An Efficient Federated Distillation Learning System for Multi-task Time Series Classification

DOI number:10.1109/TIM.2022.3201203

Journal:IEEE Transactions on Instrumentation & Measurement

Key Words:Data mining,deep learning,federated learning (FL),knowledge distillation,time series classification (TSC)

Abstract:This article proposes an efficient federated distillation learning system (EFDLS) for multitask time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components: a feature-based student–teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. For each user, the FBST framework transfers knowledge from its teacher’s hidden layers to its student’s hidden layers via knowledge distillation, where the teacher and student have identical network structures. For each connected user, its student model’s hidden layers’ weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance (LSD) used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user’s and its partner’s weights are the closest among all the weights uploaded. The server exchanges and sends back the user’s and its partner’s weights to these two users which then load the received weights to their teachers’ hidden layers. Experimental results show that compared with a number of state-of-the-art federated learning (FL) algorithms, our proposed EFDLS wins 20 out of 44 standard UCR2018 datasets and achieves the highest mean accuracy (70.14%) on these datasets. In particular, compared with a single-task baseline, EFDLS obtains 32/4/8 regarding “win”/“tie”/“lose” and results in an improvement of approximately 4% in terms of mean accuracy.

Co-author:Huanlai Xing,Zhiwen Xiao*,Rong Qu,Zonghai Zhu,Bowen Zhao

Document Code:10.1109/TIM.2022.3201203

Volume:71

Page Number:1-12

ISSN No.:0018-9456

Translation or Not:no

Date of Publication:2022-08-24

Included Journals:SCI