A Federated Multi-Task Learning Model Based on Adaptive Distributed Data Latent Correlation Analysis
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Federated learning provides an efficient integrated model for distributed data, allowing the local training ofdifferent data. Meanwhile, the goal of multi-task learning is to simultaneously establish models for multiplerelated tasks, and to obtain the underlying main structure. However, traditional federated multi-task learningmodels not only have strict requirements for the data distribution, but also demand large amounts of calculationand have slow convergence, which hindered their promotion in many fields. In our work, we apply the rankconstraint on weight vectors of the multi-task learning model to adaptively adjust the task’s similarity learning,according to the distribution of federal node data. The proposed model has a general framework for solvingoptimal solutions, which can be used to deal with various data types. Experiments show that our model hasachieved the best results in different dataset. Notably, our model can still obtain stable results in datasets withlarge distribution differences. In addition, compared with traditional federated multi-task learning models, ouralgorithm is able to converge on a local optimal solution within limited training iterations.
Abstract
Federated learning provides an efficient integrated model for distributed data, allowing the local training ofdifferent data. Meanwhile, the goal of multi-task learning is to simultaneously establish models for multiplerelated tasks, and to obtain the underlying main structure. However, traditional federated multi-task learningmodels not only have strict requirements for the data distribution, but also demand large amounts of calculationand have slow convergence, which hindered their promotion in many fields. In our work, we apply the rankconstraint on weight vectors of the multi-task learning model to adaptively adjust the task’s similarity learning,according to the distribution of federal node data. The proposed model has a general framework for solvingoptimal solutions, which can be used to deal with various data types. Experiments show that our model hasachieved the best results in different dataset. Notably, our model can still obtain stable results in datasets withlarge distribution differences. In addition, compared with traditional federated multi-task learning models, ouralgorithm is able to converge on a local optimal solution within limited training iterations.
참고문헌 (30)
- "Federated learning:strategies for improving communication efficiency" (2017)[URL]
- H. Brendan McMahan, "Communication-efficient learning of deep networks from decentralized data", Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pp.1273~1282 (2017)
- Q. Yang, "Federated machine learning : concept and applications", ACM Transactions on Intelligent Systems and Technology (TIST), 10(2) (2019)[DOI]
- "Federated learning of deep networks using model averaging" (2016)[URL]
- Y. Xue, "Multi-task learning for classification with Dirichlet process priors", Journal of Machine Learning Research, 8, pp.35~63 (2007)
- S. J. Pan, "A survey on transfer learning", IEEE Transactions on Knowledge and Data Engineering, 22(10), pp.1345~1359 (2010)
- X. T. Yuan, "Visual classification with multitask joint sparse representation", IEEE Transactions on Image Processing, 21(10), pp.4349~4360 (2012)
- L. Argote, "Organizational learning : from experience to knowledge", Organization Science, 22(5), pp.1123~1137 (2011)
- C. Vens, "Decision trees for hierarchical multi-label classification", Machine Learning, 73(2), pp.185~214 (2008)
- T. Evgeniou, "Regularized multi-task learning", Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.109~117 (2004)
- S. Rosen, "Appprofiler: a flexible method of exposing privacy-related behavior in android applications to end users", Proceedings of the 3rd ACM Conference on Data and Application Security and Privacy, pp.221~232 (2013)
- J. Wang, "Distributed multi-task learning", Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, pp.751~760 (2016)
- R. Tibshirani, "Regression shrinkage and selection via the lasso : a retrospective", Journal of the Royal Statistical Societ B, 73(3), pp.273~282 (2011)
- R. G. Brereton, "Support vector machines for classification and regression", Analyst, 135(2), pp.230~267 (2010)
- Robust principal component analysis : exact recovery of corrupted low-rank matrices via convex optimization, "J. Wright", Coordinated Science Laboratory, University of Illinois, ~, pp.Report No. UILU-ENG-09-2210(DC-243) (2009)
- X. Ding, "Camera identification based on domain knowledge-driven deep multi-task learning", IEEE Access, 7, pp.25878~25890 (2019)
- D. Mateos-Nunez, "Distributed optimization for multi-task learning via nuclear-norm approximation", IFAC-PapersOnLine, 48(22), pp.64~69 (2015)
- M. Zhao, "Joint lp-and l2, p-norm minimization for subspace clustering with outlier pursuit", Proceedings of 2016 International Joint Conference on Neural Networks (IJCNN), pp.3658~3665 (2016)
- M. Zhang, "L2, p-norm and sample constraint based feature selection and classification for AD diagnosis", Neurocomputing, 195, pp.104~111 (2016)
- R. Caruana, "Multitask learning", Machine Learning, 28(1), pp.41~75 (1997)
- D. Zhang, "Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer's disease", NeuroImage, 59(2), pp.895~907 (2012)
- Z. Hu, "Time-and cost-efficient task scheduling across geo-distributed data centers", IEEE Transactions on Parallel and Distributed Systems, 29(3), pp.705~718 (2018)
- Y. Wang, "Learning geographically distributed data for multiple tasks using generative adversarial networks", Proceedings of 2019 IEEE International Conference on Image Processing (ICIP), pp.4589~4593 (2019)
- X. Cai, "Multi-class l2,1-norm support vector machine", Proceedings of 2011 IEEE 11th International Conference on Data Mining, pp.91~100 (2011)
- P. Heins, "Locally sparse reconstruction using the l1, ∞-norm", Inverse Problems & Imaging, 9, pp.1093~1137 (2015)
- P. E. Gill, "SNOPT : an SQP algorithm for large-scale constrained optimization", SIAM Review, 47(1), pp.99~131 (2005)
- N. Tottenham, "The NimStim set of facial expressions : judgments from untrained research participants", Psychiatry Research, 168(3), pp.242~249 (2009)
- K. S. Kim, "Greedy subspace pursuit for joint sparse recovery", Journal of Computational and Applied Mathematics, 352, pp.308~327 (2019)
- S. Yi, "Dual pursuit for subspace learning", IEEE Transactions on Multimedia, 21(6), pp.1399~1411 (2019)
- "Federated multi-task learning" (2017)[URL]