Distributed Edge Computing for DNA-Based Intelligent Services and Applications: A Review
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Nowadays, Data-Network-AI (DNA)-based intelligent services and applications have become a reality to provide a new dimension of services that improve the quality of life and productivity of businesses. Artificial intelligence (AI) can enhance the value of IoT data (data collected by IoT devices). The internet of things (IoT) promotes the learning and intelligence capability of AI. To extract insights from massive volume IoT data in real-time using deep learning, processing capability needs to happen in the IoT end devices where data is generated. However, deep learning requires a significant number of computational resources that may not be available at the IoT end devices. Such problems have been addressed by transporting bulks of data from the IoT end devices to the cloud datacenters for processing. But transferring IoT big data to the cloud incurs prohibitively high transmission delay and privacy issues which are a major concern. Edge computing, where distributed computing nodes are placed close to the IoT end devices, is a viable solution to meet the high computation and low-latency requirements and to preserve the privacy of users. This paper provides a comprehensive review of the current state of leveraging deep learning within edge computing to unleash the potential of IoT big data generated from IoT end devices. We believe that the revision will have a contribution to the development of DNA-based intelligent services and applications. It describes the different distributed training and inference architectures of deep learning models across multiple nodes of the edge computing platform. It also provides the different privacy-preserving approaches of deep learning on the edge computing environment and the various application domains where deep learning on the network edge can be useful. Finally, it discusses open issues and challenges leveraging deep learning within edge computing.
Abstract
Nowadays, Data-Network-AI (DNA)-based intelligent services and applications have become a reality to provide a new dimension of services that improve the quality of life and productivity of businesses. Artificial intelligence (AI) can enhance the value of IoT data (data collected by IoT devices). The internet of things (IoT) promotes the learning and intelligence capability of AI. To extract insights from massive volume IoT data in real-time using deep learning, processing capability needs to happen in the IoT end devices where data is generated. However, deep learning requires a significant number of computational resources that may not be available at the IoT end devices. Such problems have been addressed by transporting bulks of data from the IoT end devices to the cloud datacenters for processing. But transferring IoT big data to the cloud incurs prohibitively high transmission delay and privacy issues which are a major concern. Edge computing, where distributed computing nodes are placed close to the IoT end devices, is a viable solution to meet the high computation and low-latency requirements and to preserve the privacy of users. This paper provides a comprehensive review of the current state of leveraging deep learning within edge computing to unleash the potential of IoT big data generated from IoT end devices. We believe that the revision will have a contribution to the development of DNA-based intelligent services and applications. It describes the different distributed training and inference architectures of deep learning models across multiple nodes of the edge computing platform. It also provides the different privacy-preserving approaches of deep learning on the edge computing environment and the various application domains where deep learning on the network edge can be useful. Finally, it discusses open issues and challenges leveraging deep learning within edge computing.
참고문헌 (89)
- "International Data Corporation"[URL]
- L. R. Zheng, "Smart Electronic Systems: Heterogeneous Integration of Silicon and Printed Electronics" (2018), John Wiley & Sons
- M. Anandhalli, "A novel approach in real-time vehicle detection and tracking using Raspberry Pi", Alexandria Engineering Journal, 57(3), pp.1597~1607 (2018)
- M. Syafrudin, "Performance Analysis of IoT-Based Sensor, Big Data Processing, and Machine Learning Model for Real-Time Monitoring System in Automotive Manufacturing", Sensors, 18(9), pp.2946~ (2018)
- H. Khelifi, "Bringing deep learning at the edge of information-centric internet of things", IEEE Communications Letters, 23(1), pp.52~55 (2019)
- Y. Liu, "Intelligent Edge Computing for IoT-Based Energy Management in Smart Cities", IEEE Network, 33(2), pp.111~117 (2019)
- M. Merenda, "A Real-Time Decision Platform for the Management of Structures and Infrastructures", Electronics, 8(10), pp.1180~ (2019)
- S. E. Bibri, "The IoT for smart sustainable cities of the future : An analytical framework for sensor-based big data applications for environmental sustainability", Sustainable Cities and Society, 38, pp.230~253 (2018)
- C. Ieracitano, "A novel multi-modal machine learning based approach for automatic classification of EEG recordings in dementia", Neural Networks, 123, pp.176~190 (2020)
- A. Rajkomar, "Machine learning in medicine", The New England Journal of Medicine, 380, pp.1347~1358 (2019)
- K. Y. Ngiam, "Big data and machine learning algorithms for health-care delivery", The Lancet Oncology, 20(5), pp.e262~e273 (2019)
- J. Schmidhuber, "Deep learning in neural networks : An overview", Neural Networks, 61, pp.85~117 (2015)
- B. Arshad, "Computer Vision and IoT-Based Sensors in Flood Monitoring and Mapping : A Systematic Review", Sensors, 19(22), pp.5012~ (2019)
- B. Ravandi, "A self-learning scheduling in cloud software defined block storage", 2017IEEE 10th International Conference on Cloud Computing (CLOUD), pp.415~422 (2017)
- Q. Pu, "Low latency geo-distributed data analytics", Proc. Of ACM SIGCOMM (2015)
- Y. Sahni, "Data-aware task allocation for achieving low latency in collaborative edge computing", IEEE Internet of Things Journal, 6(2), pp.3512~3524 (2019)
- J. Ren, "An edge-computing based architecture for mobile augmented reality", IEEE Network, 33(4), pp.162~169 (2019)
- M. Satyanarayanan, "The emergence of edge computing", Computer (Long. Beach. Calif), 50(1), pp.30~39 (2017)
- Y. Mao, "A survey on mobile edge computing : The communication perspective", IEEE Communications Surveys & Tutorials, 19(4), pp.2322~2358 (2017)
- S. Weisong, "Edge computing : Vision and challenges", IEEE Internet of Things Journal, 3(5), pp.637~646 (2016)
- Y. LeCun, "Deep learning", Nature, 521(7553), pp.436~444 (2015)
- I. Goodfellow, "Deep Learning, Vol.1" (2016), MIT Press
- S. Verma, "A Survey on Network Methodologies for Real-Time Analytics of Massive IoT Data and Open Research Issues", IEEE Communications Surveys and Tutorials, 19(3), pp.1457~1477 (2017)
- R. Collobert, "Natural language processing(almost)from scratch", Journal of Machine Learning Research, pp.2493~2537 (2011)
- A. Krizhevsky, "ImageNet classification with deep convolutional neural networks", Communications of the ACM, 60(6), pp.84~90 (2017)
- M. McClellan, "Deep Learning at the Mobile Edge : Opportunities for 5G Networks", Applied Sciences, 10(4), pp.4735~ (2020)
- A. L. Caterini, "Design and Applications, Vol.5" (2018)
- P. Agarwal, "Lightweight Deep Learning Model for Human Activity Recognition on Edge Devices", Procedia Computer Science, 167 (2020)
- E. Li, "Edge AI : On-Demand Accelerating Deep Neural Network Inference", IEEE Transactions on Wireless Communications, 19(1) (2019)
- B. Jacob, "Quantization and training of neural networks for efficient integer-arithmetic-only inference", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2204~2713 (2018)
- W. Meng, "Two-bit networks for deep learning on resource-constrained embedded devices", arXiv preprint arXiv : 1701. 00485, ~ (2017)
- P. Gysel, "Hardware-oriented Approximation of Convolutional Neural Networks", arXiv preprint arXiv : 1604. 03168, ~ (2016)
- "Intel Developer Zone"[URL]
- C. Yang, "Structured Pruning of Convolutional Neural Networks via L1 Regularization", IEEE Access, 7, pp.106385~106394 (2019)
- F. Tung, "Deep Neural Network Compression by In-Parallel Pruning-Quantization", IEEE Transactions on Pattern Analysis and Machine Intelligence (2020)
- H. Hu, "Network trimming : A data-driven neuron pruning approach towards efficient deep architectures", arXiv preprint arXiv : 1607. 03250, ~ (2016)
- J. -H. Luo, "Thinet: a filter level pruning method for deep neural net- work compression", Proceedings of the IEEE International Conference on Computer Vision (2017)
- S. Yao, "DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework", Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, pp.1~14 (2017)
- T. J. Yang, "Designing energyefficient convolutional neural networks using energyaware pruning", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.5687~5695 (2017)
- G. Hinton, "Distilling the Knowledge in a Neural Network", arXiv preprint arXiv : 1503. 02531, ~ (2015)
- M.-C. Wu, "Multi-teacher knowledge distillation for compressed video action recognition on deep neural networks", ICASSP 2019-2019IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2202~2206 (2019), IEEE
- M. Gao, "An embarrassingly simple approach for knowledge distillation", arXiv preprint arXiv : 1812. 01819, ~ (2018)
- Y. Lin, "Deep Gradient Compression : Reducing the Communication Bandwidth for Distributed Training", arXiv preprint arXiv : 1712. 01887, ~ (2017)
- S. Teerapittayanon, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices", 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp.328~339 (2017), IEEE
- X. Wang, "In-edge AI : Intelligentizing mobile edge computing, caching and communication by federated learning", IEEE Network, 33(5), pp.156~165 (2019)
- S. B. Carlo, "Edge Computing Architecture for applying AI to IoT", IEEE International Conference on Big Data (Big Data), pp.3012~3016 (2017), IEEE
- S. Sureddy, "Flexible Deep Learning in Edge Computing for Internet of Thinkgs", International Journal of Pure and Applied Mathematics, 119(10), pp.531~543 (2018)
- H. Li, "Learning IoT in Edge : Deep Learning for the Internet of Things with Edge Computing", IEEE Network, 32(1), pp.96~101 (2018)
- Z. Zhou, "Edge intelligence: Paving the last mile of artificial intelligence with edge computing", Proceedings of the IEEE, 107(8), pp.1738~1762 (2019)
- J. Chen, "Deep Learning with Edge Computing : A Review", Proceedings of the IEEE, 107(8), pp.1655~1674 (2019)
- M. Merenda, "Edge Machine Learning for AI-Enabled IoT Devices : A Review", Sensors, 20(9), pp.2533~ (2020)
- M. Abadi, "Tensorflow, A system for large-scale machine learning", The 12th USENIX Conference on Operating Systems Design and Implementation (2016), OSDI
- J. Dean, "Advances in Neural Information Processing Systems, Vol.25", pp.1223~1231 (2012)
- R. Mayer, "Scalable Deep Learning on Distributed Infrastructures : Challenges, Techniques, and Tools", ACM Computing Surveys (CSUR), 53(1), pp.1~37 (2020)
- Z. Jia, "Beyond data and model parallelism for deep neural networks", arXiv preprint arXiv : 1807. 05358, ~ (2018)
- J. Dean, "Advances in Neural Information Processing Systems, Vol.25", pp.1223~1231 (2012)
- J. Konečný, "Federated learning : Strategies for improving communication efficiency", arXiv preprint arXiv : 1610. 05492, ~ (2016)
- J. Konečný, "Federated optimization : Distributed optimization beyond the datacenter", arXiv preprint arXiv : 1511. 03575, ~ (2015)
- C. Gentry, "Fully homomorphic encryption using ideal lattices", Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing, pp.169~178 (2009)
- P. Li, "Multi-key privacy-preserving deep learning in cloud computing", Future Generation Computer Systems, 74, pp.76~85 (2017)
- E. Bresson, "A simple public-key cryptosystem with a double trapdoor decryption mechanism and its applications", International Conference on the Theory and Application of Cryptology and Information Security, pp.37~54 (2003), Springer
- C. Gentry, "Fully homomorphic encryption using ideal lattices", Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing, pp.169~178 (2009)
- T. Zhang, "Privacy-preserving machine learning through data obfuscation", arXiv preprint arXiv : 1807. 01860, ~ (2018)
- R. Shokri, "Privacy-preserving deep learning", Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp.1310~1321 (2015)
- M. Abadi, "Deep Learning with Differential Privacy", Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp.308~318 (2016)
- J. Wang, "Not Just Privacy: Improving Performance of Private Deep Learning in Mobile Cloud", Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.2407~2416 (2018)
- Y. Mao, "Learning from differentially private neural activations with edge computing", 2018 IEEE, 2018, 90~102
- D. Li, "Deep Learning in Natural Language Processing" (2018), Springer
- Team Siri, "Hey Siri : An On-Device DNN-Powered Voice Trigger for Apple’s Personal Assistant", Apple Machine Learning Journal, 1(6) (2017)
- M. S. Mehrjardi, "SelfAttentional Models Application in Task-Oriented Dialogue Generation Systems", arXiv preprint arXiv : 1909. 05246, ~ (2019)
- S. P. Singh, "Machine translation using deep learning: An overview", 2017 International Conference on Computer, Communications and Electronics (Comptelix), pp.162~167 (2017)
- X. Jiang, "Deep Learning in Object Detection and Recognition" (2019), Springer
- V. Aggarwal, "A review : deep learning technique for image classification", ACCENTS Transactions on Image Processing and Computer Vision, 4(11), pp.21~25 (2018)
- C. C. Hung, "VideoEdge : Processing Camera Streams using Hierarchical Clusters", 2018 IEEE, 2018, 115~131[DOI]
- S. Mittal, "Integration of edge computing with cloud computing", 2017 International Conference on Emerging Trends in Computing and Communication Technologies (ICETCCT), pp.1~6 (2017)
- C. S. Lai, "A Review of Technical Standards for Smart Cities", Clean Technologies, 2(3), pp.290~310 (2020)
- J. Souza, "Data Mining and Machine Learning to Promote Smart Cities : A Systematic Review from 2000 to 2018", Sustainability, 11(4), pp.1077~ (2019)
- F. Cicirelli, "An edge-based platform for dynamic smart city applications", Future Generation Computer Systems, 76, pp.106~118 (2017)
- A. Essien, "A deep-learning model for urban traffic flow prediction with traffic events mined from twitter", World Wide Web, 1~24 (2020)
- Z. Liu, "Urban traffic prediction from mobility data using deep learning", IEEE Network, 32(4), pp.40~46 (2018)
- Y. Liu, "Intelligent edge computing for IoT-based energy management in smart cities", IEEE Network, 33(2), pp.111~117 (2019)
- L. Greco, "Trends in IoT based solutions for health care:Moving AI to the edge", Pattern Recognition Letters (2020)
- S. Tuli, "Healthfog: An ensemble deep learning based smart healthcare system for automatic diagnosis of heartdiseases in integrated iot and fog computing environments", Future Generation Computer Systems, 104, pp.187~200 (2020)
- M. Chen, "Edge cognitive computing based smart healthcare system", Future Generation Computer Systems, 86, pp.403~411 (2018)
- A. Esteva, "Dermatologist-level classification of skin cancer with deep neural networks", nature, 542(7639), pp.115~118 (2017)
- L. Tsochatzidis, "Deep Learning for Breast Cancer Diagnosis from MammogramsA Comparative Study", Journal of Imaging, 5(3), pp.37~ (2019)
- T. Ben-Nun, "Demystifying parallel and distributed deep learning : An in-depth concurrency analysis", ACM Computing Surveys (CSUR), 52(4) (2018)
- R. Xu, "CryptoNN: Training neural networks over encrypted data", 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp.1199~1209 (2019), IEEE