본문 바로가기
KERD
논문KCI 등재

Spiking 신경망을 통합하는 프레임워크: 리뷰

Frameworks that integrate Spiking Neural Networks: A Review

김수현|아셀 캠바이
한국차세대컴퓨팅학회 (KINGComputing)|2022.12|한국차세대컴퓨팅학회 논문지 (The Journal of Korean Institute of Next Generation Computing)|제18권 제6호|pp. 93-105|DOI 10.23019/kingpc.18.6.202212.009

0

조회

0

다운로드

0

인용

0

참고문헌

키워드

프레임워크딥러닝Deep Learning스파이킹 신경망파이토치Spiking Neural NetworksFrameworksPyTorch
주제분류컴퓨터학

초록

스파이킹 신경망(SNN)은 효율적인 시간 처리와 에너지 소비로 기존 인공 신경망(ANN)의 잠재적인 대안으로 부상했다. 그러나 심층 SNN을 학습하는 데 어려움이 있어 적용 가능성이 제한된다. 개선된 학습 알고리즘 제공 및 SNN학습을 가능하게 하는 프레임워크 개발에 대한 중요성이 확대되고 있다. 본 논문에서는 기계 학습 응용 프로그램을 향상시키기 위해 그라디언트 기반 학습을 지원하는 기존 SNN 프레임워크를 활용하여 다양한 실험을 진행한다. 각 프레임워크를 SNN을 학습한 후, 최종 Top-1 accuracy, time efficiency를 비교 분석한다.

Abstract

Spiking Neural Networks (SNNs) have emerged as a potential alternative to traditional artificial neural networks (ANNs) because of their temporal processing capabilities and low energy consumption. However, the difficulties in training deep SNNs have limited their applicability. Thus, improving learning algorithms and the frameworks that implement them is one of the primary research focuses in the SNN field. This paper surveys SNN frameworks that specifically support gradient-based learning to enhance machine learning applications. A comparison of the different libraries is described with experimental analysis.

참고문헌 (47)

  1. Y. LeCun, "Deep learning", Nature, 521, pp.436~444 (2015)
  2. S. Esser, "Convolutional networks for fast, energy-efficient neuromorphic computing", Proceedings of the National Academy of Sciences of the United States of America, 113(41), pp.11441~11446 (2016)
  3. A. Kugele, "Efficient processing of spatio-temporal data streams with spiking neural networks", Frontiers Neurosci, 14, pp.439~ (2020)
  4. Y. Cao, "Spiking deep convolutional neural networks for energyefficient object recognition", Int. J. Comput. Vis, 113(1), pp.54~66 (2015)
  5. S. Deng, "Optimal conversion of conventional artificial neural networks to spiking neural networks", Proc. Int. Conf. Learn. Represent, pp.1~11 (2021)
  6. B. Rueckauer, "Conversion of continuous-valued deep networks to efficient event-driven networks for image classification", Neurosci, 11 (2017)
  7. T. Bekolay, "Nengo: a python tool for building large-scale functional brain models", Frontiers in Neuroinformatics, 7 (2014)
  8. "Lava: A software framework for neuromorphic computing" (2021)[URL]
  9. W. Fang, "Spikingjelly", ~
  10. H. Hazan, "BindsNET : A machine learningoriented spiking neural networks library in Python", Frontiers Neuroinform, 12, pp.89~ (2018)
  11. T. Masquelier, "Unsupervised learning of visual features through spike timing dependent plasticity", PLOS Comput. Biol, 3(2) (2007)
  12. E. O. Neftci, "Surrogate gradient learning in spiking neural networks : bringing the power of gradient-based optimization to spiking neural networks", IEEE Signal Process. Mag, 36, pp.51~63 (2019)
  13. M. Mozafari, "SpykeTorch : Efficient simulation of convolutional spiking neural networks with at most one spike per neuron", Frontiers in Neuroscience, 13, pp.625~ (2019)
  14. "Pysnn" (2019)[URL]
  15. "Norse - A deep learning library for spiking neural networks" (2021)[URL]
  16. A. Paszke, "Pytorch : An imperative style, high-performance deep learning library", Proc. Adv. Neural Inf. Process. Syst, pp.8026~8037 (2019)
  17. Y. Kim, "Revisiting batch normalization for training low-latency deep spiking neural networks from scratch", Front. Neurosci, 15, pp.773954~ (2021)
  18. H. Zheng, "Going deeper with directly-trained larger spiking neural networks", Proc. AAAI Conf. Artificial Intelligence, 35, pp.11062~11070 (2021)
  19. W. Fang, "Incorporating learnable membrane time constant to enhance learning of spiking neural networks", Proc. IEEE, 2021, 2661~2671
  20. W. Fang, "Deep residual learning in spiking neural networks", Advances in Neural Information Processing Systems, 34 (2021)
  21. P. Qu, "A review of basic software for brain-inspired computing", CCF Transactions on High Performance Computing (2022)
  22. A. M. Garcia-Vico, "A preliminary analysis on software frameworks for the development of spiking neural networks", International Conference on Hybrid Artificial Intelligence Systems (HAIS), pp.564~575 (2021), Springer
  23. C. D. Schuman, "A software framework for comparing training approaches for spiking neuromorphic systems", IJCNN, pp.1~10 (2021)
  24. D. L. Manna, "Frameworks for SNNs: a review of data science-oriented software and an expansion of SpykeTorch", International Conference On Neuromorphic Systems (2022)
  25. L. Abbott, "Lapicques introduction of the integrate-and-fire model neuron (1907)", Brain Research Bulletin, 50(5), pp.303~304 (1999)
  26. A. N. Burkitt, "A review of the integrateand-fire neuron model : I. Homogeneous synaptic input", Biol. Cybern, 95(1), pp.1~19 (2006)
  27. E. M. Izhikevich, "Simple model of spiking neurons", IEEE Trans. Neural Netw, 14(6), pp.1569~1572 (2003)
  28. E. M Izhikevich, "Which model to use for cortical spiking neurons?", IEEE Trans. Neural Netw, 15(5), pp.1063~1070 (2004)
  29. A. L. Hodgkin, "A quantitative description of membrane current and its application to conduction and excitation in nerve", J. Physiol, 117(4), pp.500~544 (1952)
  30. J. Kaiser, "Synaptic plasticity dynamics for deep continuous local learning(DECOLLE)", Frontiers in Neuroscience, 14, pp.424~ (2020)
  31. G. Bellec, "A solution to the learning dilemma for recurrent networks of spiking neurons", Nature Commun, 11(1), pp.3625~ (2020)
  32. F. Zenke, "Superspike : Supervised learning in multilayer spiking neural networks", Neural computation, 30(6), pp.1514~1541 (2018)
  33. S. M. Bohte, "Error-backpropagation in temporally encoded networks of spiking neurons", Neurocomputing, 48(1), pp.17~37 (2002)
  35. F. Akopyan, "TrueNorth : Design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip", IEEE Trans. Comput. -Aided Design Integr. Circuits Syst, 34(10), pp.1537~1557 (2015)
  36. M. Davies, "Loihi : A neuromorphic manycore processor with on-chip learning", IEEE Micro, 38(1), pp.82~99 (2018)
  37. A. D. Brown, "SpiNNaker : Event-Based Simulation-Quantitative Behavior", IEEE Trans. Multi-Scale Comput. Syst, 4(3), pp.450~462 (2018)
  38. J. Schemmel, "Live demonstration : A scaled-down version of the BrainScaleS waferscale neuromorphic system", Proc. IEEE Int. Symp. Circuits Syst.(ISCAS), pp.702~ (2012)
  39. O. Gewaltig, "NEST (NEural Simulation Tool)", Scholarpedia, 2, pp.4~ (2007)
  40. N. T. Carnevale, "The Neuron Book. Cambridge" (2006), Cambridge Univ. Press
  41. M. Stimberg, "Brian 2, an intuitive and efficient neural simulator", eLife, 8 (2019)
  42. E. Yavuz, "GeNN : A code generation framework for accelerated brain simulations", Sci. Rep., 6 (2016)
  43. G. Bellec, "Advances in Neural Information Processing Systems 31", pp.795~805 (2018), Curran Associates
  44. S. Hochreiter, "Long shortterm memory", Neural computation, 9(8), pp.1735~1780 (1997)
  45. B. Cramer, "The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks", IEEE Transactions on Neural Networks and Learning Systems, 1(14), pp.2020~ (2020)
  47. P. J. Werbos, "Backpropagation through time : What it does and how to do it", Proceedings of the IEEE, 78(10), pp.1550~1560 (1990)

관련 논문AI추천

스파이킹 신경망 추론을 위한 심층 신경망 가중치 변환이정수, 허준영(사)한국스마트미디어학회 · 스마트미디어저널2022 · 제11권 제3호동적 시간 지연 인코딩 기반 멀티스케일 스파이크 신경망박성수, 최영석대한의용생체공학회 · 의공학회지2025 · 제46권 제1호IoT 컴퓨팅 환경을 위한 뉴로모픽 기반 플랫폼의 추론시간 단축김재섭, 이승연, 홍지만(사)한국스마트미디어학회 · 스마트미디어저널2022 · 제11권 제2호패턴 기반 정렬을 활용한 고속-에너지 효율적인 컨볼루션 스파이킹 신경망 하드웨어 가속기의 모델 기반 동적 가지치기박대진, 윤희지대한임베디드공학회 · 대한임베디드공학회논문지2024 · 제19권 제6호뉴로모픽 환경에서 QoS를 고려한 최적의 SNN 모델 파라미터 생성 기법김봉재, 김서연, 정진만(사)한국스마트미디어학회 · 스마트미디어저널2023 · 제12권 제4호FPGA상에서 스파이킹 뉴럴 네트워크 지원을 위한 모델 최적화김봉재, 김서연, 윤영선, 이건명, 정진만, 홍지만(사)한국스마트미디어학회 · 스마트미디어저널2022 · 제11권 제2호

이 간행물의 다른 논문 (13건)

경제학 지식 분석을 위한 BERT 기반 관계추출 모델과 지식그래프강영선, 김수정, 석준희pp. 7-20공공부문 웹아카이빙을 위한 원격 수집 친화적인 웹사이트 구축 연구최동준, 권혁상, 장진호, 이규모pp. 21-31이진인코딩을 이용한 순위다중패턴매칭 알고리즘공준호, 심정섭, 김영호pp. 32-39저지연 엣지 컴퓨팅 서비스를 위한 시스템 소프트웨어 설계 및 성능 평가임철수pp. 40-48EfficientNet과 ONNX를 이용한 온라인 검체 분류제어 시스템 개발황영배, 김성웅pp. 49-60
원문 보기