V. 결론
5.3 향후 과제
본연구에서는스마트폰과 같은모바일기기의에너지효율성을고 려하여딥러닝모델을신경망아키텍처탐색을통해더욱성능이좋은딥 러닝모델을탐색하고학습이가능하다는것을확인하였다.향후모바일 기기뿐만아니라IoT센서와같이더욱성능이제약된하드웨어로확장하 여본연구방법을적용하면,마이크로컨트롤러기반에매우작은용량의 메모리에서실행되는환경에서보다효율적인딥러닝모델설계가 가능 할것으로보인다.또한본연구에서는신경망아키텍처탐색과관련해서
ENAS를베이스라인모델로선정하였지만,보다학습시간이짧고성능이
좋은탐색영역과전략을가진모델을기반으로소비전력과지연시간을 고려하여학습한다면보다성능이개선된모델을탐색하는방법에대해 연구할계획이다.
참고 문헌
[1] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethink- ing the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826, 2016.
[2] A. Ignatov, R. Timofte, A. Kulik, S. Yang, K. Wang, F. Baum, M. Wu, L. Xu, and L. Van Gool, “Ai benchmark: All about deep learning on smartphones in 2019,” in 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3617–3635, IEEE, 2019.
[3] E. Cai, D.-C. Juan, D. Stamoulis, and D. Marculescu, “Neuralpower:
Predict and deploy energy-efficient convolutional neural networks,” in Asian Conference on Machine Learning, pp. 622–637, PMLR, 2017.
[4] C. F. Rodrigues, G. Riley, and M. Luj´an, “Synergy: An energy mea- surement and prediction framework for convolutional neural net- works on jetson tx1,” in Proceedings of the International Confer- ence on Parallel and Distributed Processing Techniques and Applica- tions (PDPTA), pp. 375–382, The Steering Committee of The World Congress in Computer Science, Computer . . . , 2018.
[5] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Good- fellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Man´e, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Vi´egas, O. Vinyals, P. War- den, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow:
Large-scale machine learning on heterogeneous systems,” 2015. Soft- ware available from tensorflow.org.
[6] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, “Caffe: Convolutional architecture for fast feature embedding,” in Proceedings of the 22nd ACM interna- tional conference on Multimedia, pp. 675–678, 2014.
[7] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differenti- ation in pytorch,” 2017.
[8] E. Garc´ıa-Mart´ın, C. F. Rodrigues, G. Riley, and H. Grahn, “Estima- tion of energy consumption in machine learning,”Journal of Parallel and Distributed Computing, vol. 134, pp. 75–88, 2019.
[9] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mo- bilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510–4520, 2018.
[10] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for im- age recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
[11] Y. Moon, I. Shin, Y. Lee, and O. Min, “Recent research & development trends in automated machine learning,”Electronics and Telecommuni- cations Trends, vol. 34, no. 4, pp. 32–42, 2019.
[12] T. Elsken, J. H. Metzen, and F. Hutter, “Neural architecture search: A survey,” The Journal of Machine Learning Research, vol. 20, no. 1, pp. 1997–2017, 2019.
[13] B. Baker, O. Gupta, N. Naik, and R. Raskar, “Designing neural network architectures using reinforcement learning,” arXiv preprint arXiv:1611.02167, 2016.
[14] M. Suganuma, S. Shirakawa, and T. Nagao, “A genetic programming approach to designing convolutional neural network architectures,” in
Proceedings of the genetic and evolutionary computation conference, pp. 497–504, 2017.
[15] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning trans- ferable architectures for scalable image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8697–8710, 2018.
[16] B. Zoph and Q. V. Le, “Neural architecture search with reinforcement learning,”arXiv preprint arXiv:1611.01578, 2016.
[17] Z. Zhong, J. Yan, and C.-L. Liu, “Practical network blocks design with q-learning,”arXiv preprint arXiv:1708.05552, vol. 6, 2017.
[18] R. Miikkulainen, J. Liang, E. Meyerson, A. Rawal, D. Fink, O. Fran- con, B. Raju, H. Shahrzad, A. Navruzyan, N. Duffy,et al., “Evolving deep neural networks,” in Artificial intelligence in the age of neural networks and brain computing, pp. 293–312, Elsevier, 2019.
[19] E. Real, S. Moore, A. Selle, S. Saxena, Y. L. Suematsu, J. Tan, Q. V.
Le, and A. Kurakin, “Large-scale evolution of image classifiers,” inIn- ternational Conference on Machine Learning, pp. 2902–2911, PMLR, 2017.
[20] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” inProceedings of the IEEE con- ference on computer vision and pattern recognition, pp. 4700–4708, 2017.
[21] M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, and Q. V. Le, “Mnasnet: Platform-aware neural architecture search for mo- bile,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2820–2828, 2019.
[22] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Im- ageNet: A Large-Scale Hierarchical Image Database,” in CVPR09, 2009.
[23] H. Pham, M. Guan, B. Zoph, Q. Le, and J. Dean, “Efficient neural ar- chitecture search via parameters sharing,” inInternational Conference on Machine Learning, pp. 4095–4104, PMLR, 2018.
[24] S. Bianco, R. Cadene, L. Celona, and P. Napoletano, “Benchmark anal- ysis of representative deep neural network architectures,” IEEE Ac- cess, vol. 6, pp. 64270–64277, 2018.
[25] K. Deb, “Multi-objective optimization,” in Search methodologies, pp. 403–449, Springer, 2014.
[26] A. Wong, “Netscore: towards universal metrics for large-scale perfor- mance analysis of deep neural networks for practical on-device edge usage,” inInternational Conference on Image Analysis and Recogni- tion, pp. 15–26, Springer, 2019.
[27] “Snapdragon neural processing engine sdk reference guide.”https:
//developer.qualcomm.com/docs/snpe.
[28] A. Krizhevsky, G. Hinton,et al., “Learning multiple layers of features from tiny images,” 2009.
[29] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for con- volutional neural networks,” inInternational Conference on Machine Learning, pp. 6105–6114, PMLR, 2019.
[30] Sergio Guadarrama, Nathan Silberman, “TensorFlow-Slim:
A lightweight library for defining, training and evaluating complex models in tensorflow.” https://github.com/
google-research/tf-slim, 2016. [Online; accessed 29- June-2019].
Abstract
Neural Architecture Search considering energy efficiency of
mobile device
Youngyun Kim Graduate School of Practical Engineering Seoul National University
The demand for on-device AI service-based image analysis technology that can be used in embedded devices such as mobile and IoT devices is in- creasing. On-device AI has advantages such as low latency and enhanced security, but AI performance is dependent on hardware performance and consumes excessive power by requiring a lot of computing resources such as processor and memory for AI operations. For this reason, there is a need to improve energy efficiency for on-device AI models.
In this study, we propose ELP-NAS as a method of constructing a deep learning model considering the energy efficiency of mobile devices. ELP- NAS trains deep learning models using neural network architecture search to design optimal architectures in automatic machine learning. By applying the algorithm to predict the end-to-end energy consumption and latency of the deep learning model, the predicted energy consumption and latency of