Service Caching Strategy based on Edge Computing and Reinforcement Learning

doi:10.23940/ijpe.22.05.p5.350358

Int J Performability Eng ›› 2022, Vol. 18 ›› Issue (5): 350-358.doi: 10.23940/ijpe.22.05.p5.350358

Previous Articles Next Articles

Service Caching Strategy based on Edge Computing and Reinforcement Learning

Chengjie Xu^a, Dongcheng Li^b, W. Eric Wong^b, and Man Zhao^a,*

^aSchool of Computer Science, China University of Geosciences, Wuhan, 430074, China;
^bDepartment of Computer Science, University of Texas at Dallas, 75082, USA

Submitted on ; Revised on ; Accepted on
Contact: * E-mail address: zhaoman@cug.edu.cn

Abstract

Abstract: With the rapid development of the Internet of Things in recent years, there has been a dramatic increase in terminal units and new computationally and data-demanding applications. A terminal unit uploads data to the cloud server, which will be transmitted back to the terminal unit after certain operations. However, such a traditional cloud service is troubled by growing latency. Mobile edge computing emerges in such an environment. A short distance between the edge network and end-users mitigates this problem. However, the edge network has finite resources, making it impossible to deliver all service caching requests. To this end, a strategy is required to selectively cache services on the edge cloud. This study simulates the selection of edge services with a multi-armed bandit model and conducts a comparative study to analyze the impact that different algorithms have on performance.

Key words: edge computing, service caching, reinforcement learning, multi-armed bandit

Chengjie Xu, Dongcheng Li, W. Eric Wong, and Man Zhao. Service Caching Strategy based on Edge Computing and Reinforcement Learning [J]. Int J Performability Eng, 2022, 18(5): 350-358.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. Ashton K.That ‘internet of things’ thing. RFID journal, 22(7), pp.97-114, 2009.
2. Atzori, L., Iera, A. and Morabito, G.The internet of things: A survey. Computer networks, 54(15), pp.2787-2805, 2010.
3. Armbrust M., Fox A., Griffith R., Joseph A.D., Katz R.H., Konwinski A., Lee G., Patterson D.A., Rabkin A., Stoica I. and Zaharia M.Above the clouds: A berkeley view of cloud computing, 17, 2009.
4. Brutlag J. Speed matters for Google web search, 2009.
5. Tran T.X., Hajisami A., Pandey P. and Pompili D.Collaborative mobile edge computing in 5G networks: New paradigms, scenarios, and challenges. IEEE Communications Magazine, 55(4), pp.54-61, 2017.
6. Taleb T., Samdanis K., Mada B., Flinck H., Dutta S. and Sabella D.On multi-access edge computing: A survey of the emerging 5G network edge cloud architecture and orchestration. IEEE Communications Surveys & Tutorials, 19(3), pp.1657-1681, 2017.
7. Satyanarayanan M., Bahl P., Caceres R. and Davies N.The case for vm-based cloudlets in mobile computing. IEEE pervasive Computing, 8(4), pp.14-23, 2009.
8. Abbas N., Zhang Y., Taherkordi A. and Skeie T.Mobile edge computing: A survey. IEEE Internet of Things Journal, 5(1), pp.450-465, 2017.
9. Sutton, R.S. and Barto, A.G.Reinforcement learning: An introduction. MIT press, 2018.
10. Vermorel, J. and Mohri, M.Multi-armed bandit algorithms and empirical evaluation. In European conference on machine learning. Springer, Berlin, Heidelberg, pp. 437-448, 2005.
11. Carpentier A., Lazaric A., Ghavamzadeh M., Munos R. and Auer P.Upper-confidence-bound algorithms for active learning in multi-armed bandits. In International Conference on Algorithmic Learning Theory. Springer, Berlin, Heidelberg, pp. 189-203, 2011.
12. Chapelle, O. and Li, L.An empirical evaluation of thompson sampling. Advances in neural information processing systems, 24, 2011.

[1]	Shaili Mishra and Anuja Arora. Double Deep Q Network with Huber Reward Function for Cart-Pole Balancing Problem [J]. Int J Performability Eng, 2022, 18(9): 644-653.
[2]	Yuting Cheng, Dongcheng Li, W. Eric Wong, Man Zhao, and Dengfeng Mo. Multi-UAV Collaborative Path Planning using Hierarchical Reinforcement Learning and Simulated Annealing [J]. Int J Performability Eng, 2022, 18(7): 463-474.
[3]	Sukruta Pardeshi, chetana Khairnar, and Khalid Alfatmi. Analysis of Data Handling Challenges in Edge Computing [J]. Int J Performability Eng, 2022, 18(3): 176-187.
[4]	Haoran Li, Dongcheng Li, W. Eric. Wong, Deze Zeng, and Man Zhao. Kubernetes Virtual Warehouse Placement based on Reinforcement Learning [J]. Int J Performability Eng, 2021, 17(7): 579-588.
[5]	Gayathri D and S.P. Shantharajah. A Survey on Fusion of Internet of Things and Cloud Computing [J]. Int J Performability Eng, 2021, 17(11): 946-954.
[6]	Shujun Pei, Qinggen Zhang, and Xuehui Cheng. Workflow Scheduling using Graph Segmentation and Reinforcement Learning [J]. Int J Performability Eng, 2020, 16(8): 1262-1270.

Service Caching Strategy based on Edge Computing and Reinforcement Learning

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 6

Recommended 0