A Survey of the Analysis of Complex Systems based on Complex Network Theory and Deep Learning

doi:10.23940/ijpe.22.04.p2.241250

Abstract

Abstract: From the perspective of complex network theory, complex systems can be characterized by the interaction of microscopic units through nonlinear effects, yielding macroscopic emergent behavior. In light of the powerful capability of deep learning in feature extraction and model fitting from large amount of datasets, we try to overview the benefits of combining the complex network analysis with deep learning techniques to investigate complex systems. We first explore the existence of complexity in complex systems. In what followed, we first give a brief description of complex network theory. Then, we present an overview of deep learning technology. Subsequently, we focus on the research advances and applications in the analysis of complex systems based on complex network theory and deep learning. The last section is further discussion and prospects for the combination of these two methods. In a nutshell, the development of deep learning combined with complex network theory allows for exploring the complexity in complex systems at a higher level.

Key words: complexity, complex systems, complex networks, deep learning

Dan Lu and Shunkun Yang*. A Survey of the Analysis of Complex Systems based on Complex Network Theory and Deep Learning [J]. Int J Performability Eng, 2022, 18(4): 241-250.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. R. Gallagher, T. Appenzeller, D. Normile, et al., “Beyond reductionism,” Science, vol. 284, no.5411, p. 79, 1999.
2. M. M. Waldrop, Complexity: The emerging science at the edge of order and chaos. Simon and Schuster, 1993.
3. D. J.Watts and S. H. Strogatz, “Collective dynamics of ‘small-world’ networks,” Nature, vol. 393, pp. 440-442, jun 1998.
4. A.-L. Barabási and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, pp. 509-512, oct 1999.
5. A. Barrat, M. Barthelemy,A. Vespignani, Dynamical processes on complex networks. Cambridge university press, 2008.
6. R. Interdonato, M. Atzmueller, S. Gaito, R. Kanawati, C. Largeron,A. Sala, “Feature-rich networks: going beyond complex network topologies,” Applied Network Science, vol. 4, jan 2019.
7. T. C.Silva and L. Zhao, Machine learning in complex networks, vol. 1. Springer, 2016.
8. G. E.Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, pp. 504-507, jul 2006.
9. J. Louth, “From newton to newtonianism: Reductionism and the development of the social sciences,” Emergence: Complexity and Organization, vol. 13, no.4, p. 63, 2011.
10. J. Earman, “Laplacian determinism, or is this any way to run a universe?,” The Journal of Philosophy, vol. 68, no. 21, pp. 729-744, 1971.
11. F. Mazzocchi, “Complexity in biology: exceeding the limits of reductionism and determinism using complexity theory,” EMBO reports, vol. 9, no. 1, pp. 10-14, 2008.
12. P. W. Anderson, “More is different,” Science, vol. 177, no. 4047, pp. 393-396, 1972.
13. L. da F. Costa, F. A. Rodrigues, G. Travieso, and P. R. V. Boas, “Characterization of complex networks: A survey of measurements,” Advances in Physics, vol. 56, pp. 167-242, jan 2007.
14. R. Cohen and S. Havlin, Complex networks: structure, robustness and function. Cambridge university press, 2010.
15. L. d. F. Costa, O. N. Oliveira Jr, G. Travieso, F. A. Rodrigues, P. R. Villas Boas, L. Antiqueira, M. P. Viana, and L. E. Correa Rocha, “Analyzing and modeling real-world phenomena with complex networks: a survey of applications,” Advances in Physics, vol. 60, no. 3, pp. 329-412, 2011.
16. B. R. Jasny, L. M. Zahn,E. Marshall, “Special issue on complex systems and networks,” Science, vol. 325, no. 5939, pp. 405-432, 2009.
17. L. Euler, “Solutio problematis ad geometriam situs pertinentis,” Commentarii academiae scientiarum Petropolitanae, pp. 128-140, 1741.
18. N. Biggs, E. K. Lloyd,R. J. Wilson, Graph Theory, 1736-1936. Oxford University Press, 1986.
19. A.-L. Barabási, “Network science,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 371, no.1987, p. 20120375, 2013.
20. F. Menczer, S. Fortunato,C. A. Davis, A First Course in Network Science. Cambridge University Press, jan 2020.
21. M. Newman, Networks. Oxford university press, 2018.
22. M. E. Newman, “The structure and function of complex networks,” SIAM review, vol. 45, no. 2, pp. 167-256, 2003.
23. M. Newman, Networks: An Introduction. Oxford University Press, mar 2010. 14
24. A.-L. Barabási, “The network takeover,” Nature Physics, vol. 8, pp. 14-16, dec 2011.
25. Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabasi, “Observability of complex systems,” Proceedings of the National Academy of Sciences, vol. 110, pp. 2460-2465, jan 2013.
26. L. Lü, C.-H. Jin, and T. Zhou, “Similarity index based on local paths for link prediction of complex networks,” Physical Review E, vol. 80, oct 2009.
27. L. Weng, F. Menczer,Y.-Y. Ahn, “Virality prediction and community structure in social networks,” Scientific Reports, vol. 3, aug 2013.
28. L. Zhou, S. Pan, J. Wang,A. V. Vasilakos, “Machine learning on big data: Opportunities and challenges,” Neurocomputing, vol. 237, pp. 350-361, 2017.
29. A. K.Jain and R. C. Dubes, Algorithms for clustering data. Prentice-Hall, Inc., 1988.
30. P. Domingos, “A few useful things to know about machine learning,” Communications of the ACM, vol. 55, no. 10, pp. 78-87, 2012.
31. M. I.Jordan and T. M. Mitchell, “Machine learning: Trends, perspectives, and prospects,” Science, vol. 349, no. 6245, pp. 255-260, 2015.
32. Y. LeCun, Y. Bengio,G. Hinton, “Deep learning,” Nature, vol. 521, pp. 436-444, may 2015.
33. Y. Bengio, P. Lamblin, D. Popovici,H. Larochelle, “Greedy layer-wise training of deep networks,” Advances in neural information processing systems, vol. 19, 2006.
34. I. J. Goodfellow, A. Courville,Y. Bengio, “Scaling up spike-and-slab models for unsupervised feature learning,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1902-1914, 2012.
35. A. Krizhevsky, I. Sutskever,G. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, vol. 25, no. 2, 2012.
36. M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997. 15
37. R. Collobert and J. Weston, “A unified architecture for natural language processing: Deep neural networks with multitask learning,” in Proceedings of the 25th international conference on Machine learning, pp. 160-167, 2008.
38. K. He, X. Zhang, S. Ren,J. Sun, “Deep residual learning for image recognition,” IEEE, 2016.
39. C. Tang, Y. Ling, X. Yang, W. Jin,C. Zheng, “Multi-view object detection based on deep learning,” Applied Sciences, vol. 8, no. 9, 2018.
40. K. Hornik, M. Stinchcombe,H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, pp. 359-366, jan 1989.
41. C.-H. Chang, “Deep and shallow architecture of multilayer neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, pp. 2477-2486, oct 2015.
42. J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Networks, vol. 61, pp. 85-117, jan 2015.
43. M. Pereda and E. Estrada, “Machine learning analysis of complex networks in hyperspherical space,” arXiv preprint arXiv:1804.05960, 2018.
44. A. A. Margolin, I. Nemenman, K. Basso, C. Wiggins, G. Stolovitzky, R. D. Favera,A. Califano, “ARACNE: An algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context,” BMC Bioinformatics, vol. 7, mar 2006.
45. C. Fan, L. Zeng, Y. Sun,Y.-Y. Liu, “Finding key players in complex networks through deep reinforcement learning,” Nature machine intelligence, vol. 2, no. 6, pp. 317-324, 2020.
46. Z. Gao, W. Dang, X. Wang, X. Hong, L. Hou, K. Ma,M. Perc, “Complex networks and deep learning for EEG signal analysis,” Cognitive Neurodynamics, vol. 15, pp. 369-388, aug, 2020.
47. S. Ha and H. Jeong, “Unraveling hidden interactions in complex systems with deep learning,” Scientific reports, vol. 11, no. 1, pp. 1-13, 2021. 16
48. B. Hussain, Q. Du, S. Zhang, A. Imran,M. A. Imran, “Mobile edge computing-based data-driven deep learning framework for anomaly detection,” IEEE Access, vol. 7, pp. 137656-137667, 2019.
49. Z. Lv, A. K. Singh,J. Li, “Deep learning for security problems in 5g heterogeneous networks,” IEEE Network, vol. 35, pp. 67-73, mar 2021.
50. M. Dhilber,S. D. Bhavani.“Community detection in social networks using deep learning." International conference on distributed computing and internet technology”. Springer, Cham, 2020.
51. J. Cao, et al.“Incorporating network structure with node contents for community detection on large networks using deep learning.” Neurocomputing vol. 297, pp.71-81, 2018.
52. S. Li, et al.“A weighted network community detection algorithm based on deep learning.” Applied Mathematics and Computation, vol. 401, pp. 126012, 2021.
53. Y. Zhang, Y. Xiong, Y. Ye, T. Liu, W. Wang, Y. Zhu,P. S. Yu, “SEAL: Learning heuristics for community detection with generative adversarial networks.” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1103-1113, aug, 2020.
54. J. Chenet al., “E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction,” in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 6, pp. 3699-3712, june 2021.
55. H. Wang, X. Shi,D.Y. Yeung, “Relational deep learning: A deep latent variable model for link prediction.” In Thirty-first AAAI conference on artificial intelligence, feb, 2017.
56. C. Chiu and J.Zhan, “Deep Learning for Link Prediction in Dynamic Networks Using Weak Estimators,” in IEEE Access, vol. 6, pp. 35937-35945, 2018.
57. T. C.Silva and L. Zhao, “Network-based high level data classification,” IEEE Transactions on Neural Networks and Learning Systems, vol. 23, pp. 954-970, jun 2012.
58. T. C.Silva and L. Zhao, “High-level pattern-based classification via tourist walks in networks,” Information Sciences, vol. 294, pp. 109-126, feb 2015.
59. A. Celikyilmaz and D. Hakkani-Tur, “A graph-based semi-supervised learning for question semantic labeling,” in Proceedings of the NAACL HLT2010 Workshop on Semantic Search, pp. 27-35, 2010.
60. T. H. Cupertino, J. Huertas,L. Zhao, “Data clustering using controlled consensus in complex networks,” Neurocomputing, vol. 118, pp. 132-140, oct 2013.
61. T. C.Silva and L. Zhao, “Pixel clustering by using complex network community detection technique,” in Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007), IEEE, oct 2007.
62. P. Hamel and D. Eck, “Learning features from music audio with deep belief networks.,” in ISMIR, vol. 10, pp. 339-344, Citeseer, 2010.
63. Y. Bengio, F. Bastien, A. Bergeron, N. Boulanger-Lewandowski, T. Breuel, Y. Chherawala, M. Cisse, M. Côté, D. Erhan, J. Eustache, et al., “Deep learners benefit more from out-of-distribution examples,” in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 164-172, JMLR Workshop and Conference Proceedings, 2011.
64. J. Susskind, G. Hinton, R. Memisevic,M. Pollefeys, “Modeling the joint density of two images under a variety of transformations,” in CVPR 2011, pp. 2793-2800, IEEE, 2011. 17
65. P. Luo, X. Wang,X. Tang, “Hierarchical face parsing via deep learning,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2480-2487, IEEE, 2012.
66. S. Thomas, M. L. Seltzer, K. Church,H. Hermansky, “Deep neural network features and semi-supervised training for low resource speech recognition,” in 2013 IEEE international conference on acoustics, speech and signal processing, pp. 6704-6708, IEEE, 2013.
67. T. Mikolov, K. Chen, G. Corrado,J. Dean, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781, 2013.
68. M. Defferrard, X. Bresson,P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” Advances in neural information processing systems, vol. 29, 2016.
69. S. Zhang, H. Tong, J. Xu,R. Maciejewski, “Graph convolutional networks: a comprehensive review,” Computational Social Networks, vol. 6, nov 2019.
70. J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li,M. Sun, “Graph neural networks: A review of methods and applications,” AI Open, vol. 1, pp. 57-81, 2020.
71. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016.
72. R. Levie, F. Monti, X. Bresson,M. M. Bronstein, “Cayleynets: Graph convolutional neural networks with complex rational spectral filters,” IEEE Transactions on Signal Processing, vol. 67, no. 1, pp. 97-109, 2018.
73. M. Henaff, J. Bruna,Y. LeCun, “Deep convolutional networks on graph-structured data,” arXiv preprint arXiv:1506.05163, 2015.
74. M. Defferrard, X. Bresson,P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” Advances in neural information processing systems, vol. 29, 2016.
75. T. Nguyen and R. Grishman, “Graph convolutional networks with argument-aware pooling for event detection,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, 2018. 18
76. X. Liu, Z. Luo,H. Huang, “Jointly multiple events extraction via attention-based graph information aggregation,” arXiv preprint arXiv:1809.09078, 2018.
77. L. Yao, C. Mao,Y. Luo, “Graph convolutional networks for text classification,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp. 7370-7377, 2019.
78. V. Garcia and J. Bruna, “Few-shot learning with graph neural networks,” arXiv preprint arXiv:1711.04043, 2017.
79. S. Zhang, Y. Qin, K. Sun,Y. Lin, “Few-shot audio classification with attentional graph neural networks.,” in Interspeech, pp. 3649-3653, 2019.
80. K. Marino, R. Salakhutdinov,A. Gupta, “The more you know: Using knowledge graphs for image classification,” arXiv preprint arXiv:1612.04844, 2016.
81. X. Li, X. Yan, Q. Gu, H. Zhou, D. Wu,J. Xu, “Deepchemstable: chemical stability prediction with an attention-based graph convolution network,” Journal of chemical information and modeling, vol. 59, no. 3, pp. 1044-1049, 2019.
82. A. Fout, J. Byrd, B. Shariat,A. Ben-Hur, “Protein interface prediction using graph convolutional networks,” Advances in neural information processing systems, vol. 30, 2017.
83. D. Mrowca, C. Zhuang, E. Wang, N. Haber, L. F.Fei-Fei, J.Tenenbaum, and D. L. Yamins, “Flexible neural representation for physics prediction,” Advances in neural information processing systems, vol. 31, 2018.
84. T. Kipf, E. Fetaya, K. C. Wang, M. Welling,R. Zemel, “Neural relational inference for interacting systems.” In International Conference on Machine Learning, pp. 2688-2697, PMLR, july, 2018.
85. P. Mehta and D. J. Schwab, “An exact mapping between the variational renormalization group and deep learning,” arXiv prepr

[1]	Janarthanan Sekar and Ganesh Kumar T. Hyperparameter Tuning in Deep Learning-Based Image Classification to Improve Accuracy using Adam Optimization [J]. Int J Performability Eng, 2023, 19(9): 579-586.
[2]	Sanjay Razdan, Himanshu Gupta, and Ashish Seth. D-SVM: A Deep Support Vector Machine Model with Different Kernel Types for Improved Intrusion Detection Performance [J]. Int J Performability Eng, 2023, 19(9): 598-606.
[3]	Aashita Rajput, Muskan Yadav, Sachin Yadav, Megha Chhabra, and Arun Prakash Agarwal. Patch-Based Breast Cancer Histopathological Image Classification using Deep Learning [J]. Int J Performability Eng, 2023, 19(9): 607-623.
[4]	Kavita Pandey, and Dhiraj Pandey. Real-Time Crop Disease Detection and Remedial Suggestion through Deep Learning-based Smartphone Application [J]. Int J Performability Eng, 2023, 19(8): 491-498.
[5]	Savita Khurana, Gaurav Sharma, and Bhawna Sharma. Hybrid Machine Learning Model for Load Prediction in Cloud Environment [J]. Int J Performability Eng, 2023, 19(8): 507-515.
[6]	Manvi Khatri and Ajay Sharma. Deep Learning Approach based on Iris, Face, and Palmprint Fusion for Multimodal Biometric Recognition System [J]. Int J Performability Eng, 2023, 19(6): 407-416.
[7]	Shreshtha Singh and Arun Sharma. State of the Art Convolutional Neural Networks [J]. Int J Performability Eng, 2023, 19(5): 342-349.
[8]	Vaishali Arya and Tapas Kumar. Boosting X-Ray Scans Feature for Enriched Diagnosis of Pediatric Pneumonia using Deep Learning Models [J]. Int J Performability Eng, 2023, 19(3): 175-183.
[9]	Shikha Choudhary and Bhawna Saxena. Image-Based Crop Disease Detection using Machine Learning Approaches: A Survey [J]. Int J Performability Eng, 2023, 19(2): 122-132.
[10]	Pavneet Singh, Jigyasa Chopra, Amandeep Singh, Nikita Nijhawan, and Kritika. Deep Learning Innovations for Enhanced Drusen Detection in Retinal Images [J]. Int J Performability Eng, 2023, 19(12): 779-787.
[11]	Mansi Mahendru and Sanjay Kumar Dubey. Portable Learning Approach towards Capturing Social Intimidating Activities using Big Data and Deep Learning Technologies [J]. Int J Performability Eng, 2022, 18(9): 668-678.
[12]	Sandhya Alagarsamy and Visumathi James. RNN LSTM-based Deep Hybrid Learning Model for Text Classification using Machine Learning Variant XGBoost [J]. Int J Performability Eng, 2022, 18(8): 545-551.
[13]	K. Lavanya, Smrithi Prakash, Yash Gedam, Altamash Aijaz, and L. Ramanathan. Real Time Digital Face Mask Detection using MobileNet-V2 and SSD with Apache Spark [J]. Int J Performability Eng, 2022, 18(8): 598-604.
[14]	Houssam Lala, Sidali Bacha, Ahmed Bellaouar, and Redouane Zellagui. Modelling the Maintenance of Complex Repairable Systems based on Reliability by Comparing the Proportional Intensity Model and the Generalized Proportional Intensity Model [J]. Int J Performability Eng, 2022, 18(7): 521-528.
[15]	Rajan Prasad Tripathi, Sunil Kumar Khatri, and Darelle Van Greunen. Relative Examination of Breast Malignant Growth Analysis Utilizing Different Machine Learning Algorithms [J]. Int J Performability Eng, 2022, 18(6): 417-425.