A Distributed Storage Scheme for Remote Sensing Image based on Mapfile

doi:10.23940/ijpe.18.10.p30.25452552

Abstract

Abstract:

Hyperspectral image has a large amount of data and complex structure. The distributed storage of massive remote sensing data is a hot topic today; however, the current research mostly separates the image pixels and metadata, resulting in poor system cohesion and poor data access performance. At the same time, the needs of various upper-level remote sensing algorithms are not fully considered, which makes the system less available. In view of the above problems, this paper presents a distributed image storage model based on HDFS, which stores the entire image data model in a structure to improve the system cohesion, and provides a flexible data blocking strategy for upper-level applications to meet a variety of data access needs. The comparison experiments show that the storage model has better access performance than the existing schemes.

Submitted on July 11, 2018; Revised on August 13, 2018; Accepted on September 16, 2018
References: 22

Guangsheng Chen, Pei Nie, and Weipeng Jing. A Distributed Storage Scheme for Remote Sensing Image based on Mapfile [J]. Int J Performability Eng, 2018, 14(10): 2545-2552.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 0

	A. M. Youssef, B. Pradhan, and E. Tarabees, “Integrated Evaluation of Urban Development Suitability based on Remote Sensing and GIS Techniques: Contribution from the Analytic Hierarchy Process,” Arabian Journal of Geosciences, Vol. 4, No. 3-4, pp. 463-473, 2011
	T. Devogele, “On Spatial Database Integration,” International Journal of Geographical Information Systems, Vol. 12, No. 4, pp. 335-352, 1998
	J. Sharma, “Oracle Spatial: An Oracle Technical White Paper,” Encyclopedia of Database Systems, Vol. 46, No. 1, pp. 5-25, 2001
	A. K. W. Yeung and G. B. Hall, “Spatial Database Systems: Design, Implementation and Project Management,” 2007
	W. U. Lun and Y. Zhang, “The Integrated Framework on Distributed Multi-Spatial Database System,” Geography & Territorial Research, Vol. 18, No. 1, pp. 6-10, 2002
	J. Zhou, J. Guan, and P. Li, “DCAD: A Dual Clustering Algorithm for Distributed Spatial Databases,” Geo-Spatial Information Science, Vol. 10, No. 2, pp. 137-144, 2007
	A. Pavlo, E. Paulson, and A. Rasin, “A Comparison of Approaches to Large-Scale Data Analysis,” in Proceedings of ACM SIGMOD International Conference on Management of Data, pp. 165-178, Providence, Rhode Island, USA, June 2009
	Q. Qin, Y. Wang, and M. Huang, “Storage of Massive Remote Sensing Image Data based on MongoDB,” Journal of Beijing University of Civil Engineering & Architecture, 2015
	X. Wang, Q. Yang, and F. Chen, “Storage Model Design and Implementation of High Resolution and Hyperspectral Remote Sensing Image based on NoSQL,” Earth Science, 2015
	W. Kou, X. Yang, and C. Liang, “HDFS Enabled Storage and Management of Remote Sensing Data,” in Proceedings of IEEE International Conference on Computer and Communications, pp. 80-84, 2017
	C. Liu, “An Improved HDFS for Small File,” in Proceedings of IEEE International Conference on Advanced Communication Technology, pp. 1-1, 2016
	D. Mukhopadhyay, C. Agrawal, and D. Maru, “Addressing Name Node Scalability Issue in Hadoop Distributed File System using Cache Approach,” in Proceedings of IEEE International Conference on Information Technology, pp. 321-326, 2014
	X. Liu, J. Han, and Y. Zhong, “Implementing WebGIS on Hadoop: A Case Study of Improving Small File I/O Performance on HDFS,” in Proceedings of IEEE International Conference on CLUSTER Computing and Workshops, pp. 1-8, 2009
	Z. Chi, F. Zhang, and Z. Du, “Cloud Storage of Massive Remote Sensing Data based on Distributed File System,” in Proceedings of IEEE International Conference on Signal Processing, Communication and Computing, pp. 1-4, 2013
	L. Ni, “Fast Fractal Coding of Multispectral Remote Sensing Images,” Multispectral Image Processing and Pattern Recognition, International Society for Optics and Photonics, pp. 32-37, 2001
	Z. F. Shen, J. C. Luo, and Q. X. Chen, “Data Partition Policy of High-Resolution Remotely Sensed Image Parallel Processing,” Journal of Harbin Institute of Technology, Vol. 38, No. 11, pp. 1968-1938, 2006
	X. H. Wang, S. Wang, and W. Wei, “Study on Remote Sensing Image Metadata Management and Issue,” in Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2005
	X. Wei, X. Lu, and H. Sun, “Fast View of Mass Remote Sensing Images based-on Image Pyramid,” in Proceedings of International Conference on Intelligent Networks and Intelligent Systems, pp. 461-464, 2008
	L. Wang , Y. Ma, and A. Y. Zomaya, “A Parallel File System with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth,” IEEE Transactions on Parallel & Distributed Systems, Vol. 26, No. 6, pp. 1497-1508, 2015
	F. Azzedin, “Towards a Scalable HDFS Architecture,” in Proceedings of IEEE International Conference on Collaboration Technologies and Systems, pp. 155-161, 2013
	N. Michalakis and D. N. Kalofonos, “Designing an NFS-based Mobile Distributed File System for Ephemeral Sharing in Proximity Networks,” Applications and Services in Wireless Networks, pp. 225-231, 2004
	B. Meng, W. Guo, and G. Fan, “A Novel Approach for Efficient Accessing of Small Files in HDFS: TLB-Mapfile,” in Proceedings of IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/distributed Computing, IEEE Computer Society, pp. 681-686, 2016