Username   Password       Forgot your password?  Forgot your username? 


A Distributed Storage Scheme for Remote Sensing Image based on Mapfile

Volume 14, Number 10, October 2018, pp. 2545-2552
DOI: 10.23940/ijpe.18.10.p30.25452552

Guangsheng Chena,b, Pei Niea,b, and Weipeng Jinga,b

aCollege of Information and Computer Engineering, Northeast Forestry University, Harbin, 150040, China
bHeilongjiang Province Engineering Technology Research Center for Forestry Ecological Big Data Storage and High Performance (Cloud) Computing, Harbin, 150040, China

(Submitted on July 11, 2018; Revised on August 13, 2018; Accepted on September 16, 2018)


Hyperspectral image has a large amount of data and complex structure. The distributed storage of massive remote sensing data is a hot topic today; however, the current research mostly separates the image pixels and metadata, resulting in poor system cohesion and poor data access performance. At the same time, the needs of various upper-level remote sensing algorithms are not fully considered, which makes the system less available. In view of the above problems, this paper presents a distributed image storage model based on HDFS, which stores the entire image data model in a structure to improve the system cohesion, and provides a flexible data blocking strategy for upper-level applications to meet a variety of data access needs. The comparison experiments show that the storage model has better access performance than the existing schemes.


References: 22

                1. A. M. Youssef, B. Pradhan, and E. Tarabees, “Integrated Evaluation of Urban Development Suitability based on Remote Sensing and GIS Techniques: Contribution from the Analytic Hierarchy Process,” Arabian Journal of Geosciences, Vol. 4, No. 3-4, pp. 463-473, 2011
                2. T. Devogele, “On Spatial Database Integration,” International Journal of Geographical Information Systems, Vol. 12, No. 4, pp. 335-352, 1998
                3. J. Sharma, “Oracle Spatial: An Oracle Technical White Paper,” Encyclopedia of Database Systems, Vol. 46, No. 1, pp. 5-25, 2001
                4. A. K. W. Yeung and G. B. Hall, “Spatial Database Systems: Design, Implementation and Project Management,” 2007
                5. W. U. Lun and Y. Zhang, “The Integrated Framework on Distributed Multi-Spatial Database System,” Geography & Territorial Research, Vol. 18, No. 1, pp. 6-10, 2002
                6. J. Zhou, J. Guan, and P. Li, “DCAD: A Dual Clustering Algorithm for Distributed Spatial Databases,” Geo-Spatial Information Science, Vol. 10, No. 2, pp. 137-144, 2007
                7. A. Pavlo, E. Paulson, and A. Rasin, “A Comparison of Approaches to Large-Scale Data Analysis,” in Proceedings of ACM SIGMOD International Conference on Management of Data, pp. 165-178, Providence, Rhode Island, USA, June 2009
                8. Q. Qin, Y. Wang, and M. Huang, “Storage of Massive Remote Sensing Image Data based on MongoDB,” Journal of Beijing University of Civil Engineering & Architecture, 2015
                9. X. Wang, Q. Yang, and F. Chen, “Storage Model Design and Implementation of High Resolution and Hyperspectral Remote Sensing Image based on NoSQL,” Earth Science, 2015
                10. W. Kou, X. Yang, and C. Liang, “HDFS Enabled Storage and Management of Remote Sensing Data,” in Proceedings of IEEE International Conference on Computer and Communications, pp. 80-84, 2017
                11. C. Liu, “An Improved HDFS for Small File,” in Proceedings of IEEE International Conference on Advanced Communication Technology, pp. 1-1, 2016
                12. D. Mukhopadhyay, C. Agrawal, and D. Maru, “Addressing Name Node Scalability Issue in Hadoop Distributed File System using Cache Approach,” in Proceedings of IEEE International Conference on Information Technology, pp. 321-326, 2014
                13. X. Liu, J. Han, and Y. Zhong, “Implementing WebGIS on Hadoop: A Case Study of Improving Small File I/O Performance on HDFS,” in Proceedings of IEEE International Conference on CLUSTER Computing and Workshops, pp. 1-8, 2009
                14. Z. Chi, F. Zhang, and Z. Du, “Cloud Storage of Massive Remote Sensing Data based on Distributed File System,” in Proceedings of IEEE International Conference on Signal Processing, Communication and Computing, pp. 1-4, 2013
                15. L. Ni, “Fast Fractal Coding of Multispectral Remote Sensing Images,” Multispectral Image Processing and Pattern Recognition, International Society for Optics and Photonics, pp. 32-37, 2001
                16. Z. F. Shen, J. C. Luo, and Q. X. Chen, “Data Partition Policy of High-Resolution Remotely Sensed Image Parallel Processing,” Journal of Harbin Institute of Technology, Vol. 38, No. 11, pp. 1968-1938, 2006
                17. X. H. Wang, S. Wang, and W. Wei, “Study on Remote Sensing Image Metadata Management and Issue,” in Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2005
                18. X. Wei, X. Lu, and H. Sun, “Fast View of Mass Remote Sensing Images based-on Image Pyramid,” in Proceedings of International Conference on Intelligent Networks and Intelligent Systems, pp. 461-464, 2008
                19. L. Wang , Y. Ma, and A. Y. Zomaya, “A Parallel File System with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth,” IEEE Transactions on Parallel & Distributed Systems, Vol. 26, No. 6, pp. 1497-1508, 2015
                20. F. Azzedin, “Towards a Scalable HDFS Architecture,” in Proceedings of IEEE International Conference on Collaboration Technologies and Systems, pp. 155-161, 2013
                21. N. Michalakis and D. N. Kalofonos, “Designing an NFS-based Mobile Distributed File System for Ephemeral Sharing in Proximity Networks,” Applications and Services in Wireless Networks, pp. 225-231, 2004
                22. B. Meng, W. Guo, and G. Fan, “A Novel Approach for Efficient Accessing of Small Files in HDFS: TLB-Mapfile,” in Proceedings of IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/distributed Computing, IEEE Computer Society, pp. 681-686, 2016


                              Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

                              Download this file (IJPE-2018-10-30.pdf)IJPE-2018-10-30.pdf[A Distributed Storage Scheme for Remote Sensing Image based on Mapfile]433 Kb
                              This site uses encryption for transmitting your passwords.