Tecnologías de Big Data geoespacial en el Centro de Información Geoespacial de Geocuba

José Luis Capote Fernández; Rafael Cruz Iglesias

Authors

José Luis Capote Fernández Empresa GEOCUBA Investigación y Consultoría
Rafael Cruz Iglesias Empresa GEOCUBA Investigación y Consultoría https://orcid.org/0000-0002-7564-7779

Keywords:

Big Data Geoespacial, Centro de Información Geoespacial, Infraestructura de Datos Espaciales

Abstract

Since 2006, the GeoMIX Agency, of GEOCUBA, has been dealing with the problems related to Big Data in the framework of the Fleet Management and Control project, Movilweb. The large volume of information, its variety and the speed with which the analyzes are required have been a constant in this work. Until recently the solutions to these problems were always related to changes in architecture, index optimization, increased computing resources, etc. Although technologies related to BigData were already used in the global context, it is not until recently that, thanks to the benefits of Docker, they were enabled for wider use. Within the framework of this investigation, it allowed us to test and evaluate a variety of tools of this type and consider them as components of our architectures. During this period, three BigData Geospatial technologies that are Geomesa, Geogrellis and MrGeo have been evaluated in an experiment whose objective is to complete a management flow that includes from ingestion in the BigData cluster to its publication as a Web Service, both of vector data as raster The completion of this flow guarantees the generation of an added value chain on this primary geospatial data. The present work exposes the experiences obtained in the work with BigData Geospatial, its link to some current projects and the future perspectives of its use in the GEOCUBA Geospatial Information Center.

References

Abreus, Y. A. (2006). Tipos de fragmentación en bases de datos distribuidas. Ingeniería Industrial, 27(1), 3–6.

Cugler, D. C., Oliver, D., Evans, M. R., Shekhar, S., & Medeiros, C. B. (2013). Spatial big data: Platforms, analytics, and science. GeoJournal. https://pdfs.semanticscholar.org/c64e/b7f733cf78573e962c6b5df24860eed3aabe.pdf

de la Beaujardiere, J. (2003). OGC Implementation Specification OGC 01-068r3: OpenGIS Web Map Service. Open Geospatial Consortium. http://www.opengeospatial.org/standards/wms

Fernandez, P. M., Martínez, D. O., García, E. P., & Lorenzo, F. B. (s. f.). Vehículos aéreos no tripulados (vant) en cuba, aplicados a la Geomática. Estado actual, perspectivas y desarrollo.

Furht, B., & Villanustre, F. (2016). Introduction to big data. En Big data technologies and applications (pp. 3–11). Springer.

GDAL. (2012). GDAL - Geospatial Data Abstraction Library. http://www.gdal.org/

González Suarez, G., Capote Fernández, J. L., Cruz Iglesias, R., Batule Dominguez, M., Farre Rosales, D., & Fernández Águila, L. (2018, marzo 15). Movilweb sistema de control de flotas, estado actual y perspectivas. Informática Habana 2018. X CONGRESO INTERNACIONAL DE GEOMÁTICA 2018, La Habana, Cuba. http://www.informaticahabana.cu/es/node/4495

Graser, A. (2017, agosto 27). Getting started with GeoMesa using Geodocker | Free and Open Source GIS Ramblings. https://anitagraser.com/2017/08/27/getting-started-with-geomesa-using-geodocker/

Hughes, J. N., Annex, A., Eichelberger, C. N., Fox, A., Hulbert, A., & Ronquest, M. (2015). Geomesa: A distributed architecture for spatio-temporal fusion. Geospatial Informatics, Fusion, and Motion Video Analytics V, 9473, 94730F.

Kim, G., Humble, J., Debois, P., & Willis, J. (2015). The DevOps HandBook. IT Revolution.

Locationtech Labs. (2018). GeoTrellis for PySpark. Contribute to locationtech-labs/geopyspark development by creating an account on GitHub [Python]. LocationTech Labs. https://github.com/locationtech-labs/geopyspark (Original work published 2016)

Abreus, Y. A. (2006). Tipos de fragmentación en bases de datos distribuidas. Ingeniería Industrial, 27(1), 3–6.

Cugler, D. C., Oliver, D., Evans, M. R., Shekhar, S., & Medeiros, C. B. (2013). Spatial big data: Platforms, analytics, and science. GeoJournal. https://pdfs.semanticscholar.org/c64e/b7f733cf78573e962c6b5df24860eed3aabe.pdf

de la Beaujardiere, J. (2003). OGC Implementation Specification OGC 01-068r3: OpenGIS Web Map Service. Open Geospatial Consortium. http://www.opengeospatial.org/standards/wms

Fernandez, P. M., Martínez, D. O., García, E. P., & Lorenzo, F. B. (s. f.). Vehículos aéreos no tripulados (vant) en cuba, aplicados a la Geomática. Estado actual, perspectivas y desarrollo.

Furht, B., & Villanustre, F. (2016). Introduction to big data. En Big data technologies and applications (pp. 3–11). Springer.

GDAL. (2012). GDAL - Geospatial Data Abstraction Library. http://www.gdal.org/

González Suarez, G., Capote Fernández, J. L., Cruz Iglesias, R., Batule Dominguez, M., Farre Rosales, D., & Fernández Águila, L. (2018, marzo 15). Movilweb sistema de control de flotas, estado actual y perspectivas. Informática Habana 2018. X CONGRESO INTERNACIONAL DE GEOMÁTICA 2018, La Habana, Cuba. http://www.informaticahabana.cu/es/node/4495

Graser, A. (2017, agosto 27). Getting started with GeoMesa using Geodocker | Free and Open Source GIS Ramblings. https://anitagraser.com/2017/08/27/getting-started-with-geomesa-using-geodocker/

Hughes, J. N., Annex, A., Eichelberger, C. N., Fox, A., Hulbert, A., & Ronquest, M. (2015). Geomesa: A distributed architecture for spatio-temporal fusion. Geospatial Informatics, Fusion, and Motion Video Analytics V, 9473, 94730F.

Kim, G., Humble, J., Debois, P., & Willis, J. (2015). The DevOps HandBook. IT Revolution.

Locationtech Labs. (2018). GeoTrellis for PySpark. Contribute to locationtech-labs/geopyspark development by creating an account on GitHub [Python]. LocationTech Labs. https://github.com/locationtech-labs/geopyspark (Original work published 2016)

McCoy, M. D. (2017). Geospatial Big Data and archaeology: Prospects and problems too great to ignore. Journal of Archaeological Science, 84, 74–94.

Mohammed, F., Idries, A., Mohamed, N., Al-Jaroodi, J., & Jawhar, I. (2014). UAVs for smart cities: Opportunities and challenges. Unmanned Aircraft Systems (ICUAS), 2014 International Conference on, 267–273.

Moreno, C. S., Valenti, H. R., Rubio, D. M., & Gonzalez, C. J. (2016). Experiencias de Implementación de Despliegue Continuo con una Infraestructura “Conteinerizada”.

NGA. (2015). Home • ngageoint/mrgeo Wiki • GitHub. https://github.com/ngageoint/mrgeo/wiki

OSGeo. (2018). Geotrellis. Geotrellis. https://www.osgeo.org/projects/geotrellis/

Percivall, G., Reichardt, M., & Taylor, T. (2015). Common Approach To Geoprocessing of Uav Data Across Application Domains. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 40(1), 275.

Schut, P. (2007). OGC Implementation Specification 05-007r7: OpenGIS Web Processing Service.

Shekhar, S. (2012). Spatial big data challenges. Keynote at ARO/NSF Workshop on Big Data at Large: Applications and Algorithms, Durham, NC. http://www-users.cs.umn.edu/~shekhar/talk/2012/12.11.sbd.acmgis.pdf

Vretanos, A. (2005). OGC Implementation Specification 04-094: Web Feature Service Implementation Specification. Technical report. Open Geospatial Consortium. http://www.opengeospatial.org/standards/wfs

Willis, J. (2012). The convergence of DevOps. DevOps Blog.

Yao, X., Mokbel, M. F., Alarabi, L., Eldawy, A., Yang, J., Yun, W., Li, L., Ye, S., & Zhu, D. (2017). Spatial coding-based approach for partitioning big spatial data in Hadoop. Computers & Geosciences, 106, 60–67.