UN OPEN GIS CAPACITY BUILDING A.Albertella a, M.A. Brovelli.a*, D. Gonzalez Ferreiro b
a Politecnico di Milano, P.zza Leonardo da Vinci 32, Milan, Italy - (alberta.albertella, maria.brovelli)@polimi.it b United Nations Global Service Centre, Piazza del Vento 1, 72100 Brindisi, Italy - gonzalezferreiro@un.org
Commission IV, WG IV/4
KEY WORDS: Open source GIS, Capacity Building, UN,
ABSTRACT:
The UN Open GIS Initiative is to identify and develop, under UN guidance, an Open Source GIS bundle that meets the requirements of UN operations, taking full advantage of the expertise of mission partners (partner nations, technology contributing countries, international organizations, academia, NGO’s, private sector). The project, started in 2016, is composed by 4 working groups. One of the working group is specifically related to Capacity Building, given its importance for the success of the project.
UN Open GIS will be based on some existing open source geospatial software (packages and libraries) with many extensions specifically developed. The users of the platform will be the UN staff supporting with mapping and GIS the peacekeeping missions. Therefore, they are generally expert of this specific domain, even if they are currently using proprietary software. UN Open GIS Capacity Building is specifically thought for covering this gap, providing them the suitable background about open source geospatial software in general and the education tailored to the solution that has been being developed within the project itself.
1. INTRODUCTION 1.1 FOSS4G Software, OSGeo and GeoForAll
FOSS4G (Free and Open Source for Geospatial) are geospatial software that provide the user with the freedom to run the program for any purpose, access the source code to study how it works, change it and redistribute copies of modified versions of the software (GNU Project, 1996). The software must comply with the 10 criteria listed in the Open Source Initiative (The Open Source Initiative, 2007). Currently there is at least one mature sophisticated FOSS4G for every geo-technology area, geospatial information need and application, from data collection in the field, crowdsourcing, desktop applications, spatial extensions and Database Management Systems, to software stacks for the creation of sophisticated Web-based systems using client-server architectures (Brovelli et al., 2017; Moreno-Sanchez, 2012; Steiniger and Bocher, 2009; Steiniger and Hunter, 2012; Steiniger and Hunter, 2013). Currently there are over 350 FOSS4G projects listed in FreeGIS.org. Some of these projects have a history that dates back to the early 1980s, like GRASS GIS (Mitasova and Neteler, 2004; GRASS GIS, 2017a; GRASS GIS, 2017b) while others are more recent and yet have a wide and solid user base, such as GeoNode, a collaborative geo platform (GeoNode, 2017a; GeoNode, 2017b).
At the beginning, the different communities were spontaneously born around a project or a piece of software and only had sporadic interaction but in February 2006, some leading teams in the free and open source geospatial world joined their efforts to create OSGeo, Open Source Geospatial Foundation (OSGeo Wiki, 2017; Di Stefano, 2017; FOSSGIS, 2017), a not-for-profit organization whose mission is to support the collaborative development of open geospatial technologies and data and to promote their widespread use.
The formal birth of such a kind of organization highlights on one hand the maturity reached by the different “tribes” aware of the necessity of synergizing the tools development, requiring
the nature of geospatial software to have a high level of interaction. On the other hand, the Internet challenge, with new exigencies and consequent products, and the exit of geospatial data and applications from the niche status were determining in that choice.
From the beginning, the main benchmarks were interoperability and the choice of Open Source Initiative certified licenses allowing different technologies not only to work together but also to integrate and exchange pieces of code among them.
At the same time OSGeo guarantees the quality of the software following a procedure in which each software project, before its release and licensing, goes through an incubation process within a specific Committee of the organization, (OSGeo Wiki, 2017).
The principles of the OSGeo projects (globally called the “OSGeo Way”) are summarized by the following concepts:
· consensus/inclusiveness: the participation from all people -from beginning users to advanced developers - is welcomed;
· fostering: as most contributions are donated, projects encourage and gratify the participation of its volunteers;
· openness: projects adopt open standards and collaborate with other OSGeo projects (Rasdaman, 2017);
· responsibility: projects are responsible for checking their code integrity with respect to the open source basics.
are currently: OPticks (Desktop Applications), METACRS and ORFEO ToolBox (Geospatial Libraries) and istSOS, PyWPS, Team Engine and ZOO (Web Mapping).
For each project, even if still at the incubation stage, all the necessary materials and support for the Community are provided (forums, wikis, and mailing lists) and specific national or international short meetings are usually organized. These occasions are generally linked to ad-hoc Hackers' Code Sprints.
Besides the project teams, the Committees can be highlighted: Open GeoScience, Public Geospatial Data, System Administration, Conference, Incubation, Marketing, Website, UN and GeoForAll, which is the OSGeo Education Committee. The last two are those involved in the UN Open GIS Initiative.
Different support arrangements are made available by the global community for people embracing the usage or development of FOSS (Free and Open Source Software) tools: the OSGeo portal and specific websites for the different projects, forums, wikis, mailing lists global and related to the various projects or initiatives, blogs, Web tutorials and Webinars and the monthly Newsletter provided by the OSGeo Education Committee, which is named GeoForAll, (GeoForAll-ISE, 2017).
1.2 Open Source Software vs. Property Software
The main differences between Open Source Geospatial Software and Proprietary solutions are related to the greater freedom of the former family with respect to usage and development. FOSS4G software has improved enormously in the last years and, from the point of view of a common user, there are no significant differences in terms of performances and reliability, (FOSS4G, 2017). The main barriers of an organisation in deciding to migrate to FOSS4G can be (Brovelli et al., 2017): the prejudice on the quality of Open Source Software; the lack of awareness of software existence, relevance, or successful implementations; the shortage of technical knowledge needed to implement and use the software; already existing favourable arrangements with a proprietary vendor (e.g. discounts; training or support); big/expensive prior investments in proprietary software infrastructure with consequent problematic changing of the operating model and workflow; staff resistance in adopting new operating models; staff resistance due to fear of being deskilled if using FOSS4G instead of commercial packages; worry of shortage of providers, expertise, and traditional support; customization needs and the worry of the possible costs; lack of clarity or knowledge on licensing issues; costs related to training and customisation.
Adopting an open source solution generally reduces the overall costs although does not eliminate them completely. The main advantage of an open source solution stands in the freedom from not being bound to a particular firm; on the contrary, thanks to the interoperability of the solutions and to the open code, there is the possibility of changing suppliers and consultants. In addition, the existence of communities of developers, users and educators around the various projects guarantees the sustainability over time of the adopted GIS solution. Much progress has been done in recent years to address some of the restraining issues and enhance the facilitating ones. Major FOSS4G community groups and initiatives are enhancing awareness, facilitating access to support materials and education, establishing best practices, designing systematic approaches to ensure software can be trusted, assessing software integrity and security, and sharing successful implementation experiences. In ten years the
Community has grown not only numerically but also in experience and credibility and now FOSS4G is the main competitor of the (few) proprietary geospatial software leaders.
For the previous reasons, the United Nations (UN) initiated an effort aiming at assessing the capabilities of current open source geospatial software in front of operational requirements, as well as complementing and replacing the existing proprietary geospatial software. The product which will derive from that effort, named UN Open GIS SDI, will be based on some existing open source geospatial software (packages and libraries) with many extensions that are being specifically developed. The users of the platform will be the UN staff supporting with mapping and GIS to peacekeeping missions. Therefore, they are generally expert of this specific domain, understanding how to elicit their consensus and interest.
2. PROJECT DESCRIPTION
2.1 UN Open GIS Initiative: Background and General Project Description
The United Nations wants to further support the use of GIS technologies and software packages as key elements to aid the UN peacekeeping operations within the GIS framework, initiated the UN Open GIS Initiative and this is the reason why the Organization is promoting the adoption of widely, effective and proven technology coming from the geospatial open source world to complement and, in some cases, replace proprietary software.
UN field operations have traditionally relied on proprietary software for its mapping activities since the inception of the first peacekeeping missions in the early 2000s. The general perception at that time was that the open source technology was not mature enough to be able to compete with the existing proprietary solutions and the GIS personnel had not had any relevant exposure to open source tools that could be efficiently leveraged in the field context. The number of missions having GIS capacity has increased dramatically for the last fifteen years along with the maturity of the geospatial science and the growth of potential projects and areas where to apply the technology. Since then the scenario has substantially changed and GIS has become an ubiquitous technology that consumers of location-based tools in the missions and headquarters continuously demand. In line with these new requirements, Senior Management in UN DFS have consistently insisted in having a wider pool of software options thus in 2016 the UN launched the UN Open GIS Initiative through a diverse partnership involving the United Nations, member states willing to contribute, international organizations, academia, non-profit organizations and the private sector.
address these issues, the Geospatial Information Section (GIS) of the Information and Communications Technologies Division (ICTD), within the Department of Field Support (DFS) and its partners within the UN Open GIS Initiative established a few priorities that were organized in different working groups, called spirals. The first was to build an enterprise GIS that would encompass in an integrated manner the full GIS framework through the establishment of a web-based, SDI-like tool as an ecosystem for managing geospatial data and content sharing platform. The second was the inclusion of the capacity building as key factor to address the challenges associated with the day-to-day use and management of the open source software in order to maximize the UN’s capability for self-sustainment once the new enterprise system has been put in place and assure a smooth transition from the legacy technology to the open source in a step by step approach and without impacting the existing operations within the organization.
2.2 The Working Group Related to Capacity Building From the UN perspective one of the main priorities in establishing the capacity building program was to make sure that every UN GIS staff working in peacekeeping missions was as comfortable using and developing open source technologies as they had been in doing so with the proprietary technology. In this context, the UN Open GIS Initiative decided to establish a working group dedicated to organize and push forward the training agenda for the UN Staff’s disposal: this was called the Capacity Building Spiral or Spiral II. The working group was composed of a renowned figure of the open source geospatial academic world, Professor Maria Brovelli of Politecnico di Milano, as the chairperson, in charge of providing technological guidance about the software stack and coordinating the capacity building activities between the UN, the providers of the Beep online training platform (Politecnico di Milano), and most importantly, the tutors. On the other side, coordinating the activities from within the UN DFS, there was a co-chair, Diego Gonzalez, facilitating the selection and classification of students among the different training sessions and software options. The spiral was also composed of a Technical Committee that would give advice, support and collaborate in the establishment of the training structure. The committee was made of contributions from staff in field missions and other technical experts.
To officialise the announcement of the training opportunities for each selected software it was decided to send an official fax from the Director ICTD to the Directors of Missions Support in each field mission in order to explain the purpose of the capacity building programme, to give the necessary background about the UN Open GIS Initiative and to seek recommendations to participate in the training sessions. The responses were received from each interest peacekeeping mission and an iterative process was developed with all the staff interested to classify them in four different sessions per software stack based on their time preferences and according to a sought balance between expertise, interests and mission where each student was working in.
2.3 The UN’s Staff Survey
It was decided to send an electronic survey to the main stakeholders in all the UN peacekeeping missions where there is a GIS present to find out the level of exposure to open source geospatial software that the UN personnel had, to understand the preconception and vision about the available open source solutions and lastly the preferred educational options that the GIS staff were offered.
The survey was opened for a period of almost one month between July 7th and August 2nd, 2016 in which 32 responses were received from 10 different UN peacekeeping missions out of a total of 17 missions (including the UN Headquarters) with GIS capacity. The survey was composed of 19 questions divided in three different sections. The first one tried to know more about the general background of the staff and the knowledge, skills and experience with different software options; the second was focused on capturing the staff’s thoughts about which particular approach they thought the UN in the peacekeeping environment should take with regards to the adoption of open source geospatial technologies, what are, according to them, the advantages of open source versus proprietary software and their interaction and with and exposure to the existing GIS software, divided in desktop GIS, spatial databases, web servers and GIS frameworks and web libraries. The last part was devoted to understanding the staff’s learning preferences about the available courses, their format, modality and the type of training that they thought was ideal based on their needs, requirements and availability in terms of time. The answers that were collected helped later on shaped the course dynamics and how the online training sessions were conducted.
The background and areas of expertise that better described the respondents were, by a large majority, cartography (55.2%), followed by geography (41.4%) and environmental science (34.5%). Among the staff, the current role that better defined the current role of those who responded were GIS Analyst (37.9%), GIS Manager (31.0%) and Mapping Specialists (27.6%). The number of years of the UN staff with relevant experience in open source geospatial domains showed that they were more knowledgeable in desktop GIS software as more than half of the respondents claimed to have over five years of experience. These numbers decreased when they were asked about experience with database management since eleven of them mentioned that had five or more years of exposure to open source spatial databases. The numbers further decreased when they had to asses about web development as the big majority they had no relevant experience.
When asked about their willingness to leverage geospatial open source science and their vision about which path the UN should follow with regards to the adoption of this technology a big what open source could offer but without having to renounce to the existing acquired experience, advocating in a foreseeable future for a balanced coexistence between both technology options.
a) Customization b) Tailored Support
c) Open Standards d) Bug Fixes
e) Security f) Usability
g) Product Stability h) Training Resources
i) Overall Costs
Figure 1: Comparison of open source vs. proprietary software
The survey also enquired about the perception that the UN staff have of the open source software in comparison to the
proprietary, focusing on the advantages of one over the other in relation to nine predefined characteristics. The open source software rated better in overall costs, customizations and open standards but was perceived worse about tailored support, usability and product stability. The below Figure 1 shows the responses to the nine different topics related to the question of comparing open source versus proprietary geospatial software. With regards to the item about training resources in the open source environment, UN staff did not have a definite answer as all options had received the same number of responses.
The last part of the survey was dedicated to the experience, best options and expectations about the specific training on open source geospatial software. While 41.7% responded that they had never had any previous training experience, only 8.3% mentioned that they had received some kind of official training. In relation to the topics that the students wanted to cover were, in order of preference, the desktop mapping, database management, online mapping and web development, which have proven to be the same as the proposal during the establishment of the UN Open GIS Initiative and its panel of experts. When asked about their preferred training format, an important majority (54.2%) chose to have an on-site, instructor-led training over other options such as online tutor-supported training (20.8%), which was in the end the option selected due to the difficulties in terms of resources and time allotment to bring staff scattered throughout many missions to a selected location where the courses would be delivered. In line with the previous question, it was also asked about the preferred amount of time to be dedicated to each course and the answers matched the option selected above since a 58.3% stated that a training organized in full-day format spanned during three days was preferred over a training organized over a one-month period, with 2-3 hours of dedication per week (25.0%) and with specific milestones that had to be accomplished. Finally, the staff consulted were enquired about the importance given to having to take an exam at the end of the course and receiving a certificate upon the successful completion of the training sessions. The respondents overwhelmingly selected to include an exam (79.2%) at the end and expressed that a certificate would be an incentive to dedicate time and effort toward the finalization of the course.
An additional generic online survey was distributed to all UN staff upon completion of each training session in order to capture their feedback about how the course had been delivered and send their comments on how to improve the interaction between students and tutors. This survey was sent for the first three QGIS courses but was later on dismissed, as the number of responses was low compared to the total number of students enrolled in each session.
In general, the responses were very positive and in most cases praised the work done by the tutors and the platform’s support. Once analysed the responses and comments of the students, it was evident that there will be the need to organize at one point in the future an advanced QGIS course, ideally an on-site, instructor-led training, where more difficult functionality could be added such as creation of plugins, etc. Also another relevant comment that was expressed was the option to include data relevant to UN operations and tasks and tools that are commonly used or demanded by the UN colleagues in the field.
2.4 The preparatory phase
After the kick-off meeting in Brindisi, two calls for contributions were launched in the GeoForAll and OSGeo Discussion Mailing Lists. The first was about compiling the already available courses and training material related to Open Source geospatial software. The second was aimed at finding experts willing to revise the existing course materials and to act as tutors for the online self-paced trainings. The community, as always, responded generously. In few days a total of 19 training course were listed on the OSGeo wiki: QGIS, gvSIG, GRASS GIS, GeoServer, OpenLayers, PostGIS, GeoGig, GeoNode, Rasdaman, OSGeo-Live, Bringing GEOSS services into practice, GeoMOOSE, GET-IT, Open Source Geospatial Notebooks. Meanwhile the outcome of the UN’s staff survey was available and therefore it was decided to start revising the training materials available for QGIS, PostGIS, GeoNode, Geoserver and Openlayers. The intention was to provide the UN staff with some materials already tested and suitable for their level of knowledge. This attention was due to the fact that the staff is generally involved in peacekeeping missions and, for them, finding time for training is difficult: we wanted to put them in the most comfortable situation, providing reliable and tested material. Five teams of community members and some students of Politecnico di Milano revised the documents, going through the training step by step and advising about possible mistakes or items difficult to understand. All the reviews were sent to the authors of the training courses documentation and they provided feedbacks and revisions accordingly. For the first two courses, QGIS GeoAcademy and Boundless PostGIS were chosen because they were considered complete and suitable, (PostGIS, 2017a; PostGIS, 2017b).
The second call for tutors was again a success. In few days we had some volunteer tutors for both courses. Tutors are generally faculty already teaching these subjects, in some cases using these tutorials. This was important in order to avoid making up volunteering a too onerous task. We used Skype, email lists, the Beep Platform (see later on) and Google Docs spreadsheets to manage interactions among tutors. Some of the tutors had never met in their real life but the impression was that they felt comfortable and happy when there was an occasion to remotely meet to discuss some follow-up matters. The majority answered promptly to emails and were always available.
As chairs, we tried to reduce all barriers, by providing step-by-step instructions, to set up a schedule and work on it, to talk with tutors regularly or at least provide prompt answers to their queries. The tutors, together with the staff of Politecnico managing the Beep platform (see next section) were publicly acknowledged by a letter from the UN. The Chief of the Geospatial Information Section explicitly appreciated the dedicated work and effort supporting the courses to a large number of UN staff members.
3. ONLINE COURSES ORGANIZATION 3.1 The Beep platform
Training is conducted through the open source e-learning platform of Politecnico di Milano, namely: Beep. It is a multi-platform (working with all operative systems and all browsers) and responsive (working on PC, tablet and smartphone) web application. Beep is the LMS (Learning Management System) used by Politecnico di Milano to manage all of its courses. This involves approximately 50,000 active users every year. Beep is available in Italian as well as in English and it supports both traditional learning and online courses. It can be customized to support the specific needs of both small tutorials and larger lectures.
Beep can be tailored according to the instructors’ needs, who are thus able to manage the course by modifying the layout of online pages and inserting various functionalities as applicable. By permitting students to back-up and repeat segments of a course prior to moving on, Beep allows all students to complete activities at their own pace with the course content remaining available whenever needed.
In particular, it is possible to insert documents (i.e., slides or similar materials as uploaded by the faculty and assignments as uploaded by students), share online news and announcements, manage forums on specific topics as predefined by the instructors, and use Google utilities such as Google drive modules.
Through Beep, courses can be taught independently, since students and professors may work during different time periods and do not necessarily have real-time interaction with each other. This is crucial considering the fact that participating students of the UN Open GIS initiative are spread out around the world with different corresponding time zones.
Furthermore the platform Beep features an online conferencing system that allows for webinars and online meetings. Instructors can train, share notes or presentations and students can participate by asking questions. For students that are unable to attend live sessions, the webinar is recorded and included in the respective course’s online page.
Beep is available through specific credentials obtained from Politecnico di Milano. The technical staff of the ‘Department of Innovative Technologies and Methods for the Science of Teaching’ (Metodi e Tecnologie Innovative per la Didattica, hereafter METID) of Politecnico di Milano manages the platform. They administer the students’ enrolment process as well as the technical aspect, having resolved the few technical setbacks that have arisen. The technical team is always easily reachable through default online chat and via email correspondence (almost 700 chats and around 3,600 emails per year).
Within the framework of the UN OpenGIS initiative Beep has been and will be proposed to the tutors with its standard configuration. The tutors adapt the layout and the functionalities following the specific needs of the course.
maintain QGIS. As a free software application QGIS can be freely modified to perform different or more specialized tasks. PostGIS is a spatial database extender for PostgreSQL object-relational database. It adds support for geographic objects allowing location queries to be run in SQL (https://postgis.net/).
The QGIS course has been held by Richard Smith, Assistant Professor of Geographic Information Science at Texas A&M University, Corpus Christi, USA, assisted by Thomas Mueller (California University of Pennsylvania, USA) and Youngok Kang (Ewha Womans University, South Korea). The PostGIS course has been held by Ivana Ivanova Professor at the Department of Cartography of the Faculty of Science and Technology of the State University of São Paulo in Brazil, assisted by Mike Pumphrey (Boundless, USA), Gregory Giuliani (Institute for Environmental Sciences, University of Geneva, Switzerland), Paolo Corti (Center for Geographic Analysis Harvard University, USA), Victoria Rautenbach (Department of Geography, Geoinformatics and Meteorology at the University of Pretoria, South Africa).
For each course, 61 students have been selected by UN Global Service Centre. To make the training more efficient, both courses have been divided into four sessions; containing approximately 15 students per session (see Table 1). The average percentage of passed exams is around 50%. This result is quite satisfactory considering the difficult situations in which the students were. For example they often moved workplace with serious problems related the Internet access.
course dates students exams
Table 1. Summary of the online courses carried out.
The future planned courses for the content management and geospatial server are GeoNode and GeoServer. Finally OpenLayers will be the course intended for the online mapping library for web development.
4. CONCLUSIONS
In conclusion, we can remind that the selection of the QGIS and PostGIS software has been a successful choice. These open source GIS tools are the most common available and the widely used. Both are established since long time and very well documented. This is why they have been chosen from the software bundle suggested by Geo4All and OSGeo as starting courses.
The answers collected after each training session were definitely helpful in adapting the subsequent courses offered to the needs expressed by the students. An effort was made to understand the student’s feedback and to steer the following sessions in order to make their educational experience more meaningful and useful. By facilitating students’ participation, an excellent collaboration between tutors and an efficient communication between tutors and students, Beep confirms to be a relevant and suitable online training platform for the UN staff and the UN
Open GIS Initiative. Despite that, it could be interesting to have also an on-site training to improve the already satisfying results of such Initiative.
ACKNOWLEDGEMENTS
We thank all tutors of QGIS and PostGIS, which have make possible the courses’ progress. Special thanks are due to Gabriele Cristini, Lia Navarotto e Bianca Santolini METID, Politecnico di Milano for their constant technical support.
REFERENCES
Brovelli M., Minghini M., Moreno-Sanchez R. and Oliveira R, 2017. Free and open source software for geospatial applications (FOSS4G) to support Future Earth. Journal of Digital Earth,
FOSS4G GeoAcademy Curriculum, 2017.
http://spatialquerylab.com/foss4g-academy-curriculum/ (Accessed 16 May 2017),
FOSSGIS e.V., 2017 FreeGIS Project http://freegis.org (Accessed 17 May 2017).
GeoForAll - ISE, University of Geneva, 2017. Bringing GEOSS services into practice, http://www.geossintopractice.org/ (Accessed 17 May 2017).
GeoNode, 2017a. GeoNode tutorials,
http://docs.geonode.org/en/latest/tutorials/index.html (Accessed 17 May 2017).
GeoNode, 2017b. WFPGeoNode Training for users, http://geonode.wfp.org/trainings/4/ (Accessed 17 May 2017).
Mitasova, H., and M. Neteler, 2004. GRASS as open source free software GIS: accomplishments and perspectives. Transactions in GIS 8 (2), pp. 145-154. doi:10.1111/j.1467-9671.2004.00172.x.
Moreno-Sanchez, R., 2012. Free and Open Source Software for Geospatial Applications (FOSS4G): A mature alternative in the geotechnologies arena. Transactions in GIS 16 (2), pp. 81–88. doi:10.1111/j.1467-9671.2012.01314.x.
OSGeo Wiki, 2017. UNTraining.
https://wiki.osgeo.org/wiki/UNTraining (Accessed 15 May 2017).
OpenLayers, 2017. Boundless Workshops: introduction to OpenLayers, http://workshops.boundlessgeo.com/openlayers3/ (Accessed 17 May 2017).
PostGIS, 2017a. PostgreSQL-PostGIS Tutorial Series University of Colorado Denver FOSS4G Lab http://geospatial.ucdenver.edu/foss4g/training/tutorials
(Accessed 17 May 2017).
http://workshops.boundlessgeo.com/postgis-intro/ (Accessed 17 May 2017).
Rasdaman, 2017. OGC Big Geo Data standards and rasdaman workshops, http://rasdaman.org/wiki/Workshops (Accessed 17 May 2017).
Steiniger, S., and E. Bocher, 2009. An overview on current free and open source desktop GIS developments. International Journal of Geographical Information Science 23 (10), pp. 1345–1370. doi:10.1080/13658810802634956.
Steiniger, S., and A. J. Hunter, 2012. Free and open source GIS software for building a spatial data infrastructure. Geospatial free and open source software in the 21st century, pp. 247–261. doi:10.1007/978-3-642-10595-1_15.
Steiniger, S., and A. J. Hunter, 2013. The 2012 free and open source GIS software map–A guide to facilitate research, development, and adoption. Computers, Environment and
Urban Systems 39, pp. 136–150.
doi:10.1016/j.compenvurbsys.2012.10.003.
