This is the course outline.
Geography 585: Open Web Mapping is a 10-week online course that gives you experience with sharing geographic information on the Internet using free and open source software (FOSS) and open specifications. It is an elective course in Penn State's online geospatial education programs, including the Master of Geographic Information Systems. The lessons are freely visible online as open courseware; however, if you would like to receive official credit for the course and complete the lessons with instructor and peer feedback, you should register for the course through the Penn State World Campus.
The two main purposes of Geog 585 are to help you understand the importance of web services and to give you some experience making web maps with FOSS and open standards. You could certainly implement web services using proprietary software, too; however, the cost of a proprietary GIS server package makes FOSS an attractive area of study for basic web mapping tasks.
The purpose of Geog 585 is not to promote FOSS over proprietary software, or vice versa (you could find plenty of materials on the Internet debating this subject); however, Geog 585 should familiarize you with the capabilities and shortfalls of the current FOSS landscape to the point that you can make an informed decision about whether to use FOSS in your own web mapping tasks.
This course requires you to do some programming with JavaScript and the Leaflet API. You don't have to know anything about Leaflet yet, but it is required that you have:
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
In this lesson, you'll learn some of the history of web mapping and why web services are so important. You'll also learn about free and open source software (FOSS) and the benefits and drawbacks of using it. Finally, you'll get the chance to install and use QGIS, a FOSS solution for desktop GIS work. QGIS will help you preview and manipulate your data as you prepare to create web maps later in the course.
For decades, most digital geographic information was confined for use on desktop-based personal computers (PCs) or in-house mainframes and could not be easily shared with other organizations. GIS analysts would access data from their own workplace computers that were often connected to a central file server somewhere in the office. Specialized software was required to view or manipulate the data, effectively narrowing the audience that could benefit from the data.
With mass uptake of the Internet in the mid-1990s, people began thinking about how maps and other geographic information could be shared across computers, both within the organization and with the general public. The first step was to post static images of maps on HTML pages; however, people soon realized the potential for interactive maps. The first of these, served out by newborn versions of software such as Map Server and Esri ArcIMS, were horrendously pixelated, slow, and clunky by today's standards. Limited by these tools, cartographers had not yet arrived en masse on the web mapping scene, and most of the maps looked hideous. However, these early interactive web maps were revolutionary at the time. The idea that you could use your humble web browser to request a map anywhere you wanted and see the resulting image was liberating and exciting. See Brandon Plewe's 1997 book GIS Online (findable in many university geography libraries) to get a feel for the web mapping landscape at this time period.
These early, dynamically drawn web maps ran into challenges with speed and scalability (the ability to handle many simultaneous users). The server could only accommodate a limited number of map requests at a time before slowing down (at best) and crashing (at worst). Web maps matured significantly in these two metrics when websites began to serve out tiled map images from pregenerated caches. Why ask the server to draw every single map dynamically when you could just put forward an initial investment to predraw all possible map extents at a reasonable set of scales? Once you had the map images drawn and cached, you could serve out the images as a tiled mosaic. Each tiled map request was satisfied exponentially faster than it would take to serve the map dynamically, allowing for a server to accommodate hundreds of simultaneous users.
Following the lead of Google Maps, many sites began serving out “pre-cooked” tiled map images using a creative technique known as Asynchronous JavaScript and XML (AJAX) that eliminated the ubiquitous and annoying blink that occurred after any navigation action in earlier web maps. Now you could pan the map forever without your server gasping for breath as it tried to catch up.
Cartographers, who had largely been resigned to trading aesthetics for speed in web maps, also realized the potential of the tiling techniques. No longer would the number of layers in a map slow down the server: once you had pregenerated the tiles, you could serve a beautiful map just as fast as an ugly one. Web maps became an opportunity to exercise cartographic techniques and make the most attractive map possible. Thus were born the beautiful, fast, and detailed “web 2.0” basemaps that are common today on Google, Microsoft Bing, OpenStreetMap, and other popular websites.
As web browsers increased in their ability to draw graphics using technologies such as SVG and later WebGL, the possibilities for interactivity arose. On-the-fly feature highlighting and HTML-enriched popup windows became common elements. For several years, developers experimented with plug-ins such as Adobe Flash and Microsoft Silverlight for smooth animation of map navigation and associated widgets. More recently, developers are abandoning these platforms in favor of new HTML5 standards recognized by the latest web browsers without the need for plug-ins.
Although maps had arrived on the browser by the mid-2000s, they were still largely accessed through desktop PCs. The widespread adoption of smartphones and tablets in subsequent years only increased the demand for web maps. Mobile devices could not natively hold large collections of GIS data, nor could they install advanced GIS software; they relied on web or cellular connections to get maps on demand. These connections were either initiated by browsers on the device, such as Safari, or native applications installed on the device and built for simple, focused purposes. In both cases, GIS data and maps needed to be pulled from the organization's traditional data silos and made available on the web.
All the above web mapping scenarios are possible because of web services. If you search the Internet, you'll find many definitions of web services and can easily get yourself confused. For the purposes of this course, just think of a web service as a focused task that a specialized computer (the server) knows how to do and allows other computers to invoke. You work with the web service like this:
Examine how the following simple diagram describes this process, returning a map of current precipitation:
Now, for an example. Let's suppose that you've identified the URL of a web service out there somewhere that draws maps. You make a request by constructing a URL (http://...) containing the address of the web service and various parameters for the map, such as the format of the image you want to receive (JPG, PNG, etc.), bounding box (coordinates defining the geographic area you want to see mapped), and map scale. You paste this into your browser's address bar and the server sends you a response containing the map image you requested.
Here's an example of just such a request, using a radar map of the United States. First, see if you can identify some of the parameters in the URL. Then take a guess at what the response will look like when you get it back. Then click the link to invoke the web service and get the response:
As you examined the URL of this request, you might have noticed parameters indicating the width and height of the image, the image format, the image background transparency, and the bounding coordinates of the map to be drawn. These parameters provide specific details about how the web service should run its map drawing code. You see these parameters reflected in the response image sent back to your browser when you clicked the link. In a future lesson, you'll learn about more of the parameters in the request above.
Not all web requests invoke web service code. Some web requests just return you a file. This is how tiled maps work, and this is why they are so fast. You'll learn more about tiled maps in a later lesson, but examine the following request for a specific zoom level, row, and column of a tile:
http://a.tile.openstreetmap.org/15/11068/19742.png [7]
The request drills down into the server's folder structure and returns you the requested PNG image as a response. No special code ran on the server other than the basic file retrieval, therefore you could argue that a web service was not invoked. However, this type of simple web request is also an important part of many web maps.
At this point, you might be thinking, “I've used web maps for years and I have never had to cobble together long clunky URLs like this. Have I been using web services and other web requests?” Absolutely, yes. As you navigate Google Maps, your company's online map site, and so on, your browser is sending hundreds of web requests similar to this one. You've just never needed to know the details until now. When you begin setting up your own GIS server or designing your own client web application, it becomes important to understand the theory and architecture behind web traffic.
Not all web services use the same format of URL and parameters. In this course, you'll learn about some of the most common formats for online web services, especially ones that have been openly developed and documented to work across software packages.
Here's a simple way you can view web requests made “behind the scenes” by your browser as you navigate a website. These instructions are for the tools provided by Mozilla Firefox. Chrome and other browsers have similar tools that typically go under the name "developer tools" or "web tools" and should not be difficult to locate.
This kind of developer tool will be valuable to you later in the course as you develop your own web maps, mainly for troubleshooting scenarios where you expect a map to show up and your browser does not send you the expected response.
Plewe, B. (1997). GIS online: Information retrieval, mapping, and the Internet. OnWord Press.
The term “free and open source software” (hereafter referred to as FOSS) includes a number of ideas that can invoke complex and even emotional discussions within the technical community. To begin to describe this term, it's important to understand that software development is more enigmatic and artistic than other crafts in the tech industry [9], such as computer chip design. Software cannot be seen, touched, tasted, or described in a physical sense, other than the number of megabytes it occupies on your computer. Software often begins as ideas on a whiteboard, which are then encapsulated into intangible “classes” and "objects" of code by a programmer. These are then assembled into invokable sets of concrete tasks and dressed with a user interface of buttons, menus, and icons that require a whole other skillset of aesthetic design.
As a result of all this work, software is an empowering technology that enables a person to make practical use of computer hardware. In fact, specialized software can often cost much more than the physical machine that it runs on. Software is your window into printing, painting, calculating, storing data, and, in our case, making maps.
Given the value of software, it's no surprise that commercial businesses invest millions in researching, developing, and selling proprietary software. They protect it through patent and copyright laws. They obfuscate (scramble) the code to make it harder to copy or reverse engineer. Commercial software development has become a successful enterprise whose resulting tools have shaped our work and home environments.
At the same time, some software users and developers have advocated that there are benefits from making software source code freely visible and able to be modified or shared without legal or technical restraint. Business benefits, technical benefits, and moral arguments have been invoked in support of this concept of “free and open source software.”
It's possible to get confused when learning about FOSS, because the word “free” carries several meanings in the English language. A common analogy describing the F in FOSS is “free as in free speech, not free beer.” In other words, FOSS is “free” in the sense that it is open and amenable to use and modification. (You may sometimes see the term Free/Libre [10] Open Source Software (FLOSS) used to communicate this idea.) A whole range of license types [11] are used with FOSS that specify the conditions surrounding the modification and reuse of the software, along with the attribution required on any derived products.
While the selling of FOSS is not necessarily restricted, FOSS is usually available free of charge. All of the software we'll use in this course can be downloaded directly from the Internet and requires no fee, although donating a few dollars to your favorite project is a good way to invest in its continued development.
If FOSS is usually available without cost, why is it such a successful concept? And how do people make a living from coding with it? First, it's important to mention that many developers contribute to FOSS solely for personal enjoyment and for the desire to donate their skills to a project offering wide societal benefits. They enjoy working on virtual teams and facing the challenges of rigorous software development that their ordinary “day jobs” might not offer.
At the same time, though, many individuals make a generous living by selling services and training related to FOSS, and some of these efforts may increase the quality and number of FOSS features available. For example, suppose your company invests in a FOSS package that does everything you want at $30,000 cheaper than the proprietary software alternative; however, it's missing “Feature X” that is critical for your workflows, and you don't have a programmer who can implement Feature X. Because the code for the software is all open to examination, modification, and extension, you can contract with Rock Star Programmer to implement Feature X for you for $10,000. If Feature X is widely applicable to the good of the product, and you and the contractor are okay with the idea, Feature X may eventually be rolled into the core source code so everyone can benefit from it.
Other people may not contribute frequently to FOSS projects, but can still make a profit selling support services for FOSS software. When you buy proprietary software, you often are given access to a technical support package that allows you to call and talk to an analyst if needed. Because FOSS does not offer these official support systems, enterprising individuals have stepped in to fill this void.
Finally, several new firms are building subscription-based online services that are created using open source building blocks and may even be released under open source licenses. The services are offered for a subscription cost that is lower than most organizations could achieve if they attempted to build their own comparable infrastructure or quality control procedures. Through these FOSS-based Software-as-a-Service (SaaS) offerings, the value of the free software is passed on to many buyers.
Although there are many FOSS purists out there, the decision to use FOSS does not have to result in a full migration away from proprietary software. Many businesses and governments use what might be termed a "hybrid approach," incorporating a mix of FOSS and proprietary software depending on their budget, staff skills, and technical needs.
Let's consider some of the benefits, challenges, and other considerations that the adoption of FOSS brings into your workflows.
As you begin the endeavor of learning to use FOSS, it's helpful to understand some of the benefits that you may see:
A danger of evaluating FOSS systems is to allow the potential exciting benefits to obscure the real challenges that can accompany a FOSS deployment. Some or all of the challenges below can be mitigated, but proprietary software may offer a smoother road in these areas, if you can bear the cost.
Some aspects of FOSS and proprietary software are not as clear when it comes to deciding which type of software owns the advantage.
Quality and technical superiority – The “bugginess” of FOSS compared to proprietary software probably depends on the products in question, how mature they are, and who's developing them. FOSS advocate Eric Raymond argued that “given enough eyeballs, all bugs are shallow,” contending that it would be difficult for any major issues with FOSS to go unfixed for long among a broad community of developers. It's less clear who fixes the obscure bugs that you hit and others don't. If you can't convince someone in the FOSS community to fix it, you could at least attempt it yourself or hire a consultant.
Proprietary software vendors obviously have business incentives to fix bugs; however, they also have business incentives to NOT fix certain obscure bugs if the time, effort, and risk to do so would not provide a good return on investment. If you're a small-potatoes customer with an obscure bug holding up your project, you'd better start looking for a workaround.
FOSS has been developed to support all tiers of system architecture. You've probably heard of (or used) some of this software before. For example, the term “LAMP stack” refers to a system that is running:
Other variations of this acronym exist. For example, PostgreSQL is another open source relational database that is commonly used with GIS because of the popular PostGIS extension. This results in a LAPP stack rather than a LAMP stack.
Other general-use FOSS includes the LibreOffice suite (similar to Microsoft Office), the Mozilla Firefox web browser, the Thunderbird e-mail client, the Python scripting language, and more.
Some governments have begun mandating or encouraging the use of FOSS for government offices and projects. This is especially popular in Latin America and Europe, with one of the more recent government decrees occurring in the United Kingdom [14]. Often the FOSS is implemented first on servers and back-end infrastructure, then rolled out to desktop workstations in later phases.
These policies favoring FOSS come about for different reasons. Obviously, the savings in software licenses and the flexible security models offered by FOSS are desirable, but sometimes there are political motivations to reject proprietary software companies and their countries of origin (particularly in places where the United States is perceived as imperialist). If you are interested in more reading on this topic, I recommend Aaron Shaw's study of the FOSS movement in the Brazilian government [15] and its tie to leftist politics [here is an alternative link [16] to the paper in case the other link isn't working].
FOSS has a strong and growing presence in the GIS industry. Some tools and utilities for processing geospatial data have been around for decades. For example, GRASS GIS [17] developed by the US Army Corps of Engineers recently turned 30 years old. To see what it looked like back then, see this old GRASS promotional video [18] narrated by none other than William Shatner. I still use this video in introductory classes to teach the benefits of GIS and the main components of a GIS system.
In this course, we'll use some of these desktop workstation GIS tools for previewing and manipulating our datasets before putting them on the web. For example, later in this lesson, you'll install and explore QGIS [19] (previously known as Quantum GIS), one of the most popular and user-friendly FOSS GIS programs.
There are also various FOSS options for exposing your GIS data on the web, either within your own office network or on the entire Internet. These include Map Server [20], QGIS Server [21], and GeoServer [22], the latter of which you will learn in this course. These software offerings take your GIS datasets and make them available as web services that speak in a variety of formats. They include a web server or are designed to integrate with an existing web server, so that your web services can reach computers outside your own office or network.
FOSS can also be used to generate sets of tiled images or vector data that you can use as layers in your web maps. In this course, you'll learn how to do this using a new tool recently integrated into QGIS.
Underlying both desktop and server GIS are the databases containing your GIS data. If you need something a little more complex than a folder of shapefiles or want to make use of spatial database types, then you can use a FOSS GIS database. These include PostGIS [23] (which is an extension to the PostgreSQL relational database) and SpatiaLite [24] (which uses the SQLite database). A lighter fare option for smaller datasets is to use standalone files of GeoJSON, KML, GeoRSS, or other well-documented text-based data formats.
To pull all your map layers together and display them on a web page, you'll use a programming framework, or API. Some of the most mature web mapping APIs are OpenLayers [25] and Leaflet [26], which you will use later in this course with the JavaScript programming language. Other popular FOSS web mapping APIs include ModestMaps [27], D3 [28], and Polymaps [29].
Not to be confused with open software, open specifications are documented and mutually agreed-upon patterns of how software and digital data should behave in order to be interoperable between systems. For example, HTTP (hypertext transfer protocol) is based on a specification defining how web servers and web browsers should communicate in order to exchange information. Without open specifications, you would not be reading this web page.
In this course, we'll learn about two types of open specifications:
In this course, we'll also be using open data, which is data that has been made available to the public free of charge and bereft of most copyright restrictions. These data may be shared by government entities, researchers, not-for-profit organizations, or ordinary citizens contributing to online projects. For example, Data.gov [30] is a popular website offering datasets collected by the US government. And, later in this course, we'll learn about one significant source of open GIS data called OpenStreetMap [31]. This is an online map consisting of voluntary contributions in the style of Wikipedia.
Open datasets are often rich and exciting to include in web maps, but there are some precautionary measures you should follow in order to be successful with using them. First, be aware that even though the data is free, you are often still required to provide attribution describing where you obtained the data. You may also be restricted from redistributing the data in any way that requires a fee. When you use an open dataset, you are responsible to carefully research and adhere to any attribution requirements. You should also make an effort to verify the data quality by examining any accompanying metadata, researching the sources and collection methods of the data, and scrutinizing the data itself.
In this course, you will be getting some experience installing and using FOSS and creating web maps with it. While doing this, you will use open specifications for GIS data and web services. You'll also learn how to use open data and contribute to OpenStreetMap.
Each week, you'll complete a walkthrough explaining some FOSS tool. Following the walkthrough, I will often ask you to apply what you've learned to some of your own datasets you've found or collected. This will allow you to build up a comprehensive project as the course progresses. The goal is to produce something you can host on your personal webspace and reference in a professional portfolio.
The functionality of our web maps will be relatively simple, limited to layer display and point-and-click queries. However, the frameworks that we'll use are broadly extensible depending on how much programming you're willing and able to do.
Along the way, you'll pick up some skills with manipulating data (projecting, clipping, and so forth) with FOSS, which will hopefully prove handy throughout your GIS career. Many of these tools can be scripted with Python and other languages that you may have learned already in other GIS coursework.
The first FOSS product you'll use is a GUI-based program designed for desktop workstations. It's called QGIS (kyoo-jis), although you should know that sometimes in the past it was referred to as Quantum GIS. QGIS is somewhat similar in appearance and function to Esri's ArcMap, which you've likely used in previous courses.
In this tutorial, you'll install QGIS and make a basic vector map with it. You'll use some shapefiles of downtown Ottawa, Ontario, Canada that I originally downloaded from the OpenStreetMap database.
Download the Lesson 1 walkthrough data [32] (It is a folder of shapefiles that you should extract into a folder such as c:\data).
In this week's assignment, you'll watch and read several opinions on the use of FOSS within the GIS community. I will then ask you to respond to these in a detailed discussion below.
Make a post on the "Lesson 1 assignment forum" on Canvas that addresses both of the following questions:
You can optionally make two posts addressing the questions separately. You will be graded on the depth of the arguments that you put forward in your responses and the evidence you produce to support these arguments, not on your opinions toward FOSS.
To receive full credit, please also write at least one response to another student's post building on whatever he or she said.
I encourage you to continue thinking about these questions throughout this course as you get more hands-on experience with FOSS.
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
How do you start with some raw datasets on your computer and create a beautiful interactive web map that can be enjoyed by thousands of Internet users? This lesson will give an overview of how you can approach a web mapping project. The remaining lessons will get into the step-by-step details of how you make each part happen.
It can take several different physical machines to create, serve, and use a web map. These are often depicted in diagrams as separate levels, or tiers of architecture. In this course, you'll likely use just one machine to play all these roles; however, it's important to understand how the tiers fit together.
For example, you might have:
Again, when developing and testing a web map, you can certainly use a single physical machine to play all these roles. This is a common practice that keeps things simple and more economic. When you work for a “real world” company with its own network and are ready to deploy your web map, you will move your tested services and applications into a formal “production” environment where you have individual enterprise-grade machines for each role, as described above.
Building and using web maps is different from working with a desktop GIS for a variety of reasons:
These considerations can sometimes take people by surprise. For example, if you've only used ArcMap or QGIS in the past, you may not be accustomed to thinking about broadband speeds or sharing the machine with other people.
By far, the biggest challenge for new web mappers is understanding the amount of data that is displayed in their maps and how to get all that information drawn on the screen of a web user in sub-second speeds. Many people who have worked with desktop GIS packages are accustomed to adding dozens (or even hundreds) of layers to the map and switching them on and off as needed. Your powerful desktop machine may be able to handle the drawing of this kind of map; however, performance will be unacceptably slow if you try to move the map directly to the web. The server requires precious time to iterate through all those layers, retrieve the data, draw it, and send the image back to the client.
To address this problem, most web maps break apart the layers into groups that are handled very differently. Layers whose sole purpose are to provide geographic context are grouped together and brought into the map as a single tiled basemap. In contrast, thematic layers (the layers that are the focus of the map) are brought in as one or more separate web services and placed on top of the basemap. You might additionally decide to include a set of interactive elements such as popups, charts, analysis tools, and so forth.
Let's take a closer look at these three things--basemaps, thematic layers, and interactive elements--to understand how each is created and displayed.
A basemap provides geographic context for your map. In other words, it is usually not the main reason people look at your map, but your map would be difficult to interpret without it. The most common basemaps you've used online are vector road maps and remotely sensed imagery.
Although a basemap may consist of many sublayers (such as roads, lakes, buildings, and so forth), these are often fused together into a rasterized set of tiled images and treated as a single layer in your web map. These tiled maps consist of often thousands or millions of predrawn images that are saved on the server and passed out to web browsers as people pan around the map. Lesson 5 will explain tiled maps in greater depth and give you a chance to make your own.
In the past couple of years, it has become fashionable in some applications for the server to send the basemap as chunks of vector coordinates, sometimes known as "vector tiles." Displaying the basemap as vectors instead of a rasterized map allows for more flexibility in visualization, such as being able to rotate the map while the labels stay right-side-up. You can see a vector basemap in action if you look at the Google Maps app on a smartphone.
Sometimes two tiled layers will work together to form a basemap. For example, you may have a tiled layer with aerial imagery and a second tiled layer with a vector road overlay that has been designed to go on top of the imagery. (In Google Maps, this appears when you check the Labels item). Keeping these two tilesets separate takes less disk space and makes it easier to update the imagery.
Thematic layers (also known as business or operational layers) go on top of the basemap. They're the reason people are coming to visit your map. If placed on the basemap, they might not be of interest to everybody, but when placed on your focused web map, they are the main layer of interest. If your map is titled “Farmers markets in Philadelphia,” then farmers markets are your thematic layer. If your map is titled “Migration patterns of North American birds,” then the migration patterns are your thematic layer.
Like basemaps, thematic layers are sometimes displayed with tiles; however, this may not always be possible due to the rapidly changing nature of some data. For example, if you need to display the real time positions of police vans, you cannot rely on predrawn tiles and must use some other way to draw the data. There are various web services such as WMS (which you will learn about in Lesson 4) that are designed to draw maps on the fly in this way. You can use these for your thematic layers. Another option is to query the server for all the data and use the browser to draw it. This approach lends itself well to interactive elements such as popups and is described in Lesson 6.
Thematic layers work together with basemap layers to form an effective web map. Interestingly, the thematic layer is not always the top one. In a 2009 blog post (no longer online), Esri cartographer Charlie Frye described a “map sandwich” approach, wherein a thematic layer (which could be tiled or not) is placed in between two tiled basemap layers that give geographic context. The bottom layer has physiographic features and the top layer has labels and boundaries. This is the “bread.” The thematic layer in the middle is the “meat.” The Esri "Light Gray Canvas" basemap in ArcMap and ArcGIS Online uses this technique. FOSS technology definitely allows this approach as well.
Your map may include several thematic layers that you allow users to toggle on and off. To accomplish this, you can use a single web service with multiple sublayers, or multiple web services that each contain a single layer. However, to keep your app usable and relatively fast-performing, you should avoid including many thematic layers in your web map.
Try this
How is Bing displaying these layers? Why are they taking this approach? At the end of this class, you will have a good understanding of the different options that exist and their advantages and disadvantages. Feel free to go ahead and check out HERE WeGo [41] in the same way (layer options can be accessed at the bottom right). You should notice a couple of similarities between the two web maps.
Web maps are often equipped with interactive elements that can help you learn more about the layers in the map. These can be informational popups that appear when you click a feature, charts and graphs that are drawn in a separate part of the page, slider bars that adjust the time slice of data displayed in the map, and so forth. Some web maps allow you to edit GIS data in real time, or submit a geoprocessing job to the server and see the response drawn on the screen. Others allow you to type a series of stops and view driving directions between each.
These elements make the map come alive. The key to making an effective web map is to include the interactive elements that are most useful to your audience, without overwhelming them with options or making the tasks too complicated. Even a little bit of housekeeping, such as including user-friendly field aliases in your popups, can go a long way toward making your map approachable and useful.
Interactive elements are the part of your web map that require the most custom programming. The amount of interactivity you have the freedom to add may be strongly correlated with the amount of JavaScript programming that you know how to do. Open web mapping APIs such as OpenLayers and Leaflet provide convenient methods for doing some of the most common things, such as opening a popup.
Later in this lesson, you'll examine some web maps and discuss the interactive elements they provide. You'll also revisit this subject in Lesson 6 as you begin using the web mapping APIs to put all your layers together into a web map.
Let's get some practice identifying these elements. I will walk you through two simple web maps and point out the basemap, the thematic layers, and the interactive elements. Then, in this week's assignment, you'll get the chance to try it yourself using two web maps of your choosing.
The first map we'll look at is www.gdacs.org/ [44], published by the Global Disaster Alert and Coordination System (GDACS), a cooperation framework between the United Nations, the European Commission, and others to share disaster information and alerts. The web map on the main page showing alerts for the last four days is relatively simple in terms of interactive elements, focusing on displaying disaster related information and providing links to more detailed reports. This map has been created with OpenLayers. It is similar in scope to what you will be creating in this course. Take a minute to open this map and examine it.
https://gisco-services.ec.europa.eu/maps/tiles/NaturalEarth/EPSG3857/2/0/3.png
https://www.gdacs.org/arcgis/rest/services/GDACS/daily_rain/MapServer/export?F=image&FORMAT=PNG32&TRANSPARENT=true&SIZE=256%2C256&BBOX=-180%2C-90%2C-90%2C0&BBOXSR=4326&IMAGESR=4326&DPI=90In contrast, to the previous URL for the basemap tiles, which used nested folders for zoom level, x and y coordinates of the tiles, we here find bounding box coordinates and other information encoded as arguments following the '?' which are then decoded by the server and used to produce the requested tile image. How about the other thematic layers? It turns out these do not come as images but fortunately, they can still be identified relatively easily in the network log because the URLs for the requests contain abbreviations for the event types, like this:
https://www.gdacs.org/gdacsapi/api/events/geteventlist/MAP?eventtypes=EQThese requests are sent out immediately when the map is loaded, and the response from the GDACS server always is a GeoJSON file with a text specification of the vector features and their geometries. These vector features are then drawn by the web browser. You will learn more about client-side rendering of data from GeoJSON files in Lesson 7.
The second map is a web map at https://maps.nyc.gov/crime/ [45] showing the concentration of crimes of different types for New York City. The map is entirely based on services provided by ESRI. The map is created in Javascript using ESRI's ArcGIS Javascript API and some other Javascript libraries for the general layout of the page such as Dojo..
https://tiles.arcgis.com/tiles/yG5s3afENB5iO9fj/arcgis/rest/services/NYC_Basemap/VectorTileServer/tile/14/6161/4830.pbfSimilar to the previous web map example, the tiles are stored on the server in some folder/file organization with numbers referring to the zoom level (.../14/...), x grid coordinate (.../6161/...), and y grid coordiante (.../4830.pbf). However, one important difference is that these are not image tiles. They are vector tiles using the .pbf vector tile format in which the content of the tile is specified as vector data that then still needs to be rendered by the web map application running in the browser when the .pbf file is received. Therefore, we don't get to see an image in the Preview of the browser's Developer Tools but rather some cryptic text because the .pbf file stores the vector data in binary format. Looking at the map itself, we can see that these basemap vector tiles are rendred using a light non-intrusive design.
https://services6.arcgis.com/yG5s3afENB5iO9fj/arcgis/rest/services/NYC_Crime_Locations_PROD_view/FeatureServer/0/query?f=pbf&cacheHint=true&maxRecordCountFactor=4&resultOffset=8000&resultRecordCount=8000&where=1%3D1&orderByFields=OBJECTID&outFields=OBJECTID&outSR=102100&spatialRel=esriSpatialRelIntersectsWe see that the data is coming from arcgis.com as well. We don't have to worry about the details of these requests for now but note how the details of what exactly is requested are encoded in the URL using different parameters separated by & symbols.
Here are a few other maps you can look at if you want to keep practicing. Although some of these maps have very nice elements, I am recommending them for their variety, not for their excellence in any particular realm.
From this brief exercise, you can begin to see the various approaches that can be used to put together a web map. Hopefully, you can look at a web map and immediately begin to see how the layers were broken up into basemap layers and thematic layers. Whenever you see a web map, you should also think about the interactive elements that are present and decide whether they are usable and applicable.
GeoServer is free and open source software (FOSS) that exposes your data as geospatial web services. You'll be using GeoServer later in this course and possibly in your term project if you choose. This week, we'll take some time to get GeoServer installed and configured. This is a detour from our discussion of web map elements, but you'll soon revisit that topic in the weekly assignment.
Again, the lesson materials provide instructions for Windows. You are welcome to install on another platform, but you are on your own for instructions and troubleshooting.
set GEOSERVER_HOME=C:\Users\jow\GeoServer 2.19.6 set GEOSERVER_DATA_DIR=%GEOSERVER_HOME%\data_dirNow do the same for the file called "shutdown.bat".
You'll return to GeoServer later in this course. This walkthrough just ensures you have it up and running correctly. If technical difficulties prevented you from getting to this point, please alert your instructor.
For this week's assignment, find two web maps "in the wild" and create a post on the "Lesson 2 assignment forum" on Canvas that describes the following for each map:
Please make an effort to find these maps on your own. In other words, do not choose any of the web maps previously referenced in this lesson, nor should you browse through other students' posts in order to find maps to analyze. Except for the first point (URL), please use short paragraphs to address the different points in your description similar to what you saw in this lesson, not just bullet point enumerations.
If you are having difficulties analyzing your web map, please refer back to the section "Elements of a web map" and the examples and techniques using your browser's developer tool described there. If you check out the requests in the Network log, requests for images drawn by a server will be relatively easy to identify because the individual images sent back to the browser as a response will be image files in one of the typical image formats (.png, .jpg, etc.). You will be able to see images under 'Preview' of the Network tab. Looking at the preview images will allow you to identify which layers are included, e.g. is it just the basemap, just one thematic layer, or maybe basemap and thematic layer combined. Then the last question is whether these images are tiles, so are multiple images requested and then put together to cover the entire area of the map you are seeing, or is it always just a single image covering the entire map area (something that, for instance, you can experience with a WMS service where the browser tells the server the extent of the area covered by the map, the server then renders an image for that area and sends it back to the browser).
With requests for vector data that that is then rendered by the browser, things are more difficult. The vector data sent by the browser can come in a variety of formats (.json and .pbf are common ones) and is often in a binary format or in some other way compressed, in which case there is no direct way to check out what exactly the files contain. So if you come to the conclusion that some of the data for your map is not coming in as images, that's goood enough for this assignment. If you are able to identify some candidate requests in the network, you should mention that and the server these are going to, but in some cases even that can be difficult, e.g. when the vector data is just stored in a local file on the same web server that is hosting the page with the map.
This course contains many walkthroughs showing how to use FOSS, but the ultimate objective is to apply these skills toward your own web map using data that's interesting to you (or your supervisor at work!). As the course progresses, you'll be expected to complete a portion of the weekly assignments using your own datasets. By the end of the course, this will become a fully functioning "term project" web map that combines base layers, thematic layers, and some degree of interactivity through popups, filters, or queries.
Part of the final week of the course is reserved for putting the finishing touches on your term project; however, most of the major pieces will be in place by that time if you put a respectable effort into the weekly walkthroughs and assignments.
Please do the following to get started with the term project:
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
Underneath any web map are spatial datasets representing the entities to be placed on the map and their various attributes. In this lesson, you will learn about FOSS options for storing and processing spatial data. The broad scope of this course prohibits a full discussion of database theory and design; however, you will hopefully learn enough to select the appropriate type of data format for your project. Once you get your data in order, you'll be ready to launch GIS web services and assemble them into a web map.
This section of the lesson describes in greater detail some of the spatial data formats that have open specifications or are created by open source software. Note that these refer to files or databases that can stand alone on your hardware. We will cover open formats of web services streamed in from other computers in future lessons.
File-based data includes shapefiles, KML files, GeoJSON, and many other types of text-based files. Each of the vector formats has some mechanism of storing the geometry (i.e., vertex coordinates) and attributes of each feature. Some of the formats, such as KML may also store styling information.
Below are some of the file-based data formats you're most likely to encounter.
The Esri shapefile is one of the most common formats for exchanging vector data. It actually consists of several files with the same root name, but with different suffixes. At a minimum, you must include the .shp, .shx, and .dbf files. Other files may be included in addition to these three when extra spatial index or projection information is included with the file. This ArcGIS Resources article [57] gives a quick overview of the different files that can be included.
Because a shapefile requires multiple files, it is often expected that you will zip them all together in a single file when downloading, uploading, and emailing them.
If you want to make a shapefile from scratch, you can refer to the specification from Esri [58]. This is not for the novice programmer, and browsing this spec will hopefully increase your appreciation for those who donate their time and skills to coding FOSS GIS programs.
The GeoPackage [59] is a relatively new format for storing and transferring vector features, tables, and rasterized tiles across a variety of devices, including laptops, mobile devices, and so forth. It was defined by the Open Geospatial Consortium (OGC), a group you will learn about in more detail in Lesson 4 that consists of industry representatives, academics, practitioners, and others with an interest in open geospatial data formats. The GeoPackage actually stores the data in a SQLite database, described below in more detail in the databases section. I list the GeoPackage here in the file-based formats section because some have advocated [60] for its adoption as a more modern alternative to the shapefile.
KML gained widespread use as the simple spatial data format used to place geographic data on top of Google Earth. It is also supported in Google Maps and various non-Google products.
KML stands for Keyhole Markup Language, and was developed by Keyhole, Inc., before the company's acquisition by Google. KML became an Open Geospatial Consortium (OGC) standard data format in 2008, having been voluntarily submitted by Google.
KML is a form of XML, wherein data is maintained in a series of structured tags. At the time of this writing, the Wikipedia article for KML [61] contains a simple example of this XML structure. KML is unique and versatile in that it can contain styling information, and it can hold either vector or raster formats ("overlays", in KML-speak). The rasters themselves are not written in the KML, but are included with it in a zipped file called a KMZ. Large vector datasets are also commonly compressed into KMZs.
JavaScript Object Notation (JSON) is a structured means of arranging data in a hierarchical series of key-value pairs that a program can read. (It's not required for the program to be written in JavaScript.) JSON is less verbose than XML and ultimately results in less of a "payload," or data size, being transferred across the wire in web applications.
Following this pattern, GeoJSON is a form of JSON developed for representing vector features. The GeoJSON spec [62] gives some basic examples of how different entities such as point, lines, and polygons are structured.
You might choose to save GeoJSON features into a .js (JavaScript) file that can be referenced by your web map. Other times, you may encounter web services that return GeoJSON.
A variation on GeoJSON is TopoJSON [63], which stores each line segment as a single arc that can be referenced multiple times by different polygons. In other words, when two features share a border, the vertices are only stored once. This results in a more compact file, which can pay performance dividends when the data needs to be transferred from server to client.
Many GIS programs can read vector data out of other types of text files, such as .gpx (popular format for GPS tracks) and various types of .csv (comma-separated value files often used with Microsoft Excel) that include longitude (X) and latitude (Y) columns. You can engineer your web map to parse and read these files, or you may want to use your scripting skills to get the data into another standard format before deploying it in your web map. This is where Python skills and helper libraries can be handy.
Most raster formats are openly documented and do not require royalties or attribution. These include JPEG, PNG, TIFF, BMP, and others. The GIF format previously used a patented compression format, but those patents have expired.
Web service maps such as WMS return their results in raster formats, as do many tiled maps. A KML/KMZ file can also reference a series of rasters called overlays.
When your datasets get large or complex, it makes sense to move them into a database. This often makes it easier to run advanced queries, set up relationships between datasets, and manage edits to the data. It can also improve performance, boost security, and introduce tools for performing spatial operations.
Below are described several popular approaches for putting spatial data into FOSS databases. Examples of proprietary equivalents include Microsoft SQL Server, Oracle Spatial, and the Esri ArcSDE middleware (packaged as an option with ArcGIS Enterprise) that can connect to various flavors of databases, including FOSS ones.
PostGIS is an extension that allows spatial data management and processing within PostgreSQL (often pronounced "Postgress" or "Postgress SQL"). PostgreSQL is perhaps the most fully featured FOSS relational database management system (RDBMS). If a traditional RDBMS with relational tables is your bread and butter, then PostgreSQL and PostGIS are a natural fit if you are moving to FOSS. The installation is relatively straightforward: in the latest PostgreSQL setup programs for Windows, you just check a box after installation indicating that you want to add PostGIS. An importer wizard allows you to load your shapefiles into PostGIS to get started. The rest of the administration can be done from the pgAdmin GUI program that is used to administer PostgreSQL.
Most FOSS GIS programs give you an interface for connecting to your PostGIS data. For example, in QGIS you might have noticed the button Add PostGIS Layers . The elephant in the icon is a symbol related to PostgreSQL. GeoServer also supports layers from PostGIS.
This course, Geog 585, does not provide walkthroughs for PostGIS; however, there are a couple of excellent open courseware lessons in Geog 868: Spatial Databases [64] that describe how to install and work with PostGIS. I encourage you to make time to study these on your own (or take the instructor-led offering) if you feel that learning PostGIS will be helpful in your career.
You are welcome to use PostGIS in your term project if you feel comfortable with the other course material and want to take on an additional challenge. You can always fall back on file-based data if an excessive amount of troubleshooting is required.
SpatiaLite is an extension supporting spatial data in the SQLite database. As its name indicates, SQLite is a lightweight database engine that gives you a way to store and use data in a database paradigm without installing any RDBMS software on the client machine. This makes SQLite databases easy to copy around and allows them to run on many kinds of devices. If you are familiar with Esri products, a SpatiaLite database might be thought of as similar to a file geodatabase.
SpatiaLite is not as mature as PostGIS, but it is growing in popularity, and you will see a button in QGIS called Add SpatiaLite Layer . If you feel it would be helpful in your career, you are welcome to use SpatiaLite in your term project. If you choose to do this, I ask you to first get the project working with file-based data. Then feel free to experiment with swapping out the data source to SpatiaLite.
You will encounter a SpatiaLite database in the Lesson 9 walkthrough when you use QGIS to import data from OpenStreetMap. In that scenario, you are dealing with a large amount of data with potentially many fields of complex attributes. SpatiaLite is a more self-contained and flexible choice than shapefiles, KML, etc., for this type of task.
In the previous lesson, you learned that system architectures for web mapping include a data tier. This could be as simple as several shapefiles sitting in a folder on your server machine, or it could be as complex as several enterprise-grade servers housing an ecosystem of standalone files and relational databases. Indeed, in our system architecture diagram, I have represented the data tier as containing a file server and a database server.
The data tier contains your datasets that will be included in the web map. Almost certainly, it will house the data for your thematic web map layers. It may also hold the data for your basemap layers, if you decide to create your own basemap and tile sets. Other times, you will pull in basemaps, and quite possibly some thematic layers, from other peoples' servers, relieving yourself of the burden of maintaining the data.
Some organizations are uneasy with the idea of taking the same database that they use for day-to-day editing and putting it on the web. There is justification for this uneasiness, for both security and performance reasons. If you are allowing web map users to modify items in the database, you want to avoid the possibility of your data being deleted, corrupted, or sabotaged. Also, you don't want web users' activities to tax the database so intensely that performance becomes slow for your own internal GIS users, and vice versa.
For these reasons, organizations will often create a copy or replica of their database and designate it solely for web use. If features in the web map are not designed to be edited by end users, this copy of the database is read-only. If, on the other hand, web users will be making edits to the database, it is periodically synchronized with the internal "production" database using automated scripts or web services. A quality assurance (QA) step can be inserted before synchronization if you would prefer for a GIS analyst to examine the web edits before committing them to the production database.
You can generally increase web map performance by minimizing the number of "hops" between machines that your data has to take before it reaches the end user. If your data is file-based or is stored in a very simple database, you may just be able to store a copy of it directly on the machine that hosts your geospatial web services, thereby eliminating network traffic between the geospatial server and a data server. However, if you have a large amount of data, or a database with a large number of users, it may be best to keep the database on its own machine. Isolating the database onto its own hardware allows you to use more focused backup and security processes, as well as redundant storage mechanisms to mitigate data loss and corruption. It also helps prevent the database and the server competing for resources when either of these components is being accessed by many concurrent users.
If you choose to house your data on a machine separate from the server, you need to ensure that firewalls allow communication between the machines on all necessary ports. This may involve consulting your IT staff (bake them cookies, if necessary). You may also need to ensure that the system process running your web service is permitted to read the data from the other machine. Finally, you cannot use local paths such as C:\data\Japan to refer to the dataset; you must use the network name of the machine in a shared path; for example, \\dataserver\data\Japan.
When designing your data tier, you will need to decide whether to store your data in a series of simple files (such as shapefiles or KML) or in a database that incorporates spatial data support (such as PostGIS or SpatiaLite). A file-based data approach is simpler and easier to set up than a database if your datasets are not changing on a frequent basis and are of manageable size. File-based datasets can also be easier to transfer and share between users and machines.
Databases are more appropriate when you have a lot of data to store, the data is being edited frequently by different parties, you need to allow different tiers of security privileges, or you are maintaining relational tables to link datasets. Databases can also offer powerful native options for running SQL queries and calculating spatial relationships (such as intersections).
If you have a long-running GIS project housed in a database, and you just now decided to expose it on the web, you'll need to decide whether to keep the data in the database or extract copies of the data into file-based datasets.
To review a key point from the previous section, in this course, we will be using open data formats, in other words, formats that are openly documented and have no legal restrictions or royalty requirements on their creation and use by any software package. You are likely familiar with many of these, such as shapefiles, KML files, JPEGs, and so forth. In contrast, proprietary data formats are created by a particular software vendor and are either undocumented or cannot legally be created from scratch or extended by any other developer. The Esri file geodatabase is an example of a well-known proprietary format. Although Esri has released an API for creating file geodatabases, the underlying format cannot be extended or reverse engineered.
Some of the most widely-used open data formats were actually designed by proprietary software vendors, who made the deliberate decision to release them as open formats. Two examples are the Esri shapefile and the Adobe PDF. Although opening a data format introduces the risk that FOSS alternatives will compete with the vendor's functionality, it increases the interoperability of the vendor's software, and, if uptake is widespread, augments the vendor's clout and credibility within the software community.
Imagine you've identified some spatial data to use in your web map, but the data doesn't quite fit your purposes yet. It covers a broader region than your study area, and you want it to be in a different projection. Maybe you have a raster DEM that you need to convert into a hillshade, or perhaps you want to interpolate some raster surfaces that you can use in a time series animation. Indeed, a large portion of your datasets will probably need some kind of preprocessing before you incorporate them into your web map.
In these situations, you need to:
If you're accustomed to a proprietary GIS software package that contains hundreds of tools and uniform documentation out of the box, it may seem frustrating to move to FOSS. Cobbling together a range of tools and collecting bits or pieces of documentation may seem like a waste of precious time. This is the tradeoff that you make when you use free software. Fortunately, the number of operations to learn is finite, and most of the time you'll probably be doing one of a dozen or fewer common actions, such as selecting data, projecting, clipping, and buffering. After you've learned how to do these once, you can go back and repeat the steps with any dataset, especially if you have taken good notes. Also, scripting these actions or running them in batch may require less overhead and processing time than you experience with proprietary software.
This is not a course about spatial data processing; however, this particular lesson attempts to give you some experience doing data processing with FOSS. You will learn a few resources for addressing data processing, and you'll get a feel for how to approach new tools.
In this course, you'll use QGIS and its associated plugins as a GUI tool for data processing. You will also learn how to use the GDAL and OGR command line utilities. These are explained in more detail below.
QGIS offers a lot of the most common vector and raster processing tools out of the box. Additionally, developers in the QGIS user community have contributed plugins that can extend QGIS functionality.
Open QGIS and explore the Vector menu to see some of the operations for processing vector data. You'll notice tools for merging, summarizing, intersecting, buffering, clipping, and more.
Some of the most powerful options are hidden in the Export context menu item when you right-click a vector layer. The Save Features As... dialog box allows you to convert data between different formats (for example, convert a shapefile to GeoJSON) and reproject the data to a new coordinate system.
Now, go back up to the top of the QGIS screen and click the Raster menu to see some of the options for processing rasters. Notice that you can warp, clip, contour, and interpolate to raster formats, along with various other operations.
In addition to these common vector and raster processing options, the Processing > Toolbox menu at the top of the QGIS screen gives you a dockable side window with access to many additional functions, some of them more obscure than others. This toolbox is akin to the full "ArcToolbox" that you see in Esri software. One difference is that it contains tools from multiple software packages, including GDAL, GRASS, and SAGA (a popular open source library for processing rasters). One of the more common "algorithms" (as they are called in QGIS) that I use in this toolbox is Create Grid, which can create square or hexagon lattices for cartographic "binning" or, in other words, aggregation to uniformly shaped regions in order to get a better visualization of a point pattern. Creating a lattice of hexagons is a task not easy to do by hand, and is well-suited to a pre-canned tool.
If you don't see what you're looking for in any of the above locations, someone may have developed a plugin for it. QGIS comes with a few of the more useful plugins already installed. Click Plugins > Manage And Install Plugins to see what they are. Here, you can disable some of these plugins if you don't like them getting in the way.
If you want to add more plugins, you can do it directly from this dialog box by clicking Not installed. Do this now, and examine the descriptions of some of the plugins you can add (the list may vary from what you see below).
You can even pick one and install it if you like. The OpenLayers plugin shown above is a handy way to see OpenStreetMap, Google Maps, etc. in QGIS. This plugin is still declared "experimental" in QGIS 3, so most likely you will only be able to see the plugin if you first click on the Settings tab and select the option to show experimental plugins. The plugin will add a new entry in the Web menu, from which you can pick different basemap options. Be aware that the quality and usability of the plugins may vary.
Many spatial data processing functions use well-known logic or documented algorithms. It would cause extra tedious work and possibly introduce errors and inconsistency if every FOSS developer had to code these same operations from scratch. Therefore, many FOSS programs take advantage of a single open source code library called GDAL (Geospatial Data Abstraction Library) to perform the most common functions.
GDAL is most commonly thought of as a raster processing library. But within GDAL is an important repository of vector processing functions called the OGR Simple Features Library. You will hear the terms GDAL and OGR many times as you work with FOSS, so get used to them. You can thank a man named Frank Warmerdam for initiating and maintaining these libraries over time. Although you may have never heard of him, you've probably done something to run his code at one point or another.
One way to use GDAL and OGR is by launching functions from QGIS or some other GUI-based program, like you would get from the menu options pictured above. Another way is to write code in Python, C#, or some other language that calls directly into these libraries. A third way, which lies in the middle in terms of complexity and flexibility, is to call into GDAL and OGR using command line utilities. These utilities were installed for you when you installed QGIS. You will get a feel for them in the lesson walkthrough.
There are many other FOSS tools out there for wrangling spatial data; new ones appear all the time. Later in this course you'll be required to find one, test it out, review it, and share it with the class. Be an explorer, and if you find something that works for you, stick with it. You can certainly use any FOSS tool that works for you in order to complete the term project.
This walkthrough will first give you some experience using the GUI environment of QGIS to clip and project some vector data. Then, you'll learn how to do the same thing using the OGR command line utilities. The advantage of the command line utility is that you can easily run it in a loop to process an entire folder of data.
This project introduces some data for Philadelphia, Pennsylvania that we're going to use throughout the next few lessons. Most of these are simple basemap layers that I downloaded and extracted from OpenStreetMap; however, the city boundary is an official file from the City of Philadelphia that I downloaded from PASDA [66].
Download the Lesson 3 vector data [67]
That was easy enough, but it would be tedious, time-consuming, and possibly error prone if you had to do it for more than a few datasets at a time. Let's see how you could use the OGR command line utilities to do this in an automated fashion. Remember that OGR is the subset of the GDAL library that is concerned with vector data.
When you install QGIS, you also get some executable programs that can run GDAL and OGR functions from the command line. The easiest way to get started with these is to use the OSGeo4W shortcut that appeared on your desktop after you installed QGIS.
ogr2ogr -skipfailures -clipsrc c:\data\PhiladelphiaBaseLayers\clipFeature\city_limits.shp c:\data\PhiladelphiaBaseLayers\clipped\roads.shp c:\data\PhiladelphiaBaseLayers\roads.shp
ogr2ogr -t_srs EPSG:3857 -s_srs EPSG:4326 c:\data\PhiladelphiaBaseLayers\clippedAndProjected\roads.shp c:\data\PhiladelphiaBaseLayers\clipped\roads.shpThis one should run more quickly than the clip.
for %X in (*.shp) do ogr2ogr -skipfailures -clipsrc c:\data\PhiladelphiaBaseLayers\clipFeature\city_limits.shp c:\data\PhiladelphiaBaseLayers\clipped\%X c:\data\PhiladelphiaBaseLayers\%XYou can see the console messages cycling through all the datasets in the folder. Ignore any topology errors that appear in the console. This is somewhat messy data, and we have selected to skip failures.
cd c:\data\PhiladelphiaBaseLayers\clipped
for %X in (*.shp) do ogr2ogr -t_srs EPSG:3857 -s_srs EPSG:4326 c:\data\PhiladelphiaBaseLayers\clippedAndProjected\%X c:\data\PhiladelphiaBaseLayers\clipped\%XYou can add everything to QGIS to verify.
If you know that you'll be doing the same series of commands in the future, you can place the commands in a batch file. This is just a basic text file containing a list of commands. On Windows, you just save it with the extension .bat, and then the operating system understands that it should invoke the commands sequentially when you execute the file.
Try the following to see how you could use ogr2ogr in a batch file.
cd /d c:\data\PhiladelphiaBaseLayers set ogr2ogrPath="c:\program files\QGIS 3.16.10\bin\ogr2ogr.exe" set GDAL_DATA=C:\program files\QGIS 3.16.10\share\gdal for %%X in (*.shp) do %ogr2ogrPath% -skipfailures -clipsrc c:\data\PhiladelphiaBaseLayers\clipFeature\city_limits.shp c:\data\PhiladelphiaBaseLayers\clipped\%%X c:\data\PhiladelphiaBaseLayers\%%X for %%X in (*.shp) do %ogr2ogrPath% -skipfailures -s_srs EPSG:4326 -t_srs EPSG:3857 c:\data\PhiladelphiaBaseLayers\clippedAndProjected\%%X c:\data\PhiladelphiaBaseLayers\clipped\%%X
Notice that these are just the same commands you were running before, with the addition of a few lines at the beginning to change the working directory and set the path of the ogr2ogr utility.
Batch files can use variables, just like you use in other programming languages. You set a variable using the set keyword, then refer to the variable using % signs on either side of its name (for example, %ogr2ogrPath%). Variables created inline with loops are represented in a batch file using %% (for example, %%X), a slight difference from the syntax you use when typing the commands in the command line window.
You will use this data in future lessons. Therefore, do the following to preserve it in an easy-to-use fashion:
Now that you've seen QGIS and OGR in action with vector data, you'll get some experience processing raster data. For this exercise, we're going to start with a 30-meter resolution digital elevation model (DEM) for Philadelphia. I obtained this from the USGS National Map Viewer [69]. We'll use a combination of GDAL tools (some of them wrapped in a nice QGIS GUI) to make a nice-looking terrain background for a basemap. This will be accomplished by adding a DEM, a hillshade, and a shaded slope layer together. I have based these instructions on this tutorial by Mapbox [70] that I encourage you to read later if you would like further detail.
Download the Lesson 3 raster data [71]
Extract the data to a folder named PhiladelphiaElevation, such as c:\data\PhiladelphiaElevation. This will contain a single dataset called dem. In the interest of saving time and minimizing the download size, I have already clipped this dataset to the Philadelphia city boundary and projected it to EPSG:3857 for you. If you need to do this kind of thing in the future, you can use the Raster > Projections > Warp command in QGIS, which invokes the gdalwarp command.
0 255 255 255 90 0 0 0This creates a very simple color ramp that will shade your slope layer in grayscale with values toward 0 being lighter and values toward 90 being darker. When combined with the hillshade, this layer will cause shelfs and cliffs to pop out in your map.
gdaldem color-relief slope.tif sloperamp.txt slopeshade.tifYou just ran the gdaldem [72] utility, which does all kinds of things with elevation rasters. In particular, the color-relief command colorizes a raster using the following three parameters (in order): The input raster name, a text file defining the color ramp, and the output file name.
1 46 154 88 100 251 255 128 1000 224 108 31 2000 200 55 55 3000 215 244 244Note that the first value in the line is the elevation value of the raster, and the next three values constitute an RGB color definition for cells of that elevation. This particular ramp contains elevations well beyond those seen in Philadelphia, just so you can get an idea of how these ramps are created. I have adjusted the ramp so that lowlands are green and the hilliest areas of Philadelphia are yellow. If we had high mountains in our DEM, brown and other colors would begin to appear.
gdaldem color-relief dem.tif demramp.txt demcolor.tifWhen you add demcolor.tif to QGIS, you should see something like this:
Just like in the previous walkthrough, you will copy your final datasets into your main Philadelphia data folder for future use.
Use Windows Explorer or an equivalent program to copy hillshade.tif, slopeshade.tif, and demcolorclipped.tif into the Philadelphia folder (such as c:\data\Philadelphia) that you created in the previous walkthrough.
In this week's assignment, you'll get a chance to use some of your newfound QGIS and GDAL skills to prepare your term project data. This assignment has two distinct parts:
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
Up to this point, you have largely worked with data on your local machine using QGIS. In this lesson, you'll take a step into the web world by making some of your map layers available on GeoServer as web services.
This lesson focuses on web services that use the Open Geospatial Consortium (OGC) Web Map Service (WMS) specification. GIS professionals have been using WMS to draw FOSS web maps for years. WMS doesn't use the latest and slickest technology, but it's a functional and widespread specification that's important for you to understand and feel comfortable with.
As part of this lesson, you'll practice serving some of your term project data as a WMS; however, you are not required to use the layer in your term project if you decide there is a more appropriate format. As you'll see in the next few lessons, there are various other ways to draw layers in a web map.
There are various ways to get a map to show up in someone's web browser. One way is for the server to send predrawn map images (or image tiles) to the browser, another way is for the server to send a bunch of text-based geographic coordinates and attributes that the browser draws, and a third way is to draw the map on the server at the time of request and send the browser the completed map image. This third way is what we'll discuss in this lesson. It's sometimes called a dynamic map service because, being drawn on the fly, it reflects the most recent picture of the data. Contrast this with the image tile approach, which represents a static picture of the data taken at one point in time (when all the tiles were generated).
Because dynamically drawn map services retrieve the data and draw it at the time of request, they are useful for gaining a situational awareness of the most recent state of the data. They may be the only reasonable way to depict many features that are changing at the same time (such as the positions of each vehicle in a large fleet). Dynamically drawn maps may also be the best solution at large scales where tiled maps become too cumbersome to generate, store, or maintain.
Maps dynamically drawn through WMS allow you to apply a wide range of symbols and styles using a concept called Styled Layer Descriptors (SLDs), which is described later in this lesson. If you enjoy using the QGIS authoring environment for your maps, you can export an SLD and import it into GeoServer, thereby allowing web users to apply the same styling that you configured on the desktop. Map layers drawn by the web browser may not be able to use symbols as complex as ones drawn by the server.
Waiting for the server to draw the map can be a slow, painful experience, especially if there are many layers to render. A wait time of 2 - 3 seconds that may be considered acceptable on the desktop may not be tolerated by edgy web map users who are neither knowledgeable nor sympathetic to the type of back end technology being used. People now expect every map to perform as fast as Google Maps and, without using tiles, this can be difficult to achieve.
Dynamically drawn services are also much more susceptible to overload than tiled services if you have many users deciding they want to see the map at the same time. This presents a paradox: you want your map to be useful, but the more exposure it gets, the slower it will run if the server technology is not scalable.
If you know you have a limited audience for your application (such as a small municipality, or a team of scientists), you may safely decide that the performance of your dynamically drawn map service is "good enough." This can spare you the work of generating and maintaining a tiled service.
In this course, you'll use GeoServer for drawing dynamic map services through WMS. Other FOSS software that can do this includes:
Proprietary GIS software such as Esri ArcGIS Server also supports WMS and other OGC specifications. Although Esri has their own format for map services, many of their clients have interoperability requirements mandating that their web services be available through WMS. Hence, when you publish a map service in ArcGIS Server, you see a checkbox that can allow the service to speak through WMS specifications.
In the previous lesson, you learned about some open formats for data stored on your machine as files and databases. Now, let's examine some of the open specifications for web services that draw maps.
Recall from Lesson 1 that a web service receives a user request and sends back a response. In order for web services to work across platforms, the syntax for the requests and responses needs to be consistent and openly described in what is called a "specification" document. When software developers create a server or a client that supports the web service, they treat the specification document as their Bible; it is a contract that must be closely honored in order for web services to correctly function.
To get a feel for what a specification looks like, take a minute to browse the Open Geospatial Consortium (OGC) WMS specification [79] (especially Section 7). Don't worry too much about what this all means. We'll dive into some of that later in the lesson. Just notice that there are clearly defined methods and parameters that you and the server must use if you want to communicate with the service.
Because specifications must describe every method and parameter of the web service, they can appear to be long and complicated documents. Fortunately, you rarely have to use the specification directly because you will work with relatively user-friendly servers and clients are designed to abstract away the complexity of working with web services. For example, later in this lesson, you'll use GeoServer to create a WMS and QGIS to display the WMS on a map. You don't have to refer to the WMS specification in order to do this because these programs take care of formulating the request and working with the response. You can be certain, however, that the developers of GeoServer and QGIS had to examine the WMS specification in great detail when writing their source code.
The OGC is an industry consortium that develops open specifications for spatial data and web services. The OGC is comprised of representatives from private companies, government organizations, NGOs, and universities.
OGC has produced a series of specifications for GIS web services named in the format Web _____ Service (sometimes abbreviated as W*S). For example:
Virtually all FOSS GIS servers and some proprietary ones have engineered their web services to communicate through OGC specifications. Additionally, many client APIs for developing web maps, such as OpenLayers and Leaflet, support WMS as web map layers. Desktop GIS programs like QGIS support the use of WMS as layers. In fact, the buttons allow you to add (from left to right) WMS, WCS, and WFS web services as layers in QGIS.
In 2010, the proprietary GIS software company Esri released an open document called the GeoServices REST Specification (now the GeoServices API [80]) describing the request and response patterns used by default for its ArcGIS Server web services. (Although I mentioned that you can enable OGC communication on an ArcGIS Server service, this option is not the default.)
The GeoServices REST Specification uses a RESTful pattern of communication with the web services. Whereas the OGC services offer a simple list of methods that typically return XML (when they're not returning an image), REST exposes information in a navigable hierarchy of URLs that can return focused packets of HTML, JSON, images, or other data types. For example, using REST and the GeoServices REST specification, you can invoke http://<server>/MapServer/layers/ to get a list of layers in a map service and
Although ArcGIS Server had already been using this pattern of communication for several years, Esri eventually placed the specification online as an open document so that developers would feel encouraged (or at least legally allowed) to implement clients and servers supporting this pattern.
Some controversy around the specification erupted in 2013 when a slightly modified version of the document (with the Esri references scrubbed out) was submitted to a vote for adoption as an OGC specification. Proponents, including Esri, argued that the OGC lacked a nimble, RESTful alternative to the W*S specifications (this lesson previously linked to a 2013 Esri User Conference Q & A item elaborating this argument, but this document has since been taken offline). Opponents expressed concern [81] about functional overlap with existing service types, and claimed that OGC adoption of the specification would give Esri an unfair advantage in competing for contracts. The uproar was significant enough to get the specification pulled from the voting process.
Although the GeoServices REST Specification is not an OGC specification, I am mentioning it in this lesson as an example of a specification that was voluntarily made open by a proprietary software vendor for the purposes of encouraging interoperability and the proliferation of the vendor's platform. It is likely that you will encounter this specification at some point in your career if you do any development with web services that somebody has hosted on ArcGIS Server.
The Open Geospatial Consortium (OGC) Web Map Service (WMS) specification provides a pattern for serving maps through web services. The WMS receives a request detailing the bounding box (or extent) of the map, the layers to include, the projection to use, and other parameters. It returns a rasterized map image in a common format such as JPG, GIF, or PNG. It does not return any actual GIS data, although an optional part of the specification describes how a user can query any pixel of the service to retrieve attribute information.
WMS has been around since the year 2000 and was supported by some of the earliest FOSS GIS servers. It is still widely supported today in FOSS and proprietary GIS software. Although it has been sometimes criticized as clunky, WMS has enabled a greater degree of interoperability in the world of online mapping.
Consider WMS as an option if you do need a high level of interoperability, and you don't require lightning fast performance or the ability to handle dozens of concurrent users. If you need greater performance and scalability, you should consider tiled maps, including the OGC-endorsed WMTS (Web Map Tile Service) specification. Tiled map services are described in the next lesson.
The best way to get familiar with WMSs is to look at a real one. Therefore, make a close examination of the following URL and then open the link in a web browser:
This request uses the WMS's GetMap operation to retrieve a map with a land cover classification based on imagery and LiDAR data for the State College area. If you are familiar with the campus, you may recognize some buildings and roads in the image. This WMS is provided by the PASDA web portal [83].
Although a WMS can perform several types of operations, GetMap is the most common. It's the one that sends you back a map image.
The root piece of a WMS URL (https://imagery.pasda.psu.edu/arcgis/services/pasda/UrbanTreeCanopy_Landcover/MapServer/WmsServer? [84]) can be formed in various ways (often it will end in 'wms?' rather than 'WmsServer?'), but the parameters following the ? must be formatted according to the WMS specification. Let's examine the parameters used in the above URL.
Parameter | Example value | Notes |
---|---|---|
SERVICE | WMS | Indicates that a WMS is being called. |
VERSION | 1.1.1 | The version of the WMS specification that this WMS can be expected to comply with |
REQUEST | GetMap | This is the operation being requested from the WMS. The GetMap method is the most common operation called on a WMS and is the one that actually returns a map image. The GetCapabilities method is another common operation that returns you some XML metadata describing what the WMS can do. |
LAYERS | 10 | A comma-delimited list of layers that the WMS should include when it draws the map. Some WMSs (like this one) are treated as collections of separate layers that are not intended to be drawn together. Other WMSs have the layers designed to be overlayed or drawn in groups. In this WMS, the layers have numbers as names but these can also be real names like "landcover_2006_statecollege". |
STYLES | In this parameter, you can define your own styles to apply to the WMS when it is drawn. We did not submit this parameter, therefore we get the default styling defined on the server. | |
SRS | EPSG:4326 | The coordinate system that the WMS will use when it draws the map. Here it is specified using an EPSG code (which you saw in previous lessons). Clients can request the WMS to be drawn in WGS 84 (EPSG:4326) or any coordinate system that the WMS publisher has explicitly enabled on the WMS. If you're not sure what coordinate systems the WMS publisher has enabled, then you could use the GetCapabilities method to find out before requesting a map image. |
BBOX | -77.87304,40.78975,-77.85828, 40.80228 | The rectangular extent of the map to be returned by the WMS. This is given by specifying the coordinates of the lower left corner and the upper right corner in a comma delimited fashion. |
FORMAT | image/png | The image format that should be returned by the WMS. In this case, the server will return a PNG. Your choice can affect the performance of your service. For remotely sensed imagery, a format like image/jpeg might result in less data being transferred between the server and the client (and therefore a faster draw time). |
WIDTH | 1200 | The width, in pixels, of the image to be returned. |
HEIGHT | 900 | The height, in pixels, of the image to be returned. |
When you request a map from a WMS, there are some required parameters that you must supply and some optional parameters that you may decide to supply if the WMS publisher has implemented them. All the parameters in the above table are required. Optional parameters can display things like time and elevation dimensions. Table 8 in Section 7.3.2 of the Version 1.3.0 WMS Specification [79] shows all parameters for the GetMap request and whether they are required or optional. If you have questions about how to format the parameters, you can always refer to the specification; however, most GIS software gives you user-friendly interfaces for interacting with WMSs so that you don't have to formulate the URLs yourself.
Besides the GetMap request, WMS supports one other required operation (GetCapabilities) and an optional operation (GetFeatureInfo).
The GetCapabilities request returns metadata about the service, which you can use as a guide when making other types of requests. Here's what a GetCapabilities request would look like for ourland cover WMS:
Take a look through the XML response returned in your web browser when you hit the above URL
Notice that you can see information about each layer in the WMS, which is helpful when you are making a GetMap request and you want to know the appropriate names to specify for the LAYERS parameter.
<Layer queryable="1"> <Name>10</Name> <Title> <![CDATA[ LandCover_UTC_2006_StateCollege ]]> </Title> <Abstract> <![CDATA[ ]]> </Abstract> ... </Layer>
You can also see a list of the spatial reference systems that the service publisher has decided to support for the layer.
<CRS>CRS:84</CRS> <CRS>EPSG:4326</CRS> <CRS>CRS:4269</CRS>
GetFeatureInfo is an optional method that the service publisher can enable, allowing users to query the attribute data of a WMS layer at a specific location. This makes it possible to add interactive elements such as informational popups without enlisting the help of a second web service.
When you make a GetFeatureInfo request, you supply many of the same parameters as the GetMap request, with the addition of the screen coordinates of the pixel you want to query.
The land cover example says that the layer 9 is queryable, so try the following query URL to query for the feature at coordinates x=600 and y=300 within the map image with width=1200 and height=900:
The returned plain text response, in this case, will tell you that there that the pixel value for these coordinates is 5 which according to the description at PASDA [83] means 'building':
@10 PixelValue;OBJECTID;COUNT; 5;5;1378041;
GetFeatureInfo is somewhat of a wildcard, because the WMS specification does not mandate any particular structure or format for the returned information. If you don't know who published the service, you must make a request and examine the response in order to understand how to work with the returned information. The GetCapabilities response can show you the supported INFO_FORMATs you can request:
<GetFeatureInfo> <Format>application/vnd.esri.wms_raw_xml</Format> <Format>application/vnd.esri.wms_featureinfo_xml</Format> <Format>application/vnd.ogc.wms_xml</Format> <Format>text/xml</Format> <Format>text/html</Format> <Format>text/plain</Format> ... </GetFeatureInfo>
The OGC allows some flexibility to adjust the map symbols used in WMS layers. This is accomplished by referencing a chunk of XML called a Styled Layer Descriptor (SLD). An SLD describes all the symbol sizes, colors, and markers that should be applied within the WMS. SLDs are complex enough that they have their own OGC specification documents [87] defining how they are supposed to operate.
SLDs can be designed by either the service publisher or the client. To formulate a really useful SLD, you have to know a little bit about the layers in the WMS. You can get this by calling the optional DescribeLayer operation on the WMS.
Once you've created the XML for your SLD, you can apply it in one of several ways. The most common way is to put the SLD in its own file on a web server somewhere, and reference the URL of that file when you make a GetMap request. There is an optional SLD parameter in the GetMap request that you can use for this very purpose. Another way is to use the optional SLD_BODY parameter of the GetMap request and just provide the relevant chunk of XML directly in the URL of the request. This obviously can create long and unwieldy URLs, and requires a lot of special character encoding or escaping.
The XML used with SLDs can get verbose and tends to contain many nested levels. Because of this, it's unlikely that you'll sit down to compose an SLD from scratch. Instead, you'll probably start with a sample and adjust it to meet your needs. Alternatively, you can use the GUI environment from QGIS to style a layer and then save it out as an SLD.
Both approaches (starting an SLD from an existing sample and exporting an SLD from QGIS) are covered in the lesson walkthrough.
GeoServer maintains your data information and your style information completely separate. In the Layers page you define what datasets you want to serve, and in the Styles page you define all the SLDs you want available. The Publishing tab of the Edit Layer page is the (somewhat obscure) place where you connect the dots and define which style will be applied to your layer.
In the above graphic, the GeoServer administrator has examined the pool of all styles added to GeoServer (left) and selected three of those (right) to be available for use with the FarmersMarkets layer. The WMS will advertise these three styles for the layer and will use the one called FarmersMarkets if the user does not specify a different style in the STYLES parameter of the GetMap request.
The examples below are not functional, but they show you the URL structure for requesting different styles.
GetMap request that uses the default style ("FarmersMarkets"):
http://localhost:8080/geoserver/philadelphia/wms?service=WMS&version=1.1.0&request=GetMap&layers=philadelphia:FarmersMarkets&styles=&bbox=-8377237.031452011,4854963.883290707,-8357515.816574659,4877579.355521561&width=446&height=512&srs=EPSG:3857&format=image%2Fpng
GetMap request that overrides the default style and uses the style named "point" instead:
http://localhost:8080/geoserver/philadelphia/wms?service=WMS&version=1.1.0&request=GetMap&layers=philadelphia:FarmersMarkets&styles=point&bbox=-8377237.031452011,4854963.883290707,-8357515.816574659,4877579.355521561&width=446&height=512&srs=EPSG:3857&format=image%2Fpng
Notice that the latter URL includes styles=point, indicating that GeoServer should use the SLD named "point" to draw the layer. Alternatively, the user could have specified styles=FarmersMarketsLabeled.
Before you move on, please read the following pages of GeoServer documentation. This contains general information about SLDs, technical notes about how SLDs are applied in GeoServer, and a set of example SLDs that you can use to get started when it comes time to make your own. You'll get some practice with this technology during the walkthrough, but it is well worth your time to become familiar with this particular section of the documentation. In fact, it might be wise to read it both before and after you complete the walkthrough in order for you to confirm your learning and absorb the maximum amount of information.
In this walkthrough, you'll get some practice serving some geographic data as a WMS using GeoServer. The exercise will progress from simple to complex in the following manner:
The next walkthrough will go into more depth with SLDs and group layers.
As you complete these walkthroughs, think of some of your own data you'd like to serve as a WMS in fulfillment of the Lesson 4 assignment. You won't be required to use this WMS in your term project, but you can if you choose.
Let's start out by serving a single layer as a WMS. We'll stick with Philadelphia in this lesson so that we can eventually pull in some of the base layers that we clipped and projected previously. The first layer we'll work with is a polygon dataset of Philadelphia neighborhoods that I derived from a Zillow.com [91] shapefile.
The above WMS indeed contains the Philadelphia neighborhoods, but there are no labels and we didn't get to choose the color scheme. In this part of the walkthrough, you'll use an SLD to apply a blue outline and labels with no fill. This will allow the neighborhoods WMS to act as a thematic layer that you can place on top of other base layers.
To work with SLDs in GeoServer, you first add the SLD under the Styles list. Then you can go back into the layer properties and apply the style. Decoupling the styles and the layers in this fashion allows you to reuse a particular style on multiple layers.
The first thing we'll do is prepare the SLD, starting with an existing sample and then modifying it to meet our needs.
<?xml version="1.0" encoding="ISO-8859-1"?> <StyledLayerDescriptor version="1.0.0" xsi:schemaLocation="http://www.opengis.net/sld StyledLayerDescriptor.xsd" xmlns="http://www.opengis.net/sld" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <NamedLayer> <Name>Polygon with styled label</Name> <UserStyle> <Title>SLD Cook Book: Polygon with styled label</Title> <FeatureTypeStyle> <Rule> <PolygonSymbolizer> <Fill> <CssParameter name="fill">#40FF40</CssParameter> </Fill> <Stroke> <CssParameter name="stroke">#FFFFFF</CssParameter> <CssParameter name="stroke-width">2</CssParameter> </Stroke> </PolygonSymbolizer> <TextSymbolizer> <Label> <ogc:PropertyName>name</ogc:PropertyName> </Label> <Font> <CssParameter name="font-family">Arial</CssParameter> <CssParameter name="font-size">11</CssParameter> <CssParameter name="font-style">normal</CssParameter> <CssParameter name="font-weight">bold</CssParameter> </Font> <LabelPlacement> <PointPlacement> <AnchorPoint> <AnchorPointX>0.5</AnchorPointX> <AnchorPointY>0.5</AnchorPointY> </AnchorPoint> </PointPlacement> </LabelPlacement> <Fill> <CssParameter name="fill">#000000</CssParameter> </Fill> <VendorOption name="autoWrap">60</VendorOption> <VendorOption name="maxDisplacement">150</VendorOption> </TextSymbolizer> </Rule> </FeatureTypeStyle> </UserStyle> </NamedLayer> </StyledLayerDescriptor>Notice the places where the stroke and fill colors are specified, as well as the text weight and font. Also notice the tags marked VendorOption, which are not part of all SLDs but are supported by GeoServer as a special means of finding suitable label locations (maxDisplacement) and forcing the labels to wrap after reaching a certain length (autoWrap). See this GeoServer documentation [95] for details.
<?xml version="1.0" encoding="ISO-8859-1"?> <StyledLayerDescriptor version="1.0.0" xsi:schemaLocation="http://www.opengis.net/sld StyledLayerDescriptor.xsd" xmlns="http://www.opengis.net/sld" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <NamedLayer> <Name>Polygon with styled label</Name> <UserStyle> <Title>SLD Cook Book: Polygon with styled label</Title> <FeatureTypeStyle> <Rule> <PolygonSymbolizer> <Stroke> <CssParameter name="stroke">#133E73</CssParameter> <CssParameter name="stroke-width">2</CssParameter> </Stroke> </PolygonSymbolizer> <TextSymbolizer> <Label> <ogc:PropertyName>NAME</ogc:PropertyName> </Label> <Font> <CssParameter name="font-family">Arial</CssParameter> <CssParameter name="font-size">11</CssParameter> <CssParameter name="font-style">normal</CssParameter> <CssParameter name="font-weight">bold</CssParameter> </Font> <LabelPlacement> <PointPlacement> <AnchorPoint> <AnchorPointX>0.5</AnchorPointX> <AnchorPointY>0.5</AnchorPointY> </AnchorPoint> </PointPlacement> </LabelPlacement> <Fill> <CssParameter name="fill">#133E73</CssParameter> </Fill> <VendorOption name="autoWrap">60</VendorOption> <VendorOption name="maxDisplacement">150</VendorOption> </TextSymbolizer> </Rule> </FeatureTypeStyle> </UserStyle> </NamedLayer> </StyledLayerDescriptor>
This is still not perfect (notice the many labels overlapping features), but it is progress, and it has opened the door to editing the style through other XML options defined in the SLD and GeoServer documentation.
In general, labeling is a tricky subject with web maps. Labeling is computationally intensive and relies on complex rules that can slow down the map drawing. The labeling rules available with GeoServer and WMS are relatively simple compared to the logic used by a desktop program like QGIS or ArcMap. Because of these issues, web map authors sometimes rely on alternative mechanisms such as interactive popups or text that changes dynamically depending on where you point or click the mouse (or tap the display). Fetching this information on demand is faster than waiting for thousands of labels to be placed and eliminates the reliance on complex labeling algorithms.
There are lots of clients you can use to view a WMS. You already saw how your WMS could be previewed with OpenLayers. Now let's take a few minutes to see how QGIS works with WMS layers. This is important if you want to bring web services into your desktop maps as backgrounds or thematic layers.
If you don't have the OpenLayers plugin installed, you can still add the Stamen Watercolor map. You do this by pointing QGIS at the basemap image tiles on the Stamen servers. First, make sure your Browser panel is open. If necessary, click View > Panels > Browser. In the list of layer types in the Browser panel, find the XYZ Tiles item, right-click it, and choose New Connection. Enter the Name as "Stamen watercolor" and enter the URL as http://c.tile.stamen.com/watercolor/{z}/{x}/{y}.jpg . This is the generic URL structure for the Stamen Watercolor tiled map images (more on this in Lesson 5), where x is the column, y is the row, and z is the zoom level of the tile to fetch at any given map extent. Click OK, and then drag and drop the resulting Stamen Watercolor layer down into your map layer list in the Layers panel. See this post [98] for tips on adding all kinds of other basemap layers to QGIS 3 in the same fashion.
Congratulations! You have just made a (somewhat strange) "mashup" that brings in web services from two different servers. In future lessons, you'll learn how to do this in a more practical fashion in a web application.
QGIS not only displays WMS layers; it can also help you create SLDs. In the next walkthrough, you'll make some SLDs using QGIS and learn how to group WMS layers together using GeoServer.
You've learned how to serve and style a single layer as a WMS using GeoServer and some basic SLD examples. Now, you'll go a bit further and use QGIS to set up your styles for you. You'll also see how you can group multiple WMS layers using GeoServer.
Some of the steps require you to apply the skills you learned in the previous walkthrough. You may need to pause and review in order to complete these steps.
QGIS allows you to save out your current layer symbology as an SLD. You then apply the SLD in GeoServer using the same procedure you learned in the previous walkthrough. Styling layers in QGIS is a lot easier than writing XML. However, be aware that some features, such as labels, are not saved in the SLD. Also, the style may look slightly different when you apply it in GeoServer. Be prepared to use some trial and error to get things exactly right.
Let's style a couple of the other Philadelphia base layers that we used in previous lessons, such as roads and the city boundary. We'll eventually group these together with the neighborhoods layer within the same WMS.
In some cases, you'll use WMS for your thematic layer and lay it in on top of a non-WMS basemap that uses tiles. This is what you did in the previous walkthrough. In some cases, when you have a very simple map, you may use a WMS for both the thematic layers and the base layers. Let's take a look at how GeoServer allows you to group multiple layers together in a single WMS. We'll use the neighborhoods layer, the city boundaries layer, and the roads layer that you have already published and styled in your geog585 workspace.
If you get an error when adding the layer, try cleaning out your QGIS WMS cache by clicking the Settings menu > Options, go to the Network tab of the options box, and then click the little red garbage can icon to clear the cache.
This exercise has shown how you can style any number of layers and group them together as a dynamically drawn WMS in GeoServer. If you have many layers to combine, it's faster and more scalable to use tiles, which you'll learn about in the next lesson. However, the techniques you've learned with WMS are sufficient to launch a quick and easy web map if you'll only have a few layers or a limited number of people using the map.
As you've completed the walkthroughs, you've hopefully begun to think about ways that you could serve your own term project data as a WMS. You are not required to use a WMS in your term project, but I would like you to get some more practice with it using your own data. If you like the result, you're welcome to use it in your term project submission.
The assignment this week has two parts:
Submit these deliverables to the Lesson 4 assignment drop box on Canvas, either as a single document or as two separate documents.
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
This week's lesson focuses on tiled web maps. Tiles are relatively small square-shaped "chunks" of data (either rasterized map images or raw vector coordinates) that have been pregenerated by the server and stored in a directory called a cache. When web users navigate the map, the server can just hand out the tiles rather than generating the map on the fly.
In this lesson, you'll learn the pros and cons of tiled maps, as well as strategies for building and maintaining a tile cache. You will learn about traditional rasterized tiled image formats, as well as a newer generation of tiles that store vector coordinates. Because rasterized image tiles have been around much longer and have mature support from tile generating engines, tile servers, and clients, most of the lesson content focuses on these types of tiles. The newer vector tile format is discussed near the end of the lesson content.
There are two walkthroughs associated with this lesson, both of which involve rasterized image tiles because these currently have the most mature tools for end-to-end FOSS workflows. The first walkthrough shows how to create a simple cache of your Philadelphia neighborhoods map using the GeoWebCache software that is integrated into GeoServer. In the second walkthrough, you'll use QGIS to create a Philadelphia basemap cache with some nicer cartography than you would get using GeoServer.
As mentioned in previous lessons, the earliest web maps were typically drawn on the fly by the server, no matter how many layers were available or requested. These are the types of maps you just created using GeoServer and WMS. As you may have noticed, the symbol sets and labeling choices for this type of map are relatively limited and complex to work with. In fact, for many years, web cartographers had to build a map with minimal layer set and simple symbols to avoid hampering performance. In many cases, a cartographer was not even involved; instead, the web map was made by a server administrator tweaking XML files that defined the layer order, symbol sizes, and so forth. This was the case with both open specification web services (like WMS) and proprietary web services (like Esri ArcIMS).
Part of this approach stemmed from early efforts to make web GIS applications look exactly like their desktop counterparts. Sometimes these are referred to as “Swiss Army Knife applications” because they try to do everything (you may know one!). People expected that in a web GIS they should be able to toggle layer visibility, reorder layers, change layer symbols on the fly, and do everything else that they were accustomed to doing on the desktop. Ironically, this mindset prevailed at a time when web technology was least suited to accommodate it.
In the mid-2000s, after Google Maps, Microsoft Virtual Earth (now Bing Maps), and other popular mapping applications hit the web, people started to realize that maybe they didn't need the ability to tinker with the properties of every single layer. These providers had started fusing their vector layers together in a single rasterized image that was divided into 256 x 256 pixel images, or tiles. These tiles were pregenerated and stored on disk for rapid distribution to clients. This was done out of necessity to support hundreds or thousands of simultaneous users, a burden too great for drawing the maps on the fly.
The figure below shows how a tiled map consists of a "pyramid" of images covering the extent of the map across various scales. Tiled maps typically come with a level, row, and column numbering scheme that can be shared across caches to make sure that tile boundaries match up if you are overlaying two tile sets.
Cartographers loved the tiled maps because now they could invest all the tools of their trade into making an aesthetically pleasing web map without worrying about performance. Once you had created the tiles, you just had a set of images sitting on disk, and the server could retrieve a beautiful image just as fast as it could retrieve an ugly one. And because the tiled map images could be distributed so quickly by a web server, Google, and others were able to employ asynchronous JavaScript and XML (AJAX) programming techniques to retrieve the tiles with no page blink as people panned.
This was revolutionary. Which would you rather have: a slippy map with stunning cartography and no layer control, or a clunky and ugly map with the ability to reorder layers and adjust the color of a school? Some longtime GIS geeks had to stop to think about this, but for the common web user, the choice was a no-brainer.
Within a year or two of Google Maps' release, commercial GIS software began offering the ability to build map tiles. For many, ArcGIS Server was desirable because the map could be authored using the mature map authoring tools in ArcMap; however, cost was a concern for some. Arc2Earth was another commercial alternative. The free and open source Mapnik library could also build tiles, but it wasn't until recent years that projects like TileMill wrapped a user-friendly GUI around Mapnik. QGIS now also offers the option to create tiles, as you will see in the second walkthrough of this lesson.
Tiled maps were the only model that could reasonably work for serving complex web maps to thousands of simultaneous users. However, they eliminated the ability for users to change layer order or symbols. People started working around this by serving out their general-purpose basemap layers as tiles and then overlaying a separate layer with thematic information. The general-purpose basemap tiles could be re-used in many applications. The thematic layers could also be tiled if the data didn't change too quickly or cover too broad an area at large scales. For example, you saw in Lesson 2 that the Microsoft Bing Maps traffic layer is served as PNG tiles.
If you want faster navigation of your basemap, or you feel that more than a couple of users will be requesting maps simultaneously from your server, you should create a tile cache of your basemap. You may also choose to cache thematic layers if their features are not constantly changing attributes or position.
In both cases, be aware that the tile cache represents a snapshot of your data at the time the cache was created. To put it more bluntly, your tiles are “dumb images” that will not automatically update themselves when the back end data changes. You are responsible for periodically creating new tiles in order to update the map.
With large caches, sometimes server administrators target the cache updates at just the changed areas, rather than rebuilding tiles for the entire map. This requires keeping some kind of log about which places were edited, or comparing “before” and “after” versions of a dataset. Later in this course, you will see that OpenStreetMap.org works in this way. If you make an edit to OpenStreetMap, you'll see your changes appear within a few minutes (at least at the largest scales). This is not because the entire cache was rebuilt, but because an area of change was detected, thereby triggering a partial rebuild of the cache in just the modified area.
Building a tiled basemap requires lots of rich data, high-end map authoring software, cartographic skills, and potentially enormous amounts of time and disk space. You may still need to do it at one point or another, which is why you will get a taste of this experience in Geog 585. However, because of these challenges, general-purpose web mashups often use tiles made by somebody else. OpenStreetMap is an attractive option if you want free tiles with no restrictions. If you want to use Google's, Microsoft's, or Esri's tiles, you may be able to use them for free, or you may have to pay, depending on the nature of your map (commercial or not for profit) and how many people use your app. Other companies such as CloudMade and MapBox have marketed their own versions of tiles using OpenStreetMap data.
If you are going to build your own basemap, it's helpful to have an experienced cartographer on staff who is experienced with designing at multiple scales. The symbols, colors, and details must be adjusted appropriately at each of the scales where tiles are created. Tiled basemaps can quickly get complicated with layer and labeling scale suppression rules. Cartographers may also need to design one set of tiles to stand on its own and another set of tiles to overlay remotely sensed imagery, a task that requires very different colors and symbols.
If you are going to overlay your tiles with any of the tiles from OpenStreetMap, Google, Microsoft, or Esri (or even just attempt to look like them), you must warp your most precious GIS data into a modified spherical Mercator projection that was created solely for the convenience of fitting the world onto a set of square tiles. GIS purists balked at this idea and predicted it would fail to achieve mass uptake, but many people (at least at mid-latitudes) now hold their nose and move forward with it.
Be aware that Esri, Google, and other organizations have used other code numbers and variations of this projection in the past: things can get very confusing if you are using older software or APIs. In the past couple of years, people seem to have standardized on EPSG:3857 although even the subparameters of this projection can be interpreted in diverse ways that lead to offsets. As a side note, can you figure out the humorous reason that Google once used the code 900913 for this projection?
Even when you display your maps in EPSG:3857, you do not ever want to perform measurements in this projection. The results will be largely skewed even at mid-latitudes. It's best to make sure that geometries are projected into a more local coordinate system before performing any measurements. Most popular mapping tools such as Google Earth and the ArcGIS.com map viewer do this behind the scenes when you use their measuring tools, but it doesn't happen automatically if you're building your own solution.
Not only must you match the projection in order to overlay, you must also match the scales. These are not nice, well-rounded numbers; rather, they were derived mathematically from the starting point of putting the whole world on a 2x2 grid of tiles. For example, one of the scales is 1:36,111.98 and as you zoom in, the next one is 1:18,055.99. So much for your simple USGS 1:24000 series! Partly for this reason (and partly because many laypeople don't understand map scales), the common web map scaleset is often referred to with simple numbers such as “Level 14,” “Level 15,” etc., that increase as you zoom in. You just have to get a feel for which levels correspond to national scale, provincial scale, city scale, neighborhood scale, etc. The table in the Bing Maps Tile System [103] article is helpful for this.
Map tiles often have a simple folder structure, which makes them easy to serve. However, they can become complex to manage due to their sheer size and number. There are several different ways people have devised to serve map tiles:
At large (zoomed in) scales, the number of tiles to cache can be overwhelming, especially if you are covering a broad area such as a state or country. The irony is that at such large scales, many tiles will convey very little information. Zooming in to a neighborhood block scale at 1:2250 may show plenty of interesting features, but pan out into the desert or ocean at the same scale, and the tile may be completely blank. Do you want to spend the hours creating these tiles and gigabytes of disk space to store them?
In these situations, you may want to find software that can create tiles on demand, meaning at the time they are first visited by a user. The first person to navigate to a region will need to wait for the server to create the tiles, but subsequent visitors will enjoy the full benefits of created tiles. The most popular areas will fill up with tiles, but you won't spend resources creating and storing tiles that are never visited. Obviously, this approach varies in effectiveness based on how fast the server can actually draw the tiles on demand.
An alternative is to use a “Data not available” tile to denote areas where tiles have not been created. Web map administrators are sometimes loathe to do this, but it is often a common enough practice that lay users blame themselves when they see the tile (“Oops, I zoomed in too far!”) rather than the administrator (“Why didn't they make the map available at this scale?”).
The best approach may be to strategically create a subset of the most interesting tiles and leave the less interesting tiles to be created on demand (or returned using a “Data not available” tile). As a geographer, it may hurt your soul to call any place “less interesting,” but the bitter truth is that not all tiles of the map experience an equal number of views. Fisher (2007) showed how early visitors to Microsoft Virtual Earth (now Bing Maps) stuck mostly to the big cities, coastlines, and transportation corridors. Quinn and Gahegan (2010) built models taking into account these patterns, showing how the majority of map requests could likely be satisfied by creating a fraction of the full number of tiles that it would ordinarily take to cover the full rectangular extent of the map. Their models were cobbled together using datasets like buffered roads, coastlines, and points of interests, but more recent feeds from social media such as geotagged tweets and Flickr photos may prove even more accurate in revealing the most interesting regions of the map for the majority of users. Note that some types of specialized maps (for mineral exploration, or wilderness conservation) may have very different usage patterns than general-use basemaps.
The ability to selectively cache a subset of tiles like this depends on the tile creation software's flexibility in allowing the administrator to designate custom regions for caching. Most software just allows the submission of a rectangular bounding box for tile creation; however, interesting areas of the map such as cities and coastlines are usually not shaped like rectangles. If you identify an irregularly-shaped region where you want to create tiles, you may have to abstract it into a series of rectangles and run multiple tile creation jobs, using each rectangle as an input. If you are luckier, your tile creation software will accept a spatial dataset (such as a shapefile) as a bounding region.
Creating tiled web maps is a common task that has been addressed by various FOSS packages. The most accessible one for you at this point is GeoWebCache because it's integrated directly into GeoServer. Others include TileCache and TileStache.
The Mapnik library is a FOSS tile creation library that binds to Python and other languages. It allows a lot of advanced drawing options not found in your typical WMS layer. Working with Mapnik typically required some Linux knowledge and some trial and error; however, the for-profit company Mapbox released an open source program called TileMill that can run on Mac and Windows and puts a nice GUI around Mapnik. We have been using TileMill in this class for a long time, but a couple of years ago, Mapbox stopped developing it. TileMill has been continued as a community-driven project on GitHub [105], but the development has slowed and problems have become more frequent recently. At the same time, tile creation support has been integrated into QGIS, so at some point, we made the decision to use QGIS for this part of the class as well. In the second walkthrough in this lesson, you'll use QGIS to create a basemap of Philadelphia using some of the layers you processed earlier.
Although the rasterized tile sets we have discussed in this lesson are able to deliver nice-looking maps in a relatively rapid format, they can be cumbersome to keep updated, and they require enormous amounts of computing resources at large map scales. To work around these challenges, a data storage format called "vector tiles" has gained popularity in the past several years. Mapbox has led development efforts on vector tiles and has shared a vector tiles specification [106] under a Creative Commons license.
Vector tiles are exactly what you would guess: they store chunks of vector data instead of storing a map image. The idea behind vector tiles is that it is more efficient to keep data styling separate from the data coordinates and attributes. The client can use a predefined set of styling rules to draw tiles of raw vector coordinate and attribute data sent by the server. This allows the restyling of data on the fly, which is another serious limitation of rasterized tiles. Think about it: If you want to change the shade of green used to draw parks with your rasterized tiles, you must rebuild every tile containing a park. If you want to do the same thing with vector tiles, you just update your styling instructions in one place and the tiles themselves stay the same. Other display operations such as rotating the map also become easier to implement with vector tiles.
Vector tiles are designed to be small on disk, and employ a number of optimization approaches designed to reduce the amount of characters needed to store the geographic data and attributes, some of which are described in this video by Mapbox engineer Dane Springmeyer [107]. He also introduces a product called Mapbox Studio, [108] which works with vector tiles only and is being promoted by Mapbox as a replacement for Tile Mill. The .mbtiles file format, which originally stored rasterized tiles, now only stores vector tiles when exported from Mapbox Studio.
In reality, there continue to be use cases [109] for vector and rasterized tile formats, although it is likely that a number of organizations will see performance benefits from rebuilding some of their originally rasterized tile sets as vector tiles in the future. This is even more likely as popular commercial software packages such as ArcGIS introduce tools to work with the Mapbox vector tile specification, a strategic decision that Esri announced in a 2015 blog post [110]. On the open source side of things, GeoServer added support for vector tiles in version 2.14, as detailed in these instructions. [111]
Open source clients are also recognizing the staying power of vector tiles, exemplified by the VectorTile layer format [112] built into OpenLayers 3 and plugin support for Mapbox vector tiles in Leaflet [113]. At the time of this writing, QGIS does not natively support viewing vector tiles, although there is a relatively new plugin [114] for this purpose.
Suppose you're satisfied with the layers and symbols in your WMS, but you want it to draw faster and be available to many simultaneous users. In this situation, it might make sense to use GeoWebCache to create your tiles, because GeoWebCache is built directly into GeoServer. In this walkthrough, you'll use GeoWeb Cache to create a tile cache for the Philadelphia NeighborhoodMap group layer WMS that you published in the previous lesson.
Although performance is improved with the tile cache, you may notice some duplicate labels appearing. This is a difficult problem to avoid with map tiling, because each tile does not "know" about the labels on the adjacent tiles. To mitigate this problem, tile caching software typically draws an area much larger than a tile and then cuts it up into individual tiles. GeoWebCache calls this large area a "metatile" (Esri calls it a "supertile"). If you like, you can experiment with adjusting the metatile size; although duplicate labels can still appear at the metatile boundaries, the duplicates will be fewer and farther between. You may also find that the settings and options in the next walkthrough with QGIS make it easier to get the labeling you want.
In this walkthrough, you will use QGIS to create a general-use basemap of Philadelphia. The idea is that you will be able to overlay thematic layers on top of this map in lessons to come. The data for this walkthrough is the base data for Philadelphia that you preprocessed in Lesson 3. If you followed all instructions, you should have this data in a folder called c:\data\Philadelphia or something similar. It should be using the mercator projection EPSG:3857.
It would also be a good idea to save the project now!
Once you've finished the map design phase and your map looks good at each scale, you can start thinking about generating the tiles. For large maps, plan for this to take some time, taking into account some of the factors discussed earlier in the lesson, such as the shape of the map and the scale levels you choose to generate.
This section of the walkthrough explains how to generate tiles of your Philadelphia basemap, and host them on the web. You will then test them out in a web page.
We are setting the minimum zoom level to zero here, so that tiles will be provided all the way up to the top of the pyramid in the tiles hierarchy. However, often when working with a relatively small project area like this (e.g., in your term projects), your plan is to limit how far out the user can zoom to avoid that the displayed area becomes tiny or (almost) invisible. In that case, it makes sense to set the minimum zoom option accordingly and use Javascript programming (see next few lessons) to restrict the possible zoom levels. That said, in terms of storage space, you won't save much by excluding the lower zoom levels because almost all storage will always be taken up by the highest levels.
If you navigate around your folder of tiles, you'll notice that the images are extracted into a highly organized structure of level\column\row. This structure is understood by various mapping programs and APIs, so all you have to do at this point is put your tiles onto a web-facing server. If you already have acces to some web space, you could simply upload the folder with the tiles to that web server. What we will instead do in this course is use Amazon's cloud services, namely the S3 service for scalable storage, as a convenient place to host the tiles on the web. We will place the tiles there and then test them in a web map. However, this will require you to set up an Amazon AWS account first, including registering a credit card. Uploading and accessing the tiles over the web will then cause some costs that your credit card will be billed for at the end of each month but these will be very small, < $10 most likely, if you just use the tiles for this class and follow the instructions below. The steps below will include descriptions of how to monitor the costs and how to set up a billing warning if the accumulated monthly costs go above a certain level. Please note that the workflow below is very specific to S3. If we would instead use a normal web server to host the tiles, we would most likely use FTP or SCP to upload the tiles to some folder in our personal web space on that server.
Go to: https://portal.aws.amazon.com/billing/signup [115]
Create a Root user account using your PSU email. (If you already have created an AWS account for your PSU email, e.g. for another class, you can instead log into that account and continue with Step 3.)
You can give your account any name you want. You will then have to verify your email address and define a Root user password. After that, you have to enter your contact information and confirm your identify by entering a verification code sent to you via SMS or an automatic call to the phone number you provide. Lastly, you have to select a support plan; please choose the free "Basic support plan". When done setting up your account, you can click the button that leads to the "AWS Management Console".
Now, in the AWS Management Console, go to the search bar at the top of the screen and search for Billing . Click on the link (the name of the service at the top) to get to the "AWS Billing Dashboard".
The Billing Dashboard will show you both a cost estimate for where you will end up at the end of the month and the current costs accumulated so far this month ("Current MTD balance"). Further down below, there is a link called "View your bill" which will lead you to a page where you can explore the costs accumulated so far by unfolding the items at the bottom under "Amazon Web Services EMEA SARL charge by service".
Now type in the term Budget in the search bar at the top and select the sevice called "AWS Budgets" by clicking the name. We now want to set up a budget so that you will get alerted by email if the estimated or accumulated costs for a month pass a certain threshold.
Click on the orange button "Create a budget" at the top. On the following page, select the options "Use a template (simplified)" and "Monthly cost budget", then fill out the details for the budget as shown in the image below but using your own email address to which the warning should be sent. The $5 is just a suggestion. Feel free to use a somewhat higher or lower number. You can also change the name of the budget but the default name is fine. Lastly, click the orange "Create budget" at the bottom.
As the information text said, you will now receive messages if any one of the following conditions is true: 1) your actual spend reaches 85% of the budget you defined, 2) your actual spend reached 100%, or 3) your forecasted spend is expected to reach 100% of the budget. You can alway go back to the AWS Budgets page to select and modify you budget there, including changing the threshold value. However, here is one important final note regarding the Billing and Budgets dashboards: For some reason, AWS does not update the accumulated costs in real-time. It can actually take multiple hours until the numbers get updated. Nevertheless, this should still help you keep track of how the costs are slowly increasing while you are using the data in your S3 bucket.
In the AWS Management Console, go to the search bar at the top of the screen again and now search for S3.
Click at the first result that is indeed the S3 service.
Once in the S3 service, click the orange button to create a bucket (basically a storage container inside your S3 cloud storage).
Under "General configuration", give your bucket a unique name, we recommend starting with "geog585-" followed by some random string of character and/or numbers that you pick yourself. We here will use the name geog585-a1rt15 .
If you want you can select the region that is closest to you, or in case of a real "global" project the region closest to your clients.
Scroll down to where it says Block Public Access settings for this bucket.
Usually we want to protect our data, but in this case, the map tiles have to be fully accessible by everyone, so uncheck the first tick to allow access.
Tick the box where you acknowledge that the bucket will become public.
Leave the rest as it is, and scroll down to click Create Bucket.
Once back to the main page, you should see your new bucket in the bucket list. Click on the name to open it up.
You should now see the page for your bucket, currently still not containing any objects. Click the tab called Permissions at the top.
Scroll down to Bucket Policy and click Edit. Amazon really does not want us to create a public bucket unintentionally, so we need to do one more thing.
Copy and paste the following in the available space:
Above the text field and its title "Policy", you can find the ARN name of your bucket. Click the small icon left to it, to copy this name. Then go to the last line of the code you just pasted to change the name of the bucket there. Paste the ARN name you copied to replace the string arn:aws:s3:::geog585-a1rt15 there. Be careful to keep the initial double quote and the /*" after the bucket name.
Hit the orange button Save Changes.
Now we are almost read to upload our map tiles. If you only want to upload a couple of files (for instance, to upload and share the web map you will create in your Lesson 7 homework assignment), you can do so directly on the AWS page for your bucket by selecting the "Objects" tab at the top (you can think of "objects" as meaning "files" here) and then clicking the Upload button. This will show an area to which you can the drag and drop folder and files from the Windows File Explorer. The AWS will then scan the files and afterwards show you another Upload button that you can press to start the actual upload. However, this approach of using the web page won't work well for basemap tile sets that tend to consist of a huge amount of small files. The Philadelphia tile set, for instance, only is 200MB large in total but it consists of roughly 21,000 files. In our tests, uplading this amount of files via the web interface took around 6 hours.
Therefore, we are going to do the upload via another piece of software, the AWS Command Line Interface (CLI), which is much more efficient for small files and should be able to upload the Philadelphia tile set in just a few minutes. To set this up, we need to do two things: (1) Create a so-called Access Key that will allow us to access our S3 bucket via CLI and (2) install the CLI software itself. Let's start with creating a Root access key...
On the AWS page, click on your account name at the very top right of the page and then select "Security credentials". Scroll to the area called "Access keys". Then click the button called "Create access key".
AWS will now show you a window called "Retrieve access key" with your new access key in plain text and the corresponding secret key hidden by default. You can think of the access key as a login name and the secret key as the password. You will need both for using CLI, so please copy both of them. Important: This will be the only time you will be able to see/copy the secret key. If you loose it, you will have to go back and create a new access key. Once you have the keys stored somewhere else, click the "Done" button at the bottom. To delete or create further access keys, you can always go back to the Access Key area under Security Credentials as in Step 21.
Now, it is time to install the AWS CLI tools: Click on this link https://awscli.amazonaws.com/AWSCLIV2.msi [116] to download the software.
Double-click the downloaded file called AWSCLIV2.msi to start the installation. Then follow the instructions given by the installer.
Now open the Windows Command Shell cmd by going to the Windows start menu and typing cmd . This should open the black shell window (see next screenshot below in Step 27).
Let's first check if CLI has been installed correctly. Type in the following command and then press ENTER: aws --version
The output should look as shown in the next screenshot below starting with "aws-cli" followed by some version number.
Next, we have to give CLI some information including the access key and secret key. Type in the following command and then press ENTER: aws configure
You will then be asked for access key, secret key, default region, and default format. For the access key and secret key, copy and paste the keys that you copied from the AWS page in Step 22. For the other two items, you can simply press ENTER without typing in antyhing.
Now we are ready to upload the files to our bucket. Use the cd command to move to the folder that contains the PhillyBasemap folder to which you exported the tiles in Steps 4+5 from the Section "Exporting the tiles" (see next screenshot from Step 29).
Now type in the following command but with the name in bold replaced by the bucket name you used and then press ENTER:
aws s3 cp PhillyBasemap s3://geog585-a1rt15/PhillyBasemap --recursive
You should now see how the tiles are uploading. If instead you get some odd error messages. Try with running the command a second time; somehow this seems to happen sometimes on the first attempt.
Once everything is uploaded, you should see a prompt in the cmd shell window. Now switch back to the AWS web page and refresh the "Objects" tab on the page for your bucket. Navigate to the PhillyBasemp folder and then further into the subfolders to one random png tile.
On the screen that opens, find the Object URL and copy it. The URL is made up of the base URL (https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap [117]) followed by the folders and file name corresponding to the zoom level and x, y indices of this particular tile image. If you want, you can open a new tab in your browser and copy the URL into the address bar to open this one tile. You should see the tile image but keep in mind that you have picked an empty tile so that you will just get an image that is fully white. To later use your basemap tiles in a web map you, you will have to take the base URL and then replace the concrete numbers at the end with a string of variabels to create a template string, for instance looking like this: https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap/ [118]{z}/{x}/{y}.png . We will make use of this option in the final part of this walkthrough in which we will test out our tiles in an ArcGIS Online map.
GOOD JOB! You are done!!
Optionally, you could add other web layers on top, but, for the sake of time, this will not be described here. In future lessons, you'll learn how to build this type of web map programmatically without using ArcGIS.com.
Now that you've gone through a walkthrough and built your own tiled map, hopefully your appreciation has increased for the amount of cartographic design and effort required for producing a web basemap. In this week's assignment, you'll look at some existing tiled maps, then get some practice building your own.
Do the following:
Please produce a report containing all the information requested in the first bullet above, as well as a URL to your tileset in your AWS S3 bucket so that I can test in a web map as shown in the walkthrough, together with a few words on how you designed your tiles so that I know what to look for. Then submit the report to the Lesson 5 drop box on Canvas.
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
Note: Currently, this lesson teaches the Leaflet API. If you are looking for the earlier materials on OpenLayers, see the Lesson 6 archive page [120].
So far, you've been able to create several different types of layers, such as a dynamically-served web map service (WMS) and a static tiled web map. You've previewed these layers in various ways, but have probably come to the conclusion that they're not very useful or easy to share using these preview mechanisms. You will learn how to create more types of layers in future lessons, but, for now, we are going to pause for a week and learn how you can program a web application that combines your layers, creating an easily shareable "front end" for your web maps. You will learn what the Leaflet API is and how to use it for this purpose. It's impossible to learn Leaflet in a single week (or even in a single course), so, in this lesson, you'll just get a taste of the basics. Future lessons continue to use the Leaflet and expand on the knowledge you gain this week.
If you are new to web programming, it is recommended that you take some time to review the W3Schools HTML [121] and JavaScript [122] tutorials that you studied during orientation week. You don't need to know everything, but you do need to be familiar enough with the code and markup patterns that you can interpret most of what you are seeing when you view the examples later in the lesson. The prerequisite for this course is prior familiarity with at least one programming language, and it is expected in this lesson that you apply this familiarity to understanding the JavaScript syntax for loops, functions, decision structures, and so forth. Exerting a little extra time and effort, you should be able to see how JavaScript relates to the language(s) that you already know.
An API (application programming interface) is a framework that you can use to write a program. It provides a set of classes and functions that help you avoid writing all the low-level code to perform specific actions. For example, web mapping APIs typically include classes for maps and layers so that you don't have to write all the low-level code for displaying an interactive map image and drawing a new layer on it. Instead, you can just create a new map object, create a new layer object, and call some method such as layer.addTo(map). The API abstracts the complexity of the task and makes it easy for you to focus on the mapping aspects of your application, rather than spending time on the low-level logistics.
You've probably heard of general purpose APIs such as Java and the Microsoft .NET Framework that can be used to write all kinds of programs on desktop, web, and mobile platforms. There are also more specialized APIs built around certain products and functionalities. For example, you may have heard of Google App Engine, Amazon Web Services, and Microsoft Windows Azure that are designed for proprietary cloud computing environments.
APIs designed specifically for the purpose of making web maps include OpenLayers, Leaflet, the Google Maps API, and the ArcGIS API for JavaScript. The latter two are even more specific in that they are designed around particular proprietary platforms. This lesson introduces some of the different APIs and application development approaches, then gets into detail on how to use the Leaflet API.
Be aware that an API is not a programming language; rather, it is a set of building blocks that you invoke using a language. Some APIs are supported for use with multiple programming languages and other APIs are tied to one specific language. For example, there is both a language and an API named Java. The Java language is used to work with the Java API (and other APIs). In contrast, the .NET Framework is solely an API; there is no language called .NET. Applications using the .NET Framework are typically programmed using the C# or Visual Basic languages.
When you set out to create a web map, one of the most important choices you will make is which API to use. If your application is large in scope with many clients, this one decision can affect your professional activities and trajectory for years. How can you select an API that will be the best fit for your requirements and skill set?
The selection of an API is often tightly coupled with the decision of a platform and programming language. These two factors affect the APIs available to you. For example, if you know that your application is required to run on Android tablets, you first need to decide whether you are going to build a full-fledged native app (in other words, one that is available in Google Play and has access to the device hardware such as the camera) or one that simply runs in a web browser on the Android tablet. Developing a native app most likely means that you'll be using Java, while developing a browser-based app allows more flexibility and can be done with JavaScript and HTML, perhaps employing an API that's designed to be mobile-friendly (in other words, it supports touch gestures, resizes to device width, and so forth).
From this example, you can probably also tell that it's important to consider which languages and platforms your developers are familiar with. If you have people on staff who know Java or Objective C, your options increase for developing native mobile apps. However, knowledge of HTML and JavaScript is usually sufficient to build browser-based apps. Most FOSS APIs for web mapping are geared toward an HTML and JavaScript approach, so this is what we'll focus on in Geog 585.
Below are some examples of FOSS web mapping APIs for building browser-based apps with HTML and JavaScript.
OpenLayers [25] is a mature and richly featured JavaScript API for building web map applications. It has an extensive collection of documentation and samples, although some of the materials can be difficult for beginners to grasp. One of the nicest things about OpenLayers is the large developer community using the API. This community has created a mass of tips and examples on forums such as GIS Stack Exchange [123]. Although OpenLayers is not as approachable for beginners as some of the other APIs, its accumulation of online help resources, and its support for many layer types from both FOSS and commercial sources offer some advantage over other APIs. In 2014, OpenLayers 3 was released which was a major step because it was a complete rewrite of the library not compatible with the OpenLayers 2 branch anymore. In contrast, the current version OpenLayers 7 is still backwards compatible to previous versions of OpenLayers 3. If you feel comfortable with JavaScript and Leaflet after this lesson and decide you would like to use OpenLayers for your final project, you are welcome to go ahead, keeping in mind its strengths and weaknesses.
Leaflet [26] is a younger FOSS web mapping API that is designed to be lightweight, mobile-friendly, and easy to get started with. It has become extremely popular over the last years (one reason why we are now teaching it in this course), and quite a few companies, such as Mapbox, use it as a basis for their own APIs. Leaflet places heavy emphasis on the use of tiled maps and client-side vector graphics drawn from sources such as GeoJSON (you will learn more about the latter in the next lesson). For basic maps that use these layer types, Leaflet is an excellent choice that has already endeared itself to many GIS developers.
Leaflet contains a full API reference but only a handful of full working examples compared to OpenLayers. Going beyond the examples can be tricky for beginners; however, the simplicity of the API lends itself well to learning on the fly.
D3 [28] is a FOSS data visualization library that is frequently used for charting, but also contains many map examples. It binds data elements to the page's document object model (DOM), allowing for interesting and flexible data animations and transitions. Although it has a steeper learning curve for newbies, D3 is a nice option for composing a web app with interactive maps and charts. It also offers examples for using non-Mercator projections [124].
Polymaps [29]is a simple FOSS mapping API primarily designed for mashing up map tiles with vector features drawn from GeoJSON and other sources. However, it seems like it is not under active development anymore, even though we have not seen an official announcement in this regard. Unfortunately, the developer examples on their web site also stopped working recently so that you can currently not see the map examples showing how Polymaps can transform and overlay a raster image [125] onto an existing tile set and demonstrating Polymap's unique ability to generalize a large set of points on the fly using k-means clustering [126].
ModestMaps [27] is a lightweight FOSS API for displaying tiled maps. By design, it lacks a lot of the functionality of the other APIs mentioned above. Running JavaScript code requires transferring that code to the browser. Why load hundreds of functions if you know that you just want to display a map? Unfortunately, similarly to Polymaps, the web map examples on their pages don't seem to work anymore.
Several proprietary web mapping APIs created by commercial software companies have become very popular. In this context, "proprietary" means that the API's source code cannot be downloaded and/or is not permitted to be modified and/or cannot be deployed without paying a royalty.
I include this section on proprietary options because you will hear about them all the time, many are free to use (under various conditions), and some of them will work with the types of layers we're using in this course. Just be aware that proprietary APIs may be oriented toward the purchase of a particular product or service and may cost money if you deploy them for monetary benefit or if they incur enormous amounts of traffic. Always check the API's license agreement before you deploy any application on a server outside your own development machine.
The Google Maps API [127] part of the Google Maps Platform [128] gives developers the opportunity to overlay their own data on top of tiled map layers from Google Maps. The overlaid data is typically supplied through KML files, and is displayed as interactive vector graphics drawn on the client side. These graphics can be restyled by the developer to use custom marker symbols, and can be bound to popups or tables to show additional information on a mouse click.
Perhaps the biggest advantage of the Google Maps API is that it brings the look and feel of Google Maps to an application. Many Internet users have experience with Google Maps and may feel more comfortable when they see the Google Maps navigation control or map style, even when this is embedded in an unfamiliar third-party application. The Google Maps API is arguably no more robust or easier to use than some of the FOSS APIs described above; however, it is thoroughly documented and offers a large developer community.
In July 2018, Google Maps adopted a pay-as-you-go model wherein, all customers get $200/month of credit and must pay fees for service usage beyond this amount. Under their pricing plan at the time of this writing, that is enough to cover 100,000 static map views or 28,500 dynamic map views. See this page [129] for the most up-to-date information on Google's Pricing and Plans.
Microsoft's Bing Maps [130], another large commercial maps provider, offers APIs for web and mobile applications that are similar in scope to Google's. Bing Maps offers a free usage tier along with volume-based pricing for enterprises (see details here [131]). One difference from Google is that the Bing Maps API places less emphasis on KML usage, since Google popularized the KML format and is a primary platform used to create KML files.
The Google and Bing mapping APIs are a popular choices among developers of place-finder applications that display real estate listings [132], businesses [133], churches [134], etc. However, some sites are beginning to adopt FOSS alternatives. For example, Craigslist [135] has adopted a Leaflet + OpenStreetMap approach when showing the results of real estate searches.
Esri has created APIs for building both web apps and native mobile apps, some of which are relatively rich in function compared to the Google Maps API and many of the FOSS APIs. The ArcGIS API for JavaScript [136] is one of the most fully featured and actively developed of these APIs.
The ArcGIS APIs are primarily designed to work with web services that you have published using ArcGIS Online [137] and ArcGIS Enterprise [138] (comprised of ArcGIS Server [139] and Portal for ArcGIS [140]). However, some of the APIs can also display OGC services, KML, and generic tiled map services (such as the one we built with QGIS). One of the more distinguishing advantages of the APIs is their ability to tap into web services originating from ArcToolbox that perform geoprocessing on the server. This is an area where FOSS solutions lack an equivalent GUI experience (see the section on WPS services in Lesson 8).
The APIs are free to use for development or educational use, but require a fee if you are selling the application or embedding advertising within it. Published here are Esri's terms of use [141].
If you are interested in learning the Google Maps API and/or the ArcGIS API for JavaScript, the Penn State course materials for Geog 863 [2] provide an excellent place to get started.
A multitude of other free and proprietary APIs have appeared over the years for doing pretty much the same things as the ones listed above. Some of them, such as Mapbox-GL.js [142] (previously Mapbox.js [143]) and the CARTO Maps API [144], are associated with cloud-based mapping and location services. Please take a detour to read this GIS Stack Exchange post [145] describing available web mapping APIs. You will refer back to the post when you complete this week's assignment.
If you distinguish yourself within your organization as a person who can develop web maps, it's likely you'll eventually be called upon to use more than just one of the APIs described in the previous section. As noted, project circumstances and requirements can force the selection of different APIs for differing endeavors. As a programmer, it's important to understand the general structures, patterns, and architectures behind the APIs and languages you use, so that you can learn new ones on the fly. Technology is always changing, and you will limit your utility if you tie yourself to a single development mode.
The following section describes some patterns and features that are held in common among many (but not all) of the web mapping APIs described earlier. I include this section before diving into Leaflet, so that you get an idea of what things are not unique to Leaflet when we begin looking at the code. However, I also include example snippets to show how the concept is implemented in Leaflet.
Nearly all pages that use web mapping APIs include the following:
Before you can get off the ground, your HTML page needs to include a <script> tag pointing at the web mapping API's JavaScript files. Be aware that the more JavaScript you are referencing, the longer it will take to load your page. Some APIs are slimmer than others (hence a name like ModestMaps) but may offer fewer features. When adopting one of the larger APIs, like OpenLayers, some developers build and reference their own smaller version of the API that contains just the functions they want to offer.
There are a couple of ways to reference the API. One approach is to download and host the API on your own server, thus minimizing load times and allowing you to customize the API. The second approach is to reference the API on someone else's server. Sites called content delivery networks (CDNs) specialize in hosting commonly-referenced APIs. This is what a CDN URL looks like for Leaflet, referenced within a script tag within the page head:
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.js"></script>
In this course, we'll reference Leaflet through the CloudFlare CDN in this manner, for simplicity. This requires you to maintain an Internet connection while you are testing your code. Be aware that if you were developing an internal application, needed full control over the APIs hardware, or needed to customize the API in any way, you would need to download and host the API yourself.
Many web mapping APIs offer some stylesheets that can get you started with making nice-looking applications. You can reference these stylesheets in the form of CSS files, either by hosting them on your own server or using a CDN. You can bring a Leaflet stylesheet into your page from the CloudFlare CDN within the head using syntax like this:
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.css" type="text/css" crossorigin="">
When you want to put a map in your page, you will typically carve out some space using an HTML <div> tag. You then create a map object using the API and relate it to your div.
For example, with Leaflet, you can create a map div like this in the body of your page.
<div id="mapid"></div>
Elsewhere in your page, in your JavaScript code, you can create a Leaftlet Map [146] object and relate it to the div. The L.map constructor takes the div name as an argument (L stands for the Leaflet library here).
var myMap; myMap = new L.map('mapid');
The map contains methods for getting the current layer set, centering the map, zooming to given coordinates, and so forth. In many web mapping APIs, the map is one of the most powerful objects. Whenever you adopt a new mapping API, take a look at the reference documentation for the map object to understand what it can do and the syntax for invoking the most common methods.
Most web mapping APIs provide ways to define layer objects, which you add to the map object one by one to create your mashup. It is very important to note that a layer, in this sense, could represent a web service such as a WMS or tiled service that itself contains many underlying data layers. However, you would only need one layer object to bring this type of web service into your map. Other layer objects that reference single data files, such as KML or GeoJSON, are simpler to conceptualize.
In many web mapping APIs, the layer is an abstract (or "base") class offering a set of common properties, and is exposed to the developer only through more specific classes. Take a look at the methods and events offered in Leaftlet's Layer [147] base class. These include methods to add and remove the layer from the map and manage elements like popups and tooltips. Now look at the properties of some of the more specific derived classes such as TileLayer [148], WMS [149], and GeoJSON [150] to see some of the more specific methods associated with these layer types. You will typically be working with the API help documents at this lower level; however, it is important to remember that you still have all the properties and methods of the Layer class available to you whenever you use any of these specialized layer types.
When you create a new layer in Leaflet, you're generally expected to provide the URL or file path containing the source data for the layer. The layer will not show up on the map until you call its addTo(...) method. Adapted from the Leaflet documentation, here's one way you could create a layer referencing a WMS and add it onto your basemap:
var nexrad = L.tileLayer.wms("http://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0r.cgi", {
layers: 'nexrad-n0r-900913',
format: 'image/png',
transparent: true,
attribution: "Weather data © 2012 IEM Nexrad"
});
nexrad.addTo(myMap);
Don't worry if the meaning of all the parameters above is not immediately apparent. The important thing is to recognize that the layer required you to supply a URL before it could recognize the WMS. You were also able to toggle a transparency option and supply attribution text for the layer.
If you call the addTo(...) method on some other layer, it will be placed on top of the first layer.
Many types of tiled layers include the tile zoom level, row, and column number in each URL. When you create these layers in Leaflet, it would not make sense to supply a specific URL, since this would only point to one tile. Instead, you supply a general format using {x}, {y}, and {z} for the column, row, and zoom levels, respectively. This is sort of like what you did when you added your tiles to the ArcGIS Online map viewer in the previous lesson.
// create and add OpenStreetMap tile layer
var osm = L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
maxZoom: 19,
attribution: '© <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>'
});
osm.addTo(map);
Layers that come already drawn by the server, such as tiled maps and WMS images, already have styling applied, but for layers drawn by the browser such as GeoJSON or GeoRSS, you must define how the layer will be styled. Web mapping APIs typically offer a group of properties that you can set on a layer like this to define how the browser should draw it. These properties include things like fill width, fill color, outline width, outline color, and so forth. In Leaflet, it's common to define styles as functions that can then be referenced in the layer constructor, like so:
function parkStyle(feature) {
return {
fillColor: '#FF00FF',
fillOpacity: 1,
color: '#B04173',
};
}
var gardenLayer = new L.GeoJSON.AJAX('gardens.geojson', {
style: parkStyle
});
gardenLayer.addTo(map);
The style function returns a set of style properties (fillColor, fillOpacity, and color in this case) that will be applied when drawing the features from the layer. In this simple case, all features will look exactly the same. However, since the style function has the feature to be drawn as a parameter (function parameter feature), we can use properties of the feature to calculate its style property values. e.g., to have all features colored differently based on one of their attributes. You will learn how to do this later on in this course.
Many APIs allow you to use a custom image for a marker, rather than placing a simple dot. The Leaflet example below shows how you could add a GeoJSON layer of grocery stores to your map and style it with a shopping cart icon saved in an SVG (scalable vector graphics) file called grocery.svg. This could just as easily be a PNG file or other rasterized image type.
// create icons for supermarkets
var groceryIcon = L.icon({
iconUrl: 'grocery.svg',
iconSize: [20,20]
});
// create a vector layer of grocery stores from GeoJSON data
var groceryLayer = new L.GeoJSON.AJAX('supermarkets.geojson',{
pointToLayer: function (feature, latlng) {
return L.marker(latlng, {icon: groceryIcon});
}
});
groceryLayer.addTo(map);
The result might look something like this when put on top of a basemap layer:
Don't worry if all the code above is not completely clear to you yet. Just make sure you can see where the style is being defined via a JavaScript object. In Lesson 7, you will get some more exposure to GeoJSON and styling browser-drawn graphics.
Web mapping APIs offer interactive elements that help your map become more than just a static picture on an HTML page. The map and layer objects described above typically allow you to run code in response to certain user actions, such as clicking the mouse. The user action is called an event (or firing an event). The code you run in response is called the event handler, and it typically goes into its own function block of code. Sometimes the event can supply arguments to the handler function via an "event object" containing properties of the event, such as the screen coordinates of the mouse click that fired the event.
For example, you might instruct the map to "listen" for a mouse click event. You could then write a handler function that takes the screen coordinates of the event argument (in other words, the coordinates of the clicked pixel) and converts them to map coordinates, then writes the coordinates into a label in the HTML page so they can be seen by the user. An even greater measure of interactivity could be achieved by wiring up this handler function to a hover event, instead of a click. You would then get the effect of always seeing the coordinates of your mouse as you moved it around the screen.
Web map users often want to obtain more detailed information about specific features in the map. It is very common to handle a click event by showing a popup window with more information about the clicked feature; so common, in fact, that many web APIs have special classes and methods for popups that don't require you to write the typical amount of event listening logic. You will learn more about popups in Leaflet in future lessons in this course.
Sometimes popups are too limited in space or complexity for the content you want to show. Even if your mapping API allows you to cram large images or tabbed interfaces into a popup, it often makes more sense to show this type of content in an HTML div elsewhere on the page. Web mapping APIs allow you to perform queries on a clicked point and retrieve attribute information from the clicked features (often as part of a handler function like the ones mentioned above). You can then do anything you want with this handler function in order to display the information in HTML. You might even pass the information to a different specialized API that draws charts, queries Wikipedia, finds nearby homes for sale, and so forth.
Another common piece of desired interactivity is the ability to switch layers off and on. Remember that in this sense, a "layer" is an entire web service. It is obviously not possible to toggle the visibility of individual sublayers inside a tiled map because all the layers are "burned into" the tiled image. However, you could switch to a different tiled base map, or turn off a WMS or GeoJSON layer placed on top of it.
Leaflet offers a Layers Control [151] that acts like a table of contents for toggling visibility. You create your layers as usual, but instead of adding them to the map individually, you organize them in JavaScript objects for your base layers and overlays (thematic or business layers). Suppose you have two layer variables, grayscale and streets, representing tiled maps, and one layer variable cities representing a layer of city points. Again, from the Leaflet doc, here is how you can add the layer switching control with these defined as base layers and overlays:
var baseLayers = {
"Grayscale": grayscale,
"Streets": streets
};
var overlays = {
"Cities": cities
};
L.control.layers(baseLayers, overlays).addTo(map);
Now that you are familiar with the different elements of a web mapping API and have seen how these are expressed in Leaflet, we'll move forward and take a look at some fully functioning examples.
Now that you have received an introduction to web mapping APIs and their basic elements, you need to start experimenting with one in order to make any more significant progress. The way you will learn Leaflet, and any other API, is by taking a look at its developer code samples, trying to run them, and experimenting with adjustments or tweaks to suit your fancy. Indeed, a strategy for building any web application is to find the combination of samples that best approximates what you want to do and then start experimenting with merging those samples, testing the functionality one piece at a time until you've met all your requirements.
In this spirit of exploration and experimentation, we are going to take some time to look through some of the Leaflet developer samples and other online supplementary materials such as developer forum posts. These are the kinds of resources that you'll be working with as you complete your final project and continue Leaflet development beyond this course. I want to make sure you can use and interpret them effectively.
In the next section of the lessons, you'll complete a walkthrough where you put your PhillyBasemap tiles that you made with QGIS into the map, and then place a WMS on top of them. The user should be able to click a feature in the WMS and see a popup with basic information about that feature. The completed walkthrough will look something like this:
A good strategy when you approach a coding problem like this is to make a basic list of the things you'll need to figure out how to do. Then you can hunt for examples that show how to accomplish those tasks. In the above scenario, I can think of four key things you need to know how to do in order to get this to work.
Now take a look through some of the following Leaflet examples and developer forum posts that could help you achieve this goal. Don't worry about writing the code yet; that will be covered in the walkthrough. For now, just get familiar with the code structure and patterns. Then the material will be somewhat familiar when it's time to complete the walkthrough.
This is such a common task that you could go to the Leaflet Quick Start Guide [152] example. Indeed, this shows how to add a tiled basemap from Mapbox, using the L.tileLayer class:
L.tileLayer('https://api.tiles.mapbox.com/v4/{id}/{z}/{x}/{y}.png?access_token={accessToken}', ...)
This is even more complex than you need, though, because the sample requires you to have a Mapbox access token and project ID. If you're just adding tiles hosted from your own server, you could adapt the simple code snippet from the L.tileLayer [153] API reference:
L.tileLayer('http://{s}.somedomain.com/{foo}/{z}/{x}/{y}.png', {foo: 'bar'});
In this situation, foo just represents additional options you can apply to the tile layer. The important part is that you pass in a URL to your tiles with {x}, {y}, and {z} representing the column, row, and zoom level, respectively.
If you Google OpenStreetMap in Leaflet example, you'll find helpful sample code on the Getting Started with Leaflet [154] page that adds OpenStreetMap as a tiled basemap.
// set up the map map = new L.Map('map'); // create the tile layer with correct attribution var osmUrl='http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png'; var osmAttrib='Map data © <a href="http://openstreetmap.org">OpenStreetMap</a> contributors'; var osm = new L.TileLayer(osmUrl, {minZoom: 8, maxZoom: 12, attribution: osmAttrib}); // start the map in South-East England map.setView(new L.LatLng(51.3, 0.7),9); map.addLayer(osm);
The Leaflet examples contain a page about working with WMS in Leaflet. From the Leaflet WMS and TMS tutorial [155]:
var
wmsLayer = L.tileLayer.wms('https://demo.boundlessgeo.com/geoserver/ows?', { layers: 'nasa:bluemarble' }).addTo(map);
The above example shows how the L.tileLayer.wms [149] class can be used together with a URL and a list of layer names to add a WMS to the Leaflet map.
This one is a little harder. A WMS query is based on a GetFeatureInfo request, and Leaflet doesn't provide any kind of method for doing that. In contrast, Leaflet is more focused on working with vector data drawn by the browser from sources like GeoJSON. Its support for WMS layers is limited to display only.
So how do you query a WMS using Leaflet? It's time to head over to Google and try WMS GetFeatureInfo Leaflet. A couple of resources in the results are useful. First, a GitHub page by Ryan Clark [156] shares some JavaScript functions he wrote for doing the GetFeatureInfo request directly through the web via jQuery and AJAX. I'll describe this more thoroughly in the walkthrough, but basically, he constructs a JavaScript string variable of the URL he needs in order to do the request, then he sends the request to the web using a method from the jQuery open source JavaScript helper library. Paul Crickard III wrote a simplified implementation [157] of Clark's technique on his blog that forms the basis for the code in the walkthrough.
Both examples listen for a mouse click event on the map to trigger the query, although this is coded in slightly different ways. Crickard's pattern is sufficient for our purposes, with onMapClick being a function that runs after the map detects a click event:
map.addEventListener('click', onMapClick); function onMapClick(e){ . . . }
Please take a few minutes to review both these resources and do your best to see what's going on. You likely won't understand all the code, but pay attention to the general patterns of setting up a URL, making a request, and working with the result.
Both Clark's and Crickard's GetFeatureInfo examples above display elements of the query response inside a popup. Both use the L.Popup [158] class for this, but Crickard's example is a bit easier to follow for beginners. He does:
popup = new L.Popup({maxWidth: 400});
Then later he sets an anchor point for the popup and the content that should appear inside.
popup.setLatLng(e.latlng); popup.setContent("<iframe src='"+URL+"' width='300' height='100' frameborder='0'></iframe>"); map.openPopup(popup);
Notice that HTML can be used for the content. You can parse the WMS GetFeatureInfo response and pull out the elements you need and format them into an HTML string to act as the content.
It's natural to ask, "How do you expect me to write a bunch of code like this from scratch when I'm just barely getting familiar with JavaScript?!" The answer is that you don't have to write code like this from scratch. You only need to know enough to interpret and adapt a working example. Using the documentation and examining the code line by line, you should be able to get at least a basic idea of what is happening. If there's a parameter you don't understand, you can zero in on it. But don't worry about grasping everything.
Experienced developers sometimes say that the way to learn an API is to "play with it." This can drive beginners insane. Play with what? How do I even know what to do? In this lesson, you have hopefully learned to pick out simple code examples that are most relevant to what you want to do, then examine them line by line and see if you can tell what each line of the code is doing. Then make minor adjustments to the code to fit it to your own data and scenario. (This is the "playing with it" part.) Let's go to the walkthrough to see how the above examples could be adapted to some of your own web map services.
The goal of this walkthrough is to get some practice overlaying different kinds of web services in Leaflet. You will first publish a WMS showing farmers' markets in Philadelphia. You will then use Leaflet to place this layer on top of the Philadelphia basemap tiles you made with QGIS in the previous lesson. You'll also add code, so that a user of your application can click any farmers market and see some more information in a popup.
The first step is setting up a (passably) good-looking WMS showing farmers' markets in Philadelphia. In this application, the farmers' markets WMS will play the role of the business layer.
Setting up this WMS will be a nice review of some of the skills you learned in Lesson 4. In places where step-by-step instructions are lacking, you should be able to go back to the Lesson 4 walkthrough to remember the procedures.
<Label> <ogc:PropertyName>NAME</ogc:PropertyName> </Label>
Now you'll make a few preparations for writing a simple Leaflet application. We’re going to host this app on the mini web server called Jetty that is installed with GeoServer. Just remember that in "the real world" you would probably get your IT staff to install an enterprise grade web server, such as Apache where you could run both GeoServer and your HTML pages. If you don't have an IT staff, you may even be lucky enough to do this yourself someday! :-)
#mapid { width: 512px; height: 512px; border: 1px solid #ccc; } .leaflet-container { background: #fff; }
The above CSS sets the size of the map container, the size and color of the map border, and the color of the background.
Now you'll create an HTML page and insert the JavaScript code that configures the map and popups. The steps below do not provide the code linearly; you are expected to insert the code in the correct places as given in the instructions. Code is hierarchical, in the sense that some blocks run within others. It's more intuitive to describe the blocks of code rather than to give the code in its exact sequence. If you get confused about where the code is supposed to go, refer to the full example code at the end of this walkthrough page.
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>Farmers markets in Philadelphia</title> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.css" type="text/css" crossorigin=""> <script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.js" crossorigin=""></script> <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script> <link rel="stylesheet" href="style.css" type="text/css"> <script type="text/javascript"> </script> </head> <body onload="init()"> <h1 id="title">Farmers markets in Philadelphia</h1> <div id="mapid"> </div> <div id="docs"> <p>This page shows farmers markets in Philadelphia, Pennsylvania. Click a market to get more information.</p> </div> </body> </html>
The above code contains the HTML head and body. This should give you a shell of a page, although you won't see the map show up yet.
In the head, notice the references to the Leaflet JavaScript and CSS files, both running on the CloudFlare CDN. There's also a reference to the style.css that you placed in your web folder earlier, which in some cases overrides the Leaflet stylesheet. Finally, there is a reference to the jQuery JavaScript library, which is running on Google’s CDN. jQuery is a helper library that greatly simplifies some JavaScript tasks. We’re going to use it here to make a web request to query the WMS, since there’s no easy way to do a WMS GetFeatureInfo request in Leaflet.
In the body, notice there is a div element with id "mapid" that is intended to hold our Leaflet web map. The CSS style for this element is defined in style.css and it sets the width and the height of the map in pixels. If you wanted to change the map width and height, you would modify the CSS.
When the body of the page has loaded, it runs the init() function from the JavaScript code we’ll add later. That’s why you see <body onload="init()">.
From this point on, we will not make any changes to the head or body. We will be inserting JavaScript logic in the <script> tag.
var map; function init() { // create map and set center and zoom level map = new L.map('mapid'); map.setView([39.9526,-75.1652],13); . . . }
This sets up a global variable named map that can be used throughout your JavaScript code. Then an initialization function called init() is defined, which you’ll recall is the function you set to run when the page body loads. The first thing the function does is instantiate a Leaflet map object within your mapid div. It then centers the map at 39.9526 N 75.1652 W at zoom level 13.
In this walkthrough, you will insert the remaining JavaScript inside of the init() function where you see the . . . above.
// create tile layer and add it to map var tiles = L.tileLayer('https://<URL of your S3 bucket>/PhillyBasemap/{z}/{x}/{y}.png'); tiles.addTo(map);
The above code creates a Leaflet tile layer that references your PhillyBasemap that you made with QGIS in Lesson 5. In the code above, you must modify the URL to use your S3 bucket name as created in the L5 walkthrough. The final URL should look something to this:
https://geog585-somestring.s3.someregion.amazonaws.com/PhillyBasemap/{z}/{x}/{y}.png
The URL to the tiles is provided in a generic format where z means zoom level number, x means column number, and y means row number. As long as you give Leaflet this tile URL structure, it will be smart enough to request the correct tiles as you pan and zoom around the map.
Notice how the tileLayer.addTo() method is used to add the layer to the map. The map object is passed in as a parameter. In some other mapping APIs like OpenLayers you call an add method on the map itself and provide the layer as a parameter.
// create wms layer var farmerMarkets = L.tileLayer.wms('http://localhost:8080/geoserver/geog585/wms', { layers: 'geog585:FarmersMarkets', format: 'image/png', transparent: true }); farmerMarkets.addTo(map); . . .
This adds the WMS layer of the farmers markets to the map. Notice how Leaflet’s wms class is used for this. You give it some properties such as the URL, layers, and image format you want (all using WMS-friendly syntax), and Leaflet takes care of formatting and sending the GetMap requests and displaying the responses as the user zooms and pans around the map.
So far, creating the map and adding the layers were pretty simple. It’s clicking the WMS and seeing a popup that turns out to be more complex. In order to do this, you have to send a WMS GetFeatureInfo request to the server. Leaflet offers no options for this, so you have to do it yourself by constructing a URL, sending it to the server, and reading the response. So how do you make a web request like this using JavaScript?
One way is by using jQuery, an open source API designed for simplifying things that JavaScript programmers have to do day in and day out. One of these tasks is sending web requests to a server while the end user is interacting with a web page (ie, clicking your map). The request is sent using a technique called AJAX (Asynchronous JavaScript and XML) which avoids a total page refresh.
With this in mind, let’s write an Identify function that we can fire off whenever someone clicks the map. This function will construct a WMS GetFeatureInfo request, send the request to the server using AJAX, and put the response into a popup.
// define event handler function for click events and register it function Identify(e) { . . . } map.addEventListener('click', Identify);
Above is the Identify function that runs whenever you fire off a mouse event. I haven’t provided this entire function yet because it is long; however, I have inserted a … so you can see that together with the definition of this function, an “event listener” is added to the map to keep on the alert for any mouse clicks that occur. If someone clicks the map, the Identify function will be fired and a Leaflet event object [164] called ‘e’ containing some properties of the clicked point will be brought into the function.
// set parameters needed for GetFeatureInfo WMS request var sw = map.options.crs.project(map.getBounds().getSouthWest()); var ne = map.options.crs.project(map.getBounds().getNorthEast()); var BBOX = sw.x + "," + sw.y + "," + ne.x + "," + ne.y; var WIDTH = map.getSize().x; var HEIGHT = map.getSize().y; . . .
Some key things we need in order to construct a GetFeatureInfo request are the bounding coordinates of the map, the width of the map in pixels, and the height of the map in pixels. The Leaflet map object provides ways of getting those properties. That’s what’s happening in the code above. The southwest and northeast corner coordinates of the map are retrieved, and these are formatted into a comma-delimited string in the syntax required by the BBOX parameter. Then the width and height are also retrieved from the Leaflet map.
var X = Math.trunc(map.layerPointToContainerPoint(e.layerPoint).x); var Y = Math.trunc(map.layerPointToContainerPoint(e.layerPoint).y); // compose the URL for the request var URL = 'http://localhost:8080/geoserver/geog585/wms?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetFeatureInfo&LAYERS=geog585:FarmersMarkets&QUERY_LAYERS=geog585:FarmersMarkets&BBOX='+BBOX+'&FEATURE_COUNT=1&HEIGHT='+HEIGHT+'&WIDTH='+WIDTH+'&INFO_FORMAT=application%2Fjson&TILED=false&CRS=EPSG%3A3857&I='+X+'&J='+Y; . . .
The purpose of this code is to figure out the clicked point and construct the request URL. Although the lines above seem a bit complex, they are essentially getting the row and column of the clicked pixel from the event object (named ‘e’) generated by the mouse click. A URL is then constructed for the GetFeatureInfo request, plugging in the values we derived above for the BBOX, WIDTH, HEIGHT, I, and J parameters.
Be aware that if you named your WMS something other than FarmersMarkets, put it in a workspace other than geog585, or placed it in a folder other than geog585, you will need to modify the URL in the above code.
Now let’s add some code to send this request.
//send GetFeatureInfo as asynchronous HTTP request using jQuery $.ajax $.ajax({ url: URL, dataType: "json", type: "GET", success: function(data) { if(data.features.length !== 0) { // at least one feature returned in response var returnedFeature = data.features[0]; // first feature from response // Set up popup for clicked feature and open it var popup = new L.Popup({ maxWidth: 300 }); popup.setContent("<b>" + returnedFeature.properties.NAME + "</b><br />" + returnedFeature.properties.ADDRESS); popup.setLatLng(e.latlng); map.openPopup(popup); } } });
The above is a function that uses jQuery to make a GetFeatureInfo request using AJAX. Typically when you see $. in a piece of JavaScript code, it means a jQuery function is being invoked. Remember we brought in a reference to the jQuery library at the top of our page, allowing us to use these functions.
To make the web request, the AJAX function needs a number of options passed in, including the URL, the dataType, the type of request, etc. Another important thing is what to do with the response; therefore, we define a little function to handle the response data if the request was successful. First of all, this function checks if a feature came back, because someone could very feasibly click an empty area of the map and get no features in return. Any returned features are provided in an array, and the first feature in that array is referenced above using the variable returnedFeature. For simplicity here, we don’t handle cases where multiple features were returned.
The next order of business is to examine returnedFeature and construct a popup window using its properties. A new Leaflet popup balloon is created using the L.popup [158] class. It is then populated with a little piece of HTML constructed from some of the properties of returnedFeature, namely the NAME and ADDRESS fields of the selected farmers market.
The popup needs to be “anchored” to the map somewhere, therefore the mouse click event object ‘e’ is referenced again to construct the anchor point. A final line of code then opens the popup.
Figure 6.7 Final web page after completing the walkthrough
If you don't get the expected result at the end of the walkthrough, please verify the following:
Below is the code used in this walkthrough from start to finish (but keep in mind that the URL for the basemap tiles needs to be adapted to refer to your S3 bucket). This should help you get some context of where each block should be placed. If you’re curious how the code would look in a different API, you can download an OpenLayers 3 example [166] here.
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>Farmers markets in Philadelphia</title> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.css" type="text/css" crossorigin=""> <script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.js"></script> <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script> <link rel="stylesheet" href="style.css" type="text/css"> <script type="text/javascript"> var map; function init() { // create map and set center and zoom level map = new L.map('mapid'); map.setView([39.9526,-75.1652],13); // create tile layer and add it to map var tiles = L.tileLayer('https://<URL of your S3 bucket>/PhillyBasemap/{z}/{x}/{y}.png'); tiles.addTo(map); // create wms layer var farmerMarkets = L.tileLayer.wms('http://localhost:8080/geoserver/geog585/wms', { layers: 'geog585:FarmersMarkets', format: 'image/png', transparent: true }); farmerMarkets.addTo(map); // define event handler function for click events and register it function Identify(e) { // set parameters needed for GetFeatureInfo WMS request var sw = map.options.crs.project(map.getBounds().getSouthWest()); var ne = map.options.crs.project(map.getBounds().getNorthEast()); var BBOX = sw.x + "," + sw.y + "," + ne.x + "," + ne.y; var WIDTH = map.getSize().x; var HEIGHT = map.getSize().y; var X = Math.trunc(map.layerPointToContainerPoint(e.layerPoint).x); var Y = Math.trunc(map.layerPointToContainerPoint(e.layerPoint).y); // compose the URL for the request var URL = 'http://localhost:8080/geoserver/geog585/wms?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetFeatureInfo&LAYERS=geog585:FarmersMarkets&QUERY_LAYERS=geog585:FarmersMarkets&BBOX='+BBOX+'&FEATURE_COUNT=1&HEIGHT='+HEIGHT+'&WIDTH='+WIDTH+'&INFO_FORMAT=application%2Fjson&TILED=false&CRS=EPSG%3A3857&I='+X+'&J='+Y; //send GetFeatureInfo as asynchronous HTTP request using jQuery $.ajax $.ajax({ url: URL, dataType: "json", type: "GET", success: function(data) { if(data.features.length !== 0) { // at least one feature returned in response var returnedFeature = data.features[0]; // first feature from response // Set up popup for clicked feature and open it var popup = new L.Popup({ maxWidth: 300 }); popup.setContent("<b>" + returnedFeature.properties.NAME + "</b><br />" + returnedFeature.properties.ADDRESS); popup.setLatLng(e.latlng); map.openPopup(popup); } } }); } map.addEventListener('click', Identify); } </script> </head> <body onload="init()"> <h1 id="title">Farmers markets in Philadelphia</h1> <div id="mapid"> </div> <div id="docs"> <p>This page shows farmers markets in Philadelphia, Pennsylvania. Click a market to get more information.</p> </div> </body> </html>
This week's assignment has two parts:
Zip these deliverables into a single folder and place them in the Lesson 6 assignment drop box on Canvas.
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
Note: Currently, this lesson teaches the Leaflet API. If you are looking for the previous materials on OpenLayers, see the Lesson 7 archive page [169].
Last week, you used a server-drawn image from a WMS to display your business layer (the farmers markets). This week, you'll learn about an alternative way to display your business layers on the map; namely, sending the raw data to the client (such as a web browser) to be drawn. This is a common technique in many web maps that can provide for much interactivity and potentially faster performance if used appropriately. You may have even been exposed to this practice when you reviewed a web map for your assignment last week.
This lesson describes two common formats used for sending vector GIS data to the browser (KML and GeoJSON) and shows how you can add these types of layers in Leaflet.
Up to this point, your web maps have displayed images obtained from the server: either through pre-drawn tiles or dynamically-drawn WMS maps. An alternative approach is to send some text to the client containing the coordinates and attributes of features, then allow the client to draw the layer. This approach can improve the speed and interactivity of your web map when used wisely.
You may be asking, "How does a client like a web browser know how to draw GIS data?" Although web browsers don't "know" anything about GIS, they indeed have the ability to draw vector graphics, which is just the simple process of connecting screen coordinates with a symbol. Web mapping APIs can read a series of coordinates provided in a GeoJSON file, or a KML file, to give a few examples, and convert these into screen coordinates to be drawn by the browser.
Speed and interactivity are two principal advantages of drawing vector features on the client rather than the server. Once your web browser has read the vector data from the server, users of the web map can interact with the data in a lightning fast manner. Let's suppose I have a web map of the United States with all the NFL football teams displayed on it. The basemap is coming from an OpenStreetMap tiled layer. The football teams are coming from a GeoJSON file. When the map loads, all the GeoJSON for the teams is read by my web browser. This includes all geographic coordinates and the attribute information for each team. Now I can click on any team and see its information without making another request to the server. Contrast this with the farmers markets mashup from the previous lesson, which required a WMS GetFeatureInfo query to the server (and the associated wait time) any time you clicked a market.
Let's suppose I want to highlight a map symbol whenever a user hovers over it, thereby providing a visual cue that someone can click the symbol. A web browser can change the symbol styling quickly enough that this effect works. In contrast, if a round trip to the server were required on every hover event, my app (and possibly my server) could easily grind to a standstill.
Not all use cases are appropriate for drawing vectors on the client side. If you're displaying hundreds of features at a time, or some complex polygons with many vertices, you're probably better off asking the server to draw the map and send it to you. Browsers can get weighed down to a crawl if handed too many vector graphics to draw at once. Sending a lot of complex graphics also results in more network traffic, as all the coordinates must be downloaded by the client.
To keep performance crisp, it's a good idea to generalize the layers that you draw on the client as much as possible, at least at the smallest map scales. For example, when displaying the United States at the nationwide level, you should not be using a file that contains every tiny coastal island in the state of Maine. You might switch to loading a more detailed file when the user zooms in past a particular scale.
Labeling can also be a challenge with browser-drawn graphics. Although web browsers can draw text on the screen at a given coordinate, they have no label placement algorithms like the ones employed by GeoServer, QGIS or TileMill. Your labels will likely overlap each other. You are better off allowing the user to discover the label through interactivity, displaying the label in a popup or HTML DIV when someone clicks a feature.
Finally, the symbol choices offered by web browsers are relatively elementary. You can always instruct the browser to draw a graphic, such as an SVG file, but you won't be able to get some of the complex lines and fills available in programs like QGIS or ArcGIS. Of course, if your client happens to be a desktop application such as QGIS, you don't have to worry as much about available symbol choices.
Web mapping APIs typically provide classes for drawing vector layers with the browser; however, these go by different names with each API. For a few simple standalone vectors, you will often see a "marker" class. For more complex layers, you may see something like FeatureGroup (Leaflet) or FeatureLayer (Esri). OpenLayers has a Layer.Markers class and a Layer.Vector class for these respective purposes.
Desktop programs such as QGIS can view KML, GeoJSON, GML (described in Lesson 8), and various other text-based vector data formats.
When you define a vector layer, you need to specify the approach that your client will take for retrieving the data from the server. Remember that your client is not requesting map images from the server, but it does need to retrieve the coordinates of the vectors and any associated attribute information. Some popular approaches include:
You may see other variations on the above approaches, including the re-loading of layers at fixed intervals to represent an ever-changing phenomenon, such as the position of a fleet of ships.
KML (Keyhole Markup Language) is a popular format for vector GIS features due to its association with Google Earth and Google Maps. KML is just XML formatted according to an open specification, previously maintained by Google, but now enshrined in an OGC standard. Although KML can define the placement of raster layers, we will focus on vector KML in this lesson.
The key XML tag behind KML is the placemark. This defines a geographic feature, a symbol, and extra information that can appear in a popup. You can see some placemarks if you save the example KML file [173] and open it in a text editor. This isn't the cleanest file, but it will do for the purposes of seeing a placemark:
<Placemark> <name>Sundial, Plymouth, Devon, UK</name> <description><![CDATA[The gnonom is 27 feet high, the pool has 21 feet diameter. It was designed by architect Carole Vincent from Boscastle in Cornwall and was unveiled by Her Majesty the Queen on Friday, July 22nd, 1988 at a cost of cost £70,000. The sundial runs one hour and seventeen minutes behind local clocks. <img src="http://www.photoready.co.uk/people-life/images/sundial-fountain.jpg"> Image source:<a href="www.photoready.co.uk</a>]]> </description> <LookAt> <longitude>-4.142398271107962</longitude> <latitude>50.37145390235462</latitude> <altitude>0</altitude> <range>63.33410419881957</range> <tilt>0</tilt> <heading>-0.0001034131369701296</heading> </LookAt> <styleUrl>#msn_sunny_copy69</styleUrl> <Point> <coordinates>-4.142446411782089,50.37160252809223,0</coordinates> </Point> </Placemark>
This particular placemark has a single coordinate, contained in the Point tag. For polylines and polygons, the LineString and Polygon tags are used, respectively, although these do not appear in the above example.
Notice that the Description tag can contain HTML, which gives you more control over formatting popups. The full KML file is much longer than the snippet above, as it contains many points and descriptions.
Leaflet doesn't offer a way to read KML directly. This is an area where OpenLayers holds an advantage. However, Mapbox has produced a free Leaflet plugin called Omnivore that makes it fairly simple to read in vector file types, including KML. First, you need to put a reference to Omnivore in a script tag at the top of your page. You could reference it from a CDN like this:
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet-omnivore/0.3.4/leaflet-omnivore.js"></script>
Then you can reference the KML in a layer, like this:
var runLayer = omnivore.kml('sundials.kml') .on('ready', function() { map.fitBounds(runLayer.getBounds()); runLayer.eachLayer(function(layer) { layer.bindPopup(layer.feature.properties.description); }); }) .addTo(map);
One thing to be aware of is that Omnivore converts the KML to the GeoJSON format before displaying it; (see the next section of the lesson for more info on GeoJSON). Thus, your map may not be able to show all the styling that was originally defined in the KML. If your KML contains some kind of custom picture symbol for the points, you'll need to write Leaflet code to apply that picture to the markers. Notice, however, that the above code does bring in the KML description and applies that text in a popup. This is accomplished using the bindPopup method on the layer.
Again, don't worry about memorizing all the syntax. In most scenarios, you should just be able to tweak the above example to connect to your own KML.
GeoJSON is a widely-used data format for displaying vectors in web maps. It is based on JavaScript object notation, a simple and minimalist format for expressing data structures using syntax from JavaScript. In GeoJSON, a vector feature and its attributes are represented as a JavaScript object, allowing for easy parsing of the geometry and fields.
GeoJSON is less bulky than XML-based structures such as KML; however, GeoJSON does not always contain styling information like KML does. You must define the styling on the client, which in your case means writing JavaScript code or taking the Leaflet's default styling. This is covered in the next section of the lesson.
GeoJSON's simplicity and loading speed have made it popular, perhaps even trendy, among developers in the FOSS world. For example, in a tongue-in-cheek Internet poll, GIS practitioners recently voted "The answer is always GeoJSON" as the most likely attribute to define a 'GeoHipster' [174].
Here's what a piece of GeoJSON looks like. GeoJSON vectors are commonly bundled into a unit called a FeatureCollection. The FeatureCollection below holds just one feature (the state of Montana) but could hold other features. The bulk of the GeoJSON below contains the vertices that define the state outline, but you should also notice a few attributes, such as "fips" and "name":
{"type":"FeatureCollection","features":[{"type":"Feature","id":"USA-MT","properties":{"fips":"30","name":"Montana"},"geometry":{"type":"Polygon","coordinates":[[[-104.047534,49.000239],[-104.042057,47.861036],[-104.047534,45.944106],[-104.042057,44.996596],[-104.058488,44.996596],[-105.91517,45.002073],[-109.080842,45.002073],[-111.05254,45.002073],[-111.047063,44.476286],[-111.227803,44.580348],[-111.386634,44.75561],[-111.616665,44.547487],[-111.819312,44.509148],[-111.868605,44.563917],[-112.104113,44.520102],[-112.241036,44.569394],[-112.471068,44.481763],[-112.783254,44.48724],[-112.887315,44.394132],[-113.002331,44.448902],[-113.133778,44.772041],[-113.341901,44.782995],[-113.456917,44.865149],[-113.45144,45.056842],[-113.571933,45.128042],[-113.736241,45.330689],[-113.834826,45.522382],[-113.807441,45.604536],[-113.98818,45.703121],[-114.086765,45.593582],[-114.333228,45.456659],[-114.546828,45.560721],[-114.497536,45.670259],[-114.568736,45.774321],[-114.387997,45.88386],[-114.492059,46.037214],[-114.464674,46.272723],[-114.322274,46.645155],[-114.612552,46.639678],[-114.623506,46.705401],[-114.886399,46.809463],[-114.930214,46.919002],[-115.302646,47.187372],[-115.324554,47.258572],[-115.527201,47.302388],[-115.718894,47.42288],[-115.724371,47.696727],[-116.04751,47.976051],[-116.04751,49.000239],[-111.50165,48.994762],[-109.453274,49.000239],[-104.047534,49.000239]]]}}]}
In the GeoJSON above, notice the use of several JavaScript objects embedded within one another. At the lowest level, you have a Polygon object. The Polygon object is contained within a Feature object. The feature is part of a FeatureCollection object. The GeoJSON specification [62] (originally published in 2008 but replaced by a new specification in 2016) gives precise details about how these objects are to be structured. It's important to be familiar with these structures, although you will rarely have to read or write them directly. You will typically use convenience classes or converter programs that have been developed to simplify the experience of working with GeoJSON.
You can use GeoJSON within your main JavaScript code file, but, to keep your things looking simple, it's most common to maintain the GeoJSON in its own separate file. You then reference this file from the appropriate place in your code. With Leaflet you have to define the GeoJSON as a JavaScript variable (seen here) [175] using syntax such as
var <yourVariableName> = <yourGeoJSON>;
You would save this text in a file with a .js extension. For example, I could create a file titled myfeatures.js containing something like the following:
var myGardenJson = {"type":"FeatureCollection","features":[{"type":"Feature","id":"USA-MT","properties":{"fips":"30","name":"Montana"},"geometry":{"type":"Polygon","coordinates":[[[-104.047534,49.000239], . . . ]]}}]};
Then at the top of my HTML page, I need to put a reference to this file:
<script src="myfeatures.js"></script>
This allows me to reference the variable myGardenJson within my JavaScript code. Making a GeoJSON layer in Leaflet then becomes very simple:
var geojsonLayer = L.geoJSON(myGardenJson); geojsonLayer.addTo(map);
It is important to note that L.geoJSON(…) expects all coordinates in the GeoJSON file to be WGS84 coordinates. That is why all GeoJSON files used in this lesson and Lesson 8 will use EPSG:4326. If instead you want to directly work with a GeoJSON file that uses a different projection (so without first reprojecting it to EPSG:4326), you can use the Proj4js [176] Javascript library together with the Proj4Leaflet [177] extension to enable support for coordinate reference systems not built into Leaflet. To do so, you would first have to add the lines to load the respective Javascript libraries at the beginning of the html file
<script src="https://cdnjs.cloudflare.com/ajax/libs/proj4js/2.4.3/proj4.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/proj4leaflet/1.0.1/proj4leaflet.min.js"></script>
and then create the layer with
var geojsonLayer = L.Proj.geoJson(myGardenJson);
to have the code read what projection the GeoJSON file is in and then reproject the data accordingly on the fly. Leaflet, by the way, internally uses pixel coordinates for all data, so it understands locations with respect to the coordinate system of the map pane. That means that there is always a transformation process involved when loading vector data, independent of what projection the data comes in.
You can save any vector layer in QGIS as GeoJSON, and most web mapping APIs offer easy-to-use classes for GeoJSON as a vector display format. In the proprietary software realm, Esri has dragged its feet on GeoJSON support, offering its own JSON-based geometry formats [178] as part of the GeoServices REST Specification and the ArcGIS REST API; however, Esri has informally shared an open source JavaScript library [179]to convert between the two formats, which is now deprecated and replaced by their Terraformer tool [180].
The GeoJSON specification is not an OGC specification. At the time of this writing, OGC conspicuously lacks a JSON-based specification for defining vector GIS objects (OGC publishes the XML-based GML specification for vectors). This lack of an OGC-endorsed JSON specification played a role in the FOSS community's 2013 debate about whether the OGC should adopt the Esri-generated GeoServices REST Specification. The specification would have given OGC a JSON-based GIS data format, but some were wary of the format's association with a proprietary software company. The geoMusings "OGC Abandons the Web" [181] blog post gives one FOSS geo-developer's opinion on the episode (read the comments, too).
When you bring in vector datasets to be drawn by the browser, you may need to do some work to define the symbol that will draw the feature. With data formats like GeoJSON that don't contain any styling information, you don't get ready-made styling like you would get if you just requested an image from the server. If you don't define a symbol for your graphics, you'll probably see your API's default symbol.
In Leaflet, it's common to define styles within functions. This makes them easy to re-use. Here's an example of how you could set up a style and apply it to a GeoJSON layer. This example shows urban gardens as polygons:
// Set up style for garden polygons function gardenStyle(feature) { return { fillColor: "#FF00FF", fillOpacity: 1, color: '#B04173', weight: 4, }; } // Create layer and add it to the map var gardenLayer = L.geoJSON(myGardenJson,{ style:gardenStyle }); gardenLayer.addTo(map);
Notice that the properties include fill color, fill opacity (which controls transparency), stroke color, and stroke weight. The colors are defined using hex notation [182]. In this case, the symbol is a magenta fill with a purple outline (Sorry! I wanted it to stand out...). When writing your own code, use an online color picker [183] to get the hex values of the colors you want.
You can see a list of Leaflet's vector styling properties by reading the Path options [184] documentation. You should be able to accomplish most of what you want to do using working examples and a little bit of experimentation.
The advantage of drawing vector layers in the browser is that you can quickly change the styling in response to certain events. For example, you can change the color of a single symbol when a user clicks it. Using Leaflet, you can define various styles and connect them to map events.
The code below demonstrates how a style map could be used to "highlight" a garden feature when it's clicked. The selected garden would change to a blue fill:
Don't worry about running this code right now, because you'll use some nearly identical code in the lesson walkthrough in a working example. Just pay attention to the code patterns and what is happening in each line.
First, create functions to define the unselected and selected styles:
// Set up styles for garden polygons // Magenta symbol for gardens function gardenStyle(feature) { return { fillColor: "#FF00FF", fillOpacity: 1, color: '#B04173', weight: 4, }; } // Blue symbol for selected gardens function gardenSelectedStyle(feature) { return { fillColor: "#00FFFB", fillOpacity: 1, color: '#0000FF', weight: 4 }; }
Then create the GeoJSON layer and add functions to listen for click events on the GeoJSON features and the map itself. The code below contains a variable named selection, which holds the currently selected feature. The expression if (selection) checks to see if there is any selected feature. The expression e.target means "the feature that just got clicked". Notice how the resetStyle method can set a layer's style back to its original form, and the setStyle method can change a feature's style to something new.
var selection; // define function to handle click events on garden features function gardenOnEachFeature(feature, layer){ layer.on({ click: function(e) { if (selection) { gardenLayer.resetStyle(selection); } e.target.setStyle(gardenSelectedStyle()); selection = e.target; L.DomEvent.stopPropagation(e); // stop click event from being propagated down to other elements } }); } // Create layer and add it to the map var gardenLayer = L.geoJSON(myGardenJson,{ style:gardenStyle, onEachFeature: gardenOnEachFeature }); gardenLayer.addTo(map); // define and register event handler for click events to unselect features when clicked anywhere else on the map map.addEventListener('click', function(e) { if (selection) { gardenLayer.resetStyle(selection); selection = null; } });
A map user should now be able to click any garden and see it highlighted in blue. When the user clicks off the garden, it goes back to its original magenta symbol.
To summarize this part of the lesson, vector layers drawn by the browser require a little more work on the developer's part, because the developer must define the symbols. However, with that work comes a variety of options for interactivity that would not be possible with rasterized image tiles or layers drawn by the server. Although the resymbolize-on-click functionality seems rudimentary, think about the difficulty (if not outright futility) of getting this type of effect with a WMS or rasterized tile layer.
This section of the lesson has discussed uniform symbols for each feature in the layer (except for the occasional selected feature); however, Leaflet allows you to go further by defining style classifications based on attribute values. This opens the door for choropleth maps, proportional symbols, and so forth. Some of these techniques will be covered in Lesson 8.
Allowing developers untrained in cartographic principles to select map symbols may present some challenges of its own. It may be a good idea to have the developer consult with a cartographer in order to determine the most appropriate widths, colors, and fills for the vector symbols.
This walkthrough builds on some of the previous sections of the lesson to show how you can add interactive GeoJSON layers to your web map using Leaflet. You will build a map containing your Philadelphia basemap tiles and two GeoJSON layers on top representing urban gardens and food pantries (i.e., food banks). A user can click one of the gardens or food pantries to see the name of the feature below the map (as an alternative to using popups). The clicked feature changes color while it is selected. Hopefully, you can think of many ways to apply these pieces of functionality to the web map you're building for your term project.
Note that this is a sample dataset culled from OpenStreetMap and is not a comprehensive list of these features in Philadelphia. If you know of any other gardens or food pantries, please add them in OpenStreetMap (more about this in Chapter 9)!
Before continuing, download and unzip the data for this project [185]. Copy its contents into your Jetty home folder, which should have a path such as
c:\Users\{username}\GeoServer 2.x.x\webapps\geog585\
This is the same folder where you saved your Lesson 6 walkthrough and where your local stylesheet style.css (required for this exercise) is located.
This folder contains two JavaScript files containing GeoJSON data. gardens.js holds a gardensData variable with polygon GeoJSON and pantries.js holds a pantriesData variable with point GeoJSON.
There are also two SVG (scalable vector graphics) files that will be used for symbolizing the food pantries. The yellow symbol will be for the unselected features and the blue symbol for the selected features.
There are a couple of ways that you could get this kind of data for your own applications.
In both cases you would need to either save the data as a JS file and define the GeoJSON as a variable (the approach we took here), or use an extension like Leaflet AJAX to read the data directly out of the file (beyond the scope of this course).
To get icons, all the icons available in QGIS are available on your machine in a folder named something like:
C:\Program Files\QGIS <name of your version>\apps\qgis\svg
I used the open source program Inkscape to change the color of the icon.
Before diving into the JavaScript code, create an empty text file and insert the following code. Then save it as lesson7.html next to all the other files you just downloaded and copied into your home folder.
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>Food resources: Community gardens and food pantries</title> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.css" type="text/css" crossorigin=""> <script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.js" crossorigin=""></script> <script src="gardens.js"></script> <script src="pantries.js"></script> <link rel="stylesheet" href="style.css" type="text/css"> <script type="text/javascript"> . . . </script> </head> <body onload="init()"> <h1 id="title">Food resources: Community gardens and food pantries</h1> <div id="mapid"></div> <div id="summaryLabel"> <p>Click a garden or food pantry on the map to get more information.</p> </div> </body> </html>
If you open this, you should see a blank map frame surrounded by an HTML title and descriptive text. In the script and link tags, notice that we are loading in Leaflet code and stylesheets, as well as the gardens and pantries.js files.
Now let's add the Leaflet JavaScript code that creates the map and layers.
var map; function init() { // create map and set center and zoom level map = new L.map('mapid'); map.setView([39.960,-75.210],14); // create and add the tile layer var tiles = L.tileLayer('https://<URL of your S3 bucket>/PhillyBasemap/{z}/{x}/{y}.png'); tiles.addTo(map); . . . }
https://geog585-somestring.s3.someregion.amazonaws.com/PhillyBasemap/{z}/{x}/{y}.png
var gardenLayer; var pantryLayer; var selection; var selectedLayer; . . .Nothing is happening with these yet, but it's important that you understand their future purpose in the code:
// define the styles for the garden layer (unselected and selected) function gardenStyle(feature) { return { fillColor: "#FF00FF", fillOpacity: 1, color: '#B04173', }; } function gardenSelectedStyle(feature) { return { fillColor: "#00FFFB", color: '#0000FF', fillOpacity: 1 }; } . . .
// handle click events on garden features function gardenOnEachFeature(feature, layer){ layer.on({ click: function(e) { if (selection) { resetStyles(); } e.target.setStyle(gardenSelectedStyle()); selection = e.target; selectedLayer = gardenLayer; // Insert some HTML with the feature name buildSummaryLabel(feature); L.DomEvent.stopPropagation(e); // stop click event from being propagated further } }); } . . .
// add the gardens GeoJSON layer using the gardensData variable from gardens.js var gardenLayer = new L.geoJSON(gardensData,{ style: gardenStyle, onEachFeature: gardenOnEachFeature }); gardenLayer.addTo(map); . . .
// create icons for pantries (selected and unselected) var pantriesIcon = L.icon({ iconUrl: 'pantries.svg', iconSize: [20,20] }); var selectedPantriesIcon = L.icon({ iconUrl: 'pantries_selected.svg', iconSize: [20,20] }); // handle click events on pantry features function pantriesOnEachFeature(feature, layer){ layer.on({ click: function(e) { if (selection) { resetStyles(); } e.target.setIcon(selectedPantriesIcon); selection = e.target; selectedLayer = pantryLayer; // Insert some HTML with the feature name buildSummaryLabel(feature); L.DomEvent.stopPropagation(e); // stop click event from being propagated further } }); } // add the pantries GeoJSON layer using the pantriesData variable from pantries.js pantryLayer = new L.geoJSON(pantriesData,{ pointToLayer: function (feature, latlng) { return L.marker(latlng, {icon: pantriesIcon}); }, onEachFeature: pantriesOnEachFeature } ); pantryLayer.addTo(map); . . .
// handle clicks on the map that didn't hit a feature map.addEventListener('click', function(e) { if (selection) { resetStyles(); selection = null; document.getElementById('summaryLabel').innerHTML = '<p>Click a garden or food pantry on the map to get more information.</p>'; } }); . . .
// function to set the old selected feature back to its original symbol. Used when the map or a feature is clicked. function resetStyles(){ if (selectedLayer === pantryLayer) selection.setIcon(pantriesIcon); else if (selectedLayer === gardenLayer) selectedLayer.resetStyle(selection); } . . .Separate lines of code are needed above for the pantries and gardens layers because points represented by icons and polygons have different styling syntaxes in Leaflet.
// function to build the HTML for the summary label using the selected feature's "name" property function buildSummaryLabel(currentFeature){ var featureName = currentFeature.properties.name || "Unnamed feature"; document.getElementById('summaryLabel').innerHTML = '<p style="font-size:18px"><b>' + featureName + '</b></p>'; }The above function brings in the currently selected feature and reads its "name" attribute. It then gets the HTML element with the ID of "summaryLabel" and sets its innerHTML to a carefully constructed string of HTML into which the name (represented by the variable featureName) is inserted. Note that if our gardens and pantries layers had different attribute field names (such as "PANTRYNAME" and "GARDENNAME" then we would need to add more code above to handle those cases.
If your page does not work, carefully compare your code to the full code below to make sure you have inserted everything in the right place. Also:
An OpenLayers 3 version of the walkthrough code [187] is available for the curious. Note that the GeoJSON files must be adjusted for this version of the walkthrough to function. It must be pure GeoJSON and not contain any declared variables or JavaScript code.
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>Food resources: Community gardens and food pantries</title> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.css" type="text/css" crossorigin=""> <script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.js" crossorigin=""></script> <script src="gardens.js"></script> <script src="pantries.js"></script> <link rel="stylesheet" href="style.css" type="text/css"> <script type="text/javascript"> var map; function init() { // create map and set center and zoom level map = new L.map('mapid'); map.setView([39.960,-75.210],14); // create and add the tile layer var tiles = L.tileLayer('https://<URL of your S3 bucket>/PhillyBasemap/{z}/{x}/{y}.png'); tiles.addTo(map); var gardenLayer; var pantryLayer; var selection; var selectedLayer; // define the styles for the garden layer (unselected and selected) function gardenStyle(feature) { return { fillColor: "#FF00FF", fillOpacity: 1, color: '#B04173', }; } function gardenSelectedStyle(feature) { return { fillColor: "#00FFFB", color: '#0000FF', fillOpacity: 1 }; } // handle click events on garden features function gardenOnEachFeature(feature, layer){ layer.on({ click: function(e) { if (selection) { resetStyles(); } e.target.setStyle(gardenSelectedStyle()); selection = e.target; selectedLayer = gardenLayer; // Insert some HTML with the feature name buildSummaryLabel(feature); L.DomEvent.stopPropagation(e); // stop click event from being propagated further } }); } // add the gardens GeoJSON layer using the gardensData variable from gardens.js var gardenLayer = new L.geoJSON(gardensData,{ style: gardenStyle, onEachFeature: gardenOnEachFeature }); gardenLayer.addTo(map); // create icons for pantries (selected and unselected) var pantriesIcon = L.icon({ iconUrl: 'pantries.svg', iconSize: [20,20] }); var selectedPantriesIcon = L.icon({ iconUrl: 'pantries_selected.svg', iconSize: [20,20] }); // handle click events on pantry features function pantriesOnEachFeature(feature, layer){ layer.on({ click: function(e) { if (selection) { resetStyles(); } e.target.setIcon(selectedPantriesIcon); selection = e.target; selectedLayer = pantryLayer; // Insert some HTML with the feature name buildSummaryLabel(feature); L.DomEvent.stopPropagation(e); // stop click event from being propagated further } }); } // add the pantries GeoJSON layer using the pantriesData variable from pantries.js pantryLayer = new L.geoJSON(pantriesData,{ pointToLayer: function (feature, latlng) { return L.marker(latlng, {icon: pantriesIcon}); }, onEachFeature: pantriesOnEachFeature } ); pantryLayer.addTo(map); // handle clicks on the map that didn't hit a feature map.addEventListener('click', function(e) { if (selection) { resetStyles(); selection = null; document.getElementById('summaryLabel').innerHTML = '<p>Click a garden or food pantry on the map to get more information.</p>'; } }); // function to set the old selected feature back to its original symbol. Used when the map or a feature is clicked. function resetStyles(){ if (selectedLayer === pantryLayer) selection.setIcon(pantriesIcon); else if (selectedLayer === gardenLayer) selectedLayer.resetStyle(selection); } // function to build the HTML for the summary label using the selected feature's "name" property function buildSummaryLabel(currentFeature){ var featureName = currentFeature.properties.name || "Unnamed feature"; document.getElementById('summaryLabel').innerHTML = '<p style="font-size:18px"><b>' + featureName + '</b></p>'; } } </script> </head> <body onload="init()"> <h1 id="title">Food resources: Community gardens and food pantries</h1> <div id="mapid"></div> <div id="summaryLabel"> <p>Click a garden or food pantry on the map to get more information.</p> </div> </body> </html>
In this week's assignment, you'll make a mashup consisting of a vector layer drawn by the browser on top of your own tiled basemap. To achieve this, you need to adapt the walkthrough techniques and code to your own data. Follow the instructions below to prepare this assignment:
Above & beyond: Successful delivery of the listed requirements is sufficient to earn 90% on this assignment. The remaining 10% is reserved for efforts that go "above & beyond" these minimum requirements to improve your web map. This could include (but is not limited to): (a) using a combination of style properties not used in the walkthrough (see example [175] and look for other examples on the web) to produce a particularly nice symbology for your layer or an additional thematic layer you add, (b) producing some more advanced HTML output for selected features (e.g., showing an attribute table, including hyperlinks), and / or (c) adding other HTML elements to your page related to your map. Please add a brief description of what you did for "above & beyond" points to your text document.
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
Note: Currently, this lesson teaches the Leaflet API. If you are looking for the previous materials on OpenLayers, see the Lesson 8 archive page [188].
So far, you've learned how to create various types of web map layers and overlay them using Leaflet. Simple mashups with "dots on a map" and a few popups may be all you need in many situations. In fact, some managers are thrilled to see these basic web maps if they have never visualized their data in geographic form before; however, as a student of GIS you'll want to be aware of additional possibilities to make the map layers more interactive or informative.
This lesson introduces you to a variety of "extra" things you can do to enrich the interactivity and information exposed by your web maps. Some of these features cannot be covered in full detail in this course; however, the concepts are introduced here in case you ever encounter them or need to use them in the workplace. During the lesson assignment, you'll have the opportunity to choose one of these techniques (or other FOSS of your choice) and explore it more fully.
A word of caution is necessary here: Just because you can do something doesn't mean that you should. Many of the best web maps are effective because they are focused and uncluttered. You'll see in this lesson that Leaflet makes it easy to add all kinds of controls to your web maps, but before doing this, pause and determine whether the extra features are really needed. The same applies to WFS, WPS, widgets provided by web display frameworks, and other features mentioned in this lesson.
Choropleth maps, proportional symbols maps, and on-the-fly feature filtering all provide alternative functionality to the typical "dots on a map" mashup. How can you achieve these things with Leaflet? In this section of the lesson, we'll talk about how to create Leaflet symbols based on attributes in your data. You can then apply these symbols to vectors to get various types of maps, such as choropleth maps based on class breaks.
First, it's important to be clear that symbolizing layers on the client using Leaflet code is not the only way to achieve choropleth maps, proportional symbols, etc., on the web. Earlier in this course, you became familiar with dynamically drawn WMS maps defined by SLD styling, and tiled maps styled in QGIS. In these approaches, you do the cartography on the server rather than the client, or at least the spatial data is passed by the server to the client in an already styled format, typically as one or multiple image files. Server-side maps are most appropriate for advanced cartographic effects. They may also result in better performance when hundreds or thousands of features are in play.
Why, then, are we discussing making our specialized maps with Leaflet styles when so many other cartographic tools lie within reach? The reason is that defining symbols on the client opens the door to flexibility and interactivity.
In order to create a styling rule based on some attribute value, it's first necessary to read the attribute. How do you do this in Leaflet?
Let's consider a dataset that I downloaded from the Buenos Aires open data portal [189], showing subway ("subte") lines. Each subway line is composed of many polyline segments. Each polyline segment has an attribute stating which subway line it belongs to (e.g., "LINEA B"). I've created a GeoJSON file out of this, and I want to display it in a web map using unique colors for each subway line, like this:
First, let's take the easy case where the color value is directly coded into the attribute table. Notice the COLOR field below which contains color hex values for each line segment:
If you are fortunate enough to have this setup and the colors work for you, then you can apply them directly to the color property when you define the layer style. In Leaflet you can use the syntax feature.properties.<PROPERTY> to get the value of any feature attribute:
// Set up styles for subway lines function subteStyle(feature) { return { "color": feature.properties.COLOR, "weight": 5 }; } // Create layer and add it to the map var subteLayer = L.geoJSON(subteData, { style: subteStyle });
The above code creates a vector layer from the variable subteData which comes from a GeoJSON file. To style the layer, the subteStyle function reads the hex value from the COLOR field and inserts it in the color property of the layer. Notice how the syntax feature.properties.COLOR
is utilized to read the color value.
Although the above technique is convenient, it's not always practical. What if the colors provided in the file aren't appropriate for your map? Fortunately, in our scenario, you could work around this by providing color values in your JavaScript (i.e., your client-side) code based on the name of the subway line.
Let's look at how you'd apply some style rules on the client side using Leaflet. Examine the following variation on the code we viewed previously. This code produces the exact same map:
// Set up styles for subway lines function subteStyle(feature) { var colorToUse; var line = feature.properties.LINEASUB; if (line === "LINEA A") colorToUse = "#46C7FA"; else if (line === "LINEA B") colorToUse = "#E81D1A"; else if (line === "LINEA C") colorToUse = "#4161BA"; else if (line === "LINEA D") colorToUse = "#599C65"; else if (line === "LINEA E") colorToUse = "#65018A"; else if (line === "LINEA H") colorToUse = "#FAF739"; else colorToUse = "#000000"; return { "color": colorToUse, "weight": 5 }; } // Create layer and add it to the map var subteLayer = L.geoJSON(subteData, { style: subteStyle });
The above example employs a function subteStyle to read the attribute LINEASUB from each feature, thereby figuring out the name of the subway line (LINEA A, LINEA B, etc.). If/then logic is applied to find the appropriate color to use based on the subway line name. Finally, this color is applied to a style returned by the function.
If you have a field with some numeric attribute, such as the length of a subway line or the number of riders per day, you may want to size the symbols proportionally so that big values are represented with a bigger symbol. Let's consider an example where we have a GeoJSON dataset showing statistics for some of South America's largest metro rail systems. You'll get more familiar with this dataset in the lesson walkthrough, but here's the basic attribute table and map:
In the above image, the metro systems are all drawn using the same symbol. But let's suppose that you wanted to proportionally size the symbols so that the more stations in a metro system, the larger the map symbol. Notice the STATIONS attribute contains this information. The desired map would look something like the following:
Accomplishing the proportional symbols in Leaflet requires you to define some mathematical function that will size each symbol based on each feature's attribute value for STATIONS. The syntax below is somewhat advanced, but pay attention to the line which reads the value for STATIONS, divides it by 80, and multiplies it by 30 to derive the width and height in pixels for any given metro system symbol. These numbers signify that a metro system with 80 stations will have a symbol 30 pixels wide, a metro system with fewer stations will be less than 30 pixels, and a metro system with more stations will be greater than 30 pixels (the numbers 80 and 30, of course, are entirely arbitrary and could be adjusted to fit your own data):
// function to size each icon proportionally based on number of stations function iconByStations(feature){ var calculatedSize = (feature.properties.STATIONS / 80) * 30; // create metro icons return L.icon({ iconUrl: 'metro.svg', iconSize: [calculatedSize, calculatedSize] }); } // create the GeoJSON layer and call the styling function with each marker var metroLayer = L.geoJSON(metroData, { pointToLayer: function (feature, latlng) { return L.marker(latlng, {icon: iconByStations(feature)}); } });
In the above code, iconSize is a two-item JavaScript array containing the width and the height in pixels that should be applied to the icon. Also, notice the use of the pointToLayer property, which is necessary when you want to replace the default Leaflet markers with your own graphics.
In some situations, it may be more appropriate to break up your numerical data into various classifications that are symbolized by graduated colors or some similar approach. This is especially true for line and polygon datasets that are sometimes harder to conceptualize using proportional symbols. The boundaries for your classes (in Esri software you may have heard these referred to as "class breaks") could be determined based on equal interval, quantile, natural breaks, or some other arbitrary scheme.
For example, in the image below, metro systems with over 100 stations are symbolized using dark red. Systems with 50 to 100 stations are symbolized with red. Systems with fewer than 50 stations are symbolized with pink:
To symbolize data classes in Leaflet, we'll read one of the feature attributes, then use if/then logic to check it against our class breaks. The code below defines the three classes used in our metro rail example. Each class references a different SVG symbol with its unique hue of red:
// create metro icons var metroLowIcon = L.icon({ iconUrl: 'metro_low.svg', iconSize: [25,25] }); var metroMediumIcon = L.icon({ iconUrl: 'metro_medium.svg', iconSize: [25,25] }); var metroHighIcon = L.icon({ iconUrl: 'metro_high.svg', iconSize: [25,25] }); // function to use different icons based on number of stations function iconByStations(feature){ var icon; if (feature.properties.STATIONS >= 100) icon = metroHighIcon; else if (feature.properties.STATIONS >= 50) icon = metroMediumIcon; else icon = metroLowIcon; return icon; } // create the GeoJSON layer and call the styling function with each marker var metroLayer = L.geoJSON(metroData, { pointToLayer: function (feature, latlng) { return L.marker(latlng, {icon: iconByStations(feature)}); } });
Although the above may look like a lot of code, notice that half of it is just setting up the icons. A function that classified lines or polygons might be much simpler because a single style could be defined with a varying stroke or fill color based on the attribute of interest.
If you intend to use a classification system such as Jenks natural breaks, equal interval, quantile, etc., you must calculate the break values yourself (or find a library [190] that does it) before defining the rules. You can either do this by hand or add more JavaScript code to calculate the values on the fly.
In some situations, you may want to display only a subset of the features in a dataset, based on some attribute value or combination of values. (If you're familiar with Esri ArcMap, this is called a "definition query".) Suppose you wanted to show only the metro systems whose COUNTRY attribute was coded as "Brazil":
You can do this directly in the filter property when you create the layer in Leaflet. Here you write a function that evaluates a feature and returns a value of true or false. Leaflet will only draw features for which a value of true is returned:
// create metro icons var metroIcon = L.icon({ iconUrl: 'metro.svg', iconSize: [25,25] }); // create the GeoJSON layer and call the styling function with each marker var metroLayer = L.geoJSON(metroData, { pointToLayer: function (feature, latlng) { return L.marker(latlng, { icon: metroIcon }); }, filter: function(feature, layer) { if (feature.properties.COUNTRY === "Brazil") return true; else return false; } });
The filter function in the above example tests for features where the COUNTRY property equals "Brazil". Note that the above example simply reads an attribute, it does not do a spatial query to find the bounds of Brazil. Your website will run much faster if you can preprocess the data to put a country attribute on each metro system, rather than trying to figure that out on the fly using spatial processing in a web environment.
Now, let's look at a scenario with a numeric attribute. Suppose we wanted to show only the metro systems with over 75 stations:
This could be accomplished with the following filter:
// create metro icons var metroIcon = L.icon({ iconUrl: 'metro.svg', iconSize: [25,25] }); // create the GeoJSON layer and call the styling function with each marker var metroLayer = L.geoJSON(metroData, { pointToLayer: function (feature, latlng) { return L.marker(latlng, { icon: metroIcon }); }, filter: function(feature, layer) { if (feature.properties.STATIONS > 75) return true; else return false; } });
In the code above, note the query for a STATIONS value that is greater than 75.
Leaflet offers a variety of options for symbolizing data on the client side based on attribute values. You should not feel limited to using the default symbology or a single symbol type for the entire layer.
Creating Leaflet styles can require some trial and error, along with some skillful debugging. You'll have more success if you start with a working example and tweak it piece by piece, frequently stopping to test your alterations. This section of the lesson is deliberately code-heavy so that you will have many examples available to get you started.
Although Leaflet does not offer a point-and-click interface for building a web application, it does give you a few pre-coded "controls" that you can add to the map with just a few lines of code. These include a scale bar, layer switcher, zoom buttons, and attribution box. OpenLayers offers these same types of controls, plus a few more such as an overview map, geographic coordinates written on the page, and so on.
The example below shows a Leaflet layer switcher control in action. Hovering the mouse over (or tapping) the layer icon in the corner displays a layer list, where you can choose a basemap and toggle the visibility of thematic layers. This image shows the default OpenStreetMap Mapnik basemap selected, with the option to switch to the Stamen Toner basemap. You'll apply this code in your walkthrough later in the lesson. Only one thematic layer is available below, the "Subway lines" layer; however, you could include more layers here simply by adding them to the Leaflet map.
Here's how you would code the layer switcher above. First, set up all the layers and add the ones to the map that you want to have initially visible. Then, call a function that sets up two simple JavaScript objects containing your basemaps and thematic layers. Finally, create the layer switcher (L.control.layers [191]), passing in those two JavaScript objects as parameters.
// create and add osm tile layer var osm = L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { maxZoom: 19, attribution: '© <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>' }); osm.addTo(map); // create stamen osm layer (not adding it to map) var stamen = L.tileLayer('http://tile.stamen.com/toner/{z}/{x}/{y}.png', { maxZoom: 19, attribution: '© <a href="http://www.openstreetmap.org/">OpenStreetMap</a> and contributors, under an <a href="http://www.openstreetmap.org/copyright" title="ODbL">open license</a>. Toner style by <a href="http://stamen.com">Stamen Design</a>' }); // Set up styles for subway lines function subteStyle(feature) { return { "color": feature.properties.COLOR, "weight": 5 }; } // Create layer and add it to the map var subteLayer = L.geoJSON(subteData, { style: subteStyle }); subteLayer.addTo(map); createLayerSwitcher(); // function to create a layer switcher control function createLayerSwitcher() { // define basemap and thematic layers and add layer switcher control var basemaps = { "OSM Mapnik": osm, "OSM Stamen Toner": stamen }; var overlays = { "Subway lines": subteLayer }; L.control.layers(basemaps,overlays).addTo(map); }
Usually control frameworks are extensible, so you may even decide to pull in controls developed by others or, if you're feeling ambitious, code your own. Although customizing the out-of-the-box Leaflet controls is beyond the scope of this course, you should be aware that controls have a stylesheet you can tweak. Also, controls with images can be customized by substituting your own image. Examining the Leaflet stylesheets and example stylesheets may help you understand which CSS properties to override when applying your customizations.
The Leaflet API reference describes the controls [192] in the namespace L.Control. The best way to learn about controls is to experiment with them yourself and expand your functionality, bit by bit, using existing examples, documentation, and some trial and error. In other web mapping APIs, controls may be called "widgets," "plugins," or other names.
In Lesson 7, you saw some ways that vector data can be drawn by the web browser or client. The lesson focused on standalone KML and GeoJSON files, yet it is also possible for a web service to send the data to the client on request. The data can be in any format as long as both the server and the client are following the same specification. To standardize the process of sending vector data through web services, the Open Geospatial Consortium (OGC) has produced the Web Feature Service (WFS) specification.
You've already seen a related service, the WMS, in previous lessons. How is WMS different from WFS? WMS involves the server sending a single map image, whereas WFS involves the server sending vector data as text to be drawn by the client. In simple terms, a WMS map is drawn by the server and a WFS map is drawn by the client.
Like a WMS, a WFS supports a set of operations that typically take in parameters directly within the URL. These operations include GetCapabilities, DescribeFeatureType, and GetFeature. The GetFeature operation is the one that actually retrieves features.
Below is an example GetFeatures WFS request for the US state of Colorado (I picked something with a simple geometry). I adapted this from the Boundless WFS tutorial [193], which unfortunately doesn't seem to be available online anymore. Another recommended source for further information on WFS is the OGC e-Learning page [194]. You won't be able to use the link below yourself, but see if you can guess what each parameter signifies, then continue reading about the response:
http://suite.opengeo.org/geoserver/wfs?service=wfs&version=1.1.0&request=GetFeature&typename=usa:states&featureid=states.23
By examining the above URL parameters, you can see that a feature is requested from the WFS using version 1.1.0 of the WFS specification. The service is being hosted on GeoServer in a layer named states in the USA workspace. The feature with index 23 is returned.
A WFS returns data using Geography Markup Language (GML), a specification for expressing GIS data using XML. GML can contain both geometry and attribute information. Because it is based on XML and is designed to be flexible enough to handle many geographic feature types, GML is much more verbose (takes more text) than GeoJSON. Deep down in the GML for Colorado, you can find the geometry:
<gml:posList>37.48468400000007 -109.04348799999995 38.164690000000064 -109.04176199999989 38.27548899999999 -109.06006199999996 41.0006590000001 -109.05007599999999 41.00235900000007 -102.051717 36.993015999999955 -102.04208899999992 36.99908400000004 -109.0452229999999 37.48468400000007 -109.04348799999995</gml:posList>
You can also find attributes like this:
<usa:NAME10>Colorado</usa:NAME10>
The same type of request could be made to one of your own services running on GeoServer. Here's how I made a request for a Philadelphia neighborhood using one of the layers we published earlier in this course:
http://localhost:8080/geoserver/wfs?service=wfs&version=1.1.0&request=GetFeature&typename=geog585:Neighborhoods&featureid=Neighborhoods.12
The response looks like this and contains geometry and attributes for the Olney neighborhood:
<wfs:FeatureCollection numberOfFeatures="1" timeStamp="2014-03-03T15:07:31.822Z" xsi:schemaLocation="http://localhost:8080/geoserver/geog585 http://localhost:8080/geoserver/wfs?service=WFS&version=1.1.0&request=DescribeFeatureType&typeName=geog585%3ANeighborhoods http://www.opengis.net/wfs http://localhost:8080/geoserver/schemas/wfs/1.1.0/wfs.xsd"><gml:featureMembers><geog585:Neighborhoods gml:id="Neighborhoods.12"><geog585:the_geom><gml:MultiSurface srsDimension="2" srsName="urn:x-ogc:def:crs:EPSG:3857"><gml:surfaceMember><gml:Polygon srsDimension="2"><gml:exterior><gml:LinearRing srsDimension="2"><gml:posList>-8363968.786751106 4869301.13520122 -8363706.077778376 4871057.31164155 -8363880.846283749 4871132.918517317 -8363697.377540309 4872031.511981935 -8363780.660729433 4872179.806916264 -8363847.448310932 4872208.890548547 -8363802.926044645 4872557.878939522 -8363802.44449278 4872626.491915396 -8363025.915000884 4872530.247301338 -8361543.138729884 4872310.6731403675 -8361453.88028348 4872223.294811407 -8361493.045963939 4872015.489274301 -8361627.94355705 4871826.7318475135 -8361690.687270048 4871673.398417745 -8361627.94355705 4871403.748827802 -8361286.901117077 4870791.777211798 -8361326.368936536 4870458.7113405885 -8361498.408149585 4869986.8871721085 -8361555.111808623 4869831.380121785 -8362695.297708079 4869623.850560427 -8363168.406381819 4869548.2551895585 -8363968.786751106 4869301.13520122</gml:posList></gml:LinearRing></gml:exterior></gml:Polygon></gml:surfaceMember></gml:MultiSurface></geog585:the_geom><geog585:STATE>PA</geog585:STATE><geog585:COUNTY>Philadelphia</geog585:COUNTY><geog585:CITY>Philadelphia</geog585:CITY><geog585:NAME>Olney</geog585:NAME><geog585:REGIONID>214146.0</geog585:REGIONID></geog585:Neighborhoods></gml:featureMembers></wfs:FeatureCollection>
Although the syntax for WFS requests and responses may look intimidating, you will rarely have to worry about composing this yourself. Various FOSS and proprietary software packages include support for viewing and serving WFS.
As shown in the above examples, GeoServer can expose your layers through WFS. This is enabled by default and requires no configuration on your part. Other FOSS GIS servers such as Map Server and Deegree support the creation of WFS web services.
In the proprietary realm, Esri ArcGIS Server gives users the option to expose their web services through WFS; however, Esri has developed its own "feature service" that works through REST and accomplishes many of the same things as a WFS. Be aware that the Esri web mapping APIs and editing widgets are designed to work with the feature service rather than WFS. The communication patterns of the feature service are openly documented in the GeoServices REST Specification.
Support for WFS varies across web mapping APIs. WFS can be viewed "out of the box" as a vector layer in OpenLayers, but not within Leaflet. As mentioned above, Esri web mapping APIs are designed to work with the REST feature service rather than WFS.
Although Leaflet won't draw the GML responses sent by a WFS, it will draw GeoJSON; therefore, one pattern of working with WFS via Leaflet is to configure the server to send back GeoJSON instead of GML. You can then send WFS requests to the server via AJAX (asynchronous web requests) and use Leaflet classes to draw the geoJSON coming back. Such patterns are exposed through numerous forum posts [195], as well as third-party plugins.
On the desktop side, QGIS supports viewing and editing WFS. (Note: I got errors when trying to edit a WFS in QGIS, but I did not have a database or proxy host behind the layer.) Other FOSS clients such as uDig also recognize the WFS format. In Esri ArcMap, you use the Data Interoperability Extension to add WFS layers to the map (see "Connecting to a WFS Service [196]"). ArcGIS Pro allows users to add WFS, and the extension does not appear to be required (documentation here [197]).
The WFS specification also defines rules for feature editing, opening the door to building clients that can edit geographic data over the web. A WFS enabled for editing is known as a transactional WFS, or WFS-T. Beyond the standard WFS operations such as GetFeature, WFS-T supports an additional operation called Transaction, and may also support operations for locking features to prevent concurrent edits.
Creating web editing applications requires both a server that can handle the incoming transactions and a client that provides visual support for the edit sketching, vertex manipulation, attribute entrance into text boxes, and so forth. The client sketching operations must be wired up to the WFS-T requests. Unless you're ready to write a lot of JavaScript, it's best to start with existing samples, widgets, or web controls. Leaflet offers a Draw control [198] that can take care of the sketching part. See an example of what "Draw" [199] can do. The respec/leaflet.wfs-t [200] is an example of a plugin designed to wire up the Draw control to WFS-T requests. (Full disclosure: I have not tested this particular plugin, but there are other plugins based on the same concept).
OpenLayers also offers some basic buttons for sketching, modifying, deleting, and saving features. You can see them in action in this developer example [201].
Before you expose any dataset for editing over the web, you should carefully think through your quality control and data storage architectures. For example, perhaps you want to expose a copy of your database for web editing, then have an analyst review any changes before they are pushed to your master GIS database. If in-house editors are making changes to the master database, you'll also need a way to periodically push those to your web copy. In other cases, you may check incoming edits with a script to make sure they comply with certain topology rules or attribute schema requirements.
Both FOSS and proprietary GIS software offer spatial data processing functions such as buffer, union, contour, interpolate, and so on. You invoked some of these using QGIS and GDAL in earlier lessons. But what if you wanted to allow people to run these functions from a web browser? For example, suppose you wanted to allow users of your web map to draw a polygon and then see a calculation of the total population and number of health clinics within that polygon. You'd be able to expose the GIS to many people without them needing to install any GIS software.
OGC has released a specification for invoking spatial data processing through web services. It's called the Web Processing Service (WPS) specification. Like the other OGC services you've learned about, it offers a set list of operations you can call. These are: GetCapabilities, DescribeProcess, and Execute. Of course, the Execute operation is the one that launches the request to actually perform the processing. The server does the work to process the data and send back the result as a response. GML may be used to transfer information about vector features in either the request or the response.
As you are probably aware from running GDAL and ArcToolbox tools, spatial data processing functions can sometimes require many input parameters. For example, a buffer process might require you to specify the width of the buffer, whether it will be applied to both sides of the feature, whether the ends should be capped or rounded, etc. Each tool has its own set of parameters and syntax for describing them. Because the inputs can be so long and complex (especially if geometry is included), sometimes you can put the inputs in their own XML file and POST them to the server, rather than putting all the parameters into the URL as a GET request like you have seen with WMS and WFS in previous lessons. Some servers and browsers impost limits on the length of an HTTP GET request, whereas HTTP POST requests can typically be much longer.
The WPS spec itself doesn't say which types of spatial data processing operations a WPS must support; that's up to whoever builds and administers the service. There are hundreds of potential operations that can be included. When you first use a WPS, you can invoke the GetCapabilities operation to find out which processes are available.
GeoServer offers a WPS extension [202] that exposes a set of spatial processes from the FOSS JTS Topology Suite [203], as well as some other processes developed specifically by GeoServer. We will not install this extension in this course, but I encourage you to browse through the documentation if you think that you may use it in your workplace or other academic work.
The Zoo Open WPS Platform [204] and PyWPS [205] are other examples of FOSS WPS implementations. In the proprietary realm, Esri ArcGIS Server can serve a WPS from a ModelBuilder model that you create from ArcToolbox tools or scripts.
A few GUI-based WPS clients are available that allow you to select tools and supply their parameters in text boxes or dropdown lists. QGIS has a WPS plugin [206] that works this way, allowing you to call a WPS from the desktop.
When it comes to invoking a WPS directly from a web application, some web map APIs offer helper classes or libraries that can help you. Leaflet is not one of these (I will get to that in a minute). OpenLayers 2 supports WPS through the use of OpenLayers.WPSClient. When you instantiate this object, you supply the URL of the WPS server. You then set up JavaScript objects containing all the parameters of the process you want to invoke. Finally, you execute the process and specify what should be done with the results. See this OpenLayers developer example [207]available online.
Even when you use a WPS client plugin, library, or object, you still need to be familiar with the process and its documentation, so that you can supply the correct syntax for the parameters. One mistake in syntax can derail the entire processing operation. Furthermore, WPS servers and clients are often either in the early stages of maturity or are designed for power users who are comfortable with a lack of a GUI and extensive documentation.
Many spatial processing operations are rather common, and there is a great desire from GIS web developers to invoke these without the overhead of a WPS. The Turf.js [208] library has gained popularity for allowing geoprocessing directly on GeoJSON vectors in a JavaScript environment. In other words, Turf differs from WPS because the client machine runs the processing code, not the server. Turf was developed at Mapbox, but is free to use and modify under the open MIT license [209]. Operations available through Turf.js include dissolving, unioning, intersecting, buffering, aggregating, calculating centroids, and so forth.
You can download Turf.js to your own machine or access it online via a CDN [210]. Because Turf.js works on GeoJSON, it fits very nicely with Leaflet. Most of the abundant Turf.js developer examples are shown using Mapbox's API (for obvious reasons), but the way you would invoke them from Leaflet is essentially the same. For example, the following snippet is from the Turf.js analysis walkthrough [211]:
// Using Turf, find the nearest hospital to library clicked
var nearestHospital = turf.nearest(e.layer.feature, hospitalFeatures);
In this situation, both parameters of the turf.nearest() function are GeoJSON objects: one representing a library building, and another one representing a set of hospital buildings. The function finds the hospital nearest the clicked library and returns it as GeoJSON. Fundamentally, it doesn't matter which API you use to display the output GeoJSON.
Before implementing any kind of spatial data processing on the web, consider ways that you might preprocess the data in order to eliminate the need for on-the-fly calculations. When you invoke spatial processing on the web, it makes your server busy and increases end-user wait times. In the above example, perhaps the nearest hospital to each library could be precalculated in a desktop GIS environment and stored in the library attribute table. Then your code would just need to read the attribute, rather than performing processing.
There will be some situations where precalculation doesn't make sense due to the wide range of analysis parameters and possibilities you want to expose. For example, if there were 20 different kinds of features that you wanted to allow people to find near a library, or if a person could type in the address of some library not in the original dataset. Therefore, the decision about whether to allow real-time spatial processing should be made on a case-by-case basis.
Unless you already have some previous exposure to WPS, I do not recommend integrating it into your term project given the short amount of time that remains in the course. A Turf.js implementation would be a better fit for the scope of the term project, since it can be implemented fairly quickly with less code.
Neither of these technologies is a requirement for the project, although they may provide useful functionality in many scenarios.
Working with web pages is not always easy, especially when it's not your main area of expertise. Code for laying out the page, iterating through elements, toggling classes, etc., can get clunky and bloated in a hurry. Browsers can interpret the same pieces of HTML and JavaScript differently, and debugging the pages can be a mysterious process involving multiple code files and complex hierarchies of stylesheets and overrides.
There is no magic solution for these challenges; however, there are some JavaScript helper libraries that can simplify your life and your code if you make the effort to learn them. Your pages will probably also become more functional and attractive. Some of these libraries offer mainly back-end functions, others specialize in front-end elements, and others offer both.
Many web developers like to use special JavaScript libraries that have been developed to simplify common functions and abstract away some of the idiosyncrasies between web browsers.
For example, if you select a few web pages at random and look at the source code, chances are pretty good that you'll see someone using the jQuery library. jQuery [212] provides functions to simplify navigating and manipulating the DOM elements on your page. For example, using jQuery you can populate a dropdown list or change the CSS class of an HTML element on the fly without writing a bunch of HTML markups.
Similar alternatives to jQuery are Prototype [213] and the Dojo toolkit [214], although the latter also offers some UI elements such as menus, buttons, and charts. All these libraries are build with the goal of simplifying JavaScript coding and reducing the work for you to handle differences between browsers.
In the previous lesson examples, you've learned how to embed your map in a web page. In the Lesson 7 walkthrough, you also briefly saw how to use the innerHtml property to manipulate a DOM element and thereby change a label on a page. But how do you craft a nice looking page around your map without devoting hundreds of hours to web design study? This is where a web presentation framework can come in handy.
Web presentation frameworks consist of JavaScript libraries, stylesheets, and "widgets" that work together to give you some convenient building blocks for nice looking pages. Some of these goodies include headers, buttons, calendar date pickers, menus, etc.
Web development frameworks are typically engineered so that your page works in a similar manner across browsers. They may also give you a head start with making your page easily localizable (i.e., flexible enough to be offered in various languages such as Spanish, Arabic, Chinese, etc.).
An example of a popular web presentation framework is Bootstrap [215], distributed by the Twitter development team under an open source license. In the Lesson 8 walkthrough, you'll use Bootstrap to put a nice header on your application and divide your page into sections. You'll also get the benefit of the Bootstrap stylesheets.
Similar frameworks to Bootstrap include Groundwork [216], Zurb's Foundation [217], and Yahoo's Pure [218]. The latter used to be called YUI (yoo'-ee) and this name is still often seen in the web world. Take a few minutes and follow some of the above links to see the different styles offered by these frameworks.
Some JavaScript libraries such as the Dojo toolkit and Sencha Ext JS [219] also offer layout elements that serve a web design function. The project jQuery UI [220] offers a helpful set of user interface elements such as buttons, menus, date pickers, etc.
Even more powerful, we have web application frameworks for supporting the design and implementation of complex web applications and services, like the popular Python-based Django [221] and JavaScript-based React [222], which also contain web presentation components.
You should be aware that web presentation frameworks sometimes require and use jQuery or other libraries to accomplish their functions.
When evaluating a web presentation framework for your project, you might consider:
A web presentation framework can help produce a nicer end product than you might be able to concoct on your own; however, a framework also introduces another level of complexity. You may have to override some of the framework's stylesheets in order to get other libraries (like Leaflet or OpenLayers) to behave in the expected way. Debugging such issues often involves multiple levels of stylesheets and a lot of time in the weeds. If you just need a very simple app, you might leave the framework aside and create your own stylesheet, or at least try to select the most simple framework available for the functionality you need.
In this walkthrough, you'll put together some of the things you've learned in this lesson to make a well-rounded informational map containing thematic styling and Leaflet controls. The map will use the Bootstrap web presentation framework so that it can be extended with supplementary content in an aesthetically pleasing way. You will construct some of this content on the fly by reading the attributes of selected map features.
For this exercise, we'll stick with the metro data from South America. The final application will allow users to click any metro and see some informational text and an image from Flickr. A static legend image is also brought into the layout.
Although this lesson is called "Going beyond 'dots on a map," I have deliberately chosen point data for this walkthrough so that you can appreciate the possibilities offered through Leaflet, especially when tied together with other frameworks. Although you may not feel like a JavaScript guru at this point, you should at least know that you don't have to settle for the 'red teardrop and popup' mashups that you see all over the Internet.
Before you add JavaScript, I will just point out a few things that are important in the layout. First, notice how Bootstrap is brought into the application through a reference to a JavaScript file (.js) and a CSS file (.css) in the bolded lines below. We are referencing the files from a CDN, but you could alternatively download and host (and tweak) them yourself.
<script src="http://code.jquery.com/jquery-latest.js"></script> . . . <script src="https://maxcdn.bootstrapcdn.com/bootstrap/2.2.1/js/bootstrap.min.js"></script> <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/2.2.1/css/bootstrap.min.css">
Also take note of the stylesheet override in the <style> tag. This will prevent our page content from crashing into the top banner.
<style> body { padding-top: 60px; padding-bottom: 40px; } </style>
If you don't like something about the Bootstrap (or Leaflet) default styles, the stylesheet overrides are where you can apply a change.
In the page body, observe how classes are used to organize the page into navigation bars (navbar-*), containers (container-fluid), and spans. The span5 and span7 classes divide the page vertically so that consistent proportions of the page area are devoted to the map (on the left) and the supplementary material (on the right). Don't worry about understanding all these classes at this point. Just be aware that web development frameworks like Bootstrap, Dojo, etc., may give you CSS classes for organizing page elements in a way that should behave consistently across browsers.
Finally, note that the legend image and the "Click a metro..." text are organized into an HTML table to keep them flush. The table borders are invisible by design.
<table> <tr> <td><img src="metro_legend.png"/></td> <td id = 'summaryLabel'><p>Click a metro rail system on the map to get more information.</p></td> </tr> </table>
A div awaits for a Flickr image, but this won't appear until someone clicks a metro symbol on the map.
<div id="metroImage"></div>
Now let's breathe some life into this app by adding some JavaScript code.
You'll use JavaScript to do three main things in this application: add the map, define the symbols, and handle the click events. Let's address these in order.
In your lesson8_walkthrough.html, find the script tags. You'll put all your code immediately after: <script type="text/javascript">.
Add the following lines of code to create the map and an initialization function. This function contains some variables that we'll use to handle selections throughout, similar to what you saw in Lesson 7:
var map; var metroLayer; function init() { // create map and set center and zoom level map = new L.map('mapid'); map.setView([-28,-62],3); var selection; var selectedLayer; var selectedFeature; . . . }
'https://tiles.stadiamaps.com/tiles/stamen_toner/{z}/{x}/{y}.png?api_key=xyz'Now replace the . . . in the code above with the following. Although it's a lot, all of this should look pretty familiar if you paid attention to the code snippets earlier in the lesson:
// create and add osm tile layer var osm = L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { maxZoom: 19, attribution: '© <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>' }); osm.addTo(map); // create osm humanitarian layer (not adding it to map) var osmHumanitarian = L.tileLayer('https://{s}.tile.openstreetmap.fr/hot/{z}/{x}/{y}.png', { maxZoom: 19, attribution: '© <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>' }); // create metro icons var metroLowIcon = L.icon({ iconUrl: 'metro_low.svg', iconSize: [25,25] }); var metroMediumIcon = L.icon({ iconUrl: 'metro_medium.svg', iconSize: [25,25] }); var metroHighIcon = L.icon({ iconUrl: 'metro_high.svg', iconSize: [25,25] }); var metroSelected = L.icon({ iconUrl: 'metro_selected.svg', iconSize: [25,25] }); // add the metro GeoJSON layer var metroLayer = L.geoJson(metroData,{ pointToLayer: function (feature, latlng) { return L.marker(latlng, {icon: iconByPassday(feature)}); }, onEachFeature: metrosOnEachFeature }); metroLayer.addTo(map); . . .
You may have noticed that the metroLayer relies on two functions: iconByPassday and metrosOnEachFeature. We'll tackle those in a few minutes. But first, the layer switcher...
Replace the . . . in the code above to add a layer switcher control to the map:
// define basemap and thematic layers and add layer switcher control var basemaps = { "OSM": osm, "OSM Humanitarian": osmHumanitarian }; var overlays = { "Metro stations": metroLayer }; L.control.layers(basemaps,overlays).addTo(map); . . .Again, the pattern here is to create a JavaScript object for the basemaps and one for the thematic layers, and then pass those two objects in as parameters when you create the control.
Now for those functions I mentioned. The first one, iconByPassday, looks at the number of passengers per day that travel through each metro system, then decides which icon to use. It's the key to classifying the layer and symbolizing it with the different colors. Replace the . . . in the code above with the following:
// define functions that right icon for a given feature function iconByPassday(feature) { var icon; if (feature.properties.PASSDAY >= 2000000) icon = metroHighIcon; else if (feature.properties.PASSDAY >= 1000000) icon = metroMediumIcon; else icon = metroLowIcon; return icon; } . . .
Now for that metrosOnEachFeature function. This one is a little more lengthy. It adds a click event listener to the metros layer and then defines a function to handle that event. Replace the . . . in the code above with the following:
// define function to handle click events on metro features function metrosOnEachFeature(feature, layer){ layer.on({ click: function(e) { // reset symbol of old selection if (selection) { if (selectedLayer === metroLayer) selection.setIcon(iconByPassday(selectedFeature)); } // apply yellow icon to newly selected metro and update selection variables e.target.setIcon(metroSelected); selection = e.target; selectedLayer = metroLayer; selectedFeature = feature; // using attributes, construct some HTML to write into the page var featureName = feature.properties.CITY || 'Unnamed feature'; var country = feature.properties.COUNTRY || '(Unknown)'; var year = feature.properties.YEAR || '(Unknown)'; var passengers = feature.properties.PASSDAY || '(Unknown)'; var stations = feature.properties.STATIONS || '(Unknown)'; var length = feature.properties.LENGTHKM || '(Unknown)'; var link = feature.properties.LINK || 'http://www.wikipedia.org'; var photoHtml = feature.properties.PHOTO || '<P>Photo not available</P>'; var titleHtml = '<p style="font-size:18px"><b>' + featureName + '</b></p>'; var descripHtml = '<p>The ' + featureName + ', ' + country + ' metro opened in ' + year + ' and currently serves ' + passengers + ' passengers a day. The network consists of ' + stations + ' stations spread over ' + length + ' kilometers.</p>'; var readmoreHtml = '<p><a href="' + link + '">Read more</a></p>'; document.getElementById('summaryLabel').innerHTML = titleHtml + descripHtml + readmoreHtml; document.getElementById('metroImage').innerHTML = photoHtml; L.DomEvent.stopPropagation(e); // stop click event from being propagated further } }); } . . .Examine the code comments above to understand what each section is doing. There's a lot of code, but most of it is constructing the informational HTML describing the metro system. Recall that the attribute table looks like this:
You should notice many of these attribute field names referenced in the code above. The field value is retrieved, using some special syntax || (logical OR operator that in JavaScript returns the first operand that is TRUE) to set a fallback value in case no attribute comes back. The rest of the function constructs some HTML strings, inserting the attributes where appropriate. After the HTML strings are constructed, the innerHTML is updated for the elements "summaryLabel" and "metroImage." This causes the new text and the photo to appear on the right side of the page.
The PHOTO field deserves some additional discussion here. Anticipating that this field would be used to embed a photo in an app, the entire iframe HTML code is placed in this field as a long string. Where do you get this code if you want to make a dataset like this? Right from the Flickr embed functionality:
Finally, add a click event handler to the map that will reset the selection. This is necessary if someone clicks the map, but not a metro icon. Replace the . . . in the code above with the following:
// define and register event handler for click events to unselect features when clicked anywhere else on the map map.addEventListener('click', function(e) { if (selection) { if (selectedLayer === metroLayer) selection.setIcon(iconByPassday(selectedFeature)); selection = null; document.getElementById('summaryLabel').innerHTML = '<p>Click a metro rail system on the map to get more information.</p>'; document.getElementById('metroImage').innerHTML = '' } });Notice that some code was added here to also clear out the informational HTML and the photo.
Test the application by opening lesson8_walkthrough.html in a web browser. (If you're just opening the HTML file directly from the file system, I recommend using Firefox to test. This is because the cross-origin request security implementations in Chrome and Internet Explorer only allow the GeoJSON to load if you're hosting both the HTML and GeoJSON files on a web server.)
You should be able to switch between different base layers (note that the Map Quest open layer shown below is no longer available). Click a metro icon to see the highlighted symbol, the Flickr image, and the descriptive text.
If the walkthrough does not function, check to ensure you are connected to the Internet and that your code matches the code below:
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>Lage metro rail systems in South America</title> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.css" type="text/css" crossorigin=""> <script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.2.0/leaflet.js" crossorigin=""></script> <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script> <script src="metro.js"></script> <link rel="stylesheet" href="style.css" type="text/css"> <script src="https://maxcdn.bootstrapcdn.com/bootstrap/2.2.1/js/bootstrap.min.js"></script> <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/2.2.1/css/bootstrap.min.css"> <style> body { padding-top: 60px; padding-bottom: 40px; } </style> <script type="text/javascript"> var map; var metroLayer; function init() { // create map and set center and zoom level map = new L.map('mapid'); map.setView([-28,-62],3); var selection; var selectedLayer; var selectedFeature; // create and add osm tile layer var osm = L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { maxZoom: 19, attribution: '© <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>' }); osm.addTo(map); // create osm humanitarian layer (not adding it to map) var osmHumanitarian = L.tileLayer('https://{s}.tile.openstreetmap.fr/hot/{z}/{x}/{y}.png', { maxZoom: 19, attribution: '© <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>' }); // create metro icons var metroLowIcon = L.icon({ iconUrl: 'metro_low.svg', iconSize: [25,25] }); var metroMediumIcon = L.icon({ iconUrl: 'metro_medium.svg', iconSize: [25,25] }); var metroHighIcon = L.icon({ iconUrl: 'metro_high.svg', iconSize: [25,25] }); var metroSelected = L.icon({ iconUrl: 'metro_selected.svg', iconSize: [25,25] }); // add the metro GeoJSON layer var metroLayer = L.geoJson(metroData,{ pointToLayer: function (feature, latlng) { return L.marker(latlng, {icon: iconByPassday(feature)}); }, onEachFeature: metrosOnEachFeature }); metroLayer.addTo(map); // define basemap and thematic layers and add layer switcher control var basemaps = { "OSM": osm, "OSM Humanitarian": osmHumanitarian }; var overlays = { "Metro stations": metroLayer }; L.control.layers(basemaps,overlays).addTo(map); // define functions that right icon for a given feature function iconByPassday(feature) { var icon; if (feature.properties.PASSDAY >= 2000000) icon = metroHighIcon; else if (feature.properties.PASSDAY >= 1000000) icon = metroMediumIcon; else icon = metroLowIcon; return icon; } // define function to handle click events on metro features function metrosOnEachFeature(feature, layer){ layer.on({ click: function(e) { // reset symbol of old selection if (selection) { if (selectedLayer === metroLayer) selection.setIcon(iconByPassday(selectedFeature)); } // apply yellow icon to newly selected metro and update selection variables e.target.setIcon(metroSelected); selection = e.target; selectedLayer = metroLayer; selectedFeature = feature; // using attributes, construct some HTML to write into the page var featureName = feature.properties.CITY || 'Unnamed feature'; var country = feature.properties.COUNTRY || '(Unknown)'; var year = feature.properties.YEAR || '(Unknown)'; var passengers = feature.properties.PASSDAY || '(Unknown)'; var stations = feature.properties.STATIONS || '(Unknown)'; var length = feature.properties.LENGTHKM || '(Unknown)'; var link = feature.properties.LINK || 'http://www.wikipedia.org'; var photoHtml = feature.properties.PHOTO || '<P>Photo not available</P>'; var titleHtml = '<p style="font-size:18px"><b>' + featureName + '</b></p>'; var descripHtml = '<p>The ' + featureName + ', ' + country + ' metro opened in ' + year + ' and currently serves ' + passengers + ' passengers a day. The network consists of ' + stations + ' stations spread over ' + length + ' kilometers.</p>'; var readmoreHtml = '<p><a href="' + link + '">Read more</a></p>'; document.getElementById('summaryLabel').innerHTML = titleHtml + descripHtml + readmoreHtml; document.getElementById('metroImage').innerHTML = photoHtml; L.DomEvent.stopPropagation(e); // stop click event from being propagated further } }); } // define and register event handler for click events to unselect features when clicked anywhere else on the map map.addEventListener('click', function(e) { if (selection) { if (selectedLayer === metroLayer) selection.setIcon(iconByPassday(selectedFeature)); selection = null; document.getElementById('summaryLabel').innerHTML = '<p>Click a metro rail system on the map to get more information.</p>'; document.getElementById('metroImage').innerHTML = '' } }); } </script> </head> <body onload="init()"> <div class="navbar navbar-inverse navbar-fixed-top"> <div class="navbar-inner"> <div class="container-fluid"> <a class="brand" href="/">Large metro rail systems in South America</a> </div> </div> </div> <div class="container-fluid"> <div class="row-fluid"> <div class="span5"> <div id="mapid"> </div> </div> <div class="span7"> <div> <table> <tr> <td><img src="metro_legend.png"/></td> <td id = 'summaryLabel'><p>Click a metro rail system on the map to get more information.</p></td> </tr> </table> <div id="metroImage"></div> </div> </div> </div> </div> </body> </html>
If the amount of JavaScript in the walkthrough was intimidating, don't worry. You don't have to write that kind of code in the lesson assignment, although I hope some of it comes in handy in your term project.
For this week's assignment, we're going to do something a bit different. You will identify and report on some GIS FOSS that interests you. Your typical encounter with FOSS in the "real world" will be open-ended, with little direction other than what you can find online. Therefore, this assignment is designed to give you an experience of independent exploration.
First, read this article to get an idea of the many different FOSS products used with GIS, some of which we have not covered in this course. I have placed this article in the Lesson 8 module in Canvas:
With its many acronyms, this article approaches alphabet soup at times. But I think you are far enough along in this course that you will recognize some of them and have enough frame of reference to process the ones you don't know. Obviously, this article is several years old, but will help get you up to date on the major players in FOSS GIS. I am not aware of a more recent article as thorough as this.
Identify a FOSS solution for GIS that we have not covered in this course, download it (if applicable), and briefly experiment with it. Then use the software to accomplish something useful.
In fulfilling this requirement, keep in mind the following:
Use the Steiniger and Hunter article, as well as the Ramsey video from Lesson 1, if you need ideas about what to review.
Write a report of around 1000 words describing the following:
If you reviewed a cloud based service or a FOSS product created by a for-profit entity, please delineate which elements of the software are FOSS and which are not. Also describe how the FOSS portion fits into the business model of the provider.
Remember that free software and APIs are not always open source. The software you review for this assignment must have some portion where the source code is openly shared and can be reused or modified.
You should submit your report into the Lesson 8 assignment drop box on Canvas.
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
A GIS or web map is only as useful as the data you put into it. Just as the GIS landscape offers proprietary software and open software, you will see sources of proprietary data and open data. This lesson explores the different meanings of "open data" and provides an introduction to OpenStreetMap, a growing repository of open data that is useful in a variety of projects.
Lately, it seems that "open data" is everywhere. Reaching buzzword status, this term is often seen in tandem with phrases such as "open government," "crowdsourcing," "government transparency," and "free and open source software." But what makes data "open"? Just as you learned in Lesson 1, that different organizations, even proprietary software companies, employ the term "open source" to their advantage. There are various nuances to the term "open data" that you should consider whenever you hear someone touting this phrase.
Consider the following means of data access and how they might be placed on a continuum of more or less "open":
Consider how these levels of data access play into the following scenarios:
The most open types of data are those that allow complete download, re-use, and modification of the data in open formats. However, other levels of data openness may be more useful than not seeing the data at all. If you expose a useful dataset through a web map or a web service, you should prepare an answer for the question, "Can I download this data?" It won't be long before somebody asks.
Even when data is freely available for download in open formats at no cost, it may still be subject to licensing restrictions. There are numerous types of open data licenses that stipulate what types of applications can use the data (personal, noncommercial, commercial, etc.) and what kind of attribution must be given. The license may also state the types of modifications that are allowed on the data, especially if the modified dataset is to be redistributed.
To get a feel for some of these licenses such as Creative Commons, Open Database License, Open Government License, and Public Domain, please take a few minutes to read pages 4 - 8 of Licensing Open Data: A Practical Guide [225] by Korn and Oppenheim, 2011. Focus especially on the chart on page 6.
FOSS typically excels at working with open data formats; however, FOSS is certainly not the only option for creating, exposing, or using open data. For example, Esri has invested in building open data discovery and download mechanisms into its ArcGIS Online and Portal for ArcGIS products. The idea is that government customers will be more likely to maintain their data in the proprietary software repository if the repository is easily engineered to allow free and open downloads by the public in popular formats such as KML and CSV. Here is the video of an interview [226] with Esri's Andrew Turner from the Esri DevSummit 2017 on the topic of Open Data that you may find interesting.
If you don't have the money or means to purchase your required GIS data, or if the data doesn't exist, then you may need to collect the data yourself. If your goal is to openly share the resulting data with the public, then you may consider enlisting the public in your data collection efforts. VGI and crowdsourcing are two concepts that come into play when enlisting the public or non-domain experts in the collection of GIS data.
In 2007 Michael Goodchild published a paper in which he elaborated on the idea of volunteered geographic information (VGI). This kind of data is collected by citizens acting as sensors to gather information about the world around them. The citizens then feed this information into a centralized GIS database, often employing a user interface that has been simplified to the degree that specialized training is not required.
VGI has since become a hot term in geographic information science as thousands of people contribute to the OpenStreetMap digital map of the world (discussed later). Governments evaluate the possibilities of creating "citizen reporting" apps that allow anyone to upload information about potholes, graffiti, etc., with the objective of bringing them to the attention of local authorities.
Crowdsourcing is the idea of using the power of a crowd to collect data that is too vast, heterogeneous, or expensive to be collected by other types of sensors. Consider how many people you would have to hire in order to write an encyclopedia with 30 million articles in 250 languages. The crowdsourced website Wikipedia [227] has been able to create a project of this scope solely through crowdsourcing. Other applications of crowdsourcing include combing remotely sensed imagery [228] to find lost people or vehicles, recording old weather measurements from ship logs [229] in order to create climate databases, and transcribing census records [230] to create searchable genealogical indexes.
Crowdsourcing is a particularly good fit for tasks that require an element of human cognition not easily performed by machines. Amazon has even made a business out of crowdsourcing through its Mechanical Turk [231] service. This allows you to hire a crowd of unknown individuals to perform tasks for a particular fee, often pennies for each task. Using an architecture that is conceptually similar to cloud computing, you can scale the task up to as many volunteers as you need.
The concept of crowdsourcing is a good fit for VGI, particularly when a vast amount of data must be collected under time pressure; however, not all VGI projects use crowdsourcing. Some of them are focused on gathering information from a small sample of people or a focused group of domain experts. Cinnamon and Schuurman (2012), for example, enlisted a set of emergency medical professionals at a single hospital to submit information about the locations of local auto accidents. Using tablet computers, the paramedics tapped the screen or typed an address to record the locations of the accidents. The researchers called this type of guided process facilitated VGI (f-VGI), after Seeger (2008). These readings are available in the Lesson 9 module on the course Canvas site if you're interested in learning more about them.
The introduction of humans into the sensory element of data collection presents some interesting advantages and challenges. One advantage is that there are a lot of humans potentially available. Some of them even appear to have a lot of time on their hands! This means that tasks can be scaled up quickly and the data can be collected (or corrected) in a hurry. Humans also have the ability to care about projects and become passionate about them, increasing the amount and quality of data collected and creating an endless source of free organization and labor. It's not always necessary to hire the Mechanical Turk when you're enlisting people in a project they really believe in.
However, humans, by nature, make mistakes in some ways that computers may not. They get tired, they commit typos, they make subjective judgments, and so forth. Furthermore, the technical skills and physical infrastructure (e.g., Internet access) required for VGI participation may not be uniformly distributed throughout your study area. Finally, humans carry particular biases and interests that may skew the types of data collected.
Anyone employing VGI in scientific research or mission-critical applications should be aware of these limitations. The next section of this lesson provides some examples of how these advantages and limitations of VGI have affected OpenStreetMap.
OpenStreetMap (OSM) is a digital map database of the world built through crowdsourced volunteered geographic information (VGI). OSM is supported by the nonprofit OpenStreetMap Foundation [232]. The data from OSM is freely available for visualization, query, download, and modification under open licenses [233].
OSM works in a style similar to Wikipedia, in which virtually all features are open to editing by any member of the user community. OSM was conceived in 2004 and pretty much 10 years later reached two million registered users [234]. Although only a fraction of these are frequent map editors, the map has matured enough in some locations to the point where its detail and precision rival "authoritative" datasets from governments and commercial entities. This is particularly true in Western Europe and some parts of the US. The image below of the Penn State campus provides an idea of the intricate features that can be submitted to OSM.
OSM originally gained popularity in places where government data was not freely available, but a thriving GIS community existed. For example, in the mid-2000s, the UK Ordnance Survey data was available only for purchase, and OSM grew rapidly as an attractive free alternative. In places where governments were willing to freely share their data, bulk upload negotiations were sometimes arranged. For example, the US has fairly thorough road coverage due to a US Census TIGER street data bulk upload.
OSM volunteer efforts constitute a social event and hobby for many, who gather for group data collection events known as "mapping parties." These activities organized armies of volunteers to walk, bike, and drive through sectors of a city with GPS units and notepads, returning later to a central lab to enter the data (Perkins and Dodge 2008). Although this is still useful in cases, nowadays, many OSM beginners can get pretty far just through tracing aerial photographs in simple browser-based editors. In addition to a physical exploration of the city, mapping parties now offer training, awareness, and renewed enthusiasm of OSM (Hristova et al. 2013).
To contribute a feature to OSM, you typically digitize a geometry (a point, line, or area) and then add descriptive attributes, or tags. For example, to tag a grocery store, you trace its building footprint and tag it with shop=supermarket. There's no restriction on the tags you can use, but the data is only useful to the degree that you tag things consistent with the way other OSM users have applied the tags.
To promote consistency in tagging, the OSM community has an informal tag voting and approval process organized on the OpenStreetMap wiki [235] site. Approved tags are added to the online documentation so that others can easily find and apply them. For example, the tag shop=supermarket [236] denotes a grocery store. Before you add a tag, check the wiki to make sure that you're using the established tag and syntax. If you create your own tag and start using it on many features, consider putting it through the OSM proposal process [237].
OSM is not the only crowdsourced VGI project, but it is one of the most well known. As such, it provides a useful exemplification of the pros and cons of crowdsourcing and VGI.
Some of the main benefits of OSM include:
Some of the main challenges of OSM include:
The most basic use of OSM is to retrieve its map tiles as a background for other thematic layers. High-profile sites using OSM in this way include Foursquare, Craigslist, and Wikipedia. Some web developers switched to OSM as a basemap [241] after the Google Maps API introduced potential fees into its terms of service.
From a technical perspective, anyone can use a rendering engine like QGIS or Mapnik to draw tiles of OSM data. In fact, this is what you did with QGIS in the Lesson 5 walkthrough. The image below shows how you can select various basemap renderings on OpenStreetMap.org. Other companies such as Mapbox [242] have made their own OSM renderings that can be consumed as web services. In fact, Mapbox's business model has come to rely so heavily on OSM that the company has invested in near-real-time quality monitoring of incoming OSM edits by hosting OSMCha [243], an OSM Changeset Analyzer [244] tool originally written by Wille Marcel in 2015 (you need to sign in with a OSM account to be able to use the tool).
Let's now take a look at some of the ways OSM can be used "beyond the basemap."
OSM gained publicity as a disaster response aid in 2010 after the Haiti earthquake, as described in this PSU Geospatial Revolution video [245]. Prior to this disaster, publicly available digital data for Haiti was sparse, and OSM was limited to major roads and a handful of other features. In the weeks following the earthquake, Internet volunteers worldwide traced imagery and referenced out-of-copyright maps to create a detailed geographic database of the country in OSM. This provided helpful basemaps for humanitarian aid workers who were flocking to the country and needed maps to get around. It also served as an inventory of hospitals, churches, civic facilities, and other resources that could be used by responders.
The growth of OSM during this period was nothing short of dramatic, and a number of animations such as this video: OpenStreetMap - Project Haiti [246] have depicted the expansion of the map in Haiti during this time period. Zook et al (2010) offer an analysis of various methods of VGI and crowdsourcing used in the earthquake response, including OSM and the crisis mapping site Ushahidi.
Crowdsourced volunteer efforts work most efficiently when there is an organizing force behind the work. Using lessons from the Haiti experience, the Humanitarian OpenStreetMap Team [247] (HOT) now provides this function. After Typhoon Haiyan hit the Philippines in 2013, HOT provided tools to explain and partition the volunteer mapping work on OSM so that the most needed features and geographic areas were given priority. Volunteers visiting the HOT site could click a map sector to work on, and were given instruction about which features to trace and how to tag them. The image below shows the OSM Tasking Manager, an application used by HOT to catalog sectors completed and sectors that need work.
The efforts to rapidly assemble crisis mappers in Haiti and the Philippines are admirable, but the ideal situation would be to already have the OSM data on hand. These regions only needed the mapping because sufficient information hadn't been contributed in the first place. Lack of technical infrastructure, a shortage of human and monetary capital, civil restrictions, and other factors can cause places to remain unmapped. Graham (2010) calls these places "virtual black holes" in VGI. Unfortunately, commercial Internet maps may also neglect these places if it is believed the search and advertising functions related to the map will not produce sufficient revenue to justify the investment.
OSM has been used as a way to give a presence to communities that have previously remained unmapped. Hagen (2010) describes a project in Nairobi wherein local youth volunteers were enlisted and trained to map the sprawling slum of Kibera. Home to hundreds of thousands of people, this settlement was little more than a name on previous maps. The Map Kibera [248] effort used OSM tools to record water points, toilets, clinics, schools, pharmacies, places of worship, and NGO offices. The result is a map that the residents can use to find local services and lobby the government for infrastructure support. The features added through this project are immediately apparent when you navigate to Kibera using even the default OSM map.
Similar stories can be found elsewhere in the world. When participants in a Buenos Aires hackathon wanted to map social services in a local slum, they found the area empty in commercial maps and decided to use OSM as a basemap [249]. Even when a street network exists, other layers such as bus routes may be helpful for individuals without automobiles, opening possibilities for local travel outside of daily routines. Motivated individuals have headed up an OSM project with bus routes in India [250], noting that a detailed local map can also help with tourism promotion efforts.
One of the advantages of OSM is its flexibility to store any type of feature, given the many tags that already exist and the community-based tag proposal and voting process. In some cases, specialized thematic maps have been created around a subset of feature types. Examples of these include:
In these maps, OSM acts as a freely accessible repository for local knowledge of useful things. Some of these mapped features provide great value to a community, but are not monetarily lucrative and may be excluded from proprietary commercial maps. Even the default OSM tiles do not show all the above types of features because to do so would cause the map to be cluttered. There is a great need for developers who can retrieve custom subsets of data from OSM and display it in thematic maps.
Remember that thematic maps are only possible because OSM allows free download and re-use of the data. Sites that use OSM for thematic mapping often rely on one of the various query APIs available for OSM, such as the Overpass API that allows the submission of custom tag queries through a web service. Asking a web service to give you all features matching a certain tag is often more manageable than downloading the entire OSM dataset for a region. You will get a taste of the Overpass API in the lesson walkthrough.
The maps and queries depend heavily on users maintaining consistency with established tag syntax. For example, the Philly Fresh Food Map relies on tags described in the Food Security [255] page of the OSM wiki.
Getting data out of OpenStreetMap (OSM) presents more technical challenges than putting data into OSM. When you put data into OSM, you can use your choice of a number of different types of editors. You can use any tags that you want, attempting to stick to tagging conventions of course.
In contrast, when you get data out of OSM, you have to deal with the following:
Complicating matters is the fact that OSM returns data in its own structure of XML, which is not immediately readable by many GIS applications. Therefore, getting data from OSM often involves converting from this XML into some other format.
There are a variety of mechanisms for downloading OSM data. The easiest ones address the challenges by providing a way to filter the tags you want, allowing you to specify the output format, and allowing you to specify a geographic bounding box for the requested data, so you don't retrieve too much.
One of the most user-friendly GUI-oriented ways that I have found for retrieving OSM data is a server at BBBike.org [256]. This little web-based tool allows you to draw a bounding box interactively and specify the output format you want. After a while, you receive an e-mail with a link to download your data.
In the walkthrough, however, we'll use the OSM download mechanism that is available in QGIS via the OSMDownloader plugin. Although this way is a little more advanced than the BBBike extract service, it is more immediate and allows greater flexibility for the amount of data and tags selected.
Examine the image below of Cayenne, French Guiana. You'll notice that the city has detailed building footprint polygons available. Let's suppose that we want to get a shapefile of these building footprints using QGIS.
Note that we have defined our three pieces of essential information to filter the OSM data we want:
While in a previous version of QGIS, downloading OSM data was integrated into the main program and available under Vector > OpenStreetMap > Download data, this functionality has been removed and now requires a plugin to be installed. On the positive side, these plugins now require much fewer steps to obtain the data. There are several plugins available that can be used for downloading OSM data. We here show you the steps using the OSMDownloader plugin. Go ahead and install this plugin from the QGIS plugin manager. You should be able to find it under "Not installed". When activated, the plugin will add a toolbar with a single button to QGIS: When you click it, this button will become highlighted and you can then select an area in the map canvas for which you want to download the OSM data.
Now perform the following steps:
Under Save Location, press the Save File button and navigate to the folder you created in step 1. Then use cayenne.osm for the file name.
Click OK, which will start the download process. A progress bar will appear and finally a window informing you that the download has finished.
Figure 9.9
We are done using the download plugin now. Next, let's add the downloaded .osm file with the OSM data to our QGIS project. For this, you can simply drag the cayenne.osm file from the Windows File Explorer onto the map canvas in QGIS. The .osm file contains entities of different geometry type. However, the layer that will be added to QGIS can only contain features of a single geometry type. Therefore, you will be shown a dialog window in which you have to pick that geometry type. Since we are interested in building polygons, you should select "multipolygons" here and then click the button either called OK or Add Layers.
Figure 9.10
Behind any data retrieval mechanism from OSM is a web service request. You can send these requests directly from your web browser or an automated program using an OSM query API. One of the most powerful of these APIs is called Overpass [257]. Try the following:
http://www.overpass-api.de/api/xapi_meta?*[building=yes][bbox=-52.35,4.88,-52.25,4.98]Notice what this is requesting...It should look familiar.
You can use Python or other scripting languages to make these requests automatically. For example, here's how you could use Python to query OSM for all the farmers' markets in Philadelphia and save them to a .osm file. (You're not required to run this code).
import urllib workspace = "C:\\data\\OSMdev\\" # Make data queries to jXAPI marketsXml = urllib.urlopen("http://www.overpass-api.de/api/xapi_meta?*%5Bshop=farm%5D%5Bbbox=-75.29,39.86,-74.95,40.15%5D").read() # Make farmers markets file marketsPath = workspace + "markets.osm" marketsFile = open(marketsPath, 'w') marketsFile.write(marketsXml) marketsFile.close()
For Python junkies: The above code uses a library called urllib which is able to make web requests and read the responses. You just have to provide the URL for the request. So as not to be interpreted as defining a list, the "[" and "]" characters are escaped using the %5B and %5D sequences, respectively, but otherwise the query has the same syntax as the one you issued above for Cayenne buildings. The resulting XML is then written to a file using the standard Python write method.
A script like this might be useful if you wanted to update one or more datasets on a periodic basis. The script could be combined with GDAL processing to get the data into a format suitable for your web map. Recent versions of GDAL (1.10 and later) can read OSM XML and convert it to different formats, such as GeoJSON or shapefiles. (Be careful with shapefiles though, because GDAL plops most of the less common "other tags" into one field that gets cut off at 256 characters, a limitation of the shapefile format).
As an exclamation point at the end of all this geekiness, play around with the graphical tool overpass turbo [258] for a few minutes. This gives you an interactive environment for querying OSM and seeing the results on the map. You can save any interesting result in popular formats, such as KML. This is helpful if you just want to make a one-off query to OSM for some particular feature type.
There are many circumstances and needs that can affect the way you retrieve data from OSM. Hopefully, this walkthrough has provided enough options that you can make an informed decision about how to best get the scope and scale of data you need. Now let's go to the lesson assignment, where you'll get some experience with the other side of things: putting data into OSM.
The Lesson 9 assignment has two parts: reporting on a web map that uses OpenStreetMap (OSM), and actually editing OSM yourself. You will produce a single document describing these efforts.
Find an Internet map that uses some element of OSM. Produce a write up of several paragraphs describing the following:
What is the purpose and URL of the site, and who built it?
How is OSM being used? (i.e., Is the site simply pulling the OSM tiles, or is the source data used for creating thematic layers, etc.?)
Include at least one screenshot showing the OSM data.
What advantages and disadvantages are introduced into this map by using OSM data?
Do you see any other appropriate ways that OSM data could be used in this site?
In this part of the assignment, you'll get some practice with adding data to OSM in your town or some other place that you know well. You'll take some "before" and "after" screenshots to demonstrate the things you added to the map.
The easiest way to get started with editing OSM is using the in-browser editor at OpenStreetMap.org [31], which is called iD.
The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.
Take a moment and look back at how far you've come. In this course, you've learned what free and open source software (FOSS) is and how it fits into the practice of web mapping. You've also learned how to take raw GIS data, process it, assemble it into beautiful maps, expose it as web services, and present it as a finished product in an interactive web map.
As a demonstration of these skills, you are expected to submit a term project that exposes some theme of interest to you as a web map. You have already started creating the pieces of this project in some of your weekly lesson assignments, and a portion of this final week of the course is reserved for you to solely focus on completing the project.
After the projects are submitted, we will have a "mini-conference" where you'll get the chance to browse other students' work and review it. This may provide ideas and feedback for your future web mapping projects.
In Geog 585, you are expected to create a term project that takes some data of interest to you and fuses it into a useful web map. The term project can be simple and focused in nature, but must include:
In creating your term project, you should use at least one tool or technique that was not covered in the course materials. This could, for instance, be a tool for (pre)processing the data, or it could be a Leaflet class or method that you explored. You don't have to know this part when you make the proposal, but as you work through the different exercises in the course, you should stay aware of additional functions you could incorporate to meet this requirement.
The project must be built entirely with FOSS. This requirement is not in place to make you a FOSS "purist"; rather, it is intended to compel you to fully practice the skills you have learned in this course and discover new ways of doing things. If there's some piece of your project data processing that you don't think can be completed with FOSS, please discuss it with the instructor.
There are three parts to the term project submission:
The requirements of the submission are described in detail below. Please see the term project grading rubric on Canvas to understand exactly how these requirements will be evaluated.
To share your project with the instructor and others, you will create an online screen recording "video" explaining the purpose of the project and giving a tour of its functionality. In under 5 minutes the video must cover the following:
It is expected that the video will just record the screen (you don't need to appear on camera). There are many alternatives to produce the video, including free screen recording software and services. As Penn State students, you have free access to Kaltura [261]. Another free option that worked well in the past is Screencast-O-Matic [262]which requires you to run a small program on your computer. The recorded video can be stored as a .mp4 file, or it can be shared via their web site. Alternatively, if you have access to professional screen recording software such as Captivate, Camtasia, Fraps, etc., you may use it. Much of this software is available for free trial periods. An option for Mac users is to use QuickTime; there are descriptions of how to do this out there on the web. For sharing your video, you can also upload it to any other file sharing service, for instance, dropbox.com, GoogleDrive, or OneDrive.
It's strongly recommended that you reserve at least a day or two for creation of the video. This will allow you to accommodate any unforeseen technical challenges and do multiple "takes" if necessary. Things will go more smoothly if you prepare a script or outline of things you want to show, and refer to this during the video recording.
The video is due Sunday evening before the course end date. To submit the video, follow the description on the Term project submission and mini conference page [260].
Please also submit a 500+ word writeup recapping:
Please submit all the source code for your project together with your writeup. Please zip the original HTML, JavaScript, and all other files required by your project together. Do not copy & paste the code into your writeup. Optional: If you host your final term project map in your AWS S3 storage as in homework assignment 7, please include the public S3 URL for your main .html file in your write-up. That will then allow us to directly test out your final web map application.
The writeup should follow professional writing and grammar conventions and should be spell checked. The writeup and code are due in the term project drop box by the course end date.
Tip: If you complete the writeup before you do your video narration, the words may come more easily when you are "on camera".
The project will no doubt be a learning experience for you, but there is also plenty to learn from other students' experiences and submissions. During the final three days of the course (Monday, Tuesday, Wednesday), take some time on the "Term project video & review forum" on Canvas to browse other submissions. Select two projects of interest to you and, with each, post a brief review as a reply. Your review should include:
These reviews are due by the course end date.
Academics often attend conferences where they share their discoveries and browse the work of others to gain new ideas and offer feedback. Although we can't all physically meet together, the "Term project video & review forum" on Canvas will serve as our virtual conference location where you can get the same benefits.
By the final Sunday evening of the course, make a post in that forum containing a link to your term project video. Then, by the course end date, please reply to at least two other students' videos in that forum. See the Term project submission guidelines for further instructions about how to make these submissions.
Links
[1] https://www.e-education.psu.edu/geog485/
[2] https://www.e-education.psu.edu/geog863/
[3] http://www.w3schools.com/js/
[4] http://www.w3schools.com/html/
[5] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson1/Figure_1.1_LD.html
[6] http://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0r.cgi?SERVICE=WMS&REQUEST=GetMap&FORMAT=image/png&TRANSPARENT=TRUE&STYLES=&VERSION=1.3.0&LAYERS=nexrad-n0r&WIDTH=877&HEIGHT=276&CRS=EPSG:900913&BBOX=-15252263.28954773,2902486.4758432545,-6671748.242369267,5602853.811101243
[7] http://a.tile.openstreetmap.org/15/11068/19742.png
[8] http://ride.trimet.org/?tool=routes
[9] http://parrt.cs.usfca.edu/doc/software-not-engineering.html
[10] http://en.wikipedia.org/wiki/Libre
[11] http://choosealicense.com
[12] http://www.linuxinsider.com/story/67655.html
[13] http://www.courtnewsohio.gov/cases/2013/SCO/0307/121296.asp#.XNmp2ZNKiu7
[14] http://www.computerweekly.com/news/2240179643/Government-mandates-preference-for-open-source
[15] http://aaronshaw.org/papers/Shaw-2011-Insurgent_Expertise-JITP.pdf
[16] http://innovacionucb.pbworks.com/w/file/fetch/77872493/Aaron_2011_Insurgent%20Expertise-%20The%20Politics%20of%20Free%3ALivre%20and%20Open%20Source%20Software%20in%20Brazil.pdf
[17] http://grass.osgeo.org/
[18] https://www.youtube.com/watch?v=U3Hf0qI4JLc
[19] http://www.qgis.org/en/site/
[20] http://mapserver.org/
[21] https://docs.qgis.org/2.14/en/docs/user_manual/working_with_ogc/ogc_server_support.html
[22] http://geoserver.org
[23] http://postgis.net/
[24] https://www.gaia-gis.it/fossil/libspatialite/index
[25] http://openlayers.org/
[26] http://leafletjs.com/
[27] http://modestmaps.com/
[28] http://d3js.org/
[29] http://polymaps.org/
[30] http://www.data.gov
[31] http://www.openstreetmap.org
[32] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson1/Ottawa.zip
[33] http://www.qgis.org/
[34] https://gistbok.ucgis.org/bok-topics/web-gis
[35] https://www.youtube.com/channel/UCaM-49dLjHOiQ0k0K4w5uhw
[36] http://www.esri.com/news/arcnews/spring11articles/open-source-technology-and-esri.html
[37] http://www.esri.com/products/arcgis-capabilities/open-source
[38] http://creativecommons.org/licenses/by-sa/3.0
[39] https://www.bing.com/mapspreview
[40] https://t.ssl.ak.dynamic.tiles.virtualearth.net
[41] https://wego.here.com
[42] http://www.texastribune.org/library/data/texas-reservoir-levels/
[43] https://apps.texastribune.org/reservoirs/assets/data/reservoirs.json
[44] https://www.gdacs.org/
[45] https://maps.nyc.gov/crime/
[46] https://npcn.net/npcnWebmap/index.html
[47] http://ride.trimet.org/?tool=routes#/
[48] https://www.downtownny.com/map
[49] https://www.niu.edu/visit/maps/interactivemap.shtml
[50] http://www.java.com/en/download/index.jsp
[51] https://geoserver.org/
[52] https://sourceforge.net/projects/geoserver/files/GeoServer/2.19.6/geoserver-2.19.6-bin.zip/download
[53] http://geoserver.org/release/2.14.2
[54] http://en.wikipedia.org/wiki/Servlet
[55] https://www.youtube.com/watch?v=YEOA8WWWVCw
[56] https://www.e-education.psu.edu/geog585/node/742
[57] http://resources.arcgis.com/en/help/main/10.2/index.html#//005600000003000000
[58] http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf
[59] https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwie3PPR8IzbAhVo0oMKHXvMDF8QFggpMAA&url=https%3A%2F%2Fwww.geopackage.org%2F&usg=AOvVaw2gyTsD9CNCKY-Ix18nBU0k
[60] http://switchfromshapefile.org/
[61] http://en.wikipedia.org/wiki/Keyhole_Markup_Language
[62] https://tools.ietf.org/html/rfc7946
[63] https://github.com/mbostock/topojson/wiki
[64] https://www.e-education.psu.edu/spatialdb/
[65] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson3/Figure_3.1_LD.html
[66] http://www.pasda.psu.edu/
[67] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson3/PhiladelphiaBaseLayers.zip
[68] http://www.gdal.org/ogr2ogr.html
[69] https://apps.nationalmap.gov/viewer/
[70] https://tilemill-project.github.io/tilemill/docs/guides/terrain-data/
[71] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson3/PhiladelphiaElevation.zip
[72] http://www.gdal.org/gdaldem.html
[73] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson3/atms.qpj
[74] http://www.gdal.org/gdal_utilities.html
[75] http://www.gdal.org/ogr_utilities.html
[76] https://gdal.org/programs/index.html#raster-programs
[77] https://gdal.org/programs/index.html#vector-programs
[78] http://www.deegree.org/
[79] https://portal.ogc.org/files/?artifact_id=14416
[80] http://geoservices.github.io/
[81] http://wiki.osgeo.org/wiki/Geoservices_REST_API
[82] https://imagery.pasda.psu.edu/arcgis/services/pasda/UrbanTreeCanopy_Landcover/MapServer/WmsServer?SERVICE=WMS&version=1.1.1&REQUEST=GetMap&LAYERS=10&STYLES=&BBOX=-77.87304,40.78975,-77.85828,40.80228,-77.85828,40.80228&SRS=EPSG:4326&FORMAT=image/png&WIDTH=1200&HEIGHT=900
[83] http://www.pasda.psu.edu/uci/DataSummary.aspx?dataset=3155
[84] https://imagery.pasda.psu.edu/arcgis/services/pasda/UrbanTreeCanopy_Landcover/MapServer/WmsServer?
[85] https://imagery.pasda.psu.edu/arcgis/services/pasda/UrbanTreeCanopy_Landcover/MapServer/WmsServer?SERVICE=WMS&REQUEST=GetCapabilities
[86] https://imagery.pasda.psu.edu/arcgis/services/pasda/UrbanTreeCanopy_Landcover/MapServer/WmsServer?SERVICE=WMS&version=1.1.1&REQUEST=GetFeatureInfo&LAYERS=9&STYLES=&SRS=EPSG:4326&BBOX=-77.87304,40.78975,-77.85828,40.80228,-77.85828,40.80228& FORMAT=image/png&WIDTH=1200&HEIGHT=900&QUERY_LAYERS=10& INFO_FORMAT=text/plain&X=600&Y=300
[87] http://www.opengeospatial.org/standards/sld
[88] http://docs.geoserver.org/stable/en/user/styling/sld/introduction.html
[89] http://docs.geoserver.org/stable/en/user/styling/sld/working.html
[90] http://docs.geoserver.org/stable/en/user/styling/sld/cookbook/index.html
[91] https://www.zillowgroup.com/developers/api/public-data/neighborhood-data/
[92] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson4/Neighborhoods.zip
[93] http://localhost:8080/geoserver/geog585
[94] http://docs.geoserver.org/stable/en/user/styling/sld/cookbook/polygons.html#polygon-with-styled-label
[95] http://docs.geoserver.org/latest/en/user/styling/sld/reference/labeling.html
[96] http://localhost:8080/geoserver/geog585/wms
[97] https://www.e-education.psu.edu/geog585/node/692
[98] http://www.xyht.com/spatial-itgis/using-openstreetmap-basemaps-qgis-3-0/
[99] http://www.gebco.net
[100] https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/mapserv?
[101] https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/
[102] https://geoportaal.maaamet.ee/eng/Services/Public-WMS-Service-p346.html
[103] http://msdn.microsoft.com/en-us/library/bb259689.aspx
[104] https://mts0.google.com/vt/lyrs=m@241289412&hl=en&src=app&x=74&y=96&z=8&s=Galile
[105] https://github.com/tilemill-project/tilemill
[106] https://github.com/mapbox/vector-tile-spec
[107] https://www.youtube.com/watch?v=se2cd3BMYRY
[108] https://www.mapbox.com/mapbox-studio/
[109] http://www.azavea.com/blogs/labs/2015/05/converting-mapbox-studio-vector-tiles-to-rasters/
[110] http://blogs.esri.com/esri/arcgis/2015/07/20/vector-tiles-preview/
[111] http://docs.geoserver.org/latest/en/user/extensions/vectortiles/tutorial.html
[112] https://openlayers.org/workshop/en/vectortile/map.html
[113] https://github.com/SpatialServer/Leaflet.MapboxVectorTile
[114] https://github.com/geometalab/Vector-Tiles-Reader-QGIS-Plugin
[115] https://portal.aws.amazon.com/billing/signup
[116] https://awscli.amazonaws.com/AWSCLIV2.msi
[117] https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap
[118] https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap/
[119] http://www.arcgis.com/home
[120] https://www.e-education.psu.edu/geog585/node/783
[121] http://www.w3schools.com/html/DEFAULT.asp
[122] http://www.w3schools.com/js/DEFAULT.asp
[123] http://gis.stackexchange.com/questions/tagged/openlayers
[124] http://mbostock.github.io/d3/talk/20111018/azimuthal.html
[125] http://polymaps.org/ex/transform.html
[126] http://polymaps.org/ex/cluster.html
[127] https://developers.google.com/maps/documentation/javascript/overview
[128] https://developers.google.com/maps/
[129] https://developers.google.com/maps/licensing
[130] https://www.bing.com/maps
[131] https://www.microsoft.com/en-us/maps/licensing/options
[132] http://www.trulia.com
[133] http://www.yelp.com/
[134] http://maps.lds.org
[135] https://www.craigslist.org/about/sites
[136] https://developers.arcgis.com/en/javascript/
[137] http://www.esri.com/software/arcgis/arcgisonline
[138] http://www.esri.com/software/arcgis/arcgisserver/extensions/portal-for-arcgis
[139] https://enterprise.arcgis.com/en/server/latest/get-started/windows/what-is-arcgis-for-server-.htm
[140] https://enterprise.arcgis.com/en/portal/latest/use/what-is-portal-for-arcgis-.htm
[141] https://www.esri.com/en-us/legal/terms/full-master-agreement
[142] https://docs.mapbox.com/help/glossary/mapbox-gl-js/
[143] https://www.mapbox.com/help/define-mapbox-js/
[144] https://carto.com/docs/carto-engine/maps-api/
[145] http://gis.stackexchange.com/questions/8032/how-do-various-javascript-mapping-libraries-compare?rq=1
[146] https://leafletjs.com/reference.html#map
[147] https://leafletjs.com/reference.html#layer
[148] http://leafletjs.com/reference-1.6.0.html#tilelayer
[149] https://leafletjs.com/reference.html#tilelayer-wms
[150] https://leafletjs.com/reference.html#geojson
[151] http://leafletjs.com/examples/layers-control/
[152] http://leafletjs.com/examples/quick-start/
[153] http://leafletjs.com/reference.html#tilelayer
[154] https://switch2osm.org/using-tiles/getting-started-with-leaflet/
[155] http://leafletjs.com/examples/wms/wms.html
[156] https://gist.github.com/rclark/6908938
[157] https://paulcrickard.wordpress.com/2012/04/16/query-multiple-wms-layers-in-leaflet-js/
[158] http://leafletjs.com/reference.html#popup
[159] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson6/FarmersMarkets.zip
[160] https://docs.geoserver.org/stable/en/user/styling/sld/cookbook/points.html#point-with-styled-label
[161] http://localhost:8080/geog585/mypage.html
[162] http://localhost:8080/geog585/style.css
[163] http://localhost:8080/geog585/style
[164] http://leafletjs.com/reference.html#event-objects
[165] http://localhost:8080/geog585/markets.html
[166] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson6/l6_ol3.zip
[167] http://leafletjs.com/examples.html
[168] https://bikesharemap.com/
[169] https://www.e-education.psu.edu/geog585/node/784
[170] https://harrywood.co.uk/maps/examples/openlayers/db/bbox-map.view.html#:~:text=The%20BBOX%20strategy%20means%20that,the%20example%20without%20using%20BBOX.
[171] http://leafletjs.com/plugins.html#vector-tiles
[172] https://openlayers.org/en/latest/apidoc/module-ol_source_VectorTile-VectorTile.html
[173] http://viewer.webservice-energy.org/OpenLayers-2.13.1/examples/kml/sundials.kml
[174] http://geohipster.com/poll-tally/
[175] http://leafletjs.com/examples/geojson/
[176] https://github.com/proj4js/proj4js
[177] https://kartena.github.io/Proj4Leaflet/
[178] http://resources.arcgis.com/en/help/arcgis-rest-api/02r3/02r3000000n1000000.htm
[179] https://github.com/Esri/geojson-utils
[180] https://github.com/terraformer-js/terraformer/blob/main/packages/arcgis/README.md
[181] http://blog.geomusings.com/2013/05/30/ogc-abandons-the-web/
[182] http://www.w3schools.com/html/html_colors.asp
[183] http://www.colorpicker.com
[184] http://leafletjs.com/reference-1.0.3.html#path
[185] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson7/lesson7_data_leaflet.zip
[186] http://localhost:8080/geog585/lesson7.html
[187] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson7/l7_ol3.zip
[188] https://www.e-education.psu.edu/geog585/node/785
[189] http://data.buenosaires.gob.ar/dataset
[190] http://gis.stackexchange.com/questions/11106/tiny-js-discretization-library-for-choropleth-representation
[191] http://leafletjs.com/reference.html#control-layers
[192] http://leafletjs.com/reference.html#control-zoom
[193] http://workshops.boundlessgeo.com/geoserver-intro/overview/wfs.html
[194] http://opengeospatial.github.io/e-learning/wfs/text/basic-index.html
[195] https://gis.stackexchange.com/questions/64406/getting-wfs-data-from-geoserver-into-leaflet
[196] https://desktop.arcgis.com/en/arcmap/latest/extensions/data-interoperability/connecting-to-a-wfs-service.htm
[197] https://pro.arcgis.com/en/pro-app/help/data/services/add-wfs-services.htm
[198] https://leaflet.github.io/Leaflet.draw/docs/leaflet-draw-latest.html
[199] https://leaflet.github.io/Leaflet.draw/docs/examples/full.html
[200] https://github.com/respec/leaflet.wfs-t
[201] https://openlayers.org/en/latest/examples/draw-features.html
[202] https://docs.geoserver.org/stable/en/user/services/wps/index.html
[203] https://github.com/locationtech/jts
[204] http://www.zoo-project.org/
[205] http://pywps.org/
[206] http://plugins.qgis.org/plugins/wps/
[207] https://aviationweather.gov/lib/OpenLayers-2.13.1/examples/wps-client.html
[208] http://turfjs.org/
[209] https://opensource.org/licenses/MIT
[210] https://npmcdn.com/@turf/turf/turf.min.js
[211] https://www.mapbox.com/help/analysis-with-turf/
[212] http://jquery.com/
[213] http://prototypejs.org/
[214] http://www.dojotoolkit.org
[215] http://www.getbootstrap.com
[216] https://github.com/crgeary/groundwork
[217] http://foundation.zurb.com/
[218] http://purecss.io/
[219] http://www.sencha.com/products/extjs/
[220] https://jqueryui.com/
[221] https://www.djangoproject.com/
[222] https://reactjs.org/
[223] https://www.e-education.psu.edu/geog585/sites/www.e-education.psu.edu.geog585/files/lesson8/lesson8_walkthrough_leaflet.zip
[224] https://docs.stadiamaps.com/guides/migrating-from-stamen-map-tiles/
[225] http://www.discovery.ac.uk/files/pdf/Licensing_Open_Data_A_Practical_Guide.pdf
[226] https://www.youtube.com/watch?v=d3tbRE9RsSc
[227] http://wikipedia.org
[228] http://www.cnn.com/2014/03/11/us/malaysia-airlines-plane-crowdsourcing-search/
[229] http://www.oldweather.org/
[230] https://familysearch.org/indexing/
[231] http://aws.amazon.com/mturk/
[232] http://wiki.osmfoundation.org/wiki/Main_Page
[233] http://www.openstreetmap.org/copyright
[234] https://blog.openstreetmap.org/2015/03/12/two-million-contributors/
[235] http://wiki.openstreetmap.org/wiki/Main_Page
[236] http://wiki.openstreetmap.org/wiki/Tag:shop=supermarket
[237] http://wiki.openstreetmap.org/wiki/Proposal_process
[238] https://www.missingmaps.org/
[239] http://sterlingquinn.net/apps/crowdlens
[240] https://www.youtube.com/watch?v=01mktydWmUM
[241] https://www.techdirt.com/articles/20120405/17321218398/google-maps-exodus-continues-as-wikipedia-mobile-apps-switch-to-openstreetmap.shtml
[242] https://www.mapbox.com/data-platform/
[243] https://osmcha.org/
[244] https://labs.mapbox.com/mapping/validating-osm/
[245] https://www.youtube.com/watch?v=gxCEb5Cv4Nk
[246] https://www.youtube.com/watch?v=BwMM_vsA3aY
[247] http://hot.openstreetmap.org/
[248] http://mapkibera.org/
[249] http://blog.ilabamericalatina.org/2013_06_01_archive.html
[250] http://bitterscotch.wordpress.com/2010/04/29/mapping-a-new-way-forward-for-openstreetmap-in-india/
[251] http://www.opencyclemap.org/
[252] http://openskimap.org/
[253] http://www.wheelmap.org
[254] http://www.gis.cwu.edu/phillyfood/
[255] http://wiki.openstreetmap.org/wiki/Food_security
[256] http://extract.bbbike.org
[257] http://wiki.openstreetmap.org/wiki/Overpass_API
[258] http://overpass-turbo.eu/
[259] https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FMap_features&data=05%7C01%7Cpmg5371%40psu.edu%7C7820a0df957d466d5ead08da5dcb4a15%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C637925424940276266%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m3RmUgrFdRTI%2BKb9cjcGSpeCc2sorqDWp9zTig3V7uE%3D&reserved=0
[260] https://www.e-education.psu.edu/geog585/743
[261] https://kaltura.psu.edu/
[262] http://screencast-o-matic.com