By the end of Lesson 2, you should be able to:
Roadway data are fundamental to GIS-T and many of the most important transportation modes (e.g. highway, transit, bike). Many GIS functions and analyses rely on it including geocoding and network analysis, both of which we’ll take a close look at in the next few lessons. Roadway data also play an important role in mapping and visualization for many GIS applications.
There are a number of commercial and public sources of street data and services which are available. Some are public and freely available, and others are commercial. In this lesson, we’ll take a look at some of the most widely used sources of street data.
TIGER is a data source produced and published by the U.S. Census Bureau. These data include street data which can be used to perform geocoding or to produce a street network. TIGER data were used as a “seed” for many of the other roadway data sources, both public and commercial. We will take a closer look at TIGER data later in this section.
OSM is a rapidly growing Volunteered Geographic Information (VGI) project which got its start in 2004 and is sponsored by the OpenStreetMap Foundation. For U.S. roads, OSM initially used TIGER Line files but many updates have since been made based on input from its volunteer community which is now over a million strong. In some parts of the world, OSM data are as good, or nearly as good, as its commercial counterparts.
State-level transportation agencies have long maintained road centerline networks as well as additional networks for other modes. They have been improved greatly in accuracy and precision, and agencies are increasingly adding local and private roads and associated data. Much of this latter impetus is due to increased federal requirements for data collection and reporting. In most cases, these networks are the most complete and accurate product for network features and associated attributes for any given state.
TFTN is an evolving governmental initiative from the National States Geographic Information Council (NSGIC) and USDOT that originated in 2008. TFTN will initially be a road centerline dataset that may replace overlapping federal efforts and products. A set of centerline datasets has been created as part of state DOT submittal requirements for FHWA’s Highway Performance Monitoring System (HPMS). The next step is to try and join these across state lines.
Tele Atlas was founded in 1984 and was acquired by TomTom in 2008. Tele Atlas data was primarily collected from its own mapping vans. The company’s road products are decreasing in importance and usage.
Founded in 1985 and acquired by Nokia in 1991, NAVTEQ (now renamed HERE) operates independently and partners with third-party agencies and companies to provide its networks and services for portable GPS devices made by Garmin and others, and Web-based applications including Yahoo! Maps, Bing Maps, Nokia Maps, and MapQuest.
ESRI does not produce road data directly but instead acquires it from HERE, TomTom, and others and repackages it. ESRI StreetMap covers North America and is part of Data and Maps which is included with ArcGIS. StreetMap premium has more current data than StreetMap and also has coverage for Europe.
Google has become a major provider of mapping services. Google doesn’t make its street data available directly but instead uses it to provide services. These services are provided through products such as Google Maps, Google Earth, and various APIs. In 2008, Google released a tool called Google Map Maker to encourage individuals to submit or correct feature information. This is similar in concept to the manner in which OSM derives much of its data. Google retired Map Maker in 2017 in favor of its "Local Guides [1]" program. As a "Local Guide," you can contribute reviews of businesses or places, upload photos and suggest a new place. Recently, they also began to add capabilities to allow users to report issues with roadway geometry and missing roads. Local Guide contributions are all made directly in the Google Maps interface. Take a look at these comparisons between OSM and Google in regards to the services [2] they provide and their user contributions [3] programs. One should note these comparisons are published on the OSM Wiki site so they may be a bit biased.
Also, check out this map comparison tool [4] made available by Geofabrik, an organization who promotes OSM and provides a portal for downloading OSM data extracts. Select an area you are familiar with, and compare the OSM map, the Google map, and the HERE map.
In this section, we'll take a closer look at two of the most extensive sources of publicly available roadway data: TIGER and OSM.
The TIGER database was first created in preparation for the 1990 decennial census. In creating TIGER, not only did the Census Bureau produce the first nationwide map of roadways, it also incorporated topographical context which defined the relationship between road features as indicated in its name: Topologically Integrated Geographic Encoding and Referencing database.
In addition to the TIGER spatial database, the Census Bureau also created a Master Address File (MAF) which is a database of all known living quarters in the U.S. The MAF contains about 300,000 addresses which are identified as location addresses, mailing addresses or both. In addition, the MAF contains a record for each living unit which can correspond to a separate structure or a residence within a shared structure. There are about 200,000 living units in the MAF some of which have multiple associated addresses. Following the 2000 decennial census, the Census Bureau decided to merge the two databases into a single database known as the MAF/TIGER Database (MTdb).
The Census Bureau is planning a 3-part informational series on TIGER to commemorate its 25th anniversary. Part 1 will examine the history of TIGER, Part 2 will address efforts to improve its accuracy, and Part 3 will address the tools which provide access to the data. To date, only Part 1 of the series [5] has been made available. Spend a few minutes looking through the document to learn a little about TIGER’s history.
The TIGER data is available in a number of formats [6] including Shapefiles, geodatabases, and KML files. The Census Bureau also provides a tool called TIGERweb [7] which allow online viewing and the ability to incorporate TIGER data directly in GIS applications via web services including an OGC standard Web Mapping Service (WMS). For the exercises in this and the upcoming lesson, we will be working with the TIGER/Line shapefiles [8].
The Tiger/Line shapefiles are available for multiple years. Each year, the Census Bureau provides an updated set of Tiger/Line shapefiles in addition to associated technical documentation. The technical documentation for the 2017 Tiger/Line shapefiles can be found here [9]. It is over 120 pages long and serves as an excellent reference for the Tiger/Line Shapefiles.
With more than 3 million registered users, the OSM project has a huge community behind it. Consequently, there is plenty of documentation available for learning about the project and becoming a member of the community. A few good resources for learning about OSM are the Open Street Map Wiki [10]and the guides on LearnOSM.org [11].
OSM data is natively available in a unique file format (i.e., .osm files). However, many of the sites which provide access to OSM data serve it up in commonly used formats like shapefiles. For example, take a look at Geofabrik’s OSM data download page [12]. Also, take a look at the first few sections of the OSM Data Guide [13] which describe the .osm file format and some options for acquiring OSM data.
We often talk about spatial data in terms of points, lines, polygons, and attributes. OSM, however, uses the terms nodes, ways, relations and tags. In order to develop some understanding of these terms, take a look the descriptions of OSM data’s elements [14]on the OSM Wiki site.
In preparation for this week's webinar, you learned about the geographic areas the Census Bureau uses to tabulate and disseminate data. This week, you’ll explore the Census Bureau in greater detail. The Census Bureau is part of the U.S. Department of Commerce. The mission of the Census Bureau is to “serve as the leading source of quality data about the nation's people and economy.” To fulfill its data gathering objectives, the Bureau conducts both decennial censuses and a continuous survey known as the American Community Survey (ACS). The ACS was born in 2005 out of a need for more up-to-date information than the decennial census provided. Data from both the decennial census and the ACS are made available in a variety of ways, one of the most popular of which is the via the American FactFinder site [15].
Take a look through the American Community Survey Information Guide [16] which the Census Bureau updated in December 2017.
Data collected by the Census Bureau serve some critical functions. These data are used to:
Geography and GIS are very important to the Census Bureau.
Watch this brief presentation on the Maps of the US Census Bureau [17] (5 minutes) by Atri Kalluri, Assistant Division Chief of the US Census Bureau.
Census data have long been applied to transportation planning and research. Today, there are a number of emerging sources of data which serve to compete with or complement the role of the census data in these fields. Read this 2017 paper [18] by Gregory D. Erhardt (University of Kentucky) and Adam Dennett (Centre for Advanced Spatial Analysis, University College, London) which examines this topic.
The Census Transportation Planning Products Program (CTPP) [19] is an initiative led by the American Association of State Highway and Transportation Officials (AASHTO). AASHTO is an organization we’ll take a closer look at in an upcoming lesson. The CTPP provides special tabulations of Census data which are of particular interest to transportation planners. These datasets provide insight into how people commute and which modes of transportation they use. They are often used to validate travel demand models which themselves are used to make decisions on what types of transportation projects are needed to support regional needs, including those related to economic growth, public health, transit needs, and highway safety issues (for a quick overview of the Four Step Model (FSM) which commonly used in travel demand modeling, see this 2007 article by Michael McNally at the University of California, Irvine [20]).
To facilitate the use of the CTPP data, AASHTO created a web-based application [21]to examine travel flows. The CTPP even has a YouTube channel devoted to teaching people how to use the software (although the quality of the videos is less than stellar). Take a look at the YouTube video below (5 minutes) which shows how to generate some basic county to county commuter flow data. The CTPP data analysis tool also has the ability to display results in a variety of formats including thematic maps.
Hi, this is Penelope Weinberger. I'm the Census Transportation Planning Products Program Manager at ASHTO, recording some brief tutorials on the CTPP data access software. The tutorial you're about to watch is on selecting geography. There are two parts to the CTPP, residence and workplace, and there are two ways to select geography, by list and by map. We're going to look at both of those.
The CTPP data access software is a powerful tool to access the nearly 350 gigs of data provided by the Census Bureau. The dataset consists of almost 200 residence-based tables, 115 workplace-based tables, and 39 flow tables from (inaudible), 325,000 geographies. The data is derived from the American Community Survey Microdata record based on the 2006-2010 ACS. Looking at here is the home screen for the CTPP data access software. I'm not going to select a table. I'm going to go straight to selecting geography. As you can see, I have Residence geography by the red box and Workplace geography by the blue box. The default geography for all CTPP tables is States. We're going to change that right now. I'm going to click on Residence, and the software is going to open up and show me. On the left-hand side of the screen, I'm looking at my select level. State is what's highlighted, and States are what are selected. I have 52 states selected, that includes DC and Puerto Rico. Like all good GISs, I'm going to have to clear my selection if I don't want it in the table. So, the first thing I do is hit clear full selection. Then, I have to decide what level of geography I'm interested in. I'm interested in counties. There are 3,221 counties in the US, and I have none of them selected. So, first, I pick my level, and then it's gonna give me a list starting in Alabama. Well, I don't want to scroll all the way down from Alabama to Maryland, so I'm gonna search for Maryland, instead. I put my cursor in the search box, and I type Maryland, and then click on the search tool and also hit enter.
In the CTPP, you can have mixed levels of geography. This tutorial is just going to look at counties. Now, I have my 24 counties in Maryland. I'm gonna choose to select all of them, and I click the Select All button on the right-hand side, check marks by each one. Pretty Nifty! Now, I want to pick my workplace geography. I'm actually gonna pick the same geography. I do want you to take note that where it said all states before, now it says new set. If I want to save this set of just Maryland counties or any geography I create, we'll have to sign in, but I'm not going to do that today. Now, I'm just gonna click on workplace, and instead of picking by list, I'm gonna pick by map. So, instead of using the selection list tab, I'm going to use the selection map tab. Click on that, and it shows me a cool map of the United States. Of course, I have all my states selected since that's my default. So, I'm going to clear the selection. On the map, you do that over on the right-hand side with the little garbage can. (...they go!) Now, I pick my level. I want counties again. I want place over counties.
Now, I could do this a number of ways. I could zoom in with my tool just to where I happen to know Maryland is, and I could pick the counties one by one. That could be a little bit tedious. So, instead, I'm going to use this cool Zoom To and Select tool. Place overstate is what I want cause that is the parent. Down here, it says automatically highlight any place over County. I click that on, I type in Maryland. Hit Enter. It's loading up my Maryland counties. Now, look, 24 places over counties. It's good those two numbers match. Do I want to add all highlighted counties to my selection? I do. So, I click that, and there they all are.
Now, let's see if I can look at a table with my county residences and my county workplaces. Show CTPP tables. Of course, I want a flow table since I've got two geographies selected. Workers, let's just look at total workers. Now, one thing that's a little odd about this table, is that the residence and the workplace are both on the [INAUDIBLE], so I'm going to move one of those so that I have a matrix. I like my data as is. I'm going to grab my Residence by the textured box to the left of the word, and I'm going to drag it up till it is over output with that up-pointing arrow. I'm going to drop it. Now my output is going to nest on the Residences. It's a great-looking matrix. I have 25,000 workers that live in Allegheny and that work in Allegheny County. 153,000 live in Anne Arundel and work in Anne Arundel. It should go right down that way, the biggest numbers now in the matrix. Sure looks like it does.
Another interesting use of the commuter flow data can be seen in an application created by Mark Evans. Mark used the Google Maps API to create a GIS application called Commuter Flows [23] which facilitates the visualization of census tract level commuter flows derived from the ACS data.
This week, you will get to know a little about each other by reviewing the video autobiographies posted by your classmates in Lesson 1. You’ll also have a one-on-one chat with one of your classmates as per the schedule you were provided at the end of Week 1. The discussion should be at least 30 minutes in length. If it’s the first time you’ve chatted with each other, spend the majority of time getting to know each other. Otherwise, focus on discussing the lesson content.
Our speaker will be Dr. Ira Beckerman. Ira has been the Cultural Resources Section Chief for the Bureau of Design at PennDOT since 1998. Trained as an archaeologist (Ph.D. Anthropology, Penn State, 1986), he has worked as a field archaeologist in Mexico, Tennessee, North Carolina, and Pennsylvania. His 22 years of transportation experience is split between PennDOT and (previously) the Maryland State Highway Administration. Dr. Beckerman’s research interests include archaeological predictive modeling, pre-contact Eastern North America, and GIS. He is a member of the Society for American Archaeology, Register of Professional Archaeologists, and the Transportation Research Board’s Archaeology and Historic Preservation Committee, and has served on panels for TRB and the American Association of State Highway Transportation Officials (AASHTO). Dr. Beckerman was a 2001 recipient of the PennDOT Star of Excellence. His group also recently led an effort to develop a predictive model for archaeological sites which serves as a valuable tool for screening projects early in the planning process.
The National Historic Preservation Act (NHPA) was passed into law in 1966. The purpose of the law is to protect historic and archaeological sites of significance. One outcome of the NHPA was the creation of the National Register of Historic Places (NRHP), which is a list of districts, sites, and structures deemed worthy of protection. There are more than 1 million properties currently on the list and about 30,000 additional properties are added annually. Section 106 of the NHPA specifically requires historical and archaeological sites to be assessed for impact as part of any federally funded project.
To gain a better understanding of the Section 106 process, take a look at A Citizen’s Guide to Section 106 Review [24], a brief overview put together by the Advisory Council on Historic Preservation. Also, watch the following video which describes the process which agencies need to follow to comply with Section 106 of NHPA.
There are several areas of environmental law that local public agencies, or LPAs, might encounter on a Federal-aid project. These areas address a project's effects on: the natural environment; things like air and water quality, wetlands, wildlife or endangered species, the social environment; things that affect our quality of life, like the displacement of homes or businesses or community cohesion impacts, particularly as they relate to minority and low-income populations, historic sites, and parks and recreation areas. The National Environmental Policy Act, known as NEPA, provides a framework for environmental analyses, reviews, and series of discussions known as consultations. NEPA's process "umbrella" covers a project's compliance with all pertinent Federal environmental laws. While NEPA provides a coordinated environmental review process, the related environmental law specifies what an agency must do to comply with its particular requirements, which can vary widely. One such law, Section 106 of the National Historic Preservation Act, requires Federal agencies to consider the effects of their projects on historic properties. A historic property is any prehistoric or historic district, site, building, structure, or object that is included in or eligible for inclusion in the National Register of Historic Places. The Advisory Council on Historic Preservation and the State Historic Preservation Officers, or SHPOs, administer the federal or state historic preservation program. The National Historic Preservation Act does not mandate preservation of historic properties; however, if your project receives Federal-aid funding, your agency must participate in a consultation process that considers the effects of your project on those properties. Depending on the project, consulting parties may include: the Advisory Council on Historic Preservation, your SHPO, the Tribal Historic Preservation officer, Federally recognized Indian tribes and Native Hawaiian Organizations, local governments, and the public. Let's take a look at the consultation process and the responsibilities of agencies in complying with the National Historic Preservation Act. While the consultation process is somewhat iterative, there are four basic steps: initiate consultation, identify and evaluate historic properties, assess effects, and resolve effects To initiate consultation, a local agency typically sends a letter of correspondence to either the SHPO or the State’s department of transportation, or State DOT, identifying the project, the project location, and who is involved. And if known, an agency will also identify any historic properties in the project area. In order to properly identify and evaluate any potential archeological and historic sites, an agency needs to utilize a qualified employee or hire a consultant. The evaluator will begin by performing file and literature searches to determine if the property is on or potentially eligible for the National Registry of Historic Places. The resulting report summarizes the site's setting and historical context, project information, including maps, and any cultural resources that were identified. If there are no historic properties in the project area or no adverse effects are likely to occur to ones that are present, an agency will issue a letter with its findings, which completes the consultation process. On the other hand, if adverse effects are likely to occur, an assessment must be made. An adverse effect occurs when an element of a historic property is altered in a manner that diminishes the integrity of the property’s location, design, setting, materials, workmanship or association. For example, noise or visual blight can be considered an adverse effect. The public and the consulting members of the project team can help determine any adverse effects resulting from the project. Once potential adverse effects are identified, an agency needs to evaluate possible alternatives or modifications that would help resolve the effects by avoiding, minimizing, or mitigating them. For example, the alignment of a road might be altered to avoid affecting a historic property, or a commemorative publication or plaque might be erected. The process concludes when the agency finalizes a memorandum of agreement between consulting agencies. To illustrate the process, let’s consider a drainage and road improvement project in a town we’ll call Old Towne, which was established in the early 19th Century. To initiate the consultation process, the project manager sends a letter to the SHPO describing the project and providing a map of the project area. The city of Old Towne hires a consultant with recent experience developing similar studies for the State DOT. During the course of her literature review, she discovers evidence of an early 19th-Century tavern at the corner of 1st and Main. During a site visit, she confirms that there is no longer any evidence of the tavern from the ground's surface. The project team then conducts a subsurface investigation of the area and finds the foundation of the tavern and an outhouse. Further excavation indicates that the site might yield important information about the lifestyle of Old Towne’s earliest residents. Unfortunately, given its proximity to the street, the drainage and road improvement work would destroy the archeological site. To resolve the adverse effects, the project team consults with the SHPO, the local historical society, the State DOT, and the Federal Highway Administration’s division office. After the artifacts are analyzed and a report is prepared, the team agrees to erect an interpretive sign describing what was found and to give any recovered artifacts to the county museum. With everyone satisfied with this arrangement, the project team concludes the consultation process by writing a memorandum of agreement. As we have seen, as a sponsor of a federally-aided transportation project, your agency may be required to conduct studies and coordinate with other parties. The necessary activities and degree of your involvement depend on the nature of your project and your State DOT’s practices. Your State DOT can help you navigate the requirements and develop approaches that adequately evaluate and address your project’s effect on historic properties.
Many states have developed GIS-based systems to help state agencies and other interested parties identify historic properties and known archaeological sites and assess their proximity to planned projects. Spend some time exploring Pennsylvania’s Cultural Resources Geographic Information System (CRGIS) [26]. CRGIS was created and maintained through a collaborative partnership between the Pennsylvania Historical & Museum Commission (PHMC) and PennDOT. (note: CRGIS requires that pop-ups are allowed ... so you'll have to enable them in your browser either in general or for this site specifically. You can disable them again when you're done exploring CRGIS. Instructions for adjusting pop-up settings in Internet Explorer/Edge and Chrome are included in the "Getting Started" link in the upper right portion of the CRGIS website.)
A number of efforts have been undertaken in recent years to use predictive modeling to identify locations which are likely archaeological sites. In Pennsylvania, FHWA, PennDOT, the Pennsylvania Historical and Museum Commission (PHMC) and URS Corporation partnered to create such a predictive model which is described in brief here [27]. The model uses data from known archaeological sites together with spatial algorithms to rank areas based on their likelihood of having artifacts. These data are then used in evaluating potential projects and alternatives.
In this lesson, we discussed the importance of roadway centerline data to GIS and explored some of the public and commercial sources for this data. In particular, we took a close look at the Census Bureau’s TIGER data in addition to the data available via Open Street Map (OSM).
Our transportation organization of the week was the Census Bureau. We spent some time learning about the American Community Survey and the role that Census data plays in transportation planning.
In preparation for next week's webinar, we reviewed the role of Section 106 of NHPA in transportation projects and explored a cultural resources GIS application.
Finally, you had the opportunity to learn a little bit about your classmates by reviewing their video autobiographies and by having your first one-on-one conversation.
If there is anything in the Lesson 2 materials about which you would like to ask a question or provide a comment, submit a posting to the Lesson 2 Questions and Comments discussion. Also, review others’ postings to this discussion and respond if you have something to offer or if you are able to help.
Links
[1] https://maps.google.com/localguides/howto
[2] http://wiki.openstreetmap.org/wiki/Comparison_Google_services_-_OSM
[3] http://wiki.openstreetmap.org/wiki/Google_Maps_user_contributions
[4] http://tools.geofabrik.de/mc/#15/49.0094/8.3902&num=4&mt0=mapnik&mt1=google-map&mt2=here-map&mt3=mapnik-german
[5] http://census.maps.arcgis.com/apps/MapJournal/index.html?appid=2b9a7b6923a940db84172d6de138eb7e
[6] https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.html
[7] https://tigerweb.geo.census.gov/tigerwebmain/tigerweb_main.html
[8] https://www.census.gov/geo/maps-data/data/tiger-line.html
[9] https://www2.census.gov/geo/pdfs/maps-data/data/tiger/tgrshp2017/TGRSHP2017_TechDoc.pdf
[10] http://wiki.openstreetmap.org/wiki/Main_Page
[11] http://learnosm.org
[12] http://download.geofabrik.de/
[13] http://learnosm.org/en/osm-data/data-overview/
[14] http://wiki.openstreetmap.org/wiki/Elements
[15] http://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml
[16] https://www.census.gov/content/dam/Census/programs-surveys/acs/about/ACS_Information_Guide.pdf
[17] http://www.esri.com/videos/watch?videoid=2182&isLegacy=true&title=the-maps-of-the-us-census-bureau
[18] http://onlinepubs.trb.org/onlinepubs/conferences/2017/censusdata/KeepingCensusRelevant.pdf
[19] https://ctpp.transportation.org/
[20] https://escholarship.org/uc/item/0r75311t
[21] https://ctpp.transportation.org/2012-2016-5-year-ctpp/
[22] https://www.youtube.com/channel/UChNXwjJAIWUJ-VdMP9B-76Q
[23] http://bigbytes.mobyus.com/commute.aspx
[24] https://www.achp.gov/digital-library-section-106-landing/citizens-guide-section-106-review
[25] https://www.youtube.com/channel/UClOFVzt6KnhH1cRm17LxBgA
[26] https://www.dot7.state.pa.us/CRGIS/
[27] http://pahistoricpreservation.com/mapping-probability-pre-historic-archaeological-sites/