GEOG 865
Cloud and Server GIS

GIS databases and web editing


GIS vector datasets come in many formats. Some of these are better suited to web editing than others. Since we are working with ArcGIS Server in this exercise, we'll talk about some of the data formats that Esri offers and which ones are required for web editing. You'll then load some GIS data into a database on your EC2 instance.

Whether you're working with Esri software or not, one of the most ubiquitous formats for exchanging GIS datasets is the shapefile. This is a data format developed and openly documented by Esri, meaning that other software companies are allowed to use, create, and share shapefiles. A shapefile actually consists of multiple files with the same root name and different suffixes (.shp, .dbf, .prj, etc.) that store the data's geometry, attributes, projection information, and so on. You'll often see shapefiles available on GIS data warehouse sites that allow you to browse and download geographic datasets.

A shapefile is handy for exchanging data, but it's not very useful for web editing. Because the shapefile is an openly documented file format, it may be possible for a web developer to write an application that edits shapefiles. However, this would be a significant amount of work and ArcGIS does not supply out-of-the-box web editing functionality for shapefiles. Nor does ArcGIS support web editing with the shapefile's more advanced (but less openly documented) cousin, the file geodatabase.

In order to perform web editing with ArcGIS Server, your data must be stored in a relational database management system (RDBMS) and accessed through ArcSDE. Here's what those terms mean:

  • An RDBMS is a heavy-duty database used by enterprises to store large amounts of data. This includes GIS data, but RDBMSs also store many other types of datasets. Any time you apply for a driver's license, fill out a hospital admittance form, or buy something online, your information is probably getting pushed into an RDBMS of some sort. Common RDBMSs include Microsoft SQL Server, Oracle, and the open source PostgreSQL. In this course, you'll use a scaled down version of an RDBMS called SQL Server Express that is offered for free by Microsoft and included in the ArcGIS Server Amazon Machine Image (AMI).
  • ArcSDE is Esri technology that allows data stored in an RDBMS to be easily used within ArcGIS. It provides features such as versioning and replication that allow you to maintain different branches and copies of your data to accommodate enterprise workflows.
    ArcSDE comes in Workgroup and Enterprise levels. You'll use the Workgroup level in this course because it's designed to work with SQL Server Express. You might have noticed that Cloud Builder offers the choice to use a SQL Server Standard database (or PostgreSQL when you build sites on Linux) but that is beyond the scope and cost of this course.
    In recent years Esri has attempted to downplay the "ArcSDE" name. The ability for ArcMap and other applications to connect to RDBMSs is available without installing any separate component, as long as you have the database client software installed on the connecting computer. If you don't have the client software installed, you must install and license the ArcSDE application server component on the machine running the database.

Why are these things required for web editing with ArcGIS Server? One thing you have to consider is that when you configure editing on the web, you may not want to expose your main production database to everyone on the network. Your data is valuable. You may have spent thousands of dollars collecting it. It may be required to meet certain quality standards. To protect your data, you'll probably choose to expose a copy, or replica, of it for web editing. This replica goes on your EC2 instance. You'll keep a separate replica of the data in your on-premises environment. This on-premises replica can be protected by your firewall, data quality checks, and so on.

From time to time, you can synchronize the two replicas using ArcGIS software tools. This means that one replica gets sent the changes that were made to the other replica, and vice versa. ArcGIS Server even provides a special type of web service for synchronizing two replicas, called a geodata service.

Other terms you may see during this lesson include geodatabase and feature class. Geodatabase is an Esri-coined term to describe a database containing related GIS datasets, tables, relationship classes, topologies, and so on. A feature class is a vector dataset within a geodatabase. In the following section you'll create a geodatabase in SQL Server Express and load in some feature classes that you've copied to your EC instance.

Creating a geodatabase in SQL Server Express

Let's load some data onto your EC2 instance and prepare it for web editing. Your whole goal is to make a map on your instance and expose it through an ArcGIS feature service, which is the type of service that you can edit over the web. The first step is to get the data onto your instance and load it into SQL Server Express. The Amazon Machine Images (AMIs) used behind the scenes by Cloud Builder already contain a SQL Server Express database.

  1. If necessary, use Cloud Builder to start your ArcGIS Server site.
  2. Log in to your instance through Windows Remote Desktop Connection.
  3. Download the Bighorn Sheep data to your EC2 instance and extract it so that the datasets lie immediately under D:\data\BighornSheep. You can choose to download the data onto your local computer and copy it to the cloud using Remote Desktop (like we did in the previous lesson) or you can try to download it directly onto the instance through this web page.

    The data is a map document and a bunch of shapefiles showing Rocky Mountain bighorn sheep habitat and sightings in Carbon County, Utah. These were obtained from the State of Utah Automated Geographic Reference Portal (AGRC) except for the sightings, which are fictional.

    You will prepare a feature service out of this data and create a web editing application that people can use to report bighorn sheep sightings. You'll also see how to expose the habitat boundaries for web editing.
  4. On your EC2 instance, start ArcMap with a new empty map and make sure the Catalog window is displayed (click the Windows menu, then Catalog).

    The Database Servers node of the Catalog tree is where you work with SQL Server Express. The Database Connections node is where you work with full-featured enterprise databases such as Oracle or SQL Server Standard.
  5. Expand the Database Servers node of the Catalog tree. Double-click the connection to a database server called LOCALHOST.GDS. This connects to ther SQL Server Express database that was preconfigured on this instance.This database server has some precreated databases in it named egdb and geodata that come from the ArcGIS Server AMI.

    If you right-click LOCALHOST.GDS and click Permissions, you'll see that the ArcGIS Server account has been given permissions to your database server. This is also preconfigured on the instance.

    If you don't see LOCALHOST.GDS then double-click Add Database Server, enter the Database Server name localhost and click OK. (This is per the Esri instructions when working with this AMI. Usually when connecting to SQL Server Express you type something like myserver\sqlexpress or localhost\sqlexpress)

    Typically you would create a new geodatabase underneath LOCALHOST.GDS and put your data in it, but we'll just use the precreated database named 'geodata' due to the small amount of data we're using.
  6. Right-click your geodatabase 'geodata' and click Import > Feature Class (multiple).
  7. For Input Features, browse to the lesson data folder you extracted at D:\data\BighornSheep and select all the shapefiles contained therein. (You may need to add a folder connection to the D: drive.) Use the Shift key to help you select multiple shapefiles.
    screen shot. Selection of Shape Files.
    Figure 3.1 Selection of Shape Files
  8. Click Add and then click OK.
    You won't see anything except a small status bar in the lower-right corner of your ArcMap window. ArcGIS is using one of its geoprocessing tools to import the datasets into the geodatabase, and ArcGIS 10 introduces the concept of background processing that allows you to continue using ArcMap while a geoprocessing job is running. After a few minutes you should see a small success message appear in the lower-right corner temporarily. If you miss the message, or if you just want to see how things are going, you can click Geoprocessing > Results and expand the tree to see messages about the current job.

    When the job completes successfully, you should see your datasets appear in the Catalog window under "geodata".

    In the end, you are going to allow editing for the sightings and sheep habitat layers. Now that you are accessing your data through ArcSDE, you are required to register these datasets as versioned before you can edit them. Versioning is an ArcGIS feature that allows you to have multiple working versions, or branches, of your dataset available. Edits made to these versions can then be incorporated into the master database as needed. Versioning allows you to have multiple editors working on a dataset at the same time.

    Since you're required to register the datasets as versioned, perform the following steps.
  9. In the Catalog tree, find the datasets you just imported. Right-click your dataset named <Database name>.DBO.Sightings and click Manage > Register As Versioned.
  10. Check the option Register the selected objects with the option to move edits to base, then click OK.
  11. Repeat the above two steps to also register <Database name>.DBO.RockyMountainBighornSheep as versioned. This is the habitat layer.

    You don't need to do any more work with versioning beyond the above steps. However, if you want to learn more about versioning, you can browse A quick tour of versioning in the ArcGIS Desktop Help.

    Now that you've got the database all set up, you'll register it with ArcGIS Server.
  12. In the Catalog window, find the GIS Servers node and make sure you are connected to your ArcGIS Server (this requires double-clicking the node if you see a little red x). Then right-click your ArcGIS Server connection, and click Server Properties.
  13. Click the Data Store tab, and in the top section Registered Databases, click the plus (+) button.
  14. Enter a Name such as Carbon County sheep database and click the Add button.
  15. Enter the information as follows:
    • Database Platform - SQL Server
    • Instance - localhost
    • Authentication Type - Operating system authentication
    • Database - geodata

    Then click OK.

  16. Ensure that Same as publisher database connection is checked, then click OK. However, please note that if you were going to be doing any service publishing from your home computer, or some computer other than the server, you would add database connection information in this lower section of the dialog box representing the database seen by the publisher's machine. Don't try this with this lesson. 
  17. Click Validate All - and make sure you see a green check box next to your newly registered database. Click OK. If you don't see a connection, right-click your database geodata (VERSION:dbo.DEFAULT) and click Save Connection. This should make it appear in the Database Connections folder with a .sde file extension.

Your data is now loaded, prepared, and registered with ArcGIS Server. In the next section, you'll start working with some maps that use this database. You will prepare them to run as feature services that can be edited over the web.