In this walkthrough, you will use QGIS to create a general-use basemap of Philadelphia. The idea is that you will be able to overlay thematic layers on top of this map in lessons to come. The data for this walkthrough is the base data for Philadelphia that you preprocessed in Lesson 3. If you followed all instructions, you should have this data in a folder called c:\data\Philadelphia or something similar. It should be using the mercator projection EPSG:3857.
It would also be a good idea to save the project now!
Once you've finished the map design phase and your map looks good at each scale, you can start thinking about generating the tiles. For large maps, plan for this to take some time, taking into account some of the factors discussed earlier in the lesson, such as the shape of the map and the scale levels you choose to generate.
This section of the walkthrough explains how to generate tiles of your Philadelphia basemap, and host them on the web. You will then test them out in a web page.
We are setting the minimum zoom level to zero here, so that tiles will be provided all the way up to the top of the pyramid in the tiles hierarchy. However, often when working with a relatively small project area like this (e.g., in your term projects), your plan is to limit how far out the user can zoom to avoid that the displayed area becomes tiny or (almost) invisible. In that case, it makes sense to set the minimum zoom option accordingly and use Javascript programming (see next few lessons) to restrict the possible zoom levels. That said, in terms of storage space, you won't save much by excluding the lower zoom levels because almost all storage will always be taken up by the highest levels.
If you navigate around your folder of tiles, you'll notice that the images are extracted into a highly organized structure of level\column\row. This structure is understood by various mapping programs and APIs, so all you have to do at this point is put your tiles onto a web-facing server. If you already have acces to some web space, you could simply upload the folder with the tiles to that web server. What we will instead do in this course is use Amazon's cloud services, namely the S3 service for scalable storage, as a convenient place to host the tiles on the web. We will place the tiles there and then test them in a web map. However, this will require you to set up an Amazon AWS account first, including registering a credit card. Uploading and accessing the tiles over the web will then cause some costs that your credit card will be billed for at the end of each month but these will be very small, < $10 most likely, if you just use the tiles for this class and follow the instructions below. The steps below will include descriptions of how to monitor the costs and how to set up a billing warning if the accumulated monthly costs go above a certain level. Please note that the workflow below is very specific to S3. If we would instead use a normal web server to host the tiles, we would most likely use FTP or SCP to upload the tiles to some folder in our personal web space on that server.
Go to: https://portal.aws.amazon.com/billing/signup [1]
Create a Root user account using your PSU email. (If you already have created an AWS account for your PSU email, e.g. for another class, you can instead log into that account and continue with Step 3.)
You can give your account any name you want. You will then have to verify your email address and define a Root user password. After that, you have to enter your contact information and confirm your identify by entering a verification code sent to you via SMS or an automatic call to the phone number you provide. Lastly, you have to select a support plan; please choose the free "Basic support plan". When done setting up your account, you can click the button that leads to the "AWS Management Console".
Now, in the AWS Management Console, go to the search bar at the top of the screen and search for Billing . Click on the link (the name of the service at the top) to get to the "AWS Billing Dashboard".
The Billing Dashboard will show you both a cost estimate for where you will end up at the end of the month and the current costs accumulated so far this month ("Current MTD balance"). Further down below, there is a link called "View your bill" which will lead you to a page where you can explore the costs accumulated so far by unfolding the items at the bottom under "Amazon Web Services EMEA SARL charge by service".
Now type in the term Budget in the search bar at the top and select the sevice called "AWS Budgets" by clicking the name. We now want to set up a budget so that you will get alerted by email if the estimated or accumulated costs for a month pass a certain threshold.
Click on the orange button "Create a budget" at the top. On the following page, select the options "Use a template (simplified)" and "Monthly cost budget", then fill out the details for the budget as shown in the image below but using your own email address to which the warning should be sent. The $5 is just a suggestion. Feel free to use a somewhat higher or lower number. You can also change the name of the budget but the default name is fine. Lastly, click the orange "Create budget" at the bottom.
As the information text said, you will now receive messages if any one of the following conditions is true: 1) your actual spend reaches 85% of the budget you defined, 2) your actual spend reached 100%, or 3) your forecasted spend is expected to reach 100% of the budget. You can alway go back to the AWS Budgets page to select and modify you budget there, including changing the threshold value. However, here is one important final note regarding the Billing and Budgets dashboards: For some reason, AWS does not update the accumulated costs in real-time. It can actually take multiple hours until the numbers get updated. Nevertheless, this should still help you keep track of how the costs are slowly increasing while you are using the data in your S3 bucket.
In the AWS Management Console, go to the search bar at the top of the screen again and now search for S3.
Click at the first result that is indeed the S3 service.
Once in the S3 service, click the orange button to create a bucket (basically a storage container inside your S3 cloud storage).
Under "General configuration", give your bucket a unique name, we recommend starting with "geog585-" followed by some random string of character and/or numbers that you pick yourself. We here will use the name geog585-a1rt15 .
If you want you can select the region that is closest to you, or in case of a real "global" project the region closest to your clients.
Scroll down to where it says Block Public Access settings for this bucket.
Usually we want to protect our data, but in this case, the map tiles have to be fully accessible by everyone, so uncheck the first tick to allow access.
Tick the box where you acknowledge that the bucket will become public.
Leave the rest as it is, and scroll down to click Create Bucket.
Once back to the main page, you should see your new bucket in the bucket list. Click on the name to open it up.
You should now see the page for your bucket, currently still not containing any objects. Click the tab called Permissions at the top.
Scroll down to Bucket Policy and click Edit. Amazon really does not want us to create a public bucket unintentionally, so we need to do one more thing.
Copy and paste the following in the available space:
Above the text field and its title "Policy", you can find the ARN name of your bucket. Click the small icon left to it, to copy this name. Then go to the last line of the code you just pasted to change the name of the bucket there. Paste the ARN name you copied to replace the string arn:aws:s3:::geog585-a1rt15 there. Be careful to keep the initial double quote and the /*" after the bucket name.
Hit the orange button Save Changes.
Now we are almost read to upload our map tiles. If you only want to upload a couple of files (for instance, to upload and share the web map you will create in your Lesson 7 homework assignment), you can do so directly on the AWS page for your bucket by selecting the "Objects" tab at the top (you can think of "objects" as meaning "files" here) and then clicking the Upload button. This will show an area to which you can the drag and drop folder and files from the Windows File Explorer. The AWS will then scan the files and afterwards show you another Upload button that you can press to start the actual upload. However, this approach of using the web page won't work well for basemap tile sets that tend to consist of a huge amount of small files. The Philadelphia tile set, for instance, only is 200MB large in total but it consists of roughly 21,000 files. In our tests, uplading this amount of files via the web interface took around 6 hours.
Therefore, we are going to do the upload via another piece of software, the AWS Command Line Interface (CLI), which is much more efficient for small files and should be able to upload the Philadelphia tile set in just a few minutes. To set this up, we need to do two things: (1) Create a so-called Access Key that will allow us to access our S3 bucket via CLI and (2) install the CLI software itself. Let's start with creating a Root access key...
On the AWS page, click on your account name at the very top right of the page and then select "Security credentials". Scroll to the area called "Access keys". Then click the button called "Create access key".
AWS will now show you a window called "Retrieve access key" with your new access key in plain text and the corresponding secret key hidden by default. You can think of the access key as a login name and the secret key as the password. You will need both for using CLI, so please copy both of them. Important: This will be the only time you will be able to see/copy the secret key. If you loose it, you will have to go back and create a new access key. Once you have the keys stored somewhere else, click the "Done" button at the bottom. To delete or create further access keys, you can always go back to the Access Key area under Security Credentials as in Step 21.
Now, it is time to install the AWS CLI tools: Click on this link https://awscli.amazonaws.com/AWSCLIV2.msi [2] to download the software.
Double-click the downloaded file called AWSCLIV2.msi to start the installation. Then follow the instructions given by the installer.
Now open the Windows Command Shell cmd by going to the Windows start menu and typing cmd . This should open the black shell window (see next screenshot below in Step 27).
Let's first check if CLI has been installed correctly. Type in the following command and then press ENTER: aws --version
The output should look as shown in the next screenshot below starting with "aws-cli" followed by some version number.
Next, we have to give CLI some information including the access key and secret key. Type in the following command and then press ENTER: aws configure
You will then be asked for access key, secret key, default region, and default format. For the access key and secret key, copy and paste the keys that you copied from the AWS page in Step 22. For the other two items, you can simply press ENTER without typing in antyhing.
Now we are ready to upload the files to our bucket. Use the cd command to move to the folder that contains the PhillyBasemap folder to which you exported the tiles in Steps 4+5 from the Section "Exporting the tiles" (see next screenshot from Step 29).
Now type in the following command but with the name in bold replaced by the bucket name you used and then press ENTER:
aws s3 cp PhillyBasemap s3://geog585-a1rt15/PhillyBasemap --recursive
You should now see how the tiles are uploading. If instead you get some odd error messages. Try with running the command a second time; somehow this seems to happen sometimes on the first attempt.
Once everything is uploaded, you should see a prompt in the cmd shell window. Now switch back to the AWS web page and refresh the "Objects" tab on the page for your bucket. Navigate to the PhillyBasemp folder and then further into the subfolders to one random png tile.
On the screen that opens, find the Object URL and copy it. The URL is made up of the base URL (https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap [3]) followed by the folders and file name corresponding to the zoom level and x, y indices of this particular tile image. If you want, you can open a new tab in your browser and copy the URL into the address bar to open this one tile. You should see the tile image but keep in mind that you have picked an empty tile so that you will just get an image that is fully white. To later use your basemap tiles in a web map you, you will have to take the base URL and then replace the concrete numbers at the end with a string of variabels to create a template string, for instance looking like this: https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap/ [4]{z}/{x}/{y}.png . We will make use of this option in the final part of this walkthrough in which we will test out our tiles in an ArcGIS Online map.
GOOD JOB! You are done!!
Optionally, you could add other web layers on top, but, for the sake of time, this will not be described here. In future lessons, you'll learn how to build this type of web map programmatically without using ArcGIS.com.