Overview

This course brings into the classroom the ideas and developments that surround both 3D modeling and immersive technologies such as Virtual Reality (VR). The excitement is universal in academia (for over 30 years on and off) and the private sector (more recently) with the expectation of an 80 billion dollar industry by 2025 according to Goldman Sachs (reported on CNBC [1]). It is fair to assume that the geospatial sciences will never be the same once immersive technologies, everything from Augmented Reality (AR), Mixed Reality (MR), Augmented Virtuality (AV), to Virtual Reality, have moved into most people’s living rooms. In the following, we use the term xR or 3D/xR as a combination, to refer to a range of immersive technologies that are available.

Each of the topics that you will encounter in this course could (and is) easily be approached in individual courses! The point of this course is to give students an overview of current developments in 3D modeling and xR that we will keep up-to-date to the best of our abilities, but things are moving and developing fast. Students will encounter some hands-on experience with some of the most versatile and powerful tools for modeling and creating virtual realities such as SketchUp™, Unity3D™, or ArcGIS Pro (not in the residential version). Given the tremendous advancements in 3D modeling and xR that are currently taking place, the course is designed as a timely entry-level course with the goal to give students an understanding of the game-changing nature that recent developments will have for the geospatial sciences. This is an evolving course and any feedback you might have is greatly appreciated.

Learning Outcomes

By the end of this lesson, you should be able to:

• Define 3D modeling
• Define Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Augmented Virtuality (AV) (in short: xR)
• Identify, describe, and classify opportunities for geospatial sciences that are not possible in 2D / on regular desktops
• Find examples of 360 videos
• Find examples of 3D models
• Explain the difference between models and videos
• Explain the differences between VR, AR, AV, and MR
• Detail how immersive technologies will change the geographical/spatial sciences

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

1.1 Distinguishing VR, AR, and MR Systems

To give you an idea of current developments in the field of Virtual Reality, Augmented Reality, Augmented Virtuality, and Mixed Reality, we would like you to watch the following videos and think about and reflect on the core differences in the technologies they use.

The first video gives you a quick overview of a collaboration between Microsoft and NASA using Microsoft’s Hololens. Please do feel free to dive deeper into the diverse application scenarios that are envisioned for the Hololens and related products!

Video: Mixed-Reality Tech Brings Mars to Earth (1:40)

The second video provides you with an overview of the next generation MR headsets. The new HoloLens 2 by Microsoft provides infinite possibilities for users to interact with high-quality holograms that can be in the form of animated models of objects, conversational agents, natural phenomena, etc. The term that Microsoft is using to characterize the HoloLens products is “mixed reality”. What do you think this actually means?

Video: HoloLens 2 AR Headset: On Stage Live Demonstration (10:59)

The third video is a nice way of showing experiences in VR by placing users in front of a green screen. This adds a bit more information than filming people who are experiencing VR and then communicating the experience through their reactions. You probably have seen plenty of those videos showing people of hesitating to walk over a virtual abyss they know physically does not exist, playing horror games, or experiencing a different body. The HTC Vive that is demoed in the video below, like the Oculus Rift, is a high-end head-mounted VR system that requires a relatively powerful computer.

Video: Virtual Reality - SteamVR featuring the HTC Vive (4:29)

The fourth video is a potpourri of systems that are referred to as augmented reality. It is rumored that Apple will be focusing on augmented reality technologies and you may have played Pokemon Go. It is important to note that there is a difference between Mixed and Augmented reality. In the former, virtual elements are embedded in the real environment in such a way that they provide the illusion that they are part of the environment. This illusion is enforced through virtual elements abiding with the laws of physics that govern our reality (e.g. a virtual robot that can collide with a physical table). Augmented reality, on the other hand, does not necessarily follow this principle.

Video: Augmented Reality Demo (2:15) This video is not narrated, music only.

The last video demonstrated the concept of Augmented Virtuality. The concept of AV refers to the idea of augmenting the virtual experience with elements of reality. An example of this approach could be the user of 360 degrees images and videos in a VR experience.

1.3 3D Modeling and VR in the Geospatial Sciences

"At the time of writing, virtual reality in the civilian domain is a rudimentary technology, as anyone who has work a pair of eyephones will attest that the technology is advancing rapidly is perhaps less interesting than the fact that nearly all commentators discuss it as if it was a fully realized technology. There is a desire for virtual reality in our culture that on can quite fairly characterize as a yearning." (Penny, 1993: 18; in Gillings, 2002)

The Geospatial Sciences have an intricate relation to 3D modeling and VR. While representations of our
environments have been largely in 2D, that is, maps in the past centuries, there also have been efforts to make 3D information part of the geospatial toolset. This does not come as a surprise if we look outside the USA where GIScience is often part of Geomatics (surveying) departments, for example, in Canada and Australia. Interest in VR and 3D modeling has always come in waves so far, that is, people got interested and excited about the prospects of having 3D models and virtual access to these models (see quote by Penny above), started implementing systems and theorizing about the potential, and then gave up as they ran into too many obstacles. Looking into one such effort, a workshop and subsequent book project led by Peter Fisher and David Unwin (2002) summarized the state of the art in geography more than a decade ago and identified prospects and challenges of using VR:

• Ergonomic issues such as feeling disorientated and nauseous also referred to as motion or cybersickness.
• Substantial costs for everyone who wants to use a VR system.
• A rather complex technology for getting content to users in comparison with other media such as the World Wide Web.

If we look at these problems from today’s perspective, we can confidently assert that most of these problems have been resolved (if not all of them). Motion sickness is largely taken care of by advancements in computing technology, that is, better sensor technology and higher framerates allow for almost perfect synchronization of proprioceptive and visual senses reducing or eliminating motion sickness. And, if anything, this is a development that will get even better in the near future as graphics card manufacturers such as NVIDIA or AMD are now developing new products for a growing market, not just a niche. Check out the specs of the new NVIDIA GTX 1080 in comparison to older models GEFORCE [16]. Some problems still persist. It is still not the case that everyone feels comfortable in a VR environment and some people still get motion sick; unfortunately, it is worse if you are old or not getting enough VR time. However, we are getting very close to weed out ergonomics issues that have been standing in the way of mass deployment of VR.

As you see with the Cardboard version of a VR experience, the one that we have sent you, it is possible to get content into people’s hands rather easily. All you need is a smartphone and some apps. This still does not include high-end applications such as Oculus Rift or HTC Vive as they require hardware as well as a beefy gaming computer, but with the advancements in computing power and the availability of low-cost VR ready system, it is only a matter of time until essentially everyone will be able to access a VR (or AR or MR) system.

Likewise, the issue of VR being a rather complex technology does certainly not stand in the way any longer. While VR and 3D modeling are still requiring some training for people who would like to dive deep into software and content development, creating a simple VR experience is as easy as pressing a button. The intro video that you can find on the introductory page is created using a Ricoh™ Theta S, a 360-degree camera that can be purchased via Amazon™ for about $350. During the development of the course, the software app that comes with this camera has drastically improved and allows now for watching videos in the viewer and has a direct upload to YouTube (rather than having to infuse 360-degree videos with additional information, which was the case in May 2016).

So what is remaining as a challenge for VR to become omnipresent in the geospatial sciences and to enter the mass market? Interestingly, in a recent survey, the lack of content has been identified as one of the main remaining obstacles (Perkins Coie LLP & UploadVR Inc., 2016). This is, in my humble
opinion, one of the biggest opportunities for the geospatial sciences. With a long-standing interest in organization information about the world in databases, scanning and sensing the environment, and, more recently modeling efficiently in 3D with geospatial specific approaches and technologies, geospatial is potentially playing a critical role in the developing VR market, and benefitting from it in return. We shall see whether this prediction holds.

For a long time, people have been building 3D models not in the computer but by using physical miniature versions of buildings. The Figure below shows a nice example of the 1630 city of Göttingen, Germany. To get an idea of the current status quo of 3D modeling from a GIS perspective we would like you to read a white paper published by Esri in 2014. Please be advised that there will be questions in quizzes and the final exam on the content of the white paper.

Download paper linked below

3D Urban Mapping: From Pretty Pictures to 3D GIS (An Esri White Paper, December 2014) [2]

Figure xxx. Göttingen, Stadtmuseum, model of city, ca. 1630. The forest is fiction, but typical for the region.

Credit: Wikimedia Commons [17]

1.4 Applications of 3D Modeling

3D modeling was expensive in the past. With the availability of efficient 3D workflows and technologies that either sense the environment directly (such as LiDAR and photogrammetry) or use environmental information to create 3D models (such as Esri CityEngine's procedural modeling, see Lesson 2 and 4) it is now possible to properly use 3D models across academic disciplines. While there is still an ongoing debate to which extend 3D is superior to 2D from a spatial thinking perspective and there is, indeed, beauty in abstracting information to reveal what is essential (see London subway map [19]), there is also, without a doubt, a plethora of tasks that greatly benefit from using 3D information. For example, whether or not you are gifted with high spatial abilities, inferring 3D information from 2D representations is challenging. Below is a quick collection of topics and fields that benefit from 3D modeling and analysis (see also Biljeki et al. 2015):

• Estimation of shaded areas to calculate reduced growth areas for agricultural planning.
• Estimation of solar irradiation.
• Energy demand estimation.
• Urban planning.
• Visualization for navigation.
• Forest management.
• Archeology.
• Visibility analysis.
• Identifying sniper hazards.
• Orientation and wayfinding.
• Surveillance camera positioning.
• Noise propagation in urban environments.
• Flooding.
• Cultural Heritage.
• Communicating Climate Change.

References

1.6 Examples

There is an almost infinite number of examples of VR content that is pushed on the market with new and exciting opportunities being added on a daily basis. We have selected a couple that we are looking at in a little more detail.

Google Cardboard / WITHIN

We have sent you a Google Cardboard. It is actually a slightly more advanced version than the standard one and it does help to adjust the distance between the lenses. If you are using Google Cardboard, there are many, many options to choose from that offer you content from Google itself but also other VR companies such as Within. You are strongly encouraged to explore content on your own! As one of the assignments we would like to give you, please watch the 360° video offered by Within called Clouds over Sidra. You can watch it either in the Cardboard viewer, or, if you prefer or have not received your viewer yet, you can also watch it on your phone without the viewer, or through other VR headsets, or on your computer via the web. The video Clouds over Sidra has been embedded below for your convenience.

Video: WITHIN 360° Video, Clouds Over Sidra (8:35)

To watch this 360 video you can click on the video, visit WITHIN [21] or download and use the WITHIN app for a cardboard experience (recommended).

When watching the video, especially in the Cardboard / immersive headset option, take note on a couple of things:

• How does filming a 360° video change how you follow the storyline of the video. Take note of aspects of the video that may or may not be unintentional.
• What is the role that audio plays in watching the video and providing guidance on what to focus on (it is best to use headphones).
• What does being immersed add to the video, how do your attention and your connection to the place change?
• Did you try watching the video standing? I personally feel that it adds to my sense of place (and the camera was at a somewhat higher viewpoint).
• Anything else?

Google Expeditions

All major technology companies are investing heavily in 3D/VR technology. Google’s Expeditions Pioneer Program is one example. Expeditions is a professional virtual reality platform that is built for the classroom. Google has teamed up with teachers and content partners from around the world to create more than 150 engaging journeys - making it easy to immerse students in entirely new experiences. Feel free to explore. This tool is geared toward classroom experiences but may offer some insights into how geography education may change in the future. Maybe geography will return to the K12 curriculum?

Screenshot of Google's Expeditions Pioneer Program

Credit: Google Expeditions [22]

New York Times Virtual Reality (NYT VR)

The New York Times, as one of the first major news providers, has created an NYT VR app [23] that can be downloaded for both Apple and Android platforms.

The app can be used in combination with Google Cardboard or any other viewer that accepts your smartphone. By now all major news corporations have 360-degree video offerings.

Screenshot of the NYT VR website with content examples. While 360-degree videos can be watched as a desktop version, the main purpose is to provide content for immersive experiences using a combination of the mobile phone app and viewer (e.g. Google cardboard).

Credit: NYT VR [23]

Summary and Tasks Reminder

In this introductory lesson, we went through a plethora of topics contributing to the excitement and opportunities that 3D modeling and VR have for the geospatial sciences but also for society at large. It is a legitimate question whether or not communication about places through time and space will be the same after 3D/VR technologies have entered mainstream fully. There are a lot of opportunities and there are still a lot of unknowns. The lack of content, despite a booming market, has been identified as the main obstacle for VR; geospatial sciences through 3D modeling developments are in an excellent position to play a key role as content providers and also benefit from a booming VR market in return. We are at the verge of a new era of information systems that will allow for the development of immersive experiences alongside classic visualization techniques.

In the next lesson, we will talk more about the different workflows that are in place or developing that allow for the creation of 3D content.

Reminder - Complete all of the Lesson 1 tasks!

You have reached the end of Lesson 1! Double-check the to-do list on the Lesson 1 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 2.

Overview

As discussed in Lesson 1, 3D modeling and the availability of 3D data are going mainstream. Technology, while it has always developed as part of society for thousands of years, has now reached a level of maturity that allows non-experts to make substantial contributions in several fields. 3D modeling and VR are at a point where cartography was 20 years ago when map-making was placed into the hands of the masses (for better or worse). The workflows that we are discussing in this lesson are examples of how affordable and accessible 3D modeling has become but also provide insights into higher-end solutions. As with all aspects of this course, we will only scratch the surface of a number of possibilities and what was true at the beginning of the course may even be replaced by newer technologies, better algorithms, and lower-cost solutions at the end. We are living in exciting times.

Learning Outcomes

By the end of this lesson, you should be able to:

• Compare input modalities such as sensed data versus modeling
• Discuss options for using remotely sensed data such as LiDAR
• Compare modeling platforms such as CityEngine, AutoCAD, SketchUp
• Discuss options on how to interact with 3D models
• Define the concept of a game engine such as Unity3D
• Discuss hardware available for interacting with 3D models in VR

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

2.1 Prerequisite: The Level of Detail

The Figure below shows different models of Penn State campus and Penn State buildings, respectively. It can easily be imagined that the level of detail of the model on the top may suffice for certain planning or analysis purposes, but that a detailed model (bottom) is required for applications such as architectural planning and engineering. The level of detail is not easy to define for 3D models and there is not a single standard for the "right" level of detail of 3D models. There are related discussions in cartography such as what is a schematic map (or are all maps schematic?). The level of detail is important though as it has implications on aspects such as performance, for example, in virtual reality environments. While there is not a direct correspondence between level of detail and polygon count (the number of individual surfaces that need to be processed), it can be easily imagined that complex models require a lot of computing power to interact with while, simple models allow for keeping up sufficient frame rates, for example, in immersive VR experiences (see Lesson 8 and 9). Interestingly, in many planning scenarios, people may decide to have more than one version of a model, one with a low and one with a higher level of detail in order to use them flexibly in different applications. Unlike map generalizations, there are only a few approaches that allow for 3D model generalization.

Below are examples of different levels of detail of 3D models.

3D model extruded from 2D polygons

Credit: ChoroPhronesis Lab [25]

3D model of the HUB with indoor design and details

Credit: Penn State Office of Physical Plant [26]

It is advisable to think about the level of detail that one requires before starting the modeling process. It is still the case that the more detail you need, the higher the cost will be in terms of time, software product, and computing resources.

A good starting point for thinking of the levels of detail of 3D models is an approach referred to as CityGML [27] that envisioned to standardize 3D models. While the active development of CityGML has slowed down, its classification of levels of detail provides a good basis for discussion. CityGML identifies five levels of detail based on accuracy and minimal dimensions of objects that can roughly be characterized as follows (see also Figure 2.1.2, and Kolbe et al. 2005):

• LoD0: essentially a digital terrain model with potentially 2.5D information.
• LoD1: the positional and height accuracy of points may be 5m or less, while all objects with a footprint of at least 6m by 6m have to be considered.
• LoD2: this level requires positional accuracy of 2m, while the height accuracy is 1m. In this LoD, all objects with a footprint of at least 4m by 4m have to be considered.
• LoD3: Both height and positional accuracy are 0.5m, and the minimal footprint is 2m by 2m.
• LoD4: Both height and positional accuracy are 0.2m.

Figure: Shown here are 5 levels of detail roughly following the specification of CityGML.

Credit: ChoroPhronesis Lab [28]

As this brief discussion illustrates, the level of detail has substantial consequences on the visual appearance of 3D models, as well as on cost and efficiency. Cost is associated with time but also actual software products (e.g., high-end CAD programs for a higher level of detail; procedural rule software for a lower or medium level of detail). Efficiency is related to the computing power necessary to turn a 3D model into a smooth interactive experience.

2.2 Workflows for 3D Model Construction

Overview

Creating 3D models of buildings, artifacts, and cities enable potentially better decision making, education, and planning in many fields such as archeology, architecture, urban sciences, and geography. Creating virtual cities, for example, allows for presenting scenarios of future development plans with more realism to assess their feasibility and plan their implementation. There are informational values in creating virtual cities including better communication with non-technical audiences that is, the community. Additionally, there are new approaches in 3D analysis, for instance, dealing with views, shadows, the reflected heat of building blocks, and the calculation of hours of sunshine.

3D modeling is the process of digitally representing the 3D surface of an object. However, the model is usually rendered to be displayed as a 2D image on a computer screen. Experiencing the model in 3D can be achieved using 3D printing devices or presenting it in a virtual reality (VR) environment. There are different methods of 3D model construction and methods of accessing them. Figure 2.2.1 provides an overview of a couple of options for both creating and accessing 3D models that we will discuss in more detail later in this lesson. Many of the tools that are shown in the figure will be introduced throughout the course while others are challenging to work within an online environment.

Figure: 3D/VR Tools Examples (environmentally sensed data, UAVs, 360-degree cameras, manual modeling, 3D modeling platforms, VR devices, geospatial databases, stereo tables and cameras, interactive controller)

Credit: ChoroPhronesis

The Figure above just name a couple of examples for efficiently collecting and accessing 3D data and information. A booming market for 3D technologies such as photogrammetry, 360-degree cameras, scanning tablets, but also the efficient use of existing data such as Open Street Map, have made content creation for 3D environments accessible to a wide audience. At the same time, VR technology is going mainstream with international companies such as Facebook™ or HTC™ weeding out roadblocks such as cyber/motion sickness.

2.3 Manual Static 3D Modeling

To give you an idea of modeling, let us first consider modeling by hand. In this case, a model of an object, whether existent or not in the past, present, or future is constructed by a human using software modeling tools manually. Creating 3D models manually can be done using software such as SketchUp (see Lesson 3), Maya, 3Ds Max, Blender and Cinema 4D, or any one of a number of software that exists for 3D modeling. There are two main techniques in manually creating 3D models: solid modeling and shell modeling. Solid modeling often uses CSG (constructive solid geometry) techniques. CAD packages such as Rhino, SolidWorks, and SketchUp are considered solid modeling platforms. Complex 3D models can be created with the use of solid geometries. The Figure below shows a couple of solid geometries that are used to create potentially very complex models. An example of a complete 3D model is shown below: a model of the historic Old Main created as part of Geography 497 in Spring 2016.

Figure: SketchUp solids

Credit: SketchUp [29]

Figure: Model of historical Old Main created in Sketchup

Credit: Penn State Geog 497 Student Matt Diaz, refined by Niloufar Kiroumasi

Shell modeling only represents the surface of an object. These types of models are easier to visualize and work with. An example of shell modeling is the use of polygon meshes consisting of a collection of triangles or any other type of convex polygon (from other geography lessons on interpolation you might be familiar with triangulation). Polygon meshes only represent the exterior of a model. To make the model smoother and more realistic, a method called tessellation can be used to break down the polygon meshes to higher resolution poly meshes. Programs such as Maya, 3D Studio Max, and Blender all utilize different methods of working with polygonal meshes.

Figure: Polygonal surface modeling, Modeling an Organized Head Mesh in Maya

Credit: Pluralsight [30]

Figure: Tessellation-creating higher poly mesh

Credit: AutoDesk [31]

While there is a distinct difference between CSG and shell modeling, many of the programs mentioned and even those that were not, provide support for both types in a limited capacity. Understanding what type of modeling best fits your project needs helps with choosing the best software for 3D modeling tasks.

2.4 Data-Driven Modeling

Introduction

Shell modeling can be achieved based on photographs/ aerial imagery or scanning model geometry from the real world using, for example, laser scanners. In this section, we will learn about A) Reality modeling and B) Scanning: Remotely Sensed Data such as LiDAR Point Clouds.

2.5 Procedural Modeling

For realistic scenes, more complex models are needed. Creating these types of models can be tedious. One solution is to construct 3D models or textures algorithmically. Procedural modeling has been used in some techniques in computer graphics to create 3D models based on sets of rules. Procedural computing is the process of creating data algorithmically instead of manually. In computer graphics, it is used for creating textures. Mostly, procedural rules are used for creating complex models such as plants, buildings, cities or landscapes which require more specialized tools and are time-consuming for a person to build. With procedural rules, developing complicated models is simplified through repeating processes, automatic generation, and if necessary randomization.

One of the methods in procedural modeling is the use of fractals (fractal geometry) to create self-similar objects. Fractals are complex geometries with never-ending patterns. In other words, they can have infinite patterns at different resolutions. This type of approach is useful in creating natural models such as plants, clouds, coastlines, and landscapes. For instance, a tree can be automatically created based on branching properties and type of leaves. Then a forest can be generated automatically by tree randomization.

Figure: Fractals

Credit: Wikimedia Commons [38]

Figure: Example of a 3D Weedy tree

Credit: 3D Tree [39]

Figure: An example of a Sierpinski's tree

Credit: 3D Tree [39]

Another method in procedural modeling is generating models based on grammars. Grammar-based modeling defines rules that repeatedly can produce more details and create finer models as in the image below. Tree shape data structure or model hierarchy plays an important role in grammar-based modeling both in terms of programming efficiency and adding more details to a shape.

Figure: Grammar-based modeling

Credit: CityEngine [40]

Instances of procedural modeling platforms are ESRI CityEngine, Vue, and Fugu. Xfrog is a component that can be added to a different platform such as Maya and CAD to build plants, organic shapes or abstract objects.

Procedural modeling will be the topic of Lesson 4.

2.6 3D and VR Application Building Workflows

Typically, 3D models are created with a concrete final application or product in mind. This could be presentational material for a web site, a standalone 3D application like a virtual city tour or computer game, a VR application intended to be run on one of the different consumer VR platforms available today, or some spatial analysis to be performed with the created model to support future decision making, to name just a few examples. In many of these application scenarios, a user is supposed to interact with the created 3D model. Interaction options range from passively observing the model, for example, as an image, animation, or movie, over being able to navigate within the model, to being able to actively measure and manipulate parts of the model. Usually, the workflows to create the final product require many steps and involve several different software tools, including 3D modeling software, 3D application building software like game engines, 3D spatial analysis tools, 3D presentation tools and platforms, and different target platforms and devices.

The figure below illustrates how 3D application building workflows could look like on a general level: The workflow potentially starts with some measured or otherwise obtained input data that is imported into one or more 3D modeling tools, keeping in mind such input could also simply be sketches or single photos. Using 3D modeling tools, 3D models are created manually or using automatic computational methods, or a combination of both. The created models can then either be used as input for some spatial analysis tools, directly used with some presentational tools, for example, to embed an animated version of the model on a web page, or the models are exported/imported into a 3D application building software, which typically are game engines such as Unity3D or the Unreal engine, to build a 3D application around the models. The result from the 3D application building work can again be a static output like a movie to be used as presentation material, or an interactive 3D or VR application built for one of many available target platforms or devices, for example, a Windows/macOS/Linux stand-alone 3D application, an application for a mobile device (for instance, Android or iOS-based), or a VR application for one of the many available VR devices. The figure below is an idealization in that the workflows in reality typically are not as linear as indicated by the arrows. Often it is required to go back one or more steps, for instance from the 3D application building phase to the 3D modeling phase, to modify the previous outputs based on experiences made or problems encountered in later phases. It is also possible that the project is conducted by working on several stages in parallel, such as creating multiple levels of detail (LoD) 3D models to work with the intended platform. Moreover, while we list example software for the different areas in the figure, these lists are definitely not exhaustive and, in addition, existing 3D software often covers several of these areas. For instance, 3D modeling software often allows for creating animation such as camera flights through a scene, or in the case of Esri's CityEngine/ArcGIS Pro allow for running different analyses within the constructed model.

Figure: General 3D/VR application building workflow

Credit: ChoroPhronesis [41]

Click here for a detailed description of this image.

A schematic diagram outlines the following information (in summary):

• Input data (in a variety of formats) imported into either a 3D modeling software or a 3D application building software such as a game engine.
• In 3D modeling software, 3D models are created either manually or automatically based on computational methods The resulting models could be either directly used for visualization (i.e. on the web) or imported into 3D application building software such as Unity3D for more elaborate functionalities.
• The application made from game engines (e.g. Unity 3D) can be deployed on both web and a variety of platforms including VR headsets.
• Depending on the application and the deployment platform attributes of models such as levels of detail (LoD) can be manipulated

A 3D modeling project can also aim at several or even many target formats and platforms simultaneously. For instance, for the PSU campus modeling project, that we are using as an example in this course, the goal could be to produce (a) images and animated models of individual buildings for a web presentation, (b) a 360° movie of a camera flight over the campus that can be shared via YouTube and also be watched in 3D with the Google Cardboard, (c) a standalone Windows application showing the same camera flight as in the 360° video, (d) a similar standalone application for Android-based smartphone but one where the user can look around by tilting and panning the phone, and (e) VR applications for the Oculus Rift and HTC Vive where the user can observe and walk around in the model.

The next figure illustrates the concrete workflow for this project (again in an idealized linear order) with the actual software tools used and providing more details on the interfaces between the different stages. The main input data for the campus model is extracted from Airborne LiDAR point clouds. Based on this data, building footprints, buildings, and trees height, Digital Elevation Model (DEM) are extracted. Roads (streets and walkways) are 2D shapefiles received from the Office of Physical Plant at PSU. Roof shapes and tree types information are collected based on surveys. DEM which is in raster format and other data that are in vector format (polygons, lines, and points) are imported to CityEngine which is a generation engine for creating procedural models. Procedural modeling using CityEngine will be explained in more detail in Lesson 4. The result of the procedural model can be shared on the web as an ArcGIS Web Scene for visualization and presentation. It can also be exported to 3D/VR application building software.

Figure: Specific 3D/VR workflow for the campus project.

Credit: ChoroPhronesis [41]

Click here for a detailed description of this image.

A schematic diagram outlines the following information (in summary):

• Extraction of mode data from LiDAR point cloud.
• Digital Elevation Model (DEM) are imported into CityEngine for generation of procedural models.
• Result can be shared on the web as an ArcGIS Web Scene, or exported to 3D/VR application building software such as Unity3D.

The 3D/VR application building software we are going to use in this course will be the Unity3D game engine (Lessons 8 and 9). A game engine is a software system for creating video games, often both 2D and 3D games. However, the maturity and flexibility of this software allows for implementing a wide range of 3D and now VR applications. The entertainment industry has always been one of the driving forces of 3D and VR technology, so it is probably not surprising that some of the most advanced software tools to build 3D applications come from this area. A game engine typically comprises an engine to render 3D scenes in high-quality, an engine for showing animations, a physics and collision detection engine as well as additional systems for sound, networking, etc. Hence, it saves the game/VR developer from having to implement these sophisticated algorithmic software modules him- or herself. In addition, game engines usually have a visual editor that allows for producing games (or other 3D application) even by non-programmers via mouse interaction rather than coding, as well as a scripting interface in which all required game code can be written and integrated with the rest of the game engine or in which tedious scene building tasks can be automated.

Unity can import 3D models in many formats. A model created with CityEngine can, for instance, be exported and then imported into Unity using the Autodesk FBX [42]format. SketchUp models can, for instance, be imported as COLLADA .dae [43] files. In the context of our PSU campus project, Unity3D then allows us to do things like creating a pre-programmed camera flight over the scene, placing a user-controlled avatar in the scene, and implementing dynamics (e.g., think of day/night and weather changes, computer-controlled moving object like humans or cars) and user interactions (e.g., teleporting to different locations, scene changes like when entering a building, retrieving additional information like building information or route directions, and so on).

The different extensions and plugins available for Unity allow for recording images and movies of an (animated) scene as well aw to build final 3D applications for all common operating systems and many VR devices. For instance, you will learn how to record 360° images for each frame of a camera flight over the campus in Unity. These images can be stitched together to form a 360° movie that can be uploaded to YouTube and from there be watched in 3D on a smartphone by using the Google Cardboard VR device (or on any other VR device able to work with 360° movies directly). You will also learn how to build a standalone application from a 3D project that can be run on Windows outside of the Unity editor. Another way to deploy the campus project is by building a standalone app for an Android-based phone. Unity has the built-in functionality to produce Android apps and push them to a connected phone via the Android SDK (e.g., coming with Android Studio [44]). Finally, Unity can interface with VR devices such as the Oculus Rift and HTC Vive to directly have the scene rendered to the head-mounted displays of these devices, interact with the respective controllers, etc. There exist several Unity extensions for such VR devices and the VR support is continuously improved and streamlined with each new release of Unity. SteamVR [45] and the OVR [46], and VRTK [47]for Unity currently take a central role for building Unity applications for the Rift and the Vive.

2.7 Photogrammetry

We have briefly mentioned photogrammetry and several of you have found related examples in the previous week. Amazing things are happening in the area of photogrammetry! Photogrammetry, for those who are not familiar with it, can be loosely defined as the art and science of measuring in photos. For the case of 3D modeling this has vast applications:

• Imagine that a house you would like to model no longer exists in the real world but you still have access to images through, for example, a library. Penn State and its main campus have undergone drastic changes in the last 170 years. Naturally, not all buildings that were on campus are still there. Penn State’s library has taken on the task to preserve some of the knowledge we have about previous states of the campus and has created digital archives for images but also maps. Here is an example: Maps: State College Change Through Time [48].
• Imagine that you are an archeologist and that you are interested in preserving the heritage of an ancient site. There are high-level approaches through companies like Bentley, who recently modeled the Penn State Campus but they are really expensive and may or may not be necessary for your purposes. A substantially less expensive approach that nonetheless provides you with a 3D model of a site can be accomplished with so-called structure from motion mapping. Below is an example from a project of the University of Ghent in Belgium and Penn State's Obelisk that a ChoroPhronesis member (Jiayan Zhao) created.

Example: The Mayan site of Edzna in Campeche (Mexico)

Example: Penn State's Obelisk

Photogrammetry has traditionally been within the field of geodesy and remote sensing, but its applications are becoming ubiquitous. Originally, it was used to measure distances, define the extent of areas, and essentially identify and geo-locate terrain features with the purpose to complement databases and/or to create maps.

Tasks and Deliverables

Summary and Tasks Reminder

This lesson hopefully conveyed how creative and exciting the developments are that allow for the creation of 3D models and VR content. There are classic approaches coming out of photogrammetry, such as structure from motion mapping (SFM), that existed for a long time but only recently reached a mature level. There are creative minds that combine different technologies such as Google Earth and SFM to simulate fly overs and much more. The article we integrated by Chen and Clarke was only been presented in September 2016, so very recent. As we have been successfully using it in our own work, we thought we would integrate it into this lesson. It is a really fast-moving field and if anything, many products will become even more mature over the next years with numerous start-ups pushing into the market.

Reminder - Complete all of the Lesson 2 tasks!

You have reached the end of Lesson 2! Double-check the to-do list on the Lesson 2 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 3.

Overview

SketchUp is one of the most versatile and easiest to learn 3D modeling tools on the market. It offers an excellent balance between functionality and quality on the one hand and ease of use on the other. It allows users with relatively little experience to create exceptional 3D models of almost anything. SketchUp is not tied to a single domain (e.g., architecture) but you will find that every discipline and industry in need of 3D models is using SketchUp: architects, landscape architect, architectural engineers, interior designers, construction professionals, urban planners, woodworkers, artists, sculptors, mapmakers, historians, 3D game designers, and more. SketchUp was part of the Google™ family but is now owned by Trimble™. We will only scratch the surface of this versatile modeling tool that through its warehouse for models (assets) from users and companies might become one of your favorite tools if you are planning to create building models of the past, present, or future, if you want to design room interiors, model your dream car, or, through its large expansion opportunities you might be interested in creating solutions for augmented reality. One of our colleagues said that he treats SketchUp like Minecraft™ but instead of largely using blocks you have every geometric form under the sun at your disposal.

While geographers are only slowly yet steadily entering the world of 3D, certainly one of the biggest trends at recent ESRI user conferences, the results of using SketchUp in our residential courses are quite remarkable. Here are some models that some students have created after a weeklong introduction. More models and more information can be found on our websites (Sketchfab [54]; Sites [55]).

The model below shows Old Main, the landmark building of Penn State, in its historic form (before 1930). The model has been created by Matt Diaz and curated by Niloufar Kiromasi, both interns at ChoroPhronesis (Summer 2016). Modeling has been possible thanks to an ongoing collaboration with the Penn State Library who is offering historic images and floor plans for buildings that do not exist anymore on campus. Our efforts make a contribution to what has come known as cultural heritage research, in our case, we are working toward a strategy to recreate environments long gone. If you would like to know more about Old Main’s history, visit Penn State's Historic Old Main [56].

Click the image below to move the building around using your mouse, stylus, or finger on the screen. If you want to view the building using your Cardboard Viewer, go to Sketchfab Penn State University 1922: Old Main [57].

SketchUp model of Old Main [57]

Learning Outcomes

By the end of this lesson, you should be able to:

• Set up your workspace in SketchUp
• Draw basic 2D shapes such as rectangles, triangles, cylinders in SketchUp
• Turn 2D shapes into 3D objects in SketchUp
• Differentiate between inference, groups, and components
• Use textures in SketchUp
• Optimize your model
• Create your own building models in SketchUp
• Export your model from SketchUp and share it with the world via Sketchfab (optional)

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

3.1 Start Modeling

I would like to offer you a word of caution: not everyone is a modeler. That does not mean and should not mean that you cannot create a beautiful model (or pass this lesson!). It simply means that some people develop a love for creating models while others really do not like it. One piece of advice that works on almost everything in these situations is to practice, practice, and practice. We will start off everyone on the right foot by going through several tutorials; we will encourage you to practice (and practice, and practice). We will also request a two-tier exercise at the end acknowledging that this is a 2-week max exercise and not a course on SketchUp. However, there is also the opportunity to make a contribution to an exciting project we started, that is, we have been creating historic campus models for Penn State. The video below shows a fly-through of part of the campus model for 1922 (slightly outdated as this is an ongoing project). On Sites at Penn State: Penn State Campus - Past, Present, and Future [55], we have collected building models from students who made contributions and if you are interested, there are several buildings yet to be modeled. If you ever have a chance to visit One World Center, they have a tour through the history of New York projected onto their elevator walls, visit World Trade Center Elevator Video [59]. Penn State does not have long enough elevators but it is a feasible way to communicate the dramatic changes Penn State and State College have gone through using 3D models and VR.

Video: World Trade Center elevator video (00:58) This video is not narrated.

The model that we are building can be turned into a VR experience by using a game engine such as Unity (see Modules 8 and 9). Once in Unity, there are numerous options for interacting and experiencing the models: they can be perceived in HMDs such as Oculus Rift or HTC Vive, there are extensions to create mobile phone apps, and, as you will learn, you can create a fly-through using a 360 degree camera and exporting the video, for example, to YouTube. Just to give you an example of the latter, below is a YouTube video taken from our 1922 Campus model. You can also view the video at YouTube Penn State campus in 1922 flythrough (non-360 degree version) [60].

Video: Penn State campus in 1922 flythrough (non-360 degree version) (2:01) This video is not narrated, music only.

One of the best resources to get you started with SketchUp, which covers all essential basics necessary for efficiently creating building models, is the SketchUp: Essential Training course on Lynda.com [61]. If you are new to Lynda.com I can highly recommend its repertoire of courses that are well designed, especially for technical skills. Lynda.com is free for all Penn Staters and as the essential course is perfect for our introduction and ‘only’ three hours long, we will use it and have designed exercises around its content. It is not the idea that you simply watch the videos, but that you work through the exercises. After going through the essential training course you are well set up for your own modeling project (see final assignment).

In addition to the Lynda.com course, the following sites also provide valuable resources for learning SketchUp.

1. Lynda, in addition to the essential training course we will be using, offers a number of advanced topics on SketchUp. Just search for SketchUp.
2. SketchUp offers a variety of introductory video lessons through their own SketchUp learning website. [62]
3. One very useful tool is the SketchUp Quick Reference Card [63].
4. Last, but certainly not least there are many devoted users out there who post their solutions on YouTube. Again, just search for SketchUp or more specific topics (e.g., geolocation).

3.2 Installing SketchUp

SketchUp comes in two versions, Free and Pro. The Free version is now (since 2018) web-based and allows for modeling a variety of objects and offers very decent functionality. The Pro version offers additional functionality and is also more flexible and open for additional plug-ins (e.g., specialized export options or augmented reality).

Some of the tasks that SketchUp Pro will allow you to do are (see also Schreyer 2016):

• Export into a variety of formats such as pdf, eps, 3ds, ifc, fbx, obj, vrml
• IFC import/export
• Component-based report-generation
• Dynamic component authoring
• Solid tools for Boolean modeling
• LayOut software for drawing preparation and presentations
• BIM classifier

As part of this course, we recommend using the free trial of SketchUp Pro. Most of what you learn will be also available in the Free version but the Lynda course we are using is tailored to the pro version (they, too, demo with the trial version). To download SketchUp, go to SketchUp [64], click ‘trysketchUp'. On the next page, under "I use SketchUp for...", choose Higher Education and click Create Account to start your 30-day free trial. Please make sure that your computer meets the minimum requirements and keep in mind that using a 3-button/wheel mouse is kind of essential. In other words, if you are a digital nomad like myself, and work everywhere on your laptop, you might want to invest in a wireless mouse. It will make your modeling life drastically easier. After downloading, simply follow the instructions.

3.3 SketchUp: Essential Training

Introduction

As a Penn State student, you have access to a catalog of free self-paced courses on a huge variety of both technical and non-technical topics. These courses are high-quality video-based courses presented by professionals. One of those topics happens to be SketchUp, which you will be using in this lesson. Instead of recreating the training for you here, you will need to access the SketchUp Essentials training through Lynda.com via Penn State's subscription.

The Lynda course offers an introduction to SketchUp for both, Windows and Macintosh systems. Our team primarily uses Windows but we will try to answer questions for Mac users, too (if possible). Read through the tips and the tasks and deliverables below before starting the course. They will make more sense after you get into the course and start learning. Once you begin the Lynda course, go through it at your own pace.

Accessing the SketchUp Course

To access the SketchUp click the link below to access the full course on the Penn State Lynda website. If you aren't logged into a Penn State service like LionPATH, email or Canvas, you may need to log in using your Penn State Access Account user id and password (i.e. abc123). Once you arrive at the course, bookmark the page using your preferred browser to make it easier to return quickly. The course takes over 3 hours to complete but is broken into sections to make it easier to choose a resting point.

SketchUp 2020 Essential Training [66] by Tammy Cody

(You have subscription for courses on Linkedin Learning through Penn State)

Some tips before you get started

• Make your own template and adjust program settings to your liking.
• As always, save early and often.
• Group faces.
• Use components for repeating objects.
• Model as precisely as possible.
• Keep the polygon count low.
• Use proxy components if needed.
• Don't use textures that are too large.

3.4 SketchUp Essential Concepts Summary

Below are a few concepts that you either have encountered or are worth noticing about SketchUp.

Accurate modeling

While the name contains the word "sketch", it is a bit misleading. If you want to, you can model extremely precisely and accurately in SketchUp.

Object snapping

Like many other programs these days, SketchUp allows for snapping to various locations of/within objects such as endpoints, midpoint, edges, or circle centers.

Inference

This is one of SketchUp's most powerful features. There are several types of inferences that make it easier to create 3D models. Examples are point, line, or shape inferences. Different inferences are indicated by a color code. To learn more about inferencing, check out the SketchUp Help Center [68].

Direct value entry

Being able to enter values for objects characteristics directly is extremely useful, especially if you have measured an object you plan to model outside SketchUp.

Groups and components

This is a crazy important feature of SketchUp. Groups and components are not only making modeling easier and faster, but they are also allowing for improving the performance of 3D models. A group is, as the name implies, is a selection of entities grouped/joined together. Important: if you copy a group, all its contents are copied, too, adding to the required storage space. A component, in contrast, is a set of entities that are centrally stored and each time it is being inserted into a model, it is considered an instance. What this practically means is that the storage space of your model is drastically reduced (just think of hundreds of windows and imagine modeling them individually or as a component).

Geo-based modeling

As SketchUp originally was Google SketchUp it does not come as a surprise that it has rather convenient geo-location capabilities (latitude/longitude coordinates). Georeferencing a model is handy especially for embedding it into larger projects and for further analysis (e.g., shadow analysis). Once a model is georeferenced, other information sources become available such as:

• Ortho-photographs
• Terrain models
• Street view images

3.5 Optimization and Rendering

3D modeling allows us to recreate objects or invent new objects using different 3D modeling programs yet; there are limitations to what even modern-day computers can handle in terms of 3D modeling. What exactly does this mean? Essentially, 3D modeling involves lots of information relating to points and lines connected to one another across a coordinate plane. The more points and lines you use in a 3D model, the more work a computer must do to allow you to visualize those points and lines as a 3D model. Aside from the hardware, many software also has limits on the number of points and lines or geometry it can handle. Now think ahead to software such as game engines. Such software can now be used to add information to the geometry you provide in 3D models to produce interactions such as movement. The added information can start to cause lag, slowing the intended interaction so that it is no longer occurring in real-time. So how do you prevent this?

The gaming industry has developed several techniques, termed optimization methods that assist with this issue of too much geometry. Optimization helps you to maintain the highest level of detail (LoD) with as little geometry as possible. So, what exactly is level of detail? Level of detail relates to the amount of realism a 3D model and ultimately a 3D scene can have through geometry and materials. Typically, the more geometry a 3D model has, the more realistic it will appear.

Three different levels of detail

Credit: SketchUp [64]

Geometry is generally measured in terms of weight by either Faces (Quads) or Triangles. Some software refers to Faces as Polys or Polygons as they have four sides. Heavy models have more Faces or Triangles while Light models are the opposite. The goal is to maintain the same base shape but have different amounts of detail. For example, curved shapes take up a lot more geometry than square shapes.

Thinking back to our original issue of preventing lag or slow movement, obviously, Light models are best, but are they realistic enough? Level of detail refers to a scale of geometry weight. Some software is capable of handling 3D scenes with a mix of LoD models, where 3D models that are in focus are Heavy models but models out of focus are Light. This suggests that one needs to have both Heavy and Light models of all geometry in a scene. While this is often the case for video game development, it certainly is not good for everyone. So how else can we address this issue?

Rendering is one of the keys to creating realistic models and scenes. It requires you to add Materials (otherwise known as Textures) to your 3D models. Additionally, you introduce Light sources to the scene your 3D model(s) are in to increase the realism. Increasing the realism through materials and light sources offsets the need for detailed geometry. You use a Render Engine, there are many options in SketchUp as Extensions, to calculate the influence of the Light sources and Materials on your geometry in a scene. Keep in mind Rendering in itself only generates a static image. It can be used to render animations as video files as well, but this does not necessarily help with our issue for interactive environments. Rendering in real-time is another option but it will take up even more computer resources when combined with all your geometry and interactions in a scene. So, how do we make static rendering appear as if it were in real-time?

The gaming industry considered another solution to this issue, Texture-Baking, Render-to-Texture or Light-Mapping. This allows your 3D geometry to be associated with a render-generated material that stores all of the render results on each surface or face. This can be a time-consuming process but what it allows you to do is to add detailed information in the form of an image file like a JPEG or PNG instead of having a lot of extra geometry. We will not go into the details of this now but this technique allows you to make sure what you see, in say Sketch Up rendered also shows up in other software where you may want to import your 3D model/3D scene.

What we will introduce here, however, is Face Normals. Face Normals are part of the Material process. Have you ever noticed in SketchUp, before you add materials that some surfaces appear as a different color? This has to do with the Face Normal. Face Normals indicate which direction a material should be facing. If all face normals are correct, all your geometry would have the same color from the same viewpoint. In rendering terms, normal tells the computer where the front side of a surface is so it can show materials and apply the lighting correctly.

Face Normals

Credit: SketchUp [64]

Flipping the Face Normals on the Grey geometry in the image will correct the color difference so all normals are facing the same direction. Now, why is this important? There are two main reasons, one of which I mentioned earlier. First, if you use a Rendering Engine, even in SketchUp, you need the computer to know which side is up so the lights work correctly. Second, while not necessarily critical in SketchUp itself but when importing your model to other software, the materials may not show up at all because the other software cannot tell which side of a Face is the front side where a material is applied. This can be challenging to correct in another software that may not be designed to handle this issue, such as a game engine. It is easiest to fix Face Normals before you apply materials but it can be done after materials have been added in your 3D modeling software.

Ultimately, the solution you choose is up to you and based on the needs of your project. However, the best practice is to focus on Lightweight models and if the detail is not enough, to add a little extra geometry to see what will work. Relying on materials is necessary for Lightweight models as the materials can replace details you did not model. Correcting Face Normals is necessary for making sure those materials and also light sources show correctly in other software. Whether you use a Rendering Engine or not, you still need to use materials that help with adding detail to your 3D models.

To learn more about Optimization for SketchUp please check out the Knowledge Base article Making a detailed model in a lightweight way [69].

3.6 Create a Building

3.7 Test Your Knowledge

This is an ungraded assignment to test your knowledge.

3.8 SketchUp and Sketchfab

Once you have completed your model and would like to proudly share it with the world (non-mandatory), there are again different ways to accomplish that. One of the most prominent platforms for sharing all kinds of 3D models is Sketchfab [73]. The models that you saw earlier in this module, embedded into the course website, are hosted on Sketchfab. Many modelers allow you to use their models in different contexts and Sketchfab is continuously developing ways that will make your models even more accessible such as built-in VR features. We will be using Sketchfab to organize the final submission of this module and to give you an idea of how to share 3D models that you create.

To finalize your modeling assignment, please create a Sketchfab account.

3.9 Resources

SketchUp official home page [74]

The official SketchUp YouTube channel [75]

The official SketchUp Community Forum [76]

SketchUp Help Center: Quick Reference Card [63]

In addition to the essential training course we used in this module, there are numerous courses that might be of interest. Simply search for "SketchUp" at Lynda.com [77].

Summary and Tasks Reminder

You have gone through a hands-on modeling process in this lesson using SketchUp. While there is still a learning curve and every powerful tool requires practice, SketchUp has many, many features that make modeling easier. From its incredible warehouses to its geo-location features, it offers modelers inside and outside the geospatial sciences a lot of options for exploring the third dimension. For many applications, it is important to have a tool like SketchUp as it may be the only option to create a 3D model. It is also a good learning experience to come to appreciate the effort that has gone into designing software such as City Engine, which uses an entirely different approach as we will explore in the next lesson.

Reminder - Complete all of the Lesson 3 tasks!

You have reached the end of Lesson 3! Double-check the to-do list on the Lesson 3 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 4.

Overview

Lesson 3 has introduced you to hands-on 3D modeling using an efficient approach for various modeling purposes. Instances of when you need hands-on modeling are: (a) you are interested in a substantial amount of detail in your models, (b) your models are non-existent anymore and cannot be, for example, scanned, or (c) you’d like to make use of a substantial asset warehouse that provides you with professionally designed items such as fridges or couches. As you can easily imagine, this approach will be prohibitive if you are interested in modeling entire cities or any large number of buildings, if you would like to test changes quickly (e.g., increzasing the number of floors), or if you need flexible access to your models, for example, through the internet.

In contrast to modeling by hand (also referred to as static modeling), the general concept of procedural modeling has been discussed in Module 2. Rather than modeling individual buildings, procedural computing (as a basis for procedural modeling) is the process of creating models algorithmically instead of manually. This chapter will provide a more detailed introduction to procedural modeling as it is the basis for modeling cities and other geographic scale information in GIS environments. However, we decided not to teach a specific software as a generating platform in this course. We are already introducing you to a number of software options and ESRI CityEngine, although a powerful tool, requires a bit more time than we have in one or two lessons. We will provide you with resources and a demo (in later lessons) that show some of the functionality but do not expect you to learn CE for this course. Yet, we would like you to understand procedural modeling. Later in Module 5, you will explore how procedural rule packages (that we created for you) can be applied in ArcGIS Pro (this is also in line with the long term strategy of ESRI). What this means is that you will be able to modify rules created by a procedural modeling software but you do not have to learn the specifics of the software (City Engine). As Arc GIS Pro is a rather new software product the functionality to change procedural rules is still limited but improving from version to version. The software has already been updated twice since we started designing this course.

Learning Outcomes

By the end of this lesson, you should be able to:

• Define the concept of procedural modeling rules and coding (.cga or computer-generated architecture)
• Contrast procedural rule modeling and hands-on modeling
• Identify advantages and challenges of procedural modeling rules
• Demonstrate the results of procedural modeling rules for the campus

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

4.1 The Concept of Procedural Modeling

As discussed in Module 2, procedural modeling is about constructing detailed textured 3D models by formulating rules instead of manually modeling them (CityEngine [79]). These rules are constructed based on a specific type of shape grammar (Stiny & Gips 1971). Stiny and Gips invented shape grammars and defined shape grammars as a method of two and three-dimensional shape generation. A shape grammar is a set of shape transformation rules to generate a language or set of design (Knight, 1989; Duarte & Beirao, 2010). They are spatial rather than symbolic and also non-deterministic. It means that the designer is not constrained by a limited choice of rules (Knight, 1989).

Shape Grammar is a way of defining algorithms. It makes design, generally speaking, more understandable. In order to read and use shape grammars, we need to have a short review of the basic terms below (Stingy, 1980):

• Grammar: A set of rules of transformation.
• Language: A set of all the possible designs can be generated with the grammar rules.
• Shapes: points, lines, planes, or volumes.
• Labels: symbols, numbers or words that restrict the rules that apply.
• Initial Shape: The first labeled shape that the rules are applied to.
• Rule: A process transfer shapes and label from the left side to the right side (α-->β).
• Transformation: Usually is Euclidean transformations, changing location, orientation, reflection, or scale of a shape.
• Derivation: apply the rules to the initial shape recursively and generate new shapes.

A shape rule specifies how the existing part of a shape can be transformed. It consists of two labeled shapes on each side of the arrow (α-->β). Each labeled shape is made up of a finite set of shapes which are points, lines, planes, or volumes. For instance, in the figure below a simple shape grammar consisting of two rules is defined. The first rule specifies that a triangle with a dot can be modified by adding an inscribed triangle and removing the dot from the original triangle. The second rule states that one can also simply remove the dot from a triangle with a dot. Given the initial shape of a triangle with a dot, a shape can be derived by applying these two shape rules in a different order, namely, rule a -> rule a -> rule b in this case.

Figure: Shape Grammar

Credit: ChoroPhronesis Lab [28]

To create an urban plan or design, the complexity and characteristics of that environment can be interpreted as shape grammars to increase flexibility in suggesting the number of solutions and scenarios. Duarte & Beirao (2010) indicate that old urban plans are not compatible with contemporary urban societies and that there is a need for flexible designs. They propose that there are two ways to use shape grammars in urban environments: (1) as an analytical tool to find the structure of an existing design, or (2) as a synthetic tool to create a new language for design. Both methods automate the process of design generation.

Although a shape grammar can be generated by hand, for more complex rules and choices, a generation engine is needed to create and display geometries and decide which shape rule to apply. ESRI CityEngine is a generation engine for creating shape grammars, applying rules, and generating shapes. CityEngine’s programming language to define shape rules is called CGA (computer-generated architecture) shape grammar. The next section briefly explains the CGA shape grammar and how it can be applied to create virtual 3D models of cities.

Creating a model based on rules is less labor-intensive. However, the preparation of rules is time-consuming. The strength of this modeling approach is that once the rules are created, generating and creating models takes a small amount of time. The following diagram shows the total cost of procedural modeling vs handcrafted modeling.

Figure: Total cost of procedural modeling VS handcrafted modeling

Credit: CityEngine [79]

4.2 Introduction to CityEngine and its CGA Shape Grammar

Introduction

In this section, we focus on an introduction to ESRI CityEngine as an instance of procedural (grammar-based) modeling software. Although this software won’t be used in this course, the result of its procedural modeling will be applied in lessons 5 to 7 that will introduce you to ArcGIS Pro. CityEngine is standalone desktop software. ‘The computer is given a code-based ‘procedure’ which represents a series of commands - in this context geometric modeling commands - which then will be executed. Instead of the "classical" intervention of the user, who manually interacts with the model and models 3d geometries, the task is described "abstractly", in a rule file’ (CityEngine [79]). The set of generic procedural rules can be used or modified when creating virtual cities. Moreover, new procedural rules can be defined, modified, or rewritten to create more detailed and specific designs.

Screenshot of CityEngine user interface showing the University Park campus.

Credit: ChoroPhronesis Lab [28]

Please watch the video below to learn more about the user interface and navigation in CityEngine.

Video: CityEngine Essential Skills: Exploring the User Interface and Navigation Controls (15:47)

4.3 Introduction to Procedural Modeling for UP Campus

Introduction

Airborne LiDAR data provides information on variable elevations. This elevation information is useful in both creating the DEM (digital elevation model) and estimating buildings’ height. The airborne LiDAR data collected for Penn State campus is in 25 cm intervals. You can see the Campus LiDAR point clouds of buildings and vegetation in the Figure below. All noises and irrelevant categories are removed for presentation purpose.

3D presentation of LiDAR data, University Park Campus in ArcGIS.

Credit: ChoroPhronesis Lab [28] (Data collected by the Office of Physical Plant)

LiDAR returns information for two types of elevation models: (1) DSM or digital surface model (Figure below) which is a first return surface including anything above the ground such as buildings and canopy. (2) Topography: the ground or bare earth which is referred to as DEM (digital elevation model).

Normal DSM based on LiDAR data of University Park campus

Credit: ChoroPhronesis Lab [28]

DEM layer based on LiDAR data of University Park campus

Credit: ChoroPhronesis Lab [28]

Tasks and Deliverables

Summary and Tasks Reminder

We would like to acknowledge that we have by now given you a quite intense tour through various modeling approaches. It is, however, important to have at least an overview of what is currently available as the project you may be working on in the future (past this course) may require you to design a custom made solution. In addition to various workflows including structure from motion mapping and hand-on modeling that we explored previously, this lesson focused more deeply on procedural modeling and its origins in theories of shape grammar. Procedural modeling is a wonderful tool to both explore the organization of built environments in theory but also, more recently, as a powerful tool to make modeling more efficient. ESRI has realized this potential and bought City Engine which was founded at the ETH Zurich (their technical university). Keeping the different approaches in mind it important as we move on in the course.

Reminder - Complete all of the Lesson 4 tasks!

You have reached the end of Lesson 4! Double-check the to-do list on the Lesson 4 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 5.

Overview

In this module, you will build a 2D map of the University Park campus including buildings, trees, roads, walkways, and a digital elevation model (DEM). You will learn how to convert 2D maps to 3D presentations. Then procedural symbology that we have created for you in CityEngine will be imported and applied to the 3D map to create a realistic scene.

Learning Outcomes

By the end of this lesson, you should be able to:

• Explain options for 3D modeling within Arc GIS Pro
• Understand ArcGIS Pro's integrated 2D–3D environment (Local and Global Scenes)
• Convert a 2D map to a 3D scene
• Explain 3D symbology
• Extrude features to 3D symbology
• Find an example of a 3D CityEngine/ArcGIS Pro model: campus model
• Compare procedural modeling rules (.cga) vs procedural rule package (*.rpk)
• Create a 3D model of elevation (DEM)
• Create a 3D scene from a 2D building footprints of campus-based on the DEM
• Employ a procedural rule symbology based on the rule package
• Link procedural rule symbology to attributes table
• Distinguish static modeling from procedural modeling
• Restructure the map and represent the 3D scene based on the imported model

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

Section One: Create a Map of University Park Campus

Introduction

The first step into 3D modeling using ArcGIS Pro is making a map. First, you will create a project. Then you will add necessary data to your map where you have all the tools you need. Then, you will explore the University Park Campus with navigation tools and bookmarks. This 3D model of the University Park Campus will be the final result of this lesson.

3D model of the University Park Campus

Credit: ChoroPhronesis Lab [41]

Section Two: Symbolize Layers and Edit Features

Introduction

You probably noticed during your map-exploration that it was hard to distinguish some of the features because of how they were symbolized. In this section, you will symbolize your map, for example, to enhance readability.

Section Three: Explore Raster Data

Introduction

To present a 3D scene of University Park Campus, you need an elevation base for your building footprints and other layers such as roads and trees. In this section, we focus on preparing the Digital Elevation Model (DEM) for University Park Campus. The Digital Elevation Model (DEM) is a bare earth elevation model. It has been extracted from LiDAR data.

Section Four: Explore 3D Data

Introduction

In this section, you’ll visualize 3D data. You will learn how to convert a 2D map to 3D scene, visualize the DEM layer and other feature classes such as trees and buildings.

Section Five: Display a Scene with More Realistic Details

Introduction

As you can see, the extruded buildings in the previous section, are blocks with no details, just the elevation. In this section, you will learn how to present part of west campus with more details. To do so, you should remove some of the features from UP_BUILDINGS that overlap with the layer that has more detailed information. This new layer (UP_Roof_Segments), that you will add to the map, consists of roof segments with detailed elevation. This information has been extracted from LiDAR data. In other words, instead of treating buildings as a big, undifferentiated chunk, every change in shape or elevation in each building has been detected and a new layer that represents those change/segments have been created. In Module 4, we have explained the process of creating this layer in detail.

Tasks and Deliverables

Summary and Tasks Reminder

In this lesson, you have learned how to symbolize 2D and 3D maps. You also have learned the differences between static modeling and procedural modeling. Most importantly, you practiced how to create a 3D model of elevation and how to incorporate City Engine procedural rule symbology into GIS.

Reminder - Complete all of the Lesson 5 tasks!

You have reached the end of Lesson 5! Double-check the to-do list on the Lesson 5 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 6.

Overview

Importance of 3D Models in Spatial Analysis

In the previous lesson, you created a 3D map of the University Park campus. You learned how to symbolize layers based on procedural rule packages. In this lesson, you will learn two types of 3D spatial analysis. First, you will demonstrate the buildings that are affected in the case of a thunderstorm and a flash flood in State College. Second, you will learn how to do Sun Shadow analysis. It will give you a visual sense of how different sun positions will create various shadow volumes in different directions. It is a useful tool for building design and understanding the sun’s path in a given coordinate and also for figuring out the surface area that a building shades during a day.

Learning Outcomes

By the end of this lesson, you should be able to:

• Understand the importance of 3D models in certain geoprocessing analysis
• Present an example of 3D/spatial analysis
• Analyze the visibility of buildings and locations
• Calculate line of sights
• Calculate viewsheds
• Create layers
• Explore panning, zooming, and tilting
• Choose buildings and search for them
• Apply rules to extruded features
• Calculate, for example, a flood layer
• Share your work on Google Earth

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

Section One: Flood Analysis

Introduction

In this section, you will determine which areas of campus are more affected by flooding, by creating a raster layer representing accumulated water resulting from a thunderstorm. You will use the analytical 3D scene you created in the previous Lesson to visualize the flooding. In this study, you will use a FEMA flood map to find out how much of campus might be affected by the 1-percent annual chance flood, which is referred to as 100-year flood¹. You will buffer the floodways around campus to present the flood plain and calculate the campus area affected by the 100-year flood. The floodway data is downloaded from FEMA flood map service center². On flood plains, if you consider that water accumulation or the height of the water is a constant value, other factors that affect which areas are more affected by the flood are dependent on various factors. Examples of these factors are land elevation, surface material (pervious or impervious surfaces), vegetation coverage, and surface condition (saturated or dry). In this study, we only consider elevation as a factor. Therefore, you will extrude the flood data to see which buildings or trees are more affected.

For more information, refer to FEMA: Flood Zones [108]

²FEMA Map Service Center (Borough of State College) [109]

Section Three: Share Your Results on Google Earth

You would like to share the results of shadow volumes over the imagery. One way is to use Google Earth. It has both the imagery and 3D building models and is a good way to present your results.

Step 1: Download and install Google Earth

If you already have Google Earth skip this step. Otherwise, go through the following process:

1. Open your web browser. Go to Google Earth [119].
2. Click Earth Versions.
3. Choose Google Earth Pro on Desktop.
4. Click download Google Earth Pro.
5. Install Google Earth Pro.

Step 2: Exporting to KML

KML (Keyhole Markup Language) is an XML based file format that is used in Earth browsers such as Google Earth and Google Maps. Consider you would like to share shadow surface in the morning, at noon and before sunset to demonstrate the shaded surface during the day. To do so, you will select the time of the day you are interested in and you will convert the Layer to KML.

1. Open UniversityParkCampus_Lesson6_Shadow Project in ArcGIS Pro.
2. On the top ribbon, click Select by Attributes.
   

   Credit: 2016 ArcGIS [120]
3. A Geoprocessing Pane on the right-hand side of the interface will open. Select SunShadow_January1. Add a clause that DATE_TIME value is equal to 8.36 am. Click Run.
   

   Credit: ArcGIS [120], 2021
4. You can see selected shadow volumes highlighted in blue.
   

   Credit: ArcGIS [120], 2021
5. On the top ribbon, click Analysis. Select Tools in the Geoprocessing group.
6. In the Geoprocessing Pane on the right-hand side of the interface, search KML. Select Layer to KML (Conversion Tools).
   

   Credit: ArcGIS [120], 2021
7. The input Layer will be SunShadow_January1. Only selected areas will be converted.
8. Select the output folder and name the output Shadow_7.54am.
9. Option Clamped features to the ground create shadow surface on the ground. Make sure it is checked. Click Run.
   

   Credit: 2016 ArcGIS [120]
10. This tool creates a KMZ file which might be confusing. We talked about KML and now the result is KMZ. A KMZ is a compressed KML. KMZ is smaller in size and easier to upload to the web. It contains the same data in KML format. You can unzip the file and see the KML inside the KMZ file.
    

    Credit: ArcGIS [120], 2021
11. Repeat this process for 11:54 am and 3.54 pm shadow volumes.

Step 3: Opening KMl/ KMZ in Google Earth

In this step, you will open three KMZ files you have created in Google Earth. Since Google Earth is free to use, then you can share your Google Earth file with anyone.

1. Open Google Earth (Pro).
2. Open a file explorer or windows explorer and navigate to the place you saved your KMZ files.
   

   Credit: 2021 Google Earth
3. Move and resize the windows explorer and Google Earth, to see both at the same time next to each other. Drag and drop Shadow_7.54am into Google Earth.
   

   Credit: 2021 Google Earth
4. In the Google Earth interface, you will be directed to University Park Campus and you can see that Shadow_7.54am is added as a layer to Places Pane.
   

   Credit: 2021 Google Earth
5. Right Click on Shadow_7.54am and click properties. Go to the Style, Color tab. Change both Lines and Area to yellow with 60% opacity. Click Ok.
   

   Credit: 2021 Google Earth
6. Use the navigation tool on the up right-hand of the interface to move around and rotate the map.
   

   Credit: 2021 Google Earth
7. Drag and drop Layer Shadow_11.54am to Google Earth. Go to the Style, Color tab. Change both Lines and Area to orange with 80% opacity. Click Ok.
   

   Credit: 2016 Google Earth
8. Drag and drop Layer Shadow_3.54m to Google Earth. Go to the Style, Color tab. Change both Lines and Area to red with 80% opacity. Click Ok.
   

   Credit: 2021 Google Earth
9. You can uncheck any of the shadow layers, just to see a shadow surface in a particular time. Zoom in to Old Main and see how the shadow surface changes during the day.
10. On the Places Pane, right-click Temporary Places. Click Save Places As. Name the file "UniversityParkCampus_Shadow.kmz" and save it to your desired place. This is a file that you can share with anyone and they can promptly open it on Google Earth.
    

    Credit: 2021 Google Earth

Submit your file

Upload "UniversityParkCampus_Shadow.kmz" to Lesson 6 Assignment: Google Earth (2).

Guidelines and Grading Rubric

Tasks and Deliverables

Summary and Tasks Reminder

In this lesson, you have learned the importance of 3D data and 3D maps for spatial analysis. To explore various geoprocessing analysis, you have practiced two different methods to analyze flood and sun shadow analysis. In terms of 3D data,  it is important to remember the difference between multipatch as 3D data and extruded 2D data. In the end, you experienced how to share your work on other platforms.

Reminder - Complete all of the Lesson 6 tasks!

You have reached the end of Lesson 6! Double-check the to-do list on the Lesson 6 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 7.

Overview

In this and the next two lessons, you will be working with the Unity3D game engine, simply referred to as Unity throughout the rest of the course. Unity is a rather complex software system for building 3D (as well as 2D) computer games, and other 3D applications. Hence, we will only be able to scratch the surface of all the things that can be done with Unity in this course. Nevertheless, you will learn quite a few useful skills related to constructing 3D and VR applications in Unity dealing with virtual environments. Furthermore, you will become familiarized with C# programming (the main programming language used in this engine) through a series of exercises that guide you on developing your first interactive experience in a game engine.

This lesson will begin with an overview to acquaint you with the most important concepts and the interface of Unity. It will cover basic project management in Unity, the Project Editor interface, manipulation of a 3D scene, importing 3D models and other assets, and building a stand-alone Windows application from a Unity project. Moreover, you will learn how to create a very simple “roll-the-ball” 3D game using C#, which will prepare you for more advanced interaction mechanics we will be implementing in the next lessons.

Learning Outcomes

By the end of this lesson, you should be able to:

• Explain the role of interaction for 3D/VR projects and the role of game engines for enabling access to 3D models
• Create 3D projects in Unity3D, import 3D models, and understand the basic concepts and interface elements of the Unity editor
• Manipulate scenes and add script components to game objects
• Import custom Unity packages into a unity project
• Develop a basic understanding of the physics engine of Unity (e.g. gravity, collision detection)
• Run projects inside Unity and build stand-alone applications from them
• Have a basic understanding of how to make an experience come “alive” by programming behaviours for game objects

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

The Unity3D Game Engine

Unity is a game engine developed by Unity Technologies. Development started over a decade ago and the initial version of Unity was released in 2005. Unity has experienced rapid growth, in particular over the last few years, and is undoubtedly the most popular game engine out there. According to the Unity website, 50% of mobiles games, and 60% of AR/VR content are made with unity. The company has over 2000 employees and more than 3 billion devices worldwide are running unity [1]. There have been several major releases of Unity over the years with new minor versions being released on a regular basis. The current major release is Unity 2019 with the latest stable version being version 2019.1.10 at the time of this writing. In 2010, Unity Technologies launched the Unity Asset Store, an online marketplace where developers can obtain and sell assets for their Unity projects including tools and extensions, scripts, artwork, audio resource, etc. (Unity Asset Store [121]). Today there exists many side products and activities around the Unity platform. For instance, Unity offers its own Certification Program (Unity Certification [122]) to interested individuals and has its own VR/AR division called Unity Labs (Unity Labs [123]).

One of Unity’s main advantages is that it is cross-platform both during development time and deployment. Unity applications can be deployed for many platforms including Windows, macOS, Linux, and different mobile operating systems. That means not only can you choose on which operating system you want to create your 3D application but once it has been created, it can be deployed to a large variety of devices. For instance, you can build stand-alone programs for the common desktop PC operating systems (Windows, macOS, Linux, etc.) as well as versions for different mobile phones (e.g., Android-based phones and iPhones). Of course, the differences in performance and hardware resources between a PC and a mobile phone need to be considered, but there is support for this in Unity as well. In addition, Unity has been a front-runner when it comes to building VR applications and supports or provides extensions for a large variety of consumer-level VR devices such as the Google Cardboard, Gear VR, Oculus Rift, HTC Vive, and other mainstream HMDs. Three of the most popular VR development plugins (a series of libraries and pre-made game objects that facilitate access to HDMs and enable developers to program different functionalities) for unity are SteamVR (by Valve corporation), OVR (also known as Oculus integration, by Oculus), and VRTK (a community-based plugin by Sysdia solutions).

While its origin is in the gaming and entertainment industry, Unity is being used to develop interactive 3D software in many other application areas including:

• simulation (Universe Sandbox [124]);
• educational/training simulations (Surgical Anatomy of the liver demo [125]);
• design, architecture, and planning;
• virtual cities and tours;
• data visualization (NOAA [126]);
• history and cultural heritage;
• cognitive science research;
• and last but not least geography and GIS.

More examples can be found at Made with Unity [127]

In this course, we will use Unity to build simple interactive and non-interactive demos around the 3D environmental models we learned to create in the previous lessons. You will also learn how to produce 360° videos from these demos such that these can be watched in 3D using the Google Cardboard. Lastly, you will learn how easy it can be to add physics to different geometrical objects in unity and make them “behave” the way you want.

References

[1] Unity [128] (accessed on July 16, 2019)

Downloading and Installing Unity 

Before providing an overview of Unity’s main concepts and interface, let us start by downloading and installing Unity so that you can follow along and immediately start to experiment with the things you will learn about. Unity comes in three different versions: the “Personal” is completely free, while the “Professional” comes at a monthly fee ($125 at the time of this writing), and the “Plus” at a monthly fee (25$ at the time of this writing). For our purposes, the Personal Edition is completely sufficient. If you are interested in knowing more about the differences between the versions, an overview is provided at unity Store [129].

Please follow the instructions below to download and install the Personal Edition of Unity:

1. Go to Download Unity [130] and click on “Download Unity Hub“. The Hub is a new mechanism for managing the different versions of the Unity editor you want to install on your machine.

2. Execute the downloaded file (UnityHubSetup.exe) and go through the installation dialog. Once the installation is finished, if you have checked the “Run Unity Hub” option at the end of the installation process, when you click on Finish, the Unity Hub will open. Alternatively, you can also navigate to your desktop and double click on the newly created Unity icon named “Unity Hub” to start it.

3. You should be able to see a window as depicted below. Locate the profile icon on the top right corner of this window, click on it, and then select “Sign in”. A new window will pop up asking for your login information. Sign in with your unity account. If you do not have a unity account, go ahead and create one using the “create one” link on the pop-up window, and then sign in.

   

   Credit: Unity [131]
   

   Credit: Unity [131]

4. Once you are signed in, make sure the “Installs” category is selected on the left panel (the one highlighted with the red area in the figure below) and click on the blue “Add” button. A new window with pop up where you can select which version of unity you want to install. At the time of writing this, the latest stable version is 2019.1.10f1. Please download this version as it is compatible with the rest of the instructions given in the course (if you choose to install a new version of the editor, you might need to change some of the c# codes we will be using). To do so either click on the "Download archive" on the window that has poped up, or follow this link (https://unity3d.com/get-unity/download/archive [132]). This action will open a new tab in your default web browser. On that web page, select the Unity 2019.X, navigate to almost the end of the page and then click on the green button next to "Unity 2019.1.1" that says "Unity Hub". This will open this version of Unity editor in the Unity Hub for you to download and install. 
   This action will automatically proceed to the next dialog window in the Unity Hub, where you can select which extra modules to install. If you do not have Microsoft Visual Studio, it will be automatically selected for you on this list. Moreover, if you are developing on Linux or Mac, make sure to select the relevant “Build” module for your OS, so can deploy your virtual experience as a desktop application later. Click “Next” and wait until the installation is finished (This could be a lengthy process, so be patient).

   

   Credit: Unity [131]

5. Once the installation is complete, you should have a Unity icon (with a specific version) on your Desktop and “Unity” in your Start Menu. Before Starting Unity, we need to create a project. To do this, make sure the “Projects” category is selected in the Unity Hub (the one highlighted with the red area in the figure below) and click on the blue “New button”. Unity provides several templates for us to choose from (depending on the purpose of the project, e.g. 2D, 3D, VR lightweight RP, etc.). For the purpose of this lesson, we will use the default 3D template. Change the name of your project to “MyFirstUnityProject”. You can also change the location where the project will be saved to another address if you want (this is optional and solely based on your preference). Press “Create”, and after a few minutes the unity editor will be ready for us to use.

   

   Credit: Unity [131]

Getting More Familiar with the Editor

It is time to get more familiar with the interface and components of the Unity Project Editor and get some practical experience. The Unity web page [134] provides video tutorials on the basics and some advanced aspects of the Project Editor that we are going to rely on here. So please grab a cup of coffee, tea, or whatever you prefer, and watch the videos listed below. Please pause often to try out and practice the things explained there on the project you have. In particular, make sure to practice changing the view by panning and zooming, as well as quickly changing the perspective with the View Gizmo and manipulating objects in the Scene view. If something goes completely wrong, you can always delete all the folders and re-import the unity package and then start over.

Video: Interface Overview (2:17 min)

Video: The Scene View (4:20 min)

Video: The Game View (3:11 min)

Video: The Hierarchy and Parent-Child Relationships (3:48 min)

Video: The Project Panel and Importing (2:34 min)

Video: The Inspector (4:19 min)

Video: Build and Player Settings (1:50 min)

Video: Game Objects and Components (2:12 min)

Advanced Unity Concepts

That was quite a bit of new material but now you a very good understanding of the unity editor. We are almost done with our video tutorials before we engage in some fun practical work in the next section where we will build a very simple game! Please take a few minutes to learn about some of the more advanced features of unity using the following three videos.

Video: Prefabs (1:32 min)

Video: Tags (1:35 min)

Video: Layers (2:38 min)

Walkthrough: Using Unity to Build a Stand-Alone Windows Application

Introduction

In this walkthrough, we will use Unity to build a stand-alone Windows application from a SketchUp model. It will be a rather simple application, just featuring a keyboard-controlled camera that allows for steering the camera in a 3D scene with a single building.

Close Unity for now and download the Old Main model we are going to use here as a SketchUp .skp file. [104] You should still have the Unity project “MyFirstUnityProject” from the beginning of the lesson with just an empty scene ("SampleScene" in the scenes folder). This is what we will use for this walkthrough. The first task we have to solve is to export the 3D model from SketchUp into Unity.

Note: If the Old main model is too heavy for your personal computer, you can use the model for Walker Laboratory [135].

First Game in Unity: Roll-the-ball

The objective of this section is to get you familiar with the basics of C# programming in unity and the different awesome components (e.g. physics) offered by Unity. By the end of this section, you will be able to create a simple “Roll the ball” game as shown below:

Video: Roll the ball game (00:16) This video is not narrated.

The objective of the game is to roll a ball inside a box and collect all the available coins. The coins should simulate a rotation animation, capturing the attention of the player. Each time the ball hits a coin one point should be awarded to the player, a “cashier” sound effect played, and then that coin should disappear from the scene (i.e. collected).

The following steps will help you to first conceptualize what needs to be implemented and how it should be done, and second, guide you on how to utilize the Unity3D game engine to realize the needed game objects, mechanics, and functionalities.

1. Close your unity editor and open the Unity Hub. Create a new project (you should already know the steps from the previous sections). Name your project “Roll the ball” and make sure the template is set to “3D”.

2. Before beginning the programming of the game behaviors described above, we need to set up the play area (i.e. the box) in which we will be moving the ball. To achieve this, we will use the default empty scene that comes with every new unity project. The scene should be already open and ready. To add the floor, in the Unity editor, go to the GameObject menu on top, then 3D Object -> Plane. This will add a plane to the scene. Make sure the Position and Rotation Parameters of the Transform component in the Inspector menu (Red box in the figure below) are all set to zero:

   

   Credit: Unity [133]

   Next, we need to add some color to our plane GameObject. To do this, in the project folder structure, right click on “Assets” and go to Create -> Folder. Rename this folder to “Materials”.

   

   Credit: Unity [133]

   Now right-click on the newly created materials folder and go to Create -> Material. Rename this material to “PlaneMat”.

   

   Credit: Unity [133]

   Select the newly created “PlaneMat” and change its color from the Inspector panel on the right by clicking on the color picker next to the “Albedo”. Select a color of your liking.

   

   Credit: Unity [133]

   Drag the “PlaneMat” onto the Plane GameObject in the scene. The Plane should now have color.

   Next, we need to add four walls to the sides of the Plane so the Ball wouldn’t fall off. To do this, go to the GameObject menu on top, then 3D Object -> Cube. While the newly added Cube game object is selected set its transform parameters in the Inspector menu as depicted below:

   

   Credit: Unity [133]

   Doing so will result in a wall placed on the right side of the plane. To have the proper view of the play area when running the scene or inspecting the scene output in the “Game” tab, set the position and rotation parameters of the “Main Camera” game object as shown below:

   

   Credit: Unity [133]

   Create a new material with a color of your choice inside the materials folder and name it “WallMat”. Drag this material onto the Cube game object that acts as the wall.

   To add more meaning to our scene, rename the Plane and Cube game object to “Floor” and “Wall”, by right clicking on each and then selecting “Rename”.

   Select the “Wall” game object, hold down the Ctrl button and press D (you can also right click on the wall gameobject and click on duplicate). if done correctly your “Wall” object should be duplicated and labeled “Wall (1)”. Change the X value of the position parameter of “Wall (1)” from -5 to 5. Using this simple process, a new wall is added to the left side of the plane.

   Repeat the same procedure to add walls to the up and down sides of the planer. Simply duplicate the “Wall” object twice more and change its parameters as shown below:

   

   Credit: Unity [133]

   At this stage, we should have the play area ready as this:

   

   Credit: Unity [133]

   Video: Roll the ball 1: A Summary of what we have done so far (Step 2) (03:19) This video is not narrated.

3. Next, we need to add a ball to our scene and make it controlled by the player and roll around the play area. To add a ball to the scene, go to GameObject -> 3D Object -> Sphere. Change the name of this game object to “Ball”. Set its transform parameters as shown below. And add a material (Create “BallMat”) with a color to it.

   

   Credit: Unity [133]

   If we play ( ) the scene, you will realize that the ball will float in the air instead of dropping on the plane. The reason for this is that the ball game object does not have any physics attached to it yet. To fix this, select the ball game object, and in the Inspector panel, click on “add component”, search for “Rigidbody” and hit enter.

   

   Credit: Unity [133]

   Doing so will add a Rigidbody component to the ball game object. You may notice that the “use gravity” parameter of this component is checked by default, which will add gravity to this game object, and therefore it will drop onto the plane. Press play and see the result.

4. Rolling the ball: To enable the player to control the movement of the ball, we will use the arrow keys (the up, down, left, and right keys) on the keyboard. The idea is that by pressing the up arrow key we will make the ball roll forward, the down arrow key, backward, and the left and right arrow keys, to the sides. To achieve this, we will need to do a little bit of C# programming using the physics engine of Unity.

   Create a new folder in the project panel called “Scripts”. Right-click on this folder, select Create -> C# Script. Rename this script to “BallMovement” (Make sure you use the same exact spelling for the script name -case sensitive- if you are planning to copy the source code!). Drag this script onto the Ball game object. Double click on the script to open it in Visual Studio (this could take a while, and you might be asked to log in. You can either create an account or explore different options provided to you). Our simple ball movement script should look like this (Please read the comments in the code for further explanation). Either type or copy (make sure the name of the class i.e. “BallMovement” is spelled exactly as you have name named the script file) the following code into your script (note, the lines in green are comments are not executed by the engine, they are just extra explanation for you to understand the logic behind the code. Therefore, if you decide to type the following source code into your script, you do not need to write the lines in green):

   using System.Collections;
   using System.Collections.Generic;
   using UnityEngine;
   
   public class BallMovement : MonoBehaviour
   {
       // we need to define a variable that controls for the speed with which the ball rolls
       // the declaration of variables is done here
       // the type of the variable is float and its label Speed. The float type means that the value of Speed can be with precision points e.g. 15.5 
       public float Speed;
   
      
   
       // FixedUpdate is called several times per frame which will create the illusion of smooth movement
       void FixedUpdate()
       {
           // we need to take the input from the arrow keys pressed by the player. By default in unity, the left and right arrow keys are labeled as "Horizontal" and the up and down arrow keys as "Vertical"
           // we will define two variables where the value of each will be set "several times per frame" if the player presses the HorizontalMovement (i.e. the left or right key) and/or the VerticalMovement (i.e. the up or down key)
           float HorizontalMovement = Input.GetAxis("Horizontal");
           float VerticalMovement = Input.GetAxis("Vertical");
   
           // since we are dealing with three dimensional space, we will need to define a 3D vector based on which the ball should move. As such, we define a Vector3 called MoveBall that takes three parameters of X,Y,Z
           // movement on the X axis will be our HorizontalMovement (i.e. the left or right key) and on the Z axis the VerticalMovement (i.e. the up or down key). As we do not want to move the ball in the Y axis in the 3D space (i.e. up and down in the 3D space), the Y value will be set to 0
           Vector3 MoveBall = new Vector3(HorizontalMovement,0,VerticalMovement);
   
           //lastly, we will need to access the physics component of our ball game object (i.e. the Rigidbody) and add a force to it based on the values of the vector we just defined. We will multiple this force value with our Speed variable to be able to control the Speed of the force as we wish.
           gameObject.transform.GetComponent<Rigidbody>().AddForce(MoveBall * Speed);
       }
   }
   
   

   Once you are done populating your script with the source code above, make sure you hit save. It is absolutely important to remember that every time we modify a script, we save it so that Unity updates the editor based on the changes we have made.

   

   Credit: Unity [133]

   If we go back to unity and select the ball game object, we will see that in the inspector panel under the “Ball Movement” script, there is now a “Speed” variable that we can set. Change its value to 25 and hit play. Use the arrow keys to roll the ball inside the play area.

   Video: Roll the ball 2: A Summary of what we have learned so far (Steps 3 and 4) (02:11) This video is not narrated.

5. Adding coins to the scene: download the unity package file roll the ball assets [104]. Double click on this file, and it will open an “Import Unity Package” window inside unity. Click on import. Doing this will add two new folders in your assets hierarchy. One is called “Coin” which contains the model we will be using for the collectibles in our game. The other is called “Sounds” which will be covered in the next steps.

   Go to the “Coin” Folder and select the Coin model. Drag this model into your scene and place it somewhere inside the play area. Change its transform parameters as follows:

   

   Credit: Unity [133]

   We will now add a rotation effect to this coin to catch the attention of the player. To do this, create a new script inside the “Scripts” Folder called “CoinRotation”. Drag this script onto the coin game object. Double click on the script to open it in visual studio. Our simple coin rotation script will look like this (Please read the comments in the code for further explanation):

   using System.Collections;
   using System.Collections.Generic;
   using UnityEngine;
   
   public class CoinRotation : MonoBehaviour
   {
       // we need to define a variable that controls for the speed with which the coin rotates
       public float RotationSpeed;
   
       
       
       void FixedUpdate()
       {
           // We need to instruct this object to constantly rotate around its own Y axis (i.e. up)
           // this is done by accessing the transform component of this gameobject and using the built-in RotateAround unity function
           // this function takes three parameters. first one indicates the point in the 3D space where the rotation should happen. Since we want the coin to rotate around itself, we will pass the position of the coin itself as this parameter
           // the second parameter indicates around which axis should the game object rotate. we will set this to the up axis so the coin rotates around the Y
           // the third parameter indicates the speed with which the coin should rotate. for this we will use our RotationSpeed variable
           gameObject.transform.RotateAround(gameObject.transform.position, transform.up, RotationSpeed);
           
       }
   }
   

   If we go back to unity and select the coin game object, we will see that that in the inspector panel under the “Coin Rotation” script, there is now a “Rotation Speed” variable that we can set. Change its value to 15 and hit play. The coin should be continuously rotating around its Y-axis.

   Use the arrow keys to roll the ball inside the play area and hit the coin. You will notice that the ball will go right through the coin. The reason for this is that there is no collision happening between these two game objects. The ball game object has a Sphere collider attached to it by default, but the coin game object does not have a collider. Select the coin game object, in the Inspector panel select add component, search for “Box collider”, and hit enter.

   Play the scene and try to hit the coin with the ball again. You will notice that this time, the two objects will collide. We would need to have colliders on both objects to be able to determine when they “hit” each other. We need this to be able to collect the coins (i.e. remove the coin from the scene when the ball hits it). You may have noticed that during the collision, the coin will momentarily block the movement of the ball until it slides to either side of the coin and keeps rolling based on the force behind it. We do not want the player to experience this effect. In contrast, we want the game to be able to detect exactly when the two objects are colliding, but we do not want them to block each other in any way. The collision system of Unity can help us in achieving this goal using “Trigger”. If you select the coin game object, you will notice that under the “Box Collider” component that we just added, there is a parameter called “Is Trigger”. Enable this parameter by clicking on the CheckBox next to it. If we hit play, we can see that although we have colliders attached to both objects, the ball will not be blocked by the coin anymore.

   

   Credit: Unity [133]

6. Coin collection: create a new script called “CoinCollection” and attach it to the Coin game object. Open the script in visual studio and modify the code based on the instructions below:

   using System.Collections;
   using System.Collections.Generic;
   using UnityEngine;
   
   public class CoinCollection : MonoBehaviour
   {
    
       // this is the default unity function that will be called whenever two objects collide and one has the Trigger parameter enabled
       // the argument "other" refers to the object that hits the game object this script is attached to. 
       private void OnTriggerEnter(Collider other)
       {
           // In this case the we want to check if the "other" is the "Ball"
          // Make sure your ball gameobject is named “Ball”, otherwise you will receive an error!
           if (other.name == "Ball")
           {
               // if this condition is correct and indeed the Ball game object has hit this game object where the script is attached (i.e. the coin), we will remove this game object
               Destroy(this.gameObject);
           }
       }
   }
   

   Duplicate the coin game object several times and spread the coins across the play area. Play the scene and hit the coins. You will see that as soon as the ball hits a coin, that particular coin will be “destroyed”.

   Video: Roll the ball 3: A summary of what we have learned so far (Steps 5 and 6) (03:41) This video is not narrated.

7. Point system: Before destroying a coin upon collision with the ball, we want to reward the player with a point for managing to collect it. To implement this functionality, create an empty gameobject by going to GameObject -> “Create Empty”. Rename this empty game object to “Point System”. Create new script called “PointSystem” and attach it to the “Point System” game object. Open the script and modify the code based on the instructions below:

   using System.Collections;
   using System.Collections.Generic;
   using UnityEngine;
   
   public class PointSystem : MonoBehaviour
   {
       // we need to have a variable that holds the value of the points we aggregate
       public int Points;
   
       // we need to defined a GUIStyle profile for our graphical user interface (GUI) that shows the value of points to change its font size, shape, etc.
       public GUIStyle PointStyle;
   
       // this is the default unity graphical user interface (GUI) function. It will render GUI elements on the screen on a specific 2D coordinate 
       private void OnGUI()
       {
           // a GUI label to show the value of the aggregated points, place at the X=10, Y=10 coordinates, with width and height of 100, the next argument is the content that the label shows which is in this case
           // a numerical value. we need to convert the numerical value to a string (text) to be able to show it on the label. The last argument indicates which GUIStyle profile should be used for this label
           GUI.Label(new Rect(10,10,100,100),Points.ToString(),PointStyle);
       }
   }
   

   The script described above is responsible for holding the value of the points the player aggregates and visualizing it on the screen. To increase the points however, we will add a line to the script responsible for detecting when the ball hits a coin. For this go back to the “CoinCollection” script and modify it based on the instructions below:

   using System.Collections;
   using System.Collections.Generic;
   using UnityEngine;
   
   public class CoinCollection : MonoBehaviour
   {
        // this is the default unity function that will called whenever two objects collide and one has the Trigger parameter enabled
       // the argument "other" refers to the object that hits the game object this script is attached to. 
       private void OnTriggerEnter(Collider other)
       {
           // In this case the we want to check if the "other" is the "Ball"
           if (other.name == "Ball")
           {
               //if this condition is correct and indeed the Ball game object has hit us, access the Point System game object, then the PointSystem script attached to it, and increase the Points variable by 1
               GameObject.Find("Point System").GetComponent<PointSystem>().Points += 1;
   
               // we will remove this game object where the script is attached to (i.e. the coin)
               Destroy(this.gameObject);
           } }
   }
   

   Next, select the “Point System” game object. On the inspector panel on the right, in the “PointSystem” Script attached to it expand the “Point Style” GUIStyle and change it as instructed below:

   

   Credit: Unity [133]

   Press play and start aggregating points!

8. Collection sound: another feature we need to add to our game is a sound effect. We want to play a “cashier” sound effect each time the ball hits a coin. To achieve this. Create a new empty game object and rename to “Sound Management”. Select this newly created game object and click on “Add Component”, search for “Audio Source” and hit enter.

   Navigate to the “Sounds” folder in your project hierarchy and drag the “CashierSound” clip to the “AudioClip” field in the “AudioSource” component we just added to our audio management game object. Uncheck the “Play On Awake” parameter of the “AudioSource” component so the sound clip does not play as soon as we start the scene.

   

   Credit: Unity [133]

   Next, we need to instruct the game to play the audio clip whenever we collect a coin. To achieve this, modify the “CoinCollection” script based on the instructions below:

   using System.Collections;
   using System.Collections.Generic;
   using UnityEngine;
   
   public class CoinCollection : MonoBehaviour
   {
    
       // this is the default unity function that will called whenever two objects collide and one has the Trigger parameter enabled
       // the argument "other" refers to the object that hits the game object this script is attached to. 
       private void OnTriggerEnter(Collider other)
       {
           // In this case the we want to check if the "other" is the "Ball"
           if (other.name == "Ball")
           {
                //if this condition is correct and indeed the Ball game object has hit us, access the Point System game object, then the PointSystem script attached to it, and increase the Points variable by 1            
   GameObject.Find("Point System").GetComponent<PointSystem>().Points += 1;
   
               // access the audio management game object, then the AudioSource component, and play
               GameObject.Find("Sound Management").GetComponent<AudioSource>().Play();
   
               // we will remove this game object where the script is attached to (i.e. the coin)
               Destroy(this.gameObject);
           }
       }
   }
   

   Video: Roll the ball 4: A summary of what we have learned so far (steps 7 and 8) (03:13)

   Hit play and enjoy your creation!

Tasks and Deliverables

Assignment

The homework assignment for this lesson is to report on what you have done in this lecture. The assignment has two tasks and deliverables with different submission deadlines, all in week 8, as detailed below. Successful delivery of the two tasks is sufficient to earn the full mark on this assignment. However, for efforts that go "above & beyond" by improving the roll-the-ball game, e.g. create a more complex play area -like a maze- instead of the box, and/or make the main camera follow the movement of the ball, you could be awarded one point if you fall short on any of the other criteria.

To help with the last suggested extra functionality, use the following script and attach it to the main camera:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class FollowBall : MonoBehaviour
{
    // reference to the ball game object
    public GameObject Ball;

    // a vector3 to hold the distance between the camera and the ball
    public Vector3 Distance;

    // Start is called once before the first frame update
    void Start()
    {
        // assign the ball game object
        Ball = GameObject.Find("Ball");

        //calculate the distance between the camera and the ball at the beginning of the game
        Distance = gameObject.transform.position - Ball.transform.position;
    }
    // This called after the Update finishes
    void LateUpdate()
    {
        // after each frame set the camera position, to the position of the ball plus the original distance between the camera and the ball 
// replace the ??? with your solution
        gameObject.transform.position = ???;
    }
}

Task 1: Find and Review Unity 3D/VR Application

Search the web for a 3D (or VR) application/project that has been built with the help of Unity. The only restriction is that it is not one of the projects linked in the list above and not a computer game. Then write a 1-page review describing the application/project itself and what you could figure out how Unity has been used in the project. Reflect on the reasons why the developers have chosen Unity for their project. Make sure you include a link to the project's web page in your report.

Deliverable 1: Submit your review report to the appropriate Lesson 7 Assignment.

Grading Rubric Unit 3D/VR Application Review

Grading Rubric

Criteria	Full Credit	Half Credit	No Credit	Possible Points
URL is provided, the application/project uses Unity, is not already listed on this page, and is not a game.	2 pts	1 pts	0 pts	2 pts
The report clearly describes the purpose of the application/project and the application itself.	3 pts	1.5 pts	0 pts	3 pts
The report contains thoughtful reflections on the reasons why Unity has been chosen by the developers.	3 pts	1.5 pts	0 pt	3 pts
The report is of the correct length, grammatically correct, typo-free, and cited where necessary.	2 pts	1 pts	0 pts	2 pt
				Total Points: 10

Task 2: Build a Simple “Roll-the-ball” Game

If you follow the detailed instructions provided in this lecture, you should be able to have a fully working roll-the-ball game.

Deliverable 2: Create a unity package of your game and submit it to the appropriate Lesson 7 Assignment.

You can do this by simply going to the Assets in the top menu and then selecting “Export Package”.

Credit: Unity [133]

Make sure all the folders and files of your project are checked and included in the package you export (you can test this and import your package in a new Unity project to see if it runs).

Credit: Unity [133]

If you choose to implement the functionality of the main camera to follow the ball movement, please explain your solution as a comment in your script. You can add comments (texts in green) in C# by using “//”.

Helpful advice:

We highly recommend that, while working on the different tasks for this assignment, you frequently create backup copies of your Unity project folder so that you can go back to a previous version of your project if something should go wrong.

Grading Rubric for Roll-the-Ball game

Grading Rubric

Criteria	Full Credit	Half Credit	No Credit	Possible Points
Your game runs smoothly and as intended without any errors or strange behavior	6 pts	3 pts	0 pt	6 pts
Your project is complete and has all the required functionalities (coin rotation, collection, and sound effect).	3 pts	1.5 pts	0 pt	3 pts
Your Unity package has a proper folder structure, and all the files have meaningful names like it was done in the instructions	1 pt	.5 pts	0 pts	1 pt
				Total Points: 10

Summary and Tasks Reminder

In this lesson, you learned about the Unity3D game engine and the role it can play in creating 3D and VR applications. You should now be familiar with the interface of the Unity editor and the main concepts of Unity and be able to create simple projects and scenes. In the next lesson, we will explore other features of Unity for building 3D applications, including adding an avatar to a scene, creating a camera animation, importing resources from the Unity Asset Store, and creating a 360° movie that can be uploaded to YouTube and be watched in 3D on a mobile phone via the Google Cardboard.

Reminder - Complete all of the Lesson 7 tasks!

You have reached the end of Lesson 7! Double-check the to-do list on the Lesson 7 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 8.

Overview

This will be another practical lesson with several hands-on walkthroughs to explore other features of Unity that are useful for building 3D applications. This includes adding an avatar to a scene such that the user can walk in an environment around buildings; and creating a camera animation to generate a video with a pre-programmed camera flight over the scene. We will learn to import resources from SketchUp, the Unity Asset Store, and to create a 360° movie that can be uploaded to YouTube and then watched in 3D on a mobile phone via the Google Cardboard.

Learning Outcomes

By the end of this lesson, you should be able to:

• Import models from SketchUp into Unity
• Create camera flights and use the Unity Animation View
• Explain different ways to create VR applications with Unity
• Import assets from the Unity Asset Store and add them to a scene
• Create a 360° video in Unity viewable with the Google Cardboard

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

3D Applications in Unity

In Lesson 2, we introduced 3D application building software like Unity as part of a general 3D/VR application building workflow. The main role of 3D application building software can be seen in providing the tools to combine different 3D models into a single application, to add dynamics (and, as a result, making the model more realistic), and to design ways for a user to interact with the 3D environment. We also learned that 3D application building software typically provides support to build stand-alone applications for different target platforms including common desktop operating systems, different mobile platforms, and also different VR devices.

Figure: General 3D/VR application building workflow

Credit: ChoroPhronesis

Click here for a detailed description of this image.

A schematic diagram outlines the following information (in summary):

• Input data (in a variety of formats) imported into either a 3D modeling software or a 3D application building software such as a game engine

• In 3D modeling software, 3D models are created either manually or automatically based on computational methods

• The resulting models could be either directly used for visualization (i.e. on the web) or imported into 3D application building software such as Unity3D for more elaborate functionalities

• The application made from game engines (e.g. Unity 3D) can be deployed on both web and a variety of platforms including VR headsets

In the previous lesson, you already use some of the tools available in Unity for adding dynamics and interaction possibilities to a 3D scene in the form of giving the user control over the movement of gameobjects, animating gameobjects via scripting, collision detection, basic graphical user interface, and scripting more advanced behaviors, e.g. collecting coins and awarding points. As you have already experienced, adding behavior to your gameobjects is done either through standard unity components (e.g. rigidbody, colliders, etc.) or through scripting.

In this lesson, you will gain more practical experience with other tools available in Unity. But first, let's start the lesson with a brief overview of general tools utilized by Unity for 3D application building. As we go through this first part of the lesson, you will begin to realize that you have already used some of these tools in the previous lesson! We roughly divided the following list into ways to make a scene dynamic and ways to add interactions. However, this is a very rough distinction with certain aspects or tools contributing to both groups. In particular, scripting can be involved in all of these aspects and can be seen as a general-purpose tool capable of extending Unity in almost unlimited ways (e.g. in the previous lesson we used scripting to both animate our coins by constantly rotating them, as well as collecting them upon collision with the ball).

Tools to make a scene dynamic:

• Animations: Animations [139] obviously are one of the main tools to make a scene dynamic. We will talk about animations quite a bit in this lesson and you will create your own animation clip to realize a camera flight over a scene. Here are a few examples of animations that can make a scene more realistic and dynamic:

  • animations of otherwise static objects to make them appear more realistic, e.g. a plant or flag moving slightly as if affected by wind, opening and closing animations of a door
  • animations to simulate time of day: This can largely be accomplished by using animation to move a Directional Light object along a pre-programmed trajectory to simulate the movement of the sun
  • animation to move the main camera along a pre-programmed trajectory (see the third walkthrough of this lesson)
  • animations for movement of dynamic objects populating the scene, e.g. humans or cars moving through the scene in different ways (walking vs. running) and along different trajectories
• State machines: Many objects in a dynamic 3D environment can always be in one of several possible states. For instance, a traffic light may be in the states red, yellow, green; a car may be either parked, standing still with engine running, or driving; and an artificial humanoid avatar in the scene can have many possible states associated with such as sitting, standing, walking, running, reaching, etc. The transition between two possible states of a GameObject can be triggered by events. For instance, a human avatar can be programmed to switch from the "sitting" state to the "standing" state when another humanoid avatar enters the same room. State machines [140] are the Unity tool for realizing this idea and are often used to provide acting agents with some behavior, potentially involving AI methods to make the behavior appear realistic. Often, each state in the state machine of a GameObject is associated with a particular animation.
• Particle systems: While animations allow for adding dynamics to solid objects modeled as 3D meshes, there are visible effects in a dynamic environment that do not lend themselves to be modeled and handled in this way. Examples are smoke and flames, liquids, clouds, rain, and snow, etc. Particle systems [141] are the most common approach to model these things and can be easily created in Unity. A particle system consists of several, up to a large number of particles, which are simple images or meshes, that are moved and displayed together, for instance, to form the impression of smoke. A particle system has many parameters to control the properties of the particles, their movement and dynamics, their lifetime, etc.
• Physics: Physics is an important part of a really dynamic scene and needed to simulate the behavior of objects under gravity, collisions, and other forces. Unity's physics engine [142] is capable of simulating the behavior of solid objects under these forces. You have already used some parameters of the physics engine in the previous lesson including rigidbody and collider.
• Sound: Audio is another important input mode to make a 3D scene appear more realistic and increase its immersiveness. Sometimes you may also just want to add some nice music in the background of your application or add some commentary that is played to explain something in the scene or give instructions to the user. Unity's audio support [143] is essentially based on the idea of attaching AudioListener and AudioSource components to GameObjects in the scene. The AudioListener is typically attached to the main camera. An AudioSource attached to an object can be set up so that the played audio track gets louder when the camera approaches that object. Unity supports real 3D spatial sound which also allows the user to tell which direction a sound is coming from. You have already used an example of the AudioSource in the previous lesson for coin collection.

Tools to add interactions:

• User input: We already saw how keyboard input can be used to control the movement of an object (the ball in our previous lesson) in the scene based on some scripting and manipulation of the transformation parameters of an object. Similarly, user input can be used to interact with the objects in the scene in some other way such as opening doors, pushing/pulling some other object, pressing a button, etc.Unity's Input Manager [144] can be used to configure the input devices for a project. We will see the use of mouse as in input controller in the next lesson.
• GUI: 3D applications still make use of classical 2D graphical user interface (GUI) elements to display information to the user, provide selection choices, or collect other user input. For instance, a GUI menu could be used to allow the user to immediately jump to one of several pre-defined locations in the nice or a dialog box can be used to allow the user to enter input values for important variables like movement speed. However, Unity's UI system [145] is a bit out of the scope of this introductory course. The Unity web page [146] contains a lot of video tutorials explaining the different UI elements if you are interested in this aspect. We have already experienced creating a very simple GUI label that holds the value of the “points” awarded to a player in the previous lesson. In addition, here is a very short video demonstrating a simple Unity UI for triggering different animation states of a face, connecting back to the things we discussed above, regarding animations and state machines:Overview of An Interactive Interface Demo in Unity [147]. The Unity Assets Store contains thousand of assets related to creating menus and GUI in general.
• Complex 3D/VR interactions: IIn particular with the advent of special VR controllers to complement consumer-level head-mounted displays, new ways to interact with objects in a 3D or VR setting are conceived every day. Modern VR controllers, such as the HTC Vive Controller [148], not only combine buttons, trackpads, and triggers that allow for triggering interactions by certain clicks, but the fact that the position and orientation of the controllers in the room are tracked allows for completely new interactions by touching or pointing at objects in the scene, or by performing complex gestures with the controllers, e.g. for moving, rotating, or scaling other objects, teleporting in the scene, or measuring distances or volumes. Template scripts for many of these interactions are already publicly available. Devising and implementing completely new ways to interact with objects in the scene typically requires quite a bit coding though.

Since this is a Geography/GIS class, we also want to briefly mention that support for importing GIS data into 3D modeling and application building software to employ it as part of 3D or VR application is slowly increasing. The image below shows a world map GameObject consisting of meshes for all countries in the Unity editor. The meshes were created from an ESRI shapefile using BlenderGIS, an add-on for importing/exporting GIS data into Blender. Then the meshes were exported and imported into Unity as an Autodesk .fbx file.

World map GameObject consisting of meshes for all countries in the Unity editor

Credit: Unity

Recently, Mapbox announced the Mapbox Unity SDK [149], providing direct access to Mapbox hosted GIS data and Mapbox APIs within Unity. We believe that more initiatives like this can be expected for the near future, partially driven by the huge potential that current developments in VR and AR technology offer to GIS software developers.

Walkthrough: From SketchUp Model to Unity

In this walkthrough, we will see how we can import a 3D model from a SketchUp into Unity. Although the Unity Asset Store contains a large repository of free 3D models that you can use in your projects, depending on the requirements, it might be needed to import custom models into Unity. As you have already gained some experience in modeling 3D objects in SketchUp, it will be useful to see how you can import your model directly into Unity and add behaviors to it. In this walkthrough, we will import a SketchUp model of the “Walker” building into Unity, extract its materials and textures, and add it to a scene. Then, we will add a first-person controller prefab from the standard assets of unity which will enable us to walk around our scene.

You should still have the Unity project “MyFirstUnityProject” from the beginning of the previous lesson. Open your Unity Hub and open the “MyFirstUnityProject” (if you do not have this project anymore, follow the instructions from the previous lesson on how to create it. Make sure you also import the first package you used in the previous lesson – download here [104]). We will use this project to perform this walkthrough. Download the following Walker Laboratory [135] SketchUp file and once the download is completed, navigate to the location where it is downloaded (by default it should be in your User\[your username]\Downloads).

In the Unity editor, create a new folder in your project folder structure and rename it to “Walker Building”. Drag and drop your downloaded SketchUp file into this folder. Create a new scene in the same folder, by right-clicking on the folder from the folder structure panel -> create -> Scene. Name this scene “WalkerScene”. Open this scene and drag your WalkerLabaratory SketchUp model into the scene. Reset its position and rotation values to zero. Change the position of the Main Camera in the scene so the building is within its view. Change the X rotation of the directional light to adjust the lighting/show of your scene.

You may have noticed that the Building object we have imported looks rather dull and is missing lots of textures and materials. To fix this problem, we need to extract all the materials and textures from the prefab of the model so that Unity knows how to map them. Select the prefab of the model you have imported into your folder. You can see that in the inspector menu it is detected that this is a SketchUp model, but all the materials for the individual components of this model are missing (labeled as none).

Credit: Unity

To fix this, click on the “Extract Textures …” button in the inspector panel. Create a new folder in the pop-up window and rename it to “Textures”, and then click on “Select Folder”. Do the same for Materials by clicking on the “Extract Materials …” and creating a new folder called “materials”. Once done, the model should have all the materials and textures added to it.

Unity SketchUp (01:39) This video is not narrated.

To add a first-person controller, we will use the default FPSController prefab on unity. This is the same prefab that you have used in the first part of the previous lesson. Navigate to the “Standard Assets” folder -> “Characters” -> “FirstPersonCharacter” -> “Prefabs”. Drag and drop the FPSController prefab into your scene. Place the FPSController somewhere on the plane (you can start with X=0 and Z=0 for position and then move the FPSController along these axes) and make sure the Y value of its position parameter is set to 1. Rotate the FPSController GameObject so it faces the building. If you play the scene now, you will notice that the FPSController will fall through the plane. This is because we have not yet attached colliders to our objects. The FPSController prefab already has a collider so all we need to do is to add colliders to the plane and all the children of the Walker GameObject.

To achieve this, you have two options. You should either select all the child objects of the building prefab that has a “Mesh renderer” component attached to it and add a “Mesh collider” to that object manually. Since these objects have meshes, the physics engine of Unity will use the shape of the mesh to create a collider with the same shape around that object. Or you can select the Walker prefab from your project folder and check the “Generate Colliders” parameter in the inspector menu under SketchUp option, and then press “Apply”. Press play, and you should be able to navigate on this plane using the WASD keys on your keyboard and look around using your mouse.

Unity SketchUp 2: 01:22 This video is not narrated.

Once you are done experimenting with your new creation, disable the FPSController GameObject by clicking on it once, and then unchecking it from the inspector panel.

Credit: Unity

In the next walkthrough, we will cover animating a camera and for this, we need to use the static Main Camera in our scene.

Animations and State Change in Unity

Animations are one of the main methods to bring some life into a 3D scene in Unity and make it dynamic. Objects in the scene may be permanently going through the same looped animation (think of the leaves of trees moving in the wind) or they may be in a particular animation state that can change, potentially triggered by some external event (think of a human avatar that can switch from a walking animation to a running animation and that may have many other animations associated with).

The theoretical concept behind this idea is the notion of a State Machine: Each GameObject is always in one of a pre-defined set of states (state for walking, state for running, etc.), potentially associated with a particular animation. The transition from one state to another is triggered by some event which, for instance, can be some user input, a collision with another object, or simply a certain amount of time passed. The states with possible transitions between them form the object's state graph as shown in the image below. The state graph together with a pointer to the object's current state when the project is running constitute the object's state machine.

Enter caption here

Credit: Unity [140]

In this course, we will use animations only to animate the main camera to create a camera flight over the scene. Other uses for animations for 3D/VR applications could be making objects like plants more realistic, creating movable objects like doors, or populating the environments with dynamic entities like people or cars. Our camera flight will only need a single animation and, hence, state. So we won't have to deal with complex state graphs and the transitions between states. Nevertheless, please watch at least the first 90 seconds of the following video to get a bit of an understanding of the Unity's Animator component and controller and what happens under the surface when working with Animation State Machine. Don't worry if some of the details don't get completely clear.

The Animator Component (03:45)

The main tool for creating an animation in Unity is the Animation View. It's a very powerful but also slightly complex tool so that we will only get to know its basic features here. For details, please see the documentation available here [150]. The Animation View is for creating and editing animation clips as well as previewing them. Please watch the following short video to familiarize yourself with the idea of creating a camera animation clip in Unity using the Animation View. You do not have to try out what is shown in the video yourself; we will be doing this in the walkthrough in the next section (the walkthrough will slightly deviate from what is shown in the video though).

The Animation View (06:20)

The Unity web page [151] has more videos related to animations available, dealing for instance with importing animations created in other software, creating animations for a humanoid avatar, etc. You won't need to know about these more advanced topics for this course, but if this a topic of interest to you, feel free to check these out.

Unity-based VR Applications for Mobile Devices

You already learned about building stand-alone 3D applications for Windows with Unity. Creating stand-alone software that allows for experiencing the application in real 3D with the help of a VR head-mounted display connected to the Windows computer is not much different. While you probably won't be able to try this out yourself (unless you own a device such as the Oculus Rift or HTC Vive), Unity VR Overview [153] provides a good overview on VR support in Unity and building VR stand-alone applications. Additional information can be found at Unity Learn [154].

Figure: Building for different platforms.

Credit: ChoroPhronesis

Similarly, you can build stand-alone applications for mobile platforms such as Android or iOS and even have the built app directly pushed to your mobile device. This requires additional software to be installed on your computer, such as the Android SDK if you are building for an Android phone or tablet. However, despite a lot of progress in hardware and, in particular, graphics performance on modern mobile devices, smoothly rendering 3D scenes in high quality on a mobile phone, for instance, can still be problematic. This is particularly true if the scene needs to be rendered for the left and right eye separately to support a VR device like the Google Cardboard. We will, therefore, take a different approach in the following walkthrough and instead use Unity to produce a 360° video that can be shared, for instance via YouTube, and then be experienced in real 3D on pretty much any(!) smartphone with the Google Cardboard. While this approach is limited in that no interaction between the user and the objects in the scene is possible, advantages of the approach in addition to being less dependent on high-performance hardware are that we do not need to produce different builds for different mobile platforms and no additional software needs to be installed. If you, however, are interested in exploring how to build stand-alone (VR) applications for Android devices, the Android environment setup instructions [155] and the video below are good starting points.

Video: Live Training 10 Mar 2014 - Mobile Development (1:08:46)

Tasks and Deliverables

Part 1: Participate in a Discussion

Please reflect on your 3D spatial analysis experience, using ArcGIS Pro; anything you learned or any problems that you faced while doing your assignment as well as anything that you explored on your own and added to your 3D spatial analysis experience.

Part 2: Assignments

The homework assignment for this lesson is to apply the techniques you learned in this lecture to the historic campus scene that you already saw in the previous lesson (it should be already in your “MyFirstUnityProject” project under the “Historic Campus” folder). You are supposed to create a nice camera animation and produce a stand-alone windows application as well as a 360° movie. The video should be uploaded to your YouTube channel and shared together with your Windows application with the rest of the class by posting the links and a short description on the forums. Finally, you are supposed to provide constructive feedback on a video created by one of your classmates and submit a write-up reflecting on this assignment.

The assignment has several tasks and deliverables with different submissions deadlines, all in week 10, as detailed below. Successful delivery of the tasks 1-6 is sufficient to earn 90% on this assignment. The remaining 10% is reserved for efforts that go "above & beyond" by improving the historic campus model, e.g. in ways suggested below, before creating the video and stand-alone Windows application.

Task 1: Create camera animation

For this assignment, you will use the historic campus scene from Lesson 8. We suggest you start with a fresh download of the Unity project: Historic Campus [164]. The campus models have already been imported and placed in a scene (main) and things should look like in the screenshot below when opening the project in Unity. You are now supposed to create a ~1.5min camera flight over the scene using what you learned about animating the camera in this lesson. But before you start with the camera flight, you may want to think about what you want to do for "above & beyond" efforts (see details below) as this may have an effect on the camera trajectory you want to create (e.g. in case you are going to add additional models to the scene).

Credit: Unity

Task 2: Produce a stand-alone Windows application

Produce a stand-alone Windows application showing your camera animation as you learned in this lesson. Please note that if you put in work for above & beyond points, this needs to be done before creating the Windows application so that these efforts are included.

Deliverable 1: Create a .zip file containing all the files and folders of your built.

Credit: Unity

Upload the application to the PSU Box Comunity Folder [165] and include the link to your application when posting on the Lesson 8 Video Discussion (see Task 4).

Task 3: Produce 360° video

Produce a 360° video showing your camera animation as you learned in this lesson. Don't forget to adapt the meta information before uploading the video to your personal YouTube channel. Please note that if you put in work for above & beyond points that should be visible in the video, this needs to be done before creating the video so that these efforts are included.

Deliverable 2: Upload the 360° video to your YouTube channel and share it via a link. Include the link when posting on the Lesson 8 Video Discussion (see Task 4).

Task 4: Share stand-alone application & video

Deliverable 3: Create a post on the Lesson 8 Video Discussion including the links for your Windows stand-alone application & 360° video. Add a one-paragraph description of what you did to improve the model for above & beyond points, if anything. If your above & beyond efforts are only visible/audible in either the 360° video or the stand-alone application, please make this clear as well.

Task 5: Constructive feedback on other student's video

Watch a video posted by one of your classmates and write down 1-2 paragraphs in which you provide constructive feedback. If possible, please pick a video that has not yet received any feedback. Comment on the technical and aesthetic quality of the video and, if applicable, the extensions made to the model. Make suggestions on how the work can be improved or extended. If the description mentions that some additional effort is only included in the Windows stand-alone application, you can feel free to ignore that part.

Deliverable 4: Post your feedback as a reply to the respective video post on Lesson 8 Video Discussion.

Task 6: Reflection Paper

Create a ~700 words write-up reflecting on what you learned in this assignment, what problems you ran into, if and how you were able to resolve these, etc. Please also include a complete description of things you did for above & beyond efforts. Provide some final assessment of your experience with Unity.

Deliverable 5: Submit your write-up to the Lesson 8 Assignment.

Above & beyond:

This part of the assignment is for allowing you to let your creativity run free to extend or improve the historical campus scene and/or explore some features of Unity development on your own based on one of the many tutorials available on the web. What you do is up to you; the following are just two suggestions for directions in which you can think:

1. Add additional models to the historical campus scene: You can extend the scene by adding additional objects. This could be your own building model from Lesson 3, some free assets you find by browsing the Unity Asset Store [121] (view How to use the Asset Store [166] for instructions), or some other free 3D models available on the web. The purpose of this assignment is to demonstrate your Unity skills to us, so realism is not important here. But what you add should generally fit the historic campus scene (additional buildings, other tree or plant models, models of people, etc. but no spaceships, robots, or the likes). If you want to include your own building model but the building is already there, feel free to import it and place your model at a different location in the scene. Please don't go overboard with the number of objects you add; a very few to show that you are able to import models and place them in the scene is all that it takes.
2. Add some background music, other sound effects, or commentary: Adding sounds and background music to a scene in Unity is not complicated but you should only try to include this into the Windows stand-alone application with the standard camera object because recording sound won't work with the 360° video recording. The first four minutes of this Beginner Unity Tutorial [167] will give you a basic understanding of what needs to happen: The audio file (.mp3, .wav, etc.) you want to use needs to put into the Assets folder of your project. The main camera object needs to have an AudioListener component and this component needs to activated (checkbox at the top left). Then you can attach an AudioSource component either to the camera itself or any other GameObject in the scene. Attaching it to the camera itself works fine for background music or commentary while attaching it to objects in the scene works best for 3D sound that should get louder when the camera approaches a particular object or location in the scene. Finally, the audio track needs to be dragged from the Assets window onto the AudioClip property of the AudioSource component.

If you are looking for some relaxing background music, you can browse freesound [168]for a suitable track that is freely available. Alternatively, or in addition to using music, you can record your own commentary describing either the scene or what you did for this assignment and have this played during the camera flight. If you experience any problems with adding sound, feel free to help each other out in Lesson 8 Discussion.

Helpful advice:

We highly recommend that, while working on the different tasks for this assignment, you frequently create backup copies of your Unity project folder so that you can go back to a previous version of your project if something should go wrong.

For the camera animation, we recommend using many intermediate keyframes unless the camera is just moving in a straight line. In particular, try to not have a rotation of more than 90° between two successive keyframes in the animation.

Grading Rubric: Deliverables 1 and 2 (Windows Stand-Alone Application & 360° Video)

Criteria	Full Credit	Half Credit	No Credit	Possible Points
Camera flight has the correct length, is smooth and aesthetically pleasing.	3 pts	1.5 pts	0 pts	3 pts
Windows stand-alone application is shared via Box and shows the camera flight.	3 pts	1.5 pts	0 pts	3 pts
Video is shared via YouTube and shows the camera flight.	3 pts	1.5 pts	0 pts	3 pts
Video has been correctly created and uploaded as a 360° video allowing to look around while the camera is moving over the scene.	3 pts	1.5 pts	0 pts	3 pts
Above & Beyond: The number of points varies depending on how much effort was required (e.g. placing an additional 3rd-party 3D model in the scene will give fewer points than placing a self-made model or adding background music to the scene).	3 pts	1.5 pts	0 pts	3 pts
				Total Points: 15

Grading Rubric: Deliverable 5 (Reflection Paper)

Criteria	Full Credit	Half Credit	No Credit	Possible Points
Paper clearly communicates what you learned in this assignment, what problems you ran into, if and how you were able to resolve these, etc. Paper also includes a complete description of things you did for above & beyond efforts and a final assessment of your experience with Unity.	5 pts	2.5 pts	0 pts	5 pts
Write-up is of the correct length and is well and thoughtfully written.	5 pts	2.5 pts	0 pts	5 pts
The document is grammatically correct, typo-free, and cited where necessary.	2 pts	1.0 pts	0 pts	2 pts
				Total Points: 12

Summary and Tasks Reminder

In this lesson, you significantly extended your Unity skills by learning how to add a user-controlled avatar and enable colliders for the objects in the scene, and how to create a camera animation with the Animation View. We also spoke about the different ways to create VR experiences for different devices with Unity, including using 360° videos for the Google Cardboard as an efficient option for devices that may not have the hardware performance to run VR applications in real-time. We have now come full circle with regard to the 3D and VR application building workflows we discussed in Lesson 2. We hope that these last two lessons focusing on Unity and the skills you acquired in them will prove to be a helpful complement to your 3D modeling and analysis repertoire.

Reminder - Complete all of the Lesson 8 tasks!

You have reached the end of Lesson 8! Double-check the to-do list on the Lesson 8 Overview page to make sure you have completed all of the activities listed there before you begin Lesson 9.

Overview

This will be both a practical and theoretical lesson that shows you a complete Unity project with some advanced functionalities. You will be provided with a simple “City builder” unity game where you can instantiate buildings in a city and animate cars to patrol over a specified path. The objective is not to teach you how to program this game, as it will be outside of the scope of this course, but rather demonstrate to you what is possible to be done in Unity with a few classes and some built-in features of the game engine. Furthermore, we will explore some of the VR specific mechanics that are used in game engines such as Unity for making the experiences more interactive (e.g. different types of locomotion, and interaction with objects).

Learning Outcomes

By the end of this lesson, you should be able to:

• Have an impression of how much effort is needed to create a simple game such as City Builder
• Have a general understanding of some of the advanced features of Unity such as AI (artificial intelligence) and dynamic GameObject instantiation
• explain different locomotion mechanics used in VR to move the player around a scene
• explain different interaction mechanics used in VR to interact with objects

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

The City Builder Game

Introduction

Download the City Builder Unity Package [104]. The package includes all the resources used in the city builder game. This is a very simple game, where the user can select different buildings from a menu and place them in specific locations in a city (only some areas allow for placing a building) using their mouse. When selecting a building to be placed in the city, its base will change color. The red color indicates that the current location of the building is illegal (and therefore it cannot be placed there), whereas the blue color indicates a legal location. The color of the base will continuously change as the user moves the building in the city to find a location to place it. Once the building is in a legal location, the user can click down on the mouse (left click) to place the building in the city. If the user clicks on an already placed building, the base of the building will become purple, indicating that it has been selected. Once selected, buildings can be removed from the city by pressing the “D” button. Moreover, the user can move the camera along the X and Z axes using the arrow keys on the keyboard (left, right, up, and down keys). This will help the user navigate the environment when placing buildings. Lastly, there is a car that patrols a street over a specified path. A quick video of this game can be seen below:

Video: City Builder Demo (00:54)  This video is not narrated.

Import this package into a new Unity project. Once the import is done, navigate to the folder “City Builder”, and open the scene with the same name “City builder”. Make sure your “maximize on play” option in your game window is enabled, and then play the scene.

Credit: ChoroPhronesis [41]

When the scene is playing, you can see a blue car patrolling over a specified path. You can use the arrow keys on your keyboard (left, right, up, down keys) to move the camera along the X and Z axes. If you click on the buttons of the menu on the left side of the screen, you can select different buildings to place in the city. Select a building, move it over a legal position and left click with your mouse to place it down. Place a few more buildings in the city. Click on a building that is already in the city. You will notice that its based will change color to purple. Press the “D” button on your keyboard to delete this building.

Setting up the City Builder from Scratch

To give you an idea of how such a scene can be set up, we will go through a series of steps to recreate the City Builder scene. All the resources you need are already inside the “City Builder” folder.

1. Create a new scene in the City Builder folder, and name it “My City Builder”
2. Navigate to the “Prefabs” sub-folder and drag the “Plane” prefab into the scene. Reset its position to (64,-0.2,0) and change its scale to (50,1,50). Notice that the plane object already has a Box Collider attached to it.
3. Select the Main Camera GameObject and set its position to (65,85,-85) and orientation to (74,0,0)
4. Drag the “Roads” prefab onto the scene and set its position to (42,1.1,-17). This is a rather detailed object, with many child objects under it. If you expand this GameObject and examine its children, you will notice that some of them have the tag “Street”, and some of them do not. We use these tags later when we want to decide whether the building, we want to add to the city is being placed on the street or not. More precisely, if the building object that is following our mouse pointer collides with another object with either the tag “Street” or “Building” it is considered an illegal location.
   

   Credit: ChoroPhronesis [41]
5. Drag the “Car” Prefab onto the scene and set its position to (59.5,0,-1.2). If you look at the inspector panel of the car GameObject you have just added to your scene, you will notice that it has a script attached to it called “Car AI”. This script utilizes the AI engine of Unity which implements a path finding algorithm, so the car moves through several “Patrol points”. These points are not assigned yet, but we will do them shortly. Apart from this script, the Car object also has a component called “Nav Mesh Agent”. This is a native unity component as part of its AI engine that marks a GameObject as a navigatable agent. Since we want the car to act as such an agent, we need to have this component added it. There are many parameters that you can set for the Nav Mesh Agent including the movement and turn speed, acceleration, obstacle avoidance, etc. for more information watch NavMesh Agent - Unity Official Tutorials [169].
   

   Credit: ChoroPhronesis [41]
6. Before adding the patrol Points to the Car AI script, we must tell unity which parts of our scene are “walkable” by the Nav Mesh Agent. To do this, go to the “Window” menu on top of the Unity editor, Select AI -> Navigation.

7. A new panel called Navigation will be opened on the right side of the editor. Select the “Bake” tab on this panel and click on the Bake button.

   

   Credit: ChoroPhronesis [41]

   After a few seconds your scene should look like this:

   

   Credit: ChoroPhronesis [41]

8. Navigate to the Prefabs folder again and drag the “Waypoints” prefab onto the scene, and set its position to (62.5,0,-70.2). This GameObject has four children, where each act as a patrol point for our Car AI script. You can inspect the position of each these points in the scene. Now, if we select our Car GameObject again, and look at its properties in the inspector menu, we can add patrol points to the Car AI script. Simply drag each waypoint (children of the WayPoint object named Waypoint 1 to 4) to each element (0 to 3) place in the Car AI script.

   

   Credit: ChoroPhronesis [41]
9. Hit play, and you should have your car move from one Patrol Point to the next over a loop. As an exercise (not evaluated), change the placement of the Waypoints in the scene so the car goes around a block.

10. Inside the City Builder folder, navigate to the “Scripts” subfolder. Drag the “BuildingManager”, “BuildingPlacement”, and “MoveCamera” scripts onto the Main Camera GameObject in your scene.

    

    Credit: ChoroPhronesis [41]

11. Most of our functionalities are covered by these three scripts, so we will briefly go over them. The “Building Manager” script takes care of generating the menu on the left side of the screen and adding buttons to it to the number of buildings we want to have in our menu. You can see that this script has a property called “Buildings” in the editor. If you expand this option, you will see a placeholder for “Size”. Change this value to 3. This means that we want our menu to hold three buttons each representing a different building, so we can select them and place them in the city. When you change the value to 3, you will notice that three new empty placeholders are shown. Go to the Prefabs subfolder again, and drag the Blue, Brown, and Green building prefabs to each of those placeholders.

    

    Credit: ChoroPhronesis [41]

    Now to the number of buildings we have added to this list, we will have buttons with the same name as the building prefabs generated for us. Clicking on each button will inform the “Building Placement” Script, which building to generate and attach to the mouse pointer. For more detail, please examine the fully commented script of “Building Manager”.

12. Next, we will examine the “Building Placement” script. There are three properties for this script in the editor that we need to change. First, we need to add different materials for the “base” of the buildings to indicate illegal and legal positions of the building by means of color. Navigate to the “Materials” subfolder and drag the “Valid” material to its corresponding placeholder in the inspector panel. Do the same for the “Invalid”. Once the “Building Manager” script informs the “Building Placement” script which building is requested to be generated, the “Building Placement” script will instantiate (create a copy on the fly) that building and will attach it to the position of the mouse pointer. Furthermore, it will use the valid and invalid materials we’ve just added to the component, so it can change the color of the base of the buildings to red when they collide with buildings and streets (i.e. invalid), or otherwise blue (i.e. valid).

    

    Credit: ChoroPhronesis [41]

    The “Building Placement” script is also monitoring the clicking of the mouse by the user. If there is a building attached to the mouse pointer and the position is legal and the user clicks on the mouse, it will place the building on that exact location and will disable its base (so it looks realistic when placed in the city). Notwithstanding, we also want the user to be able to click on a building that is already placed in the city so they can remove it. This means that our “Building Placement” implements a second check condition to see if no building is attached to the mouse pointer at the moment when the user clicks on the left mouse button. If that is the case, it also checks whether what the user has clicked on is actually a building or not. For this check, our script uses the “Building Mask” property, as shown in the editor. All the building prefabs we have added to the “Building Manager” script already belong to the “Building” layer. All we need to do is tell unity which layer to look for when the user clicks on objects in the city. As such, we need to create a new layer and set that as our Buildings Mask. For this, click on the “Layer” dropdown menu on top of this panel, and select “Add Layer”.

    

    Credit: ChoroPhronesis [41]

    Create a new layer called “Building” and set the Building Mask to that layer (this layer should already exist in your layer list, if not, create it).

    

    Credit: ChoroPhronesis [41]

13. The last property we need to set in the unity editor is related to the “Move Camera” script. We need to indicate a speed value for the movement of the camera, so we can move it using the arrow keys. Set this value to 15.

    

    Credit: ChoroPhronesis [41]

14. That is all you needed to do in order to set up a fully working City Builder scene with the assets that were provided to you. Press play and enjoy your creation. You should be able to move the camera around, place buildings in designated areas of the city and remove them. You may notice that we have not covered which script handles the color changing of the base of buildings when they are selected, as well as the function for deleting them. This is because, each of the building prefabs we added to the “Building Management” script has a script called “PlacableBuilding” that takes care of these functions, and we do not need to set anything up for them to work. However, in short, once a building that is already placed in the city is clicked on, the “Building Placement” script detects which exact building was clicked and triggers the “PlacableBuilding” script attached to it. This script will then change the base color of that building to purple, and if the user presses the “D” key, it will remove that building from the scene.

You can also watch this complete video tutorial on how to assemble this scene.

Video: CityBuilder Instructions (6:39) This video is not narrated.

Note: Although we did not directly go over the scripts line by line, I strongly recommend that you look at the scripts inside the “Scripts” subfolder to have a better understanding of how some of these functions are implemented in C#. All the scripts are fully commented for your convenience.

Common Mechanics Used in VR Development

Most of the mechanics used in desktop experiences can also be used in VR. However, not all of them are best choices for VR experiences. Two of the most prominent examples are locomotion and interaction mechanics. In this section, we will briefly explore the different locomotion and interaction mechanics that are designed specifically for VR experience.

Locomotion in VR

Locomotion can be defined as the ability to move from one place to another. There are many ways in which locomotion can be implemented in games and other virtual experiences. Depending on the employed camera perspective and movement mechanics, the users can move their viewpoint within the virtual space in different ways. Obviously, there are fundamental differences in locomotion possibilities when comparing 2D, 2.5D, and 3D experiences. Even within the category of 3D experiences, locomotion can take many different forms. To give you a very general comparison, consider the following:

Locomotion in 2D games is limited to the confines of the 2D space (X and Y axes). The camera used in 2D games employs an orthogonal projection. Therefore, the game space is seen as “flat”. In these experiences, users can move a character using mechanics such as “point and click” (as is the case in the most 2D real-time strategy games) or using keystrokes on a keyboard or other types of controllers. The movements of the orthogonal camera in these experiences is also limited to the confines of the 2D space. Consequently, the users will not experience the illusion of perceiving the game through the eyes of the character (e.g. Cuphead [170], Super Mario Bros, [171] etc.). Evidently, the same type of locomotion can be employed in the 3D space as well. For instance, in the City Builder game, we have seen in this lesson, the camera uses a perspective projection. However, the locomotion of the user is limited to the two axes of X and Z. The three-dimensional perspective of the camera (viewpoint of the user), however, creates an illusion of depth and a feeling of existing “in the sky”. In more sophisticated 3D games, such as First-person shooters (FPS [172]) where it is desired that the players experience the game through the eyes of their character, the feeling of movement is entirely different. We stress the word “feeling” since the logic behind translating (moving) an object within a 2D space compared to a 3D space from one point to another is not that different. However, the differences in the resulting feeling of camera movements are vast (unattached distant orthogonal vs. perspective projection through the eyes of the character). In many modern games (not necessarily shooters) with a first-person camera perspective, the players can be given six degrees of freedom for movement and rotation.

Six degrees of freedom

Credit: Six degrees of freedom [173] by Horia Ionescu via Wikimedia Commons [174] (Public Domain)

Video: Making an FPS Game with Unity using the Asset Store (3:32)

The FPS Controller we used in the previous lessons is an example of providing such freedom (except for rotation along the Z-axis). The mechanics for the movement in such games, however, are almost all the time through a smooth transition from one point to another. For instance, in the two different locomotion mechanics we used in this course (FPS Controller, and camera movement) you have seen that we gradually increase or decrease the Position and Rotation properties of the Transform component attached to GameObjects. This gradual change of values over time (for as long as we hold down a button for instance) creates the illusion of smoothly moving from one point to another.

As was previously mentioned, there are many ways in which locomotion can be realized in virtual environments, depending on the type and genre of the experience, and the projection of the used camera. Explaining all the different varieties would be outside the scope of this course. Therefore, we will focus on the one that is most applicable in VR.

The experience of Virtual Reality closely resembles a first-person perspective. This is the most effective way of using VR to create an immersive feeling of perceiving a virtual world from a viewpoint natural to us. It does not come as a surprise that in the early days of mainstream VR development, many employed the same locomotion techniques used in conventional first-person desktop experiences in VR. Although we can most definitely use locomotion mechanics such as “smooth transition” in VR, the resulting user-experiences will not be the same. As a matter of fact, doing so will cause a well-known negative effect associated with feelings such as disorientation, eyestrain, dizziness, and even nausea, generally referred to as simulator sickness.

According to Wienrich et al. “Motion sickness usually occurs when a person feels movement but does not necessarily see it. In contrast, simulator sickness can occur without any actual movement of the subject” [1]. One way to interpret this is that simulator sickness is a form of physical-psychological paradox that people experience when they see themselves move in a virtual environment (in this case through VR HMDs) but do not physically feel it. The most widely accepted theory as to why this happens is the “sensory conflict theory” [2]. There are, however, several other theories that try to model or predict simulator sickness (e.g. the poison theory [3], the model of negative reinforcement [4], [5], the eye movement theory [4-5], and the and the postural instability theory [6]). Simulator sickness in VR is more severe in cases where the users must locomote, particularly using smooth transition, over a long distance. As such, different approaches have been researched to reduce this negative experience. One approach suggested by [1] is to include a virtual nose in the experience so the users would have a “rest frame” (a static point that does not move) when they put on the HMD.

A Virtual Nose as a Rest-Frame

Credit: C. Wienrich, CK. Weidner, C. Schatto, D. Obremski, JH. Israel. A Virtual Nose as a Rest-Frame-The Impact on Simulator Sickness and Game Experience. 2018, pp. 1-8

Other approaches such as dynamic field of view (FOV) reduction when moving or rotating have also shown to be an effective way to reduce simulator sickness.

Dynamic Field of View (FOV)

Credit: Tom's Hardware [176]

In addition to these approaches, novel and tailored mechanics for implementing locomotion, specifically in VR, have also been proposed. Here we will list some of the most popular ones:

• Physical movement: In the earlier version of VR HMDs, no external sensors were used for tracking the position of the users. As such, physically moving around a room and experiencing translation (movement) inside the virtual environment was not easily achievable (in few examples other forms of sensors such as Microsoft Kinect were used for this purpose to some extent). In the newer models of HMDs however, external sensors were added to resolve this shortcoming by providing room-scaled tracking. For instance, Oculus Rift, and HTC Vive both have sensors that can track the position of the user within a specific boundary in a physical space. This allows the users to freely and naturally walk around (as well as rotate, sit, and jump) within that boundary and experience the movement of their perspective in VR. As we have already seen in Lesson 1, the latest generations of HMDs such as Oculus Quest employ the inside out tracking technology which eliminates the need for external sensors. Using these HMDs, the users are not bound to a specified physical confine, and they can freely move around in a much larger physical space. This method of locomotion is the most natural one we can use in VR.

Video: Oculus Insight VR Positional Tracking System (Sep 2018) (02:39)

• Teleportation: teleportation is still considered the most popular locomotion system in VR (although this may change soon due to the emergence of inside out tracking technology). It allows users to jump (teleport) from one location to another inside a virtual environment. There are different types of teleportation as well. The most basic form is instant teleportation where the perspective of the user is jumped from one location to another instantaneously when they point to a location and click on their controller.

Other forms of teleportation include adding effects when moving the user’s perspective from one location to another (e.g. fading, sounds, seeing a project of the avatar move, etc.), or providing a preview of the destination point before actually teleporting to that location:

Another interesting and yet different example of teleportation is the “thrown object teleported”, where instead of pointing at a specific location, the user throws an object (using natural gestures for grabbing and throwing objects in VR as we will discuss in the next section) and then teleport to the location where the object rests.

• Arm swing: this is a semi-natural way to locomote in VR. Users must swing their arms while holding the controllers, and the swinging gesture will translate their perspective in the virtual environment. The general implementation of this mechanic is in such a way that the faster users swing their arms, the fast their viewing perspective moves in the virtual environment. This is a rather useful locomotion mechanic when the users are required to travel a relatively long distance, and you do not want them to miss anything along the way by jumping from point to point.

Arm Swing GIF

Credit: Get Good Gaming [181]

• Grabbing and locomoting: Imagine a rock-climbing experience in VR, where the user must climb a surface. An arm swing gesture is probably not the best locomotion mechanic in this case to translate the perspective of the user along the Y-axis (as the user climbs up). By colliding with and grabbing GameObjects such as rocks however, the user can locomote on the X, Y, or Z axes in a more natural way. This locomotion mechanic is used in many different VR experiences for climbing ladders, using zip-lines, etc.

Rock Climbing GIF

Credit: Ctop [182]

• Dragging: this is a particularly interesting and useful locomotion technique, specifically for situations where the user a top-down (overview) perspective of the virtual environment. Consider a virtual experience where the size of the user is disproportionate to the environment (i.e. they are a giant) and they need to navigate over a large terrain. One way to implement locomotion in such a scenario is to enable users to grab the “world” and drag their perspective along the X or Z axes.

Grabbing GIF

Credit: lionheartx10 [183]

There are many other locomotion mechanics for VR (e.g. mixing teleportation and smooth movement, run in-place locomotion, re-orientation of the world and teleportation together, etc.) that we did not cover in this section. However, the most popular and widely used ones were briefly mentioned.

Interaction in VR

Interactions with GameObjects in non-VR environments are limited to conventional modalities and their affordances. In desktop experiences, for instance, interactions are limited to pointing at objects and graphical user interface element. In certain gaming consoles, however, such as Nintendo Wii [184] and Xbox Kinect, users can perform natural gestures to interact with objects in a game or to perform actions.

Interaction with GameObjects in VR can be considerably more natural compared to desktop experiences. The immersive nature of VR HDMs combined with the possibility of locomotion in a natural point of view affords a much closer interaction experience to real-life compared to any other gaming console or desktop. Thanks to the power of sensors and controller data in VR headsets, we can constantly track the position, orientation, and intensity of hand movements in VR. As such, users can use natural gestures for interaction with different types of objects while perceiving them from a natural viewpoint.

Users can interact with objects by reaching out and grabbing them when they are in their proximity, or they can grab them from distance using a pointer. Once an object is grabbed, users can use the physics properties to place them somewhere in the virtual environment, throw them, and even change their scale and rotation.

Video: Natural Virtual Hand Interaction with Puzzle Cube in VR (1:13)  This video is not narrated.

An example of grabbing an object from distance using pointers:

Point Grab GIF

Credit: ChoroPhronesis [41]

More natural forms of interaction in VR are via gestures. For instance, users can spin a wheel using a circular gesture or pull down a lever using a pull gesture. The popular VR archery game is a prime example of using natural gestures to interact with game objects.

Similar to the bow and arrow, some objects such as a fire extinguisher are used by two hands in real-life. The same principle could be applied in VR, where the user grabs the capsule with one hand and the hose with another. An overview of some of these interaction mechanics is demonstrated in the videos below from one of the most popular VR plugins for Unity called VRTK.

Video: [Basics] Grab attach mechanics (5:57)

Video: Unity VRTK v3 ( VR Development ) Testing Interactables Demo Scene (10:34)  This video is not narrated.

In the videos posted above, you can see that users can naturally interact with different types of objects, by sitting down and pulling out a drawer, flipping switches, etc. The same principle can be applied to graphical user interface (GUI) objects. Users can use natural gestures to collide with different GUI objects such as buttons to interact with them, or they can use pointers to point at a specific GUI element. Users can grab a selector on a slider and move their hand to the left or right to decrease/increase a value, or they can move a text element in the scene.

As the inspiration of different interaction mechanics for your assignment, you can watch the following video:

Video: Top 10 PC VR Games

Tasks and Deliverables

Summary and Tasks Reminder

In this lesson, you experienced what it takes to create a very simple game in Unity. You used some of the more advanced features of Unity such as AI and experienced setting up a semi-complex scene in the editor. Furthermore, you were provided with some slightly complex C# scripts that made the City Builder game come alive and were explained the interrelations among the scripts. In the second part of the lesson, you were introduced to the most common locomotion and interaction mechanics used in VR development. We hope that this lesson has helped to broaden your knowledge on the design and development of virtual experiences in Unity and the more novel interaction and locomotion mechanics employed in VR experiences these days.

Reminder - Complete the task on Lesson 9!

You have reached the end of Lesson 9! Double-check the to-do list on the Lesson 9 Overview page to make sure you have completed the activity listed there.

Overview

In the final lesson of this course, we ask you to reflect on the potential of 3D modeling and VR (or AR/ MR) for the geospatial sciences. We have seen in this course that there is a lot of development and advancement in technology taking place that has made 3D modeling more accessible and is pushing virtual reality into the mainstream.

This is not to say that there are no issues left or that there is not a lot of work to be done on all accounts. But, with a consumer market industry behind current developments we are looking into a future of communication, sharing, and understanding environmental information that will be vastly different from our current approaches.

Learning Outcomes

By the end of this lesson, you should be able to:

• Discuss applications of 3D/VR
• Discuss evaluating the added value of immersive VR
• Discuss how spatial sciences will change in the light of VR

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

Two Current Projects

The course has provided an overview of a number of topics surrounding 3D modeling and analysis and VR. The goal was to give students an overview of the technologies and software products that are available or emerging and how geospatial sciences will change in the near future. Many topics had to be left out, but we would like to show at least two short videos of projects done by our research team.

Historic Campus

The reason to include these projects here is that they provide very natural extensions to the topics we have addressed in the course. The first project is the historic campus modeling effort that you have engaged within lesson 3, using the armory, and in the Unity lessons (7 and 8). This is an ongoing project that is self-funded and lives of contribution largely from students like yourself. The one aspect we would like to focus on here is that the model that we created and that you have explored can be used and refined in numerous ways. Using a different software tool, LumenRT, we created a more realistic rendering of the historic campus. LumenRT can transform 3D models into real-time 3D immersive scenes called LiveCubes. The LiveCubes can be shared with other people and let them navigate in the scene without installing the software. It also automatically adds accurate lighting, shadows, and reflections to the objects in the scene with real-time global illumination. For demonstration purposed please watch the following video. Keep this video in mind when you think of your audience. While many people of the younger generation are perfectly happy with game-like environments, in our experiences people of advanced age prefer a more realistic representation. While this representation is still challenging for interactive immersive VR experiences, they can be used in current technology, for example, in the form of 360-degree videos or images, which can also be shared via YouTube.

Video: Penn State Campus in 1922, new version (2:04) This video is not narrated.

LiDAR Volcano Visualization

The second example we’d like you to explore is from a project in collaboration with geosciences professor Peter La Femina. Pete is a volcanologist and has a repertoire of fascination journeys. Sometimes he is in the position to take students along, but this is not always the case. Virtual field trips have always been a dream of geography but so far they have never fully taken off. We are working on changing the acceptance of virtual field trips by implementing a VR experience that will allow students to explore the insight of a volcano. This is also an example of a workflow as discussed in lesson 2. Pete took a drone equipped with both LiDAR and photo cameras with him to Iceland and visited the volcano Thrinukagigur on Iceland. If you are interested, there is a lot more information on this Inside the Volcano [193], a famous volcano website. The volcano is reasonable dormant such that it is safe to fly, for example, a drone into it (you can actually visit the volcano).

Video: LiDAR Volcano Visualization (00:30) This video is not narrated.

Workflow for the visualization of LiDAR data in VR using drone data collected by Peter La Femina and Unity as a platform for realizing interactivity on different VR systems (e.g., HTC Vive).

Credit: ChoroPhronesis [41]

Reading Assignment

Finally, we would like you to read another article that discusses, from a more theoretical perspective, the role that using the landscape itself (especially the local landscape someone may be used to) as a means of communication. As an example, we have two projects that deal with VR and climate change. Both projects provide an opportunity to think through the options we have to communicate with people about how climate change may change their local environments, as well as what sustainable living can look like. The article is written by Stephen Sheppard and has just been published in Landscape and Urban Planning. You can download the article Making climate change visible: A critical role for landscape professionals [194] using Science Direct, through the Penn State Libraries. You may need to log in using your Penn State Access Account user id and password when you follow the link.

If you need to search for the article, the complete information is:

Journal title and issue: Landscape and Urban Planning 142 (2015) 95–105

Article title: Making climate change visible: A critical role for landscape professionals

Author: Stephen R.J. Sheppard

Deliverables

Summary and Tasks Reminder

Lesson 10 provided a short outlook on how 3D modeling and VR have the potential to change how we communicate environmental change. Discussing Sheppard's article is one example of how researchers start to add theory to the technological developments that are happening. The video of the historic campus project provided a glimpse of how realistic 3D models can become with off-the-shelf software opening new possibilities for communicating with a wide audience. The Thrinukagigur example shows that immersive technologies will have an impact on a wide spectrum of geo-spatial sciences and that we are finally in the position to realize virtual field trips.

Reminder - Complete all of the Lesson 10 tasks!

You have reached the end of Lesson 10! Double-check the to-do list on the Lesson 10 Overview page to make sure you have completed all of the activities listed there. Congratulation! You have finished this course!

Syllabus: Undergrad Course 

GEOG 497: 3D Modeling and Virtual Reality
Spring 2022

It is essential that you read this entire Syllabus document as well as the material covered in the Course Orientation. Together these serve as our course "contract."

Instructor

Instructor: Mahda M. Bagher, PhD

Office: Remote
Department of Geography, College of Earth and Mineral Sciences, The Pennsylvania State University

• E-mail: mmm6749@psu.edu [195]. Please use the course e-mail system (see the Inbox in Canvas).
• Office Hours: Fridays 12:00 PM-2: 00 PM and by appointment via Zoom

NOTE: I will read and respond to e-mail and discussion forums Frequently.

Class Support Services

Penn State Online [196] offers online tutoring to World Campus students in math, writing, and some business classes. Tutoring and guided study groups for residential students are available through Penn State Learning [197].

Course Overview

Description: 3D Modeling and VR for the Geospatial Sciences is an introductory-level science course that introduces students to emerging topics at the interface of concepts and tools that capture/sense environmental information (e.g., LiDAR) and concepts/tools that allow for immersive access to geospatial information (e.g., HTC Vive). The course offers a high-level perspective on the major challenges and opportunities facing the development of current 3D technologies, differences between classic modeling and procedural rule-based modeling, the development of VR technologies, the role of game engines in the geospatial sciences. Topics that will be covered include an introduction to the 3D Modeling and 3D sensing technologies, 360 degree cameras, VR apps and tools, workflows how to integrate environmental information into VR environments, an introduction to procedural rule modeling, hands-on experience in creating 3D models of Penn State Campus, the creation of a virtual Penn State Campus, accessing and exploring a virtual campus in Unity (game engine), and the export of flyovers to platforms such as YouTube 360.

Prerequisites and concurrent courses: None.

Course Objectives

When students successfully complete this course, they will be prepared to:

• Understand and apply the concepts of 3D modeling and VR, they will be in a position to distinguish concepts such as virtual, mixed, and augmented reality.
• Explain and implement workflows to created 3D content from existing and historic and future environments.
• Use a variety of software solutions for 3D model creation such as SketchUp, CityEngine (theoretical), Unity.
• Understand the emerging possibilities of environmentally sensed information.
• Create 3D models and make them accessible in an interactive way through the use of game engines. Evaluate scenarios for the future of food considering resilience in the context of climate change, human population growth and socio-economic, and cultural factors.

Expectations

On average, most students spend 12-15 hours per week working on course assignments. Your workload may be more or less depending on your study habits.

I have worked hard to make this the most effective and convenient educational experience possible. The Internet may still be a novel learning environment for you, but in one sense it is no different from a traditional college class: how much and how well you learn is ultimately up to you. You will succeed if you are diligent about keeping up with the class schedule and if you take advantage of opportunities to communicate with me as well as with your fellow students.

Specific learning objectives for each lesson and project are detailed within each lesson. The class schedule is published under the Calendar tab in Canvas (the course management system used for this course).

Required Course Materials

Required textbooks: None

Recommended textbooks:

There are numerous books relevant to the course content. Resources are added throughout the course allowing students to follow up on specific topics.

Required materials: 

 D-scope Pro Google Cardboard Kit with Straps may be purchased online [198]. 

Online lesson content

All other materials needed for this course are presented online through our course website and in Canvas. In order to access the online materials, you need to have an active Penn State Access Account user ID and password (used to access the online course resources). If you have any questions about obtaining or activating your Penn State Access Account, please contact the World Campus Helpdesk [199].

Assessment: Assignments, Quizzes, and Exams

This course relies on a variety of methods to assess and evaluate student learning, including:

• Practical assignments - Assignments are required for each module that require students to demonstrate their understanding of topics and processes presented in each module.
• Reflective writing assignments - These writing assignments ask students to reflect on various aspects of the topics and technologies presented in the course.
• Participation - Class participation is important. Students will be expected to join the discussions on topics related to the course.

Citation and Reference Style
I personally prefer APA as a citation style but as long as you are complete and consistent you may choose any style.

It is important that your work is submitted in the proper format to the appropriate Canvas Assignment or Discussion Forum and by the designated due date. I strongly advise that you not wait until the last minute to complete these assignments—give yourself time to ask questions, think things over, and chat with others. You will learn more, do better...and be happier!

Due dates for all assignments are posted on the mini-syllabus and course calendar in Canvas.

Grading

I will use the Canvas gradebook to keep track of your grades. You can see your grades in the Grades page of Canvas. Overall course grades will be determined as follows. Percentages refer to the proportion of all possible points earned.

Curve
I am not planning to curve grades.

Late Policy

I do not accept any "late work." In exceptional circumstances, you should contact me. The earlier you contact me to request a late submission, the better. Requests will be considered on a case-by-case basis. Generally, late assignments will be assessed a penalty of at least 25% and will not be accepted more than one week after the original due date.

Technical Requirements

For this course, we recommend the minimum technical requirements outlined on the World Campus Technical Requirements [203] page, including the requirements listed for same-time, synchronous communications. If you need technical assistance at any point during the course, please contact the IT Service Desk [204].

Internet Connection

Access to a reliable Internet connection is required for this course. A problem with your Internet access may not be used as an excuse for late, missing, or incomplete coursework. If you experience problems with your Internet connection while working on this course, it is your responsibility to find an alternative Internet access point, such as a public library or Wi-Fi ® hotspot.

Course Policies

Mahda M. Bagher

Mahda is a doctoral candidate in Geography, specializing in GIS and Immersive technologies. She is part of the team at the Center for Immersive Experiences. Her interest is in facilitating embodied learning through designing immersive learning experiences in immersive VR.  she is specifically interested in research related to memory and embodied cognition in the field of education. Her main focus has been on embodiment in the context of geospatial data visualization in VR. She also works on procedural modeling, rebuilding historical data, and integrating geospatial data in VR, including LiDAR data, DEM, shapefiles, raster data, and any point clouds.  She has received several awards for academic and research achievement in her graduate research, most notably the William and Amy Easterling Outstanding Graduate Research Assistant Award and the Willard Miller Award, 1st place for the Ph.D. proposal. She will be graduating in May 2022.

Tasks and Deliverables

Overview

In this module, you will build a 2D map of the University Park campus including buildings, trees, roads, walkways, and a digital elevation model (DEM). You will learn how to convert 2D maps to 3D presentations. Then procedural symbology that we have created for you in CityEngine will be imported and applied to the 3D map to create a realistic scene.

Learning Outcomes

By the end of this lesson, you should be able to:

• Explain options for 3D modeling within Arc GIS Pro
• Understand ArcGIS Pro's integrated 2D–3D environment (Local and Global Scenes)
• Convert a 2D map to a 3D scene
• Explain 3D symbology
• Extrude features to 3D symbology
• Find an example of a 3D CityEngine/ArcGIS Pro model: campus model
• Compare procedural modeling rules (.cga) vs procedural rule package (*.rpk)
• Create a 3D model of elevation (DEM)
• Create a 3D scene from a 2D building footprints of campus-based on the DEM
• Employ a procedural rule symbology based on the rule package
• Link procedural rule symbology to attributes table
• Distinguish static modeling from procedural modeling
• Restructure the map and represent the 3D scene based on the imported model

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

Tasks and Deliverables

This Lesson tasks have 4 parts.

Part 1: Participate in a Discussion

Please reflect on your 3D modeling experience, using ArcGIS Pro; anything you learned or any problems that you faced while doing your assignment as well as anything that you explored on your own and added to your 3D modeling experience.

Instructions

Please use Lesson 5 Discussion (Reflection) to post your response to reflect on the options provided above and reply to at least two of your peer's contributions.

Please remember that active participation is part of your grade for the course.

Due Dates

Your post is due Saturday to give your peers time to comment. All comments are due by Tuesday at 11:59 p.m.

Tasks and Deliverables

Overview

In this and the next two lessons, you will be working with the Unity3D game engine, simply referred to as Unity throughout the rest of the course. Unity is a rather complex software system for building 3D (as well as 2D) computer games, and other 3D applications. Hence, we will only be able to scratch the surface of all the things that can be done with Unity in this course. Nevertheless, you will learn quite a few useful skills related to constructing 3D and VR applications in Unity dealing with virtual environments. Furthermore, you will become familiarized with C# programming (the main programming language used in this engine) through a series of exercises that guide you on developing your first interactive experience in a game engine.

This lesson will begin with an overview to acquaint you with the most important concepts and the interface of Unity. It will cover basic project management in Unity, the Project Editor interface, manipulation of a 3D scene, importing 3D models and other assets, and building a stand-alone Windows application from a Unity project. Moreover, you will learn how to create a very simple “roll-the-ball” 3D game using C#, which will prepare you for more advanced interaction mechanics we will be implementing in the next lessons.

Learning Outcomes

By the end of this lesson, you should be able to:

• Explain the role of interaction for 3D/VR projects and the role of game engines for enabling access to 3D models
• Create 3D projects in Unity3D, import 3D models, and understand the basic concepts and interface elements of the Unity editor
• Manipulate scenes and add script components to game objects
• Import custom Unity packages into a unity project
• Develop a basic understanding of the physics engine of Unity (e.g. gravity, collision detection)
• Run projects inside Unity and build stand-alone applications from them
• Have a basic understanding of how to make an experience come “alive” by programming behaviours for game objects

Lesson Roadmap

Questions?

If you have any questions, please post them to our "General and Technical Questions" discussion (not e-mail). I will check that discussion forum daily to respond. While you are there, feel free to post your own responses if you, too, are able to help out a classmate.

Tasks and Deliverables

Assignment 1:Unity 3D/VR Application Review

Search the web for a 3D (or VR) application/project that has been built with the help of Unity. The only restriction is that it is not one of the projects linked in the list above and not a computer game. Then write a 1-page review describing the application/project itself and what you could figure out how Unity has been used in the project. Reflect on the reasons why the developers have chosen Unity for their project. Make sure you include a link to the project's web page in your report.

Assignment 2: Create the walkthrough of the Old Main

To practice Unity project management, scene editing, and project building, we want you to apply the steps from this walkthrough: Using Unity to Build a Stand-Alone Windows Application [206]. 

Please set up a new Unity project for this assignment. Then follow the steps from the walkthrough to add the building model to the scene, make sure that the lighting source and camera are placed in a reasonable way relative to the building model, and make the camera keyboard controlled. In the end, build a stand-alone Windows application from your project. 

Deliverable 1

Three screenshots should be taken in Play(!) mode showing the entire Unity editor window. Each screenshot should show the building from a different perspective. The keyboard control of the camera will allow you to navigate the camera around to achieve this.

Deliverable 2

Uploading the zip file of the Windows stand-alone application you built (consisting of both the .exe file and the ..._Data folder) or a link to the uploaded zip file on clouds such as OneDrive or Dropbox.

Assignment 3: “Roll-the-ball” Game

The homework assignment for this lesson is to report on what you have done in this lecture. The assignment has two tasks and deliverables with different submission deadlines, all in week 8, as detailed below. Successful delivery of the two tasks is sufficient to earn the full mark on this assignment. However, for efforts that go "above & beyond" by improving the roll-the-ball game, e.g. create a more complex play area -like a maze- instead of the box, and/or make the main camera follow the movement of the ball, you could be awarded one point if you fall short on any of the other criteria.

To help with the last suggested extra functionality, use the following script and attach it to the main camera:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class FollowBall : MonoBehaviour
{
    // reference to the ball game object
    public GameObject Ball;

    // a vector3 to hold the distance between the camera and the ball
    public Vector3 Distance;

    // Start is called once before the first frame update
    void Start()
    {
        // assign the ball game object
        Ball = GameObject.Find("Ball");

        //calculate the distance between the camera and the ball at the beginning of the game
        Distance = gameObject.transform.position - Ball.transform.position;
    }
    // This called after the Update finishes
    void LateUpdate()
    {
        // after each frame set the camera position, to the position of the ball plus the original distance between the camera and the ball 
// replace the ??? with your solution
        gameObject.transform.position = ???;
    }
}

If you follow the detailed instructions provided in this lecture, you should be able to have a fully working roll-the-ball game.

Deliverable 1: Create a unity package of your game and submit it to the appropriate Lesson 7 Assignment.

You can do this by simply going to the Assets in the top menu and then selecting “Export Package”.

Credit: Unity [133]

Make sure all the folders and files of your project are checked and included in the package you export (you can test this and import your package in a new Unity project to see if it runs).

Credit: Unity [133]

If you choose to implement the functionality of the main camera to follow the ball movement, please explain your solution as a comment in your script. You can add comments (texts in green) in C# by using “//”.

Helpful advice:

We highly recommend that, while working on the different tasks for this assignment, you frequently create backup copies of your Unity project folder so that you can go back to a previous version of your project if something should go wrong.

Grading Rubric

Criteria	Full Credit	Half Credit	No Credit	Possible Points
Your game runs smoothly and as intended without any errors or strange behavior	6 pts	3 pts	0 pt	6 pts
Your project is complete and has all the required functionalities (coin rotation, collection, and sound effect).	3 pts	1.5 pts	0 pt	3 pts
Your Unity package has a proper folder structure, and all the files have meaningful names like it was done in the instructions	1 pt	.5 pts	0 pts	1 pt
				Total Points: 10