1.2 VR Systems

Introduction

It is impossible to write a complete list of all the [A,M,V]R systems and content providers that are currently throwing their products on the market. Essentially, all major tech company and numerous startup companies are venturing into the field of VR. We are providing a short overview of some of the main systems that are available or are filling niche products. The image below provides a visual overview and a little farther down the line we have a quick memory game for you.

Overview of VR technologies

Credit: Courtesy of Danielle Oprean

Oculus Rift

It is fair to say that the current hype around VR has been dramatically spurred by Oculus and the fact that a little startup company after only two years got purchased by Facebook for two billion dollars (see Mark Zuckerberg's announcement [3]) [4].

Oculus started with a then teenager, Palmer Luckey, who was building VR prototypes in his garage (are not all stories like this starting in garages)? Luckey sent one of his prototypes, the PR6 that he called the Rift, to famous game designer John Carmack (Doom, Quake, founder of id Software). Carmack a little later would demo the Rift at an E3 event where it created a lot of traction. Luckey teamed up with Brendan Iribe, Michael Antonov, and Nate Mitchell, and together they launched a Kickstarter Campaign, that raised not the anticipated $250K but over 2 million dollars (for more details see "How Palmer Luckey Created Oculus Rift [5]".

From there on it was a wild ride. The biggest change for the young company came when Facebook in 2014 bought it for two billion dollars. Had VR, even up until March 2014 created some hype and fantasies, at this point it had everyone’s attention.

After two developer versions, the consumer version started shipping in the Spring of 2016! Below is a figure that shows the transition from DK1 to the consumer version. The latter offers an OLED display, a resolution of 2160 x 1200, a 90 Hz refresh rate, a 110-degree field of view, and a tracking field of up to 5 x 11 feet. The Oculus Touch controllers have not yet been released. If you want to run an Oculus on your computer the recommended minimum specs are an NVIDIA GTX 970 / AMD 290 equivalent or greater, Intel i5-4590 equivalent or greater, 8GB+ RAM, Compatible HDMI 1.3 video output, 2x USB 3.0 ports, Windows 7 SP1 or newer.

From top to bottom, the three official releases for the Oculus Rift: development kit (top), SDK2 (middle), consumer version (bottom)

Credit: Wikipedia Commons & Oculus Press Kit

The latest versions of Oculus Rift can be used with either a remote controller or a more complicated
input modality such as Oculus Touch.

Oculus remote

Credit: Oculus [6]

Oculus Touch Controllers

Credit: Oculus [6]

While the remote is rather limited as an interaction modality (only a few buttons), Oculus Touch provides a wide variety of interaction possibilities (e.g., finger tracking, gesture recognition) and a larger set of buttons on each controller.

Moreover, by default two Oculus sensors are used to track and translate the movements of users (head movements based on the VR headset and hand movements based on the Touch controllers) in VR. This technology provides a 6-DOF (degrees of freedom) (3-axis rotational tracking + 3-axis positional tracking) tracking.

Oculus Sensor

Credit: Oculus [6]

On March 20th, 2019 Oculus announced a new model to the Rift family called Oculus Rift S. The new member of the Rift family uses a Fast-switch LCD display with a resolution of 2560×1440 (1280×1440 per eye) with a refresh rate of 80 Hz. It utilizes a new technology called “inside-out tracking” using five cameras embedded into the headset, which eliminates the need for external sensors for tracking while maintaining a 6-DOF tracking. In short, the headset itself can track and translate the movements of itself and the controllers, which allows the users to freely move around a physical environment without restrictions. If you want to use an Oculus Rift S on your computer the recommended specs are an NVIDIA GTX 1060 / AMD Radeon RX 480 or greater graphics card, Intel i5-4590 / AMD Ryzen 5 1500X or greater, 8GB+ RAM, DisplayPortTM 1.2 / Mini DisplayPort, 1x USB 3.0 ports, and Windows 10.

Oculus Rift S

Credit: Oculus [6]

Since the emergence of cost-effective VR HMDs developers and users have always been restricted to limited mobility options due to the requirements for these HMDs to be connected to powerful computers via cables (i.e. tethered).

On October 11th, 2017 a new generation of Oculus HDMs called “Oculus Go” was announced and later on May 1st, 2018 released to the public. Go is described as an untethered all-in-one headset meaning that it can function as an independent device without the need to be connected to a computer. Evidently, the first generation of these untethered all-in-one headsets was not as powerful as their predecessors in terms of computing power. Oculus Go utilizes a Snapdragon 821 SoC processor and is powered by a 2600 mAh battery, 5.5" fast-switching LCD with a resolution of 2560×1440 (1280×1440 per eye) and refresh rate of 72 Hz, Adreno 530 internal graphics card, with non-positional 3-DOF tracking due to the lack of sensor use. Unlike the Rift, Go does not use rich controllers such as “Touch” but instead, uses a simple remote controller with limits interaction possibilities. On December 4th of 2020, Oculus discontinued the Go HMD and officially replaced it with the next generation, Quest.

Oculus Go

Credit: Oculus [6]

On May 21th, 2019, Oculus released a new breed of untethered all-in-one headsets called Oculus Quest. This HMD brings the best features of Rift and Go together in one. Similar to Go, it is an all-in-one standalone headset, and similar to the Rift it uses inside-out-tracking with 6-DOF. Furthermore, it utilizes a more powerful GPU and processor (4 Kryo 280 Gold processor with 4GB RAM, and an Adreno 540 graphics card), capable of running high-quality applications. It has a PenTile OLED display with a resolution of 1440 × 1600 per eye, and a refresh rate of 72 Hz. Furthermore, unlike the other untethered Oculus product Go, Quest uses the latest version of touch controllers.

Oculus Quest

Credit: Oculus [6]

On October 13th, 2020, Oculus released the new generation of Quest, called Quest 2. This new generation of the Quest HMD has superior specifications in the processor (Qualcomm Snapdragon XR2), RAM (6 GB), and display resolution (1832 x 1920 per eye) and starts at a very competitive price of $299.

Oculus Quest 2

Credit: Oculus [6]

HTC Vive

The HTC Vive

Credit: HTC Press Kit

The HTC Vive is a collaboration project of software company Valve and tech giant HTC. Not going into any details of how they came together, the Vive is a truly awesome product that offers substantial flexibility and sophisticated interaction opportunities. The main difference to the Rift is that it does not require users to remain in front of a computer screen (or a fixed location). The Vive comes with room sensors that allow users in small spaces to actively move around (you saw the intro VR video [7]), which is awesome. Additionally, the controllers allow for tracking locations of hands, provide advanced interactivity, and provide haptic feedback in the form of vibrations.

It is always very difficult to explain experiences one has in a VR environment like the Vive to someone via text … one of the best solutions without actually experiencing it oneself is this video filmed using a green background (see intro video [8] again).

The Vive also has a 2160 x 1200 resolution, offers a 90 Hz refresh rate, and a 110-degree field of view. In contrast to the Rift, the tracking field is 15 x 15 feet. The minimum computing requirements are similar to the Rift: NVIDIA GeForce GTX 970 /Radeon R9 280 equivalent or greater, Intel Core i5-4590 equivalent or greater, 4GB+ of RAM, Compatible HDMI 1.3 video output, 1x USB 2.0 port.

On January 8th, 2018, HTC released an updated version of Vive called Vive Pro. The new HMD has a higher resolution at 1440 x 1600 resolution per eye and a refresh rate of 90 Hz. Thanks to its sophisticated tracking sensors (called lighthouses) Vive provides up to 22’11” X 22’11” room-scale tracking of headset and controllers. Vive Pro is also equipped with two cameras embedded into the headset, facing outward. These cameras can be used to bring the real world into VR (Sounds similar to a type of xR previously discussed?). Moreover, Vive affords multi-user interaction in the same physical space. Notwithstanding, each user requires their own dedicated computer and play-area.

VIVE Pro

Credit: VIVE Enterprise [9]

In November 2018, VIVE released their first untethered all-in-one headsets called Vive Focus. The focus utilizes inside-out-tracking technology and affords 6-DOF tracking. It has a 3K AMOLED display, with a resolution of 2880 x 1600 and a refresh rate of 75 Hz, and a Qualcomm Snapdragon™ 835 processor and two controllers.

VIVE Focus

Credit: VIVE Enterprise [9]

In addition to awesome and popular HMDs, VIVE also provides a variety of external sensors and controllers that can be used to create a much richer and close-to-reality VR experiences. These include trackers that can be attached to arms and legs to provide a feeling of embodiment of users in VR (tracking and projecting the hand and leg movements of users in VR), controllers that look like weapons, and VR gloves that cater for hand and finger actions in VR while providing haptic feedback to users.

VIVE external sensors and controllers

Credit: VIVE Enterprise [9]

Other HMDs

There are numerous other HMD brands such as Samsung GearVR, Nintendo Lab VR kit, Sony PlayStation VR, Lenovo Mirage, etc. with different specifications. Going into details about each and every one of these different brands would be out of the scope of this lecture. Notwithstanding, it would be of great interest to become familiar with some of the HMDs are go beyond the conventional specifications.

Although not new, Samsung’s GearVR is of the most affordable (but limited) HMDs. Instead of requiring the user to purchase his or her own computer, it works with various Samsung Galaxy phones. You can think of it as a fancier version of the Google Cardboard viewer. While the Cardboard viewer essentially works with every smartphone, the GearVR requires a Samsung phone although in its newest release the GearVR accepts a range of Samsung phones. While the Cardboard viewer is somewhat limited in its interactive capabilities (gaze and the little magnet button on the side), the GearVR has a back button and a little trackpad on the side. Computing power and display characteristics will depend on your phone.

Samsung Gear VR

Credit: Samsung [10]

Google Cardboard

Credit: Google VR [11]

While most VR HDMs have a field of view (FOV) of around 110 degrees, Pimax is an HMD with a FOV of 170-200 degrees (close to human vision). It has an 8K resolution (3840 x 2160 per eye) with a refresh rate of 80 Hz and is tethered. In addition to being compatible with a variety of controllers (e.g. HTC Vive controllers), Pimax has a special Leap Motion [12] module embedded in the HMD. This module provides real-time and accurate hand tracking and can enable users to solely use their hands as the interaction modality. Furthermore, as another addition to its modular system, Pimax utilizes an eye-tracking module that can provide real-time tracking of users’ eye movements while interacting with a VR experience. Such information will be useful for an array of research topics, including usability studies, user-behavior analysis, and learning content adaptation. The high FOV provided by Pimax enable users to perceive the virtual environment in a way much closer to real life. This can potentially set the basis for more effective training of visual-spatial skills in VR and a more successful transfer of these skills to real-world contexts.

Pimax

Credit: Pimax VR [13]

As is evident from the different HMDs we have seen so far, the resolution of these devices has improved significantly over the past few years. Notwithstanding, they are still not comparable to high-end desktop experiences in that respect. As one of the main applications of VR HMDs is training via realistic simulations, display quality and high-resolution imagery can play a determining role in the successful realization of learning goals in these applications. For this exact reason, the Varjo company has released a VR HMD called Varjo VR-1. Standing at almost 6000$, it is one of the most expensive HMDs in the market, but it possesses unique characteristics. It has a Bionic display – human eye-like resolution of 60 Pxl per degree and combines a 1920x1080 low-persistence micro-OLED and a 1440x1600 low-persistence AMOLED. It is important to note that this HDM is not targeted at average consumers, but rather for AEC (architecture, engineering, and construction), industrial design, virtual production, healthcare, and training and simulation. Given the fact that resolution is one of the drawbacks of VR which impedes its proper utilization in simulations and application where detailed graphical elements are important (e.g. creation of artwork in VR, reading long texts in VR, surgery simulation, interacting with a cockpit panel with lots of buttons and small labels, etc.), this HDM opens the door for interesting research opportunities. Furthermore, it has a built-in eye-tracking module that provides researchers with a great opportunity to monitor the behavior of users when interacting with a VR experience for the purposes of usability studies, measurements of attention/interest, user behavior studies, and content adaptation.

Varjo VR-1

Credit: Varjo [14]

The bottom line is that there is a tremendous amount of development on the VR market that is spurring developments in industry and academia. We just gave you a little glimpse of some of the most prominent examples but there are many, many more. To further acquaint you with some of the technologies we have two tasks outlined on the next page.

Valve Index

Credit: Valve Software [15]

On June 28th, 2019, Valve released the first of their second-generation HMDs called Index. Valve Index has a display resolution of 1440 x 1600 per eye and a refresh rate of 120 Hz. Valve index is backward compatible with all HTC products including the Vive HMDs and the lighthouses (base stations used for tracking). It comes with a new generation of the lighthouse sensors (base station 2.0) that offer improved tracking range, better field of view, and potentially a 300 percent larger tracking space. Perhaps one of the most delightful features of the Valve Index is the novel controllers (called Knuckles) that are significantly more ergonomic compared to other controllers on the market.

This is an ungraded assignment to test your knowledge.

1.5 Important VR Concepts

Introduction

Complementing developments in 3D modeling it is fair to say that we are entering the age of immersive technologies. There are, unsurprisingly, many concepts that characterize VR systems. Scientist think about them to describe systems and interactions on the basis of specific characteristics rather than on the basis of brands. One could ask is the Rift superior to the Vive, or, alternatively one could ask whether it makes a difference that people have more than twice as much space to move around, the Vive allows people to turn a room into a VR experience while the Rift is spatially more restrictive (although this may change). Other such characteristics that potentially make a difference are the field of view, the resolution of the display, or the level of realism of the 3D models.

We will discuss here three concepts, immersion, presence, and interaction that have received considerable attention in the study of VR.

Immersion, Presence, and Interaction

We have already discussed that the renewed interest in virtual technology, the fidelity, and overall quality of information presented in virtual environments are vastly improving. These improvements are not only propelling VR technologies into living rooms and research facilities, but they also have the potential to impact on spatial experiences that users have with virtual environments from the inside of a volcano to the Mayan ruins to informal settlements in Rio. In several studies, it has been shown that these spatial experiences may have positive influences on knowledge development and transfer to a real-world application (Choi & Hannafin, 1995). In other words, by enhancing one’s intensity of the experience, it may offer a way to facilitate learning through understanding and engagement. The literature on spatial experiences within virtual environments, however, is full of ambiguous concepts centered around the relationship between presence, immersion, and interactivity. In bringing conceptual clarity to distinguish these concepts, knowledge derived from previous research can become easier to implement and evaluate. This knowledge can then help to clarify the importance of spatial experiences for systems that enable experiences that only virtual environments can offer. We start the discussion by distinguishing between presence and immersion, defining dimensions of presence, and examining knowledge built from spatial experiences.

How difficult it is to characterize these concepts should become apparent when you try to define, for example, immersion, yourself. The problem with most scientific concepts is the term we use to describe a concept often also has meaning in everyday language (mathematicians get out of this mess by using symbols, e.g., π, and then define what it means). When we think of immersion we have a couple of options defining its meaning. When my kids watch their 20 minute TV show every other evening (Masha and the Bear for the 2-year-old, Word Girl for the 5-year-old), I experience the power of immersion first hand. I would describe their state as being glued to the TV and if unresponsiveness is a measure for immersion than I could make an argument that we really do not need VR. Everyone probably has made similar experiences reading a book whether It or Harry Potter that has taken you under their spell. In becoming engaged with the media, you start to feel a part of the story, feeling present in the space.

Yet, from a technology perspective immersion can be defined differently. For example, in the work by Slater (1999), immersion is distinguished from presence with the former indicating a physical characteristic of the medium itself for the different senses involved. Furthermore, according to Witmer and Singer (1998) immersion is defined as “a psychological state characterized by perceiving oneself to be enveloped by, included in, and interacting with an environment that provides a continuous stream of stimuli and experiences” (ibid, page 227). Witmer and Singer have also argued that immersion is a prerequisite for experiencing presence.

To give you a simple example of immersion, the figure below shows three setups that we used in a recent study on how different levels of immersion influence the feeling of being present (more below) in a remote meeting. We compared a standard desktop setting (the lowest level of immersion) with a three-monitor set up (medium level of immersion) against using an immersive headset (here the Oculus Rift). You can order these technologies as we have just done along a spectrum of immersiveness which then helps to design, for example, experiments to test whether or not being physically immersed has an effect, for example, on the subjective feeling of team membership. Imagine that instead, you are using Skype (or zoom, or any other video conferencing tool) but instead of staring at your single monitor, the other person is streaming a 360-degree video directly into your VR HMD.

Different levels of immersion.

Credit: ChoroPhronesis

In contrast, the nature of presence in and of itself is highly subjective, making it hard to distinguish from concepts like immersion (well, we just did). The sense of presence is indeed the subjective sense of being in the virtual environment (Bulu, 2012). Many smart people have thought about this and to make things more fun have developed further distinctions such as spatial presence, co-presence, and telepresence. Sense of presence seems to be, interestingly enough, related to immersion and several studies have shown that immersive capabilities of technology, specifically VR technology, impact the subjective feeling of presence (Balakrishnan & Sundar, 2011). Therefore, different configurations of technology to increase immersive capabilities should increase the sense of presence experienced by a user (well, it is a plausible hypothesis).

The relationship between immersion and presence forms at the point when user attention is focused (or absorbed) enough to allow for involvement in the content to become engaged with the digital environment (Oprean, 2014). This relationship allows for a clear distinction between immersion and presence while enabling a framework for measuring the influence of one on the other. On similar grounds, and as was previously mentioned, Witmer and Singer (1988) argue that both immersion and involvement are prerequisites for experiencing presence in a virtual environment. Involvement from their perspective is defined as “a psychological state experienced as a consequence of focusing one's energy and attention on a coherent set of stimuli or meaningfully related activities and events” (ibid, page 227), and also related to other concepts similar to engagement such as the “flow state” introduced by Csikszentmihalyi (1992).

Presence is a large and heavily measured construct in research on spatial experiences. The construct of presence consists of a number of dimensions specific to different contextual purposes: spatial, co-, tele-, social, etc. Each dimension of presence is considered with the same foundational meaning: that a user feels ‘present’ within a given medium, perceiving the virtual content (Steuer, 1992). The different dimensions, however, help to further distinguish the context or situation in which presence occurs.

The dimensions of presence cover a range of contextually different concepts that are often used interchangeably (Zhao, 2003). Telepresence distinguishes itself through its original context, teleoperation work (Sheridan, 1992). Steuer (1992) defines telepresence as distinguishable from presence by suggesting it refers to the experience of a secondary, such as virtual, environment through a communication medium. Taking both of these distinctions into account, telepresence, as related to its context of teleoperation, suggests that perceptions relate to experiencing a virtual environment in which work (i.e., interaction) can occur through a communication medium. This definition relates to the development occurring in collaboration research with telepresence systems; systems meant to mediate a sense of presence to enable work on tasks, individually or collectively.

Co-presence is similar, yet a different dimension of presence, than telepresence. Nowak and Biocca (2003) note the distinctions between telepresence and co-presence as pertaining to connection with others. Whereas telepresence can occur without other individual’s involvement, co-presence is dependent on another human being present to connect with through a communication medium. This human connection is what distinguishes co-presence from telepresence, a focus on a human-human relation (Zhao, 2003).

Social presence, another dimension of presence, has a similar definition to co-presence but requires a group of humans to connect with (Nowak & Biocca, 2003). Gunawardena and Zittle (1997) further the definition of social presence stating it is the degree of perceived realness of a person in a mediated communication. This perception of realness is dependent on two concepts: intimacy and immediacy (Gunawardena & Zittle, 1997).

Spatial presence builds on the foundational definition of presence, the feeling of being in an environment. Steuer (1992) notes that with any mediated communication there is overlap with the physical (real world) presence. Spatial presence there relates to the believability of being present within the mediated space, making spatial presence purely experiential.

In distinguishing the dimensions of presence, it becomes easier to establish factors which impact the level at which a user’s perception of being a part of a mediate space. In particular, such a distinction is necessary to understand the true implications of telepresence systems and how the initial sense of being a part of a mediated space impacts teamwork.

Interaction, in contrast, seems to be somewhat more straight forward and less troublesome to define than the concepts of immersion and presence. However, like with most things, once you dig deeper into it you will find that it is not so straight forward after all to make distinctions, especially when more than one modality is involved. Interaction can be broken down into dimensions such as navigation (whether an environment is navigable or not), does the interface allow for interaction and which modality is involved (gaze used often in cardboard environments versus haptic gloves), and human agency, which could relate to navigability but it also could be a measure of how much a user is able to interact with and/or manipulate a particular environment.

Instead of trying to further define these concepts we ask you to have a look at the Figure below. Most concepts we discussed can be arranged on a spectrum from high to low (especially immersion and interactivity). This distinction is useful when we think about different VR technologies.

Interactivity and immersion along a dimension from low to high with example technologies.

Credit: Danielle Oprean

To summarize, this section provided a glimpse into the more academic discussion of how to conceptualize virtual reality. We briefly discussed important concepts such as immersion, presence, and interactivity and showed exemplarily how everyday use of the terms may interfere with their more academic definitions (although even academics get lots in these discussions). Distinguishing VR technologies along the dimensions of these concepts, for example, from high to low levels of immersion, is useful to a) talk about categories of technology rather than individual products; b) to start investigations of which technology might actually help in communicating or making decisions about space and place in the past, present, and future.

References

Balakrishnan, B., & Sundar, S. (2011). Where am I? How can I get there? Impact of navigability and narrative transportation on spatial presence. Human-Computer Interaction, 26(3), 161-204.

Bulu, S. T. (2012). Place presence, social presence, co-presence, and satisfaction in virtual worlds. Computers & Education, vol. 58, no. 1, pp. 154–161.

Csikszentmihalyi, M. & Csikszentmihalyi, I. S. (1992). Optimal experience: Psychological studies of flow in consciousness, Cambridge university press.

Choi, J. I., & Hannafin, M. (1995). Situated cognition and learning environments: Roles, structures, and implications for design. Educational technology research and development, 43(2), 53-69.

Gunawardena, C. N., & Zittle, F. J. (1997). Social presence as a predictor of satisfaction within a computer‐mediated conferencing environment. American journal of distance education, 11(3), 8-26.

Nowak, K. L., & Biocca, F. (2003). The effect of the agency and anthropomorphism on users' sense of telepresence, copresence, and social presence in virtual environments.Presence: Teleoperators & Virtual Environments, 12(5), 481-494. doi:10.1162/105474603322761289.

Oprean, D. (2014). Understanding the immersive experience: Examining the influence of visual immersiveness and interactivity on spatial experiences and understanding. Unpublished doctoral dissertation, University of Missouri.

Sheridan, T. B. (1992). Musings on telepresence and virtual presence.Presence: Teleoperators & Virtual Environments, 1(1), 120-126.

Slater, M. (1999). Measuring presence: A response to the Witmer and Singer presence questionnaire. Presence: Teleoperators and Virtual Environments,8(5), 560-565.

Steuer, J. (1992). Defining virtual reality: Dimensions determining telepresence. Journal of communication, 42(4), 73-93.

Witmer, B. G. & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire, Presence 7 (3), 225–240.

Zhao, S. (2003). Toward a taxonomy of copresence. Presence: Teleoperators and Virtual Environments, 12(5), 445-455.