Immersion – a working paper

status: work in progress 1/17/19

What’s immersion? Who’s using the term in which contexts? Has there been any kind of onthological drift over the years?

Initial Literature Research

Morton Heilig’s Sensorama

Morton Heilig

Ivan Sutherland’s Ultimate Display

Ivan Sutherland: The Ultimate Display, 1965:

the display […] should serve as many senses as possible. […] So far as I know, no one seriously proposes computer displays of smell, or taste. Obviously, Sutherland is not aware of the Sensorama. Excellent audio displays exist […] kinesthetic display […] "joysticks" with force feedback […] The computer can easily sense the positions of almost any of our body muscles. […] Machines to sense and interpret eye motion […]

Ivan Sutherland et al.: The Sword of Damocles – Head Mounted Display 1966:

Ivan Sutherland, Bob Sproul: Virtual Reality Before it Had That Name, Computer History Museum 1996:

…immerse an individual into an synthetic environment… (0:10')

…the project was an attention focusser (0:50')Ben Shneiderman: Choose a good driving problem

1990s

Carrie Heeter: Being There: The Subjective Experience of Presence, 1992:

Three dimensions of presence:

Victor Lombardi: Virtual Reality vs. Immersive Environments: Choosing the Right Computer Interface for the Future, 1994:

Virtual Reality describes, for some, the ultimate interface between human and computer. Using Virtual Reality, or VR, the user's senses are fed perceptual material exclusively from the computer. This feedback from the computer comprises a grand array of sensory information compared to traditional GUIs. The intention is that enough realistic information will be presented to the user so that another set of surroundings will be perceived, one that is virtually real. […]

Immersive Environments. Another view on the future of the computer interface says that the machinery should not be strapped to the person's body, but instead should surround the user naturally, as in a room of one's house. These interfaces are referred to as "immersive environments."

2000s

Slater: How Colorful Was Your Day? Why Questionnaires Cannot Assess Presence in Virtual Environments, 2004:

According to Lombard and Ditton (1997):

presence as a perceptual illusion of nonmediation

According to Draper, Kaber and Usher (1998):

experiential presence [is] a mental state in which a user feels physically present within the computer- mediated environment.

According to Zahoric & Jenison, 1998; Flach & Holden, 1998

A fundamentally different view […] is presence is “. . . tantamount to successfully supported action in the environment.” It is argued that reality is formed through action, rather than through mental filters and that “. . . the reality of experience is defined relative to functionality, rather than to appearances.” What is im- portant in this approach is action (how things are done) and the affordances offered in the virtual environment, rather than just appearances, and that the sense of “being there” is grounded on the ability to “do” there. […]

On Immersion

A distinction is made between immersion and presence by many researchers (e.g., Slater & Wilbur, 1997; Draper et al., 1998). Immersion is a description of overall fidelity in relation to physical reality provided by the display and interaction systems. […]

On presence:

A fundamental problem with the vast majority of presence research is that there is no real evidence at all for the existence of the phenomenon. As researchers in the field we “know” that it exists through introspection of our own states and behaviors while experiencing immersive virtual environments.

2010s

James J. Cummings, Jeremy N. Bailenson and Mailyn J. Fidler: How Immersive is Enough? A Foundation for a Meta-analysis of the Effect of Immersive Technology on Measured Presence, 2012:

The concept of presence, or a “sense of being there,” is a frequently emphasized factor when discussing virtual environments (VEs). Indeed, the assumption that achieving presence should be a goal of the design of VEs pervades both applied and academic work with such environments. An increased sense of presence is often thought to magnify user effects (e.g., the extent to which user responses to virtual stimuli and virtual interactions resemble parallel responses to “real world” counterparts) and, in turn, to increase the effectiveness of VE applications Users, goals and context of use need to be defined in order to measure effectiveness as part of usability of the VE application. (e.g., the practical use of such environments as tools for learning, training, or therapy).

[…] Slater and Wilbur (1997) distinguish it from another related concept – immersion. Slater and Wilbur suggest that presence in a VE is inherently a quality of the user’s psychologypresence as an aspect of the user experience (ISO), representing the extent to which an individual experiences the virtual setting as the one in which they are consciously present. On the other hand, immersion can be regarded as a quality of the system’s technologyimmersion as a technical dimension, an objective measure of the extent to which the system presents a vivid virtual environment while shutting out physical reality. By this accountimmersion ⇒ presence, the technological level of immersion afforded by the VE system facilitates the level of psychological presence. I doubt this. As long as the illusion is not convincing enough the mind is looking for hints to distinguish the real world from the virtual world. Furthermore, the better the provided quality of a VR envirnoment, the less imagination is needed to complete the ficticious world. cf. the concept of closure in film making and comics; lean-back vs. lean forward; McLuhans’s cold vs. hot medium. This relationship has implications, then, for how one might operationally design for increased presence.

Slater and Wilbur note that a system is more likely to be immersive – or to shut out physical reality – if it

  1. offers high fidelity simulations through multiple sensory modalities,
  2. finely maps a user’s virtual bodily actions to their physical body’s counterparts, and
  3. removes the participant from the external world through self-contained plots and narratives.

Such features are thought to make the interface of the system more transparent, permitting the user to then become psychologically engaged in the virtual task at hand rather than attending to the input mechanisms themselves. That is, the more immersive the system, the more likely an individual will feel present within the virtual environment, and the more likely that the virtual setting will dominate over physical reality in determining user responses. […]

How immersive is enough? […]

measure of presence (including “presence”, “telepresence”, “spatial presence”, or “social presence”) […]

a number of researchers have suggested the effect is composed of two component causalities:

  1. increases in immersion components leading to greater presence and
  2. increases in presence in turn leading to improvements in performance. […]performance ≟ efficiency

The common, modal independent variables from these studies permit us to examine the relative effect of a number of common immersive technology features. […] These features include:

Jose Luis Rubio-Tamayo, Manuel Gertrudix Barrio, and Francisco García García: Immersive Environments and Virtual Reality: Systematic Review and Advances in Communication, Interaction and Simulation, 2017:

Immersive Environments: Conceptual Approach: Immersive environments are defined in different approaches by authors from the 90s, in parallel to virtual reality and augmented reality. Slater and Wilbur [22] propose a framework to define presence and other factors in virtual environments. In the early 2000s, Conroy [23] tries to describe them by constructing experimental immersive worlds based on data coming from the physical world, and establishing an analogy between both. Kalawsky [24] also compares presence in virtual environments with presence in the physical world by adding the main sensory inputs “such as vision, sound, proprioception, and smell”. This idea, where different senses are used through devices to make users interact with information is common in VR and immersive environments, and then, for designing interactions and conceptualising and classifying the different components of the environment composed by artificial information. And, even though VR and immersive environments are not the same concept, a framework to define presence and describe interaction and immersion should keep similar components and features in mind.

Wikipedia: Immersion (virtual reality):

Immersion into virtual reality is a perception Here immersion is used as a psychological term, hence part of the ISO-UX domain. of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment. […]

According to Ernest W. Adams, author and consultant on game design,[1] immersion can be separated into three main categories:

Staffan Björk and Jussi Holopainen, in Patterns In Game Design,[2] divide immersion into similar categories, but call them sensory-motoric immersion, cognitive immersion and emotional immersion, respectively. In addition to these, they add a new category:

Theatre and Philosophy

Matthias Hartmann, 2012:

Die Bühne ist ein Ort, wo mit dem Publikum in Interaktion getreten wird. Deshalb spielt uns die Medienzeit ja ganz stark in die Hände. Menschen haben starke Sehnsucht danach, ernst genommen zu werden und in eine Form von Kommunikation zu treten. Je mehr sie durch illusionistische Medien unmündig gemacht werden, desto stärker wollen sie in ihrer FantasieMan darf und sollte dem Publikum ruhig zutrauen, dass es mit seiner Fantasie das auf der Bühne Dargestellte komplettiert. Das führt sogar zu einer noch größeren Identifikation und Aufmerksamkeit, als ein 4K-360-3D-Filmereignis einzeln angesprochen werden. Ich erlebe es gerade ganz stark, dass immer mehr Leute in das Theater gehen. Die Menschen suchen einen Ort, an dem sie lebendig Kontakt aufnehmen mit einem Vorgang. Das ist ein wesentliches Ritual, das das Kollektiv nur noch im Theater findet.

Jahrbuch immersiver Medien 2012 - Einleitung:

Dabei ist zubetonen, dass kein Medium – nicht einmal die immersiven Medien – immersive Prozesse induzieren muss und nicht zwingend bei den Rezipierenden zum Präsenzerleben führt, da auch mit immersiven Medien distanziert bzw. reflektiert umgegangen werden kann – schließlich sind immer semiotische Prozesse im Spiel. Zudem existieren kaum total immersive und illusionistische Medien, die vorrangig die sinnliche Wahrnehmung adressieren, da sie «technisch enorm aufwendig und in vielen Punkten bis heute schlicht unmöglich sind» (Glaubitz & Schröter 2009: 284) und aufgrund der produktiven Rolle der Imagination nicht notwendig zu sein scheinen.D.h, dass auch low-fidelity VR-Systeme zu überzeugenden Ergebnissen führen können; mit den hier verwendeten Begriffen: trotz niedriger Immersion kann es dank der produktiven Rolle der Imagination zu Päsenzerleben kommen.

Über die skizzierten Begrifspaare Realismus / Abstraktion und Perzeption / Imagination können nun diejenigen medialen Strategien extrahiert werden, durch die sich der (metaphorische) Weg in das Medium hinein öffnet. Ziel dieser Strategien ist die Entgrenzung des Dargestellten bzw. des Mediums:

Diese hier lediglich skizzierten immersiven Strategien sollten es in zukünftigen Ausführungen zu diesem Themenkomplex erlauben, das Konzept der Immersion zu segmentieren und so die Grundzüge der Strukturierung medialer Erlebnisräume präziser zu bestimmen. […] Die Rolle der Imagination scheint dabei – wie bereits angedeutet – ein Schlüssel zum tieferen Verständnis dieses Phänomens zu sein.

A Chat with Erik Champion

9-Dec-2017, edited version

Matthias:

I suspect that different people have a different understanding of the term immersion, which makes it a little bit tricky to use the term at all.

Some use it as a technical term that means the means of VR. That’s too narrow from my point of view. Or to phrase it differently, I would like to use immersion for an important concept and not just for measuring the amount of VR gear.

Presence is offered in return to talk about the dimension of… what…? total attention. For Mihaly Csikszentmihalyi it is flow – and there the visual high-fideltiy of the medium does not matter at all. It is a dimension of attention. I can be fully focussed on a narrative – in cinema / in a theatre / while reading a book – and never be confused about myself sitting on a chair…! I do not need to feel "presence" to be fully immersed in a story… IMO.

Erik:

It is old but Schuemie wrote a great survey on it. Witmer and Singer have a totally different concept to Slater and Garau but this paper by Slater [2004, PDF] may be similar to your concern.

We use different notions of immersion interchangeable, actually I think Frank Biocca wrote about immersion in VR and immersion in books about 15 years ago and Carrie Heeter did as well.

Heeter differentiated presence in beingthere [1992] and Biocca recently in mitpressjournals.org/doi/abs/10.1162/PRES_a_00255

[immersion as a technical term that means the means of VR] is not a commonly held idea of Presence in the Presence community

Matthias:

I can be fully focussed on a narrative (in cinema / in a theatre / while reading a book) and never be confused about myself sitting on a chair…! I do not need to feel "presence" to be fully immersed in a story… IMO."

Erik:

2 issues, the book is present to you; but yes, relationship between presence and immersivity is complex even if there does seem to be a relationship, is it subjective or inter-subjective?

Mel Slater: Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire

Matthias:

I do not want to disregard low-fidelity UI/VR/AR in order to engage the user by inviting him to use her imagination to complete the scene. I don’t have to trick them to make them feel to be on a dangerous powerplant for instance.

Erik:

And treatment of phobias in low res VR shows the more technology is not always necessary.

Narcis Parés and Roc Parés, 2006: Towards a Model for a Virtual Reality Experience: The Virtual Subjectiveness

Matthias:

The user interface should be usable, clear, learnable, straight forward, predictive, etc. Then the user gets used to it, builds trust and can eventually forget the tool and focus on the content/problems to solve.

A musician focusses on the music and not on his fingers and the instrument anymore. That’s why I pull the idea of flow into the discussion.

I think the equation >better immersive tools equals better presences equals better performance< is not true. Sometimes it might be. But is an assumption, and there are other ways for better user satisfaction and joy of use.

And_ Setting research goals and arguing about them is almost impossible if people use the same words for different stuff. Thanks a lot for your refs, thoughts, time!

Erik:

Final link, But final link. this is a useful survey paper: Martijn J. Schuemie et al: Research on Presence in VR: a Survey

Movies

Welt am Draht (World on a Wire), Rainer Werner Fassbinder, 1973; IMDBpart 1 | part 2

Disclosure (Enthüllungen), Barry Levinson, USA 1994; based on a book by Michael Crichton, 1994, IMDB

13th Floor, 1999, IMDB

Matrix, 1999, IMDB

References

HH

Theater

Feedback – please leave a message

I'd be happy to receive and read your feedback via_

à propos