Virtual reality seems to share some qualities with the mythical Phoenix. Every few years it is hyped up enormously, only to die fairly quickly again. Every decade or so, the cycle reboots and the hype rises from its ashes. The most recent examples are the Oculus Rift and the HTC Vive and they might actually break this cycle. What is so special about these latest VR-incarnations? And how can we, in the words of philosopher David Chalmers, use them to address Big Questions in Philosophy?
Both the Rift and the Vive are helmets which provide a virtual reality experience with two lenses, at a resolution of 1080×1200 pixels per eye. Although impressive, the video resolution is low enough that users are still confronted with pixel-y environments. What sets them apart from previous VR experiences is the fact that they come with hand-held controllers—the Vive already includes these while the Rift will have them later this year. For the first time, virtual reality devices that allow users to actively engage with a virtual environment will be available for a large consumer market. Google will soon be jumping on the band-wagon with an even more affordable “Daydream” product.
The fact that these visors merge with controllers might seem like nothing more than a nifty trick, but the development is more fundamental than that. In his 2003 book Natural Born Cyborgs, philosopher Andy Clark observed why the then-current VR-devices failed to impress: “it will not be enough to present the eyes with a fully realized, rich three-dimensional (3D) scene if we cannot also in some way move and act within the scene itself” (p. 110). Why is engaging with the environment so important? The answer to this lies in recent developments in the philosophical and scientific movement called embodied, enactive cognition.
Where traditional cognitive science tends to understand cognition from the brain, the embodied, enactive approach understands cognition from the constant interaction between an individual’s brain, body, and environment. It also provides an answer to the question of why acting in a virtual reality environment, makes so much more of an impact. A reviewer from the Dutch technology website Tweakers.net, while using the Vive, exclaimed: “It’s like I’m completely immersed in this [virtual] 3D world !” Why does this type of interaction feel so much more engaging than merely using a keyboard and mouse, or some other similar interface? Clark, this time in Supersizing the Mind (2008, p. 10), explains:
“It seems unlikely that immersive virtual reality (VR) is by its very nature disconnected in this sense. Rather, it is just one more domain in which a skilled agent may act and perceive. But skill matters, and most of us are as yet unskilled in such situations. Moreover, the modes of sensing and interaction supported by current technologies often remain limited and clumsy, and this turns the user experience into that of a kind of alert game player rather than that of an agent genuinely located inside the virtual world.”
Compare this to one of the reviewers of the Vive: “Because I see my hands, I feel more like I’m actually in this space.” and “I don’t have to remember what all the buttons on my controller do. [..] I don’t have to think about what I want to do, I just do it.”
Precisely because these controllers feel like an extension of the body, interacting with a game feels much more intuitive. When Clark mentions the “clumsy” technologies, he is talking about interfaces that were designed to work despite the human body, not with the body. Consider how unfit the design of a computer keyboard is for interaction with a human body. It has a ton of keys, and is designed for producing texts—that is in fact part of its heritage, being a descendant of mechanical and electronic typewriters. Yet nowadays we use keyboards for video editing, for moving around in a game, for, in combination with a mouse, graphic design. These are all types of interactions that could be done so much easier by other interfaces. In fact, if today one would give a computer engineer the assignment to design the perfect interface for interacting with a 3D virtual reality environment, it seems very unlikely she would come up with a keyboard and mouse.
This development is hard to explain from the perspective of mainstream cognitive science, given that it centres on the brain as the seat of cognition. A rough sketch of its current paradigm goes like this: when we interact with the world we build a representation of it. Interacting with the world means manipulating that representation to predict how to get the intended desired effect in the world. For an elaborate argument of why this view is misguided, I refer to Robert Epstein’s excellent recent essay. It suffices to say here that an embodied instead of brain-based approach is much better suited to explain the impact of these new VR technologies and guide development to make them even more immersive.
“It is time to turn this secret affair into a proper marriage.”
What we are wired to see when we look at the world is not a model of the world, or even the world ‘as it is’. The world we perceive is one in which potential actions to be performed by us stand out, as psychologist J.J. Gibson already argued in The Ecological Approach to Visual Perception (1979). The world we perceive affords certain actions: a button is there to be pushed, a chair to sit upon, an apple to be eaten. The reason why embodied gaming works has such a wow-factor is because it plays into our tendency to perceive the world as something with which to be engaged. Using the controller of a Rift or Vive means becoming that much more immersed in a reality that invites you to act in it. And that is why this VR evolution can have so much more of an impact that previous developments had. For this to go all the way, it is important that tech developers realize it is not the resolution of the graphics that makes VR immersive (though it helps): it is the way the interface connects the human to the virtual reality. Research at the University of Minnesota, by Thomas Stoffregen and his colleagues, shows that enactivism and VR make a good couple. But it is time to turn this secret affair into a proper marriage.
If tech developers go further down the line of enactive virtual reality, and they should, it would put a nice twist on a conclusion Clark draws in Surfing Uncertainty (2016). Some cognitive scientists claim that what we perceive of as ‘reality’ is indirect, and so we actually live in a virtual reality. As I’ve shown, Clark argues against this by claiming there is no “representational veil between agent and world” (p. 170). I would go even further. Because this new kind of virtual reality is cast in terms of physical affordances, it allows us to engage with it so convincingly that we should not say reality is virtual, but that the virtual is real.
Thanks to Miguel Segundo Ortin and Michael Kirchhoff for some helpful comments. Image is a still from the HTC Vive review by Tweakers.net. Some of the thoughts from this blog post have found their way into the following paper: Peeters, A., & Segundo-Ortin, M. (2019). Misplacing memories? An enactive approach to the virtual memory palace. Consciousness and Cognition, 76, 1–12. https://doi.org/10.1016/j.concog.2019.102834
