What is it Like to be a Bat?
Fifty years ago, philosopher Thomas Nagel pondered the question: what is it like to be a bat? He then went on to provide a rather unsatisfying answer: We — in principle — cannot know. The essay argued that it is impossible to re-experience the experience of other creatures. A bat, for example, primarily perceives the world through its highly sensitive sonar echolocation system. Not only is its sensory apparatus radically different from ours, but also its whole brain, and so is the inner life of a bat.
I call Nagel’s answer ‘rather unsatisfactory’ because we as humans are constantly occupied with what is going on inside the heads of others. Psychologists call this phenomenon “theory of mind” (TOM), although rather than forming theories we hypothesise, or make guesses, about thoughts, motives and goals. All too often our guesses are wrong because we cannot know. In contrast, cognitive scientists seem to suggest that the electrodes that record neural activity and the images that show blood flow inside the brain are somehow able to tell us the whole story of our mind, supported by Artificial Intelligence, at some point in the future. Like all those predictions, it is permanently located in the future, but definitely soon.
Yet a map, one would like to shout in the directions these scientists (or marketeers), is not the territory (Alfred Korzybski). Rather, all we have besides our intimate, firsthand account of what it is like to be a human is communication. It is crucial to recognise that communication does not occur between minds or brains, but within social systems, as observed by Niklas Luhmann (a topic to be explored in a future essay).
Although we are not in a position to understand either bats or other humans completely, what remains is a substantial difference between the levels of human-to-bat and human-to-human understanding (leaving open the question of how bats might understand humans). This gap is due to what Humberto Maturana calls “structural determinism”. While we likewise cannot grasp or explain the mind or consciousness, despite claims to the opposite (which sometimes come in mathematical clothing), different biological structures that underlie these phenomena keep up insurmountable boundaries to their respective territories.
Take the Red-eyed Tree Frog (Agalychnis callidryas), whose eyes are covered during sleep with a translucent membrane to help it detect approaching predators. Do we really think we are able to fully grasp or simulate the experience of seeing through such an eye?
Artistic Explorations
Nagel’s scepticism has not discouraged artists, scientists and philosopher from imagining the sensory experiences of ‘the other’. Human echolocation and synaesthesia are two widely explored phenomena both in art and in cognitive neuroscience. Sensory implants have enabled people to substitute vision with tactile and auditory sensations. The cyborg movement, at one point represented by a foundation and by a formally registered society located in Berlin, is longing for new senses and sense prosthetics.1
Other artists are creating mixed reality experiences around a variety of phenomena like psychosis (Labyrinth Psychotica by Jennifer Kanary Nikolov(a)), empathy (The Machine to be Another by BeAnotherLab), and blindness (Notes on Blindness: Into Darkness by Arnaud Colinart). Then, there are audio and video games that play with sense modalities like Papa Sangre (Somethin’ Else), BlindSide (Aaron Rasmussen, Michael T. Astolfi) or Perception (The Deep End Games). Some of these experiments not only play with the reduction and augmentation of our current sensory apparatus, but also attempt to provide their so-called “able bodied” audience a glimpse into the reality of a person with a disability.
A Walk in the Forest
Imagine you are walking up a small path through a scenic forest, a few minutes away from where you parked your car, almost an hour from Lancaster, right in the Lake District of North West England. Some unobtrusive objects guide you along the way. After a few minutes, you arrive at a small meadow, surrounded by the trees and bushes of Grizedale forest. You see several tree stumps, inhabited by strange creatures. From the shoulders down, they look human, but their heads appear strangely different. Their faces are covered with mossy spheres that appear to be helmets, cables dangling from their backs. Tacitly, you come closer. One of the helmets is empty, waiting for you. A strange attraction emanates from the device, drawing you towards it. Hesitantly, you put it over your head and enter…
In her seminal book Hamlet on the Holodeck, Janet Murray characterised immersion as “…the sensation of being surrounded by a completely other reality, as different as water is from air that takes over all our attention, our whole perceptual apparatus.” Marshmallow Laser Feast created In the Eyes of the Animal, in Virtual Reality, as a vehicle for this kind of immersion that lets us deeply connect to the ‘other’2. They deploy VR to let us experience different modes of perception of various animal inhabitants of the forest. While the modalities of the individual animal’s perception are based on solid research, the result is shaped by artistic interpretation. In the Eyes of the Animal, as well as their other projects, such as Observations on Being and Distortions in Spacetime, are playful virtual explorations of the world beyond our senses.
Immersed in Point Clouds
Whereas other developers are pushing towards more and more realism3, MLF has chosen a different aesthetic route. To visualise the forest, they utilise LIDAR, the technology that is used in self-driving cars to scan the environment in real-time. The results are point clouds consisting of almost a billion individual particles. This massive amount of data must be drastically reduced before it can be imported and processed within the visual interactive environment vvvv (later replaced by Unreal Engine). To achieve sufficient performance, the particles are run through highly optimised shader code that generates the unique aesthetics of the artwork. Finally, the visuals are sent to the custom-made helmets, which contain Oculus Rift headsets and headphones. A binaural soundtrack running in Max/MSP can be felt through SubPac tactile bass wearables, which contribute to the immersive quality of the piece.4
The natural setting for In the Eyes of the Animal is the forest and it is best experienced in this environment. The piece also has been travelling to other places such as Sundance, Playgrounds Festival, Toronto Film Festival, Yamaguchi Center and Sonar.
Virtual Storytelling
If we peek beyond the technology-driven consumer hype cycles, there is a huge creative space in VR, one that asks to be explored through artistic and scientific experiments, games and environments. Examples of these kinds of explorations include Disconnected by Thorsten Wiedemann and Sarah Lisa Vogl in which Thorsten stayed 48 hours in VR, VR Rollercoaster where the physical sensation of a rollercoaster ride is combined with virtual experiences and the creepy cooperative game Taphobos that invites its players to crawl into coffins.
The medium poses new challenges as well as opportunities for interactive storytelling. VR is unique in its physiological and cognitive affordances and its growing but still somewhat limited capacity to convey embodied experiences and to create virtual social spaces.
Half-Real
With its immersive qualities and three-dimensional interaction, Virtual Reality still has the potential to replace the smallish and flat screens that are now our predominant interface. Games can provide significant pathways towards this future, as powerful experiential vehicles that allow us to access new identities, take up different roles, and rehearse unfamiliar views and perspectives. The game studies scholar Jesper Juul describes games as a hybrid form that combines real rules with fictional worlds. They constitute a medium in which we can experience possible worlds through play.
In the end, it is on us, the creators and inhabitants of these worlds, to choose and to shape the realities we want to experience. The visions of large corporations keen on selling us their latest entertainment content, or the creative, quirky, touching, and open experiences that offer us access to a deeper meaning of what it is to be human, even through the eyes of an animal.
This is an updated and edited version of an essay published in A MAZE Magazine issue 4 in August 2016, alongside an interview with Barney Steel and Ersin Han Ersin of Marshmallow Laser Feast. I mixed in some topics from my talk “How to Perceive the Virtual Image? On the Distinction Between Virtual and Real”, delivered at the Transimage conference at the University of Plymouth, UK, also from 2016.
Since I wrote the essay in 2016, by 2025 both entities mentioned have stopped their activities. In my opinion this rather reflects on the long-term engagement, or a lack thereof, of certain artists and hackers than on the cyborg concept itself.
As mentioned above, this "other" can also be us, when we question and rediscover our own perceptions, as the works "Lichtbilder" (Light Pictures) by Ruth Schnell impressively demonstrate.
One example is Sólfar Studios, creators of Everest VR.
A biographical note: For a couple of months during my PhD in England in 2016, I shared a warehouse apartment in Hackney Wick with a member of MLF. The gorgeous place sometimes would transform into a VR production studio, photoshoot location or rooftop party space.