In the many conversations around the âmetaverseâ sparked by the transition of the business name from Facebook to Meta, much of it has focused on the visuals. What is barely mentioned is the audio. Yet the voice really matters in bringing a virtual environment to life.
Sometimes it is all.
Just ask Spike Jonze. The director dismissed the original voice actor in the title role of his 2013 film “She” and replaced the sultry timbres of Scarlett Johansson. Although Samantha, a computer operating system, never appeared in the flesh, Jonze felt that the original actress failed to capture the emotion required to create a three-dimensional character.
The voice was essential in creating a fleshed out character that could absorb the viewer into the premise of the story and make it fully believable.
Like The Washington Post Noted, many keystones of the Meta vision of the Metaverse already exist in video games – only in disconnected game worlds. And in the gaming world, the voice plays an increasingly important role. Meta promises a unified and interoperable experience, but without a rich set of highly textured and realistic digital voices, the Metaverse will be incomplete rather than inclusive and immersive.
the Research on the McGurk effect of the mid-1970s observed cognitive dissonance resulting from inadequate audio and visual perceptions; voices that do not fully match an avatar can tear the participant away from the virtual environment.
Express the real you
Humans are social beings, and the Metaverse as currently promoted is a social environment where participants create unique characters both at home and at work. Avatars will allow players to express themselves the way they want to be seen – as a human, alien, animal, plant, cartoon, or a myriad of other options. Players can temporarily try on new “looks” just like they try on new outfits. Both genus and species are fluid.
Changing identity is crippled, however, if people are not able to change their sound along with their visual presence. Matching your voice to the personality presented to other people is a central part of a personalized player identity. This is a situation that many people are already used to in video games.
If you come across a gritty, bearded, and towering knight in a game you play, you would expect this character to have a deep, gruff voice, accompanied by the clatter of armor. Game companies make sure to deliver that, with carefully crafted non-player characters (NPCs) by voice actors and audio specialists to deliver an immersive experience.
Yet in online gaming environments or in the future metaverse, where the knight is the representation of a real person, you will have a very different experience. You might be surprised to hear a high-pitched teenager with poor microphone quality instead of the ripe, gritty voice you expect. The drastic incongruity between sound and vision tears apart the immersive quality of the experience. Metaverse avatars can only be fully immersive if they allow people to create complete digital experiences.
In addition to allowing immersion, sound identity technology can also allow players to slip into a ârealâ alias. They can fully become the person (or being) they want others to see – which for many people is a powerful safeguard against a sometimes hostile online environment. This can mask a geographic focus so that the participant can more easily fit into a community of gamers (a capability that an offshore customer support call center could benefit from). For people with vocal tics, this may mask a physical impairment that they would rather not reveal.
Voice change technology can also help reduce discrimination and harassment online. A research study published in the International Journal of Mental Health and Substance Abuse in 2019, notes that female players frequently avoid verbal communication with other players to reduce unpleasant interactions. Voice switching technology can allow them to participate in fully pseudonymous, genderless conversations in which they might feel more comfortable expressing themselves.
Regardless of the “why”, the researchers of the scholarly journal Human-machine interaction concluded in 2014 that âthe voice radically transforms the experience of online gaming, making virtual spaces more intensely socialâ.
From internal data from my own company, it’s clear that gamers who communicate with vocal alter egos feel more engrossed in the game, engage it longer, and spend more money in the game as a result.
What is missing in the metaverse
A truly comprehensive immersive experience requires a combination of 3D visuals and real-time audio to allow people to express themselves the way they want to be heard. Participants want a sound representation of themselves that is just as original and unique as their visual avatar – and they want the tools to personalize their voice as meticulously as their appearance. There has to be a harmonious marriage of augmented audio and 3D video to keep the player immersed and engaged.
Realtime audio defines how people can bring the ultimate individuality to their content, making audio the great equalizer of the metaverse. Unfortunately, the current vocal experience is challenged to deliver the kind of immersive qualities that will live up to the promise of a global metaverse.
Real-time audio personas are still restrictive at best, despite the experimentation of stubborn early adopters. The tools for shaping a person’s voice to match their digital self are limited, and sound quality does not yet match visual quality.
Yet recent advancements in available audio technology make it much easier for gamers to create a unique sound identity. New solutions available to platform and game developers allow writers, producers and sound engineers to integrate voice modification technology into their games to produce natural and fantastic voices on demand, in real time .
It offers the ability to generate new avenues of monetization by delivering an inclusive and immersive hearing experience that draws players in and keeps them fully focused and engaged in the experience rather than walking away.
Businesses are investing in powerful tools that enable people to shape the visual representation of themselves in a digital space. They should not neglect the personalized sound identity for a corresponding social audio experience that makes the digital representation transparent.
The Metaverse wouldn’t be complete without him.