As the metaverse grows, so does the demand for engaging and lifelike VR avatars. To foster a sense of social presence and immersion, it is crucial to Uncanny Valley Effect when designing these avatars. The Uncanny Valley phenomenon occurs when an avatar closely resembles human appearance but still evokes uneasy feelings in users. This blog post explores strategies for creating VR avatars that bridge the Uncanny Valley and enhance the metaverse experience.
Prioritizing Visual Realism
Achieving visual realism is crucial for compelling VR avatars. However, the pursuit of photorealism isn't always the best approach. Instead, a balance between realism and stylization should be sought. The focus should be on key elements that contribute to human recognition, such as facial features, eyes, and skin texture. By employing detailed modeling, texturing, and Shading-Techniques can create visually appealing avatars without triggering the uncanny valley effect.
Realistic Avatars
A realistic avatar offers many customization options to get as close as possible to one's own appearance. Especially in a professional environment where user credibility plays a role, an avatar with realistic features is often the first choice. However, the realistic look also harbors problems: the so-called „Uncanny Valley“ effect describes the point at which an artificial avatar wants to look as real as possible but only approximates this goal – which in turn makes it appear unbelievable or even creepy.
Hardware limitations play a role here, as do limited options for creating a realistic avatar with little effort. A common method involves taking a webcam photo of the user and projecting it onto a generic avatar. This serves its purpose, as the avatar is recognizable as a person, but the simple representation quickly breaks with the desired reality. Additionally, avatars mostly float depending on the chosen VR headset due to the lack of tracking capabilities for legs and face, and facial expressions appear relatively stiff and therefore also strange.
Avatars from photogrammetry data
A more technically sophisticated approach is to generate avatars from photogrammetry data.
The creation of VR avatars from photogrammetry data is a complex procedure that is divided into several steps. First, a 3D scan created by photographing the person from various angles. These photos are then processed using photogrammetry software to create a detailed 3D model. Finally, the model is converted into a VR-compatible format. This process makes it possible to create quite realistic avatars for VR applications.
3D Scan Apps for phones like Polycam
3D scanning of people on the iPhone is enabled by scanning software applications such as PolyCam significantly easier. This app uses the powerful LiDAR technology integrated into newer iPhone models to create detailed 3D scans of objects and environments.
Cartoonish avatars
Cartoony avatars generally don't have an uncanny valley problem because they don't attempt to replicate reality. Nevertheless, it offers many possibilities for individual design. For example, a user's characteristic features can be transferred to a cartoony avatar based on a photo – similar to what the app ReadyPlayerMe already does. Alternatively, users can design their own avatar using a character creator, as is the case in „The Sims,“ for example. The result is an avatar that has a consistent style, similar to other avatars, and is not dependent on external factors such as lighting and the quality of the user's webcam.
The cartoonish style, of course, doesn't appeal to everyone – a playful look in a professional setting, especially when presenting or showcasing a realistic product, can put people off or make presenters seem less credible. Also, cartoonish avatars have no legs and limited facial expressions.
Mastery of facial expressions and emotions
Facial expressions are crucial for human communication. To avoid the uncanny valley, it is important for VR avatars to show a wide range of emotions that align with their speech and interactions. Advanced animation and motion capture techniques enable natural and seamless facial expressions thanks to Face Capturing. VR headsets like Meta Quest Pro and Pico 4 Enterprise support Face and eye tracking. Special attention should be paid to eye movement, eyebrow movement, and lip movement, as these subtle cues play a significant role in the authenticity of the avatar's emotions. Facial Motion Capture allows facial movements from a real person to be transferred to the digital avatar. For example, there are face capture solutions that typically use an iPhone to record facial movements. For instance, iClone and Metahuman Animator from Unreal. With them, it is possible to transfer realistic emotions to the characters with moderate effort. With the help of automatic blinking, it is also possible to make a character blink realistically without much effort.
Realistic animation of avatars and NPCs through motion capture (not real-time)
Natural and fluid animation is crucial for avatar believability. Stiff or robotic movements can instantly shatter the illusion of realism. The use of Motion capture technology allows for the capture of real actors' movements and gives avatars an additional layer of authenticity. Furthermore, attention should be paid to body language and gestures. These should accompany the avatar's speech and enhance the communication experience rather than detract from it. Afterwards, in 3D software such as IClone or Motion Builder, the motion capture animations will be revised to enable even better, more realistic results. For example, the character's gaze can be further adjusted to the 3D world, or collisions with the 3D body can be removed.
No standard formats for avatars
Unfortunately, there are no standard formats for avatars, so an avatar optimized for one software solution cannot easily be integrated into another.
Realistic animation of avatars in real-time
The matter becomes more difficult when player movements with VR glasses are to be captured in real-time in VR using the headset's sensor technology and transferred to the player avatar. Meta has made great progress in this area recently, so that Meta's avatars can now even have legs in the virtual world. Furthermore, the very disruptive overlaps between arms and the upper body have been significantly reduced. To enable more realism in real-time avatar movements on the hardware side, headsets like the Quest 3, which, thanks to improved cameras and sensor technology, can better track the position of the wearer's body file. In addition to the hardware, the software, Avatar SDK, is also largely responsible for how the avatars' movements are reproduced. Here, hardware manufacturers such as Meta and Pico have their own solutions with different features. However, there are also plugin implementations for Unreal and Unity that are device-agnostic.
Focus on language and audio integration
Voice is another important aspect of the metaverse experience. Ensure that the avatar's voice matches its appearance and conveys emotions appropriately. Integrating real-time voice modulation and lip-syncing can significantly enhance the avatar's realism. It is better to use voices of real people rather than relying on AI voices. AI voices currently still sound quite robotic and convey fewer emotions in their voices, unlike real people. Furthermore, spatial audio techniques can help anchor the avatar's voice to its position in the virtual environment, further immersing users in the experience. With lip-sync technology such as AccuLips From iClone, audio files can be automatically transferred into realistic lip movements on the avatar.
Customization and Personalization
The ability to customize avatars can help users feel more connected to their virtual representations. Offer a diverse selection of customization options for facial features, hairstyles, clothing, and accessories. Personalization not only improves user engagement but also mitigates the uncanny valley effect, as individual preferences for realism can vary. With programs like Character Creator 4 or even Unreals MetaHuman Can personal avatars be created. Using images and 3D scans, for example with the Character Creator 4 plugin.„Headshot“and its improved successor„Headshot 2“Avatars can be created that are virtually identical in appearance to their real-life counterparts. Precise facial recognition and advanced AI technology enable remarkable accuracy and attention to detail, resulting in avatars that exhibit a striking resemblance to the real individuals.
Iteration and collecting user feedback
Creating realistic VR avatars is an iterative process. Continuously gather user feedback and observe how users interact with the avatars in different scenarios. Identifying specific elements that trigger the uncanny valley effect can help optimize design and animations. It's important to actively involve users in the development process to create avatars that align with community and customer preferences.
Conclusion
- Avoiding the uncanny valley effect is crucial for VR avatars in the metaverse.
- Prioritizing visual realism and mastering facial expressions are important aspects.
- Animation and movement create believable avatars.
- Focus on language and audio integration enhances the reality of avatars.
- Customization and personalization enable individual user experiences.
- Iteration refined the avatars based on user feedback.
- Continuous advancements in technology and innovation in the field of avatar integration are opening up new immersion possibilities in the metaverse.
Frequently Asked Questions (FAQ)
The uncanny valley refers to the effect where an avatar appears more likeable the more human-like it becomes—until shortly before perfection, when the effect reverses and the figure appears unsettling or eerie. The triggers are usually dead eyes, stiff facial expressions, and movement that does not match the realistic appearance.
Most reliably through deliberate stylization instead of maximum photorealism. More important than skin pores are coherent eye movement, blinking, head movement, and lip-sync—a slightly stylized avatar with good animation almost always feels more pleasant than a photorealistic one with bad animation.
The usual method is photogrammetry: the person is photographed from many angles, from which the software calculates a 3D model, which is then cleaned up and rigged with a skeleton. For simpler requirements, LiDAR-supported scanning apps like Polycam on the iPhone are sufficient today.
Character Creators for quickly building customizable figures and Unreal's MetaHuman for high-resolution, realistic characters are widespread. Both can use photos or 3D scans as source material and be further processed in Unity or Unreal.
Body movements come from motion capture, facial expressions from face tracking. Headsets like the Meta Quest Pro and Pico 4 Enterprise capture eye and facial movements directly and transmit them in real-time to the avatar—allowing gaze and expression to be realistically portrayed even in virtual meetings.
No. To this day, there is no end-to-end standard between platforms and engines, which is why avatars generally have to be adapted for each target system. Anyone who wants to reuse avatars should take this into account from the very beginning when choosing the creation tool and topology.
