Compelling User Experiences in VR and MR | Immersive UX Design

Crafting captivating user experiences in VR and MR Immersive UX Design Virtual Reality (VR) and Mixed Reality (MR) are long no longer

Compelling User Experiences in VR and MR | Immersive UX Design

Creating captivating user experiences in VR and MR
Immersive UX Design

Virtual Reality (VR) and Mixed Reality (MR) are no longer futuristic fantasies; they are rapidly evolving technologies that are changing how we interact with information, entertainment, and each other. However, as these immersive worlds become increasingly accessible, the User Experience (UX) Design into the spotlight. Designing intuitive, engaging, and comfortable experiences in these new dimensions is crucial for their widespread adoption and success.

Unlike traditional screen-based design, UX in VR/MR presents a unique set of challenges and opportunities. We are no longer confined to flat surfaces and mouse clicks. Instead, we design for three-dimensional spaces, leveraging body movements, spatial audio, and novel forms of interaction. This paradigm shift requires a fresh perspective and a deep understanding of human perception and behavior in immersive environments.

Navigating Unfamiliar Terrain: Key Considerations for VR/MR UX Design

The design for VR/MR requires a differentiated approach that takes the following critical aspects into account:
    • Immersion and Presence The goal is to create a credible and captivating sense of „being there.“ This requires careful attention to visual fidelity, spatial audio, and responsive interactions that make the virtual or augmented environment feel real and tangible. Poorly designed interfaces or sluggish performance can shatter this illusion, leading to frustration and discomfort.
    • Interaction Design in 3D Space: Forget mouse and keyboard. We are now working with Hand tracking, gesture recognition, voice commands, and specialized controllers. Designing intuitive and natural ways for users to navigate in 3D spaces, select objects, and manipulate them is crucial. This requires rigorous testing and iteration to find the most comfortable and efficient solutions. This often involves choosing between two primary approaches for interface design, each of which has distinct characteristics and implications for the user experience:
        • Diegetic Interfaces (Interfaces embedded in the world): At its core, diegetic interface design aims to create virtual controls and displays that Part These elements exist within the narrative and spatial context of the environment. Think of the dashboard in a virtual car, the buttons on a virtual elevator panel, holographic controls projected by a virtual device, or even information etched into the walls of a virtual environment. The defining characteristic is that these interfaces would be conceivable within the reality of the virtual world.

          The advantages:

          • Deep immersion and presence
          • Intuitive and discoverable interaction
          • Narrative Integration
          • Improved spatial awareness

          The disadvantages:

           
            • Significant design and development effort
            • Restrictions on Information Display
            • Accessibility considerations
            • Maintaining consistency across different elements

      • Non-diegetic interfaces (interfaces as overlays or tools): In contrast, non-diegetic interfaces exist outside in the fictional world and serve as tools or overlays that provide information and control. They are often depicted as 2D panels, menus, or HUD elements that float in the user's field of view or are attached to their head. Think of a settings menu that appears when you press a button, a health bar that displays at the bottom of your field of view, or a laser pointer used to interact with distant objects. These interfaces are functional but are not intended to be perceived as part of the virtual world itself.

        The advantages:

        • Efficient Information Delivery
        • Established UI Conventions
        • Rapid prototyping and implementation
        • Flexibility and scalability

        The disadvantages:

        • Interruption of immersion and presence
        • Cognitive load from switching
        • Lack of naturalness and embodiment
        • Potential for visual clutter and obstruction

      The choice between these approaches or a combination of both is a critical design decision that significantly impacts the overall user experience in VR/MR. It influences immersion, intuitiveness, learnability, and overall user satisfaction. Designers must carefully consider the pros and cons of each approach within the context of their specific application and target audience.
 
    • Spatial Awareness and Ergonomics: Design for VR/MR means considering the user's physical space and limitations. Overly complex movements or interactions that require excessive physical exertion can lead to fatigue and discomfort. Designers must prioritize ergonomics, ensuring users can interact comfortably for extended periods, regardless of the interface approach.
 
    • Information Architecture in 3D The effective display of information in a 3D environment is a significant challenge. Overloading the field of view or presenting too much information at once can be overwhelming, whether using diegetic or non-diegetic interfaces. Designers must carefully consider how to layer information, utilize spatial cues, and guide the user's attention in a natural and intuitive way.
 
    • Onboarding and Learning Curves: Immersive technologies can be unfamiliar to many users. Effective onboarding experiences are crucial to familiarize them with new interaction paradigms and specific interface elements, whether diegetic or non-diegetic. Clear tutorials and intuitive visual cues are essential.
 
    • Managing Potential Discomfort: Motion sickness, eye strain, and disorientation are potential pitfalls of VR/MR experiences. Interface design, particularly the placement and movement of non-diegetic elements, can contribute to these issues if not carefully considered.
 
    • Accessibility and Inclusivity: VR/MR design must consider users with disabilities. The choice of interface approach can affect accessibility. For example, diegetic interfaces might be more intuitive for some, while non-diegetic panels could be easier for others to read or operate with alternative input methods.

 

Target audience preferences and input method considerations

The choice of input method significantly impacts the user experience in XR, and the effectiveness of different interface approaches can vary depending on the input method.

    • 6DoF Controller Well-suited for interacting with both diegetic elements (pressing virtual buttons) and non-diegetic interfaces (pointing and clicking on floating panels). Controller precision can be advantageous for both.


 

  • Hand tracking Excellent for interacting with diegetic interfaces that mimic real-world manipulations. Grabbing a virtual lever or pressing a virtual button with your hand feels natural. However, delays and inaccuracies can make interacting with smaller, non-diegetic targets more difficult.
  • Eye-Tracking Can be used to select elements on both diegetic and non-diegetic interfaces. However, precision limitations for small targets are particularly relevant when interacting with densely packed non-diegetic panels or small diegetic controls.

Best Practices for Designing Engaging VR/MR Experiences

Given these considerations, here are some best practices for designing effective VR/MR experiences:

  • Apply user-centered design: As with any UX endeavor, understanding your target audience and their needs is paramount. Conduct thorough user research and testing, gathering feedback throughout the design process, paying attention to interface preferences.
  • Prioritize natural and intuitive interactions: Utilize real-world metaphors and design interactions that feel natural and intuitive. Mimicking physical actions, especially in diegetic interfaces and hand tracking where appropriate, can lead to a more seamless experience.
  • Offer multiple input options: Ideally, XR applications should support multiple input methods (controllers, hand tracking, voice) to accommodate different user preferences and the requirements of various tasks, and the interface approach should be compatible with these options.
  • Strategically select interface approaches based on experience goals:
    • Diegetic interfaces prioritize for deep immersion: If the primary goal is to create a highly immersive and believable experience, prioritize diegetic interfaces for core interactions. Invest in the design and development to make these elements functional and intuitive.
    • Implementing non-diegetic interfaces for utility and information management: When efficiency in information delivery and access to system-wide controls are paramount, non-diegetic interfaces can be effective. Design them thoughtfully to minimize immersion breaks and potential clutter.
    • Leveraging Hybrid Approaches for Practicality and Balance: A hybrid approach often offers the best balance. Use diegetic elements for interactions directly related to the objects in the virtual world, and non-diegetic panels for additional information or controls that don't naturally fit within the environment. Ensure a clear visual and interaction hierarchy between these elements.
    • Consciously weigh compromises: Consciously weigh the trade-offs between immersion and information density, and between naturalness and efficiency, when choosing an interface approach for specific tasks.
  • Test both approaches comprehensively: Prototype and test both diegetic and non-diegetic interface elements with your target audience for specific tasks. Pay close attention to user feedback regarding intuitiveness, usability, immersion effect, and information accessibility.
  • Maintain consistency: Regardless of the approach (or combination thereof), strive for consistency in visual design, interaction patterns, and feedback mechanisms across all interface elements.
  • Optimize for input method: Consider how different input methods (controllers, hand tracking, eye tracking) interact with both diegetic and non-diegetic interfaces. Design interfaces that are operable and comfortable with the intended input methods. For example, ensure that touch targets on non-diegetic panels are large enough for reliable hand tracking input.
  • Create a clear visual hierarchy: Whether diegetic or non-diegetic, ensure a clear visual hierarchy to guide user attention and make interactive elements and important information easily recognizable.
  • Minimize overload: Especially with non-diegetic interfaces, avoid overwhelming the user with too much information or too many panels at once. Consider gradually revealing information as needed.
  • Consider ergonomics and comfort: Ensure that interaction with both types of interfaces is comfortable and does not require excessive physical exertion. For example, avoid placing frequently used non-diegetic panels in awkward positions that require constant head turning.
  • Iterate and prototype quickly: VR/MR design is highly iterative. Prototype early and often to test your ideas in the target environment and gather valuable feedback on the usability of the interface.
  • Focus on performance and stability. Delays and performance issues can quickly destroy immersion and make interacting with interfaces frustrating, regardless of the approach.
  • Collaborate across disciplines Successful VR/MR experiences require close collaboration between designers, developers, artists, and sound designers to ensure seamless interface integration.
  • Provide clear onboarding and tutorials: Explain clearly how to interact with both diegetic and non-diegetic interface elements within the experience.
  • Provide clear visual and haptic feedback: Enhance the sense of physicality for diegetic interactions and provide clear feedback for selections on non-diegetic panels.
  • Continuous learning and adaptation: Stay up-to-date on the latest interface design patterns and best practices for VR/MR.

Frequently Asked Questions (FAQ)

In VR there is no fixed screen area, but rather a space around the user. Mouse and keyboard are replaced by hand tracking, gestures, gaze, voice, and controllers, and every design decision directly affects posture, fatigue, and well-being.

Diegetic interfaces are part of the virtual world—such as a control panel on a machine or a display on a virtual wrist. Non-diegetic interfaces float as a panel or HUD above the scene. Diegetic solutions feel more immersive, while non-diegetic ones convey information faster. In practice, a combination often proves successful.

Crucial factors are consistently high frame rates, movement initiated by the user themselves, and the avoidance of forced camera movements. Teleportation instead of free gliding, a fixed visual reference point, and a brief darkening of the edge of the field of view during movement significantly reduce motion sickness.

Controls should be located within the easily reachable area in front of the upper body, approximately at chest height and within arm's reach. Anything permanently positioned above head height, behind the back, or on the floor quickly leads to fatigue with prolonged use.

As little as possible, but always something. A short introduction integrated into the scene, in which the central interaction is practiced once, has proven effective. For trade fair and museum applications with changing users, the operation should work without any explanation.

Spatial effect, tangibility, and fatigue cannot be judged on a screen—they only become apparent in the headset. Therefore, VR UX is highly iterative: testing early, simple prototypes in the headset, readjusting, testing again.

Clarence Dadson

Interested in an XR project?

Whether Virtual Reality, Augmented Reality, 360° Video or 3D scan — we accompany you from the initial idea to the finished application.

Clarence Dadson · CEO Design4real

+49 179 3927971
·
info@design4real.de
·
Lindwurm St. 95a, Munich