In recent years, Hand tracking developed into an integral feature of virtual reality (VR) devices. It allows users to use their hands as an input method instead of traditional VR controllers. This feature opens up a variety of opportunities and challenges for developers, especially concerning user-friendliness. In this article, we will delve deeper into the concept of hand tracking and examine the supporting VR headsets, along with their advantages and disadvantages.
Hand-tracking allows VR device users to perform tasks and actions in the virtual environment through simple hand gestures, such as pinching, grabbing, pushing, or pulling. Accurate tracking of hands in the virtual world is crucial for a realistic VR experience. A fully articulated tracking system and hardware are required to determine the position of the user's hands in virtual space relative to the real world, and to capture finger and joint movements as quickly and accurately as possible. This enables natural interactions and a more immersive experience.
Some VR headsets now natively support hand tracking. Examples include the HoloLens 2, Magic Leap 2, Meta Quest 2/3, Meta Quest Pro, and Pico 4 Enterprise.
Magic Leap 2 features an impressive 70° diagonal field of view, dynamic dimming, and outstanding 16GB RAM performance.
The HoloLens 2 is the second generation of Microsoft's revolutionary mixed-reality headset. It was officially introduced in 2019 and offers more processing power, better sensors, longer battery life, and a wider field of view compared to its predecessor. The HoloLens 2 features eye-tracking functionality that allows it to follow the user's eye movements. Instead of dedicated controllers, the HoloLens 2 is operated through gesture control and voice commands. New and more intuitive hand gestures enable easy control of content.
The Meta Quest 2 is a very popular standalone VR headset. The Meta Quest 2 also supports controller-free hand tracking. Using machine learning, the headset recognizes finger positions, allowing for the control of objects and actions in the virtual environment.
The Pico 4, similar to the Meta Quest, features optional hand tracking control in addition to VR controller input.
Leap Motion is a small device that can be connected to a computer via USB or mounted on the front of VR headsets, allowing users to manipulate digital objects with hand gestures and movements. With Ultraleap and the included SDK, you can upgrade VR headsets without built-in hand tracking to have hand tracking.
The Varjo XR-3 premium headset has Ultraleap built-in to enable the best hand tracking.
The Quest 3 features an improved hand tracking system. The system uses depth sensors to track the position and movement of hands in 3D. Hand tracking on the Quest 3 is more accurate and reliable than that of previous models. It can now also recognize more complex hand gestures, such as grabbing and releasing objects.
There are three ways users can interact with the virtual environment: direct manipulation, hand rays, and gesture recognition.
Direct ManipulationUsers can access and interact with objects in the VR environment with their hands. This method allows for intuitive control, where objects react realistically. Users can pick up objects, press buttons, and activate user interface components as if they were interacting with a touchscreen.
Sandblasting (laser)This method allows users to generate a beam from their hand to access remote objects. Through gestures, users can grab, rotate, or push objects from a distance. Users may even feel haptic feedback at the beginning or during interaction with the object.
Gesture recognitionWith this method, the VR device analyzes the user's hand and finger poses and triggers corresponding reactions. One example is a system that allows a user to take a screenshot by bringing the tips of their thumbs to their index fingers to create a square shape, mimicking looking through a camera's frame.
It is advisable to include gestures in the design when the inherent meaning of a gesture aligns with the desired outcome.
Intuitive Interaction: Hand tracking allows users to use natural and intuitive hand gestures to control objects in the virtual environment.
Increased Immersion: By being able to see one's own hands in real-time within the VR environment, a higher degree of immersion is achieved.
No additional controllers required: Hand tracking eliminates the need for additional controllers, improving comfort and usability.
Limited Functionality: The interaction possibilities with hand tracking are limited compared to traditional controllers. More complex actions and games may still require the use of controllers.
Limited Capture Range: The camera's capture range for hand tracking is smaller than the area where controllers are detected. Users must therefore be more careful about how they move.
Are you interested in developing a virtual reality or 360° tour application? You may still have questions about budget and implementation. Feel free to contact me.
I am looking forward to you
Clarence Dadson CEO Design4real
