Design4real VR AR Agency

Tracking technology in augmented reality

augmented reality tracking

What does tracking actually mean in the field of AR.

Augmented reality (AR) has taken on an increasingly important role in the world of technology and marketing in recent years. With AR, users can view their surroundings through a mobile device such as a smartphone or AR glasses like the Hololens 2 and supplement them with virtual objects or information. An important component of AR is tracking, which enables the precise positioning of virtual objects in the real world.

Tracking technology in AR is therefore of great importance as it creates the immersive experience of AR and allows users to interact with virtual objects in an intuitive way. However, there are some limitations that may arise in terms of the hardware and framework used.

An important limitation is the limitation of hardware power. Typically, AR tracking requires a lot of computing power to accurately determine the position of virtual objects in the real world. If the device's hardware is not powerful enough, this can lead to delayed or inaccurate tracking, which can negatively impact the AR experience. Especially on low performing mobile devices, it becomes difficult to drive AR applications. Depending on how many polygons the virtual objects shown bring, less or more performance is left for tracking. This means that the more polygonal the objects, the more difficult it is to track them cleanly in space.

Another problem can arise due to the limitations of the framework used. AR tracking technologies are usually implemented in AR frameworks such as ARKit from Apple or ARCore from Google. If the framework is not robust enough or does not provide sufficient features, this can limit tracking and negatively impact the user experience. Not all frameworks perform equally well.

Another challenge is that tracking in AR is a continuous experience that takes place in real time. It requires a high level of accuracy because the virtual objects must move in real time with the real world. A small error in tracking can result in the virtual object no longer appearing in the right place, which can significantly affect the user experience.


What are the different tracking methods.

Marker-based tracking is one of the oldest AR tracking technologies and is based on the use of physical markers placed in the real world. These markers serve as reference points for positioning virtual objects. The system detects the markers through the device's camera and tracks their position in real time to correctly position the virtual objects. Marker-based tracking is relatively easy to implement and provides high accuracy, but it requires users to carry physical markers, which can affect user comfort. This augmented reality marker method ensures accurate and stable representation of digital elements, synchronizing the spatial relationship between the marker and the AR content. In professional applications, especially in sectors such as retail, education and industry, marker-based tracking enables a consistent and reliable user experience.

Worldspace Tracking is a more advanced AR tracking technology that positions virtual objects in the real world without the need for physical markers. This is made possible by the use of sensors such as GPS, gyroscope and accelerometer to determine the position of the device in the real world. Worldspace Tracking provides greater freedom and flexibility for users as they do not need physical markers to place virtual objects. However, the accuracy of Worldspace Tracking depends on the availability of sensor information, which can cause problems indoors or in areas with poor GPS reception. Worldspace tracking is often a substitute solution in the WebAR domain for Plane Detection. The reason for this is that plane detection tracking is difficult to implement without access to all sensors of a smartphone.


World Space Tracking WebAR


Plane Detection -Layer detection in augmented reality is a critical aspect of creating immersive and realistic augmented reality experiences. Plane detection refers to the ability of an augmented reality system to accurately track and recognize flat surfaces in the real world, such as floors, tables, or walls. This tracking capability is important for placing virtual objects or information on these surfaces to create the illusion that they are part of the real world. Augmented reality tracking algorithms use computer vision techniques to identify and track these surfaces in real time so that users can seamlessly interact with virtual content. Accurate surface detection is critical to a smooth and immersive augmented reality experience, ensuring that virtual objects have stable footing and thus appear anchored in the real world.

AR Tracking Plane Detection Sample


Object Tracking is an AR tracking technology specifically designed for tracking objects in the real world. Object Tracking uses artificial intelligence and machine learning algorithms to detect and track objects in real time without the need for physical markers. This technology is particularly useful for AR applications focused on product demonstrations, shopping experiences, or games where users manipulate virtual objects in the real world. Object tracking offers high flexibility and accuracy, but requires high computational power to enable real-time tracking.

Since tracking has so much impact on the user experience, you should carefully familiarize yourself with the tracking capabilities and performance of the chosen framework beforehand.


Augmented reality glasses like the Hololens 2 or the Magic Leap use different tracking methods than smartphones and tablets.

The tracking system of the HoloLens 2:


Microsoft's HoloLens 2 represents a significant advance in the field of mixed reality headsets. One of the main advantages of this device over its predecessor and many other devices on the market is its advanced tracking system. This system is not only responsible for positioning holograms in physical space, but also enables the detection and interpretation of user gestures. Here we examine the key elements of this system and its importance to the user experience.

1. spatial tracking: HoloLens 2 uses a tracking system without external sensors. Instead, multiple cameras are integrated into the device that scan the environment in real time to detect the headset's position in space. This allows the user to move freely while holograms remain stable and consistently anchored in the real world.

2. hand and finger tracking: A notable feature of HoloLens 2 is the ability to track the user's hands and individual fingers in real time. This allows for a much more intuitive interaction with the holograms, as users can use natural gestures to interact with, move, rotate or scale digital objects.

3. eye tracking: Another impressive feature of HoloLens 2 is eye tracking. The device is equipped with sensors that track the user's eye movements, enabling a new level of interactivity. For example, applications can detect where a user is looking and respond accordingly. This can also help improve the user experience, for example by highlighting menus or options that the user is looking at.

4. reliability and latency: The HoloLens 2's tracking system is not only advanced, but also reliable. The latency between the user's real movements and the response of the digital content is minimal, contributing to a compelling and immersive experience.

5. privacy concerns: While the advanced tracking system offers many benefits, it has also raised privacy concerns. The constant collection of spatial and biometric data raises questions about data security and privacy. However, Microsoft has emphasized that the data is mainly processed locally and users have control over their data.

More about Hololens 2

Conclusion: The HoloLens 2 tracking system represents a significant step forward in augmented reality. By combining spatial tracking with hand, finger and eye tracking, it allows users to interact with digital content in an intuitive and natural way. While privacy is always a concern, the technology shows the potential to redefine the way we interact with the digital world.

From our blog