AR Tracking:
How do SLAM, Plane Detection, and Image Tracking work?
AR Tracking is the technical foundation for ensuring that virtual content remains stably aligned with the real world in augmented reality. An AR system must know where the smartphone, tablet, or headset is located, how it is aligned, and what structures, surfaces, or reference points are present in the environment.
Therefore, in 2026 AR Tracking does not consist of a single method. Modern systems combine cameras, IMU sensors, visual inertial odometry, SLAM, plane detection, depth maps, image tracking, and – in location-based applications – GPS or visual positioning.
This article explains the technical tracking methods. If you are already planning a specific AR project, you can find our services on the pages Augmented Reality Agency and Augmented Reality Development.
Tracking in Augmented Reality refers to the process of determining the position and orientation of a device (like a smartphone or AR glasses) in real-world space. This allows AR applications to accurately overlay virtual objects onto the real environment, making them appear as if they are part of it.
Tracking describes the ongoing determination of Position and orientation of a device or relevant objects. This allows an AR application to display virtual content in such a way that it remains stable in the same place in the room despite the user’s movement.
Technically, it usually involves three tasks:
- Pose Tracking: Where is the device located and in which direction is it facing?
- Understanding the environment: What areas, depths, objects, or visual features does the system recognize?
- Anchoring: How is a virtual content anchored to a specific real position?
Current smartphones and XR headsets combine camera and IMU data for this purpose. This combination is significantly more robust than the previous idea that AR is merely a camera image with an overlaid 3D object.
1st World Tracking, VIO and 6DoF
The basis of many modern AR systems is World Tracking. The device tracks its movement with six degrees of freedom – that is, three rotational axes and three position axes.
For this reason, it is often Visual-Inertial Odometry (VIO) Used. Camera images provide information about salient points in the environment, while gyroscopes and acceleration sensors capture very fast motion data. From both data sources, a continuous estimate of the device pose is generated.
The principle is similar to the 6DoF inside-out tracking in modern VR and MR headsets.
2nd SLAM: Determine your position and simultaneously build a card
SLAM stands for Simultaneous Localization and Mapping. The system localizes itself in an environment and simultaneously creates or updates a spatial model of that environment.
In practice, visual features are tracked over multiple frames and combined with IMU data. This allows an AR system to reorient itself even when the user moves or looks back at a familiar environment after a brief interruption.
SLAM is today a fundamental component of many native AR and mixed reality systems. It is therefore more meaningful to speak of a combination of VIO, mapping, and anchoring than to consider SLAM as an isolated „tracking technique.“.
3. Plane detection and hit testing
Plane Detection recognizes flat surfaces such as the floor, table, or wall. This allows an application to decide where a digital object can be placed in a meaningful way.
A typical example is a AR product visualizationA piece of furniture or a machine should not float somewhere in the camera image, but should be properly placed on the floor.
At Hit Test A point or beam is sent from the screen or camera perspective into the recognized environment. If it hits a recognized area or depth information, an anchor or a virtual object can be positioned there.
Modern ARCore systems support horizontal and vertical surfaces. Additionally, Depth Tracking can improve placement when a classic plane detection alone is not sufficient.
4. Depth Tracking, Scene Understanding and Occlusion
An important development compared to previous AR applications is the use of Deep information. For example, ARCore can generate depth maps from camera motion and, if available, incorporate additional hardware depth sensors.
Depth Maps help to better estimate distances to the real environment. This results in more precise hit tests, more realistic collisions, and especially Occlusion Possible: A real object can properly cover up a virtual object.
For high-quality AR and mixed reality applications, this spatial understanding is often more important than simply detecting a ground surface.
5. Marker tracking and image tracking
Marker Tracking and Image Tracking both use visual references, but have different approaches.
Marker Tracking
At Marker Tracking The camera recognizes a deliberately designed, clearly identifiable pattern. Such markers provide a robust reference for position and orientation and are particularly suitable for controlled applications, installations, training or industrial scenarios.
A QR code is not automatically an AR marker: A QR code is often only used to launch a URL or a web AR application. A true tracking marker, on the other hand, serves as a spatial reference for the pose of the digital content.
Image Tracking
At Image Tracking The camera, on the other hand, recognizes a natural image – for example, a poster, packaging, graphics, or a print motif – and uses this as a spatial reference.
The image must not contain any artificial markers. What matters is sufficient characteristic features, contrasts, and details that can be reliably recognized.
Image tracking is particularly suitable for AR Marketing, Packaging, print, museums and exhibition scenarios, because the starting point of the experience is clearly defined.
What makes a good image target?
Images that have many clearly distinguishable details, contrasts, and features distributed over the surface work well. Large monochromatic areas, strong repeating patterns, or very simple logos are often less trackable.
Therefore, before production, an image target should be tested within the framework used and not just evaluated in terms of design.
6. Geospatial Tracking: GPS alone often isn't enough
For outdoor AR, GPS remains important, but GPS alone is often too inaccurate for precise spatial positioning. Modern geospatial systems therefore combine multiple sources.
The ARCore Geospatial API For example, it uses device sensors and GPS together with Google's Visual Positioning System (VPS). VPS matches visual features of the environment with a localization model based on Street View data. This allows AR content to be anchored much more precisely in real locations than with traditional GPS coordinates alone.
Such systems are suitable for navigation, tourism, urban space, large outdoor areas, and location-based experiences.
7. Face, Hand, Body and Object Tracking
Not all AR tracking tracks the device's position. Depending on the application, faces, hands, bodies, or defined real objects are also captured.
Face Tracking For example, it is the basis of many camera effects and virtual rehearsals. Hand Tracking allows for natural interaction with virtual content. Body Tracking can transfer body movements to avatars or digital elements.
Object Tracking and Object Recognition
At Object Tracking In object recognition, a real three-dimensional object serves as the reference. This can be a product, a component, or a machine. This allows digital information to be aligned with defined areas of the real object.
The difference from image tracking is important: In image tracking, a two-dimensional image reference is recognized. In object tracking, it is the shape or characteristics of a real 3D object that are considered, and often also its spatial position.
These methods are technically part of advanced AR tracking, but serve a different purpose than World Tracking or SLAM.
Native AR and WebAR: not every tracking function is available everywhere
An important difference exists between native apps and WebAR.
Native frameworks like ARCore and ARKit generally have deeper access to cameras, sensors, Depth APIs, Anchors, and platform-specific features. In the browser, tracking quality depends more on the device, browser, WebXR support, and the webAR framework used.
WebXR specifies things like hit testing and anchors. However, advanced features like plane detection, depth, or persistent spatial data are not available on all platforms and browsers in the same way. Therefore, it is advisable to first check whether a project is supported on the following platforms and browsers:, Which tracking function is really needed, Before deciding on WebAR or a native app.
Which AR tracking method is suitable for which project?
| Application | Typical tracking basis |
|---|---|
| Place product in the room | World Tracking + Plane Detection + Depth |
| Print / Packaging / Posters | Image Tracking / Marker Tracking |
| Outdoor / City / Tourism | Geospatial Tracking + GPS + VPS |
| Virtual try-on | Face, hand or body tracking |
| Industry / Machine | SLAM / World Tracking + Image/Object Anchors |
| Mixed-Reality Training | 6DoF + Scene Understanding + Anchors |
Where AR Tracking is used in enterprise projects
Tracking is technical infrastructure. The actual added value only comes from the respective use case:
- Product presentation: Visualize products and machines in the real world.
- AR Marketing: Print, packaging, campaigns and interactive brand communication.
- AR Training: Present workflows and information directly in the real context.
- Industry and mechanical engineering: spatial information, technical visualization and assistance.
For the technical implementation of an individual app, our page is the right place. Augmented Reality Development the appropriate depth.
Typical causes of poor AR tracking
Even in 2026, tracking is not perfect under all conditions. Common problems are:
- insufficient light or strong motion blur,
- large unstructured areas without recognizable features,
- Mirrors, glass and highly reflective surfaces,
- fast camera movements,
- too few or unsuitable characteristics in image targets,
- GPS without additional visual localization,
- missing depth or occlusion information,
- High requirements on browsers or end devices in WebAR.
Therefore, a good AR application is tested not only in the studio but also in the actual lighting, space, and device conditions of the later deployment.
Conclusion: AR Tracking is a combination of several methods
The question „What tracking technology does AR use?“ can no longer be answered with a single method today. Modern systems combine world tracking, VIO, SLAM, plane detection, depth, anchors, and – depending on the application – image tracking, geospatial tracking, or body tracking.
For project planning, it is crucial to determine what spatial information is really needed. A packaging campaign requires different tracking than a machine visualization or outdoor navigation.
If you are planning a specific AR project, you can find it under Augmented Reality Agency an overview of our services.
Technical sources
FAQ: AR Tracking
AR Tracking refers to the ongoing determination of position, orientation, and relevant characteristics of the real environment. This allows virtual content to be stably aligned with real positions.
SLAM or related mapping and localization methods help the device determine its position in an environment. Plane Detection, on the other hand, detects specific flat surfaces such as the ground, table, or wall on which content can be placed.
With image tracking, the camera recognizes a previously defined image and uses it as a spatial reference. This is useful, for example, for packaging, posters, print materials, and exhibitions.
For rough localization, GPS can suffice. For more precise AR positioning, additional methods are often supplemented. Google’s ARCore Geospatial API, for example, combines GPS and device sensors with the Visual Positioning System (VPS).
Drift can arise, among other things, from poor lighting, few visual features, motion blur, reflections, or insufficient depth information. Good anchors and a suitable tracking method reduce this effect.
Not always. WebAR can cover many typical AR scenarios, but is more dependent on the browser, device, and framework. Native ARCore or ARKit apps often have deeper access to tracking, depth, and platform features.
For products placed freely in the room, World Tracking, Plane Detection, and, if applicable, Depth are typically used. If a specific print motif or product is intended to serve as a starting point, Image Tracking may be more suitable.
Marker Tracking uses deliberately designed, clearly recognizable patterns as spatial reference. Image Tracking, on the other hand, can recognize natural images such as posters, packaging, or graphics. Both methods can anchor digital content to a visual reference.
In object tracking or object recognition, a real three-dimensional object such as a product, component, or machine is recognized. Digital information can then be aligned with specific areas of this object.
