Who with Gaussian Splatting When scanning real spaces, vehicles, or architecture, one quickly notices: not every surface behaves the same. Matte walls, seats, floors, or furniture can often be captured surprisingly well. It becomes more difficult where reflections come into play. It is precisely at this point that a topic begins, which initially seems almost a little mysterious: the apparent „world behind the mirror.“.
What at first looks like a curious mistake is in truth a direct consequence of how such Scans arise. The scanner cannot properly distinguish between real geometry and reflection. And this is precisely what causes the typical problems that you see over and over again with mirrors, windows, shiny floors, and other reflective surfaces.
Why reflections are so difficult with Gaussian Splatting
When working with LiDAR-based devices like the PortalCam or the K1, we don't just capture a photo of a scene. Instead, a spatial model is created from depth information and image data. This works very well with clear, physical geometry. However, it becomes complicated with reflective surfaces.
The scanner cannot clearly detect a reflection. It treats the reflected information as if geometry were actually present, thereby creating a spatial mirror world.
Why the scanner is behaving like this
Reflective and transparent surfaces are difficult for LiDAR because the signal does not return as cleanly as from matte surfaces. This can lead to missing measurements, distorted depth information, or spatially incorrect structures. The fact that the „mirror world“ was already visible in the LiDAR preview in our case suggests that the effect begins at the point cloud level.
The Mirror World is not a random error
This mirrored world is not just an artifact. It is precisely the mechanism that ensures a reflection appears credible from the right perspective.
The problem is not that Gaussian Splatting fundamentally fails with mirrors. The problem is that the logic by which the reflection appears credible can break down elsewhere.
When the illusion works – and when it fails
As long as one moves from the perspective from which this mirrored world appears meaningful, the result can be surprisingly convincing. The mirror or the window then seems plausible because the system has actually built a kind of reflected counter-space.
It becomes difficult when there is actually no physical space for this mirror world or when you look at the scene from a completely different side. Then it becomes apparent that what looked like a correct reflection on one side has actually been pushed outwards spatially.
The example from the video: the bus scan
In my video, I demonstrate this effect on a bus that I scanned from the inside. Viewed from the interior, the reflections in the windows initially appear plausible. However, as soon as you move to the exterior, the illusion breaks. The reflected elements of the interior seem to float spatially in front of the glass – as if the interior of the bus is being pushed outward through the windows.
This isn't a minor beauty problem, but a structural one. From this perspective, the scene is essentially unusable.
Two ways to deal with the problem
When one accepts that reflective surfaces in Gaussian Splatting represent a category of challenge in itself, one quickly arrives at two practical approaches.
Why artifacts can't just be cut away: This exact geometry is often the basis for the reflection appearing credible from the inside. If you remove it on the outside, you simultaneously destroy the effect on the other side.
Solution 1: Transitions between multiple scans
The first approach is to not expect a single scan to perfectly solve both the interior and exterior simultaneously. Instead, you work with multiple states and blend between them – one version for the interior, one for the exterior.
This can be an elegant solution, especially in interactive applications. The user doesn't perceive that a switch between two states is happening in the background, but simply experiences a coherent scene.
Solution 2: Fake Reflections in the Editor
The second approach involves artificially adding or replacing problematic reflections later. Instead of hoping that windows or mirrors are perfectly reconstructed in the raw scan, one rebuilds the reflection deliberately in the editor – using reflection maps, fake mirrors, or similar dummy constructions.
Just in PlayCanvas is something like this feasible. Also tools like StorySplat offer no-code approaches for this.
Conclusion: Behind the mirror lies a second world.
The „world behind the mirror“ is not a mystical special case, but a logical consequence of how Gaussian Splatting deals with reflective information. The scanner cannot cleanly distinguish whether it sees real geometry or a reflection.
Those who understand this principle can make decisions much more precisely: when a scan is sufficient on its own, when to work with multiple states, and when artificial reflections are the better solution.
Frequently Asked Questions (FAQ)
The scanner cannot distinguish between real geometry and a reflection. It treats the reflected information as if space actually existed there – thereby creating a spatial mirror world behind the surface.
No. Exactly this mechanism ensures that a reflection even looks believable from the right perspective. It only becomes problematic when the mirror world visibly protrudes into the walkable area.
On reflective and transparent surfaces, the signal does not return cleanly. The result is missing measured values, distorted depth information, and structures that appear spatially incorrect—often visible right in the point cloud, long before the finished splat.
Matte surfaces can be captured very reliably – walls, floors, seats, furniture. It gets difficult with mirrors, windows, shiny floors, chrome, and glass.
Intentionally plan for reflective surfaces: capture from multiple angles, clean up critical areas in post-processing, and cover mirrors or glass during the scan wherever possible.
