The challenges in Gaussian Splatting do not begin with the reconstruction. They start already at the camera, the recording planning, the coverage and the question of how freely the user can move through the scene later. This is exactly where a quick demo scan separates from a professionally produced 3DGS project.
Gaussian Splatting Today it is surprisingly easy to access. With a smartphone, DSLR, 360° camera, drone, or specialized 3D scanners, real places can be transformed into photorealistic 3D scenes. Creating a splat is now relatively easy. Creating a really good splatter is something else.
In our projects, we constantly encounter the same problems: blurriness, floaters, moving people, missing upper parts, incomplete coverage, huge amounts of data, and the question of how to deliver the finished scan effectively on the web or in XR.
This article summarizes the most important problems and shows which countermeasures have proven effective in practice.
The key challenges in Gaussian Splatting
1. Gaussian splats appear slightly blurred quickly
One of the most common weaknesses of Gaussian Splatting is a certain softness in the image. From a distance of a few meters, a scan can appear extremely photorealistic. However, as you move very close to surfaces, fonts, fine material structures, or small edges quickly become softer.
This is not solely due to the 3DGS process. The quality achieved starts already during capture. What the camera did not capture clearly can not be recreated later simply by reconstructing it.
Among other things, objective quality, sensor resolution, focus, motion blur, image noise, shooting distance, overlap, and the number of usable viewing angles are crucial factors.
Especially with mobile 3D scanners, a constructive compromise is created here. A scanner combines cameras, LiDAR, processors, memory, and power supply into a mobile device. The integrated cameras therefore do not automatically achieve the image quality of a high-quality system or DSLR camera.
HD Enhancement as a Quality Lever
With XGRIDS, this problem can be solved, among other things, via HD Enhancement in Lixel CyberColor Studio This involves incorporating additional high-resolution images from an external camera into the reconstruction.
These can be shots taken with a DSLR, a system camera, or a modern smartphone. For projects where visual quality is more important than pure capture speed, this hybrid workflow is particularly interesting: the scanner provides spatial stability and coverage, while the additional camera provides finer image details.
2. Large scenes generate very quickly very large amounts of data
A bathroom and a kilometer-long outdoor area are two completely different Gaussian-Splatting projects.
The larger the area covered, the more spatial and visual information must be stored. At the same time, with freely navigable scenes, it is not always clear how close a user will later approach a specific object.
A tree that looks excellent from 50 meters away requires relatively few visible details. However, if the user can get to the bark within a few centimeters, the requirement changes completely.
This creates a fundamental dilemma: The greater the freedom of movement of the audience, the more detail would theoretically have to be kept in place everywhere.
With large environments, this is not scalable in an arbitrary way. Therefore, one has to work with levels of detail, streaming, spatial prioritization, and sometimes conscious quality adjustments.
3. Floaters: When suddenly something floats in the air
The most typical 3DGS artifacts include so-called Floaters. In this case, Gaussians appear at positions where there was no object in the real world at all.
These can be blurred halos, small areas or freely floating structures. Causes include, for example, reflections, lens flares, motion blur, transparent materials, repetitive structures, or simply incorrect spatial associations.
In purely image-based workflows, the spatial structure must be reconstructed entirely from the images. A LiDAR-based scanner, on the other hand, additionally has a geometric reference as a point cloud. This does not completely prevent artifacts, but it can significantly stabilize the reconstruction.
Those who would like to delve deeper into the differences between visual reconstruction and geometric reference can find our comparison to that effect. Gaussian Splatting vs. Photogrammetry.
4. Humans, vehicles, and other changes cause ghosting
Classical 3D Gaussian Splatting ideally starts from a largely static scene. Moving people, cars, bicycles, or animals contradict this assumption.
If a person walks through the scene during the capture, they will be located in different positions in the different images. The reconstruction then attempts to reconcile this contradictory information.
The result can be ghosts, semi-transparent people, double contours, shadow forms, or blurry areas.
The problem doesn’t even have to arise quickly. A sunshade can be opened on the first pass and closed ten minutes later. If both conditions are recorded in the same dataset, an unsightly hybrid can result.
XGRIDS also points out in its own capture instructions that slowly moving or standing people can be reconstructed incorrectly. For clean results, a scene should therefore change as little as possible during the capture.
5. 360°-Capture has an additional problem: The operator is in the image
360° cameras have an enormous advantage: they capture almost all directions at the same time. However, this also has a disadvantage.
In a true 360° shot, there is practically no „behind the camera“ element. The operator is therefore almost inevitably present in the image.
The same applies to the tripod, extension pole, and some other parts of the capture setup. In a custom 360° workflow, these areas must be masked out or removed from the image before training.
In the XGRIDS workflow, masks are created for reconstruction, which, among other things, mark people and capture devices as unsuitable image areas. The current LCC Studio documentation even lists these masks separately in the Creator data export.
This step is not limited to the XGRIDS workflow today. For example, the 360° video material is prepared by 360 Video – Still Prep Tool The image preprocessing and masking capabilities allow the operator to mask selected objects with AI. According to the provider, it combines SAM2 with RT-DETR and recognizes more than 30 object classes. Another option is 360 Gaussian ProThe software provides SAM3 keyword masks, for example for people and vehicles, as well as a brush for areas that are not automatically recognized. The masks ensure that these image areas are excluded during alignment or training; they do not automatically reconstruct the hidden background. Therefore, the masks should be tested before training and corrected manually if necessary.
6. Mirrors, water, glass and monotonous surfaces need more planning
Reflective surfaces are particularly demanding for visual reconstruction: Their visible pattern changes with viewing angle and light. Mirrors, glass, water, and shiny metals therefore do not provide stable image characteristics like a matte, textured surface. However, they are not automatically unusable. What is crucial is to incorporate them purposefully and with significantly denser coverage of angles.
Spiegel: Capture the mirror from many different positions and angles, including slightly from the side and from different heights. This way, the reconstruction process gets more overlapping views instead of a single frontal mirror shot. Be careful not to get yourself in the mirror: Hold the scanner with a long extension pole as far forward as possible or extend your arm as far as possible. Check the preview regularly, as the operator, the scanner, and the pole can show up in the reflection.
Reflecting or calm water: Capture the shoreline and water areas from multiple overlapping positions, with enough angles of view and consistent movement. In water, waves, wind, and changing reflections change the appearance between shots; as much as possible quiet conditions and a consistent light pattern help. If the water surface itself is important for the later free camera movement, plan additional shots from shallower and steeper angles. A perfect, physically stable reflection cannot, however, guarantee it for every desired view.
Monochrome or very bright matte surfaces: A large white wall offers the alignment hardly recognizable features. Capture them with plenty of overlap and as uniform illumination as possible. Temporary, contrast-rich reference markers or structured objects near by can help in alignment; if they are not intended to remain visible in the result, they must be removed subsequently with masks or in post-processing. LiDAR or other geometric data can support the shape and position of the area, but do not create missing surface structure.
Similar caution is advisable when it comes to glass, shiny metals, regular facades, tiles, grass and fine wire mesh: More angles of view and narrower, overlapping shots reduce misidentifications. For the special case of reflections, we have a separate article: Reflections in Gaussian Splatting.
7. Backlighting is especially problematic with scanners without HDR
How much the ambient light affects a scan depends largely on whether the capture system supports HDR images. It is particularly critical for cameras and scanners without HDR: The difference in brightness between a brightly lit window and the interior can exceed the sensor's dynamic range.
When an automatically-exposed camera is pointed at the bright window, it reduces the exposure. The interior becomes dark. When the camera turns away from the window again, it increases the exposure again.
This results in images of the same wall with visibly different brightness levels. Conversely, an exposure adjusted to the interior can cause windows to burn completely white.
In the outdoors, similar things happen with objects in front of a bright sky. A tree can become almost black against the sky, even though it looks correct from another direction.
With DSLR and system cameras, manual exposure, RAW, HDR, or exposure series help to better control the contrast. If a scanner has HDR, this function should be used for such scenes. If HDR is missing, avoid strong backlighting as much as possible, choose more uniform lighting conditions, and keep the exposure constant throughout the entire process, provided the device allows for this setting.
8. Smartphone scanning is impressive – but it doesn't scale as well as it could
Smartphone apps have made getting started with Gaussian Splatting extremely easy. With Scaniverse or the new XGRIDS app LCC Scan An iPhone can actually be used today as a 3D capture device.
This is very attractive for small objects, individual rooms, quick tests, and additional detailed shots.
However, with large environments, this approach quickly reaches practical limits. The problem is not only the camera quality; modern smartphones have very good cameras and ultra-wide-angle lenses.
The decisive factor is the Capture efficiency per square meter.
With a room, you can systematically walk around the objects and ensure sufficient overlap. With a large park, a long street stretch, or a kilometer-long area, the same approach quickly becomes impractical.
Niantic currently limits a single site scan in Scaniverse to up to five minutes or 500 square meters. Larger sites are built from multiple overlapping scans. This very clearly shows that large-scale capture requires a different workflow than a single smartphone scan.
The smartphone remains valuable nevertheless: especially as an additional camera for detail areas or as an additional image source for HD enhancement.
9. The right capture system depends on the project
The question „Which camera is the best for Gaussian Splatting?“ cannot be answered in a general way.
| Capture system | Strengths | Typical boundaries |
|---|---|---|
| Smartphone | Very available, fast, good cameras | Large-scale capture is time-consuming, coverage must be highly self-controlled |
| DSLR / System camera | Highest image quality, full control over optics and exposure | Slower capture, spatial coverage must be planned |
| 360° camera | Very high coverage per recording | Lower pixel density per viewing direction, operator in the image |
| LiDAR scanners | Geometric reference, coverage feedback, efficient in large rooms | Expensive, integrated cameras do not always reach DSLR levels |
| Drone | Roofs, upper sides and large outdoor areas | no need for Ground Capture, flight planning, and alignment |
The XGRIDS PortalCam For example, it is strongly focused on visual 3DGS production. It combines four cameras with LiDAR. According to XGRIDS, the two fisheye cameras each have a 200° × 200° field of view, while the LiDAR has a 180° × 180° field of view.
The Lixel K2 and the more recent Lixel L3 follow a different focus and combine visual reconstruction more closely with surveying and geometric data.
10. Coverage is more important than „I was already there“
One of the most common misperceptions when scanning is: „I already covered this area.“
However, what is crucial is not only whether an area was present in the image at some point; it must have been captured from suitable perspectives and with sufficient overlap.
In particular, niches, corners, under-sides, tops, areas behind furniture, transitions between rooms and surfaces that are only visible at very shallow angles are problematic.
In professional scanning systems, the software helps to assess the areas already captured and the density of the dot cloud. When photographing, filming, or scanning with a smartphone, the operator must plan the coverage much more independently.
For large or complex scenes, XGRIDS also recommends looping through the scan path and multiple viewpoints so that parts of the scene can be connected robustly with each other.
11. The roof problem: You can't see everything from the ground
Coverage has a second dimension: height.
Anyone who scans exclusively from the ground level mostly captures the environment at eye level. This works well – until the later user uses their virtual camera to climb up.
A classic example is a car. At eye level, doors, windows, and the bonnet are well captured by the scanner. However, the scanner only sees the car roof at a very low angle or not at all.
From a normal running perspective, this hardly noticeable. As soon as the virtual camera moves over the car, the roof can suddenly appear blurry, cracked, or heavily affected by artifacts.
The same applies to stretched sunshades, wall crowns, containers, machines and other objects whose upper part is covered by the ground.
Why an extension pole is worth it
For such situations, an extension pole is one of the most useful accessories available. The PortalCam, for example, comes with an optional 2-meter extension rod offered.
This allows the scanner to be lifted over cars, fences, machines, or other obstacles. The top sides are captured from significantly better angles and are actually available to the later free camera.
At first glance, this seems quite simple – basically, it’s like a very sturdy selfie stick. In practice, however, the additional height can determine whether a scene remains convincing even from unusual perspectives.
12. At some point, only a drone can help with building roofs anymore
A two-meter-long pole helps with the car roof. In a multi-story building, of course, it is no longer sufficient.
Therefore, for large outdoor areas, the combination of Ground Capture and Drone Capture especially useful.
The ground scanner provides images of facades, entrances, paths, underpasses and areas under trees or overhanging roofs. The drone provides images of roofs, upper floors, courtyards and the overall spatial context.
XGRIDS provides the workflow in LCC Studio for this purpose. Aerial-Ground Fusion. In this process, ground data from compatible L-, K-, and PortalCam devices is combined with drone photos. The current XGRIDS documentation supports up to ten ground data segments and up to 10,000 drone images in the desktop workflow.
The important thing is that this automated workflow is tied to XGRIDS floor data. It is not a universal „photo editor“ where you can throw any smartphone, DSLR, or 360° photo as a ground data set.
13. Own ground-/drone workflows need real alignment expertise
Of course, ground-based and drone materials can also be combined outside the XGRIDS ecosystem.
For example, you can work with a 360° camera or DSLR on the ground and capture the same terrain with a drone as well. The challenge then is AlignmentBoth image groups must be reliably located in the same spatial context.
Tools like RealityScan 2 or Agisoft Metashape come into play here. They allow significantly deeper interventions in feature matching, camera poses, control points, and problematic cameras than a pure one-click workflow.
Postshot can calculate Camera Poses itself and is ideal for many common datasets. However, for difficult materials, it may be useful to solve the alignment externally beforehand. Jawset explicitly documents the import of Camera Poses and Sparse Point Clouds from RealityScan into Postshot.
A typical professional workflow can therefore look like this:
Ground and drone images → RealityScan / Metashape → Alignment and Camera Poses → Postshot → Gaussian-Splatting Training.
That's powerful, but it's not a one-click solution. Anyone who wants to go this route should really master the alignment tools.
An overview of other software can be found in our article. Gaussian Splatting Workflow: Tools for Web, Unity, Unreal & 4DGS.
14. Large splats require correspondingly large computers
After the capture, the data must be processed. And here too, the workload increases with the size of the dataset.
Relevant factors are GPU performance, VRAM, memory, SSD speed, the number of images, the number of Gaussians, and the size of the scene.
GPU-based training and reconstruction methods in particular can require a lot of graphics memory. An RTX 3090 with 24 GB VRAM is still a very usable workstation card for many 3DGS workflows – but it is not a universal minimum requirement.
Small projects work with less hardware. Very large datasets, in turn, can require significantly higher requirements. XGRIDS recommends correspondingly more RAM and fast SSD storage for complex fusion workflows.
The crucial rule is: With a large scan, it should be clear before capturing where and how the data will be processed afterwards.
15. The finished scan also needs to reach the user
A beautiful Gaussian Splat can be several gigabytes in size. Without that, you won't have a usable web experience for a long time.
The classic PLY format remains very suitable as a working, exchange and archive format. For distribution over the web, it is often too cumbersome for large scenes.
PlayCanvas has partnered with SOG – Spatially Ordered Gaussians an optimized format for runtime and web delivery has been established. According to PlayCanvas, SOG is typically about 15 to 20 times smaller as a comparable PLY.
For very large scenes, Streamed SOG Even further. The scene is broken down spatially into chunks and several levels of detail. The viewer only loads the parts and detail levels that are actually needed for the current camera position.
The principle is reminiscent of Google Earth: More detail is loaded near the object, less in the distance.
This is a crucial advancement especially for web projects. You can find out more in our article. Create Gaussian Splatting Website: Integrating 3DGS into the Browser.
The most important rule: Scan for future camera freedom
From almost all of the problems mentioned, a common principle emerges:
Gaussian Splatting does not reconstruct information that was never recorded.
It sounds trivial, but during the capture it is surprisingly easy to forget.
The operator moves about on the ground and thinks: „I’ve seen everything.“ The later user may be able to fly, look over a car, look down at a sun umbrella, or fly over a building roof.
With this, the virtual camera suddenly has freedoms that the real camera never had during the capture.
That is exactly why the planning of recordings should always be based on the later use:
- Where can the user move?
- How close can he approach objects?
- Should he leave his equal status?
- Are the roofs and upper sides visible?
- Will the scene be delivered in the browser, on a desktop PC, or in XR?
- What file size and streaming bandwidth are realistic?
Only from these questions does the right combination of scanner, smartphone, camera, 360° capture, extension pole, drone, and processing workflow emerge.
Conclusion: Good Gaussian splats are created before the reconstruct button.
Many so-called „Gaussian splattering problems“ are not weaknesses of the rendering process itself.
They arise earlier: with the camera, the exposure, the recording path, a missing roof surface, a person in the image, a poorly planned alignment, or too much data for the later target device.
Our experience is therefore: The quality of the entire workflow is more important than actually pressing the reconstruct button.
Anyone who already knows before recording which device, in what quality and with what freedom of movement the scene is to be shown later can better coordinate the Capture method, coverage, detail depth and data volume.
And that is exactly what turns an interesting scan into a truly useful medium.
FAQ: Challenges with Gaussian Splatting
Why is my Gaussian Splat blurry?
The cause often lies in the source material itself. Blurred images, motion blur, long focal lengths or too few perspectives reduce the achievable level of detail. Additionally, reconstruction parameters, compression and level-of-detail systems can affect perceived sharpness.
What are Floaters in Gaussian Splatting?
Floaters are Gaussians that appear in spatially false positions and become visible, for example, as floating halos or blurred structures. They can arise from reflections, lens flares, missing perspectives, or incorrect image alignments.
Does LiDAR completely prevent floaters?
No. However, LiDAR provides an additional geometric depth reference and can therefore reduce spatial misinterpretations. Also, LiDAR-based workflows can still contain artifacts.
Can Gaussian Splatting be created with just a smartphone?
No. Smartphones are only one possible capture method. Depending on the project, DSLR cameras, system cameras, 360-degree cameras, drones, or specialized LiDAR scanners may be more useful. For large environments, professional or hybrid capture workflows are usually more efficient.
Why is an extension pole sometimes needed when scanning?
The upper sides of cars, awnings, walls or machinery are often obscured from normal eye level. An extension pole allows the scanner to pass through these obstacles and thus improves vertical coverage.
How to capture building roofs for Gaussian Splatting?
In higher-rise buildings, Ground Capture usually isn’t enough. A drone can provide the missing perspectives from above. XGRIDS Aerial-Ground Fusion combines compatible ground scans with drone photos in a unified workflow.
Why are people and cars problematic when scanning?
Static Gaussian Splatting assumes that the scene will change as little as possible during the capture. Moving objects can therefore produce ghosting, double images, or blurred areas.
Does the operator need to be removed from a 360° camera?
In 360° shots, operators, tripods or poles often need to be masked so that they do not interfere with the final result. 360 Video – Still Prep Tool offers AI masks for the operator and selected object classes. 360 Gaussian Pro SAM3 masks and offers manual correction. The masks exclude image areas from alignment or training; they do not automatically fill in hidden backgrounds.
What is SOG?
SOG stands for Spatially Ordered Gaussians. The format developed by PlayCanvas is optimized for runtime and web delivery and is typically about 15 to 20 times smaller than a comparable PLY.
How do you bring very large Gaussian splats to the web?
For large scenes, the entire scan should not be loaded at once. Streamed SOG divides a scene into chunks and detail levels and only loads the data needed at the moment, depending on the camera position.
