A virtual object can be perfectly positioned and still look wrong. When it draws over a real chair that should be in front of it, the illusion breaks. Depth and occlusion address that visual relationship, but adding them is not simply a matter of turning on a realism switch.
A useful implementation must handle missing information, uncertain boundaries, and the interaction between visual effects and task clarity. This guide proposes a development workflow for those decisions. Start with the spatial AR API hub and keep the spatial AI hub nearby when considering the difference between geometry and semantic understanding.
Distinguish depth from occlusion
The ARCore Depth API overview describes depth images as information about the size and shape of real objects and discusses uses including occlusion and interaction. Occlusion is an application of that information: virtual content is rendered so nearer real surfaces can hide parts of it.
For design purposes, keep the evidence separate from the effect. A depth estimate is input. The visual decision about whether to hide a virtual fragment is output. That distinction gives you somewhere to inspect errors instead of treating the final image as an unexplained result.
Do not infer object meaning from distance alone. A nearer region may affect rendering without telling your application whether it is a chair, a person, or a temporary obstruction. Semantic labels and geometric evidence answer different questions.
Begin with a scene that exposes mistakes
Choose a simple virtual object with a clear silhouette and place it near a real object with an obvious boundary. Avoid starting with transparent materials, particles, or complex shadows. Those effects can make it hard to determine whether a visual problem comes from depth handling or from the material itself.
Create a development mode that alternates between the normal view, a depth visualization, and a no-occlusion view. The comparison should be easy to trigger during testing. It helps a developer see whether an object vanished because of the mask or because it was never rendered correctly.
Keep the test asset and placement repeatable. A controlled fixture does not replace real-world trials, but it gives you a baseline when a renderer, content asset, or preprocessing step changes.
Align the data before tuning the effect
Check that the depth information and the rendered view refer to the intended perspective, timing, and coordinates. Write down the transformations involved. A small alignment error can look like a bad threshold even when the actual problem is that the two inputs do not correspond.
Use a diagnostic overlay with simple edges or a conspicuous boundary. Move the viewpoint and observe whether the relationship stays consistent. Test both a stationary view and motion. The goal is not to produce a flattering screenshot; it is to expose disagreement between the inputs.
Do not compensate for a structural mismatch by applying increasingly complicated smoothing. First verify the conventions and the path from source data to the renderer. Tuning is easier when the underlying comparison is meaningful.
Define behavior for missing information
A missing or unusable depth sample needs an explicit policy. Your application might render the virtual content without environmental occlusion, fade a preview, or temporarily suspend a precision-dependent action. The right choice depends on the task, not on a universal rule.
For a decorative scene, a graceful visual fallback may be sufficient. For an instruction that relies on identifying a specific physical relationship, continuing with an apparently precise overlay could be misleading. Separate these cases in the design document.
Make the fallback consistent. A user should not see an unexplained sequence of disappearing and reappearing content as individual samples fluctuate. At the same time, avoid hiding uncertainty behind a stable-looking image that suggests more reliable knowledge than the system has.
Treat boundary quality as a product decision
Most attention naturally goes to the edges where virtual and real content meet. Test thin objects, moving obstructions, and partial overlap as distinct cases. Record what a tester notices and whether it interferes with the task.
A slightly imperfect decorative boundary may be acceptable in a playful scene. The same defect may make a technical annotation confusing. Define an acceptance criterion in terms of readability and task completion, not simply whether a screenshot looks impressive.
When evaluating visual filtering, compare several controlled cases rather than tuning to one favorite view. A change that improves one edge can create lag or loss of detail elsewhere. Keep before-and-after captures linked to the exact content and settings used.
Preserve the readability of important content
Not every interface element should be treated like a physical object inside the scene. A critical instruction may need a stable presentation layer or an alternative text location so the user can still understand the task when scene content is obscured.
Distinguish world-attached labels from application controls. Then decide which may be hidden by the environment, which should remain available, and how the user can recover a lost label. These rules should be deliberate and visible in the interaction design.
For example, a virtual exhibit may use realistic occlusion for a decorative model while keeping a clearly separate instruction panel available. That separation avoids forcing the user to hunt for basic controls behind real-world objects.
Measure the whole rendering change
Introducing depth handling changes more than the final appearance. Evaluate the full experience after adding the feature: loading, preparation, rendering, interaction, and sustained use on representative hardware. Do not assume an effect is free because its code is short.
Build quality tiers that preserve the task. A simpler mask or a non-occluded fallback may be preferable to an experience that becomes difficult to interact with. Keep the selection of a tier explicit so a tester can identify which path was active.
Record measurements with scene and device context. Avoid publishing universal performance claims based on a single scene. Your purpose during development is to find a reliable operating envelope for the application you actually intend to ship.
Use a room-scale test script
Write a repeatable sequence: acquire the scene, place the test object, move around it, introduce an obstruction, remove it, interrupt the experience, and return. Keep observations about geometry, visual stability, and instructions in separate columns.
Then vary one environmental condition at a time where practical. This makes a failure easier to interpret. If the camera path, lighting, object arrangement, and asset all change together, an apparent improvement tells you very little.
Do not describe the result as a safety assessment. A visual depth effect is not evidence that an application can reliably identify hazards or guide a person through an environment. Keep the feature's purpose bounded to the tested visual and interaction task.
Questions to resolve before release
Does depth automatically recognize objects?
No. Distance-related information and semantic recognition are separate inputs. A visual occlusion effect can be useful without naming the obstructing object. When object categories matter, evaluate a suitable scene-understanding approach separately and account for its uncertainty.
Should occlusion always be enabled?
Use it when it improves the intended experience and can be supported with a meaningful fallback. A complex effect that hides essential information or creates unstable transitions may make the task worse. Compare the complete user journey with and without it rather than treating realism as the only goal.
Conclusion: make depth useful, not merely impressive
Depth becomes valuable when its evidence is aligned, its uncertainty is handled, and its visual consequences support the task. Start with a controlled fixture, define missing-data behavior, and protect important instructions from disappearing without explanation.
The next step is the spatial AI scene-understanding guide. It explores how semantic predictions can complement geometry while preserving the distinction between what the system measures, what it infers, and what the application is justified in doing.
