Spatial AI API

Give “understanding” a bounded meaning

Start with the decision the application needs to make. A decorative region suggestion, a placement aid, and a verified object identity are different goals. Do not use the broad promise of spatial intelligence to blur their evidence requirements.

The scene-understanding guide separates geometry, semantic predictions, and application decisions. It includes a documented vendor example with a limited scope, then develops an application-level workflow for uncertainty, evaluation, and correction.

Keep context with every observation

For your own data contract, consider the source, timestamp, coordinate-space identifier, predicted category, confidence representation, and validity state. Document what these fields mean rather than presenting them as a universal API format.

A missing signal is not the same as an observed absence. A retained user choice is not the same as a current model prediction. Preserving these distinctions gives the application a useful response when observations are stale, incomplete, or in disagreement.

Make abstention useful

An application needs a way to decline an unsupported action without making the whole experience unusable. That may mean showing a suggestion rather than a commitment, asking for confirmation, or keeping an ordinary content view available.

Evaluate errors according to their consequences for the actual task. Separate false suggestions, missed opportunities, unstable transitions, and stale overlays. A single aggregate score can hide a problem that makes the product difficult to understand.

Connect intelligence to responsible interaction

Keep data collection proportionate to the task. Identify what is processed, what leaves the device, and what is retained, then verify those statements against the deployed implementation before making privacy claims.

Read the depth guide for the geometric side of scene integration and Accessible XR for interaction flexibility. A spatial feature is more useful when people can understand its limits, correct its suggestions, and complete the task through an appropriate alternative path.