View-Aware Mesh Optimization

General-purpose simplification tries to preserve a model from every direction. View-aware optimization uses a known camera distribution to spend detail where the experience can reveal it. A product locked to a front hemisphere, a distant stadium background, and a WebXR object each justify different geometry.

This method can outperform uniform object-space reduction, but it creates a contract between asset and camera behavior. If the product later enables a back view, previously safe hidden-side changes become defects. Camera fixtures and assumptions must ship with the optimization report.

Define views as a distribution#

A single camera is fragile. Define possible views and their importance:

initial camera and its expected share of viewing time;
orbit azimuth and elevation limits;
zoom or distance range;
perspective field of view;
viewport sizes and device pixel ratios;
product-controlled animation of the model;
user gaze or selection regions when available without invasive tracking;
shadow, reflection, and secondary-camera views;
future AR or screenshot modes.

Represent these as weighted samples or a continuous probability model. If 70% of sessions remain near the front, 20% inspect sides, and 10% reach the back, a weighted optimizer can favor the front while retaining a hard error ceiling everywhere allowed.

Do not assign zero weight to an allowed view unless failure there is acceptable. A probability-weighted average can erase a rare but important backside label.

Projected error#

For a perspective camera, a small lateral displacement d at depth z projects approximately to:

pixels = focalLengthPixels × d / z

At a 1080-pixel viewport with 50-degree vertical field of view, focal length is about 1158 pixels. A 1-millimeter change at 1 meter projects to roughly 1.16 pixels. At 5 meters it is about 0.23 pixel.

This connects geometric error to visibility. An object-space tolerance can scale with depth, while silhouette and contrast still affect perception.

For each candidate collapse, estimate projected movement of affected vertices and feature lines across camera samples. Weight by view probability and screen coverage. Reject any operation that exceeds a hard maximum, even if average cost is low.

Use exact camera matrices for final evaluation. The approximation helps reason about thresholds; it does not handle arbitrary orientation or depth variation across a large object.

Front-facing, grazing, and hidden regions#

A surface facing the camera occupies more stable projected area. A grazing surface can form a silhouette and deserves high geometric accuracy even though its foreshortened area is small. A fully occluded surface may be safe to reduce under fixed conditions.

Classify contributions per view:

visible interior surface;
silhouette or near-silhouette;
occluded by the same object;
occluded by stable environment;
back-facing but shadow-casting;
visible in reflection;
visible only after animation or variant change.

Silhouette weight should be higher than simple projected-area weight. Otherwise thin contour regions receive almost no area contribution and collapse.

Self-occlusion is more stable than environment occlusion. A building hidden behind another building can become visible if level layout changes; an internal face inside a permanently sealed opaque shell has a stronger invisibility claim.

Camera sampling#

For a full orbit, distribute directions over a sphere or permitted spherical patch. A Fibonacci sphere avoids the pole clustering of a latitude-longitude grid. Add exact critical cameras regardless of distribution.

A practical fixture can contain:

32 general orbit cameras;
8 grazing cameras near critical rims;
6 closest-detail cameras;
4 initial or marketing cameras at higher weight;
4 shadow-light views;
2 reflection-probe views where required.

Increase the set until worst screen error and candidate choice stabilize. Store view and projection matrices as data. Recreating “roughly the same orbit” later invalidates comparisons.

Animated cameras need path samples. Sample more densely where view direction changes quickly or passes a thin feature. At 60 Hz, a 5-second path contains 300 frames; automated mask rendering of every frame may be reasonable.

View-weighted QEM#

Quadric error approximates object-space plane deviation. View-aware variants can weight face or error contributions based on visibility, projected area, or view direction. Another approach keeps standard QEM for candidate generation and adds a screen-space penalty or filter.

Possible cost terms include:

projected endpoint displacement;
change in triangle screen area;
contour movement;
normal change under a view-dependent highlight model;
disappearance of a visible component;
depth change near another surface;
texture-coordinate change in displayed texels.

Dynamic contour classification makes exact per-collapse evaluation expensive. Approximate with precomputed visibility and feature weights, then evaluate completed candidates through rendering.

Normalize cost across cameras. A 4K camera naturally counts more pixels than a 720p camera; that may reflect target importance or may accidentally overpower weights. Define whether error is absolute pixels, fraction of object screen size, or a perceptual function.

Hidden-surface reduction#

If some geometry is never visible, options range from stronger simplification to deletion. Deletion is appropriate only under firm opacity, camera, animation, shadow, reflection, and interaction constraints.

Before removing a region, test:

direct visibility from every allowed camera;
visibility after every animation and variant;
shadow contribution from supported lights;
reflection and refraction paths;
cutaway, exploded, or transparency modes;
collision, selection, and semantic use;
AR or developer-camera access;
future product roadmap commitments.

Mark removed source face IDs in the report. If camera policy expands, regeneration can restore them from the canonical source.

Avoid baking view assumptions destructively into the only editable asset.

Texture and shading importance#

View-aware geometry cannot ignore material. A front label may occupy few surface centimeters but many high-contrast pixels. A rough hidden underside can tolerate lower UV and normal precision.

Weight regions using:

projected texel density;
image gradient or edge energy;
material roughness and reflection sensitivity;
alpha coverage;
semantic labels;
user interaction heat maps;
normal-map frequency;
contrast against expected background.

Behavioral analytics can inform weights, but use aggregated privacy-respecting signals and keep hard quality floors. Current user behavior may reflect a UI that makes certain views difficult, not lack of interest.

Texture streaming and LOD can follow the same distribution. A front-first viewer can load front-relevant high detail first, then acquire other resources when orbit begins.

Progressive meshes and runtime selection#

View-dependent continuous LOD can choose or refine mesh regions based on current projected error. Progressive meshes encode a base mesh plus operations that reconstruct detail. Clustered systems select meshlets or patches.

Runtime refinement offers precise detail allocation but adds:

data structures and transfer bytes;
CPU or GPU selection work;
crack prevention between levels;
temporal stability requirements;
streaming and cache logic;
more complex validation;
target-specific renderer integration.

For many web assets, three or four discrete LODs are simpler and adequate. View-aware offline generation can still make each one better.

Use continuous systems for massive terrain, city models, or inspection datasets where one global LOD wastes substantial work and the renderer is controlled.

Prevent temporal instability#

If camera movement changes weights or local refinement every frame, geometry can flicker. Use hysteresis, error bands, and stable refinement histories.

For example, refine when projected error exceeds 1.2 pixels and coarsen only when it falls below 0.8 pixel. The 0.4-pixel band prevents toggling around 1.0. Add a minimum residency time when streaming cost is high.

Changes should not alter object bounds or pivot. Cross-fade or geomorph can soften transitions, but both have runtime costs. Stable topology selection is preferable.

Record triangle count and transition events along a repeatable camera path. A system switching 500 patches per second may meet static error yet create temporal noise and CPU overhead.

Foveated and gaze-aware detail#

In XR, foveated rendering reduces shading detail outside the central region. Geometry LOD can also consider gaze, but eye tracking availability, privacy, latency, and prediction matter.

Never remove safety-critical or interaction-critical geometry merely because it is peripheral. Head and eye motion can move a feature into focus quickly. Prefetch or retain enough detail to avoid visible catch-up.

A conservative design uses gaze as a bonus refinement signal while base LOD meets the normal screen-space floor. Evaluate motion-to-photon timing and transition comfort on device.

For non-XR pointer interaction, hover or selection intent can request higher detail. Keep the basic model responsive before the upgrade.

Multiview costs#

WebXR renders two eyes with slightly different views. A surface hidden in one eye can be visible in the other, especially near edges. Evaluate both view matrices. Shadow maps and mirrors add other viewpoints.

An optimization weighted only to the main color camera can break:

stereoscopic contours;
shadow silhouettes;
planar reflections;
minimaps or thumbnails;
object-ID picking cameras;
screenshots at higher resolution.

List every render consumer and decide whether it uses the same LOD or a specialized representation. A simplified shadow caster can be separate, but its mismatch must be acceptable.

Validation without overfitting#

Split cameras into optimization and holdout sets. The simplifier sees the training set; acceptance includes unseen holdout views inside the permitted envelope. This catches a candidate that exploits gaps in sampling.

One approach:

40 optimization cameras;
20 fixed holdout cameras;
10 risk cameras around thin features;
one continuous holdout path;
production initial and transition cameras.

Do not regenerate the holdout randomly for each candidate. Store it. Add newly discovered failures to the regression set without removing existing coverage.

Compare object-space error too. It provides a safety floor between sampled views. View-aware optimization should add efficiency, not eliminate all geometric checks.

Candidate evaluation#

For every candidate, report:

weighted mean pixel error;
maximum and p95 pixel error across all views;
silhouette maximum and p95;
failures on holdout cameras;
hidden surface removed or reduced;
triangles and rendered vertices per view;
draw and memory changes;
transition events along paths;
source-to-candidate surface distance;
camera fixture and policy version.

Rank only candidates passing hard per-view, semantic, topology, and runtime gates. A low weighted average cannot excuse one broken permitted view.

When not to use view-aware output#

Avoid aggressive view coupling when:

assets are distributed to unknown viewers;
users can freely enter AR or VR;
the camera policy changes often;
the model is a reusable library master;
technical accuracy is required independent of view;
reflection, shadow, or animation coverage is not understood;
the team cannot maintain camera fixtures and regeneration.

Generate a separate view-aware derivative while retaining a general delivery version when one asset serves multiple products.

Workflow#

Document camera, projection, device, and render-consumer policy.
Build weighted optimization cameras plus fixed holdouts.
mark semantic landmarks and non-negotiable regions.
compute visibility, silhouette, projected-area, and material weights.
generate constrained candidates across ratios or error targets.
measure screen, contour, object-space, and attribute error.
test unseen cameras and continuous paths.
profile runtime selection and transitions if dynamic.
choose the smallest candidate passing every hard view.
version the output with its camera-policy fixture.

View-aware checklist#

Allowed cameras and future modes are explicit.
View weights include hard per-view quality ceilings.
Silhouette receives stronger treatment than raw projected area.
Hidden-surface claims include animation, shadows, reflections, and interaction.
Camera samples cover risk features and continuous paths.
Optimization and holdout views are separate.
Pixel metrics are normalized intentionally across resolutions.
Runtime refinement uses hysteresis and bounded transition work.
Both XR eyes and secondary render consumers are considered.
Object-space error supplies a safety floor.
General source and delivery derivatives remain available.
Camera policy and output are versioned together.

Sources and further reading#

View-aware optimization turns camera knowledge into a resource advantage. It is safe when the view distribution is explicit, rare views retain hard quality floors, holdouts prevent overfitting, and the asset can be regenerated whenever the camera contract changes.