Mesh Validation Tolerances: Why Exact Geometry Tests Report False Self-Intersections

A mesh validator can be too precise for the file it is reading. That sounds like a contradiction — precision is the point — but it is the most common way an otherwise correct geometry check produces failures nobody can act on.

Here is the shape of the problem, using a running example: a dragon figurine 100 millimeters tall, hollowed to a wall of 1.8 millimeters for resin printing. The result comes back watertight, one connected shell, every face wound consistently, no boundary edges, no non-manifold edges. Then the self-intersection check reports four pairs of intersecting faces, and the model is rejected. Open it in any inspection tool and the geometry looks perfect, because it is. The four "intersections" penetrate each other by about 2 nanometers — roughly one ten-thousandth the width of a human hair, and less than the file format can represent.

This article is about that failure mode: why it happens, how to tell it apart from a real defect in one measurement, and how to choose tolerances that separate the two reliably.

Every mesh file has a precision floor#

Start with the number that governs everything else. The glTF 2.0 specification requires vertex positions to be stored as 32-bit floats — the Khronos glTF 2.0 specification mandates FLOAT as the component type for the POSITION accessor. STL is the same in practice. Your mesh is float32 whether you designed it that way or not.

A single-precision float carries 24 bits of significand: 23 bits stored plus an implicit leading one, as described in the single-precision floating-point format reference. That gives roughly seven decimal digits, but "seven digits" is the wrong mental model for geometry. The useful quantity is the unit in the last place — the gap between one representable number and the next — because that is the finest distinction your file can express at a given coordinate magnitude. It scales with the magnitude:

| Coordinate magnitude | float32 ULP | Maximum rounding error | | --- | --- | --- | | 50 mm | 3.7 nm | ±1.9 nm | | 100 mm | 7.5 nm | ±3.7 nm | | 1 m | 119 nm | ±60 nm | | 2 m | 238 nm | ±119 nm |

Two consequences follow, and both matter for validation.

First, coordinates closer together than one ULP are the same number. Not "nearly the same" — identical, bit for bit. A file cannot describe a gap of 2 nanometers between two vertices 50 millimeters from the origin, because no such gap exists in the representation.

Second, the floor moves with your units. The same object modeled in meters has a precision floor 1000 times coarser in absolute terms than one modeled in millimeters, and a model authored around the origin has a finer floor than the same model translated to a distant coordinate. Any tolerance you hard-code as an absolute number will be wrong for some scale.

What "exact" predicates actually guarantee#

Modern geometry code leans on exact predicates, and for good reason. Naive floating-point orientation tests famously produce contradictory answers — a point reported simultaneously left of, right of, and on a line — which crashes algorithms that assume consistency. Shewchuk's adaptive precision floating-point arithmetic and fast robust geometric predicates solved this by computing orientation and in-circle tests to whatever precision the inputs demand, and CGAL built an entire library around the same idea, described in the CGAL exact geometric computation philosophy.

Here is the part that gets misread. These techniques guarantee that the arithmetic is exact for the numbers you hand them. They make no claim that the numbers describe reality. Feed an exact predicate two triangles whose vertices were rounded to float32 on export, and it will tell you with perfect rigour whether those two rounded triangles overlap. Whether the two surfaces overlap is a different question, and the difference is exactly one ULP wide.

So an exact test on quantized input is answering a slightly wrong question, very accurately. On most inputs the distinction is irrelevant. On surfaces that meet exactly, it is the whole ball game.

Meshes are full of surfaces that meet exactly#

This is the piece that makes the problem systemic rather than exotic. Exactly-coincident geometry is not a pathology — it is what constructive modelling produces by design:

• Boolean cut rims. Subtract a cylinder from a shell to make a drain hole, and the bore wall meets the shell's inner and outer surfaces along a curve where the two are, mathematically, in contact along their whole length.
• Union seams. Join two overlapping solids and the result contains surfaces that were coplanar or tangent in the input.
• Welded shells and shared boundaries. Any operation that stitches two patches produces surfaces meeting along an edge with zero separation.
• Offset surfaces that touch their source, wherever the offset distance reaches zero.

Every one of these is a surface pair with exactly zero separation in exact arithmetic. Round the result to float32 and each pair lands somewhere within one ULP of zero — sometimes cleanly separated, sometimes exactly coincident, and sometimes a nanometer or two on the wrong side. That last case is a reported self-intersection, produced entirely by the export, in a model where nothing is wrong.

The rate is low but it is not zero, and it scales with how much coincident geometry your pipeline creates. A validator that rejects on any nonzero count will reject a steady trickle of perfectly good assets, and the failures will look random because they are: they depend on which way individual coordinates happened to round.

Contact is not penetration#

The fix begins with a distinction the usual boolean intersection test throws away. "Do these triangles intersect?" collapses two very different situations into one bit:

• Contact. Two surfaces meet and do not pass through each other. The overlap is zero, or as close to zero as the format allows.
• Penetration. Two surfaces pass through each other by a measurable distance.

Only the second is a defect. The first is how solids are built.

Make it quantitative with penetration depth: the smallest translation along either face's normal that separates the pair. Computing it is cheap, because you only need it for pairs the intersection test already flagged:

1. Take face A's plane and the signed distances of face B's three vertices to it.
2. If all three distances share a sign, B does not cross A's plane; the closest approach is the smallest absolute distance.
3. Otherwise take the smaller of the largest positive distance and the largest negative distance — that is how far B would have to move to clear A's plane.
4. Repeat with the roles swapped and take the minimum of the two, since either direction separates them.

Now the two cases are numerically distinguishable. Contact sits at or below one ULP. Real penetration sits orders of magnitude above it.

Diagnose by magnitude before you hunt for a cause#

The most expensive mistake in this whole area is to trust the count and go looking for the geometry that produced it. The count carries almost no information. The magnitude carries nearly all of it, and it takes one pass to compute.

Measurements from a production hollowing pipeline make the separation concrete. Two populations, both measured with the same code — one from hollowed outputs that were flagged despite being sound, one from meshes with genuine modelling defects:

| Mesh | Flagged pairs | Minimum penetration | Median penetration | | --- | --- | --- | --- | | Hollowed shell (artifact) | 19 | 0.00 nm | ~1 nm | | Character mesh, intersecting limbs | 47 | 1,271 nm | 1.35 mm | | Assembled figure, overlapping parts | 86 | 9,643 nm | 188 µm | | Character with interpenetrating shells | 39 | 30,941 nm | 563 µm | | Rigged character, arm through torso | 5 | 236,944 nm | 456 µm |

The artifacts sit at the float32 floor, which for a model measured in meters is around 119 nanometers per 1 meter of coordinate magnitude. The tightest genuine defect in the set penetrates 1.27 micrometers — about 340 times deeper than the worst artifact, and the rest run from 9.6 micrometers to nearly 5 millimeters. Three orders of magnitude of empty space separate the populations.

Some symptoms that should send you to measure magnitude before you start rebuilding geometry:

• The count is small and stable — a handful of pairs on a mesh with tens of thousands of triangles.
• The count is invariant under changes that should matter. If rebuilding the geometry a different way, with different parameters, different tessellation, or a different algorithm entirely, keeps producing three to twelve flagged pairs, you are not looking at a property of the construction.
• The flagged faces cluster where two surfaces meet — around a cut rim, along a seam, in a thin wall — rather than where geometry visibly overlaps.
• A tool whose output is correct by construction disagrees with you. This one is decisive. Libraries such as Manifold, now used as a boolean backend in several CAD and mesh tools, guarantee manifold, non-self-intersecting output. If your validator flags their output, the disagreement is evidence about your validator, not about the mesh. Treat it as a bug report against your own tolerance.

Give your validator one rule for "the same place"#

Most mesh analyzers already contain a tolerance: the weld tolerance used to merge coincident vertices before building topology. Two vertices within it are treated as one point, which is what makes edge and component counting work on meshes whose seams do not share indices.

The common bug is that this tolerance is not applied consistently. An analyzer will weld vertices at, say, one ten-millionth of the model span, then test intersections with an epsilon a hundred times tighter — in the same pass, on the same mesh. The result is a validator that declares two vertices ten nanometers apart to be the same point, and then reports that their faces cross by three nanometers. Both statements come from the same function. They cannot both be right.

The repair is to use one rule for what counts as the same place:

• Derive the tolerance from the model span, not from a constant. Something near span × 1e-7 sits roughly three times the float32 ULP for coordinates at that scale — about 10 nanometers on a model 100 millimeters across, and about 200 nanometers on one 2 meters across, which is comfortably above rounding noise and far below any real feature. Because it scales, it stays correct whether the asset is authored in millimeters or meters.
• Apply it to penetration, not to the predicate. Leave the intersection test exact and reject flagged pairs whose penetration falls below the tolerance. This keeps the geometry logic honest: you still detect the intersection, you classify it as contact.
• Do not simply enlarge the predicate's epsilon. In typical implementations that epsilon does double or triple duty — a dimensionless barycentric tolerance in the ray-triangle test, an area-like tolerance in two-dimensional orientation tests, a length tolerance elsewhere. Scaling it up by a hundred changes the meaning of each differently, and in the coplanar path it can start treating real overlapping regions as zero area. A separate, physically meaningful threshold is both safer and easier to justify.

What must stay exempt#

One case must not be filtered by penetration depth: coplanar overlap.

Two coincident or partially overlapping faces lying in the same plane overlap sideways. Their penetration depth is zero by definition, no matter how badly they overlap — a duplicated face covering an entire surface has exactly as much penetration as one that barely clips a corner. Applying a penetration threshold to the coplanar case would suppress duplicate and overlapping faces entirely, and those are real defects: they make inside-outside classification ambiguous for a slicer, which is precisely the property the 3MF core specification requires a printable mesh to have.

So the rule splits cleanly. Transversal intersections — faces crossing at an angle — get the contact tolerance. Coplanar intersections stay reportable at any depth. Most triangle-triangle tests already branch on coplanarity internally, so this is a small change where the branch already exists; the Real-Time Rendering intersection reference collects the standard formulations if you are writing one from scratch.

Validate the tolerance instead of guessing it#

A tolerance chosen by intuition is a liability, because the failure mode is silent: too loose, and you stop reporting real defects without ever noticing. Validate it against two populations before you ship it.

1. Assemble a known-artifact set. Outputs you have independently confirmed are sound — ideally including some produced by a construction that cannot self-intersect, such as an isosurface.
2. Assemble a known-defect set. Real assets with real problems: interpenetrating limbs, shells pushed through each other, geometry assembled without cleanup.
3. Measure the penetration distribution of both, not just the counts. Percentiles, and specifically the minimum.
4. Compare the tightest genuine defect against the tolerance, not the median. The median tells you nothing about safety. In the data above the medians differ by five orders of magnitude, but the number that decides whether the tolerance is safe is the 1,271-nanometer minimum against a 190-nanometer tolerance — a factor of about seven, which is adequate but is the real margin.
5. Re-run both sets after the change and confirm the artifact counts drop to zero while every genuine count is unchanged. Anything else means the tolerance is in the wrong place.

If any genuine defect sits within an order of magnitude of your tolerance, do not ship it. Either the tolerance is too loose or your defect set contains something you have misclassified — and both are worth knowing before the check goes live.

What tolerance cannot fix#

Tolerance solves the measurement problem. It does not solve geometry, and conflating the two will cost you real defects.

The clearest example is inward offsetting, the standard way to hollow a solid: push every surface point inward along its normal by the wall thickness. Wherever the solid is thinner than twice that wall, the offset surface has to pass through itself — this is not an implementation flaw but a property of offset surfaces, the same phenomenon that makes parallel curves self-intersect wherever the offset exceeds the local radius of curvature, with the medial axis marking exactly where it happens. On the dragon figurine, a wall of 1.8 millimeters against wings, spikes and claws thinner than 3.6 millimeters produced 890 genuinely self-intersecting pairs, penetrating at millimeter scale. No tolerance should hide those, and none of the ones recommended here would.

Note how easily magnitude tells the two regimes apart:

• Hundreds of pairs penetrating at millimeter scale — the wall does not fit the model's thin features. Real, and reported.
• A handful of pairs penetrating at nanometer scale — surfaces in contact, quantized. An artifact.

The fix for the real one is construction, not tolerance. Instead of offsetting a surface, erode a volume: convert the solid to a distance or density field, mesh the isosurface at the depth you want, and subtract that. Blender exposes this through its Mesh to Volume and Volume to Mesh modifiers over OpenVDB grids, and the extracted isosurface cannot self-intersect, because marching cubes and its relatives emit a closed manifold by construction. Regions too thin to hold the wall simply come out solid, which is the physically correct answer.

That approach brings its own obligation. Eroding a model with thin limbs splits the interior into several disconnected voids, and a hollow void with no path to the outside traps uncured resin, which can crack the part as it cures — the reason hollowing guidance such as the Formlabs wall thickness guide pairs hollowing with vent holes. Any erosion-based hollowing needs to identify which voids a drain actually reaches and leave the rest solid.

A tolerance checklist#

• Know your file's precision floor: float32 ULP at your model's coordinate magnitude, not a generic "seven digits".
• Express every tolerance relative to the model span so it survives a change of units or a translated origin.
• Use one tolerance for what counts as the same place — the same number that welds your vertices should govern surface contact.
• Keep intersection predicates exact; classify with a separate penetration threshold rather than loosening the predicate.
• Exempt coplanar overlap from any penetration threshold; coincident faces are a defect at any depth.
• Measure penetration depth, not just pair counts, in every self-intersection report you surface to a user.
• Validate the tolerance against a known-defect set and a known-artifact set, and judge it on the tightest genuine defect.
• Treat disagreement with a construction-guaranteed tool as a bug in your validator until proven otherwise.
• Report the two regimes differently: a large count at feature scale is a modelling problem; a small count at the precision floor is not.

A validation gate earns trust by being both strict and actionable. Strictness alone is easy, and a check that rejects sound geometry at the precision floor is not rigorous — it is miscalibrated, and it teaches everyone downstream to ignore it. The discipline is to keep the test exact, make the tolerance explicit and scale-aware, and let the magnitude of a defect decide whether it is one.

Sources and further reading#

• Khronos: glTF 2.0 specification (accessor component types and POSITION requirements)
• Single-precision floating-point format
• Unit in the last place
• Machine epsilon
• Jonathan Shewchuk: Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates
• CGAL: exactness and the exact geometric computation philosophy
• Manifold: a geometry library guaranteeing manifold boolean output
• Real-Time Rendering: intersection algorithms reference
• Möller–Trumbore ray-triangle intersection algorithm
• Sweep and prune broad-phase collision detection
• Parallel curve (offset self-intersection)
• Medial axis
• Marching cubes
• OpenVDB: sparse volume data structure and tools
• Blender Manual: Volume to Mesh modifier
• Blender Manual: Boolean modifier (solver options)
• Blender Manual: 3D Print Toolbox (intersecting-face checks)
• 3MF Consortium: 3MF Core Specification
• Formlabs: Minimum Wall Thickness for 3D Printing