HaNDF uses object shape to guide more plausible digital hand poses

A hand-pose guide that takes objects into account

A hand configuration that looks reasonable on its own may make little sense around a particular object. Fingers can pass through its surface, miss contact or bend into an unlikely arrangement. Reconstructing hands from images becomes especially difficult when the object hides much of the hand.

Researchers at Imperial College London propose HaNDF, short for Object-Conditioned Geometric Neural Hand Distance Fields, to help address this problem. Their paper was posted to arXiv on November 5, 2026; the supplied record lists it as accepted at NeurIPS 2026. The work presents a learned hand-pose prior—a guide to plausible configurations—not a complete image-reconstruction system or a demonstrated robotic controller.

Its central idea is to learn how far a proposed hand articulation lies from plausible articulations for a given object, then use that estimate to guide corrections.

Distance through possible poses, not physical space

The distance in HaNDF is not the gap between a fingertip and an object's surface. It measures separation in the space of joint rotations.

The model represents the hand using MANO, a parametric hand model, and describes 15 local finger-joint rotations with unit quaternions. These mathematical representations follow a curved geometry rather than an ordinary flat coordinate space. HaNDF uses geometry-aware optimization so that successive corrections remain valid rotation representations. That alone does not guarantee anatomically or physically valid poses; those constraints are learned from examples.

Object shape enters through a compact, 64-dimensional code fitted using a DeepSDF-style shape model. A transformer processes the joint representations and object code together, predicting a nonnegative distance. Plausible articulations are represented by the field's ideal zero-level set; in practice, optimization seeks sufficiently small predicted values.

Training uses recorded hand–object pairs with synthetic perturbations added to the hand poses. Supervision comes from the distance to the nearest recorded clean articulation associated with that particular training object. The resulting field therefore approximates plausibility from the available examples rather than deriving it from a complete physical model.

The same learned guide supports several tasks. It can correct noisy poses, fill in missing finger rotations while preserving observed joints, or generate articulations by refining random initial rotations. Reconstruction systems can also combine it with terms that enforce agreement with image or geometric observations.

What the benchmark tests show

The authors evaluate HaNDF using HOI4D, OakInk and DexYCB. Their main HOI4D split holds out object instances for testing, although those objects belong to categories represented during training.

In an HOI4D denoising test with a noise setting of 0.8 radians, HaNDF reduced the root-mean-square joint-rotation error to 5.52 degrees, compared with 7.16 degrees for H-NRDF, an unconditional hand adaptation of an earlier geometry-aware distance-field method. Mean hand-vertex distance from the reference was 6.58 millimeters versus 8.60 millimeters. These results support a benefit from conditioning the prior on the object in this setting.

For completion with one entire finger masked, HaNDF achieved 12.57 degrees of rotation error on the missing joints, compared with H-NRDF's 16.60 degrees. The comparison evaluates the concealed joints rather than giving methods credit for the observed joints held fixed.

Reconstruction results need an important qualification: the main hand-recovery comparison supplies ground-truth object geometry to every method. This isolates hand reconstruction but does not reproduce the harder situation in which object shape must also be estimated.

A separate HOI4D experiment using EasyHOI-predicted objects also showed improvement: adding HaNDF reduced mean joint-position error from 17.06 to 15.88 millimeters. However, improvements were not universal across every measure. Adding HaNDF to AlignSDF slightly increased hand–object intersection volume on OakInk, despite reducing hand-position errors.

Plausibility remains a qualified claim

HaNDF relies on reasonably accurate object geometry and does not jointly model the full hand–object configuration as a single distance field. The authors identify contact dynamics and a properly normalized joint field as further work. Consequently, better pose scores or reduced mesh penetration should not be read as guarantees of stable, safe manipulation.

Generalization is also incomplete. A supplementary test on three object categories absent from training improved noisy poses, but remained worse than a model trained with those categories represented. Most experimental comparisons lack uncertainty intervals or significance tests, although selected results include repeated-seed analysis.

The contribution is a reusable, object-aware guide for hand articulation. Its reported results suggest that combining learned examples with rotation-aware geometry can improve several reconstruction tasks, while leaving broader physical validation and generalization unresolved.