PoseBench
A benchmark can isolate interaction reasoning from single-person motion modelling in multi-agent pose prediction, by calibrating how much a partner’s pose is worth and evaluating every model on matched, user-specified contrasts.
Multi-agent pose benchmarks average position error across people, joints, and time, so a strong score cannot say whether it came from better single-person motion modelling or from genuine interaction reasoning. PoseBench introduces a calibrated axis, partner pose information demand, and lets a researcher describe a comparison in plain language, which the system compiles into matched test cells and runs every predictor on exactly the same samples.
Contributions
An evaluation axis, partner pose information demand: the improvement in predicting a target’s future local pose that comes from seeing the partner’s articulated motion, once the target’s own history and the partner’s root trajectory and orientation are already given. It tests joint-wise and body-part-wise interaction modelling directly.
An offline construction stage that standardises captured interactions, expands them with interaction-preserving physical retargeting across body proportions, joint configurations, and spatial execution, and calibrates the axis into frozen bands with uncertainty rules.
An on-demand evaluation stage that compiles a natural-language contrast into constraints, retrieves or generates candidates, builds matched test cells, and scores every predictor on the same accepted samples, reporting pose, trajectory, and relational accuracy alongside interaction gain and comparison fidelity.
Axis
Partner pose information demand, calibrated against independent reference predictors
4
Source datasets: Inter-X++, InterHuman, Hi4D, and CHI3D
NL
Comparisons requested in plain language and compiled into matched test cells
The gap
An average error does not say where the gain came from
Gains reported in prior work from interaction components, as the proposal cites them: T2P and TBIFormer interaction-modelling ablations on JRDB-GMP (JPE at 5 s), and the EMPMP distance-embedding ablation on CMU-Syn (MPJPE). These are the papers’ own numbers, not PoseBench results.
125.7 / 212.7
APE / JPE of an independent single-person baseline on MOCAP-UMPM, ahead of the interaction-aware T2P at 151.7 / 262.7
1 s → 1 s
Example contrast: one second of history to one second of local pose
200
Sequences per matched cell in the example request
How it works
Five operations, from capture to comparison
| Stage | Operation |
|---|---|
| Prepare | Standardise skeletons, frames, coordinates, actor identities, and event annotations; partition source sequences. |
| Augment | Jointly retarget interacting characters, refine their motions, and validate interaction and physical quality. |
| Calibrate | Estimate the incremental value of partner pose with independent reference predictors; freeze bands and uncertainty rules. |
| Assemble | Translate the requested contrast into constraints; retrieve candidates, search supported augmentations, and match cells. |
| Evaluate | Run predictors on shared samples; report pose, trajectory, and relational accuracy, interaction gains, and comparison fidelity. |
How it was done
Approach
Two stages: offline construction (standardise, augment, calibrate) and on-demand evaluation (compile a request, assemble matched cells, evaluate)
Datasets
Inter-X++, InterHuman, Hi4D, CHI3D
Measures
Local pose and trajectory error, interaction use along the partner-pose axis, distribution consistency, and comparison fidelity