HiveBoard:
An Open, Modular, 3D-Printed Benchmark of
Industrial Mechanisms for Robotic and Prosthetic Manipulation
Abstract
Manipulation is mostly benchmarked on free objects, through grasping, pick-and-place, and assembly of loose parts. The mechanisms that make manipulation useful in the world are not free objects: a valve resists rotation, a screw advances only along its thread, a lock opens only after a key is inserted and turned. Performance on such constrained, force-dependent, often multi-step devices is poorly predicted by grasping scores, and the physical benchmarks that do cover mechanisms are built from purchased components, which makes them hard to reproduce or modify. We present HiveBoard, an open, modular, and accessible, fully 3D-printed benchmark that lets any laboratory evaluate grippers, hands, and manipulation policies on functional mechanisms. A hexagonal honeycomb base accepts 12 interchangeable attachment designs modeled on industrial fittings and grouped into three skill categories — torque, precision, and composed assembly — all fully functional, printable on consumer-grade FDM printers, and released with articulated URDF and USD models for direct import into common simulators. A cross-laboratory validation of six systems spans quadruped and mobile manipulators, fixed-base arms, a VR-driven legged manipulator, and a wearable prosthetic hand. Each system completed 65 trials across 13 conditions, including the ball valve with and without its friction ring. Only the ring-fitted ball valve succeeded in every trial on all six platforms; none completed peg insertion. Difficulty-ordering correlations range from 0.20 to 0.88, and the failure causes reveal different limits across the systems.
Torque-based tasks
A ball valve, two gate valves, and a circuit breaker. A four-piece snap-on friction ring set modulates the breakaway torque of the ball valve without printing a second one.
Precision-based tasks
A light bulb socket, M8 and M30 threaded fasteners, and a peg-insertion plate with 8 mm threaded pins that demand tight-clearance alignment followed by sustained axial rotation.
Composed assembly
A covered push button, a key-and-lock mechanism, a sliding drawer, and a two-cell shock absorber. Each requires several sub-actions and is scored stage by stage, so partial competence is visible in the data.
Interactive 3D Benchmark
Explore the HiveBoard interactively in 3D. The base is a seven-cell hexagonal honeycomb in which every cell is an open frame that accepts an attachment through a shared press-fit interface, with no fasteners required. Rotate the perspective, snap attachments into free cells, and compose the evaluation scene you intend to print, in the large, medium, or small board version.
Interactive 3D Benchmark Explorer
Click to interact with the modular boards and tasks
Attachments & Specifications
The library contains 12 attachment designs evaluated in 13 conditions across three task categories. Select a category below to compare each printed part with its digital twin and review the success criterion, timeout, and stage decomposition used by the evaluation protocol.
Loading...
| Parameter | Value |
|---|
Simulation Compatibility
Every attachment and the base are released as articulated CAD assets, organized per assembly with URDF descriptions, USD exports, and OBJ visual and collision meshes, so the same task can be run physically or in simulation.
Newton
GPU-accelerated physics engine for robotics. HiveBoard’s URDF and USD assets can be used as the starting point for Newton scenes.
TestIsaac Sim
USD assets for HiveBoard assemblies and the honeycomb base are available for composing and simulating scenes.
ManiSkill
Robotics simulation and learning framework powered by SAPIEN. Use the released URDF and mesh assets to build custom HiveBoard task environments.
Try it in the browser
Still under developmentMuJoCo compiled to WebAssembly — real physics, no install
Pick a platform and a task, and the arm runs a fixed trajectory for it against a seven-cell board, the swappable centre cell holding whichever attachment the task calls for. These are demonstrations, not benchmark runs — they do not follow the evaluation protocol. They are here to show what each task looks like: most replay an authored attachment motion alongside the arm rather than manipulating it through contact, and only a few — the circuit breaker, and the ball valve on Spot and the SO-101 — are driven by the arm for real. Every trajectory is solved offline against the actual scene, so a platform lists only the tasks it could reach; the rest show as coming soon, which is why the lists differ. Robot models from mujoco_menagerie (Apache-2.0); interactive simulator interface based on Tune to Learn; board and attachments built from the URDF release above.
Video Demonstrations
Recorded trials from the cross-laboratory validation. Each laboratory printed the board from the released files and ran the same protocol on its own hardware, so a single attachment can be compared across a quadruped-mounted gripper, a legged manipulator, tabletop arms, the Alter-Ego mobile manipulator, and a prosthetic hand worn on an operator’s forearm.
Platform A · Spot with Spot Arm
Platform B · LeRobot SO-101
Platform C · ANYmal with DynaArm
Platform D · Macao Prosthetic Hand
Platform E · Franka Research 3
Platform F · Alter-Ego with Pisa/IIT SoftHand
Platform G · Prosthetic User with SoftHand Pro
Platform H · UR20 with Robotiq 2F-140
Isaac Lab simulations
Simulated HiveBoard tasks with ANYmal D and Franka FR3. Explore the Isaac Lab integration.
ANYmal D with DynaArm
Franka FR3
Results
Each laboratory prints the board from the released files and runs the same protocol on its own hardware: five recorded trials per condition, a fixed neutral start pose, task-specific timeouts, no scripted assistance. Select any cell, platform, or condition to see what happened in those trials.
Success rates carry 95% Wilson confidence intervals over N = 5 trials per cell. The intervals are wide by design: this is a per-laboratory reproducibility check, not a high-resolution ranking. Each platform was operated by a single operator at a single laboratory, so operator skill and platform capability are not separable in these data.
Same seven, same six
A tablet-teleoperated quadruped arm and a VR-driven legged manipulator share no arm, gripper, or interface, yet complete an identical set of seven conditions and fail on the same six — for entirely different logged reasons.
Assembly tasks separate the platforms
The Spot Arm and ANYmal failed both the drawer and shock absorber in every trial. The SO-101 and Macao, SoftHand Pro, and UR20 completed both; the FR3 completed the shock absorber, and Alter-Ego completed one drawer trial.
Two new platform results
SoftHand Pro completed 60/65 trials, including both threaded fasteners. The UR20 completed 50/65 and was the only platform to complete peg insertion (2/5).
Cross-platform analysis
Each platform ranks the 13 conditions by unsuccessful trials first and normalized median time second, and the eight orderings are then compared pairwise with a Spearman correlation. A high correlation means the board is measuring properties of the tasks rather than idiosyncrasies of one system.
range 0.20 to 0.89
the two legged bases
highest pair in the study
lowest pair in the study
SoftHand Pro correlates most with Alter-Ego (0.89), while the UR20 correlates most with the FR3 (0.80). The lowest pair is Alter-Ego and SO-101 (0.20).
HiveScore
Success weighted by completion speed relative to the unimpaired human reference.
| Platform | Successes | Conditions | Median slowdown | HiveScore |
|---|---|---|---|---|
| A · Spot Arm | 35/65 | 7 | 9.8× (6.6–12.8) | 5.8 (5.4–6.1) |
| B · SO-101 | 44/65 | 9 | 5.8× (3.8–19.3) | 12.0 (11.3–12.6) |
| C · ANYmal | 35/65 | 7 | 6.1× (4.6–10.1) | 8.7 (8.2–9.2) |
| D · Macao hand | 53/65 | 11 | 1.6× (0.9–14.7) | 41.6 (40.0–43.2) |
| E · FR3 | 40/65 | 9 | 10.9× (8.3–19.1) | 5.7 (5.4–6.0) |
| F · Alter-Ego | 34/65 | 9 | 16.6× (6.0–28.7) | 3.7 (3.5–3.9) |
| G · Prosthetic User with SoftHand Pro | 60/65 | 12 | 1.6× (0.9–11.7) | 51.1 (50.5–52.3) |
| H · UR20 | 50/65 | 12 | 7.6× (4.6–20.9) | 9.7 (9.6–10.0) |
13
Evaluation conditions across 12 attachment designs in three task categories
520
Recorded trials, 65 on each of the eight platforms, under one fixed protocol
1.6–103 s
Span of median completion times, from the circuit breaker to the M30 thread
0
Purchased parts, metal inserts, or post-machining steps needed to reproduce the board
Data Collection API
To standardize manipulation data collection across laboratories and robotic platforms, we created DataHive — a dedicated API and client toolkit designed to collect, validate, annotate, and record HiveBoard manipulation episodes.
DataHive & datahive-tools
What it's for: Provides a streamlined pipeline to record robot trials, validate episode integrity (verifying timestamps, frequencies, and synchronized multi-camera MP4 streams), annotate outcomes in a local web interface, and upload structured datasets directly to our official repository on the Hugging Face Hub (HiveBoard/<lab_id>).
Contributing data: All contributions go directly into our central repository. Register your laboratory using the registration form to receive your designated Lab ID and Hugging Face Token, allowing DataHive to authenticate and upload directly to HiveBoard/<lab_id>. See our Open Calls for full dataset requirements and authorship eligibility.
Reference integration: Looking for a starting point? See the fr3_datahive repository for a Dockerized Franka Research 3 (FR3) setup with ROS and joystick teleoperation.
Installation
pip install datahive-tools
Installs the datahive CLI and Python CollectClient for recording, validating, and managing datasets.
Official HiveBoard Repository
Submit the registration form to receive your unique Lab ID and Hugging Face Token. All uploads go directly to our official repository on the Hub (HiveBoard/<lab_id>).
Open Calls for Contributions
We invite laboratories, individual researchers, and teams to contribute manipulation datasets using HiveBoard or develop functional attachments for two benchmark extensions. Accepted contributions may establish eligibility for co-authorship on future papers.
Learning Datasets
Demonstrations and robot rollouts with synchronized states, actions, camera streams, and task outcomes for imitation and visuomotor policy learning. No fixed episode count or 13-condition requirement.
Daily Living (ADL)
Functional mechanisms representing everyday manipulation tasks (container lids, latches, dispensers, household controls). Must feature repeatable initial states and observable success criteria.
Bimanual Tasks
Functional attachments requiring coordinated two-hand manipulation (e.g. holding while turning, maintaining spring tension during insertion, aligning movable components).
Authorship & Submission Process
Substantial accepted datasets and complete functional attachments (CAD, printables, simulation model, task definition, and validation) establish eligibility for co-authorship on future benchmark papers. Contact the team before starting fabrication or collection.
Citation
@article{hiveboard2026,
title = {HiveBoard: An Open, Modular, 3D-Printed Benchmark of Industrial Mechanisms
for Robotic and Prosthetic Manipulation},
author = {Godoy, Ricardo V. and de Souza, Enzo F. and Angarola, Matheus P. and
de Lange, Rudy De-Xin and Negri, Juliano and Marsicano, Jo\~{a}o A. and
Aléssio, João H. and van Halst, Victor and
Elanjimattathil Vijayan, Aravind and Capezzuto, Gianluca and
Tommaselli, Felipe and Milazzo, Giuseppe and
Ramírez Sánchez, Amy M. and Affonso, Francisco and Baptista, Rafael R. and
van Berge, Meiko A. and Chowdhary, Girish and Bezerra, Ranulfo and Lahr, Gustavo J. G. and Ferrari Gerez, Lucas and
Bicchi, Antonio and Becker, Marcelo},
journal = {Under review},
year = {2026},
url = {https://github.com/hiveboard-bench/HiveBoard}
}Acknowledgments
HiveBoard, the CAD library, the evaluation protocol, and the logging templates are released under a permissive license at github.com/hiveboard-bench/HiveBoard. We thank the collaborating laboratories that printed the board locally and ran the protocol on their own manipulation systems for the cross-laboratory validation reported here, and Luiz Felipe Rodrigues Dantas for assistance in validating the simulated assets.
This work was supported by the São Paulo Research Foundation (FAPESP), Grants #2025/08520-0, #2025/22381-3, #2025/20858-7, and #2025/04308-7; by Petróleo Brasileiro S/A — Petrobras, using resources from the ANP R&D clause, in partnership with the University of São Paulo (USP) and the Fundação de Apoio à Física e à Química (FAFQ), under Cooperation Agreements #2023/00016-6 and #2023/00013-7; by ANYbotics AG; and by the Royal Society Research Grant (grant number 251405).
