Skip to content

How to perform each task ​

Select a task below for its starting state, required actions, success criterion, and reset. Record five trials for each of the 13 conditions (65 trials total). The two ball-valve configurations are separate conditions.

TaskWhat to doTimeout
Ball valveRotate the handle 90° from closed to open60 s
Ball valve + ringPerform the same rotation with the ring fitted90 s
Gate valve (small)Rotate the stem one full turn from closed90 s
Gate valve (large)Rotate the stem one full turn from closed120 s
Circuit breakerMove the toggle to the opposite state and hold it60 s
Light bulbThread the bulb into the socket until seated120 s
Thread (M8)Align the nut with the screw and thread it along the available length120 s
Thread (M30)Align the nut with the screw and thread it along the available length120 s
Peg insertionAlign and thread the free peg into the socket until seated120 s
ButtonOpen the cover, then press the button until actuation60 s
Lock and keyGrasp the key, insert it vertically, and rotate to unlock180 s
DrawerGrasp the drawer, align it with its housing, and insert it fully120 s
Shock absorberGrasp the loose part, align its hole with the pin, and insert the part fully onto the pin180 s

Task names follow Table 2 of the paper. See Terminology to match them to trial IDs and older file names.

Before every trial ​

Fully seat the attachment and restore the starting state described below. For loose parts, document their starting pose and use the same pose between trials. For Thread (M8) and Thread (M30), start with the nut completely removed from the fixed screw. Alignment and initial thread engagement are part of the task.

Start the end-effector from the same neutral pose for the platform. Record the complete physical trial with an external camera. Start timing with the first commanded task motion; when using the Evaluation Runner, begin that motion when the countdown reaches zero. Stop at success, a safety stop, or the timeout.

The timed sequence includes the initial approach and grasp. For the key, drawer, and shock absorber, grasping remains stage 1. For the bulb, nuts, and peg, picking up the loose part is also included in the trial time.

For familiarization, outcome labels, and handling broken or displaced parts, follow the evaluation protocol. Example videos remain available in the runner.

Ball valve ​

Task ID: valve_ball

StepInstruction
Starting stateHandle closed; no friction ring fitted; attachment fully seated.
ActionEngage the handle and rotate it 90° toward the open position.
SuccessThe handle reaches the open position after a 90° rotation.
Timeout60 s.
ResetReturn the handle to closed and check attachment seating.

The 360° variant allows overshoot without reaching a quarter-turn stop during development. The task target remains 90°. Record which variant is used and mark the initial and target orientations.

Ball valve + ring ​

Task ID: valve_ball_ring

StepInstruction
Starting stateFriction ring fitted; handle closed; attachment fully seated. Document which ring is used.
ActionEngage the handle and rotate it 90° toward the open position with the ring fitted.
SuccessThe handle reaches the open position after a 90° rotation.
Timeout90 s.
ResetKeep the same ring fitted, return the handle to closed, and check attachment seating.

Gate valve (small) ​

Task ID: valve_gate_small

StepInstruction
Starting stateStem in the closed starting position. Identify its initial orientation so one full turn can be observed.
ActionTurn the valve to rotate the stem through one complete revolution from closed.
SuccessThe stem completes one full turn (360°). Full opening travel is not required.
Timeout90 s.
ResetReverse the motion to restore the closed starting position and initial orientation. Keep the board position unchanged.

Gate valve (large) ​

Task ID: valve_gate_large

StepInstruction
Starting stateStem in the closed starting position. Identify its initial orientation so one full turn can be observed.
ActionTurn the valve to rotate the stem through one complete revolution from closed.
SuccessThe stem completes one full turn (360°). Full opening travel is not required.
Timeout120 s.
ResetReverse the motion to restore the closed starting position and initial orientation. Keep the board position unchanged.

Circuit breaker ​

Task ID: circuit_breaker

StepInstruction
Starting stateToggle in the documented initial state.
ActionMove the toggle to the opposite state and hold it there.
SuccessThe toggle has changed state and is held in that state.
Timeout60 s.
ResetReturn the toggle to the initial state and check that it moves freely.

Light bulb ​

Task ID: light_bulb

StepInstruction
Starting stateBulb removed from the socket and placed in its documented starting pose; thread clear.
ActionAlign the bulb with the socket and rotate it to engage and advance the thread.
SuccessThe bulb is fully threaded until seated.
Timeout120 s.
ResetUnscrew and remove the bulb, restore its starting pose, and inspect the thread.

Thread (M8) ​

Task ID: thread_m8

StepInstruction
Starting stateNut completely removed from the fixed screw, with no thread engagement. Document the nut's starting pose and keep it consistent across trials.
ActionAlign the nut with the screw, engage the thread, and rotate the nut along the available length.
SuccessThe nut is fully threaded along the available length of the screw.
Timeout120 s.
ResetFully unthread and remove the nut, restore its starting pose, and inspect both threads.

Thread (M30) ​

Task ID: thread_m30

StepInstruction
Starting stateNut completely removed from the fixed screw, with no thread engagement. Document the nut's starting pose and keep it consistent across trials.
ActionAlign the nut with the screw, engage the thread, and rotate the nut along the available length.
SuccessThe nut is fully threaded along the available length of the screw.
Timeout120 s.
ResetFully unthread and remove the nut, restore its starting pose, and inspect both threads.

The runner's M8 and M30 images show the assembled mechanism, with the nut on the screw. These images do not define the required starting state. Begin each new trial with the nut completely clear of the screw. Identify earlier trials that began with the nut engaged when reporting results.

Peg insertion ​

Task ID: peg_insertion

StepInstruction
Starting stateSocket empty; free 8 mm threaded peg in its documented pose next to the socket.
ActionPick up the peg, align it with the socket, and rotate it to engage and advance the thread.
SuccessThe peg is threaded into the socket until seated. Simply placing it in the opening is insufficient.
Timeout120 s.
ResetUnscrew and remove the peg, leave the socket empty, and restore the peg's starting pose.

Button ​

Task ID: button

StepInstruction
Starting stateCover closed; button returned to its unpressed state.
Stage 1Open the cover to expose the button.
Stage 2Press the button until actuation.
SuccessBoth stages complete within the timeout.
Timeout60 s for the complete sequence.
ResetRelease the button, confirm that it returns, and close the cover.

Record stage_reached = 0 if no stage completes, 1 if only the cover is opened, or 2 when the button is also actuated.

Lock and key ​

Task ID: lock

StepInstruction
Starting stateLock in its locked initial state; key removed and placed in its documented starting pose.
Stage 1Grasp the key.
Stage 2Insert the key vertically into the lock.
Stage 3Rotate the key to unlock.
SuccessThe lock is unlocked after all three stages.
Timeout180 s for the complete sequence.
ResetRestore the lock's initial state, remove the key, and return the key to its starting pose.

Record the last completed stage: 0, 1, 2, or 3. Inserting the key without unlocking is stage 2, not full success.

Drawer ​

Task ID: drawer

StepInstruction
Starting stateDrawer completely removed from its housing and placed in its documented starting pose. Housing fixed to the board and empty.
Stage 1Grasp the drawer.
Stage 2Align the drawer with the housing opening.
Stage 3Insert the drawer fully into the housing.
SuccessThe drawer is correctly seated in the fully inserted position.
Timeout120 s, including the initial approach and grasp.
ResetRemove the drawer completely from its housing and restore its documented starting pose.

Record the last completed stage: 0, 1, 2, or 3. Record 2 if the drawer is aligned but only partly inserted. Full success requires complete insertion. The task ends with the drawer inserted.

Identify earlier trials that used an open-and-close sequence in notes when reporting results.

Shock absorber ​

Task ID: shock_absorber

Grasp the loose part of the shock absorber and insert it onto the pin. The loose part moves while the pin stays in place. The complete sequence has three stages and a single 180 s timeout.

StepInstruction
Starting stateLoose part removed from the pin and placed in its documented starting pose. Leave the pin in place and check that the attachment is seated in both occupied board cells.
Stage 1 — graspApproach and grasp the loose part of the shock absorber with the end-effector.
Stage 2 — alignAlign the hole in the grasped part with the pin.
Stage 3 — insertMove the aligned part onto the pin until it is fully inserted.
SuccessAll three stages are complete, with the loose part fully inserted onto the pin.
Timeout180 s total, beginning with the first commanded task motion. Do not restart the timer between stages.
ResetRemove the loose part from the pin and return it to its starting pose. Leave the pin in place and check attachment seating in both board cells. Restore the end-effector's neutral pose before the next trial.

Record the last completed stage:

stage_reachedWhat was completed
0The loose part was not grasped.
1The loose part was grasped, but alignment with the pin was not completed.
2The hole in the part was aligned with the pin, but the part was not fully inserted onto it. This includes partial insertion.
3The part was fully inserted onto the pin.

For example, if the part is aligned but only partly inserted onto the pin when 180 s expires, record outcome = timeout, stage_reached = 2, and leave completion_time_s blank. Also record attempts, regrasps, strategy, and the dominant failure cause. See trial logging for the field definitions.

See the source protocol for success criteria and timeouts. The protocol does not specify a numeric insertion depth, holding duration, or loose-part pose. Document the starting pose and use the same setup for every trial.

HiveBoard documentation