48 trials · total cost $12.48 · the 8 Claude Fable and Opus E1 episodes (stress1_anthropic, $34.36) are not on this page; they are in the dataset's episodes table and the Results page · click a trial to step through it
| agent | task | trials | harness errors | submitted | all pass | mean end score | cost | cost / trial |
|---|---|---|---|---|---|---|---|---|
| sol | t1_connectivity | 12 | — | 12 | 10 / 12 | 0.99 | $3.9510 | $0.3293 |
| sol | t3_light | 3 | — | 3 | 3 / 3 | 0.944 | $1.2496 | $0.4165 |
| sol | t5_traces | 9 | — | 9 | 9 / 9 | 0.998 | $2.3814 | $0.2646 |
| sol | all tasks | 24 | — | 24 | 22 / 24 | 0.987 | $7.5820 | $0.3159 |
| terra | t1_connectivity | 12 | — | 12 | 7 / 12 | 0.982 | $2.3970 | $0.1998 |
| terra | t3_light | 3 | — | 3 | 3 / 3 | 0.944 | $0.7886 | $0.2629 |
| terra | t5_traces | 9 | — | 9 | 9 / 9 | 0.997 | $1.7114 | $0.1902 |
| terra | all tasks | 24 | — | 24 | 19 / 24 | 0.983 | $4.8970 | $0.2040 |
Each trial has one end grade, computed once after the run ends. Harness failures (the run stopped on a provider or harness error) are listed but have no score: they are left out of "all pass" and the means.
| headline | sol | t1_connectivity | loose_wire | trial 0 | submitted | 1 | pass | $0.2596 | 31 | 48 |
| headline | sol | t1_connectivity | loose_wire | trial 1 | submitted | 1 | pass | $0.2504 | 28 | 49 |
| headline | sol | t1_connectivity | loose_wire | trial 2 | submitted | 1 | pass | $0.3799 | 38 | 60 |
| headline | sol | t1_connectivity | mask_flip | trial 0 | submitted | 1 | pass | $0.3262 | 41 | 62 |
| headline | sol | t1_connectivity | mask_flip | trial 1 | submitted | 0.957 | fail | $0.4454 | 48 | 72 |
| headline | sol | t1_connectivity | mask_flip | trial 2 | submitted | 1 | pass | $0.3787 | 43 | 66 |
| headline | sol | t1_connectivity | red_led_aged | trial 0 | submitted | 1 | pass | $0.2733 | 28 | 46 |
| headline | sol | t1_connectivity | red_led_aged | trial 1 | submitted | 1 | pass | $0.3066 | 36 | 57 |
| headline | sol | t1_connectivity | red_led_aged | trial 2 | submitted | 0.923 | fail | $0.3423 | 36 | 59 |
| headline | sol | t1_connectivity | stale_frames | trial 0 | submitted | 1 | pass | $0.3146 | 37 | 59 |
| headline | sol | t1_connectivity | stale_frames | trial 1 | submitted | 1 | pass | $0.3676 | 31 | 59 |
| headline | sol | t1_connectivity | stale_frames | trial 2 | submitted | 1 | pass | $0.3064 | 34 | 55 |
| headline | terra | t1_connectivity | loose_wire | trial 0 | submitted | 1 | pass | $0.1246 | 26 | 42 |
| headline | terra | t1_connectivity | loose_wire | trial 1 | submitted | 1 | pass | $0.1488 | 26 | 43 |
| headline | terra | t1_connectivity | loose_wire | trial 2 | submitted | 1 | pass | $0.1642 | 27 | 42 |
| headline | terra | t1_connectivity | mask_flip | trial 0 | submitted | 0.888 | fail | $0.2237 | 33 | 51 |
| headline | terra | t1_connectivity | mask_flip | trial 1 | submitted | 1 | pass | $0.1855 | 41 | 62 |
| headline | terra | t1_connectivity | mask_flip | trial 2 | submitted | 0.969 | fail | $0.2222 | 39 | 60 |
| headline | terra | t1_connectivity | red_led_aged | trial 0 | submitted | 0.927 | fail | $0.2645 | 36 | 61 |
| headline | terra | t1_connectivity | red_led_aged | trial 1 | submitted | 1 | pass | $0.1496 | 22 | 38 |
| headline | terra | t1_connectivity | red_led_aged | trial 2 | submitted | 1 | pass | $0.2417 | 31 | 51 |
| headline | terra | t1_connectivity | stale_frames | trial 0 | submitted | 1 | pass | $0.1750 | 26 | 46 |
| headline | terra | t1_connectivity | stale_frames | trial 1 | submitted | 0.998 | fail | $0.2293 | 30 | 56 |
| headline | terra | t1_connectivity | stale_frames | trial 2 | submitted | 0.998 | fail | $0.2679 | 29 | 62 |
| headline | sol | t3_light | converging_led_low | trial 0 | submitted | 0.973 | pass | $0.3872 | 56 | 82 |
| headline | sol | t3_light | converging_led_low | trial 1 | submitted | 0.929 | pass | $0.3815 | 43 | 76 |
| headline | sol | t3_light | converging_led_low | trial 2 | submitted | 0.929 | pass | $0.4808 | 58 | 93 |
| headline | terra | t3_light | converging_led_low | trial 0 | submitted | 0.973 | pass | $0.2812 | 61 | 99 |
| headline | terra | t3_light | converging_led_low | trial 1 | submitted | 0.929 | pass | $0.2387 | 31 | 65 |
| headline | terra | t3_light | converging_led_low | trial 2 | submitted | 0.929 | pass | $0.2686 | 58 | 93 |
| headline | sol | t5_traces | bleached_and_dropped | trial 0 | submitted | 0.995 | pass | $0.2658 | 1 | 24 |
| headline | sol | t5_traces | bleached_and_dropped | trial 1 | submitted | 0.999 | pass | $0.2089 | 1 | 20 |
| headline | sol | t5_traces | bleached_and_dropped | trial 2 | submitted | 0.999 | pass | $0.2789 | 1 | 24 |
| headline | sol | t5_traces | everything | trial 0 | submitted | 0.998 | pass | $0.2036 | 1 | 18 |
| headline | sol | t5_traces | everything | trial 1 | submitted | 0.997 | pass | $0.2064 | 1 | 20 |
| headline | sol | t5_traces | everything | trial 2 | submitted | 0.993 | pass | $0.5219 | 1 | 33 |
| headline | sol | t5_traces | light_and_frozen | trial 0 | submitted | 1 | pass | $0.2439 | 1 | 23 |
| headline | sol | t5_traces | light_and_frozen | trial 1 | submitted | 1 | pass | $0.2535 | 1 | 18 |
| headline | sol | t5_traces | light_and_frozen | trial 2 | submitted | 0.999 | pass | $0.1985 | 1 | 17 |
| headline | terra | t5_traces | bleached_and_dropped | trial 0 | submitted | 0.998 | pass | $0.2098 | 1 | 31 |
| headline | terra | t5_traces | bleached_and_dropped | trial 1 | submitted | 1 | pass | $0.1944 | 1 | 22 |
| headline | terra | t5_traces | bleached_and_dropped | trial 2 | submitted | 0.999 | pass | $0.2622 | 1 | 31 |
| headline | terra | t5_traces | everything | trial 0 | submitted | 0.992 | pass | $0.1853 | 1 | 25 |
| headline | terra | t5_traces | everything | trial 1 | submitted | 0.994 | pass | $0.1980 | 1 | 27 |
| headline | terra | t5_traces | everything | trial 2 | submitted | 0.996 | pass | $0.1805 | 1 | 25 |
| headline | terra | t5_traces | light_and_frozen | trial 0 | submitted | 0.999 | pass | $0.1485 | 1 | 22 |
| headline | terra | t5_traces | light_and_frozen | trial 1 | submitted | 0.997 | pass | $0.1715 | 1 | 25 |
| headline | terra | t5_traces | light_and_frozen | trial 2 | submitted | 0.998 | pass | $0.1610 | 1 | 25 |