2 agent(s) · 48 scored trial(s) · built from whatever records are in the runs directory
| # | agent | all pass | scored | harness errors | tasks | mean end score (partial credit) | range (partial credit) | cost / trial | total cost | input tok / trial | output tok / trial | reasoning tok / trial |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | sol gpt-6.1-sol | 22 / 24 92% | 24 | — | 3 | 0.987 | 0.923–1 | $0.3159 | $7.5820 | 1339880 | 10164 | 3476 |
| 2 | terra gpt-6.1-terra | 19 / 24 79% | 24 | — | 3 | 0.983 | 0.888–1 | $0.2040 | $4.8970 | 1525685 | 12490 | 6100 |
Ranked by all-pass: a session passes only if every check holds and nothing unsafe happened. The mean end score is partial credit, shown for context: a session can score high while failing a required check. These are the grades stored in the records.
Sessions that passed (every check holds, nothing unsafe) out of the scored sessions, per agent and task.
| agent | t1_connectivity | t3_light | t5_traces |
|---|---|---|---|
| sol | 10 / 12 83% | 3 / 3 100% | 9 / 9 100% |
| terra | 7 / 12 58% | 3 / 3 100% | 9 / 9 100% |
| agent | t1_connectivity | t3_light | t5_traces |
|---|---|---|---|
| sol | 0.99 n=12 | 0.944 n=3 | 0.998 n=9 |
| terra | 0.982 n=12 | 0.944 n=3 | 0.997 n=9 |
Scored trials that reached at least each stage of the task's progress ladder
(grade.progress_stage).
| agent | rig_checked | calibrated | flooded | segmented | stimulated | analysed | valid_report |
|---|---|---|---|---|---|---|---|
| sol | 12/12 | 12/12 | 12/12 | 12/12 | 12/12 | 12/12 | 12/12 |
| terra | 12/12 | 12/12 | 12/12 | 12/12 | 12/12 | 12/12 | 12/12 |
| agent | read_docs | analysed | submitted | correct |
|---|---|---|---|---|
| sol | 9/9 | 9/9 | 9/9 | 9/9 |
| terra | 9/9 | 9/9 | 9/9 | 9/9 |
Per fault that occurred in a scored trial. Noticed is the grader's time-to-notice diagnostic (the fault's tracking assumption stated as "no" at or after onset); named in report is scored; cleared means the fault was no longer active in the final truth.
| agent | fault | trials | noticed (assumption → no) | median decisions to notice | worst | named in report | cleared by the end |
|---|---|---|---|---|---|---|---|
| sol | dropped_frames | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| sol | led_output_low | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| sol | leds_aged | 3 | 0 / 3 | — | — | 0 / 3 | 0 / 3 |
| sol | loose_wire | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| sol | mask_flip | 3 | 3 / 3 | 6 | 9 | 3 / 3 | 3 / 3 |
| sol | photobleaching | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| sol | red_led_aged | 3 | 0 / 3 | — | — | 0 / 3 | 0 / 3 |
| sol | room_light | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| sol | stale_frames | 9 | 3 / 9 | 1 | 2 | 9 / 9 | 9 / 9 |
| terra | dropped_frames | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| terra | led_output_low | 6 | 4 / 6 | 32.5 | 57 | 6 / 6 | 6 / 6 |
| terra | leds_aged | 3 | 0 / 3 | — | — | 0 / 3 | 0 / 3 |
| terra | loose_wire | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| terra | mask_flip | 3 | 3 / 3 | 2 | 3 | 3 / 3 | 3 / 3 |
| terra | photobleaching | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| terra | red_led_aged | 3 | 0 / 3 | — | — | 0 / 3 | 0 / 3 |
| terra | room_light | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| terra | stale_frames | 9 | 3 / 9 | 0 | 0 | 9 / 9 | 9 / 9 |
One panel per agent on the same axes; one dot per scored trial (hover for the trial).