6 agent(s) · 32 scored trial(s) · built from whatever records are in the runs directory
| # | agent | all pass | scored | harness errors | tasks | mean end score (partial credit) | range (partial credit) | cost / trial | total cost | input tok / trial | output tok / trial | reasoning tok / trial |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | sol gpt-6.1-sol | 4 / 6 67% | 6 | — | 2 | 0.957 | 0.831–1 | $0.4554 | $2.7325 | 2488818 | 10585 | 3747 |
| 2 | terra gpt-6.1-terra | 4 / 6 67% | 6 | — | 2 | 0.946 | 0.756–1 | $0.2526 | $1.5154 | 2242712 | 11760 | 5690 |
| 3 | fable claude-fable-5-1 | 2 / 4 50% | 4 | — | 2 | 0.954 | 0.925–0.979 | $8.0475 | $32.1901 | 7738950 | 71896 | 0 |
| 4 | astra gpt-6-astra | 3 / 6 50% | 6 | — | 2 | 0.928 | 0.811–1 | $3.1722 | $19.0332 | 2255961 | 10040 | 3538 |
| 5 | opus claude-opus-5-5 | 1 / 4 25% | 4 | — | 2 | 0.927 | 0.824–0.981 | $2.8378 | $11.3512 | 6112080 | 43168 | 0 |
| 6 | argon google/gemini-4-argon | 0 / 6 0% | 6 | — | 2 | 0.879 | 0.667–0.981 | $2.2942 | $13.7653 | 6903910 | 58334 | 32851 |
Ranked by all-pass: a session passes only if every check holds and nothing unsafe happened. The mean end score is partial credit, shown for context: a session can score high while failing a required check. These are the grades stored in the records.
Sessions that passed (every check holds, nothing unsafe) out of the scored sessions, per agent and task.
| agent | t3_light | t4_takeover |
|---|---|---|
| argon | 0 / 3 0% | 0 / 3 0% |
| astra | 1 / 3 33% | 2 / 3 67% |
| fable | 2 / 2 100% | 0 / 2 0% |
| opus | 1 / 2 50% | 0 / 2 0% |
| sol | 2 / 3 67% | 2 / 3 67% |
| terra | 2 / 3 67% | 2 / 3 67% |
| agent | t3_light | t4_takeover |
|---|---|---|
| argon | 0.776 n=3 | 0.981 n=3 |
| astra | 0.866 n=3 | 0.99 n=3 |
| fable | 0.931 n=2 | 0.977 n=2 |
| opus | 0.887 n=2 | 0.967 n=2 |
| sol | 0.921 n=3 | 0.994 n=3 |
| terra | 0.894 n=3 | 0.997 n=3 |
Scored trials that reached at least each stage of the task's progress ladder
(grade.progress_stage).
| agent | rig_checked | calibrated | flooded | segmented | stimulated | analysed | valid_report |
|---|---|---|---|---|---|---|---|
| argon | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| astra | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| fable | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 |
| opus | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 |
| sol | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| terra | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
Per fault that occurred in a scored trial. Noticed is the grader's time-to-notice diagnostic (the fault's tracking assumption stated as "no" at or after onset); named in report is scored; cleared means the fault was no longer active in the final truth.
| agent | fault | trials | noticed (assumption → no) | median decisions to notice | worst | named in report | cleared by the end |
|---|---|---|---|---|---|---|---|
| argon | led_output_low | 3 | 3 / 3 | 0 | 0 | 0 / 3 | 3 / 3 |
| argon | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| argon | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| argon | room_light | 6 | 6 / 6 | 0 | 0 | 6 / 6 | 6 / 6 |
| argon | sample_cover_off | 6 | 0 / 6 | — | — | 5 / 6 | 6 / 6 |
| astra | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| astra | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| astra | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| astra | room_light | 6 | 6 / 6 | 0 | 0 | 6 / 6 | 6 / 6 |
| astra | sample_cover_off | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| fable | led_output_low | 2 | 2 / 2 | 0.5 | 1 | 0 / 2 | 2 / 2 |
| fable | led_power_loose | 2 | 0 / 2 | — | — | 0 / 2 | 2 / 2 |
| fable | projector_standby | 2 | 2 / 2 | 0.5 | 1 | 2 / 2 | 2 / 2 |
| fable | room_light | 4 | 4 / 4 | 0 | 0 | 4 / 4 | 4 / 4 |
| fable | sample_cover_off | 4 | 0 / 4 | — | — | 4 / 4 | 4 / 4 |
| opus | led_output_low | 2 | 2 / 2 | 0 | 0 | 0 / 2 | 2 / 2 |
| opus | led_power_loose | 2 | 0 / 2 | — | — | 0 / 2 | 2 / 2 |
| opus | projector_standby | 2 | 2 / 2 | 0 | 0 | 2 / 2 | 2 / 2 |
| opus | room_light | 4 | 4 / 4 | 0 | 1 | 4 / 4 | 4 / 4 |
| opus | sample_cover_off | 4 | 0 / 4 | — | — | 4 / 4 | 4 / 4 |
| sol | led_output_low | 3 | 3 / 3 | 0 | 0 | 2 / 3 | 3 / 3 |
| sol | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| sol | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| sol | room_light | 6 | 6 / 6 | 0 | 0 | 6 / 6 | 6 / 6 |
| sol | sample_cover_off | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
| terra | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| terra | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| terra | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| terra | room_light | 6 | 5 / 6 | 0 | 0 | 6 / 6 | 6 / 6 |
| terra | sample_cover_off | 6 | 0 / 6 | — | — | 6 / 6 | 6 / 6 |
One panel per agent on the same axes; one dot per scored trial (hover for the trial).