6 agent(s) · 18 scored trial(s) · built from whatever records are in the runs directory
| # | agent | all pass | scored | harness errors | tasks | mean end score (partial credit) | range (partial credit) | cost / trial | total cost | input tok / trial | output tok / trial | reasoning tok / trial |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | opus claude-opus-5-5 | 2 / 3 67% | 3 | — | 1 | 0.99 | 0.971–1 | $2.1070 | $6.3211 | 3840879 | 33909 | 0 |
| 2 | argon google/gemini-4-argon | 2 / 3 67% | 3 | — | 1 | 0.983 | 0.949–1 | $2.8301 | $8.4903 | 7423859 | 64343 | 34186 |
| 3 | sol gpt-6.1-sol | 2 / 3 67% | 3 | — | 1 | 0.667 | 0–1 | $0.4573 | $1.3720 | 2108674 | 10437 | 4123 |
| 4 | astra gpt-6-astra | 1 / 3 33% | 3 | — | 1 | 0.99 | 0.971–1 | $3.4665 | $10.3995 | 2262920 | 10089 | 3366 |
| 5 | sonnet claude-sonnet-5-5 | 1 / 3 33% | 3 | — | 1 | 0.983 | 0.968–1 | $1.4488 | $4.3463 | 5167717 | 47353 | 0 |
| 6 | terra gpt-6.1-terra | 0 / 3 0% | 3 | — | 1 | 0.333 | 0–0.998 | $0.2213 | $0.6638 | 1910358 | 11806 | 5685 |
Ranked by all-pass: a session passes only if every check holds and nothing unsafe happened. The mean end score is partial credit, shown for context: a session can score high while failing a required check. These are the grades stored in the records.
Sessions that passed (every check holds, nothing unsafe) out of the scored sessions, per agent and task.
| agent | t4_takeover |
|---|---|
| argon | 2 / 3 67% |
| astra | 1 / 3 33% |
| opus | 2 / 3 67% |
| sol | 2 / 3 67% |
| sonnet | 1 / 3 33% |
| terra | 0 / 3 0% |
| agent | t4_takeover |
|---|---|
| argon | 0.983 n=3 |
| astra | 0.99 n=3 |
| opus | 0.99 n=3 |
| sol | 0.667 n=3 |
| sonnet | 0.983 n=3 |
| terra | 0.333 n=3 |
Scored trials that reached at least each stage of the task's progress ladder
(grade.progress_stage).
| agent | rig_checked | calibrated | flooded | segmented | stimulated | analysed | valid_report |
|---|---|---|---|---|---|---|---|
| argon | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| astra | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| opus | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| sol | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| sonnet | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| terra | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
Per fault that occurred in a scored trial. Noticed is the grader's time-to-notice diagnostic (the fault's tracking assumption stated as "no" at or after onset); named in report is scored; cleared means the fault was no longer active in the final truth.
| agent | fault | trials | noticed (assumption → no) | median decisions to notice | worst | named in report | cleared by the end |
|---|---|---|---|---|---|---|---|
| argon | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| argon | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| argon | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| argon | room_light | 3 | 3 / 3 | 0 | 3 | 3 / 3 | 3 / 3 |
| argon | sample_cover_off | 3 | 0 / 3 | — | — | 3 / 3 | 3 / 3 |
| astra | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| astra | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| astra | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| astra | room_light | 3 | 3 / 3 | 2 | 2 | 3 / 3 | 3 / 3 |
| astra | sample_cover_off | 3 | 0 / 3 | — | — | 3 / 3 | 3 / 3 |
| opus | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| opus | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| opus | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| opus | room_light | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| opus | sample_cover_off | 3 | 0 / 3 | — | — | 3 / 3 | 3 / 3 |
| sol | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| sol | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| sol | projector_standby | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| sol | room_light | 3 | 3 / 3 | 2 | 2 | 3 / 3 | 3 / 3 |
| sol | sample_cover_off | 3 | 0 / 3 | — | — | 3 / 3 | 3 / 3 |
| sonnet | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| sonnet | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| sonnet | projector_standby | 3 | 3 / 3 | 0 | 0 | 2 / 3 | 3 / 3 |
| sonnet | room_light | 3 | 3 / 3 | 1 | 1 | 3 / 3 | 3 / 3 |
| sonnet | sample_cover_off | 3 | 0 / 3 | — | — | 3 / 3 | 3 / 3 |
| terra | led_output_low | 3 | 3 / 3 | 0 | 0 | 3 / 3 | 3 / 3 |
| terra | led_power_loose | 3 | 0 / 3 | — | — | 0 / 3 | 3 / 3 |
| terra | projector_standby | 3 | 3 / 3 | 0 | 0 | 2 / 3 | 3 / 3 |
| terra | room_light | 3 | 3 / 3 | 2 | 2 | 3 / 3 | 3 / 3 |
| terra | sample_cover_off | 3 | 0 / 3 | — | — | 3 / 3 | 3 / 3 |
One panel per agent on the same axes; one dot per scored trial (hover for the trial).