ct run monitor2
Score a finished eval2 log with more monitors, writing a monitored copy
Usage
ct run monitor2 [OPTIONS] SOURCE
Arguments
| Argument | Type | Required |
|---|---|---|
SOURCE | text | Yes |
Options
| Option | Description |
|---|---|
-m, --monitors TEXT | The monitors, as YAML: a list of packaged prompt names or mappings of Monitor fields, e.g. '[simple, {prompt: strong, model: judge, cot_level: cot, samples: 2}]'. model names a role bound with --model-role. simple asks about an action, trajectory about a whole run, so pick the one --kind is for. [required] |
--kind [action|trajectory] | Score every action before it ran, or the whole trajectory once. [default: action] |
--model-role TEXT | Bind a model role, as name=model or name=<JSON model spec> (multiple allowed). A monitor answers on the role its model names, monitor by default, falling back to trusted. |
-o, --save-dir PATH | Where to write the monitored copy (default: ./data/eval2_monitoring). |
--help | Show this message and exit. |