Tasks

List tasks

Use gage task list to list available tasks.

gage task list

Command help:

Show available tasks

Usage: gage task list [OPTIONS]

Options:
  -p, --path <PATH>  Path to find tasks
  -h, --help         Print help

Run a task

Use gage task run or the short form gage run to run a task from the command line.

gage run TASK

Command help:

Run a task

Usage: gage task run [OPTIONS] [TASK]

Arguments:
  [TASK]
          Task to run

Options:
  -i, --input <INPUT>
          Task input

  -T <TASK_ARG>
          Task argument NAME=VALUE (may use more than once)

  -m, --model <MODEL>
          Task model

          If not specified, environent variables GAGE_MODEL and
          INSPECT_EVAL_MODEL are used. Otherwise the model defined by
          the task itself is used.

      --target <TARGET>
          Expected output

      --score
          Score the result (implied when --target is specified)

  -t, --tag <TAG>
          Task tag (may use more than once)

  -p, --path <PATH>
          Path to find tasks

      --log-dir <LOG_DIR>
          Inspect log dir

  -y, --yes
          Don't prompt for input

          --input is required when this option is used.

  -h, --help
          Print help (see a summary with '-h')

Evaluate a task

Use gage task eval or the short form gage eval to evaluate a task.

gage eval TASK

This is a dialog oriented version of inspect eval. Use inspect eval to access any options that aren’t supported by Gage. If you’d like an option added to gage eval, please open an issue.

Command help:

Evaluate tasks

Usage: gage task eval [OPTIONS] [TASK]...

Arguments:
  [TASK]...  Tasks to evaluate

Options:
  -m, --model <MODEL>      Task model (may be used more than once)
  -T <TASK_ARG>            Task argument NAME=VALUE (may use more than once)
  -d, --dataset <NAME>     Evaluation dataset
  -l, --limit <N>          Limit the number of samples to evaluate
  -s, --sample <ID>        Task model (may be used more than once)
      --sandbox <TYPE>     Sandbox environment type
      --shuffle            Suffle sample order
  -e, --epochs <N>         Number of times to evaluate dataset
  -p, --path <PATH>        Path to find tasks
  -y, --yes                Don't prompt to for input
      --log-dir <LOG_DIR>  Log directory
  -h, --help               Print help

Show task info

Use gage task info to show information about a task.

gage task info TASK

Command help:

Show task info

Usage: gage task info [OPTIONS] <TASK>

Arguments:
  <TASK>  Task name

Options:
  -p, --path <PATH>  Path to find tasks
  -h, --help         Print help