Skip to content

test tool

Use test while you write project rules, as xpd test does. It checks the valid and invalid examples of each rule with the same scanner as check, and compares the snapshots. It changes snapshot files only with update_snapshots.

test(options=null)
  • May change files
  • May overwrite what is there
  • A second call with the same arguments changes nothing more
  • Stays inside the project

Tags: authoring, rules.

The server gives the agent this description of the tool.

Check valid and invalid examples using xpd's scanner and compare snapshots.

Changes snapshots only when update_snapshots is true. Import, version and proof checks use the same rules as linting, including project rules.

Type: object or null. Optional, default null.

The project file, the test folder and filter, and what to do with snapshots.

Field Type Required Default Description
options.config string or null no null A project file to read instead of the nearest sgconfig.yml, relative to the project root.
options.test_dir string or null no null Read the test cases in this folder instead of the testConfigs of sgconfig.yml.
options.filter string or null no null Test only the rules whose id matches this regular expression.
options.skip_snapshots boolean no false Check only the valid and invalid examples, and no snapshot.
options.update_snapshots boolean no false Accept every changed snapshot and write it.
Field Type Required Default Description
cases list of object yes
cases[].id string yes
cases[].source string yes
cases[].status string yes
cases[].actual object or null no null
failed integer yes

The commands below make a project with the rule team/no-print, and the test file gives the rule one valid and one invalid case.

Terminal window
$ xpd new project --pack team --yes --format json | jq -c .item
"project"
$ xpd new rule team/no-print --pattern 'print($X)' --message 'Use the project logger.' --with-test --yes --format json | jq -c .id
"team/no-print"
rule-tests/team/no-print-test.yml
id: team/no-print
valid:
- "log(1)"
invalid:
- "import team\nprint(1)"

The call writes the snapshot of the invalid case.

{"options":{"filter":"^team/","update_snapshots":true}}

The result has cases, each with its id, source and status, and the failed count.

{"cases":"...","failed":0}

When no test case matches, the call fails. A rule that new just made has an empty case, so it fails this way until the case has examples.

xpd test --skip-snapshot-tests -- exited with 2: xpd: no cases matched

skip_snapshots and update_snapshots together fail with "Choose either skipping or updating snapshots."