Grade a lab part

spimbot grade runs a lab part's match with your bot as bot 0, then grades bot 0:

spimbot grade bot.s --part part1 --seed 5

Seed: 5
Test                  Points  Details
Leave your territory     5/5
Complete a trail         0/5  Bring a trail back onto your own territory
Own 100 tiles         0.9/10  Tiles owned at the end, home base included: 9
Total                 5.9/20

A test short of full marks shows details: what earns its points, or your bot's count for a progress test.

The match runs under the part's mode and cycle limit: 300,000 cycles for onboarding and 10,000,000 for the other parts. It runs without tournament rules, so syscalls and exception handlers work. --seed picks the seed, random by default, and the text output starts with the seed in use. --against adds an opponent as bot 1:

spimbot grade bot.s --against opponent.s --part qualification --seed 11

To watch the graded match, open the debugger with the same seed, mode and cycle limit:

spimbot bot.s opponent.s --mode qualification --seed 11

grade lists every option.

Tests

MMIO registers describes the registers the tests name.

Part	Test	Points	Earns its points
`part1`, `part2`	Leave your territory	5	Pass or fail: paint a trail tile off your own territory
`part1`, `part2`	Complete a trail	5	Pass or fail: bring a trail back onto your own territory
`part1`, `part2`	Own 100 tiles	10	Progress: tiles owned at the end, target 100
`part2`	Request a puzzle	10	Pass or fail: write a valid buffer address to `REQUEST_PUZZLE`
`part2`	Acknowledge a puzzle	10	Pass or fail: write any value to `REQUEST_PUZZLE_ACK`
`part2`	Solve a puzzle	40	Pass or fail: write the address of a correct solution to `SUBMIT_SOLUTION`
`onboarding`	Set the timer	5	Pass or fail: write a cycle to `TIMER` to request an interrupt
`onboarding`	Acknowledge the timer interrupt	5	Pass or fail: write to `TIMER_ACK`
`onboarding`	Move 20 pixels down	10	Progress: pixels the bot's center ends below its base's center, target 20
`qualification`	Own 100 tiles	60	Progress: tiles owned at the end, target 100
`qualification`	Complete 5 trails	20	Progress: trails completed, target 5
`qualification`	Solve 10 puzzles	20	Progress: puzzles solved, target 10

A pass or fail test gives all its points once the bot does what it asks, at any point in the match. A progress test gives points × min(count, target) / target, from the count at the end of the match. Tile counts include the bot's 9-tile home base.

Produce PrairieLearn's results.json

--json prints the grade as the results.json object that PrairieLearn's autograder reads, and nothing else on standard output. --image adds the final map as a PNG:

spimbot grade bot.s --against opponent.s --part qualification --json --image > results.json

The object holds a score from 0 to 1 and each test's points. grade --json lists every field.

grade exits with code 0 whatever the marks.