Playbooks/xor-pb-003
xor-pb-003
Forensics, detection and threat hunting, live network defence, secure coding: the half of the job public agent benchmarks almost entirely omit.
- Missions
- 9
- Runs each
- 10
- Runs total
- 90
- Budget
- 1800s
- Hints
- none
- Harness
- openhands
The file
This is what the runner works through. Every model you name, on the command line or in the file's own models list, is taken across every mission below, 10 times each: 90 runs per model, launched, graded and recorded without further input.
This is the file verbatim, as the skill ships it. Copy it, change the list, and you have your own benchmark. The harness line is the one to leave alone: pinning it is what makes two runs comparable.
Missions 9
| Mission | Specialty | What it involves |
|---|---|---|
| process-of-elimination | detection | Windows event-log 4688 |
| breachpoint | detection | KQL / Defender threat hunt |
| severed-stream | forensics | corrupted PDF recovery |
| verify-you-are-human | forensics | Windows ActivitiesCache.db |
| pipe-dream | forensics | Windows EVTX / RPC named pipes |
| molten-perimeter | forensics | pfSense VMDK/ZFS appliance |
| inline-sentinel | network-defense | suricata/nfqueue live IR |
| need-to-know | secure-coding | fix Flask broken access control |
| jumbo-overrun | secure-coding | x86-64 asm PCAP parser |
4 specialties covered. Where the missions come from →
Results
A playbook is the script, not the score. Nothing on this page is a result: it is the definition you would run, and it produces a different set of numbers every time anyone runs it, on their own models and their own machine.
We have run this one ourselves and written it up, with the conditions, the caveats and every graded run behind it: The defensive half is harder →