Fewer Than Half of Skills Check Their Own Work
The workflow tells the agent what to do. Almost none of them tell it how to know it worked.
Three counts across the 765-skill corpus:
| Safety behaviour | Skills | Share |
|---|---|---|
| Mentions verification or checking the result | 328 | 42.9% |
| Describes failure handling | 60 | 7.8% |
| Requires confirmation before an irreversible action | 14 | 1.8% |
These are the behaviours the rubric weights most heavily — verification sits inside workflow design (25 points), confirmation and cost controls inside security (20 points, with veto power). The corpus is thinnest exactly where the consequences are largest.
What verification looks like when it is present
The good versions are concrete and falsifiable:
> After editing, re-read the file and confirm the changed line appears exactly once.
That is checkable by the agent, in the same session, without another human. It is worth more than ten sentences of "ensure quality".
The common bad version names a goal without an action:
> Make sure the final deck looks professional.
An agent cannot do that. It can only assert it.
The failure-handling gap
Only 60 skills say what happens when a step does not work. The rest assume the happy path — API responds, file exists, package installs, test passes.
That assumption is usually wrong. In our own experience building the site, the failures were almost never in the documented path:
- a network call that returned HTML instead of JSON,
- a build that silently uploaded a
.DS_Store, - an evaluator that wrote
NaNbecause one field was dropped in a mapping.
None of these would be caught by a workflow that only describes success. A single line — "if the response is not JSON, stop and report the status code" — would have saved hours.
The confirmation gap is the serious one
Fourteen skills out of 765 ask before doing something that cannot be undone. The corpus contains rm -rf in three files, sudo in five, and curl in thirty-one. Many of those commands are benign, and some skills do gate them in prose the scanner did not catch. But the ratio is telling: agents are being handed destructive capability far more often than they are being told to ask first.
A confirmation rule does not need to be elaborate:
> Before deleting files or force-pushing, show the exact command and wait for explicit approval.
One sentence. It is the highest-value sentence in most engineering skills, and 98.2% of the corpus does not have it.
A three-line checklist
If you are editing a SKILL.md, add these if they are missing:
- Verify: one concrete check the agent can run after producing the output.
- Fail: what to do when the check fails — retry, stop, or ask.
- Confirm: what the agent must never do without approval.
Together they take less than fifty words. They are also the difference between a skill that assists and a skill that improvises.