Longer SKILL.md Files Score Higher — Which Sits Awkwardly With Our Own Advice
Our rubric penalises bloated files. Our data says longer files score better. One of those needs a more careful sentence.
For 591 fully-evaluated skills we have both the SKILL.md text and its score. Splitting by file length:
| Quartile | Median length | Average score |
|---|---|---|
| Shortest 25% | 3,350 chars | 8.14 |
| Middle 50% | 7,755 chars | 8.57 |
| Longest 25% | 9,058 chars | 8.82 |
Pearson r between length and score: 0.24. A weak relationship, but consistent across the distribution — the longest files are not just a few outliers dragging an average.
This is awkward. The structure dimension explicitly rewards a lean SKILL.md with detail pushed into references/, and penalises "bloated single files". If length were a defect, the correlation should run the other way.
Why both can be true
Length is not the thing the rubric penalises. The rubric penalises unstructured length — a 900-line monolith with no references/ directory, where every detail loads on every activation.
A 9,000-character file is roughly 130-160 lines. That is a normal, well-organised SKILL.md: frontmatter, a trigger section, a workflow, boundaries, output format. It is longer than a 3,000-character file because it actually contains a workflow, not because it dumped an appendix inline.
The shortest quartile at 8.14 is the more revealing group. A 3,000-character SKILL.md frequently has a description and four bullet points — the skeleton of a skill without the operational detail that makes it usable. Length, in that band, is a proxy for "did the author actually write the steps".
The confound I have to disclose
Our description column stores the full SKILL.md, and for evaluated skills we append the evaluation rationale to it. That means part of the measured length is our own writing, not the author's.
A better-scored skill tends to get a longer rationale, which makes the correlation partly mechanical. The quartile gap is too large to be entirely explained by appended rationales — the shortest quartile's median is 3,350 characters, and rationales alone run 1,500-3,000 — but I cannot cleanly separate the two without re-measuring against the original repo files.
So treat 0.24 as an upper bound.
What to do with it
- Do not pad. The relationship is weak, and the rubric will catch inline bloat through the structure dimension.
- Do write the workflow. If your SKILL.md is under 4,000 characters, check whether it actually contains numbered steps an agent can execute. The short-file penalty is probably a thinness penalty wearing a length costume.
- Move detail, don't delete it. The
references/pattern exists so you can be thorough without being monolithic.
The full rubric is at methodology, including how structure is scored out of 15.