Why 174 Skills Show Zeros in Three Dimensions
We published a number that was a placeholder, and a fifth of the directory carries it. This is the least flattering post we have written, and it is the most useful one.
> Update, 2026-10-22: This backlog has been cleared. All 174 entries now carry full six-dimension scores and valid rationales; the unevaluated state described below no longer exists in the directory. The post is left as written, because the explanation of how the defect happened is the part worth keeping.
Open any of 174 skill pages and you will see a contradiction: an overall score around 8.0, next to three dimension bars at zero.
| Group | Skills | Share |
|---|---|---|
| Fully evaluated | 591 | 77.3% |
| Placeholder dimensions | 174 | 22.7% |
For the affected rows, structure, content, and engineering are all exactly 0.0. That is not a measured value — it is the default the import pipeline writes when it has not run an evaluation.
How it happened
Our import process has two stages: ingest a repository, then evaluate each SKILL.md against the six-dimension rubric. Ingestion fills a placeholder score so the record is valid; evaluation overwrites it.
When a batch is ingested and not evaluated — interrupted, skipped, or imported before the evaluator was wired up — the record keeps the placeholder. Nothing fails. The list view shows a normal skill; only the dimension bars reveal the gap.
The affected rows cluster by source repository, which is the signature of a batch:
| Repository | Affected skills |
|---|---|
| gamedev-skills/awesome-gamedev-agent-skills | 74 |
| seranking/seo-skills | 32 |
| jykim/claude-obsidian-skills | 20 |
| agiprolabs/claude-trading-skills | 15 |
| Others (5 repositories) | 33 |
By import date: 128 arrived on 21 October, 21 on 6 October, 15 on 20 September, 10 on 18 October.
Why it is worse than it looks
Two harms.
The visible one. A skill page showing an 8.0 overall score with three zero bars is incoherent. A reader who notices will reasonably stop trusting every other number on the site.
The invisible one. Every aggregate we publish silently mixes these rows in. The 174-strong set has an average score of 7.97, close enough to the true average of 8.53 that the overall number barely moves. But the dimension averages are badly distorted:
| Dimension | All 765 | Fully evaluated 591 |
|---|---|---|
| Structure | 6.47 | 8.37 |
| Content | 6.86 | 8.88 |
| Engineering | 6.59 | 8.53 |
A reader looking at a site-wide "structure averages 6.47" would conclude the ecosystem writes badly organised skills. Actually, the ecosystem averages 8.37; the low figure is our missing evaluations.
What we are doing
- Stopping the placeholder from presenting as a score. An unevaluated skill should show "not yet scored", not a number.
- Evaluating the backlog. 174 skills is tractable in batches, and the pipeline now works end to end.
- Making missing data loud. Zero is a legitimate score, so it cannot double as "unknown". The field needs a distinct state.
This post exists because the alternative — quietly fixing it and saying nothing — would leave every number we published this week standing on a mixed dataset. The correction is part of the record.
What to take from it
If you run a directory with any kind of scoring, check for complete-looking records that were never actually processed. The failure mode is universal: a default value that is plausible enough to survive every sanity check.
The test is simple — take the dimensions of your records and ask whether any of them could only have come from a default. If yes, you do not know how much of your data is real.