What a Placeholder Row Does to Your Averages
A single bad default can corrupt an average by two points while leaving the headline number looking fine. Here is the arithmetic.
> Update, 2026-10-22: This backlog has been cleared. All 174 entries now carry full six-dimension scores and valid rationales; the unevaluated state described below no longer exists in the directory. The post is left as written, because the explanation of how the defect happened is the part worth keeping.
We have 765 rows in the skills table. 174 of them were imported and never evaluated: their structure, content, and engineering values are exactly zero.
The damage, dimension by dimension:
| Dimension | All 765 rows | Fully evaluated 591 | Distortion |
|---|---|---|---|
| Structure | 6.47 | 8.37 | −1.90 |
| Content | 6.86 | 8.88 | −2.02 |
| Engineering | 6.59 | 8.53 | −1.94 |
| Security | 9.21 | 9.69 | −0.48 |
| Workflow | 8.11 | 8.20 | −0.08 |
| Trigger | 7.83 | 7.72 | +0.11 |
| Overall score | 8.40 | 8.53 | −0.13 |
The headline number barely moves
The overall score drops by 0.13 points — small enough to pass unnoticed, and small enough that every published aggregate would have looked plausible.
The reason is that the placeholder rows carry a plausible score (average 7.97) rather than zero. Whoever designed the default chose a reasonable-looking value, which is exactly what made it invisible.
The dimension averages collapse
The three zeroed dimensions move by roughly two points each. A reader told that "structure averages 6.47 across the ecosystem" would draw a strong conclusion about how skill authors write. The true figure is 8.37. The conclusion would be backwards.
This is the practical hazard of missing data: it does not look missing, it looks like a measurement. And it distorts exactly the columns where it appears, while leaving the summary statistic intact.
The test that catches it
Before computing any average, ask of each column: could this value have come from a default?
- A dimension score of exactly 0.0 across several columns is a default, not a measurement — real evaluations produce decimals like 8.3 and 9.1.
- A description matching
A skill from <repo>is a fallback string. - A score of exactly 8.0 with no rationale is a placeholder.
In all three cases the giveaway is suspicious roundness or repetition. Real data is lumpy; defaults are uniform.
The fix, in order of value
- Use null, not zero, for "unknown". Zero is a measurement. The absence of a measurement needs a different state, and every aggregation must decide explicitly what to do with it.
- Make the default loud. An unevaluated record should be visibly incomplete in the UI, not silently presentable.
- Run a completeness report with every batch. Count rows with null dimensions, fallback strings, and placeholder scores; refuse to publish aggregates that include them.
None of this is specific to skill directories. It is the standard data-quality discipline that a fast-moving importer tends to skip, and the cost is a published statistic that is wrong in the direction of looking reasonable.