What the September and October Batches Look Like

Import batches are not a neutral detail. They determine how much discovery time a skill has had, and they move the averages accordingly.

The directory was built in two waves:

The October batch scores half a point higher and has a third of the downloads. Both numbers are real, and neither means what it looks like.

The score gap is probably selection

October's imports were chosen with more context than September's. By the second wave we knew which repositories produced clean SKILL.md files, so the hit rate of usable, well-formed entries went up.

There is also a subtler effect. Skills evaluated later were evaluated by a pipeline that had been through several rounds of prompt and parsing fixes. The model has not changed, but the harness has. We measured harness effects before — a generation-budget change between two evaluation runs produced score movement that was not the model's judgment.

We have not re-evaluated the September batch with the current harness, so we cannot say how much of the 0.5 is quality and how much is instrumentation.

The download gap is time

October skills have had days, not weeks, to be discovered. A third of the downloads is what you would expect from a third of the exposure window. There is no evidence here that newer imports are less popular; there is only evidence that they are newer.

This is the same point as the zero-download analysis: the download count measures exposure, and exposure is dominated by age and placement.

What the batches tell us about the pipeline

The most useful thing in the two-batch comparison is not in the table. It is that a batch can be imported without being fully evaluated, and the difference is visible only if you check the dimensions.

A subset of imports — 174 in the current directory — carry placeholder values: a score around 8, but structure, content, and engineering set to zero because the evaluation never ran. Those entries look normal in a list and broken on a skill page. We are treating that as a pipeline defect and writing about it separately.

The lesson generalizes: whenever you add data in batches, check the batch for completeness before comparing it to anything else. A missing evaluation and a bad evaluation look identical in an average.

What to do with this

• Do not compare across batches without controlling for age. The October batch is younger, not worse.
• Record which harness produced which score. Without that, every re-evaluation looks like a quality change.
• Verify completeness at import time. A placeholder is not a neutral default; it is a wrong number that will be averaged with right ones.