The Data Science Skills Have Perfect Structure and No Guardrails on Conclusions

The two data-science skills in our directory are structurally the cleanest things we have evaluated. That is exactly why the interesting question about them is no longer whether the agent will compute the right number — it is whether the right number will mean what someone says it means.

Perfect scaffolding

ml-pipeline — 9.0 overall — designs production-grade ML infrastructure: experiment tracking, reproducible runs, configuration management. pandas-pro — 9.0 — handles DataFrame operations: the merges, group-bys, reshapes, and dtype traps that eat data-analysis sessions. Their dimension lines are nearly identical, and the striking part is at the top: Structure 10, Engineering 10, Security 10 on both. Out of 815 skills, a clean triple-ten is rare — the overwhelming majority of our directory loses at least one of those to shortcuts.

Structure at 10 means the SKILL.md files are the right size, with reference material offloaded and progressive disclosure done properly. Engineering at 10 means the operational guidance is real: versioned data, pinned environments, tracked experiments, verification steps. These skills were written by people who have run pipelines in production and been burned by the things that burn you.

The weak dimension is the front of the funnel: trigger quality, 7.5 for ml-pipeline and 8.0 for pandas-pro. Both descriptions lead with capability ("performs pandas DataFrame operations") more than with the situations that should summon them. For a skill competing with the model's own considerable built-in pandas knowledge, the trigger has to argue for why the skill should win that contest — and it currently does not.

The third skill is the tell

The most interesting member of this set is spec-miner — 9.0 — a reverse-engineering specialist that extracts specifications from existing codebases. Its presence in the same family points at the actual workflow: mine the requirements out of legacy code, build the pipeline, manipulate the data. Three skills, one arc from "what does this system even do" to "here is the number."

That arc is where the risk lives. Everything upstream of the number — the pipeline, the transformations — is now guarded by triple-ten scaffolding, verification steps, and reproducible runs. Everything downstream is one human sentence: "the model shows a 12% lift." The scaffolding guarantees the number was computed correctly. Nothing guarantees the number means that.

Numbers do not lie; conclusions do

Data science has a failure mode that code does not have. A broken build fails loudly and immediately. A subtly wrong analysis — a leaky feature, a survivorship-biased sample, a metric that will not replicate — runs to completion, prints a confident number, and gets put in a deck. The better the tooling around the analysis, the more authoritative the wrong conclusion looks. Perfect reproducibility means you can reproduce the error exactly.

This is why the pandas-pro and ml-pipeline scores make me more careful, not less. An agent with a triple-ten pipeline skill will produce output that looks more rigorous than most human analysts' work — tracked, versioned, reproducible. If the interpretation is wrong, the polish actively hides it. The skills do include verification steps, and their engineering scores reflect that honestly — but verification as written checks the computation, not the claim.

What is missing

The missing skill in this family is a conclusion auditor: something that interrogates the finding rather than the run — checks for leakage, asks what would falsify the result, states the assumptions the number depends on, and refuses to let "significant" appear in a sentence without the sample size next to it. Trigger quality is the cheap thing to fix in the existing two; that is a description rewrite. The conclusion gap is a design gap, and it is the one worth caring about.

Until it exists, the correct posture with these skills is to let them do everything up to the number and then slow down at exactly the moment they make you feel fastest. The scaffolding is perfect. The inference is still on you.