falsify-the-problem Checked
Installation
Copy the prompt below and send it to your AI assistant (e.g. Claude Code) — it will follow the instructions and install automatically.
Install the "falsify-the-problem" skill by following the instructions at https://skill123.me/install/falsify-the-problem.
Run in your terminal — downloads and installs to ~/.claude/skills/.
curl -fsSL https://skill123.me/install/falsify-the-problem.sh | bash
Download the zip and extract it into your skills directory (e.g. ~/.claude/skills/), then restart your session.
⬇ Download v1.0.0 · 343 KBNot live-tested yet — this skill is rated on static analysis only.
About this skill
Challenge the current problem formulation before solving it. Use when explicitly invoked, or when meaningful framing uncertainty and non-trivial intervention cost or risk coexist; do not auto-activate for verified diagnoses or mechanical edits.
falsify-the-problem
> Falsify the problem before solving it.
Answer **Are we solving the right problem?** Challenge the current formulation,
seek disconfirming evidence, and hand the result back to the host agent.
Do not design or implement the solution inside this skill.
Use the same protocol across domains; examples are not domain-specific workflows.
Documentation
Challenge the current problem formulation before solving it. Use when explicitly invoked, or when meaningful framing uncertainty and non-trivial intervention cost or risk coexist; do not auto-activate for verified diagnoses or mechanical edits.
falsify-the-problem
> Falsify the problem before solving it.
Answer **Are we solving the right problem?** Challenge the current formulation,
seek disconfirming evidence, and hand the result back to the host agent.
Do not design or implement the solution inside this skill.
Use the same protocol across domains; examples are not domain-specific workflows.
1. Decide whether to enter
- Explicit invocation of
falsify-the-problem: always enter. Strong existing
evidence can justify a brief KEEP without alternatives or a new test.
- Automatic activation: require BOTH meaningful formulation uncertainty AND
non-trivial intervention cost or risk. Neither alone is enough.
- Useful signals include solution-shaped requests, a hypothesis treated as fact,
repeated failed attempts, and costly or hard-to-reverse proposed changes.
Signals are reasons to check the gate, not unconditional triggers.
- Usually do not auto-activate for a verified diagnosis, obvious syntax or
compilation error, typo, translation, formatting, mechanical transformation,
deterministic local edit, or very cheap reversible intervention.
- Mere logical possibility of another explanation is insufficient.
BYPASS is an activation decision, never an evidence verdict. When bypassing,
leave the original task with the host; do not manufacture an adversarial report.
If explicitly invoked on a supported diagnosis, use KEEP, not BYPASS.
2. Separate the claims and preserve their sources
Distinguish these five objects, even when the user combines them in one sentence:
| Object | Meaning | Example |
| --- | --- | --- |
| Observation | Something observed, measured, or reported | User reports P99 rose from 300 ms to 2.8 s |
| Interpretation | Meaning assigned to an observation | The database is slow |
| Causal Hypothesis | A specific mechanism proposed to explain it | Lock contention delays requests |
| Problem Formulation | What kind of problem is being solved | Database service time dominates the slow tail |
| Solution | A proposed intervention | Add Redis |
Observation != Interpretation != Causal Hypothesis != Problem Formulation != Solution.
For each consequential observation, retain the source and relevant scope:
tool-observed: name the file, command, trace, or other inspected artifact.user-reported: retain attribution, including measurements supplied by a user.document-reported: identify the document and its claim.experimentally observed: identify the experiment and conditions.measurement-derived: identify inputs, transformation, and sampling limits.
View full documentation
These labels can combine. Reading a report with a tool verifies what the report
says, not that its underlying claim is true. A local simulated fixture is evidence
about that fixture, not independent verification of a real incident.
Never upgrade user reports into independently verified facts.
Record uncertainty about coverage, freshness, comparability, or missing data
where it could change the conclusion. Do not fabricate observations or execution.
State the **Current Problem Formulation** explicitly. If inferred from a proposed
solution, mark it as inferred; do not attribute it to the user as a stated belief.
Keep this original formulation as the reference for the overall verdict through
this pass. A replacement is REFORMULATE, not a silently relabeled KEEP.
Preserve claim strength: a possible contributor, a primary explanation, and an
exclusive explanation are different claims. Do not infer exclusivity from a
proposed solution or refute a stronger claim than the request requires.
3. Build materially different live formulations
Ask what would change the kind of problem being investigated. Competing causes
within one formulation are not automatically competing problem formulations.
Two formulations should materially differ in at least one consequential dimension:
failing variable, mechanism class, abstraction level, system boundary, expected
evidence, investigation path, or intervention target.
If two formulations predict nearly the same evidence and lead to nearly the same
investigation path, they are probably superficial variants. Merge them.
For example, weak embeddings, small embeddings, and old embeddings usually belong
to one representation-quality formulation. Retrieval construction, ranking,
generation grounding, evaluation validity, and query distribution can change the
framing when they predict meaningfully different evidence.
Keep only plausible, decision-relevant alternatives. Do not mechanically fill a
quota. Let evidence and the pending decision determine the live set; more
candidates are not evidence of better reasoning. Presentation limits in section 9
do not justify dropping a materially different, decision-relevant candidate.
Candidates need not be mutually exclusive. If several mechanisms coexist, test
which framing explains the material failure at the relevant scope; do not force a
winner from evidence that supports a mixed or narrower formulation.
4. Find load-bearing assumptions and distinct predictions
Prioritize assumptions whose failure would materially weaken or collapse a live
formulation. Avoid exhaustive lists or generic questions about everything.
Include measurement validity only when it bears on the decision.
For each important live formulation, connect:
Formulation -> load-bearing assumption -> observable prediction -> contrary evidence
A prediction must distinguish live formulations, not restate their names.
Example: a service-time framing predicts elevated service spans in slow requests;
a capacity framing predicts rising pre-service wait with concurrency; a measurement
framing predicts disagreement between raw samples and reported aggregation.
State the population, comparison, and meaningful difference when available.
Do not invent numerical thresholds without a basis; explain qualitative criteria.
Before choosing the test, make one lightweight **Shared-Frame / Coverage Check**:
do the live formulations all rely on one consequential, untested premise about
the relevant boundary, measurement path, causal layer, population, time scope,
or optimization target? Select what matters here; do not audit a category checklist.
If that premise failing would undermine the whole set, and a plausible alternative
would change the decision and predict different evidence, add at most **ONE
Outside-Frame Challenger**. Name the shared premise and its contrasting prediction
briefly when decision-relevant. Generic "measurement could be wrong" is insufficient;
ground the challenger in this task. If none is warranted, proceed without one.
Do not chain coverage audits or expand the set merely to satisfy a quota.
5. Select ONE Primary Discriminating Test for this round
When beginning an investigation round under unresolved uncertainty, choose
**exactly ONE Primary Discriminating Test**. This is one per round, not one forever.
Choose the cheapest currently available test with substantial discriminatory
information relative to cost, risk, and execution time.
Ask: would plausible different outcomes materially change the relative support
of the live formulations? If not, choose a better test.
An outcome predicted by several live formulations may localize the failure without
distinguishing them. Prefer a matched comparison that separates the relevant
factor; otherwise retain those formulations rather than assigning the outcome
to one by default.
Specify:
- **Test:** one bounded comparison or observation, its source, and its scope.
- **Why this first:** which live formulations it separates and why its value
exceeds the cost of the next plausible option.
- **Possible outcomes:** what would strengthen, weaken, or kill which candidates.
Include inconclusive or mixed outcomes where plausible.
- **Remaining uncertainty:** what this test cannot establish.
One test can require several retrieval or computation steps if they serve one
predeclared comparison and
Score breakdown
Assessment methodology
Version history
Imported from ordinary-s/falsify-the-problem@master