Distributions over Best Fits

Many parameter settings, and many model structures, explain the same data equally well. A single best fit hides all the others. The useful output is the range of models that fit, what they share, where they disagree, and which alternatives failed. That record, not the winning model, is where the insight is (Anthropic — Teaching AI to Build Scientific Models).

Two levels of the same problem

• Parameters. Different combinations of gate settings in a nerve-cell model give near-identical output, like many orders summing to the same restaurant bill. Bayesian inference returns all of them as a distribution. Simulation-based inference makes that practical for any runnable simulator: ~100,000 simulations train a network that then returns the plausible range in seconds.
• Structure. The same is true "one level up": differently shaped equations fit equally well. An LLM agent run twice may return two different, equally good models. The fix is the same: propose many, weight by support (penalizing models that need a lot of flexibility to fit), keep the unlikely ones too, and iterate.

What a distribution tells you that one model can't

• Shared features are what the data favors. If every surviving model has it, believe it.
• Disagreement marks thin evidence and points at the next experiment.
• Interpretation needs the failures. Hodgkin and Huxley's four-particle hypothesis carried weight only because lower powers had failed. With one model, "it's easy to read meaning into parts that are not" constrained.
• The best experiment rules out the most models, not the one a scientist finds most interesting. Climate ensembles and LHC exclusion maps already work this way.

The process carries the insight

Hodgkin and Huxley "understood what the model was telling them because they had built it themselves, and they knew what hadn't worked." When AI does the building in an afternoon, the builder's knowledge of the dead ends doesn't arrive automatically. The AI must hand it over explicitly: which parts are pinned down, which are open, which alternatives failed, what new puzzles appeared. This is a concrete form of the worry in Critical Thinking Is Domain-Bound: offloading the work can offload the understanding that came with doing it.

Limits

• The authors' own caveat: model-space agents have mostly rediscovered known models; evidence on open problems is sparse.
• Parsimony still matters. Hodgkin and Huxley chose the fourth power over higher powers that fit slightly better because the improvement wasn't worth "the additional complication." A distribution doesn't remove the judgment call; it makes it visible.
• At large scale (whole-brain models) there may be no single correct model at all (Box: "All models are wrong but some are useful").

Connections

• Falsification Is Asymmetric — the same epistemics at model scale: what you eliminate is more informative than what fits, and experiments should be chosen to eliminate.
• Verification Requires Domain Knowledge — a list of failed alternatives and unconstrained parts is something a reviewer can check without rebuilding the model. Like Provable Inference, it narrows how much knowledge verification needs.
• AI-Accelerated Science — fitting is accelerating fastest; interpretation is not.
• AI Use Tracks AI Capability — fit has an objective; "understanding" doesn't, which is why interpretation is hardest to hand to AI.
• The Bottleneck Shift — after build and constrain move to AI, interpretation and experiment choice are the remaining human-heavy steps.
• Agent Memory — tension: memory research favors salience (keep little), while this page says the discarded alternatives carry the insight. See Method Reflection T5.
• Method Reflection — T4: if insight comes from the process, an auto-ingest that hands over only the product skips it.

Sources

• Anthropic — Teaching AI to Build Scientific Models — the bill analogy, SBI, model-space search, Hodgkin–Huxley, "the process of constructing it… provides the insight."