Repairing a model instead of retraining it
MIT researchers had one AI model find and remove a fault in another by editing 0.016% of its weights. What that shows, and what it does not.
- Research
- MIT CSAIL, published 2 November 2026; not run or reproduced by Dscvry
- The edit
- 131,072 weights, 0.016% of Qwen3.5-0.8B, with no training
- What it fixed
- Reasoning loops that stopped the model finishing its answers
- What it did not do
- Improve Gemma 3 270M, a model that rarely loops
A small language model kept failing maths problems, mostly because it started repeating itself and ran out of room before reaching an answer. Researchers at MIT CSAIL asked a frontier AI model to make it better at maths by changing its weights directly, with no retraining. The AI found the looping, traced it inside the model and edited 0.016% of its weights. Its score on a standard maths test rose from 7.9% to 48.1%.
The research, published on 2 November 2026 by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, is an early experiment. It raises a practical question: could fixing some models become a matter of diagnosing one defect and correcting it, rather than rebuilding the model?
What is being changed
A neural network is a very large set of numbers, usually called weights. Together they decide how the model turns an input into an output. No single weight holds a fact or a rule; behaviour comes from how millions of them combine. Normally, training sets them, a little at a time, over a great many examples.
The researchers call their idea weight-space programming: a general-purpose AI model reads a network’s weights, reasons about what they do, and writes new values directly. They tried it on small networks, then on language models.
| The usual route | In this experiment | |
|---|---|---|
| Find the problem | Scores fall short on an evaluation | Diagnose one specific behaviour |
| Change the model | Adjust settings, fine-tune or retrain | Edit a small set of weights directly |
| Then | Evaluate again | Evaluate again |
The model that kept getting stuck
The model was Qwen3.5-0.8B, a small open-weight model, run with its step-by-step “thinking” mode switched on. Its model card warns that in this mode it is prone to loops. On the MATH-500 benchmark, 93% of its answers hit the 32,768-token limit.
The editing model, GPT-6 Astra, worked in a Python session over the weights. Training was not allowed, but it had a budget of forward passes: runs of the model on up to 512 practice problems, none from the test sets. With 50 passes, it traced the looping to one direction written by two internal layers and subtracted it from 131,072 weights.
Under the same sampling settings, MATH-500 rose from 7.9% to 48.1%, and answers hitting the limit fell from 93% to 33%. GSM8K, school maths problems, rose from 13.8% to 67.1%; IFEval, which tests instruction following, from 21.0% to 62.3%. None of the benchmarks the authors report fell. A re-run of the published model by Hugging Face scored 51.0% on MATH-500.
Removing a fault is not adding ability
On the authors’ own reading, the edit did not teach the model maths. It stopped the model getting stuck, so it finished answers it could already work out. Astra’s own check on practice problems it had held back points the same way: 576 of the 579 answers it rescued were ones the original model had cut off, and accuracy on answers the original finished was unchanged.
A second experiment tests that reading. The researchers tried the same approach on Gemma 3 270M, a smaller model that rarely loops: 8.5% of its answers hit its limit. Astra found ordinary mistakes, wrong arithmetic and invalid algebra, with no single component behind them. No edit moved its MATH-500 score by more than about a point.
Correcting a behaviour that hides an ability is different from creating an ability that was never there. This work shows the first. It does not show the second.
Where this could matter
More organisations now run open-weight models: specialist, compact, often private or air-gapped. When one underperforms, the usual answer is to change its settings or prompts, fine-tune it or replace it. Direct edits could one day join that list for some kinds of failure, and save some retraining. That has not been shown yet.
Two conditions apply. Editing weights needs the weights, so it is open to an organisation that owns its model, not to one calling a closed model through an API. And the cheapest fix should be ruled out first. Qwen’s model card says this model runs without thinking mode by default, and suggests tuning its sampling settings to curb repetition. The research does not compare its edit with those simpler changes, so we do not know how much of the gain a setting would recover.
How would you know the repair worked?
A weight edit is a new version of the model, and it needs the same testing as any other. Changing 0.016% of the weights is not inherently safer than changing more; a small edit can still alter behaviour nobody measured.
Here, every benchmark was run with the same thinking-mode settings, and the model’s default mode was not reported. The process was not yet consistent either: runs given 10 or 25 forward passes made different edits and gained far less. In its favour, the edit is published as an exact difference from the original weights, so it can be inspected, rebuilt and undone.
For a model behind consequential decisions, any change has to answer the same questions. Did it fix the intended problem, did it damage anything else, and does it hold on unseen cases and under other operating conditions? That is what our evaluations are built to answer.
Understand the failure first
The interesting part is not that one AI model edited another. It is that a model’s weights may become something you can investigate and correct, rather than the sealed output of training. For organisations that own and run specialist models, the first question when one underperforms may be why it fails, before deciding whether to retrain it or replace it.