MIT researchers use GPT-6 Astra to edit another model's weights directly

Overview

Updated 1 hour ago

On November 2, researchers at MIT's Computer Science and Artificial Intelligence Laboratory reported that OpenAI's GPT-6 Astra edited the internal weights of Qwen3.5-0.8B without training or gradient descent. The edit changed 131,072 of the model's roughly 800 million parameters, or 0.016%, and raised its score on the MATH-500 benchmark from 7.9% to 48.1%.

The gain came from stopping repetitive reasoning loops, not from teaching new mathematics. On Gemma 3 270M, a model without the loop problem, the same approach produced no meaningful improvement. The result points to a narrow but real capability: reading and repairing existing weights, not creating new skills.

Why it matters

If a model can read and rewrite another model's weights from a text prompt, safety auditing and cheap fixes could bypass training entirely.

Questions about this story

Free account needed to ask — your question is kept and asked for you right after sign-up. Answers are public.

No questions yet — be the first to ask.

Key Indicators

48.1%

MATH-500 after edit

Qwen3.5-0.8B's score after GPT-6 Astra edited 0.016% of its weights, with 50 forward passes.

7.9%

MATH-500 before edit

The base Qwen3.5-0.8B score under the same sampling settings.

67.1%

GSM8K after edit

Grade-school math score rose from 13.8% after the same weight correction.

62.3%

IFEval after edit

Instruction-following score rose from 21.0%, showing the fix generalized beyond math.

131,072

Weights changed

The number of individual parameters Astra edited in Qwen3.5-0.8B.

0.016%

Share of model weights edited

The edit touched a tiny fraction of the model's roughly 800 million weights.

33.1%

Answers hitting token limit after edit

Down from 93.0% before the edit, showing the loop behavior was reduced.

Voices

Curated perspectives — historical figures and your fellow readers.

People Involved

Rikiya Takehi

MIT CSAIL researcher and lead co-author

Researcher at MIT CSAIL

Jayden Teoh

MIT CSAIL researcher and co-author

Researcher at MIT CSAIL

Alex Zhang

MIT CSAIL researcher and co-author

Researcher at MIT CSAIL

Omar Khattab

MIT CSAIL faculty and co-author

Faculty at MIT CSAIL

Lucas Beyer

AI researcher who reviewed the team's prompt

AI researcher

Organizations Involved

MIT Computer Science and Artificial Intelligence Laboratory

Research laboratory

Published the Weight-Space Programming results

MIT's main computer science and artificial intelligence research lab, where the weight-editing experiments were run.

OpenAI

AI company

Maker of GPT-6 Astra, the model that performed the edits

The AI company whose GPT-6 Astra model read and rewrote another model's weights in the MIT experiments.

Alibaba (Qwen / Tongyi Lab)

AI research team

Maker of Qwen3.5-0.8B, the model that was edited

The team behind Qwen3.5-0.8B, the small language model whose weights GPT-6 Astra edited.

Google DeepMind

AI company

Maker of Gemma 3 270M, the control model that did not improve

The AI company whose Gemma 3 270M model served as a control in the weight-editing experiments.

Timeline

3 events Latest: Yesterday

November 2026
MIT CSAIL publishes Weight-Space Programming results
Latest Research

Researchers report that GPT-6 Astra can read and write neural network weights directly, without gradient descent.
Yesterday
Astra's 0.016% edit lifts Qwen MATH-500 from 7.9% to 48.1%
Result

The edit targets a repetition direction in layers 15 and 19, cutting answers that hit the token limit from 93% to 33.1%.
Yesterday
Prompt hint draws scrutiny
Discussion

Lucas Beyer notes the prompt told Astra that fixing Qwen's repetition counted as success; Jayden Teoh says Claude Code added the hint.
November 2026

Scenarios

Independent labs replicate weight-space edits on other models

Possible Resolves by Jun 1, 2027

Discussed by: MIT CSAIL researchers and AI research commentators

Other groups try the same approach on different open-weight models. If at least two independent teams report non-gradient weight edits that improve a benchmark by more than 10 points, the capability looks general rather than tied to one model's loop bug.

Weight editing proves limited to fixing loop-style failures

Likely Resolves by Jun 1, 2027

Discussed by: MIT CSAIL researchers and technology press

Follow-up work finds that weight editing only helps models with a specific, localized failure like Qwen's repetition loop. On models without such a bug, edits produce small or inconsistent gains, as the Gemma 3 270M result suggested.

AI labs ship weight-editing tools for safety auditing

Unlikely Resolves by End of 2027

Discussed by: MIT CSAIL researchers and AI safety commentators

A major AI lab releases a product or API that lets users inspect or edit model weights in natural language. The MIT blog lists auditing without evaluation and zero-data capability steering as possible uses.

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

March 2019

The Lottery Ticket Hypothesis (2019)

Jonathan Frankle and Michael Carbin at MIT showed that dense neural networks contain sparse subnetworks that can be trained from the same starting weights to match the full network's accuracy. The finding suggested that much of a network's capability sits in a small fraction of its parameters.

Then

The paper sparked a wave of work on pruning and sparse training.

Now

It established that weight structure matters more than raw parameter count, a premise behind targeted weight editing.

Why this matters now

The MIT weight edit changed 0.016% of Qwen's parameters and shifted behavior sharply, echoing the idea that small subsets of weights carry outsized function.

June 2021

Low-rank adaptation, or LoRA (2021)

Edward Hu and colleagues at Microsoft showed that large language models could be adapted to new tasks by training small low-rank matrices rather than updating all weights. The method cut the number of trainable parameters by orders of magnitude while preserving performance.

Then

LoRA became a standard technique for cheap model fine-tuning.

Now

It normalized the idea that small, targeted parameter changes can produce large behavioral shifts.

Why this matters now

LoRA still uses gradient descent. The MIT result goes further by changing weights through a model's reasoning, with no training loop.

March 2017

Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017)

OpenAI researchers showed that evolution strategies, a gradient-free optimization method, could train neural networks on reinforcement learning tasks by perturbing weights and selecting better performers. The approach matched some gradient-based results but needed far more compute.

Then

It revived interest in gradient-free optimization for neural networks.

Now

Gradient-free methods remained a niche because they scale poorly and lack the efficiency of backpropagation.

Why this matters now

Earlier gradient-free methods found weights by trial and error. The MIT claim is different: a language model reasons about what each weight computes and edits it directly.