AI Research3 mins read

GPT-6 Astra Can Rewrite Other AI Models Through Their Weights, MIT Researchers Report

MIT researchers show that OpenAI’s GPT-6 Astra can analyze and modify other AI models directly through their weights, a method called Weight-Space Programming. The approach improved a Qwen model’s math accuracy in one test, but the results also show clear limits.

What MIT Researchers Say Astra Can Do

MIT researchers report that OpenAI’s GPT-6 Astra can analyze and deliberately modify the internal numerical settings of other AI models, known as weights. Instead of relying on the usual step-by-step optimization process, Astra can work directly with raw numbers or use programming tools and feedback from test runs.

The researchers call the approach Weight-Space Programming: a language model acts less like a trainer and more like a mechanic adjusting individual knobs inside another model. The core takeaway is that model optimization may not always require full retraining, though the reported capabilities remain limited.

Astra Reads Weights to Infer Model Behavior

In several experiments, researchers fed Astra the raw weights of trained networks and asked what those networks had learned. Astra correctly identified which of four computer games a network was designed to play, recognized basic shapes from an image generator with some confusion between filled shapes and outlines, and matched the training language of small language models without training data or execution access.

A more subtle test involved identifying which of two fine-tuned language models was trained to always agree with users. Astra could not determine that from weights alone, but when allowed to observe what the models attended to during text processing, it correctly identified the sycophantic model across all three model pairs.

Generated Weights Beat Some Trained Baselines

Astra also generated weights for models intended to play three simple computer games, using only game rules and source code rather than conventional training. According to the researchers, these models outperformed ones trained with a standard method over 100 million rounds; in Breakout, Astra’s model averaged 517 points compared with 25 for the trained baseline.

The results were not uniformly dominant. In two other games, Astra fell behind the longest-trained versions, showing that direct weight generation can be powerful in some cases but is not a blanket replacement for training.

The Practical Test: Fixing Qwen’s Math Performance

The most practical experiment involved Qwen3.5-0.8B, where Astra used a coding environment and 50 test iterations with practice problems to improve math skills. Evaluation was done on separate problems Astra had not seen.

Astra identified a thinking-loop problem already noted in Qwen’s own documentation, then changed just 0.016 percent of all weights. The result was a jump in MATH-500 accuracy from 7.9 to 48.1 percent, while instruction-following improved from 21 to 62 percent.

The fix did not fully eliminate the loop issue: a third of responses still hit the maximum text length, down from 93 percent before. Astra also produced no measurable improvement on Gemma 3 270M, suggesting the method may remove bottlenecks in existing capabilities rather than add skills a model lacks.

Why It Matters—and Where It Stops

The researchers see possible uses in security audits directly on model weights and targeted behavior changes without expensive retraining. A benchmark for systematic measurement is next on their list, and the experiment logs and edited Qwen model are available through the channels cited in the original report.

The work also drew debate: AI pioneer Jürgen Schmidhuber called the approach “old hat,” while Khattab responded that earlier papers were linked and “far ahead of its time.” The claimed new element is how well today’s language models can translate between human instructions and a neural network’s numerical values.

Discover More