SEAL shows how AI can think and evolve continuously

Artificial intelligence is becoming increasingly versatile – it generates images, writes poetry and builds apps. Yet one key limitation remains: today’s systems struggle to truly evolve beyond their initial programming. That’s exactly where a new concept from the Massachusetts Institute of Technology (MIT) comes in. Called SEAL, or Self-Adapting Language Models, this framework enables large language models to behave more like learning beings. SEAL allows them to process new information, generate their own insights and update their knowledge in real time – without relying on external datasets or extensive developer intervention. The research paper was published on June 12 on arXiv.

Continuous learning without developer intervention

“Especially in companies, it is not enough to simply retrieve data – systems must be able to adapt continuously,” says MIT PhD student Jyothish Pari. SEAL is designed to do exactly that, using a continuous two-step process. First, the AI summarizes new information, generates relevant examples and adjusts its internal settings. These changes are referred to as “self-edits.”

The system then immediately puts its self-edits to the test: it undergoes brief retraining with the new adjustments and is evaluated to see whether its responses actually improve. SEAL only retains the changes if the results show a clear performance gain. Comparative tests confirm the effectiveness of this method: in a question-and-answer quiz without supporting text, the accuracy of the Qwen 2.5-7B model rises from 33.5% to 47%. In the more challenging ARC puzzles – logic-based tasks from the Abstraction & Reasoning Corpus – performance even climbs to 72.5%, more than triple the model’s original score.

Thanks to this cycle, SEAL behaves almost like a thinking entity: whenever new facts or questions arise, the model “reflects” on what matters, generates its own examples and adjusts its settings to better apply what it has learned. Since this process runs continuously, the AI is always learning. It no longer relies on separate developer fine-tuning but instead uses incoming texts as training material – generating its own data on the fly.

SEAL unlocks several possibilities at once. In the future, chatbots could naturally adapt to users' personal preferences without needing to send sensitive data to external servers. Development and research tools could also evolve more independently – adjusting to shifting project requirements without having to be retrained each time. And even if publicly available text data becomes scarce, SEAL can generate its own training material through self-created examples, offering a smart way to sidestep potential data shortages.

High potential, but not without hurdles

Although SEAL holds significant promise for advancing AI development, the researchers point to three main challenges:

• First, there's the issue of catastrophic forgetting: as the model continuously integrates new self-edits, its ability to perform earlier tasks gradually declines. The study already shows early signs of this effect.
• Second, the computational cost is substantial, as each self-edit requires a brief fine-tuning step. According to the study, a full cycle takes between 30 and 45 seconds, significantly increasing the operational cost of running large models.
• Third, verifying the accuracy of self-edits remains a challenge. The performance tests primarily assess how convincing an answer sounds, rather than whether it is actually correct. Users on Reddit have already raised concerns that the system might accept plausible-sounding but incorrect self-edits as improvements – and then internalize these errors permanently.