Google Nested Learning Solves Catastrophic Forgetting in AI Models

Machine learning has made extraordinary strides over the past decade, yet one stubborn problem has persisted beneath the surface of every breakthrough: catastrophic forgetting. When a neural network learns new tasks, it often overwrites the knowledge it acquired from previous ones, sacrificing old competencies for new ones. In November 2025, Google Research introduced a paradigm it calls Nested Learning, published at NeurIPS 2025, that proposes a fundamentally different way to think about how models learn — and may bring AI one step closer to the continual learning capabilities of the human brain.

The Problem of Catastrophic Forgetting

Large language models and other deep learning systems are impressive, but their knowledge is frozen at the moment their training concludes. Once deployed, they cannot organically absorb new information without being retrained from scratch or fine-tuned in ways that risk erasing prior learning. This is the essence of catastrophic forgetting (CF): the more a model adapts to new data, the more it forgets what it already knew.

Researchers have traditionally attacked this problem from two angles — tweaking the model’s architecture or improving its optimization rules. Google Research’s Ali Behrouz and Vahab Mirrokni argue that this separation is itself the problem. For years, the machine learning community has treated a model’s network structure and its training algorithm as two distinct concerns. Nested Learning dissolves that boundary.

What Is Nested Learning?

Nested Learning reframes a machine learning model not as a single optimization process but as a system of interconnected, multi-level optimization problems that run simultaneously. Each level has its own internal flow of information — what the researchers call “context flow” — and its own update frequency, meaning different parts of the model adjust their weights at different rates.

The key insight is that a model’s architecture and its optimization algorithm are fundamentally the same concept viewed at different levels of abstraction. A transformer’s attention mechanism, for instance, can be formalized as an associative memory module that maps one token to another. Similarly, the backpropagation process itself can be modeled as associative memory, where the network learns to associate a data point with how surprising or unexpected it was.

By assigning each component an update frequency — how often its weights change — researchers can organize these nested problems into ordered levels. This hierarchy forms the core of the Nested Learning paradigm and opens what Google describes as a “previously invisible dimension” for designing more capable AI.

Lessons From the Human Brain

The biological inspiration for Nested Learning is neuroplasticity — the brain’s ability to restructure itself in response to new experiences, memories, and learning. Without neuroplasticity, a person is limited to immediate context, much like a patient with anterograde amnesia who cannot form new long-term memories. Current LLMs suffer from a strikingly similar limitation: their knowledge is confined to either their input context window or the static information absorbed during pre-training.

The human brain achieves continual learning through uniform, reusable structures and multi-time-scale updates. Different brain regions and circuits update at different frequencies, allowing new knowledge to integrate without overwriting existing memories. Nested Learning mirrors this principle by allowing each model component to update at its own rate, preserving older knowledge while accommodating new information.

Deep Optimizers and Continuum Memory Systems

The Nested Learning perspective yields two immediate practical advances:

• Deep optimizers: By treating standard optimizers (such as momentum-based methods) as associative memory modules, researchers identified that many rely on simple dot-product similarity that fails to account for how different data samples relate to each other. By reformulating the optimizer’s objective using more standard loss metrics like L2 regression, they derived new formulations for core concepts like momentum, making optimizers more resilient to imperfect data.
• Continuum memory systems (CMS): In a standard transformer, the sequence model serves as short-term memory while feedforward networks store long-term knowledge from pre-training. Nested Learning extends this binary into a spectrum — a continuum of memory modules, each updating at a distinct frequency. This creates a richer, more effective memory architecture purpose-built for continual learning.

Hope: A Self-Modifying Architecture

To validate the theory, Google built a proof-of-concept model called Hope, a variant of the Titans architecture. Titans are long-term memory modules that prioritize memories based on how surprising they are. While powerful, standard Titans only support two levels of parameter updates, limiting them to first-order in-context learning.

Hope goes further. It is a self-modifying recurrent architecture capable of leveraging unbounded levels of in-context learning, augmented with continuum memory system blocks to scale to larger context windows. The model can essentially optimize its own memory through a self-referential process, creating an architecture with what the researchers describe as “infinite, looped learning levels.”

Experimental Results

Google evaluated Hope across language modeling, long-context reasoning, continual learning, and knowledge incorporation tasks. The results were compelling:

• Language modeling: Hope demonstrated lower perplexity and higher accuracy compared to modern recurrent models and standard transformers across diverse public benchmarks.
• Long-context tasks: On Needle-In-A-Haystack (NIAH) downstream tasks — tests that require finding specific information buried in long documents — Hope showcased superior memory management, proving that continuum memory systems offer a more efficient way to handle extended sequences.
• Common-sense reasoning: Hope outperformed architectures including Titans, Samba, TTT, and Mamba2 across multiple difficulty levels of reasoning benchmarks.

Why This Matters for the Future of AI

Catastrophic forgetting is not merely an academic curiosity. It is one of the most significant barriers to building AI systems that can genuinely learn and adapt over time. Today’s models are essentially snapshots — brilliant at what they know, but incapable of growing without expensive, full-scale retraining. This limitation has real-world consequences:

• Enterprise AI: Companies deploying models on proprietary data must retrain or fine-tune whenever new information arrives, incurring significant compute costs and risking performance regression on existing tasks.
• Personal AI assistants: A truly useful assistant would need to remember conversations, preferences, and context across sessions — something current architectures struggle with without workarounds like retrieval-augmented generation.
• Scientific discovery: Models that can continuously incorporate new findings without forgetting prior knowledge could accelerate research in fields from drug discovery to materials science.

Nested Learning suggests that the path forward may not require entirely new model families, but rather a deeper understanding of the relationships between components we already use. By recognizing that architecture and optimization are two sides of the same coin, researchers gain a new design dimension — one that could produce models with the depth and flexibility needed for genuine continual learning.

Looking Ahead

The Nested Learning paradigm is still in its early stages. Hope is a proof-of-concept, not a production system, and scaling self-modifying architectures to the parameter counts of today’s largest models remains an open challenge. But the theoretical framework is significant because it unifies two previously separate domains of machine learning research under a single lens.

Google’s researchers are optimistic that the community will adopt this perspective to explore what they call a “new dimension” of model design. If Nested Learning proves scalable, it could mark a turning point in the quest for AI that learns the way humans do — continuously, adaptively, and without forgetting what came before.

As the field moves beyond the era of static, frozen-knowledge models, paradigms like Nested Learning offer a compelling blueprint for the next generation of self-improving AI. The gap between artificial and biological learning remains wide, but it is narrowing — one nested optimization at a time.

---

Edited by Palawan @QUE.COM
Website: https://QUE.COM Intelligence
Sponsored by: https://MAJ.COM AI Autonomous