Inside RRSI: Google’s Framework for Regularized Self-Improving AI Agents

Google Cloud AI Research, in collaboration with UNC-Chapel Hill, Stanford and Washington University in St. Louis, has open-sourced RRSI (Regularized Recursive Self-Improvement). This framework enables a language model agent to iteratively refine its own harness—prompts, tools, memory, control flow and sub-agents—while keeping model weights unchanged and preventing overfitting to the evolve set.

Availability: Released under the Apache 2.0 license, RRSI requires Python 3.10 or higher and accepts any LiteLLM model string. By default, it runs on Claude Opus 4.8 hosted on Vertex AI.

Overfitting in Self-Improving Harnesses

Traditional harness evolution loops generate candidate edits, evaluate them on a fixed set of tasks and retain the highest-scoring change. Reusing the same evolve set exposes three key failure modes (RRSI paper):

Benchmark-specific fitting: repetitive tasks lead to memorization rather than generalization.
Noise chasing: small score fluctuations are mistaken for real improvements.
Complexity accumulation: unchecked edits increase cost without genuine gains.

RRSI Methodology

Instead of editing the model weights, RRSI opens every harness component to modification but regularizes the search process itself.

Proposal Regularization

Annealed edit budget: a cosine schedule transitions from multi-edit proposals early to single, attributable edits in later rounds.
Evidence-aware credit: each candidate is logged with its component, hypothesis, diff, score change and cost change, preventing redundant retries.
Structured exploration: when improvements stall within the noise band, the budget shifts to untouched components.

Selection Regularization

Leakage critic: filters out edits exploiting task names, entities or benchmark-specific logic before scoring.
Noise-adjusted floor: only accepts edits that exceed baseline variance on the frozen harness.
Cost rule: extra inference tokens must be offset by commensurate score gains.
Pruning: removes components that fail to produce gains over a pruning window.

These regularizers mirror classic norms: the annealed budget approximates L₀, pruning acts as L₁ (Lasso) and the cost rule as L₂ (Ridge).

Performance on Eight Benchmarks

Terminal-Bench 2.1 (evolve split): 74.2% → 80.2%.
SWE-bench Verified (never used in selection): 82.0% → 83.8%.
Out-of-distribution: JobBench +4.7, GDPval +3.5, APEX-Agents +3.7 points.
EngDesign (evolve) +4.9 medal points; Frontier-Eng +4.3.
Harvey LAB: +1.1 on evolve split, +2.3 on held-out split.

All six held-out splits improved. With Gemini 3.5 Flash as policy, Terminal-Bench 2.1 rose from 64.6% to 78.7% and SWE-bench Verified from 76.8% to 79.0%. RRSI also cuts inference cost: 2.42 M policy tokens per trial versus 3.80 M for unregularized evolution (30–36% fewer).

Comparison with Other Frameworks

The table below, adapted from Table 1 of the RRSI paper, compares RRSI against competing self-improvement approaches using the same baseline harness, policy, evolve split and candidate budget:

Feature	RRSI	Meta-Harness	AHE	TTHE	HarnessX
Core idea	Regularized proposal & selection	Agentic proposer over code, scores & traces	Observability-driven loop	Evolves harness at inference	Modular, trace-driven adaptation
Model weights	Frozen	Frozen	Frozen	Frozen	Frozen
Cost rule & pruning	Yes	No	No	No	No
Harvey LAB evolve score	90.5	93.0	90.7	91.1	91.8
OOD avg (H0 = 39.7)	43.6	40.6	39.2	38.0	39.7

*Per RRSI authors; OOD average is mean of JobBench, GDPval and APEX-Agents.

Interactive Demonstration

How RRSI Regularizes Agent Self-Improvement

The model stays frozen. The harness (prompts, tools, memory, control flow) evolves, but every edit must pass the regularizers.

Getting Started

git clone https://github.com/google-research/rrsi.git && cd rrsi
pip install -e " .[dev]"
python3 rrsi.py --domain coding baseline
python3 rrsi.py --domain coding run

Each round creates two git worktrees to draft candidate harnesses, filters and evaluates them, then fast-forwards to the superior harness. The coding domain also requires Docker and harbor. New domains can be integrated via a single adapter module.

Key Takeaways

RRSI evolves prompts, tools, memory and workflows while keeping model weights fixed.
Leakage critic, noise-adjusted floor, cost rule and pruning ensure authentic, generalized improvements.
Terminal-Bench 2.1 improves from 74.2% to 80.2% with Claude Opus 4.8.
SWE-bench Verified, unseen during selection, rises from 82.0% to 83.8%.
Apache 2.0 code on GitHub; research framework for exploring self-improving agents.

Explore the paper, GitHub code and project page for technical details.