Tiny Recursive Model (TRM): A Tiny 7M Model that Surpass DeepSeek-R1, Gemini 2.5 pro, and o3-mini at Reasoning on both ARG-AGI 1 and ARC-AGI 2

«`html

Understanding the Tiny Recursive Model (TRM)

The Tiny Recursive Model (TRM) is a compact 7M-parameter model developed by Samsung SAIT (Montreal) that demonstrates superior reasoning capabilities compared to larger models such as DeepSeek-R1, Gemini 2.5 Pro, and o3-mini on both ARC-AGI 1 and ARC-AGI 2 benchmarks. TRM achieves a test accuracy of 44.6–45% on ARC-AGI-1 and 7.8–8% on ARC-AGI-2, outperforming larger models that have significantly more parameters.

Key Innovations of TRM

TRM introduces several architectural advancements:

Single tiny recurrent core: TRM replaces the two-module hierarchy of the Hierarchical Reasoning Model (HRM) with a single two-layer network that maintains a latent scratchpad z and a current solution embedding y.
Deeply supervised recursion: The model employs a think-act block that is unrolled up to 16 times with deep supervision, allowing for effective signal propagation across steps.
Full backpropagation through the loop: Unlike HRM, TRM backpropagates through all recursive steps, which is essential for generalization.

Performance Metrics

TRM’s performance on various benchmarks is noteworthy:

ARC-AGI-1: 44.6% accuracy
ARC-AGI-2: 7.8% accuracy
Sudoku-Extreme: 87.4% accuracy, surpassing HRM’s 55.0%
Maze-Hard: 85.3% accuracy, compared to HRM’s 74.5%

Why a 7M Model Can Outperform Larger LLMs

TRM’s architecture allows it to outperform larger models due to:

Decision-then-revision approach: TRM drafts a full candidate solution and then refines it through iterative consistency checks, reducing exposure bias.
Effective depth from recursion: The model’s depth is achieved through recursion rather than stacking layers, leading to better generalization at constant compute.
Tighter inductive bias for grid reasoning: For smaller fixed grids like Sudoku, TRM utilizes attention-free mixing to enhance performance.

Conclusion

The Tiny Recursive Model represents a significant step in architectural efficiency, demonstrating that a compact model can achieve competitive performance on complex reasoning tasks. The research team has made the code available on GitHub, contributing to the ongoing exploration of efficient AI models.