Google Proposes TUMIX: Multi-Agent Test-Time Scaling With Tool-Use Mixture

Understanding the Target Audience

The target audience for TUMIX includes AI researchers, business leaders in technology, and data scientists focused on improving the efficiency and effectiveness of AI systems. Their pain points typically involve:

• High inference costs associated with tool-augmented AI models.
• Need for improved accuracy in complex reasoning tasks.
• Desire for innovative approaches to enhance model performance without significant resource expenditure.

Goals include:

• Maximizing the accuracy of AI outputs while minimizing costs.
• Leveraging multi-agent systems to improve decision-making processes.
• Staying updated with the latest advancements in AI methodologies.

Interests often revolve around:

• New AI frameworks and methodologies.
• Collaboration between academia and industry in AI research.
• Practical applications of AI in business settings.

Communication preferences lean towards clear, data-driven insights with an emphasis on practical applications and technical specifications.

Overview of TUMIX

TUMIX (Tool-Use Mixture) introduces a test-time framework that ensembles heterogeneous agent styles—text-only, code, search, and guided variants. By allowing agents to share intermediate answers and stop early based on consensus, TUMIX achieves higher accuracy at lower costs on challenging reasoning benchmarks such as HLE, GPQA-Diamond, and AIME (2024/2025).

Key Innovations of TUMIX

The TUMIX framework distinguishes itself through:

• Mixture over Modality: TUMIX utilizes approximately 15 agent styles, including Chain-of-Thought (CoT), code execution, web search, and guided variants. Each agent refines its answer based on the original question and previous responses from other agents, enhancing early accuracy while managing diversity.
• Adaptive Early-Termination: An LLM-based judge determines when to halt the refinement process, ensuring that answers with strong consensus are finalized early. This approach maintains accuracy at about 49% of the inference cost compared to fixed-round refinement, reducing token costs to approximately 46%.
• Auto-Designed Agents: TUMIX enables the base LLM to generate new agent types, increasing average accuracy by approximately +1.2% without additional costs. The optimal number of agent styles for maximum effectiveness is around 12–15.

How TUMIX Works

TUMIX operates by running a diverse group of agents in parallel. Each agent undergoes a series of refinement rounds where it considers the original question and the previous answers from other agents. This structured note-sharing process allows for improved candidate coverage while managing token and tool budgets. The LLM-judge evaluates the consensus after each round, deciding whether to continue refining or to finalize the output through methods like majority vote.

Results and Performance

Compared to strong tool-augmented baselines, TUMIX exhibits superior average accuracy:

• HLE (Humanity’s Last Exam): Pro: 21.6% → 34.1% (TUMIX+); Flash: 9.7% → 23.1%.
• GPQA-Diamond: Pro: up to 88.3%; Flash: up to 82.1%.
• AIME 2024/25: Pro: 96.7%; Flash: 86.7% with TUMIX(+) at test time.

Overall, TUMIX achieves an average improvement of +3.55% over the best prior tool-augmented test-time scaling baseline at similar costs and +7.8% / +17.4% gains over no-scaling for Pro/Flash, respectively.

Conclusion

TUMIX presents a compelling approach to test-time scaling by treating it as a search problem across heterogeneous tool policies. The parallel committee structure enhances candidate coverage, while the LLM-judge allows for early stopping, preserving diversity and reducing costs—an essential consideration for applications with latency budgets.

Further Reading

For more information, check out the original paper. Explore our GitHub page for tutorials, codes, and notebooks. Join our community on Telegram and subscribe to our newsletter for the latest updates.