Scaling AI security research for a multi-agent world
Over the last decade, advancements in artificial intelligence have primarily focused on enhancing individual AI models to be more powerful, helpful, and secure. Today, Google DeepMind, alongside partners like Schmidt Sciences, the Cooperative AI Foundation, the Advanced Research and Invention Agency, and with support from Google.org, has announced a significant initiative: a call for technical research funding of up to $10 million for researchers worldwide.
Entering a New Era of AI Interactions
As AI technology scales, we are on the brink of a new era where millions of AI agents, developed by various organizations, will interact within digital environments. These agents will communicate, negotiate, and transact with one another, necessitating a safe and predictable framework for interactions. This shift presents a critical opportunity to enhance the security and stability of the AI ecosystem from its inception.
Understanding Multi-Agent System Risks
The funding call is aimed at studying the behaviors of large, multi-agent AI systems as a collective. The focus is on developing frameworks to understand and mitigate potential risks. By empowering researchers globally, the initiative seeks to address the “invisible” security risks that arise when independent systems interact across diverse networks.
Why the Agent Ecosystem Matters
The interaction of large groups of AI agents can lead to the emergence of new collective behaviors and capabilities. Currently, tools to predict, measure, and monitor these transitions are lacking. While most security assessments analyze models in isolation, interacting autonomous agents can produce complex, emergent behaviors that are difficult to predict.
Given the novelty of this research area, understanding these changes is crucial. Potential outcomes could include unpredictable economic shifts or new security challenges. The primary goal is to understand how to manage these system-wide behaviors effectively.
Pushing the Boundaries of Multi-Agent Security Research
Despite existing frameworks for multi-agent security, the rapid development of these systems necessitates an immediate and large-scale expansion of research.
Our 2025 research provided a framework for understanding these interactions, and our recent work on AI agent traps examines the vulnerabilities agents face in adversarial environments. The complexity of multi-agent interactions now exceeds existing security models, demanding accelerated research efforts.
The aim of this funding call is to expedite progress by supporting a global network of independent researchers. A diverse community is essential to ensure that safety standards are transparent and binding for everyone.
These efforts also support the mission of Schmidt Sciences’ Science of Trustworthy AI and AI Agents programs, which focus on understanding and mitigating risks of frontier AI systems. Additionally, ARIA’s Scaling Trust program aims to unlock new forms of cyber-physical multi-agent coordination.
A Common Call to Action
No single laboratory can solve the safety challenges of multiple agents alone. We invite academic and independent researchers to submit proposals in four focus areas:
- Sandboxes and testbeds: Develop realistic, reproducible environments to assess, compare, and accelerate progress across all areas of multi-agent security. This includes virtual marketplaces, simulated ecosystems, and cross-organizational workflows.
- The Science of Agent Networks: Study the safety-related properties of interacting agent populations, including how collective capabilities emerge and scale, network failures or volatility, and detection of dangerous, unexpected properties at the population level.
- Strengthening the agent infrastructure: Conduct stress testing protocols for identity, reputation, and engagement that ensure secure cross-platform agent interactions.
- Supervision and control: Develop methods to monitor deployed agent populations and mitigate collective harm on a large scale.
How to Take Part
We invite researchers to consider our call for proposals and join us in building a secure foundation for a multi-agent future.
The application deadline ends on August 8, 2026. The award winners are expected to be announced in autumn 2026.
You can find further information about the technical requirements and the application process in our application portal. Here
“`
