Resources

AI agents are increasingly granted access to sensitive information and external tools to perform tasks on behalf of users, increasing data leakage risks. This report presents a joint evaluation by the SG and KR AISI examining whether agents can complete routine workflows without leaking data. Across 12 realistic tasks spanning areas such as enterprise productivity and customer service, the evaluation finds that agents may successfully complete tasks while still exhibiting data-handling failures, suggesting that task correctness alone is insufficient to assess agent safety. This report is a more detailed follow-up to an earlier blog post on this topic.

Korea and Singapore AISIs jointly tested how AI agents behave in real-world tasks. The exercise examined whether agents can complete multi-step tasks in common settings such as customer service and enterprise productivity without leaking sensitive data. This blogpost shares the key findings, evaluation challenges, and methodological learnings.

This evaluation report details the various methodological components and findings behind this joint testing exercise, which will work towards advancing the science of AI agent evaluations and building common best practices for testing AI agents.

The 3rd Joint Testing Exercise builds on the insights from the previous two exercises, to work towards advancing the science of AI agent evaluations and building common best practices for testing AI agents. Bringing their collective technical and linguistic expertise, participating AISIs worked together to conduct testing for sensitive information leak, fraud, and cybersecurity threats.

Following the blogpost released earlier this year, this evaluation report takes a closer look at the various methodological components and findings behind this joint testing exercise, which aims to seed a common approach for multilingual safety testing of frontier models at scale.

The Singapore Conference on AI (SCAI): International Scientific Exchange (ISE) on AI Safety saw over 100 of the best global minds from academia, industry and government, collectively identify and demonstrate consensus around technical AI safety research priorities. In shaping reliable, secure and safe AI, the outcomes of the discussion at SCAI:ISE are synthesized into the Singapore Consensus on Global AI Safety Research Priorities, a living document that continues to welcome views from the global research community.