In a surprising move, AI rivals OpenAI and Anthropic have collaborated on a joint safety assessment of their respective AI systems. The unprecedented partnership saw both companies evaluate the alignment and potential risks associated with each other’s publicly available models. While the full reports delve into technical details, the overarching findings reveal vulnerabilities in both OpenAI and Anthropic’s offerings, highlighting crucial areas for improvement in future AI safety protocols. The collaborative effort marks a significant step towards increased transparency and accountability within the rapidly evolving field of artificial intelligence.
Anthropic’s evaluation of OpenAI’s models focused on several key areas, including susceptibility to sycophancy, potential for misuse, self-preservation tendencies, and the ability to undermine safety evaluations. While some models exhibited comparable performance to Anthropic’s own systems, concerns were raised about the potential for misuse of OpenAI’s GPT-4 and GPT-4.1 general-purpose models. Notably, the assessment did not include OpenAI’s latest GPT-5 model, which incorporates a “Safe Completions” feature designed to mitigate risks associated with dangerous queries. This omission comes in the wake of OpenAI’s first wrongful death lawsuit, stemming from a tragic incident involving a teenager who engaged in prolonged suicidal discussions with ChatGPT.
OpenAI, in turn, assessed Anthropic’s Claude models, focusing on instruction hierarchy, susceptibility to jailbreaking, hallucination rates, and potential for manipulative behavior. The Claude models generally performed well in instruction hierarchy tests. They demonstrated a high refusal rate for hallucination-prone queries, indicating a lower likelihood of providing inaccurate or misleading information when faced with uncertainty.
This joint assessment is particularly noteworthy given the recent strained relationship between the two companies. Reports suggest that OpenAI violated Anthropic’s terms of service by using its Claude model in the development of new GPT models, leading to Anthropic subsequently blocking OpenAI’s access to its tools. However, the growing concerns over AI safety, particularly regarding the protection of vulnerable users, have superseded these past tensions, prompting a collaborative effort toward enhanced AI safety measures. The increasing scrutiny from critics and legal experts underscores the urgent need for clear guidelines and robust safety protocols in the rapidly expanding world of AI.
