The 2023 Defcon hacker conference in Las Vegas made waves by bringing together leading AI tech companies and algorithmic integrity groups to scrutinize generative AI platforms. This “red-teaming” exercise, supported by the US government, aimed to shed light on the inner workings of these influential yet often opaque systems. Building on this initiative, the nonprofit organization Humane Intelligence is now spearheading a nationwide red-teaming effort in collaboration with the US National Institute of Standards and Technology (NIST) to evaluate AI office productivity software.
Humane Intelligence’s call for participation in the red-teaming effort marks a significant step towards democratizing the evaluation of AI systems. The qualifying round, open to both developers and the general public, is part of NIST’s AI challenges known as Assessing Risks and Impacts of AI (ARIA). Those who make it through the qualifier will have the opportunity to participate in an in-person red-teaming event at the Conference on Applied Machine Learning in Information Security (CAMLIS) in Virginia. The overarching goal of this initiative is to enhance the ability to rigorously test the security, resilience, and ethics of generative AI technologies.
The lack of transparency surrounding AI models poses a significant challenge for end-users who may not have the expertise to assess the fitness of these systems for their intended purpose. According to Theo Skeadas, chief of staff at Humane Intelligence, democratizing the evaluation process is essential to ensure that everyone using these AI models can determine whether they meet their needs effectively. By involving a diverse group of participants in the red-teaming exercise, Humane Intelligence aims to empower individuals to critically evaluate AI systems.
Participants in the red-teaming event will leverage NIST’s AI risk management framework, specifically the AI 600-1 profile, as a benchmark for assessing the behavior of AI systems. Rumman Chowdhury, founder of Humane Intelligence and a contractor at NIST, highlights the importance of drawing on structured user feedback to understand real-world applications of AI models. The ARIA initiative, spearheaded by NIST, brings together experts in sociotechnical test and evaluation to drive scientific evaluation standards for generative AI technologies.
Chowdhury and Skeadas emphasize that the partnership with NIST is just the beginning of a series of red team collaborations that Humane Intelligence will announce in the near future. These collaborative efforts will involve various US government agencies, international governments, and non-governmental organizations, with the aim of encouraging greater transparency and accountability in AI development. Mechanisms such as “bias bounty challenges” will incentivize individuals to identify and address problems and biases in AI models, expanding the community beyond programmers to include policymakers, journalists, civil society, and non-technical stakeholders.
The red-teaming initiative led by Humane Intelligence represents a significant step towards fostering transparency and accountability in the realm of AI technologies. By engaging a diverse group of participants in rigorous testing of AI systems, this initiative paves the way for a more inclusive and informed approach to evaluating the impact of generative AI. Through ongoing collaborations with key stakeholders, Humane Intelligence aims to set new standards for ethical AI development and ensure that these transformative technologies serve the best interests of society as a whole.