Earlier this month, the largest-ever public Generative Red Team Challenge drew 2,244 people to test artificial intelligence (AI) generative large language models (LLM) built by Anthropic, Cohere, Google, Hugging Face, Meta, NVIDIA, OpenAI, and Stability AI, with participation from Microsoft, on a testing and evaluation platform built by Scale AI.
Organized by SeedAI, AI Village, and Humane Intelligence in the AI Village at DEF CON 31, the GRT Challenge was designed to advance AI by gaining a better understanding of the risks and limitations of this technology at scale, because open and extensive evaluation and testing bring us closer to inclusive development.
Over two and a half days in Las Vegas, thousands of participants – including 220 community college students and others from organizations traditionally left out of the early stages of technological change from 18 states – engaged with leading AI models. Participants exchanged 164,208 messages in 17,469 conversations while probing for bias, potential harms, and security vulnerabilities in 21 challenges designed to uncover the AI models’ possible gaps in trust and safety.
As organizers planned the event, they prioritized and engaged with diverse and traditionally underserved community partners.
Also Read: Phishing, Remote Work Raise Asian Cybersecurity Concerns
“Black Tech Street led more than 60 Black and Brown residents of historic Greenwood to DEF CON as a first step in establishing the blueprint for equitable, responsible, and accessible AI for all humans,” said Tyrance Billingsley II, founder and executive director of Black Tech Street. “AI will be the most impactful technology that humans have ever created and Black Tech Street is focused on ensuring that this technology is a tool for remedying systemic social, political and economic inequities rather than exacerbating them.”
In addition to the delegation from Black Tech Street, SeedAI worked with a network of community colleges, with leadership by Houston Community College, and other organizations including the Knight Foundation to identify underserved students to sponsor to participate in the GRT Challenge.
“AI holds incredible promise, but all Americans — across ages and backgrounds — need a say on what it means for their communities’ rights, success, and safety,” said Austin Carson, founder of SeedAI and co-organizer of the GRT Challenge. “Generative AI is a representation of human language, and as such it is entirely about context: how you speak with it, use it, and interact with it. We can’t rely on a small group that lacks the context of our lives to make sure it is useful rather than harmful.”
Organizers chose DEF CON 31 as the venue for the GRT Challenge not only due to its built-in access to the hacker community who could help address risks and opportunities, but also so that participants could engage with the hacker community and exchange ideas and information.
SOURCE: Businesswire
