As the X-verse should already know, Lizzie Kumar is amazing and no you can't work with her because Sherri Rose has already stolen her for a postdoc :). Well of course you can, because Lizzie does amazing things in so many different areas.

We're back for day 2! In the first session focused on auditing, we start with Ryan Steed, presenting one of the two distinguished papers, focused on sorting through the vast literature on auditing.

lol Ryan Steed doing an incredible job presenting our paper in a literal castle 😄

I am particularly pleased that Ojewale Victor is now at Brown.

we are so thrilled and honoured to have received the SaTML distinguished paper award for our work 'AI auditing: The Broken Bus on the Road to AI Accountability' 🙌🏾😍🙏🏾

Such an incredible honor!

Congrats to the entire OAT team Abeba Birhane, Ryan Steed, Ojewale Victor, Briana Vecchione for this award- winning work 🎉🎉

zhifeng kong kamalikac The other #SaTML2024 Distinguished Paper Award goes to SoK: AI auditing: The Broken Bus on the Road to AI Accountability by Abeba Birhane Ryan Steed Ojewale Victor
Briana Vecchione Deb Raji arxiv.org/abs/2401.14462 2/4

Deep Learning Indaba, the biggest African AI conference is entirely run by volunteers. We have great entries this year for our Awards and are looking for experts to help review. If you have the expertise & capacity, please come review for us by entering your details in form 👇🏾

Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Authors : Ojewale Victor , Ryan Steed , Briana Vecchione, Abeba Birhane, Deb Raji

TLDR: Despite many tools designed for setting standards and evaluating AI systems, these often fall short in practice for…

🧵Delving into this substantial paper mapping AI audit tools ecosystem from Ojewale Victor Deb Raji Abeba Birhane Ryan Steed & Briana Vecchione.

I'm thrilled to see World Privacy Forum's complementary report on AI Governance Tools around the world & emerging problems is cited throughout.🙏 😊

Already so proud of this team and their policy impact!

Incredible to see our OAT comment cited multiple times in the NTIA 'Artificial Intelligence Accountability Policy Report': ntia.gov/sites/default/…

Fascinating thread about the risk that privacy research and tools merely facilitates even greater surveillance infrastructure. Also because my student Rui-Jie Yew and newly graduated lucy qin have just written a paper on this very topic: arxiv.org/abs/2402.06609

as part of the Mozilla OAT project, we examined and taxonomized current audit tools, and the paper is finally out

One of the biggest lessons learnt after our study on AI audit tooling (arxiv.org/abs/2402.17861) was how serious a pain point adequate model & data access continues to be.

I signed this letter bc building the tech infra is not enough - we'll need policy interventions as well!

We spent over a year scavenging for AI audit tools and interviewing audit practitioners about their process.

What we found: the audit process is more complicated than we think, and the tasks we need tooling for extends far beyond just evaluation.

See: tools.auditing-ai.com

We found lots of the tools I expected (fairness & explainability toolkits) but also some other kinds of tools I’d like to see more often (databases documenting AI harms & audit results, interview guides, organizing kits, & more).

Read or see tools.auditing-ai.com for more!