Jeremy Howard featured in Yejin Choi’s talk today at iclr

.Yejin Choi makes a prediction that I can get behind: “30% chance that within 3 years, we will have a language-only Al that is perceived as AGl-enough by ~30% of people”. This seems right. People—including scientists—easily (over-)attribute intelligence to machines.

What explains the effectiveness and staying power of truncation based inference algorithms (top-p, top-k) for LLMs? Matthew Finlayson @ ICLR is presenting our poster right now at #iclr24 , go find out!

🎉We're celebrating our OLMo team's big win last night for Innovation of the Year at the GeekWire Awards! Thank you to the panel for recognizing our efforts towards open science, and to the other nominees for pushing us all towards better AI technology! geekwire.com/2024/geekwire-…

Talk: 'OLMo: Findings of Training an Open LM' from Hanna Hajirshizi at AI2 from OSGAI.

Extremely interesting overview of the 4 parts (Data, Training, Adaptation, Eval) of the OLMo open LLM project. Rare insight into how these processes work at scale.

youtube.com/watch?v=qFZbu2…

Our new naacl paper showcased most of LLMs achieve less than 40% chance of success in creative problem solving and out of the box thinking. Though under certain domain specific settings, llms capabilities found to be complementary to human capabilities:
arxiv.org/abs/2311.09682

Adversarial Triggers For LLMs Are 𝗡𝗢𝗧 𝗨𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹!😲

It is believed that adversarial triggers that jailbreak a model transfer universally to other models. But we show triggers don't reliably transfer, especially to RLHF/DPO models.

Paper: arxiv.org/abs/2404.16020

time for a paper post: A Roadmap to Pluralistic Alignment

what happens when you take alignment down to its most granular form? you arrive at individual alignment... aka personalization. but to date, most of the utility found in LLMs has come from a global alignment approach

Published in Nature Machine Intelligence today, our new article explores the trade-offs of personalised alignment in large language models ⚖️ Personalisation has potential to democratise decisions over how LLMs behave, but brings its own set of risks...
nature.com/articles/s4225…

We created reviewing guidelines for Conference on Language Modeling. Not intended to automate the committee work, or dictate constraints. But, to inspire a thoughtful reviewing process, for an exciting and impactful program of the highest possible quality. We have a wonderful program committee ❤️

Yes, both the 8B and 70B are trained way more than is Chinchilla optimal - but we can eat the training cost to save you inference cost! One of the most interesting things to me was how quickly the 8B was improving even at 15T tokens.

We took this on Day2 of #TED2024 .
Some #AI ROCK⭐️'s...Fei-Fei Li Daniela Rus MIT CSAIL Helen Toner Yejin Choi ruchowdh.bsky.social Niceaunties. And speaking today Dr. Catie Cuan + Prof. Anima Anandkumar

...oh...and then there's me🤣

Just tried the new GPT4+v on our New Yorker caption contest task (arxiv.org/abs/2209.06293).

It does OK! (70%, good for second on leaderboard). But, w/ performance ~25% below human, it still doesn't quite 'get the joke'. Maybe your model does? :-) capcon.dev

Introducing our best OLMo yet. OLMo 1.7-7B outperforms LLaMa2-7B, approaching LLaMa2-13B at MMLU and GSM8k. High-quality data and staged training are key.

I am so proud of our team making such significant improvement in a short period after our first release.

We released this new agenda on LLM-safety yesterday. This is VERY comprehensive covering 18 different challenges.

My co-authors have posted tweets for each of these challenges. I am going to collect them all here!

P.S. this is also now on arxiv: arxiv.org/abs/2404.09932

I’m super excited to release our 100+ page collaborative agenda - led by Usman Anwar - on “Foundational Challenges In Assuring Alignment and Safety of LLMs” alongside 35+ co-authors from NLP, ML, and AI Safety communities!

Some highlights below...

I will be talking about what differential privacy is, what it is not and what some common misconceptions are in privacy for generative AI in a couple hours The GenLaw Center in DC!

Join us on the live stream: tinyurl.com/genlaw-stream
Slides: tinyurl.com/genlaw-dp-2024

🥰Excited to share that I will be joining AI2 Allen Institute for AI MOSAIC this September as a predoctoral young investigator!! So excited to continue working with amazing Yejin Choi Nouha Dziri Liwei Jiang Kavel Rao and can't wait to collaborate with others!

Can we uncover memorization of pre-training data in LLMs, using other LLMs?

Our iterative prompt optimization method finds prompts that propel an LM to output training data using other LMs. We show higher avg. data reconstruction & extract 1.4X more PII!
arxiv.org/abs/2403.04801

The infini-gram paper is updated with the incredible feedback from the online community 🧡 We added references to papers of Jeff Dean (@🏡) Yee Whye Teh Ehsan Shareghi Edward Raff et al.

arxiv.org/abs/2401.17377

Also happy to share that the infini-gram API has served 30 million queries!