'What I cannot create, I do not understand'
–Richard Feynman

'What I can create, I do not understand'
–Yoshua Bengio, Geoffrey Hinton, and Yann Lecun

ok so i'm at ICLR; this is my first machine learning conference. as you might imagine, it's all very fun and exciting.

but these poster sessions are absolutely INSANE

this is an airplane hanger crammed with hundreds of posters, each with dozens of people talking over each other

i'm in vienna 🇦🇹 for #ICLR2024 ! dm me if you want to chat or grab a beer!

also come hang out at the poster session for Language Model Inversion on Friday, i'll tweet about it once i know when and where. excited to DELVE in!

wrote a short blog post on the latent variable model behind STaR: justintchiu.com/blog/star/

i can feel the friday energy but this is where i'm at debugging-wise

🙄: posting about conference acceptances, reviewers, how many papers you published

😀: posting about PROJECTS and RESEARCH and the IDEAS that make you EXCITED

okay so what's up with this

- jan: no big open LLM releases
- feb: no big open LLM releases
- mar/apr: cohere Command-R, databricks/mosaic DBRX, Reka Core, meta LLAMA-3...
- may: no big open LLM releases

where can I go to read the most cutting-edge research in computers and AI from 50 or 75 years ago?

are there academic journals/periodicals/conference proceedings I can access from the 1950s? (would consider paying for a really good resource)

well that didn’t take long at all

i have a dataset of vector pairs. millions of them. i want to use a diffusion model to learn to map vectors from one column to the other. i do not want to deeply understand diffusion. i do not want to get into the details. is there a way to do this? like sklearn for diffusion?

large language model buzzwords

• it's not a language model, it's an intelligent assistant
• that's not a line that goes up in to the right, that's an emergent capability
• you're not running a hyperparameter sweep and making a graph, you're proposing a new scaling law

one of the most important things I know about deep learning I learned from this paper: 'Pretraining Without Attention'

this what I found so surprising:
these people developed an architecture very different from Transformers called BiGS, spent months and months optimizing it and

right now people think of data and code as very separate things, but I imagine in the future the line will really start to blur

datasets will become algorithms, syntax will soften. eventually we won’t know where a model’s source code ends and its parameters begin

i wonder if the AI-is-going-to-kill-all-the-humans hype may have crested a wave, at least for now?

i remember when a few accounts that would tweet 'quit your job, AGI is coming' and 'scaling is all it takes' type stuff nonstop. don't see nearly as much compared to a year ago