Been wild to hear from Yi how things in the wild are, definitely worth a read!

Gemini 1.5 has arrived. Pro 1.5 with 1M tokens available as an experimental feature via AI Studio and Vertex AI in private preview.

Then there’s this: In our research, we tested Gemini 1.5 on up to 2M tokens for audio, 2.8M tokens for video, and 🤯10M 🤯 tokens for text. From…

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.

And there are many more results in the paper so don’t forget to check it out. This is joint work with my awesome colleagues: Vinh Q. Tran & Mostafa Dehghani. We believe we are scratching the surface and we’ll see more applications of fractals in LLMs in the future
arxiv.org/abs/2402.01825

fun fact: I was working on DSI right after being released from the hospital for losing all remaining kidney function -- semantic ids were thought of and implemented from a dialysis chair!

google

at emnlp if anyone wants to hang!

Congratulations Yi and Reka on the launch!! Going from zero to the most multimodal model to date in 6 months is insane 😮

Our work 'Recommender Systems with Generative Retrieval' got accepted to NeurIPS 😊🎉

Congrats again to my co-authors Shashank Rajput, Nikhil Mehta, Vinh Q. Tran, Yi Tay, jonah, Maciej Kula, Ed H. Chi

Latest version at arxiv.org/abs/2305.05065

Quentin Berthet Just be free and live your best age.

Excited to announce my newest breakthrough project!!

🔥🔥 State-of-the-art results (100%!!) on widely used academic benchmarks (MMLU, GSM8K, HumanEval, OpenbookQA, ARC Challenge, etc.) 🔥🔥

1M param LLM trained on 100k tokens 🤯

How??

Introducing **phi-CTNL**

🧵👇

1/6

Thanks TensorFlow for covering our work.

This is based on our TIGER paper preprint on Generative Retrieval (arxiv.org/abs/2305.05065).

cc co-authors: Shashank Rajput, Nikhil Mehta, Yi Tay, jonah, Ed H. Chi, Vinh Q. Tran, Maciej Kula

to live, eat, and breathe the new york summer with friends and loved ones, what more could you ask for really

'I would never wish to incorporate this technology into my work at all. I strongly feel that this is an insult to life itself.' -miyazaki