Overall Common Crawl is a snapshot of some part of what was published, primarily in English and other European languages from 2010-2024 (and even then, stuff prior to 2020 is often disregarded). More data has been dreamt before or aside that than you can ever conceive.

Yes but no. If we completely disregard copyright (I won’t) LLM labs are only running of data if they refuse to become what they should be: digitization and data orgs. Books alone are 150-200m and this is still dwarfed by the massive amount of magazines and newspapers content.

Dream job alert! 👇

Great to see the openTSNE paper finally published! Pavlin's openTSNE is by far the best t-SNE implementation out there. Very fast, very easy to use, but also very flexible and easy to extend and build upon. We use it in all our t-SNE-related projects. Great job Pavlin Poličar!

Job alert Nomic AI ! I'm hiring a front end/Web engineer to build the next generation of data interfaces for curating, exploring, and model-building from text and image data. Apply here or pass it on: jobs.ashbyhq.com/nomic.ai/42e7a…

OH: “classical NLP approaches like BERT”

In last decade’s renovation of the Library Mall space at UW-Madison, the open speaker’s podium that was built into campus post-Vietnam specifically to facilitate protest & assembly was torn down & replaced with a non-functional sculpture called “Both/And – Tolerance/Innovation.”

Very rarely do I get to pull a “I wrote a book on this subject and you are very mistaken” card, but the moment has arrived.

is phi-3 better than Meta's llama3? try both in #gpt4all and decide for yourself! download now at gpt4all.io

As someone who played absolutely no part in this, let me say—this got incorporated into GPT4all *fast*.

As a constant cynic about Harvard, my model for humanities PhDs at hyper-wealthy schools is not cheap labor, but a perk that admins give to faculty. Great researchers love teaching grad seminars, advising disses, etc. more than they want, say, a foosball table in the lounge.

'In 2020, the number of humanities bachelor’s degrees awarded (for the entire range of disciplines) fell below 200,000 for the first time since 2002—and then fell again in both 2021 and 2022' bit.ly/3xENm32

Just a reminder you can literally train nomic embed from scratch, the training data is public.

The only above 62% on mteb below 500M params to do so 😀

Our paper 'The landscape of biomedical research' is out in Patterns journal! Great job by Rita González Márquez.

cell.com/patterns/fullt…

Amazing interactive explorer by Ben Schmidt / @[email protected] from Nomic AI: static.nomic.ai/pubmed.html

For details see my original Twitter thread: x.com/hippopedoid/st….

The fake accounts here seem to be getting a lot worse, no? I've now got multiple bots with *the same profile picture* liking old reposts. Seems hard for me to believe that this is either useful spamming, *or* something that would be hard for functioning auto-moderation to find.