🔥 another win for open source AI

SNOWFLAKE ARCTIC:

1] Mixture of Experts model with 128 (!) experts

2] open weights, open code

3] Company is sharing 'cookbook' and data and teaching people how to build their own world class MoE models.

4] very cheap model to train (16x…

Interesting Wes Roth video on Snowflake's #SnowflakeArctic model exploring the implications of large, capable, and truly open models.

model = learn(data)

Synthetic data is great, but it’s not data. It’s an intermediate quantity created by learn(). Data is created by people and has privacy and copyright considerations. Synthetic “data” does not - it’s internal to learn().

Often hear companies that want alignment as a service.

In other words, configurable bar on what to refrain from answering.

For example, my chatbot should only answer questions relating to my domain and not general purpose questions like [who is the president of the United…

Amazon ad load is ridiculous.

Ad load generally correlates inversely with health of underlying core growth

Screenshots below passed on without comment (okay this is as bad as yahoo.com or ask.com search)

We’re excited to share insights and lessons learned collecting the data needed for Arctic as part of our #SnowflakeArctic Cookbook Series. 📖

Our third edition covers the filtering, processing, and composition techniques we used, including what worked and what didn't.

Huge shout out to Ben Firshman and the team at Replicate for being amazing launch partners for Snowflake. Using Replicate inference, the #SnowflakeArctic live demo has been running smoothly and crazy fast all day long! arctic.streamlit.app

❄️ We're excited to introduce Snowflake’s innovative large language model, Arctic, into our model catalog. Discover Arctic in Microsoft Azure AI: msft.it/6018YKJDn

<marketing_message>
Wonder how to filter web data for high quality?

Wonder how you compose the code part of your training data?

Read on for what Nathan Wiegand and friends did as part of building #SnowflakeArctic ...
</marketing_message>

Super excited to work w/ MS Azure ...

#SnowflakeArctic is now available in Microsoft Microsoft Azure's AI model catalog 🎊

This collaboration increases transparency in AI, and we're excited to bring our open source model to users everywhere.

Read on for an inside view into optimizing training systems