Someone please make an auto optimizing pipeline that can reduce SWE-agent iterations for a specific problem domain (ie. Unit test generation) using Dspy. Pretty please

I think the most exciting announcement from Google I/O is Gemini Pro 1.5's 2 million token input window! 📚

Amongst many things, long input LLMs are going to be a game changer for search!

Search is typically a two stage process between retrieval and ranking. Beginning with

This is why DSPy, from the Stanford NLP team, is becoming a strategic component of any LLM workflow (here is an overview for SEOs that I wrote a few weeks ago wordlift.io/blog/en/dspy-s…) cc Omar Khattab

Julian Harris Matt Boyle Thanks Julian. You have shown a whole new world to me: github.com/stanfordnlp/ds…

API achieved internally

Andrew's list for working with large context models:
(1) Write quick, simple prompts
(2) Iteratively, flesh out a mega-prompt
(3) Few-shot or many-shot examples
(4) Break into subtasks / agentic workflow

I want to suggest an alternative 'Eval Driven' workflow:
(1) Write quick,

Which of the following resources would you rather share with four other grad students today?

Although your data is distributed throughout numerous databases, you can still use ALL of it for your RAG application 🍱

In this notebook, we build an end-to-end RAG pipeline that uses #Google 's Big Query and Weaviate • vector database, using DSPy!

Context Fusion with Agents will first

Lately we've been exploring several open-source frameworks, in addition to building our own in-house abstractions to create more robust AI systems. Take a look at our learnings and deep dive into DSPy.

🔗 gradient.ai/blog/achieving…
✅ GPT-4 Performance at 10x Lower Cost
✅

The future of working with LLMs is not about writing gigantic prompts!

Get to know DSPy: Programming—not prompting—LMs.

As an analogy, DSPy is to RAG as PyTorch is to DNNs.

Here's a studio on getting started with DSPy: lightning.ai/lightning-ai/s…

Quick-n-dirty 🧵on understanding DSPy as 4 related but different things.

1. A new category of ML models, Language Programs.
2. A programming model (abstractions) for expressing & optimizing LPs
3. New optimizers: ML algorithms to tune the parameters of LPs
4. A library for 1-3⤵️

👍 DSPy is good at dynamically adapting prompts and fine-tuning language models.
💪 Milvus is good at high-performance semantic similarity search.
👉 DSPy and Milvus together make highly efficient RAG pipelines.

Read more: bit.ly/3wopS1V