Llama 3 RAG Demo with DSPy Optimization, Ollama, and Weaviate!

Hey everyone! Thank you so much for watching this overview of Llama 3 looking at the release notes and seeing a demo of how to integrate it with DSPy through Ollama and how to use DSPy's MIPRO to find the optimal prompt when using this new large language model for RAG!
We are hosting an event in San Francisco on May 1st with Arize AI and Cohere, featuring a talk from Omar Khattab, the lead author of DSPy! Hope to see you there! lu.ma/dspy
Introducing Meta Llama 3: ai.meta.com/blog/meta-llama-3/
Ollama Llama 3: ollama.com/library/llama3
Weaviate Recipes: github.com/weaviate/recipes/b...
Chapters
0:00 Llama3!!
1:28 Release Notes
5:35 Performance Reporting
9:50 Training Details
17:32 DSPy Demo!

[@LaHoraMaker]

Dear Connor, that was the fastest release ever!

[@dianaliu7543]

This is so different from RAG using GPT. Lots to learn

[@cipritom]

Is it me, or at least the last part is a digital avatar?

[@charismaowojoameh7681]

I tried the implementation but i keep getting the error "model not found"

[@Tarun_Mamidi]

Is llama really OSS if we don’t know how or what it is trained on?

[@VincentFulco]

Would love to see an interface to groq please!

[@tobkin]

Two questions: - Why use gpt-4 instead of gpt-4-turbo for the teleprompter? - What are you using to make your pointer act like that?

[@catchychazz]

Great video, Connor. Have you tested out if SAMMO is better than DSPy for production?

[@larsbell1569]

I thought SAMMO was primarily prompt templating? Does it have some new features we are not aware of?

[@catchychazz]

@@larsbell1569 No, you're right. I was working on the assumption that in production, you'd be using the most capable language models. At the risk of running token-heavy pre-compiled DSPy prompts, having a simpler prompt that automatically augments few-shot prompting only on specific user inputs/triggered events might be a more cost-efficient solution.

[@MrjbushM]

Thank you.

[@420_gunna]

this mf cookin

[@koljanos]

Hey, what version of Weaviate-client you are using????

[@connorshorten6311]

Hey! I am using v4 and the latest version of dspy-ai, can you please share any error messages as an Issue on Weaviate recipes? It might be easier to help debug there instead of KZbin comments.

[@koljanos]

Allright, will check the issues!

[@human_agi]

how to install ollama

[@PeterWilliams97]

I ran your notebook and got the following error. print(RAG()("What is binary quantization?").answer) AttributeError Traceback (most recent call last) Cell In[7], line 1 ----> 1 print(RAG()("What is binary quantization?").answer) File ~/code/vector_search/weaviate/recipes/.wenv/lib/python3.11/site-packages/dspy/primitives/program.py:26, in Module.__call__(self, *args, **kwargs) 25 def __call__(self, *args, **kwargs): ---> 26 return self.forward(*args, **kwargs) Cell In[6], line 16 15 def forward(self, question): ---> 16 context = self.retrieve(question).passages 17 pred = self.generate_answer(context=context, question=question).answer 18 return dspy.Prediction(context=context, answer=pred, question=question) File ~/code/vector_search/weaviate/recipes/.wenv/lib/python3.11/site-packages/dspy/retrieve/retrieve.py:30, in Retrieve.__call__(self, *args, **kwargs) 29 def __call__(self, *args, **kwargs): ---> 30 return self.forward(*args, **kwargs) File ~/code/vector_search/weaviate/recipes/.wenv/lib/python3.11/site-packages/dspy/retrieve/retrieve.py:39, in Retrieve.forward(self, query_or_queries, k) 36 # print(queries) 37 # TODO: Consider removing any quote-like markers that surround the query too. 38 k = k if k is not None else self.k ---> 39 passages = dsp.retrieveEnsemble(queries, k=k) 40 return Prediction(passages=passages) ... 79 .do() 81 results = results["data"]["Get"][self._weaviate_collection_name] 82 parsed_results = [result[self._weaviate_collection_text_key] for result in results] AttributeError: 'WeaviateClient' object has no attribute 'query'

[@connorshorten6311]

Hey Peter! Apologies we have upgraded the WeaviateRM to use the Weaviate v4 client, can you please try upgrading dspy with `!pip install dspy-ai --upgrade` ?

[@connorshorten6311]

Can you please share any error messages as an Issue on Weaviate recipes? It might be easier to help debug there instead of KZbin comments.

[@LyuboslavPetrov]

@connorshorten6311 Please do update the video with accurate setup instructions. I have been fighting to get this running (DSPY + Weaviate + OLLAMA) for the past 2-3 hours to no avail. Tried multiple weaviate-client/server combinations, ran trough docker and standalone, configured, updated/downgraded dspy-ai. Went through so many help pages, cannot count now. I am tired, but still would like to play with this set of technologies. Thanks

[@leeme179]

maybe just me but the blur/smooth filter to the face cam makes me suspect that face cam is AI generated 🤣🤣🤣🤣🤣🤣

[@dixon1e]

Uncanny valley feel.