Zephyr 7b beta: paper deep dive, code, & RAG

Is Zephyr 7b beta the best fine-tuned smaller open-source LLM? It is fine-tuned on the best small open-source foundation model Mistral 7B, outperforms all small models, and even outperforms LLama2-Chat-70B and ChatGPT on multiple benchmarks.
What are the difference between Zephyr 7b alpha and Zephyr 7b beta?
From the author: "The main differences are (a) better dataset filtering of UltraChat, namely fixing grammatical errors and some unwanted responses and (b) training for longer, ie 3 DPO epochs vs 1"
Model: huggingface.co/HuggingFaceH4/...
Technical report: arxiv.org/abs/2310.16944
Colab: colab.research.google.com/git...
LlamaIndex Tweet (colab link in the tweet): / 1718054631413363196
🔔 SUBSCRIBE to my channel: kzbin.info...
⭐ Stay in touch ⭐
📚 DS/ML Book Club: dsbookclub.github.io/
▶ KZbin: / sophiayangds
✍️ Medium: / sophiamyang
🐦 Twitter: / sophiamyang
🤝 Linkedin: / sophiamyang
💚 #ai

Thanks a lot. Using Zephyr with ollama and it's pretty cool and fast.

[@SophiaYangDS]

Awesome! Ollama is really nice.

[@jstevh]

Thanks for that analysis. Like a good overview of a solid paper.

[@SophiaYangDS]

Thank you!

[@taeyangoh7305]

great review, Sophia! I want to give it a try for care robot use case!!

[@SophiaYangDS]

Thanks! Sounds like a really cool use case. Good luck!

[@Hash_Boy]

many many thanks

[@prestonmccauley43]

Thanks for the information I look forward to giving this model a try though it does seem like we keep producing models every week. They really need to be run for many months to determine how well they work.

[@SophiaYangDS]

Thanks! Yeah too many models coming out. One of the takeaways is Mistral 7B is a great foundation model to fine-tune on.

[@talhayousuf4599]

Your channel is really informative, good thing is that you remain updated with new things coming. Are you a researcher?, do you have any blog? / related to llm research?

[@SophiaYangDS]

Thanks 🙏 I have a casual blog with book reading notes and other random things: sophiamyang.medium.com/

[@AkarshanBiswas]

I am using this model as my default model currently. It is not perfect but I prompted to according to my liking. I wish the prompt template was a bit simpler. 😅

[@SophiaYangDS]

Nice!

[@DistortedV12]

It's okay..I tried it with different prompts. Can't wait till they scale it.

[@SophiaYangDS]

It's a good 7B model.

[@jsalsman]

Do they mention the cost of the GPT-4 calls during the training steps?

[@SophiaYangDS]

They did not mention cost in the paper.

[@jsalsman]

@@SophiaYangDS Any idea roughly how many API calls they must have used?

[@SophiaYangDS]

Must be a lot. Just to process and create this UltraFeedback dataset (huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized?row=0), it's around 74k x 5 API calls (4 models responses + 1 GPT4 ratings). @@jsalsman

[@TheReferrer72]

Is this the Model that does not reason or code well? How can this model be useful?

[@SophiaYangDS]

Still better than llama2 70B on coding and math

[@mshonle]

Does distilling help to remove overfitting?

[@SophiaYangDS]

I'm not sure. You can overfit in distillation.

[@mshonle]

I wonder how much this cost them to make, in terms of compute? (Or, maybe more relevant: how much would it cost someone else to do the same?)

[@SophiaYangDS]

​@@mshonle I think they might be spending most of the compute/API calls in generating the datasets. The actual SFT and DPO might not be that crazy.

[@mshonle]

Hmm, and those datasets are available?

[@SophiaYangDS]

@@mshonle Yes they are public on Hugging Face

[@serkhetreo2489]

Sometimes, u discover new channels

[@SophiaYangDS]

Thanks for discovering my channel :)