Evaluating LLM-based Applications

Evaluating LLM-based applications can feel like more of an art than a science. In this workshop, we'll give a hands-on introduction to evaluating language models. You'll come away with knowledge and tools you can use to evaluate your own applications, and answers to questions like:
- Where do I get evaluation data from, anyway?
- Is it possible to evaluate generative models in an automated way?
- What metrics can I use?
- What's the role of human evaluation?
Talk by: Josh Tobin
Here’s more to explore:
LLM Compact Guide: dbricks.co/43WuQyb Big Book of MLOps: dbricks.co/3r0Pqiz
Connect with us: Website: databricks.com
Twitter: / databricks
LinkedIn: / databricks
Instagram: / databricksinc
Facebook: / databricksinc

[@AnandShah-ds]

Evaluations aside, I really enjoyed the presentation. I was hooked. Great story-telling skills Josh. Thanks for sharing your experience. We count on volunteers like you to spread knowledge.

[@ndamulelosbg8887]

This is an exellent coverage of the challenging task of llm evaluatuon

[@ndamulelosbg8887]

"Your opininon on LLMs does not matter" - I found this to be a great quote

[@vaishnavipatil3319]

Thank you for clearing this concepts. Would like to see more videos from you on evaluation frameworks, methods.

[@asfandiyar5829]

Just what I was after. Thanks

[@manishsharma2211]

Good work

[@SpartanPanda]

Great storyline

[@bharath_v]

Good One!

[@threevia.travel]

Very generic, expected something more tangible! Sounds common sense which might work or might not work