Are Claude 3.5 Sonnet, Llama-3 and Gemini choosing speed over quality?

in this video chris looks at how model providers are trending towards using grouped query attention vs traditional multi-headed attention in transformer models and how this is impacting output in areas such as summarization. in this video chris shows that you get better coherent output from models such as llama-2 or claude 3-opus over new models such as llama-3 or gemini or gemma. in the end, in certain scenarios such as summarization or generative content, gpt-4o still beats sonnet.
repo
github.com/chrishayuk/mha_gqa...

[@makepeace88]

I just attended detailed anatomy of LLM session.. and it’s just wow! Nobody’s telling these details. Thanks very much Chris ❤

[@chrishayuk]

Glad it was useful, I skipped a lot of details, as I wanted to keep the focus on MHA vs GQA. I will probs do some other videos on some of the other details

[@trsd8640]

Great video! I don’t understand it fully, had to watch it again, but I‘m getting a idea of what is happening! Thank you!

[@chrishayuk]

it was quite a tough one to record, as i'm trying to avoid explaining the entire transformers architecture and attention fully (i'll do that in another video), but do enough to just show how this architectural change is affecting models output. it was a weird balance and apologies that i never explained it enough

[@danielhenderson7050]

This was very interesting

[@chrishayuk]

Glad you enjoyed, definitely a fun rabbit hole

[@everyhandletaken]

Interesting! Claude 3.5 Sonnet is definitely great for code, much better than cgpt 4-o & has really helped me solve things that are well beyond my brain capacity in the last few days.

[@chrishayuk]

totally agree, much better for code than gpt-4o

[@Leo-ph7ow]

Excelent content! Thanks!

[@chrishayuk]

Glad you liked it!

[@seanknowles9985]

Intel agencies are having their fill first. Its obviously being slowed down so three letter agencies can get ahead of this.

[@chrishayuk]

lol, i'm sure 3 letter agencies are having their say but i suspect it's not on MHA vs GQA but would love to hear that conversation if they were