Gopher Explained: 280 BILLION Parameter Model Beats GPT-3

[@Astyer18]

You appear to be misreading the paper. The 300 tons of carbon is for the entire jet trip, not for each passenger.

[@EdanMeyer]

sure did, oops

[@Eternal36595]

@@EdanMeyer how do I use gopher? Can I use it now?

[@Eternal36595]

@@EdanMeyer and what u think ever happen to Google Meena?

[@simonmasters3295]

Yeah. The passengers aren't headed for Mars!

[@YouKnowJohnny]

I's like to see video with detailed explanation of the new type of transformer they introduced. Great job for this video!

[@MasterOfMoep]

Bump

[@BoNaha]

Thank you for your work and explanation, looking forward to more information about Gopher. I'm so glad that we have now a competitor for GPT-3 and we are talking about Google, who has a dedicated processor for that and definitely knows something about data :0 Great time to be alive, for real.

[@Xaelum]

Awesome video. Very much worth the subscribe I would love to watch a video about the new type of transformer btw!

[@turner-tune]

Absolutely loved this video, this is the type of content that really needs more coverage. Only seen this one of your videos but I'm certainly subscribing! Great work man!

[@oflameo8927]

Gopher is already an application protocol. Port 70. We need to get better at naming things.

[@McMurchie]

Wow, what an amazing vid - subbed!

[@KastanDay]

Nice work! Would love to see these videos and more, will be great for your resume haha. KZbin knows me because they put this front and center on my homepage.

[@MrSchweppes]

A video explaining the RETRO model/system would be fascinating!

[@Nico79489]

Great video, just subscribed. You saved me a lot of time. Another video about improving language models by retrieving millions of tokens would be excellent.

[@anjaylah]

Does anyone know how I get to the website where I can use this? When I use chat GPT it was boom right there. I can’t find anything else.

[@SandroAerogen]

But how can we use it? Is it open for the public yet?

[@EdanMeyer]

Unfortunately it is not, and I doubt it ever will be. Not that its a good thing, but its unlikely that most people would even be able to run the model haha

[@melkenhoning158]

4:38 A video on that sounds super interesting

[@countofst.germain6417]

Do anyone know if I can use any of the alternatives?

[@lcb931023]

Very interested in the ethics paper on large language models.

[@Graverman]

quick question here, you have have said it somewhere but I can’t find it, you are ex google and amazon employee, but where do you work now?

[@EdanMeyer]

I actually quit to go to grad school and focus on research, so that’s what I’m doing now

[@i4ko95]

Hey, nice video! I have a quick question. What tool (extension) do you use to annotate in the browser? Thanks!

[@EdanMeyer]

I recently started using this: chrome.google.com/webstore/detail/paint-on-web-online-draw/ejllkedmklophclpgonojjkaliafeilj Not the best but it works okay

[@i4ko95]

@@EdanMeyer Thanks!

[@IceMetalPunk]

This video is 4 months old, and already Gopher has been made obsolete by PaLM. The rate of AI development, especially in the VLLM field, is astounding.

[@EdanMeyer]

While it may not be state or the art, that definitely doesn’t mean the research is obsolete. Although everyone has their own criterion, that’s one of the things that I think is important to make a good paper, that it goes beyond just getting SOTA on a benchmark and provides additional value. Gopher did this by providing an analysis on how models scale, which is something that should be applicable for the currently foreseeable future!

[@IceMetalPunk]

@@EdanMeyer I thought the original GPT-3 paper already did a scale-to-performance comparison, too?

[@ianmathwiz7]

I would love to see a video on the ethics paper.

[@andersonsystem2]

Awesome mate just subscribed

[@JazevoAudiosurf]

that batch size.. imagine they used a smaller size for even better accuracy. guess it would take too long to train

[@EdanMeyer]

I've seen several people mention this idea of using a smaller batch size yielding better accuracy, but I don't know where it comes from. My impression from several papers I've seen is that larger batch sizes can actually lead to better asymptotic performance (though I'm not sure how broadly this holds if at all). What makes you think that a lower batch size would be better?

[@JazevoAudiosurf]

@@EdanMeyer I don't know about transformers, but standard neural nets should in theory tend to overgeneralize with large batch sizes, while I haven't seen the disadvantage of smaller batch sizes. From my coding experience for smaller models I don't see much difference tho

[@anonymous1943]

@@EdanMeyer For the batch size, the larger the better. In the past 6 years, I've never seen any opposite example.

[@JerryThings]

So that's why deepmind has stopped giving public updates on their chatbot Meena

[@EdanMeyer]

Haha that might be why

[@cedricvillani8502]

Do Googles 540Billion model

[@cedricvillani8502]

Gopher was a information fetch

[@markadyash]

thanks

[@flugschulerfluglehrer]

Gopher used to be a internet protocol for text.

[@-rate6326]

It still is.

[@andrewwatts1997]

Interesting Gopher saw Atheism as a religion. Because it's the complete lack of any god like belief. Very nice to see something outperform GPT3 tho.

[@EdanMeyer]

Technically Gopher did not necessarily classify Atheism as a religion, but rather the experiment designers compared the sentiment of several words of different religion and atheism.

[@-rate6326]

It's might weird but atheism is considered part of some religions. Specifically from eastern philosophy. Like Samkhya. Some religions are not religion. But philosophies of multiple regions lumped together.

[@andrewwatts1997]

@@-rate6326 I want you to deconstruct that sentence in your mind a bit... ( lack of a religion) is considered part of some religions. Also, philosophy's are not religions. Religions entail worship of an entity just like cults have the worship of a cult leader. There is dogma you need to follow, places of worship you need to visit etc etc. Atheismt has zero percent of that.

[@-rate6326]

@@andrewwatts1997 i don't think you properly understand what atheism is. It doesn't matter what you think atheism is. Atheism has a proper definition. "absence of belief in the existence of deities". Atheism in noway means "no religion". Worshiping supernatural human or deities is just part of religion. maybe your understanding of religion is trap under Abrahamic understanding of religion. There are Dharmic Religions too. They have multiple philosophies. Buddhism was nontheistic belief when it started. In Sanskrit this philosophy is called "Nastik" meaning "no believe in god". Yoga's Philosophy is one of those too. But later Buddha became godly figure. The study of India and its cultures and religions, and the definition of "Hinduism", has been shaped by the interests of colonialism and by Western notions of religion. It's obvious Hinduism is not religion. But multiple philosophies that were created in India and it's culture. Religious beliefs are different in India as travel different regions. There are monotheists, polytheists, atheist but they are considered follower of Hinduism. Basically it's lumped together word for country's diverse beliefs. I am not promoting religion. I was trying to explain that there are things other than just book followers or God believers.

[@harshraj22_]

🔥🔥😲😲