Stanford CS25: V3 I Retrieval Augmented Language Models

  Рет қаралды 161,519

Stanford Online

Stanford Online

Күн бұрын

December 5, 2023
Douwe Kiela, Contextual AI
Language models have led to amazing progress, but they also have important shortcomings. One solution for many of these shortcomings is retrieval augmentation. I will introduce the topic, survey recent literature on retrieval augmented language models and finish with some of the main open questions.
More about the course can be found here: web.stanford.e...
View the entire CS25 Transformers United playlist: • Stanford CS25 - Transf...

Пікірлер: 65
@erniea5843
@erniea5843 7 ай бұрын
I love that this content is freely accessible to everyone. Lots of helpful information being shared here
@nintishia
@nintishia 7 ай бұрын
The amount of research work on retrieval augmented generation for large language models has exploded in recent times. Thanks to the speaker for directing attention to the most significant bits.
@dongxu9013
@dongxu9013 7 ай бұрын
The best talk about RAG so far
@morespinach9832
@morespinach9832 7 ай бұрын
How many have you heard, right here on KZbin? Lots of actual hands on info out there and in 1/3rd the time. This one had loads of intro forever before getting into specifics.
@edwardmitchell6842
@edwardmitchell6842 6 ай бұрын
@@morespinach9832 Can you recommend some sources? I'm compiling a list.
@yangzju
@yangzju 5 ай бұрын
@@morespinach9832 can you recommend some videos?
@comunedipadova1790
@comunedipadova1790 4 ай бұрын
@@morespinach9832 What lecture do you suggest for a more practical (code) view?
@morespinach9832
@morespinach9832 4 ай бұрын
@@comunedipadova1790 plenty of them - search for these keywords.
@capucinnolover
@capucinnolover 6 ай бұрын
Nicely explained in just right technical details. Thank you!
@heisenberggiao9303
@heisenberggiao9303 2 ай бұрын
Thank you for the great contents!
@Arvolve
@Arvolve 7 ай бұрын
So many great ideas here! Fantastic resource, thank you.
@velociraptor75013
@velociraptor75013 6 ай бұрын
Excellent content, thanks for the references!
@loopaal
@loopaal 7 ай бұрын
this is what I needed, thank you sooooo much!!!!
4 ай бұрын
Good lecture, offers a good summary of the literature on RAG.
@ronitakhariya4094
@ronitakhariya4094 8 ай бұрын
just the thing i wanted thank you so much.
@kingki1953
@kingki1953 4 ай бұрын
My lecturers in my university never explain those things, thanks for this free lecture
@99BLACKLP
@99BLACKLP 7 ай бұрын
Great insights and video!
@NerdyX90
@NerdyX90 8 ай бұрын
Thank you.
@johntanchongmin
@johntanchongmin 7 ай бұрын
Pros: Gives a brief overview of many RAG methods Cons: No intuition given which is the key reason for why it works for the different methods Would have preferred more insights rather than just describing the papers, but overall thanks for the video!
@reslleygabriel
@reslleygabriel 7 ай бұрын
Awesome content, thanks for sharing!
@deeplearningpartnership
@deeplearningpartnership 7 ай бұрын
Really cool.
@duongkstn
@duongkstn 7 ай бұрын
Hi, about Atlas. You said that we can update the Retriever. At 42:00 is some retriever loss, but what about pair label (question, positive paragraph, negative paragraphs) like normal retrieval model - do they contribute to the retriever loss ? I read codebase of Atlas, and do not find that kind of loss
@echofloripa
@echofloripa 8 күн бұрын
Does these embeddings tools such as FAISS work with other languages other than English?
@muhannadobeidat
@muhannadobeidat 6 ай бұрын
It is good to see the whole spectrum of options, but what would be a practical way to get started on this? His old colleagues in HF actually have an excellent book that goes through many different ideas including RAG in chapter 7 where they do an excellent job explaining context and giving you options for implementation.
@蔡鉎驊
@蔡鉎驊 6 ай бұрын
3:34 What chatgpt was really about - fix the user interface to LM 12:00 Frozen RAG
@samferrer
@samferrer 7 ай бұрын
Actually we can see a first sign of language models in Shannon's 1948 paper, A Mathematical Theory of Communication.
@ssssssstssssssss
@ssssssstssssssss 7 ай бұрын
Language models started to appear in the 1800s.
@therainman7777
@therainman7777 7 ай бұрын
Language models actually appeared sometime during the Cretaceous period, though scientists aren’t quite sure of the exact year. They think the stegasaurus might have had something to do with it; all these people who think OpenAI invented them are so wrong.
@zhuolin730
@zhuolin730 Ай бұрын
where can I find the slides?
@trinityblood5622
@trinityblood5622 7 ай бұрын
Could you please share the slides?
@lambertch
@lambertch 6 ай бұрын
Shouldn’t it be Shannon, 1949?
@ginogarcia8730
@ginogarcia8730 8 ай бұрын
WE ARE IN THE FUTUREEEEEE
@ritikdua7225
@ritikdua7225 6 ай бұрын
does anyone have list of research papers metioned in this video?
@AIPoker-tj6lr
@AIPoker-tj6lr 7 ай бұрын
What about scann by google ??
@deconcoder
@deconcoder 7 ай бұрын
THe bit about where this came from was funny, but your search just gave a less silly answer. Go reread Shannon's 1948 paper that invented Information Theory. Yes he did not talk about using neural nets (which did not exist) but he did talk about probability.
@amansinghal5908
@amansinghal5908 7 ай бұрын
Are there assignments for this?
@vaibhavjoshi7961
@vaibhavjoshi7961 Ай бұрын
I did like the joke you made.
@kingki1953
@kingki1953 4 ай бұрын
why there's no framework to make the process was easy
@Arthurlvaz
@Arthurlvaz 6 ай бұрын
It's not entirely clear what "frozen rag" and "retrieve rag" refer to.
@user-wr4yl7tx3w
@user-wr4yl7tx3w 3 ай бұрын
there is too much review information
@MedTn-i3h
@MedTn-i3h 2 ай бұрын
worth dozens of "shiny" "trendy" "tik-toky" videos over there, without asking you to subscribe and leave comments..
@jstello
@jstello 7 ай бұрын
I love how everyone tries to hide the fact that OpenAI is 100% the reason everyone is watching this video
@pw7225
@pw7225 7 ай бұрын
No one is hiding anything
@a3mia3mi82
@a3mia3mi82 7 ай бұрын
“Ignorance” is not equal to “facts”
@therainman7777
@therainman7777 7 ай бұрын
I know, isn’t it hilarious? The bitterness and jealousy is so transparent. It reminds me of the scene from The Social Network where they’re trying to claim credit for his creation and he says “You know it’s really not that complicated. If you guys were the inventors of Facebook, you’d have invented Facebook.” Yann LeCun likes to drone on on Twitter about how everything OpenAI is doing is “old” technology. Well if it’s so old, how come hundreds of different companies filled with smart people were trying for years to make a chatbot that was worth using but until OpenAI no one seemed to manage it? Yes, OpenAI didn’t invent the Transformer. We know. Who cares? They clearly solved dozens of incredibly difficult engineering problems that no one else had been able to solve, and gave the world a language-based AI that was actually _useful._ As evidenced by the fact that it was the fastest growing app in human history, by an extremely wide margin. And as soon as they do it all these idiots turn up basically saying “I could have done that 10 years ago, I just chose not to.” Yeah, ok. It’s so pathetic.
@Gingnose
@Gingnose 7 ай бұрын
Maybe because only you are the one that is obsessed by the idea of starting learning something new but merely due to its "novelty and coolness" is something to be ashamed of and you project that idea on others which make you think others are watching this video with the same reason but hinding that information (and trying to look smart) because it is emabrassing to declare and you think that way too. However I will remind you that trying to learn new things whether it is because it is fashion or cool is nothing to be ashamed of and a great excuse to start new things.
@srh80
@srh80 7 ай бұрын
Whats an Open Al?
@starmountpictures
@starmountpictures 7 ай бұрын
meh
@mahkhi7154
@mahkhi7154 7 ай бұрын
Us Mandalorians CYBERMEN: Understand The Material - Given To You By Minions, better Than The Minions. Your Gods Gods. Ask Them.
@therainman7777
@therainman7777 7 ай бұрын
Could you try writing that again but having it not be nonsense this time?
@mahkhi7154
@mahkhi7154 7 ай бұрын
You're a P1G. Memorising and Repeating without Understanding. What is a Sentence? A Collection of Concepts. The Brain can Learn 10's of Thousands of concepts. It Has Hardware To Help it. Concept: "Doing" brings up pictures of Making a Cake or Assembling a Chair. Concept: "What" brings up Pictures of "Asking someone a Question" or Deciding Colour Show to Buy. A Computer Doesn't Work Like That. All it Understands is Numbers and Maths. All These "Concepts" have To Be Encoded in Numbers and Maths and Rules Applied To Them. The Computer Doesn't Know What The Character "B" is. All it Know That if it Finds The Number "66" in Memory, it Draws Something That Looks like "B". Spelling. It Doesn't Know What Spelling is. All it Knows is "66,65,84" Valid for word BAT. BAB 66,65,66 is an Invalid String of Numbers and Not in the Dictionary. 10,000 Concepts Need - Modelling and Rules Creation (100's of Thousands) - For the Computer To Understand. That isn't an Easy Thing To Do: Trillions of Lines of Java Code.
@therainman7777
@therainman7777 7 ай бұрын
Maybe if you were a computer maybe you would understand that you Don’t Need To Capitalize Every Word when writing English. And trillions of lines of code? You know nothing about coding; in all of human history everyone out together has not written a trillion lines of code, let alone in a single computer or a single program. P.S. Your comment was utter nonsense, it contributed nothing, and it meant nothing. Have a nice day.
@dongmo6546
@dongmo6546 7 ай бұрын
Thank you
A Survey of Techniques for Maximizing LLM Performance
45:32
Stanford CS25: V4 I Overview of Transformers
1:17:29
Stanford Online
Рет қаралды 55 М.
Touching Act of Kindness Brings Hope to the Homeless #shorts
00:18
Fabiosa Best Lifehacks
Рет қаралды 20 МЛН
Minecraft Creeper Family is back! #minecraft #funny #memes
00:26
The Most Important Algorithm in Machine Learning
40:08
Artem Kirsanov
Рет қаралды 428 М.
Large Language Models (LLMs) - Everything You NEED To Know
25:20
Matthew Berman
Рет қаралды 95 М.
Official PyTorch Documentary: Powering the AI Revolution
35:53
How to build Multimodal Retrieval-Augmented Generation (RAG) with Gemini
34:22
Google for Developers
Рет қаралды 60 М.
Jensen Huang, Founder and CEO of NVIDIA
56:27
Stanford Graduate School of Business
Рет қаралды 1,4 МЛН
Stanford CS25: V4 I Aligning Open Language Models
1:16:21
Stanford Online
Рет қаралды 22 М.
What's the future for generative AI? - The Turing Lectures with Mike Wooldridge
1:00:59