Stanford CS25: V3 I Retrieval Augmented Language Models

  Рет қаралды 162,463

Stanford Online

Stanford Online

Күн бұрын

December 5, 2023
Douwe Kiela, Contextual AI
Language models have led to amazing progress, but they also have important shortcomings. One solution for many of these shortcomings is retrieval augmentation. I will introduce the topic, survey recent literature on retrieval augmented language models and finish with some of the main open questions.
More about the course can be found here: web.stanford.e...
View the entire CS25 Transformers United playlist: • Stanford CS25 - Transf...

Пікірлер: 65
@erniea5843
@erniea5843 8 ай бұрын
I love that this content is freely accessible to everyone. Lots of helpful information being shared here
@nintishia
@nintishia 7 ай бұрын
The amount of research work on retrieval augmented generation for large language models has exploded in recent times. Thanks to the speaker for directing attention to the most significant bits.
@kingki1953
@kingki1953 4 ай бұрын
My lecturers in my university never explain those things, thanks for this free lecture
4 ай бұрын
Good lecture, offers a good summary of the literature on RAG.
@heisenberggiao9303
@heisenberggiao9303 2 ай бұрын
Thank you for the great contents!
@Arvolve
@Arvolve 8 ай бұрын
So many great ideas here! Fantastic resource, thank you.
@loopaal
@loopaal 8 ай бұрын
this is what I needed, thank you sooooo much!!!!
@johntanchongmin
@johntanchongmin 8 ай бұрын
Pros: Gives a brief overview of many RAG methods Cons: No intuition given which is the key reason for why it works for the different methods Would have preferred more insights rather than just describing the papers, but overall thanks for the video!
@99BLACKLP
@99BLACKLP 8 ай бұрын
Great insights and video!
@ronitakhariya4094
@ronitakhariya4094 8 ай бұрын
just the thing i wanted thank you so much.
@NerdyX90
@NerdyX90 8 ай бұрын
Thank you.
@reslleygabriel
@reslleygabriel 7 ай бұрын
Awesome content, thanks for sharing!
@deeplearningpartnership
@deeplearningpartnership 7 ай бұрын
Really cool.
@dongxu9013
@dongxu9013 7 ай бұрын
The best talk about RAG so far
@morespinach9832
@morespinach9832 7 ай бұрын
How many have you heard, right here on KZbin? Lots of actual hands on info out there and in 1/3rd the time. This one had loads of intro forever before getting into specifics.
@edwardmitchell6842
@edwardmitchell6842 6 ай бұрын
@@morespinach9832 Can you recommend some sources? I'm compiling a list.
@yangzju
@yangzju 5 ай бұрын
@@morespinach9832 can you recommend some videos?
@comunedipadova1790
@comunedipadova1790 5 ай бұрын
@@morespinach9832 What lecture do you suggest for a more practical (code) view?
@morespinach9832
@morespinach9832 5 ай бұрын
@@comunedipadova1790 plenty of them - search for these keywords.
@蔡鉎驊
@蔡鉎驊 6 ай бұрын
3:34 What chatgpt was really about - fix the user interface to LM 12:00 Frozen RAG
@muhannadobeidat
@muhannadobeidat 7 ай бұрын
It is good to see the whole spectrum of options, but what would be a practical way to get started on this? His old colleagues in HF actually have an excellent book that goes through many different ideas including RAG in chapter 7 where they do an excellent job explaining context and giving you options for implementation.
@samferrer
@samferrer 8 ай бұрын
Actually we can see a first sign of language models in Shannon's 1948 paper, A Mathematical Theory of Communication.
@ssssssstssssssss
@ssssssstssssssss 8 ай бұрын
Language models started to appear in the 1800s.
@therainman7777
@therainman7777 8 ай бұрын
Language models actually appeared sometime during the Cretaceous period, though scientists aren’t quite sure of the exact year. They think the stegasaurus might have had something to do with it; all these people who think OpenAI invented them are so wrong.
@duongkstn
@duongkstn 7 ай бұрын
Hi, about Atlas. You said that we can update the Retriever. At 42:00 is some retriever loss, but what about pair label (question, positive paragraph, negative paragraphs) like normal retrieval model - do they contribute to the retriever loss ? I read codebase of Atlas, and do not find that kind of loss
@echofloripa
@echofloripa 13 күн бұрын
Does these embeddings tools such as FAISS work with other languages other than English?
@trinityblood5622
@trinityblood5622 8 ай бұрын
Could you please share the slides?
@ginogarcia8730
@ginogarcia8730 8 ай бұрын
WE ARE IN THE FUTUREEEEEE
@deconcoder
@deconcoder 8 ай бұрын
THe bit about where this came from was funny, but your search just gave a less silly answer. Go reread Shannon's 1948 paper that invented Information Theory. Yes he did not talk about using neural nets (which did not exist) but he did talk about probability.
@lambertch
@lambertch 6 ай бұрын
Shouldn’t it be Shannon, 1949?
@ritikdua7225
@ritikdua7225 6 ай бұрын
does anyone have list of research papers metioned in this video?
@zhuolin730
@zhuolin730 Ай бұрын
where can I find the slides?
@vaibhavjoshi7961
@vaibhavjoshi7961 Ай бұрын
I did like the joke you made.
@kingki1953
@kingki1953 4 ай бұрын
why there's no framework to make the process was easy
@user-wr4yl7tx3w
@user-wr4yl7tx3w 3 ай бұрын
there is too much review information
@Arthurlvaz
@Arthurlvaz 6 ай бұрын
It's not entirely clear what "frozen rag" and "retrieve rag" refer to.
@MedTn-i3h
@MedTn-i3h 3 ай бұрын
worth dozens of "shiny" "trendy" "tik-toky" videos over there, without asking you to subscribe and leave comments..
@mahkhi7154
@mahkhi7154 8 ай бұрын
Us Mandalorians CYBERMEN: Understand The Material - Given To You By Minions, better Than The Minions. Your Gods Gods. Ask Them.
@therainman7777
@therainman7777 8 ай бұрын
Could you try writing that again but having it not be nonsense this time?
@starmountpictures
@starmountpictures 8 ай бұрын
meh
@mahkhi7154
@mahkhi7154 8 ай бұрын
You're a P1G. Memorising and Repeating without Understanding. What is a Sentence? A Collection of Concepts. The Brain can Learn 10's of Thousands of concepts. It Has Hardware To Help it. Concept: "Doing" brings up pictures of Making a Cake or Assembling a Chair. Concept: "What" brings up Pictures of "Asking someone a Question" or Deciding Colour Show to Buy. A Computer Doesn't Work Like That. All it Understands is Numbers and Maths. All These "Concepts" have To Be Encoded in Numbers and Maths and Rules Applied To Them. The Computer Doesn't Know What The Character "B" is. All it Know That if it Finds The Number "66" in Memory, it Draws Something That Looks like "B". Spelling. It Doesn't Know What Spelling is. All it Knows is "66,65,84" Valid for word BAT. BAB 66,65,66 is an Invalid String of Numbers and Not in the Dictionary. 10,000 Concepts Need - Modelling and Rules Creation (100's of Thousands) - For the Computer To Understand. That isn't an Easy Thing To Do: Trillions of Lines of Java Code.
@therainman7777
@therainman7777 8 ай бұрын
Maybe if you were a computer maybe you would understand that you Don’t Need To Capitalize Every Word when writing English. And trillions of lines of code? You know nothing about coding; in all of human history everyone out together has not written a trillion lines of code, let alone in a single computer or a single program. P.S. Your comment was utter nonsense, it contributed nothing, and it meant nothing. Have a nice day.
@dongmo6546
@dongmo6546 8 ай бұрын
Thank you
@capucinnolover
@capucinnolover 6 ай бұрын
Nicely explained in just right technical details. Thank you!
@velociraptor75013
@velociraptor75013 6 ай бұрын
Excellent content, thanks for the references!
@AIPoker-tj6lr
@AIPoker-tj6lr 7 ай бұрын
What about scann by google ??
@amansinghal5908
@amansinghal5908 7 ай бұрын
Are there assignments for this?
@jstello
@jstello 8 ай бұрын
I love how everyone tries to hide the fact that OpenAI is 100% the reason everyone is watching this video
@pw7225
@pw7225 8 ай бұрын
No one is hiding anything
@a3mia3mi82
@a3mia3mi82 8 ай бұрын
“Ignorance” is not equal to “facts”
@therainman7777
@therainman7777 8 ай бұрын
I know, isn’t it hilarious? The bitterness and jealousy is so transparent. It reminds me of the scene from The Social Network where they’re trying to claim credit for his creation and he says “You know it’s really not that complicated. If you guys were the inventors of Facebook, you’d have invented Facebook.” Yann LeCun likes to drone on on Twitter about how everything OpenAI is doing is “old” technology. Well if it’s so old, how come hundreds of different companies filled with smart people were trying for years to make a chatbot that was worth using but until OpenAI no one seemed to manage it? Yes, OpenAI didn’t invent the Transformer. We know. Who cares? They clearly solved dozens of incredibly difficult engineering problems that no one else had been able to solve, and gave the world a language-based AI that was actually _useful._ As evidenced by the fact that it was the fastest growing app in human history, by an extremely wide margin. And as soon as they do it all these idiots turn up basically saying “I could have done that 10 years ago, I just chose not to.” Yeah, ok. It’s so pathetic.
@Gingnose
@Gingnose 7 ай бұрын
Maybe because only you are the one that is obsessed by the idea of starting learning something new but merely due to its "novelty and coolness" is something to be ashamed of and you project that idea on others which make you think others are watching this video with the same reason but hinding that information (and trying to look smart) because it is emabrassing to declare and you think that way too. However I will remind you that trying to learn new things whether it is because it is fashion or cool is nothing to be ashamed of and a great excuse to start new things.
@srh80
@srh80 7 ай бұрын
Whats an Open Al?
Stanford CS25: V4 I Aligning Open Language Models
1:16:21
Stanford Online
Рет қаралды 22 М.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, with Patrick Lewis, Facebook AI
1:22:32
Natural Language Processing presentations
Рет қаралды 13 М.
когда не обедаешь в школе // EVA mash
00:57
EVA mash
Рет қаралды 2,9 МЛН
Миллионер | 1 - серия
34:31
Million Show
Рет қаралды 1,7 МЛН
💩Поу и Поулина ☠️МОЧАТ 😖Хмурых Тварей?!
00:34
Ной Анимация
Рет қаралды 1,9 МЛН
From Small To Giant Pop Corn #katebrush #funny #shorts
00:17
Kate Brush
Рет қаралды 69 МЛН
The Turing Lectures: The future of generative AI
1:37:37
The Alan Turing Institute
Рет қаралды 604 М.
The Most Important Algorithm in Machine Learning
40:08
Artem Kirsanov
Рет қаралды 436 М.
Stanford CS25: V4 I Overview of Transformers
1:17:29
Stanford Online
Рет қаралды 55 М.
Official PyTorch Documentary: Powering the AI Revolution
35:53
What's the future for generative AI? - The Turing Lectures with Mike Wooldridge
1:00:59
[1hr Talk] Intro to Large Language Models
59:48
Andrej Karpathy
Рет қаралды 2,2 МЛН
AI and Quantum Computing: Glimpsing the Near Future
1:25:33
World Science Festival
Рет қаралды 432 М.
Machine Learning for Everybody - Full Course
3:53:53
freeCodeCamp.org
Рет қаралды 6 МЛН
когда не обедаешь в школе // EVA mash
00:57
EVA mash
Рет қаралды 2,9 МЛН