Stanford CS25: V1 I Decision Transformer: Reinforcement Learning via Sequence Modeling

[@ryanhewitt9902]

From a presenter's perspective the interrupting questions may have been mildly frustrating. From a viewer's perspective they were indispensable. I had exactly the same questions and the answers really helped me to understand the choices at a deeper level. Thank you!

[@adibkarimi1133]

Great talk! It's super exciting how RL with transformers performs so good.

[@prof_shixo]

For the question at timepoint 20:20 regarding the Markovian property and the fact that we are shifting the state using positional embedding, I think that the presenter reply was not that accurate. While he referred to it as state, actually what is there is an observation instead (e.g., an RGB frame) while the actual state is the encoder embedding given the encoded observation sequence along other token types. That view makes the state representation still to be a Markovian one as the encoder's embedding (the true state) is encapsulating all the history in the sequence and it is consistent (i.e., gives the same state given the same sequence), once the encoder is trained and frozen. In brief, I guess that was a confusion between observations and states in the RL paradigm.

[@sh4ny1]

Great observation !

[@rtBen]

On sequence modelling: - talk dated in the future oct 11, 2022 - streamed in July 2022 - “to appear in” neurIPS 2021

[@AdityaGrover]

ha thanks for pointing the typo! The talk was in Oct 2021.

[@empi_ai]

brilliant talk! very clear and interesting

[@albertwang5974]

This talking illustrate what's time travelling, what a future talking!

[@djethereal99]

Great talk, really interesting work!

[@LucasOSouza]

nice talk! got a bit spooked watching this today and seeing the date in the first slide 😅

[@m.h.w285]

How time flies!

[@jamesnatchwey1961]

🤣🤗

[@productlog5895]

beautiful lecture

[@imolafodor4667]

hi, how is this method considered to be offline rl if at some point you are calling env.step() aka an online step?

[@miquelnogueralonso2576]

Are the slides available ?

[@amrahmed2009]

Are the slides available, please?

[@markcarter6333]

Good stuff. Would it be better to leave questions in the chat? Or at the end of talk?

[@zyzhang1130]

Are LLMs stable to train though

[@sdfafds6823]

at the third page, "Larger model require fewer samples to reach the same performance ." Isn't that a little bit anti common sense? In my original opinion, more complex model needs to ingest more data.

[@paulcreaser9130]

October 11, 2022 ? Time travel?

[@paulcreaser9130]

Ahh, typo

[@sitrakaforler8696]

Future talks 😝👓