Finding Policies Two - Georgia Tech - Machine Learning

Watch on Udacity: www.udacity.co...
Check out the full Advanced Operating Systems course for free at: www.udacity.co...
Georgia Tech online Master's program: www.udacity.co...

[@chaitanyasharma6270]

I liked the analogies ypu were using previously, i think of this as telling someone to go higher or lower when you ask them to guess a number, they eventually converge

[@bharadwajadi18]

Great video. Thanks for making this video :)

[@ZohreYahyaee]

I have a question, what is U(s)? in the previous videos it is defined to have U(s0, s1, s2, .....)= sigma of R(si) (with/ without gamma (discount factor)) as R(si) is defined in the model, U(si) is also defined. we don't need to select it arbitrary.

[@MinhVu-fo6hd]

I think it does help by the fact that the incorrect initial guess being discounted when added to the true reward. This makes the next guess move toward the correct direction faster, so faster converge to the true Utility.

[@oldcowbb]

they mentioned that actually

[@bikcrum]

Is solving non-linear equation iteratively somehow related to Newton-Rapson method? (Take initial guess and converge recursively).

[@IndrajitRajtilak]

Why isn't U(S) equal to the R(S) + max(U(S')) where max(U(S')) is the max utility of its next neighbour? Each of the immediate neighbours capture their immediate neightbours. Why have a summation there? Why not define it as the Reward in current state + max utility of next state, what am I missing?

[@Arik1989]

If still relevant: As I understand it , in the general case the model (T) is stochastic , so when deciding to take action (a), you have to consider that there's are several states (s') that you can end up in. In the grid example, you pick action (a) to go up, you have a chance to go up, left or right, so you have to calculate the EXPECTED VALUE: The probability of reaching each possible state (T(s,a ,s')) - up(0.8), left(0.1), right(0.1) multiplied by the respective utility U(s')

[@mcab2222]

Also you miss the decaying factor. Just summing them would be neglecting decaying factor

[@googoonight858]

it feels like that Charles does not quite know what he's talking about