16. Markov Chains I

MIT 6.041 Probabilistic Systems Analysis and Applied Probability, Fall 2010
View the complete course: ocw.mit.edu/6-0...
Instructor: John Tsitsiklis
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu

[@MsAlice729]

This guy literally helped me pass all my stats courses! He is a bomb... If i ever visit MIT, i will drop by and thank him in person lol

[@yubarajpoudel1]

i can remeber this guy Borat in MIT all the way from kazakistan

[@alimustafa2682]

Make sure he doesnt explode beside you

[@trejohnson7677]

Your playlist containing this video is god-tier

[@phyowaihan2492]

Really?

[@randobianco]

@@yubarajpoudel1 great success

[@ashishpaniya5805]

sir your style of explaining is outstanding...Thanks, to MIT for doing this noble work which benefits hundreds of thousands of students in the world......keep up the great work!!

[@MrSinalta]

Not only students , some people like to learn during their part time and this video is excellent

[@ajayeswar1409]

I like how well he is trying to give us those intuitions

[@Marion89gr]

The professor's accent sounded exceptionally understandable and familiar to me, and then I saw that this brilliant teacher is from my country! Thank you so much for the lessons, μεγάλο ευχαριστώ από την Ελλάδα!

[@KBSINN]

greece

[@wuzhai2009]

Master class in presenting complex concepts -- state by state.

[@dawveed84]

Such clarity and elocuence! Great lecture

[@ApiolJoe]

This is amazing! I studied chemistry at university few years ago, and that definition of Markov Chains really makes me think of what we did with the equilibrium of the reactions with the different molecules. This is exactly the same kind of definitions: the different states are our different molecules, the probabilities have exactly the same role as our "reaction speed", and the conclusion is the same: the equilibrium is unique for a given system. Actually during the lecture I was trying to guess just from the diagram what the equilibrium would be hahaha. I feel sad that we were not even given a mention about markov chains back then. I was struggling with the last 3 lectures in this course, and even more in the assignments, but I'm so happy to see these descriptions I am already intuitively familiar with that my pain just flew away!

[@ACTHdan]

typically part of a 3rd semester calc course.

[@areliastorga4942]

it all happened that i found this lecture where in fact that i got a case study with regards of Markov analysis. it really helps me a lot, and very comprehensive lectures.

[@alexlu2844]

Sir, Ty for your video. Easy to understand your teaching, I don't need to go school anymore.

[@mrturnables1]

This is the best introduction to markov chains ever!!!!!!

[@TheMariacg031]

A million times better than my professor

[@benabbouaissa1991]

+Maria Gutierrez he is the best Probability teacher , and of course this is why MIT costs too much ;

[@jonathandobrowolski6941]

guys this guy is the best no cap.

[@kolo6518]

What a brilliant professor. This was so so helpful

[@tonyleung2442]

Man. 1.5X speed helps me to get this done in half hour. Thanks!

[@giuliom4886]

My teacher was a total failure in teaching prob and stat. He is making my worst nightmare in a pleasant discovery.

[@imadelachiri5475]

That's the best probability teacher ever!

[@ruili6415]

I was so suprised when the rij(101) = rij(100). Beautiful.

[@quixata]

Simply wonderful teaching

[@algebra5766]

Brilliant Job done here ....

[@neslihansahin1067]

very understandable and fluent . I ilked it. thank you

[@nngnn152]

great lecture. i wanted to give a standing ovation when the video finished. lol.

[@raynumehra]

very well explained!

[@atxvet]

Jeez, slackers... Had I been lucky and/or wealthy enough to attend MIT, I would not have shown up late to my classes!

[@Damion00000]

While economic advantage is understandable, what is completely irrational is that you attribute being admitted to an institution of this nature to luck - this thought alone could sabotage your life.

[@abhishekshivkumar734]

Stephanie P being born rich is luck, being well connected is luck, being a legacy is luck, going to a good school is luck. get ur libertarian nonsense out of here.

[@videofountain]

I also attended a few class at MIT and was born with a thrift store well worn stainless steel spoon in my mouth. At least at one point in the past, there are student loans and financial assistance at a number of expensive schools.

[@computerscientist5953]

that explains why not everyone works at the top positions after graduation. There's always those "top 5%" of students who get the cherry, and the "bottom 5%" who end up at "meh-" positions on average (or can't find a job at all)

[@MrCmon113]

@@computerscientist5953 None of which has to do with when you arrive at a lecture. Being on time for lectures is just about the least important thing about studying and there is a myriad of good reasons to be late.

[@taniarahman9028]

very good teaching

[@zhangzezhou145]

thx, now i know how is like a course in MIT..

[@raniab7585]

you're THE BEST wow thank you !

[@zoozoo5491]

He is teaching probability through telling a story instead of saying again the formulas and definitions - what most teachers do.

[@Bandoolero]

really great! much better than my professor.

[@sumitkhangura6006]

I love this prof

[@NoobishAlpha]

16:20 This phrase was inspiring.

[@saadhamama2112]

nice explaination, very usefull thank a lot

[@ashwinireddypingali2994]

nice explanation..thanks

[@a3090102735]

Great, super clear. I like his accent now

[@napoleonpolymeneas8056]

Wang Yi is a Greek accent i believe!

[@kokitoboy]

very good lecture

[@t-gee7516]

Fantastic lecture!

[@adityasahu96]

If a teacher makes it complicated then he is not a good teacher. If he makes it super easy then only he is a good teacher. :-)

[@cemisgezeksakini406]

Ευχαριστώ ΕυχαριστώΕυχαριστώΕυχαριστώΕυχαριστώΕυχαριστώΕυχαριστώΕυχαριστώΕυχαριστώ!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[@incxxxx]

Excellent!

[@weinerTech]

just i can say great

[@HidayaRegragui190]

I love this man

[@adityakanade8271]

Why did we use condition probability for r21(n) @46:00. Why is r21(n) not 0.3 instead?

[@yw834]

r21(n) = 0.3 when n = 1. However, when n goes to infinity eventually you will get out of state 2 and you have equal probabiliy to go to state 1 or to state 3

[@raidayoub8379]

Very clear

[@alejandroquintoschoy3919]

I'm taking the course Probability (EDX MITx) which really worth it. His book is one the best. Introduction to probability, highly recommended.

[@zishanliu9]

excellent！

[@aashishraina2831]

excellent

[@PeruanoEnTurquia]

excellent!

[@zaza2010full]

Thank you

[@lihuil3115]

what's the sample space, experiments of Markov Chains? If Markov Chains has two steps, is the experiment of the first step the same as the experiment of the second step?

[@JK-sy4ym]

smart example.

[@mpbasics8285]

the way he talks hooks me...

[@cemsavasaydn7053]

Why is he using recursion but not a transition matrix, is it because recursion is a more general notation?

[@gabreil047]

Great! Thanks.

[@ramilezernest7296]

how can i compute the probability of been at one point before other point, starting from any point. for exemple, been in point 4 before the point 2 starting at any point?

[@freemanguess8634]

I'm interested in this but the application is more as predictive software that can take the data that's collected and make predictions based on everything my question comes in at is it possible to use several other programs I guess like they use in Linux pipeline many large programs together to create a super program I'm interested in a program that can predict everything from everywhere and trying to get the predictive error down to less that 1 percent

[@shidharthrouth]

pardon me for not being much bright ... but ... can anyone tell me how to calculate the probability of a change of state from 1 to 2 (suppose) if time step n is known with no existing states in between. Any help is appreciated. Thanks in advance.

[@SandeepG118]

Can anyone clarify my question....! At 32.01 it was told that r12(n) = 1- r11(n)... it is correct intuitively but if i calculate r12(n) using normal method i got it as r11(n-1)0.5+r12(n-1)0.8 which is not same as 1- r11(n) (Here r11(n) = r11(n-1)0.2+r12(n-1)0.5)

[@ambastashobhit]

Sandeep G Hi Sandeep...r12(n) = 1-r11(n) intuitively as well as mathematically. Just for verification add up the RHS of both the equations r11(n) = r11(n-1)0.5+r12(n-1)0.2 and r12(n) = r11(n-1)0.5+r12(n-1)0.8. Addition of RHS will give us r11(n-1)+r12(n-1) which equals the LHS: r11(n)+r12(n) In other words you will notice that: r11(n)+r12(n) = r11(n-1)+r12(n-1) Continuing the same process till initial stage is reached, r11(n)+r12(n) = r11(n-1)+r12(n-1) = r11(n-2)+r12(n-2) = r11(i)+r12(i) = r11(0)+r12(0); where (i) will denote any subsequent stage and (0) is the initial stage. Now we can see that either r11(0)=0 or 1 as in the initial stage either we will be in state 1 or in state 2, hence the total probability, r11(0)+r12(0)=1 I hope that you could understand it....in case you don't just write down the equations on paper, it will be easier.

[@thilinawickramasinghe6235]

nice lec

[@gathersteel]

Is he Markov? :P

[@kaursingh637]

thank u sir for excellent lecture --pls divide long lecture in to short lectures

[@mrturnables1]

Did he just start by saying this is a lot simpler and more intuitive?? Then why did my lecturer always sound like he was from outer space???

[@MrCmon113]

Because there is a lot of terminology around Markov processes.

[@SomeHeavensStation]

In the r21(n) scenario (47:10) it was said that the probability is 1/2 (due to the oscillation between the two possibilities), however if the sum does not converge, then --by design -- doesn't it have no sum ? In other words, is it not false to say it equals 1/2? (And how am I looking at this incorrectly, if this is, in fact, not the case?)

[@andrzejkwasniewski1266]

The probability of it staying in 2 is (0,4)^n which converges to 0. So for large n the probability of leaving 2 is 1, leaving you with r21=1/2

[@Alakeshkalita]

in r11 column after two transitions the value should be .225 not .35... @34.50minutes

[@vaibhavkasotiya8452]

♥♥♥♥♥♥♥♥♥

[@riccardokiefer5387]

Wtf i’m spending 2+k euros per year in my university to attend classes where professors aren’t even half as good as John explaining stuffs, education is fucked

[@holalluis]

I created an interactive table that reproduces the simple example described in this lecture! dl.dropboxusercontent.com/u/2642357/markov.html

[@holalluis]

***** with Javascript programming language. If you right click in the page, you will view the source code

[@HUEHUEUHEPony]

@@holalluis the link is dead

[@sumanthbalaji1768]

Where my IIIT Hyderabad people at

[@mkutkarsh]

✌

[@sreekanthpallavoor3048]

Excellent!