5. Stochastic Processes I

MIT 18.S096 Topics in Mathematics with Applications in Finance, Fall 2013
View the complete course: ocw.mit.edu/18-S096F13
Instructor: Choongbum Lee
*NOTE: Lecture 4 was not recorded.
This lecture introduces stochastic processes, including random walks and Markov chains.
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu

[@takashikashiwase3461]

when you don't wanna read or write anymore but still wanna do some math, well you've got to the right place.

[@SeikoVanPaath]

Some notable Timestamps 0:00:33 Stochastic Process 0:10:57 (Simple) Random Walk 0:32:43 Markov Chain 0:58:41 Martingale 1:06:47 Stopping time / Optional Stopping Theorem

[@luismoreyra6804]

Thanks pal!

[@biaschatterjee9836]

Thank you

[@HeitorSilvadeAlvarenga]

thank you

[@aliciaterok49]

thanks!

[@louislee1574]

Thanks!

[@edwardantonian7296]

This guy is absolutely fantastic. Could not have been explained more clearly, with a sound logical structure. People complaining about him should probably try lecturing themselves before offering their criticism.

[@nickfleming3719]

And people like you are confusing people even more when they get caught up in one of this guy's many mistakes and think that THEY are the ones who are wrong.

[@realwaynesun]

@@nickfleming3719 No offense, this is a free course for us, it's our own responsibility to find out wether the information is right or not when we get caught up in the instructors' mistakes. I mean the most important ability for self-taught learners like us is to be skepticism and check other information sources when we feel confused, not only in a free course but also in other paid courses. We can certainly say whatever we want in comments and I always learned a lot by some critical comments, however, I think it would be better to be grateful when we have chance to access high quality educations like this.

[@minutichaudhary4542]

@@nickfleming3719 aaaaaaaaaaaaaaaaaaa_aaa$zzzzq xzxzzxxzxaa$zzz azxaaaa_x¢

[@maxpopkov1432]

Let’s see you lecture, I really want to see your descriptions on such topics as: Real analysis, Complex Analysis, Functional Analysis, or Harmonic Analysis; oh please it would be delightful to see such confidence coming from you.

[@adamfattal9602]

@@maxpopkov1432 Easy game

[@bigollameo]

This guy has the most elegant writing style and manner of presentation.

[@AE-cj8ch]

Top universities have the best lecturers, making it easier for the students. It’s like a “poverty trap” for higher education.

[@chrstfer2452]

Luckily the best ones (MIT, Stanford) recognize that and release things like this OCW

[@sylvienguyen1010]

So you're talking about the boot theory in higher education?

[@jerryzhang7124]

insane lecture, tried so many different online materials, this one is clear af!

[@mattiascardecchia799]

Recursive argument at 28:00: Call p the probability you hit -50 first. There’s a 50% chance you hit -50 before you hit 50, by symmetry. Once you hit 50, the game is reversed, by stationary property. Hence p = 0.5 + 0.5 * (1 - p), from which p is 1/3.

[@Tyokok]

Thank you!

[@HUEHUEUHEPony]

Ahh, yeah idk why I didn't get that the first time

[@sahilsood1664]

0:00:33 Stochastic Process 0:10:57 (Simple) Random Walk 0:32:43 Markov Chain 0:58:41 Martingale 1:06:47 Stopping time / Optional Stopping Theorem For my reference

[@sahilsood1664]

49:03 ahh

[@aidanokeeffe7928]

This is a really useful comment!

[@francoisallouin1865]

Bravo for the stopping time definition . Very helpful

[@masterofallhesurveys]

Wow ! What a clear and concise lecturer. His ability in minimizing excess data to keep to the pure path of understanding is excellent. He is a star.

[@user-oz8mj1uj6e]

Thanks for ur efforts, I was just preparing for my first class about stochastic.

[@frasersmall181]

There is a reason he teaches at MIT this guy explains things so clearly and with ease! Im in H.S and can understand this! Absolutely amazing

[@Eizengoldt]

Stop the cap

[@intom1639]

This guy is amazing. His explanation is clear.

[@Grey_197]

OMFG! This guy is genius in explaining and presenting concepts.

[@ApiolJoe]

27:00 The argument to make it work the way the intuition of the student worked is via markov chains. Set up the states -50, 0, 50 and 100, write the transition probabilities, then calculate the absorption probabilities of the two recurrent states (-50 and 100) from 0 which give 1/4 and 1/2. The probability to end up with $100 is the probability of ending up becomes 1/4 / (1/4 + 1/2) (since the two other states will eventually bleed into either one of these states we know their steady state probability will be 0) which indeed gives 1/3.

[@samgao7996]

I am currently working on understanding the stochastic processes, and I am very confused by the concept of “a collection of random variables”, but the trajectory thing given by the lecturer helped me understand the concept a lot easier. For a continuous random process, if I sample at very high frequency, I will get several curves in the “x(t)-t” plain (the curve depending on the setting of the random process).

[@Boringpenguin]

49:03 ahh the "click" moment, seeing all the maths pieces coming together is really satisfying

[@ComposingGloves]

you sir are a gift! Thankyou for your clear lecturing!

[@michaelwatt5007]

Absolutely fantastic video, presented with such clarity. Extremely helpful. Thank you.

[@fidelesteves6393]

Would be a honor to be part of your class, professor. Your content is just awesome and your care with the understanding of the students can be noticed by your looks. Thank you.

[@bigollameo]

They have the audacity to call Choongbum Lee an instructor, when he can give a presentation so complete, elegant, and accessible that he could (and maybe should) teach ALL of the other professors at MIT a thing or two about how to give a lecture.and communicate ideas throughout it. This guy is @#$%ing amazing! What a beast. God, I feel stupid in comparison.

[@MrCmon113]

What is your problem with the word "instructor"? "How dare they call him a teacher! He is too good at teaching for that!"

[@xinkeguo-xue]

@@MrCmon113 I think they mean that he should be promoted to the position of professor. Instructors are not generally permanent positions at a university.

[@jamesfullwood7788]

MIT is a top research university, and as such, professors at MIT (and other research institutions) are judged mostly according to the quality of their research, not teaching.

[@caunesandrew1476]

I have seen quite a few MIT courses and every time, the teachers were amazing. This teacher is honestly not the best, although he is very much alright.

[@thedan2]

Amazing lecture. Made it A LOT easier to understand the concepts and applications. Books on subject don't usually give examples, which makes it that much harder to understand.

[@aliciaa470]

the best intuition behind stochastic processes !, really good

[@69erthx1138]

In the 1st and 2nd cases he's talking about delta hedge parity (in trading/market practice) as reflected by trend lines. In the 3rd case he's referring to the vol of vol, in this situation one must employee stochastic volitility models.

[@phillipthompson1580]

This is great and simple stuff for students studying the particle theory and Brownian motion

[@ajarivas72]

In 1996 I took the most mathematical advanced course I have ever taken: RANDOM VIBRATIONS. This course reminded me of that great course.

[@user-wu9zj1ro6o]

Don't spend your time for another channels. It is the best one!

[@4mb127]

Great lecture. Learned a lot.

[@user-bh9ei9fl1z]

stopping time 개념이 헷갈렸었는데, 정말 직관적으로 이해가 가네요. 감사합니다!

[@victorolagunju]

Thanks a lot. Very clear explanation.

[@user-ok4wr4zm5i]

a completely different level can not be compared with the first lectures

[@michaelcheng7597]

28:00 Following the thought process of the student from the audience, after the balance reaches $50, there is a 1/2 chance for the balance to reach $100 (overall probability = 1/4) or fall back to $0 (overall probability = 1/4). If the balance falls back to zero, we can consider that as the start of the second cycle, where the distribution of the conditional probability is the same as the first cycle (1/2 chance to reach $-50, 1/4 chance to reach $100, and 1/4 to reach $50 first then return to $0). Same for the third cycle, forth cycle, etc. Therefore, we can express the overall probability for the balance to reach $100 as the infinite series of 1/4 + (1/4)^2 + (1/4)^3... which gives us 1/3.

[@gamebm]

yes, and this is also consistent with Lee's solution, except that in the equation, one only needs to consider three (large) steps/grids, instead of a total of A+B steps/grids :)

[@CubeCubesen]

very good presentation, enjoyed it!

[@marcoardanese6013]

simply amazing

[@salmakrichene844]

OMG you are a genius stochastic process never looked this simple and intuitive

[@qinweizhang2849]

Continue the reasoning from 27:22: Assume the probability of the game ends at 100 is x. As probability of the game reaches 50 is 0.5; The probability from 50 to 100 is actually (1-x). So x=0.5*(1-x) --> x=1/3

[@leangsivlinh9372]

thank so much for MIT...it very helpful for my short time study at University.

[@youtubeiscruel3946]

To get variance, applied variance to both sides, var(sum(yi) over i). because yis are iid variance becomes sum(var(yi)). Var of each yi is one, and so variance is t. Var of each yi is one by computational formula of variance, E[yi^2]-E[yi]=1

[@KevinLanguasco]

Good presentation

[@danieldasilva2057]

I wish my lecturers could lecture in such a well structured way :(

[@nickfleming3719]

All you people praising this lecturer, saying how easy and simple he makes everything, are not helping. He's making tons of mistakes, and I'm thinking I must be going crazy since everybody else seems to think this is the best lecture ever.

[@faisalajin491]

What mistakes?

[@nickfleming3719]

@@faisalajin491 47:02

[@lucasgarcia78]

@@nickfleming3719 please explain further what is the mistake

[@HUEHUEUHEPony]

​​@@lucasgarcia78matrix values not in the right position

[@biliatersinaga720]

ank you for lecture

[@carolinaaldana5205]

Thanks a lot!!! Very good teacher :)

[@nkuduuchevictor7824]

WOW... THANKS FOR THIS....

[@EulerNumber_e_2.7183]

He is sooo good!

[@bereketyisehak5584]

Awesome lecture. Just found out he went to the same college for his undergrad as me

[@davidhashford9874]

Very good explanation.

[@DilanChecker]

I mean i dont get all these praises. The guy gives an overview of the topic but not rigorously at all. This is not the level of depth I would have expected but it serves me well in my preparations. It feels like I have to dive deeper on my own to get real understanding of the topic.

[@Nikifuj908]

It's a class for finance people. Did you expect a graduate course?

[@DilanChecker]

@@Nikifuj908 To me it seems it's more taylored towards Math Majors who want to specialize in quantitative finance.

[@vijayk7387]

Very easy solution for 28:00. P(B), P(A) be probabilities that B,A occur first respectively. Probability that we hit 50$ before -50$ is 1/2 and also probability that we hit -50$ before 50$ is 1/2. If we reach 50$ first, we see problem is flipped now, we are 50$ closer to B and -100$ closer to A. So P(B/start at 50$) = P(A/start at 0$) So we can write P(B) = 1/2(P(A)) = 1/2(1-P(B)) Solving this simple equation we get P(B) = 1/3 In fact for any A,B there is a point where we can flip the problem, so try to generalize this and come up with a proof.

[@kbisht3680]

this guy is a genius

[@ucleminh1616]

Who is this guy? His explanation on the subject is awesome

[@TheLukeStein]

Choongbum Lee

[@cmarkoz]

Very clear!

[@TamNguyen-bt7lc]

56:13 I think the confusion here comes from the fact that for the other eigenvalue, which actually is less than 1 and greater than 0, the corresponding eigenvector will converge to the 0 vector. The “sum trick” he did earlier wouldn’t work because v_1 + v_2 = \lambda (v_1 + v_2) doesn’t imply that \lambda = 1 when both v_1 and v_2 are 0. Hope I didn’t overlook anything!

[@gustavallen4992]

great job

[@fernandoiglesiasg]

Interesting to see a proof that the simple random walk is expected to take t steps in order to move sqrt(t), which is relevant in Markov chain Monte Carlo theory.

[@conoroneill8067]

Also, if the Riemann Hypothesis is true, then it means the variance of the number of prime numbers up to x compared to the expected number given by the Prime Number Theorem is proportional to sqrt(x), which is connected to this as well.

[@shakesbeer00]

1:15:16 you might want to say that E(X_\tau) = E(X0). Remember that X0 is a random variable too.

[@mariushav]

Or condition on the value of X_0

[@housemagicians]

@42:00 Isn't the transition prob matrix incorrect. Where the lower left corner should be P_{m,1} instead of P_{2,m}

[@kellybrower301]

Yes

[@user-xt3jo3sk6u]

In 47:42 Multiplying a 2x2 matrix with a vector (1,0) will give back the p11 and p21 which stands for working today and working tomorrow(p11) and broken today but working tomorrow(p21) not the probability working and not working.

[@N4mch3n]

it gives the probability of the machine working tomorrow, no matter if it's broken or not today therefore p reflects the probability of the machine working in 10 years. however he should've multiply with a vector (1,1) to adjust the same for q, since if you multiply the matrix with (1,0) the value of q will be 0

[@ghale10]

N4mch3n there cannot be a vector (1,1) as they represent probabilties of the machine working and not working.The rows of the vector must add upto 1. With (1,1) it implies that the machine is working and not working at the same time.

[@francoisallouin1865]

You are right. The error is that the entrees which should sum up to one are the ones in ROWS not columns. Because he is not multiplying A^3650 by the correct vector, he had to amend the matrix A when computing the eigenvector in 52:00.

[@divyakrishnamalik3933]

Does anyone knows about more basic content so as to form a stonger intuition and be able fathom this deeper? Also recommendations for time series analysis will be appreciated as I'm basically working on that.

[@kingshukdutta2064]

At 41:35, it should be P_m1 instead of P_2m.

[@kenichimori8533]

確率方程式＝Stochastic Processes I

[@michal234486]

the last corollary is neat indeed, but the assumption of the theorem seems not be fulfilled. there does not exist T>tau, since it's possible for the random walker to bump between the lines -50 and 100 as long as it likes... can sb clarify?

[@gamebm]

58:10 Someone asked whether the algebraic manipulation led to the (seeming incorrect) conclusion that all eigenvalues lambda are 1. That was not true, since the assumption for that equation is that we are dealing with a stationary state, and therefore, the conclusion is for a stationary state, its eigenvalue must be 1, as stated by Lee.

[@eigentejas]

The equation was just an eigenvalue equation for A - it didn’t assume anything about stationary state. The correct argument, against the incorrect conclusion that all eigenvalues of A is 1, is that (v1 + v2) can be 0 and hence you can’t divide that out to conclude much about lambda. The case where you can do it turns out to be when v1 and v2 are positive - thus the theorem about the unique highest eigenvalue isn’t broken.

[@gamebm]

@@eigentejas You are correct. If one assumes a stationary state (some vector (p, q) of probability that remains unchanged by further multiplying A from the left), it simply implies the existence of an eigenvalue of 1.

[@rationalmind3567]

what is the prerequisite for this course, does anywhere i can find a detail simplified version of all the explanation relating to this topic.

[@mitocw]

Here are the prerequisites for this course: 18.01 Single Variable Calculus, 18.02 Multivariable Calculus, 18.03 Differential Equations, 18.05 Introduction to Probability and Statistics or 18.440 Probability and Random Variables, 18.06 Linear Algebra. We did a quick search of our videos and maybe this video would help? kzbin.info/www/bejne/bXS8iaxvf5l0l9E See the course on MIT OpenCourseWare for more info and materials at: ocw.mit.edu/18-S096F13. Best wishes on your studies!

[@Juoa794]

Isn’t discrete the same as continuous, at the limit?

[@Connie2216]

Thanks man

[@ryan_chew97]

choong bum the humble korean god

[@sandeepjangir6079]

Amazing Lecture, I think at 57:56 , the equation v1 + v2 = lambda(v1+v2) only holds for lambda = 1(the only case where both v1 and v2 can be positive) , for the other eigenvalue v1+ v2 =0. This Should extend to any dimension.

[@jianingzhuang104]

Brilliant! Thank you.

[@alvaroalfonso8754]

i think all people that writes "I changed video´s speed to 2" is trying to say: "i am more brilliant than anyone here".

[@legendariersgaming]

And I bet if you ask them a week later what they learned, 99% will not remember a thing from the lecture. Unless this material is just review for them, math like this needs to be savored and digested for complete understanding

[@pranayvenkatesh8815]

@@legendariersgaming Not necessarily. I suppose those who say it are probably boasting. But sometimes, it isn't difficult to get everything even I you watch in 2x

[@ButerWarrior44]

I changed video speed to x4

[@nermeenal-najam1034]

درس رائع جدا

[@HenriqueSantos-xd1eg]

Show me the lectures of the Poisson process

[@moneyeye24]

@48:24 "probability distribution of day 3651 and day 3650 are the same." @54:04 if av=v, day 3651=day3650, then the machine of his example last forever?

[@adwoayeboah1537]

This is a good video. just that there is a little mistake under the transition matrix. With the matrix provided, the last entry under the first column should have been P subscript 3m and not 2m.

[@RandomPerson-pp7ti]

I believe it should have been m1.

[@AReasonableName]

I'm confused about the machine working/broken example. At 0:49:09 I believe it should be [1 0]*A^3650 = [p q]. Then for eigenvector at 1:17:40 it should be A(transpose)*[v1,v2] = [v1,v2], as you can see he modified the matrix from A to A transpose. With the way it is shown here p, q should have different meaning.

[@mathisdifficult666]

i understand now😂 the matrix A at 0:49:09 is wrong😂

[@satvikp.s2688]

Yeah I was having this exact same confusion, what you've said seems to be perfectly right, now it all makes sense to me. Thanks a lot!

[@keerthijessy4861]

Thanks

[@TheAlx32]

There is a mistake at 1;15:23 . An Expectation of a random variable is a number not a random variable. So E(Xtau)=E(X0).

[@dharmiknaik1772]

49:10 blew my mind!

[@moabdi1760]

Tanks

[@sonalimahajan8960]

does stochastic process varies linearly with time? because in your first example function f(t) varies linearly with the time. in some books it is referred as random process. quite confusing ,guide me

[@tomofadown]

Not necessarily. You may have some stochastic process with linear delta to time but you can also have stochastic processes with non linear delta to time. For instance think about the process X(t) = t**2 for all t.

[@ARIZABEST]

Can someone explains me whats the difference of the stochastic processes number 2 and 3 defined at minute 4:30 ? Thank you so much

[@Pedritox0953]

Very interesting

[@Adam-rt2ir]

In the definition of p_ij, was homogeneity assumed anywhere? Maybe I missed it, but it definitely needs to be a homogeneous process! That means, p_ij shouldn't depend on t.

[@alexanderchristiansson2335]

I noted this too. I don't think it was mentioned anywhere.

[@dhruvvansrajrathore2148]

Thanks. I was also wondering about this and now the computation at 43:15 makes sense.

[@Marmann100]

Can someone explain why tau would be bounded in the case (i) at 1:12:23 ?

[@shubhamsumanvishwakarma7113]

47:52 Shouldn't we premultiply here,. i.e [1 0](A^3650) = [p q] pre-multiply (with [1,0] as 1x2 vector) instead of post-multiply.

[@user-dh3wo1zp5k]

This guy is the best; makerere shd employ him

[@debmallyachanda5384]

I don't understand how 2 and 3 are different? They seem same to me. 6:00

[@HUEHUEUHEPony]

Uhm one is 2 paths and the other is infinite paths

[@TheAwesomoe]

19:14 what's the name of that theorem?

[@yamiashigaru]

Chanson reflection principle of Wiener process / brownian motion

[@Boringpenguin]

and it is useful for pricing barrier options and lookback options

[@123TeeMee]

Can technically everything be a markov chain if the history is included in the current state?

[@ahmedsikander7707]

This is Dr Ahmed i am referring your lecture to my students

[@kakkar2468]

At 19:12 , the probability of a N(0,1) to be between -1 and 1 is ~68%, not close to 90% or more as said. Otherwise, great lecture.

[@MaximPodkolzine]

shailesh kakkar I believe he meant the probability to be within 100 standard deviations (which is virtually 100%, not close to 90% =). And there are a lot of minor mistakes in this video and the two before, the instructor is not very well prepared. But it's still useful

[@serrjosl]

+Maxim Podkolzine No, the answer is right, he means that the total area under the bell curve its 1, or 100%, but in the real word, you nead just 2 standard deviations boths sides to the total area to stay very close to 100%

[@dicksonh]

+serrjosl p(-1

[@serrjosl]

+dicksonh Well if you do that, you miss 1/3 of the boundaries values and your forcast will be completely wrong, but Who Am I to change your point of view.😉

[@Rannosaurus]

I think it should be N(0, 1/4) at 17:13

[@remlatzargonix1329]

Could these process also be index by both time and space (or location) where location may be derived from GIS Co-ordinates? So they could be used for spatial data analysis or spatio-temporal data analysis.

[@LeCoolCroco]

Remlat Zargonix yes, but space is just another discrete space variable

[@mvmlego1212]

I think that I don't understand the independence property of random walks, given around 21:00. His verbal explanation sounds a lot like the Markov property, but I doubt that he would define the same thing two different ways without saying that they're equivalent. Are there any systems with the independence property, but not the Markov property, or vice-versa?

[@nazaninrahimirad7344]

wonderful teacher, but I couldn't understand the last example. Why the probability is=0?

[@nirmalkumarsingh1092]

At time before 44:40 he said random walk does not have finite set.. But he earlier said that the values are limited under a curve with standard deviation of root t.? Anyone please help

[@HUEHUEUHEPony]

But not finite

[@haneulkim4902]

@17:13 Can anyone elaborate on 1/sqrt(t) X_t ~N(0,1)? I understood high level conecpt of C.L.T. however cannot really understand what X_t is referring to. is it mean of set of observations? or one random variable.

[@HUEHUEUHEPony]

X_t is the random variable at time t ~ means it approaches a normal distribution mean 0 std 1

[@user-ok4wr4zm5i]

The first time I see a teacher who rewrites everything

[@vaishnav4035]

Thank you Sir 😃😃

[@Grassmpl]

at the very last corollary, what is the T from the theorem thats applied to the corollary ?

[@Grassmpl]

Personally I argue that this T doesn't exist, so the thm cant be used. Although P(tau = infinity) = 0, we still have that for all natural number k, P(tau>k) >0, even though this probability can be really small as k gets large. So whatever T you pick, I can always argue against your claim using k=T+1.

[@pycool7595]

Shouldn't it be [1 0] * A^3650 = [p q] ?

[@kaydenwoodsmusic]

I believe it has to do with the eigenvector relationship... Av = (lamba)(v)

[@aborucu]

@23:00 how can simple random walk be stationary when variance grows with time ? Did he mean increments are stationary ?