Data Science - Part XIII - Hidden Markov Models

[@brunovasconcelosmontoni9706]

After looking for a good introduction on HMM (including letures from prestigious universities worldwide), this is BY FAR the best material out there! Kudos, Derek! And thank you SO much!

[@jameshurliman8951]

This is one of the best videos on HMM out there; this should seriously have more love

[@sheikhakbar2067]

One of the best videos on Hidden Markov Models on KZbin!

[@jonimatix]

Hi Derek, Fantastic tutorial, very nicely explained! Since RHMM is no longer available, which R library you suggest we use?

[@cooky123]

Great video

[@Chitrang62]

Very Good Explanation. Thank you very much Derek.

[@mehdirostaee2828]

I have a question ... How much is the percent of winrate of this model ( for example if we have 100 days how many days can we predict correctly ? THX. and thanks for this speech it's very valuable for me.

[@saadabou5665]

thank you very much for all the videos

[@boningli7158]

very good, easy to understand, upvote

[@ramili0711]

I like that the slides are self-contained. Though, I would maybe explain equations in more details, what is each parameter and what does it correspond to, and maybe not read line by line, it's kind of boring for the listener :p Though, so far this is the most complete yet understandable HMM tutorial I've found on the internet.

[@DerekKaneDataScience]

+ramili0711 That is really help feedback and thank you for sharing. I am thinking about trying to add more supplemental content that is going through the actual code execution and less formal and scripted. I pobably used the lectures more as a crutch than I should have. My main goal was to give a reference to go back to here. I do think that I should spend some more time now fleshing out the mathematical underpinnings of the techniques. It is always challenging to balance the practical use of the technique, underlying theory/evaluation, and statistical underpinnings.

[@samidelhi6150]

Hi Derek , would you kindly do a video even short one regarding Hierarchical version of such hmm, and if possible how can I model / fit the emissions probabilities which follow skewed- t distribution , and and how can we deal in practice with the time- Independence assumption of such type of models ? Best Regards

[@saiwuppalapati2632]

Thanks a lot! This is awesome. It cleared all my doubts

[@rampysri5399]

Hi , I would like to ask that the stock example you game is just for presentation purposes, right ? In reality the stock series data normally has a upward trend. So the HMM will always have an upward bias.

[@DerekKaneDataScience]

+Rampy sri Absolutely.... and I wish it was a very strong ML algorithm that predicts stocks. The HMM will certainly help to decompose the stock signal but it does not always give a reliable prediction. Buyer beware and proceed with caution here. :0)

[@lotteharryvan7746]

Thanks a lot for the clear explanation! I was wondering if for the trained dataset you need to have the difference between de open and close price? Or is the open price enough?

[@alphazhang8965]

Hi Derek, I am new to R and trying to follow your example. But RHmm 2.0.3 is not in the normal package. I tried the followings but didn't go anywhere. Can you help? > install.packages("RHmm") Warning message: package ‘RHmm’ is not available (for R version 3.3.0) > install.packages("RHmm", repos="R-Forge.R-project.org") Package which is only available in source form, and may need compilation of C/C++/Fortran: ‘RHmm’ These will not be installed

[@hollowjohnny]

@Derek Kane all of the HMM examples i've seen use very few states. i.e: 3-5. Will HMM work effectively with say >100 hidden/observable states? thanks!

[@DerekKaneDataScience]

+Steve Hiehn - That is a fantastic question. From what I understand, you can build out more hidden states (100+) than the 3-5 we typically see. Nothing really precludes us from doing so. I wouldn't advise to taking it to the extreme and just remember that this may not offer significantly more strength in terms of predictive performance. There could also be risks of over fitting the model. Additionally, the multiple hidden states can become computationally expensive and might not be worth the added time and effort for processing. I do think that the principle of parsimony comes into play here when we are launching the HMM. Check it out with a smaller dataset and let me know what you are seeing. I would be curious to see your thoughts versus what you would expect to see.

[@AnekwongYoddumnern]

Dear sir, If I use Markov model with PIR motion sensor to detect human of distance between 1-7 m., how many state that I can set for this case ? Thanks.

[@oraz.]

How hard is it to implement hmm yourself?

[@mathankumars896]

Hey Derek, thanks for the video, Can you explain how to calculate the probability of the States(9.20).? please!

[@brunovasconcelosmontoni9706]

you can try using historical observations for those states and come up with an estimation for the transitions probabilities

[@mmontene]

Note: RHmm package is no longer available on CRAN

[@iandrumm6855]

So did tomorrow's stock prediction turn out to be correct? :-)

[@DerekKaneDataScience]

+Ian Drumm Probably not as much as we would have hoped for... ;0) Otherwise, we are in the wrong industry, my friend. But the HMM might help to give the general direction of the stock (assuming it is somewhat stable)?

[@iandrumm6855]

+Derek Kane Ah well, I'll have to find some other way to make a fortune :D, I'm finding your youtube lecture series very useful and enjoyable, thanks!

[@ernestamoore4385]

2:41 sequence of phenoms! :D

[@poisonza]

well the slides did help me understand the concept... .. THX... but I soon realized that you were just reading the entire slides line by line..... .. made me sleepy :( "The Reader"

[@DerekKaneDataScience]

+Hankyol Lee - Thanks and your comment made me smile... :) I wanted to make sure that the content came through and the presentation was laid out to be self contained. I definitely deviate a little bit in some of the other lectures to build on an idea, however, my main goal was to ensure that you had something you could come back to if you need it in a pinch. Thanks for commenting and hopefully they were of some use for you. I'm glad that you had checked them out.

[@alptugtokgoz5912]

That video was amazing and helped me to understand the whole hidden markov model!! Thanks!! If you are still interesting in HMMs (predictions especially) i have a question for you, can you give me your e-mail?

[@rameshkumarkrishnaswamy2340]

Great Video. I would like to implement for Tool condition monitoring. I need your help for the implementation. I will contact you through your mail.

[@nincako]

Pretty nice explanation but your voice tone is very monotone and not fluent. Maybe you need to speak a little faster. You are making silences between words unnecessarily. Example in 16:53. I couldn't understand anything from your sentence so I needed to read the slide and stop a little to understand the sentence in detail.

[@piotrrybka318]

For God's sake, please, have some mercy and give explanations to your formulas. It's not so obvious (except for you) what do you mean by "b", "k", "j", "i"...

[@nincako]

Pretty nice explanation but your voice tone is very monotone and not fluent. Maybe you need to speak a little faster. You are making silences between words unnecessarily. Example in 16:53. I couldn't understand anything from your sentence so I needed to read the slide and stop a little to understand the sentence in detail.

[@aysesalihasunar9563]

This might be useful for English as second language users. Also if you stream the video with x1.25 speed, it is still understandable and fast.