Inverse Transform Sampling : Data Science Concepts

Let's take a look at how to transform one distribution into another in data science!
Note: I should have included a lambda in front of the exponential PDF. I mistakenly forgot it. I appreciate the comments which helped me realize this mistake.
---
Like, Subscribe, and Hit that Bell to get all the latest videos from ritvikmath ~
---
Check out my Medium:
/ ritvikmathematics

[@shivamathghara2870]

pdf of exponential is (lambda)*e^(-lambda x)

[@gomolemartifex]

This video just transformed my life

[@yaningwang8629]

omg you saved my stats degree, much thanks

[@mjf1422]

Thank you so much for doing these videos.

[@fyaa23]

I can't agree with you more.

[@sepidet6970]

That was a great intuitive explanation of inverse Transform Sampling. It seems so easy to me after watching this video,. Thanks a lot.

[@Arriyad1]

It only seems easy. Inverting the cdf is difficult. The exponential distribution is kind enough to let itself invert, but many other ones are mean.

[@bhaskarroy8753]

Great video. It made the underlying concept crystal clear. Thanks a lot, Ritvik.

[@thiagobarreto9056]

Just saw two 20 minutes long videos before this, none made me understand this at all. Then, I saw this 10 minutes long video of yours and it made this subject so much clearer than before. Amazing professor, congratulations!

[@ritvikmath]

Great to hear!

[@EubenM]

You solved a big curiosity I had. I learned about the power of MonteCarlo analysis and how easy it is to get a uniform distribution from Excel, but knew I would always need more specific distributions. So the question was how to get any distribution from a set of randomly generated numbers from the usual Excel Rand() generator. Thanks for the brilliant and easy demonstration! Congrats for your terrific work!

[@ritvikmath]

Great to hear!

[@Bksemsem]

I really want to thank you because your clear explanation helped me get an A in my statistical programming exam. You are a hero.

[@therockbottom2539]

Love how calm you are. I'm shitting myself when I have to explain topics like these to someone.

[@elias043011]

You have brilliantly and simply explained a topic that I have been struggling with for a whole semester. Thank you so much! :)

[@ritvikmath]

Glad it was helpful!

[@roayadiamond]

He is going to be a fabulous professor

[@ritvikmath]

Haha I appreciate the kind words :)

[@MegaNightdude]

Brilliant!!!!

[@ritvikmath]

thanks!

[@shueibsharif9955]

I can't thank you enough. You have been of help in many subjects from time series analysis to this. I would like to see EM algorithm, latent class models, and hidden Markov models in the future.

[@lilmoesk899]

Thanks for the video! I'm still struggling with this, but your explanation definitely helped!

[@ritvikmath]

Thank you!

[@hahahaYL-h3x]

Brilliant teacher! I guess it is a sort of gift.

[@aytekin8669]

thanks for good explanation about Inverse Transform sampling !

[@ritvikmath]

Glad it was helpful!

[@Maikpoint11]

Super helpful, thank you very much!

[@dedecage7465]

This was super pedagogical, thank you very much.

[@phuongdinh3769]

Trying to wrap my head around this in class but to no avail. Thank you so much for your amazing explanation

[@joaopedroxavier8474]

Thanks for the video! I was struggling to understand the motivation behind it, but your explanation has made it much easier for me :)

[@ritvikmath]

Glad it helped!

[@EubenM]

João, veja meu comentário acima para um exemplo de aplicação.

[@ahmetkarakartal9563]

you saved my life

[@mostafaalkady6556]

Great explanation! Thanks.

[@ritvikmath]

Glad you enjoyed it!

[@caiyunwurslin2468]

Thank you. Our instructor did not explain it and just gave the theorem. I was confused like I have three heads.

[@teojunwei2000]

hi, is there an error with the PDF function? f(x) = lambda * exp^(-lambda)(x)? thank you for this video!

[@samersheichessa4331]

Just fantastic ! keep it up man great videos and great explanation

[@emilioalfaro4365]

very clear explanation, thanks for sharing!

[@realimaginary5328]

Excellent. !

[@rishikalodha1236]

Thank you for this

[@adityasaini491]

That subtle pen flip at 5:49.. Damnn

[@EubenM]

LOL

[@adityasaini491]

@@EubenM You replied :DD Great videos man! Your channel is awesome :DD

[@markusnascimento210]

Greatly explained! Thanks!

[@malhajed]

I love your explanation always produce the best please don’t stop

[@liamobrien8610]

Great video! Your exponential density is missing it's normalizing constant, though. Since your CDF is correct, no harm , no foul, but it might confuse some people.

[@algrant33]

Yep, I'm looking for the lambda*e^ -(lambda*x).

[@praburocking2777]

great explanation

[@HadiAhmed7546]

Thanks a lot bro, so helpful

[@learnphysics6455]

Gem level bhau

[@berkayyucel1538]

That was awesome. Thank you !!!!!

[@ritvikmath]

no problem!

[@fredericoamigo]

Excellent explanation! Keep ut the good work!

[@lancelofjohn6995]

Nice lecture!

[@emanelsheikh6344]

Thank you 🙏

[@andyak93]

nice! Thanks for the work. Like the way you explained concepts in a straightforward and smooth way. Please keep it up ! :)

[@stipepavic843]

this guy is epic!!!

[@nishitshukla4139]

Lets say u = 0.25. Then 1 - u = 0.75, right? Could someone explain how 1- u = u in the uniform distribution?

[@awangsuryawan7320]

Up

[@築夢者-w8o]

Actually the magic for this inverse transform to work is the equation P(T(U)

[@farhadbatmanghelich278]

Thanks!

[@maximegrossman2146]

excellent

[@ronborneo1975]

Quite an amazing explanation. Well done!!

[@ramn9071]

Well explained .. thanks. One minor suggestion .. if there is a way you can make the video screen capture friendly or leave a screen capture slides to the video, that would be super helpful. Thanks for the clear presentation.

[@katiedunn7369]

very helpful, thanks for this video!

[@jindai5850]

Yo Ritvik not sure if you still remember me we talked during orientation (I was the guy work with Tasty). We had a class last week about MCMC and I was confused about certain parts and KZbin directed me to this video lol. Great job man keep it up. Hope we can catch up when things get back to normal after the pandemic

[@dwightsablan3571]

Thank you, this helped a ton! :)

[@ritvikmath]

Glad it helped!

[@lm58142]

Thanks for sharing. Just one small comment....pdf of the exponential is lambda*e^(-lambda*x).

[@annabelseah920]

perfect!

[@shubhamthakur3461]

Great Explaination! Thanks so much :)

[@sheeta2726]

Thank you!!!!!!!!!!!

[@fionnmcglacken35]

Brilliant, thank you so much.

[@kevincannon2269]

TLDR: The distribution of the CDF of _any_ PDF is uniform, so if you want to sample from a PDF that has an invertible CDF, you can sample from the uniform distribution and convert it to the desired distribution with the inverse of the CDF.

[@gavinresch1144]

Hey - great video! I think you might have forgotten the lambda in front of the exponential for the exponential PDF. If you calculate the CDF from what you have written you will get a 1/lambda factor.

[@ritvikmath]

Yup you’re definitely right !

[@adishumely]

great video! thanks!

[@aryang5511]

Great video, it really helps me out a lot. One thing I still dont really understand is why we might do this. As in, why would we use the inverse transformation method to find the exponential random variable instead of just using the exponential PDF directly if we have lamda?

[@phalanxz11_]

Can you please do a video about Copulas? For example in a (credit) risk management context

[@Juanlufg]

Thank you for this, it has helped me a lot! :)

[@grjesus9979]

Then, why is important the uniform pdf?. I mean you could sample directly from one distribution to another just by putting the value returned from the CDF of the first pdf as input to the inverse CDF of pdf you want to arrive at. Am I wrong?

[@trollingenstrae2207]

great explanation, thanks a lot!

[@konstantinkulagin]

I probably missed this moment: why transformation to CDF actually gives you desired distribution?

[@ec-wc1sq]

thanks, this is a great video!

[@musondakatongo5478]

Well explained. Thanks a mil

[@TheiPodMac1234]

Very helpful thank you !

[@yelnady]

Thank you man

[@OscarBedford]

What is the role of lambda? I've seen other videos that don't include it, so now I'm curious. Amazing explanation btw!

[@whoami6821]

could you make more advance time series tutorial? really like your videos and i'm struggling in grad level time series course

[@ritvikmath]

More time series vids coming up soon!

[@sebastianmathalikunn]

Hi Ritvik, great videos! would be interested to have a set of videos explaining variational bayes, ELBO etc. in order to perform bayesian optimisation on hyper-parameters

[@rachidwatcher5860]

Thx body u the best

[@tianjoshua4079]

Great video. Quick question: at the end of the video, you said we could swap 1- u for u. That means 1 - u = u, which translates into u = 1/2. Yet u is a random variable, it is not necessarily 1/2, right? What am I missing?

[@ritvikmath]

Good question! We are not swapping 1-u for u in an algebraic sense (in which case you would be absolutely correct). Rather, we note that u is a uniform random variable between 0 and 1. Therefore 1-u is also a uniform random variable between 0 and 1. Thus, it does not matter (in terms of probability) whether we use 1-u or u. And using just u makes the formula look a bit nicer.

[@tianjoshua4079]

@@ritvikmath Oh. I understand. RVs are not really variables. When it comes to RVs, what matters is not the specific value of the RV, yet it is the distribution of the RV that matters. Since u and 1-u are both RVs with the same distribution, they are interchangeable.

[@_anastasia_wagner]

Hi! I loved the video, but I've got a question. What are the cases when the CDF is not invertible? And what are the strategies then? Should we try to make the CDF invertable by interpolating it or should we use another random variate generation technique? Thank you in advance! Happy New Year.

[@ritvikmath]

Happy new year! And great question, indeed this technique is good only if you can find the inverse of the CDF, so if that is not possible, interpolation is a great idea as long as the fit is "good enough"

[@jonatangarcia9285]

You can use the generalized inverse of the function. This is a function g such that g(y) is the infimum of the x such that F_X(x) >= y. Since F_X is a continuous function from the right this is always a minimum. So this function is such that F(g(y)) =y, it works like the inverse and the difference is that if there are other values with the same image you take the least of them and you can always do that. This is the same function to calculate quaintiles, so Q_{0.5} = g(0.5). Take in account that g(0) = -infinity and g(1) = infinity, to get the values right. More information here en.wikipedia.org/wiki/Probability_integral_transform

[@nicnicco]

Are there any resources I can look at to understand why it's valid to assume that p(T(U)

[@kerguule]

I don't get it why the exponential distribution is called memoryless? Yes, I know that that lambda or hazard rate is constant but isn't that just the speed or rate of the probability (not the actual probability because the lambda can be more than 1). From the exponential PDF, you can clearly see that the chances in the early phase are bigger than in the later phases so why is it called memoryless? If I sampled time to failures, should I get more numbers early than later because of that decreasing curve?

[@BlueSkyGoldSun]

In data science can we transform weibull distribution into Gamma or poison distribution?

[@zhoucyrus5797]

there is an error for the pdf of the exponential distribution, the lambda is missing.

[@geoffreyanderson4719]

I have question about the math, on how to derive other inverse transformations especially for datasets that predict number of clicks on a web page for example. Some of them are tricky and might even need estimation by iterative numerical methods or ML, because the Poisson is simple to find the inverse function for. And then how do you put the inverse transform into an sklearn pipeline exactly? Here's why I ask this: Sometimes I am using a Generalized Linear Model which provides a convenient link function already built-in, but we are not always going to just use a linear model as we might need to use for example the large feature vectors that an NLM model is producing to describe some text. GLM is not necessarily the only tool to consider. Besides for random sampling, Transforms are also good for ML preprocessing and postprocessing pipelines to help your model learn easier. The log(Y) and e(Y) are the Poisson distributions transformations when your response Y is a count. Quasipoisson and Negative Binomial are good for count data when the mean and variance are not staying equal as the Poisson requires, but instead are showing some overdispersion or underdispersion. There's also zero inflation model which combines a logistic model and a Poisson model together in sort of an ensemble to help pre-predict the count = 0 case when 0 appears a lot more often than plain old Poisson can account for alone.

[@piyushsinha3344]

in order to find the inverse of CDF, we just find the value of x..why? in other word, how come x is the inverse of CDF?

[@tj9796]

Great video. Could you do one on copulas, building on this one?

[@hp-qx7tf]

beauty

[@chonglizhao2699]

If I understand correctly, the reason why uniform distribution is used because its output range from 0 to 1. Just out of curiosity, can we use beta distribution to replace uniform distribution?

[@kobi981]

Very nice video! thank you! The uniform should be (0,1] without 0 right? so the ln will be defined.

[@PavelSTL]

Was hoping to hear more about motivations for WHY i need to know this method for DS. "that's how computer gives you random samples from a distribution" is not enough to care about it. What about cases where maybe I don't have a pdf or it cannot be integrated or I get only proportionality of pdf (like in Bayesian model) so I can't just plug in the variable into the proportional pdf and get accurate samples..... maybe that's when I need to use this method.....

[@ritvikmath]

I appreciate the feedback!

[@Ivan_e9812]

Not very proficient in statistics, but in sum, if I do the transformation and have the final function, given a number u that is randomly generated from a uniform distribution, I will get an equivalent randomly generated number that falls under an exponential distribution? great video, I will subscribe and continue to watch them!

[@shivamchoudhary1681]

What if CDF is not invertible?

[@scarlettwang2643]

if the distribution we want is not the exponential distribution, are the steps are still the same？

[@martinschulze5399]

its not a ''datascience'' method (which sounds like it comes from modern era). it is known as smirnov method who lived around 1900 and likely known before

[@庭瑞赵]

Having a hard time understanding EWMA and GARCH model ,can you make some videos introducing them?thx

[@ritvikmath]

GARCH is coming up soon!

[@David-ku9li]

Cumulative density??? You mean Cumulative Distribution?

[@unnikrishnanadoor]

I have a question if we graph the inverse function of that exponential function how it will looks like? whether it looks similar to graph of uniform distribution? otherwise how this can be equal?

[@Yohan87845]

I think the graph is pdf, not cdf,

[@7ignatios]

Can you do (or recommend) a video on Granger Causality?

[@ritvikmath]

Thanks for the suggestion! I'll look into it

[@hashbrowncookie8444]

So if I had some other distributions apart from exponential one, I just need to derive its inverse, and set the number of simulations I will like to do with a U that is unif from 0 to 1? I just need clarification in that part.

[@athantas]

what if the function is not invertible? any way to deal with that?

[@kocur4d]

no, this method works only with invertible functions. You need other sampling methods for those. like MCMC or variants.

[@deepanshu7714]

sir hindi me bhi kux baniye sir hindi me bhi kux baniye

[@deepanshu7714]

sir hindi me bhi kux baniye

[@RagaveshDhandapani]

Thanks a lot. Can u make a video on generalised normal distribution and inverse to uniform. Please

[@vshssvs7]

Nice explanation, the trick seems to be that p(u