MIN-LESSON 8b: (Power Laws) Q& A

[@tommyhegarty97]

That last train of thought on nature, power laws, and domain constraints was a gem

[@Achrononmaster]

But was wrong. Nature is fundamentally constrained (energy conservation in engineering, and GDP constraint on profit, being good examples). Only mathematicians would think like Taleb. Any engineer or physicist would take the reverse attitude: *_nothing_* is power law or gaussian (infinite support) until you can prove otherwise. Engineers and physicists use infinite support distribution only for convenience as idealizations, not because they are thinking nature is that way absent constraints. If you in fact remove constraints physics calculations blow-up, practically every wave-function computed from a least action principle, or anything computed from an integration by parts blows up at the boundaries without constraints. Lack of constraint is basically _death._ So the correct way to put things is "Why might there be no constraints?" the burden is on mathematicians to justify why not, not on the engineers and physicists to justify why there are constraints.

[@johnnycochicken]

@@Achrononmaster he specifically mentioned GDP as a constraint. he doesn't mean the power-law-like behavior empirically observed goes literally to infinity, rather he is implying that it occurs over a wide range (as opposed to other matters like human height where constraints are tighter and cause thin tails)

[@sillyfarmerbilly8872]

Thanks so much for taking the time. I can only read so much in a day, it's so great to just sit back and listen to something nice like this and still be sharpening my mind.

[@guitarmaniaxx]

The notion that everything is power law distributed and that only some processes are not, due to constraints is such an eye opener. It makes the world almost easy. Even though I know there are no cookie cutter rules, but I can now look at a process and assume that it's power law, then I can check if there are constraints instead of saying something is thin tailed until a large sigma event occurs and convinces me otherwise. Thank you.

[@sillyfarmerbilly8872]

I'm writing this down.

[@Achrononmaster]

For any engineer (or physicist) your mindset has to be reversed: *_nothing_* is power law or gaussian because _everything in nature is constrained._ Power laws and infinite support distributions (gaussians) are the mathematical idealizations, not the real world. If either NT or Pinker understood the constraints on modern warfare (one being mutual annihilation threat) you'd see the distribution of symmetric war is no longer a power law, it is highly constrained. The extreme case being we can in fact suppose we get a complete annihilation event, in which case P(war)=0 afterwards (no humans around). Small asymmetric wars post twentieth century still have fat tails, because they are not so constrained. But they could be constrained by (for example) UN collective security agreements (which would mean abolishing the Security Council veto power, so states like the USA and China ceding some sovereignty). It is only because the UN diplomats are lap dogs and bullsh*tters and puppets to the _Cult of Intelligence_ (Marchetti, 1974) that we do not have such arrangements.

[@reynoldtimotius7109]

I've missed that opening "friends" since you haven't posted for a while

[@Achrononmaster]

@7:50 the other way to show student a mean can be non-finite is with the balancing (centre of mass) equivalence. If a 1-D distribution has a finite mean you can balance the thing on your finger as a pivot somewhere. There are trumpet shapes that go infinitely thin but with diverging mass, so they have no point of balance at finite _x._ When a student says, "but nothing in nature ever goes to actual infinity" then you can pull out the log transform card.

[@daniels.8723]

A thousand thanks, NNT. Your explanations highly influence how I do empirical research 🙏. Ps - I'm an epi/health researcher, and we're not all believers in pandemic modeling. الف شكر لك.

[@eriprolan4049]

From Russia with love, mr. Taleb! ) Thank you for your books.

[@nntalebproba]

My pleasure!

[@Mrk3b]

imagine being a genius like him. such a goat

[@PiotrKolmanowski]

Finally! A piece of clear information on when we can expect power laws and when not. Thank you, thank you NNT :) BTW also thanks for a precise argument against Pinker's optimism.

[@nntalebproba]

Glad it was helpful!

[@Achrononmaster]

Pinker's "optimism" is not wrong. The justification for it is. A weapon technology has advanced it has become impossible for states to seriously "go to war" on weapon parity, because the result would be mutual annihilation. Asymmetric war has no such constraint and can persist with fat tailed distribution until human civilization learns to place constraints on military power with, for example, collective security. The mutual annihilation constraint never was a possibility in the past, so post Twentieth Century --- but _only_ post-20th century --- war distributions have changed. They're longer power law due to these constraints!!! If you do not take full heed of what Taleb says you'll likely be wrong. Do not take a lesson by halves.

[@SonaliSingh7107]

As a non-math/science-background person seeking to understand, thank you. Loved the example of the storefront.

[@nntalebproba]

Glad it was helpful!

[@nishantjoshi5174]

Thank you, Monsieur Taleb for this. Expecting more on lessons in probability in the coming days.

[@yarongoldstein2372]

Question about how to apply awareness of Power Laws into real world tasks in Data Science: In all type of internet companies, it’s popular to pick the “best” out of different release options via an A/B experiment: A randomly picked # users get a Blue button, an independently picked # users a Red button. You “start” the experiment, you wait some time, and then you measure which “experiment arm” has “more success”. If its just about button clicks, and all users can click a button only 1 time, the “usual” approach of simply comparing total number of clicks in both arms and declaring the larger value to be “the better arm” might make sense, because we have the constraint that each user only gets 1 vote on saying “this works for me” or “no, not interested”. In the real world though, we often have Power Laws that come in place: If most users watch only 2-3 KZbin videos each day, their impact on the total outcome of their arm is significantly smaller than a user who watches 1000 videos a day, and could theoretically click 1000 play buttons or subscribe buttons or whatever your interested in. Given that these “big whale users” are rare (distributed by power laws), there’s quite a good chance that the originally randomized selection of users into arms A vs B is actually quite unfair to compare from the get go, given that arm A might have a super rare power user that doesn’t have an equivalent peer in arm B. Is there any way to use knowledge about power laws to correct for this possibility in the experiment setup? Does this only work for when we can identify the “big whales” already before the experiment starts? Should we “fix” the issue by changing the success metric from something that follows a power law (“# clicks”) to something that is constraints (“# users who clicked at least once”)? How should one plan the experiment size (# users per arm) if we already know that the outcome will be powerlaw distributed, how can we meaningfully get confidence intervals on the observations? (Does using Jackknife or similar fix the issue?) Dear Mr Taleb, thank you for opening my eyes for the often ignored impact on Power Laws in the business world. I would highly appreciate if you could point or guide me around the problem outlined above, that follows me through my day to day job, and reading your work and watching your videos makes me more thoughtful each day

[@klausgrobysfinancechannel5464]

Dear Nassim, I just would like to Thank You for what you do here! I'm a big fan of yours and nowadays I use your book "The Black Swan" in association with your guest lecture at Cambridge University in my lectures for doctoral students. They need to learn Power Laws and the statistical properties. I might borrow some of the content here too. Thank you!

[@stoic-999]

No one teaches maths in a more simpler way than you Maestro. What would be your suggestion for someone to watch or read for understanding the basics of statistics and probability as there is a yuuuge demand from your followers (non math background) regarding this. Thanks!

[@nishantjoshi5174]

Absolutely, Rahul. Maestro aces in simplifying and explaining that which others cannot.

[@nishantjoshi5174]

@Dan Campbell Yes, know them well, they are very advanced, suitable for graduate course.

[@WillemvanLier]

Answer from NNT's twitter: "To the freq. question, "which book to learn probability?" DO NOT start w/books. Do zillions of Monte Carlo, play & play until you get it."

[@sillyfarmerbilly8872]

@@WillemvanLier so which book teaches you how to do Monte Carlo simulations?

[@mehdiAbderezai]

Thanks for simplifying complex ideas and making them available. Question, do folks ever combine a memory less model with a model that retains memory, and maybe scale the memory portion down with the time?

[@programahepintersg3745]

When I bought that book of Pinker I had an intuition that something was not right.

[@MZKR94]

One of the best videos so fat

[@_N0_0ne]

Thank you kindly ✍️

[@adlos6168]

Thank u Nassim

[@2mohammad]

Taleb, the kind of guy to know all the math in world and still use the free version of AOL.

[@myyoutubechannel2858]

Maestro, and readers of this comment: 50 million dying due to war events every 80 years seems very high to me. In my current understanding, only 2 such war events have ever happened (WW1 & WW2). I would like to be corrected. Please tell me more about such data. The store-front power-law example (small, possibly random advantages lead to larger, structural advantages) was simply fantastic.

[@Jebcbeb]

I'm assuming he adjusts for global population size. The chance of 50 million people dying in a war in 30,000 BC is 0% even though each warrior was more likely to die and more of the population were warriors

[@myyoutubechannel2858]

@@Jebcbeb OK, that makes sense, the statistic is the equivalent number killed as a proportion of the overall population.

[@Clyde.artwork]

My man, love you. But please, get a nice camera and good mic.

[@Jack-lg9mq]

How can you figure out the average time taken for a war to kill > 50 million people? Surely it depends on the window you're using? If you calculated that over the last million years then it would only be very recently that any such wars would kill so many?

[@Senecamarcus]

Professor, using old cam just so Greenspan and others wont watch his videos!

[@reynoldtimotius7109]

Ooo that makes sense. Maybe he recorded this with old cam purposely to deter certain people from watching it. Looking back, other maestro videos, there are some that is recorded with good cam and some with old cam

[@JohnSmith-rr3qn]

Could you think of evolution through natural selection in terms of power laws? That is, the mutations and/or changes in environment that create new species which could result in the phenomena described at the end of lecture

[@ManuSolaArjona]

Thanks

[@jeanlewith5112]

What's alpha (war, pandemics) referring to here? Since it's not sounding like it has anything to do with a test statistic, which is all I'm (slightly) familiar with

[@bogdanlevi]

Distributions with a power law right tail are defined as P(X>x) = L(x)*x^{-alpha} for all x large enough, where L is a slowly varying function. So alpha is just a parameter of a power law tail. If I recall correctly, mr. Taleb says that number of victims of a war or a pandemic has a distribution with a power law tail.

[@jeanlewith5112]

@@bogdanlevi Awesome, thank you very much, especially for the formula inclusion!

[@Sadjina]

Choosing assets/portfolios with high upsides and lower downsides: does it make sense to select for the ones with high alpha on the downsides and low alpha on the upsides? Or am I completely off track here?

[@reynoldtimotius7109]

Maestro, is it just me or your camera is a bit blurry? Still, thank you for the lesson and you look great

[@Senecamarcus]

For love of god lets get him a camera

[@GuruPrasad-qu4vg]

@@Senecamarcus he is a millionaire,he can buy one himself fella

[@Senecamarcus]

@@GuruPrasad-qu4vg it was a fun comment for professor Taleb. We all know he is next level and worth millions.

[@GuruPrasad-qu4vg]

@@Senecamarcus Ja

[@Senecamarcus]

4 people that down voted this are: 1. Mr. Pinker 2. Mrs. Pinker 3. Fan of Pinker 4. Fan of Pinker

[@sillyfarmerbilly8872]

Who is pinker, anyway?

[@gordonf4723]

"The world is naturally power-lawed, except when you have constraints" So this means that eventually any constraint we know of can potentially be removed/eliminated?

[@elieobeid77]

no, sometimes physics play a role, physics is a constraint that cant be removed. After all we live in a world with limited resources and bound by physical constraints. The only way to eliminate constraints is to have unlimited constraints which is impossible. The perfect example is the stars or the blackholes. They can become huge until they collapse. The size of animals is bound by many things, including food, and you can't just create food. If anything resources are decreasing, not increasing, huge animals like dinosors went instinct.

[@elieobeid77]

I'm not sure if your hypothesis is theoretically correct at least, but in reality constraints can't be removed.

[@gordonf4723]

@@elieobeid77 I see, thank you

[@Boonton2010]

I wonder if making an event 'equilivant' in terms of its killing makes sense. Is killing 8M people just like killing 1M if it happened at a point in history when the earth had 1/8th the population? Was Cain killing Able just like killing 2 or 3 billion people today because back then there was only like 4 or so people on earth (taking it literally of course)? On the other hand killing 8M is easier if you have a population of 500M and 1% are put in the army to fight one on one Lord of the Rings style than if you only had a population of 50M....

[@jarenmoorman8168]

Can someone direct me to information in order to better understand how lower Alpha is representative of a thicker tailed dist.? {My naive understanding is that the thicker tailed dist would have a larger Alpha; ie-area under the tails}

[@bogdanlevi]

Assume X belongs to a power law distribution with an alpha of, say, 4, and Y to a distribution with an alpha of 2. Then lim_{x->infty}P(X>x)/P(Y>x)=lim_{x->infty}x^-4/x^-2=lim_{x->infty}x^-2=0. Which means, for large enough x, the probability of X exceeding x is smaller than the probability of Y exceeding x. In other words, x^4 grows faster then x^2, and thus x^-4 decreases faster then x^-2. The lower the alpha, the slower a function decreases and the thicker its tail.

[@jarenmoorman8168]

Bogdan Levinskiy: That was extremely helpful. I not only appreciate you sharing that information, but I also appreciate the quality of it and your time. It makes a lot more sense to me now. Thanks again, and hope you continue to enjoy life

[@bmk4851]

The storefront idea was really cool. You blocked me on twitter by the way. These videos & your books are prob more productive anyway, get more from them

[@johannebertinus5753]

Journalist Stephen Pinker...... = George Will

[@vijayanpp186]

Full glair

[@roverinosnarkman7240]

Friend, when you said the world is power-law (implying non-Gaussian) I wonder if you mean that the measures you find interesting are non-Gaussian? Food production, body weight and myriad other variables are all Gaussian, right? Violence, War, Pandemics, Starvation, and almost all other hell-scape scenarios are Power-Law. You really made me reconsider all of my assumptions about the world with these lectures. Thank you! Now I can state with certainty that we do know something of the future: That it isn’t predictable, at least not from any data from the past. Once we get our TimeMachine(™) working, watch out!