BBC - The Code - The Wisdom of the Crowd

[@ashmann3783]

I would be fascinated to see how differently the collective accuracy of guesses changes when the people witness every other person's guess, and when each person guesses separately and secretly. To see if social factors play any impact on guesses.

[@jfriesne]

When asked, I always estimate that there are 10 trillion jelly beans in the jar. No crowd with me in it is ever going to be wise... ;)

[@sasha-2574]

maybe he should've mentioned that you shouldn't ask the devil

[@Marnige]

Naah, any statistician would've ruled you out as an anomaly.

[@jetcaspian2882]

Lol that defeats the point.

[@kennarajora6532]

I think it's closer to negative ten trillion jelly beans.

[@alejorabirog1679]

Outliers are discriminated in statiatic analysis.

[@Khazar01]

This is why we should always stick together and let nothing divide us and let nothing come between ourselves......we simply are smarter when we're together

[@HackingDutchman]

In a group it’s possible for nobody to be correct but for everybody to be right.

[@Wyzzkyd]

People think the Asian girl who gave 50,000 was way off. When in fact she had estimated how wrong the 159 people were and gave a number that would lead to the correct answer. This is the power, the power of Asian.

[@betanixd5083]

Das is racist

[@capelandpermaculture5808]

Saw this yesterday and it became one of my favourite videos - instantly.

[@Ciruchan]

Anyone came here from Vsauce?

[@dillondasilva6223]

i did

[@EirikAnd99]

I did

[@kennarajora6532]

I want to do an experiment. I want to see what percentage of people came here from Vsauce. Since there's no way of really figuring it out. I'm going to try and see what Wisdom Of the Crowds has to say about this whole thing. I'll write the percentage I think came from Vsauce at the bottom, before you check my answer, have an answer in your head to make sure mine doesn't influence yours. - - - - - - - - - - - - - - - - - - - - - - I think it's around 70% of all the viewers, that came from Vsauce.

[@mttlsa686]

Guys this is really mind blowing. I've studied statistics at school when i was young but i've never seen this with the curiosity and awareness that i've today at almost 40. Do you realize how this simple experiment opens up to interesting discussions about reality, consciousness, collective consciousness, and many other existential mysteries? This is a very underrated topic that should be taken more into consideration for its importance, imho.

[@FiatLux47]

Engineers use this all the time. If you're trying to measure something that is extremely difficult to measure (noisy data for instance), considering only one single measurement would be very dangerous, since it might be extremely imprecise. However, if you measure 1000 times and take the average, the errors cancel out and you have a pretty accurate measurement :)

[@mttlsa686]

Take the??? I wanna know!!! 🤣

[@michaelwang1730]

@@mttlsa686 average?

[@mttlsa686]

@@michaelwang1730 the words after "take the" were not showed when i clicked on "see more". Today yes, i don't know why...

[@CascadianConsiderations]

I'm pretty sure I've read something that tested this over and over, and it kept coming up right. It's like the hive mind of a beehive.

[@ArturoStojanoff]

That guy who guessed there was half a bean... we all hate that guy...

[@fauxjebus]

There are many more examples of this phenomenon in James Surowiecki's book 'The Wisdom of the Crowd'. Definitely worth checking out if this video interested you.

[@obsideonyx7604]

Am I the only one creeped out by the wisdom of crowds. It's too OP, God, please nerf.

[@benjaminparker5044]

Lol

[@crimsonglory7823]

No you're not. I mean, I understand the mathematics behind it but it's still startling.

[@redfish337]

There is an absolute minimum, but no absolute maximum. Taking the average cannot provide a reliable answer because no one can guess -75000 to cancel out ridiculously high guesses. In such a situation, anyone that guesses over twice the real answer is doing more than one person's worth of damage to the mean.

[@JakeLikesTech]

That dude's face when he saw how close they were was amazing lol

[@KevinLarsson42]

Who is here because of Vsauce?

[@bigpips3051]

I am blown away you ask this.. I have to ask why you ask that! Indeed, I was watching VSAUCE about 15 minutes ago, but it didn't directly bring me here. I don't even remember which video I was watching, but it reminded me of a video I saw a long time ago about jellybeans and Google data and I couldn't find it.

[@justindoan4258]

+bigpips3051 ...there's a video where this video was linked/mentioned/used as an example. Funny coincedence though

[@jaduyare]

Which Vsauce video?

[@jayasuriyas2604]

Can you tell which Vsauce video?, I wanna see it.

[@flafichi]

Me

[@theblackboyjoe]

If I knew this in primary school I would have won many jelly beans.

[@wizertechnologies]

We have harnessed the power of the crowd into social technology for decision making. So fun to connect people to the right crowd

[@ItsSansom]

"Somebody thought there was half a bean in there" LOL

[@epistemocrat]

this is absolutely amazing. Markets do this after all.. I wonder why no-one actually exploit this to make decisions. We talk so much of AI but maybe crowds are actually the cleverest thing could ever exist.

[@underdog_factor]

It's not clever if it's manipulative

[@hasangarmarudi2178]

The security agencies actually do use some of these tricks to predict future.

[@DoctorWhy777]

Is this group thought or a group influenced. If the woman at 2:00 had stuck with her original 80,000 instead of reducing it to 50,000 then the math would have worked out differently. When she submitted to doubt she reassessed her guess to 50,000 (its even more interesting why she did? It is a trivial question, maybe it is human nature to second guess your self.) So the math 722.383.5 / 160 = 4,514.896875 (eureka!) However if she had stuck with her original 80,000 guess it would have been 752,383.5 / 160 = 4,702.396875 (not so eureka) So influence must play a part in this, the interesting thing is why did she choose to reduce the number instead of increasing.. for example if she said 100,000 then it would have been 772,383.5 / 160 = 4,826.396875 (not eureka!) ---- Is this group though or individual influence over the group that creates this mathematical picture?

[@mrgomelonsolaris]

I wonder what the median was (really cutting off the outliers).

[@Threedog1963]

Well, even if she said, 100,000, averaging it to 4,826 isn't that bad for such an awful guess. I would think you'd need a larger group guessing to offset stupid guesses.

[@The_Big_Dawg]

Oh VSauce, we thank thee for this link of wisdom!! :)

[@oglommi]

This is amazing, there is no other word for it.

[@TheAnachronist]

The right way to do this scientifically would be to have the guy running the experiment have no idea how many beans were in the jar, in fact not even see the jar himself only asking others to look (after the end, of course, they can look). Also, the contest needs to be announced ahead of time otherwise you run the risk of only contests which produce the shocking result being revealed publicly. Reminds me of XKCD #882, "Significant" ("Found a link between green jelly beans and acne, p>.05").

[@iFilmR]

With the one guess of 30,000, if you subtract that from the total 722,383.5, and put a more "educated" or similar guess to some of the others, with 3,000 and average it accounting for the 160 people. The final average comes out to be 4346.2. Which is a few hundred different to the actual number of beans compared to the 4 bean difference with the 30,000 guess. So although it is still accurate, that 30,000 guess was actually pretty lucky.

[@mrwho995]

The concept of Wisdom of the crowds IS repeatable and it is a rather thoroughly grounded phenomenon. Do the experiment for yourself if you want (as long as you get an appropriate sample size). Or you can just look up the effect if you want. Of course it's not going to always be reliable but the effect is well documented.

[@Son0fHobs]

To tackle this, a Fermi Estimate is the best way to go. The crowd gave a kind of fermi estimate in a way. (google: fermi estimate less wrong)

[@carykh]

What if someone guesses a googol as a joke? Even if there are a million guesses, that one guess will push up the average 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 beans, which is pretty far form the real answer. You could take the logarithms of the guesses, but then someone might guess a googolplex.

[@badkat5536]

carykh this video is probably only taking serious guesses into effect here

[@failstreet1556]

It’s easy to recognize and sort out outliers

[@JenkoTV]

I think leaving out the 80,000 or 50,000 would make sense because you would have to have some kind of filter. For example in crowd sourcing you would try to avoid people with significant mental disability. And without counting normal people would know 80,000 is was way off.

[@brettblyth1857]

The accuracy of the group is far greater than the individual. wisdom of the crowd = direct democracy

[@TheTaltiko]

they should do this multiple times. To proove that it's not just luck

[@Nguyening_music]

I think he meant the .5 bean one, because it was added up to something and .5 bean, which is half a bean. only after it was divided by 160 that it was caused by fraction. The .5 was from someone guessing half a bean.

[@FiatLux47]

I'm not sure I understand your question, but what you might be referring to is what engineers call noise with non-zero mean. That means that when averaging all measurements, the noise cancels out but with a biais. For instance if humans had a tendency to underestimate things, the average number in the experience in the video would be much lower. But apparently, at least when it comes to beans, humans seem to be pretty unbiaised estimators and thus the "noise" they produce has a mean of zero :)

[@pollumG]

Now you know why corporations are collect our data...they can pretty much predict the future with the amount of data these days.

[@HugabearNo7]

That is so extraordinary.

[@duck-fil-a3606]

I used this principle today to guess the weight of a pumpkin at a work event. The person to guess the weight correctly or guess closest to the actual weight won a $10 gift card. Pretty cool. There were only 17 guesses when i averaged and put my guess in. Even crazier is my guess was right on the money.

[@chrisg3030]

Ten years on and it's still the best KZbin vid on this fascinating subject. Thanks Marcus. Here's a thought experiment that occurred to me. Get a couple of dozen ordinary people to each shoot an arrow or gun or whatever as accurately as they can at a small spot of light projected on a barn door or similar. Afterwards switch the light off, and deduce where the target was just from the grouping of punctures. I'm pretty sure you'd be spot on just by finding their centre. Would this mean the crowd is collectively a better shot than the individual?

[@finthefish2525]

This is an experiment I’ve wanted to try for a while, or at least something similar. I didn’t think of using a light, but good idea.

[@15cocopuffs]

im actually kind of surprised there is no response trying to duplicate this and se if it came true for them too, i myself want to try it now after seeing this

[@awsomenesscaleb]

No. We call that a normal distribution. Had he analyzed all of the guesses, he would have probably found out that guesses close to the true amount were the most frequent.

[@TheAnachronist]

Which shows that it's pretty likely there's some bias in there. If the average is sensitive to one answer, if you get very close to the right answer, you're lucky or you're cheating, even if subtly and unconsciously (like second-guessing the original 80,000 guess that she gave). If he hadn't biased the girl's original response of 80,000 down to 50,000, the answer would've been ~4702, which is off significantly. This subtle guiding of people's answers when you know the right answer is cheating.

[@spacedew]

The same method is used at statistics when we take an average and use it and the result is very accurate.

[@billevans9407]

It's interesting how close the average was to the "real" answer ("real" because the experimenter could have made a small counting error either way also). However, I would also like to see what the standard deviation (StDev) was. If the average was less than a StDev or so from the answer, we could say they were "right on." How wide was that range? I mean, an Avg of 4515 with a StDev of 20 is much more certain than an Avg of 4515 with a StDev of 200. And yes, I came here from VSauce. :)

[@allenshih1]

What if they do the geometric mean?

[@kennarajora6532]

I was thinking the same.

[@tradejournal]

if she said 80000, the final average would be 4701, quite away from 4510

[@kennarajora6532]

still though. I mean obviously being 'just a few beans off' was nothing more than BBC drama and bad experimentation. But 200 beans off isn't all that bad, all things considered. Off by less than 5% of its original value.

[@BassDownLow513]

Seems interesting.. however, the girl that was hesitant in her number choice coincidentally allowed the experiment to be a success. If she had chosen one of the other numbers that she guessed first, the final number would be off from 4510.

[@daxsama]

This blew my mind when I watched this at school :)

[@mattlm64]

It kind of proves that out brains are capable of making accurate guesses, only with large errors on top of the accurate guesses which go equally either way.

[@Dazza3500]

indeed the interquartile range would be interesting to see in this case

[@oglommi]

@timb6 I've seen all of them. The planets with liffe thingy would'nt work because everybody would just say random numbers. Wich is not the same as this video.

[@Threedog1963]

So, would the answer become more accurate with a larger number of guesses? Or worse with smaller group guessing? I go with more accuracy with more people. Thoughts?

[@kennarajora6532]

I think that the number of people does make a difference. If we only take two guesses, for example, 400 and 50,000 then the average is 25200 (I know those guesses are outliers but it makes the point). If we graph the accuracy with the number of participants, I'd expect an s curve, but that's just a guess as well.

[@Davidvp]

This is not about "crowd intelligence" is purely about measures. We could say the same thing about laser meters or voltimeters. Any measure comes with an error, in these cases it's what we call a random error. When we say to a inaccurate tool to measure something it will give us a value. An error will be present in that measure. Measure it again with a 1000 more tools, calculate the average and obiously we'll have a more accurate result with an error that we can equal to the standard deviation.

[@milanpaunovic3416]

This is why we live in an amazing time. Now with the power of internet, all people around the world can participate in thought exchange and apply The Wisdom of the Crowd on many other subjects making the humanity prosper faster than ever!

[@blackout2240]

Go for it. Try it yourself with something you know and ask the crowd. Then try it with something you dont know.

[@mrjkamm123]

this video is about to blow up

[@timb6]

@oglommi i reccomend watching all 3 parts (1 hours each) from The Code. Can find it on BBC website. It's all pretty amazing! and i was wondering the last few days, why don't we guess alot of things in this way? Like howmany planets with life you think there are in the universe? Ask 1 million people, take the average, and we will know it :)

[@ultort]

I wish I can have a glance at the distribution, he could have use a spreadsheet instead of a paper and a phone...

[@TheAssholeish]

ultort are you karrom rabiah?

[@Asdam14]

The only thing I don't like about this is that the higher guesses have much more bias than the small answers. I wish the original data could be found so an 'exponential average' could be found...

[@OldSchool9690]

This just shows what we can get accomplished when we think collectively.

[@zetadroid]

4:10 aka the central limit theorem

[@toastsandwich]

does this mean that the direct democracy is the best system?

[@JoshuaStowers]

I was thinking about this myself recently, and I feel that it shouldn't apply in politics. You're not predicting the outcome of something, or the size/number of something, you're determining the best policies. When voting on policies or politicians you're trying to decide what's fair and good. The problem there is that we don't all know what's best for us. We say that war in the middle east is bad, but if we hadn't gone in the entire world's economy may have collapsed. We want Government to get us out of debt, but doing that could could wreck the value of the Euro and bankrupt entire countries in Europe. These are just the examples which I can think of as a student of Economics, I'm sure that other fields' have more to add to this. The point is, though, that a purely democratic system may not have come to these realizations, and the world would be in trouble. That's my thought on the matter, but I'm curious as to what others might have to say.

[@weirjwerijrweurhuewhr588]

Joshua Stowers Actually there's an interesting study where children get asked 'who would you chose to be the captain of your ship?' purely based on photographs on presidential candidates. The outcomes of those are highly correlated with the real outcome of the elections.

[@doublegi2880]

Fellow economist, it's been 2 years since you was a student, you stil think that? Cause when I was done with my college I was in full favour for direct democracy. "The problem there is that we don't all know what's best for us." Yeah but on average you take a big population and I think they'll be a lot closer to figuring out what's best for us, then a few politicans. Besides, you can't have lobbies pushing agenda on whole population. The error gets smaller with bigger population. Anyway, at the end of the day, I think it comes down to calculation power. And if you have brains of all nation working as a single unit, they'll outperform and on average take better decisions, then a few people. I don't really think intelligence has a lot to do with it, but it helps in the process. There are single cell organisms that can plan road network of Japan more efficiently then what high IQ Japanese engineers did. And the freaking thing has IQ - 0. The thing was basicly they made a small map of japan and put the thing in the middle and instead of making small cities they've put the pieces of food where cities should be and it connected between them more efficiently then the real road network was. There are actually bunch of examples from natural world and how it organises itself to be efficient. So with computing power of all brains combined, power of evolution and with some other help from nature, this could work way better then expected. Just look at what happens when 4chan uses hive mind and what they manage to achieve. That hive mind can process huge amount of data and solve some really crazy fucking problems. You don't really need to have every individual to know what's best for him or the group, as long as the group knows what's best for it and what group will do is what evolution forces it to do: try to evolve and survive and be as efficient as possible while at it. No one was counting marbles here either, no doctorate from marbles would help, no one knew what the number was, no inteligence helped, but they got really close to what reality is anyway. People were wrong and they didn't know what best guess/decision is, but the group wasn't :)

[@aseredy]

This maths is a better example of a republic. Not a direct democracy. Where someone (a representative) "samples" the crowd for an answer, then averages their answers. An example of direct democracy in this case would be to ask the crowd for an answer, then use the guess the most people replied with. Its a subtle difference.. Even the Demos need a check on their power. Socrates found this out the hard way.

[@gentbar7296]

not really cause politics are very predictable and the power never changes hands. in a field of no guesswork or competition things are very linear. so could economics, those are deliberately thrown into a speculation rodeo shown, but that's not what is happening behind the scenes.

[@TheNearFantastica]

Mindblowing.

[@DerTonpilz]

I had an eerie thought while watching Vsauce and this... What would happen if a study asked hundreds, thousands or even millions of people to give a random real number each? Would the average of those numbers be similar with every sizeable set of results? Would it lead us to THE number?

[@Draugh39]

The answer is 42. It was established long ago ..... 🙂

[@timb6]

@oglommi True. Thought about that later as wel... I want it to work though! There would be no unanswered problems around anymore!

[@bramsnijders]

You forgot that the average before the 50000 was added could be a lot lower than 4515. ((4515 * 160) - 50000) / 159 = 4229 That would be the average before he added the number of the lady. This still means that if the number of the lady wasn't added, the average would be way more off than 5 jellybeans (4510 - 4229 = 281 jellybeans), so it seems like they got lucky this time, or did a bit of TV magic like you suggested.

[@magmasceptre]

you should draw a normal distribution curve of the whole thing

[@MASsurvival]

I'll show it to my math class!

[@Cohonees]

... and this is why we put our heads together!

[@ChadWorthman]

If you exclude the guess of 50,000 and average the remainder out of 159, the answer is much further off than if her guess was actually as high as 80,000. It made the average guess 4228 rather than 4514.

[@kennarajora6532]

I'd be interested to find out the standard deviation actually.

[@MrTranceNinja]

Actually, it would've been 4229 - assuming her guess is removed entirely, the total is then divided by 159 rather than 160. But the point still stands. :P

[@Invalid571]

Please explain why there is an equal chance of over/underestimating.I only said 10^100 as an example of an estimation tha would disturb your average in a most apparent manner.Consider the possibility of the majority of people answering like the gal who estimated 80000 only to withdraw it and say 50000.So in a sample of a 1000 people if 700 of them estimated something like 80000 to 50000 your average would be way of the real number.

[@MudballDon]

I just used this to win a bag of candy at a baby shower.

[@mrgyani]

OMG, the average of my two guesses was bang on. I first guessed 3000, then 6000.

[@XBassSlayerX]

"for example if someone gave as an estimation 10^100" they aren't going to. this wisdom of the crowd thing is basing itself on people trying to guess the number, not fuck the system up. and therefore even barely logical guesses (10 times higher than the number) could be adjusted for. " underestimated the number then the average would be smaller,if they overestimated it would be larger" There is an equal chance to over/underestimate, so large groups will have similar values.

[@JawsofFreedom]

Actually I think it's the use of "hear" rather than "here" that actually matters.

[@larbremord]

I've guessed between 4 and 5 thousand! That makes 1 people more to rock! =D

[@juliocamacho8354]

There is a reason why airplanes are not flown by passengers inputs.

[@Kahantheflash]

Yes, but its those extremely high and extremely low guesses that cancel out each other. If you take out the lowest guess as well as the highest, it would average out to be very close to 5,010 again.

[@ConceptOfJustice]

Can we guess the laws of physics this way?

[@Yuri-bl4ec]

I think so. That's why science is a community and theories are approved by committees. But your question carries a very high degree of complexity, because laws of physics are highly abstract ideas, very different from a single number as shown on this video.

[@devun1999]

THX Vsauce for letting me see this amazing video... It is so complicated but at the same time so easy. At first I thought quantum mechanics was at work when they were talking about the "code" but after wards I realised it's just crazy math!

[@Re_Kitty]

Someone needs to do a larger, online and photo-based test of this.

[@10outofTenley]

I would not calculate this on an iPhone calculator. Those calculators are SO easy to miss-type on.

[@eliteproboy]

Anyone come from Vsauce?

[@آَّ-ض8ك]

ye

[@fev4]

And without this effect in what matters have this method (the guessing of a human) been used? Besides counting or organizing something.

[@mungojack]

vsauce

[@fishwishpishpish9165]

vsauce again

[@fishwishpishpish9165]

new vid

[@second_sunset]

In this case, it probably isn't actually the same number of people overestimating than underestimating. The guess of 50,000 is over 45,000 off the mark, and the lowest, 400, was only just over 5,000 off the true value. Therefore, chances are that more people underestimated than overestimated the number of beans.

[@stfnclmr]

If the guy influenced the answers of other respondents as he did with the answer of the Asian woman who first came up with a way higher number but changed it when faced with a very skeptical verbal response..well, we cannot trust this particular result at all. Seems fabricated for the sake of the show.

[@Kochos]

how can we use this principle in investing?

[@PelaoyFamoso]

If someone were to say something like that, you wouldn,t take that guess into consideration.

[@AGfrom83]

That deep intro.

[@nn101198]

I find this very hard to believe. Surely, given that pessimistic guesses can only range from 4509 to 1 and the optimistic guesses can be anything from 4511 to infinity the chance of them cancelling out is rather unlikely. I mean, look at the person who guessed 80000, to balance that out ~18 people would've had to guess a very low value to cancel that out.

[@CrystalClaritySan]

It's like Who Wants to be a Millionaire. I always thought "ask the audience" was the best lifeline.

[@CrazyKnows]

The same guy that counted the jelly beans individually as he put them in the jar.

[@SHNUCAU]

Singing Banana did this too. I think his explanation was a bit more thorough.

[@requisitemaxim]

Android ads. u r making it incredibly good.

[@1934853]

Still, the 50,000 also balances out ridiculously low answers, and even at 4228 beans, it still at a 93.75% near the answer; nonetheless, it's still amazing how this is even in the 90%+ accuracy mark.

[@ThisNameIsBanned]

Yea, a too big error is nearly impossible to fix for the crowd. Just see someone with 50.000. So it means its also totally "random", as you can easily get a small crowd in which everyone guesses way too less, and you can't fix the error by someone with a extrem high number. It works for something like the beans, but it doesnt for a ton of other things, especially if it does not have to do with guessing a more or less "random" number.

[@Xanketh]

Awesome, now I'm gonna do this to find the amount of candy in the jar at festivals :)

[@hugoooobr]

Why exclude it? What matters is the final opinion of everybody, the number that come to our head in the first second that we're trying to guess doesn't matter at all

[@KyleJack101]

The proof is simple: The correct answer is “A”. Each person’s guess (Xi) (i=1,2,3,..,N) is Gaussian distributed around the average A. The normalised sum of N guesses: Y = sum(Xi)/N Average of Y: avg(Y) = sum(avg(Xi))/N = sum(A)/N = (N*A)/N = A