An Embarrassing Survey - Randomised Response

Toss a coin. If you get Heads simply respond with 'Yes' in the comments below. If you get Tails answer, truthfully, the question 'Do you pick your nose?'
If you don't have a coin, roll a die instead. If you get an even number respond with 'Yes'. If you get an odd number answer the question 'Do you pick your nose?'

[@XPimKossibleX]

occasionally

[@XPimKossibleX]

shit

[@SolracFS]

+michael benzur :')

[@jacobclaassen4709]

www.dtic.mil/dtic/tr/fulltext/u2/743852.pdf page 20 he lied about vietnam and you just happened to be tc

[@PMiah]

I smoke a bit

[@tremkl]

Watching this video 11 years later, the phrase “Like Will Smith” seems like it’s going much different.

[@singingbanana]

Currently we have 472 answers. 451 people said Yes, 21 people have said No. By chance alone 236 of these people probably threw Heads. This means the other 215 Yes answers, out of the remaining 236, are answering question 2. That's about 91.1%.

[@schadenfreudebuddha]

instructions unclear: coin stuck in my nose.

[@qorilla]

This way a "no" clearly shows that someone claims not to pick their nose. With some other questions it may be better to provide plausible deniability for both answers: Toss a coin. If it's heads, toss it again and you answer the question: "is the second toss heads?". If your first toss way tails, you answer truthfully. Afterwards, you "just" subtract the noise out by repeating it on many people.

[@idlingdove]

Yes. This is a clever way of finding an approximate percentage of the population that does an embarrassing (or illegal) thing. However, there are two caveats: 1) Some people will tend to say "no" to the embarrassing question anyway, even if the truth is "yes". 2) Some people will tend to say "yes" to the embarrassing question even if the truth is "no", because that is the only way to guarantee anonymity (you are not divulging any information about yourself specifically if you say "yes"; you are if you say "no"). Of course, if 1) balances 2), we have no problem...

[@NoriMori1992]

"Some people will tend to say "no" to the embarrassing question anyway, even if the truth is "yes"." They would anticipate that. For surveys such as the marijuana one, it sounds like the primary concern is to find out if _any_ of them do, and at least get a lower bound on how many. The results tell them that _at least_ 10% do.

[@kemkyrk8029]

The irony: i was picking my nose when he asked the two questions !

[@PMiah]

did you find any slop

[@coryman125]

"Is the sky blue?" It sure isn't where I live. Clouds, clouds and more clouds...

[@PMiah]

and slop?

[@keyar87]

Imagine if 55% had been answering "no" :)

[@raffaelender9464]

It would have meant a lot of the soldiers were high during this experiment

[@thomastakeshita4979]

Sometimes

[@U014B]

I didn't need to flip a coin. Yes.

[@harrisonshaw513]

who the fuck has a dice and not a coin lmao

[@singingbanana]

Harrison Shaw Not all countries have coins with Heads and Tails.

[@hanzimaster]

singingbanana Give an example?

[@bramluiken9643]

I spent all my coins to buy a dice.

[@marcius447]

hanzimaster a coin with number and emblem

[@hanzimaster]

Then pick one side to be the Head, and the other side is the Tail.

[@bandit467]

My response is as such: I don't need to flip the coin

[@thoperSought]

+bandit467 if you just answer 'yes' without flipping the coin, you're skewing the results.

[@guacamolegeneral1060]

+ThoperSought not Really

[@thoperSought]

Alexander Ledl how do you figure? it seems like the point is that you know the statistical distribution of the coin flip, so you know how many 'yes' answers to ignore so that you can calculate a percentage. if you answer 'yes' without flipping the coin, then you're effectively answering yes _whether you got heads or not,_ which means that the distribution of the results isn't quite right-in a sense, you're taking away half of a 'no' answer. if enough people do this-and there were *loads* of comments similar to this-the results are no longer representative. if even one person does this, the results are fractionally not representative, anymore, i.e., skewed.

[@bandit467]

+ThoperSought I don't see how that is affecting the result. For someone who picks his/her nose, there is literally 0% chance of him/her answering a no, even after repeating this trick for n times. If I flip the coin and have a 100% chance of answering 'yes' and if I don't flip the coin and answer yes, I don't see how that's statistically different. The reason why I said I don't need to flip the coin is that I know, for sure, that I'm going to answer a 'yes' without the need to perform the flip. In other words, I DO pick my nose. Let's say coin flips are not biased, I already have 50% chance of answering a 'yes', the other 50% of the time, I am going to truthfully answer the embarrassing question, in which case is another 'yes'.

[@guacamolegeneral1060]

+ThoperSought sorry

[@Zilarrezko]

If I don't pick my nose... Then I am unable to breathe through my nose... And I'd rather pick my nose than to suffer looking weird by having my mouth open all the time to breathe.

[@Danicker]

Zilarrezko nah breathing's not important

[@qwerty.760]

@@Danicker You two have similar dps :)

[@UrnestHemingouey]

Yes. The cool thing is if you pick your nose, you don't even need a coin. Ah, the pleasure to be lazy.

[@rosebuster]

Reading the comments here, I noticed a lot of people answered something like "Yes, and I didn't need to flip a coin". You broke it guys! You were not supposed to give that much info!

[@Connorses]

Maybe... It landed on it's side.

[@EGarrett01]

Noteworthy...James Grime's explanation for how to get the real result (throw out 50 of the yes's since those are the result of the roughly 50 heads results) is much more intuitive than the one on wikipedia (double the percentage of "no's") which is awkward and had reasoning that didn't make sense on the first few glances. Shows again that the ability to explain something well is an uncommon talent by itself.

[@me_fault]

did you fix the Wikipedia article?

[@EGarrett01]

+darkcity darkcity No. Did it change?

[@me_fault]

no it still needs work en.wikipedia.org/wiki/Randomized_response

[@NoriMori1992]

Explaining complex topics in a way that a layman can understand is one of Wikipedia's major weaknesses.

[@singingbanana]

We now have 500 answers. 479 people said Yes, 21 people have said No. By chance alone 250 of these people probably threw Heads. This means the other 229 Yes answers, out of the remaining 250, are answering question 2. That's about 91.6%. That seems a nice round number, so I might use that in the follow-up.

[@singingbanana]

@Pluvialis The alternative method is to ask two opposing questions, such as: Question 1: I steal from work; Question 2: I do not steal from work. In this case you will get yes/no answers for each question (in fact a yes answer to question 1 would be a no answer to question 2). This was Warner's original set-up. However the probability cannot be 50-50 or the sums do not work, any other figure is fine. Also, this gives less accurate results.

[@tennicktenstyl]

except when like 20% answer "yes" so then you know everyone smokes

[@selassieprophet6637]

'marijana'

[@PMiah]

sloppy marijana?

[@MrDowntownjbrown]

He didn't even have to look at the coin.

[@insylem]

Screw the coin I pick my nose

What? Isn't this the numberphile guy? How did I end up here?

[@singingbanana]

***** I've had my own channel for years.

singingbanana Which is something I regret not knowing. I'm pleasantly surprised KZbin actually decided to show me something I WOULD be interested in. I'm also furious that it kept this from me for all this years...

[@nightmare9566]

singingbanana 9 years, wow....I really like your videos btw :)

[@Vojtaniz01]

Numberphile channel is Brady Haran's channel where James is a guest in many videos. This is James' own personal channel.

[@singingbanana]

Currently we have 416 answers. 398 people said Yes, 18 people have said No. By chance alone 208 of these people probably threw Heads. This means the other 190 Yes answers out of the remaining 208 are answering question 2. That's about 91.3%

[@sebostien]

That Will Smith intro aged like fine wine

[@ConManAU]

Yes. Also, when I learned about this trick it was set up with a deck where p% of the cards asked "Have you ever stolen?" and the rest asked "Have you never stolen?". As long as p isn't close to 50, you can estimate the number of people who have stolen something pretty well.

[@singingbanana]

@Metric60 1. They are ignored. That is not a problem. 2. Their answer is assuredly anonymous and they are asked to be truthful. However I believe there are other more sophisticated set-ups to deal with this problem.

[@singingbanana]

@fulvo Thank you! Looks like we have some statisticians in the house! This video is just for fun, it's an excuse to show the general public some clever ideas :)

[@martyfriedman1881]

Bro u are grt

[@martyfriedman1881]

How did u become a mathematician. Can u plzz share with me

[@singingbanana]

@Maximara Hey, thanks for the reference. Just looked it up. Fox and Tracy say it doesn't have a 'design flaw' in the words of Miller and Cisin, but it depends on the design of your experiment, they still seem for it. And the method is still used of course. I'm not a statistician but I am aware of the variance problem. This video however is just an excuse to show off a new interesting idea to the general public :)

[@jonathonjubb6626]

20% - must have been way more than that!!

[@singingbanana]

@plazmotech There have been a couple of similar comments. To ask the whole population a question would be expensive and time consuming, so the idea of statistics is to ask a smaller sample of people a question and extrapolate conclusions to the whole population. So statistician work on the assumption that the smaller sample reflects what is true for the whole population. There are ways to collect the data and ways to remove any bias. And the larger your sample the more accurate the result.

[@EGarrett01]

This is fucking brilliant. You can use this to gather so much useful social information.

[@Thereadingmusiclover]

yes, didn't even need to flip a coin!

[@Error081688]

Forgive my ignorance Jim. Isn't this dependent on people who choose question 1 to actually flip a coin (or roll a die)? I feel like a large number of people may choose to answer question 1 with a hypothetical coin flip done in their mind. That would introduce a bias because I think more people would choose heads. Just my 2 cents. Would that be insignificant?

[@singingbanana]

@Error081688 Yup. My instructions were to flip a coin, if they don't do that there's nothing I can do about it. Of course, this is just for fun so let's not worry too much.

[@singingbanana]

@EstBoyIscool Doesn't matter. I am essentially throwing away half of my results for the sake of anonymity. If I wanted to I could just interview twice as many people.

[@HighCrazyDiver]

In an exam I had there was a question like this. It went: When 540 people were surveyed 135 people ticked yes: "Roll a die if the answer is 6 tick yes. If it's anything else answer the question. Have you ever lied about being ill to get the day off work?" Estimate the proportion of people that lied about being sick. Round to one d.p. 540/6= 90 135-90= 45 540-90= 450 45/450*100= 10% Thankyou SingingBanana I finally understand!!

[@WombatSlug]

I hope it wasn't a cloudy day when they asked the questions.

[@singingbanana]

@Pluvialis Thank you! I'm obviously going to have to explain the purpose of statistics in the follow-up.

[@singingbanana]

@singingbanana In this case, the sample we are interested in are in fact only half the people we actually interview. The other people are just there as a method to preserve anonymity. We could perform the same study and reach the same conclusion if we only interviewed the people who answered the sensitive question - but we lose the anonymity. Don't get too hung up on individuals being in one group or another.

[@winmine0327]

Obviously I pick my nose. Nobody else is going to do it.

[@thoperSought]

+Wizmut you can pick your friends, and you can pick your nose, but you can't pick your friend's nose

[@Rakkurai]

I don't even understand how this makes sense. Is it "50 of them are saying yes to the sky blue and 10 to i smoke" because half the deck is "is the sky blue"?

[@luwn00bz]

So why not just make an anonymous survey? It is exactly the same thing.

[@thoperSought]

+luwn00bz I would say two things: *1.* sometimes, you can't anonymize it-like when you're interviewing each person as a part of something else (such as a medical checkup), which is the only way you'll be able to use a pack of cards to do it, and is therefore likely how they did it in Dr. Grimes' example *2.* sometimes people don't trust that anonymous surveys are _really_ anonymous, but it's easier to satisfy yourself that a physically random thing is really (more or less) random.

[@NoriMori1992]

People often don't answer honestly even when surveys are anonymous. It's a known problem in research that can be hard to get around.

[@scottyzacharia8386]

It depends on probability aspects. There is a 1/2 chance of landing on heads (in which sky is blue) and 1/2 chance on landing tails (nose picking). Now that everyone is saying yes, we can assume that everyone picks their nose because the only way to see a no is to have someone land on tails

[@Tanukosauro]

Excellent point! Who gets tail is obliged to response: "Yes, I do" XD

[@MajorSleeve]

If you are a “Yes”, you can just skip the coin flip and write yes haha

[@fractuz]

This seems to me like a simple form of secure function evaluation. Although you could do much better with a good protocol (and you could even do away entirely with the outside moderator who is asking the questions/dealing the cards), this is still a nice quick-and-dirty solution.

[@MelancholicPsycho]

Couldn't a person who pick their nose just skip the coin toss and say yes?

[@opaldragon75]

yes - but I am almost broken both habits, but this video made me have a relapse on one of them!

[@juubes5557]

Yea. Wait am I supposed to throw a dice or something...?

[@tjv323]

just because you tell someone to be truthful, doesn't mean they're going to. Especially when it comes to drugs and possibly getting kicked out of the military.

[@liujinkua9063]

Greeting! Professor, I really wonder How could I know 50% of them say "Yes" because the sky is blue? We can't see the question which that soldier chose.

[@WilliamRussell314]

There is a problem though, in that you can ignore the experiment and just answer "no" to advertise your innocence in committing the heinous act in question. I thought it was a perfect test before I realised that fact :( Is there any way around it?

[@justnotme3002]

Hey Jim, How can you determine how many people threw heads and how many pick their noses? I don't fully comprehend it.

[@NotKyleChicago]

How do you know the distribution of the 50 sky vs 10 marijuana in the group of 60 "yes" responses? How large is the deck of cards? How often is it shuffled?

[@AndrewKay]

You either read that coin very quickly, or already knew what your answer would be before it landed.

[@singingbanana]

@CogitoErgoCogitoSum 95% confidence interval for 100 coin tosses would be 50 +/- 10, i.e. repeat the experiment and the number of heads will be between 40 and 60 95% of the time. So we can be confident our answer will be between 0 and 33.33%. But this was just for illustration. We got 479 yes answers out of the first 500 responses. I expect 250 were heads, which means 91.6% people pick their nose. The 95% confidence interval is 250 +/- 22, so I am confident the answer is between 90.8% and 92.3%

[@adi8oii]

I am taking a course on coursera (Stats and Prob) and this topic was covered in the course, I find this idea really interesting and hence onto your video!

[@singingbanana]

Awesome! I like it too.

[@adi8oii]

@@singingbanana 11 years and ur still replying damn!

[@singingbanana]

Ha. They all come into the same inbox.

[@geographymathmaster]

ugh oh. lots of heads.

[@Sdawkminn]

Yes.

[@vivekyadav273]

I didnt flip the coin...and the answer is YES

[@notsecure6855]

What if I am not embarrassed about picking my nose?

[@dns_error]

If, the survey is "assuredly anonymous" why is there a need to develop such technique!?

[@wood_croft]

Serious question: how do you account for troll responses?

[@singingbanana]

@darshakjust4u The technique is the anonymous bit.

[@carolinacarvalho2254]

am i the only one who giggles when he says "if you get heads"?

[@JC-cr5ty]

I have never ever ever picked my nose!!!

[@martinpaulsen1592]

The coin flip variant is better, though, because it doesn't need to be face-to-face and respondents don't have to trust you when you say that half the cards are innocuous.

[@xALLBuTNuTx]

So is this experiment is only viable when the number of sample is large enough?

[@botmin]

Yes! And I couldn't be bothered to get a coin or die..

[@tinpot1978]

The amount of yes answers compared to no answers indicates most people are nose pickers. If that is the case why is it such a social no-no?

[@garrettducat5769]

I pick my nose.

[@zwz.zdenek]

That's not even an embarrassing question. And picking one's nose isn't even illegal. I pick my nose all the time and I wipe it into my t-shirt or just flick it in random direction, sometimes getting it on my face. Only in front of my boss I do palm the booger and flick it inconspicuously under the table. Being casual is the key; if you try too hard to hide it, people will notice.

[@Science4allOrg]

Hi James! I have been learning about "differential privacy" lately and in the book I'm reading, early on, they talk about a slight variation of this randomized response survey. But they did not mention the Vietnam war. Would you have a reference about the American Army having used this (btw very cool!) method?

[@singingbanana]

+Science4All I tried "randomised response vietnam marijuana" in google. Here's one link files.eric.ed.gov/fulltext/ED133636.pdf

[@Science4allOrg]

It looks so old-style and confidential. So cool! Thanks.

[@Toldasor]

I think they do because everybody picks their nose from time to time, but it's still a somewhat embarrassing thing... Yes, by the way.

[@piginaphonebox]

There is a 66.6% chance it will be yes, 33.3% chance it will be no.

[@rainjar]

You were very quick to say yes, which proves you pick your nose! I was tempted to say yes without tossing a coin, would that affect the outcome?

[@Maximara]

Miller and Cisin (1980) pointed out that Randomised Response had a fatal flaw of design decision has too high a variance cost. This was reiterated in Fox, James Alan; Paul E. Tracy (1981)) "Reaffirming the Viability of the Randomized Response Approach" American Sociological Review Vol. 46, No. 6 (Dec., 1981), pp. 930-933

[@1212JackJohnson]

There are very few no's!!!

[@Booskop.]

Yes, the sky is blue, I smoke marihuana, I pick my nose and no, I did not toss a coin. Cheers!

[@purplegill10]

No. Got tails, due to germaphobia I do not.

[@Error081688]

@singingbanana You are right. This is a fascinating method of collecting data. I'm trying to figure out a way to do it on my own. I want to do it with something already known though so I can compare my results with reality.

[@jeremyashford2115]

The poor colourblind guy answering truthfully that he didn't know (if the sky was blue on that day) could be taken for a stoner. Sorry I can't answer your question but I seem to have lost the coin up my nose.

[@jumbo6498]

On the occasion... Oh wait, is this not how the survey goes?

[@emibulls]

Yes, and...

[@MrFreaky0]

So What's the result so far (if we ignore all those who admitted it in the first place :D ) ? :D

[@musikSkool]

But if someone says "no", then you know that they got tails... And are lying...

[@iamchickensafe]

What if the respondent doesn't answer truthfully?

[@iamthecringemaster7867]

Obviously there would be some people who don't answer truthfully, but this experiment reduces that by making a fixed amount of people answer yes always

[@EdenNov]

I don't understand what keeps us from lying if we got tails and what kept the soldiers from lying if they got the marijuana question...

[@wenyichu3486]

I am confused, how do you know for sure 50% are saying yes to "sky is blue"?

[@kzoo_]

idk if you still need help since its been 4 years, but to answer your question: we know for sure 50% are saying yes to "the sky is blue" because 50% of the cards in the deck ask the question "is the sky blue?". The other 50% asks the questions "do you smoke marijuana?" and since the the answer to the first question is obviously yes, 50% of the yes responses can be discarded, and the other 50% are yes responses to the second question. hope this helped:)

[@Roar1910]

Question?------ how did you get 20% out of 10 honest yeses... did you double it to assume for another 10% of "not so honest" answers?

[@HjijijingMC]

yes

[@dammilo]

yes (predicting ~99% of people picking his nose)

[@phffeir]

my question is: who doesn't?

[@aj19bcx]

my coin landed on it's edge, so no.

[@eneses93]

I think there's a small chance that everyone who say "yes" have Heads and are forced to answer with "yes". btw, my answer is Yes. :)

[@truthsmiles]

Who DOESN’T pick their nose?

[@haoduong6565]

Hi, please let me know how to analyze it in the complex design, with sample weight and design effect? Thanks.