Data Mining: How You're Revealing More Than You Think

[@dexis9412]

Could you imagine finding out your daughter is pregnant from a bloody target coupon...

[@maythesciencebewithyou]

"...I'm will become a granddaddy...thank you target"

[@Engel-ol5rm]

Power of statistics

[@gillianross7225]

Wait why is it covered in blood?

[@nemesis_rc]

Lol ngl you had us in the first half. XD

[@angeliquea9557]

I’ve always noticed this because I search really strange things out of curiosity and notice my ads change due to it. To the point which I can tell what someone spends their time looking at based off their KZbin, Hulu, etc. ads lol

[@JasperKlijndijk]

I love how this show keeps getting better!

[@SciShow]

Thank you! We work really, really hard at that. :)

[@maxmusterman3371]

​@@SciShowPlease go back to becoming better :( rip sci show space and psych and quiz show. And the old topics kinda sometimes seem more interesting to me than the new ones. I mean this video for example is so relevant

[@maxmusterman3371]

​@@SciShowThe technical complexity of the new content has gone a bit downhill :S

[@QuinnsIdeas]

The only Data I know has a brother named Lore.

[@scholarmonk3380]

IdeasOfIceAndFire ha

[@JoshHowardChannel]

Okay. Who is Lore a uncle to?

[@PaleGhost69]

Joshua Howard Lal. What do I win?

[@JoshHowardChannel]

PaleGhost69 here's your prize. A youtube comment reply.

[@JadeDragoness77]

I LOVE YOU for that reference!! haha Trekkie for life

[@gabrielr.trindade7557]

Hi guys, your channel is very good, I'm from Brazil, and I watch your videos to learn English.

[@user-wq3rl9jt8q]

Gabriel Ramos Wow, that's impressive because they use advanced english and I know for sure that I wouldn't understand if I was learning English.

[@gabrielr.trindade7557]

It's Adnan I can understand the videos, by the way, a lean something interesting. keep it up, thank you!

[@arzigogolato8944]

They use good english and, because of the different subjects, it really is a good exercise :) just remember to practise and read too: watching videos is somehow a passive exercise, so do not binge watch ;)

[@ellawong4748]

I do the same thing, although I can’t understand what he’s talking about 😂

[@Bbi_Official]

Using the English captions to improve reading skills is also very helpful!

[@musclehank6067]

dang right I'm revealing 💪💪💪

[@haloborn6785]

W h o a h hANK hill

[@themonkeyspaw7359]

Muscle Hank SWOLE HANK BEST HANK

[@HTYM]

Muscle Hank Congrats!

[@firstnamethenalastname]

your comments on every video trying to farm likes aren't funny and you just come off as desperate and pathetic, inb4 fanboys get angry and defensive

[@demianmaurer9374]

hank the tank

[@ecoKady]

As a statistician, I'm quite happy with this video. The simplifications the script writers had to make to fit into 11 minutes were really good. Thanks for a well thought out summary!

[@Failedprodegy42]

This is a bit late. I've already given my soul to the digital God Gooogle.

[@pacman8551]

They're gonna take away fiat currencies and they're gonna convert us all into cyborgs

[@apple54345]

i remember that Target pregnancy thing from Vsauce :)

[@aendir1927]

apple54345 same

[@52hands]

It was also described in the book "The signal and the noise" by Nate Silver. A good read if you like these kind of things.

[@furiko9989]

apple54345 yes :p

[@FedJimSmith]

Yeah I think it's featured on many books

[@Markd315]

Hey, computer engineer here. Great video! Even I learned from it.

[@lasarousi]

markd315 how does being an "engineer" prevents you from learning something new? Just got your paper eh?

[@QuadraticBajeena]

lasarousi He never said that you can't?

[@sebastianelytron8450]

LMAO can't believe SciShow hearted this comment, about as arrogant and condescending as you can get. "Even I learned from it". I really wonder who's in charge of the comments at SciShow cause it's embarrassing.

[@angeldude101]

"You're revealing more than you think." What? That can't be possible! How could I be revealing more than absolutely everything?

[@ConstantChaos1]

angeldude101 same

[@fajaradi1223]

You ... My friend only reveal everything from now and the past. Not from the future ...

[@ConstantChaos1]

fajar adi Pradana I'm way too sporadic to be defined by an algorithm that works for normal people, also I will tell peo0le what I plan to do as well so for all intents and purposes I do reveal the truth (to the same degree of accuracy or better that they get)

[@juliantheivysaur3137]

Imagine if you want to surprise your husband with a child, and then he sees ads for baby food in the mail.

[@PaleGhost69]

Julian the Ivysaur this already happened to a 16 year old girl who got diaper coupons while living at her parent's

[@JustIn-sr1xe]

How do you know it won't traumatize him instead?

[@JustinY.]

Gotta love Deep AI learning

[@best_youtube_channel]

Justin Y. Reply

[@genderfr3ak]

is Justin Y. even human?

[@kanji6376]

Now you even appear on education videos. Stop following me!

[@pierrecurie]

Most of the techniques he described aren't "deep learning"

[@baranorak4080]

I feels as if the Scishow people have there cursor on the upload button and just let it rip right when it turns 4 o'clock

[@nathnlturner68]

Baran Orak, it would be 3 o clock. They live in Mountain Time Zone, like me.

[@baranorak4080]

Agent 187 *smacks lips* crap Well I'm in the central time zone

[@PaleGhost69]

You can schedule when a video is published when you upload it. A lot of these are probably filmed on the same days a week or two in advance. Or however long their editor needs...

[@OEpistimon]

Well, youtube's algorithm (which also uses some forms of data mining, appropriately enough) loves consistent upload schedules...

[@ncc.voyager]

From a technical standpoint, it's more consistent release schedules, rather than uploads directly.

[@volkerherbst1251]

Yeah about Spotify being soooo good in predicting what I want to hear: Why is it always trying to force that silly gangsta rap playlist on me then while I'm listening to Heavy Metal?

[@ancientswordrage]

Don't forget it's trying to sell you stuff

[@volkerherbst1251]

Yeah, so? That's the whole point, you think Spotify is selling their subscription to a 30 year old mostly listening to Heavy Metal, Punk and Indie Rock by forcing him to listen to a genre that is completely irrelevant to him? You could argue that it's a negative reinforcement to buy a subscription to get rid of the ads. However positive reinforcement ALWAYS works better (e.g. "Oh look, Spotify really knows my taste and I didn't have to dig for hours for these really sweet tracks!") But even when I'm not subscribing, they can sell their commercial breaks for a fortune to external companies, IF they can prove it's aimed at the right demographic. And obviously they don't even try...

[@kayleerosemarks4055]

Or crappy TV stars wants be big shot singers when I am listing to people who actually can sing... (No hate if your into them but they can't sing)

[@RandomCowboy420]

Because you Listen to Shitty Music!!!

[@chaliciah8199]

@@RandomCowboy420 facts

[@downup]

Data miners are mining Nasa to leak Despacito 2

[@finanov6646]

I took the Dual Enrollment Computer Science class at UT Austin and one of the modules explained data-mining and data collection. I learned so much from that class. Data-mining doesn't surprise me anymore as a result. And it makes sense: Google is practically the King of data-mining. Even if you're not using any of their services, if you visit a website with Google Ads, unless you turn off third-party cookies, they can still collect your data. Be careful of what you do on the Internet.

[@kannan3801]

Five ways to do data mining- classification,regression,clustering,anomaly detection,association mining. 3 types of machine learning -supervised learning -unsupervised learning -reinforcement learning Algorithms to perform classification,regression and clustering 1.K-means 2.decision tree 3.log/linear regression 4.naive Bayes 5.random forest 6.neural network 7.etc...

[@harrisonfackrell]

". . . help cell phone companies predict massacres in the Congo." *doesn't bring up that point for the next nine minutes*

[@anthonycaruso1781]

9:40

[@harrisonfackrell]

Anthony Caruso, I stand corrected.

[@terrenmaplethorpe4001]

I guess you could say.. this was a good data get out of bed

[@SciShow]

lol

[@BigDaddyWes]

"The messy human stuff." Is what I expected this video to explore.

[@tadpole53]

I can understand being suggested things online after I've actually searched them myself, like being suggested videos about, I don't know, rabbit food if I've typed it before on KZbin or Google and my device catches that. But what about all the times I've literally just talked about things, never typed them in, and then started getting suggestions for them. It's happened many times, most recently KZbin was suggesting me videos about barracudas after I was talking about them with my dad. I didn't look up anything about them on my phone, I don't even know if I've ever even typed the word on my phone, yet I get suggested videos after talking about them.

[@FriedEgg101]

I ditched gmail a couple of years ago. The final straw was when my dad forwarded me an email he had received with flight details, so the email was from my dad and not the airline company. I then had an email from google saying "we noticed that you're going away on these dates, so we added the dates to your google calander". At the time I didn't even know I had a google calander, and certainly had no plans to start using it. I was so annoyed and creeped out.

[@AndreiTache]

In theory I’m all for data mining if it makes the product better, but I’d really like if companies would just ask me for it and I would give them the data that I want instead of them just finding every little scrap of information (like age and sexual orientation... that’s just creepy)

[@kitcoffey7194]

Andrei Tache Yes, the vid should have discussed ethics and policy more, as you point out.

[@kitcoffey7194]

Consent and control over one's personal data is important

[@gigglysamentz2021]

So many things covered ! That was nice.

[@Mirudesuu]

Love your videos!

[@SciShow]

Thank you!

[@BothHands1]

Wondering how your name is spelled. 美流 seems like it would be pretty. Mi seems pretty obvious, but not sure about the second character tbh :P (Sorry, been drinking 😂)

[@Mirudesuu]

Danielle Spargo haha Miru is “見る” or “To see” in Japanese. I’m an entertainer, and I perform for people to watch and enjoy. So Miru is my stage name :)

[@BothHands1]

Miru あっ、そっか。ステージ名として、素敵だわ。やっぱり考えすぎだったねw

[@MegaElderscroll]

I thank Hank Green. He helps feed my children and defends feminist scum.

[@yosoyysoyyo]

Y'know, I straight-up filled out google's ad targeting and interest questionnaire because I hoped it would stop showing me (under 21, a vegetarian, no driver's license) ads that were irrelevant to me, but I still get cheeseburgers driving specific cars down to the local brewery. Kinda wish the algorithms were even better at creepily mining all my deepest darkest secrets.

[@Master_Therion]

Can data miners mine minors' data? I mind if miners mined my minor's data. Mind your own business.

[@Master_Therion]

#wontsomeonethinkofthechildren

[@cwtrain]

Yes. We do regularly.

[@emiliepullen2082]

Good one.

[@FishFind3000]

Kids have much stricter privacy

[@dutchik5107]

Meh. Legally can consent to it at age 13. People do it before it that though.

[@jasminewood395]

Wow... I've not been keeping close track but 5+ m subs is a lot! Has is been growing really fast lately or have I not paid enough attention? Gratz on 5m! Awesome and deserved, lots of hard work and time but its paid off!

[@WildEngineering]

In my probability course I wrote MATLAB code that did a linear regression on a set of data points I gathered. I was trying to model male/female palm ratio( height / width of their hand without digits) to the length of their middle finger.

[@cthulhutheendless1587]

Damn, this is the Minority Report of Capitalism

[@celinak5062]

Jack Timothy yep

[@jwessel1969]

Makes me think of the TV show Person of Interest.

[@xmaswitguns]

Or perhaps China’s social credit score program... All this information would be of great interest to police states.

[@crimfan]

Nice video. Regression and classification are indeed very, very similar. The math is almost identical. Anomaly detection is like model diagnosis, looking for points that don't fit the model. The one big thing people need to recall is Goodhart's Law: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." So these regularities will frequently die when they start being used for some purpose.

[@McHaven07]

I never stopped liking Limp Bizkit, some of their songs, at least.

[@HTYM]

McHaven07 I like cake.

[@FishFind3000]

Yep

[@aaronwise1089]

Everyone does

[@kassiman5307]

*THATS A VERY INTERESTING VIDEO .....!!! THANKS FOR THAT ,,!*

[@TurtlesC4nFly]

It's so weird, I DO like having Spotify recommend music to me that I will probably like, but it IS scary how many companies can know so much about you. Mixed feelings, which makes it tough to find the perfect spot for data mining.

[@ZaBiMaRuSz]

the LimpBizkit Joke kill it. my god you got me out of my concentration state in work. i was Loling my ass off. Thanks SciShow

[@pH7Programming]

Doing a Master in Data Science, and this video is actually a decent good overview 👌

[@jerryhanzel2953]

I like the way this is edited

[@alexlandherr]

That example of the upset father is the prime example of this, being cited in documentaries such as “Terms and Conditions May Apply”.

[@peterbrdar3661]

Nicely done guys. Once thing isn't quite right at 2.27, there are pretty clear distinction in what those are. I can see people using data mining and machine learning interchangeably, even that is a bit sloppy

[@pascha-j]

One of the best videos on the topic, thanks for the overview!!

[@baileydustin2139]

This helped me wonders, thanks!

[@Kabbinj]

This was shockingly similar to the first lecture i had in a data mining course at university. You even used some of the exact same examples :O

[@FArkhanor]

His forehead is so shiny that I think Stefan has officially become a rare Pokemon. By the way, what a cool episode, thanks SciShow ! :D

[@earthbjornnahkaimurrao9542]

I love this topic. Imagine all the things we can discover!

[@anny23108]

Who disliked the video? it is aboslutely brilliant! thank you very much for this!

[@XenomorphYo]

The limp bizkit line killed me hahahaha

[@BenTajer89]

Target sent me those pregnancy ads when I was 17, I am a man, and I did not even live in a country that had target. I also did not shop online.

[@Nipponcakes]

Watching this in the break room... at Target lol

[@francischic7854]

Once in awhile you guys have a really great video.

[@mattkuhn6634]

Great episode! A minor quibble: I would have put a bit more emphasis on the fact that clustering doesn't require any training data, whereas both classification and regression usually do, though there are many unsupervised methods for all of these tasks.

[@bluejedi723]

this channel is why I'm passing most of my college classes

[@SlyPearTree]

That was a very interesting and informative episodes. I'm ambivalent about data mining so companies can figure me out, I consider it a plus to get targeted advertisements but I don't want it used to influence how I think. I do see everyday that it is not perfect by the number of car advertisements I get despite the fact that I do not drive and will never drive because I'm bad at not letting one thing capture my complete attention, I've collided with parked car while riding a bicycle.

[@rareroe305]

The targeted ads can also be shockingly inaccurate. Like when I get them in languages I don't know.

[@MikefromTexas1]

"If you had an off switch, Doctor, would you not keep it secret?" -Data

[@sketchstuffs]

I look up motorcycles a lot. Now all my ads are nothing but motorcycle helmets, jackets, etc. on every computer and device I happen to be on.

[@LilyMyLolita]

As a machine learning practitioner, I think Stefan said it pretty well 😉

[@nikolademitri731]

Freak Folk is a real sub-genre, and it’s peak was Animal Collective’s 2004 album, “Sung Tongs”. Now go; go and listen! You’re welcome. ✌🏼

[@banditpsy]

That has to be the most positive insurance example I have heard... EVER come to think about it.

[@bobb6924]

1:44 SATURDAY IS ALSO A DAY :(

[@StrongButAwkward]

Missed a real opportunity to sneak in a picture of Commander Data during the "Data, data, data, data!" bit at the start.

[@sethbrooks8889]

First 100 comments, John Green should make one scishow episode

[@ConstantChaos1]

Seth Brooks 100000000% agree

[@gephc4]

#P4A2018 ...And it should be about Mars.

[@grandpastoner7527]

Mind Blown! Very informative. Thanks 😊

[@adhaarsharma753]

Future is here!

[@brendanberg9523]

Best description of k-means I have heard so far (-;

[@jwfelde]

Machine learning techniques such as these are also used with satellite imagery to detect patterns and classify features on the Earth’s surface

[@ubermench3000]

Hierarchical clustering/Hierarchical linear modelling is very useful in research of schools.

[@Euphytoseful]

I have to admit I had never heard of this kind of data mining before, for me data mining had always been decrypting things you're not supposed to in search of valuable information.

[@psychoactivednb]

I recognize that number. I lived in Great Falls for 8 years as a kid...1 area code for the whole state.

[@shanehagan]

fantastic video

[@Jake-bt7ig]

Also, what’s scary about data mining? If target knows you are pregnant, what’s the harm? That you get access to coupons that save money on the things you are gonna purchase for your baby anyway? I remember a time when I bought a sous vide, added modernest cuisine and an isi dispenser to my wishlist, watched a few cool science videos on liquid nitrogen and suddenly a video was suggested about cryosteak. It’s a really cool thing, I learned about something I didn’t know before. The coolest part was that something was suggested to me that was right in line with my interest at the time. This sounds like education at its finest. I know it’s education targeting my wallet, I went straight to amazon and added A liquid nitrogen dewar to my wishlist. None the less, I learned something extremely relevant to my interests because a huge set of data about people with similar interests was both available and examined by computers to suggest that video. So maybe I’m missing something. I like learning so please point it out if I am. But what’s so scary about collecting data. It’s not like that judgie suburban housewife down the street is poking through the data collected about me so she can judge more. And if she is, what do I care if she’s got nothing better to do than judge others? I don’t expect she can be a very enlightened, satisfied, happy person if that’s what she is choosing to spend her time doing. And I can comfort myself with the fact that I’m having a far nicer time than she. Please, someone, tell me why I should be scared and why I should prevent the tracking at all costs? What am I missing?

[@DasAntiNaziBroetchen]

You're missing the grand scheme of things: Prediction peoples' actions, invasion of privacy, and population control. He literally mentioned it right at the beginning.

[@Mr6Sinner]

I’d like to believe that the title is meant to be literal.

[@XxlololalalexX]

Felt that Limp Bizkit joke LOL

[@jaybea365]

I've learned one thing about data mining, I am an outlier. I have yet to find a system that manages to do anything other then offend me, KZbin ads, my grocery store points card, etc. They all just fail.

[@CompBioQuest]

Clearly explain. Good Job!

[@vettaimannan2082]

1.classification 2.regression 3. Clustering 4. Anomaly detection 5.association learning

[@rufusneumann9703]

Anamoly

[@PaleGhost69]

Why didn't Star Trek have Data in a miner's hat. What a great meme lost... :(

[@fayecbb]

PaleGhost69 you could just make it

[@JoshHowardChannel]

PaleGhost69 make it I'm pretty sure TNG had away missions in caves

[@TheAgamemnon911]

I do remember a scene where the android was literally a phone.

[@VezixHaikal]

THE POWER OF CONVINIENCE STORE/SHOPPING MALL/SUPERSTORE

[@two-face1041]

00:20 have a feeling this video could possibly be about data just a feeling lol

[@saeedalamry3857]

very helpful, thank you

[@c2lredstone946]

I don't care how much data companies get about me as long as it's not my passwords or credit card info because I always do research before decidions. Common sense absolutely destroys targeted ads.

[@c2lredstone946]

The only problem is that I don't get coupons because what I want are fairly priced PC parts.

[@MrSthotwhelz]

Solid episode. Script needed a bit of editing.

[@TwilightBinder]

How is this not considered a blatant violation of privacy and not illegal as hell? Also noticed the minority report set up this vid mentioned

[@AlexiaSAGITARIO]

Great one, thank you !

[@nakedpwr]

Omg! They said you would say those words exactly!

[@Elitekross]

came here from my Master's Level data mining class

[@rare523]

Every time i see the data usage for decision making and control by s company it gives me West world vibes... well done video.

[@gordonlawrence4749]

Arthur C Clerk wrote about this in the foundation series way back in the 60's.

[@greyturner3114]

Awesome, I first saw data mining on mobile games I'm playing few months ago.

[@Azrage]

I operate under the assumption that the companies know basically everything about me, but that random people that aren’t related to the companies don’t.

[@YellowPenetrator]

I always knew that Tuesday’s weather was important

[@Coach_Stefan_Brule]

Love the video great jov

[@Jake-bt7ig]

With all this data mining and predictive ability, how come every store in Minnesota still sell out of vehicle snow brushes every year with the first snowfall?

[@skagern]

Damn, the prepaid phone cards correlation is some dark stuff, but the connection seems logical.

[@galenrichter41]

wow, i've never heard the term data mining in this way. the only times i've heard it is to talk about people going through all the data in a game program. to like find things that are hidden until a later date, or other hidden bits in the code, like some underlying formulas.

[@trebortrahrebe2575]

Ya but every time I watch Jeopardy they have a question about what I just googled online. How's that possible? And TV commercials already seem to know what dog food I eat, and what pet products I use. And I don't even use money.

[@eliseholowicki]

Can you link to the video mentioned at the end?

[@GreenTea13121]

Please tell me I'm not the only one that thought of Data from Star Trek Next Generation every time he said Data in the intro of this video. I kind of want to edit it so Data appears on screen every time he says Data. Not going to, but it would be funny :)

[@TECHINSPEC]

Thanks.

[@myalxaccount2]

Google flue trends case shows how big data implementations might fail with the wrong assumptions.