Intro to Big Data: Crash Course Statistics #38

Today, we're going to begin our discussion of Big Data. Everything from which videos we click (and how long we watch them) on KZbin to our likes on Facebook say a lot about us - and increasingly more and more sophisticated algorithms are being designed to learn about us from our clicks and not-clicks. Today we're going to focus on some ways Big Data impacts on our lives from what liking Hello Kitty says about us to how Netflix chooses just the right thumbnail to encourage us to watch more content. And Big Data is necessarily a good thing, next week we're going to discuss some of the problems that rise from collecting all that data.
Crash Course is on Patreon! You can support us directly by signing up at / crashcourse
Thanks to the following Patrons for their generous monthly contributions that help keep Crash Course free for everyone forever:
Sam Buck, Mark Brouwer, James Hughes, Kenneth F Penttinen, Trevin Beattie, Satya Ridhima Parvathaneni, Erika & Alexa Saur, Glenn Elliott, Justin Zingsheim, Jessica Wode, Eric Prestemon, Kathrin Benoit, Tom Trval, Jason Saslow, Nathan Taylor, Brian Thomas Gossett, Khaled El Shalakany, Indika Siriwardena, SR Foxley, Sam Ferguson, Yasenia Cruz, Eric Koslow, Caleb Weeks, Tim Curwick, D.A. Noe, Shawn Arnold, Malcolm Callis, Advait Shinde, William McGraw, Andrei Krishkevich, Rachel Bright, Mayumi Maeda, Kathy & Tim Philip, Jirat, Ian Dundore
--
Want to find Crash Course elsewhere on the internet?
Facebook - / youtubecrashcourse
Twitter - / thecrashcourse
Tumblr - / thecrashcourse
Support Crash Course on Patreon: / crashcourse
CC Kids: / crashcoursekids

[@oliverblake6725]

Anyone else have to watch this for home work

[@sarahebert2510]

I am honestly so proud of these two brothers. They're so intelligent and ambitious, and they really are the definition of changing the world. Looking for Alaska will always be my favorite book, and I am really happy for how these two men have turned out. They're brilliant, and they're good people. I will always support these two.

[@SatishchandraSalam]

"Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it..."

[@gardenhead92]

Remember: it's now how big your data is, but how you use it

[@stevephillips8083]

My data is bigger so I can do more with it?

[@HyperGnome]

To be fair as a student of data science, that's inaccurate, there is such a thing as too little data.

[@mikail5682]

There is no better data than more data

[@HyperGnome]

Accurate @@@mikail5682

[@dominickoquin3849]

Lol.

[@bigmike4133]

Personally it seems unsettling that one's Behavior can be so accurately maped with an algorithm and sufficient data. Doesn't that lend itself to large populations being heavily influenced if not outright controlled?

[@2modern2Bgenius]

Populations are heavily controlled, the psychological effects of the impacts of big data would be an interesting sociological topic to explore. It probably already has been explored. The reason why you do the many things you do is influenced by big data and the media around you and what category that media thinks you fit it. And the more data you put in, the more you reinforce big data with your choices and actions, the more the big data reinforces you to continue those actions until it can slowly change the actions you do.

[@Mr_Wallet]

I mean, everybody talks a big game about big data, but it took Netflix and KZbin about a decade before _any_ of their recommendations were remotely passable for me. (I mean _any_ of them.) They're still clearly basing huge parts of their algorithm on what attracts the average viewer with similar topics of interest but which I personally have absolutely no chance of clicking. Netflix abandoned their "not interested" button and KZbin's doesn't seem to do anything but stop just that one specific video from being recommended, failing to dissuade similar videos from appearing for weeks even if many such videos are similarly flagged every day. It seems like if they were really capable of chugging through every piece of data about me they would both absolutely love to use that to pre-filter stuff that I probably don't want to watch. Basically what I'm saying is that big data has some impressive results for casting people into big buckets (heterosexual male is over 45% of the population - it's not surprising that it takes _very, very little_ information to make that a safe bet), but there's still severe limits on accuracy.

[@dontdoit6986]

Michael Fregoso welcome to marketing 101

[@dummypg6129]

There is no such thing as privacy when you put everything on the internet. It is just a matter of who is looking on it.

[@dlissett821]

Or use a smart phone or watch Netflix or walk down the street.

[@zanemiracle6121]

06:35 "information such as location and SPEED..." 06:40: "This information isn't particularly useful to anyone..." Imagine if virtual ticketing was a thing. "Sir you were going 75 on a 50 mph road according to Google maps at 03:27. Per this email (or mail), you are now being fined for (insert amount here) gg"

[@MilitanT07]

If they wanted to do that, they would force the car manufacturers to send back a more accurate data.

[@joshuajoshua46]

The moment you realize Orwell was onto something.

[@alanfuryhh4744]

The moment that thing happen, we might've already have a self driving car

[@MokhetheaLenyalo]

i hope it never comes to this,..

[@jameswhitfield618]

Just thinking there should be a 10 sec summary of, "this is what we learned in this video. " good stuff.

[@predatoryanimal6397]

Big data aka big brother

[@camilaavila7679]

This is amazing! Thank you for existing!

[@Jake-hc5xs]

That’s what I call BIG DATA ENERGY

[@geographconcept7523]

I haven't been following Crash Course Statistics but this sure is interesting!

[@jeremywenrich]

Latitude Brown, same, but I’ll be paying more attention now. This was great!

[@Smico85]

same here, glad they kept popping up in my subscription feed

[@RussTeeTrombone]

I think a lot of us ended up here from elsewhere z

[@rkpetry]

*_...'yeah' but when people watch a video, the maker wants to know where, they're stopping or skipping, where they've backed-over and rewatched a scene, when they've used the slide bar to find interesting images, what portion they've actually, watched, not just listened-to... big data needs get much, bigger, before it'll be useful, data... this is partly that question I asked episodes ago-as to whether statistics can reconstruct its source..._*

[@gigachad8338]

Nice vid!! And Pleasee do Multivariate Analysis :(

[@soundbroom3798]

Does anyone else enjoy privacy?

[@DeadlyNerotoxins]

Here in NH we just added privacy as a fundamental right to our state constitution.

[@AvailableUsernameTed]

Sure do, Michael.

[@stevephillips8083]

Don’t worry, I will look after your privacy for you...

[@PeeosLock]

privacy is nothing before you are well known

[@criskity]

Liked. And stored in the Big Data corpus.

[@imcaetano]

Im going to use this video in my industrial economy class presentation

[@keverenguillou9825]

We're discussing big data in my Communications course at the moment, so this is definitely gonna come in handy for my final

[@BaneLoki]

Screw ads, I’ll pay for premium just to avoid it.

[@catalepsy8916]

Don't you know what ad block is?

[@pyroicarus1203]

That might help you personal, but they still collect and sell data on you.

[@BaneLoki]

Catalepsy There’s ad block for iPhone? Do tell

[@solhsa]

Maybe big data shows that most people will prefer ads to paying, even if they do hate ads.

[@dummypg6129]

@@BaneLoki yeah thats called android

[@abhijitborah]

Mind reading is almost there. Skynet, when it takesover, may never get noticed. Great video, thanks.

[@pyroicarus1203]

Just because you got an ad blocker doesn't mean your data isn't being collected.

[@deathmetalweedman3572]

no but it means i dont have to watch ads

[@zanemiracle6121]

"00:05 you may have seen an ad before this video" ads...been a long time since I've seen those on KZbin (KZbin premium user)

[@PaulsPubAndBrew]

I'm premium also but KZbin music just started ads on me the other day.

[@zanemiracle6121]

Paul Gardner really? I dont get ads there. Do you have them purchased separately or the same? Then again I am an old KZbin red subscriber so I still pay the old fee for both so there's that. Did it change?

[@PaulsPubAndBrew]

I'm an only Red user also. Was $7.99 for years but upgraded to family access at $14.99 a year ago. I never get KZbin ads (2+ hours per day) and never on Play Music (1 hour usage per day). Just in the KZbin Music app. Like 5 seconds each.

[@zanemiracle6121]

Paul Gardner That's really bizarre. I use KZbin music premium as my go-to audio streaming source instead of play music and KZbin premium. Haven't seen an ad from either lately. I wish I could explain why you get ads for the former

[@deathmetalweedman3572]

ublock origin on pc , i dont watch youtube on my phone , crazy i use my phone only for phone calls , i dont understand why anyone would pay for youtube

[@RangerRuby]

There is so much data in this world, like... over a google amount.

[@dummypg6129]

Over *googol* amount

[@MichaelSHartman]

Big Brother is drooling.

[@ThiruMahal]

That's a kurzgesagt poster in the animation

[@shuashua958]

I am definitely using this and apparently the next video in my GP class. Thanks for posting.

[@frankservant5754]

Not all videos that have a million views are credible this one has surprisingly less views for the quality of content it has

[@acho8387]

Freedom becomes more and more of an illusion....

[@Robbya10]

Video has been out for 20 seconds and it has dislikes lol what.

[@gutstallion]

Pre-haters or KZbin itself, all about conspirology.

[@TacticusPrime]

Don't forget that Alibaba is a Chinese government front group.

[@zhubajie6940]

A Faustian Bargain for the free software or even ad-ridden software. This is the worst thing about living in this century. Even when then they send ads that aren't applicable after knowing your every step.

[@jeanqnguyen4542]

This is depressing, but what the hell, Liked !

[@victorzurkowski2388]

Around 3:22 the paper by Kosinski et alt in the Proceedings of the National Academy of Science is mentioned. This paper analyzing the predictive power of Facebook "likes" has received considerable attention, in part on account of the unauthorized used of the data by Cambridge Analytica in the US 2016 elections. The measure of model accuracy reported in the paper is the area under the curve, which is a rather weak measure of model accuracy. Moreover, unlike most Statistics papers, the closing paragraphs of the paper do not point towards possible future research directions, open questions, etc. but are intended to alarm the reader of impeding risk - a warning for which there are more proper channels - The results reported in the paper are not as spectacular as they would be if the measure of accuracy was a stronger measure.

[@akbrahma7739]

Thanks for this vid, learned a lot.

[@amedeelefroiddemereaux2865]

Big data. I n t e r e s t i n g

[@joedasilva134]

awesome video Thanks!

[@avi12]

Which company generates most of the data per year?

[@biograde7084]

'Lets us do things like recognise the faces of criminals with machines' Who decides who's a criminal? Don't open Pandora's box if you don't want what's inside

[@batoola4986]

Hello

[@bkallday80]

My mans next to the young lady @59 seconds wasn't typing a dam thing...! Lol

[@neilcidial-masrysandagesid7796]

Beards. That's a short list. It's not an impossible life. But sleep depriving is not cool.

[@TheSteveo909]

i've got a presentation coming up in a month on big data in my comp sci course so you couldn't of started this series at a better time.. although i hope you get a few more episodes out before it's due hahahaha

[@Drake00075]

The nefarious things you can do with big data are more dangerous than nuclear power plants being everywhere.

[@lubnahazem3972]

Have a nice day😊😊

[@tw06le1]

Netflix makes you wonder if good movies are actually finite... getting boring now...

[@0MoTheG]

I learned nothing about Statistics. And nothing of this was news.

[@KaitlynArgyle]

Just because it wasn't for you doesn't mean it wasn't for others

[@0MoTheG]

@@KaitlynArgyle True, but I expect better of CrashCourse. They did not even let us know anything about the specific scientific problems of big data. We can gather it, we can store it but how do you make it useful?

[@capriphonix8863]

When. Are. You. Gonna. Do. Calculus. Please. Like John I know you hate math, but I think we're all waiting for it.... ;.;

[@shawdow357]

Calculus is boring tho

[@lerolerolerolerolero256]

You'll have to wait for cc math for that cos calculus does not mostly come under statistics . I think they've touched upon calculus in one of the videos of cc physics .u could check it out

[@BlokenArrow]

It’s Hank that manages CC.

[@bartliu3158]

谢谢老师

[@Metacognition88]

Can you a segment on medium data please.

[@TH3H0uS3Fr3aK]

Psycho Pass. mark my words...

[@1nomadic]

I learned that I'm definitely not a fan of Hello Kitty fans!

[@wildmansolow2]

its no privacy, everything thing is collected and stored wow its scary

[@blinimujka1416]

who else is going to watch random movies/shows on Netflix just to confuse the Netflix algorithm ?

[@marcushead9985]

Internet of things, aka internet of sh!t. Avoid wifi-enabled appliances if possible, because all other problems aside, one day the manufacturer will just stop supporting it in favor of a new model and you will have a house full of expensive oddly-shaped bricks.

[@koolaid-jammers6714]

game end

[@grainfrizz]

Splunk can handle it.

[@JishinimaTidehoshi]

At 9:44 why are you talking about Ben Affleck? Didn't you mean Matt Damon??? He's the one in the movie and he's from Boston.

[@gobyg-major2057]

Shows Instagram and talks about amazon, shows amazon and talks about Instagram...therefore Instagram = amazon

[@masaratech]

🇧​🇮​🇬​ 🇩​🇦​🇹​🇦​

[@opsimathics]

"google privacy*"

[@cinnamonbay]

[@deathmetalweedman3572]

i hate all of this

[@athenastarlight6151]

First

[@rkpetry]

*_...so with big data the automotive traffic center could change the billboards so that more people steer around a certain zone meanwhile your car is being driven into that zone, (maybe you're a VIP, or ER support, or a target)..._*

[@gopro_audio]

Try 15 billion people on the planet, where is your big data from lol.

[@michaeljorfi3552]

To be clear,not everyone likes Beyonce.nor should they

[@PeeosLock]

随着电子科技进步, 不管你喜不喜欢大数据时代已经急速降临。大数据时代是中国和印度一个多人口的黄金时代；但是除了人口和科技，中国优势比印度明显：人口+文化+科技+中产数+法律。西方已经被“个人隐私优先权“法律绑死，再加上人口稀少，西方再不改革注定在这时代是陪玩的大输家。

[@PeeosLock]

大数据在各种行业里有不同用途， 警察，更好打击犯罪 政府，维持远观治安 个人，牺牲隐私被动享受各种好处 公司，找对客户群赚大钱 医疗，更好分析各种疾病 。 。 。

[@Nixitur]

Almost all of this video is talking about personalized data rather than Big Data. Not all personalized data is Big Data, and a ton of Big Data isn't about personalizing anything at all. When you talk about "the dangers of Big Data" in this context, you're not really saying anything about Big Data at all. It feels a bit misleading.

[@unleashingpotential-psycho9433]

I think people spend way too much time watching TV and Netflix.

[@pucovsky]

And youtube ;)

[@kgashane2226]

You can't give a "crash course" on statistics without any graphs or terms like regression, equation, sample, standard deviation, histogram....

[@gigachad8338]

Nice vid!! And Pleasee do Multivariate Analysis :(