Data Wrangling with PySpark for Data Scientists Who Know Pandas - Andrew Ray

[@AlessandroBottoni]

Fantastic introduction to PySpark for beginners. Hope to see Andrew Ray again on the stage for other presentations.

[@ratkush]

Must watch Q n A session in the end. I loved it.

[@fiddlepants5947]

Really nice how we see pandas and pyspark functions side-by-side!

[@javonnii436]

yea I thought the same!

[@pratikmehta1152]

Volume is low! :(

[@konstantinrebrov675]

use detachable speakers

[@santil.7072]

Does it mean that using pyspark sql is the best practice in data wrangling using spark?

[@raphaels2103]

19:12, now pandas has an SQL support

[@enes-the-cat-father]

Thank you for such a great presentation for beginners!

[@tanishasharma3665]

he provided with a really good comparison between the two!

[@ZenvilleErasmus]

Cool talk and key differences nicely illustrated.

[@harjeetkumar4632]

Here are some more videos on spark Spark Interview Questions: kzbin.info/aero/PL9sbKmQTkW05mXqnq1vrrT8pCsEa53std

[@kevinlin5486]

This a great video. Exactly what I'm looking for thanks very much.

[@abrahamf80]

My path to data was a little bit unsual to say the least, started to work in the financial industry using databricks and now on side projects started to work on pandas... funny that I actually used this video backwards hehe

[@thedarkknight579]

Thank you so much for the Session ❤️

[@toygraphers240]

Thank you very much for your contribution.

[@1over137]

PySpark is great with it's read only. It all goes badly wrong when you try and write anything with a typed schema.

[@Arjun147gtk]

I think I need a soundbox on full volume to hear this.

[@jaspreet0305]

I've the same issue, thanks to the captions, I saved a lot of money

[@VishalSharma16]

Super helpful, thanks for sharing!

[@francischab2262]

7:49

[@musasall5740]

by just downloading and writing this code it will not work. You have to create a session.

[@willwright5181]

Great intro!

[@alexnim4873]

great presentation!

[@goedzo4361]

Really helpful

[@elliottharris4526]

Would this be a good tool for combining large numbers of csvs into a single dataframe quickly and then performing manipulations on that dataframe before outputting a single csv?

[@krishnakishorepeddisetti4387]

Which is better in databricks environment?? Python or R or SQL..reply in comments

[@jimbocho660]

Most people seem to find SQL better.

[@Rabixter]

Whats with the volume?

[@xiaoyunzhang6878]

Nebraska Alumni

[@myshkinovavich]

Too quiet please fix

[@Tyokok]

great tech video, but volume really ...

[@Drivebyeasy]

Hey Andrew could you send me your Github link

[@kaixianghuang8589]

LOL good presentation, but unprepared for the Q &A

[@TheBjjninja]

Why did someone ask about uDF? What does UDF have to do with spark?

[@Atlas-ck9vm]

Just use koalas.