Visualizing Correlation Table - Data Analysis with Python and Pandas p.4

[@johnr2290]

you're honestly a legend to me, I've learned so many cool python tricks from your videos. Gotta love a nice correlation matrix.

[@systm0ne]

Fax

[@chases4951]

Quick jupyter tip for anyone: You can run commands in the notebook by starting your line with "!". You can even use python variables in the command. Example: path = 'data/minwage.csv' !head {path} or !pip install matplotlib

[@DevashishThakur]

You can run bash commands by putting %%bash in the cell as well.

[@hectoralarcon4888]

You are so cool, this is like a mega tutorial while learning neat tricks. Real problem solving you got there my dude!

[@David-zg4sk]

Nice tutorial @sentdex, thanks for sharing!. Just to let you know, there are some shortcuts that may help you. You can have the help of a function with Shift+Tab, so you can see all the arguments that you can pass to that function. Also Shift+enter, commit and creates a new row, so you don't need to create 10 rows in advance

[@sentdex]

I'll check out the shift+tab thing, thanks. As for making 10 at a once, I do that not because I think I need to make them in advanced, I do it to move what I am working on to the top. I want it eye level :P

[@TheVijaySaravana]

Fun tip: Pause all sentdex videos at 0:01 -> You can see him making a judgemental "ooooo" face xD

[@beansgoya]

for those who are using jupyter notebook: at 2:20, for you to show the graph, you need to just add import matplotlib.pyplot as plt %matplotlib inline

[@True_lines-JM]

Thanks

[@RabeezRiaz]

The text printed at 7:53 was the return value of set_xticklabels(), you can suppress it by using a semicolon at the end of that line (very un-pythhonic, I know :D) Also you can remove the plt.show() call by running `%matplotlib inline` in a cell somewhere before your plots.

[@ErichBSchulz]

Or simply assign it to a value

[@MegaTRIANGULUM]

Amazing stuff man!! Learned a lot :D Thank you so much!

[@Mistercapi0]

Great video, I also recommend to rotate x axis labels: ax1.set_xticklabels(labels, rotation=90)

[@nadyamoscow2461]

Many thanks for the fantastic course. I`m not sure but it feels like instead of catching the keys in dict by hand you could do: labels = [abbv_dict[c] if c in abbv_dict else c for c in min_wage_corr.columns] bad = [s for s in labels if len(s) > 2] print(labels, bad, sep = ' ') At least it worked for me (the basic data file has changed by now so there`s no FLSA ) #['AK', 'AR', 'CA', 'CO', 'CT', 'DE', 'DC', 'Guam', 'HI', 'ID', 'IN', 'KY', 'ME', 'MD', 'MA', 'MI', 'MN', 'NE', 'NV', 'NH', 'NJ', 'NM', 'NY', 'NC', 'ND', 'OH', 'OK', 'OR', 'PA', 'RI', 'SD', 'UT', 'VT', 'WA', 'WV', 'WI', 'WY'] #['Guam'] bad = {'Guam':'GU', 'AK': 'blue'} # for instance there are more values labels = [ labels[i] if labels[i] not in bad else bad[labels[i]] for i in range(len(labels)) ] print(labels) #['blue', 'AR', 'CA', 'CO', 'CT', 'DE', 'DC', 'GU', 'HI', 'ID', 'IN', 'KY', 'ME', 'MD', 'MA', 'MI', 'MN', 'NE', 'NV', 'NH', 'NJ', 'NM', 'NY', 'NC', 'ND', 'OH', 'OK', 'OR', 'PA', 'RI', 'SD', 'UT', 'VT', 'WA', 'WV', 'WI', 'WY']

[@ssmith7075]

I came for the knowledge, but I stayed for the wholesome typos.

[@RabeezRiaz]

Hey Harrison. You could use the plt.subplots() function to generate the figure and the axes together, it's nrows and ncols arguments feel more easy to use in my experience than the 111/221 thing of add_subplots()

[@oliviero1756]

do we have a real reason to plot something still coding while we can use a software like Tableau, or Qilk (both have free versions) that are much more nice and explanatory (in my opinion)?

[@Projyots]

Plot is loaded for only few columns in plt.show(), please suggest

[@piyushparadkar1]

i am getting the same thing.. were you able to solve the issue???

[@harikrishnan-pp2un]

I cannot install jupterlab because it says Could not find a version that satisfies the requirement jupiterlab , please help me out to solve this issue

[@laxmanbisht2638]

I am getting the error "". And, saved figure is coming blank.

[@lifeislarge]

Sir I'm trying matplot lib in pycharm but plot doesn't show up. What can be the issue?

[@paulosergioschlogl9550]

Just for a note! Whats means that corr graphic 8)

[@juniormoraba7465]

Where can i get this mugs ??

[@andyn6053]

Nice material but you sometimes go a bit fast and do many steps at once. It would be great if you explained more step by step exactly what everything means. You also have a tendency to rush things in the end. Else, your videos are awesome. Keep it up man!

[@Jakob6174]

At what point should I opt for using 'to_pickle(), to_json()' or any of the other local storage methods? Are they only used for extremely large databases or should I stick with 'to_csv()' for the sake of simplicity?

[@pythonocean7879]

when did you made this series,i was searching alot for this but couldnt find any good material then just saw your series :D

[@ShaizeOn1]

wow, i didnt know about *read_html* , i used *plt.xticks(rotation = 90)* in my code but yours is was cooler

[@beansgoya]

Question - At 4:37 you said adD_subplot(111). What if you wanted a 12x12 at the 5th position?

[@sentdex]

I'd prob use subplot2grid instead. I think you can add commas and get away with it this way tho. I honestly haven't needed a huge group of figures like that in years though

[@osman_gedik]

If I do this: abbv_dict = state_abbv[["State/District"]].to_dict() abbv_dict = abbv_dict["Postal Code"] abbv_dict I get the following error: KeyError: 'Postal Code'

[@jfishing27]

Another awesome tutorial @sentdex just letting you know there is some overlapping on the x axis where 'DC' 'FLSA' 'GU' are. FLSA is too long maybe we could consider dropping that column entirely? if we're really nit picky. Have a great day!

[@sentdex]

You can also rotate the labels 45 or 90 degrees, should solve that :D

[@sery152]

Thanks sentdex! Can you or anyone explain this line: "abbv_dict[c] for c in min_wage_corr.columns"? I can't see any shorhand for this anywhere. how com the abbv_dict[c] before the "for loop" is a correct syntax?

[@jamesdunbar2386]

This is called 'List Comprehensions' and it's a short hand instead of writing out the whole 'for' loop. blog.teamtreehouse.com/python-single-line-loops

[@afbdreds]

2:20, My graph was different, y axis was ploting 1 2 3 4, I guess he kept some sort of index sort from the previous tutorial?

[@Patrick-ig3cn]

I don't know if you're still interested, but it's probably because you left the .head() when you created the min_wage_corr variable

[@alexelliott6283]

Big fan, learning a ton from you. Quick question: your methods (such as .read_csv and _head() and .DataFrame() ) come up in blue for you but just in normal black text in my jupyter notebook... how can I change that?

[@tunaozates]

For some reason, index_col = 1 worked for me instead of index_col = 0. Otherwise I get numbers as dictionary keys.

[@SomeGuyInSandy]

Can attest, coffee actually does taste better out of a sentdex coffee cup!

[@andyn6053]

What does this line do? What is c[:2] ? labels = [c[:2] for c in min_wage_corr.columns]

[@maisha-safari]

Using a list comprehension, the state names are abbreviated and stored in labels. c[:2]

[@KevinTempelx]

Thank you!

[@Bergerons_Review]

Hi. I have a problem getting Jupiter to work with Sublime text 3. Anyone can help me out? I googled it, tried it failed. Please and thank you.

[@sentdex]

Jupyter is it's own thing. You just dl then type jupyter lab in console/terminal

[@pafnutiytheartist]

I don't think that it would be a huge problem to access the website that denies bots in a tutorial as you a not actually causing any harm, but using a different website is a polite thing to do anyway

[@gautamj7450]

Omg!!! I spent the last 8 hours learning the correlation plot, now you post this!! 🤣🤣🤣

[@sentdex]

:]

[@gautamj7450]

@@sentdex Could you pls explain the correlation graph in the upcoming videos, mainly on how it can be used for selecting the right features for machine learning?

[@TheBeastBony]

Hey! I love the video and follow it step by step but for some reason when I make the dictionary for the state labels the keys are just numbers from 0 to 50. How do I get the state names as keys?

[@sentdex]

You probably missed something. Compare your code to the text-based tutorial linked in the description.

[@amyliu2394]

I was having the same issue, was resolved after I opened up the csv file in excel to delete the first column containing numbers and adding in index_col = 0 when reading the csv file.

[@oberoidhruv]

Is that the Not a Flamethrower flamethrower in the background?

[@yousufazad6914]

yeah yeah yeah, Data Science is cool and all but how many weird cup you got?

[@yousufazad6914]

My life is not complete (somewhat)

[@jay-hinddoston8364]

Yes we did it

[@llJoDall]

where the heck did you get all these mugs??? xD

[@michaelshoemaker5635]

A bit confused with the lines "labels = [abbv_dict[c] for c in min_wage_corr.columns]" and "labels = [c[:2] for c in min_wage_corr.columns]" wish there was a bit more explanation with this, but I'll play with it until it makes sense.

[@philinnc]

As one of the partners in the first law firm I worked for used to ask, "But what does it all _mean_?"

[@assida143]

i need a OP for raspberrypi that u know %100 that u can ssH in to it my rasp3b,rasp0w i have 2-3b,5-0w and no vnc,or ssH is working did raspi-config

[@michaelbaumgarn4733]

nice, thank you for this helpful series! Its quite usefull

[@chaitanyasharma6270]

the only problem i have is, i watch your pandas and matplotlib ancd machine learning with scikit learn tutorials all in like a week or 2, but now i feel like i have had way tooo much information feeding into my brain and its alllll a mess now

[@Nypriot]

you have to keep applying your knowledge so that it becomes more like muscle memory

[@samha1513]

Awesome

[@pythonocean7879]

plz do a series on seaborn

[@yashpandey350]

Link of the state abbreviations website: www.infoplease.com/us/postal-information/state-abbreviations-and-state-postal-codes

[@ali51717]

So, basically all your tutorials are like, jump and do it.

[@osman_gedik]

you really have nice cups :D Can you share a pic and show your cups? :D

[@mycreations3452]

Is it jupyter or something else..?

[@sufiyanpatel8106]

Yep. it is jupyter

[@mycreations3452]

@@sufiyanpatel8106 then why it is different from mine

[@sufiyanpatel8106]

I think it depends on version

[@jfishing27]

He's using Jupyter Lab which also can use notebooks within it so its the same thing. You can open it from the Andaconda Navigator, I'm following along using just Jupyter Notebook

[@sentdex]

This is jupyter lab. it's a py module. Like anything else, you may need to update.

[@ianault8599]

Why post so early in the morning? Not good for the algorithm lol.

[@ianault8599]

I know you made a video talking about how you don't care about the algorithm.

[@sentdex]

Then you answered your own question. It's the algorithms job to figure me out. Not the other way around. I've done just fine with that way of thinking. Think I'll continue with it for a bit longer.

[@ahmedhany5037]

Dude it's 6:31 PM in Egypt. Time zones exist !

[@dwx8248]

@@ahmedhany5037 Egypt represent!

[@ahmedhany5037]

@@dwx8248 yeah !

[@259_parthpatidar9]

that is a big ass mug 😂

[@sufiyanpatel8106]

😍

[@sifiso5055]

@_NextElonMusk - Twitter page

[@anandiborade6349]

please go to pythonprogramming.net-->-->Data Analysis it will help you a lot

[@sifiso5055]

First

[@blueskies1254]

Zero

[@sentdex]

Hmm, no idea what this means.