How do I rename columns in a pandas DataFrame?

  Рет қаралды 152,060

Data School

Data School

Күн бұрын

You will often want to rename the columns of a DataFrame so that their names are descriptive, easy to type, and don't contain any spaces. In this video, I'll demonstrate three different strategies for renaming columns so that you can choose the best strategy to fit your particular situation.
SUBSCRIBE to learn data science with Python:
www.youtube.co...
JOIN the "Data School Insiders" community and receive exclusive rewards:
/ dataschool
== RESOURCES ==
GitHub repository for the series: github.com/jus...
"rename" documentation: pandas.pydata.o...
"read_csv" documentation: pandas.pydata.o...
"str.replace" documentation: pandas.pydata.o...
== LET'S CONNECT! ==
Newsletter: www.dataschool...
Twitter: / justmarkham
Facebook: / datascienceschool
LinkedIn: / justmarkham

Пікірлер: 328
@dataschool
@dataschool 7 жыл бұрын
Starting in pandas version 0.21 (released October 2017), you can rename columns using a "mapper" along with the "axis", similar to how you specify which columns you want to drop. Learn more in my latest video, "5 new changes in pandas you need to know about": kzbin.info/www/bejne/qpaYe6WJeLxggrs
@NitinSaseendran
@NitinSaseendran 7 жыл бұрын
Thanks for your amazing videos. It helped me a lot.
@dataschool
@dataschool 7 жыл бұрын
You're very welcome!
@IncredibleGrim
@IncredibleGrim 4 жыл бұрын
2:05 why are names like u'City' instead of just 'City'?
@xyxyirish
@xyxyirish 6 жыл бұрын
You are a good, kind man. One day people are going to write poems about you.
@dataschool
@dataschool 6 жыл бұрын
HA! Thank you :)
@Elias-ce3ze
@Elias-ce3ze 6 жыл бұрын
Hello, I am from Brazil and I'm usually write my codes in Fortran and Matlab. However I've decided to learn python. Your channel is amazing, it is clear and easy to understand and straight to the goal. Congratulations !!!!
@dataschool
@dataschool 6 жыл бұрын
Awesome! Thanks so much for your kind words!
@felipecaannavarro5441
@felipecaannavarro5441 5 жыл бұрын
Opa conterrâneo (=-) Estou começando a aprender programação e estou muito ansioso para fazer aqueles projetos massa com pandas e matplotlib 00 TAMU JUNTO!! BORA FICAR RICO!! kkk
@fet1612
@fet1612 6 жыл бұрын
I found the Data School's videos to be super helpful for beginners or absolute beginners alike. One thing every learner needs to do is PRACTICE ALONG as Kevin of Data School walks them through each ITEM of EACH video. PAUSE and PRACTICE, PAUSE and PRACTICE, PAUSE and PRACTICE. This way you will be surprised how fast this PAUSE and PRACTICE exercise elevates your rate of understanding. I experimented with the THEMATICS employing this PEDAGOGY, and it worked WONDERS. Data School short videos are most helpful and my most favorite. In an hour-long video, your study material might GET BURIED in HEAPs. But here, you can easily LOCATE your MATERIAL. Good luck with anyone learning at DATA SCHOOL, and BECOME DATA SCIENTISTS of the FUTURE.
@dataschool
@dataschool 6 жыл бұрын
Thanks so much for your kind words!
@nikhilkorra5712
@nikhilkorra5712 4 жыл бұрын
During this Quarantine, thought to gain a new skill but fortunately, I got the best tutor. Thank you, Sir.
@mokhadra9289
@mokhadra9289 8 жыл бұрын
That bonus worth the whole 9 minutes :) you are ROCK ... thank you :D
@dataschool
@dataschool 8 жыл бұрын
Awesome! You're welcome!
@yucelozyazgan
@yucelozyazgan 8 жыл бұрын
Simply, he is a great tutor !
@dataschool
@dataschool 8 жыл бұрын
Thanks!
@travelogue.brothers
@travelogue.brothers 2 жыл бұрын
how everything you teach gets stored in the brain library. can import what you have taught anytime. I am a begineeer to data science and the way you teach topi by topic it is so smooth and satisfying to learn. Thanks a lot
@dataschool
@dataschool 2 жыл бұрын
Thanks very much for your kind words!
@СтепанЦыбин-ю9д
@СтепанЦыбин-ю9д 5 жыл бұрын
Great teacher. I could not find an explanation for all this nuance in my own language. And here is a gift. Thank you so much, I am waiting for any of your lessons on the channel.
@dataschool
@dataschool 5 жыл бұрын
Thank you so much! Have you seen my latest pandas tutorial? kzbin.info/www/bejne/moHafZ-AoLh7gtE
@ramlabasharat4334
@ramlabasharat4334 5 жыл бұрын
best tutorials on youtube... your all videos are really informative and easy to understand. I'm watching continuously but didn't get bored. your way of teaching is really awesome
@dataschool
@dataschool 5 жыл бұрын
Thanks so much, I really appreciate it! :)
@AlexanderMoen
@AlexanderMoen 4 жыл бұрын
Thank you, man. I've been searching forever for this seemingly simple thing, and minute 1-3 did it for me. Thanks again.
@dataschool
@dataschool 4 жыл бұрын
Awesome!
@nilewilliams9613
@nilewilliams9613 4 жыл бұрын
Incredible tutor, definitely going to watch the entire series.
@dataschool
@dataschool 4 жыл бұрын
Great to hear!
@shankeybharadwaj1561
@shankeybharadwaj1561 4 жыл бұрын
Awesome video lectures... For the bonus tip , I used ufo.columns = list(map(lambda s:s.replace(' ','_') , ufo.columns))
@fet1612
@fet1612 6 жыл бұрын
DataSchool: Kevin Markham 05:29 Renaming columns in a DataFrame: Kevin: You can actually RENAME the COLUMN while you're reading in the file. Now when I'm reading in the file, here is what I can do to OVERWRITE the COLUMN NAMES. 06:01
@pranitgandhi6832
@pranitgandhi6832 5 жыл бұрын
Where were you?? If I had found you earlier my job would have been so easy. Nevertheless, you have done a world of good to me. Thanks a lot for saving my time on reading this stuff from books.
@dataschool
@dataschool 5 жыл бұрын
That's so kind of you to say! I really appreciate it 😊
@anantgupta305
@anantgupta305 4 жыл бұрын
Bro u are simply awesome, you dont know how much of my time u saved and made everything so simple
@mohamedel-gazzar5818
@mohamedel-gazzar5818 3 жыл бұрын
This dude is the man. You have helped me soooo much you have no idea!
@dataschool
@dataschool 3 жыл бұрын
🙌
@mahnoorafridi9872
@mahnoorafridi9872 4 жыл бұрын
Best tutor I ever had ,you did a great job
@dataschool
@dataschool 4 жыл бұрын
Wow, thanks!
@bhavyakothari635
@bhavyakothari635 6 жыл бұрын
Simple and easy to understand. You are the best !!! :))
@dataschool
@dataschool 6 жыл бұрын
Thanks so much! :)
@syedramishhussain5362
@syedramishhussain5362 4 жыл бұрын
short and to the point, was v helpful! Thank you
@satyamagrawal1191
@satyamagrawal1191 6 жыл бұрын
Your bonus tips are awesome!
@dataschool
@dataschool 6 жыл бұрын
Thanks!
@tanned_cosines_
@tanned_cosines_ 3 жыл бұрын
helped a lot, this playlist; you've provided multiple ways to do a single task thanks a lot
@dataschool
@dataschool 3 жыл бұрын
Glad it was helpful!
@airevancarmen
@airevancarmen 5 жыл бұрын
very simple and complete overview of methods!
@dataschool
@dataschool 5 жыл бұрын
Thanks!
@s.baskaravishnu22
@s.baskaravishnu22 7 жыл бұрын
I very much congratulate you for sharing code used in video with us. Many thanks for that. It is very much useful to me. My warm regards to you.
@dataschool
@dataschool 7 жыл бұрын
Glad to hear that it is useful to you! All of the code from this video series can be found in the following notebook: nbviewer.jupyter.org/github/justmarkham/pandas-videos/blob/master/pandas.ipynb
@nataliaagudelo8635
@nataliaagudelo8635 5 жыл бұрын
All of your videos are amazing, thank you again!
@dataschool
@dataschool 5 жыл бұрын
Thanks!
@sdoken
@sdoken 7 жыл бұрын
Thanks for the cool trick at 8:06 A question about this tip: How could we have known (without you telling us here) that we can call .str. method on ufo.columns here? >>> type(ufo.columns) returns pandas.core.indexes.base.Index object. You showed in another video that we can call .str. on pandas dataframes and linked us to pandas api which shows this under string handling. But I did not know you could run .str. method on ufo.columns because ufo.columns is not a dataframe object. I will appreciate your help!
@dataschool
@dataschool 7 жыл бұрын
I think I just figured out that string methods works on the columns attribute by trying it! I'm not sure if it's explicitly documented.
@mohitseventeen
@mohitseventeen 3 жыл бұрын
How can we use the way explained at 4:40 but in method chaining way.
@lukerengel8533
@lukerengel8533 7 жыл бұрын
When you renamed the columns to all be lower case by first creating a list and then assigning that list to your df.columns object you could also use your tip at the end to do df.columns = df.columns.str.lower() to get the same result in one line of code.
@dataschool
@dataschool 7 жыл бұрын
Great suggestion, thank you!
@superfed2742
@superfed2742 7 жыл бұрын
i have data which has date format as 20170710 and time(in another column) as 115900(11:59:00) and i want to change the format to be 2017-07-10 11:59:00 in the same column. Because this is pandas format if i understand correctly. How to do like i said?
@dataschool
@dataschool 7 жыл бұрын
I think you would need to combine those two columns into the same column, and then define the format when using the 'to_datetime' function. See here for details: pandas.pydata.org/pandas-docs/stable/generated/pandas.to_datetime.html Hope that helps!
@maikeruschannel9002
@maikeruschannel9002 2 жыл бұрын
Great video. Really helpful for beginners
@dataschool
@dataschool 2 жыл бұрын
Thank you!
@TR3NDSETR
@TR3NDSETR 4 жыл бұрын
Fabulous, and on to the next.
@dataschool
@dataschool 4 жыл бұрын
Thank you!
@rainfrog4116
@rainfrog4116 8 жыл бұрын
Another great video! Thank you, Kevin!
@dataschool
@dataschool 8 жыл бұрын
You're very welcome! Thanks for your comment.
@DennisAllard
@DennisAllard 7 жыл бұрын
There are several ways to run Python in a command line REPL (read eval print loop). NOT all of them have the pretty format shown in these videos. And some have annoying screen width limitations and different behavior for TAB, ,shift-TAB, etc. To see the exact same GUI being used in in these videos I use: Anaconda Navigator > Jupyter > New > Notebook: Python3
@dataschool
@dataschool 7 жыл бұрын
Thanks for sharing!
@void2258
@void2258 8 жыл бұрын
I was typing out your bonus question to ask it just as you got to it.
@dataschool
@dataschool 8 жыл бұрын
+Elliot Jenner Ha! Great :)
@joescanlon7502
@joescanlon7502 8 жыл бұрын
ha - i had the exact same query!
@samiagharib3796
@samiagharib3796 4 жыл бұрын
Thanks for the tutorial especially the bonus tip .
@dataschool
@dataschool 4 жыл бұрын
Glad it was helpful!
@jaquesderasmo5496
@jaquesderasmo5496 5 жыл бұрын
You are the best man. Thank you
@dataschool
@dataschool 4 жыл бұрын
Thank you!
@jordiblanco5223
@jordiblanco5223 7 жыл бұрын
Many thanks for the tutorials. Great job!
@dataschool
@dataschool 7 жыл бұрын
You're very welcome!
@irinaelisova1618
@irinaelisova1618 8 жыл бұрын
Thank you so much! Bonus tip was quite useful!
@dataschool
@dataschool 8 жыл бұрын
You're very welcome!
@leojames22
@leojames22 6 жыл бұрын
This is very useful video. Thanks for sharing. Well explained as well.
@dataschool
@dataschool 6 жыл бұрын
You're welcome!
@riteshsingh118
@riteshsingh118 3 жыл бұрын
Love your work ❤️❤️❤️
@dataschool
@dataschool 3 жыл бұрын
Thank you!
@akira_asahi
@akira_asahi 2 жыл бұрын
Thank you for the video. I am grateful for your time and contribution. Kind regards, Akira.
@dataschool
@dataschool 2 жыл бұрын
You're welcome!
@salayhin
@salayhin 8 жыл бұрын
Following line will lowercase the column name and replace space with underscore: ufo.columns = ufo.columns.str.lower().str.replace(' ', '_')
@dataschool
@dataschool 8 жыл бұрын
Indeed it will! The chaining together of multiple string methods is often useful.
@annayakovleva6983
@annayakovleva6983 5 жыл бұрын
Thank you for this! I was just wondering how to save the newly formatted column names to the df!
@rajarshichowdhury8617
@rajarshichowdhury8617 4 жыл бұрын
I have a question whether you can suggest for a python and R course of data science
@robindong3802
@robindong3802 7 жыл бұрын
Very detailed lesson. thank you!
@dataschool
@dataschool 7 жыл бұрын
You're welcome!
@goldensleeves
@goldensleeves 4 жыл бұрын
Why do your column names get that "u" next to them????? Like at 8:34 . Thanks!
@ChartExplorers
@ChartExplorers 4 жыл бұрын
The "u" in front of the string values indicates that it is a Unicode string instead of ASCII. This will happen when using Python 2. Python 3 strings are Unicode by default so you will not see this.
@dataschool
@dataschool 4 жыл бұрын
Exactly!
@jongcheulkim7284
@jongcheulkim7284 3 жыл бұрын
Thank you so much. This is very helpful.
@dataschool
@dataschool 3 жыл бұрын
Great to hear! 🙏
@AshishSahu25
@AshishSahu25 7 жыл бұрын
Thanks for the lucid explanation and bonus tips. I'm wondering using your last bonus tip code snippet, how would I replace or rename few columns' white spaces with underscore, without applying it on all.
@dataschool
@dataschool 7 жыл бұрын
Glad you liked the video! Regarding your question, my methodology would depend on the exact circumstances. There's not one obvious way to do what you are describing... sorry!
@ScholarStream_25
@ScholarStream_25 4 жыл бұрын
Brilliant 😉🔥. But can anyone tell me how did he get the URL from??
@lucasmoraes5488
@lucasmoraes5488 4 жыл бұрын
From his GitHub repository. If you click on raw in some file you'll be redirected to a page
@cutedevil069
@cutedevil069 6 жыл бұрын
Best Tutorial ever
@dataschool
@dataschool 6 жыл бұрын
Thanks!
@isaiaspereira2796
@isaiaspereira2796 4 жыл бұрын
Dude, you are awesome!!!
@dataschool
@dataschool 4 жыл бұрын
Thank you!
@jingchu4629
@jingchu4629 6 жыл бұрын
Thanks for the awesome tutorial!
@dataschool
@dataschool 6 жыл бұрын
You're welcome!
@ankhts
@ankhts 4 жыл бұрын
Thank you for all the videos.It provides almost all fundamental operations that can be done on Data frames. What understood from this video is that df.columns is used as series along the row 0.Can we do similar thing along column 0 (i.e. index column ) and apply similar operations ? Many thanks
@szamoransky
@szamoransky 11 ай бұрын
Does str.replace also have an inplace argument?
@yandongbi1800
@yandongbi1800 7 жыл бұрын
Very grateful for your perfect contribution. I want to ask you what "header = 0" means.Look forward to your reply.
@dataschool
@dataschool 7 жыл бұрын
header=0 means that the first row (row zero) is contains the header information.
@assafcohen3173
@assafcohen3173 3 жыл бұрын
I was looking for a way to easily replace column names using regex and got lost in an endless stack overflow loop - thanks for providing an easy solution.
@dataschool
@dataschool 2 жыл бұрын
You're welcome!
@ramakanthrayanchi8888
@ramakanthrayanchi8888 8 жыл бұрын
Excellent video. Cheers :) :)
@dataschool
@dataschool 8 жыл бұрын
Thanks! :)
@sdoken
@sdoken 8 жыл бұрын
If I have column labels that are of time datetime, and I want to convert them to strings, how can I do that? Without using a loop. (I already figured out how to do it with a loop by using the rename function at each iteration) Thanks!
@dataschool
@dataschool 8 жыл бұрын
This should work: df.column_name.astype(str)
@IBRAHIMDEMIREZ
@IBRAHIMDEMIREZ 2 жыл бұрын
Hi Kevin! Your videos are really helpful and very informative, my question is how am i suppose to rename column name on the base of condition? I mean df.columns = df.iloc[16] . row 16 has some NaN values so only for these case I want pd take value from row 17. Thank you!
@phuccoiinkorea3341
@phuccoiinkorea3341 8 жыл бұрын
Easy to a person bad English like me! Thank so much!
@dataschool
@dataschool 8 жыл бұрын
You're very welcome!
@chd9841
@chd9841 7 жыл бұрын
some of my variables have spaces and some have dots. I want to replace the spaces dots by underscore how can I do that in one code?
@dataschool
@dataschool 7 жыл бұрын
You could chain two str.replace operations together in the same line.
@alexandermuir8160
@alexandermuir8160 5 жыл бұрын
Excellent information provided. Qn: Can you use more than one DataFrame simultaneously? If so what is the protocol so the program does not confused as to which one it is using?
@dataschool
@dataschool 4 жыл бұрын
You reference DataFrames by name, that's all. Glad you liked the video!
@gokulkannan788
@gokulkannan788 4 жыл бұрын
How do we make our DataFrame or table representation in our jupyter notebook look like yours - with borders? :) We get tables without any borders only.
@ChartExplorers
@ChartExplorers 4 жыл бұрын
Sometimes when you use print() the table will not have borders. You can also put this into any cell and you should get borders. %%HTML table.dataframe td, table.dataframe th { border: 1px black solid !important; color: black !important; }
@sameeral-janabi5966
@sameeral-janabi5966 4 жыл бұрын
Thank you .... excellent information.
@rhettsmedia
@rhettsmedia 4 жыл бұрын
Hi there this is the right place to do this when you paste in a line from your snippets please read the entire line character by character if possible for those of us with visual impairment we do not look at the screen but we must hear all the characters the same as with your URL secure paste in it was very hard to find that this was the I T.LY because we can’t really see the tail the Y from the eye thank you
@juanmoctezuma9225
@juanmoctezuma9225 3 жыл бұрын
awesome tutorial thanks!
@terrancedejesus625
@terrancedejesus625 8 жыл бұрын
Great video! I would like to know how I can look for certain values in each row or column and then replace those values with a specific value. An example would be, for each cell in a dataframe where the Shape Reported is TRIANGLE and the State is NY, change the cell value in the City column to Buffalo. I hope that makes sense. Thank you!
@dataschool
@dataschool 8 жыл бұрын
You can definitely do that. Try watching these two videos (in order), and see if you are able to figure out: kzbin.info/www/bejne/rqfTf3Rtl6hrmdU kzbin.info/www/bejne/aoOXiKZ6f5J-jMU Let me know if that helps! :)
@keerthiallam830
@keerthiallam830 7 жыл бұрын
Thanks for the video! While replacing the column names with .rename method, I am getting 'TypeError: 'set' object is not callable'. Could you please explain this?
@dataschool
@dataschool 7 жыл бұрын
I'm pretty sure you have a typo... could you check that you are typing the code exactly the same way?
@nitishrele007
@nitishrele007 6 жыл бұрын
Why the name of the columns are of the form u'var_name'? (Name of the columns are preceded by the letter u and enclosed in a quote)
@dataschool
@dataschool 6 жыл бұрын
Python 2 puts a u in front of Unicode strings, whereas Python 3 does not.
@sdoken
@sdoken 8 жыл бұрын
Is that .str() method at the end of the video (bonus tip) a method of pandas Series or is it a method of string type in python? I am just curious about this. Thanks for the video.
@dataschool
@dataschool 8 жыл бұрын
You're welcome! 'str' is a namespace where pandas string methods are stored, and 'replace' is a Series method. It's designed to work similar to a Python string method, but it is actually implemented in pandas code. More information about string methods is here: kzbin.info/www/bejne/mKDJknZmfsieftE Hope that helps!
@vivektiwari3459
@vivektiwari3459 8 жыл бұрын
Can we create a transition table and use the translate method instead of replace?
@dataschool
@dataschool 8 жыл бұрын
I've never used the translate method in pandas, but it looks like translate is for character-by-character translation, so it wouldn't be appropriate in this case.
@vivektiwari3459
@vivektiwari3459 8 жыл бұрын
I tried using translate method on a data set to replace ' ' with '_' and '-' with '_' and it worked in a single line of code. Though you are right it would not be appropriate in this case and not likely to be of significant use in any other case, I guess. Thanks for the video series.
@dataschool
@dataschool 8 жыл бұрын
You're welcome!
@merumomo
@merumomo 8 жыл бұрын
Hi! I got a question. How do you undo the changes? How do you change the column names back to what they were before.
@dataschool
@dataschool 8 жыл бұрын
If you use the "inplace=True" parameter, the changes are permanent, and there is no undo. If you use "inplace=False" (the default), then the change is only temporary. I have a video about inplace coming out on Tuesday!
@merumomo
@merumomo 8 жыл бұрын
+Data School ok. cool. thank you!
@IncredibleGrim
@IncredibleGrim 4 жыл бұрын
2:05 why are names like u'City' instead of just 'City'
@dataschool
@dataschool 4 жыл бұрын
It stands for 'u', meaning a Unicode string. That's how Unicode strings were displayed in Python 2.
@vaibhavdesai6998
@vaibhavdesai6998 5 жыл бұрын
How do we change data types of multiple columns from object to integer in a pandas dataframe? Do you have a video on this? If not could you please make one. That would be really helpful.
@dataschool
@dataschool 5 жыл бұрын
Great question! This video includes the answer: kzbin.info/www/bejne/Y3_Fimp7bs1-rs0m45s
@liaqatali1415
@liaqatali1415 6 жыл бұрын
hello can you kindly help me, whatever text editor i use as pycharm, anaconda.. it doesn't shows full table.. it only shows first and last column and other in centers shows in dots.. i am looking in your videos you are getting a proper table.. i dont get full table
@liaqatali1415
@liaqatali1415 6 жыл бұрын
its shows like this City Colors Reported ... State Time 0 Ithaca NaN ... NY 6/1/1930 22:00 1 Willingboro NaN ... NJ 6/30/1930 20:00 2 Holyoke NaN ... CO 2/15/1931 14:00 3 Abilene NaN ... KS 6/1/1931 13:00 4 New York Worlds Fair NaN ... NY 4/18/1933 19:00
@dataschool
@dataschool 6 жыл бұрын
This video might be helpful to you: kzbin.info/www/bejne/a5C0iYCtmt-DgKc
@swagatmishra9350
@swagatmishra9350 5 жыл бұрын
hey..what is the difference btw header= None(as stated by you in the previous v d o) and header=0?
@dataschool
@dataschool 4 жыл бұрын
I believe header=None means "there is no header" and header=0 means "the first row is the header", but I'd have to look it up to be sure.
@hightimeframe
@hightimeframe 8 жыл бұрын
super, clear and helpful
@dataschool
@dataschool 8 жыл бұрын
Great! Thanks for your kind words.
@AsifMarazi
@AsifMarazi 5 жыл бұрын
Hello... I created a dataframe (Particular month data) from a CSV file I want to add column names or labels. Can I do that...Or I have to write a separate CSV file then add the column names in excel and come back to python...Thats what I am doing right now....Any kind of help will be highly appreciated....I am a absolute beginner....
@dataschool
@dataschool 5 жыл бұрын
You can just overwrite the columns attribute with the list of column names: df.columns = ['col1', 'col2', 'col3'] Does that help?
@subashkattel7033
@subashkattel7033 7 жыл бұрын
Kevin, you are doing great job with nice tutorials. Thanks. I have a question, how can we assign name to the series rather than renaming? Say I have 100 columns in a csv file with common index and header are empty(no names). Now I want to assign name to each column like P1, P2, ......P100. Can I do that?
@dataschool
@dataschool 7 жыл бұрын
The columns attribute of a DataFrame always has values, it is never empty. If you want to change those values, you could use a string method. Hope that helps!
@varunupadhyay3836
@varunupadhyay3836 8 жыл бұрын
Hi Kevin, Does header=0 and header=None means the same as they both are producing the same results?
@dataschool
@dataschool 8 жыл бұрын
Actually, header=0 means that row 0 of the file is the header row, whereas header=None means that there is no header row in the file. Does that help?
@varunupadhyay3836
@varunupadhyay3836 8 жыл бұрын
Yes. Sorts things out. Thanks
@dishonfano7599
@dishonfano7599 5 жыл бұрын
Great job man...
@dataschool
@dataschool 5 жыл бұрын
Thanks!
@dharmendra811
@dharmendra811 7 жыл бұрын
nice to have these videos, helping me alot...would you please make on video to add existing dataframe to predefined worksheet.
@dataschool
@dataschool 7 жыл бұрын
Do you mean an Excel worksheet?
@dharmendra811
@dharmendra811 7 жыл бұрын
Yes.
@dataschool
@dataschool 7 жыл бұрын
Great, thanks for the suggestion!
@valeriapir1268
@valeriapir1268 6 жыл бұрын
Your tutorials are just great! Thank you for that!! One question: Is it possible to change the cells depending on its content? I mean, if we see "ABC12" in cell that should be renamed with "Sample"? Thank you in advance! :)
@dataschool
@dataschool 6 жыл бұрын
You could use the map or replace methods to do that, I think!
@valeriapir1268
@valeriapir1268 6 жыл бұрын
Thanks to your tutorials i start my journey with Python language!
@govindrao6479
@govindrao6479 4 жыл бұрын
hi, am a beginer in Padas. Why does your output of ufo.columns has prefix by 'u' whereas when I tried there is no prefix
@ChartExplorers
@ChartExplorers 4 жыл бұрын
The "u" in front of the string values indicates that it is a Unicode string instead of ASCII. This will happen when using Python 2. Python 3 strings are Unicode by default so you will not see this.
@dataschool
@dataschool 4 жыл бұрын
Exactly!
@abhilash8492
@abhilash8492 7 жыл бұрын
type(ufo_cols) is list whereas type(ufo.columns) is pandas.core.indexes.numeric.Int64Index. How come we can equate them?what does int64index mean?
@dataschool
@dataschool 7 жыл бұрын
ufo_cols is a Python list, whereas ufo.columns is a list-like object called an index. They are just different data structures - list comes from Python, whereas index comes from pandas. I recommend watching this video to learn more about the index: kzbin.info/www/bejne/hYq9f55tj56mbKs
@ish694
@ish694 7 жыл бұрын
Hey man ! Loving this series. Just one question. How do you remember all these commands and functions and where to put where ? i am new to coding and so i keep forgetting what the commands were. :/
@dataschool
@dataschool 7 жыл бұрын
I hear you! It's easy to forget when you are getting started. I just remember them more and more over time! Plus, I make cheat sheets for myself like this: nbviewer.jupyter.org/github/justmarkham/pandas-videos/blob/master/pandas.ipynb
@ish694
@ish694 7 жыл бұрын
Thanks a lot :D I think these will do the work
@Mufasa1482
@Mufasa1482 7 жыл бұрын
your first method for renaming columns was "df.rename.columns=". when you used "ufo.rename" you included the argument "inplace=True" to make the change permanent. however, you also presented some other methods for doing the same thing, like defining a list and then applying it by "ufo.columns=list" but this time you did not need to specify "inplace=True". why not?
@dataschool
@dataschool 7 жыл бұрын
Great question! When you are overwriting an attribute, that will always be immediate and permanent. But when using a method such as rename, that will only be permanent when you use the inplace parameter. Does that help?
@davidlang32
@davidlang32 5 жыл бұрын
How would you suggest dynamically rename column headers while looping through multiple files? The files are server resource data for CPU. There's over 100 servers which are being read into a Data Frame, one at a time. The script merges data based on the time stamp and only adds the average and 95 percentile of each file to a new Data Frame. That data frame continues to get updated with each file until it has all the average and 95th percentile data from all files. This issue is that since each file has the same format the columns names are all the same. This doesn't differentiate from each of the servers. Through the process of merging the data into one master file I lose the ability to see which column belongs to which file. How would you tackle this issue?My current method: I used your method in this video to rename all the columns prior to completing the final save. It just writes all the file names as they should appear. The data is very structured so this works. However, this would not work in an environment where the data naming patterns were not as structured. Thanks for any insight you maybe able to provide.
@dataschool
@dataschool 4 жыл бұрын
Great question! There's no obvious answer here, so I'd have to see more of the specifics to advise. Sorry I can't help!
@salq6585
@salq6585 6 жыл бұрын
How to create data structure for my dataset? Note : I opened the data in the same way you did it but I want to help creating the data structure like dictionary. Thanks
@dataschool
@dataschool 6 жыл бұрын
I'm sorry, I don't understand your question. Could you clarify? Thanks!
@abhishekp9423
@abhishekp9423 7 жыл бұрын
Hi Kevin, Thanks for the awesome videos on Panda . In the second method to rename columns , we have to be cautious about the sequence in which the original name of the column has.Right??
@dataschool
@dataschool 7 жыл бұрын
Right!
@abhishekp9423
@abhishekp9423 7 жыл бұрын
Thanks Sir
@freeschool_
@freeschool_ 4 жыл бұрын
let us suppose i have hundred columns and i have to remove all the columns from my dataset which have null values more than half of its records. How to do it? I don't want to check each column and do it drop it one by one. Please help
@dataschool
@dataschool 4 жыл бұрын
I think this will work: df.dropna(thresh=len(df)*0.5, axis='columns')
@freeschool_
@freeschool_ 4 жыл бұрын
@@dataschool It worked plus i found few other techniques to do that after watching other videos of you. Thanks a lot!!!
@dataschool
@dataschool 4 жыл бұрын
You're welcome!
@komaljaswani7497
@komaljaswani7497 4 жыл бұрын
awesome video as usual :D
@dataschool
@dataschool 4 жыл бұрын
Thanks again!
@ChandraShekhar-ds8cp
@ChandraShekhar-ds8cp 4 жыл бұрын
Hey! How can we rename column names using assign function or replace function? Is that even possible? Thanks in Adv
@ChartExplorers
@ChartExplorers 4 жыл бұрын
df.columns = df.columns.str.replace('what you are replacing', 'what you want it replaced with') See 3:25 kzbin.info/www/bejne/bJatnYWMrsx4g80
@kev5187
@kev5187 6 жыл бұрын
Hi what if the dataframe doesn't have a name in columns how can I give names for the first time ??? Thanks
@dataschool
@dataschool 6 жыл бұрын
You can overwrite the columns attribute.
@hollyzhang6491
@hollyzhang6491 4 жыл бұрын
thanks for these wonderful videos. but here is a error I met when I run the http code:ConnectionResetError: [Errno 54] Connection reset by peer. Do you have any solutions? thanks anyway.
@dataschool
@dataschool 4 жыл бұрын
Sorry, I'm not sure!
@rohiniik
@rohiniik 3 жыл бұрын
can u plz tell me how to anonymize data
@samratchoudhury2797
@samratchoudhury2797 7 жыл бұрын
Hi, Could you tell me how can I import a binary excel file in python. excel file with extension .xlsb
@dataschool
@dataschool 7 жыл бұрын
I don't know if that's possible, I'm sorry!
@alanjoseph3190
@alanjoseph3190 5 жыл бұрын
Sir u r a very great teacher.Thanks for your class.But i have one doubt.eg: a=pd.Dataframe(['a','b',]) we get a single column.but if it is a=pd.Dataframe([['a','b']]) we get a single row.the difference only is a extra [ ].i cant understand that
@dataschool
@dataschool 5 жыл бұрын
Great question! The DataFrame constructor converts a Python list (your first example) to a Series. It converts a list of lists (your second example) to a DataFrame, but you only have one "inner" list so there's only one row. Try this and maybe it will make more sense: pd.DataFrame([['a','b'], ['c','d']]) Hope that helps!
@xiao2634
@xiao2634 7 жыл бұрын
I don't get it. The type of ufo.columns is pandas.core.indexes.base.Index. I could not find this in pandas documentation. Is it a class? Where could I find the documentation of all the methods like str that I can use?
@dataschool
@dataschool 7 жыл бұрын
I'm actually not sure that there is documentation of everything you can do to a pandas index! I'm sorry, I just don't know.
@xiao2634
@xiao2634 7 жыл бұрын
Thank you for your reply. I checked the source code of pandas. It is complex. pandas.core.indexes.base.Index is a class that extended several classes, one of which is StringAccessorMixin. StringAccessorMixin use another class AccessorProperty to vectorized all the methods into 'str'. All the str methods can be found at: github.com/pandas-dev/pandas/blob/master/pandas/core/strings.py
@dataschool
@dataschool 7 жыл бұрын
Thanks for researching this!
@mdmantejsingh
@mdmantejsingh 8 жыл бұрын
Hi Kevin, for me ufo.head() is not working. I am using Python 3.5.2 default editor. After saving everything when i press F5.. nothings happens. No output
@mdmantejsingh
@mdmantejsingh 8 жыл бұрын
I can use ipy notebook, but i want it in my default Pythons shell editor
@dataschool
@dataschool 8 жыл бұрын
If you're running a script, rather than running commands one by one in interactive mode, you might not see the output. I'd encourage you to run the lines of code one by one, since that tends to be most effective for data exploration.
@mdmantejsingh
@mdmantejsingh 8 жыл бұрын
Data School Thank you, later that day i installed ipy notebook. Your videos helped me a lot. Thank you so much.
@dataschool
@dataschool 8 жыл бұрын
You're very welcome!
@swethavenkataramaniyer89
@swethavenkataramaniyer89 6 жыл бұрын
How do I rename the data in a particular column in Python. There can be one or more records to be renamed.
@dataschool
@dataschool 6 жыл бұрын
You would filter and then assign... this video might help: kzbin.info/www/bejne/aHKpeIOag9NnfK8
How do I remove columns from a pandas DataFrame?
6:36
Data School
Рет қаралды 138 М.
How do I filter rows of a pandas DataFrame by column value?
13:45
Data School
Рет қаралды 280 М.
24 Часа в БОУЛИНГЕ !
27:03
A4
Рет қаралды 7 МЛН
번쩍번쩍 거리는 입
0:32
승비니 Seungbini
Рет қаралды 182 МЛН
УНО Реверс в Амонг Ас : игра на выбывание
0:19
Фани Хани
Рет қаралды 1,3 МЛН
How do I select multiple rows and columns from a pandas DataFrame?
21:47
Learning Pandas for Data Analysis? Start Here.
22:50
Rob Mulla
Рет қаралды 127 М.
How do I make my pandas DataFrame smaller and faster?
19:06
Data School
Рет қаралды 67 М.
25 Nooby Pandas Coding Mistakes You Should NEVER make.
11:30
Rob Mulla
Рет қаралды 279 М.
How do I merge DataFrames in pandas?
21:49
Data School
Рет қаралды 159 М.
21 more pandas tricks
24:40
Data School
Рет қаралды 48 М.
How do I avoid a SettingWithCopyWarning in pandas?
13:30
Data School
Рет қаралды 45 М.
24 Часа в БОУЛИНГЕ !
27:03
A4
Рет қаралды 7 МЛН