No video

How do I remove columns from a pandas DataFrame?

  Рет қаралды 137,534

Data School

Data School

8 жыл бұрын

If you have DataFrame columns that you're never going to use, you may want to remove them entirely in order to focus on the columns that you do use. In this video, I'll show you how to remove columns (and rows), and will briefly explain the meaning of the "axis" and "inplace" parameters.
SUBSCRIBE to learn data science with Python:
www.youtube.co...
JOIN the "Data School Insiders" community and receive exclusive rewards:
/ dataschool
== RESOURCES ==
GitHub repository for the series: github.com/jus...
"drop" documentation: pandas.pydata.o...
== LET'S CONNECT! ==
Newsletter: www.dataschool...
Twitter: / justmarkham
Facebook: / datascienceschool
LinkedIn: / justmarkham

Пікірлер: 252
@dataschool
@dataschool 6 жыл бұрын
Starting in pandas version 0.21 (released October 2017), you can now drop rows and columns using the 'index' or 'columns' keywords, which is slightly simpler than the method outlined in the video! Learn more in this video: kzbin.info/www/bejne/qpaYe6WJeLxggrs
@osenihabeeb5498
@osenihabeeb5498 5 жыл бұрын
How do you specify a range of columns that you want to drop instead of dropping by explicitly stating them
@stopPlannedObsolescence
@stopPlannedObsolescence 4 жыл бұрын
ty
@stopPlannedObsolescence
@stopPlannedObsolescence 4 жыл бұрын
@@osenihabeeb5498 maybe with slice method
@vivian_rchen2618
@vivian_rchen2618 4 жыл бұрын
​@@osenihabeeb5498 I use list comprehension, like this : ufo.drop([x for x in range(1,5)],axis = 0, inplace = True) # drop index 1,2,3,4 but I'm not sure if there's a better way😅
@ghania8938
@ghania8938 4 жыл бұрын
kindly make a video on how to undo or redo dropped columns or rows in pandas
@solme8577
@solme8577 3 жыл бұрын
This is the first video that I watch from u and 100% sure won't be the last. THANK U
@dataschool
@dataschool 3 жыл бұрын
You're very welcome!
@sherlockom22
@sherlockom22 3 жыл бұрын
hey i got to know that we can also use : del(ufo['City']) for example , to remove city column. this will work to delete a single column only.
@goldensleeves
@goldensleeves 3 жыл бұрын
Love your videos! QUESTION: Do you have a video that explains when it's a good idea to drop certain columns? For example, how do we determine if a column should be dropped before using it as a feature to predict a target in machine learning problems? Which columns are worth keeping and which ones are "noisy" or unhelpful? Thanks so much for all you do in the world of teaching!
@erfannazari6110
@erfannazari6110 4 жыл бұрын
Amazing Teacher! Every tutorial has something that i would do lots of searching to find it out! Thank you Kevin!
@dataschool
@dataschool 4 жыл бұрын
You're very welcome!
@mohnasnassr9816
@mohnasnassr9816 4 жыл бұрын
we can use also ufo.drop(list(range(1,5)), axis=0, inplace=True) to drop more rows
@abhishek-hb1vg
@abhishek-hb1vg 5 жыл бұрын
Your end tips are always the best.
@dataschool
@dataschool 5 жыл бұрын
Thanks!
@afrozaaktermousumi4535
@afrozaaktermousumi4535 3 жыл бұрын
Sir, just excellent, we want your videos on pandas on more difficult topics
@dataschool
@dataschool 3 жыл бұрын
Thanks! You might be interested in my DataCamp course: www.datacamp.com/courses/analyzing-police-activity-with-pandas?tap_a=5644-dce66f&tap_s=280411-a25fc8
@sarikakazi6793
@sarikakazi6793 3 жыл бұрын
Very well explained! The Data analysis in Python series is so useful!!!
@dataschool
@dataschool 3 жыл бұрын
Thank you! 🙏
@revolution77N
@revolution77N 7 жыл бұрын
Thank you for all your efforts. Your videos are really organized and the material is thoroughly explained. It's very noble of you.
@dataschool
@dataschool 7 жыл бұрын
You're very welcome! Thank you so much for your kind words!
@9zQx86LT
@9zQx86LT 7 жыл бұрын
really ! videos are very well orchestrated.
@dataschool
@dataschool 7 жыл бұрын
Thanks!
@amandal8170
@amandal8170 4 жыл бұрын
You youtube always clear and with structures. This is is handy. Waiting you to have more videos - like data Wrangling + Modeling.
@rogersteven4989
@rogersteven4989 Жыл бұрын
Sir Kevin Markham, this is indeed very helpful
@dataschool
@dataschool Жыл бұрын
Glad to hear that!
@andrenevares7543
@andrenevares7543 2 жыл бұрын
Such a nice explanation. I think one of the best explanations on KZbin so far. Nice Job!
@dataschool
@dataschool 2 жыл бұрын
Wow, thanks!
@anisity-n9q
@anisity-n9q 7 жыл бұрын
I didn't quite understand what 'inplace=True' means from this video. But thanks anyways! I like this series a lot and will keep watching it.
@dataschool
@dataschool 7 жыл бұрын
I have a video that explains the inplace parameter: kzbin.info/www/bejne/jpKmhJeobNWDo7s Hope that helps!
@Nurlan_Turganov
@Nurlan_Turganov 3 жыл бұрын
I am very glad that you made these videos! Thank you!
@dataschool
@dataschool 3 жыл бұрын
My pleasure!
@dianavallejo7268
@dianavallejo7268 5 жыл бұрын
you are the best explaining python!!!!
@dataschool
@dataschool 5 жыл бұрын
Thanks!
@spicytuna08
@spicytuna08 5 жыл бұрын
you are a talented instructor.
@dataschool
@dataschool 5 жыл бұрын
Thanks!
@gargikulkarni42
@gargikulkarni42 2 жыл бұрын
Thank you so much for these amazing videos!
@dataschool
@dataschool 2 жыл бұрын
You're very welcome!
@dianavillescas
@dianavillescas 3 жыл бұрын
My favourite teacher!!!
@dataschool
@dataschool 3 жыл бұрын
Thank you!
@victorbrylew1775
@victorbrylew1775 6 жыл бұрын
Thanks for video. It is also possible to delete columns by "del ufo['Time']" type of command. What are advantages and disadvantages of del vs drop?
@dataschool
@dataschool 6 жыл бұрын
del is a Python statement, whereas drop is a pandas method. When working with pandas objects, it's best to use pandas methods and functions whenever possible. Great question!
@VishalSharma-rn7mt
@VishalSharma-rn7mt 4 жыл бұрын
Thanks buddy for this very clear video.
@dataschool
@dataschool 4 жыл бұрын
You're welcome!
@asneogy
@asneogy 8 жыл бұрын
For a single column it is also possible to use the 'del' command. Eg, del df['col_name']. However this is not the recommended way and .drop is preferrable. I like to use only one way to do things, as long as it is the most generic / flexible way.
@dataschool
@dataschool 8 жыл бұрын
I agree. pandas is so flexible that there are sometimes too many ways to do the same thing... it's best to focus on a single approach.
@goodmorninghumans3647
@goodmorninghumans3647 6 жыл бұрын
Good stuff! I do have a question, but I'm sure I can find the answer...it does seem to be a difficult one to hunt up though. I am importing data for use in training deep learning trading strategies, and the timestamp is in unix. I am having difficulty converting those into datetime. I am sure by the time you respond, if you do...I will have solved it, but it might make for a helpful tutorial, since I have seen I am not the only one with this difficulty. But, I would also like to say that this is an excellent series of tutorials. Very well done. You are a good instructor. Thanks for the great lessons! +
@dataschool
@dataschool 6 жыл бұрын
Thanks for your kind words! Search for "unix epoch time" on this page and I think that's your answer: pandas.pydata.org/pandas-docs/stable/generated/pandas.to_datetime.html
@myravat
@myravat 6 жыл бұрын
Thanks for posting the videos. Very easy to understand.Keep up the good work
@dataschool
@dataschool 6 жыл бұрын
You're welcome! :)
@elilavi7514
@elilavi7514 8 жыл бұрын
Thanks for informative video ! While watching your video I thought about several questions : 1. What about reading from csv file only two columns and ignore others ? 2.what is fastest method to read from csv file ? 3. How to define type of each column before actually reading the csv. Thanks !!
@dataschool
@dataschool 8 жыл бұрын
+Eli Lavi Great questions! I'm not sure what you mean by number 2, but here is an example of both number 1 and number 3: ufo = pd.read_csv('bit.ly/uforeports', usecols=['City', 'State'], dtype={'State':str}) The 'usecols' parameter tells pandas to only read those two columns, and the 'dtype' parameter tells pandas to make the 'State' column a string. (That's unnecessary in this case, but the point is that you could convert the data type of the column this way.) Does that help?
@elilavi7514
@elilavi7514 8 жыл бұрын
+Data School Thanks ! It help . About second question , I've heard about other methods to load csv files to pandas , beside the read_csv method . Another question , how to load a very big file to pandas , file contains 100G - 150G of data ?
@dataschool
@dataschool 8 жыл бұрын
+Eli Lavi I don't know what method you would use to read a CSV file into pandas other than read_csv. Regarding a large file, here's how you could read it into pandas: pandas.pydata.org/pandas-docs/stable/io.html#io-chunking
@dataschool
@dataschool 8 жыл бұрын
+Eli Lavi I featured two of your questions in my latest video! :) kzbin.info/www/bejne/eF7VaomrgJ1jms0
@soukainaguaoua6875
@soukainaguaoua6875 4 ай бұрын
Thankyou for this video, it's very clear. But I have a question please, how can we remove differents columns in differents positions or index, it may be in the beginnig, in the middle or at the end of the dataframe? Thanks in advance
@anAlokDubey
@anAlokDubey 2 жыл бұрын
Old video but still very useful 😊
@dataschool
@dataschool 2 жыл бұрын
Thank you!
@robertorolo
@robertorolo 6 жыл бұрын
Your classes are awesome! Thank you very much. It is veery useful.
@dataschool
@dataschool 6 жыл бұрын
Thanks!
@kennyl7542
@kennyl7542 8 жыл бұрын
thanks a lot for this wonderful short video! very helpful!
@dataschool
@dataschool 8 жыл бұрын
+Kenny L You're very welcome! You can check out the entire video series here: kzbin.info/aero/PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y
@FabioRBelotto
@FabioRBelotto 2 жыл бұрын
What if I want to exclude all rows that match a certain criteria, for example, all oval shapes?
@ahmedtalha1952
@ahmedtalha1952 4 ай бұрын
what is the code for removing the row in range from 0 to 50 ?
@mannurajput7969
@mannurajput7969 Жыл бұрын
"['Colors Reported'] not found in axis" i followed all the steps properly, still i get this error. what can be the issue ?
@greenishengland
@greenishengland 4 жыл бұрын
How to drop the float value in any row of the object column? could you please help me
@juanguirao
@juanguirao 2 жыл бұрын
This is very useful. Thank you Kevin :)
@dataschool
@dataschool 2 жыл бұрын
You're welcome!
@dhruvsharma1611
@dhruvsharma1611 4 жыл бұрын
Hi ,the tips are very useful.I have a doubt instead of passing columns names or rows.I want to pass a boolean expression like , if i want to drop values less than 2 in dataframe ?
@leythecg
@leythecg 4 жыл бұрын
A really great series!
@dataschool
@dataschool 3 жыл бұрын
Thanks!
@KhalilYasser
@KhalilYasser 3 жыл бұрын
Thank you very much. At the end of the video after removing the first two rows, now the index begins with 2. Is it possible to reindex the dataframe?
@008abc1
@008abc1 5 жыл бұрын
I'm having a problem in removing unnamed columns from my data set. The drop method shown here does not work, instead the error shows "Unnamed: 0" not found in axis. Please help
@dataschool
@dataschool 5 жыл бұрын
I'd have to see the DataFrame in order to help, I'm sorry!
@shripadsg
@shripadsg 4 жыл бұрын
print('Even I can understand') print('You are brilliant teacher!') great job Sir. Do you have something similar to numpy? thanks in advance!
@umar19960910
@umar19960910 5 жыл бұрын
if suppose you have multiple columns say 40+ columns and you are suppose to keep only specified columns like Name,Age,Address,Sex,DOB,Salary and delete or drop rest of it how do you do that ???
@dataschool
@dataschool 5 жыл бұрын
df[['Name', 'Age', 'Address',...]]
@vimultivi
@vimultivi 7 жыл бұрын
Thank you! please provide us tutorials in plotting different types of graphs with the data .
@dataschool
@dataschool 7 жыл бұрын
Thanks for your suggestion, I'll definitely consider it for the future!
@surajnaik6393
@surajnaik6393 4 жыл бұрын
Hey how I can apply multiprocessing while reading huge data frame
@user-nz8sd5kb2m
@user-nz8sd5kb2m 7 жыл бұрын
thanks, when I study with this vedio I feel the time is slowly
@kevennguyen3507
@kevennguyen3507 Жыл бұрын
How would you remove rows of data that are in ALL CAPS? I have a table columns A, B, C with 100 rows. The rows of texts for column A has several rows that has ALL CAPS which are irrelevant data autogenerated. How can I remove those rows that has all caps from the table? Love your videos and instructions! You're awesome!
@dataschool
@dataschool Жыл бұрын
Great question! You would have to filter using a string method. Thanks for your kind words!
@ningwang5639
@ningwang5639 6 жыл бұрын
great video! a quick question, what's the IDE you are using in the video? looks like a web based? how to get it?
@GiuseppeSerraonline
@GiuseppeSerraonline 6 жыл бұрын
Hi you can get that IDE from jupyter.org/, or downloading the all distribution from www.anaconda.com/download/
@amruthab.s706
@amruthab.s706 2 жыл бұрын
What is the difference between array, series, and dataFrame? why sometimes do we need to use series.values method to convert the series to array?
@dataschool
@dataschool 2 жыл бұрын
Great question! A Series is a 1-dimensional pandas object, whereas a DataFrame is a 2-dimensional pandas object. You can think of a DataFrame as a table and a Series as a column in that table. An array is a NumPy object, and both the Series and DataFrame objects are actually arrays "under the hood". It's rare that you are required to access the values attribute directly, but it would depend on the situation. Hope that helps!
@saraths9044
@saraths9044 4 жыл бұрын
Hello there , Thank you for the video but if the axis represents direction of the process, how come the axis is 1 while deleting a row? the deletion process happens in the vertical direction doesn't it???
@kartikeybist
@kartikeybist 4 жыл бұрын
Axis=1 for vertical columns and axis =0 for horizontal rows. All 3 examples are based on this statement. The default value is axis=0.
@anandpatil5210
@anandpatil5210 5 жыл бұрын
In a CSV sheet there are 5 columns A,B,C,D,E i want to plot only colA vs colB. Right now python is selecting all 5 columns to plot.How to select only 2 columns.
@dataschool
@dataschool 5 жыл бұрын
I think this video will help you: kzbin.info/www/bejne/rqfTf3Rtl6hrmdU
@SurendraAkumalla
@SurendraAkumalla Жыл бұрын
Nice teaching! I have a question her. How can we delete rows based on condition ? Like i want to delete rows that fall under 1st Quarter (Q1) of each year ?
@dataschool
@dataschool Жыл бұрын
Great question! You would use a filter to only include the data you do want.
@sdoken
@sdoken 6 жыл бұрын
Thanks for showing this. I will use the df.drop(['column_name'], axis=1, inplace=True) method you showed here. In the past, I also used del df['column_name']. Any comments on why we should avoid using the second method with del keyword?
@dataschool
@dataschool 6 жыл бұрын
No strong reasons to avoid del that I know of, though I would recommend using pandas methods on pandas objects, since you know those methods will always be supported.
@galymzhankenesbekov2924
@galymzhankenesbekov2924 4 жыл бұрын
thanks for the video. But how to delete certain values within a particular column without deleting entire row?
@kartikeybist
@kartikeybist 4 жыл бұрын
ufo.at[,]=None ufo
@galymzhankenesbekov2924
@galymzhankenesbekov2924 4 жыл бұрын
@@kartikeybist i meant with specific values. Let's say i wanna keep value where colour=red, and wanna delete other values within a specific column.
@sharathnandalike8108
@sharathnandalike8108 5 жыл бұрын
When removing a single row 0 , may we just type 0 in the drop () . Why row labels are not typed as strings in 'drop' method and column names are typed as strings, Thanks.
@dataschool
@dataschool 5 жыл бұрын
Great question! Row labels are typed as strings if they are strings, but in the case you're describing, they are stored as integers. Column labels can also be integers, in which case they are always typed as integers. For more info, see this video: kzbin.info/www/bejne/hYq9f55tj56mbKs
@sharatainapur
@sharatainapur 2 жыл бұрын
Thank you for the great videos !! Quick Question - How to reset the row index (here eg- 0,1) after dropping certain records/rows? I wish to have row's numbered 0,1,2,3 after dropping certain rows.
@dataschool
@dataschool 2 жыл бұрын
Use the reset_index() method. Hope that helps!
@hackytech7494
@hackytech7494 4 жыл бұрын
Thankyou so much
@gdorman619
@gdorman619 6 жыл бұрын
Hello, great tutorials! How would the location of a particular column be changed? I want the data associated with the column moved as well. Thanks!
@dataschool
@dataschool 6 жыл бұрын
There are a few options here: stackoverflow.com/questions/13148429/how-to-change-the-order-of-dataframe-columns Hope that helps!
@mecook
@mecook 4 жыл бұрын
Love this video series. I am working with an excel spreadsheet that has numbers in the 1st column for the rows I want to read. Some rows without a number are completely empty, which I figured out how to exclude those, but some have data in the other columns excluding the 1st column. So I want to only read rows where there is a value in the 1st column no mater what is in the other columns. Can this be read into Pandas this way? Thanks in advance.
@dataschool
@dataschool 4 жыл бұрын
I suspect it's possible, but it's hard to say... you will just have to experiment with the parameters for read_csv. Alternatively, you can try reading directly from Excel into pandas. Hope that helps!
@faizsubhani3763
@faizsubhani3763 4 жыл бұрын
How to edit/alter text inside the columns present.?
@dataschool
@dataschool 4 жыл бұрын
Use loc and an assignment statement. Hope that helps!
@pengxu7393
@pengxu7393 7 жыл бұрын
Thanks for the video. Quick question: after removing the first two rows, the index starts from 2. Is there a way to make the index starts from 0 again? Thank you.
@dataschool
@dataschool 7 жыл бұрын
I think this is what you are looking for: stackoverflow.com/questions/16167829/in-pandas-how-can-i-reset-index-without-adding-a-new-column Does that help?
@refiknureddin3758
@refiknureddin3758 4 жыл бұрын
Hello dear, how can add or insert from pandas to postgresql table? Example: From Darksky api, adding response result in a column on a table on postgresql. Thank you
@dataschool
@dataschool 4 жыл бұрын
Export from pandas, and then import into Postgres. Does that help?
@khachatur2000
@khachatur2000 6 жыл бұрын
Thank you for first. But what if we need to remove a range of rows, for example from 0 to 50. ran = [i for i in range(51)] ufo.drop(ran, axis=0, inplace=True)
@dataschool
@dataschool 6 жыл бұрын
Great question! I think this would be better: ufo.drop(list(range(51)), axis=0, inplace=True)
@20101970carlos
@20101970carlos 7 жыл бұрын
Thank you for the excellent video. Quick question. How do you drop certain values (lets says value equal zero) from a column?
@dataschool
@dataschool 7 жыл бұрын
Glad you liked the video! When you say "drop certain values", do you mean to mark those values as missing values, or to drop rows that contain that value?
@renansereninis
@renansereninis 7 жыл бұрын
I'd like to drop rows that contian a certain value
@dataschool
@dataschool 7 жыл бұрын
I think what you need to do is to write a filter that identifies the rows you want to keep. This video should help: kzbin.info/www/bejne/aHKpeIOag9NnfK8
@zeeshancs1646
@zeeshancs1646 4 жыл бұрын
I LOVE YOU GUY Kevin Really you are a king of pandas Library LOL Thank you
@dataschool
@dataschool 4 жыл бұрын
Thank you so much! 😄
@alvarosoto1524
@alvarosoto1524 3 жыл бұрын
Thank you very much !
@wolfy2680
@wolfy2680 2 жыл бұрын
Hi Kevin can you please show us to slice a particular word in a cell of one complete column lets say we have a column name as contact numbers of thousands of employee along with the extension number of that country code so if we want only the numbers excluding the country code how do i get that using pandas dataframe
@dataschool
@dataschool 2 жыл бұрын
Does this help? kzbin.info/www/bejne/mKDJknZmfsieftE
@robstechchannel3687
@robstechchannel3687 6 жыл бұрын
Great tutorial! I got a question about the drop method though: Lets say I have a dataframe df1 and then I create another one out of df1 like this: df2 = df1 If I now use the drop method on df2, it will also delete the regarding column in df1. How can I avoid that? I want it only deleted in df2.
@dataschool
@dataschool 6 жыл бұрын
Great question! When making a copy of a DataFrame, use this instead: df2 = df1.copy() From that point forwards, they will be independent.
@MoniqNansy
@MoniqNansy 4 жыл бұрын
Thank you so much
@erikahuerta2140
@erikahuerta2140 7 жыл бұрын
Rather than removing columns from a DataFrame, how would i go about removing subgroups from one column. For example, I have a column named 'Status', and within status there is 'successful', ' failed', 'live', 'canceled', and 'suspended'. My goal is to remove live, canceled, and suspended. Any advice or suggestions would be enormously appreciated. Thank you!
@Xruba1981
@Xruba1981 7 жыл бұрын
You can use isin() method: df = pd.DataFrame({'Status': ['successful', ' failed', 'live', 'canceled', 'suspended'], 'Score': [1, 2, 3, 4, 5]}) remove_status = ['live', 'canceled', 'suspended'] df[~df['Status'].isin(remove_status)] ~ is the negation of isin(), so you can try it without ~ and change the remove_staus to subgroups to keep instead of subgroups to remove. If I understand your question correctly, this should help :)
@dataschool
@dataschool 7 жыл бұрын
Great suggestion! I also cover 'isin' at the end of this video: kzbin.info/www/bejne/j4GspZmHbZykoK8
@erikahuerta2140
@erikahuerta2140 7 жыл бұрын
Thank you both!
@nguyensyuc7325
@nguyensyuc7325 4 жыл бұрын
When we dropped the rows, the started index is 2 , How to re-index the data ?
@dataschool
@dataschool 4 жыл бұрын
You can use the reset_index method.
@user-jb7vz7ml8g
@user-jb7vz7ml8g 6 жыл бұрын
Hello, can you tell me how to drop the missing data for the columns of the dataframe?
@dataschool
@dataschool 6 жыл бұрын
This video might be helpful to you: kzbin.info/www/bejne/nHSwo4KVi9-Ygpo
@danialmonachan114
@danialmonachan114 3 жыл бұрын
is there a way to delete all the values of a column in a data frame without deleting its name
@dataschool
@dataschool 3 жыл бұрын
Yes, you can set it to a constant value through an assignment statement: df['col'] = 0 (or an empty string or np.nan)
@abdkumar1300
@abdkumar1300 6 жыл бұрын
what to do if we want to remove 10 rows? can we use slicing method? we cant type for more number of rows to be removed.
@dataschool
@dataschool 6 жыл бұрын
If the rows are next to one another, I would probably use iloc: kzbin.info/www/bejne/rqfTf3Rtl6hrmdU
@MrBigmit37
@MrBigmit37 8 жыл бұрын
If I wanted to remove the last 30 rows from my dataframe, making a list of those 30 rows will be a little cumbersome. Is there a more efficient method to remove those rows? I tried playing around with iloc and drop but I couldn't get it to work.
@dataschool
@dataschool 7 жыл бұрын
Great question. This should work: new = ufo.iloc[:-30, :].copy()
@MrBigmit37
@MrBigmit37 7 жыл бұрын
Thanks for the reply. Keep up the great work.
@jungkookpark1859
@jungkookpark1859 7 жыл бұрын
Hello. First of all, Thank you so much for posting a great series of videos. I have a question regarding removing rows. how can I remove multiple rows that have the same name? Thanks again!
@dataschool
@dataschool 7 жыл бұрын
Glad you like the videos! Are you asking how to remove multiple rows with the same index? I believe if you drop using axis=0, and specify an index that is shared by multiple rows, then all rows with that index will be removed. Does that help?
@MrSubir87
@MrSubir87 4 жыл бұрын
I think the question is how to drop rows based on certain conditions
@DrewBive
@DrewBive 7 жыл бұрын
Hello!Will u make videos abt matplotlib?Tutorials abt pandas are absolutely awesome.Thank you for them
@dataschool
@dataschool 7 жыл бұрын
Glad you like the pandas tutorials! Sure, I will consider matplotlib for future videos.
@tomyshelby345
@tomyshelby345 5 жыл бұрын
what if i have 100s of columns in my data set and i want to remove 50 or more column, do we have any shortcut method for it or we have type the name of all columns ?
@dataschool
@dataschool 5 жыл бұрын
There are lots of possible approaches, but it depends on the column names, ordering, etc.
@MrSagu83
@MrSagu83 4 жыл бұрын
can u show an example pls.... the way range works for dropping rows does not seem to work for columns
@user-bd4vn9ky6l
@user-bd4vn9ky6l 4 жыл бұрын
Thank You , But how can remove line : city, colores reported; shap reported;state;time AND column 0;1;2;4 ???
@dataschool
@dataschool 4 жыл бұрын
Sorry, I don't quite follow your question, good luck!
@amersanjak8959
@amersanjak8959 5 жыл бұрын
what's the difference between: ufo.drop('Colors Reported', axis=1, inplace=True) ufo.drop(columns='Colors Reported', axis=1, inplace=True) I tested both and they return the same results.
@dataschool
@dataschool 5 жыл бұрын
They are the same.
@rajeshkhanna8276
@rajeshkhanna8276 6 жыл бұрын
Hi, I have 5 types of data in single column, but I want only one type of data from that column, how can I export that with pandas. Kindly suggest me on this. And thank you very much for your great sessions.
@dataschool
@dataschool 6 жыл бұрын
Sounds like you need to extract the portion of the column that is of interest. This might help: kzbin.info/www/bejne/mKDJknZmfsieftE
@samb834
@samb834 7 жыл бұрын
Great videos, I'm finding them very useful. Once a column from a dataframe has been selected, is there a way to remove rows that contain a specific value (i.e., deleting all rows that contain a 0)?
@dataschool
@dataschool 7 жыл бұрын
This video discusses how to filter the rows of a DataFrame by column value: kzbin.info/www/bejne/aHKpeIOag9NnfK8 Once you have filtered, you can then copy the filtered DataFrame into a new DataFrame. That should accomplish your objective. Hope that helps!
@samb834
@samb834 7 жыл бұрын
Fantastic, thank you! Keep up the great work.
@dataschool
@dataschool 7 жыл бұрын
Thanks!
@pankajnagpal3110
@pankajnagpal3110 7 жыл бұрын
Hi Is there any way to undo drop operations.. ?
@dataschool
@dataschool 7 жыл бұрын
If you use inplace=True, there is no way to undo it. If you use inplace=False, the drop operation is temporary and there is no need to undo it. Hope that helps!
@podliza123
@podliza123 8 жыл бұрын
Hi, thanks for the series! What's the best way to drop every non-numeric column from a DataFrame?
@podliza123
@podliza123 8 жыл бұрын
Figured it out. NameOfYourDataFrame._get_numeric_data() does the trick.
@dataschool
@dataschool 8 жыл бұрын
+podliza123 Although that does work, it's not generally recommended in Python to access methods that begin with an underscore (loosely thought of as "internal" or "private" methods). The preferred way would be to use the DataFrame method 'select_dtypes': pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.select_dtypes.html For example: import numpy as np df.select_dtypes(include=[np.number]) Glad you are enjoying the series!
@podliza123
@podliza123 8 жыл бұрын
+Data School Thanks for the clarification! Will keep this in mind.
@dataschool
@dataschool 8 жыл бұрын
+podliza123 You're very welcome!
@dataschool
@dataschool 8 жыл бұрын
+podliza123 I featured your question in my latest video :) kzbin.info/www/bejne/eF7VaomrgJ1jms0
@mohammadyasir24
@mohammadyasir24 4 жыл бұрын
how can one element from the row or column be dropped ?
@dataschool
@dataschool 4 жыл бұрын
You can overwrite a particular "cell" using the loc accessor and an assignment statement. Does that help?
@RA-pi1lg
@RA-pi1lg 5 жыл бұрын
What if I want delete value based on condition for example Drop any thing > 10 How can I do that??
@dataschool
@dataschool 5 жыл бұрын
This should help: kzbin.info/www/bejne/aHKpeIOag9NnfK8
@Jared-Pace
@Jared-Pace 7 жыл бұрын
Thank you. Very nice!
@dataschool
@dataschool 7 жыл бұрын
You're welcome!
@jaspindersingh7866
@jaspindersingh7866 7 жыл бұрын
How do I replace a column with another series/list?
@dataschool
@dataschool 7 жыл бұрын
I think the best process would be to drop the original column, create a new Series with the Series constructor, and then concatenate the DataFrame and the new Series. Check out videos 6 and 29 in this playlist: kzbin.info/aero/PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y Hope that helps!
@priyankasharma6738
@priyankasharma6738 4 жыл бұрын
You are phenomenal
@dataschool
@dataschool 4 жыл бұрын
Thank you! 😊
@9zQx86LT
@9zQx86LT 7 жыл бұрын
Thank you thank you !!
@dataschool
@dataschool 7 жыл бұрын
You're very welcome!
@Cloud-Radio
@Cloud-Radio 4 жыл бұрын
axis =0 (row), axis=1(column)
@dataschool
@dataschool 4 жыл бұрын
Right!
@alexanderpeshkov6416
@alexanderpeshkov6416 7 жыл бұрын
Great video, thank you
@dataschool
@dataschool 7 жыл бұрын
You're welcome!
@prestonchen2046
@prestonchen2046 6 жыл бұрын
Great content, thanks a lot!
@dataschool
@dataschool 6 жыл бұрын
You're welcome!
@yandongbi1800
@yandongbi1800 7 жыл бұрын
Hello, can you tell me what the website you edit your python code on. Is it an online code editor?
@dataschool
@dataschool 7 жыл бұрын
It is the Jupyter notebook, previously known as the IPython notebook.
@yandongbi1800
@yandongbi1800 7 жыл бұрын
URLError: .
@dataschool
@dataschool 7 жыл бұрын
When are you getting that error message?
@Triple109
@Triple109 5 жыл бұрын
How to remove the ... in the dataset?
@dataschool
@dataschool 4 жыл бұрын
Sorry, I'm not sure what you mean?
@adriantoledano
@adriantoledano 5 жыл бұрын
how can i hide/drop Index column?
@dataschool
@dataschool 5 жыл бұрын
No, the index is part of every DataFrame or Series. Check out this video: kzbin.info/www/bejne/hYq9f55tj56mbKs
@sorayaanvari5906
@sorayaanvari5906 7 жыл бұрын
Thank you :) I write this :df.drop['job',axis=1,inplace=True] and I give error invalid syntax why????
@dataschool
@dataschool 7 жыл бұрын
Maybe your DataFrame doesn't have a column named 'job'? Keep in mind that Python is case-sensitive. Hope that helps!
@sorayaanvari5906
@sorayaanvari5906 7 жыл бұрын
thank you so much, I got this. my csv file contain ; I need to remove them first. then open it in excel
@dataschool
@dataschool 7 жыл бұрын
Makes sense! Glad you were able to get it to work!
@kevinchoi8135
@kevinchoi8135 7 жыл бұрын
How do you remove the very first column that one without any title or index (the one only report 0, 1,2,3,4...so on)
@dataschool
@dataschool 7 жыл бұрын
That column is known as the index, and it can't be removed. I recommend watching this video to learn more: kzbin.info/www/bejne/hYq9f55tj56mbKs Hope that helps!
@kevinchoi8135
@kevinchoi8135 7 жыл бұрын
Thank you. Btw do you have a video on lookup (like vlookup in Excel?) function in Python? Thanks again
@dataschool
@dataschool 7 жыл бұрын
I don't have a video, but this might be what you are looking for: stackoverflow.com/questions/25493625/vlookup-in-pandas-using-join
@kevinchoi8135
@kevinchoi8135 7 жыл бұрын
Thanks I think I saw this page before, but I am confused. I am new to Python and I found your videos very helpful. So many thanks for all your efforts
@CuriousIndic
@CuriousIndic 7 жыл бұрын
what is the way to make a copy of data.frame so that anychange in that doesn't change the original dataframe Suppose my dataframe name is trans I tried trans1=trans when I made changes in trans1 the same changes was occurring in trans too
@dataschool
@dataschool 7 жыл бұрын
You would use the copy method: trans1 = trans.copy() This video might also be helpful to you: kzbin.info/www/bejne/aoOXiKZ6f5J-jMU
@tonycardinal413
@tonycardinal413 3 жыл бұрын
Great video.But I receive the following error: /usr/local/lib/python3.6/dist-packages/pandas/core/indexes/base.py in drop(self, labels, errors) 5016 if mask.any(): 5017 if errors != "ignore": -> 5018 raise KeyError(f"{labels[mask]} not found in axis") 5019 indexer = indexer[~mask] 5020 return self.delete(indexer) KeyError: "['Close/Last' 'Volume' 'High' 'Low'] not found in axis"
@MrMukulpandey
@MrMukulpandey Жыл бұрын
well explained
@dataschool
@dataschool Жыл бұрын
Thank you!
@AbdallahProgrammer
@AbdallahProgrammer 7 ай бұрын
Thank you
@dataschool
@dataschool 7 ай бұрын
You're welcome!
@rajbir_singh0517
@rajbir_singh0517 5 жыл бұрын
what if , we want to drop multiple Row from their name "index"
@dataschool
@dataschool 5 жыл бұрын
I'm sorry, I don't understand your question. Could you clarify? Thanks!
How do I sort a pandas DataFrame or a Series?
8:57
Data School
Рет қаралды 117 М.
How do I rename columns in a pandas DataFrame?
9:37
Data School
Рет қаралды 150 М.
Gli occhiali da sole non mi hanno coperto! 😎
00:13
Senza Limiti
Рет қаралды 9 МЛН
Бутылка Air Up обмани мозг вкусом
01:00
Костя Павлов
Рет қаралды 1,6 МЛН
ПОМОГЛА НАЗЫВАЕТСЯ😂
00:20
Chapitosiki
Рет қаралды 26 МЛН
How do I select multiple rows and columns from a pandas DataFrame?
21:47
How do I select a pandas Series from a DataFrame?
11:11
Data School
Рет қаралды 158 М.
How do I use the MultiIndex in pandas?
25:01
Data School
Рет қаралды 173 М.
How do I find and remove duplicate rows in pandas?
9:48
Data School
Рет қаралды 106 М.
When should I use a "groupby" in pandas?
8:25
Data School
Рет қаралды 243 М.
How do I handle missing values in pandas?
14:28
Data School
Рет қаралды 196 М.
How do I apply a function to a pandas Series or DataFrame?
17:58
Data School
Рет қаралды 201 М.
How do I change display options in pandas?
14:56
Data School
Рет қаралды 41 М.
How do I read a tabular data file into pandas?
8:55
Data School
Рет қаралды 208 М.
How do I merge DataFrames in pandas?
21:49
Data School
Рет қаралды 158 М.
Gli occhiali da sole non mi hanno coperto! 😎
00:13
Senza Limiti
Рет қаралды 9 МЛН