Python Pandas Tutorial: Different ways to filter Pandas DataFrame #9

In this video lecture you will learn different ways to filter the data from Pandas DataFrame.
Introducing Pandas Query - Dynamic Data Filtering
What to do when the filtering criteria move too much?
Pandas has easy-to-use tools to filter your DataFrames, however the basic methods - like loc, iloc, or passing a boolean expression - lack the ability to use them comfortably when the filtering parameters are not static. The query DataFrame method offers a solution for this problem.
Data Filtering is one of the most frequent data manipulation operation. It is similar to WHERE clause in SQL or you must have used filter in MS Excel for selecting specific rows based on some conditions. In terms of speed, python has an efficient way to perform filtering and aggregation. It has an excellent package called pandas for data wrangling tasks.
You will learn:
1.) Apply filter single value.
2.) How to use Boolean value in Filter?
3.) Filter the criteria based on multiple columns.
4.) Filter the multiple cells in single column or multiple column.
5.) Understanding the logical operators in filters.
6.) How to apply nested criteria in filter?
7.) Apply the filter using ~ (tild) operator.
8.) Filter with string data.
9.) Apply the filter using isin method.
10.) How to use Lambda within the filter?
11.) Filter with Query method, nlargest, nsmallest and loc.
12.) Examples....
Jupyter Notebook: github.com/Abh...
DataSet: github.com/Abh...
Python Teaser: • A Beautiful Python Pro...
Python Pandas Tutorial: • Python Pandas Tutorial...
Python Playlist: • Python Tutorial for Be...
Python Data Structure Playlist: • Python Data Structure
Python OOPs Playlist: • Object Oriented Progra...
Python Excel Automation: • Excel Automation Using...

[@mosama22]

Thank you 😊 So to Sum up, we can filter data in pandas through: - Boolean Mask, - loc (and unnecessarily iloc), - query(), - lambda, - isnull, and which you discussed in detail in the next video, Thx body 🙂 🙂

[@vimalchawla6740]

>700 and =

[@dss1871]

Thanks Sir 😊😊 very informative session..👍👍👍👍

[@ProgrammingIsFunn]

I am happy, its useful for you.

[@hanumantgavali8543]

hey bro thank you, you did it as ur channel name...!!

[@ProgrammingIsFunn]

Thanks Hanumant..... 😊

[@Avdheshkumar-kd9qz]

awesome content with nice explanation !! thx !!

[@ProgrammingIsFunn]

Glad you liked it!

[@raymundofelipeolvera5594]

In the lambda df{df.apply(lambda x :x["City"] =="Houston" and x["Category"] == "Technology", axis = 1)] how can I add one more value to Category (x["Category"] == "Technology" and "Office Supplies, axis = 1] for example? Similar that the example that you show on excel filtering multiple values in two different columns

[@chint4383]

I am facing problems to merge two dataframe will you hepe ???

[@GrowCube-z8z]

Can u please show how to folder based on OrderDate

[@GrowthMindset_J]

I have multiple conditions to validate. if ‘Country’ column has specific value and ‘Age’ column is less than 11 or ‘DOB’ column has date after 2010 or ‘Comments’ column contains few different strings from a list and create a new column that prints value ‘Valid’ if all conditions meet else prints ‘Invalid’. Need to define a function and apply to multiple data frames. I’m so lost

[@ProgrammingIsFunn]

Thanks for you comment. First of all try to apply filter on country , age and DOB column right, it is easy, and I explained in the video, after that you can go for other condition with the same logic.

[@Moksha_Rahasyam]

thank you very much

[@daljeetghag3321]

This was amazing

[@vimalchawla6740]

Where are the sample files you use in your examples?

[@ProgrammingIsFunn]

Vimal, it is mentioned in the description of video. And I am sharing here also. DataSet: github.com/AbhisheakSaraswat/RawData/blob/main/Superstore.xls

[@bijaynayak6473]

Very nice explained

[@visasimbu]

date was stored in string format in dataframe. I am struggling on Converting string to date and filtering

[@ProgrammingIsFunn]

Hi, if the date is stored in string format then try to convert using pd.to_datetime or you can use datetime index and strptime method to convert string in date time object

[@visasimbu]

@@ProgrammingIsFunn so sweet of you. I have got answer under this video. You have replied for some one else. Thanks

[@ramaisgod]

Wow.. Very beautifully explained... Now my data filter concept is clear...

[@abhisheksaraswat6753]

Great 👌

[@jitendratrivedi7889]

Very informative and too much information in one video , keep posting videos

[@marvhan888]

this is very nice video. Thank you so much

[@mcquezchima6402]

How to filter by date

[@ProgrammingIsFunn]

df[df['Column Name'] == 'YourDate'] see below example:- df[df['Order Date'] == '2016-11-08']

[@mcquezchima6402]

@@ProgrammingIsFunn thanks and nice content. I have a file with data's from 2008 to 2012 but I just want to select data for 2010 how do I go about it

[@ProgrammingIsFunn]

@@mcquezchima6402 Thanks. We can apply filter by couple of ways: 1.) df1["Order Date"] = pd.to_datetime(df['Order Date']) df[df1['Order Date'].dt.year == 2008] 2.) df[df['Order Date'].str.contains('2008')] Try and let me know.

[@mcquezchima6402]

@@ProgrammingIsFunn first one worked out thanks

[@visasimbu]

@@ProgrammingIsFunn awesome. case1 worked for me.

[@vloggergargfamily]

Thanku sir for such kind of information that you are provinding through your vedios

[@funwithfamily6906]

So declarative vedio thanks ...sir

[@chetanpatil2510]

very well explain thanx