You are really good at explaining things. One of the better teachers on youtube. Thanks a ton for this video and I hope there's more coming.
@dataschool2 жыл бұрын
Thank you!
@NoName-qx9zc4 жыл бұрын
I'd like to thank the author. You really do a great job. Everything is structured, decomposed and coherent. Some guys just jump in complex coding without really explaining what's going on there.
@LuisRivera-ce9lm2 жыл бұрын
I just wanted to thank you for such a great explanation of joins. I did not have it explained to me and struggled for the longest time to understand them. It takes a good teacher and someone who can understand it simply for one to understand it. Seriously, you are amazing!!
@dataschool2 жыл бұрын
Thank you so much! 🙏
@amrita3011572 жыл бұрын
This is one of the best ever videos on pandas functions that I have watched. Well done Data School. I will look forward to more such videos.
@dataschool2 жыл бұрын
Thank you so much! 🙏
@mschuer1004 жыл бұрын
This, by far, is the best explanation of these concepts. Thanks for sharing.
@dataschool4 жыл бұрын
Wow, thank you so much for your kind words! 🙏
@lolkids78334 жыл бұрын
Thanks, Kevin.. this is the clearest explanation of the merge I have seen.
@dataschool4 жыл бұрын
Thank you so much!
@lenny95sehs2 жыл бұрын
Your videos are fantastic. I really appreciate the simple "common sense" approach to the teaching. It is quite easy for instructors to dive right into python lingo.
@dataschool2 жыл бұрын
Thank you so much for your kind words!
@gardnmi4 жыл бұрын
The best new feature with merge is the validate option to make sure your join is 1:1, 1:M, etc. This is very useful for machine learning projects or end user reports that rely on upstream data that is updated regularly. It's saved me headaches a few times.
@dataschool4 жыл бұрын
The "validate" option is great, I agree! I also like "indicator", which I explained here: twitter.com/justmarkham/status/1153653794829418496
@pogoclub84954 жыл бұрын
This is 1st time i walked into your video and i am very much impressed by your explaination and your english speaking pace is perfect. loved your content. Thanks a lot. :)
@dataschool4 жыл бұрын
Thanks so much for your kind words!
@danielchacreton24014 жыл бұрын
Your videos are always amazing. You are a national treasure in my book. Don't change a thing, but for viewer 1.75 speed is the speed to watch these in.
@dataschool4 жыл бұрын
Thank you! 🙌
@JesperHolmPedersen4 жыл бұрын
Supercool. Very impressive how you manage to explain the pretty complicated functionality of merge. Thanks.
@dataschool4 жыл бұрын
Glad it was helpful!
@DJoeyJordi5on3 жыл бұрын
This was a well-paced, clear and complete explanation of the topic, thank you very much! It helped me a lot
@dataschool3 жыл бұрын
That's awesome to hear!
@CristianBittel4 жыл бұрын
Great as teacher, calm, taking your time to clearly explain fundamentals!
@dataschool4 жыл бұрын
Thanks so much for your kind words, I truly appreciate it!
@summerzhang94844 жыл бұрын
Thanks for the videos Kevin! I love your teaching style and how you make each concept so crystal clear. Please keep making these videos! Just signed up to become a patron of yours and am taking your course on Data Camp (I wish you taught more courses on there!) Once I master Pandas will try out your machine learning course too :) ps your son is so adorable
@dataschool4 жыл бұрын
You are too kind, Summer! Thank you SO much for your kind words AND for becoming a patron! 🙌
@kickassbass4 жыл бұрын
Kevin you are a super hero of Data science, best videos on tube...
@dataschool4 жыл бұрын
Thank you!
@jonass74563 жыл бұрын
Dude! Let me tell you, you saved me a lot of time and work! Thank you so much!
@dataschool3 жыл бұрын
Great to hear!
@vipinamar83233 жыл бұрын
Nice teaching method. precision over pace.
@dataschool3 жыл бұрын
Glad it was helpful!
@sanjay1236443 жыл бұрын
Excellent way of teaching. Thanks Kevin
@dataschool3 жыл бұрын
Glad it was helpful! 🙌
@saraghafelehbashi58082 жыл бұрын
Please keep making these videos! You are awesome!
@dataschool2 жыл бұрын
Thank you!
@jcbcorner84644 жыл бұрын
Finally a clear explaination of merge function !! Thanks, subscribed
@brunoreighner17804 жыл бұрын
You're an amazing teacher. Thanks a lot for these.
@dataschool4 жыл бұрын
Thank you! 😃
@bongi_nkuna3 жыл бұрын
This video is pure GOLD, absolutely wonderful, loved the clear explanations , thank you...
@dataschool3 жыл бұрын
So glad to hear it was helpful to you! 🙌
@BC-gc7bv4 жыл бұрын
You are an excellent teacher!!! I'm a fan. TY.
@rawanfouda22914 жыл бұрын
That was honestly really good! thank you so much for your work
@shashi_kamal_chakraborty2 жыл бұрын
Thanks! very nicely explained. Now, I can perform joins using Pandas, quite effortlessly.
@dataschool2 жыл бұрын
Glad it helped!
@shashi_kamal_chakraborty2 жыл бұрын
@@dataschool Yeah! beside books, I follow you, especially for Pandas. Great help. Thanx...
@dataschool Жыл бұрын
You're welcome!
@tommonks24904 жыл бұрын
Excellently explained as always. Keep up the great work!!
@dataschool4 жыл бұрын
Thank you!
@autonish4 жыл бұрын
Brilliant Stuff, All videos are awesome. Clearly explained all fundamentals...Thanks for making this stuff easy. On a different line, you remind me of "Sheldon" from the TV series The Big bang theory and this is a compliment. :)
@dataschool4 жыл бұрын
Ha! So many people have said that 😄
@da_ta4 жыл бұрын
Thanks Kevin I have been looking for this for long time!
@dataschool4 жыл бұрын
Awesome! I'm so glad to hear this is the video you needed! 🙌
@vitoroliveira63634 жыл бұрын
wonderfull, loved your slow passed english, that helped me a lot
@dataschool4 жыл бұрын
Glad it helped!
@hieungotrung54114 жыл бұрын
Great to see you again as well as your high-quality content in your video
@dataschool4 жыл бұрын
Thanks so much for your kind words! 😄
@g.jignacio4 жыл бұрын
Excelent video! keep sharing content like this. Greetings from Argentina
@dataschool4 жыл бұрын
Thanks!
@themustknowfacts5103 жыл бұрын
I'm not able to read that file "u.item" , I copied the same code from GitHub but pandas wasn't able to read that. It showed me Unicode Error... How do I solve that issue..
@ChrisMao_7083 жыл бұрын
insert this encoding='latin-1' and you will be fine
@mr.stemedutv55144 жыл бұрын
Very easy to follow, and thanks for making very useful video!
@dataschool4 жыл бұрын
Thank you!
@citizen_deb4 жыл бұрын
Thank you so much Kevin, your neat explanation along with the file you share makes it so clear, was really needing it!
@SR-lf3ic2 жыл бұрын
hi, when I used pd.concat([df1,df2]), I got a tuple object instead of a dataframe object. I am using Python 3.9 environment. I would like to know what should I do to get a dataframe object rather than a tuple object?
@cmovilidad14 жыл бұрын
Máster! Regresó! chévere.
@dataschool4 жыл бұрын
😄
@avelinoamado45683 жыл бұрын
This video was very helpful and clear. Thank you for this content.
@dataschool2 жыл бұрын
You're welcome!
@TheNobody043 жыл бұрын
Wow, I've seen some of your videos and I just can say THANK YOU. It's so easy to understand you :3
@dataschool3 жыл бұрын
Thanks for your kind words! Glad you like my videos!
@dannylockett94452 жыл бұрын
I really enjoy your tutorials, thanks so much! I have 5 csv files that come out daily each containing a date column. i want to merge them all using the date as the merge field. i tried a basic merge with 2 of the csv files and date was used as the merge-on field by default - so it worked. ultimately i just need one date column in my masterfile with all the other column data merged. should I continue to do this or is it better to set the date column as the index, or something else?
@nowyouknow22494 жыл бұрын
Thanks a lot Kevin We have missed you.
@dataschool4 жыл бұрын
Thank you! 😊
@jaydhanwant40724 жыл бұрын
I wish we had 3x on youtube, great video!
@JainmiahSk4 жыл бұрын
Good to see you. I love the logic you teach.
@dataschool4 жыл бұрын
Thank you! Glad my videos are helpful to you 👍
@NiireNolweva3 жыл бұрын
Very clear and informative. Thank you very much.
@dataschool3 жыл бұрын
You're very welcome!
@joseluisbeltramone5993 жыл бұрын
Thank you very much for the precise explanation, just what I needed to know!
@dataschool3 жыл бұрын
You're very welcome! 🙏
@fschmidkonz4 жыл бұрын
You're great teacher! I see the despite having a large 100K row file, the number of rows do not get expanded after the merge. They beautifully stay the same and just add the movie titles to the reviews. Can you comment on why this is not always the case. I have tried and my output file gets expanded by a few rows (17 out of 1000) and I have not been able to figure out why. I have checked multiple videos and some come absurd not practical solutions (like the files are the same size) or arbitrarily eliminate any dups (despite some may be valid rows), but none explain the reason and how to identify those rows that could be dups. Your comments are appreciated.
@SamSam-mh5jt3 жыл бұрын
Thank you so much for the clear and concise explanation
@dataschool3 жыл бұрын
You're welcome!
@alankarshukla43854 жыл бұрын
Not wait too much to watch this.
@dataschool4 жыл бұрын
I hope the video is helpful to you!
@mochammadirfanbaihaqi2793 жыл бұрын
Love the way you explain it, thanks for your vids. Keep it up (thumbs)
@feroncia4 жыл бұрын
Thank you so much for explaining it clearly. Now I understand on merging dataframe more. TQVM
@dataschool3 жыл бұрын
You're welcome!
@cutestbear3327 Жыл бұрын
thnx for the video, that's awesome, particular the parts on explaining joins. clear and concise
@dataschool Жыл бұрын
Great to hear!
@svengunther76534 жыл бұрын
You are doing a really great job with this. Thank you so much! :)
@dataschool4 жыл бұрын
Thanks!
@jaysoni78124 жыл бұрын
Were is the link of the data set which has been used in this video. I want to practice this with your data set can you please send me link?
@maxvinella9414 жыл бұрын
Missing your pandas tutorials.. thanks
@dataschool4 жыл бұрын
It's nice to be missed! You can find all of my pandas tutorials here: kzbin.info/aero/PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y
@JustJoelTV2 жыл бұрын
Great video, informative and clear. Thanks
@dataschool2 жыл бұрын
You're welcome!
@jalego800 Жыл бұрын
Hi Kevin, thanks to your turtoring, I learn a lot from your channel, it's amazing! Since I just learn Pandas, I'm a little bit confused about concat(), melt(), merge(), pivot(), stack()...They're really annoying to me >< I really hope we have a one for all solution of how to use these functions XD Thank you!
@dataschool Жыл бұрын
I agree, it's tricky to separate out when you should use each one of those!
@gregf91604 жыл бұрын
Thank you so much for the concise clear explanation. Much appreciated.
@bilalahmad91776 ай бұрын
You are a great instructor. I have learned a lot from you regarding pandas. The video with title "How do I merge DataFrames in pandas?" has left some queries in my mind. I would be thankful to you if you clear those too. What type of join is used here movie_ratings = pd.merge(movies , ratings)? if it is inner join it should result in 1682 rows in total in movie_ratings dataframe, as movies dataframe has 1682 rows. But in video i have observed that movie_ratings results in 100,000 rows of data.
@zezodiaa10252 жыл бұрын
great video. my question is when im working on project when exactly i have to combine ?
@sch0ll13 жыл бұрын
Thanks man! You saved my weekend :*
@dataschool3 жыл бұрын
Glad I could help!
@omidadib50523 жыл бұрын
Awesome tutorial, Thank you very much man!
@dataschool3 жыл бұрын
You're welcome!
@eliasaudi28773 жыл бұрын
What would we use to show ONLY all the values that do not match ? .... i.e. anything other that inner join
@Octaphea2 жыл бұрын
Great video. However I have a little issue. I have 3 data frames that I am trying to merge together. The first is a pretty long database with columns (cust_id, gained_on gained_from_supplier, lost_to_supplier, sales_channel_id) the second is the supplier data frame (supplier_name, supplier_id) what I am trying to do is merge the supplier id and name from the second data frame, to the database frame which has the ID so supplier id to the number using the lefton/right on but instead it returns both columns - the supplier ID and name of both dataframes. Then the same with the channel data frame (sales_channel_name, sales_channel_id) and merge this with the sales_channel_id in the database dataframe and show the name instead. Any help would be appreciated, thank you!
@dhirajp46774 жыл бұрын
Hello Data school,I need to convert below dataframe into datetime dtypes period 0 28.02.2020 10:32:17:640 1 28.02.2020 10:32:18:656 2 28.02.2020 10:32:19:656 3 28.02.2020 10:32:20:671 4 28.02.2020 10:32:21:687 5 28.02.2020 10:32:22:687 6 28.02.2020 10:32:23:703 df['period'] = pd.to_datetime(df['period']) i used above code but it is throwing error ValueError: ('Unknown string format:', '28.02.2020 10:32:17:640') how do i go ahead..?
@dataschool4 жыл бұрын
Not sure, sorry!
@cgpmth64492 жыл бұрын
How to merge multiple large dataframes in a fast way? I joined with usual merge() but it seems too slow. I found a clue of using pandas.Index() with the merge method, but i don't know how to use it.
@ayodejiakinfenwa2 жыл бұрын
Plesae i am trying to merge two datasets as you have explained but it is giving an error that i should check for duplicates
@vinayakchikkorde81513 жыл бұрын
I have the source file and target file. so in that, I have to compare 140 columns and show the result if it matches or not. for example, there is a column as Country1 in source and in target as Country2. to compare that i will use if(source['country1]==target['country2])return True else return false. to compare 140+ columns it will take time to compare 140 columns. and in both of the file columns are not in ordered. so how can I solve this?
@zapy4224 жыл бұрын
Thank you for this video. I have been struggling with merge and concat today :)
@dataschool4 жыл бұрын
You're very welcome! Glad it's helpful to you!
@Moc2Talk4 жыл бұрын
slowly talk is very helpfull to me. I have 2 questions. The first is : What's if i want merge only one certain column (rating) from df rating to df movie . The second: What's if I want to sum the rate of each Movie_Id . Tks you so much and looking for your answer.
@osmanhussein38933 жыл бұрын
This is very helpful. Thank you so much.
@dataschool3 жыл бұрын
You're very welcome!
@alndr4u2 жыл бұрын
How to merge two dataframes based on 4 common columns with repatative elements?
@adedolapoogungbire70883 жыл бұрын
Just what I needed.
@dataschool3 жыл бұрын
Great to hear!
@shivamsaway68034 жыл бұрын
Does it happen while merging two data frames, only heads get to merge, No data get merged inside the new data frame?
@juliakristavilladiego2454 жыл бұрын
Thank you! Crystal clear explanation.
@saikiranhr2 жыл бұрын
Thanks for the amazing video. One simple question. How to join tables on multiple indices (like 4 or 5)?
@akinsikuelizabeth57804 жыл бұрын
Superb!!! I got Evey explanation, thanks
@dataschool4 жыл бұрын
You're welcome!
@ruthliganad82744 жыл бұрын
how about not a specific file? for example all .csv or all .tsv file? how to concatenate a header to that file? Thanks
@mehnazjabeen3 жыл бұрын
How to verify if all the columns are incorporated in the merged DataFrame by using simple comparison Operator in Python after merging two DataFrame?
@christleiroezi88784 жыл бұрын
I have a data frame and I have a list and a tuple , I want to merge all three together . I am aware merge can only do two tables at a time, but do you have any helpful hints on how to go about merged the table , list and df. I want make to make the result a new data frame
@jeevakumara55992 жыл бұрын
hi bro, I am currently working in a project. The mentors says that use foreign keys and primary keys in pandas and create table with the keys. so my question is, the usage of foreign and primary keys in pandas is possible or if we can't what shall I do to merge the two tables contains the same column which we are doing in the MYSQL coding. Thank you.
@AsMa-eg3 жыл бұрын
thank u so much. very clear and to the point.
@dataschool3 жыл бұрын
You're welcome!
@hardikvegad35084 жыл бұрын
Sir if we have hundreds of columns without the name. Then how can we name them using pandas and a for loop or lambda function because if we try to name them using names=[] it will be a very time-consuming process. The name of the columns can be col1, col2 , col3...etc.
@sivababu27532 жыл бұрын
Thanks for the video, I have a query sir, Let's consider if I have a table 1 with features (order Id) and (product Id) and table 2 with features (order Id) and (product Id).How to fetch the observations which is present in table 1 that not present in table 2
@dataschool2 жыл бұрын
Great question! See trick 16 in this video: kzbin.info/www/bejne/qoipgqSXh9mGh6M
@jqts64903 жыл бұрын
Thanks for the video. I was able to successfully meagre and find some errors from Ids I did not find using VBA vlookups. I was curious. Is there a way to highlight difference between columns in this merged database. example: Number of Vehicles_SS: 7 vs Number of Vehicles_SA: 2 and it would highlight the row, or even just those those values, base on the ID it was merged on? I am having a hard time find this. Trying to get rid of VBA, which i have doing this, But it is SUPER slow with the data I have to process.
@ДмитрийИгнатьев-з5т4 жыл бұрын
Hello, Many thanks for you tutorial. It's great!!! But i.m stuck is any techics to join two dataframes if one of them stack other not stack?
@Isabel-ec2sq4 жыл бұрын
Thank you!! I finally got the dataframe I wanted!
@ramachalprajapati11764 жыл бұрын
How to get the common mobile number from two different csv file having the different column name
@ericmindyc3 жыл бұрын
Hello. Great vid. But how do I follow along? Other videos had the bitly link. I can’t find the dataset for this exercise.
@dataschool3 жыл бұрын
Datasets are here: github.com/justmarkham/pandas-videos/tree/master/data
@Zahraa_0054 жыл бұрын
This is the best explanation Thanks so much!
@dataschool4 жыл бұрын
You're very welcome!
@lukaszeroone2304 жыл бұрын
Can pandas execute query reading sql from file or filename.sql?
@wilsonmupfururirwa65234 жыл бұрын
Hi wanted to ask how you check for data consistency in columns. Like checking for integers in a string column or trying to find values like 2A in a column with double letter values eg. AA, BB etc
@dataschool4 жыл бұрын
Great question, though there's no "one way" to catch all of these issues! Here are some tricks that might be helpful, though: kzbin.info/www/bejne/iJ2smombnsxmnsU
@OriginalBernieBro4 жыл бұрын
what is the support column in sklearn's classification_report represent and what parameters can adjust it, I'm struggling on an highly imbalanced dataset, smoted it but this metric 'support' is off showing highly imbalanced!
@hectoralvarorojas19184 жыл бұрын
Great work as allways. Very useful. Thanks for sharing it! By the way, any chance you get some video done about PySpark? It will be very usefull to treat this from the biginning considering examples based on a local connection (one computer) first and then a couple of examples emulating a cluster connection.
@dataschool4 жыл бұрын
Thanks for your kind words as always, Hector! Sorry, I don't have any videos about PySpark, but I appreciate the suggestion! 👍
@hectoralvarorojas19184 жыл бұрын
@@dataschool I would love for you to do that. I am possitive that you will get a lot of interested guys, among them me of course. My best regards!
@shaheenalhirmizy96484 жыл бұрын
Hi kevin how are you doing, is there any way using pandas or another library for conditional merging?, if I want to choose from two data Thank you very much
@dataschool4 жыл бұрын
Could you describe in more detail what you mean by "conditional merging"? Thanks!
@shaheenalhirmizy96484 жыл бұрын
I mean if we have two different tables has same numbers of columns and We want to merg them but, not all data only the rows of data we want using condisonal formulas
@dataschool4 жыл бұрын
You should perform the operation in two steps: first do the filter, and then do the merge.
@comparethis-p1g2 жыл бұрын
the resulting dataset I got has a value of null. What do i do?
@philongtran94553 жыл бұрын
hello, i cant retrieve merged df in another cell, how can i fix that ?
@karthikbandamidi4304 жыл бұрын
can we merge more than two dataframes using pandas?
@Code-og3ew4 жыл бұрын
Brilliant lecture. I have a question: should the series information be the same before you can merge columns of different names?
@dataschool4 жыл бұрын
Thank you! I'm not sure what you mean by "should the series information be the same"?
@lualmeidasilveira4 жыл бұрын
How do I merge df1 and df2 by two columns (fiels) at clausula on? For example: dfUltStatus = pd.merge(dfUltStatus, dfDescStatus, on=['CODIGO_STATUS','SUB_CODIGO_STATUS'], how = 'left') The object is merge the two data frames through these two fields to bring the description field.
@robertc21214 жыл бұрын
Love your videos!! excellent tutorial - by chance does Pandas have a facility to do a semi_join() like Dplyrs function?
@dataschool4 жыл бұрын
Thanks for your kind words! I'm not familiar with semi_join, sorry...
@AnoNymous-dh2sv2 жыл бұрын
What's the concat video? You say there is one, but I can't find it with search.
@dataschool2 жыл бұрын
It's at the end of this video: kzbin.info/www/bejne/Z2bUXpypbbWSfpY Hope that helps!