How to combine DataFrames in Pandas | Merge, Join, Concat, & Append

[@sujatamajhi6186]

The most detailed explanation with visuals of how the data frames are merging and joining ... #Kudos

[@T_tolu]

The most simplified tutorial I’ve seen about this topic, now I understand perfectly. Putting all tables and codes in discussion side by side made it easy. Thank you 🙏🏾

[@hadidy13]

Thanks so much! The only detailed explanation on KZbin, every other tutorial just write the code and you should guess what really happened.

[@leongkern6746]

you just saved my final year project sir! Much respect for the concise and visual learning material

[@khbye2411]

Note to self1: At 10:37 the df1.join(df2, on='ID', lsuffix='_l', rsuffix='_r') is the same as df1.merge(df2, how='left', left_on='ID', right_index=True, suffixes=('_l', '_r'))

[@manavsaxena5579]

Amazing content, I am more of a visual learner and you have explained the concepts brilliantly with examples. Please keep it coming, I am gonna binge watch all yours videos. Subscribed!

[@ChartExplorers]

Hi Manav, thanks for your kind words and support!

[@manangoyal5026]

Amazing video! Wish I could like it 1000s of times myself. Within 13 minutes, using rightly put graphics you've taught these 4 topics to me with such an ease, that's highly admirable! Please keep making more videos, the world could use more people like you :D

[@anthimostsirigotis5483]

Thank you man!!! In just 12min I managed to understand everything. I follow you from now on!

[@PeaceAction6464]

Very simple and easy to understand explanation, the concept has been explained in almost the first minute

[@chrisbagshaw5676]

Ugh I have a deadline tomorrow and this video actually saved me

[@ChartExplorers]

Wow, I'm glad this helped!

[@akshaypatil-qq9jv]

ya Saved me too 😅 Thanks

[@AnilAnvesh]

The best video on KZbin on #Python #Pandas Merge/Join/Concatenate/Append. Gratitude for your efforts ❤️

[@ChartExplorers]

Thanks Anil! 😀

[@PP-im6lu]

Wow. I can't believe how clear this explanation is! It helped me a lot! Just one thing though. It seems the concat (inner) is not exactly the same as the merge (default). concat (inner) seems to just get rid of the rows or the columns with NaNs, but merge would make sure there are matching values (whereas concat (inner) doesn't).

[@KarlosEPM]

Good observation. Also, merge, unless used on index, generates an entirely new index. This can be problematic if you are dealing with large datasets and are using few columns, relying on index for identification of values (rows). In the latter case, I create a column with the indexes (I hope someone smarter and more experienced shares a better solution :p). Cheers!

[@AlexFariaOliveira]

THANK YOU! I was days trying to merge 2 lists and was not finding a way your video made it all clear and half way though it I stopped and solved my problem!

[@quornnugget7799]

At 9:20 you spoke about the data type error. I am struggling with this as my table will not change the data type no matter what I try. Do you have any fixes for this?

[@Rishabh_Sar_Di]

Hey...why am I getting Error as 'You are trying to merge on float64 and object columns' while using pd.merge(df1,df2) ?? Though u could easily merge float64 and object columns !! Plzz assist

[@rolfbecker4512]

The explanations are easy to understand and comprehensive. Perfect! I will recommend your material to my students. Thank you!

[@ChartExplorers]

Thank you Rolf!

[@kamrulfst]

Thanks for your video about merge,concate method. Its more clear to me now.

[@sashi6]

Practical and concise explanation. Enjoyed watching your learning videos

[@ellyothim355]

Superb brother, Wish I met this channel earlier

[@leulsegedmesfin7239]

amazing explanation man,thanks.

[@munirbhatti1103]

You are making too good videos, Please make more videos on python, Data science and ML. It will be really helpful. Thankyou!

[@TippsausEuropa]

Coooool. Clear and easy and very useful . Best regards from Technischen Universität Berlin

[@hetpatel7918]

I was very confused about this topic until i watched this video great explanation

[@ChartExplorers]

That's great to hear! Thank you.

[@andthatswhy5198]

best quick and easy to follow tutorial

[@splashstrike]

You're a great teacher. Subscribed.

[@ChartExplorers]

Thanks!

[@hedibena3697]

I would like to thank you for sharing this knowledge in an incredible and such easy way! This really helped me, thanks a lot!

[@trustedai]

Excellent tutorial with lucid and simplified explanations.

[@EricJ-f9m]

Great video that just clear my questions!

[@darkmatter4701]

thank you for making this video it really helped with my problem that i was stuck in, thank you.

[@stevej9678]

i want to set columns side by side 'outer' even with concatenate and axis = 1 i am still getting nan values like in 12:12 instead of 12:20, . ... why ???? oh why?????, thanks a lot for this wonderful well explained perfect tutorial

[@hoaralatoan845]

Super clearly, a helpful explanation. Thank you so much.

[@RishabRebala]

BEST VIDEOOOO!!!!!🤩🤩🤩🤩

[@Olimpico230]

thank you so much sir, you saved me too much stress

[@vivekshindeVivekShinde]

amazing loved the visual explaination

[@АлександрХевен]

Absolutely brilliant 🔥 Thank you very much!

[@basicallyeditor]

Hi, I need to merge oldxl data and new xldata, label column is same in both xls I need to do outer join. If label value is same in both xls it should keep the row of new xl Can you pls help in this. Thank you

[@mohammadyahya78]

Thank you very much. Very clear explanation. At 4:26 please, is there a way to avoid comparing the row that has same ID from datrafeme 1 to all these that have same ID in row 2 please?

[@NikhilSangle]

I have to combine 34 pyspark df to one df and all df has different no of rows and only one column. Pls any one can let me know......

[@keerthanasrinivas474]

This was exactly what I needed. Thank you.

[@eldhose101]

Thankyou , this really helped me to understand.

[@Puma_178]

hi, I loved your video, very clear explanation and good example, thank you

[@helengh5493]

Hi , I love the video , that was so clear and useful would you please explain what is the best way to merge and or concat the data frame which are not created in current working directory , should we read them one by one or is there any other short way?

[@inthemakinh]

if we want to merge the text in all the cells of a row or column into a single cell, or just keep the merged cell then how can we do that.

[@josesanchezpalacios8991]

With pd.concat([df1,df2], join='inner') I was expecting to see Col_4 once ? why is repeated ? any way of avoiding that?

[@mooventhan110]

Neat and Clean Easy to understand Thanks man :-)

[@ChartExplorers]

Thanks! This is something that always tripped me up, so I'm glad you found it easy to understand 😀

[@freedoom4090]

I'm geting crazy... at 7:25 : (...) f = pd.merge([zero, a], how="right") (try to run and get:) TypeError: merge() missing 1 required positional argument: 'right' I don't know what else to do. Please, someone give me a light about were's my mistake, 'cos I'm about to broke my notebook in two :(

[@ChartExplorers]

It is looking for the second (i.e., right) dataframe. Because you added the brakets around the two dataframes it thinks it is one argument. Try it WITHOUT the brackets around your two dataframes. You want to use brakets [ ] with concat not merge. You will also need to specify what you are merging on f = pd.merge(zero, a, how='right')

[@freedoom4090]

@@ChartExplorers it worked! thanks! you saved me haha

[@ChartExplorers]

@@freedoom4090 Awesome, I'm glad it worked!

[@sarthakshandilya8412]

hey, thanx for such a good explanation 😊😊

[@aandyrea]

Thank you SOOOO much! This was perfect!

[@jimmyhilman5612]

Cannot use df.append(), dataframe attribute append not found, any ideas ?

[@rhysbennett1667]

This Helped me out so much! Thank you!

[@ChartExplorers]

I'm so glad!😀

[@BlueSkyGoldSun]

So Join and merge are just the same in pandas and they are equivalent for SQL joins ?

[@talalsafaq]

Ver Ver Very Good explaining ,, thank you very much

[@business_central]

Great video and love the visuals ! Just one question, any video/explanation on merge_asof techniques

[@richarda1630]

fantastic tutorial! very well done. I was a bit confused by the "join" section starting at 10.00 It doesn't seem to be joining on the index. Could you explain that a bit more?

[@ChartExplorers]

Thanks Richard. If you delete the on='ID' argument then it should join on the index by default. I see that you are on discord so I'll explain in more detail there.

[@richarda1630]

@@ChartExplorers Thanks so much! LF to it! :)

[@jananisivasankar9784]

kzbin.info... It's a great explanation on this topic..i understand it very well.. I also got confused in join topic after 10.00.. it's given on = 'ID' but default left and also inner join not happened based on the common values of that ID column..need some explanation here to understand this .pls help

[@Raja-cp8uo]

How to add two rows when column value same in pandas?

[@easydatascience2508]

you can watch mine. The playlists provide most of the fundamentals for Python and R.

[@Nightmare78hAlo]

A note for anyone getting errors for append, apparently that is now deprecated as of 2023/2024 for pandas , ie it's not part of pandas anymore so you can ignore append.

[@kawsydaisy]

Helpful! Thank you!

[@JuanPerez-iu9vk]

Great! Thank you very much!

[@TraderSam96]

Simplified,thanks alot

[@neuralNudge123]

hey, sooo useful tahnks. Could you please please please share the ntb?

[@vikram2105]

Very nicely explained

[@keifer7813]

Watching this on my lunch break because my manager thinks I'm an excel whizz. She told me to run some macros but really I'm using python to clean up some excel files lmao

[@lala-jy4kz]

Is there any source code? I want to use it as lecture notes to review.

[@sarbanchakraborty2053]

Nice explanations, Thanx

[@dishantshah4383]

Amazing Video.

[@samt3825]

it was really well said

[@BridgeHK]

Hi Your explanation is so clear and easy to follow. Do you have a Github and people can run the code directly? Thanks

[@ChartExplorers]

Hi @Bridge2019, thank you. Here is a link to my GitHub with the code I used for the video. github.com/bvalgard/combine-dataframes

[@BridgeHK]

@@ChartExplorers Thanks! You may put the Github link under your description next time. People can access it immediately . Thanks anyway!

[@ChartExplorers]

@@BridgeHK Great idea. Thanks!

[@bonefishboards]

Perfect. Thanks!

[@iliesboukhatem7803]

a great video, thank you so much

[@chilledvibes5700]

very efficient video.

[@injilaislam2275]

why does concat didn't match the value on basis of the column values in the inner join at 12.38

[@ChartExplorers]

I'm not sure what you are asking? Inner join on axis = 0 will match the columns that match between both dataframes. In this case that is the ID column and Col_4. It doesn't match with the original pd.concat([df1, df2)] because the original is doing an outer join, but the 12:38 example is doing an inner join. Does that address your concern?

[@injilaislam2275]

@@ChartExplorers hi thankyou for replying, my concern is regarding inner join, why here inner join is fetching values which are not common ,ie value from second index row, according to the law inner join only fetch common value , it more seems like outer join at 12.38

[@ChartExplorers]

Hi @Injila Islam, good critical thinking! What is happening here is that because axis is set to 0 we are grabbing the matching values based on columns not rows. You will notice when axis=1 and join='inner' (@12:30) what you are expecting is what happens. @12:38 however it is grabbing the similar columns (ID and Col_4) that are shared between the two dataframes.

[@magi4k]

Amzing explanation to be honest

[@pankajgoikar4158]

You are superhero

[@abegailherrera3337]

Thank you 😊

[@tarblood]

thanks Mr.

[@bonnyongom1176]

Nice 👍

[@nandineeuma6659]

How to combine multiple row into single row separated with comma using python

[@ChartExplorers]

Could you provide more details about your specific situation? Do you really want to combine rows? Or are you trying to combine columns? combining multiple columns: df['combined_columns'] = [', '.join(str(x) for x in y) for y in map(list, df[['col_1', 'col_2','col_3']].values)] For this switch out df with the name of your dataframe (two places) switch out the column names with your column names (here i have three columns but you could use more or less depending on your needs) If you are combining rows are you then appending that new row to the dataframe? or do you just want a variable to store that information? combining multiple rows: rows_combined = dfm[:5]['col_1'].values

[@nandineeuma6659]

@@ChartExplorers Thanks you for the response want to combine rows only for example if i combine values in A1:A5 cells and the concatenated values should be in B1 Cell, sample input is A1:X0007DFG A2:X0006FGH A3:X0005HJK OUTPUT X0007DFG,X0006FGH,X0005HJK

[@tindo0038]

amazing

[@hanyelshafey2722]

Thanks

[@ChartExplorers]

You're welcome!

[@shahzan525]

👍👍

[@richardnes1403]

👏🏻👏🏻👏🏻

[@jagdishprajapati9574]

.append has been deprecated

[@ryansossou646]

note:: append going to be remove from future version of pandas

[@ram-my6fl]

This is G.O.A.T shit

[@ChartExplorers]

Thanks!!!

[@NatalieCruz-h7k]

It's possible a better video than this one?

[@prizepig]

Exactly what I was looking for. Thanks!

[@InteligenciadeNegocios]

Thank you!

[@noone-kg8ry]

what if the identifiers have the same values but they don't have the same titel for example: product_id and item_id. How do i merge these sort of dataframes?

[@idongessien2245]

HI, I'm trying to merge datasets but keep getting memory error issues. How can I get pass that? Now my computer just freezes each time I attempt. I always have to forcefully shut down my laptop and reboot it all over again. still no progress made. Please help