Python Pandas Tutorial 8. Concat Dataframes

[@codebasics]

Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced

[@supremeleader8796]

Hi, your tutorials are very good for intuitive understanding of what the code is doing. I find that every time I have a technical problem and need to study for solutions, an indian lecturer will have something on youtube that helps. Thank you indian lecturers, thank you all for sharing.

[@gauravsharma-zb1ku]

One of the best pandas tutorial.Thanx alot.

[@codebasics]

Machine learning tutorials with exercises are available at: kzbin.info/www/bejne/nZ7Zp5Sll9Jqm7M

[@rajshah7940]

Dear Dhaval, can your lecture be able to crack any data analytics positions?

[@codebasics]

Step by step guide on how to learn data science for free: kzbin.info/www/bejne/jJ_CnqCFqraeiaM Machine learning tutorials with exercises: kzbin.info/www/bejne/nZ7Zp5Sll9Jqm7M

[@arturcarvalhodossantos7263]

I can't express how much your course is thrilling! I had no interest at all in DataScience, but you made me want to learn it more and more, never stop teaching.

[@vamshikumar7336]

My doubts are cleared totally with this tutorial, covered each and every small details.

[@flamboyantperson5936]

I love your videos. Respect you for providing free knowledge in this age of business. Thank you so much. You are doing a social work you will get blessings of people.

[@flamboyantperson5936]

For sure you will see good things happening in your life. You could become a great Professor. You teach everything in the simplest way I just love it. Thank you.

[@vfr45rfv]

Your tutorials are the best I have gone through so far, my sincere thanks to you!

[@callback_ai]

best basics tutorial I've used so far

[@shantanuhule8784]

Thanks you sir , your videos help me a lot , I am a second year student , I Know basic language like python ,c++, java , but I had no idea what to do after that ,, Your videos are helping me a lot and gaining more n more knowledge in python After the end of every video I feel like I am getting closer to my dream to become a data scientist Thanks you sir 🙂

[@codebasics]

All the best Shantanu :)

[@in_ashish]

Thank you sir 3000! I will update once I get my bread and butter, honestly enjoying learning like never before!

[@followthepassion1530]

Really helpful tutorials for the newbies like me....thank you so so much...

[@codebasics]

Glad it was helpful!

[@joshnoni5609]

Hello, At 12:04 in this video, I noticed you passed an index in both temperature and windspeed data frames. it was quite easy to figure out the order of the index. What happens when you're working with a dataset of millions of values and indexes. We cannot manually sort out each index. How can we go about that?

[@kashifaziz5019]

You can use merge in that case

[@ibtissamchouja7113]

thank you i was looking for this kind of tuto great job

[@hocexcelonline4943]

Thank you so much. Best pandas tutorial, very easy to understand due to your explanation.

[@tymothylim6550]

Thank you very much for this video! It helped me understand how the concat function works!

[@codebasics]

I am happy this was helpful to you.

[@bt78646]

This video is blurry. Btw,Thanks a lot for this great playlist.May Allah bless u and keep you on the right path.

[@darshikaverma9170]

Best and clear explanation. Thanks a lot :)

[@codebasics]

I am glad it was helpful

[@siddhantsachdeva3388]

Hi, at 10:16 when you concatenated with axis=1 it appended the dataframe with the 'city' column again. Is there a way to concatenate with axis=1 without this column appearing twice in your final dataframe? Basically I would like my final dataframe to have just 'city', 'temperature' and 'windspeed' (and not 'city', 'temperature', 'city', 'windspeed')

[@siddhantsachdeva3388]

thank you

[@faisalsal1]

Joining should be based on Column Values like Mumbai to Mumbai, instead of manually making sure if Mumbai have index 1 in first dataframe, it must have 1 in second dataframe as well. In a small set, may be possible but if 1000s of records, would be tedious to making sure that related entries in different dataset to have matching index value. Besides the above issue, overall good tutorial.

[@BlazeXth]

So, should 'city' be set as the index instead of the default numerical value?

[@harimurugesan8367]

difficult concepts are explained in a easy a manner

[@abhi9029]

Nice tutorial. Keep up the good work.

[@codebasics]

Thanks AK

[@SophiaandHaakenPedersen]

Thank you for your excellent tutorials

[@juandiegocaballero5404]

nice tutorial! Does anyone know where i can find some good exercise for pandas?

[@dpeduchem7393]

Great skill of explanation. Thanks

[@surajm6236]

merged_df = X.merge(Y, left_on = 'key', right_on = 'key') for easier way of merging two pds

[@Kitaplayolculuk]

your lecture is brilliant . Thank you ;)

[@basupm7558]

you are my God .. please let me know if you are condcting any training .. I am very happy to join your classes

[@jugrajsingh9450]

codebasics have u uploaded tutorials for simple python

[@oneaboveallthe7398]

when you use index at 12:24 you are using it for small dataset right when you will use millions of data then how are we going to solve the same problem bcoz we cant write the index of every columns in the big dataset, so how you will solve the problem??

[@CoolGuy-ez8lj]

Hi, I have a question @ 10.26 and 12.11 in the above video. You have concatenated using axis or index reference fine. How to ignore or avoid the city names getting repeated in as new additional row ?

[@CoolGuy-ez8lj]

Thanks !!!!

[@haneulkim4902]

what was the answer?

[@shivadumnawar7741]

Thank you so much, sir. great work

[@Al-Ahdal]

Boss, as requested earlier.... when I expect to get "Comprehensive Data Analysis Series" from you. Waiting for a detailed series covering all aspects of data analysis. Thank you Sir.

[@aashah747]

I have no words I can only say: "Matchless"

[@spicytuna08]

thanks for a great tutorial.

[@sandeepmesa]

Very clear explanation..however when i have small data set ican set index ..what if i am dealing with millions of data set..how to ensure the indexing is fine?

[@keshavswami6618]

sir , ar 12:10 , still we have 2 same column...how to delete that one

[@codebasics]

You can use drop function to delete a column, pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.drop.html

[@karthikak2193]

@CodeBasics, How to remove an 'CITY' column which showing twice?

[@malasubramanian9654]

I have a single table and I want to merge two-row cells together in a particular column. how do I do that in python?

[@girishjahagirdar4379]

Tutorial is very helpful. However I have a question on DataFrame. How to create a DataFrame when I have an URL that returns a JSON data ? Any hints would help

[@youcefyahiaoui1465]

Great tutorials. Except that I keep understanding that the speaker is saying "Not book" instead of notebook. Otherwise, perfect work. Kudos!

[@mickaelfonseca2962]

Thank you for the tutorial. However when i try on a specific application (one df containing only the collumn names, and another one with exact same column names) part of the data appears in new collumns (duplicate) leaving blank collumns. What could be the problem?

[@StrikerTide]

What's the difference between merge, append and join in general data world? A good video but would have preferred if the terms were not used interchangeably and confusingly.

[@codebasics]

append is used only to append using index which means you can add rows but not the columns in a dataframe. merge and join are kind of similar, they are more like database joins where you can join or merge two dataframess at column levels.

[@soumikdam6442]

Good Tutorial. Appreciate it. But Bangalore has lower temperature and humidity

[@akki6071]

Nice I got one issue in sheet 1 I have multiple data in column one cell like (1,2,3,4) many cells I have to compare them with another column in sheet 2 but sheet 2 I have only single digit number in column cell like (1) if sheet 1 matches with sheet 2 then should replace with new values which is also present in sheet 2

[@gileswhiting2891]

Manually amending the indexes such that the rows align before concatenating the data frames seems like a very manual solution, particularly given that each data frame could have thousands of rows... Surely this is a sub-optimal solution?

[@gileswhiting2891]

I wrote this just before the end of the video where he talks about "merge". IF you have the same question as me - watch the next tutorial lol!

[@codebasics]

👍😊

[@gokulyc]

how to remove extra city column in df at 11:03?

[@darshikaverma9170]

df.drop('City', axis = 1, inplace = True)

[@gokulyc]

@@darshikaverma9170 There are 2 city cols, i want to remove second one.

[@jiefu295]

@@gokulyc df = df.loc[:,~df.columns.duplicated()]

[@SOMEONE-eq5bu]

@@jiefu295 thanks

[@PhucPham-my2qb]

i wonder the file named DataScience, What is it for?

[@shaikansarbasha4169]

Sir you told axis=1 means row but you are telling now column.I have confusion sir.Please help me

[@debatradas9268]

thank you so much

[@rKingSalami]

how will you extract the month name from the date in one of the files you used?

[@codebasics]

If month is in mm/dd/yyyy format then you can do dt.split('/')[0] to get month

[@shobhamourya8396]

Yes, merge is more like the sql join than concat

[@jarirafaelcabarcasbolivar3946]

If I have three columns: year,month and day in a data set, is possible convert in a column ( date) year-month-day? I want to generate a series time, but the data set Have four columns!!!

[@codebasics]

Yes you can generate a new column by using apply function.

[@johnmwansa4180]

brother help us the video how to change this number 76896 to replac 2 to remove 8 to be come like this 76296 using python.

[@arkabose89]

Hey man, what if i have huge data frames with cities in different indexes? I cant surely type indexes like u did in the video.

[@techytalky3678]

I think you would have to look at the Merge video where he says that indices arent required.

[@bishalkumardas1169]

but you have not told about join inner/outer parameter

[@hayacheemaceema926]

Aoa. sir plz tell me how i can download csv file which you provide.. i try much butstill unsuccessful

[@hayacheemaceema926]

thnkx

[@yashpandey350]

without talking much about it let's start it🤣🤣🤣🤣🤣

[@haneulkim4902]

what is the difference with pd.append ?

[@codebasics]

Pd.append appends new dataframe only on x axis whereas concat can do it both on x and y axis. Pandas documentation explains this difference pretty well: pandas.pydata.org/pandas-docs/stable/user_guide/merging.html

[@jayshreedonga2833]

Thanks sir

[@SOMEONE-eq5bu]

👍👍👍

[@mayankverma7287]

thank you

[@jayshreedonga2833]

thanks sir

[@subee128]

Thanks

[@harshsinghv]

How can I retrieve the combination of most common value in two columns Lets Say Column1 Column2 A B C D A B now i want to return AB

[@BlazeXth]

A little late but in SQL you'd want to count the times each combination appears and then retrieve the max ammount of all those apparitions, i think. Surely there's something similar in Pandas that can do that, hopefully.

[@manishsharma2211]

Nice

[@shahlanoura4347]

good...

[@omharne1386]

like this

[@motiongraphic5145]

جميل جدا

[@jackroark6928]

Brilly!

[@codeministry]

How do I apply the index if I have thousands of rows?