Good 👍

Very nice

Hi Bro, I want a problem solution. Can you help me out on this? See, I have created a table with 4 columns and then I added one more column "salary" based on another column. But when I apply group by on derived column, it's getting error. show like - salary col can not be resolved. I just want to know if we create a col from an existing col in dataframe so if we want to use this derived col("salary") further for any operation. How can we do that? Hope you are getting what I am trying to say

I'M highly interested in taking the azure course.Could you please share the online course details and let me know when will you start the new batch is scheduled to begin?

I need your contact

can we use glue in free tier service?? let me know i hv to pratice small project

super explanation bro, can you share your id i want to talk with you tell me the way or platform

Anda bhurji

Hi In this video, you didn't selected json file format when you created outputjson, so how it is showing .json file?

it is the best video I have seen so far, thanks a lot. I have a question please, when you created the stored procedure linked service you used the sql server name, could you please mention from where can i get it?

sales decimal mein kyu hai

Json file is not in gdrive

Hi bro can you make video for step function

Bhai beginner log ke liye yeh video bahut faydemand hai iske sath aap vo resource delete karne ko bi note kardo varna beginners mistake karsakte hai.

Very nice

OK

Good 👍

Nice

Amazing

Hello Sir, As I have zero knowledge about pyspark,in this series what ever you are teaching,is it enough or should I study your other py spark module videos. Please suggest

The way of teaching is so good . Thank you sir.

Employe_department use case 1)upload emp and dept csv files in adls 2)check if the files are empty or not. If it is not empty process as below. Otherwise, display it as empty file and stop the pipeline. 3)Get all employee details from research and operations department and save the result in azure sql db. 4)get the details of managers belonging to sales and increment the salary of those employees you 5% and update it. Save the result as JSON file in ADLS Please help me to solve this Thanks in Advance

Nice

Please Help me Out through this ERROR Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'MANASAGADDI\SQLEXPRESS01', Database: 'sales', User: 'pizza'. Check the linked service configuration is correct, and make sure the SQL Database firewall allows the integration runtime to access. A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.) The certificate chain was issued by an authority that is not trusted

i think always inferschema is not a good way to do in production because sometime some of the column will be wrong so better we can create our own schema .while creating the dataframe set our schema in schema parameter

type converions ke erroe are hai uske vaje se pipline fial horha hai

what happens when failure is happening again and again at rdd2? BTw you explained is so well. I am learning a lot. Thank you

Excellent tutorial ..for a person from.any field ❤

I dont see the data tab in the left pane with the community edition login. What could be the reason

can you suggest some other good playlist to refer

Good explanation

Please make a video how to connect from SQL server configuration (permission, certificate authentication, reaming )and adf with linked service configuration because all are getting errors while test the connection in linked service

I am getting error while sql server to adf to build a linked service in test connection check """"Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'MADHU-VALLEPU\madhu', Database: 'AdventureWorksLT2017', User: 'mrk'. Check the linked service configuration is correct, and make sure the SQL Database firewall allows the integration runtime to access. A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.) The certificate chain was issued by an authority that is not trusted""" Please make a video on this or give solution for this it's helpful for all

is databricks free for only 14 days ?? sir

hi sir, small correction in the description ( import findpark-- spelling mistake )

Nice explanation

Thank you so much for your great effort, very helpful contents,,,Just one question, in really time scenario normally we are automating data loading activities so in that case how we can get help from parametrized table selection. In that case single pipeline will work or we need to create multiple pipelines for loading multiple tables?

upsert if --------- expression is (1==1) expression should apply for all the rows then it should be (emp_id==emp_id) ????

but how to copy multiple files

Thanks brother..great content

Hi, are there any upcoming videos in this pyspark playlist or all the concepts covered ?

for each operations it will create each dag graph or it will total one dag graph for both loading and tramsformation ??

bhai aapko idea hai kya macbook mei hum sql server docker ki help se chalate right, then sql server ko azure data factory se kaise connect karenge .. coz woh chl e docker ke through hai

can you do end to end azure data engineering project please, full video

Bro instead of filter we can use the same condition in join say as df1.dept_name == df2.dept_name and df1.max_sal == df2.sal.

Hi bro, Thanks for the video. I created df using simple json which you mentioned in the video, it is working. When I tried to create df with complex nested jason file, I am not getting expected output. Can you please create a video for complex json df example

Amazingly easy to understand! Keep making please!

Hi, Can you please share the slides or ppt that could be more helpful thx

Hi Brother - Keep uploading more projects & thanks !

How exist and count serve same purpose? And how union and union all serve same purpose?