this video is helpful for training people who are learning to transition to cloud computing.keep posting
@kabilannagarajan8 ай бұрын
Great work Brother, please add more projects like this end to end, you have covered 3 months project in 45 mins, great
@lifeitnow96006 ай бұрын
Great work...and most common scenario that's widely used has been covered....Keep posting such quality work in a simple and understandable to all viewers!!!!
@yadi4diamond7 ай бұрын
Your videos motivates an engineer to become a Data Engineer. Thanks for great teaching.
@revcuts46485 ай бұрын
Idk how to express my appreciation to you, this is very helpful. Main apse aur sikhna chahta hon pls videos dalte rahiye 👑
@hariprasad62522 ай бұрын
One of the best session I have ever seen this much simple and clear explanation and sir plz try to provide complete course for Azure data engineer .... 👏👏💐💐
@fredoderazin75419 ай бұрын
This guy is king! Thank you for making this presentation. It’s a very practical training project.
@gtanation47297 ай бұрын
Thankyou sooooo muchhh. Before this playlist i was 0 in this field but after watching all the videos of this playlist i am very confident that i can easily do azure data engineering. Thankyou againn keep uploading... We want more videos
@kirantulluri3 ай бұрын
This wonderful video, thanks taking time to explain in detail.
@user-rh3dr5tc8t9 ай бұрын
Great Explanation and easy to understand... I learnt lot process knowledge from this video. Thanks for sharing this awesome video. Please add more end to end videos for different scenarios.
@shabeerkhan37911 күн бұрын
Hi Brother - Keep uploading more projects & thanks !
@aneerabdulkareem15249 ай бұрын
Great Work. Good tutorial. Thanks for all your efforts
@kirigayakazutuu4 ай бұрын
You saved my life man, Thanks bro
@vedvikatechzone11 ай бұрын
Great work Brother, please add more projects like this end to end.
@USpilla1437 ай бұрын
Very helpful,thanks for making this happened.Really worthy Vedioes ,Thanks Bro
@user-gu5mn9zj5h5 ай бұрын
great work bro, so much usefull information. Thanks for the video
@srinivasaraog31803 ай бұрын
Very good information
@gauranshbhutani5928 ай бұрын
Amazing work my man, kudos, thank you so so much, Can you make more of such project based on healthcare data KPIs end to end and not use CSVs but connect data from Azure to power BI , Thanks a million, amazing project
@SyedliyakathaliAli2 ай бұрын
Neat and clear explanation, Thank you
@maheshvenkat92562 ай бұрын
Excellent work
@babarhussain70188 ай бұрын
You got another subscriber. Love your videos, very concise and easy to to understand. Keep posting😊
@rabbanishaik23639 ай бұрын
Very informative video. Good Job. Thank you so much.
@kumareshkrm709211 ай бұрын
Great work. Please keep do end to end projects like this.
@balamuralibattala328311 ай бұрын
Thank you so much for sharing the knowledge
@learnbydoingit11 ай бұрын
Thanks
@Rajesh-gm3fb25 күн бұрын
Good Job bro...I love you...
@apurvavaidya126 ай бұрын
THANK YOU FOR THE VIDEO KEEP DOING THE GREAT WORK
@bachushankaraih51887 ай бұрын
Good
@happyheart94319 ай бұрын
Million thanks..you are absolutely great..
@PraveenKumar-ev1uv7 ай бұрын
Awesome job..Appreciated
@keshavofficial45427 ай бұрын
Super explanation bro..keep it up ..
@srinuvaas-lf2xs3 ай бұрын
Great job. Thanks a lot !!
@saivems84528 ай бұрын
Suggestion - Go slow please. It's like chasing the topics....
@Optimus_prim333 ай бұрын
Good good.
@chiranthyn112610 ай бұрын
Great work.
@wajidturi10 ай бұрын
Great work
@harshpatel33847 ай бұрын
Good work
@user-cn4rq4mu7e6 ай бұрын
great bro
@swaroopblazing956811 ай бұрын
thank you for your contribution
@learnbydoingit11 ай бұрын
Thanks
@user-jl5ky6qe8o10 ай бұрын
Great work bro
@madhusudhanmm340610 ай бұрын
Good explanation bro and one small request could you please post the end to end steps on note format/bullet points/short point to point in notepad also?
@josemanuelgutierrez40955 ай бұрын
do you have any projects using event hubs , stream analytics ,or synapse analyrtics ?
@priyankadubey25283 ай бұрын
Could you please share some videos for SQL projects, which i can include in my resume. Thanks
@Akash1331511 ай бұрын
Great..thank you
@learnbydoingit11 ай бұрын
Thanks
@mnirani523011 ай бұрын
Thanks pls upload more projects
@learnbydoingit11 ай бұрын
Sure
@sanskritisharma544711 ай бұрын
Thanks this is really helpful
@learnbydoingit11 ай бұрын
Good to hear
@digitalsevasolutions10 ай бұрын
Very helpful
@dhananjaymali62424 ай бұрын
great explanation only one doubt, databricks community edition isn't available now then how to practice databricks portion
@learnbydoingit4 ай бұрын
Will make video for that
@nareshm94737 ай бұрын
Great efforts. One time process is fine,but what if we want this to happen as a continuous process, I mean refresh of data every 1 hour?
@learnbydoingit7 ай бұрын
Do you mean hourly refresh ?
@vallepumadhu36508 күн бұрын
Please make a video how to connect from SQL server configuration (permission, certificate authentication, reaming )and adf with linked service configuration because all are getting errors while test the connection in linked service
@learnbydoingit8 күн бұрын
Ok I will upload
@vallepumadhu36508 күн бұрын
@@learnbydoingit below i commented exact error
@chitrarekhatiwari662911 ай бұрын
Thank you so much for this
@learnbydoingit11 ай бұрын
Thanks
@fahadmahmood71509 ай бұрын
Very Great explanation.! Could you please post some more content where u have moved the data from on-premise to ADLS gen using incremental/Delta approach. Once we did the full load on daily baisis i want to load the Incremental / change data based on last modified date of file which we have generated and take the MAX time stamp of source column.
@learnbydoingit9 ай бұрын
Incremental load video is a available in channel
@mukuljha974011 ай бұрын
Thanks a Lot Bro.
@learnbydoingit11 ай бұрын
Thanks 👍
@mareswararaomalle99582 ай бұрын
Please do video on health care project
@nikhilrothe34198 ай бұрын
Thank you bro
@TheHani8211 ай бұрын
Kindly maintain low pitch voice while talking in the video
@learnbydoingit11 ай бұрын
Sure will take care.. Mic prblm
@prateekkumar-xc6uo10 ай бұрын
Can you also show us how to pass key & scope instead of passing directly ?
@learnbydoingit10 ай бұрын
Will upload that too
@hassansaleem3784Ай бұрын
On second slide you have mentioned ' We have to build one pipeline which will transfer data and run daily' What do you mean by run daily ?
@learnbydoingitАй бұрын
Daily schedule
@hassansaleem3784Ай бұрын
@@learnbydoingit But how the pipeline will trigger daily ? Because we haven't set any schedule!!
@someshpatil41516 күн бұрын
@@hassansaleem3784 u need to setup trigger for that. refer any schedule trigger video that will help u to go further
@ruckyA11 ай бұрын
What is the benefit of using data bricks in here , i think you can also use power BI to do some data processing and manuplation
@learnbydoingit11 ай бұрын
Yes but if you have millions of records and that too keep on increasing so we create aggregate and then we do reporting on top of it... Here we have less columns and less data so directly u can use power bi no doubt.. But think about big data
@user-dq4yf6mk7m11 ай бұрын
Thanks alot
@learnbydoingit11 ай бұрын
Thanks
@mihirit71377 ай бұрын
I wanted to know if the pipeline is real time , i.e. if there are changes made in the Flatfile or SQL or data is added will it be reflected in powerbi ?
@learnbydoingit7 ай бұрын
Yes
@user-pk8iz4bu7c7 ай бұрын
Can anyone provide the notebook document for databricks part.
@SH-cy6lh11 ай бұрын
Bro are you on LinkedIn?
@bhartimankar40522 ай бұрын
Hello! I really appreciate this excellent tutorial. Thank u for the amazing content. However when I tried to copy data , I encountered an error "Schema import failed: Please select a table" . however my table is showing and selected still got that error in 12:03 .Could you kindly assist me with this issue? Thank you in advance for your help!
@learnbydoingit2 ай бұрын
Do select import schema as none while selecting table I have created video for this error yesterday please do watch
Thanks for your fast reply and all your efforts. Its worked .
@user-gt8bz4tw6u3 ай бұрын
Power bi needs to data engineer jobs? ,i think its for data analytics right
@learnbydoingit3 ай бұрын
Yes it's for analytics but in few companies you have to build report also
@SomeOne-qv2tfАй бұрын
aggregate table creation error: IllegalArgumentException: All week-based patterns are unsupported since Spark 3.0, detected: e, Please use the SQL function EXTRACT instead
@learnbydoingitАй бұрын
Not sure about this error ...which part u are stuck
@yashsharma-td6ii6 ай бұрын
This project is done on azure cloud right ??
@learnbydoingit6 ай бұрын
Yes
@World_Exploror10 ай бұрын
Please share notebook it will be helpful
@Rahul-wu1ig6 ай бұрын
bro when we say we need to learn python for azure data engineering. Is that mean that we are learning python for pyspark? please clear this doubt, coz I understand pyspark, but I don't want to do python in deep, I know the basics of python.
@learnbydoingit6 ай бұрын
Yes pyspark for data engineer..
@Rahul-wu1ig6 ай бұрын
@@learnbydoingit so it means we are learning python just to understand pyspark and implement it?
@user-gg5kt7qy6b4 ай бұрын
one interview qs from tech mahindra is how you give the names to tables in sql ,comment the ans pls
@Canadian_DrizzlesАй бұрын
Hi. I can't connect Data Factory with the server. I couldn't find the username and password. Can you please help me to fix this issue?
@learnbydoingitАй бұрын
Hope which creating server u have given authentication method and pass username and password
@matthewmark72249 ай бұрын
great video but i was unable to use the integration runtime using postgresql. how do i download the sql you are using? is it free?
@learnbydoingit9 ай бұрын
Yes it's free use sql server download in google
@matthewmark72248 ай бұрын
@@learnbydoingit ok thanks. it worked.
@logsofzany5 ай бұрын
what about the pipeline bro? doesn't require?
@logsofzany4 ай бұрын
bro just told me, I love you but as a friend by liking my doubt comment but not answering it💀💀🤡
@YogeshSingh-b1v13 күн бұрын
any idea how to connect sql server via azue data studio to azure data factory in macbook
@learnbydoingit12 күн бұрын
I haven't uploaded video but you will get similar video in youtube
@YogeshSingh-b1v14 сағат бұрын
@@learnbydoingitthanks for reply but i dint find any if you can share really would help me
@dianeavwenaghagha119328 күн бұрын
what happens if connection is not successful after download of the integration runtime
@someshpatil41516 күн бұрын
reinstall and try again it will work
@mainuddinali95619 ай бұрын
i m not able to find user name, server name and password in my sql
@learnbydoingit9 ай бұрын
Will upload video for that
@mainuddinali95619 ай бұрын
@@learnbydoingit if it is possible,plz do hurry
@user-pk8iz4bu7c8 ай бұрын
Facing same issue. Unable to get the password from SQL server. Please reply urgently @learnbydoingit
@Kalyan9126 күн бұрын
What is the title of the project so that i can add it in my resume?
@learnbydoingit25 күн бұрын
Data migration and analysis
@zahidalam78315 ай бұрын
How to connect sql server and what would be the user and password , i already installed sql server management ???? Kindly help me out .
@learnbydoingit5 ай бұрын
While installing there is process to setup login and password and if you havent done then also there is process to add
@zahidalam78315 ай бұрын
@@learnbydoingit how to do now, it's already installed sir
@sandeepivs26 күн бұрын
Please share us the ppt
@user-fm8qd8bu6m7 ай бұрын
hie can i do this project entirely on the cloud without on premises servers
@learnbydoingit7 ай бұрын
Yes use sql db in azure
@yogeshvanare1336 ай бұрын
brother can you send this power point sheet
@LuckyAzherАй бұрын
integration run time issue cannot connect to sql server
@learnbydoingitАй бұрын
Have u created Integration run timr
@LuckyAzherАй бұрын
@@learnbydoingit still no struck to that error since that day now the latest error im getting make sure sql db firewall allows integration run time to acess please help
@insidehead6 ай бұрын
How much amount I will be charged to do perform these workflow?
@learnbydoingit6 ай бұрын
SQL server free , power bi free , databricks free edition and only for azure if you have not created account you can create 30 days free and practice.
@insidehead6 ай бұрын
@@learnbydoingit I know actually my ac expired. That's y I asked
@sanjud1986 ай бұрын
i am unable to mount using this way. Can you please help me in this regard
@learnbydoingit6 ай бұрын
What is the error need to debug
@learnbydoingit6 ай бұрын
What is the error need to debug
@asthasatija-f1yАй бұрын
What does KPI means?
@learnbydoingitАй бұрын
KPI stands for Key Performance Indicator, which is a quantifiable metric used to track progress towards a specific business objective.
@madheshrajs167511 ай бұрын
Bro the azure services you are using is free of cost?? If not, is there any way to use it free. Can you please help me on this. I'm trying to do by practice but stuck in the above questions.
@learnbydoingit11 ай бұрын
Free for one month
@user-pq7yc3fl7o11 ай бұрын
hi where is the data file
@learnbydoingit11 ай бұрын
It's there in telegram and description also I will add in hour
@vijaykanaparthi98011 ай бұрын
Ur lecture so fast...
@learnbydoingit11 ай бұрын
Will take care but each step wr have covered you will definitely like... You can slow KZbin speed
@therestfulmedia3636 ай бұрын
How to check sql server password I can see user name but password I can’t see in my SSMS
@learnbydoingit6 ай бұрын
Is this windows authentication?
@learnbydoingit6 ай бұрын
While setting up ssms there is option.to setup username and password... Second way is to add login ur name and password
@asitadeka4010Ай бұрын
can anyone help me if i wan to put this project in resume what description shoulf i write
@learnbydoingitАй бұрын
U can mention about data pipeline , pyspark databricks and visualization part
@Sandy-zn7vy11 ай бұрын
I m not able to find in SQL server
@learnbydoingit11 ай бұрын
Will upload today
@sejalahire137010 ай бұрын
@@learnbydoingit have u uploaded it sir? i m not able to find server name and ur vdo related to that as well..please help sir
@learnbydoingit10 ай бұрын
@@sejalahire1370 server name is while login you see name in sql
@SomeOne-qv2tfАй бұрын
i got 3 errors 154 commented memebers not got any single error woow great please help
@learnbydoingitАй бұрын
What is the error
@therestfulmedia3636 ай бұрын
11:31 what is username and password, how can I know
@learnbydoingit6 ай бұрын
Sql ?
@therestfulmedia3636 ай бұрын
Yes
@learnbydoingit6 ай бұрын
If urs showing windows authentication then there is process to setup user name and password you can search in youtube
@Sandy-zn7vy11 ай бұрын
Where to find SQL server user name and password
@learnbydoingit11 ай бұрын
While installing we have option to setup or you can add in logic and do i will upload video for that
@Sandy-zn7vy11 ай бұрын
@@learnbydoingit please make a video on this
@grandhisandeep00710 ай бұрын
Please keep the ppt we will verify
@learnbydoingit10 ай бұрын
What you want to verify?
@learnbydoingit10 ай бұрын
Uploaded in telegram
@shyammaths57057 ай бұрын
could you provide csv file
@learnbydoingit7 ай бұрын
Available in description
@user-xo7ju5rr8z5 ай бұрын
Speak slow😮
@learnbydoingit5 ай бұрын
Will try that
@user-xo7ju5rr8z5 ай бұрын
@@learnbydoingit thanks for accepting...
@user-xo7ju5rr8z5 ай бұрын
@@learnbydoingit Sir I have one question..I joined MNC as a fresher and its been 2 and half year.But I Have worked only for 10 to 11 months and around 4 to 5 was training period. Other than this I was on bench and they are not putting me in project. But Now I planning to change the company but I dont know how to clear interviews as I didnt work much..Dont know what exactly the projects Are...Please give me an idea how to prepare I have knowledge on sql, Sql server, Power BI, Adf, Adb, Synapse, Snowflake basics. Please help me sir😌
@shaqibiqbal18397 ай бұрын
Hey at 22:59 , I am doing same step but i am getting an error that "SyntaxError: invalid syntax (, line 1) File :1 select * from online_sale_analysis ^ SyntaxError: invalid syntax" Even though i have created "df.createOrReplaceTempView('online_sale_analysis')" and it is successfully done. What is this error ?
@shaqibiqbal18397 ай бұрын
I have done that mouting part and created df
@learnbydoingit7 ай бұрын
Do check df is created or not
@shaqibiqbal18397 ай бұрын
yes it created even though i can see dataframe data@@learnbydoingit
@shaqibiqbal18396 ай бұрын
yes it is created i can access my df and can see df data @@learnbydoingit
@shaqibiqbal18396 ай бұрын
Yes i checked df is already created and able to access df but still having same issue@@learnbydoingit
@manasactis4-99965 күн бұрын
Please Help me Out through this ERROR Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'MANASAGADDI\SQLEXPRESS01', Database: 'sales', User: 'pizza'. Check the linked service configuration is correct, and make sure the SQL Database firewall allows the integration runtime to access. A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.) The certificate chain was issued by an authority that is not trusted
@learnbydoingit5 күн бұрын
Check linked service and connection string to access ir