BigData Thoughts

Пікірлер

@krutikajadhav3325 20 күн бұрын

How to calculate RTO for application server are active active but database is active passive

@surendar8637 20 күн бұрын

Well done..thanks

@maheshpanuganti6977 21 күн бұрын

Thanks a lot for providing good content... It was given good understanding about Meta data driven architecture. You covered metadata driven for data ingestion & Data Transformation, not covered for Data governance prospective. if possible, could you plz help on below... 1.How Data Dictionary useful for a ETL/ELT developer (means how to make use of while designing a pipeline) 2.What are the things to know about data dictionary, as ADF developer/Bigdata developer? 3.It would be great, if you can provide the resources links for it.

@rushikeshshisode5947 28 күн бұрын

very useful video......

@nsec1216 Ай бұрын

@12:04 read is Transformation or Action?

@shivarajmiraje6672 Ай бұрын

Great explanation ❤

@sky-i8d Ай бұрын

Absolutely amazing, the way content is delivered!!

@BigDataThoughts Ай бұрын

thanks

@sky-i8d Ай бұрын

Amazing explanation!! Just posting as a doubt, if someone can answer it will be highly appreciated. How do stages and jobs relate to tasks and data partitions? For example, if a job is split into multiple stages due to data shuffling, how does this affect task execution on data partitions? How do executors and cores manage tasks across different stages within a job? suppose a dataset of 2 GB is split into data partitions , and we need to perform a group by operation ...now since different nodes will have different chunks of data, I am trying to understand how that operation is performed?

@Jhola_lekr_chal_padhenge_ji 2 ай бұрын

couldn't have been simplified further! amazing explanation.

@BigDataThoughts 2 ай бұрын

thanks

@saidulchoudhury7172 2 ай бұрын

This is what I was looking for. Too good

@BigDataThoughts 2 ай бұрын

thanks

@VamsiKrishna-dy8ww 2 ай бұрын

nice explanation

@NirajChawale 2 ай бұрын

Well explained! Thank you!!

@ravindranaths513 3 ай бұрын

very good explanation

@atulgupta5181 3 ай бұрын

its standalone or yarn architecture

@durairaj4296 3 ай бұрын

I am searching this kind of topics from the web in long time.. Thank you very much.. And i appreciate your stuff..

@BigDataThoughts 3 ай бұрын

@@durairaj4296 thanks

@itriggerpeople 3 ай бұрын

Very informative ! As a ETL Tester, It helped clear my concept. Thanks Mam

@boseashish 4 ай бұрын

very useful video after a long search. thank you

@BigDataThoughts 3 ай бұрын

@@boseashish thanks

@Soulfulsubham 4 ай бұрын

This indeed one of the best channels to understand the spark optimization techniques...

@BigDataThoughts 4 ай бұрын

@@Soulfulsubham thanks

@TimePlaysLifePasses 4 ай бұрын

Thanks!

@AI-ML-IoT 4 ай бұрын

Thanks for sharing your thoughts.

@BigDataThoughts 4 ай бұрын

@@AI-ML-IoT thanks

@mohitupadhayay1439 4 ай бұрын

Please try to make a project where one can find all these challenges. These statements won't be printed in mind unless you do some Hands on.

@zub3rahmed76 4 ай бұрын

Nice mam

@KavyaPristha 5 ай бұрын

This is what I was looking for. Crisp, To the point and rich in info. well done girl

@BigDataThoughts 5 ай бұрын

thanks

@isaackodera9441 5 ай бұрын

Wonderful explanation

@Themotivationstationpower 5 ай бұрын

Really appreciate your hard work. Thank you for the great explanation.

@BigDataThoughts 5 ай бұрын

thanks

@shubhamdaundkar8327 5 ай бұрын

Hello Shreya, Can you make a video of hand on Data ingestion in AWS S3?

@yashawanthraj8872 6 ай бұрын

Can Node/Thread have more partition than no of executors, if yes where the no of partition information will be stored.

@gvnreddy2244 6 ай бұрын

Very good session mam if it was a practically show means it is very useful. thank you for your efforts

@BigDataThoughts 6 ай бұрын

Thanks

@nishchaysharma5904 6 ай бұрын

Thank you for this video.

@BigDataThoughts 6 ай бұрын

Thanks

@vaibhavjoshi6853 6 ай бұрын

Getting confidence in spark because of you only. Thanks so so much!

@BigDataThoughts 6 ай бұрын

Thanks

@ambar752 6 ай бұрын

To summarize, what the Datamarts are for a DataWarehouse, same are the DataMesh for a DataLake

@rovashri566 6 ай бұрын

How did you make such a good visual explanation? Which tool you used to draw sketches ? Pls guide 🙏

@muralichiyan 6 ай бұрын

Data mesh and snowflake same..? Data mesh and microsoft fabric same?

@Learn2Share786 7 ай бұрын

Thanks, appreciate it.. is there a plan to post practical videos around spark performance tuning?

@AI-ML-IoT 7 ай бұрын

Thank you for sharing your thoughts.

@BishalKarki-pe8hs 7 ай бұрын

this is not excatly asnwer

@ranyasri1092 7 ай бұрын

Please do videos with sample data sets so that it would help for hands on

@mindwithcuriosity5347 7 ай бұрын

Seems it is PAAS as mentioned on Microsoft website

@sanketdhamane5941 7 ай бұрын

Really Thanks to Good And Indepth Explantion

@BigDataThoughts 7 ай бұрын

Thanks

@sindhuchowdary572 8 ай бұрын

lets say there is no change in records for the next day.. then.. does the data gets overwrite again?? with same records..??

@BigDataThoughts 8 ай бұрын

No we are only taking the new differential data when we do CDC

@sunnyd9878 8 ай бұрын

This is excellent and valuable knowledge sharing... Easily one can make out these trainings are coming out of personal deep hands-on experience and not the mere theory ..Great work

@BigDataThoughts 8 ай бұрын

thanks

@Learn2Share786 8 ай бұрын

Thank you, pls also post some practical videos around the same topic

@AI-ML-IoT 8 ай бұрын

Thank you for sharing thoughts

@KiranKumar-cg3yg 8 ай бұрын

First one to monitor the notification from you

@shreyachakravarty1347 8 ай бұрын

Thanks

@ahmedaly6999 8 ай бұрын

how i join small table with big table but i want to fetch all the data in small table like the small table is 100k record and large table is 1 milion record df = smalldf.join(largedf, smalldf.id==largedf.id , how = 'left_outerjoin') it makes out of memory and i cant do broadcast the small df idont know why what is best case here pls help