125. Databricks | Pyspark| Delta Live Table: Data Quality Check - Expect

[@uttamkumarreddygaggenapall2070]

Good to see you back.

[@rajasdataengineering7585]

Thanks

[@anishdahiya5811]

Thank you Guru. For creating one of the best databricks playlist 🎉.can you create playlist for one end to end implementation of matilion architecture?

[@rajasdataengineering7585]

Thanks for your comment! Sure I will create a video for end to end implementation of medallion architecture

[@sumitchandwani9970]

please create a video on CDF and how it is helpful in structured streaming

[@simanchalmaharana2927]

Please give one session over salting techniques.

[@rajasdataengineering7585]

Sure, will create a video on salting technique

[@prajanna9696]

Hai sir ..Iam intrested to join dataengineering course can share your contact

[@sanrhn]

thanks for your vides as DLT is but confusing topic for me, can you create a video on DLT constraint in SQL and Python in this DLT series

[@rajasdataengineering7585]

Sure, I will create a video on this requirement

[@ravulapallivenkatagurnadha9605]

add more videos on this delta live tables

[@rajasdataengineering7585]

Sure, will do

[@sravankumar1767]

Superb explanation Raja 👌 👏 👍

[@rajasdataengineering7585]

Glad you liked it! Thanks Sravan

[@tamijuddinaymadar9674]

Can you pls create an end to end project using ADF pipeline where we get data from s3 bucket, transform the data using Databricks and store in a sink incrementaly following medallion architecture.

[@shravyakulal5756]

Do you have any complete end to end course with project?

[@AkashMishra-hn5qj]

Hey! I have a doubt. In the past I have used Hadoop and hive to process data and we used to create partitions/ buckets for large files sometimes. Do we need to create those when we use spark, since when we load big files in spark data is automatically partitioned or we can use repartition/coalesce? so the concepts of partition/bucketing applies here also?

[@rajasdataengineering7585]

Concepts of partition and bucketing are applicable in spark too

[@b.yussef4875]

Hi, thanks sir for your efforts. i need your help if you have any idea or roadmap to help me with it: i am doing now my last year internship and the topic that the organization gave me is : THE USAGE OF AI TO IMPROVE DATA QUALITY

[@naveenkrishnaonly1523]

Any separate batch available to join

[@dasariudaykiran5787]

I have one doubt sir already i learned python my SQL html css javascript bootstrap so i shift to azure side so my skills useful or not

[@rajasdataengineering7585]

Python and SQL are much needed skills for azure data engineering

[@dasariudaykiran5787]

@@rajasdataengineering7585 sir where are you from sir

[@amanpirjade9]

Make azure data engineer course

[@djsash8999]

Are you taking any courses? I would like to enroll. Can you share your email or contact?