25. What is Delta Table ?

If you need any guidance you can book time here, topmate.io/bhawna_bedi56743
Follow me on Linkedin
/ bhawna-bedi-540398102
Instagram
bedi_foreve...
You can support my channel at: bhawnabedi15@okicici
What is Delta Lake?
Delta Lake is an open source storage layer that brings reliability to data lakes.
Delta Lake is a data format based on Apache Parquet. Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing.
Delta Lake runs on top of your existing data lake and is fully compatible with Apache Spark APIs.
Delta Lake on Data-bricks allows you to configure Delta Lake based on your workload patterns.
It’s an open source project (github.com/delta-io/delta), delivered with Data bricks runtimes and it’s the default table format from runtimes 8.0 onwards.
You can use Delta Lake format through notebooks and applications executed in Data bricks with various APIs (Python, Scala, SQL etc.) and also with Data bricks SQL.
Delta features overview
Delta Lake is an open-source storage layer that brings reliability to data lakes.
ACID transactions on Spark - Delta is not RDBMS right away.
Scalable metadata handling
Streaming and batch unification - A table in Delta Lake is a batch table as well as a streaming source and sink.
Schema enforcement - Automatically handles schema variations to prevent insertion of bad records during ingestion
Time travel - Data versioning enables rollbacks and full (or rather up to your retention settings) historical audit trails.
Upsert and deletes - Supports merge, update and delete operations to enable complex use cases like change-data-capture, slowly-changing-dimension (SCD) operations, streaming upsert, and so on.
Data-bricks hands on tutorials
• Databricks hands on tu...
Azure Event Hubs
• Azure Event Hubs
Azure Data Factory Interview Question
• Azure Data Factory Int...
SQL leet code Questions
• SQL Interview Question...
Azure Synapse tutorials
• Azure Synapse Analytic...
Azure Event Grid
• Event Grid
Azure Data factory CI-CD
• CI-CD in Azure Data Fa...
Azure Basics
• Azure Basics
Data Bricks interview questions
• DataBricks Interview Q...

[@MonikaGuha-tx9ly]

Clear explanation in the very simple language. Thanks

[@rashmitripathi1583]

Appreciate your efforts . Very good explanations 👏👏.

[@shadabsiddiqui28]

Thank you so much, you are one of the best explainer

[@muhammedsuhail9865]

Great explanation 👌, got the idea💡 dont hesitate to create more content. Thanks

[@samk_jg]

Awesome Tutorial!! 👋

[@Rafian1924]

Data Queen. Thank god you exist.

[@phanikiran8288]

The best content on spark in youtube.👋👋

[@anandhu_sekhar733]

Wonderful effort👏👏👏

[@sravankumar1767]

Nice explanation 👌 👍

[@vedantshirodkar]

Great video. Thanks for all your efforts !

[@lifechamp007]

Super helpful - Thank you so much !! #StayBlessednHappy

[@rajdeepsinghborana2409]

Informative ❤️

[@karthigeyang3183]

Very useful 👍

[@maheshm-sh1dg]

Wow really it’s very helpful Sister

[@subhamdas5609]

Helped a lot..

[@avinashkaushik2313]

great video.. Maam

[@sravankumar1767]

Please do more videos on Delta lake, how can we read and apply busses logics

[@narendrakishore8526]

Great explanation. I got many concepts related to Delta tables. thank you

[@anujaambesange1149]

your videos are informative. Just a small request can you pls make use of micophone as the audio is too harsh for all of your videos.

[@RaviKumar-oy5jq]

Hello Bhawana, Great explanation👌👍. Just looking for some inputs b/n Delta table vs Live table in data bricks. See that both can process streaming and batch. What is the difference and whihc one is preferred?. Please share if you have any information on this .thanks!

[@JanUnitra]

How to read a specific delta partition? Or partitions for ex. Between dates?

[@vinuthaudupa4800]

What is the difference between .delta (databricks default dataset) and .parquet files (when we write using spark.write.format(delta)) delta tables ?

[@FranciscoRuizA]

Any chance of doing similar comparison with Apache Iceberg?

[@brijesh0808]

Please add the links to the other videos discussed in the tutorial.

[@user-vp3lf9lr6r]

Great content!! Just one request could you please share the notebook commands used.

[@brijesh0808]

It feels good when women are also creating youtube technical videos. Otherwise 95-99% are only men.

[@vishnupriya407]

Can we bulk insert delta table by ssms from blob storage?

[@Abbulikesplanes]

Hello - Can you please share how to deploy delta tables to other environments

[@sravankumar1767]

I have one doubt, if we have a files at adl gen2 we have to create mount point and work on the files. if we have delta lake is it required create to mount point.

[@cloudfitness]

Yes you need to mount

[@somesh512]

At 10:32 are the .crc files checkpoint file or after every 10 transactions we have get a checkpoint file created? I am bit confused here.

[@muhammadzakiahmad8069]

Nope they are .crc files, check point files are created in parquet format.

[@karmetpsg1986]

Nice explanation..Will you please share me the code used in this video ?

[@MrAnshrockers]

Very informative, best content on spark on youtube, thank you

[@sreelakshmikp]

❤❤

[@ashredsa]

ara maam your voice is too sharp keep the mic in some more distance

[@zaffer2024]

I wonder how she got 433 likes

[@suhel.choudhury]

Really poor explanation. You are all over the place.

[@ADY_SR]

mind turning off ur video? im getting distracted

[@karthigeyang3183]

Very useful 👍