What is a Data Lake?

[@MayankDhuria]

I was way too distracted by the way he was writing in reverse. I was switching between being in awe as how well he was writing in reverse, and his explanation. Thus, I have to watch it again after accepting the fact that he is a genius, and then finally get to understand everything. Great explanation.

[@aacastrocomandante]

I think he was writing normally and the video was flipped/mirrored afterwards.

[@dreamdrifter]

He writes normally on a clear glass screen, with a camera on the other side recording it in reverse - then flips the image. Still pretty genius.

[@karrcatmusic5360]

Amazing that you can write mirroring letters ....

[@IBMTechnology]

Read this post on our community page that explains it all ibm.co/2SA1vGd

[@hdebbache2000]

You know you can invert in post processing right? Lol

[@jpro2222]

@@hdebbache2000 He is actually left handed and married in real life but not in this video jajaja

[@leonardofriedrichmagro3785]

@@hdebbache2000 But it would not have the same impact...for sure, all of viewers take a look on that and said: hes good!

[@danzinde]

I think the concept of data lake should have just focused on explaining the "Store" box.

[@datasleek7950]

Interesting but Data Lake is not only used by ML. It usually used to store unstructured raw data. Some governance can be applied, however, you don’t build Dashboard out of the data lake. You first need to model that data into a Data Warehouse using dimensional modeling (allowing you to extract different dimension of your data). This multi dimension represented by few tables will allow you to slice the data in multiple ways, making reporting, thus dashboards easy to build. This is why Airbyte/Fivetran + Snowflake + DBT are the most popular data stack on the market right now.

[@casuallistener618]

😯

[@enyart91]

You almost had me with the mirrored letters, until I realized- your wedding ring is on your right hand in this video. Very nice camera trick.

[@b00psie]

it could be on his right hand. not everyone wears it on their left.

[@nIrUbU01]

@@b00psie yes it could be, but it most likely isnt

[@sreenivasamadenahall]

Good explanation, thank you. However, talk can get started with "Big Data" - which means data lakes are intended to store, manage and serve large Big volume, variability, velocity. Data is ingested in native format. It need to be kept organized, controlled and managed - governance. Data needs to be served in native or processed further for other needs - reporting and visualization, recommendations, process automations and more. Some real-life use cases to start the discussion. If the viewer already knows bits of data world (databases, datawarehouse, data lake etc), this helps to consolidate that understanding.

[@BlackLibertyGT]

Need to explain this to Snr Management, this video is very helpful in breaking it down into something I can explain to others.

[@johnnytorres277]

Why is it that every data lake explanation is full of theory without any concrete examples? Aren't all of us here because we're SQL or Cube programmers and want to know whats so great about Data Lakes? All I see is the same thing I do with sql databases: import the data, prep and transform it and then query it directly or create dashboard applications.

[@joshuamintz8852]

Hi Mauro, does this help? kzbin.info/www/bejne/f4HOgqN4mcmYa7s

[@kuppaigopuram8751]

I agree. This video is very similar to what we do with SQL. He has not really told what data lake is. But, whatever he told is true about Data Lake. Here is a list of differences in Data Lake that are not possible in standard SQL based RDBMSs. 1. Big Data 2. On the Cloud (this is possible) 3. Separation of Data from Data Processing Engine 4. Self Service Model 5. ML (this can be done) 6. Data in native format (csv/parquet/json/avro/...) All the above are common to Big Data. Here is the list of data lake differentiator. 7. Central Repository; means single source of truth.

[@surfh3r0]

data is really the new oil! nice explanation

[@bl8nc]

Great presentation, thank you!

[@IBMTechnology]

Thanks for watching Malcom!

[@carthur]

Very clear and helpful. He writes like Ira Joe Fisher

[@DaRyteJuan]

"Data Lake" a piece of unnecessary jargon that adds nothing to the conversation. We've been dealing with these principles for decades already.

[@miloslekic1]

Great explanation, thank you!

[@ashtarathena488]

Mirroring letters was a bit spooky/distracting at first. But great and simple content. Thanks.

[@meryemLux]

Thanks, that was very informative

[@ahmeddraz962]

Simple and very informative. thx alot

[@kekuko]

Very clear and helpful. Many thanks!

[@IBMTechnology]

Glad it was helpful Brandon!

[@Ali-ds5iy]

Man this is an awesome idea to stand behind the mirror and write it..👍🏼

[@KuldeepSingh-cm3oe]

Very good explanation

[@IBMTechnology]

Thank you, Kuldeep!

[@jeroboam4486]

I still have no idea what concretely is a data lake!

[@mzimmerman1988]

thanks

[@Thee.Mighty]

Left-handed.... It explains everything

[@lounacrea8179]

He s actually using his right hand to write

[@MasterofPlay7]

ibm is still using the traditional rdbms right? what about hadoop?

[@hariwaz100]

good explanation

[@pogococo2246]

do you use data warehouse at the "store" phase?

[@justincooke5888]

So does a Data Lake fall under Document Store due to it ingesting all types of Meta-Data such as Audio /Media , and text ?

[@BourbonDrinker]

Great video. I posted this to my LinkedIn.

[@IBMTechnology]

Hi Cotton Hollow Distilling! Thanks for the link love! -Sai

[@ajathaindira]

awesome !!!

[@aeremthirteen2771]

I dont think this actually explained the concept of Data Lake. Is it just a simple design pattern?

[@centurion09]

What does “infuse” mean in this context ? I could not find an answer searching on the Internet.

[@sase1017]

Infuse to business decisions for managers (Dashboard), consume by other part of the service in an app(Application), or automate to make the entire process smarter with AI (Automation)

[@shantnuchaturvedi5080]

Nice Fancy Arrows on your diagram

[@VaibhavPatil-rx7pc]

E X C L L E N T !!!

[@bavuvan6298]

Thank you! can you make video compare with data lake and data warehouse?

[@IBMTechnology]

We're glad you enjoyed it! 😃 We'll pass your feedback on to our team.

[@adityaprasad465]

Thanks, but I don't understand how this differs from data warehouses and ETL.

[@RAJATTHEPAGAL]

Actually it doesn't, what makes it largely different is the kind of features a data lakes gives. Its catalogues data and makes it more usable traceable for external data operations. So ya u can say I can simply extract data out of my warehouse/etl system and then operationile for my spark jobs .... Chances are in a data lake solution this solution is already inbread in it with it's own ui or api for easy operationalisation ( spark job related transformation of data , munging cleaning etc) ... A data lake is a full blown solution more importantly an overlay over the existing data infrastructure u have. Maybe an onprem hadoop, or clustered mongodb. A data lake software should primarily be able to create a single view of these and make sense of it. It's a thin line but the data lakes are supposed to be more organized.....

[@David-2501]

I would say it depends on the underlying tech. Data warehouses (DWHs) and Extract-Transform-Load (ETL) is focused on relational databases (Postgres/Oracle/Microsoft/MariaDB/MySQL/SQLite), whereas a Data Lake also includes "Not Only SQL" (NoSQL) technologies like Kafka (data streams), Hadoop (Document Store/csv file storage), Impala (SQL query engine for Hadoop), etc. When it comes to concepts, it *heavily* overlaps, IMO.

[@sachinfulsunge9977]

@@David-2501 I think there's a correction, DW are not focused of relational but dimensional databases

[@scottswanson5358]

Next time you do this you should write in the direction relative to yourself then just upload a horizontally flipped video.

[@IBMTechnology]

Hi Scott...actually that is what we did :) Check out this blog post for the details: kzbin.info/door/KWaEZ-_VweaEx1j62do_vQcommunity?lb=Ugzf5SL_yh9NglCJzgF4AaABCQ

[@shaunfrench5057]

@@IBMTechnologyThe wedding ring on the "wrong" hand gives it away. Can you post-production CGI it onto the correct hand? ;)

[@read89simo]

Data about data, is that same as metadata?

[@joshuamintz8852]

Yep!

[@GianetanSekhon]

Hope all the window panes in your office have not become data lakes?

[@khalidjaradat]

where the data lake ?

[@MrBamshy]

More like what is Business Intelligence

[@peliusrex2730]

Why dont' you just write forwards and then flip the video?

[@mariam7677]

tbh it wasn't really clear(ive been watching several vedios from this channel but this one is not clear)