Optimizing Your Data Infrastructure - How To Become A Better Data Engineer

[@insightsxdesign]

Excellent information! Great job 🙌

[@SeattleDataGuy]

Thank you!

[@thedatadoctor]

Another great one! I have also found Estuary to be a great solution to reduce cost on data loading... But as you said, usually we set it to run in batches to reduce cost on the warehouse side.

[@SeattleDataGuy]

Do you have any real time use cases youve implemented. Always curious to hear where people are using real time

[@Yavin4]

Amazing video. One of your best. Time to data access is a huuuuuuuuggggggeeeee factor in your data cost. How often do you need to access the data? The accurate answer to that question can save a corporation literally millions. I've seen data that's hardly ever used accessible in a second instead of being put into deep storage or better yet completely deleted.

[@dougdataeng]

Great job! Keep going and bring us valuable content

[@SeattleDataGuy]

Thank you! Will do!

[@nicky_rads]

great video, always important to understand the cost implications. Keep it up

[@SeattleDataGuy]

Thanks, will do!

[@firefoxmetzger9063]

Another good question is: "If we turn this off today, how much money would you loose?" This is a good way to get business people thinking about value. Paying 200k may be a lot but if its "cashflow positive" its nom-critical to fix. If it isnt you now have the entire business supporting your optimization effort and will face much fewer rodeblocks while exploring alternatives.

[@avikchatterjee7854]

Amazing video as always. Sorry to be a bit cringe and annoying but do you think it will be better if you lower the exposure of your camera setting a bit?

[@SeattleDataGuy]

Thanks for the support. Hmm, what do you mean, like remove the blur?

[@avikchatterjee7854]

@@SeattleDataGuy No your face looks a bit too bright imo, bg blur is fine

[@SeattleDataGuy]

@@avikchatterjee7854 Oh yeah I filmed this one a while ago. I have tried to reduce the brightness in other videos see this one - kzbin.info/www/bejne/nXiapqOda7aUeKs

[@robertlong6]

You say in the video you'll talk about data modelling in January. You okay your videos a year in advance? 😂

[@SeattleDataGuy]

hahahha! I filmed the video in January...I gotta stop referencing other videos or get faster and putting out videos... I wanted to put out this data modeling video -kzbin.info/www/bejne/nXiapqOda7aUeKs

[@internetexplorer1593]

My organization has an issue with interdepartmental meetings with lots and lots of different excel spreadsheets/reports that are only on people's local devices. This turns into a "my data" vs "your data." Unfortunately, we haven't collectively agreed upon a data source "gospel." There are these completely silo'd legacy systems that have permissions barriers, so there's not an obvious stand out source to immediately make "gospel truth." Any suggestions on how to pitch a centralized data source to be agreed upon? Any advice helps. Great channel, amazing content