21 күн бұрын
Пікірлер
@Haiti2011Harold 23 сағат бұрын
Great explanation
@giantbush4258 Күн бұрын
Bro where is your tripod?
@goldydoesyt 2 күн бұрын
Awesome vid
@joestrinka9738 2 күн бұрын
This is just reading the website with added motion sickness. Terrible content.
@techienomadiso8970 8 күн бұрын
Kafka should be compared to Rabbit Mq while Flink to Spark
@uchihadayne6506 8 күн бұрын
We couldn’t see the dog! 🥹 thanks for the vid. Ever look at Palantir as an option to an overall edp?
@KKKBarracuda 8 күн бұрын
Thank you for the video, I am just starting to learn airflow it is great knowledge, would be great if you could do a video about executors of airflow in depth and another video of airflow architecture in-depth include the secret backend and meta database, kind a confusing me with the purpose and practical use of secret backend since there is already a database.
@ccc_ccc789 9 күн бұрын
Thanks! You're doing a good job!
@cdgtopnp 9 күн бұрын
Man you provided so much info but documented none of it. If the background were a slide deck instead of a static image, the video would feel much more structured and engaging. Nonetheless you are a great orator and hope your channel grows exponentially !!
@sblowes 11 күн бұрын
Great explanation. Please invest in a can of WD-40.
@michael_day 12 күн бұрын
The more I hear about SQLMesh, the more I'm convinced. My org much prefers open source and thus we need a fuller solution from the get-go.
@yayif7699 13 күн бұрын
Hi this is helpful!! Do you have a github profile?
@hugoclarke3284 14 күн бұрын
Are ya winning JSON
@FhariyaAseem 15 күн бұрын
Amazing!!
@santoshkumargouda6033 16 күн бұрын
Please make a video on medallion architectures in data pipelines.
@nixondanielhutahaean44 17 күн бұрын
can you explain how is Airflow do in peoduction? Like how they deploy the DAG, collaborate for building DAG, and another production thigns
@razor-b2d 4 күн бұрын
different clouds have their managed airflow versions. ex: google cloud composer
@murugesanrajasekaran5032 18 күн бұрын
Thanks. Is there any GitHub link you can share to get the code snippets used in this example
@ballettyishappy2254 18 күн бұрын
thank you sir
@thedataguygeorge 18 күн бұрын
You're very welcome!
@shresthaupadhyay5739 18 күн бұрын
Hey curious me wants to know can we transfer 150 million records of data with that ?
@thedataguygeorge 18 күн бұрын
Definitely!
@JacobThorwarth 19 күн бұрын
Just getting started and and developing a huge interest in the field of Dara Engineering. I never leave comments but I think your content is simply amazing and invaluable, I have learnt so incredibly much from you, I cannot thank you enough for your effort and insight. Greetings from Germany ❤
@thedataguygeorge 18 күн бұрын
Thank you so much from New York!
@razor-b2d 20 күн бұрын
Can you run it
@ken-zlai 20 күн бұрын
Excellent video but the audio is a bit quiet :)
@thedataguygeorge 18 күн бұрын
Thank you for the heads up!
@whramijg 20 күн бұрын
so you came up with this all by yourself?
@thedataguygeorge 18 күн бұрын
All by reading severally articles online lol
@groundingtiming 24 күн бұрын
YEs, could you please update on Git? makes it easier to follow along
@Achilles585 24 күн бұрын
Could you upload it on git?
@luisrc99 25 күн бұрын
Thanks for this video! 🔥🔥🔥
@thedataguygeorge 18 күн бұрын
No problem, my pleasure
@rahuldsouza1985 26 күн бұрын
What about DB2?
@thedataguygeorge 18 күн бұрын
Never heard of it!
@canhnguyen9960 26 күн бұрын
Can you give me the source code in the video?
@jaimernandez94 27 күн бұрын
Hello, I use cdk to create all my infra. Then I have an airflow DAG that runs some tasks, including a lambda function created by cdk. I'm able to trigger the lambda form the DAG but I'm not able to wait for the lambda's callback to continue with the rest of the tasks present in the DAG after the lambda finishes, any idea?
@thedataguygeorge 18 күн бұрын
You should use a sensor to detect the completion of the lambda job as an intermediary step since the operator itself won't wait
@jaimernandez94 17 күн бұрын
@thedataguygeorge hey, thanks a lot. Btw, I wasn't able to do this with any type of sensor whatsoever. The solution for me was to override the LambdaInvokeFunctionOperator in order to be able to modify the tcp_keepalive, read_timeout and connect_timeout. A bit weird that there is not something out of the box that allows us to do it without complicating things this much. Happy to share if you are interested!
@rubayetsabbirfaruque3629 28 күн бұрын
I keep running into an error with the execution path. Even though I entered the container with astro dev bash and saw the dbt_venv, cosmos can't seem to it.
@thedataguygeorge 18 күн бұрын
Thank you so much and will do!
@BinPham-x1k Ай бұрын
you da goat my guy
@thedataguygeorge 18 күн бұрын
Thanks big dawg!
@itzcallmepro4963 Ай бұрын
can you recommend resources for tooics such as airflow,dbt,spark ?
@thedataguygeorge 18 күн бұрын
Not sure about dbt/spark but for Airflow check out this link! academy.astronomer.io/
@jeffrey6124 Ай бұрын
Wazzup! Captain America with eye glasses 🤓 I was searching for "Data Architecture" and Google recommended your video .... only your video!!! 🤩 say hi to Data Dog 😍
@thedataguygeorge 18 күн бұрын
Wow crazy i'm the only one out there! Thanks so much for the kind words, the Data Dog says hi right back!
@sohanmsoni Ай бұрын
Thanks for detailed video on comparison, but now I would go deep in Kafta steams vs Flink. Are they competing services with each other and that too from Same owner ? (Apache) And on a lighter note, time to do servicing of that chair or get a new one 😊
@thedataguygeorge 18 күн бұрын
Apache is just an umbrella open source organization, so semi-competing projects but also designed differently, and thank you, saving up!
@jay_wright_thats_right Ай бұрын
I feel like I'm being sold something. No doubt, thank you for your efforts.
@thedataguygeorge 18 күн бұрын
I don't work for either org, all unbiased!
@boseashish Ай бұрын
God bless you!!! may you get success
@thedataguygeorge 18 күн бұрын
Thank you so much!
@LuigiMolinaro Ай бұрын
You need a new chair :D
@thedataguygeorge 18 күн бұрын
Hahahaha yes this comment section has made that clear
@PengyuHou Ай бұрын
Hello this is Pengyu from the Chronon team. Great job on explaining the concepts and even getting a demo working!
@thedataguygeorge 18 күн бұрын
Thanks so much Pengyu, thanks for making a great project!
@infamousprince88 Ай бұрын
Very useful information! Are there end to end (or zero to hero) videos you’d recommend to get up to speed with this domain? I’ve been in and around data analytics for some years and have the Python, SQL, BI/Tableau portion. Just would like to see the Data Modeling, DBT, engineering, data source integration aspects
@thedataguygeorge 18 күн бұрын
Working on creating some myself right now!
@Wakeful_Being Ай бұрын
also i think the start and stop bits might be switched @4:26
@Wakeful_Being Ай бұрын
thank you!!
@joefitzy Ай бұрын
Thanks for the video, but when it came to DBT/Cosmos it was pretty unclear what was happening.
@thedataguygeorge 18 күн бұрын
Apologies, didn't spend too much time since I have other videos on it but great point, will improve in the future
@user-xx3zp3qr1k Ай бұрын
Nice guide! but i cannot figure out how to personalize this installation, i would like to deploy airbyte on my postgres, my previous installation was through a docker-compose file, now everything has changed! what can i do to "personalize" this tool like i did before with the compose file? i can't find a lot on the official documentation! thank you very much!
@thedataguygeorge 18 күн бұрын
You can customize the docker compose file for Airbyte too! They don't have great docs on how to do it but follows the same paradigms as other dockerized apps
@anibara Ай бұрын
I am just surprised with the quality of content you put out on regular basis. Thanks a lot and yes please do more AWS content.
@phethosilas8781 Ай бұрын
Is there a way to reach you? I would love to be mentored by you. All the way from South Africa
@thedataguygeorge 18 күн бұрын
Yes! Join the Data Guy discord I just created!
@thedataguygeorge 18 күн бұрын
discord.gg/JkjvyYmFcx
@phethosilas8781 18 күн бұрын
Thank you
@PhilipPetersen-c1j Ай бұрын
Shoutout Mr data guy 🙌
@dongtandung9671 Ай бұрын
do you have this on a repo so that we can take a look at the whole thing?
@itzcallmepro4963 Ай бұрын
great , any good source to learn airflow indepth ?
@TheMustafa-b8j Ай бұрын
Nice content can you share the link of the github repo of the project