Introduction to Workflow Orchestration with Prefect- Kevin Kho | SciPy 2022

Workflow management systems are used for scheduling and monitoring data pipelines. This includes managing task dependencies, retrying failed tasks, and sending notifications to users. This talk will show data engineers and scientists how to orchestrate their data workflows with Prefect, an open-source modern workflow management system designed with Dask natively built-in. After this talk, attendees should understand the basics of workflow orchestration and how to get started implementing it for their use cases