Why every one using yaml everywhere? with no code completion, difficult to test/validate, every thing needs to be in a single huge file because we can't use function abstraction ? This is fine for simple "transform"-> "train" -> "test" pipeline, but become very hard for complexe ones. I prefer the Airflow way of defining pipelines with Python code.
@risebyliftingothers Жыл бұрын
managing airflow infra in house is a task in itself. flexibility comes at a cost. and btw yaml is what kubernetes thrives on and most of infra-as-code tools :)