Skip to main content
DATA-PIPELINE-ENGINEERING4 MIN READ

Spot Airflow smells before they become incidents

Identify common DAG smells and the reliability risk each one creates.

Place each DAG smell into the operational risk it mainly creates. Scheduler load Unsafe retry Worker portability Secret exposure Calling a source API while the DAG file is imported Appending rows to a fact table every time the task runs Writing /tmp/orders.csv in one task and reading it in the next task Putting a warehouse password directly in an operator argument Importing a heavy ML library at the top of a DAG used only by one task Reading latest available records instead of the DAG's data interval Assuming two tasks run on the same machine because it worked locally Logging…

Read the full lesson

Sign up free — one personalized lesson every day, matched to your role and goals.

Already have an account? Sign in

← Back to library
Contact us