How we deleted a dozen files and 10,000 lines of code and got control of our Airflow DAGs
Language: English
The presentation was given on 2024.09.16 at PyCon Israel 2024 - Conference.
Airflow is great, but it's notoriously hard to avoid duplicate code. We'll show you how we got into technical debt, and how we got out of it. We'll share our powerful and reusable class library, and the stories behind its development.
The challenge of writing reusable Airflow code, and the nightmare of technical debt. How we built a powerful and easy to use class library for our Airflow DAGs. What to do -- and what not to do -- when refactoring. Development tradeoffs, and managing technical debt.