Airflow -Why does everyone use it yet no one seems to cherish it?

Airflow is not flawless, but it serves as a framework rather than a tool. The design principles it enforces are more suited for batch-style data pipelines than many other technologies. Also, commercial orchestration solutions like Astronomer and Dagster don’t seem to be hurting Airflow all that much, which seems odd given the attitudes around Airflow in general.

Airflow is made up of cron + dependencies + little green/red boxes that you may click for additional information or to retry. For that combination of characteristics, the airflow is still the best in class.

The majority of difficulties that emerge in Airflow may be “solved” by using suitable development processes, such as code reviews. The query cost and query performance in general are equally interesting, but they have nothing to do with Airflow.

Airflow is a fantastic productivity tool that appears to improve with each new release. It includes a an extensive number of connectors/hooks and supported technologies, as well as workflow recovery/resumability and graph display for workflow units. Airflow comes with a plethora of cloud service providers, some of…