MWAA Migration: Move Apache Airflow Without Relearning Lessons
An operator’s MWAA migration plan—what moves as‑is, what to redesign, exact checks, and a parallel‑run cutover with rollback. Use it in‑house or bring us in.
7 min read
Airflow & orchestration
Airflow and data orchestration
Building DAGs that survive production — idempotency, retries, SLAs, deferrable operators, and observability.
Airflow DAG Testing: Catch Breaks Before Deploy
A practitioner’s guide to testing Airflow DAGs so they don’t break the scheduler, the UI, or your downstream data. Concrete checks, code, and CI examples.
4 min read
Airflow Scheduler Performance: Find the Bottleneck
Delayed task starts? Follow this ordered diagnostic to find the real constraint in Airflow: parsing, scheduler loop, DB, executor path, pools, caps, mapping, or workers.
7 min read
Airflow Monitoring That Warns You Before Users Do
What to actually monitor in Airflow, with concrete queries, configs, and alert routing that find problems before users do—plus real data freshness checks.
6 min read
Airflow + dbt: Run dbt From Airflow With Model-Level Visibility
A practitioner’s guide to running dbt from Airflow with model-level visibility. Compare Bash, dbt Cloud API, Cosmos, and Kubernetes—with real code and trade-offs.
9 min read
Airflow DAG Not Running: A Systematic Debugging Path
Your DAG isn’t firing and the clock is ticking. Use this ordered decision tree—commands, log paths, and configs included—to isolate and fix the cause fast.
3 min read
Managed Airflow Compared: MWAA vs Astronomer vs Composer
A practitioner comparison of MWAA, Astronomer, and Cloud Composer—with upgrade cadence, scaling, observability, pricing, and clear pick‑X‑if guidance.
6 min read
Airflow Best Practices From Someone Who's Been Paged at 3am
Reliability-first Airflow guidance from production incidents: idempotency, retries, deferrable sensors, SLAs, secrets, CI tests, and MWAA/Astronomer/Composer nuances.
9 min read
Want us to do this work?
Start a conversation
Tell us what's slowing your data team down.
We maintain a small client roster on purpose. If we're the wrong fit, we'll say so — and usually we know somebody who isn't.
- Replies within 2 business days
- NDA before specifics
- Fixed-scope first engagement, retainer if it works