dbt Cloud Migration: Move Without Breaking Production

Your team is running models on cron or Airflow and considering dbt Cloud for scheduling, IDE, permissions, and lineage. You want the benefits without breaking production or paying for two systems. Here’s the plan. We’ll cover when Cloud is and isn’t worth the cost, then walk the migration: inventory, environment and credential mapping, job parity, Slim CI with state deferral, exposures, and a cutover checklist with rollback. Run both platforms in parallel before switching. If you need outside help, Vertex Data Consulting has shipped this move for platforms at scale, but this write-up is complete enough for an in-house team. For deeper tuning, see our articles on why your run is slow and where the Snowflake credits go.

Should you move from core to Cloud? Cost, fit, and timing

Cloud is worth it when your team needs managed scheduling, role-based access control, SSO, IDE, and APIs, but doesn’t want to own build runners, permissions, and alerting in perpetuity. If your Airflow is stable, cheap to operate, and already tied into on-call, you may stay put. If you’re spending cycles on brittle YAML, secrets, and flaky runners, Cloud consolidates that toil.

Cost: compare subscription to internal engineer hours maintaining orchestration, plus the impact of late dashboards. Measure actual time spent per month on failed deploys, broken runners, and permissions tickets. If most work is steady-state and you rarely change jobs, savings may be modest; if change velocity is high, the platform lift pays back faster.

Capability	Core + Airflow	Cloud
Scheduling	DAGs you own	Built-in jobs
Access	IAM you build	SSO, RBAC
Dev UX	CLI + editor	Hosted IDE
APIs	Airflow + scripts	Job/metadata API

Reference design docs on the dbt Developer Hub for the latest from dbt Labs.

Inventory and mapping: environments, credentials, roles

Create a complete inventory before you migrate from dbt core. List repositories, targets, schedules, environment variables, secrets, external dependencies, and alert routes. Capture current warehouse roles and grants, including service accounts. Note every DAG and cron that calls the command-line interface.

Map targets in profiles.yml to environments in Cloud. Example Snowflake mapping:

# profiles.yml
my_profile:
  target: prod
  outputs:
    prod:
      type: snowflake
      account: "acct_abc.us-east-1"
      user: "svc_transform"
      role: "TRANSFORMER"
      database: "ANALYTICS"
      warehouse: "XSMALL"
      schema: "TRANSFORM"
      authenticator: "externalbrowser"
      threads: 8

In Cloud, mirror these via Environment + Connection with the same database, schema, role, and warehouse. Prefer SSO for users and key-pair or OAuth for service jobs. If you’re on Amazon Web Services, decide whether to continue IP allowlisting or switch to network policies. Keep names identical to reduce surprises in exposures.

Check secrets: rotate anything currently set as env vars in Airflow and store them in Cloud’s credentials. Document who owns each permission—no orphaned roles at cutover.

Job parity: schedules, Slim CI with state deferral, tests and exposures

Recreate every schedule one-for-one. For jobs triggered by upstream loaders, preserve those external triggers by calling Cloud’s API rather than removing the dependency. If your pipeline fans out across DAGs, consolidate where it helps but keep outputs identical until post-cutover refactors.

Set up Slim CI with state deferral so pull requests build only changed models. Example commands for a Cloud job stage:

dbt deps

dbt build \
  --select state:modified+ \
  --defer \
  --state s3://my-state-artifacts/prod/ \
  --profiles-dir . \
  --target ci

Expose critical dashboards so lineage remains accurate after the move. Minimal exposure YAML:

exposures:
  - name: revenue_dashboard
    type: dashboard
    url: https://bi.local/dash/42
    depends_on:
      - ref('fct_revenue')
    owner:
      name: Finance Ops
      email: finops@example.com

Run freshness and tests in the same job stages and keep alerting destinations unchanged initially. For a 40M-row orders table on an X-Small warehouse, validate runtime parity by comparing build durations over a week.

Run both systems in parallel and cut over safely

Run core and Cloud in parallel for at least one full business cycle. Point BI to the old schemas while you verify the new ones. Write guardrails that prevent both systems from writing to the same schema at once—use feature flags or per-environment schemas.

Cutover checklist:

• Job parity validated; all runs green for 7 consecutive days.
• Alerts from Cloud reach on-call and owners.
• Permissions and grants audited on target schemas.
• BI connections updated to new schemas during a freeze window.
• Airflow DAGs disabled but retained for rollback.

Rollback plan: keep Airflow artifacts and last-known-good environment variables intact. To validate data equality, compare row counts and sample hashes:

-- Snowflake validation snippet
select
  sum(case when a.hash = b.hash then 1 else 0 end) as match_rows,
  count(*) as total_rows,
  1 - (sum(case when a.hash = b.hash then 1 else 0 end) / nullif(count(*),0)) as mismatch_rate
from (
  select id, md5(object_construct_keep_null(*)) as hash from PROD_TRANSFORM.fct_orders
) a
join (
  select id, md5(object_construct_keep_null(*)) as hash from CLOUD_TRANSFORM.fct_orders
) b using (id);

Use COALESCE-based spot checks to compare critical metrics before flipping BI.

Failure modes that keep teams paying for two systems

Common traps, with fixes:

• Partial migration: a few Airflow-only tasks remain (exports, emails). Fix by moving them to a lightweight worker or calling Cloud’s API from Airflow solely as a trigger during a short deprecation window.
• Secret sprawl: env vars still referenced by old DAGs. Centralize in one vault and rotate; remove CLI fallbacks.
• Permission drift: roles created ad hoc in Cloud that don’t exist in the data warehouse. Manage grants in version control and apply via jobs.
• Package gaps: Python dependencies assumed by macros on build images. Pin versions and test via the command-line interface locally before enabling deployment.
• Monitoring split: alerts come from two places. Choose one alert channel during parallel runs; the other is informational only.
• Slow models: teams blame Cloud when the issue is SQL. Profile queries and refactor. See our dbt repo performance service and 7 fixes that work.

Finally, don’t skip documentation updates. Broken lineage or missing metadata keeps people clinging to the old workflow.

FAQ: core vs Cloud, ETL, database moves, and more

Is core deprecated? No. It’s actively maintained by dbt Labs and the community. Cloud adds managed features on top.

What’s the difference between the tool and Cloud? Core is the open-source transformation engine; Cloud layers IDE, job scheduler, SSO, APIs, and governance.

Is this an ETL tool? It’s primarily the T in ELT. Use loaders for extract/load; use models for transformation. Treat each model like a function over upstream relations—compose DAGs accordingly.

Could you use it to migrate operational databases? You can, but we don’t recommend it for live OLTP moves. Use dedicated replication tools and validate with transformations. For analytics, migrating from dbt-like workflows is straightforward; for operational schemas, risk is high.

Seven types of cloud migration commonly cited: rehost (lift/shift), replatform, refactor, repurchase, retire, retain, and relocate. Your move here is closer to refactor/replatform of transformation deployment.

How does Datafold help? By diffing data between old and new builds, surfacing column-level changes, and catching regressions quickly during parallel runs.

Do you need SDLC fundamentals to productionize code? Yes—branching with git, code review, tests, and CI are table stakes when using dbt.

Where to learn more? Start with the dbt Developer Hub. For tuning runs and credits, see our Snowflake guide and our dbt articles.

Ready for a seamless move or want a second set of eyes on your dbt migration plan? See our dbt Cloud migration service or start a project. We also help optimize warehouses and automate orchestration end to end.