You probably don't need the modern data stack

The standard answer to "we need analytics" in 2026 is a subscription bundle: a managed ingestion service, a cloud warehouse, a hosted orchestrator, and a BI tool. Each one is excellent. Each one bills monthly. And for a large class of teams, the data would fit comfortably on a laptop.

This article makes a concrete claim: for simple use cases, one virtual machine running open source tools can replace that bundle. Not a toy setup. It has schedules, a control plane, authenticated run history, cancellation, and disaster recovery. I'll use a real deployment as the example: a warehouse that ingests repositories, commits, events, and Actions runs from a GitHub organization into ClickHouse and builds dbt marts on top. The GitHub data is incidental. The shape of the solution is the point.

The bill you're replacing

For a small workload, the SaaS stack usually means a managed extractor like Fivetran or Airbyte Cloud, a cloud warehouse like Snowflake or BigQuery, and a managed orchestrator like MWAA or Astronomer. Each is reasonable on its own. Together they bill four figures a month for even a modest setup. You also get three vendor consoles, three auth systems, and three places to look when a pipeline fails at 3am.

Here is the division of labor, with list prices as of August 2026:

Take three dbt seats and the bundle lands around $1,700 a month, roughly $20,000 a year, before usage overruns. The entire right-hand column runs on one Vultr vc2-4c-8gb instance: 4 vCPUs, 8 GB of RAM, 160 GB of SSD, 4 TB of bandwidth, $40 a month. That is a 40x gap. Prices move and usage-based bills vary, so check the current pages before repeating these numbers. The pattern will outlive them. Every SaaS row scales its bill with rows, seats, or hours. The VM bills $40 whether you load once a day or every 15 minutes.

The first three rows are almost uncontroversial by now. dlt gives you declarative, incremental extraction in plain Python, merging on stable IDs. A single ClickHouse node is absurdly capable below a terabyte. dbt is the same tool you'd use in the SaaS stack anyway. The interesting rows are the last three, because orchestration, UI, and logs are the reasons people believe they need Airflow.

What Airflow actually sells you

Airflow earns its complexity when you have many pipelines with cross-dependencies, per-task retries, sensors waiting on external events, backfills over years of partitions, and a team sharing one scheduler. That is a real product. It is also wildly oversized for the most common pipeline in existence:

extract → transform → test

Three steps in a row, once an hour. That is not a DAG problem.

If a step fails at this scale, the honest recovery is not a checkpointed resume with exponential backoff. It's running the whole thing again. Extraction is idempotent because loads merge on stable IDs. dbt is idempotent by construction. A whole-run retry costs almost nothing, and it removes an entire category of state you would otherwise have to store, display, and debug.

The example repo writes this down as explicit non-goals: no task queues, no sensors, no XComs, no per-step retries, no backfill scheduler. A retry is a new run. Every feature on that list is something Airflow would have given us for free. Every one of them is state we would then be operating.

One workflow engine, two DAGs

There is still a DAG. Two, actually. The infrastructure has to be provisioned, and the data has to move. OpenTofu creates the VM and DNS, then Ansible converges ClickHouse, PocketBase, Caddy, and the runtime. The trick is using one small workflow engine for both jobs. In Colors Blue, a step is a function from options to options, and the graph is a wiring function:

def wire_fn(step, run_opts):
    event = run_opts.get("blue/event")
    if event == "run":
        return {"start": (start_step, "dlt"),
                "dlt":   (dlt_step, "dbt-run"),
                "dbt-run":  (dbt_run_step, "dbt-test"),
                "dbt-test": (dbt_test_step,)}.get(step)
    return {"start": (start_step, "tofu"),
            "tofu":  (tofu_step, "ansible"),
            "ansible": (ansible_step,)}.get(step)

create walks provision and converge. run walks extract, transform, test. Same engine, same dry-run mode, same progress reporting, same validation. Desired state is a single colors.yml in git. Secrets are environment variables. ./blue build and ./blue create --dry-run work on a fresh checkout with zero credentials, so a configuration change is reviewable before anything touches a provider. The whole warehouse, infrastructure and pipeline both, is one repo you can diff.

The SaaS stack structurally cannot give you this: the infrastructure DAG and the data DAG in the same model, versioned together, pinned to the same commit.

PocketBase instead of the Airflow UI

Here's the quiet reason teams keep Airflow long after their DAGs stopped needing it: the UI. Someone needs to see whether last night's load ran, trigger a manual refresh, and cancel a stuck one. That's a real requirement. It's also a CRUD app, not an orchestrator.

PocketBase is a single-binary backend with SQLite, auth, a REST API, and an admin UI. The warehouse defines three collections named pipelines, schedules, and runs, and holds a hard line: PocketBase stores intent and whole-run evidence, never workflow internals. A schedule says "run this hourly." A run record says queued, running, succeeded, or failed, with a start time, an exit code, and a summary that contains no secrets. One run record equals exactly one workflow invocation.

Because the control plane is collections behind auth, the UI on top can be specific to your domain instead of Airflow's generic grid of DAGs and colored squares. Your operators see "GitHub warehouse, last load 09:00, 4,312 commits, refresh now", not dag_run_id: scheduled__2026-08-22T09:00:00+00:00.

About 140 lines of Python connect PocketBase to the machine. A systemd timer runs a dispatcher every 15 minutes. The dispatcher materializes due schedule slots into run records, with an idempotency key so timer restarts are safe. It stops the systemd unit of any run whose record requests cancellation. And it launches queued runs, each as a transient unit:

command = ["systemd-run", "--unit", unit, "--collect",
           "--property=EnvironmentFile=/etc/github-dwh/environment",
           "/opt/github-dwh/.venv/bin/python", "/opt/github-dwh/run.py", run_id]

That is the entire executor. systemd supervises the process, journald keeps the full logs, and a 22-line wrapper patches the run record with the outcome. Cancellation is systemctl stop, not a distributed-systems problem. Log retention is journald configuration, not a product. The operating system already shipped an orchestrator. You just have to not rebuild it.

What you give up

Selling this honestly means naming the trade-offs.

• One host. You own backups and recovery. The repo ships a tested recovery runbook. ClickHouse data and PocketBase state are the only things worth saving, since everything else regenerates from git.
• Whole-run retries only. Fine when runs are cheap and idempotent. Wrong for a six-hour pipeline where step two is expensive.
• No cross-pipeline dependencies, no backfill machinery. If you have twenty interdependent pipelines and a data team, buy the orchestrator.
• A scale ceiling. A single ClickHouse node goes much further than people expect, but "fits on one big machine" is the honest boundary of the whole argument.

The failure mode to avoid is letting the control plane grow. The moment PocketBase acquires task states, step queues, or retry policy, you have rebuilt Airflow badly and lost the argument. One run record, one process, one journald unit. That discipline is what keeps the system at 140 lines.

The point

The modern data stack solved real problems for companies with big data and big teams, then became the default for everyone else. If your data fits on one machine and your pipelines are extract, transform, test, the honest architecture is dlt and dbt for the data, ClickHouse for the warehouse, a small workflow engine for both infrastructure and pipeline, PocketBase for the humans, and systemd for the processes. One VM. One repo. One bill.

The GitHub warehouse in this article is live. Everything described here sits in its two repositories, the package getcolors/github-dwh and its deployment getcolors/github-dwh-vultr, with desired state in git down to the Vultr plan and the dbt tests. Swap the dlt source and it's your CRM, your billing exports, your product events. The data was never the hard part.