In 256 BCE, a governor named Li Bing faced a river that flooded every spring and drowned the harvest below it. The obvious fix was a dam. He didn’t build one.
He split the Min River with a stone levee shaped like a fish’s mouth and let the split do the work — most of the flow to the fields in dry months, the excess shed back to the river in flood, the silt carried off with it. No gates, no crew. The river regulated itself, and still waters the Chengdu plain today, twenty-three centuries on.
I thought about Li Bing a lot this month. My hardest test had the river’s disease: it flooded on every run.
The flow I couldn’t fake
OwlMeans is an AI development pipeline: describe a product as user stories, and specialized AI roles build it into a full-stack app you own. To test that end to end you can’t mock it — you run the real thing.
The test registers a user, types a prompt, waits for a spec, confirms it, and then the hard part: a brand-new Kubernetes sandbox spins up for that project, an agent develops the first story inside it, and the test waits until the story is done. Ten to forty minutes, every kind of async in between — and every kind of flaky. Leftover state, a half-provisioned sandbox, an agent that hangs, a UI that flips while you watch. The naive version is an eighty-minute timeout and a prayer.
The half I didn’t write
The first part of the trick: I didn’t build most of it.
The browser side runs on our own harness, @owlmeans/test-ui — one of the shared OwlMeans common libraries every project here is built on. It launches a browser, logs in through our supervisor keys instead of a real Google sign-in, and wraps the Playwright plumbing so my test reads like intent.
The sandbox didn’t need me either. The ephemeral Kubernetes slot — namespace, storage, deployment, network policy — is provisioned and kept alive by another common library, @owlmeans/kephemeral: it watches the pod and heals it if it crashes. It already knew how to stay standing.
So the only river left to tame was the wait: how do you depend on a flow that legitimately takes forty minutes, without babysitting it?
The fish-mouth
You can’t use a short timeout — the agent really might think for forty minutes, and that’s fine. You also can’t wait the full eighty to learn it died at minute three. A fixed timer is a dam: too low and it floods, too high and it’s useless.
So I stopped timing the flow and started reading its pulse. The agent spends tokens as it works — that isn’t a proxy for progress, it is the progress. So the test watches the token balance. Every twenty seconds: still dropping? If yes, the agent is alive — keep waiting, however long it takes. If it goes flat for ten minutes, the agent has hung, and the test fails right then — instead of staring at the wall for another hour.
That’s the whole idea: the flow’s own behavior tells me whether it’s healthy, the way the current tells Li Bing’s levee how much to pass. Thirty lines of code turned the flakiest test I owned into the most trustworthy one.
Clean slate in, clean slate out
One more leak. A run killed mid-flight leaves a whole sandbox behind — deployment, storage, certs, routes, namespace — and the next run trips over the wreckage and won’t start.
The fix is boring and total: before every run and after, the test destroys every project on its one reused profile. However last time ended — clean exit, crash, closed laptop — the next run starts from a known, empty state. No manual cleanup, ever — a clean slate per run is the best thing you can do against flaky tests, and here it’s free.
The aqueduct
One problem left, the one most never solve cleanly: the cluster runs on a server at home — no public IP, behind an ISP that won’t take inbound traffic. To test it from anywhere, and put Cloudflare in front, I had to carry the flow out of the house.
That’s a separate little project, flt-pf-tnl: a cheap cloud VM with one reserved static IP, running WireGuard and an Nginx proxy. The path is simple — Cloudflare → that relay → an encrypted tunnel → the home server → the Kubernetes sandbox. Two commands stand it up — just Terraform, Ansible, and a few shell scripts. A watchdog re-checks the firewall every sixty seconds and repairs drift; the static IP survives teardowns, so the public address never moves.
I set it up once. It has stayed up. An aqueduct: built once, running on gravity.
You can just have it
Step back. None of this is exotic — a self-healing sandbox, a feedback loop that reads the flow’s own pulse, a tunnel that carries a private cluster to the world. A few days of the right structure around a flow that ate my afternoons. That structure is the product — the same promise OwlMeans makes about every app it ships: a typed monorepo and the infrastructure around it, yours to keep.
So here’s the honest pitch.
If you’re building anything real, you’ll hit this wall — slow flows, external dependencies, a sandbox to stand up and tear down, tests that pass Monday and flake Tuesday. You can do what I did: burn weeks learning that a longer timeout never fixes a flaky flow, that you read its pulse instead, that reaching your own cluster takes an aqueduct. That was worth it for me; it’s my job.
It is not your job.
This is part of OwlMeans Services: we take this exact harness, sandbox, and tunnel, tailor them to your stack, and deploy them in days. You own all of it — code, infrastructure, no lock-in, maintainable with any coding agent.
A flow that runs itself, the way a river has for twenty-three centuries. Skip the month I spent — you can just have it.
OwlMeans builds full-stack TypeScript SaaS from your user stories — typed, SSO-ready, and yours to keep building with any agent. See what it can do →