From AI-Orchestrated to AI-Practical: What Two Weeks of Obsession Actually Taught Me

In my previous post, I wrote about building an orchestration system — a company of AI agents working together, resembling a team, pushing PRs, collaborating on tasks. I was excited. I'd glimpsed what felt like the future of software engineering.

Then I had to put a full stop to the whole thing.

Not because it didn't work. It did, kind'a. But because I came to a new realization: I was spending more time caring for the system that I built than doing the actual work. And worse, I was caught in what Rachel Thomas from fast.ai calls "dark flow" — that state of intense absorption that feels productive but isn't quite what it seems.

This post is about what I landed on after pulling myself out of that loop.

The Addiction Nobody Warns You About

Thomas draws a parallel between vibe coding and gambling. The constant feedback loops, the celebratory feeling when something works, the sunk-cost momentum of "just one more prompt." I lived that. For about two weeks, I was dead set on making my workflow as efficient as possible. Going AI-first, I was building tools, refining pipelines, orchestrating agents — and barely sleeping. When not in front of the screen, I was thinking about how to improve the next thing. The output felt massive. The actual usable output? Much less impressive.

The METR study Thomas references found that developers thought they were 20% faster with AI tools, while actually being 19% slower. I can't put exact numbers on my own experience, but the pattern fits. I was producing a lot of stuff alongside my daily tasks. Whether it was producing the right stuff, or producing it efficiently — that's a different question.

Recognizing this was the first real takeaway.

What I Actually Kept

After the reset, I stripped all the "fanciness" away and restructured my workflow, keeping only the good parts. No orchestration. No agent company. Just a practical setup that lets me move faster without the overhead of babysitting a system.

Here's what it looks like now:

I have a dedicated development machine — my old computer with Ubuntu, clean, holding no data beyond the carefully created access tokens and the code needed to work. On it, I run isolated virtual machines for each project or context: one for my day work, one for my invoicing app, and one for other experiments. Within each of those environments, AI has full rein. Outside too, but that one I watch more closely. I give it a task, it works, and I get a PR. No stops, unless I want. Same result as the orchestration setup, without the fancy infrastructure. Plus, I keep the full power of cli tools at my fingertips.

Then there's my daily machine, the one I actually live in, which connects to the dev environments over SSH with a tmux setup that keeps everything organized. Two machines, clear separation of concerns, no window orchestrator needed — well, tmux does that, but no custom UI.

And I still find the need to have an IDE at hand — connected through remote SSH. It's not glamorous. But it works, and — critically — I spend my time on actual work rather than maintaining the meta-system.

Four Things I Now Believe

Rethinking the setup was more valuable than the setup itself. The orchestration experiment forced me to think deeply about how I work, where the bottlenecks are, and what I actually need from AI tooling. The system I ended up discarding taught me more than the system I kept. Sometimes the best outcome of building something is the clarity you gain about what you shouldn't build.

Orchestration doesn't pay for itself — yet. The promise of agent orchestration is compelling: multiple AI workers tackling different parts of a problem in parallel. In practice, I found that the coordination overhead eats the gains. You end up as a project manager for a team of unreliable interns who each need constant supervision. The individual AI sessions are powerful when directed at defined and small enough tasks. The meta-layer on top? More trouble than it's worth, at least today.

AI is excellent at well-scoped tasks, not at coherent systems. Ask an AI agent to implement a specific API endpoint with clear requirements? Great results. Ask it to build you a coherent application from scratch? You'll get something, but "something" and "a coherent, maintainable application" are very different things. The people claiming they built entire apps with orchestrated agents aren't lying — they did build something. Whether it holds up under real use, real users, and real maintenance cycles is another matter. One that I was unable to replicate.

Governance still needs a human, and the work needs to be well-chunked. My truth is that you still need someone who understands the architecture, who can break work into pieces that are right-sized for AI execution, and who can evaluate whether what comes back makes sense at a structural level. The line-by-line code review might be fading, but the architectural oversight isn't going anywhere. And this work is damn hard!

The Middle Ground

I'm in no way saying AI isn't powerful. After these two weeks, I'm more convinced than ever that AI-first development is the direction we're heading. But I also know better where the practical limit lies today.

The gap between what's theoretically possible and what's practically useful matters. The orchestration future might be six months away. It might be two years. The people who will navigate it best aren't the ones who bet everything on the bleeding edge — they're the ones who keep a clear head about what's working now versus what makes for a good demo.

Thomas quotes Jeremy Howard: "People who go all in on AI agents now are guaranteeing their obsolescence. If you outsource all your thinking to computers, you stop upskilling, learning, and becoming more competent." I think he's right, but I'd add a nuance: you also stop building the judgment you'll need to govern these systems when they do become capable enough to orchestrate.

The craziest thing? Two weeks from now, this will probably be outdated. That's the pace we're living in. But the fundamentals — scope your work, maintain governance, stay honest about what's actually working — those aren't going anywhere.