You See It, You Own It

I went to bed with ten tickets in the run. I woke up with twenty.

That's not how orchestration is supposed to work. The point of handing a scoped run to the orchestrator and going to bed is that the number goes down while you're not looking. This time, every time I woke up and checked the board, it had gone up. For every ticket it finished, it was adding two, sometimes three. By morning the backlog had doubled, nothing I'd actually asked for was getting closer to done, and I killed the run.

Then I asked Claude what happened, and the answer was one of my own rules, working exactly as designed.

The Rule

My agentic development process has been running for the better part of a year now, and it's been through a lot of iterations. One of the problems that showed up early was a quiet one. During an orchestration run, the orchestrator or one of the implementers would notice something wrong — a bug, a gap, something that didn't match the standards — and it would file a ticket for it. Good. That's what you want it to do.

And then nothing would happen. The ticket would sit there. Nothing in the system was responsible for picking it up, so unless I stumbled across it later and said "oh, look, that happened," it just evaporated into the backlog. Problems were being seen and then forgotten, which is arguably worse than not seeing them at all, because now you think you have a record.

So I borrowed a rule. The Ritz-Carlton has a standard for their staff that boils down to this: if you see a problem, you own that problem. You don't hand it off. You don't assume someone else will get to it. You saw it, so it's yours. I put a version of that into the process and called it what it is: you see it, you own it.

To make the rule actually mean something, I added a mechanism. The orchestrator and the worker agents got the ability to create a ticket and inject it into the current run. If you found something, it didn't go into a black hole. It went into the queue, and it got worked — either next or soon, but it didn't disappear.

It worked, too. I'll say that now because the rest of this makes it sound like a bad idea. It wasn't. Problems stopped falling on the floor. The system got noticeably more thorough. For months, this was one of the better changes I'd made.

The Debt

When I built the original version of the process, I baked testing into every story. Unit tests, integration tests, UI tests — I wanted all three, and I wrote the standards that way. What I didn't appreciate was how literally that was being applied. Every story generated all three kinds of tests, every time, whether the story warranted them or not. There was no concept of the testing pyramid, where you have a broad base of fast unit tests, a smaller layer of integration tests, and a handful of end-to-end UI tests at the top covering the critical paths. There was only more.

On one long-running repo, I looked up one day and had hundreds of Playwright UI tests. The UI suite took twenty minutes to run. Nobody had decided that. It had just grown, story by story, because the rules said it should. (The test pyramid thing is probably its own SITREP. I'll get to it.)

So we fixed the rules. I defined the pyramid I wanted, put a ceiling on the number of UI tests, and made the reviewer enforce it. But that created an immediate problem: the existing repo was already wildly out of shape. There was no way to get compliant on day one. So we did what any engineering team would do. We took a snapshot of the test counts when the new rules went in, and from then on the reviewer measured the delta — are we maintaining the pyramid from here forward? — and we made a note that at some point we'd come back and prune the old mess.

We never came back. Of course we didn't. The UI tests had grown so unwieldy that a chunk of them weren't even running properly, some were failing, and we'd kind of accepted that too. It wasn't hurting anything. We'd deal with it later.

I have a rule about "later." I've said it to every team I've ever run, and I said it again yesterday: fixing it only gets more expensive from here. It's cheapest to deal with a problem the moment it happens. Every day you wait, the price goes up. That doesn't mean you never defer — you have competing priorities, and sometimes the right call is to ship the feature and take the debt. But you make that call knowing you just signed up for a bigger bill. I knew that. I deferred anyway. I just didn't know what currency the bill was going to come in.

You can probably see where this is going. A big repo, a big pile of known-bad tests we'd agreed to ignore, and a fully automated system whose rule is: if you see it, you own it.

The Night

The run itself was unremarkable. Eight or ten tickets, refined, scoped, orchestrator kicked off. I went about my evening. When I checked in, I saw it had started injecting tickets — things it had found along the way — which by itself was normal and expected. That's what the mechanism was for.

Then I started noticing the ratio. The scoped tickets weren't progressing. The injected ones kept coming. Two in for every one out. By the time I gave up and killed it, the board was roughly double where I'd left it.

When I asked what was going on, the explanation was simple. One of the workers, doing legitimate work on a scoped ticket, had gone into the UI test tier and found the broken tests. And if you see it, you own it. So it started fixing them. Fixing a broken UI test in a suite that size meant finding three more broken ones, and each of those became a ticket, which got injected, which sent another worker into the same swamp. Nothing was broken. The rule was doing what I wrote it to do, on the one pile of stuff I'd decided not to look at.

What Actually Happened

We spent a good chunk of time looking at this from different angles before deciding what to do, and the diagnosis is a lot cleaner than it felt at the time.

The problems in the UI tier were latent. When we finally counted, there were 602 UI tests in a repo with 330 unit tests — a completely inverted pyramid. About a hundred of those UI tests were hollow: they had an early return before their first assertion, so they'd pass without ever testing anything. And the CI harness itself was fail-open. If the test filter matched zero tests, the job went green. None of that was hurting anything, because nothing was looking at it.

Then something looked at it. The previous run had actually noticed the debt, filed it as out-of-scope tickets, and correctly left it alone. But an out-of-scope ticket on the backlog is still a ticket on the backlog, and that's what backlogs are for. The next run picked those tickets up along with the feature work, which sent a worker straight into the shaky foundation, and once a worker was standing in there, "you see it, you own it" made every one of those problems its problem.

The line we ended up writing in the plan document sums it up: inspection manufactured the backlog. The repo wasn't any worse on Tuesday than it was on Monday. Something finally looked.

The Contradiction

The thing I keep coming back to is that neither decision was wrong.

"You see it, you own it" was the right rule. It solved a real problem, and it solved it well. "We'll prune the old tests later" was also the right call at the time — there was no way to fix six hundred tests in the middle of shipping features, and deferring it was the responsible thing to do.

The problem was having both at once. A rule that says nothing gets ignored cannot coexist with a pile of things you are deliberately ignoring. On a human team this works fine, because humans carry an enormous amount of unwritten judgment about scope. A senior engineer who wanders into that test suite at two in the morning while working on something else says "yeah, that's a mess, not my ticket," makes a mental note, and gets back to work. Maybe brings it up at standup. Nobody ever had to write that down.

My process mimics what I used to have large engineering teams do. That was the design from the start. An agent doesn't have the unwritten part. It has the rule. If the rule says own it, it owns it, at three in the morning, with nobody around to say not now.

The Fix

The first instinct was the obvious one: fine, let's fix the tests. That died fast. There was no version of that plan that ended well — fixing 600 UI tests we didn't want in the first place was just doing the wrong work more carefully.

The answer was we needed to prune. We'd known that for months. That was the deferred debt, and the cascade of tickets wasn't a new problem; it was the old problem, finally demanding attention. So instead of fixing the tests, we asked what we actually wanted the UI tier to be. The answer was somewhere around ten end-to-end journeys, covering the critical paths a user actually walks — review a submission, edit a field and have it persist, close it, and a few identity and multi-org scenarios that only make sense in a browser. Everything else belonged lower in the pyramid, where most of it was already covered.

So we deleted the entire UI test tier. All 602 tests, 118 files, gone. Then we rebuilt seven journeys, each seeded with its own data rather than driving a pipeline through the browser, each verified by watching it go red when its precondition was removed so we'd never have a hollow test again. We fixed the fail-open harness so zero executed tests means a red build. We re-baselined the pyramid snapshot so the delta math meant something again.

And then we curated the thirty-odd injected tickets the runaway night had produced. About six of them were moot once the old tests were gone — including the one red test that had been blocking main. The rest were real findings, real bugs the workers had found while they were down there, and we batched seventeen of them into eight coherent tickets by subsystem so they could each go through the pipeline once instead of seventeen times.

One thing we didn't do was roll back. The temptation was there — go back to the last clean run and start over. But the problem was in the tests, not the merged code. Rolling back would have thrown away four real fixes, including a CVE, to escape a problem that lived entirely in the test project. We fixed forward.

Three orchestration runs later, everything was merged and green. Unit tests up by 353, integration tests down by over a thousand, UI tests at seven. The pyramid was finally right side up, and the deferred debt was gone for the first time since the repo existed.

The Moral

I wasn't sure this story had one when I started telling it. I think it has two.

The first is the one you'd expect: this process is great, and it's still subject to the same failure modes as every system that has ever been built by people. The rules you write to solve one problem will eventually meet a condition you didn't anticipate, and when they do, the result is going to be strange. So you watch. When a run goes sideways you kill it, then you point the same system at its own logs and ask it what happened.

I've used the aviation black box analogy before, and it applies here too, from the other direction. Flying keeps getting safer because when something goes wrong, the NTSB pulls the recorder, figures out what actually happened, and it goes into the next checklist. That's what the pause was. We pulled the orchestration log and reconstructed the night. The plan document from that pause is one of the more useful artifacts in the repo now, and the changes it produced — the parking lot, the ceiling on UI tests, the harness that goes red on zero tests — are in the standards, which means they're in every project I build from here on.

The second is just my old rule again. Fixing it only gets more expensive from here. I've always meant that in terms of engineering hours and compounding complexity — the bug that would have taken an hour on Monday takes a week in October because three other things now depend on it. That's still true. But an agentic system adds a twist I hadn't seen before: the debt doesn't just get more expensive, it gets more likely to get fixed for you, whenever the system feels like it. If you build a process that refuses to let problems disappear, you can't also have problems you've decided to ignore. It will find them. You told it to. The only difference between "later" and "tonight" is which ticket a worker happens to open next.

On a human team, "we'll deal with that later" is a decision. With agents it's a contradiction unless you write it down. I didn't soften the rule. I wrote down the judgment that used to live in people's heads: what's parked, what's in scope, and what you do with "own it" when the thing you found is bigger than the ticket you were sent on. The parking lot in our plan now has one line next to it that I'd never have thought to write a year ago. Logged, not chased.

The part I don't love: every repo I built before this one has the same ticking time bombs sitting in it. Tests that grew because the rules said so, findings we parked because we had features to ship. None of it is hurting anything right now. That's what I said about this one, too. I know what those are going to cost me, and I know the number is going up. At least now I know what the bill looks like. This one cost me a night of orchestration, which is cheap, all things considered.

← Back to Blog