Claude Hallucinations, or: How I Learned to Stop Worrying and Trust But Verify

I have a skill. It's a custom prompt template that converts a PRD into a structured backlog. Breaks the product requirements down into milestones, stories, and vertical slices based on how I like to work. I've used it on several projects now, and it's become one of the core pieces of my agentic development workflow.

So I'm wrapping up the setup on a new project. The repo is created, the PRD is written, engineering standards are loaded, and Claude has just finished all the pre-work. Everything's in order. Time to convert the PRD into stories.

And Claude says, helpfully: "Your next step should be /convert-to-prd followed by the filename and a couple of parameters."

I looked at that for a second. It looked right. The syntax was clean. The parameters made sense. But something in the back of my head said: I don't remember setting that up.

Maybe I had. Maybe Claude had done it for me at some point and I'd forgotten. It's happened before. These tools do things in the background that you don't always track. But I've been doing this long enough to know that when something looks right but you can't remember making it, you ask.

So I asked. "Claude, is that command correct for that skill?"

It paused. Went back and checked. And came back with what amounted to: "No. My bad. That's not how the skill works."

It had hallucinated a command. Confidently. With correct-looking syntax, reasonable parameters, and just enough plausibility that if I hadn't questioned it, I would have run it, gotten an error, and spent time debugging something that was never real in the first place.

Why This Matters

This is the part of agentic development that makes it into the negative hype cycle. You hear it all the time: "AI hallucinates, you can't trust it, it makes things up." And they're not wrong — but hype is hype, whether it's positive or negative. The breathless enthusiasm and the breathless fear are both missing the point. What struck me about this particular hallucination wasn't that it happened. It was the absolute conviction with which it was offered. No hesitation, no hedging. It looked exactly like every other correct thing Claude has ever told me.

If you're working with AI tools, you need to internalize this: they will hallucinate, and they won't warn you. There's no asterisk. No "I'm not sure about this." The only difference between a hallucination and a correct answer is that one of them is wrong. And they look identical.

The good news? When you catch it and call it out, the model corrects immediately. No defensiveness, no doubling down. It just goes, "You're right, here's the actual answer." Best case: you catch it, it fixes it, you move on. Worst case: you don't catch it, and you build on top of something that was never real.

The Silver Lining

Here's the funny part. When I looked at what Claude had hallucinated, my first thought wasn't frustration. It was: that's actually a great idea.

The command it made up, a shorthand that takes the PRD filename and kicks off the conversion with specific parameters, would actually be a useful addition to the skill. It just didn't exist yet. Claude had basically designed a feature improvement by accidentally pretending it already existed.

I added it to my backlog.

The Rule

If you're going to work this way, and I think more people should, you need one rule tattooed on the inside of your eyelids: trust but verify.

Trust the tool. It's capable. It will do things that blow your mind. But verify everything that matters. If it suggests a command, check that the command exists. If it writes a function, make sure it actually does what it claims. If it tells you a library has a method, go look at the docs.

This isn't a flaw that makes the tools unusable. It's a characteristic you manage, like any other tool in your kit. A table saw is incredibly useful, but you don't close your eyes when you use it. Same principle.

I caught this one because something felt off. Next time, I might not have that instinct. So I verify. Every time.

← Back to Blog