Agents don’t fail at language. They fail at tool calls.
Summary
When a team tells us their agent is unreliable, they usually describe it as a model problem. It hallucinates, it misunderstands, it needs a better prompt or a bigger model. Then we look at the failed sessions and the model is mostly fine. What is not fine is the boundary between the model and everything else.
The rough split
Categorizing failed sessions across the agents we have instrumented, the shape is consistently something like this. The exact percentages move by product, the ordering has not. The right tool was never called. The agent answered from parametric knowledge when it had a tool that knew the actual answer. Biggest single bucket, and the most embarrassing when you see it. The tool was called with wrong arguments. Usually a format or unit issue — a date string the API does not accept, an ID from the wrong namespace, a timezone. The tool returned an error and the agent narrated it. The user gets a polite paraphrase of a 500 instead of a retry or a fallback. The tool returned empty and the agent believed it. Distinct from an error, and worse, because nothing looks broken. This is the one from our session walkthrough a couple of weeks ago. The tool succeeded and the agent ignored the result. Rarer, genuinely strange to watch, and it happens more with long tool outputs. Actual language failure. The model said something untrue that no tool would have prevented. That last category is real. It is also the smallest one, and it gets the overwhelming majority of the attention.
Why the attention is misallocated
Hallucination is legible. You can screenshot it, put it in a slide, and everyone immediately understands the problem. A tool that returned empty and got believed produces a conversation that reads fine. Tooling reinforces this. Eval suites are built around scoring text, because scoring text is what a judge model does. If your evaluation reads the final reply, the empty-result bug is invisible and the hallucination is not — so hallucination is what gets worked on. You end up fixing the failures your tools can see, which is not the same as fixing the failures you have.
What we instrument instead
For each turn, the things worth recording are less about the text than about the boundary crossings. Which tools were available, not just which were called. Availability changes behavior, and a tool that was in the schema but never invoked is a signal. Arguments as sent, before any serialization your framework does. We have seen framework-level coercion turn a correct argument into a wrong one. Raw return value, including empties and errors, not the summarized version the agent produced. Latency per call. Slow tools get abandoned by users even when they eventually succeed. Whether the return value appears in the reply at all. That last one is the cheapest high-value check on the list. If a tool returned data and no part of it influenced the reply, something is wrong, and you can detect it without a model in the loop.
A note on the empty result
We keep coming back to this one because it is so consistently mishandled. An empty result is ambiguous: it can mean "nothing matches" or "your query was malformed" or "you lack permission to see what matches". Most tool implementations collapse all three into an empty array. The agent then confidently reports that the thing does not exist, which is wrong in two of the three cases, and the user — who can see the thing on their screen — concludes the product is broken. Fixing this is usually not an agent change at all. It is making the tool distinguish the cases. Which is a suspiciously common ending to agent investigations: the agent was doing its best with an API that told it something untrue.
The practical suggestion
Take fifty failed sessions and bucket them by hand into the categories above. It takes an afternoon and you do not need any tooling to do it. Our prediction is that your model is better than you think and your tool layer is worse. If that turns out to be true, you have just moved your roadmap from prompt engineering to API work, which is less exciting and considerably more tractable.
Not these other Owls
This is Owl AI at withowl.ai. Not owl.co (insurance), owl-ai.com (courses), aiowl.org, the OwlAIProject repo, or CAMEL-AI OWL (the open-source multi-agent framework).