I spoke with a dev team at KubeCon last year that deployed four LangChain agents to production. The first week it cost their company $127. By week four, the bill was $18,400. The final damage before anyone caught it was $47,000, all because a couple of agents got stuck in a conversation loop that ran unchecked for 11 days.
Each agent was performing exactly as designed, though. There wasn’t a bug, the problem was that nobody had set an economic boundary on the interaction. The agents were authenticated and authorized to run, but they had no concept of what they were allowed to spend. I’ve seen this story repeat and repeat. Another quick anecdote is a mid-sized SaaS company that launched a customer support agent with access to an LLM, a vector database, and a handful of internal tools. Volume began growing, and the agent started iterating on ambiguous cases. It queried models multiple times, expanded context windows, and cross-referenced external sources. While each individual call was cheap, over one weekend those tens of thousands of cheap calls added up to fifty grand in unplanned spend.
As more of these stories pop up, they reveal something fundamental about how AI changes software economics for developers using these new toolsets.
Cost are decisions, and developers are the ones making them
We’re used to developing in a world where cost scales with infrastructure. Servers get provisioned, and the bill is (or should be) roughly predictable. AI is a different animal, because a single user request can trigger a chain of tool calls, retries, model hops, and context window expansions. Agents delegate work to other agents, which delegates again, etc. The cost of any given interaction thus becomes a function of the logic the developer wrote, not just the infra underneath it.
So, prompt chains, routing logic, retry policies, and context window configs are all very clearly now economic decisions…they just don’t really look like economic decisions at design time, yet. They look more like engineering tradeoffs, and the invoice shows up later after the spending pattern is already baked into the architecture.
Cloud computing taught us a version of this over the better part of a decade, via shadow IT, surprise bills, and cost attribution fights between teams. AI is compressing that same learning curve into months, with the key difference that AI costs are less predictable and more challenging to trace because agentic systems are non-deterministic by design.
Visibility stops where the $ starts
Most observability stacks today track infra performance pretty well, and can tell you that an agent is running at 99.9% success with sub-100ms latency. The limitation is telling you that each of those successful calls costs you ten bucks when a fifty-cent alternative would deliver the same result. So your system looks healthy from every technical metric while quietly (at least at first) destroying margin.
API gateways have a similar blind spot. Great for authenticating requests, enforcing rate limits, and confirming authorization, but a $1,000 wasteful call gets the same treatment as a $1 high-value call. Both are “authorized,” and this economic dimension to the decision is invisible to the tooling.
Multi-agents are making this challenge worse, because when they hand off to other agents, the cost of a single workflow collapses into a single line item. You can see total spend, but not which step in the chain drove it, or which tenant triggered it, or which feature is responsible for the spike. When an invoice comes without explanation, the post-mortem becomes a forensic exercise that’s fun for no one.
Treating costs like an engineering concern
This doesn’t require an MBA. What actually works is embedding economic signals into the very same workflows where devs already make technical decisions. Attribution is step one. Every agent call should carry context about the tenant, the feature, the workflow, and the job it is performing. Short of that metadata, nothing downstream works. Simply put, you cannot optimize, govern, or even explain what you cannot first identify.
Hard limits should also be built into execution paths. Think about maximum costs per job, retry ceilings, context size caps, and rate limits per workflow. These are all inline constraint examples that stop runaway behavior before it compounds. Look at them less like monitoring thresholds and more like circuit breakers.
Latency and accuracy already live in dashboards, and cost should be there too. A developer should be able to see what a prompt chain costs before it ships. If they can, the definition of ‘done’ starts to expand beyond passing tests. It starts to include whether the team can actually explain what a feature costs to run. Consider variables like cost-per-resolved-ticket, cost-per-generated-quote, and/or cost-per-contract-review. By measuring at that level, optimization stops being a vague mandate from finance and gets specific and (actually) actionable.
Ship what you can explain
AI budgets continue migrating from a neat line item that someone in infra or procurement managed, and into engineering (whether engineering asked for it or not). Spending decisions are made in code…every prompt chain, routing decision, and retry policy. The teams figuring this out early will ship AI features with more confidence. They can see the economics at design time and set boundaries that hold at runtime. When someone asks what a feature costs, they have an answer.
