Software has spent two decades with roughly zero marginal cost per user action. AI features break that. Every generation costs real money, the cost scales with usage rather than with customers, and heavy users can cost more than they pay.
This is a product problem long before it is a finance problem, because the decisions that determine the cost — how much context you send, how often you call, which model, whether you cache — are all product decisions.
Size it before you build
The calculation takes ten minutes and should happen during discovery, not after launch.
Cost per call × calls per user per month × users = monthly cost.
Cost per call is driven mostly by tokens in, not tokens out. A feature that stuffs a whole document into context on every request is expensive in a way that is invisible in a demo with one small file.
Then the question that matters:
Monthly cost per user ÷ monthly revenue per user = the share of your margin this feature eats.
On a $20/month plan, a feature costing $6 per active user has taken 30% of that revenue before support, hosting or salaries. That may still be a good trade. But it should be a decision, and it is frequently a discovery.
AI feature usage is heavily skewed — a small fraction of users generate most of the calls. Model the top decile separately. Your unprofitable customers are not hypothetical, they are the ones who love the feature most, and on a flat-rate plan they are the ones you acquired most successfully.
Where the cost actually goes
| Driver | Typical mistake | Lever |
|---|---|---|
| Context size | Sending the entire document or full history every call | Retrieve only relevant chunks; truncate history |
| Model choice | Using the largest model for every task | Route simple tasks to a smaller, cheaper model |
| Call frequency | Regenerating on every keystroke or page load | Debounce; make generation explicit and user-triggered |
| Retries | Silent retry loops on malformed output | Cap retries; fix the prompt or validate cheaply first |
| Cache misses | Recomputing identical requests | Cache by input hash; many workloads repeat more than you expect |
Model routing is usually the largest single win available and the most neglected. A great deal of production traffic is classification, extraction or short rewriting — tasks a smaller model handles at a fraction of the cost. Reserve the expensive model for the calls that genuinely need it.
Four pricing models, and what each one breaks
Included in the base plan
Simplest, best for adoption, and the one that exposes you most. Fine when cost per user is genuinely small or the feature drives retention worth more than it costs. Always pair it with a fair-use ceiling, even a generous one, so a single automated integration cannot run up an unbounded bill.
Credits or a usage allowance
Aligns cost with revenue and is now the common pattern. The cost is comprehension: users must understand what consumes a credit, and "it depends on document length" is a genuinely bad answer at the point of purchase. Whatever you choose, make one action equal one visible unit if you possibly can.
Separate paid add-on
Cleanest margin story, and it caps adoption hard. Sensible when the feature serves a distinct segment. Risky as a default, because you learn much less about a feature only your most committed users touch.
Higher tier gate
Push AI features into a premium plan. Good for upsell, and it means your free and entry tiers never demonstrate the thing you most want to demonstrate. Consider a small taste at the lower tier — enough to establish the value, capped enough to protect the margin.
Log tokens and cost per request, attributed to a user and a feature. Retrofitting this after launch is painful, and without it you cannot answer the two questions finance will eventually ask: which customers are unprofitable, and which feature is responsible.
The direction of travel
Model prices have fallen sharply and repeatedly, which tempts teams to build as though cost will solve itself. Two cautions. First, capability tends to expand into the available budget — cheaper tokens get spent on longer context and more calls, so bills often stay flat while quality rises. Second, "it will get cheaper" is not a plan you can present when the current burn is real.
Build for today's prices with the levers above in place. If costs fall, you get margin back automatically. If they do not, you are still solvent.
For whether the feature should exist at all, see when AI is the wrong answer. For knowing whether it works once shipped, see evaluating AI features.