The compounding, made concrete
Take claude-sonnet-4-6 ($3 input, $15 output per million), a 1,000 token system prompt, and turns averaging 100 tokens of user input and 250 tokens of reply:
turn N input = 1000 + (N-1) * 350 + 100 tokens // prompt + history + new msg
turn N output = 250 tokens
5 turns: ~7,900 input + 1,250 output -> ~$0.043
10 turns: ~26,750 input + 2,500 output -> ~$0.118
20 turns: ~88,500 input + 5,000 output -> ~$0.34
40 turns: ~316,000 input + 10,000 output -> ~$1.10Four times the turns, eight to ten times the cost. This is why chat features that looked affordable in a five turn demo surprise their builders in production, and why the cost of adding AI to your app puts chat a full tier above single shot features.
The four levers, in the order to pull them
- Cache the prefix. The system prompt and settled early history are byte stable across turns, exactly what prompt caching wants: reads at about a tenth of fresh input, per prompt caching economics. On the 20 turn example, caching well cuts the bill by more than half.
- Summarize old history. Past a threshold, replace verbatim old turns with a model written summary. One cheap summarization call buys thousands of tokens off every subsequent turn.
- Route by conversation weight. Casual chat on a budget model, escalation to the premium model when the conversation gets hard, per model routing for LLM apps.
- Bound the marathon. Nudge fresh sessions in the product, and back it with per user caps that downgrade the model rather than cutting the user off.
Budgeting per user: the tail is the budget
Realistic monthly ranges on a mid tier model: light users $1 to $3, engaged users $5 to $15, power users past $30. The catch is that your power users are your most engaged, most valuable, and least profitable accounts simultaneously, the exact pattern from one user costing more than they pay. A flat $19 plan across that distribution loses money precisely on the users who love the product most, which is a strange thing to discover from an invoice. Per user measurement with conversation level detail, plus a cap that softens rather than blocks, is what makes chat shippable on flat pricing. Weckr provides exactly that pair from two lines around your client, with cost recomputed at current rates and margin against each user’s plan.
FAQ
How much does an AI chatbot cost per conversation?
On a mid tier model, a short conversation of 5 turns costs a few cents, but cost grows superlinearly with length because every turn resends the conversation so far. A 20 turn conversation is not 4 times a 5 turn one, it is closer to 10 times, and on claude-sonnet-4-6 can reach 30 to 60 cents. Cost per conversation is dominated by the long tail of long conversations.
Why does chatbot cost grow faster than message count?
Context resend. Turn N includes the system prompt plus all previous turns as input tokens, so input cost grows roughly with the square of conversation length while output grows linearly. This is the single most important fact about chat economics, and it is why per message pricing intuitions fail: the 30th message can cost ten times the 3rd.
How do I reduce AI chatbot costs without ruining quality?
Four levers in order: cache the stable prefix (system prompt and early history read at about a tenth of the fresh rate), cap or summarize old history past a threshold instead of resending everything verbatim, route to a cheaper model for casual conversations and escalate for hard ones, and bound conversation length in the product with a fresh session nudge. Most teams get 40 to 70 percent off with the first two alone.
What should I budget per user for a chat feature?
Ranges at current mid tier rates: a light user having a few short conversations a week runs $1 to $3 a month, an engaged daily user $5 to $15, and a power user with long daily conversations can exceed $30. The average is misleading because the distribution is heavy tailed, budget for the p95 and cap beyond it, or a flat plan quietly loses money on its best loved users.
How do I track chatbot cost per conversation and per user?
Tag every call with the user id and a conversation scoped feature label, log usage tokens including cached counts, and price at current rates. Rolling up by the label gives cost per conversation, by user gives monthly cost against plan price. Weckr does this from two lines and adds the piece chat needs most: per user caps with model downgrade, so the marathon conversations get cheaper instead of unbounded.
Keep reading
Love your power users. Meter them anyway.
Chat rewards engagement with compounding cost, so the feature’s success is exactly what threatens its margin. Weckr shows cost per user and conversation as it happens and caps the tail with downgrades instead of walls, free for 50,000 requests a month. See it on the live demo, or start with the AI cost and margin guide.