This is the anatomy of an LLM request — where tokens come from and why enterprise AI bills explode. It traces every source of tokens in a single request so you can control cost.
In simple terms
Every part of a request adds tokens — and tokens are what you pay for.
How it works
- 1A user request hits the API gateway (auth, rate limiting, validation).
- 2A prompt builder combines the system prompt, chat history and retrieved docs (RAG).
- 3The user prompt, tool results and structured data all add tokens.
- 4Everything is combined into the complete prompt sent to the LLM.
- 5The model responds; billing = input tokens + output tokens.
Key points
- Input tokens (prompt side) + output tokens (response side) = your bill.
- Bills explode from huge system prompts, long chat history, too much RAG context, verbose tool results and no optimization.
- The context window limits how much fits at once.
- You pay for tokens, not characters — and they add up fast.
Why it matters
Understanding where tokens come from is the first step to controlling AI cost — trimming prompts, history and retrieved context is often the biggest lever on the bill.
Frequently asked questions
- Why do enterprise AI bills explode?
- Long system prompts, full chat history, too many retrieved docs and verbose outputs all multiply token usage.
- How do I cut cost?
- Trim and cache context, retrieve only what's needed, compress history and route to smaller models.