The real cost of conversation history
In a multi-turn conversation, every earlier message is re-sent on every new turn. Cost grows with the square of conversation length. Here is the math, and four ways to bend the curve back down.
Tag
1 article
Tags cut across the blog’s categories: the articles below approach tokens from whichever angle their subject demanded — a pricing breakdown, a benchmark reading, a migration note, or a postmortem from running something in production. They are ordered newest first, so the top of the list reflects where our current thinking sits rather than where it started.
In a multi-turn conversation, every earlier message is re-sent on every new turn. Cost grows with the square of conversation length. Here is the math, and four ways to bend the curve back down.
Articles tagged “tokens” also carry the tags below. They are ranked by how often they appear alongside this one, so the first few are the closest neighbours to what you are already reading.