Estimate AI API Cost With Scenarios

Act as a careful finance analyst who costs AI API usage for small businesses.

Task: Estimate the monthly cost of A customer support assistant for an online furniture shop. It handles about 400,000 chats a month. Each chat sends roughly 1,800 input tokens, including a long fixed policy prompt, and returns about 350 output tokens. About 70% of input tokens are expected to be served from cache. using the price list in Per 1M tokens, off-peak: cached input $0.003, uncached input $0.15, output $0.60. Peak prices are exactly double each off-peak price. Peak hours are 01:00-04:00 and 06:00-10:00 UTC, Monday to Friday., and show which assumption moves the total most.

Steps:
1. Convert the workload into three monthly totals in millions of tokens: cached input, uncached input and output. Treat the cache share as a share of input tokens, not of requests.
2. Split each total into peak and off-peak amounts using 30% of every token category is processed at peak, because most chats happen in UK evenings and weekends, with a small share in weekday mornings., apply the matching prices to each, and add the subtotals. Show every calculation.
3. Build a base case, then change exactly one assumption at a time: no caching, strong caching, shorter output, longer output, all peak, all off-peak.
4. Name the single assumption that changes the total most, and list what I should measure from real usage to replace it.

Output: A calculation block for the base case, a table of scenarios with monthly cost and the difference from base, then a three-sentence conclusion. Flag any figure you assumed. Do not recommend a model on cost alone.

# Monthly cost estimate: furniture shop support assistant ## Base case calculation **Step 1: monthly token totals (M = million tokens)** - Input: 400,000 × 1,800 = 720 M - Cached input: 720 × 70% = 504 M - Uncached input: 720 × 30% = 216 M - Output: 400,000 × 350 = 140 M **Step 2: peak/off-peak split (30% peak, a given assumption) and pricing** Peak prices are exactly double: cached $0.006, uncached $0.30, output $1.20. | Category | Peak M | Off-peak M | Peak cost | Off-peak cost | Subtotal | |---|---|---|---|---|---| | Cached input | 151.2 | 352.8 | 151.2 × 0.006 = $0.9072 | 352.8 × 0.003 = $1.0584 | $1.97 | | Uncached input | 64.8 | 151.2 | 64.8 × 0.30 = $19.44 | 151.2 × 0.15 = $22.68 | $42.12 | | Output | 42.0 | 98.0 | 42 × 1.20 = $50.40 | 98 × 0.60 = $58.80 | $109.20 | **Base total = 1.9656 + 42.12 + 109.20 = $153.29 per month** (about $0.00038 per chat). **Cross-check:** the all-off-peak cost is 1.512 + 32.40 + 84.00 = $117.912. With 30% of tokens at double price, the blended multiplier is 0.7 + 0.3 × 2 = 1.3, and 117.912 × 1.3 = $153.29. ✔ ## Scenarios (one change at a time) | Scenario | Change | Off-peak-priced base | Monthly cost | Δ vs base | Δ % | |---|---|---|---|---|---| | **Base** | 70% cache, 350 out, 30% peak | $117.91 | **$153.29** | n/a | n/a | | No caching | cache share 0% | 720 × 0.15 + 84 = $192.00 | $249.60 | +$96.31 | +62.8% | | Strong caching* | cache share 90% | 1.944 + 10.80 + 84 = $96.74 | $125.77 | −$27.52 | −18.0% | | Shorter output* | 250 tokens (100 M) | 1.512 + 32.40 + 60 = $93.91 | $122.09 | −$31.20 | −20.4% | | Longer output* | 500 tokens (200 M) | 1.512 + 32.40 + 120 = $153.91 | $200.09 | +$46.80 | +30.5% | | All peak | 100% peak | $117.91 × 2 | $235.82 | +$82.54 | +53.8% | | All off-peak | 0% peak | $117.91 × 1 | $117.91 | −$35.37 | −23.1% | *The 90% cache share, 250-token output and 500-token output are my own illustrative values; you didn't specify them. ## Assumptions flagged - **Given by you:** 400,000 chats, 1,800 input tokens, 350 output tokens, 70% cache share, 30% peak share, and the prices. - **Assumed by me:** the scenario values above, and the "no caching" case (0% cached). - **Not modelled:** cache-write or storage fees (none were in the price list), retries, tool calls, and growing conversation history in multi-turn chats. Any of these would raise the real figure. - **Check:** a 30% peak share looks high for a UK-evening-heavy shop. Peak covers only 7 hours on weekdays, about 21% of weekly hours, so measure it. ## Conclusion Within the ranges I tested, the cache share moves the total most (losing caching adds $96.31, or 62.8%), followed by the peak share (all peak adds $82.54), but the cache range is the widest one I chose, so this ranking depends on my scenario values. Output tokens are structurally the biggest cost driver, at about 71% of off-peak spend, so a realistic output-length error could matter as much as a cache miss. From real usage, measure the actual cached-token share (and any cache-write charges), the distribution of output tokens per chat, the hourly UTC chat distribution to find the true peak share, the fixed versus variable input tokens per chat, and the average number of turns and retries per conversation.