Summary

• Even though every model charges its API customers per token, tokens are far from a standard unit.
• Each model has its own tokenizer that decides how many tokens your prompt becomes.
• Tokenization is a non-transparent billing unit that each vendor controls unilaterally.
• The smart move does seem to be multi-model capability (even if 95% of volume goes to one vendor) plus internal benchmarks run on your actual prompts.

A couple of weeks ago, I highlighted a couple of pieces by Anjali Shrivastava, who made the point that a token is not a fixed unit of cost. This piece is a follow-up driving the point home that even

787 Followers