tokens

Tokenization, context windows, token economics

Topics

Cost and cachingLLM costs are driven by tokens: input, output (including thinking), and cached input. Most savings come from not reprocessing the same context, choosing the right effort and model for each task,…
Memory for agentsModels have no memory between requests: each call sees only what is in its context. Agents that work for long periods or across sessions need ways to keep what matters and drop what does not.…
Tokens and context windowsModels read and write tokens, not characters or words. A tokenizer splits text into sub-word pieces from a fixed vocabulary. Everything about using an LLM is measured in tokens: price, speed, rate…