Why Token Counts Matter
Large language models do not read characters or words. They read tokens, which are chunks of text from a fixed vocabulary. Context windows, rate limits and API prices are all measured in tokens, so knowing the count before you send a prompt tells you whether it fits and what it will cost.
How Tokenization Works
OpenAI models use byte-pair encoding (BPE). Common words such as " the" become a single token, while rare words, code identifiers and many non-English words are split into several. A leading space is usually part of the token, which is why " hello" and "hello" have different IDs. The coloured view above shows exactly where each boundary falls.
- o200k_base: used by GPT-4o, GPT-4.1, GPT-5 and the o-series reasoning models.
- cl100k_base: used by GPT-4, GPT-3.5 Turbo and text-embedding-3 models.
Tips for Using Fewer Tokens
- Minify JSON you send as context; indentation and spaces are tokens too. Try the JSON Minifier.
- Remove repeated boilerplate and duplicate lines with Remove Duplicates.
- Put stable instructions first so providers that support prompt caching can reuse them.
Private by Design
Prompts often contain unreleased product details or customer data. This counter runs the tokenizer in your browser, and your text is never uploaded. For plain word and character statistics, see the Word Counter.