How to Count Tokens Before Sending a Prompt to an LLM

Every LLM charges by the token and thinks in a fixed-size window, so a prompt you never measured is a prompt you do not control. Paste a long document into a chat window and one of three things happens: it works, it gets silently truncated, or your API bill surprises you. Counting tokens before you send takes ten seconds and prevents all three. A token is not a word and not a character — it is the chunk an LLM's tokenizer actually sees — and once you understand how text turns into tokens, you can predict cost, fit prompts into context windows, and trim the fat without losing meaning.

What a token actually is

Large language models never see raw text. Before your prompt reaches the model, a tokenizer splits it into tokens: common words become one token, rare words split into several, and punctuation and spaces get their own. The word 'hello' is one token. 'Unbelievable' might be three: 'un', 'believ', 'able'. Code splits even more aggressively — an identifier like getElementById can become four or five tokens. This is why two prompts with the same character count can cost very different amounts. The tokenizer is fixed per model, so the same text always produces the same tokens for a given model, but different models tokenize differently.

Why the count matters

Three limits in every LLM workflow are measured in tokens. The context window caps how much the model can consider at once — input plus output combined. Pricing is per thousand tokens, usually split between cheaper input tokens and pricier output tokens. And output length caps how long a response can be. A prompt that fits in one model's 8,000-token window overflows a smaller one. A 'cheap' task repeated ten thousand times in a script is only cheap if you know the per-call token count. Counting first turns all of these from surprises into arithmetic.

The rough math

When you need a quick estimate without a tokenizer, the standard rule of thumb is one token per four characters of English text, or about 0.75 words per token. A thousand words of prose is roughly 1,300 tokens. That estimate holds for plain English. It breaks for code, JSON, and other languages, which tokenize less efficiently. You can estimate in a line of JavaScript:

function estimateTokens(text) {
  return Math.ceil(text.length / 4);
}

const prompt = "Summarize the following report in three bullet points.";
console.log(estimateTokens(prompt)); // ~14 tokens

Use this only for ballpark figures. For real budgeting or window checks, count with the actual tokenizer or a counter that approximates it closely, because the four-characters rule can be off by 30% or more on dense text.

What inflates your token count

Some content is token-hungry. Watch for these:

• JSON and code: brackets, quotes, and indentation each cost tokens. A 500-line JSON file can be several thousand tokens.
• Repetition: repeated boilerplate, headers, and disclaimers cost the same every time they appear.
• Non-English text: languages the tokenizer saw less often split into more tokens per word.
• Whitespace: long runs of spaces and blank lines still count. Minified text is cheaper than pretty-printed text.
• Long identifiers and URLs: a single 80-character URL can be 20 tokens on its own.

Trimming a prompt without losing meaning

When a prompt is over budget, cut in this order. First, remove what the model does not need: repeated instructions, verbose greetings, and context the model cannot use. Second, compress structured data: strip JSON to the fields the task needs, and shorten long identifiers in code samples. Third, summarize instead of pasting: a three-sentence summary of a document usually carries the facts a prompt needs at a tenth of the tokens. What you should not cut is the actual instruction — vague prompts produce vague output, and saving fifty tokens is not worth a useless answer. Keep the instruction crisp and cut the filler around it.

Try it

Paste any prompt into the LLM Token Counter to see its estimated tokens, characters, and words before you send it. If you write prompts often, the Prompt Template Generator builds reusable structured prompts with role, task, and constraints filled in. And if your prompt carries structured data, How to Format JSON in JavaScript shows how to minify it and cut tokens before pasting.