NLP

The smallest unit of text that a language model processes. One token is roughly 4 characters in English or 1–2 characters in Russian. API pricing is typically calculated per token.

Related terms