Token
In natural language processing, a token is a unit of text that is processed by the AI, typically representing a word or a part of a word. However, a token does not have a fixed length in terms of characters or words. Instead, a token can vary based on the complexity of the language and the content.
MaintainedLast reviewed
Source: Hong Kong Judiciary GenAI Guidelines (Annex) · checked 2026-10-10