AI does not read words — it reads tokens. A token is roughly a word or part of a word. Each token gets converted into a number (an embedding vector) that represents its position in a learned meaning space. The colour shows the token type: key meaning words, ambiguous words, stop words (words AI often filters out as low meaning).
For each possible interpretation, AI assigns a probability based on how often these words appear together in that context across its training data. Higher probability does not mean correct — it means statistically common.
The AI picks the highest probability interpretation as its best guess. This is not understanding — it is the most statistically likely meaning based on pattern matching against training data.
When the top two interpretations are close in probability, the sentence is highly ambiguous. Humans resolve ambiguity through tone, body language, relationship and context. AI only has the words.
This panel shows which specific features of the sentence create difficulty for AI systems. These are real problems in natural language processing — not simplifications.
Humans understand language through a lifetime of social experience, emotion, relationship, tone and physical context. AI has none of these. It has only patterns from text.
Tokens not words
AI splits text into tokens and converts each to a number. It never truly "reads" — it processes numerical vectors.
Probability not meaning
Every interpretation has a score. AI picks the highest. That can be wrong when context is unusual, regional or emotional.
Slang breaks AI
Regional phrases, new slang and neurodivergent communication styles often appear rarely in training data — so AI scores them poorly.
Sarcasm is hard
Sarcasm reverses meaning entirely. Without tone of voice or facial expression, AI often misses it completely.
Context shifts everything
"I am in pain but enjoy it" means something completely different in a gym, a tattoo parlour or a crisis conversation.
Ambiguity is normal
Most human language is ambiguous. Humans fill the gaps with social knowledge. AI fills them with statistics.