Think of the billions of numbers inside a large language model as a vast spreadsheet that captures the statistical likelihood that certain words will appear alongside certain other words.

But when a large language model scores well on such tests, it is not clear at all what has been measured. Is it evidence of actual understanding? A mindless statistical trick? Rote repetition?

Wonder what is really powering your ChatGPT or Gemini chatbots? This is everything you need to know about large language models.

It looks like a bug, but it’s just the LLM doing what it always does. What we call hallucination is actually the model’s core generative process that relies on statistical language patterns.