This paper addresses two closely related questions, "What is a statistical model?" and "What is a parameter?" The notions that a model must "make sense," and that a parameter must "have a well-defined ...

Have a general understanding of the current state-of-the art in statistical language models. Understand how at least one statistical language model is implemented and can be applied (via the course ...

This paper presents a novel method to segment/decode DNA sequences based on n-gram statistical language model. Firstly, we find the length of most DNA “words” is 12 to 15 bps by analyzing the ...

But when a large language model scores well on such tests, it is not clear at all what has been measured. Is it evidence of actual understanding? A mindless statistical trick? Rote repetition?

It looks like a bug, but it’s just the LLM doing what it always does. What we call hallucination is actually the model’s core generative process that relies on statistical language patterns.