Natural Language Processing · free preview
Perplexity as a Language Model Score
Training, development and test splits, the perplexity formula as a geometric mean, branching-factor intuition, and the cross-entropy view.
1. Perplexity as a Language Model Score
Perplexity as a Language Model Score
2. The perplexity formula and its meaning
The perplexity formula and its meaning
3. Computing perplexity under two models
Computing perplexity under two models
4. Uniform branching factor
A signal lamp flashes one of 12 distinct colors, each with equal probability, and each flash is independent of the last. What is the perplexity of any sequence of flashes under this model?
5. Geometric mean of inverse probabilities
A model assigned these probabilities to the 3 tokens of a test sequence: 0.5, 0.4, 0.25. Compute the perplexity, rounded to two decimal places.
6. A skewed three-color lamp
A lamp emits amber with probability 0.6 and white and green each with probability 0.200. A test sequence holds 3 ambers and one white. Compute its perplexity under this model, rounded to two decimals.
7. Unigram perplexity on held-out text
A unigram model is trained on these padded sentences: the gulls circle; the tide turns; the fog thickens. Score the padded test sentence "<s> the fog turns </s>" with unigram probabilities (predict each of the 4 tokens after the start symbol). Round to two decimals.
8. From bits to perplexity
A language model achieves a cross-entropy of 6.2 bits per word on a test set. What is its perplexity on that test set?
Practice this interactively
Free account · instant grading · spaced review that schedules itself.
Start this course for free