Smoothing, Interpolation, and Backoff

Smoothing, Interpolation, and Backoff

Adding counts, blending orders, backing off

Smoothing a bigram table three ways

A word occurs 29 times in a corpus of 1,089,757 tokens with vocabulary size 5,873. What is its Laplace-smoothed (add-one) unigram probability? Round to six decimal places.

In a corpus with vocabulary size 1,534, the word "tide" occurs 120 times and the sequence "tide turns" occurs 18 times. What is the add-one smoothed estimate of $P(\text{turns} \mid \text{tide})$? Use four decimal places.

With $k = 0.1$, vocabulary size 5,733, prefix count $C(\text{fog}) = 78$ and pair count $C(\text{fog drift}) = 8$, compute the add-k smoothed $P(\text{drift} \mid \text{fog})$. Round to four decimal places.

A trigram model is smoothed by interpolation with $\lambda_1 = 0.4$ on the unigram estimate, $\lambda_2 = 0.6$ on the bigram estimate, and $\lambda_3 = 0$ on the trigram (its count is zero). Suppose $P(w) = 0.03$ and $P(w \mid h) = 0.5$ for the word in question. What is the interpolated probability? Use three decimals.

A stupid backoff model uses weight $\alpha = 0.4$ and terminates in the unigram distribution. For the next word $w$, the trigram count is zero, the bigram count is positive with relative frequency 0.5 and the bigram estimate is 0.5. Compute the stupid backoff score $S(w \mid \text{context})$. Round to four decimal places.