N-gram Language Models and the Markov Assumption

N-gram Language Models and the Markov Assumption

Estimating n-gram probabilities by relative frequency

A bigram model over a small logbook corpus

In a corpus the word "fog" occurs 90 times, and the sequence "fog horn" occurs 12 times. What is the maximum likelihood estimate of $P(\text{horn} \mid \text{fog})$? Give the answer rounded to three decimal places.

A bigram model assigns these probabilities along the sentence "the harbour lights burn": 0.4 0.2 0.25. What probability does the model give the whole sentence? Use five decimal places.

A model stores four factor probabilities as natural logs: -3.45, -3.07, -1.7, -3.82. What is the sum of the log probabilities, and hence the log of the sentence probability? Give the log-space total rounded to two decimals.

A corpus consists of these sentences: the tide turns; fog hides the buoys; the fog lifts. Every sentence is padded with a start symbol $\langle s\rangle$ and an end symbol $\langle /s\rangle$. Compute $P(tide \mid the)$ under the bigram MLE. Use three decimals.

With the same corpus (the tide turns; fog hides the buoys; the fog lifts, padded with boundary symbols), what probability does the bigram model assign to the sentence "the fog lifts" including its boundary symbols? Round to four decimals.