Word Vectors from Co-Occurrence Counts

Word Vectors from Co-Occurrence Counts

The word-context matrix

Counting the harbour logbook

Two words have context vectors over the same dimensions. Word A has a nonzero count (1) in a dimension where word B is zero, and word B has a nonzero count (3) in a dimension where word A is zero, and they share no dimension where both are nonzero. What is their cosine similarity? Answer as a decimal.

Compute $\cos(\mathbf{v}, \mathbf{w})$ for $\mathbf{v} = [3, 1]$ and $\mathbf{w} = [1, 2]$ and round to three decimals.

A raw dot product ranks ferry as more similar to a very frequent word than to tug, even though ferry and tug share most of their contexts. Why does the cosine fix this?

In a corpus, the word "gull" occurs in 3 sentences, and in every one of those sentences "mast" appears exactly once inside the context window around "gull". What value goes in the cell for row "gull", column "mast" of the word-context matrix? Answer with the number.

Two words accumulate exactly the same context counts over a corpus, every dimension matching. What is their cosine, and what does the outcome mean?