稠密嵌入与「Skip-Gram」

Dense Embeddings with Skip-Gram

Negative sampling and its loss

Training skip-gram in NumPy

A target and context embedding have dot product $\mathbf{c} \cdot \mathbf{w} = -2$. Compute the classifier's probability $P(+ \mid w, c) = \sigma(\mathbf{c} \cdot \mathbf{w})$ and round to three decimals.

A two-word lexicon has unigram probabilities $0.99$ and $0.01$. Compute the smoothed probability of the frequent word under $P_{0.75}$, that is $\frac{0.99^{0.75}}{0.99^{0.75} + 0.01^{0.75}}$ and round to three decimals.

A positive pair has $\mathbf{c}_{\text{pos}} \cdot \mathbf{w} = 1$ and each of the $2$ noise words has $\mathbf{c} \cdot \mathbf{w} = -1$. Compute the loss $L = -[\log \sigma(1) + 2 \log \sigma(1)]$ and round to three decimals.

A training pair has label $y = 1$ and the classifier currently outputs $\sigma(\mathbf{c} \cdot \mathbf{w}) = 0.7$. According to the update $\mathbf{{w}} \leftarrow \mathbf{{w}} - \eta [\sigma(\mathbf{{c}} \cdot \mathbf{{w}}) - y]\,\mathbf{{c}}$, what happens to the dot product $\mathbf{{c}} \cdot \mathbf{{w}}$ after the step?

Which set of statements about static embeddings is correct?