Natural Language Processing · 免费预览
基于tf-idf的稀疏检索
向量空间模型、词频与逆文档频率、余弦评分、BM25,以及使排序快速的倒排索引。
1. Sparse Retrieval with tf-idf
Sparse Retrieval with tf-idf
2. Inverse document frequency
A collection has documents and a term occurs in of them. Compute , rounding to three decimals.
3. Damped term frequency
A term appears 1 times in a document. Compute its tf as , rounding to three decimals.
4. A perfect cosine
A one-word document contains exactly the query's single term, with everything else zero. What cosine does the tf-idf scoring assign, and why?
5. What the postings list holds
A user queries two terms. How does the inverted index make scoring efficient?
6. The two knobs of BM25
In BM25, what do the parameters and control, and what happens at the extremes?