Embeddings for Retrieval

Setup

Section 0 — Corpus + labeled queries

Section 1 — Tokenization + vocabulary

Section 2 — Similarity metrics

Section 3 — TF-IDF retrieval baseline

Section 4 — Dense-ish baseline: co-occurrence + SVD word vectors

Section 5 — Evaluation: Recall@k and MRR