Data Engineering
Vector search: multiple-choice review
Crux Multiple-choice synthesis across the vector-search unit — recall vs latency, HNSW vs IVF-PQ, distance metrics, post-filtering, and the silent-recall failure mode.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitSix questions that cut across the whole unit. Each mirrors a decision you make staring at a RAG pipeline that returns ten ranked rows, fast, with no errors — and the wrong ten.
Goal
Confirm you can connect embeddings, ANN index choice, the recall–latency–memory triangle, distance metrics, and the production traps — the synthesis the lesson built toward.
Quiz
Completed
A RAG search returns ten ranked rows in 2ms with zero errors, but support reports the bot 'can't find' docs it clearly has. What is the most likely root cause, and how do you confirm it?
Heads-up It returns the full ten rows in 2ms with no errors — the DB is healthy. Low recall is invisible: it looks identical to high recall in row count and latency.
Heads-up Queries succeed in 2ms; the symptom is wrong results, not slow or failed ones. That points at recall, not the embedding call.
Heads-up Throughput is not the problem — correctness is. Replicating a low-recall index just multiplies the wrong answers.
Quiz
Completed
A RAG service over ~5M chunks has continuous ingestion, needs high recall, and has RAM to spare. Which index is the right default and why?
Heads-up IVF centroids are fixed at build time, so continuous ingestion steadily drifts recall until you reindex. With RAM available and high recall required, you would be choosing the index that drifts for a memory win you do not need.
Heads-up PQ is for corpora that will not fit in RAM at all (100M+). At ~5M with RAM to spare, you pay PQ's quantization recall loss for compression you do not need.
Heads-up Exact kNN is recall 1.0 but O(N·d); at 5M times 1536 dims every query is billions of multiply-adds, so p99 is unshippable. It is the offline baseline, not the serving path.
Quiz
Completed
Raising hnsw.ef_search from 100 to 500 moves recall from ~85% to ~98% but latency from ~1ms to ~5ms. How should a senior engineer read this?
Heads-up No ANN index improves recall and latency together at query time; they trade against each other along ef_search. Those numbers are the normal shape of the dial.
Heads-up M is a build-time knob affecting graph density and memory; it does not replace ef_search. Rebuilding is far more expensive than turning the runtime dial to hit the target.
Heads-up IVF has the same recall/latency tradeoff via probes, plus centroid drift. Switching indexes does not escape the triangle — every ANN method trades recall for speed.
Quiz
Completed
A team stores normalized embeddings and debates cosine similarity vs inner (dot) product. What is the correct call?
Heads-up Cosine is dot product divided by magnitudes. On already-normalized vectors the magnitudes are 1, so the two produce identical rankings — cosine just adds redundant work.
Heads-up L2 ranks differently from cosine unless vectors are normalized, and using a metric the embeddings were not trained for degrades ranking. The metric must match the embedding model, not default to L2.
Heads-up The metric defines what 'nearest' means and changes ranking unless vectors are normalized. Picking the wrong one for the model silently worsens relevance.
Quiz
Completed
A tenant-scoped search adds WHERE tenant_id = ? on top of an HNSW query, and recall drops sharply for small tenants. Why, and what is the fix?
Heads-up A B-tree on tenant_id speeds the filter but does not change that ANN candidates are gathered before the filter is applied — the recall collapse is from post-filtering order, not filter speed.
Heads-up Even with plenty of matching docs, post-filtering throws most ef_search candidates away because they belong to other tenants. The problem is ordering, not corpus size.
Heads-up The metric is irrelevant here. The recall loss comes from the filter being applied after the graph walk has already narrowed to ef_search candidates.
Quiz
Completed
Users search for the exact error string 'ECONNREFUSED' and pure vector search buries the matching doc beneath vaguely-related paragraphs. Best fix?
Heads-up More candidates of the same kind do not help — the need is lexical, and embeddings smear an exact token like an error code into 'semantically near'. ef_search tunes vector recall, not exact-match precision.
Heads-up A bigger model still measures semantic similarity; it does not give exact-token precision. Error codes, SKUs, and identifiers are a keyword need that BM25 handles.
Heads-up Metric choice nudges ranking but never gives lexical precision. Vector search lacks exact-match capability regardless of metric; hybrid adds the keyword leg that has it.
Recap
The through-line is one triangle — recall, latency, memory — and one habit: recall drops are silent. The candidate pool (ef_search / probes) sets recall vs latency, the index family sets memory and drift (HNSW for changing data, IVFFlat for static, IVF-PQ when it will not fit RAM), the metric must match how embeddings were trained, post-filtering wrecks selective filters (use iterative scan), and exact-token needs demand hybrid BM25 + vector with rank fusion. Above all, measure recall@k against an exact baseline or you are flying blind.