Notes · Series
Search and retrieval
How systems find and rank what is relevant: from classic ranking models to neural retrievers and evaluation, through retrieval-augmented generation (RAG) to search agents.
Parts
16From asking a question to search agents
Why we look for information, how we express a need, and the ideas that connect a question to useful evidence.
What makes a document relevant?
Why a page can be on topic without helping, and how the person, task, and moment shape what counts as a useful result.
From raw text to an indexWork in progress
From Boolean search to BM25Work in progress
How do we know a search system works?Work in progress
Improving the query, feedback, expansion, and fusionWork in progress
The web changes the game, links, authority, and diversityWork in progress
Learning to rank, and learning from clicksWork in progress
From words to meaning, embeddings and transformersWork in progress
Reading closely, reranking with BERTWork in progress
Dense retrieval, searching by meaningWork in progress
Back to the index, learned sparse retrievalWork in progress
Asking a language model to rankWork in progress
Can a language model judge relevance?Work in progress
Retrieval-augmented generation, from the retrieval sideWork in progress
Search agents, models that search for themselvesWork in progress