Retrieval-augmented generation
Retrieval-augmented generation (RAG) lets a language model answer a question with a set of readings open in front of it. The system searches a collection, then gives the model the question and the passages it found.
How does the answer get its sources?
Imagine a student asking how AI affects inequality. A search of the reading list brings back an argument about wages and the income earned by innovators. The language model can now use the authors’ argument to answer the student’s question.
- Student’s question
“How does AI affect inequality?”
- Search the reading list
Find passages by meaning in the vector database and by keyword.
- Large language model (LLM)
Read the question + passages and draft a reply.
- Reply to the student
“Innovators can capture some of the surplus.” [1]
The reading list is built ahead of time. Each new question determines which passages the model gets to read.
A reading assistant for the economics of AI
The vector database lesson showed how to prepare and search a collection. Here we use those ideas to answer questions about AI, growth and the distribution of income. A researcher chooses the literature. The language model uses passages from it while writing, so adding a paper changes the evidence available for the next answer without retraining the model.
1 Start with a reading list
For the downloadable example I use short notes on three public works. Korinek and Stiglitz examine how AI can affect the distribution of income and, in a separate paper, development strategies. Trammell and Korinek ask how automation could change economic growth. A question about inequality could turn up relevant ideas in all three, but the papers address different outcomes.
The notebook divides the notes into six passages. Each carries its text, a numerical representation for search and a reference to its paper. That gives the reader a route back from a claim in the answer to the work it draws on.
2 Find passages for the question
How does AI affect inequality?
There are two clues a search can use. A passage about income distribution can be close in meaning to the question, even if it uses different words. A passage that uses the question’s own vocabulary is also worth checking. The notebook searches both ways and combines the two rankings.
- Development strategiesCould automation erode the advantage of low labor costs in export industries?
- Income distributionWho receives the surplus from an innovation, and how do wages and returns to capital change?
- Economic growthHow does labor’s share of aggregate income differ from inequality across households?
The development passage comes first, even though the second passage more directly answers the student. A search ranking tells the model what to read. The order in which to explain the economics is a separate judgment.
How the notebook combines the rankings
The vector search puts income distribution first and development second. The word search, called BM25, reverses them. The notebook combines positions using reciprocal rank fusion: each search contributes more for a passage near the top. These two passages tie, so the notebook orders them by passage ID. Their order here carries no economic significance.
3 Write an answer from the passages
The system puts the original question, the three passages and their source labels into one message for the writer. This message is the prompt. It asks for two sentences explaining the mechanisms, with a reference to the passage that supports each claim.
“Innovators can capture some of the surplus. Changes in wages and returns to capital can also shift income among households with different endowments.”
The excerpt identifies two channels. A household can benefit by owning the innovation itself. It can also gain or lose because the value of its labor or capital changes. A numerical prediction would require knowing who owns what and how much wages and returns move.
The full draft opens with export strategy and repeats much of the supplied text. A reader can use the citation to check the claim, then decide whether a detour into development helps answer this particular question.
4 Ask a follow-up
A student can now ask about “those channels.” To make that phrase meaningful to the search, the notebook adds the first question to the follow-up and searches the library again.
How much did those channels change the US Gini coefficient in 2024?
The retrieved notes do not cover 2024. They cannot support the requested estimate.
That reply comes from a simple rule in the notebook: if the student asks for a year absent from all the selected passages, report the gap. Even if a passage mentioned 2024, a reader would still need to check whether it estimates the US Gini coefficient and attributes a change to those mechanisms.
What else could an economist build?
A tutor for a course. Give the assistant the syllabus, assigned papers and lecture notes. When a student asks why a falling labor share need not imply higher household inequality, it could point to the relevant reading, ask who owns the capital in the student’s example, and use the reply to guide the next turn.
A literature review assistant. Ask it to draft a paragraph comparing how several papers model technical change. Keep the source attached to each assumption so a researcher can check where two models differ before using the paragraph.
A modelling or coding assistant. Include model appendices and their replication code in the collection. A researcher could ask where a paper’s elasticity parameter enters the equations and which line of code sets its value. A good answer would point to both.
When would the language model initiate the search?
In the example, retrieval always comes first. In agentic RAG, the model can decide to search, read what comes back and ask the library a narrower question before answering. For a comparison across papers, it might need a second search for a missing assumption. A graph can also help a retriever follow a citation from one paper to another. LangChain and LlamaIndex offer ways to connect these components.
Try the offline example
Open the RAG notebook in Jupyter or VS Code, choose a Python 3 kernel and select Run All. It rebuilds the index from the included notes and saved embeddings, runs both searches and reconstructs the prompt. For the first turn it displays the reply Qwen generated for that exact prompt. The second turn runs the evidence check. To ask your own question and generate a fresh reply, follow the optional local-model instructions in the notebook.