DIY #11 - Build RAG System with LangChain

💊 Pill of the week

In the rapidly evolving world of artificial intelligence, the need for accurate, contextually relevant information has become essential. Traditional language models, while powerful, sometimes generate responses that lack factual accuracy or up-to-date information. This is where Retrieval-Augmented Generation (RAG) comes into play.

RAG systems enhance the performance of generative models by grounding their outputs in real-world, retrievable documents

This approach combines the best of both worlds: the creativity and fluency of generative models with the factual accuracy of retrieval-based systems.

In this issue of MLPills, we will explore how to build a RAG system using LangChain, a versatile framework that simplifies the integration of language models with external data sources. We'll cover everything from the basics of RAG and LangChain to slightly more advanced optimization techniques and real-world applications.

👨‍💻 We include the notebook and data used at the end of the issue!

What is LangChain?

LangChain is an open-source framework designed to assist developers in creating applications that leverage the capabilities of large language models (LLMs). It provides a set of tools, components, and abstractions that make it easier to integrate language models with various data sources and services, whether for retrieval, processing, or generation tasks.

Key features of LangChain include:

1. Chainability: Allows the creation of complex workflows by chaining together different components.

2. Prompts Management: Offers tools for creating, managing, and optimizing prompts for language models.

3. Memory Integration: Provides mechanisms to give language models short-term and long-term memory capabilities.

4. Agent Framework: Enables the creation of AI agents that can make decisions and take actions.

5. Data Connection: Facilitates easy integration with various data sources and vector stores.

LangChain is particularly useful for building complex AI applications, such as RAG systems, where multiple components need to work together seamlessly. It supports various language models, including those from OpenAI, Hugging Face, and others, making it a flexible choice for developers.

Why Use RAG?

Before proceeding maybe you want to revise what RAG is:

Issue #56 - Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) offers several key advantages over traditional language generation techniques:

1. Improved Accuracy: By grounding generative outputs in real-world data, RAG systems produce responses that are more accurate and less prone to hallucinations. This is crucial for applications where factual correctness is paramount.

2. Real-Time Information Retrieval: RAG systems can retrieve up-to-date information from their knowledge base, making them ideal for scenarios where current data is crucial. This is particularly useful in fields like news reporting, market analysis, or providing the latest scientific information.

3. Context-Aware Responses: By using retrieved documents as a basis for generation, RAG systems can provide more contextually relevant answers. This is particularly useful in specialized domains like legal advice, medical information, or technical support, where domain-specific knowledge is critical.

4. Transparency and Explainability: RAG systems can provide the sources of their information, allowing users to verify the generated content. This transparency builds trust and allows for fact-checking.

5. Customizability: The knowledge base used by RAG systems can be easily updated or customized for specific use cases, allowing for greater flexibility compared to traditional language models with fixed training data.

6. Reduced Training Costs: Instead of fine-tuning large language models on domain-specific data (which can be computationally expensive), RAG systems allow for the integration of domain knowledge through the retrieval component.

7. Handling of Long-Context Tasks: RAG systems can effectively handle tasks that require processing or generating long pieces of text by breaking them down into manageable chunks and leveraging relevant information from the knowledge base.

Setting Up Your Environment

Before diving into the code, let's set up the environment necessary for building a RAG system with LangChain.

1. Install Dependencies

To get started, you'll need to install a few Python packages. Open your terminal and run the following command:

pip install langchain openai faiss-cpu python-dotenv langchain-community tiktoken

• LangChain: The core framework we'll be using to build our RAG system.

• OpenAI: To access OpenAI's language models (you can replace this with other supported LLMs if preferred).

• FAISS: A library for efficient similarity search and clustering, which we'll use for document indexing.

• python-dotenv: For managing environment variables, including API keys.

• langchain-community: This library contains a collection of community-contributed utilities, integrations, and modules that extend LangChain's capabilities, enabling easier interaction with a variety of tools and services.

• tiktoken: A library for tokenizing text in a way that's compatible with OpenAI's models, which helps manage token limits and optimize performance when working with large language models.

2. Get API Keys

You'll need API keys for the services you plan to use. For this guide, we'll primarily use OpenAI's API. Follow these steps to obtain and set up your API key:

1. Go to the OpenAI website and sign up for an account if you haven't already.

2. Navigate to the API section and create a new API key.

3. Create a .env file in your project directory and add your API key:

OPENAI_API_KEY=your_api_key_here

4. In your Python script, use the following code to load the environment variables:

from dotenv import load_dotenv 
import os 

load_dotenv() 
openai_api_key = os.getenv("OPENAI_API_KEY")

Step-by-Step Guide to Building a RAG System

Let's walk through the process of building a RAG system using LangChain, step by step.

1. Document Preparation

The first step in creating a RAG system is to prepare the documents that your system will retrieve information from. These documents will serve as the knowledge base for your system.

We will use the text from the following NYTimes article about Climate Change that we manually cleaned and split in subdocuments (TXT files) for the purpose of the explanation.

LangChain supports various document loaders for different file types and sources. Here's an example of how you can load text documents:

from langchain.document_loaders import DirectoryLoader, TextLoader

# Load text files from a directory
loader = DirectoryLoader('./data', glob="**/*.txt", loader_cls=TextLoader)
documents = loader.load()

# Process the documents
from langchain.text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
texts = text_splitter.split_documents(documents)

In this example, we're using the DirectoryLoader to load all .txt files from a data directory. We then use a RecursiveCharacterTextSplitter to split the documents into smaller chunks, which is important for efficient indexing and retrieval.

You can learn more about text chunking in the following issue:

Issue #64 - Text chunking for RAG systems

2. Indexing Documents

Once your documents are loaded and processed, the next step is to index them so that they can be efficiently searched when generating responses. We'll use FAISS (Facebook AI Similarity Search) for this purpose.

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS

# Initialize the embeddings
embeddings = OpenAIEmbeddings()

# Create the vector store
vectorstore = FAISS.from_documents(texts, embeddings)

# Save the vector store
vectorstore.save_local("faiss_index")

In this code snippet, we use OpenAI's embeddings to convert the document chunks into vectors, which are then indexed using FAISS. We also save the index locally for future use.

What is a vector store? Here you have more info:

Issue #55 - Vector Databases and their importance

3. Setting Up the Retrieval Mechanism

Now that we have our documents indexed, we can set up the retrieval mechanism. This will allow our system to find and return the most relevant documents based on a user's query.