Loading repository data…
dbabbs / repository
This repository is a sample application and guided walkthrough for a semantic search question-and-answer style interaction with custom user-uploaded documents. Typescript, NextJS, OpenAI, Langchain, Pinecone
This repository is a sample application and guided walkthrough for a semantic search question-and-answer style interaction with custom user-uploaded documents.
Users can upload a custom plain text document (.txt file) and ask the AI questions about the content.
This project is tailored for web developers who are looking to learn more about integrating LLMs and vector databases into their projects.
This guide is also available as a Medium article
Why create a custom semantic search application?
While ChatGPT can handle questions about documents, it does have certain limitations:
By Dylan Babbs, built with guidance from Nader Dabit's AI Semantic Search YouTube video.
.env file with the following environment variablesOPENAI_API_KEY=XXXXXXXXXXXXXX
PINECONE_API_KEY=XXXXXXXXXXXXXX
PINECONE_ENVIRONMENT=XXXXXXXXXXXXXX
PINECONE_INDEX_NAME=XXXXXXXXXXXXXX
yarn
yarn dev
This is a high-level guide and walkthrough to creating the full-stack application. The full application code is available for reference in this repository.
Head to Pinecone, sign up for an account, and create an index. Pinecone is a fully managed and hosted vector database that runs in the cloud. With Pinecone, there's no need to setup or configure a local database.
The name that you give the database will be the same value you use as an environment variable in the .env file (PINECONE_INDEX_NAME).
For the Dimensions field, enter 1536. Why 1536 dimensions?
In the LangChain framework, when creating a new Pinecone index, the default dimension is set to 1536 to match the OpenAI embedding model text-embedding-ada-002 which uses 1536 dimensions.
For the metric field, select cosine.
Then, click Create Index. The new index will be created in a few minutes. You'll need to have the vector database created before we move onto the next steps.
Don't forget to also update the environment variables with the Pinecone details (PINECONE_API_KEY, PINECONE_ENVIRONMENT, PINECONE_INDEX_NAME).
For the application's UI, we'll be using NextJS (a React framework) and Uber's Baseweb for the UI components.
When cloning the repo, you'll see we are using the following file structure. (The repo has additional files, but this diagram only shows those that are relevant.)
src/
├─ components/
│ ├─ chat-view.tsx
│ ├─ document-upload-modal.tsx
| ├─ header.tsx
├─ pages/
│ ├─ api/
│ | ├─ chat.ts
│ | ├─ upload-document.ts
│ ├─ index.ts
├─ ai-utils.ts
The files within the src/components/ directory are UI components:
The file index.tsx within the src/pages/ directory is the main front-end file, which will be rendering the application. Here, we'll be placing most of the application's front-end logic and data fetching.
The files within the src/pages/api/ directory are the NextJS API routes. These files are the functions which will run on the backend, such as the query to the LLM and the logic for uploading files and upserting them to the Pinecone database.
Document upload modal
In order to search a custom document, we'll need to make it easy for the user to upload a document to the application.
Inside on the src/pages/index.tsx file, we've added the following DocumentUploadModal component:
// src/components/document-view.tsx
<DocumentUploadModal
isOpen={uploadModalIsOpen}
setIsOpen={setUploadModalIsOpen}
setActiveDocument={setActiveDocument}
/>
This code for this component lives inside the src/components/document-upload-modal.tsx file. This front-end component will:
Sending the file on the front-end and handling it file on the backend
Once the user has selected the file and clicks the Upload button, we'll send it to the backend with a fetch request:
// src/components/document-upload-modal.tsx
// `handleSubmit` is called when the user clicks the upload button.
const handleSubmit = useCallback(async () => {
setIsLoading(true);
//Read contents of file
const text = await readText(document);
//Send file to backend
const response = await fetch('/api/upload-document', {
method: 'POST',
body: JSON.stringify({
text,
name: documentName,
}),
});
const json = await response.json();
}, [document, documentName, setActiveDocument]);
On the backend, we'll receive this request in the file src/pages/api/upload-document.ts. This API route will:
Document object with the document's content (pageContent) and the name of the document as metadata// src/pages/api/upload-document.ts
import {insertDocument} from '../../ai-util';
async function handler(req: NextApiRequest, res: NextApiResponse) {
const {name, text} = JSON.parse(req.body);
const doc = new Document({
pageContent: text,
metadata: {documentName: name},
});
const client = new Pinecone();
const index = client.Index(process.env.PINECONE_INDEX_NAME || '');
await insertDocument(index, doc);
res.json({success: true});
}
export default handler;
Let's dive into the insertDocument function that's inside the src/ai-utils.ts file.
The first thing we'll do is start splitting the text content into chunks. Chunking is the process of splitting large pieces of text into smaller groups. Through chunking, we can ensure we are splitting the larger content into semantically relevant smaller groups. This will help the LLM better understand the input data.
We can use the popular LangChain RecursiveCharacterTextSplitter method to create the chunks.
// src/ai-utils.ts
const textSplitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
});
const chunks = await textSplitter.createDocuments([text]);
There are a few strategies for choosing chunk size. We are using a larger chunk size number (1000) to retain additional context in each chunk. On the other hand, choosing a smaller chunk size number will capture granular semantic information.
Next, we'll create the OpenAI vector embeddings for the document. Embeddings measure the relatedness of text strings. Embeddings can be used for search (our case). The distance between two vectors measures their relatedness.
Langchain includes a helper function for us to work with the OpenAI Embeddings API. We'll pass the chunks to the function:
// src/ai-utils.ts
const embeddingsArrays = await new OpenAIEmbeddings().embedDocuments(
chunks.map((chunk) => chunk.pageContent.replace(/\n/g, ' ')),
);
One we have access to the embeddings, we can begin to upsert the vectors into the Pinecone database:
// src/ai-utils.ts
const batch = [];
for (let i = 0; i < chunks.length; i++) {
const chunk = chunks[i];
const vector = {
id: `${documentName}_${i}`,
values: embeddingsArrays[i],
metadata: {
...chunk.metadata,
loc: JSON.stringify(chunk.metadata.loc),
pageContent: chunk.pageContent,
documentName: documentName,
},
};
batch.push(vector);
}
await index.upsert(batch);
The vectors have now been inserted into Pinecone! You can verify they have been uploaded successfully by checking out the index details in the Pinecone web application. You'll be able to see some items under the "Browser" tab:
Now, let's configure the logic of handling the question and answering within the custom document. To facilitate this, we're going to create a new API endpoint to handle this request: src/pages/api/chat.ts
// src/pages/api/chat.ts
async function handler(req: NextApiRequest, res: NextApiResponse<LLMResponse>) {
const {documentName, question} = JSON.parse(req.body);
const client = new Pinecone();
const index = client.Index(process.env.PINECONE_INDEX_NAME || '');
//Query Pinecone client
const queryResponse = await queryPinecone(index, question, documentName);
if (queryResponse.matches.length > 0) {
//Query LLM
const result = await queryLLM(queryResponse, question);
res.json(result);
} else {
res.json({
result: "Sorry, I don't know the answer to that question.",
sources: [],
});
}
}
export default handler;
When we receive the request, we'll want to:
Selected from shared topics, language and repository description—not editorial ratings.
animesh-sh /
This is the repository for my personal blog website built using Next.js, Tailwind CSS, and Typescript. The content of the blog is managed using Notion as a CMS. The website primarily focuses on topics related to development and data science.