Graph-RAG-Engine

An explainable Graph + Vector Retrieval system that indexes Markdown documents, builds a lightweight knowledge graph, retrieves relevant chunks with FAISS, expands results through graph concepts, and returns source-grounded answers with citations, reasoning paths, and retrieval traces.

Important: This project is an LLM-ready retrieval system, not a fully autonomous knowledge engine.

By default, it uses extractive answer mode, meaning it answers from retrieved passages instead of generating unsupported claims. Optional LLM mode can be enabled with an API key for grounded answer generation.

---

Table of Contents

• Project Overview
• What This Project Does
• What This Project Does Not Do
• Key Features
• System Workflow
• Project Structure
• Installation
• Building the Index
• Running the Backend
• Running the Streamlit App
• Answer Modes
• API Usage
• Retrieval Trace
• Retrieval Evaluation
• Testing and CI
• Code Quality
• Limitations
• Security Notes
• Responsible Use
• Future Improvements
• Tech Stack
• Author
• License

---

Project Overview

Many RAG projects behave like black boxes: a question goes in, an answer comes out, and the user cannot easily see why the system selected certain sources.

Graph-RAG-Engine takes a more transparent approach. It combines dense vector search with a lightweight graph layer so answers can include retrieved passages, citations, related concepts, graph paths, and score-level retrieval traces.

The goal is to demonstrate how a RAG workflow can be turned into an explainable retrieval system, not just a chatbot wrapper around an embedding index.

This project focuses on:

• document ingestion from local Markdown files
• chunking and lightweight concept extraction
• dense vector search with FAISS
• graph-enhanced retrieval with NetworkX
• hybrid reranking using similarity, concept overlap, and PageRank
• extractive answers by default
• optional LLM-generated answers when credentials are provided
• golden-query retrieval evaluation
• FastAPI and Streamlit interfaces
• tests and CI for safer iteration

---

What This Project Does

This project can:

• Read Markdown documents from data/docs/
• Split documents into retrievable text chunks
• Extract lightweight concepts from each chunk
• Create sentence embeddings with Sentence Transformers
• Build a FAISS vector index
• Build a NetworkX graph connecting documents, chunks, and concepts
• Store repository-relative source paths inside index artifacts
• Retrieve relevant chunks for a user question
• Expand retrieval through graph concepts and document relationships
• Rerank results using embedding similarity, normalized concept overlap, and document PageRank
• Return extractive answers with citations
• Return graph-based reasoning paths
• Return retrieval traces with score components
• Recommend related documents
• Run retrieval evaluation with golden queries
• Run unit tests and CI checks
• Optionally generate grounded LLM answers from retrieved context
• Use Streamlit to ask questions in extractive or optional LLM mode

---

What This Project Does Not Do

This project does not:

• Guarantee that every retrieved or generated answer is correct
• Replace human review for important research or business decisions
• Verify facts against the live web
• Use a production graph database by default
• Provide large-scale observability or monitoring
• Make LLM access mandatory
• Claim that graph expansion always improves retrieval quality
• Protect against unsafe index artifacts from untrusted sources
• Provide production-grade access control, audit logging, or permissioning

A production RAG platform would need stronger retrieval evaluation, source governance, monitoring, feedback loops, access control, observability, data-security review, and deployment hardening.

---

Key Features

• FAISS vector search over normalized sentence embeddings
• SentenceTransformer embeddings using all-MiniLM-L6-v2
• NetworkX knowledge graph connecting documents, chunks, and concepts
• Graph expansion through shared concepts
• Hybrid reranking using vector similarity, concept overlap, and PageRank
• Normalized concept-overlap scoring for more balanced reranking
• Retrieval traces showing score components and matched concepts
• Extractive answer mode as the default behavior
• Optional LLM answer mode with fallback to extractive mode
• Streamlit answer-mode selector for extractive or LLM mode
• Repository-relative index paths instead of local machine paths
• Expanded demo corpus with multiple RAG engineering topics
• Golden-query retrieval benchmark
• Retrieval metrics: Hit@K, Precision@K, Recall@K, MRR, and nDCG@K
• FastAPI backend
• Streamlit frontend
• Unit tests and GitHub Actions CI
• CI retrieval evaluation smoke workflow

---

System Workflow

Markdown documents
        ↓
Chunking + concept extraction
        ↓
Sentence embeddings ───────────────→ FAISS vector index
        ↓
Docs / chunks / concepts ──────────→ NetworkX graph
        ↓
User question
        ↓
Vector search
        ↓
Graph expansion
        ↓
Hybrid reranking
        ↓
Extractive answer or optional LLM answer
        ↓
Citations + graph paths + retrieval traces

---

Project Structure

Graph-RAG-Engine/
│
├── .github/
│   └── workflows/
│       └── ci.yml
│
├── backend/
│   ├── __init__.py
│   ├── api.py
│   ├── llm.py
│   ├── rag.py
│   └── retriever.py
│
├── data/
│   ├── docs/
│   │   ├── api_contracts.md
│   │   ├── chunking_strategies.md
│   │   ├── citation_grounding.md
│   │   ├── graph_rag_overview.md
│   │   ├── hybrid_retrieval.md
│   │   ├── retrieval_evaluation.md
│   │   └── ...
│   └── index/                  # generated by ingestion (gitignored)
│       ├── chunks.json
│       ├── docs.json
│       ├── faiss.index
│       ├── graph.json
│       └── vectors.npy
│
├── env/
│   └── requirements.txt
│
├── evaluation/
│   ├── __init__.py
│   ├── evaluate_retrieval.py
│   ├── baseline.json
│   ├── golden_queries.json
│   ├── metrics.py
│   └── results/              # generated by evaluation (gitignored)
│       └── retrieval_eval.json
│
├── graph/
│   ├── __init__.py
│   └── graph_store.py
│
├── ingest/
│   ├── __init__.py
│   ├── ingest_docs.py
│   └── split.py
│
├── tests/
│   ├── test_evaluation_baseline.py
│   ├── test_graph_store.py
│   ├── test_llm.py
│   ├── test_project_integrity.py
│   ├── test_retrieval_evaluation_dataset.py
│   ├── test_retrieval_evaluation_smoke.py
│   ├── test_retrieval_metrics.py
│   ├── test_retrieval_trace.py
│   ├── test_retriever_api_contract.py
│   ├── test_split.py
│   └── test_ui_contracts.py
│
├── ui/
│   ├── __init__.py
│   └── app.py
│
├── pipeline.py
├── run_backend.py
├── Makefile
├── pyproject.toml
├── .gitignore
├── .pre-commit-config.yaml
├── README.md
└── LICENSE

---

Installation

1. Clone the Repository

git clone https://github.com/AmirhosseinHonardoust/Graph-RAG-Engine.git
cd Graph-RAG-Engine

2. Create a Virtual Environment

On Windows CMD:

python -m venv .venv
.venv\Scripts\activate

On Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1

On macOS/Linux:

python -m venv .venv
source .venv/bin/activate

3. Install the Package

Install in editable mode. This pulls in all dependencies and registers the graph-rag-ingest, graph-rag-eval, and graph-rag-serve commands:

python -m pip install --upgrade pip
pip install -e .

For development tools (Ruff, Black, mypy, pre-commit):

pip install -e ".[dev]"

The pinned dependency list is also available at env/requirements.txt. The first embedding run may download the SentenceTransformer model, so an internet connection is required for initial setup.

---

Quick Start

Once installed, build the index and validate retrieval quality in one command:

graph-rag-pipeline

This ingests the documents, builds the index, runs the golden-query evaluation, and fails if any metric drops below evaluation/baseline.json. Common tasks are also wrapped in a Makefile:

make dev       # install with dev tools and git hooks
make pipeline  # ingest + evaluate
make serve     # run the FastAPI backend
make lint      # ruff + black + mypy
make test      # run the unit tests

On platforms without make, use the console commands directly (graph-rag-ingest, graph-rag-eval, graph-rag-serve, graph-rag-pipeline).

---

Building the Index

Run ingestion (from anywhere once installed):

graph-rag-ingest

This will:

• read Markdown files from data/docs/
• sort document loading deterministically
• split each document into chunks
• extract lightweight concepts
• create sentence embeddings
• save a FAISS index
• build and save the graph
• store repo-relative source paths
• write retrieval artifacts to data/index/

Generated index artifacts are saved in:

data/index/

Security note: Index artifacts (chunks.json, docs.json, graph.json) are stored as JSON, so loading them cannot execute code. The FAISS index and vectors.npy are binary numeric files. Only the embedding vectors and FAISS index require trusting the producer; none of the artifacts are pickled.

---

Running the Backend

Start the FastAPI backend:

graph-rag-serve

This serves backend.api:app on 127.0.0.1:8000 with reload enabled. You can also run uvicorn directly: uvicorn backend.api:app --reload --port 8000.

Open the API docs:

http://localhost:8000/docs

Health check:

curl http://localhost:8000/health

Expected response:

{
  "ok": true
}

---

Running the Streamlit App

In another terminal, run:

streamlit run ui/app.py

The UI opens at:

http://localhost:8501

By default, the UI calls the backend at:

http://localhost:8000

You can override this with:

export GRAPH_RAG_API_URL="http://localhost:8000"

On Windows PowerShell:

$env:GRAPH_RAG_API_URL="http://localhost:8000"

The Streamlit app supports both answer modes:

• extractive
• llm

If LLM mode is selected without a valid API key, the backend falls back to extractive mode and returns an LLM error message.

---

Answer Modes

The project supports two answer modes.

1. Extractive Mode

Extractive mode is the default.

It uses retrieved passages directly and avoids generating unsupported claims.

{
  "question": "What is FAISS?",
  "mode": "extractive"
}

This mode is useful for:

• local demos
• transparent retrieval debugging
• source-first answer display
• running the system without API keys

2. Optional LLM Mode

LLM mode sends retrieved passages to an OpenAI-compatible chat-comple