Project RAG - MCP Server for Code Understanding

A Rust-based Model Context Protocol (MCP) server that provides AI assistants with powerful RAG (Retrieval-Augmented Generation) capabilities for understanding massive codebases.

Overview

This MCP server enables AI assistants to efficiently search and understand large projects by:

• Creating semantic embeddings of code files
• Storing them in a local vector database
• Providing fast semantic search capabilities
• Supporting incremental updates for efficiency

Features

• Local-First: All processing happens locally using fastembed-rs (no API keys required)
• Hybrid Search: Combines vector similarity with BM25 keyword matching using Reciprocal Rank Fusion (RRF) for optimal results
• AST-Based Chunking: Uses Tree-sitter to extract semantic units (functions, classes, methods) for 12 languages
• Comprehensive File Support: Indexes 40+ file types including code, documentation (with PDF→Markdown conversion), and configuration files
• Git History Search: Search commit history with smart on-demand indexing (default: 10 commits, only indexes deeper as needed)
• Multi-Project Support: Index and query multiple codebases simultaneously with project filtering
• Smart Indexing: Automatically performs full indexing for new codebases or incremental updates for previously indexed ones
• Cross-Process Locking: Filesystem-based locks prevent multiple processes (e.g., multiple Claude Code sessions) from indexing the same codebase simultaneously
• Concurrent Access Protection: Safe lock management prevents index corruption when multiple agents try to index simultaneously
• Stable Embedded Database: LanceDB vector database (default, no external dependencies) with optional Qdrant support
• Language Detection: Automatic detection of 40+ file types (programming languages, documentation formats, and config files)
• Advanced Filtering: Search by file type, language, or path patterns
• Respects .gitignore: Automatically excludes ignored files during indexing
• Code Navigation: Find definitions, references, and call graphs (lightweight LSP-like features)
• Adaptive Search Thresholds: Automatically lowers similarity threshold when no results found (0.7 → 0.6 → 0.5 → 0.4 → 0.3)
• Slash Commands: 9 convenient slash commands via MCP Prompts

MCP Slash Commands

The server provides 9 slash commands for quick access in Claude Code:

1. /project:index - Index a codebase directory (automatically performs full or incremental)
2. /project:query - Search the indexed codebase
3. /project:stats - Get index statistics
4. /project:clear - Clear all indexed data
5. /project:search - Advanced search with filters
6. /project:git-search - Search git commit history with on-demand indexing
7. /project:definition - Find where a symbol is defined (LSP-like)
8. /project:references - Find all references to a symbol
9. /project:callgraph - Get call graph for a function (callers/callees)

See slash-commands.md for detailed usage.

Supported File Types

Project RAG automatically indexes and searches 40+ file types across three categories:

Programming Languages (24 languages)

Supports AST-based semantic chunking for these languages:

• Rust (.rs)
• Python (.py)
• JavaScript (.js, .mjs, .cjs), TypeScript (.ts), JSX (.jsx), TSX (.tsx)
• Go (.go)
• Java (.java)
• C (.c), C++ (.cpp, .cc, .cxx), C/C++ Headers (.h, .hpp)
• C# (.cs)
• Swift (.swift)
• Kotlin (.kt, .kts)
• Scala (.scala)
• Ruby (.rb)
• PHP (.php)
• Shell (.sh, .bash)
• SQL (.sql)
• HTML (.html, .htm)
• CSS (.css), SCSS (.scss, .sass)

Documentation Formats (8 formats)

With special handling for rich content:

• Markdown (.md, .markdown)
• PDF (.pdf) - Automatically converted to Markdown with table preservation
• reStructuredText (.rst)
• AsciiDoc (.adoc, .asciidoc)
• Org Mode (.org)
• Plain Text (.txt)
• Log Files (.log)

PDF Conversion Features:

• Extracts text content using pdf-extract library
• Converts to Markdown format automatically
• Preserves table structures (detects tab/space-separated columns)
• Detects and formats headings (ALL CAPS lines and section markers)
• Handles multi-column layouts intelligently
• Chunks like any other text file (50 lines per chunk by default)

Configuration Files (8 formats)

For complete project understanding:

• JSON (.json)
• YAML (.yaml, .yml)
• TOML (.toml)
• XML (.xml)
• INI (.ini)
• Config files (.conf, .config, .cfg)
• Properties (.properties)
• Environment (.env)

Example Use Cases

# Index documentation PDFs in your project
query_codebase("API authentication flow")  # Finds content in .pdf, .md, .rst files

# Search configuration files
query_codebase("database connection string")  # Finds .yaml, .toml, .env, .conf files

# Find code implementations
search_by_filters(query="JWT validation", file_extensions=["rs", "go"])

MCP Tools

The server provides 9 tools that can be used directly:

1. index_codebase - Smartly index a codebase directory

   • Automatically performs full indexing for new codebases
   • Automatically performs incremental updates for previously indexed codebases
   • Respects .gitignore and exclude patterns
   • Returns mode information (full or incremental)

2. query_codebase - Hybrid semantic + keyword search across the indexed code

   • Combines vector similarity with BM25 keyword matching (enabled by default)
   • Returns relevant code chunks with both vector and keyword scores
   • Configurable result limit and score threshold
   • Optional project filtering for multi-project setups

3. get_statistics - Get statistics about the indexed codebase

   • File counts, chunk counts, embedding counts
   • Language breakdown

4. clear_index - Clear all indexed data

   • Deletes the entire vector database collection
   • Prepares for fresh indexing

5. search_by_filters - Advanced hybrid search with filters

   • Always uses hybrid search for best results
   • Filter by file extensions (e.g., ["rs", "toml"])
   • Filter by programming languages
   • Filter by path patterns
   • Optional project filtering

6. search_git_history - Search git commit history using semantic search

   • Automatically indexes commits on-demand (default: 10 commits, configurable)
   • Searches commit messages, diffs, author info, and changed files
   • Smart caching: only indexes new commits as needed
   • Regex filtering by author name/email and file paths
   • Date range filtering (ISO 8601 or Unix timestamp)
   • Branch selection support

7. find_definition - Find where a symbol is defined (LSP-like)

   • Specify file path, line number, and column
   • Returns definition location with symbol metadata
   • Uses hybrid approach: high-precision stack-graphs (Python, TypeScript, Java, Ruby) or AST-based RepoMap fallback
   • Reports precision level of results

8. find_references - Find all references to a symbol

   • Specify file path, line number, and column
   • Returns all locations where the symbol is used

Prerequisites

• Rust: 1.88+ with Rust 2024 edition support
• protobuf-compiler: Required for building (install via sudo apt-get install protobuf-compiler on Ubuntu/Debian)

Vector Database Options

LanceDB (Default - Embedded, Stable)

No additional setup needed! LanceDB is an embedded vector database that runs directly in the application. It stores data in ./.lancedb directory by default.

Why LanceDB is the default:

• Embedded - No external dependencies or servers required
• Stable - Production-proven with ACID transactions
• Feature-rich - Full SQL-like filtering capabilities
• Hybrid search built-in - Tantivy BM25 + LanceDB vector with Reciprocal Rank Fusion
• Columnar storage - Efficient for large datasets with Apache Arrow
• Zero-copy - Memory-mapped files for fast queries

Qdrant (Optional - Server-Based)

To use Qdrant instead of LanceDB, build with the qdrant-backend feature:

cargo build --release --no-default-features --features qdrant-backend

Then start a Qdrant instance:

Using Docker (Recommended):

docker run -p 6333:6333 -p 6334:6334 \
    -v $(pwd)/qdrant_data:/qdrant/storage \
    qdrant/qdrant

Using Docker Compose:

version: '3.8'
services:
  qdrant:
    image: qdrant/qdrant
    ports:
      - "6333:6333"
      - "6334:6334"
    volumes:
      - ./qdrant_data:/qdrant/storage

Or download standalone: https://qdrant.tech/documentation/guides/installation/

Installation

# Navigate to the project
cd project-rag

# Install protobuf compiler (Ubuntu/Debian)
sudo apt-get install protobuf-compiler

# Build the release binary (with default LanceDB backend - stable and embedded!)
cargo build --release

# Or build with Qdrant backend (requires external server)
cargo build --release --no-default-features --features qdrant-backend

# The binary will be at target/release/project-rag

Usage

Running as MCP Server

The server communicates over stdio following the MCP protocol:

./target/release/project-rag

Configuring in Claude Code

Add the MCP server to Claude Code using the CLI:

# Navigate to the project directory first
cd /path/to/project-rag

# Add the MCP server to Claude Code
claude mcp add project --command "$(pwd)/target/release/project-rag"

# Or with logging enabled
claude mcp add project --command "$(pwd)/target/release/project-rag" --env RUST_LOG=info

After adding, restart Claude Code to load the server. The slash commands (/project:index, /project:query, etc.) will be available immediately.

Configuring in Claude Desktop

Add to your Claude Desktop config:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Linux: ~/.config/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "project-rag": {
      "command": "/absolute/path/to/project-rag/target/release/project-rag",
      "env": {
        "RUST_LOG": "info"
      }
    }
  }
}

Note: Claude Code and Claude Desktop are different products with different configuration methods.

Example Tool Usage

Index a codebase:

{
  "path": "/path/to/your/project",
  "include_patterns": ["**/*.rs", "**/*.toml"],
  "exclude_patterns": ["**/target/**", "**/node_modules/**"],
  "max_file_size": 1048576
}

Query the codebase:

{
  "query": "How does authentication work?",
  "limit": 10,
  "min_score": 0.7
}

Advanced filtered search:

{
  "query": "da