🐳 Docker Services

🚀 Quick Start

Prerequisites

• Docker Desktop (with GPU support for vLLM)
• NVIDIA GPU (Recommended, 8GB+ VRAM)
• Git

1. Clone & Configure

git clone https://github.com/yourusername/Advanced-RAG.git
cd Advanced-RAG

# Copy environment template
cp .env.example .env
# Edit .env with your API keys (OpenRouter, etc.)

2. Start All Services

docker compose up -d

3. Access the UI

• Chat UI: http://localhost:3000 (Open WebUI)
• Langfuse Dashboard: http://localhost:3001
• API Docs: http://localhost:5002/docs

Default Langfuse Credentials

• Email: admin@rag.local
• Password: ragadmin123

⚙️ Environment Variables

See .env.example for the full list.

📁 Project Structure

Advanced-RAG/
├── src/
│   ├── main.py                 # FastAPI app & endpoints
│   ├── config.py               # Model & provider configuration
│   ├── ingestion/              # Document processing pipeline
│   │   ├── router.py           # Ingestion orchestrator
│   │   ├── docling_parser.py   # PDF/DOCX parser
│   │   ├── deepseek_ocr.py     # OCR for images (optional)
│   │   ├── metadata.py         # LLM-based metadata extraction
│   │   └── chunking.py         # Hierarchical chunking
│   ├── retrieval/              # Search & retrieval
│   │   ├── engine.py           # Query rewriting, HyDE
│   │   ├── qdrant_client.py    # Vector DB operations
│   │   └── reranker.py         # Cross-encoder reranking
│   ├── generation/             # Response generation
│   │   ├── agents.py           # Multi-agent orchestration
│   │   ├── router.py           # Model routing (local/cloud)
│   │   └── semantic_cache.py   # Query caching
│   └── observability/          # Monitoring
│       └── config.py           # Langfuse setup
├── docker-compose.yml          # All services
├── Dockerfile                  # RAG backend image
├── pyproject.toml              # Python dependencies
└── requirements.txt            # Pip dependencies

🔄 How It Works

Ingestion Pipeline (Upload a Document)

1. File Detection → Route to Docling (PDF/DOCX) or OCR (images)
2. Text Extraction → Preserve structure (tables, headers)
3. Metadata Enrichment → LLM extracts department, date, summary
4. Hierarchical Chunking → Parent (1024 tok) + Child (256 tok) chunks
5. Vector Upsert → Dense + Sparse embeddings to Qdrant

Query Pipeline (Ask a Question)

1. Semantic Cache Check → Return cached answer if similarity > 0.95
2. Query Rewriting → Expand ambiguous queries
3. Hybrid Search → Dense (semantic) + Sparse (keyword) in Qdrant
4. Re-ranking → Cross-encoder scores top 50 → keep top 5
5. Model Routing → Simple → Local vLLM, Complex → OpenRouter
6. Response Generation → Stream answer with context
7. Cache Update → Store Q&A for future queries

📊 Observability (Langfuse)

Access the Langfuse dashboard at http://localhost:3001

Features

• Traces - Full execution path for each request
• Sessions - Group traces by conversation (chat thread)
• Users - Track usage per user
• Costs - Token usage and cost breakdown
• Scores - User feedback (thumbs up/down)

Session Tracking

Open WebUI automatically sends session headers when ENABLE_OPENWEBUI_USER_HEADERS=true:

• X-OpenWebUI-Chat-Id → Groups all messages in a conversation
• X-OpenWebUI-User-Id → Links traces to users

🛠️ Development

Running Locally (without Docker)

# Install dependencies
pip install poetry
poetry install

# Start backend
poetry run uvicorn src.main:app --reload --port 8000

Poetry Convenience Scripts

After running poetry install, you can use these commands to manage Docker services:

Example workflow:

# Start everything at once
poetry run start-all

# Or start services individually
poetry run trace-up      # Start observability first
poetry run llm-up        # Start local LLM
poetry run app-up        # Start the RAG application

# Check what's running
poetry run status

# Stop everything
poetry run stop-all

Adding New Models

Edit src/config.py to add new models:

ModelConfig(
    id="your-model-id",
    name="Display Name",
    provider=Provider.OPENROUTER,  # or Provider.VLLM
    context_window=8192,
)

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing)
5. Open a Pull Request

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• LlamaIndex - RAG framework
• vLLM - Fast LLM inference
• Qdrant - Vector database
• Langfuse - LLM observability
• Open WebUI - Chat interface
• Docling - Document parsing