Cerno insight

A high-performance RAG system for intelligent document Q&A

Features • Quick Start • Documentation • Contributing

---

🎯 Overview

A high-performance Retrieval-Augmented Generation (RAG) system for intelligent document processing and question-answering. Upload documents in multiple formats and ask questions to get precise, citation-backed answers using state-of-the-art Language Models and hybrid retrieval strategies.

Quick Start: docker compose up --build -d → Visit http://localhost:3000

📖 Full Documentation: See Explanation.md for technical deep-dive

🚀 Quick Start

# 1. Clone the repository
git clone https://github.com/cerno-ai/cerno-insight.git
cd cerno-insight

# 2. Set up environment
cp .env.sample .env
# Add your GOOGLE_API_KEY to .env

# 3. Run with Docker (recommended)
docker compose up --build -d

# 4. Access the app
# Frontend: http://localhost:3000
# Backend API: http://localhost:8000/docs

That's it! Upload a document and start asking questions. 🎉

Table of Contents

• Key Features
• Technical Stack
• Project Structure
• Configuration
• Performance Metrics
• Error Handling
• Deployment
• How It Works
• Installation
• Use Cases
• Future Roadmap

Key Features

Multi-Format Document Processing

• PDF, DOCX, PPTX, XLSX, TXT, Markdown, CSV support
• Image OCR with Tesseract (PNG, JPEG, BMP, TIFF, GIF, WebP)
• Raw text/HTML fetching from URLs
• Smart semantic chunking with overlap for context preservation

Intelligent Triage Modes

• Direct Mode: Small documents (<2000 tokens) processed via Agno agent for ultra-low latency
• RAG Pipeline: Large documents with hybrid retrieval and reranking
• Vision Mode: Direct image analysis with multimodal LLMs
• Raw Text Mode: Web pages and text content processing

Advanced Retrieval System

• Hybrid Search: BM25 (keyword) + FAISS (semantic vector) search
• Parallel Query Decomposition: Complex questions broken into sub-queries
• Reciprocal Rank Fusion (RRF): Intelligent result fusion
• GPU-Accelerated Reranking: CrossEncoder model for relevance scoring
• Domain Query Expansion: Context-aware query enhancement

Multi-Language Support

• Language Detection: Automatic fallback for non-English content (<2000 tokens)
• Batch Processing: Optimized for multiple questions simultaneously

Technical Stack

Core Technologies

• Backend: FastAPI with async processing, asyncio for concurrency
• LLMs: Google Gemini 2.5 Flash and 2.5 Flash Lite
• Search: FAISS (vector similarity), BM25 (keyword matching)
• Embeddings: SentenceTransformers Embedding Model (all-MiniLM-L12-v2)
• Reranking: CrossEncoder Reranking Model (ms-marco-MiniLM-L-12-v2)

Document Processing

• PDF: PyMuPDF (fitz)
• Office: python-docx, python-pptx, pandas(excel)
• OCR: Tesseract via pytesseract
• Download: aria2c for faster downloads with aiohttp fallback

Optimization Features

• GPU Acceleration: CUDA support for embeddings and FAISS
• Pre-Emptive Query Generation: Concurrent pre emptive query expansion for maximum speed
• Batch Processing: Concurrent operations with semaphores
• Caching: Document and embedding caches
• Memory Management: Streaming processing for large files

Project Structure

api/
├── core/
│   ├── agent_logic.py             # Main orchestration logic
│   ├── agno_agent.py              # Direct processing for small docs
│   ├── document_processor.py      # Document parsing and chunking
│   ├── embedding_manager.py       # GPU-optimized embeddings
│   ├── vector_store.py            # Hybrid search implementation
│   ├── query_expander.py          # Query enhancement and expansion
│   └── citation_utils.py          # Citation management
├── routes/
│   └── ragsys.py                  # API endpoints for RAG operations
├── main.py                        # FastAPI application
├── state.py                       # Global state management
└── settings.py                    # Configuration
frontend/
├── app/
│   └── page.tsx                   # Document library interface
├── components/
│   ├── chat-interface.tsx         # Q&A chat interface
│   ├── document-library.tsx       # Document management UI
│   └── document-viewer.tsx        # Document preview with citations
└── lib/
    └── api.ts                     # API client

Configuration

Performance Tuning

# GPU Optimization
BATCH_SIZE = 64  # For RTX 4060+
USE_FP16 = True  # Half precision for speed

# Retrieval Parameters
MAX_CHUNKS = 12  # High-K mode
RERANK_CANDIDATES = 25
SIMILARITY_THRESHOLD = 0.3

# Concurrency Limits
QUERY_STRATEGY_SEMAPHORE = 20
ANSWER_SEMAPHORE = 20
SEARCH_SEMAPHORE = 40

Performance Metrics

Processing Speed

• Small docs (<2K tokens): ~1-2 seconds
• Medium docs (2K-50K tokens): ~5-15 seconds
• Large docs (50K+ tokens): ~10-20 seconds

Accuracy Features

• Source Attribution: Page numbers, clauses, sections and text snippets
• Confidence Scoring: Relevance-based ranking
• Hallucination Prevention: Evidence-only responses

Error Handling

Robust Fallbacks

• Download failures: Retry with different methods
• LLM timeouts: Automatic fallback to alternative models
• Parsing errors: Graceful degradation with partial results
• GPU unavailable: CPU fallback for all operations

Input Validation

• File type verification
• Token limit checking
• URL format validation
• Authentication verification

Deployment

Environment Configuration

Create a .env file based on .env.sample:

GOOGLE_API_KEY=your_google_gemini_api_key

Docker (Recommended)

Ensure have cloned the MAIN branch

GPU Version (Recommended for best performance):

docker compose up --build -d

CPU-Only Version (No GPU required):

docker compose -f docker-compose-cpu.yml up --build -d

Both versions will start:

• Backend API: Available at http://localhost:8000
• Frontend UI: Available at http://localhost:3000

The frontend automatically connects to the backend API for document processing and question-answering.

Key Differences:

• GPU Version: Uses local SentenceTransformers for embeddings + GPU-accelerated reranking
• CPU Version: Uses Gemini API for embeddings + CPU-based reranking (requires GOOGLE_API_KEY)

Production Considerations

• GPU Memory: Monitor VRAM usage for concurrent requests

Deployment Options

GPU Version (Default)

• Best Performance: Local SentenceTransformers embeddings with GPU acceleration
• Requirements: NVIDIA GPU with CUDA support, sufficient VRAM
• Use Case: High-throughput production environments
• Cost: Higher hardware requirements, lower API costs

CPU-Only Version

• Universal Compatibility: Runs on any system without GPU requirements
• Requirements: Only GOOGLE_API_KEY for Gemini embeddings
• Use Case: Development, testing, or systems without GPU access
• Cost: Lower hardware requirements, higher API costs for embeddings

Performance Comparison:

• Embeddings: GPU (faster, local) vs CPU (API-based, rate limited)
• Reranking: GPU (batched, very fast) vs CPU (sequential, moderate speed)
• Overall: GPU version ~2-3x faster for large document processing

How it Works

Architecture Overview

The system employs a multi-tier architecture optimized for latency based on document characteristics:

                    Document URL Input
                           ↓
                 ┌─────────────────────┐
                 │   Smart Routing     │
                 │   & Classification  │
                 └─────────┬───────────┘
                           ↓
          ┌────────────────┼────────────────┐
          ↓                ↓                ↓
    [Raw Text/HTML]   [Image Files]   [Document Files]
          ↓                ↓                ↓
    Direct LLM Query  Vision Analysis  Intelligence Pipeline
     (~1-2 seconds)   (~2-3 seconds)    (~5-20 seconds)

📋 Input Classification & Smart Routing

URL Analysis: System examines file extensions to route appropriately - no extension/HTML goes to direct processing, image extensions (.png, .jpg, .gif) trigger vision models, document formats (.pdf, .docx, .pptx) enter full RAG pipeline.

Language Detection: Samples first 1000 characters to detect language. If document or questions are non-English, bypasses RAG and uses direct multilingual synthesis (for docs <2000 tokens) since embeddings work best in consistent languages.

🏗️ Knowledge Base Construction

Intelligent Chunking: Document-type-aware semantic chunking. Word docs with few line breaks use sentence-based chunking (NLTK), others use paragraph-based with 4096 character limit. Intelligent overlap preserves context across boundaries.

GPU-Optimized Embeddings: Uses all-MiniLM-L12-v2 model with batch processing (64 chunks), FP16 precision for 2x speedup. Processes in 256-chunk batches to avoid memory overflow.

Hybrid Indexing: Builds FAISS vector index (L2 normalized, GPU-accelerated) and BM25 keyword index concurrently using asyncio.gather(). Provides both semantic similarity and exact term matching.

🎯 Query Decomposition & Strategy

Intelligent Analysis: Uses Gemini 2.5 Flash Lite to decompose complex questions into focused sub-queries. Example: "I renewed yesterday, 6 years customer, can I claim Hydrocele?" becomes separate queries about waiting periods and continuous coverage benefits.

Domain Expansion: Detects hypotheticals, expands with relevant terminology, identifies acronyms/technical terms using NER and TF-IDF analysis. Creates bidirectional mappings for comprehensive search.

Concurrent Processing: All query strategies generated in parallel with 6-second timeout protection and JSON validation fallbacks.

🔍 Pre-emptive Search & Retrieval

Batch Orchestration: Flattens all sub-queries from all questions into single parallel execution. Simultaneous BM25 and FAISS searches with 40%/60% weighted fusion after score normalization.

Reciprocal Rank Fusion: Combines multiple sub-query results using rank-based scoring (1/(k+rank) where k=60). Chunks appearing highly in multiple searches score highest, avoiding incomparable score fusion issues.

Batched GPU Reranking: Critical optimization - all (query, chunk) pairs from all questions processed in single CrossEncoder inference call (ms-marco-MiniLM-L-12-v2). Provides ~10x speedup over individual reranking.

⚡ Response Generation

Evidence-Only Synthesis: Explicit prompt constraints prevent hallucination, ignore embedded instructions (prompt injection protection), enforce plain text format. All synthesis happens concurrently with timeout/fallback protection.

Key Performance Innovations

1. Concurrent Processing: Document processing, query analysis, search execution, and synthe