Loading repository data…
Loading repository data…
AdilShamim8 / repository
A comprehensive, end-to-end learning path for Retrieval-Augmented Generation — from core concepts and document processing to agentic RAG systems, evaluation, and production deployment.
A transparent discovery signal based on current public GitHub metadata.
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
A comprehensive, end-to-end learning path for Retrieval-Augmented Generation — from core concepts and document processing to agentic RAG systems, evaluation, and production deployment.
This roadmap has been updated for June 2026 to reflect the latest advancements in the RAG ecosystem, vector databases, embedding models, and agentic frameworks.
text-embedding-3-small and text-embedding-3-large as default embedding modelsThis repository provides a structured, step-by-step curriculum for mastering Retrieval-Augmented Generation (RAG) — one of the most impactful patterns in production AI engineering today.
RAG addresses core limitations of large language models: hallucination, knowledge cutoffs, and the inability to cite sources. This roadmap takes you from understanding those problems to building production-grade systems that solve them.
| Milestone | Description |
|---|---|
| 📄 Document Pipelines | Load, parse, chunk, and enrich documents from PDFs, URLs, CSVs, and more |
| 🧠 Embedding Workflows | Compare and implement embedding models for semantic search |
| 🗄️ Vector Store CRUD | Build full create/read/update/delete vector store applications |
| 🔎 Retrieval Systems | Similarity search, MMR, hybrid search, contextual compression, self-query |
| 🤖 Advanced RAG Patterns | RAG Fusion, HyDE, CRAG, Self-RAG, Graph RAG |
| 🕸️ Agentic RAG | LangGraph-based agents with retrieval tools, ReAct loops, and self-correction |
| 📊 RAG Evaluation | RAGAS-based evaluation pipelines with all key metrics |
| 🚀 Production Deployment | Monitoring with LangSmith, caching, cost optimization, security |
RAG has moved from an experimental technique to the default architecture for enterprise AI systems that require accurate, traceable, and domain-specific responses. Key forces driving this:
| Generation | Approach | Key Limitation |
|---|---|---|
| Naive RAG | Chunk → Embed → Retrieve → Generate | Poor precision, no diversity control |
| Advanced RAG | Query expansion, re-ranking, compression | Higher latency, more complexity |
| Modular RAG | Swappable retrieval components | Integration overhead |
| Agentic RAG (2025–2026) | Agents that decide when/how to retrieve | Reasoning overhead, requires orchestration |
| Graph RAG (2025–2026) | Knowledge graphs + vector retrieval | Graph construction cost |
| Category | Tools |
|---|---|
| Orchestration | LangChain v1.2.x, LlamaIndex, Haystack |
| Agentic Frameworks | LangGraph v1.1.10, AutoGen, CrewAI |
| Vector Stores | Chroma, FAISS, Qdrant, Pinecone, Milvus, Weaviate |
| Embedding Models | OpenAI text-embedding-3, Ollama/Gemma, Cohere, HuggingFace |
| Re-ranking | Cohere Rerank, FlashRank, cross-encoders |
| Evaluation | RAGAS, TruLens, DeepEval |
| Observability | LangSmith, Arize, Weights & Biases |
| Graph Databases | Neo4j, Amazon Neptune |
Before starting this course, you should be comfortable with:
RAG Roadmap — 10 Modules
│
├── Module 1 — RAG Fundamentals and Architecture [Week 1]
├── Module 2 — Document Processing and Chunking [Week 2]
├── Module 3 — Embeddings and Vector Representations [Week 3]
├── Module 4 — Vector Stores [Week 4]
├── Module 5 — Basic Retrieval Techniques [Week 5]
├── Module 6 — Advanced Retrieval Techniques [Week 6]
├── Module 7 — Advanced RAG Patterns [Week 7–8]
├── Module 8 — Agentic RAG with LangGraph [Week 9]
├── Module 9 — Evaluating RAG with RAGAS [Week 10]
└── Module 10 — Capstone Project with Deployment [Week 11–12]
Duration: ~1 week | Goal: Understand what RAG is, why it exists, and how data flows through the system
Knowledge Base → Retriever → Generator
↑ ↓
[Documents] [Final Response + Sources]
User Query
│
▼
[Query Embedding]
│
▼
[Vector Store Similarity Search]
│
▼
[Top-K Relevant Chunks]
│
▼
[Prompt Assembly: Query + Context]
│
▼
[LLM Generation]
│
▼
[Final Answer with Citations]
| Dimension | Prompt Engineering | RAG | Fine-Tuning |
|---|---|---|---|
| Cost | Very Low | Low–Medium | High |
| Latency | Lowest | Medium | Low |
| Knowledge Freshness | Static | Dynamic ✅ | Static |
| Domain Accuracy | Limited | High ✅ | Very High |
| Source Attribution | ❌ | ✅ | ❌ |
| Maintenance | Easy | Medium | High |
| Data Required | None | Documents | Labeled examples |
Duration: ~1 week | Goal: Load and prepare any document type for embedding and retrieval
| Loader | Source Type | Notes |
|---|---|---|
PyPDFLoader | Fast, page-level splits | |
PDFMinerLoader | Better text extraction for complex layouts | |
PDFPlumberLoader | Best for tables and structured PDFs | |
UnstructuredLoader | PDF, DOCX, PPTX, HTML | Multi-format powerhouse |
WebBaseLoader | Web pages | HTML parsing |
RecursiveUrlLoader | Entire sites | Crawls links recursively |
CSVLoader | CSV | Row-per-document loading |
JSONLoader | JSON | jq-based field selection |
TextLoader | .txt | Simplest loader |
🧪 Hands-on: Load documents from 5 different sources (PDF, URL, CSV, JSON, plain text)
["\n\n", "\n", " ", ""]chunk_size, chunk_overlapfrom langchain_text_splitters import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=512,
chunk_overlap=50,
separators=["\n\n", "\n", " ", ""]
)
chunks = splitter.split_documents(docs)
| Splitter | Best For |
|---|---|
CharacterTextSplitter | Simple single-separator splits |
TokenTextSplitter | Token-budget-aware splitting |
SemanticChunker |