Welcome to TheGradientPath 🚀
Your comprehensive learning journey through modern Machine Learning, Deep Learning, and Artificial Intelligence - from fundamentals to production systems.
This repository is a complete educational resource that bridges theory with practice, covering everything from foundational neural networks to cutting-edge AI agent systems and production ML deployment. Each project is designed to be hands-on, practical, and production-ready, with clear documentation, video tutorials, and runnable code.
📚 Table of Contents
AI Agents
Comprehensive AI Agent Framework Benchmark
Location: AiAgents/AgentFrameworkBenchmark/
A production-grade comparison of 7 major AI agent frameworks implementing identical multi-agent systems to provide objective, real-world benchmarks.
Video Tutorial
Watch on YouTube - Complete framework comparison and implementation guide
Frameworks Compared
- LangChain/LangGraph (🥇 284/360) - Best overall, maximum flexibility
- OpenAI Agents (🥈 277/360) - Minimal code, native MCP support
- CrewAI (🥉 249/360) - Rapid prototyping, simple delegation
- LlamaIndex (227/360) - Balanced workflow architecture
- AutoGen (195/360) - Enterprise async infrastructure
- Semantic Kernel (178/360) - Microsoft ecosystem integration
- Vanilla Python - Baseline with zero framework overhead
What's Benchmarked
- ✅ Agent Orchestration - Multi-agent coordination and routing
- ✅ Tool Integration - Custom tool creation and execution
- ✅ State Management - Complex state handling across agents
- ✅ Memory Management - Persistent conversation history
- ✅ MCP Server Integration - Model Context Protocol support
- ✅ Production Features - Guardrails, token tracking, structured output
System Architecture
Each implementation includes:
- Orchestrator Agent - Routes queries to specialized agents
- Legal Expert Agent - Handles law and legal topics
- Operational Agent - Manages programming and general queries
- Tools - Weather API, calculator, web search
- MCP Integration - Extended capabilities via Model Context Protocol
Full Documentation
Deep Learning with Keras
Modern Neural Network Implementations
Location: Keras/
Production-ready deep learning implementations using TensorFlow and Keras, from fundamentals to advanced architectures.
1. Image Classification with MLP
Path: Keras/ImageClassificationWithMLP/
- Dataset: MNIST handwritten digits
- Architecture: Multi-layer perceptron with Dropout and BatchNormalization
- Features: TensorBoard integration, Visualkeras architecture diagrams
- Tools: Dense layers, functional API, comprehensive logging
2. Transformer-Based Text Generation
Path: Keras/transformers/text_generation/
- Task: Natural language generation from scratch
- Architecture: Complete Transformer implementation
- Components: Multi-head self-attention, positional encoding, feed-forward networks
- Features: Custom training loop, text preprocessing, generation sampling
3. Text Generation with KV Cache
Path: Keras/transformers/kv_cache_for text_gen/
- Optimization: Key-Value cache for efficient inference
- Performance: Dramatically reduced computation during generation
- Architecture: Modified Transformer with caching mechanism
- Use Case: Production LLM inference optimization
4. Time Series Forecasting with Transformers
Path: Keras/transformers/time_series_forecast/
- Task: Stock price prediction using Transformers
- Data: Synthetic financial time series
- Architecture: Transformer adapted for sequential prediction
- Features: Temporal embeddings, MinMax scaling, visualization
Key Learning Points:
- Building Transformers from scratch in Keras
- Multi-head attention mechanisms
- Positional encoding strategies
- KV cache optimization techniques
- Adapting Transformers for different domains
PyTorch Projects
CNN Image Classification
Location: Pytotch/CnnImageClassification/
Full README
Fashion-MNIST CNN Classifier
- Dataset: 70,000 images of 10 clothing categories
- Architecture: 2-layer CNN with BatchNorm
- Conv2d(1→16) + BatchNorm + ReLU + MaxPool
- Conv2d(16→32) + BatchNorm + ReLU + MaxPool
- Fully Connected (512→10)
- Performance: ~85-90% validation accuracy
- Features:
- Automatic dataset download
- GPU acceleration support
- Model checkpointing
- Training visualization
- Real-time progress monitoring
Key Learning Points:
- Convolutional neural networks fundamentals
- Batch normalization for training stability
- PyTorch DataLoader and Dataset classes
- Model training and evaluation pipelines
LLM Fine-Tuning
Advanced Parameter-Efficient Fine-Tuning Techniques
Location: LLMFineTuning/
State-of-the-art techniques for efficiently fine-tuning large language models for specific tasks.
1. All PEFT Techniques From Scratch
Path: LLMFineTuning/all_peft_tecniques_from_scratch/
Complete implementation of Parameter-Efficient Fine-Tuning methods:
- LoRA (Low-Rank Adaptation) - Inject trainable low-rank matrices
- Prefix Tuning - Learn soft prompts prepended to inputs
- Adapter Layers - Small bottleneck layers inserted into models
- IA³ (Infused Adapter by Inhibiting and Amplifying Inner Activations)
Why PEFT?
- Train only 0.1-1% of model parameters
- Reduce memory requirements by 90%
- Maintain performance close to full fine-tuning
- Enable multi-task learning with parameter isolation
2. GRPO Reasoning with Unsloth
Path: LLMFineTuning/GRPO_REASONING_UNSLOTH/
Advanced reasoning capabilities through Gradient-based Reward Policy Optimization:
- Model: Google Gemma 3 1B with 4-bit quantization
- Technique: GRPO (combines PPO benefits with gradient-based optimization)
- Task: Mathematical reasoning with structured outputs
- Features:
- LoRA rank-32 adaptation
- 4-bit quantization for memory efficiency
- vLLM acceleration for fast inference
- Structured reasoning format (
<reasoning> and <answer> tags)
Performance Gains:
- Models learn to show reasoning steps
- Improved accuracy on complex problems
- Better interpretability of model decisions
3. Supervised Fine-Tuning with Tool Choice
Path: LLMFineTuning/SFT_HF_TOOL_CHOICE/
Teaching models to intelligently select tools:
- Model: HuggingFace SmolLM2-135M
- Task: Tool selection based on user queries
- Dataset: 10,000 synthetic examples with tool annotations
- Technique: Supervised Fine-Tuning with custom special tokens
- Use Case: Building function-calling capabilities in smaller models
Real-World Application:
- Enable LLMs to use external tools (calculators, APIs, databases)
- Reduce reliance on large models for specialized tasks
- Build cost-effective AI assistants
RAG Systems (Retrieval-Augmented Generation)
Advanced RAG Architectures
Location: Rag/
Production-ready Retrieval-Augmented Generation systems that enhance LLM responses with external knowledge.
1. Dartboard RAG
Path: Rag/dartboard/
Full README
Balanced Relevance and Diversity Retrieval
Based on the paper: "Better RAG using Relevant Information Gain"
Key Innovation:
- Problem: Standard top-k retrieval returns redundant documents
- Solution: Optimize combined relevance-diversity score
- Result: Non-redundant, comprehensive context for LLMs
Features:
- Configurable relevance/diversity weights
- Production-ready modular design
- FAISS vector store integration
- Oversampling for better candidate selection
Algorithm:
combined_score = diversity_weight * diversity + relevance_weight * relevance
When to Use:
- Dense knowledge bases with overlapping information
- Queries requiring diverse perspectives
- Avoiding echo chambers in retrieval
2. Hybrid Multivector Knowledge Graph RAG
Path: Rag/hybrid_multivector_knowledge_graph_rag/
Full README
The Most Advanced RAG System - 11+ Graph Traversal Algorithms
Revolutionary Features:
- Knowledge Graph Engineering with Neo4j
- Multi-Vector Embeddings for nuanced retrieval
- 11+ Graph Traversal Algorithms:
- K-hop Limited BFS
- Depth-Limited DFS
- A* Search with heuristics
- Beam Search
- Uniform Cost Search (UCS)
- Context-to-Cypher query generation
- LLM-powered intelligent filtering
Architecture:
- Vector Retrieval - Initial similarity search
- Graph Traversal - Navigate knowledge relationships
- Entity Extraction - LLM-powered entity identification
- Dynamic Querying - Context-aware Cypher generation
- Intelligent Ranking - Multi-factor relevance scoring
Why Knowledge Graphs?
- Discover hidden connections across concepts
- Follow chains of reasoning
- Understand complex relationships
- Navigate multi-hop queries intelligently
Use Cases:
- Research and academic knowledge bases
- Legal document analysis
- Scientific literature review
- Complex domain expertise systems
3. Vision RAG
Path: Rag/vision_rag/
Multimodal RAG for Documents with Images
Capabilities:
- PDF Processing - Extract text and images from documents
- Image Embeddings - CLIP-based visual understanding
- Unified Retrieval - Search across text and images simultaneously
- PostgreSQL + pgvector - Scalable vector storage
- Docker Deployment - Production-ready containerization
Architecture:
- Text extraction and chunking
- Image extraction and captioning
- Dual embedding spaces (text + vision)
- Unified query interface
- Relevance-based ranking
Use Cases:
- Architectural design documents
- Scientific papers with diagrams
- Product catalogs
- Technical manuals
- Medical imaging reports
Real-World Production Projects
ML Cyber Attack Prediction System
Location: RealWorldProjects/CyberAttackPrediction/
Full README | Video Tutorial
Enterprise-grade cloud-native ML system for real-time network threat detection.
Complete Production Stack
Infrastructure (AWS CloudFormation)
- Application Load Balancer - HTTPS/HTTP traffic distribution
- Auto Scaling Groups - Elastic capacity management
- EC2 Instances - Ubuntu 22.04 LTS compute
- Target Groups - Health-checked backend pools
- Security Groups - Network isolation and access control
- IAM Roles - Least-privilege security model
CI/CD Pipeline
- AWS CodePipeline - Automated deployment workflows
- AWS CodeBuild - Application compilation and testing
- AWS CodeDeploy - Zero-downtime deployments
- S3 Artifact Storage - Build artifact management
- GitHub Integration - Source control via CodeStar
ML Architecture
Multi-Stage Pipeline:
- Data Preprocessing - Mixed numerical/categorical feature handling
- AutoEncoder - Learn normal traffic patterns, detect anomalies
- Feature Selection (ORC) - Dynamic relevance-based feature selection
- **SG