AI Engineering Toolkit🔥

Build better LLM apps — faster, smarter, production-ready.

A curated, list of 100+ libraries and frameworks for AI engineers building with Large Language Models. This toolkit includes battle-tested tools, frameworks, templates, and reference implementations for developing, deploying, and optimizing LLM-powered systems.

📋 Table of Contents

• 🛠️ Tooling for AI Engineers
  • Vector Databases
  • Orchestration & Workflows
  • RAG (Retrieval-Augmented Generation)
  • Evaluation & Testing
  • Model Management
  • Data Collection & Web Scraping
• 🤖 Agent Frameworks
• 📦 LLM Development & Optimization
  • Open Source LLM Inference
  • LLM Safety & Security
  • AI App Development Frameworks
  • Local Development & Serving
  • LLM Inference Platforms
  • Structured Generation
• 🤝 Contributing

🛠️ Tooling for AI Engineers

Vector Databases

Tool	Description	Language	License
Pinecone	Managed vector database for production AI applications	API/SDK	Commercial
Weaviate	Open-source vector database with GraphQL API	Go	BSD-3
Qdrant	Vector similarity search engine with extended filtering	Rust	Apache-2.0
Chroma	Open-source embedding database for LLM apps	Python	Apache-2.0
Milvus	Cloud-native vector database for scalable similarity search	Go/C++	Apache-2.0
FAISS	Library for efficient similarity search and clustering	C++/Python	MIT
Deep Lake	AI-native data lake with versioned datasets, optimized for embeddings and multimodal storage	Python	Apache-2.0
Vectara	Managed RAG platform with APIs for retrieval and generation	Python/Go	Commercial

Orchestration & Workflows

Tool	Description	Language	License
LangChain	Framework for developing LLM applications	Python/JS	MIT
LlamaIndex	Data framework for LLM applications	Python	MIT
Haystack	End-to-end NLP framework for production	Python	Apache-2.0
DSPy	Framework for algorithmically optimizing LM prompts	Python	MIT
Semantic Kernel	SDK for integrating AI into conventional programming languages	C#/Python/Java	MIT
Langflow	Visual no-code platform for building and deploying LLM workflows	Python/TypeScript	MIT
Flowise	Drag-and-drop UI for creating LLM chains and agents	TypeScript	MIT
Promptflow	Workflow orchestration for LLM pipelines, evaluation, and deployment	Python	MIT
Dify	Production-ready open source platform combining RAG pipelines, agent capabilities, model management, and observability in one UI	TypeScript/Python	Apache-2.0

PDF Extraction Tools

Tool	Description	Language	License
Docling	AI-powered toolkit converting PDF, DOCX, PPTX, HTML, images into structured JSON/Markdown with layout, OCR, table, and code recognition	Python	MIT
pdfplumber	Drill through PDFs at a character level, extract text & tables, and visually debug extraction	Python	MIT
PyMuPDF (fitz)	Lightweight, high-performance PDF parser for text/image extraction and manipulation	Python / C	AGPL-3.0
PDF.js	Browser-based PDF renderer with text extraction capabilities	JavaScript	Apache-2.0
Camelot	Extracts structured tabular data from PDFs into DataFrames and CSVs	Python	MIT
Unstructured	Parse PDFs, DOCX, HTML into structured JSON for LLM workflows	Python	Apache-2.0
pdfminer.six	Detailed PDF text extraction and layout analysis	Python	MIT
Llama Parse	Structured parsing of PDFs and documents optimized for LLMs	Python	Apache-2.0
MegaParse	Universal parser for PDFs, HTML, and semi-structured documents	Python	Apache-2.0
ExtractThinker	Intelligent document extraction framework with schema mapping	Python	MIT
PyMuPDF4LLM

RAG (Retrieval-Augmented Generation)

Tool	Description	Language	License
RAGFlow	Open-source RAG engine based on deep document understanding	Python	Apache-2.0
Verba	Retrieval Augmented Generation (RAG) chatbot	Python	BSD-3
PrivateGPT	Interact with documents using local LLMs	Python	Apache-2.0
AnythingLLM	All-in-one AI application for any LLM	JavaScript	MIT
Quivr	Your GenAI second brain	Python/TypeScript	Apache-2.0
Jina	Cloud-native neural search framework for multimodal RAG	Python	Apache-2.0
txtai	All-in-one embeddings database for semantic search and workflows	Python	Apache-2.0
FastGraph RAG	Graph-based RAG framework for structured retrieval	Python	MIT
Chonkie	Chunking utility for efficient document processing in RAG	Python	-
FlashRAG	Low-latency RAG research toolkit with modular design and benchmarks	Python	-
Llmware	Lightweight framework for building RAG-based apps	Python	Apache-2.0

Evaluation & Testing

Tool	Description	Language	License
Evals	OpenAI's framework for creating and running LLM evaluations	Python	MIT
Ragas	Evaluation framework for RAG pipelines	Python	Apache-2.0
Opik	DevOps platform for evaluation, monitoring, and observability	Python	Apache-2.0
Phoenix	ML observability for LLM, vision, language, and tabular models	Python	Apache-2.0
DeepEval	LLM evaluation framework for unit testing LLM outputs	Python	Apache-2.0
TruLens	Evaluation and tracking for LLM experiments	Python	MIT
UpTrain	Open-source tool to evaluate and improve LLM applications	Python	Apache-2.0
Giskard	Testing framework for ML/LLMs with bias and robustness checks	Python	Apache-2.0
Weave	Experiment tracking, debugging, and logging for LLM workflows	Python	Apache-2.0
Lighteval	Lightweight and fast evaluation framework from Hugging Face	Python	Apache-2.0
Langfuse	Open source LLM engineering platform with tracing, evals, prompt management, and metrics	TypeScript/Python	MIT

Model Management

Tool	Description	Language	License
Hugging Face Hub	Client library for Hugging Face Hub	Python	Apache-2.0
MLflow	Platform for ML lifecycle management	Python	Apache-2.0
Weights & Biases	Developer tools for ML	Python	MIT
DVC	Data version control for ML projects	Python	Apache-2.0
ClearML	End-to-end MLOps platform with LLM support	Python	Apache-2.0

Data Collection & Web Scraping

Tool	Description	Language	License
Firecrawl	AI-powered web crawler that extracts and structures content for LLM pipelines	TypeScript	MIT
Scrapy	Fast, high-level web crawling & scraping framework	Python	BSD-3
Playwright	Web automation & scraping with headless browsers	TypeScript/Python/Java/.NET	Apache-2.0
BeautifulSoup	Easy HTML/XML parsing for quick scraping tasks	Python	MIT
Selenium	Browser automation framework (supports scraping)	Multiple	Apache-2.0
Newspaper3k	News & article extraction library	Python	MIT
Crawl4AI	Fast, lightweight, and modern web crawling & scraping library for AI data pipelines	Python	Apache-2.0
Colly	High-performance scraping framework for Go	Go	BSD-2
Trafilatura	Extract clean text from web pages for LLM training corpora	Python	MIT
ScrapeGraphAI	Use LLMs to extract structured data from websites and documents	Python	MIT
Crawlee	Web scraping and crawling framework for large-scale data collection	TypeScript	Apache-2.0

🤖 Agent Frameworks

Framework	Description	Language	License
Google's ADK	Flexible and modular framework for developing and deploying AI agents