Awesome MLOps

A curated list of awesome MLOps tools.

Inspired by awesome-python.

• Awesome MLOps
  • AutoML
  • CI/CD for Machine Learning
  • Cron Job Monitoring
  • Data Catalog
  • Data Enrichment
  • Data Exploration
  • Data Management
  • Data Processing
  • Data Validation
  • Data Visualization
  • Drift Detection
  • Feature Engineering
  • Feature Store
  • Hyperparameter Tuning
  • Knowledge Sharing
  • Machine Learning Platform
  • Model Fairness and Privacy
  • Model Interpretability
  • Model Lifecycle
  • Model Serving
  • Model Testing & Validation
  • Optimization Tools
  • Simplification Tools
  • Visual Analysis and Debugging
  • Workflow Tools
• Resources
  • Articles
  • Books
  • Events
  • Other Lists
  • Podcasts
  • Slack
  • Websites
• Contributing

---

AutoML

Tools for performing AutoML.

• AutoGluon - Automated machine learning for image, text, tabular, time-series, and multi-modal data.
• AutoKeras - AutoKeras goal is to make machine learning accessible for everyone.
• AutoPyTorch - Automatic architecture search and hyperparameter optimization for PyTorch.
• AutoSKLearn - Automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator.
• EvalML - A library that builds, optimizes, and evaluates ML pipelines using domain-specific functions.
• FLAML - Finds accurate ML models automatically, efficiently and economically.
• H2O AutoML - Automates ML workflow, which includes automatic training and tuning of models.
• MindsDB - AI layer for databases that allows you to effortlessly develop, train and deploy ML models.
• MLBox - MLBox is a powerful Automated Machine Learning python library.
• Model Search - Framework that implements AutoML algorithms for model architecture search at scale.
• NNI - An open source AutoML toolkit for automate machine learning lifecycle.

CI/CD for Machine Learning

Tools for performing CI/CD for Machine Learning.

• ClearML - Auto-Magical CI/CD to streamline your ML workflow.
• CML - Open-source library for implementing CI/CD in machine learning projects.
• KitOps – Open source MLOps project that eases model handoffs between data scientist and DevOps.

Cron Job Monitoring

Tools for monitoring cron jobs (recurring jobs).

• Cronitor - Monitor any cron job or scheduled task.
• HealthchecksIO - Simple and effective cron job monitoring.
• Heartbeat.pm - Monitoring aliveness of any sensor/cron job.

Data Catalog

Tools for data cataloging.

• Amundsen - Data discovery and metadata engine for improving productivity when interacting with data.
• Apache Atlas - Provides open metadata management and governance capabilities to build a data catalog.
• CKAN - Open-source DMS (data management system) for powering data hubs and data portals.
• DataHub - LinkedIn's generalized metadata search & discovery tool.
• Magda - A federated, open-source data catalog for all your big data and small data.
• Metacat - Unified metadata exploration API service for Hive, RDS, Teradata, Redshift, S3 and Cassandra.
• OpenMetadata - A Single place to discover, collaborate and get your data right.

Data Enrichment

Tools and libraries for data enrichment.

• Snorkel - A system for quickly generating training data with weak supervision.
• Upgini - Enriches training datasets with features from public and community shared data sources.

Data Exploration

Tools for performing data exploration.

• Apache Zeppelin - Enables data-driven, interactive data analytics and collaborative documents.
• BambooLib - An intuitive GUI for Pandas DataFrames.
• DataPrep - Collect, clean and visualize your data in Python.
• Deepnote - Drop-in replacement for Jupyter and an AI-native workspace for modern data teams.
• Google Colab - Hosted Jupyter notebook service that requires no setup to use.
• Jupyter Notebook - Web-based notebook environment for interactive computing.
• JupyterLab - The next-generation user interface for Project Jupyter.
• Jupytext - Jupyter Notebooks as Markdown Documents, Julia, Python or R scripts.
• Pandas Profiling - Create HTML profiling reports from pandas DataFrame objects.
• Polynote - The polyglot notebook with first-class Scala support.

Data Management

Tools for performing data management.

• Arrikto - Dead simple, ultra fast storage for the hybrid Kubernetes world.
• BlazingSQL - A lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.
• Delta Lake - Storage layer that brings scalable, ACID transactions to Apache Spark and other engines.
• Dolt - SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
• Dud - A lightweight CLI tool for versioning data alongside source code and building data pipelines.
• DVC - Management and versioning of datasets and machine learning models.
• Git LFS - An open source Git extension for versioning large files.
• Hub - A dataset format for creating, storing, and collaborating on AI datasets of any size.
• Intake - A lightweight set of tools for loading and sharing data in data science projects.
• lakeFS - Repeatable, atomic and versioned data lake on top of object storage.
• Marquez - Collect, aggregate, and visualize a data ecosystem's metadata.
• Milvus - An open source embedding vector similarity search engine powered by Faiss, NMSLIB and Annoy.
• Pinecone - Managed and distributed vector similarity search used with a lightweight SDK.
• Potato - Portable annotation tool for creating labeled datasets.
• Qdrant - An open source vector similarity search engine with extended filtering support.
• Quilt - A self-organizing data hub with S3 support.

Data Processing

Tools related to data processing and data pipelines.

• Airflow - Platform to programmatically author, schedule, and monitor workflows.
• Azkaban - Batch workflow job scheduler created at LinkedIn to run Hadoop jobs.
• Dagster - A data orchestrator for machine learning, analytics, and ETL.
• Hadoop - Framework that allows for the distributed processing of large data sets across clusters.
• OpenRefine - Power tool for working with messy data and improving it.
• Spark - Unified analytics engine for large-scale data processing.

Data Validation

Tools related to data validation.

• Cerberus - Lightweight, extensible data validation library for Python.
• Cleanlab - Python library for data-centric AI and machine learning with messy, real-world data and labels.
• Great Expectations - A Python data validation framework that allows to test your data against datasets.
• JSON Schema - A vocabulary that allows you to annotate and validate JSON documents.
• TFDV - An library for exploring and validating machine learning data.

Data Visualization

Tools for data visualization, reports and dashboards.

• Count - SQL/drag-and-drop querying and visualisation tool based on notebooks.
• Dash - Analytical Web Apps for Python, R, Julia, and Jupyter.
• Data Studio - Reporting solution for power users who want to go beyond the data and dashboards of GA.
• Facets - Visualizations for understanding and analyzing machine learning datasets.
• Grafana - Multi-platform open source analytics and interactive visualization web application.
• Lux - Fast and easy data exploration by automating the visualization and data analysis process.
• Metabase - The simplest, fastest way to get business intelligence and analytics to everyone.
• Redash - Connect to any data source, easily visualize, dashboard and share your data.
• SolidUI - AI-generated visualization prototyping and editing platform, support 2D and 3D models.
• Superset - Modern, enterprise-ready business intelligence web application.
• Tableau - Powerful and fastest growing data visualization tool used in the business intelligence industry.

Drift Detection

Tools and libraries related to drift detection.

• Alibi Detect - An open source Python library focused on outlier, adversarial and drift detection.
• Frouros - An open source Python library for drift detection in machine learning systems.
• ml3-drift - Drift detection algorithms seamlessly integrated with ML and AI frameworks.
• TorchDrift - A data and concept drift library for PyTorch.

Feature Engineering

*Tools a