Deep-Learning-in-Production

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

Convert PyTorch Models in Production:

• PyTorch Production Level Tutorials [Fantastic]
• The road to 1.0: production ready PyTorch
• PyTorch 1.0 tracing JIT and LibTorch C++ API to integrate PyTorch into NodeJS [Good Article]
• Model Serving in PyTorch
• PyTorch Summer Hackathon [Very Important]
• Deploying PyTorch and Building a REST API using Flask [Important]
• PyTorch model recognizing hotdogs and not-hotdogs deployed on flask
• Serving PyTorch 1.0 Models as a Web Server in C++ [Useful Example]
• PyTorch Internals [Interesting & Useful Article]
• Flask application to support pytorch model prediction
• Serving PyTorch Model on Flask Thread-Safety
• Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX
• Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX (Another Version)
• Deep Dive into ONNX Runtime
• EuclidesDB - multi-model machine learning feature database with PyTorch
• EuclidesDB - GitHub
• WebDNN: Fastest DNN Execution Framework on Web Browser
• FastAI PyTorch Serverless API (with AWS Lambda)
• FastAI PyTorch in Production (discussion)
• OpenMMLab Model Deployment Framework
• PyTorch Native Architecture Optimization: torchao
• TorchServe [Great Tool]
• TorchServe Video Tutorial
• LitServe: Easily serve AI models Lightning fast

Convert PyTorch Models to C++:

• Loading a PyTorch Model in C++ [Fantastic]
• PyTorch C++ API [Bravo]
• An Introduction To Torch (Pytorch) C++ Front-End [Very Good]
• Blogs on using PyTorch C++ API [Good]
• ATen: A TENsor library
• Important Issue about PyTorch-like C++ interface
• PyTorch C++ API Test
• PyTorch via C++ [Useful Notes]
• AUTOGRADPP
• PyTorch C++ Library
• Direct C++ Interface to PyTorch
• A Python module for compiling PyTorch graphs to C

Deploy TensorFlow Models in Production:

• How to deploy Machine Learning models with TensorFlow - Part1
• How to deploy Machine Learning models with TensorFlow - Part2
• How to deploy Machine Learning models with TensorFlow - Part3
• Neural Structured Learning (NSL) in TensorFlow [Great]
• Building Robust Production-Ready Deep Learning Vision Models
• Creating REST API for TensorFlow models
• "How to Deploy a Tensorflow Model in Production" by Siraj Raval on YouTube
• Code for the "How to Deploy a Tensorflow Model in Production" by Siraj Raval on YouTube
• How to deploy an Object Detection Model with TensorFlow serving [Very Good Tutorial]
• Freeze Tensorflow models and serve on web [Very Good Tutorial]
• How to deploy TensorFlow models to production using TF Serving [Good]
• How Zendesk Serves TensorFlow Models in Production
• TensorFlow Serving Example Projects

Convert Keras Models in Production:

• Deep learning in production with Keras, Redis, Flask, and Apache [Rank: 1st & General Usefult Tutorial]
• Deploying a Keras Deep Learning Model as a Web Application in Python [Very Good]
• Deploying a Python Web App on AWS [Very Good]
• Deploying Deep Learning Models Part 1: Preparing the Model
• Deploying your Keras model
• Deploying your Keras model using Keras.JS
• "How to Deploy a Keras Model to Production" by Siraj Raval on Youtube
• Deploy Keras Model with Flask as Web App in 10 Minutes [Good Repository]
• Deploying Keras Deep Learning Models with Flask
• keras2cpp

Deploy MXNet Models in Production:

• Model Server for Apache MXNet
• Running the Model Server
• Multi Model Server (MMS) Documentation
• Introducing Model Server for Apache MXNet
• Single Shot Multi Object Detection Inference Service
• Amazon SageMaker
• How can we serve MXNet models built with gluon api
• MXNet C++ Package
• MXNet C++ Package Examples
• MXNet Image Classification Example of C++
• MXNet C++ Tutorial
• An introduction to the MXNet API [Very Good Tutorial for Learning MXNet]
• GluonCV
• GluonNLP
• Model Quantization for Production-Level Neural Network Inference [Excellent]

Deploy Machine Learning Models with Go:

• Cortex: Deploy machine learning models in production
• Cortex - Main Page
• Why we deploy machine learning models with Go — not Python
• Go-Torch
• Gotch - Go API for PyTorch
• TensorFlow Go Lang
• Go-onnx

General Deep Learning Deployment Toolkits:

• OpenVINO Toolkit - Deep Learning Deployment Toolkit repository [Great]
• ClearML - ML/DL development and production suite
• Model Deployment Using Heroku: A Complete Guide on Heroku [Good]
• NVIDIA Triton Inference Server [Great]
• NVIDIA Triton Inference Server - GitHub [Great]
• Cohere Boosts Inference Speed With NVIDIA Triton Inference Server
• NVIDIA Deep Learning Examples for Tensor Cores [Interesting]
• Deploying the Jasper Inference model using Triton Inference Server [Useful]
• Nvidia MLOPs Course via Triton
• Awesome Production Machine Learning [Gr