Deep Learning - The Straight Dope (Deprecated Please see d2l.ai)

This content has been moved to Dive into the Deep Learning Book freely available at https://d2l.ai/.

Abstract

This repo contains an incremental sequence of notebooks designed to teach deep learning, MXNet, and the gluon interface. Our goal is to leverage the strengths of Jupyter notebooks to present prose, graphics, equations, and code together in one place. If we're successful, the result will be a resource that could be simultaneously a book, course material, a prop for live tutorials, and a resource for plagiarising (with our blessing) useful code. To our knowledge there's no source out there that teaches either (1) the full breadth of concepts in modern deep learning or (2) interleaves an engaging textbook with runnable code. We'll find out by the end of this venture whether or not that void exists for a good reason.

Another unique aspect of this book is its authorship process. We are developing this resource fully in the public view and are making it available for free in its entirety. While the book has a few primary authors to set the tone and shape the content, we welcome contributions from the community and hope to coauthor chapters and entire sections with experts and community members. Already we've received contributions spanning typo corrections through full working examples.

Implementation with Apache MXNet

Throughout this book, we rely upon MXNet to teach core concepts, advanced topics, and a full complement of applications. MXNet is widely used in production environments owing to its strong reputation for speed. Now with gluon, MXNet's new imperative interface (alpha), doing research in MXNet is easy.

Dependencies

To run these notebooks, you'll want to build MXNet from source. Fortunately, this is easy (especially on Linux) if you follow these instructions. You'll also want to install Jupyter and use Python 3 (because it's 2017).

Slides

The authors (& others) are increasingly giving talks that are based on the content in this books. Some of these slide-decks (like the 6-hour KDD 2017) are gigantic so we're collecting them separately in this repo. Contribute there if you'd like to share tutorials or course material based on this books.

Translation

As we write the book, large stable sections are simultaneously being translated into 中文, available in a web version and via GitHub source.

Table of contents

Part 1: Deep Learning Fundamentals

• Chapter 1: Crash course

  • Preface
  • Introduction
  • Manipulating data with NDArray
  • Linear algebra
  • Probability and statistics
  • Automatic differentiation via autograd

• Chapter 2: Introduction to supervised learning

  • Linear regression (from scratch)
  • Linear regression (with gluon)
  • Binary classification with logistic regression (gluon w bespoke loss function)
  • Multiclass logistic regression (from scratch)
  • Multiclass logistic regression (with gluon)
  • Overfitting and regularization

Part 2: Applications

• Chapter 8: Computer vision (CV)

  • Roadmap Network of networks (inception & co)
  • Roadmap Residual networks
  • Object detection
  • Roadmap Fully-convolutional networks
  • Roadmap Siamese (conjoined?) networks
  • Roadmap Embeddings (pairwise and triplet losses)
  • Roadmap Inceptionism / visualizing feature detectors
  • Roadmap Style transfer
  • Visual-question-answer
  • Fine-tuning

• Chapter 9: N