Jupyter Notebooks for "Ensemble Methods for Machine Learning"

This repository contains companion material: data, Python code and Jupyter notebooks for Ensemble Methods for Machine Learning (Manning Publications). The code and notebooks are released under the MIT license.

These notebooks primarily use Python 3.7, scikit-learn 0.22 and matplotlib 3.2.1, though other packages such as pandas, seaborn and Keras make guest appearances as well.

This book is a work in progress and expected to be released some time in Fall 2022.

List of notebooks

• Chapter 1. Ensemble Methods: Hype or Halleujah?

  • 1.3. Fit vs. Complexity in Machine-Learning Models
  • 1.4. A Simple Model Averaging Ensemble

• Chapter 2. Homogeneous Parallel Ensembles: Bagging and Random Forests

  • 2.2, 2.3. Bagging and Random Forest
  • 2.5. Case Study: Breast Cancer Diagnosis

• Chapter 3: Heterogeneous Parallel Ensembles: Combining Strong Learners

  • 3.1 Base Estimators for Heterogeneous Ensembles
  • 3.2 Combining Predictions by Weighting
  • 3.3 Combining Predictions by Meta-Learning
  • 3.4 Case Study: Sentiment Analysis

• Chapter 4: Sequential Ensembles: Boosting

  • 4.1, 4.2 AdaBoost: Adaptive Boosting
  • 4.3 AdaBoost in Practice
  • 4.4 Case Study: Handwritten Digit Classification
  • 4.5 LogitBoost: Boosting with the Logistic Loss

• Chapter 5: Sequential Ensembles: Gradient Boosting

  • 5.1 Gradient Descent for Minimization
  • 5.2 Gradient Boosting: Gradient Descent + Boosting
  • 5.3, 5.4 LightGBM in Practice
  • 5.5 Case Study: Document Retrieval

• Chapter 6: Sequential Ensembles: Newton Boosting

  • 6.1 Newton's Method for Minimization
  • 6.2 Newton Boosting: Newton's Method + Boosting
  • 6.3, 6.4 XGBoost in Practice
  • 6.5 Case Study: Document Retrieval

• Chapter 7: Learning with Continuous and Count Labels

  • 7.1 A Brief Review of Regression
  • 7.2, 7.3 Parallel and Sequential Ensembles for Regression
  • 7.4 Case Study: Demand Forecasting

• Chapter 8: Learning with Categorical Features

  • 8.1, 8.2 Encoding and Training with Categorical Features with category_encoders & CatBoost
  • 8.3 Case Study: Income Prediction
  • 8.4 Encoding High-Cardinality String Features

• Chapter 9: Explaining Your Ensembles

  • 9.1 What is Interpretability? Black-Box vs. Glass-Box Models
  • 9.2 Case Study: Data-driven Marketing
  • 9.3 Black-Box Methods for Global Explainability
  • 9.4, 9.5 Black-Box Methods for Local Explainability, Glass-Box Ensembles