Awesome Feature Engineering for Machine Learning

A curated list of resources dedicated to Feature Engineering Techniques for Machine Learning

Maintainers - Andrei Khobnia

This page is licensed under Creative Commons Attribution-Noncommercial-ShareAlike 3.0 Unported License

Please feel free to create pull requests.

Contents

• Numeric Data
  • Scaling
  • Ranking
  • Quantization and Binning
  • Box-Cox Transformation
  • Yeo-Johnson Transformation
  • Feature Interactions
  • Clustering Features
  • t-SNE Features
  • PCA Features
• Textual Data
  • Bag of Words
  • Phrase Detection Features
  • TFIDF
  • Word Embeddings
  • Subword Embeddings
  • Pattern Features
  • Lexicon Features
  • PoS Features
• Image Data
  • Computer Vision Algorithm Features
  • Image Statistics Features
  • OCR Features
  • Deep Learning Features
• Categorical Data
  • One Hot Encoding
  • Count Encoding
  • Label Encoding
  • Dummy Encoding
  • Mean Encoding
  • Hashing
• Time Series Data
  • Rolling Window Features
  • Lag Features
• Geospatial Data

Numeric Data

• Understanding Feature Engineering (Part 1) -- Continuous Numeric Data

Scaling

• sklearn.preprocessing.MinMaxScaler
• sklearn.preprocessing.StandartScaler

Ranking

• Ranking
• scipy.stats.rankdata

Quantization and Binning

• Data Binning
• Bucketing Continuous Variables in pandas
• pandas.cat

Box-Cox Transformation

• scipy.stats.boxcox
• np.log (x + const)

Yeo-Johnson Transformation

• Yeo-Johnson Transformation

Feature Interactions

• Featuretools
• sklearn.preprocessing.PolynomialFeatures
• Divisions
• Other interactions

Clustering Features

• How to create New Features using Clustering!!

t-SNE Features

• t-SNE
• Automatic feature extraction with t-SNE

PCA Features

• Principal component analysis (PCA)
• sklearn.decomposition.PCA

Textual Data

• Understanding Feature Engineering (Part 3) -- Traditional Methods for Text Data

Bag of Words

• Bag-of-words model
• A Gentle Introduction to the Bag-of-Words Model
• sklearn.feature_extraction.text.CountVectorizer
• sklearn.feature_extraction.DictVectorizer
• sklearn.feature_extraction.FeatureHasher

Phrase Detection Features

• sklearn_api.phrases – Scikit learn wrapper for phrase (collocation) detection

TFIDF

• tf-idf
• sklearn.feature_extraction.text.TfidfVectorizer

Word Embeddings

• Word embedding
• GloVe: Global Vectors for Word Representation
• Gensim: models.word2vec – Word2vec embeddings
• fastText
• Word2Vec and FastText Word Embedding with Gensim
• Do Pretrained Embeddings Give You The Extra Edge?

Subword Embeddings

• Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)

Pattern Features

• ClearTK - Feature Extraction Tutorial
• Regular Expressions

Lexicon Features

• Named Entity Recognition with Bidirectional LSTM-CNNs (arXiv:1511.08308)

PoS Features

• Part-of-Speech_Tagging
• NLTK Categorizing and Tagging Words
• How to use PoS features in scikit learn classfiers

Image Data

Computer Vision Algorithm Features

• Feature extraction and similar image search with OpenCV for newbies
• OpenCV -- Feature Detection and Description
• SimpleCV.Features package
• Scikit-image feature module

Image Statistics Features

• ImageStat Module -- Pillow

OCR Features

• A Python wrapper for Google Tesseract

Deep Learning Features

• Keras pre-trained models feature extraction
• Using Keras’ Pre-trained Models for Feature Extraction in Image Clustering

Categorical Data

• Understanding Feature Engineering (Part 2) -- Categorical Data

One Hot Encoding

• Why One-Hot Encode Data in Machine Learning?
• How to One Hot Encode Sequence Data in Python
• sklearn.preprocessing.OneHotEncoder
• Keras - to_categorical

Count Encoding

• Feature engineering: Count encoding

Label Encoding

• Label encoding in scikit-learn
• Feature engineering: Label encoding

Dummy Encoding

• Dummy Coding: The how and why
• pandas.get_dummies
• One-Hot vs Dummy encoding

Mean Encoding

• Likelihood encoding of categorical features
• Python target encoding for categorical features
• Adding variance column when mean encoding

Hashing

• Feature Hashing on Wikipedia
• Feature hashing and Extraction in VowpalWabbit
• Feature hashing in scikit-learn

Time Series Data

• Automatic extraction of relevant features from time series
• Basic Feature Engineering With Time Series Data in Python

Rolling Window Features

• pandas.DataFrame.rolling

Lag Features

• Use pandas to lag your timeseries data in order to examine causal relationships

Geospatial Data

• Geospatial Feature Engineering and Visualization
• Intro to Geospatial Data using Python

Back to Top