Skip to content
ML-From-Scratch GitHub Details, Stars and Alternatives | OpenRepoFinder
Home / Repositories / rohanmistry231/ML-From-Scratch rohanmistry231 / repository
ML-From-Scratch A compact collection of classic machine learning algorithms implemented from scratch in Python for learning and experimentation.
#algorithms #classification #from-scratch-in-python #machine-learning #ml-algorithms #python
View Repository on GitHub ↗ REPOSITORY OVERVIEW Live repository statistics ★ 3 Stars
⑂ 1 Forks
◯ 0 Open issues
◉ 3 Watchers
74 /100
OPENREPOHUB HEALTH SIGNAL Healthy signals A transparent discovery signal based on current public GitHub metadata.
Recent activity35% weight
100 Community adoption25% weight
7 Maintenance state20% weight
100 License clarity10% weight
100 Project information10% weight
75 This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
README preview 🧠 ML From Scratch
Every Machine Learning Algorithm, Implemented From Scratch in Python.
The only repo you need to understand how ML truly works under the hood — and ace your interviews.
🎯 What Is This?
This is a comprehensive, interview-ready collection of 48 machine learning algorithms — all coded from scratch using only Python and NumPy. No scikit-learn. No TensorFlow. No PyTorch. Just pure math turned into working code.
Every algorithm comes with:
✅ Clean Python implementation — class-based, well-commented, importable
✅ README with theory — intuition, math, pseudocode, complexity analysis
✅ Runnable example — demo on a toy dataset with printed output
✅ Interview Q&A — real questions you'll face, with crisp answers
🤔 Who Is This For?
ML/AI learners who want to go beyond calling model.fit() and actually understand what happens insideEngineers preparing for interviews at FAANG, startups, or ML-focused companiesStudents supplementing coursework with hands-on implementationsAnyone who believes the best way to learn an algorithm is to build it
📁 Repository Structure
ml-from-scratch/
│
├── README.md ← you are here
├── pyproject.toml ← pip install -e . to install as package
├── requirements.txt
├── LICENSE
│
├── utils/ ← shared helpers used across all algorithms
│ ├── __init__.py
│ ├── metrics.py ← accuracy, precision, recall, F1, MSE, R², etc.
│ ├── distances.py ← euclidean, manhattan, cosine, minkowski
│ ├── activations.py ← sigmoid, relu, tanh, softmax + derivatives
│ ├── losses.py ← MSE, cross-entropy, hinge loss, etc.
│ └── preprocessing.py ← standard scaler, train/test split, one-hot
│
└── algorithms/
├── 01_supervised_regression/
│ ├── linear_regression_normal_equation/
│ ├── linear_regression_gradient_descent/
│ ├── polynomial_regression/
│ ├── ridge_regression/
│ ├── lasso_regression/
│ └── elastic_net/
│
├── 02_supervised_classification/
│ ├── logistic_regression/
│ ├── k_nearest_neighbors/
│ ├── naive_bayes_gaussian/
│ ├── naive_bayes_multinomial/
│ ├── perceptron/
│ ├── svm_linear/
│ ├── svm_kernel/
│ ├── decision_tree_classifier/
│ └── decision_tree_regressor/
│
├── 03_ensemble_methods/
│ ├── bagging/
│ ├── random_forest/
│ ├── adaboost/
│ ├── gradient_boosting/
│ └── stacking/
│
├── 04_clustering/
│ ├── k_means/
│ ├── k_means_plus_plus/
│ ├── hierarchical_clustering/
│ ├── dbscan/
│ ├── mean_shift/
│ └── gaussian_mixture_models/
│
├── 05_dimensionality_reduction/
│ ├── pca/
│ ├── svd/
│ ├── lda/
│ └── tsne/
│
├── 06_neural_networks/
│ ├── single_layer_perceptron/
│ ├── multi_layer_perceptron/
│ ├── cnn/
│ ├── rnn_vanilla/
│ ├── lstm/
│ └── autoencoder/
│
├── 07_optimization/
│ ├── batch_gradient_descent/
│ ├── stochastic_gradient_descent/
│ ├── mini_batch_gradient_descent/
│ ├── momentum/
│ ├── rmsprop/
│ └── adam/
│
├── 08_recommender_systems/
│ ├── collaborative_filtering/
│ └── matrix_factorization/
│
├── 09_reinforcement_learning/
│ ├── q_learning/
│ └── sarsa/
│
└── 10_probabilistic/
├── hidden_markov_model/
└── apriori/
Each algorithm folder contains:
<algorithm_name>/
├── README.md ← theory, math, pseudocode, interview Q&A
├── <algorithm_name>.py ← from-scratch implementation (class with fit/predict)
├── example.py ← runnable demo with output
🖼️ Visual Gallery Run the notebooks in notebooks/ to generate these visualizations:
Category What You'll See Regression 6 regression lines (Linear, Polynomial, Ridge, Lasso, Elastic Net) on the same scatter plot — compare fits visually Classification Decision boundaries for Logistic Regression, KNN, Naive Bayes, Decision Tree — 2×2 grid on 2D data Ensembles Single tree vs Random Forest vs AdaBoost decision boundaries — see how ensembles reduce overfitting Clustering K-Means, DBSCAN, GMM side-by-side on blob data — compare cluster assignments Dimensionality Reduction PCA, SVD, LDA 2D projections from 10D data — see which preserves class separation Neural Networks MLP learning XOR — decision boundary before/after training + loss curve Optimizers Momentum, RMSProp, Adam converging on a simple 1D loss surface — see trajectory differences
cd notebooks/
jupyter notebook
📋 Full Algorithm Index
0. Utils — Shared Building Blocks File What It Covers metrics.pyAccuracy, Precision, Recall, F1, MSE, RMSE, MAE, R² distances.pyEuclidean, Manhattan, Cosine, Minkowski activations.pySigmoid, ReLU, Tanh, Softmax + all derivatives losses.pyMSE, Binary/Categorical Cross-Entropy, Hinge preprocessing.pyStandardScaler, MinMaxScaler, train_test_split, one-hot encoding
1. Supervised Learning — Regression | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 1 | Linear Regression (Normal Equation) | 01_supervised_regression/linear_regression_normal_equation/ | ✅ |
| 2 | Linear Regression (Gradient Descent) | 01_supervised_regression/linear_regression_gradient_descent/ | ✅ |
| 3 | Polynomial Regression | 01_supervised_regression/polynomial_regression/ | ✅ |
| 4 | Ridge Regression (L2) | 01_supervised_regression/ridge_regression/ | ✅ |
| 5 | Lasso Regression (L1) | 01_supervised_regression/lasso_regression/ | ✅ |
| 6 | Elastic Net | 01_supervised_regression/elastic_net/ | ✅ |
2. Supervised Learning — Classification | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 7 | Logistic Regression | 02_supervised_classification/logistic_regression/ | ✅ |
| 8 | K-Nearest Neighbors | 02_supervised_classification/k_nearest_neighbors/ | ✅ |
| 9 | Naive Bayes (Gaussian) | 02_supervised_classification/naive_bayes_gaussian/ | ✅ |
| 10 | Naive Bayes (Multinomial) | 02_supervised_classification/naive_bayes_multinomial/ | ✅ |
| 11 | Perceptron | 02_supervised_classification/perceptron/ | ✅ |
| 12 | SVM (Linear) | 02_supervised_classification/svm_linear/ | ✅ |
| 13 | SVM (Kernel) | 02_supervised_classification/svm_kernel/ | ✅ |
| 14 | Decision Tree (Classifier) | 02_supervised_classification/decision_tree_classifier/ | ✅ |
| 15 | Decision Tree (Regressor) | 02_supervised_classification/decision_tree_regressor/ | ✅ |
3. Ensemble Methods | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 16 | Bagging | 03_ensemble_methods/bagging/ | ✅ |
| 17 | Random Forest | 03_ensemble_methods/random_forest/ | ✅ |
| 18 | AdaBoost | 03_ensemble_methods/adaboost/ | ✅ |
| 19 | Gradient Boosting | 03_ensemble_methods/gradient_boosting/ | ✅ |
| 20 | Stacking | 03_ensemble_methods/stacking/ | ✅ |
4. Clustering | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 21 | K-Means | 04_clustering/k_means/ | ✅ |
| 22 | K-Means++ | 04_clustering/k_means_plus_plus/ | ✅ |
| 23 | Hierarchical Clustering | 04_clustering/hierarchical_clustering/ | ✅ |
| 24 | DBSCAN | 04_clustering/dbscan/ | ✅ |
| 25 | Mean Shift | 04_clustering/mean_shift/ | ✅ |
| 26 | Gaussian Mixture Models | 04_clustering/gaussian_mixture_models/ | ✅ |
5. Dimensionality Reduction | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 27 | PCA | 05_dimensionality_reduction/pca/ | ✅ |
| 28 | SVD | 05_dimensionality_reduction/svd/ | ✅ |
| 29 | LDA | 05_dimensionality_reduction/lda/ | ✅ |
| 30 | t-SNE | 05_dimensionality_reduction/tsne/ | ✅ |
6. Neural Networks | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 31 | Single-Layer Perceptron | 06_neural_networks/single_layer_perceptron/ | ✅ |
| 32 | Multi-Layer Perceptron | 06_neural_networks/multi_layer_perceptron/ | ✅ |
| 33 | CNN | 06_neural_networks/cnn/ | ✅ |
| 34 | RNN (Vanilla) | 06_neural_networks/rnn_vanilla/ | ✅ |
| 35 | LSTM | 06_neural_networks/lstm/ | ✅ |
| 36 | Autoencoder | 06_neural_networks/autoencoder/ | ✅ |
7. Optimization Algorithms | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 37 | Batch Gradient Descent | 07_optimization/batch_gradient_descent/ | ✅ |
| 38 | Stochastic Gradient Descent | 07_optimization/stochastic_gradient_descent/ | ✅ |
| 39 | Mini-Batch Gradient Descent | 07_optimization/mini_batch_gradient_descent/ | ✅ |
| 40 | Momentum | 07_optimization/momentum/ | ✅ |
| 41 | RMSProp | 07_optimization/rmsprop/ | ✅ |
| 42 | Adam | 07_optimization/adam/ | ✅ |
8. Recommender Systems | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 43 | Collaborative Filtering | 08_recommender_systems/collaborative_filtering/ | ✅ |
| 44 | Matrix Factorization | 08_recommender_systems/matrix_factorization/ | ✅ |
9. Reinforcement Learning | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 45 | Q-Learning | 09_reinforcement_learning/q_learning/ | ✅ |
| 46 | SARSA | 09_reinforcement_learning/sarsa/ | ✅ |
10. Probabilistic & Other | # | Algorithm | Folder | Status |
|---|---|---|---|---|
| 47 | Hidden Markov Model | 10_probabilistic/hidden_markov_model/ | ✅ |
| 48 | Apriori | 10_probabilistic/apriori/ | ✅ |
🚀 Getting Started
1. Clone the repo git clone https://github.com/rohanmistry231/ml-from-scratch.git
cd ml-from-scratch
2. Install dependencies pip install -r requirements.txt
Only 3 dependencies : numpy, matplotlib, and scikit-learn (used only in examples for loading datasets and benchmarking — never in the algorithm code itself).
3. Run any algorithm cd algorithms/01_supervised_regression/linear_regression_normal_equation/
python example.py
🛤️ Suggested Learning Path If you're starting fresh, follow this order. Each builds on the last.
Phase 1 — Foundations (do first)
utils/ → Linear Regression (Normal Eq) → Linear Regression (GD) → Polynomial Regression
Phase 2 — Core Supervised Learning
Logistic Regression → KNN → Naive Bayes → Perceptron → Decision Tree
Ridge → Lasso → Elastic Net
Bagging → Random Forest → AdaBoost → Gradient Boosting
K-Means → DBSCAN → PCA → GMM
Phase 6 — Neural Networks
Single Perceptron → MLP → CNN → RNN → LSTM
Phase 7 — Advanced & Interview Prep
SVM → t-SNE → Optimizers (SGD → Adam) → Recommenders → RL
🔑 Rules of This Repo Rule Why No ML library in algorithm code The whole point is learning what's inside the box numpy is allowed Vectorized math is how real implementations work scikit-learn only in example.py For loading data and validating our results Every algo has a README Code without explanation teaches nothing Every algo has interview Q&A Make your study immediately useful
🤝 Contributing
Fork the repo
Pick an algorithm marked ⬜ in the index
Follow the folder template exactly (README.md, implementation, example.py)
Ensure python example.py runs and prints output
Open a PR
See CONTRIBUTING.md for detailed guidelines.
📜 License MIT — use it, learn from it, build on it.
