Loading repository data…
taldatech / repository
Jupyter Notebook tutorials for ECE 046211 Deep Learning course at the Technion
Jupyter Notebook tutorials for the Technion's ECE 046211 course "Deep Learning"
| File | Topics Covered | Video |
|---|---|---|
Setting Up The Working Environment.pdf | Guide for installing Anaconda locally with Python 3 and PyTorch, integration with PyCharm and using GPU on Google Colab | - |
ee046211_tutorial_01_machine_learning_recap.ipynb/pdf | Supervised and Unsupervised Learning, Model Evaluation, Bias-Variance Tradeoff, Feature Scaling, Linear Regression, Gradient Descent, Regularization (Ridge, LASSO) | Video Link |
ee046211_tutorial_02_single_neuron_recap.ipynb/pdf | Discriminative models, Perceptron, Logistic Regression (also in PyTorch), Softmax Regression, Activation functions | Video Link |
ee046211_tutorial_03_optimization_gradient_descent.ipynb/pdf | Unimodal functions, Convexity, Hessain, Gradient Descent, SGD, Learning Rate, LR Scheculing / Annealing, Momentum, Nesterov Momentum, Adaptive Learning Rate Methods, Adagrad, RMSprop, Adam, AdaBelief, MADGRAD, Adan, Schedule-free Optimization (SGD, Adam) | Video Link - Part 1Video Link - Part 2 |
ee046211_tutorial_04_differentiation_autograd.ipynb/pdf | Lagrange Multipliers, Automatic Differentiation (AutoDiff) Forward Mode and Reverese Mode, PyTorch Autograd | Video Link |
ee046211_tutorial_05_multilayer_nn.ipynb/pdf | Multi-Layer Perceptron (MLP), Backpropagation, Neural Netwroks in PyTorch, Weights Initialization - Xavier (Glorot), Kaiming (He), Deep Double Descent | Video Link |
ee046211_tutorial_06_convnets_visual_tasks.ipynb/pdf | 2D Convolution (Cross-correlation), Convolution-based Classification, Convolutional Neural Networks (CNNs), Regularization and Overfitting, Dropout, Data Augmentation, CIFAR-10 dataset, Visualizing Filters, Applications of CNNs, The problems with CNNs (adversarial attacks, poor generalization, fairness-undesirable biases) | Video Link - Part 1Video Link - Part 2 |
ee046211_tutorial_07_sequential_tasks_rnn.ipynb/pdf | Sequential Tasks, Natural Language Processing (NLP), Language Model, Perplexity, BLEU, Recurrent Neural Network (RNN), Backpropagation Through Time (BPTT), Long Term Short Memory (LSTM), Gated Recurrent Unit (GRU), RWKV, xLSTM, Multi-head Self-Attention, Transformer, BERT and GPT, Teacher Forcing, torchtext, Sentiment Analysis, Transformers Warmup, Intialization, GLU variants, Pre-norm and Post-norm, RMSNorm, SandwichNorm, ReZero, Rectified Adam (RAdam), Relative Positional Encoding/Embedding | Video Link - Part 1Video Link - Part 2Video Link - Part 3 |
ee046211_tutorial_08_training_methods.ipynb/pdf | Feature Scaling, Normalization, Standardization, Batch Normalization, Layer Normalization, Instance Normalization, Group Normalization, Vanishing Gradients, Exploding Gradients, Skip-Connection, Residual Nlock, ResNet, DenseNet, U-Net, Hyper-parameter Tuning: Grid Search, Random Search, Bayesian Tuning, Optuna with PyTorch | Video LinkVideo Link - Optuna Tutorial |
ee046211_tutorial_09_self_supervised_representation_learning.ipynb/pdf | Transfer Learning, Domain Adaptation, Pre-trained Networks, Sim2Real, BERT, Low-rank Adaptation - LoRA, DoRA, Representation Learning, Self-Supervised Learning, Autoencoders, Contrastive Learning, Contrastive Predictive Coding (CPC), Simple Framework for Contrastive Learning of Visual Representations (SimCLR), Momentum Contrast (MoCo), Bootstrap Your Own Latent (BYOL), DINO, CLIP | Video Link - Part 1 - Transfer LearningVideo Link - Part 2 - Self-supervised Learning |
ee046211_tutorial_10_compression_pruning_amp.ipynb/pdf | Resource Efficiency in DL, Automatic Mixed Precision (AMP), Quantization (Dynamic, Static), Quantization Aware Training (QAT), LLM Quantization, Pruning, The Lottery Ticket Hypothesis | Video Link |
pytorch_maximize_cpu_gpu_utilization.ipynb/pdf | Tips and Tricks for efficient coding in PyTorch, Maximizing the CPU and GPU utilization, nvidia-smi, PyTorch Profiler, AMP, Multi-GPU training, HF Accelerate, RL libraries | Video Link |
You can view the tutorials online or download and run locally.
| Service | Usage |
|---|---|
| Jupyter Nbviewer | Render and view the notebooks (can not edit) |
| Binder | Render, view and edit the notebooks (limited time) |
| Google Colab | Render, view, edit and save the notebooks to Google Drive (limited time) |
Jupyter Nbviewer:
Press on the "Open in Colab" button below to use Google Colab:
Or press on the "launch binder" button below to launch in Binder:
Note: creating the Binder instance takes about ~5-10 minutes, so be patient
Press "Download ZIP" under the green button Clone or download or use git to clone the repository using the
following command: git clone https://github.com/taldatech/ee046211-deep-learning.git (in cmd/PowerShell in Windows or in the Terminal in Linux/Mac)
Open the folder in Jupyter Notebook (it is recommended to use Anaconda). Installation instructions can be found in Setting Up The Working Environment.pdf.
For the complete guide, with step-by-step images, please consult Setting Up The Working Environment.pdf
environment.yml file by running: conda env create -f environment.yml which will create a new conda environment named deep_learn. If you did this, you will only need to install PyTorch, see the table below.Anaconda Prompt from the start menu, in Mac/Linux open the terminal and run conda create --name deep_learn. Full guide at https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commandsAnaconda Prompt in Windows) and run conda activate deep_learn| Library | Command to Run |
|---|---|
Jupyter Notebook | conda install -c conda-forge notebook |
numpy | conda install -c conda-forge numpy |
matplotlib | conda install -c conda-forge matplotlib |
pandas | conda install -c conda-forge pandas |
scipy | conda install -c anaconda scipy |
scikit-learn | conda install -c conda-forge scikit-learn |
seaborn | conda install -c conda-forge seaborn |
tqdm | conda install -c conda-forge tqdm |
opencv | conda install -c conda-forge opencv |
optuna | pip install optuna |
pytorch (cpu) | conda install pytorch torchvision torchaudio cpuonly -c pytorch (get command from PyTorch.org) |
pytorch (gpu) | conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia (get command from PyTorch.org) |
torchtext | conda install -c pytorch torchtext |
torchdata | conda install -c pytorch torchdata + pip install portalocker |
jupyter notebook in the terminal (or Anaconda Prompt in Windows) while the deep_learn environment is activated.Selected from shared topics, language and repository description—not editorial ratings.
A curated list of Machine Learning and Deep Learning tutorials in Jupyter Notebook format ready to run in Google Colaboratory
lilyprojectwork /
(1) Use this walkthrough to setup Python with Keras in Google Drive: https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d (You don't have to read the whole thing---you might find it helpful/interesting to do so, but for now you only need to get up to the point where tensorflow 2.0 and keras are installed and imported.) (2) Put the attached two Jupyter notebook files in your Google drive colab folder and read through them line-by-line and try to understand what each command does. (There's no comments, so you might have to look up some of the commands, but most of them are self-explanatory so I think you'll be able to figure it out just by looking at the code.) If you're stuck/confused by any parts of it, ask me by email (I'll also quickly walk through it next week at the beginning of class). (3) Play around with the hyperparameters and neural net architecture (feel free to be creative and experiment and try different things even if you don't know how well they'll do!) and see what is the best validation accuracy you can acheive for a MLP network (fully connected layers of hidden neurons, the kind we've been studying before) and for a CNN network (the convolution kind we just started studying). HINT: you don't need to use all the code in these files for this, only the part up to the "history" where the model is trained and then the following command that computes the validation accuracy (all the code after that is just to randomly sample an image in the database and see how the model labels it). You don't have to turn anything in here, but we'll start class next week with a fun informal competition to see which student in the class got the highest accuracies for the two types (MLP and CNN) and then have the student(s) either describe their choices or share their screen and show the rest of us.
wkambale /
This repository contains the official source code and Jupyter notebooks for the "Google Colab in VS Code: A Deep Dive into the New Extension" tutorial.
A curated list of Machine Learning and Deep Learning tutorials in Jupyter Notebook format ready to run in Google Colaboratory
stevenalexander /
Jupyter notebook run on Google colab from "Deep learning turkey" tutorial
Naagar /
Here I have done some code on Jupyter notebook (google Colab) starting Zero to GAN and Normalization Flow DataSets - MNIST and CIFAR10