Clear Face

Clear Face is python project with C++/Python libraries for facial landmark detection(in 2d and 3d), tracking faces and multiple models detection from faces such as:

• Gender, Expressions, Illumination, Pose, Occlusion, Age, and Makeup.

Content Tabel

• Download and Setup

• Running Options

• Gender Expressions and other models

• Face Tracking

• Facial Landmark Detection

• Credits

Requerments

• Python 3.*
• Cmake
• C++
• make commad

Download and setup

Download Repo

git clone https://github.com/DiaaZiada/ClearFace.git

cd ClearFace

Virtual Environment

# create virtual environment
python -m venv ./venv

# activate virtual environment
source venv/bin/activate

Install Requirements

pip install -r requirements.txt

Download libtorch for c++ (CPU)

./download.sh

note: this command will download and extract the libarary

Build all c++ files

./build.sh

Running Options

to use ClearFace execute run.py file with various options

usage: run.py [-h] [--show] [--_2d] [--_3d] [--_2d3d] [--tracking]
              [--delay DELAY] [--inputs_path INPUTS_PATH]
              [--video_name VIDEO_NAME] [--outputs_path OUTPUTS_PATH]
              [--models_path MODELS_PATH] [--cam_num CAM_NUM]
              [--models MODELS [MODELS ...]]

Clear Face is python project with C++/Python libraries for facial landmark detection(in 2d and 3d), tracking faces and multiple models detection from faces such as:
 - Gender, Expressions, Illumination, Pose, Occlusion, Age, and Makeup.

optional arguments:
  -h, --help            show this help message and exit
  --show                set this parameter to True value if you want display
                        images/videos while processing, default is False
  --_2d                 set this parameter to True value if you 2d landmarks
                        from from the 2d model, default is False
  --_3d                 set this parameter to True value if you 3d landmarks
                        from from the 3d model, default is False
  --_2d3d               set this parameter to True value if you 2d landmarks
                        from from the 3d model, default is False
  --tracking            set this parameter to True value if you want tracking
                        faces in images/videos, default is False
  --delay DELAY         amount of seconds to wait to switch between images
                        while show the precess
  --inputs_path INPUTS_PATH
                        path for directory contains images/videos to process,
                        if you don't use it webcam will open to start the
                        record
  --video_name VIDEO_NAME
                        name of recorded video from camera
  --outputs_path OUTPUTS_PATH
                        path for directory to add the precesses images/videos
                        on it, if you don't use it output directory will
                        created and add the precesses images/videos on it
  --models_path MODELS_PATH
                        path for directory contains pytorch model
  --cam_num CAM_NUM     number of camera to use it
  --models MODELS [MODELS ...]
                        first index refers to gender model, second index
                        refers to expression model, and third index refers to
                        multiple models

Examples for process images/videos in directory $python run.py --inputs_path /path/to/inputs_dir --tracking --show --_2d3d --_3d
to use webcam $python run.py ---tracking --show --_2d3d --_3d

Gender Expressions and other models

1. Gender : Male, Female
2. Expressions : Anger, Happiness, Sadness, Surprise, Fear, Disgust
3. Illumination : Bad, Medium, High
4. Pose : Frontal, Left, Right, Up, Down
5. Occlusion : Glasses, Beard, Ornaments, Hair, Hand, None, Others
6. Age : Child, Young, Middle and Old
7. Makeup : Partial makeup, Over-makeup

Datasets

• IMDB ~ 500K image
• WIKI ~ 60K image
• FER ~ 35K image
• IMFDB ~ 3K image
• JAFFE ~ 250 image

Models

models in this project are based on Real-time Convolutional Neural Networks for Emotion and Gender Classification paper model architecture:

this part of the project consist of 3 models:

1. Gender
2. Expression
3. Multiple

Gender Model

trained done by using IMDB , WIKI, IMFDB datasets. consist of ~ 600 K image, and by using tencrop data augmentation dataset increased to be ~ 6 M image Accuracy ~ 78% using only 6 epochs and it will reach higher accuracy expected to be ~ 96 %

Expression Model

trained done by using FER , IMFDB, JAFFE image datasets. consist of ~ 40 K image, and by using tencrop data augmentation dataset increased to be ~ 400 K image Accuracy ~ 60% using 15 epochs and it will reach higher accuracy expected to be ~ 66 %

Multiple Models

this model is little bit different form Gender & Expression models in this model we use one feature extractor model and 5 different classifiers each classifier predict specific features form faces and they are:

• Illumination : Bad, Medium, High
• Pose : Frontal, Left, Right, Up, Down
• Occlusion : Glasses, Beard, Ornaments, Hair, Hand, None, Others
• Age : Child, Young, Middle and Old
• Makeup : Partial makeup, Over-makeup

trained done by using IMFDB image datasets consist of ~ 3 K image, and by using tencrop data augmentation dataset increased to be ~ 30 K image Accuracy ~ 77% using 15 epochs

Train

all training process done on notebook using Google Colab cloud

Convert to TorchScript

we convert ower models to torchscript to load them in c++ and use them as extention of ClearFace all convertions about this part will found in this notebook

Face Tracking

we used the centroid tracking algorithm

Facial Landmark Detection

DataSet

kaggle youtube faces with facial keypoints ~ 150K image

Train

all training process for the 2d and 3d model done in this notebook using Google Colab cloud

Pre-Trained Models

You can email me on diaa.elsayedziada@gmail.com to get the pre-trained models

Credits

• Real-time Convolutional Neural Networks for Emotion and Gender Classification paper
• Simple object tracking with OpenCV by pyimagesearch
• Extending TorchScript with Custom C++ Operators
• CMake Tutorial
• Ryan Reynolds and Jake Gyllenhaal interview for LIFE, DEADPOOL - UNCENSORED Youtube
• 1adrianb face alignment repo