facial-landmark-dnn

Facial landmark detection using Convolutional Neural Networks for Mobile Device

[ Requirements ]

• tensorflow==1.5.0
• Keras==2.1.3
• coremltools==0.8

[ Data Preprocessing ]

• Raw-Data

  1. 300W
  2. 300VW
  3. Ibug
  4. Our own Korean face dataset

• Dataset in Google Drive: Image Dataset Landmarks Dataset

# Example: Load Dataset
X = np.load(PATH + "basic_dataset_img.npz")
y = np.load(PATH + "basic_dataset_pts.npz")
X = X['arr_0']
y = y['arr_0'].reshape(-1, 136)

[ Modeling ]

• VGG_like_model
• Mobilenet_based_model

[ Result ]

• Face detector: opencv ssd detector
• Facial Landmark detector: Mobilenet based model

Youtube Video 1 Youtube Video 2

[ Converter for Mobile ]

• IOS - Keras to CoreML

• Android (TODO)

  1. Import model in Tensorflow
     • Convert Keras to Tensorflow use this code
     • Build Android app and call tensflow. check this tutorial and this official demo from google to learn how to do it.
  2. Import model in Java
     • deeplearning4j a java library allow to import keras model: tutorial link
     • Use deeplearning4j in Android: it is easy since you are in java world. check this tutorial

[ Reference ]

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Yinguobing github