中文版本

TengineKit - Free RealTime Face Landmarks 212 Points For Mobile

TengineKit, developed by OPEN AI LAB.
TengineKit is an easy-to-integrate face detection and face landmarks SDK. At present, it can run on various mobile phones at very low latency.

Effect

DEMO

Gif

Video( YouTube | BiliBili )

Have a try

• Apk can be directly downloaded and installed on the phone to see the effect.

or

• scan code to download apk

Goals

• Provide best performance in mobile client
• Provide the simplest API in mobile client
• Provide the smallest package in mobile client

Features

• face detection
• face landmarks
• face attributes like age, gender, smile, glasses

Performance

Release

The latest release is available on Maven Central.

    implementation 'com.tengine.android:tenginekit:1.0.2'

Usage

...

import com.tenginekit.FaceManager;

public class CameraActivity extends AppCompatActivity implements Camera.PreviewCallback{
    private static final String TAG = "CameraActicity";

    // camera preview width
    protected int previewWidth;
    // camera preview height
    protected int previewHeight;
    // content display screen width
    public static float ScreenWidth;
    // content display screen height
    public static float ScreenHeight;

    // nv21 data from camera
    protected byte[] mNV21Bytes;

    ...

    public void Init() {
        mNV21Bytes = new byte[previewHeight * previewWidth];

        /**
         * init
         * */
        FaceManager.GetInstance().init(this,previewWidth, previewHeight, ScreenWidth, ScreenHeight,FaceManager.ImageFormat.YUV_NV21);
    }

    /**
     * Callback for android.hardware.Camera API
     */
    @Override
    public void onPreviewFrame(final byte[] bytes, final Camera camera) {
        if (isProcessingFrame) {
            return;
        }
        isProcessingFrame = true;
        try {
            if (mNV21Bytes == null) {

                ...

                Camera.Size previewSize = camera.getParameters().getPreviewSize();
                previewHeight = previewSize.height;
                previewWidth = previewSize.width;
                Init();
                
                ...
            }
        } catch (final Exception e) {
            MyLogger.logError(TAG, "onPreviewFrame: " + e);
            return;
        }
        
        processImage();
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
        /**
         * release
         * */
        FaceManager.GetInstance().release();
    }


   

    protected void processImage() {

        ...

        FaceInfo[] faceInfos = FaceManager.GetInstance().getFaceInfo(mNV21Bytes);

        if (faceInfos != null && faceInfos.length > 0) {
            Rect[] face_rect = new Rect[faceInfos.length];
            List<List<LandmarkInfo>> face_landmarks = new ArrayList<>();
            for (int i = 0; i < faceInfos.length; i++) {
                face_rect[i] = faceInfos[i].faceRect;
                face_landmarks.add(faceInfos[i].landmarks);
            }

            // do something with face_rect, face_landmarks
        }

        ...

    }
}

System

Android

• Min Sdk Version 19

API

The located Function under com.tenginekit.FaceManager.

init

Parameter

• context: context in activity，
• preview_w: Camera preview width
• preview_h: Camera preview height
• screen_w: Screen width
• screen_h: Screen height
• format: Data format(YUV, RGB)

    FaceManager.getInstance().init(Context context, int preview_w, int preview_h, float screen_w, float screen_h, ImageFormat format);

    enum  ImageFormat
    {
        YUV_NV21,
        RGB,
        RGBA,
        GRAY
    }

get face info

Parameter

• imageData: Input data

    FaceInfo[] info = FaceManager.getInstance().getFaceInfo(byte[] imageData);

set rotation

Parameter

• ori: Rotation angle (vertical to 0°)
• is_screen_rotate: whether the display area follows rotation
• screen_w: screen width
• screen_h: screen height

    FaceManager.getInstance().setRotation(int ori, boolean is_screen_rotate, float screen_w, float screen_h);

switch camera

Parameter

• back:Whether it is a rear camera

    FaceManager.getInstance().switchCamera(boolean back);

turn on other function

Parameter

• FuncName: Function name
• state: Whether the additional function type is enabled

    FaceManager.getInstance().changeFuncState(ExtraFunc FuncName, boolean state);

    enum ExtraFunc
    {
        Attribution, // face attrubution Function
    }

release

    FaceManager.getInstance().release();

Data structure

• FaceInfo

• FaceInfo -- Attribution

• LandmarkInfo

Landmark Points Order

LandmarksOrder

Permission

<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.INTERNET"/>

<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_PHONE_STATE"/>

<uses-permission android:name="android.permission.CAMERA"/>

Sample

Android code under the "sample/Android" folder.

Access Guide

In setRotation of TengineKit Api, there are two parameters ori and is_screen_rotate, which are the rotation angle and whether to follow the screen rotation. Whether the android:screenOrientation parameter in the Manifest follows the screen can be set. Not setting this parameter is to follow the screen rotation.

Process

1.Device preview

This part is to get data from Camera, as the SDK input.

2.Angle

We use the vertical screen as an angle of 0 degrees. Since the data collected by the Android camera always deviates by 90, it is necessary to set + (-90) when setting the ori parameter. The actual rotation angle of Android is to add a function through the sensor Calculate to get. For details, see the example in the Demo project.

3.Rendering

When rendering, it is rendered at an angle of 0°, which is the normal output that people see under normal circumstances. The Android part has Canvas and Opengl rendering. Using Opengl rendering can make your apk better.

Contact

About the use of TengineKit and face-related technical exchanges, you can join the following QQ groups(Group Answer:TengineKit):

• TengineKit communication QQ group: 630836519
• Scan to join group