English | 简体中文

⚡️FastDeploy is an accessible and efficient deployment Development Toolkit. It covers 🔥critical CV、NLP、Speech AI models in the industry and provides 📦out-of-the-box deployment experience. It covers image classification, object detection, image segmentation, face detection, face recognition, human keypoint detection, OCR, semantic understanding and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform .

Image Classification	Object Detection	Semantic Segmentation	Potrait Segmentation
Image Matting	Real-Time Matting	OCR	Face Alignment
Pose Estimation	Behavior Recognition	NLP	Speech
			input:Life was like a box of chocolates, you never know what you're gonna get. output:

📣 Recent Updates

• 🔥 【Live Preview】2022.11.09 20:30～21:30，《Covering the full spectrum of cloud-side scenarios with 150+ popular models for rapid deployment》
• 🔥 【Live Preview】2022.11.10 20:30～21:30，《10+ AI hardware deployments from Rockchip, Amlogic, NXP and others, straight to industry landing》
• 🔥 【Live Preview】2022.11.10 19:00～20:00，《10+ popular models deployed in RK3588, RK3568 in action》
  • Slack：Join our Slack community and chat with other community members about ideas
  • WeChat：Scan the QR code below using WeChat, follow the PaddlePaddle official account and fill out the questionnaire to join the WeChat group

• 🔥 2022.10.31：Release FastDeploy release v0.5.0
  

  • 🖥️ Data Center and Cloud Deployment: Support more backend, Support more CV models
    • Support Paddle Inference TensorRT, and provide a seamless deployment experience with other inference engines include Paddle Inference、Paddle Lite、TensorRT、OpenVINO、ONNX Runtime；
    • Support Graphcore IPU through paddle Inference;
    • Support tracking model PP-Tracking and RobustVideoMatting model；
    • Support one-click model quantization to improve model inference speed by 1.5 to 2 times on CPU & GPU platform. The supported quantized model are YOLOv7, YOLOv6, YOLOv5, etc.

• 🔥 2022.10.24：Release FastDeploy release v0.4.0
  

  • 🖥️ Data Center and Cloud Deployment: end-to-end optimization, Support more CV and NLP model
    • end-to-end optimization on GPU, YOLO series model end-to-end inference speedup from 43ms to 25ms;
    • Support CV models include PP-OCRv3, PP-OCRv2, PP-TinyPose, PP-Matting, etc. and provides end-to-end deployment demos;
    • Support information extraction model is UIE, and provides end-to-end deployment demos;
    • Support TinyPose and PicoDet and TinyPosePipeline deployment.
  • 📲 Mobile and Edge Device Deployment: support new backend，support more CV model
    • Integrate Paddle Lite and provide a seamless deployment experience with other inference engines include TensorRT、OpenVINO、ONNX Runtime、Paddle Inference；
    • Support Lightweight Detection Model and classification model on Android Platform，Download to try it out.
  • Web-Side Deployment: support more CV model
    • Web deployment and Mini Program deployment New OCR and other CV models capability.

Contents

• 📖 Tutorials（click to fold）

  • Install
    • How to Install FastDeploy Prebuilt Libraries
    • How to Build and Install FastDeploy Library on GPU Platform
    • How to Build and Install FastDeploy Library on CPU Platform
    • How to Build and Install FastDeploy Library on IPU Platform
    • How to Build and Install FastDeploy Library on Nvidia Jetson Platform
    • How to Build and Install FastDeploy Library on Android Platform
  • A Quick Start - Demos
    • Python Deployment Demo
    • C++ Deployment Demo
    • A Quick Start on Runtime Python
    • A Quick Start on Runtime C++
  • API (To be continued)
    • Python API
    • C++ API
  • Performance Optimization
    • Quantization Acceleration
  • Frequent Q&As
    • 1. How to Change Inference Backends
    • 2. How to Use FastDeploy C++ SDK on Windows Platform
    • 3. How to Use FastDeploy C++ SDK on Android Platform(To be Continued)
    • 4. Tricks of TensorRT
    • 5. How to Develop a New Model(To be Continued)
  • More FastDeploy Deployment Module
    • deployment AI Model as a Service
    • Benchmark Testing

• 🖥️ Data Center and Cloud Deployment
  • A Quick Start for Python SDK
  • A Quick Start for C++ SDK
  • Supported Data Center and Cloud Model List
• 📲 Mobile and Edge Device Deployment
  • Paddle Lite NPU Deployment
  • Supported Mobile and Edge Model List
• Web and Mini Program Deployment
  • Supported Web and Mini Program Model List
• Community
• Acknowledge
• License

🖥️ Data Center and Cloud Deployment

pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2

pip install fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg

# For deployment of GPU/TensorRT, please refer to examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision


model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                 "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                 "ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im)
print(result)

vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)

wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg

// For GPU/TensorRT deployment, please refer to examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"

int main(int argc, char* argv[]) {
  namespace vision = fastdeploy::vision;
  auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                          "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                          "ppyoloe_crn_l_300e_coco/infer_cfg.yml");
  auto im = cv::imread("000000014439.jpg");

  vision::DetectionResult res;
  model.Predict(&im, &res);

  auto vis_im = vision::VisDetection(im, res, 0.5);
  cv::imwrite("vis_image.jpg", vis_im);
  return 0;
 }

For more deployment models, please refer to Vision Model Deployment Examples .

Supported Data Center and Web Model List🔥🔥🔥🔥🔥

Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Available;

📲 Mobile and Edge Device Deployment

Paddle Lite NPU Deployment

• Rockchip-NPU / Amlogic-NPU / NXP-NPU

Supported Mobile and Edge Model List 🔥🔥🔥🔥

Web and Mini Program Deployment

Community

• If you have any question or suggestion, please give us your valuable input via GitHub Issues
• Join Us👬：
  • Slack：Join our Slack community and chat with other community members about ideas
  • WeChat：join our WeChat community and chat with other community members about ideas

Acknowledge

We sincerely appreciate the open-sourced capabilities in EasyEdge as we adopt it for the SDK generation and download in this project.

License

FastDeploy is provided under the Apache-2.0.