inference-optimization

High-efficiency floating-point neural network inference operators for mobile, server, and Web

The Tensor Algebra SuperOptimizer for Deep Learning

BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

Batch normalization fusion for PyTorch

[MLSys 2021] IOS: Inter-Operator Scheduler for CNN Acceleration

Optimize layers structure of Keras model to reduce computation time

Graphsignal Python agent

A set of tool which would make your life easier with Tensorrt and Onnxruntime. This Repo is designed for YoloV3

cross-platform modular neural network inference library, small and efficient

Modified inference engine for quantized convolution using product quantization

Batch Partitioning for Multi-PE Inference with TVM (2020)

🤖️ Optimized CUDA Kernels for Fast MobileNetV2 Inference

A constrained expectation-maximization algorithm for feasible graph inference.

PyTorch Mobile: iOS examples

Batch estimation on Lie groups

Interface for TensorRT engines inference along with an example of YOLOv4 engine being used.

MIVisionX Python Inference Analyzer uses pre-trained ONNX/NNEF/Caffe models to analyze inference results and summarize individual image results

ncnn is a high-performance neural network inference framework optimized for the mobile platform

PyTorch Mobile: Android examples of usage in applications

A simple tool that applies structure-level optimizations (e.g. Quantization) to a TensorFlow model