Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.

A flexible, high-performance serving system for machine learning models

The open big data serving engine. https://vespa.ai

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

Serve, optimize and scale PyTorch models in production

⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.

Database system for building simpler and faster AI-powered applications

TensorFlow template application for deep learning

DELTA is a deep learning based natural language and speech processing platform.

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

Generic and easy-to-use serving service for machine learning models

A unified end-to-end machine intelligence platform

A scalable inference server for models optimized with OpenVINO™

Ray Aviary - evaluate multiple LLMs easily

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

ML pipeline orchestration and model deployments on Kubernetes, made really easy.

[Machine Learning Engineer Basic Guide] 부스트캠프 AI Tech - Product Serving 자료

MLOps Platform