The Wayback Machine - https://web.archive.org/web/20231217070428/https://github.com/topics/llm-inference

#

llm-inference

Here are 193 public repositories matching this topic...

nomic-ai / gpt4all

gpt4all: open-source LLM chatbots that you can run anywhere

Updated Dec 16, 2023
C++

microsoft / autogen

Enable Next-Gen Large Language Model Applications. Join our Discord: https://discord.gg/pAbnFJrkgZ

chat chatbot gpt chat-application agent-based-framework agent-oriented-programming gpt-4 chatgpt llmops gpt-35-turbo llm-agent llm-inference llm-framework

Updated Dec 17, 2023
Jupyter Notebook

bentoml / OpenLLM

Operating LLMs in production

Updated Dec 16, 2023
Python

anything-llm

Mintplex-Labs / anything-llm

Open-source ChatGPT experience for both open and closed source LLMs, embedders, and vector databases. Unlimited documents, messages, and concurrent users with permission management in one app.

Updated Dec 16, 2023
JavaScript

mistralai / mistral-src

Reference implementation of Mistral AI 7B v0.1 model.

llm llm-inference mistralai

Updated Dec 16, 2023
Jupyter Notebook

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

nlp natural-language-processing ai computer-vision deep-learning transformers inference speech-recognition yolo recommendation-system performance-boost good-first-issue openvino diffusion-models stable-diffusion generative-ai llm-inference optimize-ai deploy-ai

Updated Dec 17, 2023
C++

SuperDuperDB / superduperdb

🔮 SuperDuperDB. Bring AI to your database; integrate, train and manage any AI models and APIs directly with your database and your data.

Updated Dec 17, 2023
Python

deepsparse

neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

nlp performance computer-vision inference machinelearning pruning object-detection pretrained-models quantization cpus onnx sparsification llm-inference deepsparse

Updated Dec 16, 2023
Python

liguodongiot / llm-action

本项目旨在分享大模型相关技术原理以及实战经验。

llm llmops llm-serving llm-training llm-inference

Updated Dec 14, 2023
Jupyter Notebook

liltom-eth / llama2-webui

Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.

llm llm-inference llama2 llama-2

Updated Nov 27, 2023
Jupyter Notebook

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

llama cuda-kernels deepspeed llm fastertransformer llm-inference turbomind internlm llama2 codellama

Updated Dec 16, 2023
C++

intel / intel-extension-for-transformers

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

cpu gpu chatbot xeon pc habana large-language-model chatpdf llm-inference 4-bits speculative-decoding llm-cpu streamingllm neurips2023 attention-sink intel-optimized-llamacpp neural-chat neural-chat-7b-v3-1 gaudi2

Updated Dec 17, 2023
C++

FasterDecoding / Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

llm llm-inference

Updated Dec 10, 2023
Jupyter Notebook

ray-project / ray-llm

RayLLM - LLMs on Ray

distributed-systems transformers ray serving large-language-models llm llmops llm-serving llm-inference

Updated Dec 6, 2023
Python

llmflows

stoyan-stoyanov / llmflows

LLMFlows - Simple, Explicit and Transparent LLM Apps

python machine-learning ai openai question-answering vector-database gpt-4 llm prompt-engineering llms chatgpt llmops llm-inference

Updated Oct 18, 2023
Python

predibase / lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

transformers pytorch llama gpt lora model-serving fine-tuning llm llmops llm-serving llm-inference

Updated Dec 16, 2023
Python

lean-dojo / LeanCopilot

LLMs as Copilots for Theorem Proving in Lean

machine-learning theorem-proving lean lean4 llm-inference

Updated Dec 13, 2023
C++

FlagAI-Open / Aquila2

The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.

llm llm-training llm-inference

Updated Dec 14, 2023
Python

anarchy-ai / LLM-VM

irresponsible innovation. Try now at https://chat.dev/

machine-learning deep-learning artificial-intelligence distillation distillation-model llm llm-agent llm-training llm-inference llm-local

Updated Dec 15, 2023
Python

ray-project / ray-educational-materials

This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.

deep-learning ray distributed-machine-learning ray-tune ray-train ray-distributed llm generative-ai ray-serve ray-data llm-serving llm-inference

Updated Dec 15, 2023
Jupyter Notebook

Improve this page

Add a description, image, and links to the llm-inference topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the llm-inference topic, visit your repo's landing page and select "manage topics."