This repo explores how to build GenAI apps locally using Docker Model Runner with step-by-step examples.

Run a simple LLM (e.g. llama2, mistral) locally using Docker Model Runner.

View Article ➜

A simple Python script that sends prompts to the running model and prints the response.

docker model pull ai/mistral
docker model run ai/mistral

cd docker-model-runner-llm-demo
pip install -r requirements.txt
python app/main.py

View Article ➜

Wrap your prompt logic in a FastAPI app and serve it as a local GenAI API using Docker Compose.

cd docker-llm-fastapi-app
docker compose up --build

Access the API at http://localhost:8000/generate

View Article ➜

Build a web UI with role-based templates using FastAPI + Jinja2. Run it locally with Docker and connect to Model Runner via OpenAI-compatible APIs.

cd docker-genai-ui
docker compose up --build

Open in browser: http://localhost:8000

⏳ Note: First LLM response may take 1–2 minutes to load due to model warm-up.

View Article ➜

How to Set Up Your Company for Success with Docker — standardize dev environments, improve security, and streamline team adoption of Docker.

• Docker Desktop v4.41+ with Model Runner enabled
• Enable host-side TCP support (Docker Settings → Beta Features → Model Runner → TCP)
• Enable experimental features in Docker Desktop
• Python 3.10+ for running the scripts

Stay tuned! 🚀