A RAG support chatbot — a pet project on the path to becoming an AI/LLM Application Engineer.
Answers questions based on a loaded knowledge base. Built incrementally: MVP → RAG → evals → agent → production.
| Python 3.12 + FastAPI | web framework + ASGI |
| Pydantic v2 | data validation and configuration |
| Anthropic SDK | Claude Haiku 4.5 (demo mode) |
| Ollama | llama3.2 locally (dev mode) |
| SlowAPI | rate limiting |
| SSE | response streaming without WebSocket |
| React 19 + TypeScript + Vite | UI |
| TanStack Query v5 | server state management |
| Zod v4 | runtime API response validation |
| eventsource-parser | SSE stream parsing |
| Feature-Sliced Design | architecture |
| CSS Modules | styling |
Model calls are behind a single LLMProvider interface. Switching is one variable in .env, no logic changes needed:
LLM_PROVIDER=ollama # local development, free
LLM_PROVIDER=claude # demo, Claude Sonnet 4.6
llm/
├── base.py # abstract LLMProvider class
├── claude.py # ClaudeProvider
├── ollama.py # OllamaProvider
└── factory.py # get_provider() — selects by env var
src/
├── app/ # global providers (QueryClient)
├── pages/ # composition only, no logic
├── features/
│ ├── chat/
│ │ ├── api/ # zod schemas, request functions
│ │ ├── hooks/ # useChat
│ │ ├── components/# ChatWidget, MessageList, ChatInput, MessageBubble
│ │ ├── types.ts # domain types (Message, etc.)
│ │ └── index.ts # public feature API
└── shared/
├── api/ # fetch client, getErrorMessage
├── config/ # env variables
├── lib/ # cn (classnames)
└── ui/ # Button
Layers import only downward: pages → features → shared. Boundaries are enforced by eslint-plugin-boundaries — a wrong import is a lint error.
support-bot/
├── Makefile
├── .pre-commit-config.yaml
├── backend/
│ ├── chat/
│ │ ├── router.py # HTTP endpoints
│ │ ├── schemas.py # Pydantic models
│ │ └── service.py # business logic
│ ├── llm/ # provider abstraction
│ ├── prompts/
│ │ └── system.txt # system prompt
│ ├── config.py # Pydantic Settings
│ ├── limiter.py # SlowAPI rate limiter setup
│ ├── main.py # FastAPI app, CORS, /health
│ ├── requirements.txt
│ └── requirements-dev.txt
└── frontend/
└── src/
├── app/
├── pages/
├── features/
└── shared/
make installCreates backend/.venv, installs Python packages from requirements-dev.txt, and runs npm install in frontend/.
backend/.env:
LLM_PROVIDER=ollama # or claude
OLLAMA_MODEL=llama3.2
# ANTHROPIC_API_KEY=sk-... # required only when LLM_PROVIDER=claude
ALLOWED_ORIGINS=["http://localhost:5173"]frontend/.env.local:
VITE_API_URL=http://localhost:8000make backend # FastAPI on :8000
make frontend # Vite dev server on :5173make setup-hooksRuns before every commit: ruff (lint + format check), mypy, eslint, and tsc.
| Command | What it does |
|---|---|
make backend |
Start FastAPI dev server |
make frontend |
Start Vite dev server |
make install |
Install all dependencies |
make lint |
ESLint + Ruff across the project |
make check |
mypy + tsc type checking |
make format |
Ruff format + Prettier |
make setup-hooks |
Install pre-commit hooks |
| Stage | Status | Description |
|---|---|---|
| MVP | ✅ done | document in system prompt, streaming chat |
| v1 — RAG | 🔜 next | chunking, embeddings, Supabase pgvector, retrieval |
| v2 — quality | ⏳ | hybrid search, reranking, evals (RAGAS / Promptfoo) |
| v3 — agent | ⏳ | tool use, agentic RAG |
| v4 — production | ⏳ | Langfuse tracing, guardrails, caching, Render deploy |
- Secrets live only in
.envand are never committed.ANTHROPIC_API_KEYstays on the backend only. - Architecture is designed for the full roadmap upfront, implemented incrementally — the stack does not change mid-way.
- Model calls go only through
llm/factory.py → LLMProvider, never directly to a provider. - Industry standard over the path of least resistance: the right library for the right job.