CognitiveOS inference engine — local LLM runtime for both Raw Model (firmware guardrail) and Wide Model (operational inference). Contains two binaries:

• coginfer — Ollama-compatible HTTP inference server for the Wide Model
• cograw — Root-level RPC server for the Raw Model (firmware GGUF guardrail)

cmd/coginfer             — Wide Model inference server (HTTP, Ollama-compatible)
cmd/cograw               — Raw Model RPC server (Unix socket, JSON-RPC 2.0)
internal/server/         — HTTP server with all API handlers
internal/llm/            — Backend interface + implementations (mock, cgo)
internal/model/          — Model scanning and .gguf metadata discovery

Exposes an Ollama-compatible HTTP API for the general-purpose Wide Model.

# Standard build (detects llama.cpp, falls back to mock)
make build

# With CGo (production — requires vendored llama.cpp build)
CGO_ENABLED=1 go build -tags=cgo -o bin/coginfer ./cmd/coginfer

# Without CGo (mock backend only, no llama.cpp needed)
CGO_ENABLED=0 go build -o bin/coginfer ./cmd/coginfer

# Start with mock backend (no llama.cpp needed)
./bin/coginfer --backend mock --models /cognitiveos/models

# Start with CGo llama.cpp backend (production)
./bin/coginfer --backend cgo --models /cognitiveos/models

# Custom port and log file
./bin/coginfer --addr 127.0.0.1:11434 --log /cognitiveos/logs/inference.log

• mock — Simulated token generation with delays. Default for development.
• cgo — In-process llama.cpp via CGo bridge. Requires CGO_ENABLED=1 and vendored llama.cpp build.

A small, always-on RPC server that acts as the firmware-level guardrail between the human and the Wide Model. Runs as root, communicates over a Unix socket via JSON-RPC 2.0.

Key constraint: cograw has no knowledge of MCP tools or registries — it is a pure guardrail GGUF. Tool routing is the daemon's responsibility.

For the authoritative specification, see raw-model.md in the product-specs repo.

make build    # Builds both coginfer and cograw
make test     # Run tests
make lint     # Run go vet
make clean    # Remove build artifacts

Requires CGo and vendored llama.cpp for production builds. See scripts/build.sh for the self-contained bootstrap.

./bin/cograw \
  --socket /cognitiveos/run/raw.sock \
  --model /cognitiveos/models/raw/raw-model.gguf

Detailed parameter and response schemas are in the product-specs RPC methods table.

Human → cli → cognitiveosd → cograw (validate) → Wide Model (generate) → cognitiveosd (parse, route MCP) → cli

1. Human input arrives at the daemon via CLI
2. Daemon sends the prompt to cograw for guardrail validation
3. If allowed, daemon forwards to Wide Model for inference
4. Daemon parses the Wide Model response and routes tool calls to MCP servers
5. Final output returned to CLI

cograw must start before cognitiveosd — the daemon hard-fails if the raw socket is unavailable at boot.

The raw model file is stored at /cognitiveos/models/raw/raw-model.gguf on a read-only squashfs partition:

• Owner: root:root
• Permissions: 0400
• The Wide Model has no read access
• Only a full firmware update (physical reflash or signed update) can change it

• Product Specs — authoritative specs for raw-model, architecture, API, and security model
• cognitiveosd — daemon that manages the Wide Model and routes tool calls
• CognitiveOS Project — GitHub organization

make build    # Compile to build/bin/cognitiveos-inference and build/bin/cograw
make test     # Run tests
make lint     # Run go vet
make clean    # Remove build artifacts

1. Branch from main
2. Use topic branches: feature/<name>, fix/<name>
3. Open a PR to main with a clear title and description
4. Merge after review

See the SDLC repo for the full contribution guide, code review standards, and testing strategy.

Jean Machuca — GitHub · Sponsor

MIT