Skip to content

open-delivery-spec/cli

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

94 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ODS CLI

Zero-config AI code quality gate for teams using Claude Code, Copilot, or Cursor. These tools already stamp Co-Authored-By trailers on every commit, so ODS attributes AI-generated code automatically in CI — then analyzes quality, scores technical debt, and enforces policy on every PR. No disclosure forms, no manual tagging.

CI Go Version License


The Problem

A PR arrives: 8 commits, 6 written by Copilot, 2 by a human. The branch says feature/add-sarif-output. But two changed files touch the authentication module — nothing to do with SARIF. The reviewer doesn’t know. The merge happens. A bug ships.

This is the new reality of AI-assisted development. AI code increases technical debt in predictable ways:

AI Failure Mode Real-world impact
Hallucinated APIs AI invents functions, packages, and endpoints that don’t exist
Redundant error handling AI over-defends: 3+ identical if err != nil blocks in the same function
Over-commenting AI writes 35%+ comment-to-code ratio with self-explanatory comments
No test coverage AI PRs average 22% test coverage vs 68% for human PRs
Invisible AI code Teams can’t distinguish AI-generated from human-written changes
Scope drift AI changes files unrelated to the stated feature

ODS is the CI gate that detects AI code, analyzes its quality, scores technical debt impact, and enforces enterprise policy — on every pull request.


In Production

ODS runs on every PR in the open-delivery-spec org (dogfooding):

ODS on spec ODS on cli ODS on validate-action

See ADOPTERS.md for the full list and pending external adoption.


Quick Start

# Install
go install github.com/open-delivery-spec/cli/cmd/ods@latest

# Detect AI code in a PR
ods detect --diff-base origin/main --branch feature/my-feature

# Analyze code quality
ods analyze --json

# Score technical debt
ods score

# Enforce policy
ods check

Command Reference with Real Output

ods detect — AI Code Attribution

Attributes AI-generated code using Co-Authored-By trailers, the Linux kernel's Assisted-by: trailers, commit AI-assisted: footers, branch names, PR disclosure, and diff heuristics.

Co-Authored-By trailers emitted by Claude Code, GitHub Copilot, and Cursor are the primary signal — no additional configuration needed. The Linux kernel coding-assistants convention is recognized as a first-class disclosure with the same confidence:

Assisted-by: Claude:claude-3-opus coccinelle sparse

parses the agent (Claude) and model version (claude-3-opus) into the evidence; the trailing analysis-tool list is not attribution and is ignored. A bare Assisted-by: Claude without the model also counts.

Repos using git-ai get the highest-fidelity signal: its authorship logs under refs/notes/ai (Git AI Standard v3) record which lines each agent wrote. When notes are present on commits in the diff range, per-file AI line counts are measured from them instead of estimated by the diff heuristics, and the evidence names the agent and model (AI-assisted commit a1b2c3d (git-ai: 6 AI line(s), cursor/claude-sonnet-4-5)). AI lines are capped at each file's changed lines so authorship recorded outside the change can't inflate the ratio. Nothing changes on repos without git-ai. Note for CI: git notes aren't fetched by default — run git fetch origin +refs/notes/ai:refs/notes/ai after checkout.

This is attribution from signals the tools (or authors) volunteer, not forensic detection: stripping the trailer evades it, and the diff heuristics are only a low-confidence fallback.

$ ods detect --diff-base origin/main --branch feature/ai-oauth
🤖  AI code detected — 85% confidence (PR shows AI disclosure)
   Sources: pr-body
   Evidence:
     • [pr-body] AI disclosure checkbox is checked (85%)
$ ods detect --diff-base origin/main --branch feature/add-login
👤  No AI code detected (0% confidence)

JSON output for CI pipelines:

$ ods detect --diff-base origin/main --json
{
  "ai_generated": true,
  "confidence": 0.85,
  "summary": "AI code detected — 85% confidence (PR shows AI disclosure)",
  "sources": ["pr-body"],
  "evidence": [
    {
      "source": "pr-body",
      "value": "AI disclosure checkbox is checked",
      "confidence": 0.85
    }
  ],
  "files": [
    {
      "path": "internal/scanner/sarif.go",
      "ai_lines": 180,
      "total_lines": 195,
      "confidence": 0.92
    }
  ]
}
Flag Default Description
--diff-base HEAD~1 Git ref to diff against
--branch auto Branch name
--pr-body PR description body text
--pr-file File containing PR body
--commits 10 Max commits to scan
--json false JSON output
--format summary Output format: summary, detail, json

ods analyze — AI Code Quality Analysis

$ ods analyze --file internal/scanner/sarif.go --json
{
  "issues": [
    {
      "file": "internal/scanner/sarif.go",
      "line": 42,
      "rule": "ai-over-commenting",
      "severity": "info",
      "message": "Comment-to-code ratio is 47%",
      "suggestion": "If comments restate the code, prefer explaining why over what; documentation comments are fine"
    },
    {
      "file": "internal/scanner/sarif.go",
      "line": 88,
      "rule": "ai-inconsistent-pattern",
      "severity": "medium",
      "message": "Mixed naming conventions in the same file",
      "suggestion": "Standardize on one naming convention; run gofmt/prettier"
    }
  ],
  "total_lines": 195,
  "summary": "2 quality issues found (0 critical, 0 high, 1 medium, 0 low, 1 info)"
}
Flag Default Description
--file, -f Analyze a single file
--dir, -d Analyze a directory (recursively)
--ai-only false Only files detected as AI-generated
--json false JSON output
--format summary Output format: summary, detail, json

ods score — Technical Debt Impact

$ ods score
⚠️  Technical Debt Score
   +4.2 (increase)
   Verdict: increase (Moderate risk: review recommended, ensure adequate tests)
$ ods score --json
{
  "technical_debt_delta": 4.2,
  "verdict": "increase",
  "recommendation": "Moderate risk: review recommended, ensure adequate tests",
  "breakdown": {
    "ai_code_ratio": 0.75,
    "defect_density": 1.2,
    "critical_issues": 0,
    "test_coverage": 0.3,
    "duplication_rate": 0.1
  }
}
Flag Default Description
--json false JSON output
--format summary Output format: summary, detail, json
--test-dir Test directory path (auto-detected)
--sarif SARIF file whose findings are merged into the score

ods check — Enterprise Policy Enforcement

$ ods check
✅  Policy check passed
   Policy: .ods/policy.rego
$ ods check --json
{
  "allowed": false,
  "denials": ["AI code with low test coverage"],
  "warnings": ["High-confidence AI code with multiple quality issues"]
}
Flag Default Description
--policy, -p .ods/policy.rego Path to Rego policy file
--json false JSON output
--sarif SARIF file whose findings are merged into the policy input

Review routing: review_tier

Beyond allow/deny, a policy can answer a second question: how much human attention does this change need? Define a review_tier rule returning one of auto (low risk — eligible for expedited review or auto-merge), standard (normal review, the default), or elevated (high risk — request extra reviewers):

default review_tier := "standard"

review_tier := "auto" {
    input.technical_debt_delta <= 1.0
    not has_high_or_critical
}

review_tier := "elevated" {
    input.ai_generated == true
    has_high_or_critical
}

The tier is reported in the text output and as "review_tier" in --json. Semantics: deny always wins — a blocked PR is never routed; the tier is an advisory routing signal for changes that may merge, and it never affects the exit code. Policies that define no review_tier behave exactly as before (consumers should treat the absent tier as standard). An unknown tier value falls back to standard with a warning instead of failing the gate. ods init scaffolds these rules (with explanatory comments) into new policies.

ods hook install — Git Hooks

$ ods hook install
✅  pre-commit hook installed at .git/hooks/pre-commit
✅  prepare-commit-msg hook installed at .git/hooks/prepare-commit-msg
✅  pre-push hook installed at .git/hooks/pre-push

ods init — Project Scaffolding

$ ods init
  ✅ Created: .github/workflows/ods-ai-quality.yml
  ✅ Created: .ods/policy.rego

── ODS initialized ──

Next steps:
  1. Edit .ods/policy.rego to add custom enforcement rules
  2. Install git hooks:  ods hook install
  3. Commit and push — ODS will run on your next PR

init is idempotent — existing files are skipped, never overwritten.

ods rules — Rule Catalogue

$ ods rules
ODS Analysis Rules (4)

🔴 [high] ai-unsafe-deserialization
  json.Unmarshal into interface{} without type validation — AI commonly skips type checking.
  → Use a concrete struct type or validate the unmarshalled data before use.
...

ods rules --json emits the machine-readable catalogue (id, name, description, default severity, category, suggestion). It is the single source of truth for every rule the analyzer can emit.

ods report — AI Attribution Report

A governance view over recent history: how much delivered work is AI-assisted, and trending which way. Attribution comes from the Co-Authored-By trailers AI tools emit automatically and the kernel-style Assisted-by: trailers — the same signals as ods detect. Assisted-by commits aggregate under their agent name in the per-tool breakdown.

$ ods report --since "90 days ago"
ODS AI Attribution Report — since 90 days ago

  Commits:        64 total · 5 AI-assisted (8%) · 59 human
  Changed lines:  30056 total · 1697 AI-assisted (6%)

  By tool:
    Claude               3 commit(s)
    Claude Sonnet 4.6    2 commit(s)

64 commit(s): 5 AI-assisted (8%), 59 human — AI touched 6% of changed lines
Flag Default Description
--since 90 days ago History window (any git --since expression)
--max-commits 0 Cap commits scanned (0 = no cap)
--json false Machine-readable output (commit/line shares, per-tool counts)

This is attribution, not forensic detection: it counts what the tools disclose. Coverage/quality history is not reconstructable from git alone, so the report focuses on the signals git carries reliably — AI share of commits and churn.


Debugging

Every command accepts a global --debug flag (or set ODS_DEBUG=1) to print decision diagnostics to stderr. JSON written to stdout stays clean, so --debug is safe to combine with --json in pipelines.

$ ods check --json --debug
[ods:debug] check: diff base = HEAD~1
[ods:debug] check: no policy file found, using built-in default policy
[ods:debug] check: detection ai_generated=true confidence=0.90 sources=[commit-trailer]
[ods:debug] check: analysis issues=0 (changed lines=2, test lines=0)
[ods:debug] check: coverage source=unknown value=-1.00
[ods:debug] check: score delta=0.00 verdict=decrease (ai_ratio=0.00 ...)
[ods:debug] check: policy result allowed=true denials=0 warnings=0
{
  "allowed": true
}

This answers "why did this PR pass/block?" by exposing the detection signals, score breakdown, coverage source, and every policy denial/warning.


Installation and CI Integration

Install

go install github.com/open-delivery-spec/cli/cmd/ods@latest

Requires Go 1.25+.

GitHub Actions (Recommended)

Use the one-step validate-action:

name: ODS AI Code Quality
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  ods:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - uses: open-delivery-spec/validate-action@v1

Or use individual CLI steps:

- name: Detect AI code
  run: ods detect --diff-base origin/main --branch ${{ github.head_ref }} --json

- name: Analyze quality
  run: ods analyze --json

- name: Score tech debt
  run: ods score --json

- name: Enforce policy
  run: ods check --json

See Also


License

Apache License 2.0

About

Reference CLI tool for Open Delivery Spec — validate and generate delivery artifacts

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors