Skip to content

Refactor log DB into LogWriter interface#19234

Merged
rasmusrygaard merged 9 commits into
mainfrom
dev/rasmus/add-feedback-log-sink
Apr 24, 2026
Merged

Refactor log DB into LogWriter interface#19234
rasmusrygaard merged 9 commits into
mainfrom
dev/rasmus/add-feedback-log-sink

Conversation

@rasmusrygaard
Copy link
Copy Markdown
Contributor

@rasmusrygaard rasmusrygaard commented Apr 23, 2026

Why

This prepares feedback log capture for a future remote app-server hook sink without changing the current local SQLite upload path. The important boundary is now intentionally small: a log sink is a tracing Layer that can also flush entries it has accepted.

That keeps the existing SQLite implementation simple while giving the upcoming gRPC sink a place to fit beside it. SQLite and gRPC have different worker/write semantics, so this PR avoids introducing a shared buffered-sink abstraction and instead lets each LogWriter own the buffering mechanics it needs.

What Changed

  • Added LogSinkQueueConfig with the existing local defaults: queue capacity 512, batch size 128, and flush interval 2s.
  • Added LogDbLayer::start_with_config(...) while preserving LogDbLayer::start(...) and log_db::start(...) defaults.
  • Introduced the LogWriter trait as the minimal shared interface: tracing_subscriber::Layer plus flush().
  • Made LogDbLayer implement LogWriter.
  • Kept tracing event formatting inside LogDbLayer; it still creates one LogEntry per tracing event before queueing it for SQLite.
  • Kept normal event capture best-effort and non-blocking via bounded try_send.

Behavior Notes

This does not change the SQLite schema, retention behavior, /feedback/upload, or Sentry upload behavior. Normal log events still drop when the queue is full; explicit flush() still waits for queue capacity and receiver processing before returning.

Verification

  • cargo test -p codex-state log_db
  • cargo test -p codex-state
  • just fix -p codex-state

The added tests cover configured batch-size flushing, configured interval flushing, queue-full drops, and the flush barrier semantics.

@rasmusrygaard rasmusrygaard changed the title Carve out log DB interfaces for new sinks Refactor log DB into buffered sink interface Apr 24, 2026
@rasmusrygaard rasmusrygaard marked this pull request as ready for review April 24, 2026 18:54
@pakrym-oai
Copy link
Copy Markdown
Collaborator

There are a lot of new abstractions there. Can we introduce a single trait LogWriter that implements tracing_subscriber::Layer and flush and make LogDb layer implement that?

@rasmusrygaard rasmusrygaard changed the title Refactor log DB into buffered sink interface Refactor log DB into LogWriter interface Apr 24, 2026
@rasmusrygaard rasmusrygaard merged commit 5378ccc into main Apr 24, 2026
25 checks passed
@rasmusrygaard rasmusrygaard deleted the dev/rasmus/add-feedback-log-sink branch April 24, 2026 23:27
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 24, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

2 participants