Skip to content

Conversation

@kolesnikovae
Copy link
Collaborator

@kolesnikovae kolesnikovae commented Feb 26, 2025

The PR adds asynchronous log writer.

Recently implemented buffered logger (#3947) does not address the issue with blocking log writes:

image

The implementation we use flushes the buffer synchronously, blocking all callers. To address the problem, we should use asynchronous writer. The implementation in the PR implements a queue that is processed in background:

  • when the queue is full, new writes are discarded.
  • by default, the queue is large enough to store 20K messages or 5MB.
  • errors are ignored since we can't do anything if stderr is not available (which is the cause of errors).
@kolesnikovae kolesnikovae marked this pull request as ready for review February 26, 2025 14:01
@kolesnikovae kolesnikovae requested a review from a team as a code owner February 26, 2025 14:01
@kolesnikovae kolesnikovae merged commit 22db84d into main Mar 5, 2025
21 checks passed
@kolesnikovae kolesnikovae deleted the perf/async-log-writer branch March 5, 2025 05:39
shelldandy pushed a commit to shelldandy/pyroscope that referenced this pull request Mar 14, 2025
* perf: add async log writer

* clean up tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants