Skip to content

app/vlinsert: splunk support#710

Merged
valyala merged 1 commit into
masterfrom
splunk
Apr 14, 2026
Merged

app/vlinsert: splunk support#710
valyala merged 1 commit into
masterfrom
splunk

Conversation

@AndrewChubatiuk
Copy link
Copy Markdown
Contributor

@AndrewChubatiuk AndrewChubatiuk commented Sep 26, 2025

Describe Your Changes

  • replaced fastjson.Parser with fastjson.Scanner, since one line can contains multiple log entries
  • added separate endpoint for Splunk HEC events
  • since no collector, that supports Splunk events provides an ability to set extra HTTP headers or extra args, -splunk.* cmd arguments were added

Checklist

The following checks are mandatory:

@AndrewChubatiuk AndrewChubatiuk marked this pull request as draft September 26, 2025 10:57
@AndrewChubatiuk AndrewChubatiuk force-pushed the splunk branch 4 times, most recently from 0fb01c9 to 77ab347 Compare December 15, 2025 19:18
@AndrewChubatiuk AndrewChubatiuk marked this pull request as ready for review December 15, 2025 19:24
@AndrewChubatiuk AndrewChubatiuk force-pushed the splunk branch 4 times, most recently from 9005319 to 43b45c8 Compare December 20, 2025 07:50
@AndrewChubatiuk
Copy link
Copy Markdown
Contributor Author

@valyala could you please take a look at this PR?

Comment thread app/vlinsert/splunk/splunk.go
Comment thread app/vlinsert/splunk/splunk.go Outdated
Comment thread app/vlinsert/splunk/splunk.go Outdated
@AndrewChubatiuk AndrewChubatiuk force-pushed the splunk branch 6 times, most recently from 351d07f to 972e4b0 Compare February 4, 2026 12:00
Copy link
Copy Markdown
Member

@vadimalekseev vadimalekseev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Left some minor comments

Comment thread app/vlinsert/splunk/splunk.go Outdated
Comment thread docs/victorialogs/data-ingestion/Splunk.md
Comment thread lib/logstorage/json_parser.go Outdated
Comment thread docs/victorialogs/data-ingestion/Splunk.md Outdated
Comment thread app/vlinsert/main.go
Comment thread app/vlinsert/splunk/splunk.go Outdated
Comment thread app/vlinsert/splunk/splunk.go Outdated
Comment thread app/vlinsert/splunk/splunk.go Outdated
Comment thread lib/logstorage/json_parser.go Outdated
Comment thread deployment/docker/victorialogs/fluentbit/otlp/fluent-bit.conf
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 16 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="app/vlinsert/datadog/datadog.go">

<violation number="1" location="app/vlinsert/datadog/datadog.go:38">
P3: Metrics path labels still reference "/insert/datadog/api/v2/logs" even though the handler now serves "/api/v2/logs", so observability will be misleading. Update the metric path labels to match the new endpoint.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread app/vlinsert/datadog/datadog.go Outdated
Comment thread app/vlinsert/main.go Outdated
Comment thread app/vlinsert/splunk/splunk.go Outdated
@AndrewChubatiuk AndrewChubatiuk force-pushed the splunk branch 2 times, most recently from f1d54e5 to 202f095 Compare February 11, 2026 13:02
@AndrewChubatiuk AndrewChubatiuk force-pushed the splunk branch 3 times, most recently from 665dda7 to 7ccd80d Compare April 7, 2026 16:12
@valyala valyala merged commit b9ef628 into master Apr 14, 2026
8 checks passed
@valyala valyala deleted the splunk branch April 14, 2026 17:26
@valyala
Copy link
Copy Markdown
Contributor

valyala commented Apr 14, 2026

@AndrewChubatiuk , thank you for implementing data ingestion for Splunk agent at VictoriaLogs!

valyala added a commit that referenced this pull request Apr 14, 2026
- Clarify the description for -splunk.* command-line flags
- Clarify the docs at https://docs.victoriametrics.com/victorialogs/data-ingestion/splunk/
- Add descriptions for Splunk-related command-line flags to VictoriaMetrics docs.
- Consistently use 'Splunk' instead of 'splunk' across the docs.
- Verify the correctness of stream field names passed via -splunk.streamFields command-line flags.
- Clarify the error message when -splunk.tenantID command-line flag cannot be parsed,
  so users could understand how to fix the error.
- Add missing docs for JSONScanner.
- Fix errors in docs for commonJSON.
- Properly reset JSONScanner.err on JSONScanner.reset() call.
- Return back the order of parseLogMessage() args, because it was changed without real need.
- Add a link to Splunk data ingestion docs at https://docs.victoriametrics.com/victorialogs/data-ingestion/
- Add a CHANGELOG line about the addition of the support for Splunk data ingestion protocol.

Updates #710
@valyala
Copy link
Copy Markdown
Contributor

valyala commented Apr 14, 2026

@AndrewChubatiuk , please see the follow-up commit 1b71b92 .

@valyala
Copy link
Copy Markdown
Contributor

valyala commented Apr 14, 2026

FYI, this pull request has been included in VictoriaLogs v1.50.0.

valyala added a commit that referenced this pull request Apr 15, 2026
…nk data ingestion protocol

This feature has been implemented in the commit b9ef628

Updates #710
See also #1305
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants