The Wayback Machine - https://web.archive.org/web/20200903030356/https://github.com/arduino/arduino-cli/pull/573
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export Prometheus telemetry in daemon mode #573

Merged
merged 39 commits into from Mar 10, 2020
Merged

Export Prometheus telemetry in daemon mode #573

merged 39 commits into from Mar 10, 2020

Conversation

@rsora
Copy link
Contributor

rsora commented Jan 31, 2020

This PR implements the basis for a telemetry feature for the CLI in daemon mode.

Rationale:
Enhance the observability of the CLI internals at runtime, as we are proceeding internally to integrate the CLI as a gRPC server in all our services.

Module telemetry
Leverages https://github.com/segmentio/stats library to configure and expose a Prometheus endpoint when activated (in cli/daemon/daemon.go). It also provides helpers for transformation of commands input data in stats.Tags objects, using the installation id and secret generated by the inventory module.

The telemetry activation and endpoint config are available via CLI configuration.

Module repertory (feel free to ask for a name change 😄) inventory
This module leverages an additional Viper instance, in order to create a configuration repository that contains values that may create confusion in the user-facing arduino-cli.yaml file, polluting it.

This is the case for the installation values stored in the inventory.yaml file, that are used to generate server specific UUIDs that help the monitoring tools to perform statistics aggregation, when monitoring multiple CLI gRPC server instances.

The inventory file is created at the CLI first launch and it is stored in the directories.Data folder.

Instrumentation
Code instrumentation is done via standard stats apis for each command (only for the compile and board list in this PR):

  1. Create the set of tags for the command
tags := map[string]string{
		"fqbn":            req.Fqbn,
		"sketchPath":      telemetry.SanitizeSketchPath(req.SketchPath),
		"showProperties":  strconv.FormatBool(req.ShowProperties),
		"preprocess":      strconv.FormatBool(req.Preprocess),
		"buildProperties": strings.Join(req.BuildProperties, ","),
...
                "success":          "true",
  1. Modify the success tag according to each case
tags["success"] = "true"
  1. Push the data to the stats engine
stats.Incr("compile", stats.M(tags)...)

Tests

  • test_inventory_creation: python integ test
  • test_inventory_prometheus_endpoint: python integ test

Notes

  • Upgraded dependencies (Viper, pyinvoke as pyinvoke/invoke#660 was closed)
  • Updated README.rst adding details about telemetry endpoint and config
@rsora rsora force-pushed the rsora/stat branch from ff47ccf to 7c82b10 Feb 5, 2020
@gvarisco gvarisco requested review from gvarisco and removed request for gvarisco Feb 11, 2020
@rsora rsora force-pushed the rsora/stat branch from 18b68c8 to 964812a Feb 11, 2020
@rsora rsora force-pushed the rsora/stat branch from 964812a to 0c8d49b Mar 3, 2020
@rsora rsora added this to the 0.10.0 milestone Mar 4, 2020
Copy link
Member

cmaglie left a comment

Added some change requests.

I see that prometeus does not keep stats when terminated (so metrics are reset on each run), is this expected?

repertory/repertory.go Outdated Show resolved Hide resolved
repertory/repertory.go Outdated Show resolved Hide resolved
commands/compile/compile.go Outdated Show resolved Hide resolved
commands/compile/compile.go Outdated Show resolved Hide resolved
Copy link
Contributor

masci left a comment

Left few comments but looks good overall

configuration/defaults.go Outdated Show resolved Hide resolved
cli/daemon/daemon.go Outdated Show resolved Hide resolved
commands/compile/compile.go Outdated Show resolved Hide resolved
test/conftest.py Outdated Show resolved Hide resolved
repertory/repertory.go Outdated Show resolved Hide resolved
telemetry/telemetry.go Outdated Show resolved Hide resolved
@rsora rsora force-pushed the rsora/stat branch from 9c005b2 to 8b425a7 Mar 9, 2020
@rsora rsora requested review from masci and cmaglie Mar 9, 2020
)

# we block here until the test function using this fixture has returned
yield runner

This comment has been minimized.

@masci

masci Mar 10, 2020

Contributor

🍰

@masci
masci approved these changes Mar 10, 2020
Copy link
Contributor

masci left a comment

🚀

commands/board/list.go Outdated Show resolved Hide resolved
@rsora rsora requested a review from cmaglie Mar 10, 2020
@rsora rsora merged commit 2791756 into master Mar 10, 2020
5 checks passed
5 checks passed
build
Details
test-matrix (ubuntu-latest) test-matrix (ubuntu-latest)
Details
test-matrix (windows-latest)
Details
test-matrix (macOS-latest)
Details
license/cla Contributor License Agreement is signed.
Details
@rsora rsora deleted the rsora/stat branch Mar 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.