Measurement harness: JSON steady-state, correctness, and R2R coverage

Part of #130524.

Status: draft / issue candidate · Area: CoreCLR-on-WASM, measurement, diagnostics/EventPipe · Related: #130518 (PGO trace capture), #130522/#130516 (entry-point upgrades)

Goal

The measurement harness for the prototype: a JSON steady-state benchmark, a correctness gate across execution modes, and an R2R coverage signal. Coverage — which methods actually executed as R2R native rather than interpreted — is obtained by consuming the runtime's existing per-method entry-point event over EventPipe on WASM, at the same scope and granularity as the existing PGO EventPipe trace. Together these are the prototype's success gate (correctness plus steady-state JSON speedup) and the acceptance signal for the lazy tier-up loop.

Why

The prototype's success bar (correctness plus steady-state JSON speedup) needs proof that the speedup came from code running native, and the lazy loop (#130516 trigger, #130522 entry-point patch) needs an observable confirmation that shards loaded and entry points upgraded. Rather than invent a bespoke counter, reuse the per-method entry-point eventing already flowing through EventPipe — the same trace mechanism used to capture the PGO profile — so coverage is just another view of that trace.

Direction

• JSON steady-state benchmark. Drive a representative JSON parse/serialize workload and measure steady-state throughput, so the R2R speedup is measured on the intended hot path.
• Correctness across execution modes. Assert identical results across interpreter-only, startup-R2R, and after-lazy-tier-up runs — splitting, precompiling, and streaming must not change behavior.
• Report timing, don't over-gate it. Report steady-state throughput (the success signal) and keep startup timing informational for this prototype.
• Consume the existing per-method entry-point event on WASM, using the same EventPipe trace path as the profile capture ([browser][coreCLR] R2R: Dedicated JS trigger to capture and download the PGO trace #130518). Coverage granularity is whatever that trace already provides (per-method entry-point resolution).
• Produce a coverage view for the harness: the set of methods that resolved to R2R native, plus an aggregate derived from it — enough to confirm the JSON hot path ran native and to quantify overall coverage.
• Confirm the event fires on WASM. Verify the per-method entry-point event is actually emitted on WASM, including for the entry-point upgrades that [browser][coreCLR] R2R: R2R that loads against IL already in memory #130522/[browser][coreCLR] R2R: Interpreter live hotness trigger for on-demand R2R shard download #130516 perform; if it isn't, scope the minimal emission needed.
• Measurement-time capability. Coverage is collected in a traced run; timing runs can be separate so tracing overhead does not distort the speedup numbers.

Scope

In: the JSON steady-state benchmark; the correctness gate across execution modes; steady-state timing; and coverage — enabling/consuming the existing per-method entry-point event over EventPipe on WASM to produce a per-method-plus-aggregate coverage view, verifying the event fires on WASM.

Out: a new bespoke coverage counter (rejected in favor of the existing event); the profile-production trigger (#130518); the runtime features under test (the other R1 work items).

Relationship to other items

Shares the EventPipe trace mechanism with the profile capture (#130518) — same transport, different events. It observes the entry-point upgrades produced by #130522 and #130516, so it doubles as the acceptance signal that the lazy loop actually activated shards.

Success criteria

• From a traced WASM run, the harness obtains the set of methods that executed as R2R native, confirms the JSON hot path among them, and reports an aggregate coverage figure.
• Coverage collection is separable from the timing run so it does not distort speedup numbers.

Open questions

• Whether the per-method entry-point event actually fires on WASM today, and if not, what minimal emission is needed for the [browser][coreCLR] R2R: R2R that loads against IL already in memory #130522/[browser][coreCLR] R2R: Interpreter live hotness trigger for on-demand R2R shard download #130516 upgrade path.
• How to collect the EventPipe trace in the browser for this purpose (shares the browser-capture mechanics of [browser][coreCLR] R2R: Dedicated JS trigger to capture and download the PGO trace #130518).
• Overhead of tracing during a measured run, and keeping timing and coverage runs cleanly separated.