Skip to content

outputMonitor: fix two false-positive families pausing the agent loop#315485

Merged
meganrogge merged 1 commit into
microsoft:mainfrom
ba-work:ba-work/output-monitor-input-pattern-315476
May 11, 2026
Merged

outputMonitor: fix two false-positive families pausing the agent loop#315485
meganrogge merged 1 commit into
microsoft:mainfrom
ba-work:ba-work/output-monitor-input-pattern-315476

Conversation

@ba-work
Copy link
Copy Markdown
Contributor

@ba-work ba-work commented May 9, 2026

Fixes #315476.

This change addresses two related false-positive families in RunInTerminalTool's input-needed heuristic that pause the agent loop on benign terminal output. Both surfaced live: the $Ri family on log lines ending in : or ?, and the Vyo family on every command return when the shell uses an oh-my-zsh-style git-aware PS1 prompt.

Family 1 — detectsInputRequiredPattern's broad fallbacks match log output

detectsInputRequiredPattern augmented detectsHighConfidenceInputPattern's strict pattern set with two over-broad fallbacks:

/: +$/                              // any line ending colon + spaces
/\? *(?:\([a-z\s]+\))? +$/i     // any line ending question + spaces

These fire on benign log output like:

[INFO] Starting:
Last Command:
find: /tmp/x: No such file:
Are you sure you want to do this?

The strict guard at the original call site is shell-integration's isActive flag, which is only true when the foreground process is actively holding stdin open. _handleIdleState was using the broad pattern unconditionally, with no isActive evidence.

Fix: split the function into two:

  • detectsInputRequiredPattern(line): strict-only — delegates to detectsHighConfidenceInputPattern. Safe to call from any path, including unconditionally on the last line of a finished command.
  • detectsLikelyInputRequiredPattern(line): strict + broad fallbacks. Should only be called when an out-of-band signal already confirms the process is awaiting input (isActive === true).

Update the callers:

  • _handleIdleState: switch to the strict detectsInputRequiredPattern. Eliminates the log-line false positives.
  • _waitForIdle: keep using the broad detectsLikelyInputRequiredPattern. This call is gated by recentlyIdle && isActive === true, which provides the out-of-band evidence the broad fallbacks need.

Family 2 — detectsHighConfidenceInputPattern's paren-default rule matches every oh-my-zsh prompt

detectsHighConfidenceInputPattern contained this rule:

/:\s*\([^)]*\) +$/      // "package name: (test) "

intended for npm-init / yarn-init style prompts. Because \s* allows zero spaces between : and (, it also matches every git-aware shell prompt:

➜  myrepo git:(main)
➜  vscode git:(ba-work/output-monitor)

(oh-my-zsh's robbyrussell theme; bash with __git_ps1; etc.) Live trigger rate: every shell-prompt return, on every benign command, on every machine running oh-my-zsh. The agent loop pauses for ~2 minutes (the _handleTimeoutState recovery window) before resuming on every command, making multi-step tasks unusable.

Fix: tighten \s*\s+ so at least one space is required between : and (. npm-init prompts always render a space (name: (default)); shell-prompt PS1 themes never do (git:(main)). Single-byte regex change, no behavioural change to the npm-init / yarn-init matches.

Tests

  • Update Line ends with colon to use the strict path and assert that bare Enter your name: , File to overwrite: , and log lines like Last Command: , [INFO] Starting: , find: /tmp/x: No such file: , error: subprocess returned: , and 2025-05-09 12:34:56 INFO Starting: all return false under the strict detectsInputRequiredPattern.
  • Update detects trailing questions similarly.
  • Add a new suite for detectsLikelyInputRequiredPattern covering strict-set parity, bare-colon/question matching, non-prompt rejection, and the documented false positives (log lines with colons) so the broad surface stays understood.
  • Add regression cases in detectsHighConfidenceInputPattern asserting that oh-my-zsh / git-aware prompts return false while npm-init prompts continue to return true.
  • Add the same oh-my-zsh regression cases in detectsInputRequiredPattern (strict) and detectsLikelyInputRequiredPattern so the rule's flow through both call paths is pinned.
  • Add three end-to-end OutputMonitor integration tests asserting onDidDetectInputNeeded does NOT fire for (a) log lines ending in : , (b) lines ending in ? , (c) oh-my-zsh git-aware shell prompts. These pin the wiring at _handleIdleState, not just the regex behaviour in isolation.

How to verify

Family 2 (oh-my-zsh prompts): in the integrated terminal with shell integration enabled, on a machine using oh-my-zsh's robbyrussell theme (or any git-aware PS1 that renders git:(branch) with no space after the colon), run any command from agent mode that returns to the prompt:

cd ~/anywhere && git status

Without the fix: the agent pauses for ~2 minutes after every command return. With the fix: the agent continues immediately.

Family 1 (broad-fallback log lines): from agent mode, run a slow-ish command whose visible buffer ends in : while the command is still being polled (the strict path is what fires onDidDetectInputNeeded; running command output, not the post-prompt return, is what _handleIdleState evaluates):

printf 'Compiling: \n' && sleep 2

Without the fix: the agent pauses with the input-needed wrapper because detectsInputRequiredPattern's broad /: +$/ fallback matches the trailing : . With the fix: _handleIdleState calls the strict path, which no longer carries those fallbacks, and the wrapper does not fire.

Scope

This PR is intentionally limited to the two false-positive families called out in #315476. While reading the surrounding code I noticed detectsGenericPressAnyKeyPattern is invoked with outputTail rather than outputLastLine (line ~370), which is inconsistent with sibling line-oriented detectors. That predates this PR and is out of scope here; happy to file a separate fix if a maintainer agrees it's worth addressing.

Copilot AI review requested due to automatic review settings May 9, 2026 18:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refines the terminal OutputMonitor input-needed heuristics used by the chat agent terminal tooling to reduce false positives that pause the agent loop on benign output (e.g., log lines ending in : / ? and git-aware shell prompts like git:(main)).

Changes:

  • Split “input-needed” detection into a strict path (detectsInputRequiredPattern → high-confidence only) and a broader, gated path (detectsLikelyInputRequiredPattern).
  • Tighten the parenthesized-default regex to avoid matching oh-my-zsh / git-aware prompts (:\s*:\s+).
  • Expand unit/integration-style tests to cover the reported false-positive families and regressions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/monitoring/outputMonitor.ts Splits strict vs broad prompt detection, updates idle polling gating, and tightens the parenthesized-default regex.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/browser/outputMonitor.test.ts Adds regressions and new suites validating strict vs likely behavior and prompt false-positive avoidance.
@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from c198177 to 1367c62 Compare May 9, 2026 18:40
@ba-work ba-work requested a review from Copilot May 9, 2026 18:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from 1367c62 to e712877 Compare May 9, 2026 18:57
@ba-work ba-work requested a review from Copilot May 9, 2026 19:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from e712877 to 0aae89a Compare May 9, 2026 19:15
@ba-work ba-work requested a review from Copilot May 9, 2026 19:16
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Fixes microsoft#315476.

This change addresses two related false-positive families in
RunInTerminalTool's input-needed heuristic that pause the agent loop
on benign terminal output.

### Family 1 — ``'s broad fallback regexes match log output

`detectsInputRequiredPattern` ($Ri minified) augmented `Vyo`'s
strict pattern set with two over-broad fallbacks:

  /: +$/                                  (any line ending colon + spaces)
  /\? *(?:\([a-z\s]+\))? +$/i         (any line ending question + spaces)

These fire on log output like:

  [INFO] Starting:
  Last Command:
  find: /tmp/x: No such file:
  Are you sure you want to do this?  (rhetorical)

The strict guard at the call site is shell-integration's `isActive`
flag, which is only true when the foreground process is actively
holding stdin open. `_handleIdleState` (line ~388/408) was using
the broad pattern unconditionally, with no isActive evidence.

Fix: split the function into two:

  - detectsInputRequiredPattern(line): strict-only — delegates to
    detectsHighConfidenceInputPattern. Safe to call from any path,
    including unconditionally on the last line of a finished command.
  - detectsLikelyInputRequiredPattern(line): strict + broad fallbacks.
    Should only be called when an out-of-band signal already confirms
    the process is awaiting input (isActive === true).

Update the callers:

  - _handleIdleState (lines ~388, ~408): switch to the strict
    detectsInputRequiredPattern. Eliminates the log-line false
    positives that were pausing the agent.
  - _waitForIdle (line ~532): keep using the broad
    detectsLikelyInputRequiredPattern. This call is gated by
    `recentlyIdle && isActive === true`, which provides the
    out-of-band evidence the broad fallbacks need.

### Family 2 — `Vyo`'s paren-default rule matches every oh-my-zsh prompt

`Vyo` (detectsHighConfidenceInputPattern) contained a rule:

  /:\s*\([^)]*\) +$/      // "package name: (test) "

intended for npm-init / yarn-init style prompts. Because `\s*`
allows zero spaces between `:` and `(`, this also matches every
git-aware shell prompt:

  ➜  myrepo git:(main)
  ➜  vscode git:(ba-work/output-monitor)

(oh-my-zsh's robbyrussell theme; bash with __git_ps1; etc.) Live
trigger rate: every shell-prompt return, on every benign command,
on every machine running oh-my-zsh. The agent loop pauses for
~2 minutes (the _handleTimeoutState recovery window) before
resuming on every command, making multi-step tasks unusable.

Fix: tighten `\s*` -> `\s+` so at least one space is required
between `:` and `(`. npm-init prompts always render a space
("name: (default)"); shell-prompt PS1 themes never do
("git:(main)"). Single-byte regex change, no behavioural change to
the npm-init / yarn-init matches.

### Tests

- Update test 'Line ends with colon (strict path)' to assert that
  bare "Enter your name: ", "File to overwrite: ", and log lines
  like "Last Command: ", "[INFO] Starting: ", "find: /tmp/x: No
  such file: ", "error: subprocess returned: ", and
  "2025-05-09 12:34:56 INFO Starting: " all return FALSE under the
  strict detectsInputRequiredPattern.
- Update test 'detects trailing questions (strict path does not match
  bare ? prompts)' similarly.
- Add new suite for detectsLikelyInputRequiredPattern with parity
  tests, bare-colon/question matching, non-prompt rejection, and
  documented false positives (e.g. log lines with colons).
- Add regression cases in detectsHighConfidenceInputPattern asserting
  that oh-my-zsh / git-aware prompts return FALSE while npm-init
  prompts continue to return TRUE.
- Add regression cases in detectsInputRequiredPattern (strict) and
  detectsLikelyInputRequiredPattern asserting the same for the
  oh-my-zsh case (since the strict rule flows through both paths).

### How to verify

Reproduce in the integrated terminal with shell integration enabled
on a machine using oh-my-zsh's robbyrussell theme:

  cd ~/anywhere && git status

Without the fix: agent pauses for ~2 minutes after the command
returns. With the fix: agent continues immediately.
@ba-work ba-work force-pushed the ba-work/output-monitor-input-pattern-315476 branch from 0aae89a to 14ba5fb Compare May 9, 2026 19:39
@ba-work ba-work requested a review from Copilot May 9, 2026 19:41
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@meganrogge meganrogge assigned meganrogge and unassigned hediet May 11, 2026
@meganrogge meganrogge added this to the 1.121.0 milestone May 11, 2026
Copy link
Copy Markdown
Collaborator

@meganrogge meganrogge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@meganrogge
Copy link
Copy Markdown
Collaborator

Good catch on detectsGenericPressAnyKeyPattern being invoked with outputTail rather than outputLastLine — that does look inconsistent with the sibling detectors. Please go ahead and file a separate issue/PR for it. Thanks for flagging it and for keeping this PR focused.

@meganrogge meganrogge enabled auto-merge (squash) May 11, 2026 15:30
@meganrogge meganrogge merged commit 44551a9 into microsoft:main May 11, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants