[2/4] Implement executor HTTP request runner by aibrahim-oai · Pull Request #18582 · openai/codex

aibrahim-oai · 2026-04-19T18:25:57Z

Why

Remote streamable HTTP MCP needs the executor to perform ordinary HTTP requests on the executor side. This keeps network placement aligned with experimental_environment = "remote" without adding MCP-specific executor APIs.

What

Add an executor-side http/request runner backed by reqwest.
Validate request method and URL scheme, preserving the transport boundary at plain HTTP.
Return buffered responses for ordinary calls and emit ordered http/request/bodyDelta notifications for streaming responses.
Register the request handler in the exec-server router.
Document the runner entrypoint, conversion helpers, body-stream bridge, notification sender, timeout behavior, and new integration-test helpers.
Add exec-server integration tests with the existing websocket harness and a local TCP HTTP peer for buffered and streamed responses, with comments spelling out what each test proves and its setup/exercise/assert phases.

Stack

[1/4] Add executor HTTP request protocol #18581 protocol
[2/4] Implement executor HTTP request runner #18582 runner
[3/4] Add executor-backed RMCP HTTP client #18583 RMCP client
[4/4] Honor Streamable HTTP MCP placement #18584 manager wiring and local/remote coverage

Verification

just fmt
cargo check -p codex-exec-server -p codex-rmcp-client --tests
cargo check -p codex-core --test all compile-only
git diff --check
Online full CI is running from the full-ci branch, including the remote Rust test job.

Co-authored-by: Codex noreply@openai.com

aibrahim-oai · 2026-04-19T18:26:10Z

This change is part of the following stack:

[1/4] Add executor HTTP request protocol #18581
- [2/4] Implement executor HTTP request runner #18582 ◀
  - [3/4] Add executor-backed RMCP HTTP client #18583
    - [4/4] Honor Streamable HTTP MCP placement #18584

_{Change managed by git-spice.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a1b28d67b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-19T18:30:40Z

+    let body = response.bytes().await.map_err(|err| {
+        internal_error(format!("failed to read http/request response body: {err}"))
+    })?;


Enforce a max size when buffering HTTP response bodies

response.bytes().await reads the entire payload into memory with no limit before returning bodyBase64. A remote endpoint can return an arbitrarily large body and force the exec-server process into excessive memory use or OOM, making this an availability bug. Add a bounded read (or require streaming) for buffered mode.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: afcf57105c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-19T18:42:45Z

+    let timeout = params
+        .timeout_ms
+        .map(Duration::from_millis)
+        .unwrap_or(DEFAULT_HTTP_REQUEST_TIMEOUT);
+    let headers = build_headers(params.headers)?;
+    let client = reqwest::Client::builder()
+        .timeout(timeout)
+        .build()


Disable total timeout for streamed HTTP responses

Client::builder().timeout(timeout) sets a whole-request deadline, including reading the entire body. In streamResponse mode this truncates long-lived/SSE responses at the default 30s when timeoutMs is omitted, emitting a terminal error even if upstream is healthy. That breaks the intended streaming behavior for remote MCP HTTP flows.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: afcf57105c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-19T18:43:58Z

+    response: reqwest::Response,
+    notifications: RpcNotificationSender,
+) {
+    tokio::spawn(async move {


Cancel body-stream task when RPC connection closes

spawn_body_stream detaches a task that owns RpcNotificationSender. If the JSON-RPC transport disconnects, that sender keeps the outbound channel alive until the HTTP body stream ends. Because timeout_ms is caller-controlled, a peer can request a very long timeout (or a never-ending stream), causing run_connection teardown to hang and leak per-connection tasks/resources.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6a38e3191

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-19T19:01:45Z

+        // The JSON-RPC response carries status and headers; the body follows as
+        // ordered notifications so callers can begin parsing streaming content
+        // without waiting for the server to close the HTTP response.
+        spawn_body_stream(request_id, response, notifications);


Send http/request response before bodyDelta notifications

run_http_request starts spawn_body_stream before the handler enqueues the JSON-RPC response. The spawned task can push http/request/bodyDelta messages first, so headers/status may be delayed behind body frames. With bounded channels, large/fast streams can fill notification queues and stall http_request_stream() while it is still waiting for the response.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b85c91f52b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-19T19:30:49Z

+    let request_id = if params.stream_response {
+        Some(params.request_id.clone().ok_or_else(|| {
+            invalid_params("http/request streamResponse requires requestId".to_string())
+        })?)
+    } else {


Reject duplicate stream request IDs before spawning streams

Validate requestId uniqueness for active streamed requests. run_http_request accepts any caller-supplied requestId, and each spawn_body_stream starts seq at 1. If two in-flight streamed calls reuse the same requestId, their http/request/bodyDelta frames can interleave under one key with colliding sequence numbers, violating ordered-stream semantics and causing consumers to corrupt or reject the body.

Useful? React with 👍 / 👎.

Co-authored-by: Codex <noreply@openai.com>

starr-openai · 2026-04-21T23:42:56Z

+        params: HttpRequestParams,
+    ) -> Result<HttpRequestResponse, JSONRPCErrorError> {
+        self.require_initialized_for("http")?;
+        let request_id = if params.stream_response {


lets just make request_id required always?

starr-openai · 2026-04-21T23:45:19Z

+            Ok((response, pending_stream)) => {
+                if let Some(pending_stream) = pending_stream {
+                    let mut http_body_streams = self.http_body_streams.lock().await;
+                    http_body_streams.pending.push_back(pending_stream);


can we just drive the body stream with a tokio task here? why do we enqueue the stream and drive them all separately?

if it's to ensure we clean-up successfully, maybe use a JoinSet + CancellationToken

starr-openai · 2026-04-21T23:46:49Z

+    let status = response.status().as_u16();
+    let headers = response_headers(response.headers());
+
+    if let Some(request_id) = request_id {


same as above - I think making request_id required simplifies the code a bit and is reasonable to always have

Co-authored-by: Codex <noreply@openai.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fa029b87db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-22T05:15:40Z

+    async fn reserve_http_body_stream(&self, request_id: &str) -> Result<(), JSONRPCErrorError> {
+        let mut body_streams = self.body_streams.lock().await;
+        if body_streams.reserved_ids.contains_key(request_id) {
+            return Err(invalid_params(format!(
+                "http/request streamResponse requestId `{request_id}` is already active"
+            )));
+        }
+        body_streams
+            .reserved_ids
+            .insert(request_id.to_string(), None);
+        Ok(())


Cap concurrent streamed HTTP requests per connection

reserve_http_body_stream only rejects duplicate requestIds and accepts every unique id, and each accepted stream later gets its own spawned body task. With timeoutMs allowed to be null, a client can open many long-lived streams and keep them active indefinitely, leading to unbounded task/socket growth and potential executor DoS. Enforce a per-connection active-stream limit.

Useful? React with 👍 / 👎.

Co-authored-by: Codex <noreply@openai.com>

starr-openai · 2026-04-22T18:24:50Z

    session_registry: Arc<SessionRegistry>,
    notifications: RpcNotificationSender,
    session: StdMutex<Option<SessionHandle>>,
+    body_streams: Mutex<HashMap<String, Option<JoinHandle<()>>>>,


from quick read of the rust docs, if we can use a tokio_util::task::TaskTracker and don't need the key here, that might generalize so we can just use this for any background coros.

so e.g. background_tasks: tokio_util::task::TaskTracker<...>

then you can add the background body tailing tasks there and don't need the explicit cleanup in the request handler.

if it's not easy to adapt/use the current PR is fine tho

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb86b7a6bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-22T20:00:39Z

+            None => reqwest::Client::builder(),
+            Some(timeout_ms) => {
+                reqwest::Client::builder().timeout(Duration::from_millis(timeout_ms))
+            }
        }
+        .build()
+        .map_err(|err| internal_error(format!("failed to build http/request client: {err}")))?;


Build HTTP runner client with shared custom-CA policy

ExecutorHttpRequestRunner::new builds a raw reqwest::Client, which bypasses Codex’s shared CA policy (CODEX_CA_CERTIFICATE / SSL_CERT_FILE). As a result, http/request can fail TLS to enterprise/intercepting proxies even when other Codex HTTP paths succeed. This outbound path should use the shared reqwest-client construction helper instead of direct Client::builder().build().

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-04-22T20:00:39Z

+            tokio::select! {
+                _ = shutdown.cancelled() => {}
+                _ = ExecutorHttpRequestRunner::stream_body(pending_stream, notifications) => {}
+            }


Cancel HTTP body streams immediately on session eviction

Stream tasks only terminate on background_task_shutdown in shutdown(). But eviction (is_session_attached == false) is checked only after incoming_rx.recv() returns, so an otherwise idle old connection may never reach shutdown. Existing http/request stream tasks can keep emitting bodyDelta notifications on the evicted socket after session resume.

Useful? React with 👍 / 👎.

This was referenced Apr 19, 2026

[4/4] Honor Streamable HTTP MCP placement #18584

Merged

[1/4] Add executor HTTP request protocol #18581

Merged

[3/4] Add executor-backed RMCP HTTP client #18583

Merged

chatgpt-codex-connector Bot reviewed Apr 19, 2026

View reviewed changes

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from d0f4fc9 to 1fc4cd7 Compare April 19, 2026 18:38

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from 1a1b28d to afcf571 Compare April 19, 2026 18:38

aibrahim-oai changed the title ~~Implement executor HTTP request runner~~ [2/4] Implement executor HTTP request runner Apr 19, 2026

chatgpt-codex-connector Bot reviewed Apr 19, 2026

View reviewed changes

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from 1fc4cd7 to 5b3829a Compare April 19, 2026 18:48

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from afcf571 to 81ff713 Compare April 19, 2026 18:48

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from 5b3829a to 53977b8 Compare April 19, 2026 18:57

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from 81ff713 to d6a38e3 Compare April 19, 2026 18:57

chatgpt-codex-connector Bot reviewed Apr 19, 2026

View reviewed changes

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from 53977b8 to 8edc173 Compare April 19, 2026 19:07

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from d6a38e3 to 0a24aa2 Compare April 19, 2026 19:07

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from 8edc173 to 1a9358f Compare April 19, 2026 19:15

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from 0a24aa2 to 4690cae Compare April 19, 2026 19:15

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from 1a9358f to 3018783 Compare April 19, 2026 19:22

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from 4690cae to b85c91f Compare April 19, 2026 19:22

chatgpt-codex-connector Bot reviewed Apr 19, 2026

View reviewed changes

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from 3018783 to c3159bc Compare April 20, 2026 15:45

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from b85c91f to f14bc16 Compare April 20, 2026 15:45

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from c3159bc to 888b4d7 Compare April 20, 2026 15:52

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from f14bc16 to dbee178 Compare April 20, 2026 15:52

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from 888b4d7 to f33c92e Compare April 20, 2026 16:02

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from dbee178 to 5baf804 Compare April 20, 2026 16:02

aibrahim-oai force-pushed the dev/remote-mcp-http-protocol-full-ci branch from f33c92e to b003868 Compare April 20, 2026 16:11

aibrahim-oai force-pushed the dev/remote-mcp-http-runner-full-ci branch from 5baf804 to 941b42e Compare April 20, 2026 16:11

aibrahim-oai and others added 5 commits April 20, 2026 23:10

codex: fix exec-server timeout test on linux

b43e995

Co-authored-by: Codex <noreply@openai.com>

codex: fix clippy failure on PR #18582

b79f32f

Co-authored-by: Codex <noreply@openai.com>

codex: simplify exec-server timeout tristate

222046f

Co-authored-by: Codex <noreply@openai.com>

codex: use nested option for http timeout

c299270

Co-authored-by: Codex <noreply@openai.com>

codex: drop implicit exec-server http timeout

cc7ee09

Co-authored-by: Codex <noreply@openai.com>

starr-openai reviewed Apr 21, 2026

View reviewed changes

aibrahim-oai and others added 5 commits April 21, 2026 17:15

codex: start http body streams after responses

649450c

Co-authored-by: Codex <noreply@openai.com>

codex: move executor http streaming into client

7beb501

Co-authored-by: Codex <noreply@openai.com>

codex: require request ids for executor http

d93a319

Co-authored-by: Codex <noreply@openai.com>

codex: update executor http timeout test

fa029b8

Co-authored-by: Codex <noreply@openai.com>

codex: simplify http body stream registry

e4e2525

Co-authored-by: Codex <noreply@openai.com>

chatgpt-codex-connector Bot reviewed Apr 22, 2026

View reviewed changes

aibrahim-oai and others added 12 commits April 21, 2026 22:18

codex: move executor http response flow into handler

53d813f

Co-authored-by: Codex <noreply@openai.com>

codex: route http requests through exec router

1d550d8

Co-authored-by: Codex <noreply@openai.com>

codex: inline executor http state into handler

83e0ad6

Co-authored-by: Codex <noreply@openai.com>

codex: clarify exec-server outbound sender naming

06616d5

Co-authored-by: Codex <noreply@openai.com>

codex: derive exec-server notifications from outbound sender

cec5da2

Co-authored-by: Codex <noreply@openai.com>

codex: reuse rpc notification sender for responses

b0c4c26

Co-authored-by: Codex <noreply@openai.com>

codex: trim exec-server http routing churn

9bd9c44

Co-authored-by: Codex <noreply@openai.com>

codex: simplify exec-server http routing

ab5e359

Co-authored-by: Codex <noreply@openai.com>

codex: wrap executor http requests in a runner

b602d47

Co-authored-by: Codex <noreply@openai.com>

codex: drop thin executor http wrappers

61f15c9

Co-authored-by: Codex <noreply@openai.com>

codex: reorder executor http client functions

93f1a99

Co-authored-by: Codex <noreply@openai.com>

codex: fix exec-server clippy cleanup

1febfa7

Co-authored-by: Codex <noreply@openai.com>

starr-openai reviewed Apr 22, 2026

View reviewed changes

starr-openai approved these changes Apr 22, 2026

View reviewed changes

codex: use task tracker for exec-server background tasks

cb86b7a

chatgpt-codex-connector Bot reviewed Apr 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[2/4] Implement executor HTTP request runner#18582

[2/4] Implement executor HTTP request runner#18582
aibrahim-oai merged 29 commits into
mainfrom
dev/remote-mcp-http-runner-full-ci

aibrahim-oai commented Apr 19, 2026 •

edited

Loading

aibrahim-oai commented Apr 19, 2026 •

edited

Loading

chatgpt-codex-connector Bot left a comment

chatgpt-codex-connector Bot Apr 19, 2026

chatgpt-codex-connector Bot left a comment

chatgpt-codex-connector Bot Apr 19, 2026

chatgpt-codex-connector Bot left a comment

chatgpt-codex-connector Bot Apr 19, 2026

chatgpt-codex-connector Bot left a comment

chatgpt-codex-connector Bot Apr 19, 2026

chatgpt-codex-connector Bot left a comment

chatgpt-codex-connector Bot Apr 19, 2026

starr-openai Apr 21, 2026

starr-openai Apr 21, 2026

starr-openai Apr 21, 2026

starr-openai Apr 21, 2026

chatgpt-codex-connector Bot left a comment

chatgpt-codex-connector Bot Apr 22, 2026

starr-openai Apr 22, 2026

chatgpt-codex-connector Bot left a comment

chatgpt-codex-connector Bot Apr 22, 2026

chatgpt-codex-connector Bot Apr 22, 2026

Labels

2 participants

Conversation

aibrahim-oai commented Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Stack

Verification

aibrahim-oai commented Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

chatgpt-codex-connector Bot Apr 19, 2026

Choose a reason for hiding this comment

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

chatgpt-codex-connector Bot Apr 19, 2026

Choose a reason for hiding this comment

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

chatgpt-codex-connector Bot Apr 19, 2026

Choose a reason for hiding this comment

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

chatgpt-codex-connector Bot Apr 19, 2026

Choose a reason for hiding this comment

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

chatgpt-codex-connector Bot Apr 19, 2026

Choose a reason for hiding this comment

starr-openai Apr 21, 2026

Choose a reason for hiding this comment

starr-openai Apr 21, 2026

Choose a reason for hiding this comment

starr-openai Apr 21, 2026

Choose a reason for hiding this comment

starr-openai Apr 21, 2026

Choose a reason for hiding this comment

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

chatgpt-codex-connector Bot Apr 22, 2026

Choose a reason for hiding this comment

starr-openai Apr 22, 2026

Choose a reason for hiding this comment

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

chatgpt-codex-connector Bot Apr 22, 2026

Choose a reason for hiding this comment

chatgpt-codex-connector Bot Apr 22, 2026

Choose a reason for hiding this comment

Labels

2 participants

aibrahim-oai commented Apr 19, 2026 •

edited

Loading

aibrahim-oai commented Apr 19, 2026 •

edited

Loading