Fix macOS SIGSEGV in task execution by using fork+exec

[kaxil]

Summary

Discovered while testing the AIP-99 LLMSQLQueryOperator example DAGs from #64824 on macOS. Tasks that make network calls (LLM API requests, HTTP calls) crash intermittently with SIGSEGV or SIGABRT when running via airflow standalone or any executor on macOS.

Root cause: WatchedSubprocess.start() in supervisor.py uses bare os.fork() to create task child processes. On macOS, the forked child inherits corrupted Objective-C runtime state from the parent. When the child later triggers ObjC class initialization -- for example via socket.getaddrinfo() -> macOS system DNS resolver -> Security.framework -> +[NSNumber initialize] -- the ObjC runtime detects the half-initialized state and deliberately crashes.

The fix: Add a use_exec: bool = False kwarg to WatchedSubprocess.start(). ActivitySubprocess.start() passes use_exec=True; DAG processor and triggerer use the default. On platforms in _FORK_EXEC_PLATFORMS (currently {"darwin"}), the fork+exec path runs os.dup2 on the socketpairs to place them on FDs 0/1/2, then os.execv a fresh Python interpreter. os.dup2 (with the default inheritable=True) clears FD_CLOEXEC on the destination FDs, so they survive execv. The structured log channel is NOT inherited; instead, the exec'd child sets _AIRFLOW_FORK_EXEC=1, and after startup() in task_runner.main the existing reinit_supervisor_comms() requests the log FD from the supervisor via ResendLoggingFD + SCM_RIGHTS (the same mechanism used by the sudo / virtualenv re-exec path).

The crash chain

supervisor.py: os.fork()
  -> child runs pydantic_ai.Agent.run_sync()
    -> httpx creates ThreadPoolExecutor for DNS
      -> socket.getaddrinfo() in worker thread
        -> macOS system resolver (not glibc)
          -> _scproxy / Security.framework
            -> ObjC runtime detects fork-unsafe state
              -> SIGABRT / SIGSEGV

The faulthandler traceback that identified the root cause:

Current thread (most recent call first):
  File "socket.py", line 978 in getaddrinfo
  File "concurrent/futures/thread.py", line 59 in run
  ...

Why macOS only

Apple's ObjC runtime, CoreFoundation, and libdispatch are not fork-safe. This is why CPython changed multiprocessing's default start method from fork to spawn on macOS in Python 3.8 (BPO-33725). Linux uses glibc's resolver which has no ObjC dependency, so bare fork() works fine there. The _FORK_EXEC_PLATFORMS set is the extension point if another platform needs the same treatment later.

Scope: task execution only (for now)

This PR only applies fork+exec to task execution (ActivitySubprocess). DAG processor and triggerer also use WatchedSubprocess and can hit the same macOS fork-safety crash -- DAG files often have top-level network calls (secret backends, connection / variable lookups, in-process API server threads), and triggerers run user-defined async triggers that poll APIs. We observed the DAG processor crash earlier in this investigation (the InProcessExecutionAPI + a2wsgi background thread initializing httpx / ssl / Security.framework).

Extending fork+exec to them requires reworking _child_exec_main() to rehydrate arbitrary targets across execv (it currently hardcodes _subprocess_main; DAG processor passes target=_parse_file_entrypoint, triggerer passes target=TriggerRunnerSupervisor.run_in_process). Tracked as follow-up in #65691.

Scope: all executors on macOS, not just LocalExecutor

This affects any executor running on macOS (Local, Celery worker, etc.) because the fork happens inside supervise() in the Task SDK, not in the executor itself. The executor spawns a worker process (which is safe -- multiprocessing.Process uses spawn on macOS), but that worker then calls supervise() which does the bare os.fork().

The two-fork architecture:

Executor -> multiprocessing.Process (spawn, safe)
  -> worker calls supervise()
    -> os.fork() (bare fork, UNSAFE on macOS without exec)
      -> child runs task

How the log channel is obtained after exec

_fork_main needs a log FD to set up structured logging. In the bare-fork path it's inherited. In the fork+exec path:

1. _child_exec_main (the exec'd entry point) calls _fork_main(..., log_fd=0, ...) -- zero means skip structured-log setup. It also sets _AIRFLOW_FORK_EXEC=1.
2. _fork_main runs through its usual setup (signals, stdio, etc.) with logging going to the stderr socket.
3. task_runner.main calls startup() to receive StartupDetails from the supervisor.
4. After startup(), main checks _AIRFLOW_FORK_EXEC and calls the existing reinit_supervisor_comms(), which sends ResendLoggingFD() over the requests socket and receives a fresh log FD via SCM_RIGHTS.

The timing works because the supervisor enters its event loop (process.wait()) after _on_child_started has sent the startup message. By the time the child requests the log channel, the supervisor is in the loop and ready to handle it.

What we tried that didn't work

Approach	Why it failed
Lazy imports (pydantic_ai, datafusion)	Crash happens at runtime during DNS resolution, not at import time
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES	Undocumented Apple debug knob. Suppresses the abort but doesn't fix the underlying memory corruption. Cached at ObjC runtime init, unreliable on newer macOS
NO_PROXY='*'	Python-level env var; socket.getaddrinfo() is a C library call that uses the macOS system resolver directly
Pre-initialize _scproxy before fork	Fragile -- any new dependency that touches ObjC frameworks would break it again
subprocess.Popen instead of os.fork()	Loses the socketpair FDs. The child can't communicate with the supervisor

Why fork + exec and not spawn

Python's multiprocessing.Process(start_method='spawn') would also work, but it requires pickling the target function and arguments. The supervisor's communication is built around socketpairs created before fork, with FDs inherited by the child. fork + exec preserves this design: the three socketpairs are placed at FDs 0/1/2 via dup2 (which clears FD_CLOEXEC), and the log channel is obtained after startup via the existing ResendLoggingFD protocol. No new cross-process communication path needed.

References

• Crash in urllib/request.py proxy_bypass_macosx_sysconf python/cpython#105912 -- getaddrinfo SIGSEGV after fork on macOS
• exception error in _scproxy.so when called after fork python/cpython#58037 -- _scproxy crash after fork
• Task exited with return code Negsignal.SIGSEGV when dag triggered from UI #24463 -- Airflow SIGSEGV reports on macOS
• https://bugs.python.org/issue30385 -- original macOS fork-safety discussion
• Extend macOS fork+exec fix to DAG processor and triggerer #65691 -- follow-up to extend fork+exec to DAG processor and triggerer

---

Was generative AI tooling used to co-author this PR?

• Yes -- Claude Code

[ashb]

I wonder if we should also set the env var we have to not load settings in this ecec'd process to speed up airflow import?

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Fixes intermittent macOS task crashes (SIGSEGV/SIGABRT) caused by fork-unsafe Apple Objective‑C runtime state by switching the task-execution subprocess path to fork + immediate exec.

Changes:

• Add macOS-only fork+exec path for task execution in the supervisor (WatchedSubprocess.start) with _child_exec_main as the post-exec entrypoint.
• Reinitialize supervisor comms/logging channel in the task runner when started via the fork+exec path.
• Add a unit test for _child_exec_main wiring of FDs 0/1/2 and signaling task runner via _AIRFLOW_FORK_EXEC.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File	Description
task-sdk/src/airflow/sdk/execution_time/supervisor.py	Introduces macOS fork+exec execution path and _child_exec_main entrypoint.
task-sdk/src/airflow/sdk/execution_time/task_runner.py	Requests structured logging FD after startup when running under fork+exec.
task-sdk/tests/task_sdk/execution_time/test_supervisor.py	Adds test coverage for _child_exec_main FD handling and _AIRFLOW_FORK_EXEC env signaling.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

[ashb]

Few nits but LGTM over all.

We don't want a way to let users disable this do we?

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.