Summary
An intermittent, unhandled System.ObjectDisposedException: The CancellationTokenSource has been disposed. is thrown on a ThreadPool thread from inside the SDK's own progress-callback marshaling during FoundryLocalManager.DownloadAndRegisterEpsAsync(...). Because it is unhandled on a ThreadPool thread, it fail-fasts the entire host process (0xc0000409). The calling application has no way to catch it.
Environment
- Package:
Microsoft.AI.Foundry.Local.WinML 1.2.3 — faulting module Microsoft.AI.Foundry.Local.Core.dll, assembly/file version 1.2.0.0
- Host: WinUI 3 / Windows App SDK desktop app targeting
net10.0-windows (the .NET Runtime crash event reported CoreCLR Version: 9.0.17)
- OS: Windows 11 Pro, build 26200 (x64)
- GPU / EP: NVIDIA GeForce RTX 5090 (CUDA execution provider)
What the app was doing
One-time initialization. After FoundryLocalManager.CreateAsync(...) we register the hardware execution providers with a progress callback, then fetch the catalog:
await manager.DownloadAndRegisterEpsAsync(
(epName, pct) => { /* report progress to the UI */ },
cancellationToken);
The crash occurs during this call. It is intermittent — in our logs, EP registration completed successfully 23 times and fail-fasted once. It appears most likely when the EPs are already cached, i.e. the download_and_register_eps command returns very quickly.
Crash details
Windows Error Reporting — Application Error (Event ID 1000):
Faulting application name: <host>.exe
Faulting module name: Microsoft.AI.Foundry.Local.Core.dll, version: 1.2.0.0
Exception code: 0xc0000409
.NET Runtime (Event ID 1026):
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ObjectDisposedException: The CancellationTokenSource has been disposed.
at Microsoft.AI.Foundry.Local.NativeInterop.<>c__DisplayClass13_0.<ExecuteCommandWithCallbackManaged>b__9(String, Double)
at System.Progress`1.InvokeHandlers(Object)
at System.Threading.QueueUserWorkItemCallbackDefaultContext`1.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.WindowsThreadPool.DispatchCallback(IntPtr, IntPtr, IntPtr)
Root cause (analysis)
ExecuteCommandWithCallbackManaged marshals native progress updates through a System.Progress<T>, whose handler invocations are posted to the ThreadPool (the Progress<T> is constructed on a background thread with no SynchronizationContext). The command also uses a per-command CancellationTokenSource.
When the download_and_register_eps command finishes — quickly, when EPs are already cached — the SDK disposes that CTS, but a progress callback (b__9) is still queued on the ThreadPool. When that queued callback runs, it touches the now-disposed CTS, throwing ObjectDisposedException. Since it runs as a ThreadPool work item with no handler, the runtime terminates the process.
The throw happens inside the SDK's own wrapper (b__9), before control reaches the app-supplied (epName, pct) callback, so the calling application cannot observe or handle it (it is not surfaced through AppDomain.UnhandledException in a way that can prevent termination).
Note: appears already addressed on main
This managed-interop code path does not appear to exist in the current open-source tree — ExecuteCommandWithCallbackManaged, ExecuteWithTracker, ExecuteCommandManaged, and a Progress<T>-based NativeInterop all return no hits — and the current EP-download paths use disposal-safe patterns:
sdk/cs routes EP progress through a plain string ICoreInterop.CallbackFn (no Progress<T>).
sdk_v2/cs checks ct?.IsCancellationRequested (safe after dispose) inside the native callback.
So this looks specific to the compiled 1.2.x WinML assembly and likely resolved by the SDK v2 / unified-package refactor.
Ask
- Can you confirm this
Progress<T> EP-progress / disposed-CTS race is gone in the unified/v2 package?
- Would you consider a 1.2.x servicing backport — e.g. guard the progress handler so it cannot touch the CTS after disposal, or defer disposing the per-command CTS until all queued
Progress<T> callbacks have drained?
Thanks!
Summary
An intermittent, unhandled
System.ObjectDisposedException: The CancellationTokenSource has been disposed.is thrown on a ThreadPool thread from inside the SDK's own progress-callback marshaling duringFoundryLocalManager.DownloadAndRegisterEpsAsync(...). Because it is unhandled on a ThreadPool thread, it fail-fasts the entire host process (0xc0000409). The calling application has no way to catch it.Environment
Microsoft.AI.Foundry.Local.WinML1.2.3 — faulting moduleMicrosoft.AI.Foundry.Local.Core.dll, assembly/file version 1.2.0.0net10.0-windows(the.NET Runtimecrash event reportedCoreCLR Version: 9.0.17)What the app was doing
One-time initialization. After
FoundryLocalManager.CreateAsync(...)we register the hardware execution providers with a progress callback, then fetch the catalog:The crash occurs during this call. It is intermittent — in our logs, EP registration completed successfully 23 times and fail-fasted once. It appears most likely when the EPs are already cached, i.e. the
download_and_register_epscommand returns very quickly.Crash details
Windows Error Reporting — Application Error (Event ID 1000):
.NET Runtime(Event ID 1026):Root cause (analysis)
ExecuteCommandWithCallbackManagedmarshals native progress updates through aSystem.Progress<T>, whose handler invocations are posted to the ThreadPool (theProgress<T>is constructed on a background thread with noSynchronizationContext). The command also uses a per-commandCancellationTokenSource.When the
download_and_register_epscommand finishes — quickly, when EPs are already cached — the SDK disposes that CTS, but a progress callback (b__9) is still queued on the ThreadPool. When that queued callback runs, it touches the now-disposed CTS, throwingObjectDisposedException. Since it runs as a ThreadPool work item with no handler, the runtime terminates the process.The throw happens inside the SDK's own wrapper (
b__9), before control reaches the app-supplied(epName, pct)callback, so the calling application cannot observe or handle it (it is not surfaced throughAppDomain.UnhandledExceptionin a way that can prevent termination).Note: appears already addressed on
mainThis managed-interop code path does not appear to exist in the current open-source tree —
ExecuteCommandWithCallbackManaged,ExecuteWithTracker,ExecuteCommandManaged, and aProgress<T>-basedNativeInteropall return no hits — and the current EP-download paths use disposal-safe patterns:sdk/csroutes EP progress through a plain stringICoreInterop.CallbackFn(noProgress<T>).sdk_v2/cschecksct?.IsCancellationRequested(safe after dispose) inside the native callback.So this looks specific to the compiled 1.2.x WinML assembly and likely resolved by the SDK v2 / unified-package refactor.
Ask
Progress<T>EP-progress / disposed-CTS race is gone in the unified/v2 package?Progress<T>callbacks have drained?Thanks!