Is it safe to have multiple Executors.newCachedThreadPool() in a Java program?

Question

The spec for this method: https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Executors.html#newCachedThreadPool()

Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.

It's not clear to me from this description - is it safe to have several of these pools in a single program? Or would I potentially run into a situation where one pool stalls on many threads and freezes up other pools?

Answer 1

I don't think there is a clear yes / no answer on this.

On the one hand, there is not a finite number of threads that ThreadPoolExecutor instances consume. The JVM architecture itself doesn't limit the number of threads.

On the the second hand, the OS / environment may place some limits:

• The OS may have hard limits on the total number of native threads it will support.

• The OS may restrict the number of native threads that a given process (in this case the JVM) can create. This could be done using ulimit or cgroup limits, and potentially other ways.

• A Java thread stack has a size of 1MB (by default) on a typical 64 bit JVM. If you attempt to start() too many threads, you may run out of memory and get an OOME.

• If there are a large enough number of threads and/or too much thread context switching, the thread scheduler (in the OS) may struggle.

  (Context switching typically happens when a thread does a blocking syscall or has to wait on a lock or a notification. Each time you switch context there are hardware related overheads: saving and restoring registers, switching virtual memory contexts, flushing memory caches, etc.)

On the third hand, there are other things than just the number and size of thread pools that could cause problems. For example, if the thread tasks interact with each other, you could experience problems due to:

• deadlocking when locking shared objects,
• too much contention on shared locks leading to resource starvation,
• too much work leading to timeouts, or
• priority inversion problems ... if you try to use priorities to "manage" the workload.

So ...

Is it safe to have several of these pools in a single program?

Or would I potentially run into a situation where one pool stalls on many threads and freezes up other pools.

It is unlikely you would get a "stall" ... unless the tasks are interacting in some way.

But if you have too many runnable threads competing for CPU, each one will get (on average) a smaller share of the finite number of cores available. And lock contention or too much context switching can slow things down further.