Out of Memory Error when using Executors in java

Question

I am using the Executors framework(fixed thread pool with unbounded blocking queue) to execute tasks concurrently.

But when I run a load test with about 10000 tasks created, there is a huge build up of heap memory (2.1 GB) with about 3.5 million Executable objects.

I am not sure if the unbounded queue is causing this build up.

Memory Analyzer report :

One instance of "java.util.concurrent.ThreadPoolExecutor" loaded by "" occupies 2,299,506,584 (94.97%) bytes. The instance is referenced by com.test.ScheduleBean @ 0x743592b28 , loaded by "org.jboss.modules.ModuleClassLoader @ 0x741b4cc40".

Any pointers appreciated!

        //The Executors are loaded in a hashmap
        HashMap<String,Executor> poolExecutorMap = new HashMap<String,Executor>();

       //Executor is a fixed thread pool one
       Executor poolExecutor = Executors.newFixedThreadPool(threadCount);

      //then add the executor to the hashmap
       poolExecutorMap.put("Executor", poolExecutor);



    //then a list of tasks are pulled from a database and passed as runnable objects to the executors

      Class<?> monitorClass=null;

      List<Task> list = getAllTasksToProcess();

     for (int i = 0; i < list.size(); i++) {
      Task task = list.get((int) i);

     monitorClass = Class.forName(task.getTask_event_name());
        Constructor<?> ctor;
        ctor = monitorClass.getConstructor(Task.class);
        Object object = ctor.newInstance(task);
        logger.debug("Adding  task number : "+task.getTask_sequence_id());
        poolExecutorMap.get("Executor").execute((Runnable) object);
}



// the executor classes have an execute method which sends a http notification.

I don't understand how the map is used. There's appears to be only one Executor in it, with the key "Executor". But then you are get()-ing others, keyed by a class name. Where do these come from? — erickson
– erickson, Commented Apr 29, 2015 at 19:31
I have modified it a bit, assume there is only one executor with the name "Executor" — DntFrgtDSemiCln
– DntFrgtDSemiCln, Commented Apr 29, 2015 at 19:36
Take a look at the retained heap of the Task objects in MemoryAnalyzer. You might want to dig down into the object graph if the number looks high to see where the memory is used. After all, you are creating thousands of them. — Mike Tunnicliffe
– Mike Tunnicliffe, Commented Apr 29, 2015 at 20:08

Mohan Raj · Accepted Answer · 2015-04-30 10:59:05Z

2

Write OQL in MemoryAnalyzerTool

select * from java.util.concurrent.ThreadPoolExecutor

and execute the query. It will list the object in the separate window. Then right click on the instances generated then

path to GC roots --> Exclude soft/weak/phantom references

It will help you in understanding who is holding strong reference of the suspected objects.

answered Apr 30, 2015 at 10:59

Mohan Raj

1,1321 gold badge9 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

DntFrgtDSemiCln · Accepted Answer · 2015-04-30 15:02:23Z

0

Thanks for all the responses.

The real problem was the way tasks were being pulled from the database. Duplicate tasks were being added to the task queue and hence the queue build up.

answered Apr 30, 2015 at 15:02

DntFrgtDSemiCln

1,2892 gold badges18 silver badges35 bronze badges

Collectives™ on Stack Overflow

Out of Memory Error when using Executors in java

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related