JVM: Non-heap memory causes OutOfMemoryError because the Garbage Collector is not run. What am I doing wrong?

Question

I've encountered a weird issue where the Groovy application I'm working on grows to consume more memory (far beyond the limits of the xmx-argument, so it can't be the heap) until the computer runs out of RAM, at which point the JVM behaves in one of two different ways - it either suddenly frees (almost) all the memory it has taken, or it crashes with an OutOfMemoryError. This problem can be avoided by regularly calling System.gc().

What seems to happen to me is, despite allocating more and more memory, the JVM does not call the Garbage Collector. It is not clear whether it always (tries to) call it once the computer runs out of RAM and only sometimes succeeds, or whether it sometimes throws an OOME without calling the GC (even though that would violate the specifications). It might be interesting to note that the ResourceMonitor does not report the java.exe instance commiting more memory (it stays at ~500MB), but the commit charge goes up all the same.

The only thing I'm doing while this is going on is to have a Timer start a new thread every 33ms to call a repaint() of a JComponent. I have heard that each new thread is allocated some memory outside the heap, so I suspect that the issue may be that that memory is never collected, but I could be wrong (I really feel out of my depth here, TBH).

I could obviously solve things by just having a Timer invoke System.gc() regularly (though no less often than once every few seconds), but that seems very bad practice to me, and I'm honestly hoping that there is something I'm doing wrong, rather than this being some weird issue with the JVM.

The only code I'm running at the time is the one below (I've removed some comments and some logging to the console). There is, of course, a whole bunch more code, but the only thing active is, as mentioned, the Timer calling repaint().

//snip: package and imports
    
@groovy.transform.CompileStatic
class MapWindow extends BasicWindow {
//BasicWindow provides a constructor that stores its two arguments as windowX and windowY (and set dimensions accordingly) and creates and stores a basic frame
//It also overrides setVisible() to call frame.setVisibile() as well. It does nothing else.

    int xPos
    int yPos

    MapWindow(int x, int y) {
        super(x, y)
        frame.setTitle("EFG")
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE)
        frame.pack()
    }

    @Override
    public void paint(Graphics gA) {
        VolatileImage img = createVolatileImage(windowX, windowY)
        Graphics2D g = (Graphics2D) img.getGraphics()

        VolatileImage tmp = getTestImage()

        g.drawImage(tmp, 0, 0, null)
        g.dispose()
        gA.drawImage(img, 0, 0, windowX, windowY, null)

        if (Game.game.rnd.nextInt(100) == 0) {
            //System.gc()  <--If I uncomment this, things work
        }
    }

    VolatileImage getTestImage() {
        VolatileImage img = createVolatileImage(windowX, windowY)
        Graphics2D g = img.createGraphics()

        for (int x = 0; x < tileSet.x; x++) {
            for (int y = 0; y < tileSet.y; y++) {
                g.drawImage(tileSet.images[x][y], x * tileSet.resolution, y * tileSet.resolution, null)
            }
        }

        char[] msg = "test complete".toCharArray()
        for (int x = 0; x < msg.length; x++) {
            char c = msg[x]
            if (!c.isWhitespace()) {
                g.drawImage(tileSet.getImage("symbol.$c"), x * tileSet.resolution, tileSet.resolution * tileSet.y, null)
            }
        }

        g.dispose()

        return img
    }
}

    //Located in a different class, called once during startup. It also subject to a @CompileStatic
    void startTimer() {
        timer = new java.util.Timer()
        int period = config.getInt("framePeriod")
        boolean fixed = config.getBoolean("frameFixedRate")
        if (fixed) {
            timer.scheduleAtFixedRate(new TimerTask() {
                @Override
                public void run() {
                    activeWindow?.frame?.repaint()
                }
            }, period, period)
        } else {
            timer.schedule(new TimerTask() {
                @Override
                public void run() {
                    activeWindow?.frame?.repaint()
                }
            }, period, period)
        }
    }

I can provide more code/info if needed, but I didn't want to clog this up by essentially posting the entire program. It seems exceedingly likely the issue is somewhere here, probably in paint() or getTestImage() (or the JVM).

Windows 7 64-bit
16GB RAM, no pagefile
SDK 1.8.0_25 (issue also confirmed with 13.0.1)
Groovy 2.5.8
I use IntelliJ IDEA, but the issue also occurs if I build a .jar and run it independently.

EDIT: ewramner has pointed out that I should call flush() on (or reuse) the VolatileImages, so I've accepted that as the solution. I'd still be interested if anyone could explain why the GC doesn't act earlier, though, especially if it leads to the JVM crashing with an OOME.

Answer 1

If you read the documentation for VolatileImage it says:

When a VolatileImage object is created, limited system resources such as video memory (VRAM) may be allocated in order to support the image. When a VolatileImage object is no longer used, it may be garbage-collected and those system resources will be returned, but this process does not happen at guaranteed times. Applications that create many VolatileImage objects (for example, a resizing window may force recreation of its back buffer as the size changes) may run out of optimal system resources for new VolatileImage objects simply because the old objects have not yet been removed from the system.

The solution is to call flush (where you call System.gc) or perhaps to reuse the image instead of re-creating it for every paint operation.