Thursday, May 15, 2008

JDK GC tuning for app servers

A quickie post here.

Lets say you get called in to review a server that's getting OutOfMemoryErrors

This is a bad thing, obviously! How do you fix it?

Well, it depends on the root cause, first of all you have to figure out what the problem is.

On old JVMs you turn on -verbose:gc and that's it, on newer ones you can turn on more details, some timestamps etc - all things that help you, but that's not the point of the post.

The point of this post is to send you over to get GCViewer from tagtraum at http://www.tagtraum.com/gcviewer-download.html

It's really the best option for visualizing what's going on.

It's not perfect though, and a trick I have to use nearly every time I'm visualizing the GC behavior in an app server via GCViewer, I have to filter the log file so it has nothing but GC lines:

cat $log | grep '\[' | grep GC > clean-gc.log

Then you fire GCViewer up and see what you've got

usually you get a line going "up and to the right" which is great for finance, but bad for GC signatures. That means you have a memory leak, or you have misconfigured your server to hold on to too many objects in caches, or to accept too many sessions that themselves have too many objects.

Sometimes you realize that you never run out of heap, but if you still get OOM errors, you are either running out of PermGen space or in rare cases are running out of operating system file handles by spawning too many threads (that error gets mapped to OOM, believe it or not)

Without GCViewer, it's difficult to clearly articulate the problem though, and also hard to see trends. Combined with how fast it is to use, I end up reaching for this particular utility every time.

Cheers

No comments: