The part of memory Java doesn't tell you about

Java doesn't just use heap. The JVM also allocates:

• Metaspace — class metadata, loaded by the JVM itself
• Code cache — JIT-compiled native code
• Thread stacks — each thread gets its own
• Direct byte buffers (NIO) — allocated off-heap by many libraries
• Internal JVM bookkeeping

This is called native memory, and it's invisible to your usual heap monitoring. When your container hits its cgroup memory limit, the kernel doesn't care how much heap you have left — it kills the process when the total RSS exceeds the limit.

A 512MB container running a JVM with 256MB heap can easily OOM at around 350–400MB total RSS, because metaspace, code cache, and buffers have already eaten into the headroom you didn't know you needed.

The fix nobody explains properly

The old way: -Xms256m -Xmx256m. Fixed heap size, ignores container limits.

The better way:

-XX:MaxRAMPercentage=75.0

This tells the JVM to size the heap as a percentage of the container's actual memory limit, not some fixed number. If your container has 512MB, the heap gets roughly 384MB. The remaining ~128MB is left for native memory, JIT overhead, and everything else the JVM allocates outside the heap.

For most workloads, 75% is a reasonable starting point. If you're running into native memory pressure (you'll see it in jcmd VM.native_memory), dial it down to 70%.

A few other flags worth knowing:

# Pre-touch heap pages at startup instead of on first access
-XX:+AlwaysPreTouch

# Cap metaspace growth so it can't run away
-XX:MaxMetaspaceSize=256m

AlwaysPreTouch is a tradeoff — it increases startup time but prevents those surprise OOMs when a traffic spike touches cold heap pages for the first time.

How to actually see what's happening

Heap usage comes from your app, but native memory is opaque by default. Enable native memory tracking:

-XX:NativeMemoryTracking=detail

Then query it at runtime:

jcmd <pid> VM.native_memory summary

Output looks like:

Native Memory Tracking:
Total: reserved=618MB, committed=412MB
- Heap         : 256MB reserved, 180MB committed
- Class        :  45MB reserved,  38MB committed
- Thread       :  12MB reserved,  12MB committed
- Code         :  28MB reserved,  22MB committed
- Internal     :   8MB reserved,   8MB committed

That's the total picture. Watch the "committed" column under Heap against the overall RSS — if RSS is consistently 100–150MB above committed heap, that's native overhead you need to account for when sizing your container.

The short version

Your container limit needs to cover heap plus native memory. If you only tune the heap, you're flying blind. Switch to -XX:MaxRAMPercentage, enable NativeMemoryTracking so you can actually see what's being used, and you'll stop getting OOMs when heap looks fine.

It's a 15-minute change and it eliminates one of those "but the monitoring said we had headroom" incidents that show up at 2am.