Proxmox newbie here, although I have been using it for Home Assistant for a couple of years, but almost a set and forget aside from updates. I recently moved Omada from my Synology NAS to Proxmox on a newish Lenovo mini because the new Omada versions won't run on a Celeron CPU. The used memory just keeps rising to the point where it maxes out and the CPU then jumps from 1.5% to 100%. A reboot brings it back to earth and it starts again.

I've looked at several threads including:

Jan 22 15:12:16 pve kernel: immich-api invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Jan 22 15:12:16 pve kernel: CPU: 2 UID: 100999 PID: 2133645 Comm: immich-api Tainted: P           O        6.17.4-2-pve #1 PREEMPT(voluntary)
Jan 22 15:12:16 pve kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
Jan 22 15:12:16 pve kernel: Call Trace:
Jan 22 15:12:16 pve kernel:  <TASK>
Jan 22 15:12:16 pve kernel:  dump_stack_lvl+0x5f/0x90
Jan 22 15:12:16 pve kernel...

In line with that thread, I have increased the RAM and Swap File size, but the trend is still up.
The Proxmox overview :

My Container Summary:
CPU usage 1.08% of 2 CPU(s)
Memory usage 57.60% (2.30 GiB of 4.00 GiB)
SWAP usage 0.00% (0 B of 1.00 GiB)

Memory use started at about 1.96gb. The memory limit was set at 3gb and Swap 512mb. It would fill up in about 36 hours and the swap was running at about 15% (not 0% like now)

In the past ten minutes the top -co%MEM result has gone from
MiB Mem : 4096.0 total, 1208.4 free, 2339.9 used, 547.8 buff/cache
MiB Swap: 1024.0 total, 1024.0 free, 0.0 used. 1756.1 avail Mem
to
MiB Mem : 4096.0 total, 1199.7 free, 2344.3 used, 552.1 buff/cache
MiB Swap: 1024.0 total, 1024.0 free, 0.0 used. 1751.7 avail Mem

At first I thought it was the issue with the Intel NIC offloading and tried the offloading fix but that did not help the memory issue. I have PBS running in a VM alongside the Omada container and it is fine.

I am assuming this is an Omada issue, not a Proxmox issue, but grateful for any help. Otherwise it's a reboot every two days.

Omada is a software controller for TP-Link's range of Omada switches, router, APs etc.

It was running in Synology's version of Docker. It was v5 which uses an older version of the MongDB. Omada v6 needs a new version of MongoDB and that will not run in a Celeron, hence the move to the i5. The v5 version was running quite happily on a small allocation of RAM and I thought was quite low weight. v6 is a fairly big update. It ran without issue in the Synology "Container Manager".

BTW - since my original post memory use has jumped to 71%, including a big hop up 90 minutes ago. I can't see anything the logs to show why.

SIgh.......

- thanks for those links. I had not seen them. I might try an earlier version if I can work out how to do that.

I'd been hoping that Docker was immune but not so.

Sorry - formatting fixed. On start parameters I don't know how to get/display that unless it is these:
arch: amd64
cores: 2
hostname: omada
memory: 4096
net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:56:F4:19,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-101-disk-0,size=8G
swap: 512
tags: community-script;controller;tp-link
timezone: Australia/Brisbane
unprivileged: 1

[ATTACH type="full" width="523px" size="709x480"]96968[/ATTACH]

In case anyone's curious about those parameters

# pgrep -af omada
6505 omada -cwd /opt/tplink/EAPController/lib -pidfile /var/run/omada.pid -home /usr/lib/jvm/temurin-21-jdk-amd64/ -cp /usr/share/java/commons-daemon.jar:/opt/tplink/EAPController/lib/*:/opt/tplink/EAPController/properties -outfile /opt/tplink/EAPController/logs/startup.log -errfile /opt/tplink/EAPController/logs/startup.log -user omada -procname omada -showversion -server -XX:MaxHeapFreeRatio=60 -XX:MinHeapFreeRatio=30 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tplink/EAPController/logs/java_heapdump.hprof -Djava.awt.headless=true -Djdk.lang.Process.launchMechanism=vfork com.tplink.smb.omada.starter.OmadaLinuxMain start

I tried to reproduce this (I hate working with these scripts) but if there's a leak it's not very much. The helper script doesn't appear to modify the upstream code so you probably have to wait for TP Link to fix this or restore backup of a older version or manually install it.

In case anyone's curious about those parameters

Bash:

# pgrep -af omada
6505 omada -cwd /opt/tplink/EAPController/lib -pidfile /var/run/omada.pid -home /usr/lib/jvm/temurin-21-jdk-amd64/ -cp /usr/share/java/commons-daemon.jar:/opt/tplink/EAPController/lib/*:/opt/tplink/EAPController/properties -outfile /opt/tplink/EAPController/logs/startup.log -errfile /opt/tplink/EAPController/logs/startup.log -user omada -procname omada -showversion -server -XX:MaxHeapFreeRatio=60 -XX:MinHeapFreeRatio=30 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tplink/EAPController/logs/java_heapdump.hprof -Djava.awt.headless=true -Djdk.lang.Process.launchMechanism=vfork com.tplink.smb.omada.starter.OmadaLinuxMain start

It appears to be claiming anything up to 0.25gb every two hours, so I have allocated more memory for headroom. But I've also set up a crontab to reboot the container every 24 hours, so that is a band-aid until TP-Link comes up with a solution. It has been raised with them and they have acknowledged it. I'll just hurry up and wait.

The memory growth is almost certainly the JVM not respecting the LXC container's memory limit. By default, Java allocates its heap based on what it sees as the system's total RAM. In an LXC container, older JVM versions (and even some newer ones depending on cgroup configuration) see the host's full memory rather than the container's allocated amount, so they happily use far more than the container allows.

Temurin 21 should support cgroup v2 memory detection, but in Proxmox LXC containers this can still be unreliable depending on the cgroup delegation setup. The safest fix is to explicitly cap the JVM heap.

Edit /opt/tplink/EAPController/jre/bin/java.conf (or wherever the Omada startup parameters are defined) and add:

This caps the heap at 512MB and starts at 256MB. Adjust based on your network size, for a small home network with under 20 devices 512MB is plenty.

If that file does not exist or is not read, you can modify the systemd service or the startup wrapper directly. For the Docker/LXC based Omada installs, the environment variable JAVA_OPTS works:

export JAVA_OPTS="-Xmx512m -Xms256m"

Then restart the Omada service. The memory usage should now stay within the cap rather than climbing indefinitely.

Also worth checking: MongoDB (which Omada uses for its database) can be equally greedy with its WiredTiger cache. If you see mongod consuming memory alongside Java, add

storage.wiredTiger.engineConfig.cacheSizeGB: 0.25

to the MongoDB config.

Thank you @RianKellyIT

Option 1 is no go, so I'll try the environment variable JAVA_OPTS path. I assume that goes into the LXC Console, not the node Shell?

Interestingly I noticed there was an Omada update to v6.2.x. This came out around the time the memory issues were being reported to TP-Link so I don't believe they have applied a fix yet. However after the upgrade RAM growth has slowed to about 0.1gb in 24 hours, rather than 0.1gb every hour or so. I will monitor the system for another day before trying your fix.

Latest version of Omada seems to have fixed the problem. It now maxes out at 3Gb. The swap file doesn't like being constrained to 500mb, so I've upped that to 768mb and it's loitering around 310mb.

I think I will declare victory.... or at least - resolved.