Hello,
I would like to confirm whether Proxmox Backup Server currently has any configuration option to limit or throttle the request rate when using the S3 backend.
This is not intended as a feature request. I would first like to know whether such a setting already exists, or whether there is any recommended configuration on the PBS side for this situation.
I am testing backups from Proxmox Backup Server to an S3-compatible object storage provider. When backing up a VM, the backup sometimes fails with 429 Too Many Requests.
According to the object storage provider’s documentation, the cause seems clear:
- The provider enforces a limit of 100 requests per second.
- If this limit is exceeded, the service returns 429 Too Many Requests.
- The provider cannot relax this limit.
- They recommend that clients limit the number of requests per second, queue requests, or implement similar client-side handling.
A small LXC backup of around 8 GB succeeds, so I assume the object storage provider is not rejecting PBS backups in general. The issue seems to occur only when the backup generates enough chunk upload requests to exceed the provider’s request rate limit.
Environment:
Bash:
root@pvenode-hv01:~# proxmox-backup-manager versions --verbose
proxmox-backup unknown running kernel: 7.0.6-2-pve
proxmox-backup-server 4.2.1-1 running version: 4.2.1
proxmox-kernel-helper 9.2.0
proxmox-kernel-7.0 7.0.6-2
proxmox-kernel-7.0.6-2-pve-signed 7.0.6-2
proxmox-kernel-7.0.2-6-pve-signed 7.0.2-6
ifupdown2 3.3.0-1+pmx12
libjs-extjs 7.0.0-5
proxmox-backup-docs 4.2.1-1
proxmox-backup-client 4.2.1-1
proxmox-mail-forward 1.0.3
proxmox-mini-journalreader 1.6
proxmox-offline-mirror-helper 0.7.4
proxmox-widget-toolkit 5.2.3
pve-xtermjs 6.0.0-1
smartmontools 7.5-pve2
zfsutils-linux 2.4.2-pveI have attached the logs for the backup tasks on Proxmox VE and the backup server.
Thank you.