Summary: VictoriaMetrics is optimized to run more effectively on ARM hardware by introducing a CGO build and fine-tuning its compression library usage, narrowing the performance gap with x86 systems. While x86 still maintains an advantage in query latency, the enhancements make ARM processor performance with VictoriaMetrics much more cost-efficient and competitive.
ARM processors become more popular and more cost-effective according to many benchmarks. One of them was made by Percona for MySQL.
Some of our users reported issues with VictoriaMetrics at AWS Graviton instances. The main concerns were higher CPU and disk IO usage compared to x86 instances of the same size and for the same workload.
By that time, we verified that VictoriaMetrics works fine for raspberry and IoT devices, but didn’t do any optimizations for ARM builds.
The main difference between x86 and ARM builds is the library we use for data encoding. x86-build uses gozstd library, a wrapper over Facebook’s zstd written in C.
Cross-compiled ARM64 build uses compress library by Klaus Post written in native Go.
So in many aspects, the performance difference between x86-build and cross-compiled ARM64 build heavily depends on the performance of these libraries.
That’s why, in order to improve performance of the ARM builds we added CGO build with some tweaks.
For my test I created 3 instances at AWS:
m5.4xlarge - intel x86 based instances with 16 CPU and 64 RAM $0.768 hour
to run x86 build of VictoriaMetrics.m6g.4xlarge - graviton2 based instance with 16 CPU and 64 RAM $0.61 hour
to run cross-compiled ARM64 build without CGO.m6g.4xlarge - graviton2 based instance with 16 CPU and 64 RAM $0.616 hour
to run CGO build for ARM64.And installed vmsingle v1.72.0 on each of the instances.
For workload generation, I’ve used our benchmark suite and set up a separate vmsingle node for metrics collection.
Initial CPU profiling proves this theory and shows performance improvements with gozstd lib:
CPU usage by VictoriaMetrics before optimizations
CPU usage by VictoriaMetrics after optimizations
Optimized ARM and x86 versions show almost the same result for disk IO usage:
Disk writes/reads during the benchmark
Query performance for x86 version outperforms optimized ARM by ~10% and unoptimized ARM by ~25%:
Query latency during the benchmark
One of major challenges was to add CGO build into our cross-compilation pipeline. We are using musl based builds and the default musl compiler isn’t aware of how to build code for ARM. Instead, special aarch64-musl-gcc compiler must be used:
CC=/path_to_folder/bin/aarch64-linux-musl-gcc \
GOOS=linux GOARCH=arm64 CGO_ENABLED=1 go build main.go
Important note, your C-lang dependencies must be built with the same compiler. In my case, I had to rebuild gozstd lib.
Yes. In our benchmark, x86 still had about 10% lower query duration than optimized ARM. For read-heavy workloads where query latency is the top priority, x86 may still have an edge, though ARM delivered strong cost-performance overall.
We recommend testing both with your own workload. ARM can offer better cost-performance, especially on AWS Graviton, while x86 may be preferable for query-latency-sensitive workloads. Compare ingestion, queries, disk IO, and total instance cost.
The main difference is compression. x86 builds used gozstd, while cross-compiled ARM64 builds used the native Go compress library. Much of the performance gap came from these encoding and compression differences.
For best ARM performance, we added CGO builds with zstd-related optimizations. When building manually, use ARM64 CGO with the correct aarch64 musl compiler and rebuild C dependencies with the same compiler.
We continue to improve performance where possible. Since v1.73.0, we have provided production-ready ARM builds, prebuilt binaries, and Docker images, making ARM deployments easier and more practical.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。