TrueNAS uses OpenZFS as the foundation for its data management layer and is the deployment vehicle for the majority of OpenZFS storage systems used today. Here at TrueNAS, we love OpenZFS, and it continues to improve with the branching of OpenZFS 2.3 on October 4th, 2024.

The TrueNAS Engineering team has been significantly contributing to the codebase of OpenZFS 2.3, and Electric Eel (TrueNAS 24.10) includes several new and long-anticipated OpenZFS 2.3 features. The current development version of TrueNAS, “Fangtooth”, aligns with the full OpenZFS 2.3 release and will use this version of OpenZFS throughout its version lifecycle.

With every new version of both TrueNAS and OpenZFS, additional features, test cases, and bug fixes are included. The previous OpenZFS 2.2 brought dRAID and block cloning to TrueNAS Dragonfish (24.04) and CORE 13.3. TrueNAS 24.10 adds a number of new features, including two highly anticipated enhancements: Fast Dedup and RAIDZ expansion.

This blog highlights these new capabilities and their status within TrueNAS.

Fast Dedup Development is Complete

Deduplication is highly desirable for many workloads, including virtualization and several file storage use cases. Where there is naturally a high ratio of redundant data within a pool, deduplication effectively increases the usable capacity of the drives and the efficiency of the Adaptive Replacement Cache (ARC) and Level 2 ARC (L2ARC).

One of the primary challenges with traditional ZFS deduplication has been keeping the large deduplication metadata tables in memory at all times to avoid significant performance penalties. This existing functionality was not performant and led to usability issues during operation. As the size of the deduplication tables increased, the ZFS ARC would shrink, and performance would degrade as the pool filled, leading to a poor user experience and challenges when scaling to higher storage capacities.

With the inclusion of Fast Dedup, the metadata size can now be automatically constrained to fit in either primary RAM or dedicated flash devices to avoid hitting the performance penalty wall. In addition, the metadata structure for Fast Dedup has been completely re-engineered to enable efficient updates and the ability to prune non-duplicate blocks, effectively shrinking the memory footprint of the deduplication tables by 90% in many cases.

Combining these metadata improvements with properly configured storage will improve deduplication performance by an order of magnitude for larger systems. Performance is more stable as the pool is filled, leading to predictable behavior and enhanced space-efficiency.

This Fast Dedup project started in 2023 and was committed to the OpenZFS project as a “Valentine’s Day Gift” in 2024. Allan Jude and Klara Systems collaborated with Alexander Motin and the TrueNAS Engineering team along with members of the OpenZFS community, and development was completed in September of this year. We appreciate the hard work and dedication shown by all contributors and testers to help bring this project through to completion.

With development completed, Fast Dedup is now ready for testing but not yet suggested for serious production use. Within TrueNAS, it is marked as Experimental. We expect to provide test results along with any necessary code improvements in early 2025.

RAIDZ Expansion is Finally Available

A much-anticipated feature for smaller systems and home users of TrueNAS, RAIDZ expansion allows a small pool (e.g., a single RAIDZ vdev) to be gradually expanded with one drive at a time. Existing data is preserved with its original parity level and rewritten across all drives, while new data is written with the new parity configuration. This simplified administrative process gives smaller TrueNAS systems the flexibility to expand in single drive increments, rather than adding a full vdev of drives. The same expansion feature works regardless of the parity level used – RAIDZ1, Z2, or Z3 – but cannot migrate between protection levels.

The expansion process is done while the ZFS pool is online, similar to the resilvering process when a drive fails and is replaced. Once completed, the larger pool’s full performance is available. The new disk is used immediately, with additional capacity being reclaimed as existing data is rewritten.

This project took several years to complete and test, will be included in OpenZFS 2.3, and is available now in TrueNAS 24.10. TrueNAS sponsored this work to benefit smaller systems and is fully supported within TrueNAS Electric Eel.

OpenZFS Direct IO Improves NVMe Performance

Direct IO is one of the latest features included in OpenZFS 2.3 and was provided by Los Alamos Labs. It provides the option to bypass the ARC when storing data directly on NVMe drives. Removing memory copies can increase a system’s bandwidth by over 30%. The primary use case for Direct IO is storing checkpoint data in High-Performance Computing (HPC) clusters. This specific use case sees very little benefit from the read caching of the ZFS ARC (Adaptive Replacement Cache).

The Direct IO feature will only be available via Fangtooth. We are looking forward to testing for use cases that benefit our customers. Most workloads, however, will continue to benefit significantly from the default settings with ARC fully enabled.

What else is in TrueNAS 24.10?

In addition to the major features highlighted above, TrueNAS 24.10 includes an upgraded and improved webUI, enhancements to cloud backup integration, the replacement of Kubernetes with Docker for TrueNAS Apps, improved hardware support and drivers, and much more. For more details, see the Release Notes and join the discussion on the TrueNAS Forums, where some of the over 5,000 testers of 24.10 pre-release versions are sharing their feedback and tips.

TrueNAS 24.10 “Electric Eel” is planned for formal release near the end of October 2024. Want to learn more about TrueNAS solutions in your business? Contact us to speak to a product specialist, and find out how to harness the power of open source storage.