Tidehunter: Sui’s Next-Generation Database Optimized For Low Latency And Reduced Write Amplification

by
Alisa Davidson

Printed: February 03, 2026 at 2:10 am Up to date: February 03, 2026 at 2:10 am

by Victor Dey

Edited and fact-checked:
February 03, 2026 at 2:10 am

To enhance your local-language expertise, typically we make use of an auto-translation plugin. Please notice auto-translation is probably not correct, so learn original article for exact data.

Sui, a Layer 1 blockchain community, has launched Tidehunter, a brand new storage engine engineered to align with the efficiency calls for, knowledge entry traits, and operational constraints generally present in modern blockchain infrastructures. 

The system is positioned as a possible successor to the prevailing database layer utilized by each validators and full nodes, reflecting a broader effort to modernize core infrastructure in response to the evolving scale and workload profiles of manufacturing blockchain environments.

Sui initially relied on RocksDB as its major key–worth storage layer, a broadly adopted and mature answer that enabled speedy protocol improvement. Because the platform expanded and operational calls for elevated, basic limitations of general-purpose LSM-tree databases grew to become more and more obvious in production-like environments. 

Intensive tuning and deep inside experience couldn’t totally deal with structural inefficiencies that conflicted with the entry patterns typical of blockchain methods. This led to a strategic shift towards designing a storage engine optimized particularly for blockchain workloads, ensuing within the improvement of Tidehunter.

A central issue behind this resolution was persistent write amplification. Measurements beneath life like Sui workloads confirmed amplification ranges of roughly ten to 12 instances, which means that comparatively small volumes of utility knowledge generated disproportionately giant quantities of disk site visitors. Whereas such habits is frequent in LSM-based methods, it reduces efficient storage bandwidth and intensifies rivalry between background compaction and skim operations. In write-intensive or balanced read-write environments, this overhead turns into more and more restrictive as throughput scales. 

Load testing on high-performance clusters confirmed the influence, with disk utilization nearing saturation regardless of average utility write charges, highlighting the rising mismatch between standard storage architectures and fashionable blockchain efficiency necessities.

Tidehunter Structure: A Storage Engine Optimized For Blockchain Entry Patterns And Sustained Excessive-Throughput Workloads

Storage habits in Sui and comparable blockchain platforms is dominated by a small set of recurring knowledge entry patterns, and Tidehunter is architected particularly round these traits. A big portion of state is addressed utilizing cryptographic hash keys which can be evenly distributed and usually map to comparatively giant data, which removes locality however simplifies consistency and correctness. 

On the similar time, blockchains rely closely on append-oriented constructions, akin to consensus logs and checkpoints, the place knowledge is written so as and later retrieved utilizing monotonically growing identifiers. These environments are additionally inherently write-heavy, whereas nonetheless requiring quick entry on latency-critical learn paths, making extreme write amplification a direct risk to each throughput and responsiveness.

On the heart of Tidehunter is a high-concurrency write pipeline constructed to take advantage of the parallel capabilities of recent solid-state storage. Incoming writes are funneled by way of a lock-free write-ahead log able to sustaining extraordinarily excessive operation charges, with rivalry restricted to a minimal allocation step. 

Information copying proceeds in parallel, and the system avoids per-operation system calls by utilizing writable memory-mapped information, whereas sturdiness is dealt with asynchronously by background companies. This design produces a predictable and extremely parallel write path that may saturate disk bandwidth with out turning into constrained by CPU overhead.

Lowering write amplification is handled as a major architectural goal moderately than an optimization step. As an alternative of utilizing the log as a brief staging space, Tidehunter shops knowledge completely in log segments and builds indexes that reference offsets immediately, eliminating repeated rewrites of values. 

Indexes are closely sharded to maintain write amplification low and to extend parallelism, eradicating the necessity for conventional LSM-tree constructions. For append-dominated datasets, akin to checkpoints and consensus data, specialised sharding methods maintain latest knowledge tightly grouped in order that write overhead stays steady at the same time as historic knowledge grows.

For tables addressed by uniformly distributed hash keys, Tidehunter introduces a uniform lookup index optimized for predictable, low-latency entry. Quite than issuing a number of small and random reads, the index reads a barely bigger contiguous area that statistically incorporates the specified entry, permitting most lookups to finish in a single disk spherical journey. 

This strategy intentionally trades some learn throughput for decrease and extra steady latency, a tradeoff that turns into sensible as a result of decreased write amplification frees substantial disk bandwidth for learn site visitors. The result’s extra constant efficiency on latency-sensitive operations akin to transaction execution and state validation.

To additional management tail latency at scale, Tidehunter combines direct I/O with application-managed caching. Massive historic reads bypass the working system’s web page cache to stop cache air pollution, whereas latest and regularly accessed knowledge is retained in user-space caches knowledgeable by application-level entry patterns. Together with its indexing format, this reduces pointless disk spherical journeys and improves predictability beneath sustained load.

Information lifecycle administration can be simplified. As a result of data are saved immediately in log segments, eradicating out of date historic knowledge might be carried out by deleting whole log information as soon as they fall outdoors the retention window. This avoids the advanced and I/O-intensive compaction mechanisms required by LSM-based databases and allows sooner, extra predictable pruning at the same time as datasets broaden.

Throughout workloads designed to replicate actual Sui utilization, Tidehunter demonstrates larger throughput and decrease latency than RocksDB whereas consuming considerably much less disk write bandwidth. Probably the most seen enchancment comes from the close to elimination of write amplification, which permits disk exercise to extra intently match application-level writes and preserves I/O capability for reads. These results are noticed each in managed benchmarks and in full validator deployments, indicating that the features lengthen past artificial testing.

Analysis is carried out utilizing a database-agnostic benchmark framework that fashions life like mixes of inserts, deletions, level lookups, and iteration workloads. Checks are parameterized to replicate Sui-like key distributions, worth sizes, and read-write ratios, and are executed on {hardware} aligned with advisable validator specs. Beneath these circumstances, Tidehunter constantly sustains larger throughput and decrease latency than RocksDB, with the most important benefits showing in write-heavy and balanced situations.

Validator-level benchmarks additional verify the outcomes. When built-in immediately into Sui and subjected to sustained transaction load, methods utilizing Tidehunter preserve steady throughput and decrease latency at working factors the place RocksDB-backed deployments start to undergo from rising disk utilization and efficiency degradation. Measurements present decreased disk strain, steadier CPU utilization, and improved finality latency, highlighting a transparent divergence in habits beneath comparable load.

Tidehunter represents a sensible response to the operational calls for of long-running, high-throughput blockchain methods. As blockchains transfer towards sustained moderately than burst-driven workloads, storage effectivity turns into a foundational requirement for protocol efficiency. The design of Tidehunter displays a shift towards infrastructure constructed explicitly for that subsequent stage of scale, with additional technical element and deployment plans anticipated to comply with.

Alisa, a devoted journalist on the MPost, makes a speciality of cryptocurrency, zero-knowledge proofs, investments, and the expansive realm of Web3. With a eager eye for rising tendencies and applied sciences, she delivers complete protection to tell and have interaction readers within the ever-evolving panorama of digital finance.

• 
  
  

More articles