How Microsoft Organizes Data in Multi-Tier Storage Systems

A method for organizing computer data into three specific tiers—log, hash, and journal stores—to make writing and reading data faster and more efficient.

What it covers

This patent describes a way to manage data storage by moving information through three distinct stages. First, incoming data is written into a 'log store,' which acts as a quick, temporary landing zone. Periodically, these logs are converted into a 'hash store,' where metadata is organized using a specific structure that includes hash portions and offsets. Finally, multiple hash stores are merged into a 'journal store,' which acts as a more permanent, indexed archive. By using this tiered approach, the system reduces the amount of work the storage hardware has to do when writing data, which improves overall speed and efficiency.

What it doesn't cover

• —Does not cover simple file systems that write directly to a disk without a tiered log-to-hash-to-journal conversion process.
• —Does not cover storage systems that do not utilize the specific metadata and hash-offset structure defined in claim 1.
• —Does not cover data storage that relies solely on traditional relational database indexing methods without the defined multi-tier conversion steps.

The clever bit

The system uses a specific conversion process that calculates disk offsets for metadata before writing, allowing it to merge multiple smaller data stores into a larger, indexed journal without needing to rewrite the entire data set from scratch.

Why it matters

As data centers handle massive amounts of information, 'write amplification'—where a small change requires writing a large amount of data—becomes a major bottleneck. This patent provides a specific architectural solution to minimize that overhead. It is significant for cloud infrastructure providers like Microsoft Azure who need to maximize the throughput of solid-state drives (SSDs).

Real-world examples

• 1.Cloud-based storage backends
• 2.High-performance solid-state drive (SSD) controllers
• 3.Enterprise-grade database storage engines