What is Zoned Storage and Why Does it Matter?

A critical issue for the storage industry is coming up with innovative ways to help customers store the sheer volume, velocity, and variety of data types generated from video and the Internet of Things (IoT) to 5G, analytics, and self-driving cars. By 2024, enterprises, machines, industries, consumers, science, and more will be creating, capturing, copying, and consuming 143 zettabytes per year, according to IDC.¹ How can IT scale storage architectures and produce higher-density, TCO-optimized storage devices and related technologies to serve the Zettabyte Era? Enter Zoned Storage.

Today, large-scale data-center infrastructure is moving toward open, purpose-built, scalable solutions. I sat down with Ted Marena, senior director in the Office of the CTO at Western Digital to find out more about Zoned Storage and what it means for the data center of the future.

Senior Director, RISC-V Ecosystem & Machine Learning Business Development, Western Digital

What is Zoned Storage all about?

In a nutshell, Zoned Storage enables cloud service providers and enterprise data-center customers to architect robust, highly scalable, data-storage tiers with improved quality of service (QoS) at a competitive total cost of ownership (TCO). It’s an open-source, standards-based initiative to enable data centers to scale efficiently for the Zettabyte Era that’s upon us.

Tell us about the inception of the Zoned Storage Initiative. What’s the back story?

As organizations continue to generate vast amounts of content per year, today’s data-center infrastructures struggle to keep pace and scale economically to meet demand. New innovations were needed to help data-center architects design purpose-built solutions (versus using general-purpose compute). The Zoned Storage Initiative, announced in 2019, gives developers and architects the tools and resources to intelligently place data on the media, and optimize for better performance, higher densities, predictable QoS, and most importantly, lower TCO.

There are currently two Zoned Storage technologies—shingled magnetic recording (SMR) for HDDs and zoned namespaces (ZNS) for SSDs. Both standards are supported by a unified software stack backed by Linux. Zoned Storage brings together open-source, standards-based tools, libraries, and resources, and supports a technical documentation site, ZonedStorage.io, to enable data-center architects to understand and leverage the advantages of zone block management.

The technology behind zoned block management means both host and storage applications can intelligently place data at scale to take full advantage of the highest available storage capacities in SMR HDDs and ZNS for NVMe SSDs.

What is SMR and how does it work?

SMR enables today’s highest-capacity hard drives. SMR is patterned much like the shingles on a roof (Fig. 1). By overlapping or “shingling” the HDD data tracks on top of each other, much higher areal density and capacity can be achieved within the same footprint. Overlapping tracks are grouped into zones of fixed capacity for more effective data organization and partial update capability.

1. Host-managed shingled-magnetic-recording (SMR) hard-disk drives offer far more areal density and capacity than conventional HDDs. Data is overlapped in tracks that resemble shingles on a roof. Zones further increase the organization and capacity of a drive with the same footprint as a conventional drive.

Today’s SMR types include device-managed SMR (DMSMR) and host-managed SMR (HMSMR) technologies, and each are designed for different applications and use cases. In both cases, the data is sequentially recorded on the HDD media. Host-managed SMR drives are purpose-built for the data center and allow for optimized performance, economies of scale, and system-level intelligence of data placement.

Tell us about ZNS SSDs. How do they work?

ZNS divides flash media into zones, where each zone is an isolated namespace. These zones can only be written sequentially, much like data written sequentially to tracks on SMR HDDs. Cloud providers can, for example, separate workloads or data types to different zones so that usage patterns are predictable among multiple users or workloads. The end result is SSDs that deliver better density and are far more efficient.

For example, Zoned Storage devices reduce the need for background garbage collection and require less over-provisioning of the media without sacrificing performance, endurance, or QoS.

What’s new in the Zoned Storage Initiative?

The most recent milestone is that the ZNS command set specification has been ratified by the NVM Express consortium. This means that ZNS is now an extension of the NVMe standard. With an approved standard, ZNS-based NVMe SSDs are poised to become an integral part of the Zoned Storage device ecosystem, complementing SMR HDDs.

By enabling the sequential Zoned Storage model, ZNS allows the host and the SSD to coordinate data placement onto the SSD (Fig. 2). This provides higher write endurance, increased density, and improved I/O access latencies, while allowing technologies such as QLC NAND to proliferate.

2. Data placement in regular SSDs is sequential across tracks and is controlled by the device. ZNS SSDs feature intelligent data placement where data from each application is managed in zones for higher density and efficiency.

As data centers continues to scale, ZNS together with SMR will be critical in supporting growing storage needs.

How can companies take advantage of this new ZNS spec ratification?

Even as the ZNS Command Set specification introduces a new Zoned Storage block interface for SSDs, much of the software changes required to adopt the model are already mature. This is due to the existing SMR HDD software ecosystem accelerating the adoption of ZNS SSDs.

With a small set of changes to the software stack, users of host-managed SMR HDDs can deploy ZNS SSDs into their data centers, and new adopters can take advantage of the existing software ecosystem. They can utilize the existing open tools to accelerate support in their applications.

The unifying of the zoned block interface for both HDDs and SSDs enables software developers to support a single interface, accelerating storage deployments and ultimately taking advantage of the benefits of Zoned Storage (capacity, higher QoS, and TCO).

Why should an enterprise adopt Zoned Storage now?

Adopting a Zoned Storage approach today can enable an organization to gain a competitive advantage for a successful deployment to address the data generated from emerging workloads such as video, IoT, and self-driving cars. It may require some effort initially, but the density and TCO benefits can be substantial.

The Zoned Storage Initiative gives data-center architects the capabilities to start designing for SMR HDDs and ZNS SSDs today—not only to take advantage of the highest-capacity data-storage solutions, but to think futuristically about their long-term data strategy and optimize their workloads for higher efficiency and performance with the best possible TCO.

By leveraging a unified software stack for both SMR HDDs and ZNS SSDs, cloud providers and data-center architectures will recoup this investment and allow scale for future data growth.

What are next steps?

IT leaders must prepare for the Zettabyte Era and need economical ways to capture exponentially more data to extract more value. It’s time to rethink existing architectures to enable newer technologies that give the best TCO. Zoned Storage offers a solution to redefine data centers for zettabyte scale. Check out ZonedStorage.io for tools and resources that can help organizations architect and intelligently optimize workloads for the Zettabyte Era and beyond.

Reference

1. “Source IDC Worldwide Global DataSphere Forecast, 2020–2024: The COVID-19 Data Bump and the Future of Data Growth,” April 2020, DOC #US44797920.