HDD Remains Dominant Storage Technology

What’s the outlook for enterprise hard disk drives (HDD) in the modern data center?

Are solid state drives (SSD), fueled by long-term declines in NAND prices, on track to eclipse enterprise HDD as the dominant storage technology in the data center—and, if so, when? Is the end nigh for the trusty hard disk drive?  

The answer is simple and counter-intuitive perhaps. Despite the growing (and welcome) adoption of solid state drives in consumer and enterprise computing, reports of the demise of the hard disk drive are routinely overblown. In fact, the 2020s are set to prove the strongest decade yet for shipped HDD capacity.

Sharp Exabyte Growth

The data is compelling. In 2018, there were a total of 912 exabytes of combined SSD/HDD storage shipped, comprising more than 800 exabytes of HDD. That’s almost a zettabyte (or 1 billion terabytes) of freshly-minted storage capacity,  a whopping 89% of which consisted of spinning platters.

Although the full-year data is not yet confirmed for 2019, all signs point to year-on-year growth in shipped capacity, with HDD continuing to dominate. No surprise there, and the outlook through 2025 is even more promising, once the world has moved through the worst of the coronavirus pandemic.

For one, Seagate, which has kept most of its storage eggs in the HDD basket, is projecting staggering levels of data growth in the first half of the 2020s. In 2018 research conducted by IDC, the storage leader predicted an increase in global data from 33 zettabytes in 2018 to 175 zettabytes by 2025. That’s 175 sextillion bytes to be precise, or one to the power of 21.  

Of course, it’s unclear how COVID-19 will affect these projections and not all the data generated will require storage anyway—much of it will evaporate into cyberspace. But an element of it, depending on the use case, will end up stored somewhere, whether locally or in the cloud.

And that drives demand for fresh storage. For this reason, IDC predicts that 22 ZB of new storage capacity will ship in the period 2018-25. Of this fresh supply, hard disk drives will make up approximately 60%.  This amounts to about 13 ZB of additional HDD capacity entering the supply chain during the seven-year period. The vast majority of that total (approximately 90%) will ship between 2020 and 2025. 

Storage expert Tom Coughlin offers a similar forecast. In his most recent analysis, he foresees a quadrupling in annual shipped capacity between 2019 (1 ZB) and 2025 (4 ZB).  

Impact Of Coronavirus

Despite the recent optimism, there’s now an 800-pound gorilla in the room in the form of the coronavirus. It remains far from clear how the devastating effects of the outbreak on the world economy will affect previous industry projections on fresh storage capacity.

What is clear is the critical role that large technology firms are now playing in keeping the lights on for many aspects of everyday life.

Speaking on March 24 to CNBC as the United States prepared to overtake China as having the largest number of confirmed coronavirus cases on the planet, Microsoft CEO Satya Nadella said the crisis was underscoring the role of cloud and SaaS services as “digital first responders.”

Indeed, demand for remote access file sharing and video conferencing to support the needs of teleworking surged in the weeks following the U.S. national shutdown.

The spike in home-based working might be good news for the cloud giants and video conferencing firms such as Zoom. However, there is no obvious correlation between the consumption of cloud-based services and the demand for data storage. Will the data storage manufacturers end up shipping more or less capacity because of this historic disruption to the world economy? How will it affect end user demand?

Most analysts and industry leaders, including Nadella, agree the IT supply chain will rebound from its shock fairly quickly. However, they also agree that the longer-term impact on demand remains to be determined.

Writing for the Motley Fool regarding the appointment of Cisco’s David Goeckeler as Western Digital’s new CEO, analyst Leo Sun said the potential ramifications of COVID-19 on storage product sales were wide-ranging.

“The coronavirus crisis could reduce demand for [WD’s] HDDs, SSDs, and flash memory products across multiple industries across the world. Sales of PCs and servers could also stagnate and end WD’s delicate rebound in sequential revenue growth,” he warned.

That said, given the essential role data plays in the functioning of modern life, it’s unlikely data storage demand will fall off the cliff. The more likely scenario is that the ebullient projections from Seagate / IDC for the period through 2025 will broadly hold up, albeit suppressed over the next six to 12 months as the outbreak reaches peak around the world and the global economy begins to recover its footing.

New HDD Technologies

Pandemic aside, there remain good grounds for the three main hard drive manufacturers to feel confident about the medium to long-term outlook for HDD technology.  

Seagate is particularly bullish on this point: it believes HDD will retain its price advantage over SSD until at least the middle of the 2030s. It points to projected advances in areal density—the relative volume of data each spinning disk can hold— enabled by its development of HAMR (heat-assisted magnetic recording) technology. 

This upbeat assessment comes after several years of areal densities flatlining. To offset the slowdown, the HDD manufacturers have been incrementally adding 2TB platters to enterprise drives as a way to increase capacity. Now, new HDD technologies are looking to up the ante on areal density again.  

While Seagate is hot on HAMR, rival Western Digital initially touted its MAMR (microwave-assisted magnetic recording) technology as the optimal solution to the challenge of areal density. However, in a succession of recent statements WDC has been careful to acknowledge that not only is HAMR also a good bet for the long term but that WD has itself been investing in HAMR development (to the tune of $500 billion, according to reports) alongside its MAMR initiative.   

 “We’re not going to sit here and tell you that it’s MAMR versus HAMR. I think that’s a losing argument,” recently retired Western Digital CEO Steve Milligan remarked on a analyst call in December. “We believe that both technologies are viable.”

It will be indicative to hear what Milligan’s successor David Goeckeler, who took over the reins on March 9, has to say about HAMR when he starts making public statements on the company’s technology roadmap.

Meanwhile, Seagate continues to double down on hard disk drives. It points not only to its investment in HAMR but its development of dual actuator technology, designed to unlock additional input/output operations per second for improved read/write performance. And it continues to project bullishness about its single-minded commitment to HAMR.

What appears certain is that HDD capacities remain poised for significant expansion in the coming years. Notwithstanding Seagate and WD’s pronouncements on the subject, Japanese platter manufacturer (and Toshiba supplier) Showa Denko recently announced the development of new HD media that would pave the way for HDD capacities up to 80TB.

HAMR represents a recording method in which magnetic film is locally heated at the time of recording. This technology has been developed to solve the “magnetic recording trilemma”: difficulty in simultaneously meeting the three requirements of fine-particle structure, resistance to thermal fluctuation, and ease of magnetization.

Compared with the recording density of approx. 1.14 Tb/in² for HD media based on conventional magnetic recording methods, it is said that HAMR-based HD media will achieve recording density of 5-6 Tb/in² in the future. Provided that the same number of disks are used, it is estimated that a 3.5-inch HDD will achieve storage capacity of approx. 70-80 TB per unit.”

Showa Denko K.K.

At capacities of up to 80 TB, that’s a truly massive data-bearing piece of kit.

HDD Vs SSD

None of this technological development reflects poorly on solid state drives—the opposite in fact.

SSD adoption will continue to grow as a proportion of the overall pie, and SSD and HDD pricing will keep falling over the long term (although the medium-term outlook for enterprise SSD pricing is somewhat volatile as higher demand boosts 2Q20 prices by up to 15%, according to analysis by TrendForce.)

Advances in the NVMe protocol and a growing appetite for composable architectures built around flash technology and storage-class memory will further overhaul the established order, and place less of an outright emphasis on HDD in the modern data center.

Indeed, outside of the data center flash is already the default choice for storage.  For example, HDD manufacturers have long recognized the inevitability of the notebook entirely transitioning to SSD as consumers rely increasingly on the cloud for storage, remarks Stephen Buckler, chief operating officer of Horizon Technology.

However, this does not spell the end of the hard drive, Buckler insists.

“Increasingly storage is all about the cloud as artificial intelligence and big data ever more play a role in our day-to-day lives, increasing the need for cheap storage. Storage leaders such as Seagate and Western Digital continue to invest in improvements to hard disk technology, signalling that the game is far from up for HDD.”

Despite the headwinds around areal density and the increasing need for higher availability storage to handle AI workloads, confidence about the outlook for HDD has stayed constant in recent years.

“HDD’s installed base is massive and works well with all but the fastest high-performance applications,” found a 2018 report on data storage trends from Enterprise Storage Forum. “It makes no sense for IT to rip and replace disk-based or hybrid systems, and may not for years to come.”

Nearline As The Sweet Spot

Much of the optimism around HDD centers on nearline capacity. Nearline represents a reliable sweet spot for hard disk drive manufacturers as they meet demand from enterprise and cloud vendors, themselves looking to keep pace with the explosive growth in data generation. 

For its part, Toshiba, the third of the main enterprise manufacturers, remains eager to grow its share of the nearline sector.

Recently unveiling plans to drive up its nearline market share, Toshiba insists its strategic commitment to enterprise HDD is nothing new. The company points to a near tripling in shipped HDD exabytes since the beginning of 2018, which it attributes to the introduction of helium-filled hard drives.

As Rainer Kaese of Toshiba Electronics Europe remarks, “There is still no way around HDDs for the cost-effective provision of storage capacity.”

Nearline is certainly the growth area for enterprise HDD, particularly at the larger capacities that the industry promises. Storage expert Enrico Signoretti goes as far to argue that the higher-cap, next-gen HDD promised by the OEMs will not easily find a place in small to medium-sized data centers. Few companies need to store petabytes of data locally, he maintains.  

On the flip side of the equation, it seems unlikely that HDD will develop into a credible threat to tape as a medium for cold storage in the hyperscale data center. Not only does tape continue to hold a clear price advantage over HDD for archival storage, but there is a raft of technologies (many of them centering around the potential of DNA for long-term storage) that may come to threaten tape’s dominance anyway.

What’s clear is that Toshiba, Seagate, and Western Digital are targeting the cloud data center as the prime area for capacity HDD growth in the coming years. By 2025, approximately half of the world’s stored data will live in the public cloud, according to Seagate/IDC’s research. 

An appetite for cloud data storage runs strong among both consumers and enterprise. “Nearly 50% of cloud architects report almost half of their organization’s infrastructure is in the cloud, with 77% sharing their desired end state is running most or all applications on public cloud infrastructure,” according to a 2019 survey conducted on behalf of data protection firm Veritas.

The sales data backs it up. Demand among cloud vendors for high-cap nearline HDD drove Q3 shipments to record highs “with an unprecedented volume of nearline shipping to just a handful of U.S.-based tier-1 hyperscale customers,” according to analysis from TrendFocus.

Catch A Rising Tide

Once the coronavirus crisis is under control, there’s every indication that demand for storage capacity will continue growing, largely driven by exabytes of fresh HDD capacity.

Just as tape still has its place in the storage mix, hard disk drives have cemented their position as the data storage mainstay for enterprise and cloud workloads. At the same time, flash continues to strengthen across storage use cases, and will grow as a proportion of an expanding pie.

As they say, a rising tide lifts all boats.