You may have heard of the IBM® Spectrum Storage™ family, a select group of storage products that together provide software defined storage (SDS) infrastructure. And if not, now you have. It is all a bit of a mouthful: there is a new deployment style for storage (software defined), with an ambitious user value proposition that needs a new supporting cast of technologies, and now IBM has delivered a select family of offerings to meet that need.

By now SDS has become an industry-wide quest; So can we cut through the jargon and salesmanship and explain why SDS matters and why this particular offering family is right for the task? Yes we can, glad you asked.

The SDS idea has been floating around for a few years now, long enough for IDC to propose a definition of what it is (WW SDS Taxonomy March 2014 #247700) and begin to count actual revenue for qualifying products. The definition is a useful start: SDS offerings require no unique hardware, they run on a variety of platforms from various hardware vendors, and provide a complete standalone system. But we haven’t said yet why this is so important or such a departure from other ways to deal with storage. The value of SDS is in the ability to define through software the access, movement, placement, protection and handling of data across its entire lifecycle—from first point of capture to the final disposal of its last copy.

By any measure, we are only at the very start of the information age. Data is being captured at exponentially growing rates as we build digital instrumentation for practically every human activity. The smart meter on my house samples my power use 8,000 times more often than the old once-a-month approach. You order a taxi, track its arrival and pay for the ride with a smartphone. Our cameras are our phones and our photo albums are at a cloud service. Data is being kept for longer periods of time, and combinations of data from disparate sources are being aggregated to provide valuable insights in business, medicine, science, education and government. Many of the hard-wired, fixed storage assets of the past are not up to the dynamic task of providing large-scale, low-cost infrastructure that can be quickly re-configured to meet the need of a new business idea or a new regulation. We need the speed and flexibility of software, the surety of automated management, and the scale and performance of specialized storage services to deliver on the promise of the information age.

Enter the Spectrum Storage family.

To be useful, SDS must provide the data interfaces and performance that applications have been written to, and be able to accommodate new data interfaces as these appear. Many existing applications today expect to address storage through block or file protocols. There is a good deal of optimization baked into world-class block stores, setting expectations for performance and function that SDS must meet. IBM has chosen two mature technologies for block storage: Spectrum Accelerate™ for iSCSI attachment to virtual machine servers, and Spectrum Virtualize™ to bring along existing Fibre Channel infrastructure commonly found in enterprise data centers. Both of these software technologies can help you fully exploit new flash devices and are designed to perform on par with the best storage appliances.

Much of the data growth in SDS is driven by applications that use file and object protocols for data. These environments can start at modest size and grow to be huge as data collects over time. To meet the needs of these applications, IBM chose a world-class file technology, Spectrum Scale™, and added a collection of data interfaces including POSIX, NFS, a Hadoop connector and the OpenStack Swift object store software. Data from various sources can be brought into Spectrum Scale and shared across multiple applications and geographies. Further, Spectrum Scale supports a collection of storage pools, each with its own characteristics of cost, performance, protection and so on. One such pool can be Spectrum Archive™, a facility to provide a tape storage pool that Spectrum Archive can use for cold data.

SDS must do the entire storage job, and that means preventing data loss from hardware failure, software bugs or human error. Data protection, including backup and disaster recovery copies, is provided by Spectrum Protect™, a mature technology already in use by tens of thousands of users. As big data analytics and other new application types become business-critical, data protection will be added to what are often rudimentary implementations today.

Finally, all of this must be managed. The sheer scale and performance of large facilities makes it impossible to manage through human operators. Instead, we need automation that not only can keep up with the rate of demand, but can provide a goof-proof operation where human error cannot lead to data loss. To accomplish this, Spectrum Control™ is management software that manages through policy. Decisions about security, data protection, performance or disposal of data are made by administrators in business terms, and then mapped to policies that Spectrum Control provides to the various data products. These in turn enforce the proper placement of data in fast or slow pools, create remote replicas or not, and retain data for the authorized period of time.

As the world becomes more and more dependent on data, reliable, highly capable, low-cost storage infrastructure will be of critical importance. SDS is about much, much more than software running on standard servers; it is about delivering the data required to keep food and power flowing, planes in the air, patients healthy and people connected. IBM has brought forward a well-planned collection of mature technologies ready to deliver on today’s needs and ready to add the features and capacity required in the future. And don’t forget to let me know what you think using the comments feature below.