The SSD caching market is on fire.  This week, EMC announced VFCache, IBM announced XIV Gen3 SSD Caching, Flashsoft was acquired by Sandisk, and many storage vendors acknowledged that flash features in their long-term roadmap.  Activity has certainly picked up. It is interesting to explore what is causing the rapid development of the SSD caching market.

Here are my 4 key takeaways:

1. Flash has arrived

Reliability: Flash technology (particularly in the SSD form factor) has matured enough to be used in the 24/7 enterprise environments. Through a combination of more reliable SSD and better management of data on SSD, suppliers have managed to create solutions that, while expensive, provide the bullet-proof reliability required in enterprise environments.

Real results: Businesses have experienced the benefits of higher SSD IOPS in production environments. Higher SSD performance has translated to up to 10+ times improvement in performance for applications like databases (Oracle, SQL Server, MySQL, etc.), Financial analysis, research, simulation, modeling, enterprise search, E-commerce and social networking.

Enterprise Budgets:  2012 is the first year where IT teams have dedicated budgets for deploying SSD.

2. The storage industry has accepted that storage architectures will change

EMC’s launch of VFCache and similar announcements from other storage vendors show that the industry has accepted a paradigm shift in data storage.  Gone are the days when all data was stored and served from fibre-channel SAN arrays. Storage arrays will continue to be used to consolidate data for backup and sharing, but performance will be driven by a layer of flash in the server that holds a copy of primary data.  Chris Mellor’s article in The Register “Inside the Mind of EMC, Is Storage Just a Launchpad” and David FLoyer's article in Wikibon “Designing Systems and Infrastructure in the Big Data IO Centric Era” provide more details on the shift in storage architectures.

3.  SSD caching offers an easier path to SSD deployment

SSDs offer many advantages over HDDs, including much higher I/O performance, lower power consumption, and non-volatile memory. However, SSDs are expensive and deploying SSD often disrupts existing data protection and data management infrastructures. Deploying SSD as a cache overcomes both limitations.

 Lower Cost: By caching only important data on SSD and keeping primary data on lower-cost HDD, application performance can be increased at a lower cost

 Non-Disruptive Deployment: When SSD is deployed as a cache, existing primary storage remains unchanged. There is no need to pause or change existing backup and data management infrastructure.

 Best of both worlds:Deploying SSD as a cache provides the low latency and high performance of server-based SSD, while preserving the ability to uniformly protect and share primary data stored in the storage array.

4. SSD cache solutions will soon pack more intelligence

SSD is not just a faster disk: data is structured, accessed, and protected in a fundamentally different way in SSDs. Unfortunately, applications today are optimized for working with HDD and do not take into account the asymmetric read/write operations and SSD wear caused by write/erase operations. One can expect that over time applications will be re-written to account for the specific properties of SSD; however, the process of rewriting applications will be slow and is likely to drag over 10-15 years. SSD Caching software is a natural application translation layer that can optimize data for SSD.

Data optimization for SSD: The table below illustrates the asymmetric nature of SSD. Read speed is 10x faster than write speed and 80x faster than erase speed. Furthermore, erase operations wear down the SSD and shorten its life. An SSD performs best if the caching software structures data in a way that maximizes read operations and minimizes write and erase operations. SSD performance is also impacted by the structure and size of data blocks stored on SSD. Intelligent SSD caching software would structure data blocks in a way that optimizes SSD performance and reliability while minimizing cost and wear.

Performance: Existing caching algorithms were developed for HDD and do not account for the asymmetric properties of SSD (the blog “Why Standard Cache Algorithms Won’t Work For SSDs” by Prof. Qing Yang, CTO of VeloBit, discusses this in further detail). As a result, when used with SSD, –existing caching algorithms do not perform well and shorten the life of SSD. New caching algorithms, designed with SSD in mind, can increase SSD performance by 3-5x relative to existing caching solutions and can enable use of less expensive SSD.

Data pattern intelligence: Tiering and caching solutions today prioritize data based on its recency or frequency of use and, as a result, are not very effective at predicting future data use. The intelligent SSD caching software of the future will self-tune based on the data pattern to maximize performance on every application.

What do you think about the recent wave of SSD announcements and acquisitions? Where do you think the industry is going?