This week Nimble Storage announced an all-flash array in a series of events around the globe that were based on presentations at the main event in San Francisco.  Many bloggers were flown in specially from around the world, to help get the all-flash message out.  Nimble has been a long-time provider of hybrid storage solutions and has traded publicly since December 2013, after IPO on the NYSE.  Moving to an all-flash architecture is an interesting move for a company that even two years ago seemed to think that hybrid was really the only way to go (read this good interview article in Techworld).  So why all-flash and why now?


The architecture of Nimble’s storage platform is based on a feature called CASL or Cache Accelerated Sequential Layout.  The company doesn’t seem to talk about CASL much any more, however you can still find a deep dive video on the technology on YouTube.  To summarise, CASL converts random write requests into sequential writes by coalescing the updates in NVRAM before committing to hard disk.   Flash storage is used to cache active or hot data based on adaptive algorithms that make intelligent decisions about what needs to be moved immediately to cache for subsequent re-read.  Anyone familiar with NetApp’s original Data ONTAP architecture will recognise this approach and in fact in March 2015 Nimble finally settled two outstanding lawsuits from NetApp over alleged misappropriation of confidential information and poaching of NetApp staff.

So CASL is designed to work with disk media by committing data in large sequential writes.  Flash is only used to service reads because at the time of designing the platform, it was assumed that flash would wear out too quickly.  The two potential issues for CASL performance occur when there are large numbers of writes or if data being read is not in the flash cache.  In the first instance, performance can drop as the NVRAM flush process has to wait for data to be written to disk.  Sizing of NVRAM therefore becomes critically important.  Second, the ability of the read cache to be effective again is determined by size; if the read cache isn’t sufficient, then data has to be read from disk and at that point the I/O latency increases dramatically.

Flash Evolution

As a design, CASL is an elegant way of continuing to use hard drives and (what was at design time) expensive flash.  However since the company was founded in 2008, flash technology has advanced significantly.  At the macro level, prices have continued to drop (in the same manner as HDDs) while capacity and endurance have increased.  At the micro level, new technologies like 3D-NAND and TLC have allowed the development of drives with capacities already at 4TB, with the prediction of 16TB drives within a couple of years.  So it’s becoming increasingly practical to use lower endurance, cheaper flash as the bulk storage medium in todays storage appliances.  It’s clear that Nimble have needed to follow the market and release an all-flash version of their Adaptive Flash platform, if not to compete on technical merit, but simply from a marketing position.  All-flash systems are increasingly attractive, with many vendors pushing (effective) capacity prices down to around $1-$1.50/GB, so Nimble has to have a story, even if the original premise for the platform design predicated that all-flash wasn’t necessary.  Incidentally, Nimble have not quoted raw or effective $/GB system prices, but instead chosen to talk about TCO (which is 33-66% lower than competing products, apparently).  This in itself speaks volumes.

If we look at CASL, the use of flash as a cache becomes somewhat redundant with an all-flash system.  Reads can simply come from the media itself.  When processing writes, the NVRAM flush can be synchronised to match the size of the blocks of the underlying NAND.  So, rather than have garbage collection continuously managing the block/page allocations, flushing to match block size should provide a more consistent performance experience and reduce the impact of wear on the flash.  As the write process is well controlled, this should also mean that CASL is good for working with larger capacity drives (specifically greater than 4TB) without too much of a problem.

The Architect’s View

There are some interesting aspects to Nimble’s technology, including InfoSight and some features of the all-flash systems like Triple+ RAID and rack density.  These will have to get covered in another post.  With some of those features, Nimble have moved ahead of the competition, however it won’t take long for the other all-flash vendors to start using 4TB drives in their systems.  This week’s announcement looks more to me as a catch-up than a deliberate move to wait until the market was right for an all-flash Nimble system.  However the markets did react quite favourably to the news with an 8.5% increase in the Nimble stock price to around $7.20.  Remember though that the share price peaked at almost $53 in February 2014, so there’s a long way to go to get back to their former glory.

Further Reading

There are a large number of videos to watch on the Nimble page over at the Tech Field Day website.  Alternatively, the links below provide other information.


Comments are always welcome; please read our Comments Policy first.  If you have any related links of interest, please feel free to add them as a comment for consideration.  

Copyright (c) 2009-2016 – Chris M Evans, first published on, do not reproduce without permission.

Please consider sharing!Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInBuffer this pageEmail this to someoneShare on RedditShare on StumbleUpon

Written by Chris Evans

  • klstay

    I was able to spend a couple of hours with two Nimble engineers earlier this week and pick their brains a bit on details of the new all flash system. Garbage collection is particularly well suited to an all flash system as you indicated. Pages are now 18MB up form 4.5. Dynamically dealing with multiple block sizes all the way to “disk” has always been a strength and still is though for different reasons now.

    At the same time this is clearly a re-working of a platform designed for hybrid SSD caching in front of spinning disks. The whole concept of caching is now gone, though it could be added back in later and I believe it will be eventually. Not caching with a higher performance NAND tier while access host access is still iSCSI or even FC is probably without much of a downside. However, if NVMeF becomes viable having just TLC NAND with nothing in front of it could become a real weakness for some workloads.

    Also, per controller NVRAM has been NVDIMM for some time now. Having access to that space at memory bus latency is great except all those blocks must also be written to the standby controller NVRAM at PCIe bus latency before an ack goes back upstream. IF (and I remain unclear on this) alignment, de-dupe, compress etc. processing begins before waiting for packet commit to standby controller NVRAM to finish (which to me seems likely) then that design is of some benefit.

    Having stated all that, I like what I see so far. Once there are some real world “field proven reliability” numbers behind it this could be a real winner for FC/iSCSI connected tier 1 (and even “1.5” at these prices) workloads. After all, the last thing any customer wants to live through is another Xtremio debacle. So, I am optimistic, but as usual will let someone else do the 1.0 release field testing…