Here’s a quality piece of reporting from TechCrunch on the state of Facebook and their data problems. I mentioned just last week in this post about their data growth. It’s incredible that they’re purchasing a new Netapp 3070 filer each week!
I’m surprised that Facebook would be continually purchasing NAS filers to grow their content. There must be a rolling set of pictures, thumbnails and so on that are frequently looked at, but there also must be a significant amount that aren’t and could be archived to super-dense nearline type technology akin to the Copan products.
Unfortunately when data growth is so intense, it isn’t always easy to see the wood for the trees and from previous and current experience, using Netapp creates the risk of wasted resources.
In my experience, looking at just block-based arrays, I’ve always seen around 10-15% of orphan or unused resources and sometimes higher. When host-based wastage is taken into consideration, the figure can be much worse, although host reclamation is a much more intense process.
I’m willing to offer to anyone out there who has more than 50TB of storage on storage arrays a free analysis of their environment – for a 50:50 split of any savings that can be made. As budgets tighten, I think there will be more and more focus on this kind of work.