S3, or Simple Storage Service is Amazon’s high capacity, high resiliency object store.  The company recently announced a new feature, which allows versioned objects (those objects where multiple versions are retained for historical archive purposes) in S3 to be managed with lifecycle rules that either move data to AWS Glacier or assign a time period for the deletion of historical copies.

Versioning is a feature of S3 which allows multiple copies of an updated object to be retained, based on policies set by the user.  Policies are established using AWS Rules, which can be applied to an entire bucket (a collection of objects) or a subset based on a folder or object prefix.  From this point the user defines an action on either the primary or the versioned copies of the object.  In many instances the primary object may be left alone (“do nothing”) whereas the older versions get archived or deleted after a fixed period of time.  Archiving means moving to Glacier, Amazon’s low cost storage.

Although the Lifecycle rules a pretty simplistic, they are actually quite powerful and offload a lot of work customers would have done themselves.  As the objects never leave S3/Glacier, all of their metadata information should be maintained automatically.

How could this feature be used? Well think of something as simple as backup; a lightweight backup client could replicate into S3 with versioning enabled with rules/policies in place to move all but the primary version immediately into Glacier.  Even backups taken with something as simple as rsync could be made to be cost effective and more useful.

The Architect’s View

Amazon continues to innovate with AWS, to the extent that it’s almost a full time job keeping up with the enhancements.  Versioning of objects and archiving to Glacier is a “must have” feature that will make S3 even more attractive to the customers already using the platform and perhaps help to jump start the use of Glacier.

Related Links

Comments are always welcome; please read our Comments Policy.  If you have any related links of interest, please feel free to add them as a comment for consideration.  

Copyright (c) 2007-2018 – Post #15BC – Chris M Evans, first published on https://blog.architecting.it, do not reproduce without permission. Photo credit iStock.


Written by Chris Evans

With 30+ years in IT, Chris has worked on everything from mainframe to open platforms, Windows and more. During that time, he has focused on storage, developed software and even co-founded a music company in the late 1990s. These days it's all about analysis, advice and consultancy.