FrieslandCampina

Our customer FrieslandCampina is a global dairy company operating across a 100+ countries. TerraAlto have been supporting them on their journey to AWS, work closely with them on multiple projects and provide 24-7 global managed services around their AWS environments.

As part of ongoing cost optimization and control our client wanted to implement more proactive and automated housekeeping on AMIs and associated snapshots. They have been using a 3rd party tool for scheduling and managing retention of AMI backups of EC2 instances, with a retention period of 14 days there a numerous AMIs and snapshots left in place after an EC2 instance is terminated. They did not want to use the AWS EBS LifeCycle Management solution, as their backup strategy is strongly AMI based. The customer requested TerraAlto to provide an solution.

As part of ongoing cost optimization and control they wanted to implement more proactive and automated housekeeping on AMIs and associated snapshots. They have been using a 3rd party tool for scheduling and managing retention of AMI backups of EC2 instances, with a retention period of 14 days there a numerous AMIs and snapshots left in place after an EC2 instance is terminated. They did not want to use the AWS EBS LifeCycle Management solution, as their backup strategy is strongly AMI based.  The customer requested TerraAlto to provide an solution.

Solution

  • An EC2 instance termination event activates a Lambda function;
    Cloudwatch Event Rule -> SNS Topic -> Lambda Function
  • The Lambda function identifies all associated AMIs and deletes all but the most recent two. It also deletes the associated snapshots for each AMI being deleted. Finally it tags the remaining two AMIs and their associated snapshots a ‘snapshot’ tag which indicates how they were created and the expiration date. Example tag value == man-181118.
  • Additionally there is a Lambda function that runs a daily check of all AMIs and snapshots, deleting any that passed their expiration date.
    CloudWatch Event Rule -> Lambda Function

  • All above is deployed using Cloudformation and SAM framework.

Challenges
Some of the challenges involved in this project were to:

  • We needed to be able to identify AMI’s associated with an EC2 instance being terminated, and the snapshots associated with each AMI.
  • We implemented a new tag for snapshots and AMIs to identify type and expiration date.
  • We still want to retain the last 2 AMIs backups of an EC2 instance after termination for specified or standard period of time.
  • Automated the deployment of the solution, to make it easily repeatable for other clients.

Benefits Delivered

  • Immediate cleanup on AMIs and snapshots that provided a 30% reduction in monthly spend related to snapshots.
  • New tagging allowed for easy implementation of custom retention of AMIs and snapshots.
  • Reduction in operational resource time required for completing decommissioning checklists and for ongoing housekeeping related to AMIs and snapshots.