Answers: <p>I think you're on the right track. Just need a tiny bit more logic on the backup process, then S3 can handle the rest.</p>
  <p>Consider this:</p>
  <ol>
  <li>A backup is uploaded every day.</li>
  <li>If today is January 1st (for example), the backup is given a &quot;yearly&quot; tag.</li>
  <li>Otherwise, if today is the 1st of the month, the backup is given a &quot;monthly&quot; tag.</li>
  <li>Otherwise, if today is a Sunday, the backup is given a &quot;weekly&quot; tag.</li>
  <li>Otherwise the backup is given a &quot;daily&quot; tag.</li>
  </ol>
  <p>Next, set up S3 Bucket Lifecycle Policies with each tag as the filter.</p>
  <ol>
  <li>Transition &quot;yearly&quot; backups to Glacier after 365 days (kept indefinitely)</li>
  <li>Delete &quot;monthly&quot; backups after 365 days (1 year)</li>
  <li>Delete &quot;weekly&quot; backups after 183 days (6 months)</li>
  <li>Delete &quot;daily&quot; backups after 31 days (1 month)</li>
  </ol>
  <p>References:</p>
  <ul>
  <li><a href="https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html</a></li>
  <li><a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-configuration-examples.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-configuration-examples.html</a></li>
  </ul>
  
Answers: CC BY-SA 4.0
Answers: 2020-07-08T10:05:34.097Z
Answers: I settled by first listing all S3 backups then if most recent yearly was > 365 days ago then tag today's backup a yearly ... else if most recent monthly > 31 days ago then make today's backup a monthly ... etc ...
Answers: Scott Stensland
Answers: CC BY-SA 4.0
Answers: 2020-07-03T18:22:59.430Z
Answers: 2020-07-03T18:22:59.430Z
Answers: Woodland
Body: <p>I need to automate deletion of aws S3 backups ... evidently if I tag the backup with one of daily, weekly, monthly, yearly then aws will delete based on my desired retention counts per those periods however I see no easy way to determine which of those tags I give to my daily backup process which is</p>
  <pre><code>//   Upload backup file
  
  aws s3 cp /tmp/backup_2007020644.20200703.1248_blobs.tar  s3://foo-bar-baz-someaccount-us-east-2/backup_2007020644.20200703.1248_blobs.tar --region  us-east-2 --only-show-errors 
  
  
  aws s3api put-object-tagging --bucket foo-bar-baz-someaccount-us-east-2  --key backup_2007020644.20200703.1248_blobs.tar --tagging  --region  us-east-2 TagSet=[{Key=backuptype,Value=blobs}]
  
  //   now lets list this s3 bucket
  
  aws s3 ls s3://foo-bar-baz-someaccount-us-east-2 --region  us-east-2 
  
  2020-07-01 22:55:57   31904428 backup_2007010938.20200701.2233_blobs.tar
  2020-07-01 22:55:43     893239 backup_2007010938.20200701.2233_mongo.tar
  2020-07-02 15:30:36   34343354 backup_2007010938.20200702.1508_blobs.tar
  2020-07-02 15:30:22     893676 backup_2007010938.20200702.1508_mongo.tar
  2020-07-03 01:20:04   30596405 backup_2007020644.20200703.0055_blobs.tar
  2020-07-03 01:19:51     893741 backup_2007020644.20200703.0055_mongo.tar
  2020-07-03 12:48:44   34658003 backup_2007020644.20200703.1226_blobs.tar
  2020-07-03 12:48:30     895294 backup_2007020644.20200703.1226_mongo.tar
  2020-07-03 15:05:00   34657972 backup_2007020644.20200703.1248_blobs.tar
  2020-07-03 15:04:46     895279 backup_2007020644.20200703.1248_mongo.tar
  </code></pre>
  <p>Alternatively I can code up my own logic to parse above listing then issue the delete command per backup file after my code keeps track of my retention policy. There must be a better way, any advice?</p>
  
Tags: <amazon-s3>
Title: auto delete aws S3 backups
UserDisplayName: Scott Stensland
ThreadText: # auto delete aws S3 backups
  
  (Asked by: username_0 on 2020-07-03)
  
  I need to automate deletion of aws S3 backups ... evidently if I tag the backup with one of daily, weekly, monthly, yearly then aws will delete based on my desired retention counts per those periods however I see no easy way to determine which of those tags I give to my daily backup process which is
  
  ```
  //   Upload backup file
  
  aws s3 cp /tmp/backup_2007020644.20200703.1248_blobs.tar  s3://foo-bar-baz-someaccount-us-east-2/backup_2007020644.20200703.1248_blobs.tar --region  us-east-2 --only-show-errors 
  
  aws s3api put-object-tagging --bucket foo-bar-baz-someaccount-us-east-2  --key backup_2007020644.20200703.1248_blobs.tar --tagging  --region  us-east-2 TagSet=[{Key=backuptype,Value=blobs}]
  
  //   now lets list this s3 bucket
  
  aws s3 ls s3://foo-bar-baz-someaccount-us-east-2 --region  us-east-2 
  
  2020-07-01 22:55:57   31904428 backup_2007010938.20200701.2233_blobs.tar
  2020-07-01 22:55:43     893239 backup_2007010938.20200701.2233_mongo.tar
  2020-07-02 15:30:36   34343354 backup_2007010938.20200702.1508_blobs.tar
  2020-07-02 15:30:22     893676 backup_2007010938.20200702.1508_mongo.tar
  2020-07-03 01:20:04   30596405 backup_2007020644.20200703.0055_blobs.tar
  2020-07-03 01:19:51     893741 backup_2007020644.20200703.0055_mongo.tar
  2020-07-03 12:48:44   34658003 backup_2007020644.20200703.1226_blobs.tar
  2020-07-03 12:48:30     895294 backup_2007020644.20200703.1226_mongo.tar
  2020-07-03 15:05:00   34657972 backup_2007020644.20200703.1248_blobs.tar
  2020-07-03 15:04:46     895279 backup_2007020644.20200703.1248_mongo.tar
  
  ```
  
  Alternatively I can code up my own logic to parse above listing then issue the delete command per backup file after my code keeps track of my retention policy. There must be a better way, any advice?
  
  **Tags:** amazon-s3
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-03. Score: 4 | Accepted answer
  
  I think you're on the right track. Just need a tiny bit more logic on the backup process, then S3 can handle the rest.
  
  Consider this:
  
  1. A backup is uploaded every day.
  2. If today is January 1st (for example), the backup is given a "yearly" tag.
  3. Otherwise, if today is the 1st of the month, the backup is given a "monthly" tag.
  4. Otherwise, if today is a Sunday, the backup is given a "weekly" tag.
  5. Otherwise the backup is given a "daily" tag.
  
  Next, set up S3 Bucket Lifecycle Policies with each tag as the filter.
  
  1. Transition "yearly" backups to Glacier after 365 days (kept indefinitely)
  2. Delete "monthly" backups after 365 days (1 year)
  3. Delete "weekly" backups after 183 days (6 months)
  4. Delete "daily" backups after 31 days (1 month)
  
  References:
  
  * <https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html>
  * <https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-configuration-examples.html>
  
  > **Comments:**
  > **username_0**: I settled by first listing all S3 backups then if most recent yearly was > 365 days ago then tag today's backup a yearly ... else if most recent monthly > 31 days ago then make today's backup a monthly ... etc ...  
  
  ---
  
  