Answers: <p>My solution was based on the one of @rombob.
  I used S3 and created a daily S3 inventory file of the source bucket (which is replicated). Then I created a Athena DB and table using Glue (in CloudFormation). 
  Then I implemented an S3 event which triggers a lambda. Every time a new manifest.checksum file was uploaded (= new inventory is finished). 
  The lambda executes a query on the Athena table and checks if there are objects which <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-status.html" rel="nofollow noreferrer">replication status</a> is FAILED. If there are then the lambda publishes to a SNS topic to send an email. </p>
  
Answers: CC BY-SA 4.0
Answers: 2019-07-15T18:50:38.430Z
Answers: 2019-07-15T18:50:38.430Z
Answers: lvthillo
Answers: <p>Still no answers, and still no CRR monitoring in place on AWS, here's minimalistic solution, any feedback and suggestions are welcome. This monitor will only show which S3 objects are missing from destination, it will not provide visibility on versions of replicated objects.</p>
  
  <p>Prerequisites</p>
  
  <hr>
  
  <ol>
  <li><a href="https://docs.aws.amazon.com/AmazonS3/latest/user-guide/configure-inventory.html" rel="nofollow noreferrer">Enable inventory reports</a> on <code>src</code> (Source) and <code>dst</code>
  (Destination) buckets</li>
  <li><a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-inventory.html#storage-inventory-athena-query" rel="nofollow noreferrer">Create Athena tables</a> (<code>src_inventory</code>, <code>dst_inventory</code>) based on inventory reports for each bucket</li>
  </ol>
  
  <p>Monitor</p>
  
  <hr>
  
  <p>Run Athena query after both inventory reports generated e.g. on daily basis, can be executed from <a href="https://docs.aws.amazon.com/athena/latest/APIReference/API_StartQueryExecution.html" rel="nofollow noreferrer">Lambda</a>, <a href="https://github.com/anselmo/step_function_activity_patterns/blob/master/athena_query_pull_mode/fns/athena.js" rel="nofollow noreferrer">Step Function</a>, build server or any type of cron job.</p>
  
  <p>Example of Athena query which should show number of missing objects on destination bucket at the time of generation of inventory reports:</p>
  
  <pre><code>FROM src_inventory src 
  LEFT JOIN dst_inventory dst 
  ON src.key = dst.key 
  WHERE dst.key is NULL
  AND src.dt = '2018-11-11-08-00';
  </code></pre>
  
  <p>The result can be reported as CloudWatch metric and monitored with CloudWatch alarm. </p>
  
Answers: CC BY-SA 4.0
Answers: 2018-11-14T20:10:08.953Z
Answers: 2019-05-31T14:57:46.150Z
Answers: 2019-05-31T14:57:46.150Z
Answers: rombob
Answers: <p>At this point using S3 inventory to monitor replication is the best solution for large buckets, and <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-status.html" rel="nofollow noreferrer">the solution suggested by AWS</a> themselves. Since AWS don't report on the status directly, any other solution is likely to be both expensive and time consuming when that many objects are involved. It seems like you're on the right (and only?) track.</p>
  
  <p>An obvious idea would be to talk to AWS Support about it, ask them to add such a feature. But that's obvious.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-02-11T17:55:51.913Z
Answers: 2019-02-11T17:55:51.913Z
Answers: bgdnlp
Body: <p>Amazon S3 has a cross-region replication which will handle copy of new/updated objects to additional region.</p>
  
  <p>The problem is that solution does not provide visibility on state for replication process, for example at the moment there's no way to easily monitor missing objects on destination or any possible permission issues that can interfere with the process and can result with replication not doing it's job, as well missing feedback for replication lag.</p>
  
  <p>We found several tools similar to CRR Monitor <a href="https://aws.amazon.com/answers/infrastructure-management/crr-monitor/" rel="nofollow noreferrer">https://aws.amazon.com/answers/infrastructure-management/crr-monitor/</a>
  but we didn't have good experience with it as we manage tens of millions of objects which can cost several thousand of dollars per month for CloudWatch events, as well as additional management for services required by solution (DinamoDB, Kinesis etc.) another layer of "monitoring the monitor" required.</p>
  
  <p>For now we are using daily comparison of S3 objects inventory between source and destination using AWS Athena query, which also feels like a hack and it's not optimal for problem detection as inventory generated with daily frequency.</p>
  
  <p>I hope to see some feedback, suggestions, ideas or experience for alternatives. Ideal solution would be managed or "serverless" solution which will improve visibility on the process and will provide fast detection of errors.</p>
  
Tags: <amazon-web-services><monitoring><amazon-s3>
Title: How to monitor AWS S3 cross-region replication process?
UserDisplayName: rombob
ThreadText: # How to monitor AWS S3 cross-region replication process?
  
  (Asked by: username_0 on 2018-10-04)
  
  Amazon S3 has a cross-region replication which will handle copy of new/updated objects to additional region.
  
  The problem is that solution does not provide visibility on state for replication process, for example at the moment there's no way to easily monitor missing objects on destination or any possible permission issues that can interfere with the process and can result with replication not doing it's job, as well missing feedback for replication lag.
  
  We found several tools similar to CRR Monitor <https://aws.amazon.com/answers/infrastructure-management/crr-monitor/>
  but we didn't have good experience with it as we manage tens of millions of objects which can cost several thousand of dollars per month for CloudWatch events, as well as additional management for services required by solution (DinamoDB, Kinesis etc.) another layer of "monitoring the monitor" required.
  
  For now we are using daily comparison of S3 objects inventory between source and destination using AWS Athena query, which also feels like a hack and it's not optimal for problem detection as inventory generated with daily frequency.
  
  I hope to see some feedback, suggestions, ideas or experience for alternatives. Ideal solution would be managed or "serverless" solution which will improve visibility on the process and will provide fast detection of errors.
  
  **Tags:** amazon-web-services, monitoring, amazon-s3
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2019-07-15. Score: 1 | Accepted answer
  
  My solution was based on the one of @username_0.
  I used S3 and created a daily S3 inventory file of the source bucket (which is replicated). Then I created a Athena DB and table using Glue (in CloudFormation).
  Then I implemented an S3 event which triggers a lambda. Every time a new manifest.checksum file was uploaded (= new inventory is finished).
  The lambda executes a query on the Athena table and checks if there are objects which [replication status](https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-status.html) is FAILED. If there are then the lambda publishes to a SNS topic to send an email.
  
  
  ---
  
  ### Answer by username_0 on 2018-11-14. Score: 2
  
  Still no answers, and still no CRR monitoring in place on AWS, here's minimalistic solution, any feedback and suggestions are welcome. This monitor will only show which S3 objects are missing from destination, it will not provide visibility on versions of replicated objects.
  
  Prerequisites
  
  ---
  
  1. [Enable inventory reports](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/configure-inventory.html) on `src` (Source) and `dst`
     (Destination) buckets
  2. [Create Athena tables](https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-inventory.html#storage-inventory-athena-query) (`src_inventory`, `dst_inventory`) based on inventory reports for each bucket
  
  Monitor
  
  ---
  
  Run Athena query after both inventory reports generated e.g. on daily basis, can be executed from [Lambda](https://docs.aws.amazon.com/athena/latest/APIReference/API_StartQueryExecution.html), [Step Function](https://github.com/anselmo/step_function_activity_patterns/blob/master/athena_query_pull_mode/fns/athena.js), build server or any type of cron job.
  
  Example of Athena query which should show number of missing objects on destination bucket at the time of generation of inventory reports:
  
  ```
  FROM src_inventory src 
  LEFT JOIN dst_inventory dst 
  ON src.key = dst.key 
  WHERE dst.key is NULL
  AND src.dt = '2018-11-11-08-00';
  
  ```
  
  The result can be reported as CloudWatch metric and monitored with CloudWatch alarm.
  
  
  ---
  
  ### Answer by username_2 on 2019-02-11. Score: 1
  
  At this point using S3 inventory to monitor replication is the best solution for large buckets, and [the solution suggested by AWS](https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-status.html) themselves. Since AWS don't report on the status directly, any other solution is likely to be both expensive and time consuming when that many objects are involved. It seems like you're on the right (and only?) track.
  
  An obvious idea would be to talk to AWS Support about it, ask them to add such a feature. But that's obvious.
  
  
  ---
  
  