title: Caching and invalidating AWS Lambda response
body: <p>I am trying to implement a solution on AWS which is as follows:</p>
  
  <p>I have a crawler that will run once a day to index certain sites. I want to cache this data and expose it the the form of an API since after crawling, this data will not change for an entire day. After the crawler refetches, I want to invalidate and rebuild this cache to serve the updated data. I'm trying to use serverless architecture to build this.</p>
  
  <p><strong>Possible Solutions</strong></p>
  
  <p>It is clear that the crawler will run on AWS Lambda. What is unclear to me is how to manage the cache that will serve the data. Here are some solutions I thought of</p>
  
  <ol>
  <li><p><strong>S3 and Cloudfront for caching:</strong> After crawling, store the data in the form of .json files in S3 that will be cached using AWS Cloudfront. When the crawler refetches new data, it will rebuild these files and ask Cloudfront to invalidate the cache.</p></li>
  <li><p><strong>API Gateway DynamoDB</strong>: After Crawling store the data in DynamoDB which will be then served by API Gateway which is cached. The only problem here is how can I ask for this cache to be invalidated at the end of the day when the crawler re-crawls? Since the data will be static for a day, how can I not pay for the extra time that DynamoDB will be running (because if I implement caching on API Gateway, there will only one call to DynamoDB for caching after that it will be sitting idle for a day)</p></li>
  </ol>
  
  <p>Is there any other way that I am missing?</p>
  
  <p>Thanks!</p>