Answers: <p>Netflix have presented their method of dealing with the problem of resource sprawl. They created the <a href="http://techblog.netflix.com/2013/01/janitor-monkey-keeping-cloud-tidy-and.html" rel="nofollow noreferrer">Janitor Monkey</a> which is responsible to clean-up resources when it is apparent that those are not being used.</p>
  
  <p>To track the created resources in Amazon AWS, generate an audit log, and allow to search through time (and history) they also created <a href="http://techblog.netflix.com/2012/11/edda-learn-stories-of-your-cloud.html" rel="nofollow noreferrer">Edda</a>. Edda allows to store and search through your AWS resources.</p>
  
  <p>There is some mention that Janitor Monkey might be integrated with Edda, but they have not published much information about these since.</p>
  
  <p>Today with <a href="https://aws.amazon.com/cloudtrail/" rel="nofollow noreferrer">AWS CloudTrail</a> and <a href="https://aws.amazon.com/config/" rel="nofollow noreferrer">AWS Config Rules</a> it is possible to achieve similar results without third-party software. When Edda and Janitor Monkey were announced (2012-2013) both of these services were not yet ready to solve these kinds of problems.</p>
  
  <p>With CloudTrail and Config, the tracking of resources is already in place. All to do is write a script that will decide what to do with these by going over the list from time to time.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-02T04:22:07.583Z
Answers: Thanks, looking at your links, aws may have a facility for this, so i may be looking in the wrong spot.
Answers: Chris
Answers: CC BY-SA 3.0
Answers: 2017-03-01T11:39:49.987Z
Answers: 2017-03-01T11:39:49.987Z
Answers: Evgeny Zislis
Answers: <p>A judicious usage of one of the cloud "orchestration" tools, such as <a href="https://www.terraform.io/" rel="noreferrer">Terraform</a> or possibly <a href="https://fugue.co/" rel="noreferrer">Fugue</a> seems to be the best way.</p>
  
  <p>You can start small, pick a less important and not very extensive environment, carefully decipher it into automatable code and proceed from there.</p>
  
  <p>Broadly this is referred to as infrastructure-as-code, for googling and other buzzword-oriented media applications.</p>
  
  <p>More specifically, say, if you had your environment described in Terraform's HCL files, you would have required ID's exported as "outputs" and could operate on those from scripts/Jenkins jobs/etc</p>
  
Answers: CC BY-SA 3.0
Answers: 2018-02-06T06:34:07.650Z
Answers: That is one way to handle the visibility issue. I am currently using TotalCloud [https://totalcloud.io]. It provides a visual topological view of the resources that are present on my cloud accounts.
Answers: Veer Abheek Singh Manhas
Answers: CC BY-SA 3.0
Answers: 2017-03-02T04:21:23.717Z
Answers: Thanks, looks like terraform might be an alternative to what i am using.  Doing some research.
Answers: Chris
Answers: CC BY-SA 3.0
Answers: 2017-03-03T15:30:38.517Z
Answers: Using a cloud orchestration tool to do my deployments and updates is the answer.  I am actually using CloudFormation to automate the initial build, I just didn't realize I could use it to update the piece I was looking for yet.
Answers: Chris
Answers: CC BY-SA 3.0
Answers: 2017-03-01T11:16:03.497Z
Answers: 2017-03-01T11:16:03.497Z
Answers: Michael Bravo
Body: <p>I am trying to automate my AWS application deployment with Jenkins. </p>
  
  <p>Right now if we want to update the application in any of the environments, say UAT, we build our docker images, find the current ECS Task and update it with the new image, find the running ECS Cluster, and update the tasks.  </p>
  
  <p>Broadly, what is best practice for keeping track of cloud resource Id's (ECS Cluster Id's, ECS Task Id's, EC2 Id's, etc) in your Continuous Integration environment?</p>
  
Tags: <continuous-integration><terraform><automation>
Title: How can I keep track of my Cloud Resource use?
UserDisplayName: Chris
ThreadText: # How can I keep track of my Cloud Resource use?
  
  (Asked by: username_0 on 2017-02-28)
  
  I am trying to automate my AWS application deployment with Jenkins.
  
  Right now if we want to update the application in any of the environments, say UAT, we build our docker images, find the current ECS Task and update it with the new image, find the running ECS Cluster, and update the tasks.
  
  Broadly, what is best practice for keeping track of cloud resource Id's (ECS Cluster Id's, ECS Task Id's, EC2 Id's, etc) in your Continuous Integration environment?
  
  **Tags:** continuous-integration, terraform, automation
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2017-03-01. Score: 10 | Accepted answer
  
  A judicious usage of one of the cloud "orchestration" tools, such as [Terraform](https://www.terraform.io/) or possibly [Fugue](https://fugue.co/) seems to be the best way.
  
  You can start small, pick a less important and not very extensive environment, carefully decipher it into automatable code and proceed from there.
  
  Broadly this is referred to as infrastructure-as-code, for googling and other buzzword-oriented media applications.
  
  More specifically, say, if you had your environment described in Terraform's HCL files, you would have required ID's exported as "outputs" and could operate on those from scripts/Jenkins jobs/etc
  
  > **Comments:**
  > **username_0**: Thanks, looks like terraform might be an alternative to what i am using.  Doing some research.  
  > **username_0**: Using a cloud orchestration tool to do my deployments and updates is the answer.  I am actually using CloudFormation to automate the initial build, I just didn't realize I could use it to update the piece I was looking for yet.  
  > **username_3**: That is one way to handle the visibility issue. I am currently using TotalCloud [https://totalcloud.io]. It provides a visual topological view of the resources that are present on my cloud accounts.  
  
  ---
  
  ### Answer by username_1 on 2017-03-01. Score: 1
  
  Netflix have presented their method of dealing with the problem of resource sprawl. They created the [Janitor Monkey](http://techblog.netflix.com/2013/01/janitor-monkey-keeping-cloud-tidy-and.html) which is responsible to clean-up resources when it is apparent that those are not being used.
  
  To track the created resources in Amazon AWS, generate an audit log, and allow to search through time (and history) they also created [Edda](http://techblog.netflix.com/2012/11/edda-learn-stories-of-your-cloud.html). Edda allows to store and search through your AWS resources.
  
  There is some mention that Janitor Monkey might be integrated with Edda, but they have not published much information about these since.
  
  Today with [AWS CloudTrail](https://aws.amazon.com/cloudtrail/) and [AWS Config Rules](https://aws.amazon.com/config/) it is possible to achieve similar results without third-party software. When Edda and Janitor Monkey were announced (2012-2013) both of these services were not yet ready to solve these kinds of problems.
  
  With CloudTrail and Config, the tracking of resources is already in place. All to do is write a script that will decide what to do with these by going over the list from time to time.
  
  > **Comments:**
  > **username_0**: Thanks, looking at your links, aws may have a facility for this, so i may be looking in the wrong spot.  
  
  ---
  
  