Answers: <p>I will give it a try and explain all required steps with examples.</p>
  
  <p>Please add comments with questions and improvement suggestions if anything is not well enough explained.</p>
  
  <p>The OP (original poster) refers to <code>awscli</code>; while I provide corresponding examples, I also discuss limitations of this approach and give examples of doing same with Python.</p>
  
  <h1>EC2 deployment</h1>
  
  <p>The purpose of this answer is to demonstrate that as OP correctly assumed, no Web UI access neither human interaction is mandatory to deploy a container on an AWS EC2 instance.</p>
  
  <p><strong>Scope and limitations</strong></p>
  
  <p>In this use case, we want to start just one container manually for demo/research purposes, therefore we explicitly decide to do no <em><a href="/questions/tagged/orchestration" class="post-tag" title="show questions tagged &#39;orchestration&#39;" rel="tag">orchestration</a></em>. </p>
  
  <p>For a complex and possibly more realistic deployment scenario, this example is therefore not reusable. For same simplicity reasons, we will use EC2 service and create there a single virtual machine.</p>
  
  <p>According to a short documentation research, there seems to be no official AMI with pre-installed Docker daemon. Thus we need to deploy some default EC2 instance and install there Docker daemon, then start the daemon. </p>
  
  <p>While using a private registry, a login there is required; for simplicity purposes, I give also an example to run a container from the public Docker Hub registry.</p>
  
  <p>Also, I am not going to provide a copy-paste automation script but single steps which can be easily adopted towards individual needs. In any case, intermediate results require parsing and passing operations which is then a pure programming activity concern.</p>
  
  <p>A boilerplate automation could be done either with bash using awscli or with Python, using the <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html" rel="nofollow noreferrer">boto3 SDK</a> for AWS.</p>
  
  <p>Python is worth to consider because you can do better processing of JSON responses you get from AWS. You can also tell Python to execute SSH login and bash commands on the remote host.</p>
  
  <p><strong>Execution plan</strong></p>
  
  <ul>
  <li>A. Identify recent Linux AMI (Amazon Machine Image) ID</li>
  <li>B. Create keypair and instance (further, also referenced as VM)</li>
  <li>C. Connect to the instance</li>
  <li>D. Install and configure Docker daemon</li>
  <li>E. Run container</li>
  </ul>
  
  <p><strong>A. Identify recent Linux AMI image ID</strong></p>
  
  <p>Search for a recent and up-to-date Amazon Linux image following the official <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/finding-an-ami.html#finding-an-ami-aws-cli" rel="nofollow noreferrer">documentation</a>:</p>
  
  <pre><code>$ aws ec2 describe-images --owners amazon --filters 'Name=name,Values=amzn2-ami-hvm-2.0.????????.?-x86_64-gp2' 'Name=state,Values=available' --query 'reverse(sort_by(Images, &amp;CreationDate))[:1].ImageId' --output text
  ami-01f14919ba412de34
  </code></pre>
  
  <p><strong>B. Create keypair and VM</strong></p>
  
  <p>With <code>awscli</code>, you would just say:</p>
  
  <pre><code>$ aws ec2 create-key-pair --key-name ec2-docker-test
  </code></pre>
  
  <p>However, the API sends you your public key wrapped in a JSON data message. Therefore, using <a href="https://blog.ipswitch.com/how-to-create-an-ec2-instance-with-python" rel="nofollow noreferrer">Python</a> it is way easier to process this data message to a <em>.pem</em> file:</p>
  
  <pre><code>import boto3
  ec2 = boto3.resource('ec2')
  
  # create a file to store the key locally
  outfile = open('ec2-keypair.pem','w')
  
  # call the boto ec2 function to create a key pair
  key_pair = ec2.create_key_pair(KeyName='ec2-docker-test')
  
  # capture the key and store it in a file
  KeyPairOut = str(key_pair.key_material)
  print(KeyPairOut)
  outfile.write(KeyPairOut)
  </code></pre>
  
  <p>With the AMI ID we have got previously we can now create the VM:</p>
  
  <pre><code>instances = self.ec2r.create_instances(
                   ImageId=AMI_ID,  
                   MinCount=1,
                   MaxCount=1,
                   InstanceType=instance_type,
                   KeyName='ec2-docker-test'
               )
  </code></pre>
  
  <p>AWS API allows also lookup of instance state and IP. With little bit more  Python/<em>boto3</em> stanzas you can acquire this data to know when you can proceed.</p>
  
  <p><strong>Note: Firewall consideration</strong></p>
  
  <p>To configure firewall access, please <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/ec2-example-security-group.html" rel="nofollow noreferrer">create a security group and link it to machines's VPC</a>. For readability reasons, example is not included.</p>
  
  <p><strong>C. Test SSH connection</strong></p>
  
  <p>As soon as the instance is up and running, you can do everything you like by means of <em>remote command execution</em>.</p>
  
  <p>In a Bash script, you would use <code>ssh</code> only or <code>scp</code> to upload a local bash script and run it remotely.</p>
  
  <pre><code> ssh -t &lt;REMOTE_IP&gt; -i &lt;KEY_FILE&gt; &lt;COMMAND&gt;
  </code></pre>
  
  <p>With Python, you use further modules like <a href="http://www.paramiko.org/" rel="nofollow noreferrer">paramiko</a> to establish SSH connection and run commands remotely. </p>
  
  <pre><code>client = paramiko.SSHClient()
  client.connect(ip, username=username, key_filename=key_filename, port=port,
                             timeout=timeout,
                             auth_timeout=timeout)
  
  stdin, stdout, stderr = ssh.exec_command("whoami")
  </code></pre>
  
  <p>For readability purposes, I list further commands unwrapped. In your script, you might want to use elementary helper subroutines to have calls like <code>ssh_call(command)</code>.</p>
  
  <p><strong>D. Install and configure Docker daemon</strong></p>
  
  <p>As said above, anything else from here is running remote commands. </p>
  
  <p>That is, the concluding steps are, following an example in the <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html" rel="nofollow noreferrer">official documentation</a>; proceeding is same as you would do it locally, but you need to wrap it in your script and pass over intermediate results if required:</p>
  
  <ul>
  <li>install and configure Docker daemon </li>
  <li>login to your Docker registry if needed</li>
  <li>pull image</li>
  <li>run Docker container</li>
  </ul>
  
  <pre><code>    sudo yum update -y
      sudo amazon-linux-extras install docker
      sudo service docker start
      sudo usermod -a -G docker ec2-user
      # relogin or continue with sudo, which you shouldn't
      aws ecr get-login --no-include-email --region region
  
  
  </code></pre>
  
  <p><strong>E. Run container</strong></p>
  
  <p>Pull custom image and run container from a private registry in AWS.</p>
  
  <pre><code>    docker pull aws_account_id.REGISTRY.ecr.REGION.amazonaws.com/REPOSITORY/IMAGE:TAG
      docker run --name MYSERVICENAME -d -p PORT_HOST:PORT_CONTAINER IMAGE:TAG
  
  </code></pre>
  
  <p>Pull an official public  image and run another container listening on port 8001 in a single step:</p>
  
  <pre><code>    docker run --name nginx -d -p 8001:80 nginx:latest
  
  </code></pre>
  
  <hr>
  
  <h1>Alternative scenario with ECS</h1>
  
  <p>As described in these Python/boto3 examples <a href="https://github.com/kgoedecke/python-ecs-example/blob/master/python_ecs_example/deployment.py" rel="nofollow noreferrer">here</a> and <a href="https://github.com/AlexIoannides/py-docker-aws-example-project/blob/master/deploy_to_aws.py" rel="nofollow noreferrer">here</a>, it's possible to define an machine cluster with the AWS ECS service and run containers there.</p>
  
  <p>Please note that ECS cluster offers you an <a href="https://aws.amazon.com/de/ecs/pricing/" rel="nofollow noreferrer">abstraction</a> either from AWS EC2 or AWS Fargate to deliver the actual virtual machines.</p>
  
  <p>If we go for EC2, again we need to <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/retrieve-ecs-optimized_AMI.html" rel="nofollow noreferrer">pick up</a> a recent machine image, but this we need a machine optimized for ECS. In a nutshell, these machines will have an ECS agent; through its configuration, we assign a machine to our target cluster. </p>
  
  <p>A small bonus is also that ECS machines have Docker daemon already preinstalled.</p>
  
  <p>So, we create an ECS cluster and add a virtual machine to it.</p>
  
  <pre><code>ecs_client.create_cluster(clusterName=cluster_name)
  
  ec2_client.run_instances(
          ImageId=AMI_ID,
          MinCount=1,
          MaxCount=1,
          InstanceType="t2.micro",
          IamInstanceProfile={
              "Name": "ecsInstanceRole"
          },
          UserData="#!/bin/bash \n echo ECS_CLUSTER=" + cluster_name + " &gt;&gt; /etc/ecs/ecs.config"
      )
  
  </code></pre>
  
  <p>In the cluster, we can create a task definition of the service which are both abstract of the container concept.</p>
  
  <pre><code>ecs_client.register_task_definition(
          containerDefinitions=[
          {
            "name": "&lt;MY_SERVICE&gt;",
            "image": "&lt;MY_IMAGE&gt;",
            "portMappings": [
              {
                "containerPort": 80,
                "hostPort": 80
              }
            ]
          }
          ],
          family="hello_world"
      )
  </code></pre>
  
  <p>Now, possibly you ask yourself, how would the Docker daemon on the instance know about your private registry to pull the specified image? For that, the machine's ECS agent needs <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth-container-instances.html" rel="nofollow noreferrer">additional configuration</a>.</p>
  
  <p>Now, you can launch the service, which will use the previously defined task to tell the previously authorized Docker daemon to pull the previously built and published image from ECR and run the container on the machine you've included in the ECS cluster <s>in the data center house that Jeff built.</s></p>
  
  <pre><code>ecs_client.create_service(
          cluster=cluster_name,
          serviceName=service_name,
          taskDefinition=task_name,
          desiredCount=1,
          clientToken='request_identifier_string',
          deploymentConfiguration={
              'maximumPercent': 200,
              'minimumHealthyPercent': 50
          }
      )
  </code></pre>
  
  <p>Configuration of security groups in the ECS context has been highlighted on StackOverlow <a href="https://stackoverflow.com/questions/57827251/change-aws-ecs-services-security-groups">already</a>. In short, these are further calls to the API parts related to security group management.</p>
  
  <h1>Conclusion</h1>
  
  <p>Both Bash and Python implementation options have their benefits and drawbacks.</p>
  
  <p>Seen on long term, Python being a general purpose programming language with large community, allows complex string manipulation and abstractions from a cloud API or a system process.</p>
  
  <p>Without knowing the real goal of the deployment, an end-to end of above steps might not be worth the effort in terms that for deploying a set of related containers which is more typical scenario, other approaches could be more pragmatic. </p>
  
  <p>Before stepping into technical implementation, best is to make well elaborated decisions and choices addressing business goals and budget:</p>
  
  <ul>
  <li>how many times will automation need to run and in which time period vs. time to implement it end-to-end from the beginning?</li>
  <li>assess whether you need only EC2, or ECS with EC2 either <a href="https://www.dragonspears.com/blog/aws-container-orchestration-101-ecs-vs-fargate-vs-eks" rel="nofollow noreferrer">Fargate</a> for stateless tasks, or maybe you need Kubernetes?</li>
  </ul>
  
  <p>Further reading: "<em>The Benefits of Managed Kubernetes vs. Amazon ECS</em>" (short argument from a cloud consultant <a href="https://www.fairwinds.com/blog/the-benefits-of-managed-kubernetes-vs-amazon-ecs" rel="nofollow noreferrer">blog</a> as of August 2019).</p>
  
  <blockquote>
    <p>A <a href="https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/" rel="nofollow noreferrer">2018 CNCF</a> survey cites <strong>83%</strong> of organizations use Kubernetes as
    their container orchestration solution vs. <strong>24%</strong> for ECS.</p>
  </blockquote>
  
Answers: CC BY-SA 4.0
Answers: 2020-01-04T19:54:47.417Z
Answers: Thanks for the detailed explanation. Do you might expanding on your answer how this process would be automated when ECS is used instead of EC2? The documentation mainly refers to using AMI's that are already, but not how to run my custom docker image that's on the ECR.
Answers: Kyu96
Answers: CC BY-SA 4.0
Answers: 2020-01-04T20:40:27.760Z
Answers: With boto3, you can wire up an ECS client object (where you define clusters and services) with ECR client (where you get repository deployment token). I've found a small and nice GitHub example on this, please tell me if you want me to add an answer section along these lines. https://github.com/AlexIoannides/py-docker-aws-example-project/blob/master/deploy_to_aws.py
Answers: Ta Mu
Answers: CC BY-SA 4.0
Answers: 2020-01-04T21:11:44.660Z
Answers: Hey. Thanks for the GitHub link. Yes would be nice if you could expand your answer. Assume the same situation - the docker image is already uploaded to the ECR and just needs to be prepared and started up, and the network settings (inbound and outbound ports) must be adjusted.
Answers: Kyu96
Answers: CC BY-SA 4.0
Answers: 2020-01-05T07:32:44.207Z
Answers: Will do so in a couple of hours.
Answers: Ta Mu
Answers: CC BY-SA 4.0
Answers: 2020-01-05T09:53:48.733Z
Answers: Alright, awesome. Because it would be very interesting by seeing how to solve this issue by using different approaches - by using ECS and by using EC2.
Answers: Kyu96
Answers: CC BY-SA 4.0
Answers: 2020-01-06T14:28:35.317Z
Answers: Done. Please review.
Answers: Ta Mu
Answers: CC BY-SA 4.0
Answers: 2020-01-06T18:12:15.040Z
Answers: Seems good for the most part. Thanks for adding the additional details. I am trying to replicate the ECS scenario at the moment, but the IAM profiles seem to cause problems: `botocore.exceptions.ClientError: An error occurred (InvalidParameterValue) when calling the RunInstances operation: Value (ecsInstanceRole) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name`
Answers: Kyu96
Answers: CC BY-SA 4.0
Answers: 2020-01-06T21:02:56.557Z
Answers: Yes, you need to provide ECS client a role it can come up with to ECR. As a quick shot: please check this https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html and compare to boto3 iam specs and tell me whether it helps.
Answers: Ta Mu
Answers: CC BY-SA 4.0
Answers: 2020-01-07T00:38:34.360Z
Answers: Okay, after several issues I managed to got the instance up and running. However it is not accessible the way it should be. Locally a .sh script is executed with the CMD command that ensures that the services in the container are started. My guess is this command got overwritten by AWS. Do I have to specify it manually in the task/service definition? How can I ensure my command gets executed so that the container is actually started correctly. I don't know how I can verify whats going on.
Answers: Kyu96
Answers: CC BY-SA 4.0
Answers: 2020-01-07T06:37:05.070Z
Answers: Possibly you need to step into some debugging: SSH to ec2 instance, check container logs, open a terminal to the container and validate what is happening. Maybe post a new question :-)
Answers: Ta Mu
Answers: CC BY-SA 4.0
Answers: 2020-01-07T12:04:23.830Z
Answers: I'll open a new question for this issue then :)
Answers: Kyu96
Answers: CC BY-SA 4.0
Answers: 2020-01-03T11:50:29.897Z
Answers: 2020-01-09T19:14:43.570Z
Answers: 2020-01-09T19:14:43.570Z
Answers: Ta Mu
Body: <p>This question is related to my <a href="https://devops.stackexchange.com/questions/10180/aws-ecr-start-uploaded-docker-image">previous one</a> - but a bit more descriptive.
  I am trying to build automate the generation and deployment of a docker container. The docker container is created with <a href="https://www.packer.io/" rel="noreferrer">packer</a>.
  After the creation of the docker image, I am using packer's post processing modules to upload it to the AWS ECR:</p>
  
  <pre><code>{
      "type": "docker-push",
      "ecr_login": true,
      "aws_access_key": "&lt;snip&gt;",
      "aws_secret_key": "&lt;snipe&gt;",
      "login_server": "https://&lt;snip&gt;.dkr.ecr.eu-west-1.amazonaws.com/"
  }
  </code></pre>
  
  <p>This works just fine. The docker image got uploaded to my repository in the ECR.
  Now, the next step is where I am stuck. 
  I want to run start the container on AWS.</p>
  
  <p>I am thinking a little script that uses the aws-cli commands could be suited for that.
  I do not want to use the web ui - and I am sure this can be done over the command line, especially when someone wants to integrate this into a pipeline of some sort.</p>
  
  <p>However, I have a few questions, as I am not sure if I understand the big picture correctly. My docker image is available over the ECR. To run it, I need some base instance, right? According to a reply in the previous thread, the are various way of running docker containers, either by using the ECS or by spinning up a EC2 instance. Since this is related to some research project where docker and virtualization is analyzed I am questioning what would be the best option for my case. Probably EC2? It's important that I can automate it so that no user interaction is required.</p>
  
  <p>I read the docs and checked several tutorials but I still am not entirely sure how to everything correctly in the right order. Especially part where the environment is set up and the image is started.</p>
  
  <p>I know that I can list my images with:</p>
  
  <p><code>aws ecr list-images --repository-name prototypes</code>.</p>
  
  <p>How would I fire up a new instance and run my image on it?</p>
  
  <p>If anything is unclear, please let me know so that I can update my question to be as detailed as possible.</p>
  
Comments: CC BY-SA 4.0
Comments: 2019-12-28T12:11:02.967Z
Comments: I'm interested in doing so as well, i.e. deploy Docker containers on AWS. 
  If this is anyway a research project with example deployments maybe there is some GitHub write-up so that a direct code contribution would be possible? Are you going to deploy a bunch of containers i.e. after deployment, what is the orchestration strategy? I.e. maybe you need is EKS?
Comments: Ta Mu
Comments: CC BY-SA 4.0
Comments: 2020-01-02T00:07:59.993Z
Comments: @Peter I probably can not publish the code on github, because the research project is under NDA. However I can maybe update this thread when someone provides an answer that helped me to achieve the scenario described above. 
  I still haven't manage to solve the problem and I hope that someone who is experienced with AWS can give a descriptive explanation.
Comments: Kyu96
Tags: <docker><amazon-ec2><deployment><virtualization><aws-ecr>
Title: Automate custom docker image deployment on AWS
UserDisplayName: Kyu96
ThreadText: # Automate custom docker image deployment on AWS
  
  (Asked by: username_0 on 2019-12-25)
  
  This question is related to my [previous one](https://devops.stackexchange.com/questions/10180/aws-ecr-start-uploaded-docker-image) - but a bit more descriptive.
  I am trying to build automate the generation and deployment of a docker container. The docker container is created with [packer](https://www.packer.io/).
  After the creation of the docker image, I am using packer's post processing modules to upload it to the AWS ECR:
  
  ```
  {
      "type": "docker-push",
      "ecr_login": true,
      "aws_access_key": "<snip>",
      "aws_secret_key": "<snipe>",
      "login_server": "https://<snip>.dkr.ecr.eu-west-1.amazonaws.com/"
  }
  
  ```
  
  This works just fine. The docker image got uploaded to my repository in the ECR.
  Now, the next step is where I am stuck.
  I want to run start the container on AWS.
  
  I am thinking a little script that uses the aws-cli commands could be suited for that.
  I do not want to use the web ui - and I am sure this can be done over the command line, especially when someone wants to integrate this into a pipeline of some sort.
  
  However, I have a few questions, as I am not sure if I understand the big picture correctly. My docker image is available over the ECR. To run it, I need some base instance, right? According to a reply in the previous thread, the are various way of running docker containers, either by using the ECS or by spinning up a EC2 instance. Since this is related to some research project where docker and virtualization is analyzed I am questioning what would be the best option for my case. Probably EC2? It's important that I can automate it so that no user interaction is required.
  
  I read the docs and checked several tutorials but I still am not entirely sure how to everything correctly in the right order. Especially part where the environment is set up and the image is started.
  
  I know that I can list my images with:
  
  `aws ecr list-images --repository-name prototypes`.
  
  How would I fire up a new instance and run my image on it?
  
  If anything is unclear, please let me know so that I can update my question to be as detailed as possible.
  
  **Tags:** docker, amazon-ec2, deployment, virtualization, aws-ecr
  
  > **Comments:**
  > **username_1**: I'm interested in doing so as well, i.e. deploy Docker containers on AWS.  If this is anyway a research project with example deployments maybe there is some GitHub write-up so that a direct code contribution would be possible? Are you going to deploy a bunch of containers i.e. after deployment, what is the orchestration strategy? I.e. maybe you need is EKS?  
  > **username_0**: @Peter I probably can not publish the code on github, because the research project is under NDA. However I can maybe update this thread when someone provides an answer that helped me to achieve the scenario described above.  I still haven't manage to solve the problem and I hope that someone who is experienced with AWS can give a descriptive explanation.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-01-03. Score: 3 | Accepted answer
  
  I will give it a try and explain all required steps with examples.
  
  Please add comments with questions and improvement suggestions if anything is not well enough explained.
  
  The OP (original poster) refers to `awscli`; while I provide corresponding examples, I also discuss limitations of this approach and give examples of doing same with Python.
  
  # EC2 deployment
  
  The purpose of this answer is to demonstrate that as OP correctly assumed, no Web UI access neither human interaction is mandatory to deploy a container on an AWS EC2 instance.
  
  **Scope and limitations**
  
  In this use case, we want to start just one container manually for demo/research purposes, therefore we explicitly decide to do no *[orchestration](/questions/tagged/orchestration "show questions tagged 'orchestration'")*.
  
  For a complex and possibly more realistic deployment scenario, this example is therefore not reusable. For same simplicity reasons, we will use EC2 service and create there a single virtual machine.
  
  According to a short documentation research, there seems to be no official AMI with pre-installed Docker daemon. Thus we need to deploy some default EC2 instance and install there Docker daemon, then start the daemon.
  
  While using a private registry, a login there is required; for simplicity purposes, I give also an example to run a container from the public Docker Hub registry.
  
  Also, I am not going to provide a copy-paste automation script but single steps which can be easily adopted towards individual needs. In any case, intermediate results require parsing and passing operations which is then a pure programming activity concern.
  
  A boilerplate automation could be done either with bash using awscli or with Python, using the [boto3 SDK](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) for AWS.
  
  Python is worth to consider because you can do better processing of JSON responses you get from AWS. You can also tell Python to execute SSH login and bash commands on the remote host.
  
  **Execution plan**
  
  * A. Identify recent Linux AMI (Amazon Machine Image) ID
  * B. Create keypair and instance (further, also referenced as VM)
  * C. Connect to the instance
  * D. Install and configure Docker daemon
  * E. Run container
  
  **A. Identify recent Linux AMI image ID**
  
  Search for a recent and up-to-date Amazon Linux image following the official [documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/finding-an-ami.html#finding-an-ami-aws-cli):
  
  ```
  $ aws ec2 describe-images --owners amazon --filters 'Name=name,Values=amzn2-ami-hvm-2.0.????????.?-x86_64-gp2' 'Name=state,Values=available' --query 'reverse(sort_by(Images, &CreationDate))[:1].ImageId' --output text
  ami-01f14919ba412de34
  
  ```
  
  **B. Create keypair and VM**
  
  With `awscli`, you would just say:
  
  ```
  $ aws ec2 create-key-pair --key-name ec2-docker-test
  
  ```
  
  However, the API sends you your public key wrapped in a JSON data message. Therefore, using [Python](https://blog.ipswitch.com/how-to-create-an-ec2-instance-with-python) it is way easier to process this data message to a *.pem* file:
  
  ```
  import boto3
  ec2 = boto3.resource('ec2')
  
  # create a file to store the key locally
  outfile = open('ec2-keypair.pem','w')
  
  # call the boto ec2 function to create a key pair
  key_pair = ec2.create_key_pair(KeyName='ec2-docker-test')
  
  # capture the key and store it in a file
  KeyPairOut = str(key_pair.key_material)
  print(KeyPairOut)
  outfile.write(KeyPairOut)
  
  ```
  
  With the AMI ID we have got previously we can now create the VM:
  
  ```
  instances = self.ec2r.create_instances(
                   ImageId=AMI_ID,  
                   MinCount=1,
                   MaxCount=1,
                   InstanceType=instance_type,
                   KeyName='ec2-docker-test'
               )
  
  ```
  
  AWS API allows also lookup of instance state and IP. With little bit more Python/*boto3* stanzas you can acquire this data to know when you can proceed.
  
  **Note: Firewall consideration**
  
  To configure firewall access, please [create a security group and link it to machines's VPC](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/ec2-example-security-group.html). For readability reasons, example is not included.
  
  **C. Test SSH connection**
  
  As soon as the instance is up and running, you can do everything you like by means of *remote command execution*.
  
  In a Bash script, you would use `ssh` only or `scp` to upload a local bash script and run it remotely.
  
  ```
   ssh -t <REMOTE_IP> -i <KEY_FILE> <COMMAND>
  
  ```
  
  With Python, you use further modules like [paramiko](http://www.paramiko.org/) to establish SSH connection and run commands remotely.
  
  ```
  client = paramiko.SSHClient()
  client.connect(ip, username=username, key_filename=key_filename, port=port,
                             timeout=timeout,
                             auth_timeout=timeout)
  
  stdin, stdout, stderr = ssh.exec_command("whoami")
  
  ```
  
  For readability purposes, I list further commands unwrapped. In your script, you might want to use elementary helper subroutines to have calls like `ssh_call(command)`.
  
  **D. Install and configure Docker daemon**
  
  As said above, anything else from here is running remote commands.
  
  That is, the concluding steps are, following an example in the [official documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html); proceeding is same as you would do it locally, but you need to wrap it in your script and pass over intermediate results if required:
  
  * install and configure Docker daemon
  * login to your Docker registry if needed
  * pull image
  * run Docker container
  
  ```
      sudo yum update -y
      sudo amazon-linux-extras install docker
      sudo service docker start
      sudo usermod -a -G docker ec2-user
      # relogin or continue with sudo, which you shouldn't
      aws ecr get-login --no-include-email --region region
  
  ```
  
  **E. Run container**
  
  Pull custom image and run container from a private registry in AWS.
  
  ```
      docker pull aws_account_id.REGISTRY.ecr.REGION.amazonaws.com/REPOSITORY/IMAGE:TAG
      docker run --name MYSERVICENAME -d -p PORT_HOST:PORT_CONTAINER IMAGE:TAG
  
  ```
  
  Pull an official public image and run another container listening on port 8001 in a single step:
  
  ```
      docker run --name nginx -d -p 8001:80 nginx:latest
  
  ```
  
  ---
  
  # Alternative scenario with ECS
  
  As described in these Python/boto3 examples [here](https://github.com/kgoedecke/python-ecs-example/blob/master/python_ecs_example/deployment.py) and [here](https://github.com/AlexIoannides/py-docker-aws-example-project/blob/master/deploy_to_aws.py), it's possible to define an machine cluster with the AWS ECS service and run containers there.
  
  Please note that ECS cluster offers you an [abstraction](https://aws.amazon.com/de/ecs/pricing/) either from AWS EC2 or AWS Fargate to deliver the actual virtual machines.
  
  If we go for EC2, again we need to [pick up](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/retrieve-ecs-optimized_AMI.html) a recent machine image, but this we need a machine optimized for ECS. In a nutshell, these machines will have an ECS agent; through its configuration, we assign a machine to our target cluster.
  
  A small bonus is also that ECS machines have Docker daemon already preinstalled.
  
  So, we create an ECS cluster and add a virtual machine to it.
  
  ```
  ecs_client.create_cluster(clusterName=cluster_name)
  
  ec2_client.run_instances(
          ImageId=AMI_ID,
          MinCount=1,
          MaxCount=1,
          InstanceType="t2.micro",
          IamInstanceProfile={
              "Name": "ecsInstanceRole"
          },
          UserData="#!/bin/bash \n echo ECS_CLUSTER=" + cluster_name + " >> /etc/ecs/ecs.config"
      )
  
  ```
  
  In the cluster, we can create a task definition of the service which are both abstract of the container concept.
  
  ```
  ecs_client.register_task_definition(
          containerDefinitions=[
          {
            "name": "<MY_SERVICE>",
            "image": "<MY_IMAGE>",
            "portMappings": [
              {
                "containerPort": 80,
                "hostPort": 80
              }
            ]
          }
          ],
          family="hello_world"
      )
  
  ```
  
  Now, possibly you ask yourself, how would the Docker daemon on the instance know about your private registry to pull the specified image? For that, the machine's ECS agent needs [additional configuration](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth-container-instances.html).
  
  Now, you can launch the service, which will use the previously defined task to tell the previously authorized Docker daemon to pull the previously built and published image from ECR and run the container on the machine you've included in the ECS cluster ~~in the data center house that Jeff built.~~
  
  ```
  ecs_client.create_service(
          cluster=cluster_name,
          serviceName=service_name,
          taskDefinition=task_name,
          desiredCount=1,
          clientToken='request_identifier_string',
          deploymentConfiguration={
              'maximumPercent': 200,
              'minimumHealthyPercent': 50
          }
      )
  
  ```
  
  Configuration of security groups in the ECS context has been highlighted on StackOverlow [already](https://stackoverflow.com/questions/57827251/change-aws-ecs-services-security-groups). In short, these are further calls to the API parts related to security group management.
  
  # Conclusion
  
  Both Bash and Python implementation options have their benefits and drawbacks.
  
  Seen on long term, Python being a general purpose programming language with large community, allows complex string manipulation and abstractions from a cloud API or a system process.
  
  Without knowing the real goal of the deployment, an end-to end of above steps might not be worth the effort in terms that for deploying a set of related containers which is more typical scenario, other approaches could be more pragmatic.
  
  Before stepping into technical implementation, best is to make well elaborated decisions and choices addressing business goals and budget:
  
  * how many times will automation need to run and in which time period vs. time to implement it end-to-end from the beginning?
  * assess whether you need only EC2, or ECS with EC2 either [Fargate](https://www.dragonspears.com/blog/aws-container-orchestration-101-ecs-vs-fargate-vs-eks) for stateless tasks, or maybe you need Kubernetes?
  
  Further reading: "*The Benefits of Managed Kubernetes vs. Amazon ECS*" (short argument from a cloud consultant [blog](https://www.fairwinds.com/blog/the-benefits-of-managed-kubernetes-vs-amazon-ecs) as of August 2019).
  
  > A [2018 CNCF](https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/) survey cites **83%** of organizations use Kubernetes as
  > their container orchestration solution vs. **24%** for ECS.
  
  > **Comments:**
  > **username_0**: Thanks for the detailed explanation. Do you might expanding on your answer how this process would be automated when ECS is used instead of EC2? The documentation mainly refers to using AMI's that are already, but not how to run my custom docker image that's on the ECR.  
  > **username_1**: With boto3, you can wire up an ECS client object (where you define clusters and services) with ECR client (where you get repository deployment token). I've found a small and nice GitHub example on this, please tell me if you want me to add an answer section along these lines. https://github.com/AlexIoannides/py-docker-aws-example-project/blob/master/deploy_to_aws.py  
  > **username_0**: Hey. Thanks for the GitHub link. Yes would be nice if you could expand your answer. Assume the same situation - the docker image is already uploaded to the ECR and just needs to be prepared and started up, and the network settings (inbound and outbound ports) must be adjusted.  
  > **username_1**: Will do so in a couple of hours.  
  > **username_0**: Alright, awesome. Because it would be very interesting by seeing how to solve this issue by using different approaches - by using ECS and by using EC2.  
  > **username_1**: Done. Please review.  
  > **username_0**: Seems good for the most part. Thanks for adding the additional details. I am trying to replicate the ECS scenario at the moment, but the IAM profiles seem to cause problems: `botocore.exceptions.ClientError: An error occurred (InvalidParameterValue) when calling the RunInstances operation: Value (ecsInstanceRole) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name`  
  > **username_1**: Yes, you need to provide ECS client a role it can come up with to ECR. As a quick shot: please check this https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html and compare to boto3 iam specs and tell me whether it helps.  
  > **username_0**: Okay, after several issues I managed to got the instance up and running. However it is not accessible the way it should be. Locally a .sh script is executed with the CMD command that ensures that the services in the container are started. My guess is this command got overwritten by AWS. Do I have to specify it manually in the task/service definition? How can I ensure my command gets executed so that the container is actually started correctly. I don't know how I can verify whats going on.  
  > **username_1**: Possibly you need to step into some debugging: SSH to ec2 instance, check container logs, open a terminal to the container and validate what is happening. Maybe post a new question :-)  
  > **username_0**: I'll open a new question for this issue then :)  
  
  ---
  
  