Question: How do I resolve the "DockerTimeoutError" error in AWS Batch?

The jobs in my AWS Batch compute environment are failing and are returning the following error: "DockerTimeoutError: Could not transition to created; timed out after waiting 4m0s." How do I troubleshoot "DockerTimeoutError" errors in AWS Batch?

Answer: "The jobs in my AWS Batch compute environment are failing and are returning the following error: "DockerTimeoutError: Could not transition to created; timed out after waiting 4m0s." How do I troubleshoot "DockerTimeoutError" errors in AWS Batch?Short descriptionIf your docker start and docker create API calls take longer than four minutes, then AWS Batch returns a DockerTimeoutError error.Note: The default timeout limit set by the Amazon Elastic Container Service (Amazon ECS) container agent is four minutes.The error can occur for a variety of reasons, but it's commonly caused by one of the following:The ECS instance volumes of the AWS Batch compute environment are under high I/O pressure from all the other jobs in your queue. These jobs, which are created on and run on the ECS instance, can deplete the burst balance. To resolve this issue, follow the steps in the Resolve any burst balance issues section of this article.Stopped ECS containers aren't being cleaned fast enough to free up the Docker daemon. You can experience Docker issues if you're using a customized Amazon Machine Image (AMI) instead of the default AMI provided by AWS Batch. The default AMI for AWS Batch optimizes your Amazon ECS cleanup settings. To resolve this issue, follow the steps in the Resolve any Docker issues section of this article.If neither of these issues is causing the error, then you can further troubleshoot the issue by doing the following:Check your Docker logs to identify the source of the error.Run the Amazon ECS logs collector script on the ECS instances in the ECS cluster associated with your AWS Batch compute environment.ResolutionResolve any burst balance issuesCheck the burst balance of your ECS instance1.    Open the Amazon ECS console.2.    In the navigation pane, choose Clusters. Then, choose the cluster that contains your job.Note: The name of the cluster starts with the name of the compute environment, followed by _Batch_ and a random hash of numbers and letters.3.    Choose the ECS Instances tab.4.    From the EC2 Instance column, choose your instance.Note: To find the failed job's instance ID, run the AWS Batch describe-jobs command. The instance ID appears in the output for containerInstanceArn.5.    On the Descriptions tab in the Amazon EC2 console, under Block devices, choose the link for your volume.6.    On the block device pop-up window, for EBS ID, choose your volume.7.    Choose the Monitoring tab. Then, choose Burst Balance to check your burst balance metrics. If your burst balance drops to 0, then your burst balance is depleted.Create a launch template for your managed compute environmentNote: If you change the launch template, you must create a new compute environment.1.    Open the Amazon EC2 console, and then choose Launch Templates.2.    Choose Create launch template.3.    For AMI ID, select the default Amazon ECS optimized AMI.4.    In the Storage (Volumes) section, choose a volume type in the Volume type column. Then, enter an integer value in the Size(GiB) column.Note: If you choose Provisioned IOPS SSD (io1) for your volume type, enter an integer value that's permitted for IOPS.5.    Choose Create launch template.6.    Use your new launch template to create a new managed compute environment.Create an AWS Batch compute environment with your AMINote: If you change the AMI, you must create a new compute environment because the AMI ID parameter can't be updated.1.    Open the Amazon EC2 console.2.    Choose Launch instance.3.    Follow the steps in the setup wizard to create your instance.Important: On the Add Storage page, modify the volume type or size of your instance. The larger the volume size, the greater the baseline performance is and the slower it replenishes the burst balance. To get better performance for high I/O loads, change the volume to type io1.4.    Create a compute resource AMI from your instance.5.    Create a compute environment for AWS Batch that includes your AMI ID.Resolve any Docker issuesBy default, the Amazon ECS container agent automatically cleans up stopped tasks and Docker images that your container instances aren't using. If you run new jobs with new images, then your container storage might fill up with Docker images you aren't using.1.    Use SSH to connect to the container instance for your AWS Batch compute environment.2.    To inspect the Amazon ECS container agent, run the Docker inspect ecs-agent command. Then, review the env section in the output.Note: You can reduce the values of the following variables to speed up task and image cleanup:ECS_ENGINE_TASK_CLEANUP_WAIT_DURATIONECS_IMAGE_CLEANUP_INTERVALECS_IMAGE_MINIMUM_CLEANUP_AGEECS_NUM_IMAGES_DELETE_PER_CYCLEYou can also use tunable parameters for automated task and image cleanup.3.    Create a new AMI with updated values.-or-Create a launch template with the user data that includes your new environment variables.To create a new AMI with updated values1.    Set your agent configuration parameters in the /etc/ecs/ecs.config file.2.    Restart your container agent.3.    Create a compute resource AMI from your instance.4.    Create compute environment for AWS Batch that includes your AMI ID.To create a launch template with the user data that includes your new environment variables1.    Create a launch template with user data.For example, the user data in the following MIME multi-part file overrides the default Docker image cleanup settings for a compute resource:MIME-Version: 1.0Content-Type: multipart/mixed; boundary="==MYBOUNDARY=="--==MYBOUNDARY==Content-Type: text/x-shellscript; charset="us-ascii"#!/bin/bashecho ECS_IMAGE_CLEANUP_INTERVAL=60m >> /etc/ecs/ecs.configecho ECS_IMAGE_MINIMUM_CLEANUP_AGE=60m >> /etc/ecs/ecs.config--==MYBOUNDARY==--2.    Use your new launch template to create a managed compute environment.Related informationAWS services that publish CloudWatch metricsCompute resource AMIsamazon-ecs-agent (AWS GitHub)Follow"


Question: How can I troubleshoot the "Waiting for the slave SQL thread to free enough relay log space" error in Amazon Aurora MySQL?

I received the following error in the output of the SHOW SLAVE STATUS command that is working as a replica of binary log replication in Amazon Aurora MySQL:"Waiting for the slave SQL thread to free enough relay log space"How can I troubleshoot and resolve this error?

Answer: "I received the following error in the output of the SHOW SLAVE STATUS command that is working as a replica of binary log replication in Amazon Aurora MySQL:"Waiting for the slave SQL thread to free enough relay log space"How can I troubleshoot and resolve this error?Short descriptionWhen Aurora MySQL is a replica of binary log replication, it runs the I/O thread and the SQL thread in the same way as MySQL. The I/O thread reads binary logs from the primary, and then saves them as relay logs in the replica DB instance. The SQL thread processes the events in the relay logs, and then deletes the relay logs when the events in the relay logs are processed.If the SQL thread doesn't process events fast enough to catch up with the speed that the relay logs are being generated at, the amount of relay logs increase.When the global variable relay_log_space_limit is set to larger than 0 and the total size of all relay logs reach the limit, new relay logs aren't saved. Until the relay log space becomes available again, the output of SHOW SLAVE STATUS shows the message "Waiting for the slave SQL thread to free enough relay log space" in the Slave_IO_State field.In Aurora MySQL, the relay_log_space_limit is set to 1000000000 (953.6 MiB) and can't be modified. This prevents the cluster volume from becoming unnecessary large. When the total size of all relay logs reaches 1000000000 bytes (953.6 MiB), the I/O thread stops saving relay logs. It waits for the SQL thread to process events and delete the existing logs. Slave_IO_State then shows the message "Waiting for the slave SQL thread to free enough relay log space". If the SQL thread isn't stopped, the relay logs are eventually deleted, and the I/O thread resumes saving new relay logs.This also means that replication lag exists because the SQL isn't fast enough to catch up with the generation of relay logs by the I/O thread. Even if relay_log_space_limit is modified to a larger value, the relay logs still further accumulate, and the issue isn't resolved until the SQL thread catches up.You can view the current relay log space, the status of the I/O thread, and the status of the SQL thread in the output of the SHOW SLAVE STATUS command.Slave_IO_State: Waiting for the slave SQL thread to free enough relay log spaceMaster_Log_File: mysql-bin-changelog.237029Read_Master_Log_Pos: 55356151Relay_Master_Log_File: mysql-bin-changelog.237023Exec_Master_Log_Pos: 120Relay_Log_Space: 1000002403Master_Log_File and Read_Master_Log_Pos show the binary log file name and the position where the I/O thread completed reading and saving. Relay_Master_Log_File and Exec_Master_Log_Pos show the binary log file name and the position where the SQL thread is processing. Although what the SQL thread actually reads are relay logs, the corresponding binary log file name in the primary DB instance and the position is displayed.When Master_Log_File is different from Relay_Master_Log_File, the SQL thread isn't fast enough. If Master_Log_File and Relay_Master_Log_File are the same, the I/O thread might be contributing to the lag.The following factors can cause insufficient performance of the SQL thread:Long-running queries on the primary DB instanceInsufficient DB instance class size or storageParallel queries performed on the primary DB instanceBinary logs synced to the disk on the replica DB instanceBinlog_format on the replica DB instance is set to ROWFor more information on resolving these issues, see How can I troubleshoot high replica lag with Amazon RDS for MySQLAdditionally, the following factors can also impact the performance of the SQL thread:A very large Transaction History List Length (HLL) on the replica DB instanceLess-than-efficient I/O operations on the replica DB instanceTables with lots of secondary indexes on the replica DB instanceResolutionAs long as there are writes happening in your replica, you don't need to worry about relay log space. You can monitor this using the Write Throughput metric in Enhanced Monitoring.Instead, focus on troubleshooting the replica's performance. For more details, see How can I troubleshoot high replica lag with Amazon RDS for MySQL and Why did my Amazon Aurora read replica fall behind and restart?Related informationMySQL documentation for Replica server options and variablesFollow"


Question: How do I test if my delegated subdomain resolves correctly?

"I want to test and confirm if my delegated subdomain resolves correctly. If the subdomain doesn't resolve correctly, then I want to troubleshoot it."

Answer: "I want to test and confirm if my delegated subdomain resolves correctly. If the subdomain doesn't resolve correctly, then I want to troubleshoot it.Short descriptionYou can configure a parent zone for your apex domain (such as example.com) using Amazon Route 53 or a third-party DNS provider. You can also use your DNS provider to set up a delegation set for the subdomain (such as www.example.com).If you use a separate delegation set for your subdomain, then you can have the following configurations:An apex domain and a subdomain that both use Route 53An apex domain that uses a third-party DNS service and a subdomain that uses Route 53An apex domain that uses Route 53 and a subdomain that's delegated to a third-party DNS serviceTo verify if your subdomain resolves correctly and troubleshoot as needed, complete the following steps depending on your DNS provider and configuration.ResolutionNote: The following commands apply only to Amazon Elastic Compute Cloud (Amazon EC2) Linux instances. If you use Amazon EC2 for Windows, you can use third-party web tools such as DiG GUI and Dig web interface for troubleshooting.An apex domain and a subdomain that both use Route 531.    To check that your subdomain resolves correctly, run the dig command:dig RECORD_TYPE DESIRED_SUBDOMAIN_RECORDNote: Replace RECORD_TYPE and DESIRED_SUBDOMAIN_RECORD with your relevant details.2.    In the output, verify that you have a record type of your choice under your subdomain’s hosted zone. In the following example output, there's an A record for www.example.com under the subdomain:$ dig www.example.com;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48170;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0;; QUESTION SECTION:;www.example.com.                IN         A;; ANSWER SECTION:www.example.com.     60          IN         A          127.0.0.13.    If the lookup is successful against other DNS servers, then your local resolver might have caching issues. To bypass your local resolver, run the dig @ command with another resolver and your domain name. For example, the following lookup uses Google’s public resolver:dig @8.8.8.8 www.example.comPerform the lookup directly against one of the authoritative AWS name servers for the apex domain’s hosted zone:dig @ns-***.awsdns-**.com www.example.com4.    If the DNS lookup fails, then use the dig +trace command:dig +trace www.example.comThen, review the output to identify where the lookup fails along the DNS chain. See the following example output for a dig +trace command:$dig +trace www.example.com; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.62.rc1.56.amzn1 <<>> +trace www.example.com;; global options: +cmd.                                   518400 IN         NS        G.ROOT-SERVERS.NET.......                                   518400 IN         NS        F.ROOT-SERVERS.NET.;; Received 228 bytes from 169.xxx.xxx.xxx#53(169.xxx.xxx.xxx) in 21 mscom.                            172800 IN         NS        c.gtld-servers.net......com.                            172800 IN         NS        i.gtld-servers.net.;; Received 498 bytes from 199.xxx.xxx.xxx #53(199.xxx.xxx.xxx) in 198 ms.example.com.   172800          IN         NS        ns-xxx.awsdns-xx.com..example.com.   172800          IN         NS        ns-xxx.awsdns-xx.net..example.com.   172800          IN         NS        ns-xxx.awsdns-xx.co.uk..example.com.   172800          IN         NS        ns-xxx.awsdns-xx.org.;; Received 207 bytes from 192.xxx.xxx.xxx #53(192.xxx.xxx.xxx) in 498 mswww.example.com.     172800 IN         NS        ns-xxx.awsdns-xx.com.www.example.com.     172800 IN         NS        ns-xxx.awsdns-xx.net.www.example.com.     172800 IN         NS        ns-xxx.awsdns-xx.co.uk.www.example.com.     172800 IN         NS        ns-xxx.awsdns-xx.org.;; Received 175 bytes from 205.xxx.xxx.xxx #53(205.xxx.xxx.xxx) in 345 mswww.example.com.     900      IN         SOA      ns-xxx.awsdns-xx.com. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400$ dig www.example.com.com;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22072;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; QUESTION SECTION:www.example.com.com.          IN         A;; AUTHORITY SECTION:www.example.com.com. 60       IN         SOA      ns-xxx.awsdns-xx.com. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 864005.    Depending on the information in your output, follow the relevant troubleshoot steps.dig returns a NOERROR status with no ANSWER section, and the dig +trace output only includes the apex domain’s name serversThe NS record for your delegated subdomain is missing from the hosted zoned of your apex domain. Also, the record type for the subdomain under the root domain is wrong. For example, an MX record is listed instead of an A record.To fix this issue, create an NS record under your apex domain’s hosted zone for your subdomain with the correct name servers. Also, remove the non-NS record for the subdomain under the apex domain’s hosted zone. Then, place the non-NS record under the subdomain’s hosted zone.dig returns NXDOMAIN status and dig +trace output only includes apex domain’s name serversThe NS record for your delegated subdomain is missing under your apex domain’s hosted zone.To fix this issue, create an NS record under your apex domain’s hosted zone with the correct name servers.dig +trace returns your name servers for the delegated subdomain, but dig returns NOERROR status with no ANSWER sectionThe hosted zone contains a record that's the wrong type for your delegated subdomain. For example, a TXT record exists instead of an A record for your subdomain under the subdomain’s hosted zone.To fix this issue, create a new A record for the delegated subdomain under the subdomain’s hosted zone.dig +trace returns name servers for the delegated subdomain, but dig returns the SERVFAIL statusThe Route 53 name servers for your delegated subdomain under your apex domain’s hosted zone are incorrect in the NS record. Confirm this problem with a DNS lookup against one of the delegated subdomain’s name servers using the dig @ command. An incorrect name server returns a REFUSED status.To fix this issue, modify the NS record with the correct name servers for your subdomain’s hosted zone.An apex domain that uses a third-party DNS service and a subdomain that uses Route 531.    Check if the name servers for the subdomain are properly configured in the parent zone.2.    If the name servers aren't properly configured, then look up the NS records for your subdomain’s hosted zone. Then, add those records to the apex domain’s hosted zone or zone file with the third-party DNS provider.3.    To verify that the subdomain resolves correctly, use the dig @ command with one of the subdomain's hosted zone name servers:dig @ns-***.awsdns-**.com www.example.comNote: If the DNS resolution fails, then follow the methods in step 4 of An apex domain and a subdomain that both use Route 53.An apex domain that uses Route 53 and a subdomain that is delegated to a third party1.    Check if the name servers for the subdomain are properly configured in the parent zone.2.    If the name servers aren't properly configured, then add NS records under the hosted zone for your apex domain in Route 53.3.    To verify that the subdomain resolves correctly, use the dig @ command with your third-party DNS service's authoritative name server:dig @THIRD_PARTY_NAME_SERVER www.example.comNote: If the DNS resolution fails, then follow the methods in step 4 of An apex domain and a subdomain that both use Route 53.Related informationHow do I create a subdomain for my domain that's hosted in Route 53?Follow"


Question: Why can't I copy an object between two Amazon S3 buckets?

"I'm trying to copy an object from one Amazon Simple Storage Service (Amazon S3) bucket to another, but it's not working. How can I troubleshoot this?"

Answer: "I'm trying to copy an object from one Amazon Simple Storage Service (Amazon S3) bucket to another, but it's not working. How can I troubleshoot this?Short descriptionTo troubleshoot issues with copying an object between buckets, check the following:Bucket policies and AWS Identity and Access Management (IAM) policiesObject ownershipAWS Key Management Service (AWS KMS) encryptionAmazon Simple Storage Service Glacier (Amazon S3 Glacier) storage classRequester Pays enabled on the bucketAWS Organizations service control policyCross-Region request issues with Amazon Virtual Private Cloud (VPC) endpoints for Amazon S3ResolutionBucket policies and IAM policiesTo copy an object between buckets, you must make sure that the correct permissions are configured. To copy an object between buckets in the same AWS account, you can set permissions using IAM policies. To copy an object between buckets in different accounts, you must set permissions on both the relevant IAM policies and bucket policies.Note: For instructions on how to modify a bucket policy, see How do I add an S3 bucket policy? For instructions on how to modify the permissions for an IAM user, see Changing permissions for an IAM user. For instructions on how to modify the permissions for an IAM role, see Modifying a role.Confirm the following required permissions:At minimum, your IAM identity (user or role) must have permissions to the s3:ListBucket and s3:GetObject actions on the source bucket. If the buckets are in the same account, then set these permissions using your IAM identity's policies or the S3 bucket policy. If the buckets are in different accounts, then set these permissions using both the bucket policy and your IAM identity's policies.At minimum, your IAM identity must have permissions to the s3:ListBucket and s3:PutObject actions on the destination bucket. If the buckets are in the same account, then set these permissions using your IAM identity's policies or the S3 bucket policy. If the buckets are in different accounts, then set these permissions using both the bucket policy and your IAM identity's policies.Review the relevant bucket policies and IAM policies to confirm that there are no explicit deny statements conflicting with the permissions that you need. An explicit deny statement overrides an allow statement.For specific operations, confirm that your IAM identity has permissions to all the necessary actions within the operation. For example, to run the command aws s3 cp, you need permission to s3:GetObject and s3:PutObject. To run the command aws s3 cp with the --recursive option, you need permission to s3:GetObject, s3:PutObject, and s3:ListBucket. To run the command aws s3 sync, then you need permission to s3:GetObject, s3:PutObject, and s3:ListBucket.Note: If you're using the AssumeRole API operation to access Amazon S3, you must also verify that the trust relationship is configured correctly.For version-specific operations, confirm that your IAM identity has permissions to version-specific actions. For example, to copy a specific version of an object, you need the permission for s3:GetObjectVersion in addition to s3:GetObject.If you're copying objects that have object tags, then your IAM identity must have s3:GetObjectTagging and s3:PutObjectTagging permissions. You must have s3:GetObjectTagging permission for the source object and s3:PutObjectTagging permission for objects in the destination bucket.Review the relevant bucket policies and IAM policies to be sure that the Resource element has the correct path. For bucket-level permissions, the Resource element must point to a bucket. For object-level permissions, the Resource element must point to an object or objects.For example, a policy statement for a bucket-level action such as s3:ListBucket must specify a bucket in the Resource element, like this:"Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET"A policy statement for object-level actions like s3:GetObject or s3:PutObject must specify an object or objects in the Resource element, similar to the following:"Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"Object ownershipIf the bucket policies have the correct permissions and you're still having problems copying an object between buckets, check which account owns the object. The bucket policy applies only to objects owned by the bucket owner. An object that's owned by a different account might have conflicting permissions on its access control list (ACL).Note: The object ownership and ACL issue typically occurs when you copy AWS service logs across accounts. Examples of service logs include AWS CloudTrail logs and Elastic Load Balancing access logs.Follow these steps to find the account that owns an object:1.    Open the Amazon S3 console.2.    Navigate to the object that you can't copy between buckets.3.    Choose the object's Permissions tab.4.    Review the values under Access for object owner and Access for other AWS accounts:If the object is owned by your account, then the Canonical ID under Access for object owner contains (Your AWS account).If the object is owned by another account and you can access the object, then the following is true:The Canonical ID under Access for object owner contains (External account).The Canonical ID under Access for other AWS accounts contains (Your AWS account).If the object is owned by another account and you can't access the object, then the following is true:Canonical ID fields for both Access for object owner and Access for other AWS accounts are empty.If the object that you can't copy between buckets is owned by another account, then the object owner can do one of the following:The object owner can grant the bucket owner full control of the object. After the bucket owner owns the object, the bucket policy applies to the object.The object owner can keep ownership of the object, but they must change the ACL to the settings that you need for your use case.AWS KMS encryptionIf the object is encrypted using an AWS KMS key, then confirm that your IAM identity has the correct permissions to the key. If your IAM identity and AWS KMS key belong to the same account, then confirm that your key policy grants the required AWS KMS permissions.If your IAM identity and AWS KMS key belong to different accounts, then confirm that both the key and IAM policies grant the required permissions.For example, if you copy objects between two buckets (and each bucket has its own KMS key), then the IAM identity must specify the following:kms:Decrypt permissions, referencing the first KMS keykms:GenerateDataKey and kms:Decrypt permissions, referencing the second KMS keyFor more information, see Using key policies in AWS KMS and Actions, resources, and condition keys for AWS Key Management Service.Amazon S3 Glacier storage classYou can't copy an object from the Amazon S3 Glacier storage class. You must first restore the object from Amazon S3 Glacier before you can copy the object. For instructions, see How do I restore an S3 object that has been archived?Requester Pays enabled on bucketIf the source or destination bucket has Requester Pays enabled, and you're accessing the bucket from another account, check your request. Make sure that your request includes the correct Requester Pays parameter:For AWS Command Line Interface (AWS CLI) commands, include the --request-payer option.Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.For GET, HEAD, and POST requests, include x-amz-request-payer : requester.For signed URLs, include x-amz-request-payer=requester.AWS Organizations service control policyIf you're using AWS Organizations, check the service control policies to be sure that access to Amazon S3 is allowed.For example, the following policy results in a 403 Forbidden error when you try to access Amazon S3 because it explicitly denies access:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Deny",            "Action": "S3:*",            "Resource": "*"        }    ]}For more information on the features of AWS Organizations, see Enabling all features in your organization.Cross-Region request issues with VPC endpoints for Amazon S3VPC endpoints for Amazon S3 currently don't support cross-Region requests. For example, suppose you have an Amazon Elastic Compute Cloud (Amazon EC2) instance in Region A with a VPC endpoint configured in its associated route table. Additionally, the Amazon EC2 instance can't copy an object from Region B to a bucket in Region A. Instead, you receive an error message similar to the following:"An error occurred (AccessDenied) when calling the CopyObject operation: VPC endpoints do not support cross-region requests"To troubleshoot this cross-Region request issue, you can try the following:Remove the VPC endpoint from the route table. If you remove the VPC endpoint, the instance must be able to connect to the internet instead.Run the copy command from another instance that's not using the VPC endpoint. Or, run the copy command from an instance that's in neither Region A nor Region B.If you must use the VPC endpoint, send a GET request to copy the object from the source bucket to the EC2 instance. Then, send a PUT request to copy the object from the EC2 instance to the destination bucket.Related informationCopying objectsHow do I troubleshoot 403 Access Denied errors from Amazon S3?Follow"


Question: How do I utilize CDK escape hatches to retrieve lower-level construct objects from L3 and L2 constructs?

I want to use AWS Cloud Development Kit (AWS CDK) escape hatches to retrieve child objects of L2 and L3 constructs.

Answer: "I want to use AWS Cloud Development Kit (AWS CDK) escape hatches to retrieve child objects of L2 and L3 constructs.Short descriptionThere are three AWS CDK abstraction layers:L1 constructs have 1:1 relationships that map to the related AWS CloudFormation resource types. This is the most fundamental construct layer for AWS CDK.L2 constructs can wrap a number of L1 constructs and its default child object is the relevant resource type's L1 construct. Other L1 construct child objects are synthesized into AWS CloudFormation templates based on the L2 child object's specified properties.L3 constructs are the highest level of AWS CDK abstraction layers and can wrap a number of L2 and L1 constructs.For more information, see abstractions and escape hatches.ResolutionUse AWS CDK escape hatches to retrieve child objects from an Amazon Elastic Kubernetes Service (Amazon EKS) cluster with an L3 construct.Note: These steps use the Python programming language. The steps are similar for any other programming languages. Make sure to adjust code syntax for the programming language you're using.An example Amazon EKS cluster with a L3 construct in Python:vpc = ec2.Vpc(self, "Vpc",            ip_addresses=ec2.IpAddresses.cidr("192.168.0.0/25")        )        eks_object = eks.Cluster(self, "HelloEKS",            version=eks.KubernetesVersion.V1_25,            vpc=vpc,            vpc_subnets=[ec2.SubnetSelection(subnet_type=ec2.SubnetType.PRIVATE_WITH_EGRESS)]        )1.    Retrieve all the child objects of a L3 construct in an Amazon EKS cluster by using the node.find_all() attribute:for child in eks_object.node.find_all():            print(child.node.id)After using the preceding command, all child IDs of the L3 construct print.Example printout:HelloEKS...NodegroupDefaultCapacityNodeGroupRole...2.    After printing the child IDs of the L3 construct, retrieve the desired child ID by using the node.find_child() attribute:Important: Make sure to check all AWS Command Line Interface (AWS CLI) commands and replace all instances of example strings with your values. For example, replace example_child_id with your target child ID.l2_nodeGroup = eks_object.node.find_child(example_child_id)After using this command, the L2 construct of the desired child ID prints.Example printout:<aws_cdk.aws_eks.Nodegroup object at 0x7ffa9c7b2910>Note: You can use variables l2_nodeGroup to invoke the Nodegroup properties, attributes, and methods to modify the associated resources.3.    Retrieve all the child objects of the L2 construct by using the node.find_all() attribute:for child in l2_nodeGroup.node.find_all():         print(child.node.id)After using the preceding command, all child IDs of the L2 construct will print.Example printout:NodegroupDefaultCapacityNodeGroupRoleImportNodeGroupRole4.    After the child IDs of the L2 construct print, retrieve the desired child ID by using the node.find_child() attribute:l2_nodeGroup_role = l2_nodeGroup.node.find_child(example_child_id)     print(l2_nodeGroup_role)After using the preceding command, an object at the L2 layer will return at the aws_iam.Role level.5.    When you are at the aws_iam.Role level, use the following node.default_child attribute to reach the L1 CfnRole construct object:l1_nodeGroup_role = l2_nodeGroup_role.node.default_child        print(l1_nodeGroup_role)After using the preceding command, the default child at the L1 layer will return.Note: When you use node.find_all() or node.default_child to retrieve child objects, you can use that construct's functionalities for increased controls over a CloudFormation template.If you still can't retrieve child objects, contact AWS Support or create a new issue at the GitHub website for AWS CDK issues.Related informationHow do I customize a resource property value when there is a gap between CDK higher level constructs and a CloudFormation resource?Follow"


Question: "Do Classic Load Balancers, Application Load Balancers, and Network Load Balancers support SSL/TLS session resumption?"

"I want to know if Classic Load Balancers, Application Load Balancers, and Network Load Balancers support Secure Sockets Layer/Transport Layer Security (SSL/TLS) session resumption."

Answer: "I want to know if Classic Load Balancers, Application Load Balancers, and Network Load Balancers support Secure Sockets Layer/Transport Layer Security (SSL/TLS) session resumption.ResolutionAll types of load balancers support SSL/TLS session resumption. However, the connection methods that they support varies.SSL/TLS connection methodsThere are two types of TLS handshakes: full and abbreviated. The full handshake is performed only once. After the handshake, the client establishes an SSL/TLS session with the server. On subsequent connections, the abbreviated handshake is used to resume the previously negotiated session more quickly.There are two ways to establish or resume a TLS connection:SSL session IDs – This method is based on both the client and server keeping session security parameters for a period of time after a fully negotiated connection ends. A server that intends to use session resumption assigns a unique identifier for the session, called the session ID. The server then returns the session ID to the client in the ServerHello message. To resume an earlier session, the client must submit the appropriate session ID in its ClientHello message. If the server finds the corresponding session in its cache and accepts the request, then the server returns the samesession identifier. The server then continues with the abbreviated SSL handshake. Otherwise, the server issues a new session identifier and switches to a full handshake.SSL session tickets – This method doesn't require server-side storage. The server gathers all session data, encrypts it, and then returns it to the client in the form of a ticket. On subsequent connections, the client submits the ticket back to the server. Then, the server checks the ticket integrity, decrypts the contents, and uses the information in it to resume the session. If the server or client doesn't support this extension, then fall back to the session identifier mechanism built into SSL.Supported SSL/TLS connection methods for each load balancer typeClassic Load BalancersClassic Load Balancers support session ID-based SSL/TLS session resumption but don't support session ticket-based SSL session resumption. SSL session caching is supported at the node level. For example, suppose that a client connects to node B using the SSL session ID received from node A. When this happens, the SSL handshake reverts to a full handshake. After that, a new SSL session ID is generated by node B.Application Load BalancersApplication Load Balancers support both session ID and session ticket-based SSL session resumption. Both session IDs and session tickets are supported at the node level. For example, suppose that a client connects to node B using the SSL session ID or session ticket received from node A. When this happens, the SSL handshake reverts to a full handshake. After that, a new SSL session ID and session ticket are generated by node B.Network Load BalancersNetwork Load Balancers support only session tickets for session resumption. Resumption using session tickets is supported at the Regional level. Clients can resume TLS sessions with a Network Load Balancer using any of its IP addresses.Follow"


Question: How do I share manual Amazon RDS DB snapshots or Aurora DB cluster snapshots with another AWS account?

I want to share manual Amazon Relational Database Service (Amazon RDS) DB snapshots or Amazon Aurora DB cluster snapshots with another account.

Answer: "I want to share manual Amazon Relational Database Service (Amazon RDS) DB snapshots or Amazon Aurora DB cluster snapshots with another account.Short descriptionYou can share manual DB snapshots with up to 20 AWS accounts. You can start or stop sharing manual snapshots by using the Amazon RDS console, except for the following limitations:You can't share automated Amazon RDS snapshots with other AWS accounts. To share an automated snapshot, copy the snapshot to make a manual version, and then share that copy.You can't share manual snapshots of DB instances that use custom option groups with persistent or permanent options. For example, this includes Transparent Data Encryption (TDE) and time zone.You can share encrypted manual snapshots that don't use the default Amazon RDS encryption key. But you must first share the AWS Key Management Service (AWS KMS) key with the account that you want to share the snapshot with. To share the key with another account, share the AWS Identity and Access Management (IAM) policy with the primary and secondary accounts. You can't restore shared encrypted snapshots directly from the destination account. First, copy the snapshot to the destination account by using an AWS KMS key in the destination account. Then, restore the copied snapshot.To share snapshots that use the default AWS managed key for Amazon RDS (aws/rds), encrypt the snapshot by copying it with a customer managed key. Then, share the newly created snapshot.You can share snapshots across AWS Regions. First share the snapshot, and then copy the snapshot to the same Region in the destination account. Then, copy the snapshot to another Region.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Open the Amazon RDS console.In the navigation pane, choose Snapshots.Choose the DB snapshot that you want to copy.Choose Actions, and then choose Share Snapshot.Choose the DB snapshot visibility:Public allows all AWS accounts to restore a DB instance from your manual DB snapshot.Private allows only AWS accounts that you specify to restore a DB instance from your manual DB snapshot.In the AWS Account ID field, enter the ID of the AWS account that you want to permit to restore a DB instance from your manual DB snapshot. Then, choose Add.Note: You can repeat this step to share snapshots with up to 20 AWS accounts.Choose Save.To stop sharing a snapshot with an AWS Account, select the Delete check box next to the account ID from the Manage Snapshot Permissions pane.Choose Save.You can restore a DB instance or DB cluster from a shared snapshot by using the AWS CLI or Amazon RDS API. To do this, you must specify the full Amazon Resource Name (ARN) of the shared snapshot as the snapshot identifier.Related informationSharing a DB snapshotCreating a DB snapshotRestoring from a DB snapshotFollow"


Question: Why is my AWS DMS task in an error status?

"My AWS Database Migration Service (AWS DMS) task is in an error status. What does an error status mean, and how can I troubleshoot and resolve the error?"

Answer: "My AWS Database Migration Service (AWS DMS) task is in an error status. What does an error status mean, and how can I troubleshoot and resolve the error?Short descriptionAn AWS DMS task that is in an error status means that one or more of the tables in the task couldn't be migrated. A task in an Error status continues to load other tables from the selection rule, but a failed task stops with fatal errors.ResolutionTo identify the table that has an error, open the AWS DMS console.Choose Database migration tasks from the navigation pane.In the Tables errored column, the number of tables that have errors are listed.Choose the name of the task that has an error status.From the Table statistics section, check the Load state column to see which table names have errors. Or you can run describe-table-statistics.To troubleshoot error messages further, turn on Amazon CloudWatch logging. If you haven't turned on logging, stop the task, and modify the task to turn on logging. Then, restart the task.From the Logs page for your task, filter the timestamps for events that have ]E: and ]W: in them.After you resolve the errors, reload the tables, or restart the task for to the error status to be resolved.Related informationCommon errors for AWS DMSMigration strategy for relational databasesFollow"


Question: Why isn’t my EBS volume increase reflected in my OS or Disk Management?

"I increased the size of my Amazon Elastic Block Store (Amazon EBS) volume, but the change isn’t reflected in my operating system or Disk Management."

Answer: "I increased the size of my Amazon Elastic Block Store (Amazon EBS) volume, but the change isn’t reflected in my operating system or Disk Management.ResolutionModifying an EBS volume requires two steps. First, increase the size of the EBS volume using the Amazon Elastic Compute Cloud (Amazon EC2) console or the AWS Command Line Interface (AWS CLI). Then, extend the volume’s file system to use the new storage capacity. For instructions, see:Extend a Windows file system after resizing a volumeExtend a Linux file system after resizing a volumeNote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Related informationRequirements when modifying volumesMonitor the progress of volume modificationsMap disks to volumes on your Windows instanceInitialize Amazon EBS volumes on LinuxInitialize Amazon EBS volumes on WindowsFollow"


Question: How does ELB DNS and traffic flow operate with different cross-zone load balancing configurations?

"I configured my Elastic Load Balancer (ELB) for two Availability Zones, but it shows only one IP address in DNS."

Answer: "I configured my Elastic Load Balancer (ELB) for two Availability Zones, but it shows only one IP address in DNS.Short descriptionWith Application Load Balancers, cross-zone load balancing is always turned on at load balancer level. Cross-zone load balancing can't be turned off, but it can be changed at the target group level.However, with Network Load Balancers and Gateway Load Balancers, cross-zone load balancing is turned off by default.When cross-zone load balancing is turned off, an Availability Zone must have at least one healthy target in each target group. When cross-zone load balancing is turned on, there must be at least one healthy target in each target group in any Availability Zone. Each condition keeps the Availability Zone healthy, and adds the corresponding Elastic Load Balancer node IP address to the Elastic Load Balancer DNS.ResolutionWhen cross-zone load balancing is turned offThe following is an example of when cross-zone load balancing is turned off between two Availability Zones, AZ1 and AZ2.Availability Zone 1 (AZ1) has two target groups, A and B, each with its own target, A1 and B1. Target A1 is unhealthy and target B1 is healthy. Because target A1 is unhealthy, AZ1 is also unhealthy.Availability Zone 2 (AZ2) also has two target groups, A and B, each with its own target, A2 and B2. Targets A2 and B2 are both healthy. Because each target in both target groups is healthy, AZ2 is healthy.The Elastic Load Balancer includes the IP address of AZ2 in the Elastic Load Balancer's DNS because AZ2 is the only healthy Availability Zone. As a result, when you resolve the domain of the Elastic Load Balancer, the IP address of AZ2 is the only one that appears.Traffic then gets routed through the Elastic Load Balancer node in AZ2 to the healthy target in the corresponding target group. If there are multiple healthy targets in a target group, then one target is selected based on the routing algorithm of the load balancer.If both Availability Zones are unhealthy, then the Elastic Load Balancer fails open. Each Elastic Load Balancer IP address is then added to the DNS of the load balancer.When cross-zone load balancing is turned onThe following is an example of when cross-zone load balancing is turned on using the same Availability Zones, AZ1 and AZ2:In AZ1, target A1 is unhealthy and target B1 is healthy. In AZ2, both targets A2 and B2 are healthy. Because each Availability Zone has at least one healthy target, Elastic Load Balancer includes both IP addresses in the DNS for the Elastic Load Balancer hostname.Traffic then gets routed to any of the Elastic Load Balancer nodes and forwarded to the targets in the corresponding target groups. If there are multiple healthy targets in a target group, then a target is selected based on the routing algorithm of the load balancer.If target B1 in AZ1 is unhealthy and target B2 in AZ2 is also unhealthy, then both Availability Zones are unhealthy. Because neither Availability Zone is healthy, the Elastic Load Balancer fails open. Each Elastic Load Balancer IP address is then added to the DNS of the load balancer. As a result, when you resolve the domain, the IP addresses for both Availability Zones appear.Related informationApplication Load Balancers now support turning off cross zone load balancing per target groupFollow"


Question: Why is AWS Global Accelerator failing health checks with endpoints?

I want to know why the endpoints registered to my AWS Global Accelerator aren't healthy.

Answer: "I want to know why the endpoints registered to my AWS Global Accelerator aren't healthy.Short descriptionFor Standard Accelerators, AWS Global Accelerator automatically checks the health of the endpoints that are associated with your static IP addresses. It then directs the user traffic only to healthy endpoints.Global Accelerator supports four types of endpoints for Standard Accelerators: Amazon Elastic Compute Cloud (EC2) Instance, Elastic IP address, the Application Load Balancer, and the Network Load Balancer.You can specify health check options in Global Accelerator when configuring your endpoints. However, the accelerator uses this configuration only for EC2 and Elastic IP endpoint types. For the Application Load Balancer and the Network Load Balancer endpoints, Global Accelerator reuses the already-configured health checks associated with those endpoints.However, health checks might fail for the different endpoint types supported by Global Accelerator. To resolve these failures, review these solutions.ResolutionIdentify your endpoint type. Then, follow the steps in that section to review the health check status.Endpoint type: EC2 Instance or Elastic IP address1.    Log in to the Global Accelerator console.2.    Choose an accelerator for a health check from the list of accelerators.3.    Under Listeners, choose the listener that you want to review.4.    Under Endpoint groups, open the health check details.5.    Review these health check details: the path, the port, and the protocol associated with the endpoint group.6.    Locate the section labeled Endpoints. This section shows whether the endpoint passed or failed the health check. The section flags a failed health check status.7.    If the endpoint health check failed, then make sure the Firewall, Security Groups (SG), and Network Access Control List (NACL) have access to the Amazon Route 53 health checker IP addresses and the appropriate health check port.Global Accelerator requires that your router and firewall rules allow inbound traffic from the IP addresses associated with Route 53 health checkers. This lets the accelerator complete the health checks for the EC2 Instance or the Elastic IP address endpoints. The health check fails if the port or the IP addresses are blocked. The accelerator reports these endpoints as unhealthy. For more information about approved IP addresses, see IP address ranges of Amazon Route 53 servers.8.    Make sure you have a TCP, HTTP, or HTTPS server at your endpoint for health checks, irrespective of UDP or TCP listeners. Then, follow these steps:Check whether the application is listening on the required port and IP address (for health check ports and application ports) using the netstat command. If the application isn't listening on the IP address and port, then configure your application and make sure that it's working locally on the instance.On Windows: netstat -ano | findstr endpoint_IP_address:portOn Linux: netstat -anp | grep endpoint_IP_address:portNote: Replace endpoint_IP_address:port with your endpoint's IP address and port number.Use these tools to check the connectivity to the endpoints on the health check ports. The tests must be successful without any errors on all endpoints and target application instances. Make sure that the application can accept the configured health check requests according to these settings:For TCP health checks: telnet endpoint_IP_address health_check_portFor HTTP health checks: curl -vko /dev/null http://endpoint_IP_address:portFor HTTPS health checks: curl -vko /dev/null https://endpoint_IP_address:portNote: Replace endpoint_IP_address with your endpoint's IP address and health_check_port with the associated port number.Check if iptables (for Linux) and firewall (for Windows) are dropping the application traffic.Endpoint type: Application Load Balancer or Network Load BalancerIf the endpoint type is an Application Load Balancer or a Network Load Balancer, then Global Accelerator uses the load balancer's health check information to determine the health of the endpoints. There are a few unique considerations for how Global Accelerator calculates health for these endpoints:Application Load Balancer considerations1.    All target groups in your Application Load Balancer must be healthy for Global Accelerator to consider the load balancer healthy. For more information on how to configure the target group in the Application Load Balancer, see Target group health.2.    If ALL target groups are empty, then Global Accelerator considers the Application Load Balancer unhealthy.Network Load Balancer considerations1.    All target groups in your Network Load Balancer must be healthy for Global Accelerator to consider the load balancer healthy. For more information on how to configure the target group in the Network Load Balancer, see Target group health. 2.    If ANY single target group is empty, then Global Accelerator considers the Network Load Balancer unhealthy.Refer to the following articles if the Elastic Load Balancing (ELB) target groups are reporting unhealthy results:See How do I troubleshoot and fix failing health checks for Application Load Balancers? to make sure that the Application Load Balancer's targets are healthy.See Troubleshoot Your Network Load Balancer to make sure that the Network Load Balancer's targets are healthy.Related informationChanging health check optionsFollow"


Question: How do I troubleshoot helper scripts that won't bootstrap in a CloudFormation stack with Windows instances?

My helper scripts won't bootstrap in an AWS CloudFormation stack with Microsoft Windows instances. How do I resolve this issue?

Answer: "My helper scripts won't bootstrap in an AWS CloudFormation stack with Microsoft Windows instances. How do I resolve this issue?Short descriptionIn a Windows instance, UserData that runs as a child process of EC2ConfigService invokes cfn-init.exe. Certain steps that cfn-init.exe performs might require a system reboot.For example, you must reboot the system if you rename a computer, or join a computer to a domain. After the system reboots, cfn-init continues to run the rest of the configurations that remain in AWS::CloudFormation::Init with the help of C:\cfn\cfn-init\resume_db.json.For helper scripts that don't run after rebooting the Windows instance, complete the steps in the Troubleshoot bootstrapping issues section.If you receive the following error, then you might have issues with cfn-signal:"Received 0 conditions when expecting X or Failed to receive X resource signal(s) within the specified duration"To resolve the preceding error, complete the steps in the Troubleshoot cfn-signal issues section.For both issues, complete the steps in the Follow the best practices for using a Windows operating system with CloudFormation section.ResolutionTroubleshoot bootstrapping issuesIf your script doesn't run after reboot, then complete the following steps:1.    In the commands section of your cfn-init configuration set, confirm that waitAfterCompletion is set to forever. For example:"commands": { "0-restart": { "command": "powershell.exe -Command Restart-Computer", "waitAfterCompletion": "forever" } }Note: The forever value directs cfn-init to exit and resume only after the reboot is complete. For more information, see AWS::CloudFormation::Init.2.    Check the following logs for errors:Amazon Elastic Compute Cloud (Amazon EC2) configuration log at C:\Program Files\Amazon\Ec2ConfigService\Logs\Ec2ConfigLog.txt (versions before Windows 2016)Amazon EC2 configuration log at *C:\ProgramData\Amazon\EC2-Windows\Launch\Log* (Windows 2016 and later)cfn-init log at C:\cfn\log\cfn-init.logWindows Event logs at C:\Windows\System32\winevt\logsTroubleshoot cfn-signal issues1.    Confirm that the cfn-signal is configured correctly.Important: Use -e $lastexitcode in PowerShell scripts, and use -e %ERRORLEVEL% for Windows cmd.exe scripts.For PowerShell scripts in UserData, use the following tags:<powershell></powershell>Example PowerShell script:UserData:        Fn::Base64:          Fn::Sub : |            <powershell>            $LASTEXITCODE=0            echo Current date and time >> C:\Temp\test.log            echo %DATE% %TIME% >> C:\Temp\test.log            cfn-init.exe -s ${AWS::StackId} -r SInstance --region ${AWS::Region}            New-Item -Path "C:\" -Name userdata -ItemType directory            cfn-signal.exe -e $LASTEXITCODE --stack ${AWS::StackId} --resource WindowsInstance --region ${AWS::Region}            </powershell>For cmd.exe scripts in UserData, use the following tags:<script></script>Example cmd.exe script:UserData:  Fn::Base64: !Sub |    <script>    cfn-init.exe -v -s ${AWS::StackId} -r WindowsInstance --configsets ascending --region ${AWS::Region}    cfn-signal.exe -e %ERRORLEVEL% --stack ${AWS::StackId} --resource WindowsInstance --region ${AWS::Region}    </script>2.    Increase the Timeout of the WaitCondition to 1800/3600 seconds based on the example of bootstrapping a Windows stack.Note: Step 2 is necessary because Windows instances usually take longer than Linux instances to complete their initial boot process.3.    If you're using a custom Amazon Machine Image (AMI), then you must use Sysprep to create the AMI before you start. If you don't use Sysprep, then you might experience metadata issues and get the following error in the user data log for metadata:Failed to get metadata: The result from http://169.254.169.254/latest/user-data was emptyUnable to execute userdata: Userdata was not providedFollow the best practices for using a Windows operating system with CloudFormationInclude $ErrorActionPreference = "Stop" at the top of your PowerShell scripts to catch exceptions.When you refer to a Windows path in your CloudFormation template, you must add a forward slash (/) at the beginning of the path. For example:"commands" : {  "1-extract" : {    "command" : "C:\\SharePoint\\SharePointFoundation2010.exe /extract:C:\\SharePoint\\SPF2010 /quiet /log:C:\\SharePoint\\SharePointFoundation2010-extract.log"  }For Windows stacks, you must base64-encode the WaitCondition handle URL again. For example:cfn-signal.exe -e %ERRORLEVEL% ", { "Fn::Base64" : { "Ref" : "SharePointFoundationWaitHandle" }}, "\n"Follow"


Question: How do I change the WorkSpace that I'm connecting to through the Amazon WorkSpaces client?

I want to change the WorkSpace that I connect to through the Amazon WorkSpaces client application. How can I do this?

Answer: "I want to change the WorkSpace that I connect to through the Amazon WorkSpaces client application. How can I do this?ResolutionTo change the WorkSpace that you connect to, follow these steps:Retrieve the user name and registration code for the new WorkSpace from your invitation email. If you haven't already completed the registration process by opening the link in your invitation email, then do so now. If you can't find your invitation email, ask your administrator to resend it.Note: All WorkSpaces that are launched from the same directory have the same registration code.Verify that you are using the latest Amazon WorkSpaces client. Then, open the client application.On the client login window, choose Change Registration Code. For more information, see WorkSpaces Windows client application or WorkSpaces macOS client application.The dropdown list displays all saved registration codes and the associated Region. You can select a different registration code from the list, or clear the text box and add a new registration code. Then, choose Continue.Sign in to the WorkSpace using the user name and password associated with the WorkSpace.To rename or remove a saved registration code, choose Settings, then choose Manage Login Information.Related informationGet started with WorkSpaces Quick SetupFollow"


Question: Why was I charged by Amazon Web Services when I don't have an AWS account?

"I received a bill for Amazon Web Services, but I don't have an AWS account. Why was I billed?"

Answer: "I received a bill for Amazon Web Services, but I don't have an AWS account. Why was I billed?ResolutionFirst, make sure that the charges were from Amazon Web Services.If your payment method is billed, then that payment method is associated with an AWS account with running resources. Here are some of the most common reasons why you might be billed for an account that you don't remember creating:Someone is using your payment method without your permission. If you're the individual owner of the payment method and don't have an AWS account, then someone might be using your payment information without permission. Speak with your credit card issuer to dispute the charges.Note: AWS Support can't resolve charges if you're not affiliated with an AWS account. For a stolen credit card or similar situation, contact your financial institution.A test account's Free Tier promotion expired. The AWS Free Tier covers new accounts for a year, allowing new customers to test many AWS services for free. Any resources left running after the Free Tier expires are billed at the On-Demand rates.You or another authorized user of the card might have forgotten that the account has running resources on it. For more information, see I unintentionally incurred charges while using the Free Tier. How do I make sure that I'm not billed again?Someone in your organization created the account. Check with other authorized users of the card to see if they've opened an AWS account using the card. This might be a spouse or another family member. If this is a corporate credit card, check with technical department heads within your organization. Common users of AWS include application or service developers, website designers, or systems administrators.A contractor or third party is using AWS to provide you a service. A contractor or third party that you hired might be building your website or app using AWS. Contact the third party for more details about these charges.If you need to contact AWS Support, use the I'm an AWS customer and I'm looking for billing or account support form. You can use this form to submit a case if you don't have an AWS account, or if you're unable to sign in. If you can't resolve your concern with the information provided, respond to the email you receive from AWS Support.Important: AWS Support can't discuss any account-related information if account security isn't verified.Related informationI can't sign in because my credentials don't workAvoiding unexpected chargesFollow"


Question: Why did the DeliveryFrequency of my AWS Config configuration snapshot change?

"The deliveryFrequency of my AWS Config configuration snapshot changed, and there are PutDeliveryChannel events logged in AWS CloudTrail. However, I didn't change the AWS Config delivery channel. What caused this change?"

Answer: "The deliveryFrequency of my AWS Config configuration snapshot changed, and there are PutDeliveryChannel events logged in AWS CloudTrail. However, I didn't change the AWS Config delivery channel. What caused this change?ResolutionThe frequency that AWS Config delivers configuration snapshots is controlled by:The deliveryFrequency parameter in the PutDeliveryChannel API using the AWS Command Line Interface (AWS CLI).The MaximumExecutionFrequency parameter in the PutConfigRule API set with creating or changing AWS Config periodic rules.Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.You might see PutDeliveryChannel events logged in CloudTrail if:The deliveryFrequency parameter isn't configured for the delivery channel.The value of the MaximumExecutionFrequency parameter is less than the deliveryFrequency value set on the delivery channel.If no value for the deliveryFrequency is set, then AWS Config calls the PutDeliveryChannel API to update the MaximumExecutionFrequency value for the periodic rule.You can view the deliveryFrequency using the DescribeDeliveryChannnels command similar to the following:$ aws configservice describe-delivery-channels --region your-region{    "DeliveryChannels": [        {            "configSnapshotDeliveryProperties": {                "deliveryFrequency": "Twelve_Hours"            },             "name": "default",             "s3BucketName": "config-bucket-123456789012-your-region"        }    ]}You can view the PutDeliveryChannel API using AWS CloudTrail similar to the following:"eventSource": "config.amazonaws.com",    "eventName": "PutDeliveryChannel",    "awsRegion": "your-region",    "sourceIPAddress": "192.0.2.0",    "userAgent": "console.amazonaws.com",    "requestParameters": {        "deliveryChannel": {            "name": "default",            "configSnapshotDeliveryProperties": {                "deliveryFrequency": "Twelve_Hours"            },            "s3BucketName": "config-bucket-123456789012-your-region"        }    },Related informationHow can I recreate an AWS Config delivery channel?Follow"


Question: How can I concatenate Parquet files in Amazon EMR?

"I'm using S3DistCp (s3-dist-cp) to concatenate files in Apache Parquet format with the --groupBy and --targetSize options. The s3-dist-cp job completes without errors, but the generated Parquet files are broken. When I try to read the Parquet files in applications, I get an error message similar to the following:"Expected n values in column chunk at /path/to/concatenated/parquet/file offset m but got x values instead over y pages ending at file offset z""

Answer: "I'm using S3DistCp (s3-dist-cp) to concatenate files in Apache Parquet format with the --groupBy and --targetSize options. The s3-dist-cp job completes without errors, but the generated Parquet files are broken. When I try to read the Parquet files in applications, I get an error message similar to the following:"Expected n values in column chunk at /path/to/concatenated/parquet/file offset m but got x values instead over y pages ending at file offset z"Short descriptionS3DistCp doesn't support concatenation for Parquet files. Use PySpark instead.ResolutionYou can't specify the target file size in PySpark, but you can specify the number of partitions. Spark saves each partition to a separate output file. To estimate the number of partitions that you need, divide the size of the dataset by the target individual file size.1.    Create an Amazon EMR cluster with Apache Spark installed.2.    Specify how many executors you need. This depends on cluster capacity and dataset size. For more information, see Best practices for successfully managing memory for Apache Spark applications on Amazon EMR.$  pyspark --num-executors number_of_executors3.    Load the source Parquet files into a Spark DataFrame. This can be an Amazon Simple Storage Service (Amazon S3) path or an HDFS path. For example:df=sqlContext.read.parquet("s3://awsdoc-example-bucket/parquet-data/")HDFS:df=sqlContext.read.parquet("hdfs:///tmp/parquet-data/")4.    Repartition the DataFrame. In the following example, n is the number of partitions.df_output=df.coalesce(n)5.    Save the DataFrame to the destination. This can be an Amazon S3 path or an HDFS path. For example:df_output.write.parquet("URI:s3://awsdoc-example-bucket1/destination/")HDFS:df=sqlContext.write.parquet("hdfs:///tmp/destination/")6.    Verify how many files are now in the destination directory:hadoop fs -ls "URI:s3://awsdoc-example-bucket1/destination/ | wc -l"The total number of files should be the value of n from step 4, plus one. The Parquet output committer writes the extra file, called _SUCCESS.Follow"


Question: How can I check the amount of backup storage being used by my Aurora PostgreSQL-Compatible DB instances?

I want to check the amount of storage used by the backup of my Amazon Aurora PostgreSQL-Compatible Edition DB instances.

Answer: "I want to check the amount of storage used by the backup of my Amazon Aurora PostgreSQL-Compatible Edition DB instances.ResolutionNote: If you receive errors when running AWS Command Line Interface commands, make sure that you’re using the most recent version of the AWS CLI.To check the storage usage of your Aurora DB instances backup, you can use these Amazon CloudWatch metrics:BackupRetentionPeriodStorageUsed - the amount of backup storage used for continuous backups at a given time.SnapshotStorageUsed - the amount of backup storage used for storing manual snapshots beyond the backup retention period.TotalBackupStorageBilled - the sum of BackupRetentionPeriodStorageUsed and SnapshotStorageUsed minus the mount of free backup storage.For more information on using these metrics, see Understanding Amazon Aurora backup storage usage.For more information on accessing these metrics using Amazon CloudWatch, see Monitoring Amazon Aurora metrics with Amazon CloudWatch.You can also using the AWS CLI to view Amazon CloudWatch metrics about your Aurora instances. For instructions on how to view these metrics in the AWS CLI, see Viewing DB cluster metrics in the CloudWatch console and AWS CLI.Note: Aurora databases don't have the FreeStorageSpace metric.Preformatted code block (delete if article contains no code)Related informationOther considerations for Aurora backup storage and pricingFollow"


Question: How do I troubleshoot issues with an Amazon Cognito user's email_verified attribute?

I want to resolve any issues with the email_verified attribute for Amazon Cognito users.

Answer: "I want to resolve any issues with the email_verified attribute for Amazon Cognito users.Short descriptionAn Amazon Cognito user pool has a set of standard attributes that are used to identify individual users. The email_verified attribute that indicates whether a user's email address has been verified can change in the following situations:A user updates their email address. When a user updates their email address, Amazon Cognito changes the email_verified attribute to unverified.An email address is configured as an alias. Then, a user with a duplicate email address is created. When an email address is set as an alias, only one user can hold the email address value as the email_verified attribute. If the newer user's account confirmation succeeds, then the email address alias is transferred to the newer user. The former user's email address is then unverified. For more information, see User pool attributes and review the Customizing sign-in attributes section.A federated user or a user linked to a federated user signs in with an email mapping. When a federated user signs in, a mapping must be present for each user pool attribute that your user pool requires. If an email attribute is mapped, the email_verified attribute changes to unverified by default.ResolutionTo resolve issues with the email_verified attribute, follow the steps that apply to your situation.Important: In the following example AWS Command Line Interface (AWS CLI) commands, replace all instances of example strings with your values. (For example, replace "example_access_token" with your access token value.)Verification after an email address updateTo verify the email address after a user update:1.    For Amazon Cognito to send the verification code to an updated email address, configure the email verification setting for the user pool.2.    If necessary, update the email address by calling the UpdateUserAttributes API or the AdminUpdateUserAttributes API.An example update-user-attributes command:aws cognito-idp update-user-attributes --access-token "example_access_token" --user-attributes Name="email",Value="example_new_email"An example admin-update-user-attributes command:aws cognito-idp admin-update-user-attributes --user-pool-id "example_user_pool_id" --username "example_username" --user-attributes Name="email",Value="example_new_email"Important: The AdminUpdateUserAttributes API can also be used to automatically verify the email by setting the email_verified attribute to True. If the email address is automatically verified with the AdminUpdateUserAttributes API, the next step isn't necessary. The next step is necessary when using the UpdateUserAttributes API.3.    Check your new email inbox for the verification code.4.    Call the VerifyUserAttribute API. Specify the parameters for AccessToken and AttributeName as "email" and enter the verification code from the previous step.An example verify-user-attribute command:aws cognito-idp verify-user-attribute --access-token "example_access_token" --attribute-name "email" --code "example_verification_code"To verify the email address after the initial code expires:1.    Sign in to your application with your user name to retrieve your access token.2.    Call the GetUserAttributeVerificationCode API. Set the AttributeName parameter as "email".An example get-user-attribute-verification-code command:aws cognito-idp get-user-attribute-verification-code --access-token "example_access_token" --attribute-name "email"3.    Call the VerifyUserAttribute API. Specify the parameters for AccessToken and AttributeName as "email". Enter the verification code from the previous step.Confirm a new user with a duplicate email addressTo allow the confirmation of a new user with a duplicate email address:1.    If necessary, call the SignUp API to sign up a user with a configured email address.An example sign-up command:aws cognito-idp sign-up --client-id "example_client_id" --username "example_username" --password "example_password" --user-attributes Name="email",Value="example_user_email"2.    Call the ConfirmSignUp API with the ForceAliasCreation parameter set to True.An example confirm-sign-up command:aws cognito-idp confirm-sign-up --client-id "example_client_id" --username "example_username" --confirmation-code "example_confirmation_code" --force-alias-creationTo deny the confirmation of a new user with a duplicate email address after sign up:1.    Call the ConfirmSignUp API with the ForceAliasCreation parameter set to False.Note: ForceAliasCreation is False by default. Therefore, it's not required to be passed as a parameter in the request.An example deny-sign-up command:aws cognito-idp confirm-sign-up --client-id "example_client_id" --username "example_username" --confirmation-code "example_confirmation_code" --no-force-alias-creation2.    By setting the ForceAliasCreation parameter to False, the API returns the following error:An error occurred (AliasExistsException) when calling the ConfirmSignUp operation: An account with the email already exists.Create a new user with a duplicate email address as an administratorTo create a new user with a duplicate email address as an administrator:1.    Call the AdminCreateUser API with a configured email address, with the email_verified attribute set to True and the ForceAliasCreation parameter set to True.An example admin-create-user command:aws cognito-idp admin-create-user --user-pool-id "example_user_pool_id" --username "example_username" --user-attributes Name="email",Value="example_user_email" Name="email_verified",Value="True" --force-alias-creationMap the email_verified attribute to a third-party identity provider (IdP)To keep the email_verified attribute verified after federation:1.    From the Amazon Cognito console, map the IdP attribute for verification status to the email_verified attribute.Note: Most OpenID Connect (OIDC) providers include the email_verified attribute.Related informationVerifying updates to email addresses and phone numbersFollow"


Question: How can I set up a Direct Connect public VIF?

I want to set up a AWS Direct Connect public VIF.

Answer: "I want to set up a AWS Direct Connect public VIF.Short descriptionA public VIF uses a public IP address to access all AWS public services such as Amazon Elastic Compute Cloud (Amazon EC2).Note: Public VIFs can't be used to access the Internet.ResolutionFollow these instructions to set up a AWS Direct Connect public VIF based on your scenario.IPv4 address allocation and addressing and Border Gateway Protocol (BGP) Autonomous System Number (ASN)For IPv4 addresses, use one of the following options:Use a public IPv4 CIDR block that you own.If you don't own a public IPv4 block, then check with your partner in the AWS Direct Connect Partner Program or ISP to see if they can provide you with a public IPv4 CIDR. Be sure to include the LOA-CFA authorization form stating that they authorize you to use those public IP prefixes.You can also contact AWS Support to request a public IPv4 CIDR. Be sure to provide your use case. Note that AWS can't guarantee approval for all public IPv4 CIDR requests. For more information, see Prerequisites for virtual interfaces.A public or private BGP ASN for your side of the BGP session. If you are using a public ASN, you must own it. If you are using a private ASN, it must be in the 1 to 2147483647 range. Autonomous System (AS) prepending does not work if you use a private ASN for a public virtual interface.Note: For IPv6 addresses, AWS automatically allocates you a /125 IPv6 CIDR. You can't specify your own peer IPv6 addresses.Approving prefixes and BGP ASN over public VIFWhen you create a public virtual interface, the following information is subject to approval by the Direct Connect team:The BGP Autonomous System Number (only if it's a public ASN)The Public peer IP addressesThe Public prefixes that you plan to advertise over the virtual interfaceIf you advertised the prefixes before they are approved, you might need to clear the BGP session and then re-advertise the prefixes after approval.For more information, see My Direct Connect public virtual interface is stuck in the "Verifying" state. How can I get it approved?Advertising prefixes over public VIFYou must advertise at least one public prefix using BGP.The public IP addresses used for peering and public IP addresses advertised can't overlap with other public IP addresses announced or used in Direct Connect. You can verify ownership of BGP ASN and IP address prefixes using a WHOIS query.Example output:AS    | IP        | BGP Prefix   | CC | Registry | Allocated  | AS Name12345 | 192.0.2.0 | 192.0.2.0/24 | US | arin     | 1991-12-19 | EXAMPLE-02, USAWS prefixes received on premises over public VIFAfter BGP is established over your public VIF, you receive all available local and remote AWS Region prefixes. To verify the available prefixes, check that the BGP communities on the prefixes received from AWS. For more information, see How can I control the routes advertised and received over the AWS public virtual interface with Direct Connect?AWS Direct Connect applies the following BGP communities to its advertised routes:7224:8100—Routes that originate from the AWS Region where the Direct Connect point of presence is located7224:8200—Routes that originate from the continent where the Direct Connect point of presence is locatedNo tag—Global (all public AWS Regions)Connecting to AWS over public VIFDirect Connect performs inbound packet filtering to validate that the source of the traffic originated from your advertised prefix.Be sure that you connect from a prefix that's advertising to a public VIF. You can't connect from a prefix that isn't advertised to a public VIF.Related informationHow do I connect my private network to AWS public services using an AWS Direct Connect public VIF?Follow"


Question: How can I use BGP communities to control the routes advertised and received over the AWS public virtual interface with Direct Connect?

"How can I control the routes advertised and received over the AWS public virtual interface (VIF) to a specific Region, continent, or globally?"

Answer: "How can I control the routes advertised and received over the AWS public virtual interface (VIF) to a specific Region, continent, or globally?Short descriptionAWS Direct Connect locations in AWS Regions or in the AWS GovCloud (US) Region can access public services in any AWS Region (excluding the China (Beijing) Region). Direct Connect advertises all local and remote AWS Region prefixes where available, and includes on-net prefixes from other AWS non-Region points of presence (POPs) where available, such as Amazon CloudFront. For more information, see Routing policies and BGP communities.ResolutionDirect Connect supports a range of Border Gateway Protocol (BGP) community tags to help control the scope (Regional, continent, or global) of routes advertised and received over a public VIF.Direct Connect BGP community tags that AWS advertises to your customer gateway device over the public VIF include:7224:8100—Routes that originate from the AWS Region where the Direct Connect point of presence is located.7224:8200—Routes that originate from the continent where the Direct Connect point of presence is located.No tag—Global (all public AWS Regions).If you have a public VIF in the us-east-1 Region, then AWS advertises the routes associated for public resources in us-east-1 Region with a community tag of 7224:8100. For routes for public resources in North America, AWS advertises the routes with a community tag of 7224:8200. For all other prefixes, there is no tag.Direct Connect BGP community tags that you can use to select the scope of your prefixes to AWS:7224:9100—Local AWS Region where the Direct Connect point of presence is located.7224:9200—All AWS Regions for the continent (for example, North America) where the Direct Connect point of presence is located.7224:9300 or no tag—Global (all public AWS Regions).If you have a public VIF in the us-east-1 Region, you can limit the scope of the routes you advertise to us-east-1 Region with the community tag of 7224:9100. If you tag your routes with the community tag of 7224:9200, then your prefixes are advertised to all US Regions (North America continent). If you tag your routes with the community tag of 7224:9300, or if you do not tag your prefixes with a community tag, then your prefixes are advertised to all AWS Regions.For example, to limit the routes received and advertised over the public VIF to a specific local Region, make sure that you configure a prefix filter and a route map that matches the routes received from AWS with the community tag of 7224:8100, and then install only those routes. You also must advertise your prefixes to AWS with a community tag of 7224:9100. This makes sure that the routes received and advertised over the public VIF are limited to the local Region.You can use any combination of the community tags to control the routes advertised and received over an AWS public VIF.AWS Direct Connect advertises all public prefixes with the NO_EXPORT BGP community tag.For the current list of prefixes advertised by AWS, download the AWS JSON IP address ranges. For more information, see AWS IP address ranges.Related informationRouting policies and BGP communitiesFollow"


Question: How do I resolve the error "GENERIC_INTERNAL_ERROR" when I query a table in Amazon Athena?

"When I query my Amazon Athena table, I receive the error "GENERIC_INTERNAL_ERROR"."

Answer: "When I query my Amazon Athena table, I receive the error "GENERIC_INTERNAL_ERROR".Short descriptionThe different types of GENERIC_INTERNAL_ERROR exceptions and their causes are the following:GENERIC_INTERNAL_ERROR: null: You might see this exception under either of the following conditions:You have a schema mismatch between the data type of a column in table definition and the actual data type of the dataset.You're running a CREATE TABLE AS SELECT (CTAS) query with inaccurate syntax.GENERIC_INTERNAL_ERROR: parent builder is null: You might see this exception when you query a table with columns of data type array, and the SerDe format OpenCSVSerDe. OpenCSVSerde format doesn't support the array data type.GENERIC_INTERNAL_ERROR: Value exceeds MAX_INT: You might see this exception when the source data column is defined with the data type INT and has a numeric value greater than 2,147,483,647.GENERIC_INTERNAL_ERROR: Value exceeds MAX_BYTE: You might see this exception when the source data column has a numeric value exceeding the allowable size for the data type BYTE. The data type BYTE is equivalent to TINYINT. TINYINT is an 8-bit signed INTEGER in two’s complement format with a minimum value of -128 and a maximum value of 127.GENERIC_INTERNAL_ERROR: Number of partition values does not match number of filters: You might see this exception if you have inconsistent partitions on Amazon Simple Storage Service (Amazon S3) data. You might have inconsistent partitions under either of the following conditions:Partitions on Amazon S3 have changed (example: new partitions added).Number of partition columns in the table do not match that in the partition metadata.GENERIC_INTERNAL_ERROR: Multiple entries with same key: You might see this exception due to Keys (columns) in the JSON data when:The same name is used twice.The same name is used when it’s converted to all lowercase.ResolutionGENERIC_INTERNAL_ERROR:nullColumn data type mismatch: Be sure that the column data type in the table definition is compatible with the column data type in the source data. Athena uses schema-on-read technology. This means that your table definitions are applied to your data in Amazon S3 when the queries are processed.For example, when a table created on Parquet files:Athena reads the schema from the filesThen Athena validates the schema against the table definition where the Parquet file is queried.If the underlying data type of a column doesn't match the data type mentioned during table definition, then the Column data type mismatch error is shown.To resolve this issue, verify that the source data files aren't corrupted. If there is a schema mismatch between the source data files and table definition, then do either of the following:Update the schema using the AWS Glue Data Catalog.Create a new table using the updated table definition.If the source data files are corrupted, delete the files, and then query the table.Inaccurate syntax: You might get the "GENERIC INTERNAL ERROR:null" error when both of the following conditions are true:You created the table using the CTAS query.You used the same column for table properties partitioned_by and bucketed_by.To avoid this error, you must use different column names for partitioned_by and bucketed_by properties when you use the CTAS query. To resolve this error, create a new table by choosing different column names for partitioned_by and bucketed_by properties.GENERIC_INTERNAL_ERROR: parent builder is nullTo resolve this error, find the column with the data type array, and then change the data type of this column to string. To change the column data type to string, do either of the following:Update the schema in the Data Catalog.Create a new table by choosing the column data type as string.Run the SHOW CREATE TABLE command to generate the query that created the table. Then view the column data type for all columns from the output of this command. Find the column with the data type array, and then change the data type of this column to string.To update the schema of the table with Data Catalog, do the following:Open the AWS Glue console.On the navigation pane, choose Tables.Select the table that you want to update.Choose Action, and then choose View details.Choose Edit schema.Scroll to the column with data type array, and then choose array.For Column type, select string from the dropdown list.Choose Update.On the Edit schema page, choose Save.GENERIC_INTERNAL_ERROR: Value exceeds MAX_INTTo resolve this error, find the column with the data type int, and then update the data type of this column from int to bigint. To change the column data type, update the schema in the Data Catalog or create a new table with the updated schema.Run the SHOW CREATE TABLE command to generate the query that created the table. Then view the column data type for all columns from the output of this command. Find the column with the data type int, and then change the data type of this column to bigint.To update the schema of the table with Data Catalog, do the following:Open the AWS Glue console.On the navigation pane, choose Tables.Select the table that you want to update.Choose Action, and then choose View details.Choose Edit schema.Scroll to the column with data type int, and then choose int.For Column type, select bigint from the dropdown list.Choose Update.On the Edit schema page, choose Save.GENERIC_INTERNAL_ERROR: Value exceeds MAX_BYTETo resolve this error, find the column with the data type tinyint. Then, change the data type of this column to smallint, int, or bigint. Or, you can resolve this error by creating a new table with the updated schema.Run the SHOW CREATE TABLE command to generate the query that created the table. Then, view the column data type for all columns from the output of this command. Find the column with the data type tinyint, and change the data type of this column to smallint, bigint, or int.To update the schema of the table with Data Catalog, do the following:Open the AWS Glue console.In the navigation pane, choose Tables.Select the table that you want to update.Choose Action, and then choose View details.Choose Edit schema.Scroll to the column with data type tinyint, and then choose tinyinit.For Column type, select smallint, bigint, or int from the dropdown list.Choose Update.On the Edit schema page, choose Save.GENERIC_INTERNAL_ERROR: Number of partition values does not match number of filtersTo resolve this error, do either of the following:Create a new table using an AWS Glue Crawler.Drop the partitions using the ALTER TABLE DROP PARTITION statement. Then, add the same number of partitions as in the table definition with the ALTER TABLE ADD PARTITION statement. For example, suppose you have two partition columns date and country in the table definition and a partition that has only one column date. Drop the partition with the date column, and then add both partitions to the table.ALTER TABLE doc_example_table DROP PARTITION (date = '2014-05-14');    ALTER TABLE doc_example_table ADD PARTITION (date = '2016-05-14', country = 'IN');GENERIC_INTERNAL_ERROR: Multiple entries with same keyIf rows have multiple columns with the same key, pre-processing the data is required to include a valid key-value pair. If only some of the records have duplicate keys, and if you want to ignore these records, set ignore.malformed.json as SERDEPROPERTIES in org.openx.data.jsonserde.JsonSerDe.If the key names are same but in different cases (for example: “Column”, “column”), you must use mapping. This is because hive doesn’t support case sensitive columns. To do this, you must configure SerDe to ignore casing.Do the following:CREATE TABLE mytable (    time1 string,   time2 string) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'WITH SERDEPROPERTIES ("case.insensitive" = "false", --tells hive to ignore key case"mapping.time1"= "time", -- lowercase 'time' mapped into 'time1'"mapping.time2"= "Time") -- uppercase to 'time2'Related informationData types in Amazon AthenaPartitioning data in AthenaFollow"


Question: How can I use an SSH tunnel through AWS Systems Manager to access my private VPC resources?

I want to use an SSH tunnel through AWS Systems Manager to access my private VPC resources. How can I do this?

Answer: "I want to use an SSH tunnel through AWS Systems Manager to access my private VPC resources. How can I do this?Short descriptionTo create an SSH tunnel, you can use Session Manager, a capability of AWS Systems Manager that lets you use port forwarding for remote hosts. This feature is supported on SSM Agent versions 3.1.1374.0 and later. Port forwarding is an alternative to the steps below. For more information about remote host port forwarding, see Start a session.Session Manager uses the Systems Manager infrastructure to create an SSH-like session with an instance. Session Manager tunnels real SSH connections, allowing you to tunnel to another resource within your virtual private cloud (VPC) directly from your local machine. A managed instance that you create acts as a bastion host, or gateway, to your AWS resources.The benefits of this configuration are:Increased Security: This configuration uses only one Amazon Elastic Compute Cloud (Amazon EC2) instance (the bastion host), and connects outbound port 443 to Systems Manager infrastructure. This allows you to use Session Manager without any inbound connections. The local resource must allow inbound traffic only from the instance acting as bastion host. Therefore, there's no need to open any inbound rule publicly.Ease of use: You can access resources in your private VPC directly from your local machine.Note: For instructions to access your EC2 instances with a terminal or a single port forwarding using Systems Manager, see Setting up Session Manager.PrerequisitesComplete the Session Manager prerequisitesInstall the Session Manager plugin for the AWS Command Line Interface (AWS CLI)Allow SSH connections through Session Manager and make sure that SSH connection requirements are met.Note: You must have the following installed to use the SSH feature:1.    SSM Agent v2.3.672.0 or newer.2.    Session Manager Plugin v1.1.23 or newer on your local machine.3.    AWS CLI v1.16.12 or newer on your local machine.ResolutionTo start the SSH tunnel using Session Manager, follow these steps:Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.1.    To start the SSH tunnel, run the following command:ssh -i /path/my-key-pair.pem username@instance-id -L localport:targethost:destport2.    To test access to the tunnel on the target port created in step 1, run the following command:telnet 127.0.0.1 localportIn the preceding example, 127.0.0.1 and localport translate to access targethost:destport.Example ConfigurationsScenario 1Create a tunnel from your local machine to access a MySQL database running on an EC2 instance using the SSM host as a bastion host.Resources usedinstance1: An EC2 instance acting as a bastion host and managed by AWS Systems Manager.    Hostname = ec2-198-51-100-1.compute-1.amazonaws.com Instance id = i-0123456789abcdefainstance2: An EC2 instance running MySQL Database on the default port 3306.    Hostname = ec2-198-51-100-2.compute-1.amazonaws.comInstructions1.    From a local machine (for example, your laptop), run the SSH command to connect to instance1, using Session Manager-based SSH. This command establishes a tunnel to port 3306 on instance2, and presents it in your local machine on port 9090.ssh -i /path/key-pair_instance1.pem username_of_instance1@i-0123456789abcdefa -L 9090:ec2-198-51-100-2.compute-1.amazonaws.com:3306Note: In the preceding example, port 9090 is available on the local machine.2.    From the local machine, access the database using the available port used in step 1 (in this example, 9090).mysql -u user -h 127.0.0.1 -P 9090 -p passwordNote: Any security groups, network access control list (network ACL), security rules, or third-party security software that exist on instance2 must allow traffic from instance1. In the preceding example, instance2 must allow port 3306 access from instance1.Scenario 2Create three tunnels over a single SSH connection from your local machine to:Connect to the SSH port in instance1Access a MySQL database in RDS instanceAccess a webserver in instance3Resources usedinstance1: An EC2 instance acting as a bastion host and managed by AWS Systems Manager.    Hostname = ec2-198-51-100-1.compute-1.amazonaws.com Instance id = i-0123456789abcdefaRDS instance: A MySQL RDS instance located in a private subnet.    Hostname = DBinstanceidentifier.abcdefg12345.region.rds.amazonaws.cominstance3: An EC2 instance located in a private subnet    Hostname = ec2-198-51-100-3.compute-3.amazonaws.comInstructions1.    Start the session with three tunnels using the SSH command.Note: There are three separate tunnel invocations in the command.ssh -i /path/key-pair_instance1.pem username_of_instance1@i-0123456789abcdefa -L 8080:ec2-198-51-100-1.compute-1.amazonaws.com:22 -L 9090:DBinstanceidentifier.abcdefg12345.region.rds.amazonaws.com:3306 -L 9091:ec2-198-51-100-3.compute-1.amazonaws.com:80Note: In the preceding example, ports 8080, 9090, and 9091 are available on the local machine.2.    Access SSH from the local machine to instance1. The local port 8080 tunnels to the SSH port (22) on instance1. The key-pair and username are for the instance you are tunneling to (instance1, in this example).ssh -i /path/key-pair_instance1.pem username_of_instance1@127.0.0.1 -p 80803.    Access the database on RDS instance. The local port 9090 tunnels to port 3306 on RDS instance. You can use MySQL workbench, which allows you to access the DB server using the GUI, with 127.0.0.1 as hostname and 9090 as port. Or, run the following command in the shell command prompt:mysql -u user -h 127.0.0.1 -P 9090 -p password4.    From the local machine, to access the website on instance3, open the browser and navigate to the website.http://127.0.0.1:9091Important: Any security groups, network ACL, security rules, or third-party security software that exist on RDS instance and instance3 must allow traffic from instance1. In the preceding example, instance3 must allow port 80 access from instance1.Related informationAutomated configuration of Session Manager without an internet gatewaysession-manager-without-igwSecurely connect to an Amazon RDS or Amazon EC2 database instance remotely with your preferred GUIFollow"


Question: How do I mount an EFS file system on an ECS container or task running on EC2?

I want use Amazon Elastic File System (Amazon EFS) with Amazon Elastic Container Service (Amazon ECS) container or tasks using an Amazon Elastic Compute Cloud (Amazon EC2) launch type. How can I do this?

Answer:
"I want use Amazon Elastic File System (Amazon EFS) with Amazon Elastic Container Service (Amazon ECS) container or tasks using an Amazon Elastic Compute Cloud (Amazon EC2) launch type. How can I do this?Short descriptionYou can mount an EFS file system on a task or container running on an EC2 instance. To do this, create a task definition that provides the file system ID in the volume task definition parameters. This allows the EFS file system to automatically mount to the tasks that you specify in your task definition.Required resources:Amazon ECS cluster (EC2 launch type)Amazon EFS file systemResolutionNetwork requirementsThe EFS file system and ECS cluster must be in the same VPC.The security groups associated with the EFS file system must allow inbound connections on port 2049 (network file system, or NFS) from the ECS container instance and the ECS task.Security groups of the ECS instance or tasks must allow outbound connections on port 2049 to the EFS file system's security group.Create a task definition1.    Open the Amazon ECS console and select Task Definitions, Create new Task Definition.2.    Choose EC2 for the launch type compatibility, then select Next step.3.    In Configure task and container definitions, enter a name for your task definition.4.    In the Volume section, choose Add volume.5.    Enter the name of the volume, and then select EFS from the Volume types drop down menu.6.    For the File system ID, select the ID of the file system to use with the ECS tasks.7.    (Optional) Specify the Root directory, Encryption in transit, and EFS IAM authorization if needed based on your requirements. If no options are modified, then the default root directory "/" is used.8.    Select Add.9.    While creating the container, under Container definitions, select Add container to use the previously created volume. Then, under Storage and Logging in the Mount points sub-section, select the volume that you created in step 4.10.    For container path, choose the directory path within the container for your application, and then choose Add.11.    Complete the remaining required fields in the task definition wizard and then choose Create.In the following example, the task definition creates a data volume named efs-ec2-test. The nginx container mounts the host data volume at the /usr/share/nginx/html path.{  "containerDefinitions": [    {      "memory": 128,      "portMappings": [        {          "hostPort": 80,          "containerPort": 80,          "protocol": "tcp"        }      ],      "essential": true,      "mountPoints": [        {          "containerPath": "/usr/share/nginx/html",          "sourceVolume": "efs-ec2-test"        }      ],      "name": "nginx",      "image": "nginx"    }  ],  "volumes": [    {      "name": "efs-ec2-test",      "efsVolumeConfiguration": {        "fileSystemId": "fs-1324abcd",        "transitEncryption": "ENABLED"      }    }  ],  "family": "efs-test"}Note: Replace the  fileSystemid, containerPath, and other task definition parameters based on the values for your custom configuration.In the preceding example, you can create a sample index.html file in the file system's root directory with the following content:
You are using an Amazon EFS file system for persistent container storage.
  Run an ECS task1.    Run the ECS task using the task definition created earlier.2.    Make sure that the EFS file system mounts successfully to the EC2 container by accessing the website using the ECS instance's public IP address.Follow"