Question: How do I remove non-valid characters from my Amazon Redshift data?

There are non-valid characters in my Amazon Redshift data. How do I remove them?

Answer: "There are non-valid characters in my Amazon Redshift data. How do I remove them?Short descriptionIf your data contains non-printable ASCII characters, such as null, bell, or escape characters, you might have trouble retrieving the data or unloading the data to Amazon Simple Storage Service (Amazon S3). For example, a string that contains a null terminator, such as "abc\0def," is truncated at the null terminator, resulting in incomplete data.ResolutionUse the TRANSLATE function to remove the non-valid character. In the following example, the data contains "abc\u0000def". The TRANSLATE function removes the null character "\u0000" and replaces it with an empty value, which removes it from the string:admin@testrs=# select a,translate(a,chr(0),'') from invalidstring;   a    | translate --------+----------- abc    | abcdef abcdef | abcdef(2 rows)To remove specified non-valid characters from all rows in a table, run the UPDATE command with the TRANSLATE function, as shown in this example:admin@testrs=# select * from invalidstring;   a    -------- abc abcdef(2 rows)admin@testrs=# update invalidstring set a=translate(a,chr(0),'') where a ilike '%'||chr(0)||'%';UPDATE 1 admin@testrs=# select * from invalidstring;   a    -------- abcdef abcdef(2 rows)Related informationCHR FunctionFollow"


Question: Why can't I delete my Amazon S3 bucket?

I can't delete my Amazon Simple Storage Service (Amazon S3) bucket and I'm not sure why.

Answer: "I can't delete my Amazon Simple Storage Service (Amazon S3) bucket and I'm not sure why.ResolutionPrerequisitesBefore you delete an Amazon S3 bucket, confirm the following points:All AWS accounts share the Amazon S3 namespace. If you delete a bucket name, then the name becomes available for all users. If another AWS account claims the bucket name, then you can't reuse the bucket name. It's a best practice to empty the bucket instead of deleting it entirely.For buckets that are hosted as a static website, review and update Amazon Route 53 hosted zone settings that relate to the bucket.If the bucket receives log data, then stop the delivery of logs to the bucket before you delete it.Amazon S3 bucket isn't emptyTo delete an Amazon S3 bucket, the bucket must be empty. Use the AWS Management Console, AWS Command Line Interface (AWS CLI), or SDK to delete a bucket manually. If the bucket is large and has versioning configured, then it takes a long time to delete the objects manually. In these cases, use Amazon S3 Lifecycle configuration to empty the buckets.Note: If you receive errors when running AWS CLI commands, make sure that you're using the most recent AWS CLI version.For buckets that have versioning configured or are in a suspended status, include the following rules in your lifecycle configuration:Rule 1: Expire all current versions of objects after X days of creation. Permanently delete all noncurrent versions of objects after Y days of becoming noncurrent.Rule 2: Expire all lone delete markers and incomplete multipart uploads after Z days.For buckets with versioning not configured, include the following rules in your lifecycle configuration:Rule 1: Expire all current versions of the objects after X days of object creation.Rule 2: Expire all incomplete multipart uploads after Z days.Note: To delete the bucket quickly in this example, set X, Y, and Z to 1 day.Amazon S3 runs the lifecycle rules daily at 12:00AM UTC. After the lifecycle rules run, all objects that are eligible for expiration are marked for deletion. Because the lifecycle policy actions are asynchronous, it takes several days for the objects to be physically deleted from the bucket. After an object is marked for deletion, you're no longer charged for the storage that's associated with the object.Using the AWS CLIRun the following command to permanently delete objects from an Amazon S3 bucket with versioning not configured:aws s3 rm s3://bucket-name --recursiveRun the following command to permanently delete all objects in an Amazon S3 bucket with versioning configured or suspended:aws s3api delete-objects --bucket BUCKET_NAME --delete "$(aws s3api list-object-versions --bucket BUCKET_NAME --output=json --query='{Objects: Versions[].{Key:Key,VersionId:VersionId}}')"Using the Amazon S3 consoleOpen the Amazon S3 console.In the navigation pane, choose Buckets.Under Buckets, select the bucket that you want to empty. Then, choose Empty.On the Empty bucket page, type permanently delete in the text field to delete all objects in the bucket. Then, choose Empty.(Optional) Review the Empty bucket: Status page to see the emptying progress.Note: If object versions in an Amazon S3 bucket are locked in governance mode, then the AWS Identity and Access Management (IAM) identity requires "s3:BypassGovernanceRetention" permissions. To bypass governance mode, you must include the "x-amz-bypass-governance-retention:true" header in your request. For requests made in the AWS Management Console, the Console applies the header automatically to requests with the required permissions to bypass governance mode.During the retention period, an IAM identity can't delete object versions protected in compliance mode. This includes the root user of the account. After the retention period, delete the objects that are protected in compliance. After the bucket is empty, delete the Amazon S3 bucket.Amazon S3 bucket has access points associated with itBefore you delete the Amazon S3 bucket, delete all access points attached to the bucket. For more information, see Deleting an access point.IAM identity making the DeleteBucket request doesn't have sufficient permissionsGrant the IAM identity that is deleting the Amazon S3 bucket DeleteBucket permissions on the IAM policy. Or, grant the Amazon S3 bucket policy permissions to perform the DeleteBucket action.Explicit DENY statement preventing the deletionAn explicit DENY statement takes precedence over an explicit ALLOW statement. Confirm that the following policies don't contain any explicit DENY statements:IAM policy of the IAM identityAmazon S3 bucket policyVirtual Private Cloud (VPC) endpoint policyWhen AWS Elastic Beanstalk creates a bucket, the policies contain explicit DENY statements by default. Before you delete the Amazon S3 bucket, delete the explicit DENY statement or the bucket policy.Related informationEmptying a bucketHow S3 Object Lock worksDeleting a bucketFollow"


Question: How can I increase the binlog retention in my Aurora MySQL-CompatibleDB cluster?

I have an Amazon Aurora MySQL-Compatible Edition DB cluster. I want to increase the binlog retention to increase the performance of binlog extraction. How can I do this?

Answer: "I have an Amazon Aurora MySQL-Compatible Edition DB cluster. I want to increase the binlog retention to increase the performance of binlog extraction. How can I do this?Short descriptionYou can increase the availability of your Aurora MySQL-Compatible DB cluster's binlogs by increasing the binlog retention period of the DB cluster.Note: Turning on binlog on your Aurora MySQL-Compatible DB cluster has the following performance effects:Causes additional write overhead (turn it on only when necessary)Increases engine start-up time at reboot because of the binlog recovery processAs a best practice, turn on binary logging in your Aurora MySQL-Compatible DB cluster in the following circumstances:For Aurora cross-Region read replicaFor Aurora manual replication to an external MySQL-compatible databaseNote: Aurora MySQL-Compatible doesn't use binlogs for intra-cluster replication. Aurora MySQL-Compatible global databases don't use binlogs.ResolutionTurn on binary logging in Aurora MySQL-Compatible DB cluster1.    Open the Amazon Relational Database Service (Amazon RDS) console.2.    In the navigation pane, choose Parameter groups.Note: If you're using the default Aurora DB cluster parameter group, then create a new DB cluster parameter group. For Type, choose DB Cluster Parameter Group.3.    Choose the DB custom cluster parameter group. Then, choose Parameter group actions.4.    Choose Edit.5.    Change the value for the binlog_format parameter. For example: ROW, Statement, or MIXED.6.    Choose Save changes.For more information, see How do I turn on binary logging for my Aurora MySQL-Compatible cluster?Increase the binlog retention in Aurora MySQL-Compatible DB clusterMake sure that the binlog files on your replication source are retained until the changes are applied to the replica.Note: Make sure that you choose a time frame to retain the binlog files before they are deleted. The retention time frame must be long enough to make sure that changes are applied to your replica before they are deleted.To increase the binlog retention of the DB cluster, use the mysql_rds_set_configuration procedure. You can run the following command and sample parameters on the writer instance to retain the binlog files for 7 days:CALL mysql.rds_set_configuration('binlog retention hours', 168);Note: For Aurora MySQL 5.7-compatible, the maximum value for binlog retention hours is 168 (7 days). If you enter a higher value, then 168 is used by default.For other Aurora MySQL compatible versions, the maximum value for binlog retention hours is 2160 (90 days). If you enter a higher value, then 2160 is used by default.Related informationRetain binary logs on the replication source until they are no longer neededFollow"


Question: How do I troubleshoot SSH or RDP connectivity to my EC2 instances launched in a Wavelength Zone?

I can't connect to my Amazon Elastic Compute Cloud (Amazon EC2) instance launched in a Wavelength Zone using SSH or Windows remote desktop protocol (RDP). I can still ping my instance.

Answer: "I can't connect to my Amazon Elastic Compute Cloud (Amazon EC2) instance launched in a Wavelength Zone using SSH or Windows remote desktop protocol (RDP). I can still ping my instance.Short descriptionThere are restrictions on connecting to an EC2 instance launched in a Wavelength Zone using the public IP address provided by the 5G service provider.In a Wavelength Zone, the carrier gateway turns on the following controls for internet flows by default. You can't remove these controls.TCP is allowed for outbound and response. This means that TCP traffic is allowed only in one direction, from the EC2 instance to the internet.UDP is denied. This includes both inbound and outbound UDP traffic.ICMP is allowed. This means that the carrier gateway allows ICMP inbound and outbound traffic.Pinging an EC2 instance in a Wavelength Zone works with these controls. However, connecting to the instance using SSH or RDP from the internet fails.ResolutionUnlike public IPs, private IP connectivity works exactly the same way as it does for any other EC2 instance in the Region. To connect to your EC2 instance in a Wavelength Zone, do the following:Launch the bastion host in the same VPC as the Wavelength Zone in the Region.Connect through a bastion host using your instance's private IP address.Note: Public IP connectivity restrictions apply only when connecting to your instances in a Wavelength Zone from internet. Connectivity to instances in a Wavelength Zone works as expected if the following conditions are true:Your security group and Network ACL are set up correctly.The SSH or RDP client is located in the carrier network.UDP traffic also works from within the carrier network.Related informationQuotas and considerations for Wavelength ZonesHow AWS Wavelength worksFollow"


Question: How do I provision an Amazon SageMaker Project?

"I want to provision an Amazon SageMaker Project, but I don't know how."

Answer: "I want to provision an Amazon SageMaker Project, but I don't know how.ResolutionSageMaker Projects use templates that AWS Service Catalog imports to your AWS account. Creating a domain activates the import when one of the following actions happens:You turn on Amazon SageMaker Project templates and Amazon SageMaker JumpStart for the account.The EnableSagemakerServicecatalogPortfolio API is called for an existing SageMaker Studio domain.You can provision a SageMaker Project directly from a SageMaker Studio domain. You can also create a SageMaker Project through an API or AWS Command Line Interface (AWS CLI). However, you must provide the ProductId and ProvisioningArtifactId.Before you begin, turn on project templates for your SageMaker Studio domain. Also, provide AWS Identity and Access Management (IAM) access to your SageMaker Studio users.Check the Service Catalog status from the Domain Settings tab in the Amazon SageMaker console. You can also invoke the GetSagemakerServicecatalogPortfolioStatus API, or run the following AWS CLI command:$aws sagemaker get-sagemaker-servicecatalog-portfolio-statusNote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.If you get Access Denied errors or the projects aren't active, then turn on the project templates and grant the required permissions for users. For more information, see SageMaker Studio permissions required to use projects.After you turn on the projects templates, you must get the ProductID and ProvisioningArtifactID values for the templates to create projects.To get the the product and provisioning artifact IDs, complete the following steps:Open the Service Catalog console.In the navigation pane, under Administration, choose Portfolios.Choose the Imported tab.In the search bar, enter Amazon SageMaker Solutions and MLOps products.Choose Amazon SageMaker Solutions and MLOps products.Review the Products page, and get the Product ID.Select the product or project template that you want to use (example: MLOps template for model building and training).Review the Product list page, and get the Provisioning Artifact ID.Use the project and provisioning artifact IDs in your CreateProject API or the following AWS CLI command:$aws sagemaker create-project --project-name myproject--service-catalog-provisioning-details ProductId="prod-xxxxxx",ProvisioningArtifactId="pa-xxxxxx"Follow"


Question: How do I troubleshoot connectivity issues with my Amazon EMR notebook?

I want to troubleshoot connectivity issues with my Amazon EMR notebook.

Answer: "I want to troubleshoot connectivity issues with my Amazon EMR notebook.Short descriptionWhen connecting to your Amazon EMR notebook, you might receive errors similar to the following:Workspace (notebook) is stopped. Internal Error.Workspace (notebook) can now be used in local mode.Notebook is stopped. Service Role does not have the required permissions.Notebook is stopped. Notebook security group sg-xxxxxxxx does not have an egress rule to connect with the master security group sg-yyyyyy. Please fix the security group or use the default option.Notebook is stopped. Notebook security group sg-xxxxxxx should not have any ingress rules. Please fix the security group or use the default option.ResolutionCheck the service role in the EMR notebooks1.    Validate that the notebook's AWS Identity and Access Management (IAM) role has the minimum required permissions. For more information, see Service role for EMR notebooks.2.    Verify that the notebook has all of the permissions contained in AmazonElasticMapReduceEditorsRole. Use the AWS managed policy S3FullAccessPolicy for full access to Amazon Simple Storage Service (Amazon S3). For more information, see AWS managed policy: Amazon S3FullAccess.3.    Remove any permissions restrictions from the bucket policy attached to the S3 bucket where the notebook is located.Check the security groups in the EMR notebooks1.    Validate that the security group used for your notebook has at least the minimal rules required. For more information, see Specifying EC2 security groups for EMR notebooks.2.    It's a best practice to use different security groups for the EMR cluster and the EMR notebook. The security groups for the notebook and the cluster have different inbound and outbound rule requirements.The notebook security group ElasticMapReduceEditors-Editor has an egress rule that allows connection to the master security group ElasticMapReduceEditors-Livy. This connection uses tcp/18888. Remove any outbound rules added in the notebook security group ElasticMapReduceEditors-Editor routing to 0.0.0.0/0.The master security group ElasticMapReduceEditors-Livy has an ingress rule that allows connection to the notebook security group ElasticMapReduceEditors-Editor. This connection uses tcp/18888. Remove any ingress rules added in the master security group ElasticMapReduceEditors-Livyrouting to 0.0.0.0/0.EMR cluster requirements1.    Verify that the attached cluster is compatible and meets all cluster requirements.2.    When Livy impersonation is turned on, verify that hadoop-httpfs is running on the EMR cluster master node.Use the following command to check the status of hadoop-httpfs:$ sudo systemctl status hadoop-httpfsUse the following command to turn on hadoop-httpfs:$ sudo systemctl start hadoop-httpfsFollow"


Question: Why can't I find my DataSync task logs in the CloudWatch log group?

"I ran an AWS DataSync task, but I can't find the logs in the relevant Amazon CloudWatch log group. How can I troubleshoot this?"

Answer: "I ran an AWS DataSync task, but I can't find the logs in the relevant Amazon CloudWatch log group. How can I troubleshoot this?ResolutionConfirm that CloudWatch Logs has a resource policy that allows DataSync to upload logs. Follow these steps to review the CloudWatch Logs resource policies in the AWS Region of your DataSync agent:Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.1.    Run the describe-resource-policies command using the AWS Command Line Interface (AWS CLI):aws logs describe-resource-policies --region us-east-1Note: Each AWS account is allowed up to 10 resource policies per Region for CloudWatch Logs. If you exceed this limit, then you receive an error message when creating your resource policy.2.    Review the output of the command. If a resource policy isn't set up, then the output is similar to the following:{  "resourcePolicies": []}Important: Confirm that the resource policy for DataSync is enabled in the correct AWS Region. The policy must be in the same Region as the DataSync agent that you're using.Follow these steps to create a resource policy that grants DataSync permissions for uploading logs:1.    Create a JSON file that grants DataSync the minimum permissions for uploading logs:{  "Statement": [    {      "Sid": "DataSyncLogsToCloudWatchLogs",      "Effect": "Allow",      "Action": [        "logs:PutLogEvents",        "logs:CreateLogStream"      ],      "Principal": {        "Service": "datasync.amazonaws.com"      },      "Resource": "*"    }  ],  "Version": "2012-10-17"}You can name the file policy.json.2.    Run the put-resource-policy command using the AWS CLI to create a resource policy using the JSON file:aws logs put-resource-policy --policy-name trustDataSync --policy-document file://policy.json --region <Region>Important: Set the AWS Region of your DataSync agent as the value for --region.3.    Run the describe-resource-policies command to confirm that the resource policy was created:aws logs describe-resource-policies --region <Region>Note: Each AWS account is allowed up to 10 resource policies per Region for CloudWatch Logs. If you exceed this limit, you receive an error message when creating your resource policy. Use the put-resource-policy command to verify if you've reached the limit.4.    After you create the resource policy, the command output is similar to the following:{  "resourcePolicies": [    {      "policyName": "trustDataSync",      "policyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"DataSyncLogsToCloudWatchLogs\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"datasync.amazonaws.com\"},\"Action\":[\"logs:PutLogEvents\",\"logs:CreateLogStream\"],\"Resource\":\"*\"}]}",      "lastUpdatedTime": 1577448776606    }  ]}5.    To test the resource policy, run a DataSync task. A few minutes after the task runs, confirm that you're seeing the log stream from the relevant CloudWatch log group.Related informationOverview of managing access permissions to your CloudWatch Logs resourcesFollow"


Question: How do I resolve "403 Error - The request could not be satisfied. Request Blocked" in CloudFront?

Amazon CloudFront is returning the error message "403 Error - The request could not be satisfied. Request Blocked."

Answer: "Amazon CloudFront is returning the error message "403 Error - The request could not be satisfied. Request Blocked."Short descriptionThe error message "403 Error - The request could not be satisfied. Request Blocked." is an error from the client. This error can occur due to the default actions of AWS WAF rules associated with the CloudFront distribution. The following settings might cause a Request Blocked error:When the default action is set to Allow, the request matches a rule that has Action set to Block.When the default action is set to Block, the request matches the conditions of a rule that has Action set to Block.-or-When the default action is set to Block, the request doesn't match the conditions of any rule that has Action set to Allow.For information on troubleshooting other types of 403 errors, see How do I troubleshoot 403 errors from CloudFront?ResolutionTo resolve the Request Blocked error:Open the CloudFront console.Choose the ID for the distribution that you want to update.Choose the General tab.Under Settings, in the AWS WAF web ACL list, choose the web access control list (web ACL) associated with your distribution.In the AWS WAF console, choose Web ACLs.On the Web ACLs page, for AWS Region, choose Global (CloudFront).Choose the web ACLs that require review. Check that the AWS WAF default action is set on the web ACL.To resolve the Request Blocked error when the default action is Allow, review the requests. Be sure that they don't match the conditions for any AWS WAF rules with Action set to Block.If valid requests match the conditions for a rule that blocks requests, then update the rule to allow the requests.To resolve the Request Blocked error when the default action is Block, review the requests. Be sure that they match the conditions for any AWS WAF rules with  Action set to Allow.If valid requests don't match any existing rules that allow requests, then create a rule that allows the requests.Note: For more troubleshooting, use theAWS WAF console to review a sample of requests that match the rule that might cause the Request Blocked error. For more information, seeTesting and tuning your AWS WAF protections.Related informationHow do I resolve "403 ERROR - The request could not be satisfied. Bad Request" in Amazon CloudFront?How AWS WAF worksUsing AWS WAF to control access to your contentFollow"


Question: How do I resolve issues with deleting my Amazon EBS snapshot?

"I'm trying to delete my Amazon Elastic Block Store (Amazon EBS) snapshot, but I can't. How do I resolve this issue?"

Answer: "I'm trying to delete my Amazon Elastic Block Store (Amazon EBS) snapshot, but I can't. How do I resolve this issue?Short descriptionThe following are common reasons why Amazon EBS snapshot deletion fails:The AWS Identity and Access Management (IAM) user or role doesn't have permission to run the DeleteSnapshot API action.Another account owns the snapshot and shares it with your AWS account.The snapshot of the EBS volume root device is used by a registered Amazon Machine Image (AMI).The snapshot is in the Recycle Bin.The snapshot is created in AWS Backup, or the snapshot that's created in AWS Backup is restored from the Recycle Bin.The snapshot is created using Amazon Data Lifecycle Manager and is in or restored from the Recycle Bin.DeleteSnapshot API results aren't immediately visible to subsequent commands.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.The IAM user or role doesn't have permission to run the DeleteSnapshot API actionIn AWS CloudTrail, you receive the error message: "You are not authorized to perform this operation. Encoded authorization failure message: Bght_tAZ......"To decode the authorization failure message, run the following command:$  aws sts decode-authorization-message --encoded-message encoded_messageNote: Replace encoded_message with the encoded authorization failure message that you received.You can also use the IAM policy simulator to troubleshoot. Check the policy that's related to the IAM user or role to see if it has a rule that denies the ec2:DeleteSnapshot action.Example JSON policy that denies the ec2:DeleteSnapshot action:{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "VisualEditor0",      "Effect": "Deny",      "Action": "ec2:DeleteSnapshot",      "Resource": "*"    }  ]}Also, check for rules that deny any conditions that must be satisfied for the operation to complete, such as ec2:SnapshotID. Update the IAM user or role policy to allow snapshot deletion.For a list of related condition keys, see the DeleteSnapshot section of Actions, resources, and condition keys for Amazon EC2.The snapshot is owned and shared by a different AWS accountYou receive the error message: "The snapshot 'snap-abcdef1234567890' does not exist."You can't delete a snapshot from your account that another account owns and shares with your account. If you have access to the account that owns the snapshot, then you can delete it. If not, then you must contact the owner of that account.To check the owner of the snapshot, run the following describe-snapshots AWS CLI command:$ aws ec2 describe-snapshots --snapshot-id snap-abcdef1234567890Note: Replace snap-abcdef1234567890 with your snapshot's ID.You can also find information about the snapshot in the Amazon Elastic Compute Cloud (Amazon EC2) console. For more information see, View Amazon EBS snapshot information.If you own the snapshot and want to revoke sharing the snapshot with other accounts, then follow these steps:Open the Amazon EC2 console.In the navigation pane, choose Snapshots.Select the snapshot that you shared, and then choose Actions, Modify permissions.Under Shared accounts, select the account ID of the account that you want to revoke snapshot sharing from. Then, choose Remove selected.Choose, Save changes.The snapshot of the Amazon EBS volume root device is used by a registered AMIYou receive the error message: "The snapshot 'snap-abcdef1234567890' is currently in use by ami-abcdef1234567890."Use the AWS Management Console or AWS CLI to deregister your AMI. Then, delete the snapshot.You can find the AMI ID in the error message. Or, you can run the following describe-snapshots AWS CLI command:$ aws ec2 describe-snapshots --snapshot-ids snap-abcdef1234567890You can find the AMI ID in the Description section:{  "Snapshots": [    {      "Description": "Created by CreateImage(i-abcdef1234567890) for ami-abcdef1234567890",      "Encrypted": false,      "OwnerId": "111122223333",      "Progress": "100%",      "SnapshotId": "snap-abcdef1234567890",      "StartTime": "2022-11-12T03:15:16.272000+00:00",      "State": "completed",      "VolumeId": "vol-abcdef1234567890",      "VolumeSize": 8,      "StorageTier": "standard"    }  ]}The snapshot is in the Recycle BinYou receive the error message: "An error occurred (InvalidSnapshot.NotFound) when calling the DeleteSnapshot operation. The snapshot 'snap-abcdef1234567890' does not exist."If you delete a snapshot using the AWS CLI and receive the preceding error message, then the snapshot might be in the Recycle Bin. You can't delete a snapshot that's in the Recycle Bin. The snapshot is deleted only when the retention period expires.To check if the snapshot is in the Recycle Bin, run the list-snapshots-in recycle-bin AWS CLI command:aws ec2 list-snapshots-in-recycle-bin --snapshot-id snap-abcdef1234567890 --region regionNote: Replace region with your AWS Region.Example output:{  "Snapshots": [    {      "SnapshotId": "snap-0460a240fc523552e",      "RecycleBinEnterTime": "2022-11-13T16:33:54.707000+00:00",      "RecycleBinExitTime": "2022-11-14T16:33:54.707000+00:00",      "Description": "",      "VolumeId": "vol-08d1428974b817a18"    }  ]}If you need to delete the snapshot before the retention period expires, then you can restore the snapshot from the Recycle Bin. Make sure that your IAM user or role has the correct permissions to view and recover snapshots that are in the Recycle Bin.Then, check your AWS Region's retention rules. For a tag-level retention rule, modify the snapshot tags so that they don't match the retention rule. Then, delete the snapshot. For a Region-level rule, delete the retention rule, and then delete the snapshot. Deleting the retention rule doesn't affect the other snapshots in the Recycle Bin.The snapshot is created in AWS Backup, or the snapshot that's created in AWS Backup was restored from the Recycle BinYou receive the error message: "snap-abcdef1234567890 This snapshot is managed by AWS Backup service and cannot be deleted via EC2 APIs. If you wish to delete this snapshot, please do so via the Backup console."You can't use the Amazon EC2 console or AWS CLI to delete a snapshot that's created and managed in AWS Backup. You must delete the snapshot from the AWS Backup console. Note the snapshot ID, and then follow the steps for Deleting backups.However, you can't use the AWS Backup console to delete a snapshot that's created in AWS Backup, sent to the Recycle Bin, and then restored. You must delete the snapshot using the Amazon EC2 console or AWS CLI.The snapshot is created using Amazon Data Lifecycle Manager and is stored in the Recycle BinAmazon Data Lifecycle Manager doesn't manage snapshots in the Recycle Bin that are created using Amazon Data Lifecycle Manager or snapshot policies. You must use the Amazon EC2 console or AWS CLI to delete the snapshot.DeleteSnapshot API results aren't immediately visible to subsequent demandsAll Amazon EC2 APIs follow an eventual consistency model. This means that when you use the DeleteSnapshot API, the results might not be immediately visible to subsequent commands that you run.To check the status of a recently deleted snapshot, run the following describe-snapshots AWS CLI command:$ aws ec2 describe-snapshots --region region --snapshot-ids snap-abcdef1234567890If you receive the following error message, then the snapshot is successfully deleted: "An error occurred (InvalidSnapshot.NotFound) when calling the DescribeSnapshots operation: The snapshot 'snap-abcdef1234567890' does not exist."Follow"


Question: How do I resolve the AWS CloudFormation error "Unable to assume role and validate the listeners configured on your load balancer" when I launch an Amazon ECS resource?

I get an error message when I use AWS CloudFormation to launch an Amazon Elastic Container Service (Amazon ECS) resource (AWS::ECS::Service).

Answer: "I get an error message when I use AWS CloudFormation to launch an Amazon Elastic Container Service (Amazon ECS) resource (AWS::ECS::Service).Short descriptionIf I'm using a Classic Load Balancer, I get an error message similar to this:"12:21:48 UTC+0100 CREATE_FAILED AWS::ECS::Service ECSService Unable to assume role and validate the listeners configured on your load balancer. Please verify the role being passed has the proper permissions."If I'm using an Application Load Balancer, I get an error message similar to this:"12:21:48 UTC+0100 CREATE_FAILED AWS::ECS::Service ECSService Unable to assume role and validate the specified targetGroupArn. Please verify that the ECS service role being passed has the proper permissions."If you create an Amazon ECS service with an independent AWS Identity and Access Management (IAM) policy resource that specifies an instance profile, the Amazon ECS service can fail and return an error message.ResolutionTo resolve the error for both Classic Load Balancers and Application Load Balancers, try one or more of the following solutions:Confirm that the IAM role for the Amazon ECS service has the right permissions to register and deregister container instances with your load balancers.Tip: You can use this AWS CloudFormation template as a reference to build out your Amazon ECS architecture components with the right dependencies. The architecture components include an Amazon ECS cluster, service, load balancers, container instances, and IAM resources.Confirm that your AWS Auto Scaling group or Amazon ECS container instance has an instance profile associated as an attribute.Use the DependsOn attribute to specify the dependency of the AWS::ECS::Service resource on AWS::IAM::Policy. Or, use a custom resource to delay the stack creation process and give service role permissions time to propagate.Follow"


Question: How do I resolve problems when accessing a service in the AWS Management Console?

I'm unable to access a service from the AWS Management Console.

Answer: "I'm unable to access a service from the AWS Management Console.Short descriptionYou might receive one of the following error messages:You are not subscribed to this serviceThe AWS Access Key Id needs a subscription for the serviceYour service sign-up is almost complete!ResolutionResolve "You are not subscribed to this service" or "The AWS Access Key Id needs a subscription for the service" errorsWhen you created your AWS account, all available services at that time were activated. However, as new services are released, they aren't automatically put into an active state without your permission. You must subscribe to each service individually as they are released.To activate all existing services and new services as they are released, you must sign in as the AWS account root user. Then, in your account settings page, update your account subscription.Resolve "Your service sign-up is almost complete!" errorsFinish the account sign-up process.You might need to provide a valid payment method, verify your account, or choose a support plan.Related informationCreating your first IAM admin user and groupManaging your AWS payment methodsFollow"


Question: How do I avoid throttling when I call PutMetricData in the CloudWatch API?

I receive the error "400 ThrottlingException" for PutMetricData API calls in Amazon CloudWatch.

Answer: "I receive the error "400 ThrottlingException" for PutMetricData API calls in Amazon CloudWatch.Short descriptionWhen you receive the error "400 ThrottlingException" for PutMetricData API calls in CloudWatch, you also receive the following message:<ErrorResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">  <Error>    <Type>Sender</Type>    Throttling    <Message>Rate exceeded</Message>  </Error>  <RequestId>2f85f68d-980b-11e7-a296-21716fd2d2e3</RequestId></ErrorResponse>To help service performance, CloudWatch throttles requests for each AWS account based on the AWS Region. For current PutMetricData API request quotas, see CloudWatch service quotas.Note: All calls to the PutMetricData API in a Region count towards the maximum allowed request rate. This number includes calls from any custom or third-party application. Examples of such applications include the CloudWatch Agent, the AWS Command Line Interface (AWS CLI), and the AWS Management Console.ResolutionIt's a best practice to use the following methods to reduce your call rate and avoid API throttling:Distribute your API calls evenly over time rather than making several API calls in a short time period. If you require data to be available with a 1-minute resolution, then you have an entire minute to emit that metric. Use jitter (randomized delay) to send data points at various times.Combine as many metrics as possible into a single API call. For example, a single PutMetricData call can include 1,000 metrics and 150 data points. You can also use pre-aggregated data sets, such as StatisticSet, to publish aggregated data points. This reduces the number of PutMetricData calls per second.Retry your call with exponential backoff and jitter.Related informationCloudWatch API referenceFollow"


Question: Why are the items with expired Time to Live not deleted from my Amazon DynamoDB table?

Some of the items with expired Time to Live (TTL) aren't yet deleted from my Amazon DynamoDB table.

Answer: "Some of the items with expired Time to Live (TTL) aren't yet deleted from my Amazon DynamoDB table.ResolutionThe DynamoDB TTL feature allows you to define a per-item timestamp to determine when an item is no longer needed. Shortly after the date and time of the specified timestamp, DynamoDB deletes the item from your table without consuming any write throughput. TTL is provided at no extra cost as a means to reduce stored data volumes by retaining only the items that remain current for your workload’s needs. When you activate TTL on a DynamoDB table, you must identify a specific attribute name that the service will look for when determining if an item is eligible for expiration. After you activate TTL on a table, a per-partition scanner background process automatically and continuously evaluates the expiry status of items in the table.Some of the common causes for deletion of expired items taking longer are the following:The actual delete operation of an expired item might vary depending on the size and activity level of your table. Because TTL is a background process, the nature of the capacity used to expire and delete items using TTL is variable.The TTL process typically deletes expired items within 48 hours of expiration. However, this timeline isn't an SLA. TTL deletes are done on best effort basis and deletion might take longer in some cases. During the deletion of objects using TTL, DynamoDB uses the backend capacity of the table to delete the objects instead of using the provisioned capacity. The deletion process might take longer if there are a large number of delete requests and not enough backend capacity to continuously delete those items.To check if TTL is working properly, do the following:Be sure that you activated TTL on the table and the related settings are correct.The item must contain the attribute that you specified when you activated TTL on the table.The TTL attribute’s value must have the datatype Number.The TTL attribute’s value must be a timestamp in Unix epoch time format in seconds.The TTL attribute value must be a datetimestamp with an expiration of no more than five years in the past.The TTL processes run on the table only when there is enough spare capacity so that these processes don't interfere with table operations. If the table or table partitions are using most of the allocated capacity, TTL processes might not run due to insufficient spare capacity.Items that are expired, but aren't yet deleted by TTL, still appear in reads, queries, and scans. If you don't want expired items in the result set, you must filter them out. To do so, use a filter expression that returns only items where the Time to Live expiration value is greater than the current time in epoch format. For more information, see Filter expressions for scan.Related informationExpiring items by using DynamoDB Time to Live (TTL)Follow"


Question: Why did I receive the GuardDuty finding type alert Recon:EC2/PortProbeUnprotectedPort for my Amazon EC2 instance?

Amazon GuardDuty detected alerts for the Recon:EC2/PortProbeUnprotectedPort finding type for my Amazon Elastic Compute Cloud (Amazon EC2) instance.

Answer: "Amazon GuardDuty detected alerts for the Recon:EC2/PortProbeUnprotectedPort finding type for my Amazon Elastic Compute Cloud (Amazon EC2) instance.Short descriptionThe GuardDuty finding type Recon:EC2/PortProbeUnprotectedPort means that an Amazon EC2 instance has an unprotected port that is being probed by a known malicious host.ResolutionUse the following best practices to protect the unprotected port or remove inbound rules:Follow the instructions to view and analyze your GuardDuty findings.In the findings detail pane, note the port number.If the unprotected port is 22 for Linux, you can restrict access by following the instructions for authorizing inbound traffic for your Linux instances.If the unprotected port is 3389 for Windows, you can restrict access by following the instructions for authorizing inbound traffic for your Windows instances.If the unprotected port is 80 or 443 and you must keep these ports open, you can put the EC2 instance behind a load balancer.If the port doesn't have any application running on it and doesn't need to be open, you can remove the inbound rule for the EC2 instance security group and iptables rules.If you don't need to protect the unprotected port, you can ignore the Recon:EC2/PortProbeUnprotectedPort finding type.Related informationMonitoring GuardDuty findings with Amazon CloudWatch EventsFinding typesFollow"


Question: How can I find which Availability Zones I can use in my account?

"I want to use an Availability Zone (AZ), but it's unavailable for my account. How can I find which AZs I can use in my account?"

Answer: "I want to use an Availability Zone (AZ), but it's unavailable for my account. How can I find which AZs I can use in my account?Short descriptionAs AZs grow, the ability for AWS to expand them can become constrained. As a result, you might notice the following:Your account has fewer available AZs than what's publicly listed.One of your accounts has a different number of available AZs in a Region than another account.ResolutionTo find which AZs you can use to launch resources in any of your accounts, do the following:Open the Amazon Elastic Cloud Compute (Amazon EC2) console.From the navigation bar, view the options in the Region selector.On the navigation pane, choose EC2 Dashboard.In the Service Health section, view the list of AZs under Availability Zone Status.Also, you can run the following command in the AWS Command Line Interface (AWS CLI) to generate a list of AZs in your account.Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure you confirm that you’re running a recent version of the AWS CLI.Note: Be sure to replace region-name with your Region's name. For example, ap-northeast-1.aws ec2 describe-availability-zones --region region-nameRelated informationAvailability ZonesFollow"


Question: How do I submit a Digital Millennium Copyright Act (DMCA) notice to AWS?

Material that I own the copyright for is being distributed using AWS resources without my permission. How do I submit a Digital Millennium Copyright Act (DMCA) notice to AWS?

Answer: "Material that I own the copyright for is being distributed using AWS resources without my permission. How do I submit a Digital Millennium Copyright Act (DMCA) notice to AWS?ResolutionThe Digital Millennium Copyright Act (DMCA) is the United States law governing claims that copyright-protected material is being distributed online.If you believe that AWS resources are being used to host or distribute copyrighted material that you own the copyright for without your permission, first consider whether there is an exception to copyright that applies in this case (such as fair use). If after considering this you still want to make a copyright complaint, you can reach out directly to the site operator using the contact details provided on the site. Or, to submit a DMCA notice to AWS, follow the steps outlined below. Remember that only the copyright owner or their authorized representative can file a report of copyright infringement.Steps to submit a DMCA notice to AWSGather the documentation described at Notice and procedure for making claims of copyright infringement.Send the information to ec2-abuse@amazon.com or abuse@amazonaws.com.Note: The AWS Abuse team won't open attachments under any circumstance. You must provide any necessary information in plaintext.If you provide your notice in the body of an email and send it to ec2-abuse@amazon.com or abuse@amazonaws.com, you don't need to provide a physical signature.What happens next?After we receive your DMCA notice, you'll receive an automated email acknowledging receipt of your notice. If we have questions about your notice, we'll email you asking for more information. We request that you respond to our email so that we can continue looking into your report.If your DMCA notice is complete and valid and the reported content is on AWS, we'll take action expeditiously including contacting our customer to remove or disable access to the infringing content. Whenever we remove content in response to a DMCA notice, we provide a copy of the original complaint as well as your contact information to our customer. If our customer doesn't believe that the content is infringing or that the content shouldn't be removed, they might reach out to you directly to resolve the issue or they might submit a counter-notice to us under the DMCA.Repeat infringer policyAWS terminates the accounts of repeat infringers in appropriate circumstances.The information above is provided for general informational purposes only and is not legal advice. If you have any questions about this information or your specific situation or rights, seek legal advice from a professional.Related informationHow do I provide a counter-notice to a DMCA notice?Follow"


Question: Why did I get an AWS Config error after turning on AWS Security Hub?

How to troubleshoot AWS Config errors after turning on AWS Security Hub.

Answer:
"How to troubleshoot AWS Config errors after turning on AWS Security Hub.Short descriptionWhen setting up AWS Security Hub, you might have one of the following errors:"AWS Config is not enabled on some accounts.""AWS Config is not enabled in all regions.”"An error has occurred with AWS Config. Contact AWS Support."ResolutionUse the following best practices for configuring and troubleshooting AWS Config with Security Hub:Note: AWS Config rules created by Security Hub do not incur any additional costs.Verify that AWS Config is turned on in the same AWS Region as Security HubManually turn on AWS Config in the same Region as Security Hub as follows:1.    Open the AWS Config console in the same Region that you have Security Hub turned on.2.    If AWS Config is not turned on, follow the instructions for setting up AWS Config with the Console.Note: If you have Security Hub configured in multiple Regions, repeat these steps for each Region.Verify AWS Config is recording all resources including global in your RegionModify the type of resources AWS config records as follows:1.    Open the AWS Config console, and choose Settings.2.    In Settings, confirm Recording is on.3.    In Resource types to record, select Record all resources supported in this region.4.     In Resource types to record, select Include global resources (e.g., AWS IAM resources).5.    Choose Save.Note:These settings apply to all of your AWS accounts that are configured with Security Hub, including AWS Organizations member accounts.You do not have to record all resource types in AWS Config. However, be sure that the required resource types for CIS, PCI DSS, and AWS foundational security best practices controls are recording.You do not need to turn on global resources in all Regions. To avoid duplicate configuration settings, you can turn on global settings in only the same AWS Region as Security Hub per AWS account.It can take up to 24 hours for the recorder settings to complete.Use Amazon CloudWatch log filter patterns to search AWS CloudTrail log dataSearch for and troubleshoot AWS Config error messages as follows:1.    Follow steps 1-4 in Search log entries using the console.2.    In Filter, paste the following example syntax, and then choose enter on your device:EventSource: config.amazonaws.com
3.    Note the error. Then, follow the instructions for How can I troubleshoot AWS Config console error messages?Verify the permissions on the Security Hub service-linked roleAWS Security Hub uses service-linked roles to provide permissions to AWS services. The following AWS Identity and Access Management (IAM) permission allows access to AWS Config with Security Hub:{
"Effect": "Allow",
"Action": [
"config:PutConfigRule",
"config:DeleteConfigRule",
"config:GetComplianceDetailsByConfigRule",
"config:DescribeConfigRuleEvaluationStatus"
],
"Resource": "arn:aws:config:*:*:config-rule/aws-service-rule/*securityhub*"
}
For more information, see Using service-linked roles for AWS Security Hub.Related informationAWS Security Hub now generally availableFollow"


Question: I unintentionally incurred charges while using the AWS Free Tier. How do I make sure that I'm not billed again?

I accidentally provisioned resources that are not covered under the AWS Free Tier and I was billed. How do I make sure I'm not billed again?

Answer: "I accidentally provisioned resources that are not covered under the AWS Free Tier and I was billed. How do I make sure I'm not billed again?Short descriptionWith the AWS Free Tier, you can explore and try out some AWS services free of charge within certain usage limits. To learn more about AWS Free Tier, see What is the AWS Free Tier, and how do I use it?When using AWS Free Tier, you might incur charges due to the following reasons:You exceeded the monthly free tier usage limits of one or more services.You're using an AWS service, such as Amazon Aurora, that doesn't offer free tier benefits.Your free tier period expired.To be sure that you aren't billed again:Verify that your account is covered by the free tier.Identify the resources that are generating charges.Delete, stop, or terminate any resources that are generating charges.Proactively monitor your usage to be sure that you don't exceed the free tier offering again.ResolutionVerify that your account is covered by the free tierOpen the Billing and Cost Management console, and then choose Free Tier from the navigation pane.If the table doesn't appear, then your account is no longer covered under the AWS Free Tier. You're billed at standard rates for any resources provisioned on your account. Delete, shut down, or terminate any resources that you don't want to keep or be billed for.If the table appears, then you see additional information about any services that you're using. This table also provides a Month-to-Date (MTD) actual usage percentage.Identify the resources that are generating chargesTo identify active resources in your account, do the following:Open the Billing and Cost Management console.In the navigation pane, on the left side of the screen, choose Bills.The Details section shows all the charges that are being incurred by various AWS services on your account. Make note of which services have active resources. If you use the consolidated billing feature in AWS Organizations, the Bills page lists totals for all accounts on the Consolidated Bill Details tab. Choose the Bill Details by Account tab to see the activity for each account in the organization.Note: It's a best practice to check the Details section for the previous month to identify all services that are generating charges.Under Details, expand each service to identify the Regions where the services have incurred charges. The Bill details by account section shows all the charges incurred in different AWS Regions. Note the resources in those Regions.Note: The Billing and Cost Management console takes about 24 hours to update usage and charge information for active resources.Charges listed under a service are for usage not covered under the AWS Free Tier.For more information on finding your active Amazon Elastic Compute Cloud (Amazon EC2) resources, see Why can't I find an Amazon EC2 instance that I launched on my account?Important: AWS Free Tier benefits cover the aggregated billable total of your usage for all AWS Regions together, but not each Region individually.Delete, stop, or terminate any resources that you don't want to be billed forAfter identifying the AWS resources that are incurring charges, you can stop the billing by deleting, stopping, or terminating the resources.For information on how to delete, stop, or terminate resources that aren't covered by the free tier, see How do I terminate active resources that I no longer need on my AWS account?Proactively monitor your usage of AWS Free Tier resourcesYou can track your free tier usage with the AWS Free Tier usage alerts. AWS provides free tier usage alerts using AWS Budgets. These alerts notify you when your free tier usage exceeds 85 percent of your monthly limit.View the Top AWS Free Tier services table for tracking the free tier usage limit of your services along with your current usage amount.Related informationHow do I make sure I don't incur charges when I'm using the AWS Free Tier?Using the AWS Free TierAWS Free Tier FAQsAvoiding unexpected chargesHow do I close my AWS account?Follow"


Question: How can I troubleshoot issues with Accelerated VPN?

How can I troubleshoot issues with AWS Accelerated VPN?

Answer: "How can I troubleshoot issues with AWS Accelerated VPN?ResolutionConfirm that your firewall configuration meets all requirementsFor more information, see Configuring a firewall between the internet and your customer gateway device.Confirm that NAT-traversal is activated on the customer gateway deviceNAT-traversal (NAT-T) is required for an Accelerated VPN connection. NAT-T is activated by default. If you downloaded a configuration file from the Amazon Virtual Private Cloud (Amazon VPC) console, check the NAT-T setting and start it if necessary.Note: If NAT-T is deactivated on the customer gateway device the tunnel will still come up. However, in this scenario the issue could still persist for the data traffic.For more information, see Your customer gateway device.Confirm that lifetime parameters matchThe IKE tunnel lifetime parameter must match what's set on your AWS Virtual Private Network (AWS VPN). By default, these settings are:28,800 seconds (8 hours) for phase 13,600 seconds (1 hour) for phase 2If necessary, change the AWS VPN parameters to match your IKE tunnel parameters.Confirm the connection's compatibility with Global Accelerator (if applicable)If your Site-to-Site VPN connection uses certificate-based authentication, it might not be compatible with AWS Global Accelerator. There's limited support for packet fragmentation in Global Accelerator. If you require an Accelerated VPN connection that uses certificate-based authentication, your customer gateway device must support IKE fragmentation. Otherwise, don't activate your VPN for acceleration. For more information, see How AWS Global Accelerator works.Confirm that acceleration was configured in the proper sequenceAcceleration can't be activated or deactivated for an existing Site-to-Site VPN connection. Instead, create a new Site-to-Site VPN connection with acceleration activated or deactivated as needed. Then, configure your customer gateway device to use the new Site-to-Site VPN connection. Finally, delete the previous Site-to-Site VPN connection. For more information on Accelerated VPN restrictions, see Rules and restrictions.Follow"


Question: How do I resolve the "Server.InternalError: Internal error on launch" error for a failed stack in AWS CloudFormation?

"I tried to create an Amazon Elastic Compute Cloud (Amazon EC2) instance with an AWS CloudFormation stack, but my stack creation failed. Then, I received the "Server.InternalError: Internal error on launch" error message. How can I resolve this error?"

Answer: "I tried to create an Amazon Elastic Compute Cloud (Amazon EC2) instance with an AWS CloudFormation stack, but my stack creation failed. Then, I received the "Server.InternalError: Internal error on launch" error message. How can I resolve this error?Short descriptionYou receive this error if duplicate or invalid device mappings are specified in your AWS CloudFormation template. You can't have two block devices map to the same location (for example, /dev/sdb).Note: If you're using a Nitro-based instance type (for example, c5, m5, or t3), then you won't receive this error, because /dev/sdb and /dev/xvdb are mapped to two different NVMe devices in the operating system.ResolutionIn the BlockDeviceMappings property of your AWS CloudFormation template, confirm that your block devices aren't mapping to the same location by checking the value of DeviceName for each block device.In the following JSON and YAML example templates, the block devices specified are /dev/xvdb and /dev/xvdc. The root volume is automatically provisioned for the instance, and the block devices are associated as secondary volumes.JSON:    "Ec2Instance" : {      "Type" : "AWS::EC2::Instance",       "Properties" : {        "...OtherProperties..."        "BlockDeviceMappings" : [          {            "DeviceName" : "/dev/xvdb",            "Ebs" : { "VolumeSize" : "100" }          },{            "DeviceName" : "/dev/xvdc",            "Ebs" : { "VolumeSize" : "100" }          }        ]      }    }YAML:EC2Instance:    Type: AWS::EC2::Instance    Properties:        ...OtherProperties...        BlockDeviceMappings:        -          DeviceName: /dev/xvdb          Ebs:            VolumeSize: 100        -          DeviceName: /dev/xvdc          Ebs:            VolumeSize: 100Related informationBlock device mappingEC2 block device mapping examplesDevice names on Linux instancesFollow"


Question: How do I troubleshoot distribution errors for encrypted AMIs in Image Builder?

I receive errors when I try to distribute encrypted AMIs to another account in EC2 Image Builder. How do I troubleshoot this?

Answer: "I receive errors when I try to distribute encrypted AMIs to another account in EC2 Image Builder. How do I troubleshoot this?Short descriptionThe following scenarios can cause distribution errors in EC2 Image Builder when you distribute an encrypted Amazon Machine Image (AMI) to another account:The AMI that's distributed is encrypted using the default AWS managed key for Amazon Elastic Block Store (Amazon EBS).The AWS Key Management Service (AWS KMS) or AWS Identity and Access Management (IAM) entity doesn't have the required permissions.ResolutionThe AMI that's distributed is encrypted using the default AWS managed key for Amazon EBSYou receive the following error:Distribution failed with JobId 'XXXXXXXXXXXXXX', status = 'Failed' for ARN 'arn:aws:imagebuilder:us-east-1:xxxxxxxxxxxx:image/test-recipe/0.0.1/1'. 'Not all distribution jobs are completed. 1) EC2 Client Error: 'Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot.' when distributing the image from the source account (ID: xxxxxxxxxxxx) to the destination account (ID: xxxxxxxxxxxx) in Region us-east-1.'You can't share AMIs that are encrypted with the default AWS KMS key. For more information see, Share an AMI with specific AWS accounts.Scenarios to check include:The AWS managed KMS key is specified in the recipe's storage configuration.The AWS managed KMS key is specified in the distribution configuration along with one or more target accounts.The parent AMI is encrypted using the AWS managed KMS key.The parent AMI has multiple snapshots, and at least one is encrypted with the AWS managed KMS key.Encryption by default is activated in your AWS Region, and it's using the AWS managed KMS key.To resolve this issue, create a new version of the image recipe, and specify a customer managed KMS key for encryption in the recipe's storage configuration. For KMS keys in the distribution configuration, specify a customer managed KMS key for encryption when you distribute AMIs to other accounts.The AWS KMS or IAM entity doesn't have the required permissionsYou can distribute AMIs in Image Builder using either the launchPermissions or targetAccountIds configurations.launchPermissionsWhen you distribute an AMI using launchPermissions, Image Builder uses the IAM role AWSServiceRoleForImageBuilder in the source account. By default, AWSServiceRoleForImageBuilder has the required AWS KMS permission for the resources in the source account.The KMS key policy has a statement that allows the "kms:*" action for the root user. If this statement isn't in the key policy, then the service-linked role can't access the key in the source account. If the "kms:*" action isn't allowed for the root user, then modify the policy to allow the service-linked role to use the key.For example:{  "Sid": "Enable IAM User Permissions",  "Effect": "Allow",  "Principal": {    "AWS": "arn:aws:iam::source_account_id:role/aws-service-role/imagebuilder.amazonaws.com/AWSServiceRoleForImageBuilder"  },  "Action": [    "kms:Decrypt",    "kms:Encrypt",    "kms:GenerateDataKeyWithoutPlaintext",    "kms:DescribeKey",    "kms:CreateGrant",    "kms:ReEncryptFrom",    "kms:ReEncryptTo"  ],  "Resource": "*"}Note: Replace source_account_id with your source account's ID.targetAccountIdsIf the destination account doesn't have the IAM role EC2ImageBuilderDistributionCrossAccountRole, or the source account isn't listed in the trust policy, then you receive the following error:Distribution failed with JobId 'xxxxxxxxxxxxxx', status = 'Failed' for ARN 'arn:aws:imagebuilder:us-east-1:XXXXXXXXXX:image/testdistribution/2.0.0/3'. 'Not all distribution jobs are completed. 1) STS Client Error: 'User: arn:aws:sts::XXXXXXXXXXXX:assumed-role/AWSServiceRoleForImageBuilder/Ec2ImageBuilderIntegrationService is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::xxxxxxxxx:role/EC2ImageBuilderDistributionCrossAccountRole'. Please make sure your 'EC2ImageBuilderDistributionCrossAccountRole' is setup with correct permission policies. If you are copying AMI to opt-in regions, please make sure the region is enabled in the account when distributing the image from the source account (ID: XXXXXXXXXXXX) to the destination account (ID: XXXXXXXXXXXX) in Region us-east-1.'STS Client Error User is not authorized to perform: sts:AssumeRole on resource.To resolve this issue, create the role EC2ImageBuilderDistributionCrossAccountRole. Then, attach the Ec2ImageBuilderCrossAccountDistributionAccess policy to allow cross-account distribution. Then, list AWSServiceRoleForImageBuilder in the EC2ImageBuilderDistributionCrossAccountRole trust policy:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "Service": "ec2.amazonaws.com",        "AWS": "arn:aws:iam::XXXXXXXXXX:root"      },      "Action": "sts:AssumeRole"    }  ]}You might receive the following error due to issues with cross-account AWS KMS permissions:Distribution failed with JobId 'xxxxxxxxxxxxxx', status = 'Failed' for ARN 'arn:aws:imagebuilder:ap-southeast-2:11111111111:image/test/1.0.0/1'. 'Not all distribution jobs are completed. 1) AMI Copy Reported Failure For 'ami-0047623fbcxxxxx' when distributing the image from the source account (ID: 11111111111) to the destination account (ID: 222222222222) in Region ap-southeast-2.'When you distribute an AMI using targetAccountIds, Image Builder uses the role AWSServiceRoleForImageBuilder in the source account. In the destination account, it uses the role EC2ImageBuilderDistributionCrossAccountRole. Make sure that you give permission to EC2ImageBuilderDistributionCrossAccountRole to use the AWS KMS keys in the distribution configuration and recipe's storage configuration.For example:{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "VisualEditor0",      "Effect": "Allow",      "Action": [        "kms:Decrypt",        "kms:Encrypt",        "kms:GenerateDataKeyWithoutPlaintext",        "kms:DescribeKey",        "kms:CreateGrant",        "kms:ReEncryptFrom",        "kms:ReEncryptTo"      ],      "Resource": "*"    }  ]}Note: You can also restrict the Resource section by specifying the ARN of the AWS KMS key.If the customer managed KMS keys belong to the destination account, then you must share the AWS KMS key with the source account. If the customer managed KMS keys belong to the source account, then you must share the AWS KMS keys with the destination account.Complete the following steps to share the AWS KMS keys:1.    Log in to the account where the KMS key is.2.    Open the AWS KMS console in the same AWS Region.3.    In left navigation pane, choose Customer managed keys.4.    Select the KMS key ID.5.    Choose the Key Policy tab.6.    In the Other AWS accounts section, choose Add other AWS accounts.7.    Specify the ID of the account that you want to share the KMS key with.8.    Choose Save Changes.Related informationSet up cross-account AMI distribution with Image BuilderHow do I share my AWS KMS keys across multiple AWS accounts?Follow"


Question: How do I prepare for end of support for Kubernetes 1.15 on Amazon EKS?

I want to prepare for the end of support for Kubernetes 1.15 on Amazon EKS in Amazon Elastic Kubernetes Service (Amazon EKS).

Answer: "I want to prepare for the end of support for Kubernetes 1.15 on Amazon EKS in Amazon Elastic Kubernetes Service (Amazon EKS).Short descriptionKubernetes 1.15 on Amazon EKS reaches end of support on May 3, 2021.Note: After Open Source community support for Kubernetes 1.15 ended on May 6, 2020, AWS provided an additional one year of support for Kubernetes 1.15 on Amazon EKS. This means that no bug fixes or security vulnerability patches will be back-ported to Kubernetes 1.15 on Amazon EKS. For more information, see Kubernetes Patch Releases on the Kubernetes GitHub site.ResolutionTo prepare for end of support, complete the following steps:Understand the impact of end of support.Complete the Kubernetes 1.16 update prerequisites.Update your Amazon EKS clusters by May 3, 2021.For more information on preparing for deprecation, see Preparing for Kubernetes API deprecations when going from 1.15 to 1.16.Note: If you're having issues with your Amazon EKS pods or apps, see Kubernetes 1.16 update prerequisites. The most common issue is kube-proxy failing to start because the --resource-container flag is still in the kube-proxy DaemonSet. To resolve this issue, remove the --resource-container flag.Follow"


Question: How do I create a subdomain for my domain that's hosted in Route 53?

"I want to create a subdomain for my domain that's hosted in Amazon Route 53, but I don't know how."

Answer: "I want to create a subdomain for my domain that's hosted in Amazon Route 53, but I don't know how.Short descriptionUsing a separate hosted zone to route internet traffic for a subdomain is known as delegating responsibility for a subdomain to a hosted zone. It might also be referred to as subdomain delegation through name servers.PrerequisitesBefore you begin, be sure to implement the following requirements:A valid registered domain (regardless of the registrar)An authoritative hosted zone for the registered domain in Route 53ResolutionCreate a hosted zone for the subdomain in Route 53Create a hosted zone with the same name as the subdomain that you want to route traffic for, such as acme.example.com. To do this, complete the following steps:Open the Route 53 console.In the navigation pane, choose Hosted zones.Choose Create hosted zone.In the navigation pane, enter the name of the subdomain (such as acme.example.com).Note: For more information, see DNS domain name format.For Type, accept the default value of Public hosted zone.Choose Create hosted zone.Find the name servers that Route 53 assigned to the new hosted zoneWhen you create a hosted zone, Route 53 automatically assigns four name servers to the zone. To start using the hosted zone for the subdomain, create a new name server (NS) record in the hosted zone for the domain (example.com). The name of the NS record must be the same as the name of the subdomain (acme.example.com).After creating the hosted zone for the subdomain, expand the Hosted zone details dropdown list for the subdomain in the hosted zone (acme.example.com). In the right pane, copy the names of the four servers listed as the Name servers under Hosted zone details.Add NS records to route traffic to your subdomainComplete the following steps to route traffic to your subdomain. These steps also apply to cross-account scenarios where the hosted zone for the domain and the subdomain are in different accounts.Select the hosted zone for the domain (example.com). Be sure not to select the name of the subdomain (some.example.com).In the hosted zone for the domain, choose Create record.For Name, enter the name of the subdomain.For Value, enter the names of the name servers.For Record type, choose NS - Name servers for a hosted zone.For TTL (Seconds), select a more common value for an NS record, such as 172,800 seconds.For Route Policy, choose Simple routing.Choose Create Records.Note: To remove the subdomain delegation (acme.example.com), first delete the NS record in the parent hosted zone (example.com). Then, delete the subdomain hosted zone. These steps protect your subdomain from an unauthorized takeover.Create records in the subdomain hosted zoneCreate your records in the newly-created subdomain hosted zone. Test the record resolution using the dig/nslookup command.Example dig/nslookup command:dig acme.example.comFollow"


Question: How do I set up UDP load balancing with my Network Load Balancer?

I want to register a User Datagram Protocol (UDP) service to my Network Load Balancer. How can I do this?

Answer: "I want to register a User Datagram Protocol (UDP) service to my Network Load Balancer. How can I do this?ResolutionCreate a new target group using UDPOpen the Amazon Elastic Cloud Compute (Amazon EC2) console.From the navigation pane, choose Target Groups.Choose Create target group.For Target type, choose Instances or IP addresses.For Target group name, enter a name for the target group.For Protocol, choose UDP or TCP_UDP.For Port, choose the port that the service is listening on.For VPC, choose your VPC.For Health check protocol, choose TCP, HTTP, or HTTPS.Expand Advanced health check settings, and then choose Override.For Healthy threshold, enter the number of health checks.Choose Next, and then choose Create target group.For more information, see Create a target group for your Network Load Balancer.Register targets to the target groupOpen the Amazon EC2 console.From the navigation pane, in Load Balancing, choose Target Groups.In Target groups, choose your target group, choose Actions, and then choose Register targets.For Available instances, choose your Instance ID or IP address that you created previously, and then choose Include as pending below.Choose Register pending targets.For more information, see Register targets with your target group.Modify the health check for the target groupOpen the Amazon EC2 console.From the navigation pane, in Load Balancing, choose Target Groups.In Target groups, choose your target group.Choose the Health checks tab, and then choose Edit.In Health check protocol, choose your protocol (TCP, HTTP, or HTTPS).Expand Advanced health check settings.In Port, choose either Traffic port or Override, and then choose Save changes.For more information, see Modify the health check settings of a target group.Add a listener to your Network Load BalancerOpen the Amazon EC2 console.From the navigation pane, in Load Balancing, choose Load Balancers.Choose your Network Load Balancer, choose the Listeners tab, and then choose Add listener.Choose the Protocol dropdown list, and then choose your protocol (UDP or TCP_UDP).Choose the Select a target group dropdown list, choose your target group, and then choose Add.For more information, see Create a listener for your Network Load Balancer.Related informationUDP load balancing for Network Load BalancerFollow"


Question: Why am I receiving an error when I try to create an Amazon EC2 Auto Scaling lifecycle hook?

I'm receiving a validation error when I try to create an Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling lifecycle hook. The error reads "Unable to publish test message to notification target" or "Please check your target and role configuration and try to put lifecycle hook again." How do I troubleshoot these errors?

Answer: "I'm receiving a validation error when I try to create an Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling lifecycle hook. The error reads "Unable to publish test message to notification target" or "Please check your target and role configuration and try to put lifecycle hook again." How do I troubleshoot these errors?Short descriptionTo publish a message to the Amazon Simple Queue Service (Amazon SQS), the lifecycle hook's AWS Identity and Access Management (IAM) role must:Be different from the IAM role assigned to the instance.Be listed as a key user on the AWS Key Management Service (AWS KMS) key policy.Have a trust policy attached for the Auto Scaling service.Include specific managed policy actions.Be associated with the Amazon EC2 Auto Scaling group.Have access to the encryption key used by Amazon SQS.Resolution1.    Confirm that you're using an IAM role for the lifecycle hook that's different from the IAM role you've assigned to the instance.Note: You can create an IAM role, or use the following AWS managed role that has all of the necessary permissions:arn:aws:iam::aws:policy/service-role/AutoScalingNotificationAccessRole2.    Verify that the role is included as a key user on the KMS key policy. To do this:Open the AWS KMS console.Select the KMS key.Verify that the role is listed under Key users on the Key policy tab. If the role isn't listed, search for it, and then select Add.3.    Be sure that the IAM role for the lifecycle hook has a trust policy attached for the Amazon EC2 Auto Scaling service.{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "Service": "autoscaling.amazonaws.com"      },      "Action": "sts:AssumeRole"    }  ]}4.    Verify that the managed policy for the lifecycle hook's IAM role includes the following actions:For SQS messagessqs:SendMessagesqs:GetQueueUrlFor SNS notificationssns:Publish5.    In the AWS Command Line Interface (AWS CLI), run the aws autoscaling put-lifecycle-hook command.6.    Run the command below to confirm that the lifecycle hook is associated with the Auto Scaling group.aws autoscaling describe-lifecycle-hooks --auto-scaling-group-name "ExampleSQSQueueName"Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Related informationAmazon EC2 Auto Scaling lifecycle hooksFollow"


Question: Why did the AWS Config auto remediation action for the SSM document AWS-ConfigureS3BucketLogging fail with the error "(MalformedXML)" when calling the PutBucketLogging API?

I want to troubleshoot the errors that I get when I set up auto remediation for non-compliant Amazon Simple Storage Service (Amazon S3) resources.

Answer: "I want to troubleshoot the errors that I get when I set up auto remediation for non-compliant Amazon Simple Storage Service (Amazon S3) resources.Short descriptionUsing auto remediation to address non-compliant Amazon S3 buckets can generate errors. The following types of errors may occur when you use the AWS SSM Automation document AWS-ConfigureS3BucketLogging with the AWS Config managed rule s3-bucket-logging-enabled:AWS Config console error “Action execution failed (details)."AWS Systems Manager console error "Step fails when it is Execute/Cancelling action. An error occurred (MalformedXML) when calling the PutBucketLogging operation: The XML you provided was not well-formed or did not validate against our published schema. Please refer to the Automation Service Troubleshooting Guide for more diagnosis details."AWS CloudTrail event PutBucketLogging error "The XML you provided was not well-formed or did not validate against our published schema."Remediation fails when an Amazon S3 bucket, configured as a target bucket to receive server access logging, does not allow the Log Delivery group write access.ResolutionGrant the Amazon S3 Log Delivery group write access in the target bucket's access control list (ACL). See How do I set ACL bucket permissions? for more information.Related informationHow do I enable server access logging for an S3 bucket?How can I be notified when an AWS resource is non-compliant using AWS Config?How can I troubleshoot AWS Config console error messages?Follow"


Question: How can I create queues on my Amazon EMR YARN CapacityScheduler?

How do I create queues on my Amazon EMR Hadoop YARN CapacityScheduler?

Answer: "How do I create queues on my Amazon EMR Hadoop YARN CapacityScheduler?Short descriptionEMR clusters have a single queue by default. You can add additional queues to your cluster and allocate available cluster resource capacity to your new queues.ResolutionCreate a reconfiguration commandThe following example reconfiguration does the following:Creates two additional queues, alpha and beta.Allocates 30% of the total resource capacity of your cluster to each of the new queues. When adding queues and allocating cluster capacity, the sum of capacities for all queues must be equal to 100. So, in the following example reconfiguration, the capacity of the default queue decreases to 40%.Provides full access (designated by the "*" label) to both queues. This allows both queues to access labeled core nodes.To submit to particular queue specify the queue in the yarn.scheduler.capacity.queue-mappings parameter. This parameter maps users to a queue with the same name as the user. The parent queue name must be the same as the primary group of the user, such as u:user:primary_group.user. In the following example, the parameter is set to u:hadoop:alpha. This maps to the newly created queue alpha.Note: The capacity for each queue’s access to the core label matches the capacity of the queue itself. So, the core partition splits between queues at the same ratio as the rest of the cluster.- Classification: capacity-scheduler  Properties:    yarn.scheduler.capacity.root.queues: 'default,alpha,beta'    yarn.scheduler.capacity.root.default.capacity: '40'    yarn.scheduler.capacity.root.default.accessible-node-labels.CORE.capacity: '40'    yarn.scheduler.capacity.root.alpha.capacity: '30'    yarn.scheduler.capacity.root.alpha.accessible-node-labels: '*'    yarn.scheduler.capacity.root.alpha.accessible-node-labels.CORE.capacity: '30'    yarn.scheduler.capacity.root.beta.capacity: '30'    yarn.scheduler.capacity.root.beta.accessible-node-labels: '*'    yarn.scheduler.capacity.root.beta.accessible-node-labels.CORE.capacity: '30'- classification: yarn-site  properties:    yarn.scheduler.capacity.queue-mappings: 'u:hadoop:alpha'  configurations: []Note: If you want to override the default queue mapping settings, set parameter yarn.scheduler.capacity.queue-mappings-override.enable to true. By default, this parameter is set to false. When set to true, users can submit jobs to queues other than the designated queue. For more information, see Enable override of default queue mappings on the Hortonworks Docs website.Verify your modificationsAccess the YARN ResourceManager Web UI to verify that your modifications have taken place.The following is an example of a Spark job submitted on Amazon EMR 6.4.0 that has the preceding example reconfiguration:spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster --conf spark.driver.memoryOverhead=512 --conf spark.executor.memoryOverhead=512 /usr/lib/spark/examples/jars/spark-examples.jar 100SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/usr/lib/spark/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class].........22/11/29 07:58:07 INFO Client: Application report for application_1669707794547_0001 (state: ACCEPTED)22/11/29 07:58:08 INFO Client: Application report for application_1669707794547_0001 (state: RUNNING)This application application_1669707794547_0001 is submitted to queue "alpha"Related informationHadoop: Capacity Scheduler on the Apache Hadoop websiteConfigure Hadoop YARN CapacityScheduler on Amazon EMR on Amazon EC2 for multi-tenant heterogeneous workloadsFollow"


Question: How can I configure an AWS Glue ETL job to output larger files?

I want to configure an AWS Glue ETL job to output a small number of large files instead of a large number of small files.

Answer: "I want to configure an AWS Glue ETL job to output a small number of large files instead of a large number of small files.ResolutionUse any of the following methods to reduce the number of output files for an AWS Glue ETL job.Increase the value of the groupSize parameterGrouping is automatically enabled when you use dynamic frames and when the Amazon Simple Storage Service (Amazon S3) dataset has more than 50,000 files. Increase this value to create fewer, larger output files. For more information, see Reading input files in larger groups.In the following example, groupSize is set to 10485760 bytes (10 MB):dyf = glueContext.create_dynamic_frame_from_options("s3", {'paths': ["s3://awsexamplebucket/"], 'groupFiles': 'inPartition', 'groupSize': '10485760'}, format="json")Note: The groupSize and groupFiles parameters are supported only in the following data formats: csv, ion, grokLog, json, and xml. This option is not supported for avro, parquet, and orc.Use coalesce(N) or repartition(N)1.    (Optional) Calculate your target number of partitions (N) based on the input data set size. Use the following formula:targetNumPartitions = 1 Gb * 1000 Mb/10 Mb = 100Note: In this example, the input size is 1 GB, and the target output is 10 MB. This calculation allows you to control the size of your output file .2.    Check the current number of partitions using the following code:currentNumPartitions = dynamic_frame.getNumPartitions()Note: When you repartition, targetNumPartitions should be smaller than currentNumPartitions.   Use an Apache Spark coalesce() operation to reduce the number of Spark output partitions before writing to Amazon S3. This reduces the number of output files. For example:dynamic_frame_with_less_partitions=dynamic_frame.coalesce(targetNumPartitions)Keep in mind:coalesce() performs Spark data shuffles, which can significantly increase the job run time.If you specify a small number of partitions, then the job might fail. For example, if you run coalesce(1), Spark tries to put all data into a single partition. This can lead to disk space issues.You can also use repartition() to decrease the number of partitions. However, repartition() reshuffles all data. The coalesce() operation uses existing partitions to minimize the number of data shuffles. For more information on using repartition(), see Spark Repartition on the eduCBA website.Use maxRecordsPerFileUse the Spark write() method to control the maximum record count per file. The following example sets the maximum record count to 20:df.write.option("compression", "gzip").option("maxRecordsPerFile",20).json(s3_path)Note: The maxRecordsPerFile option acts only as an upper limit for the record count per file. The record count of each file will be less than or equal to the number specified. If the value is zero or negative, then there is no limit.Related informationFix the processing of multiple files using groupingFollow"


Question: My Amazon S3 bucket has data files created using the UNLOAD command from an Amazon Redshift cluster in another account. Why can't I access those files?

"My Amazon Simple Storage Service (Amazon S3) bucket has data files created using the UNLOAD command from an Amazon Redshift cluster in another AWS account. However, I'm getting 403 Access Denied errors when I try to access those files from my own account. How can I fix this?"

Answer: "My Amazon Simple Storage Service (Amazon S3) bucket has data files created using the UNLOAD command from an Amazon Redshift cluster in another AWS account. However, I'm getting 403 Access Denied errors when I try to access those files from my own account. How can I fix this?Short descriptionBy default, an S3 object is owned by the AWS account that uploaded it. This is true even when the bucket is owned by another account. Therefore, when Amazon Redshift data files are put into your bucket by another account, you don't have default permission for those files.To get access to the data files, an AWS Identity and Access Management (IAM) role with cross-account permissions must run the UNLOAD command again. Follow these steps to set up the Amazon Redshift cluster with cross-account permissions to the bucket:1.    From the account of the S3 bucket, create an IAM role with permissions to the bucket. This is the bucket role.2.    From the account of the Amazon Redshift cluster, create another IAM role with permissions to assume the bucket role. This is the cluster role.3.    Update the bucket role to grant bucket access, and then create a trust relationship with the cluster role.4.    From the Amazon Redshift cluster, run the UNLOAD command using the cluster role and bucket role.Important: This resolution doesn't apply to Amazon Redshift clusters or S3 buckets that use server-side encryption with AWS Key Management Service (AWS KMS).ResolutionCreate a bucket roleFrom the account of the S3 bucket, create an IAM role with permissions to the bucket:1.    From the account of the S3 bucket, open the IAM console.2.    Create an IAM role. As you create the role, select the following:For Select type of trusted entity, choose AWS service.For Choose the service that will use this role, choose Redshift.For Select your use case, choose Redshift - Customizable.3.    After you create the IAM role, attach a policy that grants permission to the bucket. You can use a policy that's similar to the following:{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "Stmt1234537676482",      "Action": [        "s3:ListBucket",        "s3:PutObject"      ],      "Effect": "Allow",      "Resource": [        "arn:aws:s3:::awsexamplebucket/*",        "arn:aws:s3:::awsexamplebucket"      ]    }  ]}4.    Get the bucket role's Amazon Resource Name (ARN). You need the role's ARN for a later step.Create a cluster roleFrom the account of the Amazon Redshift cluster, create another IAM role with permissions to assume the bucket role:1.    From the account of the Amazon Redshift cluster, open the IAM console.2.    Create an IAM role. As you create the role, select the following:For Select type of trusted entity, choose AWS service.For Choose the service that will use this role, choose Redshift.For Select your use case, choose Redshift - Customizable.3.    After you create the IAM role, attach the following policy to the role:Important: Replace arn:aws:iam::123456789012:role/Bucket_Role with the ARN of the Bucket Role that you created.{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "Stmt1234537501110",      "Action": [        "sts:AssumeRole"      ],      "Effect": "Allow",      "Resource": "arn:aws:iam::123456789012:role/Bucket_Role"    }  ]}4.     Get the cluster role's ARN. You need the role's ARN for a later step.Update the bucket role to create a trust relationship with the cluster role1.    From the account of the S3 bucket, open the IAM console.2.    In the navigation pane, choose Roles.3.    From the list of roles, open the Bucket Role that you created.4.    Choose the Trust relationships tab.5.    Choose Edit trust relationship.6.    For the Policy Document, replace the existing policy with the following:Important: Replace arn:aws:iam::012345678901:role/Cluster_Role with the ARN of the Cluster Role that you created.{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "AWS": "arn:aws:iam::012345678901:role/Cluster_Role"      },      "Action": "sts:AssumeRole"    }  ]}7.    Choose Update Trust Policy.From the Amazon Redshift cluster, run the unload operation using the Cluster Role and Bucket Role1.    Connect to the Amazon Redshift cluster.2.    Run the UNLOAD command with both the IAM roles that you created, similar to the following:Important: Replace arn:aws:iam::012345678901:role/Cluster_Role with the ARN of your Cluster Role. Then, replace arn:aws:iam::123456789012:role/Bucket_Role with the ARN of your Bucket Role.unload ('select * from TABLE_NAME')to 's3://awsexamplebucket' iam_role 'arn:aws:iam::012345678901:role/Cluster_Role,arn:aws:iam::123456789012:role/Bucket_Role';After you run the UNLOAD command, the data files are owned by the same account as the bucket that they're stored in.Related informationUNLOAD ExamplesFollow"


Question: How do I fix a compute environment that's not valid in AWS Batch?

My compute environment in AWS Batch is in the INVALID state. How do troubleshoot the error?

Answer: "My compute environment in AWS Batch is in the INVALID state. How do troubleshoot the error?Short descriptionYou receive the error: "CLIENT_ERROR - Your compute environment has been INVALIDATED and scaled down because none of the instances joined the underlying ECS Cluster. Common issues preventing instances joining are the following: VPC/Subnet configuration preventing communication to ECS, incorrect Instance Profile policy preventing authorization to ECS, or customized AMI or LaunchTemplate configurations affecting ECS agent."Issues preventing your instances from joining an Amazon Elastic Container Service (Amazon ECS) cluster include:Amazon Virtual Private Cloud (Amazon VPC) subnet configuration settings are preventing successful communication to Amazon ECS.An incorrect setting within the instance profile policy preventing authorization to Amazon ECS.Customized Amazon Machine Images (AMI) or launch template configurations affecting the ECS agent.The CLIENT_ERROR message indicates that the Amazon Elastic Compute Cloud (Amazon EC2) instances created by the AWS Batch compute environment have failed to join the ECS cluster. When the CLIENT_ERROR message occurs, AWS Batch automatically terminates the EC2 instance and then moves the compute environment into an INVALID state.If your compute environment is in the INVALID state, choose one of the following resolutions based on the error message that you receive:CLIENT_ERROR - Not authorized to perform sts:AssumeRoleComplete the steps in the Fix a service role that's not valid section.CLIENT_ERROR - Parameter: SpotFleetRequestConfig.IamFleetRole is invalidComplete the steps in the Fix a Spot Fleet role that's not valid section.CLIENT_ERROR - The specified launch template, with template ID [xxx], does not existComplete the steps in the Deactivate and delete your compute environment section.CLIENT_ERROR - Access deniedCreate a service role with the correct permissions or choose an existing service role with the correct permissions.Internal ErrorComplete the steps in the Deactivate and then activate your compute environment section.INVALID CLIENT_ERROR - nullComplete the steps in the Deactivate and then activate your compute environment section.CLIENT_ERROR - The request uses the same client token as previous, but non-identical requestComplete the steps in the Deactivate and then activate your compute environment section.CLIENT_ERROR - You are not authorized to use launch templateCheck the following:Review your Service Role to see if permissions related to Amazon Elastic Compute Cloud and Auto Scaling groups are granted. Then, complete the steps in the Fix a service role that's not valid section.Review if your account is part of AWS Organizations and if any service control policies are blocking access to your Amazon EC2 permissions. Then, update any service control policies, if needed.ResolutionFix a service role that's not valid1.    Open the AWS Batch console.2.    In the navigation pane, choose Compute environments.3.    Choose the compute environment that's in the INVALID state.Note: If your compute environment is in the DISABLED state, choose Enable to activate your compute environment.4.    Choose Edit.5.    For Service role, choose a service role with the permissions needed for AWS Batch to make calls to other AWS services.Important: Your service role manages the resources that you use with the service. Before you can use the service, you must have an AWS Identity and Access Management (IAM) policy and role that provides the necessary permissions to AWS Batch. You must create a service role with permissions if you don't have one.6.    Choose Save.Fix a Spot Fleet role that's not validFor managed compute environments that use Amazon EC2 Spot Fleet Instances, you must create a role that grants the Spot Fleet the following permissions:Bidding on instancesLaunching instancesTagging instancesTerminating instancesIf you don't have a Spot Fleet role, complete the following steps to create one for your compute environment:1.    Open the IAM console.2.    In the navigation pane, choose Roles.3.    Choose Create role.4.    Choose AWS service. Then, choose EC2 as the service that will use the role that you're creating.5.    In the Select your use case section, choose EC2 Spot Fleet Role.Important: Don't choose the similarly named EC2 - Spot Fleet.6.    Choose Next: Permissions.7.    Choose Next: Tags. Then, choose Next: Review.8.    For Role name, enter AmazonEC2SpotFleetRole.9.    Choose Create role.Note: Use your new Spot Fleet role to create new compute environments. Existing compute environments can't change Spot Fleet roles. To get rid of the obsolete environment, deactivate and then delete that environment.10.    Open the AWS Batch console.11.    In the navigation pane, choose Compute environments.12.    Choose the compute environment that's in the INVALID state. Then, choose Disable.13.    Choose Delete.Deactivate and delete your compute environmentYou must deactivate and delete your compute environment because the launch template associated with your compute environment doesn't exist. This means that you can't use the compute environment associated with your launch template. You must delete that compute environment, and then create a new compute environment.1.    Open the AWS Batch console.2.    In the navigation pane, choose Compute environments.3.    Select the compute environment that's in the INVALID state. Then, choose Disable.4.    Choose Delete.5.    Create a new compute environment.Deactivate and then activate your compute environment1.    Open the AWS Batch console.2.    In the navigation pane, choose Compute environments.3.    Choose the compute environment that's in the INVALID state. Then, choose Disable.4.    Choose the same compute environment from step 3. Then, choose Enable.Related informationTroubleshooting AWS BatchWhy is my Amazon ECS or Amazon EC2 instance unable to join the cluster?Why is my AWS Batch job stuck in RUNNABLE status?Follow"