Question: How can I migrate my Amazon DynamoDB tables from one AWS account to another?

I want to perform a cross-account Amazon DynamoDB table migration.

Answer: "I want to perform a cross-account Amazon DynamoDB table migration.Short descriptionYou can migrate your DynamoDB tables to a different AWS account by choosing one of the following methods depending on your use case:AWS BackupDynamoDB import and export to Amazon Simple Storage Service (Amazon S3)Amazon S3 and AWS GlueAWS Data PipelineAmazon EMRResolutionAWS BackupYou can use AWS Backup to create cross-account DynamoDB backups. For more information, see Creating backup copies across AWS accounts and AWS Backup Demo: Cross-Account & Cross-Region Backup.In the target account, do the following:Create an AWS Backup vault in the target account in the Region where your DynamoDB is present.When you create the vault, use the AWS Key Management Service (AWS KMS) key that you already configured. This is the key that you already shared with the source account in the same organization.After you create the vault, add an AWS Identity and Access Management (IAM) policy to the vault that allows other accounts within the same organization to copy into the vault. You can do by selecting the option Allow access to a backup vault from organization.In the source account, do the following:In the source account where your DynamoDB is present, create an AWS Backup vault in the Region where your table data needs to be migrated.When you create the vault, use the AWS KMS key that you already configured. This is the key that you shared with other accounts in the organization.Add an IAM policy to the vault that allows other account in the organization to copy into the vault. You can do by selecting the option You can do by selecting the option Allow access to a backup vault from organization.Create a backup plan to generate backups of the DynamoDB tables in the source account to the target account.For Backup vault, be sure to choose the vault that you created in the source account.Then, select the Copy to another account's vault option.For Assign resources, be sure to include the resources that you need to back up. You can choose Include specific resource types.For Select specific resource types, select DynamoDB. You can choose all tables or only those tables that you need to back up.In the target account, do the following:In the target account, navigate to the vault that you created.You can see that the Recovery points are the same as in the source account.You can restore your DynamoDB table in the target account.Note: For this approach, the source and target accounts must be in the same organization.DynamoDB import and export to Amazon S3Migrate the DynamoDB table data by exporting the table to an Amazon S3 bucket in the target account. Be sure that DynamoDB has s3:ListBucket permissions to this S3 bucket. Be sure that the S3 bucket doesn't have any access control lists that deny access to the exported data.After the export is completed, import the data from the S3 bucket into a new table in the target account. For more information, see DynamoDB data import from Amazon S3: how it works.Note: Exporting the table doesn't consume any read capacity on the table and has no impact on table performance or availability. Also, importing the table doesn't consume any writer capacity. Therefore, you don't need additional capacity during the import process.Amazon S3 and AWS GlueYou can migrate your DynamoDB table to a different AWS account using an S3 bucket and an AWS Glue job.You can perform the initial migration of the DynamoDB table by exporting the tables to an Amazon S3 bucket in the other account.When you export your tables from Account A to an S3 bucket in Account B, the objects are still owned by Account A. The AWS Identify Access Management (IAM) users in Account B can't access the objects by default. The export function doesn't write data with the access control list (ACL) bucket-owner-full-control. As a workaround to this object ownership issue, include the PutObjectAcl permission on all exported objects after the export is complete. This workaround grants access to all exported objects for the bucket owners in Account B. For more information, see Why can't I access an object that was uploaded to my Amazon S3 bucket by another AWS account?Use a Glue job to read the files from the S3 bucket and write them to the target DynamoDB table. After you export the data to an S3 bucket in the target account as mentioned in step 1, do the following in the target account:Run an AWS Glue crawler on the data in Amazon S3. The crawler infers the schema and creates an AWS Glue Data Catalog table with that schema definition.Use the AWS Glue Studio to create an ETL job. After you specify a source, transformation, and target, AWS Glue Studio automatically generates the PySpark code based on these inputs. For this job, specify the AWS Glue Data Catalog table as the source and ApplyMApplyMapping as the transformation. Don't specify a target. AWS Glue Studio generates the PySpark code to create a DynamicFrame from S3.Be sure that the key name and data type mapping of the code generated by AWS Glue Studio are correct. If the mappings aren't correct, then modify the code, and correct the mappings. Because you didn't specify the target when creating the AWS Glue job, add a sink operation similar to the following in this example. Adding this operation allows the job to write directly to the target DynamoDB table:glueContext.write_dynamic_frame_from_options ( frame = Mapped, connection_type = "dynamodb", connection_options = { "dynamodb.region": "", "dynamodb.output.tableName": "", "dynamodb.throughput.write.percent": "1.0" } )To load the data to the target table, run the job from AWS Glue Studio or the Jobs page on the AWS Glue console.After exporting the tables to the Amazon S3 bucket, use DynamoDB Streams and AWS Lambda to migrate data insertions and updates in the source table to the destination table in another account. For more information, see Cross-account replication with Amazon DynamoDB.AWS GlueAWS Glue ETL jobs support both reading data from and writing data into another account's DynamoDB table. Use the dynamodb.sts.roleArn parameter to assume a cross-account role in the job script. Assuming this role allows you to get the temporary credentials that must be used for cross-account access to DynamoDB. For more information, see Cross-account cross-Region access to DynamoDB tables and How to export an Amazon DynamoDB table to Amazon S3 using AWS Step Functions and AWS Glue.Data PipelineTo move a DynamoDB table to different account using Data Pipeline, you can also do the following:Export the DynamoDB table from the source account to Amazon S3 in the destination account.Import the exported data from the S3 bucket.For more information, see How can I use Data Pipeline to back up a DynamoDB table to an S3 bucket that's in a different account? and How to migrate Amazon DynamoDB tables from one AWS account to another with AWS Data Pipeline.Note: The destination account can't access the DynamoDB data in S3 bucket. To work with the data, restore it to a DynamoDB table. Data Pipeline provides the easiest method to move the table with the least manual effort. However, there are fewer options for customization.Amazon EMRWhen you use Amazon EMR to migrate DynamoDB tables, use one of the following options depending on your use case:If you can afford downtime during the migration, then stop write operations to the source table. This is to make sure that the target table is in sync with the source table.If you can't afford downtime, then you must store all transactions that happen during the migration in a staging table. After the original table is migrated to the other AWS account, push the new transactions from the staging table to the target table.Note: The time required to migrate tables with Amazon EMR can vary significantly depending on the DynamoDB table's provisioned throughput network performance, and the amount of data stored in the table.To migrate a DynamoDB table using Amazon EMR, do the following:Launch EMR clusters in both the source and destination accounts. In the Software configuration section, be sure that you choose an option that includes Apache Hive. Note: It's a security best practice to launch Amazon EMR clusters into private subnets. The private subnets must have an Amazon S3 VPC endpoint and a route to DynamoDB. For more information, see Private subnets. If the clusters need to access the internet, then use a NAT gateway that resides in a public subnet. For more information, see VPC with public and private subnets (NAT).Be sure that the EMR_EC2_DefaultRole IAM roles in both accounts have permission to write to the S3 bucket in the destination account. For more information, see Configure IAM service roles for Amazon EMR permissions to AWS services and resources.In the source account, connect to the master node using SSH.In the source account, use Hive commands to export the DynamoDB table data to the S3 bucket in the destination account.In the destination account, import the Amazon S3 data to the new DynamoDB table.If you're using a staging table to capture writes that happened during the migration, repeat steps 4 and 5 on the staging table.Related informationCopying data between DynamoDB and Amazon S3Follow"


Question: How do I troubleshoot common errors with API calls in Amazon ECS?

I want to troubleshoot common errors with API calls in Amazon Elastic Container Service (Amazon ECS).

Answer: "I want to troubleshoot common errors with API calls in Amazon Elastic Container Service (Amazon ECS).Short descriptionThe Amazon ECS APIs might fail with one of the following errors:AccessDeniedExceptionClientExceptionClusterNotFoundExceptionInvalidParameterExceptionServerExceptionServiceNotActiveExceptionPlatformTaskDefinitionIncompatibilityExceptionPlatformUnknownExceptionServiceNotFoundExceptionUnsupportedFeatureExceptionYou might also experience API issues with the application that's running inside your Amazon ECS tasksResolutionYour API requests are recorded in AWS CloudTrail as events. When activity occurs in Amazon ECS, that activity is recorded in a CloudTrail event along with other AWS service events in Event history. You can view, search, and download recent events in your AWS account.To view the CloudTrail event history and locate the API errors, do the following:Open the AWS CloudTrail console.In the navigation pane, choose Event history.Choose the gear icon.Under Select visible columns, select Error code.Choose Confirm.In the Event history page, for Lookup attributes, select Event name.For Enter an event name, enter the action that failed.Note: If you don't know the event name, then do the following:For Lookup attributes, select Event source.For Enter an event source, select ecs.amazonaws.com to filter all events related to your ECS service.From the list of results, choose the events with error codes of your choice to view the event details.AccessDeniedExceptionThis error is logged when the AWS Identity and Access Management (IAM) user or role making the API call doesn’t have the required permissions to perform the requested action.The AccessDeniedException error looks similar to the following:An error occurred (AccessDeniedException) when calling the CreateCluster operation: User: arn:aws:sts::123456789012:assumed-role/test-role/test-session is not authorized to perform: ecs:CreateCluster on resource: * because no identity-based policy allows the ecs:CreateCluster actionYou can view the following details in the related CloudTrail event record:User information:"type": "AssumedRole","principalId": "AROAZEPPWYLQU45ZDJY6V:test-session","arn": "arn:aws:sts::123456789012:assumed-role/test-role/test-session"Event name:"eventName": "CreateCluster"Error message:"errorMessage": "User: arn:aws:sts::123456789012:assumed-role/test-role/test-session is not authorized to perform: ecs:CreateCluster on resource: * because no identity-based policy allows the ecs:CreateCluster action"To test a policy that is not attached to a user, user group, or role, use the IAM policy simulator.To resolve this error, do the following:Open the IAM console.In the navigation pane, choose Roles or Users depending on the user identity.Filter the role or user using the search filter.Choose the role or user.Choose the Permissions tab.Expand the permissions policy to view the permissions associated with the user.Be sure that the policy includes ecs:your-event-name in the Actions list and Allow for Effect. If the policy doesn't include these parameters, then update the policy to include these changes. Or, create a new policy allowing the mentioned action and attach the policy to the IAM role or user. For more information, Editing customer managed policies (console).ClientExceptionThis error is logged when the ECS client specifies an identifier or resource that isn't valid or doesn't exist. For example, if you try to start a task using the RunTask or StartTask API and refer an incorrect task definition, you get this error:$ aws ecs run-task --cluster example-cluster --task-definition centos --region ap-southeast-2An error occurred (ClientException) when calling the RunTask operation: TaskDefinition not found.$ aws ecs start-task --cluster example-cluster --task-definition centos --container-instances 765936fadbdd46b5991a4bd70c2a43d4 --region ap-southeast-2An error occurred (ClientException) when calling the StartTask operation: TaskDefinition not found.To prevent this error, be sure that the resources referred in the command, your code, or API calls exist and are valid.ClusterNotFoundExceptionThis error is logged when the specified cluster isn't found.Example:$ aws ecs run-task --task-definition CentOS --cluster example-cluster --region ap-southeast-2An error occurred (ClusterNotFoundException) when calling the StartTask operation: Cluster not found.To avoid this error, be sure that the cluster name that you pass in the command, your code, or API calls is correct. You can run the following command to list the existing ECS clusters. With the list that's returned, you can verify that the cluster mentioned in the API call exists.$ aws ecs list-clusters --region example-region{    "clusterArns": [        "arn:aws:ecs:ap-southeast-2:123456789012:cluster/my-cluster",        "arn:aws:ecs:ap-southeast-2:123456789012:cluster/my-private-cluster"    ]}InvalidParameterExceptionThis error is logged when the parameter passed in the command isn't valid. Suppose that you mentioned a version of the task definition that doesn't exist:$ aws ecs run-task --task-definition CentOS:3 --cluster example-cluster --region ap-southeast-2Then, the error looks similar to the following:An error occurred (InvalidParameterException) when calling the RunTask operation: TaskDefinition not found.To avoid this error, be sure that the parameters passed in the command are valid.ServerExceptionThis error is logged when there is a server error related to the API call. ServerException is usually caused due to HTTP error code 500. This exception occurs when there is an issue with the ECS service in the AWS Region. This error is usually temporary and subsequent attempts to run the API should be successful. However, if the issue persists for a long time, contact AWS Support.ServiceNotActiveExceptionThis error occurs when the ECS service that's being updated isn't active. Verify that the ECS service that's being updated is present in the ECS cluster and is in active state.Run the following AWS Command Line Interface (AWS CLI) command to list all the services in the cluster:$ aws ecs list-services --cluster example-clusterIn the output, verify whether the service that's being updated is displayed.Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Then, run the following command to verify that the service is in the active state.$ aws ecs describe-services --services example-service-name --cluster example-clusterThe output might look similar to the following:{    "services": [{        "serviceArn": "arn:aws:ecs:ap-southeast-2:111122223333:service/my-cluster/example-service",        "serviceName": "example-service",        "clusterArn": "arn:aws:ecs:ap-southeast-2:111122223333:cluster/example-cluster",        "loadBalancers": [],        "serviceRegistries": [],        "status": "ACTIVE",        ......    }]}PlatformTaskDefinitionIncompatibilityExceptionThis error occurs when a task is launched on a platform that doesn’t meet the capabilities required in the task definition. Suppose that you try to create a service with an Amazon EFS volume attached on platform version 1.3.0:$ aws ecs create-service \    --cluster example-cluster \    --task-definition Test-fargate-EFS \    --launch-type FARGATE \    --service-name example-service \    --desired-count 1 \    --network-configuration="awsvpcConfiguration={subnets=["subnet-ed7d31b5","subnet-833ef1cb"],securityGroups=["sg-eeb28aa1"]}" \    --platform-version 1.3.0Then, you get the following error:An error occurred (PlatformTaskDefinitionIncompatibilityException) when calling the CreateService operation: One or more of the requested capabilities are not supported.To resolve this issue, be sure to use the platform version that supports the capability requirements in the task definition. For information about supported capabilities in the various platform versions, see AWS Fargate platform versions.PlatformUnknownExceptionThis error occurs if you specify an unknown or wrong platform version when you launch a task. Suppose that you provide an incorrect platform version 1.3 instead of version 1.3.0:$ aws ecs create-service \    --cluster example-cluster\    --task-definition example-task \    --launch-type FARGATE\    --enable-execute-command \    --service-name example-service\    --desired-count 1 \    --network-configuration="awsvpcConfiguration={subnets=["subnet-ed7d31b5","subnet-833ef1cb"],securityGroups=["sg-eeb28aa1"]}"\    --platform-version 1.3Then, you get the following error:An error occurred (PlatformUnknownException) when calling the CreateService operation: The specified platform does not exist.For more information, see Linux platform versions and Windows platform versions.ServiceNotFoundExceptionThis error occurs when the ECS service that's specified in your command or code doesn’t exist. Verify that the service name in your command or code is correct and the service is present in the cluster.To view all the services in the cluster, run the following command:$ aws ecs list-services --cluster example-clusterUnsupportedFeatureExceptionThis error occurs when an ECS feature isn't available in a specific Region. For example, the AWS Fargate feature might not be immediately available in a newly launched Region. If a Fargate task is launched in this Region, then you get the UnsupportedFeatureException error.Application API issuesThe following are some of the most commonly seen HTTP 5xx errors that you might get when you access the application hosted inside an ECS task:500 - Internal Server Error: You get this error when the application encounters an unexpected condition. This error might occur due to application misconfiguration or an error with the application.503 - Service Unavailable: You get this error under the following conditions:The ECS task is experiencing heavy workload and unable to service the request.The application running inside your task is down for maintenance.To troubleshoot these errors, do the following:Analyze the application logs for the ECS tasks in Amazon CloudWatch Logs. You can find information about the log group from the task definition. Each task is associated with an individual long stream that contains the application logs from the task.To view the log group and log stream for your task, run the following command:$ aws ecs describe-task-definition —task-definition example-taskdefinitionThe output looks similar to the following:...                "logConfiguration": {                    "logDriver": "awslogs",                    "options": {                        "awslogs-group": "/ecs/example-task",                        "awslogs-region": "ap-southeast-2",                        "awslogs-stream-prefix": "ecs"                    }                }...Related informationAPI failure reasonsFollow"


Question: How can I troubleshoot Route 53 private hosted zone DNS resolution issues?

"I created a private hosted zone for my domain in Amazon Route 53. However, DNS isn't working in my virtual private cloud (VPC)."

Answer: "I created a private hosted zone for my domain in Amazon Route 53. However, DNS isn't working in my virtual private cloud (VPC).ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Confirm that the correct VPC ID is associated with the private hosted zone. Use the AWS CLI command get-hosted-zone to get a list of VPCs associated with your hosted zone. Or, use list-hosted-zone-by-vpc to get a list of private hosted zones associated with a specific VPC.Note: Make sure that you're querying the domain from within the same VPC.Confirm that the DNS hostnames and DNSSupport parameters are set to true in your VPC. To do this, check your VPC settings.Check the VPC settings to validate that you configured custom DNS servers in the DHCP option set. If configured, confirm that the DNS servers are set to forward DNS queries for the private domain to the Amazon-provided DNS servers of your VPC. For example, if the CIDR range for your VPC is 10.0.0.0/16, then the IP address of the VPC DNS server is 10.0.0.2. The IP address is the VPC network range plus two.Note: Private hosted zones are resolvable only through the VPC DNS.Review the resolver configuration settings of your source instance.For Linux instances, use the cat /etc/resolv.conf and cat/etc/hosts files.For macOS, see Change DNS Settings on Mac in the macOS user guide.For Windows, complete the following steps:Choose Start, Network & Internet Settings.Choose Change adaptor options under Advanced network settings.Right-click on the network connection, and then choose Properties.Choose IPv4 properties, and then enter the preferred DNS IP address in DNS server addresses.Example: You configure the Rotate option in resolv.conf to load balance DNS queries between an Amazon-provided DNS server and a public name server. This configuration causes intermittent DNS resolution. Queries that reach the VPC DNS resolver associated with your private hosted zone are resolved. Queries routed to the public name server fail.Check for multiple private hosted zones with overlapping namespaces such as example.com and test.example.com. If there are overlapping namespaces, then the Resolver routes traffic to the hosted zone based on the most specific match. If there's a matching zone but no record that matches the domain name and type, then Resolver returns NXDOMAIN (non-existent domain) to the client. The Resolver doesn't forward the request to another zone or public DNS resolver in this case. Confirm that the correct record is configured in the most specific private hosted zone for successful DNS resolution.Confirm that you don't have an NS record configured for the subdomain in the private hosted zone of the parent domain.Note: In a private hosted zone, name server (NS) records aren't supported for delegating the responsibility for a subdomain.Confirm that you configured a routing policy that's supported by a private hosted zone. The supported routing policies are:Simple routingFailover routingMultivalue answer routingWeighted routingLatency-based routingGeolocation routingCheck that you're using Resolver with an outbound endpoint. If both of the following conditions are true, then the Resolver rule takes precedence:You have a Resolver rule to route traffic to your network for your private hosted zone's domain.You have a Resolver rule associated to the same VPC that's also associated to the private hosted zone.For more information, see Resolving DNS queries between VPCs and your network.Confirm that the Resolver rule and inbound endpoint of the VPC aren't causing a query loop. You create a loop if you do following:Create target IP addresses in a Resolver forwarding rule point to inbound endpoints of a VPC.Associate the endpoints with the private hosted zone and,Associate the same resolver rule with the VPC.Confirm that the on-premises resolver sends a recursive request. Verify the resolution type using a packet capture on the DNS resolver (on-premises). Then, review the DNS flags (recursion desired = 0). You can also test the resolution by sending an iterative request using +norecurse with the dig command, or set "norecurse" with nslookup.Confirm that you configured the correct rule priorities for the Amazon-provided DNS. In general, the most specific rule takes priority. If the rules are at the same level, the priority is resolver rule, private hosted zone rule, and internal rule, respectively.Related informationWorking with private hosted zonesWhich Amazon VPC options do I need to activate to use my private hosted zone?Follow"


Question: How can I troubleshoot Amazon S3 endpoint connection test failures when using AWS DMS?

I'm using Amazon Simple Storage Service (Amazon S3) as the source or target endpoint for my AWS Database Migration Service (AWS DMS) task. I received an error message because my endpoint connection test failed.

Answer: "I'm using Amazon Simple Storage Service (Amazon S3) as the source or target endpoint for my AWS Database Migration Service (AWS DMS) task. I received an error message because my endpoint connection test failed.Short descriptionYou must have the right permissions configured for the AWS Identity and Access Management (IAM) role that is using the Amazon S3 endpoint. If you don't have the right permissions, then you see one of these entries:Message:Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to connect to S3 endpoint. Access denied.Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to list bucket wan-dms Failed to connect to database., Application-Detailed-Message: failed to list bucket wan-dms AWS failed to list bucket List bucket failed with exception 'AccessDenied', message 'Access Denied', error type '15' AWS failed to list bucket Not retriable error: Access DeniedTest Endpoint failed: Application-Status: 1020912, Application-Message: Failed to connect to database.Error Details: [message=putDatabase call failed, errType=ERROR_RESPONSE, status=1020414, errMessage= Failed to connect to database., errDetails=]ResolutionNote: When you use Amazon S3 as a source or target for AWS DMS, be sure to first review the following articles:Prerequisites when using Amazon S3 as a source.Prerequisites for using Amazon S3 as a target.To resolve these errors, grant the minimum IAM permissions that are required to access the Amazon S3 endpoint. Amazon S3 source and target endpoints have different minimum permissions for AWS DMS.See this example of an IAM policy for an Amazon S3 source endpoint:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Action": [        "s3:GetObject",        "s3:ListBucket"      ],      "Resource": [        "arn:aws:s3:::awsexamplebucket",        "arn:aws:s3:::awsexamplebucket/*"      ]    }  ]}Note: Replace awsexamplebucket with your own bucket name.See this example of an IAM policy for an Amazon S3 as target endpoint:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Action": [        "s3:PutObject",        "s3:DeleteObject",        "s3:PutObjectTagging"      ],      "Resource": [        "arn:aws:s3:::awsexamplebucket/*"      ]    },    {      "Effect": "Allow",      "Action": [        "s3:ListBucket"      ],      "Resource": [        "arn:aws:s3:::awsexamplebucket"      ]    }  ]}Note: Replace awsexamplebucket with your own bucket name.Troubleshoot permissions errors1.    Confirm that the IAM role has the minimum permissions required to access the Amazon S3 endpoint.2.    Confirm that the bucket name specified for the Amazon S3 endpoint matches the resource allowed in the IAM role. For example, the IAM role allows access to only one bucket, like awsexamplebucket in the previous example policy. But if awsexamplebucket2 is specified in the Bucket name of the Amazon S3 endpoint, then the test connection fails.3.    Confirm that dms.amazonaws.com is the trusted entity associated with the IAM role. For more information, see Editing the trust relationship for an existing role and this example trust policy for AWS DMS:{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "",      "Effect": "Allow",      "Principal": {        "Service": "dms.amazonaws.com"      },      "Action": "sts:AssumeRole"    }  ]}4.    If you use AWS Organizations, then confirm that the IAM role used is part of an organization that allows access to Amazon S3. If your organization doesn't allow access, then AWS DMS can't connect to the Amazon S3 endpoint using that IAM role. This is the case even if the role has all the required permissions. If your organization doesn't allow access, contact your account administrator to allow Amazon S3 access to your organization.5.    If your replication instance version is 3.4.7 or higher, then take one of these actions:        Make your replication instance subnets publicly routable. Add an Internet Gateway (IGW) route to the VPC that's used by your AWS DMS replication instance.        -or-        Create VPC endpoints so that your replication instance can access all source and target endpoints that are used by AWS DMS. For more information, see Preparing a migration to AWS DMS versions 3.4.7 and higher.Related informationHow can I troubleshoot AWS DMS endpoint connectivity failures?Follow"


Question: How can I grant public read access to some objects in my Amazon S3 bucket?

"I want some objects in my Amazon Simple Storage Service (Amazon S3) bucket to be publicly readable. However, I don't want to change the permissions on other objects that are in the same bucket. How can I do that?"

Answer: "I want some objects in my Amazon Simple Storage Service (Amazon S3) bucket to be publicly readable. However, I don't want to change the permissions on other objects that are in the same bucket. How can I do that?Short descriptionGrant public read access in one of these ways:Update the object's access control list (ACL) using the Amazon S3 consoleUpdate the object's ACL using the AWS Command Line Interface (AWS CLI)Use a bucket policy that grants public read access to a specific object tagUse a bucket policy that grants public read access to a specific prefixImportant: Granting public access through bucket and object ACLs doesn't work for buckets that have S3 Object Ownership set to Bucket Owner Enforced. In most cases, ACLs aren't required to grant permissions to objects and buckets. Instead, use AWS Identity Access and Management (IAM) policies and S3 bucket policies to grant permissions to objects and buckets.By default, new buckets, access points and objects don't allow public access. If block public access is activated for all buckets within the account, the message “Bucket and objects not public” is shown. To lean more, see Configuring block public access settings for your account.ResolutionImportant: Before you begin, confirm that you don't have any block public access settings at the account level or the bucket level. Your settings must not prevent you from making the objects public. By default, block public access settings are set to True on new S3 buckets.Update the object's ACL using the Amazon S3 consoleTo make several objects public at once, follow these steps:Warning: After you make several objects public, there's no option to undo this action for several objects at once. To remove public access, you must go into each object in the Amazon S3 console. Then, from the Permissions tab of the object, modify Public access. You must do this for every object where you want to undo the public access that you granted. Make sure to carefully review the list of objects before you make them public.1.    Open the Amazon S3 console.2.    From the list of buckets, choose the bucket with the objects that you want to update.3.    Navigate to the folder that contains the objects.4.    From the object list, select all the objects that you want to make public.5.    Choose Actions, and then choose Make public.6.    In the Make public dialog box, confirm that the list of objects is correct.7.    Choose Make public.To make an individual object public, you can repeat the previous process or follow these steps:1.    From the Amazon S3 console, choose the bucket with the object that you want to update.2.    Navigate to the folder that contains the object.3.    Open the object by choosing the link on the object name.4.    Choose the Permissions tab.5.    Choose Edit.6.    In the Everyone section, select Objects Read.7.    Select I understand the effects of these changes on this object.8.    Choose Save changes.Update the object's ACL using the AWS CLIFor an object that you've already stored in Amazon S3, you can run this command to update its ACL for public read access:aws s3api put-object-acl --bucket DOC-EXAMPLE-BUCKET --key exampleobject --acl public-readNote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.Or, you can run this command to grant full control of the object to the AWS account owner and read access to everyone else:Note: For the value of --grant-full-control, enter the account's canonical user ID.aws s3api put-object-acl --bucket DOC-EXAMPLE-BUCKET --key exampleobject --grant-full-control id="008exampleA45666666668889999008853" --grant-read uri=http://acs.amazonaws.com/groups/global/AllUsersUse a bucket policy that grants public read access to a specific object tagImportant: Before you begin, be sure to review the pricing for S3 Object Tagging.First, add a bucket policy that allows public read access to any objects with a specific tag. For example, this policy allows public read access for any object that's tagged with the key-value pair public=yes:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": "*",      "Action": "s3:GetObject",      "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*",      "Condition": {        "StringEquals": {          "s3:ExistingObjectTag/public": "yes"        }      }    }  ]}Then, add the tag to the objects that you want to be publicly readable. You can add or manage object tags by using the Amazon S3 console. Or, you can use the AWS CLI.To check if an object has any existing tags, run this AWS CLI command:aws s3api get-object-tagging --bucket DOC-EXAMPLE-BUCKET --key exampleobjectTo add a tag to an object that doesn't have any existing tags, run this command:Warning: This command overwrites any existing object tags.aws s3api put-object-tagging --bucket DOC-EXAMPLE-BUCKET --key exampleobject --tagging 'TagSet={Key=public,Value=yes}'To add a tag to an object that has existing tags, run the following command. Make sure to include the new object tag, as well as the existing tags that you want to keep.aws s3api put-object-tagging --bucket DOC-EXAMPLE-BUCKET --key exampleobject --tagging 'TagSet=[{Key=public,Value=n},{Key=exampletag1,Value=one},{Key=exampletag2,Value=two}]'After you add the object tag, run this command to review the tags of all the objects:aws s3api get-object-tagging --bucket DOC-EXAMPLE-BUCKET --key exampleobjectUse a bucket policy that grants public read access to a specific prefixWarning: The following bucket policy grants public read access to all objects under a specific prefix. Before you use this bucket policy, confirm that your use case supports all publicly readable objects within the prefix. This policy doesn't grant list access for the prefix. The user can access the object only if the object path is known. When accessing an object that doesn't exist in the prefix, the user receives a 403 error.To grant public read access to a specific object prefix, add a bucket policy similar to the following:{  "Version":"2012-10-17",  "Statement":[    {      "Sid":"AddPerm",      "Effect":"Allow",      "Principal": "*",      "Action":["s3:GetObject"],      "Resource":["arn:aws:s3:::DOC-EXAMPLE-BUCKET/publicprefix/*"]      }  ]}Then, copy the objects into the prefix with public read access. You can copy an object into the prefix by running a command similar to the following:aws s3 cp s3://DOC-EXAMPLE-BUCKET/exampleobject s3://DOC-EXAMPLE-BUCKET/publicprefix/exampleobjectNote: Depending on the object's prefix, copying the object isn't required to grant public read access.Related informationConfiguring ACLsFollow"


Question: How do I cancel my AWS Support plan?

I want to cancel my Developer or Business Support plan.

Answer: "I want to cancel my Developer or Business Support plan.ResolutionTo cancel your AWS Support plan, sign in to your AWS account as the root user. You can also use an AWS Identity and Access Management (IAM) user with access permissions for AWS Support Plans. For more information, see Manage access for AWS Support Plans.The Basic Support plan is the standard Support plan. This plan is free with all AWS accounts.To cancel a Developer or Business Support plan, switch your account back to the Basic Support plan by following these steps:Open the AWS Support Center. Or, in the AWS Management Console, choose the question mark icon, and then choose Support Center.In the navigation pane, choose Change next to your current AWS Support plan.Choose Review downgrade for the Basic Support plan.Important: If you have an Enterprise On-Ramp or Enterprise Support plan, then on the Change plan confirmation dialog box, choose Contact us. Next, fill in the form, and then choose Submit.For Change plan confirmation, expand the support items to see the features included with the plan.Choose Accept and agree.You'll receive a prorated refund for the remainder of your current month's AWS Support charge to the original payment method. This prorated refund is subject to your AWS Support plan's minimum monthly charge.If you have the Basic Support plan, you can't cancel it. To close your AWS account, see How do I close my AWS account?Related informationAWS SupportFollow"


Question: Why aren't Amazon S3 object-level API actions appearing in my CloudTrail Event history?

I activated AWS CloudTrail object-level event logging for an Amazon Simple Storage Service (Amazon S3) bucket. Why aren't Amazon S3 object-level API actions appearing in my CloudTrail Event history?

Answer: "I activated AWS CloudTrail object-level event logging for an Amazon Simple Storage Service (Amazon S3) bucket. Why aren't Amazon S3 object-level API actions appearing in my CloudTrail Event history?Short descriptionAmazon S3 object-level API actions are CloudTrail data events. By default, trails don't log data events, and data events aren't viewable in CloudTrail Event history.To view Amazon S3 object-level API actions after you've activated data event logging, you must query your CloudTrail logs.To query CloudTrail logs for object-level API actions, you can use either of the following AWS services and features:Amazon CloudWatch logs filter patternsAmazon Athena queriesFor more information, see Logging Amazon S3 API calls using AWS CloudTrail.Note: If you have other data sources configured for the same S3 bucket as your trail, then the logs are combined with CloudTrail events.ResolutionTo use CloudWatch logs filter patterns to view object-level API actionsFollow the instructions in Search log data using filter patterns.Example CloudWatch Logs filter syntax for the DeleteBucket API action{$.eventName = "DeleteBucket"}To use Athena queries to view object-level API actionsNote: To use Athena to query CloudTrail logs, you must have a trail configured to log to an Amazon S3 bucket. You can use Athena to query CloudTrail logs over the last 90 days.Follow the instructions in How do I automatically create tables in Amazon Athena to search through AWS CloudTrail logs?Example query for the GetBucketAcl API actionNote: Replace example-cloudtrail-log with your CloudTrail log's name.SELECT *FROM example-cloudtrail-logWHERE eventname = 'GetBucketAcl';For more information, see Understanding CloudTrail logs and Athena tables.Related informationCloudTrail tracking with Amazon S3 SOAP API callsHow do I analyze my Amazon S3 server access logs using Athena?Analyze security, compliance, and operational activity using AWS CloudTrail and Amazon AthenaFollow"


Question: How do I import existing resources into a nested stack whose parent is the root stack in CloudFormation?

"I have a nested stack in AWS CloudFormation whose parent stack is a root stack. I want to import an existing resource into the nested stack. But when I try to import the resource in the root stack, I get the error “This template does not include any resources to import.”"

Answer: "I have a nested stack in AWS CloudFormation whose parent stack is a root stack. I want to import an existing resource into the nested stack. But when I try to import the resource in the root stack, I get the error “This template does not include any resources to import.”Short descriptionWhen the nested stack's parent stack is the root, it's a best practice to modify only the root stack.Note: The following resolution is for importing a resource into a nested stack whose parent stack is the root stack.Important: Always run drift detection on stacks after an import operation to verify that the imported resource isn't drifted.ResolutionTo import resources into a nested stack, follow these steps:Update the root template with a Retain DeletionPolicy on the nested AWS::CloudFormation::Stack resource to detach the stack from the root stack without deleting it.Update the root stack by removing AWS::CloudFormation::Stack from the template.Note: After you update the root stack, the nested stack is still marked as Nested.Perform an import operation on the detached nested stack to import the existing resource.Note: Because the stack is still considered Nested, a warning appears that recommends you to update the root stack instead. Ignore it and continue to update the nested stack. Don't modify any other resources during the import.Move the detached nested stack out of IMPORT_COMPLETE status by updating it again with an easily removable change, such as adding outputs or tags.Perform an import operation on the root stack to add the nested stack back into its nested hierarchy.Note: Make sure that the root template includes a reference for the nested stack's updated template so that it contains the newly imported resource.Related informationCloudFormation Workshop: Nested stacks labFollow"


Question: How do I optimize the performance of my Amazon EBS volumes?

I want to make sure that I get optimal performance from my Amazon Elastic Block Store (Amazon EBS) volumes.

Answer: "I want to make sure that I get optimal performance from my Amazon Elastic Block Store (Amazon EBS) volumes.ResolutionMake sure that your Amazon Elastic Compute Cloud (Amazon EC2) instance is a type that can be optimized for use with Amazon EBS. An EBS-optimized EC2 instance provides dedicated network throughput for volumes. For a description of the instance types, see Amazon EBS-optimized instances - Supported instance types.Improve the performance of Amazon EBS volumes by doing the following:For best practices and performance tuning, see Amazon EBS volume performance on Linux instances.Initialize EBS volumes restored from a snapshot before you use them.Use AWS Trusted Advisor to get a list of best practices that you can use to improve the performance of your Amazon EBS volumes.Related informationAmazon Elastic Block Store (Amazon EBS)Amazon EBS-optimized instancesFollow"


Question: Why do I receive a MaxNumberOfConfigurationRecordersExceededException error when I create an AWS Config configuration recorder?

"When I try to create an AWS Config configuration recorder, I get a MaxNumberOfConfigurationRecordersExceededException error. How can I troubleshoot this issue?"

Answer: "When I try to create an AWS Config configuration recorder, I get a MaxNumberOfConfigurationRecordersExceededException error. How can I troubleshoot this issue?Short descriptionAWS Config uses a configuration recorder to record changes in your resource configuration. AWS Config currently allows only one configuration recorder per Region in an account. The MaxNumberOfConfigurationRecordersExceededException error indicates that you can’t create a new configuration recorder because one is already present on your account for the Region. The error occurs regardless of whether the recorder is created using the AWS Management Console, the AWS Command Line Interface (AWS CLI), or AWS CloudFormation.ResolutionTo resolve the MaxNumberOfConfigurationRecordersExceededException error and create a new configuration recorder, you must identify and then delete the existing configuration recorder.Note: It's not possible to delete a configuration recorder from the console. Deletions must be performed programmatically.First, verify that the correct AWS Identity and Access Management (IAM) permissions are in place to run the required commands. Then, you can identify and delete the existing configuration recorder using the AWS CLI or CloudFormation.IAM permissionsTo describe and delete a configuration recorder, add the following IAM permissions:{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": [                "config:DescribeConfigurationRecorders",                "config:DeleteConfigurationRecorder"            ],            "Resource": "*"        }    ]}To start and stop a configuration recorder, add the following IAM permissions:{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": [                "config:PutConfigurationRecorder",                "iam:PassRole",                "config:DescribeConfigurationRecorders",                "config:StopConfigurationRecorder",                "config:StartConfigurationRecorder",                "config:DeleteConfigurationRecorder"            ],            "Resource": "*"        }    ]}Identify and delete a configuration recorder (AWS CLI)Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Identify the existing configuration recorder nameRun the following command to identify the existing configuration recorder in a specific Region. Replace RegionID with your AWS Region.$aws configservice describe-configuration-recorders --region RegionIDReview the output. The existing configuration recorder name is listed next to “name” in the output.Delete the existing configuration recorderRun the following command to delete the existing configuration recorder. Replace RecorderName and RegionID with your values.$aws configservice delete-configuration-recorder --configuration-recorder-name RecorderName --region RegionIDNote: The delete-configuration-recorder command doesn't return an output when the recorder is successfully deleted.API calls to start and stop the recorderYou can use API calls if you need to start or stop a configuration recorder.Run the following command to start a configuration recorder. Replace RecorderName with the name of your configuration recorder.$aws configservice start-configuration-recorder --configuration-recorder-name RecorderNameRun the following command to stop a configuration recorder. Replace RecorderName with the name of your configuration recorder.$aws configservice stop-configuration-recorder --configuration-recorder-name RecorderNameIdentify and delete a configuration recorder (CloudFormation)If you enabled AWS Config using a CloudFormation StackSet template, then follow the guidance in this section to identify and delete a configuration recorder.You can use an AWS Lambda-backed custom resource to write code logic to identify and delete a configuration recorder.Use the DescribeConfigurationRecorders API call to identify the name of the existing recorder.Use the DeleteConfigurationRecorder API to delete the existing recorder.After the configuration recorder is deleted, you can use the cfn-response module for custom resources to continue creating a new configuration recorder.After the new configuration recorder is created, you can run the StartConfigurationRecorder API to start the recorder.Related informationdescribe-configuration-recordersdelete-configuration-recorderCloudFormation custom resource creation with Python, AWS Lambda, and crhelperFollow"


Question: How do I create an AppStream 2.0 image for Windows and deploy it to my fleet?

"I want to create an Amazon AppStream 2.0 image, and then deploy the image to my fleet."

Answer: "I want to create an Amazon AppStream 2.0 image, and then deploy the image to my fleet.ResolutionTo create an AppStream 2.0 image and deploy the image to your fleet, follow these steps:Launch an image builder from the AppStream 2.0 console.Create a custom image with the image builder.Launch a fleet. For Choose an Image, choose your custom image.Create a stack. From the Create Stack wizard, for Fleet, select your new fleet.Create a user, and then assign the user to a stack. You can provide access to users through the AppStream 2.0 user pool, SAML 2.0, or the AppStream 2.0 API. For more information, see User pool administration and Amazon AppStream 2.0 Integration with SAML 2.0.Note: If you use a fleet that's joined to an Active Directory domain, you must use SAML.Related informationWhat is Amazon AppStream 2.0?Amazon AppStream 2.0 resourcesFollow"


Question: Can I increase the IAM role chaining session duration limit?

"I used the AssumeRole API to assume an AWS Identity and Access Management (IAM) role using temporary credentials, but I received an error similar to the following:"The requested DurationSeconds exceeds the 1 hour session limit for roles assumed by role chaining"."

Answer: "I used the AssumeRole API to assume an AWS Identity and Access Management (IAM) role using temporary credentials, but I received an error similar to the following:"The requested DurationSeconds exceeds the 1 hour session limit for roles assumed by role chaining".Short descriptionYou can use role chaining to assume a role with temporary security credentials using the AWS Command Line Interface (AWS CLI). For more information, see the role chaining section in roles terms and concepts.Note: Role chaining limits your AWS CLI or AWS API role session to a maximum of one hour and can't be increased. For more information, see Roles terms and concepts.ResolutionUse the following best practices with role chaining:Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.The operation fails if the DurationSeconds parameter value for the temporary credentials is greater than one hour.The role chaining one hour limit only applies to the AWS CLI or API.The AWS Management Console doesn't support role chaining. You can use the switch role feature in the Console to get a role's temporary credentials. The Console uses the credentials of the IAM or federated user to switch to another role. For more information, see switching to a role (console).Multi-Factor Authentication (MFA) users with the AWS CLI use temporary credentials to assume another role. The temporary credentials use the AWS STS GetSessionToken API and are limited to one hour.If role chaining is used to assume Role B for the same AWS account as Role A, then assign additional permissions to Role A to avoid role chaining into Role B.Related informationTroubleshooting IAM and Amazon Elastic Compute Cloud (Amazon EC2)Follow"


Question: How can I resolve the "MalformedJson" error when importing CSV or TSV files into DynamoDB using the default Data Pipeline template?

"When I try to import CSV or TSV files into Amazon DynamoDB using the default AWS Data Pipeline template, I get a "MalformedJson" error."

Answer: "When I try to import CSV or TSV files into Amazon DynamoDB using the default AWS Data Pipeline template, I get a "MalformedJson" error.ResolutionNote: This resolution is valid for Amazon EMR 4.7.0 and later release versions.The Import DynamoDB backup data from S3 template works only if you first export the data using the Export DynamoDB table to S3 template. If you didn't use the Export DynamoDB table to S3 template, then create a new pipeline using a DynamoDBDataFormat object with a HiveActivity object. Use the following script for the HiveActivity object. This script removes incompatible jars from the Hive classpath.delete jar /usr/lib/hive/lib/hive-contrib.jar ;\ndelete jar /mnt/taskRunner/emr-hadoop-goodies.jar ;\ndelete jar /mnt/taskRunner/emr-hive-goodies.jar ;\ndelete jar /mnt/taskRunner/open-csv.jar ;\ndelete jar /mnt/taskRunner/oncluster-emr-hadoop-goodies.jar ;\ndelete jar /mnt/taskRunner/oncluster-emr-hive-goodies.jar ;\ndelete jar /mnt/taskRunner/pipeline-serde.jar ;\nINSERT OVERWRITE TABLE ${output1} SELECT * FROM ${input1};Here's an example of a pipeline definition that exports CSV files from Amazon Simple Storage Service (Amazon S3) to DynamoDB. The example pipeline launches the Amazon EMR resources into a private subnet, which is more secure than a public subnet. For more information, see Configure an Amazon EMR cluster in a private subnet. The CSV files in this example contain the following data:AnyCompany1,100AnyCompany2,20AnyCompany3,30Note: The DynamoDB table must exist before you run the pipeline. Be sure to specify your values for the variables listed in the "values" section. For more information, see Add myVariables to the pipeline definition.{  "objects": [    {      "name": "DefaultEmrCluster1",      "id": "EmrClusterId_kvKJa",      "releaseLabel": "emr-5.23.0",      "type": "EmrCluster",      "subnetId": "#{mySubnetId}",      "emrManagedSlaveSecurityGroupId": "#{myCoreAndTaskSecurityGroup}",      "emrManagedMasterSecurityGroupId": "#{myMasterSecurityGroup}",      "serviceAccessSecurityGroupId": "#{myServiceAccessSecurityGroup}",      "terminateAfter": "24 Hours"    },    {      "dataFormat": {        "ref": "DynamoDBDataFormatId_YMozb"      },      "name": "DefaultDataNode2",      "id": "DataNodeId_WFWdO",      "type": "DynamoDBDataNode",      "tableName": "#{myDDBTableName}"    },    {      "directoryPath": "#{myInputS3Loc}",      "dataFormat": {        "ref": "DataFormatId_ciZN3"      },      "name": "DefaultDataNode1",      "id": "DataNodeId_OZ8Nz",      "type": "S3DataNode"    },    {      "column": [        "company string",        "id bigint"      ],      "name": "DefaultDynamoDBDataFormat1",      "id": "DynamoDBDataFormatId_YMozb",      "type": "DynamoDBDataFormat"    },    {      "column": [        "company string",        "id bigint"      ],      "name": "DefaultDataFormat1",      "id": "DataFormatId_ciZN3",      "type": "CSV"    },    {      "output": {        "ref": "DataNodeId_WFWdO"      },      "input": {        "ref": "DataNodeId_OZ8Nz"      },      "stage": "true",      "maximumRetries": "0",      "name": "DefaultHiveActivity1",      "hiveScript": "delete jar /usr/lib/hive/lib/hive-contrib.jar ;\ndelete jar /mnt/taskRunner/emr-hadoop-goodies.jar ;\ndelete jar /mnt/taskRunner/emr-hive-goodies.jar ;\ndelete jar /mnt/taskRunner/open-csv.jar ;\ndelete jar /mnt/taskRunner/oncluster-emr-hadoop-goodies.jar ;\ndelete jar /mnt/taskRunner/oncluster-emr-hive-goodies.jar ;\ndelete jar /mnt/taskRunner/pipeline-serde.jar ;\nINSERT OVERWRITE TABLE ${output1} SELECT * FROM ${input1};",      "id": "HiveActivityId_AwIZ9",      "runsOn": {        "ref": "EmrClusterId_kvKJa"      },      "type": "HiveActivity"    },    {      "failureAndRerunMode": "CASCADE",      "resourceRole": "DataPipelineDefaultResourceRole",      "pipelineLogUri": "s3://awsdoc-example-bucket/dplogs/",      "role": "DataPipelineDefaultRole",      "scheduleType": "ONDEMAND",      "name": "Default",      "id": "Default"    }  ],  "parameters": [    {      "description": "Input S3 folder",      "id": "myInputS3Loc",      "type": "AWS::S3::ObjectKey"    },    {      "description": "Destination DynamoDB table name",      "id": "myDDBTableName",      "type": "String"    }  ],  "values": {    "myDDBTableName": "companyid",    "myInputS3Loc": "s3://awsdoc-example-bucket1/csvddb/",    "mySubnetId": "subnet_id",    "myCoreAndTaskSecurityGroup": "core and task security group",    "myMasterSecurityGroup": "master security group",    "myServiceAccessSecurityGroup": "service access security group"  }}Related informationExporting data from DynamoDB to Amazon S3Follow"


Question: How do I troubleshoot errors when I'm trying to create a connector using Amazon MSK Connect?

I get an error when I try to create a connector using Amazon Managed Streaming for Apache Kafka (Amazon MSK) Connect.

Answer: "I get an error when I try to create a connector using Amazon Managed Streaming for Apache Kafka (Amazon MSK) Connect.Short descriptionWhen you create a connector using MSK Connect, you might receive one of these error messages:There is an issue with the connector Code: UnknownError.UnknownMessage: The last operation failed. Retry the operation.Invalid parameter connectorConfiguration: The following required field is missing or has invalid value: tasks.maxInvalid parameter serviceExecutionRoleArn: A service linked role ARN cannot be provided as service execution role ARNorg.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches...org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadataorg.apache.kafka.common.errors.SaslAuthenticationException: Too many connectsorg.apache.kafka.connect.errors.ConnectException: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to s3.eu-central-1.amazonaws.com:443 [s3.eu-central-1.amazonaws.com/52.219.47.235 ] failed: connect timed outorg.apache.kafka.connect.errors.ConnectException: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to firehose.us-east-2.amazonaws.com:443 [firehose.us-east-2.amazonaws.com/52.95.23.168 ] failed: connect timed outConnection to node - 1 (broker endpoint) failed authentication due to : Access DeniedERROR Connection to node -3 (b-1.<cluster>.<region>.amazonaws.com/INTERNAL_IP) failed authentication due to: An error: (java.security .PrivilegedActionException: javax.security .sasl.SaslException: Failed to find AWS IAM Credentials [Caused by aws_msk_iam_auth_shadow.com.amazonaws.SdkClientException: Unable to load AWS credentials from any ...........Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))ResolutionThere is an issue with the connector Code: UnknownError.UnknownMessage: The last operation failed. Retry the operation.You get this error when MSK Connect can't create the connector, and the connector is moved to Failed state.To find the root cause for the failure, review the log events for MSK Connect. MSK Connect writes log events that you can use to debug your connector. When you create a connector, you can specify zero log destinations or one of the following log destinations:Amazon CloudWatch logsAmazon Simple Storage Service (Amazon S3)Amazon Kinesis Data FirehoseInvalid parameter connectorConfiguration: The following required field is missing or has invalid value: tasks.maxIf you use a carriage return (/r) character at the end of a configuration, then you receive the following error:Invalid parameter connectorConfiguration: The following required field is missing or has invalid value: tasks.maxTo resolve this error, try the following troubleshooting steps:Manually enter the configuration information in the connector configuration dialog box instead of copying and pasting from another source, such as documentation.If you're on a Windows operating system, then use a text editor (such as Notepad++). You can use the text editor to check and remove the carriage return (CRLF) and any end-of-line (EOL) characters. To remove the carriage return, copy and paste the configuration into a text editor. From your text editor, choose View and Show Symbol. Then, choose Show All Characters to review any CRLF or EOL characters that might exist in a configuration. Replace all the CRLF characters ("\r\n") with LF characters ("\n").Invalid parameter serviceExecutionRoleArn: A service linked role ARN cannot be provided as service execution role ARN.You get this error when you use a service-linked role to create a connector. MSK Connect doesn't support using the service-linked role as the service execution role. You must create a separate service execution role. For instructions on how to create a custom AWS Identity and Access Management (IAM) role, see Creating a role to delegate permissions to an AWS service. Specify the role that you want the connector to work with. This role must be different from the service-linked role AWSServiceRoleForKafkaConnect that the service uses internally to create the connector resources.org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches...To resolve this error, try the following troubleshooting steps:Remove any carriage return (/r) characters that might exist in the connector configuration.If multiple files are required for the connector plugin, then include these files in your zipped file. The zipped file is used during the creation of the connector plugin. The JAR files in the zipped file must also have the expected file structure documented for that plugin. It's a best practice to turn on logs for MSK Connect and review the logs to confirm that the file structure is correctly set.org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadataIf the connector can't reach your MSK cluster, then you receive a TimeOutException error. To resolve this error, try the following troubleshooting steps:Check if bootstrap servers that are specified in the properties, including port number, are valid and accurate.Be sure that the security group for your cluster allows inbound traffic from the client's security group that's associated with MSK Connect. If an MSK cluster is used, then add a rule that allows inbound traffic from itself. For more information about setting up these rules, see Security group rules.org.apache.kafka.common.errors.SaslAuthenticationException: Too many connectsIf your MSK cluster is running on a kafka.t3.small broker type with IAM access control, then be aware of the connection limit. The kafka.t3.small instance type accepts only one TCP connection per broker per second. When this connection limit is exceeded, your creation test fails. As a result, you receive a SaslAuthenticationException error, indicating an invalid credentials error. For more information about MSK clusters and IAM access control, see How Amazon MSK works with IAM.To resolve the SaslAuthenticationException error, take one of the following actions:In your MSK Connect worker configuration, update the values for reconnect.backoff.ms and reconnect.backoff.max.ms to "1000" or higher.Upgrade to a larger broker instance type (such as kafka.m5.large or higher). For more information about Amazon MSK broker types and choosing the right broker type, see Broker types and Right-size your cluster.org.apache.kafka.connect.errors.ConnectException: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to s3.us-east-1.amazonaws.com:443 failed: connect timed outYou get this error when the connector can't connect to Amazon S3. To troubleshoot this error, make sure that you created the Amazon Virtual Private Cloud (Amazon VPC) endpoint from the cluster's VPC to Amazon S3. To create an Amazon VPC endpoint from the cluster's VPC to Amazon S3, complete the following steps:Open the Amazon VPC console.In the navigation pane, choose Endpoints.Choose Create endpoint.For Service category, choose AWS services.Under Services, choose the Service Name filter, and then select com.amazonaws.<region>.s3. Replace <region> with your AWS Region.Choose the Type filter, and then choose Gateway.For VPC, select the cluster's VPC.Under Route tables, select the route table that's associated with the cluster's subnets.Choose Create endpoint.org.apache.kafka.connect.errors.ConnectException: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to firehose.us-east-2.amazonaws.com:443 failed: connect timed outYou get this error when the connector can't connect to Amazon Kinesis Data Firehose. To troubleshoot this error, make sure that you created the Amazon VPC endpoint from the cluster's VPC to Kinesis Data Firehose.To create an Amazon VPC endpoint from the cluster's VPC to Kinesis Data Firehose, follow the steps from the preceding section. Use the Service name filter com.amazonaws.<region>.kinesis-firehose.Connection to node - 1 (b1.<cluster>.<region>.amazonaws.com) failed authentication due to : Access DeniedYou get this error when the IAM user for MSK Connect doesn't have the required permissions to create a connector.When you create a connector with MSK Connect, you must specify an IAM role to use with it. Your service execution role must have the following trust policy so that MSK Connect can assume this role:{    "Version": "2012-10-17",    "Statement": [{        "Effect": "Allow",        "Principal": {            "Service": "kafkaconnect.amazonaws.com"        },        "Action": "sts:AssumeRole",        "Condition": {            "StringEquals": {                "aws:SourceAccount": "Account-ID"            },            "ArnLike": {                "aws:SourceArn": "MSK-Connector-ARN"            }        }    }]}If the MSK cluster that you want to use with your connector uses IAM authentication, then you must add the following permissions policy to the connector's service execution role:{    "Version": "2012-10-17",    "Statement": [{            "Effect": "Allow",            "Action": [                "kafka-cluster:Connect",                "kafka-cluster:DescribeCluster"            ],            "Resource": [                "cluster-arn"            ]        },        {            "Effect": "Allow",            "Action": [                "kafka-cluster:ReadData",                "kafka-cluster:DescribeTopic"            ],            "Resource": [                "ARN of the topic that you want a sink connector to read from"            ]        },        {            "Effect": "Allow",            "Action": [                "kafka-cluster:WriteData",                "kafka-cluster:DescribeTopic"            ],            "Resource": [                "ARN of the topic that you want a source connector to write to"            ]        },        {            "Effect": "Allow",            "Action": [                "kafka-cluster:CreateTopic",                "kafka-cluster:WriteData",                "kafka-cluster:ReadData",                "kafka-cluster:DescribeTopic"            ],            "Resource": [                "arn:aws:kafka:region:account-id:topic/cluster-name/cluster-uuid/__amazon_msk_connect_*"            ]        },        {            "Effect": "Allow",            "Action": [                "kafka-cluster:AlterGroup",                "kafka-cluster:DescribeGroup"            ],            "Resource": [                "arn:aws:kafka:region:account-id:group/cluster-name/cluster-uuid/__amazon_msk_connect_*",                "arn:aws:kafka:region:account-id:group/cluster-name/cluster-uuid/connect-*"            ]        }    ]}For information on finding your cluster's UUID and construct topic ARNs, see Resources.ERROR Connection to node -3 (b-1.<cluster>.<region>.amazonaws.com/INTERNAL_IP ) failed authentication due to: An error: (java.security .PrivilegedActionException: javax.security .sasl.SaslException: Failed to find AWS IAM Credentials [Caused by aws_msk_iam_auth_shadow.com.amazonaws.SdkClientException: Unable to load AWS credentials from any ...........Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))You get this error when the IAM role that you used to create the connector doesn't have the required permissions.Review the access policies and trust relationship of the IAM role for the connector that you use to access Amazon MSK. For more information, see Service execution role.Related informationHow do I connect to my Amazon MSK cluster using the Kafka-Kinesis-Connector?MSK ConnectTroubleshooting your Amazon MSK clusterFollow"


Question: Why is my deleted EBS volume still showing as compliant or noncompliant in my AWS Config rule?

I deleted my Amazon Elastic Block Store (Amazon EBS). Why is my Amazon EBS volume still showing as compliant or noncompliant in my AWS Config rule?

Answer: "I deleted my Amazon Elastic Block Store (Amazon EBS). Why is my Amazon EBS volume still showing as compliant or noncompliant in my AWS Config rule?Short descriptionAmazon EBS volumes are durable, block-level storage devices that you can attach to Amazon Elastic Compute Cloud (Amazon EC2) instances. You can delete Amazon EBS volumes by:Stopping the Amazon EC2 instance, detaching the Amazon EBS volume, and then running the DeleteVolume API call.Setting the DeleteOnTermination attribute to true, and then running the TerminateInstances API call.ResolutionAmazon EBS volumes deleted with the DeleteVolume API callAmazon EBS volumes deleted using the DeleteVolume API call invoke a DescribeVolumes API call on volume. The DescribeVolumes API call returns an InvalidVolume.NotFound error code, and the Amazon EBS volume is removed from the list of resources in AWS Config. The updated configuration of the volume is recorded as a configuration item with a status as ResourceDeleted, and then delivered to an Amazon Simple Storage Service (Amazon S3) bucket.Amazon EBS volumes deleted with the TerminateInstances API callTerminated Amazon EC2 instances use the DeleteOnTermination attribute for each attached EBS volume to determine to delete the volume. By default, the DeleteOnTermination attribute for the root volume of an Amazon EC2 instance is set to true, and set to false for all other volume types. Amazon EC2 deletes the Amazon EBS volume that have the DeleteOnTermination attribute set to true, but it does not publish the DeleteVolume API call. This is because AWS Config uses the DeleteVolume API call as a trigger with the rule, and the resource changes aren't recorded for the EBS volume. The EBS volume still shows as compliant or noncompliant.AWS Config performs a baseline every six hours to check for new configuration items with the ResourceDeleted status. The AWS Config rule removes the deleted EBS volumes from the evaluation results. If you choose the "Include deleted resources" check box, you can safely ignore any deleted Amazon EBS volumes.Note: Amazon EBS volumes associated with Amazon EC2 instances created by Auto Scaling groups use the TerminateInstances API call to terminate the instance.Related informationDeleting an Amazon EBS volumeHow can I prevent my EBS volumes from being deleted when I terminate my EC2 instances?Follow"


Question: Why did I receive an Amazon GuardDuty finding type Trojan:EC2 alerts for my Amazon EC2 instance?

Amazon GuardDuty detected finding type Trojan:EC2 alerts for my Amazon Elastic Compute Cloud (Amazon EC2) instance.

Answer: "Amazon GuardDuty detected finding type Trojan:EC2 alerts for my Amazon Elastic Compute Cloud (Amazon EC2) instance.ResolutionWhen GuardDuty detects anomalous Amazon EC2 activity, GuardDuty responds with a Trojan alert. Check each reference in this list to find the reason for the alert. Then, follow the instructions for remediating a compromised EC2 instance.Attempting to communicate with an IP address of a remote host that is a known black hole.Attempting to communicate with an IP address of a remote host that is known to hold credentials and other stolen data captured by malware.Querying a domain name that is being redirected to a black hole IP address.Querying a domain name of a remote host that is a known source of drive-by download attacks.Querying a domain name of a remote host that is known to hold credentials and other stolen data captured by malware.Querying algorithmically generated domains.Exfiltrating data through DNS queries.Querying domains involved in phishing attacks.For more information, see GuardDuty EC2 finding types and How Amazon GuardDuty uses its data sources.Related informationCreating custom responses to GuardDuty findings with Amazon CloudWatch EventsHow to use Amazon GuardDuty and AWS Web Application Firewall to automatically block suspicious hostsSecurity solutions in AWS MarketplaceGuardDuty finding formatFollow"


Question: How can I use a CloudFormation resource import to create an Amazon S3 notification configuration for Lambda on an existing S3 bucket?

I want to create an Amazon Simple Storage Service (Amazon S3) notification configuration for AWS Lambda on an existing S3 bucket. I want to do this by using AWS CloudFormation to import a resource.

Answer: "I want to create an Amazon Simple Storage Service (Amazon S3) notification configuration for AWS Lambda on an existing S3 bucket. I want to do this by using AWS CloudFormation to import a resource.Short descriptionTo configure an Amazon S3 notification without using a custom resource, do the following:Create a template with the Lambda function S3NotificationLambdaFunction. This function adds the existing bucket NotificationS3Bucket notification configuration.Use a resource import to bring the existing NotificationS3Bucket S3 bucket that's specified in the template into CloudFormation management.Update the CloudFormation stack to include the properties that you want to activate in your S3 bucket.To use a custom resource, see How can I use CloudFormation to create an Amazon S3 notification configuration for Lambda on an existing S3 bucket?ResolutionImportant: The following steps override any existing or manually created notification configurations in an S3 bucket. Follow these steps to add a new notification configuration to an imported S3 bucket.Create a CloudFormation template with a Lambda functionThe following example template creates a Lambda function with a running role and permissions to invoke the function. To use an existing Lambda function, use the function's Amazon Resource Name (ARN) in your CloudFormation template for the LambdaConfiguration property in the S3 bucket.AWSTemplateFormatVersion: 2010-09-09Description: >-  Sample template that shows you how to import an existing S3 bucket as an event source for a Lambda functionParameters:  NotificationBucket:    Type: String    Description: S3 bucket that's used for Lambda event notificationResources:  S3NotificationLambdaFunction:    Type: 'AWS::Lambda::Function'    Properties:      Code:        ZipFile: !Join          - |+          - - import json            - 'def lambda_handler(event,context):'            - '    return ''Welcome... This is a test Lambda Function'''      Handler: index.lambda_handler      Role: !GetAtt LambdaIAMRole.Arn      Runtime: python3.6      Timeout: 5  LambdaInvokePermission:    Type: 'AWS::Lambda::Permission'    Properties:      FunctionName: !GetAtt S3NotificationLambdaFunction.Arn      Action: 'lambda:InvokeFunction'      Principal: s3.amazonaws.com      SourceAccount: !Ref 'AWS::AccountId'      SourceArn: !Sub 'arn:aws:s3:::${NotificationBucket}'  LambdaIAMRole:    Type: 'AWS::IAM::Role'    Properties:      AssumeRolePolicyDocument:        Version: 2012-10-17        Statement:          - Effect: Allow            Principal:              Service:                - lambda.amazonaws.com            Action:              - 'sts:AssumeRole'      Path: /      Policies:        - PolicyName: root          PolicyDocument:            Version: 2012-10-17            Statement:              - Effect: Allow                Action:                  - 's3:GetBucketNotification'                  - 's3:PutBucketNotification'                Resource: !Sub 'arn:aws:s3:::${NotificationBucket}'              - Effect: Allow                Action:                  - 'logs:CreateLogGroup'                  - 'logs:CreateLogStream'                  - 'logs:PutLogEvents'                Resource: 'arn:aws:logs:*:*:*'Import an existing S3 bucket to your CloudFormation stack1.    Open the AWS CloudFormation console.2.    On the navigation pane, choose Stack, and then select the stack that you created.3.    Choose Stack actions, and then choose Import resources into stack.4.    Review the Import overview page, and then choose Next.Important: In your CloudFormation template, each resource that you import must have a DeletionPolicy attribute, and all other resources must remain the same. For example:AWSTemplateFormatVersion: 2010-09-09Description: >-  Sample template that shows you how to import an existing S3 bucket as an event source for a Lambda functionParameters:  NotificationBucket:    Type: String    Description: S3 bucket that's used for Lambda event notificationResources:  S3NotificationLambdaFunction:    Type: 'AWS::Lambda::Function'    Properties:      .      .  LambdaInvokePermission:    Type: 'AWS::Lambda::Permission'    Properties:      .      .  LambdaIAMRole:    Type: 'AWS::IAM::Role'    Properties:      .      .  <The preceding resources were already created. Now, add the DeletionPolicy to the preceding stack to import the S3 bucket.>  NotificationS3Bucket:    Type: 'AWS::S3::Bucket'    DeletionPolicy: Retain    Properties:      BucketName: myenv-bucket          #Bucket name to import5.    In the Specify template section, choose Amazon S3 URL or Upload a template file based on your requirements, and then choose Next.6.    Complete the rest of the steps in the wizard to import your existing resources. For more information, see Import an existing resource into a stack using the AWS CloudFormation console.7.    Wait for the stack to reach the IMPORT_COMPLETE state.Update your CloudFormation template1.    Update the stack with the CloudFormation template that you modified.2.    Wait for the stack to reach the UPDATE_COMPLETE state, and then verify the NotificationConfiguration on the S3 bucket.3.    Turn on Amazon S3 event notifications on your S3 bucket. In the following example template, the bucket name is myenv-bucket:AWSTemplateFormatVersion: 2010-09-09Description: >-  Sample template that shows you how to import an existing S3 bucket as an event source for a Lambda functionParameters:  NotificationBucket:    Type: String    Description: S3 bucket that's used for Lambda event notificationResources:  S3NotificationLambdaFunction:    Type: 'AWS::Lambda::Function'    Properties:      .      .  LambdaInvokePermission:    Type: 'AWS::Lambda::Permission'    Properties:      .      .  LambdaIAMRole:    Type: 'AWS::IAM::Role'    Properties:      .      .  <The preceding resources were already created. Now, add the DeletionPolicy to the preceding stack to import the S3 bucket.>  NotificationS3Bucket:    Type: 'AWS::S3::Bucket'    DeletionPolicy: Retain    Properties:      BucketName: myenv-bucket      NotificationConfiguration:   #Update stack with NotificationConfiguration          LambdaConfigurations:              - Event: 's3:ObjectCreated:Put'                Function: !GetAtt S3NotificationLambdaFunction.Arn              - Event: 's3:ObjectRemoved:*'                Function: !GetAtt S3NotificationLambdaFunction.ArnFollow"


Question: How do I restrict CIDR IP addresses for a LoadBalancer type service in Amazon EKS?

I want to restrict CIDR IP addresses for a LoadBalancer type service in Amazon Elastic Kubernetes Service (Amazon EKS).

Answer: "I want to restrict CIDR IP addresses for a LoadBalancer type service in Amazon Elastic Kubernetes Service (Amazon EKS).Short descriptionIf you create a service of type:LoadBalancer, then requests from the source 0.0.0.0/0 are allowed by default. If your load balancer is in a public subnet, then requests are routed to worker nodes from anywhere on the internet.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Set up your environment1.    Create an Amazon EKS cluster.2.    Create and launch worker nodes.3.    Set up kubectl.4.    Set up the AWS CLI.5.    Set up the AWS Load Balancer Controller.Note: The AWS Load Balancer Controller supports the following versions of the AWS Network Load Balancer for Services of type LoadBalancer: NLB ip mode (version 2.0.0 or above) and NLB instance mode (2.2.0 or above)Important: It's a best practice to use the AWS Load Balancer Controller when you provision a new Network Load Balancer for Services of type LoadBalancer. Use the AWS Load Balancer Controller instead of the Kubernetes in-tree service load balancer controller.Restrict CIDR IP addresses1.    In your service manifest file (svc.yaml), add the .spec.loadBalancerSourceRanges field. For example:apiVersion: v1kind: Servicemetadata:  labels:    app: nginx  name: nginx  annotations:    service.beta.kubernetes.io/aws-load-balancer-type: "external"    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "instance"    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"spec:  ports:  - port: 80    protocol: TCP    targetPort: 80  selector:    app: nginx  type: LoadBalancer  loadBalancerSourceRanges:  - "143.231.0.0/16"2.    To apply the manifest file, run the following command:$ kubectl apply -f svc.yaml3.    To confirm that the inbound rules on the security group are modified, run the following AWS CLI command:$ aws ec2 describe-security-groups --group-ids sg-XXXXXXXXXXXXXXXXX...    "CidrIp": "143.231.0.0/16"...Finally, consider the following:If you use NLB ip mode, then the .spec.loadBalancerSourceRanges field is ignored by default. In this case, use the following annotation:service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: preserve_client_ip.enabled=trueThis annotation enables Client IP preservation.For a service with a Network Load Balancer type, consider the maximum security group limit. For each node port and subnet CIDR range, the controller creates rules on the worker node's security group.Related informationCluster authenticationKubernetes ServiceFollow"


Question: How do I access the internet using Site-to-Site VPN in my on-premises network?

I want to connect my on-premises network to the internet using AWS Site-to-Site VPN and the AWS Transit Gateway.

Answer: "I want to connect my on-premises network to the internet using AWS Site-to-Site VPN and the AWS Transit Gateway.Short descriptionAWS Site-to-Site VPN allows secured connectivity between AWS resources and the on-premises network such as a data center or a branch office.AWS Site-to-Site VPN provides two tunnels per connection, using the virtual private gateway or the AWS Transit Gateway. The virtual private gateway provides connectivity to a single Amazon Virtual Private Cloud (Amazon VPC) in a Region. The transit gateway provides connectivity to multiple Amazon VPCs in a region as well as to the internet.For more information on how AWS Site-to-Site VPN works, see What is AWS Site-to-Site VPN?ResolutionEstablish the Site-to-Site VPN connection through a transit gateway to get centralized internet access from your on-premises network. This centralized setup uses AWS Network Translation Service (NAT gateway) to connect to the internet.Follow these steps to gain internet access from your on-premises network:1.    Create a transit gateway.2.    Create a Site-to-Site VPN using a transit gateway.3.    Attach your Amazon VPC to a transit gateway in your Region.4.    Create a public NAT gateway in a public subnet in the same Amazon VPC.5.    Create a default-route 0.0.0.0/0 in the VPC’s subnet routing table associated with transit gateway. This route points to the NAT gateway that you created earlier.Destination  Target0.0.0.0/0    nat-123456789012345676.    Create a route for the on-premises CIDR in the public subnet routing table you used earlier. This route points to the transit gateway.Destination  Target10.0.0.0/16  tgw-123456789098765430.0.0.0/0    igw-123456789012345677.    Create a default route 0.0.0.0/0 in the transit gateway routing table associated with VPN attachment. Then, point it to the Amazon VPC attachment:CIDR        Attachment ID                  Resource ID        Resource type       Route type        Route state    Prefix list ID0.0.0.0     tgw-attach-98765432109876      vpc-987654321      VPC                 Static            Active         -8.    Create a route (Static route or propagate) for the on-premises CIDR in the transit gateway routing table associated with the Amazon VPC attachment. Then, point it to the VPN attachment:CIDR          Attachment ID               Resource ID         Resource type      Route type       Route state     Prefix list ID10.0.0.0/16   tgw-attach-1234567890123    vpn-1234567897      VPN                Static           Active          -Note: The Site-to-site VPN encryption domain must allow traffic between the on-premises CIDR and any (0.0.0.0) destination, a policy-based VPN.Related informationCreating a transit gateway VPN attachmentTransit gateway route tablesFollow"


Question: "How do I troubleshoot missing X-Ray traces, segments, or services in the service map?"

"I want to troubleshoot my missing AWS X-Ray traces, missing segments, and missing services in the service map."

Answer: "I want to troubleshoot my missing AWS X-Ray traces, missing segments, and missing services in the service map.Short descriptionMissing services in the service map are caused by missing traces. The causes that lead to missing traces are:Improper instrumentationX-Ray SDK can't reach the X-Ray daemon over the daemon address or port numberX-Ray daemon can't reach the X-Ray service endpointTracing isn't configured at the individual service levelMissing AWS Identity and Access Management (IAM) permissionsSampling rules configurationMissing configuration in the OpenTelemetry CollectorMissing segments are caused by the absence of open parent segments to the sub segments. For more information, see troubleshooting AWS X-Ray.ResolutionImproper instrumentationImproper instrumentation when using X-Ray SDKIf your application code isn't instrumented properly for X-Ray SDK to patch the supported libraries and frameworks, then your traces will be missing. Missing traces are also caused by trace IDs that aren't passed to the downstream services.To troubleshoot this, turn on debug-level logging on the SDK to output more detailed logs to your application log file. This allows you to further isolate issues related to the instrumentation and to trace the flow of the trace ID across the application. The logs also indicate if a trace is missing because it's not sampled or because of instrumentation misconfiguration. For more information on debug logging, see X-Ray SDK for Java, X-Ray SDK for Node.js, X-Ray SDK for Python, X-Ray SDK for .NET, X-Ray SDK for Ruby.Note: If the application is using an auto-instrumentation agent for Java, then it doesn't capture traces for asynchronous requests and traces will be missing. Consider manual instrumentation with SDK if you have missing traces.Improper instrumentation when using OpenTelemetry SDKIf you use OpenTelemetry SDK and the application code isn't instrumented properly to patch the supported libraries and frameworks, then your traces will be missing. Missing segments are also caused by trace IDs that aren't passed to the downstream services.To troubleshoot this, do the following:Turn on debug-level logging on the SDK to send more detailed logs to your application log file. This allows you to further isolate issues related to the instrumentation and to trace the flow of the trace ID across the application. Logs indicate if a trace is missing because it's not sampled or because of instrumentation misconfiguration. For more information on debug logging, see OpenTelemetry JavaScript SDK, OpenTelemetry Java SDK, OpenTelemetry Python SDK, OpenTelemetry Ruby SDK, or OpenTelemetry PHP SDK.Make sure that your instrumentation isn't turned off. For more information, see suppressing specific auto-instrumentation.Note: For enriching traces with AWS infrastructure information, make sure that the AWS Detector is supported for that AWS service in the OpenTelemetry SDK. For more information, see using the AWS resource Detectors.X-Ray SDK can't reach the X-Ray daemon over the daemon address or port numberIf X-Ray SDK can't reach the X-Ray daemon over the default or configured daemon port number and address, then traces will be missing. The X-Ray daemon address is passed to SDK with the environmental variable AWS_XRAY_DAEMON_ADDRESS. By default, the X-ray Daemon listens on port 2000 UDP. This port can be changed through the command line option and the configuration file passed to the X-ray daemon. For more information, see using a configuration file to configure the X-Ray daemon.If you are running the X-Ray daemon with debug logs, then identify the UDP port configured for the X-Ray daemon using the -l debug command:./xray -l debugNote: The following example identifies 3000 as the port number.Example output:2023-03-28T15:15:43-07:00 [Debug] Listening on UDP 127.0.0.1:3000If you aren't running the X-Ray Daemon with debug logs, then turn on the option to debug logs and re-run the daemon. For more information, see Using a configuration file.Note: If your X-Ray daemon isn't running on the same machine as your application and X-Ray SDK, then check the inbound rule and ingress rule. Check that the security group inbound rule and network access control list (network ACL) ingress rule are allowing traffic. Traffic must be allowed for X-Ray SDK over the listener port of the X-Ray daemon.X-Ray daemon can't reach the X-Ray service endpointIf your X-Ray daemon can't reach the X-Ray service endpoint, then check the following:If you have proxy configured at the source end, then check whether proxy is inhibiting the traffic from reaching the X-Ray service endpoint.If you're using the X-Ray Amazon Virtual Private Cloud (Amazon VPC) endpoint com.amazonaws.region.xray to connect to the X-Ray service, then check the security group. Make sure that the security group attached has outbound traffic that allows HTTPS traffic to push data to the X-Ray service API. Also, make sure that the security group allows inbound traffic on port 443 to allow X-Ray daemon.Verify that the associated network ACL and X-Ray Amazon VPC endpoint have an egress rule that allows traffic on port 443 for everywhere 0.0.0.0/0. The X-Ray daemon must send data to the X-Ray service endpoint. Also, verify that the network ACL ingress rule allows traffic from the X-Ray daemon to the Amazon VPC endpointTracing isn't configured at the individual service levelWhen X-Ray configuration isn't turned on for certain AWS services, traces will be missing. The following is a list of AWS services that require X-Ray configuration to be turned on at the service level:AWS AppSyncAWS App RunnerAmazon API GatewayAmazon CloudWatch real user monitoring (CloudWatch RUM)AWS LambdaAmazon Simple Notification Service (Amazon SNS)AWS Step FunctionsMissing AWS Identity and Access Management (IAM) permissionsIf you receive Access Denied errors in the X-Ray daemon logs, then check the IAM permissions associated with the IAM role that used by the X-Ray daemon. To send the traces to the X-Ray service endpoint, the X-Ray daemon requires permissions to be defined in the managed policy AWSXRAYDaemonWriteAccess. If you're using a tags-based condition in your IAM policy, then make sure that the IAM policy refers to the correct tag. For more information, see Authorization based on X-Ray tags.If X-Ray daemon is running in Amazon Elastic Container Service (Amazon ECS) as a sidecar container in the task, then check the Amazon ECS task role. Make sure that the Amazon ECS task role has the AWSXRayDaemonWriteAccess policy attached.If X-Ray daemon is deployed in Amazon Elastic Kubernetes Services (Amazon EKS), then check the sidecar container in the pod. Make sure the container is using IAM roles for service accounts (IRSA) - Amazon EKS and the IAM role associated has the AWSXRayDaemonWriteAccess policy attached.If X-ray daemon is configured in Amazon Elastic Compute Cloud (Amazon EC2), then check the associated instance profile. Make sure that the instance profile role associated with that EC2 Instance has the AWSXRayDaemonWriteAccess policy attached.If X-Ray daemon is configured in AWS Lambda, then check the Lambda execution role. Confirm whether the Lambda execution role has the AWSXRayDaemonWriteAccess policy and if it must be attached.If you have active tracing turned on in Amazon SNS and you still can’t see your traces, then configure a resource policy in X-Ray.Sampling rules configurationIf traces aren't sampled by X-Ray SDK and AWS services that support active tracing with sampling configuration, then the traces will be missing. Use sampling rules to determine the requests to record. By default, the X-Ray SDK records the first request each second and five percent of any additional requests. You can customize this, and you can change the sampling rate to decrease or increase the number of traces recorded.Example scenario:Reservoir set to 1. This indicates first request of each second is taken.Sampling rate set to 0.1. This indicates ten percent of additional requests of each second is recorded.Due to the limit on the number of requests recorded by X-Ray SDK, all requests might not get traced. To record more traces, increase the reservoir and sampling rate settings. For more information, see sampling rule examples.Note: If multiple sampling rules are configured and the number of traces sampled fluctuates, then check the sampling rule that's used by X-Ray SDK. For more information, see viewing sampling results.Missing configuration in the OpenTelemetry CollectorIf you're using OpenTelemetry Collector instead of X-Ray daemon, then the required receiver and exporter configuration might be missing. A missing configuration cause the traces to not be sent to the X-Ray service endpoint. Based on your receiver, check the receiver configuration in the config.yaml file to verify there is no missing configuration.Example configurations:X-Ray exporterX-Ray receiverZipKin receiverNote: Check whether your collector has the required IAM permission defined in the managed policy AWSXRayDaemonWriteAccess.Follow"