Question: How do I troubleshoot S3 related errors while setting up ELB access logging?

I'm getting an error while setting up Elastic Load Balancing (ELB) access logs using an Amazon Simple Storage Service (Amazon S3) bucket.

Answer: "I'm getting an error while setting up Elastic Load Balancing (ELB) access logs using an Amazon Simple Storage Service (Amazon S3) bucket.Short descriptionTo use access logs with your load balancer, the load balancer and the Amazon S3 bucket must be in the same account. You must also attach a bucket policy to the Amazon S3 bucket that allows ELB permission to write to the bucket. Depending on the error message that you receive, see the related resolution section.Note: Network Load Balancers (NLB) support access logs only for Transport Layer Security (TLS) listeners. The log contains information about TLS requests made to the Network Load Balancer. Transmission Control Protocol (TCP) is not supported.Resolution"S3Bucket: my-access-log-bucket is not located in the same region with ELB: app/my-load-balancer/50dc6c495c0c9188"This error indicates that your Amazon S3 bucket and load aren't located in the same AWS Region. The Amazon S3 bucket can be in a different Region but must be in the same account as the load balancer."Access Denied for bucket: my-access-log-bucket. Please check S3bucket permission"This error indicates that the Amazon S3 bucket doesn't have a policy that grants permission to write the access logs.To resolve this error, verify that the bucket policy grants permission to write logs to your bucket. Confirm that you have the correct placeholders for the name and prefix of your bucket. Confirm you have the correct ID of the AWS account for Elastic Load Balancing, based on the Region for your load balancer.For more information on the required permissions, see:Application Load Balancer: Bucket permissionsNetwork Load Balancer: Bucket requirementsClassic Load Balancer: Attach a policy to your S3 bucketServer-side encryption with Amazon S3 managed keys (SSE-S3) can be used to encrypt access logs for ELB. Additionally, Network Load Balancers support AWS Key Management Service (AWS KMS) customer managed keys to encrypt access logs. You can't use AWS KMS managed keys for encrypting ELB access logs. "The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again."If you receive this error, verify that your access logs bucket prefix doesn't include "AWSLogs."Additional troubleshootingIf you verified your S3 bucket policy and configuration but still can't view logs, verify that the load balancer is receiving traffic. To verify whether the load balancer is receiving traffic, check the ActiveConnectionCount and RequestCount metrics.Follow"


Question: How do I use AWS Batch as a target for my EventBridge rule?

I want to use AWS Batch as a target for my Amazon EventBridge rule.

Answer: "I want to use AWS Batch as a target for my Amazon EventBridge rule.ResolutionThe following example shows how to configure Amazon EventBridge to submit an AWS Batch job when a new Amazon Simple Storage Service (Amazon S3) bucket is created.Create a job definition1.    Open the AWS Batch console.2.    From the navigation bar, select your AWS Region.3.    In the navigation pane, choose Job definitions, and then choose Create.4.    For Job definition name, enter a unique name for your job definition.Note: You can use up to 128 letters (uppercase and lowercase), numbers, hyphens, and underscores in your unique name.5.    For Container image, enter amazonlinux.6.    For Command, enter the following:echo Ref::S3bucket7.    For vCPUs, enter 2.8.    For Memory (MiB), enter 500.9.    Choose Next, and then choose Create.Get the ARN of your job queue and job definition1.    Open the AWS Batch console.2.    In the navigation pane, choose Job queues.3.    Choose your job queue.4.    In the Job queue details section, copy the Queue ARN, and then save it for later.5.    In the navigation pane, choose Job definitions, and then choose the job definition that you previously created.6.    In the Job definition details section, copy the Job definition ARN, and then save it for later.Enable CloudTrail data events logging for objects in an S3 bucketTo trigger AWS Batch jobs on Amazon S3 object-level operations (for example, when a new object is uploaded to an existing bucket), see Enabling CloudTrail event logging for S3 buckets and objects.Create the EventBridge rule1.    Open the EventBridge console.2.    Select Create rule.3.    Enter a Name for your rule. You can optionally enter a Description.4.    In Define pattern, select Event pattern.5.    Select Pre-defined pattern by service.6.    For Service provider, choose AWS.7.    For Service name, choose Simple Storage Service (S3).8.    For Event type, choose Bucket-Level API Call via CloudTrail.9.    Choose Any operation.10.    In the Select targets section, choose Batch job queue from the Target dropdown list.11.    For Job queue, paste in the job queue ARN that you copied earlier.12.    For Job definition, paste in the job definition ARN that you copied earlier.13.    For Job name, enter a name for your AWS Batch job.14.    In the Configure input section, choose Input Transformer.15.    In the first input box, enter the S3 bucket values to be sent when the event is triggered:{"S3BucketNameValue":"$.detail.requestParameters.bucketName"}Note: Replace S3BucketNameValue with your own value.16.    In the second input box, enter the Parameters structure to be passed to the Batch job:{"Parameters" : {"S3bucket": S3BucketNameValue}}Note: Replace S3BucketNameValue with your own value. Replace S3bucket with the name of the parameter that you want to define in your AWS Batch job.17.    Choose either Create a new role for this specific resource or Use existing role.Note: If you choose an existing role, that role should have an AWS Identity and Access Management (IAM) policy that allows the batch:SubmitJob action.18.    Select Create.Test the new rule that you created1.    Open the Amazon S3 console.2.    Choose Create bucket.3.    In the Bucket name field, type a unique DNS-compliant name for your new bucket.4.    For Region, choose the same Region where you created the EventBridge rule.5.    Choose Create.Check your logs1.    Open the AWS Batch console.2.    In the navigation pane, choose Jobs.3.    Choose your job with a Status of SUCCEEDED.4.    On the Job details page, in the Attempts section, choose View logs. The log displays your bucket name in the CloudWatch console.Related informationTutorial: Use input transformer to customize what EventBridge passes to the event targetFollow"


Question: How do I troubleshoot network performance issues between EC2 Linux or Windows instances in a VPC and an on-premises host over the internet gateway?

Packet loss or latency issues exist between my Amazon Elastic Compute Cloud (Amazon EC2) instances and on-premises host over the internet gateway. How can I troubleshoot these issues with network performance?

Answer: "Packet loss or latency issues exist between my Amazon Elastic Compute Cloud (Amazon EC2) instances and on-premises host over the internet gateway. How can I troubleshoot these issues with network performance?Short descriptionTo diagnose network issues such as packet loss or latency, first test the network to isolate the source of the issue. The following resolution can help determine if the source of the issue is a network or an application. It's a best practice to benchmark the performance results so that you can compare the results when you're observing the performance issues.Before you begin troubleshooting, check the following:Be sure that the network utilities are installed on both endpoints (on the EC2 instance and the on-premises host).Use an EC2 instance that supports enhanced networking, and be sure that the drivers are up to date. Enhanced networking provides higher I/O with low CPU utilization, which helps avoid instance-level issues when running performance tests. If enhanced networking isn't turned on, see Enhanced networking on Linux or Enhanced networking on Windows.Connect to your EC2 instance to access the instances and be sure that there's end-to-end connectivity between your EC2 instance and your on-premises host.ResolutionInstall the following tools to help troubleshoot and test your network:AWSSupport-SetupIPMonitoringFromVPC to collect network metrics such as packet loss, latency, MTR, tcptraceroute, and tracepath.MTR to check for ICMP or TCP packet loss and latency problems.Traceroute to determine latency or routing problems.Hping3 to determine end-to-end TCP packet loss and latency problems.Tcpdump to analyze packet capture samples.Review hops on traceroute or MTR reports using a bottom-up approach. For example, check for loss on the last hop or destination, and then review the following hops. If the packet loss or latency issues continue through the last hop, there might be a network or routing issue. Packet loss or latency on one hop in the path might occur if there's an issue with the control plane rate limiting on that node. Check if the last hop reported is the destination noted in the command. If it isn't, then there might be an issue caused by a restrictive security group.Test performance using AWSSupport-SetupIPMonitoringFromVPCThis built-in tool collects many of the metrics that you need to troubleshoot your network. For more information, see Debugging tool for network connectivity from Amazon VPC.Performance troubleshooting for Linux instancesCheck the Linux performance statisticsIf you have access to the source instance or destination instance, check for issues with the CPU, memory utilization, and load average.Test performance using MTRThe Linux MTR command provides continual, updated output. This output allows you to analyze network performance. This diagnostic tool combines the functionality of traceroute and ping utilities. Most Linux distributions come with traceroute and MTR preinstalled. You can also download it from your distribution’s software package manager.To install MTR, run the following commands:Amazon Linux:sudo yum install mtrUbuntu:sudo apt-get install mtr-tinyTo test your network's performance using MTR, run this test bidirectionally between the public IP address of your EC2 instances and your on-premises host. The path between nodes on a TCP/IP network can change if the direction is reversed. Therefore, it's important to obtain MTR results for both directions. You can use a TCP-based trace instead of ICMP, because most internet devices deprioritize ICMP-based trace requests.Review your packet loss. Packet loss on a single hop usually doesn't indicate an issue. The loss can be the result of a control plane policy that causes the "ICMP time exceeded" messages to be dropped. If you notice sustained packet loss until the destination hop, or packet loss over several hops, this loss might indicate a problem.Note: It's common to see a few requests time out.ICMP-based MTR:mtr -n -c 200 <Public IP EC2 instance/on-premises host> --reportTCP-based MTR:mtr -n -T -c 200 <Public IP EC2 instance/on-premises host> --reportThe argument -T performs a TCP-based MTR, and the --report option puts MTR into report mode. MTR runs for the number of cycles specified by the -c option. Print the statistics, and then exit.Note: The TCP-based MTR tests the destination TCP port 80, to MTR a specific destination TCP port, appended with -P, followed by the port number. The following is an example to MTR destination TCP port 443:mtr -n -T -c 200 <Public IP EC2 instance/on-premises host> -P 443 --reportTest performance using tracerouteThe Linux traceroute utility identifies the path taken from a client node to the destination node. The utility records the time in milliseconds for each router to respond to the request. The utility also calculates the amount of time that each hop takes before reaching its destination.To install traceroute, run the following commands:Amazon Linux:sudo yum install tracerouteUbuntu:sudo apt-get updatesudo apt-get install tracerouteNote: Traceroute isn't necessary if you run an MTR report. MTR provides latency and packet loss statistics to a destination.Be sure that port 22 or the port that you're testing is open in both directions. To troubleshoot network connectivity using traceroute, run the command from the client to the server, and from the server back to the client. The path between nodes on a TCP/IP network can change if the direction is reversed. Use a TCP-based trace instead (your application port) of ICMP, because most internet devices deprioritize ICMP-based trace requests.ICMP-based traceroute:sudo traceroute -I <Public IP of EC2 instance/on-premises host>TCP-based traceroute:sudo traceroute -n -T -p 22 <Public IP of EC2 instance/on-premises host>The argument -T -p 22 -n performs a TCP-based trace on port 22.Note: You can use your application specific port for testing. Use the specific port to understand if there are any intermediate devices in the path dropping your application traffic.Test performance using hping3Hping3 is a command-line oriented TCP/IP packet assembler and analyzer that measures end-to-end packet loss and latency over a TCP connection. In addition to ICMP echo requests, hping3 supports TCP, UDP, and RAW-IP protocols. Hping3 also includes a traceroute mode that can send files between a covered channel. Hping3 is designed to scan hosts, assist with penetration testing, test intrusion detection systems, and send files between hosts.MTRs and traceroute capture per-hop latency. However, hping3 yields results that show end-to-end min/avg/max latency over TCP in addition to packet loss. To install hping3, run the following commands:Amazon Linux 2. Install the EPEL release package for RHEL 7, then activate the EPEL repository.sudo amazon-linux-extras install epel -yAmazon Linux 2:sudo yum --enablerepo=epel install hping3Ubuntu:sudo apt-get install hping3The following command sends 50 TCP SYN packets over port 0. By default, hping3 sends TCP headers to the target host's port 0, with a window size of 64 and without a TCP flag:sudo hping3 -S -c 50 -V <Public IP of EC2 instance/on-premises host>The following command sends 50 TCP SYN packets over port 22:sudo hping3 -S -c 50 -V <Public IP of EC2 instance/on-premises host> -p 22Note: Be sure that port 22 or the port that you're testing is open.Test packet capture samples using tcpdumpIt's a best practice to perform simultaneous packet captures on your EC2 instance and on-premises host when diagnosing packet loss/latency issues. Doing so can help to identify the request and response packets so that we can isolate the issue at the networking and application layers. It's also a best practice to first start the packet capture, then initiate the traffic. This helps capture all packets for the flow. To install tcpdump, run the following commands:Amazon Linux:sudo yum install tcpdumpUbuntu:sudo apt-get install tcpdumpAfter tcpdump is installed, you can run the following command to capture the tcp port 22 traffic and save it in a pcap file.sudo tcpdump -i eth0 port 22 -s0 -w samplecapture.pcapNote: The tcpdump flag -i specifies the interface on the instance where tcpdump captures the traffic. You might need to change the interface from eth0 to the configured interface in your environment.Performance troubleshooting for WindowsCheck for ECN capability1.    Run the following command to determine if Explicit Congestion Notification (ECN) capability is turned on:netsh interface tcp show global2.    If ECN capability is activated, run the following command to deactivate it:- netsh interface tcp set global ecncapability=disabled3.    If you don't see an improvement in performance, you can re-activate ECN capability using the following command:netsh interface tcp set global ecncapability=enabledReview hops and troubleshoot TCP port connectivityFirst, use MTR or tracert to review hops:MTR method:1.    Download and install WinMTR.2.    Enter the destination IP in the Host section, and then choose Start.3.    Let the test run for a minute, and then choose Stop.4.    Choose Copy text to clipboard and paste the output in a text file.5.    Look for any losses in the % column that are propagated to the destination.Note: Ignore any hops with the No response from host message. This message indicates that those particular hops aren't responding to the ICMP probes.6.    Review hops on the MTR reports using a bottom-up approach. For example, check for loss on the last hop or destination, and then review the preceding hops.Tracert method:If you don't want to install MTR, you can use the tracert command utility tool.1.    Perform a tracert to the destination URL or IP address.2.    Look for any hop that shows an abrupt spike in round-trip time (RTT). An abrupt spike in RTT might indicate that there's a node under high load, which in turn induces latency or packet drops in your traffic.Note: The -d option doesn't resolve IP addresses to hostnames. Remove -d if IP to hostname resolution is required.tracert -d <Public IP of EC2 instance/on-premises host>Then, check TCP port connectivity.Note: Because WinMTR and tracert are both ICMP-based, you can use tracetcp to troubleshoot TCP port connectivity.1.    Download WinPcap and tracetcp.2.    Extract the tracetcp ZIP file.3.    Copy tracetcp.exe to your C drive.4.    Install WinPcap.5.    Open the command prompt and root WinPcap to your C drive using the *C:\Users\username>cd * command.6.    Run tracetcp using the following commands: tracetcp.exehostname:port or tracetcp.exe ip:port.Check the Windows Task ManagerIf you have access to the source instance or destination instance, check the Windows Task Manager. Look for issues with CPU and memory utilization, or load average.Take a packet captureNote: It's a best practice to perform simultaneous packet captures on your EC2 instance and your on-premises host when diagnosing packet loss or latency issues. This helps to identify the request and response packets to isolate the issue at the networking and application layers. It's also a best practice to first start the packet capture and then initiate the traffic. This helps capture all packets for the flow.1.    Install Wireshark and take a packet capture.2.    Use the following filter to isolate the traffic between particular sources in the packet capture: (ip.addr eq source_IP) &&(tcp.flags.syn == 1). The output shows all the tcp streams initiated by that source IP.3.    Select the row with the relevant source IP and destination IP.4.    Choose the context (right-click) menu, and then choose Follow, TCP Stream. This results in a TCP flow between the source IP and destination IP that you want to investigate.5.    Look for retransmissions, duplicate packets, or TCP window size notifications like TCP window full or Window size zero. These notifications might indicate that the TCP buffers are running out of space.If you find packet loss, or if the number of hops changes significantly from your benchmarks, refer to your networking equipment vendor documentation. If working within a multi-homed network environment, perform these tests using a different ISP.Related informationEnhanced networking on LinuxEnhanced networking on WindowsFollow"


Question: How do I resolve a preupgrade check failure in Amazon RDS for MySQL?

"I want to perform a major upgrade from Amazon Relational Database Service (Amazon RDS) for MySQL 5.7 to MySQL 8.0. However, I'm experiencing a preupgrade check failure. How do I resolve this?"

Answer: "I want to perform a major upgrade from Amazon Relational Database Service (Amazon RDS) for MySQL 5.7 to MySQL 8.0. However, I'm experiencing a preupgrade check failure. How do I resolve this?Short descriptionIf your DB instance experiences a preupgrade check failure, then your Amazon RDS for MySQL version upgrade is stopped. When incompatibilities are detected, Amazon RDS provides a log file for you to review the details of the failed precheck.Note: There are different prechecks that are built into the MySQL database and Amazon RDS. Also, prechecks aren't run for upgrades for versions lower than MySQL 5.7. For upgrades from MySQL 5.6 to MySQL 5.7, prechecks are limited to detecting orphan tables and sufficient storage space to rebuild tables.ResolutionWhen you perform an upgrade from MySQL 5.6.x to 5.7.x, or from MySQL 5.7.x to 8.0.x, Amazon RDS automatically runs prechecks. The prechecks detect any data incompatibilities that might affect your upgrade. Prechecks are designed to minimize any unplanned downtime during a version upgrade.If you experience a preupgrade check failure during one of these version upgrades, review the PrePatchCompatibility log file generated by Amazon RDS. You can view the PrePatchCompatibility log file under Logs & eventsin the Amazon RDS console. Then, review the details of the listed incompatibility issues and correct the issue. In most cases, the log entry will include a link to the Amazon RDS for MySQL documentation on how to correct the incompatibility.Tip: It's a best practice to test your DB instance before a major version upgrade.Follow"


Question: How can I resolve 400 errors with access denied for AWS KMS ciphertext in AWS Glue?

I get a 400 error when running an AWS Glue crawler or AWS Glue ETL job.

Answer: "I get a 400 error when running an AWS Glue crawler or AWS Glue ETL job.Short descriptionWhen using AWS Glue, you might receive this error when you try to access AWS resources like Amazon Simple Service Solution (Amazon S3) buckets:"The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access."You receive this error when you access any AWS service that's encrypted with AWS Key Management Service (AWS KMS) keys using AWS Glue jobs or crawlers. The error is most likely to happen for one of these reasons:The principal trying to access the encrypted key doesn't have the required AWS KMS permissionsThe AWS KMS key doesn't exist or doesn't exist in a specific AWS RegionResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you're using the most recent AWS CLI version.Check if your AWS Glue role policy has the required permissionsFirst, check that the AWS Glue role policy you're using has the permissions needed to access the resource's AWS KMS key.For example, assume that you have an AWS Glue role called AWSGlue-MyGlueCustomRole. This role is accessing an S3 bucket that's encrypted with the AWS KMS key arn:aws:kms:us-east-1:XXXXXXXX9645:key/64d3e826-1a87-4cbc-95b8-fb386730a07. The AWS Glue role policy must have the following context in addition to its other permissions:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "kms:Decrypt",                "kms:Encrypt",                "kms:GenerateDataKey"            ],            "Resource": "arn:aws:kms:us-east-1:XXXXXXXX9645:key/64d3e826-1a87-4cbc-95b8-fb386730a07"        }    ]}Check if your AWS KMS key policy allows the AWS Glue roleThe AWS KMS key policy must allow the AWS Glue role as part of its allowed entity.Using the previous example, the AWS KMS key arn:aws:kms:us-east-1:XXXXXXXX9645:key/64d3e826-1a87-4cbc-95b8-fb386730a07 policy must have the following permissions attached:{    "Sid": "Allow use of the key",    "Effect": "Allow",    "Principal": {        "AWS": [            "arn:aws:iam::XXXXXXXX9645:role/AWSGlue-MyGlueCustomRole "        ]    },    "Action": [        "kms:Encrypt",        "kms:Decrypt",        "kms:ReEncrypt*",        "kms:GenerateDataKey*",        "kms:DescribeKey"    ],    "Resource": "*"}Make sure that the AWS KMS key exists in the same Region as your AWS Glue jobTo check which Region your AWS KMS key exists in, follow these steps:1.    Open the AWS KMS console.2.    Choose the Region that you're encountering the error in.3.    In the navigation pane, choose either AWS managed keys or Customer managed keys. Then search for the AWS KMS key that you're using. You can also use the AWS CLI to search for the AWS KMS key. The following command returns a NotFoundException error if the key doesn't exist in the Region:aws kms describe-key --key-id <Key-ID> --region <region-name>Note: Be sure to replace <Key-ID> and <region-name> in this example with the Key ID and name of the Region you're using.4.    Check the AWS CloudTrail API calls kms:Decrypt and kms:Encrypt to see if you observe any failures. Or, you can view all event logs by using kms.amazonaws.com as the event source.Cross-account exampleIf the AWS Glue role and the AWS KMS key are in different accounts, use the customer managed key, and not the AWS managed key. AWS KMS keys can be used only when the principals are from the same account. You can't modify the key policy for AWS managed keys. When you access the AWS KMS key in the second account, make sure that permissions are present both on the source and the destination.Assume you want to allow an AWS Identity and Access Management (IAM) user X in Account Y to an AWS KMS key in Account Z. In this case, the IAM policy must have the required AWS KMS actions. The same actions must be allowed from the AWS KMS key policy. For more information, see Allowing users in other accounts to use an AWS KMS key.Here are the most common issues that occur when accessing an AWS KMS key from a cross account.If the error message is caused by a service control policy (SCP), then check all the applied SCPs on your account. This helps you to identify the SCP that's restricting the kms:Decrypt action. Update the affecting SCP, and try your action again. For more information, see Listing the policies attached to a root, OU, or account and Updating an SCP. See the following example of this error:User: arn:aws:iam::XXXXXXXX9645:user/KMSDemO is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:us-east-1:XXXXXXXX9645:key/64d3e826-1a87-4cbc-95b8-fb386730a07 with an explicit deny in a service control policy"If the IAM entity has a permission boundary attached to it, then the boundary sets the maximum permissions that the entity has. Check the permissions boundary on the IAM principal to make sure that the required AWS KMS decrypt action is allowed. Then, try the action again. See the following example of this error:User: arn:aws:iam::XXXXXXXX9645:user/KMSDemO is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:us-east-1:XXXXXXXX9645:key/64d3e826-1a87-4cbc-95b8-fb386730a07 because no permissions boundary allows the kms:Decrypt action"If you're connecting to AWS KMS using a VPC endpoint, then check the applied VPC endpoint policy. Make sure that the kms:Decrypt action is allowed in the policy. See the following example of this error:User: arn:aws:iam::XXXXXXXX9645:user/KMSDemO is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:us-east-1:XXXXXXXX9645:key/64d3e826-1a87-4cbc-95b8-fb386730a07 because no VPC endpoint policy allows the kms:Decrypt action"Related informationSetting up encryption in AWS GlueKey policies in AWS KMSFollow"


Question: How can I troubleshoot Amazon SageMaker Ground Truth labeling errors?

I want to troubleshoot Amazon SageMaker Ground Truth labeling errors.-or-My SageMaker workers are idle.-or-It's taking a long time for tasks to show up for my SageMaker workers.

Answer: "I want to troubleshoot Amazon SageMaker Ground Truth labeling errors.-or-My SageMaker workers are idle.-or-It's taking a long time for tasks to show up for my SageMaker workers.ResolutionSageMaker Ground Truth first sends a batch of 10 tasks to your SageMaker workers for annotations. This batch is used to check and make sure that the labeling job is correctly configured. Then, Ground Truth sends larger batches of tasks to workers for annotations based on the MaxConcurrentTaskCount value.MaxConcurrentTaskCount defines the maximum number of data objects that can be labeled by human workers at the same time. If you use the console, this parameter is set to 1,000. If you use CreateLabelingJob, you can set this parameter to any integer between 1 and 1,000, inclusive.After Ground Truth receives the labels, it processes the labels with a consolidation AWS Lambda function. With this function, the final annotations are written to the manifest file or Amazon Simple Notification Service (Amazon SNS) output. Then, Ground Truth loops back to read another batch of tasks based on the MaxConcurrentTaskCount value from the input manifest file or Amazon SNS topic.Troubleshoot task latency and idle workersBe sure that the MaxConcurrentTaskCount value is set to a size that enables workers to complete the entire batch within the given TaskAvailabilityLifetimeInSeconds. The maximum value for this parameter is 1000.Be sure that NumberOfHumanWorkersPerDataObject is set to a value that fits your use case. For example, if the number of workers per object to label is set to 3, then each object needs to be labeled by three workers. If two of the workers finish the current batch, the next batch isn't assigned until the third worker has finished their batch. If a private worker notices that a job disappears from the portal, the worker might have finished one batch and is idle while they wait for a new batch to be available.Be sure that the TaskAvailabilityLifetimeInSeconds is set to a value that fits your use case. This value represents the total time that the tasks are available to the workers. The maximum value that you can set for this parameter is 864,000 seconds (10 days). It's a best practice to split your input dataset into multiple jobs and point them to same work team under the following conditions:The number of objects in the labeling job is high.Your job failed because the wait time exceeded the TaskAvailabilityLifetimeInSeconds value.Be sure that TaskTimeLimitInSeconds is set to a value that fits your use case. If you need to control the time taken by workers to complete a task to make sure that tasks are annotated and the next batch is assigned, consider setting an appropriate value for this time limit.Troubleshooting labeling errorsCheck permissionsBe sure that you have the right permissions to create a labeling job, access input data, and access the Amazon Simple Storage Service (Amazon S3) bucket for output data. For more information, see Step 1: Before you begin.Be sure of the following:The Amazon S3 bucket is in the same Region as the Ground Truth labeling job.The bucket has a CORS policy attached. For more information, see CORS permission requirement.Check the output manifest fileCheck the output manifest file that you specified in the S3 bucket to store the output files. In this output dataset, you can see the metadata for any failed annotations that might have led to failed labeling jobs.Example:{"source-ref":"s3://sagemaker-output-labeling-bucket-example/example.jpeg","example-metadata":{"retry-count":1,"failure-reason":"ClientError: Annotation tasks expired.  Probable Reasons are 1) TaskAvailabilityLifetimeInSeconds parameter is too small.  2) Reward is too low for workers to work on the task.  3) If you use a custom html template, your template may be broken.  4) Data (image/video/text) sent for annotation is broken or too big, preventing completion.  5) All workers declined the tasks.","human-annotated":"true"}}Workers are allowed to decline tasks because of unclear instructions, input data being broken (not displaying correctly), or some other issue with the task. If all workers decline, the object is marked as expired and not sent to any other workers.You can monitor whether workers decline, submit, or return a task using Amazon CloudWatch Events. For more information, see Monitor labeling job status.Check the input manifest fileBe sure that the input manifest file meets all the listed JSON object requirements. For more information, see Use an input manifest file.Related informationCreate a labeling jobControl the flow of data objects sent to workersMonitor labeling jobsFollow"


Question: Why can't I start or launch my EC2 instance?

I can't launch or start my Amazon Elastic Compute Cloud (Amazon EC2) instance.

Answer: "I can't launch or start my Amazon Elastic Compute Cloud (Amazon EC2) instance.Short descriptionThere are multiple reasons why you might not be able to launch or start your EC2 instance. If you don't receive an error code when you try to start the instance, first run the AWS Command Line Interface (AWS CLI) describe-instances command. Then, specify the instance ID to see the reason for the problem.In the following example, replace MYINSTANCE with the instance ID that you're trying to start.Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Run the describe-instances command:aws ec2 describe-instances --instance-id MYINSTANCE --output jsonCheck the StateReason message in the JSON response that the command returns.In the following example, the StateReason that's returned is Client.InternalError:"StateReason": {     "Message": "Client.InternalError: Client error on launch",     "Code": "Client.InternalError"   },ResolutionThe following are common errors that can prevent you from starting your instance:InsufficientInstanceCapacityYou receive the following error message:"An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 4). We currently do not have sufficient capacity in the Availability Zone you requested."This error occurs if there isn't enough On-Demand capacity to complete your request.For information on resolving this error, see How do I troubleshoot InsufficientInstanceCapacity errors when starting or launching an EC2 instance?InstanceLimitExceededYou receive the following error message:"Your quota allows for 0 more running instance(s)."This error occurs if you launched the maximum allowed instances in your AWS Region.For information on resolving this error, see How do I troubleshoot InstanceLimitExceeded errors when starting or launching an EC2 instance?UnauthorizedOperationThe UnauthorizedOperation error indicates that the user who's trying to launch the instance doesn't have the required permissions. To resolve this error, see How can I decode an authorization failure message after receiving an "UnauthorizedOperation" error during an EC2 instance launch?The requested configuration is currently not supportedEither of the following two issues cause this error:1.    The specified instance type isn't supported in the requested Availability Zone. To check the availability of the instance types in specific Availability Zones, run the aws ec2 describe-instance-type-offerings command in the AWS CLI. Then, launch the instance in a supported Availability Zone.For example, use the following command to check availability of the c5.2xlarge instance type across all Availability Zones in us-east-1:aws ec2 describe-instance-type-offerings --location-type availability-zone --filters Name=instance-type,Values=c5.2xlarge --region us-east-12.    The Amazon Machine Image (AMI) that's used to launch the instance doesn't support the selected instance type. Before you select the instance type or family, refer to the AMI descriptions in the AWS Marketplace. This issue commonly occurs when you use AWS Marketplace AMIs, and launch them using the AWS CLI. When you launch instances from the Amazon EC2 console, unsupported instances are unavailable.Client.InternalErrorThere are several reasons why you might receive this error:An Amazon Elastic Block Store (Amazon EBS) volume isn't correctly attached to the instance.An EBS volume that's attached to the instance is in an ERROR state.An encrypted EBS volume is attached to the instance. However, you don't have permissions to access the AWS Key Management Services (AWS KMS) for decryption.For information on resolving these issues, see How do I troubleshoot an Amazon EC2 instance that stops or terminates when I try to start it?Disk full errorsWhen you run out of space on an instance and the audit service stops the machine as soon as it's booted, a disk full error occurs.Note: Before you proceed, it's a best practice to create a snapshot to back up your volumes' data.Important: These resolution steps require you to start and stop the instance. Be aware of the following:When the instance is stopped, you lose the data in your instance store volumes. For more information, see Determine the root device type of your instance.If you stop an instance that's part of an Amazon EC2 Auto Scaling group, then you might terminate the instance. Instances that are launched with Amazon EMR, AWS CloudFormation, and AWS Elastic Beanstalk might be part of an AWS Auto Scaling group. Instance termination in this scenario depends on the instance scale-in protection settings for your Auto Scaling group. If your instance is part of an Auto Scaling group, then temporarily remove it from the Auto Scaling group before you proceed.Stopping and starting your instance changes instance's the public IP address. When you route external traffic to your instance, it's a best practice to use an Elastic IP address instead of a public IP address.To resolve disk full errors, do the following:1.    Open the Amazon EC2 console.2.    Choose Instances from the navigation pane, and then select the instance.3.    Stop the instance.4.    Detach the EBS root volume (/dev/xvda for Linux) from the stopped instance.5.    Launch a new EC2 instance in the same Availability Zone as the impaired instance. The new instance becomes your rescue instance.6.    Attach the root volume that you detached in step 4 to the rescue instance as a secondary device.Note: When you attach secondary volumes, you can use different device names.7.    Connect to your rescue instance using SSH.8.    Create a mount point directory for the new volume attached to the rescue instance in step 6. In the following example, the mount point directory is /mnt/rescue.Note: The following $ sudo commands apply only to Linux environments:$ sudo mkdir /mnt/rescue9.    Mount the volume at the directory that you created in step 8:$ sudo mount /dev/xvdf /mnt/rescueNote: The device (/dev/xvdf, in this example) might have a different device name for the rescue instance that it's attached to. To determine the correct device names, use the lsblk command to view your available disk devices along with their mount points.10.    Increase the root volume size.11.    Extend the partition.12.    Run the umount command to unmount the volume:$ sudo umount /mnt/rescue13.    Detach the volume from the temporary instance.14.    Attach the volume to the original instance with device name (/dev/xvda for Linux), and then start the instance to confirm that it boots successfully.15.    Review the audit service configuration to make sure that it doesn't have the option to halt (shutdown) the machine if it runs out of space. For Amazon Linux and Amazon Linux 2, make sure that the audit service configuration includes the following options in /etc/audit/auditd.conf:max_log_file_action = ROTATEadmin_space_left_action = SUSPENDdisk_full_action = SUSPENDdisk_error_action = SUSPENDRelated informationTroubleshoot instance launch issues - LinuxTroubleshoot instance launch issues - WindowsWhy is my instance terminating immediately after launching or starting an instance?Amazon ECS best practices guideFollow"


Question: How do I troubleshoot Amazon S3 Batch Operations issues?

"I want to create an Amazon Simple Storage Service (Amazon S3) Batch Operations job for objects in my bucket. However, Amazon S3 keeps returning an error or my batch job keeps failing."

Answer: "I want to create an Amazon Simple Storage Service (Amazon S3) Batch Operations job for objects in my bucket. However, Amazon S3 keeps returning an error or my batch job keeps failing.Short descriptionIf an Amazon S3 Batch Operations job encounters an issue that prevents it from running successfully, then the job fails. For example, if Amazon S3 can't read the specified manifest, or objects in your manifest don't exist in the specified bucket, then the job fails. A failed job generates one or more failure codes and reasons. Amazon S3 Batch Operations stores the failure codes and reasons with the job so that you can view them by requesting the job's details. You can also review your failure codes and reasons in the completion report for the job.To prevent jobs from running a large number of unsuccessful operations, Amazon S3 also imposes a task-failure threshold on every Batch Operations job. Amazon S3 monitors the task failure rate after at least 1,000 tasks run. If a job exceeds the failure rate of 50%, then the job fails. To resolve this failure, review the causes of the failures and correct them before resubmitting the job.Here are some common reasons that Amazon S3 Batch Operations fails or returns an error:Incorrect manifest file format (.csv or JSON)The manifest file specifies multiple bucket names or contains multiple header rowsMissing permissions to read the manifest fileDifferent batch job AWS RegionMissing target bucket for your S3 Inventory reportMissing AWS Identity Access Management (IAM) role's trust policyMissing IAM role permissions for creating a batch jobMissing IAM role access to the source bucket, S3 Inventory report, or destination bucketRestrictive AWS Organizations service control policy (SCP)ResolutionIncorrect manifest file format (.csv or JSON)Amazon S3 Batch Operations supports .csv and JSON (S3 Inventory report) manifest files. When you create a new batch job in Amazon S3, select or specify the correct manifest format for your manifest file:For the Amazon S3 Inventory report, make sure to use a CSV-formatted report. Also, specify the manifest.json file that's associated with the inventory report.For .csv files, each row in your manifest file must include the bucket name and object key. You can also include the object version. Object keys must be URL encoded. The manifest must either include version IDs for all objects or omit version IDs for all objects.Note: If the objects in your manifest are in a versioned bucket, then you must specify the version IDs for the objects. Otherwise, the batch job might fail. Or, Amazon S3 might apply the batch job to the latest version of the object, instead of the object that existed when you created the job.For more information about manifest files and formats, see Specifying a manifest.The manifest file specifies multiple bucket names or contains multiple header rowsWith S3 Batch Operations, all batch job operations require that all objects that are listed in the manifest file also exist in the same bucket. Otherwise, you receive the following error:Reasons for failure:Cannot have more than 1 bucket per Job. JOB_IDIf you perform any batch job operations, then make sure that your manifest file specifies only one bucket name. Additionally, the manifest file can't contain any header rows. For example, the following manifest file contains multiple header rows, and therefore Amazon S3 returns an error:bucket,keymy-batch-bucket,object001.txtmy-batch-bucket,object002.txtmy-batch-bucket,object003.txtmy-batch-bucket,object004.txtMissing permissions to read the manifest fileVerify that the IAM role that you use to create the S3 Batch Operations job has GetObject permissions to allow it to read the manifest file. You can do this by checking the object's metadata. Look for any mismatches in access with S3 Object Ownership. Also, look for any unsupported AWS Key Management Service (AWS KMS) keys that encrypt the manifest file.If you don't have permission to read the manifest file, then you get the following errors when you try to create an S3 Batch Operations job.See the following AWS CLI error example:Reason for failureReading the manifest is forbidden: AccessDeniedSee the following Amazon S3 console error example:Warning: Unable to get the manifest object's ETag. Specify a different object to continueNote: S3 Batch Operations supports CSV inventory reports that are AWS KMS-encrypted. S3 Batch Operations doesn't support .csv manifest files that are AWS KMS-encrypted. For more information, see Configuring inventory using the S3 console or Specifying a manifest.Different batch job RegionYou must create S3 Batch Operations copy jobs in the same Region as the destination bucket where you want to copy your objects to. Therefore, when you create your batch job, make sure to select the same Region as your destination bucket. For example, if your destination bucket is in the us-west-2 Region, then select us-west-2 as the Region for your batch job.Missing target bucket for your S3 Inventory reportConfirm that the target bucket for your S3 Inventory report exists. Also, confirm that the S3 bucket policy doesn't deny the s3:PutObject action. If the report is delivered to another AWS account, then confirm that the target bucket allows the IAM role to perform the s3:PutObject action.Missing IAM role's trust policyNote: Make sure that you specifying an IAM role and not an IAM user.Unlike an IAM user, an IAM role has a trust policy that defines which conditions must be met for other principals to assume it. To allow the S3 Batch Operations service principal to assume the IAM role, attach a trust policy to the role.The following example trust policy delegates access to Amazon S3 and reduces any risks that are associated with privilege escalation:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "Service": "batchoperations.s3.amazonaws.com"      },      "Action": "sts:AssumeRole"    }  ]}Missing IAM permissions for creating a batch jobBefore you create and run S3 Batch Operations jobs, grant the required permissions. If your IAM role is missing the required permissions to perform the S3 Batch Operations job, then the batch job fails.To create an S3 Batch Operations job, you must grant s3:CreateJob permissions. The same entity that creates the job must also have iam:PassRole permissions to pass the IAM role that's specified for the batch job. For more information about specifying IAM resources, see IAM JSON policy elements: Resource .Missing IAM role access to the source bucket, S3 Inventory report, or destination bucketCheck to make sure that the IAM role that you use for S3 Batch Operations has the required permissions to perform the batch job. For example, the IAM policy for the copy operation looks like the following example:{  "Version": "2012-10-17",  "Statement": \[    {      "Action": \[        "s3:PutObject",        "s3:PutObjectAcl",        "s3:PutObjectTagging"      \],      "Effect": "Allow",      "Resource": "arn:aws:s3:::{{DestinationBucket}}/\*"    },    {      "Action": \[        "s3:GetObject",        "s3:GetObjectAcl",        "s3:GetObjectTagging",        "s3:ListBucket"      \],      "Effect": "Allow",      "Resource": \[        "arn:aws:s3:::{{SourceBucket}}",        "arn:aws:s3:::{{SourceBucket}}/\*"      \]    },    {      "Effect": "Allow",      "Action": \[        "s3:GetObject",        "s3:GetObjectVersion"      \],      "Resource": \[        "arn:aws:s3:::{{ManifestBucket}}/\*"      \]    },    {      "Effect": "Allow",      "Action": \[        "s3:PutObject"      \],      "Resource": \[        "arn:aws:s3:::{{ReportBucket}}/\*"      \]    }  \]}For more information, see Granting permissions for Amazon S3 Batch Operations.Restrictive AWS Organizations SCPIf you use AWS Organizations, then confirm that there aren't any deny statements that might deny access to Amazon S3. For example, if your service control policy explicitly denies all S3 actions, then you might get an Access Denied error when you create a batch job.See the following example policy that explicitly denies all S3 actions:{  "Version": "2012-10-17",  "Statement": [    {      "Principal": "*",      "Effect": "Deny",      "Action": "s3:*",      "Resource": "*"    }  ]}If you intend to apply a restrictive policy, then you can allowlist the IAM role that S3 Batch Operations uses to perform the operation. See the following example:{  "Version": "2012-10-17",  "Statement": [    {      "Principal": "*",      "Effect": "Deny",      "Action": "s3:*",      "Resource": "*",      "Condition": {        "StringNotLike": {          "aws:userId": [            "AROAEXAMPLEID:*",            "AIDAEXAMPLEID",            "111111111111"          ]        }      }    }  ]}Follow"


Question: How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?

I want to allocate memory to work as a swap file in an Amazon Elastic Compute Cloud (Amazon EC2) instance. How do I do that?

Answer: "I want to allocate memory to work as a swap file in an Amazon Elastic Compute Cloud (Amazon EC2) instance. How do I do that?Short descriptionWhen physical RAM is already in use, Amazon EC2 instances use swap space as a short-term replacement for physical RAM.Contents of RAM that aren't in active use or that aren't needed as urgently as other data or instructions can be temporarily paged to a swap file. This frees up RAM for more immediate use.You can also create swap space on a partition. For more information, see How do I allocate memory to work as swap space on an Amazon EC2 instance using a partition on my hard drive?Note: It's a best practice to create swap space only on ephemeral storage instance store volumes.ResolutionBefore beginning this resolution, verify that your file system supports the usage of swap files.Note: Most of the common file systems (ext3, ext4, XFS) are supported. This may not be the same for other filesystems. It is the best practice to review the latest file system documentation.Calculate the swap space sizeFor information on how to calculate the size of your swap space, see the following:All about Linux swap space on the linux.com website.How much swap do I need? on the help.ubuntu.com website.Create a swap file1.    Use the dd command to create a swap file on the root file system. In the command, bs is the block size and count is the number of blocks. The size of the swap file is the block size option multiplied by the count option in the dd command. Adjust these values to determine the desired swap file size.The block size you specify should be less than the available memory on the instance or you receive a "memory exhausted" error.In this example dd command, the swap file is 4 GB (128 MB x 32):$ sudo dd if=/dev/zero of=/swapfile bs=128M count=322.    Update the read and write permissions for the swap file:$ sudo chmod 600 /swapfile3.    Set up a Linux swap area:$ sudo mkswap /swapfile4.    Make the swap file available for immediate use by adding the swap file to swap space:$ sudo swapon /swapfile5.    Verify that the procedure was successful:$ sudo swapon -s6.    Start the swap file at boot time by editing the /etc/fstab file.Open the file in the editor:$ sudo vi /etc/fstabAdd the following new line at the end of the file, save the file, and then exit:/swapfile swap swap defaults 0 0Related informationRHEL - Adding swap spaceFollow"


Question: How do I share AWS Glue Data Catalog databases and tables cross-account using AWS Lake Formation?

I want to share AWS Glue Data Catalog databases and tables cross-account using AWS Lake Formation.

Answer: "I want to share AWS Glue Data Catalog databases and tables cross-account using AWS Lake Formation.ResolutionWith Lake Formation's cross-account feature, you can grant access to other AWS accounts to write and share data to or from the data lake. The resources can be shared either through tag-based access control or named resources. This article focuses on granting cross-account access to Data Catalog resources using the named resource method.Be sure that the prerequisites are metKeep in mind the following prerequisites before you share your Data Catalog resources with another account or access resources shared from another account:Revoke Lake Formation permissionsRevoke all Lake Formation permissions from the IAMAllowedPrincipals group for the Data Catalog resource.Prevent new tables from having Super permissionsFor Data Catalog databases that contain tables that you might share, prevent new tables from having a default grant of Super to IAMAllowedPrincipals:Open the Lake Formation console.In the navigation pane, under Data Catalog, choose Databases.Select the database that you want to update.Choose Actions, and then choose Edit.Under Default permissions for newly created tables, clear Use only IAM access control for new tables in this database.Choose Save.For more information, see Super.Add permissions required for cross-account accessIf the AWS Glue Data Catalog resource policy is already enabled in the account, then you can either remove the policy or add new permissions to the policy that are required for cross-account grants. The following is an example resource policy for providing cross-account AWS Glue access to account 5555666677778888 from account 1111222233334444.For more information, see Granting cross-account access.{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "Service": "ram.amazonaws.com"      },      "Action": "glue:ShareResource",      "Resource": [        "arn:aws:glue:us-east-1:1111222233334444:table/*/*",        "arn:aws:glue:us-east-1:1111222233334444:database/*",        "arn:aws:glue:us-east-1:1111222233334444:catalog"      ]    },    {      "Effect": "Allow",      "Principal": {        "AWS": "arn:aws:iam::5555666677778888:root"      },      "Action": "glue:*",      "Resource": [        "arn:aws:glue:us-east-1:1111222233334444:table/*/*",        "arn:aws:glue:us-east-1:1111222233334444:database/*",        "arn:aws:glue:us-east-1:1111222233334444:catalog"      ]    }  ]}Enable sharing with organizationsIf the Data Catalog resources are shared across organizations, then enable sharing with AWS Organizations using the AWS RAM console. The AWS Identity and Access Management (IAM) user or role enabling this option must have the ram:EnableSharingWithAwsOrganization IAM permission.For more information, see Cross-account access prerequisites.Grant the required IAM permissionsSource account: To use the named resources method to grant cross-account permissions, you must have the required IAM permissions for AWS Glue and AWS Resource Access Manager (AWS RAM). You can choose the AWS managed policy AWSLakeFormationCrossAccountManager that grants these permissions or create a new policy based on this policy.Target account: Data lake administrators in target accounts must have the following additional policy. This policy allows the administrator to accept the AWS RAM resource share invitations and enable resource sharing with organizations:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Action": [        "ram:AcceptResourceShareInvitation",        "ram:RejectResourceShareInvitation",        "ec2:DescribeAvailabilityZones",        "ram:EnableSharingWithAwsOrganization"      ],      "Resource": "*"    }  ]}Note: The IAM user or role receiving the resource share invitation in AWS RAM must have the required IAM Permissions for glue:PutResourcePolicy.Share a database and its tables with the target accountTo share a database and all the tables in the database with the target account that's not part of the organization, do the following:Note: If you share all the tables in a database in the source account, then any new table created in the source account is automatically shared with the target account.In the source account, do the following:Open the Lake Formation console, and sign in as a data lake administrator.In the navigation pane, choose Databases.Select the database that you want to share.Choose Actions, and then choose Grant.Select External account.For AWS account ID or AWS organization ID, enter the account ID of the target account.For Table, be sure that All tables is selected.For Table permissions and Grantable permissions, select the access permissions that you want to grant.Choose Grant.In the target account, do the following:Open the AWS RAM console.In the navigation pane, under Shared with me, choose Resource shares.Review the list of resource shares to which you've been granted access.To accept the invitation for the resource shared from the source account, select the resource share ID, and choose Accept resource share.Open the Lake Formation console.In the navigation pane, choose Databases.You can view shared database in the listing. The Owner account ID for this database shows the account ID of the source account.Select the shared database, and then choose Actions.Choose Create resource link.In the Create resource link page, do the following:For Resource link name, enter the name of the resource link.For Shared database, be sure that the name of the shared database is selected.For Shared database's owner ID, enter the account ID of the source account.Choose Create.The resource link is created.Resource links are Data Catalog objects that are links to metadata databases and tables, typically to shared databases and tables from other AWS accounts. They help to enable cross-account access to data in the data lake. After the resource link is created, you can query the tables in the shared database with the data lake administrator access.To grant access to the IAM users/principals for the shared database, grant the required permissions for the resource link and the shared database. This allows the IAM users/principals to view the shared database and resource link in their Lake Formation console. The IAM users can also view the database and resource link in their Amazon Athena console or Amazon Redshift Spectrum.To grant access to IAM users for the resource link, do the following:Open the Lake Formation console, and sign in as a data lake administrator.In the navigation pane, choose Databases.Select the resource link that you created.Choose Actions, and then choose Grant.Under Principals, select IAM users and roles.For IAM users and roles, select the IAM user or principal for which you need to grant access.Under Resource link permissions, select Describe.Choose Grant.To grant access to the IAM users for the shared databases, do the following:Open the Lake Formation console, and sign in as a data lake administrator.In the navigation pane, choose Databases.Select the shared database.Choose Actions, and then choose Grant.Under Principals, select IAM users and roles.For IAM users and roles, select the IAM user or principal for which you need to grant access.Under Database permissions, select Describe.Note: This step provides the minimum permissions for the users to view the shared database.Choose Grant.To grant access to all or specific tables in the database, select the All tables option:Select the resource link.Choose Actions, and then choose Grant.Select IAM users and roles.For IAM users and roles, select the user/principal for which you want to grant access.Under LF-Tags or catalog resources, do the following:To grant access to all tables in the database, for Tables - optional, select All tables.To grant access to only specific tables in the database, for Tables - optional, select the tables.For Table permissions and Grantable permissions, select Select and Describe.Choose Grant.Note: You can grant only those permissions that you selected for Grantable permissions in the source account.After granting the required permissions, you can successfully query the table in Athena from the target account.Share only tables with the target accountTo share individual tables with the target account, follow the instructions from the previous section with the following changes.Source account:To grant access to the target account from the Lake Formation console, select the individual tables instead of selecting the database.Target account:Accept the resource share in the AWS RAM console to access the shared table in the Lake Formation console.Create a resource link for the shared table. After the resource link is created, you can query the shared table with the data lake administrator access.To grant access to the IAM users/principals for the shared table, you must grant permissions for the resource link.Review additional considerationsWhen you grant permissions on the table, you can restrict access only for the particular columns in the table. If you do so, the target account can view only those columns in the shared table.Be sure that the IAM users/principals from the target account have access to the Amazon Simple Storage Service (Amazon S3) path in the source account.If you revoke the permissions that were granted earlier from the source account, then the target account can't access the shared database/table. However, the resource link that you created in the target account isn't automatically deleted. You must manually delete the resource link.When you delete a database/table, the resource shares in AWS RAM aren't deleted automatically. Therefore, you must manually revoke the cross-account permissions before deleting a shared database or table.Related informationViewing shared Data Catalog tables and databasesCreating resource linksGranting data location permissions (external account)Granting and revoking permissions on Data Catalog resourcesHow AWS Lake Formation cross-account feature worksFollow"


Question: Why did my Amazon Redshift query status change from "Completed" to "Aborted" when no changes were made?

"The Amazon Redshift console shows that the query status is "Completed", but the status then changes to "Aborted". However, no updates were made to the table when I queried the results from a previous session or transaction. Why is this happening?"

Answer: "The Amazon Redshift console shows that the query status is "Completed", but the status then changes to "Aborted". However, no updates were made to the table when I queried the results from a previous session or transaction. Why is this happening?Short descriptionSQL statements that manipulate data or create database objects don't persist until the transaction is committed. This doesn't apply to TRUNCATE statements, which implicitly perform a COMMIT.The Amazon Redshift console shows that the query status is "Completed" for a SQL statement if it's still in an open transaction. The status changes to "Aborted" if the transaction is rolled back. The STL_QUERY system table also shows that the SQL statement is successfully completed when the aborted column value is 0.If the transaction is later committed, then the changes will appear. However, if the transaction can't be committed, the Amazon Redshift console shows that the query is aborted. To identify the reason that your transaction can't be committed, check the STL system tables.ResolutionTo confirm whether a transaction has been committed or rolled back, use the output of the following query on SVL_STATEMENTTEXT system table. Then, filter by the transaction ID (xid) of the SQL statement:SELECT *FROM SVL_STATEMENTTEXTWHERE xid IN (SELECT xid FROM STL_QUERY WHERE query = [Query ID]) ORDER BY starttime, sequence;The output on the query displays an "Undoing 1 transactions" statement for the rolled back transaction.If a transaction starts with a BEGIN statement, then the statement was explicitly opened by the user or application. The statement has to also be explicitly committed. Transactions that are not initiated with a BEGIN statement are usually auto-committed by the SQL client or driver's AUTO COMMIT option. If the option is disabled, the user has to explicitly send a COMMIT.When a transaction is properly committed, transaction's changes are durable (persisted) and can be seen by other XIDs that have been initiated after the COMMIT statement. For more information, see Serializable isolation.When there is no END, COMMIT, or "Undoing 1 transactions" message seen on the SVL_STATEMENTTEXT system table, the XID might still be open. Use the SVV_TRANSACTIONS view to identify open transactions and LOCK contention.The systems tables STL_COMMIT_STRATS and STL_UNDONE can also be used to confirm whether a transaction has ended with a COMMIT or a ROLLBACK.Run the following query to find out if the changes have been committed:SELECT q.query, q.xid, NVL2 (cs.endtime, cs.endtime::text, 'NO COMMIT') AS commit_endtimeFROM STL_QUERY q LEFT JOIN STL_COMMIT_STATS cs ON q.xid = cs.xid AND cs.node = -1WHERE q.query = [QUERY ID];Run the following query to find out if the changes have been rolled back:SELECT *FROM STL_UNDONEWHERE xact_id_undone IN (SELECT cid from STL_QUERY where query = [QUERY ID]);Transaction changes aren't persisted due to an explicit ROLLBACK command or if it doesn't run until completion. Explicit ROLLBACKS can't occur when there is a serializable isolation violation. They also can't occur when an administrator TERMINATES a session or CANCELS a query. Timeouts in a network connection can also keep transaction changes from persisting.If a rollback occurs, then the client receives an error message with more details. It's best practice to configure your client to log errors. For more information, see Configure logging (JDBC) or LogLevel (ODBC).Related informationSTL_DDLTEXTFollow"


Question: "I'm using OpenSSL to import my key into AWS KMS, but I'm getting an "InvalidCiphertext" error. How can I fix this?"

"I created an AWS KMS key in AWS Key Management Service (AWS KMS). I specified the KMS key material origin as external, chose RSAES_OAEP_SHA_256 as the algorithm for the wrapping key, and then used OpenSSL to encrypt my key material with the wrapping key. However, when I import my encrypted key material, the request fails, and I get an error message regarding "InvalidCiphertext". How can I avoid this import error?"

Answer: "I created an AWS KMS key in AWS Key Management Service (AWS KMS). I specified the KMS key material origin as external, chose RSAES_OAEP_SHA_256 as the algorithm for the wrapping key, and then used OpenSSL to encrypt my key material with the wrapping key. However, when I import my encrypted key material, the request fails, and I get an error message regarding "InvalidCiphertext". How can I avoid this import error?Short descriptionBy default, OpenSSL uses the SHA-1 hash function.To avoid import errors when you use the RSAES_OAEP_SHA_256 algorithm (SHA-256 hash function), encrypt your key material with OpenSSL using the openssl pkeyutl command, and specify the parameters –pkeyopt rsa_padding_mode:oaep and –pkeyopt rsa_oaep_md:sha256.The full procedure to import key material into AWS KMS using OpenSSL and RSAES_OAEP_SHA_256 is as follows:1.    Create a KMS key using external for the key material origin.2.    Download the wrapping (public) key and the import token.3.    Generate a 256-bit symmetric key.4.    Verify that your OpenSSL version supports openssl pkeyutl.5.    Encrypt your key material with the wrapping key.6.    Import your encrypted key material.Note: If you're using the RSAES_OAEP_SHA_1 encryption algorithm for your wrapping key, see How to import key material.ResolutionCreate a KMS key using External for the key material originRun the following AWS Command Line Interface (AWS CLI) commands to create a KMS for external key material.Note:If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Be sure to replace the export REGION and export KEY_ALIAS values with the details for your key.export REGION=us-east-1export KEY_ALIAS=kms_key_with_externalmaterialexport KEY_ID=`aws kms create-key --region $REGION --origin EXTERNAL --description $KEY_ALIAS --query KeyMetadata.KeyId --output text`aws kms --region $REGION create-alias --alias-name alias/$KEY_ALIAS --target-key-id $KEY_IDYour newly created KMS key status is Pending Import until you import the key material. Run the following command to view the key's status.aws kms --region $REGION describe-key --key-id $KEY_IDDownload the wrapping (public) key and the import tokenRun the following AWS CLI commands to get the PublicKey (wrapping key) and ImportToken values, decode both values using base64, and then store these values into separate files. Note that the RSAES_OAEP_SHA_256 wrapping algorithm is specified in the get-parameters-for-import command.export KEY_PARAMETERS=`aws kms --region $REGION get-parameters-for-import --key-id $KEY_ID --wrapping-algorithm RSAES_OAEP_SHA_256 --wrapping-key-spec RSA_2048`echo $KEY_PARAMETERS | awk '{print $7}' | tr -d '",' | base64 --decode > PublicKey.binecho $KEY_PARAMETERS | awk '{print $5}' | tr -d '",' | base64 --decode > ImportToken.binThese commands store the wrapping key in PublicKey.bin and the import token in ImportToken.bin.Generate a 256-bit symmetric keyThe key material must be a 256-bit (32-byte) symmetric key. Run one of the following commands to generate the key:OpenSSLopenssl rand -out PlaintextKeyMaterial.bin 32-or-dddd if=/dev/urandom of=PlaintextKeyMaterial.bin bs=32 count=1Verify that your OpenSSL version supports openssl pkeyutlTo encrypt your key material with RSAES_OAEP_SHA_256 as the wrapping algorithm, use the openssl pkeyutl command.Important: This command is available only on OpenSSL 1.0.2 and later versions.If you're using a RHEL-based Linux computer, such as an Amazon Elastic Compute Cloud (Amazon EC2) instance launched with an Amazon Linux AMI, then follow these steps.1.    Run the following command to check your OpenSSL version.openssl version2.    To update OpenSSL, run the following command.sudo yum –y update opensslIf you're using macOS, then follow these steps.1.    Run the following Homebrew commands.brew updatebrew upgrade opensslbrew info openssl2.    The last command shows you that OpenSSL is installed in /usr/local/opt/openssl/bin/. Confirm the OpenSSL version by running this command./usr/local/opt/openssl/bin/openssl version3.    To always use the most recent OpenSSL version, edit ~/.bash_profile and add the following line at the end of the file:export PATH="/usr/local/opt/openssl/bin:$PATH"4.    After adding the line, run the following command.source ~/.bash_profile5.    To verify the change in your macOS environment, run the following command.echo $PATHopenssl versionEncrypt your key material with the wrapping keyRun the following command to encrypt your key material using the most recent version of OpenSSL and the wrapping key that you previously downloaded.Note: The wrapping key is stored in PublicKey.bin.openssl pkeyutl -in PlaintextKeyMaterial.bin -out EncryptedKeyMaterial.bin -inkey PublicKey.bin -keyform DER \            -pubin -encrypt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256The command generates EncryptedKeyMaterial.bin. You import this value as the encrypted key material into the KMS.Import your encrypted key materialRun the following AWS CLI command to import the encrypted key material into the KMS key.aws kms --region $REGION import-key-material --key-id $KEY_ID --encrypted-key-material fileb://EncryptedKeyMaterial.bin --import-token fileb://ImportToken.bin --expiration-model KEY_MATERIAL_DOES_NOT_EXPIREAfter the import completes, the key's status changes to Enabled. Run the following command to verify the key's status.aws kms --region $REGION describe-key --key-id $KEY_IDRelated informationAbout imported key materialFollow"


Question: How can I see the patches that Patch Manager will install on my Amazon EC2 instance?

I want to see the patches that AWS Systems Manager Patch Manager will install on my Amazon Elastic Compute Cloud (Amazon EC2) instances. How can I do that?

Answer: "I want to see the patches that AWS Systems Manager Patch Manager will install on my Amazon Elastic Compute Cloud (Amazon EC2) instances. How can I do that?Short descriptionSystems Manager Patch Manager allows you to orchestrate your patching operations. You can scan instances to only view a report of missing patches, or you can scan and automatically install all missing patches.Patch Manager uses patch baselines, which include rules to auto-approve patches within days of their release. Patch baselines also include a list of approved and rejected patches. During a Scan operation, Patch Manager determines the instance patch compliance state based on the patch baseline. For more information about how patch baselines define the patches that will install on your instances, see About predefined and custom patch baselines.Patch Manager provides predefined patch baselines that can be customized for each supported operating system (OS). If the predefined patch baselines don’t meet your requirements, you can create your own custom patch baselines. Custom patch baselines allow you more control over the patches that are approved or rejected for your environment. You can also choose to use a patch group to associate instances with a specific patch baseline.ResolutionBefore you begin, confirm that you meet the Patch Manager prerequisites.Review or create a patch baselineReview the predefined patch baseline for each OS that you use. If the default baseline doesn’t meet your needs, create a custom patch baseline to define a standard set of patches for the selected instance type.If you choose to create a custom patch baseline, then set the custom baseline as the default patch baseline.(Optional) Organize instances into patch groupsYou can choose to organize your instances into patch groups using Amazon EC2 tags.Note: The AWS-RunPatchBaseline Systems Manager RunCommand document performs patching operations on instances for security and other types of updates. If a patch group isn’t specified, the document uses the patch baseline currently specified as the default for an OS type. If a patch group is specified, the document uses the patch baselines associated with the patch group.Run the Scan operationChoose an AWS Systems Manager tool to run a Scan operation. For Operation, select Scan.Note: To generate the patch states report, "AWS-RunPatchBaseline" document must run at least once on the instance.View the list of patches that Patch Manager will installUsing the Systems Manager consoleYou can use the Patch Manager Reporting tab in the Systems Manager console to find the patch compliance states reported by the AWS-RunPatchBaseline.Open the Systems Manager console, and then choose Patch Manager from the navigation pane.From the Reporting tab, select the instance that you want to see missing patches for.From the bottom pane, choose the Missing patches count hyperlink to see the list of missing patches.(Optional) To generate patch compliance reports, see Generating .csv patch compliance reports (console).Using the AWS Command Line Interface (AWS CLI)Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Before you begin, verify that you have AWS Identity and Access Management (IAM) permissions for the DescribeInstancePatches API.You can use the describe-instance-patches command to find the patch compliance states reported by the AWS-RunPatchBaseline.Run the following command to get an overall count report of the patch compliance states, including the missing count. Replace InstanceID with your EC2 instance ID.aws ssm describe-instance-patch-states --instance-ids "InstanceID"To isolate the patches that are approved in the baseline but that aren’t installed on the instance, apply the Missing state filter. The output lists the missing patches. Replace InstanceID with your Amazon EC2 instance ID, and RegionID with your AWS Region.aws ssm describe-instance-patches --instance-id "InstanceID" --filters Key=State,Values=Missing --region RegionIDRelated informationTroubleshooting PatchBaseline module is not downloadableHow security patches are selectedHow patches are installedUpdating or deleting a custom patch baseline (console)Get patch compliance details for an instanceFollow"


Question: How can I troubleshoot VPC endpoints in my private Elastic Beanstalk environment network configuration?

I want to troubleshoot networking problems in my completely private AWS Elastic Beanstalk environment.

Answer: "I want to troubleshoot networking problems in my completely private AWS Elastic Beanstalk environment.Short descriptionCompletely private environments have the following setup andallow access to your web application only from within the same Amazon Virtual Private Cloud (Amazon VPC).Load balancer visibility: PrivateLoad balancer subnets: Both private subnetsInstance public IP: DisabledInstance subnets: Both private subnetsAll communication across the VPC takes place through configured VPC endpoints. Also, all communication within the VPC takes place through the local route in the route table. The following are common errors that you might see in the environment because of networking configurations:The Amazon Elastic Compute Cloud (Amazon EC2) instances failed to communicate with Elastic Beanstalk. Communication failure is due to configuration problems with the VPC or a failed EC2 instance. Check your VPC configuration, and try launching the environment again.The instances didn't respond in the allowed command timeout time.ResolutionWhen creating a completely private Elastic Beanstalk environment, make sure that you have the following configurations set up:You must have a VPC with at least two private subnets in different Availability Zones.The VPC must have DNS hostnames and DNS resolutions turned on. Having these turned on adds a DNS entry in your VPC that maps the public service endpoint to the interface VPC endpoint.You must create VPC endpoints so that AWS services outside of the VPC and the instance inside of the VPC can communicate through these endpoints.Check VPC endpointsA standard Elastic Beanstalk environment must have the following VPC endpoints for successful creation:com.amazonaws.your-region.elasticbeanstalk (Elastic Beanstalk interface endpoint)com.amazonaws.your-region.elasticbeanstalk-health (Elastic Beanstalk health interface endpoint)com.amazonaws.your-region.cloudformation (AWS CloudFormation interface endpoint)com.amazonaws.your-region.logs (logs interface endpoint)com.amazonaws.your-region.sqs (Amazon Simple Queue Service (Amazon SQS) interface endpoint)com.amazonaws.your-region.s3 (Amazon Simple Storage Service (Amazon S3) gateway endpoint)Environments running on the Docker platform that runs on 64 bit Amazon Linux 2, must have the following additional VPC endpoints:com.amazonaws.your-region.ecr.dkrcom.amazonaws.your-region.ecr.apiEnvironments running on the Amazon Elastic Container Service (Amazon ECS) platform that runs on 64 bit Amazon Linux 2, must have the following additional VPC endpoints:com.amazonaws.your-region.ecscom.amazonaws.your-region.ecr.apicom.amazonaws.your-region.ecr.dkrNote: To make sure that your Docker and Multi-Docker environment work properly, provide a private Docker image.Check the security groupWhen creating the interface VPC endpoint, you must choose the subnets and the security group for your VPC endpoint. Make sure that you choose private subnets and the security group with the appropriate inbound rules.The security group that's attached to the interface VPC endpoints must have an inbound rule for port 443 (HTTPS):Inbound rule:TypeProtocolPort rangeSourceHTTPSTCP443IP address of client (refer note)Note: To protect incoming traffic through the VPC endpoint, consider specifying only the IP address or network of the client in the Source.Also, verify that the security group that's associated to the instance allows HTTP traffic from the load balancer’s security group:Inbound rule:TypeProtocolPort rangeSourceHTTPTCP80Load balancer’s security groupVerify that the security group that's associated with the load balancer allows all HTTP traffic in the inbound rule:Inbound rule:TypeProtocolPort rangeSourceHTTPTCP800.0.0.0/0Note: You don't need to restrict the egress traffic in any of the preceding security groups.Check for route table entriesWhen creating gateway endpoints, you have the option to choose a route table. Make sure that you choose the route table that's associated with the private subnet of the VPC. The route table that's associated with the private subnet must have a route for each gateway VPC endpoint. For example, you must have a route for each gateway VPC endpoint that's associated with Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB.Route table entries:DestinationTarget172.xx.x.0/16localpl-63a5xxxvpce-04xxxCheck the VPC endpoint policiesThe VPC endpoint policies must allow your Elastic Beanstalk environment to communicate privately to the service. When you create an interface or gateway endpoint, you can attach an endpoint policy. The endpoint policy controls which AWS principals can use the VPC endpoint to access the endpoint service. AWS principals include AWS accounts, AWS Identity and Access Management (IAM) users, and IAM roles. If you don't specify an endpoint policy when you create an endpoint, then by default the endpoint has full access.The following is a sample S3 Gateway VPC endpoint minimum policy that's required for an S3 endpoint when working with Elastic Beanstalk.{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "EBBucketAccess",            "Effect": "Allow",            "Principal": "*",            "Action": [                "s3:GetObject",                "s3:ListBucket",                "s3:PutObject",                "s3:PutObjectAcl",                "s3:DeleteObject",                "s3:PutBucketPolicy",                "s3:PutObjectVersionAcl"            ],            "Resource": [                "arn:aws:s3:::elasticbeanstalk-*",                "arn:aws:s3:::elasticbeanstalk-*/*",                "arn:aws:s3:::cloudformation*/*"            ]        }    ]}Related informationAccess an AWS service using an interface VPC endpointFollow"


Question: Why am I unable to DROP an object in my Amazon Redshift cluster?

I'm unable to drop a table or a view in my Amazon Redshift cluster. Why is this happening?

Answer: "I'm unable to drop a table or a view in my Amazon Redshift cluster. Why is this happening?Short descriptionYou might not be able to drop an object (such as a table or a view) in your Amazon Redshift cluster for the following reasons:Insufficient permissions: The user doesn't have the proper permissions to drop the object. (The user must be an owner of the object or have admin permissions.)Object dependency: The table columns are being referred to by another view or tableLock contention: A transaction is holding a lock on the object, which causes the drop operation to hang.ResolutionInsufficient permissionsIn Amazon Redshift, only the owner of the table, the schema owner, or a superuser can drop a table.To confirm user permissions and ownership, create a view using the v_get_obj_priv_by_user.sql script:CREATE OR REPLACE VIEW admin.v_get_obj_priv_by_userASSELECTObject dependencyYour drop operation might fail with the following error message:Invalid operation: cannot drop table/view <object_name> because other objects depend on itThe Invalid operation error indicates object dependencies on the target object.To identify the objects that depend on the target table, create these three views:A view to identify the constraint dependencyA view to identify the dependent viewsAn object view which aggregates the two previous viewsAfter you create these three views, use the v_object_dependency.sql script to get the dependent objects of the target object:select * from admin.v_object_dependency where src_objectname=<target object>After checking all the dependent objects, drop all the related objects along with the target object by using the CASCADE parameter:drop table <target object> cascade;Lock contentionIf the drop command hangs or doesn't output anything when you perform a drop, a transaction might be holding a lock on the object. As a result, you can't acquire the AccessExclusiveLock on the table. The AccessExclusiveLock is required to drop an object.To identify any locks, use the following syntax:select a.txn_owner, a.txn_db, a.xid, a.pid, a.txn_start, a.lock_mode, a.relation as table_id,nvl(trim(c."name"),d.relname) as tablename, a.granted,b.pid as blocking_pid ,datediff(s,a.txn_start,getdate())/86400||' days '||datediff(s,a.txn_start,getdate())%86400/3600||' hrs '||datediff(s,a.txn_start,getdate())%3600/60||' mins '||datediff(s,a.txn_start,getdate())%60||' secs' as txn_durationfrom svv_transactions a left join (select pid,relation,granted from pg_locks group by 1,2,3) b on a.relation=b.relation and a.granted='f' and b.granted='t' left join (select * from stv_tbl_perm where slice=0) c on a.relation=c.id left join pg_class d on a.relation=d.oidwhere  a.relation is not null;And once you identify the locking transaction either COMMIT the blocking transaction or terminate the session of the blocking transaction if it is no longer necessary by :select pg_terminate_backend(PID);After you identify the locks, use PG_TERMINATE_BACKEND to release the locks. For more information about detecting and releasing locks in Amazon Redshift, see How do I detect and release locks in Amazon Redshift?Follow"


Question: How can I use SCPs and tag policies to prevent users in my AWS Organizations member accounts from creating resources?

I want to prevent users in my AWS Organizations member accounts from creating AWS resources using a Service control policies (SCPs) or tag policies.

Answer: "I want to prevent users in my AWS Organizations member accounts from creating AWS resources using a Service control policies (SCPs) or tag policies.Short descriptionSCPs can be used to manage permissions in your organization, but not to grant permission. For more information, see Service control policies (SCPs).Tag policies can be used to maintain standardized tags with AWS resources for accounts with Organizations. For more information, Tag policies.ResolutionUse the following SCP or tag policy based on your use case.Use tag policies to prevent tagging on existing resourcesThe tag policies are checked when you perform operations that affect the tags on an existing resource. For example, tag policies can enforce that users can't change the specified tag on AWS resources to a non-compliant tag.The following example tag policy allows the tag key-value pair as Environment-Production enforced for Amazon Elastic Compute Cloud (Amazon EC2) instances. The policy prevents users from changing this tag on existing Amazon EC2 instances, but it doesn't prevent launching new instances with non-compliant tags, or no tags.{  "tags": {    "Environment": {      "tag_key": {        "@@assign": "Environment"      },      "tag_value": {        "@@assign": [          "Production"        ]      },      "enforced_for": {        "@@assign": [          "ec2:instance"        ]      }    }  }}Use SCPs to prevent tagging for creating new resourcesYou can use SCPs to prevent the creation of new AWS resources that aren't tagged for your Organization’s tagging restriction guidelines. To make sure that the AWS resources are created only if a certain tag is present, use the example SCP policy to require a tag on specified created resources.Related informationWhat's the difference between an AWS Organizations service control policy and an IAM policy?Follow"


Question: How do I prevent an IAM user or role from setting up Amazon SageMaker Canvas?

I want to prevent AWS Identify and Access Management (IAM) users and AWS IAM Identity Center (successor to AWS Single Sign-On) users from setting up Amazon SageMaker Canvas.

Answer: "I want to prevent AWS Identify and Access Management (IAM) users and AWS IAM Identity Center (successor to AWS Single Sign-On) users from setting up Amazon SageMaker Canvas.ResolutionTo prevent an IAM user or role from setting up the SageMaker Canvas app, first create an IAM policy to deny the required permissions. Then, attach this policy to the SageMaker execution role.Do the following:1.    Open the IAM console.2.    In the navigation pane, choose Policies.3.    Choose Create policy, and then choose the JSON tab.4.    Copy and paste the following IAM policy in the policy editor:{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "AllowSageMakerCreateAppOperations",      "Effect": "Allow",      "Action": "sagemaker:CreateApp",      "Resource": "*"    },    {      "Sid": "DenySageMakerCanvasCreateApp",      "Effect": "Deny",      "Action": "sagemaker:CreateApp",      "Resource": "arn:aws:sagemaker:example-region:1111222233334444:app/example-domain/example-user-name/canvas/*"    }  ]}Be sure to replace the following in the policy:example-region with the Region of your choice1111222233334444 with your account ID.example-domain with your SageMaker Studio domain ID.example-user-name with your SageMaker Studio user profile name.5.    Resolve any security warnings, errors, or general warnings generated during policy validation, and then choose Review policy.6.    Choose Next: Tags.7.    On the Review policy page, enter a Name and a Description (optional) for the policy that you're creating. Review the policy Summary, and then choose Create policy to save your work.8.    In the list of policies displayed, choose the policy that you created.9.    Choose the Policy usage tab, and then choose Attach.10.    From the list of IAM users and roles that appears, select the SageMaker execution role for the Studio user.11.    Choose Attach policy.If you try to set up the SageMaker Canvas app after completing the preceding steps, you get the following error:SageMaker is unable to use your associated ExecutionRole [<SageMaker Studio User Execution Role>] to create app. Verify that your associated ExecutionRole has permission for 'sagemaker:CreateApp'.Related informationAmazon SageMaker CanvasFollow"


Question: How do I troubleshoot the "ResourceLimitExceeded" error in Amazon SageMaker?

I want to troubleshoot the "ResourceLimitExceeded" error in Amazon SageMaker.

Answer: "I want to troubleshoot the "ResourceLimitExceeded" error in Amazon SageMaker.ResolutionWhen you create a SageMaker resource, such as a processing job, training job, endpoint, or Studio app, you might get the ResourceLimitExceeded error. See the following example error message:"The account-level service limit 'ml.m5.xlarge for endpoint usage' is 0 Instances, with current utilization of 0 Instances and a request delta of 1 Instances. Please contact AWS support to request an increase for this limit."This error usually occurs when you exceed the account level service quotas that are specified for your SageMaker resources.To resolve this error, complete the following steps:1.    Open the Service Quotas console.Note: To use the Service Quotas console, you need the corresponding AWS Identity and Access Management (IAM) permission in your user or role.2.    From the AWS Region selector on the navigation bar, select the Region where you get the error.3.    In the navigation pane, choose AWS services.4.    In the Search bar, enter Amazon SageMaker.5.    Choose Amazon SageMaker.6.    Select the quota that you want to increase. For the preceding example error message, select ml.m5.xlarge for endpoint usage.7.    Choose Request quote increase.8.    For Change quota value, enter the desired value.9.    Choose Request.This sends your request to AWS Support. Based on your use case and current usage, your request is either approved, denied, or partially approved.Related informationAWS service quotasSageMaker service quotasFollow"


Question: Why can't my Amazon EC2 instance in a private subnet connect to the internet using a NAT gateway?

"I created a NAT gateway to reach the internet from my Amazon Elastic Compute Cloud (Amazon EC2) instance. My instance uses HTTP or HTTPS ports in a private subnet, but isn't able to reach the internet."

Answer: "I created a NAT gateway to reach the internet from my Amazon Elastic Compute Cloud (Amazon EC2) instance. My instance uses HTTP or HTTPS ports in a private subnet, but isn't able to reach the internet.ResolutionVerify that the instances meet the following conditions:1.    The destination is reachable by pinging the destination from another source using a public IP address.2.    The NAT gateway is in the Available state. If the NAT gateway is in the Failed state, then see NAT gateway creation fails.Note: A NAT gateway in the Failed state automatically deletes after about an hour.3.    You created your NAT gateway in a public subnet, and the public route table has a default route pointing to an internet gateway.4.    The private subnet's route table has a default route pointing to the NAT gateway.Important: Make sure that you're not using the same route table for both the private and the public subnet. Using the same route table means that traffic isn't routed to the internet.5.    The enableDnsSupport attribute is set to true in the VPC. For more information, see View and update DNS attributes for your VPC.Note: Turn on DNS to prevent DNS resolution failure.6.    Firewalls aren't blocking traffic over ports 80 (for HTTP traffic) and 443 (for HTTPS traffic). Make sure that you check for a firewall blocking traffic on the destination host. You can use the following example command to check for firewalls:$ telnet PUBLIC_IP TCP_PORT7.    The security group attached to the instance's elastic network interface allows outbound traffic to ports 80 and 443. For more information, see Amazon EC2 security groups for Linux instances or Amazon EC2 security groups for Windows instances.8.    The following examples have rules allowing inbound and outbound traffic on ports 80 and 443 using the destination IP address 0.0.0.0/0:The network Access Control Lists (ACLs) associated with the private subnet where the instance is located.The network ACLs associated with the public subnet where the NAT Gateway is located.To allow your Amazon EC2 instances to access an HTTPS website, the network ACL associated with the NAT gateway subnet must have the following rules:Inbound rulesSourceProtocolPort RangeAllow / DenyVPC CIDRTCP443ALLOWPUBLIC_IPTCP1024-65535ALLOWOutbound rulesDestinationProtocolPort RangeAllow / DenyPUBLIC_IPTCP443ALLOWVPC CIDRTCP1024-65535ALLOWFor more information on configuring network ACLs, see Work with network ACLs.Related informationNAT gatewaysConfigure route tablesFollow"


Question: Why does my Amazon OpenSearch Service cluster have more nodes than I originally provisioned?

"In the Monitoring tab of the Amazon OpenSearch Service console, a cluster appears to have twice the number of nodes than originally provisioned. Why is this happening?"

Answer: "In the Monitoring tab of the Amazon OpenSearch Service console, a cluster appears to have twice the number of nodes than originally provisioned. Why is this happening?ResolutionAmazon OpenSearch Service uses blue/green deployments to make most cluster configuration changes. During this process, OpenSearch Service provisions a new cluster with the specified number of nodes, copying the entire dataset over from the existing cluster. When the data migration is complete, OpenSearch Service terminates the existing cluster, and the number of nodes returns to normal.In most cases, the following changes don't trigger a blue/green deployment:Access policiesAutomated snapshot hourEnabling auto-tune or disabling auto-tune without rolling back changesIf your domain has dedicated master nodes, changing the data node or UltraWarm node countFor more information about configuration changes and blue/green deployment, see Making configuration changes in OpenSearch Service.Related informationCreating and managing Amazon OpenSearch Service domainsFollow"


Question: Why is my AWS CloudFormation stack stuck in the REVIEW_IN_PROGRESS state?

My AWS CloudFormation stack is stuck in the REVIEW_IN_PROGRESS state.

Answer: "My AWS CloudFormation stack is stuck in the REVIEW_IN_PROGRESS state.Short descriptionA stack that's stuck in the REVIEW_IN_PROGRESS state means that you tried to create the stack using a change set that wasn't executed. When you create a change set for a new stack, CloudFormation creates a unique stack ID but no resources. If you don't execute the change set, then the stack remains in the REVIEW_IN_PROGRESS state.Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.ResolutionComplete the following steps to execute a change set using your preferred method.Using the AWS CloudFormation consoleOpen the AWS CloudFormation console.In the navigation pane, choose Stacks, and then find the stack that's stuck.Choose the Change Sets tab.Select the latest change set and review the changes.Choose Execute to create the stack.Using AWS Command Line Interface (AWS CLI)1.    Run the following list-change-sets command to list all the change sets:aws cloudformation list-change-sets --stack-name StackName2.    Identify the change set, and run the following execute-change-set command:aws cloudformation execute-change-set -–change-set-name ChangeSetNameUsing AWS CodePipeline consoleUsing AWS CodePipeline to deploy to CloudFormation with the Create or replace a change set action mode, only creates or replaces the change set. It doesn't automatically execute it. To execute the change set, you must add the Execute a change set action mode.Open the AWS CodePipeline console.Identify the pipeline that's deploying to CloudFormation.Choose Edit, and then choose Edit Deploy stage.Choose Add Action group.In the Action name, enter the name of your action.For Action provider, select AWS CloudFormation.For Action mode, select Execute a change set.Choose Save.Using AWS CLI1.    Run the following get-pipeline command to copy the pipeline structure into a JSON file:aws codepipeline get-pipeline --name MyPipeline >pipeline.json2.    Add CHANGE_SET_EXECUTE as the ActionMode in the Deploy stage:{  "name": "Deploy",  "blockers": null,  "actions": [    {      "name": "Deploy",      "actionTypeId": {        "category": "Deploy",        "owner": "AWS",        "provider": "CloudFormation",        "version": "1"      },      "runOrder": 1,      "configuration": {        "TemplatePath": "SourceArtifact::ssm.yml",        "ActionMode": "CHANGE_SET_REPLACE",        "Capabilities": "CAPABILITY_IAM,CAPABILITY_NAMED_IAM,CAPABILITY_AUTO_EXPAND",        "ChangeSetName": "CFNTest-CS",        "RoleArn": "arn:aws:iam::xxxxx:role/xxx",        "StackName": "CFNTest"      },      "outputArtifacts": [],      "inputArtifacts": [        {          "name": "SourceArtifact"        }      ],      "roleArn": null,      "region": "us-east-1",      "namespace": "DeployVariables"    },    {      "name": "ExecuteChangeSet",      "actionTypeId": {        "category": "Deploy",        "owner": "AWS",        "provider": "CloudFormation",        "version": "1"      },      "runOrder": 2,      "configuration": {        "ActionMode": "CHANGE_SET_EXECUTE",        "ChangeSetName": "CFNTest-CS",        "StackName": "CFNTest"      },      "outputArtifacts": [],      "inputArtifacts": [        {          "name": "SourceArtifact"        }      ],      "roleArn": null,      "region": "us-east-1",      "namespace": null    }  ]}3.    Run the following update-pipeline command to copy the pipeline into a JSON file:aws codepipeline update-pipeline --cli-input-json file://pipeline.jsonFollow"