Question: How can I resolve SageMaker Python SDK rate exceeded and throttling exceptions?

How can I resolve throttling errors such as "botocore.exceptions.ClientError: An error occurred (ThrottlingException)" when using the Amazon SageMaker Python SDK?

Answer: "How can I resolve throttling errors such as "botocore.exceptions.ClientError: An error occurred (ThrottlingException)" when using the Amazon SageMaker Python SDK?Short descriptionAPI calls to any AWS service can't exceed the maximum allowed API request rate per account and per AWS Region. These API calls might be from an application, the AWS Command Line Interface (AWS CLI), or the AWS Management Console. If the API requests exceed the maximum rate, then you receive the “Rate Exceeded” error, and the API calls are throttled.You might get this error when calling the SageMaker APIs because of the default retry configuration in Boto3. You can override this configuration to increase the number of retry attempts and the timeouts for connecting and reading a response.You can resolve this error by adding a SageMaker boto3 client with a custom retry configuration to the SageMaker Python SDK client.Resolution1.    Create a SageMaker boto3 client with a custom retry configuration. Example:import boto3 from botocore.config import Configsm_boto = boto3.client('sagemaker', config=Config(connect_timeout=5, read_timeout=60, retries={'max_attempts': 20}))print(sm_boto.meta.config.retries)2.    Create a SageMaker Python SDK client using the boto3 client from the previous step. Example:import sagemakersagemaker_session = sagemaker.Session(sagemaker_client = sm_boto)region = sagemaker_session.boto_session.region_nameprint(sagemaker_session.sagemaker_client.meta.config.retries)3.    Test a SageMaker API with multiple requests from the SageMaker Python SDK. Example:import multiprocessingdef worker(TrainingJobName):    print(sagemaker_session.sagemaker_client          .describe_training_job(TrainingJobName=TrainingJobName)          ['TrainingJobName'])    returnif __name__ == '__main__':    jobs = []    TrainingJobName = 'your-job-name'    for i in range(10):        p = multiprocessing.Process(target=worker, args=(TrainingJobName,))        jobs.append(p)        p.start()4.    Create an instance of the sagemaker.estimator.Estimator class with the sagemaker_session parameter. Example:estimator = sagemaker.estimator.Estimator(container,                                             role,                                              train_instance_count=1,                                              train_instance_type='ml.c4.4xlarge',                                             train_volume_size = 30,                                             train_max_run = 360000,                                             input_mode= 'File',                                             output_path=s3_output_location,                                             sagemaker_session=sagemaker_session )5.    To confirm that the retry configuration resolves the throttling exceptions, launch a training job from the estimator that you created in the previous step:estimator.fit()Related informationBoto3 documentationFollow"


Question: What's the difference between Lambda function execution role permissions and invocation permissions?

I want to understand the differences between AWS Lambda function execution role permissions and invocation permissions. How are the two types of AWS Identity and Access Management (IAM) permissions for Lambda different?

Answer: "I want to understand the differences between AWS Lambda function execution role permissions and invocation permissions. How are the two types of AWS Identity and Access Management (IAM) permissions for Lambda different?ResolutionImportant: When you create IAM policies, it's a best practice to grant only the permissions required to perform a specific task. For more information, see Grant least privilege in the IAM User Guide.Lambda function execution role permissionsLambda execution role permissions are IAM permissions that grant a Lambda function permission to access specific AWS Cloud services and resources. If you don't grant your function execution role permissions for an AWS Cloud service or resource, then the function can't access that service or resource.Important: For Lambda functions that poll other AWS Cloud services, execution role permissions that allow the function to retrieve records from those services are required.How execution role permissions are grantedLambda execution role permissions are given in the form of specific API action identifiers (for example, s3:GetObject). These action identifiers are listed in the Action element of an IAM policy statement. The policy is then attached to an IAM role called an execution role that your Lambda function assumes whenever it's invoked.For services that generate a queue or data stream, you must configure an event source mapping to have Lambda poll the queue or data stream. An event source mapping uses permissions in the function's execution role to read and manage items in the event source.Note: Permissions, event structure, settings, and polling behavior vary by event source. For an example IAM policy that includes Lambda polling permissions, see IAM policy to allow an AWS Lambda function to access DynamoDB stream records.Lambda function invocation permissionsLambda function invocation permissions are IAM permissions that grant a specific AWS Cloud service or resource permission to invoke a Lambda function. If you don't grant a service or resource invocation permissions for your Lambda function, that service or resource can't invoke your function.Important: For Lambda function's that are invoked by AWS Cloud services that push event sources to the function, invocation permissions are required.How invocation permissions are grantedLambda invocation permissions are given in the form of a specific API action identifier: lambda:InvokeFunction. This action identifier is listed in the Action element of a Lambda function's IAM resource-based policy statement. A Lambda function's resource-based policy is configured separately from the function's execution role.Note: This is a special case where the action identifier (lambda:InvokeFunction) is different than the API operation (Invoke). For other actions, the action identifier is the operation name prefixed by lambda:.When you use the Lambda console to add a trigger to your function, the required invocation permissions are added to the function's resource-based policy automatically. To grant permissions to accounts or services that aren't available in the console, you must use the Lambda AddPermission API action.For more information, see Granting function access to AWS services.Related informationAWS Lambda permissionsApplying the principles of least privilege (Lambda Operator Guide)Follow"


Question: Why am I getting an Access Denied error when I open the URL to an Amazon S3 object that I have access to?

"I have permission to access an object in Amazon Simple Storage Service (Amazon S3) bucket. I opened the URL in the Amazon S3 console for that object. However, I got an Access Denied error message. How can I fix this?"

Answer: "I have permission to access an object in Amazon Simple Storage Service (Amazon S3) bucket. I opened the URL in the Amazon S3 console for that object. However, I got an Access Denied error message. How can I fix this?Short descriptionThe URL to the Amazon S3 object doesn't include your user credentials, so the request to the object is anonymous. Amazon S3 returns an Access Denied error for anonymous requests to objects that aren't public.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent version of the AWS CLI.Do one of the following:Download the object using the Amazon S3 console, AWS CLI, AWS SDKs, or REST API. For instructions, see Downloading an object.-or-Include the user credentials in the object request using a presigned URL. For instructions, see Sharing objects using presigned URLs.Important: It's a security best practice to keep objects private and to set up users with a presigned URL for authenticated requests to access objects. However, if your use case supports public read access to your objects that requires you to access using the URL, then modify the objects permission. To set up public read access to the object, see Granting read-only permission to an anonymous user.Related informationConfiguring ACLsIdentity and access management in Amazon S3Follow"


Question: How can I automate Amazon EBS snapshot management using Amazon Data Lifecycle Manager?

"I want to automate the creation, retention, and deletion of Amazon Elastic Block Storage (Amazon EBS) snapshots used for backing up my Amazon EBS volumes. How do I do that?"

Answer: "I want to automate the creation, retention, and deletion of Amazon Elastic Block Storage (Amazon EBS) snapshots used for backing up my Amazon EBS volumes. How do I do that?Short DescriptionAmazon Data Lifecycle Manager (Amazon DLM) is an automated procedure to back up the data stored on your Amazon EBS volumes. Use Amazon DLM to create lifecycle policies to automate snapshot management.ResolutionCreate a lifecycle policy in Amazon DLM:1.    In the Amazon Elastic Compute Cloud (Amazon EC2) console, under Elastic Block Store, select Lifecycle Manager.2.    Select Create Snapshot Lifecycle Policy.3.    Enter a Description for the policy.4.    Select a resource type. Select Volume if you want to schedule snapshots for volumes with a specific tag. Select Instance if you want to schedule snapshots for all volumes attached to an instance with a specific tag.5.    Select the tags associated with the Amazon EBS volume or Amazon EC2 instance, depending on the option chosen above.6.    Add a Schedule name to your policy. Any snapshot created with this policy is automatically tagged with the schedule name entered here.7.    Enter the number of hours that will elapse between policy runs.8.    Enter the policy run start time. Snapshot creation starts within one hour of the specified start time.9.    Set the Retention rule to the maximum number of snapshots that you want to retain. The oldest snapshot beyond this number will be deleted.10.    Optionally, select Copy Tags from volume to copy all user-defined tags on a source volume to snapshots of the volume created by this policy.11.    Optionally, add additional tags to the created snapshots by selecting Add Tag. These tags are in addition to any tags that were copied from the volumes or added as default by Amazon DLM.12.    Select an AWS Identity and Access Management (IAM) role that has snapshot create and delete permissions. The Default role has the required permissions. If you don't want to use the default role, you can create a new role with the required permissions using the IAM console. Select the new role when creating your lifecycle policy. The following permissions are required for a role to use Amazon DLM."ec2:CreateSnapshot",     "ec2:CreateSnapshots", "ec2:DeleteSnapshot","ec2:DescribeVolumes", "ec2:DescribeInstances",   "ec2:DescribeSnapshots"13.    Review the Policy Summary to verify that the specified rules meet your requirements.14.    Select Enable policy to start the policy runs at the next scheduled time. If Enable policy is not selected, the policy will not create or delete snapshots.15.    Select Create Policy.Related InformationAmazon Data Lifecycle Manager - Permissions for IAM UsersFollow"


Question: Why does my IAM credential report show my AWS Config managed rules as not compliant?

My AWS managed config rules aren't compliant when I used the API to turn on multi-factor authentication (MFA) for my AWS Identity and Access Management (IAM) users or rotated IAM access keys.

Answer: "My AWS managed config rules aren't compliant when I used the API to turn on multi-factor authentication (MFA) for my AWS Identity and Access Management (IAM) users or rotated IAM access keys.Short descriptionThe AWS managed config rules mfa-enabled-for-iam-console-access,  iam-user-mfa-enabled,  access-keys-rotated, and  iam-user-unused-credentials-checkaren't compliant after invoking the API GenerateCredentialReport. This is because these rules rely on the credential report generated by the API.When a GenerateCredentialReport call is invoked, IAM checks if there is an existing report. If the report was generated within the past four hours, the API call will use the most recent report rather than generating a new report. If the most recent report is older than four hours or there are no previous reports, the GenerateCredentialReport API generates a new report. For more information, see Getting credential reports for your AWS account.ResolutionChange the MaximumExecutionFrequency parameter to more than 4 hours. MaximumExecutionFrequency indicates the maximum frequency with which AWS Config runs evaluations for an AWS Managed Periodic rule.Open the AWS Config console, and then choose Rules.In Rule name, select your AWS Config rule, and then choose Edit.In Trigger, select the Frequency dropdown menu, and choose 6, 12, or 24 hours.Choose Save.To update the rule trigger frequency using the AWS Command Line Interface (AWS CLI), run the put-config-rule command.Related informationConfigRuleGetCredentialReportFollow"


Question: Why does the AWS Glue crawler classify my fixed-width data file as UNKNOWN when I use a built-in classifier to parse the file?

"When I parse a fixed-width .dat file with a built-in classifier, my AWS Glue crawler classifies the file as UNKNOWN."

Answer: "When I parse a fixed-width .dat file with a built-in classifier, my AWS Glue crawler classifies the file as UNKNOWN.Short descriptionBuilt-in classifiers can't parse fixed-width data files. Use a grok custom classifier instead.ResolutionCreate the grok custom classifier1.    Open the AWS Glue console.2.    In the navigation pane, choose Classifiers.3.    Choose Add classifier, and then enter the following:For Classifier name, enter a unique name.For Classifier type, choose Grok.For Classification, enter a description of the format or type of data that is classified, such as "special-logs."For Grok pattern, enter the built-in patterns that you want AWS Glue to use to find matches in your data. To parse a .dat file, no delimiter is required between fields. Because each field has a known length, you can use a regex pattern to find matches.Example:(?<col0>.{7})(?<col1>.{8})(?<col2>.{14})(?<col3>.{52})(Optional) For Custom patterns, enter any custom patterns that you want to use. These patterns are referenced by the grok pattern that classifies your data. Each custom pattern must be on a separate line. For more information, see Custom classifier values in AWS Glue.4.    Choose Create.Create and run the crawler1.    In the navigation pane, choose Crawlers.2.    Choose Add crawler.3.    For Crawler name, enter a unique name.4.    Choose the arrow next to the Tags, description, security configuration, and classifiers (optional) section, and then find the Custom classifiers section.5.    Choose Add next to the customer classifier that you created earlier, and then choose Next.6.    On the Specify crawler source type page, choose Data stores, and then choose Next.7.    On the Add a data store page, enter the following:For Choose data store, choose your preferred data store.For Include path, enter the path to your .dat file.8.    Choose Next,and then confirm whether or not you want to add another data store.9.    On the Choose an IAM role page, select an existing AWS Identity and Access Management (IAM) role or create a new one. Choose Next.10.    For Frequency, choose Run on demand, and then choose Next.11.    On the Configure the crawler's output page, for Database, choose the database that you want the table to be created in. Choose Next.12.    Choose Finish to create the crawler.13.    When the crawler status changes to Ready, select the crawler name, and then choose Run crawler.14.    Wait for the crawler to finish, and then choose Tables in the navigation pane. The Classification must match the classification that you entered for the grok custom classifier (for example, "special-logs").Related informationWorking with classifiers on the AWS Glue ConsoleWriting grok custom classifiersAdding classifiers to a crawlerFollow"


Question: How do I install a GUI on my Amazon EC2 instance running Amazon Linux 2?

I want to install a graphical user interface (GUI) in my Amazon Elastic Compute Cloud (Amazon EC2) instance running Amazon Linux 2. How do I do this?

Answer: "I want to install a graphical user interface (GUI) in my Amazon Elastic Compute Cloud (Amazon EC2) instance running Amazon Linux 2. How do I do this?Short descriptionBefore you proceed, first consider one of these alternatives to manually installing a GUI:Launch from an Amazon Linux 2 AMI that comes with MATE pre-installed. For more information, see Configure the Amazon Linux 2 MATE desktop connection.Start using Amazon WorkSpaces, a fully managed, persistent desktop virtualization service. For more information, see Amazon WorkSpaces.Start using NICE DCV, a remote display protocol that delivers remote desktops and application streaming securely from any cloud or data center to any device. For more information, see NICE DCV.If you want to proceed with manual GUI installation, then do the following:Install the MATE desktop environment, a lightweight GUI based on GNOME 2 available as an extra for Amazon Linux 2. For more information about MATE, see the MATE desktop environment website.Install a virtual network computing (VNC) service, such as TigerVNC. For more information about TigerVNC, see the tigervnc.org website.Connect to the GUI using the VNC.(Optional) Turn off password authentication for VNC.(Optional) Install a web browser, such as Chromium or Firefox. For more information on Chromium, see the chromium.org website. For more information on Firefox, see the mozilla.org website. Note that both Chromium and Firefox are only available in the X86 variant.Note: These instructions apply only to Amazon Linux 2. To confirm the version that you're running, run the following command:$ grep PRETTY_NAME /etc/os-releasePRETTY_NAME="Amazon Linux 2"For instructions on accessing an EC2 macOS instance using a GUI, see How can I access my Amazon EC2 Mac instance through a GUI?ResolutionBefore you begin1.    Connect to your instance as EC2-user using SSH.The instance must be in a public subnet and accessible through a public IP address or an Elastic IP address. Or, you can establish a connection using AWS VPN or AWS Direct Connect that allows you to access your instance through a private IP. These instructions don't apply to environments using a bastion host. For security reasons, traffic to the VNC server is tunneled using SSH. It's a best practice to avoid opening VNC ports in your security groups.2.    Update your instance by running sudo yum update, and then  reboot the instance. Failure to update and reboot can lead to unpredictable results when following these steps.Install the MATE desktop environment1.    Install MATE packages.sudo amazon-linux-extras install mate-desktop1.x2.    Define MATE as your default desktop for all users.sudo bash -c 'echo PREFERRED=/usr/bin/mate-session > /etc/sysconfig/desktop'Install TigerVNC1.    Install TigerVNC Server:sudo yum install tigervnc-server2.    Configure a VNC-specific password containing from six to eight characters for this user. When asked if you want to enter a view-only password, press "n".vncpasswd3.    Restrict VNC network access to the localhost, so that VNC can be accessed only by using a secure SSH tunnel.Create the tigervnc configuration directory:sudo mkdir /etc/tigervncCreate a mandatory configuration file containing the localhost option:sudo bash -c 'echo localhost > /etc/tigervnc/vncserver-config-mandatory'4.    Start the VNC Server on display number 1, and cause it to always start at boot time.Create a new systemd unit.sudo cp /lib/systemd/system/vncserver@.service /etc/systemd/system/vncserver@.serviceUse the sed command to replace all occurrences of USER in the new unit with ec2-user.sudo sed -i 's/<USER>/ec2-user/' /etc/systemd/system/vncserver@.serviceReload the systemd manager configuration.sudo systemctl daemon-reloadTurn on the service.sudo systemctl enable vncserver@:1Start the service.sudo systemctl start vncserver@:1Connect to the GUI using VNC1.    Install the TigerVNC software on your local computer, if it's not already installed. TigerVNC is available for Windows, Linux, and macOS. See the TigerVNC website to access the download.Linux: Thetigervnc package is available in the repositories of many distributions, and can be installed using their respective package managers.macOS: Download and install TigerVNC-x.y.z.dmg, where x.y.z represents the latest version. Windows: Download and install tigervnc64-x.y.z.exe (64-bit) or tigervnc-x.y.z.exe (32-bit), where x.y.z represents the latest version.2.    On your local computer, use SSH to connect to your instance while creating a tunnel to forward all traffic on local port 5901/TCP (VNC) to the instance's VNC server:Linux and macOSWhen connecting to your instance using SSH, add the -L parameter to activate port forwarding. Replace PEM_FILE with your private key, and INSTANCE_IP with your instance's public or private IP, as appropriate.ssh -L 5901:localhost:5901 -i PEM_FILE ec2-user@INSTANCE_IPWindowsWhen opening the connection with PuTTY, configure port forwarding, and then open the connection:From the Connection menu, select SSH, and then select Tunnels.Enter 5901 in the Source Port field.Enter localhost:5901 in the Destination field.Select Add.3.    Open the TigerVNC Viewer on your local computer. When asked for the VNC server hostname, enter localhost:1 and then connect to it.4.    Enter the VNC password that you set up in step 2 of the Install TigerVNC section. If an alert appears stating that the connection isn't secure, disregard it. You're accessing the VNC server using an encrypted SSH tunnel.Your MATE desktop environment appears.(Optional) Turn off password authentication for VNCIt's not a best practice to turn off VNC password authentication. Any users who are able to use SSH to connect into your instance will be able to create a tunnel and access your VNC desktop. If this is a single-user instance (that is, no other people can log in), and you'd like to turn off password authentication for VNC anyway, follow these steps:1.    Change the Security Types setting of the VNC Server to None:echo SecurityTypes=None >> ~/.vnc/config2.    Restart the VNC server.sudo systemctl restart vncserver@:1(Optional) Install a web browser (X86 only)AWS doesn't provide web browsers such as Chromium or Firefox in Amazon Linux 2 repositories. To install one of these browsers, follow these steps. Note that AWS doesn't provide support for software installed from third-party sources.The following instructions apply to EC2 instances based on 64-bit Intel and AMD processors (x86_64). These third-party sources don't provide web browsers for instances based on Graviton processors (aarch64). To confirm your instance's architecture, run the following command:$ uname -mx86_64To install Chromium (open-source project behind Google Chrome), do the following:1.    Activate the EPEL repository. See the EPEL documentation on the Fedoraproject.org website for more information.sudo amazon-linux-extras install epel2.    Install Chromium packages.sudo yum install chromiumChromium installs on MATE under Applications, Internet, Chromium Web Browser.To install Mozilla Firefox (for current user only), do the following:1.    Download the latest Firefox version for Linux 64-bit to your local computer. See the mozilla.org website to access the download.2.    After downloading Firefox, copy the file to your instance. See the respective websites for Linux and Windows for instructions.3.    Connect to your EC2 instance using SSH.4.    Extract the file contents. This example command indicates that the file is in the ec2-user's home directory. Change the path and the Firefox version as needed.tar jxf ~/firefox-xx.y.tar.bz2 -C ~/5.    Using vim or your favorite editor, create a desktop icon. Create the file ~/Desktop/Firefox.desktop with the contents as shown in the following example.[Desktop Entry]Version=1.0Type=ApplicationTerminal=falseIcon=/home/ec2-user/firefox/browser/chrome/icons/default/default128.pngIcon[en_US]=/home/ec2-user/firefox/browser/chrome/icons/default/default128.pngName[en_US]=FirefoxExec=/home/ec2-user/firefox/firefoxComment[en_US]=Firefox web browserName=FirefoxComment=Firefox web browserGenericName[en_US.UTF-8]=Firefox web browserCategories=Network;WebBrowser;6.    Use the desktop icon that you created in step 5 to launch Firefox.7.    If a prompt appears about the application launcher not being marked as trusted, choose Mark as Trusted, and then double-click to open Firefox.Follow"


Question: How can I troubleshoot an error that I received when deleting an Amazon Aurora cluster?

"I want to delete an Amazon Aurora cluster, but:I received an error message.        - or -The option to delete is unavailable (grayed out) in the Amazon Relational Database Service (Amazon RDS) console.How can I troubleshoot this issue and delete my cluster?"

Answer: "I want to delete an Amazon Aurora cluster, but:I received an error message.        - or -The option to delete is unavailable (grayed out) in the Amazon Relational Database Service (Amazon RDS) console.How can I troubleshoot this issue and delete my cluster?Short descriptionYou can use the Amazon RDS console or the AWS Command Line Interface (AWS CLI) to delete an Amazon Aurora cluster. For more information, see Deleting a DB instance.To delete an Aurora cluster using the AWS CLI, first delete all DB instances inside the cluster. After you delete all DB instances inside a cluster, you can then delete the cluster by using delete-db-cluster. If you delete the last DB instance in the cluster using the Amazon RDS console, the empty cluster is automatically deleted.ResolutionNote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Before deleting a cluster, consider the following:If you have a cluster with only one DB instance and you delete that DB instance using the Amazon RDS console, then delete both that DB instance and the cluster.If you have a cluster with one writer instance and one or more reader instance/read replicas, and you delete the reader instance, then the writer instance and the cluster aren’t affected. If you delete the writer instance, then the reader instance is promoted as a writer, automatically. This failover operation causes downtime.Delete an Aurora cluster with no instances inside it using the Amazon RDS consoleFollow these steps in the Amazon RDS console:Open the Amazon RDS console.Choose Databases from the navigation pane, and then select the cluster.Choose Actions, and then choose Delete Cluster.Note: If Delete Cluster is gray or unavailable, there might be an instance inside the cluster that you must delete. In this case, delete all the instances in the cluster. When you delete the last instance in the cluster using the Amazon RDS Console, then the cluster is deleted automatically.If you receive the following error, then follow the steps below."This database has deletion protection option enabled, to be able to delete the last Aurora DB instance, modify the Aurora cluster and disable deletion protection"Open the Amazon RDS console.Choose Databases from the navigation pane, and then select the cluster.Choose Actions, and then choose Modify Cluster.Choose Disable deletion protection, and then choose Continue.Choose Apply immediately.Delete an Aurora cluster using the AWS CLINote: The following steps are applicable for both Aurora MySQL and Aurora PostgreSQL.1.    Delete an Aurora clusterRun the following command:aws rds delete-db-cluster --db-cluster-identifier sample-cluster --final-db-snapshot-identifier mydbinstancefinalThis command deletes the cluster, and takes the final DB cluster snapshot. If you don't want to take the final snapshot, use the ---skip-final-snapshot option when you run this command. You might receive the following error when you run this command:"An error occurred (InvalidDBClusterStateFault) when calling the DeleteDBCluster operation: Cluster cannot be deleted, it still contains DB instances in non-deleting state"This indicates that the cluster can't be deleted because there are instances present in the cluster. Be sure to delete all instances associated with the cluster before you delete the cluster.2.    Delete an instance in the clusterRun the following command:aws rds delete-db-instance --db-instance-identifier sample-instanceYou might receive one of the following error messages when trying to delete an instance in the cluster."InvalidParameterCombination: An error occurred (InvalidParameterCombination) when calling the DeleteDBInstance operation: FinalDBSnapshotIdentifier cannot be specified when deleting a cluster instance"You receive this error when you use the --final-db-snapshot-identifier option when you run the command to delete the instance in the cluster. This error warns you that you can't take a final snapshot when deleting an instance in the cluster. Either remove the --final-db-snapshot-identifier option or use the --skip-final-snapshot option, and then run the command again."InvalidDBClusterStateFault: Cannot delete the last instance of the read replica DB cluster. Promote the DB cluster to a standalone DB cluster in order to delete it”This error indicates that the Aurora cluster that you are trying to delete is acting as a replica, but the last instance of the cluster can't be deleted until the cluster is promoted to standalone.3.    Promote a replica clusterPromote a replica cluster by running this command:aws rds promote-read-replica-db-cluster --db-cluster-identifier rds-mysql-aurora-replica-clusterRelated informationHow do I delete or terminate RDS resources?Managing an Amazon Aurora DB clusterDeleting a DB instance in an Aurora DB clusterFollow"


Question: How do I get a Lambda function in a VPC to access Systems Manager Parameter Store?

"I have an AWS Lambda function in an Amazon Virtual Private Cloud (Amazon VPC). I want to connect the function to AWS Systems Manager Parameter Store, but I can't establish the connection. Why can't my function access Parameter Store, and how do I configure the function so that it can?"

Answer: "I have an AWS Lambda function in an Amazon Virtual Private Cloud (Amazon VPC). I want to connect the function to AWS Systems Manager Parameter Store, but I can't establish the connection. Why can't my function access Parameter Store, and how do I configure the function so that it can?Short descriptionLambda functions within an Amazon VPC don't have internet access. The network interfaces that Lambda creates within an Amazon VPC have private IP addresses, and can't use an internet gateway to gain internet access. This configuration blocks the function from accessing other AWS resources outside of the Amazon VPC, such as Parameter Store.Note: It's a best practice to not put your Lambda function in an Amazon VPC unless the function must access other resources in the VPC.ResolutionTo allow a Lambda function that's in an Amazon VPC to access Parameter Store, do one of the following:Give your function internet access.Set up an interface endpoint in your Amazon VPC that allows your function to access Systems Manager.Note: AWS PrivateLink VPC endpoints rely on DNS. If you use AWS PrivateLink, make sure that you configure the DHCP options for your Amazon VPC so that DNS is set up for it. For more information, see Using DNS with your VPC.Related informationConfiguring a Lambda function to access resources in an Amazon VPCVPC endpointsFollow"


Question: What information should I include in my AWS Support case?

"I have a Developer or Business Support plan, and I’m opening a technical support case to get help from AWS Support."

Answer: "I have a Developer or Business Support plan, and I’m opening a technical support case to get help from AWS Support.ResolutionAll the following information can be helpful to AWS Support engineers attempting to diagnose the issue:A description of your use case. What is the intended behavior for this resource / application / project?The environment in which you’re observing the issue. Is this your production, staging, development, or test environment? Who is the intended audience for this content / application?Region. Include information on Regions from which you launched or are attempting to launch your resources.Resource IDs of the affected resources. Include Amazon Elastic Compute Cloud (Amazon EC2) instance IDs, Amazon Simple Storage Service (Amazon S3) object and bucket names, Amazon CloudFront distribution IDs, API request IDs, and so on.Dates / times when you experienced the issue, and how long the issue persisted. Be sure to include your time zone.Log files from the approximate time the issue occurred. You can attach these directly to your case.Any error messages that you’re receiving.The steps you took, or the steps a Support engineer might take, to reproduce the issue.For networking and DNS issues, the outputs of tcptraceroute / TCP MTR / telnet / netcat / curl / dig / nslookup commands.Related informationAWS Support CenterAWS SupportFollow"


Question: How can I troubleshoot when I'm unable to initiate a remote PowerShell session with FSx for Windows File Server over an external trust?

"I want to configure shadow copies on Amazon FSx for Windows File Server. However, I'm unable to do so because I get the following error while I set up a remote PowerShell session:"enter-pssession : Connecting to remote server <FSx_DNS_Name> failed with the following error message : WinRM cannot process the request. The following error occurred while using Kerberos authentication: Cannot find the computer <FSx_DNS_Name>. Verify that the computer exists on the network and that the name provided is spelled correctly. For more information, see the about_Remote_Troubleshooting Help topic.""

Answer: "I want to configure shadow copies on Amazon FSx for Windows File Server. However, I'm unable to do so because I get the following error while I set up a remote PowerShell session:"enter-pssession : Connecting to remote server <FSx_DNS_Name> failed with the following error message : WinRM cannot process the request. The following error occurred while using Kerberos authentication: Cannot find the computer <FSx_DNS_Name>. Verify that the computer exists on the network and that the name provided is spelled correctly. For more information, see the about_Remote_Troubleshooting Help topic."ResolutionFirst, confirm that your configuration meets the following prerequisites:You must create a trust relationship between the on-premises Microsoft Active Directory and AWS Directory Service for Microsoft Active Directory.Note: At minimum, you must have two one-way trust relationships. One one-way trust relationship must be an outgoing trust from the AWS Managed Microsoft AD where your Amazon FSx computer objects are. The other one-way trust relationship must be an incoming trust on the on-premises Microsoft AD.You must have an on-premises AD user that is part of the Amazon FSx administrators group in the AWS Managed Microsoft AD directory.Your Amazon FSx security group must have rules that allow traffic to and from the IP addresses or security group IDs associated with the client that you want to manage Amazon FSx and enable shadow copies from. The Amazon FSx security group must allow this inbound and outbound traffic on the following ports:TCP/UDP 445 (SMB)TCP 135 (RPC)TCP/UDP 1024-65535 (Ephemeral ports for RPC)If your configuration meets the prerequisites, but you're still getting the "Kerberos authentication: Cannot find the computer" error, then the error might be related to the Microsoft external trust. For Kerberos authentication to work with Microsoft external trust, the application building the service principal names (SPNs) must build three-part SPNs. The enter-pssession cmdlet requests only a two-part SPN, so the request fails. If you run a network trace while you run the PowerShell cmdlet, then the request returns an error message similar to "Kerberos: KRB_ERROR - KDC_ERR_S_PRINCIPAL_UNKNOWN."To resolve this error and enable Kerberos authentication over a Microsoft external trust, configure Kerberos Forest Search Order (KFSO) for the Kerberos client in the on-premises AD domain. In the Use forest search order window, for Forests to Search, add the domain names for your self-managed AD and the AWS Managed Microsoft AD. Add the domain names in the following format and order:selfmanaged.example.com;awsmanaged.example.comAfter you apply these settings on the machine that you use to maintain Amazon FSx, you can proceed with enabling shadow copies for Amazon FSx.Related informationUsing DNS names with Kerberos authenticationFollow"


Question: How do I troubleshoot latency with my Amazon SageMaker endpoint?

"When I invoke my Amazon SageMaker endpoint, I experience high latency."

Answer: "When I invoke my Amazon SageMaker endpoint, I experience high latency.Short descriptionUse Amazon CloudWatch to monitor the latency metrics ModelLatency and OverheadLatency for a SageMaker endpoint that serves a single model.ModelLatency is the amount of time that a model takes to respond to an inference request, as viewed by SageMaker. This duration includes the local communication time for the model to send the request and fetch the response. It also includes the completion time of the inference inside the model container.OverheadLatency is the amount of time that SageMaker takes to respond to an invocation request with overheads. This measurement lasts from when SageMaker receives a request until it returns a response, minus ModelLatency.When you use a SageMaker multi-model endpoint, the following additional metrics are available in CloudWatch:ModelLoadingWaitTime: The amount of time that an invocation request waits for the target model to download or load, before performing inference.ModelDownloadingTime: The amount of time to download the model from Amazon Simple Storage Service (Amazon S3).ModelLoadingTime: The amount of time to load the model from the container.ModelCacheHit: The number of InvokeEndpoint requests that are sent to the endpoint where the model previously loaded.Multi-model endpoints load and unload models throughout their lifetime. You can use the LoadedModelCount CloudWatch metric to view the number of loaded models for an endpoint.ResolutionHigh ModelLatencyTo reduce this latency, take any of the following actions:Benchmark the model outside of a SageMaker endpoint to test performance.If SageMaker Neo supports your model, then you can compile the model. SageMaker Neo optimizes models to run up to twice as fast with less than a tenth of the memory footprint with no loss in accuracy.If AWS Inferentia supports your model, then you can compile the model for Inferentia. This offers up to three times higher throughput and up to 45% lower cost per inference compared to the AWS GPU-based instances.If you use a CPU instance and the model supports GPU acceleration, then use a GPU instance to add GPU acceleration to an instance.Note: The inference code might affect the model latency depending on how the code handles the inference. Any delays in code increase the latency.An overused endpoint might cause higher model latency. To dynamically increase and decrease the number of instances that are available for an endpoint, add auto scaling to an endpoint.High OverheadLatencyMultiple factors might contribute to OverheadLatency. These factors include the payload size for request and responses, request frequency, and the authentication or authorization of the request.The first invocation for an endpoint might have an increase in latency because of a cold start. This is expected with the first invocation requests. To avoid this issue, send test requests to the endpoint to pre-warm it. Note that infrequent requests might also lead to an increase in OverheadLatency.Follow"


Question: What are the differences between the Amazon EBS volume types?

I want to choose an Amazon Elastic Block Store (Amazon EBS) volume type that's right for my workload. What are the differences between the volume types?

Answer: "I want to choose an Amazon Elastic Block Store (Amazon EBS) volume type that's right for my workload. What are the differences between the volume types?ResolutionWhen determining which Amazon EBS volume type is right for your needs, consider the following:DurabilityVolume sizeMaximum IOPS per volumeMaximum throughput per volumeWhether it supports Amazon EBS Multi-AttachWhether it supports boot volumeAmazon EBS provides the following volume types: previous generation, hard disk drives (HDD), and solid state drives (SSD).Note: Before you change your volume type, see Limitations.Previous generationYou can use this volume type as a backup and for archiving, or for workloads with small datasets that you access infrequently. For more information on this volume type, see Previous generation Magnetic volumes.Hard disk drives (HDD)The dominant performance attribute for this volume type is the throughput. HDD-backed volumes are optimized for workloads that require a large I/Osize, or workloads that have a synchronized I/O. It's a best practice to use an Amazon EBS-optimized Amazon Elastic Compute Cloud (Amazon EC2) instance for this volume type. HDD-backed volumes don't support boot volume, so you can't choose it as a root volume.HHD-backed volumes fall under two categories: Throughput Optimized HDD (st1) and Cold HDD (sc1). If optimal performance is your priority, then consider st1. If cost optimization is your priority, then consider sc1. Both st1 and sc1 offer the same level of durability. For more information on Throughput Optimized HDD and Cold HDD, see Overview of HDD-backed volumes.Solid state drives (SSD)The dominant performance attribute for this volume type is IOPS. SSD-backed volumes are optimized for a small I/O size and random I/O. It's a best practice to use an Amazon EBS-optimized instance for this volume type. SSD-backed volume types include General Purpose SSD volumes and Provisioned IOPS SSD volumes.General Purpose SSD volumesGeneral Purpose SSD volumes are good for a wide variety of transactional workloads that require less than the following:16,000 IOPS1,000 MiB/s of throughput160-TiB volume sizeGeneral Purpose SSD volumes include gp2 and gp3 types. Both gp2 and gp3 can reach up to 16000 IOPS, and offer the same level of durability. However, gp2 volumes reach a lower maximum throughput per volume (250 MiB/s) than gp3 volumes (1000 MiB/s). IOPs performance for gp2 volumes scales linearly with volume size, and gp2 burst performance is good for workloads that have a high IOPS burst rate. gp3 volumes don't use burst performance. However, gp3 volumes provide a consistent 3,000 IOPS baseline performance and 125 MiB/s throughput performance regardless of volume size. With gp3 volumes, you can provision IOPS and throughput independent of storage size.If you're currently using gp2 and want to change to gp3, see Migrate to gp3 from gp2.Note: If you don't specify IOPS or throughput performance when changing your volume type from gp2 to gp3, then Amazon EBS provisions whichever performance is higher.Provisioned IOPS SSD volumesProvisioned IOPS SSD volumes include io2 volumes, io2 Block Express volumes, and io1 volumes. Each one requires 16,000 KiB I/O minimum IOPS per volume. If your workload requires up to 64,000 IOPS per volume and up to 1,000 MiB/s of throughput, then use io1 and io2. If your workload requires up to 256,000 IOPS per volume and up to 4,000 MiB/s of throughput, then use io2 Block Express. If you require more durability and more ration per GiB, then use either io2 or io2 Block Express.Note: To achieve optimal performance for these volume types, consider the following:io1: If you created your io1 volume before December 6, 2017, then you must modify the volume.io1 and io2: Make sure that you provisioned 64,000 IOPS to reach the maximum 1,000 MiB/s throughput.io2 Block Express: Only c7g, r5b, trn1, x2idn, and x2iedn instances support io2 Block Express volumes.The maximum values for IOPS, throughput, and latency are based on an I/O size of 16 KiB. A larger I/O size can't reach these maximum values.Follow"


Question: How can I troubleshoot errors in my Amazon EKS environment after I create a cluster?

I get errors in my Amazon Elastic Kubernetes Service (Amazon EKS) environment after I create a cluster.

Answer: "I get errors in my Amazon Elastic Kubernetes Service (Amazon EKS) environment after I create a cluster.Short descriptionYou can use the AWSPremiumSupport-TroubleshootEKSCluster automation workflow to troubleshoot common errors with your Amazon EKS environment. The workflow provides a detailed report that lists errors, warnings, and links to Amazon Web Services (AWS) recommended best practices.Note: To use the AWSPremiumSupport-TroubleshootEKSCluster automation workflow, you must be subscribed to an Enterprise or Business Support plan.You can use the AWSPremiumSupport-TroubleshootEKSCluster automation workflow if you experience any of these issues:Your Kubernetes Cluster Autoscaler isn't working.Your internal load balancer doesn't get created in private or public subnets.You don't know if your worker nodes are using the latest Amazon Machine Image (AMI).You get access denied errors in your aws-node pods.You can't pull Amazon Elastic Container Registry (Amazon ECR) images.Your managed nodes don't stabilize, or your self-managed nodes don't join a cluster.The AWSPremiumSupport-TroubleshootEKSCluster automation workflow runs these checks:Checks Auto Scaling groups for the Cluster AutoScaler subnet tags for internal and internet-facing load balancers and worker security groups.Important: If you have more than one security group associated to your worker nodes, then you must apply a tag to one of your security groups. Set the key of the tag to kubernetes.io/cluster/your-cluster-name. Set the value to owned. This tag is optional if you have only one security group associated to your worker nodes.Checks worker nodes for the latest AMI.Checks security group rules for minimum and recommended settings. For example, the workflow checks for ingress and egress from the cluster's security group (that is, all security groups attached to the cluster) and from the worker security group.Checks custom network access control list (network ACL) configurations.Checks AWS Identity and Access Management (IAM) policies on worker nodes, including the AmazonEKSWorkerNodePolicy and AmazonEC2ContainerRegistryReadOnly policies.Checks if worker nodes can connect to the internet by checking the route tables of the subnets where they're located.ResolutionMake your worker nodes discoverableTo make your work nodes discoverable, use the AWSPremiumSupport-TroubleshootEKSCluster automation workflow and your Amazon EKS cluster:1.    Open the Amazon EC2 console.2.    On the navigation pane, choose Instances.3.    Select the Amazon Elastic Compute Cloud (Amazon EC2) instances for your worker nodes, and then choose the Tags tab.4.    Choose Add/Edit Tags.5.    For Key, enter kubernetes.io/cluster/your-cluster-name. For Value, enter 'owned'/ 'shared'.6.    Choose Save.The AWSPremiumSupport-TroubleshootEKSCluster automation workflow runs with the following policy. This policy must include minimum permissions to access the cluster. The cluster can be accessed only by AutomationAssumeRole.{  "Version": "2012-10-17",  "Statement": [    {      "Sid": "AWSPremiumSupportTroubleshootEKSCluster",      "Effect": "Allow",      "Action": [        "eks:ListNodegroups",        "eks:DescribeCluster",        "eks:DescribeNodegroup",        "iam:GetInstanceProfile",        "iam:ListInstanceProfiles",        "iam:ListAttachedRolePolicies",        "autoscaling:DescribeAutoScalingGroups",        "ec2:DescribeInstanceTypes",        "ec2:DescribeInstances",        "ec2:DescribeNatGateways",        "ec2:DescribeSecurityGroups",        "ec2:DescribeVpcs",        "ec2:DescribeSubnets",        "ec2:DescribeNetworkAcls",        "ec2:DescribeRouteTables"      ],      "Resource": "*"    },    {      "Sid": "GetPublicSSMParams",      "Effect": "Allow",      "Action": "ssm:GetParameter",      "Resource": [        "arn:aws:ssm:*:*:parameter/aws/service/eks/optimized-ami/*/amazon-linux-2/recommended/image_id",        "arn:aws:ssm:*:*:parameter/aws/service/ami-windows-latest/Windows_Server-2019-English-Core-EKS_Optimized-*/image_id",        "arn:aws:ssm:*:*:parameter/aws/service/ami-windows-latest/Windows_Server-2019-English-Full-EKS_Optimized-*/image_id",        "arn:aws:ssm:*:*:parameter/aws/service/ami-windows-latest/Windows_Server-1909-English-Core-EKS_Optimized-*/image_id",        "arn:aws:ssm:*:*:parameter/aws/service/eks/optimized-ami/*/amazon-linux-2-gpu/recommended/image_id"      ]    },    {      "Sid": "UploadObjectToS3",      "Effect": "Allow",      "Action": [        "s3:GetBucketPolicyStatus",        "s3:GetBucketAcl",        "s3:PutObject"      ],      "Resource": [        "<BUCKET_ARN>",        "<BUCKET_ARN>/AWSPremiumSupport-TroubleshootEKSCluster-*"      ]    }  ]}Run the automation workflow from the AWS Systems Manager console1.    Open the Systems Manager console.2.    In the navigation pane, choose Automation.Note: If the Systems Manager homepage opens first, choose the menu icon to open the navigation pane, and then choose Automation.3.    Choose Execute automation.4.    In the Automation document search box, enter AWSPremiumSupport-TroubleshootEKSCluster, and then press Enter.5.    From the search results, select AWSPremiumSupport-TroubleshootEKSCluster.Note: The document owner is Amazon.6.    In the Document description tab, verify that Document version is set to Default version at runtime.7.    Choose Next.8.    In the Execute automation document section, choose Simple execution.9.    In the Input parameters section, set the following parameters:Set ClusterName to the name of the cluster. This cluster must be running in your AWS account.Set AutomationAssumeRole to the IAM role that you want to use for the workflow.(Optional) If required, set BucketName to the Amazon Simple Storage Service (Amazon S3) bucket where you want to upload the report.Note: It you want to specify the AutomationAssumeRole, create an IAM role and S3 bucket that apply to the policy. Then, upload the report to your own S3 bucket ( BucketName) for the SSM automation.10.    Choose Execute.11.    To monitor the progress, choose the running Automation, and then choose the Steps tab.12.    When the process is finished, choose the Descriptions tab, and then choose View output to view the results.Note: The results appear as a report that lists all the errors and warnings for your cluster.Run the automation workflow from the AWS CLINote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version. Create an IAM role and S3 bucket that apply to the policy if you would like to specify AutomationAssumeRole and upload report to your own S3 bucket ( S3BucketName) for running the automation workflow.If you want the report in an S3 bucket, run this command:aws ssm start-automation-execution --document-name "AWSPremiumSupport-TroubleshootEKSCluster" --document-version "\$DEFAULT" --parameters '{"ClusterName":["your-eks-cluster"],"AutomationAssumeRole":["arn:aws:iam::123456789012:role/eks-troubleshooter-role"],"S3BucketName":["your_bucket"]}' --region your_regionIf you don't want to upload the report to an S3 bucket, run this command:aws ssm start-automation-execution --document-name "AWSPremiumSupport-TroubleshootEKSCluster" --document-version "\$DEFAULT" --parameters '{"ClusterName":["your-eks-cluster"],"AutomationAssumeRole":["arn:aws:iam::123456789012:role/eks-troubleshooter-role"]}' --region your_regionFollow"


Question: How do I install PECL 7 modules on Elastic Beanstalk environments running on PHP with Amazon Linux 1 stacks?

"I want to install Redis, Memcached, or other modules for PECL 7 on an AWS Elastic Beanstalk environment running on PHP 7.3 using Amazon Linux 1 stacks."

Answer: "I want to install Redis, Memcached, or other modules for PECL 7 on an AWS Elastic Beanstalk environment running on PHP 7.3 using Amazon Linux 1 stacks.Short descriptionTo install PECL 7 modules, you must:Install any dependencies.Create a Linux configuration file (.ebextensions) to install and run the PECL 7 modules.Important: The following PHP extensions are already included with the release of the PHP platform on Amazon Linux 2:php-pecl-redis (for communication with the Redis key-value store)php-pecl-memcached (for the Memcached caching daemon)Note: You can install PECL PHP extensions on any PHP environment that has PEAR support. PECL is installed by default on the Elastic Beanstalk PHP stack and is provided by the php-pear RPM. For more information, see redis on the PECL website.ResolutionInstall the Redis extension1.    In the .ebextensions directory in the root of the source bundle of your application, create a .ebextension file based on the following example:commands:  01_install_redis:    command:  /usr/bin/yes 'no'| /usr/bin/pecl7 install redis    test: '! /usr/bin/pecl7 info redis'  02_rmfromphpini:    command: /bin/sed -i -e '/extension="redis.so"/d' /etc/php.ini  03_createconf:    command: /bin/echo 'extension="redis.so"' > /etc/php-7.3.d/41-redis.ini    test: /usr/bin/pecl7 info redisThe .ebextension does the following:Installs the Redis extension (01_install_redis)Removes the entry that the pecl7 command creates on the /etc/php.ini file to load the extension (02_rmfromphpini)Creates a configuration file that loads the Redis module (03_createconf), which requires that the default JSON module loads before the Redis module2.    Deploy the new version of your application to your Elastic Beanstalk environment.3.    Connect to your instance using SSH, and then run the following command:php -m | egrep '(redis)'Output:$ php -m | egrep '(redis)'redisInstall the Memcached extension1.    In the .ebextensions directory in the root of the source bundle of your application, create a .ebextension file based on the following example:packages:  yum:    libmemcached-devel: []The preceding .ebextension installs libmemcached-devel, which is a prerequisite for installing the Memcached extension.2.    In the .ebextensions directory in the root of the source bundle of your application, create a .ebextension file based on the following example:commands:  01_install_memcached:    command:  /usr/bin/yes 'no'| /usr/bin/pecl7 install memcached    test: '! /usr/bin/pecl7 info memcached'  02_rmfromphpini:    command: /bin/sed -i -e '/extension="memcached.so"/d' /etc/php.ini  03_createconf:    command: /bin/echo 'extension="memcached.so"' > /etc/php-7.3.d/41-memcached.ini    test: '/usr/bin/pecl7 info memcached'The .ebextension does the following:Installs the Memcached extension (01_install_memcached)Removes the entry that the pecl7 command creates in the /etc/php.ini file to load the extension (02_rmfromphpini)Creates a configuration file that loads the Memcached extension (03_createconf)3.    Deploy the new version of your application to your Elastic Beanstalk environment.4.    Connect to your instance using SSH, and then run the following command:php -m | egrep '(memcached)'Output:$ php -m | egrep '(memcached)'memcachedFollow"


Question: How can I use Elastic Load Balancing to troubleshoot the reason that my website can't be accessed?

My website can't be accessed. How can I use Elastic Load Balancing (ELB) to understand why end users can't access my website?

Answer: "My website can't be accessed. How can I use Elastic Load Balancing (ELB) to understand why end users can't access my website?Short descriptionThere are a number of reasons that you might not be able to reach your website. To find the best path to troubleshoot, first answer these questions:Can the website not be reached, and are you getting a browser default timeout page?Is the issue intermittent?Is the page loading after a delay?Is the page showing an error like HTTP 5XX?ResolutionDo a DNS lookup and comparisonConfirm that the domain that is facing this issue is correctly resolving to the IP address of the load balancer nodes. This makes sure that the connection attempt that clients are trying is sent to the correct IP address.1.    Do a DNS lookup:dig <affected domain>ornslookup <affected domain>2.    Now, do a DNS lookup against the DNS of the load balancer for comparison. For example:dig lb-xxxxxxxxxx.us-east-1.elb.amazonaws.comThis check is successful if the list of IP addresses match. If the load balancer is in a scale-out state, then there might be more IP addresses than a single DNS response can show. In this use case, this comparison might not be sufficient. You might need to run multiple DNS queries to get all the IPs listed.Check the status of backend/targetsCheck the backend/target health check status. Load balancers have a configured health check that they use to check the status of backend/targets. Depending on the type of load balancer, unhealthy backend/targets can affect routing. Check the health of your load balancer depending on the type:Health checks for Application Load Balancer (ALB)Health checks for Network Load Balancer (NLB)Heath checks for Classic Load Balancer (CLB)Note:Enabling or disabling cross-zone can impact how requests are routed with NLB and CLBs.You can configure health checks on ports other than traffic ports. In this case, even if the target/backed is healthy, the target might not respond to traffic sent to it.If the health check shows the targets as unhealthy, then the service might down at the target/backend instances. Or, the load balancers can't get a response from the target/backend instances.Connect to the nodes of the load balancer directlyConnect to the nodes of the load balancer directly. This test confirms if all the IP addresses of the load balancer that were DNS resolved previously are reachable. This test also confirms whether the issue is happening only with a specific Availability Zone's (AZ) subnet.Note the IP addresses of the load balancer, and then attempt a connection to these IP addresses, one at a time. Using a browser for this test might not work due to certificate validations and the host header. Instead, it's a best practice to use the curl tool to perform this check with this format. Note that you must replace <protocol>, <ELB IP Address>, <port>, <path>, and <domain name> with the correct values from your web setup.curl -lvk <protocol>://<ELB IP Address>:<port>/<path> -H "Host: <domain name>"For example:Eg: curl -lvk https://3.2.1.1:443/test -H "Host: example.com"If your attempt gets a response, then check whether this is the expected response or an error. If the response is satisfactory, then check the next IP address.Check the load balancer for backend/target issuesIf you confirm connectivity to all IP addresses used by the load balancer, then confirm the load balancer's communication status with the target/backend.1.    First, check the security groups, network ACL, and routing.2.    If they are allowing access, then log in to the target/backend. After you establish access, confirm that the web service is listening on the configured port using this command. Replace “port" with the listening port of the target/backend. This is the port used when registering the target/backend to the load balancer.Windows:netstat -an | findstr "LISTEN" | findstr "port"Linux:netstat -an | grep LISTEN | grep "port"3.    If you don’t see anything listed after mentioning the expected listening port, then the web service is down. Continue to troubleshoot the service status.4.    If the response shows that the port is listening, then try to connect to it using curl. Replace <protocol> with the listening application service and <port> with the localhost port:curl -lvk <protocol>://localhost:<port>If you get a response from this command, then your web service is functioning. During this phase of testing, you can initiate a connection from another Amazon Elastic Compute Cloud (Amazon EC2) instance in the same Amazon Virtual Private Cloud (Amazon VPC). Check if this connection attempt reaches and gets a response from the web service. This response also shows what's received by the load balancer when you attempt to connect to the target/backend.Related informationHow do I troubleshoot an unresponsive website hosted on my EC2 instance?Follow"


Question: How do I transfer an Amazon EC2 instance or AMI to a different AWS account?

I want to transfer an Amazon Elastic Compute Cloud (Amazon EC2) instance or Amazon Machine Image (AMI) from one Amazon Web Services (AWS) account to another. How can I do this?

Answer: "I want to transfer an Amazon Elastic Compute Cloud (Amazon EC2) instance or Amazon Machine Image (AMI) from one Amazon Web Services (AWS) account to another. How can I do this?Short descriptionIt's not possible to transfer existing instances or AMIs between AWS accounts. Instead, you can manually migrate an EC2 image from one AWS account to another.ResolutionNote: The following steps apply only to instances with unencrypted Amazon Elastic Block Store (Amazon EBS) volumes.Create a custom AMI from the instance that you want to use as the source account. Be sure to include all required EBS data volumes in the AMI. Be sure that the AMI doesn't have a static IP configuration, as this can result in connectivity issues. For instructions, see one of the following:Create a Windows AMI from a running instanceCreate a Linux AMI from an instanceNote: Data stored on instance store volumes isn't preserved in AMIs. This data won't be on the instance store volumes of the instances that you launch from the AMI.Share the AMI with the target account using either the Amazon EC2 console or the AWS Command Line Interface (AWS CLI). For instructions, see one of the following:User Guide for Windows Instances - Share an AMI with specific AWS accountsUser Guide for Linux Instances - Share an AMI with specific AWS accountsNote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.From the target account, find the AMI using the EC2 console or the AWS CLI.Launch a new instance from the shared AMI on the target account. To import a key pair from a previous account, see Import your own public key to Amazon EC2.Note: For Amazon Virtual Private Cloud (Amazon VPC) instances, the private IP address is different in the new account unless specifically set during launch.Create a custom AMI from the target account instance. It's a best practice to use this copy to avoid complications after the source account deregisters the originally shared AMI.Deregister the AMI on the source account if you don't need to run any instances from it. For information about how to terminate running instances and delete associated snapshots, see Clean up your Amazon EBS-backed AMI.Note: If you share an AMI from account A to account B, and then deregister the AMI from account A, you can't launch new instances from the AMI in account B. You can leave the AMI registered in account A so that you can launch new instances from it later. Or, use one of the following workarounds so that you can deregister the AMI and retain the capability to launch new instances:Launch new instances from the shared AMI in account B before you deregister the AMI in account A. Deregistering the AMI in the original account doesn't impact instances launched from the shared AMI. If you need to launch new instances after deregistering the AMI, you can create a new AMI from one of the new instances.Copy the AMI to the same Region or a different Region in account B. This allows you to preserve the original AMI and also creates a backup without using it in account B.Related informationHow do I create and copy an Amazon Machine Image (AMI) from one AWS Region to another?How to share encrypted AMIs across accounts to launch encrypted EC2 instancesHow do I share an Amazon Machine Image (AMI) privately with another AWS account?Share an encrypted snapshot using the console - LinuxShare an encrypted snapshot using the console - WindowsFollow"


Question: How do I configure an AWS Backup Vault Lock?

I want to configure an AWS Backup Vault Lock for my backup vault.

Answer: "I want to configure an AWS Backup Vault Lock for my backup vault.ResolutionAWS Backup Vault Lock is an optional feature of a backup vault. For more information, see AWS Backup Vault Lock.Lock a backup vault using the consoleTo add a vault lock to your backup vault, complete the following steps:Open the AWS Backup console.In the navigation pane, choose Backup vaults. Then, choose Backup Vault Lock.Under How vault locks work or Vault locks, choose + Create vault lock.Under Vault lock details, choose the backup vault to which you want your lock applied.Under Vault lock mode, choose Governance mode or Compliance mode. For more information on choosing your modes, see Vault lock modes.For Retention period, choose the minimum and maximum retention periods (maximum retention period is optional). Only backup jobs within the retention periods are successful.If you chose Compliance mode, then Vault lock start date is shown. A compliance mode vault lock has a cooling-off period from the creation of the vault lock until the vault and its lock become unchangeable. Choose a grace time for the duration of the cooling-off period. The period must be at least three days (72 hours).Important: After the grace time is expired, the vault and its lock are immutable. It can't be changed or deleted by any user or by AWS.Choose Create vault lock.Enter confirm in the text box and check the box to add the vault lock.If the steps have been completed successfully, then a Success banner appears at the top of the console.Lock a backup vault programmaticallyTo configure an AWS Backup Vault Lock programmatically, use the PutBackupVaultLockConfiguration API.To create a vault lock in governance mode, don't include the parameter ChangeableForDays. If the ChangeableForDays parameter is included, then the vault lock is created in compliance mode. For more information, see Lock a backup vault programmatically.Note: If you receive errors when running AWS CLI commands, make sure that you're using the most recent version of the AWS CLI.The following are examples of using the put-backup-vault-lock-configuration API:The following command locks the backup vault my_vault_to_lock1 with governance mode:aws backup put-backup-vault-lock-configuration --backup-vault-name my\_vault\_to\_lock1 --min-retention-days 1 --region us-eastThe following command locks the vault my_vault_to_lock2 with compliance mode. The --changeable-for-days parameter is added to configure the grace time:aws backup put-backup-vault-lock-configuration --backup-vault-name my\_vault\_to\_lock --min-retention-days 1 --changeable-for-days 3 --region us-eastDetermine if a vault is locked in compliance or governance mode using the consoleTo review an AWS Backup Vault Lock details using the console, do the following:Open the AWS Backup console.In the navigation pane, choose Backup vaults. Then, choose Backup Vault Lock.The vault lock status is shown.Determine if a vault is locked in compliance or governance mode programmaticallyTo review the AWS Backup Vault Lock details on a vault use the DescribeBackupVault or ListBackupVaults APIs.The following is an example for the DescribeBackupVault command:aws backup describe-backup-vault --backup-vault-name s3Backup {    "BackupVaultName": "s3Backup",    "BackupVaultArn": "arn:aws:backup:us-east-1:XXXXX5457:backup-vault:s3Backup",    "EncryptionKeyArn":    "arn:aws:kms:us-east-1:XXXXX5457:key/xxxxxxxxx-e4e294b5e1ff",        "CreationDate": "2022-02-23T08:45:08.904000+00:00",    "CreatorRequestId": "xxxxxxxxx-5903d602b45a",    "NumberOfRecoveryPoints": 0,    "Locked": true,       "MinRetentionDays": 1,    "LockDate": "2023-03-26T12:05:24.117000+01:00" }The following is an example for the ListBackupVaults command:aws backup list-backup-vaults --region us-east-1{    "BackupVaultList": \[        {            "BackupVaultName": "Vault100",            "BackupVaultArn": "arn:aws:backup:us-east-1:XXXXXX15457:backup-vault:Vault100",            "CreationDate": "2021-02-21T18:45:12.611000+00:00",            "EncryptionKeyArn": "arn:aws:kms:us-east-1:XXXXX5457:key/xxxxxxxx-e4e294b5e1ff",            "CreatorRequestId": "xxxxxxxx-8f3d5b584447",            "NumberOfRecoveryPoints": 6,            "Locked": true        },        {           "BackupVaultName": "destinationvault",           "BackupVaultArn": "arn:aws:backup:us-east-1:XXXXXXX15457:backup-vault:destinationvault",           "CreationDate": "2022-10-03T22:56:44.129000+01:00",           "EncryptionKeyArn": "arn:aws:kms:us-east-1:XXXXXXX15457:key/xxxxxxxxx-aa4f-48834efceebe",           "CreatorRequestId": "xxxxxxxxxx-ea7cb20a2a01",           "NumberOfRecoveryPoints": 5,           "Locked": false       },      {            "BackupVaultName": "s3Backup",            "BackupVaultArn": "arn:aws:backup:us-east-1:XXXXXX5457:backup-vault:s3Backup",            "CreationDate": "2022-02-23T08:45:08.904000+00:00",            "EncryptionKeyArn": "arn:aws:kms:us-east-1:XXXXXXX5457:key/xxxxxxxxx-e4e294b5e1ff",            "CreatorRequestId": "xxxxxxxxxx-5903d602b45a",            "NumberOfRecoveryPoints": 0,            "Locked": true,            "MinRetentionDays": 1,            "LockDate": "2023-03-26T12:05:24.117000+01:00"        }The preceding example output provides the following information:The s3Backup and Vault100 vaults have a vault lock turned on because Locked is set to true.The destinationvault vault doesn't have a vault lock turned on because Locked is set to false.The s3Backup vault uses compliance mode because a LockDate is populated.The Vault100 vault uses governance mode because it doesn't contain a LockDate.Related informationEnhance the security posture of your backups with AWS Backup Vault LockAWS account closure with a locked vaultFollow"


Question: How do I handle Spot termination notices in AWS Fargate Spot tasks?

I want to know how to handle Spot termination notices in AWS Fargate Spot tasks.

Answer: "I want to know how to handle Spot termination notices in AWS Fargate Spot tasks.Short descriptionYou can use Fargate Spot to run interruption tolerant Amazon Elastic Container Service (Amazon ECS) tasks. A termination notice is a two-minute warning that you receive before the Fargate Spot task is due for termination. This warning helps you to manage Spot interruptions by giving your applications time to prepare for a graceful shutdown. The termination notice is created as soon as the Fargate Spot task is marked for termination. The notice indicates the time when the Fargate Spot task that's running will be terminated. The warning is sent as a task state change event to Amazon EventBridge and as a SIGTERM signal to the running task.To be sure that the containers on Fargate Spot exit before the task stops, specify a stopTimeout value of 120 seconds or less in the container definition used by the task. Specifying a stopTimeout value gives the container the time to exit normally. After this duration is elapsed, the container is forcefully stopped.Note: You can specify a maximum value of 120 seconds for stopTimeout. If you don't specify any value for this parameter, then the default value of 30 seconds is used.When the interruption signal is received on Amazon ECS services using Fargate Spot, the service scheduler determines if additional capacity is available. The service scheduler uses the minimumHealthyPercent and maximumPercent values to make this determination. If capacity is available, then the service scheduler attempts to launch additional tasks on Fargate Spot. However, if the service scheduler fails to find the capacity for new tasks, then the old tasks are terminated after the stopTimeout duration elapses.Consider the following when you use Fargate Spot with a load balancer:Tasks that are run as FARGATE_SPOT might not be deregistered from a load balancer’s target group until the task transitions to a STOPPED state.With FARGATE_SPOT, you get only two minutes to deregister the target group before the task is shut down. This means that you must set the deregistration delay for any target groups associated with FARGATE_SPOT to a value less than two minutes.ResolutionWhen capacity is unavailable, or the capacity is taken back for Fargate Spot, the ECS service scheduler can't launch new tasks. Instead, the scheduler terminates the existing tasks after providing a two-minute notice. However, these events aren't reported in the Amazon ECS console.The events from ECS are delivered to EventBridge in near real time. Therefore, it's a best practice to write simple rules to indicate your chosen events and automated actions for when an event matches a rule. This article covers EventBridge rules for the following use cases:A FARGATE_SPOT task is shut down due to Fargate Spot interruption.A FARGATE_SPOT task can't be placed due to the unavailability of Fargate Spot capacity.A FARGATE_SPOT task is shut down due to Fargate Spot interruptionThe following is a snippet of a task state change event displaying the stopped reason and stop code for a Fargate Spot interruption:{  "version": "0",  "id": "a99d3f53-4a7c-4153-a1a5-48957fc83b8f",  "detail-type": "ECS Task State Change",  "source": "aws.ecs",  "account": "1111222233334444",  "resources": [    "arn:aws:ecs:ap-southeast-2:1111222233334444:task/4be29e5b-b05c-42a2-a596-be62090eea9b"  ],  "detail": {    "clusterArn": "arn:aws:ecs:ap-southeast-2:1111222233334444:cluster/default",    "createdAt": "2022-02-25T10:13:08.455Z",    "desiredStatus": "STOPPED",    "lastStatus": "RUNNING",    "stoppedReason": "Your Spot Task was interrupted.",    "stopCode": "SpotInterruption",    "taskArn": "arn:aws:ecs:ap-southeast-2:1111222233334444:task/4be29e5b-b05c-42a2-a596-be62090eea9bEXAMPLE",    ...  }}Note that stopCode is mentioned as SpotInterruption when a task is stopped due to Fargate Spot interruption. You can create an EventBridge rule that sends an Amazon Simple Notification Service (Amazon SNS) alert whenever a FARGATE_SPOT task is stopped by SpotInterruption.Do the following:1.    Create an Amazon SNS topic.2.    Create an EventBridge rule for this use case.To create an EventBridge rule for this use case, do the following:1.    Open the Amazon EventBridge console.2.    In the navigation pane, choose Rules.3.    Choose Create rule.4.    Enter a name and description for the rule.5.    For Event bus, select AWS default event bus.6.    For Rule type, select Rule with an event pattern.7.    Choose Next.8.    For Event source, select AWS services.9.    For Event pattern, choose Custom patterns (JSON editor), and add the following pattern:{  "source": [    "aws.ecs"  ],  "detail-type": [    "ECS Task State Change"  ],  "detail": {    "stopCode": [      "SpotInterruption"    ],    "clusterArn": [      "arn:aws:ecs:exampleregion:1111222233334444:cluster/examplecluster"    ]  }}10.    Choose Next.11.    For Target types, select AWS service.12.    For Select a target, select SNS topic.13.    For Topic, select the SNS topic that you created.14.    Choose Next.15.    In the configure tags - optional page, choose Next.16.    Review the options and choose Create rule.A FARGATE_SPOT task can't be placed due to the unavailability of Fargate Spot capacityThe following is a snippet of the service task placement failure event that occurred under the conditions:The task was trying to use the FARGATE_SPOT capacity provider.The service scheduler couldn't acquire any Fargate Spot capacity.{  "version": "0",  "id": "403b98b2-616e-4ec7-8dff-b2cba8d5bf64",  "detail-type": "ECS Service Action",  "source": "aws.ecs",  "account": "1111222233334444",  "time": "2022-02-25T14:56:32.756Z",  "region": "ap-southeast-2",  "resources": [    "arn:aws:ecs:ap-southeast-2:1111222233334444:service/default/servicetest"  ],  "detail": {    "eventType": "ERROR",    "eventName": "SERVICE_TASK_PLACEMENT_FAILURE",    "clusterArn": "arn:aws:ecs:ap-southeast-2:1111222233334444:cluster/default",    "capacityProviderArns": [      "arn:aws:ecs:ap-southeast-2:1111222233334444:capacity-provider/FARGATE_SPOT"    ],    "reason": "RESOURCE:FARGATE",    "createdAt": "2022-02-25T14:21:04.163Z"  }}When a task can't be placed due to unavailable Fargate Spot capacity, the eventName is mentioned as SERVICE_TASK_PLACEMENT_FAILURE. This means that you can create an EventBridge rule that sends out an SNS alert whenever a FARGATE_SPOT task can't be placed.Do the following:1.    Create an SNS topic.2.    Create an Amazon EventBridge rule for this use case. To do this, use the instructions provided in the section A FARGATE_SPOT task is shut down due to Fargate Spot interruption except for the following change:For Event pattern, choose Custom patterns (JSON editor), and add the following pattern:{  "source": [    "aws.ecs"  ],  "detail-type": [    "ECS Deployment State Change"  ],  "detail": {    "eventName": [      "SERVICE_TASK_PLACEMENT_FAILURE"    ],    "clusterArn": [      "arn:aws:ecs:example-region:1111222233334444:cluster/example-cluster"    ],    "reason": [      "RESOURCE:FARGATE"    ]  }}Related informationHandling Fargate Spot termination noticesCreating Amazon EventBridge rules that react to eventsFollow"


Question: How can I troubleshoot Amazon EKS pods on AWS Fargate that are stuck in a Pending state?

My Amazon Elastic Kubernetes Service (Amazon EKS) pods that are running on AWS Fargate instances are stuck in a Pending state.

Answer: "My Amazon Elastic Kubernetes Service (Amazon EKS) pods that are running on AWS Fargate instances are stuck in a Pending state.Short descriptionHere are some common scenarios that cause pods to remain stuck in the Pending state on Amazon EKS using AWS Fargate:There's a capacity error because a particular vCPU and memory combination is unavailable.You created the CoreDNS pods with a default annotation that maps them to the Amazon Elastic Compute Cloud (Amazon EC2) compute type. To schedule them on a Fargate node, remove the Amazon EC2 compute type.The pod didn't match any Fargate profiles when you created it and isn't assigned to the fargate-scheduler. If a pod isn't matched on creation, then it isn't automatically rescheduled to Fargate nodes. This is true even if you create a matching profile later. In this case, the pod is assigned to the default-scheduler.If the pod is assigned to the fargate-scheduler but remains in a Pending state, then the pod might require additional troubleshooting.Before troubleshooting, note the Fargate following pod rules:You must configure namespace and match labels for your pod selectors. Fargate workflow matches pods to a Fargate profile only if both conditions match the pod specification.If you specify multiple pod selectors within a single Fargate profile, then fargate-scheduler schedules the pod if it matches any of the selectors.If a pod specification matches with multiple Fargate profiles, then the pod is scheduled according to a random Fargate profile. To avoid this, you can use the annotation eks.amazonaws.com/fargate-profile:fp_name within the pod specification.Note: Replace fp_name with your Fargate profile name.ResolutionImportant: The following steps apply only to pods launched with AWS Fargate. For information on pods launched on Amazon EC2 instances, see How can I troubleshoot the pod status in Amazon EKS?Find out the status of your pod1.    Run the following command to check your pod state:kubectl get pods -n <namespace>2.    To get more error information about your pod, run the following describe command:kubectl describe pod YOUR\_POD\_NAME -n <namespace>Refer to the output of the describe command to evaluate which of the following resolutions will help troubleshoot your issue.Resolving capacity errorIf your pods have a capacity issue, then the describe output is similar to the following:Fargate capacity is unavailable at this time. Please try again later or in a different availability zoneThis output means that Fargate can't provision compute capacity based on the vCPU and memory combination that you selected.To resolve the error:Retry creating the pod after 15-20 minutes. Because the error is capacity-based, the exact amount of time can vary.Change the request (CPU and memory) within your pod specification (from the Kubernetes website). The Fargate workflow then provisions a new combination of vCPU and memory.Note: You're billed based on one of your combinations. For more information on how the combination is finalized based on your pod specification, see Pod CPU and memory. Performing a kubectl describe node command from your terminal/IDE can give you a much higher vCPU and memory combination value. Fargate doesn't always have available capacity based on your requests and provisions resources from a capacity pool on a best effort basis. However, you're billed only for pod usage and equivalent vCPU and memory combination.Resolving CoreDNS pods in a Pending stateIf CoreDNS pods are in a Pending state, then you see an output that's similar to the following message:kubectl get pods -n kube-system  NAME                                     READY   STATUS     RESTARTS      AGEcoredns-6548845887-qk9vf                 0/1     Pending    0             157mThis might be because CoreDNS deployment has the following default annotation: eks.amazonaws.com/compute-type : ec2.To resolve this and re-assign the pods to the Fargate scheduler, see Update CoreDNS.Troubleshooting pods assigned to fargate-schedulerThere are multiple reasons why pods assigned to fargate-scheduler might be stuck in Pending state, ranging from misconfiguration of pod annotation to networking issues. If your pods remain in a Pending state, then the describe output is similar to the following message:Events:Type       Reason              Age                     From     ----       ------              ----                    ----     Warning    FailedScheduling    2m25s (x301 over 5h3m)  fargate-schedulerTo troubleshoot this error:Delete and recreate the pods.Confirm the following specifications aren't set in the pod specification YAML:node selector<>node nameschedulerNameThese specifications cause the fargate-scheduler to skip the pod.Confirm that the subnets selected in your Fargate profile have enough free IP addresses to create new pods. Each Fargate node consumes one IP address from the subnet.Confirm that the NAT gateway is set to a public subnet, and has an Elastic IP address attached to it.Confirm that the DHCP option sets that are associated with your virtual private cloud (VPC) have an AmazonProvidedDNS or a valid DNS server hostname for domain-name-servers.Confirm that DNS hostnames and DNS resolution is turned on for your VPC.If your Fargate pods use private subnets with only VPC endpoints configured for service communication, then you must allow these endpoints with DNS names:ECR - APIECR - DKRS3 Gateway endpointConfirm the security group that's attached to the VPC endpoint allows communication from Fargate to and from the API server. The VPC endpoint security group must allow port 443 ingress from the cluster VPC CIDR. You must also turn on private endpoint access for your cluster.Resolving pods assigned to default-schedulerTo determine the scheduler that your pods are assigned to, run the following command:kubectl get pods -o yaml -n <namespace> <pod-name> | grep schedulerName.In the output, confirm that the schedulerName is fargate-scheduler. If it's listed as default-scheduler, then the fargate-scheduler skipped this pod. To troubleshoot this issue, check your pod configuration for compute-type annotations and refer to AWS Fargate considerations.Follow"


Question: How do I troubleshoot the latency of Amazon EBS volumes caused by an IOPS bottleneck in my Amazon RDS instance?

I have an Amazon Relational Database Service (Amazon RDS) DB instance. I want to troubleshoot the latency of the Amazon Elastic Block Store (Amazon EBS) volumes in my Amazon RDS instance.

Answer: "I have an Amazon Relational Database Service (Amazon RDS) DB instance. I want to troubleshoot the latency of the Amazon Elastic Block Store (Amazon EBS) volumes in my Amazon RDS instance.ResolutionThe most common reasons for latency in an Amazon RDS instance that's caused by an IOPS or throughput bottleneck include the following:An IOPS bottleneck at the instance levelAn IOPS bottleneck at the volume levelA throughput bottleneck at the instance levelA throughput bottleneck at the volume levelMicro-burstingUse the following troubleshooting steps based on your use case.RDS instance with General Purpose SSD (gp2)Perform the following checks:Check the configuration information of the Amazon RDS instance, such as the DB instance class and storage size. This information can help you track the IOPS and throughput limits. You must know these values when you troubleshoot issues causing an IOPS or throughput bottleneck.Use the Amazon CloudWatch graphs to check for any spikes in the values of DiskQueueDepth, ReadLatency, and WriteLatency. Under normal circumstances, it's a best practice to use a DiskQueueDepth of one per minute for every 1000 IOPS. ReadLatency and WriteLatency are expected to be less than 10 milliseconds. If you notice spikes, then identify the time of the spike.Use the CloudWatch graphs to view the ReadIOPS and WriteIOPS metrics. Check if the IOPS limit was breached at the volume level during the timeframe of the spikes in the values of DiskQueueDepth, ReadLatency, and WriteLatency.Use the CloudWatch graph to check if there is a drop in the value of BurstBalance. This check is applicable only for volumes with a size of less than 1 TB. A drop in the value of BurstBalance confirms the occurrence of an IOPS bottleneck during the timeframe of the spike.Use the CloudWatch graphs to view the ReadThroughput and WriteThroughput metrics. Check if the throughput limit was breached at the volume level during the timeframe of the spikes in the values of ReadThroughput and WriteThroughput.If you're using an EBS-optimized RDS instance class, then use the CloudWatch graphs to check for throttling of IOPS or throughput. For instance classes with burst capacity, view the EBSIOBalance% and EBSByteBalance% metrics in the CloudWatch graphs. Consistently low values of EBSIOBalance% or EBSByteBalance% indicate an IOPS or throughput bottleneck at the instance level.Throttling of IOPS, throughput, or both indicates that the IOPS or throughput is inadequate for your workload at the storage level. To fix this issue, do the following:Locate the SQL queries that create more load on the database, and then optimize these queries. If the workload is as expected, or there is no scope for tuning the SQL queries, then you might need to increase the storage size to get a higher IOPS capacity.Note: After you increase the storage size of an RDS instance, you can't reduce the size to the previous value.Consider switching the volume from General Purpose (gp2) to Provisioned IOPS (io1). If the DB instance is Single-AZ and you're using a custom parameter group, then switching between gp2 and io1 might cause a brief downtime. If your instance is Multi-AZ, then you don't experience any downtime.If you notice throttling of IOPS or throughput at the instance level, then you must scale up the instance class to get a higher IOPS or throughput capacity.RDS instance with Provisioned IOPS (io1)Check the configuration information of the Amazon RDS instance, such as the DB instance class and defined Provisioned IOPS, to determine the IOPS limit or throughput limit for the DB instance class.Use the CloudWatch graphs to check for any spikes in the values of DiskQueueDepth, ReadLatency, and WriteLatency. Under normal circumstances, it's a best practice to use a DiskQueueDepth of one per minute for every 1000 IOPS. ReadLatency or WriteLatency are expected to be within 10 milliseconds. If you notice spikes, then identify the time of the spike.Use the CloudWatch graphs to view the ReadIOPS and WriteIOPS metrics. Check if the IOPS limit was breached during the timeframe of spikes in the values of DiskQueueDepth, ReadLatency, and WriteLatency.Use the CloudWatch graphs to view the ReadThroughput and WriteThroughput metrics. Check if the throughput limit was breached during the timeframe of spikes in the values of ReadThroughput and WriteThroughput.If you're using an EBS-optimized RDS instance class, then use the CloudWatch graphs to check for throttling of IOPS or throughput. For instance classes with burst capacity, view the EBSIOBalance% and EBSByteBalance% metrics in the CloudWatch graphs. Consistently low percentage values of EBSIOBalance% or EBSByteBalance% respectively indicate an IOPS or throughput bottleneck at the instance level.Throttling of IOPS or throughput indicates that the IOPS or throughput is inadequate for the workload at the storage level. To fix this issue, do the following:Locate the SQL queries that create more load on the database, and then optimize these queries. If the workload is as expected or there is no scope for tuning the SQL queries, then you might need to increase the IOPS provisioned.If you notice throttling of IOPS or throughput at the instance level, then you need to scale up the instance class to get a higher throughput or IOPS capacity.Micro-burstingMicro-bursting occurs when an EBS volume "bursts" high IOPS or throughput for significantly shorter periods than the collection period. CloudWatch metrics are collected at an interval of 60 seconds. Because the volume bursts high IOPS or throughput for a shorter time than the collection period, CloudWatch doesn't reflect the bursting. You can use Enhanced Monitoring to identify if microbursting causes the latency. Turn on Enhanced Monitoring with a granularity of 1 second. You can use the Read IO/s and Write IO/s metrics to determine the actual IOPS utilization. You can use the Read Kb/s and Write Kb/s to determine the actual throughput utilization per second. For more information, see Enhanced Monitoring metric descriptions.Related informationAmazon RDS metricsUnderstanding burst vs. baseline performance with Amazon RDS and GP2Hardware specifications for DB instance classesHow can I identify if my EBS volume is micro-bursting and how can I prevent this from happening?Determining the IOPS needs for Oracle database on AWSFollow"


Question: Why does my IAM user have access to Data Catalog resources even after their permissions are restricted in Lake Formation?

"I restricted table permissions for a user in AWS Lake Formation (for example, SELECT with ExcludedColumnNames). However, the user can still query the table using Amazon Athena."

Answer: "I restricted table permissions for a user in AWS Lake Formation (for example, SELECT with ExcludedColumnNames). However, the user can still query the table using Amazon Athena.Short descriptionLake Formation grants the Super permission to the IAMAllowedPrincipals group by default. This permission applies to all existing AWS Glue Data Catalog resources. If Use only IAM access control is enabled, then Lake Formation also grants the Super permission on new resources. This means that the AWS Identity and Access Management (IAM) policies control access to Data Catalog resources and Amazon Simple Storage Service (Amazon S3) locations. For more information, see About upgrading to the Lake Formation permissions model.To restrict access, change the default settings to make sure that you're using the Lake Formation permissions model for new and existing Data Catalog resources.ResolutionTo update the security settings for all new databases and tables in your data lake, see Change Data Catalog settings.To update the security settings only for certain new resources, create the resource, and then update the permissions only on that resource. For more information, see Secure existing Data Catalog resources.The change in the default security settings applies only to the resources that are created after security settings are updated.To change the security settings for existing databases, explicitly remove the permissions from IAMAllowedPrincipals on each database by doing the following:Open the AWS Lake Formation console.In the navigation pane, under Data catalog, choose Databases.Select the circle next to the database that you want to update.Choose Actions, and then choose Edit.Under Default permissions for newly created tables, clear Use only IAM access control for new tables in this database.Choose Save.On the Databases page, be sure that the database is selected. Choose Actions, and then choose Revoke.In the Revoke permissions dialog box, in the IAM users and roles list, scroll down to the Group heading, and then choose IAMAllowedPrincipals.Under Database permissions, be sure that Super is selected, and then choose Revoke.To change the security settings for existing tables, explicitly remove the permissions from IAMAllowedPrincipals on each table by doing the following:Open the AWS Lake Formation console.In the navigation pane, choose Tables.On the Tables page, select the circle next to the table that you want to update.Choose Actions, and then choose Revoke.In the Revoke permissions dialog box, in the IAM users and roles list, scroll down to the Group heading, and then choose IAMAllowedPrincipalsUnder Table permissions, be sure that Super is selected, and then choose RevokeNote: Be sure that the Amazon S3 path to the database or table is registered with Lake Formation.After changing the security settings for existing resources, grant table permissions to the IAM user or role. You can do this using the Lake Formation console or the AWS Command Line Interface (AWS CLI).Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.The following are examples of CLI commands that grant the SELECT permission on specific columns:Allow SELECT on a specific column only:aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT"  --resource '{ "TableWithColumns": {"DatabaseName":"retail", "Name":"inventory", "ColumnNames": ["prodcode","location","period","withdrawals"]}}'Exclude columns (allow SELECT on all columns except the column that's specified under ExcludedColumnNames):aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT"  --resource '{ "TableWithColumns": {"DatabaseName":"retail", "Name":"inventory", "ColumnWildcard": {"ExcludedColumnNames": ["location"]}}}'To perform Lake Formation operations, principals must have Lake Formation and IAM permissions. In addition to this to query the table in Athena user must also have permission to run Athena queries. For example, to query a table from Athena when Lake Formation is managing permissions, the user must have the following IAM permissions:{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "minimumAWSGlueLakeFormationAthenapermissiontorunselectqueryontable",            "Effect": "Allow",            "Action": [                "lakeformation:GetDataAccess",                "glue:SearchTables",                "glue:GetTables",                "glue:GetPartitions",                "glue:GetDatabases",                "glue:GetTable",                "glue:GetDatabase",                "athena:GetNamespace",                "athena:ListWorkGroups",                "athena:GetCatalogs",                "athena:GetNamespaces",                "athena:GetExecutionEngine",                "athena:GetExecutionEngines",                "athena:GetTables",                "athena:GetTable"            ],            "Resource": "*"        },        {            "Sid": "AthenaPermissionToRunQueryInAWorkergroup",            "Effect": "Allow",            "Action": [                "athena:StartQueryExecution",                "athena:GetQueryResults",                "athena:DeleteNamedQuery",                "athena:GetNamedQuery",                "athena:ListQueryExecutions",                "athena:StopQueryExecution",                "athena:GetQueryResultsStream",                "athena:ListNamedQueries",                "athena:CreateNamedQuery",                "athena:GetQueryExecution",                "athena:BatchGetNamedQuery",                "athena:BatchGetQueryExecution"            ],            "Resource": [                "arn:aws:athena:*:*:workgroup/*"            ]        },        {            "Sid": "allowS3PermissionToSaveAthenaQueryResults",            "Effect": "Allow",            "Action": [                "s3:PutObject",                "s3:GetObject",                "s3:ListBucketMultipartUploads",                "s3:AbortMultipartUpload",                "s3:CreateBucket",                "s3:ListBucket",                "s3:GetBucketLocation",                "s3:ListMultipartUploadParts"            ],            "Resource": [                "arn:aws:s3:::aws-athena-query-results-*"            ]        }    ]}Related informationChanging the default security settings for your data lakeFollow"