Question: How can I troubleshoot managed node group update issues for Amazon EKS?

I'm trying to update my Amazon Elastic Kubernetes Service (Amazon EKS) managed node group and am experiencing issues.

Answer: "I'm trying to update my Amazon Elastic Kubernetes Service (Amazon EKS) managed node group and am experiencing issues.Short descriptionWhen you update your Amazon EKS managed node group, you might receive one of the following errors:"PodEvictionFailure Reached max retries while trying to evict pods from nodes in node group nodegroup-1234""Error: Nodegroup health has issues other than AsgInstanceLaunchFailures, InstanceLimitExceeded, InsufficientFreeAddresses, ClusterUnreachable""Error: InvalidParameterException: Launch template details can't be null for Custom ami type node group"          -or-         "An error occurred (InvalidParameterException) when calling the UpdateNodegroupVersion operation: You cannot specify the field kubernetesVersion when using custom AMIs""UPDATE_FAILED Resource handler returned message: Requested release version 1.xx is not valid for kubernetes version 1.yy"ResolutionTo resolve Amazon EKS managed node group update errors, follow these troubleshooting steps.Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.PodEvictionFailure Reached max retries while trying to evict pods from nodes in node group nodegroup-1234The PodEvictionFailure error occurs when the upgrade can't drain all the pods from the node. If a pod disruption budget (PDB) is preventing pods from being drained from the node, then this can cause the issue. For example, if an application has a PDB of two, then at least two pods for that application must be running.To check if your application is using a PDB, run the following kubectl command:$ kubectl get pdb —all-namespacesIf you're using a PDB, then take one of the following actions:Increase the number of available IP addresses for your Amazon Elastic Compute Cloud (Amazon EC2) nodes to increase the number of pods.Temporarily remove the PDB.Use the force update option.The force update option doesn't recognize PDBs. Updates occur regardless of PDB issues by forcing the node to restart.Note: If pods from a Kubernetes controller aren't spread across the nodes, then this option might cause downtime for your applications.To use the force option, run the AWS CLI command similar to the following one:$ aws eks update-nodegroup-version --cluster-name cluster-123 --nodegroup-name nodegroup-1234 --force-or-Run the following eksctl command:$ eksctl upgrade nodegroup --cluster OneCluster --name managed-ng --force-upgradeTroubleshooting PodDisruptionBudget eviction failures with CloudWatch Logs InsightsYou can use Amazon CloudWatch Logs Insights to search through the Amazon EKS control plane log data. For more information, see Analyzing log data with CloudWatch Logs Insights.Important: You can view log events in CloudWatch Logs only after you turn on control plane logging in a cluster. Before you select a time range to run queries in CloudWatch Logs Insights, verify that you turned on control plane logging. For more information, see How can I retrieve Amazon EKS control plane logs from CloudWatch Logs?To identify the pod that failed eviction and the failure count, run a query similar to the following one:fields @timestamp, @message | stats count(*) as count by objectRef.name| filter @logStream like /audit/ | filter user.username == "eks:node-manager" and requestURI like "eviction" and requestURI like "pod" and responseStatus.code > 400| sort count descThe maximum number of retries for an eviction pod is 20. If the count for a displayed pod is greater than or equal to 20 failures, then this is the pod that failed eviction.To identify the pod disruption budget name that's blocking the preceding pod from being evicted, run the following query.filter @logStream like /^kube-apiserver-audit/  | fields @logStream, @timestamp, @message  | sort @timestamp desc  | filter user.username == "eks:node-manager" and requestURI like "eviction" and requestURI like "pod_name" and responseStatus.code > 400  | limit 999  | display responseObject.details.causes.0.message,objectRef.name,objectRef.namespace,objectRef.resourceNote: Replace pod_name with your pod name.The output has a message similar to following one, where pod_distruption_budget is the object that's causing the eviction failures:The disruption budget pod_distruption_budget needs 1 healthy pods and has 1 currentlyError: Nodegroup health has issues other than AsgInstanceLaunchFailures, InstanceLimitExceeded, InsufficientFreeAddresses, ClusterUnreachableIf you receive this error message, then check the managed node group details and locate the health issues. For more information and troubleshooting, check for managed node group errors and the Amazon EKS issue API reference.Error: InvalidParameterException: Launch template details can't be null for Custom ami type node group-or-An error occurred (InvalidParameterException) when calling the UpdateNodegroupVersion operation: You cannot specify the field kubernetesVersion when using custom AMIs.For managed node groups with a custom AMI, you must create a new AMI version with the Kubernetes version that you want to upgrade to. During the upgrade, specify the launch template and the version.If you're using the AWS CLI, then use the flag --launch-template. For eksctl, use the flag --launch-template-version.Note: Avoid using the --kubernetes-version flag with these commands.UPDATE_FAILED Resource handler returned message: "Requested release version 1.xx is not valid for kubernetes version 1.yy"This error occurs when upgrading the managed node group with the Amazon EKS cluster from the same AWS CloudFormation stack with the UpdateStack API call.CloudFormation tries to roll back the stack, but it fails because the Amazon EKS cluster upgraded successfully and can't be reverted. The Amazon EKS cluster can't match 1.xx with 1.yy (for example, 1.21 and 1.22).Check the first error that occurred for the node group in the CloudFormation stack for more details on how to fix the issue. Then, update your CloudFormation stack again.For more information, see Managed node update behavior.Related informationHow do I troubleshoot Amazon EKS managed node group creation failures?How do I resolve managed node group errors in an Amazon EKS cluster?Amazon EKS troubleshootingFollow"


Question: What do I do if I receive the error "Your account is in an invalid state" when I try to access services through my AWS account?

"I received the error message “Please contact customer service, your account is in an invalid state” when I tried to access the services through my AWS account."

Answer: "I received the error message “Please contact customer service, your account is in an invalid state” when I tried to access the services through my AWS account.ResolutionAWS accounts can take up to 24 hours to fully activate. If your account was created recently, wait a few hours.If you created your AWS account more than 24 hours ago, then check the inbox of the email address associated with the AWS account root user of your account. Occasionally, AWS requires additional documents or verification steps to fully activate your account.If the error message persists, create a case with AWS Support, and then choose Account and billing support.Related informationHow do I create and activate a new AWS account?Follow"


Question: What actions do I need to take for an ElastiCache self-service update?

I received a notification for an Amazon ElastiCache self-service update and I want to know what actions I need to take.

Answer: "I received a notification for an Amazon ElastiCache self-service update and I want to know what actions I need to take.Short descriptionAmazon ElastiCache automatically monitors your fleet of Redis and MemCached nodes to apply service updates as they become available. Service updates help strengthen security, reliability, and operational performance of Amazon ElastiCache. Typically, you set up a predefined maintenance window so that Amazon ElastiCache can apply these updates. However, in some cases you might find this approach too rigid and likely to constrain your business flows. Self-service update features allow you to apply updates at your discretion.ResolutionHow can I view service update notifications?If you have qualifying clusters, Amazon ElastiCache notifies you of available updates through the following methods:ElastiCache console notificationEmailsAmazon Simple Notification Service (SNS) messagesAWS Personal Health Dashboard (PHD) notificationsAmazon CloudWatch EventsYou might receive multiple notifications for the service update throughout the life cycle of the update. The number of notifications that you receive for a service update depends on the type of update, such as mandatory or non mandatory. Notifications also depend on what actions you perform on your cluster. For example, if you apply the mandatory update before the deadline, you don't receive reminders or auto update notifications.For information on setting up SNS notifications, see Monitoring ElastiCache events.Note: Notifications aren't sent for nodes launched or replaced after the release date specified in the Service Update description.How can I check available service updates?After receiving notification about available service updates, you can retrieve them using the AWS Command Line Interface (AWS CLI) describe-service-updates command. For more information, see describe-service-updates.Note: If you receive errors when running AWS CLI commands, make sure that you're using the most recent AWS CLI version.You can also check available service updates from the ElastiCache console. Use the following instructions to view updates from the console:Open the ElastiCache console.Choose Service updates. You can view all service updates, descriptions, and the status for your ElastiCache fleet.How do I determine what's being fixed in the service update?Check the Update description section in the service update details to understand the fix or improvement.How do I apply the service update?You can apply the service updates as soon as it's released and is available. It's a best practice to apply the update as soon as you can after receiving the service update notification.To apply a service update on your cluster, run the AWS CLI batch-apply-update-action command. For more information, see batch-apply-update-action.To apply the service update using the ElastiCache console, use the following instructionsOpen the ElastiCache console.Choose Service Updates.Select the service update that you want to apply and then choose Apply now. For more information, see Applying the service updates using the console.Note: You can apply only those service updates with an Available or Scheduled status.What is the impact of applying a service update?For Redis, the update is applied to no more than one node at a time within each shard. The nodes being updated experience downtime of few seconds, while the rest of the Redis cluster continues to serve traffic. The replacement might take longer in certain instance configurations and traffic patterns. For example, Redis primary nodes might not have enough free memory, and might be experiencing high write traffic.For Memcached nodes, the replacement process brings up an empty new node and terminates the current node. The cache contents are removed to start fresh. The new node is unavailable for a short period during the switch. After they're switched, your application might see performance degradation while the empty new node is populated with cache data. To minimize the impact, the update is applied to one node at a time. Because Memcached nodes don't need to sync, their replacement completes faster, regardless of node sizes.Note: For both Redis and MemCached, there is no change in the cluster configuration. You might see a delay in your Amazon CloudWatch metrics that catch up as soon as possible.To minimize impact during the service update, refer to the following three questions in Amazon ElastiCache managed maintenance and service updates help page:How long does a node replacement take?How does a node replacement impact my application?What best practices should I follow for a smooth replacement experience and minimize data loss?Is the service update mandatory?If the Auto-update after due date attribute is set to Yes, then it's a mandatory service update and must be applied.What do I do if the service updates Apply-by date has passed?If you don't apply the updates by the Apply-by date, Amazon ElastiCache starts scheduling the update in your maintenance window. The update is scheduled in the cluster’s maintenance window. You will receive a new notification one week before the scheduled date. Amazon ElastiCache won't apply the update in your maintenance window without advance notification.Note: You can still apply the service update before the scheduled automatic update date even after the Apply-by date. After you apply the update, the scheduled automatic update is voided.What do I do with expired service updates?You can't apply an expired service update. However, new nodes contain all the applicable service updates. You can manually replace the existing nodes that weren't updated to get the latest updates. Nodes launched after the service update's Release date automatically receive past updates. For more information on replacing nodes manually, see Replacing nodes (Redis) and Replacing nodes (MemCached).Note: If you miss applying the update before the Update expiration date, the update is included in the next service update.What do I do if I there are multiple service updates for the same cluster?Service updates are always cumulative. If you have more than one service update for the same cluster for security and performance improvement, you can apply the latest one. All outstanding applicable updates are applied.How do I reschedule a mandatory service update?Security is a shared responsibility. So, it's a best practice to apply the service update as soon as possible. If the scheduled time isn't suitable for you, you can defer the service update by changing the maintenance window. After you change the maintenance window and the scheduled date has passed, the service update is rescheduled to the newly specified window in the following weeks. You receive a new notification one week before the new date.How do I reschedule an optional service update?It's a best practice to apply all self-service updates in a timely manner to maintain compliance, regardless of whether the update is mandatory or not. If the service update has the attribute Auto-update after due date set to No, Amazon ElastiCache won't auto apply the update. If you don't install the update by the Update expiration date, it's skipped and there is no action needed. If you want to apply expired updates, see the previous section: What do I do with expired service updates?How do I get additional help?If you find an issue during or after applying a service update, contact the AWS Support team.Related informationService updated in ElastiCache for RedisFollow"


Question: How do I troubleshoot issues with contact trace records or applying filters in Amazon Connect?

I want to troubleshoot issues when viewing contact trace records or applying filters in Amazon Connect.

Answer: "I want to troubleshoot issues when viewing contact trace records or applying filters in Amazon Connect.ResolutionContact trace records (CTRs) capture the events associated with a contact in your contact center. Real-time and historical metrics are based on the data captured in the contact records.CTRs are generated in the following ways:After the call has ended and the Agent Status is moved from ACW to Available or Offline status.When a call is transferred to another agent or queue.Abandoned contacts, where a contact is disconnected by the customer when in queue.Not all calls result in CTRs. For example, short-lived contacts, such as a customer hanging up immediately after dialing, won't create a CTR. Amazon Connect doesn't create a CTR when there's no agent engagement. For more information, see Contact records data model.Amazon Connect delivers contact records at least once. Contact records can be delivered again for multiple reasons such as when new information arrives after initial delivery. For example, if you use update-contact-attributes to update a contact record, Amazon Connect delivers a new contact record. This contact record is available for 24 months from the time that the associated contact was initiated. For more information, see the Contact records data model.To troubleshoot issues with CTRs or applying filters, consider the following:Contact trace recordsTo troubleshoot issues with accessing contact trace records, review the following:User permissionsFor a user to view all contact records, be sure that the required permissions are assigned to the user. To add permissions to an existing security profile, see Update security profiles.Overlapping security profilesOverlapping security profiles might occur if a user has both a privileged security profile, such as Admin, and a less privileged profile, such as Agent. To avoid unexpected access issues from overlapping security profiles, make sure that all security profiles attached to a user have the Restrict contact access turned off.Agent hierarchyIf agent hierarchy isn't configured, then remove the user’s security profile permission for Restrict contact access. Turning off the Restrict contact access permission allows the user to view all contact records.Note: If Restrict contact access is turned on, then contact search results are restricted based on the agent's hierarchy.If agent hierarchy is configured, then a user's access to contacts is based on their agent hierarchy. This means that the user has access only to contacts handled by an agent within their hierarchy. This is because contact records must be classified with the user's agent hierarchy before they can be filtered. Be sure that you're checking the CTRs after the agent hierarchy is set up.Agent security profileTo view and edit the security profile for an agent, you must log in as a user with the following permissions:Users - View and Edit permissionsSecurity profiles - View and Edit permissionsTo access the security profile for an agent, do the following:Log in to your contact center at https://instance name.my.connect.aws/. You must be logged in with an Amazon Connect account that has permissions to update security profiles.In the navigation pane, choose Users, and then choose User management.Select the agent, and then choose Edit. The Security profiles section lists the security profiles associated with the agent.To add or remove permissions to an existing security profile, see Update security profiles.If the user has the correct permissions, security profile, and agent hierarchy, but still can't see the CTRs, then create a Support case. Be sure to include screenshots with the following information:The user's security profileThe agent hierarchyAn example CTR of Contact IDs that can't be accessed by that userUsing the agent filterTo use the Agent filter on the Contact search page, the user must have the required permissions to use the Agent search filter. To change security profiles, see Update security profiles.Using the custom contact attribute filterTo use the custom contact attribute filter, the user must have the required permissions to configure searchable contact attributes.After assigning the required permissions in the security profile, refresh the Contact search page, choose Add filter, and then choose Custom contact attribute.When using the custom contact attribute filter, consider the following:By default a custom attribute isn't indexed for search until a user with appropriate permissions specifies that it's searchable. Only a user with the Configure searchable contact attributes permission can configure the custom contact attributes search key.If the Contact attributes permission isn't present, then the user can't view the filter option of custom contact attribute.If the Configure searchable contact attributes permissions isn't present, the user can apply only previously configured searchable contact attributes. The user can't create new searchable contact attributes.If a Custom contact attribute is added, then users can search on Attribute keys for any future contacts. But the user can't search on Attribute keys for previously formed contacts.Related informationAnalyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSightFollow"


Question: How do I stream SES notifications across different Kinesis Data Streams accounts?

I want to set up Amazon Simple Email Service (Amazon SES) event notifications to send data to an Amazon Kinesis data stream in another account. How can I do this?

Answer:
"I want to set up Amazon Simple Email Service (Amazon SES) event notifications to send data to an Amazon Kinesis data stream in another account. How can I do this?Short DescriptionTo set up Amazon SES event notifications to an Amazon Kinesis data stream in another account, do the following:1.    Start at the SES source. Update your configuration settings in SES to push to an Amazon Simple Notification Service (Amazon SNS) topic. The updated setting configures AWS Lambda to subscribe to the topic as a trigger.2.    Subscribe to SNS.3.    Set up cross-account access.4.    Set up Lambda to send data to Amazon Kinesis Data Streams. Lambda then uses the code to grab the full event from the Amazon SNS topic. Lambda also assumes a role in another account to put the records into the data stream.5.    Use the Amazon Kinesis Client Library (KCL) to process records in the stream.ResolutionStart at the SES sourceCreate an SES environment and add your email account to the list. Verify the email account to send emails through the service to your own email address. Set up a configuration set for the emails in order to track all of the possible event types that can occur during testing. In the configuration set, set the destination to Simple Notification Service (SNS), using the default tracking domain.Note: You can create the topic when creating your configuration set, or you can create it before creating the set.For more information about configuration sets, see Using Amazon SES Configuration Sets.Subscribe to SNSTo test the notification topic created, you can temporarily subscribe your own email address in the SNS topic. This ensures that you receive notifications of the messages that come through from SES. Note that you'll need to reference the subscription list when configuring Lambda.Important: When testing, any of the openor clickevents can be tracked only in emails that are in HTML format. This is because AWS embeds an invisible GIF image, which contains a tracking token. This tracking token tracks any notification emails that are opened, submitting feedback for the event back to the topic.After SES and SNS are integrated, you can verify whether the integration was successful by using the SES console. Choose Email Addresseson the left side of the SES console. Then, choose the send a Test Emailbutton and select rawas the email format.Note: If you are using the formatted option, the email will not be sent in HTML. As a result, feedback for the openand clickemail events are also not sent.You can use the following template in the console:Subject: Amazon SES Raw Email TestMIME-Version: 1.0Content-Type: text/htmlX-SES-CONFIGURATION-SET: SESxLMxKIN<!DOCTYPE html>
This text should be large, because it is formatted as a header in HTML.

Here is a formatted link: Amazon Simple Email Service Developer Guide.
In this example template, note that the configuration set name is included in the headers. The configuration set name is included to use the configuration for the chosen events. This configuration set sends you an email and notifications for all of the actions that are taken with the email in your inbox. For additional reading on SNS, see Setting Up Amazon SNS Notifications.Set up cross-account accessCreate the setup to send the messages over to another account. This requires roles and policies to allow you to assume roles in another account.**Note:**You must have a role to assume in the destination account. The execution role for Lambda and the role that is going to be assumed must also be created. The role that is going to be assumed has the permissions to interact with the services needed.Here's an example template for the account with SES and Lambda, which you'll need to attach to the Lambda execution role:{    "Version": "2012-10-17",    "Statement": {        "Effect": "Allow",        "Action": "sts:AssumeRole",        "Resource": "arn:aws:iam::222222222222:role/role-on-source-account"    }}Here's an example template for the account with the data stream, which should be attached to the new role that Lambda assumes:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::111111111111:role/my-lambda-executionrole-source-account"            },            "Action": "sts:AssumeRole"        }    ]}This template defines the trust relationship of a Lambda execution role. The Lambda execution role assumes the role that will deliver data to the Kinesis data stream.**Note:**The role assumed by Lambda must have permissions to put records into a Kinesis data stream.Set up Lambda to send data to Kinesis Data StreamsYou can set up Lambda to send data to Data Streams by using a template like this:from __future__ import print_functionimport base64 #Kinesis only accepts base64 encoded payloads. import boto3  #Allow programmatic access to AWS services.import jsonprint('Loading function')def lambda_handler(event, context):    # Setting up the assumeRole for cross-account    sts_connection = boto3.client('sts')    acct_b = sts_connection.assume_role(        RoleArn="arn:aws:iam::[ADD_ACCOUNT_ID]:role/AssumeLambdaCrossAccount", #TODO: Add relevant role ARN        RoleSessionName="cross_acct_lambda"    )    print(acct_b)        ACCESS_KEY = acct_b['Credentials']['AccessKeyId']    SECRET_KEY = acct_b['Credentials']['SecretAccessKey']    SESSION_TOKEN = acct_b['Credentials']['SessionToken']        #Gets the SNS message that was posted (cliks, opens, sends, deliveries etc.)    message = event['Records'][0]['Sns']['Message'] #Fetching the message in the JSON structure received.    print("From SNS: " + message) #Checking the message received from SNS.        #TODO:Generate a random partitionKey for kinesis.     #If a random partitionKey is not used in a multi shard stream it will lead to performance problems    #PartitionKey = RandomKey        print("Add data to cross-account stream")    kinesis_client = boto3.client('kinesis',region_name="us-east-1",aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY,aws_session_token=SESSION_TOKEN) #Initiates the kinesis service with cross-account credentials..     kinesis_client.put_record(StreamName="CrossAccountTest",Data=json.dumps(message),PartitionKey="partitionKey")In this Python code example, replace RoleArnwith your stream name and Region to fit your environment. This function takes the SNS message received from SES, assuming the role from the destination account. The assumed role then puts messages into the destination stream.To configure Lambda as a subscriber to the SNS topic, you can follow these steps:1.    Sign in to the Amazon SNS console.2.    On the navigation pane, choose Topics.3.    On the Topicspage, choose a topic.4.    In the Subscriptionssection, choose Create subscription.5.    On the Create subscriptionpage, verify the chosen Topic ARN.6.    Choose AWS Lambdaas your protocol type.7.    Enter the Amazon Resource Name (ARN) of your function to establish the endpoint.8.    Choose Create subscription.For more information about using Lambda as a subscriber to SNS, see Using Amazon SNS for system-to-system messaging with an AWS Lambda function as a subscriber.If a message is published to an SNS topic that is subscribed by Lambda, Lambda gets invoked with the payload of the published message. For more information about how to create a sample message history store, see Invoking AWS Lambda functions via Amazon SNS.Note: If you don't want to set up cross-account access, the Credentialssections can be removed and written to a local Kinesis data stream. For more information about using Kinesis with Lambda and Python, see Kinesis in the AWS Boto3 documentation.Use the KCL to process recordsYou can use the KCL to process records in the Kinesis data stream. After the data is in the Kinesis data stream, you can build custom applications and implement Kinesis Data Analytics for analysis and visualization.Follow"


Question: How do I create VPC endpoints so that I can use Systems Manager to manage private EC2 instances without internet access?

My Amazon Elastic Compute Cloud (Amazon EC2) instance doesn't have internet access. How can I manage my instance using AWS Systems Manager?

Answer: "My Amazon Elastic Compute Cloud (Amazon EC2) instance doesn't have internet access. How can I manage my instance using AWS Systems Manager?ResolutionAmazon EC2 instances must be registered as managed instances to be managed with AWS Systems Manager. Follow these steps:Verify that SSM Agent is installed on the instance.Create an AWS Identity and Access Management (IAM) instance profile for Systems Manager. You can create a new role, or add the needed permissions to an existing role.Attach the IAM role to your private EC2 instance.Open the Amazon EC2 console, and then select your instance. On the Description tab, note the VPC ID and Subnet ID.Create a virtual private cloud (VPC) endpoint for Systems Manager.For Service Name, select com.amazonaws.[region].ssm (for example, com.amazonaws.us-east-1.ssm). For a full list of Region codes, see Available Regions.For VPC, choose the VPC ID for your instance.For Subnets, choose a Subnet ID in your VPC. For high availability, choose at least two subnets from different Availability Zones within the Region.Note: If you have more than one subnet in the same Availability Zone, you don't need to create VPC endpoints for the extra subnets. Any other subnets within the same Availability Zone can access and use the interface.For Enable DNS name, select Enable for this endpoint. For more information, see Access an AWS service using an interface VPC endpoint.For Security group, select an existing security group, or create a new one. The security group must allow inbound HTTPS (port 443) traffic from the resources in your VPC that communicate with the service.If you created a new security group, open the VPC console, choose Security Groups, and then select the new security group. On the Inbound rules tab, choose Edit inbound rules. Add a rule with the following details, and then choose Save rules:For Type, choose HTTPS.For Source, choose your VPC CIDR. For advanced configuration, you can allow specific subnets' CIDR used by your EC2 instances.Note the Security group ID. You'll use this ID with the other endpoints.Optional: For advanced setup, create policies for VPC interface endpoints for AWS Systems Manager.Note: VPC endpoints require AWS-provided DNS (VPC CIDR+2). If you're using a custom DNS, then use Amazon Route 53 Resolver for the correct name resolution. For more information, see the following:Access an AWS service using an interfaced VPC endpointResolving DNS queries between VPCs and your networkRepeat step 5 with the following change:For Service Name, select com.amazonaws.[region].ec2messages.Repeat step 5 with the following change:For Service Name, select com.amazonaws.[region].ssmmessages.After the three endpoints are created, your instance appears in Managed Instances, and can be managed using Systems Manager.Note: To use Session Manager, create the following VPC endpoints:System Manager: com.amazonaws.region.ssmSession Manager: com.amazonaws.region.ssmmessagesKMS: com.amazonaws.region.kms (Optional. This endpoint is required only if you want to use AWS Key Management Service (AWS KMS) encryption for Session Manager.)Amazon CloudWatch Logs (Optional. This endpoint is required only if you want to use Amazon CloudWatch Logs for Session Manager, Run Command).The EC2 VPC endpoint isn't required for connecting the instance to Session Manager. The EC2 VPC endpoint is required to create VSS-enabled snapshots of the instance.For more information, see Creating VPC endpoints for Systems Manager.Related informationAWS Systems Manager endpoints and quotasSystems Manager prerequisitesCreating VPC endpoints for Systems ManagerUse AWS PrivateLink to set up a VPC endpoint for Session ManagerFollow"


Question: How can I change the VPC of an Amazon Aurora for MySQL or PostgreSQL cluster?

How can I change the Amazon Virtual Private Cloud (Amazon VPC) that is associated with an Amazon Aurora cluster?

Answer: "How can I change the Amazon Virtual Private Cloud (Amazon VPC) that is associated with an Amazon Aurora cluster?Short descriptionYou can't modify the VPC of an Aurora cluster or instance. However, you can change the VPC of an Aurora cluster by using one of the following methods:Create a clone in a different VPC.Take a snapshot and then restore the snapshot in a different VPC.Set up replication using binary logging (MySQL only).ResolutionCreate a clone in a different VPCIf you clone a database in an Aurora cluster, then you can change the VPC of the clone. However, the subnets in the VPC must map to the same set of Availability Zones. For more information, see Cloning databases in an Aurora DB cluster.Take a snapshot and then restore the snapshot in a different VPCIf you create a snapshot of an Aurora instance or cluster, then you can restore the snapshot to the new VPC. For more information, see Restoring from a DB cluster snapshot.Set up replication using binary logging using MySQLIf you use Amazon Aurora for MySQL and you set up replication using binary logging, then you can change the VPC of an Aurora cluster with minimal downtime.Create a new Aurora cluster in the target VPC.Set up manual MySQL replication between the two Aurora clusters.Promote the replica to be a standalone cluster.Related informationWorking with an Amazon RDS DB Instance in a VPCHow do I change the VPC for an RDS DB instance?Follow"


Question: Why do I get the error "The service is unavailable. Please try again shortly" when modifying my Amazon EBS volume?

"When I try to modify my Amazon Elastic Block Store (Amazon EBS) volume, I get the error "The service is unavailable. Please try again shortly.""

Answer: "When I try to modify my Amazon Elastic Block Store (Amazon EBS) volume, I get the error "The service is unavailable. Please try again shortly."ResolutionWhen you're modifying an Amazon EBS volume, you might get the error "The service is unavailable. Please try again shortly." This error occurs because an operational activity is being performed on the systems that host your volume. The operational activity doesn't affect your volume performance or availability. You can continue to read or write during the operational activity. However, you can't modify your volume.You can view notifications about the AWS services that you use by logging in to the AWS Health Dashboard. You receive notifications on the dashboard when an operation for an AWS service begins and another one after the operation is complete. Continue to monitor the dashboard until you receive the notification that the operation is complete. Then, you can begin modifying your volume.If you can't wait until the operation is complete, then you can create a new volume with your desired size and performance. After you create the new volume, restore the data from the snapshot of your existing volume. For more information, see Replace a volume using a previous snapshot. If you don't have a snapshot of your volume, then see Create Amazon EBS snapshots.Follow"


Question: How can I troubleshoot failed remediation actions in AWS Config?

"I followed the instructions for Remediating Noncompliant AWS Resources by AWS Config Rules. However, the remediation action failed, and the AWS Config console displays the Action status error "Action execution failed (details)". I opened the Details page, but it doesn't contain enough information to troubleshoot the issue."

Answer: "I followed the instructions for Remediating Noncompliant AWS Resources by AWS Config Rules. However, the remediation action failed, and the AWS Config console displays the Action status error "Action execution failed (details)". I opened the Details page, but it doesn't contain enough information to troubleshoot the issue.ResolutionFollow these instructions to troubleshoot the remediation actions failure using the AWS Command Line Interface (AWS CLI) or AWS CloudTrail event history.Note: If you receive errors when running the AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.AWS CLI1.    Run the AWS CLI command for describe-remediation-execution-status for a more detailed error message, state, and timestamps for remediation action steps similar to the following:aws configservice describe-remediation-execution-status \     --config-rule-name example-rule \     --region example-region \     --resource-keys resourceType=example-resource-type,resourceId=example-resource-IDNote: Replace example-rule, example-region, example-resource-type, and example-resource-ID with your AWS Config rule name, Region, resource type, and resource ID.2.    You receive an output similar to the following:{    "RemediationExecutionStatuses": [        {            "ResourceKey": {                "resourceType": "AWS::EC2::Volume",                "resourceId": "vol-0b399a24561582586"            },            "State": "FAILED",            "StepDetails": [                {                    "Name": "createDocumentStack",                    "State": "FAILED",                    "ErrorMessage": "Automation Step Execution fails when it is creating a CloudFormation stack. Get Exception from CreateStack API of cloudformation Service. Exception Message from CreateStack API:[User: arn:aws:sts::xxxxx:assumed-role/config-remediation-sshpublic-role-zkga0ot3/config-remediation-sshpublic is not authorized to perform: cloudformation:CreateStack on resource: arn:aws:cloudformation:eu-west-2:xxxxx:stack/DetachEBSVolumeStack2f6d3590-ea2c-424a-97ea-045749f07164/* (Service: AmazonCloudFormation; Status Code: 403; Error Code: AccessDenied; Request ID: b8f41dd6-9020-11e9-897d-f9719db1a9e6)]. Please refer to Automation Service Troubleshooting Guide for more diagnosis details.",                    "StartTime": 1560680582.675,                    "StopTime": 1560680582.884                },                {                    "Name": "detachVolume",                    "State": "PENDING"                },                {                    "Name": "deleteCloudFormationTemplate",                    "State": "PENDING"                }            ],            "InvocationTime": 1560680582.419,            "LastUpdatedTime": 1560680583.67        }    ]}3.    In the StepDetails list, note the error message and cause of the failure.CloudTrail event history1.    Open the AWS CloudTrail console.2.    Follow the instructions for Viewing CloudTrail events in the CloudTrail console.3.    The StartAutomationExecution API action is invoked when AWS Config initiates a remediation action. Filter Event name with the StartAutomationExecution API, and in the event details page, copy the requestID.4.    Open the AWS Systems Manager console, and then choose Automation in the navigation pane.5.    Paste the requestID in the Automation document search field.6.    Then, find the action that the remediation performed. Verify that the error relates to AWS Identity and Access Management (IAM) permissions, syntax issues, or incorrect parameters configured in the remediation action.Related informationGranting custom permissions for AWS Config UsersHow can I troubleshoot AWS Config console error messages?View, update, and delete rules (AWS CLI)How can I resolve the error "NoSuchRemediationConfigurationException" or "unexpected internal error" when trying to delete a remediation action in AWS Config?Follow"


Question: How can I use the CloudWatch agent to view metrics for Performance Monitor on a Windows server?

I want to use the Amazon CloudWatch agent on a Windows server to get the metrics that are collected for Performance Monitor. How can I do this?

Answer: "I want to use the Amazon CloudWatch agent on a Windows server to get the metrics that are collected for Performance Monitor. How can I do this?Short descriptionTo use the CloudWatch agent to get metrics for Performance Monitor on a Windows server, you need:An Amazon Elastic Compute Cloud (Amazon EC2) instance on a public subnet (with access to internet). Or, an EC2 instance on a private subnet (with access to VPC endpoints).An AWS Identity and Access Management (IAM) profile and role for the instanceSSM agentThe CloudWatch agentFollow the steps in this article to install and configure the CloudWatch agent on your Windows server, and then view the metrics for Performance Monitor.ResolutionStart your instance1.    Start your EC2 Instance with Windows Server 2019.2.    After your EC2 instance has started, use a remote desktop protocol (RDP) to log in to your instance.Install the CloudWatch agentInstall the CloudWatch agent using one of these two methods:Use the SSM document AWS-ConfigureAWSPackageDownload the CloudWatch package manuallyUse the SSM document AWS-ConfigureAWSPackageTo use the AWS-ConfigureAWS package to install the agent, see Installing the CloudWatch agent on EC2 instances using your agent configuration.Download CloudWatch agent manuallyRun this command to download the installer package:Invoke-WebRequest -Uri https://s3.amazonaws.com/amazoncloudwatch-agent/windows/amd64/latest/amazon-cloudwatch-agent.msi -OutFile $env:USERPROFILE\Downloads\amazon-cloudwatch-agent.msi -UseBasicParsingInvoke-Item $env:USERPROFILE\Downloads\amazon-cloudwatch-agent.msiAfter you download the .msi file, create the JSON configuration file using one of these methods:Use the agent configuration file wizardCreate the configuration file manuallyCreate the CloudWatch agent configuration file with the wizard1.    Follow the steps in Create the CloudWatch agent configuration file with the wizard. This is an example of the file that the wizard generates:{  "logs": {    "logs_collected": {      "files": {        "collect_list": [          {            "file_path": "C:\\Users\\Administrator\\Desktop\\CWMetricsLogs",            "log_group_name": "CWMetricsLogs",            "log_stream_name": "{instance_id}",            "retention_in_days": -1          }        ]      },      "windows_events": {        "collect_list": [          {            "event_format": "xml",            "event_levels": [              "VERBOSE",              "INFORMATION",              "WARNING",              "ERROR",              "CRITICAL"            ],            "event_name": "CloudWatchAgent",            "log_group_name": "CloudWatchAgent",            "log_stream_name": "{instance_id}",            "retention_in_days": -1          }        ]      }    }  },  "metrics": {    "aggregation_dimensions": [      [        "InstanceId"      ]    ],    "append_dimensions": {      "AutoScalingGroupName": "${aws:AutoScalingGroupName}",      "ImageId": "${aws:ImageId}",      "InstanceId": "${aws:InstanceId}",      "InstanceType": "${aws:InstanceType}"    },    "metrics_collected": {      "LogicalDisk": {        "measurement": [          "% Free Space"        ],        "metrics_collection_interval": 30,        "resources": [          "*"        ]      },      "Memory": {        "measurement": [          "% Committed Bytes In Use"        ],        "metrics_collection_interval": 30      },      "Paging File": {        "measurement": [          "% Usage"        ],        "metrics_collection_interval": 30,        "resources": [          "*"        ]      },      "PhysicalDisk": {        "measurement": [          "% Disk Time",          "Disk Write Bytes/sec",          "Disk Read Bytes/sec",          "Disk Writes/sec",          "Disk Reads/sec"        ],        "metrics_collection_interval": 30,        "resources": [          "*"        ]      },      "Processor": {        "measurement": [          "% User Time",          "% Idle Time",          "% Interrupt Time"        ],        "metrics_collection_interval": 30,        "resources": [          "*"        ]      },      "TCPv4": {        "measurement": [          "Connections Established"        ],        "metrics_collection_interval": 30      },      "TCPv6": {        "measurement": [          "Connections Established"        ],        "metrics_collection_interval": 30      },      "statsd": {        "metrics_aggregation_interval": 60,        "metrics_collection_interval": 30,        "service_address": ":8125"      }    }  }}2.    You can also configure your CloudWatch agent config.json file for other Windows Performance counters. This example shows a generic syntax that you can use to add performance counters that are available in Performance Monitor. Be sure that the counters are visible in Windows Performance Monitor or the agent can't push any datapoint for that counter."PerformanceCounter-Family(processor, logical disk, memory etc.)": {  "measurement": [      "Counter-name ("%Usage, "% free space" etc.)"    ],  "metrics_collection_interval": 30,  "resources": [    "Counter Property ( "_total", "C:/" etc.)  ]},3.    After the wizard generates the file, you can find it in C:\Program Files\Amazon\AmazonCloudWatchAgent. Rename this file, and then copy it to this location: C:\ProgramData\Amazon\AmazonCloudWatchAgent\Configs.4.    Start the CloudWatch agent:Once the file has been put in the CloudWatchAgent Configs, Start the cloudWatch Agent Service with.\amazon-cloudwatch-agent-ctl.ps1 -a fetch-config -m EC2 -c file:$env:USERPROFILE\config.json -sCreate the configuration file manuallyFor more information on creating the JSON file manually, see Manually create or edit the CloudWatch agent configuration file.View the schema definition for the configuration file for Windows servers:installation-directory/amazon-cloudwatch-agent-schema.jsonAfter you create the file, copy it to the other servers where you want to install the agent in.View the Performance Counter metrics in CloudWatchAfter you configure and start the CloudWatch agent, view the Windows Performance Counters metrics in the CloudWatch console.1.    Open the CloudWatch console.2.    From the navigation pane, under Metrics, choose All Metrics.3.    From the Custom namespaces section, choose CWAgent.Sort by ImageID, InstanceID, InstanceType, instance, objectname, or by InstanceID. Your metrics are then uploaded and you can view them on the console.Related informationCollecting metrics and logs from Amazon EC2 instances and on-premises servers with the CloudWatch agentFollow"


Question: "What are Elastic IP addresses, and how do I use them?"

I want to have a public IP address for my EC2 instance that will persist even if my instance is stopped and restarted. How do I do that?

Answer: "I want to have a public IP address for my EC2 instance that will persist even if my instance is stopped and restarted. How do I do that?ResolutionAn Elastic IP address is a reserved public IP address that you can assign to any EC2 instance in a particular region, until you choose to release it. To allocate an Elastic IP address to your account in a particular Region, see Allocate an Elastic IP address.When you associate an Elastic IP address with an EC2 instance, it replaces the default public IP address. If an external hostname is allocated to the instance from your launch settings, it also replaces this hostname. Otherwise, it creates one for the instance. The Elastic IP address remains in place through events that normally cause the address to change, such as stopping or restarting the instance.Note: This description applies to most AWS accounts, which are EC2-VPC by default.Related informationAWS service quotasFollow"


Question: Why am I getting a "WAFInvalidParameterException" error in AWS WAF?

Why am I getting the WAFInvalidParameterException error?

Answer:
"Why am I getting the WAFInvalidParameterException error?Short descriptionIf the operation you are performing has parameters that AWS WAF can't recognize, the operation fails and one of the errors that AWS WAF returns is WAFInvalidParameterException. The most common reasons for this error are:A parameter name or value specified isn't valid.The ARN referenced in the request is from a resource that can't be associated to the web ACL.ResolutionIf the value of a parameter isn't recognized by AWS WAF, you see the following example error message:An error occurred (WAFInvalidParameterException) when calling the CreateRuleGroup operation: Error reason:
The scope is not valid., field: SCOPE_VALUE, parameter: <parameter value>For example, a Scope parameter value that is other than REGIONAL or CLOUDFRONT returns this error.To troubleshoot the error, do the following:identify the parameter that has a value that can't be recognized by AWS WAF.Modify the parameter to a value supported by AWS WAF. For supported values, see AWS WAFV2 API Reference.AWS WAF can be associated with an Amazon CloudFront distribution, Amazon API Gateway REST API, Application Load Balancer, or AWS AppSync GraphQL API. If you try to associate with a different resource type, you see the following example error message:An error occurred (WAFInvalidParameterException) when calling the AssociateWebACL operation: Error reason:
The ARN isn't valid. A valid ARN begins with arn: and includes other information separated by colons or slashes., field: RESOURCE_ARN, parameter: <resource arn>To troubleshoot the error, do the following:Identify the parameter that has a value that can't be recognized by AWS WAF.Modify this parameter to a resource supported by AWS WAF.Follow"


Question: How can I troubleshoot high target latency on an AWS DMS task?

"I'm running a full load and change data capture (CDC) AWS Database Migration Service (AWS DMS) task. The source latency is not high, but the target latency is high or it is increasing. How can I troubleshoot high target latency on an AWS DMS migration task?"

Answer: "I'm running a full load and change data capture (CDC) AWS Database Migration Service (AWS DMS) task. The source latency is not high, but the target latency is high or it is increasing. How can I troubleshoot high target latency on an AWS DMS migration task?Short descriptionYou can use Amazon CloudWatch metrics to monitor your replication task's statistics. Specifically, you can monitor CDCLatencySource and CDCLatencyTarget to identify replication latency in the ongoing replication phase (CDC). The CDCLatencySource metric is the latency between the source and the replication instance. The CDCLatencyTarget metric is the latency between replication instance and target. For more information, see Replication task metrics.High CDCLatencySource means that the process of capturing changes from the source is delayed. And high CDCLatencyTarget means that the process of applying the change events to the target is delayed. If both CDCLatencySource and CDCLatencyTarget are high, investigate CDCLatencySource  first, because target latency is always the same or greater than the source latency. High CDCLatencyTarget is most likely the result of the delay in capturing the change events from the source. If the CDCLatencySource isn't high, but the CDCLatencyTarget is high, the latency could be caused by the following:There are no primary keys or indexes in the targetThere are resource bottlenecks in the targetThere are resource bottlenecks in the replication instanceThere is a network issue between the replication instance and the targetTo resolve these issues, see Best practices and troubleshooting.ResolutionNo primary keys or indexes in the targetBy default, AWS DMS writes changes to the target by data manipulation language (DML) statements—such as INSERT, UPDATE, or DELETE—like any other application. If the required indexes aren't in place, then changes like UPDATEs and DELETEs can result in full table scans. Full table scans can cause performance issues on the target. These scans then result in target latency. Check your target database schemas, especially if you created the target schema manually. Identify slow queries using target database mechanisms, like the slow query log for MySQL, pg_stat_activity for PostgreSQL, or a query plan. If your target is Amazon Redshift, also check the distribution style for your table. All distribution styles can cause target latency because they take longer to INSERT or UPDATE data into tables.Resource bottlenecks in the targetIf your target doesn't have sufficient resources, then the target can’t accept changes at the rate that AWS DMS sends them. This can cause resource bottlenecks on the target and target latency. This can also happen if other processes are consuming resources in the target. If the target is hosted on AWS, then check the resource statistics from the CloudWatch metrics.Resource bottlenecks in the replication instanceChoose a replication instance that has enough resources to handle your migration—CPU, memory, network, or iOPS. You can monitor your replication instance resources using CloudWatch metrics.Network issue between replication instance and targetNetwork bandwidth and latency issues can also cause latency issues, especially when your target is an on-premises database, or if you use AWS DMS for cross-Region replication.Best practices and troubleshootingIf your target is Amazon Relational Database Service (Amazon RDS) follow the best practices for Improving the performance of an AWS DMS migration. Amazon RDS has an automated backup mechanism that starts within the backup window, and Amazon RDS backs up the moved data. If a snapshot of the target DB instance was in the process of capture, AWS DMS can have issues applying changes to the target. As a result, the target latency increases until the snapshot capture is completed. If your target is Amazon Elastic Compute Cloud (Amazon EC2) or an on-premises database, check your target database's backup mechanism.Some task settings can cause changes to be written slowly to the target. If you run ongoing replication from a source where the rate of change is high, consider using BatchApplyEnabled. For more information, see the BatchApplyEnabled section of Debugging your AWS DMS migrations: What to do when things go wrong?To set BatchApplyEnabled to True, run the modify-replication-task command using the AWS Command Line Interface (AWS CLI):aws dms modify-replication-task --replication-task-arn arn:aws:dms:ap-northeast-1:123456789012:task:ABCDEFGHIJKLMNOPQRSTUVWXYZ --replication-task-settings "{\"TargetMetadata\":{\"BatchApplyEnabled\":true}}"Related informationChange processing tuning settingsFollow"


Question: Why was I charged for AWS Business Support when I have an AWS Activate Portfolio package?

"I was charged for my AWS Support plan, even though I signed up for an AWS Activate Portfolio package that includes a credit for AWS Business Support. How can I resolve this issue?"

Answer: "I was charged for my AWS Support plan, even though I signed up for an AWS Activate Portfolio package that includes a credit for AWS Business Support. How can I resolve this issue?ResolutionIf you had an AWS Support subscription other than a Business-level Support plan before you were approved for an AWS Activate Portfolio package, see AWS Premium Support FAQs, and follow the instructions in Q: How do I cancel my AWS Support subscription? to cancel your support subscription. Then, follow the instructions in the welcome email you received from the AWS Activate team.If you can't locate the email from the AWS Activate team, or if you have questions about the AWS Activate program, then contact the AWS Activate team at AWS Activate Contact Us.Related informationAWS ActivateAWS SupportFollow"


Question: Why is MFA failing on my AWS Managed Microsoft AD directory or my AD Connector?

"I've enabled multi-factor authentication (MFA) on my AWS Directory Service for Microsoft Active Directory (AWS Managed Microsoft AD) directory or AD Connector. However, MFA is failing. How can I troubleshoot this?"

Answer: "I've enabled multi-factor authentication (MFA) on my AWS Directory Service for Microsoft Active Directory (AWS Managed Microsoft AD) directory or AD Connector. However, MFA is failing. How can I troubleshoot this?ResolutionThe security group associated with your AWS Managed Microsoft AD or AD Connector must have a rule that allows outbound traffic on port UDP 1812 to the security group associated with your RADIUS server.Note: If you're using a custom UDP port for MFA authentication, then allow the custom UDP port traffic under the following:Outbound rules on the security group associated with your AWS Managed Microsoft AD or AD Connector.Inbound rules on the Security group associated with your RADIUS server.Verify that port UDP 1812 or your custom UDP port for MFA is allowed under outbound traffic on the AWS Managed Microsoft AD or AD Connector security groupTo find the security group associated with your DNS servers, open the AWS Directory Service console, and note the IP addresses under DNS address.Open the Amazon Elastic Compute Cloud (Amazon EC2) console, and then choose Network Interfaces.In the search field, enter one of the DNS IP addresses found in step 1 and select the checkbox for that interface.Under Details, select the security group listed in Security Groups.Select View outbound rules. Verify that there is a rule allowing outbound traffic on port UPD 1812 for UDP, or your custom UDP port for MFA, to the IP address space or security group associated with your RADIUS EC2 instances.Verify that the secret key for directory services is the same key configured on the RADIUS serverThe RADIUS client and server must use the same shared password or key. Check the RADIUS server logs for further information. The method for checking Radius logs depends on your configuration. Review the documentation for your configuration for instructions on accessing the logs.Related informationEnable multi-factor authentication for AWS Managed Microsoft ADEnable multi-factor authentication for AD ConnectorFollow"


Question: What's the IP address range for Device Farm?

"I want to test my company's app using AWS Device Farm, but our internal systems are behind a corporate firewall. I need the IP address range of Device Farm to allow device traffic through."

Answer: "I want to test my company's app using AWS Device Farm, but our internal systems are behind a corporate firewall. I need the IP address range of Device Farm to allow device traffic through.ResolutionFor mobile devicesIn your internal systems, allow requests from the IP address ranges:54.244.50.32/2799.78.197.0/2915.248.40.40/2954.239.50.200/29Note: This IP CIDR range allows traffic coming from Device Farm devices during mobile automation testing only.For desktop browsersIt's a best practice to use the Amazon Virtual Private Cloud (Amazon VPC) for desktop browser testing. Amazon VPC's Test Grid feature is more secure than an IP allow list, and your VPC will have direct access to your Device Farm resources.Note: If your VPC is located in an AWS Region other than us-west-2, then you must create a VPC in the us-west-2 Region and activate VPC peering.To use an allow list for internet-facing browser testing, make sure to allow requests from the following IP range:100.20.244.10344.225.225.3244.227.49.13335.82.102.6Note: These IPs are stable but not permanent. If you are unable to access Device Farm with these IPs, then they might have changed since the most recent update of documentation.Related informationAWS Device Farm FAQsAWS IP address rangesWhat is AWS Device Farm?Follow"


Question: How do I resolve the ErrorPortAllocation error on my NAT gateway?

"I keep getting the ErrorPortAllocation error on my NAT gateway, and several concurrent connections to my destination host are failing. How do I resolve this port allocation error?"

Answer: "I keep getting the ErrorPortAllocation error on my NAT gateway, and several concurrent connections to my destination host are failing. How do I resolve this port allocation error?Short descriptionNAT gateways support up to 55,000 simultaneous connections to each destination. If this threshold is crossed, then new connections to the destination fail and the ErrorPortAllocation metric for the NAT gateway increases in Amazon CloudWatch.To resolve this issue, find the source clients and destinations that are causing these errors. Then, follow the steps in the Resolve port allocations errors Resolution section.Prerequisites:Make sure that the VPC flow log is activated at the VPC level or to the subnets where the NAT gateway is used.Make sure that the VPC flow log is configured to deliver to CloudWatch.ResolutionFind source clients and their connection destinations1.    Open the CloudWatch console.2.    In the navigation pane, choose Insights.3.    For Log group, choose the log group where the flow logs are recorded.4.    Find where traffic is going:filter (srcAddr like 'NAT gateway Private IP' and dstAddr not like 'xxx.xxx.')| stats count(*) as numaccept by dstAddr| sort numaccept desc| limit 10The results show the destination IP with the greatest number of responses during the time frame when you received the port allocation errors.Note: Replace xxx.xxx. with the first two octets of your VPC CIDR. Also, replace NAT gateway Private IP with the private IP of your NAT gateway.5.    Find the source clients that are sending traffic to the destination:filter (dstAddr like 'Public IP from above filter' and srcAddr like 'xxx.xxx.') | stats sum(bytes) as bytesTransferred by srcAddr, dstAddr| sort bytesTransferred desc| limit 10Note: Replace xxx.xxx. with the first two octets of your VPC CIDR. Also, replace Public IP from above filter with the destination IP that you received from the preceding query.Resolve port allocation errorsCreate a NAT gateway in each Availability Zone, and then distribute your clients across Availability Zones. Route traffic to the internet using a NAT gateway in the same Availability Zone as your client to reduce cross Availability Zone data charges.If you notice an increase in the IdleTimeoutCount metric in CloudWatch, then configure your application or private instance to close idle connections. This lets the NAT gateway allocate the source port to new connections.Limit the number of connections that your clients can make to a single destination.For traffic going to an Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB public IP address in the same AWS Region, use a gateway VPC endpoint. There are no data processing or hourly charges for using gateway VPC endpoints.Related informationSample queriesMonitoring NAT gateways using Amazon CloudWatchFollow"


Question: How do I troubleshoot problems establishing communication over VPC peering?

I can't establish communication between my Amazon Virtual Private Cloud (Amazon VPC) over VPC peering.

Answer: "I can't establish communication between my Amazon Virtual Private Cloud (Amazon VPC) over VPC peering.Short descriptionVPC peering network connectivity failures might occur due to incorrect or missing route tables, firewall rules, or DNS (if applicable). To route traffic between VPCs in a peering connection with private IP addresses, the VPCs must be able to communicate with each other. You must allow the VPCs to communicate as if they are in the same network.ResolutionVerify that the VPC peering connection is in the Active state.Update your route tables for your VPC peering connection. Verify that the correct routes exist for connections to the IP address range of your peered VPCs through the appropriate peering ID.Verify that an ALLOW rule exists in the network access control (network ACL) table for the required traffic.Verify that the security group rules allow network traffic between the peered VPCs.To use a domain name to reach a destination in the peer VPC, see Why can't I resolve domain names over my VPC peering connection?To verify that a source or destination doesn't reject the required traffic, use VPC Flow Logs. This rejection might occur due to the missing rules under security groups or network ACLs of the source or destination.Related informationVPC peering basicsFollow"


Question: How can I troubleshoot Lambda function connectivity issues to an Amazon VPC using Reachability Analyzer?

I want to troubleshoot AWS Lambda function connection issues to an Amazon Virtual Private Cloud (Amazon VPC) using VPC Reachability Analyzer.

Answer: "I want to troubleshoot AWS Lambda function connection issues to an Amazon Virtual Private Cloud (Amazon VPC) using VPC Reachability Analyzer.Short descriptionYou can configure a Lambda function to access resources inside an Amazon VPC. The Lambda function is assigned to a network interface that's used to connect the Amazon VPC to an elastic network interface. To troubleshoot network connectivity issues, you can use the configuration analysis tool Reachability Analyzer.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Get the network interface IDs for Lambda functions using the AWS Management Console1.    Open the Functions page of the Lambda console, and then choose your function.2.    Choose the Configuration tab.3.    In General configuration, choose VPC. Copy or note the VPC ID, subnet IDs, and security group IDs.4.    Open the Amazon Elastic Compute Cloud (Amazon EC2) console.5.    In the navigation pane, from Network & Security, choose Network Interfaces.6.    In the search bar, enter the following API filters: VPC ID = <your VPC ID>Subnet ID = <your subnet ID>Security group IDs = <your security group ID>Interface Type = lambda7.    Copy the network interface ID from the Details section.Get the network interface IDs for Lambda functions using the AWS CLI1.    Run the AWS CLI command get-function-configuration to retrieve the Lambda function's Amazon VPC configuration similar to the following:aws lambda get-function-configuration --function-name <function_name> --query 'VpcConfig'2.    Copy the VpcId, SecurityGroupIds, and SubnetIds from the output.3.    Run the AWS CLI command describe-network-interfaces to retrieve the network interface ID similar to the following:Note: Replace the values in this example with your IDs from step 2.aws ec2 describe-network-interfaces --filters 'Name=vpc-id,Values=vpc-12345678' 'Name=subnet-id,Values=subnet-12345678' 'Name=interface-type,Values=lambda' 'Name=group-id,Values=sg-12345678' --query 'NetworkInterfaces[*].NetworkInterfaceId'Copy the network interface ID in the output for "NetworkInterfaceId".Create and analyze a path with Reachability Analyzer1.    Open the AWS Network Manager console.2.    In the navigation pane, choose Reachability Analyzer.3.    Choose Create and analyze path.4.    Choose the Source type dropdown list, and then choose Network Interfaces.5.    Choose the Source dropdown list, and then enter the network interface ID.6.    Choose the Destination type dropdown list, and then choose your destination type.7.    For Protocol, choose TCP or UDP, depending on your use case.8.    Choose Create and analyze path.View the results of the path analysisAfter the path analysis completes, you can view the results of the path analysis.If the reachability status matches your intent, then there's no further action required. If the reachability status doesn't match your intent, then you can change your network configuration and analyze the path.For more information, see Getting started with Reachability Analyzer.Related informationHow do I use the VPC Reachability Analyzer to troubleshoot connectivity issues with an Amazon VPC resource?New – VPC Reachability AnalyzerFollow"


Question: How do I allow only specific IP addresses to access my API Gateway REST API?

I want to allow only specific IP addresses access to my Amazon API Gateway REST API. How do I restrict access to my API Gateway REST API?

Answer: "I want to allow only specific IP addresses access to my Amazon API Gateway REST API. How do I restrict access to my API Gateway REST API?Short descriptionCreate a resource policy for your REST API that denies access to any IP address that isn't specifically allowed.After the resource policy is attached to your REST API, users who call the API from specified IP addresses (allowed users) can access the API. Calls from any other IP addresses are denied access and receive an HTTP 403 Forbidden error.To block specific IP addresses and allow access for all others, see Example: Deny API traffic based on source IP address or range.Note: API Gateway HTTP APIs currently don't support resource policies.ResolutionNote: For the following procedure, you can use either your existing API Gateway REST API or create an example REST API for testing. If you use the example API (PetStore), skip to the second section: Create and attach a resource policy that allows only specific IP address access to your API Gateway REST API.Set up an API method1.    Open the API Gateway console.2.    Choose your REST API.3.    In the Resources pane, choose Actions. Then, choose Create Method.4.    In the dropdown list under the / resource node, choose ANY. Then, choose the check mark icon.5.    On the / - ANY - Setup pane, for Integration type, choose Mock.6.    Choose Save.Note: A mock integration responds to any request that reaches it, which helps later with testing.Create and attach a resource policy that allows only specific IP addresses access to your API Gateway REST API1.    Open the API Gateway console.2.    In the left navigation pane, choose Resource Policy.3.    In the Resource Policy text box, paste the following example resource policy:Example resource policy{  "Version": "2012-10-17",  "Statement": [{      "Effect": "Allow",      "Principal": "*",      "Action": "execute-api:Invoke",      "Resource": "execute-api:/*/*/*"    },    {      "Effect": "Deny",      "Principal": "*",      "Action": "execute-api:Invoke",      "Resource": "execute-api:/*/*/*",      "Condition": {        "NotIpAddress": {          "aws:SourceIp": ["sourceIpOrCIDRBlock", "sourceIpOrCIDRBlock"]        }      }    }  ]}3.    For aws:SourceIp, replace the sourceIpOrCIDRBlock value with the specific IP addresses that you want to grant access to. You can also specify a range of IP addresses using CIDR notation.Example public IP address string that's listed in CIDR notation["52.95.36.0/22", "15.230.39.196/31", "52.93.178.219/32"]Important: The aws:SourceIp condition value works only for public IP address ranges. To allow access to private IP address ranges, use the condition value aws:VpcSourceIp instead. Then, for the aws:VpcSourceIp value, enter the private IP address of your HTTP client that's invoking your private API endpoint through the interface VPC endpoint. Note that you can invoke private API endpoints in API Gateway only through an interface VPC endpoint. For more information, see aws:VpcSourceIp.4.    Choose Save.Note: The $context.identity.sourceIp variable in your Amazon CloudWatch Logs lists any private IP addresses that call your REST API. For more information, see $context variables for data models, authorizers, mapping templates, and CloudWatch access logging.Deploy your API1.    In the Resources pane of your REST API, choose Actions.2.    Choose Deploy API.3.    For Deployment stage, choose [New Stage].4.    For Stage name, enter a name. For example: v1 or demo.5.    Choose Deploy.Important: You must redeploy your REST API each time you make changes its resource policy.Test the resource policyConfirm that your REST API returns an HTTP 200 OK response to allowed IP addresses by testing your API's invoke URL using curl or the Postman app. Then, run another test from an environment without an allowed IP address to confirm that your REST API returns an HTTP 403 Forbidden error.Note: If you don't have access to multiple environments for testing, then you can set up an Amazon Elastic Compute Cloud (Amazon EC2) instance. If you use an Amazon EC2 instance for testing, make sure that you add the instance's IP address to your API's resource policy first. Then, redeploy your API.To test your REST API for an HTTP 200 OK response or HTTP 403 error using curlBased on your operating system, run one of the following commands.Important: Replace https://yourInvokeUrl/ with your REST API's invoke URL.For Linux, Unix, and macOS environments, run the following command:curl -IX GET https://yourInvokeUrl/For Windows PowerShell, run the following command:curl https://yourInvokeUrl/If the resource policy is configured correctly, the allowed environment receives an HTTP 200 OK response. The denied environment receives an HTTP 403 Forbidden error.Related informationCreate and attach an API Gateway resource policy to an APIAPI Gateway resource policy examplesAccess policy language overview for Amazon API GatewayControlling and managing access to a REST API in API GatewayFollow"


Question: How can I add Oracle Enterprise Manager Database Control or Database Express to an Amazon RDS Oracle DB instance?

How can I add Oracle Enterprise Manager Database Control or Database Express to an Amazon Relational Database Service (Amazon RDS) Oracle DB instance?

Answer: "How can I add Oracle Enterprise Manager Database Control or Database Express to an Amazon Relational Database Service (Amazon RDS) Oracle DB instance?Short DescriptionUse the Oracle Enterprise Manager (OEM) Option Group to configure OEM Database Control or Database Express based on the following versions of Amazon RDS Oracle:Oracle Enterprise Manager Database Express for Oracle 19c, Oracle 18c, and Oracle 12cOracle Enterprise Manager 11g Database Control for Oracle 11gFor more information, see Oracle Enterprise Manager Database Express.ResolutionOpen the Amazon RDS console in the same AWS Region as your DB instance.Choose Databases from the navigation pane, and then choose your DB instance.Choose Option groups from the navigation pane. Then, create an option group, if you don't already have an option group that is associated to your DB instance.Confirm that the option group uses the Oracle EE or Standard engine, and then choose one of the supported major DB engine versions, such as 12.1.After you create the option group, select the new option group, and then choose Add option.From the Add option page, choose OEM from the Option name dropdown list.Choose the DB engine version, such as 12.1.0.2, from the Version dropdown list.Confirm that your Amazon Virtual Private Cloud (Amazon VPC) is configured to allow inbound access to the OEM port.From the Security Groups list, select the VPC or DB security groups that you want to enable.Note: You might see that the DB security group is different from the VPC or Amazon Elastic Compute Cloud (Amazon EC2) security group. This is because a DB instance on an EC2 Classic instance might use a DB security group, but DB instances on a VPC use the VPC/EC2 security group.Select Apply Immediately, and choose Add Option.Note: If you add the OEM option for an Oracle 11g DB instance, then there is no outage, and you don't need to restart your DB instance. If you add the OEM option for an Oracle 19c, Oracle 18c, or Oracle 12c DB instance, then a brief outage occurs while your DB instance is automatically restarted.Associate this option group with the DB instance by modifying your DB instance.After you enable the OEM option, you can use either Database Control or Database Express from your web browser. The URL of the OEM option includes the Amazon RDS endpoint and OEM port. For example, https://mydb.f9rbfa893tft.us-east-1.rds.amazonaws.com:5500/em for Oracle 19c, Oracle 18c, or Oracle 12c DB instances or https://mydb.f9rbfa893tft.us-east-1.rds.amazonaws.com:1158/em for Oracle 11g.After the page loads, enter the master user name and master password for your DB instance.You are now ready to manage your Oracle databases.Related InformationOracle on Amazon RDSFollow"


Question: How do I identify and troubleshoot performance issues and slow-running queries in my RDS for PostgreSQL or Aurora PostgreSQL instance?

My Amazon Relational Database Service (Amazon RDS) for PostgreSQL instance is slow. I want to identify and troubleshoot the slow-running queries.

Answer: "My Amazon Relational Database Service (Amazon RDS) for PostgreSQL instance is slow. I want to identify and troubleshoot the slow-running queries.ResolutionThe performance of your Amazon RDS for PostgreSQL instance might be affected for multiple reasons, such as:Undersized hardwareChanges in workloadIncreased trafficMemory issuesSuboptimal query plansIdentify the causeUse a combination of these tools to identify the cause of slow-running queries:Amazon CloudWatch metricsEnhanced Monitoring metricsPerformance Insights metricsDatabase statisticsNative database toolsCloudWatch metricsTo identify performance bottlenecks caused by insufficient resources, monitor these common CloudWatch metrics available for your Amazon RDS DB instance.CPUUtilization - Percentage of computer processing capacity usedFreeableMemory - Available RAM on the DB instance (in megabytes)SwapUsage - Swap space used by the DB instance (in megabytes)A higher percentage of CPU utilization generally indicates an active workload on the instance and the need for more CPU resources. Higher memory utilization along with swap space consumption indicates frequent swapping due to low memory availability for the workload. This might mean that your instance can't keep up with the workload. A high utilization of CPU and memory resources are typically caused by long-running queries, suddenly increased traffic, or a large number of idle connections.Run this command to view the active queries with runtime:SELECT pid, usename, age(now(),xact_start) query_time, query FROM pg_stat_activity WHERE state='active';Run this command to find the number of idle connections present in the database:SELECT count(*) FROM pg_stat_activity WHERE state='idle';Then, run this command to terminate idle connections that consume resources:Note: Be sure to replace example-pid with the PID of the idle connection that you got from pg_stat_activity:SELECT pg_terminate_backend(example-pid);Verify that the desired network throughput is achieved on the RDS Instance by reviewing these metrics:NetworkReceiveThroughputNetworkTransmitThroughputThese metrics indicate the incoming and outgoing network traffic in bytes/second. Undersized or non-Amazon Elastic Block Service (Amazon EBS)-optimized instance classes might affect network throughput, resulting in slow instances. A low network throughput can result in slow responses for all the application requests irrespective of the database performance.Evaluate the I/O performance by reviewing these metrics:ReadIOPS and WriteIOPS - Average number of disk read or write operations per secondReadLatency and WriteLatency - Average time taken for a read or write operation in millisecondsReadThroughput and WriteThroughput - Average number of megabytes read from or written to disk per secondDiskQueueDepth - Number of I/O operations that are waiting to be written to or read from diskFor more information, see How do I troubleshoot the latency of Amazon EBS volumes caused by an IOPS bottleneck in my Amazon RDS instance?Enhanced Monitoring metricsWith Enhanced Monitoring, you can view metrics at the operating system level and the list of top 100 processes that consume high CPU and memory. You can activate Enhanced Monitoring at per-second level to identify intermittent performance issues on your RDS Instance.You can evaluate the available operating system metrics to diagnose performance issues that might be related to CPU, workload, I/O, memory, and network.From the process list, identify the process with high values for CPU% or Mem%. Then, find the related connection from the database.For example:NAMEVIRTRESCPU%MEM%VMLIMITpostgres: postgres postgres 178.2.0.44(42303) SELECT [10322]250.66 MB27.7 MB85.932.21unlimitedConnect to the database, and then run this query to find the connection and query-related information:SELECT * FROM pg_stat_activity WHERE pid = 10322;Performance Insights metricsPerformance Insights allows you to evaluate database workloads sliced by waits, SQL, host, or users. You can also capture the database and SQL level metrics.The Top SQL tab on the Performance Insights dashboard shows the SQL statements that contribute the most to DB load. A DB load or load by waits (AAS) that's higher than the Max vCPU value indicates throttled workload on the instance class.The average latency per call in SQL statistics provides the average running time of a query. It’s common to see a different SQL to be the top DB load contributor than the one having the highest average running time. This is because the top SQL list is based on total running time.Database statisticsThe following statistics can help you evaluate database performance in PostgreSQL:Data distribution statisticsExtended statisticsMonitoring statisticsFor information on how to read and understand these statistics, see Understanding statistics in PostgreSQL.Native database toolsTo identify slow queries, use the native pgBadger tool. For more information, see Optimizing and tuning queries in Amazon RDS for PostgreSQL based on native and external tools.Optimize performanceTune memory settingsPostgreSQL DB server allocates a certain memory area for its entire lifetime for caching data to improve read and write accesses. This memory area is referred to as shared buffers. The amount of memory the database uses for shared memory buffers is controlled by the shared_buffers parameters.Apart from the shared memory area, each backend process consumes memory for performing operations within a DB server. The amount of memory that's used is based on the values set for work_mem and maintenance_work_mem parameters. For more information, see PostgreSQL documentation for Server configuration.If you continuously observe high memory pressure on the DB Instance, consider lowering the values of these parameters. You can lower the values of these parameters in the custom parameter group that's attached to your DB Instance.Aurora PostgreSQL Query Plan ManagementUse Amazon Aurora PostgreSQL-Compatible Edition query plan management to control how and when query execution plans change. For more information, see Best practices for Aurora PostgreSQL-Compatible query plan management.Troubleshoot slow-running queriesYou typically experience slow-running queries when there are infrastructure issues or the overall resource consumption is high. Slow-running queries might also be the result of suboptimal query planning by the query planner. PostgreSQL query planner uses statistics created for the table to create query plans. These plans might be affected due to schema changes and stale statistics. Bloat in the table and indexes might also result in slow-running queries.The autovacuum daemon is responsible for creating autovacuum worker processes that remove the dead tuples from tables whenever the dead tuple threshold is met. The autovacuum daemon is also responsible for running the ANALYZE operation that refreshes the statistics stored for a particular table.Run the following query to find information about:Dead tuplesNumber of autovacuum or vacuum operationsNumber of autoanalyze or analyze runsWhen these operations were last runSELECT schemaname, relname, n_live_tup,n_dead_tup, last_autoanalyze, last_analyze, last_autovacuum, last_vacuum,autovacuum_count+vacuum_count vacuum_count, analyze_count+autoanalyze_count analyze_count FROM pg_stat_user_tablesORDER BY 5 DESC;You can use the pg_stat_activity view to find data related to current activities. This view provides the backend pid, query, and other details. To find long-running queries, run this query:SELECT pid, datname, query, extract(epoch from now()) - extract(epoch from xact_start) AS duration, caseWHEN wait_event IS NULL THEN 'CPU' ELSE wait_event_type||':'||wait_event end wait FROM pg_stat_activityWHERE query!=current_query() AND xact_start IS NOT NULL ORDER BY 4 DESC;Note that queries waiting for locks might be slow. Therefore, verify whether the query is waiting for locks by running this query:SELECT pid, virtualtransaction AS vxid, locktype AS lock_type, mode AS lock_mode, granted,fastpath,CASEWHEN virtualxid IS NOT NULL AND transactionid IS NOT NULLTHEN virtualxid || ' ' || transactionidWHEN virtualxid::text IS NOT NULLTHEN virtualxidELSE transactionid::textEND AS xid_lock, relname, page, tuple, classid, objid, objsubidFROM pg_locks LEFT OUTER JOIN pg_class ON (pg_locks.relation = pg_class.oid)WHERE -- do not show our view’s lockspid != pg_backend_pid() ANDvirtualtransaction IS DISTINCT FROM virtualxidORDER BY 1, 2, 5 DESC, 6, 3, 4, 7;Your RDS for PostgreSQL instance allows you to create the pg_stat_statements extension from within the database:CREATE EXTENSION IF NOT EXISTS pg_stat_statements;Using pg_stat_statements, you can view the statistics of queries. Be sure to add the pg_stat_statements entry to shared_preload_libraries before creating the extension.Note: You can modify parameters for this module only when a custom parameter group is attached to your DB Instance.Use this queries to identify the top SQL queries that affect the performance of your instance.To find queries that spend more time in the database, run this query for PostgreSQL versions 12 and earlier:SELECT query, calls, total_time, mean_time FROM pg_stat_statements ORDER BY 3 DESC;Run this query for PostgreSQL versions 13 and later:SELECT query, calls, total_plan_time+total_exec_time AS total_time, mean_plan_time + mean_exec_time AS mean_time FROM pg_stat_statements ORDER BY 3 DESC;To find queries with a lower buffer cache hit ratio, run this query for PostgreSQL versions 12 and earlier:SELECT query, calls, total_time, rows, 100.0 * shared_blks_hit / nullif(shared_blks_hit + shared_blks_read, 0) AS hit_percentFROM pg_stat_statements ORDER BY total_timeDESC LIMIT 10;Run this query for PostgreSQL versions 13 and later:SELECT query, calls, total_plan_time+total_exec_time as total_time, rows, 100.0 * shared_blks_hit / nullif(shared_blks_hit +shared_blks_read, 0) AS hit_percentFROM pg_stat_statements ORDER BY 5 ASCLIMIT 10;To capture long-running queries or query plans in database error logs, appropriately set the log_min_duration_statement parameter for your instance, and then use the auto_explain module. The log_min_duration_statement parameter causes the duration of each completed statement to be logged if the statement ran for at least the specified amount of time. For example, if you set this parameter to 250 ms, then all SQL statements that run for 250 ms or longer are logged. The auto_explain module allows you to capture the explain plan of queries that are run in the database.You can also capture the plan using explain and explain analyze commands. Identify query tuning opportunities based on the auto_explain module or explain commands for the query. For more information, see the PostgreSQL Documentation for Using EXPLAIN.If your system is well tuned and you're still facing resource throttling, consider scaling the instance class upwards. Scale the instance class upwards to allocate your DB Instance with more compute and memory resources. For more information, see Hardware specifications for DB instance classes.Related informationHow can I troubleshoot high CPU utilization for Amazon RDS or Amazon Aurora PostgreSQL-Compatible Edition?Working with parameters on your RDS for PostgreSQL DB instanceWhy is my Amazon RDS DB instance using swap memory when I have sufficient memory?Follow"


Question: How do I troubleshoot issues when creating a Kubernetes persistent volume object dynamically using the Amazon EFS CSI controller?

I'm getting errors when I create an Amazon Elastic Kubernetes Service (Amazon EKS) pod that uses the persistent volume claim. The persistent volume is created dynamically and is in Pending state.

Answer: "I'm getting errors when I create an Amazon Elastic Kubernetes Service (Amazon EKS) pod that uses the persistent volume claim. The persistent volume is created dynamically and is in Pending state.ResolutionThe Amazon Elastic File System (Amazon EFS) CSI driver uses the AWS Identity and Access Management (IAM) role for service account (IRSA) feature. This feature requires that various components, including the OpenID Connect (OIDC) provider, IAM role, and access permissions are correctly configured. These components are configured using the IAM role policy and Kubernetes service account. Try the following troubleshooting steps based on the error that you get.Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.storageclass.storage.k8s.io "<STORAGE_CLASS_NAME>" not foundThis error indicates that the storage class referenced by the parameter storageClassName in your PersistentVolumeClaim object definition doesn't exist and must be created.To resolve this error, do the following:1.    Create a Kubernetes storage class object.2.    Download a storage class manifest for Amazon EFS:curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/examples/kubernetes/dynamic_provisioning/specs/storageclass.yaml3.    Edit the downloaded file. Find the following statement, and then replace the value for fileSystemId with your file system ID.fileSystemId: fs-582a03f344f0fc633 # Replace the filesystem id4.    Deploy the storage class:kubectl apply -f storageclass.yamlfailed to provision volume with StorageClass "<STORAGE_CLASS_NAME>": rpc error: code = InvalidArgument desc = File System does not exist: Resource was not foundThis error indicates that the fileSystemId that's referenced by the storage class object either doesn't exist in the Region or is incorrect.To troubleshoot this error, verify whether the Amazon EFS file system referenced in the storage class is correct and exists in the Region:kubectl get storageclass `kubectl get pvc ${PVC_NAME} -o jsonpath='{.spec.storageClassName}'` -o jsonpath='{.parameters.fileSystemId}'Note: Be sure to replace PVC_NAME with the name of your PersistentVolumeClaim.If the EFS file system (fileSystemId) that's returned doesn't exist in the Region, then delete the Kubernetes storage class object. Then, create it again by including the correct file system ID for the fileSystemId field.failed to provision volume with StorageClass "<STORAGE_CLASS_NAME>": rpc error: code = Internal desc = Failed to fetch File System info: Describe File System failed: WebIdentityErr: failed to retrieve credentials caused by: InvalidIdentityToken: No OpenIDConnect provider found in your account for https://oidc.eks.<REGION-CODE>.amazonaws.com/id/<OIDC ID> status code: 400, request id: <REQUEST ID>This error indicates that the IAM OIDC identity provided isn't created in IAM for the Amazon EKS cluster.1.    Retrieve your cluster's OIDC provider ID and store it in a variable:oidc_id=$(aws eks describe-cluster --name ${CLUSTER_NAME} --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5); echo $oidc_idNote: Be sure to replace CLUSTER_NAME with the name of your cluster.2.    Create an IAM OIDC identity provider for your cluster:eksctl utils associate-iam-oidc-provider --cluster ${CLUSTER_NAME} –-approveNote: If you can't use the eksctl utility to create the IAM OIDC provider, use the AWS Management Console.failed to provision volume with StorageClass "<STORAGE_CLASS_NAME>": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access deniedThis error indicates that the IRSA doesn't have the required access permissions (Example: elasticfilesystem:CreateAccessPoint).1.    Retrieve the service account that's used by the EFS CSI controller deployment:kubectl get deploy -n kube-system efs-csi-controller -o=jsonpath={'.spec.template.spec.serviceAccount'}2.    Find the IAM role that's used by the service account:kubectl get sa -n kube-system ${SERVICE_ACCOUNT} -oyaml -o=jsonpath={'.metadata.annotations.eks\.amazonaws\.com/role-arn'}3.    Download the IAM policy document from GitHub:curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/docs/iam-policy-example.json4.    Create the IAM policy if it doesn't exist:aws iam create-policy --policy-name AmazonEKS_EFS_CSI_Driver_Policy --policy-document file://iam-policy-example.json5.    Attach this IAM policy to the IAM role that you previously retrieved, annotated with the service account that's used by the EFS CSI controller deployment.aws iam attach-role-policy --policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/AmazonEKS_EFS_CSI_Driver_Policy --role-name ${IAM_ROLE_NAME}Note:You can retrieve the account ID by running the following command:aws sts get-caller-identity --query "Account" --output textYou can retrieve the IAM role by running the following command:echo $IAM_ROLE_ARN | cut -d "/" -f2Failed to fetch File System info: Describe File System failed: WebIdentityErr: failed to retrieve credentials caused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentityYou get this error because of one of the following reasons:The IAM permission sts:AssumeRoleWithWebIdentity isn't provided.The IAM OIDC identity provider listed in the trust relationship document that's attached to the IAM role isn't correct.The Kubernetes service account (example: system:serviceaccount:kube-system:efs-csi-controller-sa) mentioned doesn't match the one that's used by the EFS CSI controller deployment.To troubleshoot this error, do the following:1.    Retrieve your cluster's OIDC provider ID and store it in a variable.oidc_id=$(aws eks describe-cluster --name ${CLUSTER_NAME} --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5); echo ${oidc_id}Be sure to replace CLUSTER_NAME with the name of your cluster.2.    Verify that the IAM OIDC provider ID exists in the AWS account:aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f43.    Review the trust relationship document that's attached to the IAM role:aws iam get-role --role-name ${IAM_ROLE_NAME} --output json --query "Role.AssumeRolePolicyDocument"Verify that the Action sts:AssumeRoleWithWebIdentity is allowed. Also, confirm that the IAM OIDC ID matches the OIDC ID returned by the preceding command.Note:You can retrieve the IAM role by running the following command:echo $IAM_ROLE_ARN | cut -d "/" -f2Related informationIAM roles for service accountsAmazon EFS CSI driverFollow"


Question: How do I troubleshoot keyboard issues when using Amazon WorkSpaces?

My keyboard isn't responding or I have keyboard issues when using Amazon WorkSpaces. How can I fix this?

Answer: "My keyboard isn't responding or I have keyboard issues when using Amazon WorkSpaces. How can I fix this?ResolutionTo troubleshoot keyboard issues when using Amazon WorkSpaces, follow these steps:Note: If you are using macOS Catalina 10.15, see Amazon WorkSpaces macOS client application.Confirm that the keyboard works as expected when using local machine to rule out an issue with the keyboard hardware.Verify that you are using the latest Amazon WorkSpaces client.Reboot the WorkSpace from the Amazon WorkSpaces console.Disable any secure keyboard features in your antivirus software settings.Note: Secure keyboard configuration prevents other applications on your computer or the network from detecting and recording what you type in a particular application. Review the documentation for your antivirus software for instructions on how to disable secure keyboard.Disable any custom key remapping applications.If you're using an international keyboard for a Windows WorkSpace, verify that the keyboard layout on the WorkSpace and the client match. To verify and change keyboard language settings for Windows, see Manage the input and display language settings in Windows 10 on the Microsoft website. To verify and change keyboard language settings for macOS, see Type in another language on your Mac with input sources on the Apple website.macOS only: If enabled, the Secure Keyboard Entry option on macOS can prevent the Amazon WorkSpaces client from receiving input from the operating system. To disable Secure Keyboard Entry on macOS, follow these steps:Open a Terminal window.On the menu bar, choose Terminal, then clear Secure Keyboard Entry.If you've completed the above troubleshooting steps and haven't resolved the issue, contact AWS Support for assistance.Related informationAmazon WorkSpaces Windows client applicationAmazon WorkSpaces Linux client applicationFollow"