Question: How do I make sure that the CodeDeploy agent is deployed when I use CloudFormation to provision Amazon EC2 instances?

I'm provisioning my infrastructure and AWS CodeDeploy resources using AWS CloudFormation. How do I make sure that the CodeDeploy agent is installed and running on my Amazon Elastic Compute Cloud (Amazon EC2) instances?

Answer: "I'm provisioning my infrastructure and AWS CodeDeploy resources using AWS CloudFormation. How do I make sure that the CodeDeploy agent is installed and running on my Amazon Elastic Compute Cloud (Amazon EC2) instances?Short descriptionIf you create either of the following resources as the target of a CodeDeploy deployment, then use the cfn-init and cfn-signal helper scripts:AWS::EC2::InstanceAWS::AutoScaling::AutoScalingGroupBefore you provision your infrastructure deployment, you can use the helper scripts to fully provision your target instances with the CodeDeploy agent.ResolutionTo use the cfn-init and cfn-signal helper scripts to deploy the CodeDeploy agent, complete the following steps to correctly configure the AWS CloudFormation template.1.    In a code editor, create a JSON or YAML file to use as the AWS CloudFormation template for your resources.2.    For the Metadata key, use the AWS::CloudFormation::Init type to include metadata for your instance or launch configuration.Example Metadata key that uses the AWS::CloudFormation::Init type"SampleLinuxInstance" : {            "Type" : "AWS::EC2::Instance",            "Metadata" : {                "AWS::CloudFormation::Init" : {                    "config" : {                        "packages" : {                            "yum" : {                                "ruby" : []                            }                        },                        "files" : {                            "/home/ec2-user/install" : {                                "source" : {"Fn::Join" : ["", ["https://aws-codedeploy-", {"Ref" : "AWS::Region"}, ".s3.amazonaws.com/latest/install"]]},                                "mode" : "000755"                            }                        },                        "commands" : {                            "00-install-agent" : {                                "command" : "./install auto",                                "cwd" : "/home/ec2-user/"                            },                            "01-cfn-signal" : {                                "command" : {"Fn::Join" : ["", ["/opt/aws/bin/cfn-signal -e 0 --stack ", { "Ref": "AWS::StackName" }, " --resource SampleLinuxInstance --region ", { "Ref" : "AWS::Region" }]]}                            }                        }                    }                }            },            "Properties" : {                ...            }        }Note: The AWS::CloudFormation::Init type includes installation steps for the CodeDeploy agent. It also includes the command to signal back to the CloudFormation stack after you complete the following steps.3.    Set the UserData property to invoke the cfn-init script.Example UserData property that invokes the cfn-init script"Properties" : {                "InstanceType" : "t2.micro",                "ImageId" : {"Ref" : "AMI"},                "IamInstanceProfile" : {"Ref" : "SampleInstanceProfile"},                "KeyName" : {"Ref" : "SSHKey"},                "UserData" :  {"Fn::Base64" : {                    "Fn::Join" : ["", [                        "#!/bin/bash\n",                        "yum -y update\n",                        "yum -y install cfn-bootstrap\n",                        "/opt/aws/bin/cfn-init -v",                        " --stack ", { "Ref" : "AWS::StackName" },                        " --resource SampleLinuxInstance",                        " --region ", { "Ref" : "AWS::Region" },                         "\n"                    ]]                }},                "Tags" : [                    {                        "Key" : {"Ref" : "InstanceTagKey"},                        "Value" : {"Ref" : "InstanceTagValue"}                    }                ]            }4.    For the instance resource, configure the CreationPolicy attribute to do the following: Require the Amazon EC2 instance or Amazon EC2 Auto Scaling group to send success signals to CloudFormation within a specific time period.Example CreationPolicy attribute that requires an Amazon EC2 instance to send success signals to CloudFormation within a specific time period"SampleLinuxInstance" : {            "Type" : "AWS::EC2::Instance",            "CreationPolicy" : {                "ResourceSignal" : {                    "Count" : "1",                    "Timeout" : "PT5M"                }            },5.    To create an explicit dependency for your deployment group resource on the Amazon EC2 instance or Amazon EC2 Auto Scaling group, use the DependsOn attribute.Example DependsOn attribute that creates an explicit dependency for a deployment group resource"SampleLinuxCDDeploymentGroup" : {            "Type" : "AWS::CodeDeploy::DeploymentGroup",            "DependsOn" : "SampleLinuxInstance",            "Properties" : {                "ApplicationName" : {"Ref" : "SampleCDApp"},                "ServiceRoleArn" : {"Ref" : "ServiceRole"},                "Ec2TagFilters" : [                    {                        "Key" : {"Ref" : "InstanceTagKey"},                        "Value" : {"Ref" : "InstanceTagValue"},                        "Type" : "KEY_AND_VALUE"                    }                ],                "Deployment" : {                    "Revision" : {                        "RevisionType" : "S3",                        "S3Location" : {                            "Bucket" : {"Ref" : "BucketName"},                            "BundleType" : {"Ref" : "BundleType"},                            "Key" : {"Ref" : "BundleKey"}                        }                    }                }            }        }6.    To complete the CloudFormation template, include any other additional resources, properties, or outputs required for your environment.7.    Use the completed template to create a CloudFormation stack.Related informationAWS CloudFormation templates for CodeDeploy referenceCreate an Amazon EC2 instance for CodeDeploy (AWS CloudFormation template)How can I use launch configurations to automatically install the CodeDeploy agent on an EC2 instance running Amazon Linux or Ubuntu?Follow"


Question: How do I troubleshoot liveness and readiness probe issues with my Amazon EKS clusters?

I want to troubleshoot issues related to liveness and readiness probes in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster.

Answer: "I want to troubleshoot issues related to liveness and readiness probes in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster.Short descriptionKubelets that are running on the worker nodes use probes to check pod status periodically. Kubernetes currently supports three states in probes: success, failure and unknown. Kubelet considers the pod as successful or healthy under the following conditions:The application running inside the container is ready.The application accepts traffic and responds to probes that are defined on the pod manifest.Kubelet considers an application pod as failed or unhealthy when the probe doesn't respond. Kubelet then marks this pod as unhealthy and sends SIGTERM to the pod. Either of the following happen based on the lifecycle policy and restartPolicy that are defined on the deployment:The pod is terminated immediately.The pod is shut down gracefully after it stops to accept traffic.Example:spec: containers: - name: "example-container"  image: "example-image"  lifecycle:   preStop:    exec:     command: ["sh", "-c", "sleep 10"]In this example, if example-container that runs inside example-pod becomes unhealthy by not responding to probes, then the pod stops accepting traffic in 10 seconds. Then, kubelet shuts down the pod gracefully. If the pod isn't terminated even after 30 seconds, kubelet forcefully removes the pod. Kubelet considers the application pod as unknown when kubelet can't determine the status of the pod using probes that are defined on the deployment manifest. In this case, kubelet performs additional checks to determine the pod status.Kubernetes provides probe checks on liveness probe, readiness probe, and startup probe.Kubelet uses liveness probes to know the state of the application that's running inside your pod.Kubelet uses readiness probes to know when your application is ready to start serving the incoming traffic.Kubelet uses startup probes for slow-starting applications inside the pod. When a startup probe is configured, liveness and readiness probes don't check the pod until startup is considered successful.If none of these probes are defined on the pod manifest, kubelet indefinitely marks the pods as successful or healthy. You can configure one of the following probes to check pod health:HTTP probeCommand probeTCP socket probegRPC probeResolutionApplication healthcheck fail errors due to client timeoutsWhen your liveness or readiness probes fail, you get the following error messages:Liveness probe failed: Get "http://podIP:8080/health ": context deadline exceeded (Client.Timeout exceeded while awaiting headers)Readiness probe failed: Get "http://podIP:8080/health ": context deadline exceeded (Client.Timeout exceeded while awaiting headers)To troubleshoot these errors, do the following:Check whether you configured the liveness and readiness probes for the application pods correctly.If you're using the Amazon Virtual Private Cloud (Amazon VPC) CNI version 1.11.0 or later, then be sure that POD_SECURITY_GROUP_ENFORCING_MODE is set to standard in the aws-node DaemonSet. If this setting is incorrect, run the following command:kubectl set env daemonset aws-node -n kube-system POD_SECURITY_GROUP_ENFORCING_MODE=standardBe sure to set DISABLE_TCP_EARLY_DEMUX to true for amazon-k8s-cni-init that's the container under initcontainers when the following conditions are true:You're using an Amazon VPC CNI version that's earlier than version 1.11.0.Your pods are configured with security groups.ENABLE_POD_ENI is set to true.kubectl patch daemonset aws-node -n kube-system \-p '{"spec": {"template": {"spec": {"initContainers": [{"env":[{"name":"DISABLE_TCP_EARLY_DEMUX","value":"true"}],"name":"aws-vpc-cni-init"}]}}}}'Amazon VPC CNI failure errorsYour aws-node DaemonSet might fail with the following errors:Liveness probe failed: OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: read init-p: connection reset by peer: unknownWarning  Unhealthy  11m (x3 over 12m)    kubelet            Liveness probe failed:Normal   Killing    11m                  kubelet            Container aws-node failed liveness probe, will be restartedReadiness probe failed: OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: process_linux.go:99: starting setns process caused: fork/exec /proc/self/exe: resource temporarily unavailable: unknownWarning  Unhealthy  11m (x9 over 13m)    kubelet            Readiness probe failed:You can resolve these errors by increasing the value of timeoutSeconds to 60 seconds on the aws-node DaemonSet.To view the current values of fields in the VPC CNI, run the following command:$kubectl get daemonset aws-node -n kube-system -o yamlThe output looks similar to the following:"livenessProbe":          exec:            command:            - /app/grpc-health-probe            -addr=:50051            -connect-timeout=5s            -rpc-timeout=5s          failureThreshold: 3          initialDelaySeconds: 60          periodSeconds: 10          successThreshold: 1          timeoutSeconds: 60Application connection errorsWhen you run the describe command on custom application pods, you get the following errors if the pods fail the liveness and readiness probe checks:2m 25s Warning  Unhealthy  Liveness probe failed: Get "http://podIP:8081/health ": dial tcp 192.168.187.28: 8081: connect: connection refused2m 25s Warning  Unhealthy   Readiness probe failed: Get "http:// podIP:8081/health": dial tcp 192.168.187.28:8081: connect: connection refusedWarning  Unhealthy  39s (x4 over 2m19s)  kubelet            Liveness probe failed: HTTP probe failed with statuscode: 500Warning  Unhealthy  29s (x5 over 2m19s)  kubelet            Readiness probe failed: HTTP probe failed with statuscode: 500To troubleshoot this error, do the following:1.    Manually curl the health check path that's defined on the pod manifest from the worker node.[ec2-user@ip-10-0-0-11 ~]$ curl -ikv podIP:8081/health2.    Exec into the application pod that fails the liveness or readiness probes. Then, curl the health check path that's defined on the pod manifest:local@bastion-host ~ % kubectl exec <pod-name> -- curl  -ikv "http://localhost:8081/_cluster/health?"3.    Check the kubelet logs of the worker node where the pod is running for any errors:[ec2-user@ip-10-0-0-11 ~]$ journalctl -u kubelet //optionally 'grep' with pod name4.    Run the describe pod command on the pod and check the current status for the containers that are running in the pod. Also, check the pod logs:$ kubectl describe pod <pod name> -n <namespace>$ kubectl logs <pod name>5.    If you still don't have information on the error, then consider increasing the verbosity of the underlying kubelet that's running on the worker node:$ sudo systemctl status kubelet$ vi /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf    [Service]   Environment='KUBELET_ARGS=--node-ip=192.168.31.211 --pod-infra-container-image=602401143452.dkr.ecr.us-east-2.amazonaws.com/eks/pause:3.5 --v=2'In the configuration file, change --v=2 to --v=9, and then save the file.Restart the kubelet for the changes to take effect:$ sudo systemctl daemon-reload && sudo systemctl restart kubelet && sudo systemctl enable kubeletRun the following command to check the verbosity of the kubelet:$ systemctl status kubelet -lThe output must look like the following:CGroup: /system.slice/kubelet.service        └─5909 /usr/bin/kubelet --cloud-provider aws --config /etc/kubernetes/kubelet/kubelet-config.json --kubeconfig /var/lib/kubelet/kubeconfig --container-runtime docker --network-plugin cni --node-ip=10.0.0.11 --pod-infra-container-image=602401143452.dkr.ecr.us-east-1.amazonaws.com/eks/pause:3.1-eksbuild.1 --v=9Restart the pod that fails with liveness or readiness checks. Then, make sure that this pod is being deployed on the Amazon EKS worker node where the preceding changes were made. You can check the kubelet logs by running the following command:$ journalctl -u kubelet //optionally 'grep' with pod name6.    Run the same container image on bastion host and check whether you can curl the health check path that's defined on the probes in manifest. Also, check the container logs.7.    For HTTP probes, check if the custom http header is configured correctly. Kubelet uses the golang code that's equivalent of curl to HTTP test your pod.Example:"livenessProbe":   httpGet:        path: /healthz        port: 8080        httpHeaders:        - name: Custom-Header          value: Awesome      initialDelaySeconds: 3      periodSeconds: 38.    If you're using an exec probe, check whether your pod is configured correctly. In the following example, the pod passes for 30 seconds, and then fails:apiVersion: v1kind: Podmetadata:  labels:    test: liveness  name: liveness-execspec:  containers:  - name: liveness    image: registry.k8s.io/busybox    args:    - /bin/sh    - -c    - touch /tmp/healthy; sleep 30; rm -f /tmp/healthy; sleep 600    livenessProbe:      exec:        command:        - cat        - /tmp/healthy      initialDelaySeconds: 5      periodSeconds: 5Follow"


Question: How do I use Auto Scaling to improve the fault tolerance of an application behind my load balancer?

I have an application that's running on Amazon Elastic Compute Cloud (Amazon EC2) instances behind a load balancer. I want to use Auto Scaling to improve the fault tolerance of my application.

Answer: "I have an application that's running on Amazon Elastic Compute Cloud (Amazon EC2) instances behind a load balancer. I want to use Auto Scaling to improve the fault tolerance of my application.ResolutionElastic Load Balancing (ELB) supports the following types of load balancers: Application Load Balancer, Network Load Balancer, and Classic Load Balancer. If you have a preexisting load balancer, then you can create an Auto Scaling group to automatically terminate unhealthy instances and launch new, healthy ones. This improves the fault tolerance of your application. You can also configure Auto Scaling to dynamically scale up your application in response to demand.You can create an Auto Scaling group that launches copies of instances that you've already configured. You can also create a launch template that uses an Amazon Machine Image (AMI) instead. After you create the Auto Scaling group, attach your load balancer to it.Note: You must create your Auto Scaling group in the same AWS Region and Availability Zone as your load balancer.Related informationTutorial: Set up a scaled and load-balanced applicationFollow"


Question: How do I use Instance Scheduler with CloudFormation to schedule EC2 instances?

I want to use the AWS Instance Scheduler with AWS CloudFormation to schedule Amazon Elastic Compute Cloud (Amazon EC2) instances.

Answer: "I want to use the AWS Instance Scheduler with AWS CloudFormation to schedule Amazon Elastic Compute Cloud (Amazon EC2) instances.Short descriptionUse CloudFormation templates to automate the deployment of the AWS Instance Scheduler.Important: If you use the Instance Scheduler for EC2 instances with an encrypted Amazon Elastic Block Store (Amazon EBS), then your instances can't be started. To start your instances, you must grant the Instance Scheduler a key user role with a key policy to encrypt or decrypt EBS volumes. You must add the key policy to the AWS Key Management Service (AWS KMS) key to allow the key user role to use this key.ResolutionInstall the Instance SchedulerIf you haven't already done so, follow the instructions to install the Instance Scheduler command line interface (CLI).To verify that the installation is successful, run the following command:$ scheduler-cli --versionCreate a CloudFormation stack with the Instance Scheduler templateThe stack deploys an AWS Lambda function, an Amazon DynamoDB table, an Amazon EventBridge rule, and Amazon CloudWatch custom metrics.Open the AWS Management Console.Open CloudFormation with the Instance Scheduler template. Or, go to the Step 1. Launch the instance scheduler stack page, and choose Launch Solution.Note: The template is launched in the US East (N. Virginia) Region by default.In the navigation bar, select the AWS Region where you want to launch your stack with the template, and then choose Next.For Stack name, name your stack.For Instance Scheduler TagName, you can keep the default value as Schedule, or customize it.For Frequency, choose a frequency in minutes to run your scheduler. For example, you can choose 5 minutes.Note: The frequency is the number of minutes that pass before EventBridge initiates the Lambda function again for the Instance Scheduler. If you have a large number of instances, then use the highest frequency possible to avoid throttling. If the frequency isn't often enough for your needs, then you can adjust the Frequency property later.For Enable CloudWatch Metrics, choose Yes.For Enable CloudWatch Logs, choose Yes.(Optional) For Started tags, enter state=started.(Optional) For Stopped tags, enter state=stopped.For cross-account scheduling, provide the Cross-account roles parameter. Put in the ARNs for every role from the secondary accounts, separated by commas. If you aren't using cross-account scheduling, then leave the parameter empty.For all other parameters, customize the stack for your needs.Choose Next.On the Options page, choose Next.Review your settings, and then select I acknowledge that AWS CloudFormation might create IAM resources.Choose Create.Create the periodsTo create periods, you can use the Instance Scheduler CLI, DynamoDB console, or Custom resources. For more information on time periods, see Start and stop times.The following example shows you how to create instances that:Start at 9 AM and stop at 5 PM on Monday through FridayStart at 9 AM and stop at 12 PM on SaturdayFor this example, you must create two periods. For your own scenario, create the appropriate number of periods.Using the Instance Scheduler CLIConnect to the Instance Scheduler CLI, and then run the following commands:$ scheduler-cli create-period --stack your_stack_name --region eu-west-1 --name mon-fri-9-5 --begintime 9:00 --endtime 16:59 --weekdays mon-fri$ scheduler-cli create-period --stack your_stack_name --region eu-west-1 --name sat-9-12 --begintime 9:00 --endtime 11:59 --weekdays satNote: Replace your_stack_name with the stack name that you chose in step 4 and eu-west-1 with your own Region.Using the DynamoDB consoleOpen the DynamoDB console.Choose Tables, and then choose the configuration table.Note: The Instance Scheduler template automatically creates two DynamoDB tables: state and configuration. The state table stores the state of instances that the template stops and starts. The configuration table allows you to specify the periods and schedules for your requirements.Choose Explore Table Items.Choose Create Item.Choose the JSON view, and then use the following JavaScript object notation (JSON) template:{  "type": {    "S": "period"  },  "name": {    "S": "mon-fri-9-5"  },  "begintime": {    "S": "9:00"  },  "endtime": {    "S": "16:59"  },  "weekdays": {    "SS": [      "mon-fri"    ]  }}Note: The preceding JSON template creates the first period. Use a similar JSON template for the second period. Make sure to edit the templates for your requirements.Create a scheduleTo create a schedule, you can use the Instance Scheduler CLI, DynamoDB console, or Custom resources.Using the Instance Scheduler CLIRun the following command:$ scheduler-cli create-schedule --stack your_stack_name --name m-f9-5-sat9-12 --region eu-west-1 --periods mon-fri-9-5,sat-9-12 --timezone UTCUsing the DynamoDB consoleOpen the DynamoDB console.Choose Tables, and then choose the configuration table.Choose Explore Table Items.Choose Create Item.Choose the JSON view, and then use the following JSON template:{  "type": {    "S": "schedule"  },  "name": {    "S": "m-f9-5-sat9-12"  },  "timezone": {    "S": "UTC"  },  "periods": {    "SS": [      "mon-fri-9-5"    ]  }}Tag the instance and test the scheduleWhen you use a CloudFormation stack with the Instance Scheduler, you must define the Instance Scheduler TagName parameter. The default value for this parameter is Schedule.The Instance Scheduler monitors tags on instances. If the instance tag key matches the defined scheduler tag, then the Instance Scheduler applies the schedule that's set for the instance tag value. For example, a tag's key is set to Schedule and the value is set to m-f9-5-sat9-12. In this example, the instances start at 9 AM and stop at 5 PM on Monday through Friday. The instances also start at 9 AM and stop at 12 PM on Saturday.Note: Tag keys and values are case sensitive. The Instance Scheduler doesn't stop running instances if they're manually started outside of the running period. The Instance Scheduler also doesn't start an instance if the instance is stopped manually during the running period, unless the schedule is enforced. For more information, see Schedule definitions.Use predefined schedulesIn addition to custom schedules, you can also use any of the predefined schedules from the configuration table. For example, the following steps test the predefined schedule named running:Open the Amazon EC2 console.Choose the stopped instances that you want to tag.Choose the Tags view, and then choose Manage Tags.Choose Add Tag.For Key, enter Schedule.For Value, enter running.Choose Save.Refresh the Amazon EC2 console, and then wait for the Lambda function to be initiated.Note: When the Lambda function is initiated and runs without errors, then the Instance State displays as running, depending on the schedule that you're testing. In the CloudWatch console, you can check Lambda metrics for invocations and errors.Open the DynamoDB console.Choose Tables, and then choose the state table.Choose the Explore Table Items and confirm that the tagged instance is started.Note: The state data is stored in the state table.Important: You can be charged additional costs based on the frequency and duration of the Lambda function that you're using. You can also be charged additional costs for the DynamoDB tables or EventBridge rules that you create.For cross-account scheduling: Launch the remote stack in secondary accountsTo schedule instances in secondary accounts using the Instance Scheduler, deploy the aws-instance-scheduler-remote CloudFormation  template. This template creates the role that allows the Instance Scheduler from the primary account to manage instances in the secondary account.Note: You must provide the role's ARN as a parameter for the Instance Scheduler stack in the primary account. Make sure to create or update your Instance Scheduler stack with the correct parameter.Open the AWS Management Console of the secondary account and launch the aws-instance-scheduler-remote CloudFormation template. You can also  download the template for future use.Note: The template is launched in the US East (N. Virginia) Region by default.In the navigation bar, select the AWS Region where you want to launch your stack with the template, and then choose Next.On the Select Template page, verify that you selected the correct template, and then choose Next.On the Specify Details page, assign a name to your remote stack.Under Parameters, review and modify the Primary account parameter. Put in the account number of the primary account.Choose Next.On the Options page, choose Next.Review your settings, and then select I acknowledge that AWS CloudFormation might create IAM resources.Choose Create.Choose the stack Outputs tab, and then copy the CrossAccountRole value.From the primary account, choose your CloudFormation stack, and then choose Update.On the Update stack page, choose Use current template.In the Cross-account roles parameter, paste the CrossAccountRole value.Choose Next, and then select I acknowledge that AWS CloudFormation might create IAM resources.Choose Update Stack.Follow"


Question: How do I troubleshoot using my on-premises Active Directory login to my RDS for SQL Server instance?

I'm unable to access my Amazon Relational Database Service (Amazon RDS) for Microsoft SQL Server when using my on-premises Active Directory login. How can I troubleshoot this?

Answer: "I'm unable to access my Amazon Relational Database Service (Amazon RDS) for Microsoft SQL Server when using my on-premises Active Directory login. How can I troubleshoot this?Short descriptionWhen setting up Windows Authentication with Amazon RDS, a forest trust must be created. This is created with the AWS Directory Service for Microsoft Active Directory (AWS Managed Microsoft AD). A forest trust is set up whether you are using an on-premises or self-hosted AWS Managed Microsoft AD. While using an on-premises login after configuring the trust relationship, the following login error might appear for multiple reasons:"Login Failed. The Login is From an Untrusted Domain and Cannot be Used with Windows Authentication"ResolutionTo troubleshoot Active Directory login errors, check the following:Amazon RDS domain statusAfter you create or modify your DB instance, the instance becomes a member of the domain. The RDS console indicates the status of the domain membership for the DB instance. For more information on DB instance status', seeUnderstanding Domain membership. If you receive a "Failed" error when joining a DB instance to a domain or directory status on the RDS console, seerejoining a DB instance.If you receive an AWS Identity and Access Management (IAM) error, this might be due to not using the default IAM role rds-directoryservice-access-role. If you're using a custom IAM role, then attach the default policy AmazonRDSDirectoryServiceAccess to resolve the error.Trust relationshipYou can configure one and two-way external and forest trust relationships between your AWS Managed Microsoft AD and self-managed (on-premises) directories. You can also configure one and two-way external and forest trust relationships between multiple AWS Managed Microsoft ADs in the AWS Cloud. AWS Managed Microsoft AD supports all three trust relationship directions: incoming, outgoing, and two-way. To access the RDS console using an on-premises login, make sure that the trust status is in the "verified" state. For more information on verifying trust relationships, see Create, verify, or delete a trust relationship.Forest-wide and selective authenticationsWhile creating a forest trust using AWS Directory Service console, you have an option to turn on "selective authentication". If this option isn't turned on, then the authentication is treated as "forest-wide authentication".Forest-wide authenticationWhen forest-level authentication is turned on, the domain controllers of the forest authenticate all access requests made by users of the trusted forest. After authentication is successful, access to the resource is granted or rejected based on the resource Access Control List (ACL).There is a risk in this approach. After the foreign user (from trusted forest) us successfully authenticated, they become a member of the "Authenticated User" group. This group doesn't have any permanent members, and membership is computed dynamically based on authentication. After an account is a member of the "Authenticated User" group, that account can access all resources where the group "Authenticated user" has access.Selective authenticationTo have control of the authentication, you can opt for the selective authentication level. In this level, not all users are authenticated by domain controllers by default. Instead, when a domain controller detects that an authentication request is coming from a trusted forest, the domain controller validates the user account. The domain controller validates that the user account was granted exclusive permission on the resource that is holding the object.When selective authentication is turned on, you must add the respective users and groups of the on-premises Active Directory. Users and groups must be added to the "AWS Delegated Allowed to Authenticate Objects" group of AWS Managed AD. The "AWS Delegated Allowed to Authenticate Objects" is assigned the "Allowed to Authenticate" permission. All users that are part of this group can access the RDS instance. Users that are not part of this group can't access the Amazon RDS SQL Server.Note: "AWS Delegated Allowed to Authenticate Objects" group is created by default after configuring AWS Managed AD. Members of this group are provided the ability to authenticate to computer resources in the AWS reserved organizational units (OU). This is needed only for on-premises objects with selective authentication trusts.Login and password statusOn-premises Active Directory login passwords and statuses can't be expired or locked. If so, check the login status using the following command:net user username/domainYou only need to change the username to the user you want to check the status of. Leave the domain as it is.Duplicate Service Principal Names (SPN)By default, Amazon RDS creates an SPN as required. Creating additional SPN's for the on-premises Active Directory login for other uses might cause a login failure. For more information, see Identify, remove, and verify an SPN.Security patchesIf you are seeing an on-premises Active Directory login error and have verified the trust relationship, then check the latest security patches. Check the Distributed Control System (DCS) or Domain Name System (DNS) servers for known issues with Windows updates (KB). If there are any issues due to security patches or KBs, you might have to roll back the updates. If rolling back the updates doesn't fix the issue, then try to apply the fix from Microsoft, if available.Related informationEverything you wanted to know about trusts with AWS Managed Microsoft ADFollow"


Question: Why is my on-demand DynamoDB table being throttled?

My on-demand Amazon DynamoDB table is being throttled.

Answer: "My on-demand Amazon DynamoDB table is being throttled.Short descriptionYour DynamoDB on-demand table might be throttled due to one of the following common reasons:The traffic is more than double the previous peak.The traffic exceeds the per-partition maximum.The traffic exceeds the per-table account quota.The global secondary index of your table is being throttled.ResolutionDynamoDB tables that use on-demand capacity mode automatically adapt to your application’s traffic volume. However, tables that use the on-demand mode might still throttle.The traffic is more than double the previous peakIf you exceed double your previous traffic's peak within 30 minutes, then you might experience throttling. Before you exceed your previous traffic's peak, it's a best practice to spread your traffic growth over at least 30 minutes. Use the ConsumedReadCapacityUnits metric in Amazon CloudWatch to monitor traffic to the table. For more information, see DynamoDB metrics and dimensions.For new on-demand tables, you can immediately drive up to 4,000 write request units or 12,000 read request units, or a linear combination of both. For an existing table that you switched to on-demand capacity mode, the previous peak is either half the previous provisioned throughput for the table. Or, the previous peak is the settings for a newly created table with on-demand capacity mode.For more information, see Initial throughput for on-demand capacity mode.The traffic exceeds the per-partition maximumEach partition on the table can serve up to 3,000 read request units or 1,000 write request units, or a linear combination of both. If the traffic to a partition exceeds this limit, then the partition might be throttled. To resolve this issue, take the following actions:Use CloudWatch Contributor Insights for DynamoDB to identify the most frequently accessed and throttled keys in your table.Randomize the requests to the table so that the requests to the hot partition keys are distributed over time. For more information, see Using write sharding to distribute workloads evenly.The traffic exceeds the per-table account quotaFor on-demand tables, the table-level read throughput and table-level write throughput quotas apply at the account level. By default, the table throughput is set to a maximum of 40,000 read requests units and a maximum of 40,000 write requests units. If the traffic to a table exceeds the per-table account quotas for throughput, then the table might be throttled. To resolve this issue, use the Service Quotas console to increase the table-level read throughput and write throughput quotas for your account.The global secondary index of your table is being throttledIf your DynamoDB table has a secondary global index that's being throttled, then the throttling might create back pressure throttles on the base table. For more information, see How does throttling on my global secondary index affect my Amazon DynamoDB table? and Using global secondary indexes in DynamoDB.Related informationPeak traffic and scaling propertiesHow do I choose the right primary key for my Amazon DynamoDB table?Why are the metrics on the DynamoDB console different from the CloudWatch metrics?Service, account, and table quotas in Amazon DynamoDBFollow"


Question: Why am I seeing high or increasing memory usage in my ElastiCache cluster?

I'm seeing high or increasing memory usage in my Amazon ElastiCache cluster. How is memory usage determined on ElastiCache cluster nodes?

Answer: "I'm seeing high or increasing memory usage in my Amazon ElastiCache cluster. How is memory usage determined on ElastiCache cluster nodes?ResolutionTo determine overall memory usage on your cluster and its nodes, review these Redis metrics. These metrics are published in Amazon CloudWatch for each node in a cluster:BytesUsedForCache: The total number of bytes allocated by Redis for dataset, buffers, and so on. This metric is derived from a Redis node's INFO command output. Use this metric to determine the memory utilization of your cluster.FreeableMemory: This host level metric shows the amount of free memory available on the host. When memory usage increases due to cache data or by overhead, you can see the decrease in FreeableMemory. A decrease in FreeableMemory suggests low free memory on the host. Swap might occur if FreeableMemory is too low.DataBaseMemoryUsagePercentage: This metric is derived from the Redis INFO command output. This is the percentage of memory used by the cluster node. Redis initiates the Redis maxmemory eviction policy after this metric reaches 100% of the threshold.Keep in mind that by default ElastiCache for Redis reserves 25% of the max-memory for non-data usage, such as fail-over and backup. If you don't specify enough reserved memory for non-data usage, the chance of swapping increases. For more information, see Managing reserved memory.Causes of sudden high memory usageRecently added keys: Adding new key-value pairs causes an increase in memory usage. Adding elements on existing keys also increases memory usage. Check the SetTypeCmds metric to determine if there are recent data changes on the node. This metric logs the total number of write type commands and is derived from the Redis commandstats statistic.Increase in buffer usage: Clients are connected to Redis over the network. If the client isn't reading from the cache fast enough, Redis keeps the response data in a memory space called the client output buffer. The client can continue reading from the buffer space. This is also true for Pub and Sub clients if the subscribed clients aren't reading fast enough.If there is a bottleneck in network bandwidth or if the cluster is continuously under heavy load, then the buffer usage might continue to accumulate. This accumulation causes memory exhaustion and performance degradation. By default, ElastiCache for Redis doesn't restrict growth in output buffer and each of the clients have their own buffer. Use the client-list command to check buffer usage.Large number of new connections: A large number of new connections might elevate memory usage. All new connections create a file descriptor that consumes memory. The aggregate memory consumption with a large number of new connections might be high, leading to data eviction or OOM errors. Check the NewConnections metric for the total number of accepted new connections.High swap usage: It's normal to see some swap usage on a cache node when there is free memory. However, too much swap usage might lead to performance issues. High swap usually starts happening in a node that's running under memory pressure, resulting in low freeable memory. Use the SwapUsage metric to monitor swap on the host.High memory fragmentation: A high memory fragmentation indicates inefficiencies in memory management within the operating system. Redis might not free up memory when keys are removed. Use the MemoryFragmentationRatio metric to monitor the fragmentation ratio. If you're running into fragmentation issues, turn on the activedefrag parameter can for active memory defragmentation.Big keys: A key with a large data size or a large number of elements in it is called a big key. You might see high memory usage as a result of big key even if the CurrItems metric stays low. To detect big keys in your data set, use the redis-cli --bigkeys command.Best practices to control high memory usageUse TTL on keys: You can specify TTL on keys for expiration. Doing this removes keys after expiration without waiting for memory pressure. This prevents cluttering Redis with unnecessary keys. A small number of evictions isn't a concern, but a high number of evictions means your node is running on memory pressure.Use eviction policy: When cache memory starts to fill up, Redis evicts the keys to free up space based on the maxmemory-policy. The default maxmemory-policy policy is set to volatile_lru. It's a best practice to choose an eviction policy that is specific to the needs of your work load.Allocate reserved memory: To avoid issues during failover or backup, it's a best practice to set reserved_memory_percentage to at least 25% for non-data usage. If there is not enough reserved memory to perform failover or backup, swap and performance issues occur.Use connection pooling: Connection pooling helps control high numbers of new connections attempted by the Redis client. Review the AWS best practice guidelines for handling a large number of new connections.Adjust output buffer size limits: You can adjust the output buffer limit to control the buffer space usage. ElastiCache for Redis parameter groups provide several parameters starting with client-output-buffer-limit-* to avoid unlimited growth of client output buffer usage. Be aware that there isn't a suggested limit for these parameters as every workload is unique. It's a best practice to benchmark your workload so that you can choose an appropriate value.Consider using hash mapping: In Redis, the total memory footprint of the Redis DB is linear. It takes more memory with fewer individual keys than a single hash-mapped key with fewer fields. Hash mapping helps with data structures that have a large number of keys. In addition to hash mapping, you can take advantage of ziplist encoding, which reduces the memory footprint compared to hash-tables. Note that using hash mapping might cause a spike in Redis engine usage because this is a complex command that needs more CPU than set operations.Scale the cluster: Sometimes you might experience memory pressure after taking the necessary precautions. If this occurs and if the usage is due to expected workload, consider performing an appropriate scaling to ease the memory bottleneck.Set an alarm for memory usage. You can use CloudWatch alarms to initiate an alarm when memory usage crosses a preset threshold. Use the BytesUsedForCache or DatabaseMemoryUsagePercentage metric to create an alarm from the CloudWatch console for monitoring and scaling purposes.Related informationMetrics for RedisRedis 4.0.10 parameter changesHow do I resolve the error message "OOM command not allowed" for an ElastiCache for Redis cluster node?Key evictionMemory optimization for RedisFollow"


Question: How do I get the ID and IP address of an Amazon EC2 instance for an AWS Batch job?

I want to get the ID and IP address of an Amazon Elastic Compute Cloud (Amazon EC2) instance for an AWS Batch job. How do I find that information?

Answer: "I want to get the ID and IP address of an Amazon Elastic Compute Cloud (Amazon EC2) instance for an AWS Batch job. How do I find that information?Short descriptionTo get information about an AWS Batch job's Amazon EC2 instance, first get the job's Amazon Elastic Container Service (Amazon ECS) container instance ID. Then, use the container instance ID to get the associated Amazon ECS cluster name and Amazon EC2 instance ID. You can use the Amazon EC2 instance ID to get the instance's IP address.Note: Amazon ECS resources created after April 1, 2021 use a new long Amazon Resource Name (ARN) format that now includes the ECS cluster name. This article provides instructions for using either the new or the old ARN formats. For more information, see ARNs and IDs in the Amazon ECS Developer Guide.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you're using the most recent AWS CLI version.Get the AWS Batch job's Amazon ECS container instance ID, cluster name, and Amazon EC2 instance IDFor the new ARN formatGet your AWS Batch job's Amazon ECS container instance ID, cluster name, and Amazon EC2 instance ID by running the following bash script:Important: Replace <your_job_ID> with your AWS Batch job's job ID.#!/bin/bashJOB_ID=<your_job_ID>CONTAINER_INSTANCE_ARN=$(aws batch describe-jobs --jobs "$JOB_ID" --query 'jobs[0].container.containerInstanceArn' --output text);TMP=${CONTAINER_INSTANCE_ARN#*/}CLUSTER_NAME=${TMP%/*}EC2_ID=$(aws ecs describe-container-instances  --container-instances "$CONTAINER_INSTANCE_ARN" --cluster "$CLUSTER_NAME" --query "containerInstances[0].ec2InstanceId" --output text)echo ${EC2_ID}For the old ARN format1.    Get your AWS Batch job's Amazon ECS container instance ID by running the following describe-jobs command:$ aws batch describe-jobs --jobs <your_job_ID> --query 'jobs[0].container.containerInstanceArn' --output textThe output returns a container instance ARN value that includes the container instance ID.2.    Find the Amazon ECS cluster that corresponds to your AWS Batch compute environment by doing the following:Open the Amazon ECS console.Choose Clusters.Choose the Services tab.Then, search for the cluster name that appears in the following format: ComputeEnvironmentName_Batch_RandomStringOfCharacters3.    Get the cluster's associated Amazon EC2 instance ID by running the following describe-container-instances command:Important: Replace <Your_Cluster_Name> with your Amazon ECS cluster's name. Replace <Your_Container_Instance_ID> with your AWS Batch job's Amazon ECS container instance ID.$ aws ecs describe-container-instances --cluster <Your_Cluster_Name> --container-instances <Your_Container_Instance_ID> --query "containerInstances[0].ec2InstanceId"The output returns the job's associated Amazon EC2 instance ID.Use the Amazon EC2 instance ID to get the instance's IP addressYou can use either Amazon EC2 console or the AWS CLI to get an Amazon EC2 instance's IP address.Amazon EC2 console1.    Open the Amazon EC2 console.2.    Choose Running instances.3.    In the Instance ID column, find your Amazon EC2 instance's instance ID.4.    In the same row as your instance ID, look at the IPv4 Public IP address column. The value listed is the Amazon EC2 instance's IP address.AWS CLIRun the following describe-instances command:Important: Replace <EC2_ID> with your Amazon EC2 instance ID.aws ec2 describe-instances --instance-ids <EC2_ID> --query 'Reservations[0].Instances[0].{"PrivateIP":PrivateIpAddress,"PublicIP":PublicIpAddress}'To get instance information for multi-node AWS Batch jobs1.    Get the container instance ID for a child job by running the following describe-jobs command on an individual child job:Important: Replace Your_Job_ID with your AWS Batch job ID.$ aws batch describe-jobs --jobs Your_Job_ID#1 --query 'jobs[0].container.containerInstanceArn' --output textImportant: Because each child job is on a unique node, the parent job doesn't have information on the container instance. For child jobs, use node notation to append the child job's index to the job ID. In this example describe-jobs command, #1 accesses the first node of the job. Adding a number #2 accesses the second node, and so on. For more information, see Multi-node parallel jobs.2.    Get the associated Amazon EC2 instance ID by running the following describe-container-instances command:Important: Replace Your_Cluster_Name with your Amazon ECS cluster's name. Replace Your_Container_Instance_ID with your AWS Batch job's Amazon ECS container instance ID.$ aws ecs describe-container-instances --cluster Your_Cluster_Name --container-instances Your_Container_Instance_ID --query "containerInstances[0].ec2InstanceId"Follow"


Question: How do I create Amazon Data Lifecycle Manager policies from selected EBS volumes in the EC2 console?

I want to create Amazon Data Lifecycle Manager policies from my Amazon Elastic Block Store (Amazon EBS) volumes in the Amazon Elastic Compute Cloud (Amazon EC2) console. How do I do this?

Answer: "I want to create Amazon Data Lifecycle Manager policies from my Amazon Elastic Block Store (Amazon EBS) volumes in the Amazon Elastic Compute Cloud (Amazon EC2) console. How do I do this?ResolutionYou can select one or more Amazon EBS volumes in the Amazon EC2 console to create an Amazon Data Lifecycle Manager policy that's used to manage snapshots of the EBS volumes. An Amazon Data Lifecycle Manager policy creates and deletes snapshots according to the policy schedule. This saves you on costs and effort because you don't have to match the snapshots manually.To create an Amazon Data Lifecycle Manager policy:Open the Amazon EC2 console.Make sure that all the EBS volumes that you want Amazon Data Lifecycle Manager to manage have at least one tag with a key-value pair.Select the desired EBS volumes, choose Actions, and then choose Create snapshot lifecycle policy.All the tags for the EBS volumes selected in the previous step are automatically prefilled under Target resource tags.Define the rest of the Amazon Data Lifecycle Manager policy, including the creation frequency and retention period.For step-by-step instructions, see Automating Amazon EBS snapshot and AMI management using Amazon Data Lifecycle Manager.After creating the policy, Amazon Data Lifecycle Manager automatically creates and deletes snapshots based on the policy schedule.Troubleshooting tipsTo create an EBS snapshot lifecycle policy, your EBS volumes must have at least one tag with both a key and a value. If you select one or more volumes that don't have any tags, and then select Create a snapshot lifecycle policy in the Amazon EC2 console, you receive an error.When you select an EBS volume with multiple tags, and at least one tag has both a key and a value, Create snapshot lifecycle policy is prefilled with tags with both keys and values. Any tags with a key but no value aren't prefilled.All the tags of your selected EBS volumes are prefilled with both keys and values. When other volumes aren't selected, but they have at least one of the prefilled tags, the Amazon Data Lifecycle Manager policy creates and manages the snapshots for those volumes.The total number of unique tags in your selected EBS volumes can't be greater than 45.When you remove tags from EBS volumes after a policy is created, the Amazon Data Lifecycle Manager can no longer create snapshots of those volumes. Refer to the Amazon Data Lifecycle Manager documentation for more information.Follow"


Question: How can I use Apache Iceberg with a cross-account AWS Glue Data Catalog in Spark?

I want to use Spark with Amazon EMR or AWS Glue to interact with Apache Iceberg from a cross-account AWS Glue Data Catalog.

Answer: "I want to use Spark with Amazon EMR or AWS Glue to interact with Apache Iceberg from a cross-account AWS Glue Data Catalog.ResolutionSet the following parameters to use Spark to interact with Apache Iceberg tables from the AWS Glue Data Catalog:--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \  --conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog \  --conf spark.sql.catalog.dev.glue.id=<CROSS_ACCOUNT_ID> \  --conf spark.sql.catalog.glue_catalog.warehouse=s3://<your-warehouse-dir>/ \  --conf spark.sql.catalog.glue_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog \  --conf spark.sql.catalog.glue_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIOYou can set these parameters in a number of ways, depending on whether you use an AWS Glue job or an Amazon EMR cluster.For AWS Glue jobs, use job parameters. For example:Key:  --conf  Value: spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions --conf spark.sql.catalog.dev=org.apache.iceberg.spark.SparkCatalog --conf spark.sql.catalog.dev.glue.id=<CROSS_ACCOUNT_ID> --conf spark.sql.catalog.dev.warehouse=s3://<WAREHOUSE_DIR>/ --conf spark.sql.catalog.dev.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog --conf spark.sql.catalog.dev.io-impl=org.apache.iceberg.aws.s3.S3FileIOFor an Amazon EMR cluster that runs version 6.5 or later, set the parameters when you submit the job. Or, use the Spark default configuration (/etc/spark/conf/spark-defaults.conf). For more information, see Use an Iceberg cluster with Spark.Note: For cross-account scenarios, you must always use the glue.id property to specify the corresponding AWS Glue Data Catalog ID (AWS account ID).If you're using Amazon EMR version 6.5 or later, then use the following spark-defaults configuration:]    }          "configurations": []          },              "spark.sql.catalog.dev.io-impl": "org.apache.iceberg.aws.s3.S3FileIO"              "spark.sql.catalog.dev.catalog-impl": "org.apache.iceberg.aws.glue.GlueCatalog",              "spark.sql.catalog.dev.warehouse": "s3://<WAREHOUSE_DIR>/",              "spark.sql.catalog.dev.glue.id": "<CROSS_ACCOUNT_ID>",              "spark.sql.catalog.dev": "org.apache.iceberg.spark.SparkCatalog",              "spark.sql.extensions": "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",              "spark.jars": "/usr/share/aws/iceberg/lib/iceberg-spark3-runtime.jar",          "properties": {          "classification": "spark-defaults",      {      },          "configurations": []          },              "iceberg.enabled": "true"          "properties": {          "classification": "iceberg-defaults",      {  [Note: The Amazon EMR or AWS Glue job must have sufficient AWS Identity and Access Management (IAM) permissions to access the cross-account AWS Glue Data Catalog. For more information, see Making a cross-account API call.Related informationRead an Iceberg table from Amazon S3 using SparkFollow"


Question: How do I troubleshoot scaling issues with my Amazon EC2 Auto Scaling group?

My Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling group isn't scaling correctly. How do I troubleshoot this?

Answer: "My Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling group isn't scaling correctly. How do I troubleshoot this?ResolutionBased on your scaling issues, perform the following checks on your Amazon EC2 Auto Scaling configurations:My Amazon EC2 Auto Scaling group isn’t following the scaling policy I configuredCheck your scaling policies to see whether an event triggers more than one policy. If two policies are evaluated at the same time, Amazon EC2 Auto Scaling follows the policy with the greater impact. For example, assume you have one policy to add two instances and another policy to add four instances. In this case, Amazon EC2 Auto Scaling adds four instances when both policies are triggered at the same time.My Amazon EC2 Auto Scaling group is scaling out instead of scaling inVerify if a scale-out policy and a scale-in policy are triggered at the same time. When both scale-out and scale-in policies are triggers at the same time, Amazon EC2 Auto Scaling follows the scale-out policy to confirm availability.To verify if scale-out and scale-in policies are triggered at the same time:Review your alarm history from the Amazon CloudWatch console to check if the two associated alarms were triggered at the same time.Review your Auto Scaling group's activity history from the Amazon EC2 console to check if the associated scheduled actions were triggered at the same time.My Amazon EC2 Auto Scaling group isn’t responding to a CloudWatch alarm or health check replacementCheck if your Auto Scaling group already reached its minimum or maximum number of instances. To find the specified minimum or maximum capacity, review your Amazon EC2 Auto Scaling group's details using the Amazon EC2 console. Amazon EC2 Auto Scaling doesn't show an error message in the group's activity history when scaling is blocked due to minimum or maximum capacity.Check suspended processes for your Auto Scaling group. Amazon EC2 Auto Scaling can't evaluate a CloudWatch scaling policy for suspended Terminate, Launch, or Alarm Notification processes. Amazon EC2 Auto Scaling can't evaluate a health check replacement for suspended ReplaceUnhealthy or HealthCheck processes.If you're using a simple scaling policy, check if your instances are in a cooldown period. Simple scaling policies remain suspended until after the cooldown period.If you're using a step scaling policy or target tracking policy, check for an in progress instance warmup. If you specified a warm-up time, the Auto Scaling group doesn't count instances toward the group metrics until after the warmup.Check your CloudWatch alarm to be sure that it's triggering scaling activity correctly. For more information, see How do I ensure that Amazon CloudWatch alarms trigger scaling of my Auto Scaling group?Check if there is a lifecycle hook configured for your Amazon EC2 Auto Scaling group. If there is a lifecycle hook, you might need to either complete the lifecycle hook or wait for the timeout period to end. For more information, see the following section "My instance is in stuck in the Pending:Wait or Terminating:Wait state during scaling activity".My instance is stuck in the Pending:Wait or Terminating:Wait state during scaling activityIf you configured a lifecycle hook for your Amazon EC2 Auto Scaling group, an instance might be paused in the Pending:Wait or Terminating:Wait state. Except for step scaling policies, other scaling activities are suspended until the instance moves out of the Pending:Wait or Terminating:Wait state.To determine if you have a lifecycle hook configured, run the following AWS Command Line Interface (AWS CLI) command:aws autoscaling describe-lifecycle-hooks --auto-scaling-group-name my-asg-nameNote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.To continue the scaling process, wait for the timeout period to end (one hour by default), or complete the lifecycle hook.For information on managing lifecycle hooks from the Amazon EC2 console, see Amazon EC2 Auto Scaling lifecycle hooks.My Amazon EC2 Auto Scaling group isn’t responding to a scheduled actionCheck the time zone you configured for the scheduled action. You must schedule actions in Coordinated Universal Time (UTC).Review your Amazon EC2 Auto Scaling group’s activity history from the Amazon EC2 console. Verify if there is any scaling activity that conflicts with your scheduled action.If you're scheduling scale-out and scale-in actions, check that you scheduled one action for scaling out and another action for scaling in. You can't use the same scheduled action to both scale in and scale out.I reached my Amazon EC2 instance quotaIf your Amazon EC2 Auto Scaling group isn't scaling due to your EC2 instance quota, you receive a message similar to the following:Launching a new EC2 instance. Status Reason: Your quota allows for 0 more running instance(s).You requested at least 1. Launching EC2 instance failed.To increase the quota, contact AWS Support. For more information on quotas, see AWS service quotas.Related informationAuto Scaling lifecycleTroubleshooting Amazon EC2 Auto ScalingFollow"


Question: How can I resolve "Exit status: -100. Diagnostics: Container released on a *lost* node" errors in Amazon EMR?

My Amazon EMR job fails with an error message similar to the following:ExecutorLostFailure (executor 12 exited caused by one of the running tasks) Reason: Container marked as failed: container_1572839353552_0008_01_000002 on host: ip-xx-xxx-xx-xx Exit status: -100. Diagnostics: Container released on a lost node

Answer: "My Amazon EMR job fails with an error message similar to the following:ExecutorLostFailure (executor 12 exited caused by one of the running tasks) Reason: Container marked as failed: container_1572839353552_0008_01_000002 on host: ip-xx-xxx-xx-xx Exit status: -100. Diagnostics: Container released on a lost nodeShort descriptionThis error commonly occurs during either of the following situations:A core or task node is terminated because of high disk space utilization.A node becomes unresponsive due to prolonged high CPU utilization or low available memory.This article focuses on disk space issues.When disk usage on a core or task node disk (for example, /mnt or /mnt1) exceeds 90%, the disk is considered unhealthy. If fewer than 25% of a node's disks are healthy, YARN ResourceManager gracefully decommissions the node. To resolve this problem, add more Amazon Elastic Block Store (Amazon EBS) capacity to the EMR cluster. You can do this when you launch a new cluster or by modifying a running cluster.ResolutionDetermine the root causeTo determine the cause of the error, check the following Amazon CloudWatch metrics for the EMR cluster:MR unhealthy nodes: If this metric shows an unhealthy node, the issue is caused by a lack of disk space.MR lost nodes: If this metric shows a lost node, it indicates that a node was lost due to a hardware failure, or that the node couldn't be reached due to high CPU or high memory utilization.Use one of the following options to resolve lost node errors that are caused by a lack of disk space.New clusters: Add more EBS capacityTo add more EBS capacity when you launch an EMR cluster, choose a larger Amazon Elastic Compute Cloud (Amazon EC2) instance type. Larger EC2 instances include more EBS storage capacity. For more information, see Default EBS Storage for Instances. (You can also modify the volume size or add more volumes when you create the cluster, regardless of the instance type that you choose.)New or running clusters: Add more core or task nodesChoose a larger number of core or task nodes when you launch a new cluster.Add more core or task nodes to a running cluster.Running clusters: Add more EBS volumesDo the following to attach more EBS volumes to a running cluster:1.    If larger EBS volumes don't resolve the problem, attach more EBS volumes to the core and task nodes.2.    Format and mount the attached volumes. Be sure to use the correct disk number (for example, /mnt1 or /mnt2 instead of /data).3.    Connect to the node using SSH.4.    Add the path /mnt1/yarn inside the yarn.nodemanager.local-dirs property of /etc/hadoop/conf/yarn-site.xml. Example:<property>    <name>yarn.nodemanager.local-dirs</name>    <value>/mnt/yarn,/mnt1/yarn</value></property>5.    Restart the NodeManager service:sudo stop hadoop-yarn-nodemanagersudo start hadoop-yarn-nodemanager6.    Enable termination protection.If you still have disk space issues, try the following:Remove unnecessary files.Increase the disk utilization threshold from 90% to 99%. To do this, modify the yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage property in yarn-default.xml on all nodes. Then, restart the hadoop-yarn-nodemanager service.Related informationCluster terminates with NO_SLAVE_LEFT and core nodes FAILED_BY_MASTERWhy is the core node in my Amazon EMR cluster running out of disk space?Follow"


Question: How can I grant access to Cost Explorer for an IAM user with a member account of an AWS Organizations?

I want to allow an AWS Identity and Access Management (IAM) user belonging to an AWS Organizations member account access to AWS Cost Explorer.

Answer: "I want to allow an AWS Identity and Access Management (IAM) user belonging to an AWS Organizations member account access to AWS Cost Explorer.ResolutionFollow these instructions to allow IAM users for an Organizations member account to access Cost Explorer.1.    Using the Organizations management account, turn on Cost Explorer access for all your organization accounts by following the instructions for Controlling member accounts' access using Cost Explorer preferences.Note: You can't grant access to individual member accounts. All member accounts in your organization are granted access. Member account owners can't view Cost Explorer information for other accounts in your organization.2.    Follow the instructions for Activating access to the AWS Billing console.Note: Only the root user of an account can access the Billing and Cost Management console until this step is completed.3.    Attach an IAM policy to the IAM user to allow access to Cost Explorer information. For example policies, see AWS Billing policy examples.Related informationBest practices for the management accountBest practices for member accountsManaging the AWS accounts in your organizationAWS Organizations terminology and conceptsFollow"


Question: How are the metrics and logs formatted for the count rule action in AWS WAF?

I've set up AWS WAF and some of the rules in my web ACL are in the Count action. How do I analyze the Amazon CloudWatch metrics and logs for the rules in the count rule action?

Answer: "I've set up AWS WAF and some of the rules in my web ACL are in the Count action. How do I analyze the Amazon CloudWatch metrics and logs for the rules in the count rule action?Short descriptionYou can use the count rule action to test rules before you implement them with their normal action settings. AWS WAF evaluates the requests against rules in the count rule action and reports matches in metrics, request samples, and logs.You can also use the count rule action to troubleshoot rules that are generating false positives. False positives occur when a rule or a rule group blocks traffic that you aren't expecting it to block. If you identify a rule blocking requests that you want allowed, you can update the action for that rule to count. This excludes the rule from acting on your requests. For more information on detecting and mitigating false positives, see How can I detect false positives caused by AWS Managed Rules and add them to a safe list?ResolutionTo understand how the metrics and logs are formatted for the count rule action, consider the following three scenarios:Set the rule action for a custom rule to count mode.Set the rule action for individual rules to count mode in an  AWS Managed Rules rule group.Override an entire rule group's action to count.Set the rule action for a custom rule to count modeExampleYou have a web ACL (Test1-ACL) that has a single custom rule ( Geo-Count) with a count rule action. The Geo-Count rule counts the request if the request originates from the United States. If the client IP belongs to a different country, then the web ACL default action with an Allow action is matched.The following is the JSON representation of this rule:{  "Name": "Geo-Count",  "Priority": 0,  "Statement": {    "GeoMatchStatement": {      "CountryCodes": [        "US"      ]    }  },  "Action": {    "Count": {}  },  "VisibilityConfig": {    "SampledRequestsEnabled": true,    "CloudWatchMetricsEnabled": true,    "MetricName": "Geo-Count"  }}If the request originates from a client IP located in the United States, the following occurs:The request is counted.The CloudWatch metric for the custom rule Geo-Count shows a data point in the CountedRequests metric.Note: CloudWatch reports the CountedRequests metric only when there is a nonzero value. To view the data, switch the statistic to Sum on the Graphed metrics tab in the CloudWatch console.Because Geo-Count is a custom rule with the Count action, AWS WAF counts the request.Then, AWS WAF evaluates the request against the web ACL default action. Since the web ACL default action is set to Allow, the request is allowed.A data point for this action appears in the AllowedRequests metric for the web ACL.For more information on metrics and dimensions included under the AWS WAF namespace, see AWS WAF metrics and dimensions.When you update the action of a custom rule to Count, the rule is considered as a non-terminating rule from a logging perspective. So, if the request matches a custom rule in count mode, the following fields are logged:nonTerminatingMatchingRules: The list of non-terminating rules that match the request.ruleId: The ID of the rule that matched the request and was non-terminating.action: This is always COUNT.ruleMatchDetails: Detailed information about the rule that matched the request. This field is only populated for SQL injection (SQLi) and cross-site scripting (XSS) match rule statements.If the Geo-Count rule matches the request, the logs show the nonTerminatingMatchingRules details as shown in the following example:"nonTerminatingMatchingRules": [{  "ruleId": "Geo-Count",  "action": "COUNT",  "ruleMatchDetails": [  ]}]For more information on all possible log fields, see  Log fields.To turn on logging for a web ACL, see Managing logging for a web ACL.Set the rule action for individual rules to count mode in an AWS Managed Rules rule groupYou can override the actions of the rules inside a rule group, setting them to count for some or all of the rules. For rule actions configured inside the rule group to something other than count, the override changes that action so that only matching requests are counted.If a web request matches the rule, AWS WAF processes the match as a count and then continues evaluating the subsequent rules in the rule group.For more information on how to set the rule action to count in a rule group, see Setting rule actions to count in a rule group.ExampleYou have a web ACL (Test2-ACL) that includes a AWS Managed Rules rule group (AWSManagedRulesKnownBadInputsRuleSet). There are no other rules in this web ACL and the web ACL default action is configured with an Allow action.You update the Log4JRCE rule inside AWSManagedRulesKnownBadInputsRuleSet rule group to a Count action. Now, if a component (such as a URI, query string, and so on) in the client request has a Log4j vulnerability, then that request is counted by the Log4JRCE rule. If the request doesn't match any other rules in the rule group, then it's allowed by the web ACL default action.The following is the JSON representation of this rule:{  "Name": "AWS-AWSManagedRulesKnownBadInputsRuleSet",  "Priority": 0,  "Statement": {    "ManagedRuleGroupStatement": {      "VendorName": "AWS",      "Name": "AWSManagedRulesKnownBadInputsRuleSet",      "ExcludedRules": [        {          "Name": "Log4JRCE"        }      ]    }  },  "OverrideAction": {    "None": {}  },  "VisibilityConfig": {    "SampledRequestsEnabled": true,    "CloudWatchMetricsEnabled": true,    "MetricName": "AWS-AWSManagedRulesKnownBadInputsRuleSet"  }}If you set the rule action to count for rules inside the rule group, these rules are considered excluded rules in the rule group. If you have metrics turned on, you receive COUNT metrics for each excluded rule.If the request matches the Log4JRCE rule, the following occurs:The request is counted.The CloudWatch metric for the Log4JRCE rule shows a data point in the CountedRequests metric.If no other rule inside the AWSManagedRulesKnownBadInputsRuleSet matches the request, the following occurs:The request is allowed by the web ACL default action.A data point appears in the AllowedRequests metric for the web ACL.For excluded rules, the following fields are captured in AWS WAF logs:excludedRules: The list of rules in the rule group that you have excluded. The action for these rules is set to Count.exclusionType: A type that indicates that the excluded rule has the action Count.ruleId: The ID of the rule within the rule group that is excluded.If the Log4JRCE rule matches the request, the logs show the excludedRules details as shown in the following example:"ruleGroupList":[  {    "ruleGroupId":"AWS#AWSManagedRulesKnownBadInputsRuleSet",    "terminatingRule":null,    "nonTerminatingMatchingRules":[            ],    "excludedRules":[      {        "exclusionType":"EXCLUDED_AS_COUNT",        "ruleId":"Log4JRCE"      }    ]  }]If an individual rule inside a rule group is updated to the count rule action, then the following occurs:The logs for the request matching against this rule won't contain a Count action for the rule under the nonTerminatingMatchingRules field.The AWS WAF logs show this rule under the excludedRules field.Override an entire rule group's action to countFor information on how to override a rule group's action to count, see Overriding the resulting rule groups action to count.For information on how to override a rule group's action to count, see Overriding a rule group's action to count.ExampleYou have a web ACL (Test3-ACL) that includes an AWS Managed Rules rule group (AWSManagedRulesKnownBadInputsRuleSet). There are no other rules in this web ACL and the web ACL default action is Allow. All of the rules in the AWSManagedRulesKnownBadInputsRuleSet rule group are using the default action defined in the rule. This means that the rules inside this rule group aren't set to Count. However, the Override rule group action to count option is turned on.The following is the JSON representation of this rule:{  "Name": "AWS-AWSManagedRulesKnownBadInputsRuleSet",  "Priority": 0,  "Statement": {    "ManagedRuleGroupStatement": {      "VendorName": "AWS",      "Name": "AWSManagedRulesKnownBadInputsRuleSet"    }  },  "OverrideAction": {    "Count": {}  },  "VisibilityConfig": {    "SampledRequestsEnabled": true,    "CloudWatchMetricsEnabled": true,    "MetricName": "AWS-AWSManagedRulesKnownBadInputsRuleSet"  }}If a component (such as a URI, query string, and so on) in the client request has a Log4j vulnerability, the following occurs:The request is blocked by the Log4JRCE rule.Because Block is a terminating action, AWS WAF stops evaluating the rule group and returns the terminating action result to the web ACL.At this point, the Override rule group action to count takes effect. The rule group's terminating action is overridden to a count.AWS WAF then continues processing the rest of the rules in the web ACL.If a request matches the Log4JRCE rule (or any other rule) inside the AWSManagedRulesKnownBadInputsRuleSet rule group, the following occurs:CloudWatch shows a data point in the CountedRequests metric corresponding to the rule group name AWSManagedRulesKnownBadInputsRuleSet.AWS WAF continues processing the rest of the rules in the web ACL after the override takes place. So, the request is ultimately matched against the web ACL default action. This shows as a data point in the AllowedRequests metric for the web ACL.In the AWS WAF logs, the matching request contains a count action in the nonTerminatingMatchingRules field.Additionally, the ruleGroupList shows the rule inside the rule group that originally blocked the request before the terminating action was overridden to count.If the Log4JRCE rule matches the request, the logs show the ruleGroupList and nonTerminatingMatchingRules details as shown in the following example:"ruleGroupList": [{  "ruleGroupId": "AWS#AWSManagedRulesKnownBadInputsRuleSet",  "terminatingRule": {      "ruleId": "Log4JRCE",      "action": "BLOCK",      "ruleMatchDetails": null    },  "nonTerminatingMatchingRules": [  ],    "excludedRules": null  }],"rateBasedRuleList": [],"nonTerminatingMatchingRules": [{    "ruleId": "AWS-AWSManagedRulesKnownBadInputsRuleSet",    "action": "COUNT",    "ruleMatchDetails": [  ]}]Related informationBasic handling of the rule and rule group actions in a web ACLFollow"


Question: How do I create an IAM policy to control access to Amazon EC2 resources using tags?

I want to create an AWS Identity and Access Management (IAM) policy that controls access to Amazon Elastic Compute Cloud (Amazon EC2) instances using tags.

Answer: "I want to create an AWS Identity and Access Management (IAM) policy that controls access to Amazon Elastic Compute Cloud (Amazon EC2) instances using tags.Short descriptionControl access to smaller deployments of Amazon EC2 instances as follows:1.    Add a specific tag to the instances you want to grant the users or groups access to.2.    Create an IAM policy that grants access to any instances with the specific tag.3.    Attach the IAM policy to the users or groups that you want to access the instances.ResolutionAdd a tag to your group of EC2 instancesOpen the Amazon EC2 console, and then add tags to the group of EC2 instances that you want the users or groups to be able to access. If you don't already have a tag, then create a new tag.Note: Be sure to read and understand the tag restrictions before tagging your resources. Amazon EC2 tags are case-sensitive.Create an IAM policy that grants access to instances with the specific tagCreate an IAM policy that does the following:Allows control over the instances with the tag.Contains a conditional statement that allows access to Amazon EC2 resources if the value of the condition key ec2:ResourceTag/UserName matches the policy variable aws:username. The policy variable ${aws:username} is replaced with the friendly name of the current IAM user when the policy is evaluated by IAM.Allows access to the ec2:Describe* actions for Amazon EC2 resources.Explicitly denies access to the ec2:CreateTags and ec2:DeleteTags actions to prevent users from creating or deleting tags.Note: This prevents the user from taking control of an EC2 instance by adding the specific tag to it.The finished policy looks similar to the following:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Action": "ec2:*",      "Resource": "*",      "Condition": {        "StringEquals": {          "ec2:ResourceTag/UserName": "${aws:username}"        }      }    },    {      "Effect": "Allow",      "Action": "ec2:Describe*",      "Resource": "*"    },    {      "Effect": "Deny",      "Action": [        "ec2:CreateTags",        "ec2:DeleteTags"      ],      "Resource": "*"    }  ]}Note: This policy applies to Amazon EC2 instances that use the ec2:ResourceTag condition key. To restrict launching new Amazon EC2 instances using tags, see How can I use IAM policy tags to restrict how an EC2 instance or EBS volume can be created?Attach the IAM policy to the users or groups you want to access the instancesFinally, attach the IAM policy that you created to the users or groups you want to access the instances. You can attach the IAM policy using the AWS Management Console, AWS CLI, or AWS API.Related informationGranting required permissions for Amazon EC2 resourcesIAM policies for Amazon EC2Follow"


Question: How do I prevent ThrottlingException or Rate exceeded errors when using AWS Glue?

"When I make an AWS Glue API call, I receive a ThrottlingException error message indicating that my rate is exceeded."

Answer: "When I make an AWS Glue API call, I receive a ThrottlingException error message indicating that my rate is exceeded.Short descriptionWhen you make an AWS Glue API request, you might receive an error similar to one of the following:"Error Code: ThrottlingException; Request ID: 07512ede-123f-49c8-8933-1298742297b9; Proxy: null)" } }{ "taskSubmitFailedEventDetails": { "resourceType": "glue", "resource": "startJobRun.sync", "error": "Glue.AWSGlueException", "cause": "Rate exceeded (Service: AWSGlue; Status Code: 400;""TaskSubmitFailed": {"resourceType": "glue", "resource": "startJobRun.sync", "error": "Glue.AWSGlueException", "cause": "Rate exceeded (Service: AWSGlue; Status Code: 400; Error Code: ThrottlingException; Request ID: 70ea5ab7-2bb8-4a47-b957-799ef46c129e; Proxy: null)" }""botocore.exceptions.ClientError: An error occurred (ThrottlingException) when calling the GetCrawler operation (reached max retries: 4): Rate exceeded cause": "Rate exceeded (Service: AWSGlue; Status Code: 400; Error Code: ThrottlingException; Request ID: 70ea5ab7-2bb8-4a47-b957-799ef46c129e; Proxy: null)"AWS Glue API requests are throttled for AWS accounts on a per-Region basis to help service performance. For more information, see AWS Glue endpoints and quotas.ResolutionFollow these best practices to mitigate a ThrottlingException or rate exceeded error:Reduce the frequency of the API calls.Stagger the intervals between your API calls so that they don't all run at the same time.Use APIs that return more than one value. For example, GetPartitions supports a call of 1000 values with one API call. Similarly, GetTables supports a call of 100 parameters with one API call.Implement error retries and exponential backoff when you make API calls.Using the AWS CloudTrail console, check which and how many API calls are sent during a given period of time.If you've tried these troubleshooting steps but still receive rate exceeded errors, then request a service quota increase.Note: Before submitting a request to increase your quota, identify the API call that caused the error and the call rate.Related informationCommon errorsFollow"


Question: How do I troubleshoot BGP connection failure over AWS VPN or Direct Connect?

I want to debug BGP sessions that fail to establish a connection because of parameter mismatch over VPN or over AWS Direct Connect.

Answer: "I want to debug BGP sessions that fail to establish a connection because of parameter mismatch over VPN or over AWS Direct Connect.Short descriptionWhen configuring dynamic AWS Site-to-Site VPN and AWS Direct Connect connections, you might encounter connectivity issues related to the Border Gateway Protocol (BGP). To address this, you must first analyze the BGP logs and identify the specific point of failure within the BGP negotiation process.Note: For troubleshooting a BGP session that can't establish a connection or is in an Idle state over a VPN tunnel, see How do I troubleshoot BGP connection issues over VPN?For troubleshooting a BGP session that can't establish a connection or is in an Idle state over Direct Connect, see How can I troubleshoot BGP connection issues over Direct Connect?ResolutionMake sure you have a verified BGP configuration on the customer gateway and the pings between the BGP peer IPs are working. Then, collect the packet captures for traffic between the BGP peer IPs from the customer gateway device. Finally, analyze the data as follows for each BGP state.BGP statesIdleThis is the first state where BGP waits for a “start event.” The start event occurs when you configure a new BGP neighbor or when you reset an established BGP peering. Next, BGP initializes some resources, resets a ConnectRetry timer and initiates a TCP connection to the remote BGP neighbor.ConnectDuring this stage, BGP waits for the TCP three-way handshake to complete. If this stage completes successfully, the connection moves to the OpenSent state.2021-07-04 22:50:20 169.254.60.146 169.254.60.145 TCP 74 34516 → 179 [SYN] Seq=0 Win=2920 Len=0 MSS=1460 SACK_PERM TSval=3030456 TSecr=0 WS=12021-07-04 22:50:20.719228 169.254.60.145 169.254.60.146 TCP 74179 → 34516 [SYN, ACK] Seq=0 Ack=1 Win=26844 Len=0 MSS=1375 TSval=64921081 TSecr=3030456 WS=1282021-07-04 22:50:20.719453 169.254.60.146 169.254.60.145 TCP 6634516 → 179 [ACK] Seq=1 Ack=1 Win=2920 Len=0 TSval=3030476 TSecr=64921081If the connection or ConnectRetry fails, it remains in the Active state and doesn't go to OpenSent state.Check the Connect logs to identify the cause of failure:In the case of dynamic VPN, ensure that there is TCP connectivity between the two BGP neighbors by performing a "telnet" test on TCP port 179If there is no TCP connectivity, check the logs to see if there were any errors or dropped packets during the TCP connection.Check that the neighbor's IP address is correctly configured on both BGP, customer gateway, and AWS.If using the Direct Connect's virtual interface, check if you've entered the correct BGP authentication (MD5 password) on the routers.OpenSentAfter sending an OPEN message to the peer, BGP waits in this state for the OPEN reply. If it receives a successful reply, the BGP state moves to OpenConfirm and sends a KEEPALIVE to the peer. A failed connection puts BGP back into the Idle or Active state.Border Gateway Protocol - OPEN MessageType: OPEN Message (1)  Version: 4   My AS: 65000  Hold Time: 30  BGP Identifier: 54.241.242.80In the OpenSent message, the peer sends its BGP parameters, such as a version number, AS number, Hold Timer (default value: 30 seconds for BGP over VPN and 90 seconds for BGP over Direct Connect) and BGP Identifier IP address. If BGP fails to establish, check the logs to ensure that the OPEN message was sent and received correctly by the neighbor with the BGP parameters. Also, check the OpenSent logs to identify the cause of failure.Note: AWS doesn't accept 0 as a Hold Time value.OpenConfirmThe BGP state is one step away from reaching its final state (Established). BGP waits in this state for KEEPALIVEs from the peer. If successful, the state moves to Established. Otherwise, the state moves back to Idle or Active state, based on the errors.652021-07-04 22:50:20.744297169.254.60.146169.254.60.145BGP85KEEPALIVE Message662021-07-04 22:50:20.765323169.254.60.145169.254.60.146BGP85KEEPALIVE MessageCheck the logs to ensure that the KEEPALIVE message was sent and received correctly.EstablishedIn this state, BGP exchanges information between the peers. The information consists of updates (route advertising ), KEEPALIVES, or notification.Border Gateway Protocol - UPDATE MessagePath attributesPath Attribute - AS_PATH: 65000 Path Attribute - NEXT_HOP: 169.254.60.146 Network Layer Reachability Information (NLRI)   192.168.0.0/16 If a connection isn't established, do the following:Check the logs to ensure that the routers are exchanging updates correctly. Verify that the advertised prefixes match the expected routes.Check if any BGP filters or prefix lists are preventing the propagation of routes within the route table.Check the advertised route entries on peer route tables.You might see the BGP session going from an Established to an Idle state for VPN or Direct Connect Virtual Interface on a virtual gateway. Verify that the peer is advertising less than 100 routes over the BGP session.Follow"


Question: How can I create notifications for when an Amazon EMR cluster or step changes state?

I want to get a notification when my Amazon EMR cluster or step changes state. How can I accomplish this with Amazon EventBridge?

Answer: "I want to get a notification when my Amazon EMR cluster or step changes state. How can I accomplish this with Amazon EventBridge?Short descriptionWhen you create an EventBridge rule, you choose a target, such as an Amazon Simple Notification Service (Amazon SNS) topic. The target triggers when a specified event occurs, such as when a cluster state changes.ResolutionTo create an EventBridge rule that sends an SNS message when an Amazon EMR cluster or step changes state, do the following:Create an SNS topic. This topic is the target for the EventBridge rule.Follow the steps for Creating Amazon EventBridge rules that react to events.For Define pattern, choose Event pattern.Choose Pre-defined pattern by service.For Service provider, choose AWS.For Service Name, choose EMR.For Event type, choose State Change.For Target, select SNS topic.For Topic, select the topic that you created in step 1.Note: You can choose more than one target per rule. For example, you can perform an AWS Lambda function, send an SNS notification, and send a message to an Amazon Simple Queue Service (Amazon SQS) queue at the same time, in the same rule.If the EventBridge rule doesn't behave as expected, see Troubleshooting Amazon EventBridge.Related informationAmazon EMR eventsFollow"


Question: How can I use the AWS CLI to register a Lambda function as a target behind my Application Load Balancer?

I want to register a Lambda function as a target behind my Application Load Balancer using the AWS Command Line Interface (AWS CLI). How can I do this?

Answer: "I want to register a Lambda function as a target behind my Application Load Balancer using the AWS Command Line Interface (AWS CLI). How can I do this?ResolutionNote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.1.    Create a target group with the target type set to Lambda. Be sure to replace [your target group name] with the name of your target.aws elbv2 create-target-group \    --name [your target group's name] \    --target-type lambda2.    Use the add-permission command to allow Elastic Load Balancing to invoke your Lambda function. Be sure to replace [your Lambda function's name] and [your target group's Amazon Resource Name (ARN)] with your respective names.aws lambda add-permission \    --function-name [your Lambda function's name] \    --statement-id load-balancer \    --principal elasticloadbalancing.amazonaws.com \    --action lambda:InvokeFunction \    --source-arn [your target group's Amazon Resource Name (ARN)]3.    Use the register-targets command to register Lambda as the target. Be sure to replace [your target group's ARN] and **[your Lambda function's ARN]**with your ARNs.aws elbv2 register-targets \     --target-group-arn [your target group's ARN] \     --targets Id=[your Lambda function's ARN]Follow"