Question: How do I export an ElastiCache for Redis backup to S3?

How do I export my Amazon ElastiCache for Redis backup to Amazon Simple Storage Service (Amazon S3)?

Answer: "How do I export my Amazon ElastiCache for Redis backup to Amazon Simple Storage Service (Amazon S3)?Short descriptionTo copy an ElastiCache snapshot (rdb file) to an S3 bucket, first make sure that the bucket is in the same Region as the ElastiCache cluster. Then, you need to set a bucket policy and Identity and Access Management (IAM) permissions so that ElastiCache and IAM users can access the bucket. Amazon ElastiCache requires permission in the bucket policy to access the S3 bucket and copy the .rdb file. Additionally, the users' IAM permissions must be set to access the object (.rdb file) in the S3 bucket.PrerequisitesComplete the following steps before you export your backup:Create or identify an Amazon S3 bucket within the same Region as ElastiCache cluster.Grant Amazon ElastiCache service access to S3.Allow the IAM user permission to copy and paste to the bucket.ResolutionComplete the prerequisitesStep 1: Create or identify an Amazon S3 bucket to export to within the same Region as the ElastiCache cluster.Step 2: Grant Amazon ElastiCache access to the S3 bucket.1.    Open the Amazon S3 console.2.    Choose the name of the Amazon S3 bucket that you want to copy the backup to.3.    Choose the Permissions tab.4.    Scroll down on the Permissions page, select Access control list (ACL), and then select Edit.Add grantee Canonical Id: 540804c33a284a299d2547575ce1010f2312ef3da9b3a053c8bc45bf233e43535.    Select the following options listed:Objects: List, WriteBucket ACL: Read, Write5.    Choose Save.Note: GovCloud Regions use a different ID:Canonical Id: 40fa568277ad703bd160f66ae4f83fc9dfdfd06c2f1b5060ca22442ac3ef8be6Step 3: Make sure that the user responsible for exporting the backup has the correct permissions to interact with the bucket.The following is an example policy that provides the correct permissions:{  "Version": "2012-10-17",  "Statement": [{    "Effect": "Allow",    "Action": [      "s3:GetBucketLocation",      "s3:ListAllMyBuckets",      "s3:PutObject",      "s3:GetObject",      "s3:DeleteObject",      "s3:ListBucket"    ],    "Resource": "arn:aws:s3:::*"  }]}After you complete these preceding prerequisites, follow the steps in the following Export an ElastiCache for Redis backup to S3 section.Export an ElastiCache for Redis backup to S31.    Open the ElastiCache console.2.    In the left navigation pane select Backups.3.    From the list of backups, choose the box to the left of the name of the backup that you want to export.4.    Choose Copy.5.    Enter a name for your backup in New backup name. Amazon ElastiCache adds an instance identifier and .rdb to the value that you enter here. For example, if you enter my-exported-backup, Amazon ElastiCache creates my-exported-backup-0001.rdb.6.    From the Target S3 location list, choose the name of the Amazon S3 bucket that you want to export your backup to.7.    Choose Copy.Related informationExporting a backupSeeding a new cluster with an externally created backupFollow"


Question: "For Amazon S3 request rates, what's the difference between prefixes and nested folders? How many prefixes can I have in an S3 bucket?"

"For Amazon Simple Storage Service (Amazon S3) request rates, what's the difference between key prefixes and nested folders? How many prefixes can I have in an S3 bucket?"

Answer: "For Amazon Simple Storage Service (Amazon S3) request rates, what's the difference between key prefixes and nested folders? How many prefixes can I have in an S3 bucket?ResolutionPrefixesA key prefix is a string of characters that can be the complete path in front of the object name (including the bucket name). For example, if an object (123.txt) is stored as BucketName/Project/WordFiles/123.txt, the prefix might be “BucketName/Project/WordFiles/123.txt”. The prefix can be any length, including the entire object key name.If the 123.txt file is saved in a bucket without a specified path, Amazon S3 automatically adjusts the prefix value according to the request rate. Partitions can be automatically formed at any point in the prefix string.A partitioned prefix in a bucket can support 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second. There is no limit to the number of prefixes that you can have in a bucket. However, be aware that a spike in the request rate can cause throttling.Note: In Amazon S3, there are no partitions for keys or objects. Partitions exist only at the prefix level, and not at the object level. For more information about using prefixes in Amazon S3, see Organizing objects using prefixes.FoldersIn Amazon S3, folders are used to group objects and organize files. Unlike a traditional file system, Amazon S3 doesn't use hierarchy to organize its objects and files. Amazon S3 console supports the folder concept only as a means of grouping (and displaying) objects.More specifically, a folder is the value between the two "/" characters. For example, if a file is stored as BucketName/Project/WordFiles/123.txt, the file path indicates that there is a folder ("Project") and subfolder ("WordFiles"). Both "Project" and "WordFiles" are considered to be folders. If the 123.txt file is saved in a bucket without a specified path, then no folders are used to store the file.Note: The folder structure might not indicate any partitioned prefixes that support request rates.Difference between prefixes and foldersThe difference between a prefix and a folder is the significance of the "/" character. For folders, the "/" character signifies a subfolder or object name. For prefixes, "/" is just another character. The "/" does not indicate a partition placement.Note: The folder structure only applies to the Amazon S3 console. For more information, see Organizing objects in the Amazon S3 console using folders.Related informationAWS re:Invent 2018: Best practices for Amazon S3 and Amazon S3 GlacierOrganizing, listing, and working with your objectsFollow"


Question: How do I create alias resource record sets in Route 53 using the AWS CLI?

How do I create alias resource record sets in Amazon Route 53 using the AWS Command Line Interface (AWS CLI)?

Answer: "How do I create alias resource record sets in Amazon Route 53 using the AWS Command Line Interface (AWS CLI)?Short descriptionYou can use an alias record in Route 53 to point to an AWS resource or an Amazon Simple Storage Service (Amazon S3) bucket. You can create an alias record through the Route 53 console or through the AWS CLI.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Before you create alias resource record sets, create a hosted zone to contain the records for routing traffic to your domain. The hosted zone and your domain must have the same name. Depending on your use case, there are two choices. You can create a public hosted zone for routing Internet traffic. Or, you can also create a private hosted zone for routing traffic within an Amazon Virtual Private Cloud (Amazon VPC).Create your alias resource record setModify the following sample JSON syntax to create your own alias resource record set, and then specify your own values for the alias records. Save the file (for example, sample.json).Warning: Be sure that you use the hosted zone ID of your AWS resources, and not your domain name, in your configuration. In the JSON file, this value is specified in the key-value pair HostedZoneId. Before proceeding, locate the HostedZoneId for Elastic Load Balancing, AWS Elastic Beanstalk, Amazon S3, and Amazon CloudFront endpoints for each Region.This example creates the alias resource record set for a domain ( elb.example.com) to point to a load balancer endpoint ( ALB-xxxxxxxx.us-west-2.elb.amazonaws.com). The request body includes a list of change items, known as a change batch. Change items are included with a ChangeResourceRecordSetsRequest element.{  "Comment": "Creating Alias resource record sets in Route 53",  "Changes": [    {      "Action": "CREATE",      "ResourceRecordSet": {        "Name": "elb.example.com",        "Type": "A",        "AliasTarget": {          "HostedZoneId": "Z1H1FL5HABSF5",          "DNSName": "ALB-xxxxxxxx.us-west-2.elb.amazonaws.com",          "EvaluateTargetHealth": false        }      }    }  ]}Next, use a change-resource-record-sets command to create your resource record set in a hosted zone. The values for record creation are specified in the JSON configuration file that you created previously.Modify the following command for your configuration. For the --hosted-zone-id, provide your hosted zone ID for the domain name:$ aws route53 change-resource-record-sets --hosted-zone-id ZXXXXXXXXXX --change-batch file://sample.jsonElastic Load Balancing settingsIf you're pointing to a load balancer, always include dualstack in the value for the DNSName key-value pair of the JSON file. For example, if your load balancer's Amazon-provided DNS name is ALB-xxxxxxxx.us-west-2.elb.amazonaws.com, use:"DNSName": "dualstack.ALB-xxxxxxxx.us-west-2.elb.amazonaws.com"CloudFront distribution settingsIf you're pointing to a CloudFront distribution, specify the Amazon-provided CloudFront distribution domain name in the DNSName key-value pair of the JSON file. For example, if your CloudFront distribution Amazon-provided domain name is d111111abcdef8.cloudfront.net, use:"DNSName": "d111111abcdef8.cloudfront.net"Warning: You must include an alternate domain name to use instead of the CloudFront-assigned one. The alternate domain name must match the domain for which you created the alias resource record set. For example, if you want to access your CloudFront distribution using the domain example.com, add the domain to the Alternate Domain Names for the distribution.Amazon S3 bucket settingsIf pointing to an S3 bucket, specify the domain name of the bucket's website endpoint for the DNSName key-value pair in the JSON file. Don't use the S3 endpoint that appears in the static website hosting from your S3 management console for the value. The domain name for which you create the resource record set must match the name of your Amazon S3 bucket.You must also specify the Region where the bucket is hosted (for example, us-east-1).s3-website-us-east-1.amazonaws.comThe following is example JSON syntax for pointing to an S3 bucket:{  "Comment": "Alias record for S3 bucket",  "Changes": [    {      "Action": "CREATE",      "ResourceRecordSet": {        "Name": "new.example.com",        "Type": "A",        "AliasTarget": {          "HostedZoneId": "Z3BJ6K6RIION7M",          "DNSName": "s3-website-us-west-2.amazonaws.com",          "EvaluateTargetHealth": false        }      }    }  ]}Note: In addition to creating Alias records pointing to the AWS resources such as CloudFront, Amazon S3 or ELB, you can create an Alias resource record. The Alias resource record points to another record in the same hosted zone that you’re creating the alias record in. The alias record must have the same type as the record you're routing to.Related informationHow do I troubleshoot errors with creating Amazon Route 53 resource record sets using the AWS CLI?Follow"


Question: How do I connect to my Amazon RDS for MySQL instance by using MySQL Workbench?

I want to connect to my Amazon Relational Database Service (Amazon RDS) DB instance that's running MySQL using MySQL Workbench.

Answer: "I want to connect to my Amazon Relational Database Service (Amazon RDS) DB instance that's running MySQL using MySQL Workbench.ResolutionFollow the steps below to connect MySQL Workbench to your Amazon RDS DB instance:Download and install MySQL Workbench.Open MySQL Workbench, and then choose the ⊕ sign beside MySQL Connections to set up a new connection.In the Setup New Connection dialog box, enter a name for your connection.In the Parameters section, enter these details:Host name: Enter the RDS endpoint.Port: Enter the Port number.Username: Enter the primary user.Note: You can get these details from the Amazon RDS console. From the Databases section, choose Instances, and then select the instance that you are connecting to. From the Connectivity and Security tab, choose Configuration. The primary user is listed here.Choose Test Connection.In the pop-up window that appears, enter the password that you configured when you created the DB instance, and then choose OK.After testing your connection, from the Setup new connection dialog box, choose OK to save the connection.If you receive an error when you test the connection, check that you configured the network correctly.Related informationHow can I troubleshoot connectivity to an Amazon RDS DB instance that uses a public or private subnet of a VPC?Scenarios for accessing a DB instance in a VPCControlling access with security groupsFollow"


Question: How does Amazon DynamoDB handle spiky loads in short intervals?

I want to know how Amazon DynamoDB handles spiky workloads in short intervals.

Answer: "I want to know how Amazon DynamoDB handles spiky workloads in short intervals.ResolutionWhen you use provisioned mode, DynamoDB provides a degree of flexibility in your per-partition throughput provisioning by providing burst capacity. DynamoDB currently retains up to 5 minutes (300 seconds) of unused read and write capacity. Suppose that a DynamoDB table has its capacity provisioned at 150 RCUs and has no usage in the last 5 minutes. For this configuration, the table has 45,000 read units of burst capacity (150 * 300). Therefore, if a sudden spiky load pushes consumption to 200 read units, then DynamoDB can leverage burst capacity to accommodate 200 RCUs for 15 minutes.Example: 45,000 / (200 - 150) = 900 seconds = 15 minutesHowever, consider the following:Burst capacity is provided on a best effort’s basis and isn't guaranteed. Therefore, don't consider the burst capacity when you configure a table's provisioned capacity.DynamoDB might consume the burst capacity for background maintenance and other tasks without prior notice.Burst capacity can't alleviate partition-level throttling. A partition key that doesn't have a high cardinality can result in many requests that target just a few partitions. If a resulting hot partition exceeds the per partition limits of 3,000 RCU or 1,000 WCU per second, or a combination of both, your table throttles.For more information, see Using burst capacity effectively. In on-demand capacity mode, DynamoDB automatically adapts to your application's traffic volume. On-demand capacity mode instantly accommodates up to double the previous peak traffic on a table. Suppose that your application's traffic pattern varies between 25,000 and 50,000 consistent reads per second and 50,000 reads per second is the previous traffic peak. In this case, the on-demand capacity mode instantly accommodates sustained traffic of up to 100,000 reads per second. If your application sustains the traffic of 100,000 reads per second, then that peak becomes your new previous peak. With this new peak, subsequent traffic might reach up to 200,000 reads per second.However, if your spikes are nearly immediate and more than double the previous peak, your table might throttle. For more information, see Read/write capacity mode.Follow"


Question: How do I troubleshoot an OIDC provider and IRSA in Amazon EKS?

My pods can't use the AWS Identity and Access Management (IAM) role permissions with the Amazon Elastic Kubernetes Service (Amazon EKS) account token.

Answer: "My pods can't use the AWS Identity and Access Management (IAM) role permissions with the Amazon Elastic Kubernetes Service (Amazon EKS) account token.Short descriptionTo troubleshoot issues with the OpenID Connect (OIDC) provider and IAM roles for service accounts (IRSA) in Amazon EKS, complete the steps in one of the following sections:Check if you have an existing IAM OIDC provider for your clusterCheck if your IAM role has an attached needful IAM policy with required permissionsVerify that the IAM role trust relations are correctly setCheck if you created a service accountVerify that the service account has the correct IAM role annotationsVerify that you correctly specified the serviceAccountName in your podCheck the environment variables and permissionsVerify that the application uses a supported AWS SDKCheck the pod user and groupRecreate podsVerify that the audience is correctVerify that you configured the correct thumbprintFor the AWS China Region, check the AWS_DEFAULT_REGION environment variableResolutionCheck if you have an existing IAM OIDC provider for your clusterIf a provider already exists, then you receive an error that's similar to the following message:"WebIdentityErr: failed to retrieve credentials\ncaused by: InvalidIdentityToken: No OpenIDConnect provider found in your account for https://oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E\n\tstatus code: 400"1.    Check your cluster's OIDC provider URL:$ aws eks describe-cluster --name cluster_name --query "cluster.identity.oidc.issuer" --output textSee the following example output:https://oidc.eks.us-west-2.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E2.    List the IAM OIDC providers in your account. Replace EXAMPLED539D4633E53DE1B716D3041E (include < >) with the value returned from the previous command:aws iam list-open-id-connect-providers | grep EXAMPLED539D4633E53DE1B716D3041ESee the following example output:"Arn": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E"If the preceding command returns an output, then you already have a provider for your cluster. If the command doesn't return an output, then you must create an IAM OIDC provider.Check if your IAM role has an attached needful IAM policy with required permissions1.    Open the IAM console.2.    In the navigation pane, choose Roles.3.    Choose the role that you want to verify.4.    Under the Permissions tab, verify if this role has the required policy attached.Verify that the IAM role trust relations are correctly setWith the AWS Management Console:1.    Open the IAM console.2.    In the navigation pane, choose Roles.3.    Choose the role that you want to check.4.    Choose the Trust Relationships tab to verify that the format of your policy matches the format of the following JSON policy:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E"      },      "Action": "sts:AssumeRoleWithWebIdentity",      "Condition": {        "StringEquals": {          "oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E:sub": "system:serviceaccount:SERVICE_ACCOUNT_NAMESPACE:SERVICE_ACCOUNT_NAME",          "oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E:aud": "sts.amazonaws.com"        }      }    }  ]}To verify trust relations, run following command with your role name in the AWS Command Line Interface (AWS CLI):$ aws iam get-role --role-name EKS-IRSANote: Replace EKS-IRSA with your IAM role name.In the output JSON, look for the AssumeRolePolicyDocument section.See the following example output:{  "Role": {    "Path": "/",    "RoleName": "EKS-IRSA",    "RoleId": "AROAQ55NEXAMPLELOEISVX",    "Arn": "arn:aws:iam::ACCOUNT_ID:role/EKS-IRSA",    "CreateDate": "2021-04-22T06:39:21+00:00",    "AssumeRolePolicyDocument": {      "Version": "2012-10-17",      "Statement": [        {          "Effect": "Allow",          "Principal": {            "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E"          },          "Action": "sts:AssumeRoleWithWebIdentity",          "Condition": {            "StringEquals": {              "oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E:aud": "sts.amazonaws.com",              "oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E:sub": "system:serviceaccount:SERVICE_ACCOUNT_NAMESPACE:SERVICE_ACCOUNT_NAME"            }          }        }      ]    },    "MaxSessionDuration": 3600,    "RoleLastUsed": {      "LastUsedDate": "2021-04-22T07:01:15+00:00",      "Region": "AWS_REGION"    }  }}Note: Check that you have specified the correct AWS Region, Kubernetes service account name, and Kubernetes namespace.Check if you created a service accountUse the following command:$ kubectl get sa -n YOUR_NAMESPACENote: Replace YOUR_NAMESPACE with your Kubernetes namespace.See the following example output:NAME      SECRETS   AGEdefault   1         28dirsa      1         66mIf you don't have a service account, see Configure service accounts for pods (from the Kubernetes website).Verify that the service account has the correct IAM role annotationsUse the following command:$ kubectl describe sa irsa -n YOUR_NAMESPACENote: Replace irsa with your Kubernetes service account name. Replace YOUR_NAMESPACE with your Kubernetes namespace.See the following example output:Name:                irsaNamespace:           defaultLabels:              noneAnnotations:         eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/IAM_ROLE_NAMEImage pull secrets:  noneMountable secrets:   irsa-token-v5rtcTokens:              irsa-token-v5rtcEvents:              noneVerify that you correctly specified the serviceAccountName in your podUse the following command:$ kubectl get pod POD_NAME  -o yaml -n YOUR_NAMESPACE| grep -i serviceAccountName:Note: Replace POD_NAME and YOUR_NAMESPACE with your Kubernetes pod and namespace.See the following example output:serviceAccountName: irsaCheck the environment variables and permissionsLook for AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE in the pod's environment variables:$ kubectl -n YOUR_NAMESPACE exec -it POD_NAME -- env | grep AWSSee the following example output:AWS_REGION=ap-southeast-2AWS_ROLE_ARN=arn:aws:iam::111122223333:role/EKS-IRSAAWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/tokenAWS_DEFAULT_REGION=ap-southeast-2Verify that the application uses a supported AWS SDKThe SDK version must be greater than or equal to the following values:Java (Version 2) — 2.10.11Java — 1.11.704Go — 1.23.13Python (Boto3) — 1.9.220Python (botocore) — 1.12.200AWS CLI — 1.16.232Node — 3.15.0Ruby — 2.11.345C++ — 1.7.174.NET — 3.3.659.1PHP — 3.110.7To check for latest supported SDK version, see Using a supported AWS SDK.Check the pod user and groupUse the following command:$ kubectl exec -it POD_NAME -- iduid=0(root) gid=0(root) groups=0(root)Note: By default, only containers that run as root have the proper file system permissions to read the web identity token file.If your containers aren't running as root, then you can receive the following errors:Error: PermissionError: [Errno 13] Permission denied: '/var/run/secrets/eks.amazonaws.com/serviceaccount/token-or-WebIdentityErr: failed fetching WebIdentity token: \ncaused by: WebIdentityErr: unable to read file at /var/run/secrets/eks.amazonaws.com/serviceaccount/token\ncaused by: open /var/run/secrets/eks.amazonaws.com/serviceaccount/token: permission deniedTo provide the proper file system permissions, make sure that your containers run as root. For clusters 1.18 or lower, provide the following security context for the containers in your manifest:apiVersion: apps/v1kind: Deploymentmetadata:  name: my-appspec:  template:    metadata:      labels:        app: my-app    spec:      serviceAccountName: my-app      containers:      - name: my-app        image: my-app:latest      securityContext:        fsGroup: 1337...Note: The fsGroup ID is arbitrary. You can choose any valid group ID. The preceding security context setting is not required for clusters 1.19 or later.Recreate podsIf you created pods before you applied IRSA, then recreate the pods.See the following example command:$ kubectl rollout restart deploy nginxSee the following example output:deployment.apps/nginx restartedFor daemonsets or statefulsets deployments, you can use the following command:$ kubectl rollout restart deploy DEPLOYMENT_NAMEIf you have created only one pod, then you must delete the pod and recreate it.See the following example command to delete the pod:$ kubectl delete pod POD_NAMESee the following example command to recreate the pod:$ kubectl apply -f SPEC_FILENote: Replace SPEC_FILE with your Kubernetes manifest file path and file name.Verify that the audience is correctIf you created the OIDC provider with the incorrect audience, then you receive the following error:Error - An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: Incorrect token audienceCheck the IAM identity provider for your cluster. Your ClientIDList is sts.amazonaws.com:$ aws iam get-open-id-connect-provider --open-id-connect-provider-arn arn:aws:iam::ACCOUNT_ID:oidc-provider/oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041ESee the following example output:{  "Url": "oidc.eks.AWS_REGION.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E",  "ClientIDList": [    "sts.amazonaws.com"  ],  "ThumbprintList": [    "9e99a48a9960b14926bb7f3b02e22da2b0ab7280"  ],  "CreateDate": "2021-01-21T04:29:09.788000+00:00",  "Tags": []}Verify that you configured the correct thumbprintIf the thumbprint that's configured in the IAM OIDC is not correct, you can receive the following error:failed to retrieve credentials caused by: InvalidIdentityToken: OpenIDConnect provider's HTTPS certificate doesn't match configured thumbprintTo automatically configure the correct thumbprint, use eksctl or the AWS Management Console to create the IAM identity provider. For other ways to obtain a thumbprint, see Obtaining the thumbprint for an OpenID Connect identity provider.For the AWS China Region, check the AWS_DEFAULT_REGION environment variableIf you use IRSA for a pod or daemonset that's deployed to a cluster in the AWS China Region, then set the AWS_DEFAULT_REGION environment variable in the pod specification. If you don't, the pod or daemonset can receive the following error:An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation: The security token included in the request is invalidUse the following example to add the AWS_DEFAULT_REGION environment variable to your pod or daemonset specification:apiVersion: apps/v1kind: Deploymentmetadata:  name: my-appspec:  template:    metadata:      labels:        app: my-app    spec:      serviceAccountName: my-app      containers:      - name: my-app        image: my-app:latest        env:        - name: AWS_DEFAULT_REGION          value: "AWS_REGION"...Follow"


Question: I can't connect to my Amazon Redshift cluster

I'm having trouble connecting to my Amazon Redshift cluster. How do I troubleshoot this?

Answer: "I'm having trouble connecting to my Amazon Redshift cluster. How do I troubleshoot this?Short descriptionTo connect to your Amazon Redshift cluster, check the following:If your Amazon Redshift cluster is in a public subnet and you want to access it over the internet, then set Publicly accessible to Yes.If your Amazon Redshift cluster is in a private subnet, then use a NAT gateway in the public subnet to access the internet. A cluster in a private subnet can connect to the internet using the NAT gateway, but the internet can't initiate a connection to the cluster.To enable your Amazon Redshift cluster to communicate with resources that are in a different virtual private cloud (VPC) account, create a VPC peering connection.ResolutionMy cluster is in a public subnet, but I can't connect to it over the internet from my local computerNote: Before completing the steps below, make sure that your VPC has an internet gateway attached to it.1.    Open the Amazon Redshift console.2.    Choose Clusters.3.    Select the cluster to which you are trying to connect.4.    Choose Actions.5.    Choose Modify publicly accessible setting.6.    For Allow instances and devices outside the VPC to connect to your database through the cluster endpoint, choose edit publicly accessible.7.    Choose Enable.8.    (Optional) Select your Elastic IP address.9.    Choose Confirm to save your changes.My cluster is in a private subnet, and I can't connect to it from my local computerYour cluster must be in a public subnet for it to connect to a local machine.To access a cluster from a local machine, perform the following steps:1.    Open the Amazon Redshift console.2.    Choose Clusters.3.    Select the cluster to which you are trying to connect.4.    Choose the Configuration tab to open the Cluster Properties page.5.    Choose View VPCs to open the Amazon VPC console.6.    In the navigation pane, choose Route Tables.7.    Select the route table that is associated with the subnet where your cluster resides.8.    Choose Edit.9.    Choose Add route.For IPv4 traffic, enter "0.0.0.0/0" in the Destination field, and then select the internet gateway ID for the Target field.For IPv6 traffic, enter "::/0" in the Destination field, and then select the internet gateway ID for the Target field.10.    Choose Save.My cluster can't be accessed by an Amazon Elastic Compute Cloud (Amazon EC2) instance that is in a different VPCCreate a VPC peering connection between the VPCs. A VPC peering connection allows two VPCs to communicate with each other using private IP addresses.1.    Follow the steps for Creating and accepting a VPC peering connection.Note: If the VPCs are in the same AWS account, make sure that the IPv4 CIDR blocks don't overlap.2.    Update both route tables.3.    Update your security groups to reference peer VPC groups.4.    On the EC2 instance, test the VPC peering connection using a networking utility (such as netcat). For example:nc -zv <hostname> <port>If the VPC peering connection is successful, then the output will look like this:$ nc -zv example.123456789.us-east-1.redshift.amazonaws.com 5439found 0 associationsfound 1 connections:     1:    flags=82<CONNECTED,PREFERRED>    outif en0    src xx.xxx.xxx.xx port 53396    dst xx.xxx.xxx.xxx port 5439    rank info not available    TCP aux info availableConnection to example.123456789.us-east-1.redshift.amazonaws.com port 5439 [tcp/*] succeeded!Related informationAuthorizing access to the clusterManaging clusters in an Amazon Virtual Private Cloud (Amazon VPC)Working with VPC peering connectionsWorking with route tablesFollow"


Question: How do I change the name on my AWS account?

How do I change my AWS account name?

Answer: "How do I change my AWS account name?Short descriptionYour AWS account name is the name that appears in the AWS Management Console next to your account ID. You choose this name when you first create the account, but you can update the name through the AWS Billing and Management console.Changing an account name is a task that requires root user credentials. Therefore, only the AWS account root user can modify the AWS account name.Note: The root user credentials are different than the account name. The root user logs in with the email and password that were used to first create the account. Unlike the account name, the root user identity can't be changed.To change the name of an AWS Identity and Access Management (IAM) user, see Renaming an IAM user.Resolution1.    Use your root user credentials to sign in to the AWS account that you want to modify.2.    Choose your name from the navigation bar, and then select Account.3.    Next to Account Settings, choose Edit.            Note: If you aren't signed in as the root user, then the Edit option isn't available.4.    Enter the new AWS account name, and then choose Update.Note: If your AWS Organizations has service control policies (SCPs) that restrict access to aws-portal, then this method doesn't work. Instead of choosing Account in the AWS Management Console, choose Security Credentials. Additionally, make sure that the iam:UpdateAccountName action isn't restricted by your SCPs.The updated account name can take up to four hours to appear in the AWS Management Console.Related informationAfter I use AWS Organizations to create a member account, how do I access that account?Permissions required to access IAM resourcesIAM identities (users, user groups, and roles)Follow"


Question: How can I modify the storage capacity of my Storage Gateway?

"I want to modify the storage capacity for my gateway on AWS Storage Gateway. How can I remove, resize, or allocate disks to my gateway?"

Answer: "I want to modify the storage capacity for my gateway on AWS Storage Gateway. How can I remove, resize, or allocate disks to my gateway?Short descriptionYou can modify the storage capacity on your gateway by adding, allocating, or removing cache disks.Note: These steps apply only to Volume Gateway or Tape Gateway. For managing FSx File Gateway or S3 File Gateway, follow the steps on maintaining a file gateway.ResolutionRemoving or decreasing storage capacityDon't remove disks that are allocated as cache disks for an existing gateway—doing this can break your gateway's functionality. You can't decrease the size of a cache disk after it's allocated to an existing gateway. Instead, you must create a new gateway with the cache space that you need. Then, you can migrate your data to the new gateway. Use the Storage Gateway Console to migrate data on Volume Gateway. Use the Virtual Tape Library (VTL) interface to migrate data on Tape Gateway.For disks that are allocated as upload buffer for an existing gateway, you must first shut down the gateway before removing the disk. Be sure to follow the proper steps for shutting down a gateway. If you directly shut down the instance without following these steps, it can prevent access to your gateway for days. After the gateway is shut down, you can decrease the disk size or remove it. Then, you can allocate a new disk with the reduced upload buffer size.Note: When decreasing disk size or removing disks, your gateway must still have at least one buffer disk with a minimum of 150 GB.Increasing storage capacityDon't expand disks that you previously allocated as upload buffer or cache storage for your gateway. To increase the storage capacity of your gateway, add new disks to the gateway instead.Before you add more storage to your gateway, review the minimum recommended capacity for your use case:Upload buffer: It's a best practice to allocate at least 150 GiB of upload buffer. Use the following formula to determine the size that you should allocate:(Application throughput (MB/s) - Network throughput to AWS (MB/s) * Compression factor) * Duration of writes (s) = Upload buffer (MB)Note: 1 GiB equals 1073.741824 MB. Even if this formula returns less than 150 GiB for your use case, it's still a best practice to allocate at least 150 GiB.Cache storage: For tape and volume gateways, it's a best practice to allocate cache storage that's at least 1.1 times the upload buffer size. Because file gateways don't have an upload buffer in their architecture, it's a best practice to allocate at least 150 GiB of cache storage.After you determine the recommended storage capacity for your use case, use the Storage Gateway console to allocate the appropriate local disk capacity.Follow"


Question: How do I resolve the "This template does not include any resources to import" error in AWS CloudFormation?

I receive the following error in the AWS CloudFormation console: "This template does not include any resources to import. Learn more."

Answer: "I receive the following error in the AWS CloudFormation console: "This template does not include any resources to import. Learn more."Short descriptionYou receive this error when you use the AWS CloudFormation console to import resources into an existing stack that was created outside of CloudFormation.This error can occur when you use the AWS CloudFormation console in scenarios with any of the following:Conditional resources. The resource that you're importing has a condition key associated with a condition that evaluates to false.AWS Serverless Application Model (AWS SAM) templates. The AWS CloudFormation console doesn't support the Transforms section when importing resources. You can't import a resource with a template that uses the AWS::Serverless transform.Fn::Transform. The AWS CloudFormation console doesn't support the use of the intrinsic function Fn::Transform when importing resources.You can use the AWS Command Line Interface (AWS CLI) instead of the AWS CloudFormation console to resolve this error for templates using:AWS SAMFn::TransformTo resolve this error for conditional resources, make sure that the condition specified under the condition key evaluates to true for the resource being imported.The AWS CLI requires you to explicitly provide imported resources using the CloudFormation command, create-change-set.Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.ResolutionIn the following example, the AWS CLI is used to import an existing AWS::ECS::Cluster resource into a CloudFormation stack:Resources:  ...  ECSCluster2:    Condition: MyCondition    Type: AWS::ECS::Cluster    DeletionPolicy: Retain    Properties:      ClusterName: Cluster2Note: Before you proceed to the next steps, make sure that the condition MyCondition evaluates to true.To import the resource using the AWS CLI, complete the following steps:Note: If your stack isn't in your default AWS Region, then add --region to your commands, or change the default Region by setting and exporting the AWS_DEFAULT_REGION environment variable.1.    Create a resource import file called import.txt. For example:[    {        "ResourceType": "AWS::ECS::Cluster",        "LogicalResourceId":            "ECSCluster2"        ,        "ResourceIdentifier": {            "ClusterName":"Cluster2"        }    }]2.    To create a change set against your stack, run the following create-change-set command:ID=$(aws cloudformation create-change-set --stack-name testStack --change-set-name testSet --resources-to-import file://import.txt --change-set-type IMPORT --template-body file://template.yaml --capabilities CAPABILITY_AUTO_EXPAND  --query 'Id' --output text)Note: Replace testStack with your stack name and template.yaml with your CloudFormation template file name. The preceding command returns the Amazon Resource Name (ARN) of the change set and stores the ARN in the environment variable ID.Note: You must use CAPABILITY_AUTO_EXPAND only if your template uses transformations.3.    (Optional) To wait for the change set to be created successfully, run the following command:aws cloudformation wait change-set-create-complete --change-set-name ${ID}4.    View the change set using the AWS CloudFormation console. Or, use the following describe-change-set command:aws cloudformation describe-change-set --change-set-name ${ID}5.    To apply the change set and import your resource into the stack, run the following command:aws cloudformation execute-change-set --change-set-name ${ID}6.    (Optional) To verify that all properties in your template match with your resource, use drift detection on the resource.Related informationImport an existing resource into a stack using the AWS CLIResources that support import and drift detection operationsFollow"


Question: How do I troubleshoot 403 Access Denied errors from an Amazon S3 bucket with public read access?

"I'm trying to access an object in my Amazon Simple Storage Service (Amazon S3) bucket that allows public read access. However, I'm getting a 403 Access Denied error."

Answer: "I'm trying to access an object in my Amazon Simple Storage Service (Amazon S3) bucket that allows public read access. However, I'm getting a 403 Access Denied error.ResolutionTo diagnose issues with accessing objects from a public S3 bucket, run the AWSSupport-TroubleshootS3PublicRead automation document (on AWS Systems Manager). This article analyzes some permissions settings that affect the bucket and objects, such as the bucket policy and object access control lists (ACLs).Note: The AWSSupport-TroubleshootS3PublicRead document analyzes 403 errors from publicly readable objects. The document doesn't evaluate permissions for private objects.Follow these steps to run the AWSSupport-TroubleshootS3PublicRead automation document using the Systems Manager console:1.    Open the Systems Manager console.2.    In the navigation pane, choose Automation.3.    Choose Execute automation.4.    Under Choose document, choose the Owned by Amazon tab.5.    In the Automation document search bar, enter AWSSupport-TroubleshootS3PublicRead, and then press Enter.6.    Select AWSSupport-TroubleshootS3PublicRead.7.    Select Execute automation.8.    Choose Simple execution.9.    (Optional) For AutomationAssumeRole, you can select an AWS Identity and Access Management (IAM) role that Systems Manager can assume to send requests to your bucket. If you leave this field blank, then Systems Manager uses the IAM identity that you're using to set up the document.Important: The trust policy of the IAM role that you select must allow Systems Manager Automation to assume the role. Additionally, the IAM role must have permissions for running the AWSSupport-TroubleshootS3PublicRead automation document.10.    For S3BucketName, enter the name of the S3 bucket that you want to troubleshoot.11.    (Optional) For S3PrefixName, you can specify a prefix to analyze. If you leave this field blank, then the document lists the bucket and evaluates the first few objects lexicographically.12.    (Optional) For StartAfter, you can specify the key name that you want the document to start listing from.13.    For MaxObjects, enter the maximum number of objects that you want the document to evaluate. The default number is 5.14.    For IgnoreBlockPublicAccess, it's a best practice to leave the value as false.Warning: Changing the value to true ignores Amazon S3 Block Public Access settings that might be blocking access.15.    For HttpGet, leave the value as true if you want the document to perform a partial HTTP GET request (the first byte) for each object. Change the value to false if you want the document to perform a full GET request.16.    For Verbose, enter true if you want to see detailed information during the analysis. Enter false if you only want to see warning and error messages.17.    (Optional) For CloudWatchLogGroupName, you can enter an Amazon CloudWatch log group name that you want to send the analysis results to. If you specify a name and the log group doesn't exist, then the document will try to create a log group with that name.18.    (Optional) For CloudWatchLogStreamName, you can enter a CloudWatch log stream name that you want to send the analysis results to. If you specify a name and the log group doesn't exist, then the document tries to create a log group with that name. If you leave this field blank, then the document uses the document's execution ID as the log stream name.19.    For ResourcePartition, select the partition that the S3 bucket is in. The options are aws, aws-us-gov, or aws-cn.20.    (Optional) For Tags, enter up to five key-value pair tags.21.    Choose Execute.22.    Use the Execution status to track the progress of the document.23.    After the status indicates a Success, review the results listed in Outputs. The results might include error codes for each object evaluated. The error codes can help diagnose the cause of the Access Denied errors for anonymous requests to each object.Tip: To review the result of an individual step in the evaluation, choose the relevant Step ID under Executed steps. The Executed Steps are listed below the Execution status.Follow"


Question: How do I set dynamic environment variables for Docker multi-stage builds in Elastic Beanstalk?

I want to set dynamic environment variables for Docker multi-stage builds in AWS Elastic Beanstalk.

Answer: "I want to set dynamic environment variables for Docker multi-stage builds in AWS Elastic Beanstalk.Short descriptionIn the Elastic Beanstalk Docker platform, Elastic Beanstalk environment variables are set and made accessible for the final container in the Dockerfile. This applies to scenarios involving multiple containers (that is, multi-stage builds). You can set hard-coded/static variables in the Dockerfile only in build stages. For example, you can use the "ENV" key to set static variables. Because Elastic Beanstalk variables aren't accessible by the other containers in the Dockerfile that are getting built in earlier stages, you can follow the steps in the resolution to set dynamic environment variables during the build stage.To set dynamic environment variables during the build stage, use an SSM parameter or an Elastic Beanstalk variable. It's a best practice to use an SSM parameter, because you're required to set the variable one time only. Also, you can follow the same steps in the Use an SSM parameter section to create and update an Elastic Beanstalk variable.Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.ResolutionNote: The following resolution assumes that a React application is installed on Alpine Linux. However, you can use the package manager of any other distribution to build your container and install the required packages.Use an SSM parameterImportant: To allow the instance to retrieve the SSM parameter, confirm that your instance profile has the ssm:GetParameters permission.1.    Create an SSM parameter. For example, you can use the AWS CLI to create the SSM parameter:aws ssm put-parameter --name PRODURL_SSM --value http://myproddomain.com --type String2.    Deploy your application using the following Dockerfile. The Dockerfile retrieves the SSM parameter value inside the temporary/build container, and then exports the variable during the build.# Build environmentFROM node:13.12.0-alpine as buildWORKDIR /appENV PATH /app/node_modules/.bin:$PATHCOPY package.json ./COPY package-lock.json ./RUN npm install react-scripts@3.4.1COPY . ./## Install the required packages for awscli, curl, and jq. ##RUN apk -v --no-cache add \        py-pip \        curl \        jq \        && \        pip install awscli --upgrade --user## Export the region dynamically retrieving it from the instance metadata, and then retrieve the SSM parameter value using awscli. Then, place the parameter in a text file, and export the variable using the value from the text file. Finally, run the build. ##RUN export REGION=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region) && /root/.local/bin/aws ssm get-parameters --names PRODURL_SSM --region $REGION | grep Value | cut -d '"' -f4 >> /tmp/SSMParameter.txt && export DYNAMIC_SSM_VAR=$(cat /tmp/SSMParameter.txt) && npm run build# Production environmentFROM nginx:stable-alpineCOPY --from=build /app/build /usr/share/nginx/html## Copy the text file that has the SSM parameter value from the build container to the production container to confirm the variable was retrieved successfully. ##COPY --from=build /tmp/SSMParameter.txt /tmpEXPOSE 80CMD ["nginx", "-g", "daemon off;"]3.    To confirm that the deployment is successful, check the file that you copied from the build container. This file should include the value of the SSM parameter variable. For example:# docker exec -it <Container-Id> sh# cat /tmp/SSMParameter.txt http://myproddomain.comUse an Elastic Beanstalk variable1.    Copy the config.yml file used by the Elastic Beanstalk Command Line Interface (EB CLI) to the project root directory. For example:cp .elasticbeanstalk/config.yml .2.    To set the Elastic Beanstalk variable with the EB CLI, run the following command to update or deploy to an existing environment:eb setenv PRODURL_EB=http://myproddomain.com3.    To create a new Elastic Beanstalk environment, add the new environment name in the config.yml file in the project root directory, and then run the following command:eb create --envvars PRODURL_EB=http://myproddomain.com4.    Deploy your application using the following Dockerfile. The Dockerfile retrieves the value of the Elastic Beanstalk variable inside the build container, and then exports the variable during the build.# Build environmentFROM node:13.12.0-alpine as buildWORKDIR /appENV PATH /app/node_modules/.bin:$PATHCOPY package.json ./COPY package-lock.json ./RUN npm install react-scripts@3.4.1COPY . ./## Create an .elasticbeanstalk directory to place the config.yml file in, so that the eb cli can interact with the Elastic Beanstalk environment. ##RUN mkdir .elasticbeanstalk## Copy config.yml to /app/.elasticbeanstalk inside the build container as it will be used by eb cli ##COPY config.yml /app/.elasticbeanstalk## Install required packages for awsebcli ##RUN apk -v --no-cache add \        gcc \        musl-dev \        openssl \        openssl-dev \        make \        py-pip \        libffi-dev \        python \        python-dev \        && \        pip install awsebcli --upgrade --user ## Retrieve the Elastic Beanstalk variable using awsebcli and place it in a text file. Then, export the desired variable using the value from the text file, then run the build. ##RUN /root/.local/bin/eb printenv | grep PRODURL_EB | awk '{print $3}' >> /tmp/EBVar.txt && export DYNAMIC_EB_VAR=$(cat /tmp/EBVar.txt) && npm run build# Production environmentFROM nginx:stable-alpineCOPY --from=build /app/build /usr/share/nginx/html## Copy the text file that has the Elastic Beanstalk variable value from the build container to the production container to confirm the variable was retrieved successfully ##COPY --from=build /tmp/EBVar.txt /tmpEXPOSE 80CMD ["nginx", "-g", "daemon off;"]5.    To confirm that the deployment worked, check the file that you copied from the build container. The file should include the value of the Elastic Beanstalk variable.# docker exec -it <Container-Id> sh# cat /tmp/EBVar.txt http://myproddomain.comRelated informationInstance identity documentseb createeb printenveb setenvget-parametersput-parameterFollow"


Question: Why did my CloudWatch canary fail with the error "Node is either not visible or not an HTMLElement"?

"I created an Amazon CloudWatch canary using the GUI Workflow Builder. However, my canary failed and I received the following error: "Node is either not visible or not an HTMLElement". How can I troubleshoot this error?"

Answer: "I created an Amazon CloudWatch canary using the GUI Workflow Builder. However, my canary failed and I received the following error: "Node is either not visible or not an HTMLElement". How can I troubleshoot this error?ResolutionWhen creating a CloudWatch canary, the "Node is either not visible or not an HTMLElement" error is typically caused by:Using a script that references a non-existent element (Wrong selector)Using a script that references an invisible element (Correct selector but the element is invisible)Note: If you use a global selector as a class element in your script, the selector uses the first element in the resulting array by default. If the first element has a "display:none" CSS value, the element will be invisible.To fix this error, follow the steps in How can I identify a DOM selector or XPath to use in a CloudWatch canary GUI Workflow Builder? Be sure to use a selector that has a result in a desired element.Related informationGUI Workflow BuilderFollow"


Question: Why can’t I create a CNAME record in Route 53?

I'm getting errors when trying to create a CNAME record in Amazon Route 53.

Answer: "I'm getting errors when trying to create a CNAME record in Amazon Route 53.ResolutionError: InvalidChangeBatch 400: RRSet of type CNAME with DNS name example.com. is not permitted at apex in zone example.comThe preceding error occurs when creating a CNAME record for an apex domain, such as example.com.DNS protocol doesn't allow creation of a CNAME record for the top node of a DNS namespace (the zone apex). For example, if you register the DNS name example.com, then the zone apex is example.com. You can't create a CNAME record for example.com. But, you can create CNAME records for www.example.com, newproduct.example.com, and so on.To avoid this error, create an alias record for your apex domain. The record can point to your resources, such as your Amazon CloudFront distribution, load balancer, another record in the same hosted zone, and so on.Note: Alias records that have the same name as the hosted zone (the zone apex) can't route traffic to a CNAME record.For more information, see How do I create alias records for services hosted in AWS?Error: InvalidChangeBatch 400: RRSet of type CNAME with DNS name test.example.com. is not permitted as it conflicts with other records with the same DNS name in zoneA CNAME record can't coexist with any other data. A CNAME record set for a subdomain name can't have an MX record, an A record, or a TXT record for that subdomain. Conversely, if there's an MX record for a subdomain, then you can't have a CNAME record for that subdomain.To avoid this restriction, delete the conflicting DNS record, if it's not in use. Or, create an A-alias record pointing the subdomain to your resource, such as a CloudFront distribution, and so on. An example of a subdomain is test.example.com.Error: Bad request. (InvalidChangeBatch 400: DomainLabelTooLong (Domain label is too long) encountered with 'org/services-and-resources/resources/classes-events/trauma-nurse'', Unparseable CNAME encountered)Domain names consist of a series of labels separated by dots. Each label can be up to 63 bytes long. The total length of a domain name can't exceed 255 bytes, including the dots. Route 53 supports any valid domain name.Error: Can't create a CNAME record that has different values for the same CNAME recordTo avoid the preceding error, keep the following parameters in mind when creating CNAME records:CNAME records must contain only one value.CNAME records must point to another domain name and never to an IP address.Follow"


Question: How can I add my own security group to my Elastic Beanstalk environment?

I want to add my own security group to my AWS Elastic Beanstalk environment.

Answer: "I want to add my own security group to my AWS Elastic Beanstalk environment.Short descriptionBy default, Elastic Beanstalk creates a security group for your environment, but you can also add your own security group to your environment. To do this, add one or more configuration files to a .ebextensions directory in your application deployment package. The .ebextensions make Elastic Beanstalk attach a pre-existing security group to your Auto Scaling group.Resolution1.    Use a pre-existing security group, or create a new security group.2.    Note the name of your security group (For example, ebtest).3.    Create a .ebextensions/ directory in your local application code directory.Note: For more information, see Advanced environment customization with configuration files (.ebextensions).4.    In the .ebextensions/ directory, create a file named securitygroup.config. See the following example:option_settings:  - namespace: aws:autoscaling:launchconfiguration    option_name: SecurityGroups    value: ebtest5.    Deploy your code and the new .ebextensions/ directory together as a new application version to your Elastic Beanstalk environment.After deployment, your security group is attached to your Amazon Elastic Compute Cloud (Amazon EC2) instances. Your environment is now successfully using your existing security group in addition to the default security group.Note: If you use Amazon Virtual Private Cloud (Amazon VPC) with Elastic Beanstalk so that your instances are launched within a VPC, specify security group IDs instead of security group names.Related informationAWS GitHub Elastic Beanstalk SamplesHow do I use my own security group for my load balancer when I deploy an AWS Elastic Beanstalk application?Follow"


Question: How do I set up OneLogin as a SAML identity provider with an Amazon Cognito user pool?

I want to use OneLogin as a Security Assertion Markup Language 2.0 (SAML 2.0) identity provider (IdP) with an Amazon Cognito user pool. How do I set that up?

Answer: "I want to use OneLogin as a Security Assertion Markup Language 2.0 (SAML 2.0) identity provider (IdP) with an Amazon Cognito user pool. How do I set that up?Short descriptionAmazon Cognito user pools allow sign-in through a third party (federation), including through a SAML IdP such as OneLogin. For more information, see Adding user pool sign-in through a third party and Adding SAML identity providers to a user pool.To set up OneLogin as SAML IdP, you need an Amazon Cognito user pool and a OneLogin account with an application on it.ResolutionCreate an Amazon Cognito user pool with an app client and domain nameFor more information, see these articles:Tutorial: creating a user poolSetting up the hosted UI with the Amazon Cognito ConsoleAdding a domain name for your user poolNote: When creating a user pool, the standard attribute email is selected by default. For more information about user pool attributes, see Configuring user pool attributes.Create a OneLogin accountFor more information, see the OneLogin website, and then choose Start a free trial.On the account creation page, under Your OneLogin Domain, note the domain that OneLogin provides.Create a OneLogin application1.    On the OneLogin portal page (https://your-new-domain.onelogin.com/portal/), choose Administration.2.    At the top of the Administration page, pause on Apps, and then choose Add apps.3.    In the search bar under Find Applications, enter saml, and then choose SAML Test Connector (IdP) to open the Add SAML Test Connector (IdP) page.4.    (Optional) Do any of the following:For Display Name, enter a name and description. For example, Cognito Setup (IdP).For Rectangular Icon and Square Icon, upload thumbnail icons following the specifications on the page.For Description, enter a short summary description. For example, For Amazon Cognito user pool.5.    Choose Save.Edit your OneLogin application configuration1.    Choose Configuration.2.    On the Configuration page, do the following:For RelayState, enter any valid URL, such as https://www.example.com.For Audience, enter urn:amazon:cognito:sp:yourUserPoolId.Leave Recipient blank.For ACS (Consumer) URL Validator, enter https://yourDomainPrefix.auth.region.amazoncognito.com/saml2/idpresponse.For ACS (Consumer) URL, enter https://yourDomainPrefix.auth.region.amazoncognito.com/saml2/idpresponse.Leave Single Logout URL blank.Note: For Audience, replace yourUserPoolId with your Amazon Cognito user pool ID. Find the ID in the Amazon Cognito console on the General settings tab of the management page for your user pool.For ACS (Consumer) URL Validator and ACS (Consumer) URL, replace yourDomainPrefix and region with the values for your user pool. Find them in the Amazon Cognito console on the Domain name tab of the management page for your user pool.Edit your OneLogin application's parameters1.    Choose Parameters.Note: One parameter (NameID (fka Email)) is already listed—this is expected.2.    Choose Add parameter to create a new, custom parameter.3.    In the New Field dialog, for Field name, enter http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier.4.    For Flags, select the Include in SAML assertion check box.5.    Choose Save.6.    For Value, choose Email from the list.7.    Choose Save.Copy the IdP metadata for your OneLogin application1.    Choose SSO.2.    Under Issuer URL, copy the URL to your clipboard.3.    Choose Save to save all your changes to your OneLogin application.Configure OneLogin as the SAML IdP in Amazon CognitoFor more information, see Creating and managing a SAML identity provider for a user pool. Follow the instructions under To configure a SAML 2.0 identity provider in your user pool.When creating the SAML IdP, for Metadata document, paste the Issuer URL you copied.Map the email address from the IdP attribute to the user pool attributeFor more information, see Specifying identity provider attribute mappings for your user pool. Follow the instructions under To specify a SAML provider attribute mapping.When adding a SAML attribute under Attribute mapping, for SAML Attribute, enter http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier. For User pool attribute, choose Email from the list.Change the app client settings in Amazon CognitoNote: This is an example setup for testing purposes. For a production setup, it's a best practice to use the Authorization code grant OAuth flow for your app client settings. When you use that flow, you receive an authorization code after authentication in your redirect URL. You must exchange the authorization code for JSON web tokens (JWTs) by making a request to the token endpoint.1.    In the Amazon Cognito console management page for your user pool, under App integration, choose App client settings. Then, do the following:Under Enabled identity providers, select the Select all check box.For Callback URL(s), enter a URL where you want your users to be redirected after logging in. For testing, you can enter any valid URL, such as https://www.example.com.For Sign out URL(s), enter a URL where you want your users to be redirected after logging out. For testing, you can enter any valid URL, such as https://www.example.com.Under Allowed OAuth Flows, select at least the Implicit grant check box.Under Allowed OAuth Scopes, select at least the email and openid check boxes.2.    Choose Save changes.For more information, see App client settings terminology.Test the login endpoint1.    In your browser, enter https://yourDomainPrefix.auth.region.amazoncognito.com/login?response_type=token&client_id=yourClientId&redirect_uri=redirectUrl.Note: Replace yourDomainPrefix and region with the values for your user pool. Find them in the Amazon Cognito console on the Domain name tab of the management page for your user pool.ReplaceyourClientId with your app client ID, and replaceredirectUrl with your app client callback URL. Find them in the Amazon Cognito console on theApp client settings tab of the management page for your user pool.For more information, see How do I configure the hosted web UI for Amazon Cognito? and login endpoint.2.    Choose OneLogin.Note: If you're redirected to your app client's callback URL, then you're already logged in to your OneLogin account in your browser. Everything is set up correctly.3.    On the OneLogin page, for Username, enter your OneLogin account user name.4.    Choose Continue.5.    For Password, enter your OneLogin account password.6.    Choose Continue.If you're redirected to your app client's callback URL, then everything is set up correctly.Related informationIntegrating third-party SAML identity providers with Amazon Cognito user poolsSAML user pool IdP authentication flowFollow"


Question: How can I find the resource that currently owns an unknown IP addresses in my Amazon VPC?

I found unknown IP addresses in my Amazon Virtual Private Cloud (Amazon VPC) flow logs or firewall logs. The unknown IP addresses aren't assigned to an instance. How can I find the resource that currently owns these unknown IP addresses in my Amazon VPC?

Answer: "I found unknown IP addresses in my Amazon Virtual Private Cloud (Amazon VPC) flow logs or firewall logs. The unknown IP addresses aren't assigned to an instance. How can I find the resource that currently owns these unknown IP addresses in my Amazon VPC?ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Several resources can create elastic network interfaces in your Amazon VPC. To find the network interface that owns an IP address:Open the Amazon Elastic Compute Cloud (Amazon EC2) console.In the navigation pane, choose Network Interfaces.Click in the search box, and then choose Primary private IPv4 address. If you're looking for a public IP, then choose Public IPv4 address when you click in the search box.Enter the IP address that you're looking for in the search box. The network interfaces that meet your search criteria appear.Note: You can also enter the IP address into the search box without selecting a filter.In the bottom pane, read the Description of the network interface to identify the resource that owns it.Or, use the AWS CLI to find the network interface that owns an IP address:aws ec2 describe-network-interfaces --filters Name=addresses.private-ip-address,Values=IPv4 addressNotes:Replace IPv4 address with the IP address that you're investigating.Confirm that you have the correct Region set in the configuration. Otherwise, manually specify the Region with the --region parameter.Related informationdescribe-network-interfacesFollow"


Question: How do I integrate Amazon Connect with Amazon Kinesis?

I want to stream contact trace records (CTRs) and agent events from Amazon Connect to Amazon Kinesis.

Answer: "I want to stream contact trace records (CTRs) and agent events from Amazon Connect to Amazon Kinesis.Short descriptionActivating data streaming in your Amazon Connect instance lets you stream agent events and CTRs to Amazon Kinesis Data Firehose or Amazon Kinesis Data Streams. Agent events provide near real-time reporting of agent activity within your instance. CTRs provide near real-time data of contact events such as, voice calls, chat, and tasks in your instance.To integrate Amazon Connect with Kinesis Data Firehose or Kinesis Data Streams, complete the following steps:Create a Kinesis Data Firehose delivery stream or Kinesis data stream.Activate data streaming for your instance.Test your setup.ResolutionCreate a Kinesis Data Firehose delivery streamTo integrate Amazon Connect with Kinesis Data Firehose, complete the following steps:Configure the source, destination, and name for your Kinesis Data Firehose delivery stream.Note: Direct PUT lets you create a Kinesis Data Firehose that can stream data from Amazon Connect without having to create a Kinesis data stream. Kinesis stream requires that you create a Kinesis data stream for your Amazon Connect data.(Optional) Configure the Kinesis Data Firehose to transform and convert your record data.(Optional) Configure the backup and advanced settings for your Kinesis Data Firehose delivery stream.Activate data streaming for your instance.Test your delivery stream using sample data.Create a Kinesis data streamTo integrate Amazon Connect with Kinesis Data Streams, complete the following steps:Create a Stream using the AWS Management Console.Activate data streaming for your instance.To test your set-up, complete one of the following sets of steps depending on your use case:You activated data streaming for contact events:Place an outbound or inbound call to create a contact record in your instance.Confirm that the contact record is generated in the Contact search UI.Check the destination configured in your delivery stream for the contact record event. Contact events are delivered in bulk. You might see multiple contact IDs delivered to the destination at the same time. For an example, see Sample contact event for when a voice call is connected to an agent.-or-You activated data streaming for agent events:Have an agent change their status or connect with a contact to create an agent event.Check the destination that's configured for the Kinesis data stream for the agent event. The agent ARN is the partition key that's used when data is sent to the data stream. All events for a single agent are sent to the same shard and any resharding events in the stream are ignored. For an example, see Sample agent event stream.Identifying and troubleshooting errorsI don't see data published in my Kinesis delivery stream destination.To troubleshoot not seeing data published, take one or more of the following actions:Confirm that the Kinesis data stream that you selected is activated for server-side encryption. If server-side encryption is activated due to lack of permission to call the kms:GenerateDataKey, then Amazon Connect can't publish data to the stream. To resolve this issue, follow the steps in using server-side encryption for Kinesis stream.Test your delivery stream using the Test demo data option in Kinesis Data Firehose to verify that your delivery stream is configured as expected.Review the data delivery frequency for Kinesis Data Firehose. Data delivery frequency might vary depending on the destination that's selected. If five minutes pass and data isn't being delivered, then review the delivery stream metrics for incoming records. You can configure error logging for your Kinesis Data Firehose to capture data delivery errors.I can't select the Kinesis Data Firehose or Kinesis Data Stream I that created when I activated the data stream for Amazon Connect.First, confirm that the Kinesis Data Firehose or Kinesis Data Stream exists in the same AWS Region as the Amazon Connect instance. If the services are in the same Region, then there might be a permission missing that allows you to list the stream. Confirm that the following permissions are granted: ListDeliveryStreams for Kinesis Data Firehose, or ListStreams for Kinesis Data Streams.You can also use the AWS Command Line Interface (AWS CLI) command associate-instance-storage-config to activate data streaming.Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Where can I access the agent events or contact trace records (CTRs)?If agent events and CTRs are sent to Kinesis Data Streams, then use the data viewer in the Kinesis Console to see the data. Also, you can configure a consumer to process the data from the Kinesis data stream. For example, you can create a Kinesis Data Firehose to consume the data from the Kinesis data stream for the agent events.If CTRs are sent to Kinesis Data Firehose, then the CTRs are accessible in the destination that's configured for the delivery stream. For example, if the destination is an Amazon Simple Storage Service (Amazon S3) bucket, then you can access the CTRs in the chosen S3 bucket. You choose the Amazon S3 destination bucket when you create the Kinesis Data Firehose.Can I configure more than one Kinesis delivery stream for contact trace records (CTRs)?By default, only one delivery stream is supported for CTRs. But you can use multiple consumers to have multiple delivery streams for CTRs to a Kinesis Data Stream. To use this configuration, the Kinesis data stream must act as a producer for the CTR data. Then, you can configure multiple consumers to ingest the data. For more information on consumers for Kinesis Data Streams, see Reading data from Amazon Kinesis Data Streams.I see duplicate data from Amazon Connect published in my Kinesis Data Streams application.Duplicate data can occur because of producer retries and consumer retries. It's a best practice to have the destination of the final data handle duplicates to be resilient to retries.Related informationAmazon Connect agent event streamsAmazon Connect contact eventsWhat Is Amazon Kinesis Data Streams?What Is Amazon Kinesis Data Firehose?Follow"


Question: Why am I unable to connect to a port on an EC2 Windows instance?

I can't connect to an Amazon Elastic Compute Cloud (Amazon EC2) Windows instance on a certain port. How do I determine if the port is open?

Answer: "I can't connect to an Amazon Elastic Compute Cloud (Amazon EC2) Windows instance on a certain port. How do I determine if the port is open?Short descriptionThe following issues can prevent a connection to an EC2 Windows instance on a specific port:The service that uses the port isn't running on the instance.Windows Firewall is blocking traffic to the port.A security group is blocking traffic.Network access control lists (NACLs) are blocking traffic.Incorrect routes are in the route table or local Windows routes.ResolutionTo identify the source of the connectivity issue, follow these steps:1.    Verify that the application is running properly and is listening on the correct network port. If the application is stopped, start it.To accept traffic on a port, an EC2 Windows instance must host an application or service that listens on the specified port. From the EC2 Windows instance that is hosting the service, run the netstat command to display active connections and ports.2.    Perform a port test using Telnet or Test-NetConnection locally on the instance to confirm that the port can be connected to locally.Using a command prompt, enter the following:telnet <Private IP> <port number>Using Windows PowerShell, enter the following:tnc <Private IP> -port <port number>If the port test fails, check the antivirus and security software that runs on the instance. Set up an allow list for the application or service that uses this port on the antivirus or security software, and then test again.3.    Repeat step 2 using a different instance. If possible, choose an instance on the same subnet. If the port test on another instance also fails, the issue might be with the Operating System (OS) or EC2 instance configuration.Troubleshoot OS issuesVerify that Windows Firewall, and any other security software that runs on the instance, allows the required port.Troubleshoot EC2 instance configuration issuesVerify that the security groups associated with the instance allow inbound or outbound traffic on the specified port.Verify that network ACLs have rules to allow inbound or outbound traffic on the specified port.Check the route table to determine if there is a route from the source and destination EC2 Windows instance.Related informationI host a website on an EC2 instance. How do I allow my users to connect on HTTP (80) or HTTPS (443)?Follow"


Question: "Why isn't Amazon SNS invoking my AWS Lambda function, and how do I troubleshoot the issue?"

"I'm using an AWS Lambda function to process Amazon Simple Notification Service (Amazon SNS) notifications, but my Amazon SNS topic isn't invoking the function. How do I troubleshoot Amazon SNS notifications not invoking my Lambda function?"

Answer: "I'm using an AWS Lambda function to process Amazon Simple Notification Service (Amazon SNS) notifications, but my Amazon SNS topic isn't invoking the function. How do I troubleshoot Amazon SNS notifications not invoking my Lambda function?Short descriptionThere are five reasons why an Amazon SNS message event doesn't invoke a Lambda function that's subscribed to an SNS topic:The AWS Identity Access and Management (IAM) identity publishing to the SNS topic doesn't have the permissions required to publish to the SNS topic.The Lambda function doesn't have the permissions required to allow Amazon SNS to invoke the function.The Lambda function's SNS topic subscription has a filter policy that doesn't match the message attributes being sent from the SNS topic.The Lambda function is throttled because it's close to exceeding the account-level concurrency quota in the AWS Region that it's hosted in.The Amazon SNS topic is hosted in an AWS Region and the Lambda function is hosted in an opt-in Region.Note: For the current list of AWS Regions that Amazon SNS supports, see Amazon Simple Notification Service endpoints and quotas.ResolutionImportant: If you haven't done so already, configure Amazon CloudWatch delivery status logging for your Amazon SNS topic. For more information, see Monitoring Amazon SNS topics using CloudWatch.Confirm that the IAM identity publishing to the SNS topic has the permissions required to publish to the SNS topicLook at the NumberOfMessagesPublished metric in your Amazon CloudWatch metrics for Amazon SNS. If the Publish requests made by the IAM entity that you're using to invoke the function aren't recorded in the NumberOfMessagesPublished metric, do the following:1.    Confirm that the IAM entity making the Publish API request has the permissions required to publish to the SNS topic. For more information and specific policy statement examples, see Example cases for Amazon SNS access control.2.    Confirm that the permissions policy for the SNS topic allows the IAM entity making the Publish API call to use the "SNS:Publish" action. For more information and example permissions policies, see Permissions for the Amazon SNS topic.Confirm that your Lambda function has the permissions required to allow Amazon SNS to invoke the functionTo view your function's permissions policy, follow the instructions in Using resource-based policies for AWS Lambda. If your function's policy doesn't include the "lambda:invokeFunction" action for your SNS topic, update the policy to include the required permissions.Confirm that your Lambda function's filter policy matches the message attributes being sent from the SNS topicReview the NumberOfNotificationsFilteredOut metric in your CloudWatch metrics for Amazon SNS. If the Publish requests made by the IAM entity that you're using to invoke the function appear in the NumberOfNotificationsFilteredOut metric, then do the following:1.    View your Lambda function's SNS topic subscription filter policy by doing the following:Open the Amazon SNS console.On the left navigation panel, choose Subscriptions.On the Edit page, expand the Subscription filter policy section.2.    Review the subscription filter policy to confirm that the Publish request message attributes match the attributes required by the filter policy. If the attributes don't match, then update your Publish request message attributes so that they match the attributes required by the filter policy.Note: For more information, see Amazon SNS subscription filter policies.Confirm that your Lambda function isn't being throttledFollow the instructions in How do I troubleshoot Lambda function throttling with "Rate exceeded" and 429 "TooManyRequestsException" errors?Confirm that your Lambda function is hosted in the same Region as your SNS topicAn Amazon SNS topic hosted in an AWS Region can't invoke a Lambda function that's not in the same AWS Region. For the current list of Regions that Amazon SNS supports, see Amazon Simple Notification Service endpoints and quotas.Look at the Amazon Resource Names (ARNs) of your Lambda function and your Amazon SNS topic. If the Amazon SNS topic is hosted in a commercial AWS Region and the function is hosted in an opt-in Region, do the following:1.    Create a new Lambda function in the same Region as your Amazon SNS topic.2.    Configure the function's settings so that it makes an Invoke API call to the function that's hosted in the opt-in Region.Important: Make sure that you update your SDK Region settings.Example Python (Boto 3) command to change Region settings#us-east-1 clientlambda_us_east_1_client = boto3.client('lambda',  region_name='us-east-1')#us_west_1 clientlambda_us_west_1_client = boto3.client('lambda',  region_name='us-west-1')Example JavaScript/Node.js command to change Region settings//us-east-1 clientvar lambda_us_east_1_client = new AWS.Lambda({apiVersion: '2015-03-31',region: 'us-east-1'});//us_west_1 clientvar lambda_us_east_1_client = new AWS.Lambda({apiVersion: '2015-03-31', region: 'us-west-1'});Follow"


Question: How can I troubleshoot unhealthy Route 53 health checks?

The Amazon Route 53 health checks that I created are reporting as unhealthy.

Answer: "The Amazon Route 53 health checks that I created are reporting as unhealthy.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.First, determine the reason for the last health check failure using the AWS Management Console. Or, use the get-health-check-last-failure-reason command in the AWS CLI.Then, complete the corresponding troubleshooting steps in the following section to identify and fix the issue.Note: Regardless of health check type, check the status of the Invert health check status option. If this option is set to true, then Route 53 considers the health check unhealthy even if it is marked as healthy.Troubleshoot a health check that monitors an endpointError: The health checker couldn't establish a connection within the timeout limitThe preceding error occurs when the health checkers' attempt to connect with the configured endpoint times out. The minimum times to establish a connection are as follows:For TCP health checks, the TCP connection between the health checkers and the endpoint must happen within ten seconds.For HTTP and HTTPS health checks, the TCP connection between the health checkers and the endpoint must happen within four seconds. The endpoint must respond with a 2xx or 3xx HTTP status code within two seconds after establishing a connection.For more information, see How Amazon Route 53 determines whether a health check is healthy.To avoid the timeout error, complete the following steps:1.    In the heath check configuration, note the Domain name or IP address of the endpoint.2.    Access the endpoint. Confirm that the firewall or server allows connections from the Route 53 public IP addresses for the AWS Regions designated in the health check configuration. See IP ranges and search for service: ROUTE53_HEALTHCHECKS. For endpoint resources that are hosted on AWS, configure security groups and network access control lists to allow the Route 53 health checkers' IP addresses.3.    Use the following tools to test connectivity with the configured endpoint over the internet. Replace the placeholders in the following command. In the following example commands, the variables with the values for your use case.TCP test$ telnet <domain name / IP address> <port>HTTP/HTTPS test$ curl -Ik -w "HTTPCode=%{http_code} TotalTime=%{time_total}\n" <http/https>://<domain-name/ip address>:<port>/<path> -so /dev/nullCompare the output of the preceding tests with the timeout values for the health checks. Then, confirm that your application is responding within the respective timelines.For example, if you run the following test:curl -Ik -w "HTTPCode=%{http_code} TotalTime=%{time_total}\n" https://example.com -so /dev/nullThen the output is:HTTPCode=200 TotalTime=0.001963In this example, the total time to get responses with the HTTP status code 200 is 0.001963 seconds.For HTTP connections, the connection time must be within four seconds. The endpoint must respond with the HTTP status code within two seconds after connecting. The total time is six seconds. A value higher than six seconds indicates that the endpoint is slow to respond and the health check fails. In this case, check your endpoint to make sure that it responds within the timeout period.If your output from the test commands shows an HTTP code other than 200, then check the following configurations:Firewall rulesSecurity groupsNetwork access control listsWhen checking the preceding configurations, confirm that your endpoint allows connections from Route 53 public IP addresses.4.    If turned on, then use the latency graph option in the health check configuration to check the metrics graph for the following:TCP connection timeTime to first byteTime to complete SSL handshakeFor more information, see Monitoring the latency between health checkers and your endpoint.Note:If the Latency graph isn't turned on, then you can't edit existing health checks. Instead, you must create a new health check.If the Elastic IP address of the endpoint that you're monitoring is released or updated, then the health check might fail.Error: SSL alert handshake_failureThe handshake failure error indicates that SSL or TLS negotiation with the endpoint failed. When you turn on SNI (HTTPS only), Route 53 sends the hostname in the "client_hello" message to the endpoint during TLS negotiation. This action allows the endpoint to respond to the HTTPS request with the applicable SSL or TLS certificate.If your monitored hostname isn't part of the common name in the endpoint's SSL or TLS certificate, then you receive an "SSL alert handshake_failure" error.Note: To turn on SNI, the monitored endpoint must support SNI.Troubleshoot health checks with the string match conditionThe endpoint server returns "200 OK", but Route 53 marks the health check as unhealthyHealth checkers must establish a TCP connection with the endpoint within four seconds. Health checkers must then receive an HTTP status code of 2xx or 3xx in the next two seconds. Then, the configured string must appear in the first 5,120 bytes of the response body within the next two seconds. If the string isn't present in the first 5,120 bytes, then Route 53 marks the health check as unhealthy.To verify that the string appears in the first 5,120 bytes of the response body, use the following command. Replace domain-name, port, and $search-string with your values.$ curl -sL <http/https>://<domain-name>:<port> | head -c 5120 | grep $search-stringTroubleshoot a health check that monitors a CloudWatch alarmRoute 53 doesn't wait for the Amazon CloudWatch alarm to go into the ALARM state.The preceding situation occurs when Route 53 monitors the metric data stream instead of the state of the CloudWatch alarm.To resolve this error, complete the following steps:1.    Verify the configuration of the health check that's in the INSUFFICIENT DATA state. If the metric data stream provides insufficient information to determine the state of the alarm, then the health check status depends on the InsufficientDataHealthStatus configuration. The status options for the InsufficientDataHealthStatus setting are healthy, unhealthy, or last known status.2.    When you update the configuration of a CloudWatch alarm, the new settings don't automatically appear in the associated health check. To synchronize the health check configuration with the updated CloudWatch alarm's configuration:In the Route 53 console, choose Health Checks.Select the health check, and then choose Synchronize configuration.Follow"


Question: Which metrics should I use to monitor and troubleshoot Kinesis Data Streams issues?

I want to monitor incoming and outgoing data for Amazon Kinesis Data Streams. Which metrics should I use?

Answer: "I want to monitor incoming and outgoing data for Amazon Kinesis Data Streams. Which metrics should I use?ResolutionUsing stream-level metricsYou can use Amazon CloudWatch metrics to continuously monitor the performance of your Amazon Kinesis data stream and its throughput. The following metrics can help you monitor producer and consumer issues:GetRecords.IteratorAgeMilliseconds: Measures the age in milliseconds of the last record in the stream for all GetRecords requests. A value of zero for this metric indicates that the records are current within the stream. A lower value is preferred. To monitor any performance issues, increase the number of consumers for your stream, so that the data is processed more quickly. To optimize your application code, increase the number of consumers to reduce the delay in processing records.ReadProvisionedThroughputExceeded: Measures the count of GetRecords calls that throttled during a given time period, exceeding the service or shard limits for Kinesis Data Streams. A value of zero indicates that the data consumers aren't exceeding service quotas. Any other value indicates that the throughput limit is exceeded, requiring additional shards. This metric confirms that there are no more than five reads/second/shard or 2 MB/second/shard in the stream. You can enable enhanced monitoring to validate that there are no hot shards in the stream.WriteProvisionedThroughputExceeded: Measures the PUT or data producer (like ReadProvisionedThroughputExceeded) to help determine if the stream is being throttled. This exceeds the service quotas for Data Streams when writing into a shard. Be sure that the PUT requests don't exceed 1 MB/second/shard or 1,000 records/shard/second. Be sure that the partition key is evenly distributed and that enhanced monitoring is enabled to verify hot shards in the stream. Depending on shard saturation, consider updating shard count in the stream to allow for increased throughput.PutRecord.Success and PutRecords.Success: Measure the count of successful records of PutRecords request over a given period by data producers into the stream. This metric confirms effective retry logic for failed records.GetRecords.Success: Measures the count of successful GetRecords requests for a given time period in the stream. It confirms effective retry logic for failing records.GetRecords.Latency: Measures the time taken for each GetRecords operation on the stream over a specified time period. Confirms sufficient physical resources or record processing logic for increased stream throughput. Processes larger batches of data to reduce network and other downstream latencies in your application. For the Kinesis Client Library (KCL), investigate the ProcessTask.Time metric to monitor the processing time of the application that is falling behind. The GetRecords.Latency metric confirms that the IDLE_TIME_BETWEEN_READS_IN_MILLIS setting is set to keep up with stream processing.PutRecords.Latency: Measures the time taken for each PutRecords operation on the stream over a specified time period. If the PutRecords.Latency value is high, aggregate records into a larger file to put batch data into the Kinesis data stream. You can also use multiple threads to write data. Throttling and retry logic on the PutRecords API can impact latency and the time taken for each PutRecords operation on the stream.Then, use the Average statistic for the listed metrics to monitor performance and throughput of the stream.Note: For GetRecords.IteratorAgeMilliseconds, the Maximum statistic should be used to reduce the risk of data loss for consumers that are lagging behind any read operations. Configure a CloudWatch alarm to trigger any data points to be evaluated for a metric. For more information about CloudWatch alarms, see Using Amazon CloudWatch alarms.If you are using the enhanced fan-out feature, use the following metrics to monitor Kinesis Data Streams:SubscribeToShard.RateExceeded: Measures the number of calls per second exceeded that are allowed for the operation or when a subscription attempt fails because an active subscription already exists.SubscribeToShard.Success: Verifies whether the SubscribeToShard operation succeeds.SubscribeToShardEvent.Success: Verifies the successful publication of an event for active subscription.SubscribeToShardEvent.Bytes: Measures the number of bytes received in the shards over the specified time period.SubscribeToShardEvent.Records: Measures the number of records received in the shards over the specified time period.SubscribeToShardEvent.MillisBehindLatest: Measures the difference of current time and last record of the SubscribeToShard event written to the stream.Enabling enhanced shard-level metricsEnable shard-level metrics in CloudWatch to monitor specific tasks and to troubleshot data producers and consumers. For example, enabling shard-level metrics can help you identify issues like uneven workload distributions.To enable enhanced monitoring, perform the following steps:Note: You can also use theEnableEnhancedMonitoring API request or enable-enhanced-monitoring AWS CLI.1.    Open the Kinesis console.2.    Choose a specific Region.3.    From the navigation pane, choose Data Streams.4.    Under Data Stream Name, select your Kinesis data stream.5.    Choose Configuration.6.    Choose Edit under Enhanced (shard-level) metrics.7.    From the dropdown menu, select your metrics for enhanced monitoring.8.    Choose Save Changes to apply your configuration settings.Additional troubleshooting with API callsYou can also use the following API calls to read or write data from Kinesis Data Streams:CreateStream: Limit of five transactions per second per account.DeleteStream: Limit of five transactions per second per account.ListStreams: Limit of five transactions per second per account.GetShardIterator: Limit of five transactions per second per account per open shard.MergeShards: Limit of five transactions per second per account.DescribeStream: Limit of ten transactions per second per account.DescribeStreamSummary: Limit of twenty transactions per second per account.When you use these API calls, you can monitor any throttling in the AWS CloudTrail logs. For more information about Kinesis Data Streams API calls and CloudTrail, see Logging Amazon Kinesis Data Streams API calls with AWS CloudTrail.Related informationAmazon CloudWatch pricingMonitoring Amazon Kinesis Data Streams Service with Amazon CloudWatchFollow"


Question: How can I use Cost Explorer to analyze my spending and usage?

I want to use Cost Explorer to analyze my spending and usage.

Answer: "I want to use Cost Explorer to analyze my spending and usage.Short descriptionWith Cost Explorer, you can do the following:Review charges and usage associated with your account.View and create reports for your cost and usage data.Forecast estimated spending for the next three months.Get recommendations on utilization and coverage of your Reserved Instances (RIs).Get recommendations on utilization and coverage of your Savings Plans.ResolutionFirst, launch Cost Explorer for your account:Open the Billing and Cost Management console.From the navigation pane, choose Cost Explorer.Choose Launch Cost Explorer.After you launch Cost Explorer, you might need to wait for up to 24 hours for cost and usage data to populate.To view pre-configured reports for your cost and usage data, do the following:Open the AWS Cost Management console.Choose Reports from the navigation pane.Select a report name to analyze your costs.To create custom reports for your cost and usage data, do the following:Open the AWS Cost Management console.Choose Cost Explorer from the navigation pane.Choose New Report and select a report type you want to create.Choose Create Report. Then, save the report for later use by choosing Save as.Here are some common uses for Cost Explorer:Compare your month-to-month costsTo compare your month-to-month costs, use the Monthly costs by service report. By default, the chart displays the five services with the highest costs. However, the table includes cost data for all the services that you used.You can also investigate temporary spikes in your billing by opening the Monthly costs by service report. Then, under Monthly, choose Daily instead.Understand costs for your resources by serviceOpen the Service filter.Select the services that you want to investigate.Choose Apply Filters.For Group by, choose Usage Type.For example, to see how much you're billed for data transfer for Elastic Load Balancing, choose the EC2-ELB service in FILTERS. Then, choose Usage Type for Group by. Then, look for data transfer.View costs by cost allocation tagTo view costs by cost allocation tag, choose Tag for Group by. The Tag option isn't available if you have no cost allocation tags. When you create a new cost allocation tag, you might need to wait up to three days for the tag to appear in Cost Explorer.View unblended cost data for an organizationTo view unblended cost data for member accounts in an organization, select Unblended costs for Show costs as in ADVANCED OPTIONS.View taxes by accountFor Group by, choose Linked account.Open the Service filter, and then choose Tax.View RI utilizationTo view your RI utilization, select Reports from the navigation pane and choose RI Utilization under Report name.View the combined usage of all your purchased RIs in the chart and the utilization of individual RIs in the table.View the utilization of your RIs as the percentage of purchased RI hours in the chart.View the number of RI hours used against the number of RI hours purchased in the table.Select a single RI or a group of RIs in the table to view their respective utilization in the chart.Use the information in the table to track the number of RI hours that you have reserved but did not use.For more information, see RI utilization reports.View RI coverageTo view your RI coverage, select Reports from the navigation pane and choose RI Coverage under Report name.See the number of instance hours covered by your RIs in the chart.Track the number of hours that you have totally used and how many of those are covered by your RIs.View the number of hours covered by RIs against the On Demand hours in the table.Select a single RI or a group of RIs in the table to view their respective coverage in the chart.For more information, see RI coverage reports.View your RI recommendationsTo view your Amazon Elastic Compute Cloud (Amazon EC2) RI recommendations and estimated annual savings, choose Recommendations in the navigation pane. The RI recommendations are provided based on your past EC2 usage.View your Savings Plans recommendationsTo view your Savings Plans recommendations and estimated savings:Open the AWS Cost Management console.Then choose Recommendations, located under Savings Plans in the navigation pane.Use the Recommendation parameters to choose a Savings Plans type. Then, select Recommendation options based on your preferred term length, payment option, and past usage.The Recommendations table provides Monthly On-Demand Spend, Estimated Monthly Spend, Estimated Monthly Savings, and Recommended Savings Plans.Note: It might take up to 24 hours after a Savings Plans purchase for the Savings Plans Recommendations to update.View Savings PlansTo view purchased Savings Plans and when they expire:Open the AWS Cost Management console.Then choose Inventory, located under Savings Plans in the navigation pane.View Savings Plans utilizationTo view your Savings Plans utilization, choose Utilization report, located under Savings Plans in the navigation pane. The Utilization report shows the percentage of your Savings Plans commitments that you are utilizing across your applicable usage.View Savings Plans coverageTo view your Savings Plans coverage, choose Coverage report, located under Savings Plans in the navigation pane. The Coverage report shows how much of your applicable usage was covered by Savings Plans.Related informationAnalyzing your costs with Cost ExplorerUnderstanding your reservations with Cost ExplorerTag your resources for billingFollow"