Question: How do I monitor IAM events and set up event notifications with AWS CloudFormation?

"I want to monitor AWS Identity and Access Management (IAM) activity with AWS CloudFormation. For example, I want to receive an email notification whenever a certain IAM event occurs."

Answer: "I want to monitor AWS Identity and Access Management (IAM) activity with AWS CloudFormation. For example, I want to receive an email notification whenever a certain IAM event occurs.Short descriptionYou can use the AWS CloudFormation template in the following resolution to monitor IAM events and set up notifications with Amazon EventBridge.Consider the following:You must deploy the template in the US East (N. Virginia) - us-east-1 AWS Region.You can modify the AWS::SNS::Topic resource to include additional email addresses.You can edit the AWS::Events::Rule resource to add or remove additional API calls.ResolutionThe following template includes the AWS::Events::Rule resource that contains the EventPattern property. You can use the EventPattern property to give different event sources and API calls the ability to add or restrict specific events for your use case. Each API call has different parameters and information in the events. It isn't possible to create one-size-fits-all events rule. The following sample template provides only two rules that group together many different API calls. However, you can use as many rules as you need to create emails or notifications for different API calls. You can customize and define which API calls to monitor and define customized emails and information for each call.1.    Copy the following JSON or YAML version of the template, and then update it with the values for your environment.JSON:{    "AWSTemplateFormatVersion": "2010-09-09",    "Description": "Monitor IAM events with EventBridge rules with AWS CloudFormation. This Stack must be deployed in 'us-east-1' (IAM).",    "Parameters": {        "EmailList": {            "Type": "String",            "Description": "Email to notify!",            "AllowedPattern": "[a-zA-Z0-9]+@[a-zA-Z0-9]+\\.[a-zA-Z]+",            "Default": "mail@example.com"        },        "SNSTopicName": {            "Type": "String",            "Description": "Name for the notification topic.",            "AllowedPattern": "[a-zA-Z0-9_-]+",            "Default": "iam-monitoring-topic"        },        "MonitorStatus": {            "Type": "String",            "Description": "Enable / Disable monitor.",            "AllowedValues": [                "ENABLED",                "DISABLED"            ],            "Default": "ENABLED"        }    },    "Resources": {        "SNSMonitoringTopic": {            "Type": "AWS::SNS::Topic",            "Properties": {                "Subscription": [                    {                        "Endpoint": {                            "Ref": "EmailList"                        },                        "Protocol": "email"                    }                ],                "TopicName": {                    "Fn::Sub": "${AWS::StackName}-${SNSTopicName}"                }            }        },        "SNSMonitoringTopicTopicPolicy": {            "Type": "AWS::SNS::TopicPolicy",            "Properties": {                "Topics": [                    {                        "Ref": "SNSMonitoringTopic"                    }                ],                "PolicyDocument": {                    "Version": "2012-10-17",                    "Statement": [                        {                            "Sid": "SnsIAMTopicPolicy",                            "Effect": "Allow",                            "Principal": {                                "Service": "events.amazonaws.com"                            },                            "Action": [                                "sns:Publish"                            ],                            "Resource": {                                "Ref": "SNSMonitoringTopic"                            }                        },                        {                            "Sid": "AllowAccessToTopicOwner",                            "Effect": "Allow",                            "Principal": {                                "AWS": "*"                            },                            "Action": [                                "sns:GetTopicAttributes",                                "sns:SetTopicAttributes",                                "sns:AddPermission",                                "sns:RemovePermission",                                "sns:DeleteTopic",                                "sns:Subscribe",                                "sns:ListSubscriptionsByTopic",                                "sns:Publish",                                "sns:Receive"                            ],                            "Resource": {                                "Ref": "SNSMonitoringTopic"                            },                            "Condition": {                                "StringEquals": {                                    "AWS:SourceOwner": {                                        "Ref": "AWS::AccountId"                                    }                                }                            }                        }                    ]                }            }        },        "EventRulePolicyMonitor": {            "Type": "AWS::Events::Rule",            "Properties": {                "Name": {                    "Fn::Sub": "${AWS::StackName}-policy-monitor"                },                "Description": "This EventBridge rule will capture IAM API Calls and events related to creation and deletion of policies.\n",                "State": {                    "Ref": "MonitorStatus"                },                "EventPattern": {                    "source": [                        "aws.iam"                    ],                    "detail-type": [                        "AWS API Call via CloudTrail"                    ],                    "detail": {                        "eventSource": [                            "iam.amazonaws.com"                        ],                        "eventName": [                            "CreatePolicy",                            "DeletePolicy",                            "PutGroupPolicy",                            "DeleteGroupPolicy",                            "PutRolePolicy",                            "DeleteRolePolicy",                            "PutUserPolicy",                            "DeleteUserPolicy",                            "CreatePolicyVersion",                            "DeletePolicyVersion",                            "AttachRolePolicy",                            "DetachRolePolicy",                            "AttachUserPolicy",                            "DetachUserPolicy",                            "AttachGroupPolicy",                            "DetachGroupPolicy"                        ]                    }                },                "Targets": [                    {                        "Arn": {                            "Ref": "SNSMonitoringTopic"                        },                        "Id": "iam-policy-monitor",                        "InputTransformer": {                            "InputPathsMap": {                                "eventName": "$.detail.eventName",                                "policyName": "$.detail.requestParameters.policyName",                                "policyArn": "$.detail.requestParameters.policyArn",                                "eventTime": "$.detail.eventTime",                                "userIdentity": "$.detail.userIdentity.arn",                                "sourceIPAddress": "$.detail.sourceIPAddress"                            },                            "InputTemplate": "\"API Call '<eventName>' was issued on policy '<policyName><policyArn>'. This occurred at '<eventTime>' and was initiated by '<userIdentity>' from IP '<sourceIPAddress>'. Please review the details here: https://console.aws.amazon.com/iam/home?region=us-east-1#/policies/<policyArn>$jsonEditor?section=attached_entities .\"\n"                        }                    }                ]            }        },        "EventRulePrincipalsMonitor": {            "Type": "AWS::Events::Rule",            "Properties": {                "Name": {                    "Fn::Sub": "${AWS::StackName}-principals-monitor"                },                "Description": "This EventBridge rule will capture IAM API Calls and events related to creation and deletion of users, groups and roles.",                "State": {                    "Ref": "MonitorStatus"                },                "EventPattern": {                    "source": [                        "aws.iam"                    ],                    "detail-type": [                        "AWS API Call via CloudTrail"                    ],                    "detail": {                        "eventSource": [                            "iam.amazonaws.com"                        ],                        "eventName": [                            "CreateUser",                            "CreateGroup",                            "CreateRole",                            "UpdateUser",                            "UpdateGroup",                            "UpdateRole",                            "DeleteUser",                            "DeleteGroup",                            "DeleteRole"                        ]                    }                },                "Targets": [                    {                        "Arn": {                            "Ref": "SNSMonitoringTopic"                        },                        "Id": "iam-user-monitor",                        "InputTransformer": {                            "InputPathsMap": {                                "eventName": "$.detail.eventName",                                "userName": "$.detail.requestParameters.userName",                                "roleName": "$.detail.requestParameters.roleName",                                "groupName": "$.detail.requestParameters.groupName",                                "eventTime": "$.detail.eventTime",                                "userIdentity": "$.detail.userIdentity.arn",                                "sourceIPAddress": "$.detail.sourceIPAddress"                            },                            "InputTemplate": "\"API Call '<eventName>' was issued on '<userName><roleName><groupName>'. This occurred at '<eventTime>' and was initiated by '<userIdentity>' from IP '<sourceIPAddress>'. \"\n"                        }                    }                ]            }        }    }}YAML:AWSTemplateFormatVersion: 2010-09-09Description: >             - Monitor IAM events with EventBridge rules with AWS CloudFormation.             - This Stack must be deployed in 'us-east-1' (IAM).Parameters:  EmailList:    Type: String    Description: "Email to notify!"    AllowedPattern: '[a-zA-Z0-9]+@[a-zA-Z0-9]+\.[a-zA-Z]+'    Default: "mail@example.com"  SNSTopicName:    Type: String    Description: "Name for the notification topic."    AllowedPattern: '[a-zA-Z0-9_-]+'    Default: "iam-monitoring-topic"  MonitorStatus:    Type: String    Description: "Enable / Disable monitor."    AllowedValues:      - ENABLED      - DISABLED    Default: ENABLEDResources:  SNSMonitoringTopic:    Type: AWS::SNS::Topic    Properties:      Subscription:        - Endpoint: !Ref EmailList          Protocol: email      TopicName: !Sub ${AWS::StackName}-${SNSTopicName}  SNSMonitoringTopicTopicPolicy:    Type: AWS::SNS::TopicPolicy    Properties:      Topics:        - !Ref SNSMonitoringTopic      PolicyDocument:          Version: '2012-10-17'          Statement:          - Sid: SnsIAMTopicPolicy            Effect: Allow            Principal:              Service: events.amazonaws.com            Action: [  'sns:Publish' ]            Resource: !Ref SNSMonitoringTopic          - Sid: AllowAccessToTopicOwner            Effect: Allow            Principal:              AWS: '*'            Action: [  'sns:GetTopicAttributes',                       'sns:SetTopicAttributes',                       'sns:AddPermission',                       'sns:RemovePermission',                       'sns:DeleteTopic',                       'sns:Subscribe',                       'sns:ListSubscriptionsByTopic',                       'sns:Publish',                       'sns:Receive' ]            Resource: !Ref SNSMonitoringTopic            Condition:              StringEquals:                'AWS:SourceOwner': !Ref 'AWS::AccountId'  EventRulePolicyMonitor:    Type: AWS::Events::Rule    Properties:      Name: !Sub ${AWS::StackName}-policy-monitor      Description: >        This EventBridge rule will capture IAM API Calls and        events related to creation and deletion of policies.      State: !Ref MonitorStatus      EventPattern:        source:           - aws.iam        detail-type:          - AWS API Call via CloudTrail        detail:          eventSource:            - iam.amazonaws.com          eventName:            - CreatePolicy            - DeletePolicy            - PutGroupPolicy            - DeleteGroupPolicy            - PutRolePolicy            - DeleteRolePolicy            - PutUserPolicy            - DeleteUserPolicy            - CreatePolicyVersion            - DeletePolicyVersion            - AttachRolePolicy            - DetachRolePolicy            - AttachUserPolicy            - DetachUserPolicy            - AttachGroupPolicy            - DetachGroupPolicy      Targets:        - Arn:            Ref: SNSMonitoringTopic          Id: iam-policy-monitor          InputTransformer:            InputPathsMap:              eventName: $.detail.eventName              policyName: $.detail.requestParameters.policyName              policyArn: $.detail.requestParameters.policyArn              eventTime: $.detail.eventTime              userIdentity: $.detail.userIdentity.arn              sourceIPAddress: $.detail.sourceIPAddress            InputTemplate: >                "API Call '<eventName>' was issued on policy '<policyName><policyArn>'. This occurred at '<eventTime>' and was initiated by '<userIdentity>' from IP '<sourceIPAddress>'. Please review the details here: https://console.aws.amazon.com/iam/home?region=us-east-1#/policies/<policyArn>$jsonEditor?section=attached_entities ."  EventRulePrincipalsMonitor:    Type: AWS::Events::Rule    Properties:      Name: !Sub ${AWS::StackName}-principals-monitor      Description: >        This EventBridge rule will capture IAM API Calls and        events related to creation and deletion of users, groups        and roles.      State: !Ref MonitorStatus      EventPattern:        source:           - aws.iam        detail-type:          - AWS API Call via CloudTrail        detail:          eventSource:            - iam.amazonaws.com          eventName:            - CreateUser            - CreateGroup            - CreateRole            - UpdateUser            - UpdateGroup            - UpdateRole            - DeleteUser            - DeleteGroup            - DeleteRole      Targets:        - Arn:            Ref: SNSMonitoringTopic          Id: iam-user-monitor          InputTransformer:            InputPathsMap:              eventName: $.detail.eventName              userName: $.detail.requestParameters.userName              roleName: $.detail.requestParameters.roleName              groupName: $.detail.requestParameters.groupName              eventTime: $.detail.eventTime              userIdentity: $.detail.userIdentity.arn              sourceIPAddress: $.detail.sourceIPAddress            InputTemplate: >                "API Call '<eventName>' was issued on '<userName><roleName><groupName>'. This occurred at '<eventTime>' and was initiated by '<userIdentity>' from IP '<sourceIPAddress>'. "2.    Deploy your template using either the AWS CloudFormation console or the AWS Command Line Interface (AWS CLI):AWS CloudFormation console1.    Download the template.2.    Open the AWS CloudFormation console.3.    From the AWS Region selector in the navigation bar, choose us-east-1.4.    Choose Create stack, and then choose With new resources (standard).5.    In the Specify template section, choose Upload a template file.6.    Choose Choose file, select the template that you downloaded in step 1, and then choose Next.7.    In the Stack name section, for Stack name, enter a name for your stack.8.    In the Parameters section, for EmailList, enter the email address where you want to receive notifications.9.    For MonitorStatus, choose ENABLED.10.    For SNSTopicName, leave the default name, or choose your own name for an Amazon Simple Notification Service (Amazon SNS) topic.11.    Complete the rest of the steps in the setup wizard, and then choose Create stack.12.    Check your inbox (using the email address that you entered in step 8) for a confirmation email, and then follow the email instructions to confirm your subscription.AWS CLINote: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.1.    Download the template, and then name the template sample-event-rule-iam-sns.yaml.2.    Set up the AWS CLI.3.    Open a command line in your operating system, and go to the folder where your template is located.4.    Run the following command:aws cloudformation --region=us-east-1 \                   create-stack \                   --stack-name iam-sample-monitor \                   --template-body file://sample-event-rule-iam-sns.yaml \                   --parameters \                   ParameterKey=EmailList,ParameterValue="mail@example.com"Note: Replace mail@example.com with the email that you want to receive notifications.5.    Check your inbox (using the email address that you entered in step 4) for a confirmation email, and then follow the email instructions to confirm your subscription.Test the monitoring notification1.    Open the IAM console.2.    Create a testing policy.3.    Check your email for a notification about the event. You will receive an email similar to the following:API Call 'CreatePolicy' was issued on policy 'test-policy'. This occurred at '2020-11-13T00:00:00Z' and was initiated by 'arn:aws:sts::123456789012:assumed-role/your-role' from IP 'X.Y.Z.T'. Please review the details here: https://console.aws.amazon.com/iam/home?region=us-east-1#/policies/$jsonEditor?section=attached_entities.Related informationCloudTrail record contentsWelcome to the IAM API referenceFollow"


Question: How can I integrate Amazon S3 with my Aurora PostgreSQL-Compatible DB cluster?

I want to turn on the Amazon Simple Storage Solution (Amazon S3) integration feature in my Amazon Aurora PostgreSQL-Compatible Edition DB cluster.

Answer: "I want to turn on the Amazon Simple Storage Solution (Amazon S3) integration feature in my Amazon Aurora PostgreSQL-Compatible Edition DB cluster.Short descriptionYou can import data that is stored in an Amazon S3 bucket into a table on an Aurora PostgreSQL-Compatible DB cluster. First, install the Aurora PostgreSQL-Compatible aws_s3 extension. This extension provides the functions that you need to import data from an S3 bucket. The data can be in a comma-separate value (CSV) file, a text file (txt), or a compressed (gzip) file. Your database must be running PostgreSQL version 10.7 or higher to import from Amazon S3 into Aurora PostgreSQL-Compatible.To integrate your Aurora DB cluster with an S3 bucket:Install the aws_s3 extensionSet up access to an S3 bucketAdd the option to the Aurora clusterUse a psql query to import or export the data from the databaseResolutionInstall the aws_s3 extensionTo install the aws_s3 extension that allows you to connect to the database, run this command:postgres=> CREATE EXTENSION aws_s3 CASCADE;To verify that aws_s3 is installed, use the psql \dx meta-command. Or you can use this query from any DB client tool:select * from pg_available_extensions where installed_version is not null;Set up access to an S3 bucketTo give your Aurora PostgreSQL-Compatible cluster access to Amazon S3, create an AWS Identity and Access Management (IAM) policy. This policy provides the bucket and object permissions that allow your DB cluster to access the S3 bucket.In the IAM policy that you create, include these required actions. These actions allow the transfer of files from the S3 bucket to your DB cluster:s3:GetObjects3:ListBuckets3:PutObjects3:AbordMultipartUploadFor more information, see Setting up access to an Amazon S3 bucket.Add the option to the Aurora clusterAdd an IAM role to your cluster using either the Amazon Relational Database Service (Amazon RDS) console or the AWS Command Line Interface (AWS CLI).Using the consoleLog in to the Amazon RDS console.From the navigation pane, choose the PostgreSQL cluster that you want to modifyChose the Connectivity & security tab.From the Manage IAM roles section, under Add IAM roles to this instance, choose the role that you want to add.Under Feature, choose s3Export, and then choose Add role.Import or export data from the databaseImport data to your PostgreSQL cluster from the S3 bucket or export data to your S3 bucket from your PostgreSQL cluster.For more information, see Export and import data from Amazon S3 to Amazon Aurora PostgreSQL.After you complete these steps, import data to an Aurora PostgreSQL-Compatible cluster from Amazon S3 using aws_s3.table_import_from_s3. You can export data from Aurora PostgreSQL-Compatible cluster to Amazon S3 using the aws_s3.query_export_to_s3 function.Follow"


Question: How do I pass values between nested stacks within the same parent stack in AWS CloudFormation?

I want to pass or share a value between two nested stacks within the same parent stack in AWS CloudFormation.

Answer: "I want to pass or share a value between two nested stacks within the same parent stack in AWS CloudFormation.Short descriptionThe resolution assumes the following:You have two nested stacks, NestedStackA and NestedStackB, that are part of the same parent stack.You want to use a value from NestedStackA in NestedStackB.Resolution1.FSPIn your AWS CloudFormation template, pass the value that you want to share as an output in your source stack ( NestedStackA). See the following JSON and YAML examples.JSON:"Outputs": {    "SharedValueOutput": {        "Value": "yourValue",        "Description": "You can refer to any resource from the template."    }}YAML:Outputs:  SharedValueOutput:    Value: yourValue             Description: You can refer to any resource from the template.2.FSPCreate a parameter in the destination stack ( NestedStackB). See the following JSON and YAML examples.JSON:"Parameters": {    "SharedValueParameter": {        "Type": "String",        "Description": "The shared value will be passed to this parameter by the parent stack."    }}YAML:Parameters:   SharedValueParameter:     Type: String    Description: The shared value will be passed to this parameter by parent stack.3.FSPPass the output value from NestedStackA as the parameter value for NestedStackB. To access this value in the parent stack, use the Fn::GetAtt function. Use the logical name of NestedStackA and the name of the output value in Outputs.NestedStackOutputName format. See the following JSON and YAML examples.JSON:{  "AWSTemplateFormatVersion": "2010-09-09",  "Resources": {    "NestedStackA": {      "Type": "AWS::CloudFormation::Stack",      "Properties": {        "TemplateURL": "<S3 URL for the template>"      }    },    "NestedStackB": {      "Type": "AWS::CloudFormation::Stack",      "Properties": {        "TemplateURL": "<S3 URL for the template>",        "Parameters": {          "SharedValueParameter": {            "Fn::GetAtt": [              "NestedStackA",              "Outputs.SharedValueOutput"            ]          }        }      }    }  }}YAML:AWSTemplateFormatVersion: 2010-09-09Resources:  NestedStackA:    Type: 'AWS::CloudFormation::Stack'    Properties:      TemplateURL: <S3 URL for the template>  NestedStackB:    Type: 'AWS::CloudFormation::Stack'    Properties:      TemplateURL: <S3 URL for the template>      Parameters:        SharedValueParameter:           Fn::GetAtt:           - NestedStackA          - Outputs.SharedValueOutputRelated informationWorking with nested stacksAWS::CloudFormation::StackAWS CloudFormation template snippetsFollow"


Question: How do I troubleshoot errors related to CloudWatch streaming for an Elastic Beanstalk environment?

I want to troubleshoot errors related to Amazon CloudWatch streaming for an AWS Elastic Beanstalk environment.

Answer: "I want to troubleshoot errors related to Amazon CloudWatch streaming for an AWS Elastic Beanstalk environment.Short descriptionStreaming your logs to CloudWatch can help safeguard your data. For example, if your Elastic Beanstalk environment experiences a terminated Elastic Compute Cloud (Amazon EC2) instance, then you can still recover your logs from CloudWatch. You can also use log rotation to protect against data loss.If your environment's instance logs aren't streaming to CloudWatch even after activating log streaming, then you must look into the following common issues:Your IAM instance profile role lacks the required IAM permissions.You launched your environment in an AWS Region that doesn't support CloudWatch Logs.One of your custom log files doesn't exist in the path you specified.Resolution1.    Verify that your Elastic Beanstalk environment instance profile role has the following permissions:{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Action": [        "logs:CreateLogStream",        "logs:PutLogEvents",        "logs:DescribeLogGroups",        "logs:DescribeLogStreams"      ],      "Resource": [        "*"      ]    }  ]}2.    If you are streaming custom logs, then use a configuration file to directly install the CloudWatch Logs agent and to configure the files. Also, you must verify the format and indentation of that configuration file. For more information on streaming custom logs to CloudWatch, refer the sample configuration file.3.    Check the CloudWatch agent status and agent logs to find out the causes of log streaming issues.To check the old CloudWatch agent (awslogsd) status and logs:agent status: systemctl status awslogsd.serviceagent log location: /var/log/awslogs.logTo check the latest CloudWatch agent (amazon-cloudwatch-agent) status and logs:agent status: systemctl status amazon-cloudwatch-agent.serviceagent logs location: /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.logFor more information on customizing log files in Elastic Beanstalk, see How can I customize my log files in Elastic Beanstalk?Related informationUsing Elastic Beanstalk with Amazon CloudWatch Logslogs-streamtocloudwatch-linux.configFollow"


Question: Why am I unable to launch Amazon SageMaker Studio?

I want to troubleshoot issues when I launch Amazon SageMaker Studio.

Answer: "I want to troubleshoot issues when I launch Amazon SageMaker Studio.ResolutionIssues with the loading screenYou might get an error that looks similar to the following:The loading screen is taking too long. Would you like to clear the workspace or keep waitingYou can resolve this error by doing the following:Clear cache and cookies from your current browser. Sign out and sign back in to the AWS Management Console. Then, open SageMaker Studio.Try using a different browser to open SageMaker Studio.For SageMaker Studio on VPC only mode, check whether all the required endpoints are attached to subnets that are associated with your domain. For more information, see Connect SageMaker Studio notebooks in a VPC to external resources.Issues with JupyterServer appYou might get an error that looks similar to the following:The JupyterServer app default encountered a problem and was stopped. Restart NowTo resolve this error, try deleting all the apps from SageMaker Studio for the user by doing the following:Open the SageMaker console.In the navigation pane, choose Studio.Under Users, choose the user that's experiencing this issue.On the User Details page, choose Delete app to delete all the apps for the user.Be sure not to delete the actual user profile.After all the apps for the user are deleted, choose Launch app, and then choose Studio.SageMaker Studio reinitializes the Jupyter notebook.Note: You might need to wait for some time for Studio to reinitialize the notebook.If the preceding approach doesn't work, delete the user profile by doing the following:Open the SageMaker console.In the navigation pane, choose Control panel.Under Users, choose the use that's experiencing this issue.On the User Details page, choose Delete app for all apps in the list that didn't fail.Be sure not to delete the actual user profile.On the Delete app dialog box, choose Yes, delete app, enter delete in the confirmation field, and then choose Delete.After the status of the apps that you deleted displays as Deleted, choose Delete user.Note: When a user is deleted, the user can't access the Amazon Elastic File System (Amazon EFS) volume containing data, including notebooks and other artifacts. The data isn't deleted and can be accessed by an administrator. After deleting the user, add a new user with the appropriate AWS Identity and Access Management (IAM) role. Then, try accessing SageMaker Studio with that user.You might also get this error if the Amazon Virtual Private Cloud (Amazon VPC) where you launched Studio uses Dynamic Host Configuration Protocol (DHCP) and isn't configured correctly. If the Amazon VPC uses DHCP, then be sure that the DHCP option set is configured correctly.Open the Amazon VPC console.In the navigation pane, choose Your VPCs.Select the VPC that you want to check, and then choose Actions.Choose Edit DHCP option set.For DHCP options set, select an existing option set, and then choose Save changes.Issues related to IAM Identity CenterYou might get an error similar to the following:You're attempting to set up Studio with SSO authentication in Europe (Frankfurt). Your AWS SSO is configured in Europe (Ireland). You must set up Studio in the same Region as AWS SSO or choose IAM authentication.Currently, SageMaker doesn't support using SageMaker Studio with AWS IAM Identity Center (successor to AWS Single Sign-On) in a Region where IAM Identity Center isn't configured. The IAM Identity Center organization account must be in the same Region as SageMaker Studio. You can also onboard to Amazon SageMaker Domain using the standard setup procedure for IAM authentification.Issues related to using different RegionsYou might get an error similar to the following:You're attempting to set up Studio with SSO authentication in US East (N. Virginia). Your AWS SSO is configured in US East (N. Virginia). You must set up Studio in the same Region as AWS SSO or choose IAM authentication.You might get this error even if you configured Studio and IAM Identity Center in the same Region. This error occurs if you didn't turn on IAM Identity Center in the management account or member accounts in your organization. This error is usually caused by the AccessDenied error that results when IAM Identity Center doesn't provide the sso:ListDirectoryAssociations permissions. You can resolve this issue by turning on the IAM Identity Center for the management account or member accounts.Follow"


Question: Why is my Lightsail load balancer failing the health check even though the website is running correctly?

I'm using an Amazon Lightsail load balancer for my Amazon Lightsail instance with a Bitnami stack. Why is the load balancer health check failing even though the website is running correctly? How can I prevent the load balancer health check from failing?

Answer: "I'm using an Amazon Lightsail load balancer for my Amazon Lightsail instance with a Bitnami stack. Why is the load balancer health check failing even though the website is running correctly? How can I prevent the load balancer health check from failing?Short descriptionLightsail load balancer performs the health check by checking the response of the URL http://ipaddress:80/healthcheckpath. The health check passes if the response status code is 200 OK. This response check isn't customizable in Lightsail load balancer. If your instance enforces HTTPS redirection, then http://ipaddress:80/healthcheckpath returns the 301 or 302 response status code instead of 200 OK. This leads to health check failure.The same issue can occur on WordPress Multisite instances because these instances redirect the URL http://ipaddress:80/healthcheckpath to http://ipaddress.nip.io/healthcheckpath by default.ResolutionNote: The file paths in the following resolution steps might change depending on whether the Bitnami stack uses native Linux system packages (Approach A), or if it's a self-contained installation (Approach B). To identify your Bitnami installation type, run the following command:test ! -f "/opt/bitnami/common/bin/openssl" && echo "Approach A: Using system packages." || echo "Approach B: Self-contained installation."The steps used to resolve this issue differ depending on the following:Redirection is set up using WordPress application plugins, such as Really Simple SSL.Redirection is set up using web server redirect rules.You're using a WordPress Multisite stack instance.Redirection is set up using WordPress application pluginsCreate an HTML file in the document root of your website. Then modify the load balancer health check configuration to add that file as the health check file. You must use this method because application level redirections don't normally affect HTML files that aren't originally part of your application.1.    Connect to your Lightsail instance.2.    Navigate to the Document root location of your website where you have stored your website files.In Bitnami stack under Approach A, the document root location is /opt/bitnami/APPNAME/ (for example, /opt/bitnami/wordpress).In Bitnami stack under Approach B, the document root location is /opt/bitnami/apps/APPNAME/htdocs (or example, /opt/bitnami/apps/wordpress/htdocs).In LAMP Bitnami stack, the document root location is /opt/bitnami/apache2/htdocs.3.    Create an empty HTML file either by uploading it or by running the following command:touch health.html4.    Go to the Lightsail home page, and then choose Networking.5.    Choose your load balancer.6.    On the Target instances tab, choose Customize health checking.7.    Type the path health.html and then choose Save.8.    Make sure that http://ipaddress:80/health.html returns the 200OK response using an HTTP Header Checker.9.    Wait for a few minutes, and then verify that the health check passes.Redirection is set up using web server redirect rulesAdd an exception rule for the health check file in the webserver redirect rules so that only the original website files redirect, but not the health check file.1.    Follow the steps 1 through 7 in the Redirection is set up using WordPress application plugins section.2.    Open the web server file where you added HTTPS redirection rules and then add the following line just before the line that starts with RewriteRule:RewriteCond expr "! %{REQUEST_URI} -strmatch '*health.html’ "The following are example redirect rules:RewriteEngine OnRewriteCond %{HTTPS} offRewriteCond expr "! %{REQUEST_URI} -strmatch '*health.html’ "RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI}Make the preceding change to the rules in the following locations:Bitnami stack under Approach A: /opt/bitnami/apache2/conf/bitnami/bitnami.conf and also any file ending with the prefix -vhost.conf in the /opt/bitnami/apache2/conf/vhosts/ directory.Bitnami stack under Approach B: /opt/bitnami/apache2/conf/bitnami/bitnami.conf.3.    Restart the web service.sudo /opt/bitnami/ctlscript.sh restart4.    Make sure that http://ipaddress:80/health.html returns the 200OK  response using this HTTP Header Checker.5.    Wait for a few minutes, and then verify that the health check passes.You're using a WordPress Multisite stack instanceWordPress Multisite redirects the URL http://ipaddress:80/healthcheckpath to http://ipaddress.nip.io/healthcheckpath by default. To fix this issue, do the following:1.    Follow the steps 1 through 7 in the Redirection is set up using WordPress application plugins section.2.    Open the file /opt/bitnami/apache2/conf/vhosts/wordpress-vhost.conf and add the following line under the section # BEGIN nip.io redirection:RewriteCond expr "! %{REQUEST_URI} -strmatch '*health.html' "The following is an example of rules with the line added:# BEGIN nip.io redirectionRewriteEngine OnRewriteCond %{HTTP_HOST} ^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})(:[0-9]{1,5})?$RewriteCond expr "! %{REQUEST_URI} -strmatch '*health.html'"RewriteRule ^/?(.*) %{REQUEST_SCHEME}://%1.nip.io%2/$1 [L,R=302,NE]# END nip.io redirection3.    Restart the web service.sudo /opt/bitnami/ctlscript.sh restart4.    Make sure that http://ipaddress:80/health.html returns the 200OK response using this HTTP Header Checker.5.    Wait for a few minutes, and then verify that the health check passes.Follow"


Question: Why am I being charged for Amazon EBS when all my instances are stopped?

"I've stopped all my Amazon Elastic Compute Cloud (Amazon EC2) instances, but I'm still receiving charges for Amazon Elastic Block Store (Amazon EBS) storage. How can I stop Amazon EBS charges accruing on my account for instances that I'm not using?"

Answer: "I've stopped all my Amazon Elastic Compute Cloud (Amazon EC2) instances, but I'm still receiving charges for Amazon Elastic Block Store (Amazon EBS) storage. How can I stop Amazon EBS charges accruing on my account for instances that I'm not using?ResolutionYou are charged for Amazon EBS storage for the amount of storage provisioned to your account, measured in "gigabyte-months." Amazon EC2 instances accrue charges only while they're running. However, EBS volumes attached to instances continue to retain information and accrue charges, even when the instance is stopped. For more information about how Amazon EBS is billed, see Amazon EBS pricing.Amazon EBS snapshots are billed at a lower rate than active EBS volumes are. You can minimize your Amazon EBS charges but retain the information stored in Amazon EBS for later use by creating a snapshot of the volume as a backup and then deleting the active volume. Later, when you need the information stored in the snapshot, you can replace the EBS volume using the snapshot for use with your infrastructure.To stop Amazon EBS-related charges, delete EBS volumes and Amazon EBS snapshots that you don't need.Related informationI've deleted snapshots of my EBS volume, and then I deleted the volume. Why aren't my data storage costs lower?Follow"


Question: How do I troubleshoot mobile text message (SMS) failures in Amazon SNS?

My mobile text messages (SMS messages) are failing to send through Amazon Simple Notification Service (Amazon SNS).

Answer: "My mobile text messages (SMS messages) are failing to send through Amazon Simple Notification Service (Amazon SNS).ResolutionTurn on and view Amazon CloudWatch Logs for your SMS messagesFor more information, see Viewing Amazon CloudWatch metrics and logs for SMS deliveries. Also, see How do I access Amazon SNS topic delivery logs for SMS messages?Check whether you exceeded your monthly SMS spending quota1.    Review your delivery status logs for failed SMS message deliveries ("status": "FAILURE").2.    Check the failed SMS delivery logs for the following value: "providerResponse": "No Quota left for account". If you see this value in the failed SMS delivery logs, then you exceeded your monthly SMS spending quota.Failed SMS message delivery CloudWatch log event example{    "notification": {        "messageId": "efgh9a7-c68f-abcd-bdbe-21ada245e18f",        "timestamp": "2020-10-21 16:32:10.981"    },    "delivery": {        "destination": "+1234567890",        "smsType": "Transactional",        "providerResponse": "No quota left for account",        "dwellTimeMs": 145    },    "status": "FAILURE"}Note: You can also check the SMSMonthToDateSpentUSD metric in your CloudWatch metrics to confirm whether you exceeded your monthly SMS spending quota.3.    If you exceeded your monthly SMS spending quota, then request a spending limit increase for SMS messages in Amazon SNS. For more information, see Requesting a spending limit increase for SMS messages in Amazon SNS.Important: After you receive an SMS spending quota increase confirmation, you must also adjust the spending quota for your account on the Amazon SNS console. If you don't update your SMS settings on the Amazon SNS console, then your SMS spend limit doesn't increase.If you use an AWS SDK to send SMS messages, then check whether you exceeded your maximum message price1.    Review your delivery status logs for failed SMS message deliveries ("status": "FAILURE").2.    Check the failed SMS delivery logs for the following value: "providerResponse": "delivery would exceed maxprice". If you see this value in the failed SMS delivery logs, then you exceeded the price limit that's configured in your AWS.SNS.SMS.MaxPrice message attribute.Note: You can also check the PublishSize metric in your CloudWatch metricsto confirm if you exceeded your message size limit. If Amazon SNS determines that sending the message incurs a cost that exceeds the maximum price that you set, then it doesn't send the message.3.    If you exceeded your maximum message price limit, then either increase your AWS.SNS.SMS.MaxPrice value or reduce the size of your SMS message. For more information, see Publishing to a mobile phone.Confirm that your account is approved to send SMS messages1.    Review your delivery status logs for failed SMS message deliveries ("status": "FAILURE").2.    Check the failed SMS delivery logs for the following value: "providerResponse": "Your account requires additional review prior to being able to send SMS messages". This value means that the Amazon SNS team doesn't have the necessary information to allow your account to send SMS messages.Note: To prevent abuse, Amazon SNS requires customers to have their spending limit and use case approved before sending SMS messages.3.    If your account doesn't have approval to send SMS messages, then open a Service limit increase case in the AWS Support Center. For more information, see Requesting a spending limit increase for SMS messages in Amazon SNS.Confirm that your originating identity is Regionally supportedThe country or AWS Region that you're sending messages to must support the originating identities that you're using.For example, to send messages to US destination phone numbers, you must have a supported US origination number (TFN/10DLC/ShortCode) in your Region. Or, to send messages to CA destination phone numbers, you must have a supported CA origination number (LongCode/ShortCode) in your Region.For a complete list of countries that support and don't support each type of originating identity, see Supported Regions and countries.Follow"


Question: How do I move users between WorkSpaces bundles?

I want to move users between Amazon WorkSpaces bundles. How can I do this?

Answer: "I want to move users between Amazon WorkSpaces bundles. How can I do this?ResolutionYou can move an existing WorkSpaces bundle to a destination bundle by using the migrate WorkSpaces functionality. User volume data is retained in the WorkSpaces migration process.Note: Migration of WorkSpaces across AWS Regions isn't supported. For more information about the limitations of WorkSpaces migration, see Migration limits.Related informationMigrate a WorkSpaceAmazon WorkSpaces migrateFollow"


Question: How can I protect my Amazon EC2 instance with AWS WAF?

I want to protect my Amazon Elastic Compute Cloud (Amazon EC2) instance from common web exploits and bots that can compromise security or consume excessive resources.

Answer: "I want to protect my Amazon Elastic Compute Cloud (Amazon EC2) instance from common web exploits and bots that can compromise security or consume excessive resources.Short descriptionAWS WAF can’t be directly associated with an Amazon EC2 instance. Instead, register your Amazon EC2 instances as a target for your Application Load Balancer (ALB) and then associate AWS WAF with the ALB. To protect your Amazon EC2 instance with AWS WAF, do the following:Create a target group and add your Amazon EC2 instance as its target.Configure a load balancer and a listener.Associate your web ACL with the ALB in AWS WAF.ResolutionCreate a target group and add the Amazon EC2 instance as its targetOpen the Amazon EC2 console.In the navigation pane, under Load Balancing, choose Target Groups.Choose Create target group.In the Basic Configuration section, set the following parameters:For Choose a target type, select Instance to specify targets by instance ID.For Target group name, enter a name for the target group.Modify the Port and Protocol as needed.For VPC, select a virtual private cloud (VPC) where your EC2 instance is located.For Protocol version, select HTTP1 when the request protocol is HTTP/1.1, select HTTP2 when the request protocol is HTTP/2, or select gRPC when the request protocol is gRPC.Choose Next.For Register targets, select one or more instances, enter one or more ports, and then choose Include as pending below.Choose Create target group.Configure a load balancer and listenerFor instructions, see Configure a load balancer and a listener.Associate your web ACL with the ALB in AWS WAFOpen the AWS WAF console.In the navigation pane, choose Web ACLs.For Region, select the Region where you created your web ACL.Note: Select Global if your web ACL is set up for Amazon CloudFront.Select your web ACL.Choose Add AWS resources.Choose the ALB you created to associate this web ACL with.Choose Add.Follow"


Question: "How can I store an Amazon Athena query output in a format other than CSV, such as a compressed format?"

"I want to store Amazon Athena query results in a format other than CSV, such as JSON or a compressed format."

Answer: "I want to store Amazon Athena query results in a format other than CSV, such as JSON or a compressed format.ResolutionAthena supports CSV output files only. The UNLOAD query writes query results from a SELECT statement to the specified data format. Supported formats for UNLOAD include Apache Parquet, ORC, Apache Avro, and JSON. CSV is the only output format used by the Athena SELECT query, but you can use UNLOAD to write the output of a SELECT query to the formats that UNLOAD supports. Although you can use the CTAS statement to output data in formats other than CSV, those statements also require the creation of a table in Athena. The UNLOAD statement is useful when you want to output the results of a SELECT query in a non-CSV format but don't require the associated table. For example, a downstream application might require the results of a SELECT query to be in JSON format, and Parquet or ORC might provide a performance advantage over CSV if you intend to use the results of the SELECT query for additional analysis.To store query output files in a different format, use a CREATE TABLE AS SELECT (CTAS) query, and then configure the format property. After the query completes, drop the CTAS table. Keep the following in mind:You can set format to ORC, PARQUET, AVRO, JSON, or TEXTFILE. If you don't specify a format for the CTAS query, then Athena uses Parquet by default.The name of the parameter, format , must be listed in lowercase, or your CTAS query fails. For an example, see Example: Writing query results to a different format.By default, all CTAS queries use GZIP compression. For Parquet and ORC, you can use the parquet_compression and orc_compression options to specify other compression types, such as SNAPPY. For an example, see Example: Specifying data storage and compression formats.For text-based formats, you can specify a field_delimiter like this: WITH (field_delimiter = ',') . If you don't specify a field delimiter, then Athena uses \001 by default.If you use the external_location parameter in the CTAS query, then be sure that the Amazon Simple Storage Service (Amazon S3) location has no data. Otherwise, you might see an error like this: "HIVE_PATH_ALREADY_EXISTS: Target directory for table 'table_name' already exists: s3://AWSDOC-EXAMPLE-BUCKET". For more information about the external_location parameter, see CTAS table properties.Related informationConsiderations and limitations for CTAS queriesFollow"


Question: How can I install CloudFormation helper scripts on Ubuntu 16.04 LTS/Ubuntu 18.04 LTS/Ubuntu 20.04 LTS/Ubuntu 22.04 LTS or RHEL 9 AMIs?

"I want to install AWS CloudFormation helper scripts on Ubuntu 16.04 LTS, Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, Ubuntu 22.04 LTS, and RHEL 9 Amazon Machine Images (AMIs)."

Answer: "I want to install AWS CloudFormation helper scripts on Ubuntu 16.04 LTS, Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, Ubuntu 22.04 LTS, and RHEL 9 Amazon Machine Images (AMIs).Short descriptionYou must install helper scripts for Ubuntu 16.04 LTS, Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, Ubuntu 22.04 LTS, and RHEL 9 AMIs during the boot process. Use the UserData property to download and install the AMIs. Then, start the cfn-hup service at launch. To get the latest binaries of the aws-cfn-bootstrap package, see CloudFormation helper scripts reference.Resolution1.    Review the templates for Ubuntu 16.04 LTS script, Ubuntu 18.04 LTS script, Ubuntu 20.04 LTS script, Ubuntu 22.04 LTS script, on RHEL 9 script on GitHub. These scripts create a service in systemd to allow the cfn-hup helper daemon to automatically start at boot.Important: Make systemd a dependency of an existing boot target. Use multi-user.target to create the dependency.2.    Review how to define the UserData property of the instance.3.    Create the cfn-hup configuration file, cfn-hup hook, and systemd file for cfn-hup /lib/systemd/systemcfn-hup.service. Refer to the instructions that are set in the files key.4.    To activate the service in systemd and start the service, use the commands key.5.    To confirm that the cfn-hup service starts after it launches the stack, run the following command:systemctl status cfn-hupIf cfn-hup starts successfully, then you receive an output that's similar to the following one:cfn-hup.service - cfn-hup daemon   Loaded: loaded (/usr/lib/systemd/system/cfn-hup.service; enabled; vendor preset: disabled)   Active: active (running) since Wed 2016-10-12 08:10:26 EDT; 1min 11s ago MainPID: 4852 (cfn-hup)   CGroup: /system.slice/cfn-hup.service                       └─4852 /usr/bin/python /opt/aws/bin/cfn-hupOct 12 08:10:26 ip-172-31-44-180.ec2.internal systemd[1]: Started cfn-hup daemon. Oct 12 08:10:26 ip-172-31-44-180.ec2.internal systemd[1]: Starting cfn-hup daemon...Note: If you use systemd, then the cfn-init helper script doesn't support the services key. Instead, use the commands key as shown in the previous example.Related informationCFNhelpernonAMZNAMIs (GitHub)Follow"


Question: How can I add multiple Firewall Manager admininstrator accounts?

I want to add multiple AWS Firewall administrators to manage firewall resources in my AWS Organizations organization.

Answer: "I want to add multiple AWS Firewall administrators to manage firewall resources in my AWS Organizations organization.Short descriptionYou can have up to 10 Firewall Manager administrators to manage the firewall resources of your organization. For example, you can have one administrator manage a set of organizational units (OUs) for your organization. You can also delegate another administrator to manage only specific Firewall Manager policy types.ResolutionFollow these steps to create another Firewall Manager administrator account to manage the security policies for your organization.Note: If you didn't set up Firewall Manager for an AWS account, then first set one up.1.    Open the Firewall Manager console with your management account.2.    In the navigation pane, expand AWS Firewall Manager, and then choose Settings. Note the Default administrator account. This is the default Firewall Manager account is the first administrator account that was created.3.    In Settings, choose Create administrator account.4.    In Administrator account ID, enter the AWS account to set as another Firewall Manager administrator.5.    Choose the Administrative Scope dropdown menu, and then choose either Full or Restricted.Note: Full scope gives the administrator full access to all of the preceding resource types. Restricted scope gives administrative permission to only a subset of the preceding resources. It's a best practice to grant administrators only the necessary permissions to perform the duties of their role.6.    Choose Create administrator account.For more information, see Working with AWS Firewall Manager administrators.Related InformationManaging the AWS accounts in your organizationAWS Firewall Manager prerequisitesFirewall Manager quotasFollow"


Question: How can I understand AWS Config billing by retrieving the number of configuration items recorded per month?

I want the number of configuration items recorded by AWS Config.

Answer: "I want the number of configuration items recorded by AWS Config.ResolutionTo help you identify previous and current billing trends, use Amazon Athena to identify the number of configuration items (CI) per month for your account.Verify that your S3 bucket contains configuration filesNote: Be sure that the AWS Config service is able to deliver configuration history files successfully to your designated Amazon Simple Storage Service (Amazon S3) bucket. Typically, CIs are delivered to the bucket as configuration history files every 6 hours. For more information, see Viewing configuration details.Open the AWS Config console, and then choose Settings from the navigation pane.In the Amazon S3 bucket section, note the Bucket name.Open the S3 console. Then, for Bucket name, choose your S3 bucket. Verify that the S3 bucket you choose contains configuration files.Note: If there are no configuration files, then your role might be missing permissions. For more information, see Identity and access management in Amazon S3.Create a table in AthenaSign in to the Athena console, and then follow the instructions to create a table using the wizard.Use the following table syntax:CREATE EXTERNAL TABLE awsconfig (         fileversion string,         configSnapshotId string,         configurationitems ARRAY < STRUCT < configurationItemVersion : STRING,         configurationItemCaptureTime : STRING,         configurationStateId : BIGINT,         awsAccountId : STRING,         configurationItemStatus : STRING,         resourceType : STRING,         resourceId : STRING,         resourceName : STRING,         ARN : STRING,         awsRegion : STRING,         availabilityZone : STRING,         configurationStateMd5Hash : STRING,         resourceCreationTime : STRING > > ) ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe' LOCATION 's3://<BUCKET-NAME>/AWSLogs/<ACCOUNT-ID>/Config/<REGION>/';For LOCATION, use the location and Region for the AWS Config items stored in your Amazon S3 bucket. For BUCKET-NAME, ACCOUNT-ID, and REGION, use your specific information.Note: If you're using Athena engine version 2, then the maximum line length for text files is 100 MB. If you have a large number of resources, then Config items stored in the designated AWS Config S3 bucket might exceed this limit. For example, AWS Config also delivers configuration snapshot files in the same bucket LOCATION, and the configuration snapshot file could exceed this limit. If you exceed the limit, then when querying the AWS Config item, you receive an error that looks similar to the following:HIVE_BAD_DATA: Line too long in text file: <s3_path_to_config_data_object>In this case, use the following table syntax to point Amazon Athena to directly query the S3 path that stores the configuration history files instead:CREATE EXTERNAL TABLE awsconfig (         fileversion string,         configSnapshotId string,         configurationitems ARRAY < STRUCT < configurationItemVersion : STRING,         configurationItemCaptureTime : STRING,         configurationStateId : BIGINT,         awsAccountId : STRING,         configurationItemStatus : STRING,         resourceType : STRING,         resourceId : STRING,         resourceName : STRING,         ARN : STRING,         awsRegion : STRING,         availabilityZone : STRING,         configurationStateMd5Hash : STRING,         resourceCreationTime : STRING > > ) PARTITIONED BY (`year` string,`month` string,`day` string)ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'LOCATION 's3://<BUCKET-NAME>/AWSLogs/<ACCOUNT-ID>/Config/<REGION>/'TBLPROPERTIES (  'projection.enabled'='true',   'projection.year.interval'='1',   'projection.year.range'='2021,2121',   'projection.year.type'='integer',   'projection.month.interval'='1',   'projection.month.range'='1,12',   'projection.month.type'='integer',   'projection.day.interval'='1',   'projection.day.range'='1,31',   'projection.day.type'='integer',   'storage.location.template'='s3://<BUCKET-NAME>/AWSLogs/<ACCOUNT-ID>/Config/<REGION>/${year}/${month}/${day}/ConfigHistory/')For LOCATION, use the location and Region for the AWS Config items stored in your Amazon S3 bucket. For BUCKET-NAME, ACCOUNT-ID, and REGION, use your specific information. The following example table partitions the Athena table with partitions projection from /2021/1/1/ to /2121/12/31/. Customize this time period as needed.Note: The AWS Config data S3 path date format isn't compatible with Athena partitions projection date type format.Example Athena queriesThe following example query retrieves the number of configuration items per day in February 2021:SELECT result.configurationitemcapturetime,         count(result.configurationitemcapturetime) AS NumberOfChangesFROM     (SELECT regexp_replace(configurationItem.configurationItemCaptureTime,         '(.+)(T.+)', '$1') AS configurationitemcapturetime    FROM default.awsconfig    CROSS JOIN UNNEST(configurationitems) AS t(configurationItem)    WHERE "$path" LIKE '%ConfigHistory%'            AND configurationItem.configurationItemCaptureTime >= '2021-02-01T%'            AND configurationItem.configurationItemCaptureTime <= '2021-02-28T%') resultGROUP BY  result.configurationitemcapturetimeORDER BY  result.configurationitemcapturetimeThe results are similar to:configurationitemcapturetime    NumberOfChanges2021-02-02    72021-02-03    32021-02-07   11...The following example query retrieves the number of changes per resource in February 2021, sorted by most frequently changed:SELECT configurationItem.resourceType,         configurationItem.resourceId,         COUNT(configurationItem.resourceId) AS NumberOfChangesFROM default.awsconfigCROSS JOIN UNNEST(configurationitems) AS t(configurationItem)WHERE "$path" LIKE '%ConfigHistory%'        AND configurationItem.configurationItemCaptureTime >= '2021-02-01T%'        AND configurationItem.configurationItemCaptureTime <= '2021-02-28T%'GROUP BY  configurationItem.resourceType, configurationItem.resourceIdORDER BY  NumberOfChanges DESCThe results are similar to:resourcetype              resourceid        NumberOfChangesAWS::EC2::VPC             vpc-9ed00bfa        7AWS::EC2::Subnet          subnet-4472e248     5AWS::EC2::SecurityGroup   sg-450c6531         4Note: When comparing the total number of CIs between Athena query results and AWS billing data for the same month and Region, a discrepancy can occur. The data queried by Athena can cross day boundaries and also include CIs billed in adjacent months. AWS Config CIs are metered based on when the time configurationItemCaptureTime was initiated.As a best practice, increment the specified end day from end of month by one.For example, change this query:AND configurationItem.configurationItemCaptureTime <= '2021-02-28T%') resultTo this query:AND configurationItem.configurationItemCaptureTime <= '2021-03-01T%') resultNote: The last day of the month is incremented by one day.Related informationAWS Config pricingAmazon Athena pricingFollow"


Question: I'm using CloudFront with an Amazon S3 origin. Why am I not able to access my files?

I'm using an Amazon Simple Storage Service (Amazon S3) bucket as the origin of my Amazon CloudFront distribution but I’m not able to access or download files in my S3 bucket. How can I troubleshoot this issue?

Answer: "I'm using an Amazon Simple Storage Service (Amazon S3) bucket as the origin of my Amazon CloudFront distribution but I’m not able to access or download files in my S3 bucket. How can I troubleshoot this issue?ResolutionTroubleshoot 403 Access Denied errorsIf CloudFront returns 403 Access Denied errors, then see the following guides based on your use case:I’m using an S3 REST API endpoint as the origin of my CloudFront distribution. Why am I getting 403 Access Denied errors?I’m using an S3 website endpoint as the origin of my CloudFront distribution. Why am I getting 403 Access Denied errors?If you’re using an AWS WAF attached to your distribution, see How do I resolve "403 Error - The request could not be satisfied. Request Blocked" in CloudFront?Check your CloudFront geographic restrictionsIf your distribution is using geographic restrictions, verify that you’re not restricting the countries that your requests are originating from.For more information see Restricting the geographic distribution of your content.Verify that you’re not enforcing HTTPS with an S3 website endpointsAmazon S3 website endpoints don't support HTTPS. If you configured a CloudFront distribution with the CLI, API, or SDK, verify that you aren’t enforcing HTTPS with your S3 website endpoint origin.To check if you’re using HTTPS, use the GetDistributionConfig API or get-distribution-config CLI command to get the distribution configuration. If the OriginProtocolPolicy is set to https-only, then modify the OriginProtocolPolicy to use http-only.To use HTTPS, use CloudFront with a S3 REST API endpoint origin. For more information, see How do I use CloudFront to serve HTTPS requests for my Amazon S3 bucket?Confirm that your S3 objects are under 30 GBThe maximum allowed file size for HTTP GET, POST, and PUT requests is 30 GB. If a file is larger than 30 GB, you receive a 400 "BadRequest" error.For more information, see General quotas on distributions.Check for special characters in S3 object key namesIf you're using special characters in your S3 object key name, they might require special handling. For more information, see Object key naming guidelines.Confirm that you're not using default root objects from a subdirectoryA CloudFront distribution doesn’t return default root objects from a subdirectory, even if the object is configured in your distribution. For more information, see Specifying a default root object.As a workaround, you can use Lambda@Edge to return the default root object from a subdirectory. For more information, see Implementing default directory indexes in Amazon S3-backed Amazon CloudFront origins using Lambda@Edge.Confirm that you're not using host headers with an S3 originFor Amazon S3 origins, caching based on the Host header or forwarding the Host header isn't supported. For more information, see Selecting the headers to base caching on.Troubleshoot the "No 'Access-Control-Allow-Origin' header is present on the requested resource" errorSee the resolution steps in How do I resolve the "No 'Access-Control-Allow-Origin' header is present on the requested resource" error from CloudFront?Related informationTroubleshooting error responses from your originFollow"


Question: What's the difference between an AWS Organizations service control policy and an IAM policy?

What's the difference between an AWS Organizations service control policy (SCPs) and an AWS Identity and Access Management (IAM) policy? How can I use them together?

Answer: "What's the difference between an AWS Organizations service control policy (SCPs) and an AWS Identity and Access Management (IAM) policy? How can I use them together?ResolutionAWS Organizations SCPsAWS Organizations SCPs don't replace associating IAM policies within an AWS account.You can use SCPs to allow or deny access to AWS services for individual AWS accounts with AWS Organizations member accounts, or for groups of accounts within an organizational unit (OU). The specified actions from an attached SCP affect all IAM identities including the root user of the member account.AWS services that aren't explicitly allowed by the SCPs associated with an AWS account or its parent OUs are denied access to the AWS accounts or OUs associated with the SCP. SCPs associated to an OU are inherited by all AWS accounts in that OU.For more information, see Example service control policies.IAM policiesIAM policies allow or deny access to AWS services or API actions that work with IAM. An IAM policy can be applied only to IAM identities (users, groups, or roles). IAM policies can't restrict the AWS account root user.For more information, see Example IAM identity-based policies.For more information on how you can use IAM to secure access to your organization, see AWS Identity and Access Management and AWS Organizations.Related informationTutorial: Creating and configuring an organizationAWS Organizations terminology and conceptsFollow"


Question: How do I create a read replica for an Amazon RDS DB instance?

I want to scale up a read-heavy database. How do I create a read replica for my Amazon Relational Database Service (Amazon RDS) database instance?

Answer: "I want to scale up a read-heavy database. How do I create a read replica for my Amazon Relational Database Service (Amazon RDS) database instance?ResolutionTo create a read replica, see Creating a read replica. If you use PostgreSQL, see Working with PostgreSQL read replicas in Amazon RDS.Note: Creating a read replica in a different AWS Region can help you serve clients in different geographic regions more quickly. For more information, see Creating a read replica in a different AWS Region.It's a best practice to create a read replica when you don't expect a heavy database load. This is because the read replica creation process might cause I/O latency until it's completed. To reduce I/O latency during read replica creation and on your source RDS DB instance, you can use a High availability (Multi-AZ) for Amazon RDS.Note: To improve performance, configure your read replica so that it's identical to its source database. If you use the Amazon RDS console to create a read replica, the source database settings are used by default.Related informationWorking with read replicasFollow"


Question: Which security protocols and ciphers does CloudFront support?

Which security protocols and ciphers does Amazon CloudFront support?

Answer: "Which security protocols and ciphers does Amazon CloudFront support?ResolutionFor detailed information on supported cipher and protocol versions, see Supported protocols and ciphers between viewers and CloudFront.The configuration of each protocol version depends on the security policy that's selected. Security policies are pre-made. Specific ciphers suites cannot be added or removed from the security policy that you select. For more information, see Requirements for using SSL/TLS certificates with CloudFront.It's a best practice to use the most secure policy: TLSv1.2_2021. CloudFront attempts to establish the most secure connection. However, the level of security depends on the ciphers and protocols supported by the end user or client.Additionally, a security policy is selected only if a custom SSL certificate is used. If using the default SSL certificate, then the distribution defaults to TLSv1 as its security policy. For more information, see Security Policy.Note: CloudFront uses s2n-tls for Transport Layer Security (TLS) implementation. CloudFront supports TLS versions 1.0, 1.1, 1.2, and 1.3. By default, TLS version 1.3 is enabled on all CloudFront distributions.Related informationIntroducing s2n-tls, a New Open Source TLS ImplementationFollow"


Question: How can I share an encrypted Amazon EBS snapshot or volume with another AWS account?

"I want to share an Amazon Elastic Block Store (Amazon EBS) snapshot or volume with another AWS account. But, I don't know how."

Answer: "I want to share an Amazon Elastic Block Store (Amazon EBS) snapshot or volume with another AWS account. But, I don't know how.Short descriptionYou can't directly share an encrypted EBS volume with another AWS account. Instead, you must first create and share an encrypted Amazon EBS snapshot with the other account. Then, create a new volume from the copy of the shared snapshot to share the volume.When sharing EBS snapshots, keep the following in mind:You can't share snapshots that are encrypted with a default AWS Key Management Service (AWS KMS) key. Instead, you must encrypt the snapshot with a customer managed key.You can't share encrypted snapshots publicly. To share a snapshot publicly, make sure that it's not encrypted.For more information, see Before you share a snapshot.ResolutionNote: To complete the following steps, you must have permissions to edit volumes and snapshots. If you create a volume from an encrypted snapshot but don't see it on the volume list, then you might not have the correct permissions. Also, a snapshot that goes into an error state indicates that there is a permissions issue.Share a snapshot that's encrypted with a customer managed keyTo share a snapshot that's encrypted with a customer managed key, you must share the customer managed key that you used to encrypt the snapshot.First, follow the steps to share a snapshot. Then, follow the steps to share an AWS KMS key.Share a snapshot that's encrypted with a default AWS KMS keyYou can't change a snapshot's default encryption after you create the snapshot. Therefore, to share a snapshot that's encrypted with a default AWS KMS key, you must first create a copy of the snapshot. Then, encrypt the snapshot copy with a customer managed key. For more information, see Encryption and snapshot copying.If you don't have a customer managed key, then see Create an AWS KMS key in a custom key store (console).Share an encrypted EBS volume with another accountTo share an encrypted volume with another account, follow these steps.From the source account:Create a snapshot of the volume.Important: If the EBS volume is attached to an instance, then first stop the instance to maintain data consistency.Complete the steps in the preceding section, Share a snapshot that's encrypted with a customer managed key.From the target account:Create a copy of the shared snapshot.Note: Be sure to select your customer managed key. Otherwise, Amazon EBS encryption uses the default key. For more information on copying a snapshot see, Copy an Amazon EBS snapshot, and review the Prerequisites, Considerations, and Pricing.Create a volume from the snapshot.Note: You can't attach a volume to an Amazon Elastic Compute Cloud (Amazon EC2) instance that's in a different Availability Zone. Therefore, make sure to create the volume in the same Availability Zone where the Amazon EC2 instance is.Important: You can restore snapshots only in the AWS Region where you created the snapshot. For EBS volumes in another Region, copy the snapshot to that Region first, and then restore the snapshot.Follow"


Question: How do I run Athena queries for pretty-print JSON files?

"When I run Amazon Athena queries on pretty-print JSON files with JSON SerDe, my queries fail with the following error message: "HIVE_CURSOR_ERROR: Row is not a valid JSON Object"."

Answer: "When I run Amazon Athena queries on pretty-print JSON files with JSON SerDe, my queries fail with the following error message: "HIVE_CURSOR_ERROR: Row is not a valid JSON Object".ResolutionAthena JSON SerDe doesn't support pretty-print JSON files, such as the following file:{    "id": "b179db57-9d5b-40c8-8f07-f75bd1fa1b3c",    "random_number": 19,    "categoires": [        "category 1",        "category 2"    ],    "dict": {        "name": "category 1",        "comment": "category 2"    }}{    "id": "bc4624ce-d752-4e38-8a30-42de158dbc50",    "random_number": 95,    "categoires": [        "category 1",        "category 2",        "category 3",        "category 4"    ],    "dict": {        "name": "category 2",        "comment": "category 1"    }}{    "id": "1d3298e8-e3bb-41c0-950c-6a146555ce3c",    "random_number": 89,    "categoires": [        "category 1"    ],    "dict": {        "name": "category 5",        "comment": "category 3"    }}Because Amazon Ion is a superset of JSON, use the Amazon Ion Hive SerDe to query non-Amazon Ion JSON datasets. Unlike other JSON SerDe libraries, the Amazon Ion SerDe doesn't expect each row of data to be on a single line. Use this feature to query JSON datasets that are in pretty print format, or break up the fields in a row with newline characters.1.    Create a table that's similar to the following one with Athena Ion Hive SerDe:CREATE EXTERNAL TABLE table_name (    id string,    random_number int,    categories array<string>,    dict struct<name:string, comment:string>)STORED AS IONLOCATION 's3://sample-bucket/prefix/'2.    Query the table:SELECT * FROM table_nameThe output looks similar to the following output:#idrandom_numbercategoriesdict1b179db57-9d5b-40c8-8f07-f75bd1fa1b3c19[category 1, category 2]{name=category 1, comment=category 2}2bc4624ce-d752-4e38-8a30-42de158dbc5095[category 1, category 2, category 3, category 4]{name=category 2, comment=category 1}31d3298e8-e3bb-41c0-950c-6a146555ce3c89[category 1]{name=category 5, comment=category 3}Related informationWhy do I get errors when I try to read JSON data in Amazon Athena?Follow"


Question: What are SWAP files and why are they consuming space on my AWS DMS instance?

"The storage space on my AWS Database Migration Service (AWS DMS) DB instance is being consumed by swap files that have been generated on my instance. What are SWAP files, and how can I reduce swap file generation?"

Answer: "The storage space on my AWS Database Migration Service (AWS DMS) DB instance is being consumed by swap files that have been generated on my instance. What are SWAP files, and how can I reduce swap file generation?Short descriptionIn AWS DMS, the SORTER component caches ongoing changes that come from the source. The component does this to manage actions such as transaction merging, filtering, and data transformations before applying them to the target.If the source processing is over-performing the target apply, the target can't cope with incoming changes. Or, if long transactions exceed the maximum memory allocation (MemoryLimitTotal) and time (MemoryKeepTime), then the transactions in memory are spilled over to the replication instance disk, known as swap or sorter files.Swap files like this are different from the OS swap files that are created when server memory isn't sufficient for processing. So, this data isn't included in the SwapUsage metric in Amazon CloudWatch.ResolutionYou might see a message similar to this in your task logs:"[SORTER ]I: Reading from source is paused. Total storage used by swap files exceeded the limit 1048576000 bytes (sorter_transaction.c:110) 00000849: 2021-04-28T07:19:20:987696 [SORTER ]I: Reading from source is resumed (sorter_transaction.c:116)]"This is an information message, and not an error. You see this message when the swap files that are used by the AWS DMS SORTER component reaches a size limit of 1GB (hardcoded limit). AWS DMS versions 3.4.2 and later include a feature that prevents storage full scenarios by limiting the maximum amount of swap files you can have on a replication instance's storage volumes. So, you see this message when the swap files that are used by the DMS SORTER component reaches the 1 GB limit.When this 1 GB limit is reached, the SORTER component pauses SOURCE CAPTURE. AWS DMS then applies changes from swap files to the target database. After the swapped changes from disk are applied or when the swap file drops below 1 GB, AWS DMS resumes source capture.Note: When this happens, the task continues to run. Only SOURCE CAPTURE is paused.Reduce swap file generationUse these best practices to help reduce or avoid swap file generation on your AWS DMS instance:Check for the common causes of high target latency on your AWS DMS task:There are no primary keys or indexes in the targetThere are resource bottlenecks in the targetThere are resource bottlenecks in the replication instanceThere is a network issue between the replication instance and the targetUse batch apply mode and ParallelApplyThreads to improve target apply, based on your engine type.If you are experiencing long running transactions at the source, then use frequent commits at the source database to help mitigate the issue.If memory is available on the replication instance, then you can increase MemoryLimitTotal and MemoryKeepTime so that changes remain in memory for longer before they are swapped to the disk. Make sure that you increase both of these parameters, because after a limit is reached, AWS DMS swaps the changes to the disk.Related informationChange processing tuning settingsFollow"


Question: How do I explicitly allow file uploads that an AWS WAF rule blocks without excluding the rule?

I want to allow users to upload files using specific file extensions without having to block the current AWS WAF configuration rule.

Answer: "I want to allow users to upload files using specific file extensions without having to block the current AWS WAF configuration rule.Short descriptionHTTP file upload content is commonly one of the following types:Form-data: Data that's sent from website forms to APIs as part of a multipart form-data.Binary-data: A file that's anything other than a text payload. The binary file can be a JPEG file, a GZip file, or a PDF file.To find out why AWS WAF blocks a POST request, review the common rules that block file uploads. If a common rule isn't blocking the upload, then review the additional options for allowing blocked files.The following rules commonly block file uploads:CrossSiteScripting_BODYSQLi_BODYSizeRestrictions_BODYCustom rules that evaluate the request BODYResolutionDetermine which rule blocks the file uploadsComplete the following steps to determine which rule blocks the file upload:1.    Open the AWS WAF console.2.    Review the sampled web requests. The sampled requests contain both information on the rule that blocks the request and the HTTP request components.3.    On the Overview page, under Rule inside rule group, find the HTTP request components. The components look similar to the following examples:Rule inside rule groupawswaf:managed:aws:core-rule-set:CrossSiteScripting_BodyRequest`POST /upload`  `User-Agent: PostmanRuntime/7.30.0 Accept: */* Host: example.amazonaws.com Connection: keep-alive Content-Type: multipart/form-data; boundary=--------------------------421232031360350156757252 Content-Length: 4060737 `4.    Review the AWS WAF comprehensive logs for terminatingRuleMatchDetails.Note: The terminatingRuleMatchDetails field populates only for SQLi_BODY and CrossSiteScripting_BODY attacks.Identify the content type of the POST dataTo identify the content type, review the HTTP header in your POST data for content-type. In the previous example, the content type is multipart/form-data.Form-data content For form-data content that has multipart/form-data for the header value, complete the following steps:1.    Open the AWS WAF console.2.    In the navigation pane, under AWS WAF, choose Web ACLs.Note: The default selection for Region is US East (N. Virginia). Choose the AWS Region where you created your web ACL. Choose Global if your web ACL is set up for Amazon CloudFront.3.    Choose your Web ACL.4.    On the Rules tab, choose Add Rules, and then choose Add my own rules and rule groups.5.    On the Add rule screen, under Rule Type, choose Rule builder.For Name, enter a name to identify this rule.For Type, choose Regular rule.For If a request, choose matches the statement.For Inspect, choose Body.For Content type, choose JSON.For JSON match scope, choose Values.For How AWS WAF should handle the request if the JSON in the request body is invalid, choose the option appropriate for you.For Content to inspect, choose Full JSON content.For Match type, choose Matches regular expression.For Regular expression, paste the following sample regex-pattern:(?:.pdf|.doc|.docx|.ppt)(Optional) For Text transformation, choose either a text transformation or None. For more information, see Text transformations.For Oversize Handling, choose the option appropriate for your configuration. For more information, see Handling oversize web request components in AWS WAF.For Action, choose Allow. For more information, see Rule action.6.    Choose Add Rule.7.    For Set Rule Priority, select your rule and move it to a higher priority than the rule that blocks the request. AWS WAF evaluates the rules according to the priority that's set. For more information, see Processing order of rules and rule groups in a web ACL.8.    Choose Save. Make sure that the rules works as expected.Binary-data contentFor binary-data content that has a value similar to application/pdf or application/ppt for the header value, complete the following steps:1.    Open the AWS WAF console.2.    In the navigation pane, under AWS WAF, choose Web ACLs.Note: The default selection for Region is US East (N. Virginia). Choose the AWS Region where you created your web ACL. Choose Global if your web ACL is set up for Amazon CloudFront.3.    Choose your Web ACL.4.    On the Rules tab, choose Add Rules, and then choose Add my own rules and rule groups.5.    On the Add rule screen, under Rule Type, choose Rule builder.For Name, enter a name to identify the rule.For Type, choose Regular rule.For If a request, choose matches the statement.For Inspect, choose Single header.For Header field name, enter Content-Type.For Match type, choose Matches regular expression.For Regular expression, paste the following sample regex-pattern:(?:pdf|jpeg)(Optional) For Text transformation, choose either a Text transformation or None. For more information, see Text transformations.For Action, choose Allow. For more information, see Rule action.6.    Choose Add Rule.7.    For Set Rule Priority, select your rule and move it to a higher priority than the rule that blocks the request. AWS WAF evaluates the rules according to the priority that's set. For more information, see Processing order of rules and rule groups in a web ACL.8.    Choose Save. Make sure that the rule works as expected.Note: To restrict the rule for a specific upload URI path, use an AND statement inside the Web ACL rule.Related informationHow do I turn on AWS WAF logging and send logs to CloudWatch, Amazon S3, or Kinesis Data Firehose?Follow"


Question: Why did my Amazon RDS DB instance fail over?

My Amazon Relational Database Service (Amazon RDS) DB instance failed over. How do I identify why the DB instance failed over?

Answer: "My Amazon Relational Database Service (Amazon RDS) DB instance failed over. How do I identify why the DB instance failed over?Short DescriptionIf there is a planned or an unplanned outage for a Multi-AZ DB instance, Amazon RDS automatically switches to a standby replica or secondary instance in another Availability Zone. Depending on your database activity at the time of the time of the outage, failover usually lasts between 60-120 seconds. For more information, see Failover Process for Amazon RDS.ResolutionTo see if a failover has occurred, open the Amazon RDS console, and then choose Events from the navigation pane. If AWS CloudTrail logging is enabled, then you can check the logs to see whether the event was planned or unplanned. For example, scaling compute or applying pending OS upgrades can trigger a failover.If there are no events listed and there are no CloudTrail logs, the failover might be caused by one of the following:Availability Zone failuresPrimary DB instance compute node failuresNetworking issues with the primary DB instanceStorage or Amazon Elastic Block Store (Amazon EBS) volume issuesWhen these types of unplanned outages occur, Amazon RDS monitors and identifies the issue. Then, Amazon RDS fails over automatically to minimize downtime.Related InformationBest Practices for Amazon RDSFollow"


Question: Why is my AWS Glue workflow not triggered?

"I created an AWS Glue workflow, but it's not starting. Or some of the constituent jobs or crawlers in my AWS Glue workflow are not running."

Answer: "I created an AWS Glue workflow, but it's not starting. Or some of the constituent jobs or crawlers in my AWS Glue workflow are not running.Short descriptionIf your AWS Glue workflow or its components aren't triggered, then confirm the following:If a scheduled trigger is used as the source trigger, then confirm that it's activated. Be sure that the schedule is mentioned in Coordinated Universal Time (UTC), and that the cron expression includes all of the required fields.If an event-based trigger is used as the source trigger, then confirm that the Amazon EventBridge rule is correct, turned on, and linked to the workflow. Confirm that the role that's used to add the workflow as a target has the necessary permissions to pass events to AWS Glue.If an external component is triggering the source trigger, then confirm that the external component isn't malfunctioning.Be sure that the predicate condition used to trigger a component isn't met by an agent external to the workflow.The component might be part of a dependency chain. Be sure that the upstream jobs/crawlers are started as part of the same workflow by a single source trigger.ResolutionWorkflow not starting with a time-based triggerIf the workflow's source trigger is scheduled, then check the following:Be sure that the trigger is in the ACTIVATED state and not in the CREATED state. If the trigger isn't in the ACTIVATED state, then activate the trigger manually.Be sure that the cron expression used in the schedule for a scheduled trigger is in UTC. Be sure that the fields in the cron expression correspond with the conversion of the local time zone to UTC. Also, check if the cron expression includes all the required fields in the correct format. For more information, see Time-based schedules for jobs and crawlers.Workflow not starting with an on-demand triggerYour source trigger might be on-demand, with an upstream entity triggering it using the StartWorkflowRun API call. For this use case, be sure that the calling entity functions correctly.Workflow not starting with a conditional triggerBe sure that the predicate conditions in the trigger aren't met by an agent external to the workflow. If the conditions are met by an external agent, then the trigger isn't fired. Conditional triggers are started only if the watched event is started by a trigger.For example, suppose that the following conditions are true:You have a workflow with a job JOB_MAIN that's triggered by the trigger TEST_TR.The trigger TEST_TR is dependent on the completion of another job JOB_DEP that's not part of the current workflow.In this case, even if JOB_DEP completes successfully, and the trigger TEST_TR's predicate logic is met, the job JOB_MAIN isn't fired. The job isn't fired because the predicate condition is met by an agent that's not part of the same workflow.Workflow not starting with an event-based triggerYou might have configured EventBridge (also known as Amazon CloudWatch Events) to start your workflows. If you are using this setup, and notice that the source trigger is not being fired, then check the following:If you're using Amazon Simple Storage Service (Amazon S3) data events as the source, confirm that you've properly created a trail for the AWS account. Then check that the events for the concerned S3 bucket are being logged in AWS CloudTrail.Verify the EventBridge rule to confirm the eventSource, eventName, and requestParameters are as expected.Confirm that the EventBridge rule is attached to the correct workflow target ARN.Confirm that the EventBridge rule state is ENABLED.Confirm that the RoleArn used to attach the rule to the workflow target has CloudWatchEventsBuiltInTargetExecutionAccess and CloudWatchEventsInvocationAccess policies attached. Then, confirm that there are sufficient IAM permissions to do 'glue:NotifyEvent' on the workflow. The following example shows the minimum required AWS Glue policy:{    "Version": "2012-10-17",    "Statement": [{        "Effect": "Allow",        "Action": ["glue:NotifyEvent"],        "Resource": ["arn:aws:glue:<region>:<account-id>:workflow/<workflow-name>"]    }]}Workflow not starting for a component job or crawler that's part of a dependency chainCheck if the constituent job or crawler depends on the completion of an upstream job or crawler that's also started by a trigger. A dependent job or crawler is started only if the job or crawler that's completed was started by a trigger. Make sure that all jobs or crawlers in a dependency chain are descendants of a single scheduled or on-demand trigger.For example, suppose that these conditions are true:Your workflow starts with a trigger named TEST_TR1 that starts the job named JOB_1.Another trigger named TEST_TR2 depends on the completion of JOB_1 to start the job named JOB_2.In this case, TEST_TR2 starts JOB_2 when the predicate conditions for TEST_TR2 are met.But if JOB_1 is run on-demand, and not started by TEST_TR1, then TEST_TR2 doesn't start JOB_2 even if the predicate conditions for TEST_TR2 are met.Related informationAWS Glue triggersWorkflow restrictions in AWS GlueStarting an AWS Glue workflow with an Amazon EventBridge eventFollow"


Question: Why is my Application Migration Service or Elastic Disaster Recovery replication process stuck at 100% with the "Finalizing Initial Sync" message?

I use AWS Application Migration Service (AWS MGN) or AWS Elastic Disaster Recovery (AWS DRS). The replication process is stuck at 100% with a console that states "Finalizing Initial Sync".

Answer: "I use AWS Application Migration Service (AWS MGN) or AWS Elastic Disaster Recovery (AWS DRS). The replication process is stuck at 100% with a console that states "Finalizing Initial Sync".Short descriptionWhen the replication process is stuck at 100% during syncs for Application Migration Service or Elastic Disaster Recovery, you see the following errors:"Finalizing Initial Sync - Flushing Backlog""Finalizing Initial Sync - Creating First Launchable Snapshot"ResolutionTroubleshooting the "Finalizing Initial Sync - Flushing Backlog" errorWait until the backlog completes flushing for the sync to initialize.If the source machine is write intensive, then the backlog can increase in size. The machine might remain stuck in the Finalizing Initial Sync state on the Application Migration Service or Elastic Disaster Recovery console. If this occurs, then complete the following steps:Test the replication speed (on the CloudEndure website).Calculate the required bandwidth for all replicating source machines. Make sure that the network throughput of the replication instance is sufficient.Under Replication Settings, verify if Network bandwidth throttling is activated. If your configuration requires activating this option, then make sure that you set the value to at least the minimum required bandwidth. For more information, see the bandwidth throttling documentation for Application Migration Service or Elastic Disaster Recovery.Use Amazon CloudWatch metrics to check the network and disk utilization of the replication server. If a resource throttles the server, then use a dedicated replication server or a larger replication server type. Or, choose SSD-based storage. For more information, see Disk settings (Application Migration Service) or Disk settings (Elastic Disaster Recovery).To verify which replication server a specific source machine uses, run the netstat command on the source machine as shown in the following example.Note the remote IP address that the machine connects to over port 1500:netstat command for Linux:$ netstat -anp | grep ":1500"netstat command for Windows:netstat -ano | findstr ":1500"Or, review the agent.log.0 file on the source machine to identify the exact replication server in use:agent.log.0 for Linux:$ sudo cat /var/lib/aws-replication-agent/agent.log.0 | grep :1500 | tail -n 1agent.log.0 for Windows:findstr /L ":1500" "C:\Program Files (x86)\AWS Replication Agent\agent.log.0"Troubleshooting the Finalizing Initial Sync - Creating First Launchable Snapshot errorTo troubleshoot this error, complete one or more of the following steps:Verify that the user's AWS Identity and Access Management (IAM) policy has all required permissions to run the Amazon Elastic Compute Cloud (Amazon EC2) APIs. For more information, see How do I use CloudTrail to review what API calls and actions have occurred in my AWS account?Confirm that the replication server communicates with Amazon EC2 endpoints (https://ec2.{region}.amazonaws.com) within the AWS Region. For more information, see AWS service endpoints.Check for recent changes in Replication settings.Verify that you use the correct proxy settings.Confirm that the Application Migration Service or Elastic Disaster Recovery agent is working properly.Check for service quotas issues.Verify that the Application Migration Service or Elastic Disaster Recovery user's AWS IAM policy has all permissions to run the required Amazon EC2 APIsFor the Application Migration Service or Elastic Disaster Recovery user’s policy, see the required AWS credentials for Application Migration Service or Elastic Disaster Recovery. Or, you can view the AWS CloudTrail Event history to confirm any API failures for the configured user.Confirm that the replication server communicates with Amazon EC2 endpoints within the RegionLaunch a new Linux machine in the same subnet as your staging area.To test connectivity, log in to the new machine and run the following commands. In the following example commands, replace us-east-1 with your Region:$ dig ec2.us-east-1.amazonaws.com  $ telnet ec2.us-east-1.amazonaws.com 443  $ wget https://ec2.us-east-1.amazonaws.comIf any of these commands fail, then network connectivity issues exist. Proceed to the following section.Identify any network connectivity blockersVerify that the virtual private cloud (VPC), subnet, security group, network access control list (network ACL), and route table settings align with the Replication Settings. A misconfiguration might block communication to Amazon EC2 endpoints from the replication servers.If the replication server launches in a public subnet, then complete the following steps:Verify that the security group, network ACLs, and the route table allow communication with Amazon EC2 endpoints on TCP port 443.Verify that the enableDnsHostnames and enableDnsSupport attributes are set to true at the VPC level:$ aws ec2 describe-vpc-attribute --vpc-id vpc-a01106c2 --attribute enableDnsHostnames  {    "VpcId": "vpc-a01106c2",   "EnableDnsHostnames": {    "Value": true   }   }$ aws ec2 describe-vpc-attribute --vpc-id vpc-a01106c2 --attribute enableDnsSupport     {   "VpcId": "vpc-a01106c2",    "EnableDnsSupport": {   "Value": true    }  }If the replication server launches in a private subnet, then complete the following steps:Verify that the security group, network ACLs, and route table allow communication with Amazon EC2 endpoints on TCP port 443.If you configured a NAT gateway or instance in the route table, then verify that outbound traffic to the EC2 endpoint on TCP port 443 works.Check if outbound traffic passes through a transit or virtual private gateway. In this case, make sure that the route table allows traffic to EC2 endpoints on TCP port 443.Check if the firewall blocks communication.If the VPC has interface VPC endpoints, then make sure that communication occurs between Amazon EC2 endpoints on TCP port 443 through a private network. To do this, complete the following steps:Verify that the enableDnsHostnames and enableDnsSupport attributes are set to true at the VPC level. Verify that the PrivateDnsEnabled value is set to true on the VPC interface endpoints:$ aws ec2 describe-vpc-attribute --vpc-id vpc-a01106c2 --attribute enableDnsHostnames --query 'EnableDnsHostnames'  {    "Value": true  } $ aws ec2 describe-vpc-attribute --vpc-id vpc-a01106c2 --attribute enableDnsSupport --query 'EnableDnsSupport'  {    "Value": true  } $ aws ec2 describe-vpc-endpoints --vpc-endpoint-ids vpce-088d25a4bbf4a7abc --query 'VpcEndpoints[0].PrivateDnsEnabled'  trueCheck for recent changes in Replication SettingsSearch for the UpdateReplicationConfiguration API call to track changes to Replication Settings from the CloudTrail Event history. Then, use the source server to filter Resource name. For example, check if there's a not valid tag inserted in the Replication resources tags field. For a list of allowed characters, see Tag restrictions.Verify that you're using the correct proxy settingsIf your replication servers use a proxy server, then make sure that the proxy settings allow communication with Regional EC2 endpoints on TCP port 443.Make sure that the allowed list for SSL interception and authentication includes mgn.<region>.amazonaws.com for Application Service Migration and drs.<region>.amazonaws.com for Elastic Disaster Recovery. For more information, see Can a proxy server be used between the source server and the Application Migration Service console? Also, see Can a proxy server be used between the source server and the Elastic Disaster Recovery Console?Confirm that the Replication Agent works correctlyConfirm that the AWS Replication Agent works correctly on the source machine. You can check the Replication Agent logs for possible errors to help pinpoint any problems. The Replication Agent logs are located in the following file locations:Linux Replication Agent logs:/var/lib/aws-replication-agent/agent.log.0Windows Replication Agent logs:C:\Program Files (x86)\AWS Replication Agent\agent.log.0Check for Amazon EC2 service quota issuesService quota issues or API throttling and rate limit issues might prevent Application Migration Service or Elastic Disaster Recovery from creating the first launchable recovery snapshot. Check the CloudTrail Event history to determine if a service quota or bandwidth throttling issue exists.Follow"