Question: How can I troubleshoot the error "An error occurred while communicating with Amazon Athena" when connecting to Amazon Athena with a JDBC/ODBC driver?

I'm experiencing connection timeout issues when I use Amazon Athena with a Java Database Connectivity (JDBC)/Open Database Connectivity (ODBC) driver.-or-I am not able to see the Athena query results on my business intelligence tool.

Answer: "I'm experiencing connection timeout issues when I use Amazon Athena with a Java Database Connectivity (JDBC)/Open Database Connectivity (ODBC) driver.-or-I am not able to see the Athena query results on my business intelligence tool.ResolutionThe most common reason for this error is the connection issue between the host where you installed the JDBC/ODBC driver and Athena or AWS Glue endpoints.To troubleshoot this error, check the connectivity between the host that you installed on the JDBC/ODBC driver and Athena/AWS Glue.Connectivity to AthenaYour host can connect to Athena with either an Athena public service endpoint or an Athena private endpoint. For more information on setting the endpoint, see Command line options and Boto3 documentation.Athena uses port 443 to connect to the host.Athena's streaming API uses port 444 to stream the query results. When you use a JDBC/ODBC driver, Athena uses this port to stream the query results to the JDBC/ODBC driver installed on the client host. Therefore, unblock this port when you use a JDBC/ODBC driver to connect to Athena. If this port is blocked, your business intelligence tool might time out or fail to show query results when you run a query.Athena public region endpointIf you're using an Athena public service endpoint (example: athena.us-east-1.amazonaws.com), then be sure to update the endpoint based on the Region that you're trying to connect.If you're using Windows, test the connectivity to the Athena public endpoint using Telnet or Test-netconnection.Note: You can use similar commands for testing connectivity to port 444.telnet athena.us-east-1.amazonaws.com 443--or--Test-NetConnection -Computer_Name athena.us-east-1.amazonaws.com -Port 443If you're using Linux or MAC operating system, then run a command similar to the following to check the Athena connectivity.Note: You can use a similar command for testing connectivity to port 444.nc -v athena.us-east-1.amazonaws.com 443Athena private interface VPC endpointIf you're using an Athena private endpoint instead of the public service endpoint, then test the connectivity to the interface VPC endpoint.If you have endpoint specific DNS hostnames, then check the connectivity to the endpoint specific DNS hostname. If not, check the connectivity to the default DNS hostname.If you've turned on private DNS for the endpoint, then use **vpce-.athena.us-east-1.vpce.amazonaws.com**  for the hostname.If you've turned off private DNS for the endpoint, then use athena.us-east-1.amazonaws.com for the hostname.If you're using Windows, then use the following commands. Be sure to test the connectivity for port 444 in addition to testing port 443.Note: Use Test-NetConnection to test the connectivity if you have trouble running the telnet command.You can use these commands if you've turned off private DNS for the endpoint.telnet vpce-<name>.athena.us-east-1.vpce.amazonaws.com 443--or--Test-NetConnection -Computer_Name vpce-<name>.athena.us-east-1.vpce.amazonaws.com -Port 443If you're using Linux/MAC, then run a command similar to the following to check the Athena connectivity. If your connection is successful, then you might receive a message similar to "Connection to athena.us-east-1.amazonaws.com port 443 [tcp/https] succeeded".nc -v vpce-<name>.athena.us-east-1.vpce.amazonaws.com 443Use the appropriate JDBC connection URLs in your business tool configuration according to your private DNS configuration for your endpoint.Use the following connection string if you turned off the private DNS: jdbc:awsathena://vpce-.athena.us-east-1.vpce.amazonaws.com:443Use the following connection string if you turned on the private DNS: jdbc:awsathena://athena.us-east-1.amazonaws.com:443Be sure that the security group attached to your VPC endpoint allows traffic from the host where you installed the JDBC/ODBC driver.Be sure that port 444 isn't blocked. If you use an AWS PrivateLink endpoint to connect to Athena, then be sure that the security group attached to the AWS PrivateLink endpoint is open to inbound traffic on port 444. Athena uses port 444 to stream query results. If port 444 is blocked, then the results aren't streamed back to your client host. In such situations, you might receive an error message similar to "[Simba]AthenaJDBC An error has occurred. Exception during column initialization". This can also cause the business intelligence tool to stop responding and not display the query results.Connectivity to AWS GlueBe sure to check the connectivity between the host that you installed on the JDBC/ODBC driver to AWS Glue. Your host can connect to AWS Glue with either an AWS Glue public service endpoint or an AWS Glue private service endpoint. For more information, see AWS Glue service endpoints. Athena uses the AWS Glue Data Catalog to store and retrieve table metadata for the Amazon S3 data in your account. Therefore, your JDBC/ODBC driver needs to connect to the AWS Glue catalog successfully to get the table metadata information.If you're using an AWS Glue public service endpoint, then test the connectivity with a command similar to the following:Note: You might need to use Test-NetConnection or nc to check the port connectivity based on your Operating System.telnet glue.us-east-1.amazonaws.com 443If you're using an AWS Glue VPC interface endpoint, then test the connectivity with a command similar to the following:Note: Use the following command if you've turned off private DNS for the endpoint:telnet vpce-<name>.glue.us-east-1.vpce.amazonaws.com 443Use the following command if you've turned on private DNS for the endpoint:telnet glue.us-east-1.amazonaws.com 443Related informationConnecting to Amazon Athena with ODBC and JDBC driversFollow"


Question: How can I find which AWS API call is causing a "Rate exceeded" error?

"I receive a "Rate exceeded" error message when I try to create, update, or access an AWS service. How do I determine which AWS API call is causing the error?"

Answer: "I receive a "Rate exceeded" error message when I try to create, update, or access an AWS service. How do I determine which AWS API call is causing the error?Short descriptionTo determine which API call is causing a Rate exceeded error in your AWS account, do the following:1.    Create an Amazon Athena table for AWS CloudTrail logs.2.    Run an Amazon Athena query to find the API call that's associated with the Rate exceeded error.To troubleshoot the issue, see the To troubleshoot Rate exceeded errors based on the associated API call and AWS service section of this article.Note: To log API activity in your AWS account, you must first create a trail in CloudTrail. Any data events that occur in your account before you create a trail aren't logged by the trail.ResolutionCreate an Amazon Athena table for CloudTrail logsFor instructions, see Using the CloudTrail console to create an Athena table for CloudTrail logs.Create and run the Athena query to find the associated API call1.    Open the Athena console.2.    Choose the New query tab.3.    Copy and paste the following example query into the query text box, and then choose Run query:Important: Replace cloudtrail-table-name with the name of your CloudTrail table.select eventname, errorcode,eventsource,awsregion, useragent,COUNT(*) count FROM cloudtrail-table-name   where errorcode = 'ThrottlingException'AND eventtime between '2018-10-14T03:00:08Z' and '2018-10-23T07:15:08Z'group by errorcode,awsregion, eventsource, useragent, eventnameorder by count desc;4.    In the query output, identify the API calls listed in eventname and eventsource that are associated with the errorcode, "ThrottlingException".To troubleshoot Rate exceeded errors based on the associated API call and AWS serviceNote: Rate exceeded errors can occur with different throttling conditions, depending on the AWS service and API call that you're using.Refer to the following troubleshooting documentation for the API call and service that's associated with the Rate exceeded error that you're receiving:For GetResourceConfigHistory or ListDiscoveredResources API callsSee How can I troubleshoot AWS Config console error messages?For the PutMetricData API callSee How do I avoid throttling when I call PutMetricData in the CloudWatch API?For API calls related to AWS Auto ScalingSee My Auto Scaling API calls are getting throttled. What can I do to avoid this?For API calls related to an AWS Lambda functionSee How do I troubleshoot Lambda function throttling with "Rate exceeded" and 429 "TooManyRequestsException" errors?For API calls related to AWS Elastic BeanstalkSee How can I resolve API throttling or "Rate Exceeded" errors in Elastic Beanstalk?Note: You can also allow AWS API calls to retry after throttling occurs by adding a wait statement. For more information, see Error retries and exponential backoff in AWS.Related informationHow do I automatically create tables in Amazon Athena to search through AWS CloudTrail logs?Follow"


Question: How can I troubleshoot Direct Connect gateway routing issues?

How do I troubleshoot routing issues between my VPC and on-premises network involving Private VIF and AWS Direct Connect gateway with Virtual Private Gateway as the association?

Answer: "How do I troubleshoot routing issues between my VPC and on-premises network involving Private VIF and AWS Direct Connect gateway with Virtual Private Gateway as the association?Short descriptionI can't connect from an on-premises data center to Virtual Private Cloud (Amazon VPC) resources over a private virtual interface associated with a Direct Connect gateway and virtual private gateway.ResolutionFollow these troubleshooting steps for your virtual private gateway.Check the Amazon VPC subnet route table. Make sure that it has a static or propagated route entry for the on premises network pointing to the virtual private gateway.Be sure that the Direct Connect gateway is associated with the correct virtual private gateway.Make sure that the virtual private gateway has the allowed prefixes for the Direct Connect gateway entered as the entire VPC CIDR, or a CIDR wider than the VPC CIDR. Note: If you specify a CIDR less than the VPC CIDR, you won't receive a route on your gateway router.Make sure that your router is advertising the on premises prefix to AWS over the Border Gateway Protocol (BGP) session of the private VIF.Verify that the security group rules and the network ACLs allow traffic to and from the on premises network.Verify that the firewall rules on your router allow traffic from the Amazon VPC subnet CIDR.To troubleshoot transit virtual interfaces, see Why can't I connect to VPC resources over a transit virtual interface using a Direct Connect connection?Related informationHow do I configure routing for my Direct Connect private virtual interface?How can I control the routes advertised and received over the AWS public virtual interface with Direct Connect?Follow"


Question: How do I find missing call recordings in Amazon Connect?

"I'm trying to find a call recording in Amazon Connect, but the recording doesn't exist."

Answer: "I'm trying to find a call recording in Amazon Connect, but the recording doesn't exist.Short descriptionAmazon Connect lets you store call recordings of customer interactions in Amazon Simple Storage Service (Amazon S3). To generate a call recording in the Amazon Connect console, you must meet the following conditions:You turn on call recording for the agent or customer in the Set recording and analytics behavior flow block.You configure the Amazon S3 bucket to store the call recording.The agent and customer are connected on the call interaction, and the call is disconnected.If you meet these conditions and you can't locate the call recording, then review the troubleshooting in the Resolution section.ResolutionComplete the following steps to verify that the call recording is present in Amazon Connect:Open the Amazon Connect console using an AWS account that has permissions to review call recording.From the navigation pane, choose Analytics, and then choose Contact Search.On the Contact search page, review the Recording/Transcript column to confirm that a call recording was generated.If a call recording is present, then select the Contact ID to verify that the recording is attached to the contact trace record (CTR).Recording isn't created in Amazon ConnectIf Amazon Connect didn't create the call recording, then review the following steps to troubleshoot.Check the contact flow configurationIn the contact flow log, review the Set recording and analytics behavior contact flow block. Verify that the Set recording and analytics behavior activates the call recording before Amazon Connect connects the customer to an agent for the interaction.{  "ContactId": "12ef1cf2-922b-9865-b18b-362564cc9be8",  "ContactFlowId": "xyz:aws:connect:alocation:123:instance/112232-c123-12bc-345a-d7e8a9b1bf23/contact-flow/b70334ed-8847-4efd-b2a7-cbbbd6da330c",  "ContactFlowName": "Sample inbound flow (first contact experience)",  "ContactFlowModuleType": "SetRecordingBehavior",  "Timestamp": "2022-06-21T17:15:51.966Z",  "Parameters": {   "RecordingBehaviorOption": "Enable",   "RecordingParticipantOption": "All"<br>  }}The following are common scenarios where calls aren't recorded:Example 1: If you add a Set recording and analytics behavior flow block after a Transfer to queue flow block, then Amazon Connect doesn't record the call. This happens because the contact exits the contact flow after it reaches the Transfer to queue flow block. In this scenario, the contact doesn't reach to the Set recording and analytics behavior flow block. This means that Amazon Connect doesn't generate a recording.Example 2: There's an external transfer. You turned on the Set recording and analytics behavior flow block in the contact flow, but Amazon Connect doesn't record the contact. To verify this, trace the contact flow logs for information on Transfer to phone number.Check the Amazon S3 bucket policyFor instances that you created before October 2018, review your S3 bucket policy to confirm that Amazon Connect instances have the appropriate permissions for the assumed role.Review the AWS CloudTrail logs to verify whether the Amazon Connect instance assumed role is getting an access denied error. The user or application that's trying to access the S3 bucket must have appropriate permissions.Validate the AWS Identity and Access Management (IAM) user’s permissions, and update the policy to grant permissions.For instances that your created after October 2018, Amazon Connect uses service-linked roles to access Amazon S3 and provides access to one S3 bucket at a time.Recording is created in Amazon Connect, you can't locate it in Amazon S3Note: You must have access to the destination S3 bucket to review the recordings.If Amazon Connect created the call recordings, but you can't locate the recordings in the S3 bucket, then take the following actions:Confirm the recording nameMost call recordings for specific contact IDs are named with the contact ID prefix itself. However, the contact IDs and name of the contact recording file don't always match. On the Contact search page, use the Contact ID to search for call recordings. To find the correct recording, refer to the audio file that's on the contact’s record.Recording name example:87a833e6-beec-46b3-bddd-fea0f190b5f7_20211104T09:06_UTC.wavImportant: Call recordings for customer callback aren't named with the contact ID prefix.Check whether the S3 bucket changedIf the S3 bucket information changed, then the instance doesn't have permissions to access the previous S3 bucket.For example, You use "bucket-A" in January and “bucket-B” in February. Amazon S3 stores all calls and recordings that Amazon Connect creates after February 1 in "bucket-B". You can see the calls and recordings from January in "bucket-A", but you can't access them because they're not in "bucket-B".Check the CallRecordingUploadError metricThe CallRecordingUploadError Amazon CloudWatch metric records the number of call recordings that failed to upload to the S3 bucket that's configured for your instance.Note: CallRecordingUploadError retries within 24 hours and doesn't always indicate complete failure. As each retry fails, a new data point appears in the metric. If you can't locate the recording after 24 hours, then open a support case. Provide the contact ID, contact flow logs, and CTR screenshots for review in your support case.Follow"


Question: How can I prevent the AWS Glue crawler from creating multiple tables?

"The AWS Glue crawler is creating multiple tables from my source data, and I want to prevent this from happening."

Answer: "The AWS Glue crawler is creating multiple tables from my source data, and I want to prevent this from happening.Short descriptionDuring the first AWS Glue crawler run, the crawler reads either the first 1000 records or the first megabyte of every file. It does this to infer the schema. The amount of data that the crawler reads depends on the file format and availability of a valid record.For example, if the input is a JSON file, then the crawler reads the first 1 MB of the file to infer the schema. If the crawler reads a valid record within the first 1 MB of the file, then the crawler infers the schema. If the crawler can't infer the schema after reading the first 1 MB, then it reads a maximum of 10 MB of the file. It does this in 1-MB increments.For CSV files, the crawler reads either the first 1000 records or the first 1 MB of data, whichever comes first. For Parquet files, the crawler infers the schema directly from the file.The crawler compares the schemas that it infers from all the subfolders and files, and then creates one or more tables.The AWS Glue crawler creates multiple tables when your source data files don't use these same configurations:Format (such as CSV, Parquet, or JSON)Compression type (such as SNAPPY, gzip, or bzip2)SchemaStructure of Amazon Simple Storage Service (Amazon S3) partitionsResolutionCheck the crawler logsCheck the crawler logs to identify the files that are causing the crawler to create multiple tables:1.    Open the AWS Glue console.2.    In the navigation pane, choose Crawlers.3.    Choose the crawler that you want to review the logs for.4.    Choose the Logs link to view the logs on the Amazon CloudWatch console.5.    If AWS Glue created multiple tables during the previous crawler run, then the log includes entries similar to the following entries:[439d6bb5-ce7b-4fb7-9b4d-805346a37f88] INFO : Created table 2_part_00000_24cab769_750d_4ef0_9663_0cc6228ac858_c000_snappy_parquet in database glue[439d6bb5-ce7b-4fb7-9b4d-805346a37f88] INFO : Created table 2_part_00000_3518b196_caf5_481f_ba4f_3e968cbbdd67_c000_snappy_parquet in database glue[439d6bb5-ce7b-4fb7-9b4d-805346a37f88] INFO : Created table 2_part_00000_6d2fffc2_a893_4531_89fa_72c6224bb2d6_c000_snappy_parquet in database glueThe log entries include the names of the files that are causing the crawler to create multiple tables.Prevent creation of multiple tablesTo prevent the creation of multiple tables, take one of the following actions:Confirm that the all data files use the same schema, format, and compression typeIn some cases, files use different schemas. For example, schema A might say that field X is type INT, and schema B says that field X is type BOOL. In this use case, take one of the following actions:Use the from_options function to run an AWS Glue extract, transform, and load (ETL) job to read the outlier data. Then, transform the outlier data types to the correct or most common data types in your source.Use the existing table DDL to manually create the table in Amazon Athena. Then, run an AWS Glue crawler to update the table metadata. Use the crawler setting to inherit the schema from the table.Combine compatible schemas when you create the crawlerThe crawler infers the schema at folder level and compares the schemas across all folders. The crawler checks if the schemas match and that the partition threshold is higher than 70 percent. If they match, then the schemas are denoted as partitions of a table. If they don’t match, then the crawler creates a table for every folder, resulting in a higher number of tables.Your data might have different schemas in some input files and similar schemas in other files. When you create the crawler, you can combine compatible schemas. On the Configure the crawler's output page, under Grouping behavior for S3 data (optional), choose Create a single schema for each S3 path. When you activate this setting and the data is compatible, the crawler ignores the similarity of schemas when evaluating Amazon S3 objects in the path.For more information, see How to create a single schema for each Amazon S3 include path.Check if your input files have different Amazon S3 pathsWhen the structure inside the Amazon S3 prefix isn’t consistent, the crawler assumes each individual path as an individual table. The crawler then creates multiple tables. If your input files have different Amazon S3 structures or paths, then the crawler creates multiple tables. This is the crawler’s default behavior.For example, a crawler crawls an Amazon S3 path s3://doc-example-bucket/doc-example-key/doc-example-table with a partition structure similar to the following structure:s3://doc-example-bucket/doc-example-key/doc-example-table/dt=2020-08-20/doc-example-file1.csvs3://doc-example-bucket/doc-example-key/doc-example-table/dt=2020-08-21/dox-example-file2.csvs3://doc-example-bucket/doc-example-key/doc-example-table/dt=2020-08-22/doc-example-file3.csvYou add the following additional files to the previous three files:s3://doc-example-bucket/doc-example-key/doc-example-table/dox-example-file4.csvs3:// doc-example-bucket/doc-example-key/doc-example-table/doc-example-file5.csvThe crawler then creates five individual tables in another crawler run. This happens because the folder partition structure is inconsistent.To avoid this issue, review these best practices:Make sure that the schema is consistent, either manually or programmatically. In the preceding example, either delete the Amazon S3 files without the partition dt=xxxx-xx-xx, or add the partition for files doc-example-file4.csv and doc-example-file5.csv.Use an exclude pattern to exclude unwanted files or folders.Use consistent headersWhen using CSV data, use headers consistently. If some of your files have headers and some don't, then the crawler creates multiple tables.Related informationManaging partitions for ETL output in AWS GlueSetting crawler configuration optionsFollow"


Question: Why am I getting a 554 or 400 "Message rejected" error with the message "Email address is not verified" from Amazon SES?

"I'm trying to send emails using Amazon Simple Email Service (Amazon SES). However, I'm getting a 554 or 400 "Message rejected" error with the message "Email address is not verified. The following identities failed the check in region." Why is this happening? How can I resolve this error?"

Answer: "I'm trying to send emails using Amazon Simple Email Service (Amazon SES). However, I'm getting a 554 or 400 "Message rejected" error with the message "Email address is not verified. The following identities failed the check in region." Why is this happening? How can I resolve this error?Short descriptionWhen you send emails using Amazon SES, you must verify the identities (domains or email addresses) that you send emails from. If you send emails using an unverified identity, then you can get "Message rejected" errors. These errors can apply to the "From", "Source", "Sender", or "Return-Path" addresses.If you're using the Amazon SES SMTP interface, then the error is similar to "554 Message rejected: Email address is not verified. The following identities failed the check in region region: identity0, identity1, identity2."If you're using the Amazon SES API (HTTPS) interface, then the error is "400 Message rejected." If you get the 400 error, check the error stack for a description of what caused the error. If you see the error message "Email address is not verified," then that means you're sending email from an identity that failed the verification check in that AWS Region.ResolutionTo resolve 554 or 400 "Message rejected" errors with the message "Email address is not verified," check the following:Open the Amazon SES console. Confirm that the identity that you're sending emails from has a verification status of verified, not pending or failure. If your identity is stuck in the pending state, see  What can I do if my domain is stuck in the "pending verification" status or in the "failed" verification status in Amazon SES?If your email address is verified, check that the email address on the message that resulted in the "Message rejected" error is correct. Email addresses are case-sensitive. The email address that you used to send the message must match your verified email address.Check whether your account is in the Amazon SES sandbox for the AWS Region that you're using to send emails. If your account is in the Amazon SES sandbox, then you must verify the recipient email address in addition to verifying your sender identity. Or, you can request to move your account out of the Amazon SES sandbox.Note: If you're using sending authorization policies to delegate sending on behalf of your Amazon SES identity to a non-verified email address, then the delegate sender account must be out of the Amazon SES sandbox. For more information, see Moving out of the Amazon SES sandbox.Confirm that your email-sending method is using the correct AWS Region. Email identity verification is separate for each Region. You must verify the identity that you use to send emails in each Region that you want to send emails from.Confirm that you're connecting to the correct Amazon SES endpoint. Amazon SES has endpoints in several Regions. You must communicate with the endpoint for the Amazon SES SMTP interface or the Amazon SES API in the correct Region.For more information on issues that can lead to errors when using Amazon SES, see Amazon SES email sending errors.Follow"


Question: How do I set up single stack ownership of an AWS::StepFunctions::Activity resource in CloudFormation?

I want my AWS::StepFunctions::Activity resource to be maintained by a single stack only in AWS CloudFormation.

Answer: "I want my AWS::StepFunctions::Activity resource to be maintained by a single stack only in AWS CloudFormation.Short descriptionYou can maintain ownership of an AWS::StepFunctions::Activity resource from multiple stacks. For example, you can create StackA with the first CloudFormation template and StackB with the second template.Template for StackA:AWSTemplateFormatVersion: "2010-09-09"Description: Template for StackAResources:  MyActivity:    Type: AWS::StepFunctions::Activity    Properties:      Name: myActivityTemplate for StackB:AWSTemplateFormatVersion: "2010-09-09"Description: Template for StackBResources: MyActivityDuplicate: Type: AWS::StepFunctions::Activity Properties: Name: myActivity Tags: - Key: Key1   Value: Val1....StackA and StackB will both show the resource as CREATE_COMPLETE. However, the Activity resource referenced in StackA and StackB has the properties that are defined in the template for StackA. The operation appears successful, but the resource's properties don't match the template used by StackA. When either one of the stacks is deleted, the activity is also deleted. There is no indication in the other stack(s) that the resource no longer exists.It's a best practice when using CloudFormation to make sure that a shared resource has its configuration maintained in a single CloudFormation stack. Doing so provides a single source of truth for the configuration of the resource. The CloudFormation stack that maintains the shared resource can provide the resource's Amazon Resource Name (ARN) as a stack output. Then, any other stacks that depend upon the shared resource can reference the shared resource, such as Step Functions activity. For more information on cross-stack references, see Walkthrough: Refer to resource outputs in another AWS CloudFormation stack.ResolutionTo resolve the issue, complete the following steps for each AWS::StepFunctions::Activity resource being maintained in multiple stacks:1.    Identify the ARNs shared in the AWS Personal Health Dashboard of the CloudFormation stacks that maintain the AWS::StepFunctions::Activity resource.2.    Designate one of the stacks as the single stack to maintain the activity going forward.Complete the following steps within the identified stack:1.    If there is no Outputs section in your CloudFormation template, then add an Outputs section to your template.2.    Add a new output to the Outputs section. Your new output must include the following:A unique logical ID for the output. For example: MyActivityArn. You use your unique logical ID as the key for the output within the Outputs section of your template.A Value property within the stack output. The Value property uses Ref with the logical ID of the activity as its argument to retrieve the activity's ARN.An Export property to the stack output, with its value as an object with a single Name property. The Name property's value is a string used to reference this activity's ARN from other CloudFormation stacks. For example:AWSTemplateFormatVersion: "2010-09-09"Description: Template for StackAResources:  MyActivity:    Type: AWS::StepFunctions::Activity    Properties:      Name: myActivityOutputs:  MyActivityArn:    Value:       Ref: MyActivity    Export:      Name: MyActivityArn3.    Update your stack with the preceding changes.4.    Validate that the resources referencing the activity are still using the correct ARN.Complete the followings steps for each stack that references the activity resource:1.    In the CloudFormation console, identify the logical ID of the activity resource on the Resources tab for your stack.2.    Add a DeletionPolicy property to the activity's resource declaration, if the property isn't there already. Set the value to Retain. For example:AWSTemplateFormatVersion: "2010-09-09"Description: Template for StackBResources:  MyActivityDuplicate:    Type: AWS::StepFunctions::Activity    DeletionPolicy: Retain    Properties:      Name: myActivity      Tags:        - Key: Key1          Value: Val1...3.    Update your stack with the preceding changes.4.    Determine which other resources in the template are referencing this activity's ARN.5.    For each of the resources that you identified in step 4, replace the reference to the activity's ARN with the Fn::ImportValue intrinsic function. Pass the export name that you specified when you created a new output earlier as the intrinsic function's argument. For example:AWSTemplateFormatVersion: "2010-09-09"Description: Template for StackBResources:  # MyActivityDuplicate:  #   DeletionPolicy: Retain  #   Type: AWS::StepFunctions::Activity  #   Properties:  #     Name: myActivity  #     Tags:  #       - Key: Key1  #         Value: Val1  MyStateMachine:    Type: AWS::StepFunctions::StateMachine    Properties:      StateMachineName: MyStateMachine      RoleArn: arn:aws:iam::111122223333:role/service-role/StatesExecutionRole-us-east-1      DefinitionString:         Fn::Sub:          - |              {                "Comment": "An example using a Task state.",                "StartAt": "getGreeting",                "Version": "1.0",                "TimeoutSeconds": 300,                "States":                {                  "getGreeting": {                    "Type": "Task",                    "Resource": "${ActivityArn}",                    "End": true                  }                }              }          - ActivityArn:              Fn::ImportValue: MyActivityArn6.    Remove the entry for the activity in the Resources section.7.    Update your stack with the preceding changes.8.    Validate that the resources referencing the activity are still using the correct ARN.Follow"


Question: How can I create a CloudWatch log group to use as a target for an EventBridge rule?

I want to add an Amazon CloudWatch log group as a target to an Amazon EventBridge rule. How can I do this?

Answer: "I want to add an Amazon CloudWatch log group as a target to an Amazon EventBridge rule. How can I do this?Short descriptionWhen you create an EventBridge rule, you must specify the target where events that are matched to the rule are sent. For a list of available targets for EventBridge, see Targets available in the EventBridge console. One of the targets that you can add to the EventBridge rule is a CloudWatch log group.ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Use these steps to configure a CloudWatch log group to be used as a target for an EventBridge rule:1.    Open the EventBridge console, and choose the Region that you want to create a rule in.2.    Choose Create rule, and then enter any information about that rule, such as the event pattern or schedule details.3.    On the Select target page, choose CloudWatch as your target.Note: To add a CloudWatch log group as a target, you can either:Create a new log groupUse an existing log groupBe sure that the log group that you use as a target for the EventBridge rule starts with /aws/events. If you create a new log group using the console when you create a rule, then EventBridge automatically creates the log group for you. If you want to add an existing log group, be aware that only log groups that start with /aws/events appear in the dropdown list.4.    To deliver the event data to the target log group, EventBridge needs permission to access the target log group. It uses this permission to create log streams and push events to those log streams. For CloudWatch log groups, EventBridge uses a resource-based policy in order to access the log group.If you use the console to add log groups to an EventBridge rule, then the resource-based policy for your log group is updated automatically. But, if you use the AWS SDK/API/CDK/CLI, then you must manually update the log group's resource-based policy. This example policy document demonstrates the permissions you must define in the resource-based policy of your log group:{  "Statement": [    {      "Action": [        "logs:CreateLogStream",        "logs:PutLogEvents"      ],      "Effect": "Allow",      "Principal": {        "Service": [          "events.amazonaws.com",          "delivery.logs.amazonaws.com"        ]      },      "Resource": "arn:aws:logs:region:account:log-group:/aws/events/*:*",      "Sid": "TrustEventsToStoreLogEvent"    }  ],  "Version": "2012-10-17"}You can't configure a log group's resource-based policy using the console. To add these permissions to a resource-based policy, use the PutResourcePolicy API call. Then, use the describe-resource-policies command to check that your policy applied correctly.Note: The current limit is 10 policies per Region, per account. If you hit this limit, then delete any unused policies or combine multiple policies.5.    The EventBridge service uses the PutRule CLI command to create rules. It then uses the PutTargets API or put-targets CLI command to add targets to an EventBridge rule. When using the AWS SDK, CDK, or CLI, you must use the PutTargets API or put-targets CLI command to add the log group to EventBridge rules.Related informationCloudWatch Logs permissionsFollow"


Question: What's the best way to transfer large amounts of data from one Amazon S3 bucket to another?

I want to transfer a large amount of data (1 TB or more) from one Amazon Simple Storage Service (Amazon S3) bucket to another bucket.

Answer: "I want to transfer a large amount of data (1 TB or more) from one Amazon Simple Storage Service (Amazon S3) bucket to another bucket.Short descriptionDepending on your use case, you can perform the data transfer between buckets using one of the following options:Run parallel uploads using the AWS Command Line Interface (AWS CLI)Use an AWS SDKUse cross-Region replication or same-Region replicationUse Amazon S3 batch operationsUse S3DistCp with Amazon EMRUse AWS DataSyncResolutionRun parallel uploads using the AWS CLINote: As a best practice, make sure that you're using the most recent version of the AWS CLI. For more information, see Installing or updating the latest version of the AWS CLI.You can split the transfer into multiple mutually exclusive operations to improve the transfer time by multi-threading. For example, you can run multiple, parallel instances of aws s3 cp, aws s3 mv, or aws s3 sync using the AWS CLI. You can create more upload threads while using the --exclude and --include parameters for each instance of the AWS CLI. These parameters filter operations by file name.Note: The --exclude and --include parameters are processed on the client side. Therefore, note that resources on your local machine might affect the performance of the operation.For example, to copy a large amount of data from one bucket to another (where file names begin with a number), run the following commands.First, run this command to copy the files with names that begin with the numbers 0 through 4:aws s3 cp s3://source-awsexamplebucket/ s3://destination-awsexamplebucket/ --recursive --exclude "*" --include "0*" --include "1*" --include "2*" --include "3*" --include "4*"Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Then, run this command on a second AWS CLI instance to copy the files with names that begin with the numbers 5 through 9:aws s3 cp s3://source-awsexamplebucket/ s3://destination-awsexamplebucket/ --recursive --exclude "*" --include "5*" --include "6*" --include "7*" --include "8*" --include "9*"Additionally, you can customize the following AWS CLI configurations to speed up the data transfer:multipart_chunksize: This value sets the size of each part that the AWS CLI uploads in a multipart upload for an individual file. This setting allows you to break down a larger file (for example, 300 MB) into smaller parts for quicker upload speeds.Note: A multipart upload requires that a single file is uploaded in not more than 10,000 distinct parts. You must be sure that the chunksize that you set balances the part file size and the number of parts.max_concurrent_requests: This value sets the number of requests that can be sent to Amazon S3 at a time. The default value is 10. You can increase it to a higher value like resources on your machine. You must be sure that your machine has enough resources to support the maximum number of concurrent requests that you want.Use an AWS SDKConsider building a custom application using an AWS SDK to perform the data transfer for a very large number of objects. The AWS CLI can also be used to perform a copy operation. However, a custom application might be more efficient at performing a transfer at the scale of hundreds of millions of objects.Use cross-Region replication or same-Region replicationAfter you set up cross-Region replication (CRR) or same-Region replication (SRR) on the source bucket, Amazon S3 automatically replicates new objects from the source bucket to the destination bucket. You can choose to filter which objects are replicated using a prefix or tag. For more information on configuring replication and specifying a filter, see Replication configuration overview.After replication is configured, only new objects are replicated to the destination bucket. Existing objects aren't replicated to the destination bucket. For more information, see Replicating existing objects with S3 Batch Replication.Use Amazon S3 batch operationsYou can use Amazon S3 batch operations to copy multiple objects with a single request. When you create a batch operation job, you specify which objects to perform the operation on using an Amazon S3 inventory report. Or, you can use a CSV manifest file to specify a batch job. Then, Amazon S3 batch operations call the API to perform the operation.After the batch operation job is complete, you get a notification and you can choose to receive a completion report about the job.Use S3DistCp with Amazon EMRThe S3DistCp operation on Amazon EMR can perform parallel copying of large volumes of objects across Amazon S3 buckets. S3DistCp first copies the files from the source bucket to the worker nodes in an Amazon EMR cluster. Then, the operation writes the files from the worker nodes to the destination bucket. For more guidance on using S3DistCp, see Seven tips for using S3DistCp on Amazon EMR to move data efficiently between HDFS and Amazon S3.Important: Because this option requires you use Amazon EMR, be sure to review Amazon EMR pricing.Use AWS DataSyncTo move large amounts of data from one Amazon S3 bucket to another bucket, perform the following steps:1.    Open the AWS DataSync console.2.    Create a task.3.    Create a new location for Amazon S3.4.    Select your S3 bucket as the source location.5.    Update the source location configuration settings. Make sure to specify the AWS Identity Access Management (IAM) role that will be used to access your source S3 bucket.6.    Select your S3 bucket as the destination location.7.    Update the destination location configuration settings. Make sure to specify the AWS Identity Access Management (IAM) role that will be used to access your S3 destination bucket.8.    Configure settings for your task.9.    Review the configuration details.10.    Choose Create task.11.    Start your task.Important: When you use AWS DataSync, you will incur additional costs. To preview any DataSync costs, review the DataSync pricing structure and DataSync limits.Follow"


Question: How do I remove local administrator rights from WorkSpaces users?

I want to turn off local administrator rights for Amazon WorkSpaces users.

Answer: "I want to turn off local administrator rights for Amazon WorkSpaces users.ResolutionTo turn off administrator rights, update the Local Administrator Setting from the WorkSpaces console. For instructions, see Manage local administrator permissions.Note: When you update the directory details to turn off access for local administrators, the change applies only to newly created WorkSpaces. To apply the change to an existing WorkSpace, rebuild the WorkSpace.To make changes to a WorkSpace's operating system after removing local administrator rights, connect to the WorkSpace using a Remote Desktop Protocol (RDP) client. When connecting with an RDP, you still must use a WorkSpaces user account that has administrator permissions, such as a domain administrator. If you connect to the WorkSpace with RDP as a regular WorkSpaces user or domain user, then you're prompted for elevated access rights.Related informationAdminister your WorkSpacesFollow"


Question: How do I provide external access to multiple Kubernetes services in my Amazon EKS cluster?

I want to provide external access to multiple Kubernetes services in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster.

Answer: "I want to provide external access to multiple Kubernetes services in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster.Short descriptionUse the NGINX ingress controller or AWS Load Balancer Controller for Kubernetes to provide external access to multiple Kubernetes services in your Amazon EKS cluster. The NGINX ingress controller is maintained primarily by NGINX. To check for issues with the NGINX ingress controller, see the list of issues on the GitHub website. The AWS Load Balancer Controller is maintained by Amazon Web Services (AWS). To check for issues with AWS Load Balancer Controller, see the list of issues on the GitHub website.Important: The ingress controller and IngressClass (from the Kubernetes website) aren't the same as the Ingress (from the Kubernetes website). The Ingress is a Kubernetes resource that exposes HTTP and HTTPS routes from outside the cluster to the services within the cluster. The ingress controller usually fulfills the Ingress with a load balancer. You can't use Ingress without an ingress controller. The IngressClass is used to identify which ingress controller to use for fulfilling the Ingress object request.Prerequisite: Install the AWS Load Balancer Controller. It's a best practice to use the AWS Load Balancer Controller to create and manage a Network Load Balancer for the LoadBalancer type service objects in Amazon EKS.ResolutionThe following resolution uses the kubernetes/ingress-nginx ingress controller on the Kubernetes GitHub website. The other ingress controller that's available for public use is the nginxinc/kubernetes-ingress on the NGINX GitHub website.Deploy the NGINX ingress controller for KubernetesYou can deploy the NGINX ingress controller for Kubernetes by either transmission control protocol (TCP) or transport layer security (TLS).Note: The following resolution is tested on Amazon EKS version 1.22, NGINX ingress controller version 1.3.0, and AWS Load Balancer Controller version 2.4.3.(Option 1) NGINX ingress controller with TCP on Network Load Balancer1.    Get the YAML file to deploy the following Kubernetes objects: namespace, serviceaccounts, configmap, clusterroles, clusterrolebindings, roles, rolebindings, services, deployments, ingressclasses, and validatingwebhookconfigurations.wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/aws/deploy.yaml2.    Edit the file. Then, in the ingress-nginx-controller service object section replace all service.beta.kubernetes.io annotations with following:service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcpservice.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"service.beta.kubernetes.io/aws-load-balancer-type: "external"service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "instance"service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"3.    Apply the manifest:kubectl apply -f deploy.yamlExample output:namespace/ingress-nginx created serviceaccount/ingress-nginx created configmap/ingress-nginx-controller created clusterrole.rbac.authorization.k8s.io/ingress-nginx created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created role.rbac.authorization.k8s.io/ingress-nginx created rolebinding.rbac.authorization.k8s.io/ingress-nginx created service/ingress-nginx-controller-admission created service/ingress-nginx-controller created deployment.apps/ingress-nginx-controller created ingressclass.networking.k8s.io/nginx createdvalidatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created serviceaccount/ingress-nginx-admission created clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created(Option 2) NGINX ingress controller TLS termination on Network Load BalancerBy default, the previous solution terminates TLS in the NGINX ingress controller. You can also configure the NGINX Ingress service to terminate TLS at the Network Load Balancer.1.    Download the deploy.yaml template:wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/aws/nlb-with-tls-termination/deploy.yaml2.    Edit the file. Then, in the ingress-nginx-controller service object section, replace all service.beta.kubernetes.io annotations with following:service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-west-2:XXXXXXXX:certificate/XXXXXX-XXXXXXX-XXXXXXX-XXXXXXXXservice.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"service.beta.kubernetes.io/aws-load-balancer-type: "external"service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "instance"service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"Note: Make sure to include your ARN for service.beta.kubernetes.io/aws-load-balancer-ssl-cert.3.    Edit the file, and change the Amazon Virtual Private Cloud (Amazon VPC) CIDR for the Kubernetes cluster:proxy-real-ip-cidr: XXX.XXX.XXX/XX4.    Apply the manifest:kubectl apply -f deploy.yamlNote: The previous manifest uses ExternalTrafficPolicy as local to preserve the source (client) IP address. Using this configuration with a custom DHCP name in the Amazon VPC causes an issue. To prevent the issue from occurring, apply the following patch to the kube-proxy:kubectl edit daemonset kube-proxy -n kube-system5.    Edit the manifest to include the following snippet:spec:  template:    spec:      containers:        - name: kube-proxy          command:            - kube-proxy            - --hostname-override=$(NODE_NAME)            - --v=2            - --config=/var/lib/kube-proxy-config/config           env:            - name: NODE_NAME              valueFrom:                fieldRef:                  apiVersion: v1                  fieldPath: spec.nodeNameVerify the deployed resourcesAWS Load Balancer controllerCommand:kubectl get all -n kube-system --selector app.kubernetes.io/instance=aws-load-balancer-controllerExample output:NAME                                                READY   STATUS    RESTARTS   AGE   IP               NODE                                           NOMINATED NODE   READINESS GATESpod/aws-load-balancer-controller-85cd8965dc-ctkjt   1/1     Running   0          48m   192.168.37.36    ip-192-168-59-225.us-east-2.compute.internal   none             none pod/aws-load-balancer-controller-85cd8965dc-wpwx9   1/1     Running   0          48m   192.168.53.110   ip-192-168-59-225.us-east-2.compute.internal   none>         none NAME                                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE   SELECTOR service/aws-load-balancer-webhook-service   ClusterIP   10.100.154.44   none          443/TCP   19h   app.kubernetes.io/instance=aws-load-balancer-controller,app.kubernetes.io/name=aws-load-balancer-controller NAME                                           READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS                   IMAGES                                                                                SELECTOR deployment.apps/aws-load-balancer-controller   2/2     2            2           19h   aws-load-balancer-controller 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-load-balancer-controller:v2.4.0   app.kubernetes.io/instance=aws-load-balancer-controller,app.kubernetes.io/name=aws-load-balancer-controller NAME                                                      DESIRED   CURRENT   READY   AGE   CONTAINERS                     IMAGES                                                                                SELECTOR replicaset.apps/aws-load-balancer-controller-85cd8965dc   2         2         2       19h   aws-load-balancer-controller   602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-load-balancer-controller:v2.4.0   app.kubernetes.io/instance=aws-load-balancer-controller,app.kubernetes.io/name=aws-load-balancer-controller,pod-template-hash=85cd8965dcNGINX ingress controllerCommand:kubectl get all -n ingress-nginx --selector app.kubernetes.io/instance=ingress-nginxExample output:NAME                                           READY  STATUS   RESTARTS  AGE   IP               NODE                                           NOMINATED NODE  READINESS GATESpod/ingress-nginx-controller-54d8b558d4-k4pdf  1/1    Running  0         56m   192.168.46.241   ip-192-168-59-225.us-east-2.compute.internal   none            noneNAME                                        TYPE           CLUSTER-IP      EXTERNAL-IP                                                                     PORT(S)                      AGE   SELECTORservice/ingress-nginx-controller            LoadBalancer   10.100.99.129   ad9bba7a8239a475297d24bd2f617782-a579e639079f8270.elb.us-east-2.amazonaws.com   80:32578/TCP,443:30724/TCP   15h   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginxservice/ingress-nginx-controller-admission  ClusterIP      10.100.190.61   none                                                                            443/TCP                      15h   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginxNAME                                       READY   UP-TO-DATE   AVAILABLE    AGE   CONTAINERS   IMAGES                                                                                SELECTORdeployment.apps/ingress-nginx-controller   1/1     1            1            15h   controller   k8s.gcr.io/ingress-nginx/controller:v1.1.1@sha256:0bc88eb15f9e7f84e8e56c14fa5735aaa488b840983f87bd79b1054190e660de   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginxNAME                                                  DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES                                                                                SELECTORreplicaset.apps/ingress-nginx-controller-54d8b558d4   1         1         1       15h   controller   k8s.gcr.io/ingress-nginx/controller:v1.1.1@sha256:0bc88eb15f9e7f84e8e56c14fa5735aaa488b840983f87bd79b1054190e660de   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx,pod-template-hash=54d8b558d4NAME                                       COMPLETIONS   DURATION   AGE   CONTAINERS   IMAGES                                                                                SELECTORjob.batch/ingress-nginx-admission-create   1/1           2s         15h   create       k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660   controller-uid=242bdf56-de16-471d-a691-1ca1dbc10a41job.batch/ingress-nginx-admission-patch    1/1           2s         15h   patch        k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660   controller-uid=a9e710d2-5001-4d40-a435-ddc8993bfe42IngressClassCommand:kubectl get ingressclassExample output:NAME    CONTROLLER             PARAMETERS                             AGE alb     ingress.k8s.aws/alb    IngressClassParams.elbv2.k8s.aws/alb   19h nginx   k8s.io/ingress-nginx   none                                   15hTest the deployment setupNote: The following step is running two microservices. The microservices are exposed internally with Kubernetes as the default type.1.    Set up your deployments or microservices. For example, hostname-app and apache-app.Example of a hostname-app-svc.yaml file for hostname-app:apiVersion: apps/v1 kind: Deployment metadata:  name: hostname-app  namespace: default spec:  replicas: 2  selector:    matchLabels:      app: hostname-app  template:    metadata:      labels:        app: hostname-app    spec:      containers:      - name: hostname-app        image: k8s.gcr.io/serve_hostname:1.1 --- apiVersion: v1 kind: Service metadata:  name: hostname-svc  namespace: default spec:  ports:  - port: 80    targetPort: 9376    protocol: TCP  selector:    app: hostname-appExample of an apache-app-svc.yaml file for apache-app:apiVersion: apps/v1 kind: Deployment metadata:  name: apache-app  namespace: default spec:  replicas: 2  selector:    matchLabels:      app: apache-app  template:    metadata:      labels:        app: apache-app    spec:      containers:      - name: apache-app        image: httpd:latest        ports:        - containerPort: 80 --- apiVersion: v1 kind: Service metadata:  name: apache-svc  namespace: default  labels: spec:  ports:  - port: 80    targetPort: 80    protocol: TCP  selector:    app: apache-app2.    Apply your configurations.hostname-app:kubectl apply -f hostname-app-svc.yamlapache-app:kubectl apply -f apache-app-svc.yaml3.    Verify that the resources are created.DeploymentsCommand:kubectl get deployment hostname-app apache-app -n defaultExample output:NAME           READY   UP-TO-DATE   AVAILABLE   AGE hostname-app   2/2     2            2           29mapache-app     2/2     2            2           29mServicesCommand:kubectl get svc apache-svc hostname-svc -n defaultExample output:NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE apache-svc     ClusterIP   10.100.73.51    none          80/TCP    29m hostname-svc   ClusterIP   10.100.100.44   none          80/TCP    29mTest the NGINX ingress controller1.    Access the DNS URL of the load balancer that you retrieved from the command line:curl -I ad9bba7a8239a475297d24bd2f617782-a579e639079f8270.elb.us-east-2.amazonaws.comExample output:HTTP/1.1 404 Not Found Date: Thu, 03 Mar 2022 14:03:11 GMT Content-Type: text/html Content-Length: 146 Connection: keep-aliveNote: The default server returns a 404 Not Found page for all domain requests that don't have defined ingress rules. Based on the defined rules, the ingress controller doesn't divert traffic to the specified backend service unless the request matches the configuration. Because the host field is configured for the Ingress object, you must supply the Host header of the request with the same hostname. In a testing environment, use a curl flag to provide the Host header. In a production environment, map the load balancer DNS name to the hostname on any DNS provider, such as Amazon Route 53.2.    Implement the ingress so that it interfaces with your services using a single load balancer provided by the NGINX ingress controller.Example micro-ingress.yaml:apiVersion: networking.k8s.io/v1 kind: Ingress metadata:  name: micro-ingress  namespace: default  annotations:     kubernetes.io/ingress.class: nginx spec:  rules:    - host: hostname.mydomain.com      http:        paths:        - backend:            service:              name: hostname-svc              port:                number: 80          path: /          pathType: Prefix  - host: apache.mydomain.com    http:      paths:      - backend:          service:            name: apache-svc            port:              number: 80        path: /        pathType: PrefixNote: For more information, see Name based virtual hosting (from the Kubernetes website).3.    Verify that the resource is created.IngressCommand:kubectl get ingress -n defaultExample output:NAME           CLASS   HOSTS                                       ADDRESS                                                                         PORTS   AGE micro-ingress  none    hostname.mydomain.com,apache.mydomain.com   ad9bba7a8239a475297d24bd2f617782-a579e639079f8270.elb.us-east-2.amazonaws.com   80      29m4.    Add the Host header to the request.First configured domain:curl -i -H "Host: hostname.mydomain.com" http://aaa71bxxxxx-11xxxxx10.us-east-2.elb.amazonaws.com/Example output:HTTP/1.1 200 OK Date: Sat, 26 Mar 2022 18:50:38 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 29 Connection: keep-aliveSecond configured domain:curl -i -H "Host: apache.mydomain.com" http://aaa71bxxxxx-11xxxxx10.us-east-2.elb.amazonaws.com/Example output:HTTP/1.1 200 OK Date: Sat, 26 Mar 2022 18:51:00 GMT Content-Type: text/html Content-Length: 45 Connection: keep-alive Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT ETag: "2d-432a5e4a73a80" Accept-Ranges: bytesAfter adding the Host header, the ingress controller redirects traffic to the backend configured service as it matches the configuration that's defined in the Ingress.To keep the same domain name but divert the traffic based on the path that's accessed, you must add path-based routing with the Ingress.For example:apiVersion: networking.k8s.io/v1 kind: Ingress metadata:  name: path-ingress  namespace: default  annotations:    kubernetes.io/ingress.class: nginxd    nginx.ingress.kubernetes.io/rewrite-target: / spec:  rules:  - host: mydomain.com    http:      paths:      - backend:          service:            name: hostname-svc            port:              number: 80        path: /hostname        pathType: Prefix  - host: mydomain.com    http:      paths:      - backend:          service:            name: apache-svc            port:              number: 80        path: /apache        pathType: PrefixNote: If requests have mydomain.com as the Host header, then the preceding example returns only the 200 response. The requests are accessed on either the /hostname or /apache paths. For all other requests, 404 responses are returned.5.    Verify that the path-based routing is added:Command:kubectl get ingress -n defaultOutput:NAME            CLASS  HOSTS                                       ADDRESS                                                                         PORTS  AGEmicro-ingress   none   hostname.mydomain.com,apache.mydomain.com   ad9bba7a8239a475297d24bd2f617782-a579e639079f8270.elb.us-east-2.amazonaws.com   80     164mpath-ingress    none   mydomain.com,mydomain.com                   ad9bba7a8239a475297d24bd2f617782-a579e639079f8270.elb.us-east-2.amazonaws.com   80     120mCommand:curl -i -H "Host: mydomain.com" http://aaa71bxxxxx-11xxxxx10.us-east-2.elb.amazonaws.com/hostname-or-curl -i -H "Host: mydomain.com" http://aaa71bxxxxx-11xxxxx10.us-east-2.elb.amazonaws.com/apacheTest the ingress using the AWS Load Balancer controller1.    Launch the Application Load Balancer using the following example Ingress manifest:apiVersion: networking.k8s.io/v1 kind: Ingress metadata:  name: micro-ingress-alb  namespace: default  annotations:    kubernetes.io/ingress.class: alb     alb.ingress.kubernetes.io/scheme: internet-facing    alb.ingress.kubernetes.io/target-type: ip spec:  rules:  - host: alb.hostname.mydomain.com    http:      paths:      - backend:          service:            name: hostname-svc            port:              number: 80        path: /        pathType: Prefix   - host: alb.apache.mydomain.com     http:       - backend:            service:              name: apache-svc              port:                number: 80         path: /         pathType: Prefix2.    Verify that the Application Load Balancer launches.Command:kubectl get ingress -n defaultOutput:NAME               CLASS  HOSTS                                               ADDRESS                                                                         PORTS  AGEmicro-ingress      none   hostname.mydomain.com,apache.mydomain.com           ad9bba7a8239a475297d24bd2f617782-a579e639079f8270.elb.us-east-2.amazonaws.com   80     164mmicro-ingress-alb  none   alb.hostname.mydomain.com,alb.apache.mydomain.com   k8s-default-microing-8a252bde81-1907206594.us-east-2.elb.amazonaws.com          80     18mpath-ingress       none   mydomain.com,mydomain.com                           ad9bba7a8239a475297d24bd2f617782-a579e639079f8270.elb.us-east-2.amazonaws.com   80     120mRequest based on first configured domain:curl -i -H "Host: alb.hostname.mydomain.com" http://k8s-default-microing-8a252bde81-1907206594.us-east-2.elb.amazonaws.comExample output:HTTP/1.1 200 OK Date: Sat, 26 Mar 2022 20:46:02 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 29 Connection: keep-aliveRequest based on second configured domain:curl -i -H "Host: alb.apache.mydomain.com" http://k8s-default-microing-8a252bde81-1907206594.us-east-2.elb.amazonaws.comExample output:HTTP/1.1 200 OK Date: Sat, 26 Mar 2022 20:46:14 GMT Content-Type: text/html Content-Length: 45 Connection: keep-alive Server: Apache/2.4.53 (Unix) Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT ETag: "2d-432a5e4a73a80" Accept-Ranges: bytesFollow"


Question: How do I troubleshoot low storage space in my OpenSearch Service domain?

"My Amazon OpenSearch Service domain is out of storage space, and I received an error."

Answer: "My Amazon OpenSearch Service domain is out of storage space, and I received an error.Short descriptionYour OpenSearch Service domain has storage space requirements for processing workloads. The ClusterBlockException error might occur because your OpenSearch Service domain ran out of storage space. For example, when you allocate storage to a cluster node, up to 20% of that space (20 GB) is reserved space. Also, operating systems such as Linux reserve 5% of the file system to support any critical processes performed by the root user. For more information about OpenSearch Service storage space requirements, see Calculating storage requirements.To resolve low storage space issues, complete the following steps:Update your cluster sharding strategy so that shards are evenly distributed across all nodes.Increase the size of the domain's Amazon Elastic Block Store (Amazon EBS) volumes. You can also add more cluster nodes to the OpenSearch Service domain.Reduce the amount of data that's stored in your domain. You can delete unnecessary files, optimize old indices, or reduce the domain's replica count. Be sure to take a manual snapshot first.Note: Reducing the domain's replica count can reduce fault tolerance. It's a best practice to configure at least one replica for each index.Use Index State Management (ISM) to manage low storage space.Use Amazon CloudWatch metrics to monitor the amount of available storage in your cluster.ResolutionUpdating your storage configuration settingsImportant: Before you update your configuration settings, verify the configuration changes that can initiate a blue/green deployment. Also, make sure that your dedicated primary node type is the recommended node type.Use the cat allocation (from the Elastic website) command to check how much storage space is available for each node in your cluster:curl -XGET "es_endpoint/_cat/allocation?v"To update your OpenSearch Service domain configuration settings, complete the following steps:1.     Open the OpenSearch Service console.2.     From the navigation pane, under Managed clusters, choose Domains.3.     Choose the domain that you want to add storage to.4.    Update your domain settings.Note: If you're using an Amazon EBS volume for storage, then update your Storage configuration settings. Or you can update the number of data nodes.5.    Choose Submit.If your domain uses EBS volumes for data storageIf the domain uses EBS volumes for storage, take one of the following actions:Increase the size of the EBS volumes. The maximum volume size depends on the node's instance type or Elasticsearch version type. For example, 512 GiB is the maximum volume size for Elasticsearch version 1.5.If you can't increase the size of the EBS volumes, then add nodes or scale up your domain and choose a new EBS limit. For more information about EBS volumes, see EBS volume size quotas.Create a backup of unwanted indices to your Amazon Simple Storage Service (Amazon S3) bucket. Then, delete the indices from your OpenSearch Service cluster to free up disk space. Or, you can use Curator to rotate data in OpenSearch Service (from the Python Package Index website). You can also use ISM to rotate indices in OpenSearch Service.Note: You must have a manual snapshot repository set up to create any backups.If your domain uses Amazon Elastic Compute Cloud (Amazon EC2) I3 instances for data storageIf you're using Amazon EC2 I3 instances for data storage, then take one of the following actions:Add more nodes to your OpenSearch Service cluster.Scale up the instance type to obtain more storage space.Delete any unwanted indices from your OpenSearch Service cluster.Delete unused or old indicesYou can check the indices creation date with the following command:GET _cat/indices?h=h,s,i,id,p,r,dc,dd,ss,creation.date.string&s=creation.date.string:descTo delete single indices, run the following command:DELETE <index-name>To delete multiple indices, run the following command:DELETE cwl-index-*Use ISM to manage low storage spaceISM allows you to automate routine tasks and then apply them to indices and index patterns in OpenSearch Service. With ISM, you can define custom management policies that help you maintain issues, such as low disk space. For example, you can use a rollover operation and an ISM policy to automate deletion of old indices based on conditions, such as index size. The rollover operation rolls over a target to a new index when an existing index meets the defined condition.The following example ISM policy deletes indices after 50 minutes:PUT _plugins/_ism/policies/delete_ism_policy{    "policy": {        "policy_id": "delete_ism_policy",        "description": "A simple default policy that deletes old unused indices“,        "last_updated_time": 1658834661281,        "schema_version": 13,        "error_notification": null,        "default_state": "example_hot_state",        #Default state        "states": [            {                "name": "example_hot_state",                "actions": [],                "transitions": [                    {                        "state_name": "delete",                        "conditions": {                            "min_index_age": “50m”   #Note that after 50 minutes index will transit from hot_state to delete state                        }                    }                ]            },            {                "name": "delete",                    #Indices moved to delete state                "actions": [                    {                        "delete": {}                    }                ],                "transitions": []            }        ],        "ism_template": [            {                "index_patterns": [                  #This ISM policy attaches to all indices for the index pattern and you can mention your index pattern here                    "sample*"                ],                "priority": 100,                     #Priority can be set from 0 to 100                "last_updated_time": 1658834436349            }        ]    }}Use the following API call to manually attach the ISM policy to the index:POST _plugins/_ism/add/your-index-*{     "policy_id": "<policy_id>"}For more information, see How do I use ISM to manage low storage space in Amazon OpenSearch Service?Use CloudWatch alarms to monitor storageYou can use the CloudWatch FreeStorageSpace metric to monitor the amount of available storage in your cluster. To receive notifications whenever storage space is low, configure your CloudWatch alarms.For more information, see Monitoring OpenSearch cluster metrics with Amazon CloudWatch.Related informationOperational best practices for Amazon OpenSearch ServiceHow do I make my Amazon OpenSearch Service domain more fault tolerant?Why is my Amazon OpenSearch Service cluster in red or yellow status?Follow"


Question: Can I merge multiple AWS accounts together?

"I have resources on multiple AWS accounts, and I want to merge all the resources together under one account."

Answer: "I have resources on multiple AWS accounts, and I want to merge all the resources together under one account.ResolutionIt's not possible to merge multiple AWS accounts together.However, you can use AWS Organizations to manage multiple AWS accounts, and then combine the billing in one consolidated bill.You can also transfer some AWS resources between AWS accounts. For example, with Amazon Elastic Compute Cloud (Amazon EC2), you can share custom Amazon Machine Images (AMIs) between accounts. This allows you to transfer instances between accounts. Because each AWS service is unique, the way to move resources between accounts varies by service. For more details, see AWS Documentation.Related informationShare an AMI with specific AWS accountsFollow"


Question: How do I troubleshoot unhealthy targets for Network Load Balancers in Amazon EKS?

I want to resolve unhealthy targets for Network Load Balancers in my Amazon Elastic Kubernetes Service (Amazon EKS).

Answer: "I want to resolve unhealthy targets for Network Load Balancers in my Amazon Elastic Kubernetes Service (Amazon EKS).Short descriptionThe following are common reasons why the targets for your Network Load Balancer are unhealthy:The health check is incorrectly configured. To resolve this issue, manually initiate the health check from a host machine that's running within the Amazon Virtual Private Cloud (Amazon VPC).There's an unexpected exception from the pod. To resolve this issue, follow the troubleshooting steps in the Check if there's an unexpected exception from the pod Resolution section.A Network Load Balancer with the externalTrafficPolicy is set to Local (from the Kubernetes website), with a custom Amazon VPC DNS on the DHCP options set. To resolve this issue, patch kube-proxy with the hostname override flag.Note: You can determine if the target group type is an IP address or instance by seeing if the service annotation service.beta.kubernetes.io/aws-load-balancer-nlb-target-type exists.ResolutionVerify if the target group is an IP address or instanceRun the following command:kubectl get service service_name -o yamlNote: Replace service_name with your service's name. If the service.beta.kubernetes.io/aws-load-balancer-nlb-target-type annotation isn't present, then the default target type is an instance.Verify that the health check is correctly configuredCheck which Elastic Load Balancing (ELB) annotations (from the Kubernetes website) are configured for your service:`kubectl get service service_name -o yaml`Example output:service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: "2"# The number of successive successful health checks required for a backend to be considered healthy for traffic. Defaults to 2, must be between 2 and 10service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "3"# The number of unsuccessful health checks required for a backend to be considered unhealthy for traffic. Defaults to 6, must be between 2 and 10service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "20"# The approximate interval, in seconds, between health checks of an individual instance. Defaults to 10, must be between 5 and 300service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "5"# The amount of time, in seconds, during which no response means a failed health check. This value must be less than the service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval value. Defaults to 5, must be between 2 and 60service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: TCPservice.beta.kubernetes.io/aws-load-balancer-healthcheck-port: traffic-port# can be integer or traffic-portservice.beta.kubernetes.io/aws-load-balancer-healthcheck-path# health check path.If the preceding annotations are incorrectly configured, then the targets can be unhealthy.Manually initiate the health check from a host machine that's running within the Amazon VPCFor instance target types, run the following curl command with NodePort:curl-ivk node_IP:NodePortNote: Replace node_IP with your node's IP address.For IP address target types, run the following curl command:curl -ivk pod_IP:pod_portNote: Replace pod_IP with your pod's IP address and pod_port with your pod's port.Check if there's an unexpected exception from the podInstance target typeCheck the service specification for the current health check configuration annotations (from the GitHub website):kubectl get service service_name -o yamlCheck if there are endpoints to verify that there are pods behind the service:kubectl get endpoints service_name -o yamlIf no endpoints exist for the service, then check that the pod labels and service labels match:kubectl describe servicekubectl describe pod pod_name or kubectl get pod --show-labelsNote: Replace pod_name with your pod's name.Check if the pods are in Running status:kubectl get pod -o wideCheck the pods' statuses to verify that the pods are running without any restarts:kubectl get pods -o wideIf there are restarts, then collect the pod logs to determine the cause:kubectl logs pod_namekubectl logs pod_name --previousLog in to a host machine in the Amazon VPC where you can communicate with the node.Use the curl command with NodePort to check if the pods are returning the expected HTTP status code:curl node_IP:NodePortIf the curl command didn't return the expected HTTP status code, then the backend pods also aren't returning the expected HTTP status code.Use the same host machine to connect to the pod's IP address and check if the pod is correctly configured:curl pod_IP:pod_portIf the curl command didn't return the expected HTTP status code, then the pod isn't correctly configured.Note: If the service's externalTrafficPolicy (from the Kubernetes website) is set to Local, then only the nodes where the service's backend pods are running are seen as healthy targets.IP address target typeCheck the service specification for the current health check configuration annotations (from the GitHub website):kubectl get service service_name -o yamlLog in to a host machine in the Amazon VPC and use the curl command to communicate with pod's IP address:curl pod_IP:pod_portIf the curl command didn't return the expected HTTP status code, then the pod isn't correctly configured.Patch kube-proxy with the hostname override flagModify the kube-proxy daemonset specification command, args, and env, with:---spec:  template:    spec:      containers:      - name: kube-proxy        command: [ "/bin/sh" ]        args:        - -c        - |          kube-proxy --v=2 --hostname-override=$(NODE_NAME) --config=/var/lib/kube-proxy-config/config        env:        - name: NODE_NAME          valueFrom:            fieldRef:              apiVersion: v1              fieldPath: spec.nodeNameFor instance target types, if the externalTrafficPolicy is set to Cluster or Local, then the node security group's default ingress setting for NodePort is 0.0.0.0/0. Also, when the externalTrafficPolicy is set to Local, an additional health check NodePort is configured to allow subnet CIDR IP address ranges.To control the source IP address on the node security group for NodePort, add loadBalancerSourceRanges in the specification and include the ranges:spec:loadBalancerSourceRanges:- "143.231.0.0/16"- "xx.yy.zz.zz/24"Note: If the .spec.loadBalancerSourceRanges isn't set, then Kubernetes allows traffic from 0.0.0.0/0 to the node security groups. If the nodes have public IP addresses, then non Network Load Balancer traffic can also reach every instance in the modified security groups.Follow"


Question: How can I resolve the Amazon SageMaker inference error "upstream timed out (110: Connection timed out) while reading response header from upstream"?

"When I deploy an Amazon SageMaker endpoint or run a BatchTransform job, the connection times out with an error like this:"upstream timed out (110: Connection timed out) while reading response header from upstream, client: 169.xxx.xxx.xxx, server: , request: "POST /invocations HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock/invocations" , host: "169.xxx.xxx.xxx:8080""

Answer: "When I deploy an Amazon SageMaker endpoint or run a BatchTransform job, the connection times out with an error like this:"upstream timed out (110: Connection timed out) while reading response header from upstream, client: 169.xxx.xxx.xxx, server: , request: "POST /invocations HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock/invocations" , host: "169.xxx.xxx.xxx:8080"Short descriptionThis error indicates a problem with the connection between NGINX and the web server. Both of these components run in the model container, regardless of whether you're using your own container or a pre-built container. These components aren't directly related to SageMaker hosting or batch transforms. However, when the connection between NGINX and the web server times out, then SageMaker can't obtain inference from the /invocations endpoint.To resolve this issue:Reduce the latency of the algorithm container or increase the container's timeout limit.Increase the NGINX.conf timeout settings.ResolutionReduce the latency of the algorithm container or increase the timeout limitIf you're running inference code for hosting services: Your model containers must respond to requests within 60 seconds. The model itself can have a maximum processing time of 60 seconds. If you know that your model needs 50-60 seconds of processing time, then set the SDK socket timeout to 70 seconds. For more information, see How your container should respond to inference requests.If you're running inference code for batch transform: Use ModelClientConfig to configure the InvocationsTimeoutInSeconds and InvocationsMaxRetries parameters.Amazon SageMaker sets environment variables specified in CreateModel and CreateTransformJob on your container. Adjust the following API parameters to reduce the latency of the algorithm container. For example, if the input is splittable, limit the payload size of each request by setting the MaxPayloadInMB field when you create a transform job.MaxPayloadInMB: The maximum size of the payload that is sent to the container. If the container can quickly process a batch transform, then increase this property. If the batch transform takes longer than expected, reduce this property.MaxConcurrentTransforms: The default is 1. Increase this setting if you have more than one NGINX worker.BatchStrategy: To fit as many records as possible in a mini-batch (up to the MaxPayloadInMB limit), set BatchStrategy to MultiRecord and SplitType to Line.If you're using a SageMaker framework container that implements Gunicorn, then pass these properties to the Docker container as environment variables:SAGEMAKER _MODEL_SERVER_TIMEOUT: The timeout for the Gunicorn server. To allow more time for the request to be processed before the connection is closed, increase this value.SAGEMAKER _MODEL_SERVER_WORKERS: The number of workers per CPU.Increase the NGINX.conf timeout settingsIf you're using one of Amazon SageMaker's prebuilt Docker containers, then you can't modify the NGINX.conf file. You can modify NGINX.conf only if you're using your own Docker container.NGINX timeouts can cause failures because Amazon SageMaker closes the connection after the timeout. If your container tries to read from or write to the closed connection, the request fails. Modify one or more of the following properties to accommodate for network overhead.proxy_read_timeout: This is the amount of time that NGINX waits for a response from the model after a request.send call. Increase this value to allow more time for Amazon SageMaker to process the request before the closing the connection.worker_processes: This is the number of threads for inbound connections. In most cases, the value should be the same or greater than the number of CPU cores. For example, for a two-core instance type such as ml.m5.large, set this property to a minimum of two.worker_connections: This is the maximum number of simultaneous connections for each worker process. It's a best practice to set the starting value for this to 1024.For more information about configuration settings, see Module ngx_http_proxy_module in the NGINX documentation.Follow"


Question: How do I remove the restriction on port 25 from my Lightsail instance?

I'm having trouble sending email over port 25 from my Amazon Lightsail instance.

Answer: "I'm having trouble sending email over port 25 from my Amazon Lightsail instance.Short descriptionBy default, AWS blocks outbound traffic on port 25 on all Lightsail instances. To send outbound traffic on port 25, request that this restriction be removed.ResolutionTo request that AWS remove the port 25 restriction on your Lightsail instance, do the following:Sign in to the Amazon Lightsail console as the AWS account root user.Open the Request to remove email sending limitations form.Enter your email address.Provide the required information in the Use case description field:A clear and detailed use case for sending email from your Lightsail instance.A statement outlining your plan to confirm that your account isn't implicated in sending unwanted emails.Confirmation that the request is for the Lightsail instance. You can also provide the Lightsail instance name.The Region where your Lightsail instance is hosted.(Optional) In the Elastic IP address field, provide the AWS owned Elastic IP addresses that you use to send outbound emails.(Optional) In the Reverse DNS record field, enter any reverse DNS (rDNS) records that AWS must associate with the Elastic IP addresses. When you send emails, it's a best practice to set up an rDNS record to help prevent outbound emails from being flagged as spam. Make sure to link the rDNS record to your static IP address using a DNS A record. For example, if mail.example.com is the rDNS record that you're setting, be sure that you create an A record for mail.example.com that points to the Static IP address. This is the domain that is returned when the reverse DNS lookup is performed.Choose Submit.Note: If you have instances in more than one Region, then submit a separate request for each Region.You receive an email with the Request ID after submitting the request form. It might take up to 48 hours to process your request. If your request is approved, you receive an email to notify you that the port 25 restriction is removed. If you don't receive an update within 48 hours after submitting the request, then reply to the initial email that you received.Related informationHow do I remove the restriction on port 25 from my Amazon EC2 instance or AWS Lambda function?Configuring reverse DNS for an email server on your Amazon Lightsail instanceFollow"


Question: Can I use Amazon S3 for Hadoop storage instead of HDFS?

I want to configure Amazon EMR to use Amazon Simple Storage Service (Amazon S3) as the Apache Hadoop storage system instead of the Hadoop Distributed File System (HDFS).

Answer: "I want to configure Amazon EMR to use Amazon Simple Storage Service (Amazon S3) as the Apache Hadoop storage system instead of the Hadoop Distributed File System (HDFS).ResolutionYou can't configure Amazon EMR to use Amazon S3 instead of HDFS for the Hadoop storage layer. HDFS and the EMR File System (EMRFS), which uses Amazon S3, are both compatible with Amazon EMR, but they're not interchangeable. HDFS is an implementation of the Hadoop FileSystem API, which models POSIX file system behavior. EMRFS is an object store, not a file system. For more information, see Hadoop documentation for Object Stores vs. Filesystems.For an overview of the storage layers in Amazon EMR, see Overview of Amazon EMR architecture.For recommendations about when to use each file system, see Work with storage and file systems.Related informationEMR File System (EMRFS)HDFS configurationFollow"


Question: How do I resolve authorization errors when trying to add subscribers to an Amazon SNS topic?

I receive AWS Identity and Access Management (IAM) authorization errors when I add subscribers to my Amazon Simple Notification Service (Amazon SNS) topic. How do I resolve the errors?

Answer: "I receive AWS Identity and Access Management (IAM) authorization errors when I add subscribers to my Amazon Simple Notification Service (Amazon SNS) topic. How do I resolve the errors?Short descriptionWhen an IAM entity tries to add a subscription to an Amazon SNS topic without Subscribe API action permissions, SNS returns the following error:An error occurred (AuthorizationError) when calling the Subscribe operation: User: your_IAM_user_or_role is not authorized to perform: sns:Subscribe on resource: YOUR_SNS_TOPIC_ARNNote: The IAM entity could be an IAM user or role.To resolve the error, grant the IAM entity permission to run the Subscribe API action on the Amazon SNS topic.If you receive a similar error that ends in an "explicit deny"Remove any policy statements that explicitly deny the IAM entity access to the SNS resource.ResolutionIf the IAM entity and the SNS topic are in different AWS accountsDo both of the following:Attach an IAM policy statement to the IAM entity that allows entity to run the "sns:Subscribe" actionFor instructions, see Adding and removing IAM identity permissions. Attach the following example IAM policy statement:Important: Replace "YOUR_SNS_TOPIC_ARN" with your Amazon SNS topic's Amazon Resource Name (ARN).{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "SNSSubscribePermission",            "Effect": "Allow",            "Action": "sns:Subscribe",            "Resource": "YOUR_SNS_TOPIC_ARN"        }    ]}Attach an Amazon SNS policy statement to your topic's access policy that allows the IAM entity to run the "sns:Subscribe" actionFor instructions, see How do I edit my Amazon SNS topic's access policy? Attach the following example Amazon SNS policy statement:Important: Replace "YOUR_IAM USER/ROLE_ARN" with your IAM entity's ARN. Replace "YOUR_SNS_TOPIC_ARN" with your Amazon SNS topic's ARN.{  "Sid": "AllowIAMEntity",  "Effect": "Allow",  "Principal": {    "AWS": "YOUR_IAM USER/ROLE_ARN "  },  "Action": "sns:Subscribe",  "Resource": "YOUR_SNS_TOPIC_ARN"}If the IAM entity and the SNS topic are in the same accountDo either of the following, but not both:Attach an IAM policy statement to the IAM entity that allows entity to run the "sns:Subscribe" action.-or-Attach an Amazon SNS policy statement to your topic's access policy that allows the IAM entity to run the "sns:Subscribe" action.For example policy statements, see the If the IAM entity and the SNS topic are in different AWS accounts section of this article.Verify that neither the IAM entity's policy or SNS topic's access policy explicitly denies access to the SNS resourceReview the IAM entity's policy and SNS topic's access policy. Then, remove any policy statements that explicitly deny the IAM entity access to the SNS resource.For more information, see The difference between explicit an implicit denies.Related informationUsing identity-based policies with Amazon SNSFollow"


Question: Why did AWS Global Accelerator fail over to an endpoint in a different Region?

I want to troubleshoot what caused AWS Global Accelerator to fail over to an endpoint in a different Region.

Answer: "I want to troubleshoot what caused AWS Global Accelerator to fail over to an endpoint in a different Region.Short descriptionThe AWS Global Accelerator uses traffic dial settings to route traffic to endpoint groups. It also uses endpoint weight to determine the proportion of traffic that's directed to an endpoint. If there's no healthy endpoint with a weight greater than zero (0), then Global Accelerator fails over to a healthy endpoint with a weight greater than 0 in another endpoint group.ResolutionPrevent Global Accelerator's failover to a different endpoint group by checking the traffic dial setting and the endpoint weights in the endpoint groups. Then, verify the health of the endpoints.Check the traffic dial settingGlobal Accelerator uses traffic dial to route traffic between different endpoint groups. An endpoint group with 0 traffic dial doesn't receive any traffic. However, if there's no healthy endpoint in an endpoint group with a weight greater than 0, then the accelerator fails over to another endpoint group with a healthy endpoint and a weight greater than 0. For this failover, the accelerator ignores traffic dial settings.Example: Let's say that you configure Global Accelerator with three endpoint groups, with traffic dial setting as shown here:EndpointGroup 1: Dial 100EndpointGroup 2: Dial 0EndpointGroup 3: Dial 0If EndpointGroup 1 doesn't have healthy endpoints with a weight greater than 0, then Global Accelerator checks up to two additional endpoint groups for a healthy endpoint. If endpoint groups 2 and 3 are closest to the client and are healthy with a weight greater than 0, then the accelerator fails over. It routes traffic to the endpoint in these endpoint groups, ignoring the traffic dial settings.Check weight of endpoints in endpoint groupsGlobal Accelerator uses weight to route traffic within an endpoint group. An endpoint with 0 weight does not receive any traffic, even if the endpoint is healthy. If all the endpoints have 0 weight, then the accelerator tries to find endpoints with a weight greater than 0 in another endpoint group. If it locates a healthy endpoint with a weight greater than 0, the accelerator routes traffic to that endpoint. If not, the accelerator fails open and includes endpoints in the nearest endpoint group.Example: Let's say that you've configured instance 1 as an endpoint in EndpointGroup 1 and instance 2 as an endpoint in EndpointGroup 2. If the weight of instance-1 is 0, then the accelerator fails over and routes traffic to instance-2 if it's healthy and has a weight greater than 0.Verify if the endpoints are healthyCheck if the endpoints are passing a health check by using health check status. For more information on how to remedy health check failures, see Why is Global Accelerator failing health checks with endpoints?Related informationDeploying multi-Region applications in AWS using AWS Global AcceleratorTraffic management with AWS Global AcceleratorFollow"


Question: How can I add periodic tasks to the worker tier of my Elastic Beanstalk environment and troubleshoot issues with periodic tasks that won't get added?

I want to add periodic tasks to the worker tier of my AWS Elastic Beanstalk environment.

Answer: "I want to add periodic tasks to the worker tier of my AWS Elastic Beanstalk environment.ResolutionNote: Elastic Beanstalk uses the aws-elasticbeanstalk-ec2-role instance profile role. This comes with the managed permissions AWSElasticBeanstalkWebTier, AWSElasticBeanstalkWorkerTier, and AWSElasticBeanstalkMulticontainerDocker. AWSElasticBeanstalkWorkerTier is required for periodic tasks to work. For more information, see Managing Elastic Beanstalk instance profiles.Add periodic tasks to a worker tierTo add a periodic task to the worker tier of your Elastic Beanstalk environment, include a cron.yaml file at the root of your source code. For example:version: 1cron: - name: "schedule"   url: "/schedule"   schedule: "0 */12 * * *"Note: In the preceding example, the schedule task periodically sends an Amazon Simple Queue Service (Amazon SQS) message that describes the task. The schedule task sends the message to the head of the queue every 12th hour. After the message transverses the queue, an instance picks up the task to HTTP POST the "/schedule" endpoint.If your periodic tasks still don't appear in the worker tier, then complete the steps in the following Troubleshoot issues with periodic tasks section.Troubleshoot issues with periodic tasks1.    In your /var/log/aws-sqsd/default.log platform file, check for a log similar to the following:schedule-parser: Successfully loaded 1 scheduled tasks from file /var/app/current/cron.yaml .2.    In your /var/log/aws-sqsd/default.log file, confirm that the Amazon SQS daemon is running and polling the correct queue.If the Amazon SQS daemon starts correctly, then you see logs similar to the following:2023-01-17T03:28:31Z init: initializing aws-sqsd 3.0.4 (2022-03-18)2023-01-17T03:28:31Z schedule-parser: Successfully loaded 1 scheduled tasks from file /var/app/current/cron.yaml .2023-01-17T03:28:31Z leader-election:  initialized leader election2023-01-17T03:28:31Z scheduler: initialized 1 job's pending time2023-01-17T03:28:31Z pollers: start initializting poller timer...2023-01-17T03:28:31Z pollers: start auto running poller...2023-01-17T03:28:31Z leader-election:  Starting leader election2023-01-17T03:28:31Z leader-election:  current role: worker2023-01-17T03:28:31Z scheduler: Starting scheduler2023-01-17T03:28:31Z start: polling https://sqs.us-east-1.amazonaws.com/111122223333/yourWorkerQueue3.    In your /var/log/aws-sqsd/default.log file, check for the HTTP POST request that the Amazon SQS daemon sends to http://localhost/ on port 80. When the daemon polls the queue item, it sends the request.You see a message similar to the following:2019-07-05T13:54:52Z message: sent to http://localhost:80/4.    To confirm that the Amazon SQS queue receives the messages, check the metric NumberOfMessagesReceived for the queue that's associated with the worker environment.5.    To invoke the scheduled task manually, run the following command:curl -d param1=value1 -d param2=value2 -H "Content-Type: application/json" -X POST http://localhost/scheduled-task-pathThe -d (--data) flag can hold the data that you're trying to pass to the periodic task.Note: Periodic tasks don't work for worker environments that are configured with Amazon SQS FIFO (First-In-First-Out) queues.Related informationHow do I create cron jobs on Amazon EC2 instances in Elastic Beanstalk environments?Periodic tasksFollow"


Question: How do I launch an Amazon EC2 instance with my AWS Marketplace subscription?

I have subscribed to an AWS Marketplace product. I want to know how to launch Amazon Elastic Compute Cloud (Amazon EC2) instances using this subscription.

Answer: "I have subscribed to an AWS Marketplace product. I want to know how to launch Amazon Elastic Compute Cloud (Amazon EC2) instances using this subscription.ResolutionYou can launch an EC2 instance from the AWS Marketplace console or directly from the AWS Marketplace.Launch an instance from the AWS Marketplace consoleOpen the AWS Marketplace console.Choose Manage subscriptions.Note: On the Manage subscriptions page, you can view all the product subscriptions on your account.Choose Manage next to the product that you want to launch an instance with, and then choose Actions.Choose Launch new instance.On the Configure this software page, select the required Delivery method, Software version, and Region, and then choose Continue to launch through EC2.You are redirected to the Amazon EC2 console.In the Choose an Instance Type page, select the required instance type, and then choose Next: Configure Instance Details.After you configure the instance, add storage, add tags, and then configure security groups, you can see the Review Instance Launch page.Review the instance details, and then choose Launch.Launch an instance using 1-Click LaunchSign in to AWS Marketplace.Choose the software subscription where you want to launch your instances.On the Launch on EC2 page, choose 1-Click Launch.Review the default settings for Version, Region, EC2 Instance Type, and Firewall Settings, and then update as needed.Expand Key Pair, and then choose an existing key pair or create a new one.Choose Accept Terms & Launch with 1-Click.Launch an instance using launch with EC2 ConsoleSign in to AWS Marketplace.Choose the software subscription where you want to launch your instances.On the Launch on EC2 page, choose Launch with EC2 console.Select an AMI version from the list.Review the default settings for Firewall Settings, Installation Instructions, and Release Notes, and then update as needed.Choose Launch with EC2 console.You are redirected to the Amazon EC2 console.Launch your instance by following the instructions in Getting started with Amazon EC2 Linux instances.The instance might take a few minutes to launch depending on your product subscription. You're charged the subscription price for all your running instances. If your instance is stopped, you might still be charged for storage.Related informationLaunching softwareGetting started as a buyer in the AWS MarketplaceAWS Marketplace Help and FAQFollow"