Question: AWS glue send an error of AccessDeniedException

"When I create any Glue crawler, connection, jobs at that time I get the below error.[gluestudio-service.ap-south-1.amazonaws.com] createJob: AccessDeniedException: Account 123456789 is denied access.FollowComment"

Answer: "0You would need to add permissions to your user - that seems to be issue here. If you have the ability to use built in permissions try adding the AWSGlueConsoleFullAccess  policy to your user ID.If your environment uses tighter access controls, I believe you need at least          {            "Effect": "Allow",            "Action": [                "glue:*",                "iam:ListRoles",                "iam:ListUsers",                "iam:ListGroups",                "iam:ListRolePolicies",                "iam:GetRole",                "iam:GetRolePolicy",                "iam:ListAttachedRolePolicies",                "s3:ListAllMyBuckets",                "s3:ListBucket",                "s3:GetBucketAcl",                "s3:GetBucketLocation",                "kms:ListAliases",                "kms:DescribeKey"            ],            "Resource": [                "*"            ]        },       {            "Effect": "Allow",            "Action": [                "s3:GetObject",                "s3:PutObject"            ],            "Resource": [                "arn:aws:s3:::aws-glue-*/*",                "arn:aws:s3:::*/*aws-glue-*/*",                "arn:aws:s3:::aws-glue-*"            ]        },        {            "Action": [                "iam:PassRole"            ],            "Effect": "Allow",            "Resource": "arn:aws:iam::*:role/AWSGlueServiceRole*",            "Condition": {                "StringLike": {                    "iam:PassedToService": [                        "glue.amazonaws.com"                    ]                }            }        }CommentShareAnanth Tirumanuranswered 8 months ago"


Question: How can I create an S3 object loading time chart in the CloudWatch custom dashboards?

"I want to create a bar chart where the Y axis should be a bucket's average S3 object loading time, and the X axis should be a bucket name. And each chart should represent a country's data. I got a similar picture for reference but not the exact one. Can anybody help me with that?FollowComment"


Question: Glue ETL PySpark Job Fails after Upgrade from Glue Version 2.0 to 3.0 error occurred while calling pyWriteDynamicFrame EOFException occurred while reading the port number from pyspark.daemon's stdout

"A PySpark Glue ETL job fails after upgrading from Glue Version 2.0 to 3.0.The job fails while writing a DynamicFrame to s3 in parquet format.error snippet from log:py4j.protocol.Py4JJavaError: An error occurred while calling o433.pyWriteDynamicFrame.: org.apache.spark.SparkException: Job abortedCaused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 30.2 failed 4 times, most recent failure: Lost task 1.3 in stage 30.2 (TID 2007594) (10.78.9.145 executor 8): org.apache.spark.SparkException: EOFException occurred while reading the port number from pyspark.daemon's stdoutFollowComment"

Answer: "0I changed the worker_type from G.1X to G.2X and the job completed successfully, albeit in 38 hours.  So then I tuned the Spark code so that the 3 Dataframes are all partitioned on the same value .repartition("attribute_name"), and also doubled the number of workers from 5 to 10.  Then the job completed successfully in 1 hr 20 mins.  The partitioning helped the JOIN that was being done to create the final dataset being written to s3.CommentSharemrjimianswered a year agoFabrizio@AWS EXPERTa year agothank you for feedback.How long was it taking with Glue 2.0?were you using the same number of nodes?Share0Hi,have you looked at the documentation about migrating Glue from version 2.0 to 3.0 ? and Spark 2 to Spark 3?Do you use external libraries?Contacting AWS Support might be the fastest way to resolve your issue if you cannot find any indication in the documentation shared, without seeing the job itself it is difficult to provide more prescriptive guidance.hope this helpsCommentShareEXPERTFabrizio@AWSanswered a year ago"


Question: "Why it is not possible to have two CloudFront distributions with the same alternate domain name, if they are not used simultaneously"

"Imagine following use case:First CloudFront distribution is used to serve the web application (multiple origins eg. ALB, Lightsail, ...)Second CloudFront distribution is used to show simple maintenance page from the S3 bucketWe want to use Route 53 to update the DNS record for our domain eg. example.com:to point to the first CloudFront distribution during the normal operation of the web applicationto point to the second CloudFront distribution during the maintenance of the web applicationIt is not possible to support this scenario (switching between two CloudFront distributions), because two CloudFront distribution cannot have the same alternate domain name. We wonder if it is not possible to do so for some special reason? Or is there any workaround?FollowComment"

Answer: "2It would be much better to do this with a single distribution - this is a fairly common requirement from customers and you can implement it using custom error pages in CloudFront, along with AWS WAF.To implement this:Create the maintenance page in an S3 bucket as you originally intendedCreate a new Origin in your CloudFront distribution that points to the S3 Bucket. Use OAI/OAC as required.Create a new Behavior in your CloudFront distribution with the S3 Bucket as the origin. Use a path that's not currently part of your application, etc /errorsConfigure Custom Errors for your distribution so that your maintenance page is served in the case of a 403 error. Add other error codes if you wish.Configure an AWS WAF WebACL and attach it to your CloudFront distibution. If you already have a WebACL attached, you can simply modify that one.Create a new custom rule in AWS WAF and place it at the top of the WebACL. The rule should match all traffic - but you may want to add an exception for your own IP address(es) so that you won't receive the maintenance page. Make sure the rule action is set to COUNTWhen you want to invoke the maintenance page, simply change the rule action to BLOCK - this will cause a 403 error to be returned to all of your viewers. The error will be handled by CloudFront, which will display your maintenance page. When you want to return the application back to service, simply switch the rule action back to COUNTIf you wish, you could use a Lambda function to toggle the rule action. You can then trigger the Lambda function in a variety of ways to suit your use case.CommentSharePaul_Lanswered 5 days agoGary Mclean 5 days agoPerfect way of doing this Paul L.. Theres no worries with DNS Cache here either..Sharepif-paf a day agoPaul L, your answer adds a component (WAF) which costs money. The OP's desired solution on the other hand, having a dormant Cloudfront maintenance distribution, and then switching DNS to that from the production distribution, would, if it were possible, carry no additional costs. Do you have a suggestion which does not cost money just to add this maintenance functionality?Sharepif-paf a day agoAlso, from AWS docs: "CloudFront can't distinguish between an HTTP status code 403 that is returned by your origin and one that is returned by AWS WAF". Maintenance pages are usually served with a 503. This solution does not nicely allow separate management of planned maintenance vs a genuine application-level "forbidden".SharePaul_L a day agoIf you prefer to use 503 status codes for your maintenance page, you can do this with a custom response action in WAF: https://docs.aws.amazon.com/waf/latest/developerguide/customizing-the-response-for-blocked-requests.htmlSharePaul_L 7 hours agoI recommended AWS WAF because it aligns to the Well Architected Framework (Apply security at all layers) and also is a best practice for DDOS mitigation. The intention is that it should be used for security use-cases, alongside this one. However, should you decide not to use AWS WAF, you could implement maintenance page functionality on a single CloudFront distribution by using a CloudFront Function to rewrite the URI of all incoming requests to that of your maintenance page. You would then enable/disable the maintenance page by attaching or removing the CloudFront Function.Share"


Question: Acount is in Suspended state

I am getting the below error while accessing the services.This account is currently blocked and not recognized as a valid account. Please contact aws-verification@amazon.com if you have questions.To Answer this we have uploaded all the required documents. But still the account is not activated.FollowComment

Answer: "0Hi,Thank you for reaching out to us. The Verifications team will take some time to review your documents. You can create a Phone or chat to talk to the AWS Support team (They are different from the AWS Verifications team).You can however express your urgency, if any by sending an email to 'aws-verification@amazon.com'.Just to understand the situation - Have you received any security emails from AWS about access keys?Please let us know. Thanks.CommentShareNithin Chandran Ranswered a year agoAWS-User-0527742 a year agoWe are on Free account, So will not be able to connect with AWS team through Chat or Phone.Kindly help us to resolve this issue at the earliest.Share0We have received only one email asking us to upload the utility bills, which we have done already.How can we initiate this Phone chat with AWS team?CommentShareAWS-User-0527742answered a year ago"


Question: allow only corporate email address in cognito

"We are building B2B saas app, we use cognito hosted login approach for sign-in. Can we restrict the access for public emails domains and allow only the corporate email ids ?FollowComment"

Answer: 0You could use a Lambda trigger to validate the domain of the entered email address: https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-pre-sign-up.html#aws-lambda-triggers-pre-registration-exampleCommentShareEXPERTBrettski-AWSanswered 2 months ago


Question: Understanding Amazon SES and SNS relationship - Email failures (bounces)

"I'm setting up Mautic and using Amazon SES to send the emails via SMTP. I'm a little confused about how SES & SNS work together...what exactly does SNS do and why did I have to set it up when I created my account.I was watching a tutorial on how to setup Amazon SES and because of that I created an SNS topic for (example):cats.comI created the SNS topic for this domain though I'm not actually using it (readon below)Later, I changed my mind for which domain I want to send emails from so I verified another domain called (example):dogs.comdogs.com has no SNS topic assignedI also verified the email that Mautic is setting as the "from" email address:marketing.dogs.commarketing.dogs.com has no SNS topic assignedYesterday I sent out a newsletter inside Mautic and then I started receiving a bunch of failure emails inside the marketing.dogs.com inbox...I'm aware that those are emails that failed to send for one reason of the other. Here are my questions, please help by answering them in point form because some are for my knowledge and some are to address the issue at hand#1 - Why did it send those failure emails to marketing.dogs.com?#2 - How did it know to send the failure emails it to marketing.dogs.com and not something like: emailfailures.dogs.com? (how do I set that?)#3 - Why am I receiving these failure emails to begin with?#4 - I want to not receive any emails about bounces/complaints etc inside the marketing.dogs.com mailbox. I see mautic can handle bounces and complaints via a webhook created in SES, see here: https://docs.mautic.org/en/channels/emails/bounce-management#amazon-webhook but in all screenshots/tutorials i come across they have this part of the topic set up that does not exist for me:So I am not sure how/why SES is sending failure emails to marketing.dogs.com mailbox (how does it know? does it just assume that "hey, you're the sender, so let me send back these failures to you"FollowComment"

Answer: "0Hello,Let me answer your question after providing a small explanation of what is going on.If your recipient's receiver (for example, an email provider) fails to deliver your message to the recipient, the receiver bounces the message back to Amazon SES. Amazon SES then notifies you of the bounced email through email or through Amazon Simple Notification Service (Amazon SNS), depending on how you have your system set up.AWS is legally obligated to send you bounce and complaint notifications. So there is no way to disable these notifications. In order to send email using Amazon SES, you must have a system in place for managing bounces and complaints. Amazon SES can notify you of bounce or complaint events in three ways: by sending a notification email, by notifying an Amazon SNS topic, or by publishing sending events. You have to enable one method of receiving bounce or complaint notifications. You can send notifications to the domain or email address that generated the bounce or complaint, or to an Amazon SNS topic. If you don't set up one of these methods of receiving bounce or complaint notifications, Amazon SES automatically forwards bounce and complaint notifications to the Return-Path address (or the Source address, if you didn't specify a Return-Path address) in the email that resulted in the bounce or complaint event, even if you disabled email feedback forwarding. If you disable email feedback forwarding and enable event publishing, you must apply the configuration set that contains the event publishing rule to all emails you send. In this situation, if you don't use the configuration set, Amazon SES automatically forwards bounce and complaint notifications to the Return-Path or Source address in the email that resulted in the bounce or complaint event.Now to your questions:#1 - Why did it send those failure emails to marketing.dogs.com?Answer:marketing.dogs.com is the one you used in either the Return-Path address, or the FROM address. Since you have disabled Feedback Forwarding, a notification will be sent to the email address specified in either the Return-Path, or the FROM headers.#2 - How did it know to send the failure emails it to marketing.dogs.com and not something like: emailfailures.dogs.com? (how do I set that?)Answer:It sent to marketing.dogs.com because it was specified in the FROM or the Return-Path headers. You can control that by providing a Return-Path header when you send out a message.#3 - Why am I receiving these failure emails to begin with?Answer:SES is legally obligated to inform the customers of all bounces and complaints (Incase SES receives a notification from the MTA "Mail Transfer Agent" that a message has been marked as SPAM). These notifications should be used by the customer to refine their mailing list to maintain their domain reputation and IP reputation if they are using dedicated IPs. If SES did not provide these notifications, and the reputation of your domain or dedicated IP is damaged and no more message can be sent out from a specific domain, as all ISPs have marked this domain as a spam generator, then the domain itself might be lost and no messages can be sent from it any more. This is why SES has to inform the customers of all the bounces and complaints if available.For more about this, please refer to the below documentation linkhttps://docs.aws.amazon.com/ses/latest/dg/monitor-sending-activity-using-notifications.htmlI believe this covers all of your concerns.Best Regards,MoCommentShareMoanswered 2 months ago"


Question: VPC Interface Endpoints and API Gateway called from Lambdas

"I have a work problem, where for security reasons we are moving all Lambdas into VPC, and using PrivateLink for all AWS services, via Interface Endpoints added to the VPC. I am having an issue trying to call a REST API in API Gateway with an Endpoint Interface (private DNS enabled) for the execute-api service.This is fine for Private APIs, where our VPC is added to API resource policy, and we can call the API with no problems. However we need to call webhooks from one lambda, which may or may not be in AWS. Outside of AWS works fine, but to our unit test webhook (REGIONAL) REST API in the same account fails with a 403 error unless I make it private and attach the endpoint id for API Gateway. The security group on the lambda allows all outgoing traffic, and I can call the webhook from the internet via postman with no issues.I'm struggling to work out why the 403 error occurs when REGIONAL, and why it needs to be PRIVATE to work. We will be calling random webhook URLs inside and outside AWS, and my understanding was that the endpoint interface was the entry point to Private link, so that all API Gateway calls were not via the public endpoints, but otherwise behaved the same. Does anyone know what I am missing here, I can't seem to work out where I've gone wrong from the docs...Much appreciated.FollowCommentImtranur_R a year agoCould you please Check the DNS setting on a VPC endpoint:If the API is invoked from an Amazon VPC that has an interface VPC endpoint, verify that the DNS setting of the interface endpoint is set correctly based on the API type.Keep in mind the following:To invoke a Regional API from inside an Amazon VPC, private DNS must be disabled on the interface endpoint. Then, the endpoint's hostname can be resolved by a public DNS. For more information, see Why do I get an HTTP 403 Forbidden error when connecting to my API Gateway APIs from a VPC?To invoke a private API from inside an Amazon VPC using the API's private DNS name, private DNS must be activated on the interface endpoint. Then, the interface endpoint's hostname can be resolved to the Amazon VPC's local subnet resources. For more information, see How to invoke a private API.Note: You don't need to set up private DNS if you're invoking the private API using its public DNS name or an Amazon Route 53 alias.https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-troubleshoot-403-forbidden/Share"

Answer: "3Accepted AnswerHi WearsTheFoxHat,If you enable "Enable Private DNS Name" when you add an API GW VPC endpoint to your vpc, all REGIONAL and PRIVATE APIs will get routed to the VPC endpoint.As is stated in [1], "However, you cannot access public APIs from a VPC by using an API Gateway VPC endpoint with private DNS enabled. " (emphasis mine)You can solve this in two ways:leave private DNS Name enabled and make sure all your public APIs are accessed from custom domain names that are Edge optimizeddisable private DNS name, but that will require that you call your private APIs thru the endpoint setting the Host header explicitly [2] or using a Route 53 alias [3]Hope this helps!Best, Alf[1] Creating a private API in Amazon API Gateway - Create an interface VPC endpoint for API Gateway execute-api - https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-apis.html#apigateway-private-api-create-interface-vpc-endpoint[2] How to invoke a private API  - Invoking your private API using endpoint-specific public DNS hostnames  - https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-api-test-invoke-url.html#apigateway-private-api-public-dns[3] How to invoke a private API  - Accessing your private API using a Route53 alias - https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-api-test-invoke-url.html#apigateway-private-api-route53-aliasCommentShareAlfanswered a year agoEXPERTHernitoreviewed a year agoWearsTheFoxHat a year agoah, I hadn't read the API gateways docs in 1 (someone else manages that bit) so hadn't come across this restriction in the Endpoint docs. I probably can't ask clients to use edge optimised Gateways, so will need to Heath-Robinson something else I think and get it past the Security team...Many thanks for your time.Share0To reach a public webhook you need to grant your VPC lambda access to internet.  This is a bit more involved than security group setting. This article might help:https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/CommentSharesinaanswered a year ago0Could you please Check the DNS setting on a VPC endpoint:If the API is invoked from an Amazon VPC that has an interface VPC endpoint, verify that the DNS setting of the interface endpoint is set correctly based on the API type.Keep in mind the following:To invoke a Regional API from inside an Amazon VPC, private DNS must be disabled on the interface endpoint. Then, the endpoint's hostname can be resolved by a public DNS. For more information, see Why do I get an HTTP 403 Forbidden error when connecting to my API Gateway APIs from a VPC?To invoke a private API from inside an Amazon VPC using the API's private DNS name, private DNS must be activated on the interface endpoint. Then, the interface endpoint's hostname can be resolved to the Amazon VPC's local subnet resources. For more information, see How to invoke a private API.Note: You don't need to set up private DNS if you're invoking the private API using its public DNS name or an Amazon Route 53 alias.https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-troubleshoot-403-forbidden/CommentShareImtranur_Ranswered a year ago"


Question: Bad performance on RDS MariaDB

"I'm migrating an ancient  on-premises 5.5.65 MariaDB to RDS.The new  RDS MariaDB 10.6 not show CPU, IOPs or Memory problems but the SQLs are too slow.One complex SELECT cost 0,89 sec on premises and 2,93 using an EC2 in the same VPC.I have compared the execution plans an are different but probably is because the different versions.On_premises executionEC2 over RDS in the same VPCI don't know if the differences on the "key" column are relevant to this issue.Any experience on bad performance with RDS and MariaDB?FollowComment"

Answer: "1Accepted AnswerIt seems different optimization was applied. Rowid filtering is an optimization available from MariaDB 10.4.3+, so please disable rowid_filter optimizer switch, and re-issue the query.SET SESSION optimizer_switch='rowid_filter=off';, then issue the query.https://mariadb.com/kb/en/rowid-filtering-optimization/https://mariadb.com/kb/en/server-system-variables/#optimizer_switchCommentShareYUTAKA_Hanswered 8 months agoCarlos Garces 8 months agoThanks! I need to investigate why a new optimization is degrading performance, but works as workaround.Share"


Question: default encryption between ALB and target groups.

"as per the AWS documentation  all traffic with in AWS cloud is encrypted which suggests the traffic from load balancer and a ec2 target should be encrypted as well.I understand there is an option to enable end-2-end encryption with self signed certs but i am trying understand if my above assumption is accurate.if it is accurate assumption what will be the use case to enable  encryption  between ALB and EC2 except for compliance reasons or to avoid eavesdropping within aws cloud, etc ?FollowComment"

Answer: "1If strict encryption of communication is required by PCI SSC requirements, etc., the communication between ALB and EC2 may also be HTTPS.CommentShareEXPERTRiku_Kobayashianswered 12 days ago1The link you posted does say this, it is physical layer encryption (layer1) and not layer7 (TLS) therefore although there are native mechanisms for authentication etc (see the highlighted paragraph, note that it says authentication and not encryption) it is recommended to use self-signed certs to encrypt the traffic between ELB and targets if you are looking to achieve end-to-end encryption.All network traffic between AWS data centers is transparently encrypted at the physical layerPlease see this link which has specific recommendations for the customers: Data protection in Elastic Load BalancingAlso, the below paragraph:The load balancer establishes TLS connections with the targets using certificates that you install on the targets. The load balancer does not validate these certificates. Therefore, you can use self-signed certificates or certificates that have expired. Because the load balancer is in a virtual private cloud (VPC), traffic between the load balancer and the targets is authenticated at the packet level, so it is not at risk of man-in-the-middle attacks or spoofing even if the certificates on the targets are not valid (1).Reference:[1] https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html#target-group-protocol-versionCommentShareEXPERTTushar_Janswered 12 days ago"


Question: How do I handle WAF Size Constraint rules?

"I'm trying to find a way to block requests that are not inspected due to the size constraints; for example, if a rule is longer than 8k and the entire rule is not inspected due to the size constraint, I would want this blocked. To satisfy this solution would I use "MATCH"?My Queston is similar to this here - https://repost.aws/questions/QU_QAauDSSTR6hmnKQj3F19g/how-to-set-oversize-handling-wafThough my confusion is regarding the blocked vs allow. If the reqeust happens to be over 8k does that mean an attacker can bypass the WAF inspection? For example, if the request is longer than 8k and the WAF doesn't inspect the requests that exceeds the 8k, would that be "ALLOWED" if I set the WAF handling to "CONTINUE"?FollowComment"

Answer: "0Hello,AWS WAF does not support inspecting very large contents for the body and cannot inspect >8KB of body of a request. To immediately block an oversized request (>8KB) without inspecting, you can use the Match option in the rule with action set to Block.With Continue option, request will be inspected normally according to the rule inspection criteria only within the size limitations (8KB).https://docs.aws.amazon.com/waf/latest/developerguide/web-request-body-inspection.htmlAnother option to block oversized request is using AWS Managed Rule groups where the managed rule group - Core Rule Set (CRS) blocks the request over 8KB.https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-baseline.html#aws-managed-rule-groups-baseline-crsCommentShareVaishnavi Ganesananswered 5 months ago0Hello AWS Customer,First of all, as you know, AWS WAF does not support inspecting very large contents for the body, headers, or cookies request components.[1]The limitations are as follows:Body and JSON Body – You can inspect the first 8 KB (8,192 bytes) of the body of a request.Headers – You can inspect at most the first 8 KB (8,192 bytes) of the request headers and at most the first 200 headers. The content is available for inspection by AWS WAF up to the first limit reached.Cookies – You can inspect at most the first 8 KB (8,192 bytes) of the request cookies and at most the first 200 cookies. The content is available for inspection by AWS WAF up to the first limit reached.For these components, you provide oversize handling instructions when you define your rule statement. Oversize handling tells AWS WAF what to do with a web request when the request component that the rule inspects is over the limits.The options for oversize handling are the following:Continue – Inspect the request component normally according to the rule inspection criteria. AWS WAF will inspect the request component contents that are within the size limitations.Match – Treat the web request as matching the rule statement. AWS WAF applies the rule action to the request without evaluating it against the rule's inspection criteria.No match – Treat the web request as not matching the rule statement, without evaluating it against the rule's inspection criteria. AWS WAF continues its inspection of the web request using the rest of the rules in the web ACL, like it would do for any non-matching rule.If you use the "Match" option in a rule with the action set to "Block", the rule will "block" a request where the component type is oversized. With other configurations, the final disposition of the request depends on various factors, such as the configuration of the other rules in your web ACL and the web ACL's default action setting.Hope you will find this information useful. Please let me know if you have any further questions or concerns. We will be more than glad to assist you.I hope you are always healthy and have a nice day.CommentShareSUPPORT ENGINEERJisoo_Kanswered 5 months ago"


Question: OpenSearch Service  - How to restore .opendistro_security index

"The cluster status was red, so I made a list of unallocated shards with this reference,The .kibana_1 and .opendistro_security shards were unassigned.curl -XGET {domain-endpoint}/_cat/shards?h=index,shard,prirep,state,unassigned.reason  | grep UNASSIGNED.opendistro_security   0 p UNASSIGNED NODE_LEFT.opendistro_security   0 r UNASSIGNED PRIMARY_FAILED.kibana_1              0 p UNASSIGNED NODE_LEFT.kibana_1              0 r UNASSIGNED REPLICA_ADDEDI was able to delete .kibana_1 and restore it from the snapshot, but when I tried to delete .opendistro_security I got the following error and could not delete it.{  "error": {    "root_cause": [       {         "type": "security_exception",         "reason": "no permissions for [] and User [name={admin-name}, backend_roles=[], requestedTenant=null]"        }     ],     "type": "security_exception",     "reason": "no permissions for [] and User [name={admin-name}, backend_roles=[], requestedTenant=null]"   },  "status": 403}How can I restore .opendistro_security from a snapshot?FollowComment"

Answer: "0The ".opendistro_security" index is an internal index and managed by AWS OpenSearch. This means that it cannot be deleted from customers end.In addition, due to special permissions on the OpenSearch Dashboards and fine-grained access control indexes, attempts to restore all indexes might fail, especially if you try to restore from an automated snapshot.If customer needs to delete/restore the ".opendistro_security" index, they would need to contact our AWS Support team for assistance.In the meantime, you may want to consider restoring all indexes except the Dashboards and fine-grained access control indexes. You can do this using the following command:curl -XPOST 'domain-endpoint/_snapshot/cs-automated/2020-snapshot/_restore' \-d '{"indices": "-.kibana*,-.opendistro*"}' \-H 'Content-Type: application/json'This will restore all indexes in your cluster except for the Dashboards and fine-grained access control indexes.For more information kindly refer below documentation:[+] Restoring snapshots in Amazon OpenSearch Service - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-snapshots.html#managedomains-snapshot-restorePlease consider opening a support case for further assistance if required.CommentShareSUPPORT ENGINEERSachin_Kanswered a day ago"


Question: Home directory - Restricted in aws transfer family using cloud formation template

"Hi,For adding a user with a home directory mapping, I tried the below stack template.However, the home directory was not created after stack was run. It was in restricted mode.If we only do to edit user configuration manually we can uncheck restricted.I want to implement this mode in yaml template.Please help me to do better.GoldcoastTvodUser:    Type: 'AWS::Transfer::User'    Properties:      HomeDirectoryMappings:        - Entry: /          Target: /goldcoast-tvod      HomeDirectoryType: LOGICAL      Policy:        'Fn::Sub': |          {            "Version": "2012-10-17",            "Statement": {              "Sid": "AllowFullAccessToBucket",              "Action": "s3:*",              "Effect": "Allow",               "Resource": [                "arn:aws:s3:::goldcoast-tvod",                "arn:aws:s3:::goldcoast-tvod/*"              ]            }          }      Role:        'Fn::Sub': 'arn:aws:iam::${AWS::AccountId}:role/TransferManagementRole'      ServerId:        'Fn::GetAtt': TransferServer.ServerId      SshPublicKeys:        - >-          ssh-rsa          AAAAB      UserName: GoldcoastTvodUserFollowComment"

Answer: "1Accepted AnswerIf you change the HomeDirectoryType to PATH, it'll remove the restricted.      HomeDirectory: "/goldcoast-tvod"      HomeDirectoryType: "PATH"P.S. although it's only a public SSH key, you might want to remove it from your question.CommentShareEXPERTRaphaelanswered 2 months agoMahendra Kumar V 2 months agoHi,Thanks much i have removed my public key and let me try your update and let you the working status.Thanks,ShareMahendra Kumar V 2 months agoHi,Thank you for the answer and YES! that was the fix.GoldcoastTvodUser:Type: 'AWS::Transfer::User'Properties:HomeDirectory: "/goldcoast-tvod"HomeDirectoryType: "PATH"Share"


Question: License for AWS Greengrass non-open source components?

How are the non-open sourced components (https://docs.aws.amazon.com/greengrass/v2/developerguide/public-components.html) for Greengrass licensed? Does it fall back to the Amazon Software License? https://aws.amazon.com/asl/FollowComment

Answer: 0Found it: https://aws.amazon.com/service-terms/#55._AWS_GreengrassDirect download: https://s3-us-west-2.amazonaws.com/greengrass-release-license/greengrass-license-v1.pdfWhy that is not actually linked from the Greengrass pages themselves remains a mysteryCommentShareoaustegardanswered 2 years ago


Question: Lambda - Streaming - Increase quota

"Hi,I would like to use RESPONSE_STREAM option on lambda.According to the documentation, we need to increase the payload size: 20 MB instead of 6MB.I tried to increase these values in quotas, but it seems impossible. Not adjustable. How to make it changeable ?Thanks for helpFollowComment"

Answer: "2Using Lambda response streaming you can return by default payloads up to 20 MB, and if you need more, you can increase it. There is no need to increase if you do not need more than 20 MB.The quotas that show in the Service Quota page are for synchronous and asynchronous invocations. Not for stream invocations. I assume that you should create a support ticket to increase the streaming payload size limit.CommentShareEXPERTUrianswered a month ago0Accepted AnswerOhw ! Seems an issue with Postman. It works perfectly in CLI with Curl. Thanks Uri !CommentSharerePost-User-7795271answered a month ago0Because when I try to implement a node function with RESPONSE_STREAM, it behaves like it is buffering, he here the code :(I followed this tutorial, and I run in Node 16)export const handler = awslambda.streamifyResponse(async (event, responseStream, context) => {    responseStream.setContentType("text/plain");        const sentences = [        "1...",        "2...",        "3...",        "4...",        "5...",        "6...",        "7...",        "8...",        "9...",    ]        const timer = ms => new Promise(res => setTimeout(res, ms));        for (let i = 0; i < sentences.length; i++) {        responseStream.write(`${sentences[i]}\n`);        await timer(1000);    }    responseStream.end();});CommentSharerePost-User-7795271answered a month ago"


Question: scheduling a AWS lex intent as push notification.

"Hai There,i have a requirement to schedule a lex intent with pinpoint push notification.consider this example everyday morning at 9 AM i would like to send a push notification to some users for a survey which will have some questions to answer which is a lex and lambda integration.currently i am able to work with lex by calling the utterance, but what i want to achieve is connect to lex without user calling the utterance. we can think of it like a bot first kinda conversation.Thanks for the help in advance,charith v.FollowComment"

Answer: "2In order to start a conversation without specific user input you can leverage PutSession API. PutSession API creates a new session or modifies an existing session. In this case if you have ability to make a PutSession call as soon as the user clicks on notification that might be a possible approach for you.PutSession API is supported both in Lex V1 and Lex V2. In Lex V2 to achieve this, you can set the intent in session state and Lex will respond with appropriate next response. You can then play/print that response back to the user continuing subsequent conversation from that point.Example PutSession Input:{    "sessionState": {        "dialogAction": {            "type": "Delegate"        },        "intent": {            "name": "BookCar"        }    }}In this case I am setting 'BookCar' as the current intent and Delegating the next step to Lex. In response header I get the bot response as x-amz-lex-messagesCommentShareSaketanswered a year agoAWS-User-1009878 a year agoHai @Saket,Thanks for the response which was really helpful for me to understand that we can invoke conversation via a HTTP API.i am using lex v1 so searching around docs i found that this is the endpoint https://runtime.lex.eu-central-1.amazonaws.com and https://docs.aws.amazon.com/lex/latest/dg/API_runtime_PutSession.html these are the docs.But when i try to invoke the api endpoint i get {"message":"Missing Authentication Token"} i am unable to find where to get and use Authentication token for LEX.And another question i have is is there any way to invoke this via amplify on ReactNative?ShareSaket a year agoTo make REST API calls you must sign the request with appropriate authentication data. More detail on this is documented at Signing and authenticating REST requests. To ease your testing you can use tools like Postman which will make it easier to sign and manage the auth tokens.Regarding integration with Amplify: I personally do not have experience with it so cannot comment on feasibility. But I will suggest you to checkout into Lex JavaScript SDK and if that can be integrated within your ReactNative app. Good Luck!ShareAWS-User-1009878 a year agoThanks @Saket,I have opened a new discussion on aws-amplify's repohttps://github.com/aws-amplify/amplify-js/discussions/9472ShareAjayM-AWS a year agoHello, just wanted to follow up and ask if there was any other assistance that we could provide. Otherwise, if you found Saket's answer useful, could you please mark it as the "Accepted Answer". This will benefit others facing similar issues.Alternatively, if you need additional assistance, we can help answer Lex specific questions.Thank you.Share"


Question: How to manage thumbnails in AWS ivs.

I want to get the thumbnails of the live streams. I am not recording the lives streams in s3 bucket. How can I retrieve their thumbnails?ThanksFollowComment

Answer: "0Hi,If you record the live stream to S3 Amazon IVS will generate the thumbnails for you in a specific directory (/media/thumbnails) under this /ivs/v1/<aws_account_id>/<channel_id>/<year>/<month>/<day>/<hours>/<minutes>/<recording_id>. There's more info on how to do it in this doc : https://docs.aws.amazon.com/ivs/latest/userguide/record-to-s3.htmlThanksCommentSharePedro_Fanswered 4 months ago"


Question: AWS File Transfer Family Server and IAM role setup

"Hi All,We have setup AWS file transfer server with AWS directory service (connected to Microsoft AD) authentication. As per use case, once user login to sftp, user should be able to see two directory within their own folder.{username}/folder1{username}/folder2I have setup below Access policy and IAM policy (attached to S3)create-access CLI:aws transfer create-access \    --home-directory-type LOGICAL \    --home-directory-mappings '[{"Entry":"/folder1","Target":"/bucket_name/${transfer:UserName}/folder1" },{ "Entry": "/folder2", "Target":"/bucket_name/${transfer:UserName}/folder2"}]' \    --role arn:aws:iam::account_id:role/iam_role \    --server-id s-1234567876454ert \    --external-id S-1-2-34-56789123-12345678-1234567898-1234access policy was created successfully.Below IAM role is attached to S3 bucket and file-transfer server.{    "Version": "2012-10-17",    "Statement": [        {            "Action": [                "s3:ListBucket",                "s3:GetBucketLocation"            ],            "Resource": [                "arn:aws:s3:::bucket_name"            ],            "Effect": "Allow",            "Sid": "ReadWriteS3"        },        {            "Action": [                "s3:PutObject",                "s3:GetObject",                "s3:DeleteObject",                "s3:DeleteObjectVersion",                "s3:GetObjectVersion",                "s3:GetObjectACL",                "s3:PutObjectACL"            ],            "Resource": [                "arn:aws:s3:::bucket_name/${transfer:UserName}/*"            ],            "Effect": "Allow",            "Sid": ""        }    ]}When user login to sftp, they do not see folder1 & folder2 in their own directory. Can anyone help if anything missing in IAM policy?Thank YouFollowComment"

Answer: "0Does the IAM role have permissions to folder1 and folder2 locally?CommentSharejschwar313answered 7 months agoUser-07051983 7 months agoHi @jschwar313Thanks for reply.Yes, IAM role has permission to all folders within user directory."arn:aws:s3:::bucket_name/${transfer:UserName}/*"When user logged in, they land into their own directory (based on username) but "folder1" and "folder2" directory are missing. I checked directly into S3 bucket as well with admin access, dont see directory auto created under username folder. However folder based on username is created automatically under S3.Share0Hello,Thank you for your response. I understand that the SFTP user is unable to see "folder1" and "folder2" when connected.Please note that these folders need to be first created in S3; these sub-folders are not automatically created on your behalf.I tested a setup at my end to validate this configuration. If these folders are not pre-created in S3, then the SFTP user will not be able to see them when connected via SFTP.Requesting you to create - "folder1" and "folder2" under "${transfer:UserName}" prefix in the S3 bucket and then try to reconnect from SFTP.Thank you.CommentShareSUPPORT ENGINEERSamreen_Tanswered 7 months ago0I tried to create folder under "${transfer:UserName}" prefix but it does not allow. I am sure, I am doing something wrong. We have 3000 users and each user will land in their own directory only and should see "folder1" and "folder2" thats the use case.CommentShareUser-07051983answered 7 months ago"


Question: "AWS Lambda, Python : Call Shell script from Lambda or Linux command"

"I am working on an AWS Lambda script written in Python where I am currently getting all the instances with specific tags, and removing the oldest one from them. After that, from the remaining instances, I would like to call a linux command on the instances. The only thing I require is to call crontab -r , as the oldest instance will have the cron set, and adding those crons in the ASG generated instances will cause duplicate emails being sent.I am done till the part of getting all the instances except the oldest one, but how can I call crontab -r on each of those instances? Any ideas. Thank you.Code :import boto.ec2import boto3conn=boto.ec2.connect_to_region("eu-central-1")reservations = conn.get_all_instances()instances_list = []process_instance_list = []for res in reservations:for inst in res.instances:if 'Name' in inst.tags:if inst.tags['Name'] == 'PROJECT_NAME' :instances_list.append(inst);instances_list.sort(key=lambda x: x.launch_time, reverse=False)non_processed_id=instances_list[0]for val in instances_list:if val.id != non_processed_id.id:// Call crontab -r here.FollowComment"

Answer: "0Accepted AnswerUse boto3 send_command to execute a command on ec2.Example for your case:boto3.client('ssm').send_command(InstanceIds=[val.id],DocumentName='AWS-RunShellScript',Parameters={'commands': ['crontab -r']},Comment='Crontab remove')CommentShareharshmanvaranswered 4 years ago"


Question: HOW TO set up the required IAM permissions for WorkMail??

Does anybody know how to set up the require IAM Permission for WorkMail??I just created a new access key and I do not know where to replace the old key with this new key...Can someone help me??FollowComment

Answer: "0Hi!  If you're trying to reset a user password for WorkMail, go to the workmail console (https://console.aws.amazon.com/workmail/), then go to organizations, then users, then select the user and reset their password.If you're looking for managing WorkMail via an IAM User or Role, you can manage credentials via the IAM Console or API: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.htmlCommentSharejsoncanswered a year ago"


Question: Unable to set up GUI using TigerVNC after starting new instance

"Sorry if this is basic but I'm new and in a bind.Previously I was able to set up a GUI on my lightsail instance by following this guide. It worked good but my project requires more RAM than I had available so I used a snapshot to start a new instance with better specs and shut the original down.Now I am unable to get the GUI working. All is well up until:Open the TigerVNC Viewer on your local computer. When asked for the VNC server hostname, enter localhost:1 and then connect to it.TigerVNC throws the error "The connection was dropped by the server before the session could be established." I'm able to ssh into the instance, everything is installed and set up correctly (as far as I can tell), but for some reason I cannot tunnel in with TigerVNC. It worked on the previous instance.FollowComment"

Answer: "0The instructions you linked to depend on an SSH tunnel being established before you can connect to the VNC server on the remote. The following line:ssh -L 5901:localhost:5901 -i PEM_FILE ec2-user@INSTANCE_IPfrom the instructions is critical. Is that, specific, ssh session running and is PEM_FILE and INSTANCE_IP properly replaced with values from your new, larger instance?Suggestions:Use a static IP so that INSTANCE_IP doesn't change.Furthermore, use a DNS name pointing to that static IP to make it even easier.I hope this helps!DonleyCommentSharedonleyatawsanswered 7 months ago"


Question: Support domain registration for .pl domains

I'd like to ask for supporting registrations/transfers for '.pl' (Poland) domains in Route53.FollowComment

Answer: "0Hello,There is already a request to have ".pl" top level domain available within Route 53. There is no ETA as AWS does not publicize roadmap items, but you can check in periodically to see if this domain extension is available by reviewing these AWS documentation pages:Domains that you can register with Amazon Route 53What's New with AWS?AWS blogsThat said, one way to use R53 (for better integration with AWS services, for example) without actually transferring the domain, is to update the name servers in your current registrar with R53 name servers. You can refer to the process here.CommentShareEXPERTTushar_Janswered 10 months ago"


Question: "Api Gateway, parse a 429 Response"

"I have an API Gateway in front of a Lambda that has reserved concurrency.  When the Lambda returns a 429 for throttling, I'd like to reply to the API client with a 429 Method Response.  I have other Integration Responses working for other Method Responses, based on the Lambda's response body.  However, I can't get the integration response to process the response body like it does with other responses; perhaps because those responses are http 200's, and this is a non-200?Wed Jun 08 20:17:53 UTC 2022 : Endpoint response body before transformations:{"Reason":"ReservedFunctionConcurrentInvocationLimitExceeded","Type":"User","message":"Rate Exceeded."} Wed Jun 08 20:17:53 UTC 2022 : Lambda invocation failed with status: 429. Lambda request id: 00000000-e2b9-4000-9f00-700000007a(I'd expect the mapping template to be able to parse the above json in the "Endpoint response body before transformations" message, like the other responses being handled, but nope...)Wed Jun 08 20:17:53 UTC 2022 : Execution failed due to configuration error: Rate Exceeded.Wed Jun 08 20:17:53 UTC 2022 : Method completed with status: 500How can I parse the response body of a non-200 response?FollowComment"

Answer: "0Have you tried creating a Method Response of 429 first and then going back to the Integration Response and trying to map 429 in the Integration Response to 429 in the Method Response and keeping the content as Passthrough. What happens? Do you get back a 429 or a 500 in the response to the client?CommentShareEXPERTIndranil Banerjee AWSanswered a year agorePost-User-9020933 a year agoI get the same experience whether trying to pass through, or trying to use a mapping templage.Share"


Question: Creating Multiple S3 Buckets at Once

I wanted to know if there is way for creating multiple storage accounts in bulk in AWS?FollowComment

Answer: "1Accepted AnswerTo create storage accounts in bulk in AWS, you can use AWS CloudFormation.You can use the following steps to create storage accounts in bulk:Create an S3 bucket to store your CloudFormation templates.Create a CloudFormation template that defines your storage accounts. You can use the AWS::S3::Bucket resource type to define your storage accounts. Here's an example:yamlResources:  MyBucket1:    Type: AWS::S3::Bucket    Properties:      BucketName: my-bucket-1      AccessControl: Private      VersioningConfiguration:        Status: Enabled  MyBucket2:    Type: AWS::S3::Bucket    Properties:      BucketName: my-bucket-2      AccessControl: Private      VersioningConfiguration:        Status: EnabledSave the template to your S3 bucket.Use the AWS CLI or AWS Management Console to create a stack based on the template. You can specify parameters such as the stack name and the S3 bucket location of the template. Here's an example AWS CLI command:cppaws cloudformation create-stack --stack-name my-stack --template-url https://s3.amazonaws.com/my-bucket/my-template.yamlThis will create two S3 buckets named "my-bucket-1" and "my-bucket-2".You can modify the template to define additional storage accounts, or you can create multiple templates for different sets of storage accounts.CommentShareTasadduq Burneyanswered 3 months ago"


Question: How is RDS for SQL Server Express charged?

"I have configured RDS for SQL Server Express (which I hopefully will not be charged).  I understand that for free tier, I have 750 hours per month.  How is 750 hours per month counted?  If I have RDS available for 24 hours, will that be counted as 24 hours usage?  Or I have to login to my EC2 server and run an application which connect to RDS database.  If my application was connected for 2 hours and that will be counted as 2 hours out of 750 hours.  Is this the right way to count the RDS hours?FollowComment"

Answer: "0Accepted AnswerFrom the docsAWS Free Tier with Amazon RDS750 hours of Amazon RDS Single-AZ db.t2.micro Instance usage running MySQL, MariaDB, PostgreSQL, Oracle BYOL or SQL Server (running SQL Server Express Edition) – enough hours to run a DB Instance continuously each month20 GB of General Purpose (SSD) DB Storage**20 GB of backup storage for your automated database backups and any user-initiated DB Snapshots**** Backup storage is free up to 20 GB or the size of your DB storage, whichever is less, and requires you to have at least one DB instance running in your account. If you exceed these limits, or if you terminate all DB instances in your account, backup storage will be charged at standard RDS rates.For your case ensure you are running a single-az db.t2.micro with no more than 20GB of DB storage. To verify if you are falling within the free offering refer to the billing console and look at your bill. From here you will be able to see if you are being charged and if you are you can determine what you have launched outside the scope of the free offering.CommentShareRoBanswered a year ago"


Question: IAM Access Analyzer - alert on roles assumed by federated github actions

"I would like IAM Access Analyzer to alert me when a role is permitted to be assumed by GitHub. Backround: I'm using GitHub actions, which use OIDC to assume roles in my AWS account. IAM Access Analyzer does allow me to define archive rules using the OIDC principal, which is part of the solution. But I also want to check that the roles are limiting based on conditions, such as the repo name. At the moment, it appears only Facebook/Google/Amazon/Cognito is supported. This is what I'd like to check for:        "Effect": "Allow",        "Principal": {            "Federated": "arn:aws:iam::xxxx:oidc-provider/token.actions.githubusercontent.com"        },        "Action": "sts:AssumeRoleWithWebIdentity",        "Condition": {            "StringLike": {                "token.actions.githubusercontent.com:sub": "repo:mygithuborg/mygithubrepo:*"            }        }FollowComment"

Answer: "0Hi,Thank you for contacting us! I understand that you'd like IAM Access Analyzer to support archiving based on federation with GitHub, and for it to support filtering with condition keys for repositories and sub key. Please correct me if I have misunderstood your concern.While the service does not yet support those condition key filters [1], we have an option to create a rule with the 'principal.Federated' criterion (it's called 'Federated User' on the console) to filter by the IdP ARN (which is arn:aws:iam::xxxx:oidc-provider/token.actions.githubusercontent.com in your case). From your correspondence I understand that you are aware of this option.I thank you for your feedback on the service and I shall create a feature request to support your use case. While we do not have an ETA for now, please feel free to keep track of such feature releases on our What's New [2] page for when they arrive.1 https://docs.aws.amazon.com/IAM/latest/UserGuide/access-analyzer-reference-filter-keys.html2 https://aws.amazon.com/new/Let us know if you run into any further questions or concerns!CommentShareSUPPORT ENGINEERSumukhi_Panswered a year ago"


Question: Kinesis Java Application limits

"I see on the Kinesis Data Analytics for Java Applications Limits page that "You can create up to 50 Kinesis Data Analytics applications per Region in your account. You can create a case to request additional applications via the service limit increase form."I take it that this means that 50 applications is a soft limit.  If I need to request an increase, is there a hard limit on the number of applications that I can request?  Thanks.FollowComment"

Answer: "0There isn't a hard limit. If you ask us for 1M apps, we may try to understand why you need this limit, but there isn't any reason we can't approve it.CommentShareAWS-User-9868313answered 4 years ago"


Question: WatchGuard Dimension on AWS

"Hello all,I was wondering if anyone had detailed steps for recreating a WatchGuard Dimension server from an AWS Snapshot? I was tasked with removing the resources, and told we had nothing that should be on AWS. After I did so, well, we found out it was hosting our Dimension server. I have tried to recreate these instances with no luck. Any, and all help is welcomed.Resources I have used:https://www.watchguard.com/help/docs/help-center/en-US/Content/en-US/Fireware/firebox_cloud/deploy_aws.htmlA PDF "NetThreat_WatchGuard_Dimension_AWS_Setup_.pdf" found from a Google search from www.watchguardonline.co.ukFollowComment"

Answer: "0I can create the Instance, however it does not stay in a running state. I don't see any errors, just that when I start the Instance, it comes up, I hit connect and then it tells me it is not in a running state.CommentShareITDruidessanswered 3 years ago0I was able to contact WatchGaurd for help.CommentShareITDruidessanswered 3 years ago"


Question: Successfully detected the MX record for subdomain but NOT WORKING

"Hi,Received this email last thursday: Congratulations! Amazon SES has successfully detected the MX record required to use email.mydomain.com as a custom MAIL FROM domain for verified identity mydomain.com in AWS region US East (N. Virginia).But nearly a week later my emails are still being sent with the following:mailed-by:amazonses.comsigned-by:mydomain.comMy settings are to use amazonses.com if no MX record is found - so this means that Amazon hasn't found the MX record for email.mydomain.com.I spoke to my web host (Bluehost) and they told me that the MX record good on their end - they suggest it is on your end.Please help!ChrisFollowComment"

Answer: 0For anyone in the future who has this problem - it's because I had my domain (mydomain.com) AND my email address for that domain (chris@mydomain.com) validated. Amazon prioritised my email address instead of my domain. So I deleted my email address identity and that fixed the problem.CommentShareChrisMonkanswered 4 years ago


Question: AWS Amplify  image doesn't come out

"AWS Amplify BitBucket has been successfully applied.React is fine.but the image doesn't come outIn local, the image is normally output.However, it is not output in the Amplify domain.What settings should I check?FollowComment"


Question: CloudFormation - Aurora Fast Cloning

"Hi,I'd like to create a provisioned clone of my serverless Aurora MySql database using CloudFormation. The docs state that this should be possible, and it is when using the AWS CLI. However, when I try to accomplish this using CloudFormation, I always end up with a serverless clone instead of a provisioned clone. Here's my cluster definition:"Type": "AWS::RDS::DBCluster","Properties": {    "SourceDBClusterIdentifier": {        "Ref": "SourceDbClusterIdentifier"    },    "RestoreType": "copy-on-write",    "EngineMode": "provisioned",    "Engine": "aurora-mysql",    "UseLatestRestorableTime": true,    "DBClusterIdentifier": "WhyAmIServerless"}There's also a AWS::RDS::DBInstance attached to this cluster, but the stack errors out telling me I can't add an instance to a serverless cluster. Fair enough, but why is this cluster serverless in the first place? It seems like the EngineMode property is being ignored. Does anyone know if creating a provisioned clone from a serverless cluster is possible in CloudFormation? Thanks.FollowComment"

Answer: "1Accepted AnswerI think I may have found an answer. There indeed seems to be an issue with some properties being ignored when creating a clone. DBSubnetGroupName, EngineMode, and VpcSecurityGroupIds included.https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/336CommentShareScottieMc7answered a year ago"


Question: SES web version email

Do emails send through AWS offer a web version?I think this would be a powerful feature to offer if there is no such ability now.FollowComment

Answer: "0Accepted AnswerHello,There is not.You can integrate SES with other 3rd party tools available.Thank youRegards,Gaurav @ AWSCommentShareAWS-User-7751712answered 4 years ago0Hello,There are several ways that you can send an email by using Amazon SES. You can use the Amazon SES console, the Simple Mail Transfer Protocol (SMTP) interface, or you can call the Amazon SES API.Amazon SES console—This method is the quickest way to set up your system and send a couple of test emails, but once you are ready to start your email campaign, you will use the console primarily to monitor your sending activity. For example, you can quickly view the number of emails that you have sent and the number of bounces and complaints that you have received.SMTP Interface—There are two ways to access Amazon SES through the SMTP interface. The first way, which requires no coding, is to configure any SMTP-enabled software to send email through Amazon SES. For example, you can configure your existing email client or software program to connect to the Amazon SES SMTP endpoint instead of your current outbound email server.The second way is to use an SMTP-compatible programming language such as Java and access the Amazon SES SMTP interface by using the language's built-in SMTP functions and data types.Amazon SES API—You can call the Amazon SES Query API directly through HTTPS, or you can use the AWS Command Line Interface, the AWS Tools for Windows PowerShell, or an AWS SDK. The AWS SDKs wrap the low-level functionality of the Amazon SES API with higher-level data types and function calls that take care of the details for you. The AWS SDKs provide not only Amazon SES operations, but also basic AWS functionality such as request authentication, request retries, and error handling.Regards,Gaurav @ AWSCommentShareAWS-User-7751712answered 4 years ago0Hello vgauranaws and thanks for the reply.The reply however is not answering whether or not there is a web version of the email we send through AWS available.CommentShareKktheochanswered 4 years ago"


Question: WorkMail Client Setup in Mac iOS not working. Asking for Internal URL and External URL. Not sure what to give

WorkMail Client Setup in Mac iOS not working. Asking for Internal URL and External URL. Not sure what to give. Try to setup the WorkMail using Microsoft Exchange option.Followed all steps on this linkhttps://docs.aws.amazon.com/workmail/latest/userguide/connect_mac_mail.htmlInstead of accepting after password screen it takes to a new screen asking for Internal and External URL. And I am not able to find a URL that can be given there and not able to setup the client.FollowComment

Answer: "0Hi,I'm sorry to hear you're experiencing problems configuring your mobile device. The documentation page you're using is for the MacOS mail client. You can follow this guide to set-up mail on your iOS device.Kind regards,RobinCommentShareMODERATORrobinkawsanswered 8 months ago"


Question: runningsum function limitations

"Hi team, I just finished QS training and have a question regarding the runningsum function.I know there’s a limitation of using it on top of sumover , can runningsum not be used on any LAC-A functions?For example, the below calculation doesn’t give a subtotal value (while the distinct_count function works):Calc - Dist:  runningSum(distinct_count({uid[wh_monthly_facts]}),[{lifeline_months_tenure} ASC],[{Approval_Month}])https://docs.aws.amazon.com/quicksight/latest/user/level-aware-calculations.htmlFollowComment"

Answer: "0Hello,In this context, I would like to inform you that I had replicated the scenario with a sample data - 'SaaS-Sales.csv' from the below link[+] https://ee-assets-prod-us-east-1.s3.amazonaws.com/modules/337d5d05acc64a6fa37bcba6b921071c/v1/SaaS-Sales.csv I first loaded my sample data in QuickSight and created the below distinct count function. Please find the logic for the same below.Column name : My_Distinct_calculation -distinct_count({Order Date})I further tried using runningSum() function and created the below calculated field.Column name :My_Running_Sum -runningSum(distinct_count({Order Date}),[{City} ASC],[Country])Once I created the above two fields, assuming that you are using the "Pivot Table" as you have mentioned "subtotals" in your question, I went ahead and added all the columns in the field wells and further I could observe that the subtotals is shown for the column "My_Distinct_calculation" but not for the "My_Running_Sum" column. Please find the screenshot of my replication below.Therefore upon further analyzing I could not completely understand the usecase of calculating subtotals of the running sum of the column therefore it would be difficult to provide any workaround at this point of time if there are any.That being said, if you would like to further know about the above please raise a support case with AWS for further information along with the sample dataset to replicate. If a support case has already been created please be assured that we will get back to you and assist you in the best way possible.Reference:[+] https://docs.aws.amazon.com/quicksight/latest/user/supported-applications.htmlAttachment :CommentShareApoorva_Janswered 3 months ago"


Question: How to Deploy and Manage AWS Workspaces via AWS Systems Manager/SMM?

"Is there a way to deploy and manage AWS Workspaces through the AWS SMM?We're trying to setup AWS Systems Manager to be a simplified "central hub" in managing AWS Workspaces without having to go through AWS Workspaces. So it has to be setup in a way that the admin can log in, go to AWS SMM, then deploy and manage AWS Workspaces from there in just a few clicks.We've reached out to AWS tech support (Workspaces and SMM), with the following solutions:[1] https://aws.amazon.com/blogs/publicsector/automating-deployment-amazon-workspaces-active-directory-group/[2] https://aws.amazon.com/blogs/desktop-and-application-streaming/manage-amazon-workspaces-lifecycle-automatically-with-users-in-active-directory/But they require a lot of effort and also mastery of additional tools such as AWS Lambda, AWS SMM Maintenance Window, Directory Services, and CloudFormation.FollowComment"

Answer: "1Accepted AnswerAWS Systems Manager doesn't have specific integration with Amazon WorkSpaces, however you can perform functions on non-BYOL Amazon WorkSpaces through a Hybrid Activation. I want to be clear though, there will be some additional tools required no matter how you proceed forward with building a solution that meets your requirements. I suggest starting with Automating AWS Systems Manager activation for Amazon WorkSpaces and then getting a good understanding of how to automate basic activities with Run Command.CommentShareEXPERTStevieStetsanswered a year ago"


Question: Receiving emails error in SES

"I 'm using SES where I have validated my domain and email address, so I'm able to send emails without problems (sent from my domain). Also, no longer in Sandbox.However, I'm struggling to receive emails using SES. I followed all steps, and included the MX record in my DNS provider, but I'm not getting emails (I did setup a rule set which is active and contains SNS notification and S3/lambda code to forward the incoming emails).Checking for MX records using online tools, I get this error;smtpinbound-smtp.us-west-2.amazonses.comProblem getting IP Address for inbound-smtp.us-west-2.amazonses.comWhat could be the problem?FollowCommentAWS-User-SOS a year agoNeverminded, the URL was wrong. The correct one is:smtp inbound-smtp.us-west-2.amazonaws.com  (not ...amazonses.com )Now it works fine...(Note; confusion came as there is another URL which isinbound-smtp.us-west-2.amazonaws.com and in this case, it ends with ...amazonses.com )Share"

Answer: "0For future reference, in case others have a similar question, here is the documentation:Publishing an MX record for Amazon SES email receivingCommentShareJesse_Tanswered a year ago"


Question: ftp error 500 I won't open a connection to private ip but public ip

"I am trying to connect ftp from aws windows server to linux server with ftp in command terminal and I am able to login but unable to load any file there. The error is as below :--ftp> open x.x.x.xConnected to x.x.x.x220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------220-You are user number 2 of 50 allowed.220-Local time is now 07:39. Server port: 21.220-This is a private system - No anonymous login220-IPv6 connections are also welcome on this server.220 You will be disconnected after 15 minutes of inactivity.User (x.x.x.x:(none)):331 User mysqldesigners OK. Password required230 OK. Current restricted directory is /ftp> prompt nInteractive mode Off .ftp> quote PASV227 Entering Passive Mode (x,x,x,x,236,118)ftp> binary200 TYPE is now 8-bit binaryftp> cd sqlbackup/SqlServerBackups/daily250 OK. Current directory is /sqlbackup/SqlServerBackups/dailyftp> mput "c:/SQLServerBackups/SQLDB_BACKUP.7z"500 I won't open a connection to x.x.x.x private IP (only to x.x.x.x public IP)425 No data connectionI have disabled windows firewall and opened port 21 in aws security inbound and outbound rules but still there is same error.Although I am able to connect ftp location from windows explorer in aws windows server without any issue but in terminal there is error.Could some one help me to resolve this error at aws windows server.Thanks,SomeshFollowComment"

Answer: "0Hi Somesh,I understand that you are are getting an ftp error 500 that won't open a connection to private ip but only public ip.Please make sure that you have attached an Elastic IP to your Windows EC2 instance so that your instance can have a static ip address.In order to attach an Elastic Ip to your instance, please refer to the reference [1].To transfer files using a client, see reference [2].Verify that the application is running properly and is listening on the correct network port. If the application is stopped, start it. To accept traffic on a port, an EC2 Windows instance must host an application or service that listens on the specified port. From the EC2 Windows instance that is hosting the service, run the netstat command to display active connections and ports.References:[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html[2] https://docs.aws.amazon.com/transfer/latest/userguide/transfer-file.htmlI hope this helps.CommentShareAsenathianswered a year ago0Hi Asenathi,Thanks for your reply.I have already set elastic IP for the windows instance and using windows built in ftp client in windows command terminal. All necessary ports are open and windows firewall is off but still get the same error.netstat tells that   TCP    172.31.23.57:56316     server:ftp             ESTABLISHEDAlso I am able to make ftp connection from windows explorer but the issue is in terminal with ftp script. Only then I get error.C:\Users\Administrator>ftp 1.2.3.4Connected to 109.203.108.4.220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------220-You are user number 1 of 50 allowed.220-Local time is now 11:17. Server port: 21.220-This is a private system - No anonymous login220-IPv6 connections are also welcome on this server.220 You will be disconnected after 15 minutes of inactivity.User (1.2.3.4:(none)): abc331 User abc OK. Password requiredPassword:230 OK. Current restricted directory is /ftp> pwd257 "/" is your current locationftp> dir500 I won't open a connection to 172.31.23.57 (only to 3.4.5.6)425 No data connectionftp>Thanks,SomeshCommentSharerePost-User-5436366answered a year ago0It looks like you are using Active mode FTP connection. With Active mode, the server has to connect to the client for a data connection. This is not possible in cases where the client is behind a NAT or Firewall. In that case, the client will send its private IP but the server sees the NATed public IP of the client.  Try to use Passive mode connection where client will make both control ( port 21) and data connection to the server.  You need to configure the Pure-FTPd server to use a predefined passive port range so that you can open those ports in the EC2 Security group ingress.Following articles has a neat explanation of  Active vs Passive FTP.[1] https://www.jscape.com/blog/active-v-s-passive-ftp-simplified[2] https://docs.cpanel.net/knowledge-base/ftp/how-to-enable-ftp-passive-mode/If you are using the EC2 instance only as a FTP server, you may also try the AWS Transfer Family for FTP.  This is a managed service and you do not need to configure the FTP server or the instance by yourself. For more details, refer tof following documentation.https://docs.aws.amazon.com/transfer/latest/userguide/what-is-aws-transfer-family.htmlCommentShareRenjith_Ranswered a year ago"


Question: State Manager Association Non-Compliance vs. Failure

"I just created a few simple Systems Manager State Manager associations, all to accomplish small tasks like running inventory and scanning for patches. I had initially assumed that an association failing on a single instance would lead that instance to be marked as non-compliant, with whatever severity level was configured for the association. In one of the walkthroughs in the SSM documentation, however, I see this:If the State Manager association fails, no compliance data is reported. For example, if Systems Manager attempts to download a Chef cookbook from an S3 bucket that the node doesn't have permission to access, the association fails, and Systems Manager reports no compliance data.Is this the case for all association failures, regardless of document? And if it is, under what scenarios would association compliance show anything other than compliant instances?FollowComment"


Question: The plain HTTP request was sent to HTTPS port

I have Jenkins running on an EC2 with NGINX running on the same EC2 listening on port 80 forwarding to 8080 for Jenkins.  In front of this I have an ALB listening to port 443 and a CERT setup.When I go to https://jenkins.example.com.  I can login then I get 400 Bad Request "The plain HTTP request was sent to HTTPS port" and the url changes to http://jenkins.example.com:443/loginError.I tried adding in another listener on port 80 with a re-direct to 443.  That did nothing.  I even changed it to just print out a message but never got the message.Any idea where I might be missing something?FollowComment

Answer: "0One thing missing here is how the target group is configured on the ALB. From the information available it would appear that it's configured to send HTTP requests to the target group. You'll get the error message you're seeing if the listener is configured for HTTPS (regardless of what port it is using).Also: I'm not sure what else NGINX is configured to do here; but it's more efficient to have ALB send the requests directly to port 8080.CommentShareEXPERTBrettski-AWSanswered 4 months ago0Unless I am reading this wrong.  I have an ALB with one listener:HTTPS:43 forwarding to a Target Group.The Target Group has one Registered instance on port 80 and is healthy.I do not care if I use NGINX to forward port 80 on the instance to port 8080 on the same instance or if I have the Target Group forward directly to port 8080.  What I do know is that if I change the Server line in NGINX to let me use the instance IP I can get in to Jenkins on port 80 with no issues what so ever.  So the problem must be with the ALB or Target Group.CommentShareSystemgeekanswered 4 months ago0Since I did not mention this before here are my security groups.Instance SGInbound: HTTP source ALB SGInbound: HTTPS source ALB SGOutbound: All 0.0.0.0/0ALB SGInbound: HTTP source 0.0.0.0/0Inbound: HTTPS source 0.0.0.0/0Outbound: HTTP source Instance SGOutbound: All source 127.0.0.1/32CommentShareSystemgeekanswered 4 months ago0I enabled logs on the ALB and here is what I am seeing:h2 2023-02-10T19:31:25.203691Z app/JJJ-Dev-JenkinsMaster-ALB/73b30d15979cbc5c 10.0.0.4:62114 172.0.0.2:80 0.001 0.042 0.000 302 302 1112 229 "POST https://jenkins.tm-ci-prod.example.com:443/j_spring_security_check HTTP/2.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 OPR/94.0.0.0" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 arn:aws:elasticloadbalancing:us-east-1:111122223333:targetgroup/JJJ-Dev-JenkinsMaster-ALB-TG80/94ff9b642572376c "Root=1-63e69b8d-5ece4b07683f68816f0148de" "jenkins.tm-ci-prod.example.com" "arn:aws:acm:us-east-1:111122223333:certificate/1ff16fb3-eeeb-48f5-98f9-fda871fdb5fb" 0 2023-02-10T19:31:25.160000Z "forward" "-" "-" "172.0.0.2:80" "302" "-" "-"https 2023-02-10T19:31:25.375811Z app/JJJ-Dev-JenkinsMaster-ALB/73b30d15979cbc5c 10.0.0.4:62115 - -1 -1 -1 400 - 814 772 "GET http://jenkins.tm-ci-prod.example.com:443/loginError HTTP/1.1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 OPR/94.0.0.0" - - - "-" "-" "-" - 2023-02-10T19:31:25.316000Z "-" "-" "-" "-" "-" "-" "-"The last log message shows clearly how its breaking.CommentShareSystemgeekanswered 4 months ago0Hi,If you don't mind, I would like to recommend... Remove the NGINX, its just ALB (With a valid cert for the listeners), 2 listeners, 1st for HTTP traffic to redirect to HTTPS:443 and 2nd listener for HTTPS to a Target Group, the target group is your Jenkins instance at port 8080, and that one, only expose in the SG or allow the Jenkins port 8080.Make sure the ALB is internal tho, not recommended to expose Jenkins to world in any case.Cheers!CommentShareDavidCaballeroanswered 4 months ago"


Question: Trying to understand "Update Root User Email Address for AWS Account"

"I received an email from AWS entitled "Update Root User Email Address for AWS Account". I'm trying to understand what it's actually asking me to do. The complete body of the email is below.Is it saying that I'm using the same email address for AWS (the cloud platform) and Amazon.com (the store) and that I need to change the email address in AWS to something different?I find this message incredibly difficult to parse.You are receiving this message because we have identified that you are currently using the same email address for this AWS account (as listed in the Subject line) and for additional AWS account(s), which are associated with your Amazon.com account. We strongly recommend that you update the root user email address [1] for this AWS account as soon as possible to separate access to your additional AWS account(s) linked to your Amazon.com account. If you do not take any action by April 10, 2023, we will require you to update your email before accessing this AWS account when you sign in next to your account.After you have changed the root user email address for this account, you will be able to use it to access your account. At that point, we can finish separating your additional AWS account(s) from your Amazon.com account.The following are your additional AWS account(s) linked to your Amazon.com account:999999999999After you receive a confirmation email from no-reply@update.signin.aws, you can then sign in with the existing root email address for the additional AWS Account(s) and access new features. This can include enhancing the security of your sign-in experience with other Multi-Factor Authentication (MFA) device types, including hardware security keys [2], and monitoring root user activity through AWS CloudTrail [3].FollowCommentajx 2 months agoYes, you are right, the request is to use another email for your AWS account.The recommendation is to separate your amazon.com user from any AWS accounts. For eg: You may have an amazon.com user account for shopping, prime etc. But when it comes to AWS, it serves better to have a different email. This gives you an advantage from a security perspective i.e AWS accounts users do not automatically get access to amazon.com services.If you have multiple AWS account, there is also a concept of using an organization to bring everything together. https://aws.amazon.com/organizations/Share"

Answer: "1Accepted AnswerHi,You are correct, it just mention to separate the email you use to log in to AWS console, from the one you use in Amazon.con e-commerce site. Reason is that if one access is compromised, so it will be the other one, and remediation is to have different emails.To change root e-mail in aws follow this guide: https://aws.amazon.com/premiumsupport/knowledge-center/change-email-address/Hope it clarifies and if it does I d appreciate answer to be accepted so that community can benefit for clarity, thanks ;)CommentShareEXPERTalatechanswered 2 months agorePost-User-6683732 2 months agoThere's not really any utility to my having multiple email addresses.We really have to jump through the hoop of getting a gmail account to use AWS ... where I host my email?Identity should be separate from authentication. It's fine to reuse identity. This is very silly.Is there no way to leave it as-is?Share0How are we suppose to sperate them if the following applies?Important: If your Amazon.com retail and AWS accounts share the same log-in information, then updating the email address for one account also changes the other account.CommentSharerePost-User-2174710answered a month ago"


Question: Date Format in DataBrew transformation

Working with a CSV that has a date format on the raw file dd/mm/yyyy which is AU/UK format.We are applying a recipe to change the data format to yyyy-MM-dd via the format column transform so we can do some ordering via asc/desc.But if I look at a before and after from the two text files.Before = FOO 3822 08/03/2023 Bob (8th of March 2023)After= FOO 3822 2023-08-03 Bob (3rd of August 2023)This only seems to apply on string where the dd is between 1-12. So i'm getting a mixture in my output of some outputs being yyyy-MM-dd & yyyy-dd-MM with a single Date Function.is their a way to parse the string as a date standard before I format it?The dataset is a list of product price updates. There is multiple rows with the same product but different update date stamps. The purpose is to order them by latest date using the format yyyy-MM-dd and then remove duplicate rows for the same product leaving a dataset with only the latest product update which will always be the first row of the same product type.FollowComment


Question: Route 53 to EC2 connection not working

"So I've followed everything I can find online about how to connect my domain name to my server's IP address via route 53 and EC2.  From what I find my connection is set up correctly.  In route 53 I've gone to my hosted zones and created a new record set.  Type A using my elastic IP address for the value.  I am able to access my server through the browser by simply putting the elastic IP address in; however, the domain is not connecting to that elastic IP address and I don't know why.FollowComment"

Answer: "0Accepted AnswerDid you ever solve your problem?  I also had the exact same problem.  My solution was that I needed to change the DNS names for my domain in the Registered Domains section to match the DNS names that were created when I made the Hosted Zone.Instruction for this were found at the below link.https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-name-servers-glue-records.html#domain-name-servers-glue-records-adding-changingCommentShareFlyingTrashcananswered 3 years ago0Hi,If you post your domain name, I'll take a look.Note that the Route 53 Developer Guide also includes an extensive troubleshooting chapter:https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/troubleshooting-route-53.htmlScottCommentShareEXPERTAWS-User-6179575answered 3 years ago"


Question: Unable to complete SSL connection on a gamelift fleet with Linux OS

"Hello, I'm new to gamelift and I'm trying to implement it with my unity game, matchmaking seems to work fine with Windows build on Windows OS fleet, but when I make linux build and create linux OS fleet I get "Unable to complete SSL connection" while executing UnityWebRequest:ReservedPlayersCount: 4Connected: 1Config: [OK], connection: 1, checksum: 0GameLift.AcceptPlayerSession : [OK]Authorized: [OK], connection: 1, gid: d1e24a64-bb1c-43d0-af49-ba832e91e099, lid: 2Joined: [OK], connection: 1, gid: d1e24a64-bb1c-43d0-af49-ba832e91e099, lid: 2Gcs.player-profiles-operation: [FAIL], Unable to complete SSL connectionConnected: 2Config: [OK], connection: 2, checksum: 0GameLift.AcceptPlayerSession : [OK]Authorized: [OK], connection: 2, gid: dd858677-1a60-4951-a991-11a5156a7bfe, lid: 1Joined: [OK], connection: 2, gid: dd858677-1a60-4951-a991-11a5156a7bfe, lid: 1Gcs.player-profiles-operation: [FAIL], Unable to complete SSL connectionConnected: 3Config: [OK], connection: 3, checksum: 0GameLift.AcceptPlayerSession : [OK]Authorized: [OK], connection: 3, gid: b3c53b31-e0bc-4226-a0c0-61a9265b3d28, lid: 3Joined: [OK], connection: 3, gid: b3c53b31-e0bc-4226-a0c0-61a9265b3d28, lid: 3Gcs.player-profiles-operation: [FAIL], Unable to complete SSL connectionConnected: 4Config: [OK], connection: 4, checksum: 0GameLift.AcceptPlayerSession : [OK]Authorized: [OK], connection: 4, gid: 9d9f3670-0de2-4f5d-8e77-c3cc8a9597bc, lid: 4Joined: [OK], connection: 4, gid: 9d9f3670-0de2-4f5d-8e77-c3cc8a9597bc, lid: 4Gcs.player-profiles-operation: [FAIL], Unable to complete SSL connectionAnyone else encountered trouble with Linux fleets?FollowComment"

Answer: "0Just to be clears are you seeing these problems with the client -> server connections? Or a connection from the server to something else ie GameLift?Looks like the later which would indicate a problem, most likely, with root ca certs on the Linux machine and ensuring they are in the Unity/Mono trust store.You could attach a custom certificate handler to UnityWebRequest and manually validate the certificate data to see whats going on. You can also override certificate validation in the short term to unblock yourself.There some really good threads on debugging these types of issues on the Unity forumshttps://forum.unity.com/threads/unitywebrequest-does-not-work-on-linux-dedicated-server.630865/https://forum.unity.com/threads/unitywebrequest-unable-to-complete-ssl-connection.566380/I would suspect Unity forums are likely the fastest route to solving this because the way certs/root CA certs handled and all the networking stack/set-up is on the Unity side.CommentSharerePost-User-5332620answered 3 years ago0Thank you, first link helped me to solve the problem, by adding custom install.sh#!/bin/bashsudo chmod 755 install.shsudo mkdir -p /etc/ssl/certscd /etc/ssl/certssudo ln -s /etc/pki/tls/certs/ca-bundle.crt /etc/ssl/certs/ca-certificates.crtCommentSharerePost-User-0692473answered 3 years ago"


Question: Aurora Serverless v2 scales up ACUs very often for seemingly no reason

"Hi, it appears around 3 times each day, for 4-5 hours each time, Aurora (MySQL) rapidly scales up and down.This is despite incredibly light use of the database. I changed the ACU limit from 1.5 to 1.0 at 5pm yesterday, which is why the effect seems to reduce those final 2 times on the graph.Back when the ACU limit was 1.5, the logs indicated the buffer pool size was increasing, then 5 min later decreasing again. A pause for around 10 min, then repeat - for around 4 hours. Since lowering ACUs to 1.0, I no longer see logs of buffer pool resizing. So I think this was a symptom rather than a cause, but it may help someone determine what is happening here.The performance insights show no slow queries, no difference in network activity, fairly consistent CPU load, memory use only changes during the scale ups/downs, and show there is minimal load on the database overall.Does anyone know what's going on or how I can prevent it? I would prefer to increase my ACU limit for redundancy, but with this issue and many periods of low load, it would be a waste of money.You can see in this 1 hour graph the particular fingerprint of event is roughly identical each time.FollowComment"


Question: Can't start up an EC2 with F1 instance-type

"I've been wanting to explore FPGA development via the AWS FPGA Developer AMI on the marketplace.While following through a tutorial, there came a point where I needed the instance to have a /dev/xfpga device on it, which I believe exists only on the F1 instance-types.So I turned off my EC2 instance, and tried switching the instance-type to f1.2xlarge. It seemed to accept the change of instance-type fine at this point.But when I tried to right-click and 'Start Instance', I was met with an error message of:Failed to start the instance i-056d8b0407f711785The requested configuration is currently not supported. Please check the documentation for supported configurations.Googling online, a few possible reasons were suggested. The one I suspect is most likely is that perhaps the F1 instance-types aren't supported in my local region (Sydney / ap-southeast-2) yet.I do recall reading an article online that mentioned F1 instance-types only being available in US-regions:https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-ec2-f1-instances-are-now-available-in-aws-govcloud--us/But that article is from 2017, and I assumed that perhaps F1 instance-types would be globally available by now, and since nothing stopped me from selecting it while in Sydney region, it added to that impression.But anyways, thought I'd ask here, in-case anyone has any insights into this.For my part, I will try spin up an EC2 instance in a US-region and see if I have any luck there.FollowComment"

Answer: "2Accepted AnswerHello,The best way to confirm an Instance type is available at any given moment in any Availability Zone; is to check the type offering (1-2) in CloudShell or anywhere where AWS CLI is deployed. The script below will provide you with the Availability Zones under ap-southeast-2 with f1.4xlarge Instances available, you can tweak it up to another Instance Type and region:$ aws ec2 describe-instance-type-offerings --location-type "availability-zone" --filters "Name=location,Values=ap-southeast-2" --filters "Name=instance-type,Values=f1.4xlarge"**Important: ** Keep in mind that these values fluctuate depending on demand and how AWS keeps on expanding the offers.If your Instance is located in a different AZ, then the message output would be expected until the type offering is available in that particular Zone. I hope this helpsHave a great day!ReferencesFinding an instance type - Finding an instance type using the AWS CLI - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-discovery.html#instance-discovery-clidescribe-instance-type-offerings -  https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instance-type-offerings.htmlCommentShareSUPPORT ENGINEERJosueAWSanswered a year agoGurce a year agoThank you, via your aws-cli command, it showed that this instance-type was only available in 'ap-southeast-2b' (and not in 'ap-southeast-2a', where my current EC2 instance was located).After making a new EC2 instance inside 'ap-southeast-2b', I'm successfully able to use the 'f1.2xlarge' instance type, and when I log in, I do see the /dev/xfpga device.Share"


Question: EC2 instance doesn't have access to internet

"Hello everybody!I have a VPC with EC2 windows instance which is located in a private subnet. I have a NAT gateway with EIP which is located in a public subnet and an internet gateway which is attached to the vpc.  Also I have a virtual private gateway which is connecting my local network with the windows instance. I have one (main) route table with 3 routes (the local target, 0.0.0.0/0 pointing to NAT gateway and a route to my local network which is accessed via virtual private gateway). The vpn tunnel is working fine and I am able to connect to my windows instance via RDP over the VPN tunnel. The problem is that I don't have access to internet from that instance. I can make a route with the destination 0.0.0.0/0 pointing to the virtual private gateway but I don't want to route all the traffic via the VPN to avoid additional charges.   ACLs and SG allows all inbound and outbound traffic. Please help me to diagnose the problem.Edited by: Altaran on Jan 10, 2020 6:22 AMFollowComment"

Answer: "0figured this out. added another route table and add a route to 0.0.0.0 via IGW and it worked for meCommentShareAltarananswered 3 years ago0Hi Altaran,It is great to hear that you were able to solve the issue. Please do not hesitate to let us know if you need any further help.Regards,awstomasCommentShareAWS-User-3892169answered 3 years ago"


Question: Proxy settings on aws-sdk-ios

Hi. I'm trying to use proxy when my ios app accesses AWS Cognito and Lambda via aws-sdk-ios(but not amplify).But I couldn't find the way of implement that.I'd like to set proxy information like URLSession.Can anyone help?Thanks!FollowComment

Answer: "0For new projects, you should not use the AWS SDK for iOS anymore.We are focusing our efforts on a new AWS SDK for Swift (in preview).The AWS SDK for Swift implements a custom HTTP client that bypass macOS / iOS system wide proxy settings.  The solution is to provide your own HTTP Client implementation, just for the duration of your testing.Eventually, we would like the SDK to include such client by default.  You can track progress and copy/paste code sample here https://github.com/awslabs/aws-sdk-swift/issues/817CommentShareSebastien Stormacqanswered 4 months ago"


Question: Getting charged for non-existent Elastic IPs

"I have been charged for an "Elastic IP address not attached to a running instance per hour (prorated)",but cannot find any Elastic IPs in that region, how to proceed?FollowComment"

Answer: "0I would start with AWS Config. If this service has been turned on, the history of the Elastic IP will be available. AWS Config will show deleted resources.CommentShareEXPERTkentradanswered 6 months ago"


Question: Services vanished after enabling all features in Organization

"I lost access to my S3 buckets, Lambda functions, and CloudFront distributions after the Account Manager of the Organization I'm a member of "Enabled all features" in the organization.  Strangely he can't see them either in his account so it's not a matter of him adjusting the permissions on the services.  How can I regain access to what I've lost?FollowComment"

Answer: "0Hi There,I understand you lost access to your S3 buckets, Lambda functions, and CloudFront distributions after the Account Manager of the Organization you are a member of "Enabled all features" in the organization.The process of enabling "All Features" in Organizations by itself will not impact any existing resources in any of the member AWS accounts, however, depending on the SCP you have applied it will have the effect on the existing policies. By default, an SCP named “FullAWSAccess" is attached to every root, OU, and account, which allows all actions and all services. The users/roles are then restricted by the IAM policies attached directly to them.Therefore, until you start creating and attaching the SCPs to accounts, all of your existing IAM permissions continue to operate as normal [1].When enabling the “All Features” mode, you continue to get all the consolidated billing features plus a set of advanced feature such as SCPs, which give you fine-grained control over which services and actions that member accounts can access.SCPs enable you to restrict, at the account level of granularity, what services and actions the users, groups, and roles in those accounts can do. SCPs are similar to IAM permission policies and use almost the exact same syntax. However, an SCP never grants permissions. Instead, SCP acts as a "filter" that enables you to restrict what service and actions can be accessed by users and roles in the accounts that you attach the SCP to.I would highly recommend you to go through the AWS documentation on “before enabling all features” in [2] linked below to have a better understanding about enabling all features. Then, please see [3] linked below for instructions to being the process of enabling all features.Additionally, AWS SSO [4] and IAM are completely separate services, so enabling SSO in your account will not affect IAM users, roles, or policies that you're already managing in IAM.With AWS SSO, your users can log into the AWS resources using the identities in your existing identity source or identities you have created in the native AWS SSO Directory. Thus, AWS SSO does not use access key to access AWS resources.Please see [4] linked below for more detail about AWS SSO and [5] for instructions to get started with AWS SSO.Please feel free to contact us in case you have any other concerns. I will be happy to assist you further until everything is addressed.References:[1] Service control policies (SCPs)https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_about-scps.html[2] Enabling all features in your organization – Before enabling all featureshttps://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_org_support-all-features.html#before-enabling-all [3] Enabling all features in your organization – Beginning the process to enable all featureshttps://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_org_support-all-features.html#manage-begin-all-featuresPlease note that I personally value your feedback and would appreciate it if you accept and thumbs up this response if it satisfies your questionCommentShareMfaneloanswered a year ago"


Question: EKS Node Group with RI

"I'm looking for the information whether  we can use Reserved Instance in the EKS Managed Node Group. How should customer proceed with the setup, should they make reservation first before creating the node group?I saw during the Node Group creation we need to specify the Launch Template; and during creation of Node Template there is only 1 option to enable purchasing option which is "Request Spot Instances".Is there any guidance how to use Reserved Instance with EKS managed node group?FollowComment"

Answer: "0Accepted AnswerHi, EKS managed node groups use standard EC2 instances, so if there are RIs or Savings Plans that match the instance types / families, they will apply just as they would with regular EC2 instances.The "managed" part refers to the fact that EKS has a bit more of control over the instance, but other than that it's no different than a standalone EC2 instance.CommentShareAWS-User-9465269answered 3 years ago"


Question: "S3 BytesUploaded, BytesDownloaded metrics are not recorded in CloudWatch"

"Hi,I have a S3 bucket to which files are being uploaded by a Kafka sink connector. I am trying to setup a monitoring dashboard for this S3 bucket. However, I could only find BucketSizeBytes and NumberOfObjects metrics in CloudWatch but not request metrics like BytesUploaded, BytesDownloaded. Metrics seems to be enabled by default since BucketSizeBytes and NumberOfObjects metrics are being recorded already. Is there any extra configuration I need to do to get request metrics reported in CloudWatch.Thank you for your time.Thank you,SreeniFollowComment"

Answer: "0Hello!You need to opt in to request metrics in the console, via the REST API, or via the AWS CLI. To opt in to request metrics for S3 using the console, you need to do the following:Sign in to the AWS Management Console and open the Amazon S3 console.In the Buckets list, choose the name of the bucket that contains the objects you want request metrics for.Choose the Metrics tab.Under Bucket metrics, choose View additional charts.Choose the Request metrics tab.Choose Create filter.In the Filter name box, enter your filter name. NOTE: Names can only contain letters, numbers, periods, dashes, and underscores. We recommend using the name EntireBucket for a filter that applies to all objects.Under Filter scope, choose This filter applies to all objects in the bucket. (You can also define a filter so that the metrics are only collected and reported on a subset of objects in the bucket. For more information, see Creating a metrics configuration that filters by prefix, object tag, or access point.)Choose Save changes.On the Request metrics tab, under Filters, choose the filter that you just created.After about 15 minutes, CloudWatch begins tracking these request metrics. You can see them on the Request metrics tab. You can see graphs for the metrics on the Amazon S3 or CloudWatch console.For more information or alternative ways of opting in via the REST API or AWS CLI, see Creating a CloudWatch metrics configuration for all the objects in your bucket. It's also helpful to be aware of the CloudWatch metrics configurations and considerations.Hope this helps!CommentShareMVR-060709answered 2 months ago"