Question: "Lambda function as image, how to find your handler URI"

"Hello,I have followed all of the tutorials on how to build an AWS Lambda function as a container image.  I am also using the AWS SAM SDK as well.  What I don't understand is how do I figure out my end-point URL mapping from within my image to the Lambda function?For example in my docker image that I am using the AWS Python 3.9 image where I install some other packages and my python requirements and my handler is defined as:summarizer_function_lambda.postHandlerMy python file being copied into the image is the same name as above but without the .postHandlerMy AWS SAM Template has:AWSTemplateFormatVersion: "2010-09-09"Transform: AWS::Serverless-2016-10-31Description: AWS Lambda dist-bart-summarizer functionMore info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rstGlobals:Function:Timeout: 3Resources:DistBartSum:Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunctionProperties:FunctionName:  DistBartSumImageUri: <my-image-url>PackageType: ImageEvents:SummarizerFunction:Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#apiProperties:Path: /postHandlerMethod: POSTSo what is my actual URI path to do my POST call either locally or once deployed on Lambda??When I try and do a CURL command I get an"{"message": "Internal server error"}% "curl -XPOST "https://<my-aws-uri>/Prod/postHandler/" -d '{"content": "Test data.\r\n"}'So I guess my question is how do you "map" your handler definitions from within a container all the way to the end point URI?FollowComment"

Answer: "1The url you are creating looks correct. You can reference the API Gateway tutorial and see that they have a section showing how to build the URL:To test the deployed API using cURL:1. Open a terminal window.2. Copy the following cURL command and paste it into the terminal window, replacing *r275xc9bmd* with your API's API ID and *us-east-1* with the region where your API is deployed.curl -v -X POST \  'https://r275xc9bmd.execute-api.us-east-1.amazonaws.com/test/helloworld?name=John&city=Seattle' \  -H 'content-type: application/json' \  -H 'day: Thursday' \  -d '{ "time": "evening" }'Based on what you provided, it looks like API Gateway's Stage name is "Prod" so I would expect the following to work after replacing r275xc9bmd with your API's API ID and us-east-1 with the region:curl -v -X POST \  'https://r275xc9bmd.execute-api.us-east-1.amazonaws.com/Prod/postHandler' \  -H 'content-type: application/json' \  -d '{ "content": "Test data.\r\n" }'You might want to confirm API Gateway and Lambda is configured correctly and look at the CloudWatch logs to find the error.CommentSharematthew_danswered a year agoAWS-User-2124136 a year agoThis makes sense for a API gateway deployment, but what is the end point URI for a Lambda Function URI?  Meaning no gateway deployment.  So in my example I have the following Lambda Function URI:https://<my_unique_id>.lambda-url.us-east-2.on.aws/When I try and do a CURL to that URL I just get:Internal Server Error%Should it be:https://<my_unique_id>.lambda-url.us-east-2.on.aws/postHandlerIn my example above with my python code?ShareAWS-User-2124136 a year agoSp just to close this whole thing it, when it comes to a container Lambda function whatever you "lambda_function.handler" is called will work with your top level Function URL, i.e:'https://r275xc9bmd.execute-api.us-east-1.amazonaws.com/-H 'content-type: application/json' -H 'day: Thursday' -d '{ "content": "something" }'Share0When you create a Lambda function, regardless if using the zip file format or the container image format, the function should have a Lambda handler. The handler is the entry point to your function.This is where you write your business logic. If you are using Python, it would look like:def handler(event, context):    print (event)    return {        'statusCode': 200    }In your case it seems that you configured the handler to be summarizer_function_lambda.postHandler which means that the handler is in a file named summarizer_function_lambda and the method is named postHandler.If you want to invoke your function using a URL you have two options: 1. Create an API in API Gateway and set the integration to be the Lambda function. or 2. Create a Lambda function URL. Looking at your template you are using the first option which also creates an API. To find the URL for that API you can either include it in the Outputs section of the template, or go to the CloudFormation console, and check the resources that were created in your stack. API Gateway should be in there.CommentShareEXPERTUrianswered a year agoAWS-User-2124136 a year agoUri,This is great for the container and I understand that mapping.  However once it is created with AWS SAM with the 'sam deploy' and I have my API gateway and Lambda function created what is my final URL i can send a POST to??  That is what I don't understand what it would be.  For example my API endpoint is:https://<my-api>.us-east-2.amazonaws.com/ProdSo what do I do an example CURL post to?https://<my-api>.us-east-2.amazonaws.com/Prod?Do I need to invoke my actual handler name, in this case postHandler?  This is what I mean.  I need to understand my final REST API endpoint that I can do a GET or POST to.ShareUri EXPERTa year agoThe URL that is used is not related to the function at all. Lambda functions is just one option for an integration. As you are using SAM you define the trigger in the Events section (see below). In your case you defined that it will be triggered by an API in API Gateway and the path is /postHandler and the method is POST.Events:  SummarizerFunction:    Type: Api      Properties:        Path: /postHandler        Method: POSTSo in your case your function will be invoked by: POST to https://<api-endpoint>/<stage>/postHandler.You can export it from your template by adding the Output section like below:Outputs:  MyApi:    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/postHandler/"Share"


Question: IAM Identity Center - what does reprovision accounts mean

"The permissions on a role in a child account managed by IAM Identity Center in our management account has incorrect permissions.  We have a message in the Identity Center that says 2 AWS accounts are using an outdated version of this permission set.  When I click Update, I'm given another screen that has the text, To update this permission set in the AWS accounts that you selected, we are reprovisioning the accounts..  This is absolutely terrifying.  I want to update these policies, and I think this is what I want to do, but the wording has be very concerned.  I don't want to "reprovision" my account if "reprovision" means to recreate it somehow.  What does it mean by "reprovision"?FollowComment"

Answer: "1Accepted AnswerBackgroundThe warning "permission set uses outdated permissions" happens because either the managed policy or custom policy attached to the permission set might have deprecated permissions. Sometimes AWS needs to add a new permission to an existing policy, such as when a new service is introduced. Adding a new permission to an existing policy does not disrupt or remove any feature or ability. However, AWS might choose to create a new policy when the needed changes could impact customers, if they were applied to an existing policy.For example, removing permissions from an existing policy could break the permissions of any IAM entity or application that depended upon it, potentially disrupting a critical operation. Therefore, when such a change is required, AWS creates a completely new policy with the required changes and makes it available to customers. The old policy is then marked deprecated. A deprecated managed policy appears with a warning icon next to it in the Policies list in the IAM console. The same applies here to permission sets as well, since you can attach managed IAM policies to those permission sets.Now, this is where the re-provisioning of the AWS accounts comes into play. Re-provisioning is a process where if any changes are made to a permission set (deprecated policies etc) or to the account, then you will have to propagate those changes to the account/s by re-provisioning it. This is why you see the warning 'Requires reapplying permission set'.QueryComing to your query, in the context of the message received "To update this permission set in the AWS accounts that you selected, we are reprovisioning the accounts..", the action of reprovisioning the permission sets will only reprovision and update the policies associated to the AWS accounts. It will update the permission's policy and there is no recreation of account. It's just reprovisioning of the permission sets(along with updated policy) to the accounts, which are associated with the permission sets.Hope above shared information helps.Thank youCommentShareSUPPORT ENGINEERVarunanswered 7 months ago"


Question: Transparent TLS proxy for Keyspaces ?

"Hi,To ease the transition from Cassandra to Keyspaces, it would be useful to provide a transparent TLS proxy.Some drivers don't support TLS at all, and legacy applications that are not using TLS encryption could be migrated without any change.The  cql-proxy project from DataStax  is pretty close to what I'm requesting, sadly, it doesn't support proxying to Cassandra TLS.Are there plans for such a thing, and in the meantime, what is the Keyspaces team recommending ?ThanksFollowComment"


Question: CDK and Route 53 Failover

"We found a quite useful CDK article. It’s using a Lambda, Route 53 “A record”, and more. It’s titled “AWS CDK: Use Lambda with Application Load Balancer” https://sbstjn.com/blog/aws-cdk-lambda-loadbalancer-vpc-certificate/However, the article does not cover failover. The straight forward question would be “what changes to this article would be made so that Active-passive failover between regions is supported?“I understand a Route 53 “A record” has the "Failover routing policy" with which one can set up an active-passive failover setup. Hypothetically: if us-east-1 is down, it would automatically switch us to us-east-2.Items of note:Unless I missed it, the latest CDK ARecord (2.10.0) does not seem to support configuring the ARecord for failover. https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_route53.ARecord.htmlI see that the ability to set the routing policy for an ARecord was requested in 2019 https://github.com/aws/aws-cdk/issues/4391 which would cover a superset of what we need. The comments mention using CfnRecordSet. Is that currently the best way?Top level concepts from the article are: Lambda with handler code, LambdaTarget, ApplicationLoadBalancer, Certificate, and Route-53-A-Record (IPv4 DNS).Other related resources:https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.htmlhttps://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-configuring.htmlhttps://aws.amazon.com/blogs/networking-and-content-delivery/lambda-functions-as-targets-for-application-load-balancers/Any insight into how to implement failover using CDK, would be quite welcome. Thanks!Update Feb 5, 2022Still hoping for an optimal solution. For now, trying to wrestle with CfnRecordSet. CfnRecordSet properties of setIdentifier, **aliasTarget ** with evaluateTargetHealth (Evaluate Target Health), and failover seem to be key.Evaluate Target Health related docs that we are looking at:Route 53 RecordSet "Evaluate Target Health" via CloudFormation templatewhich points at https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-route53-recordset.html#cfn-route53-recordset-aliastargetand points at https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-route53-aliastarget.html#cfn-route53-aliastarget-evaluatetargethealthFollowComment"


Question: LOAD FAILED while loading RDF file to Amazon Neptune DB

"I'm new in Amazon Neptune service and trying to load a simple RDF file from my S3 bucket using Neptune Loader command  but it Failed with this error : ( "errorMessage" : "Invalid data type encountered for header rdf:RDF",)any help please ?FollowComment"

Answer: "0Accepted AnswerCan you confirm that your format is set to "rdfxml"? Please post your loader command JSON (with any sensitive data removed).https://docs.aws.amazon.com/neptune/latest/userguide/load-api-reference-load.htmlCommentShareBrad Bebeeanswered 4 years ago0Thanks bradbataws , I forgot to add the format of the fileCommentSharesabulwafaanswered 4 years ago"


Question: how can i beanstalk elastic env

"my env has this error "The instance profile aws-elasticbeanstalk-ec2-role associated with the environment does not exist."so I wanna delete this envbut delete button not activate, only restore button activatehow can i solve this?FollowComment"

Answer: "1I seem to be having the same problem as here.The error occurs when "aws-elasticbeanstalk-ec2-role" is missing.As described here, if you get this error, you can solve it by creating your own IAM role..https://repost.aws/questions/QUx_V0HSWxRJSRndVaSlsvOQ/elastic-bean-stalk-environment-is-not-getting-created#ANwB9EGj4QTHycEXVvji4eEgAlso, I believe Elastic Beanstalk environments are automatically deleted if they fail to be created.CommentShareEXPERTRiku_Kobayashianswered 9 days ago"


Question: How to Transfer an AWS Domain from a Suspended AWS Account

"Hi,I have an AWS Domain that is trapped (for years) in an inaccessible AWS Account -I.e. AWS says the account is permanently closed. Assuming that this domain is lawyer.com. How do I initiate the transfer of lawyer.com to my other accessible AWS console?Thanks.FollowComment"

Answer: "1Accepted AnswerPlease refer to the following AWS documentations:My AWS account is closed, suspended, or terminated, and my domain is registered with Route 53Contacting AWS Support about domain registration issuesCommentSharejoahnaanswered a month agoEXPERTalatechreviewed a month agorePost-User-8332758 a month agoThank you.  I will explore that solution...Share0AWS Support can also help you transfer the domain to another domain registrar.Once 90 days has passed from when you closed your account, you can no longer reopen it.Route 53 doesn't automatically renew your domain registration even if automatic renewal is turned on.If you closed your AWS account, or if the account is suspended or terminated, You need contact AWS Support and ask them for help with the following options:If you don't want to keep the domain registration, AWS Support can disable automatic renewal for the domain. This prevents you from getting several reminder emails about domain renewal.If you do want to keep the domain registration, AWS Support can help you transfer the domain registration to another AWS account.CommentShareNysten Macedoanswered a month ago"


Question: Healthlake Imaging in Preview stage

"Got access to Healthlake imaging preview after making a request to AWS (In the preview stage, the service is not available in console, so I am working on CLI.)--> Able to create, list and delete Healthlake-imaging datastores.--> But when trying to start an import job, the keyword 'health-imaging' is not recognized in CLI and cloudshell. PS: Healthlake imaging is a service available only in Preview stage in US-East-1 and I have done all configurations from the same region.It would be helpful if anyone could provide any input on it. Thanks in advance.FollowComment"


Question: Machine Learning image training on EC2 with GPU

"I am training deep learning models with ten thousand images on a G4 GPU instance, using local storage. Using parallel PyTorch dataloaders, just like I do with on-prem GPU hardware. On-prem, GPU utilization is typically a constant 99% during training and varies during validation steps. On EC2, training flips between 30/maybe up to 70% util and back to zero, for an average of maybe 30-40%. Please suggest how to get more GPU utilization in this scenario.FollowCommentMichael_F EXPERTa year agoJust to be clear, by "local storage," do you mean EC2 instance storage, or do you mean the root EBS volume for your instance? The two have very different performance characteristics.Share"

Answer: "0Hello,Thank you for posting your question! You may consider below steps to optimize the GPU setting to get the best performance from the GPU: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/optimize_gpu.htmlIn the above URL you can specify GPU clock speed to maximum frequency depending on instance type.CommentShareSUPPORT ENGINEERAWS-User-8128162answered a year ago"


Question: Savings Plan for Lambda

How to calculate the $ commitment while purchasing Savings Plan for Lambda?FollowComment

Answer: "0Accepted AnswerSavings plans will give a discount to AWS Lambda duration pricing, as accrued per hour (unused saving plans don't roll over to the next hour).  Savings can be found here: https://aws.amazon.com/savingsplans/pricing/ and range from 12% to 17% depending on payment options and commitment.Once you have a savings plan in place, the Lambda function duration costs will be charged at the discounted rate, up until the savings plan limit.  Note that Savings Plans apply across compute costs in the account, and will be applied to the services that offer the biggest discounts first, which means that if you have EC2 instances, they may consume the discount (since it will save you more money).Buying a Savings Plan purely for Lambda is really hard, since the discounts apply to spend in each hour (not the overall average), and Lambda functions tend to be bursty and unpredictable in nature.  So (in a contrived example), if you buy a $1 per/hour Savings Plan, and have a Lambda function that is idle for almost all of the month which then bursts to $730 of usage in a single hour, you will get a discount of 12-17% for ~$1 of the $730 spend in that hour, and no other discount that month (since there was no Lambda spend).In essence, if you're trying to buy a Savings Plan just for Lambda usage, you should probably only look to purchase one that covers the minimum spend that's incurred each and every hour - otherwise you will have wasted Savings Plans commitment for every hour that the Lambda function usage is below the spend.Example here: https://docs.aws.amazon.com/savingsplans/latest/userguide/sp-applying.html#scenario-1CommentShareEXPERTmhjworkanswered 3 years ago"


Question: Get error message "Unable to get your server endpoints" when try to add a gateway under AWS IoT>Wireless connectivity>Get started>Add gateway

"Hello,I'm logged in to the console with IAM user.I'm tring to add LoRa Gateway under  AWS IoT>Wireless connectivity>Get started>Add gateway (https://console.aws.amazon.com/iot/home?region=us-east-1#/wireless/onboarding/lorawan/gateway)I get an error red message on the top of the screen says:"Unable to get your server endpointsYour server endpoints couldn't be retrieved, Try again later"Can you please help?Thanks,EyalFollowComment"

Answer: "0You can get the "Could not connect to the endpoint URL" error if there's a typo or error in the specified Region or endpoint.  Be sure to the region is supported.For additional help you can reference the AWS IoT Core for LoRaWAN documentation here.There is also the AWS IoT Core for LoRaWAN workshop.  Here, you'll find information about instructions for connecting gateways that are qualified for use with AWS IoT Core for LoRaWAN.CommentShareKathy_Hanswered a year ago"


Question: "AWS access credentials can not be found" getting this error when trying to deploy the react app on amplify

"Hi Guys, I need urgent help on this issue. There isn't any accurate help available on the internet for this."AWS access credentials can not be found" getting this error when trying to deploy the react app on @amplify . Everything's working fine on localhost.Here's the error log-2023-05-20T09:21:17.184Z [INFO]: # Cloning repository: git@github.com:myrepos2023-05-20T09:21:19.101Z [INFO]: 2023-05-20T09:21:19.103Z [INFO]: Cloning into 'zeeepy'...2023-05-20T09:21:19.104Z [INFO]: # Switching to commit: bd45c7afa30c9e71643a0e0a2991d0fc8440c7aa2023-05-20T09:21:19.124Z [INFO]: Note: switching to 'bd45c7afa30c9e71643a0e0a2991d0fc8440c7aa'.                                 You are in 'detached HEAD' state. You can look around, make experimental                                 changes and commit them, and you can discard any commits you make in this                                 state without impacting any branches by switching back to a branch.                                 If you want to create a new branch to retain commits you create, you may                                 do so (now or later) by using -c with the switch command. Example:                                 git switch -c <new-branch-name>                                 Or undo this operation with:                                 git switch -                                 Turn off this advice by setting config variable advice.detachedHead to false                                 HEAD is now at bd45c7a Backend integrated2023-05-20T09:21:19.180Z [INFO]: Successfully cleaned up Git credentials2023-05-20T09:21:19.180Z [INFO]: # Checking for Git submodules at: /codebuild/output/src242373753/src/zeeepy/.gitmodules2023-05-20T09:21:19.189Z [INFO]: # Retrieving environment cache...2023-05-20T09:21:19.796Z [INFO]: # Retrieved environment cache2023-05-20T09:21:19.797Z [INFO]: ---- Setting Up SSM Secrets ----2023-05-20T09:21:19.797Z [INFO]: SSM params {"Path":"/amplify/d3ia74jrzqfroo/main/","WithDecryption":true}2023-05-20T09:21:19.836Z [WARNING]: !Failed to set up process.env.secrets2023-05-20T09:21:42.406Z [INFO]: # Patching NPM package '@aws-amplify/cli' from 10.5.2 to 12.0.0...2023-05-20T09:22:02.567Z [INFO]: # Done patching NPM package '@aws-amplify/cli' to version 12.0.02023-05-20T09:22:02.630Z [INFO]: # Retrieving cache...2023-05-20T09:22:05.748Z [INFO]: # Extracting cache...2023-05-20T09:22:10.438Z [INFO]: # Extraction completed2023-05-20T09:22:11.441Z [INFO]: # Retrieved cache2023-05-20T09:22:15.475Z [INFO]: ## Starting Backend Build                                 # Starting phase: build2023-05-20T09:22:17.637Z [INFO]: Note: It is recommended to run this command from the root of your app directory2023-05-20T09:22:17.652Z [INFO]: 🛑 AWS access credentials can not be found.2023-05-20T09:22:17.652Z [INFO]: Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/2023-05-20T09:22:17.696Z [INFO]: 2023-05-20T09:22:17.697Z [INFO]: Session Identifier: 5668b0c6-8dc5-4dcd-8b00-38754fa06b842023-05-20T09:22:17.715Z [ERROR]: !!! Build failed2023-05-20T09:22:17.716Z [ERROR]: !!! Non-Zero Exit Code detected2023-05-20T09:22:17.716Z [INFO]: # Starting environment caching...2023-05-20T09:22:17.716Z [INFO]: # Uploading environment cache artifact...2023-05-20T09:22:17.824Z [INFO]: # Uploaded environment cache artifact2023-05-20T09:22:17.824Z [INFO]: # Environment caching completedTerminating logging...Here's the link to stackoverflow with error logs - https://stackoverflow.com/questions/76296636/aws-access-credentials-can-not-be-found-error-when-deploying-on-amplifycould you please help me with this?FollowComment"

Answer: 1HiIt seems that your amplify configuration lacks the required IAM permissions.Follow this guide to create the required role for Amplify -> https://docs.aws.amazon.com/amplify/latest/userguide/how-to-service-role-amplify-console.html#step-3-return-to-the-amplify-consoleCommentShareBisinaanswered 11 days ago


Question: Specific user keeps getting access denied

"Hi,I'm hoping to get some help with troubleshooting this. I setup a transfer family SFTP server and was able to connect and transfer files via Cyberduck without any issues. I setup a user for my coworker and he also is able to connect successfully. However, when creating this third user, we keep getting "Listing directory / failed. Access denied." The same role and policy is applied to this user. I checked the log and it seems to connect but then immediately gets "Access denied." I asked this user to send me their key pair and I can connect perfectly  using their username but for whatever odd reason, they keep getting "Access denied" on their laptop.Here's the policy:{"Version": "2012-10-17","Statement": [{"Sid": "VisualEditor0","Effect": "Allow","Action": ["s3:ListBucket","s3:GetBucketLocation"],"Resource": "arn:aws:s3:::people-ops-pyn"},{"Sid": "VisualEditor1","Effect": "Allow","Action": "s3:ListAllMyBuckets","Resource": ""},{"Sid": "VisualEditor2","Effect": "Allow","Action": ["s3:PutObject","s3:GetObject","s3:DeleteObjectVersion","s3:DeleteObject","s3:GetObjectVersion"],"Resource": "arn:aws:s3:::people-ops-pyn/"}]}Here's the trust relationship policy:{"Version": "2012-10-17","Statement": [{"Sid": "","Effect": "Allow","Principal": {"Service": ["transfer.amazonaws.com","s3.amazonaws.com"]},"Action": "sts:AssumeRole"}]}Edited by: calfun on Jun 23, 2021 3:08 PMFollowComment"

Answer: "0Disregard! I overlooked the fact that the user had a specific policy scoped to them. Once I set it to none, it inherited the role policy and was able to connect.CommentSharecalfunanswered 2 years ago"


Question: AWS Glue gives error: "An error occurred while calling o285.save. File already exists:"

"I'm testing a relatively simple pyspark script that I first wrote (and tested) EMR. On the EMR script works as intended, but in Glue, the script starts writing output to desired S3 location and stops midway with this error:An error occurred while calling o285.save. File already exists:s3://bucket/prefix/part-xxxx.jsonSyntax I'm using to write DF:df \.write.format('json') \.option('header', 'false') \.save('s3://...')The prefix didn't exist on S3 before running the script. I'd appreciate any and all help on how to get this fixed.FollowCommentAbdel Dadouche a year agocan you post the full script?Share"


Question: How to upload a profile picture in re:Post?

"I have completed my profile but I cant find the input were I can upload my profile picture, this is an unreleased feature?FollowComment"

Answer: "18Accepted AnswerYes, this is an upcoming feature. We plan to support this in the coming months.CommentShareMODERATORSesi Bhushan_Sanswered 2 years agoEXPERTJDBreviewed a year ago1Hi Eduardo, we just launched the image support feature. Feel free to upload your profile picture now!CommentShareSUPPORT ENGINEERNguyen_Manswered a year ago"


Question: AWS services for enterprise

"I have a new technology company and we are looking to deploy AWS services in hosting our website and clients, company project management, HR management. What would be the best AWS products we can buy?FollowComment"

Answer: "0Hey @rePost-User-4858795 I would highly suggest that you look at https://aws.amazon.com/free/ to look at taking some AWS services for a free run. You don't buy the services, you pay for them as you go.Let me know if you have any issues with this, or if it helps you then please accept my answer after you've tried it out - it would be much appreciated! Good luck :)CommentShareD Ganswered a day ago"


Question: Analyze expense returns different line item fields header with the same document with different file type.

"Hi, I'm using analyze expense to extract details from my invoice. I have an invoice document in a pdf and also in png format with the same content. My problem is that when I scan both of them, They return different Line Item Fields. I've attached screen shots for comparison. It seems that it can't detect line item fields in pdf format.Invoice PNG format:Invoice PDF format:Here's the the google drive link for the actual documenthttps://drive.google.com/drive/folders/1UC6TZbFIWb8-IEsTd2CCN6p-N3IaBfcB?usp=sharingFollowComment"

Answer: "0Hi,Thanks for using Textract service. Sorry that you are facing inconsistent behavior for different document format though in usual case they should have same outputs. Would you please share the documents so we help look into the issue.Thank you,ShayneCommentShareAWS-Shayneanswered 3 months agojeff 3 months agoHi @AWS-Shayne, I updated the question and included the link for both files.Share0Hello,Thanks again for using Amazon Textract. I am able to reproduce the issue with the documents provided, however I do notice that the pdf you shared is cropped strangely within the pdf. When converting the png image to a pdf myself, the results from AnalyzeExpense are identical between the two. Is seems likely that this cropping is changing the resolution enough to affect the results.Hope this makes sense. Let us know if you have any other questions.CommentShareAWS-Killiananswered 3 months ago"


Question: Possible quota issue

"I opened up an AWS support case to increase my quota based on the error below, but AWS support states this is not a quota issue.  They were unwilling to help me in support but directed me to this forum.I'm getting the following error when running some AWS SageMaker batch transform jobs on m5.large instances.  I'm running 8 transforms in parallel against 8 different models in a step function map.  A few of the transforms succeed, but some fail with the following error:{"resourceType": "sagemaker","resource": "createTransformJob.sync","error": "SageMaker.AmazonSageMakerException","cause": "Rate exceeded (Service: AmazonSageMaker; Status Code: 400; Error Code: ThrottlingException; Request ID: 49a80dc1-df06-4b88-a462-24e517d13531; Proxy: null)"}What is going on here?  Am I doing something wrong?  What kind of throttling exception is this (what is being throttled?)FollowComment"

Answer: "0I have seen this with batch transform when I try to trigger too many jobs too quickly (usually like 4xjobs within 1-2 seconds).What I did as a workaround was handle and recognize this quota error on my service, sleep for 5 seconds and retry the same job.CommentShareGrant-Crossfoldanswered a year ago"


Question: What Username do AWS VPN Client need when using password-encrypted private key certificate?

"We have a Client VPN Endpoint and want to use password encrypted private key certificates.When using Windows "aws vpn client", a pop-up asks username and password. What username should we specify? The CN of the certificate? Anything else?The password is the password we specified when creating client certificate with EasyRSAThanks.FollowComment"

Answer: "1I am using the aws vpn client for macos. I had the same client connection behavior, a popup asking for username/password for an endpoint configured for mutual tls. I used the same ACM-created server cert as the client cert, as it was a POC. (I would think an ACM-created client cert will work just fine.) The root cause was that I had not decrypted the private key with the passphrase. After downloading from the ACM, run this: openssl rsa -in private_key.txt -out decrypted_private_key.txt. Then cat the decrypted file and put it's contents in the client config file <key> </key> block.CommentSharemark-sawers-webomatesanswered 6 months ago1I have the same behavior.What worked for me was to decrypt de private key with : openssl rsa -in private_key.txt -out decrypted_private_key.txtand put the content of decrypted_private_key.txt into <key></key> of the vpn client config file.CommentSharegabydinuanswered 4 months ago0Hello thereClient VPN offers the following types of client authentication:-Active Directory authentication (user-based)-Mutual authentication (certificate-based)-Single sign-on (SAML-based federated authentication) (user-based)         You will only be prompted for username/password if you are using user based authentication methods (AD or SAML)With Active Directory authentication, clients are authenticated against existing Active Directory groups ,so clients have to enter credentials for existing AD groups.For SAML, the credentials are stored by their IdPs.I would like to mention that using ACM and ACM Private CA for mutual authentication isn't currently supported for use with the AWS provided client for the Client VPN application[2].However you can connect using any other OpenVPN-based client application.Reference:[1]https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/client-authentication.html[2]https://aws.amazon.com/premiumsupport/knowledge-center/client-vpn-authenticate-private-ca-acm/CommentShareNosiphiwoanswered a year ago0The above solutions work!  Thanks to those who provided the fixes above.However, they require distributing unencrypted key material in the configuration file, making life harder for admins of small projects that don't have AD or SAML built yet.  Tunnelblick has no problem supporting encrypted secrets, why not the AWS VPN Client?CommentShareottomonanswered 4 months ago"


Question: Connect Lightsail instance to AWS instance in a private subnet

"Hi - I am trying to access Windows SQL server running in a private subnet in AWS from another Windows instance in Lightsail.  There are multiple subnets in the same VPC, some are private and others are public.  How can (only) allow Lightsail instance to access DBs in SQL server?  ThanksFollowComment"

Answer: "0Hi.Lightsail has a VPC peering feature.However, you can only peer with the default VPC.https://lightsail.aws.amazon.com/ls/docs/en_us/articles/lightsail-how-to-set-up-vpc-peering-with-aws-resourcesIf you have an RDS resource other than the default VPC, you can place the NLB in front of the RDS and use the endpoint service to establish a connection route from the default VPC to the RDS with an IP address target.https://dev.classmethod.jp/articles/nlb-bastion-for-rds/I have successfully connected to Lightsail-> Endpoint on Default VPC-> NLB-> RDS with this configuration.CommentShareEXPERTiwasaanswered a year agoAWS-User-0181798 a year agoThanks for your response.  Will this configuration also work with connecting to MSSQL DB?Shareiwasa EXPERTa year agoYes. You can also connect with MSSQL.Only the port number is different.Shareiwasa EXPERTa year agoI tried your scenario and put it together in an article. I verified it with MySQL, but it should work with MSSQL as well.https://dev.classmethod.jp/articles/lightsail-connect-vpc-rds/Share"


Question: EC2 load balanced service stuck "resource creation initiated"

"***ORIGINAL TITLE: CloudFormation reports "Content of DNS Record is: null" when deploying ApplicationLoadBalancedEc2ServiceI have been using CDK for the last few months deploying ApplicationLoadBalancedFargateService. No problems there. I have been able to enable https on the ALB using the posted certificate codeNow I need to deploy an ApplicationLoadBalancedEc2Service. I added capacity to the Cluster construct and left everything the same. In the hope that the ALB would pick up the cert logic. The deployment has been stuck in the service resource for quite some time. My impression is that is related to the reported message. The domain is in Route53. Any thoughts?`String subDomain = String.format("%s.%s", id, Constants.getZoneId());        IHostedZone hz = HostedZone.fromHostedZoneAttributes(this,                 String.format("XXXX-%s", id),                 HostedZoneAttributes.builder()                        .zoneName(Constants.getZoneId())                        .hostedZoneId(Constants.getHostZoneId())                        .build());        Certificate cert = Certificate.Builder.create(this, String.format("YYYY-%s", id))                .domainName(subDomain)                .validation(CertificateValidation.fromDns(hz))                .build();`EDIT 1:It ended up being rolled back. I made some changes: removed all the https, certs, and cpu configuration from the service. It currently stands like this:cluster.addCapacity(String.format("XXXXXX-%s", id)        , AddCapacityOptions.builder()        .desiredCapacity(Constants.getMinInstanceCount())        .instanceType(new InstanceType("t2.medium"))        .vpcSubnets(SubnetSelection.builder().subnets(vpc.getPrivateSubnets()).build())        .build()); ApplicationLoadBalancedEc2Service.Builder.create(this, String.format("ZZEERRRR-%s", id))               .healthCheckGracePeriod(Duration.minutes(10))               .loadBalancer(alb)               .protocol(ApplicationProtocol.HTTP)               .protocolVersion(ApplicationProtocolVersion.HTTP1)               .memoryReservationMiB(Constants.getMaxMemory())               .targetProtocol(ApplicationProtocol.HTTP)               .cluster(cluster)               .taskImageOptions(ApplicationLoadBalancedTaskImageOptions.builder()                       .image(ContainerImage.fromEcrRepository(Repository.fromRepositoryName(this, "QQQQQQ", "XXXX"), Constants.getDockerImage()))                       .environment(getEnv(id))                       .containerPort(Constants.getPort())                   .logDriver(LogDriver.awsLogs(AwsLogDriverProps.builder().logGroup(LogGroup.fromLogGroupArn(this, "cw", "arn:aws:logs:YYYYY:*")).streamPrefix("ZZZ").build()))                       .taskRole(Role.fromRoleArn(this, "role", Constants.getTaskRoleArn()))                       .build())               .build();But it still is stuck. Any pointers on what I might be missing?FollowCommentchubbsondubs 2 months agoI'm having the same issues.  Actually my Ec2 instance isn't registered with the cluster which I think is leading to the cluster unable to run the task and CDK is waiting until everything goes healthy to finish.  But I think that might be the same issue as you.  Did you ever resolve the issue?Share"


Question: Usage of alphanumeric sender ID for SMS messaging in Russia

Is it possible to use an already registered alphanumeric sender ID for SMS messaging in Russia with second provider?I.e. ID is registered through provider A and now we would like to use this sender ID also with AWS SNS. Do we have to register it again?FollowComment

Answer: "0Hello,If you plan to send messages to a country, for example Russia, where pre-registration of sender ID is required as per AWS documentation at this link[1], then you have to register the sender ID with AWS and follow the steps at this next link [2] in order to use the same sender ID again which has already been registered with provider A.The reason is that AWS needs to register the sender ID further once more.[1] https://docs.aws.amazon.com/pinpoint/latest/userguide/channels-sms-countries.html#sms-support-note-1[2] https://docs.aws.amazon.com/sns/latest/dg/channels-sms-awssupport-sender-id.html#:~:text=Step%201%3A%20Open-,an,-Amazon%20SNS%20SMSCommentShareSUPPORT ENGINEERTim_Panswered a year ago"


Question: Domain Transfer Failed status not getting updated

"Hi Team,I have initiated a domain transfer yesterday. When I tried to make payment 1st mode got failed, but 2nd mode of payment was successful.Now even after 12 hours status is unchanged, and domain is not getting displayed in the list.Please help.FollowComment"

Answer: "0Hello,I'm so sorry to hear about this, it's not the experience we strive for.I recommend reaching out to our Support team so that they can look into this for you by creating support case here: http://go.aws/support-center.Also, you can check out this article to see how you can request a call  for assistance through our Support Center: https://aws.amazon.com/premiumsupport/knowledge-center/aws-phone-support/.Best regards,— Aimee K.CommentShareEXPERTAWS Support - Aimeeanswered 2 months ago"


Question: elbv2 api for rules - need example for http-header condition type

"Hi,I am looking for a documented example for creating listener rule with the condition type of "http-header". The API for the conditions element only allows attributes for "Field" and "Values". However, the "http-header" condition type also requires providing the "HttpHeaderName".  For eg: "cookie". Trying to specify that in various ways but getting errors.Is there an example of specifying this via the CLI?Appreciate the help.ThanksFollowComment"

Answer: "0this is a relateively new feature..upgrade your aws cli to at least:aws-cli/1.16.144then e.g:aws elbv2 create-rule --listener-arn <listener arn> --priority 90 --conditions '{"Field":"http-header","HttpHeaderConfig":{"HttpHeaderName":"Cache-control","Values":["no-cache"]}} ' --actions Type=forward,TargetGroupArn=<targetarn>CommentSharecloudbuttonanswered 4 years ago0Thanks. Upgrading the cli and using the example below works.CommentShareshardanswered 4 years ago"


Question: BodyM Dataset License Type Clarification

"BodyM Dataset, Provided by: Amazon, part of the Open Data (https://aws.amazon.com/marketplace/pp/prodview-w3762gcflmhzk#overview) has entry regarding license:"Creative Commons Attribution-Non Commercial 4.0 International Public License - https://creativecommons.org/licenses/by/4.0/legalcode "Wording states that it is for non commercial usage, however linked license is 'Creative Commons Attribution Attribution 4.0 International' which is less restrictive. Which one is valid?FollowComment"


Question: New VPC Subnets and Transit Gateway attachment

"Customer has an existing VPC with 2 subnets which are attached to a Transit Gateway. The subnet is running out of available IP space. So they added a new CIDR range to the VPC and added two subnets (Subnet 3 and 4). Subnet 1 and Subnet 3 share the same AZ while Subnet 4 is in a new AZ.When attaching the new subnets to the Transit Gateway, you can select Subnet4 to be attached. Since Subnet 1 and Subnet 3 are in the same AZ, the Transit Gateway attachment option allows you to select only subnet 1 or subnet 3.The question is whether Transit Gateway will know a route/path to subnet 3 if the transit gateway attachment is not explicitly made. In another word, would you need to detach subnet 1 from the Transit Gateway attachment and attach subnet 3?FollowComment"

Answer: "0Accepted AnswerYou can only have 1 attachment per AZ. In fact, you need to have 1 attachment per AZ that you have created subnets in. The next step is to make sure the route table associated to each subnet has the required routes pointing to the Transit Gateway. These will be static routes.On the TGW routing side, if you have route propagation enable with TGW route tables (or default route table) the entire added CIDR will be advertised automatically. if you do not have route propagation in the TGW, you can add the entire CIDR or what every subnets you want to advertise statically in the TGW.CommentShareAWS-User-0547701answered 3 years ago"


Question: Direct Connect Public VIF "Prefixes you want to advertise" clarification

"while configuring AWS Direct connect public VIF, if you add your own public CIDR of /23 (just an example: 4.4.0.0/23) to the "Prefixes you want to advertise", but advertise a /24 (4.4.0.0/24) through the BGP, what would AWS advertise within the region? will it be the longer /23 prefix added or the smaller /24 advertised through BGP? i see a doc on how prefixes work with VPN Gateway and TGW, but not on the public VIF. https://docs.aws.amazon.com/directconnect/latest/UserGuide/allowed-to-prefixes.htmlFollowComment"

Answer: "0Hello,Thank you for your question.AWS will advertise /24 only. Bcoz that is what was advertised to AWS.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 23 days ago"


Question: AWS Sending rate is lower than visible in AWS

"Hello AWS,we actually have a daily sending quota of 1 million emails per day (what is enough for now) and a maximum sending rate of 400 emails per second granted by AWS.But when we are executing email campaigns the send out of the emails is happening much slower than 400 emails/sec, it is less than 50 emails/sec and we do not see the reason for that.Does anybody has experience with this? How can we increase the **real **sending rate of our AWS account?Many thanks in advance! Best regards, StefanFollowComment"

Answer: "0In the meantime I found to approaches by myself:AWS says we should follow this documentation: https://aws.amazon.com/blogs/messaging-and-targeting/how-to-handle-a-throttling-maximum-sending-rate-exceeded-error/SAP says:How the System Reacts on Amazon's ThrottlingWhen Amazon runs into throttling, Amazon returns an error message. The SAP system reacts on it by reducing the send rate and processing the failed messages.When Amazon SES runs into throttling, it returns an error message with the following text: API error: Code "400", Reason: "Bad Request", Message: "Throttling Maximum sending rate exceeded."This happens if the maximum send rate is exceeded.The campaign execution reacts on it by reducing the number of parallel sent request to Amazon SES. Messages that failed due to the Amazon SES error are automatically reprocessed by the campaign execution.As result of throttling the campaign execution needs more time to process all requests because it sends less in parallel to avoid that the maximum send rate is exceeded again.**Recommendation**If the issue still persists, we recommend to exceed the quota at Amazon.So - no final solution yet, but hopefully one of both will act! I keep you up to date here.CommentShareAWS-User-6365177answered a year ago"


Question: AppFlow permissions error when calling CreateConnectorProfile API

"I'm calling CreateConnectorProfile from a Lambda, and I get the following error :An error occurred (ValidationException) when calling the CreateConnectorProfile operation: CreateConnectorProfile failed as AppFlow is unable to create secret in customer account {MyAWSAccountId}. Ensure the IAM user has permissions secretsmanager:CreateSecret and secretsManager:PutResourcePolicy. If customer managed KMS key was provided, ensure AppFlow also has access to the key. Please give the required permissions and try to create the connector profile again.My Lambda's execution role has the Managed Policy AmazonAppFlowFullAccess attached. The customer managed KMS Key (which is passed in as the kmsArn to the CreateConnectorProfile call) has the following policy :{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Principal": {                "Service": "appflow.amazonaws.com"            },            "Action": [                "kms:Encrypt",                "kms:GenerateDataKey",                "kms:Decrpyt"            ],            "Resource": "*"        },        {            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::{MyAwsAccountId}:root"            },            "Action": "kms:*",            "Resource": "*"        }    ]}What permissions am I missing?FollowCommentDane86 5 months agoI have the same issue but then through cloudformation, did you manage to get it working?Share"


Question: ETL automation and testing

Hey I am trying to learn what others and what the best practices are with glue for development automation and testing/validation.FollowComment

Answer: "0Hi there,ETL Job Automation Best PracticesSet up complex workflows with AWS Glue Workflow (multiple crawlers, jobs, and triggers)https://docs.aws.amazon.com/glue/latest/dg/workflows_overview.htmlStart a workflow with Amazon EventBridge event to automate your AWS services and respond automatically to system events such as application availability issues or resource changes.https://docs.aws.amazon.com/glue/latest/dg/starting-workflow-eventbridge.htmlUse job bookmarks to track data that has been processed by a previous run of an ETL job.https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.htmlDeveloping & testing AWS Job Scripts Best PracticesIf you prefer local/remote development experience, the Docker image is a good choice. This helps you to develop and test Glue job script anywhere you prefer without incurring AWS Glue cost.https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-libraries.htmlIf you prefer local development without Docker, installing the AWS Glue ETL library directory locally is a good choice.https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-libraries.htmlUnit test and deploy AWS Glue jobs using AWS CodePipeline.https://aws.amazon.com/blogs/devops/how-to-unit-test-and-deploy-aws-glue-jobs-using-aws-codepipeline/Customer Case Study"By using AWS Glue, ShopFully can automate the ETL services that had taken up time on its previous solution and create individual clusters for each campaign, reducing latency and efficiently handling petabytes of data". Read more here:https://aws.amazon.com/solutions/case-studies/shopfully-case-study/CommentShareCHELSEA IPanswered 10 months agoSUPPORT ENGINEERMonica_Lluisreviewed 10 months ago"


Question: "CloudFront S3 Endpoint Works, but Not as Website Endpoint"

"I am setting up an S3 bucket to host up static files through CloudFront (which I have done multiple times). I noticed when I set up the "Origin domain" as the S3 bucket I had configured, I received a message that said:This S3 bucket has static web hosting enabled. If you plan to use this distribution as a website, we recommend using the S3 website endpoint rather than the bucket endpoint.There was a button to "Use website endpoint."Given that this seemed to be the right advice, I clicked "Use website endpoint" and went on with configuration. However, I could not access the actual site.After working on it a bit, I went back to this setting and disabled the use of the website endpoint. This changed the Origin access settings (see screenshot below). I was able to select "Origin access control settings (recommended)" option and everything worked perfectly immediately.My confusion - why does the "Use website endpoint" not offer this option when its selected. It seems like it is needed. I have not seen any other information on anything else I need to configure if I choose this option.FollowComment"

Answer: "0Accepted AnswerMy confusion - why does the "Use website endpoint" not offer this option when its selected. It seems like it is needed.S3 buckets can use Origin Access Control (OAC) or Origin Access Identity (OAI) to authenticate CloudFront distribution which requests to the bucket.If you choose the S3 Website endpoint, CloudFront will recognize the origin as a Web Site, not a bucket. Therefore, OAC and OAI are not available.CommentSharejhashimotoanswered 4 months agoAWS-User-6328292 4 months ago@jhasimoto Okay, that makes sense to me. But, I still get the 403 no matter what. If I'm setting CloudFront as a static website, does the bucket need to be Public then?Sharejhashimoto 4 months ago@AWS-User-6328292 That's right.Sharejhashimoto 4 months ago@AWS-User-6328292for your reference: https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-serve-static-website/?nc1=h_lsShare"


Question: "Use CDK or CloudFormation -> what justifications? Currently, the CDK is still used, whether with Java or another language?"

"Use CDK or CloudFormation -> what justifications?Currently, the CDK is still used, whether with Java or another language?I quote: "IT teams have two AWS-native options for infrastructure-as-code: AWS CloudFormation and AWS Cloud Development Kit (CDK). CloudFormation templates were AWS's first foray into cloud-based infrastructure-as-code, and while are useful, CloudFormation has clear weaknesses, more specifically, it lacks built-in logic features and has a steep learning curve.The AWS CDK, an open source software development framework for defining cloud infrastructure, addresses these weaknesses. The AWS CDK supports popular programming languages that developers can use to build, automate, and manage infrastructure based on an imperative approach. Finally, developers can provision these commands through CloudFormation."FollowComment"

Answer: "0Please view my response to the question https://repost.aws/questions/QUBBSLNpleSU-G9w9b-sP96Q#AN8pnTPfd3RoOmm4Ne6nIQggCommentShareEXPERTIndranil Banerjee AWSanswered 7 months ago0Same as Indranil here, also that had my answer that you seem to have commented.I am just going to add to it responding to your comment in your question: CFN might feel like there is a steep learning curve if you never used it. I have been using it since it was only JSON, which made it both great and painful (specifically, "where is that missing, ?" sort of issue). But, it's really not, just like everything, it takes time and effort to get to know a language/service very well. Is it perfect? "God no", but I have old templates that still work today that I haven't had to change in years. CDK in these years already did change version, however.Now CDK being something you can code with makes the adoption more dev friendly but also I have seen many examples where CDK users end up creating single-use templates, simply because they hardcode values which should be parameters. Maybe someone else using CDK too might be able to run the script again and get new templates out of it, but then that's only valid for them now.Also one of my favorite things with CFN native templates is that you can host them in S3 publicly, and create links for people to "just deploy". Very useful to share and reuse. Also, for an IT department, that makes it easy to create items in Service Catalog.Again, as said in my previous post, CDK or native CFN templates is up to the person writing these. But either way, they should always think about reusability first, expandability, and then go bespoke.CommentShareJohnPrestonanswered 7 months ago"


Question: EBS volume I/O error

"I've created a new EBS volume, formatted as ext4 and mounted to a Ubuntu instance.No significant read/write volumes, about 1% of the volume used, nothing special at all.After 1 week, it has reported an I/O error and was made read-only.The following details reported in the syslog:end_request: I/O error, dev xvdf, sector 0Buffer I/O error on device xvdf, logical block 0lost page write due to I/O error on xvdfEXT4-fs error (device xvdf): ext4_journal_start_sb:327: Detected aborted journalEXT4-fs (xvdf): Remounting filesystem read-onlyEXT4-fs (xvdf): previous I/O error to superblock detectedThis is a second time this happens over last 2 weeks. The above volume has replaced an older volume with the same issue. The server has been in service for a few years with no such issues in the past. What might go wrong? What could I try?FollowComment"

Answer: "1Accepted AnswerThe ext4 file system can be configured to be remounted as read-only after encountering an error. For reference, see ext4's manual: https://man7.org/linux/man-pages/man5/ext4.5.htmlThe relevant part of this documentation is this:errors={continue|remount-ro|panic}              Define the behavior when an error is encountered.  (Either              ignore errors and just mark the file system erroneous and              continue, or remount the file system read-only, or panic              and halt the system.)  The default is set in the file              system superblock, and can be changed using tune2fs(8).According to Ubuntu's documentation: https://manpages.ubuntu.com/manpages/xenial/en/man8/mount.8.html, this is the default option for mounting ext4 file systems on Ubuntu.To prevent a file system from remounting itself as read-only, change its mounting option to not use errors=remount-ro. One alternative is to use errors=continue, which will allow the volume to still be used after encountering an error.See this documentation for instructions on how to mount an EBS volume: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.htmlIf you have configured an entry in /etc/fstab for this volume, you can change the option in there. It will prevent the volume from being remounted as read-only.CommentShareDaniel_Lanswered 3 months agorePost-User-0747912 3 months agoThank you for the answer, that makes sense. One question, though. I understand that some data might be lost following this error. If so, can I just ignore that? I mean how can I be sure that the volume is consistent afterwards?ShareDaniel_L 3 months agoThis documentation provides guidance for handling I/O errors on EBS volumes: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstances.html#DeviceBlock. Taking frequent snapshots can substantially reduce the risk of data loss due to failure. Note that newer kernels (5.10 and later) provide more options to help reduce the risk of data loss. See: https://lwn.net/Articles/842385/Share"


Question: Unable to fetch delete marker version IDs for s3 objects through Boto3 Client/Service Resource.

"When I am trying to fetch the s3 object attributes, I am getting these errors:Error Message: The request signature we calculated does not match the signature you provided. Check your key and signing method.ORError Message: The x-amz-object-attributes header specifying the attributes to be retrieved is either missing or emptyFollowComment"

Answer: "2Credentials: Make sure you're using the correct AWS Access Key ID and Secret Access Key. If you have multiple AWS accounts or users, it can be easy to mix up keys.Region: Ensure that your request is being sent to the correct AWS region. The region in your Boto3 client creation should match the region where the bucket is located.Time Sync: Make sure your system time is accurate. AWS request signing is time sensitive and if your system clock is skewed, it could lead to errors.As for the error "The x-amz-object-attributes header specifying the attributes to be retrieved is either missing or empty", it suggests that the request made to S3 is missing some required informationand please follow this article https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAttributes.htmland why and how you can set x-amz-object-attributesCommentShareEXPERTsdtslmnanswered 15 days agoEXPERTkentradreviewed 15 days ago"


Question: How can I find the Query ID for an Athena Saved Query on AWS console?

"After I create a query in Athena using the AWS console, how can I find the Query ID associated with that query?If I run the query and it fails, I can see the Query ID in the AWS console by going to "Recent Queries" and clicking the "Failed" link next to the query run (https://docs.aws.amazon.com/athena/latest/ug/querying.html), but this doesn't work for queries where the status is "completed".I can also use the API to first call ListNamedQueries, then iterate through that resulting list of query ids and call GetNamedQuery on each one to see if the name matches the query I'm looking for. But that's a really roundabout process just to get the query id for a query I just created.Is there another way to simply find a named query's id via the console that I'm not seeing?FollowComment"

Answer: "0No, I'm not looking for the Execution Id. That would be the id for a query run. I'm looking for a way to find the NamedQueryId for a Named Query from console.CommentShareAWS-User-8508909answered a year agoFabrizio@AWS EXPERTa year agoThank you for the clarification, could you also let me know how you want to use the Named query ID?As mentioned the NamedQueryId is different from the QueryId (in the Recent Queries tab for Failed execution query id = execution id).in the Saved Queries tab, you can see all the Named query in the workspace you are using, that tab shows the   Actual Name of the query , the SQL and the description.the only thing missing is the database and the Named query Id itself, https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena.html#Athena.Client.get_named_queryShare0Hi ,based on the documentation link you shared I think that the id you are looking for is the **Execution Id** . If want to check click on the failed query and look at the Query Id you will see it should match the execution id.hope this helps,CommentShareEXPERTFabrizio@AWSanswered a year ago"


Question: EC2 instance 100% CPU due to system interrupts

"We have a t2.micro instance (Windows) that every so often goes to 100% CPU and stays there indefinitely, resulting in massively slow response time. The Windows performance monitor shows all available CPU resource is consumed by system interrupts. The only solution seems to be to stop the instance and restart it. Then it runs fine for another week or two, then this happens again.I read a few accounts in the forum (old) that when a single CPU instance lands on vCPU 0 that it handles all the IO interrupts for all other VMs on that host. I don't know if that is still true, but how can I avoid this? It is not acceptable to have to restart the instance - we don't find out about this until it is already happening and our customers are getting poor service. It is not OK that the resources we PAY for are being used to service other users of the hardware.The t2.micro performs just fine otherwise so I see no need to upgrade to a larger instance (the next level would be more than 2X the cost).Any idea if this in in fact the issue, or how to avoid it?FollowComment"

Answer: "2Hello! The t2.micro instance type has just 1 GB of ram https://aws.amazon.com/ec2/instance-types/t2/. Depending on your workload, it might not have enough resources for what you need.Additionally, t2 offers "burstable" CPU performance. This means that by default, you get a baseline level of performance according to the size, and it bursts when needed until the CPU bucket credits expire. Then it will burst again when needed when the CPU bucket credits refill. For more information about the CPU burst model, please check https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-credits-baseline-concepts.htmlIn your example, your baseline performance is 10% per VCPU (as explained in the above link) and you have 1 VCPU. For more insights about your particular case, I recommend checking the CloudWatch Metrics listed here https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/viewing_metrics_with_cloudwatch.html . I would start by checking the CPUUtilization as well as the CPUCreditBalance metric. If the latter one is close to zero, it means the instance is requiring additional CPU more than the time the burst credit allows to. If this happens, I have a couple of suggestions:Increase the instance size, so you can have a larger baseline performance and amount of CPU credits.Turn on "unlimited mode", which means the instance will be allowed to burst 100% of the time. For more information about unlimited mode, please check https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-performance-instances-unlimited-mode.htmlIf you need more assistance, I encourage you to create a new support case and then an engineer will have access to see your instance metrics and provide additional suggestions.CommentShareSUPPORT ENGINEERFrancisco_Lanswered a year agoEXPERTBrettski-AWSreviewed a year agoRoB a year agoIf CPU credits are a factor then consider switching from a t2 to a t3 as by default t3 launch as unlimited.ShareFrancisco_L SUPPORT ENGINEERa year agoRoB correct, although some customers prefer to have unlimited mode as disabled by default to prevent any unforeseen billing charges.Share"


Question: Cannot see my HIT created from the API using boto3 and Python

"I am planning to do a large scale crowdsourcing experiment with approx. 1000 HITs and 3-5 assignments each. I would like to do this using the API and Python, since I am very interested in the ReviewPolicies Feature. I tested this (not only in the sandbox, but also in the marketplace) and can't find my created test HIT (reward set to 0.01).What could be the reason for this? Also I read in external forums, that HITs created with the API are not visible in the developer interface. But they must be visible to workers on the website interface, aren't they? If not, how will these HITs be found.FollowComment"


Question: A way to backup personal Quicksight dashboards/assets

"I would like to park my personal Quicksight account (enterprise billing) so that I am not billed while it is inactive.Is there a way to backup Quicksight dashboards/assets, import formulas, visualizations, other metadata, etc?Or is the only way to reduce the billing is to discard and lose all experiments?FollowComment"

Answer: "0I've located the following references which may be helpful, in terms of backing up your QuickSight resources.https://docs.aws.amazon.com/quicksight/latest/user/copying-a-dashboard.htmlhttps://docs.aws.amazon.com/quicksight/latest/user/managing-users-enterprise.htmlhttps://docs.aws.amazon.com/quicksight/latest/user/saving-changes-to-an-analysis.htmlI would suggest opening a support case with our Billing team directly for assistance & guidance with reducing the billing on your account, here: http://go.aws/support-center. This way, an accurate analysis can be made of your request, to avoid the loss of any important data on your account.-- Ben G.CommentShareEXPERTAWS Support - Benanswered a year ago0Thank you. So this means that since Quicksight does not support backup/restore and metadata export, it also cannot be integrated with source control and therefore not an enterprise-grade software yet?Are there any plans to implement source control integration?CommentShareAWS-User-0599270answered a year ago0Other than CloudFormation, there is support for Amazon QuickSight object migration and version control with the use of QuickSight APIs. Please refer to this blog post and demo/code samplehttps://aws.amazon.com/blogs/big-data/biops-amazon-quicksight-object-migration-and-version-control/https://devops.learnquicksight.online/porting-content.htmlThere already are pending feature requests for this to be easier using the console or CLI.CommentShareChan_Nanswered a year agoEXPERTFabrizio@AWSreviewed a year ago"


Question: Cloudwatch alarms with Airflow MWAA

"Hello,I am setting up a managed Airflow service (MWAA), and would like to setup alerting.I am looking at alerting via Cloudwatch metrics. I would like to be alerted when any task fails within a Dag.  (TaskInstanceFailures)Unfortunately, it seems to exist only at the '1 dag'/'1 task' level, or at the 'All dags'/'All tasks' level when creating an alarm.Whenever I try to select 'All tasks' for 1 specific Dag, nothing shows up. (Even through the query builder 'Graph with SQL query' option).I have found a way to use the 'DAGDurationFailed' metric but it really does not seem like an ideal method.Is there any other way to set up an alarm for one dag, without having to write one alarm per task ?Furthermore, my alarm would send a notification with SNS, to send emails. The email message does not contain much information about the error.We were previously using Sentry for alerting on Airflow task errors, is it possible to send more detailed information about the actual task error in the email alert via cloudwatch ?Thank you.FollowComment"

Answer: 1One option is using a callback on your task failures that emit CloudWatch metrics or alarms.CommentShareJohn_Janswered a year ago


Question: "AWS session manager, force requirement of SSH key"

"Hi,I was able to configure AWS session manager to use SSH keys over session manager tunnel as it is described here -> https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html.But now i need to force user to provide SSH keys, because now, even tho i can use SSH keys to authenticate into the EC2 instance, im still able to to it without providing SSH keys, just by using aws ssm start-session command.As i suppose i can add some kind of policy for that, something like:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::accountid:user/test-user"            },            "Action": [                "ssm:TerminateSession",                "ssm:ResumeSession"            ],            "Condition": {                "StringEquals": {                    "ssm:StartSession/RequireSSH": "True" ( parameters made up, by me )                }            }        }    ]}But im not sure what should be in the place of "ssm:StartSession/RequireSSH": "True",Any help will be appreciatedJoannFollowComment"

Answer: "1The condition you want is ssm:SessionDocumentAccessCheck. See: Controlling user permissions for SSH connections through Session Manager. Something like this:{    "Version": "2012-10-17",    "Statement": [      {            "Effect": "Allow",            "Action": "ssm:StartSession",            "Resource": [                "arn:aws:ec2:region:account-id:instance/*",                "arn:aws:ssm:*:*:document/AWS-StartSSHSession"            ]        },        {            "Effect": "Deny",            "Action": "ssm:StartSession",            "NotResource": "arn:aws:ssm:*:*:document/AWS-StartSSHSession"        }    ]}CommentShareEXPERTkentradanswered a year agoJoann Babak a year agoThank you very much Kentrad !Share0Accepted AnswerIt appeared that the solution that @Kentrad provided didn't worked for me fully as i wanted, but what did worked for me is :{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": "ssm:StartSession",            "Resource": [                "arn:aws:ec2:eu-north-1:<accountid>:instance/*",                "arn:aws:ssm:*:*:document/AWS-StartSSHSession"            ],            "Condition": {                "BoolIfExists": {                    "ssm:SessionDocumentAccessCheck": "true"                }            }        }    ]}I found this solution mainly here https://docs.aws.amazon.com/systems-manager/latest/userguide/getting-started-sessiondocumentaccesscheck.htmlCommentShareJoann Babakanswered a year ago"


Question: AWS Lightsail Container + Nginx Setup - Timeout

"I have deployed a container service with nginx as public endpoint, the nginx is configured to timeout as per the following attributes, however when a long running web request hits the server it times out in 60 seconds irrespective of overridding the default timeouts. The same setup works as expected without 60 seconds timeout in a local docker container configured with exactly identical configuration.server {  listen 80;  sendfile on;  default_type application/octet-stream;  gzip on;  gzip_http_version 1.1;  gzip_disable      "MSIE [1-6]\.";  gzip_min_length   256;  gzip_vary         on;  gzip_proxied      expired no-cache no-store private auth;  gzip_types        text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;  gzip_comp_level   9;  root /usr/share/nginx/html;  location /health/check {    access_log off;    return 200;  }  location /xyz {    proxy_pass https://xyz;    proxy_buffering off;    proxy_set_header X-Real-IP $remote_addr;    proxy_set_header X-Forwarded-Host $host;    proxy_set_header X-Forwarded-Port $server_port;    proxy_read_timeout 1200s;    proxy_send_timeout 1200s;    fastcgi_read_timeout 1200s;    uwsgi_read_timeout 1200s;  }  location / {    try_files $uri $uri/ /index.html =404;  }upstream xyz {  server xyz.domain.com:443;  keepalive 1;  keepalive_time 1100;  keepalive_timeout 1200;}I am trying to understand why the same configuration works on a local container as opposed to the one in Lightsail container service.FollowCommentPavel - AWS 2 months agore "a long running web request", can you please update the post with some examples of such requests? this will help us in offering possible answers, and other people who search the web for similar issues. For example, does this mean your application container performs some sort of slow computation before producing response?SharePavel - AWS 2 months agoThe underlying proxy that powers "public endpoints" feature has the "idle connection" timeout of 60 seconds, so this is the short answer as to why you get these timeouts.ShareNeeraj 2 months agoThank you Pavel for the response.It's the server that the container is performing the request which is performing time consuming computations. Is there any way the idle timeout can be increased on the proxy to make the content available to the container?SharePavel - AWS a month agoNeeraj, unfortunately this idle time out is currently not modifiable for public endpoints. Perhaps the server application can be changed with that limitation in mind? E.g. you can have an API that starts an expensive computation and responds immediately with an ID which can be used with a separate API that allows you to check if the computation for the given ID has concluded or still in progress. Another approach for long running HTTP interactions is to use https://en.wikipedia.org/wiki/WebSocket Both of these approaches would work well with the idle timeout as it is now.Share"


Question: Rotue 53 hosted zone not working at all after 5 days post transfer

"Transfered playoffpredictor.com from register.com to AWS. DNS records are correct in AWS. However the top level DNS servers (g.gtld-servers.net) are still pointing to register.com DNS records, which are of course refused.  This domain transfer happened 5 full days ago, so the 48 hour wait thing is long gone to clear out old DNS caches. Please help!  DIG output below:neaga@NEAGA-M-T2EB ~ % dig playoffpredictor.com @8.8.8.8; <<>> DiG 9.10.6 <<>> playoffpredictor.com @8.8.8.8;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 21025;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1;; OPT PSEUDOSECTION:; EDNS: version: 0, flags:; udp: 512;; QUESTION SECTION:;playoffpredictor.com.INA;; Query time: 143 msec;; SERVER: 8.8.8.8#53(8.8.8.8);; WHEN: Tue Oct 19 07:40:56 CDT 2021;; MSG SIZE  rcvd: 49neaga@NEAGA-M-T2EB ~ %neaga@NEAGA-M-T2EB ~ %neaga@NEAGA-M-T2EB ~ % dig playoffpredictor.com @ns-1417.awsdns-49.org; <<>> DiG 9.10.6 <<>> playoffpredictor.com @ns-1417.awsdns-49.org;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56898;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 1;; WARNING: recursion requested but not available;; OPT PSEUDOSECTION:; EDNS: version: 0, flags:; udp: 4096;; QUESTION SECTION:;playoffpredictor.com.INA;; ANSWER SECTION:playoffpredictor.com.5INA99.25.59.162;; AUTHORITY SECTION:playoffpredictor.com.60INNSns-105.awsdns-13.com.playoffpredictor.com.60INNSns-1417.awsdns-49.org.playoffpredictor.com.60INNSns-1675.awsdns-17.co.uk.playoffpredictor.com.60INNSns-708.awsdns-24.net.;; Query time: 53 msec;; SERVER: 2600:9000:5305:8900::1#53(2600:9000:5305:8900::1);; WHEN: Tue Oct 19 07:41:58 CDT 2021;; MSG SIZE  rcvd: 202neaga@NEAGA-M-T2EB ~ %neaga@NEAGA-M-T2EB ~ % dig playoffpredictor.com +trace; <<>> DiG 9.10.6 <<>> playoffpredictor.com +trace;; global options: +cmd.0INNSj.root-servers.net..0INNSg.root-servers.net..0INNSa.root-servers.net..0INNSm.root-servers.net..0INNSb.root-servers.net..0INNSd.root-servers.net..0INNSe.root-servers.net..0INNSc.root-servers.net..0INNSf.root-servers.net..0INNSi.root-servers.net..0INNSh.root-servers.net..0INNSk.root-servers.net..0INNSl.root-servers.net.;; Received 239 bytes from 173.37.87.157#53(173.37.87.157) in 82 mscom.172800INNSm.gtld-servers.net.com.172800INNSd.gtld-servers.net.com.172800INNSg.gtld-servers.net.com.172800INNSl.gtld-servers.net.com.172800INNSc.gtld-servers.net.com.172800INNSb.gtld-servers.net.com.172800INNSe.gtld-servers.net.com.172800INNSj.gtld-servers.net.com.172800INNSf.gtld-servers.net.com.172800INNSi.gtld-servers.net.com.172800INNSa.gtld-servers.net.com.172800INNSh.gtld-servers.net.com.172800INNSk.gtld-servers.net.com.86400INDS30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766com.86400INRRSIGDS 8 1 86400 20211101050000 20211019040000 14748 . UfGIF9LAypJoSmK9wAphO_ve/I8iQqQkjmNsxCha6UdKX3O_oPTxDU32 wEuajMlf873yjZ_FkBxKH_Crl2MLBTkcbmYV7E3rIZdmGeqKkPnOq/R0 JkuQdk4BbxRvyJhf_OKW_PrlX6gP2r8Q387eGTxQPuxPMblLELlto5NT nqaXYME4QD63W/UrSqNDWWL1/UYmJ_UaJhn/Dp0d_QT6wooM0yPJaRMo 5xd476vPXg5aXAcJhp/3g4jEB0CovicJfqyxBWOKcscYPNiq1z1EXiRt XsrWoy46uWhPaJqCpkHHmpus6XM8M4Ojuk1lxFnkBUARk0Co6MHRH/Ev sxr9dQ==;; Received 1180 bytes from 2001:7fe::53#53(i.root-servers.net) in 89 msplayoffpredictor.com.172800INNSdns101.register.com.playoffpredictor.com.172800INNSdns102.register.com.CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A  NS SOA RRSIG DNSKEY NSEC3PARAMCK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20211023042336 20211016031336 15549 com. Gw3gWAHJ0mo6ongxQgBkfV4GtTMyIgXoPv6oZsKbRBJsIMXSlICn5/sU 7TlEUdMfmzV_crrVV5VRlFZYgIQn8T5Nk/eDH8nxREYJNBsZ3JU6AEJz NVH70wWpmyIUtMlGV9ZiFx83tuaa_dVYN3Hsa9/+GRbYOeJiuto3Cspg lu2ez7SDrJQSaI5He5xLGaTLaBoR6eJNx9tqjzj6VW0EUA==DPILHSH4SBQN57VPSSEQ7VC3QKHLCEVE.com. 86400 IN NSEC3 1 1 0 - DPIM903TRFGMTPTV3SEO1A32PMPA5IA0  NS DS RRSIGDPILHSH4SBQN57VPSSEQ7VC3QKHLCEVE.com. 86400 IN RRSIG NSEC3 8 2 86400 20211026052044 20211019041044 15549 com. MkZIis6BHua8it3Mls90aKX9jFfXvPtMQ_hK/cVRAHWYAQ6qsBIDrcR0 vsg0kmlPUtbhl_YmY2lttI9pYWdJZ6IOG11vt5lev_4RpIHG95P2Le9X A3yRnX33BxmNQGwvERpx21mdtBPj4GW5MFilwPlVVLaMvrSlcD6HdjY2 9SO_NS6DH4sAHFchcL2nD/bPZ7pQdSJF1W8E14HLzW8Y+Q==;; Received 681 bytes from 2001:502:7094::30#53(j.gtld-servers.net) in 74 ms;; Received 49 bytes from 162.159.24.117#53(dns101.register.com) in 91 msFollowComment"

Answer: "0According internet testers, your domain registration have still old information about name servers, if you are testing for example with this: https://intodns.com/playoffpredictor.comPerhaps you transferred domain registration and name servers together and didn't update name servers? You have to go to registration details in https://console.aws.amazon.com/route53/home#DomainDetail:playoffpredictor.com and update name servers to those 4, which you can read from NS entry from your hosted zone for domain.CommentShareTommiForeanswered 2 years ago0Thank you!  I knew what to do but could not find that link in the aws console.  I was looking in hosted zones, not registered domains.Edited by: CiscoNeville on Oct 19, 2021 7:19 AMCommentShareCiscoNevilleanswered 2 years ago0I'm glad that was helpful. If you also want to rate my answer, I'm thankful... :)CommentShareTommiForeanswered 2 years ago"


Question: AWS Elemental MediaLive – Deactivate Outputs?

"Hi,is it possible to deactivate or switch off Ouputs in AWS Elemental MediaLive without removing them?Or how can I run tests with different output configurations before I go live with them?FollowCommentBruce Wilkinson 11 days agoPlease identify the type of test output you are referring to. RTP/FEC, HLS, RTMP etc.If you do not define an output Medialive will not allow you to save the configuration.A lot of times we will send to a private YouTube or Facebook channel but that only works with RTMP.You can make a fake output, properly defined, but not certain what the value of that would be. If you are only worried about the ingest flow then that would be an option.Good luck.Sharesaltarob 10 days agoThank you. Any output. To Facebook, to YouTube to a webpage, RTMP, HLS. On YouTube you can put the live stream on private. But not on a Facebook Page or on a webpage. Therefore I use usually another output to a private Facebook timeline and to a dublicate of the webpage player which is private. But If I can't switch from the private to the public output I can't really test a channel before I start the public live stream. Okay I could test it on another channel with the private outputs but then I don't know if my public live channel will work and if everything is okay with it.Share"

Answer: "0Accepted AnswerHi Saltarob,individual control of outputs is not currently a feature of MediaLive. It is possible to pause one or both pipelines form outputting, but it affects all of the outputs for that pipeline.Regards,SteveCommentShareSteve_Wanswered 12 days agosaltarob 12 days agoHi Steve, thank you so much for your quick answer. Pause could be useful. How can I pause a pipeline?Share0Hi Saltarob,Pipeline pause is achieved using the channel schedule: https://docs.aws.amazon.com/medialive/latest/ug/working-with-schedule.htmlCommentShareSteve_Wanswered 8 days agosaltarob 3 days agoThank you so much.Share"


Question: EKS Cluster Nodeport access connection refused.

"Hello,I created an EKS cluster with a self-managed node group of two ec2 nodes. And deployed AWX operator with an awx instance. everything went OK until the step of accessing the nodeport through one of the nodes private IP, I got connection refused.the SG is the same for the cluster and both nodes, with a rule to allow inbound traffic to service port.Note: the flow is open to the worker node's private IP (through VPN) I can access it using SSH.from inside the node I am able to do curl:curl -Is  http://localhost:svc_portHTTP/1.1 200 OKServer: nginxDate: Tue, 14 Jun 2022 18:31:54 GMTContent-Type: text/html; charset=utf-8Content-Length: 1054Connection: keep-aliveVary: Accept-Language, Origin, CookieContent-Language: enX-API-Total-Time: 0.032sX-API-Request-Id: 6ba658763873asdfasdf36574367Access-Control-Expose-Headers: X-API-Request-IdStrict-Transport-Security: max-age=15768000X-Frame-Options: DENYX-Content-Type-Options: nosniffCache-Control: no-cache, no-store, must-revalidateExpires: 0Pragma: no-cacheThanks,GasoulFollowComment"

Answer: "1The Security Group will not return a "connection refused" error. Security Groups act like firewalls. If the rules in the Security Group don't allow the traffic, then no response is returned, the packets will be dropped, and requests usually time out. "Connection Refused" generally means that you reached the host, on a particular network port, but the service you attempted to reach on that port is not listening on that port. For example, if you were trying to access HTTP on port 80, but the service was actually listening on port 8080, and nothing else was listening on port 80, then the attempt to request on port 80 would be refused by the host.Additional troubleshooting should include trying to access the host:port from another host, preferably in the same network segment and affected by the same Security Groups. This troubleshooting document might help as well.CommentSharejimmyrayanswered a year agoJeremy a year agoIn addition to what @jimmyray suggested, I would recommend reading https://learnk8s.io/kubernetes-network-packets#inspecting-pod-to-service-traffic. It will help you diagnose the issue you're having.SharerePost-User-8446994 a year agoThanks @Jimmyray and @Jeremy.I was able to deploy internal Loadbalancer which let me access the ansible AWX instance from premises environment.Now, I want to manage target hosts on premises.How to open the needed flow?Share"


Question: AWS Data Pipeline stuck in "WAITING FOR RUNNER"

"Hi,all my AWS Data Pipelines i create to export data from a dynamodb table to a .csv in an s3 bucket are stuck in the "WAITING FOR RUNNER" state. I am following the tutorial here: https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-importexport-ddb.htmlI also checked the values "Runs on resource" which is: "EmrClusterForBackup" and "workerGroup" which is "df-0274180EXA6BQJAA9HV_@EmrClusterForBackup_2019-01-13T10:21:37"can you tell me if these values are correct?FollowComment"

Answer: "0Common issue would be ,check the IAM access of emr ec2 role and make sure that it has datapipeline:*make sure that internet connectivity is available from emr/ec2  either nat or igwyou can add ec2 key in emr cluster and log in to emr master  (ssh) then check task runner logtail -100f /mnt/taskrunner/output/logs/tasrunner*.loghttps://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-how-task-runner-user-managed.html#dp-verify-task-runnerCommentShareShivananswered 4 years ago0Hi Shivan,thank you for your answer.I think with EMR EC2 Role you mean the DataPipelineDefaultResourceRole? The one i assign durin the creation of the pipeline to the field "EC2 instance role"?If so, i checked. It has datapipeline full access.If i activate my datapipeline, then switch to the EMR console, and check the respective cluster, it says "Install Task Runner cancelled".Do you have an idea what the issue could be?Best regardsEdited by: OlliHC on Jan 14, 2019 7:08 AMEdited by: OlliHC on Jan 14, 2019 7:14 AMCommentShareOlliHCanswered 4 years ago0check your pipeline terminateaftertimeout parameter and increase 2-3 times normal run +15 minutes for cluster launchExample,Job usual run time is 10 minutes then 30 +15 =45 minutes would be best value for terminateafterCommentShareShivananswered 4 years ago0Just to be precise:I fixed it.The reason was that our limit on EC2 instances of the respective account was reached.We texted the AWS support and their increased our limit, now it works fine.CommentShareOlliHCanswered 4 years ago"


Question: Invalid S3 request when creating Iceberg tables in Athena

"I'm having trouble creating iceberg tables in Athena.  I keep getting "Invalid S3 request" as an error message.CREATE TABLE iceberg_test (    event_date timestamp,     log string)PARTITIONED BY (     hour(event_date))LOCATION    's3://bucket/iceberg/test'TBLPROPERTIES (    'table_type'='ICEBERG');FollowComment"


Question: Getting an error configuring jupyter on Deeplens

"HelloTrying to do one of the advanced labs and run this command:jupyter notebook --generate-configHere is the output:**aws_cam@AWSDeepLens:~$ jupyter notebook --generate-configusage: jupyter [-h] [--version] [--config-dir] [--data-dir] [--runtime-dir][--paths] [--json] [--debug][subcommand]Jupyter: Interactive Computingpositional arguments:subcommand     the subcommand to launchoptional arguments:-h, --help     show this help message and exit--version      show the versions of core jupyter packages and exit--config-dir   show Jupyter config dir--data-dir     show Jupyter data dir--runtime-dir  show Jupyter runtime dir--paths        show all Jupyter paths. Add --json for machine-readableformat.--json         output paths as machine-readable json--debug        output debug information about pathsAvailable subcommands: kernel kernelspec migrate run troubleshootJupyter command jupyter-notebook not found.**Any ideas?FollowComment"


Question: Are there any examples of using the 'Photon Engine Fusion' framework on Gamelift?

"I'm using 'Photon Engine Fusion' as network solution, using shared mode. Now I am ready to migrate the game to Gamelift, Is there an example of this?FollowComment"

Answer: "0Hi,We don't have specific examples of integrating GameLift with Photon. https://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-howitworks.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-integration.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/integration-custom-intro.htmlThese documents are a great place to start learning about GameLift.From a high level overview, you need to integrate the GameLift SDK with your game server and upload it as a GameLift Build on your AWS account, then create a GameLift Fleet with that Build. Your game client can then communicate with the fleet to create game sessions, and then place players on those game sessions.CommentShareJuneHoP_AWSanswered 6 months ago"


Question: pyspark dataframe in glue notebook mode=overwrite creates empty files at each path level

"I'm writing partitioned parquet data using a Spark data frame and mode=overwrite to update stale partitions.  I have this set:  spark.conf.set('spark.sql.sources.partitionOverwriteMode','dynamic')The data is being written correctly with all the partitioning being set correctly, but I am also getting empty files created at each level of the path named <path_level>_$folder$.  Removing mode=overwrite eliminates this strange behavior.  Is there any way to prevent these zero size files from being created?  Have I misconfigured something?FollowComment"

Answer: "0Those are folder markers, they are used because S3 doesn't allow creating empty folders without objects, you can just ignore them they shouldn't do any harm, it's very common to have them.Once the corresponding directory has any files, they can be safely removed.CommentShareGonzalo Herrerosanswered 2 months ago"


Question: What is a good AMI for GPU-based tensorflow work?

"Hi,Using the ec2 instance (p3.8xlarge - 4 V100 GPUs) I cannot find an AMI that makes use of these with tensorflow [-gpu].I've tried:NVIDIA GPU-Optimized AMI - issues, very little installed, but that's ok as it is ubuntu and you can install whatever.  But, with tensorflow-gpu installed:print (tf.version)print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))2.9.1Num GPUs Available:  0Other AMI's see the GPUs, but are unable to utilize them (using nvidia-smi to monitor usage).  For this, I've tried:Deep Learning AMI GPU TensorFlow 2.9.1 (Amazon Linux 2) 20220803Deep Learning AMI GPU TensorFlow 2.9.1 (Ubuntu 20.04) 20220803Has anyone successfully deployed GPUs on a DLAMI?  This time last year I was able to use individual GPUs ok (distributed is another thing, but this year none are working).Thanks,Example nvidia-smi output:+-----------------------------------------------------------------------------+| NVIDIA-SMI 515.48.07    Driver Version: 515.48.07    CUDA Version: 11.7     ||-------------------------------+----------------------+----------------------+| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC || Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. ||                               |                      |               MIG M. ||===============================+======================+======================||   0  Tesla V100-SXM2...  On   | 00000000:00:1B.0 Off |                    0 || N/A   40C    P0    35W / 300W |      3MiB / 16384MiB |      0%      Default ||                               |                      |                  N/A |+-------------------------------+----------------------+----------------------+|   1  Tesla V100-SXM2...  On   | 00000000:00:1C.0 Off |                    0 || N/A   39C    P0    35W / 300W |      3MiB / 16384MiB |      0%      Default ||                               |                      |                  N/A |+-------------------------------+----------------------+----------------------+|   2  Tesla V100-SXM2...  On   | 00000000:00:1D.0 Off |                    0 || N/A   38C    P0    40W / 300W |      3MiB / 16384MiB |      0%      Default ||                               |                      |                  N/A |+-------------------------------+----------------------+----------------------+|   3  Tesla V100-SXM2...  On   | 00000000:00:1E.0 Off |                    0 || N/A   38C    P0    38W / 300W |      3MiB / 16384MiB |      0%      Default |FollowComment"

Answer: "0Hello,The size of your model should be a factor in selecting an instance. If your model exceeds an instance's available RAM, select a different instance type with enough memory for your application.Amazon EC2 P3 Instances have up to 8 NVIDIA Tesla V100 GPUs.Amazon EC2 P4 Instances have up to 8 NVIDIA Tesla A100 GPUs.Amazon EC2 G3 Instances have up to 4 NVIDIA Tesla M60 GPUs.Amazon EC2 G4 Instances have up to 4 NVIDIA T4 GPUs.Amazon EC2 G5 Instances have up to 8 NVIDIA A10G GPUs.Amazon EC2 G5g Instances have Arm-based AWS Graviton2 processorsRefer this link- https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html for more details on the selection of instances.Hope this helps, If yes, please click accepted answer.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 10 months agorePost-User-3205485 10 months agoYes, the size of the model is one thing - mine is pretty large (5 MByte), but the data set can be 1000's of times larger than that.  So holding batches in memory of the GPU is critical.  Hence I chose p3.8xlarge, which is the smallest I can get away with.But the question wasn't about that, it was what AMI instance to choose.  I listed three that were no use and was asking the community what they have found to work well with tensorflow, good utilization of GPUs, and potentially distributed training.Share"


Question: opensearch cross-cluster replication stuck in 'BOOTSTRAPPING' state

"I’m having trouble setting up multi-region replication between two aws opensearch domains. I’ve been following the documentation (https://docs.aws.amazon.com/opensearch-service/latest/developerguide/replication.html) but once I start the replication via PUT call and query the replication status I see that the status is stuck at ‘BOOTSTRAPPING’ and I can see that no bytes have been transferred, even hours later.PUT request payload to start the replication:{  "leader_alias": "dss-pre-prod-use2-usw2",  "leader_index": "devportal.resourcev2",  "use_roles":{    "leader_cluster_role": "all_access",    "follower_cluster_role": "all_access"  }}GET response:{  "status": "BOOTSTRAPPING",  "reason": "User initiated",  "leader_alias": "dss-pre-prod-use2-usw2",  "leader_index": "devportal.resourcev2",  "follower_index": "devportal.resourcev2",  "bootstrap_details": {    "bytes_total": 0,    "bytes_recovered": 0,    "bytes_percent": 0.0,    "files_total": 0,    "files_recovered": 0,    "files_percent": 0.0,    "start_time": 1650317767399,    "running_time": 18585600  }}There are only 5 documents in the leader index, so the replication seems stuck. I've tried deleting the follower index and restarting the replication multiple times but with the same result. How can I debug this issue?FollowComment"

Answer: "0Hi , hope this situation has been resolved by now.for the next time , or otherwise if you still have the issue. This is a case in which AWS Support should be able to help you, because no general answer may apply and something contextual to your account might be causing the issue.Your best next steps would be to open a Support case, in case you have not yet a Support subscription, consider to upgrade to Developer Support or Business support depending on the urgency of your case.Without looking at the internal logs or having more detailed information on the situation it is not possible to provide guidance, sorry.hope this helpsCommentShareSUPPORT ENGINEERVarun_Sanswered a year ago"


Question: IoT Core Rule SQL - OR operator inconsistency between IoT SQL versions

"Hello, I am trying to create an IoT Rule that selects only the fields I need from a specified MQTT topic and republishes the message to another MQTT topic. The SQL query I am using is the following:SELECT state.debug AS state.debug, state.command AS state.command, version FROM '$aws/things/my-thing/shadow/update/delta' WHERE state.debug <> '' OR state.command <> '' When the selected SQL version is the latest (2016-03-23), the right operand of the WHERE clause is completely ignored and the rule is triggered only if the state.debug property has a value. When I change the SQL version to 2015-10-08, however, the rule works just fine.It seems that there is no breaking change on the way the two SQL versions handle logical operators, according to the AWS documentation. Therefore, is it possible that there is a bug with 2016-03-23 IoT SQL version or am I missing something else here?FollowComment"

Answer: "0Hi. I tried to replicate your issue with the WHERE clause, but didn't find a problem with SQL version 2016-03-23. Are you still stuck on this problem?CommentShareEXPERTGreg_Banswered a year agoAWS-User-3181140 a year agoHello Greg, thanks for your reply! Yes, unfortunately I am still stuck on this.Could you please provide the SQL query you tried as well as the message you used it on?The messages published to the topic I am querying are structured this way:{"state": {"debug": {"remoteLogging": false},"command": "ls",...},"version": ... ,"timestamp": ... ,"metadata": ...}Should I maybe check in a different way than state.debug <> '' OR state.command <> '' whether either of these 2 properties is not empty?Thanks again!ShareGreg_B EXPERTa year agoI realize I misread your SQL statement. Now I can indeed see the behaviour. Regardless, it will be more semantic (and it will work) to use the IsUndefined() function: WHERE IsUndefined(state.debug) = False OR IsUndefined(state.command) = False. I hope this unblocks you.ShareAWS-User-3181140 a year agoHey Greg, thanks for you response. Indeed, isUndefined() seems to be exactly what I need!Out of curiosity, have you managed to identify why the OR operator behaves this way in my initial query?ShareGreg_B EXPERTa year agoHi again. No I didn't. It's odd. I raised a ticket for it internally. Nonetheless, undefined is not necessarily an empty string, so the IsUndefined function is a better choice.Share"


Question: how large traffic can ddos shield defend? 500Gbps offen occurs in games industry.

"Hi , how large traffic can ddos shield defend? 500Gbps offen occurs in games industry. I  can not find any information about it . thanksFollowComment"

Answer: "0I think specific numbers are hard because of the varying types of DDoS attacks and nuance in how they are conducted. Vendors are hesitant to make very specific promises about those numbers.The real answer is that average DDoS attacks are still far lower than 500 Gbps. (Google search "average size of DDoS attack" and comb through several recent reports and analysis). I've seen 150 Mbps to 5 Gbps reported as an average size.A 2.3 Tbps attack was reportedly mitigated by shield.. You can find reports about this on several news sites.I can't find any official numbers or limits in AWS Shield documentation. Again, likely because of nuance in how these attacks are conducted and customer's specific configurations.CommentSharenewrustanswered a year ago"


Question: setup 1-box stage in lambda ci-cd pipeline

"One box stage in CICD pipeline is a stage before production stage. This stage also will receive the production traffic. The goal of this stage is to validate the new changes with very small prod traffic and keep the blast radius to minimal incase of issues.In server based deployment, this stage will contain only 1 machine to receive very small % of traffic. The code changes will be deployed to 1-box stage before production and leave it for some time to validate the behaviour of new change.  If there is an issue in 1-box stage with the new code, the blast radius will be minimal and roll back is faster since the changes are deployed in only one machine.It is explained in "Backward compatibility and one-box testing" topic in this link - https://aws.amazon.com/builders-library/automating-safe-hands-off-deployments/How do we setup the same in lambda ci-cd ? Any leads on this will be super helpful.FollowComment"

Answer: "1Accepted AnswerWith Lambda you don't really have boxes, however, you can easily route a small percentage of your traffic to a new version and later increase that percentage all the way to 100% or decrease it to 0%. You can do this using Alias routing configuration. CodeDeploy supports different traffic shifting options.CommentShareEXPERTUrianswered a year ago"