Question: What is wrong with calling DescribeMatchmaking vs polling a SNS topic

"I am at a point where I have set up a SNS topic for GameLift, and I have subscribed a SQS queue to the topic and am able to get messages via reading from the queue. However, I am now wondering, what is the difference between doing this and calling DescribeMatchmaking to get the status of my matchmaking tickets? I have read on previous posts that it is bad to call DescribeMatchmaking at high volumes and that it should only be used in local testing. I am wondering why that is? Does DescribeMatchmaking put strain on the GameLift fleets/resources? And also what is the recommend approach then for notifying the client about a match/ticket's status? Because right now, I am currently deciding between having the client call a Lambda-function-integrated-API every 10 seconds that eitherCalls DescribeMatchmakingReads from an SQS queue that is subscribed to a SNS topic that is receiving GameLift event notificationsReads from a database like DynamoDb that gets the SNS topic messages from a Lambda function that is subscribed to the topic.Lastly, any examples would be very helpful. Thank you in advance.FollowComment"

Answer: "0Accepted AnswerThe suggestion is that you have something listening to the SNS topic that updates some storage (memcache, dyanmodb etc). Clients call that something to find out about their specifics wrt to matchmaking. Or your thing/service has a notification mechanism to call your clients to action / pass connection information.Lots of ways you could build this but it depends on if you have other requirements or other things you are integrating with (ie how you are managing player authentication etc).Sorry if thats all a bit vague but GameLift as service leaves this up to developers (which may not be ideal). Hoping that GameLif team will provide a little more direction in this space (we'll see what happens) or somethings to make this part of the integration easier.CommentSharerePost-User-5332620answered 3 years ago0I don't work for GameLift so I can't refer to the why of DescribeMatchmaking, I do know that GameLift restricts the calls per second on this API.Quoting from this threadhttps://forums.awsgametech.com/t/unable-to-compile-gamelift-3-2-0-ue4-4-18/4815/1"2. The client could just poll, but we don’t recommend this for when your game launches. There are throttling limits that GameLift will enforce on your game if across all clients they are polling at too high a TPS. For large games that could have hundreds or thousands of clients in matchmaking simultaneously, you could exceed your polling limits quickly. Hence why we don’t recommend you launch with this set up."(thread actually has a bunch of useful information - but it links to this great blog post):https://aws.amazon.com/blogs/gametech/fitting-the-pattern-serverless-custom-matchmaking-with-amazon-gamelift/ CommentSharerePost-User-5332620answered 3 years ago0[quote="chrisgong, post:1, topic:8106"]I am at a point where I have set up a SNS topic for GameLift, and I have subscribed a SQS queue to the topic and am able to get messages via reading from the queue. However, I am now wondering, what is the difference between doing this and calling DescribeMatchmaking to get the status of my matchmaking tickets? I have read on previous posts that it is bad to call DescribeMatchmaking at high volumes and that it should only be used in local testing. I am wondering why that is? Does DescribeMatchmaking put strain on the GameLift fleets/resources? And also what is the recommend approach then for notifying the client about a match/ticket’s status? Because right now, I am currently deciding between having the client call a Lambda-function-integrated-API every 10 seconds that eitherCalls DescribeMatchmakingReads from an SQS queue that is subscribed to a SNS topic that is receiving GameLift event notificationsReads from a database like DynamoDb that gets the SNS topic messages from a Lambda function that is subscribed to the topic.Lastly, any examples would be very helpful. Thank you in advance.[/quote]Hey @REDACTEDUSERIf I see any more specific information, I will forward it to you. Good luck!CommentSharerePost-User-7328120answered 3 years ago"


Question: Auto scalling group will not accept Security Group

I have an Launch Template which has the security group attached to it. I can launch an instance that works fine from this template. From the template I created an Auto Scaling gorup. When I look at the AS group I do not see the Security Group at all. I just see a dash. When I try to run the AS group it fails to create the instance. When I look at the failed instances they too do not have a security group.What am I doing wrong??? How do I add the group to the AS Group or get it to pull in the one from the Launch Template.FollowComment

Answer: 0When a launch template is specified for an ASG a version number is specified - see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-autoscaling-autoscalinggroup-launchtemplatespecification.html.  Is it possible your ASG is using an old version of the launch template?CommentShareEXPERTskinsmananswered 4 months ago0No.  I initially created the Launch Template with the SG.  So when I setup the ASG it was at version 1 to begin with.I then updated the Launch Template (version 2) with out the SG hoping that I could configure it in the ASG.  So when I told the ASG to use version 2 I found I was unable to configure the SG in the ASG.  So I moved the ASG back to version 1 of the template.As a side note.  When I look at the terminated EC2s the ASG tries to create they all have an error in Security about SG cannot be "".  Sorry I do not have the exact message right now.CommentShareSystemgeekanswered 4 months ago0I couldn't see a reason why security group is not getting associated with the instance. Can you verify SG is in fact there for ASG in the 'Launch Template' section.Security Group may have just - (dash) but there should be an id listed under "Security group IDs". Looks fine when I tested it. SG was associated with instance. I even tried removing SG and let ASG add one more instance when SG associated with ASG does not exist and it rightly reported error during instance launch. "Launching a new EC2 instance. Status Reason: The security group 'sg-0888ac0884e71f175' does not exist in VPC. Launching EC2 instance failed." Somehow couldn't reproduce the error you are seeing. Just try creating new ASG from Launch template and see  "Security group IDs" has SG ID mentioned there.CommentShareGirish Mukimanswered 4 months ago0My problem what that my EBS was encrypted but the KMS key was not allowing the ASG to decrypt the EBS.I created a new key and gave it permissions for ASG.  This allowed the ASG to bring up the instance.CommentShareSystemgeekanswered 4 months ago


Question: Troubleshooting replication pauses due to swapping

"Hi, I have a mysql -> redshift replication that is frequently pausing CDC with the following message: "Reading from source endpoint temporary paused as total storage used by swap files exceeded the limit for task"I've tried increasing the values for BatchApplyMemoryLimit & MemoryLimitTotal, but it hasn't seemed to change the behavior, any suggestion on how I should troubleshoot this issue?FollowComment"

Answer: 0We're experiencing the same issue.Did you manage to fix it?Thanks!CommentSharejasanchezanswered 2 years ago0We are getting same message in our prod DMS jobs. DMS jobs are from different sources to a RDS oracle instance.Did you find any solution?CommentSharesboraanswered 2 years ago0We are also having this issue. Is there any update here?CommentSharekaveh-airanswered 2 years ago0I am getting this as well. any ideas? tips? i'm doing a single On Prem Oracle Instance to a Single AWS RDS Oracle Instance.CommentSharemtterryanswered 2 years ago0This might help some  https://stackoverflow.com/questions/66040310/aws-dms-swap-files-consumes-all-the-spaceCommentSharemacx78answered 2 years ago


Question: Display dynamic text in AWS IoT Twinmaker scene

"Is it possible to show dynamic/real-time text information on the scene in AWS IoT Twinmaker? I could not figureout if this is possible and if so how. Any pointers here would be very helpful.Also, additionally I am looking to see if it is possible to build some preset views for the scene - like top view, front view etc. of the twin model. Please suggestFollowComment"

Answer: "1Hi mj123, dynamic text and preset cameras aren't supported yet but we will take this feedback into consideration, thanks!CommentSharejohnnyw-awsanswered 8 months ago"


Question: Amazon Simple Queue Service in Localhost

It is possible to use the AWS SQS (NODE)Uncaught exception 'cURL_Exception' with message 'cURL resource: Resource id #44; cURL error: Could not resolve host: https (cURL error code 6). See http://curl.haxx.se/libcurl/c/libcurl-errors.html for an explanation of error codes.https://'FollowComment

Answer: "0Accepted Answeryou can configure the SDk like // Load the SDK for JavaScriptvar AWS = require('aws-sdk');// Set the region AWS.config.update({region: 'us-west-2'});List your Queue like // Load the AWS SDK for Node.jsvar AWS = require('aws-sdk');// Set the region AWS.config.update({region: 'REGION'});// Create an SQS service objectvar sqs = new AWS.SQS({apiVersion: '2012-11-05'});var params = {};sqs.listQueues(params, function(err, data) {  if (err) {    console.log("Error", err);  } else {    console.log("Success", data.QueueUrls);  }});For more details please follow this guide from AWS : https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/sqs-examples-using-queues.htmlCommentShareharshmanvaranswered 4 years ago0This is what i was looking for thanks you so much for sharing actually i am leaning node and aws both same and newbie so thanks a lot.CommentSharechaganlalanswered 4 years ago"


Question: Does AWS Glue have an issue with multi-threaded loads

"I'm experiencing a strange issue with AWS Glue. In order to speed up loads, I'm running multiple threads (spark.scheduler.mode=FAIR, Python multiprocessing.pool.ThreadPool(thread_count=5)). Each thread loads up a specific JDBC database table (glueContext.create_dynamic_frame.from_options(**options)and uses job bookmarks to handle deltas.What is happening that each thread starts and logs the table they should be loading. After the log entry comes the create_dynamic_frame.from_options() and all loads seem to stop there. Nothing happens from there and ultimately the job timeouts. The next step would be to write the result in an S3 bucket but that is not happening. Sometimes, when the job is re-deployed or executed several times manually, it completes, but that's really rare. This seems like a race condition of some sort...Does Glue have any limitations / issues in using Spark threading? Does anyone have a properly functioning JDBC load running in multiple threads?FollowComment"

Answer: "0Hi ,if the issue seems limited to JDBC load, have you tried to monitor also the source database ?Are you sure you are not experiencing queries timeouts while reading from the Database?Depending on the options you are using, and the number of concurrent loads (is it 5? or more?)  you might be submitting more queries than you expect to the database and it might be starting to slowing down.Try to monitor the jobs loking at the glue metrics and at Spark UI and at the same time monitor the DB you read from to understand where the slow down may be actually occurring.Hope hit helps,CommentShareEXPERTFabrizio@AWSanswered 8 months agopor 7 months agoHi,Thanks for the tips! We've monitored the source database and there's nothing of significance there. In the beginning of the loads the queries execute and we can see them, but then, they stop. It looks like the database returns the result but Glue never processes the results.We're now in the process of dropping Glue and handling the bookmarking by ourselves.Share0I just realized we have working multi-threaded processing jobs. They load data from S3 and after transformations etc. dump the data back in S3 with new format. Those are running nicely, so the issue seems to be specific to JDBC loads.CommentShareporanswered 8 months ago0Another note - this seems to be related to number of tables. The failing job loads around 850 tables. The same code works fine with loads containing around or less than 100 tables. This might support the race condition theory.CommentShareporanswered 8 months ago"


Question: Deleting and adding subnet from an aws eks cluster

How do I delete a subnet in an eks cluster using the command:aws eks.......I need to delete the cluster subnet:subnet-0b0c38aacba4f47d2And add the subnet to the cluster:subnet-093c61d66a8293e85FollowComment

Answer: "1Hello,It is currently not possible to add/delete your subnets that are registered with your EKS cluster. There is an open feature request for this use-case and the EKS team is currently working on it.Please be advised that you can run worker nodes in any subnets of your VPC, they do not have to be the same ones registered with your EKS cluster. Please refer EKS subnet considerations for more info.Thank you!CommentShareSUPPORT ENGINEERVenkat Penmetsaanswered 3 months ago"


Question: Bug Report - AppStream 2.0 Usage Reports CloudFormation template link broken

"In the us-east-2 region, AppStream 2.0 console, Usage Reports->Report Details, clicking the link to "CloudFormation template" results in the CloudFormation console reporting that the S3 bucket does not exist.Manually loading the template from us-east-1 works around the issue.FollowComment"


Question: Where can I see the actual Packets per Second allowed for each instance type?

"A network adapter has a metric pps_allowance_exceeded now, so there is an actual value for allowed PPS for each instance type. Is it published anywhere?FollowComment"

Answer: "0It's not defined because of a couple reasons:You can alter the MTU of any OSMany instances have burst credits (see here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-credits-baseline-concepts.html)The best way to see if an instance meets your needs is to configure the OS as you desire and then perform a sustained test to ensure you are not just seeing burst maximums.CommentShareRodney Lesteranswered a year agoCashier a year agoIf there is a metric then it must be a maximum allowed packets per second which is exceeded. Hence it is not a dynamic limit came from the bandwidth. It may be different for different instances, sure. Bandwidth is also vague defined for such burstable instances but it is well defined for other instances. I am not sure why pps limits aren't listed anywhere.ShareRodney Lester a year agoMTU alters packet size.Share"


Question: Export Autopilot model to GovCloud region

"HiAs AWS Sagemaker autopilot is not available in GovCloud region, is it possible to export a model trained on non-GovCloud environment in GovCloud environment.What I have done:Ran Autopilot on non-GovCloud environmentI was able to download the output model.joblib(preprocessing) and xgboost models from output bucket in S3 bucketI was not able to load the model.joblib preprocessing model since the sagemaker-sklearn-extention gives symbol not found error(https://issuehint.com/issue/awslabs/ml-io/28)Thanks for your help and insights in exporting the model to GovCloud environment.FollowComment"


Question: problem in installing library in sagemaker studio notebook

"I have a problem while installing librosathis error message comes:OSError: cannot load library 'libsndfile.so': libsndfile.so: cannot open shared object file: No such file or directoryI tried many solutions and didn't work, please helpFollowCommentHrushi_G EXPERT7 months agoCan you elaborate on how you are installing this library and which studio image you are using. I have tried pip install librosa in both the studio notebook images - tensorflow-2-6-cpu-py-ml-t3-medium, pytorch-1-10-cpu-py38-ml-t3-medium. You may also check this stackoverflow entry - https://stackoverflow.com/questions/38480029/libsndfile-so-1-cannot-open-shared-object-file-no-such-file-or-directoryShare"

Answer: "0Hello,I had this error when I was trying to install the library ffmpeg, This is the process I followed:On notebook terminal run the following commands:$ sudo yum install -y gcc$ wget https://github.com/libsndfile/libsndfile/releases/download/1.0.31/libsndfile-1.0.31.tar.bz2 $ tar -xf libsndfile-1.0.31.tar.bz2$ cd libsndfile-1.0.31$ ./configure —prefix=/usr \    --disable-static \    --docdir=/usr/share/doc/libsndfile-1.0.31 && make$ sudo make install$ pip install sndfileThen go ahead and install librosa but However you may want to setup a new environment with the librosa package preinstalled to it.$ conda create --no-default-packages -n name_of_env python librosa$ python -m ipykernel install --user --name=name_of_envOnce completed you may create a new notebook and use this new environment.CommentShareAswath Ram_Aanswered 2 months ago"


Question: RDS instance DNS resolution to EC2 on same VPC sudden change from private to public IP

"Hi,I have a PostgreSQL RDS instance (subscriber) that lives in the same AZ and VPC as an EC2 machine with PostgreSQL (publisher). I established logical replication subscription against the publisher about a month ago using a CNAME DNS that points to the EC2 DNS name. The publisher is restricted to only allow access from private IP in the VPC and this worked up to three days ago where it suddenly stopped. I find no events that indicate what happened but the cause is clear - the DNS name is suddenly resolving to the public IP at the RDS instance and is denied access by the publisher.I fail to understand how this can happen and also seems against documentation. That is, I expect the DNS lookup to reliably resolve to the private IP.Any ideas what can explain this behaviour and/or if my expectations are wrong or right?FollowCommentrePost-User-9593392 3 months agoDid you find a workaround or did it start working again? Seeing a somewhat similar behavior, just with RDS -> RDS. When the subscriber RDS instance looks up the DNS for publisher RDS instance it resolves to the public IP, though if I look up the publisher DNS from an EC2 instance it resolves to the private IP. They are all in the same AZ and VPC.Share"


Question: Elastic Search pricing: Pay as you go or not?

"Hi everyone,I would like to connect an Elastic Search cluster to my graphQL API (AppSync) to be able to perform a geosearch, but I'm quite confused by the pricing model...On the AWS Elastic Search page, I find "Amazon Elasticsearch Service lets you pay only for what you use – there are no upfront costs or usage requirements." and on its pricing page, the pricing table's header says "on-demand pricing". For a t2.small in Frankfurt, it costs $0.048/h. But is this for a continuously running server (hence $30/month) or only for the time you need the server to be running (genuinely on-demand)?BR,ArneFollowComment"

Answer: "0Hello Arne,In respect to Amazon Elasticsearch service pricing you are charged for the hours the instance is running, data transfer costs and Amazon Elastic Block Storage (which is optional and depends on your use case for the service).In the Frankfurt region you are correct, the cost to run a t2.small.elasticsearch instance is $0.042 per hour for a continuously running instance. Please note that this is the for pricing for the instance alone, more information can be found at [1]. Currently it is not possible to pause the instances running on Amazon Elasticsearch however, rest assured our Elasticsearch service team is working hard to introduce as many features as possible to ensure customers have the best possible experience with the service.More information in regards Amazon Elasticsearch pricing you can refer to the pricing calculator [2], which allows you to gain estimates of monthly costs with your chosen configuration for Elasticsearch.I hope I was able to answer your question!Best Regards,Shane-AWSReferences:[1] Elasticsearch Service Pricing Model - https://aws.amazon.com/elasticsearch-service/pricing/[2] AWS Pricing Calculator - https://calculator.aws/#/CommentShareShane-AWSanswered 2 years ago"


Question: AnalyzeDocumentRequest VS StartDocumentAnalysisRequest - JAVA SDK Version 2

I can see humanLoopConfig in AnalyzeDocumentRequest but can't see in StartDocumentAnalysisRequest.Just wondering is there any way to add Human Review (Amazone A2I) in StartDocumentAnalysisRequest?FollowComment

Answer: "1Accepted AnswerYou can not initiate Amazon A2I from the Amazon Textract call for Start* API requests (asynchronous requests), only from AnalyzeDocument.You can create a custom A2I flow after receiving the Textract response from any API.This workflow has an example: https://github.com/aws-samples/amazon-textract-idp-cdk-stack-samples#insuranceand this blog post links to a sample for multi-page processing with A2I: https://aws.amazon.com/blogs/machine-learning/automate-digitization-of-transactional-documents-with-human-oversight-using-amazon-textract-and-amazon-a2i/CommentShareMartin Schade _AWS_answered 3 months agorePost-User-9146392 3 months agoThank you so much MartinShare"


Question: How to redirect user to a custom website when user verifies from email verification link?

"I have been trying to implement aws cognito in my react application. I can successfully sign in and login a user.  My issue is after email verification link is sent to email and user verifies it, it always redirects to a default page. I have already defined a callback url in the app client setting. But I am still redirected to that default page. Any suggestions?  TIAFollowComment"

Answer: "0Unfortunately this isn't "natively" supported by the amplify-js library.  See this Github issue for more information on that.  However, there is a clever work-around involving API gateway and a lambda.  For a detailed description of that, see this Medium post as well as this StackOverflow answer.CommentShareMiguelUTanswered 9 months ago0Cognito allows you to configure your User Pool to send an email to your users when they attempt to register an account. You can configure this email to contain a link to Cognito's Hosted UI where the user's account will be marked as confirmed.This trigger template allows you to define an email message with a link to a static S3 bucket that you control, where the user's account will be confirmed and they can then be redirected to a URL of your choice (presumably your application).Below URL has a sample React code on how you can achieve that .https://docs.amplify.aws/cli/usage/lambda-triggers/#email-verification-link-with-redirectCommentShareJoel_Ponukumatlaanswered 9 months ago"


Question: Cron Job in AWS

"Hi Team,We are planning to run a cron job in AWS. The jobs runs for 1 hr every day.If we use EC2 rest of 23 hours we simply spend money on itAs cron job runs for 1 hr, lambda function will not work. As lambda function will get timed out after 15 minuteSo could you please let me any better solution for it.Regards,Rajesh Khanna BFollowComment"

Answer: "0Hi, I would look into:EventBridge Scheduler: https://aws.amazon.com/blogs/compute/introducing-amazon-eventbridge-scheduler/EC2 spot instance. Spot Fleet (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html). An alternative to Spot instances is to use ECS or Fargate, but then I would suggest to destroy the environment in case you do not want to incur into cost, since you want to running only for one hour.If above (EventtBridge scheduler) is not an option, then you may look into AWS Batch (https://aws.amazon.com/batch/)Hope it helps ;)CommentShareEXPERTalatechanswered 2 months agorePost-User-4459426 2 months agoThe EventBridge Scheduler requires a lambda function. But our cronjob will run for 1 hr. So lambda function might not helpShare0The answer above seems very attractive to solve your isssue.I'd like to contribute with something else. If you cron job runs and depends on linux platform for a number of reason(s): Setup a Lambda to start your EC2 instance,  starts your cron as fixed schedule and after an hour, a Lambda stops your EC2 instance.CommentSharePaulanswered 2 months ago"


Question: aws amaon connect chat aws connectparticipant send-message --content-type 'text/plain' --content hi --connection-token xxxx  gives "User is not authorized to access this resource with an explicit deny

for unit testing I am exercising amazon connect chat via cliaws connect start-chat-contact   <<< comes back with ParticipantTokenaws connectparticipant create-participant-connection --type WEBSOCKET --participant-token  <<<  come back with ParticipantTokenaws connectparticipant send-message --content-type 'text/plain' --content hi --connection-token xxxx  gives "User is not authorized to access this resource with an explicit deny"user has Administrator rightsFollowComment

Answer: "0Hi,your request is rejected since you're passing the participant token on your send-message operation instead of the connection token.You can come around by changing the connection type like this:aws connect start-chat-contact --instance-id XXX --contact-flow-id YYY --participant-details DisplayName=ZZZ// Returns participantTokenaws connectparticipant create-participant-connection --type CONNECTION_CREDENTIALS --participant-token <participantToken>// Returns connectionTokenaws connectparticipant send-message --content-type text/plain --content hi --connection-token <connectionToken>/* Returns MessageId and timestamp e.g.:{    "Id": "a1b2c3d4-5678-90ab-cdef-EXAMPLE22222",    "AbsoluteTime": "2022-10-26T07:03:19.534Z"}*/CommentSharefaddianswered 7 months agoMichael Oldham 7 months agothanks. this got me to the point shown but the hi message did not come through to the agent and the event log did not show the hi. Missing another step it looks likeShare"


Question: An error occurred (UnrecognizedClientException) when calling the ListClusters operation: The security token included in the request is invalid

"https://aws.amazon.com/cn/premiumsupport/knowledge-center/iam-assume-role-cli/Refer to this tutorial, run aws sts get-caller-identity to see that the identity has been switched correctly, but encounter the following error when runningaws eks list-clusters --region ap-east-1An error occurred (UnrecognizedClientException) when calling the ListClusters operation: The security token included in the request is invalidThere is no problem in specifying other areas. I have confirmed that both account A and account B have enabled the ap-east-1 region.The trust relationship for the IAM role is as follows:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::accountA:root"            },            "Action": "sts:AssumeRole",            "Condition": {}        }    ]}I can run the same command normally in cloudshell, why is this?FollowComment"

Answer: "0Hi,Security token is invalid - It appears credentials are not correct. Assuming you're trying cross account access.Follow this article to ensure the IAM role and policies are defined correctly - https://aws.amazon.com/premiumsupport/knowledge-center/cross-account-access-iam/And then use this page to access as - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-cli.htmlCommentShareAWS-CSinghanswered 4 months agoives 4 months agoThe same operation and cli command, only in ap-east-1 region errorShare"


Question: Trigger Area does not trigger (script canvas)

"Hello, no matter what I tried I was not able to make Trigger Area work with my player controlled sphere. For some reason, it does trigger on a camera following the sphere but not the sphere itself when I set it to Activate by All Entities.Images here because of 1 pic limit: https://imgur.com/a/B8NB5e5Every help much appreciated :slight_smile:Follow"

Answer: "0Trigger area to my knowledge doesnt support PhysX which could be your problem, 1.21 though has support for trigger events so if you havent switched over to that version, i recommend doing so....otherwise you would have to use legacy physics to get the functionality you are looking for.hope that helps.SharerePost-User-9018410answered 4 years ago0Thank you for the fast response. I actually am using 1.21 (upgraded from 1.18) for 3 days now so I will try look for how to use Trigger Events.I would actually like to keep using Physx because that was the only way I figured out how to apply pressure to actually spin the ball. I guess if Trigger Events won't work, I can still attach some invisible object and place it in front of a camera to work as an trigger.By the way, do you know if static objects can use their mesh for Physx colliders?SharerePost-User-0043578answered 4 years ago0[quote="MagicD3VIL, post:3, topic:7229"]if static objects can use their mesh for Physx colliders?[/quote]yes, but you are better to make special simplified version of mesh for thishttps://docs.aws.amazon.com/lumberyard/latest/userguide/component-physx-collider.html#creating-mesh-collidersSharerePost-User-2514050answered 4 years ago0yea as fluffy stated it possible, you will want to create a pxmesh in the FBX settings to achieve this, its still needs some work so it probably hit or miss but check it out!.I also have a tutorial dealing with the triggers using physX collider so I let you know if i run into any issuesSharerePost-User-9018410answered 4 years ago0Could you please link me the tutorial?SharerePost-User-0043578answered 4 years ago0recording it today, soon as its uploaded ill post a link hereSharerePost-User-9018410answered 4 years ago0PhysX triggers now works as intended for me, thanks for clearing that out. I appreciated your help :slight_smile:SharerePost-User-0043578answered 4 years ago0@REDACTEDUSERso what you will want to do instead is to add a PhysX collider instead and tick the "Trigger" checkbox so its treated like a trigger area, resize it to how you want...make sure the other has PhysX rigidbody and Physx Collider (i believe you have this based of your screenshot) and you should be good to go...Oh switch out those trigger nodes for the one under PhysX > Trigger category in Script canvas.one last thing i recommend is using the Debug gem, for better debugging options so instead of using print you can use "draw text on screen" node so its just easier to spot when something is triggered.REMOVEDUPLOADThe number on the node represents the amount of seconds you want it to show on the screen before it disappears.SharerePost-User-9018410answered 4 years ago0np! glad i could helpSharerePost-User-9018410answered 4 years ago0OMG, I've been racking my brain over this all day...It's so simple, I kept wondering why the enter trigger was under physX, well now i know! I wish there was a way to make this more obvious...even though now it seems like it is, lol.Thank you so much MagicD3VIL for asking the question and TheDevShop for the answer! 😃SharerePost-User-5447580answered 3 years ago"


Question: Querying postgres RDS from Athena

"Whilst querying a postgres DB on RDS via a lambda function, I keep getting the below error. I'm running the query as root and i definitely have access to the bucket spill bucket it's complaining about.GENERIC_USER_ERROR: Encountered an exception[java.lang.RuntimeException] from your LambdaFunction[jdbc_connector] executed in context[retrieving meta-data] with message[You do NOT own the spill bucket with the name: arn:aws:s3XXXX]The role that runs the lambda function has the following policy:{"Version": "2012-10-17","Statement": [{"Sid": "VisualEditor0","Effect": "Allow","Action": ["athena:StartQueryExecution","lambda:InvokeFunction","athena:GetQueryResults","s3:ListBucket","athena:ListWorkGroups","s3:ListMultipartUploadParts","s3:PutObject","s3:GetObject","athena:GetWorkGroup","s3:AbortMultipartUpload","athena:StopQueryExecution","athena:GetQueryExecution","s3:GetBucketLocation"],"Resource": ""},{"Sid": "VisualEditor1","Effect": "Allow","Action": "athena:","Resource": "*"},{"Sid": "VisualEditor2","Effect": "Allow","Action": ["s3:ListBucket","s3:GetBucketLocation"],"Resource": "arn:aws:s3:::XXX"}]}Any ideas what im missingFollowComment"

Answer: 0Found the answer to the question here:https://dataengineeringacademy.medium.com/how-to-query-postgresql-with-athena-1179e31f91ccCommentSharecjb85answered 2 years ago


Question: Failed to create environments

"I'm new in aws. so I recently deploy cloud 9 IDE for testing. I use cloud 9 IDE with the EC2 instance. My basic configuration is type  t3.small, Memory 2 GiB and vCPU 2.for Storage, I use EBS only.ErrorFailed to create environments: browserIDECloud9 could not connect to the EC2 instance. Please check your VPC configuration and network settings. Go to CloudFormation stackPlz, guide me on How to solve it with easy steps. any article which guides me to feel free to tell me.FollowComment"

Answer: "0Make sure the subnet the Cloud9 instance is deployed in is a public subnet. See: VPC settings for AWS Cloud9 Development Environments.You can also use Systems Manager to access a Cloud9 instance in a private network. See: Accessing no-ingress EC2 instances with AWS Systems ManagerCommentShareEXPERTkentradanswered a year ago0Hi! We're sorry you're facing an issue with Cloud9. Could you let us know which AWS region this occurred in and at what time (including timezone)?CommentShareaws-andreanswered a year ago0Hi,If that can help, try to create the environment from the cloudshell.Open cloudshell (upright symbol [[>_]] in the aws console landing page (before notifications, support and region selection))The shell takes about 1 minute to start, then execute that command:aws cloud9 create-environment-ec2 --name c9-test-env --instance-type t3.smallYou should have a response like this one :{    "environmentId": "15258..."}It works for me and it creates a cloud9 environment in the default vpc and a default public subnet (n.b. I tested in Ireland region with an admin user).Then go to the cloud9 service page in the aws console and open your new created environment.It takes also about 1 minute to start, and then you should be able to use cloud9 IDE.If this works, try again in the the console (https://docs.aws.amazon.com/cloud9/latest/user-guide/tutorial-create-environment.html)Hope this helps,CommentShareolivier-traininganswered a year ago"


Question: AWS Send-SSMcommand parameter variable

"I am trying to use a variable inside my command list like this$Commands = @('$DiskNumber = (Get-Partition -DriveLetter $driveletter).DiskNumber','Update-Disk -Number "$DiskNumber"','$Size = Get-PartitionSupportedSize -DriveLetter $driveletter','Resize-Partition -DriveLetter $driveletter -Size $Size.SizeMax')$Parameter = @{commands = $Commands}however when I run the ssm command, its not recognizing the $driveletter variable which i defined earlier in the script.if I put in letter instead of $driveletter, it worksany idea how to pass the variable inside the $commands list?FollowComment"

Answer: "0you need to wrap variable with backticks. I also ran into this issue and to troubleshoot you just go to the location of the script on the host and look at the script, you'll notice with how you have it, if you look at the script (c:\programdata\amazon\ssm{instanceId}\document\orchestration{orchestrationID}\awsrunPowerShellScript\0.awsrunPowerShellscript_script.ps1) the variable doesnt even show up left of the equal sign.so just wrap like `$DiskNumber` = (get-partition...).. Then open the script from path and look at it to troublshoot further.CommentShareVersaceDribzanswered 9 months ago0this is the error I am seeing. With or without back ticksGet-Partition : Cannot validate argument on parameter 'DriveLetter'. The argument is null. Provide a valid value for the argument, and then try running the command again.At C:\ProgramData\Amazon\SSM\InstanceData\i-0c02b9986c3c33057\document\orchestration\e4a27d78-e4ae-4fae-b45f-f528729b81dc\awsrunPowerShellScript\0.awsrunPowerShellScript\_script.ps1:1 char:45+ `$DiskNumber` = (Get-Partition -DriveLetter $driveletter).DiskNumber+                                             ~~~~~~~~~~~~    + CategoryInfo          : InvalidData: (:) [Get-Partition], ParameterBindingValidationException    + FullyQualifiedErrorId : ParameterArgumentValidationError,Get-Partition $Size = : The term '$Size =' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.At C:\ProgramData\Amazon\SSM\InstanceData\i-0c02b9986c3c33057\document\orchestration\e4a27d78-e4ae-4fae-b45f-f528729b81dc\awsrunPowerShellScript\0.awsrunPowerShellScript\_script.ps1:3 char:1+ `$Size` = Get-PartitionSupportedSize -DriveLetter $driveletter+ ~~~~~~~~~    + CategoryInfo          : ObjectNotFound: ($Size =:String) [], CommandNotFoundException    + FullyQualifiedErrorId : CommandNotFoundException Resize-Partition : Cannot validate argument on parameter 'DriveLetter'. The argument is null. Provide a valid value for the argument, and then try running the command again.At C:\ProgramData\Amazon\SSM\InstanceData\i-0c02b9986c3c33057\document\orchestration\e4a27d78-e4ae-4fae-b45f-f528729b81dc\awsrunPowerShellScript\0.awsrunPowerShellScript\_script.ps1:4 char:31+ Resize-Partition -DriveLetter $driveletter -Size $Size.SizeMax+                               ~~~~~~~~~~~~    + CategoryInfo          : InvalidData: (:) [Resize-Partition], ParameterBindingValidationException    + FullyQualifiedErrorId : ParameterArgumentValidationError,Resize-PartitionCommentSharedragoanswered 6 months ago"


Question: Writing data to Kinesis stream from PySpark

"A customer would like to process streaming data in Spark Streaming using PySpark and would like to output the results on a Kinesis stream.While PySpark does support reading data from Kinesis (http://spark.apache.org/docs/latest/streaming-kinesis-integration.html), I do not see any support for writing data to Kinesis.DataBricks has some documentation for creating a Kinesis sink for Spark (scala) (https://docs.databricks.com/spark/latest/structured-streaming/kinesis.html), but if I understand the documentation correctly this is based on the ForeachSink which is not supported in PySpark (http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#using-foreach)The foreach operation allows arbitrary operations to be computed on the output data. As of Spark 2.1, this is available only for Scala and Java. To use this, you will have to implement the interface ForeachWriter (Scala/Java docs), which has methods that get called whenever there is a sequence of rows generated as output after a trigger. Note the following important points.Has anybody come across a way to achieve this?FollowComment"

Answer: "0Accepted AnswerIn PySpark you can use forEachPartition and call Kinesis or any external API for that partition, or you can also use map and call out to Kinesis per record.# Approach 1: Per Partitiondef pushToKinesis(iterator):   print(list(iterator)[0]   #push to kinesis using boto3 APIsrdd.foreachPartition(pushToKinesis()) # Approach 2:  Per recorddef pushToKinesis(record):   #push to kinesis using boto3 APIsrdd.map(lambda l: pushToKinesis(l)) The blog post below uses Approach 2 to call Amazon Comprehend:https://aws.amazon.com/blogs/machine-learning/how-to-scale-sentiment-analysis-using-amazon-comprehend-aws-glue-and-amazon-athena/CommentShareAWS-User-4669487answered 5 years ago"


Question: Cognito USER_SRP_AUTH Flow using CLI

"I'm testing/learning about Cognito before I implement it in my app. I have somewhat of a handle on the USER_PASSWORD_AUTH authorization flow, which seems to be the simplest, but I don't want to use that in my app, but rather the USER_SRP_AUTH flow. Before I start coding in the JavaScript SDK, which I'm not very experienced in, I'd like to use the cli in a shell script to demonstrate to myself that I understand how to use this flow. I currently have a shell script that implements the USER_PASSWORD_AUTH flow to set the AWS_*  credential env vars using the cli on Linux, and now I'd like a similar script that uses the USER_SRP_AUTH flow. But, I'm not seeing any examples or tutorials that use the cli.My current current script that uses USER_PASSWORD_AUTH is below, simplified to replace potentially sensitive values with "<my sensitive value ...>" placeholders. Can you advise on the changes that would be needed to implement USER_SRP_AUTH?Thanks.#!/bin/ksh93## ----------------------------------------## source this script to set the AWS*## environment vars so the cli will operate## as the specified Cognito user## ----------------------------------------if [[ $# -ne 1 ]] ; then    print -u2 "Usage: . $(basename ${.sh.file}) cognito_user"    return 1fi## ----------------------------------------## args## ----------------------------------------cognito_user=$1## ----------------------------------------## initiate-auth## ----------------------------------------initiate_output=$(aws cognito-idp \                      initiate-auth \                      --auth-flow USER_PASSWORD_AUTH \                      --auth-parameters USERNAME=$cognito_user,PASSWORD=<my secret password> \                      --client-id <my user pool app client id> \                      --region us-east-1)if [[ $? -ne 0 ]] ; then    print -u2 "Failed to initiate auth."    return 1fiid_token=$(print "$initiate_output" |grep '"IdToken"' |awk '{print $2}' |tr -d '",')## ----------------------------------------## get-id## ----------------------------------------getid_output=$(aws cognito-identity \                   get-id \                   --identity-pool-id <my federated identity pool id> \                   --region us-east-1 \                   --logins cognito-idp.<us-east-1.amazonaws.com/<my user pool id>=$id_token)if [[ $? -ne 0 ]] ; then    print -u2 "Failed to get identity."    return 1fiidentity_id=$(print "$getid_output" |grep IdentityId |awk '{print $2}' |tr -d '"')## ----------------------------------------## get-credentials and set env## ----------------------------------------get_credentials_output=$(aws cognito-identity \                             get-credentials-for-identity \                             --region=us-east-1 \                             --identity-id=$identity_id \                             --logins cognito-idp.us-east-1.amazonaws.com/<my user pool id>=$id_token)if [[ $? -ne 0 ]] ; then    print -u2 "Failed to get credentials for identity."    return 1fiexport AWS_ACCESS_KEY_ID=$(print "$get_credentials_output" |grep AccessKeyId |awk '{print $2}' |tr -d '",')export AWS_SECRET_ACCESS_KEY=$(print "$get_credentials_output" |grep SecretKey |awk '{print $2}' |tr -d '",')export AWS_SESSION_TOKEN=$(print "$get_credentials_output" |grep SessionToken |awk '{print $2}' |tr -d '",')## ----------------------------------------## done (assume script is sourced)## ----------------------------------------returnFollowComment"

Answer: 1If I were you I would look in to AdminInitiateAuth instead of using USER_SRP_AUTH since it offloads some of the more complex pieces of the SRP calculations to Cognito.  See the answer here for more details.CommentSharetparrottanswered 10 months ago


Question: Why can't the root user change MFA rules on a rule?

"I'm in a cognito user pool configuration, and I tried to change the MFA requirement on the pool to allow TOTP.  It gave me this error:[AccessDeniedException] Failed to update MFA configuration for us-east-1_FLnPt9UKErequestId: 0660fb15-2cc4-4f63-9c8f-657b71b320d9time: Sat May 21 2022 16:16:18 GMT-0400 (Eastern Daylight Time)code: AccessDeniedExceptionmessage: User: arn:aws:iam::384426254369:root is not authorized to perform: iam:PassRole on resource: > arn:aws:iam::745623467555:role/cognito_sms_role because no resource-based policy allows the iam:PassRole > actionI'm confused because I am logged into the console as the root user, which should have permission to EVERYTHING.  I do not, however, really have a role named "cognito_sms_role."  Is that the problem?  If so, how do I fix it?FollowComment"

Answer: "0Accepted AnswerThe AWS account numbers are different in the two resources: 384426254369 for the root account user, and 745623467555 on the role.This means that your root user doesn't have permissions to change the role, since it's outside the account. Does that make sense for your setup?Given iam:PassRole is a permission-only IAM action, it might also be that the root user account doesn't have it (it's not something I've ever tried to do).You will likely have to assume a role in the Cognito role's account, so that you can make changes to it.CommentSharerowanuanswered a year ago0I'll check that out.  I shohuldn't have two accounts so I'm not even sure why that should be!CommentSharewz2banswered a year ago"


Question: ECS: Random container failed to start - no log

"Hello,We encountered troubles with ECS on Docker on EC2.Some tasks stay a long time in provisionning state while an EC2 is available and others tasks with the same docker image and parameters succeed to start. Then, the status goes from PROVISIONNING to STOPPED with reason : "Task failed to start".I have no logs, or info on the containers in ECS console and Cloudwatch, and did not find any tips in the EC2 logs...Any idea on why we have this behaviour ? or a way to have more traces to investigate ?ThanksValentinFollowCommentMichael_F EXPERTa year agoCan you run aws ecs describe-tasks --tasks <task-id> --cluster <cluster-name> and see what it reports as the stopped reason? Task status will remain available for about an hour after the task stops.Share"


Question: How are the assumed thresholds for Amazon Rekognition Custom Labels computed?

"In the Per label performance section of my model's evaluation summary in the Amazon Rekognition console, I can see an Assumed threshold metric. How are those metrics computed?When I collected all the confidence levels computed for a given class in the test dataset, I could see that the minimum confidence level for my KO label was 0.87, while the minimum confidence level for the OK label was around 0.30. Is there some kind of heuristics going on (for example, 90% of the minimum confidence level observed for a class?), or is there a precise mathematical definition behind the assumed thresholds?Also, is there a way to get the assumed thresholds programmatically with the Amazon Rekognition Custom Labels API?FollowComment"

Answer: "0Accepted AnswerThe assumed threshold for each label is the value above which a prediction is counted as a true or false positive. The metric is calculated based on the best F1 score achieved on the test dataset during model training. For more information, see the Assumed threshold section in the Rekognition Custom Labels guide.To get the assumed thresholds programmatically, use the Rekognition SDK. The assumed thresholds are listed in the summary file output.CommentShareEXPERTJulia Beckanswered 3 years ago"


Question: VPC ACLS - ICMP Rules vs Documentation

"https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html states:If the maximum transmission unit (MTU) between hosts in your subnets is different, or your instances communicate with peers over the internet, you must add the following network ACL rule, both inbound and outbound. This ensures that Path MTU Discovery can function correctly and prevent packet loss. Select Custom ICMP Rule for the type and Destination Unreachable, fragmentation required, and DF flag set for the port range (type 3, code 4).This doesn't fit with what the console offers. I can see only Destination Unreachable as an option and none of the others.What's the correct setup? Am starting to think the VPC ACL stuff is just broken in terms of web console - encountering bugs in validation & it lacks expected features like copying an existing ACL to new?FollowComment"

Answer: "2It's not written clearly in the documentation.  For "Type" select "Destination Unreachable", and for "Port range" select "fragmentation required, and DF flag set".CommentShareEXPERTskinsmananswered a year ago1Accepted AnswerWhen editing a Security Group you can select Custom ICMP in the Type column. That then allows you to select Destination Unreachable in the Protocol column. From there you can select Fragmentation needed in the Port range column.The reason this is a little odd is because most other protocols use a port to determine the application that is being allowed. With ICMP it is a Type and a Sub-type so the ICMP sub-types (of which Fragmentation needed but DF bit set) is one.CommentShareEXPERTBrettski-AWSanswered a year ago"


Question: Visual Studio AWS Toolkit CodeWhisperer stopped providing multiple options for suggestions

"Hi team,My VS Code Codewhisperer could give me only 1 code suggestion. It did not provide nor allow me to choose other alternative suggestions via using left/right cursors.When I placed the same comment on other IDE tool such as IntelliJ IDEA, it would provide me multiple suggestions and I could use cursor key to select the appropriate one.Similar observation also being reported by my customer. May I know if there is any hidden thing that I activated on VS Code to stop providing me multiple suggestions in its feedbacks?Many thanks.RaymondFollowComment"

Answer: "0I have exactly the same problem...CommentShareLaurent Mirguetanswered 19 days ago0there was a regression in VSCode itself that broke the inline suggestion feature.We started observing this as a part of v1.78. unfortunately the bug you ran into impacts every plugin using the inline completions API (including Copilot, actually).Workaround is to either configure proxy in vscode or downgrade to vscode 1.77 until vscode releases a fix (expected for 1.79)CommentSharevamsianswered 7 days ago"


Question: CVE-2021-23017 issue for services running behind Network load balancer

"We have a couple of services running on EKS fargate accessible from internet via network load balancers.We have recently conducted external penetration testing as part of our compliance process. It identified that the network load balancer is using nginx v1.20 which is subject to a security issue CVE-2021-23017 https://alas.aws.amazon.com/cve/html/CVE-2021-23017.html#score-breakdown . This issue has been fixed in v1.20.1.Is there a patch that has been applied to network load balancer to fix CVE-2021-23017 or any mitigation we can do to overcome this from client (our) side?Could anyone provide any pointers for me to gather more information in order to make an assessment of the severity level?I searched the forum but only found a few posts about this issue with regards to elastic beanstalk, nothing about the network load balancer.Thank you.Edited by: yybc9a3 on Nov 26, 2021 3:27 AMEdited by: yybc9a3 on Nov 26, 2021 3:31 AMFollowComment"

Answer: "0Turned out that it was a massive overlook.... We did run a small nginx container after netwrok load balancer....Problem solved.Edited by: yybc9a3 on Nov 26, 2021 6:24 AMCommentShareyybc9a3answered 2 years ago"


Question: About dms  task fail

"Actually i have created few dms task but after migrate 145 table its shown error.the error is :-"Last Error Fatal error has occurred Task error notification received from subtask 2, thread 0 [reptask/replicationtask.c:2880] [1022505] Error executing source loop; Stream component failed at subtask 2, component st_2_T3EBVOARFJZOJNHCJNVOTQ6P7U7C67W5KCW4INI; Stream component 'st_2_T3EBVOARFJZOJNHCJNVOTQ6P7U7C67W5KCW4INI' terminated [reptask/replicationtask.c:2888] [1022505] Stop Reason FATAL_ERROR Error Level FATAL"After the analysis its showing"error code <LimitExceededException>: This request would have caused the ReadCapacityUnits limit to be exceeded for the account in ap-south-1. Current ReadCapacityUnits reserved by the account: 79900. Limit: 80000. Requested: 200. Refer to the Amazon DynamoDB Developer Guide for current limits and how to request higher limits. [1001789]  (ddb_table_actions.cpp:91)"that error.Please tell me how i solve this problem...FollowComment"

Answer: "1You're hitting the soft limit for DynamoDBs service quota. Check out the service quota page here.You can use the service quota console, the AWS API and the CLI to view your throughput default quotas and request quota increases for the adjustable quotas, when needed. You can also request quota increases by cutting a ticket to AWS support. This info is also linked on the above page.Sounds like you're hitting the 80,000 read capacity units and 80,000 write capacity units, but that's just a soft limit. Go ahead and request a an increase and retry it.CommentShareAWSJoeanswered a year ago"


Question: Assign Multiple Values to a Single key tag In AWS

"I will like to know if it is possible to assign multiple values to a single key tag in AWS, especially for EC2 instances.Kindly advise.FollowComment"

Answer: "0No - single tag, single value. However you might consider using (say) a comma or semicolon as a delimeter and then parsing the tag value. Note that you shouldn't do this for billing tags.What do you need multiple tag values for?CommentShareEXPERTBrettski-AWSanswered a year agoNurudeen Kamilu a year agoThanks for your comment. I have an instance hosting more than one application, hence the instance is to have two application IDs for the key name application.ShareGK a year agoBut you can do Key: Application_ID1: ABCD and Application_ID2: ABCDShareShlomo Dubrowin EXPERTa year agoWhile it is possible overload a Tag as mentioned in the other comments, it is highly not recommended https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf (see page 11)Share0HelloYou can't simply, Each key must have Uniq Name, But you can do another way for exampleName-1  Project1Name-2 Project1Name-3 Project1CommentShareGKanswered a year ago0You can have a tag with multiple value field of the tag. An example is mentioned in our documentation.The example is with the tag 'team', where more than one team is associated to that resource.team = Engineering:QAThe delimiter is ":", but could be other valid separators. Using this to control for access takes special consideration on how you craft your policy for those values.CommentShareStephenHeverin_AWSanswered 4 months ago"


Question: instance terminate and how to reduce remove bills since i used only free trail.

"Hi I am a student,I tried to use free tire from aws services 2 years ago but i didn't used it and I didn't know that it will cost.If i knew that i won't even register for free tire.I am unable to pay that bill. plese help me to solve the issue.how can I delete the activated services and how can I remove the costs that I have not even used?FollowComment"

Answer: 0Please go to support center and open "Account & billing" support caseCommentShareAWS-yl-lfcanswered 3 months ago


Question: migration from google cloud

"Hello everybody,I am a google cloud user and planning a migration to AWS. The structure to be migrated is relatively simple and well documented and I would like to know who I can contact for a reliable cost estimate to compare with the current costs on google cloud. thank youFollowComment"

Answer: "1Accepted AnswerHi,I would also like to add on top of the previous answer - please feel free to reach out to your account manager (if you have a support plan) - they can assign a Solutions Architect to your migration project. Having a solutions architect on the project along your side would help you to ensure that the best and cost effective resources are used and it is well architected in AWS - Hope this helps - CheersCommentShareEXPERTNIRUPAM_danswered 10 months ago1The AWS Pricing Calculator (https://calculator.aws) might suit your needs - it allows you to create a cost estimate for the services you'll be using.CommentShareEdanswered 10 months ago"


Question: Cognito - Azure AD SAML response

"Hi,We have **Azure AD as our identity provider **and successfully federated with Cognito user pool. This helps us in authenticating the users registered in Azure AD through Cognito hosted UI.As I understand, during the federation process, Azure AD sends a SAML token to the https://<domain_prefix>.auth.<region>.amazoncognito.com/saml2/idpresponse endpoint for Cognito to issue the id and access tokens.Can we intercept this SAML response/payload through any of the Cognito lambda triggers or other method?The need is to exchange this SAML token from Azure AD for an OAuth2 token issued by Azure AD to access other protected web api's in Azure.The Cognito issued tokens(id token, access token) are not acceptable by AzureAD protected web api's for the reasons being issued by a different provider.They work fine for the AWS resources though which is understandable.ThanksMJFollowComment"

Answer: "0Hi,It is not possible to intercept or access the original SAML response that Azure AD sends to Cognito idpresponse endpoint. This SAML response is validated by Cognito and attributes in the assertion are mapped to cognito attributes as you configured them. Is it possible to send this oauth2 token as an attribute inside the SAML assertion and map it to a custom attribute in Cognito?CommentShareEXPERTMahmoud Matoukanswered a year agomj a year agoThanks for confirming that the SAML response that Azure AD sends to Cognito idpresponse endpoint cannot be intercepted.Just was looking through the Azure AD SAML attribute mappings but it does not list either the idToken or accessToken that can be mapped as an attribute.We can choose from the attributes like first name, last name and so on individually but cannot have the token itself as an attribute in the SAML mapping.As you mentioned above, in case that was possible, it would be then mapping that as a custom attribute in CognitoShare"


Question: Instance Stuck in Stopping

There is not an option to "Force Stop" instance.Instance has been in "stopping" status for 10 minutes.Please advise on next steps.thank youFollowComment

Answer: "0Hi User,Check out this aws documentation on how to force stop an instance wich is stuck in the stopping step:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesStopping.htmlFollowing cli command force stops an instance:aws ec2 stop-instances --instance-ids i-0123ab456c789d01e --forceSincerelyHeikoCommentShareHeikoMRanswered 4 months agoEXPERTkentradreviewed 4 months ago"


Question: How can I terminate Amazon SageMaker RunInstance?

"Hello.I had an unexpected billing, and that was because the SageMaker RunInstance was still running. (Especailly DataWrangler ; see the screenshot below.)I didn't know how to terminate, so I contacted to the AWS support center.I followed all the instructions they gave,  which means that I deleted endpoints / models / notebook instances / s3buckets /  cloudwatch log groups.But after 24 hours of monitoring, AWS support center said that the SageMaker RunInstance is still running.They gave me the same instructions and one additional instruction : stop the training jobs (https://docs.aws.amazon.com/sagemaker/latest/dg/studio-tasks-stop-training-job.html)Firstly I checked  endpoints / models / notebook instances / s3buckets /  cloudwatch log groups again, and I found nothing in those tabs.Secondly I tried to stop the jobs, but I have deleted the domain since I tried to delete everything - thought it would be better (Maybe that was a fault) - So I created the domain again with quick start option (to get access to the studio), and got into the studio to terminate jobs. But all of the jobs are already completed. There's nothing with activated 'stop training jobs' button.So I followed the instruction, but there was nothing I could do.I really want to STOP this SageMaker RunInstance, but I don't know why it's still running. What should I do..? Please help me.FollowComment"

Answer: "0Be sure that you're connected to the correct AWS region.Follow the documentationt to stop Data Wrangler and you Studio Notebook.Hope it helpsCommentShareAlexanswered 10 months agorePost-User-1585177 7 months agoI confirm the suggestion, I create "for fun" a database and the region selected wasn't mine region (I don't know way, by the way), so I was not able to find wich service was generating costs IN MY REGION!Share"


Question: S3 Replication FAILED on some objects

"I have setup Cross Region and Cross Account S3 replication. Both source and destination buckets are setup with the same CloudFormation template (same permissions and class), the only difference being the destination doesn't have a replication rule. Here is the replication rule on the source.{  "Role": "arn:aws:iam::{SourceAccount}:role/{SourceBucket}",  "Rules": [    {      "Status": "Enabled",      "Priority": 1,      "DeleteMarkerReplication": { "Status": "Disabled" },      "Filter" : {},      "Destination": {          "Bucket": "arn:aws:s3:::{DesitnationBucket}",          "Account":"{DesitnationAccount}",          "AccessControlTranslation":{            "Owner":"Destination"        },          "Metrics": {            "Status": "Enabled"        }      }    }  ]}The replication works. However, out of the about 1 000 000 objects I have, 66 000 show the status as failed to replicate. When I run an S3 batch job on the bucket to replicate, it fails as well, and from the output, it states the error as "SrcObjectNotEligible" "Object is not eligible for replication. Object or object tags do not match replication configuration."As you can see in the above rule, I have no filters applied, and all objects should replicate. I do have versioning and object lock enabled on the bucket, and the objects are not encrypted (this is not preferred. However, the source that uploads to the bucket requires this to be off.)As per https://aws.amazon.com/premiumsupport/knowledge-center/s3-list-objects-failed-replication/#:~:text=Amazon%20S3%20inventory%20report,-Amazon%20S3%20inventory&text=To%20find%20objects%20that%20failed,source%20bucket%2C%20which%20triggers%20replicationThe only way to resolve a failure is "To find objects that failed replication, filter a recent report for objects with the replication status of FAILED. Then, you can initiate a manual copy of the objects to the destination bucket. Or, you can re-upload the objects to the source bucket, which triggers replication."As I would like the status to be "Replicated" I am not sure a manual copy will work. I downloaded the file from S3 and proceeded to overwrite the existing one. It does start the process of replicating it again but also fails.Does anyone have other options for me?FollowComment"

Answer: "0Hi THerePlease take a look at the list of reasons why an object wouldn't get replicated here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-what-is-isnot-replicated.html#replication-what-is-not-replicatedCommentShareEXPERTMatt-Banswered 5 months agoBobk 5 months agoI have checked those out already, and none seem to apply. There are also no lifecycle policies applied to either source or destination buckets. The source and destination have also been the same since the first setup.ShareMatt-B EXPERT5 months agoCan you do a "aws s3api head-object --bucket <bucket-name> --key <full key prefix and name>" command on one of the failed objects and post the output?  If you need help with that CLI command see https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/head-object.htmlShareBobk 5 months agoHere is the output from 1AcceptRanges: bytesContentLength: 1978ContentType: application/octet-streamETag: '"d3a7e36d53e05328a4560893fdc7e00b"'LastModified: '2022-09-29T20:17:10+00:00'Metadata:  arch-block-size: '1894'ObjectLockLegalHoldStatus: OFFObjectLockMode: COMPLIANCEObjectLockRetainUntilDate: '2023-01-20T20:06:48+00:00'ReplicationStatus: FAILEDVersionId: AHYyrnOSStfFBsOfT8WMOatsVEGX0M97Share"


Question: CFN Stack stuck at UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS

"AWS Team, Asking help in **deleting **stack "arn:aws:cloudformation:ap-southeast-1:829635768155:stack/StagingEzAPIGW/0ff480c0-d1b1-11ea-97a7-0243768bbe4a".Nested stacks are stuck at UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESSFollowComment"

Answer: "0I DELETED the root stack, keeping the nested stacks -- and today, the nested stack status shows the rollback was COMPLETED and I was able to delete the nested stack individuallyCommentShareathananswered a year ago"


Question: Calling create_passthrough_verify_fcn returns an error

"Has anybody seen the following problem in calling create_passthrough_verify_fcn?We're trying to implement a custom password verify function by calling the following from the master user account:beginrdsadmin.rdsadmin_password_verify.create_passthrough_verify_fcn(p_verify_function_name => 'VERIFY_FUNCTION_SYS',p_target_owner             => 'MASTER',p_target_function_name => 'VERIFY_FUNCTION');end;VERIFY_FUNCTION is our custom password verify function and has been compiled successfully.The follow error is returned:ORA-20199: Error in rdsadmin_password_verify.create_passthrough_verify_fcn. ORA-20002: Unable to create the unprivileged user. Please contact AWS Support.ORA-06512: at "RDSADMIN.RDSADMIN_PASSWORD_VERIFY", line 70ORA-20002: Unable to create the unprivileged user. Please contact AWS Support.ORA-06512: at line 2Calling the simpler create_verify_function is OK but we need create_passthrough_verify_fcn to call our own custom function.FollowComment"

Answer: "0Hi there,This can happen in situations where you have defined a restrictive password verification function on the "DEFAULT" profile and then run something that uses automation to create a new user in that profile. You might try setting the password verification function back to null for "DEFAULT", then retry creating the passthrough function, then re-enable password verification for "DEFAULT".Hope this helps. Thanks!MichaelCommentShareAWS-User-9938760answered 4 years ago0After setting the default profile to null, it is now working.Thanks a lot.CommentSharesnlauanswered 4 years ago"


Question: Glue Spark Job is incomparable slower and more expensive than Athena

"I've tried to use Glue Spark Job for very basic partitioning over GZIP JSON data about 50GB.The reason for trying Glue Job is my data could have more than 100 partitions and it is not really convinians to do it in Athena.The code is almost from AWS template.import sysfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom awsglue.context import GlueContextfrom awsglue.job import Jobfrom awsglue.dynamicframe import DynamicFramefrom pyspark.sql.functions import concat, col, lit, substring## @params: [JOB_NAME]args = getResolvedOptions(sys.argv, ['JOB_NAME'])sc = SparkContext()glueContext = GlueContext(sc)spark = glueContext.spark_sessionjob = Job(glueContext)job.init(args['JOB_NAME'], args)my_partition_predicate = "partition='main' and year='2022' and month='12' and day='17'"datasource0 = glueContext.create_dynamic_frame.from_catalog(database = "my_db", table_name = "my_table", push_down_predicate = my_partition_predicate, transformation_ctx = "datasource0", additional_options = {"recurse": True})datasource0 = datasource0.toDF()datasource0.printSchema()datasource0 = datasource0.withColumn("od", concat(substring('originalrequest',9,3), lit("_"), substring('originalrequest',12,3)))datasource0 = DynamicFrame.fromDF(datasource0, glueContext, "datasource0")datasink4 = glueContext.write_dynamic_frame.from_options(    frame = datasource0,     connection_type = "s3",     connection_options = {"path": "s3://my_b/glue4",  "partitionKeys": ["od"] , "compression": "gzip"},     format = "json"    )job.commit()The job were executed with 60 DPUs and after 20 minutes it timed out... This failed execution is cost $8.8.Meanwhile, totally the same job were done in Athena in about 2 minutes and cost $0.25.Am I doing something wrong, or Athena (Presto) is leaps ahead of Spark in terms of speed and cost effectiveness?FollowCommenttjtoll 4 months agoIn both cases are you writing to JSON and compressing as GZIP?ShareSmotrov 4 months agoYep. Output is exactly the same. Partitioned JSON with GZIP compression.Share"

Answer: "1Athena requires less effort to optimize the job, Glue since it allows so many ways of using it, doesn't self tune itself so well yet.Before jumping to conclusions, check if your job is making good use of those 60 DPUs (using SparkUI or Glue metrics), maybe the data is unbalanced at source.For that specific use case DynamicFrame is not as efficient as DataFrame, especially doing the partitioning.In addition, you are converting forth and back, which often degrades performance.Since your data is not dynamic, I suggest you do the same only with DataFrame and compare (e.g. spark.sql("your query").write.partitionBy("od").json("your path") )Finally, Glue 4 has significant performance improvements, I encourage you to try it out on that version.CommentShareGonzalo Herrerosanswered 3 months agoSmotrov 3 months agoThanks for getting back to me! It seems like the learning curve for Glue and Spark is quite steep, almost like climbing Everest.I was under the impression that DynamicFrame was a crucial concept in Glue Jobs, which is why I used it. Unfortunately, I'm having trouble getting Spark SQL to recognize any columns other than partitioned ones.Even after runningspark.sql('use default')spark.sql('describe my_table').show()only the partitioned columns are consistently displayed. I've spent half a day poring over output stacks and documentation with no success.As a result, I'm unable to benchmark Spark DataFrame since I just can't get my data column to show up.ShareGonzalo Herreros 2 months agoDynamicFrame is very relevant when the data is schema is not well defined but not necessarily on all cases. For catalogs it's easier to use SparkSQL. If Spark doesn't see those columns, it means the table is not well defined and Athena shouldn't see them either. How was the table created? It's better if you let tools create tables for you (e.g. Athena CTAS or Spark saveAsTable())Share0You may be doing something wrong, but Athena is generally faster and more cost-effective than Glue for ad-hoc querying of data in S3. Athena uses Presto, which is a high-performance SQL query engine optimized for reading and aggregating large data sets from S3. On the other hand, Glue is a fully-managed ETL service that is intended for data extraction, transformation, and loading, not for ad-hoc querying. So while it's possible to use Glue for basic partitioning, it may not be the most efficient or cost-effective option, especially if your data is partitioned and your queries are simple.CommentShareDivyam Chauhananswered 4 months agoSmotrov 2 months agoAnother try without conversion to data frames. 17GB of data processed during 40minuted on 50 DPU. All workers loaded ~100%.So the effective speed of processing is 17000/40/50 = 8,5Mb/m per DPU. Just would like to know is it normal processing speed  for AWS Glue? Or there is a spate for tuning? To me it is kind of slow-ish...my_partition_predicate = "partition='main' and pos='DE' and pcc='YYYY' and year='2023' and month='3' and day='8'"source_dyf = glueContext.create_dynamic_frame.from_catalog(database = "estr_db", table_name = "estr_warehouse", push_down_predicate = my_partition_predicate, additional_options = {"recurse": True})def add_od(record):    od = record['originalrequest'][8:11] + "_" + record['originalrequest'][11:14]    record['od'] = od    return recordsource_dyf = source_dyf.map(add_od)# Set a suitable number of partitions and repartition the DynamicFramenum_partitions = 100  # Adjust this value based on your data size and cluster resourcessource_dyf = source_dyf.repartition(num_partitions)# Write the output DynamicFrame to an S3 bucket in JSON format with Gzip compressionglueContext.write_dynamic_frame.from_options(    frame=source_dyf,    connection_type="s3",    connection_options={        "path": "s3://mybucket/temp/booo",        "partitionKeys": ["od"],        "compression": "gzip"    },    format="json",    transformation_ctx="sink")Share"


Question: does api gateway parse or validate XML?

"I created a Open API spec to define the API Gateway endpoints which includes a definition of the XML used in the POST body.  I guessed that API Gateway would parse or at least validate XML POSTs, but after playing with it, seems like all you can do is pass the XML thru to a lambda function to parse it?FollowComment"

Answer: 0Accepted AnswerNo. API Gateway only validates JSON schemas. Same from transformation.CommentShareEXPERTUrianswered 4 months ago


Question: StackSet stuck in Stopping state

"Hello AWS support,I have a stack-set stuck in Stopping status and it doesn't want to time out for few days now. Could you please fail the following operation ID ca498374-b491-4786-af92-28e4c8dba0dc, so I can modify the stack set? Problem seems similar to https://forums.aws.amazon.com/thread.jspa?messageID=808469This has happened to me before, but the operation timed out after few hours. This is an issue with Lambda custom resource not executing correctly in target account, which causes stack-set in administrator account to keep running without timing out, but stopping it manually hangs the operation. No other operation (Console/CLI/Boto3) is allowed due to OperationInProgressException error code. There is no option to set timeout on stack instances, neither nested stacks are supported.Thanks in advance!scoobezFollowComment"

Answer: 0duplicateCommentSharescoobezanswered 2 years ago


Question: "MWAA Metadatabase not initialized - log, dag, triggers relations do not exist"

"we have created the MWAA env, ECR, SUBNETs, private SGs, VPC - we see NICs attached to the SGaccess via UI is given - we can login and open UI via web browser.on /home access we see an Ooops screenbrowsing other menus (i.e. Security, Docs) work - all the other Dags, Admin, Browse menus result in an Ooopsno log other than webserver showing some feedback:on /home :psycopg2.errors.UndefinedTable: relation "dag" does not existhow can one run a db init ?we have tried it using via REST API (https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-cli-command-reference.html) - which works for certain commands but the db init returns with "TypeError: 'int' object is not iterable"any help is welcomewhat is the need of the security group rules (egres, ingres) ?FollowComment"

Answer: 1we fixed it in the end while going through the aws mwaa exampl line by line - the problem we found missing in our setup was a self pointing rule on the mwaa security groupwe have re-created the env and all works as designedCommentShareAWS-User-2457839answered a year ago


Question: How to remove Global Accelerator Endpoint from Group programmatically?

"I can add an endpoint to a group using aws globalaccelerator update-endpoint-group, but how do I remove an endpoint from an endpoint group?? Closest thing I can find is remove-custom-routing-endpoints but I'm using a Standard GA, not an custom one.FollowComment"

Answer: "1Accepted AnswerThe update-endpoint-group or UpdateEndpointGroup API can be used to both add or remove Endpoints. The contents of the property --endpoint-configurations is a list which must contain the end state desired endpoints ie if you wanted to add one endpoint you need to include the current endpoints and the new endpoint, if you want to remove an endpoint you supply an --endpoint-configurations list without the endpoint you want to remove and this will then remove the endpoint from the groupYou might want to consider using cloudformation or other IaC tool as you might find easier to manage adding/removing operationsCommentShareEXPERTPeter_Ganswered a year agoSmellyspice a year agoThank you for this most complete and excellent answer! To delete all endpoints in a group, calling update-endpoint-group with an empty --endpoint-configurations clears all entries -- exactly what I was looking for!It would be nice if this removal process was documented in the cli/api docs, even if just as an example.Share"


Question: Spot Fleet Instance could not be stopped

I have a spot instance(with stop behavior) which I am trying to stop but the console gives error that "Instance could not be stopped because it belongs to fleet".ScreenShotAny idea what I am missing.Thanks in advance.FollowComment

Answer: "1Interruption Behaviour is what AWS will do when your spot instance get interrupted, not what you can do on the instance.Also, when you use Spot Fleet, the instances are managed by the Fleet. So you cannot stop the instance manually.CommentShareRichardFananswered a year ago1Hello,The AWS Spot Fleet controls the instances lifetime and it automatically manages them on you behalf.Since your fleet includes Spot Instances, Amazon EC2 can attempt to maintain your fleet target capacity as Spot prices change.An EC2 Fleet request of type maintain or request remains active until it expires or you delete it.When you delete a fleet of type maintain or request, you can specify whether deletion terminates the instances in that fleet.Otherwise, the On-Demand Instances run until you terminate them, and the Spot Instances run until they are interrupted or you terminate them.To get more information about Spot Fleets, please refer to Spot Fleet configuration strategiesCommentShareLungaanswered a year ago"


Question: Are we working on improving the license plate numbers and VIN Numbers recognition from images?

I tried several license plates of different states of US. The results are very inconsistent. Sometimes  all the words even are not recognised.I have a use case to extract only the license plate numbers of all states of US from image as text.Are we already working on such requirement?FollowComment

Answer: 0You may want to pose this question to the internal answers board.  This information would not be exposed externally via re:Post.CommentShareEXPERTChris_Ganswered a year ago


Question: Error running python:3.8-alpine3.14 in docker AWS lambda

For My below code I am trying to run alipne version due to higher execution time mynormal python version.I got error: "START RequestId: 191b606b-ee6b-4dfe-8b02-7fccf7689718 Version: $LATESTRequestId: 191b606b-ee6b-4dfe-8b02-7fccf7689718 Error: exec: "generate_worldcloud.generate_wordcloud_handler": executable file not found in $PATHRuntime.InvalidEntrypointEND RequestId: 191b606b-ee6b-4dfe-8b02-7fccf7689718REPORT RequestId: 191b606b-ee6b-4dfe-8b02-7fccf7689718Duration: 0.86 msBilled Duration: 1 msMemory Size: 512 MBMax Memory Used: 2 MB"My docker file contains :python3.8 lambda base imageFROM python:3.8-alpine3.14# python3.8 lambda base imageFROM public.ecr.aws/lambda/python:3.9-x86_64installing dependenciesCOPY py3-nltk-3.8.1-r0.apk /tmp/COPY py3-wordcloud-1.8.2.2-r1.apk /tmp/COPY py3-pypdf2-2.10.9-r0.apk /tmp/RUN apk add --no-cache /tmp/py3-pypdf2-2.10.9-r0.apkRUN apk add --no-cache /tmp/py3-nltk-3.8.1-r0.apkRUN apk add --no-cache /tmp/py3-wordcloud-1.8.2.2-r1.apkcreate a directory for the NLTK dataRUN mkdir /tmp/nltk_datacopy the NLTK data to the containerCOPY nltk_data /tmp/nltk_dataset the NLTK_DATA environment variableENV NLTK_DATA=/tmp/nltk_dataCopy all .py files to containerCOPY *.py ./COPY generate_worldcloud.py ./setting the CMD to your handler file_name.function_nameCMD ["generate_worldcloud.generate_wordcloud_handler" ]i tried installing using requirements file but it doesn't work so i downloaded APK copied and installed still my lambda function doesn't work. I don't understand why!FollowComment

Answer: 0The problem comes from the fact that the image you are using does not contain the necessary code that Lambda runtime expects in the image.Please follow these instruction to build a container Lambda from alternative images than the one provided by AWS.https://docs.aws.amazon.com/lambda/latest/dg/images-create.html#images-create-from-altCommentShareEXPERTMassimilianoAWSanswered a month ago


Question: MySQL upgrade to version 5.7

I want to upgrade my Aurora RDS MySQL version 5.6 database to 5.7 (and eventually on to 8.0). But in the Modify section I do not have the option to upgrade to version 5.7 (5.7 does not appear in the version options). I suspect it has to do with my DB Instance Class - which is a db.t2.small. If this is in fact the issue- does anyone know the smallest DB Instance Class that allows for MySQL version 8.0?FollowComment

Answer: "0AWS support pointed out Aurora MySQL does not currently support version 8.0CommentShareJohnEGManswered 4 years ago0Hello,I understand that you wish to migrate your Aurora MySQL 5.6 database  to Aurora MySQL 5.7 and then eventually to Aurora MySQL 8.0.As  you know ,the MySQL version 8.0 is only available for RDS MySQL and the engineering team is already working towards getting the release for the Aurora MySQL v.8.0.I would like to inform you that MySQL 8.0 is in the roadmap for Aurora MySQL, however we do not have an ETA for the release.Having said that, all AWS new features and bug fixes are announced at the following links. Please keep an eye on them.[+] https://aws.amazon.com/new/[+] https://aws.amazon.com/blogs/database/[+] https://aws.amazon.com/releasenotes/Regarding upgrading your database from Aurora MySQL 5.6 to 5.7, I would like to inform you that you will not be able to see the 5.7 version in the Modify DB instance/cluster page.I would like to inform you that Manual Snapshot restore is the best way for you to migrate to the new aurora-mysql 5.7 version. I reviewed your account and I could see that you have aurora instances of 5.6.10aPlease refer the below steps to upgrade from Aurora 5.6.10a to 5.7.12:1. Select the Aurora instance you want to upgrade, click 'Instance Action's and then select 'Take Snapshot'.2. In the 'Snapshots' section, select the snapshot created in the previous step, click on 'Snapshot Actions' and then select Restore Snapshot.3. In the DB Engine field, select the option 'Aurora-mysql'. In 'DB Engine Version' field, select the required 5.7 version.4. Select the appropriate DB instance class (To avoid any performance issues, it is recommended to have at least the same instance type as Aurora instance 5.6.10a)5. Enter the other field appropriately and the click on 'Restore DB instance'Once your Db instance is updated just switch over to the new endpoint at your convenience.Please find the complete release notes along with their features in the below document:[+] Database Engine Updates for Amazon Aurora MySQL 2.0 - https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.20Updates.htmlHaving said this, I strongly recommend you to do the upgrade in a test environment. Perform the necessary tests on the new Aurora 5.7 instance, and if it suits your application workload, you may go ahead and do the upgrade in your production environment.I hope you find the above information helpful.Have a great day ahead!CommentShareAshikaAWSanswered 4 years ago"


Question: Even bridge how it works

"Hi,Is even Bridge is Truly FIFO. Are events in event bridge in FIFO manner or Random?RegardsJoFollowComment"

Answer: "1Accepted AnswerEventBridge does not guarantee the order of the events. It is not random, but they can get out of order sometimes.CommentShareEXPERTUrianswered a year agoRodney Lester a year agoEvent Bridge is not a queue, where FIFO or LIFO would apply.Share"


Question: add only different file to s3 archive

"I have an s3 bucket. I store there zip archive.I download this zip archive, unzip it, change some files then I zip it and upload back to s3.I'm just curious is it possible to upload not full zip archive but only files that was changed?for example I download zip from aws, change some file and I upload only this files, not full archive.Or is there any other way to optimize this scenario?FollowComment"

Answer: "0Hello, you can use CLI to make incremental updates to your zip archive. You can use the aws s3 sync command to download the zip archive, make the changes, and then upload only the modified files back to S3. Please refer to this man page https://docs.aws.amazon.com/cli/latest/reference/s3/sync.htmlAnother way-through, using  a versioning feature for your bucket. This allows you to store multiple versions of an object and easily roll back to a previous version if necessary. Check this for more information https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.htmlIt's important to choose the one that best fits your specific use case and requirements.CommentShareOlwi Mamdouhanswered 4 months ago"


Question: Athena federated query on PostgresSQL

"Hi I am trying to execute queries on a postgresql database I created in AWS.I added a data source to Athena, I created the data source for postgresql and I created the lambda function.In lambda function I set:default connection stringspill_bucket and spill prefix (I set the same for both: 'athena-spill'.  In the S3 page I cannot see any athena-spill bucket)the security group --> I set the security group I created to access the dbthe subnet --> I set one of the database subnetI deployed the lambda function but I received an error and I had to add a new environment variable created with the connection string but named as 'dbname_connection_string'.After adding this new env variable I am able to see the database in Athena but when I try to execute any query on this database as:select * from tests_summary limit 10;I receive this error:GENERIC_USER_ERROR: Encountered an exception[com.amazonaws.SdkClientException] from your LambdaFunction[arn:aws:lambda:eu-central-1:449809321626:function:data-production-athena-connector-nina-lambda] executed in context[retrieving meta-data] with message[Unable to execute HTTP request: Connect to s3.eu-central-1.amazonaws.com:443 [s3.eu-central-1.amazonaws.com/52.219.170.25] failed: connect timed out]This query ran against the "public" database, unless qualified by the query. Please post the error message on our forum  or contact customer support  with Query Id: 3366bd80-143e-459c-a4da-5350b5ab4a77What could be causing the problem?Thanks a lot!FollowComment"

Answer: "0insert the communityCommentSharerePost-User-7026277answered a year agoamul a year agoI cannot understand, Did I make a mistake during posting or what else?Share0how I setup account with amazon?CommentSharerePost-User-7026277answered a year ago"


Question: Migrating Aurora Postgres to RDS Postgres

"Hi there, Is there a way to Migrate an Aurora-based Postgres Db to an RDS Postgres Instance? I have searched on AWS docs I couldn't find any.  Any Pointer will be greatly appreciated. ThanksFollowComment"

Answer: "1Kindly note that there is no direct method to convert/migrate from Aurora PostgreSQL to RDS PostgreSQL. However, as a work-around, I have shared a couple of alternatives that you could review further.Using PostgreSQL native tools such as pg_dump and pg_restore.A) Since Aurora PostgreSQL and PostgreSQL have the same underlying engine, using the engine's native tools is usually the best way to perform migrations. Take the pg_dump from Aurora PostgreSQL and restore the dump into the newly created RDS instance. Please follow the link [1] below for detailed steps:I) Create a dump using pg_dump that contains the data to be loaded[2].II) Create the target RDS DB instanceIII) Use psql to create the database on the DB instance and load the data or use pg_restoreIV) Create a DB snapshot of the RDS DB instanceUsing AWS Database Migration Service (AWS DMS)[3].A) AWS DMS is a cloud service that makes it easy to migrate relational databases, data warehouses, NoSQL databases, and other types of data stores.B) With AWS DMS, we can perform one-time migrations (full-load), and you can replicate ongoing changes to keep sources and targets in sync[4]. AWS DMS provides minimum downtime, but the setup can be complex and might need a learning curve.C) With that said, there are also some limitations on using PostgreSQL as source and target endpoint, as specified in Document links [5-8].[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Procedural.Importing.html[2] https://www.postgresql.org/docs/8.3/backup.html[3] http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.html[4] http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.Components.html[5] http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html[6] http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Reference.Source.PostgreSQL.DataTypes.html[7] http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html[8] http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Reference.Target.PostgreSQL.DataTypes.htmlCommentShareSUPPORT ENGINEERSanketh Saliananswered 6 months ago"


Question: Amazon SQS Access Denied Exception

"I am trying to send messages to an SQS queue. For local environment I am setting up the AmazonSQS object like belowamazonSQS =AmazonSQSClientBuilder.standard()    .withCredentials(        new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey, secretKey)))    .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(endpoint, region))    .build();For my development environment I am setting up like below:amazonSQS = AmazonSQSClientBuilder.defaultClient();The tests run fine locally but in my dev environment, I am getting the below error:com.amazonaws.services.sqs.model.AmazonSQSException: Access to the resource https://sqs.us-east-1.amazonaws.com/2092384092384/error-handler-queue is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: 7bc7e074-a36a-544d-9489-bbe66133f0a8; Proxy: null)What could be the issue ? Any help is appreciatedFollowComment"

Answer: "0Hi, @sniranjan.It looks like you're not using credentials with the proper permissions.Where is your development environment hosted? EC2? Lambda?I assume you're using the AWS SDK for Java, but check the credentials below. Check your environment variables and instance profile.https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.htmlNext, if you can get IAM information, first check if the IAM policy settings are the same as your local environment. (In the end it's correct to set least privilege)CommentShareEXPERTiwasaanswered 9 months ago"


Question: Ec2 metadata service enumeration detection

How to identify suspicious EC2 metadata enumeration activity? should we detect from cloudtrail or using cli to analyze EC2 access log? thanksFollowComment

Answer: "0To inspect suspicious traffic to the instance metadata service, I would check VPC Flow Logs.  Those would have network connectivity that you could see if there are suspicious network traffic to the EC2 instance metadata service.AWS GuardDuty comes with a VPC Flow Log Finding: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_finding-types-ec2.html#unauthorizedaccess-ec2-metadatadnsrebind (Keep in mind AWS GuardDuty does have a cost associated with the service: https://aws.amazon.com/guardduty/pricing/).I would also recommend using IMDSv2 if possible which is a session-based method compared to request/response of IMDSv1: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.htmlVPC Flow Logs: https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.htmlCommentSharejsoncanswered a year ago"


Question: How to attach EBS volume from original EC2 instance to a new EC2 instance?

currently  my website is running under a t3a.small EC2 instance (Origin - N California) and the volume size is 10 GIB. Now i have launched a new EC2 t2.micro instance (Origin - Mumbai) and the volume size is 8 GIB. All i'm doing for avoiding charges as i am a free tier user and the original instance is not cover the free tier promotion. So how can i attach EBS volume from original instance to new instance without losing any data?FollowComment

Answer: 0If you shut the original instance down (not terminate!) then the EBS volume will no longer be attached to an instance; you can attach it to your new instance then.However: That will give you two volumes; a "new" root volume and the original root volume attached to a different mount point/drive letter.Note that in some circumstances it will be possible just to change the instance type and boot on the original root volume: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-resize.htmlCommentShareEXPERTBrettski-AWSanswered a year ago


Question: How to know last version available on ODC TAO Dashboard

"Hi, i'm using TRUSTED ADVISOR ORGANIZATIONAL (TAO) DASHBOARD from AWSWell-Architected Labs.I need to know the last version available for this dashboard or others.thank you.FollowComment"

Answer: "0Hello,The last version available for TAO Dashboard is from September 2022. This can be viewed here : https://www.wellarchitectedlabs.com/cost/200_labs/200_cloud_intelligence/trusted-advisor-dashboards/#last-updatedCommentShareSUPPORT ENGINEERSanjana_RKanswered 8 months ago"


Question: AWS Policy Generator bug for S3 - ARN flagged invalid despite not being

"Hi there,I'm trying to create an s3 bucket policy using the AWS policy generator found here https://awspolicygen.s3.amazonaws.com/policygen.html and I'm running in to what looks like a bug when entering the ARN of my bucket. I'm getting a message saying 'Resource field is not valid. You must enter a valid ARN.', despite my ARN being correctly formatted and valid (doesn't work either typing it or copying it from the AWS console). Has anyone found a solution to this? I know I could write the JSON myself, but I wanted to flag this issue. I noticed this question was also asked here, but the issue still persists https://repost.aws/questions/QUjnr5ei9VQm-YpIle6R4UrQ/why-my-s-3-policy-generator-dosent-workThanks.FollowComment"

Answer: "1Hi,I understand that you are having an error with creating an s3 bucket policy using the AWS policy generator. I did replicate and received the same error.As a workaround:I inserted a * in the ARN section and generated the policy.I, thereafter, replaced the * with my ARN once it has been generated.Please find the attached ARN Format documentation[1].I hope this helpsResources:-[1] https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html.CommentShareAsisiphoanswered a year agoddavies90 a year agoThat's a really helpful workaround. Thanks for that!Share"


Question: EC2 Instance Shutdown by SystemD event "Power key pressed" but no CloudTrail event were logged

"On Jan 04 12:00 GMT, our EC2 t3a.medium Instance placed in AZ ap-southeast-1c Has been Shutdown by AWS Datacenter (from our findings of SystemD event "Power key pressed")Why are this Instance Shutdown not recorded on CloudTrail event log?FollowComment"


Question: I cant login to sftp using my private key on my instance?

I cant login to sftp using my private key on my instance?ami-0fec1fb452e2ab3b0with ubuntu as usernamedo i need an sftp server or something?FollowComment

Answer: "0There are various reasons you might be unable to connect to your instance - this page offers some solutions: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.htmlDo you get any specific error message? It's worth trying to connect through SSH in verbose mode (ssh -vvv) to see if that gives any more information.CommentShareEdanswered 7 months ago0By default you won't be able to SFTP to an EC2 machine. There are many articles on the Internet on how to set this up. I went through a few and this one caught my eye - https://medium.com/@clopez.lopez/enabling-a-sftp-on-ec2-instance-with-ami2-linux-82eae613c429Disclaimer: I have not tried the instructions on this article myself and I or AWS do not endorse it in any way. You should try it out yourself and see if it works for youIf you want a fully managed SFTP solution on AWS, please take a look at https://aws.amazon.com/aws-transfer-family/Detailed documentation here - https://docs.aws.amazon.com/transfer/latest/userguide/what-is-aws-transfer-family.htmlCommentShareEXPERTIndranil Banerjee AWSanswered 7 months ago"


Question: Increase root drive in bulk for AWS workspace

We have about 4000 workspaces that were created that are too small to accept the 20H2 update. The only option we find seems to be increasing one workspace at a time but Amazon must have another option. Please advise. Thanks!FollowComment

Answer: "0Accepted AnswerMy guidance would be to use the modify-workspace-properties CLI (https://docs.aws.amazon.com/cli/latest/reference/workspaces/modify-workspace-properties.html) to do this via a script that loops through multiple sets of WorkSpace IDs at a time. Apply exponential backoff logic to not get throttled of course, but I would never do this through the AWS Console at scale.CommentShareEXPERTStevieStetsanswered a year ago"


Question: AWS Lake Formation: (AccessDeniedException) when calling the GetTable operation: Insufficient Lake Formation permission(s) on table

"I have implemented LakeFormation on my data bucket.I have a step function in which one step consists of running a GlueJob that reads and writes to the data catalog.I have upgraded my DataLake permissions as reported here.The Service Role that runs my Step Function has a root-type policy (granted just for debugging this issue):        Statement:          - Effect: "Allow"            Action:               - "*"            Resource:               - "*"   On lake formation the service role has:Administrator RightsDatabase Creation rights (and Grantable)Data Location access to the entire bucket (and Grantable)Super rights on read and write Database (and Grantable)Super rights on ALL tables within above Databases (and Grantable).The bucket is not encrypted.But, somehow, its access to the tables is denied with the error:(AccessDeniedException) when calling the GetTable operation: Insufficient Lake Formation permission(s) on tableWhat's really strange is that the Glue Job succeeds when writing to some tables, and fails on others. And there is no real substantial difference across tables: all of them are under the same S3 prefix, parquet files, partitioned on the same key.Given the abundance of permissions granted, I am really clueless about what is causing the error.Please, send help.FollowComment"

Answer: "0Hi,You have mentioned your state machine has all permission but can you please confirm if your Glue IAM role has permissions!Please make sure your Glue job IAM role has glue permissions on your Glue catalog.Reference: https://docs.aws.amazon.com/glue/latest/dg/using-identity-based-policies.html#identity-policy-examplesAlso, please verify from Lake formation "Data lake permission" that your Glue job IAM role has permission on Lake formation DB/table.I hope this helps!Thanks,TejalCommentShareTejal_Ganswered 10 months ago"