Question: put-bucket-notification-configuration question

"I am using 'aws s3api put-bucket-notification-configuration' and I'm getting the following error:An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Unable to validate the following destination configurationsThere is no actual 'output' following that statement.What I'm doing is editing a Lambda function, pushing that up using 'aws lambda update-function-code', then publishing a version using 'aws lambda publish-version', getting the new ARN and running the 'aws s3api put-bucket-notification-configuration' to install the new ARN into the S3 bucket's notification configuration.The only difference between the JSON file I'm using and the output of the 'aws s3api get-bucket-notification-configuration' command is the version number on the end of the ARN.Using the output of 'aws s3api get-bucket-notification-configuration' as the '--notification-configuration ' file works fine, but the one with the new version number fails.  Strangely, previous version numbers which I know to have been working also fail.FollowComment"

Answer: "0Unable to validate the following destination configurationsin most cases means that there is no permission to call Lambda.You can validate LambdaPermission, it probably allows notification to InvokeFunction without version, so you need to allow to invoke a particular version or use *.CommentShareMGanswered a year agowritchkrapht a year agoI have Allow '*' on '*'.Sharesagar a year agoThere are two permission concepts associated with an AWS lambda. One is the role associated with the lambda which indicates what a lambda can invoke. And there is a second thing which indicates who can invoke the lambda. Do you have a Allow * on * for who can invoke the lambda. In that case anyone on the internet can invoke the lambda, which is quite unsafe.Sharewritchkrapht a year agoI meant my login is an admin level account.  The Lambda is very private, and has only the basic permissions it needs to execute and to write to CloudWatch.  The S3 Bucket is also very private.This was a test of whether we could configure a Lambda to perform actions when it noticed a change in an S3 bucket.  I set them up last week and it works fine.  It's just (right now) logging the changes in the bucket as they occur.  I have permissions in my login which allow me to do all this through the GUI, but one of the goals is to get to where our Lambdas are published with shell scripts that call the AWS client so as to avoid the hazards of human behavior.The publishing is working splendidly, and creating a new version is as well, but telling the S3 bucket to use the new version ARN (which I can do by hand in the GUI) is the point of failure.Share"


Question: Adding new font to my custom fonts with AWS lambda not working

"I'm using AWS lambda with python and ffmpeg to add textual watermark over images. I added a custom font i.e Quicksand to my lambda function like this:1- Create a fonts directory in the root of the function and add the font file (.ttf) and a font.conf file with the following contents:<?xml version="1.0"?><!DOCTYPE fontconfig SYSTEM "fonts.dtd"><fontconfig>  <dir>/var/task/fonts/</dir>  <cachedir>/tmp/fonts-cache/</cachedir>  <config></config></fontconfig>2- Added an environment variable like thisFONTCONFIG_PATH = "/var/task/fonts"It was working fine until I chose to add more fonts. Now when I try to use some font other than the Quicksand e.g Arial, it still uses the Quicksand font. I'm not sure why? In the CloudWatch Logs I can see it is still trying to access the Quicksand font rather than the Arial one even when the code is right. Any caching issue or something else?My Folder Structure:My docker file:COPY requirements.txt ./RUN yum update -y && \    pip install -r requirements.txtCOPY . .CMD ["app.handler"]Cloudwatch logs:[Parsed_drawtext_0 @ 0x70b5bc0] Using "/var/task/fonts/Quicksand.ttf"Maybe its a caching issue? I'm not sure.FollowComment"

Answer: "0I am guessing this is something in your own code. Or maybe in your fonts.config file.CommentShareEXPERTUrianswered 9 months agoosama 9 months agoWhich fonts.config file? I only have fonts.conf file and I've added that in the question. Also, the code is fine. I've added the logs and have also tried to use hardcoded font names. But it still uses the Quicksand (added first). If I remove Quicksand font then it uses the second one. But still doesn't let me use any other.ShareUri EXPERT9 months agoI meant the font.conf file, sorry. I am not familiar with ffmpeg, What I am saying is that it is most probably not related to Lambda, but to the library itself. I did a quick search and found the following answer, which I think can help you as well.Shareosama 9 months agoThank you. I ended up storing the fonts on s3 bucket and copying them to /tmp storage for usage inside the function. Nothing else was workingShare"


Question: HPO training using Sagemaker SDK vs Sagemaker Jupyter notebook

"Hello,When training an HPO using the Sagemaker SDK It's much slower than training on Sagemaker jupyter notebook -Both variants have the same:Hyperparameters (the same Model)Data - Train / Test (exactly same data)Resources - 'ml.m5.24xlarge' machineThe SDK is slower at training and at inference (X3 more slow).Could it be due to different tensorflow versions?Is it a common issue? what things should I check?Thanks,OrFollowComment"

Answer: "0It's not quite clear exactly what comparison you're running here, but I can probably suggest some tips...The basics - Infrastructure start-upAssuming you're talking about end-to-end run times for workflows with your own algorithm/training script, the main difference between running a SageMaker job "with the Python SDK" versus in the notebook, is that a Sagemaker training (Estimator), processing (Processor), or inference (Transformer) job will spin up infrastructure to run your job on-demand.This means if I take exactly the same code and run it either 1) in my (already-running) notebook compute or 2) through a SM training/etc job on the same instance type - then (2) will probably run at similar speed but need an extra couple of minutes or so to spin up the instance and initialize the container image.The valuable trade-off here is that by using a ml.m5.24xlarge notebook instance I would be paying for the compute for all the time it's running (perhaps often at low utilization)... Whereas in a typical SageMaker workflow I might use just a ml.t3.medium for my notebook itself and only pay for the seconds of ml.m5.24xlarge that my training job(s) take to run: As soon as the training/processing/inference job gets completed, the instance is shut down automatically. Furthermore, while a notebook is typically limited to scaling up to bigger instances, SageMaker jobs can be easily scaled out to multiple instances so long as your algorithm/script supports it.If you'd like to orchestrate multiple steps together (e.g. pre-processing, training, tuning, inference), you also don't necessarily need to manage this from a notebook: You could set up a DAG with SageMaker Pipelines and just trigger that. If you're running lots of sequential jobs and would like to accelerate infrastructure start-up, check out SageMaker Warm Pools.Other ideasWhile the infra start-up period seems the most likely factor from your original question, maybe you already knew to adjust for that... There are a few other things worth mentioning in case:If initial data loading is a bottleneck, check out the available training data access modes in SageMaker. For example using FastFile mode may speed up training jobs in some cases, depending on the access patterns. The default File mode downloads all the data up-front before starting your training job, so you'd see this as added start-up time before your script runs.You mentioned "HPO", so I'm not sure if you're comparing a notebook run to a single training job with your own HPO procedure/script, or a SageMaker Automatic Model Tuning job. It's worth mentioning that SageMaker AMT spins up multiple training jobs (with configurable parallelism) so direct comparison to an all-in-notebook HPO would be difficult.If you're comparing your own code's performance to a SageMaker pre-built algorithm, then of course this is apples-to-oranges: For example using open-source XGBoost versus the SageMaker XGBoost in algorithm mode, or a public HuggingFace training script to an algorithm from SageMaker JumpStart.For training, you could try disabling SageMaker Debugger to turn off some hooks which get enabled by default (although I wouldn't usually expect the difference to be this large).As you mention, it's probably worth checking whether your TensorFlow and other key library versions are different between the notebook and the jobs.For inference in particular, a lot depends on how you're running your model... Batch transform? Processing job? Hopefully not deploying an endpoint then manually sending data through it and turning it off? Is the inference well-optimized?One last thing: I was a little surprised to read you using TensorFlow but not training on a GPU-accelerated instance type like p3, g4dn, inf1/2, etc? If the workload is parallel compute-bound like a lot of neural networks, you might be able to get higher throughput-per-dollar by using accelerators.CommentShareEXPERTAlex_Tanswered 17 days ago0Alex_T's comment already covers the most likely factors for the observed performance difference. In addition to Alex_T's comment, you could also make use of the SageMaker Debugger feature with your Python SageMaker SDK to determine if there is any issues with your job.If you require further assistance, I would recommend you to open a support case with the specific configuration and reproducers of your two approaches so one of our SageMaker Support Engineers can inspect the differences in closer detail.CommentShareSUPPORT ENGINEERCadence_Lanswered 15 days ago"


Question: Do we need Lambda extensions for accessing AWS Secrets Manager ?

"Do we need Lambda extensions for accessing AWS Secrets Manager ?I explored Vault, Lambda provides vault extensions, which can optimize the secrets fetching -https://www.hashicorp.com/blog/aws-lambda-extensions-for-hashicorp-vaultThe extension can retrieve multiple secrets from Vault, if configured to do so, and writes the full JSON response from HashiCorp Vault to the configured destination.Before a Lambda function is invoked, extensions are initialized and given the opportunity to perform tasks before signaling their readinessDo we need similar setup in AWS Secrets Manager ? Or Fetching secrets from AWS Secret Manager already optimised for Lambda ?FollowComment"

Answer: "0In order to access secrets manager you only need to define the security policy in your lambda and it should then be able to access it from your code.More info hereCommentSharenissyangelanswered a year agoNavin GV a year agoI understand, will it cause any performance issues while getting the secrets for every invocation ?I see that the Lambda extensions for vault does some work before lambda gets initialised.Share0You do not need a Lambda extension to retrieve a secret from AWS Secrets Manager in a serverless workload. You should retrieve the secret as a global variable, not in the handler, to maximize performance. To learn more, see https://aws.amazon.com/blogs/networking-and-content-delivery/securing-and-accessing-secrets-from-lambdaedge-using-aws-secrets-manager.CommentShareavniatawsanswered a year ago0This link -https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/cache-secrets-using-aws-lambda-extensions.html provides a way to run local host to cache the secrets.This setup is not necessary at all the times.  Accessing Secrets Manager from Lambda can be done via the boto3 API, but the pattern above is considered best practice as it prevent cold start, increase speed of retrieval and keep the costs for Secrets Manager access low (in cases where you expect a lot of invocations).CommentShareNavin GVanswered a year ago"


Question: The "new" Amazon ECS Console is missing fundamental options like container command overrides

"The "new" Amazon ECS Console, which is an opt-in experience, is missing important functionality. One such missing function is the ability to specify a custom entrypoint or command override for each container defined in an Amazon ECS Task Definition. This makes the usage of the "new" console almost impossible, and I have to fallback to using the old console.When is the "new" ECS console going to be usable?FollowCommentAWS-User-5726074 a year agoAgreed with original concerns here, just wasted like 20 minutes looking for those parameters on the new UI, knowing I had used them in the past to make some Task Definitions that are currently running.Share"

Answer: 0This feature has been logged with the ECS team and will be reviewed.CommentShareEXPERTChris_Ganswered a year ago


Question: Starting with Deep Glacier.

"I have 5 Terabytes of information that I want to backup in AWS deep Glacier. I hope to never download them, just wishing to archive them in the cloud besides physical disks. Is there a way to Uoload it directly from S3?FollowComment"

Answer: "0Accepted AnswerReading the question a little differently. If that data is on your local disk, you can copy objects directly to any storage class in S3 by using the command line. This avoids any charges by storing the objects other storage classes for a period of time.For example:aws s3 cp <source file> s3://<bucket name>/path/ --storage-class DEEP_ARCHIVECommentShareEXPERTBrettski-AWSanswered 2 months ago0Yes, you can upload data directly from Amazon S3 to Amazon S3 Glacier using a lifecycle policy that transitions objects from S3 to Glacier.To do this, you can create a S3 bucket and upload the 5TB of data to this bucket.Then, you can create a lifecycle policy to transition these objects to S3 Glacier.When you create the policy, you can specify a rule to transition the objects from S3 to S3 Glacier based on the object age, object size, or both.Here's an example of a lifecycle policy{  "Rules": [    {      "ID": "Archive to Glacier",      "Status": "Enabled",      "Filter": {        "Prefix": ""      },      "Transitions": [        {          "Days": 30,          "StorageClass": "GLACIER"        }      ]    }  ]}Once the objects are transitioned to S3 Glacier, you can view them in the S3 Glacier console or retrieve them using the AWS CLI or SDK.CommentShareEXPERTsedat_salmananswered 2 months agoMrSergio 2 months agoThank you very much. I guess that I have to learn to create a lifecycle policy now. Thank you very much!Sharesedat_salman EXPERT2 months ago:)https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.htmlShare"


Question: How to migrate only postgres data using EBS volumes

"Our application stores metadata in postgres database. We are using AWS ebs volumes for postgres data for persistent storage. We attach and detach this EBS volumes to AWS instance when needed. AWS instance is destroyed when not needed and will create new one when needed. EBS volume will be attached when new instance is created. The problem is, when ebs volume is detached postgres version is 9.x. While attaching EBS volume to new instance, postgres version in the new instance is 11.x. How to migrate the postgres data from 9.x to 11.x in the ebs volumes where new instance has 11.x version postgres version installed.FollowComment"

Answer: "1Hello,I assume this is a self-managed postgres database? You'll need to upgrade the database from v9 to v11, you cannot just use the database files created in one version from another db engine version.You can refer to this documentation for instructions for example: https://www.postgresql.org/docs/11/pgupgrade.htmlRegardsCommentShareAWS-ADolganovanswered a year ago0First attach to v9 , upgarde the db to v11 and from this point you have the data on EBS ready for another detach-attach as v11CommentShareArikYanswered 10 months ago"


Question: Function 'GetDynamodb' failed to execute for rule. Not able to retrieve and/or parse the requested item from dynamo.

"We have a rule where the condition is based on data from dynamodb. We wanted the filtering based on if given key had an item in the database. The query looks like:SELECT * FROM '$aws/events/presence/connected/+' WHERE NOT isNull(get_dynamodb(MY_TABLE, 'HashKey', MY_HASHKEY, 'RangeKey', MY_RANGEKEY,   MY_ROLE))When the data is present the rule works as expected, but when it is not we get these errors logged in CloudWatch:details: Not able to retrieve and/or parse the requested item from dynamo. ErrorMessage:eventType: FunctionExecutionlogLevel: ERRORdetails: Function 'GetDynamodb' failed to execute for rule MY_RULE. Not able to retrieve and/or parse the requested item from dynamo. ErrorMessage:eventType: RuleExecutionlogLevel: ERRORreason: ExternFunctionExceptionSo the questions are:Can i use the get_dynamodb function to check if a item exists in the database, or can I only use this function if I know that the data for a given key is present?Should I expect my error action to trigger in the case described above (testing indicates that my error action is not triggered)?FollowComment"

Answer: "1Accepted AnswerHi Magne.You can use it when an item is not present, but it will cause the FunctionExecution error you're seeing. So it's probably not the behaviour you're wanting.No. The error action is only triggered if a rule action suffers an error. Your rule query failed to produce a match (because the error occurred in the WHERE clause) and hence no action was triggered.CommentShareEXPERTGreg_Banswered 9 months agoMagne 9 months agoThanks for the quick answer!This was unexpected(and disappointing seen from my use case). Could it be an idea to update the documentation for that function (https://docs.aws.amazon.com/iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodb)?Sharekbrandsen 9 months agoI have created a request for the documentation to be updated.Share"


Question: IVS Player 1.3.1 - undefined property collectLogs

We upgraded to v1.3.1 and we are seeing the an undefined property on the amazon-ivs-wasmworkeramazon-ivs-wasmworker.min.js:5 Uncaught (in promise) TypeError: Cannot read property 'collectLogs' of undefinedat printErr (amazon-ivs-wasmworker.min.js:5)at amazon-ivs-wasmworker.min.js:5Has anyone else run across this yet?  Thanks in advancedFollowComment

Answer: "0Accepted AnswerHi Guys,I ran across this exact issue and when googling the error message, this was the only page that popped up.First I tried resolving this using CSP headers, as suggested by @riley-aws.Unfortunately this did not do the trick.So I dug a little deeper, and using a break point where the error occured, I was able to get the error details. This was quite interesting indeed and lead me to learn why the wasm file couldn't be executed.Here's the error:"wasm streaming compile failed: TypeError: Failed to execute 'compile' on 'WebAssembly': Incorrect response MIME type. Expected 'application/wasm'."Obviously the solution was to add the 'application/wasm' mime type header for .wasm files.I did that and it worked.Hope this helps someone else!To amazon I suggest to catch these promise errors to maybe log them with a little bit more context. Had I seen this error message in the console, I'd immediately have known what's up. If there's more I can do to help with this, pls let me know.Edited by: janpaepke on Jun 9, 2021 5:54 AMCommentSharejanpaepkeanswered 2 years ago0This error is potentially being caused by a limitation with the Content Security Policy(CSP) on the browser being used. More information on this behavior can be found https://docs.aws.amazon.com/ivs/latest/userguide/player-web.html#web-content-security-policyDue to Safari's limited CSP support, it requires a very lenient CSP policy (one including unsafe-eval). This works against the whole point of CSP, to limit dangerous JavaScript from running on a page. With Safari, to avoid having dangerous JavaScript run on your page, we recommend that you host the library assets on the same origin as your page and use our same-origin CSP policy.Seeing as the error being seen is a relatively generic error, it would be beneficial if we could better understand if there is a CSP policy in place on your webpage? As well, if there is any sample code that you are able/willing to share, please feel free to include that, as it may help indicate why this error is being thrown.CommentShareriley-awsanswered 2 years ago0Did you find a solution to this issue?I'm having the exact same problem here. I tried to add the CSP configs (didn't have it on the server), but the issue still happens.CommentShareistumpfanswered 2 years ago0We have not found a solution yet, and have been running an older version in the interim.  We will keep you posted on our findings as we find a way to complete the upgrade.CommentShareagirlinganswered 2 years ago0Thank you, this is a great find.  I checked the mime type and found it was declared as binary/octet-stream.I'll need to update my CI to set "application/wasm" properly when uploading to S3.  Attaching a sample here for future reference:aws s3 cp website/ s3://$WEBSITE_BUCKET --recursive --exclude "*.wasm"aws s3 cp website/ s3://$WEBSITE_BUCKET --recursive  \--content-type "application/wasm" \--exclude "" --include ".wasm"CommentShareagirlinganswered 2 years ago0Hi All,I am also facing the same error, having already in place the mentioned content type. Actually, I am hosting the js library and wasm file from CDN. It works in the beginning but started to throw this error after an hour and a half of smooth running.Previously, we were using version 1.1.2, which doesn't seem to have this error. However, there we were facing memory errors in that version, as we had frequent switching between different feeds on the browser.Please see attached screenshots.https://ibb.co/r6LBWRshttps://ibb.co/0CzgsgTEdited by: Sala on Jun 10, 2021 10:24 PMCommentShareSalaanswered 2 years ago0Only this solution works perfectly. Thank you very muchCommentShareAhnYoungSeonanswered 2 years ago0Thank you AWS team for adding information regarding setting the Content-Type header on hosted assets.For future reference of those reading this thread - https://docs.aws.amazon.com/ivs/latest/userguide/player-web.html - section "Setup With NPM"CommentShareagirlinganswered 2 years ago"


Question: SMS SenderID in India

"My organization has already got a sender ID from TRAI. Do I need to request for Sender ID again through a support ticket, to send SMS using SNS in India?FollowComment"


Question: When is the Aurora-Redshit zero-ETL coming up?

"I felt there was a lot of hype around https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-aurora-zero-etl-integration-redshift during the re:Invent and shortly afterward.However, the announcement says that it is only for the Aurora MySQL certain versions, in the US East (N. Virginia) Region, and only a limited preview. The limited preview link is no longer working.I am looking for Aurora PostgreSQL zero-ETL integration into Redshift.Are there any news at all since re:invent? Have I missed anything?FollowComment"

Answer: 0Please connect with your AWS account team and they will be able to set a meeting with a Redshift Specialist that can better answer your questions.CommentSharejuliaturanswered a month ago


Question: Is there a way to avoid throttling exceptions in tag editor?

I am using tag editor to tag a bunch of domains so that I can use a script to transfer them to another account.  I have selected 113 domains to tag and am trying to add a single tag to each.  When I ran it only 28 succeeded before the process failed with a rate limit exception.I thought the whole point of tag editor was to make bulk changes so failing after so few actions seems broken.  Is there a way to avoid the rate limit errors in this tool without having to keep retrying the action manually?FollowComment

Answer: "0According to the docs, there is a limit of 5 calls per second for TagResources.CommentShareEXPERTkentradanswered 2 months agoMatt 2 months agoYeah I know about the limits, I was just hoping tag editor handled this situation better.  Like maybe by staying under the rate limit or automatically using a retry mechanism or something like that.  Just seems weird to offer a bulk edit tool like this just to have it fail constantly on rate limits.Share"


Question: DNS zone transfer

"Hello all,I transferred a registered domain from bluehost to aws and its worked smoothly, but seems i cannot move the zone into a route 53 hosted zone!, i created one, changed the nameserver on registered domain section in route53 to use aws ns, but seems its still resolving from bluehost, how can i make sure that my registered dns is using 53 hosted zone, not the blue host dns ? , thanks.FollowComment"

Answer: "0domain name?CommentShareMuhammadanswered 4 months ago0complymarket.comCommentSharerePost-User-3348574answered 4 months ago0I get these results from leafdns:The parent nameserver m.gtld-servers.net. reports your nameservers as:TTLGlueIPName172800Yes205.251.192.213ns-213.awsdns-26.com.FAIL: Nameserver is not authoritative for complymarket.com.172800No205.251.194.190ns-702.awsdns-23.net.FAIL: Nameserver is not authoritative for complymarket.com.172800No205.251.196.53ns-1077.awsdns-06.org.FAIL: Nameserver is not authoritative for complymarket.com.172800No205.251.199.155ns-1947.awsdns-51.co.uk.FAIL: Nameserver is not authoritative for complymarket.com.i created hosted zone in aws, i did not change the SOA nor ns, should i do anything ?CommentSharerePost-User-3348574answered 4 months ago"


Question: AWS Backup retention in depth

"Hi everyone,I have the following situation with 2 backup plans and 1 EC2 instance for both of them:Backup plan 1:Frequency : every 12 hours (starting at 5 AM) / Retention 30 daysBackup plan 2:Frequency : every 24h (starting at 5 AM) / Retention 1 dayCopy backup to another region (same account)I had the above configuration and it generated some high costs in transferring data from one region to another one.Questions:How does retention works more exactly ? Let's say the backup takes 2h to finish. Retention will be 1 day since STARTING the backup (5AM) OR since it was successfully FINISHED (7AM) ?Backup plan 2 generated some high costs (regarding transferring data) and I want to double check if this is because the backup is expired before starting the incremental backup and now is forced to perform a FULL backup from scratch ?Since this is about same EC2 instance, on Backup Plan 2 shouldn't perform an incremental backup of the backup from Plan 1 (since it has the backup since 12h ago)? - or the incremental is separated on each Backup plan, regardless of having the same instances in both plans.FollowComment"

Answer: "0[Question] How does retention works more exactly ? Let's say the backup takes 2h to finish. Retention will be 1 day since STARTING the backup (5AM) OR since it was successfully FINISHED (7AM) ?[Answer] If the Backup creation/start time is December 6, 2022, 05:00:00 (UTC+00:00) and backup completion time is December 6, 2022, 06:16:10 (UTC+00:00) and the retention is 1 day then backup expiration would be December 7, 2022, 05:00:00 (UTC+00:00). So, expiration time is calculated from the Backup creation time rather than completion time.Please note: AWS Backup randomizes its deletions within 8 hours following recovery point expiration to maintain performance. https://docs.aws.amazon.com/aws-backup/latest/devguide/deleting-backups.html[Question 2] Backup plan 2 generated some high costs (regarding transferring data) and I want to double check if this is because the backup is expired before starting the incremental backup and now is forced to perform a FULL backup from scratch ?[Answer 2] Whether a snapshot copy is incremental is determined by the most recently completed snapshot copy. When you copy a snapshot across Regions or accounts, the copy is an incremental copy if the following conditions are met:The snapshot was copied to the destination Region or account previously.The most recent snapshot copy still exists in the destination Region or account.All copies of the snapshot in the destination Region or account are either unencrypted or were encrypted using the same KMS key.If the most recent snapshot copy was deleted, the next copy is a full copy, not an incremental copy. Refer https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-copy-snapshot.html[Question 3] Since this is about same EC2 instance, on Backup Plan 2 shouldn't perform an incremental backup of the backup from Plan 1 (since it has the backup since 12h ago)? - or the incremental is separated on each Backup plan, regardless of having the same instances in both plans.[Answer 3] Backup of the same EC2 instance created by either Plan 1 and Plan 2 will always be incremental. New backup created by either of the backup plan saves only the blocks that have changed since your last snapshot. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.htmlCommentShareAnkushanswered 6 months ago"


Question: How does authentication/authoriation work with multi region api in a DNS fail-over scenario?

"I  want to set up a multi region api with  DNS-fail over setting , by setting up health checks. I plan to set up api key for my api endpoint. say, if i create api gateway resource in two regions us-east-1 and us-west-1, with api key. i assume it will create different api keys for my api in each region. so if request comes into an api hosted in us-east-1 with a particular api key, and if us-east-1 fails and then request is routed to us-west , it should fail because the api key are different?FollowComment"

Answer: "1Accepted AnswerYou have the option to either have API Gateway generate API keys for you or you have the option to import API keys. If you use the import option, you can ensure that the same API Keys are used for a particular client for both your regional API Gateway end-points.Look at the "Import API Keys" section in https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-setup-api-key-with-console.html#api-gateway-usage-pan-import-apikeyAs a side note, you should not use API Keys for Authentication and Authorization. API Keys are only meant for implementing Usage Plans for clients.You can use mechanisms such as Cognito User Pools, Cognito Identity Pools, Lambda Authorizers etc. for Authentication and Authorization in API GatewayCommentShareEXPERTIndranil Banerjee AWSanswered 8 months agoEXPERTAWS-User-Nitinreviewed 7 months agoclouduser 8 months ago@Indranil - thanks. point noted regarding api keys and authentication/authorization. could you also please take a look at my another question here, I am having an issue on how to do this via cloudformation. i have created a hosted zone for my domain name and want to use that and map it to my api gateway cloudformation in two regions, but i'm not how to connect  -> https://repost.aws/questions/QU0SLKNMhsQdKYp3dTjsD_1g/how-to-configure-a-custom-domain-name-for-api-gateway-in-a-multi-region-scenario.Share"


Question: Copy snapshot to Europe (Milan)

"Hi all.I'm trying to copy a snapshot from EU (Frankfurt) to Milan but I receive the following error:An internal error has occurred. Please try your query again at a later time. (Service: AmazonRDS; Status Code: 500; Error Code: InternalFailure; Request ID: af9c2f07-30ea-4f92-8609-a6b598adc685)If this is not supported, is there another way to move my Oracle instance to the new region?Thanks.AlessandroFollowComment"

Answer: "0Accepted AnswerPlease take a look at https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Oracle.Procedural.Importing.htmlMany of the techniques described could be used to move a database between RDS Oracle in different regions.CommentShareHalTempanswered 3 years ago0Hi,Cross-Region snapshot copy isn't supported in the following opt-in AWS Regions:Africa (Cape Town)Asia Pacific (Hong Kong)Europe (Milan)Middle East (Bahrain)Ref: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CopySnapshot.htmlCommentShareAWS-User-6940831answered 3 years ago0Thanks for confirming.Can you advise on how I can migrate an existing Oracle RDS instance to that region? Or is this operation not currently supported?CommentSharealilomapaanswered 3 years ago0Backup & Restore!CommentShareGF7answered 3 years ago0I thought a database snapshot was exactly that: a backup of my instance.Are you saying there's another method to backup a db through the AWS console that is portable across regions?Apologies if this is a stupid question, I'm really no expert in db administration.CommentSharealilomapaanswered 3 years ago0I'm not sure about Oracle, but for all others engine it is possible.With MySql/MariaDB/Aurora MySql:-> Power on an ec2 instance and run mysqldump command. You will get the dump in your ec2 instance.-> Create the rds instance in other region and restore the dump you generated.With Postgres:-> The same as mysql: pg_dump and then restore.With SqlServer:-> You can connect with management studio and run a backup in two ways: A store procedure that uploads a native backup to S3 or backup directly from management studio.-> To restore the data you have to upload the backup to S3 and then run a store procedure to import.This is what I have tested personally. There is other ways that I didn't tried like the export of the snapshot to S3. Then you can move it and restore as you would fill the database the first time.I guess oracle has something similar if not better. Data pum, some kind of management studio where to perform these actions etc.CommentShareGF7answered 3 years ago0Thank you very much for taking time to reply.I'll have to check if/how this is doable with Oracle (I imagine it is, but never tried before).Cheers.CommentSharealilomapaanswered 3 years ago0Thank you.I'm trying to follow those instruction (specifically the section "Importing Data with Oracle Data Pump and an Amazon S3 Bucket"), but I get an error on step 3 when uploading the dump to S3:[ERROR] The DB instance doesn't have credentials to access the specified Amazon S3 bucket. To grant access, add the S3_INTEGRATION role to the DB instance.But I did add that option to my db: https://i.gyazo.com/0347a2a84582b392556fc5fa0c6b1a80.pngI also added the IAM role: https://i.gyazo.com/006590bba853cf3dba5eda64310bf614.pngDon't know what else I can try: can you help me?Edited by: alilomapa on Jun 5, 2020 4:33 PMCommentSharealilomapaanswered 3 years ago0I tried again today and now it worked: not sure if it was en error on my part or I just had to wait a bit for the changes to propagate. Anyway, I have now successfully uploaded the backup file to a S3 bucket.Unfortunately I'm now facing the same issue when uploading to the destination db:[ERROR] The DB instance doesn't have credentials to access the specified Amazon S3 bucket. To grant access, add the S3_INTEGRATION role to the DB instance.But, when I open this newly created db in the AWS console, the "Manage IAM roles" section is missing. Also the left side menu has fewer options. This is what I see:in eu-central-1: https://i.gyazo.com/378c443144f560a336b9d332e13b1d57.pngin eu-south-1: https://i.gyazo.com/14e7c5901eb59925e6a193362f02ff02.pngHave I done something wrong on db creation or is this region somehow "different"?AlessandroCommentSharealilomapaanswered 3 years ago0Never mind, I did it with a db link: way easier than fiddling around with S3.Thanks again to all who have responded.AlessandroCommentSharealilomapaanswered 3 years ago"


Question: SDK for JavaScript v3 - "Error: TypeError: NetworkError when attempting to fetch resource"

"I followed this tutorial to get started using the AWS SDK for JavaScript (version 3) within React Native apps, in order to, say, create and delete buckets stored in Amazon S3. I completed all the steps in the tutorial, but ran into an issue on Step 6: running the example React Native app. The app does run in my browser, fortunately. The problem is, the app keeps giving me the error message in the question title, whenever I type a bucket name into the text input field and subsequently press the button that calls this asynchronous JavaScript function for creating buckets:const createBucket = async () => {    setSuccessMsg('');    setErrorMsg('');      try {        await client.send(new CreateBucketCommand({ Bucket: bucketName }));        setSuccessMsg(`Bucket '${bucketName}' created.`);    } catch (e) {          setErrorMsg(e);    }};What gives? I successfully created a Cognito Identity pool and attached the AmazonS3FullAccess policy to the unauthenticated identities IAM role. And I am positive that I provided the correct credentials—region and identity pool ID—within the React Native code.If it matters, I used Expo, not the React Native CLI, to create and run the app.FollowComment"

Answer: "0Hello Ben,It's possible that you have entered a bucket name that violated S3's bucket naming rules [1].Additionally, "TypeError: Network error when attempting to fetch resource" usually indicates CORS error [2]. This might happens if you're sending your create bucket request to a region other than us-east-1, since your application will have to handle 307 Temporary Redirect [3], thus triggering a CORS error. The sample code from the tutorial allows you to change the configured region by changing the line (line 18) const REGION = "REGION" to const REGION = "us-east-1", which might resolve the CORS error.Best,1: S3 bucket naming rules2: CORS - Mozilla documentation3: S3 createbucket commandCommentShareTriet_Vanswered 10 months ago"


Question: AWS SES ip address 54.240.9.16 on spam list

"can AWS get this IP address be unblocked?or is there a way to set my SES account to not use this ip address?action":"failed","status":"5.3.0","diagnosticCode":"smtp; 553 5.3.0 Spam blocked see: http://spamcop.net/bl.shtml?54.240.9.16"}]http://spamcop.net/bl.shtml?54.240.9.16https://www.spamcop.net/w3m?action=blcheck&ip=54.240.9.16Causes of listingSystem has sent mail to SpamCop spam traps in the past week (spam traps are secret, no reports or evidence are provided by SpamCop)SpamCop users have reported system as a source of spam less than 10 times in the past weekFollowComment"

Answer: "0Hello,From the link you provided, I've queried that IP address again and it is now unlisted in that spam list.For more information about Blacklists and Amazon SES, please visit: https://docs.aws.amazon.com/ses/latest/dg/faqs-dnsbls.html#bl-q4Thank you!Regards,FrankyCommentShareFranky Chenanswered a year ago"


Question: where clause won't work in sql transform in glue

where clause doesn't work in sql transform in glue. any help is appreciated.FollowComment

Answer: "0We would need more details on what is the issue that you are exactly facing , but check if the below helps you .In the glue script below note the abc text . The query by itself just uses that literal , whereas in the later section , abc is  actually being mapped to the sql table . sparkSqlQuery is a custom function that is written to execute the sqlYou can simulate a similar script if you devise just this section of the code in  a new visual job.def sparkSqlQuery(glueContext, query, mapping, transformation_ctx) -> DynamicFrame:    for alias, frame in mapping.items():        frame.toDF().createOrReplaceTempView(alias)    result = spark.sql(query)    return DynamicFrame.fromDF(result, glueContext, transformation_ctx)# Script generated for node SQLSqlQuery0 = """select year(create_time)          as create_time_year,month(create_time)         as create_time_month,count(id)                  as count_of_records from abcwhere create_time is not null group by year(create_time),month(create_time)order by year(create_time),month(create_time)"""SQL_node = sparkSqlQuery(    glueContext,    query              =SqlQuery0,    mapping            ={"abc": sql_node},    transformation_ctx ="SQL_node",)CommentShareKoustavCanswered 5 months ago"


Question: Limitations on unauthenticated Cognito identities?

"I'm trying to use an unauthenticated Cognito identity to call ec2.describeInstanceStatus using the JavaScript for web SDK. When I try to call that or the similar ec2.describeInstances, I get an UnauthorizedOperation error and a HTTP 403 forbidden.I checked the IAM role permissions against the IAM Policy Simulator and the role should have the proper permissions to perform those actions.... I even gave it "AmazonEC2FullAccess (IAM Policy)" and nothing else on a new IAM Role since I was at a loss. I still got UnauthorizedOperation.... This leads me to my question. Do unauthenticated Cognito identities have limitations on services/calls? I am able to invoke Lambda functions using the same IAM role and using the same basic JavaScript/HTML so I know the unauthenticated identity works for at least some actions.Thanks for the help!FollowComment"

Answer: "0Answer found: From https://docs.aws.amazon.com/cognito/latest/developerguide/iam-roles.html#access-policiesFor additional security protection, Amazon Cognito applies a scope-down policy to credentials vended by GetCredentialForIdentity to prevent access to services other than the ones listed below for your unauthenticated users. In other words, this policy allows an identity using these credentials with access to only some services (list in link).EC2 is not in that list.CommentShareaj177answered 2 years ago"


Question: How can i add AWS Community Builder Badge to the re:Post profile?

How can i add AWS Community Builder Badge to the re:Post profile?FollowComment

Answer: "4Accepted AnswerHi,To add an AWS Community Builder badge in re:post fill this form: https://app.smartsheet.com/b/form/1c0373d4b099462cbabfafdd206a44efHope it helps, and If it does, I would appreciate if this answer is accepted so that community can benefit for clarity, thanks ;)CommentShareEXPERTalatechanswered 2 months agosedat_salman EXPERT2 months agoThank you for this link and clarificationShare"


Question: VMWare to AWS Linux Based Instance

I have an VMWare server which I exported into OVF format.  On import I get this error: "ClientError: Multiple different grub/menu.lst file s found."  I have also tried importing the drive of the server and I get the same: ClientError: Multiple different grub/menu.lst file s found."  Any ideas would be appreciated.FollowComment

Answer: 0The error message is saying that there are multiple GRUB menu files that the import utility can find. You'll need to locate the duplicate ones and delete them from the image before importing it.CommentShareEXPERTBrettski-AWSanswered 8 months ago


Question: How to have ECS map ports automatically when a task is deployed in EC2?

"Context:I am using Circle CI's aws-ecs/deploy-service-update orb to deploy my docker container by pulling the latest image in AWS ECR and deploy it in AWS ECS with AWS EC2 instance. This container is a Machine Learning model that accepts API requests at TCP port 3000(I am using fastAPI for this) and returns the predictions. After I deployed it I couldn't send requests to the public IP of the container instance of the task that deploys the container at port 3000 (This IP is not my EC2 instance's public IP; it only has private IP and public IP is disable).DebuggingI checked my security group and made sure that the port 3000 is open to receive requests from all IPs(0.0.0.0), as part of the inbound rule.I stopped the task(which automatically will stop the container running in the EC2 instance) with the thought that something may have gone wrong from Circle CI. Then, according to the service configuration(1 desired task) and task definition of AWS ECS, a new task has started(hence the container) automatically. But, I couldn't send requests to this either.I SSHed into my EC2 instance to know if the port 3000 is open. This is when is when I learned that ports weren't mapped at all:https://i.stack.imgur.com/rUAzF.pngAs you can see, PORTS column is empty for the container and the container has to accept requests at port 3000 from the command.And here are the open ports of the EC2 instance:https://i.stack.imgur.com/cvrtl.pngAs you can see, port 3000 is not listed here.Here is the task with port mappings which deployed the container (to AWS ECS) that you see docker ps screenshot above:https://i.stack.imgur.com/enTjv.pngIn the task definition, you can see the port mappings I have defined for the container.Here is the task running on my EC2 instance with the task-definition shown above and the network mode I am using is 'awsvpc':https://i.stack.imgur.com/0SpUt.pngHere's the "Networking" tab of ENI associated with the task, and also the inbound rule of the security group associated with the EC2 instance that the task is running inside, which accepts requests on port 3000 from all IPs.https://i.stack.imgur.com/5V9Tk.pngEDIT 1:After I diddocker run -p 3000:3000 <my-image:my-tag>  inside the EC2 machine(by SSHing from my laptop), I could send API requests and receive proper response to the container to it's public IP, of the cluster of AWS ECS. This means that ports are being mapped only when I run the container manually.I had no problems with ports when I used FARGATE, when I updated the service from Circle CI or even when I manually started tasks.So, how to automatically map ports when a task is run from AWS ECS service dashboard or from Circle CI? If I run docker container manually, I will not be able to get logs automatically from AWS Cloudwatch and will not be able to stop it from AWS ECS dashboard. Another container by AWS that is running in EC2 instance will take care of those things. It will route the logs to Cloudwatch and accepts stop the existing one and start commands to start a new container with new image stored in AWS ECR, without having to SSH everytime I would want to look at logs or start/stop containers.What has gone wrong here, which led to ports not being mapped and How do I fix it and map ports properly, so i will be able to send API requests to my container.If you would like to look at the screenshots, visit the same question on stackoverflow, here: https://stackoverflow.com/questions/64632279/port-mapping-didnt-happen-for-a-container-deployed-on-aws-ecsuses-ec2FollowComment"

Answer: "0I was using an ECS cluster optimised for FARGATE and was trying to use EC2 instances inside it. So, since the cluster's networking wasn't optimised for EC2 instances, I got problems with port mapping. Changing the cluster template to EC2 type with "bridge" network mode fixed the issue in my case.CommentShareNaveen Reddy Marthalaanswered 3 years ago"


Question: RDS Free Tier Showing Cost - MySQL Public IP

"Hello,I checked previous threads regarding estimate cost showing even for Free tier RDS DB creation.But, their configuration was different than mine.Can anyone please confirm whether I will be charged if I create instance for the following configuration:MySQL 5.7.22Templates: Free TierDB Instance size: db.t2.microStorage: General Purpose SSD 20 GBConnectivity: Default VPCSubnet group: defaultPublicly accessible: yesVPC security group: defaultAvailability zone: ap-south-1aPort: 3306Initial DB name: xxxxxxxxxxxDB Parameter group: default.mysql5.7Option group: default:mysql-5-7IAM db authentication: EnabledBackup: disabledMonitoring: disabledLog: Error, General, Slow query, Audit (all ticked)Maintenance: Enabled auto minor version updateMaintenance window: No preferenceDeleting protection: Not enabledEstimated monthly costsDB instance17.52 USDStorage2.62 USDTotal20.14 USDFollowComment"

Answer: "0Accepted AnswerThe console tells you what the costs would be IF you didn't have the Free Tier credit, or run more than one t2.micro during the month. This is a consequence of how the Free "Tier" works, which is poorly understood. The Free Tier isn't a special mode or type of instance in RDS, it is a credit that is applied at the end of the month if you are within your first year of the account's life. So during normal RDS operation it has no idea if an RDS instance is "free" or not. At the end of the month the billing system applies a credit of up to 750 hours of t2.micro time and 20GB of storage & snapshot storage to the account.Since the RDS Console doesn't know the history of the account, or how to present all the complexities around application of the credit, it can't know that this specific free-tier eligible t2.micro will actually be free. What happens if you fire up two at the same time? You will only be charged for having one, but there isn't any identification in the system which one is free and which is charged for. So the console can't really tell you which one would be free and must display the estimated charges independent of the credit that will be applied at the end of the month.I went and looked at creating a default Free Tier eligible instance in Mumbai and received the same estimated cost as in your example. So, assuming that is the only t2.micro instance you have in a month, you don't change the configuration (e.g., add more storage or turn on Multi-AZ), and you are still in the Free Tier portion of the account's life, it should be free.CommentShareHalTempanswered 4 years ago0Try thishttps://calculator.awsCommentShareMODERATORphilawsanswered 4 years ago0Thanks a lot HalTemp.CommentSharerohitvedantwaranswered 4 years ago"


Question: What's the fastest way to move a 2TB Oracle Database to EC2?

Customer has a 2TB Oracle Database that they need to lift and shift to AWS.This 2TB Database is only one schema and size of the backup.What would be fastest way with minimum downtime?Clarification:Customer has Site-to-Site VPNFollowComment

Answer: "1Accepted AnswerYou can follow the strategies listed out in this documenthttps://docs.aws.amazon.com/prescriptive-guidance/latest/migration-oracle-database/migration-oracle-database.pdfThis document has a table that lists different utilities/methods that can be employed while doing a migration. Your choices can be different if you are moving to a EC2 based instance vs RDS Oracle. Depending on whether they have license for Data Guard or Active guard and/or Goldengate, how much downtime a customer can afford, network bandwidth, it becomes more of an art by selecting and recommending the right options. Generally speaking, Oracle native tooling is preferred over DMS if you are doing a homogeneous migration (Oracle to Oracle).CommentShareAWS-User-3253199answered 3 years ago"


Question: How to avoid double writing when publishing RDS logs to Cloudwatch Logs.

"I am publishing MySQL slow-query logs to CloudWatch Logs.When I look at CloudwatchLog, I see two logs with the same log.The same log is ingested at different times.$ aws logs get-log-events --log-group-name /aws/rds/instance/db/slowquery --log-stream-name db...    "events": [        {            "ingestionTime": 1660018515125,            "timestamp": 1660018510972,            "message": "# Time: 2022-08-09T04:15:10.972506Z\n# User@Host: user[user] @  []  Id: 158209\n..."        },        {            "ingestionTime": 1660021205551,            "timestamp": 1660018510972,            "message": "# Time: 2022-08-09T04:15:10.972506Z\n# User@Host: user[user] @  []  Id: 158209\n..."        },[5] pry(main)> Time.at 1660018515.125=> 2022-08-09 13:15:15 +0900[6] pry(main)> Time.at 1660021205.551=> 2022-08-09 14:00:05 +0900It sends once at 13:15 and again at 14:00 with the same log.I think RDS is probably sending twice from the same file, which setting should I change?FollowComment"


Question: How to delete a deprecated dataset I don't own anymore but still notifying me everyday?

"Hi,I used to own a quicksight dashboard and all dataset. But I left the team and already transferred all dataset to other member, and that team memeber left the team as well. This dashboard was deprecated recently and I got the dataset refresh failed notification everyday. I tried to delete it since an existing team member confirmed it is not needed anymore, but it showed that I have no view/modify access. The team member I talked to also does not own the dataset hence he could not delete them.Can someone share how to delete those dataset or stop the notifications?Thanks,-AliceFollowComment"

Answer: "2Hello Alice!To resolve your issue, you can ask a Quicksight admin to update the permissions for your dataset.First, they will need to verify who is the current owner of the dataset by running the below command from the CLI:aws quicksight describe-data-set-permissions --aws-account-id <account-ID> --data-set-id <dataset-id>[+] https://awscli.amazonaws.com/v2/documentation/api/latest/reference/quicksight/describe-data-set-permissions.htmlIf it belongs to a current employee, you can ask them to delete the dataset or share them with the admin. Please refer the below documentations,[+] Deleting datasets - https://docs.aws.amazon.com/quicksight/latest/user/delete-a-data-set.html[+] Sharing datasets - https://docs.aws.amazon.com/quicksight/latest/user/sharing-data-sets.htmlIf the dataset belongs to ex-employee, the admin can update dataset permission using the below:aws quicksight update-data-set-permissions --aws-account-id <account-id> --data-set-id <dataset-id> --grant-permissions file://permissions.json   [+] https://awscli.amazonaws.com/v2/documentation/api/latest/reference/quicksight/update-data-set-permissions.htmlThe content of permissions.json is similar to the permissions below:[   {      "Actions":[         "quicksight:DescribeDataSet",         "quicksight:DescribeDataSetPermissions",         "quicksight:PassDataSet",         "quicksight:DescribeIngestion",         "quicksight:ListIngestions",         "quicksight:UpdateDataSet",         "quicksight:DeleteDataSet",         "quicksight:CreateIngestion",         "quicksight:CancelIngestion",         "quicksight:UpdateDataSetPermissions"      ],      "Principal": "<full arn of new user>"   }] The new owner can later delete the dataset.You can also consider deleting the user account of an ex-employee to stop the notifications. When removing an inactive user from AWS QuickSight account, it will give an option to either delete or transfer it's resources to an active user present :Transfer ownership of all orphaned resources to a different userDelete all orphaned resources.Please refer the below documentation for more information:[+] Managing user access inside Amazon QuickSight - Deleting a user account - https://docs.aws.amazon.com/quicksight/latest/user/managing-users.html#delete-a-user-accountYou can also refer to the below article to manage Orphaned Resources of a Deleted Quicksight User:[+] https://aws.amazon.com/premiumsupport/knowledge-center/quicksight-access-resource-deleted-user/I hope the above helps!CommentShareSmrithi_Ranswered 4 months agoEXPERTiwasareviewed 3 months ago"


Question: How to create SSL dblink connection from RDS to postgres on premise?

We have an rds postgres instance and a remote postgres instance(on premise) we can only access via SSL connection.To create an SSL dblink connection in postgres one has to define SSL cert file locations on on the client.I haven't been able to find a way to upload SSL cert files to our RDS postgres instance in order to establish a dblink from our RDS postgres instance to remote postgres instance(on premise).Would it be possible to upload SSL cert files from remote instance somehow so that rds instance could have access to them?Please tell me if this is possible or if there are any alternative options.We would rather use a RDS instance than to manage our own on a EC2 instance.FollowComment

Answer: "0Hi, you can use postgres_fdw extension to setup encrypted in transit connection to foreign PG based database on premise. In order to enforce ssl please use sslmode in the connection string. You can find an example here:FDW uses CREATE SERVER to create foreign (remote) server mapping as the PostgreSQL-based server from which we pull the data. A foreign server typically encapsulates connection information that an FDW uses to access an external data resource. It uses the same connection options as libpq. SSLMODE ‘require’ makes sure that the data is encrypted in transit. See the following code:pgfdwsource=> create server my_fdw_target Foreign Data Wrapper postgres_fdw OPTIONS (DBNAME 'pgfdwtarget', HOST 'pgfdwtarget.xxxx.us-west-2.rds.amazonaws.com', SSLMODE 'require');Following blog will also be helpful for more information on postgres_fdw: https://aws.amazon.com/blogs/database/federated-query-support-for-amazon-aurora-postgresql-and-amazon-rds-for-postgresql/Unfortunately, as of now you cannot upload a custom certification to your RDS or Aurora environment.CommentSharevpanswered a year ago"


Question: Managed Ethereum Blockchain: waiting endlessly while Status = Creating ...

"Hi,The docs said it would take just 60 ninutes. I have been waiting for well over 2 hours and status still says "creating"Also, I had to delete and reinstall because i thought maybe it developed a memory leak. It was timing out on my request. It worked for a month or so before that.Any feedback as to whether I should trust AWS managed blockchain or just install and configure my own instance on EC2 would be highly appreciated.FollowCommentJC 4 months agoCan you please post a link to that doc?Share"

Answer: "0Well, so much for a manged service. My node state is stuck. Token and ETH balances are out of sync with Etherscan. I'm able to send transactions thru it just fine but its state is out of sync with Mainnet.CommentSharechaosmachine1answered 3 months agoJC 3 months agoThis is clearly not normal. Please contact support with your details so they can look into the situation (if you have not done it yet).Share0Creation can take over two hours depending on the state of AMB fast sync and how big your node is (if you pick a small node (8Gb), it will be slower).If you launch your own node on EC2, synchronization will take days.CommentShareJCanswered 4 months ago"


Question: Lightsail instance inaccessible.

"I was working on building something in FastAPI, dinking around with changing responses, when I got disconnected. Since then, the server has been completely inaccessible, and I have no idea why. It doesn't respond to ping, cant connect through SSH, nothing. I've rebooted, stopped/started, and it's like it's there but dead.Metrics just shows it idling, and I was able to take a snapshot, but I'm not sure what I can even do with the snapshot. I've checked other posts, and I don't really have any idea what went wrong. I wasn't messing around with the firewall or anything that should have blocked my access.FollowCommentAlex Jordan 5 months agoCreated a new instance from snapshot, same behavior.Trying an export to EC2 to see if I can at least recover unsaved work.Share"

Answer: "0Hi, @Alex JordanUnfortunately, Lightsail has limited troubleshooting possibilities as per the following documentation:You need to export to EC2 for troubleshooting.If you can't connect after exporting to EC2, try standard EC2 troubleshooting, such as using EC2Rescue.you might need to migrate the Lightsail server to EC2 for troubleshooting further. This is because troubleshooting options are limited in Lightsail as it is for simpler workloads.https://aws.amazon.com/premiumsupport/knowledge-center/lightsail-instance-failed-status-check/?nc1=h_lsCommentShareEXPERTiwasaanswered 5 months ago"


Question: MySQL 8.3.0 storage is going to full quickly

Our database MySQL 8.30 size is 100Gib but the actual data size is 2Gib stored. Do not know what is taking storage space.How can we reclaim the storage?FollowComment


Question: How does AWS reconfigure it's network infrastructure on-the-fly?

"As a former CCNA holder and someone who has both worked for Internet Service Providers as well as visited some of the very data centres that AWS likely operates out of (though it's of course impossible to confirm), I'm curious as to how exactly AWS manages the impressive feat of reconfiguring it's network programatically.The term Software Defined Networks spring to mind.  But why is there so little transparency about it?  Security through obscurity?  I thought that died out a long time ago.    It seems like AWS doesn't talk about it.  Is this AWS policy?FollowCommentDavid Malone EXPERT10 months agoWhile it is an older video, this re:Invent session is informative on the topic of VPC: https://www.youtube.com/watch?v=Zd5hsL-JNY4. I would suggest looking for other past re:Invent sessions on the topics of networking, VPC, and global infrastructure for more information related to your questions.Share"

Answer: "1Accepted AnswerAdding to David's comment. Definitely check out the later version of that same talk because it does go into detail about how AWS builds the network that customers use. But also check out James Hamilton's Innovation at Scale video which is older but goes into other detail including EC2, EBS, etc.There are dozens (if not more) videos on YouTube that go into our technology in depth. Consider reading the Amazon Builder's Library because it talks to a lot of the lessons and patterns that we use at scale.Of course, there are many things we don't talk about because they are unique to AWS. But as a customer you should talk to your local AWS Solutions Architect - they may be able to answer more specific questions.CommentShareEXPERTBrettski-AWSanswered 10 months ago"


Question: Can I use Third party APIs with Amazon Connect

Amazon connect supports things like Amazon polly for converting text to speech. Is it possible to replace polly with a third party text to speech like Google's API? I want to use Amazon connect but want to replace parts of the infrastructure with 'non-Amazon' APIs.FollowComment

Answer: "0I am sure you can do it, but not sure you will want to. Here's how I see this happening:Identify what you want to say and create string "Hello and thank you for calling.."Have Lambda send this string to Google to generate an audio file, which will then get pushed to an S3 bucket, which will then provide you with an ARN.Save ARN location as an attribute.Play ARN location in Connect.This feels like unnecessary steps which might cause delays in your flow and not a great customer experience. If your prompts are pretty static could you just generate them all in Google and then put them in an S3 and only use Polly for dynamic information, like dates or values?CommentSharedmaciasanswered 3 months agoEXPERTAWS-samfredreviewed 2 months ago0You cannot replace the native TTS engine (AWS Polly) in Amazon Connect.As mentioned, best bet is to use Play Prompt from S3 feature so you can update the prompts dynamically built from other TTS engine to those locations as neededCommentShareClarenceChoianswered 3 months ago"


Question: "Clientvpn, error on linux client when adding route number 63"

"Hi, we have a very rare problem with aws clientvpn, we have 62 routes/authorizations and the service works fine, we have windows clients with clientvpn software and linux clients with openvpn software.But, when we add the route number 63, windows clients go fine, but linux clients (all of them) fails with messages like this:...55.0,route 10.53.4.0 255.255.255.0,route 10.1.124.0 255.255.255.0,route 10.105.0.0 255.255.0.0,route 172.25.246.0 255.255.255.0,route 172.24.191.0 255.255.255.0,route 172.25.182.0 255.255.255.0,route 10.55.36.0 255.255.255.0,route 10.53.132.0 255.255.255.0,route 172.25.196.0 255.255.255.0,route 172.25.76.0 255.255.255.0,route-gateway 10.200.0.129,topology subnet,ping 1,ping-restart 20,ifconfig'2022-03-24 09:14:12 Options error: Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:68: ifconfig (2.5.6)2022-03-24 09:14:12 OPTIONS IMPORT: timers and/or timeouts modified2022-03-24 09:14:12 OPTIONS IMPORT: --ifconfig/up options modified2022-03-24 09:14:12 OPTIONS IMPORT: route options modified2022-03-24 09:14:12 OPTIONS IMPORT: route-related options modified2022-03-24 09:14:12 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified2022-03-24 09:14:12 Using peer cipher 'AES-256-GCM'2022-03-24 09:14:12 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key2022-03-24 09:14:12 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key2022-03-24 09:14:12 net_route_v4_best_gw query: dst 0.0.0.02022-03-24 09:14:12 net_route_v4_best_gw result: via 192.168.0.1 dev enx000ec675f0d52022-03-24 09:14:12 ROUTE_GATEWAY 192.168.0.1/255.255.255.0 IFACE=enx000ec675f0d5 HWADDR=00:0e:c6:75:f0:d52022-03-24 09:14:12 TUN/TAP device tun0 opened2022-03-24 09:14:12 WARNING: OpenVPN was configured to add an IPv4 route. However, no IPv4 has been configured for tun0, therefore the route installation may fail or may not work as expected.2022-03-24 09:14:12 net_route_v4_add: 10.203.0.0/16 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 10.204.0.0/16 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 10.202.0.0/16 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 10.52.12.0/24 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 10.24.0.0/24 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 10.201.0.0/16 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 10.53.8.0/24 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 172.24.224.0/24 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 10.1.28.0/24 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed2022-03-24 09:14:12 net_route_v4_add: 172.24.0.0/24 via 10.200.0.129 dev [NULL] table 0 metric -12022-03-24 09:14:12 sitnl_send: rtnl: generic error (-101): Network is unreachable2022-03-24 09:14:12 ERROR: Linux route add command failed...Any help is welcome, thanks in advance!Bye,FollowComment"

Answer: "0Hello Angel,Thank you for reaching out. To clearly understand and bifurcate,You are using CVPN endpoint and have the below client software's:Windows Client- AWS ClientLinux Client- Open VPN ClientWhen you add more than 63 routes you face an issue.If I understand your issue correctly, the best place to ask the questions regarding the same & with the detailed logs would be the Open VPN portal where you can post the same and connect with a wider audience using the same. Since it is not an AWS controlled client, it is very difficult to try and reproduce the same.To the best of my knowledge, this should not happen on the linux client if you have the correct routing command unless there is some device level issues blocking the same due to internal settings.Also, the client logs has been taken on the device or the Open VPN client ? Is there any detailed logs that can be enabled and pushed to be see details ?Also, try checking the device level logs to check for limits. Adding to that, I can also see that the limits mentioned on the routing are not OS dependent and are common for both on the Client VPN endpoint side.https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/limits.htmlHope this helps. Thanks & stay safe. Feel free to reach out in case of any queries.Thanks & stay safeCommentShareSUPPORT ENGINEERAWS-User-Chiraganswered a year ago"


Question: Is there an option to create an EFS from an EBS snapshot?

"We currently have an instance on EC2 classic with an EBS with a directory with at least 1Tb of data. We mount that directory with Gluster on different instances.Now that EC2 Classic is ending we want to migrate the instance to a VPC and stop using Gluster in favour of EFS. Is it possible to use a snapshot of the EBS containing the data to generate an EFS from it?We haven't found any information about this in the documentation and/or internet and it seems that the only option is to launch an instance from the snapshot in the VPC, mount the EFS and transfer the data (rsync or cp...).FollowComment"

Answer: "1Accepted AnswerThe best options here are:Enable NFS Server on a Linux system that has the EBS volume mounted, and then use DataSync to copy from the resulting NFS share.Copy from the client side with rsync/cp/rclone.CommentSharekdavydanswered a year agoImobach a year agoWe want to avoid the option 2 you suggest, but option 1 could be a solution, thank you for your suggestions.Share1I am not an expert in that domain but maybe AWS DataSync can help you out here? if u can use it to sync data from on-prem to EFS it might also be helpful here.CommentShareMarcoanswered a year agoImobach a year agoI don't know how DataSync works internally and how it handles errors (in the transfer of 1Tb there can be many), we will have to study it. Thanks for the contribution.ShareImobach a year agoI have been doing some research and AWS DataSync does not support EBS, these are the supported types:Network File System (NFS)Server Message Block (SMB)Hadoop Distributed File System (HDFS)On-premises (self-managed) object storageAmazon EFSAmazon FSx for Windows File ServerAmazon FSx for LustreAmazon S3There doesn't seem to be an option for network files mounted with GlusterFS or else EBS disks on EC2 instances.Share"


Question: aws ec2 run-instances with offline and unallocated disks

"Hi Community We have created a new ec2 instance with 3 drives by aws ec2 run-instances as below.The 3 volumes have been created and were shown as attached in the aws console ec2 volumes screen.Nevertheless on the server in Disk Management, only Drive C: was available, the additional 2 drives were offline and unallocated:We needed to take them online, initialize and allocate by Windows tools.Does anybody know how we make them online and allocated by an aws bash command?Thanks!Patrickaws ec2 run-instances --instance-type t2.xlarge --image-id ami-0c6e3ed6b1dd28a39 --block-device-mappings file:///home/cloudshell-user/mapping.json --key-name xxx --security-group-ids xxx --subnet-id xxx --instance-initiated-shutdown-behavior stop --disable-api-termination mapping.json: [ { "DeviceName": "/dev/sda1", "Ebs": { "DeleteOnTermination": true, "VolumeSize": 70, "VolumeType": "gp2" } }, { "DeviceName": "/dev/sdh", "Ebs": { "DeleteOnTermination": true, "VolumeSize": 80, "VolumeType": "gp2"    } }, { "DeviceName": "/dev/sdi", "Ebs": { "DeleteOnTermination": true, "VolumeSize": 80, "VolumeType": "gp2"    } } ]FollowCommentRodney Lester 8 months agoIn order to accomplish this, you will have to provide user data with the powershell commands to mount and format the disks and then allocate them to drives as you wish. I am not well versed in powershell to provide the commands.Share"

Answer: 0Hi RodneyThanks for your comment. Do you mean I should use the '--user-data'-parameter in 'aws ec2 run-intances' to start a powershell script to mount and  attach drives?Thanks! PatrickCommentSharerePost-User-6866822answered 8 months ago


Question: restoring redshift serverless from provisioned cluster snapshot

"I recently enabled redshift serverless endpoint, and to benchmark it I want to restore it from a snapshot of a legacy provisioned cluster, so we have identical data sets to start with.However, when I select to restore the snapshot to the serverless (using console as described in https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-snapshots-recovery.html, and confirming that it will replace all my serverless databases), nothing seems to happen (no error, but also the serverless databases are not replaced with the snapshot).I already tried adding redshift full access policy to the IAM role of the serverless endpoint, but that didn't help.Has anyone already achieved this?FollowComment"

Answer: "0Kindly review that either the administrator permissionarn:aws:iam::aws:policy/AdministratorAccessor the full Amazon Redshift permissionarn:aws:iam::aws:policy/AmazonRedshiftFullAccessis attached to the IAM policy that you used. If true, then open a Support Ticket for assistance.CommentShareMilindOkeanswered a year agoAWS-User-7164347 a year agoAs indicated in my initial post, I have already attached both the arn:aws:iam::aws:policy/AmazonRedshiftFullAccess and the arn:aws:iam::aws:policy/AmazonRedshiftDataFullAccess policies to the IAM role, but that did not make any difference.Share"


Question: Cloudformation Stack Update Fails Due to Duplicate Tags Errors on IAM Role Resource

"My stack has some tags that I can see on the "Stack Info" dashboard. The IAM Role resource I am trying to add to the stack does not use any tags. Nevertheless, I get the following error during the stack update at the Role's creation:Duplicate tag keys found. Please note that Tag keys are case insensitive. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: InvalidInput; Request ID: 2e1c1359-cfa9-4f6b-9571-0b302df35355; Proxy: null)I don't really know what could be causing this. Thanks for your help!FollowComment"

Answer: "1Accepted AnswerHi,The error message "Duplicate tag keys found. Please note that Tag keys are case insensitive" occurs when there are duplicate tag keys in the IAM Role resource being created, even if the keys have different cases. Check this out:https://docs.aws.amazon.com/it_it/IAM/latest/UserGuide/access_tags.htmlBTW, this error can occur even if the IAM Role resource does not use any tags. To prevent duplicate tags with a key that varies only by case, use the "aws:TagKeys" condition to define the tag keys that your users can apply, or use tag key-name condition key to specify what tag key-value pairs can be passed in a request to tag or untag an AWS resourceYou can check all the resources in the stack that have tags defined and verify if any of them have duplicate tag keys. To do this, you can navigate to the CloudFormation console, select your stack, and view the "Resources" tab. From there, you can expand each resource and look for tags.CommentShareStefanoBanswered 22 days agoJuan Linde 22 days agoThank you Stefano. There seems to be duplicate keys in our json config file. Do you know how to define "stack-level" tags inside the yml file?Share0Hi,To define stack-level tags in a CloudFormation YAML file, you can use the Tags property at the root level of your template. Here's an example:AWSTemplateFormatVersion: '2010-09-09'Description: Your CloudFormation stack descriptionTags:StackLevelTag1: StackLevelValue1StackLevelTag2: StackLevelValue2Resources:...Make sure to replace StackLevelTag1, StackLevelValue1, etc. with your actual tag key-value pairs. Remember that stack-level tags are not specific to any particular resource. They are used to provide metadata at the stack level. Also, remember to adjust the resource type (AWS::EC2::Instance in the example) and properties according to your resource's definition.CommentShareStefanoBanswered 19 days ago"


Question: Cannot create arn:aws:iam::XXXXXXXXX:role/service-role/AWSBatchServiceRole

"Hello,I tried to create the service-linked role for Batch following this link (9 steps at its bottom):https://docs.aws.amazon.com/batch/latest/userguide/service_IAM_role.htmlbut I saw the following error after clicking on the "Create role" button (the 9th step):"Failed to create role AWSServiceRoleForBatch.Role name prefix AWSServiceRoleFor can only be used for AWS Service Linked Roles"I need this role to delete an invalid compute environment as explained by this link:https://repost.aws/questions/QUQDKKXZjnQZGwAAe6bQYB8g/how-to-delete-compute-environment-that-is-invalidThe invalid compute envrionment is showing this error:"statusReason": "CLIENT_ERROR - User: batch.amazonaws.com is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::XXXXXXXXXXXX:role/service-role/AWSBatchServiceRole (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: cab999c4-f3a6-4af2-8786-bad3a55b3109; Proxy: null)",Could someone help? Thanks!FollowCommentrePost-User-3662849 4 months agoI found a workaround......since the name ''AWSBatchServiceRole" is reserved, I change it to a different name "my_AWSBatchServiceRole". Then in the console, I went to AWS Batch/Compute environments, selected the invalid compute environment, clicked on the "Edit" button, and changed its associated role from "AWSBatchServiceRole" to "my_AWSBatchServiceRole". After doing this, I was able to delete it successfully.Share"

Answer: "0Are you deploying this in an AWS Organization? Have you checked if the service is enabled or not? This is an error that AWS returns when integrations are disabled at org level. Also check if the role already exists, these roles usually are created automatically by AWS when you use the service.CommentShareesgrano-evolateanswered 4 months agorePost-User-3662849 4 months agoThanks, esgrano. I am not deploying this in an AWS Organization. The AWSBatchServiceRole role does not exist in my account now.Share0Ok, I finally found that the right way of creating it is using CLI.........aws iam create-service-linked-role --aws-service-name batch.amazonaws.comThe arn of the role created this way will be different though:arn:aws:iam::XXXXXXXXXXXX:role/aws-service-role/batch.amazonaws.com/AWSServiceRoleForBatchCommentSharerePost-User-3662849answered 4 months ago"


Question: "Access denied for operation 'AWS::QuickSight::Analysis'."

"Hello,When using the cloudformation template in this blog, I receive access denied when the Amazon Quicksight analysis is to be created using the AWS provided template ARN: arn:aws:quicksight:us-west-2:111530706466:template/testconnectquicksight-CtrTemplateI tried using the CLI to describe template and list templates from that account but neither work. Is this template still available for use?FollowComment"

Answer: "0Hello, I hope that you are well.Please be informed that I was able to follow the blog and successfully create a Cloudformation Stack using this template. I first downloaded the template to my local machine the deployed it Cloudformation. The template creates one resource of Type:' AWS::QuickSight::Analysis' that has the logical name "QuickSightAnalysisCtr".Please kindly note that 'Access Denied' errors indicate underlying permission related issues. Access Denied errors can be caused by multiple factors such as the Role missing the required permissions to create a QuickSight Analysis resource. Please also take a look at the following document to learn more about troubleshooting Access Denied errors.I would also suggest that you check CloudTrail for any failed  "CreateAnalysis" API calls/events. Please kindly refer to the following document to learn more about how you can review CloudTrail Event history using the CloudTrail console or the AWS CLI.For further analysis of this issue I would suggest that you open a support case with the Cloudformation team so that we can investigate the issue in depth by checking the affected resources.CommentShareSandisiwe_Manswered a month ago"


Question: "Instance restart frequently, but status check is not, and the software I installed will disappear"

"My instance restart frequently(when I log in again, up time is short. Status check is normal), and the software I installed will disappear. For example, I upgraded python to 3.8, but it changed back to 3.7Instance info:Storage:Monitor:Uptime in console:FollowComment"

Answer: "0Accepted AnswerIs it the same instance or a replacement one - does the instance ID change?Are you using EC2 autoscaling or Spot instances?CommentShareEXPERTskinsmananswered a month agoawser a month agoThanks for your answer. I'm not sure if the instance id has changed, I'll record it. It doesn't look like autoscaling, because the files I uploaded are always there, but every time I upgrade the software, it will be lostShareskinsman EXPERTa month agoI think you need to provide more information about your configuration.  What sort of instance is it and what disk volumes does it have.  What tpe are those volumes?  Where are you uploading files to on your instance?Shareawser a month agoHi skinsman, I uploaded screenshots in the question. I used the git clone to upload the files in /home/cloudshell-userShare0When you upload files to /home/cloudshell-user, that's in your CloudShell persistent storage - a 1GB allocation that most likely is kept in S3.  It doesn't have anything to do with your EC2 instance so termination/replacement of your EC2 instance won't affect it.The software upgrades you do on the instance though are tied to the instance - so it looks like your instance is being replaced.  If the Instance ID changes then that will confirm it.CommentShareEXPERTskinsmananswered a month agoawser 21 days agoOh yes. I thought cloudshell was specially used by aws to log in to instances. Thank you for your helpShare"


Question: Querying Athena from Lambda function - QUEUED state?

"Hi there,I've been successfully querying s3 via athena from inside a lambda function for quite some time but it has suddenly stopped working. Further investigation shows that the response from get_query_execution() is returned a state of 'QUEUED' (which i was led to believe is not used?!)My code snippet is as follows:while (max_execution > 0 and state in ['RUNNING']):max_execution = max_execution - 1print("maxexecution=" + str(max_execution))response = client.get_query_execution(QueryExecutionId = execution_id)    if 'QueryExecution' in response and \              'Status' in response\['QueryExecution'] and \              'State' in response\['QueryExecution']\['Status']:              state = response\['QueryExecution']\['Status']\['State']              print(state)              if state == 'SUCCEEDED':                  print("Query SUCCEEDED: {}".format(execution_id))                  s3_key = 'athena_output/' _ execution_id _ '.csv'                  print(s3_key)                  local_filename = '/tmp/' _ execution_id _ '.csv'                  print(local_filename)                  rows = \[]                  try:                      print("s3key =" + s3_key)                      print("localfilename = " + local_filename)                      s3.Bucket(BUCKET).download_file(s3_key, local_filename)                      with open(local_filename) as csvfile:                          reader = csv.DictReader(csvfile)                          for row in reader:                              rows.append(row)                  except botocore.exceptions.ClientError as e:                      if e.response\['Error']\['Code'] == "404":                          print("The object does not exist.")                          print(e)                      else:                          raise                  return json.dumps(rows)              elif state == 'FAILED':                  return False      time.sleep(10)  return False  Obviously it is returning false as i don't cater for get_query_execution() returning a state of 'QUEUED' - what can cause it to return this state? From the doco i've seen this state isn't supposed to be in use yet? How should this be handled? is it as simple as adding 'QUEUED' to the while condition? e.g.     while (max_execution > 0 and state in ['RUNNING', 'QUEUED'])FollowComment"

Answer: "0Hi jaldons,David here from AWS. I'm a Premium Support engineer from the Big Data team, working with EMR, Athena and other related services.The Athena team recently deployed a host of new functionality for Athena, including more granular CloudWatch metrics for Athena queries.For more information:AWS What's New page: https://aws.amazon.com/about-aws/whats-new/analytics/?whats-new-content.sort-by=item.additionalFields.postDateTime&whats-new-content.sort-order=desc&awsf.whats-new-products=general-products%23amazon-athenaAthena docs on CloudWatch metrics: https://docs.aws.amazon.com/athena/latest/ug/query-metrics-viewing.htmlAs part of the deployment of more granular metrics, Athena now includes a "QUEUED" status for queries. This status indicates that an Athena query is waiting for resources to be allocated for processing. Query flow is roughly:SUBMITTED -> QUEUED -> RUNNING -> COMPLETED/FAILEDNote that queries that fail due to system errors can be put back into the queue and retried.I apologise for the frustration that this change has caused.It seems like the forum formatting has stripped some elements from your code snippets.However, I think that your WHILE loop is working on an array of the possible query statuses, which didn't previously cater for "QUEUED".If that is the case, then yes, adding "QUEUED" to that array will allow your application to handle the new status.Is this affecting a production system in your environment?Please feel free to message me privately with more details, and I can see what I can do to help you.Kind regards,David F.CommentShareAWS-User-1319882answered 4 years ago0Thanks for that - at least it means i'm not going crazy! Is there some way in which these sort of changes are communicated normally? Luckily the particular API's i'm dealing with aren't in production yet but it would have been disastrous if we were a month further along... luckily it's an easy fix but knowing before it breaks would be preferableCommentSharejaldonsanswered 4 years ago"


Question: Tape auto-create feature isn't creating tapes.

"HelloI've been trying to automate our off-site backup using Veeam+AWS Storage Gateway appliance. In my aws console within the storage gateway service, I have the Tape auto-create feature configured on my gateway, yet it never creates new tapes. My tape job in Veeam stalled over the weekend because it ran out of tapes. I had to log in to the console and create new tapes before Veeam could import them and resume the backup job. I must be missing something, please advise.screenshot of auto create settings:https://ashlandfood.coop/awstapes.pngFollowComment"

Answer: "0Hi,Thanks for reaching out, it looks like it should be workingWe're aware of a few cases that can cause the auto-creator to get stuck. Temporarily, a reboot should fix it, https://docs.aws.amazon.com/storagegateway/latest/userguide/MaintenanceShutDown-common.html but we expect this to be fully solved in the next releaseFeel free to PM me with your gatewayId + region and we'll be happy to take a closer lookThanksCommentShareAWS-User-4017914answered 3 years ago0Thanks for the response!I've gone ahead and shut down the gateway using the method detailed in the link you provided. I then rebooted the appliance vm, then went back to my aws console and restarted the gateway. It is running, but did not create new tapes. I opened the auto tape create configuration and deleted my old entry, and made a new entry for 6 800gb Glacier tapes. Still no tapes created.I'll PM you my gateway details.Thanks for your assistance!CommentSharegcampbellanswered 3 years ago0I waited for a few minutes, but did not wait long enough apparently, as I now can see the auto created tapes.Thankyou for your assistance and disregard my PM. Have an awesome day!CommentSharegcampbellanswered 3 years ago"


Question: Timestream Historical Data (10s of years old)

"Hi,(Quite new to timestream, old hat with SQL)I've added Millions (80M+) rows of data to a timestream table around 3 days ago.  The data is historical, and goes back to the late 1800s.When querying it with:SELECT * FROM <database>.<table> ORDER BY time ASC LIMIT 1The earliest record it shows 'time' is in the year 2016 (this is actually around the date of the first record inserted into the table), there are however millions of records before this date (but these were added later to the table), I've confirmed all the records exist in the table by running:SELECT count(id) FROM <database>.<table>Which results in a count equal to all the rows I've added (80M+)The table is setup to hold data in memory for 6 months, and in magnetic store for 200 years.  Magnetic writes are also turned on.Why does this behaviour happen? are query results or records cached/indexed by timestream in some way, if so is there a way to clear this cache or documentation about it?  Is it the normal behaviour of the magnetic store, as its such a large dataset added recently, is magnetic store still organising it?Thanks,JamesFollowComment"

Answer: "0Hi James, I am Victor from the Timestream PM team, we would love to learn more about your issue, would you please  file a customer support ticket so we can follow up from there?ThanksVictorCommentShareVictorJSanswered a year agoJames-ReworkDigital a year agoHi Victor, actually I think I worked around this original issue, my fault as I hadn't noticed an error..  But it has lead to another related problem, very similar.  When adding data from pre 1970 (epoch), the data is rejected, even if we try and add a time with a minus value, for example adding the time with value:-2169358250000 (Thu Apr 04 1901 16:29:10)We're trying to add historical (and live) precious metal prices to timestream..Share0Hi James, unfortunately Timestream as a service only supports seconds offset from Epoch. Are you planning to do time series analysis going all the way back to late 1800s?CommentShareVictorJSanswered a year agoJames-ReworkDigital a year agoHi Victor, we are yes, we are using it to store historical and current prices for gold and other precious metals..Share0Oh ok, I get it. Even when you could use multi-measure records since it provides support for having a timestamp as a measure, by doing so you will miss on most of the benefits of indexing on the time column offered by Timestream. Sadly there is not much we can do at this time, but if you file a customer support ticket we will be able to track it and reach out to you in the future if some work around becomes available. My apologies for not being able to help you at this time and thank you for reaching out.CommentShareVictorJSanswered a year ago"


Question: "I am using javascript v3 aws-sdk to communicate with AWS-API, but intermittent delay occurs in certain regions."

"Hello, I am a developer working for an AWS Partner. I've been using javascript v3 aws-sdk to get information about customers' use of AWS services.ex)  const ec2Client = new EC2Client({                            region: ec2Region,                            credentials: awsCredentials,                            tls: false,        })        // EC2        ec2Client.send(new DescribeInstancesCommand({}))I'm experiencing intermittent communication delays when requesting a URL for the 'eu-west-1' region, which is the Ireland region among several regions.The region making the AWS API request is Seoul, South Korea, and several services in other regions are communicating smoothly.Communication delays to the 'eu-west-1' region are critical, as customers must periodically update their usage of multiple services.Please help me troubleshoot the 'eu-west-1' region communication issue.FollowComment"


Question: There is an error that changes to "??" when I type Korean on the server.

"There is an error that changes to "??" when I type Korean on the server.I wonder if the hosting server environment is UTF-8.It seems that the server doesn't accept Korean, so I want to know the solution.FollowComment"


Question: Permission denied on S3 path: s3://artless-reports-prod/ServicesInScopeReport/year=2022/month=11/day=13

"Hello team,We are getting this error and cannot pull the query. Are you able to assist?Permission denied on S3 path: s3://artless-reports-prod/ServicesInScopeReport/year=2022/month=11/day=13This query ran against the "artlessetl" database, unless qualified by the query. Please post the error message on our forum  or contact customer support  with Query Id: 5c118429-7134-4469-9198-2e24069fcc22FollowComment"

Answer: "1There are a few things that could cause this. you could try these steps:Did you provide read permissions to your S3 bucket for the user - both at bucket and inside the bucket (eg."Resource": ["arn:aws:s3:::artless-reports-prod", "arn:aws:s3:::artless-reports-prod/"])? You can start with "s3:Get" and "s3:List*".Are your bucket and objects encrypted? If your bucket and objects are encrypted, I would make sure Lakeformation has the proper access to use the KMS Key (KMS Key Policy).Do you use Lakeformation? if so, has the data location registered for the datalake bucket/folder in LakeFormation?If the bucket is owned by the same account you are querying from, check out https://docs.aws.amazon.com/athena/latest/ug/s3-permissions.html.For cross account access, check https://docs.aws.amazon.com/athena/latest/ug/cross-account-permissions.htmlFor federated queries, check https://docs.aws.amazon.com/athena/latest/ug/federated-query-iam-access.htmlReference: https://aws.amazon.com/premiumsupport/knowledge-center/access-denied-athena/CommentShareAnanth Tirumanuranswered 5 months ago"


Question: Hide sensitive data in CloudWatch logs

Hello. We have enabled logging from RDS instance to the CloudWatch. I have noticed that some sensitive info like certificates and private keys are appearing in CloudWatch logs. I would like to hide this type of logs so it won't be shown in CloudWatch. Could you please advise how can I do it? Unfortunately I can not use Data protection policy because it's not supported in our AWS RegionFollowComment

Answer: "0Hi, thanks for reaching out!As I understand it, RDS logs cannot be scrubbed prior to sending to CloudWatch. As well, once the logs are in CloudWatch, and without the benefit of the CloudWatch Logs data masking feature available in enabled regions, hiding specific sensitive data already delivered to CloudWatch Logs is not straightforward as individual log messages cannot be outright deleted from a log stream. The minimum level of granularity for Delete calls is for an entire log stream.As a baseline level of protection, I would advise ensuring that log groups that may contain sensitive data be encrypted using the AWS Key Management Service. As well, IAM permissions can be adjusted such that only the IAM roles you specify can access log groups containing sensitive log data.CloudWatch Logs Actions, resources, and condition keysIdentity-based policies for CloudWatch LogsCloudWatch Logs Insights can be configured with saved queries that only return the log messages you wish to see while excluding any log messages containing sensitive data. However, the log messages with the sensitive data would still be available to be viewed by altering the query or viewing the log group directly by anyone with access to it.CommentShareSUPPORT ENGINEERJustin_Banswered 2 months ago"


Question: 2/2 checks passed but Spring Boot app with SQL database deployed on EC2 Ubuntu fails to load and connection times out

"I deployed a full stack Spring Boot application on an Ubunut server with AWS EC2, but even though the site would initially load, I have found that the connection is not consistent and more often than not, I cannot connect to the site and the connection times out. The strange thing is that the server is passing the checks. I am not sure what to do to diagnose/resolve the connection issue. Is there a better place to deploy Spring Boot applications?FollowComment"

Answer: "0Are you getting issues under heavy load or even with simple functionality testing? If it is the latter, you should look at the Springboot application logs to see if the Springboot application is getting the requests and running into issues or not.If you are getting issues under load, you will need to vertically scale your EC2 instance or better, run an auto-scaling group of EC2 machines with an Application Load Balancer in front, so your EC2 machines can scale with load. You could set that up yourself, or else use Elastic Beanstalk that does lot of the heavylifting for you as illustrated in this blog - https://aws.amazon.com/blogs/devops/deploying-a-spring-boot-application-on-aws-using-aws-elastic-beanstalk/. Note that this blog is old so some screens may have changed.You have other options as well on AWS. You could run your Springboot application as a containerized application and use ECS or EKS to orchestrate. You can get guidance from the following documents - https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/deploy-java-microservices-on-amazon-ecs-using-amazon-ecr-and-aws-fargate.html, https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/deploy-java-microservices-on-amazon-ecs-using-amazon-ecr-and-load-balancing.html and https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/deploy-a-sample-java-microservice-on-amazon-eks-by-using-amazon-ecr-and-eksctl.htmlThe third option is to deploy your Springboot application on AWS Lambda. You can refer to AWS samples that show you how to run Java applications built using frameworks like Springboot, Micronaut and Quarkus using GraalVM to reduce cold starts - https://github.com/aws-samples/serverless-java-frameworks-samplesCommentShareEXPERTIndranil Banerjee AWSanswered 7 months agoEXPERTTushar_Jreviewed 7 months ago"


Question: AWS CCP Certification Exam Voucher Invalid

"Hi,I attended the AWESOME program and was issued a voucher for 50% discount towards the cost of my exam and upon using this code, error response I got was "Voucher Invalid"I tried registering on Pearson VUE. Could you please help look into this?Thank YouFollowComment"

Answer: 0you could probably contact Xvocuher (https://help.xvoucher.com/general) if the voucher is issued by them.CommentShareSrianswered a year ago