Question: "Can a specific partition key be enforced for the Kinesis PutRecord action, via IAM policy or other means?"

"Given a producer sending records to a stream by calling the PutRecord action, is there any way to enforce that the producer is calling PutRecord with a specific partition key?I've checked this documentation page Actions, resources, and condition keys for Amazon Kinesis, and it does not seem like a condition exists to allow an IAM policy to restrict the partition key.FollowComment"

Answer: "1Hello,I understand that you have a query regarding enforcing a specific partition key for the Kinesis PutRecord API.Firstly I would like to mention that currently the IAM action - PutRecord doesn't support any condition keys. Also, it only support resource restriction in terms of stream type -> arn:${Partition}:kinesis:${Region}:${Account}:stream/${StreamName}Secondly, please note that partition key is a mandatory parameter when using PutRecord/PutRecords API call [1]. PartitionKey    Determines which shard in the stream the data record is assigned to. Partition keys are Unicode strings with a maximum length limit of 256 characters for each key. Amazon Kinesis Data Streams uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.    Type: String    Length Constraints: Minimum length of 1. Maximum length of 256.    Required: YesAs there is currently no way to restrict it from API's end using condition keys, you will have to create a custom implementation in your own application code to enforce the usage of a specific/single partition key restriction when calling the PutRecord API, however please note that the records will always land to a single shard in such a use case.Finally, I also wanted to mention that the best practice provided by service team in regards to the partition key, is to use random partition keys such as uuid.References:[1] https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.htmlCommentShareSUPPORT ENGINEERYash_Canswered a year ago"


Question: How to prevent disassociating SageMaker LifecycleConfig unintentionally

"When you go to SageMaker Notebook Instance edit screen in AWS Web Console (to change the Instance Type for example), it is sometimes the case that Lifecycle configuration is popped up as No Configuration even though the configuration is actually set earlier. This results in an unintentional disassociation of the LifecycleConfig because it's easy to save the instance change without noticing the change in Lifecycle Config. This is a serious problem for us. I was able to reproduce this issue in Chrome and Firefox (but you need to try several times to repro the issue).I am in the position of provisioning different cloud resources for the end users and I need a way to systematically prevent this disassociation to happen. I considered applying an IAM policy that denies the update operation containing the change in the LifecycleConfig of notebooks, but there seems no condition key for LifecycleConfig which makes me think this approach isn't feasible.What can I do?Thanks.FollowCommentDurga_S a year agoI can see this issue as well - intermittently it defaults to No configuration when updating a notebook instance. From your question, it sounds like you still want to allow the user to update the instance types and such. Can you raise a ticket with support?Share"


Question: ECS Tasks CPU hard limit without increase of cpu reservation

"Hi :)I am currently trying to resolve an issue with our ECS EC2-based cluster. Our Task definitions utilize the container-level soft CPU limit (set to 50 CPU units), but dont utilize the hard CPU limit from task-definition level. In our case we have more than 2000 services running, each one with a single task. These tasks have new revisions being re-deployed at a very high rate and at the same time, and upon startup they often reach levels of CPU usage above 1000%. This causes the whole EC2 instances to become unresponsive, resulting in a need to restart the whole machine. This we have solved with an alarm and a lambda for quickly rebooting failing, unresponsive instances.But this is not a solution, only a temporary fix. What we would like to achieve is to somehow limit the CPU usage of each task to not exceed the soft limit by factors of 10 or more. I have found a way to do it using the hard CPU limit, but this solution is also not great, mainly for the following reasons:*Our tasks even with the soft limit use at most 50% of this reservation, but the hard limit's minimum value for ECS with EC2 is 128 units (compared to current 50)The hard limit automatically increases the reservation value for the chosen task, meaning that setting the limit to 128 for all 2000+ servcies/tasks would require us to host more than double the amount of EC2 machines, without an actual gain, as our cluster's usage right now hangs around 5-10%.So my question is, is there a way to somehow limit the maximum cpu usage of each task/container without using the task-level hard limit? Our EC2 machines are running on UbuntuFollowComment"


Question: How long to receive soft bounce email reply

We tried sending an email on AWS SES. We received a soft bounce response from SES after several hours. How long does AWS SES send a soft bounce reply?FollowComment

Answer: "0Accepted AnswerFrom this AWS blog post:When a soft bounce event happens, SES continuously retries to deliver your email for 12 hours before giving up. There is no limit of retries during this 12-hour interval, and, as long as the ISP recovers, your email will be delivered.From the SES documentation, soft bounces ("Transient") can occur for a variety of reasons. It is recommended that you monitor your bounces and take action based on the bounce types by analyzing the bounceType and bounceSubType fields.CommentShareAaron Doddanswered 9 months ago"


Question: RDS SQL Server request more storage

Amazon RDS SQL Server DB instances have a limit of 16 TiB of storage.Is posible request more storage with AWS Support?FollowComment

Answer: 016 TiB is hard limit and cannot be increased even by support team https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SQLServer.html#:~:text=The%20maximum%20storage%20sizeCommentShareAWS-User-8174525-subhashanswered a year agoRodney Lester a year agoHe is correct. See https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.KnownIssuesAndLimitations.html#MySQL.Concepts.Limits.FileSizeShare


Question: Boto3 delete objects with BucketObjectsCollection

"I am using the boto3 S3 resource to access objects in S3 and later delete all the files.However, the deletion operation could end up leaving one file un-deleted from time to time.Here is what my code looks like:s3 = boto3.resource("s3")# Iterate through a list of prefixesfor some_prefix in list_of_prefixes:    bucket = s3.Bucket("some_bucket_name")    # For each prefix, get all the objects with the prefix, do some processing, and finally delete all the processed objects    bucket_objects = bucket.objects.filter(some_prefix)    ... <do something with the objects>    bucket_objects.delete()To troubleshoot, I added logging in the <do something with the objects> section and made sure that the un-deleted file was part of the bucket_objects BucketObjectsCollection.Here are more details:There are about ~200 files under each prefixThere were no exceptions or errors raised by the S3 resourceThank you so much for your help!FollowComment"


Question: How to change maximum cost per hour in my spot instance

I need to change maximum cost in my spot instance.how to change it? is it possible?FollowComment

Answer: 0Please refer to the below link to understand how to monitor Spot Instance limits and usage and request a Spot Instance limit increasehttps://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-limits.htmlCommentShareAWS-User-4371564answered a month ago


Question: Redshift pausing process stuck

"Hi there,I’m having a problem with pausing my Redshift cluster.  Amazon Redshift > Clusters > Snapshots indicates that creating the (automated) snapshot has stuck: it reads “219 MB backed up. Currently backing up at 57.2399 MB/second. 0 seconds until backup completes, *** seconds elapsed” where the current number of seconds *** is displayed correctly (now it exceeds 4000).Can’t see how to abort snapshot creating or reboot the cluster while the creation process is running. I would appreciate it a lot if anyone could advise me on this issue.Thank you,A.M.FollowComment"

Answer: "0Hi,Sorry to hear that happened.The best is to open a support ticket.CommentShareacmanjonanswered a month ago"


Question: Amazon QuickSight - Filters for visualizations while selecting table records

"Hello, I can only select one record in the table and that selection be the filter for other visualizations on the page. How to add filters for other visualizations when I select more than one record in the table?Is it possible?Thanks.FollowComment"

Answer: 1Try doing this here on your analysisCommentSharejthornes-awsanswered 10 months ago


Question: What new "Cache and origin request policy" matches our legacy cache settings?

"We have been creating CloudFront distributions with the following "legacy cache settings": Open ScreenshotHeaders: AllQuery strings: AllCookies: AllWe use these distributions so we can host SSL certificates from our many customers, and redirect their traffic to our web app.What is the equivalent of these settings with the new "Cache policy and origin request policy (recommended)"?We were unfortunately not able to figure that out yet. Thank you for your help.FollowComment"

Answer: "1Accepted AnswerThe legacy cache settings that you've provided effectively disable caching, and forward everything to the Origin. You can replicate this using cache policies and origin request policies, by using managed policies as follows:Cache policy: CachingDisabledOrigin request policy: AllViewerCaching in CloudFront is highly configurable, and it's very unusual that someone would need to disable caching in all cases - you will certainly need to do this for dynamic content, but you should try to maximize it for any static content that you have. I recommend that you read the documentation on optimizing caching and availability to try to maximize your use of the cache: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ConfiguringCaching.htmlAllowing your content to be served from the cache will reduce the number of requests being made to your origin, and reduce response times and latency for your viewers.CommentSharePaul_Lanswered a year ago"


Question: How to publish live stream to MediaPackage using FFmpeg?

"I want to push the stream to MediaPackage with FFmpeg instead of pushing to MediaLIve. I try the commandffmpeg -re -stream_loop -1 -i input.mp4 -c copy -f hls 'https://username:password@mypackage_domain.com/xxx/channel/main.m3u8'It seems that I push the stream succeed, but I can't play the stream with the endpoint, and there are no metrics about ingress items.FollowComment"

Answer: "0When pushing inputs to MediaPackage they need to be in HLS protocol, it does not accept MP4s directly. MediaLive has the ability to take MP4 files as a source input and convert to HLS that can be pushed to MediaPackage. This would be considered a 'live streaming' scenario.With the VOD scenario, MediaPackage can 'ingest' MP4 assets from S3 storage, yet requires a .smil file to detail all the renditions. There will need to be an MP4 file for each rendition and the .smil. MediaPackage does not have the ability to create multiple renditions from a single MP4. Keep in mind that MediaPackage is a Just-in-Time packaging/origin server. Once MP is made aware of a VOD asset (ingest process) it will pull the associated MP4 renditions only when requested from a CDN.CommentShareMike-MEanswered a year agorePost-User-8021535 a year agoThanks for your reply. The FFmpeg command will remux mp4 to mpegts segment, but it doesn't work well.Share"


Question: CloudWatch Unified Agent custom namespace not showing up on CloudWatch Metrics

"We have previously succeeded in running CWUA on our Auto Scaling Group Ubuntu EC2 servers with custom configuration from SSM Parameter store. The parameter value looks like{"agent": {"metrics_collection_interval": 60,"run_as_user": "root"},"metrics": {"namespace": "cdk-sample-asg-ASG1-asg-exp","append_dimensions": {"AutoScalingGroupName": "${aws:AutoScalingGroupName}","InstanceId": "${aws:InstanceId}","InstanceType": "${aws:InstanceType}"},"aggregation_dimensions": [ [ "AutoScalingGroupName" ] ],"metrics_collected": {"mem": {"measurement": [{"name": "mem_used_percent","unit": "Percent"}],"metrics_collection_interval": 60}}}}and the user data script in the launch template to pick it up looks likewget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/arm64/latest/amazon-cloudwatch-agent.deb -O /tmp/amazon-cloudwatch-agent.debdpkg -i /tmp/amazon-cloudwatch-agent.deb/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:AmazonCloudWatch-cdk-sample-asg-ASG1-asg-expAll these configuration and deployment were achieved with CDK. Now we're generalising into more reusable constructs with the same setup, but the test deployment for some reason does not show the custom ASG namespace in CloudWatch Metrics.Initially I thought it might be because Detailed monitoring wasn't enabled but changing that did not improve the situation. What other obstacles can prevent the custom namespace from showing up?The system log shows CWUA successfully installed and configured.[   54.811101] cloud-init[1274]: 2022-08-12 09:59:17 (6.15 MB/s) - â/tmp/amazon-cloudwatch-agent.debâ saved [57412840/57412840][   54.836742] cloud-init[1274]: Selecting previously unselected package amazon-cloudwatch-agent.[   54.864478] cloud-init[1274]: (Reading database ... 65339 files and directories currently installed.)[   54.867059] cloud-init[1274]: Preparing to unpack .../amazon-cloudwatch-agent.deb ...[   54.896405] cloud-init[1274]: create group cwagent, result: 0[   54.922558] cloud-init[1274]: create user cwagent, result: 0[   54.940742] cloud-init[1274]: create group aoc, result: 0[   54.960169] cloud-init[1274]: create user aoc, result: 0[   54.961249] cloud-init[1274]: Unpacking amazon-cloudwatch-agent (1.247354.0b251981-1) ...[   56.464911] cloud-init[1274]: Setting up amazon-cloudwatch-agent (1.247354.0b251981-1) ...[   56.534388] cloud-init[1274]: ****** processing amazon-cloudwatch-agent ******[   56.534654] cloud-init[1274]: /opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source ssm:AmazonCloudWatch-cdk-sample-asg-ASG1-asg-exp --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default[   56.540078] cloud-init[1274]: I! Trying to detect region from ec2[   56.540893] cloud-init[1274]: D! [EC2] Found active network interface[   56.544437] cloud-init[1274]: Region: ap-southeast-1[   56.544580] cloud-init[1274]: credsConfig: map[][   56.605107] cloud-init[1274]: Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-cdk-sample-asg-ASG1-asg-exp.tmp[   56.608867] cloud-init[1274]: Start configuration validation...[   56.609027] cloud-init[1274]: /opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default[   56.618549] cloud-init[1274]: 2022/08/12 09:59:19 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-cdk-sample-asg-ASG1-asg-exp.tmp ...[   56.621567] cloud-init[1274]: 2022/08/12 09:59:19 I! Valid Json input schema.[   56.622296] cloud-init[1274]: I! Detecting run_as_user...[   56.622471] cloud-init[1274]: I! Trying to detect region from ec2[   56.622597] cloud-init[1274]: D! [EC2] Found active network interface[   56.626058] cloud-init[1274]: No csm configuration found.[   56.626169] cloud-init[1274]: No log configuration found.[   56.626318] cloud-init[1274]: Configuration validation first phase succeeded[   56.628994] cloud-init[1274]: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml[   56.681402] cloud-init[1274]: Configuration validation second phase succeeded[   56.681576] cloud-init[1274]: Configuration validation succeeded[   56.695308] cloud-init[1274]: amazon-cloudwatch-agent has already been stopped[   56.950624] cloud-init[1274]: Created symlink /etc/systemd/system/multi-user.target.wants/amazon-cloudwatch-agent.service â /etc/systemd/system/amazon-cloudwatch-agent.service.[  OK  ] Started Amazon CloudWatch Agent.UPDATEComparing to the instances that have worked, I notice some extra actions missing in the older version (circa April 2022).[   30.714094] cloud-init[851]: 2022-04-27 17:02:49 (4.98 MB/s) - â/tmp/amazon-cloudwatch-agent.debâ saved [54613346/54613346][   32.203984] cloud-init[851]: (Reading database ... 110768 files and directories currently installed.)[   32.208150] cloud-init[851]: Preparing to unpack .../amazon-cloudwatch-agent.deb ...[   32.256865] cloud-init[851]: ****** processing cwagent-otel-collector ******[   32.261143] cloud-init[851]: cwagent-otel-collector has already been stopped[   32.600172] cloud-init[851]: ****** processing amazon-cloudwatch-agent ******         Stopping Amazon CloudWatch Agent...[  OK  ] Stopped Amazon CloudWatch Agent.[   33.518263] cloud-init[851]: Unpacking amazon-cloudwatch-agent (1.247350.0b251780-1) over (1.247350.0b251780-1) ...[   35.821820] cloud-init[851]: Setting up amazon-cloudwatch-agent (1.247350.0b251780-1) ...[   35.892578] cloud-init[851]: ****** processing amazon-cloudwatch-agent ******[   35.893649] cloud-init[851]: /opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source ssm:AmazonCloudWatch-Original-app-asg-dev --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default[   35.902414] cloud-init[851]: Region: ap-southeast-1[   35.903189] cloud-init[851]: credsConfig: map[][   36.010690] cloud-init[851]: Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-Original-app-asg-dev.tmp[   36.015002] cloud-init[851]: Start configuration validation...[   36.016156] cloud-init[851]: /opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default[   36.022624] cloud-init[851]: 2022/04/27 17:02:54 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-Original-app-asg-dev.tmp ...[   36.025894] cloud-init[851]: Valid Json input schema.[   36.027002] cloud-init[851]: I! Detecting run_as_user...[   36.030131] cloud-init[851]: No csm configuration found.[   36.031144] cloud-init[851]: No log configuration found.[   36.032190] cloud-init[851]: Configuration validation first phase succeeded[   36.033524] cloud-init[851]: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml[   36.065638] cloud-init[851]: Configuration validation second phase succeeded[   36.066681] cloud-init[851]: Configuration validation succeeded[   36.076898] cloud-init[851]: amazon-cloudwatch-agent has already been stopped[  OK  ] Started Amazon CloudWatch Agent.There are no additional users and groups created, and no final symlink created between the service files.However I am not experienced enough with Linux to properly grasp the significance of these differences and how they might cause report failure.FollowComment"

Answer: 0Accepted AnswerThe CWUA version difference is a red herring.Thankfully it boiled down to the simple case of forgetting to add the managed policy CloudWatchAgentServerPolicy to the EC2 instance role.CommentShareicelavaanswered 10 months ago


Question: S3 lifecycle rule

How to create a lifecycle rule that delete all files in a bucket?FollowCommentRei Cheng 7 months agocreate for testingShare

Answer: "0Accepted AnswerPlease resolve this.CommentShareRei Chenganswered 7 months ago0You need to read up on expiration rules for lifecycle rules.  You can read about them here.  There are a number of other articles you may be interested in in this area, too.  https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-expire-general-considerations.htmlCommentSharejschwar313answered 7 months ago"


Question: s3 jmespath not matching as expected

"Edit (2022-07-18)To debug I have shared more of my code and setup a public s3 bucket.And in the process it seems to be working as expected after a few learnings and tweaks. Current state is pasted below for context.Code:import jsonimport boto3s3 = boto3.client("s3")s3_paginator = s3.get_paginator("list_objects_v2")s3_iterator = s3_paginator.paginate(Bucket="jmespath-not-matching-as-expected")# [Output 1]: All keys (with content, skips empty files and folders)for key_data in s3_iterator:    print("[Output 1]")    print(        json.dumps(            {                k: [{"Key": i["Key"]} for i in v if i["Size"] != 0]                for k, v in key_data.items()                if k == "Contents"            }        )    )# Query which only returns a subset of interestquery_starts_with = f" && starts_with(Key,`sub_folder`)"# Note: Previously successfully processed folders have had an empty file 'processed' added to paths# Trying to move/archive is too slow, easier to have a query skip over what has already been processedprocessed = [    i.split("/processed")[0]    for i in s3_iterator.search("Contents[?contains(Key,'processed')].Key")]query_unprocessed = (    f" && ({' && '.join([f'!contains(Key, `{p}`)' for p in processed])})"    if processed    else "")# [Output 2]: Subset of keys (only those with content and not contained in a subfolder already processed)for key_data in s3_iterator.search(    f"Contents[?Size!=`0`{query_starts_with}{query_unprocessed}].Key"):    print("[Query]")    print(f"Contents[?Size!=`0`{query_starts_with}{query_unprocessed}].Key")    print("[Output 2]")    print(key_data)Query:Contents[?Size!=`0` && starts_with(Key,`sub_folder`) && (!contains(Key, `sub_folder/should_not_display/38286a88-eef4-4f04-827c-f7809bc04f31`) && !contains(Key, `sub_folder/should_not_display/c86fae3b-9987-42a7-b876-d995ce66a9f9`))].KeyOutputs:[Output 1]{  "Contents": [    {      "Key": "sub_folder/should_display/random"    },    {      "Key": "sub_folder/should_not_display/38286a88-eef4-4f04-827c-f7809bc04f31/random"    },    {      "Key": "sub_folder/should_not_display/c86fae3b-9987-42a7-b876-d995ce66a9f9/random"    },    {      "Key": "this/should_not_display/random"    }  ]}[Output 2]sub_folder/should_display/randomOriginal Question:Using boto3 (s3 client) I have tried the following:query = "Contents[?starts_with(Key,`sub_folder`) && (!contains(Key, `sub_folder/should_not_display/38286a88-eef4-4f04-827c-f7809bc04f31`) && !contains(Key, `sub_folder/should_not_display/c86fae3b-9987-42a7-b876-d995ce66a9f9`))].Key"s3_paginator = s3.get_paginator("list_objects_v2")s3_iterator = s3_paginator.paginate(Bucket=S3_BUCKET)for key_data in s3_iterator.search(query):    print(key_data)    # Should only print s3 objects from subfolder that do not contain:    #  - sub_folder/should_not_display/38286a88-eef4-4f04-827c-f7809bc04f31 or     #  - sub_folder/should_not_display/c86fae3b-9987-42a7-b876-d995ce66a9f9 in thier pathIllustration of s3 folder structure:{  "Contents": [    {"Key": "sub_folder/should_not_display/38286a88-eef4-4f04-827c-f7809bc04f31"},    {"Key": "sub_folder/should_not_display/c86fae3b-9987-42a7-b876-d995ce66a9f9"},    {"Key": "this/should_not_display"},    {"Key": "sub_folder/should_display"}  ]}When I paste the above query and json illustration into https://jmespath.org/ it works as intended. And yet, with the s3 boto3 client library I am not getting any matches where I shouldFollowCommentshimo 10 months agoYour query syntax works fine for me. Does s3_iterator have expected contests?Sharednk8n 10 months agoDespite things working as expected in this toy example, it is still not working in my real world scenario. Can't locate what the problem is. For now I am just going to reprocess everything available weekly but expire s3 contents after 6 days.Share"


Question: Pearson VUE revoked my SAA-C03

"Hi AWS Team,I scheduled my exam for AWS SAA-C03 on 08-10-2021 and the Pearson VUE proctor was revoked. That was not my mistake, the proctor was trying to connect through onVUE it was not connecting, then he pinged in the chat as, I am calling your mobile and just pick up the call then I picked up.Raised support with Pearson VUE on the same day and they said after 14day you can reschedule the exam, when I check with them after 14 days now they are saying to check with the AWS teamPearson VUE case ID: XXXXXXXXAWS Candidate ID: AWS03045655I have already raised the AWS support case using https://support.aws.amazon.com/#/contacts/aws-training.I haven't heard back yet. I need to complete the certification ASAP as per my company guidelines. Kindly do the needful to reschedule the exam.Thanks.Edit: Removed case ID — Aimee K.FollowComment"

Answer: "0Hello,I'm so sorry to hear about this, it's not the experience we strive for. I've passed these concerns along internally for further review & visibility into your case.For your privacy, we're unable to discuss account or case details via this platform. Rest assured, our team in the Support Center has full visibility and will follow up with you in your open support case as soon as there are next steps provided.Please check out this article to see how you can request a call through our Support Center as well, should you have further questions or concerns about this process:https://aws.amazon.com/premiumsupport/knowledge-center/aws-phone-support/.We thank you for your patience as we work to resolve this.Best regards,— Aimee K.CommentShareEXPERTAWS Support - Aimeeanswered 7 months ago"


Question: WorkSpaces - Domains and IP Addresses to Add to Your Allow List

Customer is configuring Amazon WorkSpaces. They are reviewing the Domains and IP Addresses to Add to Your Allow List page (https://docs.aws.amazon.com/workspaces/latest/adminguide/workspaces-port-requirements.html) which asks them to add a list of our service endpoints in regions where they do not operate. Their operation is within us-east-1.Do they need to add all the service endpoints listed or just the endpoints for us-east-1?FollowComment

Answer: "0Accepted AnswerThey will need to include the service endpoints for each section, but not for each region in each section. For example, your customer would not need to include the endpoints for us-west-2 where there is also an endpoint in us-east-1. There are a number of "global" endpoints that will still need to be included.CommentShareEXPERTMatt_Janswered 2 years ago"


Question: Additional information required to generate a public certificate for a project using a fhir client

I am trying to create a public certificate for a fhir client but was told I needed additional verification required. How would I go about doing this?  I cannot put in a support ticket because I do not have an upgraded account. how can i get someone to authenticate the public certificate?FollowComment

Answer: "0Hi, thank you for reaching out to us with your question. Please reference the doc link attached [1] to this message for more information regarding this issue. ACM requires additional information to process certificate requests from you as this is a fraud-protection measure if your domain ranks within Alexa's top 1000 websites list. We also might need to whitelist your domain if it indeed falls within Alexa's top 1000 websites list, for which you would need to reach out via a support case.To do this, you would need to subscribe to a proper support plan for us to troubleshoot this issue much further to dive deeper into the root cause of it. The link provided at [2] has the steps for how to subscribe to a Support Plan. Please note at the bottom of that doc link that if the Support plan you choose has a monthly cost associated with it, then your payment method is automatically charged the prorated monthly cost according to the days remaining in the current month. When this charge is processed, your AWS Support subscription is activated within a few hours.Also an important reminder to not share any confidential information(such as domain name) over re:Post as this is a public platform.[1] - https://docs.aws.amazon.com/acm/latest/userguide/troubleshooting-failed.html#failed-additional-verification-required[2] - https://aws.amazon.com/premiumsupport/knowledge-center/sign-up-support/CommentShareNelson_Panswered 10 months agoSUPPORT ENGINEERSumukhi_Previewed 10 months ago"


Question: Can't log into business account

I can't log into my business account. The account # is XXXXXXXXXXXX. The person who used to manage our account has been terminated.Edit: Removed account #. - Kita B.FollowComment

Answer: 0Accepted AnswerIs this account your root account or what?Only the terminated person used to access the account?CommentSharehimanshustallongmailcomanswered 13 days ago0Yes. He was the only one that accessed the account. I believe it was the root user. I'm new to this.CommentSharerePost-User-0728522answered 13 days agohimanshustallongmailcom 13 days agois forgot password option not working or of no use in this case?Share


Question: Errno -2: Name or service not known (AWS IoT Sitewise)

"I followed step by step the tutotrial https://docs.aws.amazon.com/iot-sitewise/latest/userguide/ingest-data-from-iot-things.html but when I tried to do the step 8 of section "Running the device client scrip" I received the error "Errno -2: Name or service not known". Apparently the problem is in mqttClient.connect(), but I haven't been able to fix it in a long time. Any help is welcome! Thanks!FollowComment"

Answer: "1Hi,it looks like the AWS IoT endpoint which you pass with -e command line parameter is not resolvable. The iot endpoint looks similar to abcdefgh012345-ats.iot.AWS_REGION.amazonaws.com.You can retrieve your endpoint for example with the AWS CLI:aws iot describe-endpoint --endpoint-type iot:Data-ATS --query 'endpointAddress' --output textYou can also validate if you can resolve your endpoint in DNS.Cheers,PhilippCommentSharePhilipp Sanswered 6 months agorePost-User-4649751 6 months agoIt worked! Cheers mate!Share"


Question: Missing Public Ip after reboot

"Hi dear Community,After an recent reboot my t2.small instance (Win-serv 2008r2) by some reason was not assigned with a Public IP.I was successfully using it for ages and all worked fine (I did reboots hundred of times without any issues). It means that the reason is not in wrong configuration on my side. VPC, routing table, networks - nothing was change and these used to work till that last reboot.I've googled the problem and reviewed a lot of AWS documents - I do not see any related answer on my problem. It looks like the most closest case is as following:https://stackoverflow.com/questions/26706683/ec2-t2-micro-instance-has-no-public-dns(it looks like I faced with exectly the following case: https://stackoverflow.com/a/60671532)I've tried all advices from the link (except recreating the instance from a image) - everything  looks configured OK but I still can't get a public IP and so CAN'T connect to my device.Does anybody know what might be a root of cause and a fix for this issue (obtain Public IP back)? Or this is really "Yes, it is possible that AWS screws up" case?Thanks in advance for any assistance!DmitryFollowComment"

Answer: "0Given that nothing has changed on your end I'd encourage you to create a support case. The support team can look at your environment and determine what is failing. As a forum participant, we can't do that.CommentShareEXPERTBrettski-AWSanswered a year agoAWS-User-2423939 a year agoThank you for your answer!Playing around with this problem I was able to find only that this is directly related to VPC, because even recreating the instance from an image but with the same existing VPC leads to the same issue. I guess that something was broken in AWS internal records that are related to particular VPC or it's part.As soon as it is impossible to reassign VPC (or ist's meaningful parts) to an existing instance  - the only way is to recreate an instance with the related AMI-image and specifying a NEW VPC!At this moment my subscription does not include support service so I can't create the ticket. As there is a workaround  - no reason to extend my subscription to create a ticket... AWS will continue to dwell in a cloud of ignorance, unfortunately =)Share"


Question: RDS AUTOMATIC BACKUP

How to disable RDS autoatic Baxkup?FollowComment

Answer: "1As documented, to disable automated backupsSign in to the AWS Management Console and open the Amazon RDS console at https://console.aws.amazon.com/rds/In the navigation pane, choose Databases, and then choose the DB instance or Multi-AZ DB cluster that you want to modify.Choose Modify.For Backup retention period, choose 0 days.Choose Continue.Choose Apply immediately.Choose Modify DB instance or Modify cluster to save your changes and disable automated backups.CommentShareadrian-AWSanswered a month ago"


Question: Failed to send one byte message to AWS IOT Wireless device through queue downlink message

When I try to send a one byte message to my LoRa device that is connected to AWS IOT Wireless server I get a warning when I enter "1" in the Payload field and try to Submit that says:Payload must follow pattern: ^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$This is under:AWS IoT > Wireless connectivity > Wireless devices > [Device ID] > Queue downlink messageFollowComment

Answer: 1Accepted AnswerHi. The payload must be entered in Base64 format. So a 1 byte value of 0x01 is AQ==.https://base64.guru/converter/encode/hexCommentShareEXPERTGreg_Banswered a year ago


Question: Don't see pillarid Sustainability

"I have created Amazon quicksight for aws well architect, I have choosed pillarid but i have seen 5 pillar , dont have pillar Sustainability in quicksight.Please guide meFollowComment"

Answer: "0Please, can you provide more information about how to you set up? Did you connected with WA Tool? You are assessing some WAFRs made before 2022?CommentShareIbrahim Cesaranswered 2 months ago"


Question: Newline encoding in url to SNS Service

"Hi,I am triggering the SNS Service through the API Gateway.My RequestBody looks like this:{"sender_id": "Alias","phone_number": "+4912345678","message_text": "Break \n here"}The Url Query Parameter is mapped like this:Messagemethod.request.body.message_textAction Publishetc.This alone should work fine, but it isn't, as it's passed to the SNS service via URL and the \n gets decoded to %5Cn."Action=Publish&Message=Break_%5Cn_here"This should be passed as %0A. Even writing "Break %0A here" in the message is not working as the % is then urlencoded as %25, which results in %250A in the URL and a '%0A' in the received SMS.I tried overwriting the Message through a Mapping Template and using $util.javaScriptEscape and so on.. Nothing works, it's always too heavy encoded. Is there a way to trick AWS into not doing this? Or a way to pass the Message other than through the URL Query?I hope this request is detailed enough, thanks in advance.FollowComment"

Answer: "0Thanks to the AWS Support the problem has been solved. It was wrong to use URL Query Parameters. I removed all of them.I needed one HTTP Header: Content-Type: 'application/x-www-form-urlencoded'Then I used a Messaging Template like this:#set($message = $input.path('$.message_text'))#set($phoneNumber = $input.path('$.phone_number'))Action=Publish&PhoneNumber=$util.urlEncode($phoneNumber)&Message=$util.urlEncode($message)&MessageAttributes.entry.1.Name=AWS.SNS.SMS.SenderID&MessageAttributes.entry.1.Value.DataType=String&MessageAttributes.entry.1.Value.StringValue=AliasHaving my JSON Object in the Request like this:{"phone_number": "+4912345678","message_text": "Break\nHere","sender_id":"Alias"}Works perfectly fine with a line breakCommentShareAnanasPizzaanswered 3 years ago"


Question: calling IAM auth function urls from Lambda

"I have a 2 lambda functionsCaller  - calling the service mentioned #2Service - with function url with IAM permission setup.From service lambda #1 I want to  invoke service #2 . I searched online, all mentioned calling the function url using IAM credentials, but we don’t use IAM credentials inside calling Lambda right ?Do we need to use the lambda role while configuring the lambda resource based policy ?Are there any CDK examples for Lambda with functions urls IAM permissions set ?Can some one share on this ?FollowComment"

Answer: "1Accepted AnswerFor invoking a Lambda function using its function URL, which is protected with IAM, you need to sign the request using SigV4. To create the signature you need to have AWS credentials which are allowed to invoke the function. If you invoke function #2 from a Lambda function #1, you need to attache an execution role to function #1 that has the right permissions.You will need to sign the request yourself. The role is not enough. Depending on your language, there are different SDKs that may sign it for you.CommentShareEXPERTUrianswered a year agoNavin GV a year agoI am going to use java/kotlin in lambda #1 to invoke lambda #2.  I referred the below link which requires the secret key.  Does this secret key represents the IAM user's secret key ? If yes, how does this IAM user and lambda role linked ?https://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-javaWhere do we store the secrets in Lambda ? What is the best practice ? We also want to rotate the key periodically right.ShareUri EXPERTa year agoThe function has an execution role. You should retrieve the secret key from the role in the Lambda function. The runtime does it for you and you can find the information in environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN.Share"


Question: EC2 to VPC migration -> can't tell what service we need to migrate

"Hello,regarding the dismissal of non-VPC instances, we keep getting the notice about resources running on EC2 and don't get what service is the 'culprit'.We have:1 EC2 instance (the name being vpc-XXX so I assume is ok)1 RDS instance (the name being vpc-XXX so I assume is ok)In addition to that we also have:1 S3 instance3 Elastic ipSecurity Groupssome volumessubnet maskskey pairsIs it possible that one of said services triggers the notice?Is there a way to 'filter' them and tell for sure what resource we need to migrate?Any help would be greatly apreciated.Thank youFabioFollowComment"

Answer: "1Use the EC2 Classic Resource Finder script to find all of the EC2-Classic resources in your account. You can run this directly in a single AWS account, or you can use the included multi-account-wrapper to run it against each account of an AWS Organization. The Resource Finder visits each AWS Region, looks for specific resources, and generates a set of CSV files.https://github.com/aws-samples/ec2-classic-resource-finderCommentShareEXPERTMatt-Banswered 8 months agoEXPERTToni_Sreviewed 8 months ago"


Question: Why do I need to setup LightSail DNS records when I have Route53 DNS records?

I don't understand I why I need to re-create all my DNS records in LightSail under new name server records when I already have this domain setup on Route 53? I have many other services running under sub-domains on Route53.FollowComment

Answer: "1Accepted AnswerI found that I am able to create an A record on my DNS records on Route53 to the static IP of LightSail. Should perhaps be something more clear on the LightSail side.CommentSharerePost-User-3355781answered 6 months ago0You don't need to setup Lightsail DNS records. As long as the Name servers for your domain name are set to the ones for your Route 53, you can use route 53 for DNS.CommentSharekdambiecanswered 6 months agon19htz 3 months agoHello, can you explain how to do that with example?Share"


Question: API Gateway HTTP + Lambda integration not enabling CORS

"I have an API Gateway with an HTTP API + route that utilizes a Lambda function integration.From the AWS documentation, I seeIf you configure CORS for an API, API Gateway automatically sends a response to preflight OPTIONS requests, even if there isn't an OPTIONS route configured for your API. For a CORS request, API Gateway adds the configured CORS headers to the response from an integration.If you configure CORS for an API, API Gateway ignores CORS headers returned from your backend integration.However, I still get the errors:Access to fetch at 'https://domain.execute-api.aws-region.amazonaws.com/dev/upload' from  origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight  request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is  present on the requested resource. If an opaque response serves your needs, set the  request's mode to 'no-cors' to fetch the resource with CORS disabled.From Cloudwatch, I can see the requests being sent to the API, but for some reason the header isn't properly configured. I do not understand this because I thought API Gateway should handle OPTIONS pre-flight requests for HTTP APIs. I cannot find out what I'm doing wrong. I'm not sure if it's because I'm not properly calling my API (link is https://domain.execute-api.aws-region.amazonaws.com/dev, stage is dev, route is /upload) or something else.FollowComment"

Answer: "0Hello!  In the example given, your URL would be https://domain.execute-api.aws-region.amazonaws.com/dev/upload.   This article from AWS Support contains some very detailed debugging steps to try: https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-cors-errors/ .  Hope it helps!CommentShareClay_Banswered a year ago"


Question: Routing/forwarding to onprem site

"Hi,Here my setup:one on prem site (range 10.1.0.0/16)one AWS Org with two account A & B, with a VPC peering between VPC-A (10.50.0.0/16) and VPC-B (10.20.0.0/16)Wireguard VPN setup between vm1 (on prem) and vmA-1 (VPC-A)VPC-A route:- 10.1.0.0/16 -> vmA-1- 10.20.0.0/16 -> peering- 10.50.0.0/16 -> local- 0.0.0.0/0 -> igwVPC-B route:- 10.0.0.0/8 -> peering- 10.20.0.0/16 -> local- 0.0.0.0/0 -> igwIP src/dest check is disable on vmA-1From an ec2 on any subnet in VPC-A, I can reach vms on prem.From an vm on prem, I can reach ec2s in VPC-A and VPC-B.From ec2 on any subnet in VPC-B, I can reach vmA-1.From ec2 on any subnet in VPC-B, I cannot reach vms on prem.Did I miss something (obviously yes) ? Is there any src/dest check that need to be disable as well at the peering, route table level ?Thanks for sharing your ideas !FollowComment"

Answer: "1This setup will not work, you can only target the VPCA CIDR over the VPC peering connection. You would need to NAT the traffic on vmA-1 to make this work the way you have it setup now. Using a TGW in place of the VPC peer will allow you to route the traffic the way you are trying to over the VPC peer as TGWs provide transitive routing and the TGW attachment subnets in the VPCA can redirect the 10.1.0.0/16 traffic to vmA-1. You could also setup the VPN to the Transit Gateway directly or from the on-prem vm1 to VGWs on both VPCs or to EC2 instances in both VPCs.CommentShareEXPERTThaddeus_Wanswered 3 months agoed 3 months agoThanks a lot for confirming that something was wrong in my setup !I'll give a try to the TGW and review my peering setup.Share"


Question: AWS Glue Dynamid Dataframe relationalize

"I load json data and use relationalize method on dynamic dataframe to flatten the otherwise nested json object and saving it into parquet format. The problem is that once saved into parquet format for faster Athena queries, the column names contain dots, which is against the Athena sql query syntax and thus I am unable to make column specific queries.In order to tackle this problem I also rename the column names in the Glue job to exclude the dots and put underscores instead. My question is which approach of the two would be better and why? (Efficiency- memory? execution speed on nodes? etc.).Also given the horrible aws glue documentation I could not come up with dynamic frame only solution. I have problems getting the column names in dynamic fashion, thus I am utilizing toDF().First approach is around getting the column names from df extracted from dynamic dfrelationalize1 = Relationalize.apply(frame=datasource0, transformation_ctx="relationalize1").select("roottable")    df_relationalize1 = relationalize1.toDF()    for field in df_relationalize1.schema.fields:        relationalize1 = RenameField.apply(frame = relationalize1, old_name = "`"+field.name+"`", new_name = field.name.replace(".","_"), transformation_ctx = "renamefield_" + field.name)Second approach would be to extract the df from dynamic df and perform the rename field on the pyspark df (instead of dynamic df), to then convert back to dynamic df and save it in parquet format.Is there a better approach? Can a crawler rename columns? How fast is .fromDF() method? Is there a better documentation on functions and methods than the pdf developer guide?FollowComment"

Answer: "0Accepted AnswerAs far as I tried, I was able to run column specific queries against parquet table which contains dot in column name.Table: parquet_tableroot|-- name: string|-- url: string|-- sample.key: stringQuery:SELECT "sample.key" FROM "parquet_table" limit 10;SELECT * FROM "parquet_table" WHERE "sample.key" LIKE 'sample%' limit 10;Can you explain bit more details? What schema does your table have? What query did you see errors in?CommentShareAWS-User-6283168answered 3 years ago0It appears that when querying with Athena I did not enclose column names with dots into double quotes, thus the error.I am still interested in removing the dots from column names and thus would like to know what would be good approach of renaming multiple columns in AWS Glue. I changed my approach to first converting the DynamicDataframe to PySpark dataframe and then using piece that I found on stackoverflow.new_column_name_list= list(map(lambda x: x.replace(".", "_"), df_relationalized.columns))    df_renamed = df_relationalized.toDF(*new_column_name_list)CommentSharecellanswered 3 years ago0Do you have any STRUCT datatype in your columns, does this solution also change the fields inside the struct ?CommentSharemahe13mkanswered 3 years ago"


Question: "Unable to enter other created instances with putty, except for first instance"

"I've created an instance, created key-pair able to connect to it with putty.  All my instances in the past worked fine, no key pair / putty client issue, my security groups are set with default open ssh accounts (open only to my ip which I've checked and matches my current ip)2nd ec2 instance was created, created a new key pair, created all settings from scratch, added my_key.ppk to putty>Auth and now I'm not getting access, putty is throwing:PuTTY Fatal ErrorNo supported authentication methods available (server sent: publickey)Now i've already been through this => https://aws.amazon.com/premiumsupport/knowledge-center/linux-credentials-error/It's basically saying problem stems from either:You're not connecting with the appropriate user name for your AMI when you negotiate an SSH session with an EC2 instance.OR 2. You're using the wrong private key when you negotiate an SSH session with an EC2 instance.I'm neither, (1) doesn't apply since I don't use an AMI.I went through resolution and nothing was relevant or went through all of those and I'm on Windows 10 so I can't use the EC2 Serial Console.I have opened several instances by now, created instances with new keys, then tried with an already existing key pairs. nothing.Any help at this point would be appreciated 🙏FollowCommentBrettski-AWS EXPERTa year agoCould you explain a little more about "I don't use an AMI"?Shareerezt a year agoI connect to my aws acct with 2fa, I connect to my instances with putty authenticating with a key pair. I have never made an AMI account.ShareBrettski-AWS EXPERTa year agoAn AMI is the image that the instance is launched from; so I think we're talking about different things here.Share"

Answer: 0I've seen issues like this when the key has been copy/pasted and an editor has added line-breaks. Could this be the issue?Also confused about AMIs (Amazon Machine Images) in this context but it could be you're using an image that needs a specific user name.HTHCommentShareAlex_Kanswered a year ago


Question: ECS auto scale-in doesn't work event though I got CloudWatch scale-in Alarm

"EnvironmentECS & FargateScaling policyAnd I got scale-in alarm like:But, ECS doesn't scale-in since 18:43I don't use scale-in protection option.Please let me find the reason if you experienced similar problems.FollowComment"

Answer: "1Accepted AnswerHello,In general, AutoScaling might choose not to scale-in when an alarm goes off for quite a few reasons (already at the min, suspended processes, in progress activities, etc).  And Target Tracking specifically has multiple additional layers of checks built in (ex, if there's multiple target tracking policies, they all need to want to scale-in at the same time).  This is all done to prevent excessive oscillation and maintain high availability in the environment.There was a recent Application AutoScaling feature launched to help add additional transparency to these not-scaling decisions.  From the AWS CLI, can you run the below command (filling in your specific info where needed) and let me know if it has information to shed light on the delayed scaling?aws application-autoscaling describe-scaling-activities \    --include-not-scaled-activities \    --service-namespace ecs \    --scalable-dimension ecs:service:DesiredCount \    --resource-id service/default/web-appIf you see an error in the CLI stating that --include-not-scaled-activities isn't a valid flag, you'll need to update the CLI (since this is a relatively recent feature).  Alternatively, you could use CloudShell in the AWS Console which should have the latest CLI buildCommentShareShahad_Canswered 3 months agorePost-User-4683956 3 months agoThank you a lot, it really helped me.Share"


Question: "DNS Record Setup, PTR Record Setup, Reverse DNS, SMTP and DMARC Authentication"

"Issue with SPF, DKIM. DMARC, DNS, Reverse DNS Setup in AWSFollowCommentsdtslmn EXPERT9 days agoCan we learn the issue?Share"


Question: What's the unit for AppRunner's CPU Utilization?

"Looking at the CloudWatch metric AppRunner > Instance metrics > CPU Utilization, I see that my instance is reaching up to 1000 of an unknown unit. Well, the question here is simple - what is the unit for CPU Utilization?Here are my theories so far:The official docs page on AppRunner metrics is not very helpful; their description is "The average CPU usage during one-minute periods."The most obvious is percentage of CPU, but that should stop at 100%, so 1000% doesn't make much sense.According to some other AWS products though, 100% = 1vCPU. This SageMaker question says 400% = 4 vCPUs. So 1000 would be 10 vCPUs.But my instances are configured to use 1vCPU & 2 GB. So I'd still expect a max of 100, not 1000. Am I allowed to go above the max for brief amounts of time? Will I be charged extra if so?Another theory is that this might be something to do with less than 1vCPU - like AWS Lambda's CPU throttling which lets you use less than 1vCPU. ie, 1000 = 1vCPU.Finally, I might be doing something wrong. I'd be happy to share some details of my setup. I'm using maximum for aggregation statistic in Cloud Watch & only 1 instance is running.But to sum up - does anyone know what the unit is for AppRunner's CPU Utilization?FollowCommentmreferre EXPERTa year agoCan you post a screenshot of the metric in CW? Thanks.Sharehowellz a year agoSure, here's a screenshot of CloudWatch: https://imgur.com/a/Ozdjj7OShare"

Answer: "0Accepted AnswerHello,The unit for CPU is percentage as you correctly stated. In the attached graph that shows "Graphed Metrics" I see that two labels have been selected. Could you try selecting only the second one (CPU Utilization) and let us know what you see? When two or more metrics are selected CloudWatch shows "various metrics" as the unit.CommentShareUgochukwu_Ianswered a year agohowellz a year agoAh, thanks! Yes, I removed that & the unit become Percent! Not sure about the graph above (I've losted it now), but a newer one shows Percent & ranges from 0 - 200%. So if that SageMaker question is right, 200% would be 2vCPUs.Share"


Question: [Announcement] AWS IoT Greengrass v2 now supports the use of hardware security modules (HSM)

"We are pleased to announce that AWS IoT Greengrass v2 now supports the use of hardware security modules (HSM) through the PKCS#11 interface for secure storage and offloading of private keys. Customers can use the new PKCS#11 provider component to securely store and access private keys in HSMs, such as Trusted Platform Modules (TPM), or other cryptographic elements. AWS IoT Device Tester for AWS IoT Greengrass (IDT-GG) now also includes tests for device manufacturers to validate if their devices with Trusted Platform Modules (TPM) can run AWS IoT Greengrass and interoperate with AWS IoT services.Managing the security of device fleets is a top priority for IoT customers, specially those with devices that are installed either outside company premises or in remote, unmanned areas. With this launch, you can configure AWS IoT Greengrass Core software to use the private key and certificate stored in a hardware security module (HSM) through the PKCS#11 interface (https://en.wikipedia.org/wiki/PKCS_11). To enable the AWS IoT Greengrass Core software to use the private key and certificate stored in the HSM, you can install and configure the PKCS#11 provider component when you install the AWS IoT Greengrass Core software.AWS IoT Device Tester for AWS IoT Greengrass (IDT-GG) is a test automation tool for your Greengrass devices. IDT-GG now includes tests for device manufacturers to validate if their devices with Trusted Platform Modules (TPM) can run AWS IoT Greengrass and interoperate with AWS IoT services.To learn more, refer to AWS IoT Greengrass developer guide (https://docs.aws.amazon.com/greengrass/v2/developerguide/hardware-security.html) and AWS IoT Device Tester for AWS IoT Greengrass developer guide (https://docs.aws.amazon.com/greengrass/v2/developerguide/dev-test-versions.html).FollowCommentNguyen_M SUPPORT ENGINEERa year agoThis is an announcement migrated from AWS Forums that does not require an answerShareclogwog a year agois there any example of how to configure on using the pkcs11 private key ? We are using the ATECC608A from microchip and for purposes of retrieving the private key we specify it like:pkcs11:model=ATECC608A;manufacturer=Microchip Technology Inc;serial=1234567890334;token=00ABC;object=device;type=privateso i assume this should be configured somewhere in the /greengrass/v2/config/effectiveConfig.yaml ?Share"


Question: How can I use python libraries spacy in aws glue jupyter notebook ?

"I tried using !pip3 install spacy to download the library but the notebook was not able to find the module, next I realized you can use %additioanl_python_modules, but I am not able to figure out how to do so...so I have to zip them even if I am not using my own custom libraries.FollowComment"

Answer: "0Have you tried using S3 to upload your library and the reference it in Glue?https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#aws-glue-programming-python-libraries-zipping:~:text=.zip%20file.-,Loading,-Python%20libraries%20inCommentSharevtjeananswered 2 months ago0If you have a zip you need to use %extra_py_files and point to s3, if you use %additional_python_modules you can point to a whl on s3 or just use a pip package name list.Notice you need to do this before you run any other cells that would start the session.CommentShareGonzalo Herrerosanswered a month ago"


Question: Possible AWS Glue Connection Bug

"Hi There,I was adding a VPC Network connection to an AWS Glue job and I got this error:JobRunId:jr_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx failed to execute with exception Could not find connection for the given criteria Failed to get catalog connections given names: xxx-xxxxx-xxxx,noneService: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: xxxxxxx-xxxxx-xxxxx-xxxxxxxxx; Proxy: nullI checked my VPC connecton and it all looked fine with correct  security.Eventually I realized that I had also added the "None" Connection to the  jobEGSurely the "None" connection should be ignored or else shouldn't be selectable.ThanksJohnFollowComment"

Answer: 0Glue will try to use that None connection and if it's not correct will fail.Please clean it up.CommentShareGonzalo Herrerosanswered 2 months ago


Question: Updated Wordpress blueprints needed

"Can we get some updated Wordpress blueprints in Lightsail? Currently 5.3.2-3 is being offered which was released on 2-19-20.Bitnami released 5.4.2-1 today, 7-7-20.  Would like to take advantage of the security updates among other updates.FollowComment"

Answer: "0Hi,Thank you for using Lightsail and your precious feedback. We have noted your request and will schedule updates on designated blueprints. Thanks again.MichaelCommentShareAWS-User-3138555answered 3 years ago"


Question: Is it possible Private EC2 Instance send outbound traffic to Public ELB?

"Here is the AWS structure I created.VPC : 1Public Subnet : 2 (1 EC2 Instance in each Subnet)Public ELB : 1 (About Public Instance)Private Subnet : 1 (1 EC2 Instance)Here, I want to make a request to the Public ELB from the instance in the Private Subnet. How can I do this? Or did I get the structure wrong?FollowComment"

Answer: "1If you're accessing a Public load balancer that is in a different VPC (owned by you or by someone else) then the answer from Jhalak is correct - NAT Gateway is the way to go.But I think your question is asking how you access the load balancer in the same VPC. You can still use NAT Gateway but there is a charge associated with that. Today, there's no simple way of getting a DNS record for the private IP of the load balancer. Instead, you might create a second private load balancer.Also: Your question mentions ELB; please consider using ALB as it has many more features.CommentShareEXPERTBrettski-AWSanswered 8 months agoEXPERTToni_Sreviewed 8 months ago0you can use NAT Gateway to send traffic out of your Private EC2/subnet.CommentShareJhalak Modianswered 8 months ago"


Question: Quicksight - Using calculated field in visualizations

"Hi,I have a very simple calculated field which is a ratio of 2 summed metrics, which I want to use in several visualizations.Other than the Table and Pivot visuals, every time I try to use the calculated field as a value (say in a horizontal bar chart) with one of the dimensions, I keep getting "Table calculation attribute reference(s) are missing in field wells".  I am assuming I can't be expected to include the 2 metrics that are used in the calculated metric into the visual, correct?It is such a basic thing that I can't imagine there is no option  to utilize a calculated metric in a simple bar chart.Any idea what I could be doing wrong?FollowComment"


Question: Convert VTL object to JSON in API gateway

"I'm trying to transform the response for API gateway, the problem is that the response is relative dynamic, and I need to make a transformation on a specific attribute.Below examples of possible responses{  "results": {    "name1": [      {        "phoneNumber": "000000000",        "attr1": "foo"      },      {        "phoneNumber": "000000001",        "attr1": "foo1"      }    ]  }}{  "results": {    "name2": [      {        "phoneNumber": "\"000000000\"",        "attr4": {          "foo1": [            "abc"          ],          "foo2": true        }      },      {        "phoneNumber": "\"000000005\"",        "attr4": {          "foo1": [            "bar"          ],          "foo2": false        }      }    ]  }}The attribute that I want to change is "phoneNumber" that is the same for any response and the location is always $.results..phoneNumber, and I need to JSON parse it.I could convert the response in VTL object with $util.path('$') but then I would need to recreate the entire response, and because it's dynamic it's very difficult, so I was wondering if there is something easier and convert the VTL to JSON, or otherwise modify the object directly without convert to VTL object.FollowComment"


Question: Live migration in EC2 (vMotion in VMware)

"Is it possible to do livemigration (equivalent to vMotion in VMware) of EC2, between physical servers?I found an article that says it is possible in GCP.We are checking from two perspectives.(1) Can the EC2 administrator perform livemigration at any given time (I couldn't find anything about it in the documentation, so I assume it is not possible)(2) Does AWS person move EC2 to another physical server on its own when, for example, the physical server is about to break down?I use the term "livemigration" to mean "moving a virtual machine running in a server virtualization environment entirely to another physical server without stopping the OS or applications.FollowComment"

Answer: "1Accepted AnswerVMWare Cloud on AWS supports this, but EC2 does not. There are methods, mechanisms, and best practices for providing high availability provided in the AWS Well-Architected Reliability Pillar.CommentShareRodney Lesteranswered 7 months ago0Thank you very much.I will peruse the links to the documents you provided.CommentSharerePost-User-20221103answered 7 months ago"


Question: AWS Client VPN silent installation

"Hello, we're looking to do a central install of AWS Client VPN (3.1.0) via Intune.  If done interactively, it forces the user to accept the EULA, I was hoping we could do a totally managed installation (you know -- like real enterprise software...), especially considering our desktop users won't have local admin privileges so they couldn't possibly install it without manual intervention from our support teams.I've tried to get silent installation working locally with typical msiexec params like /i ... /qn but nothing happens.  If I try logging the install with /log <logfilename> then I end up with an empty log file.Is there a way to silently install this?Thanks!FollowComment"

Answer: "0Well, Intune magically performed the installs silently anyway even though I didn't get it working locally with msiexec.  At this point not investigating further but will leave this here in case it helps anyone else!CommentSharerePost-User-5679787answered 7 months ago"


Question: Unable to SSH to the ubuntu managed node using instance-id

"Dear all,I set up the session manager correctly a while ago. I was able to connect to my ubuntu-managed node through:the System Manager Consolethe AWS CLI ( Session manager Plugin is installed on my local Mac)SSH using instance-id instead of host public IPI need the last one because I also need to transfer files between my local and the managed nodes.But after a few months, today I tried to connect using ssh:ssh -i :path/to/file.pem ubuntu@{instance-id}But I'm not able to connect using ssh anymoressh: Could not resolve hostname {instance-id}: nodename nor servname provided, or not knownThe other starting session methods still work very fine and I am also able to SSH to the server using the host IP.OS Version:                         Ubuntu 18.04.4 LTSamazon-ssm-agent:           3.1.1188.0Local AWS CLI:                    aws-cli/2.4.28 Python/3.8.8 Darwin/21.5.0 exe/x86_64 prompt/offsession-manager-plugin  1.2.295.0Any help is really appreciatedRegardsHamed.FollowCommentsteven a year agoHello,Is there a chance that your instance was stopped at some point then you restarted it? Was it attached to an ElasticIP before stopping it? It could be that you IP address changed after starting your instance.Share"

Answer: "1Accepted AnswerIt appears that ssh is trying to resolve the instance id instead of starting the proxy command from ~/.ssh/config file. To allow SSH connections through Session Manager, the ssh configuration file must have a proxy command that starts a Session Manager session and transfer all data through the connection. Make sure that ssh config file for the logged in user ( in your Ubuntu machine) has following entry in it.$ cat ~/.ssh/config# SSH over Session Managerhost i-* mi-*    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"For more details, refer to the following documentation.https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html#ssh-connections-enableCommentShareRenjith_Ranswered a year agorePost-User-3169671 a year agoYes, that resolved my problem. Actually, it should be configured on the local ssh configuration. Thank you.Also, the needed policy permissions were missing for the SSH connection through the session manager.Share"


Question: Using NICE DCV for game streaming - poor fps?

"Hey there,Is anyone using NICE DCV to stream games, such as games on the Steam platform?I ask because this is what we're trying to do. However, even on a good internet connection to the client, I am seeing fps (frames/sec) on the host instance side of only 60-40fps, and on the client side as low as 15fps. This is not really usable for games.But... then I see on YouTube a few people showing games like Tomb Raider running at great frame rates, even 4K video. I have no idea how they are getting these frame rates/fps. Any ideas?Our NICE DCV instances are in eu-west-1 region, and accessed in Ireland/UK. I am guessing maybe our issues is the relatively poor internet speeds in the UK compared to places like Canada, Paris, USA.I wonder if NICE DCV/AppStream is just not the proper use case for game streaming. Is it more aimed at graphical desktop/app streaming. Interested in other people's feedback. Should I just give up or is there something I can tweak on the instance/NICE DCV protocol side?Thanks a lot!FollowComment"

Answer: "0Hi,there are great examples of how well DCV works for games - e.g. https://www.youtube.com/watch?v=aJ8MO6SfvpoCan you check the FPS and bandwidth available which you can find in Settings (wheel to the upper left) -> Open Connection Metrics ?By default the FPS on the server side might be limited to 25 FPS depending on the OS. Here is more how to tune DCV: https://docs.aws.amazon.com/dcv/latest/adminguide/config-param-ref.htmlBest regards,KarstenCommentShareKarstenNISPanswered 3 years ago0Thanks Karsten for your reply.I have watched the video and a few others, and this is what prompted me to ask the community how others are achieving these frame rates at 1080p over NICE DCV. I can see in the video he is getting 60-80fps @ 1080p via the Steam fps counter, and the NICE DCV counter is reporting a locked 60fps. Bear in mind he's playing the game on HIGH graphics settings rather than ULTRA. Unfortunately there's no detailed technical info in the video regarding how he configured the NICE DCV instance.For my testing, I've reviewed the same fps counters and with a similar bandwidth and latency to his (50-70mbps and 15-20ms), the Steam fps counter never goes above 50 and fluctuates down to 30fps. The NICE DCV counter also fluctuates and never locks at 60, it's more like 30-40fps.Thanks, I also recently learnt about the DCV tuning configuration in the Windows registry. Indeed by default it's capped at 25fps. You need to add the "target-fps" registry entry and specify something like 60 (decimal) to specify that upper limit. You can also set the value to 0 for unlimited fps.Whilst that registry setting did help, it only took the NICE DCV FPS from 15 to 30fps.So, I'm still perplexed how these higher locked 60fps frame rates are being achieved. I can tell by the way videos play, and games play, over NICE DCV, that the frame rates are not smooth or consistent. I am hoping there's some other config I'm missing and someone is going to enlighten me :-)Thanks a lot.CommentShareMilkyMagicanswered 3 years ago0Hi,in case your network to AWS shows 50-70mbps and 15-20ms (which the DCV client metrics confirm i assume) then with 20ms you should be able to get 50 fps or more.Are you using the native DCV client (not the web browser) for best performance and can you check if the CPU on your client is maybe the bottleneck (process monitor)?Best regards,KarstenCommentShareKarstenNISPanswered 3 years ago0Hi Karsten,Thanks, that's a very good point. I was using the web client and forgot that the DCV client is more performant ;-) I've now switched to that and it's definately now feeling more "fluid" when navigating with the mouse in-game. The fps reported by the DCV client metrics is a little better, average 40fps, unfortunately still not a consistent 60fps.I did check the client CPU and that's looking good at 20% utilised. The DCV viewer process is only consuming around 2% CPU.What I am seeing is better fps for static screens or videos within the game. But once the real gameplay starts, the performance dips and is never consistent. Looking back at that YouTube video where he's playing Tomb Raider, he is seeing similar performance issues actually. Watch it closely... When he stops moving the character, the fps is great. When he starts moving around the world, he also sees drops to 30fps. If a discrete Nvidia GPU on a physical PC had the same vRAM as the Tesla one inside the g4dn instances (16GB vRAM), the performance would be great. But inside AWS via NICE DCV, that GPU vRAM does not seem to translate to consistent frame rates in-game.I am conducting further tests in an AWS region a little closer to me. I am also trying to obtain more detailed specs on these Tesla T4 cloud GPUs. eg, Are they optimised for 3D modelling/CAD, rather than games where you need the GPU to push out millions of polygons?Thanks!CommentShareMilkyMagicanswered 3 years ago0Hi,great to hear the progress! I would assume the T4 are the same as the physical ones you can buy but that is just a guess ;)You could also check how the load on the CPU is or if there are other bottlenecks on the server side which would explain dropping FPS on the server side when there are strong movements.If you think we have reached a conclusion please feel free to mark as solved.Best regards,KarstenCommentShareKarstenNISPanswered 3 years ago0Thanks for your continued help.Unfortunately we don't have a solution just yet. I am conducting further in-depth testing and I am finding initially that the type of game really is a factor. Potentially these cloud Teslas are optimised for AI and compute, and in a game with a high number of polygons/character assets when you navigate with the camera, the performance is taking a real hit. The YouTube Tomb Raider video also seems to confirm that.I did also already review detailed Windows performance counters on the host EC2 instance side, and ruled out any bottleneck when it comes to CPU, RAM, disk, and network. It's the GPU that's reporting 100% utilisation.Thanks again, will report back when I have more test results.CommentShareMilkyMagicanswered 3 years ago0In case the GPU is the limiting factor did you try a more powerful one like the V100 in the p3.2xl in case?CommentShareKarstenNISPanswered 3 years ago0Thanks for the suggestion.I believe only the g family instances are suitable for NICE DCV. Using the official AMI reveals they don't support the p family. Looks like that instance family is designed for machine learning, big compute, rather than graphics. Those p instances do have multiple Tesla GPUs, but bear in mind most games will not take advantage of multi-GPU (equivalent of Nvidia SLI).In fact, when I tested a larger g instance with multiple GPUs, it made no difference to fps/performance for gaming.Thanks anyway, I think the key thing here is confirming if these cloud-based Tesla GPUs have the relevant polygon/shader/texture support for games. Based on testing so far, they struggle to get above 30fps when there are many character assets/polygons in motion.CommentShareMilkyMagicanswered 3 years ago0Most games it seems will not benefit from multiple GPUs. The P instance with 1 V100 can likely be used for games as well at least with Windows. So it seems in the case of T4 the GPU is the bottleneck - did you check with nvidia-smi -a or other tools which part of the GPU is at 100%?CommentShareKarstenNISPanswered 3 years ago0Thanks, I was not aware of that command. If you have any further detail, I'll check, thanks a lot.CommentShareMilkyMagicanswered 3 years ago0I've run "nvidia-smi -a" when the game streaming fps dropped low, and forwarded that on to AWS Support.Key performance stats:Utilization      Gpu                               : 98 %      Memory                            : 47 %      Encoder                           : 3 %      Decoder                           : 0 %  Encoder Stats      Active Sessions                   : 1      Average FPS                       : 34      Average Latency                   : 2227CommentShareMilkyMagicanswered 3 years ago0Great to see you have interesting numbers from nvidia-smi confirming the assumption that the GPU and in this case the utilization seems to be the bottleneck. Please keep us posted on findings.CommentShareKarstenNISPanswered 3 years ago0Hi, How were you able to connect to Nice DCV? I have a G4 instance, with BYOL windows 10 . I have all drivers and I have been able to Stream using parsec, however now that I want to try DCV, I am just not able to connect. Port is open and included in security group, however whenever I type IP or DNS in connection string, it just keeps spinning. Any idea?ThanksCommentShareRgarza90answered 3 years ago0Hey there,I don't believe that will work as you need one of the NICE DCV AMI-based instances, these are actually hosted on Windows Server rather than Windows 10.For example, this is the AMI I've been using:https://aws.amazon.com/marketplace/pp/B082HYM34K?ref=cns_srchrowI honestly don't know if Windows 10 will work or is supported. If you scroll down on that web page link, the Usage Instructions may help you. Other than the security group inbound rule on TCP port 8443, did you make sure to grant the instance IAM role permissions to read the NICE DCV licence from the S3 bucket?Also the usual things to check, such as, is there a Network ACL in the VPC denying inbound/outbound traffic.Thanks, hope that helps.CommentShareMilkyMagicanswered 3 years ago0Hi,did you check the DCV logs about any messages? Windows DCV Logfiles:   C:\ProgramData\NICE\dcv\log\More info about installing DCV on Windows is at https://www.ni-sp.com/how-to-install-nice-dcv-on-windows/Best regards,KarstenCommentShareKarstenNISPanswered 3 years ago0Nice Karsten, I did not realise you can install this manually onto Windows client OS-based instance if you have BYOL.One point mentioned in that guide, also make sure the Windows Firewall inbound rule is configured at the OS-level, as well as the security group/NACL at the AWS level.CommentShareMilkyMagicanswered 3 years ago0Thanks for replying. Well, I opened port 8443 for UDP following a tutorial, should I be opening TCP port as well? I also was not aware that it needs to read the license from S3 bucket, however i was able to install it with no issues. Regarding the VPC i am using default VPC, so i guess it should not have an ACL blocking traffic, however i will check.I am using windows 10 as BYOL as i wanted to try some games available on Microsoft store which was not available thorugh windows server, maybe that could be the issue, that this only works with windows server.Thanks.CommentShareRgarza90answered 3 years ago0TCP 8443 needs to be opened in the security group for incoming. That seems to be the main issue right now.S3 bucket missing will give you a notice.Best regards,KarstenCommentShareKarstenNISPanswered 3 years ago0Yes, you definately only need that TCP port opened, it works fine for me without needing any UDP port rule.The NACL for the default VPC is allow all inbound & outbound, but worth checking. Also worth checking which subnet you launched your EC2 into. Obviously it needs a route to the internet (assuming your client is remote somewhere), so either put the instance in a public subnet, or place in private subnet with a NAT gateway.An update on the original fps performance issue: My support case is ongoing with AWS, they are reviewing...Edited by: MilkyMagic on Oct 14, 2020 7:39 AMCommentShareMilkyMagicanswered 3 years ago0Thanks for following up guys, I opened TCP port and there is an improve now. Before It was just not able to reach the server, now I get a prompt asking for user, password,  It says starting session and then it errors out saying "unable to connect, checkhostname/address and try again" , this is strange, Ihave checked in the server and the service is up and running. I assume the user and password is the one for the PC?Regarding the VPC, I checked the default VPC, the subnet is using a route table mapped to an internet gateway, so i would think thats not the issue.I know tit is trying to connect because it disconnects any remote connection i have using RDP or Parsec, so it is definitely reaching the computer, however something is killing the connection at the end.Hopefully you get a reply from support as i am trying to use this for gaming as well.Edited by: Rgarza90 on Oct 14, 2020 4:04 PMEdited by: Rgarza90 on Oct 14, 2020 4:15 PMCommentShareRgarza90answered 3 years ago0Good to hear you are making progress. Your VPC config is good.I would definately make sure all RDP sessions are LOGGED OFF, and not just disconnected. Also make sure Parsec service is disabled. I know these other products can conflict with NICE DCV. To get a clean test, you may want to deinstall Parsec, logout any RDP sessions, reboot, and then attempt connection to NICE DCV again.For a simple test, I normally connect to NICE DCV using "Administrator" user. The password will be the one you configured for the local Windows Administrator account. For EC2 instances, you would have a key pair and you have the associated private key. Inside the EC2 console, you would select the Instance, then go to the ACTIONS menu and select "Get Windows Password" - that's the local Administrator password you need to connect over NICE DCV.There's some detailed info here in the docs around connection configuration, and the steps required to ensure your EC2 can reach the S3 licensing bucket:https://docs.aws.amazon.com/dcv/latest/adminguide/dcv-ag.pdfI can see that Windows 10 is indeed supported as a NICE DCV server host, as Karsten commented. Also worth checking, on a Windows Server EC2 instance, the EC2Config service is preconfigured to use the metadata to grab it's credentials and then connect to S3. If you're using your BYOL Windows 10 instance, maybe this EC2Config service is missing and thus causing failure. The steps to check/configure that are in the doc above.Also, check for error messages in the NICE DCV log here:C:\ProgramData\NICE\dcv\log\server.logHope that helps.Edited by: MilkyMagic on Oct 15, 2020 12:49 AMCommentShareMilkyMagicanswered 3 years ago0Thanks, I havent tried uninstalling PARSEC, also I dont need to generate dindows password, basically since this is a Virtual machine of windows 10 that I imported to create an AMI, I created my Admin user and password once I created the virtual machine, so all of these steps I can skip.I have my admin account and this is the one i am using to log in to my windows 10, but howw can you log in to Nice DCV using your user?ThanksCommentShareRgarza90answered 3 years ago0First off, let's provide an update on the fps frame rate challenges...The answer to the original topic: Can we get good consistent 60fps via NICE DCV for game streaming, is a "maybe". After extensive analysis it really does depend which game you are trying to stream. The Tesla cloud GPUs are designed for CUDA compute so you'll never get the same experience compared to a physical gaming PC with a consumer graphics card. The NICE DCV protocol itself is extremely efficient with very low latency, it's the GPU that's the bottleneck here for gaming.I would encourage people to test the specific game(s) you want to stream via NICE DCV. If there are few character assets on-screen at once, you'll most likely get very good fps. However, if the game features hundreds of NPCs/characters you will see the frame rate drop significantly.This situation most likely will improve over time. We know that AWS are very good at constantly improving their services. Let's hope we see the game-focused optimisations of the NVIDIA 30-series Ampere consumer GPUs make their way over to the cloud GPUs in the near future maybe.I'll mark this question as answered.For the other question around Windows 10 BYOL instance and NICE DCV, let's get that set up as a separate discussion, thanks.CommentShareMilkyMagicanswered 3 years ago0Thanks for your comprehensive answer! Did you have the chance to try a V100 based P3 instance for better FPS and performance (the T4 might show half of the performance here compared to the V100)? Should be easy to switch changing the instance type. I guess the 30xx consumer cards will not make it but the A100 was announced.CommentShareKarstenNISPanswered 3 years ago0Hey Karsten, actually I did not test P3 instances, I will spin up and test over the next few days, and report back a performance comparison.Cheers for the suggestion :-)CommentShareMilkyMagicanswered 3 years ago0Yes, just to confirm, I've tested a p3 instance using both GRID and Tesla drivers. All properly licensed and confirmed detected as NVIDIA Quattro Workstation.Same fps frame rate performance issues unfortunately. Noteworthy that according to the AWS GPU docs, only the g4 instances are officially supported to use the NVIDIA vGaming licence and gaming driver.Oh well, was worth a try :-)CommentShareMilkyMagicanswered 3 years ago0Interesting! Would have expected the V100 performs significantly better and produces more FPS - apparently it does'nt. Yes, worth a try ;)CommentShareKarstenNISPanswered 3 years ago0Just providing an update here.I found an AWS instance config that gives me much better frame rates for the intensive games I was testing. So around ~100fps compared to ~40fps.  This is the p3.8xlarge instance with EGA (Elastic Graphics Acceleration) ENI attached. This boosts the GPU with an 8GB vRAM GPU-based ENI, however it only supports OpenGL vs DirectX.It seems to have fixed a lot of the poor frame rate issues. However, the graphical fidelity takes a hit, most likely because the EGA is not taking care of things via DirectX. For example, the graphics show a lot of “resolving” and gradients when moving around the environment.So not a perfect solution, but at least we have some options for improved frame rates. AWS also recently announced the first Ampere-based GPU instances in USA, the p4dn instance type. However, it still appears tuned for ML/compute, and there's no NVIDIA gaming-type driver or AMI for that instance type yet.Hope that helps others.Edited by: MilkyMagic on Nov 3, 2020 8:09 AMCommentShareMilkyMagicanswered 3 years ago0Thanks for sharing your interesting findings, MilkyMagic!Are you still using the built-in 4 V100 as well in this scenario included with the p3.8xlarge and EGA - so virtually 4+1 GPUs?CommentShareKarstenNISPanswered 3 years ago0That's correct, Windows Device Manager did detect the 4 x GPUs. The elastic ENI is hidden from Device Manager, you need to follow the AWS docs to load up a utility to support that, plus your instance needs a port rule adding to the security group:https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/elastic-graphics.htmlThe instance type is of course quite expensive, but via spot pricing it's not too bad. Just also note certain regions/AZs may not be successful launching these instances based on capacity at AWS.Also worth stating I used this AMI: NVIDIA Quadro Virtual Workstation - WinServer 2019http://aws.amazon.com/marketplace/pp/B07TS3S3ZH?ref=cns_srchrowPotentially could save a bit of money dropping to the p3.2xlarge instance which also supports this AMI.CommentShareMilkyMagicanswered 3 years ago0Hi!Just wanted to let you know that the new DCV 2020.2 version is out with QUIC based UDP support making cloud based gaming even more feasible with high FPS on the transport layer. You can enable QUIC in the DCV server and choose in the new DCV 2020.2 client.Best regards,KarstenCommentShareKarstenNISPanswered 3 years ago0Ah yes thanks for the heads-up, I just learnt that too.We'll do another test and report back, let's see what performance we gain from the NICE DCV improvements versus the underlying EC2 GPU hardware remaining the same.CommentShareMilkyMagicanswered 3 years ago0Right, I've tested this latest NICE DCV release, making sure both the server and client were updated.No improvement in frame rate and performance unfortunately, still inconsistent and averaging around ~40fps. Feels a little worse actually.To be honest, I believe the NICE DCV protocol was always performant and not the key bottleneck. That bottleneck I believe is still the lack of consumer GPU performance within the cloud instances.CommentShareMilkyMagicanswered 3 years ago0Indeed - the main FPS limiter seemed to be in your case the rendering GPU.To use QUIC which is offering e.g. 60 FPS in our demo cases also with higher latencies you can enable it in the registry (https://docs.aws.amazon.com/dcv/latest/adminguide/enable-quic.html) and configure to use it in the new DCV client as well.Here is some background on the configuration: https://www.ni-sp.com/nice-dcv-tips-and-tricks/#h-how-to-enable-the-udp-based-quic-transport-protocol-in-dcvCommentShareKarstenNISPanswered 3 years ago0Good point, I may have spoken too soon! Let me double-check this config, and re-test, thank you.Edited by: MilkyMagic on Nov 13, 2020 3:57 AMCommentShareMilkyMagicanswered 3 years ago0OK I've retested, no DICE unfortunately (pun intended).I've enabled QUIC UDP on both the server and client, and I've confirmed via the client's streaming monitor that it's using QUIC.We are still getting ~40fps locally on the instance, and DCV is reporting ~18fps on the client side via NICE client over QUIC. I believe the QUIC improvements will help with latency over the network. But the issue lies not with the network or the streaming protocol, it's  the GPU hardware locally in the instance itself  that is not performant for games, it was never designed to be.CommentShareMilkyMagicanswered 3 years ago"


Question: Can't migrate Classic Elastic IP with Reverse DNS to VPC Elastic IP

We're trying to move our infrastructure from EC2 Classic to VPC as requested by Amazon. We have an Elastic IP address with Classic scope which has a reverse DNS record associated with it (managed by Amazon) and we need to migrate this Elastic IP address to VPC. When we try we receive via the console we receive the message:Elastic IP address could not be moved to VPC scope. Address with allocation_id:xxx has a PTR record setHow do we go about migrating this Elastic IP address to VPC?FollowComment

Answer: "0This might help. You have to contact your ISP to have the IP tied to the new domain name you assigned it.   This isn't done in your domain DNS entries.  If you have static IPs, just call them and tell them you need to update a reverse DNS PTR record for your IP address that represents your Elastic IP.Edited by: Jccrzaa on Oct 28, 2021 11:43 PMCommentShareJccrzaaanswered 2 years ago0Thanks, though the issue in this case is that Amazon assign the reverse DNS (PTR) entry as part of removing the restriction on the EC2 instance sending emails (see https://aws.amazon.com/premiumsupport/knowledge-center/ec2-port-25-throttle/)In the end I solved it by using the form listed in that article and filling it in asking the Amazon team to remove the PTR record, or assign a PTR record for a new VPC-scope Elastic IP address we created. They seem to have done the latter, so I just need to fill the form in again to ask them to remove the PTR record from the Classic-scope Elastic IP address which will then allow me to release it.CommentShareDr David Greyanswered 2 years ago"


Question: API Gateway cares about my Authorization header when it shouldn't

"My setup is:Private API Gateway REST API with a GET method that invokes a LambdaA VPC from which the API will be invokedVPC endpoint for API Gateway (execute-api). The endpoint (as well as the rest of the VPC infrastructure) is created via CloudFormation.There is no Authorization set on the method. It's just a simple test API as a proof of concept. However it will need to accept a header called "Authorization", which is something specific to our company. I expect this header to be passed through to the Lambda as part of the integration.The problem is that when I supply an Authorization header, API Gateway rejects the request with the following message (taken from API Gateway CloudWatch logs):IncompleteSignatureException Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=\[the header content here]  If I remove the Authorization header, the request goes through to the Lambda and I get the expected response.Now the interesting part is this: if I delete the execute-api VPC endpoint and create it manually via the console, everything works fine - my Authorization header is passed through to the Lambda without problems. But if I create the endpoint via CloudFormation, API Gateway suddenly cares about that header's content. But the endpoint itself appears to be the same when created manually or via CF: dns is enabled, same 3 subnets, same security group, same resource policy (allow all access for everyone to everything). I can see no difference whatsoever. But evidently something somewhere is different, but I have no idea what. I've recreated it manually and via CF several times and the result is always as described above.What I have found, not sure how relevant it is, is that the amazon headers logged by API Gateway are slightly different for the manual vs CF endpoints (this is with the Authorization header removed, as the CF one rejects before logging headers). With the CF endpoint, the "Method request headers" logged include "x-amzn-vpce-config=0" and "x-amzn-vpce-policy-url=MQ==". With the manual endpoint, I get "x-amzn-vpce-config=1" and the policy-url header isn't included. Googling these headers gives me absolutely nothing.Also possibly worth noting, I've tried both specifying the endpoint in the API (in the console, API settings, you can select the endpoint for a private API) and leaving it blank. Redeployed API gateway after. Neither change has any effect on the behaviour described above.Does anyone know why it's behaving like this?FollowComment"

Answer: "0I managed to get it working. This is an AWS bug.This is a snippet of the CF to setup this endpoint:ApiGatewayVPCEndpoint:  Type: AWS::EC2::VPCEndpoint  Properties:    PolicyDocument:      Statement:        - Action: '*'          Effect: Allow          Resource: '*'          Principal: '*'This is that policy shown in the console:{    "Statement": [        {            "Action": "*",            "Effect": "Allow",            "Resource": "*",            "Principal": "*"        }    ]}This is the policy returned when calling aws ec2 describe-vpc-endpoints"PolicyDocument": "{\"Statement\":[{\"Action\":\"*\",\"Resource\":\"*\",\"Effect\":\"Allow\",\"Principal\":\"*\"}]}"Now if I create the endpoint manually, this is the policy in the console:{    "Statement": [        {            "Action": "*",            "Effect": "Allow",            "Resource": "*",            "Principal": "*"        }    ]}And this is the policy returned when calling aws ec2 describe-vpc-endpoints"PolicyDocument": "{\n  \"Statement\": [\n    {\n      \"Action\": \"*\", \n      \"Effect\": \"Allow\", \n      \"Principal\": \"*\", \n      \"Resource\": \"*\"\n    }\n  ]\n}"Basically the same except for some whitespace and newlines you'd get when prettifying the JSON, but of course it's irrelevant to the actual JSON data. Or so you'd think. Actually that is the reason why it is failing in the YAML version. When AWS transforms the YAML to JSON, it doesn't put newlines in there. And somehow that causes the policy to not work. If I deploy this:ApiGatewayVPCEndpoint:  Type: AWS::EC2::VPCEndpoint  Properties:    PolicyDocument: '    {      "Statement": [        {          "Action": "*",          "Effect": "Allow",          "Resource": "*",          "Principal": "*"        }      ]    }'Then it works perfectly. And even if I supply the JSON on a single line, with no whitespace, it still works fine (describe-vpc-endpoints shows newlines and whitespace added).Obviously (I'd like to say) the issue here isn't whether irrelevant newlines exist in the JSON or not, but rather how that JSON is interpreted. It seems like there is string parsing going on here rather than converting the JSON to an actual object (or whatever method you might normally use to access some path in a JSON string), which is getting tripped up by lack of newline characters. Anyway, just a guess, but what isn't a guess is that this is a bug.Edited by: mspo2 on Oct 18, 2021 7:27 AMCommentSharemspo2answered 2 years ago"


Question: greengrass logfile upload on demand

"I have no need to upload all logfiles, of all Core Devices in a deployment group, all the time.I would just like to ask for logs of a specific Device if things go wrong.The documentation of logmanager indicates that this is more of a permanent continuous logfile upload.Is there an existing way to 'trigger' a single upload of all/selected greengrass logs ?(using GG v2)FollowComment"

Answer: "1Accepted AnswerHi there,Unfortunately logmanager doesn't currently support just a one off log file upload.You can choose component logs you want log manager to upload by specifying them in the log manager confighttps://docs.aws.amazon.com/greengrass/v2/developerguide/log-manager-component.html#log-manager-component-configuration . But please note these would be continuous uploads until you change the log manager config back and make another deployment.Thanks,Navya.CommentShareNavyaKuchibhotla-AWSanswered a year agoclogwog a year agothank you Navya, i was afraid so, but it is good to hear it confirmed so i can try to find a different solution.Share"


Question: AWS SF Express Workflow

"Hello,Quick question regarding AWS SF: Can you have an SF with type Express and then use it as both Synchronous and Asynchronous? For example, we want it to be Synchronous if triggered by APIGW, but Asynchronous if triggered by a lambda (that is connected to an SQS)FollowComment"

Answer: 1Yes you can. The same express workflow can be invoked both synchronously or asynchronously. Standard workflows can only be invokes asynchronously.CommentShareEXPERTUrianswered a year ago0Yes. https://docs.aws.amazon.com/step-functions/latest/dg/concepts-express-synchronous.htmlCommentShareMarcin Manswered a year ago


Question: Athena Query is not running

Answer: "0Where did this query come from?  I cut-and-pasted it into an editor and precisely at position 13527 as reported, there's a newline.  There aren't any other newlines, so looks like this is tripping it up.  It's just after "in_GeneralOrchFolderPath:string,".CommentShareEXPERTskinsmananswered 4 months ago"


Question: immigration database

"is it safe to leave my database on worbench when deploying my site on awsFollowCommentalatech EXPERT2 months agoCan you elaborate?ShareVivek Pandey 2 months agoTo migrate you can use AWS Database Migration Service (AWS DMS) to migrate your data to and from most widely used commercial and open-source databases such as Oracle, PostgreSQL, Microsoft SQL Server, Amazon Redshift, Amazon Aurora, MariaDB, and MySQL. The service supports homogeneous migrations such as Oracle to Oracle, and also heterogeneous migrations between different database platforms, such as Oracle to MySQL or MySQL to Amazon Aurora MySQL-Compatible Edition. Alternatively, you can use AWS DMS to move from existing, self-managed, open-source, and commercial databases to fully managed AWS databases of the same engine. You can refer this step by step walkthrough:-> https://docs.aws.amazon.com/dms/latest/sbs/dms-sbs-welcome.htmlShare"

Answer: "0Accepted AnswerCould you please shed some more light on your question?CommentShareVetrivelanswered 2 months ago0I deployed my java spring boot su elasticbeanstalk project and my database on rds and I manage my database via mysqlworkbench software and now I have a connection problem with my database because I can't connect another time to host rds , my question is there is a solution to immigrate my database (the tables) to an aws serviceCommentShareAchemlalanswered 2 months ago"