Question: "Athena query analysis all data, rather than specified columns"

"I've created an Athena table to analyse ALB logs stored in s3 using the recommended create table command in the AWS docs. I'm running some queries on this table to retrieve data from specific columns. However, when I look at the data analysed, it appears the data used in the query amount the the total data in the logs files. These log files contain many columns that I am not referencing in my query.I thought if I am only retrieving data from specific columns, that it would only  search the data from those columns, and therefore the data used in the query would be less?Is there something I'm missing here?Query:SELECT FROM_UNIXTIME(FLOOR(TO_UNIXTIME(parse_datetime(time,'yyyy-MM-dd''T''HH:mm:ss.SSSSSS''Z'))/60)*60) as time, ssl_protocol, count(ssl_protocol) as total FROM alb_logs WHERE ssl_protocol LIKE '%' AND request_verb = 'POST' AND REGEXP_LIKE(request_url, 'https:\/\/myapp.*\/v3\/me\/order.*') AND elb_status_code = 200 AND parse_datetime(time,'yyyy-MM-dd''T''HH:mm:ss.SSSSSS''Z') BETWEEN TIMESTAMP '2021-12-06 12:12:44' AND TIMESTAMP '2022-01-05 12:12:44' GROUP BY 1, ssl_protocol ORDER BY 1 limit 20FollowComment"

Answer: "2Accepted AnswerHi,no you are not missing anything, the behaviour you are seeing is normal, the ALB log files are compressed text files, and the table used in the documentation you mentioned is not partitioned.This means that to find the rows you need in your query Athena still needs to open and scan all file to make sure there are no rows needed by the query in that file. As for the column the log file stores all columns for a specific row (it is just a text string) in the same file; so to extract the columns you need Athena still need to scan all files.Now, if you want to optimize the cost of your query you have some options:modify the Athena table and introduce Partition Projection using a new partition column for the date (based on the documentation the log files are actually stored with a prefix structure that include yyyy/mm/dd . for example:CREATE EXTERNAL TABLE IF NOT EXISTS alb_log_partition_projection (type string,time string,elb string,client_ip string,client_port int,target_ip string,target_port int,request_processing_time double,target_processing_time double,response_processing_time double,elb_status_code string,target_status_code string,received_bytes bigint,sent_bytes bigint,request_verb string,request_url string,request_proto string,user_agent string,ssl_cipher string,ssl_protocol string,target_group_arn string,trace_id string,domain_name string,chosen_cert_arn string,matched_rule_priority string,request_creation_time string,actions_executed string,redirect_url string,lambda_error_reason string,target_port_list string,target_status_code_list string,classification string,classification_reason string)PARTITIONED BY(mydate STRING)ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'WITH SERDEPROPERTIES ('serialization.format' = '1','input.regex' ='([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*):([0-9]*) ([^ ]*)[:-]([0-9]*) ([-.0-9]*) ([-.0-9]*) ([-.0-9]*) (|[-0-9]*) (-|[-0-9]*) ([-0-9]*) ([-0-9]*) \"([^ ]*) ([^ ]*) (- |[^ ]*)\" \"([^\"]*)\" ([A-Z0-9-]+) ([A-Za-z0-9.-]*) ([^ ]*) \"([^\"]*)\" \"([^\"]*)\" \"([^\"]*)\" ([-.0-9]*) ([^ ]*) \"([^\"]*)\" \"([^\"]*)\" \"([^ ]*)\" \"([^\s]+?)\" \"([^\s]+)\" \"([^ ]*)\" \"([^ ]*)\"')LOCATION 's3://yourS3bucket/AWSLogs/youraccount/elasticloadbalancing/us-east-1/'TBLPROPERTIES ('has_encrypted_data'='false',"projection.enabled" = "true","projection.mydate.type" = "date","projection.mydate.format" = "yyyy/MM/dd","projection.mydate.range" = "2021/01/01,NOW","projection.datehour.interval" = "1","projection.datehour.interval.unit" = "DAY","storage.location.template" = "s3://yourS3bucket/AWSLogs/youraccount/elasticloadbalancing/us-east-1/${mydate}")if you want to further optimize the cost of your queries and also performance , you could use a CTAS query in Athena to create an new table in Apache Parquet format (highly compressed and columnar format) and define the partitions you need (probably just date) . In this case not only your WHERE clause will limit the files to scan based on the time interval you are interested in , but it will only scan the column you are interested in.For Example:CREATE TABLE alb_parquet3WITH (      format = 'Parquet',      write_compression = 'SNAPPY',      partitioned_by = ARRAY['mydate'],      external_location = 's3://yourS3bucket/data-lake/alb-pqt/')as select * from "alb_log_partition_projection" orCREATE TABLE alb_parquet3WITH (      format = 'Parquet',      write_compression = 'SNAPPY',      partitioned_by = ARRAY['mydate'],      external_location = 's3://yourS3bucket/data-lake/alb-pqt/')as select *, substr(time,1,10) as mydate from "alb_logs" with approach 2, you will then need to schedule a batch to insert data in the new table. Having partitioned the table by day you will need to do the insert once a day to avoid duplicate.Otherwise you can make the process to convert the data a bit more complex and you could drop the partition for the current day, delete the prefix for the current date from S3 and finally call the insert with a where clause on data only for the current day. In this case you could even run the incremental load process every few hours or every few minutes.In this case having the original table with partition projection allows you to scan less data during the incremental process.you can also check this knowledge base article.hope this helpsCommentShareEXPERTFabrizio@AWSanswered a year agoAWS-User-0601396 a year agoAm I right in thinking that if option 2 is chosen, option 1 in no longer needed? Thanks!ShareAWS-User-5296216 a year agoHi Thanks for the thorough answer, that's helpful. About option 2, when you say "define the partitions you need (probably just date)" , by date, do you mean the actual column in the original table (in this case, named 'time'), or do you mean the 'date' partition that was created in option 1? If you mean the latter, I'm not sure I understand how this can be utilised when using CREATE TABLE AS?ShareFabrizio@AWS EXPERTa year ago@AWS-User-0601396, If you use the CTAS to create the partitioned Parquet table you are right , because the partition will be in Hive style.(you will need to consider a batch process to insert the new partitions once a day.If you have another table in parquet format but with prefixes like yyyy/month/day you could actually combine the 2 methods.Share1The ALB access logs are in fact stored as text files and not in columnar format such as Parquet. As a result, the amount of data scanned is independent from number of columns in the Select statement. To enhance your query performance (and reduce data scanned), an option would be implementing partition projection with Amazon Athena - https://aws.amazon.com/premiumsupport/knowledge-center/athena-analyze-access-logs/With this approach, you will also need to rewrite the query with the Where clause using the "year", "month" and "day" partitions/columns in addition of "time".CommentShareCorvus_Lanswered a year agoEXPERTFabrizio@AWSreviewed a year ago"


Question: AWS Control Tower - Not Governed Regions

"Hi all,We are using AWS Control Tower to manage AWS Accounts in our Landing Zone,Unfortunately one of our principal regions (eu-south-1) isn't governed by Control Tower, so in this case we need to setup/create resoures manually or using Terraform.What we are trying to achieve is to setup a Terraform Pipeline which will enable ** AWS Config** and Create some Config Rules we are using usually in the regions Not-Governed by Control Tower.But the main issue right now is the following statement of an SCP created by Control Tower to prevent any modifications to AWS Config within the AWS Organization :    {      "Condition": {        "ArnNotLike": {          "aws:PrincipalARN": "arn:aws:iam::*:role/AWSControlTowerExecution"        }      },      "Action": [        "config:DeleteConfigurationRecorder",        "config:DeleteDeliveryChannel",        "config:DeleteRetentionConfiguration",        "config:PutConfigurationRecorder",        "config:PutDeliveryChannel",        "config:PutRetentionConfiguration",        "config:StopConfigurationRecorder"      ],      "Resource": [        "*"      ],      "Effect": "Deny",      "Sid": "GRCOXXXXXX"    }We tried to disable the Deny Regions feature in out Landing Zone but no success, the SCP is still there !!Does anyone have idea how to overcome this issue  ?any help would be greatly appreciatedThanksPeterFollowCommentAnna 4 months agoThis SCP is applied by Control Tower guardrail and not by the Deny Region feature. That guardrail is mandatory.I would try and temporarily assume ControlTowerExecution role and setup whatever resources you need in that region. Just keep in mind that you would need to remove the Config setup once the region will be available for governance under Control Tower.SharePeter Eskandar 4 months agoHey Anna,Thanks for the comment, seems to be the only way to do so is to assume the ControlTowerExecutionRole !!Anyway I'm talking to our TAM to check if there are any other solutionsShare"

Answer: "0Hello. You need to remove that region from governance by your Control Tower landing zone. Go to AWS Control Tower in the console then find landing zone settings in the menu. Once there, click the modify settings button then remove the eu-south-1 region. Let me know if that fixes your issue.CommentShareGrantAWSanswered 4 months agoPeter Eskandar 4 months agoHi,actually eu-south-1 isn't among the list of regions that can be governed by Control-Tower right nowShare"


Question: Does Babelfish support Japanese character sets / collations?

Do you support collations such as Japanese_CI_AS? Any other tips for Japanese customers?FollowComment

Answer: "0Accepted AnswerAs of July 8, 2022, Babelfish 2.1 (needs Aurora 14.3+) and above has support for:Japanese_CS_ASJapanese_CI_AIJapanese_CI_ASBabelfish frequently adds new capabilities, so be sure to check the latest collations documentation for new features:https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/babelfish-collations.htmlPlease speak to your Amazon solutions architect for additional instructions for using Japanese collations with Babelfish.CommentSharelightanswered a year ago"


Question: "Help needed using AWS Encryption SDK, Python and RSA Public/Private keys"

"Hi,I've recently started using the AWS Encryption SDK with multiple KMS keys to encrypt and decrypt files.I've now been asked to investigate using Public key encryption, so that we can have a mix of KMS master key and RSA master key providers.  My thought this would be able to ensure that we can use our own KMS keys for decryption and allows a third party to supply us with their RSA Public key, which can then be referenced for them to decrypt offline.I have reviewed the following example,  https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/python-example-code.html#python-example-multiple-providers , which I thought should solve it for me, but have noted that only the private key portion of the key pair is used for both the encryption and decryption of the file.I am really lost with how to use a public key to encrypt and a private key to decrypt.Is anyone able to point me in the right direction?ThanksEdited by: WranglingData on Jun 28, 2020 10:32 PMFollowComment"

Answer: "0Hi,Take a look at this example for how to initialize a RawMasterKey with access to the public key and a RawMasterKey with access to the private key:https://github.com/aws/aws-encryption-sdk-python/blob/keyring/examples/src/master_key_provider/multi/aws_kms_with_escrow.pyThe RawMasterKey with access to the public key can be used for encryption, and the RawMasterKey with access to the private key can be used for decryption.(Note that this links to example on one of our features branches. For future readers of this post, if this link is dead, look for this example on the master branch.)CommentShareAWS-User-7412981answered 3 years ago0Hi,Thanks.  Works perfectly.Curiously, I thought I had tried a pattern similar to that, but in all honesty I am no longer sure :)Regardless, this is the tonic.  I have also tried it with aws_encryption_sdk.stream(), as I will be dealing with bigger files and as expected, it worked perfectly.I appreciate you taking the time to answer.  Thanks!CommentShareWranglingDataanswered 3 years ago"


Question: Problems with AMIs from Graviton3

"Hello,For the last 4/5 days I've tried to spin up instances from AMIs created with Graviton3. The original OS is Ubuntu 20.04 and the image was created without any apparent problem. However, these AMIs haven't worked yet: the console keeps showing the instance as "initializing" and I cannot log in on the instance. I have completed this process literally hundreds of times with different combinations of instances and OSs, but never ran into this issue.Thanks.P.S. Just for completeness, I launched a c7g.medium with ALinux2 (which is not my usual choice), updated it, created a text file in the home directory, and created the AMI. It doesn't work either, same problem.FollowCommentMichael_F EXPERTa year agoDoes it work with the standard Ubuntu 20.04 AMI for the region? You can find the list of Canonical-published AMIs here (https://cloud-images.ubuntu.com/locator/ec2/) and entering "20.04 arm64" in the search box.Shareafody a year agoHi Michael_F. Yes, I can spin up instances (e.g. c7g.medium) and play around. The problem is only when I store the image, and try to spin up another instance from the new AMI.ShareMichael_F EXPERTa year agoHi @afody, thanks for reporting this. I'm able to reproduce your issue and am escalating internally.ShareMichael_F EXPERTa year ago@afody Does it work for you if you build your AMI from a c6g.medium instance as the source? Can you then launch that on a c7g.medium instance?Shareafody a year agoSorry that I didn't see your comment earlier. I took one of the many AMIs stored in the account (and created with a Graviton2 instance); I had no problem spinning up and connecting to a c7g.medium instance.ShareShow 2 more comments"

Answer: "2Thank you for reporting this. We are able to reproduce the issue, have identified the root cause, and are working on a fix. We'll update this post when the fix has been deployed.In the meantime, we suggest creating AMIs on Graviton2 instances (c6g/m6g/r6g) as a workaround. These AMIs can be used on Graviton3 instances without issue.CommentShareEXPERTMichael_Fanswered a year agoafody a year agoI'll keep an eye on the post. Thanks!ShareMichael_F EXPERT7 months agoI just wanted to let you know that the fix for this issue has been deployed. Thank you for bringing the issue to our attention!Share1While we work on a general fix, you can work around this by executing the following command before creating a AMI on the c7g instance which removes the the EFI Internal Shell from the valid boot options so when the AMI in started on a new instance it will boot via the default media path. After executing the below, either stop your instance and create the image or create the image directly (nominally resulting in an instance reboot). After the reboot you'll have to execute this one-liner again to successfully re-create the image a second time.sudo efibootmgr -B -b `efibootmgr | grep "EFI Internal Shell" | sed -r 's/Boot0*([0-9]+).*/\1/'`CommentShareAli_Sanswered a year ago"


Question: AWSSDK .NET 6.0  Version dll

"Hi We have used AWSSDK.dll to consume S3 APIs in our.NET Framework project, and now we are trying to migrate to NET Core. To our knowledge, there is no.NET Standard or.NET 5.0 Higher version of AWSSDK.dll. Please help us.FollowComment"

Answer: "1Hi, You are correct. To consume Amazon S3 bucket from .NET core use AWSSDK.S3 nuget package which is netstandard2.0 compatible.You can also try Porting Assistant for .NET to convert .NET framework application into .NET Core.CommentSharePankaj Rawatanswered 7 months ago"


Question: Not enough space to install requirements?

My MWAA environment will not properly initialize because the requirements install fails. There is nothing in the documentation about a download size limit and no way to override this. There is only a 'recommendation' about size because it can cause slowness in updating.Error log:2023-05-09T16:13:25.377-04:00      Downloading tensorflow-2.8.2-cp37-cp37m-manylinux2010_x86_64.whl (497.9 MB) 2023-05-09T16:13:27.753-04:00      ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device FollowComment

Answer: "0Per https://github.com/pypa/pip/issues/5816#issuecomment-425410189 you may be able to use --no-cache-dir by installing via startup script, i.e.:#!/bin/shif [[ "${MWAA_AIRFLOW_COMPONENT}" != "webserver" ]]then     # place requirements.txt in DAGs folder, and replace existing requirements.txt with an empty text file     pip3 install --no-cache-dir -r /usr/local/airflow/dags/requirements.txtfiSee https://docs.aws.amazon.com/mwaa/latest/userguide/using-startup-script.html#create-startup-scriptYou may also install the files from WHL per https://docs.aws.amazon.com/mwaa/latest/userguide/best-practices-dependencies.html#best-practices-dependencies-python-wheelsCommentShareJohn_Janswered a month ago"


Question: com.aws.greengrass.lambdamanager.StatusTimeoutException mysterious lambda error

"I am trying to use lambdas on greengrass v2. I created and packaged the hello world lambda from the greengrass tutorial and deployed it to the core but it continues to error out. It doesnt seem to run to completion, or the lambda isnt getting called correctly and the core reports it as timed out. This is the stacktrace. How do I get this lambda to work in greeengrass v2?com.aws.greengrass.lambdamanager.StatusTimeoutException: Lambda status not received within timeoutat com.aws.greengrass.lambdamanager.Lambda.lambda$createInstanceKeepAliveTask$5(Lambda.java:282)at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)at java.base/java.lang.Thread.run(Thread.java:833)aws.greengrass.LambdaManager 2.2.2 using Python Python 3.7With the following hello world lambdaimport jsonimport awsiot.greengrasscoreipcimport awsiot.greengrasscoreipc.client as clientfrom awsiot.greengrasscoreipc.model import (    QOS,    PublishToIoTCoreRequest)TIMEOUT = 10ipc_client = awsiot.greengrasscoreipc.connect()def greengrass_hello_world_run():    topic = "my/topic"    message = "Hello, World"    qos = QOS.AT_LEAST_ONCErequest = PublishToIoTCoreRequest()request.topic_name = topicrequest.payload = bytes(message, "utf-8")request.qos = qosoperation = ipc_client.new_publish_to_iot_core()operation.activate(request)future = operation.get_response()future.result(TIMEOUT)Timer(5, greengrass_hello_world_run).start()greengrass_hello_world_run()def lambda_handler(event, context):    returnFollowComment"

Answer: "1Hi,The common cause for the status timeout is that your memory limit is set too low on the lambda, and so Linux kills the container, thus the status doesn't update. You can address this by changing the memory limit when creating your component version or by updating the configuration in a deployment. If you create a new Greengrass component version to update the memory limit you must use "reset": [""] in the configurationUpdate for the deployment in order for the new default values to be used. https://docs.aws.amazon.com/greengrass/v2/developerguide/update-component-configurations.html#reset-configuration-updateIf the above does not solve your issue, please provide the version of Lambda manager which is being used, and if you are not using the latest version, then do use the latest version. Please also provide the lambda language (Python, NodeJS, or Java).CommentShareEXPERTMichaelDombrowski-AWSanswered a year agoAWS-User-3908852 a year agoI have updated the question. Sorry about the formatting.ShareMichaelDombrowski-AWS EXPERTa year agoPlease try my suggested solution, and then let me know how it goes.ShareAWS-User-3908852 a year agoTurns out it works, but gives errors in cloudwatch.Share"


Question: How does an EC2 instance assume an IAM Role?

"I am working through the Security Learning Plan and was recently watching the video about AWS Secrets Manager.The real question that concerns me is why using AWS Secrets Manager is better than say, store encrypted credentials in a config file. I know that there are a couple of aspects where Secrets Manager is obviously a lot better than a config file (credential rotation, central point of maintenance of credentials) but these don't answer my question.To clarify, let me give you an example. Let's say I have an EC2 instance running Tomcat and Tomcat needs credentials to connect to a DB. I store the creds in Secrets Manager and use the Java API to let Tomcat retrieve them. I guess (I didn't try it out yet) I need to grant the required permissions to the EC2 instance by assigning it an IAM role with appropriate permissions (maybe this assumption is incorrect, if so, how would I do it instead?). And now come my questions:How does Secrets Manager know the credential request is legimitate?How will the HTTP Query Tomcat sends authenticate against Secrets Manager?Will Secrets Manager see that a Java process is querying or will it only see a request coming from instance i-something?If the answer of 3 is "Secrets Manager sees only the instance querying" then how can I prevent another process on the same box querying the DB creds? If I chose the solution with the config file one of the major security drawbacks is that any intruder on the box can read the files, reverse the code the encrypts the creds and decrypt them. I would like to know if Secrets Manager provides a good solution for this fundamental problem.I didn't find any posts discussing this in the required detail (e.g. https://repost.aws/questions/QUAsOpdhR-QAKVZEL0nRGTkw/aws-secrets-manager-with-boto-3-in-python).I hope I explained the problem well enough. Thanks for every answer.FollowComment"

Answer: "1Accepted AnswerHere is a great explanation of how an EC2 instance assumes the role using instance profiles: Using an IAM role to grant permissions to applications running on Amazon EC2 instances.You can also read about the advantages of using IAM roles instead of distributing the credentials manually.And now back to your questions.You are correct: you need to grant the required permissions to the EC2 instance by assigning it an IAM role with appropriate permissions (for accessing the Secrets Manager service and obtaining the stored secrets).Checking if a request is legitimate is not something that will be done specifically by the Secrets Manager service. This is a core functionality of AWS - every API call will be checked for validity. Every API request must be signed with credentials to access the AWS services. AWS will check if the credentials are valid and if they grant permissions to call that specific API.When you use the Java SDK and make a call to the Secrets Manager from an instance with a role assigned, the SDK will first call the AWS Security Token Service and obtain the credentials for you. This call will happen under the hood if you attach a role to your EC2 instance. The AWS STS will issue temporary credentials that will expire after some time. Those credentials will sign the subsequential request sent to the Secrets Manager, so this is how the call will be authenticated. For more information, you can read this blog post explaining what exactly happens on assuming a role.The Secrets Manager service will observe an API call coming from the EC2 service. There is no way to restrict the access to a single process only. All processes running on the EC2 instance with attached role will have all permissions that role grants.By using resource-based permission policies, you control which IAM roles and services have access to your secrets in Secrets Manager. However, this won't prevent the intruder from accessing the secrets if an instance with role assigned will be compromised. In that case, follow the security incident response best practices and isolate the instance. Since you haven't stored any long-term credentials on that instance, the compromised temporary credentials will soon expire, so the intruder will not be able to access the services anymore. You can also configure an automatic secrets rotation in Secrets Manager, so your database passwords will automatically change too (without needing to distribute them to all your instances).CommentShareDmitry Balabanovanswered a year ago1The API call to Secrets Manager will be signed by the credentials (Access Key Id/Secret Key) that link back to the role assigned to the instance.Not sure specifically about Tomcat, but the API signing is built into the AWS SDKs. And the SDKs know to check the instance metadata service to retrieve the credentials.It will just see the request signed by credentials asserting the role that was assumed by the instance and then have the policies that allow it to retrieve the secret.Yes, any process on that instance can call APIs with the credentials of the role assigned to the instance.No, I don't think there is a solution to the problem you are describing, at some point your app needs the plaintext credentials. However, using Secrets Manager and EC2 Instance Profiles keeps your secrets and encryption keys out of config files and therefore out of any code repo. Focus on making sure the policy assigned to the role follows least privilege and restricting access to instance via network controls and login permissions.CommentShareEXPERTkentradanswered a year ago"


Question: Pinpoint Journeys - Where is the "EXIT" condition?

"Hi,I've been testing out Pinpoint Journeys and getting ready to use them.However, I'm having trouble understanding how to make sure a user (an Endpoint) exits the journey when it's no longer relevant to them?For example, if the Entry Condition is "Segment" (when a new Endpoint is part of a Segment, add them to the Journey)...If this user (Endpoint) is no longer part of that Segment, do they automatically exit the Journey? Because, if so, I haven't seen it anywhere in the docs.Please help. I don't want users to enter a Journey and continue receiving its emails when it's no longer relevant to them.Thanks so much,FollowCommentFab a year agoDid you find an answer? the documentation is so incompleteShare"

Answer: "0It looks like that Journey branch (in a Yes/No split for example) that has no activity and leads nowhere... IS an Exit of the journey.So, a solution would be to check for an event (purchase for example). If the event:paying_user is present, do nothing, if not then continue the journey and send the next marketing emails.A little burdensome, as you have to check before every next Communication Action, but this is what I found so far.CommentSharePSTwentyTwentyanswered 3 years ago0Correction: Any Journey branch that leads to no further actions is a Journey Exit.CommentSharePSTwentyTwentyanswered 3 years ago"


Question: How to retrieve attributes of the thing associated with an IoT Core for LoRaWAN device

"An AWS IoT Core for LoRaWAN device's message payload and associated wireless data is received at the rules engine with the DevEUI and WirelessDeviceID easily accessible.  When a LoRaWAN device is created, it can be associated with a thing, which is given an automatically generated name.  A usable name can be given to the thing as an attribute.We would like to retrieve the associated thing and attributes based on the WirelessDeviceID as part of the lambda function that parses the binary payload and generate the topic name based on this information.  I have not been able to find a way to retrieve this information from a lambda function.  If this is not possible from lambda, is there a query that can be used as part of IoT Analytics to get at this information?We are trying to avoid having to program each individual LoRaWAN device with its name and other identifying information using either downlink messages.  Though this would allow the device to include the information in its payload, we are trying to minimize the size of each message transmitted.  We have considered using the Shadow Service as a means to store the device's name, building location, etc., but we're not sure this is the right way to go.Any direction on how best to show the meaningful name on historic data plots and in alert conditions detected by the rules engine would be appreciated.FollowCommenttkobet 3 days agoThe idea with Fleet Hub is to, as you say, use it for alarms and general observability of device data.  For demo purposes, we have been showing alarms by setting up a query for SNS in a rule action, but if the customer eventually wants to modify or add alarms, the thinking is that Fleet Hub would provide a way for them to manage this without having to bring up the rule and edit the query for SNS.  There are two types of devices in the system, and an alert condition on one of the device types depends on the state of another, so the query might get complicated.The devices will be reporting sensor data once per hour, and we're looking at using either DynamoDB or Timestream to store the data for reports and graphs that go back up to some number years, so for that part of the system we have to get at the thing attributes using the wireless ID as the link to the Thing.Share"

Answer: "0Accepted AnswerHi. You can use the WirelessDeviceId that comes in the Lamba event, and call the GetWirelessDevice API. Python/boto3 snippet:    iotwireless = boto3.client('iotwireless')        response = iotwireless.get_wireless_device(        Identifier=event['WirelessDeviceId'],        IdentifierType='WirelessDeviceId'    )    thing_name = response['ThingName']You need to be careful about the API TPS limits though: https://docs.aws.amazon.com/general/latest/gr/iot-core.html#wireless-limitsCommentShareEXPERTGreg_Banswered 17 days agotkobet 16 days agoThis will get us the device name, but after getting into it there are a few issues.  The lambda function will have to be given permission to use getWirelessDevice (we're using javascript), which is ok, but this function returns a large amount of information that would not be needed.  Also, I can see us eventually running into the TPS limit.  If we did go this route, once the thing name is acquired from the data returned, we would then still need additional steps to get at the thing attributes (if that is possible from lambda).If the WirelessDeviceID is included in the topic data output by the lambda function, could this be used down the chain from the lambda function to get at the thing name and attributes?  If the goal is to get some of the thing attributes returned in a database query result along with the sensor data from the LoRaWAN device to fully identify the source of the sensor readings, how best to either retrieve the attributes and write them to the table along with the data, or retrieve them as part of the database query?I can paste the previous paragraph into a new question post if that would be more appropriate.ShareGreg_B EXPERT15 days agoThe TPS limit is adjustable, so you can request a quota increase. Alternatively, you might, for example, have a DynamoDB table to look up the thing name for the Wireless Device ID. Once you have the thing name, you can use DescribeThing to get the attributes. It has a soft limit of 350 TPS in most regions. Will you use fleet indexing? If not, the data you wish to store in Thing attributes, might perhaps also go well in a DynamoDB table.ShareGreg_B EXPERT15 days agoPrevious question on this point: https://repost.aws/questions/QUHQC_cN58QJeRPcKAbKR_YwShareGreg_B EXPERT15 days agoAnd please be aware there is an SQL function to help with accessing DynamoDB tables in IoT Core rules: https://docs.aws.amazon.com/iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodbSharetkobet 12 days agoWe are planning to use fleet indexing as it's my understanding this is required in order to use Fleet Hub.  This being the case, we have a choice between using AWS.Iot.getStatitistics() to query the fleet index based on WirelessDeviceId, or populating a DynamoDB table as each new device is added and using get_dynamodb() to query the table for the attributes we would need.  Is that correct?ShareShow 1 more comment"


Question: Amazon Elastic Kubernetes Service (EKS) maximum pod calculation with secondary VPC CIDR

"Amazon EKS includes scripts in the bootstrap.sh file to pass in the maximum number of pods per instance type based on the number of Interfaces and IPs per interface, and this works well for single CIDR VPCs.However, it seems that when using a secondary CIDR range, the instance ends up with 1 less ENI available per node, and this can impact pod scheduling.Does anyone have a solution to this?References:https://docs.aws.amazon.com/eks/latest/userguide/cni-custom-network.htmlhttps://github.com/awslabs/amazon-eks-ami/blob/master/files/bootstrap.shhttps://github.com/awslabs/amazon-eks-ami/issues/375FollowComment"

Answer: 0Accepted AnswerHere is a solution that can be implemented placed into the worker node userdata to dynamically calculate the number of pods.# get instance details to feed into bootstrap script`az=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)instance_type=$(curl -s http://169.254.169.254/latest/meta-data/instance-type)    # set region for aws cliaws configure set region `curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region`    # calculate number of max-pods# https://docs.aws.amazon.com/eks/latest/userguide/cni-custom-network.htmlmax_if=$(aws ec2 describe-instance-types  --instance-types $${instance_type} --query  InstanceTypes[*].NetworkInfo.MaximumNetworkInterfaces --output text)max_ips_per_if=$(aws ec2 describe-instance-types  --instance-types $${instance_type} --query  InstanceTypes[*].NetworkInfo.Ipv4AddressesPerInterface --output text)max_pods=$((($${max_if} -1) * ($${max_ips_per_if} -1 ) + 2))    /etc/eks/bootstrap.sh  --use-max-pods false --kubelet-extra-args "--node-labels=k8s.amazonaws.com/eniConfig=$${az} --max-pods=$${max_pods}" '${cluster_name}CommentShareShawn_Oanswered 3 years ago


Question: AWS Opensearch Service (1.1) with email alerting (AWS SNS)

"Hi everyone, I was trying to make an alerting system that every time a specific value is stored in one OpenSearch index, that triggers an email alarm configured with AWS Simple Notification Service. The problem is that, when I go to create the destination, in the “Alerting” configuration section, it’s supposed to show me the option to integrate it with SNS but it doesn’t, it only shows me 3 options: Slack, custom webhook, and Amazon Chime (as mentioned in their doc: “Compared to OpenSearch, alerting in Amazon OpenSearch Service has some notable differences… OpenSearch Service supports Amazon Simple Notification Service (Amazon SNS) for notifications.”)Does anyone know what should I do to integrate SNS with the AWS OpenSearch Service? Because also, supposing that it is not possible for any reason, if I wanted to do the same thing, using a Lambda function, it’s nonsense because as OpenSearch is a service hosted in AWS, why should I go to the Internet, to consume an API (also hosted in AWS) with all that implies (time, security, etc.), just to send an alert mail message?Thanks for your time.FollowComment"

Answer: "1I have retraced your steps from doc. When I try to Add destination in the "Alerting" configuration, the dropdown does show 4 options, including Amazon SNS. I can't be sure why it doesn't show up for you; I would suggest raising a support ticket to get this resolved.CommentShareSarika_Sanswered a year agoArian Calabrese a year agoThanks for your reply. I was going to create the ticket and i found that i need a subscription to get technical support from AWS. Do you know if this is the only way to ask for a solution? Because as I see it, this is more an AWS problem than something I'm doing wrong.Share1I have the same issue. My solution is: I downgraded! Use v1.0 - it has SNS email support.CommentShareonursavasanswered a year agoArian Calabrese a year agoWell, that worked! So thank you for that. My question now is, because I can't find any clarified response about. Is OpenSearch v1.0 safe from the log4j vulnerability? I was searching if is it safe to downgrade and I found myself in the position of giving up the SNS integration to maintain the security of the nodes.Share0There are specific directions here: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/alerting.htmlCommentShareRodney Lesteranswered a year agoArian Calabrese a year agoSorry but that's exactly the documentation I attached to the question. I tried to follow those steps, but SNS is not a valid option on the "Type" drop-down values.Share"


Question: Permission denied on S3 path

"(image removed)Hi,I got the above error when I run the query "select * from mco_bi_data_definition". The table is created with the given file path. What do I need to do to resolve the issue?Here are the policies in my S3 bucket:(image removed)Many thanks.*Edit: Removed Images - Ben G.FollowComment"

Answer: "0Did you see this article about athena permissions?  There are other articles you may want to read in this same grouping that might help you, too.  For example, there are specific athena permissions you are missing.https://docs.aws.amazon.com/athena/latest/ug/federated-query-iam-access.htmlCommentSharejschwar313answered 8 months ago"


Question: "Server process exited without calling ProcessEnding(), exitCode(-1073740940)"

"So there are a good amount of posts regarding this issue but I believe I have implemented the shutdown process according to that advice and I am still getting the error. Also, the only post with this particular code appears to be rather old and related to the Lumberyard Engine itself (We are using UE4).In my shutdown server process function, I am first calling TerminateGameSession, then checking that the outcome was successful, then calling ProcessEnding and again om a successful outcome then and only then do I call FGenericPlatformMisc::RequestExit(false); which tells Unreal to cleanly shutdown.I am getting no obvious errors in the game log, it looks like it is shutting down cleanly.I even inserted a 5-second sleep after both successful outcomes and still get the error.Any help would be appreciated. Thanks!FollowComment"

Answer: "0I do the same thing you just described, so I believe that what you're doing is fine, especially considering your game is shutting down cleanly. Here's the code just to make sure we're on the same page:auto TerminateGameSessionOutcome = Aws::GameLift::Server::TerminateGameSession();if (TerminateGameSessionOutcome.IsSuccess()) {auto ProcessEndingOutcome = Aws::GameLift::Server::ProcessEnding();if (ProcessEndingOutcome.IsSuccess()){FGenericPlatformMisc::RequestExit(false);}}This is a longshot, but what does your OnProcessTerminate Lambda function that you pass to Aws::GameLift::Server::ProcessParameters, which is passed to Aws::GameLift::Server::ProcessReady look like? Perhaps something in that function is causing that error message?CommentSharerePost-User-5100160answered 3 years ago0Hey @REDACTEDUSERYep we are on the same page, however, I was calling TerminateGameSession and ProcessEnding through the gameLiftSdkModule instance rather than statically like you are doing. I tried calling them statically though just to test and got the same error message.As for the OnProcessTerminate function, I haven't touched the bind from the standard code that aws advised when adding GameLift hereCommentSharerePost-User-4435701answered 3 years ago0Hmmmm, in that case, are you able to test your game locally and check if the exit code is still -1073740940? Because usually if it was a clean shutdown, then the exit code would be 0.CommentSharerePost-User-5100160answered 3 years ago0Unfortunately the connection logic is integrated with Flexmatch at this time, whose API is not exposed to gamelift local. So we have not be using gamelift local at all and might need to make some dev-only logic to work with it. Its a good idea though, might have to bite the bullet.I'm also using a custom child class of FGameLiftServerSDKModule so maybe that is causing an issue. Switching back to the standard FGameLiftServerSDKModule to test it now.CommentSharerePost-User-4435701answered 3 years ago0Ended up getting GameLift local working for at least the server (no client connections) and found that the server wasn't exiting cleanly. One of the plugins was trying to free memory that had already been freed. Now resolved and I don't get the error from GameLift anymore. Thanks @REDACTEDUSERCommentSharerePost-User-4435701answered 3 years ago"


Question: CloudWatch search expression

"Hi Everybody,I am doing CloudWatch search expression for graph metrics with a syntax:SEARCH(' {AWS/EC2,InstanceId} MetricName="CPUUtilization" ', 'Average', 300)But query will show all instances EC2. I want filter just instance name is: "ABC-asia-bastion-host". I did try with syntax bellow but maybe syntax false:SEARCH(' {AWS/EC2,InstanceId,Instance name} MetricName="CPUUtilization" Instance name ="ABC-asia-bastion-host" ', 'Average', 300)SEARCH(' {AWS/EC2,InstanceId} MetricName="CPUUtilization" Instance name="ABC-dev-asia-bastion-host" ', 'Average', 300)Evidence in files attach: https://ibb.co/1GyCWCGPlease help me filter the exact instance name as expected.Thank you so much.Best regard.FollowComment"

Answer: "1Hi there,It seems you are trying to search the EC2 metric based on Instance Name tag which is unfortunately not supported by Search expression. The supported characteristics you can search based on are "metric names, namespaces, dimension names, and dimension values" as per 1.On the contrary, you can use Metrics Explorer which supports Tag based searching - you can get more detail information from 2.When you are on the Metrics Explorer section of the CloudWatch console select CPUUtilization metric from EC2 namespace, and on the following forms you can specify which Tag you want to filter the metric on and in this case you are using Name tag.For your reference, I have attached above image as a showcase. As shown on the image, you can filter by the "Name: ABC-asia-bastion-host" for the Tag instead of "Name: test-js-server" which I used for example.I hope you find above information helpful.Thanks,MunkhbatCommentShareSUPPORT ENGINEERMunkhbat_Tanswered a year agoAWS-User-7754781 a year agoHi Munkhbat,Thank you so much. So easy.Best regard.Share0Hi Munkhbat,By the way, If I use Metrics Explorer when attaching to my dashboard maybe not be good? It looks worse than I want. To use CloudWatch search expression for widgets, how to search to filter to InstancesID or InstanceNameThank you so much.Best regard.CommentShareAWS-User-7754781answered a year agoMunkhbat_T SUPPORT ENGINEER10 months agoHi there,As mentioned previously, Search Expression only supports searching Namespace, Metric Name, Dimension Names and Values. Unfortunately, InstanceName is not a actual dimension from EC2 service which is shown on console(cosmetic feature that shows the Instance Name tag on the EC2 metrics for convenience). You can use the example shown here https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-examples.html for your guidance.More importantly, please help the community by accepting answers if it resolved your issues.Thanks,MunkhbatShareAWS-User-7754781 10 months agoHi Munkhbat_T,Thank you so much.Share"


Question: Can the Amazon Lex bot quota for 'all utterances character limit' be increased?

"Hello, I was training a chatbot on Lex and found that the current character limit (for all utterances) is 200000, but I need 500000 for that certain bot. It is not a service quota increase so I am unable to raise a support ticket under the same. Is it not possible to have an increased limit? Thank you in advance.FollowComment"

Answer: "2Accepted AnswerIf you look at the ServiceQuotas console, the limit for Total characters in sample utterances per bot locale (V2) is listed as not adjustable. In other words, it is a hard limit.CommentShareTaka_Manswered 7 months ago"


Question: Free Tier RDS has been charged

"Hi,Over the past three months, I have been using an AZSingle Micro T3 RDS Postgres instance. Today, I discovered a charge on my bill for 643 hours @0.26$ per hour and 17GB for storage. It's important to note that I had not noticed this charge until yesterday, and I am aware that my account qualifies for the free tier. Interestingly, I have also been utilizing a micro T2 EC2 instance for the same duration of 643 hours, but I haven't been charged for it.FollowComment"

Answer: "3UpdateBetween May 27 and May 29, we experienced a pricing issue where the Amazon RDS free-tier resources (for RDS for SQL Server, MySQL, MariaDB, and PostgreSQL) were being metered even when the usage did not exceed the free-tier limit. The issue has been resolved and your end-of-the-month bill should correctly reflect your free-tier usage.CommentShareEXPERTLeeroy Hannigananswered 7 days ago1I have also been charged and as far I can tell, I shouldn't have because I was at 650 hrs and the free tier is up to 755 hours and I was careful not to choose any paid options when setting up the instance.I contacted support through chat after opening a ticket and got this response:"I can see that you have not crossed the free tier but you have been charged again. It's not your fault here so I can look for a billing adjustment."CommentSharerePost-User-9128055answered 7 days ago0Hi,Amazon RDS free tier includes 750 hours of Amazon RDS Single-AZ db.t2.micro, db.t3.micro, or db.t4g.micro instance, and 20 GB of General Purpose (SSD) DB Storage, running a MySQL, MariaDB or PostgreSQL database per month, across all AWS Regions except GovCloud (US) and Asia Pacific (Osaka) Local, but just during 12 months starting with the date on which you create your AWS account.If you meet this last requirement, I would recommend opening a support case via the console.Updated: There are more users reporting the same issue since yesterday both here and on Reddit, so it seems to be a general issue.CommentShareMikel Del Tioanswered 8 days agorePost-User-0905816 8 days agoHi Mikel,I created my account three months ago and it is currently in the free tier, as confirmed in the AWS console.ShareMikel Del Tio 8 days agoHi,Yes, it seems correct, and since another user complains about a similar problem, I recommend you to open a support case.Sharemomermushtaq 8 days agoIts right. Support ticket seems to be the only solution at this stage.ShareMarcio-333fff 7 days agoThe same thing happened to me!I created my db.t3.micro instance on 05/01/2023 (i have only one instance).I followed all the free tier recommendations, I didn't exceed 750 hours or 20GB of Storage, but the charges (USD 13.67 for RDS) appear on the Bills panel.Amazon Relational Database Service for PostgreSQL:USD 0.018 per db.t3.micro Single-AZ instance hour (or partial hour) running PostgreSQL  ——— 636.908 Hrs —— USD 11.46Amazon Relational Database Service Provisioned Storage:$0.115 per GB-month of provisioned gp2 storage running PostgreSQL    ———-17.148 GB-Mo —— USD 1.97ShareMikel Del Tio 7 days agoThere are more users reporting the same issue since yesterday both here and on Reddit, so it seems to be a general issue.Share0Same issue with db.t4g.micro.CommentSharerePost-User-2201286answered 7 days ago0Issue resolved. My bill got revised.CommentSharerePost-User-0905816answered 6 days ago-1CommentSharerePost-User-0905816answered 8 days ago-2The support team is actively addressing this matter, and I will provide an update as soon as they have resolved the issue.CommentSharerePost-User-0905816answered 7 days ago"


Question: From S3 bucket to SFTP server

"Hi,   Requirement is to send a file from S3 bucket to an SFTP server (which has already been configured in our AWS Transfer Family). In documentation, I read that AWS transfer family is "a secure transfer service that enables you to transfer files into and out of AWS storage services."     But, how to do that ? Is it possible to configure AWS transfer family to receive a file from S3 bucket ? Or, must I configure S3 bucket to enable sending a file to an SFTP server ( https://docs.aws.amazon.com/transfer/latest/userguide/transfer-file.html#post-processing-S3-object-metadata ) ?     Any help or working example is highly appreciated. I lost several days trying to use "Paramiko" library in a lambda function or even a Glue job and I haven't succeeded to install that library. Before using ( https://stackoverflow.com/questions/47905614/upload-s3-bucket-csv-file-to-sftp-server-in-nodejs ) to do that, I thought I may ask experts over here who may know how do it in a right and efficient way.Thanks beforehand,AnthonyFollowComment"

Answer: "2SFTP server's storage is the S3 bucket. Ie. "to send a file from S3 bucket to an SFTP server" you need to do nothing but to upload the object into the same bucket as you configured for the SFTP user. If you would want to copy automatically objects from another bucket to one configured for SFTP, then you want to look into triggering a lambda function when object is uploaded (or modified) in your source bucket.https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.htmlPlease note that your SFTP users root directory is by default s3://bucket-name/user-name/For more complex directory mappings, logical directories can be helpfulhttps://aws.amazon.com/blogs/storage/simplify-your-aws-sftp-structure-with-chroot-and-logical-directories/https://aws.amazon.com/blogs/storage/using-aws-sftp-logical-directories-to-build-a-simple-data-distribution-service/CommentShareKalluanswered 10 months ago1Hello,In regards to copying S3 bucket data to SFTP server, these are few options to do this -Use S3 Event notifications. S3 event notification is a mechanism to trigger a notification to different AWS destinations when a certain event happens in a s3 bucket. For example, you can invoke a Lambda function when there is an upload of new object to s3. With this mechanism, you can trigger a Lambda function upon upload of new object in s3 bucket, this Lambda function will have to take care of authenticating On-premise SFTP server and copy files to this server. This function can assume s3 permissions from IAM role associated to it.  You can refer below document to know more about s3 event notification.https://docs.aws.amazon.com/AmazonS3/latest/userguide/NotificationHowTo.htmlNote - In order to upload and Download from s3, you need to have s3:PutObject and s3:GetObject permissions. But, you may need to add more permission depending upon the actions you are doing while trying to access s3. For example, if you are trying to do a GetObject on previous version, you need to use s3:GetObjectVersion. For s3 actions, you can refer below document -https://docs.aws.amazon.com/AmazonS3/latest/userguide/list_amazons3.html#amazons3-actions-as-permissionsUsing shell script: You can setup an automation on an EC2 Instance (or an On-premise server) and copy files from s3 bucket to SFTP server using regular shell commands. The important thing to consider is to authenticate SFTP server. For s3 authentication, you can use IAM instance profile to assume the credentials.Refer the below document and see if it helps to authenticate SFTP using a shell script. Please note that this is a third party document and AWS doesn't own it,  I cannot guarantee any information in this document. I would recommend you to go through in detail before implementing this.https://www.golinuxcloud.com/automate-sftp-shell-script-with-password-unix/You could also consider executing shell script in the SFTP server itself. Typically, SFTP server files are stored on local disks and can be accessed directly from the OS itself. If you can automate a script, you can use aws cli s3 command to copy files directly to File system instead of authenticating SFTP server using shell script.Using AWS Transfer SFTP server that exposes objects in s3 bucket using SFTP protocol. You can authenticate using ssh key pair using supported SFTP client and access files from s3 bucket.https://aws.amazon.com/aws-transfer-family/I want to explain a little about the Transfer Family service. The AWS Transfer Family provides fully managed support for file transfers directly into and out of Amazon S3 or Amazon EFS. When you create a SFTP enabled Transfer Family server and create a user that connects to a S3 bucket, then once you connect to this Transfer server via the user from your machine, you can then view all of the current files already in your S3 bucket.The transfer family server views what is in the S3 bucket and what you could do is download/transfer the files from the S3 bucket into your local machine or you can upload/transfer files from your local machine to your S3 bucket.For more information on getting started with Transfer Family service, such as creating servers and users, please refer here -https://docs.aws.amazon.com/transfer/latest/userguide/getting-started.htmlhttps://docs.aws.amazon.com/transfer/latest/userguide/how-aws-transfer-works.htmlCommentShareSUPPORT ENGINEERYash_Canswered 10 months ago"


Question: Can't update Key/Pair assigned at launch

We have an EC2 instance where one of the previous users updated the key/pair for SSH. We have tried most of the articles we found with no change in the key/pair.I have updated the authorized_keys file with one of our pre-existing public keys but am not seeing any update/change on the instance details page or success connecting using SSH.Is there something step or pre/post activity that I'm missing? Right now the only way we have to access that instance is via Session ManagerFollowComment

Answer: "1Editing the public key in authorized_keys does not change the key pair name listed on the instance details page.However, if you have updated authorized_keys with the correct public key, you should be able to ssh setzu with the private key corresponding to the public key.You can also try the following method to change the public key.https://repost.aws/knowledge-center/user-data-replace-key-pair-ec2CommentShareEXPERTRiku_Kobayashianswered 2 months ago0The instance details page shows the key pair that was assigned at launch. The field name is "Key pair assigned at launch". If you change the key pair, or add key pairs to the instance, the value of "Key pair assigned at launch" does not change.CommentShareMarilynAWSanswered 2 months ago"


Question: Does Linux instance have the connection limit?

"I have a Debian Linux 10 instance running on t3a.xlarge EC2, it's an application server. Every time when the request connects reached to about 150-200, the server response is extremely slow.  We don’t have such a problem when the “same Linux” was running as a physical machine that was a lot less powerful than the t3a.xlarge instance.  Does the Linux instance have connection limit? if yes, how to increase the connection limit?Also the connection is not the  heavy workload, I don't think it's the AWS bandwidth issue either.  thxFollowComment"

Answer: "1Short answer: No, there aren't any limits imposed on connections other than what the operating system does.However. T-series instance work on a CPU credit system. When they aren't busy they build up credits. When they are, they draw down those credits and if the credits run out the performance level returns to a baseline which may not be enough to service your requests in a timely manner.You can monitor your CPU credits which would be a good thing to look at in this case.If you are running out of CPU credits three suggestions:Use a larger T-sized instance. This will cost more but it also comes with a higher baseline performance. Not a great suggestion if your performance issue happens with low frequency as you're paying for capacity you don't need.Choose Unlimited mode for your instance. If it runs out of CPU credits we will give you additional CPU credits - for a charge. This is a good option if you only see this issue occasionally but if it is happening more often then the charges may be higher.Choose a non-T series instance that doesn't rely on CPU credits. This has a higher cost but now you don't have to worry about performance changing under load.This is all assuming that CPU is the problem in this case. It may not be. You should be looking at memory usage, disk I/O and other system metrics to determine why performance suffers when you have many requests.CommentShareEXPERTBrettski-AWSanswered 10 months agoAWS-User-8431016 10 months agoThanks for your answer.  I don't think it's the CPU credit issue, I checked in Cloudwatch showing the "CPUCreditBalance" is always 2304, the "CPUCreditUsage" was always 0.x. Far away to reach the credit limit, Any other areas should we check?Our source server has Intel CPU, t3a has AMD CPU, Does this make a difference?  Can we change the instance type from t3a to t3?  thanks!ShareBrettski-AWS EXPERT10 months agoChanging from t3a to t3 is unlikely to make a difference. In this case I'd recommend monitoring the processes on the instance to determine what is causing the slowdown as it accepts more connections. Is it running out of memory and swapping? Is it running out of CPU capacity? Is it overloading a database which is returning data slowly? There are many causes for servers being slow.Share"


Question: Two EC2 Instances stuck "stopping"

"Hi, two of my ec2 instances in ca-central-1a hung at approx 15:20 (UK time)I've tried to stop them, but they've been stuck in the "stopping" state for over 10 minutes now.Can anyone help?i-02d3a7e67a35ad2d6i-04cef95ec8cffc2f6Many thanksFollowComment"

Answer: 0both stopped now - not sure if that was AWS action or just time passing... anyway.  resolved.CommentSharefatPabloanswered 3 years ago


Question: Lambda (C# NetCore) timing out when trying to retrieve ThingShadow

"I have a Lambda function (Runtime: .NET Core 3.1 (C#/PowerShell)) which tries to retrieve a Shadow document using the GetThingShadow API call. This Lambda was working until yesterday and suddenly stopped working today with a timeout error. The Lambda is configured with a Role and a policy that grants the Lambda to access the ThingShadow. This whole setup was working until yesterday. Any clue on what might cause this issue suddenly?FollowCommentMaulik_Makadiya a year agoAt first step, Worth to check Cloudtrail/Audit log to see is something was changed by someone else or not.Share"

Answer: "1Hi,May I confirm if the issue is permanent or is it intermittent? Are you only seeing a timeout error or are there other errors as well? Can you provide a short snippet of the error in the Cloudwatch log here? Please make sure it does not contain any sensitive information.A timeout error implies that Lambda is unable to reach the IoT Endpoint. Is it possible that the Lambda function has been modified recently and VPC configuration has been added? By default, a Lambda function loses internet connectivity when it is connected to a VPC and requires some additional configuration in order to have internet access. You can read more about this here in our knowledge center article https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/On the other hand, if your Lambda function is already VPC enabled, certain changes to the VPC configuration may lead to the Lambda function losing internet access. Therefore, even if there were no changes to the Lambda function itself, it could still end up losing internet access. For example, the route to the NAT gateway/instance may have been modified. This is one way I can think of that a Lambda might suddenly stop working where the Lambda's API calls are leading to timeoutsIf you provide me with the region of your Lambda and Thing then I can double check if there may be some sort of ongoing issue. I'm not aware of any issue at the moment and I do not think this is the reason but I will double check once I know the region.CommentShareSUPPORT ENGINEERRyan_Aanswered a year agoAWS-User-2664896 a year agoHi, The Lambda was redeployed as part of our regular maintenance yesterday. VPC was already enabled on this Lambda to communicate with our internal resources over a VPN tunnel (which is also having issues currently). The Lambda is running in us-east-2 region and the thing is 32465. We confirmed that by removing the VPC settings the API call is successful. There wasn't any change to the VPC or its related settings recently. Also the Cloud watch log it just says timed out. The issue is also permanentShareRyan_A SUPPORT ENGINEERa year agoHi,Thanks for your response and for performing that test. I was just about to suggest removing the VPC settings to see if the API call will be successful. However, this actually confirms that there is an issue with the VPC settings.I know you have mentioned that there were no changes to the VPC settings but I would recommend double checking this and making sure that the VPC settings are correct. You can refer to the following resources:https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html#vpc-internetAs a possible alternative solution, AWS IoT core supports VPC endpoints(https://aws.amazon.com/about-aws/whats-new/2021/07/aws-iot-core-supports-vpc-endpoints/) so you can use this instead of configuring your Lambda for internet access.Share"


Question: "SageMaker Model Registry, Model Monitor and Hyperparameter Tuning jobs - Pricing?"

"Hello,The SageMaker pricing page has no mention of SageMaker Model Registry, Model Monitor and Hyperparameter Tuning jobs - https://aws.amazon.com/sagemaker/pricing/and I can't add them using the AWS calculator.Can someone help me determine the pricing and why it's not in the official calculator?Thank you!FollowComment"

Answer: "1Accepted AnswerThere's no price for using SageMaker Model Registry - you only pay once you start hosting the models.For hyperparameter tuning jobs, you pay for the number of training jobs that the tuning job spins up, so you can plan accordingly in your pricing calculator (for example if you have 12 combinations of hyperparameters, then plan for 12 training jobs in the estimator).Similarly, model monitoring runs processing jobs on schedules (see docs).CommentShareDurga_Sanswered 8 months ago"


Question: Installing CUDA on p2.xlarge (Deep Learning AMI GPU PyTorch 2.0.0 (Ubuntu 20.04))

"I am trying to install CUDA to run my python script on a GPU. However, even though I am following the guidelines on the nvidia webpage, the driver is never running after installation (nvidia-smi says it "failed to communicate with the driver", even though nvcc --version shows the installed driver).p2.xlarge has a GPU, but there no driver installed.Is there a step by step guide?FollowComment"


Question: Reset All AWS Infrastructure to Defaults

"Good afternoon.I am using my AWS account to learn AWS.  During this process things have been created and destroyed.  How can I get everything reset to where it was before I started, like it was a brand-new AWS account? I am able to find information on password resets, but nothing on resetting the AWS account infrastructure to defaults (factory reset all AWS infrastructure in an account).Thanks in advance.FollowComment"

Answer: "2There is no built-in way to accomplish this task. When using AWS services, you might change your account configuration and/or create various resources like IAM roles or CloudWatch log groups. Additionally, you might change global or regional service configurations (e.g. for AWS Config or Amazon API Gateway). Further more, while using the services, your account's internal metrics will change. So basically, there is no way to do a 'factory reset' of an AWS account.Even when you deploy your resources using the IaC approach (AWS CloudFormation or AWS CDK) and then delete the stacks, you still might have some data or configuration that differs from defaults (e.g. CloudWatch metrics - you cannot delete them, they expire automatically).The only way to fully reset everything is to delete your existing account and create a new one.CommentShareDmitry Balabanovanswered 6 months ago1Open-source tools may serve your purpose.amazon web services - Factory reset AWS account - Stack OverflowHowever, please use it at your own risk.Warning: these open source tools are not provided by AWS, you should take every precaution to ensure that nothing critical gets deleted, and you are responsible for your choices and actions.CommentSharejhashimotoanswered 6 months agoLorenzo 6 months agoThank you for your answer. However, this is exactly what I do NOT want to do. I do not want something that traverses the account and deletes things.  I have seen this answer before. It does not answer the question but only provides a hack.I want to know if there is a way to actually do a real reset, that removes everything then ensures the default objects are recreated.  Neither AWS-Nuke nor Cloud-Nuke do this.  They just destroy everything without recreating the default objects (VPC, subnets, etc.).So, back to my original question: is there a way to do a factory-type reset of an AWS account?ShareRoss Wickman 6 months agoThis is the only option and does almost exactly what you want.The only Default configuration you would need to worry about in the account after the nuke IS the launch of a "Default" VPC which I would never use for anything other than tinkering, and you can create a new Default VPC right from the VPC Console.Or if you are looking for a way to automate this, you can just add the CLI to create it at the end of your automaiton.Found here: https://docs.aws.amazon.com/cli/latest/reference/ec2/create-default-vpc.htmlShare"


Question: Unable to remove SNS subscription

"I can't remove an SNS subscriptionI already have tried the options described in this linkshttps://aws.amazon.com/premiumsupport/knowledge-center/sns-invalid-parameter-error-api-call/https://aws.amazon.com/premiumsupport/knowledge-center/sns-cannot-delete-topic-subscription/https://repost.aws/questions/QURO0Hgy8vTfe95rKT_cIT2Q/how-do-i-delete-unconfirmed-subscriptions-for-removed-topicBut my situation is different. Is a SNS Subscription with "Confirmed" status but without ARNaws sns list-subscriptions show the subscription with "Deleted" SubscriptionArn$ aws sns list-subscriptions{    "Subscriptions": [        {            "SubscriptionArn": "Deleted",            "Owner": "111111111111",            "Protocol": "email",            "Endpoint": "foo@bar.com",            "TopicArn": "arn:aws:sns:us-east-1:111111111111:foobar"        }    ]}Recreate the SNS Topic ant delete it again not delete the subscriptionMaybe the problem was created by aws-nuke, but I'm not sure.https://github.com/rebuy-de/aws-nukeThe error show in the console isCouldn't delete subscription.Error code: InvalidParameter - Error message: Invalid parameter: SubscriptionArn Reason: An ARN must have at least 6 elements, not 1I assume that is trying to pass "Deleted" as ARNSomething like:aws sns unsubscribe --subscription-arn DeletedAn error occurred (InvalidParameter) when calling the Unsubscribe operation: Invalid parameter: SubscriptionArn Reason: An ARN must have at least 6 elements, not 1Any ideas?FollowComment"


Question: What are Amazon Translate charges on batch job failure.

"When Amazon translate batch job fails mid way when processing documents, does the customer still get billed? If so, what is the charge based on?FollowComment"

Answer: "2Accepted AnswerRequests where the source language equals the target language (whether user designated or automatically identified), and when an error occurs and no translation is returned, are not charged for. Requests where the content is non-translatable (e.g., “&*^%((**&(^”) are charged for.CommentShareCristian_Canswered 2 years ago0If Amazon Translate batch translation job fails, that request is not charged for. Further information on pricing can be found here:https://aws.amazon.com/translate/faqs/https://aws.amazon.com/translate/pricing/CommentShareBasak_Yanswered 2 years ago"


Question: Creating ALIAS Record for Zone Apex in Route 53

"A customer is currently using Akamai CDN and are evaluating Route53 for their DNS needs. They only get CNAME records for their CDN from Akamai. They want the zone apex of their domain to also point to their CDN. Given that, in Route53, you can’t redirect queries from an ALIAS record at the zone apex to a CNAME record, what is the best way to accomplish this? I was thinking making the ALIAS record at the zone apex point to a CloudFront distribution (which Route53 allows) and then using Lambda@Edge triggered by Viewer Request events to redirect the requests to their Akamai CDN. Any thoughts on this?FollowComment"

Answer: "0Accepted AnswerAs apex records must be A records, and Route53 aliases effectively lookup their destination to return the IP addresses when they are queried, aliases have to point to other AWS resources (such as CloudFront CDNs). Your suggestion would be a good solution. It is also recommended to redirect HTTP requests for apex domains to their www records.My recommendation is to make sure the Lambda@Edge is configured for the origin-facing request event, and sets a high Cache-Control max-age header in the response. Also make sure that you can handle both HTTPS and HTTP requests.CommentShareEXPERTLeeAtk@AWSanswered 3 years ago"


Question: RDS t3.small multi-az capacity in us-east-1

"Hi,Trying to modify my t3.small instance in us-east-1a to be multi-az but after many retries I keep receiving an InsufficientDBInstanceCapacity error. I read here that purchasing a reserved DB instance should resolve this:https://forums.aws.amazon.com/thread.jspa?threadID=224370"4. Try purchasing Reserved Instances. Reserved Instances are a long-term capacity reservation. For more information, see: Amazon EC2 Reserved Instances.http://aws.amazon.com/ec2/purchasing-options/reserved-instances/"...however I am still getting the error. Did I just needlessly purchase a reserved instance or is there some way I am missing in the console/CLI to claim my reserved capacity?Edited by: davegravy on Dec 13, 2019 7:18 AMFollowComment"

Answer: "0Sometimes it isn't useful to apply EC2 documentation, and certainly not forum postings, to RDS. So the RI might not help with your immediate problem but still will help you save costs over time. (You could also contact customer service, explain why you purchased by mistake, and try to get a refund.)I suspect what is happening is that you are trying to put the Multi-AZ standby into us-east-1e, which for some reason (likely too full to add racks) doesn't have, and doesn't appear to have ever had, t3.micro instances. The question is, why would RDS try to put the standby node there when it won't allow you to select it manually. If I had to guess I'd say that you probably only have two DB Subnets defined, one for the AZ your primary lives in and one for us-east-1e. So RDS, or more specifically the console, tries to put your standby into 1e and EC2 returns the ICE error.Make sure you have subnets for at least one AZ other than where your primary lives, and other than 1e. If you don't have one for another AZ, create it and try modifying to Multi-AZ again. You could also try using the CLI and explictly specifying which AZ you want the standby in via SecondaryAvailabilityZone in the modify-db-instance command.HalPs: If I had to guess this is probably poor behavior in the Console rather than at the core API level in RDS. That is probably where they look at your existing subnets and try to determine the appropriate place to create the secondary. If I'm right, you can use the Feedback link in the Console to tell the development team how their trying to make your life easier actually made it harder.CommentShareHalTempanswered 3 years ago0HalTemp wrote:I suspect what is happening is that you are trying to put the Multi-AZ standby into us-east-1e, which for some reason (likely too full to add racks) doesn't have, and doesn't appear to have ever had, t3.micro instances. The question is, why would RDS try to put the standby node there when it won't allow you to select it manually. If I had to guess I'd say that you probably only have two DB Subnets defined, one for the AZ your primary lives in and one for us-east-1e. So RDS, or more specifically the console, tries to put your standby into 1e and EC2 returns the ICE error.This indeed seems to be what was happening. I've added another subnet that supports the t3 class, and we'll see after maintenance tonight how it goes but the console error at least disappeared.Thanks!CommentSharedavegravyanswered 3 years ago"


Question: AWS Elemental Live - Secure remote kickstart

Managing a fleet of Elemental Live encoders that feed MediaLive.  What's the best way to manage performing kickstarts on the on-prem devices if you aren't on site to use a thumb drive on the unit itself?  PXEBoot?  Some other solution?  Any thoughts are welcome!FollowComment

Answer: 0You can use the DELL iDRAC management port of the Appliances to remotely kickstart the appliancesCommentShareNuno_Qanswered 2 years ago


Question: How to export / import rows reliably from one MySQL RDS server to another

"I have looked, but haven't found a reliable way of exporting rows from a MySQL DB on one RDS instance and importing them on another RDS MySQL instance.  I have tried MySQL Workbench, and it mostly works, but fails when there are double quotes in a field.FollowComment"

Answer: "0Hello Ron,I understand that you are looking for a reliable way of exporting rows from a MySQL DB on one RDS instance and importing them on another RDS MySQL instance. Please note that there is currently no way to specify rows when performing exporting of importing of data. However, you can perform replication to export and import data from an RDS for MySQL DB instance to another RDS MySQL DB instanceThe replication process is as follows:Prepare the source MySQL DB instanceFor this process, please see reference [1]: Prepare the source MySQL DB instanceCopy the databaseSee reference [1]: Copy the DatabaseImport the data into the target RDS MySQL DB instanceThe method/technique for this action is to create a read replica for ongoing replication. Promote the read replica for one-time creation of a new DB instance. To create and configure a MySQL read replica, see reference [2].You can also consider using mysqldump which is a separate tool where you can specify databases to export and import, see reference [3] for MySQL documentation.References:[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Exporting.NonRDSRepl.html[2] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_MySQL.Replication.ReadReplicas.html[3] https://dev.mysql.com/doc/refman/8.0/en/mysqldump.htmlCommentShareAsenathianswered a year ago0Hi RonYou can replicate tables using AWS DMS. DMS can connect to a variety of source and target endpoints. On the task definition, you need to specify the schema and table that you want to replicate. There are 3 method to replicate; full, full + cdc and cdc only.You can find more information in the following linkshttps://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MySQL.htmlhttps://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.htmlBest RegardsAgustiCommentShareAgusti_Lumbrerasanswered a year ago"


Question: About changing the scope of the OAuth Scope for Client Credentials grant and the suggestion of implementation for Authorization Code Grant flow without using hosted UI

"Hello,We are now facing an issue about the scope of "aws.cognito.signin.user.admin" after using initiateAuth method for signing in to Cognito, as we found this scope has given a lot of API actions when the user gain the token. For the condition of using initialAuth() method from Cognito with USER_PASSWORD_AUTH and Client Credentials grant flow, any possible solution to change the scope?And if Authorization Code Grant flow is needed to be selected in order to switch to other possible scopes, is it possible for not using the hosted UI as the login form and retain the initialAuth() method with USER_PASSWORD_AUTH and applying the new grant flow? Thanks.Regards,Jimmy ShumFollowComment"


Question: S3-S3 Datasync Fails with error: The endpoint has already opened a volume for this session

"I'm trying to clone a number of buckets between two AWS accounts. Of the 6 buckets that are being copied, 2 consistently fail to sync with the error:The endpoint has already opened a volume for this sessionThe sync task starts and goes into a "Preparing" state, it then stops here for up to a few hours, then just errors.The two buckets that fail are the largest buckets from this batch (one is 10 TB the other is 1 TB), but object count looks like it shouldn't be exceeding the quota for a task (12.3 million and 7.6 million respectively).I've tried various combinations of only synchronising some directories and turning off only synchronising changes, neither has helped. I've also tried completely recreating the synchronisation locations and tasks and this also didn't make any impact.Looking at the logs, I see a normal startup sequence:[INFO] Request to start task-xxx.[INFO] Execution exec-xxx started.[INFO] Started logging in destination hostId: host-xxx for Execution exec-xxx[INFO] Started logging in destination hostId: host-xxx for Execution exec-xxxThen no further log messages are recorded.Has anyone had any similar experiences with s3-s3 sync? or any ideas what could be going wrong?FollowComment"


Question: When I try searching up my site i get 502 error cloudfront failed to establish a connection with the origin.

This is what I see when I look up my site.FollowComment

Answer: "0Hi. As the error mentions, CloudFront is having issues connecting to the origin. What is the origin (S3, ALB, HTTPS endpoint)? Does it require any authentication? Also, what happens if you bypass CloudFront and try accessing the content directly from the origin?CommentShareJeremyAWSanswered 4 months ago"


Question: How to find deadlock process in RDS My SQL and set alarm notification?

"My customers encounter the problem of SQL process deadlock when using RDS MySQL, I would like to know how to find these processes in RDS MySQL and set up alarm notification when triggered？and how to process the issue?FollowComment"

Answer: "2Accepted AnswerTo identify deadlocks on a database, please enable the innodb_print_all_deadlock and innodb_deadlock_detect parameter in the parameter group to log all deadlocks to errorlog.The INFORMATION_SCHEMA INNODB_TRX table can be queried. This gives you precise details about every running transaction in INNODB, including waits.Next, you can get error logs through Cloudwatch log, and filter keywords for alarm notification.CommentShareColin Tanganswered a year agorePost-User-0067846 a year agothanks Colin, your advice is very helpful for me to solve the issue. and the problem is solved.Share1thanks very much. it's helpful and the issue is solved by your advice.CommentSharerePost-User-0067846answered a year ago"


Question: Billed for OnDemandStreamHr in Kinesis despite no instances running

"I've been billed about $30/month for Kinesis "OnDemandStreamHr" at about 720hr/month (so running 24hrs) for the last few months. I was playing around with Kinesis in January when studying for the ML cert, which is when the billing started, so that tracks, but I was very careful to shut down all my instances. I still don't see any running kinesis instances despite the billing continuing to happen. Any insight would be much appreciated.FollowComment"

Answer: "1Hello there,Sorry to hear about this trouble! If you haven't had the chance, this guide offers steps to troubleshoot the resources generating charges:https://go.aws/resources-unexpected-charges.If you still find yourself with questions or concerns, our Billing team has the visibility needed to take a closer look. To begin an investigation, please open a secure correspondence by creating a case here:https://go.aws/support-center.Thank you,- John M.CommentShareEXPERTAWS Support - Johnanswered a year ago"


Question: Make Cloudfront accessible and serve content from S3 Bucket

Make Cloudfront accessible and serve content from S3 Bucket.Restrict direct access to S3 bucket.Website should only be accessible from cloudfront url.FollowComment

Answer: "1Start with this article and my recommendation is to use OAC as described.CommentShareEXPERTBrettski-AWSanswered 3 months ago0Hi,if above is a question, I suggest looking in this article: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html. It also outlines to use OAC (Origin Access Control) over OAI (Origin Access Identity).Hope it helps.CommentShareEXPERTalatechanswered 3 months ago"


Question: AWS IoT TwinMaker launches asset synchronization with AWS IoT SiteWise

"Starting today, AWS IoT TwinMaker will support asset synchronization with AWS IoT SiteWise, making it easier for AWS IoT SiteWise customers to bring their assets and asset models into AWS IoT TwinMaker.Now, customers can start using AWS IoT TwinMaker without having to recreate AWS IoT SiteWise assets and asset models, and any updates will be automatically synced. There is no charge for entities synced from AWS IoT SiteWise. If synced assets and asset models are no longer needed, customers can simply delete the sync setup.Customers can enable the feature in the AWS IoT TwinMaker console by providing a workspace to host assets and asset models from AWS IoT SiteWise. To learn more, visit our developer guide and API reference.This feature is available in the following AWS Regions: US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Europe (Frankfurt), and Europe (Ireland).FollowComment"


Question: Does AWS DMS support ARRAY data type for RDS for PostgreSQL on EC2 to Aurora PostgreSQL migration?

"I am currently migrating an Amazon RDS for PostgreSQL database from Amazon EC2 to Amazon Aurora PostgreSQL-Compatible Edition. I am using AWS DMS and have encountered the following issue:One of the columns in a particular table stores the values of water pressure measured within a second. This column is an array of decimal numbers (Ex: {2.44, 5.66, 8.55}). I received the following error message during the migration from AWS DMS:"1 unsupported data type '_float4' on table 'data1', column 'pressure'".Does AWS DMS support ARRAY data type for double or floating point numbers? The AWS documentation indicates that the arrays can't be migrated. However, further down on the same page, it's mentioned that AWS DMS supports arrays from a source RDS for PostgreSQL database and that arrays are mapped to CLOBs in AWS DMS. I'm looking for some guidance on whether ARRAY data type is supported by AWS DMS during migration. The reports return the following error:Note: You can see that the pressure column is indicated with real[] .1 unsupported data type '_float4' on table 'data1', column 'pressure'pipeminder=# \d data1Table "public.data1"Column | Type | Modifiers---------------+--------------------------+-----------device_id | bigint | not nulltimestamp | timestamp with time zone | not nullpressure | real[] | not nullpressure_min | real | not nullpressure_mean | real | not nullpressure_max | real | not nullflow | real | not nullIndexes:"data1_unique_device_time" UNIQUE CONSTRAINT, btree (device_id, "timestamp")FollowComment"

Answer: "0Accepted AnswerIf your RDS for PostgreSQL version on Amazon EC2 is PostgreSQL 10 or higher, then you can use the native logical replication to Aurora. In this case, you don't need to use AWS DMS. For more information, see Migrating PostgreSQL from on-premises or Amazon EC2 to Amazon RDS using logical replication. The native logical replication handles all native PostgreSQL data types, including arrays. If the source database is PostgreSQL 9.4, 9.5, or 9.6, then you can use the pglogical extension. Pglogical extension can handle arrays as well.CommentShareAWS-User-3171097answered 3 years ago"


Question: FBX Animation/Character Import Problems with Blender

"I've spent several hours trying to figure out the process of character/animated model importing from Blender to LY via FBX. After fumbling around with a more complicated character I decided to distill the process down to its very VERY basic form. I've come up with this .blend file that when exported with the settings indicated, will produce the .fbx file.https://agdpublic.s3.amazonaws.com/ly/test01.blendhttps://agdpublic.s3.amazonaws.com/ly/test01.fbxNotes from file:Root bone of Armature group must be pointing Y+and does not need to contribute to the animation.All transforms must be applied (easier if this isdone before skinning).FBX Export Settings:FBX 7.4 binaryAll default settings except:Z UP, Y FORWARDNo leaf bones on ArmatureFor Geppetto process see last part of this video:https://youtu.be/0nVhS-576owStill having issues with what appears to be anincorrectly scaled skeleton once imported intoLumberyardNotes as of 3-6-17As indicated, I'm still having issues with what appears to be scaling with the bones/skeleton. If your guru's could examine the files and determine where I went wrong, I will be one step closer to being able to put together a nice video on the process so everyone can learn.Please advise. Thank you.EDIT, found that a good setup for a Blender scene is to use Metric units with a .01 Unit Scale. See new test files, these should import flawlessly:https://agdpublic.s3.amazonaws.com/ly/test02.blendhttps://agdpublic.s3.amazonaws.com/ly/test02.fbxFollow"

Answer: "0Accepted AnswerCould you try again with a fresh scene and the units in blender set to metric, please? The scale on the animation of your file is indeed off when imported into lumberyard. When I export a scene using metric units everything looks ok, however changing the units setup on your scene didn't help, hence please try a fresh one with units set to metric right from the start.EDIT: Scratch that.Set the units to metric in blender and use a units scale (right underneath the units setting) of 0.01. Then rescale your mesh to the desired size. That works for me.4018-test01-modified.zip|attachment (76.4 KB)test01-modified.zipSharerePost-User-0887237answered 6 years ago0You need a separate file with just the not animated skeleton the file with the skinned model(i don't know if it also works without the skinned model.I have not noticed your files I will give a look at them while I watch the champion's league(Soccer).SharerePost-User-7621041answered 6 years ago0First of all try to make the skeleton file right. without animation.reset that armature dummy and if you can try to use a bone instead as I had problems with dummies and fbx.SharerePost-User-7621041answered 6 years ago0It appears as if it's scaled improperly in the other direction now. Almost as if the scaling of the bones is independent of the mesh, even though the scaling has been applied. I experienced the same issue when exporting an FBX from 3DS 2013.https://agdpublic.s3.amazonaws.com/ly/3ds-test-01.FBXhttps://agdpublic.s3.amazonaws.com/ly/ly-anim-metric-01.blendhttps://agdpublic.s3.amazonaws.com/ly/ly-anim-metric-01.fbxSharerePost-User-4796539answered 6 years ago0This is the correct answer! Your Blender scene needs Metric scaling with .01 unit scale.I also updated notes removing stipulation to remove leaf bones in export settings. Also updated OP to reflect correct answer. Thank you!!SharerePost-User-4796539answered 6 years ago0A little teaser. It was a long hard road, and he even lost a hand in the process, but he's a survivor!REMOVEDUPLOADSharerePost-User-4796539answered 6 years ago0Great work on resolving this. I've made a note to have this documented better (at all).SharerePost-User-8353690answered 6 years ago0It's just a flesh wound! Nice work.SharerePost-User-8353690answered 6 years ago0The links above should still work:https://agdpublic.s3.amazonaws.com/ly/test01.blendhttps://agdpublic.s3.amazonaws.com/ly/test01.fbxThese have all transforms applied, but the skeleton is scaled down by 100 times once you play the anim in geppetto.SharerePost-User-4796539answered 6 years ago0https://agdpublic.s3.amazonaws.com/ly/test01-ascii.fbxFYI Blender is a free ~85MB download.SharerePost-User-4796539answered 6 years ago0Hey @REDACTEDUSERDo you still have the original files that were not working before? We'd like to use them for a test case to help improve the FBX Importer.SharerePost-User-7951591answered 6 years ago0Perfect, thank you!SharerePost-User-7951591answered 6 years ago0Hey @REDACTEDUSERWould you mind uploading an ASCII version of the FBX file? I don't have Blender on my system to do it myself, otherwise I can see if someone else on our side has Blender if you don't want to.SharerePost-User-7951591answered 6 years ago0Thanks again!SharerePost-User-7951591answered 6 years ago"


Question: How can I define Slot Priority for Slot Values in an Intent in LexV2

"I am trying to build my Lex Bot but receiving this error messageDetails Recommended actionSlot ids [NumberTwo, Number] don't define a slot priority. Update the intent to add a priority to these slots.-Slot ids [Number] don't define a slot priority. Update the intent to add a priority to these slots.-Slot ids [Answer] don't define a slot priority. Update the intent to add a priority to these slots.-Slot ids [Number] don't define a slot priority. Update the intent to add a priority to these slots.-I am using AWS SDK for Lex with Python to create the bot.How can I update the intent to add slot priority for my slots?ThanksFollowComment"

Answer: 0Accepted AnswerHelloThe typical steps when creating bots via api are as followsCreate botCreate bot version or use draftCreate bot localeCreate intents in the localeCreate slots inside the intentUpdate the intent with the slot priority list - This is the step you are missing. Once you have the slot ids you should populate the slotPriorities list inside intent - https://docs.aws.amazon.com/lexv2/latest/APIReference/API_UpdateIntent.html#lexv2-UpdateIntent-request-slotPrioritiesBuild the botLet us know if this works for you.Thanks!CommentShareDonanswered 4 months ago0Thanks. Using this UpdateIntent operation was the solution.The python AWS Lex v2 SDK method I used to add slot priorities to and Intent slot were:create_intentcreate_slotupdate_intentCommentSharerePost-User-3428371answered 4 months ago


Question: [SES] Emails considered as SPAM by Outlook servers because reverse DNS checks fail

"Hello,ContextWe have been sending emails from an application using SES for several years without any issue.A verified identity is used for sending the emails.ProblemStarting from last week, a few emails are considered as SPAM by Outlook servers. This is a small proportion of the emails.For example, it concerns at least two mails received yesterday at 15 h 25 and 15 h 26 UTC + 2.When we look at the message's headers, this one seems interesting:x-forefront-antispam-report:CIP:54.240.3.11;CTRY:US;LANG:fr;SCL:5;SRV:;IPV:NLI;SFV:SPM;H:a3-11.smtp-out.eu-west-1.amazonses.com;PTR:ErrorRetry;CAT:SPOOF;SFS:(13230016)(83170400001)(356005)(81166007)(6486002)(5660300002)(34206002)(1096003)(9316004)(71280400001)(28085005)(8676002)(4326008)(79850400001)(7696005)(63370400001)(6506007)(336012)(26005)(9686003)(36736006)(956004)(42882007);DIR:INB;CAT:SPOOF indicates that the mail is considered as spoofing.This is due to PTR:ErrorRetry which indicates the reverse DNS check is failing.When I manually do this reverse DNS check (with dig -x 54.240.3.11), it works.Settings that could matterSPFSPF is active and seems working:received-spf: Pass (protection.outlook.com: domain of eu-west-1.amazonses.comdesignates 54.240.3.11 as permitted sender) receiver=protection.outlook.com;client-ip=54.240.3.11; helo=a3-11.smtp-out.eu-west-1.amazonses.com; pr=CDKIMDKIM signature is also enabled and DKIM authentication passes (we have  dkim=pass (signature was verified) in the authentication-results header.SES eventsIn case that matters, we are tracking the "bounce", "complaint", "delivery delays" and "reject" events and there was no event for the impacted mails.QuestionsI think that the origin of the error can:an issue with the AWS DNS record at the time of the mail sending ;a network issue impacting the Outlook server ;?Have you already encountered this kind of issue ?If so, were you able to solve it ?If you need more details, do not hesitate.Thank you for your help and have a nice Day.RomainFollowComment"


Question: How to avoid Personalize SIMS to recommend only popular items?

"I ran into a case where Personalize SIMS recipe returns as top similar items the same list for almost every product, which are pretty much the top popular items.How to avoid Personalize doing that? I see there is a popularity_discount_factor parameter but its use is unclear. Happy to be clarified and to hear real stories of this being fixed.FollowComment"

Answer: "0Accepted AnswerAccording to the Amazon Personalize documentation "In the absence of sufficient user behavior data for an item, this recipe recommends popular items.".You could change the popularity_discount_factor hyperparameter close to 1 to push the model to pay less attention to an items' popularity during calculation of similarity.CommentShareAleksandr_Panswered 3 years ago0I'm having the exact same issue. SIMS is only recommending the most popular items for that recently added items (i.e. new items with, I presume, a very small number of interactions). That behavior is not much sense to me. Using HPO does not solve the issue, and adjusting popularity_discount_factor neither. Also, tuning the rest of the solution hyperparamenters changes nothing for that new items getting only popularity items as recommendations.Please, if you have any suggestion it will be nice to hear. I've used SIMS in other projects without problems.CommentSharerePost-User-7232721answered a year ago"


Question: Where is domain join directory option in the new launch experience?

"We have been launching windows instances using the Seamless domain join instructions using our AD Connector for several years. Since being automatically opted-in to the new launch experience, I cannot find the option join a domain no longer appears. I can still get to it by opting-out to the old experience, but that is going to go away at some point.Am I missing this or did this feature not make it into the new experience?FollowComment"

Answer: "0Accepted AnswerHi Sean,I also have noticed that the domain join feature is not available on a new EC2 console. However, when you open the info link next to launch an instance it is stated that AWS is working on adding this feature on the new console. Hopefully by the time the old console gets deprecated this feature will be available on the new console.CommentShareCebianswered a year agoSean a year agoThanks. I opened that link as a new tab and just got the EC2 dashboard and assumed their documentation was broken too.Share"


Question: Unknown session property with UNLOAD

"I've tried to UNLOAD() data from MySQL to S3 by Athena engine 3 and got an error: INVALID_SESSION_PROPERTY: Unknown session property ***.manifest_location.*** - this is my Data source.But it works with Athena engine 2. I can't find any info about this error.UNLOAD (SELECT * FROM database.table)TO 's3://my-bucket/table/'WITH (format = 'PARQUET',compression = 'SNAPPY')FollowComment"


Question: Newbie: Unable to use string with "like" in a query

I have a database of large.csv files which I am trying to query but it does not fetch any results when I use the string variable with "like" function. Here is my query:abc is declared as INTpc is declared as STRING-- View ExampleCREATE OR REPLACE VIEW view_name ASSELECT abcFROM "test_log"WHERE pc like 'IG%';FollowCommentSanjeeb 3 months agoCan you please verify your select statement and see whether you are able to see any data or not. For example :-Test the query - select * from test_log limit 100;Test the query - select * from test_log where pc like 'IG%';If the above are working then you should create the view and see the result.Share

Answer: "0We need more information to diagnose.Does the query return data if you don't specify WHERE statement?Does the value for pc starts with 'IG'?This page might help you investigating the issue.https://aws.amazon.com/premiumsupport/knowledge-center/athena-empty-results/CommentShareYota_Hanswered 3 months agorePost-User-8576385 3 months agoThanks Yota for the quick response. Here are my comments:Yes the query returns data if I remove the WHERE statementYes the value for pc must start with 'IG'ShareYota_H 3 months agoI tried on my end but couldn't reproduce the issue. LIKE clause works as expected.If you're okay would you share minimal DDL and CSV content here for reproduction? (please deduct identifiable info)Alternatively you can contact AWS support for private support.SharerePost-User-8576385 3 months agoSure Yota,DDL:CREATE EXTERNAL TABLE IF NOT EXISTS xxxxxx.xxxxxx_xxx (priceper double,year int,dateoftransfer date,propertytype string,duration string,price int,postcode string,lad21cd string,transactionid string,id int,tfarea string,numberrooms string,classt string,CURRENT_ENERGY_EFFICIENCY string,POTENTIAL_ENERGY_EFFICIENCY string,CONSTRUCTION_AGE_BAND string)ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'WITH SERDEPROPERTIES ('field.delim' = ',')STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'LOCATION 's3://xxxxxx/xxxxxx/'TBLPROPERTIES ('classification' = 'csv');-- View ExampleCREATE OR REPLACE VIEW view_name ASSELECT "year"FROM "xxxxxxx_xxx";CSV:priceperyeardateoftransferpropertytypedurationpricepostcodelad21cdtransactionididtfareanumberroomsclasstCURRENT_ENERGY_EFFICIENCYPOTENTIAL_ENERGY_EFFICIENCYCONSTRUCTION_AGE_BAND4069.767442202022/05/2020FL350000IG11 0ABE09000002{BA558B33-27A3-76EF-E053-6B04A8C0B4B7}1844750186NA118383NO DATA!4818.181818202006/08/2020FL265000IG11 0ABE09000002{BA558B33-2824-76EF-E053-6B04A8C0B4B7}1850436055NA118383NO DATA!3837.209302202027/03/2020FL330000IG11 0ABE09000002{CB0035E6-46A2-58AE-E053-6B04A8C091AF}1846649886NA118484NO DATA!4050.632911202129/01/2021FL320000IShareYota_H 3 months agoTested after updating your DDL with WITH SERDEPROPERTIES ('field.delim' = '\t') and TBLPROPERTIES ('classification' = 'csv', 'skip.header.line.count'='1'). It's working fine, I can filter by WHERE postcode like 'IG%' for both table and view. Sorry I don't have further guidance on your issue - please wait for other comments or contact AWS support.Share"