Question: Job on  parallel cluster using hpc6a.48xlarge not running

"Hello,I want to run a simulation on a parallel cluster (alinux2) using 2 hpc6a.48xlarge instances (192 CPUs). I created the cluster and submitted the job using slurm. The problem is that the job stays waiting in the queue and never runs (I left it for more than 1 day). I tried running the same job using another kind of instance, with the same number of CPUs and it worked perfectly, so it is an issue with this specific instance (hpc6a.48xlarge).I also tried using only 1 hpc6a.48xlarge instance (96 CPUs) but did not work either. I copy the squeue information at the end of the message. It shows some 'BeginTime' reasons, although I have not programmed my job to start later.What may be the reason for this issue?I am creating the cluster on a new company account. May the issue be related to the usage of the account?  I ask this because I have already configured the same cluster on a personal account (with significantly more usage than the company account) and the job is executed almost immediately.I would appreciate any advice on resolving this issue[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (BeginTime)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (BeginTime)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (BeginTime)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (None)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (None)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (BeginTime)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (BeginTime)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (BeginTime)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (BeginTime)[ec2-user@ip- OpenFOAM]$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)2   compute  foam-64 ec2-user PD       0:00      1 (None)FollowComment"

Answer: "1Can you take a look at clustermgtd and slurm_resume.log/var/log/parallelcluster/ for details of the launch of the compute nodes?CommentShareAWS-User-1193008answered a year agoAWS-User-9531109 a year agoThese are the last lines of the clustermgtd file:2022-04-01 02:01:44,064 - [slurm_plugin.clustermgtd:_get_config] - INFO - Reading /etc/parallelcluster/slurm_plugin/parallelcluster_clustermgtd.conf2022-04-01 02:01:44,065 - [slurm_plugin.common:read_json] - INFO - Unable to read file '/opt/slurm/etc/pcluster/run_instances_overrides.json'. Using default: {}2022-04-01 02:01:44,066 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Managing cluster...2022-04-01 02:01:44,072 - [slurm_plugin.clustermgtd:_manage_compute_fleet_status_transitions] - INFO - Current compute fleet status: RUNNING2022-04-01 02:01:44,072 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Retrieving nodes info from the scheduler2022-04-01 02:01:49,148 - [slurm_plugin.clustermgtd:_get_ec2_instances] - INFO - Retrieving list of EC2 instances associated with the cluster2022-04-01 02:01:49,211 - [slurm_plugin.clustermgtd:_perform_health_check_actions] - INFO - Performing instance health check actions2022-04-01 02:01:49,211 - [slurm_plugin.clustermgtd:_maintain_nodes] - INFO - Performing node maintenance actions2022-04-01 02:01:49,211 - [slurm_plugin.clustermgtd:_maintain_nodes] - INFO - Following nodes are currently in replacement: (x0) []2022-04-01 02:01:49,211 - [slurm_plugin.clustermgtd:_maintain_nodes] - INFO - Found the following unhealthy dynamic nodes: (x1) ['compute-dy-defaultcompute-9(compute-dy-defaultcompute-9)']ShareAWS-User-9531109 a year agoAnd these are the last lines of slurm_resume.log:2022-04-01 02:03:41,449 - [slurm_plugin.common:is_clustermgtd_heartbeat_valid] - INFO - Latest heartbeat from clustermgtd: 2022-04-01 02:02:49.260630+00:002022-04-01 02:03:41,449 - [slurm_plugin.resume:_resume] - INFO - Launching EC2 instances for the following Slurm nodes: compute-dy-defaultcompute-102022-04-01 02:03:41,513 - [slurm_plugin.instance_manager:add_instances_for_nodes] - INFO - Launching instances for slurm nodes (x1) ['compute-dy-defaultcompute-10']2022-04-01 02:03:42,293 - [slurm_plugin.instance_manager:_launch_ec2_instances] - ERROR - Failed RunInstances request: d40261ce-e840-4e92-857b-a86c2820c73b2022-04-01 02:03:42,293 - [slurm_plugin.instance_manager:add_instances_for_nodes] - ERROR - Encountered exception when launching instances for nodes (x1) ['compute-dy-defaultcompute-10']: An error occurred (VcpuLimitExceeded) when calling the RunInstances operation: You have requested more vCPU capacity than your current vCPU limit of 0 allows for the instance bucket that the specified instance type belongs to. Please visit http://aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit.2022-04-01 02:03:42,294 - [slurm_plugin.resume:_resume] - INFO - Successfully launched nodes (x0) []2022-04-01 02:03:42,294 - [slurm_plugin.resume:_resume] - ERROR - Failed to launch following nodes, setting nodes to down: (x1) ['compute-dy-defaultcompute-10']Share1There is a chance that your new account doesn't have the requisite quota for hpc6a. Please open a support case asking for a limit increase, or reach out to your account manager or solution architect to help.CommentShareChris Pollardanswered a year ago0Thank you for all your comments.I have requested a new limit increase to 514 vCPUs (I had 240 vCPUs). Hopefully, that will solve the issue.CommentShareAWS-User-9531109answered a year ago0There is a specific quota for the hpc6a. Probably you need to increase it tooCommentShareAWS-User-1193008answered a year ago"


Question: cloudFront distribution using Domain without using Route 53

"Hello, I'm trying to create a distribution on Cloudfront and S3 and use a domain outside of AWS, with route 53 I can use my domain without problems, but my question is whether it would be possible to use my domain without using Route 53. on the site that hosts my domain, since I don't have ns, it's like something is missing. I can manage to configure the cname from the certificate generated in ACM, the txt and the alias that points to the cloudFront, but I could not configure the ns. So, is Route 53 fundamental in this process?FollowComment"

Answer: "0Hi Denise,Route 53 isn't mandatory at all to use CloudFront. You can use another DNS provider like CloudFare or Godaddy or any other. For example here is a guide using CloudFare DNS.You need to define your domain, create a certificate in ACM and associate it with cloudfront, and then with your DNS provider point the CNAME to the cloudfront distribution.Hope this helps you!CommentSharePablo Guzmananswered 2 months agoEXPERTkentradreviewed 2 months ago0Yes, I've done all that and it didn't work. I think the problem is with my DNS provider, because in AWS everything is ok, the certificate... But I wanted to have a confirmation that it was indeed possible. Thank you very much!CommentShareDeniseanswered 2 months ago"


Question: Restricting incoming NLB traffic to internal IP addresses

Customer has a use case which involves handling TCP traffic (not http or https) in their internal VPC.Is there a way to restrict source IPs? For an NLB there are no security groups.FollowComment

Answer: "0Accepted AnswerIf the clients are in the same VPC you could simply use an internal NLB, which will only have private IP addresses, and not be available to clients outside the VPC's private connectivity.When you use an instance type target group on your NLB, the security group rules s of the targets are applied if they refer to the client's source IP or source network CIDR.CommentShareEXPERTzobAWSanswered 3 years ago"


Question: AWS 50% Coupon - Cancelling the exam

"Hello,I have scheduled for AWS Developer (DVA 01) which is expiring on Feb 27.I have used my 50% coupon for that and I am not keeping well and may need to cancel based on my health situation.I cannot reschedule as this is expiring in 4 days, hence I have to register for DVA 02 and in that situation, If I cancel this exam scheduled will I get the coupon back to be used for the another exam.Please clarifyFollowComment"

Answer: "0Accepted AnswerHi,Yes, if you cancel the exam, the voucher can still be used for the next scheduleCommentShareEXPERTalatechanswered 3 months ago"


Question: Operational excellence vs. ...

"The first pillar of the well-architected framework is Operational Excellence. However, then the remaining five are, also after consulting the docs, just more specialized aspects of the first pillar. Is there a good way to think about Operational Excellence capturing something totally different than the other pillars? Thanks!FollowComment"

Answer: "0Accepted AnswerI understand your viewpoint. That being said, we know that customers are often concerned with only a subset, sometimes just one, of the six pillars. There is some known overlap. For example, not implementing a security best practice of rotating credentials could impact your reliability if the credentials are no longer valid and your workload is unavailable. The best practices are grouped according to customer feedback on where the best practice applies and where we have seen application enable success.CommentShareRodney Lesteranswered a year agopeisenha a year agoThanks a lot, in particular for the background information!Share"


Question: EndPoint Connection Error while connecting to AWS SecretManager from MWAA

I was just trying to complete the connection configuration for MWAA which has been specified in the article https://docs.aws.amazon.com/mwaa/latest/userguide/connections-secrets-manager.html but i'm ending up with the Endpoint Connection Error. Here is the Error log from Airflow"botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://secretsmanager.secretsmanager.amazonaws.com/"FollowComment

Answer: "0Please ensure that the Secrets Manager endpoint is accessible via PrivateLink, Internet Gateway (IGW), and/or other routing, and that security groups and endpoint policies allow access, as well as that the MWAA execution IAM role is authorized to access Secrets Manager.CommentShareJohn_Janswered 23 days agoNitin 23 days agoI've attached the required policies and opened all traffic in the security groups. The example dag to read the variables stored in AWS Secretsmanager works fine. but for reading connections it's giving the above error!Here is some more info on error log,[2023-05-05, 11:49:43 UTC] {{base.py:73}} INFO - Using connection ID 'aws_default' for task execution.[2023-05-05, 11:49:45 UTC] {{taskinstance.py:1768}} ERROR - Task failed with exceptionTraceback (most recent call last):File "/usr/local/airflow/.local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_connconn = connection.create_connection(File "/usr/local/airflow/.local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connectionfor res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfofor res in _socket.getaddrinfo(host, port, family, type, proto, flags):socket.gaierror: [Errno -2] Name or service not knownShare0Logs looks like connection error , usually hooks use aws_default connection to connect secretes manager or any AWS service ( if you use operator hook to connect). I believe your environment is private with out NAT , in such case connecting to endpoint secretsmanager.secretsmanager.amazonaws.com use public IP and it will not receive response from secretsmanager.secretsmanager.amazonaws.com as a workaround you can use endpoint secretsmanager.{region}.amazonaws.com  (secretsmanager.us-east-2.amazonaws.com ) to communicate.procedure to regional endpoint this :Update connection in webUI-->Connection-->AWS_Default-->Extras{"role_arn":"arn:aws:iam::AccountID:Role/rolename","region_name":"region like us-east-2"}orCreate a new connection with name aws-newdefault with connection type " aws web service" and in extras, provide below .{"role_arn":"arn:aws:iam::AccountID:Role/rolename","region_name":"region like us-east-2"}and in your task code make sure you use conn_id=aws-newdefaultIf your environment is not private then you need to make sure if there are no restrictions in networks like policies , NACL,SG ,TGW etc or do partial testing by creating Ec2 instance using same VPC and SG . SSH to ec2 and try to do telnet, Dig, nc with and without region to make sure your network is good.Telnet secretsmanager.amazonaws.comtelnet secretsmanager.amazonaws.com 443nc -v secretsmanager.amazonaws.com 443ping secretsmanager.amazonaws.comsimilarly do above for secretsmanager.us-east-2.amazonaws.comif you still see issue , I would suggest to open a support case.CommentSharepriyathamanswered 22 days agoNitin 17 days agoIt turns out to be an error in the sample code provided in the AWS documentation,setting the parameter "client = hook.get_client_type()" or hardcoding the region for example "client = hook.get_client_type(region_name='us-east-1')" solved the issue!Share"


Question: Retrieving the full text of a SQL statement - value for "--group-identifier"

"the --group-identifier is db.sql.id. In this example, my-sql-id represents a SQL ID retrieved by invoking pi get-resource-metrics or pi describe-dimension-keyswhen I try using --group-identifier db.sql.idI get:An error occurred (InvalidArgumentException) when calling the GetDimensionKeyDetails operation: This Key ID is not valid.and there is no guidance on what to use from  get-resource-metrics or pi describe-dimension-keysCan someone provide more information?FollowComment"


Question: How to disable the Lex error handling for sending contents length higher than 1024 from Slack

"We are integrated the AWS lex into our Slack public channel as an answer bot. But if someone talks in the channel with a context length that is higher than 1024, the AWS lex bot will return an error that is kind of annoying and not related. Is there any way to turn off the error handling or make it mute for error reporting in Slack?Thanks for any help/support.FollowComment"

Answer: "1You can improve the user experience by these two steps:(1) provide a hint message for your user: "we recommend you to upload the audio within 90 secs or 210 words limit." So that the user understands how to interact with the bot properly. The number is based on the assumption from the internet: average Speaking Speed Rate is 150 wpm and the average word in the English language is 4.7 characters. Feel free to adjust it accordingly.(2) Overwrite the default handling: This documentation is showing you how to overwrite the default error handling by updating the fallback intent setting. So that the user can receive the relevant message.CommentShareMia Canswered 10 months ago0Maximum utterance lengthDefines how long Amazon Lex V2 waits before speech input is truncated and the speech is returned to your application.Default this is 13 seconds. You may need to set max-length-ms to a particular number of seconds, so that user does not talks something that is not more than 1024 chars.https://docs.aws.amazon.com/lexv2/latest/dg/session-attribs-speech.html#voice-input-time-outCommentShareprabuanswered a year ago"


Question: DynamoDB "explore items" showed me an error "Key not found in KeySchema."

"DynamoDB "explore items" showed me an error "Key not found in KeySchema" and items didn't show up. Actually, we have a key in this table and 30+ items. We had no problem when we checked the items by using the old dashboard. This problem only occurs in the new dashboard.There was a "Feedback" button before, but now it disappeared.How can I feedback this issue to the team?FollowComment"


Question: Enabling Identity Federation with AD FS 3.0 and Amazon AppStream 2.0

"I’m troubleshooting problems with the SSO integration of AppStream2.0 with ADFS. I followed step by step the guide indicated at the following link :https://aws.amazon.com/it/blogs/compute/enabling-identity-federation-with-ad-fs-3-0-and-amazon-appstream-2-0/but when I try to navigate via browser in the RelayState URL , the ADFS page returns an error.Need to enable some other AWS service or is there a more detailed guide? As an image for APPStream2.0 I used a standard image of the Appstream2.0 service while the AD FS is resident on our windows machine. AD FS is 3.0 .Any suggestion?ThanksFollowComment"

Answer: "1At the bottom of the blog there is a few steps that are commonly missed for AD FS 3.0  under "Enable RelayState and forms authentication".What is the exact error message you are getting?You can use a browser extension, like SAML-tracer to capture the SAML message. Are you seeing the correct attributes and values? Can you paste a redacted copy of the SAML summary?CommentShareJeremy_Sanswered a year agoAWS-User-6631477 a year agoHere is an error from the SAML-tracer:HTTP/1.1 302 FoundContent-Length: 0Content-Type: text/html; charset=utf-8Location: https://xxxxxxxxxxxxxxxxxxxxxxx:443/adfs/ls/idpinitiatedsignon?client-request-id=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxServer: Microsoft-HTTPAPI/2.0P3P: ADFS doesn't have P3P policy, please contact your site's admin for more detailsSet-Cookie: MSISSamlRequest=; expires=Sun, 27 Mar 2022 10:29:04 GMT; path=/adfsMSISAuthenticated=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==; path=/adfs; HttpOnly; SecureMSISLoopDetectionCookie=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==; path=/adfs; HttpOnly; SecureShare0The part about the "Enable RelayState and forms authentication" has been configured in all parts.It's an error message that returns ADFS server(error id 364 AD FS), does not allow me to login into the application. SAML tracer does not return any error.The RelayState URL at which point I built it on the example shown in the AS2 setup page(Enabling Identity Federation with AD FS 3.0 and Amazon AppStream 2.0).Is there any other way to get it?In the url I also checked the characters paid attention to the key sense.ThanksCommentShareAWS-User-6631477answered a year ago0Attributes seem correct, I try to recheck.Here is an error from the SAML-tracer:HTTP/1.1 302 FoundContent-Length: 0Content-Type: text/html; charset=utf-8Location: https://xxxxxxxxxxxxxxxxxxxxxxx:443/adfs/ls/idpinitiatedsignon?client-request-id=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxServer: Microsoft-HTTPAPI/2.0P3P: ADFS doesn't have P3P policy, please contact your site's admin for more detailsSet-Cookie: MSISSamlRequest=; expires=Sun, 27 Mar 2022 10:29:04 GMT; path=/adfsMSISAuthenticated=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==; path=/adfs; HttpOnly; SecureMSISLoopDetectionCookie=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==; path=/adfs; HttpOnly; SecureHere instead an error of AD FS:Microsoft.IdentityServer.Service.IssuancePipeline.CallerAuthorizationException: MSIS5007: The caller authorization failed for caller identity DOMAIN\user for relying party trust https://signin.aws.amazon.com/saml.Thanks in advance for any suggestions or helpCommentShareAWS-User-6631477answered a year ago"


Question: aws iot rule sql

"AWS IoT Rule Engine can handle MQTT Payload values through SQL statements. When MQTT Client sends a message by putting a byte array value in the payload, can the byte array value be decoded and selected through SQL statements? For example,Publishtopic : test/payloadpayload : [72,101,108,108,111,119,111,114,108,100](( Hello World ))SQLSQL SELECT * FROM 'test/payload' I wonder if it is possible to SELECT 'Hello World' by converting the above payload.FollowComment"

Answer: "0Hi,there are no native functions in the SQL supported by Rules engine to decode binary data, but you can pass the data to a Lambda function that can return the decoded payload. Since Lambda functions do not support binary input, you need to convert the input to base64.SELECT aws_lambda("arn:aws:lambda:us-east-1:account_id:function:lambda_function", {"message":encode(*, 'base64'}) as output FROM 'test/payload'CommentShareEXPERTMassimilianoAWSanswered 2 months ago0Hi rePost-User-1228067,Thanks for your question!If I understand correctly:You have a Byte array as your MQTT payloadYou are trying to perform a filter on the decoded content from the binary array to know if the decoded (plain text) equals 'Hello World' for example. Then sending it out to some downstream services.Is that correct?If that's the case, then your answer is yes (provided that your binary data is Base64 or Protobuf encoded). The native function is 'decode' and the syntax isdecode(value, decodingScheme) where 'value' is the encoded string payload and decodingScheme is either 'base64' or 'proto'.In your specific example, depending on the client flexibility, you'll either ensure first that your payload is a Base64 binary (or Protobuf) encoded value instead of a Byte Array, or have a lambda function cloud side to transform Byte Array to Base64 binary and republish to a new topic for processing. Then do the following:SELECT decode(payload,"base64") AS myfilterresult from 'test/payload' where decode(payload,"base64") = 'Hello World' ; or anything similar, depending on the filtering conditions you'd like to apply.Here are some documentation links for reference:https://docs.aws.amazon.com/iot/latest/developerguide/iot-sql-functions.html#iot-sql-decode-base64.https://docs.aws.amazon.com/iot/latest/developerguide/binary-payloads.htmlCommentShareCharlyscott237answered 2 months ago"


Question: "Chaning AMI and apply it to ASG for refresh old instacnes automatically, once I deploy"

"Hello,recently, I use Windows 2019 of EC2 instances and using launchconfigurations for AutoScaling.I used own tool for deploy for ec2.Is there anyway I can make AMI and apply to ASG and also exchanging old ec2 to new AMI image automatically?I thought I could use 2 lambda to make it happend, but it didnt go well.My scnerio isdeploy app -> API gateway -> lambda 1 ( making AMI) -> when the status chaged to 'available' -> trigger lambda 2 -> changing AMI image of ASG and refresh.anyone have experience of this?FollowComment"

Answer: "0The way I've commonly seen it done is as follows.ASG resources are defined in CloudFormation with rolling update of instances, and the template has an "AMI" parameter set to the ID of your custom AMI.  When you want to update it to a new AMI ID, run a CloudFormation Stack update with the new value for the AMI parameter.  You could trigger the Stack update as soon as your new AMI is available, or have something scheduled that regularly checks for new AMIs for Stacks that have an "AMI" parameter, for example.BTW Launch Configurations are being deprecated, so best to update to using Launch Templates.CommentShareEXPERTskinsmananswered 6 months agoHeesu 6 months agoThank you for comment.first of all I know Launch Configuration will be deprecated but it will support until I have it, so I won't change yet.and you told me that I can use run cloudformation Stack update. However, I want to do all the process automatically.is it possible using cloudformation stack update as automation?Share0Does this automation template work for you?  It uses launch templates instead of launch configs, but otherwise it sounds like what you're asking for:https://docs.aws.amazon.com/systems-manager/latest/userguide/automation-walk-patch-windows-ami-autoscaling.htmlCommentShareShahad_Canswered 6 months ago"


Question: EMR with Specific AL2 kernel 5 or AL2022 kernel5

"When does AWS EMR be available with AL2 with kernel 5.x or AL2022 with kernel 5.x.If we wanted to use custom AMI's, can you provide documentation of how to create custom AMI's for EMR.FollowComment"

Answer: "0Accepted AnswerYes, AWS EMR do support customized AMI with Amazon Linux 2. You can launch an EC2 with AL2 and upgrade the kernel to 5.x or install any other packages you want following by the actions in the link below.https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-custom-ami.htmlOn the other hand, as AL2022 is still in public preview stage, it's better to wait for it's generally available to make customized AMI on it. If you're interested in the AL2022, you can try to ask for preview and try the actions above to see if EMR can run on AL2022.CommentShareYuxianganswered a year ago"


Question: access ssh with different port in browser web lightsail

access ssh with different port in browser web lightsailFollowComment

Answer: "0Accepted Answerrelated q: https://repost.aws/questions/QUedvb-mz_RUqB19ytXj8__g/cannot-ssh-to-lightsail-instance-via-aws-console-port-22-is-openCommentSharePrincy Aanswered 3 months agoGK 3 months agoHow come the related post to ssh not to change the port, i think your goal is to change the port and acess from the browser, So you are able to acess Lightsail from browser with different SSH portShare0HiIf you would like to change the port of the Lighsail instance, You can do that by using the following settingsSSH into the machineEdit file /etc/ssh/sshd_config change #portsudo service sshd restartThen you need to allow PORT In the inbound rules from the Lightsail consoleRemember Light sail WEB SSH works Only  with default 22 port, So if you change the port you need to use any SSH client like putty or termius, Lightsail web ssh doesnt workThank YouGKCommentShareGKanswered 3 months ago"


Question: My EC2 tg4.micro simply disappeared today with no trace

"Hello,My EC2 tg4.micro simply vanished today, until yesterday night I was accessing it fine.This instance was running fine since 23 July, Region us-east-1 (N. Virginia), with AMI Amazon Linux 2 arm64, and almost vanilla with nothing fancy installed yet.Nothing new or scheduled was On, The EBS is gone too, no trace.Uniq tip I have is my cloud watch cpu alarm(credit balance spot), which stopped receiving data today at 3 AM (UTC), the EC2 Events have no logs.I have 2 security groups on it, internal user ec2-user disable, my user was using ssh pairs so still no password ssh login allowed...I use AWS for a couple of years (since 2018) and have never experienced something like that, I am with no clue here.Thanks in advance!FollowComment"

Answer: "0Well that's terrifying! I don't have an answer sadly. Out of curiosity, was it a spot instance?CommentSharestraubdevanswered 2 years ago0Me too =/ ...Never thought that this kind of thing could happen with no trace...Yup, it was a spot instance, but even if it was the case that my credits expire or reached some kind of limit, my instance should be stopped but never deleted and I have no policy/role for that so...I even lookup Spot Instance pricing history for any changes but its completely stable for t4gmicro in that region...I already opened a support case because I still have hope that inside AWS they can find what happen reading logs that customers don't have access to.CommentSharebrunofanswered 2 years ago0brunof wrote:Me too =/ ...Never thought that this kind of thing could happen with no trace...Yup, it was a spot instance, but even if it was the case that my credits expire or reached some kind of limit, my instance should be stopped but never deleted and I have no policy/role for that so...I even lookup Spot Instance pricing history for any changes but its completely stable for t4gmicro in that region...I already opened a support case because I still have hope that inside AWS they can find what happen reading logs that customers don't have access to.Hello, just an update here.It really was an EC2 Spot related cause, when I though interrupted I didn't assimilate it as erased =PI just received a helpful reply from AWS support.So I probably set a “maximum Spot price" when creating the instance, and it was claimed and terminated after a 2-minute notification(which I thought was an email).See more:You can read more about Spot Instance requests and terminations in the "Understanding Spot Instances and Spot Price" section of our Spot Instances page:http://aws.amazon.com/ec2/spot-instancesThe EBS cause was also explained by the AWS:When a Spot instance is terminated:1) All data on instance-storage will be lost.2) All EBS volumes that are attached which are set to "Delete on Terminate" will be deleted.3) All EBS volumes that are attached which are set to NOT "Delete on Terminate" will be left unattached to any instance. You can then attach them to another instance and get to your data.So, my mistake folks, still faithful using AWS again ;)CommentSharebrunofanswered 2 years ago0brunof wrote:Me too =/ ...Never thought that this kind of thing could happen with no trace...Yup, it was a spot instance, but even if it was the case that my credits expire or reached some kind of limit, my instance should be stopped but never deleted and I have no policy/role for that so...I even lookup Spot Instance pricing history for any changes but its completely stable for t4gmicro in that region...I already opened a support case because I still have hope that inside AWS they can find what happen reading logs that customers don't have access to.Hello, final update here.Yesterday I found the log at CloudTrails -> Event history with one entry with name BidEvictedEventBy default the behavior is to terminate the Spot instance, checked on cli with:aws ec2 describe-spot-instance-requestsUseful link:[Why is my Spot Instance terminating even though the Spot price doesn't exceed the maximum price?]:https://aws.amazon.com/pt/premiumsupport/knowledge-center/ec2-spot-instance-termination-reasons/[Specify the interruption behavior]https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#specifying-spot-interruption-behaviorEdited by: brunof on Aug 6, 2021 6:28 AMCommentSharebrunofanswered 2 years ago"


Question: Why is the number 0 saved as Null(true) in DynamoDB sometimes?

"I have a DynamoDB data model where I've set the default value to a particular field explicitly as the number 0 on item generation, which is done in Flutter. The new item is sometimes saved correctly, but not always. The number 0 is occassionally replaced with Null(true), which is a completely different AttributeValue according to my understanding of the Rust AWS SDK. This impacts the requirement to .uwrap() something from .get() from the same data table in my lambda function, but I am not sure what the difference is caused by and hence having trouble figuring out how best to solve.The field is not a required field, but is always defaulted.// imported modelclass Donate {  final int? _timstamp;  ...// initialization code  final donate = aws.Donate(    timstamp: 0    ...This results in the s/s attached.FollowComment"

Answer: "0Hello,Thank you for using the AWS DynamoDB service.From the question, I understand that you are observing that occasionally  the integer value '0' is saved as "NULL" in one of your DynamoDB data model which later impacts the lambda function and wants to know how to overcome this issue as it happens with random records.To better assist you on this issue, we would like to replicate the scenario and identify the root cause of this random behavior with integer value 0 for your DynamoDB data model. As this medium is not the best place to ask for your code snippet and other configurations details that you are using at your end, so I'd recommend you to please open a Support case with our Support engineer team so that the DynamoDB engineer can assist you in identifying the root cause of the issue. To open a support case with AWS using the link: https://console.aws.amazon.com/support/home?#/case/createCommentShareSUPPORT ENGINEERKartik_Ranswered 5 months ago"


Question: Bug: Glue 4.0 | Python 3.10 | pandas library

"I have been experimenting using Glue 4 which supports Python 3.10 and pandas.I am adding pandas as a zipped library through the --extra-py-files functionality for a gluetl job.When running my job, it fails importing pandas (version 1.4.3) (import pandas as pd) with the following which I copy-pasted from the cloudwatch logs:2022-12-06 16:49:09,450 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(77)): Error from Python:Traceback (most recent call last):  File "/tmp/database_monitoring.py", line 2, in <module>    import pandas as pd  File "/home/spark/.local/lib/python3.10/site-packages/pandas/__init__.py", line 48, in <module>    from pandas.core.api import (  File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module>    from pandas.core.groupby import (  File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>    from pandas.core.groupby.generic import (  File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 76, in <module>    from pandas.core.frame import DataFrame  File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/frame.py", line 170, in <module>    from pandas.core.generic import NDFrame  File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/generic.py", line 147, in <module>    from pandas.core.describe import describe_ndframe  File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/describe.py", line 45, in <module>    from pandas.io.formats.format import format_percentiles  File "/home/spark/.local/lib/python3.10/site-packages/pandas/io/formats/format.py", line 105, in <module>    from pandas.io.common import (  File "/home/spark/.local/lib/python3.10/site-packages/pandas/io/common.py", line 8, in <module>    import bz2  File "/usr/local/lib/python3.10/bz2.py", line 17, in <module>    from _bz2 import BZ2Compressor, BZ2DecompressorModuleNotFoundError: No module named '_bz2'2022-12-06 16:49:09,450 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(77)): Error from Python:Traceback (most recent call last): File "/tmp/database_monitoring.py", line 2, in <module> import pandas as pd File "/home/spark/.local/lib/python3.10/site-packages/pandas/__init__.py", line 48, in <module> from pandas.core.api import ( File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/api.py", line 47, in <module> from pandas.core.groupby import ( File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/groupby/__init__.py", line 1, in <module> from pandas.core.groupby.generic import ( File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 76, in <module> from pandas.core.frame import DataFrame File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/frame.py", line 170, in <module> from pandas.core.generic import NDFrame File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/generic.py", line 147, in <module> from pandas.core.describe import describe_ndframe File "/home/spark/.local/lib/python3.10/site-packages/pandas/core/describe.py", line 45, in <module> from pandas.io.formats.format import format_percentiles File "/home/spark/.local/lib/python3.10/site-packages/pandas/io/formats/format.py", line 105, in <module> from pandas.io.common import ( File "/home/spark/.local/lib/python3.10/site-packages/pandas/io/common.py", line 8, in <module> import bz2 File "/usr/local/lib/python3.10/bz2.py", line 17, in <module> from _bz2 import BZ2Compressor, BZ2Decompressor ModuleNotFoundError: No module named '_bz2'I believe this is a bug in AWS Glue 4.0 as opposed to a user issue. Is anyone able to advise or confirm? And if so, is there a bug fix planned for this?FollowComment"

Answer: "0Do you need a specific / higher version than the included one? If not, no need to provide any zip at all.I can't / won't test Glue v4 at all - since the documentation (e.g. https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html) and other actual service functionality like the aws_glue_interactive_sessions module is not updated. So much about "general availability".CommentSharerePost-User-4559153answered 6 months agoJDay 6 months agoI did attempt this but found it was more trouble that it was worth. I am using libraries which also use pandas so I would need to add custom logic to ignore  the pandas dependency when installing those libraries. And even then, that is supposing the pandas version AWS offers is compatible.This would add significant overhead to something which is supposed to be an out-of-the-box solution. Hence, I just won't use Glue 4.0 and will think of an alternative unless this is resolved.SharerePost-User-4559153 6 months agoI guess there was some context missing and neither me nor you did actually google the error message ;)Share0The integrated pandas version faces the same error and that must be fixed on the system level, e.g. https://stackoverflow.com/questions/50335503/no-module-named-bz2-in-python3.CommentSharerePost-User-4559153answered 6 months agoJDay 6 months agoYep, this is what I thought. I did google and found this post initially (https://stackoverflow.com/questions/74623358/pandas-not-working-in-aws-glue-4-0-version) but seeing as it had no traction I thought i would post here hoping that someone from AWS would be able to offer a timeline for a fix.Share"


Question: Disable Cookie popup in AppStream Web client

"I'm using AWS Appstream 2.0 to deliver windows applications through a browser to students in university (including Europe). To achieve this, we use the webclient embedded in another page.However, every time a student starts a session, they are presented with a Cookie popup from AWS.How do I disable cookies when embedding the AWS Appstream 2.0 web client?FollowComment"

Answer: "0If I understood your issue correctly, Appstream web client (web broswer with appstream link) is embedded with another webpage, right? In that case, Appstream web client is basically a browser, it could be Chrome, Firefox. So check the browser specific documentation on how to disable it.CommentShareMayank_Janswered 6 months agorePost-User-2609373 6 months agoIt is not really answering the question.We are using AppStream to build a service for other users. I don't want to bother these users with a cookie popup. I actually want AppStream to stop asking my customers to accept cookies from AWS.Share"


Question: No option to retry failed App Runner service deployments

"This is a real pain. After configuring all the ENV variables and the service options, the deployment will fail if, for example, there is a bad source image or the source image repo is empty. Afterwards, there is no option to retry it - the "resume" is greyed out, and there is no button that will redo the deployment. Why? Why do I have to delete this now and enter everything again? This is very tedious.FollowCommentAWS-User-Mike a year agoYes, that's a pain. While waiting for a fix, a workaround is to export service definition json with describe-service command, edit it, and run create-service with that json as input.Share"

Answer: "1Hello there,Thanks for bring this up to our attention. This use case totally makes sense and I would like to let you know that we already had a open feature request for this. Please track the progress via the Github issue. Meanwhile, I would recommend you to deploy a sample app or sample image in your initial deployment, then deploy you own app, this way we can make sure that even your app's deployment failed, you can roll back to the previous working state (which is sample app), then make necessary changes to your app and re-deploy.CommentShareSUPPORT ENGINEERZhiyuan_Lanswered a year agoHooHost a year agoIn our case we just need to make sure that there's something in ECR before deploying (for now). Thanks for looking into it!Share"


Question: PG_CRON doesn't work

"Hi, I have a RDS instance, I was used to running maintenance sql statements using pg_cron but for some reason it has stopped working and I can't seem to work out why. I no longer can see any activity in cron.job_run_details and my parameters in my RDS instance stay the same. I have also tried the following UPDATE cron.job SET nodename = ''; with no luck. Can anyone please advise?FollowComment"

Answer: "0Hi ,I understand you are trying to run maintenance sql statements using pg_cron and it has stopped working.Please correct me if I misunderstood.The pg_cron extension is supported on RDS for PostgreSQL engine versions 12.5 and higher. Please verify your engine version if it is updated to the supported one.Please also verify if you have done the following :1)Is cron_job enabled2)Enabling the pg_extension,granting permissions to the pg_cron,scheduling pg_jobs following the link[1].3)For further assistance you can contact AWS Customer Support to review your RDS instance[2].I hope the above is helpful.Reference[1]https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL_pg_cron.html[2]https://aws.amazon.com/premiumsupport/knowledge-center/CommentShareKaraboanswered a year ago"


Question: user data on EC2 not working with CDK

"Hi team,I created an ec2 instance with CDK and added used data on it :   const userData = readFileSync("lib/my-script.sh", "utf8");    instance.addUserData(userData);    instance.applyRemovalPolicy(RemovalPolicy.DESTROY);my-script.sh :#! /bin/bashsudo suyum -y update# adding MYSQL client to connect to RDS(MYSQL) instance.sudo yum install mysqlbut when i connect to my ec2 instance via the session manager tab and do MySQL command I have this error :sh: mysql: command not foundlooks like the user data script was not run.I'm I missing something?how can I know if the user data script was successfully run on the ec2 instance?found this on : /var/log/cloud-init-output.logSep 15 00:17:50 cloud-init[xxxx]: util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]Sep 15 00:17:50 cloud-init[xxxx]: cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)Sep 15 00:17:50 cloud-init[xxxx]: util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failedci-info: no authorized ssh keys fingerprints found for user ec2-user.thank you!FollowComment"

Answer: 1I think you need to do sudo yum install mysql -y so that the yum utility doesn't try and prompt the user.CommentShareEXPERTBrettski-AWSanswered 9 months ago0Accepted Answerremoved  "sudo" from the script as User Data scripts are run as root user as per this answer:https://stackoverflow.com/questions/67740072/user-data-not-running-command-in-mysqlhttps://stackoverflow.com/questions/41603517/installing-mysql-client-on-ec2-via-cloudformationalso noticed that ec2 add #!/bin/bash on top of the script even if it already existsmy final script :yum -y update# adding MYSQL client to connect to RDS(MYSQL) instance.yum -y install mysqlCommentShareJessanswered 9 months ago


Question: Robomaker - IoT reserved topics?

"Hi,Are there any reserved topics being sent over greengrass IoT by a robomaker deployment job that can be accessed while a deployment is occuring?AustinFollowComment"

Answer: "0Hi Austin,AWS RoboMaker Fleet Management uses AWS Greengrass to deploy and launch ROS applications. When you create a new robot, it will ask you to install the AWS GreenGrass Core Software (for more information:  https://docs.aws.amazon.com/greengrass/latest/developerguide/module2.html). When the Greengrass software is running and a RoboMaker deployment job is invoked, a local lambda function will pull down the robot application bundle defined in the RoboMaker Robot Application. The status updates of the deployment job are displayed in the RoboMaker console, or can be retrieved programmatically through the DescribeDeploymentJob API call https://docs.aws.amazon.com/cli/latest/reference/robomaker/describe-deployment-job.html.There are no additional IoT topics reserved in this process, other than those reserved through Greengrass. For more information about reserved topics in Greengrass and AWS IoT, check out our documentation here: https://docs.aws.amazon.com/iot/latest/developerguide/reserved-topics.htmlHowever, there are some environment variables which are reserved for RoboMaker. For example, you can use ROS_ROOT to define a local installation of ROS to use instead of the bundle. For troubleshooting deployments, here are some common pitfalls: https://docs.aws.amazon.com/robomaker/latest/dg/troubleshooting-deployments.html. To look at the environment variables included with the AWS Lambda function deployed on your behalf, you can open the Greengrass console, select the Lambda function (it should be postpended with DO_NOT_DELETE, as you can probably conclude - do not delete or save any changes here!!) and press edit. In here, you will find a list of environment variables used with the Lambda function.If you are looking to determine the status of a deployment job on a given robot, you could use the DescribeRobot API call https://docs.aws.amazon.com/cli/latest/reference/robomaker/describe-robot.html. This will return the following robot states: Available, Registered, PendingNewDeployment, Deploying, Failed, InSync, and NoResponse.AWS Step Functions is a great tool to manage more complex deployment workflows in conjunction with AWS RoboMaker. With AWS Step Functions, you could define a loop with a wait condition that checks on the status of all deployments (and run the above describe API calls). On the event of a failure, you could then define a custom AWS Lambda function to process the failure.Happy building!CommentSharejeranswered 3 years ago"


Question: Embedding a Quicksight QSearchBar for a user in a namespace (other than default)

"I am able to successfully embed Quicksight dashboards using the multitenancy (namespaces-based) approach described in Quicksight's documentation. However, when I try to embed a Q Search Bar, the request completes successfully, but it generates a blank iFrame. Are Q Search Bars only allowed to be embedded for users in the default namespace? If not, how can I go about granting access to Q Topics to users in different namespaces and successfully embedding the Q Search Bar?Here are some additional details:I've followed the Amazon Quicksight documentation for embedding dashboards for registered users as well as embedding the Q search bar for registered users. I've created a workflow that creates namespaces and uses row level security to restrict access based on namespaces / groups. While the dashboard embedding works fine for users created in their respective namespaces, the Q Search Bar embedding does not.This applies even if I create a Q Topic from the same dashboard that I can embed successfully for the namespace user. In both cases, the result is simply a blank iframe where the Q Search Bar should appear. The actual call to GenerateEmbedUrlForRegisteredUser does not throw an error when attempting to embed the Q Search Bar for a user from the namespace - the returned link just generates a blank iFrame.However, if I use same operation ( GenerateEmbedUrlForRegisteredUser) using the OWNER account in the default namespace, the embedded Q Search Bar works fine. This appears to suggest that there is some sort of permissions issue at play. I cannot find any documentation that covers how to use the API to grant access to a Q Topic to a user in a specific namespace. I've tried removing RLS as well, but the iFrame is still empty. Any help would be greatly appreciated.FollowCommentrePost-User-4275458 10 days agosame issue here. did you receive an answer from support? I dont see a way to share the topic with the group in a specific namespaceShare"

Answer: "0Did you try it after creating and sharing the topic with that user? If it still doesn't work, please contact Customer Support with the details.CommentShareVijayanswered a year ago0This is the expected behavior as of now, since the search bar in embedded functions same as website. The user can utilize the onError callback in the SDK to be made aware when there are no topics. However, to make this more clear, we could provide a UX (in embedded mode only) that surfaces the error as wellCommentShareGregTanswered a year ago0https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/quicksight/client/update_topic_permissions.htmlyou need to update the topic permission using the SDK to the group/arn arn which includes the namespace in the url as well.I was able to get it to work.CommentSharerePost-User-4275458answered 10 days ago"


Question: cloudformation script fails when invoked via lambda

"i have a cloudformation stack file on s3 which works just fine when I launch it from console. but when I trigger the same script using boto3 from AWS lambda, i get an encoded error message. Decoding it then gives me the following json output. (converted to yaml for readability)---allowed: falseexplicitDeny: falsematchedStatements:  items: []failures:  items: []context:  principal:    id: AROA3H7OMTCQKOORZGNNY:Spawn-reward-service    arn: arn:aws:sts::773057xxxxxx:assumed-role/Spawn-reward-service-role-g0vgzj5h/Spawn-reward-service  action: ec2:RequestSpotFleet  resource: arn:aws:ec2:us-east-1:773057xxxxxx:subnet/subnet-e18921ca  conditions:    items:    - key: ec2:Vpc      values:        items:        - value: arn:aws:ec2:us-east-1:773057xxxxxx:vpc/vpc-aa2884c1    - key: aws:Resource      values:        items:        - value: subnet/subnet-e18921ca    - key: aws:Account      values:        items:        - value: '773057xxxxxx'    - key: ec2:AvailabilityZone      values:        items:        - value: us-east-1e    - key: ec2:ResourceTag/Name      values:        items:        - value: 1e subnet    - key: ec2:SubnetID      values:        items:        - value: subnet-e18921ca    - key: aws:Region      values:        items:        - value: us-east-1    - key: aws:Service      values:        items:        - value: ec2    - key: aws:Type      values:        items:        - value: subnet    - key: 773057xxxxxx:Name      values:        items:        - value: 1e subnet    - key: ec2:Region      values:        items:        - value: us-east-1    - key: aws:ARN      values:        items:        - value: arn:aws:ec2:us-east-1:773057xxxxxx:subnet/subnet-e18921caThe Lambda role has AmazonEC2SpotFleetTaggingRole, NetworkAdministrator and AWSCloudFormationFullAccess policies attached. So not sure where the hangup is. Any help appreciated.Here is the lambda function in full for reference.import json, boto3def lambda_handler(event, context):    cloudformation = boto3.client('cloudformation')    response = cloudformation.create_stack(        StackName='yj',        TemplateURL='https://s3.amazonaws.com/cf-templates-xxxxxxxxxxxx-us-east-1/reward-service-01-spot.yaml',        Parameters=[            {                'ParameterKey': 'InstanceType',                'ParameterValue': 't2.large'            }        ]    )    return responseFollowComment"

Answer: "1Are you providing CloudFormation with a different role to use for execution of the Stack?  If not it will use the credentials you've used to invoke CloudFormation.  In this case that's the role with NetworkAdministrator and AWSCloudFormationFullAccess policies which wouldn't contain anything to allow ec2:RequestSpotFleet.CommentShareEXPERTskinsmananswered 4 months agorePost-generalenthu 4 months agoI am specifying the role in my template - Do you mean to say, it inherits the lambda role and overrides the role in my script? In any case, i added the AmazonEC2SpotFleetTaggingRole which allows request spot instances, but with no luck. I realized my json was truncated. updating my question and posting it as yaml in case it is hiding any clues.SharerePost-generalenthu 4 months agoI am not sure it has anything to do with VPCs though. I did try to move the lambda into a VPC, but that just causes the lambda to timeout and never trigger the stack even after a minute.Shareskinsman EXPERT4 months agoThe "different role" I was talking about is passing the following to create_stack():RoleARN (string) --The Amazon Resource Name (ARN) of an Identity and Access Management (IAM) role that CloudFormation assumes to create the stack. CloudFormation uses the role's credentials to make calls on your behalf. CloudFormation always uses this role for all future operations on the stack. Provided that users have permission to operate on the stack, CloudFormation uses this role even if the users don't have permission to pass it. Ensure that the role grants least privilege.  If you don't specify a value, CloudFormation uses the role that was previously associated with the stack. If no role is available, CloudFormation uses a temporary session that's generated from your user credentials.As it looks like you're not providing a RoleARN, your Lambda's role will be used to execute all of CloudFormation's AWS API calls while creating the Stack.SharerePost-generalenthu 4 months agoi did try passing the role ARN using        RoleARN='arn:aws:iam::773057xxxxxx:role/aws-ec2-spot-fleet-tagging-role', unfortunately I get Role arn:aws:iam::773057xxxxxx:role/aws-ec2-spot-fleet-tagging-role is invalid or cannot be assumed". Since this is an AWS managed role, i cannot edit it, but it is setup to be assumed based on my reading of the policy.Share0Accepted AnswerThe whole request spotfleet approach seems to be deprecated. The recommended approach after further research seems to be to use the create_fleet api which moves a bunch of config to launch templates from the cloudformation scripts to launch templates.I got onboard this train and have just a lambda function with launch template and no cloudformation stuff.CommentSharerePost-generalenthuanswered 4 months ago"


Question: error message: when calling the DetectDocumentText operation

"Good morning,I use boto3 to extract with aws textract the image text (see part of my code below).If the test file is 'image.png' I have a result otherwise the following error message: when calling the DetectDocumentText operation: Unable to get object metadata from S3. Check object key, region and/or access permissions.In my account, the access keys are active.What is the problem?Thanks for your feedback.my code :import boto3from trp import Documents3BucketName = "textract-console-eu-west-3-a6efc625-6d9c-44ec-a867-7d946e8a0c29"textractmodule = boto3.client('textract',region_name='eu-west-3')response = textractmodule.detect_document_text(Document={S3Object': {Bucket': s3BucketName,Name': test}})FollowComment"

Answer: "0Your code is the same of myhere is my Role/user :Id -   Last utilisation(last hour) - region (eu-west-3)-  last used service (textract) - Statut(Active)I dont know why with image.png is correct , and with another file ist incorrect.CommentSharerePost-User-4710275answered 3 months agoAlexR 3 months agoYou found the problem, correct?Share0Hi,thanks a lot for your response.I find the solution and it works;the codefrom PIL import Imageimport boto3import ioimport pandas as pdfrom trp import Documentimage=images[0]image.save(path+"image.png", format="png")im = Image.open(path+"image.png")buffered = io.BytesIO()im.save(buffered, format='PNG')client = boto3.client('textract')response = client.analyze_document(Document={'Bytes': buffered.getvalue()},FeatureTypes=['TABLES'])for item in response["Blocks"]:if item["BlockType"] == "LINE":tata=item["Geometry"]["BoundingBox"]X0 , Y0, width, height = tata['Left'] , tata['Top']  ,tata['Width'] , tata['Height']dim = item["Text"].upper(), X0 , Y0 , width , heightDetail_page.append(dim)df = pd.DataFrame (Detail_page,columns = ['text','X0','Y0','width','height'])CommentSharerePost-User-4710275answered 3 months agoAlexR 3 months agoGood to hear that.May I ask you please to accept the question, so it will help also others.ThanksShare0Please try this way:import boto3 #import Document# Add your file to your bucket and change the bellow 2 liness3BucketName = "dus-idp-textract" document = "The_river_effect_in_justified_text.jpg"textractmodule = boto3.client('textract',region_name='eu-west-1')response = textractmodule.detect_document_text( Document={ 'S3Object': { 'Bucket': s3BucketName, 'Name': document } })responseI just tried it and it works.Also please add the permissions (Textract Policy) to the Role/User you are using in order to have permission to call Amazon Textract APIsIf you consider that this answer helped, please accept itCommentShareAlexRanswered 3 months ago"


Question: Amazon WorkSpace Registration Code

I have just installed and now trying to setup the WorkSpace app for Windows and is prompting me for a registration code. I never received an email for said code. Can I find that online somewhere or generate a new one?FollowComment

Answer: 0Have you created an Amazon WorkSpace environment within your AWS account and have you provisioned an Amazon WorkSpace desktop.  The WorkSpaces client requires that you have an already provisioned Amazon WorkSpace to connect.  You can find your registration code by viewing the properties of you AWS Directory Connector.  It may looks something like SLIAD-XXXXXX depending on which region you provisioned your Amazon WorkSpace.CommentShareRobert_Fanswered 10 months ago


Question: EMR on EKS local files

"Currently for EMR on EC2 we pass in job configuration using a local file my.jar --cfg /local/path/config.cfg ..... Now I'm migrating jobs to both EMR on EKS and EMR Serverless. How can I achieve the same with EMR on EKS or EMR Serverless?I don't see much in documentation on how that should be handled.Regards,AtharvaFollowComment"

Answer: "0Hello,Can you please confirm, if this (my.jar --cfg /local/path/config.cfg) is a spark config? If yes, then you can pass this spark config through spark-submit.The spark-submit script in Spark’s bin directory is used to launch applications on a cluster. [1]Also, please feel free to check out our AWS Integration and Automation Github page on EMR on EKS here -https://github.com/aws-ia/terraform-aws-eks-blueprints/tree/main/examples/analytics/emr-on-eksFurthermore, information regarding sparkSubmit and example in terms of AWS CLI command for EMR Serverless has been mentioned on our AWS docs here -https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/jobs-spark.htmlconf = An arbitrary Spark configuration property.Configuration that you provide as part of sparkSubmitParameters using --conf.Additionally, in regards to containers, you can pass config files to a container using configmaps as a ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume. [2]A ConfigMap allows you to decouple environment-specific configuration from your container images, so that your applications are easily portable.Caution: ConfigMap does not provide secrecy or encryption.References:[1] https://spark.apache.org/docs/latest/submitting-applications.html[2] https://kubernetes.io/docs/concepts/configuration/configmap/CommentShareSUPPORT ENGINEERYash_Canswered 9 months ago"


Question: My Postgres database is rebooting forever

"My database stopped responding and I rebooted it.Now, it is forever rebooting and there is an error message in the log:2023-01-11 15:39:16.131 GMT [30125] LOG:  skipping missing configuration file "/rdsdbdata/config/recovery.conf"2023-01-11 15:39:16 UTC::@:[30125]:LOG:  database system is shut downFollowComment"

Answer: "1Based on my understanding, RDS PostgreSQL instance is experiencing an Out of Memory issue. It appears that the instance was experiencing memory pressure and that this prevented the host from contacting the monitoring system. To relieve the instance's memory burden, it was restarted. Operating System Process, RDS Process, Monitoring Process, and Database Engine Process typically utilize the memory allotted to the underlying host. The engine process takes precedence over all other processes running on the host whenever the customer workload requires memory. As a result, the monitoring agent was unable to communicate heartbeat data to the monitoring application because of the limited free memory. The Monitoring Application assumes there may be a problem because it hasn't received any heartbeat data from the Monitoring Agent located on the same RDS host. As a result, it starts the reboot procedure.Steps to try:Check for the workload/queries consuming more memory and work on further Query tuning from Performance Insight tool.Check for the RDS PostgreSQL memory related parameters and tune them if needed.Please refer to the link (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/creating_alarms.html)which will guide you to set up cloudwatch alarms in RDS Metrics and pro-actively monitor those. You can create a CloudWatch Alarm to monitor and send alerts when the RDS metric goes beyond a certain threshold limit. CloudWatch uses Amazon SNS to send an email.You can scale up the RDS instance class to next higher level so that it can have more memory. You need to perform load testing on the Non-Prod instance first before working on the Prod as we are not sure of the impact of the new workloads on the instanceIf none of these work, please open a support case with AWS using the following link. (https://console.aws.amazon.com/support/home#/case/create)CommentSharePriyanka_Sanghvianswered 5 months agorePost-User-1095545 5 months agoThank you for your answer. However, the database is not running anymore and I can't modify the instance. I read some logs and realized that the database is corrupted on RDS.Share"


Question: Next Steps in Athena Based Project

"Hello, I have a project where I am using Athena to allow users to look up people in a table. The next step of the project would have them record when they have a match and then to be able to store those validated records in a different table .I would like to be able to record the person who did the search in athena, the date they did so, and then to be able to allow them to fill out some additional fields.Can all of this happen in Athena? Or do i need to use other services for recording the matches.?FollowComment"

Answer: "0Without knowing the full details of your use-case it is hard to recommend a solution.In general Athena is used for Big Data Analytics use cases, for running queries on a data lake.In your use-case, if you want to support fuzzy text-based search, then using Amazon OpenSearch may be a better option for the peoples database that users will search. To record the search history, you could use DynamoDB or RDS/Aurora MySQL or Postgres database. If you use DynamoDB, you could have the user that did the search as the partition key and the current datatime as the sort key. You could create other fields based on what information you want to capture.Assuming you are planning to build a GUI application, you could create APIs in Lambda or ECS/EKS that will implement the backend functionality. The front-end could be integrated with Amazon Cognito and you can pass JWT tokens from Cognito to the backend APIs so they can determine who the user doing the search is and accordingly log them into your database.Just throwing out some ideas without knowing a whole lot of details.CommentShareEXPERTIndranil Banerjee AWSanswered 9 months ago"


Question: How IP addresses get logged when using Lambda and API Gateway

A customer is asking if IP addresses get logged when using Lambda and API Gateway? He mentioned that this is important for them to be GDPR compliant once the new regulation comes into force. Thank you!FollowComment

Answer: "1Accepted AnswerYou can enable Amazon CloudWatch Logs to trace API calls. Once enabled, API Gateway will log API calls in CloudWatch. The API gateway access log will have the information of the IP address "ip": "$context.identity.sourceIp":https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.htmlOr check the value of "X-Forwarded-For" if you want to build a function in API Gateway by using Lambda Proxy integration:https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.htmlCommentShareMODERATORAWS-User-8412605answered 5 years ago"


Question: AWS SSO/IdC User Access Portal Timeouts

"Hello,Is there a way to change the session duration of the AWS SSO/Identity Center user access portal itself (i.e. the page at companyname.awsapps.com/start)? That portal seems to have an 8 hour session duration, but our organization prefers a much shorter duration. We're using Active Directory as our identity source.We've been able to configure the session duration of applications displayed in the portal to be shorter, which works as expected, but we need the portal itself to have a shorter session duration.FollowComment"

Answer: "1Unfortunately, this doesn't address our needs or use case. We're not using the User Portal to provide users access to the AWS management console. We use the user portal to provide users access to SAML federated applications.We have the timeout of those applications set to one hour. However, if a user's session with that application times out, they just go back to the AWS user portal, click the icon of that app, and are immediately back in. We need the session to the AWS user portal to time out after one hour as well, forcing them to sign back in to AWS SSO/IdC before they can re-access one of these SAML-provided applications.CommentShareash-gobillsanswered 5 months ago1That's for session manager within SSM, not for the user portal of AWS SSO. Doesn't seem like this is a possibility. We'll switch to another IdP instead of AWS SSO, then.CommentShareash-gobillsanswered 5 months ago0Hello,AWS Single Sign-On (SSO) enables you to customize the session duration to AWS accounts ranging from 1 hour up to 12 hours. You can configure session duration for each permission set so that you can optimize how long your users can access AWS Management Console and AWS CLI for your AWS accounts. For example, when your users need to run long-running operations, you can increase the session duration so that your users can complete the operation using a single session. To learn more about configuring session duration, see Set Session Duration.Link- https://docs.aws.amazon.com/singlesignon/latest/userguide/howtosessionduration.htmlHope this helps you with the question !!CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 5 months ago0I checked your use-case again & I believe, In that case you need to configure the session lifecycle on the Azure AD side. (see: https://docs.microsoft.com/en-us/graph/api/resources/tokenlifetimepolicy?view=graph-rest-1.0). The lifetime of the session set the maximum time a user can use the Amazon SSO web portal without re-authenticating to the external IDP.Check out the link pasted above for additional details.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 5 months ago0Hi,I am a Software Dev Engineer working in AWS Identity Center team.The feature being requested to configure the session duration for access portal is currently supported only for IdentityCenter and ExternalIdp identity source.More information here-https://docs.aws.amazon.com/singlesignon/latest/userguide/configure-user-session.htmlUnfortunately, this feature is currently not supported for AD Identity source: https://docs.aws.amazon.com/singlesignon/latest/userguide/manage-app-session.htmlCommentShareAarushi Chawlaanswered 5 months ago"


Question: Unable to Import Custom slot type from a zip file

"Hello everyone.I am trying to create a translator chatbot. I have a compressed json file in zip format and I want to use the zip file to create a slot type. The zip file contains 100 enumeration values. However, I do not see the option to create a slot type from an import on my aws interface. I am stucked. I need help please.FollowComment"

Answer: "0Hi,In Lex V2, we currently only support Import/Export feature for Bot and BotLocale resource. If you want to add a slotType to one of the locales, you can use our CreateSlotType API instead: https://docs.aws.amazon.com/lexv2/latest/dg/API_CreateSlotType.html.Feel free to reach out if you have questions.CommentShareShreyBanswered a year ago"


Question: CPU Credit Balance Getting Zero

"We have the Jenkins Server on EC2 instance, the CPU Credit Balance is automatically going to zero without any load on the instance.What are the alternatives for this issue so that CPU Credit Balance stays above zero?Instance Size is 8gb,Instance Type is t3.mediumFollowComment"

Answer: "0here are some key concepts about burstable instancesother than that, check if you are seeing the right metric aggregation is set to average or maximum, see the screenshot belowhere is some documentation about how to monitor the credit usagefinally, if you want to continue providing CPU credit even if you are running out of credits you can enable Unlimited modeCommentShareEXPERTDamian Olguinanswered 17 days agoAWS-ADolganov 16 days agoJust a small addition on the "unlimited mode" - it can be less cost effective than running a normal non-burstable instance type, it's worth looking into the m, r or c instance types insteadShare"


Question: OAuth Scope based Authorization in AppSync

"Assume we have Cognito user pools with one application (ex: photos app) with custom scopes defined (ex: photos.read). In https API GW, we can restrict the access using the scope and audience. This article is explains using group claims -  https://aws.amazon.com/blogs/mobile/graphql-security-appsync-amplify/.  Can we do similar authorization based on OAuth scope ?FollowComment"

Answer: "1Hello!I was taking a peek at your question and conversing with some colleagues, and I think I may have found an example CDK (AWS Cloud Development Kit) example that you may be able to test this with:https://serverlessland.com/patterns/eventbridge-api-appsync-cdkThe serverlessland.com website is a site that our serverless developer advocates curate serverless patterns on, and after taking a peek at the page and corresponding GitHub page, I think this may be able to get you a head start on testing this.Note it may spin up some infrastructure you don't need, but it will give you start on automating your deployments.CommentShareSUPPORT ENGINEERTim_Panswered a year ago"


Question: SES DMARC Google fail

"Hi everyone,We are using SES to send our emails from ec2 servers, we had already setup the DKIM / SPF, however the google is still failing the DMARCHere is the google raw message dkim=pass header.i=@amazonses.com header.s=7v7vs6w47njt4pimodk5mmttbegzsi6n header.b=j5RPkmox;       spf=pass (google.com: domain of 010101667a5912cb-e91361bf-4377-49b3-88b9-8397f33f8425-000000@us-west-2.amazonses.com designates 54.240.27.53 as permitted sender) smtp.mailfrom=010101667a5912cb-e91361bf-4377-49b3-88b9-8397f33f8425-000000@us-west-2.amazonses.com;       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=showpo.comThe DMARC report from google is failed as well<record><row><source_ip>54.240.27.19</source_ip><count>2468</count><policy_evaluated><disposition>none</disposition><dkim>fail</dkim><spf>fail</spf></policy_evaluated></row><identifiers><header_from>domain.com</header_from></identifiers><auth_results><dkim><domain>amazonses.com</domain><result>pass</result><selector>7v7vs6w47njt4pimodk5mmttbegzsi6n</selector></dkim><spf><domain>us-west-2.amazonses.com</domain><result>pass</result></spf></auth_results></record>Any advise will be appreciated here. Thanks in advancedCheersFollowComment"

Answer: "0Hi,We have how to comply with DMARC described here https://docs.aws.amazon.com/ses/latest/DeveloperGuide/dmarc.html, you can follow the instruction to set up everything.CommentShareYiwei-AWSanswered 5 years ago"


Question: Find old Forum Thread ID for migrating our Simple AD to a new AWS Managed Microsoft AD

I'm looking for the info formerly included in an old forum thread that has disappeared with the migration to the new AWS re:Post forum.Is it possible to find the info in this thread? https://forums.aws.amazon.com/thread.jspa?threadID=261247I'm looking for info on migrating our existing Simple AD directory to a new AWS Managed Microsoft AD directory.FollowComment

Answer: "0Hi Mr PRSTO,Unfortunately, AWS forum has been deprecated and only select threads were migrated into re:Post.For information about migrating SimpleAD, you can look that this blog post:How to Migrate Your Microsoft Active Directory Users to Simple AD or AWS Managed Microsoft ADCommentShareEXPERTDzung_Nanswered a year ago"


Question: GameLift Server SDK5 WebSocket returning throttling errors on InitSDK() call

"I am using GameLift through the Go version of Server SDK5. When I instantiante the fleet with 50 game server processes on an instance (there is only one instance), I get this error in game server process logs :2023/05/03 09:56:02.812774 websocket.go:55: [DEBUG]:Response header is: map[Content-Length:[113] Content-Type:[application/json; charset=UTF-8] Date:[Wed, 03 May 2023 09:56:02 GMT] Via:[1.1 ffe68b4a5d64737b8a3ccde75553a7ac.cloudfront.net (CloudFront)] X-Amz-Apigw-Id:[EVxj9F6WjoEFufA=] X-Amz-Cf-Id:[TYcHtwXIHyFPVlBRLHFuptYeLk7tueWkCyUj7Iuyqf20C2eK2u2lNQ==] X-Amz-Cf-Pop:[DUB56-P1] X-Cache:[Error from cloudfront]]2023/05/03 09:56:02.812782 websocket.go:56: [DEBUG]:Response body is: {"Action":"$connect","RequestId":"$connect","StatusCode":429,"ErrorMessage":"ThrottlingException: Rate exceeded"}2023/05/03 09:56:02.812795 gamelift.go:61: GameLift InitSDK failed with error : [GameLiftError: ErrorType={19}, ErrorName={Local connection failed.}, ErrorMessage={[GameLiftError: ErrorType={24}, ErrorName={WebSocket Connection Failed}, ErrorMessage={connection error 429 Too Many Requests:websocket: bad handshake}]}]It seems that the WebSocket server is throttling the connections because there are too many at a time. GameLift creates and runs all 50 processes of game servers within a timeframe of 50 milliseconds , each process then calls the SDK function gameLiftServer.InitSDK(), and so throttling happens. As a result, we have to do random sleeps to avoid the processes calling InitSDK() at the same time, and retrying if the call still fails with a 429 error.Is that normal behavior ? With SDK4, we did the same thing, same number of calls, and no errors happened. Is there a way to increase the throttling maximums on the GameLift WebSocket server side ?FollowComment"

Answer: "0Thanks for bringing this to our attention. This is something we're looking into currently, but in the meantime I would suggest implementing a backoff and retry strategy with jitter to ensure your calls aren't getting throttled. If you'd like to track this with the service team please make sure to cut a customer support case.Thanks!CommentShareShashankJ_AWSanswered 23 days ago"


Question: data type converstion in DocumentDB using aggregation pipelines

"I have a MongoDB aggregation pipeline that converts a string to a double.  I just realized that DocumentDB doesn't support the $convert or $toDouble operators.  Is there any workaround for this problem while still using an aggregation pipeline?Below is an extract from my pipeline that works in MongoDB Atlas:...{    $convert:{        input:$subDoc.stringVar,        to:'double',        'onError':0,        'onNull':0    }}, ...FollowComment"

Answer: "0Hello. You could use the Javascript parseFloat() method and programmatically update those strings to double in the existing collection.  Or use an $out stage in the aggregation and update the results afterwards.Something like this will loop through all documents in the collection and update the string:db.collection.find({}).forEach( function(doc) {  doc.subDoc.stringVar = parseFloat( doc.subDoc.stringVar );  db.collection.update( { _id: doc._id }, {$set: { "subDoc.stringVar": doc.subDoc.stringVar } } ) });CommentShareMihai Aanswered a year ago"


Question: Amplify and Google Search

"Hi, I hope I can explain this succinctly! I have a Docusaurus site hosted on AWS Amplify. I have the domain set up and it resolves correctly to https://docs.cavalry.scenegroup.co/The problem I'm facing is that the url that shows up in Google Search is the original amplify url. e.g. https://main.d1fnr9j76bdr39.amplifyapp.com/I'm no expert (!) but it looks like the canonical metadata is the bit I need to change. Is there anything I can do via the console (or otherwise) to resolve this?Thanks.FollowComment"

Answer: "0Since your website can be accessed by multiple urls, you need to tell the search engine which URL is the canonical URL. One way to do it is to add a link tag in your pages <head> to point to the canonical url.example: <link rel="canonical" href="https://docs.cavalry.scenegroup.co" />More info can be here: https://developers.google.com/search/docs/advanced/crawling/consolidate-duplicate-urlsCommentShareMODERATORMohamed_Banswered a year agoChris Hardcastle a year agoThanks Mohamed. When I do a URL inspection in Google Search the Amplify link is the 'User-declared canonical'. I have not explicitly/ intentionally declared this so was hoping there might be something in the Console to allow me to specify this (e.g. it seems AWS is setting it). This site is for documentation so we've got 100s of pages - I'm hoping not to need to do this for each page individually!ShareMohamed_B MODERATORa year agoI inspected your website and see that you have <link data-react-helmet="true" rel="canonical" href="https://main.d1fnr9j76bdr39.amplifyapp.com/"> in your website. You need to edit that.ShareMohamed_B MODERATORa year agoI am not a Docusaurus expert but I think you are defining the url https://main.d1fnr9j76bdr39.amplifyapp.com somewhere in your config, and Docusaurus is automatically injecting the canonical tag link for you https://docusaurus.io/docs/next/seo#single-page-metadataShare"


Question: AWS not responding within 24 hours on System impaired

"Hi everyone,Has anyone had experience with AWS not responding within 24 hours on System impaired case ?Do you have suggestions on what to do in such a case?In case the AWS team sees this question, the number of my case is xxxxxxxxxx.Please help.Edit: Removed case ID — Chrissy B.FollowComment"

Answer: "0Hello,I'm so sorry for the delay you experienced in receiving assistance. I have reviewed your case and can see that one of our support engineers has followed up with you. Thank you for your patience!Best regards,Kita B.CommentShareEXPERTAWS Support - Kitaanswered a year ago"


Question: vscode builder id fails to connect

"I setup a dev environment in codecatalyst yesterday and it would trigger vscode when I connected.Today, tried to use it again, but when vscode is launched it fails to connect and checking the AWS Toolkit logs, I see2023-01-12 17:33:44 [ERROR]: log level: info2023-01-12 17:33:45 [INFO]: Retrieving AWS endpoint data2023-01-12 17:33:45 [INFO]: OS: Darwin x64 22.1.02023-01-12 17:33:45 [INFO]: Visual Studio Code extension host:  1.74.32023-01-12 17:33:45 [INFO]: AWS Toolkit:  1.60.02023-01-12 17:33:45 [INFO]: node: 16.14.22023-01-12 17:33:45 [INFO]: electron: 19.1.82023-01-12 17:33:45 [WARN]: AwsContext: no default region in credentials profile, falling back to us-east-1: 2023-01-12 17:33:45 [WARN]: CloudFormationTemplateRegistry: addWatchPattern(**/*.{yaml,yml}): no workspace2023-01-12 17:33:45 [WARN]: CodelensRootRegistry: addWatchPattern(**/requirements.txt): no workspace2023-01-12 17:33:45 [WARN]: CodelensRootRegistry: addWatchPattern(**/package.json): no workspace2023-01-12 17:33:45 [WARN]: CodelensRootRegistry: addWatchPattern(**/*.csproj): no workspace2023-01-12 17:33:45 [WARN]: CodelensRootRegistry: addWatchPattern(**/go.mod): no workspace2023-01-12 17:33:45 [WARN]: CodelensRootRegistry: addWatchPattern(**/build.gradle): no workspace2023-01-12 17:33:45 [WARN]: CodelensRootRegistry: addWatchPattern(**/pom.xml): no workspace2023-01-12 17:33:52 [WARN]: auth (codecatalyst): removing saved connection "codecatalyst.savedConnectionId" as it no longer exists2023-01-12 17:33:52 [ERROR]: aws.codecatalyst.openDevEnv: Error: Connection is invalid or expired. Try logging in again. [InvalidConnection]2023-01-12 17:36:21 [ERROR]: aws.codecatalyst.openDevEnv: Error: Connection is invalid or expired. Try logging in again. [InvalidConnection]2023-01-12 17:38:46 [INFO]: telemetry: sent batch (size=20)2023-01-12 17:38:46 [INFO]: telemetry: sent batch (size=14)2023-01-12 17:41:23 [ERROR]: aws.codecatalyst.createDevEnv: Error: Connection is invalid or expired. Try logging in again. [InvalidConnection]2023-01-12 17:43:46 [INFO]: telemetry: sent batch (size=13)I've trieduninstalling and re-installing the AWS Toolkit extension,creating a new dev environment in codecatalystclicking on the builder ID icon in AWS Toolkit extensionLogging out and back in to codecatalyst.aws with builder idand none of them worked. Any suggestions?FollowComment"

Answer: "1Accepted AnswerHi there,Based on the error message, it looks like your session was expired. Could you please try logging into CodeCatalyst from the Toolkit ? Please feel free to reach out if the error still occurs.CommentShareMODERATORSundaresh_Ianswered 5 months ago1For anyone who couldn't find how to do this -Click on the AWS extension icon in the left hand tab of VSCode -Choose CodeCatalyst > Start from the developer tools extensionFollow any prompts from VSCode about allowing Code to open an external websiteIn your browser, allow aws-toolkit to access your dataIn VSCode, you should now have an option Open Dev Environment - click thisAfter a short delay, you should get a list of the available dev environments for your Builder ID - choose the appropriate environment.CommentShareSimon Hanmeranswered 5 months ago"


Question: "Is it possible for Amplify DataStore to skip a table on client, but to create a DynamoDB table for a scheme type"

"Hi, In our amplify app we have a table that can contain huge text fields, so we would prefer to work with this data on AppSync/QraphQL  level without DataStore.Is it possible to declare the scheme in such a way so the DataStore doesn't even try to sync that model?If I get it right from the docs [1] @model directive is required for Amplify to generate DynamoDB table (and for the @searchable to work).So we currently added an DataStore config on syncExpressions to prevent the sync, but that approach seems odd, so I hope it's possible to exclude the table from DataStore altogether?for the example:type Document @model(subscriptions: { level: off }) @auth(rules: [{ allow: owner}]) @searchable {    id: ID! @primaryKey    content: AWSJSON!}Amplify.configure({    ...awsExports,    DataStore: {        syncExpressions: [            syncExpression(Document, doc => doc?.id('eq', '0'))        ],We still receive warnings with this approach, I guess they are caused by the subscription off param in @model derectiverecoverable error {error: {…}}error: cause: {provider: AWSAppSyncRealTimeProvider, error: {…}}errorType: "Unknown"localModel: nullmessage: "Connection failed: {\"errors\":[{\"message\":\"Validation error of type FieldUndefined: Field 'onUpdateDocument' in type 'Subscription' is undefined @ 'onUpdateDocument'\"}]}"model: "Document"operation: "Update"process: "subscribe"recoverySuggestion: "Ensure app code is up to date, auth directives exist and are correct on each model, and that server-side data has not been invalidated by a schema change. If the problem persists, search for or create an issue: https://github.com/aws-amplify/amplify-js/issues"remoteModel: null[[Prototype]]: Object[[Prototype]]: ObjectoverrideMethod @ react_devtools_backend_compact.js:2367errorHandler @ index.tsx:39(anonymous) @ subscription.ts:450step @ mutation.ts:574...there are similar errors for onCreateDocument and onDeleteDocument[1] - https://docs.amplify.aws/cli-legacy/graphql-transformer/model/FollowComment"

Answer: "1Accepted AnswerHello,From your query, I understand that you would like to use the Document table directly without configuring the DataStore for it. You have tried to configure the DataStore configuration for syncExpression on the table to prevent the syncing of the data. However, you are observing the error on subscription for the generated schema.Please note that, unfortunately, at the moment, the disabling of DataStore for a specific model is not supported in Amplify applications. DataStore can be enabled or disabled for the entire Amplify project at once using the amplify update api command. There is an active feature request with Amplify service team on this issue. You can track this FR on the same GitHub issue. It also mentions the workaround of disabling the sync as you mentioned. We can leverage the same workaround to prevent the DataStore sync of the table.Having said that, for the error you are observing, this is being returned as the subscriptions are not being generated for the Document model. I tried replicating this warning in my sample application however, I did not received this error. To further investigate on this, we would need to check if there is no subscription connection being established for the Document model. For more information, please refer subscribing to data.I hope you find above information helpful. In case you require further assistance specific to the above error being observed, please share if the subscription connection is being established from the application. You may also reach out to us via a Support case.Have a nice day ahead!CommentShareSUPPORT ENGINEERShadab_Aanswered a month ago0Hi @Shadab_A, I was thinking today, that what if I don't even need the GraphQL API for that model. All work with it is done through custom backend code on Node.js. I save the data through AWS SDK DynamoDB API, and query it on client only through my custom NodeJS endpoints and the OpenSearch AppSync API generated from the @searchable.Can I than skip the table on front-end  by the workaround I mentioned at the github  issue[1]?(in short, simply by removing the model from the scheme before running amplify codegen models)Or will it have some other impact that I don't see at the moment?[1] - https://github.com/aws-amplify/amplify-js/issues/10062#issuecomment-1531139785CommentShareArsenanswered a month agoShadab_A SUPPORT ENGINEERa month agoHello,Thank you for writing back.In case you generate the models using the method defined in the GitHub issue, it may work until the changes are not being make to the GraphQL API or the command amplify codegen model. These would generate the models again and might cause issues in future. This has been addressed by Amplify service team on the GitHub comment: https://github.com/aws-amplify/amplify-js/issues/10062#issuecomment-1531636954Let me know in case you have any follow up questions!Share"


Question: AMS Windows 2012 server...failed to authenticate with service

Service successfully installed on source.   "How to solve this?" does not appear to have anything relevant on this particular error.Why am I getting this error in AWS and how do I solve it?FollowComment

Answer: "1We had this same problem - Data replication stalled: Failed to authenticate with Service.In our case the problem was that the Replication Server was unable to access the management service from the custom Subnet we had created for the migration.Apart from the authentication failure error message, we also noticed that Replication EC2 instances were being created, then terminated after about 10 minutes of inactivity.To diagnose this we created a new EC2 instance and edited the Network settings to use the same VPC and Subnet as used by the Replication Server. From an SSH terminal we were able to confirm that the instance could not reach the management API when running the following commandcurl https://mgn.{region}.amazonaws.com/On the Replication VPC this command would timeout,  but on servers in a different VPC this command would return a response immediately.In our case the problem was that we had modified the Network ACL rules to restrict traffic inbound, and response traffic from the outbound http request was being blocked (as Network ACLs are stateless).To fix it we needed to add a Network ACL allowing all Inbound TCP traffic on the ephemeral ports 32768 - 65535.CommentShareJoeanswered 10 months ago0Hello,From the information provided, I am able to determine that this could be an issue with the Service Linked role not created with the right permissions.Application Migration Service uses AWS Identity and Access Management (IAM) service-linked roles. A service-linked role is a unique type of IAM role that is linked directly to Application Migration Service. Service-linked roles are predefined by Application Migration Service and include all the permissions that the service requires to call other AWS services on your behalf.A service-linked role makes setting up Application Migration Service easier because you don’t have to manually add the necessary permissions. Application Migration Service defines the permissions of its service-linked roles, and unless defined otherwise, only Application Migration Service can assume its roles. The defined permissions include the trust policy and the permissions policy, and that permissions policy cannot be attached to any other IAM entity.Link- https://docs.aws.amazon.com/mgn/latest/ug/using-service-linked-roles.htmlCommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 10 months agojake 10 months agoThank you for the prompt reply.  Sorry to say that I'm not really sure where this was supposed to lead.I deleted the server from AMS, uninstalled/deleted from my source machine, and deleted the IAM user.I then tried it all again, letting AMS recreate the user.I then went to the AMS settings and 'reinitialized' the permissions.Got a little further, but this issue is at least resolved.Still no idea why it happened or what the error even meant.Share0I am having the same issue, trying to migrate on-prem windows 2016 and  this syncing won't start I see EC2 instance started and intializing, then it shuts-down and gets terminated. I get this error "Data replication stalled. Failed to authenticate with service"I deleted the source server, I created new account for migration, and I tried to use global admin, I tried public vs private ip, nothing is passing this error, logs are not really helpful unless I am missing specific log that can help?When I add a server in AMS, I am able to download the installer and use the powershell command-line on the source server it runs with no errors and it completes installing the AWS replication agent.I tried deleting the server from AMS and creating again with no luck.CommentSharesamsalamaanswered 3 months ago"


Question: API Gateway -> Kinesis Mapping Issue

Changing the value of the integration mapping results in bad kinesis input.FollowComment

Answer: "0Accepted Answerthis request works fine for me. I simulated the same settings in my acc and i'm able to successfully the receive the payload in the lambda.Attaching the screenshot of my api request, api gateway mapping template and lambda logs:so it should work with 'payload' key also. Please check if your api request body is correct.API Gateway Mapping Template (Same as your's)API Request Body:Lambda Logs:CommentSharerePost-User-6922853answered 10 months agorePost-User-3647664 10 months agoThank you for your confirmation, this helped. It ended up being the case that the terraform version I was using was unevenly applying/refreshing API gateway resources and this was messing things up.Share"


Question: Secrets Manager PHP caching

Hello.I'm trying to find a good way to cache secrets from Secrets Manager in PHP to avoid calling the SM constantly.Can anyone suggest a good and easy cache for PHP 8 in Elastic Beanstalk.Thank you.FollowComment

Answer: "0Apparently, there are no official examples of caching in PHP. [1] [2][1] Retrieve secrets from AWS Secrets Manager - AWS Secrets Managerhttps://docs.aws.amazon.com/secretsmanager/latest/userguide/retrieving-secrets.html[2] Caching mechanism for PHP - Issue #15 - aws/aws-secretsmanager-caching-pythonhttps://github.com/aws/aws-secretsmanager-caching-python/issues/15I found an example of caching in PHP on Stack Overflow, but I'm not familiar with PHP, so I'm not sure if this will meet your requirements. [3][3] php - F3 Framework: how to make sure that caching is working correctly?https://stackoverflow.com/questions/69310949/f3-framework-how-to-make-sure-that-caching-is-working-correctlyI hope this is helpful.CommentSharemn87answered 8 months ago"


Question: Cluster created with ParallelCluster will not run jobs

"UPDATE: I answered this question for myself. I re-created the AMI but manually (following these docs: https://docs.aws.amazon.com/parallelcluster/latest/ug/pcluster.update-cluster-v3.html#modify-an-aws-parallelcluster-ami )  this time and it worked.Odd because the documentation cautions against this but it worked better than creating the AMI using pcluster.Can't delete the question so here it is for the record.I created a slurm cluster using AWS ParallelCluster (the pcluster tool). Creation works fine and I can ssh to the head node. But when I submit jobs they do not run. Using srun:$ srun echo hello worldsrun: error: Node failure on queue1-dy-t2micro-1srun: Force Terminated job 1srun: error: Job allocation 1 has been revokedUsing sbatch:$ sbatch t.shSubmitted batch job 2$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)    2    queue1     t.sh   ubuntu CF       0:02      1 queue1-dy-t2micro-2Above it looks like it is going to start a job on host queue1-dy-t2micro-2 but that host never comes up, or at least does not stay up, and after a little bit, I see this:$ squeueJOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)    2    queue1     t.sh   ubuntu PD       0:00      1 (BeginTime)And then subsequently, the job is never run.Anyone know what is going on? I did use a custom AMI which I also built with pcluster, but I am not sure if that is the issue, because the head node comes up just fine and it is using the same AMI.FollowComment"

Answer: "0Hi!pcluster build-image should work as well, it would be interesting to understand the reason because it seems that the compute nodes were not able to bootstrap correctly.The logs of the cluster are preserved in CloudWatch Logs for 2 weeks so you could try to check in the cfn-init and cloud-init* log files of one of the failed compute instances (more info in Integration with Amazon CloudWatch Logs doc).The next time you're facing issues and you have a running cluster, it is possible to investigate the root cause of the bootstrap failure by following the troubleshooting guide and checking the logs of one of the failing instances.Anyway glad to know that you've been able to find a working solution.EnricoCommentShareenrico-awsanswered a year ago"


Question: Deactivate or delete access keys for root user

"In the IAM console I see the message:Deactivate or delete access keys for root userDeactivate or delete the access keys for the root user. Instead, use access keys attached to an IAM user to improve security.I'm concerned that if I do this I'll be unable to log into my account.FollowComment"

Answer: "1As a security best practice, it is not recommended for the root user to have an access key. Per the AWS documentation https://docs.aws.amazon.com/accounts/latest/reference/credentials-access-keys-best-practices.html,One of the best ways to protect your account is to not have access keys for your AWS account root user. Unless you must have root user access keys (which is rare), it is best not to generate them. Instead, the recommended best practice is to create one or more AWS Identity and Access Management (IAM) users. Grant those IAM users the necessary permissions, and use them for everyday interaction with AWS.You'll still have access to your AWS account by using an IAM user. And, if you really need to access the AWS console using your root user, you can still do so by using your root user credentials (email and password). As a security best practice as well, ensure that you have an MFA configured for your root user credentials.CommentSharejoahnaanswered 10 months agoEXPERTkentradreviewed 10 months ago"


Question: How do you use the local shadow service from a connected device in GG v2?

"How do I interact with local shadows from a connected device? I see information about configuring the Shadow component to sync various IoT Things' shadows to a core device, and I can see that from a GG v2 component running on that same core I would be able to access the shadow, but I'm hoping that there's a way to have a connected device interact with its local shadow. Thanks!Edited by: jjvic on Sep 22, 2021 11:21 AMFollowComment"

Answer: "0Hi,Thank you very much for using Greengrass V2. The integration between client devices and shadow is being worked on right now. It will be available to use in a couple of months.Until then, we advise the customers to create a new GG v2 component which will act as a bridge between the Client device and Shadow manager. This new component can convert the local MQTT message into the shadow operation using the GG v2 SDK. When deploying the new component, the customer should add some configuration in the MQTT bridge to translate the local MQTT messages to local PubSub messages (on which the new component is listening on). The customers should also add the specific configurations in MQTT bridge to translate the local PubSub messages to local MQTT messages.eg MQTT Bridge configuration:{"mqttTopicMapping": {"ClientDevice1Mapping": {"topic": "$aws/things/MyThingName/shadow/update/accepted","source": "Pubsub","target": "LocalMqtt"},"ClientDevice2Mapping": {"topic": "$aws/things/MyThingName/shadow/update/rejected","source": "Pubsub","target": "LocalMqtt"}}}Cheers,NikkhilEdited by: nikkhilataws on Sep 29, 2021 10:36 AMCommentSharenikkhilatawsanswered 2 years ago"


Question: transfer of the domain name

"my domain name will expire in the beginning of June,please let me know how to transfer it to AWS and what is the priceFollowComment"

Answer: "2When you transfer the registration of a domain to Route 53, the transfer fee applied to your AWS account depends on the top-level domain, such as .com or .org. For more information, see Route 53 Pricing.Steps to transfer the domain: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-transfer-to-route-53.htmlCommentShareFranke-AWSanswered a month agoEXPERTkentradreviewed a month ago"