Question: Authoriztion failure when publishing to IoT Core MQTT topic

"Hello,I'm currently running into some trouble setting up an IoT Core MQTT broker. I am able to connect to my broker using my terminal and mosquitto, but when I try to publish a message to any topic, the mosquitto client disconnects and reconnects without being able to publish.I have validated this connect/disconnect behaviour through the $aws/events/presence/# topic and the mosquitto client in debug mode for which I can provide a sample output :Client william_terminal sending CONNECTClient william_terminal received CONNACK (0)HELLOClient william_terminal sending PUBLISH (d0, q0, r0, m1, 'test', ... (5 bytes))Client william_terminal sending CONNECTClient william_terminal received CONNACK (0)Using the AWSIotLogs set at debug level, I was able to find out that this behaviour is caused by an authorization problem happening at publish time. Here are consecutively sampled logs for the stream :{    "timestamp": "2022-09-29 15:16:55.406",    "logLevel": "INFO",    "traceId": "5697ba84-38f7-eefc-08e9-b6dd00096727",    "accountId": "673559919736",    "status": "Success",    "eventType": "Connect",    "protocol": "MQTT",    "clientId": "$GEN/af403525-5e3b-4f81-9888-a31f16e300f0",    "principalId": "49964471e92f354742f5394e648c97d9ac3aa940081cccf0962918bf97fcdf09",    "sourceIp": "10.240.100.18",    "sourcePort": 46898}{    "timestamp": "2022-09-29 15:16:59.554",    "logLevel": "ERROR",    "traceId": "067b15e5-9bcb-5c6d-2061-9bbefbccb3d0",    "accountId": "673559919736",    "status": "Failure",    "eventType": "Publish-In",    "protocol": "MQTT",    "topicName": "sim/2",    "clientId": "$GEN/af403525-5e3b-4f81-9888-a31f16e300f0",    "principalId": "49964471e92f354742f5394e648c97d9ac3aa940081cccf0962918bf97fcdf09",    "sourceIp": "10.240.100.18",    "sourcePort": 46898,    "reason": "AUTHORIZATION_FAILURE",    "details": "Authorization Failure"}The certificates I use to authenticate to my account have the following policy attached :{  "Statement": [    {      "Action": [        "iot:Connect"      ],      "Condition": {        "Bool": {          "iot:Connection.Thing.IsAttached": [            "true"          ]        }      },      "Effect": "Allow",      "Resource": "*"    },    {      "Action": [        "iot:Publish"      ],      "Effect": "Allow",      "Resource": "*"    },    {      "Action": [        "iot:Subscribe"      ],      "Effect": "Allow",      "Resource": "*"    },    {      "Action": [        "iot:Receive"      ],      "Effect": "Allow",      "Resource": "*"    }  ],  "Version": "2012-10-17"}The only restrictive part of this permissions being set on the connection action, I don't understand how it is possible to have a publication authorization failure.I will deeply appreciate any help on this topic.Cheers,William DidierFollowComment"

Answer: "2Accepted AnswerIn AWS IoT, IoT policies are attached to the device certificate and optionally can be inherited from IoT policies attached to the Thing Group. To validate the effective applied IoT policy , best is to work backwards from the device certificate:Specify the device certificate file name to test, confirm the AWS region and AWS Accountexport DEVICE_CRT=dd001.crtexport ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)export AWS_REGION=eu-west-1Compute the sha256 fingerprint using the device certificate file nameexport CERTIFICATE_ID=$(openssl x509 -noout -fingerprint -sha256 -inform pem -in $DEVICE_CRT | cut -c 20- | tr -d : | tr A-F a-f)export PRINCIPAL_ID=arn:aws:iot:${AWS_REGION}:${ACCOUNT_ID}:cert/${CERTIFICATE_ID}Check if the certificate is registered in AWS IoTaws iot describe-certificate --certificate-id $CERTIFICATE_IDFind the Thing attached to this certificate in AWS IoT. The command below takes the first Thing attached to the certificate, if you have more than one Thing attached to your certificate you need to adapt the command below.export THING_NAME=$(aws iot list-principal-things --principal $PRINCIPAL_ID --query 'things[0]' --output text)Retrieve the effective IoT policy for this Thing, so the IoT policies attached to the certificate and the inherited policies from Thing Groups.aws iot get-effective-policies --thing-name $THING_NAME --principal $PRINCIPAL_ID {  "effectivePolicies": [    {      "policyName": "E81FC...",      "policyArn": "arn:aws:iot:eu-west-1:299807768844:policy/E81FCE....",      "policyDocument": "{ \"Version\": \"2012-10-17\", \"Statement\": [{ \"Effect\": \"Allow\", \"Action\":[\"iot:*\"], \"Resource\": [\"*\"] }] }"    }  ]}Verify that you don't have any policies inherited from Thing Groups that deny publishing.CommentShareEXPERTJan_Banswered 8 months agoEXPERTGreg_Breviewed 8 months ago0Hello Jan,Thank you so much for your answer. Indeed, it appears that the policy I gave you did not correspond to the effectivePolicies policy document I retrieved using your method.I can indeed publish to the topics indicated by the policies attached to my certificate.CommentShareWilliam_Didier_Idemiaanswered 8 months ago0If you don't mind, Please you share a document guide to connect AWS IoT Core MQTT by Mosquitto.I have some problem that it maybe like as your problem but my problem is occurred when Mosquitto try to connect to AWS IoT Core.I'm find some way to connect to it and that document may help me for this situation.Thank YouCommentSharerePost-User-1457441answered 4 months ago"


Question: TAO Dashboard deployment failed (table `ta_organizational_view_reports` doesn't exist)

"I'm trying to create a TAO Dashboard following the steps in TAO DASHBOARD DEPLOYMENT - Option 3: CloudFormation Deployment.I'm executing the CloudFormation template from Terraform, and I get the following error:"The following resource(s) failed to create: [taoQuickSightDataSet]. Rollback requested by user." "Resource handler returned message: \"SYNTAX_ERROR: line 3:6: Table awsdatacatalog.athenacurcfn_billing_cur_report_definition.ta_organizational_view_reports does not exist [Execution ID:..."The DB athenacurcfn_billing_cur_report_definition already exists (it was created some time ago, during the deployment of CID & CUDOS dashboards).The table ta_organizational_view_reports does not exist. However, in the lab instructions it's stated:6. Update your Athena Database Name with the name of the CUR Athena Database where you want to deploy table for TA reports. Leave default if you are not sure which database name provideI understand the table will be created by the CloudFormation template (tao.cfn.yml).Is my understanding wrong?FollowComment"

Answer: "2Hello,Currently the TA Orginizational view is only supported in the Manual report generation method, as mentioned in the previous step for Create and uploading  the Trusted Advisor Report under the " Trusted Advisor Organizational View" dropdownPlease attempt to re-deploy your TA Orginizational report by using the manual deployment method.CommentShareSUPPORT ENGINEERTrishen_Canswered 7 months agoEXPERTFabrizio@AWSreviewed 7 months ago1Accepted AnswerProblem turned out to be in the template itself. I've been using the template found in:https://aws-well-architected-labs.s3.us-west-2.amazonaws.com/Cost/Labs/200-cloud-intelligence-dashboards/tao.cfn.ymlAs indicated in lab:https://wellarchitectedlabs.com/cost/200_labs/200_cloud_intelligence/trusted-advisor-dashboards/dashboards/3_deployment/#option-3-cloudformation-deploymentLater I used the template located in GitHub repository:https://github.com/aws-samples/aws-cudos-framework-deployment/Specifically, the release 0.2.3.This template is exactly the same than the other one, but it changes the table name:Old: Name: ta_organizational_view_reports.New: Name: !Join [ '' , [ !FindInMap [ "Definitions", "taoAthenaTable", "Value" ] ,  !Ref taoSuffix ] ].With this template, TAO dashboard was deployed successfully.CommentShareOmaranswered 7 months ago"


Question: Built a dynamic website using Wordpress hosted on a 3-tier architecture

"I created my presentation-tier ( Web layer) with 3 public subnets containing one EC2 each and use an internet facing ELB to distribute traffic to all of them. I also install Apache to all of the instances. The Elb healthcheck is healthyand so far everything is working.On my Application layer, I created 3 private subnets containing one EC2 each and use an internal facing ALB to distribute traffic to all of them. My Alb receives traffic only from my Web-servers, and I installed Wordpress on all 3 of them ( The script to install Wordpress also include Apache and MySQL). The Alb HealthCheck says that healthcheck failed  and the reason being " unhealthy threshold 2 consecutive health check failures". I also created a NAT gateway for these application-servers.I created my dababase on the batabase-layer with its security group that allow traffic only from App-servers through port 3306.From my understanding of a 3-tier architecture, they are all connected to one another through the security group and even the route table. Since I can use session manager to connect to all my Web-servers and App-servers, I would like to believe that my security groups ports are "ok". Here is their flow:INTERNET-->Internet facing ELB-SG-->Web-SG--> Internal facing ALB-SG-->App-SG-->DB-SG. The flow is unsecure using Http (80).1-How do I troubleshoot "Unhealthy threshold 2 consecutive health check failures?"2- How do I built my application so that it will be accessible using only the DNS name of the Internet facing ELB?FollowComment"

Answer: "0What is your healthcheck? You can customise it to be something that you know Wordpress will respond to given a healthy instance. I believe the default check for an ALB is to check that / responds with an HTTP 200 - can you confirm this is the case for your app tier? To validate it:Check the health check configured on your internal ALB from Web->App tierLog into a Web tier instance and use curl to check that the health check resolves correctlyYour architecture will allow you to ensure your site is only accessible via the internet facing ALB. Ensure that the ALB has a public IP, and no instance does, and that your security groups only allow traffic from the appropriate tiers. This is a good reference: https://docs.aws.amazon.com/whitepapers/latest/best-practices-wordpress/reference-architecture.htmlCommentShareCameron Todanswered a year ago0Are you using two load balancers? I can't tell for sure from your post. If so, you only need one (see the link in Cameron's answer). Getting that sorted might help your issues.CommentShareJohn Voldanswered a year ago"


Question: Updating a tag on an EC2 KeyPair resource results in CloudFormation attempting to recreate the resource.

"I created a KeyPair using CloudFormation. As per the documentation, the KeyName property is required, making this a custom named resource.Howeve, the Tags property says that updates require no interruption, so I wouldn't expect the resource to be replaced in that case. The generated change set suggests that will be the case - notice that replacement = False, and details.requiresRecreation is never.[  {    "resourceChange": {      "logicalResourceId": "KeyPair",      "action": "Modify",      "physicalResourceId": "<resource>",      "resourceType": "AWS::EC2::KeyPair",      "replacement": "False",      "moduleInfo": null,      "details": [        {          "target": {            "name": "Tags",            "requiresRecreation": "Never",            "attribute": "Properties"          },          "causingEntity": null,          "evaluation": "Dynamic",          "changeSource": "DirectModification"        },        {          "target": {            "name": "Tags",            "requiresRecreation": "Never",            "attribute": "Properties"          },          "causingEntity": "Version",          "evaluation": "Static",          "changeSource": "ParameterReference"        }      ],      "changeSetId": null,      "scope": [        "Properties"      ]    },    "hookInvocationCount": null,    "type": "Resource"  }]Yet when I make the change above, which is only updating one tag, and redeploy the CloudFormation template, the update fails.CloudFormation cannot update a stack when a custom-named resource requires replacing. Rename <resource> and update the stack again.I'm guessing there is a bug somewhere, but how can I avoid this, as this means that I can't update my CF template.Thanks,DavidFollowComment"


Question: ssh resets on connections to external interface.

"We notices on Monday that ssh connections to our bastion hosts would fail about 80% of the time. If you did connect, your connection would be stable.Running tcptrack on the bastion host, I seet the failed connections are showing as "reset".I've tried patching/upgrading to the latest Ubuntu and SSH versions and the problem has persisted.I have also verified that we are not even getting close to the max number of ports we can have open on the host.I have confirmed this by trying to connect to the bastion hosts eternal interface from an EC2 instance in our AWS network and my home system with a similar failure rate. Connecting to the bastions internal interface from another EC2 instance does not have this problem.Bastion OS: Ubuntu18.04 LTS 5.4.0-1066-aws #69~18.04.1-UbuntuSSH version: OpenSSH7.6p1-4ubuntu0.6Anyone know what changed over the weekend and how to fix this problem?FollowComment"

Answer: "0Accepted AnswerThanks to whom ever fixed the issue. After two days straight of connection problems, this morning the problem magically cleared up.CommentShareAdam Harrisonanswered a year ago"


Question: Migrating Multi-container Docker running on Amazon Linux to ECS on Amazon Linux 2

Hi I'd like to ask if anyone was able to successfully migrate their existing Multi-container Docker running on 64bit Amazon Linux which is about to be deprecated to the newer ECS running on 64bit Amazon Linux 2?I followed the steps outlined in this document: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/migrate-to-ec2-AL2-platform.html using the AWS CLI to upgrade our environment. It did upgrade it to the new platform however I'm getting a 502 Bad Gateway in my webpage.The document says no configuration needs to be changed on the source code as well as in the Dockerrun.aws.json v2 file. Is there any additional configuration I'm missing? When I rolled-back the platform to the previous one (Multi-container Docker running on 64bit Amazon Linux) - it went back to normal.FollowComment


Question: What is allowed in the redirect_uri parameter of "https://signin.aws.amazon.com/oauth?Action=logout&redirect_uri=https://aws.amazon.com"?

"When I'm logged into the AWS Console already and use https://signin.aws.amazon.com/federation to login to the AWS Console with a different account, the error page says "You must first log out before logging into a different AWS account. To logout, click here" and "here" links to https://signin.aws.amazon.com/oauth?Action=logout&redirect_uri=https://aws.amazon.com.My ultimate goal is to be able to logout of one account and login to another in the same request so that I don't have to force human beings to keep micromanaging this themselves.The most promising way I've come up with to achieve this is to set redirect_uri to be a https://signin.aws.amazon.com/federation?Action=login&... URL (properly escaped) but I'm always redirected to https://aws.amazon.com/console.Can someone from AWS please advise on what's allowed in this redirect_uri? Is it related to the Issuer used in the https://signin.aws.amazon.com/federation?Action=login&... request? Something else?Is there a one-step logout-then-login variant of https://signin.aws.amazon.com/federation?Action=login&... I can use?FollowComment"


Question: "if I changed Ec2 type to Spot instances in autoscaling that is already deployed, does EC2 stop service while it is changing?"

"I used ECS with EC2 not fargate and also have autoscalingas we used gpu instances, we are considering to change ec2 type as spot instance.I have question that if i do change instance as spot, the ec2 ,which is belong to autoscaling now, could be changed to spot without stop or service stopFollowComment"

Answer: "0This depends on your setupIf the ASG is using a MixedInstancePolicy, and you're changing the percent from 100% OnDemand to 100% spot (or any other changes), then yes, AutoScaling will proactively rebalance.  You could avoid this by enabling scale-in protection on the instances, but they won't be able to be terminated for any graceful scale-in.  So you'd need to manually manage cycling them out when they're idleIf there's no MixedInstancePolicy, and you're just updating the launch template or launch config to use spot, then no change will immediately happen.  The ASG will just use the new settings when launching new instances in the futureMake sure your application is setup in a way to handle spot.  Spot capacity is best effort (by definition, its EC2s spare capacity).  This means your instances might get terminated with 2 minutes of warning from spot reclaiming them, or you might have times when no spot capacity is available and you'd get launch failures.  To reduce capacity issues, use as many instance types and AZs as possible.CommentShareShahad_Canswered 6 months ago"


Question: Scripting Entity Creation in Sumerian

"Hey folks, I want to script the creation of entities in Sumerian.  I started with:import * as s from 'module://sumerian-common/api/entity';export default function CreationAction(ctx) {ctx.start([ s.entity.CreateEntityAction, { name:"thing", addToWorld:(true) } ]);}This is attached to an existing entity.  However, when I play the scene, nothing seems to happen.  So:Can anyone point out where I am going wrong?Note I have specifically imported from api/entity/ - is this necessary, or will importing from api/ include subfolders such as entity/?Eventually, I want to source the entity to be created from an asset.  How would that change the commands I need?I've been wrestling with this for longer than I care to admit, so any help is massively appreciated! :-)ThanksFollowComment"

Answer: "1Accepted AnswerI've been wrestling with this for longer than I care to admitUnderstandable. Spawning entities programmatically in Sumerian isn't super intuitive. I hope this will help...The CreateEntityAction only creates an empty entity which is probably not very useful to you, especially given your stated goal #3. A more useful option is to use the LoadEntityInstanceAction. This action can be used to create new instances of an entity that already exists in your scene.Side note: There's no way to directly reference assets that exist in the library but not in your scene. In fact, assets that aren't used in your scene get stripped from the final output during the publishing process.Below is an example script that should be pretty close to what you've described. To use it, first add a Box or other entity that you'd like to use as the "template" for your spawned entities. Then, add this script to an Empty entity in your scene (I often use an empty entity that I name "Scripts".) After attaching the script, with the "Scripts" entity still selected, you'll see that a parameter is exposed called "Entity To Spawn". Simply drag-drop your Box entity from the Entities list into the "Entity To Spawn" field. Then run the scene and you should see that the entity gets duplicated and placed in a random position.import * as s from 'module://sumerian-common/api';export const PROPERTIES = {// This property will allow you to specify an existing entity in your scene that you would like to use as the basis// for new entities.entityToSpawn: {type: s.type.Entity}}export default function(ctx, props) {// Uncomment the line below if you would like to hide the template entity.//props.entityToSpawn.hide();// Below we use the LoadEntityInstanceAction which allows us to create a new instance of an entity that exists// in our scene. This is an asynchronous action. So if we want to do anything to that new instance—such as change// its default location—we must wait until the action notifies us that it has completed its work. The mechanism the// action provides for this is the onCreate callback property.ctx.start([s.entity.LoadEntityInstanceAction, { entity: props.entityToSpawn, onCreate: onSpawnedEntityReady}]);// This function will be triggered as soon as the new entity instance has completed its initialization and is ready// for use.function onSpawnedEntityReady(spawnedEntity) {// Here we're just moving the new entity to a random location.const x = Math.random() * 2;const y = Math.random() * 2;const z = Math.random() * 2;spawnedEntity.position.setDirect(x, y, z);}}CommentShareKrisanswered a year agobatfink a year agoFantastic, thanks for a detailed and helpful answer!Share"


Question: ALBRequestCountPerTarget auto-scaling metric not available on an ECS Service

"Hi, I'm trying to set my ECS service to scale up and down based on the ALBRequestCountPerTarget metric, but it's always greyed out. I can set the the average CPU or Memory metric just fine. I've been going over the documentation and the only information I can find on this metric is that it's only applies to spot fleets and ECS Services, which is where I'm trying to use it. There's no other indication I can see as to why it's not available for selection. The service in question is using the EC2 launch type in bridge network mode. It's associated to two target groups currently. Any help would be appreciated. Thanks!FollowComment"

Answer: "0HiThe reason  the "ALBRequestCountPerTarget" option was greyed out was due to not having an application load balancer configured for the ECS cluster service.When configuring the service for the ECS cluster, you will need to specify the Application Load Balancer that will be used to distribute incoming traffic across the tasks running in your service [1].Once the Load balancer has be configured, you will  be able to Configure Service Auto Scaling to adjust your service’s desired count [2], then be able to select the “ALBRequestCountPerTarget” as the ‘ECS service metric’ on the target tracking scaling policy [2].[1] https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-create-loadbalancer-rolling.html[2] https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-configure-auto-scaling.htmlCommentSharerePost-User-9017359answered a year ago"


Question: Does FSx for OpenZFS allow updating SSD storage capacity after creation?

"What the title says. I created a FSx filesystem with OpenZFS and set the SSD storage capacity to 1024 GB. I understand I can add volumes and update volume sizes as long as it all fits under the filesystem's total storage capacity, but is it possible to update the filesystem's total SSD storage capacity?If not, what solutions are there if I need more than the initially provisioned SSD storage capacity?The best idea I can think of is create snapshots of each volume, create a new FSx filesystem with greater SSD storage capacity, create volumes from snapshots and add to the new FSx filesystem.It would be really nice if I could just update the FSx filesystem's total SSD storage capacity in place.FollowComment"

Answer: "0Accepted AnswerThe SSD storage capacity of Amazon FSx for OpenZFS file systems is immutable post file system creation and only supports intra file system origin snapshot transfers. The recommended path to have more SSD storage capacity is to create a new file system with greater SSD storage capacity and copy the files over using standard Linux copy tools, like rsync, fpsync, mcp, etc. Throughput capacity can be updated post file system creation, so make sure both file systems have adequate throughput capacity. The EC2 instance used to copy files from file system to file system should have high network performance (e.g. network optimized instances like c5n, c6gn, etc.) and the Linux kernel should be 5.3 or later. Both file systems should be mounted using NFSv3 using the nconnect=16 mount option. The nconnect mount option is available on Linux kernel 5.3 and later and will establish multiple TCP connections from the EC2 instance to the FSx for OpenZFS file system, allowing for optimal throughput. See the performance section of the FSx for OpenZFS User Guide for more tips on maximizing performance - https://docs.aws.amazon.com/fsx/latest/OpenZFSGuide/performance.html#performance-tips-zfs.CommentShareDarryl_Oanswered a year ago"


Question: How to retrieve path parameters from websocket url in lambda functions

"When we deploy the websocket in API Gateway, it gives us a url like: wss://<example>.amazonaws.com/productionBased on our use case, we need to pass a parameter in the url, like:  wss://<example>.amazonaws.com/production/idNow when we try to connect to the first url, is works fine. But if we connect to the second url, we get a 403 error:Error: Unexpected server response: 403Handshake DetailsRequest URL: https://<example>.amazonaws.com/production/idRequest Method: GETStatus Code: 403 ForbiddenThere seems no way in AWS API Gateway to configure access to the second url.Can anyone offer a solution for this?FollowComment"

Answer: 1Accepted AnswerThis is correct. You can't append anything after the stage name. If you do you get an error.To work around this you can deploy a CloudFront distribution in front of API Gateway and then use a CloudFront Function to modify the URL by removing the ID and placing it in a header or in a request parameter.CommentShareEXPERTUrianswered a year agoRolf Recoy a year agoMany thanks for pointing me in the right direction!Share


Question: Migrate repository from CodeCommit to Gitlab

"Hello,I am using AWS CodeCommit in my current project. And We have a two requirementsWe need to merge branch-a of repository-b(which is in CodeCommit, mirrored from gitlab), to original  repository-b staying in Gitlab.We need to migrate some of the repositories to Gitlab from CodeCommit.Can you please help us with this requirement, on how to achieve this?FollowComment"

Answer: "0You can migrate an existing Git repository to a CodeCommit repository. The procedures in this topic show you how to migrate a project hosted on another Git repository to CodeCommit. As part of this process, you:Complete the initial setup required for CodeCommit.Create a CodeCommit repository.Clone the repository and push it to CodeCommit.View files in the CodeCommit repository.Share the CodeCommit repository with your team.Detailed steps are available at https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-migrate-repository-existing.htmlCommentShareNirmalanswered a month ago"


Question: Detect data types of random files in S3 bucket

I want to detect all different data types in a random file stored in the S3 bucket. Is there an AWS API for this case to get a list of the data types of each column in the unknown file? Is it possible to automatically convert the individual columns to the data types in Redshift?FollowComment

Answer: "0Amazon Textract does not classify text it has read into data types. The API Actions / Data demonstrate how information is returned when you call the API. Information within a page is identified as different block types (e.g. Word, Line Key-Value, Table) as described here: https://docs.aws.amazon.com/textract/latest/dg/API_Types.htmlCommentShareChris_Ranswered 10 months ago"


Question: Event bus: UPDATE_FAILED: Unsupported Operation

"All of a sudden we started to get UPDATE_FAILED errors with "Unsupported Operation" error message for the event bus.We use infra-as-code (CDK) approach and I am tracking the changes, and there were NO infra changes, other than tags changing (e.g. build ID, git sha).I am also doing a cdk diff and it produces no meaningful changes, other than tags.Yet, CloudFormation has been failing for days with the aforementioned message.We most definitely did not make any changes to event bridge (bus) itself or anything surrounding that.Any ideas?FollowCommentMichaelW 8 months agoYou are not alone as this same thing happened with our CDK deployed event bus deployments.  Yesterday when we went to release latest fixes to production it failed with this same "Unsupported Operation" error with no further detail and this is now happening on all updates when tested on multiple previously deployable versions and on multiple AWS accounts.Sharem0ltar 8 months agoThat is super annoying! Thanks for the confirmation. Have you reached out to support yet?SharerePost-User-2378349 8 months agoSame issue on a standard AWS stack, using SAM, any solution / workaround ? It's getting very problematicSharerePost-User-5370515 8 months agoSame issue using SAM: I have to first manually delete my stack every time I need to update it, which is absolutely not feasible on production environments.SharerePost-User-8240684 8 months agoSame issue with CDK, I'm currently unable to deploy anything as my Stack containing the EventBus has a tag with my commit sha, so anything I deploy is seen as a change and is failing.ShareShow 5 more comments"

Answer: "1Accepted AnswerGot a response from the Amazon support team:To begin with I would like to inform you that this is a known issue by our CloudFormation service team, and they are already working on a fix for this issue. The issue is caused by updating stack-level tags(not resource level tags i.e., even without specifically having made changes to the resource) on “AWS::Events::EventBus” resource type. The current workaround is to not make any changes to the stack-level tags used by any stacks containing an “AWS::Events::EventBridge“ resource. This will result in this resource not being updated thus allowing you to resume with update of your stacks without facing this error.Also, we were able to replicate the scenario at our end. We created a stack with an "AWS::Events::EventBus" resource with Resource Level Tags and no error was shown after multiple updates requested. However, when we tried to add Stack Level Tags, we encountered the same error you are facing. Hence, it is confirmed that the issue is with the "Stack level tags".Unfortunately, I’m not able to provide ETA on when this issue might be fixed since AWS Support Engineers are not privy to development road-maps. I would like to apologize for the inconvenience caused.Rest assured, I will keep the case in Pending Amazon Action and I will keep you updated on the resolution of this issue as soon as I have an update from the internal Team.CommentSharem0ltaranswered 8 months agoMichaelW 8 months agoI got the same support response as you and while they haven't yet reported that it is fixed it worked fine for me now and all my production deployments are now completed with no issues.  So it seems their internal team has already made the change to resolve the issue.Share"


Question: Can RDS Custom use an existing CloudTrail?

I would like to reduce costs by allowing RDS Custom to use an existing trail rather than create a new one. Is this possible?FollowComment

Answer: "0Accepted AnswerYes, this is possible. Please speak to an Amazon AWS representative or AWS Solutions Architect for further details.CommentSharelightanswered 4 months ago"


Question: Need help getting an AWS built tutorial pipeline to build

"Hi, I am trying to get the codebuild to work from the following AWS ML Blog post.https://aws.amazon.com/blogs/machine-learning/automate-model-retraining-with-amazon-sagemaker-pipelines-when-drift-is-detected/The article has a link to a cloudformation stack that when clicked, imports correctly into my account. When I follow the steps to run it, all things appear to build. Following the steps in the tutorial, it becomes clear that the necessary sagemaker pipelines that are built as part of the stack failed to build.I reached out to the authors on twitter,  and they noted:"something went stale indeed: CDK dropped support for node v12 sometimes back. Quick and dirty fix: pin the CDK installed version in the CodeBuild ProjectSpec."I navigated around and found that I could force a specific version of CDK in the codebuild buildspec for the failed build of the pipeline, the relevant line being here, changing the npm line from  "commands": [    "npm install aws-cdk,    "npm update",    "python -m pip install -r requirements.txt"  ]to  "commands": [    "npm install aws-cdk@1.5",  # arbitrary number, was going to trial-and-error with version numbers until something worked    "npm update",    "python -m pip install -r requirements.txt"  ]When I attempt to re-run the failed build, I get the below error:Build failed to start Build failed to start. The following error occurred: ArtifactsOverride must be set when using artifacts type CodePipelinesWhen I open the 'Build with Overrides' button and select disable artifacts, the closest option I can find to meeting the above suggestion, the build starts, but still fails, presumably because it is not pulling in necessary artifacts from a source.If there is another way to unstick this build I would be extremely grateful. This tutorial is greatly needed for a project I am working on and I am not very familiar with CodeBuild, but am trying to get to the materials in sagemaker as that is the focus of what I am trying to fix with some time sensitivity.ANY help you can give me would be greatly appreciated. If it is something else that is wrong, please do let me know. Other options the author suggested:"Two possible paths here:** update node to v16, python to 3.10, and then change the project image to standard 6.0 **. Alternative, pin CDK to an older version npm install cdk@x.x.xx . Not sure which version to suggest right now, it might need some trial and error"If I try this suggestion, I have to switch the environment from AL2 to Ubuntu, then look for Standard 6.0. I have to uncheck "Allow AWS CodeBuild to modify this service role so it can be used with this build project", otherwise I get an error of "Role XXX trusts too many services, expected only 1." Unchecking that lets the changes save, but same ArtifactsOverride issue when trying to run the build.Looking for the least friction solution to getting this tutorial to build as it has exactly what I need to finish a project. Please advise and thank you very much!Sample from log with error:ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.awscli 1.25.18 requires botocore==1.27.18, but you have botocore 1.23.54 which is incompatible.awscli 1.25.18 requires s3transfer<0.7.0,>=0.6.0, but you have s3transfer 0.5.2 which is incompatible.Successfully installed amazon-sagemaker-drift-detection-deployment-pipeline-0.0.1 aws-cdk.aws-applicationautoscaling-1.116.0 aws-cdk.aws-autoscaling-common-1.116.0 aws-cdk.aws-cloudwatch-1.116.0 aws-cdk.aws-iam-1.116.0 aws-cdk.aws-sagemaker-1.116.0 aws-cdk.cloud-assembly-schema-1.116.0 aws-cdk.core-1.116.0 aws-cdk.cx-api-1.116.0 aws-cdk.region-info-1.116.0 boto3-1.20.19 botocore-1.23.54 cattrs-22.1.0 constructs-3.4.67 exceptiongroup-1.0.0rc8 jsii-1.64.0 publication-0.0.3 s3transfer-0.5.2 typeguard-2.13.3WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[notice] A new release of pip available: 22.1.2 -> 22.2.2[notice] To update, run: pip install --upgrade pip[Container] 2022/08/13 16:20:07 Phase complete: INSTALL State: SUCCEEDED[Container] 2022/08/13 16:20:07 Phase context status code:  Message: [Container] 2022/08/13 16:20:07 Entering phase PRE_BUILD[Container] 2022/08/13 16:20:07 Phase complete: PRE_BUILD State: SUCCEEDED[Container] 2022/08/13 16:20:07 Phase context status code:  Message: [Container] 2022/08/13 16:20:07 Entering phase BUILD[Container] 2022/08/13 16:20:07 Running command npx cdk synth -o dist --path-metadata falseUnexpected token '?'[Container] 2022/08/13 16:20:07 Command did not exit successfully npx cdk synth -o dist --path-metadata false exit status 1[Container] 2022/08/13 16:20:07 Phase complete: BUILD State: FAILED[Container] 2022/08/13 16:20:07 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: npx cdk synth -o dist --path-metadata false. Reason: exit status 1[Container] 2022/08/13 16:20:07 Entering phase POST_BUILD[Container] 2022/08/13 16:20:07 Phase complete: POST_BUILD State: SUCCEEDED[Container] 2022/08/13 16:20:07 Phase context status code:  Message: [Container] 2022/08/13 16:20:07 Expanding base directory path: dist[Container] 2022/08/13 16:20:07 Assembling file list[Container] 2022/08/13 16:20:07 Expanding dist[Container] 2022/08/13 16:20:07 Skipping invalid file path dist[Container] 2022/08/13 16:20:07 Phase complete: UPLOAD_ARTIFACTS State: FAILED[Container] 2022/08/13 16:20:07 Phase context status code: CLIENT_ERROR Message: no matching base directory path found for dist```FollowComment"

Answer: "1Hey, I had a quick look at trying to go through the tutorial but I hit the same issues as you did 😕 However, I was able track down the Githib repo that the CloudFormation template was generated from: https://github.com/aws-samples/amazon-sagemaker-drift-detection.If you clone that repo, you should be able to deploy the stack using the instructions in BUILD.md. The overall project is built using AWS CDK, so you should be able to find where the older version of node.js is specified, update it, then deploy the stack using the instructions.Hopefully that points you in the right direction at least! I'm sorry I don't have time to figure out exactly how to fix it but hopefully that helps you a little. Let me know how you get on - it seems like a really interesting tutorial so if you can't crack it, I may have another go when I have some more time!!Thanks,Marrick.CommentShareMarrickanswered 10 months agorePost-User-3027927 10 months agoThanks for the pointers! I started hitting some IAM problems that I don't want to add cascading issues to - if you have the chance to try do let me know if it works for you? I made edits to the yaml file in .github/workflows that referred to node v12 (moved it to 16) and python 3.8 to 3.9. Got errors at the cdk bootstrap command though!SharerePost-User-3027927 10 months agoJust tried acting on every single IAM issue that arose, but in the end got to some arcane issues with the stack itself I think, though it's probably me simply not doing it right. Got a lot of these errors: Cannot delete entity, must detach all policies first.  -- I wouldn't even know where to dig for that -- real shame, I very much would have benefited from getting this tutorial working. Let me know if you have any success building it?Share"


Question: Moving to AWS Transit Gateway from VPC Peering

How would an organization that currently uses VPC peering move to Transit Gateway given that the same routes that currently exist between VPC peers would need to be set up for Transit Gateway routing.FollowComment

Answer: "0Accepted AnswerYou can use [Serverless Transit Network Orchestrator] (https://docs.aws.amazon.com/solutions/latest/serverless-transit-network-orchestrator/deployment.html) to deploy hub and spoke model using Transit Gateway. There's an option to add Route to TGW, this will add route to VPC route table associated with the subnet but it will not overwrite existing route. So if you have route going to VPC peering that will not get overwritten.You can deploy this and have default route or wider CIDR range (less specific route) under Default Route to TGW Example if you have 10.1.0.0/24 -- VPC-peer1 and 10.2.0.0/24 -- VPC-peer1in VPC route table, you can have 10.0.0.0/8 under Default Route to TGW. Once the solution is deployed and you have correct route in place, you'll have to manually delete VPC peering route.CommentSharejkapsanswered 3 years ago"


Question: Unable to connect to AWS via RDP

"I started using a free tier AWS instanceI connected via RDP client from Android .Today I cant connect to it from Android or from LaptopI am getting below error1)Remote access to the server  not enabledThe remote computer is turned off3)The remote computer not available on networkError Code : 0x204Extended error code: 0x0Pls help how to move forwardinstance details : ec2-13-126-78-199.ap-south-1.compute.amazonaws.comFollowCommentrePost-User-9012036 a month agoI have a similar error and on free tier AWS instance also that started a month ago. I'm not sure if this post was resolved. Back then, it was working fine. I temporarily disabled the windows firewall as an app installation is getting a proxy server error. I got disconnected and unable to connect since then and getting this error.Remote access to the server is not enabledThe remote computer is turned offThe remote computer is not available on the networkMake sure that the remote computer is turned on and connected to the network and that remote access is enabled.Error code: 0x204Extended error code: 0x0Share"

Answer: "0You could RDP from an Android device previously but not today, from the same device?  Verify everything is exactly the same as it was before on the client and server side.  Check in the AWS console that your instance is still running.  Verify its Security Group has an ingress rule allowing RDP, that its subnet has open ACLs and a default route for 0.0.0.0/0 to the Internet Gateway.You could try briefly running another small instance to help debug, it will cost very little.CommentShareEXPERTskinsmananswered 2 months ago0What's the inbound rule of your security group ? Ensure the port 3389 is open in that rule and the specified  IP range or IP address matches the source of the connection.CommentSharePaulanswered 2 months ago0When you setup the security group, did you select access from anywhere (0.0.0.0/0) or only for your IP? if it's the latter then that will allow access from whatever your IP was on that device at the time the security group was created, which may be different to today (you will get a different IP on cellular as opposed to home or office wifi, and also your home wifi's IP may change periodically depending on your ISP).Check the current IP of your device on whatismyip.com and verify that address is allowed ingress on port 3389.CommentShareRWCanswered 2 months ago"


Question: Unable to purchase prepaid Hits

"Hi,I am new to Mturk and very confused about the process for purchasing prepaid Hits.I was following the process described in the FAQ of the Amazon Mturk page (https://www.mturk.com/help#enable_aws_billing):How do I purchase prepaid HITs on Amazon Mechanical Turk?Follow these steps to purchase prepaid HITs:From your Amazon Mechanical Turk account, go to My Account -> Purchase Prepaid HITs.Enter in the amount you would like to purchase.Select the credit or debit card on file or enter in new credit or debit card information.Confirm your purchase.Note: As a US Requester, you may be prompted to establish a verified Amazon Payments account if you plan to make a purchase above certain amounts. You can create a verified Amazon Payments account at any time here.=========================================First of all, I am NOT ABLE TO find "Purchase Prepaid HITs" on "My Account" page.So,  I tried to establish "a verified Amazon Payments account" as it directs,  and I am in the stage when I encounter "We’re verifying your identity now, and we’ll send you an email when the verification is complete. This can take up to 24 hours. You can’t use your account until we’ve verified your identity."But it has been more than two weeks since I saw that message.What is wrong with my whole process? I really do want to purchase prepaid HITs but I am not able to...FollowComment"

Answer: "0I am having the exact same issue. It's a really bad experience and to add, I was connected to AWS support who hung up on me and then connected me to an empty call center ( was listening to people in the background). It's really disappointing because I'm trying to use Mturk for my Master's thesis.CommentSharerePost-User-0389717answered 8 months agocustomer 6 months agoDid you resolve the issue?Share"


Question: Connect On-premises⇔AWS in the same segment with Direct Connect (DX)

"Is it possible to connect on-premises and AWS with DX in the same segment?example)・192.168.10.0/24(AWS) <->  192.168.10.0/24(On-premises)I searched for DX, but couldn't find the content "This is the answer"FollowComment"

Answer: "0AWS doesn't natively support stretched subnets between on premises and VPC. There are some partner solutions in this space that will allow you to do that in limited circumstances. Note that this will introduce extra cost to you and add complexity to the network.But: I would question why you want to do this. Generally, stretched layer 2 subnets aren't a great idea - it's why routers were invented in the first place. What's the use case?Generally, I see customers ask this question because "I want to migrate into AWS but I don't want to change IP addresses because..." and there are a bunch of reasons that come down to hard-coded IP addresses in applications. In these cases I encourage customers to do the hard yards and fix the root cause of the problem rather than resorting to brittle and (probably) expensive network-based solutions which are just being used as a band-aid.CommentShareEXPERTBrettski-AWSanswered a year ago"


Question: Delete CloudWatch Log Groups using AWS Lambda function.

"Hi AWS, I need to write a lambda function to clean CloudWatch log if it has not been accessed more than certain days (30 days) or it is orphaned.FollowComment"

Answer: "0You will want something like the following. Note that it has a deliberate sleep command in it to try and avoid API throttling. This isn't perfect - I could probably do better by checking the return value from the delete_log_stream call but it will only be an issue if you're deleting many logs.It also checks for a keyword (in this case "keyword") to skip those logs. And it sets the retention for log groups that don't have a retention period set to 7 days.What the middle does is delete logs older than the retention time. They should already be deleted but there are cases where the retention period is set after logs have been created and those logs are retained.This doesn't completely answer your question but it gives you something to start with.import boto3import timelogs = boto3.client('logs')def lambda_handler(event, context):       logGroups = logs.describe_log_groups()['logGroups']    for group in logGroups:       if 'keyword' not in group['logGroupName']: continue       daysRetention = group.get('retentionInDays', 0)       if daysRetention != 7:           logs.put_retention_policy(logGroupName=group['logGroupName'], retentionInDays=7)           continue       maxRetention = time.time()-(daysRetention*86400)       logStream = logs.describe_log_streams(logGroupName=group['logGroupName'])['logStreams']       for stream in logStream:           if (stream['creationTime']/1000) < maxRetention:               print(f'Deleting: {region} {group["logGroupName"]} {stream["logStreamName"]}')               logs.delete_log_stream(logGroupName=group['logGroupName'], logStreamName=stream['logStreamName'])               time.sleep(0.2)CommentShareEXPERTBrettski-AWSanswered 10 months agoUri EXPERT10 months agoI would change the stream['creationTime'] to stream[lastIngestionTime]ShareBrettski-AWS EXPERT10 months ago^^^ What he said. ;-)Share"


Question: Rollback using CodePipeline

We can see the history now in CodePipeline. But what is the best way to rollback to or re-execute a pipeline from the history?FollowComment

Answer: "0Hello,At this time there is no way to perform a rollback or re-execution from the CodePipeline History. While this is an active feature request for this, we recommend watch the What's New Blog[1] or announcements about CodePipeline Feature Releases.[1] https://aws.amazon.com/new/#dev-toolsRegards,MohsinCommentShareAWS-User-8355257answered 5 years ago0Two years on, still no sign of a rollback feature on the horizon. I mean, I would have thought being able to rollback is a fundamental part any pipeline tool; at least, any pipeline tool that wants to be taken seriously. Frankly the absence of this fundamental feature in 2020 still baffles me. What exactly is the thinking behind this?My team was considering switching to CodePipeline and this is a very serious downside for us, and I don’t think we’ll be going ahead with CodePipeline because of the lack of this essential feature.CommentShareGbadeanswered 3 years ago0I totally agree. It's a crucial functionality and it shouldn't be complex to implement by the pipeline team.CommentShareMokusanswered 3 years ago0I literally have been pushing our company to get off Jenkins and onto CodeBuild/CodePipeline.I can't believe there is no rollback.  Why not have a list of artifact (triggers) and allow a redeploy of an older one?  No need to mess with any other aspects of the pipeline, just re-run an older trigger?  Next commit/build still triggers off a deployment as usual.. seems easy to implement.I keep hearing "use code deploy if you want rollbacks" but in our case, we use ECS FARGATE and the only CodeDeploy action is this blue/green, which has another major limitation.  It only allows 1 targetgroup + listener combo.  We have a 443 listener as our gateway, and then an 8080 listener with host header rules to point to different target groups, guess what, it can't handle the rules.  So that basically removes CodeDeploy as an option, which means there is no rollback, and therefore AWS CI/CD tools aren't able to work for us.Crazy.CommentShareaws-nick19392392answered 3 years ago0This is a really surprising gap to run into with CodePipelines. CodeDeploy rollbacks are insufficient, as infra changes like IAM policy updates can cause issues sometimes and need rolling back. We really need to be able to rollback CFN updates. Even if the implementation is a roll-forward of a historical pipeline build artifact, that's fine.CommentSharebilalanswered 2 years ago0This is a bit of a ugly workaround, but what we do the following.CodeCommit repo - used by the CodePipelineBitBucket repo - used by all developers dailyFor deployment, we push all changes from the BitBucket repo into the CodeCommit repo (git mirror). This triggers the CI/CD and deployment goes on as expected.Rollback: we reset the AWS CodeCommit repo (BitBucket repo remains untouched and developers can focus on fixing the cause which lead to the rollback in the first place) to a prev commit which does a new build and deploy but to a previous version of the codeCommentSharechrisjanswered 9 months ago"


Question: Pearson Vue does not have CLF-C01 available to schedule

"I was scheduled to take the AWS Cloud Practitioner test on Thursday afternoon, but decided I want a couple more weeks to shore up my knowledge of the fine details. Well I learned this morning that the cert is no longer being offered by PSI and Pearson will be handling the tests going forward into the new year. So I called Pearson to schedule it and the support staff on the other end said I needed a passcode to take the test and then gave me a number that's in North Carolina that has 2 people working is and they are closed for Christmas/New Years. So I just ended up canceling the test since PSI doesn't offer a way to reschedule it and hope that I can schedule in 2023 on Pearsons Website as they currently don't offer CLF-C01. Really confusing this morning.and to show everyone I'm not crazy here are the available tests they offer when I go through the links.https://imgur.com/a/bgQyVunFollowComment"

Answer: "1The CLF exam will be offered by Pearson VUE. Since you just cancelled with PSI, please give the system 24hrs to regenerate your authorization for this exam, and you will be able to reschedule with VUE.If you have any other issues, please submit a ticket to the AWS Training & Certification customer support team, and they will provide assistance. Please allow at least 48 hrs to get a response.CommentShareNelliT-AWSanswered 5 months ago"


Question: How to get multi-dimensions forecast ?

"Hello,I've imported my raw historical sales data which contains several attributes: item, date, store, day in the week, week in the month, season, and the corresponding demand for each data row. When I run the forecast, I only get the total forecasted demand for 1 item only. Two inquiries here:1- I need to get the forecast for the item for each different store in the specified period.2- I need to get the forecasted data as a full report that can be exported, for example if I have 5 items in 5 stores, I want to get forecast report which contains 25 forecast points.Could you please help in this?FollowComment"

Answer: "0Hi,One possible approach would be to use scikit-hts in SageMaker, as demonstrated in this example:https://github.com/aws-samples/amazon-sagemaker-hierarchical-forecasting/blob/main/hierarchical-forecasting-retail.ipynbYou would need to re-adjust for your data, but in principle it should work.CommentShareDanielanswered a month ago"


Question: Lightsail enable SSL certificate AttachCertificateToDistribution error

"I've set up an nginx Lightsail instance + Lightsail Distribution. The Distribution URL is live and the site loads via the Distribution URL w/ no errors.Then for DNS I'm using Route53 from another AWS account (this is unavoidable due to policy, cannot be moved to the same account as the Lightsail instance). I have an A record pointing to the Lightsail static IP, a www CNAME pointing to the Distribution URL.When creating the certificates in Lightsail, I created one for non-www and www combined. It would not enable under the Lightsail Custom Domain tab (error below). A www-only certificate produced the same error (below). A non-www only certificate enabled just fine. The certs have been verified as having the us-east-1 region w/ awscli ($ aws lightsail get-certificates).Error AttachCertificateToDistribution[us-east-1]Alternative Domain Names [domain.com, www.domain.com] have one or more parameter that is already associated with a different resource.InvalidInputExceptionWith the non-www cert the site loads fine via the non-www domain, however www (http and https) brings up an entirely different site. I expected at least a privacy warning and the ability to view regardless of a valid cert. I've doubled checked the www CNAME and it is pointing to my Lightsail Distribution. The www certificate that is in use for www under Site Information (browser address bar) isn't the certificate created by my Lightsail account.The domain names associated with the SSL/TLS certificate cannot be in use by another distribution across all Amazon Web Services (AWS) accounts, including distributions on the Amazon CloudFront service. You will be able to create the certificate for the domains, but you will not be able to use it with your distribution.Can an existing certificate for this domain created on any other AWS account cause it to still be in use? If so, what happens if I cannot get it removed? I thought new DNS pointing at my Lightsail would avoid resolving to anything other than where it currently points to.FollowComment"


Question: AMI created by Server Migration Service

we had used Server Migration Service in past to migrate On Premise servers to AWS and this activity created many AMI with name Created by SMS can we delete that?FollowComment

Answer: 1Accepted AnswerYes if you do not need to launch any instances from that ami anymore. There is actually a feature released yesterday to see when the ami was lastlaunchedCommentShareEXPERTPeter_Ganswered a year ago


Question: Charts for IOT data

"Hi Guys,I am collecting IOT data during the day from many devices, I am trying to build static and live charts from these data. I need to do multiple calculation for hourly, daily for some period of time and some other time interval, need to calculate lot of average values of data.What is the best way to approach this.Should I just do raw calculation on tables and prepare data for charts, or calculate everything ahead of time? Or any AWS service  I can use for this?I will build chart on UI and need data for them.ThanksFollowComment"

Answer: "1Use IoT Core to trigger a rule action: Invoke Lambda or direct putItem on DynamoDB.Lambda will save data into DynamoDB or Keyspaces (Apache Cassandra).Use API Gateway and Lambda to query your data. If you use Keyspaces as a DB, you can use aggregation functions.CommentShareEddyanswered 2 months ago1For the use case you are describing you can get all the required functionality out-of-the-box by using AWS IoT SiteWise. AWS IoT SiteWise is a fully managed service providing aggregation of the streamed data and visualization components, either via the managed monitor or via the IoT AppKit components that can be embedded in your own application.You can ingest data into Sitewise from IoT Core using IoT Rules, or if you have industrial data sources you can use a SiteWise Gateway.To learn-by-doing you can run this workshopIf you would nevertheless prefer building your own solution, I would suggest the following services:AWS IoT Core for data ingestionAmazon Timestream for data storage and data aggregation. You can run adhoc queries on Timestream so that you do not have to define the aggregations ahead of time. You can also define scheduled query for the most commonly used aggregation to reduce cost.Managed Grafana for data visualization or any charting library if you want to build your own (CloudScape provide pretty nice components)CommentShareEXPERTMassimilianoAWSanswered 2 months ago0Use AWS IoT Core to collect data from your IoT devices.Use AWS IoT Rules Engine to process and forward the data to other AWS services like Amazon Kinesis Data Streams for real-time processing or Amazon S3 for storage.Use Amazon DynamoDB for storing time-series data, which is suitable for IoT data.For data visualization, you can use Amazon QuickSight or a custom web application using popular JavaScript chartinglibraries like Chart.js, D3.js, or Highcharts. You can query the pre-aggregated data from the chosen data store to display it on the charts.CommentShareEXPERTsedat_salmananswered 2 months agoozy cozy 2 months agoThanks for the suggestion, I will use Highcharts. Only challange is, it is hard to prepare data ahead of time. Let say for this want to see average value  between 8 AM to 10 AM or other interval dynamically, so I need to act on raw data. trying to find best way to query data and do average calculation.Share"


Question: DMS -> Opensearch 2 - BULK failed: contains an unknown parameter [_type]

"Hello,I'm trying to configure a Database migration task to copy data from RDS Aurora (MySQL compatible) to Opensearch 2.3.If I enable the properties ParallelLoadThreads & ParallelLoadBufferSize then I have an error and nothing is copied:Elasticsearch: BULK send request failed SourceTable:xxxxxxxxxxxxx TargetIndex:xxxxxxxxxxxxxxx Operation:BULK Failed Completely  DocumentCount:xxxx   ES HttpCode:400 ESErrorResponse: {    "error": {        "root_cause": [            {                "type": "illegal_argument_exception",                "reason": "Action/metadata line [1] contains an unknown parameter [_type]"            }        ],        "type": "illegal_argument_exception",        "reason": "Action/metadata line [1] contains an unknown parameter [_type]"    },    "status": 400}I guess it is due to the new version as the type has been removed.Is there a specific property to enable to solve this issue? I can't find anything in the documentation except the property UseNewMappingType but it only solves the issue with _doc. Or just DMS is not compatible with opensearch 2.3 at the moment?Thanks in advance.Best,BenFollowComment"

Answer: "0I am facing the same issue, DMS 3.4.7 with UseNewMappingType = true, I can import the data in single thread, if enable the multi thread, unknown parameter [_type] error returned.Looks like the DMS still sending the _type to OpenSearch Bulk API, hope they will resolve the issue soon.PhilipCommentSharePhilipanswered 3 months ago"


Question: KVM DataSync Agent partial network connectivity failure

"I am attempting to deploy a KVM DataSync Agent on my CentOS7 host, and I am stuck on the activation step.After the DataSync agent starts it is supplied with an address 192.168.122.19I have configured an AWS VPC (CIDR 10.0.0.0/16) and established a site-to-site VPN tunnel to my on-prem private network.I can ping and ssh from my site to an EC2 instance (10.0.1.252) in my VPC.I can ping from an EC2 instance in my VPC back to various machines in my private network (192.168.1.0/24).I have created a datsync Endpoint in my VPC it was assigned 10.0.1.138 addressWhen I run the "Test Network Connectivity" option from within the DataSync Agent console it partially fails10.0.1.138:443 FAILED10.0.1.138:1024-106410.0.1.138:1026 FAILED10.0.1.138:1027 FAILED10.0.1.138:1029 FAILED54.201.223.107:22 PASSED0.amazon.pool.ntp.org:123 PASSED1.amazon.pool.ntp.org:123 PASSED2.amazon.pool.ntp.org:123 PASSED3.amazon.pool.ntp.org:123 PASSEDAny suggestions on what I might have configured incorrectly?thanksFollowComment"

Answer: "0Well apparently my Libreswan IPsec VPN tunnel is a bit fragile. It seems to have gone dead at some point between my ping/ssh connectivity tests and configuring my DataSync Agent. I restarted IPsec service (systemctl restart ipsec), and now the Network Connectivity tests complete successfully, and I am able to activate the agent.CommentSharedigital-janitoranswered 2 years ago0Good to hear!  Network issues are most often the cause of activation errors.Regards,JeffAWS DataSyncCommentShareJeff_Banswered 2 years ago"


Question: Persistent Servers - GameLift or EC2 Only? (newb questions)

"Hey folks, please excuse my ignorance because I'm fairly new to the world of dedicated server hosting.I'm working on a game that requires a persistent world. I've managed to get an EC2 instance up and running and I'm able to start my dedicated server process(es) and connect as a client.I'm trying to figure out how to fetch all running dedicated server processes from all EC2 instances (currently just one, but the idea is to create a server list to show the client).Ideally we also want to keep track of the server age and have our servers get "wiped" after X weeks (kind of like Rust).My current understanding is that GameLift is still using EC2 Instances, it's just managing them for you to spin up additional servers to handle the extra traffic. Like there's no special "GameLift server", am I correct?If so, I guess I don't understand the best way to go about it so my questions are:Is GameLift needed for a long term servers or is it specifically for session-based games?If GameLift can be used for persistent servers, what features does it provide that would prove to be useful? (I don't think auto-scaling would be necessary as we'd have a "fixed" amount of official servers - although that might need to change if the game does well so it's not completely out of the question)If sticking with EC2 and not integrating with GameLift is the way to go, is there a way to "fetch" the dedicated processes from an EC2 instance and extract some custom data that we need from them? (i.e. server name, region, age since wipe, next wipe date, current/max players, etc.)Thanks for your time :slight_smile:PS. We're developing our game using Unreal Engine.FollowComment"

Answer: "0First, yes GameLift servers are EC2 servers(which is the backbone of AWS Services) but they are specials server that have been installed with the GameLift Framework. They handle all the server admin, and updates and what not for you, and give you a connection framework so people can connect to the server easier.So GameLift can support long term servers and session-based server and mix. What you would do is a "persistent server" be a fleet on to it self, where you can set the min number of server to 1, and disable auto-scaling.no matter the option that you go with, you should have the server client submit that data to like a database or http end point, because most OS really dont like the idea of reading memory(data) from a process(hackers would have a field day if that was possible).Most of the data that you have comes into two flavors: static data, and changing data. The static data(like server name, region, last wipe time,  max player), the server client can submit when the server client boot up. You can give the server client an id that links to your records about the server(that links to server name, region, max player), then it just tells you when it bootup again(that way your records can calculate the next boot time. This solution pushes alot of the info from the server and to your database/record, that way it safe if the server ever get hacked and what not. For the current number of players you can have the server give you that information on like 5 mins base.CommentSharerePost-User-3292008answered 2 years ago0Another thing to bear in mind is that all compute is fallible.  You do need to plan for instances that go bad due to memory, thread leaks etc, network failures that prevent communication or instances that require reboots due to security patching etc.So you should have a plan for instance migration of your simulation/persistent data so players get the experience you want.My two cents on your questionsYou could use GameLift for a persistent worlds but I would make sure you are getting value add out of it. Are there other GameLift features that make sense to you ie player latency matching, Flexmatch etc? Note that GameLift is not going to be able to transfer your world data or players if you need to migrate an instance (in GameLift terms they would just start a new session on a new instance).Scaling/Flexmatch could still help you if you are grouping x players per server. What happens when more players come to your game? Do you need to spin up new worlds for them, are they on slices of the same world instance? What happens when your player population falls?Theres lots of ways of achieving this. You could do things via ServiceManager/SSM https://aws.amazon.com/systems-manager/ and step functions (if you want a poll model/query model). As @REDACTEDUSERCommentSharerePost-User-5332620answered 2 years ago0[quote="Bry, post:1, topic:10034"]PS. We’re developing our game using Unreal Engine[/quote]To allow your server client to talk to your backend services, I would recommend VaRest pluging, and wrap the calls into a server only module since your game client wont need this code. This Plugin allows make rest calls from UE4 and parse json responseCommentSharerePost-User-3292008answered 2 years ago0This is Advice i got from game tech solutions Architect.The gamelift is for session based if you are hosting matchmaking, or games like Call of Duty,If you want to host games like GTA or Minecraft , you would use EC2.Both are able to be scaled, and you can allow people to use the same image to host on AWS with them being charged (For if you want users to host lobbies as well).If you want more help, feel free to reach me in the Lumberyard Discord, :)CommentSharerePost-User-1354821answered 2 years ago"


Question: Change the drive letter of the default disk before migration

"What if an existing instance on another cloud platform contains the same drive letter as the default/temp disk of the AWS virtual machine? Before migrating, is it possible to modify the default drive? What will occur if we fail to update the drive letter?FollowComment"

Answer: "1Yes, it is possible to modify the default drive letter before migrating to AWS. If you fail to update the drive letter, the existing instance may not be able to access the data on the AWS virtual machine's default or temp disk. This can lead to data loss or corruption.CommentShareDivyam Chauhananswered 4 months agoJD 4 months agoHi @Divyam Chauhan, how about we are migrating servers using MGN service, do we need to change the default/temp disk of aws vm?Share"


Question: Exclude resources from AWS Config Managed rules?

"I've been searching for a way to exclude resources from AWS Config managed rules for backup plans but so far to no avail.I'm governing multiple accounts and to see if they have backup plan on resources i have attached the "resources protected by backup plan" rules that do exist (e.g., https://docs.aws.amazon.com/config/latest/developerguide/aurora-resources-protected-by-backup-plan.html).The rules will be in NON-COMPLIANT state if they have no backup plan attached, but some resources we do not want to backup. The rule is good to have if we add resources but accidently forget to attach a backup plan- but I would also like the account owners to have the possibility to actively "allow-list" a resource so the rule can be COMPLIANT.There is two parameters for these rules: "resourceTags" and "resourceId", but with these it's only possible to include resources for the rule. What I would like is the opposite, something like "excludeResourceTags" and "excludeResourceId". That would make it possible to give the account owners the possibility to keep the rule in COMPLIANT state, but we wouldn't miss backups for resources where we need it.Any suggestions are welcomed!FollowComment"

Answer: 0We are really sorry but excluding resources is currently not possible. I suggest that you reach out to your AWS contact person and raise this demand so that it gets properly tracked.Depending on your development appetite you may want to have a look at the AWS Rules Development Kit for creating a custom config rule based on the examples in the awslabs github repository: https://github.com/awslabs/aws-config-rulesAs you can pass own parameters to your custom config rule you could specify a specific tag name. You can protect this tag via Service Control Policies in AWS Organizations to make sure that not everyone can put their resources on the exclude list.CommentShareEXPERTAndreas Seemuelleranswered a year ago


Question: aws-sdk:sesv2:sendEmail failing with BadRequestException

"Inside a Step Function I am trying to send a simple email and I get the following errorService returned error code BadRequestException (Service: SesV2, Status Code: 400 Request ID: etcIs there something wrong/missing? Could this be a permission problem? I'm testing it with a real email address (not user@mail.com)This is my Parameters JSON"Content": {"Simple": {"Body": {"Text": {"Data": "Body"}},"Subject": {"Data": "Test Email Send"}}},"ReplyToAddresses": ["user@email.com"],"Destination": {"ToAddresses": ["user@email.com"]}FollowComment"


Question: No option to add Lambda functions in Amazon Lex intent under fulfillment

"I select the fulfillment toggle in the intent and as I understand it, a dropdown box should appear with lambda functions. This isn't happening.I have created a Lambda functionI have confirmed my lambda instance and Lex instane are in the same regionI have signed out of the dashboard console and cleared my browsing dataI may be missing somethingFollowComment"

Answer: "0In the fulfillment section, Did you click Advanced options > Use a Lambda function for fulfillment?If you did, then you only enabled the use of lambda function in that Amazon Lex Intent. It doesn't let you choose the Lambda function in that configuration.You must do the following:Click the "Save Intent" button.Click "Build" button.Click "Test" button when Build is done.Click the Gear Icon on the Test chat that popped up and there you will see the Lambda functions.You can also do it via AWS CLI: https://docs.aws.amazon.com/cli/latest/reference/lex-models/put-intent.htmlCommentSharevtjeananswered 2 months ago"


Question: Elastic Beanstalks

How can I vew the console output for a nodejs app using docker on Elastic Beanstalks?ThanksFollowComment

Answer: "0You can view the logs generated by your application in the console through the Logs section of your environment. Once you click on Request Logs, you can download output from various log files. You can also use the eb logs command in the Elastic Beanstalk CLI.https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.logging.htmlIn a Docker environment, you should be able to see any logs sent to stdout in the /var/log/eb-docker/containers/eb-current-app/* files. If your app logs to a file, it might be worth looking into how you can extend the log configuration: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.logging.html#health-logs-extendCommentShareEdanswered a year ago"


Question: How to install BOTO3 - - > Python boto install throws exceptions

"Hello,I am trying to install Python Boto3 using the PIP installer.  I can't seem to get it to install successfully as it is throwing exceptions. See below:C:\WINDOWS\system32>pip install boto3ERROR: Exception:Traceback (most recent call last):File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_internal\cli\base_command.py", line 173, in _mainstatus = self.run(options, args)File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_internal\cli\req_command.py", line 203, in wrapperreturn func(self, options, args)File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_internal\commands\install.py", line 315, in runrequirement_set = resolver.resolve(File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_internal\resolution\resolvelib\resolver.py", line 94, in resolveresult = self._result = resolver.resolve(File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 472, in resolvestate = resolution.resolve(requirements, max_rounds=max_rounds)File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 341, in resolveself._add_to_criteria(self.state.criteria, r, parent=None)File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 172, in _add_to_criteriaif not criterion.candidates:File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_vendor\resolvelib\structs.py", line 151, in boolreturn bool(self._sequence)File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_internal\resolution\resolvelib\found_candidates.py", line 140, in boolreturn any(self)File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_internal\resolution\resolvelib\found_candidates.py", line 128, in <genexpr>return (c for c in iterator if id(c) not in self._incompatible_ids)File "C:\Users\tdrod\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_internal\resolution\resolvelib\found_candidates.py", line 29, in _iter_built..........C:\WINDOWS\system32>aws --versionaws-cli/1.19.48 Python/3.6.0 Windows/10 botocore/1.20.48C:\WINDOWS\system32>python --versionPython 3.10.2Is there anything that can help me to resolve this to get boto3 installed??FollowComment"

Answer: "0Accepted AnswerYou could try the Windows MSI installer. Step by step instructions within AWS docs. You will need a 64-bit version of Windows XP or later with admin rights to run the installer.CommentShareRoBanswered a year agoAWS-User-ChefPrivColl a year agoAfter extensive research, eliminating possible issues, etc, I de-installed the CLI and Python and re-installed both. It appears it was a configuration mis-match somewhere along the line.Share"


Question: Monitron Asset Monitoring

"So when I am adding my own sensors, instead of creating a drop down with different positions it makes two different assets. All other that were installed have a drop down feature and the two I'm trying to add do not. I wanted to be able to correct this before I continue with the sensor installations.FollowComment"


Question: Redshift Spectrum giving IERR_READ_ERROR (Access Denied) error

"I'm trying to use Redshift Spectrum to query gz JSON data in S3. I've used a Glue crawler to create the database and table schema which all appears to be correct. I've created the external schema usingcreate external schema spectrum from data catalog ...which populates my SQL IDE with the schema from Glue. But when I go to query data from this table, I get the following error:S3 Query Exception (Fetch). Task failed due to an internal error. Error: HTTP response error code: 403 Message: AccessDenied Access Deniedx-amz-request-id: xxxx-amz-id-2: xxxIon/JSON Stream error while opening buffer: IERR_READ_ERROR (s3://<bucket>/.../<file...>...The S3 data is in a separate AWS account from the Redshift cluster, but I have the S3 bucket policy set up to allow the spectrum-role access based on this link: https://docs.aws.amazon.com/redshift/latest/dg/c-spectrum-iam-policies.htmlI've given the spectrum-role full get and list access:{    "Sid": "Redshift Spectrum permissions",    "Effect": "Allow",    "Principal": {        "AWS": "arn:aws:iam::<account>:role/spectrum-role"    },    "Action": [        "s3:Get*",        "s3:List*"    ],    "Resource": [        "arn:aws:s3:::<bucket>",        "arn:aws:s3:::<bucket>/*"    ]}This thread https://forums.aws.amazon.com/thread.jspa?threadID=280879 suggests that IERR_READ_ERROR is always malformed gz JSON, but seeing "AccessDenied" in the error message makes me question that.Can anyone help me diagnose what the problem is?FollowComment"

Answer: "0The Redshift AWS account did not have permissions for spectrum-role to read from the cross-account AWS S3 bucket (the glue role did, but not the spectrum-role). Adding Get** and List** permissions to the spectrum-role allowed the query to complete.CommentSharekwhiteanswered 3 years ago"


Question: AWS dns_probe_finished_nxdomain - aws dns out of sync for external host

"Hello,For some time now, a "dns_probe_finished_nxdomain" type error has magically appeared when it worked fine before. After several days I was able to temporarily fix the problem by passing my DNS settings from the external host I use instead of the AWS DNS. However I would like to reuse AWS DNS as before.To give some information, I have an EC2 instance with different S3 services...I had put parameters in the DNS Route 53 towards a subdomain. But the host was not AWS but another host in which there are the DNS settings of my domain name.When the error appeared my DNS for the domain name was working fine, the problem came from my subdomains and therefore from the AWS DNS.If you have any suggestions for me that could help me explain why it suddenly crashed and fixed it, thank you :)FollowComment"


Question: I'm Unable to use my VPS machine

"Hi Everyone, I've mentioned lines of error that may help you better understand my issue.I'm trying to use RDP and while connecting to Remote Desktop it gives the following error that I'm unable to get into my Machine:Remote desktop cannot connect to the computer because of the following reasons:1: remote access to the server is not enabled2: the remote computer is turned off3: the remote computer is not available on the networkPlease help me in troubleshooting the error so that I can continue using servicesFollowComment"

Answer: "1you need to check the followingsecurity group  --> RDP port --> 3389 ALLOWinstance must be in public subnet (subnet connected to internet gateway(you need to check the route table of subnet))instance must have public IP or elastic IP associatedthis are the possible reasons,thank you,chaitanya.CommentSharechaitanyaanswered a year ago"


Question: SES Domain verification: DKIM-Signature header returns empty key. Missing required Public key ("p" tag).

"As the title says.  One, out of the three DKIM CNAME records provided by the Easy DKIM setup agent does not resolve after 72 hours.I'm utilizing https://dmarcian.com/dkim-inspector/ tool to query.  On the third CNAME Name the query consistently returns an empty value, where it should resolve to the public key,  aka 'p' tag.As such, it appears that the SES Domain verification process has not completed, however it has not failed.  It says in the dashboard, "Verification Pending" The domain is registered and administered via google domains.  I've verified there are no misplaced characters, etc in my domains DNS config.My initial assumption is that keys may be rotated, and thus at any given time one key may intentionally return a blank. However, I could not find documentation to support my theory.   And it's been 4 days, and I'm getting the same result on this particular CNAME query.Anyone have any ideas?Thanks in advance!FollowComment"

Answer: "0Accepted AnswerAnswering my own question.The DKIM tag was a red herring.  I wasn't able to figure out why the verification failed.  But I reinitiated the process, and removed the CNAME records from the DNS on my domain.  After re-entering all the details, the verification was done in under 30 minutes.  I'm not certain what caused the first attempt to hang.  But re-doing it solved the issue.CommentSharejnilssonanswered a year ago"


Question: rekognition JobId ResourceNotFoundException

"Hello,I used the NodeJS AWS SDK v3 to create Face Detection Jobs on AWS Recognition.I created a bunch of jobs a week ago.Now when I try to fetch the faces found in the job, I am getting a JobId ResourceNotFoundException.What I tried before posting this post :I made sure that the JobId is correctI tried every region of AWSMy feeling is that jobs expire over time. Is this a thing ?Or are job available for ever ?I hope some of you had the same problems :)FollowComment"

Answer: "0Hello,Amazon Rekognition retains the results for a video analysis operation for 7 days. You will not be able to retrieve the analysis results after this time. Once a job is complete, you can get the output of video analysis using Get* API and store it in a data store such as S3 bucket for later usage. More details on the Get operation can be found here - https://docs.aws.amazon.com/rekognition/latest/dg/api-video.html#api-video-getThanks!CommentShareGowthamAtAWSanswered 2 years ago"


Question: ShadowManager deployment failed

"Hi,When I am deploying ShadowManager , I am getting the following error2022-10-25T08:06:25.546Z [INFO] (aws.greengrass.ShadowManager-lifecycle) com.aws.greengrass.shadowmanager.ShadowManager: service-set-state. {serviceName=aws.greengrass.ShadowManager, currentState=ERRORED, newState=NEW}2022-10-25T08:06:25.619Z [ERROR] (pool-2-thread-18) com.aws.greengrass.shadowmanager.ShadowManager: service-errored. {serviceName=aws.greengrass.ShadowManager, currentState=NEW}java.lang.IllegalStateException: File corrupted in chunk 1401, expected check value 1854, got 1850 [1.4.200/6]at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:950)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.Chunk.readBufferForPage(Chunk.java:379)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.readBufferForPage(MVStore.java:1214)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.readPage(MVStore.java:2209)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVMap.readPage(MVMap.java:672)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVMap.readOrCreateRootPage(MVMap.java:688)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVMap.setRootPos(MVMap.java:682)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.openMap(MVStore.java:576)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.openMap(MVStore.java:535)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.tx.TransactionStore.openMap(TransactionStore.java:534)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.tx.Transaction.openMap(Transaction.java:544)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.db.MVPrimaryIndex.<init>(MVPrimaryIndex.java:62)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.db.MVTable.<init>(MVTable.java:120)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.db.MVTableEngine$Store.createTable(MVTableEngine.java:253)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.db.MVTableEngine.createTable(MVTableEngine.java:127)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.db.MVTableEngine.createTable(MVTableEngine.java:41)at com.aws.greengrass.shadowmanager.lib.org.h2.schema.Schema.createTable(Schema.java:697)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Database.open(Database.java:741)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Database.openDatabase(Database.java:307)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Database.<init>(Database.java:301)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Engine.openSession(Engine.java:74)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Engine.openSession(Engine.java:192)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Engine.createSessionAndValidate(Engine.java:171)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Engine.createSession(Engine.java:166)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Engine.createSession(Engine.java:29)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:340)at com.aws.greengrass.shadowmanager.lib.org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:173)at com.aws.greengrass.shadowmanager.lib.org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:152)at com.aws.greengrass.shadowmanager.lib.org.h2.Driver.connect(Driver.java:69)at com.aws.greengrass.shadowmanager.lib.org.h2.jdbcx.JdbcDataSource.getJdbcConnection(JdbcDataSource.java:189)at com.aws.greengrass.shadowmanager.lib.org.h2.jdbcx.JdbcDataSource.getConnection(JdbcDataSource.java:160)at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:80)at org.flywaydb.core.Flyway.execute(Flyway.java:453)at org.flywaydb.core.Flyway.migrate(Flyway.java:158)at com.aws.greengrass.shadowmanager.ShadowManagerDatabase.install(ShadowManagerDatabase.java:74)at com.aws.greengrass.shadowmanager.ShadowManager.install(ShadowManager.java:215)at com.aws.greengrass.lifecyclemanager.Lifecycle.lambda$handleCurrentStateNew$5(Lifecycle.java:441)at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)at java.base/java.lang.Thread.run(Thread.java:829)Suppressed: com.aws.greengrass.shadowmanager.lib.org.h2.message.DbException: (Message 90028 not found) [90028-200]at com.aws.greengrass.shadowmanager.lib.org.h2.message.DbException.get(DbException.java:194)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.db.MVTableEngine$Store.close(MVTableEngine.java:421)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Database.closeOpenFilesAndUnlock(Database.java:1545)at com.aws.greengrass.shadowmanager.lib.org.h2.engine.Database.openDatabase(Database.java:329)... 24 moreCaused by: com.aws.greengrass.shadowmanager.lib.org.h2.jdbc.JdbcSQLNonTransientException: (Message 90028 not found) [90028-200]at com.aws.greengrass.shadowmanager.lib.org.h2.message.DbException.getJdbcSQLException(DbException.java:505)at com.aws.greengrass.shadowmanager.lib.org.h2.message.DbException.getJdbcSQLException(DbException.java:429)... 28 moreCaused by: java.lang.IllegalStateException: File corrupted in chunk 1401, expected check value 1854, got 1850 [1.4.200/6]at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:950)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.Chunk.readBufferForPage(Chunk.java:379)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.readBufferForPage(MVStore.java:1214)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.readPage(MVStore.java:2209)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVMap.readPage(MVMap.java:672)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVMap.readOrCreateRootPage(MVMap.java:688)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVMap.setRootPos(MVMap.java:682)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.openMap(MVStore.java:576)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStore.openMap(MVStore.java:535)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStoreTool.compact(MVStoreTool.java:559)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStoreTool.compact(MVStoreTool.java:506)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.MVStoreTool.compact(MVStoreTool.java:449)at com.aws.greengrass.shadowmanager.lib.org.h2.mvstore.db.MVTableEngine$Store.close(MVTableEngine.java:410)... 26 more2022-10-25T08:06:25.625Z [INFO] (aws.greengrass.ShadowManager-lifecycle) com.aws.greengrass.shadowmanager.ShadowManager: service-set-state. {serviceName=aws.greengrass.ShadowManager, currentState=NEW, newState=BROKEN}my device ：Linux raspberrypi 4.4.26-v7+ #1 SMP Sat Oct 22 21:06:04 HKT 2016 armv7l GNU/Linuxjava version ：openjdk version "11.0.17" 2022-10-18 LTSOpenJDK Runtime Environment Zulu11.60+19-CA (build 11.0.17+8-LTS)OpenJDK Client VM Zulu11.60+19-CA (build 11.0.17+8-LTS, mixed mode)FollowCommentochoanel 7 months agoHey rePost-User-3854128,In order for use to triage this could you please let us know if this was the first time you were installing the component or had you previously installed it?. Does rebooting the greengrass service resolves it, or it just keeps happening after that?Share"

Answer: "0Hello,I understand, you are getting an error when trying to deploy ShadowManager component. Could you kindly confirm if this was the first time you were installing the component or was it previously installed and whether re-booting the Greengrass service resolves it, or not?And in order to investigate this issue further, we would need to take a look at the Greengrass logs generated. You can create a technical case using AWS Support console and share with us all the logs generated under /greengrass/v2/logs directory[1].References: [1] https://docs.aws.amazon.com/greengrass/v2/developerguide/monitor-logs.html#access-local-logsCommentSharerePost-User-6150495answered 7 months ago"


Question: Calling on-prem Web API from AWS Lambda

"I have an AWS lambda function which needs to talk to an on-prem .NET Web API in our corporate network. Although it is not the right approach, I should do this until the on-prem API can be moved to the cloud.What AWS Services that I need to use to create a connection channel so the on-Prem Web API can be called from the AWS Lambda? Is there an example to show how this is done?FollowComment"

Answer: "1Accepted AnswerI'm assuming that you want to keep your on premises API private rather than expose it to the Internet. If so:You'll need to create a link between a VPC and your on premises network. The default VPC is fine but you probably want to create a new VPC with a specific IP address range that doesn't overlap with your on premises network.You can link the two together using the managed VPN service or Direct Connect. It's quicker and easier to set up a VPN (less expensive too!); you can switch to Direct Connect later if you require more consistent bandwidth and latency.Then, configure your Lambda function to run within the context of the VPC which will allow it to access resources in that VPC and on premises via the VPN (or Direct Connect) that you have.CommentShareEXPERTBrettski-AWSanswered a year ago"


Question: Use cases and resources for media-insights-on-aws

Source: https://github.com/aws-solutions/media-insights-on-awsTrying to get my head wrapped around what the MIE application's main function is.Is it strictly a framework for describing ETL jobs or does it also do the ETL jobs?Does it deploy newly isolated stacks for different ETL jobs? Or is everything maintained in the same stackIs there any resource/tutorial of it being used in actionFollowComment


Question: Reuse Snowflake connections in AWS Lambda?

Jeremy Daly talks about reusing RDS connections in AWS Lambda. I assume this principle of establishing the connection outside of the Lambda handler's context would work for Snowflake connections too?FollowComment

Answer: "0Hi there,The general advice for any SDK clients and database connections would be to initialize them outside of the function handler, and cache static assets locally in the /tmp directory. Subsequent invocations processed by the same instance of your function can reuse these resources. This saves cost by reducing function run time.This would also apply to third-party database tools such as Snowflake.Should you wish to learn more about how Lambda may re-use it's containers, please see [1].Thank you,Ahmad[1] https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/CommentShareAhmad Tariqanswered 3 months ago"


Question: IVS Player 1.14.0 - Uncaught SyntaxError

"Hi,I'm working on integrating Amazon's IVS Player with VideoJS, using React. Following the documentation here.Ran into the error below, so I've taken the sample code found here and tried to run it in my current application, but getting the same error.Error Message:When clicking into the error, it points to the below:Here is my code:// Amazon IVS Playerimport {  registerIVSTech,  registerIVSQualityPlugin,  VideoJSEvents,  VideoJSIVSTech,  VideoJSQualityPlugin,} from "amazon-ivs-player";// @ts-ignoreimport wasmBinaryPath from "amazon-ivs-player/dist/assets/amazon-ivs-wasmworker.min.wasm";import wasmWorkerPath from "amazon-ivs-player/dist/assets/amazon-ivs-wasmworker.min.js";registerIVSTech(videojs, {     wasmWorker: createAbsolutePath(wasmWorkerPath),     wasmBinary: createAbsolutePath(wasmBinaryPath),});registerIVSQualityPlugin(videojs);const livePlayer = videojs(          videoRef.current,          {            techOrder: ["AmazonIVS"],            autoplay: true,            poster: poster,            controls: false,            fill: true,            muted: false,            responsive: true,            html5: {              nativeAudioTracks: true,              nativeVideoTracks: true,              nativeTextTracks: true,            },            controlBar: {              volumePanel: {                inline: true,              },            },          },          function () {            console.warn("Player is ready to use");          }) as videojs.Player & VideoJSIVSTech & VideoJSQualityPlugin;livePlayer.enableIVSQualityPlugin();const events: VideoJSEvents = livePlayer.getIVSEvents();const ivsPlayer = livePlayer.getIVSPlayer();ivsPlayer.addEventListener(events.PlayerState.PLAYING, () => {     console.log("IVS Player is playing");});livePlayer.src([{     src: "https://vn-platform-media-converter.s3.amazonaws.com/4723ca8f-306f-4515-b9a4-105e7192a92f-Production/CMAF/4723ca8f-306f-4515-b9a4-105e7192a92f-Production.m3u8",     type: "application/x-mpegURL",     },]);FollowComment"


Question: How do I share Amazon S3 files with users who don't have AWS Management Console access?

"I want to share an HTML file in Amazon S3 with users in our organization who don't have AWS Management Console access. What's the best way to do that and have the file display on users web browsers? I've tried using a presigned URL to share the HTML file, but the URL generated by AWS Lambda downloads the file rather than displays it in the user's browser.FollowComment"

Answer: "0Accepted AnswerTo have an Amazon S3 object appear in a user's web browser without the user needing to have AWS Management Console access, configure a static website on Amazon S3.For a tutorial, see Tutorial: Configuring a static website on Amazon S3 in the Amazon S3 user guide.CommentShareMODERATORAWS-User-8855869answered 3 years ago0For easy S3 file/folder sharing and collaboration with external users using advanced SSO, please check NirvaShare - https://nirvashare.comHere the external users need not have access to aws console.  NirvaShare can be easily integrated with external identity providers such as AWS SSO, Active Directory, Salesforce, etc from another organizations.CommentShareNirvaShareanswered a year ago"


Question: Queue solution to slow down speed rate of messages

"Hello forum,I have a solutions architect question that need to be solved with some queuing strategy.An external party sends us bursts of HTTP POST messages in a very quick rate of 15,000 per second. Our backend is not able to process all those messages at that speed. We need to receive them at 10 per second.What would be the best option to slow down the speed of those messages?Thank you for any feedback.FollowComment"

Answer: "1Hi, the expected answer is most likely an SQS queue, but if you can only process 10 per second those 15,000-message bursts better not happen more often than every 1,500 seconds!  Also SQS messages are limited to 256KB each.  And if processing order needs to be preserved or duplicates removed, it would need to be a FIFO SQS queue not a standard one with "at least once" delivery.CommentShareEXPERTskinsmananswered 4 months agoEXPERTkentradreviewed 4 months agorePost-User-0583042 4 months agoThank you for your answer. No, those bursts don't happen more than once an hour and each message is only a few bytes.Share0The answer was actually easier than I originally thought. We just configured the receiving NGINX server to send 429 (Too many requests) after X messages/s.CommentSharerePost-User-0583042answered 4 months ago"


Question: aurora_oom_response value is empty

"HelloFollowing the aws link https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_Troubleshooting.htmlwe decided to enable the aurora_oom_response parameter with the value print, tune, decline, kill_query in our DB clusters' parameter groups but we have noticed that the value appears as empty when using the mysql client:mysql> show global variables like '%oom%';+---------------------+-------+| Variable_name       | Value |+---------------------+-------+| aurora_oom_response |       |+---------------------+-------+1 row in set (0.00 sec)mysql> We even tried rebooting the instances but the value still shows as empty in mysql client although is clearly setup in the right Parameter groups (cluster and instance).Is this a bug in Aurora? the engine used is 5.7 version 2.11Thanks!FollowComment"