Question: IoT Device Defender Metrics

"Hi guys,is it possible to access the generated metrics, specified in "additionalMetricsToRetain" without an security profile? I need the metric of "sent messages per device/thing" for a dashboard tool. It would be a waste to code it by myself if the metric is technically there. Even the aws console has an api to retrieve the data. Isn't that public or accessible via an sdk?Best,PatrickFollowComment"

Answer: "0Hi Patrick,The generated metrics in 'additionalMetricstoRetain' in Device Defender are not accessible today via API.You can create a security profile with only additional metrics to retain and observe metric trending over time per device in the AWS IoT console meanwhile.If you need the metrics for other visualization/dashboarding purpose outside of the console, you will need to implement it with custom functions.Hope this helps clarify.  Let me know if you have any other questions. ~CathyCommentSharelucathAWSanswered 3 years ago0Hi Patrick, AWS IoT Device Defender recently launched the ListMetricValues API. You can use this API to fetch the metrics that Device Defender collects and stores. Here is the documentation of the API: https://docs.aws.amazon.com/iot/latest/apireference/API_ListMetricValues.htmlCommentShareswarangaanswered a year ago"


Question: Lightsail - container "Took too long" to deploy all of a sudden (Nothing in logs)

"Hello,yesterday I was able to publish a new container into one of our Lightsails with no issues. Today when I try, the deployment fails, and the only thing visible in the logs says;[16/Dec/2021:13:49:54] [deployment:13] Creating your deployment[16/Dec/2021:13:54:28] [deployment:13] Took too longI have tried multiple of lightsails, and no matter what I try to commit, this happens. I can't debug it further since the log doesnt say anything more than this. Im trying to deploy a container with two images (php + nginx), and both images logs says the same. I have disabled the lightsails and enabled them again and they just revert back to the last working image - which is exactly the same as the one I try to deploy (Except just a minor php echo in the index.php-file which can't have any reason why it's not working..)What can I do - since the logs are so poor?FollowCommentoakleaf a year agoRegion: eu-north-1I can't even roll back to a previous working deployment - same issue..ShareJan Naumi a year agoI am having the same issue trying to Deploy a spring-boot app docker image.Set the timeout to 120 via CLI, deployment states to be finished after a few seconds, but finally fails after 120 secs with:deployment:12 took too long.How did you solve it?ShareAWS-User-3421211 a year agoHi, have any progress? I'm facing the same problem and recommendation in answer definitely can't work.Share"

Answer: "2There are two things worth trying:Adjust the logging on the containers you're trying to deploy. The logs that you see on Lightsail are the same as the logs from the containers themselves (Lightsail just adds a couple logs about deployment status), so if you're not getting enough information, you need to log more from your containers.Adjust your healthcheck settings. Some containers take longer than the default settings to become healthy. You can adjust your healthcheck settings when you create a deployment. https://docs.aws.amazon.com/lightsail/2016-11-28/api-reference/API_ContainerServiceHealthCheckConfig.htmlCommentShareAlexanswered a year agoAWS-User-3421211 a year agoI'm facing the same problem. Even redeploying of already worked container service causes the "Took too long". It has nothing to do with logging or health check settings because a service/container simply does not run.Sharevanduclddev 4 months agoAdjust healthcheck settings does the trick, In my case, I am using 1 GB RAM, 0.5 vCPUs with BlitzJS application , timeoutSeconds:60 , intervalSeconds: 120,Share0I went through this problem, your application is taking a long time to start, in my case it is a large application in spring boot. To solve it you have to increase the processing of your cluster, I was increased and testing, when I reached that 1 GB RAM, 0.5 vCPUs worked.CommentShareMauro Albranswered a year ago0I resolved this issue by adding CMD command inside my lauch script. Since my container was running php-fpm I added this code at the end of launch scriptset -- php-fpm "$@"exec "$@"CommentSharesurya_aliveanswered a year ago"


Question: Adding SecurID (hard token) MFA to root account

Is there a way to add a RSA SecurID hard token MFA to the root account? I've tried going through documentation with no luck. Whenever I try to add it I get the following error and have ensured everything is typed in correctly.FollowComment

Answer: "0HiAs per the IAM guide IAM/MFA the only hard tokens supported are Thales, as such RSA will not work.Only options are to go with software token or purchase a new token.Hope this helps.CommentShareRobin Fordanswered 5 months ago0Please cross-check steps with Enabling a hardware TOTP token (console).Additional resource that may help with a solution for RSA as third-party SAML solution.Specifically, Simplify Identity Access and Assurance Decisions on AWS with RSA SecurID and Session TagsCommentShareAWS-User-2870192answered 5 months ago0RSA SecurID token will not work, you will require FIDO certified security keys as mentioned in AWS blog. RSA ID Plus: DS100 Authenticator is FIDO certified as mentioned in FIDO certified product list.CommentShareronald_lanswered 5 months ago"


Question: Unable to troubleshoot github connector with amazon kendra as no logs are printed

Usecase: Created kendra index with github connector.Values:host: https://api.github.comOrganization: open-universities-australiagh_token: supplied from secrets managerSecret name: AmazonKendra-GitHub-indexKey name: githubTokenIAM Role: create new optionRepo: AllScan files: AllProblem:Index fails and no logs are displayed in log groupFollowComment

Answer: "0To answer your question, we require details that are non-public information. Please open a support case with AWS using the following link : https://console.aws.amazon.com/support/home#/case/createAlso, please verify if all requirements are met as per below documents. See : https://docs.aws.amazon.com/kendra/latest/dg/data-source-github.htmlCommentShareSUPPORT ENGINEERAWS-User-4315904answered 9 months ago"


Question: [AWS-SDK] Not receiving any transactional SMS

"Hello, I'm using the AWS-SDK in NodeJs to send Transactionnal SMS to usersI'm using the '2010-03-31' api version and with the method setSMSAttributes, I'm setting the DefaultSMSType and DefaultSenderID.I then publish a SMS with a Message and PhoneNumberI get the RequestId of the SMS but I'm not receiving the SMSMy SNS SMS Dashboard describes a flat activity at 0 outbound messages.Is there something I've done wrong ?Edited by: thomasdupre on Jan 16, 2019 6:48 AMFollowComment"

Answer: "0Ok this was my fault, I setted the region in the SDK configuration at eu-west-1 but SMS service is only available in us-east-1CommentSharethomasdupreanswered 4 years ago"


Question: HTTPS slow

"Solved. (DOS.ru)Edited by: T1m on Dec 5, 2019 6:34 PMFollowComment"

Answer: 0blacklisted an IPCommentShareT1manswered 3 years ago


Question: How to use a domain registered in AWS Account A for AWS GovCloud Account B?

I have a validated domain name registered with Route 53 on a commercial AWS environment. How can I use that domain name inside an AWS-GovCloud environment? Is this even possible?FollowComment

Answer: 0It is doable. See:Setting Up Amazon Route 53 with Your AWS GovCloud (US) ResourcesSetting Up Amazon Route 53 Zone Apex Support with an AWS GovCloud (US) Elastic Load Balancing Load BalancerCommentShareEXPERTkentradanswered 2 months ago


Question: Enabling lifecycle rules on a huge S3 bucket

"Hello all, I have a huge S3 bucket with almost 7 Billion objects and 270 Tb of data. This bucket receives almost 2 million objects per day, and the object size has an average of 37 kb. Enabling Intelligent tiering here is a no-go because it's really expensive scanning objects. I'm thinking in enabling Infrequent Access but I don't know if doing this could lead to high costs just for enabling it on this size of a bucket. Any advice on this would be helpful, thanks.FollowComment"

Answer: "1Accepted AnswerBefore choosing how you'll transition this, Take a look at S3 Storage Lens: https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage_lens_basics_metrics_recommendations.htmlThis will better help you to understand what your usage is like so you don't transition a bunch of objects and then have to immediate incur costs to retrieve them. Once you have a good understanding of usage pattern on buckets, then I'd implement lifecycle policies. https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.htmlCommentShareEXPERTRob_Hanswered 9 months ago1Leverage a Lambda with scheduled Cron to run the S3 Delete API for the objects. Also, follow the best practice documents for S3 with respect to Costs as below and maybe that can reduce the cost.Your use-case & data is best known by you. You can enable lifecycle rules/Intelligent Tiering approach if you feel the data in the Bucket is for the same.Best Practices- https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.htmlCost optimization- https://aws.amazon.com/s3/cost-optimization/A lot can be done to reduce the costs and optimize the bucket, however it is best to also reach out to AWS Technical account manager/AWS Support  who can closely work with you on saving the costs.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 9 months ago"


Question: Aws Lmabda snapstart option not available in ap-south-1(Mumbai) region

"Hi,Recently i have started using snapstart for java lambda functions, its really great.Today when deploying my functions to Mumbai region, i noticed there is no option for snapstart in mumbai region, i deployed the same code to us-west-2(oregon) and I can see snapstart option in oregon and i can enable it too.I am checking documentation, searching if snapstart is not available in some location/data center but could not find any information related to it.At this stage i am stuck and cant deploy my app to mumbai region which is my prod env as well.So my question isIs snapstart supposed to be available in mumbai region if yes then do i need to do anything so that i can see the option of snapstart?Is snapstart NOT supposed to be available in mumbai region if yes then when is it going to be available.?ThanksFollowComment"

Answer: "2Accepted AnswerHi, on release last November (see https://aws.amazon.com/blogs/aws/new-accelerate-your-lambda-functions-with-lambda-snapstart/) it included these regions: US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Singapore, Sydney, Tokyo), and Europe (Frankfurt, Ireland, Stockholm).  I don't think there's been any additional regions announced since.  AWS never provide a timeframe for future releases but often the progressive rollout across regions follows a similar pattern so you could guess a likely release timeframe based on what you've seen before.CommentShareEXPERTskinsmananswered 5 months agoRavi 5 months agoThanks Skinsman for the clarification, i totally missed it. I hope they will bring it in the mumbai(and other) region(s) soon.Share1As per the updates - AWS Lambda now supports SnapStart for Java functions in 6 additional regions.AWS Lambda now supports SnapStart for Java functions in 6 additional AWS Regions: Asia Pacific (Mumbai), Asia Pacific (Seoul), Canada (Central), Europe (London), South America (São Paulo), US West (N. California).CommentShareG V Navinanswered a month ago"


Question: Control Tower update to Landing Zone 3.0 causes failure in Security Hub AWS Foundational Security Best Practices rule Config.1

"I have an organization that's updating its accounts to Control Tower Landing Zone 3.0. As we do so, we're finding that the upgraded accounts fail Security Hub AWS Foundational Security Best Practices rule Config.1 "AWS Config should be enabled". The failure appears to be caused by a change to Config where global resource recording only happens in the home Control Tower region. The Config.1 failures we see are in secondary regions, and we confirmed that the failing accounts don't have global resource recording active in the secondary regions.My question is: is there a plan to update the Security Hub rule to reflect the Control Tower change? Control Tower has it right, we only need to record global resources in one region. It's also very annoying to undo the change in Landing Zone 3.0 as we have to move accounts out of CT-managed OUs or log in as the CT role to change Config.FollowComment"

Answer: "1Accepted AnswerI have been seeing this issue as well. At re:Invent this year I had many discussions around this and am working with an SA to demonstrate the problem.The SH Check Lags behind Control Tower protect that setting on Config in all regions that are not your primary/home.The alternative I am looking at currently is to globally disable the check with a description using this solution:https://github.com/aws-samples/aws-security-hub-cross-account-controls-disablerLet me know if you have any questions on that. I have successfully deployed it and testing CIS checks currently.CommentShareRoss Wickmananswered 6 months ago0Thanks, good to know that I'm not seeing things. The global enabler/disabler solution is interesting but I wish the SH team would make this a feature of delegated management.CommentShareJames Scottanswered 6 months ago"


Question: Alternative for decryption of Administrator password for EC2 Windows Server instance

"I have the .pem file with the RSA key but after launching the instance, when I click "Get password" It gets stuck in the following message.Password is not available.Please wait at least 4 minutes after launching an instance before trying to retrieve the auto-generated password.Is there an alternative for retrieving the password?Thank you in advance.FollowComment"

Answer: "0So you've waited a while and the instance is definitely healthy, and you still get that message?  Have you tried using another connection method, e.g. Session Manager (cmd line), EC2 Instance Connect (SSH) or Systems Manager Fleet Manager (RDP)?CommentShareEXPERTskinsmananswered 5 months agonoxuz 5 months agoHello, Yes the instance is healthy and the message never updates, i am trying to connect via RDP, I have not tried other methods, yet curiously this happens to any new instance I create, even if I make a new key pair.Share0Does the principal you're using to access the console have ec2:GetPasswordData permission?Is EC2Config enabled on the instance?See https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/common-messages.html#password-not-ready for more info.CommentShareEXPERTskinsmananswered 5 months ago"


Question: Auto index for workdocs

Is it possible to create an auto index for Workdocs?FollowComment


Question: ECS EC2 Instance is not register to target group

"I create a ECS service using EC2 instances, then i create an Application Load Balancer and a target group, my docker image the task definition its using follow configuration:{  "ipcMode": null,  "executionRoleArn": null,  "containerDefinitions": [    {      "dnsSearchDomains": null,      "environmentFiles": null,      "logConfiguration": {        "logDriver": "awslogs",        "secretOptions": null,        "options": {          "awslogs-group": "/ecs/onestapp-task-prod",          "awslogs-region": "us-east-2",          "awslogs-stream-prefix": "ecs"        }      },      "entryPoint": null,      "portMappings": [        {          "hostPort": 0,          "protocol": "tcp",          "containerPort": 80        }      ],      "cpu": 0,      "resourceRequirements": null,      "ulimits": null,      "dnsServers": null,      "mountPoints": [],      "workingDirectory": null,      "secrets": null,      "dockerSecurityOptions": null,      "memory": null,      "memoryReservation": 512,      "volumesFrom": [],      "stopTimeout": null,      "image": "637960118793.dkr.ecr.us-east-2.amazonaws.com/onestapp-repository-prod:5ea9baa2a6165a91c97aee3c037b593f708b33e7",      "startTimeout": null,      "firelensConfiguration": null,      "dependsOn": null,      "disableNetworking": null,      "interactive": null,      "healthCheck": null,      "essential": true,      "links": null,      "hostname": null,      "extraHosts": null,      "pseudoTerminal": null,      "user": null,      "readonlyRootFilesystem": false,      "dockerLabels": null,      "systemControls": null,      "privileged": null,      "name": "onestapp-container-prod"    }  ],  "placementConstraints": [],  "memory": "1024",  "taskRoleArn": null,  "compatibilities": [    "EXTERNAL",    "EC2"  ],  "taskDefinitionArn": "arn:aws:ecs:us-east-2:637960118793:task-definition/onestapp-task-prod:25",  "networkMode": null,  "runtimePlatform": null,  "cpu": "1024",  "revision": 25,  "status": "ACTIVE",  "inferenceAccelerators": null,  "proxyConfiguration": null,  "volumes": []}The service its using ALB and its using same Target Group as ALB, my task its running, and i can access using public ip from instance, but the target group does not have registered my tasks.FollowComment"

Answer: "0Thank you for reaching out to AWS!To fix your issue, ensure you currently have your chosen target group.You will then want to associate your target group with your application load balancer.In order to register Amazon ECS tasks, you will want to create a Cluster in the ECS console.In the configure service section, confirm you have at least one task createdAnd to use a load balancer, you must include "containerPort" in your definitionThen in the Load Balancing Section, select Application Load Balancer and the Listener port used by your applicationFinally, select the existing target group you've createdAfter creation, view your tasks and ensure they are RUNNINGThese instructions can be found here:https://aws.amazon.com/premiumsupport/knowledge-center/create-alb-auto-register/I hope this helps and if you have any more questions, please feel free to reach out!CommentShareCade_Kanswered 10 months ago"


Question: Redshift keeps lock on table after query completed

"Hi!I have the following situation:My teammate ran a query from Python, and received the data into DataFrame. So the query had been completed, but he didn't close the connection.I've tried to rename that table (using DBeaver client) and the query was running too long. When I checked the pg_locks table I found that those table was locked by my colleague.I have tested with queries from Python on my side using 2 different modules psycopg2 and pyodbc, and found that tables are locked in both cases after the query completed, until the connection is closed by me.Is it normal that Redshift keeps lock on table after the query had been completed?FollowComment"

Answer: "0Accepted AnswerHi denisaleksandrov,I think you may find this knowledge center article helpful:  https://aws.amazon.com/premiumsupport/knowledge-center/prevent-locks-blocking-queries-redshift/I suspect there is an open transaction or perhaps an open cursor on the session doing the SELECT ... that is still holding an AccessShareLock on the table when your other session it trying to get an AccessExclusiveLock while doing the ALTER TABLE ... RENAME ...I hope this helps,-KurtCommentShareklarsonanswered 4 years ago"


Question: Security Group problem while creating a load balancer

"Hello,I'm just about to end my AWS cloud technical essentials course in the last lab I have a task to create an application load balancer for that I created a new security group names load-balancer-sg, I successfully created but unable to see that security group under the security group tab while creating a load balancer any solution?FollowComment"

Answer: "1Hello,Have you selected the right VPC (in the ALB creation menu) where the security group has been created?CommentShareVincentanswered 3 months agoEXPERTkentradreviewed 3 months ago1Please first check if you are on a same region ( where your ALB in created )  - You can confirm it from top right side of the AWS console. Then make sure you are at the same VPC where you created your load balancer. You can check your VPCs from left side menu on the console or you can search it by name .CommentSharemojtothanswered 3 months agoEXPERTkentradreviewed 3 months ago"


Question: How to configure our own inference.py for two different PyTorch models in MultiDataModel to build single endpoint and call both models from there?

"I have referred this notebook to deploy PyTorch model but as per this notebook they are just calling newly trained model to get predictions by passing input payload... but I want to configure my own inference.py file to pass an input there and get predictions from there...So, can anyone help me achieving that?ThanksFollowComment"

Answer: "0I haven't got an up-to-date example, but tentatively believe this should be possible without having to build a custom container?Generally in PyTorch framework, if your model.tar.gz contains code/inference.py (root of the tarball contains a code subfolder with an inference.py script), this should get picked up... So the approach would be to pack your inference scripts into your model tarballs.Specifically with MME, I haven't tried on the most recent framework versions but last time I tried it out, you needed to also use TorchServe model archiver to package the model ready.This sample creates a MultiDataModel in PyTorch with inference scripts, but is currently pinned at framework version <1.8.1 because of this issue. Hopefully it could still help you get started?CommentShareEXPERTAlex_Tanswered 6 months agoyugaljain 6 months agoError:AttributeError                            Traceback (most recent call last)/tmp/ipykernel_8642/1469107258.py in <cell line: 47>()4546 print(type(mme))---> 47 predictor = mme.deploy(initial_instance_count=1,48                        instance_type='ml.m5.2xlarge',49                        endpoint_name=f'mme-pytorch-{current_time}')~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/multidatamodel.py in deploy(self, initial_instance_count, instance_type, serializer, deserializer, accelerator_type, endpoint_name, tags, kms_key, wait, data_capture_config, **kwargs)240             self.sagemaker_session = local.LocalSession()241--> 242         container_def = self.prepare_container_def(instance_type, accelerator_type=accelerator_type)243         self.sagemaker_session.create_model(244             self.name,~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/multidatamodel.py in prepare_container_def(self, instance_type, accelerator_type, serverless_inference_config)138         # copied over139         if self.model:--> 140             container_definition = self.model.prepare_container_def(instance_type, accelerator_type)141             image_uri = container_definition["Image"]142             environment = container_definition["Environment"]~/anaconda3/envs/pytorch_p38/lib/python3.8/siteShareyugaljain 6 months ago~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/pytorch/model.py in prepare_container_def(self, instance_type, accelerator_type, serverless_inference_config)287                 )288--> 289             region_name = self.sagemaker_session.boto_session.region_name290             deploy_image = self.serving_image_uri(291                 region_name,AttributeError: 'NoneType' object has no attribute 'boto_session'Shareyugaljain 6 months agoHey @Alex_TI just tried as per your instruction but getting following error, so can you please tell me how we can fix this error?My script:import timeimport sagemakerfrom sagemaker.pytorch.model import PyTorchModelfrom sagemaker.multidatamodel import MultiDataModelfrom sagemaker import get_execution_rolesagemaker_session = sagemaker.Session()print(sagemaker_session)role = get_execution_role()BUCKET = 'autofaiss-demo'PREFIX = 'huggingface-models'model_data_prefix = f's3://{BUCKET}/{PREFIX}/mme/'#print(model_data_prefix)output_1 = f's3://{BUCKET}/{PREFIX}/mme/model.tar.gz' # whisper-tinycurrent_time = time.time()#print(output_1)model_1 = PyTorchModel(model_data=output_1, # whisper-tiny                              role=role,                             entry_point='inference.py',                             framework_version='1.8.0',                             py_version="py38")#print(model_1)mme = MultiDataModel(name=f'mme-pytorch-{current_time}',                     model_data_prefix=model_data_prefix,                     model=model_1,                     sagemaker_session=sagemaker_session)#print(type(mme))predictor = mme.deploy(initial_instance_count=1,                       instance_type='ml.m5.2xlarge',Share0Refer to the following link to build your own Multi-Model Endpoint Container:https://docs.aws.amazon.com/sagemaker/latest/dg/build-multi-model-build-container.htmlCommentShareEXPERTAshleyMilleranswered 6 months ago"


Question: "Trying to share an S3 bucket across accounts using 'aws:PrincipalOrgPaths', how to debug?"

"We have several AWS accounts, all arranged into a tree in Organizations:o-ABCDEF / r-1234 / ou-XXXX / ou-YYYY / ou-ZZZZ / ou-<actual_accounts>The intermediate X->Y->Z OUs are just there for, well, organizational purposes.  The "actual accounts" correspond to projects and customers and stuff with a need for isolated resources, billing, yadda yadda.There's also an "actual account" OU at the same level as the ZZZZ branch.  This actual account (call it Central Account) is where we put a lot of our central internal resources:  EKS running websites, S3 buckets holding gobs of data, etc.In the interests of making new accounts a little easier to stand up (along with Account Factory out of the Control Tower service), we wanted to be able to have EC2 instances in the various "actual accounts" download some stuff from one of the S3 buckets in that Central Account.  There's an example given in AWS documentation about using aws:PrincipalOrgPaths as a condition for a bucket policy, so following that example, I came up with"Version": "2012-10-17","Statement": [    {        "Sid": "meaningful human reminder goes here",        "Effect": "Allow",        "Principal": "*",        "Action": [            "s3:GetObject",            "s3:ListBucket",            "s3:a_few_other_Get_and_List_entries but no Put or Delete"        ],        "Resource": [            "arn:aws:s3:::name-of-bucket",            "arn:aws:s3:::name-of-bucket/special-prefix",            "arn:aws:s3:::name-of-bucket/special-prefix/*"        ],        "Condition": {            "ForAnyValue:StringLike": {                "aws:PrincipalOrgPaths": [ "o-ABCDEF/r-1234/*/ou-ZZZZ/*" ]            }        }    }]That's the entire bucket policy.  "Block all public access" is on.  ACLs are disabled.The path in Organizations does have an intermediate wildcard because the AWS documentation had explicitly mentioned it.  I had originally written the intermediate OUs in there and it didn't make a difference.  The trailing wildcard was always present.  This SO question mentions that the PrincipalOrgPaths takes an array even when you're only listing a single entry, but that the AWS Console editor removes the square brackets in such cases.  I've tried it with square brackets and one entry, as well as listing the same path multiple times just so that the square brackets would be preserved; made no difference.Organizations has "Service control policies" enabled, and its FullAWSAccess (AWS managed policy) attached.  I'm not entirely certain if that matters or not.Trying to access s3://name-of-bucket/special-prefix/from an EC2 instance in one of the other accounts in the OU tree gives only Access Denied errors.  I've turned on server logging for the bucket, and the log entries showing my test attempts give the bucket name, originating instance role, the 403 response, etc, but obviously don't mention what Organization OU is involved.I'm not sure what's wrong with the policy, or if there's something else I need to change, or if there's a way to see what test S3 is applying that's failing instead of succeeding.  This should be doable with just Organizations, right?FollowComment"

Answer: "1I don't see anything wrong syntactically, though I am dubious about the multiple wildcards. If you have not tried IAM policy simulator, then I suggest testing with that. You can include a resource policy to help you trouble shoot, see https://aws.amazon.com/blogs/security/verify-resource-based-permissions-using-the-iam-policy-simulator/. Also, although Amazon S3 bucket policies will take spaces in the Sid, I suggest camel case as a general habit because there are policy statements that won't except spaces in Sids.CommentShareAWS-Everettanswered a year agorePostReReadReTry a year agoThank you for answering!  The multiple wildcards (actually, the entire syntax block) is based off this AWS blog post:  https://aws.amazon.com/blogs/security/iam-share-aws-resources-groups-aws-accounts-aws-organizations/   Specifically, the section "Example: Grant S3 bucket access to all principals in an OU in your organization".  Although they didn't mention the square bracket thing, so I've wondered if the rules have changed since that was posted...The actual SID uses underscores, but camel case is probably more conventional.  I'll tweak that up next time I'm there.   :-)And thank you for pointing us at the IAM policy simulator.  Hadn't seen that before, I'll go fool around and see what happens with it.  (I'd really love some kind of insight into which policies/rules get examined in what order -- Organizations, S3 buckets, IAM, etc -- to find out exactly what test is failing.)SharerePostReReadReTry a year agoWell, the IAM simulator shows green as long as I'm simulating a user/role with AdministratorAccess.  :-(  Listing the bucket's ARN for the S3 GetObject, ListBucket, etc actions, and checking the "include resource policy" checkbox, merely shows a "Cannot get the resource policy. Reason: The specified resource was not found" error.  I've added GetBucketPolicy to the bucket policy, with no difference.  (Trying to add a second statement to the policy's Statement array yields no syntax errors, but suddenly claims that I don't have PutBucketPolicy permissions and won't save the policy.  I have administrator access, so...?)So everything works as long as it's done by an IAM admin, but the S3 bucket itself isn't taking effect.  Simulating another user, or an EC2 instance policy, fails for that reason.Share"


Question: [AWS WAF] Can't hit the rule with managed rules included in custom rules

"Hello,I'm testing some rules on AWS WAF. As with the custom rules (such as URI), I can configure them successfully.However, when I add another "label" rule inside this custom rule, it can never hit (count) although I can see the relevant logs.Any advice? Thank you.FollowComment"

Answer: "0Can you perhaps share the rule syntax so that we can understand the logic better? Is it an "AND" or an "OR" condition?CommentShareTzoori Tamamanswered a year agoNam Tran a year agoI tried using "AND" or "OR" condition, or even just applied a single rule. Here are details of the rule:{"Name": "CustomCountRule-NoUserAgentHeader","Priority": 0,"Statement": {"AndStatement": {"Statements": [{"LabelMatchStatement": {"Scope": "LABEL","Key": "awswaf:managed:aws:core-rule-set:NoUserAgent_Header"}},{"NotStatement": {"Statement": {"ByteMatchStatement": {"SearchString": "<redacted>","FieldToMatch": {"UriPath": {}},"TextTransformations": [{"Priority": 0,"Type": "NONE"}],"PositionalConstraint": "CONTAINS"}}}}]}},"Action": {"Count": {}},"VisibilityConfig": {"SampledRequestsEnabled": true,"CloudWatchMetricsEnabled": true,"MetricName": "CustomCountRule-NoUserAgentHeader"}}Share0Similar issue. Extremely basic IP match rule with default BLOCK results in the rule never being hit and all requests blocked with the IP that should be allowed through listed in the logs and in the "sample requests".CommentSharerePost-User-4453523answered 6 months ago"


Question: Back-off restarting failed container error on pods running on fargate not Nodegroup

"Why am I getting Back off restarting failed on AWS-LOAD-BALANCER-CONTROLLER running on Fargate, not Nodegroup?How could this be resolved?FollowComment"

Answer: "0Accepted AnswerHello rePost-User-8469491,While deploying AWS Load Balancer Controller on EKS Fargate, you will have to explicitly specify the VPC ID and Region code as shown in the helm command below. Please refer this RePost article for more info.helm install aws-load-balancer-controller eks/aws-load-balancer-controller \    --set clusterName=YOUR_CLUSTER_NAME \    --set serviceAccount.create=false \    --set region=YOUR_REGION_CODE \    --set vpcId=<VPC_ID> \    --set serviceAccount.name=aws-load-balancer-controller \    -n kube-systemIf you have already done this and still facing issues, check the aws-load-balancer-controller pod logs to find out what is causing the pods to crash.It would also be helpful if you could share the pod describe output: kubectl describe pod <aws-load-balancer-controller-pod> -n kube-systemHope this helps!CommentShareSUPPORT ENGINEERVenkat Penmetsaanswered a month agorePost-User-8469491 a month agoaws-load-balancer-controller:Container ID:  containerd://***Image:         public.ecr.aws/eks/aws-load-balancer-controller:v2.5.1Image ID:      public.ecr.aws/eks/aws-load-balancer-controller@**Ports:         9443/TCP, 8080/TCPHost Ports:    0/TCP, 0/TCPArgs:--cluster-name=Network-EKS-Cluster-V2--ingress-class=alb--aws-region=us-east-1State:          WaitingReason:       CrashLoopBackOffLast State:     TerminatedReason:       ErrorExit Code:    1Started:      Mon, 01 May 2023 10:01:31 -0600Finished:     Mon, 01 May 2023 10:01:38 -0600Ready:          FalseRestart Count:  12SharerePost-User-8469491 a month agoaws-load-balancer-controller:Container ID:  containerd://***Image:         public.ecr.aws/eks/aws-load-balancer-controller:v2.5.1Image ID:      public.ecr.aws/eks/aws-load-balancer-controller@**Ports:         9443/TCP, 8080/TCPHost Ports:    0/TCP, 0/TCPArgs:--cluster-name=Network-EKS-Cluster-V2--ingress-class=alb--aws-region=us-east-1State:          WaitingReason:       CrashLoopBackOffLast State:     TerminatedReason:       ErrorExit Code:    1Started:      Mon, 01 May 2023 10:01:31 -0600Finished:     Mon, 01 May 2023 10:01:38 -0600Ready:          FalseRestart Count:  12Share0I have removed the taints but the still having CrashLoopBackOffCommentSharerePost-User-8469491answered a month agorePost-User-8469491 a month agoI used helm to install the load balancer and which i set the vpcid, is there a reason it is not populating the arg?helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=Network-EKS-Cluster-V2 --set region=us-east-1 --set vpcid=vpc-* --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controllerSharerePost-User-8469491 a month agoI used helm to install the load balancer and which i set the vpcid, is there a reason it is not populating the arg?helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=Network-EKS-Cluster-V2 --set region=us-east-1 --set vpcid=vpc-* --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controllerShareVenkat Penmetsa SUPPORT ENGINEERa month agoThe command has to be --set vpcId=vpc-abcd. From the above comment, I see that you are passing it as vpcid instead of vpcId. Can you fix that and retry?Share"


Question: Can I reuse CloudWatch Log Stream for Batch Array Jobs

Hi I created a Batch with array jobs.I have created a CloudWatch Log Group and Log Stream under it and i wanted to reuse the same CloudWatch LogStream for the Batch Array jobs when it ran.FollowComment


Question: Case ID 6135575121 Retry payment for reserved instance

"Hello,I have payment request failed first time because of limit on my card.I increased the limit but I can't retry payment again, no link Pay now.I have created case 6135575121 and it's still "Unassigned" after 20 hours.This is payment for Reserved Instance (db.t3.small Postgres) for 3 year all upfront. I requested reservation at Jun 3.Can you help me with this?I loose money now waiting for that reservation completion...Thanks!FollowComment"

Answer: 0My card was charge a few minutes ago for reserved instance and everything looks good :)CommentShareMichalZalewskianswered 4 years ago


Question: Do AWS Transfer Family support AIX?

AWS Transfer Family support AIX platformFollowComment

Answer: 0AWS Transfer Family supports the SFTP protocol which isn't tied directly to any one Operating System. The behavior of SFTP within any Operating System would be dependent upon which client you are using.We test/validate a few common clients listed here: https://docs.aws.amazon.com/transfer/latest/userguide/transfer-file.htmlIf you are having issues with a specific client on AIX I would recommend letting us know more details on the client/specific version and what the error might be or reaching out to our Premium Support team for further investigation.CommentShareBrian Canswered a year ago


Question: OpenSearch query with type "term" not returning the same results in Amazon Neptune

"I am using Amazon OpenSearch service version 1.3 alongside Amazon Neptune.I followed Neptune docs for integrating full-text search capabilities in Neptune.I needed to use a custom analyzer for one of the predicates found in Neptune, so instead of letting the stream poller lambda function create the index called amazon_neptune, I created the index manually from Opensearch dashboards.The analyzer is only used on predicates.url.value and it uses path_hierarchy tokenizer.What I want to achieve with this analyzer is that, when searching for a URL like domain.com/path, the query returns all URLs that are included in the hierarchy.E.g: domain.com/path/blog, domain.com/path/blog/1, domain.com/path/aboutThe following query in OpenSearch returns exactly what I described in the above exampleGET amazon_neptune/_search{  "query": {    "term": {      "predicates.url.value": "domain.com/path"    }  }}But, when making a query to Amazon Neptune like below, the resultset is empty:%%gremling.withSideEffect("Neptune#fts.endpoint", "opensearch-endpoint").withSideEffect('Neptune#fts.queryType', 'term').V().has("url", "Neptune#fts domain.com/path")This is the amazon_neptune index used:{  "amazon_neptune" : {    "aliases" : { },    "mappings" : {      "properties" : {        "document_type" : {          "type" : "text",          "fields" : {            "keyword" : {              "type" : "keyword",              "ignore_above" : 256            }          }        },        "entity_id" : {          "type" : "text",          "fields" : {            "keyword" : {              "type" : "keyword",              "ignore_above" : 256            }          }        },        "entity_type" : {          "type" : "text",          "fields" : {            "keyword" : {              "type" : "keyword",              "ignore_above" : 256            }          }        },        "predicates" : {          "properties" : {            "name" : {              "properties" : {                "value" : {                  "type" : "text",                  "fields" : {                    "keyword" : {                      "type" : "keyword",                      "ignore_above" : 256                    }                  }                }              }            },            "url" : {              "properties" : {                "value" : {                  "type" : "text",                  "analyzer" : "path_analyzer"                }              }            }          }        }      }    },    "settings" : {      "index" : {        "number_of_shards" : "1",        "provided_name" : "amazon_neptune",        "max_result_window" : "20000",        "analysis" : {          "analyzer" : {            "path_analyzer" : {              "tokenizer" : "path_hierarchy"            }          }        },        "number_of_replicas" : "1",      }    }  }}FollowComment"


Question: Glue Relationalize recursion / depth

"Does relationalize only go one layer in depth? I have json objects that have 2 to 3 layers of struct arrays and I get this error on anything past the first array layer :An error occurred while calling o136.pyWriteDynamicFrame. Bad dbtable value expecting either schema.table or tablewhat am I missing here?code example (works as expected with objects that have 1 layer of array depth)PostgreSQLtable_node3 = glueContext.create_dynamic_frame.from_catalog(database="tekmetric",table_name="repair_orders")resolvechoice2 = ResolveChoice.apply(frame=PostgreSQLtable_node3, choice="make_struct", transformation_ctx="resolvechoice2")dropnullfields3 = DropNullFields.apply(frame=resolvechoice2, transformation_ctx="datasource0").drop_fields("jobs.parts.dotNumbers")dfc = Relationalize.apply(frame=dropnullfields3, staging_path="s3://tekmetric-temp/repair-orders/", name="repair_orders", transformation_ctx="relationalize4")for df_name in dfc.keys():    m_df = dfc.select(df_name)    if m_df.count():        glueContext.write_dynamic_frame.from_jdbc_conf(frame = m_df, catalog_connection = "postgres", connection_options = {"dbtable": df_name, "database": "tekmetric"})FollowComment"

Answer: "1Fixed this myself - it wasn't an issue with Relationalize per se ... writing to Postgres, the write_dynamic_frame did not enjoy periods in the table names.Fix :for df_name in dfc.keys():    m_df = dfc.select(df_name)    final = df_name.replace('.val.','_')    if m_df.count():        glueContext.write_dynamic_frame.from_jdbc_conf(frame = m_df, catalog_connection = "postgres", connection_options = {"dbtable": final, "database": "tekmetric"})CommentSharewhsmaconanswered a year ago"


Question: What is the process to Signing a AMI ?

"I am trying to figure out  how to sign a AMI image, so that it can be verified that its not tampered with  before creating EC2 instance from the AMI.FollowComment"

Answer: "0AMI binary content (i.e. the disk content) is immutable, so once the AMI is created then that content can't be changed: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/modify-ami.htmlA limited set of attributes of an AMI can be changed after it is created - see the API docs for ModifyImageAttribute for details.CommentShareEXPERTJames_Sanswered 3 days ago"


Question: Update lua scripts on "Play Game" in editor

"I'm using a text editor to edit my lua scripts and pressing ctrl-g in editor to see the changes. Very frequently, the editor does not update the scripts when I change them. Maybe this works if you use the LY Lua IDE, but I don't use that for various reasons. This might also be exacerbated by the fact that some of my scripts use require() - last time I looked into this it seemed LY was doing some funny things that prevented me from even using Lua code to re-load my required scripts.It would be extremely nice if LY reloaded the scripts from file when you press ctrl-g. Maybe make this an option somewhere. As it is now I have to restart the editor every time I make any change to a script in order to see the effects. I'm not good enough with c++ to figure out how to make these changes in LY myself.Follow"

Answer: "0Accepted AnswerSuper valid! I've submitted this to our team for further action :) Thanks @REDACTEDUSERSharerePost-User-5738838answered 5 years ago0Just wanted to add that I've also been dealing with this. Even when using the in-editor Lua IDE, I would need to either remove->add the the root Lua script from the component, or re-open the level.SharerePost-User-5509042answered 5 years ago0Thanks @REDACTEDUSERSharerePost-User-5738838answered 5 years ago0Has this been fixed.  I am also experiencing this quite often.  From time to time my scripts update properly, however the last few days I need to restart the editor every time I make a change.SharerePost-User-7521022answered 3 years ago"


Question: Recent update to Code Deploy now breaking build (we made no changes)

"So we are provisioning a new server for a new project to production, and two days ago we got everything working in production. We have not configured any scripts that cover actual code execution while building our project, but we did add our build script that covers getting a few environment installations in place (update the server, and install a package called yarn)We have not made any adjustments to the build server yet, because we are not ready for it, but today when rolling out a new update our build server suddenly is crashing on a BASIC task of apt-get update -ythis makes absolutely NO sense from a developer stance, something has had to be done on the AWS side to cause this crash.I have attached the full logs and our super basic buildspec.yml file of our buildImageaws/codebuild/standard:4.0Environment typeLinuxCompute3 GB memory, 2 vCPUsPrivilegedFalseService role(not relevant)Timeout1 hour 0 minutesFollowComment"

Answer: "0After further studying of how apt-get update is crashing on the AWS provided container for Nodejs, I saw that a key is expired for yarnI contacted the package maintainer and he responded with the following:Try refreshing your GPG key:curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -so for anyone using the Nodejs container image, please make that your first script before running apt-get updateCommentSharealillandanswered 2 years ago0I found this solution did not work. The use of 'sudo' in that context produced the error "sudo not found". Without the 'sudo' it worked.My BuildProject is created via CloudFormation with the following Environment parameters:"Environment": {"ComputeType": "BUILD_GENERAL1_SMALL","Image": "aws/codebuild/standard:2.0","ImagePullCredentialsType": "CODEBUILD","Type": "LINUX_CONTAINER"}To use the solution posted here I needed to remove the 'sudo' and run the command like this:curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -Coach SpoonCommentShareCoachSpoonanswered 2 years ago"


Question: Describe_log_streams and filter_log_events calls breaks put_log_events

"We have applications that send a fairly large amount of logs into cloudwatch logs.Yesterday, someone was attempting to filter those logs, running potentially many describe_log_streams and filter_log_events call for a fairly small time window but for a time a month in the past.  (time window of 20 minutes or less)All of a sudden, the code that issues put_log_events calls started getting the following errors.400 Bad Request,  ResourceNotFoundException  The specified log stream does not existIt seems the filter and describe_log_streams requests broke the ability to send additional logs.How careful do we need to be when issuing these types of requests?FollowComment"

Answer: 1This was user error.  The error message was accurate.  The log stream had been deleted.CommentShareJackK99answered 4 years ago


Question: how to re-package sagemaker model with lambda?

"I have a sagemaker training process which dumps a model.tar file in a s3 bucket. I don't have access to this training code . I want to download the model.tar file from that s3 location, and repackage it with my own inference code , saved in some s3 bucket. i am aware that there is a way to package this in sagemaker via pipelines but are there any examples to do it in lambda?FollowComment"

Answer: "0This document is in Japanese, does it match what you want to do?https://aws.amazon.com/jp/blogs/news/build-test-and-deploy-your-amazon-sagemaker-inference-models-to-aws-lambda/CommentShareEXPERTRiku_Kobayashianswered 2 months agoclouduser 2 months ago@Riku_Kobayashi - not really, but thanks. i want to be able to download an existing model.tar file , unzip it and then add my own inference script then zip it back and upload it .Share"


Question: Anyone else getting blank password window when streaming MS Office App

When I attempt to stream and MS Office app (in my case Visio) I get the sign in request.  I successfully respond to username request then the password request never comes up.  Just a blank pane where the request would normally appear.  Has anyone else experienced this and can suggest solution?FollowComment

Answer: "0Accepted AnswerThis is a known issue and is pending a fix from Microsoft (no ETA).As a workaround, you can use your fleet in Desktop view and login to your Microsoft apps.CommentShareJeremy_Sanswered 9 months agoPaulv45 9 months agoThanks, Jeremy_S!Indeed reconfigured the fleet for stream view Desktop instead of Application and works like you said it would.You have earned a helping Gold Star -- take to nearest Starbucks along with local equivalent of $6.50 USD and have a latte.Share0Seems browser related.Did you verify the usual suspects?clearing cachecookiestemp filesTry InCognito mode(or a different browser like Firefox if you are using Chrome) to confirm whether it a browser issue.CommentShareArun_PCanswered 9 months ago0Cleared history in Firefox and Chrome -- same problem ---Firefox Private -- same problemCommentSharePaulv45answered 9 months ago"


Question: What is best way to zip large amount of objects from aws s3 (like Tbs of files)

What is best way to zip large amount of objects from aws s3 (like Tbs of files) and upload that zip file to another bucket in the same region to another storage class like Glacier?FollowComment

Answer: "0The best way to copy large numbers of objects to other buckets seems to be S3 Batch Operations. [1] [2][1] Performing large-scale batch operations on Amazon S3 objects - Amazon Simple Storage Servicehttps://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-ops.html[2] Copy objects - Amazon Simple Storage Servicehttps://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-ops-copy-object.htmlHowever, a different approach seems to be needed for how to zip existing objects. [3] [4][3] python - possible to copy s3 file and change its zip format while doing so?https://stackoverflow.com/questions/67355516/possible-to-copy-s3-file-and-change-its-zip-format-while-doing-so[4] amazon web services - Zip up folders as I upload them to AWS Glacier - Stack Overflowhttps://stackoverflow.com/questions/57108755/zip-up-folders-as-i-upload-them-to-aws-glacierHope one of the above fits your requirements.CommentSharemn87answered 8 months agombehrens 4 months agoYou might want to compare speed/time of various compression algorithms that might be available on your OS: bzip2, xz, gzip, etc.  For larger files bzip2 might be better than xz.Share"


Question: RedShift Serverless timeout when connecting with python

"I tried to use redshift_connector to connect to my redshift cluster using python but I get a timeout error..import redshift_connectorconn = redshift_connector.connect(    host="default.XXXXXXXXX.us-east-1.redshift-serverless.amazonaws.com",    database='dev',    access_key_id="XXXXXXXXX",    secret_access_key="XXXXXXX",    port=5439,    region="us-east-1" )Result :redshift_connector.error.InterfaceError: ('communication error', TimeoutError(10060, 1- I am using my default workspace which uses my default VPC which has open inbound rules for IP and ports2- I enabled public access to the workspacewasted 3 hours on this and finaly used google bigquery ...FollowCommentTim Seyfried 9 months agoHave you tracked down exactly when the error is occurring.  Is it during the connecting or is it when running an sql statement.  I have seen this error with a bad sql statement which doesn't make sense but is something I have seen.Share"


Question: IoT MQTT Usage Plan (throttling and quota) for clients

Is there any way create Usage Plan such as ApiGateway for AWS IoT MQTT? I want to control how many messages are sent by clients. Similar question with body validation (structure and size).FollowComment

Answer: "1Hi,at the time being AWS IoT does not provide any functionality to create a usage plans. You can monitor the number of messages sent by clients (either with your own solution or using Device Defender to detect anomalous behaviour) and implement counter-measures, such as disabling the certificates associated to the misbehaving devices.In terms of payload validation, you can perform validation in the AWS IoT Rule via a Lambda function or by checking that expected fields are present in the message and possibly have the right values using the WHERE clause. In case the message is not valid it will be ignored. You can monitor the number of invalid messages by comparing the rule metrics against the rule action metrics, since actions are executed only if the WHERE clause matches, ie the message is valid.Cheers,MassimilianoCommentShareEXPERTMassimilianoAWSanswered 6 months ago"


Question: AWS Greengrass COMPONENT_BROKEN

"I develop a live video detection web app using getUserMedia() to get user's webcam, but when I failed to deploy the component due to the error of COMPONENT_BROKEN.I try to run the same script on the cloud and found that the webcam fail to render. I suspect that the error comes from getUserMedia() which require secure origin. But I have no idea how to solve this. How can I solve this problem?FollowComment"

Answer: "0The fact that the component is status BROKEN in Greengrass is not related to the issue with getUserMedia(). To understand why the component is broken, you can check the component logs that you find in /greengrass/v2/logs/ folder. The log file is named after your component name.I suppose that you are trying to deploy a web application server as a Greengrass compoenent, using something like express.js. As you state, getUserMedia requires a secure context, ie an HTTPS connection. The easiest way to run a web server over HTTPS is to use a self signed certificate. See this blog for an example with express.js. You can also request a public server certificate from https://letsencrypt.org/ for your Greengrass device but for that you will also need to have a domain name.For a more advanced setup, you might want to use NGINX as a reverse-proxy for your web server.https://nginx.org/en/docs/http/configuring_https_servers.htmlhttps://nginx.org/en/docs/beginners_guide.html#proxyCommentShareEXPERTMassimilianoAWSanswered 5 months ago"


Question: Media access restricted to certain FQDN.

is there any way that media (served from cloudfront using signed URL)  can be limited to be accessed by certain FQDN?Solution using API G/W or WAF preferred.FollowComment


Question: Multiple tables in one Data Pipeline?

"I am using Data Pipelines to replicate data from RDS to Redshift with one pipeline per table they are replicating.Is it possible to consolidate the pipelines into a single pipeline for multiple tables? And if so, is there an example JSON config that would make this work?FollowComment"

Answer: "0Accepted AnswerYes, this is possible. Below I've included a demo JSON code for that pipeline.Note that the JSON isn't complete as there are more attributes required for MySql and Redshift - I've included links to the full attribute requirements as well.MySqlDataNode RequirementsRedshiftDataNode Requirements{  "objects": [    {      "output": {        "ref": "RedshiftDataNodeId_BWynX"      },      "input": {        "ref": "MySqlDataNodeId_iOnTY"      },      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultCopyActivity2",      "id": "CopyActivityId_WMYod",      "type": "CopyActivity"    },    {      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultMySqlDataNode3",      "id": "MySqlDataNodeId_9uoCY",      "type": "MySqlDataNode",      "table": ""    },    {      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultRedshiftDataNode1",      "id": "RedshiftDataNodeId_mJUEj",      "type": "RedshiftDataNode",      "tableName": ""    },    {      "output": {        "ref": "RedshiftDataNodeId_yz06G"      },      "input": {        "ref": "MySqlDataNodeId_9uoCY"      },      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultCopyActivity3",      "id": "CopyActivityId_nm5LG",      "type": "CopyActivity"    },    {      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultMySqlDataNode1",      "id": "MySqlDataNodeId_6Xe5h",      "type": "MySqlDataNode",      "table": ""    },    {      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultRedshiftDataNode3",      "id": "RedshiftDataNodeId_yz06G",      "type": "RedshiftDataNode",      "tableName": ""    },    {      "failureAndRerunMode": "CASCADE",      "schedule": {        "ref": "DefaultSchedule"      },      "resourceRole": "DataPipelineDefaultResourceRole",      "role": "DataPipelineDefaultRole",      "pipelineLogUri": "s3://athena-ab3/",      "scheduleType": "cron",      "name": "Default",      "id": "Default"    },    {      "occurrences": "1",      "period": "1 days",      "name": "RunOnce",      "id": "DefaultSchedule",      "type": "Schedule",      "startAt": "FIRST_ACTIVATION_DATE_TIME"    },    {      "output": {        "ref": "RedshiftDataNodeId_mJUEj"      },      "input": {        "ref": "MySqlDataNodeId_6Xe5h"      },      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultCopyActivity1",      "id": "CopyActivityId_G0Lps",      "type": "CopyActivity"    },    {      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultRedshiftDataNode2",      "id": "RedshiftDataNodeId_BWynX",      "type": "RedshiftDataNode",      "tableName": ""    },    {      "schedule": {        "ref": "DefaultSchedule"      },      "name": "DefaultMySqlDataNode2",      "id": "MySqlDataNodeId_iOnTY",      "type": "MySqlDataNode",      "table": ""    }  ],  "parameters": []}CommentShareMichael_Panswered 2 years ago"


Question: Not able to send sms(Text messages) through api to USA region using csharp dotnet

"We are actually trying to send sms(text messages) to US destination{.AmazonSimpleNotificationServiceClient}.  from oregon region (Us west-2), in api call its showing successful  but in aws console it shows failed.But we are  able to send sms to mobile form mumbai region  (APSouth) region  using same api service.so all i got to know is we need an Origination ID to send sms txt messages to US destination.while requesting  in excess limit it show only 10dlc option but not short codes. So even while creating an origination ID also i am facing issue.I have increased the usage limit in oregon region also.Below is the code i am using in apiAmazonSimpleNotificationServiceClient snsClient = new AmazonSimpleNotificationServiceClient("access key", "secret key", Amazon.RegionEndpoint.USWest2);PublishRequest pubRequest = new PublishRequest();// pubRequest.Message = message+ " is the OTP for your multi factor authentication.This code will expire in 10 minutes.";pubRequest.Message = message;pubRequest.PhoneNumber = mobileNo;// add optional MessageAttributes, for example:////pubRequest.MessageAttributes.Add("AWS.SNS.SMS.SenderID", new MessageAttributeValue////{ StringValue = "clicinsight", DataType = "String" });// pubRequest.MessageAttributes AWS.SNS.SMS.SenderID"] = new MessageAttributeValue { StringValue = "...", DataType = "String" };//pubRequest.MessageAttributes AWS.SNS.SMS.MaxPrice"] =// new MessageAttributeValue { StringValue = "0.50", DataType = "Number" };pubRequest.MessageAttributes AWS.SNS.SMS.SMSType"] = new MessageAttributeValue { StringValue = "Transactional", DataType = "String" };PublishResponse pubResponse = snsClient.Publish(pubRequest);FollowComment"

Answer: "1Hello,I can confirm that your code doesn't have problem and the issue you are facing relates to the breaking changes introduced by US telecom providers : "US telecom providers no longer support using person-to-person (P2P) long codes for application-to-person (A2P) communications to US destinations"You need to use one of the following types of origination numbers in-order to send SMS messages US recipients:10DLCToll-free Numbersshort CodeI would recommend re-viewing each of the above origination numbers and its associated charged/costs and see if it fits your use-case and then you can proceed to request the appropriate one.Should you have any follow-up questions related to this reported issue feel free to let us know by clicking the comment button.CommentSharesyumaKanswered a year ago"


Question: cannot connect to a postgresql database instance

I created a postgres database using standard create. I tried this -nc -zv <RDS endpoint> <port number>using my endpoint and port number and it timed out.endpoint was vcs.c2ayp2tfftm5.us-east-2.rds.amazonaws.comport was 5432FollowComment

Answer: "0This could be a number of things, start by verifying the security group that is associated with the RDS instance has port 5432/tcp opened for ingress from your IP address (just for troubleshooting you may want to broaden this to be open from 0.0.0.0/0).Also, verify that the subnet that you have provisioned the database in is accessible from the internet. Amazon Reachability Analyser may be able to help you here https://aws.amazon.com/blogs/database/troubleshoot-network-connectivity-to-amazon-rds-databases-using-vpc-reachability-analyzer/if you're still having trouble after all this, further things to try are listed in these docs https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Troubleshooting.html https://repost.aws/knowledge-center/rds-cannot-connectCommentShareRWCanswered 14 days ago"


Question: Issue with Tomcat container running on Fargate platorm 1.4

"Hi,We've been running an app in a container based on the tomcat:9.0-jdk8-corretto image.  It has been running fine under Fargate with platform 1.3.  But when we changed to platform 1.4, the app starting having issues.The application in the container provides JWT tokens for authentication and access purposes.    I notice in the Tomcat logs at start up under 1.4 that we get a message that we don't with 1.3:INFO  com.mchange.v2.uid.UidUtils - Failed to get local InetAddress for VMID. This is unlikely to matter. At all. We'll add some extra randomness java.net.UnknownHostException: bc7a746844e64dfd95a60014xxxxxxxx-yyyyyyyy: bc7a746844e64dfd95a60014xxxxxxxx-yyyyyyyy: Name or service not known When we try to obtain a token, we see this in the application log:Unexpected error reading requestjava.lang.NoSuchMethodError: io.jsonwebtoken.SignatureAlgorithm.assertValidSigningKey(Ljava/security/Key;)VIf I switch the exact same container back to 1.3, it works as expected.   I've been reading up on the differences between platforms 1.3 and 1.4, but nothing is jumping out as a reason for why we are having this issues.  Curious if anyone else has run into something similar and if there are some ideas for what to try.FollowComment"

Answer: "0Accepted AnswerThe issue turned out to be an older jar being included in the build.   I guess the Docker runtime used in 1.3 handles classpaths different than the containerd runtime in 1.4.CommentShareGreganswered 4 months ago0Probably a good Blog to read:https://aws.amazon.com/blogs/containers/aws-fargate-launches-platform-version-1-4/From here, I suspect that this change has an effect, as before 1.4 it went through a Fargate ENI not controlled by you.  In 1.4, Task ENI inherits your VPC settings.Starting with 1.4.0 that traffic flows through the Task ENI. Task ENIs will inherit the networking connectivity patterns you have enabled in your own VPCs. This is important for customers that, for example, want to have visibility of that specific traffic inside VPC Flow Logs.I suspect now that you have to update your outbound ruleIn terms of network connectivity, now either your VPCs need to allow outbound traffic to reach the same public endpoints or you need to configure Private Links for said services so that your Task ENI can reach the endpoint on your VPC. A practical example is that, previously on Fargate, if you were using private links for ECR, you only needed to set up the ecr.dkr endpoint. With platform version 1.4.0, you also need to set up the api.ecr endpoint.I am not sure if the specific error points to that, but I do suspect folks to look at their permissions since we moved control over to user in 1.4CommentShareRolyanswered 4 months agoGreg 4 months agoThanks for the info.  That post was one of the ones I read in trying to figure out the issue.  I'm nearly certain now the first message about UnknownHostException is a red herring.  We are thinking now there might be something different between Docker and containerd runtimes regarding classpath loading, and we are checking the jars of the build to see if maybe we have an older jar that isn't getting evicted in the build.Share"


Question: SNS notifications "recipients" does not contain the whole recipients list

"Hi,I've set up a SES Configuration Set with Event Notifications directed to SNS.When I receive a notification, I noticed that the "mail.destination" field does not always contains the whole recipients list, even though the docs (https://docs.aws.amazon.com/ses/latest/DeveloperGuide/event-publishing-retrieving-sns-contents.html) say:destinationA list of email addresses that were recipients of the original mail.Which implies, IMO, that all addresses should be present.See this example:{"mail": {"source": "contact@.com","messageId": "0102016ddf1f227e-5f871b08-9e8d-43af-b920-bfa47bfb6f43-000000","timestamp": "2019-10-18T13:47:02.654Z","destination": ["archives@.com"],"commonHeaders": {"to": ["CUSTOMER xxxx.yyyy@gmail.com"],"bcc": ["Archives <archives@.com>"],"date": "Fri, 18 Oct 2019 15:46:43 +0200","from": ["".com" <contact@***.com>"],"subject": "Confirmation de votre commande","messageId": "0102016ddf1f227e-5f871b08-9e8d-43af-b920-bfa47bfb6f43-000000"}},"send": [],"eventType": "Send"}We can clearly see from the headers that the message has a To: and a Bcc: address, but the "mail.destination" field only contains the Bcc address.Similarly, sometimes the commonHeaders field may not contain all the addresses; see for example:{"mail": {"source": "noreply@.lu","messageId": "0102016dd98f16a7-fc5c0bcb-0605-4347-bfcd-8dd7a3fa6faf-000000","timestamp": "2019-10-17T11:51:36.359Z","destination": ["XXX@gmail.com","YYY@yyy.fr","ZZZ@zzz.com"],"commonHeaders": {"cc": [" YYY@yyy.fr"],"to": ["*** XXX@gmail.com"],"date": "Thu, 17 Oct 2019 11:51:35 +0000","from": ["*** <noreply@***.lu>"],"messageId": "0102016dd98f16a7-fc5c0bcb-0605-4347-bfcd-8dd7a3fa6faf-000000"}},"send": [],"eventType": "Send"}This message had 3 recipients:To: XXX@gmail.comCc: YYY@yyy.frBcc: ZZZ@zzz.comHere, the event does contain all recipient addresses in the "mail.destination" field, but neither the "headers" not the "commonHeaders" fields contains the Bcc address.Are these bugs in Amazon SES?Note: I've removed or replaced sensitive information in the events above, but they're otherwise genuine, including the Message IDs if someone from Amazon wants to investigate them.Edited by: Client d'Amazon on Oct 18, 2019 7:37 AMFollowComment"

Answer: "0I finally figured it out.Depending on the recipients of the email, the email you send may be split into several SES messages (in which case, you receive several Message-IDs).When this is the case, each message will have its own, partial list of destinations (recipients).I was mislead by the fact that sometimes, you may have a single message ID, even though you're sending to multiple recipients. This happens when the recipients are on the same email domain (e.g. gmail.com). This also happens if you're sending to multiple recipients using the same email service in the background (e.g. G Suite). So I did not immediately understand that the message was being split into several Message IDs.I still haven't figured out when the Bcc header does or does not show, however.CommentShareBMRKanswered 4 years ago"


Question: CPU Utilization metrics not matching

"Hi,I'm using c5d.2xlarge EC2 instance.When compared CPU metrics from Cloudwatch with top command, they are not matching.Even though the top command shows around 90% of CPU utilization, in the cloudwatch it is showing around 26% only.Could you please help me with thisFollowComment"

Answer: "0Accepted AnswerThe c5d.2xlarge instance type has 8 vCPUs. top will report CPU usage as a percentage of a single CPU. You can press I (shift + i, capital I) to toggle Irix mode, which is explained as:    1. %CPU  --  CPU Usage       The task's share of the elapsed CPU time since the last screen       update, expressed as a percentage of total  CPU  time.   In  a       true  SMP environment, if 'Irix mode' is Off, top will operate       in 'Solaris mode' where a task's cpu usage will be divided  by       the  total  number  of  CPUs.  You toggle 'Irix/Solaris' modes       with the 'I' interactive command.Otherwise, you can press 1 within top to show a break down of all the CPUs. This should give you data that is closer to what CloudWatch shows.CommentShareEXPERTbwhaleyanswered 5 months ago"


Question: Cannot add Elasticsearch Data Source: Request failed with status code 400

"None of the fields are highlighted in red to indicate a problem.I was able to select the region and ES domain.I tried selecting an existing role, which i created by following this AWS tutorial: https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-elasticsearch-resolvers.html. That resulted in another strange error: "Error: Only amazonaws.com domains are supported at this point." My ES domain endpoint does contain amazonaws.com...I opened a support case with AWS support but it's been almost 2 days and nobody has responded yet so I am giving this forum a chance.Has anyone experienced this problem? Any ideas how to resolve it?Thank you in advance.FollowComment"

Answer: "0Turns out AppSync cannot communicate with ES domain in a VPC. I missed that mention in the documentation.A clear error message would have saved me days, though. Hopefully this will help someone.CommentShareabstractfontsanswered 3 years ago"


Question: Account Status: SHUTDOWN?

"Our SES account was seemingly hacked - we have no trace of bad emails being sent by our server. Nothing in mqueue folder, no bounces/complaints, nothing.I've been sending emails to ses-review@amazon.com for over 24 hours but have had no response. This is the latest email I sent attempting to answer the key questions below.What else can/should I do?Thanks for your help,MarkI am trying again to respond to the Shutdown of our SES account. Let me try to address the key questions:  What caused your high complaint rate?  We presume our SMTP account was hacked and we have changed the credentials - none of those bad emails were sent by us. The appropriate configuration files on our server as protected. We have not seen any complaint emails from our server associated with this email send of 24k +. To add, we have not sent any emails from our server since 7/11/19 and the hack seems to have occurred on 7/14/19.  What changes have you made in your email-sending systems or processes?  We have reset the credentials for SMTP. We have validated that no bad emails were sent from our servers. We are not using any non opt-in email addresses. Our bounce rate should revert to normal - historically less than 0.2%. Our complaint rate should revert to our normal - historically less than 0.02%.  How do these changes ensure that the issue won't occur again in the future?    New SES SMTP credentials. New AWS login. Scanned/removed all malware on our server.  New tools to monitor and alert any hacked sending from our servers. Our systems are safe to re-enable our SES service.Please let me know any questions. We really need to get our SES account live again.Edited by: Amazon Customer on Jul 15, 2019 5:41 PMFollowComment"

Answer: "0Hi,I can confirm that the account is Healthy now.Regards,Gaurav @ AWSCommentShareAWS-User-7751712answered 4 years ago0Thank you so much!CommentShareAmazon Customeranswered 4 years ago0Hi:I cannot believe it - this has happened again. We sent an email to ~18k opt-in customers yesterday. 1 complaint. Maybe 10 bounces.Just now I get another warning from SES and see somehow 29000 emails have been sent and the complaint rate is already at 0.69%.Our  server has no queue, nothing is being sent from our email app, sendmail is still shutdown on the server.I just updated the SMTP credentials again.Can I pay for some support for SES to figure out how the heck this is happening?Thanks for your help!CommentShareAmazon Customeranswered 4 years ago0A short while later and 42000 emails have been sent using our SES account. I do not understand - SMTP credentials have not even been copied from SES. Our bounce/complaint email is not getting anything - zero emails. Nothing in our catchall / spam accounts.Is there a way to find out where these emails are being sent from? I do not see anything in the SES Management Console.Any help is welcomed! Thanks.CommentShareAmazon Customeranswered 4 years ago0If someone at AWS support can shut down our SES account for now that would be great. Thanks!Again I think we need paid support to figure this out.CommentShareAmazon Customeranswered 4 years ago0Now almost 54000 emails have been sent and the complaint rate is 0.89%. SMTP credentials have been reset once more. Our apps and server show zero activity in terms of email sends. I have zero ideas about how this is happening but welcome any help. Thank you - I'm looking for help.CommentShareAmazon Customeranswered 4 years ago0Now at 54k emails sent. Complaints are up to 1.01%.Still... Nothing being sent by our server or apps. No complaints to our accounts. SMTP credentials changed and changed - that info is not even on our server and only on AWS.Absolutely out of our control.Any help appreciated. Thanks! Happy to pay for AWS support!CommentShareAmazon Customeranswered 4 years ago0Just sent this email to ses-review@amazon.com...=====What caused your high complaint rate?We presume our SMTP account was hacked and we have changed the credentials twice tonight - none of those bad emails were sent by us. The appropriate configuration files on our server are protected. We have not seen any complaint emails from our server associated with this email send of 54k +. To add, we have not sent any emails from our server since 7/16/19 and the hack seems to have occurred on 7/18/19.What changes have you made in your email-sending systems or processes?We have reset the credentials for SMTP twice tonight. We have validated that no bad emails were sent from our servers. We are not using any non opt-in email addresses. Our bounce rate should revert to normal - historically less than 0.2%. Our complaint rate should revert to our normal - historically less than 0.02%.How do these changes ensure that the issue won't occur again in the future?New SES SMTP credentials. Scanned/removed all malware on our server. New tools to monitor and alert any hacked sending from our servers. Our systems are safe to re-enable our SES service.Please let me know any questions. We really need to get our SES account live again.Happy to have paid support from Amazon to resolve this.CommentShareAmazon Customeranswered 4 years ago0The shutdown happened again.CommentShareAmazon Customeranswered 4 years ago0Opening new thread.CommentShareAmazon Customeranswered 4 years ago0Were you able to resolve this? We are facing similar problem.CommentSharehershey2answered 4 years ago0Have you tried using IP Address Filters as a temporary solution? If it is only a short number of IP addresses or a well-defined set of ranges that you need to send email from, this might fix the problem.I would still want to investigate this though...to figure out how these people are gaining access to your account.CommentSharecazortanswered 4 years ago"


Question: I want to call transcribe streaming as REST API from UI (Angular)

"Hi,Based on this link https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html provided by AWS i got to know that we can call streaming as a REST API from PostMan or from any UI (Angular) to process the audio real time.I tried using this url https://transcribestreaming.us-west-2.amazonaws.com/stream-transcription?x-amzn-transcribe-language-code=en-US&x-amzn-transcribe-sample-rate=16000&x-amzn-transcribe-media-encoding=pcm&x-amzn-transcribe-session-id=435d60c4-3f68-4871-8680-3d100eccd63dand the below json as input to the above url{"AudioStream": {"AudioEvent": {"AudioChunk": ""}}}as the streaming service accepts blob type to process the content real time and i am not sure what type of data can we send to the url.My problem is that when i try to hit the url in PostMan i get the below response{"Output": {"__type": "com.amazon.coral.service#UnknownOperationException"},"Version": "1.0"}What is the exact way of calling the transcribestreaming api form UI (Angular) and what should be the input data we need to send.Is there any possibility to call the transcribe from UI(Angular) itself ??Edited by: vjandhyala on Feb 6, 2019 1:20 AMFollowComment"

Answer: "0Hi did you get the answer for same? Have you resolved your issue?CommentSharekbhavsaranswered 4 years ago0+1, having the exact same issue. Did you ever get this solved?CommentShareMartijnPanswered 4 years ago"


Question: How to make EventBridge rule match on prefix AND suffix

"When matching s3 events through eventbridge I would like to match on the prefix AND suffix of the key. The following results in an or match.    {       "detail-type": ["Object Created"],       "source": ["aws.s3"],       "detail": {         "bucket": {           "name": [some-bucket"]         },         "object": {,           "key": [{             "prefix": "some-prefix/"           }, {             "suffix": ".extention"           }]         }       }     }Repeating the Key key one for prefix and one for suffix yields no results at all.Can someone help me out?FollowComment"

Answer: "0I am also encountering this issue and some things I found out:If you include multiple values in brackets, it's treated as an OR conditionIf you include a key more than once, the last reference will be the one used to evaluate eventsTherefore for the following event pattern, only { "suffix": ".extension" } will be taken into account when evaluating the key.    "object": {      "key": [{"prefix": "some-prefix/"}],      "key": [{"suffix": ".extension"}]    }See link: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns-content-based-filtering.html#eb-filtering-complex-example)So far, the following are the workarounds I can think of for my use case:Use the regular S3 Event notification feature as it can limit based on prefix AND suffix - whenever I only need to publish the event to Lambda, SQS or SNSIf I really need to publish to EventBridge, I will create an S3 replication to another bucket and limit the scope to a prefix. That second bucket has EventBridge notification enabled and the event pattern filters the suffixCommentSharejoahnaanswered a month ago0I also encountered this issue, I was able to solve it by creating another Eventbus.My first event comes from s3 to the default EventBus and is filtered by prefix.I then trigger an event on a second Eventbus that filters by suffix, that targets my lambdaCommentSharejswineyanswered 13 days ago"


Question: "Players dont stop joining after game has started (Battle Royale Game, UE5)"

"Main thing: my own game is about to get finished but theres some bug with matchmaking! my goal is to have more than two matches in one single server just like in any battle royale. also i have a bug where new players keep joining the ongoing game when it already started (and they shouldnt, just like in any br game). When using AWS Matchmaking. AWS will try to fill the amount of players the match is set for. Always. So, we tell AWS to stop back filling players once game has started. Here it’s not listening to our request.Yes, autobackfill its disabled. its manual.Player Sessions get updated everytime a Player Logs in the Game.Its a auto mated process & is called by the server.UpdatePlayerSessionCreationPolicy() << This is C++ function, we have BP which does the same.And, the issue has nothing to do with PlyerSession Update.Any suggestions? Has this ever happened to anyone?I’ll pay if this gets fixedFollowComment"

Answer: "0To answer your question, we require details that are non-public information. Please open a support case with AWS using the following link.CommentShareSUPPORT ENGINEERAbhilash_Kanswered 9 months ago"


Question: AWS Certified Solutions Architect - Associate : Benefits expiration?

"Hi,Do the benefits that come with passing the Certified Solutions Architect - Associate exam (i.e. discounts on your next certification exam) expire, or can you click the "Claim" button anytime?Should I click the "Claim" button asap or is there no hurry to claim the benefits?Thank you!P.S. This was originally posted in the wrong forum.FollowComment"

Answer: "0Accepted AnswerHi Vichan,CORRECTION: Once a benefit is claimed it is good until the certification for which you obtained the benefit expires.  If you never claim the benefit it will expire when the certification expires. Hope that helps!AWS Training & CertificationCommentShareAWSTMDanswered 4 years ago0Hi Vichan,Once a benefit is claimed it is good for six months or until the certification for which you obtained the benefit expires, whichever comes first.  If you never claim the benefit it will expire when the certification expires.  Hope that helps!AWS Training & CertificationCommentShareAWSTMDanswered 4 years ago"


Question: aws-cloudformation-restore-efs solution

"I want to share my solution which allows you to restore EFS from backup in your cloudformation template. I want feedback. Please test ithttps://github.com/horeca2/aws-cloudformation-restore-efsFollowCommentsetheliot a year agoVery nice!  If you would like to create a hands-on lab around this solution, we welcome Pull Requests for the Well-Architected labs.  Creating a lab will give your solution more visibilityShare"


Question: Does eks:AccessKubernetesApi allows to update any resources or it allows only readonly access?

"Hello,Please advise does "eks:AccessKubernetesApi" allows only to view the workloads or it allows to update any kubernetes resources.If you could point me to security implications of adding "eks:AccessKubernetesApi" to a role in production, that would be really helpful.ThanksFollowComment"

Answer: 0The IAM policy action eks:AccessKubernetesApi is used to allow users to view Kubernetes resources on the AWS Console. Without this the user cannot see the Overview and Workloads tabs content. You can see in the IAM action type definitions that this action is classified as read-only. You can view full list of actions and their access levels here.On top of this you also need to create IAM role mapping inside the EKS cluster to give the AWS Console access to make requests on users behalf. More details available at here.CommentShareEXPERTToni_Sanswered a year ago


Question: Cloudformation Template

"So I am making cloud formation template main.yaml file and one other yaml file from which parameters can be taken so that I  can reuse it by just making changes in the value of another yaml file where only values are other but facing issue. Please guide accordingly.Parameters:InstanceType:Description: EC2 instance typeType: StringDefault: t2.microAllowedValues: [t2.micro, t2.small, t2.medium, m4.large]SecurityGroupId:Description: Security group ID for the EC2 instanceType: AWS::EC2::SecurityGroup::IdVpcId:Description: VPC IDType: AWS::EC2::VPC::IdKeyName:Description: Name of the key pair to use for SSH accessType: AWS::EC2::KeyPair::KeyNameSubnetId:Description: Subnet ID for the EC2 instanceType: AWS::EC2::Subnet::IdResources:EC2Instance:Type: AWS::EC2::InstanceProperties:InstanceType: !Ref InstanceTypeSecurityGroupIds: [!Ref SecurityGroupId]KeyName: !Ref KeyNameSubnetId: !Ref SubnetIdImageId: ami-0f8ca728008ff5af4UserData: !Base64Fn::Sub: |#!/bin/bashsudo apt-get updatesudo apt install apache2 -ysudo systemctl start apache2sudo systemctl enable apache2Like we ave variables in terraform in params.yamlHow do I format and define these values or take user inputInstanceType: t2.microKeyName: devopsSecurityGroupIds: sg-02464c840862fddafSubnetId: subnet-0b2bbe1a860c1ec8fKeyName: devopsVpcId: vpc-01491099ac5c6857aFacing issue with formatting and defining paramsyaml file.Please guide.FollowCommentsedat_salman EXPERT2 months agoPlease copy code blocks instead of pictures and also please add the issue you have facedShareuser01 2 months agoPlease guide. how to do it in cloudformation like variables.tf we do in terraformShare"

Answer: "0Hi ThereFirst, you need to store the parameters in a json file (you cant use YAML for the parameters file).  Example parameters.json file:[   {        "ParameterKey": "InstanceType",        "ParameterValue": "t2.micro"    },    {        "ParameterKey": "SecurityGroupId",        "ParameterValue": "sg-xxxxxxxx"    }]Then, you deploy your CloudFormation stack using the --parameter-overrides option.aws cloudformation deploy --template-file /path_to_template/template.yml --stack-name my-new-stack --parameter-overrides file://path_to_parameters/parameters.jsonReference: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudformation/deploy/index.htmlCommentShareEXPERTMatt-Banswered 2 months agoEXPERTBrettski-AWSreviewed 2 months agouser01 2 months agoThank you. What about Userdata can it be also passed instead of template.yaml from parameters.json? If yes, How? please guide with fullThis is my template.yamlAWSTemplateFormatVersion: "2010-09-09"Parameters:InstanceType:Type: StringDefault: t2.microDescription: EC2 instance typeAMI:Type: AWS::EC2::Image::IdDefault: ami-0f8ca728008ff5af4Description: ID of the Amazon Machine Image (AMI) to use for the instanceKeyName:Type: AWS::EC2::KeyPair::KeyNameDefault: my-key-pairDescription: Name of an existing EC2 KeyPair to enable SSH access to the instanceSecurityGroupId:Description: Security group ID for the EC2 instanceType: AWS::EC2::SecurityGroup::IdVpcId:Description: VPC IDType: AWS::EC2::VPC::IdSubnetId:Description: Subnet ID for the EC2 instanceType: AWS::EC2::Subnet::IdRootVolumeSize:Type: NumberDefault: 0Description: Size of the root volume in GBDataVolumeSize:Type: NumberDefault: 0Description: Size of the data volume in GBResources:EC2Instance:Type: AWS::EC2::InstanceProperties:InstanceType: !Ref InstanceTypeImageId: !Ref AMIKeyName: !Ref KeyNameSecurityGroupIds:- !Ref SecurityGroupIdSubnetId: !Ref SubnetIdUserData: !Base64Fn::Sub: |#!/bin/bashsudo apt-get updatesudo apt install apache2 -ysudo systemctl start apache2Share"


Question: Error when save using file system s3a in EMR 6.6.0 with spark 3.2

"We are trying to save data using s3a and we are getting this error, but when we save the data as s3 it works fine. Below you can find the stacktrace.Closing SparkContextException in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.util.SemaphoredDelegatingExecutor.<init>(Lcom/google/common/util/concurrent/ListeningExecutorService;IZ)Vat org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:813)at org.apache.hadoop.fs.FileSystem.primitiveCreate(FileSystem.java:1235)at org.apache.hadoop.fs.DelegateToFileSystem.createInternal(DelegateToFileSystem.java:100)at org.apache.hadoop.fs.AbstractFileSystem.create(AbstractFileSystem.java:605)at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:696)at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:692)at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)at org.apache.hadoop.fs.FileContext.create(FileContext.java:698)at org.apache.spark.sql.execution.streaming.FileContextBasedCheckpointFileManager.createTempFile(CheckpointFileManager.scala:327)at org.apache.spark.sql.execution.streaming.CheckpointFileManager$RenameBasedFSDataOutputStream.<init>(CheckpointFileManager.scala:140)at org.apache.spark.sql.execution.streaming.CheckpointFileManager$RenameBasedFSDataOutputStream.<init>(CheckpointFileManager.scala:143)at org.apache.spark.sql.execution.streaming.FileContextBasedCheckpointFileManager.createAtomic(CheckpointFileManager.scala:333)at org.apache.spark.sql.execution.streaming.StreamMetadata$.write(StreamMetadata.scala:79)at org.apache.spark.sql.execution.streaming.StreamExecution.$anonfun$streamMetadata$1(StreamExecution.scala:141)at scala.Option.getOrElse(Option.scala:189)FollowComment"

Answer: "0This is a known issue and was Fixed in EMR 6.7.0  and above.Root cause :In EMR-6.6.0, Spark was upgraded to 3.2.0 which includes SPARK-33212, a change that makes Spark depend on shaded Hadoop client jars (hadoop-client-api and hadoop-client-runtime) instead of the unshaded ones. The difference between the jars is that the third-party classes in the shaded jars were relocated to different packages. However, S3A classes were built assuming that the unshaded client jars are used (i.e. third-party classes aren’t relocated).When Spark creates an S3AFileSystem, the S3AFileSystem.create() method attempts to invoke the constructor of a Hadoop utility class (SemaphoredDelegatingExecutor) while incorrectly assuming it accepts the unshaded version of a Guava class [1]. The actual parameter type of this constructor in Spark turns out to be the shaded version of this class [2]. This causes the JVM to throw the NoSuchMethodError, causing customer’s application to fail.This issue was fixed in Hadoop 3.2.2 by HADOOP-16080, but Spark in EMR-6.6.0 uses Hadoop 3.2.1 without this fix backported.[1] com.google.common.util.concurrent.ListeningExecutorService[2] org.apache.hadoop.shaded.com.google.common.util.concurrent.ListeningExecutorServiceIf upgrading to EMR 6.7 is not an option, please reach out to AWS Support.CommentShareMannem Ranswered 5 months ago"


Question: Multiple Catalog Access from an ETL Glue Job

"Hi all together,thanks to this article https://repost.aws/de/knowledge-center/query-glue-data-catalog-cross-account I know that with AWS EMR I can access the Glue Catalog from my current account and in addition by setting up the right permissions also the glue catalog from another account at the same time.My question is if this is also possible with an ETL Glue Job.I know that a cross Account Glue Catalog can be set up by using the --conf spark.hadoop.hive.metastore.glue.catalogid.. Parameter.But if I want to access tables from two other Accounts, I have a problem.Anyone any idea?Thanks for the help.BestFollowComment"

Answer: "0Hi,I understand that you are trying to access tables from two different glue catalogue accounts using a glue job. We can setup the access policies in source and target accounts and then use two different dynamic frames to access these tables. We don’t need to use the “--conf spark.hadoop.hive.metastore.glue.catalogid” option for your use case. The step by step process to setup for cross account is provided here -https://repost.aws/knowledge-center/glue-tables-cross-accountsWe can first setup the cross account permissions from Account A [Catalogue account] to Account B and Account C separately. Then in the glue job we can create two dynamic frames [df1 and df2] to access tables from the accounts B and C.For Example :df1 = glueContext.create_dynamic_frame.from_catalog(database="doc_example_DB", table_name="doc_example_table", catalog_id=“Account B”, region="us-east-1")df2 = glueContext.create_dynamic_frame.from_catalog(database="doc_example_DB", table_name="doc_example_table", catalog_id="Account C", region="us-east-1")Thank you.CommentShareSUPPORT ENGINEERSaiIndira_Chanswered 2 months agorePost-User-5899595 2 months agoThank you very much.This helps a lotShare"


Question: Frozen dashboard - servers stuck on stopping and pending?

"Hi,This morning everything was working fine, we currently only have a few EC2 instances running. For no reason the main instance that hosts our site froze. I couldn't even SSH into it:i-0f02548f4de263bb4I tried stopping it, force stopping it and its stuck in a stopping state.This guide https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesStopping.html said create a replacement instance while I post here for a resolution. Which I did but now thats stuck in the pending state:i-01c4449db2dfeb8c3Can someone let me know if there's anything else I can do? Our services are down and we've no way to recover this at the minute.JonFollowComment"

Answer: "0Our main instance has stopped. For whatever reason it took over half an hour where as it normally stops in a minute or so.CommentSharejonwrightanswered 2 years ago0We're also having issues. Our production instance was unresponsive so I rebooted it and things just got worse form there on! It's now stuck in a pending state and says the volumes are impaired. Can't stop or reboot it now and it's been stuck that way for over half an hour. Not what I wanted first thing on a monday morning :-(CommentSharecodemonkey69answered 2 years ago0There's an "EC2 operational issue" affecting euw2-az3 (eu-west-2b) - which was logged starting at February 1, 2021 at 12:24:33 PM UTC (but we were seeing issues before that).It's showing on my "Personal Health Dashboard" - but on the "Affected resources" tab of the issue, it's not showing the instance which is stuck in a pending state (due to the volume being "impaired").I cannot force detach the volume from the instance; the only instance option I have is terminate - but this fails from the AWS Management Console with the error "The instance ... may not be terminated. Modify its 'disableApiTermination' instance attribute and try again."CommentShareTommyGunanswered 2 years ago"


Question: MWAA legacy airflow and python versions

"Hi all! I am new to MWAA service. I would like to use it to manage my dags, which I deployed by myself on ec2 instances with airflow running in a docker. It would be much simpler for me to use managed airflow :)But: I already use Dynamic Task Mapping which is from Airflow 2.3 where MWAA Airflow is 2.2 + my packages are python 3.9 where MWAA python is 3.7.Would you know the roadmap of AWS concerning the updates of Airflow and Python?Would you know how to run my airflow tasks in a 3.9 environment (I read it could be possible using docker, but I don't find any tutorial to do this).Thanks a lot for your help!FollowComment"

Answer: "0For the tasks that will be running inside a container, you can utilize the ecs operator to execute a task on the Amazon Elastic Container Service (ECS).   We have some sample information to assist you in connection to the Amazon ECS utilizing the ECSOperator:https://docs.aws.amazon.com/mwaa/latest/userguide/samples-ecs-operator.htmlThis link contains the information on how to host your container within Amazon ECS and also the example DAG for invoking the ECS Operator.CommentShareCurtis_Danswered 9 months ago0Ok thanks. Finally we are going to downgrade our packages and to unuse Dynamic Task Mapping, which is not available in Airflow 2.2.2.Would you have information on the timeline for versions upgrades or Python and Airflow?Thanks a lot!CommentShareLine Rahalanswered 9 months ago"


Question: Monitron Sensor

"Hello,I am transfering monitron sensor data from S3 bucket into a self-made app.  (visualizing vibration and temperature in a different way than the monitron-app is doing)How can store the monitron data longer - right now I can only fetch the last 24 hours.Can I extend it to 7 days ?FollowComment"

Answer: "0Yes, you can store the Monitron data for longer than the default 24-hour retention period by enabling data archiving in Amazon S3. To do this, you need to create an S3 bucket and configure it to receive the Monitron data. Then, you can use the Amazon S3 Lifecycle feature to automatically move the data from the original bucket to a second bucket with longer-term storage after a specified time period, such as 7 days.To set up data archiving for Monitron data, follow these general steps:Create an S3 bucket to receive the Monitron data.Configure Monitron to store the data in the S3 bucket.Enable S3 Lifecycle on the bucket to move data to a second bucket with longer-term storage after a specified time period.Use the second bucket to store the data for longer periods of time.Here are more detailed steps:Create an S3 bucket to receive the Monitron data:Open the S3 console and click "Create bucket".Name the bucket and choose a region for it.Leave all other settings as default and click "Create bucket".Configure Monitron to store the data in the S3 bucket:In the Monitron console, go to "Settings".Under "Data Export", choose "S3 bucket".Enter the name of the S3 bucket you just created.Leave all other settings as default and click "Save".Enable S3 Lifecycle on the bucket to move data to a second bucket with longer-term storage after a specified time period:In the S3 console, click on the S3 bucket you created for Monitron data.Click the "Management" tab and then "Lifecycle".Click "Add lifecycle rule".Choose "Current version of an object" and click "Next".Set the "Expiration" period to 7 days (or the desired period for longer-term storage).Choose "Transition current version to S3 Standard-Infrequent Access" and set the number of days for this transition (for example, 7 days).Choose "Create new S3 bucket" for the transition destination and name the new bucket.Leave all other settings as default and click "Review".Review the settings and click "Create rule".Use the second bucket to store the data for longer periods of time:After the specified time period (7 days in this example), the data will be automatically moved to the second bucket.You can use this second bucket to store the data for longer periods of time.You can also apply additional S3 Lifecycle rules to the second bucket to move the data to even cheaper storage options after longer periods of time, such as Glacier or Glacier Deep Archive.With these steps, you can store Monitron data for longer periods of time and use it in your app.CommentSharemishdaneanswered 2 months ago"