Question: EMR 6.10 - local disk encryption error

"I'm trying to deploy Hadoop on EMR EC2 Cluster and having an issue with local disk encryption error on AWS Console.AWS doesn't provide me any deeper logs nor info about the issue (S3 bucket log is empty, CloudTrail doesn't have Error Codes).My Security Configuration: use KMS CMK, Encryption Local disk, IMDSv2, AuthN: Off, AuthZ: Instance Profile.Used Roles:EMR_DefaultRole (Policies : AmazonEMRServicePolicy_v2, myCustomPolicy)EMR_EC2_DefaultRole (Policies : AmazonElasticMapReduceforEC2Role, myCustomPolicy)myCustomPolicy:{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "HadoopPolicy",            "Effect": "Allow",            "Action": [                "ec2:DescribeSecurityGroups",                "ec2:AuthorizeSecurityGroupEgress",                "ec2:AuthorizeSecurityGroupIngress",                "ec2:RevokeSecurityGroupEgress",                "ec2:RevokeSecurityGroupIngress",                "ec2:CreateSecurityGroup",                "ec2:DeleteSecurityGroup",                "ec2:*"            ],            "Resource": [                "arn:aws:ec2:my-region:XXXXXXXXXXX:security-group/sg-SLAVE",                "arn:aws:ec2:my-region:XXXXXXXXXXX:security-group/sg-MASTER",                "arn:aws:ec2:my-region:XXXXXXXXXXX:subnet/my-subnet-with-internet-access",                "arn:aws:ec2:my-region:XXXXXXXXXXX:*"            ]        },        {            "Sid": "AllowReadAccesstoKey",            "Effect": "Allow",            "Action": [                "kms:Encrypt",                "kms:Decrypt",                "kms:ReEncrypt*",                "kms:GenerateDataKey*",                "kms:DescribeKey",                "kms:CreateGrant",                "kms:ListGrants"            ],            "Resource": "arn:aws:kms:my-region:XXXXXXXXXXX:key/my-kms-cmk-id"        }    ]}In addition:I'm using standard Amazon Linux AMI.In KMS CMK I pointed my EMR_DefaultRole and EMR_EC2_DefaultRole as a users and administrators of KEY.I've deployed cluster in subnet with Internet connection through IGW.Reference guides:https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-data-encryption-options.htmlhttps://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-security.htmlhttps://repost.aws/knowledge-center/emr-create-cluster-with-ebs-encryptionFollowComment"

Answer: "1Hi,IAM permissions looks correctly. Instance profile which is used to spin EMR cluster instance has to be able to interact KMS key which will be used to encrypt EBS storage. You stated also that deployment was done in subnet with IGW. Can you verify if indeed IGW is in route table for that subnet?What I would suggest to check is to make sure that VPC is properly configured.KMS service must be accessible from subnet(s) where cluster nodes are being created. Preferred way one can achieve that is by creating VPC Endpoint for KMS service (https://docs.aws.amazon.com/kms/latest/developerguide/kms-vpc-endpoint.html#vpce-create-endpoint). This way will make sure traffic will not traverse via public internet but stay inside AWS network.CommentSharemarcsamianswered 22 days ago1Hello,I see you also indicated how you setup the EMR security configuration. if no insights on Ctrail, on your S3 log directory go to the prefix /cluster_ID/node/<master_node_ID>/setup-devices/ and refer to the log DiskEncryptor.log.gz for insights.If you still not seeing any insights on WHY, please indicate will also test the same using your above stepsCommentSharerePost-User-2219661answered 18 days ago0Hi, Thanks for your answers.Finally, after many days I found the solution:DELETE aws:SourceArn and aws:SourceAccount CONDITIONS from trust relationships for Instance Profile.In my case when this conditions are in trust relationships lunching flow ends with "local disk encryption error - Internal error", logs are not publishing on S3 and there's no info in CloudTrail.EMR_EC2_DefaultRole:Trust relationships:{    "Version": "2008-10-17",    "Statement": [        {            "Sid": "",            "Effect": "Allow",            "Principal": {                "Service": "ec2.amazonaws.com"            },            "Action": "sts:AssumeRole",/*START DELETE */            "Condition": {                "StringEquals": {                    "aws:SourceAccount": "<account-id>"                },                "ArnLike": {                    "aws:SourceArn": "arn:aws:elasticmapreduce:<my-region>:<my-account-id>:*"                }            }/*END DELETE */        }    ]}I don't know why in AWS Documentation they provide this condition in Trust relationships:https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-iam-role.htmlCommentSharetwitkoanswered 16 days agomarcsami 15 days agoYou are mixing EMR Service Role and EC2 Instance Profile. In documentation you are referring to it is clearly stated that conditional block is for Service Role not Instance Profile.EMR EC2 Instance Profile creation is described here https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-iam-role-for-ec2.html.Share"


Question: Length Limit on EnvironmentVariables configuration parameter?

"Does the EnvironmentVariables configuration parameter for CodeBuild actions (https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodeBuild.html#action-reference-CodeBuild-config) have a length limit? If so, what is it?FollowComment"

Answer: "1There's no specific limit for the length of environment variables, but as noted in https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodeBuild.html#action-reference-CodeBuild-config there's an overall limit of 1000 characters.When you enter the name, value, and type for your environment variables configuration, especially if the environment variable contains CodePipeline output variable syntax, do not exceed the 1000-character limit for the configuration’s value field. A validation error is returned when this limit is exceeded.CommentShareAWS-User-5639455answered 3 years ago0Kaixiang-AWS wrote:When you enter the name, value, and type for your environment variables configuration, especially if the environment variable contains CodePipeline output variable syntax, do not exceed the 1000-character limit for the configuration’s value field. A validation error is returned when this limit is exceeded.Thank you for that! I interpret the docs as saying the "value" field for each environment variable cannot exceed 1000 characters.For example, could I have three environment variables, each with a 900-character value?Thank you for clarifying!CommentSharem4dc4panswered 3 years ago"


Question: Setting up Managed Grafana with Amazon RDS

"im trying to set up a managed grafana instance in AWS and configure a datasource to pull from a RDS database (public ip). When i test the connection i get a 504 responsei can successfully connect to the connection via mysql workbench. however, with the same connection info and credentials in the add datasource -> mysql page within grafana i get the following error: <title>504 Gateway Time-out</title>
504 Gateway Time-out
        I have tried adding the IP address of my managed grafana instance to the inbound TCP rules for the RDS security group roles for the instance as well:Non-authoritative answer:Name:   g-7b1e9d3d37.grafana-workspace.us-east-2.amazonaws.comAddress: 3.141.179.101Name:   g-7b1e9d3d37.grafana-workspace.us-east-2.amazonaws.comAddress: 52.14.201.37Name:   g-7b1e9d3d37.grafana-workspace.us-east-2.amazonaws.comAddress: 3.132.106.29FollowComment"


Question: Supporting Data Capture on Serverless Endpoints?

"Hi!I have understood that the Serverless Endpoints currently do not support the Data Capture functionality (according to the feature exclusions found here). This is unfortunate as my use case is primed for using Serverless Endpoints, but at the same time I was hoping to leverage the ease of the data capture functionality.Are there plans on enabling this feature for Serverless Endpoints too? If yes, what is the time plan?FollowComment"


Question: AWS Toll Free Number #Without Sandbox

"Hey Team,Initially we were using AWS SNS service to send SMS to a person from application. For that it seems as a pre-requisites, we were using AWS Sandbox, where we were adding numbers and verifiying those and sending SMSs via application. Now our customer is US based and for that as per new process we have to opt for Toll free number and complete the registration. We have done with toll free process and receives the toll number with active status.Now, is it ok if I exit from SMS sandbox from my AWS account ?? Or is there any additional process to be done ?? Since I tried to send one SMS to US number which has not gone successful but after verifying that number in Sandbox it was successfully received.In actual scenario we don't want US numbers to be verify first. Application can directly send SMS to US numbers without verification from toll free which is already active.If I exit from sandbox, will it impact toll free number, which we have activated ??Regards,Nalay PatelFollowComment"


Question: Container - is it a compute service?

Can a container be considered a compute service?FollowComment

Answer: "5Accepted AnswerHi,In cloud computing the term “compute” describes concepts and objects related to software computation like processing power, memory, networking, storage, and other resources required for the computational success of any program. Containers are one of the AWS compute services.Here is more info on what Compute is in the context of Cloud Computing:https://aws.amazon.com/what-is/compute/Here is an overview of AWS Compute Services where you can find all the different Container services:https://aws.amazon.com/products/compute/Hope this helps!CommentShareMarasiliaanswered 6 months agoEXPERTAWS-User-Nitinreviewed 6 months agoTLChandler 6 months agoThis is a tremedous help!!!  The explanation is wonderful!  Marvelous!  Many, many thanks for your assistance Marasilia.  Hope you're safe and well!!!Share0You're very welcome TLChandler!Glad I could help :)Have a great weekend!CommentShareMarasiliaanswered 6 months ago"


Question: Can we connect multiple File Gateways to one FSx ?

"Hello, our customer has multiple on-premises sites and would like to connect them all to use one AWS FSx for Windows File Server.Can we install FSX Storage Gateways on each of the sites, then connect them all to the same FSx on AWS?Will this have any problems with file ownsership, locking, etc.?Thanks in advance.H. BalighFollowComment"

Answer: "1This limitation makes it difficult to deploy.FSx Windows feature of file locking is unfortunately a multi-office choice we've made.CommentShareThomasBrownanswered 10 months ago0Accepted AnswerHello,Yes, you can use multiple FSx File Gateways and attach the single FSx Windows file server to multiple gateways. However, please note this would be a multiple writer scenario which could cause issues if a second writer (Gateway B) modifies written files than the specified gateway (Gateway A). To avoid running into issues with multiple writers, we recommend using dedicated folders for each FSx File Gateway without any overlap on the FSx file server.CommentShareSuryaanswered a year agoH Baligh a year agoThanks very much!Share"


Question: How to programmatically figure out which services are being actively used by an AWS account

"Outside of billing and calling each service Describe/List* method, is there a way to obtain a list of AWS services for which there is atleast one resource present in a given AWS account? (this is different from Active service).FollowComment"

Answer: "0Have your tried using awscli get cost usage to retrieve the resources in your account by different dimensions?  FYIhttps://awscli.amazonaws.com/v2/documentation/api/latest/reference/ce/get-cost-and-usage-with-resources.htmlOrhttps://docs.aws.amazon.com/zh_cn/aws-cost-management/latest/APIReference/API_GetCostAndUsageWithResources.htmlCommentShareLihu Wuanswered 5 months ago0Hi ThereThere are a few ways to to this:Use Tag EditorUse AWS ConfigUse a third party tool like aws-inventoryUse AWS Resource Explorer although not all services are supported yetCommentShareEXPERTMatt-Banswered 5 months ago0I'd consider looking in to the recently announced AWS Resource Explorer (https://aws.amazon.com/resourceexplorer/)CommentShareRoss Wickmananswered 5 months ago0Thank you for quick responses! I should have added that I have limitation of working within SecurityAudit policy permissions to be able to do this programmatically. That said, I wasn't aware of aws-inventory tool and AWS resource explorer, will try those and  respond back to the thread if it indeed solves the problem.CommentSharerePost-User-7179714answered 5 months ago"


Question: Flask application deployed through Elasticbean stalk unable to load file in S3

"Hi,I am getting below error when trying to upload files in S3 bucket. I checked policies and updated them but still getting error.2023-01-01T16:08:29.758+05:30Jan  1 10:38:29 ip-172-31-40-199 web: boto3.exceptions.S3UploadFailedError: Failed to upload static/uploads/alexa.wav to myflask/alexa.wav: An error occurred (AccessDenied) when calling the PutObject operation: Access DeniedJan 1 10:38:29 ip-172-31-40-199 web: boto3.exceptions.S3UploadFailedError: Failed to upload static/uploads/alexa.wav to myflask/alexa.wav: An error occurred (AccessDenied) when calling the PutObject operation: Access DeniedJan  1 10:58:40 ip-172-31-40-199 web: ERROR:aws_xray_sdk.core.context:cannot find the current segment/subsegment, please make sure you have a segment openBucket is public and policy applied is{    "Version": "2008-10-17",    "Statement": [        {            "Sid": "myflask-allow-permission-for-saving-audios",            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::377727317375:role/aws-elasticbeanstalk-ec2-role"            },            "Action": [                "s3:PutObject",                "s3:ListBucket",                "s3:ListBucketVersions",                "s3:GetObject",                "s3:GetObjectVersion"            ],            "Resource": [                "arn:aws:s3:::myflask",                "arn:aws:s3:::myflask/*"            ]        }    ]}FollowComment"

Answer: "0In addition to the S3 bucket policy, You should verify that your instance has the correct permissions for Amazon S3.https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-s3-bucket-instance/By default, aws-elasticbeanstalk-ec2-role is attached as an instance profile to the Elastic Beanstalk applications.Please confirm that the role has the permissions including "s3:PutObject"CommentShareKihyeonanswered 5 months agorePost-User-8055548 5 months agoHi Kihyeon,I searched a lot and later found on a forum that "s3:PutObjectAcl" is also needed along with s3:PutObject, once I added this, it worked perfectly.ShareKihyeon 5 months agoThank you for updating the statusShare0After adding "s3:PutObjectAcl" along with s3:PutObject, it worked perfectly.CommentSharerePost-User-8055548answered 5 months ago"


Question: Help with the cloud architecture

"I need some feedback on my current design.Problem: I have 5 Python scripts which need to be executed based on user selection; I also have a frontend where the user can see the list of scripts and choose which one to run; the user will give some input (strings) which will be given to script and the script start, they ran for about 3 min and sends the response back to the user. Some of the scripts may require input multiple times; a user at the front end will be notified, and he will enter the input( strings), and the script will continue.Solution 1:  I am using Nomad with a client with 8GB RAM. I have made a docker of all five scripts. I  am also using a redis which allows input to be moved from the backend to this docker. So basically, every time a user opens the browser tab with script selection, I ask the Nomad to run the job and keep waiting for input; user input is taken by the backend and placed in redis. The script then fetches input, which can be fetched multiple times. The jobs remain active until the user closes the tab for that script. As soon as the tab is closed, the nomad job is killed.Solution 2: Combine all the scripts into a single docker (which script to run on which user command is handled internally) and run two instances of that job using Nomad. Whenever a user request script execution, we take the input request from the user and put it in redis, and then it is fetched by one of the two running jobs; once the job is started, a new instance is started (Not sure how to achieve this, I am thinking to make jobs in Nomad as a batch so they stop after executing and every time a script starts execution a new job is started and it keeps on waiting from redis and other job is completed and stopped), basically meaning for N users there will be N+2 instance of job running to reduce the waiting time, but we don't need to keep waiting for the user browser tab to close to kill a job, as soon as its work is done it is killed, and two ideal job instance constraint is maintained.Let me know what the better way to design this is.I am expecting ten users per day.FollowComment"

Answer: "0My impression is that Nomad is: https://www.hashicorp.com/products/nomadI am not sure if you are specifically tied to Nomad / Docker / Redis. If so, I can't offer any insight since I have not used Nomad. The key will be which AWS services it can call / interact with.If you are not tied to your existing stack, there are a couple of simple and proven AWS / Cloud patterns that would potentially work well for you since you are running Python scripts.Use API Gateway to host endpoints that are called when the user selects a script from your list. The APIGW is a proxy to a lambda (specifically python) to execute your script logic and return a result to the client side. (https://docs.aws.amazon.com/solutions/latest/constructs/aws-apigateway-lambda.html). The lambda can take up to 15 minutes if needed. Call the API GW from Nomad (if possible) or create a simple web page to offer the options, collect input data and call APIGW. Host web page in S3/CloudFront (https://docs.aws.amazon.com/solutions/latest/constructs/aws-cloudfront-apigateway-lambda.html).USE API GW and have it proxy to SQS to store the results (https://docs.aws.amazon.com/solutions/latest/constructs/aws-apigateway-sqs.html). Then you can have a Lambda trigger (to your Python lambdas) or docker long polling the SQS queue to process the request. Same front end as #1.In any case, you appear to be using Redis as a queue. AWS has SQS which is a is probably a better option for this use case. But, I think you may be using Redis because it is a data source available to Nomad and SQS may not be.CommentShareTigerSC95answered a month ago"


Question: Unrecognized action eks:AccessKubernetesApi

"Hello!When trying to access our EKS cluster I get "Unauthorized: Verify you have access to the Kubernetes cluster" assuming a role having all rights (Allow--). Our developers using a "DeveloperRole" get an error message saying eks:AccessKubernetesApi is needed. However, when trying to add this, it lists as an unrecognized action. This behavior relates to both our Sandbox and Staging member-accounts. However, in our Production account everything is fine.Is this related to our last week upgrade from cluster version 1.16 to 1.18? However, our Production account is also on 1.18 and works fine...Anyone have any ideas of what is going on?Best regardsVidar Evenrud SeebergFollowComment"

Answer: "0Same error here with 3 clusters (21.15 + 11.18).I'm not sure this is related your migration.We can access to clusters through kubectl without any errors.CommentSharealxsbnanswered 3 years ago0Are you running your clusters in different regions? Which ones?We are getting the same with three 1.17 clusters in us-west-2 installed with eksctl and with non-managed nodegroups.  It seems to me that this may be caused due to new features being rolled out per Re:Invent.CommentSharesstoyanovanswered 3 years ago0Hello!I am actually globus68... Somehow the login messed up my user...We are having our clusters in eu-west-1 for all environments, so it is strange it works in Production, but not in Sandbox and Staging. Yes, it may very well be some re:invent news...Best regardsglobus68CommentShareglobus68-2answered 3 years ago0Same issue here. So far no solution. But it applies to also not updated clusters.Help? :-(CommentSharegnwdtxanswered 3 years ago0eu-west-1 also !CommentSharealxsbnanswered 3 years ago0This helps!{"Version": "2012-10-17","Statement": [{"Action": ["eks:AccessKubernetesApi"],"Effect": "Allow","Resource": "*"}]}CommentSharedbattulaanswered 3 years ago0I've tried, it does not solve the issue.BTW: If the user is using{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Action": "","Resource": ""}]}...I would assume it should work any way.CommentSharegnwdtxanswered 3 years ago0@dbattula: I already tried that, but got the "Unrecognized action" message.@gnwdtx: I also tried that, but still does not workCommentShareglobus68answered 2 years ago0Thank you SriSaranBalajiVR@AWS!Adding eks:AccessKubernetesApi to a policy for the developer role fixed it. However, using the Administrator role (having Allow--) we still are denied access.Best regardsglobus68CommentShareglobus68answered 2 years ago0Hello globus68,I sent a private message as well requesting for some information. If possible, can you paste the exact error message you're seeing and then we can verify whether its due to AWS Service AuthN/AuthZ issue or Kubernetes AuthN/AuthZ issue.Thank you.CommentShareAWS-User-0353704answered 2 years ago0Hello,Sorry for the inconvenience this has caused. We are updating our docs with more detailed info (https://github.com/awsdocs/amazon-eks-user-guide/blob/master/doc_source/troubleshooting_iam.md#cant-see-workloads-or-nodes-and-see-error-loading-namespaces-in-the-aws-management-console)TL;DR: Couple of days back, we launched new AWS console for EKS (https://aws.amazon.com/blogs/containers/introducing-the-new-amazon-eks-console/). As part of this launch, users can view their kubernetes objects in aws console for example, console now shows pods, deployments and other resources available in their EKS cluster.New EKS console requires following two permissions to be available to view kubernetes objectsNew IAM permission - eks:AccessKubernetesApiCurrently this permission can be updated by modifying the json policy in the console as explained here https://docs.aws.amazon.com/eks/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-console. While we are actively pushing this new Permission to be available in IAM console, you can use API to update IAM permission until then you will notice "Unrecognized action" in the console.Access to Kubernetes ClusterUser using console should have their IAM entity in the kubernetes aws-auth configmap. If the mapping is in the configmap, make sure that the user or group is bound to a Kubernetes role or clusterrole has the necessary permissions to view the Kubernetes resources. If the IAM user or role isn't in the configmap or the Kubernetes role or clusterrole doesn't have permissions to view the Kubernetes resources, the console error may include "Unauthorized: Verify you have access to the Kubernetes cluster". For more information about adding IAM users or roles to the aws-auth configmap, please see https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html.Existing content is moved under configuration tab and should be visible without any modification to the IAM entity. Also, this feature doesn't affect your cluster health or removes any of our existing permissions. Once again sorry for the inconvenience. Please do try it out and give us your feedback.Thank you.CommentShareAWS-User-0353704answered 2 years ago0Problem solved thanks to srisaranbalajiataws!I had to add the administrator role to the aws-auth configMap. The developer role was already in there.globus68CommentShareglobus68answered 2 years ago"


Question: Secure Connection Failed: the authenticity of the received data could not be verified

"I created a WordPress instance using lightsail and assigned a domain to its static IP and enabled HTTPS on the WordPress instance too.The web app is accessible everywhere, but when the user is behind proxy/firewall, he gets the following message and the browser does not allow access to it:Secure Connection Failed: the authenticity of the received data could not be verified.How to solve this issue?FollowComment"

Answer: 0Looks like the proxy is interfering.It could be the proxy is trying to inspect the SSL traffic and isn't replacing the certificate correctly.CommentShareDavid Ganswered a year ago


Question: Can't find the console login event history in ClouldTrail

"I'm new to CloudTrail, When I'm searching root user console login event history, It's logged until December 17 but no further logs are founded.Apparently other event logs are still logged, I have no clue why console login event is not logging and what I did after December 17.How come this is happening? What should I do to see console login event log again?FollowComment"

Answer: "3Accepted AnswerSince November 22, 2021, the history of global service events, including console logins, can only be checked in us-east-1.https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-concepts.html#cloudtrail-concepts-global-service-eventsAre you checking the CloudTrail history in other regions? If so, please make sure to check it in us-east-1.CommentSharehayao-kanswered a year ago0If i have a Multiregion Trail in EU-xxxx i see the us-east-1 events are saved there. How can i analyze them in what dashboard?CommentShareAlexanswered a year ago"


Question: AWS Cognito - SAML Provider Issue

"Hello,I am trying to create a new AWS Cognito SAML identity provider and I am entering the metadata document endpoint URL, but when I try to save my new provider I get the following error: "We were unable to update identity provider: Non-ok status code 403 returned from remote metadata source {here goes my provider URL} (Service: AWSCognitoIdentityProviderService; Status Code: 400; Error Code: InvalidParameterException; Request ID: bfdccf61-dcf3-41d1-88ca-50f73b5b42b4; Proxy: null)"The provider endpoint is ssl and has a valid certificate associated. I also understand that Cognito is getting a forbidden (403) response while trying to access the metadata. I can access the my provider endpoint URL from the browser (no credentials needed) and download the metadata file, and if I upload the file instead of using the endpoint it seems to work just fine. But I do not want to depend on this manually uploaded file in case the provider changes the metadata at some point.I am not sure if there is anything the provider needs t do to allow AWS Cognito to access the endpoint. Can someone please provide some light to this problem?Thank you very much in advance!FollowComment"

Answer: "0When you add a SAML provider in Cognito, the metadata document is requested from an AWS IP address. If you're able to access the metadata document in your browser without any authentication, it sounds like your identity provider has a firewall or similar blocking requests from AWS.I'm not sure which identity provider you're using, but to make it work they will need to allow the AWS IP ranges [1] to access the metadata document.[1] https://ip-ranges.amazonaws.com/ip-ranges.jsonCommentShareEdanswered 8 months agoRansel 8 months agoThank you so much for your reply, Ed! I truly appreciated.I have contacted the SAML provider with your answer and hopefully the request validation gets unblocked!Share"


Question: Infra-as-Code in SageMaker

"AWS SageMaker maintainers pushing everything into notebooks - pipelines, observability, endpoints, etcThere is some heavy scripting with boto3 for helpers and integrations.There are very good abstractions in SM SDK, however approaches like creating IAM roles and lambdas functions from boto3 scripts are from 2015, no?What am I missing?FollowComment"

Answer: "1Hi,The AWS CDK enables you to create ressources similar to CloudFormation, but in your favorite languages such as Javascript and Python. Here is the Amazon SageMaker Construct Library, which allows you to deploy SageMaker ressources with the CDK.CommentShareAWS-User-5132531answered 10 months ago"


Question: creating new iam user leads to "unknown error"

"hi,i am following the aws amplify tutorial ( https://docs.amplify.aws/start/getting-started/installation/q/integration/angular/ ) and i want to create my first iam user via console ( https://us-east-1.console.aws.amazon.com/iamv2/home#/users ) but i get an "Unknown error" every time i press the "Add users" button (see screenshot https://imgur.com/a/SX3KrGw ).trying to open "Account settings" ( https://us-east-1.console.aws.amazon.com/iam/home#/account_settings ) leads to the same problem.i am logged in with the root user and don't know what to do. Can you help me or point me in the right direction to solve the issue?Thanks and greetings!FollowComment"

Answer: "0Accepted AnswerAddendum: The error occurs in Firefox. In Chrome, creating users works. It could be that the iam management console has not been implemented in a cross-browser compatible way.CommentShareAWS-User-5899366answered a year ago"


Question: Codebuild is getting slow

"Hi Team,Since last 2 months I have noticed that my codebuild is getting very before usually it takes upto 5 mins to complete the build now a days it's taking around 2 mins to provisions and to complete the build it will take 13 minsI m using standard:3 image from code. I don't want to upgrade to standard:4 or any other latest versions due to my runtime dependencyI have created a new customer docker image with my requirement even though it's slow and even I have tried the cache option from the artifact's unfortunately it doesn't help meKindly request you to suggest me any tricks to improve the speedFollowCommentrePost-User-2738569 a year agoAny suggestions to reduce build time?Share"


Question: Confused between put request pricing and data transfer in pricing

"Hi,I have read object of any size upload (data transfer in) is free. But 'PUT' request has price tag. Isn't that put request is same as object upload?can someone make me understand, how they two differ in AWS perspective?FollowComment"

Answer: "0As you have seen, there are many different cost structures associated with S3.One way you can look at it, is that AWS breaks out the cost structures into various categories, such as:Storage pricingRequest pricingS3 Storage Management pricingData transfer pricingS3 transfer acceleration pricingCross-region replication pricingAWS Gov Cloud RegionTo determine your costs when using S3, you will need to look at how you are using S3 and which categories may be applicable.For the S3 put Object request action, you will need to look at both the Request pricing and the Data transfer in pricing (both apply to the same action, and they could have made you pay extra money based on the size of the object that needs to be transferred in over the network - for example, the could have charged an additional $0.002 per GB during the upload (object put) process. But instead, the actual data in transfer in is free. ).  Let me know if that helps.Note: The definitions of each of the categories along with their costs can be found in the following documentLink: https://aws.amazon.com/s3/pricing/-randyCommentShareRandyTakeshitaanswered 4 years ago0Hi Randy,So , PUT is a part of s3 hidden server request through API/SDK calls.Where , Data transfer in , is more of AWS network utilization related.When I pay for a virtual storage space like "DropBox",  I usually think of monthly/yearly payment to the storage space , but not in terms of number of requests I made to the dropbox server.Now I realized in AWS architechture, I pay for  AWSInternalNetworkUsage_storageManagement_APIrequests , but since data transfer in(AWSInternalNetworkUsage when I do the upload) is not charged, I only pay storageManagement+APIrequests.I hope I understood it correctly now.I have a follow up question:Why does AWS charge for API requests? Does it use AWSInternalNetwork such as DirectConnect or some other mechanism to process the PUT or any other API request?CommentSharemanikantaSamudralaanswered 4 years ago0Yep, sounds like you've got it :-)As for your follow up question - Why does AWS charge for "requests"?If no one is making a request to data, the network, CPU, and storage devices are basically sitting there doing nothing.. just taking up storage space.   However, once a request is made to storage, disk IOPS, network, and CPU processing is occurring to process that data. These resources, although plentiful in the AWS ecosystem, are "shared" hardware resources. Let's say 2 different customers are storing 1 TB of data.  Customer "A" that makes 200,000 data requests over a months time, and customer "B" makes 500 data requests over a months time. Customer "A" is using up way more resources than customer "B".  Another analogy is renting a car for 1 day.  Standard daily rental is $19.95 per day. However, if you drive more than 20 miles the rental company will add an additional $0.20 per mile to the daily rental price.-randyCommentShareRandyTakeshitaanswered 4 years ago"


Question: Code build cost reduction

"We use amazon code build to build the code from code commit and generate artifacts for deployment.It looks like it is spinning a new machine every time and executes the build, the cost for the code build is increasing for us. Are there any way to reduce the cost for code build.We use gradle for building java+kotlin packages and react & js libraries using nodejs and npm.FollowComment"

Answer: "0Please see these tips that should help you reduce the cost of using Amazon CodeBuild and optimize your build process:Run the 'aws codebuild batch-get-projects' command to determine which builds take the most time to complete. You can then optimize these builds by reducing the number of build dependencies or using a larger build environment.Look in to Amazon CloudWatch to monitor your build usage patterns and identify opportunities to optimize your builds.Look in to the AWS CodeBuild Concurrent Builds feature to run multiple builds concurrently, which can help you take advantage of idle capacity in your build environment.Using a build environment that is shared by multiple builds, such as the 'Managed image' option (Use an image managed by AWS CodeBuild), can help reduce costs.Hope this helps.CommentShareAWS-User-2870192answered 5 months ago"


Question: Cognito User Pools: Receiving Expired Session During CUSTOM_AUTH flow

"I have setup my lambda triggers for define auth challenge, create auth challenge, and verify auth challenge. I am able to successfully get through the PASSWORD_VERIFIER challenge and issue my custom challenge. However, as soon as I try to respond to newly issued custom challenge with AdminRespondToAuth, I get the following error:NotAuthorizedException: An error occurred (NotAuthorizedException) when calling the AdminRespondToAuthChallenge operation: Invalid session for the user.I am sending back the session token from the response I received after calling AdminRespondToAuthChallenge during the PASSWORD_VERIFIER challenge. I've checked the logs for my lambdas, but it looks like it's not even hitting any of themFollowComment"

Answer: "0Hello,Firstly, in regards to the Session String, it is an encrypted session received by the client in the previous step that the client must pass back as-is. The session contains state information about the current authentication. It cannot be replayed and it expires after 3 minutes.https://aws.amazon.com/jp/blogs/mobile/customizing-your-user-pool-authentication-flow/Secondly, in regards to the error faced, one way this error can happen is if two consecutive API calls were being made with the same session in your application code. Please do remember that the session key can be used only once. If RespondToAuthChallenge call is performed using the same session key again, you will get the above error. Hence, I would recommend checking the flow of your API calls in your application code. Also, checking Cloudtrail logs in your AWS account can also help in confirming if there are any multiple RespondToAuthChallenge API executions taking place.Additionally, the general flow of API and Lambda invocation in terms of custom challenges should look like below -InitiateAuth Api call with CUSTOM_AUTH authflow will invoke the DefineAuth trigger.DefineAuth trigger will then invoke CreateAuth trigger.CreateAuth trigger sends the response to DefineAuth trigger, which then invokes verifyAuth trigger.VerifyAuth trigger confirms the challenges and send the response to DefineAuth trigger, this trigger then generates token or prints error based on the response received.Thirdly, if above suggestion doesn't help, we will need to investigate your Lambda (considering you mentioned that it's not even hitting them - this would require checking the configurations of Lambda itself first to make sure that enough permissions [1] are provided for Cognito to be able to invoke them), the flow of your API calls, and test CLI vs application behavior to isolate if the issue exists in just the application end, or if it is consistent everywhere else. Also, if required capture the service request IDs (can be procured using HAR logs [2] for your app or CLI --debug [3] capabilities) for those failing Cognito API calls (RespondToAuthChallenge) to check for the issue.However, please do not post any any of the above sensitive information about your accounts or it's resources over this re:Post channel since this is a public platform. Therefore, I would like to request you to log a support case directly with our Premium Support team if you require any further assistance.As always, feel free to reach back out with any further questions or concerns in the meantime!References:[1] https://docs.aws.amazon.com/lambda/latest/dg/access-control-resource-based.html[2] https://aws.amazon.com/premiumsupport/knowledge-center/support-case-browser-har-file/[3] https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.htmlCommentShareSUPPORT ENGINEERYash_Canswered 5 months ago"


Question: Serverless lambda rest api not updating database properly

"I am invoking my lambda REST api to update tables in dynamo db but it only updates the database after the second time its invoked. e.g - 1st call I post "some data" to lambda api it doesn't do anything but 2nd call I post "other data" it will process the first call I made("some data"). If I call lambda the third time then it will process "other data". If I wait between 1st and 2nd call the "some data" is lost and never processed.Everything works fine when I run the server locally. I am using api-gateway, lambda and express serverless-http packages. I have tried logging before and after the db calls are made looks like the code is executing but the database is not updating. What am I doing wrong?FollowComment"

Answer: "3Accepted AnswerAs you are using node.js I think the issue is that the function returns before the function finished updating the database. Once the function returns, the Lambda execution environment is frozen and it will be unfrozen when the next request is received.You should either update the DB using a synchronous mode, i.e., use await when updating the DB, or make sure that the context.callbackWaitsForEmptyEventLoop is set to true. See more information here.CommentShareEXPERTUrianswered a year ago"


Question: How can I configure a custom CloudWatch alert for my Ec2 instance ?

"How can I configure a custom CloudWatch alert i.e. (Disk, Memory) for my Ec2 instance (Linux / Windows) without agent installation & any scripts over the system?FollowComment"

Answer: "1Without an agent/application or scheduled script there is no way for the operating system metrics to be sent to cloudwatch  and therefore no way to set an alert.Most commonly customers install the cloudwatch agent to send OS metrics + custom metrics to cloudwatch. If using an AMI with SSM already installed the agent can be deployed via SSMCommentShareEXPERTPeter_Ganswered 16 days agoEXPERTkentradreviewed 16 days ago1To create custom metrics without using CloudWatch Agent, you must use AWS CLI or AWS SDK to create a script to fetch the necessary values.If the only metrics you want to get are disk and memory, it would be easier to include CloudWatch Agent than to create a script.CommentShareEXPERTRiku_Kobayashianswered 16 days agoEXPERTkentradreviewed 16 days ago"


Question: Export dynamodb data as Excel files

"Hey,We just changed to AWS Dynamodb and i was wondering what is the best way for exporting the data to an Excel file. This should be automated later on . Is it with AWS S3 or with some python programming? Can't find a clear solution online so thats why i'm asking.Any suggestions?Thanks!FollowComment"

Answer: "1Depending on what is your goal, I would look into 3 options:If you want to just dump, a quick script in language you prefer would do the jobIf you want to dump periodically, I would suggest to make Lambda out of it - here is a result of quick googling with working example: https://medium.com/@jrodl3r/exporting-dynamodb-data-to-excel-using-lambda-with-s3-on-aws-e6f873681c77If you need to work dynamically with the given data, let`s say to slice & dice, filter, etc analyse it by users - consider using AWS Quicksight with DynamoDB as data source.CommentSharewieshkaanswered 7 months ago1DynamoDB does not provide the ability to export the entire table to CSV, only a small sub-set.The best way I have found is using Athena's capability to export to CSV, this is done using theAthena DynamoDB ConncectorThe next suggestion approach is to come up with your own logic/script to export the table to CSV using Lambda and Event Bridge to periodically kick off the job. There are many tools available to assist you with converting JSON to CSV and even tools to do the entire process for you such as:NPM dynamodb-to-csvAnd finally, if your table is really large you can use AWS Glue to run a periodic ETL job which will read from DynamoDB and export to CSV in an S3 Bucket:AWS GlueOverall it depends on your specific use-case needs on which option to use.CommentShareEXPERTLeeroy Hannigananswered 7 months ago0I think the second option is the one we go with. Just dumping periodically. But this is more for sending it through email? is their a way to store it in ... a bucket?CommentSharerePost-User-1452554answered 7 months agowieshka 7 months agoThat`s depending on what you implement in the given Lambda code - from there you can both send to e-mail (via AWS SES), and as well park in the S3 bucket. Or, send an e-mail of the new dump being created on S3 with a pre-signed download URL for convenience.Since you mentioned Python - boto3 is your friend.Share0I was planning to create a lambda with an Event bridge. And storing the file on aws S3. Should that be the best way? So we can download Excel files from the bucket or can we acces Excel in aws it self?CommentSharerePost-User-1452554answered 7 months agoLeeroy Hannigan EXPERT7 months agoYou can use comments to post comments under posts to avoid creating multiple answers.Yes, that would be the best way in my opinion, there is multiple libraries that allow you to convert JSON to CSV and then store in S3. You can then share pre-signed URL's with others who can then download your files.Share"


Question: "Can't connect with Elastic Beanstalk CLI: "C:\Users\Danny B>eb init 'eb' is not recognized as an internal or external command, operable program or batch file.""

"I have succesfully installed Elastic Beanstalk CLI on my local Windows machine following instrucions here:https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install-advanced.htmlI have also added these paths in my local windows environment variables:%USERPROFILE%\AppData\Roaming\Python\Scripts%USERPROFILE%\AppData\Local\Programs\Python\Python311\Scripts%USERPROFILE%\AppData\Roaming\Python\Python311\ScriptsEhe error I'm getting in my command line when I typeC:\Users\Danny B>eb initis this:'eb' is not recognized as an internal or external command,operable program or batch file.Can anybody help?Thanks!FollowComment"

Answer: "0HiIt seems that even though you have configured the Paths they have not loaded into your terminal environment.If you are using Powershell you can use the following command to reload$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User") Another way to reload is to simply restart the machine.Hope this helpsCommentShareBisinaanswered 11 days agorePost-User-1408394 11 days agoI restarted the machine, but it didn't help. I don't understand how to translate what you wrote for my situation.Share"


Question: IP access restriction on S3

Does AWS S3 allow you to configure a list of allowed IPs that can be uploaded to buckets?FollowComment

Answer: "0Hi,Answering to your question, yes you can restrict access to S3 bucket to specific IP address or range using a bucket policy.The following example bucket policy allows traffic to the bucket when the request is from specified IP address or range ( aws:SourceIp):{"Id": "SourceIP","Version": "2012-10-17","Statement": [{"Sid": "AllowSourceIP","Principal": "","Action": "s3:","Effect": "Allow","Resource": ["arn:aws:s3:::BUCKET-NAME","arn:aws:s3:::BUCKET-NAME/*"],"Condition": {"IpAddress": {"aws:SourceIp": ["11.11.11.11/32","22.22.22.22/24"]}}}]}Note: Above policy allows s3 full access to all users from the specified IP address or range, but it doesn't deny all access from outside the IP address or range. If a IAM user/role from the same account is authenticated and has s3 access via IAM policy, this policy still allows the IAM to access the bucket from outside the specified IP address or range.For example policy that denies permissions to any user to perform any S3 operations on objects in the specified S3 bucket unless the request originates from the IP address or range specified in the condition, please refer following documents.[+] https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-use-case-3[+] https://aws.amazon.com/premiumsupport/knowledge-center/block-s3-traffic-vpc-ip/Regards,PavithraCommentSharePavithra-AWSanswered 2 years ago"


Question: "Any option for large, chunked responses from Lambda"

"I'm building a data access application that sometimes needs to return large responses. I would like the code to run in a scale to zero server less system like Lambda but both API Gateway and Application Load Balancer Lambda integrations do not seem to support chunked responses and if I build my own proxy in front I will still have the 6MB limit to contend with.Other Lambda limits (running time and memory) have been increased over the years, why are we stuck with the 6MB limit?FollowComment"

Answer: "4I can't talk to why the limit hasn't been increased. But the most common solution is to place large objects into S3 and then return a presigned URL to the caller to access that object. You could also take advantage of CloudFront and the global AWS backbone network just by using those services.You need to make sure that the credentials used to generate the presigned URL are long-lived enough for the client to retrieve the object - if it is a one-time retrieval then the credentials from the Lambda function are usually fine; but many implementations use a separate set of credentials so that if the Lambda function credentials expire before the client has retrieved the object then it still works.You'd also presumably need to clean up the objects in S3 afterwards - this can easily accomplished with bucket lifecycle policies.You could implement a chunked transfer mechanisms using API Gateway and Lambda; it would require you to store the object somewhere permanent anyway (because there's no guarantee that separate calls to retrieve chunks would hit the same Lambda invocation) - so there's a bunch of work to be done if you went down that path. I wouldn't recommend it but it can be done.CommentShareEXPERTBrettski-AWSanswered a year agoEXPERTUrireviewed a year agoarnab a year agoThis solution sounds like the right approach.A note on the credentials used to generate the S3 pre-signed URLs: As @Brettski mentions, if the credentials expire before the client has retrieved the object, this mechanism will result in your client failing to retrieve the result. Since you as a customer can't control the lifecycle of Lambda functions, you don't have control over how long these credentials will last. It could expire a second after the S3 presigned URL is generated, if the Lambda container is recycled.Because of this, a separate set of credentials must be used in this case.ShareLaurence Rowe a year agoThe chunked transfer mechanism I'm looking for is an equivalent to Transfer-Encoding: chunked, a basic HTTP feature with no need for separate calls.It can be awkward to determine whether a response will exceed the 6MB limit and then require a full roundtrip redirect to the client (because API Gateway does not to my knowledge support an equivalent to Nginx's X-Accel-Redirect.)I guess I just want to be able to run an HTTP server in a scale to zero container as I can on Cloud Run. It's frustrating there's not yet an equivalent in AWS.Share"


Question: Unable to connect to instance from a newly created ami: Failed to start Service for snap ap…amazon-ssm-agent.amazon-ssm-agent

"Hello,I created an ami on from an ubuntu 18.04 instance. AMI was generated without any errors. However, when I launch a new instance from the ami, instance reachability check fails and I unable to ssh into the instance. After looking at the logs, I got the following error message:...[FAILED] Failed to start Service for snap ap…amazon-ssm-agent.amazon-ssm-agent.See 'systemctl status snap.amazon-ssm-a….amazon-ssm-agent.service' for details.Failed to start Service for snap ap…amazon-ssm-agent.amazon-ssm-agent.I cannot the command as the instance is not accessible by ssh. Can anyone please help? Thank you.FollowCommentAWS-User-2166362 a year agoDid you ever find out the solution to this? I am receiving the same error.Share"

Answer: 0Can you use the AWS Serial Console to access the instances? Then you'll be able to look at the log file or run the command above to determine where the issue is.CommentShareEXPERTBrettski-AWSanswered a year ago


Question: AWS IoT Core and Fleet Indexing connected:false response

"Hi,while I try Fleet Indexing, I found I always get a response of connected:false while the device is connected to the Iot core and publish message to iot core, anyone has faced this, thank you.us-west-2aws iot describe-index --index-name "AWS_Things"{"indexName": "AWS_Things","indexStatus": "ACTIVE","schema": "REGISTRY_AND_SHADOW_AND_CONNECTIVITY_STATUS"}aws iot search-index --index-name "AWS_Things" --query-string "pm"{"things": [{"thingName": "pm","thingId": "{thingId}","thingTypeName": "demo-iot","thingGroupNames": ["test_group"],"shadow": "{"metadata":{},"version":1}","connectivity": {"connected": false}}]}FollowComment"

Answer: "1Accepted AnswerIf your client is using a different client ID from Thing Name when connecting to AWS IoT, the connectivity status of your "thing" will not be indexed by Fleet Indexing.CommentShareEXPERTMassimilianoAWSanswered 4 years agoAWS-User-3949958 a year agogenius thanksShare"


Question: Apollo subscription with AWS Lambda and Mongodb

"Hi team,I'm implementing apollo subscription with aws lambda and mongodb. I found one documentation for the same but with dynamodb. https://github.com/aws-samples/apollo-server-on-aws-lambdaCan anyone help me to guide me to how to implement with mongodb. Actually I'm confused on cdk. The github link uses cdk. So cdk is required to create subscription env with aws lambda? I found @aws-cdk/aws-dynamodb library but not found same library for mongodb. Can anyone help me with this? ThanksFollowComment"


Question: Will We Ever Get a Forum?

"So will we ever get a developers forum like the other game engines ?You know I really like just easiness of normal forums, such as Subscribe to topic buttons normal font sizes etc... :)So is there a plan for such a thing in the future ?ThanksFollow"

Answer: "0Accepted AnswerHeya @REDACTEDUSERThis is certainly a good topic among many. Removing friction and improving the customer experience is top priority for 2018. We'd love to hear more about what specifically you are looking for -- /Community, please feel free to join in on this! The more we know, the more we can improve on this quality bar.SharerePost-User-5738838answered 5 years ago0Great to hear. Fingers crossed for this year then. :)SharerePost-User-4807048answered 5 years ago0@REDACTEDUSERWe also need a nice overhead menu bar with a "Marketplace" and "Learn" and "Community" links across the top/header. We also need a Marketplace, where users can post their assets for the community to use/share.Lumberyard needs a REAL Forum, and also needs a "standardized" menu bar (similar to this one here) that shows "Explore" (with sub-menu navigation pages/links for "Features", "Showcase", "Roadmap", "News" and "Indie Development Fund"), and then "Marketplace" (with a link to the "Marketplace" storefront where users can post their community created content (for users to download for free, or for sale, etc.), a place were users/developers can search for free community-created Assets, plugins/gems, scripts, sound effects, Full Projects, etc. A "Learn" (with sub-menu navigation links/pages for "Tutorials" and "Documentation"). A "Community" (with sub-menu navigation links/pages for "Gallery", "Forums", "Answers" and "GitHub").Create the forums/community/Lumberyard site with a black background (instead of white), for uniformity, and so if you are surfing/viewing the forums (on your mobile device) you can save/conserve energy. Using a white background on the site really eats up the battery life on mobile devices. Or at least have it so you can "toggle" between white and black backgrounds (with black being the default). That way if users like white, they can "toggle" back to white, very easily. Just have a "Lights On" and "Lights Off" toggle (for black and white backgrounds), with black being the default.Also make sure that it supports HTML and/or BBCode, and also a nice WYSIWYG Editor (similar to WordPress). With the ability to use HTML, BBCode, Smilies/Emoticons, etc. Also have the ability to create "User Groups" (users that divide the community into manageable sections/communities/groups with "like minded" interests), especially for projects/features (i.e. Flight Simulator, Train Simulator, Truck Simulator, Cars/Racing Simulator, etc.). Where users can work together, share code, have discussions, and Community Volunteers can "volunteer" to help/moderate various community "User Groups" (sub-groups) [with "public", or "private" or "hidden"/closed memberships]But we REALLY REALLY REALLY need Forums with Community Volunteers/Group Leaders, where we can create our own private (or even "hidden"/closed User Groups) with mailing lists, group discussions, and users can request to "join" a public or private group (hidden groups are "invite" only). Where community group members can discuss Game Design, plugins, tools, models, art, UI, audio, animation, materials, visual scripting, UI, etc.We also need the ability to send PM's (Private Messages), as well as have mailing lists. So developers can discuss topics/discussions (with a group/team) via Mailing List.Give "Teams" the ability to create their own "User Group" and their own various "Mailing Lists" (Dev Mailing List, Support Mailing List, User Mailing List, etc.)Also add the ability for "Friends" or "Foes" list (where we can add people/users to our "Friends" list, or block users in our "Foes" list). Also have the ability to see users (who's currently online) and see their online status, and send them private messages (messenger/PM's).Also have "template" support, so that teams can create their own "templates" and customize the look/feel of the various sub-groups.If you need help with this, let me know... but this would be a good "example" of a nice Community Forum (easy to use, well thought out, and has all the easy to use navigation and forum features that we need).Use either vBulletin or phpBB (Crytek uses phpBB). I think phpBB would be the best for a Community Forum.SharerePost-User-9149768answered 5 years ago"


Question: AWS Lambda Application Runtime Limitations

"A customer wants to use AWS Lambda Applications (not only functions) with Python runtime, but as they mentioned and I could double check internally, it seems only NodeJS support is offered (at least through the AWS Console).However, I can’t find any documentation explicitly talking about that limitation. Do you know if that is a known limitation and where I can find more about it?If you try to go now to Lambda > Applications > Create Application > Create/Author from scratch(or use this link) -> you get only NodeJS RuntimeFollowComment"

Answer: "0Accepted AnswerThere are only examples that use node.js currently with Lambda Applications. If you use Cloudformation to deploy Lambda, then they appear as applications, so can make your own. Optionally use the Serverless Application Repository to do this. Currently the "Authohr from Scratch" option only includes Node.js. This can be done via SAM, Cloudformation, serverless, CDK, etc.CommentSharesang_kanswered 3 years ago"


Question: How to set/reset password for user in Identity store using the Java SDK?

"Hello,I was able to create a user in the IAM Identity Center using the Java SDK with the IdentitystoreClient and CreateUserRequest. The user gets created in the Identity Center directory.I need to set (in future reset) the password for this user using the API. I am unable to find any examples or API to accomplish the same. Can this be done via API call? If yes, then please provide some example code/reference to the API class.Is there an API to assign the user to a group?Thanks,Binfer.FollowComment"

Answer: "0Hi Binfer,To set password via sdk look at this reference: https://docs.aws.amazon.com/IAM/latest/APIReference/API_UpdateLoginProfile.htmlto add user to a group: https://docs.aws.amazon.com/IAM/latest/APIReference/API_AddUserToGroup.htmlCommentShareEXPERTalatechanswered 4 months agoBinfer 4 months agoThanks, but that works only for IAM and not IAM Identity center. They both appear to have different API's. The identity server client had no UpdateLoginProfile mechanism and the IAM UpdateLoginProfile does not work with users created with the identity center API as identityStoreId is needed for API calls. Ref: https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/identitystore/IdentitystoreClient.htmlSharealatech EXPERT4 months agoI see. for adding user to a group you could try this? https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/identitystore/IdentitystoreClient.html#createGroupMembership(software.amazon.awssdk.services.identitystore.model.CreateGroupMembershipRequest)To rest password I am not able to find a proper methodShare"


Question: 404 Nginx Error with AWS Meta Capi

"Hi,I set up the Facebook/ Meta Conversion API with deploying it to AWS. I followed all the steps of deploying it and  the stack was sucessfully created. The deployment urlAlso the dns was setup sucessfully. If I want to enter the Capi Dashboard, it can't reach the server and if I use the the ConversionsApiGatewayInstanceURL in the CloudFormation Output it gives me a 404 Not found Nginx Error.I also can't connect to the ec2 Instance because it tells me that I use the wrong credentials (I use the user that I found in the Userdata). Session Manager ist not setup, ssh client connection timed out, EC2 Instance Connect gives me a "failed to connect to your instance" and Ec2 serial console doesn't work with my credentials.What is the problem and / or how can I have access to my EC2 Instance so I can debug the Nginx Error? Sadly I am not so familiar with AWS products.Kind regards and thank you!FollowComment"

Answer: 0Having the same issue and I also followed the same tutorialCommentSharerePost-User-5713609answered 8 months ago


Question: Glue job fail many workers

"Hi team,I have a glue job that reads CSV files S3 to MySQL RDS. I set up** 2 workers** on my glue job,the job fails for one big file with the  flowing error :An error occurred while calling o122.pyWriteDynamicFrame. Duplicate entry '123456' for key 'mySqlTable.PRIMARY'I checked on the CSV file in question I have only one single line with the value '123456'  so no duplication on the file.is that error because 2 workers try to insert the same data at the same time? so that It generates the duplicate key error?I tried with 4 workers => same errorI wanted to try with 1 worker but seems the minimum number of workers should be 2 or greater.I truncate my SQL table every time and make sure that's empty before each run.thank you.FollowComment"

Answer: "0Is the data already present in the MySQL table?AWS Glue is based on Spark, as you know, 1 node is the Spark driver and the other nodes are hosting the executors (doing the real works), in you case with just 2 workers you have only 1 executor in your cluster.Even with multiple executor, spark partitions the data without replicating it this means that even parallel writes will never try to enter the same data twice, unless as already mentioned your code has introduced the duplicates.Glue works in Append mode only. no updates, have you checked if that key is, by any chance, already present in the target database?hope this helps,CommentShareEXPERTFabrizio@AWSanswered a year agoJess a year agoI truncate my SQL table every time and make sure that's empty before each run. I tried with 2 workers and 4 workers always the same error.the previous file passes successfully this file cause always this error from glueShareFabrizio@AWS EXPERTa year ago@Jess could you please pose an anonymized snippet of the code generated?I have not experienced that and not sure how to reproduce it.ShareFabrizio@AWS EXPERTa year ago@Jess could you please pose an anonymized snippet of the code generated?I have not experienced that and not sure how to reproduce it.Share0Are you doing some transformation, join, which can produce duplicate data? Is it happening to all the keys or single key only. I would suggest to remove PK or unique constraint and let the job complete once. Then you can evaluate in RDS is it true duplicate or some transformation is generating duplicateCommentShareZahidanswered a year agoJess a year agono, it's direct mapping no transformation nor join it happened on a single key each runShare"


Question: There’s piracy of NBA AND NFL games being live streamed on the Amazon cloud front

Cloud front with more than 20 million views per month alongside child exploitations and it’s all being completely ignored since November 2021 and AWS has control over everything on the cloud front so why wouldn’t they remove that??? AWS even deleted one of my cases regarding this 9494299111FollowComment

Answer: "0Accepted AnswerIt's important that we (AWS) ensure that resources are used in legal ways. However, this forum isn't the right place to report that. Please use our AWS abuse form for this. You can read details about what is illegal/abusive content here: https://aws.amazon.com/premiumsupport/knowledge-center/report-aws-abuse/CommentShareEXPERTBrettski-AWSanswered a year agoMelaniee a year agoI have done that and made a compliance report and brought it to their attention since November and it’s completely ignored. AWS deleted my case regarding it (9494299111)Share0Also amazon Trust and safety team lied about having ever received any reports of that nature when they have multiple times prior when I emailed them asking why they lied ignore me so yes they are fully awareCommentShareMelanieeanswered 10 months ago"


Question: CEDR Project-id

Where do i find the project-id of the CEDR project which is required to run the upgrade assessment tool while migrating AWS EDR?FollowCommentAWS-User-2560483 2 months agoDid you upgrade CEDR to DRS successfully?Please share the steps performed as for me below mentioned document not fetching successful results:https://docs.cloudendure.com/#Configuring_and_Running_Disaster_Recovery/Upgrade_CEDR_to_DRS/Upgrade_CEDR_to_DRS.htm#Upgrading_from_CEDR_to_AWS%C2%A0DRS%3FTocPath%3DNavigation%7CConfiguring%2520and%2520Running%2520Disaster%2520Recovery%7CUpgrading%2520from%2520CEDR%2520to%2520AWS%25C2%25A0DRS%7C_____0Thank you.Share

Answer: "0you can find the --project-id  from the CEDR URL once you are in that project.For example:https://console.cloudendure.com/#/project/XXXXXX-7184-4736-XXXX-84b8XXXXX84aa/machines.This is the Project Id here is XXXXXX-7184-4736-XXXX-84b8XXXXX84aa.Which can be used to run the upgrade assessment tool as provided below,python3 cloudendure_to_drs_upgrade_assessment_tool.py --api-token XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX --project-id XXXXXX-7184-4736-XXXX-84b8XXXXX84aaCommentShareAWS-User-4748647answered 2 months ago"


Question: FSx for Windows appears to be blocking WinRM

"I have stood up FSx for windows and I can connect to the shares and manage them fine. I have include the necessary ports in the SG to manage the service from the documentation. However, when I tried to connect via WinRM on port 5985 to the FSx instance, it fails to connect.  I can connect via SMB 445 when running a test and we confirmed that by connecting to the service and creating shares. SMB and WinRM are opened in the same SG for FSx, so it makes no sense. All ports are allowed outbound and the NACL is still set to default all open in both directions. What is even more strange is that a connectivity analysis test says the port is open between the same instance and ENI for FSx (445 and 5985), but from inside the instance it is not open when doing a test. Local firewall is disabled and all ports open on instance SG and NACL for the same local private subnet (FSx and instance). This is very bizarre. Anyone else run into this?https://docs.aws.amazon.com/fsx/latest/WindowsGuide/self-manage-prereqs.htmlFollowComment"

Answer:
"1Hi, thanks for sharing the output of the Test-NetConnection command.This tells us that the port is open and the security group attached to FSx is correct.Please could you share the command that you are using to connect to the FSx remote powershell endpoint?Alternatively, please try this example command:$FSxWindowsRemotePowerShellEndpoint  = "ReplacewithyourRemotePowerShellEndpointDNSname"Invoke-Command -ComputerName $FSxWindowsRemotePowerShellEndpoint -ConfigurationName FSxRemoteAdmin -ScriptBlock { Get-FSxSmbOpenFile}One of the key differences between regular PS remoting and FSx, is that you need to specify the -ConfigurationName FSxRemoteAdmin parameter.Let me know how it goes :)CommentSharePeter-John Manswered 8 months agoAWS-User-2639757 8 months agoI was able to get it working. I believe I wasn't using that Windows Remote PowerShell Endpoint to connect to the instance via PowerShell and that was my issue.SharePeter-John M 8 months agoAwesome, thanks for the update, glad its working!Share0Hello there!I notice that you are having trouble connecting to your FSx instance via WinRM on port 5985.From the notes you have mentioned that you have configured the instance for testing via SMB port 445 and opened the SG for SMB and WinRM. You also mention that all ports are allowed outbound and NACL set for both directions.However I would like you to please note a couple of factors in regards to your case:For starters you have to ensure that you can connect to the Windows Remote PowerShell Endpoint.
The main point that I noticed is that there was no mention of the file system’s inbound rule that will grant access for remote PowerShell connections. It is mandatory that the SG group in the inbound direction to allow traffic on port 5985 so a remote Powershell session is established. This will allow traffic from your FSx file system, which is what is currently blocking the connection[1]. Please also check that you are not using the addresses 198.18.0.0/15 space in your VPC, which is referenced in the documentation[2]. For more information about inbound rules and associating a security group[3] have a look at the references below.References:[1] https://docs.aws.amazon.com/fsx/latest/WindowsGuide/remote-pwr-shell.html[2] https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_getting_started_prereqs.html[3] https://docs.aws.amazon.com/fsx/latest/WindowsGuide/limit-access-security-groups.html#fsx-vpc-security-groupsCommentShareAntonioanswered a year ago0Hello Antonio, when testing the endpoint and port using the command below, it comes back with True (I have removed the network-specific info). I'll need to go back and test it again. I have been just using the File Server role to manage the shares, etc. for now:PS C:\Users\username> test-netconnection {endpoint.domain.com} -port 5985ComputerName     : {endpoint.domain.com}RemoteAddress    : XXX.XX.XXX.XXXRemotePort       : 5985InterfaceAlias   : EthernetSourceAddress    : XXX.XX.XXX.XXXTcpTestSucceeded : TrueCommentShareAWS-User-2639757answered a year ago"


Question: Unable to deploy model

"Hi,I am in the early stages of building a CV demo using Qualcomm chipset and Greengrass V2.My app contains the following components components.My system is unable to do any inference though it is able to send empty MQTT packages to the cloud.I am seeing the following error when I am deploying a model using the variant.DLR'''2022-10-10T20:21:22.232Z [WARN] (Copier) variant.DLR: stderr. ++ PATH=/greengrass/v2/work/variant.DLR/greengrass_ml/greengrass_ml_dlr_venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.232Z [WARN] (Copier) variant.DLR: stderr. ++ export PATH. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.232Z [WARN] (Copier) variant.DLR: stderr. ++ '[' -n '' ']'. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.233Z [WARN] (Copier) variant.DLR: stderr. ++ '[' -z '' ']'. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.233Z [WARN] (Copier) variant.DLR: stderr. ++ _OLD_VIRTUAL_PS1=. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.233Z [WARN] (Copier) variant.DLR: stderr. ++ '[' 'x(greengrass_ml_dlr_venv) ' '!=' x ']'. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.234Z [WARN] (Copier) variant.DLR: stderr. ++ PS1='(greengrass_ml_dlr_venv) '. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.234Z [WARN] (Copier) variant.DLR: stderr. ++ export PS1. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.234Z [WARN] (Copier) variant.DLR: stderr. ++ '[' -n /bin/bash -o -n '' ']'. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.235Z [WARN] (Copier) variant.DLR: stderr. ++ hash -r. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.235Z [WARN] (Copier) variant.DLR: stderr. ++ python3 -c 'try:. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.236Z [WARN] (Copier) variant.DLR: stderr. import dlr. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.236Z [WARN] (Copier) variant.DLR: stderr. print(dlr.version). {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.236Z [WARN] (Copier) variant.DLR: stderr. except Exception as e:. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.237Z [WARN] (Copier) variant.DLR: stderr. print(e)'. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.647Z [WARN] (Copier) variant.DLR: stderr. + get_dlr_version=1.6.0. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.647Z [INFO] (Copier) variant.DLR: stdout. Skipping DLR installation as it already exists.. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.648Z [WARN] (Copier) variant.DLR: stderr. + [[ 1.6.0 == \1.\6.\0 ]]. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}2022-10-10T20:21:22.649Z [WARN] (Copier) variant.DLR: stderr. + return 0. {scriptName=services.variant.DLR.lifecycle.install.script, serviceName=variant.DLR, currentState=NEW}'''Any help would be appreciated.Thanks,FollowComment"


Question: MWAA: how to install python requirements behind proxy

"Hello everyone,we've launched a private MWAA environment. We are able to access the UI, but we're having some trouble installing our python requirements. MWAA picks up the requirements file from S3, but runs into a timeout when trying to install the python packages.This is expected, because we're behind a proxy, so here goes my question: how do we tell MWAA to use our proxy while installing our python dependencies?This is what our CloudWatch log stream (requirements_install_ip*) tells us:"WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None))after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnectionobject at 0x7fda26b394d0>, 'Connection to pypi.org timed out. (connect timeout=15)')' "FollowComment"

Answer: "0Hi,Please see https://docs.aws.amazon.com/mwaa/latest/userguide/best-practices-dependencies.htmlThanks!CommentShareJohn_Janswered 2 years agoAWSuser a year agoHi, we are trying to setup MWAA in private mode. We have included the requirements.txt as per option 1 in above link but we are still getting error. The pypi.org is accessible from other EC2 on the same subnet so there should not be any network/FW issues. Does anyone has any idea what might be wrong or what other configuration may be needed to make this work? Happy to provide additional details if required.Share"


Question: Organisation ConformancePacks: s3 bucket policy for delegated administrator?

"I am following the documented procedure to use a delegated administrator to deploy organisation-wide conformance packs:https://aws.amazon.com/blogs/mt/deploy-aws-config-rules-and-conformance-packs-using-a-delegated-admin/When using the delegated account to put the packs I get an error:An error occurred (InsufficientPermissionsException) when calling the PutOrganizationConformancePack operation: Insufficient permission to get S3 bucket ACL for awsconfigconforms-company-orgThe only way to make it work so far is to add this policy statement to allow the delegated account access to the ACL on the bucket:            "Sid": "DelegatedAdministratorAllowGetBucketAcl",            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::123456789:root"            },            "Action": "s3:GetBucketAcl",            "Resource": "arn:aws:s3:::awsconfigconforms-company-org",            "Condition": {                "StringEquals": {                    "aws:PrincipalOrgID": "o-123456"                }            }        }Is this how I can resolve the permission issue? Or is there a better way to give cross account access to the bucket in the master account?Also: is it expected that the conformancePacks are created in the delegated account and not in the master account? aws configservice describe-organization-conformance-packs lists them for the delegated one, I guess that's expected as I'm delegating?FollowComment"

Answer: 1Delegated Administrator with AWS Organizations supports a list of compatible AWS services.Take a look at How can I provide cross-account access to objects that are in Amazon S3 buckets on granting permissions cross-account.CommentShareNick_Canswered a year agogduemlein a year agoAWS Config is listed as a compatible service.AWSServiceRoleForConfigMultiAccountSetup is a service-linked role - so I cannot change it's permissions/policies.I guess I can modify the permissions/policies of the user executing PutOrganizationConformancePack - however that's a running through SSO in our case. So I believe my user is assuming a role that's then assuming the service linked role.I'll need to do more reading on that topic.Share


Question: Filters for CloudWatch Dashboard widgets

Is there a way where I can add a filter for a log table in cloud-watch dashboard to select the service and get the detailsFollowComment

Answer: 0Have a look at Custom Widgets for CloudWatch Dashboards. You can achieve what you want with one. There is a sample widget called Logs Insights query editor that can be deployed directly from within a CloudWatch Dashboard that should give you a good starting point.CommentShareAmazon-CloudWatch-01answered 9 months ago


Question: Origin SSL check fails with the NEW Letsencrypt Issuer

"Certificates on the origin from Letsencrypt worked fine for years now.Yesterday one of the ssl certificate of a websites got renewed successfully.But now cloudfront makes a 503 error only for this website (one of many)other website with an older certifacte from letsencrypt still working fine.The only difference is that LetsEncrypt now signes with a new issuerOld issuers was: "Let's Encrypt Authority X3"New issuer is: "R3"Issuer Certifacteshttps://letsencrypt.org/certificates/There are two Intermediate certifiactes,this one make the issue:https://crt.sh/?id=3479778542Issuer Statement:https://letsencrypt.org/documents/isrg-cps-v3.0/It seams that cloudfront dont trust the certifactes from the new letsencrypt issuer "R3"Edited by: Zetanova on Dec 5, 2020 9:00 AMFollowComment"

Answer: 0My hosting system virtualmin only renewed the certificate but not the Intermediate certificate.After updating the Intermediate certificate manually it is working againCommentShareZetanovaanswered 2 years ago


Question: Use Connect Contact Lens to analyze pre recorded phone conversations

"I have a number of recordings of phone conversations that were made using Amazon Connect a couple of years ago, that I want to analyze using Contact Lens.  I have reviewed the APIs for both Connect and Contact Lens and can not find any methods that would let me do this.  I am looking at the .net sdk documentation.A thorough google search has not returned anything.Asking open ai's chat gpt - returned a function that didn't align with the current .net sdks, having a mismatch of methods and models taken from the Connect and Contact Lens classes.FollowComment"

Answer: 0Contact Lens for Connect will only kick in for new incoming calls.  It cannot go back and analyze.Alternative is to use AWS Transcribe/Comprehend manually on the call recording to get the analytics that you need for them.   But these are outside of Connect/Contact Lens processing now.CommentShareClarenceChoianswered 3 months ago


Question: Domain Point to one static ip in aws

"Hai,I have purchased the domain from GoDaddy, I have deployed an application with that domain.I just want to point that domain into static IP and that static IP i want to configure into juniper firewall.When the juniper got down application should not impact.Please suggest.I tried route 53 to create a hosted zone.FollowComment"

Answer: "1You can create an Elastic IP address and then create an 'A' record for that IP with your domain in Route53. You can find how to set this up here - https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-ec2-instance.html.The Juniper firewall will be your EC2 instance.CommentShareAjitanswered 2 months agoMahendra Kumar V 2 months agoHai,My application deployed on premisses with the mentioned domain.That domain point to AWS.Share1See the instructions here: Making Route 53 the DNS service for a domain that's in use. Step #7 is where you point to the NS servers in Route 53 at GoDaddy.CommentShareEXPERTkentradanswered 2 months ago"


Question: MediaLive S3 Put / Set different filename for HLS manifest and HLS Segments

"Hello All,We are trying to migrate our existing transcode workflow over to AWS Media Live via storing the HLS segments into S3 directly for our platform to pickup. Everything is stored in the same directory in S3. When I am setting this up as expected in Media Live I don't see a configuration that would result in the main manifest having a distinct name from the child manifest and ts segments.Current ResultMain Manifest/desktop-playlist.m3u8Child Manifest/desktop-playlist-1228k.m3u8/desktop-playlist-2128k.m3u8TS Segment/desktop-playlist-1228k-00000.ts/desktop-playlist-2128k-00000.tsDesired resultMain Manifest/desktop-playlist.m3u8Child Manifest/file-1228k.m3u8/file-2128k.m3u8TS Segment/file-1228k-00000.ts/file-2128k-00000.tsPlease let me know if this is possible or if anyone has any suggestions to try.Thanks,JayFollowComment"

Answer: "0Accepted AnswerTo answer my own question, I have found a work around. I set the HLS output group to only output the child manifest and ts segments. Since our desktop-playlist content is persistent we are going to create it externally and place it into the directory as it is created.Alternatively, we may just have medialive create the Main mainfest and rename the file at time of creation to our existing naming structure.JayCommentShareKaFronanswered 3 months ago0MediaLive does offer the ability to create sub directories for each rendition in the Adaptive Bit Rate stack. This will write the top level manifest into the top directory, then create sub directories for 'Stream', where the child playlist and segments for that rendition are stored.As an example for an HLS Output Group, select the top level Output Group, then expand the Location drop down and under Directory Structure choose SUB_DIRECTORY_PER_STREAM. The Stream sub directories will take their name from the  Name Modifier defined in the individual outputs (ex. _2128k)It is not advised to attempt to create sub directory names that end in .m3u8 or .ts. Those are extensions reserved for actual components of the Streams. This may cause confusion.CommentShareMike-MEanswered 3 months ago0Thanks for the quick response!My goal is to have everything in the same directory, but have the Main manifest have a different name than the child manifest and TS segments.Currently you define the HLS Rendition Group you give it the path and the name pre-fix for that manifest. This is also where it derives the filename prefix for the child manifests and ts segments.So currently I have the below configuredHLS group destination As3:/directory/desktop-playlistHLS outputOutput 1 - Name Modifier (-5928k)Output 2 - Name Modifier (-3128k)etcThis results in this/desktop-playlist.m3u8/desktop-playlist-5928k.m3u8/desktop-playlist-3128k.m3u8/desktop-playlist-5928k-00000.ts/desktop-playlist-3128k-00000.tsI would like to make a configuration change that would result in the below/desktop-playlist.m3u8/file-5928k.m3u8/file-3128k.m3u8/file-5928k-00000.ts/file-3128k-00000.tsLet me know if that makes is clear what I'm trying to adjust.Thanks,JayCommentShareKaFronanswered 3 months ago"


Question: Feature request: Support for DynamoDB Transactions in AppSync

"Could a future build of AppSync support Dynamodb transactions directly in VTL. Possibly for batch/pipeline resolvers.Rolling back partially completed requests is a huge pain. Given that DynamoDB now supports transactions, when do you think we will see AppSync making use of this functionality?FollowComment"

Answer: "0Accepted AnswerHi,Thanks for the feature request. Could you +1 it at https://github.com/aws/aws-appsync-community/issues/14 please?Regards,RohanCommentShareAWS-User-6514131answered 4 years ago"


Question: Quota increase for Rekognition Stored Video concurrent jobs

"Hello,In the quotas and guidelines for Rekognition, it is mentioned that "Amazon Rekognition Video supports a maximum of 20 concurrent jobs per account". This number is too less for my use case. Is there any way it can be increased by raising a request?In case there cannot be more than 20 concurrent jobs, are the other jobs stored in some sort of internal queue or are they lost?FollowComment"

Answer: "0Hi,Quota increase can be requested in AWS console "Service Quotas > AWS services > Amazon Rekognition".https://console.aws.amazon.com/servicequotas/home/services/rekognition/quotas.Find "Concurrent Amazon Rekognition Video stored video jobs per account".Currently, queuing is not supported, and has to be implemented on user side. If exceeding the concurrent jobs limit, further job submission is blocked from API (eg. StartFaceDetection).Thanks,CommentShareAWS-User-5016049answered a year agoAWS-User-3217907 a year agoOhk, thanks! Could you let me know the max limit of the quota please? Can it be increased to indefinite?Share"


Question: cloud front (ssl) connection issue with ec2.

"I am implementing public ssl via clod front and connecting cloud front to ec2 machine.SSL  implemented successfully but  (Mixed Content) issue showing that the content must be served over HTTPS.in setting of clod front domain section i enabled protocol https but issue is still same on live site (talends.com )issue is :Mixed Content: The page at 'https://talends.com/' was loaded over HTTPS, but requested an insecure stylesheet This request has been blocked; the content must be served over HTTPS.please guide how can we resolve this issue.FollowComment"

Answer: "0Hi, @talends.Check the HTML source you get when you access https://talends.com.Perhaps the warned animate.css, style.css, etc. are not output with absolute paths (starting with http)?This error occurs because the browser accesses HTTP EC2 directly, ignoring HTTPS CloudFront only for stylesheets.If you modify the stylesheet path so that it is output as a relative path, the stylesheet will also be retrieved via CloudFront's HTTPS, so the problem should be resolved.CommentShareEXPERTiwasaanswered 10 months agotalends 10 months agohiall assets path come from cloud front domain name (ec2 public dns).and i select  Protocol  (https)  but on website assets loading  only with http.i can not update path for assets from code becuase path come from cloud front domain name.site open on https but not assets.may be  some setting miss on cloud front for content to load with https.please guide and check screenshots.https://prnt.sc/bOWZWhpV-GGbhttps://prnt.sc/nkEnFEXcrXIkShareiwasa EXPERT10 months ago@talends.The screenshot you posted is of the CloudFront settings screen, but we don't need to look there yet. (You may see it later, but order matters in problem solving.)The first thing to check is the content of the response you get in your browser.Open the target page in a browser to view the source and publish the capture.Sharetalends 10 months agookaythis is issue on browswer console.Mixed Content: The page at 'https://talends.com/' was loaded over HTTPS, but requested an insecure stylesheet 'http://ec2-3-235-241-28.compute-1.amazonaws.com/talends/assets/css/animate.css'. This request has been blocked; the content must be served over HTTPS.https://prnt.sc/ta782DT63KOJhttps://prnt.sc/W6eyTFghUBRnShareiwasa EXPERT10 months agoIs it possible to share the contents of the page displayed by accessing the following with Google Chrome?We need to see the HTTP response content, not the developer tools or the CloudFront console.view-source:https://talends.com/Sharetalends 10 months agoplease checkhttps://prnt.sc/jAOXEfxr7cwshttps://prnt.sc/3ftc6d4iVt2tShareShow 3 more comments0CloudFront doesn't modify the content of the message, unless you use Lambda@Edge. So the references to the EC2 instances domain name is coming from somewhere in your code.So depending on how you generate the HTML pages (if it's CSM or your generate them as part of your Node build) you will need to make sure the base URL is set to your domain. In many cases if you don't customize this the CSM or your Node build will assume to use the name of the machine.So please check with your chosen framework, technology or service how do you customize the base URL so that links to resources and other pages are generated correctly.As related note, I would recommend you protect your custom origin, so that only Cloudfront can connect to it. Use the Security Group to only allow access from CloudFront IP addresses. Please see: https://aws.amazon.com/blogs/networking-and-content-delivery/limit-access-to-your-origins-using-the-aws-managed-prefix-list-for-amazon-cloudfront/CommentShareEXPERTToni_Sanswered 10 months ago"


Question: Reliably archiving many s3 files

"I would like to be able to add many (hundreds of thousands) of small s3 files to a single archive elsewhere on s3.  It doesn't necessarily need to be fast, but it does need to be reliable.  I can stream data through an archiver and back to s3 in a single lambda on a small scale, but since I need to get every single object, at full scale it's a lot to ask from a single lambda.Could I, for instance, use step function to run archiving lambdas against a subset of the files and perform multipart uploads into a single combined archive?Are there any better ways to achieve this sort of thing?FollowComment"

Answer: 0If you can leverage something like Firehose in your application you can stream files to a data lake in S3 and then archive to glacier with lifecycle policies.CommentShareEXPERTRob_Hanswered a year ago


Question: S3 > SQS Event Notification Failure

I've been trying to setup through AWS Management Console an S3 event to send notification to SQS for newly-created objects but I'm always getting an error.Unknown ErrorAn unexpected error occurred.Unable to validate the following destination configurationsNOTE: Both S3 and SQS are in the same region.FollowComment

Answer: "1Hi.You need to attach an IAM policy to the SQS queue to allow notifications from S3.https://docs.aws.amazon.com/AmazonS3/latest/userguide/grant-destinations-permissions-to-s3.html#grant-sns-sqs-permission-for-s3If you have already set this, there are cases where the S3 notification and destination queue settings are incorrect as shown below.https://zenn.dev/fffukken/articles/5c165e8cde8e2bCommentShareEXPERTiwasaanswered a year ago"


Question: "CloudFront set to "HTTPS only", but http will still be served"

"I have a CloudFront (with a registered domain) "connected" to an S3 bucket. The Viewer protocol policy is set to HTTPS only. However, requesting http://mydomain.com will reply with an HTTP answer.Tested on:  securityheaders.comFollowComment"

Answer: "2Hi,try to select Redirect HTTP to HTTPS.https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-viewers-to-cloudfront.htmlCommentShareEXPERTalatechanswered 4 months agoPeter 4 months agoWorks as a workaround.Sharealatech EXPERT4 months agoThanks, if above helped you, I d appreciate an upvote/accepted answer so that also community benefits for clarity ;)Share"


Question: How to import/load pretrained DeepAR model into an estimator for continuous training?

"Hi, I have a trained DeepAR model saved as model.tar.gz in some S3 location. I hope to further optimize it by training for some extra epochs with extra data having same features, like transfer learning. I have read some DeepAR documentation but cannot find a way to import the trained model.I tried using Sagemaker's Estimator's model_uri config, but it reports error for deepar, saying no 'model' channel is acceptable.I am wondering does DeepAR have such functionality or it is still not developed?FollowComment"

Answer: "0Hello,Thank you for using AWS SageMaker. I understand that you are looking for Incremental training using SageMaker DeepAR algorithm where you can add trained DeepAR model saved in S3 location which further optimize your model.Unfortunately at this time, only three built-in algorithms currently support incremental training: Object Detection - MXNet, Image Classification - MXNet, and Semantic Segmentation Algorithm.If you need any further assistance or need detailed investigation please open a support case with AWSCommentShareSUPPORT ENGINEERKartik_Ranswered 4 months ago"


Question: API Gateway Upgraded and API Lost

Did AWS upgrade the API Gateway code last night as I get a new interface and all my API are now gone?FollowComment

Answer: "0Hello:No update occurred at this time that should have resulted in your APIs disappearing. I would encourage you to review your CloudTrail logs to see if/why your APIs were removed.Regards,BobCommentShareEXPERTAWS-User-0395676answered 3 years ago0Thanks Bob .CommentShareRajanGanswered 3 years ago"


Question: Want to remove sign-up option from AWS Cognito default interface

"Hello,I was trying the default Amazon Cognito hosted login page to integrate with my application for authentication purpose and wanted a way to remove the signup option from the default hosted page of Amazon Cognito. I want to add the users from the backend and don't want to allow the users to sign up.Is there any way to do so?FollowComment"

Answer: "0Accepted AnswerGo to your user pool in Amazon Cognito.Under General Settings, go to Policies.Select the option Only allow administrators to create users as shown in the screenshot below and Save Changes.CommentShareRajjat Kumaranswered 4 months agounknown 4 months agoThanks, this helped! :)Share"


Question: How to set termination protection on an Amplify app stack in CloudFormation?

"As a production best practice, I want to deploy my Amplify app with termination protection enabled on its stacks in CloudFormation. How do I do that programmatically?FollowComment"


Question: AWS DMS connected successfully to Redshift created table but not loading data. Connected to S3 but failing at Uploading files to redshift.

"Any help please? Logs is as below:: Failed to upload file '/rdsdbdata/data/tasks/KXHGVEZM7WRYI636BOJOQYTCOIGFGS6AOGE3MNY/cloud/1/LOAD00000001.gzip' to bucket 'dms-in2tox7r7fgrwfjrshecwnp6s4y2uvsre63afzy' as 'redshift-FGKQA6KH2TIZ5J6G4T3NMVU7REZGRBNUYSGQI3Y/KXHGVEZM7WRYI636BOJOQYTCOIGFGS6AOGE3MNY/1/LOAD00000001.csv', status = 4 (FAILED) [1001705] (transfer_client.cpp:514)2022-11-17T14:59:38.000+00:00 2022-11-17T14:59:38 [FILE_FACTORY ]E: Failed to upload </rdsdbdata/data/tasks/KXHGVEZM7WRYI636BOJOQYTCOIGFGS6AOGE3MNY/cloud/1/LOAD00000001.gzip> to <dms-in2tox7r7fgrwfjrshecwnp6s4y2uvsre63afzy/redshift-FGKQA6KH2TIZ5J6G4T3NMVU7REZGRBNUYSGQI3Y/KXHGVEZM7WRYI636BOJOQYTCOIGFGS6AOGE3MNY/1/LOAD00000001.csv> [1000722] (at_s3_ff.c:429)2022-11-17T14:59:38.000+00:00 2022-11-17T14:59:38 [FILE_FACTORY ]E: Failed to write entire file (second trial) [1000722] (at_universal_fs_object.c:631)2022-11-17T14:59:38.000+00:00 2022-11-17T14:59:38 [FILE_FACTORY ]E: Write entire file failed: source = '/rdsdbdata/data/tasks/KXHGVEZM7WRYI636BOJOQYTCOIGFGS6AOGE3MNY/cloud/1/LOAD00000001.gzip' target = 'dms-in2tox7r7fgrwfjrshecwnp6s4y2uvsre63afzy/redshift-FGKQA6KH2TIZ5J6G4T3NMVU7REZGRBNUYSGQI3Y/KXHGVEZM7WRYI636BOJOQYTCOIGFGS6AOGE3MNY/1/LOAD00000001.csv' open type = 3 [1000731] (at_universal_fs_object.c:316FollowComment"

Answer: "0Hi,Have you checked that the used IAM Role has the S3 Get Object permission?The last section found in this link could be helpful https://aws.amazon.com/premiumsupport/knowledge-center/dms-task-successful-no-data-s3/Thanks,CommentShareZiadanswered 7 months ago"


Question: AWS PrivateLink use case validation

"Hello,I would like to ask about my use case if it needs a private link https://docs.aws.amazon.com/vpc/latest/privatelink/concepts.html, or not.We have a VPC with 2 public subnets, 2 private subnets, one of the EC2 is in private subnet and needs to connect to ALB (Application Loud Balancer) with Internal Scheme and also exist on same private subnet on same VPCConsidering AWS Well-Architect, do we need to add any private link (endpoint) from this EC2 to the related ALB ? Is this a valid scenario?Many thanks.FollowComment"

Answer: "1Accepted AnswerWith both the instance and ALB in the same subnet and VPC privatelink is not required. Within the same subnet you can use the security group of the ALB to control who is allowed to connect (ie use the security group of the EC2 instance within the ALB inbound rule)PrivateLink can be used to expose your NLB (not ALB) to other AWS accounts or unlinked VPCsCommentShareEXPERTPeter_Ganswered 10 months agoEXPERTBrettski-AWSreviewed 10 months ago0I don't understand your question but a privatelink is to access to a service outside your VPC...If the target ALB is in the same VPC, you "just" need good configurations on NACL, Security Groups and routes.CommentSharebacattaanswered 10 months ago"


Question: Cannot load anything other than root request -- ECS + ECR + ALB 503s

"Greetings,I have a simple static site behind NGINX in a container. I have pushed the image to ECR, and can pull and run the container without any issues.However, when I put an Application Load Balancer in front of the ECS Service with Fargate, I am unable to load any assets. For that matter, anything beyond the root does not load: i.e. subdomain.example.com loads, and subdomain.example.com/* throws a 503.As mentioned, I can run the container locally without issue by pulling the :latest tag, so I am confident it's not an issue with the container, and the issue lies somewhere in the configuration of the ALB and ECS Service.Any guidance would be greatly appreciated.FollowComment"

Answer: "0This turned out to be a misconfigured Listener that was pointed at an incorrect target group. I don't see any way to delete my question, so I figured I'd share the "solution."CommentShareJakeAHarrisanswered 3 years ago"