Question: Kinesis Analytics for SQL Application Issue

"Hello,I am having trouble to properly handle query with tumbling window. My application sends 15 sensor data messages per second to Kinesis Data Stream, which is used as an input stream for Kinesis Analytics application. I am trying to run an aggregation query using a GROUP BY clause to process rows in a tumbling window by 60 second interval. The output stream then sends data to a lambda function. I expect that the messages should arrive at lambda every 60 seconds but instead, they arrive much faster, almost every second, and the aggregations don't work as expected. Here is the CloudFormation template that I am using:ApplicationCode:CREATE OR REPLACE STREAM "SENSORCALC_STREAM" ("name" VARCHAR(16),"facilityId" INTEGER,"processId" BIGINT,"sensorId" INTEGER NOT NULL,"min_value" REAL,"max_value" REAL,"stddev_value" REAL);    CREATE OR REPLACE PUMP "SENSORCALC_STREAM_PUMP" AS     INSERT INTO "SENSORCALC_STREAM"     SELECT STREAM              "name",         "facilityId",                     "processId",                     "sensorId",                MIN("sensorData") AS "min_value",        MAX("sensorData") AS "max_value",        STDDEV_SAMP("sensorData") AS "stddev_value"            FROM "SOURCE_SQL_STREAM_001"    GROUP BY "facilityId","processId", "sensorId", "name",         STEP("SOURCE_SQL_STREAM_001".ROWTIME BY INTERVAL '60' SECOND);KinesisAnalyticsSensorApplicationOutput:Type: "AWS::KinesisAnalytics::ApplicationOutput"DependsOn: KinesisAnalyticsSensorApplicationProperties:ApplicationName: !Ref KinesisAnalyticsSensorApplicationOutput:Name: "SENSORCALC_STREAM"LambdaOutput:ResourceARN: !GetAtt SensorStatsFunction.ArnRoleARN: !GetAtt KinesisAnalyticsSensorRole.ArnDestinationSchema:RecordFormatType: "JSON"I would really appreciate your help in pointing what I am missing.Thank you,SergeFollowComment"


Question: data brew capacity

How many records data brew can process for cleaning and data transformation?FollowComment

Answer: 0How can i clean Thai text in databrew to form word cloud?CommentSharerePost-User-1026054answered a year ago


Question: kinesis video fragment browser

"Does anyone know of a UI to be able to browse through the fragments that are available in a kinesis video stream ?I've seen the API for API_reader_ListFragments.html, just wondering if there already is something like this out there that i can use.FollowComment"

Answer: 0Accepted Answersomething that does something like that... not exactly but worth a look https://github.com/aws-samples/amazon-kinesis-video-streams-media-viewerCommentShareclogwoganswered a year ago


Question: When will r6 instances be available in eu-central-1a?

"Hey,when will r6 instances be available in eu-central-1a , so we can actually use R6 instances in services that need all three zones - e.g. EKS, RDS etc.greetings !FollowComment"


Question: IP of my instance not responding

"Hello, why is the public IP address 34.211.96.185 of my instance i-016b97b86e42f00e2 not responding? As well as the address https://ec2-34-211-96-185.us-west-2.compute.amazonaws.com/I tried restarting it, but nothing changed.FollowComment"

Answer: "0You might want to look at:The security group: are the inbound ports open?The Network ACLs, do they accept traffic both inbound and outbound?Is an Internet Gateway attached to the VPC your EC2 instance is running in?CommentShareFrederiqueanswered 4 months ago0I have been having the same problem. I already tried redeploying with several instances since yesterday and the public IP is not valid. I already checked my settings and they are all correct according to what the troubleshooting pages show. It is getting pretty frustrating. Does anyone have a solution? Is AWS having reported problems with instances at the moment?CommentShareeslf_panswered 4 months ago"


Question: Neptune  Notebook visualization: expand node (vertex) ?

"I'm evaluating Neptune after working with Neo4j.AWS Notebooks are analagous to Neo4j Browser.The Graph tab shows a visualization of results.Nodes can be rearranged, colored etc.In Neo4j browser, a node can be selected and all of it neigbors can be unhidden and added to the display. Nodes can be selectively hidden again.Does the Notebook's Graph tab provide for this node neighborhood expansion? If so I have not found it.Thanks.FollowComment"

Answer: "0Hello and thanks for the question. At present the graph-notebook project does not support a "click to explore" graph navigation model. However, this is a capability we are interested in adding and feedback like yours helps us to prioritize new feature work. There are several open source and commercial alternatives that can be used to work with Neptune that do support a "click to explore" experience.CommentShareAWS-KRLanswered a year ago"


Question: What are the current SLA's offered by AWS for SAP instances?

What are the current SLA's offered by AWS for SAP instances?NOTE: This is a common question asked by AWS customers. The AWS team is posting it to provide an answer that can benefit everyone.FollowComment

Answer: "1Accepted AnswerEach service used in SAP deployment has its own SLA, please refer AWS Service Level Agreements (SLAs) for details.https://aws.amazon.com/legal/service-level-agreementsNOTE: This answer is provided by the SAP on AWS speciality team and is accurate at the time of publish. Please check comments as the answer may have changed/services evolved.CommentShareDeep_Kanswered a year ago"


Question: [HELP] Instance goes unresponsive after a few minutes...

"I am trying to RESTART using AWS after several years, I have created a new VPC and Instance per the instructions and I can start it, ping it and ssh into it (it's Red Hat Linux - Free tier eligible version).  The problem is that after about five minutes it just stops responding to network traffic,  the ping and ssl session die.  Oddly enough the instance still appears to be running and I can get a screen shot showing the console looking okay.I read somewhere that this is a known issue with "reactivated or old AWS accounts" and can only be resolved with help from Amazon.  Could someone please take a look??Instance:i-0b0c25b5126971d83 (RHL 05072019 1)Network Interface:eni-00ef3fe3eb30ef3b4Public IP:  3.210.51.27StatusRunning but unresponsiveThanks, DanFollowComment"

Answer: 0This was resolved - it was an issue with a restriction of the account placed by AWSCommentSharedchallisanswered 4 years ago


Question: WooCommerce on Lightsale server

"Hi,we have a Lightsail instance with WordPress and WooCommerce (about 4000 products and 350 views/day). We would need to increase the sustainable zone because at the moment the site is slow or not accessible. How can we solve? With an upgrade of Lightsail plan (3,50$ plan actually) or with a migration on EC2 instance? Thanks for answers!Edited by: IlReDellaFesta on Feb 16, 2021 7:23 AMFollowComment"

Answer: "0The $3.50 plan is extraordinarily underpowered for, IMO, just about any application.I would suggest you snapshot your instance and create a new one with more cpu & memory from the snapshot.You might also want to consider creating an 8gb disk to use as a swap partition.  I did that and it helped my instances considerably.  I blogged about it here: https://www.geekyramblings.net/2018/02/01/amazon-lightsail/#512mb-isnt-a-lot-of-memorydavidCommentShareDavid Ganswered 2 years ago0Thanks a lot for the answer David! I upgrade my lightsail plan (from 3,50 to 5 USD) and it seem works fine, for now. I'll try with the partition swap too.CommentShareIlReDellaFestaanswered 2 years ago"


Question: EventBridge infinite loop calling API Gateway

"I have an EventBridge rule with cron to run once day. When I enable the rule, it keeps firing the event and executing the API Gateway. I tried to disabled the rule but checking the logs for API Gateway, I keep seeing executions.Is this expected behavior for a rule?FollowComment"

Answer: "0It might be helpful to show the cron expression that is being used.Is the API Gateway invocation succeeding? If it fails you might find that EventBridge will retry the operation.CommentShareEXPERTBrettski-AWSanswered 4 months ago0This is the cron expression: cron(10 * * * ? *).The API Gateway was failing and it was retrying but I also saw the same behavior on success from API Gateway.CommentSharerePost-User-7304484answered 4 months ago0That expression means that your API is being triggered at 10 minutes past the hour, every hour of the day.Looking at the documentation you probably want something like `(0 10 * * ? *) which runs at 10AM UTC every day.CommentShareEXPERTBrettski-AWSanswered 4 months ago"


Question: deploy spring boot application on ECS fargate using buildPacks

"Hi team,I have a spring boot application I want to deploy it to my fargate cluster using codePipeline,in my build buildspec.yml is it best practice to use tye Cloud Native Build packs to generate the image or to use dockerfile with docker commands :docker build -t, docker tag ...if it's best practice to use build packsI found 2 ways to do it :1 - buildpack-builder build my-app -p my-app/ --builder cloudfoundry/cnb:bionic2 - by installing the build packs in the installation phase : echo "$(curl -Ls https://github.com/buildpacks/pack/releases/download/v0.13.0/pack-v0.13.0-linux.tgz | gunzip | tar -xf - -C /usr/local/bin)"not sure what is the difference between them and which one is best to use.is there a complete example of a buildspec.yml using buildPacksThank you for your help,Regards,FollowComment"

Answer: "0Hello there,Thank you for contacting AWS support!I understand that you want an example of a CodeBuild buildspec.yml file which uses buildPacks as you want to know the best practise for the same.I would like to inform you that latest iteration of buildpacks can be thought of as decoupling the build mechanism of these platforms so that it can be consumed by the broader container community. As this project has matured, it has been adopted by a number of vendors and tools.Additionally, I would like to mention that writing custom buildspec falls out of our support scope, however, I would be providing you a sample Buidlspec.yml file for your use-case:version: 0.2env:  variables:    builder: "paketobuildpacks/builder:base"    pack_version: "0.18.1"    application_name: "default_application"  exported-variables:  # Exported the image tag to be used later in the CodePipeline  - IMAGE_TAGphases:  install:    commands:    # Download the pack linux binary    - wget -q https://github.com/buildpacks/pack/releases/download/v$pack_version/pack-v$pack_version-linux.tgz -O - | tar -xz    - chmod +x ./pack  pre_build:    commands:    # Log in to ECR    - ECR_DOMAIN="$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"    - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $ECR_DOMAIN    # Set up some derived values for subsequent phases    - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)    - ECR_REPOSITORY="$ECR_DOMAIN/$application_name"    - IMAGE_TAG="$ECR_REPOSITORY:$COMMIT_HASH"  build:    commands:    - |      ./pack build --no-color --builder $builder \      --tag $IMAGE_TAG $ECR_REPOSITORY:latest \      --cache-image $ECR_REPOSITORY:cache \      --publishPlease use the above .yml file for example and configure your buildspec files as per your use-case.I hope the information provided above is useful.Thanks!CommentShareSUPPORT ENGINEERrePost-User-0470248answered 4 months agoJess 4 months agothank you for your answer,I understand from your answer this is not the recommended way to do it from AWS.So what would be the AWS recommended way to build a spring boot application?Thank youShare"


Question: EFS pricing for tiny files

A customer wants to store millions of small files (less than 1KB each) on EFS.They want to know if we round up the size of small files to a minimum file size when calculating the storage costs. The pricing example on the service page seems to indicate we do not round up small files to a minimum size for EFS.Can someone please confirm this?FollowComment

Answer: "0Accepted AnswerYes, the minimum metering increment for EFS is 4KiB, and inodes are metered at 2 KiB.  So a 1KiB file is metered at 6KiB.  More info here: https://docs.aws.amazon.com/efs/latest/ug/metered-sizes.htmlCommentShareAWS-User-2481325answered 4 years ago"


Question: GraphQL DynamoDb filter List field

"I'm trying to use a graphQL API using AWS Amplify to list books that i've stored in DynamoDB filtered on a field that is a List type.Schemaid: ID!title: StringauthorId: Stringgenre: [String]...}I'm doing this:API.graphql(graphqlOperation(listBooks, {filter: {genre: {contains: category,}},}))But the only books being returned are the books with a single element and that element matches the category, for example when the category is "History" this will matcha list of {S: "History"}but this will not matcha list of  { "S" : "History" }, { "S" : "Politics and Social Sciences" }My ModelBookFilterInput is this:input ModelBookFilterInput {id: ModelIDInputgenre: ModelStringInput...}And the ModelStringInput has the contains: String attribute on it so it looks like it supports this.I'm not sure what i'm doing wrong, any input appreciated.Edited by: JesusGarcia94 on Aug 18, 2020 8:44 PMEdited by: JesusGarcia94 on Aug 18, 2020 8:45 PMEdited by: JesusGarcia94 on Aug 18, 2020 8:45 PMEdited by: JesusGarcia94 on Aug 18, 2020 8:46 PMFollowComment"

Answer: "0It turns out I was neglecting to use the nextToken in the payload. the query will only look at 10 elements at a time by default so it's the dev's responsiblity to continue querying until they have enough matching resultsExample of method to use nextToken:/**@desc Recursively fetch all items in a list query using nextToken@param {Object} query The query object from cda-graphql in use.@param {Object} variables The variables to pass to query.@returns {Array} Array of all items received from queries.*/import { API, graphqlOperation } from 'aws-amplify';async function fetchItemsNextToken({ query, variables, limit}) {const results = [];while (results.length < limit) {const { data } = await API.graphql(graphqlOperation(query, variables));const key = Object.keys(data).find(k => k.includes('list'));const res = data[key]; // res = { items: [], nextToken: '' }    results.push(...res.items);      if (!res.nextToken) break;      // eslint-disable-next-line no-param-reassign      variables.nextToken = res.nextToken;  }  return new Promise((resolve, reject) => {      resolve(results);  });  }export default fetchItemsNextToken;CommentShareJesusGarcia94answered 3 years ago"


Question: Workdocs user invite getting expired within 2 days

The workdocs invite sent to external users are getting expired within 2 days. Can you please help in this?FollowComment

Answer: "0Hello,I understand you have sent the work docs invite to external users and the invites get expired in two days.Below I would like to address your direct question--Since it's optional to set an expiry date on the invitation link as shown here [1], the administrator is the one who can control if the link can expire or not ,so in this case , you need to send a new link with no expiration date.i hope i was able to help.Reference link[1] https://docs.aws.amazon.com/workdocs/latest/userguide/web_share_link.htmlCommentSharestevenanswered a year agorePost-User-4703398 a year agoThanks Steven! But I'm unable to find the option to change the link expiry date as detailed in the reference link. Can you help me to identify where exactly i could find the option? Is it somewhere within the admin control Panel?Share"


Question: Is it possible to use IAM and Secure Token Service as the authentication solution for an application?

"The reasons for considering this are:The number of authenticated users is low, and leveraging the roles and permissions model in AWS would be useful.Unifies the audit processesAllows users of our application to also access AWS features like dashboards, logging etc.I think it may be possible with API Gateway, but I couldn't see something in the docs that would pass a username or a verification result through the HTTP headers.FollowComment"

Answer: "1It looks like Cognito Identity Pools is what you are looking for.  This lets you map IAM roles to identities.  For defining the actual user, password, etc to define the identity you will use with Identity Pools, you can use  Amazon Cognito user pools, but you do not have to.   Cognito Identity Pools will also work with user IDs defined on Facebook, Google, etc.CommentSharesetheliotanswered a year agoFT a year agoThanks setheliot, do you know if it is possible to convert a Cognito session's IAM role into a web session for a user to go to the AWS web console?Share0I think what you are looking for is AWS Single Sign-On.Another option would be Amazon Cognito.CommentShareEXPERTkentradanswered a year agoFT a year agoThanks kentrad, Do you know if you can use an Amazon Cognito endpoint as an AWS Single Sign-On provider? I suspect that may be the way to give a person access to the web AWS console, but it wasn't clear how to configure it that way.Share"


Question: RE: Amazon Connect - Dequeue Callers or Remove Caller from Queue

"Hi,When callers are in the queue, call flow comes out of hold music prompt and give press 1 option to caller to leave voice mail in same call flow (Default Customer Queue - Call flow).   Sometime, When caller start to leave voice message, the caller get connected with agent. Agent can hear caller messages. But Caller doesn't hear the agent talk. How can we dequeue / remove caller from queue before sending to Voice mail flow?  I will appreciate for your feedback.Thanks,SelvaFollowComment"

Answer: "0One suggestion will be to create a non-agent queue (such as NoAgentQueue) and use the Set Working Queue to this queue before invoking the voicemail flowCommentShareRickyanswered 7 months agoEXPERTAWS-User-Nitinreviewed 7 months agoSelva Maha 7 months agoThanks Ricky. I will appreciate your feedback.Share0Hi Ricky,       The Caller is in "Customer queue" flow type. In this flow type, it is not allowing to invoke "Set Working Queue" or "Transfer to flow".        We can invoke "Set Working Queue" or "Transfer to flow" when it is Contact flow type.Thanks,SelvaCommentShareSelva Mahaanswered 7 months agoRicky 6 months agoIn the customer Queue flow, you can use Transfer to Queue Block.Share"


Question: DyanamoDB connection issue - aws-sdk for NodeJs

"While connecting to DynamoDB via aws-sdk for NodeJs, i'm seeing below issue:InvalidSignatureException: Credential should be scoped to a valid region, not 'eu-east-2'.at Request.extractError (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/protocol/json.js:52:27)at Request.callListeners (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/sequential_executor.js:106:20)at Request.emit (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/sequential_executor.js:78:10)at Request.emit (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:686:14)at Request.transition (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:22:10)at AcceptorStateMachine.runTo (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/state_machine.js:14:12)at /Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/state_machine.js:26:10at Request.<anonymous> (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:38:9)at Request.<anonymous> (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:688:12)at Request.callListeners (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {code: 'InvalidSignatureException',time: 2021-12-30T17:22:35.364Z,requestId: '3CT4LSCT0395369U3SVVNMV4B3VV4KQNSO5AEMVJF66Q9ASUAAJG',statusCode: 400,retryable: false,retryDelay: 24.638717702395528I cannot believe that I'm not able to resolve this issue still after spending several hours on it and its weird that nobody else has faced or logged it. Or maybe I'm doing something wrong.Here's the config code:AWS.config.update({region: DynamoDBConfig.region,endpoint: DynamoDBConfig.endpoint,accessKeyId: DynamoDBConfig.accessKeyId,//process.env.AWS_ACCESS_KEY_ID,secretAccessKey: DynamoDBConfig.secretAccessKey//process.env.AWS_SECRET_ACCESS_KEY});const dynamodb = new AWS.DynamoDB({region: DynamoDBConfig.region});Note: The same code is working fine on the localhost DynamoDB.FollowComment"

Answer: "3Accepted AnswerHi, it looks like the issue is with the region code you're using. eu-east-2 isn't listed on our region list:https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.htmlCommentShareTed_Zanswered a year agoSaru a year agoMY GOD! That was a typo, aaahggg!Thank you so much for the input, you've saved me several days!Share2It looks like you are using a region which does not exist. Check the value of your DynamoDBConfig.region and ensure it points to a region listed here:CodeRegionus-east-2US East (Ohio)us-east-1US East (N. Virginia)us-west-1US West (N. California)us-west-2US West (Oregon)af-south-1Africa (Cape Town)ap-east-1Asia Pacific (Hong Kong)ap-southeast-3Asia Pacific (Jakarta)ap-south-1Asia Pacific (Mumbai)ap-northeast-3Asia Pacific (Osaka)ap-northeast-2Asia Pacific (Seoul)ap-southeast-1Asia Pacific (Singapore)ap-southeast-2Asia Pacific (Sydney)ap-northeast-1Asia Pacific (Tokyo)ca-central-1Canada (Central)eu-central-1Europe (Frankfurt)eu-west-1Europe (Ireland)eu-west-2Europe (London)eu-south-1Europe (Milan)eu-west-3Europe (Paris)eu-north-1Europe (Stockholm)me-south-1Middle East (Bahrain)sa-east-1South America (São Paulo)CommentShareEXPERTLeeroy Hannigananswered a year ago0Hi,InvalidSignatureException error is when your credentials for the IAM user may not have the right DynamoDB actions for that DynamoDB table or IAM user or for that specific region it is not authorised. As you have mentioned, on localhost it is working recheck the DynamoDB table or region it is using.CommentSharezachjonesnoelanswered a year ago0the issue was occurring because of the typo in the region where I had this: 'eu-east-2' instead of this: ''us-east-2'CommentShareSaruanswered a year ago"


Question: "Run stuck on a device forever, can't start new run, please help"

"We cannot start any new Android run since yesterday. It looks like one of earlier "stopped" runs is stuck and LG G7 ThinQ is stuck forever in "Stopping..." state and looks like the run cannot fully stop so it blocks all the following runs.Problematic LG G7 ThinQ run https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/a5cd451c-6593-4e2d-988d-3cfe012bcdd7/runs/5b059cff-b9f0-4e46-8c2e-d8a83abfeb80Following run that's stuck in "Your app is currently being tested. Results will appear here as tests complete." and stays there forever without ever showing any tests or results https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/a5cd451c-6593-4e2d-988d-3cfe012bcdd7/runs/5b059cff-b9f0-4e46-8c2e-d8a83abfeb80Please unblock us since we can't do any Android runs anymore as we have 1 Android slot only.Thank you.Edited by: marjanovic on May 21, 2019 5:02 AMFollowComment"

Answer: 0It works again. Thank you for solving this fast.CommentSharemarjanovicanswered 4 years ago


Question: EB instance Degraded after Platform update

"EB instance Degraded after Platform updateI got a notification from AWS that my platform needed an update, my lead Dev has done the update on our Staging server, but the instance is now Degraded with the error Following services are not running: proxyReturning to the older version, restarting app server and rebuilding the environment didn't help.What steps to rectify this? Is it an application error or a server error, we're struggling to find the causeThanks in advanceFollowComment"

Answer: "0Turns out the answer was to recreate the new environment a second time, after the update, creating the new environment using the existing code. It then automatically set the latest platform versionCommentShareChronic404answered 3 years ago"


Question: Glue Job to output in PostgreSQL throws "Something unusual has occurred to cause the driver to fail. Please report this exception."

"Hi al,Did any of you encountered this? Do you know where should I report it?The full error message looks loke this: "An error occurred while calling o111.pyWriteDynamicFrame. Glue ETL Marketplace: failed to retrieve table schema from source.driverClassName: org.postgresql.Driver, error message: Something unusual has occurred to cause the driver to fail. Please report this exception."FollowCommentGonzalo Herreros 2 months agoThat error comes from the driver, by report they mean to the Postgres community. More often than not it's a connection issue, there should be a reason in the error log. For instance: https://github.com/pgjdbc/pgjdbc/issues/217Share"


Question: Redshift recommended free space percentage

I have a customer who is performing a migration of their on-premises DW to Redshift and is concerned about the amount of free space that the Redshift cluster should have at any time in order to maintain a good performance and don't run into issues when executing heavy queries or commands.Do we have any guidance the maximum disk space percentage used within the cluster?FollowComment

Answer: "0Accepted AnswerBetween 70% to 80% should be the maximum storage space used. This is for regular queries and day to day operations.  You can configure a CloudWatch Alarm to monitor the disk usage of your cluster.If the customer is vacuuming a large table - they currently require approximately 3x the size of it in free space.As an example, a table that consumes 15% of cluster space, will require 45% free space in order to vacuum.CommentShareEXPERTTony-AWSanswered 5 years ago"


Question: How do I save data files to EFS using a Node.js application?

"I am testing my Node.js application which I deployed using Elastic Beanstalk, and I've successfully mounted an EFS volume to my EC2 instance on the "/efs-mount-point" directory. The problem is I don't know how to get my application to save data to this directory or to my EFS volume.I am using the fs module to write short text files like this:fs.writeFile(path, textData, callback);If I set the path value to something like "./data/filename.txt", then the app works fine except that it does not save the data to EFS. It saves to what I assume is the EBS volume instead. I've tried setting the path to my EFS mount point directory "./efs-mount-point/filename.txt", but then nothing is saved at all.I was under the impression that EFS storage could be treated like local file system storage in our applications. Am I simply using the wrong path location, or are there additional AWS configurations required to make a Node app save files to the EFS volume?FollowComment"

Answer: "0Thanks for trying EFS!You're under the correct impression that reading and writing from EFS should work just like local file system operations, except that the files will be available in a shared file system.At first glance, I think your issue may be that you're calling your write API with a relative file path rather than absolute by starting with a ".".  When you save to "./efs-mount-point/filename.txt", the OS will resolve to "current-directory/efs-mount-point/filename.txt", where "current-directory" is likely the workspace of your Node app. Assuming you mounted EFS to "/efs-mount-point", try changing your API call to say "/efs-mount-point/filename.txt" and see if the files appear in EFS.Will @ AWSCommentShareAWS-User-4040733answered 4 years ago0Thank you for your response, but unfortunately, I am still not able to write or read my EFS volume using my Node.js application.You were right about the relative read path causing problems. When I tried to read the directory of my EFS mount point, I was previously getting a directory does not exist error. When I changed the readFile path name from "./efs-mount-point" to "/home/ec2-user/efs-mount-point", my Node app produced a "permission denied" error instead, which leads me to believe this is a step in the right direction. Unfortunately, I am unable to determine why permission is denied.I can manually write files to my "/home/ec2-user/efs-mount-point" directory in SSH console using the "touch testFile.txt" command, so I think I've given the root user sufficient permissions. I've even tried opening up my Security Group settings to allow all traffic on both inbound and outbound, but I still get the same permission denied error in my Node app.I would note that I am using Elastic Beanstalk to launch my Node application, and the EC2 instance and EFS volume are in different security groups (as instructed in the AWS docs). I am using NSF4 to mount my EFS volume automatically on reboot. It only successfully mounts on reboot when I specify the mount path as "/home/ec2-user/efs-mount-point". The path "/efs-mount-point" does not mount successfully. Am I mounting my EFS volume incorrectly, or do you think it's some security setting I've overlooked?CommentShareSev2answered 4 years ago0Great to hear we're getting closer.  For the next issue you're running into it sounds like the user the Node.js application is running as doesn't have permissions to the data in EFS.  By default EFS file systems are owned by root.  When you SSH in as ec2-user you're operating as root, so thats why you're able to read/write from the linux shell.  I'm not super familiar with Node, but i'd guess that when you launch a node application it executes as a non-root user (maybe called "node"), and since that user hasn't been granted access to any EFS directories or files you're getting permission denied.The easiest way to fix this is to issue a 'chmod 777 /myefsmountpoint'.  This gives all users read/write/execute permissions to the root directory, allowing them to create new files and read/write to them.  You may have to 'chmod 777 /myefsmountpoint/somefile' for any data that already exists that you want Node to have access to.  We have a doc that talks a little more about this -https://docs.aws.amazon.com/efs/latest/ug/accessing-fs-nfs-permissions.htmlTo the issue about mounting at boot, the only thing I can think of is that when you reboot your instance the directory you're trying to mount to isn't there.  This may happen if EBS is only backing your home directory and not root.  If this was the case, each time you rebooted your / directory would be rebuilt from the AMI.  If true, you could workaround it by either automating the creation of the directory, or mounting to a path backed by EBS (like home).CommentShareAWS-User-4040733answered 4 years ago0Thank you for your response. You were right that I had to run the chmod 777 command to enable access to my node application, but in order to get it to work, I had to run it on the home directory, ec2-user directory, and efs-mount-point directory. Without running the command on all 3 directories, it doesn't work.  Example:sudo chmod 777 homesudo chmod 777 ec2-usersudo chmod 777 efs-mount-pointYou were also right that if I try to save my Node documents to a subdirectory, like efs-mount-point/data, then I also have to run the chmod command a 4th time. Example:sudo chmod 777 dataThe good news is I am now able to read and write files to my efs-mount-point directory! Everything seems to be working as expected, so I will mark this question as answered. Thanks again.CommentShareSev2answered 4 years ago"


Question: How to know which s3 bucket invokes the lambda in a cdk app?

"Say I have a CDK app deploying the S3 -> EventBridege -> Lambda stack, and there are many s3 buckets that will invoke the same lambda.  The lambda handler runs a program differetly based on the source bucket.Normally, we can read the bucket name from the incoming event to know which bucket invokes the lambda, but since the bucket name might be different each time a new deployment is made, depending on the bucket name seems not reliable.So, is there a better way to know what the bucket is invoking the lambda in the lambda handler code?FollowComment"

Answer: "1Accepted AnswerIf I understand correctly, you deploy multiple stacks, each with a different bucket, but all with the same Lambda function. You want the bucket name to be determined by CFN, but you need to know in the function the origin bucket, or more specifically, what the bucket is used for, not its name.If this is the case and your function performs different actions based on the origin bucket, I would recommend creating different functions. This way each stack creates a different S3 bucket with a different function, that knows what to do.CommentShareEXPERTUrianswered 10 months ago0Hi, while creating an S3 bucket using CDK you can manually set the bucketName property: https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-s3.Bucket.htmlThis will ensure that the bucket has the same name every-time it is deployed, which should address your issue.Thanks,Marrick.CommentShareMarrickanswered 10 months agostrange-folk-df787 10 months agoHi, thanks for your response, but I prefer to have the bucketName assigned by the CloudFormation as the doc said: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.Bucket.html#bucketnameShare0Fair enough! In that case, your best bet is probably to using S3 to publish event messages to SNS which would in-turn trigger your Lambda function: https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-enable-disable-notification-intro.html. Specifically, configuring each bucket to send SNS events with a different SNS Topic would allow your Lambda function to determine where the event came from.CommentShareMarrickanswered 10 months ago"


Question: Is the AWS Pinpoint SDK intended to only be used by a mobile app (iOS/Android) or a web client?

"A project that I am working on has a feature to be implemented to send data to AWS Pinpoint from the API layer of our application.  The presumption is that there are multiple clients (websites/mobile apps) that interface with this API layer and that it would make sense to send that data to Pinpoint from the API layer that is central to all of the interactions across the clients.  Almost, all of the documentation that I see regarding the AWS Pinpoint SDK appear to refer to iOS/Android or web/React Native implementations.  Are there people that interact with Pinpoint from a backend/microservice application?  Is this not a desirable implementation?  Would there be any problems in that type of implementation?FollowComment"


Question: Multiple instances of macOS Workspaces client

"Hello,Is it possible to have two simultaneous instances of Amazon Workspaces clients running on macOS Catalina?I have workspaces from two different organizations/directories that I need to connect to at the same time.Thanks,HariFollowComment"

Answer: "0You can create an applescript with the following code:do shell script "open -n /Applications/WorkSpaces.app"Save it as an application (make sure the name and path of the Workspaces app is correct). If you run it, it will launch a new copy of the Workspace app. This technique works with any Mac OS appCommentShareagardezianswered 3 years ago0Thank you, @agardezi - that is really helpfulCommentShareHariMenonanswered 3 years ago"


Question: KMS key High Usage

"Hi,In AWS account as i can see there is huge cost of Customer master key. How do I find which CMK key is getting high cost  as i can see all CMK keys  are enable and how can we reduce the costRegardsHarryFollowComment"

Answer: "1Accepted AnswerDoes your account make heavy usage of KMS encryption for S3 buckets?  If so, you may want to explore the use of bucket keys to reduce cost.In terms of monitoring your KMS keys you can monitor your keys with both CloudWatch and CloudTrail.  In order to get per-key usage stats, you would most likely need to parse the CloudTrail log data further.CommentSharetparrottanswered a year ago"


Question: AWS Cloudformation conditions

"I would like to have an IF-ELSE statement to assign subnets.If 1 AZ is being used, assign privatesubnet1 to subnetsIf 2 AZ is being used, assign privatesubnet1 and privatesubnet2 to subnetsIf 3 AZ is being used, assign privatesubnet1, privatesubnet2 and privatesubnet3 to subnetsThe IF function handles 1 and 2 AZ well, but I don’t know how to handle 3 AZSuggestions are welcome.Conditions:  SingleZone:    !Equals [!Ref NumberOfZones, 1]  TwoZones:    !Equals [!Ref NumberOfZones, 2]  ThreeZones:    !Equals [!Ref NumberOfZones, 3]Resources:  Properties:    Subnets:      - !If [TwoZones, !Join [',', [!Ref PrivateSubnet1, !Ref PrivateSubnet2]], !Ref PrivateSubnet1]      #How to handle ThreeZonesFollowComment"

Answer: "0Hi,Look at https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-conditions.html#intrinsic-function-reference-conditions-if"Fn::If": [condition_name, value_if_true, value_if_false]You have to create second If in the value_if_false of your current if.CommentShareDidier_AWSanswered 25 days ago"


Question: Is log cleaner logging anything on AWS MSK?

"I use AWS MSK cluster with brokers logging turned on to CloudWatch. I can see brokers logs. We have some topics with cleanup.policy=compact and some with cleanup.policy=delete. The system is running on the new cluster for about 2 weeks now.From my research (e.g. https://zendesk.engineering/an-investigation-into-kafka-log-compaction-5e520f4291f0) I see that kafka should run log cleaner (obviously) and there should be some traces in logs of this activity. However in my CloudWatch log group I cannot find a word "cleaner" or "cleaned" and I cannot find any trace of log cleaner running.Is it something wrong with my logging settings? Or is log cleaner not running at all (this is suspicious, because we have a lot of messages eligible for cleanup but still not cleaned)?FollowComment"

Answer: "0Hello there, in order to verify from the logs whether files related to partitions are getting deleted or not you can look for the something similar as belowDeleted log LOG_DIR/<topic>/00000000022134078627.log.deletedDeleted time index LOG_DIR/<topic>/00000000022134078627.timeindex.deletedWhen retention kicks in you will be noticing deletion of log and time index files in the broker logs. That indicates retention is successfully triggered and deleted old log segments.CommentShareSUPPORT ENGINEERHarshith_Manswered a year agoamorfis a year agoThank you! Yes I have noticed such logs, but still, the disk usage is constantly growing, while we have about 99% of records eligible for deletion. This records are in __consumer_offsets topic where we commit last message every 100 millis even if there are no new messages for hours.Share"


Question: Deployment is getting failed

"We are trying to build and deploy dotnet application on windows platform in elastic beanstalk environment but deployment is getting failed with an error "Deployment completed, but with errors: During an aborted deployment, some instances may have deployed the new application version. To ensure all instances are running the same version, re-deploy the appropriate application version. Failed to deploy application. Unsuccessful command execution on instance id(s) 'i-0031f3decb3972a8f'. Aborting the operation. [Instance: i-0031f3decb3972a8f ConfigSet: Infra-WriteRuntimeConfig, Infra-EmbeddedPreBuild, Hook-PreAppDeploy, Infra-EmbeddedPostBuild, Hook-EnactAppDeploy, Hook-PostAppDeploy] Command failed on instance. Return code: 1 Output: null. Deployment Failed: Unexpected Exception Error occurred during build: Command hooks failed"FollowComment"

Answer: "0Hello there,Looking into the error message, it seems like the command execution on the instances have failed. This can happen due to multiple reasons. At the end, it is mentioned that command hooks failed. If you are using hooks with the Elastic Beanstalk environment they might be failing too.The best way to identify the reason behind the error is to fetch the logs of the environment or RDP into the instance. You need to look at the log locations to find the reason behind it. You can use this document to fetch the logs - https://repost.aws/knowledge-center/elastic-beanstalk-windows-instance-logsIf you are still not able to find the cause behind the issue, I request you to reach out to the Technical Support team with the environment ID and region for further investigation of the issue.CommentShareSUPPORT ENGINEERGurpreet Singhanswered 2 months agorePost-User-1875604 2 months agoThanks for your support.Share"


Question: HTTPS on Elastic Beanstalk without load balancer

"I have an Elastic Beanstalk environment for test only, so I don't need an ELB on it. Is it still possible to use https?FollowComment"

Answer: "0hi,Yes. You can. But using HTTPS with ALB provides an additional security layer and would be best to use it. But you can bypass ALB through the steps below,Get a SSL/TLS cert via ACMConfigure the Elastic Beanstalk environment: Update the environment to use HTTPS by specifying the SSL certificate and port in the environment's configuration file. You will also need to specify the SSL port that your application listens to in your application code.Configure the Security Group: Open the SSL port you specified in step 2 in the environment's security group.Modify DNS settings: If you are using a custom domain, update the DNS settings to point to the HTTPS endpoint of the Elastic Beanstalk environment.CommentShareArunanswered 3 months ago"


Question: Backing up the EC2

"We have an EC2 installed running Ubuntu 18-04 LTS - Bionic-Ubuntu for a Python application that is using PostgressSQL database.Due to the criticality of the application that we are running on this instance, we need to keep a near real time backup of the data, and I would to know your suggestions please.I have taken few images, but this process needs reboot of the instance which is not practical for us.FollowComment"

Answer: "0Hello,Using AMI does seems to be the best alternative offered by AWS for creating backups on an instance. You do have the option of selecting "No Reboot" from console or passing --no-reboot from CLI when creating the image:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-ebs.htmlhttps://docs.aws.amazon.com/cli/latest/reference/ec2/create-image.htmlRegards,shubh-AWSCommentShareSUPPORT ENGINEERshubh-AWSanswered 3 years ago"


Question: Unable to create parent nitro enclaves using aws cli

"Hi,I'm following the Hello enclave Tutorial from https://docs.aws.amazon.com/enclaves/latest/user/getting-started.html.When I try to run a instance with the following command:aws ec2 run-instances --image-id ami-070b208e993b59cea --count 1 --instance-type c6a.xlarge --enclave-options 'Enabled=true'I get the error message:usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]...Unknown options: --enclave-options, Enabled=trueAm I doing sth wrong or is the documentation obsolete?FollowComment"

Answer: "0Accepted AnswerI could figure out what was going wrong.I was using an outdated version of the awscli.aws-cli/1.18.69 Python/3.8.10 Linux/5.15.0-52-generic botocore/1.16.19With the latest version it works like a charm:aws-cli/2.8.12 Python/3.9.11 Linux/5.15.0-52-generic exe/x86_64.ubuntu.20 prompt/offHopefully, this helps anyone who runs into a similar issue.CommentSharerePost-User-8052000answered 7 months agoEXPERTTushar_Jreviewed 7 months ago0can you check the aws-cli version?  maybe you can try to upgrade your aws-cli version and try again.CommentShareSailong Nianswered 5 months ago"


Question: Inject authorizer context variable into a REST API Gateway proxy request

"Is it possible to inject a context variable into a REST API Gateway proxy request?Use case: I am storing an API key in a Secrets Manager.I want a custom authorizer to read the secret, and return it in the context.An API Gateway (REST flavour) proxies requests to the target API that needs to be authorized.And the goal is through the request mapping to inject the API key returned from the authorizer into the header of the proxied request.For example: x-api-key: $context.authorizer.apiKeyFollowComment"

Answer: 0A Lambda Authorizer (custom authorizer) can only returns an IAM policy as an output. Lambda Authorizer cannot be used for your scenario.https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.htmlCommentShareDebasis Mitra AWSanswered 5 months ago


Question: Upgrade RDS Postgres from 11.19 to 15.2 takes enormous amount of time

"I am trying to upgrade an RDS Postgres instance from 11.19 to 15.2.My db is almost empty, with allocated 100GB and used ~1GB.I would expect the upgrade to be almost seemless, taking at most 5 minutes, but after waiting for more than 1.5 hour, I deleted the instance.Is there something that I didn't notice that caused this much time of waiting for this upgrade?FollowComment"

Answer: "0Accepted AnswerI found the issue by issuing step by step upgrades for each major version.When upgrading from latest 11 to latest 12, an error was thrown (and upgrade aborted) about having tables using unsupported WITH OIDS conditions.After deleting those tables, the upgrade from 11 to 15 took less than 2 minutes (as expected).However, the issue was that direct upgrade from 11 to 15 did not throw those errors, instead it simply got stuck.CommentShareIoannis Christodoulouanswered 20 days ago"


Question: How to retrieve job run id from the script itself in python shell glue job

"I need to print my python shell glue job run id from the script itself into cloudwatch logs. But I didn’t find a way to find the current job run id, I have found a partial way using boto3 functions but its giving all glue job run ids and if we want to find out current job run id then its saying to filter with job run status, but this procedure works only there is one job run at a time, in my scenario I am expecting concurrent runs. Can you help me on how to find out the current job run id??FollowComment"

Answer: "0You can pass this in your AWS Glue Scripts. See if this helpsimport sys from awsglue.utils import getResolvedOptions args = getResolvedOptions(sys.argv, ['JOB_NAME']) job_run_id = args['JOB_RUN_ID']CommentShareAWS-User-8598002answered a year agoAWS-User-4333940 a year agoI have tried this, but the job is throwing an error -> KeyError: JOB_RUN_ID, and as per your message it seems like job is trying to retrieve a job parameter which is not even passed.Without passing job run id in the job parameters how can we retrieve it from script?Share0This function should help you get the job_run_id :Call it right in the beginning of your python jobdef get_running_job_id(job_name):    session = boto3.session.Session()    glue_client = session.client('glue')    try:        response = glue_client.get_job_runs(JobName=job_name)        for res in response['JobRuns']:            print("Job Run id is:"+res.get("Id"))            print("status is:"+res.get("JobRunState"))            if res.get("JobRunState") == "RUNNING":                return res.get("Id")        else:            return None    except  botocore.exceptions.ClientError as e:        raise Exception("boto3 client error in get_status_of_job_all_runs: " + e.__str__())    except Exception as e:        raise Exception("Unexpected error in get_status_of_job_all_runs: " + e.__str__())CommentShareKoustavCanswered 5 months ago0Apparently thegetResolvedOptions(sys.argv, ["JOB_NAME","JOB_RUN_ID"]) works only for Pyspark jobs. You can confirm this by firing a shell job and Pyspark job and doing a print(sys.argv) and having a look at the entire list of arguments returned.For Job Run ID :import boto3glue_client = boto3.client("glue")response = glue_client.get_job_runs(JobName = <your job name>)job_run_id = response["JobRuns"][0]["Id"]Use this code as early as possible within the Python shell job to get the job run id of the most recent execution.For Job Name :There is a programmatic way to derive job name which I have explained below.In a scenario where the job name and the python script name are same, we can read the the first element of sys.argv and then use below :job_name = sys.argv[0].split('/')[-1]returns scriptname.py. Strip the ".py" if you need only the name part.CommentShareawsUser-Akhilr97answered 6 months ago"


Question: How to restrict user to modify instance (ec2:ModifyInstanceAttribute)

"Hi,How can I restrict users to modify instances based on tags?"ec2:ModifyInstanceAttribute" doesn't support condition key "ec2:ResourceTag". It seems no solution to this question? To be honest, it totally makes NO sense to me that "ec2:ModifyInstanceAttribute" doesn't support "ec2:ResourceTag". Why?Now the users are free to modify any of the instances. What I want to do is allow users to only modify some of the instances, based on whatever attributes (like tag which doesn't work though).Do I miss anything? Any advice is appreciated.FollowComment"

Answer: "1Hello,At this time, there isn't a way to restrict "ModifyInstanceAttribute" to specific condition or resource. The action "ModifyInstanceAttribute" does not support any resource level permissions or any condition keys.https://docs.aws.amazon.com/IAM/latest/UserGuide/list_amazonec2.htmlI completely agree with you that this is a valid use case and these actions should support resource level permissions and conditions. This feature is requested by other customers as well and is a popular feature request. We are actively working on your feedback to address the issues listed in your post.You can keep an eye on our blog[1] and news websites[2] for updates.[1] http://aws.amazon.com/blogs/aws/[2] http://aws.amazon.com/new/Thanks for bringing this to our attention. Have a nice day :)CommentShareAWS-User-5718909answered 4 years ago0Hi,I have similar requirement. I am looking to restrict to be not able to change the Instance type to anything else that is not allowed in the IAM policy. Please let me know if there is any solution currently available to implement this?Would like to restrict the user to change the instance type based on the condition below. Right now the users will be able to change the instance type to anything once they are able to create the instance with the allowed types.{"Version": "2012-10-17","Statement": [{"Sid": "Stmt1624884618883","Action": ["ec2:RunInstances"],"Effect": "Deny","Resource": "arn:aws:ec2:::instance/*","Condition": {"StringNotEquals": {"ec2:InstanceType": ["t2.micro"," t3","micro"," t3.nano"]}}}]Edited by: SukumarEnuguri on Jun 28, 2021 5:46 AMCommentShareSukumarEnugurianswered 2 years ago"


Question: AWS Workspaces + Simple AD not protected by AWS DNS Firewall

"I'm working on preventing DNS exfiltration in an environment that makes use of AWS Workspaces + Simple AD. The WorkSpaces don't need to resolve anything via private hosted zones. They do connect to services hosted on an EKS cluster in the same VPC, however they have public domain names.The Simple AD documentation reads to me that Simple AD would use our VPC's configured DNS Resolver:Simple AD forwards DNS requests to the IP address of the Amazon-provided DNS servers for your VPC.I have AWS Network Firewall and AWS Domain Firewall configured - the EKS clusters DNS queries are correctly being filtered. However the AWS WorkSpaces can initiate a DNS request and the request bypasses the AWS DNS Firewall entirely (a DNS server outside the environment receives the request).Some assumptions of mine:AWS Workspaces instances use Simple AD for configuration, and by default they use the Active Directory Domain Controller as their default DNS server. The Active Directory Controller isn't a machine we directly control, it is part of the managed simple active directory service from AWS.What I think is happening is the Simple AD directory controller doesn't use our VPC configured resolver - and is recursively resolving the DNS query from the WorkSpace member instance. I'm assuming it isn't feasible to change the workspace instances to not use the domain controller's DNS server as they won't be able to join the domain? Is there any way of configuring the directory controller's behavior in this respect?I tried creating an outbound DNS Resolver and associating it with the VPC via a DHCP option set, but that didn't impose our restrictions on the DNS queries coming from the Workspace instance.The other thing I'm considering as a potential solution is upgrading from Simple AD to Microsoft AD. I understand this would provision two domain controllers within our VPC - which may be protected behind our network and dns firewall? I don't see why this would be different for Simple AD though? In any case, this AWS security blog certainly reads like MS AD can be make to work with the AWS DNS firewall - https://aws.amazon.com/blogs/security/protect-your-remote-workforce-by-using-a-managed-dns-firewall-and-network-firewall/Appreciate any pointers!FollowCommentJeffrey_D 5 months agoWhat do you mean by associating the DNS Resolver via DHCP option set? Also does the VPC that was used with the workspace have DNS Firewall configured?ShareBrian 5 months agoOne thing I tried was setting my VPC's DHCP option set. to try get the AD controllers to use the AmazonProvidedDNS via my VPC.Yes I have both the AWS Network Firewall and AWS DNS Firewall configured, both working well for other workloads in the VPC (primarily an EKS cluster).Share"

Answer: "0Microsoft Managed AD provides DNS directly from the managed Domain Controllers. Simple AD is not actually Windows AD, and must forward DNS queries. Regardless, Simple AD is not recommended for production use-cases of Amazon WorkSpaces. AWS Network Firewall is proven to work with Amazon WorkSpaces with Windows AD for filtering traffic.CommentShareEXPERTStevieStetsanswered 5 months agoBrian 5 months agoI've created a test directory using the managed Microsoft AD and started a Windows WorkSpaces instance to test this theory and I'm afraid it behaves the same. The test is to have an external DNS server watch for any requests e.g. https://messwithdns.net/I can confirm that DNS requests going through the VPC's DNS resolver (e.g 10.0.0.2) are correctly filtered by the DNS firewall, but DNS requests from the WorkSpace instance do make it to the external server - somehow circumventing the firewall.It seems to me like the AD controller's DNS resolver (using MS AD) might be forwarding DNS requests via its management interface eth0, rather than eth1 for DNS resolution?Share"


Question: AWS DX connections - BGP and Pricing

"q1.) Is BGP mandatory when setting up AWS DX connections? I had only a few routes and wanted to check if i can setup just Static routes and not use BGP.q2.) When we use Hosted connections setup using AWS DX partner, will the partner costs will be different from that of AWS DX costs or included in AWS DX costs?FollowComment"

Answer: "1See below statement from the documentation, BGP is mandatory for using DX:q1.) Is BGP mandatory when setting up AWS DX connections? I had only a few routes and wanted to check if i can setup just Static routes and not use BGP.•  Your device must support Border Gateway Protocol (BGP) and BGP MD5 authentication.Reference: https://docs.aws.amazon.com/directconnect/latest/UserGuide/Welcome.htmlq2.) When we use Hosted connections setup using AWS DX partner, will the partner costs will be different from that of AWS DX costs or included in AWS DX costs?Yes, that is correct. Direct Connect Delivery Partner's cost would be separateSee some of the pricing examples here: https://aws.amazon.com/directconnect/pricing/CommentShareEXPERTTushar_Janswered 24 days agoEXPERTRiku_Kobayashireviewed 24 days ago"


Question: Cloud Gaming - NVIDIA GRID vGPU Support

"A customer wants to use G4dn.xlarge machines to provide a cloud gaming service. They're testing with a small set of users (around 10-20) with the possibility - if working and cost-wise interesting - to add way more users to the equation.They are evaluating the G4dn.xlarge to be way more powerful than necessary for the kind of games they wish to provide in their portal. Therefore, they would like to use NVIDIA vGPU Architecture to share the GPU with 2 (or more) VMs on top of the same EC2 instance. Is this possible? Are there customers doing this today?My understanding from some of the researches I've done online is that it shouldn't be possible, however you guys might know some workaround or alternatives. AppStream / Workspaces is not currently on the table. Thanks!FollowComment"

Answer: "0Accepted AnswerIt is possible to launch multiple CUDA contexts on a single GPU, however I would not recommend it as performance with each context can be impacted. if g4dn.xlarge is actually too "powerful" then what is needed for the customer. They can look at the g4dn.metal. It does provide the flexibility of your customer to install their own hypervisor and NVIDIA Virtual GPU Manager and will allow them the flexibility to slice the 8 T4s as they wish with different profiles (minimum in 1GB increments).CommentShareEXPERTAmr_Ranswered 3 years ago"


Question: AWS STS GetFederationToken + resource based policy

"Hello,I am trying to understand the inner workings of AWS STS GetFederationToken in combination with resource based policy as described in https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_control-access_getfederationtoken.html. Does the IAM permission policy attached to the user which calls "GetFederationToken" must contain the same IAM actions as specified in the resource policy (the s3 actions in the example) or not? The documentation is not clear for me.Thank you very much for any help.FollowComment"

Answer: "0Accepted AnswerHello ,Thank you for posting your question on the AWS Repost, my name is Rochak and it will be a pleasure assisting you with this today.I understand you would like to know if the IAM permission policy attached to the user which calls "GetFederationToken" must contain the same IAM actions as specified in the resource policy. Please, let me know if my understanding is incorrect.Please note that yes, the IAM policy attached to the user making the GetFederationToken API call must contain the necessary IAM actions to perform the operation, but it does not necessarily need to contain the same IAM actions as specified in the resource policy.The GetFederationToken API call is used to obtain temporary security credentials for a federated user, which can be used to access AWS resources for a limited period. When making the GetFederationToken call, the user must have the sts:GetFederationToken permission included in their IAM policy.On the other hand, the resource-based policy is used to control access to specific resources (e.g., S3 buckets) like you mentioned  for the federated user.The IAM permissions policy and resource-based policy are separate entities, and they serve different purposes. While the IAM policy grants permission to make the GetFederationToken API call, the resource-based policy controls access to the specific resource(s) that the federated user is authorized to access.[1]So, the main point here is the IAM policy attached to the user making the GetFederationToken call must include the sts:GetFederationToken permission, but it does not need to contain the same IAM actions as specified in the resource policy. [2]I hope this helps. If you need further info, let me know in the comments; otherwise I'd appreciate if you mark my answer as "accepted".Kind regards,Rochak from AWSReferences:[1] Identity based policy and resource-based policyhttps://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_identity-vs-resource.html[2] Policy Typeshttps://aws.amazon.com/blogs/security/iam-policy-types-how-and-when-to-use-them/CommentShareAWS-Rochakanswered 3 months agoThomas Laue 3 months agoHello Rochak,Thank you very much for your reply. You understood my question correctly and you provided a clear answer to my question. It is now clear for me what has to be defined. Thanks for that. Only the documentation does not seem clear to me as the picture in the upper section proposes that the resulting permissions are an intersection of the users permission policy and the bucket policy - like it is the case when dealing with session policies.Also the section "The following resource-based policy is attached to the bucket. This bucket policy allows a federated user named Carol to access the bucket. When the example policy described earlier is attached to the token-app IAM user, the federated user named Carol has permission to perform the s3:GetObject, s3:PutObject, and s3:DeleteObject actions on the bucket named productionapp." lead me to the conclusion that it is an intersection.Best regards, ThomasShare"


Question: Unable to find database in RDS console

"I am a complete newbie in using AWS services. Recently (15-20 days ago) I started experimenting with using AWS to possibly transfer an application from another service to AWS. I started with a free account and following the tutorials I tried several times to create a Spring Boot application that needs to connect to a MySql database. Since the beginning of the month, the costs have started to increase enormously (for a test application).This is why I started deleting the resources I was using.But, I am not able to delete one particular MySql database. Neither database is visible in my RDS console, but I can access the said database using MySql Workbench. Even though I don't see a single instance in the RDS console, my costs keep increasing.Besides using the RDS console, I tried listing the active snapshots, instances and clusters using the aws cli:aws rds describe-db-snapshotsaws rds describe-db-clustersaws rds describe-db-instancesbut I still can't see which database is active for me, because I get empty strings as query results.{"DBInstances": []}{"DBSnapshots": []}{"DBClusters": []}What can I do next? How do I find and delete that database?  How can I find other databases that I may have inadvertently created?FollowComment"

Answer: "0Accepted AnswerHello,If you can still access your MySQL database but you can't see it in your console, it is possible that your database is deployed in another AWS region.You can see the AWS region in the database endpoint. For instance, if you have this endpoint: mydb.123456789012.us-east-1.rds.amazonaws.com, it means this database is deployed in us-east-1 region which is US East (N. Virginia) AWS region.Did you check the other AWS regions ?CommentShareMickael-AWSanswered 10 months ago0Thank you. I solved the problem by starting a connection with AWS from another browser, in which I saw invisible databases. After clearing the cache and cookies, all browsers worked correctly.CommentSharerePost-User-8343321answered 10 months ago"


Question: Not able to use IAM role assigned to EC2 instance at startup

"I am trying to create an autoscaling group with EC2 instances, I need to completed few tasks as an instance starts up and have created a shell + python3(boto3 script). I need to assign some elastic Ips etc. the instance has an IAM role assigned to it to complete the tasks. I assigned this role while creating the autoscaling group and launch configuration. I add the script as 'user-data'  to the instance startup.I tested the script independently and it works, but unfortunately it doesn't work when the instance is launched and the script is invoked. I get an error "unauthorized" when it tries to run it. Although I can see that the IAM role gets correctly assigned to the instance. I also tried by adding a wait time of about 30 seconds so the role may correctly get assigned to the instance, but that still didn't work.If I manually invoke the script after logging into the instance, it works. I have tried running the script as both only boto3 and as aws cli commands, I still get unauthorized for both.Other ways to do it would be that I invoke lambda functions as a target lifecycle hook during startup, but in that case I do not have a way to send the instance id to the function, which I needed for my script.Not sure what I am doing wrong here. Is there some other way I should invoke the script?FollowComment"

Answer: "0I was able to do this now, turned out the ec2 image I was using had an aws key configured for its root user, which was conflicting with the role assigned to the instance, after removing the configuration of the root user I was successfully able to run my scenario.CommentShareashish1457answered 3 years ago"


Question: OS life cycle of AWS

"Hello, I would like to ask the following 5 questions regarding the OS life cycle of AWS.Is there any criterion for launching a new OS?For example, a new OS has to be released 6 months after its release by the manufacturer.I'd like to know the possible impacts of ending the sales of OS.When the sale of the existing server ends (in other words, when a new server is created), are there any constraints on the functionality or usage of the existing server?When the technical support for the existing servers is no longer provided, are there any constraints on the functionality or usage of the existing server?When the support for server images is no longer provided, which of the following three cases is the right one?(1) Creating a new server with the server image no longer available.(2) It is possible to create a new server with the server image, but technical support is no longer provided.(3) It is possible through the console, CLI, and API.Does the SLA specify details about release, end-of-sale, and end-of-technical support of OS?FollowComment"

Answer: "1Here are the each question's answerThere is no specific criterion for launching a new OS on AWS. AWS evaluates various factors, including customer demand and technical feasibility, when deciding whether to launch a new OS.When the sale of an existing server is ended, there are no constraints on the functionality or usage of the existing server. However, the server may no longer receive security updates or bug fixes, which could affect its security and stability.When technical support for an existing server is no longer provided, customers can still use the server but may not receive assistance from AWS if they encounter technical issues.When support for a server image is no longer provided, it is still possible to create a new server with that image, but technical support is no longer provided.The AWS Service Level Agreement (SLA) does not specify details about the release, end-of-sale, and end-of-technical support of operating systems. However, customers can find information about the life cycle of specific operating systems in the AWS documentation.CommentShareMuhammad Imrananswered 4 months ago"


Question: How to do Custom Authentication in http header for MQTT-over-Websocket using C++ SDK?

"Hi,For "MQTT over WebSockets" found in https://docs.aws.amazon.com/iot/latest/developerguide/custom-auth.html, the credential can be passed  "Through request headers or query parameters in the HTTP UPGRADE request to establish the WebSockets connection."Does anyone know how to do this using the AWS IoT SDK C++?I understand that I can use Mqtt::OnWebSocketHandshakeIntercept() to embed authorisation into the HTTP header. However, it is unclear how it can be done since Http::HttpHeader basically only has name and value data members or is there another way to do it?Any assistance is greatly appreciated,AndiFollowComment"

Answer: "0Accepted AnswerHi,The AWS IoT Device SDK for C++ v2 supports the process for custom authorizers. You can modify the pubsub sample  to test. Essentially set websocket to true, and then change the protocol as referenced here. This should then pass the additional HTTP Headers to the Custom Authorizer. The additional authParam headers are processed here.I haven't tested with C++, so please do respond if you are successful or not.Edit: The above method would be for request headers used by POST or GET. Query headers could also be used by modifying the URI while still changing the protocol name.CommentShareGavin_Aanswered a year ago0Many thanks, Gavin! I have tried it and it is working well for me.CommentShareAWS-User-2130763answered a year ago"


Question: Aurora PostgreSQL aws_s3.query_export_to_s3 compression

Is there any way to compress s3 export data using aws_s3.query_export_to_s3 on Aurora PostgreSQL?Redshift has various compression options in its UNLOAD command but I cannot see anything similar for Aurora PostgreSQL.FollowComment

Answer: "2Thank you for reaching out. I understand you wanted to know if there is any options to compress data while exporting data to s3 using aws_s3.query_export_to_s3 on Aurora PostgreSQL.Unfortunately, compression is not available in this feature set at this time.However, I would suggest to keep an eye on our what's-new page, blogs for announcements about new features and improvements to RDS and other AWS services that might be of interest to you.https://aws.amazon.com/new/#database-serviceshttps://aws.amazon.com/blogs/database/As always, Happy Cloud Computing.CommentShareSUPPORT ENGINEERArnab_Sanswered a year agoshaohong 2 months agoThis would a really handy feature to export the data as gzipped file s3. Good for the environment!Share"


Question: Automatic report schedule from dynamodb table

"Hi AllA few datas are stored in dynamodb table and I would like to get an automated report in my email based on the filters (like a day/24hours interval). Now I am exporting the data manually on a daily basisCan someone guide me th proces? I think we can achieve it using a lambda function, streams and sns. Please advisce.RegardsAshFollowComment"

Answer: "3If you are looking to run a cron job to run a DynamoDB query and send out a report, you can look into using EventBridge. EventBridge allows you to create a cron rule where you can set the cron expression. So, the architecture looks like this: EventBridge rule -> Lambda (DDB query from AWS SDK and send a message to an SNS topic - where your email is subscribed to).CommentShareTaka_Manswered 9 months agoEXPERTBrettski-AWSreviewed 9 months ago"


Question: amazon server Down in trace route to our EC2 instance

below is a trace showing a request from our office to our EC2 instance.  it dies at 52.93.28.78.  our EC2 instance is at 54.86.96.175.aws support states it's a 'freeip.amazon.com' ip address and they cannot support it.  We believe it is down. How to report it? How to request service for a dead server?Any suggestions would be very much appreciated.Traceroute:Tracing route to 54.86.96.175over a maximum of 30 hops:1    <1 ms    <1 ms    <1 ms  192.168.1.12     2 ms     2 ms     2 ms  96-92-39-30-static.hfc.comcastbusiness.net [96.92.39.30]3    12 ms    13 ms    13 ms  96.120.40.334    14 ms    11 ms    14 ms  96.110.113.855    15 ms    18 ms    17 ms  be-79-ar02.pontiac.mi.michigan.comcast.net [69.139.255.77]6     *        *        *     Request timed out.7    19 ms    24 ms    24 ms  be-2212-pe12.350ecermak.il.ibone.comcast.net [96.110.33.214]8     *        *        *     Request timed out.9    21 ms    25 ms    20 ms  150.222.212.3110     *        *        *     Request timed out.11     *        *        *     Request timed out.12     *        *        *     Request timed out.13     *        *        *     Request timed out.14     *        *        *     Request timed out.15     *        *        *     Request timed out.16    43 ms    43 ms    42 ms  52.93.28.7817     *        *        *     Request timed out.18     *        *        *     Request timed out.19     *        *        *     Request timed out.20     *        *        *     Request timed out.21     *        *        *     Request timed out.22     *        *        *     Request timed out.23     *        *        *     Request timed out.24     *        *        *     Request timed out.25     *        *        *     Request timed out.26     *        *        *     Request timed out.27     *        *        *     Request timed out.28     *        *        *     Request timed out.29     *        *        *     Request timed out.30     *        *        *     Request timed out.Trace complete.FollowComment

Answer: "0It's not clear to me which server you think is "dead".I can ping your EC2 instance (54.86.96.175) so it's definitely up and running. That the various router hops along the way don't respond (or "participate" if you like) in the traceroute process is not surprising - it's actually very common.Traceroute works by sending packets with a lower TTL (Time-to-Live) then usual. The TTL is decremented by every router hop along the path; when it gets to zero what should happen is that the device that expired the packet (where TTL went to zero) emits a "Time Exceeded" ICMP packet and sends it back to the origin. Many devices don't do that (it's extra processing overhead). There could be other devices in the network that block those packets.In any case, it's quite normal to see traceroute not work quite properly yet have connectivity to the end host - which you do in this case (at least for ICMP - ping).CommentShareEXPERTBrettski-AWSanswered 6 months agowizcom 6 months agoin this case though. the server and all websites and services it is hosting cannot be reached nor ping from that location in fort gratiot only.the traceroute is successful from a sister office and so is visiting the websites and services the ec2 is hosting.we have an additional ip address on another subnet connected to that same ec2. trace route doesn't complete at hop 21. same 52.93.28.XXX subnet is last to touch it in the list.we have placed a static host entry at the locations router and it resolves fine -- as the ec2 instance is fine, the route is where we have issue. DNS resolves as it should. just one route from one of our office locations. all other locations, that we have tested work as expected.ShareBrettski-AWS EXPERT6 months agoIf putting a static entry in the hosts file fixes the problem then it's not routing - it's DNS.Share0after review of my notes, I am not certain the ping was successful from the office in question, but the resolve of host file works --leading me to believe that it was.  another of our agents will respond back with a ping for me. thank you for any help in the interim.CommentSharewizcomanswered 6 months ago"


Question: IoT Core is not an MQTT broker?

A customer is asking if IoT Core is not an MQTT broker in the sense that a device can not publish in a topic and that other devices receive it.The communication is designed to be Device to Cloud and Cloud to Device. Is that true?FollowComment

Answer: "1Accepted AnswerIt absolutely is an MQTT broker that exists in the cloud so a device can publish to a topic and another device will receive that message if it's subscribed to the same topic. Both devices are connected to the broker which is in the cloud or if they want to use Greengrass the broker can be local. If they are asking for Device->Device, that doesn't require a broker.CommentSharetypemismatchanswered 4 years ago"


Question: Elasticbeanstalk: Failed to deploy application. Verify the permissions on the environment's service role.

"I'm having trouble deploying to ElasticBeanstalk, it says:Deployment completed, but with errors: Failed to deploy application. Failed to check health. Verify the permissions on the environment's service role and try again later. Permissions changes take up to two hours to propagate.It's not at all clear what to do here.I can see that the service role is Service role: arn:aws:iam::*:role/aws-elasticbeanstalk-service-roleI can see that the service role exists and appears to be correct.It's not clear what the actual issue is or how to diagnose it further.FollowComment"

Answer: "0Hello,The Service role exists and may be configured, however you also need to verify if the right policies are attached.Link- https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.iam.managed-policies.html -- [1]CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 8 months agojpswade 8 months agoThanks, but it's not clear what policies should be attached. That link does not describe anything about health and what is required for that to work, please could you clarify what is needed?Share"


Question: cannot modify RDS on govCloud

"We have a few RDS database on gov Cloud that are running older version of mysql (5.6.51). I found out the "Modify" button is greyed out. That we would not be able to change anything for the instance, include endpoint name, memory etc. Could you take a look to see why is it restricted? Was it because the version is too old?FollowComment"

Answer: 0According to the documentation:You can't modify a stopped DB instance.CommentShareEXPERTkentradanswered 10 months ago


Question: Why can't i use A/AAA records to redirect a www domain to a non-www domain when the latter points to a cloudfront distribution?

"Hi,I want all requests coming from www.domain.com to be redirected to https://domain.com. When i use A/AAA records to redirect www urls to non-www ones i get the following error from cloudfront in the browser.403 ERRORThe request could not be satisfied.Bad request. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.Generated by cloudfront (CloudFront)Request ID: FuncJA-r5nbfvEUYV-_NdAINj80hfCo12dKVgm-K2-lCtLcjgj3O4w==I am now using an empty s3 bucket to redirect traffic coming from www.domain.com to https://domain.com. But i was wondering why this can't be done with simple A records in route53. The cloud distribution that is hosting the app only works with https but will redirect all incoming http traffic automatically to https. Does anyone know why Alias records don't work for redirecting traffic to a domain that points to a cloudfront distribution?FollowCommentJeremy 7 months agoYou must add all alternative domain names you intend to use for the Cloudfront distribution before adding the A/AAAA ALIAS in Route53.I use non-www hostnames for my Cloudfront distributions and then have the S3 bucket static website configured for the www hostname to redirect to the non-www hostname. You would configure there A/AAAA record for the S3 bucket to point to s3-website-us-east-1.amazonaws.com if the bucket is in us-east-1 region, you would have to check for other regions. then you'd create the ALIAS entry pointing to the Cloudfront distributionShare"

Answer: "0Accepted Answer(As per RFC 2606, I will use example.com instead of domain.com here.)You cannot use an ALIAS record as a redirect. That isn't how ALIAS records work. It's mentioned in the ALIAS record docs:When Route 53 receives a DNS query for an alias record, Route 53 responds with the applicable value for that resource:...Another Route 53 record in the same hosted zone – Route 53 responds as if the query is for the record that is referenced by the alias record.Route 53 it respond "as if the query is for the record that is referenced by the alias record."In this case, if you set up www.example.com as an ALIAS record to example.com, and example.com is an ALIAS record to the CloudFront distribution, Route 53 will respond to a request for www.example.com with the IP addresses of the CloudFront distribution - the same results as it returns for example.com.However, if use the following configuration:Include both www.example.com and example.com in "Alternate domain names"Create a example.com as an ALIAS A record to the CloudFront Distribution domain nameCreate www.example.com as an ALIAS A record to example.comEnsure that your TLS certificate includes both www.example.com and example.comThen you can visit www.example.com and your CloudFront distribution will be served without error.However, again, it will not be a redirect. That is, the browser will still show www.example.com, not example.com. For that you need to use an S3 website endpoint (as it seems like you have already done), or an edge function, as outlined in this article.CommentShareEXPERTbwhaleyanswered 7 months agoViridios 7 months agoHowever, again, it will not be a redirect. That is, the browser will still show www.example.com, not example.com yes i've tried that approach and noticed it. In fact the browser will treat www.example.com as an entirely different website which just has an identical content to example.com. This approach is also very bad for SEO. Thank you for your explanation about alias records. I'll accept it as the answerShare"


Question: Free tier billing for DynamoDB missing transfers?

"DynamoDB includes:25 GB of storage and up to 200 million read/write requests per month with the AWS Free TierMy monthly billing statement for S3 shows storage and transfers:Amazon Simple Storage Service Requests-Tier1$0.00 per request - PUT, COPY, POST, or LIST requests under the monthly global free tierAmazon Simple Storage Service Requests-Tier2$0.00 per request - GET and all other requests under the monthly global free tierAmazon Simple Storage Service TimedStorage-ByteHrs$0.000 per GB - storage under the monthly global free tierWhich is exactly as expected.  But DynamoDB is missing the "monthly global free tier" section:Amazon DynamoDB PayPerRequestThroughput$0.25 per million read request units (N. Virginia)$1.25 per million write request units (N. Virginia)Amazon DynamoDB TimedStorage-ByteHrs$0.00 per GB-Month of storage for first 25 free GB-Monthsand based on this I would expect a monthly bill of $0.01 for my piddly 6,000 read/write requests (out of the 200 million free tier limit).  What needs to be done to get the billing page to show "$0.00 per request under the monthly global free tier" for DynamoDB?FollowComment"

Answer: 0The DynamoDB free tier provides for Provisioned Capacity but not for On Demand.https://aws.amazon.com/dynamodb/pricing/provisioned/"25 WCUs and 25 RCUs of provisioned capacity for tables using the DynamoDB Standard table class"CommentSharejzhunteranswered a year ago


Question: Transfer Files from Windows 10 Pro Machine to AWS S3 using AWS Datasync

"Hello everyone,I am new to AWS Datasync service and I am working as an intern. I have been given a scenario where I have to transfer data from a Windows 10 Pro machine to AWS S3 using AWS DataSync. I have enabled NFS Services on the Windows Machine.Is it possible to create an agent on EC2 and execute a task? I tried to do that. I provided the source details with the Windows 10 Pro IP address and the folder to be shared and the destination as an S3 bucket. All the ports were enabled on the Datasync agent as well as the Windows Machine. However, I got the below error message when a task was executed,Execution statusErrorTask failed to access location loc-065df9303f651147a: x40016: Failed to connect to an NFS server on host 192.168.2.101. Ensure that your DataSync agent has a route to your NFS server and can make a TCP connection to port 2049. Please also ensure that the NFS service is running and listening on port 2049. Additionally, if this is a NFSv3 mountpoint, you will also need to expose ports 111 and 635. After resolving any networking issues, retry your task.Have I followed the right procedure for transferring data from the local machine to AWS S3?FollowComment"

Answer: "0H, If I understood your requirement correctly you want to transfer data from an on-premise Windows 10 machine to an S3 bucket.You do not need to utilize an EC2 instance you need to configure the data sync agent on your on-premise device.Check the following options mentioned in the documentation -> https://docs.aws.amazon.com/datasync/latest/userguide/transferring-on-premises-storage.htmlWorkshop -> https://catalog.us-east-1.prod.workshops.aws/workshops/74237958-77c8-4e7f-a02f-ae201a04d759/en-US/migrating-data-with-datasync-lab/02-datamigration/01-copyfiletos3CommentShareBisinaanswered 10 days ago0Hello,It is possible to deploy the DataSync agent as an EC2 instance but not typically recommended due to latencies connecting to a source storage system over a network. You can deploy the agent on a supported hypervisor on premises.https://docs.aws.amazon.com/datasync/latest/userguide/deploy-agents.htmlFor an agent deployed in AWS, you would need to provide a IP address that the agent can connect to and ensure you have the proper ports opened through your VPC security group as well.https://docs.aws.amazon.com/datasync/latest/userguide/datasync-network.htmlThe agent provides a local console that enables you to test connectivity to the storage system.https://docs.aws.amazon.com/datasync/latest/userguide/local-console-vm.html#self-managed-storage-connectivityCommentShareDarryl_Dioanswered 10 days ago"


Question: WHM with SES Services problem

"I m using WHM service in Lightsail. I have many domain name in this server. I m using SES to relay the mail.the first question, some user send email with error message"message has lines too long for transport"the second question, some email can't do forwarder.A message that you sent could not be delivered to one or more of itsrecipients. This is a permanent error. The following address(es) failed:XXXXX@gmail.comhost email-smtp.ap-southeast-1.amazonaws.com [XX.XX.XX.XX]SMTP error from remote mail server after end of data:554 Message rejected: Email address is not verified. The following identities failed the check in region AP-SOUTHEAST-1:cpanel@XXXXX.com, "cPanel on XXXXX.com" cpanel@XXXXXX.comwhat's the solution of this case? is it buy a Dedicated IP to slove this problem?FollowComment"

Answer: "0This article may help. It's about WordPress, but the same steps should apply when setting up cPanel. You will need a static IP to get everything working properly.https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-enabling-email-on-wordpressCommentShareAlexanswered 10 months ago"


Question: MediaConvert Image Inserter

"When creating a MediaConvert job, using the AWS PHP SDK, i am adding an image to the input video, like this:"InsertableImages" => [["Opacity" => 50,"ImageInserterInput" => 's3-image.jpg',"ImageX" => 0,"ImageY" => 0,"Layer" => 1,]]As i understood, ImageX and ImageY are offsets to the image, from the top-left corner.My questions are:Is there an easy way to choose another corner with the same offsets (top-right for example.)?How would i center an image?Thank you very much in advance.FollowComment"

Answer: "0Hi :)In image and video, the x,y coordinates are relative to (0,0) from the bottom left of the content.(0,0): Bottom Left(Pixel of image's horizontal, 0): Bottom Right(0, Pixel of image's vertical): Top Left(Pixel of image's horizontal, Pixel of image's vertical): Top Right(Pixel of image's horizontal / 2, Pixel of image's vertical / 2): CenterThe same applies to video.If you want the image to be centered in the video, you can specify the center position for the video in the ImageX and ImageY parameters.Example) Video Resolution: 1500 x 500(0,0): Bottom Left(1500, 0): Bottom Right(0, 500): Top Left(1500, 500): Top Right(1500 / 2, 500 / 2) = (750, 250): Center"ImageX" => 750, "ImageY" => 250(+ ImageX and ImageY indicate where to put the image in the video.)CommentShareSangbeomanswered 19 days ago"


Question: "OSError: [Errno 28] No space left on device -- PyTorch, CNN, estimator"

"hey there, I'm training a convolutional neural network (CNN) on a large dataset (10k images - 50 GB) stored on S3 bucket using estimator(sagemaker infrastructure) . everything works well when I work with 2000 images which has the total size of almost 5-10 GB. however, I get an error when I increase number of images to 3000 or more. the error indicates that there is no space left on device. I also attached the estimator setup to this message, as you can see I am using ml.g4dn.12xlarge instance which has 192 GB of memory!! I also increased the volume size to 900 GB. I still don't know why I am getting space/storage error!! I know that error is related the function "_get_train_data_loader" in which it is trying to download the images!! I was reading somewhere that EFS (elastic file system) might help with this issue, if so, I don't know how to specify it in the estimator. estimator = PyTorch(entry_point="pbdl_sm.py",role=role,framework_version="1.4.0",py_version="py3",instance_count=1,instance_type="ml.g4dn.12xlarge",volume_size = 900,hyperparameters={"epochs": 6, "backend": "gloo","lr": 0.001,"train_size":2900,"n_realz":3000},)FollowComment"

Answer: "0HiThank you for reaching out to us.In general the no space left on device error occurs when there is a high disk utilization, Requesting you to review the instance metrics and cloudwatch metrics/logs of the training job for more detailed information.It also depends on the various factors like learning rate, number epochs and configuration of the training job and the estimator, However I would recommend you to try distributed training with the pytorch using the smdistributed [1] on multiple instances , Currently, the following are supported: distributed training with parameter servers, SageMaker Distributed (SMD) Data and Model Parallelism, and MPI. SMD Model Parallelism can only be used with MPI.To enable the SageMaker distributed data parallelism: { "smdistributed": { "dataparallel": { "enabled": True } } }I would also recommend to try the PIPE mode, With Pipe input mode, your dataset is streamed directly to your training instances instead of being downloaded first. This means that your training jobs start sooner, finish quicker, and need less disk space.If you are facing any issues and require further investigation on the issue, I would encourage you to open a case with the premium support along with the training job ARN and the cloudwatch logs of the job.Due to security reason, this post is not suitable for sharing customer's resource.Reference:[1] https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/sagemaker.pytorch.html[2] https://aws.amazon.com/blogs/machine-learning/using-pipe-input-mode-for-amazon-sagemaker-algorithms/[3] https://github.com/aws/amazon-sagemaker-examples/blob/80df7d61a4bf14a11f0442020e2003a7c1f78115/advanced_functionality/pipe_bring_your_own/train.py[4] https://sagemaker-examples.readthedocs.io/en/latest/training/distributed_training/pytorch/data_parallel/maskrcnn/pytorch_smdataparallel_maskrcnn_demo.htmlCommentShareAWS-User-6649274answered a year ago"


Question: .NET AWS SDK Secrets Manager Unauthorized

"Hello,I am using the .NET AWSSDK to build my app, and I'm trying to use Secrets Manager. I've configured a programatic access user for the SDK, created an IAM role that allows access to the secret in question on the "GetSecretValue" action and ran it in code, but it throws an unauthorized exception.I've double-checked the SDK's region, it is set correctly to eu-central-1.I've used IAM Simulator, and it shows up as "allowed".The secret itself has no policies set on itself.I've also looked at https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html, but found no good answer.Amazon.SecretsManager.AmazonSecretsManagerException: User: arn:aws:iam::<ACCT_ID>:user/<USER> is not authorized to perform: secretsmanager:GetSecretValue on resource: <SECRET_NAME> because no identity-based policy allows the secretsmanager:GetSecretValue actionThe IAM policy:{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": "secretsmanager:GetSecretValue",            "Resource": "arn:aws:secretsmanager:eu-central-1:<ACCT_ID>:secret:<SECRET_NAME>"        }    ]}Any help is appreciated.Thank you,AndreiFollowComment"

Answer: 2Accepted AnswerTurns out it was a PEBCAK situation.I was unaware that the ARN of the secret is different from it's name. Fixing that in the IAM policy fixed the issue.CommentShareAndrei Havaanswered 6 months ago


Question: Aurora Serverless v2 crashed with signal 11

"Hello everyone!I recently migrated my MariaDB database to Aurora Serverless v2 (3 weeks ago) and today our Aurora restarted by itself at 11:22. I got the log, but I didn't understand what could have been the cause. (Because of the time zone the db hits 14:22, but it's normal).14:19:34 UTC - mysqld got signal 11 ;Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.Thread pointer: 0x4000604d2000Attempting backtrace. You can use the following information to find outwhere mysqld died. If you see no messages after this, something wentterribly wrong...stack_bottom = 4001070825a7 thread_stack 0x40000/rdsdbbin/oscar/bin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x2b) [0x23fac0b]/rdsdbbin/oscar/bin/mysqld(handle_fatal_signal+0x3db) [0x1262adb]linux-vdso.so.1(+0x85b) [0x400007d5a85b]/rdsdbbin/oscar/bin/mysqld(bind_fields(Item*)+0x20) [0x1077e20]/rdsdbbin/oscar/bin/mysqld(sp_lex_instr::reset_lex_and_exec_core(THD*, unsigned int*, bool)+0x5bb) [0xff3c3b]/rdsdbbin/oscar/bin/mysqld(sp_lex_instr::validate_lex_and_execute_core(THD*, unsigned int*, bool)+0xe7) [0xff46a7]/rdsdbbin/oscar/bin/mysqld(sp_head::execute(THD*, bool)+0x3df) [0xfea4df]/rdsdbbin/oscar/bin/mysqld(sp_head::execute_trigger(THD*, MYSQL_LEX_CSTRING const&, MYSQL_LEX_CSTRING const&, GRANT_INFO*)+0x1c3) [0xfeac43]/rdsdbbin/oscar/bin/mysqld(Trigger::execute(THD*)+0x97) [0x1189617]/rdsdbbin/oscar/bin/mysqld(Trigger_chain::execute_triggers(THD*)+0x23) [0x118ac63]/rdsdbbin/oscar/bin/mysqld(Table_trigger_dispatcher::process_triggers(THD*, enum_trigger_event_type, enum_trigger_action_time_type, bool)+0x4b) [0x11841eb]/rdsdbbin/oscar/bin/mysqld(fill_record_n_invoke_before_triggers(THD*, COPY_INFO*, mem_root_deque<Item*> const&, mem_root_deque<Item*> const&, TABLE*, enum_trigger_event_type, int, bool, bool*)+0x457) [0x1003017]/rdsdbbin/oscar/bin/mysqld(Sql_cmd_insert_values::execute_inner(THD*)+0x4a3) [0x16ce2a3]/rdsdbbin/oscar/bin/mysqld(Sql_cmd_dml::execute(THD*)+0x62f) [0x10de1cf]/rdsdbbin/oscar/bin/mysqld(mysql_execute_command_internal(THD*, bool)+0x1fef) [0x10805ef]/rdsdbbin/oscar/bin/mysqld(mysql_execute_command(THD*, bool)+0x18b) [0x108248b]/rdsdbbin/oscar/bin/mysqld(dispatch_sql_command(THD*, Parser_state*)+0x383) [0x10831c3]/rdsdbbin/oscar/bin/mysqld(dispatch_command(THD*, COM_DATA const*, enum_server_command)+0x13f3) [0x1084a53]/rdsdbbin/oscar/bin/mysqld(do_command(THD*)+0x17f) [0x1085a1f]/rdsdbbin/oscar/bin/mysqld(THD_task::process_connection()+0x183) [0x12505a3]/rdsdbbin/oscar/bin/mysqld(Thread_pool::worker_loop()+0x1cb) [0x124ee0b]/rdsdbbin/oscar/bin/mysqld(Thread_pool::worker_launch(void*)+0x23) [0x124efe3]/rdsdbbin/oscar/bin/mysqld() [0x29421bf]/lib64/libpthread.so.0(+0x72ab) [0x40000954a2ab]/lib64/libc.so.6(+0xd5e5b) [0x4000098fae5b]There are more logs in the following lines that I can bring if necessary.In my 'Recent Events' from the database instance I got this:October 26, 2022, 11:22 (UTC-03:00)DB instance restartedFollowComment"


Question: How do I create a VPC Endpoint for S3 Interface?

"All,I am trying to create an S3 interface endpoint.I select endpoint from the VPC endpoint GUIThere was an error creating VPC S3 interface endpoint"Enabling private DNS requires both enableDnsSupport and enableDnsHostnames VPC attributes set to true for vpc-049a6e29a22695b61"So I enable DNS on the VPCNow I try creating the VPC S3 Interface endpoint again, and get this"There was an error creating VPC endpointPrivate DNS can't be enabled because the service com.amazonaws.us-east-1.s3 does not provide a private DNS name."Feel like I am stuck in a loop here. Any idea what I am doing wrong?FollowComment"

Answer: "3You are receiving above because unfortunately, the Interface Endpoints for S3 don't support Private DNS. [1]. This feature is enabled by default in the creation settings. If your intentions are to access S3 from on premises or another region, a slight change needs to be made in the creation process. Please see the below steps in order to successfully create your S3 interface endpoint [2]:=================S3 Interface endpoint-Return to your VPC-Select > Endpoints, Create Endpoint.-Select > AWS service name (S3), Type (Interface)-After making your VCP selection > there is a drop down "Additional settings" > deselect "Enable DNS name" (Selected by Default)-Select > your Subnet/s and continue as normal to complete the endpoint.*Note: S3 interfaces use public, interface specific Regional and Zonal DNS names that are created when you make an S3 interface endpoint. These DNS names are resolved to the private IP's of your respective subnet ENI's within your VPC. Which can be accessed securely within the AWS network via these Private IPs. There is no need to configure a DNS resolver. I have also listed a link that elaborates on the comparison of S3 interface and gateway endpoints [3].[1]Private DNS for interface endpointshttps://docs.aws.amazon.com/vpc/latest/privatelink/vpce-interface.html#vpce-private-dns[2] Creating an Interface Endpointhttps://docs.aws.amazon.com/vpc/latest/privatelink/vpce-interface.html#create-interface-endpoint[3] S3 Endpoint comparisonhttps://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html#types-of-vpc-endpoints-for-s3CommentShareSUPPORT ENGINEERNirmal_Kariaanswered a year ago"


Question: Polly: GetSpeechSynthesisTask return length of audio

can we get the duration of the audio returned in the success responseFollowComment

Answer: "0Hi! Amazon Polly does not currently return the length of the audio. You would have to use third party libraries such as ffmpeg or mutagen to obtain the length from the file! Hope this helps! :)CommentShareDani Manswered 10 months agoDani M 10 months agoHey, did this work for you? Remember to accept the answer if it has helped you (^_^)Share0I can provide an alternative to Dani's suggestion. There is an approach which will allow you to determine the length of generated Polly audio by using nothing more than the Polly API, though it is a little non-obvious. You can request that Polly return "speech marks" - metadata which describes the synthesized speech. One type of speech mark data you can request are "visemes". These are timestamped descriptions of the face and mouth movements that a human would use to make each component sound found in the generated audio. The viseme data will always end with a viseme labelled "sil" which represents silence. The time-stamp of this last viseme represents the end of the final sound in the generated audio.Here's the viseme metadata returned for the phrase "Hello, world"...{"time":125,"type":"viseme","value":"k"}{"time":200,"type":"viseme","value":"@"}{"time":237,"type":"viseme","value":"t"}{"time":300,"type":"viseme","value":"o"}{"time":375,"type":"viseme","value":"sil"}{"time":562,"type":"viseme","value":"u"}{"time":750,"type":"viseme","value":"E"}{"time":837,"type":"viseme","value":"t"}{"time":1000,"type":"viseme","value":"t"}{"time":1212,"type":"viseme","value":"sil"}Viseme timestamps are expressed in milliseconds. So we can see that, given the voice I selected when generating the audio, the generated audio is 1.212 seconds in duration.CommentShareKrisanswered 10 months ago"