Question: [1.6] Problem with Vegetation Applied via Terrain Layers

"I am attempting to create vegetation variation by using the Terrain Layers to distribute groups of vegetation. I am running into what seems to be an LOD issue with this method. Initially I thought it to be one of two things, a texture LOD issue, or a terrain LOD issue. I believe I have eliminated these as a cause and am at this point completely stumped. I've dug through the cvars to find every possible variable and spent hours trying to find it but was unsuccessful. I did find the two vars I thought might fix it but here is the video to demonstrate the results.Starts out with e_TerrainLodRatio at 0. Switch "Detail layer view distance" in Environment from default to 0, then to 100. Switch e_TerrainLodRatio back to default (0.5).https://youtu.be/yE1XhLN4FmwI realize this is not the recommended method to distribute vegetation, but this is the most logical for the number of plants I want to distribute and the uniformity of the distribution over multiple varying areas.I believe this behavior can be replicated easily with any project/level.Thanks!Follow"

Answer: "0Accepted AnswerThis problem is directly related to the e_ProcVegetationMaxSectorsInCache system variable, which must be set in either the game.cfg or the system<platform>.cfg file. Just add a line like this:e_ProcVegetationMaxSectorsInCache=64This setting requires an editor restart, which is why changing the value in the editor doesn't show any change.REMOVEDUPLOADSharerePost-User-4796539answered 6 years ago0Hey @REDACTEDUSERREMOVEDUPLOADThat could help with the draw distance popping you are seeing. Let us know if adjusting these settings has any desired effects.SharerePost-User-8353690answered 6 years ago0These settings don't seem to have any affect on the popping that I'm seeing. As I said, I've tried every variable I could find that seemed to pertain to LOD on the vegetation, terrain mesh, and the textures themselves.Left: placed via Terrain Layers. Right: placed by painting.https://youtu.be/R8R_Lap0GkEThanks!SharerePost-User-4796539answered 6 years ago0Thank you. I have been able to replicate on multiple machines, and others have been able to replicate as well. It seems as if the veg. that is spawned on terrain layers is affected by a different LOD mechanism than typically painted/distributed vegetation. It seems to me as if it is related to the terrain mesh LOD, however unaffected by the cvars that change the way terrain LOD is calculated. I have spent quite a bit of time trying to narrow down the lines of code responsible for the distribution via terrain layers that might vary from the standard mass distribution method, but have come up short so far.SharerePost-User-4796539answered 6 years ago0Thanks for the video. Can you give me some insight on how you created the vegetation layers? This info might help our techs.SharerePost-User-8353690answered 6 years ago0Hey @REDACTEDUSERSharerePost-User-8353690answered 6 years ago0Thanks for the info. I will add this info to the ticket.SharerePost-User-8353690answered 6 years ago0Curious to see if there is a possible workaround for this issue. Maybe I'm missing a tool or method here. I'd like to randomly and consistently disperse groups of vegetation over only a portion of a terrain, with more than one group of vegetation types.REMOVEDUPLOADSharerePost-User-4796539answered 6 years ago0Ah, good find! Thanks for sharing.SharerePost-User-8353690answered 6 years ago0Ok, after some further digging through the code, it appears as if this issue is related to the cached sectors that the procedural objects are using. It seems that the cvar e_ProcVegetationMaxSectorsInCache can't be changed, which doesn't allow for any more sectors to be allocated. FYI there is a debug mode for the procedural vegetation e_ProcVegetation = 2:REMOVEDUPLOADREMOVEDUPLOADEDIT: Changing this cvar requires a restart, so it must be set in either the game.cfg or the system*.cfg. SOLVED!SharerePost-User-4796539answered 6 years ago0Hi all, I have some new info from one of our engineers:I made some improvements a few months back to how merged mesh vegetation LOD'd out over distance. The defaults are still quite aggressive, but can be changed with cvar. Check out e_MergedMeshesLodRatio and e_MergedMeshesViewDistRatio to control total distance that merged mesh vegetation renders and where the quality starts to drop off. Also e_MergedMeshesInstanceDistShadows controls how far away to render shadows on merged mesh vegetation. User should be able to make things much better by tweaking some values, at least in the latest version of LY.Hope this helps!SharerePost-User-8353690answered 6 years ago"


Question: GraphQL save self relation in custom resolver in amplify

"I have the following models:type Field @model {  id: ID!  fieldID: ID @index(name: "byField", sortKeyFields: ["name"])  name: String!  type: String!  required: Boolean  fields: [Field] @hasMany(indexName: "byField", fields: ["id"])}type Mutation {    batchCreateField(fields: [BatchCreateField]): [Field]}input BatchCreateField {  id: ID  fieldID: ID  name: String!  type: String!  required: Boolean  fields: [BatchCreateField]}And wrote a custom resolver:$util.log.info($util.toJson($context))#set($isAuthorized = false)#set( $createdAt = $util.time.nowISO8601() )#set($fieldsArray = [])#foreach($item in \${ctx.args.fields})  $util.qr($item.put("id", $util.defaultIfNullOrBlank($item.id, $util.autoId())))  $util.qr($item.put("createdAt", $util.defaultIfNull($item.createdAt, $createdAt)))  $util.qr($item.put("updatedAt", $util.defaultIfNull($item.updatedAt, $createdAt)))  $util.qr($item.put("__typename", "Field"))  $util.qr($fieldsArray.add($util.dynamodb.toMapValues($item)))#end## [End] Initialization default values. **$util.toJson( {  "version": "2018-05-29",  "operation": "BatchPutItem",  "tables": {    "Field-INSERT_APIID-INSERT_PROJECT_ENV": $fieldsArray  }} )Saving in batch works fine, the self relation is not working. Is there any way i can save this self relation like below in a batch and the resolver autofilling the sub fields with the fieldID of the previously inserted field? Or am i thinking in the wrong way and should i be handling this in another way.let fieldInput: CreateFieldInput = {  name: field.name,  type: field.type,  required: field.required,  fields: field.fields            };batchFieldsInput.push(fieldInput)API.graphql(graphqlOperation(batchCreateField, {fields: batchFieldsInput}))The data shown is just the subset of the payload. The datasets i'm saving are api's > endpoints (sometimes more than 100 endpoints) > fields & parameters (can be more than 100 per endpoint, since fields can have subfields, like shown), resulting in a very large amount of calls, if i cannot send the whole api object in one call.FollowComment"


Question: SES domain identity not verified or verification failed

I have created an SES domain 20 days back. It was verified as it was very simple to verify and it is out of sandbox. Now the issue is my domain is not able to send emails. The domain I am using is purchased from AWS. I have tried everything but I am not to get it verified now. I am using only one email only to send emails from the same domain and is not able receive emails as it is used only to send emails.Possible issue:My SES service is running in N.Virgenia but all the emails are send to Bahrain can that be an issue. If yes then what can be the solution as SES is not available in Bahrain.FollowComment


Question: Problem with the new Viewport Interaction Model

Anyone else who used the new Viewport Interaction Model found the Terrain Tool?I was trying to find it and even created a layout with the Terrain Tool and the Terrain Layer using the old viewport but when changing to the new viewport the Terrain Tool doesn't appear.Follow

Answer: "0It's disabled in the new viewport. Of course, no one tells you this. You have to figure it out for yourself what other things don't work because Amazon doesn't care enough about its users to tell them that or to provide even a rough roadmap. They just like to push us the crumbs after developing the things they in particular or their more important "customers" (read: not "users") want.SharerePost-User-9359627answered 3 years ago0Hi @REDACTEDUSERI'm sorry this wasn't communicated super well.There was a mention of this in the Release Notes for 1.19 but when coming to this later on I totally get it isn't clear.It might be worth us updating the messaging in the 'Enable New Viewport Interaction Model' popup to callout the features that are not yet supported.Thanks for the feedback and sorry the confusion,All the best,TomSharerePost-User-0765265answered 3 years ago0I can already tell that Amazon Lumberyard is making an effort to be more community and user-driven. I think this is the right step. Thanks for your response.SharerePost-User-9359627answered 3 years ago0Not at all and thank you very much for your kind words :slight_smile:We're always looking to improve and this feedback is super useful to help us spot things we missed.I've created a task to update this and I'm sure it will be in an upcoming releaseThanks again!TomSharerePost-User-0765265answered 3 years ago0Super minor thing but we did update the wording for this for the 1.24 Release!It's not perfect but should be a little more informative than before and will link to the relevant pages for more information.Thanks!TomSharerePost-User-0765265answered 3 years ago0Hi @REDACTEDUSERI see this problem in versions 1.24 and earlier 😭REMOVEDUPLOADSharerePost-User-8224505answered 3 years ago0Hi @REDACTEDUSERAh is this with the new Viewport Interaction Model Enabled? I'm afraid I do not believe the AI/Physics button is currently supported but it's something we're looking into for a future release.I am sorry for the inconvenience. This update was just to try and make the notification of features being different more clear.Thank you for the information!TomSharerePost-User-0765265answered 3 years ago0OK , GREAT @REDACTEDUSERI  switched to UI 2.0.I think it's better that this part can be resized and the sliding mode(REMOVEDUPLOAD is bad.REMOVEDUPLOADThank you 🙂 🙏SharerePost-User-8224505answered 3 years ago0You did call it out here @REDACTEDUSERhttps://youtu.be/9tP0_f4pGD8?t=445SharerePost-User-8914159answered 3 years ago0Hi @REDACTEDUSERthank you very much for your post, but I saw this video a few month ago🙏 🙂 🎮SharerePost-User-8224505answered 3 years ago0Yes, I agree with you completely @REDACTEDUSERI hope , see soon Lumberyard Prime Stable Version 🙂My main problem is bugs in script canvas 1.24,1.23,1.22 , etc ... but I see terrain bug in LY 1.24 & some component bugs in LY 1.24 & 1.23.First Strategy in 2016Products with less features but with strong and stable   codes and having the lowest bugs, much much better than products that are immense features and has many bugs and without Stable.https://forums.awsgametech.com/t/two-strong-strategy/1477...https://forums.awsgametech.com/t/welcome-to-strategy-seven/8109SharerePost-User-8224505answered 3 years ago"


Question: [Announcement] Introducing SQL Explorer in EMR Studio

"Amazon EMR Studio is an integrated development environment (IDE) that makes it easy for data scientists and data engineers to develop, visualize, and debug big data and analytics applications written in R, Python, Scala, and PySpark. Today, we are excited to introduce SQL Explorer, a feature in your EMR Studio Workspace that allows you to browse the data catalog and run SQL queries on EMR clusters from EMR Studio. This release of SQL Explorer in EMR Studio supports running SQL queries on Amazon EMR on EC2 clusters running Presto version 0.254.1 or higher.Presto is a fast SQL query engine designed for interactive analytic queries over large datasets from multiple sources. In SQL explorer, you can connect to Amazon EMR on EC2 clusters with Presto installed to view and browse the data catalog. Supported data catalogs include AWS Glue Data Catalog and self-hosted Hive Metastore version 3.1.2 or higher. SQL Explorer also provides you an Editor to run SQL queries, view the query results in a table, and download query results in a csv format. You also have the ability to run multiple SQL statements via different Editor tabs. SQL explorer is supported for Amazon EMR versions 6.4.0+.EMR Studio is available in US East (Ohio), US East (N. Virginia), US West (Oregon), Canada (Central), Europe (Ireland), Europe (Frankfurt), Europe (London), Europe (Stockholm), Europe (Paris), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo) and South America (Sao Paulo) regions.To learn more about SQL Explorer in EMR Studio see our documentation here. To see the feature in action see the demo video here.FollowCommentSrikanth_N SUPPORT ENGINEERa year agoThis is an announcement migrated from AWS Forums that does not require an answer.Share"


Question: Bug: UserFunction.js uses existsSync

"If this is not the right place to write this please advice.This is regarding AWS Lambda 14.x and the ESM support announcement from Jan 6 2022.It looks like the functionfunction _tryRequireFile(file, extension) {  const path = file + (extension || "");  verbose("Try loading as commonjs:", path);  return fs.existsSync(path) ? require(path) : undefined;}in /var/runtime/UserFunction.js uses fs.existsSync, which will return true for folders. This means if the application contains for example a handler.js file as well as handler folder and happens to be an esm module, it would always be loaded via "require" and we would get the following error:Must use import to load ES Module: /var/task/lib/handler.jsrequire() of ES modules is not supported.require() of /var/task/lib/handler.js from /var/runtime/UserFunction.js is an ES module file as it is a .js file whose nearest parent package.json contains \"type\": \"module\" which defines all .js files in that package scope as ES modules.Instead rename handler.js to end in .cjs, change the requiring code to use import(), or remove \"type\": \"module\" from /var/task/package.json.Clearly we would expect an esm module to simply "work" without having to worry about shadow folders.Seems to be a pretty straight forward bug. Happy fixing!Cheers, vincFollowComment"


Question: False positives with AWS DMS validations.

"Hi,I am using DMS to move data (5.200.000 records) between an RDS SQL Server Express database and an RDS MySQL database.The number of records transferred is perfect : every record count per table matches exactly.However when running the validation step I get a lot of mismatched records that I believe is an issue in the DMS validation process at the driver level (charset configuration probably):RECORD_DIFF[[{'Title': 'سلام ژوند څنګه دی (1) 😀🥳'}, {'Title': 'سلام ژوند څنګه دی (1) ??'}],]If I do the select query directly in the MySQL database with MySQL Workbench I get exactly the same emojis as I the DMS validation tool gets from the SQL Server.Besides this I am also getting a lot of RECORD_DIFF[[{'Content': '<VALUE_PRINT_NOT_SUPPORTED>'}, {'Content': '<VALUE_PRINT_NOT_SUPPORTED>'}],] which I am not sure what they mean querying both database I see the values perfectly for those rows.Any help will be greatly appreciated.Thanks,Claudio.FollowComment"

Answer: "0I have faced a similar issue before. You can raise a support ticket with AWS to help address the issue.Meanwhile, here are some options.Remove the table from your validation list. You can verify it using SQL queries and hash function later.If you know the reason for the mismatch, you can adjust for it using modified validation rules (see section titled Using JSON editor to modify validation rules).If the mismatch is due to charset, you can adjust it using Character substitution task settingsYou can use a combination of the 1,2, and 3. For example, you can validate the length of the column using validation settings but then validate the content later using hash function.Please let us know how you handled it.CommentShareAWS-VGanswered a year ago"


Question: StartMatchBackfill never returns

"Hi everyone,I am currently implementing Backfill for our game and ran into a problem trying to get Backfill to work. I am using UE4.21.2 and AWS GameLift SDK Version 3.30. I trying to call StartMatchBackfill() from a Dedicated Server running on a fleet. Also I am using the UE AWS Gamelift Plugin.The problem is that the process never returns from the call to Aws::GameLift::Server::StartMatchBackfill() and becomes unresponsive. (Meaning clients time out and the CPU is at 0% constantly. No more log output is generated after that call.)I can't really debug anything for this call, because I can't remote debug to the fleet instance.Anyone have any Idea what is going on? I should at least get an outcome and not have the process stop in its tracks and drop all players.My request seems to be correctly filled out. Here is an example:TicketId: empty. (I want AWS to generate one for me, just like in StartMatchmaking())MatchmakingConfigArn: arn:******GameSessionArn: arn:******There also is one player in the request, including latency data for just the region the game session is running in. (eu-central-1)FollowComment"

Answer: "0Hi, we've had a few posts around issues with StartMatchBackfill being called from the Server SDK with empty/null ticket IDs. For now, please create unique ticket IDs on your end to get unblocked right away. I'm working on a fix for empty/null ticket IDs from the Server SDK on our end and will update the forums when we have that fix available in all regions. Sorry for the inconvenience! If you're still seeing issues and not receiving an outcome after generating unique ticket IDs, update the post and we can dig deeper to see if there's anything else happening.CommentSharerePost-User-4192873answered 4 years ago0The GameLift Server SDK should now be able to call StartMatchBackfill() without needing to specify a ticketId in all regions. Please let us know if you still have issues with blank ticketIds in the future!CommentSharerePost-User-4192873answered 4 years ago0I'm just now debugging an issue where StartMatchBackfill is blocking and never returning.We're also on version 3.30, Unreal 4.20 so we can't easily upgrade to the latest SDK.However, in our case we ARE setting a backfill ticket ID, of the format:"b-<sessionID>-<incrementing number >"Example:BackfillTicketIDToSend=b-702f4f0d-0e50-421f-a8da-bbea6f8bdbc9-0SessionID=arn:******MatchmakingConfigurationArn=arn:******Players=3Are there other circumstances that could be triggering the blocking behavior? Thanks!CommentSharerePost-User-1492204answered 2 years ago0@REDACTEDUSERI will ping the GameLift team on this.Firstly, how are you allocating your backfill request? If you are using the stack, then you may want to move to dynamically allocated memory due to the size of this request (depending on how much data you are packing in there).Obviously it shouldn't hang but it can lead to data mangling if your request is too large.Secondly, do you see any matchmaking tickets created for your request?Thirdly, try taking your backfill request to the cli first. Ensure that all your fields are correctly formatted and you can make the request via the cli. I vaguely recall there being some issues in the past where StartMatchBackfill would fail on the server sdk as it didn't handle errors very well.CommentSharerePost-User-5332620answered 2 years ago0Sorry about the delayed reply on this.  I took a look at our system logs for the time of your failed request and see the problem.  It looks like you are constructing some of the AttributeValues associated with this request as AttributeValue() (at a minimum, "ranking" looked to be like this).  We really shouldn't allow that as it creates an untyped AttributeValue which will always return this error.  Instead, be sure that you are always constructing these with the desired type, e.g. AttributeValue(3.0), or Attribute("DesiredMapName").  We also should have better error messages for these cases and have logged a bug to capture this.In the meantime, Pip's advice is solid if you encounter issues in this area -- try the request in the CLI first as the error messaging is more robust there and should help you avoid some trip-ups like this.CommentSharerePost-User-0985817answered 2 years ago0Great thank you this is very helpful. The hang did indeed go away when we explicitly set the ranking attribute in our request.It does seem to imply though that something within the Unreal plugin was setting the null ranking value, ex AttributeValue(). From our perspective the interface for attributes is to pass in a map of name/value pairs on the FPlayer struct:TMap<FString, FAttributeValue> m_playerAttributes;I haven't debugged, but my first guess is that FAttributeValue type was set to None. But those were properties sent to us by the flexmatch OnUpdateGameSession, so there may be a bug there if it's not setting the type.In any case, thank you for the help. We'll know what to do in the future if we trigger this again.CommentSharerePost-User-1492204answered 2 years ago"


Question: "Allow Internet Access based on DNS, Outbound Security Group only allow to enter IP Address not DNS name.."

"I required to give internet access for EC2 instance only a list of websites, in Outbound security group I only have provision enter the IP address.Is there any other alternative way to restrict permission/access based on DNS.Further, I have tried to find out the IP addresses of the websites allowed in Outbound security group some does work others doesn't.Thanks in Advance.FollowComment"

Answer: "0Is there any other alternative way to restrict permission/access based on DNS.Yes, there are several ways. [1] [2] [3][1] How to set up an outbound VPC proxy with domain whitelisting and content filtering | AWS Security Bloghttps://aws.amazon.com/jp/blogs/security/how-to-set-up-an-outbound-vpc-proxy-with-domain-whitelisting-and-content-filtering/[2] FAQs - AWS WAF - Amazon Web Services (AWS)https://aws.amazon.com/waf/faqs/?nc1=h_ls#:~:text=Are%20the%20existing%20matching%20conditions%20compatible%20with%20the%20Rate%2Dbase%20Rule%3F[3] Cloud firewall - AWS Network Firewall - Amazon Web ServicesCloud firewall - AWS Network Firewall - Amazon Web Serviceshttps://aws.amazon.com/network-firewall/?nc1=h_ls&whats-new-cards.sort-by=item.additionalFields.postDateTime&whats-new-cards.sort-order=desc#:~:text=Flexible%20protection%20through%20fine%2Dgrained%20controlsFurther, I have tried to find out the IP addresses of the websites allowed in Outbound security group some does work others doesn't.It is difficult to answer the above question because we do not know the details of the event.CommentSharemn87answered 8 months ago"


Question: Tests fail on Pie: AatpDeviceManager The app was built for an older version

"On some Android 9.0 devices we see AatpDeviceManager popup window and all tests fail afterwards because of lost window focus.Link to RUN: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/a5cd451c-6593-4e2d-988d-3cfe012bcdd7/runs/41642dd2-62df-4644-90b6-fcad3f1696a8Affected devices includeGoogle Pixel 2Google Pixel 3Samsung Galaxy 10+Samsung Galaxy 10ePopup dialog that steals the focus has:title: AatpDeviceManagermessage: The app was built for an older version of Android and may not work properly. Try checking for updates, or contact the developer.buttons: Check for update and OKPopup appears when "tapping on HOME button". We use following UI Automator commands to simulate HOME button:UiDevice mDevice = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation());mDevice.pressHome();We use Gradle plugin and Android instrumentation.UI Automator version: androidx.test.uiautomator:uiautomator:2.2.0Since com.amazon.aatp.aatpdevicemanager is an internal Amazon app it would be good that Amazon team have a look into this issue that limits number of usable Android 9 devices heavily to only few. Search/googling didn't reveal anything related to this issue with Aatp.Edited by: marjanovic on May 16, 2019 3:12 AMEdited by: marjanovic on May 16, 2019 3:17 AMEdited by: marjanovic on May 16, 2019 4:54 AMEdited by: marjanovic on May 16, 2019 4:55 AMFollowComment"

Answer: "0Thank you for reporting this issue.Yes you are right, we are working on a fix to get not have this popup appear on these latest devices.CommentShareAWS-User-9707822answered 4 years ago0Thank you for fast feedback. Is there maybe a process in place to track the progress on this issue or will there be maybe follow up comment in this thread when issue is solved? We're just wondering if we should recheck affected devices periodically which is time consuming or it's enough to track activity in this thread.Thank you very much for the support.CommentSharemarjanovicanswered 4 years ago"


Question: Cognito (with google) + Application Load Balancer = 414 Request-URI Too Large

"I have configured the Application Load Balancer to sit in front my application hosted in ECS. The load balancer has a rule to Authenticate using Cognito User Pool and then forward the request to a target group.I get the prompt to enter my Google credentials the login appears to be successful, with the url in the format https://{domain}/oauth2/authorize?client_id={id}&redirect_uri=https%3A%2F%2{domain}.%2Foauth2%2Fidpresponse&response_type=code&scope=openid&state={state here}The problem here is I get "414 Request-URI Too Large". I have no indication that this is from my application and this is a response from the load balancer. The length of the State in the url is 20,514 charactersMy question is this a bug or what am I doing wrong?FollowComment"

Answer: "0Hi, Thank you for using AWS re:PostThe reason why the load balancer sends you the HTTP 414: URI too long error is because the request URL or query string parameters are too large [1].In order to fix this, we can try enabling client secret for the application in AWS Cognito [2].Please keep in mind that if the ARN length exceeds the 1600 bytes limit it could also cause this error [3].Here are additional articles that might be helpful:[1] https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-troubleshooting.html#load-balancer-http-error-codes[2] https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-settings-client-apps.html[3] https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.htmlCommentShareVardhan_Panswered 9 months ago"


Question: Dynamo DB Kinesis Steams Best Practice

"Hi,My customer currently has a large DynamoDB table with DynamoDB Streams enabled and two Lambda consumers of the Stream.  They wish to be able to handle up to five additional consumers, but are aware of potential throttling if more than two consumers are reading from the same stream.  They are considering using a Kinesis Data Steam for DynamoDB to fan-out the events to the additional consumers, they have raised the following questions:Is it good practice to add the Kinesis Data Steam for DynamoDB with the existing DynamoDB Stream and two Lambda consumers still in place, or should they look at removing the existing DynamoDB Stream and use the Kinesis Data Steam for DynamoDB for all consumers?Any pitfalls they should be aware of when using a Kinesis Data Steam for DynamoDB vs DynamoDB Streams?  Ones I can thin of are:When using Kinesis Data Steam for DynamoDB records may appear in a different sequence than the actual modifications to the item, where in a DynamoDB Steam the records appear in a time-ordered sequenceThe number of Kinesis Data Steam Shards will need to be managed when using Kinesis Data Steam for DynamoDB where this is managed for you when using DynamoDB Streams.Any guidance much appreciated.ThanksFollowComment"

Answer: "2Accepted AnswerTo help answer your questions:Its always better to create 1 stream from DDB, either DDB streams or a Kinesis. So if your customer is planning to use more than 2 consumers, I would suggest them to move to Kinesis streams and then have all their Lambda consume from Kinesis. You also get the advantage of data retention over DDB streams.Besides the ones that you have mentioned:a. Stream records may appear more than once in the Kinesis data stream. You can use the timestamp attribute ApproximateCreationDateTime on each record to identify the actual order in which item modifications occurred, and to identify duplicate records in the stream.b. Using Kinesis Data Streams for DynamoDB is subject to both Kinesis Data Streams pricing for the data stream and DynamoDB pricing for the source table. So beware of the additional costs involved of Kinesis plus DynamoDB charges for change data capture for Kinesis Data Streams. They have to also calculate the red. # of shards reqd based on their DDB table.CommentShareEXPERTParnabanswered 2 years ago"


Question: Amazon S3 using Excel VBA

"How to download/upload files on PRIVATE buckets in Amazon S3 from Excel VBA? or alternatively, what are VBA instructions to create presigned URLs?FollowComment"

Answer: "0hiYou can try using the AWS .NET SDKInstall the AWS SDK for .NETSet up your AWS credentials, including your AWS Access Key and Secret Access Key.Create an S3 client using the AWS .NET SDK in your VBA code. You will need to provide the client with your AWS credentials and the S3 bucket name.To upload a file to the S3 bucket, use the PutObject method of the S3 client. You will need to provide the file path, key name (i.e., the name of the file in the bucket), and the bucket name.To download a file from the S3 bucket, use the GetObject method of the S3 client. You will need to provide the key name and the bucket name.Sample code to download a file from a private S3 bucketSub DownloadFileFromS3()    Dim s3Client As Object    Dim response As Object    Dim keyName As String    Dim bucketName As String    Dim localFilePath As String       'Set AWS credentials    Set s3Client = CreateObject("Amazon.S3.AmazonS3Client")    s3Client.Initialize("<Your Access Key>", "<Your Secret Key>", "<Region>")       'Set S3 bucket name and key name    bucketName = "<The bucket name you want to access>"    keyName = "<Object name>"       'Set local file path to download the file    localFilePath = "<Local path where the file will be stored>"       'Download the file from S3 bucket    Set response = s3Client.GetObject(bucketName, keyName)       'Save the downloaded file to local folder    response.WriteToFile localFilePathEnd SubThanksArunCommentShareArunanswered 3 months ago"


Question: Deploy step in pipeline build fails with access denied

"Hello,I am following the blog https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/set-up-ci-cd-for-aws-appsync-graphql-api-updates.htmlThe CDK deploy seems to be ok and the build starts of fine. But then, the step DeployAlpha fails withAccess Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: ....  S3 Extended Request ID: ....; Proxy: null)The only thing I have changed compared to the provided code in the blog (or the version in github) is to change the github branch name from main to 'm a s t e r' (ignore the spaces - apparently the name of the branch is no longer allowed not even in a post)Would anyone have any tips on how to get this fixed?Tx!PeterFollowComment"

Answer: "1Hi Peter,The S3 access denied appears to be to the PipelineAsset bucket (ex: pipelinestack-pipelineartifactsbucketaea9a052-xxxxxxbb1c),  but the executing role for the 'DeployAlpha' 'AppsyncCdkAppStack.Prepare' uses 'arn:aws:iam::xxxxxxxxxxxx:role/cdk-hnb659fds-cfn-exec-role-xxxxxxxxxxxxx-us-west-2'. Looking at the permissions for this role, I can see that it does not have GetObject access to the above pipeline asset bucket. As a test, manually and temporarily adding S3 GetObject permissions to the pipelinestack-pipelineartifactsbucket for this role will allow the pipeline to succeed.Note that this sample uses an older CDK version and updating to the latest will change how to run this sample; the methods used in this sample are deprecated.CommentShareRyan_Janswered a year agoPeter a year agoHello, thanks for replying! I will test your proposal asap and provide feedback here.ShareRyan_J a year agoHi Peter,Just following up. Did my comments help unblock you?Share0Sorry for the delay (getting caught up in other things). It's still not working. I even gave public access to the pipeline bucket but the same error is thrownPeterCommentSharePeteranswered a year agoPeter a year agoSo I tracked down the deployment artefact in the bucket that was given the problem. I also gave public access to that and then it looks like the pipeline continues a bit further. However, next problem"Failed to assume service-linked role ...aws-service-role/appsync.amazonaws.com/AWSServiceRoleForAppSync, please retry. (Service: AWSAppSync; Status Code: 403; Error Code: AccessDeniedException; Request ID: 26297786-44f1-4763-b38f-a0d27154b013; Proxy: null)"Seems to happen when trying to create the dynamodb tableShare0To summarize. After upgrading to version aws-cdk libs version 1.138.0 (the latest 1.x for now) and some weird struggles with npm versions it eventually started working...which is niceWhat I don't understand is how people actually develop in such an environment. This (really small) build pipeline takes ~10 minutes or so and depending on what part one is developing for there is really no escape from running your code inside aws. A 10 minute tdd cycle does not make sense whatsoever, and then I have not thought through what it would mean to debug issues.Any tips anyone?CommentSharePeteranswered a year ago"


Question: S3 Select Compressed Pricing

"If I run an s3 select query on a compressed file, will I be charged for the bytes scanned on the compressed file size or the decompressed file size? It is not clear at all from the documentation.The s3 select response returns the BytesScanned and BytesProcessed fields. BytesScanned is the compressed files size and BytesProcessed is the decompressed file size.FollowComment"

Answer: 1You will be charged for the number of bytes scanned in the compressed file. Amazon S3 Select works directly on your compressed data and does not need to decompress the data before processing the SELECT statement.You can easily test this by running a S3 select on an compressed and an uncompressed file and comparing the Usage Reports. Refer to https://docs.aws.amazon.com/AmazonS3/latest/userguide/aws-usage-report.html to create the usage reports (See Downloading the AWS Usage Report)CommentShareAnanth Tirumanuranswered 5 months ago


Question: accessing iot core topic from within docker c++ program

"I have a c++ program that runs within a docker container.docker container receipe:{"RecipeFormatVersion": "2020-01-25","ComponentName": "au.com.mycompany.smartdvr.docker.streamcontroller","ComponentVersion": "1.3.22","ComponentType": "aws.greengrass.generic","ComponentDescription": "A component that runs the smart dvr Docker container from a private Amazon ECR image. Video/Audio saving and streaming to Kinesis added logging to file, file culling,ridgun optimisation","ComponentPublisher": "MYCompany","ComponentConfiguration": {"DefaultConfiguration": {"accessControl": {"aws.greengrass.ipc.mqttproxy": {"au.com.mycompany.smartdvr.docker.streamcontroller:mqttproxy:1": {"policyDescription": "Allows access to publish/subscribe to all topics.","operations": ["aws.greengrass#PublishToIoTCore","aws.greengrass#SubscribeToIoTCore"],"resources": ["*"]}}}}},"ComponentDependencies": {"aws.greengrass.DockerApplicationManager": {"VersionRequirement": ">=2.0.0 <2.1.0","DependencyType": "HARD"},"aws.greengrass.TokenExchangeService": {"VersionRequirement": ">=2.0.0 <2.1.0","DependencyType": "HARD"},"au.com.mycompany.smartdvr.docker.gstd": {"VersionRequirement": ">=0.0.1 <5.0.0","DependencyType": "HARD"}},"Manifests": [{"Platform": {"os": "linux"},"Lifecycle": {"Run": "echo '==========>>>>>>>'; $(docker kill streamcontroller || true) ; $(docker rm streamcontroller || true); docker run --name=streamcontroller --cap-add=SYS_PTRACE --runtime=nvidia -e DISPLAY=$DISPLAY --privileged --volume /tmp/.X11-unix:/tmp/.X11-unix --net=host -e NVIDIA_VISIBLE_DEVICES=all -v $HOME/.Xauthority:/root/.Xauthority -v /run/udev/control:/run/udev/control -v /greengrass/v2:/greengrass/v2 -v $AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT:$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT  -e SVCUID -e AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT=$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT -v /dev:/dev -v /sys/firmware/devicetree/base/serial-number:/sys/firmware/devicetree/base/serial-number -v /data:/data -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics   xxxxxxxx.dkr.ecr.ap-southeast-2.amazonaws.com/smartdvr:latest2 runstreamcontroller.sh"},"Artifacts": [{"Uri": "docker:xxxxxxx.dkr.ecr.ap-southeast-2.amazonaws.com/smartdvr:latest2","Unarchive": "NONE","Permission": {"Read": "OWNER","Execute": "NONE"}}]}],"Lifecycle": {}}inside the c++ program i subscribe to an iot core topic like:void ExternalBroker::subscribe_to_video_request_topic(){    string topic("mtd/");    topic += LocalConfig::get_machine_id();    topic += "/v1/mdvr/video/request";    g_logging->info("External Broker subscribing to topic " + topic);    QOS qos = QOS_AT_MOST_ONCE;    int timeout = 10;    g_logging->info("External Broker  : --00" );    SubscribeToIoTCoreRequest request;    request.SetTopicName(topic.c_str());    request.SetQos(qos);    g_logging->info("External Broker  : --01" );    ExternalBrokerVideoRequestTopicHandler streamHandler;    SubscribeToIoTCoreOperation operation = ipcClient.NewSubscribeToIoTCore(streamHandler);    g_logging->info("External Broker  : --02" );    auto activate = operation.Activate(request, nullptr);    g_logging->info("External Broker  : --03" );    activate.wait();    g_logging->info("External Broker  : --11" );    auto responseFuture = operation.GetResult();    g_logging->info("External Broker  : --22" );    if (responseFuture.wait_for(std::chrono::seconds(timeout)) == std::future_status::timeout)     {        std::cerr << "External Broker Operation timed out while waiting for response from Greengrass Core." << std::endl;        return;    }    g_logging->info("External Broker  : auto response = responseFuture.get()" );    auto response = responseFuture.get();    g_logging->info("External Broker  : respnse" );    if (!response) {        // Handle error.        g_logging->error("External Broker  : ERROR" );        auto errorType = response.GetResultType();        if (errorType == OPERATION_ERROR) {            auto *error = response.GetOperationError();            // Handle operation error.            g_logging->error("External Broker  : operation ERROR" );                    } else {            // Handle RPC error.            g_logging->error("External Broker  : RPC ERROR" );        }    }    else    {        g_logging->info("External Broker  : got response: " );    }}but it never gets past theactivate.wait();and when i publish something on that topic using the consoles test page ( https://ap-southeast-2.console.aws.amazon.com/iot/home?region=ap-southeast-2#/test ) I never see the callback to theclass ExternalBrokerVideoRequestTopicHandler : public SubscribeToIoTCoreStreamHandlervoid ExternalBrokerVideoRequestTopicHandler::OnStreamEvent(IoTCoreMessage *response) {    g_logging->info("External Broker  : aa" );    auto message = response->GetMessage();    if (message.has_value() && message.value().GetPayload().has_value())     {        g_logging->error("External Broker  : bb" );        auto messageBytes = message.value().GetPayload().value();        std::string messageString(messageBytes.begin(), messageBytes.end());        std::string topicName = message.value().GetTopicName().value().c_str();        // Handle message.        g_logging->info("External Broker received iot message on " + topicName + " messgae: " +  messageString);        // try to load the json        json payload_json = nlohmann::json::parse(messageString);        if( payload_json )        {            g_logging->info("External Broker  : cc" );            // forward request to queue and save queue        }        else        {            g_logging->error("External Broker can't load " +  messageString);        }    }log output:[2021-12-13 05:29:59.956] [streamcontroller] [info] External Broker trying to connect to Greengrass APC[2021-12-13 05:29:59.964] [streamcontroller] [info] External Broker conneted[2021-12-13 05:29:59.964] [streamcontroller] [info] External Broker set_connected :[2021-12-13 05:29:59.964] [streamcontroller] [error] External Broker  : 22[2021-12-13 05:29:59.964] [streamcontroller] [error] External Broker  : 33[2021-12-13 05:29:59.964] [streamcontroller] [info] External Broker subscribing to topic mtd/smartdvr-1423019132001/v1/mdvr/video/request[2021-12-13 05:29:59.964] [streamcontroller] [info] External Broker  : --00[2021-12-13 05:29:59.965] [streamcontroller] [info] External Broker  : --01[2021-12-13 05:29:59.965] [streamcontroller] [info] External Broker  : --02[2021-12-13 05:29:59.965] [streamcontroller] [info] External Broker  : --03[2021-12-13 05:30:09.907] [streamcontroller] [info] ExternalBroker timer tick..[2021-12-13 05:30:19.947] [streamcontroller] [info] ExternalBroker timer tick..[2021-12-13 05:30:29.952] [streamcontroller] [info] ExternalBroker timer tick..[2021-12-13 05:30:40.003] [streamcontroller] [info] ExternalBroker timer tick..i can't see any errors in greengrass.logwhat am i missing here ?FollowComment"

Answer: "0Accepted AnswerYour hunch is correct; when the operation goes out of scope the subscription is closed.What you're trying to do is very reasonable (and embarrassingly, very basic, expected functionality), but the current API doesn't allow it since the operation types aren't movable.I initially looked at making the operations movable but ultimately decided not to due to the complexity of the web of objects that make up an operation.  Instead, we have tentatively decided on a second set of factory functions for operations that return pointers, giving you complete control over operation lifetime.  This avoids messing with the delicate relationships between the various eventstream ipc objects.You can find a preview version of this functionality here:: https://github.com/aws/aws-iot-device-sdk-cpp-v2/tree/NewIpcApiExampleI'll be doing some testing on it next and we welcome any feedback or questions (ideally on the github repository/PR) if you'd like to try it out.CommentShareAWS-BretAmbroseanswered a year agoclogwog a year agoThank you ! I look forward to it being merged. Kind regards, tomShare0Hello,I would suggest that you read https://docs.aws.amazon.com/greengrass/v2/developerguide/run-docker-container.html#docker-container-ipc if you haven't already.There is specific setup required for a docker container to be able to use Greengrass IPC as it requires access to a specific Unix domain socket and some environment variables.Greengrass.log file shows all IPC connections, so I'd recommend viewing the log and verifying that you see that your component is able to connect over IPC. Once the connection has been made, then the behavior is the same inside and outside of the container.Cheers,MichaelCommentShareEXPERTMichaelDombrowski-AWSanswered a year agoclogwog a year agoHello Michael,thank you for that. I've gone through that setup ( see the -v $AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT:$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT  -e SVCUID -e AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT=$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT part of my docker startup and the accessControl in the receipe.)I've been able to send a message to AWS IoT using a python script in the same Docker image using the same sockets and access control, but it looks like i'm doing something wrong for subscribing to an aws IoT Core topic.I'll have another good look at greengrass.log for the socket connections. thank you !ShareMichaelDombrowski-AWS EXPERTa year agoYes, please do check on the logs. Make sure that you are mounting the directory which contains Greengrass as that directory also contains the IPC socket.If you can run without Docker, I'd also suggest doing that to remove Docker as a source of issues, that way we can narrow it down to whatever the true problem is.Shareclogwog a year agoIf i Follow https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-iot-core-mqtt.html#ipc-operation-subscribetoiotcore and run it as a standalone program inside the docker then it works.at the end of the example shown above it does awhile (true) {std::this_thread::sleep_for(std::chrono::seconds(10));}however in a real program this is not an option as i need to do other stuff as well.what is the suggested way to keep a subscription alive ?i tried connecting and subscribing and waiting in a thread, but that crashes my program.My guess is that it stops subscribing as soon asSubscribeToIoTCoreOperation operationgoes out of scope. is this a good guess ?how would i get around this ?ShareMichaelDombrowski-AWS EXPERTa year agoIf it is going out of scope and causing a problem, then you need to create it on the heap rather than the stack and free the memory at an appropriate time.I would also suggest implementing onStreamError, such asbool OnStreamError(OperationError *operationError) override {        std::cout << "Error received from SubscriptionHandler: " << operationError->GetMessage().value() << std::endl;        return false;    }This will then print if there are any problems on the stream.Shareclogwog a year agoHow would i go about changing :SubscribeToIoTCoreOperation operation = ipcClient.NewSubscribeToIoTCore(streamHandler);to be created in the heap ?  It is created using the NewSubscribeToIoTCore() factory function and returns an object and SubscribeToIoTCoreOperation doesn't seem to have a default constructor that I can use to add to my class and wait for it to be created ?ShareShow 4 more comments0getting a bit closer now. Finding that right after the subscription goes all the way to the server (found it in CloudWatch in the AWSIotLogsV2 group) the streamHandler ( of type class ExternalBrokerVideoRequestTopicHandler : public SubscribeToIoTCoreStreamHandler ) that is used in        SubscribeToIoTCoreOperation operation = ipcClient.NewSubscribeToIoTCore(streamHandler);receives a OnStreamClosed callback right away.is that because in my subscribe method operation goes out of scope ?If this is the case, how do I keep the subscription going ? I can't seem to create an emptySubscribeToIoTCoreOperation operation in my class and set it during the subscription method.The example at https://docs.aws.amazon.com/greengrass/v2/developerguide/interprocess-communication.html#ipc-subscription-handler-best-practices is a simple main() program where operation is created like:        SubscribeToIoTCoreOperation operation = ipcClient.NewSubscribeToIoTCore(streamHandler);but in a real program I would like to start a subscription and then continue with other stuff that my program needs to do.Is my hunch that the subscription stops because the operation goes out of scope correct ? And if so, how can i keep the operation around ?my latest subscription method:void ExternalBroker::subscribe_to_video_request_topic(){    try    {        string topic("mtd/");        topic += LocalConfig::get_machine_id();        topic += "/v1/mdvr/video/request";        int timeout = 10;        subscription_request.SetTopicName(topic.c_str());        subscription_request.SetQos(QOS_AT_LEAST_ONCE);        SubscribeToIoTCoreOperation operation = ipcClient.NewSubscribeToIoTCore(streamHandler);        auto requestStatus = operation.Activate(subscription_request).get();        if (!requestStatus)        {            g_logging->error("Failed to send subscription request to " + topic);            return;        }        auto subscribeResultFuture = operation.GetResult();        auto subscribeResult = subscribeResultFuture.get();        if (subscribeResult)        {            g_logging->info("External Broker  : Successfully subscribed to topic: " + topic);        }        else        {            auto errorType = subscribeResult.GetResultType();            if (errorType == OPERATION_ERROR)            {                OperationError *error = subscribeResult.GetOperationError();                if (error->GetMessage().has_value())                {                    string anerr("Greengrass Core responded with an error: ");                    g_logging->error( anerr + error->GetMessage().value().c_str());                }            }            else            {                string err("Attempting to receive the response from the server failed with error code: ");                g_logging->error( err + subscribeResult.GetRpcError().StatusToString().c_str());            }        }    }    catch (std::exception &e)    {        string test("External Broker  : Exception subscribing: ");        g_logging->error(test + e.what() );    }}CommentShareclogwoganswered a year ago0Hi Michael, I don't think it is a problem with the access to the socket as a python program in the same docker can publish to a iot core topic. Also I just recently seen that the subscription makes it all the way to IoT Core.Unfortunately I need to run this in the Docker as it uses the IPC to talk to IoT Core.CommentShareclogwoganswered a year ago"


Question: Ground Truth Label Jobs - Same document repeating in the labeling job

We want to extract the entities from pdf documents. We are exploring labeling the existing document using Safemaker GroundTruth. The documents are keep repeating for some reasons which we are not able to identify the reason ? When it repeats some pages coming as read only and other pages will us to annotate. When it repeats the previously read only pages are annotable .Can some expert from this help us to understand why it is repeating ?FollowComment


Question: Error trying to detach network interface

"Hello all,I get the following error when trying to detach a network interface in CLI 2.0:An error occurred (OperationNotPermitted) when calling the DetachNetworkInterface operation: The network interface at device index 0 cannot be detached.Tried with both force and no force, instance is stopped still not able... Any help appreciated.eni-attach-0867e4aaa19f25409 is this interface in question.Thanks,FollowComment"

Answer: "0Hi jflinnergex,Sorry to hear about the inconvenience you are facing. The primary network interface is assigned to network card index 0, which is what you are trying to detach, however you can only detach a secondary network interface. Each instance has a default network interface, called the primary network interface. You cannot detach a primary network interface from an instance, and it will only get detached or released when the instance is terminated. If you need to preserve the ENI please make sure the delete on termination flag is not selected when terminating the instance: https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-network-interface-attribute.htmlFor more information: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#eni-basicsI hope this helps. Let me know if you have any questions.Warm regards,awsamyCommentShareamz99answered 2 years ago"


Question: powershell cloudtrail trying to get instance id from requestparameters

"I am trying to pull instance Id and other parameters from cloudtrail using ps like so$results = Find-CTEvent -StartTime (Get-Date).AddMinutes(-30) | ? {$_.EventName -eq "TerminateInstances"}{"eventVersion":"1.08","userIdentity":{"type":"IAMUser","principalId":"xx","arn":"arn:aws:iam::462518063128:user/awslab1","accountId":"xxx","acces                   sKeyId":"xx","userName":"awslab1","sessionContext":{"sessionIssuer":{                   },"webIdFederationData":{},"attributes":{"creationDate":"2022-05-27T14:28:44Z","mfaAuth                   enticated":"false"}}},"eventTime":"2022-05-27T17:04:12Z","eventSource":"ec2.amazonaws.c                   om","eventName":"TerminateInstances","awsRegion":"us-west-1","sourceIPAddress":"AWS                    Internal","userAgent":"AWS Internal","requestParameters":{"instancesSet":{"items":[{"in                   stanceId":"i-07efe3d31ef2cef02"}]}},"responseElements":{"requestId":"dde64a51-2fd6-40ef                   -b9d6-06fde8a2abd9","instancesSet":{"items":[{"instanceId":"i-07efe3d31ef2cef02","curre                   ntState":{"code":32,"name":"shutting-down"},"previousState":{"code":16,"name":"running"                   }}]}},"requestID":"dde64a51-2fd6-40ef-b9d6-06fde8a2abd9","eventID":"dfc1fa38-c5db-401d-                   9ac9-11cd5ab41dd8","readOnly":false,"eventType":"AwsApiCall","managementEvent":true,"re                   cipientAccountId":"462518063038","eventCategory":"Management","sessionCredentialFromCon                   sole":"true"}then convertfrom json$results.CloudTrailEvent | ConvertFrom-JsoneventVersion                 : 1.08userIdentity                 : @{type=IAMUser; principalId=xxxx;arn=arn:aws:iam::462518063128user/awslab1; accountId=xx;accessKeyId=xxxx; userName=awslab1; sessionContext=}eventTime                    : 5/27/2022 5:04:12 PMeventSource                  : ec2.amazonaws.comeventName                    : TerminateInstancesawsRegion                    : us-west-1sourceIPAddress              : AWS InternaluserAgent                    : AWS InternalrequestParameters            : @{instancesSet=}responseElements             : @{requestId=dde64a51-2fd6-40ef-b9d6-06fde8a2abd9; instancesSet=}requestID                    : dde64a51-2fd6-40ef-b9d6-06fde8a2abd9eventID                      : dfc1fa38-c5db-401d-9ac9-11cd5ab41dd8readOnly                     : FalseeventType                    : AwsApiCallmanagementEvent              : TruerecipientAccountId           : 462518061234eventCategory                : ManagementsessionCredentialFromConsole : trueBut the requestParameters            : @{instancesSet=} is missing instance id and other valuesany idea?FollowComment"

Answer: "0When you describe the object, you don't see the value but the instance ID exists under the requestParameters. Please see below for how to describe the instance IDs.$results = Find-CTEvent -StartTime (Get-Date).AddMinutes(-30) | ? {$_.EventName -eq "TerminateInstances"}($results.CloudTrailEvent |convertfrom-json).requestParameters.instancesSet.itemsCommentShareTaka_Manswered 7 months ago"


Question: Multi-tenant design

"I want to understand how the multi-tenant application are built ? Assume a product called "company.com" exist. The client want to use this company.com and create a sub domain i.e client1.company.com.In AWS how this can be achieved ?  Does Route 53 provides any way to create the sub-domain using runtime APIs.After the creation of sub domain is successful, how to identify the client1 i.e tenant from url in AWS ?Does Route 53 provides some way to identify the tenant ?Also How does cognito works in multiple tenant application? For every tenant do we need to create cognito identity pool ? What is the recommendation for this ?Can some one explain this ?FollowComment"

Answer: "2Hi, @Navin GVThe most important part to consider when designing an infrastructure for a multi-tenant application is the tenant isolation strategy.Check the following.https://aws.amazon.com/partners/programs/saas-factory/tenant-isolation/?nc1=h_lsTenant separation mainly includes silos, pools, and bridges.Let me give you an example.It's simple if you use silo strategies and subdomains. It can be separated by simply changing the destination of name resolution for each subdomain to a different infrastructure.In the case of pools and bridges, it is possible to change the behavior in the host header in the same infrastructure, but in that case it is necessary to think about how to separate tenants in the shared infrastructure.If you use Cognito, you can separate the ID pool separately, or you can include the tenant ID in the custom attribute and separate it.The answer is that AWS defines best practices for various multi-tenant applications, but which one you apply depends on your strategy.Check out the SaaS best practices below for more information on multi-tenant strategies.https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/general-design-principles.htmlCommentShareEXPERTiwasaanswered a year agoEXPERTToni_Sreviewed a year agoNavin GV a year agoSuppose if i decided to use pool model with tenant id as custom attribute in cognito, how do I restrict the cross tenant data access in dynamo ? Do I need to handle the tenant_id of data with tenant_id of the request in the application ? Or Do we have some mechanism to get STS token (temp credentials) with using tenant id ?Share"


Question: CodeCatalyst and additional extensions

"I suspect I know the answer, but I just wanted to confirm that the plan is for CodeCatalyst to have extensions to connect with BitBucket, GitLab etc. and not just GitHub?Are you also planning extensions similar to the Jira integration but for say GitHub issues, trello etc - or opening up a mechanism for developers to add extensions?FollowComment"

Answer: "1As I mentioned, probably the big one at the moment would be github issues - the enterprise clients I work with are predominantly using Jira, but a significant amount use github issues, so would be good to be able to integrate into the issue management through that route.CommentShareSimon Hanmeranswered 4 months agoJonJ-AWS MODERATOR4 months agoThanks for the feedback!Share0Accepted AnswerHi Simon! We cannot comment about specific items on the roadmap, but can say that we want to make it easy to use CodeCatalyst with other tools that developers are using.Are there specific integrations that you are interested in?CommentShareMODERATORJonJ-AWSanswered 4 months ago"


Question: DLM cloud watch matrics

"Hi All,I have setup the DLM for daily & weekly backup of EBS volumes. I have setup the cloud matrics to get the total count of the Snapshot completed & failed.The sum of data is based on the period filter not the duration i.e.,If the period =1day  and Duration : 1 months, it shows sum= 1If the period=7days and Duration : 1 Months, it shows sum = 7If the period=30days and Duration : 1 Months, it shows sum = 30I want that when I select the duration, it should show total count of completed & failed snapshot.Please suggest.Regards,Nikhil ShahFollowComment"


Question: Workmail email not being delivered.

"How can I get someone from AWS to solve the issue below?I´ve checked and I cannot do it, it has to be someone responsible for the ip(domain owners shouldn´t contact them).-----Mensagem original-----De: mailer-daemon@amazonses.com [mailto:mailer-daemon@amazonses.com]Enviada em: terça-feira, 28 de fevereiro de 2023 10:32Para: RH - Ativa ContabilidadeAssunto: Undelivered Mail Returned to SenderAn error occurred while trying to deliver the mail to the following recipients:2023-02-28 13:31:46 : An error occurred while trying to deliver the mail to the following recipients:Technical report:Reporting-MTA: dns; a11-132.smtp-out.amazonses.comAction: failedFinal-Recipient: rfc822; larissa@planningservice.com.brDiagnostic-Code: smtp; 554 5.7.1 Service unavailable; Client host [54.240.11.132] blocked using zen.spamhaus.org; https://www.spamhaus.org/sbl/query/SBL588989Status: 5.7.1Action: failedFinal-Recipient: rfc822; cintia@planningservice.com.brDiagnostic-Code: smtp; 554 5.7.1 Service unavailable; Client host [54.240.11.132] blocked using zen.spamhaus.org; https://www.spamhaus.org/sbl/query/SBL588989Status: 5.7.1FollowComment"


Question: "Codebuild always reporting build status on Github webhook events, can't turn off"

"I have a Codebuild job wired up to a Github Enterprise repository using webhook triggers on PRs. I need this codebuild job to run, but NOT report a build status on the PR back to Github (I have other lambdas that handle PR status and checks reporting).In the source configuration for the Codebuild job, I have "Report build statuses to source provider" disabled, and the Status Context and URL are blank.However, on every PR event after the "Github Hookshot" triggers a build, Codebuild still automatically reports back to Github the commit status (pending/success/failure)Why is it automatically doing this and how can I disable this build status reporting?FollowComment"

Answer: "0Hi,I'm Zainub from AWS Premium Support and it will be my pleasure assisting you on this question.I understand that:CodeBuild has been integrated with GitHub Enterprise using webhooks.It is required that the build triggers on pull requests but not report it's status back to the GitHub repository as other Lambda functions handle the PR status and checks reporting.The CodeBuild project source configuration has been updated to disable "Report build statuses to source provider" and the status and target URLs have been left blank.Despite the source configuration updates, the build status is still reported back to GitHub.Guidance on why it is reporting and how to disable it is required.Kindly confirm if the summary and understanding of the issue is correct.According to the CodeBuild documentation:The status of a build triggered by a web hook is always reported to your source provider.Therefore, a web hook-triggered build ignoring the reportBuildStatus property is by design.As I do not have insight into the custom automation (Lambda functions etc.) that has been implemented on your end, it is difficult to suggest a work-around or advice on the best way forward. In this regard, I recommend reaching out to a Solutions Architect who would be better equipped to provide the best advice on the way forward. You can get in touch with a Solutions Architect via the AWS sales page.I hope you find the above informative and have a great day further.Linked Resources:ProjectSource: Contents: reportBuildStatusAWS SalesCommentShareSUPPORT ENGINEERZainub_Aanswered 2 months ago"


Question: Cannot update "Pending Validation"Status to Issue

I have private hosted zone in Route 53. When I go to certificate manager and click "Create record in Route 53 button" I dont see the certificate and "Create Record" button is disable.AWS Certificate Manager > Certificates > Create DNS records in Amazon Route 53I see the certificate after I remove the filter Is domain in Route 53?: YesI can see my domain in Route 53. How can I resolve the issue?FollowComment

Answer: "0The requirement to create a certificate by ACM - you need to create public hosted zone, not private hosted zone. By your question, it appears you are trying to create "Request a public certificate."When someone request an ACM public certificate using DNS validation, ACM provides a CNAME record for each FQDN, and another name (if you have one) that you must add to your DNS configuration to validate your ownership of the domain. Please note, because anyone can create a private DNS zone and put records on it under any domain name, being able to make a change in a private DNS zone doesn't prove public ownership of the domain.Requesting a private certificate requires the creation of a private certificate authority (CA). To create a private CA, visit AWS Private Certificate AuthorityYou can request two types of certificate from ACM:Request a public certificate, andRequest a private certificatePlease visit the following links:https://docs.aws.amazon.com/acm/latest/userguide/dns-validation.htmlhttps://aws.amazon.com/private-ca/getting-started/CommentSharerePost-User-7136343answered 2 months ago"


Question: Bulk deletion of non-current file versions

Is it possible to have a bulk deletion of non-current versions instead deleting then one by one with specifying the version-id?FollowComment

Answer: 0We could achieve this with S3 Lifecycle.Please refer to this blog post for permanently deleting noncurrent versions of objects: https://aws.amazon.com/blogs/storage/reduce-storage-costs-with-fewer-noncurrent-versions-using-amazon-s3-lifecycle/CommentShareweidianswered 8 months agorePost-User-2128408 8 months agoI wonder if this can be done with aws cli/sdk....Share


Question: Intelligent Data platfom equivalent

"Hello! I am looking for an equivalent to this solution that MIcrosoft has flaunted called IDP intelligent data platform, it is governance + operations + analytics in one. they flaunt synapse  with aml  and purview and other stuff they did with these to make it more integrated   . I know we have RDS and Sagemaker -- but how about purview?  how are we tgo make it more cohesive?FollowComment"


Question: Boto3 'Chime' object has no attribute 'create_app_instance'

"Using the following python code in a lambda function:import boto3chime_client = boto3.client('chime', region_name='us-east-1')def lambda_handler(event, context):response = chime_client.create_app_instance(      Name='my_app_instance'  )  Expected result:Get a new app_instance object with associated arn.Current result:"errorMessage": "'Chime' object has no attribute 'create_app_instance'", "errorType": "AttributeError"Boto3 documentation includes this attribute for the chime client, but includes the following: Only Messaging SDK customers use this API. Who are "Messaging SDK customers" and how do you become one?FollowComment"

Answer: 0Solved it by installing the latest boto3 as a dependency via pip install rather than using the 'default' included version of boto3 in the Lambda service. It seems this preinstalled version does not get updated often enough to include the newest available properties.CommentShareegalvananswered 2 years ago


Question: How to create secondary private ip with AWS CDK?

"I cannot figure out how I can attach a secondary private IP to my primary ENI (eth0) with AWS CDK.This is how I create my instance:   const myVpc = new ec2.Vpc(this, "my-vpc", {      maxAzs: 3,    });   const mySecurityGroup = ...  const imageDebian10 = ...        new ec2.Instance(this, "my-instance", {      instanceName: "my-instance",      vpc: myVpc,      vpcSubnets: {        subnetType: ec2.SubnetType.PUBLIC,      },      availabilityZone: this.availabilityZones[0],      securityGroup: mySecurityGroup,      machineImage: imageDebian10,      instanceType: ec2.InstanceType.of(        ec2.InstanceClass.T2,        ec2.InstanceSize.MICRO      ),    });FollowComment"

Answer: "0HelloAs far as i know that you cannot modify the Primary interface, Here are my recommendations for you, I have done something like this with Cloud-formation both any stack could be the same.Procedure:Create Two interaces or one interface eni(0) [I have created manually because there is custom requirement for us ]You can call One is primary and Second Interface as Secondary ENIYou can refer those interfaces in your templatePrimary network interfaceEach instance has a default network interface, called the primary network interface. You cannot detach a primary network interface from an instance. You can create and attach additional network interfaces. The maximum number of network interfaces that you can use varies by instance type. For more information, see IP addresses per network interface per instance type.If you still have any questions please let me knowThank YouGKCommentShareGKanswered a year ago"


Question: AWS glue NoClassDefFoundError on job.init()

"Trying to debug AWS Glue scripts locally using Glue ETL library. I have installed aws-glue-libs and spark-3.1.1-amzn-0-bin-3.2.1-amzn-3.tgz.When I run job.init(), I get the following error trace:py4j.protocol.Py4JJavaError: An error occurred while calling z:com.amazonaws.services.glue.util.Job.init. : java.lang.NoClassDefFoundError: com/typesafe/config/ConfigMergeable at com.amazonaws.services.glue.util.Job$.init(Job.scala:93) at com.amazonaws.services.glue.util.Job.init(Job.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: com.typesafe.config.ConfigMergeable at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 13 moreFollowComment"


Question: "Action Required: Irregular activity in your AWS account."  I'm unable to do anything.

"Two days ago I got an email saying my AWS account may be compromised, so I have to take immediate action.  Sounded quite urgent, so I changed my password, deleted a couple policies, but didn't find any unauthorized account access- I think the system just flagged me because I recently started a new project so I've been creating a lot of new resources.Since my account was locked, I'm unable to access Lambda at all, the dashboard just reads "You do not have sufficient permission. Access denied."  I can't do anything until this "urgent matter" is resolved.I responded to the support ticket that I've done everything that was asked of me, been inquiring every day since then, but have received no help at all apart from being told that they're investigating.  They have told me there's no time frame for resolving this, since the service team might not be available to me.As far as I can tell, I'm just completely blocked from using AWS Lambda due to being incorrectly flagged, until someone gets around to fixing it.  Is there anything I can do?FollowComment"

Answer: "0Try this here to see if it helpCommentSharejthornes-awsanswered 10 months ago0Hello there,I understand you are facing issues with accessing resources on your account.Usually when this warning is triggered, the specific IAM users or even the root users are denied access to the account to prevent any further damage or abrupt charges to the account prior to confirming the account is secure.In your case, I understand that this might be a false alarm since you have investigated and found no compromise activity.I would recommend you do the following:Raise a chat contact on the same support caseRequest the connected engineer to expedite the review of your account's security as it is impacting your projectFurther, the engineer will reach out to the appropriate team and have this expedited.The dedicated team  experiences high case volumes and it is the reason why they are not able to provide you a confirmed SLA. However, you can reach out to the support engineers and ask it to expedited. Please note that there is no fixed SLA for expedited requests either. The engineers will try their best to have it updated within 24 to  48 hours.Hope I have been helpful.Take care.CommentShareSUPPORT ENGINEERHarshavardhan_Ganswered 10 months ago"


Question: How to connect AWS IAM endpoint without going through internet ?

"Hello,I want to get IAM User list on my private EC2 instance without opening the 443 port for the whole world and one more thing it is it possible to get user list without internet access because I am using VPC endpoint for accessing the other services but IAM doesn't support the VPC endpoint, So, How should I get things works.Thanks.FollowComment"

Answer: "1AWS STS does have a VPC Interface Endpoint so if you are assuming roles you can do this without an Internet Gateway.CommentShareEXPERTkentradanswered a year ago0Today (early 2022), IAM doesn't have a VPC endpoint. For reference the services that do are listed on this page which is updated as we add more services.Otherwise, the only way to do this (as you have already stated) is to give your EC2 instance access to the Internet via NAT Gateway or some other method.Please get in touch with your local AWS account team - they can take your feedback on this particular request and add it as a feature request.CommentShareEXPERTBrettski-AWSanswered a year ago"


Question: How to manage WAF with cloudfront signed cookie

"Hi there.There is my architecture:WAF [Check for some cookies exists, else, redirect to login] -->Cloudfront with behaviors with signed cookie for trust key groups and cache static resources -->API GW -->LB -->EKSEverything worked find until we integrated the signed cookie with key group trust and then, it's seems that the signed cookie key validation happens before the WAF staff, so we get 403 at any case, before the login itself.Any idea how to manage that the WAF will occurs before the CF signed cookie validation?Thanks.FollowComment"


Question: partitions disappearing

"I have several partitioned tables and have noticed that after a period of time  queries return no data and I need to rerun MSCK REPAIR TABLE to make them visible again.The process I have in place runs ADD PARTITIONS when new data is added to S3.  This works for a period of time, but eventually I see queries return 0 rows and am forced to run the repair.The data partition key "PARTITIONED BY (v_id string, x_id string, year int, month int, day int, hour int)".  Some tables are CSV and others are AVRO, I've seen this on both.Any suggestions?  I could automate MSCK REPAIR, but that seems like the wrong solution.Edited by: lettermuckoo on Dec 18, 2019 6:13 AMFollowComment"

Answer: "0I found the issue.There was a job that was recreating the tables during deploys.  MSCK REPAIR TABLE was being run after recreate, but it was not fully qualifying the database.tablename, so it was not discovering the existing partitions.Edited by: lettermuckoo on Dec 18, 2019 1:56 PMCommentSharelettermuckooanswered 3 years ago"


Question: 3cx storage increase

"can't find where to increase quote for installed 3cx from marketplace , only i find a VPC and even i don't know if it is related to installed 3cx, i don't find any related thing.why asking ? our recording quote on 3cx dashboard is almost 100 % and we were advised to increase the storage on AWSi can't find anything that helps me to increase storageany help ?FollowComment"

Answer: "0Hi!I found the following offer in the marketplace: 3CX Phone System.This is an AMI (Amazon Machine Image), which was likely used to boots up an EC2-instance in your default VPC. Navigate the AWS Console to EC2 and there should be your instance. To add additional storage, you may want to use EBS (Elastic Block Storage) to increase the capacity of your instance, or (depending on the nature of your recordings) you may want to copy the files to S3 or EFS (Elastic File Service).CommentShareMatthaeusPanswered 8 months ago"


Question: Can Tags for instances be included in Fleet Manager list view?

"In the SSM Fleet Manger, a list of Native and Hybrid Cloud Managed instances are displayed. Is it possible to include, in the columns on the list, and/or on the report that can be downloaded using the Download Report button on this page, any tags and their values, similar as to what can be displayed in the EC2 Instances list view?FollowComment"

Answer: "0Hi,Currently this is not possible. As you are probably aware, you can view Tag data in the Node details page by selecting the instance ID from the Fleet Manager list view, but this data cannot be visualized in the list view itself or on the report generated by the Download Report button. We are aware of customer interest regarding this functionality, and we take this into account when making improvements to our services.I have passed your feedback to the team responsible for Fleet Manager, but I can't provide any timeline or ETA on their behalf. In the meantime, you can keep track of feature updates for Systems Manager in the following places:The AWS Blog: https://aws.amazon.com/blogs/aws/category/management-tools/aws-systems-manager/In our What's New page, by filtering for Systems Manager under the 'Management & Governance' category: https://aws.amazon.com/new/CommentShareSUPPORT ENGINEERSean_Sanswered a year agoEXPERTJames_Sreviewed a year ago"


Question: What's the meaning of an AWS Glue Database name in italic?

"I have a Glue job that writes to a table. If I do this and pointing to a “non-italic” database, the table is shown afters in the table view. If I let the Glue Job write into the “italic” database, then the table is successfully written, too, but not listed in the table view.Furthermore, I can see both the newly created tables in Athena in the table list and I can do querying over both. If I create a new Glue Job, I also can see both table in the source data catalog table dropdown.I.e., I can see the table in the “italic” database everywhere but not in the table view.Is the “italic” database some kind of shared database or so? What's the meaning of it?FollowComment"

Answer: "1Accepted AnswerIf you are using LakeFormation, italics are a way to distinguish databases from resource links. To learn more about it, please refer to How resource links work in Lake FormationCommentShareSUPPORT ENGINEERManu_Ganswered a year agoEXPERTFabrizio@AWSreviewed a year ago"


Question: Lambda function for moving files from S3 bucket to FSx

I'm wondering is this even possible and how it should be configured? Do you have some ideas?I know how i will trigger that lambda function but I'm not sure for the rest.FollowComment

Answer: "0You can read this blog post about integrating a Lambda Function with an FSx storage. This blog post also offers you a .NET code example that demonstrates how to proceed.The described solution allows you to run file and directory operations such as listing, storing, retrieving, and deleting files and folders on file shares hosted on Amazon FSx for Windows File Server and Amazon FSx for NetApp ONTAP.CommentShareDmitry Balabanovanswered a year ago"


Question: EMR Serverless submitting a job using command-runner.jar?

"Hi! I'm excited to see EMR Serverless. I went through the quick start and was able to run the word count, pretty cool.Now I would like to submit a spark job using command-runner.jar like I do on a regular EMR Cluster but when submitting a job EMR Studio asks for the script S3 URI. It is not a script it is a JAR and in a regular EMR cluster I use command-runner.jar to call a spark-submit with another jar.I looked for documentation but this all seems pretty new and couldn't find any. How can I submit a job using command-runner.jar? Or do I submit the job directly?Any tips would be highly appreciated!FollowComment"

Answer: "0In EMR Serverless, you can submit a Spark job using the aws emr-containers submit-job command. This command submits a job to run in a container on an EMR-managed infrastructure. Here is an example command:aws emr-containers submit-job \--cluster arn:aws:eks:us-west-2:012345678910:cluster/my-emr-cluster \--job-name my-job \--execution-role-arn arn:aws:iam::012345678910:role/emr-containers \--region us-west-2 \--command '["/usr/bin/spark-submit", "--deploy-mode", "client", "--class", "org.apache.spark.examples.SparkPi", "local:///usr/lib/spark/examples/jars/spark-examples.jar", "10"]'In the --command option, you can specify the command you want to run inside the container. In this example, we're using spark-submit to run the SparkPi example with 10 partitions.You can also pass any other options or arguments that you would normally pass to spark-submit. For example, you can specify the path to your JAR file as an argument:aws emr-containers submit-job \--cluster arn:aws:eks:us-west-2:012345678910:cluster/my-emr-cluster \--job-name my-job \--execution-role-arn arn:aws:iam::012345678910:role/emr-containers \--region us-west-2 \--command '["/usr/bin/spark-submit", "--deploy-mode", "client", "--class", "com.example.MyJob", "s3://my-bucket/my-job.jar", "arg1", "arg2"]'Note that the JAR file needs to be located in an S3 bucket that is accessible from your EMR Serverless cluster.CommentSharehashanswered a month ago"


Question: How to configure  EC2 Instance on AWS Free tier to avoid being charged for running single instance on t2micro?

I have a sample nodeJS application running on ElasticBeanStalk on a single instance without the load balancer. I read through the Free Tier documents to make sure i am following the guidance to stay in free tier but i still keep getting these small charges despite the changes i have made.I have single instance running t2micro . I have attached the breakdown of billing below. For some reason i am charged per/hr for running t2micro in a single instance as well .Is there anything i am doing wrong? i know the charged amount is small but still it shud fall under free tier. Any help will be appreciated.I have attached the screenshot of EC2 dashboard and Billing details belowFollowComment

Answer: "1Accepted AnswerAre you sure you're in Free Tier - that your account is less than 12 months old since sign-up?Are you sure you don't have resources in another region?You can track your Free Tier usage as described here - https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/tracking-free-tier-usage.html.CommentShareEXPERTskinsmananswered 2 months agoprawegko 2 months agoThanks . Looks like i might have accidentally signed up AWS long before i actually started using it. I have been Amazon prime member a long time , i might have signed it up back then thinking something else. The oldest bill showing up in my billing dashboard dates back to 2017. so My guess is that was when my account was signed up? Is there any other way to track this ?Shareskinsman EXPERT2 months agoYes the oldest bill in the "Bills" page is a good way to tell when the account was first active.If you get around to it please Accept my answer to make it easy for others to find, thanks.Share"


Question: Why was my EC2 Instance deleted from my account?

Just wondering why my free tier EC2 instance was deleted from my account? I turned off the instance the last month or so because I was not able to work on it and did not want to keep it running. Can the instance be restored? I do not have the instance id as I have no information in my AWS management console to reflect the instance id. Any help with this is appreciated. Thank you.FollowComment

Answer: "0Hello,Which region was this instance in?I can see that the instance i-0d2196316ca6ca1d9 was stopped last month in the us-west-1 region but is currently in a running state.CommentShareAWS-User-8424448answered 3 years ago0I was able to verify the instance was still there. My mistake, I just was not looking at the right screen. Sorry, for the confusion.CommentShareAWS-User-9191320answered 3 years ago0I was able to verify the instance was still there. My mistake, I just was not looking at the right screen. Sorry, for the confusion.CommentShareAWS-User-9191320answered 3 years ago"


Question: DynamoDB False save

"I just experienced disturbing behavior and I wonder if anyone else saw anything similar and how to handle it?I'm updating a row in DynamoDb from node.js lambda and logging ALL_NEW result of the update operation.In my log, I can see the data I was trying to save but in the table itself, that data is missing.  On the next run, everything went OK.This is very problematic from a data consistency perspective.Anyone has an idea where to look and if there is some setting or way to handle this kind of behavior?FollowCommentjzhunter a year agoAre you doing a strongly consistent or eventually consistent read to check if the data is there?Share"

Answer: "0Hello,What you're seeing is the effect of an "eventually consistent read", please take a look at this doc page that describes the differences between the "eventually consistent" and "strongly consistent" reads: ttps://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html The default read mode is eventually consistent, and you have to request the strongly consistent read if you need it, using the "ConsistentRead" parameter.RegardsCommentShareAWS-ADolganovanswered a year agosetheliot a year agoEventual consistency will become consistent within 1 second or less.  In OPs case he had time check his logs, so it seems more than a secondShare"


Question: To stream indexes into elasticsearch from dynamodb table on insertion

I am a novice into all thisCan anyone explain or point out to a reference/link which could explain how to stream indexes into elasticsearch from dynamodb table on insertion i.e. on inserting an item in dynamodb table the same record is to be updated in the elasticsearch too by trigger.There are examples available in Python/Nodejs but cant find one in JavaThanksFollowComment

Answer: 0No responseCommentShareawsdineshanswered 2 years ago


Question: Anybody ever successfully ran multi-node gradient boosting on Amazon SageMaker?

"Anybody ever successfully ran multi-node gradient boosting on Amazon SageMaker?I'm looking for a SageMaker-compatible multi-node training solution for either Catboost, LightGBM or XGBoost. Knowing if it's ever been done would be nice, having a public demo link would be even better :)FollowComment"

Answer: 0Accepted AnswerWe do have an example of distributed training of XGBoost in the sagemaker-examples repo. You can find it here: https://github.com/aws/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/xgboost_abalone/xgboost_abalone_dist_script_mode.ipynbCommentShareAWS-User-4760880answered 2 years ago


Question: User-User recommendation recipe?

"Hello there - testing our Amazon Personalize for a business use-case where we want to suggest other Users with similar Items/Events.  Based on research it seems like a user-user recommendation system isn't possible with Amazon Personalize, but wanted to double check to make sure I'm not overlooking a creative solution for this.Thank you!FollowComment"

Answer: "0Accepted AnswerHello,As of today there is not a Recipe for User-User recommendations.  From a creative perspective, you could potentially upload your Users dataset as both Users and Items datasets assuming you have an Interactions dataset that shows Users interacting with Users.While not supported today, there may be a use case/recipe in the future for User-User based interactions if there is enough demand from customers like you for a feature like this.Here are the available recipes today: https://docs.aws.amazon.com/personalize/latest/dg/working-with-predefined-recipes.htmlCommentShareEXPERTChris_Ganswered 9 months ago0Hi,  it sounds like you might be interested in using the Amazon Personalize user segmentation recipes. These recipes generate segments of users based on item input data.A possible application couId be: if you want to recommend other users to user A, your input item, could be the last item user A interacted with, or the last category of items user A interacted with. Amazon Personalize will recommend users that have affinity for those items or item attributes. As long as you have enough interaction data and item metadata,  you can generate recommendations. Check the personalize cheat sheet.Alternatively, the Amazon Personalize custom dataset groups are use-case agnostic: you can use any kind of "items" to recommend to your users, including other users! You can create your items dataset with user_ids and metadata. However, in this case, the interaction data would be users interacting with other users, not with items.As long as you have enough interaction data (and ideally some user and "item" metadata) you can generate recommendations. Check the personalize cheat sheet.For both recommendation types you can also apply business rules to further improve your recommendations through filters . For example you can recommend only those users that have the same interest or type of the original user, or only users that have a similar company size.CommentShareAnna_Ganswered 9 months ago"


Question: Duplicated MySQL RDS 5.7 error.log entries in CloudWatch

"I am using MySQL RDS 5.7.33 configured to sent mysql error logs to CloudWatch.While investigating deadlocks frequency in CloudWatch, I encountered duplicated entries like2021-12-25T23:15:10.131859Z 39261418 [Note] InnoDB: Transactions deadlock detected, dumping detailed information.2021-12-25T23:15:10.131859Z 39261418 [Note] InnoDB: Transactions deadlock detected, dumping detailed information.that happens at about 10% frequency. The original error.log files for that day have only a single entry:Viewing Log: error/mysql-error-running.log.2021-12-26.0 (2.7 kB)2021-12-25T23:15:10.131859Z 39261418 [Note] InnoDB: Transactions deadlock detected, dumping detailed information.How can I avoid duplicated entries in the CloudWatch mysql error.log?FollowComment"

Answer: "0Hi! Thanks for reaching out.With the information available here, I'd be interested to learn a few more details:How are the logs being retrieved from CloudWatch? Directly in the log stream within the console, or by another method such as a GetLogEvents, FilterLogEvents, or possibly a Logs Insights query?MySQL error log files are rotated every hour with the hour each file is generated appended to the file name in UTC. In this case, I note that the log file is appended with the date 2021-12-26.0 while the log entry timestamp is denoted as 2021-12-25T23:15:10.131859Z. Was this log entry also written in the error/mysql-error.log.2021-12-25.23 file as well, leading to the duplicate entries being seen in CloudWatch?These details may help determine if the same log entry was written twice by RDS to two files between rotations, or if this may be a quirk with retrieving log entries from CloudWatch Logs.Overall, it may be best to open a case with the RDS team via the case management system in order to work directly with an engineer to determine if error logs are being written as expected to the RDS error log files.Overview of MySQL database logs  - MySQL error logs - https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.MySQL.LogFileSize.html#USER_LogAccess.MySQL.ErrorlogCommentShareSUPPORT ENGINEERJustin_Banswered a year ago0Hi Justin,How are the logs being retrieved from CloudWatch? Directly in the log stream within the console,...I am looking at the logs directly in the log stream within the console....Was this log entry also written in the error/mysql-error.log.2021-12-25.23 file as well, leading to the duplicate entries being seen in CloudWatch?No duplication observed in the mysql-error.log files. There were only two logs for 2021-12-26error/mysql-error-running.log.2021-12-26.21Sun Dec 26 2021 15:00:00 GMT-0500 177 Berror/mysql-error-running.log.2021-12-26.0Sat Dec 25 2021 18:20:00 GMT-05002.7 kBand five logs for 2021-12-25error/mysql-error-running.log.2021-12-25.20Sat Dec 25 2021 14:25:00 GMT-0500177 Berror/mysql-error-running.log.2021-12-25.13Sat Dec 25 2021 07:30:00 GMT-0500528 Berror/mysql-error-running.log.2021-12-25.7Sat Dec 25 2021 01:35:00 GMT-0500494 Berror/mysql-error-running.log.2021-12-25.6Sat Dec 25 2021 00:55:00 GMT-0500516 Berror/mysql-error-running.log.2021-12-25.5Fri Dec 24 2021 23:30:00 GMT-050013.3 kBThese details may help determine if the same log entry was written twice by RDS to two files between rotations, or if this may be a quirk with retrieving log entries from CloudWatch Logs.I found one other case of duplication of the deadlock error.log entries in the CloudWatch Logs. Filtering for deadlock in the log stream within the console for the past two months shows repeated timestamps that happen on 2021-11-10:2021-11-10T23:15:27.926126Z 200840 [Note] InnoDB: Transactions deadlock detected, dumping detailed information.2021-11-10T23:15:27.926462Z 200906 [Note] InnoDB: Transactions deadlock detected, dumping detailed information.repeated subsequently as2021-11-10T23:15:27.926126Z 200840 [Note] InnoDB: Transactions deadlock detected, dumping detailed information.2021-11-10T23:15:27.926462Z 200906 [Note] InnoDB: Transactions deadlock detected, dumping detailed information.As on 2021-12-25, these duplicates happen at the last UTC hour of 2021-11-10.Other filtered out deadlock records were not duplicated.CommentShareSasha Vanswered a year ago"


Question: EKS: ARM64 nodes fail to become ready due to CNI error

"Hi folks -k8s version: 1.21Platform version: eks.4I'm trying to get an autoscaling nodegroup using a1.metal instances working.   Nodes that are spawned never become ready due to the following error:container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitializedThe aws-node pod on the node fails with the following error:Readiness probe failed: {"level":"info","ts":"2021-12-29T04:56:34.384Z","caller":"/usr/local/go/src/runtime/proc.go:225","msg":"timeout: failed to connect service \":50051\" within 5s"}The kube-proxy pod on the node fails to start with the following log:standard_init_linux.go:228: exec user process caused: exec format errorWhich leads me to believe that at least the kube-proxy pod is likely pulling an x86 image instead of arm64.I appreciate any assistance; please let me know which other information I can provide.thanks,/-WillFollowCommentsaiteja313 a year ago50051 port represents ipamd agent (part of aws-node daemonset). Could you post the ipamd logs from following location on ARM workernode.cat /var/log/aws-routed-eni/ipamd.logShareMichael_F EXPERTa year agoHow did you set up your worker nodes? Is it a Managed Nodegroup, or are you using your own unmanaged worker nodes?Are these new nodes attached to an older cluster, or was this a freshly-created cluster?How did you install the VPC CNI controller? Did you let EKS do it, or did you update/install it yourself?Can you please post the output of kubectl -n kube-system get daemonset aws-node -o yaml ?Share"

Answer: "0Hi Will,Thanks for your post here.As per the reference  Amazon EKS optimized Arm Amazon Linux AMIs [1]If your cluster was deployed before August 17, 2020, you must do a one-time upgrade of critical cluster add-on manifests. This is so that Kubernetes can pull the correct image for each hardware architecture in use in your cluster. For more information about updating cluster add-ons, see To update the Kubernetes version for your Amazon EKS cluster . If you deployed your cluster on or after August 17, 2020, then your coredns, kube-proxy, and Amazon VPC CNI Plugin for Kubernetes add-ons are already multi-architecture capable.to validate that, please share the output of the below commandkubectl get ds kube-proxy -n kube-system -o yamland you can check for below parameterskey: "beta.kubernetes.io/arch"operator: Invalues:- amd64- arm64the same is also as referred here in the reference document [2], Updating the kube-proxy self-managed add-on.(Optional) If you're using x86 and Arm nodes in the same cluster and your cluster was deployed before August 17, 2020. Then, edit your kube-proxy manifest to include a node selector for multiple hardware architectures with the following command. This is a one-time operation. After you've added the selector to your manifest, you don't need to add it each time you update. If your cluster was deployed on or after August 17, 2020, then kube-proxy is already multi-architecture capable.kubectl edit -n kube-system daemonset/kube-proxyAdd the following node selector to the file in the editor and then save the file. This enables Kubernetes to pull the correct hardware image based on the node's hardware architecture.key: "beta.kubernetes.io/arch"operator: Invalues:- amd64- arm64after you reviewing the above information, if you find that there is need to to update the kube-proxy, please follow the steps from reference document[2].If you do not see any issue with kube-proxy configuration, you can review reference document [3], where we discussed about other possible reasons and troubleshooting steps for node not joining the cluster.based on the nature of this issue, if you need more assistance and we need to perform deep dive, you can create a case with premium support where we can get the logs reviewed and suggest specific troubleshooting steps if required.References:https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#arm-amihttps://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html#updating-kube-proxy-add-onhttps://aws.amazon.com/premiumsupport/knowledge-center/eks-worker-nodes-cluster/CommentShareSUPPORT ENGINEERKiran_Kanswered a year ago"


Question: How can I revoke tokens created through Cognito oauth/token url?

"We have an application implemented on api-gateway and lambda, the authentication is carried by tokens generated on Cognito, Cognito has the Client credentials OAuth Flow with custom scopes. We are using the oauth/token url to generate access tokens, we tried to create refresh tokens, but the oauth/authorize isn't working, because the Client credential flow restrict the Authorization code grant.On the middle of the process we need these tokens to be revoked, but the oauth/revoke needs the refresh token that we can't generate because of the chosen flow, How can we can revoke those tokens?FollowComment"

Answer: "1Hi,Currently it is not possible to revoke an access token that is issued using client-credentials flow.In a token-based authentication system like Cognito, tokens are considered valid as long as they have valid signature and they haven't expired. When these tokens are passed for authorization to back-end (like API Gateway), tokens are validated remotely by verifying its signature and validity, this remote verification doesn't involve any calls to the issuer of the token (cognito). Revoking a token on the authentication server will not invalidate the already issued token and back-end systems that verify this token remotely will continue to accept it.In your case, if you want to deny access to a token before it is expired, you will need to maintain a deny-list table in DynamoDB for example and if you want to deny access to the token you store its unique identifier (jti claim) in this table and check this table during authorization to make sure that token hasn't been denied access.CommentShareEXPERTMahmoud Matoukanswered a year ago"