Question: Lambda to Dynamo Write: Intermittently missing batch data

"Hi,I am using AWS Lamba to read data from web and store it in dynamoDB at periodic intervals during the day. Post which, I take the daily dump from DynamoDB and get a CSV file is S3 for daily reporting.This had been working perfectly fine and I was getting data for all batches during the day. But off late my S3 dump seems to have gaps and I am missing data for different batches. And these missing batches are different for different days.I am not getting any error message - so, just wanted to get some guidance on how to go about debugging this.Below are stats for my dyanmoDB table:Item count: 235,267Table size: 65.6 megabytesAverage item size: 279.03 bytesProvisioned read capacity units: 1Provisioned write capacity units: 1Also below is snapshot of key operational metrics for the table:https://ibb.co/TPhWqBjhttps://ibb.co/jVgRS93Regards,DbeingsFollowComment"

Answer: "1You are being throttled for exceeding your provisioned capacity. It working for a couple of months does not mean it will always work, it will not scale as the more data you add, the more RCU you will consume. You also mentioned you deleted CloudWatch Alarms, you likely have Autoscaling on your table and you deleted the alarms required for triggering scaling actions.In essence, you need to increase the capacity on your table. For free tier, you get 25 RCU free per region. You can divide those 25 units across all tables in a region anyway you like.I would also add try/except block to your code and log any exceptions you may receive so you can better understand what is happening when things go wrong.CommentShareEXPERTLeeroy Hannigananswered a year agodbeing a year agoHi Leeory,Thanks that clears it up. Below is screenshot for my read capacity setting and I have auto scaling enabled. Did deleting CloudWatch alarms mess up the auto scaling? Should I increase minimum read capacity to 4 as the max in RCU graph was 2.81 if I dont want to enable CloudWatch?https://ibb.co/LNxgFwgMore importantly, is there any way I can complete my task with current read capacity. Though I write to table at every 5 mins. during day time my read is just once a day. The read fetches about 6000 records and I am fine with fairly slow performance for this read - as this is just for daily MIS based on one time CSV extract.Regards,DbeingsShare0Based on the screenshot attached, it shows your read-requests are getting throttled. Try increasing RCU more than 1 or on-demand to see how much is the usage capacity now and then provision.CommentShareirshadanswered a year ago0Hi Irshad,Thanks for your response.  I can increase RCU but a bit reluctant as I am new on AWS free tier and have faced some issue with AWS charges when I change defaults. This was working perfectly fine for more than a month. The only change that I did recently was deleting Cloudwatch alarms, which I think should not impact read / write operation to Dynamo.My requirements are fairly basic: I am not doing any reads during the day and its just a write operation by Lambda at different batches. The read operation is just once a day when I pull entire day's data from the Dynamo DB and put it in a S3 bucket. I am not really bothered about the performance for the read as this is just once a day and all I need is a day's data. Below is the python script for the read operation that is being used in Lambda.def getDynamoData():dynamodb = boto3.resource('dynamodb', region_name = 'us-east-2')table = dynamodb.Table('MyTable')mystr = ''df_list = pd.DataFrame(data={'ItemName':[],'Metric'[],'UpdateTime':[],'MettricID':[]})done = Falsestart_key = None row_num = 0page_no = 0mydate = date.today()mydateStr = str(mydate.year) +  "-" + str(mydate.month) +  "-" + str(mydate.day)  while not done:       if start_key:        response = table.query(IndexName="feedDate-index", KeyConditionExpression=Key('feedDate').eq(mydateStr),ExclusiveStartKey=start_key)    else:        response = table.query(IndexName="feedDate-index", KeyConditionExpression=Key('feedDate').eq(mydateStr))            items = response.get('Items', [])        page_no = page_no+1    item_i = 0    if len(items) == 0:        break    while item_i<len(items):        for key in items[item_i]:            df_list.loc[row_num,key] = str(items[item_i][key])        item_i = item_i+ 1        row_num = row_num+1    start_key = response.get('LastEvaluatedKey', None)    done = start_key is NonelastItem = df_list.iloc[dfLen-1]['ItemName']s3 = boto3.client('s3')fileName = 's3://mybucket/MyFile' + mydateStr + '.csv'wr.s3.to_csv(df_list, fileName, index=False)CommentSharedbeinganswered a year ago0Hi,I increased RCU to minimum 5 units but still facing the issue. I observed that issue is not at the time of reading but while writing to the table.  The value are missing form DyanmoDB itself. I had escalated this through a ticket to AWS team and they indicated that they could find errors using internal tools in Lambda function for said timeslots where values were missing.But I am not able to see any thing in Lambda logs. The last log is from Feb 1 and the issue started happening in April.  There appear to be some error indicated by the graphs under lambda metrics but not able to get details on them. Here are screenshots for key lambda metrics and logs for the said function.https://ibb.co/SBmrHY0https://ibb.co/0m0LsqdIs my Dynamo table size could be an issue? I am planning to move the write to a new table to rule this out. Below are stats to my dynamo Table.Item count: 268,759   Table size: 75 megabytes Average item size279.19 bytesI would really appreciate if anyone could please provide some pointers here.Regards,DbeingsCommentSharedbeinganswered a year ago"


Question: How do I login to my 2 ec2 instances from Android mobile as I can only login from Desktop?

"I have 2 ec 2 instances running and am able to login to both from pc desktop but not from Android mobileIt is my understanding that we can have 2 connections through rdp, is these connections to the server or the instances, so my question is my having 2 instances is that my connections used up and I need to pay for rdp licence then?Or should I be able to connect through my mobile as well?If it is possible to connect through mobile, can someone detail the process for doing so?If I need to buy licences, where do I find details to do that and also associated costs, I would need a connection to each instance through mobile, so is that 2 rdp licenses required?any pointers appreciatedMany ThanksGordonFollowComment"

Answer: "0Any given Windows server has a default limit of 2 concurrent inbound RDP connections. This is a Windows default limit, not an AWS limit, and AWS doesn't have any other RDP limits.  No license is needed.  You should be able to connect from Android with an RDP client.  What happens when you try, timeout?Are your EC2 instance's Security Groups open inbound to your Android's IP address?  Are their subnet NACLs open completely, or open inbound to your IP + suitable ephemeral ports outbound for an Android client?Can you Ping your EC2 instances from your Android (after allowing this in your Security Groups & NACLs)?Are your existing desktop PC connections also connecting over the Internet or via DX or site-to-site VPN?To help narrow things down have you tried connecting your desktop PC to the internet via hotspot on your phone?CommentShareEXPERTskinsmananswered 2 months ago0inbound rules source 0.0.0.0/0When I try it times out but not sure I am using the correct login details, I am using same login as is for my computer, but I did have a pem file and it was to create login for desktopI did copy pem onto mobile but it did not make any differenceMaybe I need to create new pem file on mobile?Subnet acls open completely, not sure where to find that, will need tp read up a bit moreAgain I will need to read up on pinging to see how to ping it from the mobileThe only way I tried to connect to the server was from Desktop pc using rdp, I am not sure on all the other stuff, will need to read up on itI never tried connecting pc via hotspot on the phone but I will have a go at thatThanks for your assistance, I definitely have more reading to do in order to fully understandCommentSharegordonmartinanswered 2 months ago"


Question: AccessDeniedException: User: arn:aws:sts::764717618004:assumed-role/cloudwatch_logs_events_putter/

"Hello, I have observed that the Lambda functions are working correctly on my local machine with my NextJS application. However, when I deployed my NextJS application on the server, I encountered an error in the network tab.{"message":"AccessDeniedException: User: arn:aws:sts::764717618004:assumed-role/cloudwatch_logs_events_putter/uZUu4Ca7Slsek0AGRJnQtenJ-5c0c2554740d00c0e47b39b0106000bb2d9b492 is not authorized to perform: cognito-idp:AdminInitiateAuth on resource: arn:aws:cognito-idp:us-east-2:764717618004:userpool/us-east-2_Y93BT0AII because no identity-based policy allows the cognito-idp:AdminInitiateAuth action"}FollowComment"

Answer: "0Hi Shyam,This error comes from the AWS Lambda function from not having the ability to execute the API action cognito-idp:AdminInitiateAuth.You can correct this by adding an going to your lambda function, going to configuration, and selecting permissions. Under Execution Role, click on the role name and then edit the inline policy to allow the action.CommentShareJoe Melfianswered 25 days ago"


Question: Step Function Use Case

"Hello All,We have aurora postgres cluster up and running . We have lot of procedures in it. Is there a way to call procedure from Step function .If we need to use lambda to call procedure , then we have some procedure taking more than 15 mins for completion so in that case how lambda will be able to send back information to step function that procedure in finished ?FollowComment"

Answer: 1An alternative to Lambda would be to run an ECS Task in Fargate.  This can be triggered from Step Functions and can run for longer than 15 minutes.https://docs.aws.amazon.com/step-functions/latest/dg/connect-ecs.htmlCommentShareEXPERTChris_Ganswered 5 months ago0I tried using python shell. Thanks . Not getting option to accept answerCommentShareAWS-User-6494745answered 5 months agoVivek_S 5 months agoYou should be able to upvote an answer with the thumbs up icon.Share0You can also Glue Python Shell job. With 0.0625 DPU the cost of running a Glue Python Shell not too much different than running a AWS Lambda function and a Glue Python shell has no time limitation. So a long running Python based job can be ported to Glue Python Shell based job easily. Step function can also use .sync integration pattern with it. https://docs.amazonaws.cn/en_us/step-functions/latest/dg/connect-glue.htmlCommentShareVivek_Sanswered 5 months ago


Question: How can I create a signed GET curl request using aws cli?

"I'm trying to do something I thought was easy, but my google fu is failing me.I have been provided an API gateway endpoint that I must call a get on the d/l a file onto my EC2 instance.The request has to be signed.I see all kinds of SDK's and examples, but nothing for cli.I don't see aws cli command that will let me call a gateway endpoint (test-invoke) doesn't seem right. Is there one.If not, is there a simple way to use the aws cli to create a signed request that I can send with invoke-webrequest (powershell) to d/l the file.The IAM permissions are in place and the EC2 instance profile does have the invoke permission for the API.FollowComment"

Answer: "0Accepted AnswerAWS CLI does not provide an option to get signed request. Since you're making a HTTPS call to API gateway, using IAM user permissions, you need to use SigV4 to add authentication information to the request.More info - https://docs.aws.amazon.com/general/latest/gr/create-signed-request.htmlThis page explains the steps required to sign the request. I can think of two options - (1) try to generate the signature in power shell by following the steps described on the link above, or (2) Write a function in language you prefer using one of the code sample (links provided on this page) to generate the signature and use powershell to invoke the same and send request using invoke-webrequest.CommentShareVaibhav Janswered 2 months agorePost-User-2885274 2 months agoYea, kinda what I was trying to avoid. :)SharerePost-User-2885274 2 months agoI decided to create a lambda that will d/l the file and put it in s3 so my instance can download it from there using the AWS cli. Thanks for your suggestions.Share"


Question: AWS Greengrass - Erro with public component version

"Hello,I currently have an Ubuntu device running AWS Greengrass Core Device with different components installed. When trying to update the version of the public component aws.greengrass.clientdevices.mqtt.Bridge and other components the deploy fails. Reviewing the log file greengrass.log I discover the next error:Failed to negotiate version with cloud and no local version to fall back to. {componentName=aws.greengrass.Nucleus, versionRequirement={aws.greengrass.LogManager=>=2.1.0 <2.8.0, aws.greengrass.Cli=>=2.5.0 <2.8.0, aws.greengrass.StreamManager=>=2.0.0 <2.6.0, aws.greengrass.clientdevices.IPDetector=>=2.2.0 <2.9.0, aws.greengrass.clientdevices.Auth=>=2.6.0 <2.9.0}}The surprising thing is that it requires non-existing versions, like for the component aws.greengrass.clientdevices.IPDetector (latest version is 2.1.4) or aws.greengrass.clientdevices.Auth (latest version 2.2.3)Thank you very much in advance.All the best.AlbertoFollowComment"

Answer: "2Accepted AnswerHi Alberto,what this message is saying is that your deployment cannot find a version of the Nucleus component which is compatible with the requirements of the components being deployed.  In particular the issue comes from Auth requiring a version >=2.6.0 and StreamManager requiring a version <2.6.0.To fix, review your deployment and in the configuration step ensure you select the latest version for each one of the components included.You can also force a specific version of the Nucleus component by including it in the deployment and specifying the version in the deployment.Cheers,MassimimlianoCommentShareEXPERTMassimilianoAWSanswered 8 months agoalbdom 8 months agoThank you very much for the quick response, it was indeed solved as you mentioned!ShareJonathanKaye 7 months agoI am also having this problem but Auth's latest version is 2.2.3. I cannot find an Auth version >= 2.6.0 in the available public components list, so I reverted my Nucleus to 2.7.0 and then adjusted other component versions. That seemed to work, but I really don't want to use older Nucleus versions. Where is the mythical Auth >= 2.6.0?Share"


Question: Lua and C++ integration

"Guys, I am trying to expose a c++ function to Lua just to be able to use windows clipboard, copy and paste. I am not sure this is the best solution but it is what we are doing right now.We created a GEM and add to the gem a .h and cpp file. This is all we have right now:.h#include "StdAfx.h"#include <ScriptHelpers.h>#include <IScriptSystem.h>class ClipB:public CScriptableBase{public:ClipB(ISystem* pSystem);virtual ~ClipB() {}int GameLog(IFunctionHandler* pH, char* pText);};.cpp#include "StdAfx.h"#include <ScriptHelpers.h>#include <IScriptSystem.h>#include <ScriptHelpers.h>#include "ClipB.h"ClipB::ClipB(ISystem * pSystem){Init(pSystem->GetIScriptSystem(), pSystem);SetGlobalName("Game");#undef SCRIPT_REG_CLASSNAME#define SCRIPT_REG_CLASSNAME &ClipB::SCRIPT_REG_TEMPLFUNC(GameLog, "text");}int ClipB::GameLog(IFunctionHandler * pH, char * pText){return 0;}When we run waf to compile this code we receive this error:c:\Amazon\Lumberyard\1.8.0.1\dev\Code\CryEngine\CryCommon\ScriptHelpers.h(32): error C2061: syntax error: identifier 'IFunctionHandler'I could not find a fix for this. I am reading all the GEMs and another codes like LyShine but none of them showed me the error.Could you guys help us?Tks a lot!Follow"

Answer:
"0Accepted AnswerHonestly, everything is migrating away from ScriptableBase system and moving to Components.This details how to make a component: https://forums.awsgametech.com/t/how-to-create-a-system-component/2553/1As for exposing lua functions, you use behavior contextes.Here's an exerpt of the transform component's reflect function and take a look at it's behavior context section of it:    void TransformComponent::Reflect(AZ::ReflectContext* reflection){AZ::SerializeContext* serializeContext = azrtti_cast<AZ::SerializeContext*>(reflection);if (serializeContext){serializeContext->Class<TransformComponent, AZ::Component, NetBindable>()->Version(2)->Field("Parent", &TransformComponent::m_parentId)->Field("Transform", &TransformComponent::m_worldTM)->Field("LocalTransform", &TransformComponent::m_localTM)->Field("ParentActivationTransformMode", &TransformComponent::m_parentActivationTransformMode);}AZ::BehaviorContext* behaviorContext = azrtti_cast<AZ::BehaviorContext*>(reflection);if(behaviorContext){behaviorContext->EBus<AZ::TransformNotificationBus>("TransformNotificationBus")->Handler<AZ::BehaviorTransformNotificationBusHandler>();behaviorContext->Class<TransformComponent>()->RequestBus("TransformBus");behaviorContext->EBus<AZ::TransformBus>("TransformBus")->Event("GetLocalTM", &AZ::TransformBus::Events::GetLocalTM)->Event("GetWorldTM", &AZ::TransformBus::Events::GetWorldTM)->Event("GetParentId", &AZ::TransformBus::Events::GetParentId)->Event("GetLocalAndWorld", &AZ::TransformBus::Events::GetLocalAndWorld)->Event("SetLocalTM", &AZ::TransformBus::Events::SetLocalTM)->Event("SetWorldTM", &AZ::TransformBus::Events::SetWorldTM)->Event("SetParent", &AZ::TransformBus::Events::SetParent)->Event("SetParentRelative", &AZ::TransformBus::Events::SetParentRelative)->Event("SetWorldTranslation", &AZ::TransformBus::Events::SetWorldTranslation)->Event("SetLocalTranslation", &AZ::TransformBus::Events::SetLocalTranslation)->Event("GetWorldTranslation", &AZ::TransformBus::Events::GetWorldTranslation)->Attribute("Position", AZ::Edit::Attributes::PropertyPosition)->VirtualProperty("Position", "GetWorldTranslation", "SetWorldTranslation")->Event("GetLocalTranslation", &AZ::TransformBus::Events::GetLocalTranslation)->Event("MoveEntity", &AZ::TransformBus::Events::MoveEntity)->Event("SetWorldX", &AZ::TransformBus::Events::SetWorldX)->Event("SetWorldY", &AZ::TransformBus::Events::SetWorldY)->Event("SetWorldZ", &AZ::TransformBus::Events::SetWorldZ)->Event("GetWorldX", &AZ::TransformBus::Events::GetWorldX)->Event("GetWorldY", &AZ::TransformBus::Events::GetWorldY)->Event("GetWorldZ", &AZ::TransformBus::Events::GetWorldZ)->Event("RotateByX", &AZ::TransformBus::Events::RotateByX)->Event("RotateByY", &AZ::TransformBus::Events::RotateByY)->Event("RotateByZ", &AZ::TransformBus::Events::RotateByZ)->Event("SetEulerRotation", &AZ::TransformBus::Events::SetRotation)->Event("SetRotationQuaternion", &AZ::TransformBus::Events::SetRotationQuaternion)->Event("SetRotationX", &AZ::TransformBus::Events::SetRotationX)->Event("SetRotationY", &AZ::TransformBus::Events::SetRotationY)->Event("SetRotationZ", &AZ::TransformBus::Events::SetRotationZ)->Event("GetEulerRotation", &AZ::TransformBus::Events::GetRotationEulerRadians)->Event("GetRotationQuaternion", &AZ::TransformBus::Events::GetRotationQuaternion)->Attribute("Rotation", AZ::Edit::Attributes::PropertyRotation)->VirtualProperty("Rotation", "GetRotationQuaternion", "SetRotationQuaternion")->Event("GetRotationX", &AZ::TransformBus::Events::GetRotationX)->Event("GetRotationY", &AZ::TransformBus::Events::GetRotationY)->Event("GetRotationZ", &AZ::TransformBus::Events::GetRotationZ)->Event("SetScale", &AZ::TransformBus::Events::SetScale)->Event("SetScaleX", &AZ::TransformBus::Events::SetScaleX)->Event("SetScaleY", &AZ::TransformBus::Events::SetScaleY)->Event("SetScaleZ", &AZ::TransformBus::Events::SetScaleZ)->Event("GetScale", &AZ::TransformBus::Events::GetScale)->Attribute("Scale", AZ::Edit::Attributes::PropertyScale)->VirtualProperty("Scale", "GetScale", "SetScale")->Event("GetScaleX", &AZ::TransformBus::Events::GetScaleX)->Event("GetScaleY", &AZ::TransformBus::Events::GetScaleY)->Event("GetScaleZ", &AZ::TransformBus::Events::GetScaleZ)->Event("GetChildren", &AZ::TransformBus::Events::GetChildren)->Event("GetAllDescendants", &AZ::TransformBus::Events::GetAllDescendants)->Event("GetEntityAndAllDescendants", &AZ::TransformBus::Events::GetEntityAndAllDescendants);}NetworkContext* netContext = azrtti_cast<NetworkContext*>(reflection);if (netContext){netContext->Class<TransformComponent>()->Chunk<TransformReplicaChunk, TransformReplicaChunk::Descriptor>()->Field("ParentId", &TransformReplicaChunk::m_parentId)->Field("LocalTransformData", &TransformReplicaChunk::m_localTransform);}}Here, it's mostly with EBuses but to expose a function you use the Method function in the behavior context:behaviorContext->Class<ComponentClass>("ComponentLuaName")->Method("MethodName", Function f, BehaviorValues* defaultValues, "dbgDesc");This can be within a "Class" or global by taking out the "Class" section. The only cadavit, i believe, havent tested it yet, is that you have to assign an entity with the component in order to use the method unless it's a global method.So for the example above would be:ComponentLuaName.MethodName();Personally still trying to figure out this myself but i was able to get ebuses working fine.More "info" about behavior contextes here but it's a bit lack luster...https://docs.aws.amazon.com/lumberyard/latest/developerguide/component-entity-system-behavior-context.htmlSharerePost-User-8564951answered 6 years ago0In the Reflect function of the component there's a parameter of "AZ::ReflectContext*". You convert that to a "AZ::BehaviorContext*" using "AZ::BehaviorContext* behaviorContext = azrtti_castAZ::BehaviorContext*(reflection);" From there you have access to the component's behavior context to implement lua (and/or lumberyard's new visual scripting system).This should be bare basics of what is needed://------------------------------------//--HEADER//------------------------------------class ExampleComponent : public AZ::Component {public:~ExampleComponent() override = default;AZ_COMPONENT(ExampleComponent, "{271C6E71-8352-49F1-A495-A96AB2825335}");static void Reflect(AZ::ReflectContext* reflection);void Call(int in){ CryLog("Call Received: %i", in); }public:protected:// AZComponentvoid Init() override {};void Activate() override {};void Deactivate() override {};};class ExampleBusGroup : public AZ::EBusTraits {public:virtual float Example(int in) = 0;virtual void BusCall(int in) = 0;};using ExampleBus = AZ::EBus<ExampleBusGroup>;class ExampleBusHandler : public ExampleBus::Handler, public AZ::BehaviorEBusHandler {public:AZ_EBUS_BEHAVIOR_BINDER(ExampleBusHandler, //Name"{616AFC57-8CF2-423F-BE4F-BAB3C26AEC8C}", //TypeIdAZ::SystemAllocator, //default allocator.// List of event names to handle and support for BehaviorContext.Example);float Example(int in) { return Call(FN_Example, in); }void BusCall(int in) {}};//------------------------------------//--CODE//------------------------------------void ExampleComponent::Reflect(AZ::ReflectContext* context) {AZ::BehaviorContext* behaviorContext = azrtti_cast<AZ::BehaviorContext*>(context);if (behaviorContext) {behaviorContext->Class<ExampleComponent>("ExampleComponent")->Method("Call",&ExampleComponent::Call, nullptr, "");behaviorContext->EBus<ExampleBus>("ExampleBus")->Handler<ExampleBusHandler>()->Event("BusCall",&ExampleBus::Events::BusCall);}}
SharerePost-User-8564951answered 6 years ago0Here's a rough example on how to deal with ebus integration:class ExampleEbusGroup : public AZ::EBusTraits {public:virtual float Example(int in) = 0;virtual void Call(float in) = 0;};using ExampleBus = AZ::EBus<ExampleEbusGroup>;class ExampleHandler : public ExampleBus::Handler, public AZ::BehaviorEBusHandler {public:AZ_EBUS_BEHAVIOR_BINDER(ExampleHandler //Name"{85E7D575-CFE0-45CC-9F8E-EF12D6452070}", //TypeIdAZ::SystemAllocator, //default allocator.// List of event names to handle and support for BehaviorContext.Example);float Example(int in) { Call(FN_Example,in); }};behaviorContext->EBus<ExampleBus>("ExampleBus") //declare the ebus->Handler<ExampleHandler>() //declare the handler->Event("Call",&ExampleBus::Events::Call); //declare an event
It's a small example and i dont have any experience in declaring events but handlers are really easy to work with.How to deal with ebuses in lua is simple:local luasystem = {Properties = {}
}function luasystem:OnActivate()if self.exampleBus == nil thenself.exampleBus = ExampleBus.CreateHandler(self, self.entityId)self.exampleBus.Connect()endendfunction luasystem::OnDeactivate()if self.exampleBus ~= nil thenself.exampleBus.Disconnect()self.exampleBus = nilendendfunction luasystem:Example(in)Debug.Log("Data in: " .. in)return 10.02fendreturn luasystemFirst off, on activate you create a handler and connect to the bus. When creating the handler, you can just pass "self" or "self, self.entityId", the second one allows for targeted notification whereas the first one you would have to use broadcasting to notify it.Now for communicating to the ebuses in C++ is as followed:-For targeted notification with return:int out = 10;float ret;ExampleBus::EventResult(ret, entity->GetId(), &ExampleEbusGroup::Example, out);
-For broadcast notification without return:int out = 10;ExampleBus::Broadcast(entity->GetId(), &ExampleEbusGroup::Example, out);
to recieve a return from a broadcast it would require return aggregation which is detailed in https://docs.aws.amazon.com/lumberyard/latest/developerguide/asset-pipeline-ebus.html under "Return Values from Multiple Handlers".SharerePost-User-8564951answered 6 years ago0Glad to help. ^.^SharerePost-User-8564951answered 6 years ago0@REDACTEDUSERSharerePost-User-4308601answered 6 years ago0@alatnet tks a lot for your answer! I am trying to follow what you have posted but I am struggling with c++... would you mind to send to me the cpp and .h?I am trying to register the handler inside the cpp of the component/gem. There I can get the behaviorcontext. Am I correct?thank you very much!SharerePost-User-4308601answered 6 years ago"


Question: Get region from within SQL Server RDS instance.

"• Can I query the region from within the SQL Server itself?Anything like rdsadmin.dbo.rds_show_configuration that include instance specs?I have an SP that is clearing all backup tables from production to a secondary DB.I want to back it up to S3 and drop those tables.I have the backup command ready:declare @BackupCMD varchar(4000) = concat('exec msdb.dbo.rds_backup_database @source_db_name=''', 'BackupTables', ''', @s3_arn_to_backup_to=''arn:aws:s3:::', @bucket, '/', @file_name, '.bak'', @kms_master_key_arn=''', @arn, '''')The issue is we have instances over several regions and bucket and key need to match the same region as the instance.If I can I will be able to cross it with the ARN / buckets list that need to match.Thank you!FollowComment"

Answer: 0you could try https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/xp-cmdshell-transact-sql?view=sql-server-ver16 and give an AWS CLI command to obtain current region: 'aws configure get region'CommentShareDidier_AWSanswered 2 months agoBarak 2 months agoThis is blocked by security team. Would work...Share


Question: Route53 - NS changes not reflected to Lightsail instance

"I am trying to set up Cloudflare for several days already and experiencing issues with the updated NS records pointing to my Lightsail instance which in turn gets its domain information from Route 53.Cloudflare requires me to change the Nameserver (NS) records to point towards their servers. I have done so in Route53 several days ago but so far can see via WHOIS that no change has taken effect. Now, having spent a fair bit of time with Lightsail and R53 domains, one recurring issue seems to be that, ie. when creating subdomains, that CNAME records need to be updated in both Route53 and Lightsail > Networking > Domain Zone as they somehow are not aligned and seem to run independently doing the same thing.In the example of creating a subdomain, I found that creating a CNAME record in Lightsail Networking OR Route 53 only, it will not work, rather I have to have the same record in both places.The challenge here now is that Lightsail Networking does not provide me with the option to change my NS, and when changing them in R53 I see no effect on either WHOIS or Cloudflare.So in short:I need to update my NS records, doing so in R53 yields no result for my Lightsail instance. What do I do?Appreciate any help with this, thanks.FollowComment"

Answer: "0one recurring issue seems to be that, ie. when creating subdomains, that CNAME records need to be updated in both Route53 and Lightsail > Networking > Domain Zone as they somehow are not aligned and seem to run independently doing the same thing.In DNS, there can only be one hosted zone for a given (sub-)domain. If Route 53 is the registrar for your domain, look for the Name servers under "Registered domains > <your domain management page>" they will either match a hosted zone in Route 53 or the DNS Zone in Lightsail. Route 53 creates a Hosted zone by default when you register a domain and this can be a source of confusion later if you don't delete it.You can delegate a sub-domain from a hosted zone in Route 53 to a DNS zone in Lightsail, but that's an advanced use-case.Cloudflare requires me to change the Nameserver (NS) records to point towards their servers.If you are adding NS records on a hosted zone they must be for a sub-domain, for example www.<my-domain.com>. If you intend to use Cloudflare to manage all traffic through your domain you need to change the name servers with your registrar. Doing so will invalidate any other records you've configured in any other zone in Lightsail or Route 53.CommentShareGabriel - AWSanswered a year ago"


Question: VPC Gateway Endpoint & Security Group

"Briefly, does VPC Gateway Endpoint or VPC Load Balancer Gateway that powered by PrivateLink leverage "Security Group" ?According to the following references from AWS, it sems that VPC Gateway Endpoint does leverage "Security Group":In AWS Doc titled as  [https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html]("Gateway endpoints for Amazon S3") by one of the conisderations points the following is written:The outbound rules for the security group for instances that access Amazon S3 through the gateway endpoint must allow traffic to Amazon S3. You can use the prefix list ID for Amazon S3 as the destination in the outbound rule.re:Post Question [https://repost.aws/de/knowledge-center/connect-s3-vpc-endpoint]("Why can't I connect to an S3 bucket using a gateway VPC endpoint? " )On the contrary, in one of the online courses that I attend, the instructor claimed that VPC Gateway Endpoint does not leverage "Security Group" at all !?!Could you please clarify this misunderstanding ?FollowComment"

Answer: "1Hi,when the above comments are talking about security groups, they're talking about security groups used by the compute resources (i.e. EC2, vpc lambda etc.) rather than security groups attached to the endpoints.However, to clarify, endpoints are accessed via Elastic Network Interfaces (ENIs) - when you create an endpoint, it will create network interfaces to allow you to connect to the endpoint. Unless overridden, these interfaces use the default security group in the VPC. This is normally configured to allow all incoming network traffic. However, you can override this if required by creating a new security group and attaching it to the ENI, so you could for example, only allow access to the ENI from compute resources with a specific IP or IP range.CommentShareSimon Hanmeranswered 16 days agorePost-User-0862805 16 days agoWhat about "Load Balancer Gateway" powerd by PrivateLink. Does it leverage "Security Group" ?SharerePost-User-0862805 16 days agoYou're absolutely correct, but this is only with the type "VPC Endpoint interface" that powered by "PrivateLink". Still not getting clear answer to my question.Share1There's not two types of Security Group - only one.  Think of a Security Group as a firewall on a network interface (ENI, or Elastic Network Interface).  Anything that has an ENI in your VPC can have a Security Group; that includes EC2 instances and VPC Interface Endpoints.  It does NOT include VPC Gateway Endpoints which are kind of a "routing hack" where you enter routes into your route table to access a gateway rather than traffic being directed to an ENI via DNS.You asked about "Load Balancer Gateway" - do you mean "Gateway Load Balancer"?  This is also powered by PrivateLink so client endpoints have ENIs and Security Groups.CommentShareEXPERTskinsmananswered 16 days agorePost-User-6624931 16 days agoThanks for the clarification. Now, it is clear. The problem was my understanding of "Security Group" operates on instance level, while NACLs on subnet level. According to your explanation, anything that has an ENI in VPC can have "Security Group", which it definetely makes sense.Share0I quite get it now, but it is not clear yet for me. Basically, we have two types of "Security Groups" in AWS:VPC Security Group, e.g. "Default VPC Security Group".Instance Security Group, e.g. "launch-wizard"CommentSharerePost-User-0862805answered 16 days ago"


Question: How to brand(custom style) Amazon Connect Control Pannel?

"We are looking onto ways we can make our custom style Amazon Connect, for basic things such as changing ICON, styling background, etc?If this is not possible, is it possible to create our very simple custom DIAL pad for agents that we can host in our domain without using the AWS branded dial pad for agents to make simple things such as initializing calls, taking calls and also login?Disclaimer: I am newbie in Amazon Connect. ThanksFollowComment"

Answer: "0Accepted AnswerHello,You can create a Custom CCP using the Streams SDK. This workshop  will show a basic way to build a development environment to deploy a custom CCP for amazon connect using the Streams SDK. You can as well leverage reactive frameworks to deploy your CCP as described in this blog.Best regards!CommentShareEstivalanswered a year agofezekilePlaatyi a year agoThank you so much, this is exactly what I needed. God bless you!Share"


Question: Monitoring The body of the Workmail email

"I have to Monitor My user's email content in AWS workmail, for monitoring, I have used cloud watch, but found no metric to monitor the body part of the email sent,  how can I see what exactly my users send to each other.FollowComment"

Answer: "2Hi there -- thank you for your interest in Amazon WorkMail!If you want to perform analytics on content of your outbound messages, creating an Email Flow Rule with an asynchronous Lambda action will help you do that. You'll need to write a Lambda application for your analysis use case, but you can get started by using the applications published in our Serverless App Repository as a starting point. Using the AWS SDK in your Lambda app, you can interact with your WorkMail messages and then put the CloudWatch Metrics to build your desired monitoring.I hope that helps!Kind regards,SabrinaCommentSharesldanswered a year agoAWS-User-2001144 a year agothank you for responding Sabrina, do we have any metric in cloud watch to see the body part of an email in Amazon WorkMail, the way we can see the recipient , subject etc.?Share1Hi, again -- I am adding this as a second answer, since the comment field is too limited in character length to let me reply inline.Thank you for responding Sabrina, do we have any metric in cloud watch to see the body part of an email in Amazon WorkMail, the way we can see the recipient , subject etc.?No, the body part contents wouldn't be available in Event Logging. But you do have full access to review the content using the GetRawMessageContent API, as messages are in flight, which is possible using an EFM rule with a Lambda. You could extract the information you're looking for and put it to CloudWatch to suit your needs.To take this answer from "how can you get this to work" to "how can you get this to work well,"  though, I would recommend two additional considerations.First, do a quick cost estimate to make sure you don't get any surprises in your billing statement. CloudWatch Pricing can give you a rough idea (make sure to select the right region to match your WorkMail Organization). The AWS Calculator for CloudWatch can help you create the full picture. In addition, consider if billing alarms are right for your use case. For example, if you're building logs for content where the content may become unexpectedly high volume or very large in size -- imagine if an automated job were to begin to fail overnight and start to send many megs of backtrace every minute -- you could start logging much more than usual, and it could pile up before you realize.Second, and most importantly, please review all of your access grants and policies to ensure that any data is securely protected  in line with your company's compliance and security obligations. If you are extracting message content and moving it to CloudWatch Logs, that could disclose highly confidential email contents to, for example, someone who has access to CloudWatch Logs for troubleshooting other unrelated applications, like a service running in EC2. I'll refer to the Well-Architected Framework's Security Pillar for more on this, but just in general, I recommend that you build with the principle of least privilege in mind, and put appropriate data security controls in place.I hope that helps!Kind regards,SabrinaCommentSharesldanswered a year ago"


Question: Cognito Identity Pools Attribute-based access control - dynamic attributes

"I have hundreds of S3 buckets and dozens of users in Cognito User Pool. I want to be able to select which user can access which S3 bucket, for example:user_a can access bucket_1, bucket_2, bucket_3user_b can access bucket_2user_c can access bucket_1, bucket_4and so on.I would love to be able to do it without creating a dedicated API creating a dynamic policies. I thought about utilising Cognito Identity Pools and Attribute-based access control.There is a cool example where an user gets an attribute "department": "legal" and is then assigned a role that is allowed to query only the buckets with -legalsuffix, thanks to ${aws:PrincipalTag/department} magic.If my users were to access only one bucket, that would be a solution. However, in my case a user could get assigned to dozens or hundreds of buckets (think "multiple departments" in the example from AWS docs).I thought of using multiple custom attributes on each user:bucket_1: truebucket_2: falsebucket_3: false..and so onand creating a policy that allows you to access given bucket_n if and only if you have an attribute bucket_n: true.This would work if I had at most 50 buckets (the hard limit of Custom Attributes in Cognito).In my case, this value is slightly higher (a couple hundreds). I can have users having access to 200+ buckets as well as ones being allowed to only one bucket.Is there any way to achieve my goal with Cognito Identity Pools and IAM Policies?FollowComment"


Question: CloudWatch Dashboard to manage & monitor a budget

"In CloudWatch dashboard, is it possible to use math expressions (e.g. based on EstimatedCharges or some other metric) to create a metric that shows a budget, with month-to-month cloud spending gradually deducted from that budget?Example: Suppose there is a starting budget of $1000. The cloud spend is $200 in the first month, and $100 in the second. Hence, the metric needs to have an initial value of $1000, which decreases to $800 at the end of the first month, and $700 at the end of the second month.Can I use CloudWatch Dashboards with math expressions for this, or would other services be needed? e.g. QuickSight, Lambda?FollowComment"

Answer: "1Your use case might be met using AWS Budgets? For more details see https://aws.amazon.com/aws-cost-management/aws-budgets/.CommentShareAWS-SAanswered a year agoNathaniel Ng a year agoOkay, I will test this out to see if it works. I suppose Budget Renewal Type = "Expiring budget" and Budgeting method = "Fixed"?Share1This graph would be approximately what you need, where you are plotting 1000 less the running total of your estimate spend.{    "metrics": [        [ { "expression": "1000 - RUNNING_SUM(monthly)", "label": "Expression1", "id": "budget" } ],        [ "AWS/Billing", "EstimatedCharges", "Currency", "USD", { "id": "monthly", "visible": false } ]    ],    "view": "timeSeries",    "stacked": false,    "region": "us-east-1",    "stat": "Maximum",    "period": 2592000}However, it is difficult to get exact value of spend in a month.  To get exact values you would be better off querying AWS Cost Explorer.  CloudWatch Dashboards has a Custom Widget feature that comes with a simple AWS Cost Explorer report. It would be relatively easy to adapt the code in that to display what you need.CommentShareAmazon-CloudWatch-01answered a year agoEXPERTFabrizio@AWSreviewed a year ago"


Question: EC2 instance - reachability failed

"Hello,I received an email stating that one of my EC2 instances will be retired in two weeks. I wanted to back it up, but I couldn't connect to the instance. When I checked the instance state, it showed that:"InstanceStatus": {"Details": [{"ImpairedSince": "2023-05-09T17:06:00+00:00","Name": "reachability","Status": "failed"}],"Status": "impaired"Is there any way to create a backup?FollowComment"

Answer: "0Hello,You can create an Amazon Image (AMI) or a EBS Snapshot of your EBS volumes, you don't need access to the OS, you can create the Backup from the AWS EC2 Console Web Page  or using the command line (CLI).Also, if your EC2 is not an Instance Store based you can simple Stop the EC2 and then Start it up, sometimes this simple process remediates the access to your EC2 as long as the issue was with the underlying host or a transitory issue with your OS.Kind regards.CommentShareHigheranswered 21 days ago0If you have created a snapshot of the instance before now, you can try recovering from that https://docs.aws.amazon.com/prescriptive-guidance/latest/backup-recovery/restore.html#instance-from-snapshotIf you don't have a snapshot you can try creating one now, and then following the same process to create a new AMI and create a new instance from it (though with the proviso that you are creating a new instance from an image of an instance that is impaired, so don't be surprised if it doesn't work first time and you have to fiddle with things).Other documentation that may help here:Creating snapshots https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.htmlCreating backups with snapshots https://docs.aws.amazon.com/prescriptive-guidance/latest/backup-recovery/new-ebs-volume-backups.htmlCreating an AMI https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-ebs.htmlBackup and recovery with snapshots https://docs.aws.amazon.com/prescriptive-guidance/latest/backup-recovery/ec2-backup.htmlCommentShareRWCanswered 21 days ago"


Question: Firehose delivery stream writes every record exactly 2 times

"Events are sent once from a lambda function to a firehose delivery stream without any transformation or conversion. observing the data on s3 every record is written twice.the lambda uses "putRecordBatch" to send the object to FDS.example:{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}"id" is uniqueI understand that duplication might happen but not to 100% of the data.how can we figure out the cause of this duplication if it is abnormal.FollowComment"

Answer: 0pushRecord was called with a callback and the invoked again vai .promise()CommentSharelimeanswered 4 years ago


Question: {{iframe}} doc view error

"Hi,Seems that the function to embed in {{iframe}} a doc that is stored in WorkDocs is not working anymore. But the issue is not the embed but seems that the doc can't be opened by the function.{{iframe url="https://amazon.awsapps.com/workdocs/pdfjs/viewer.html#/documentid=docID.from.workdocs" width="900px" height="900px";}}{{/iframe}}Any fix for this or advice?error message: "We weren't able to load this document. Please try again later."FollowComment"


Question: How do I selectively override WAF rules for only specific URIs in CloudFormation?

"There are some false positives in the AWS common rule sets that make it difficult to implement as-is.For example, if one of your URI endpoints accepts XML or HTML/HTML fragments, then you will very likely have problems with the CrossSiteScripting family of rules.  Also, if you are publishing content, then there are some sequences that trigger LFI-family rules as well, like "../."It sure seems sad to lose out of all of that CrossSiteScripting goodness for all of the other URIs that don't accept that type of data.This post was very helpful and gave me hope that I could make it work for multiple URIs with a bit of work, but I wasn't able to get it to work for me for multiple URIs.What's the most efficient way of handling this for just a few URIs?FollowComment"

Answer: "0Accepted AnswerAfter many hours on a call with the wonderful AWS employees in Cape Town, South Africa, and then some more trial and error, we have settled on this for our implementation:The following is a complete example (except for actually associating the resources with WAF) with simple logging for easy debugging.Allow CrossSiteScripting_BODY and GenericLFI_BODY on /uri1, /uri2, and /uri3 only.Log WebACL traffic to CloudWatch so we don't have to worry about S3 permissions and AWS Glue and Athena to query WAF logs.Redact sensitive information.I threw inAWSTemplateFormatVersion: '2010-09-09'Description: WAF implementation that allows exclusions for specific URIsTransform: AWS::Serverless-2016-10-31Parameters:  ProjectName:    Description: Project Name    Type: StringResources:  regionalWebAcl:    Type: AWS::WAFv2::WebACL    Properties:      Scope: REGIONAL      Name: !Sub ${ProjectName}-regional-webacl      DefaultAction:        Allow: {}      VisibilityConfig:        CloudWatchMetricsEnabled: true        MetricName: !Sub ${ProjectName}-regional-webacl-metric        SampledRequestsEnabled: true      Rules:        - Name: !Sub ${ProjectName}-regional-webacl-AWSManagedRulesKnownBadInputsRuleSet          Priority: 5          OverrideAction:            None: {}          VisibilityConfig:            CloudWatchMetricsEnabled: true            MetricName: !Sub ${ProjectName}-regional-webacl-AWSManagedRulesKnownBadInputsRuleSet-metric            SampledRequestsEnabled: true          Statement:            ManagedRuleGroupStatement:              Name: AWSManagedRulesKnownBadInputsRuleSet              VendorName: AWS        - Name: !Sub ${ProjectName}-regional-webacl-AWSManagedRulesCommonRuleSet          Priority: 10          OverrideAction:            None: {}          VisibilityConfig:            CloudWatchMetricsEnabled: true            MetricName: !Sub ${ProjectName}-regional-webacl-AWSManagedRulesCommonRuleSet-metric            SampledRequestsEnabled: true          Statement:            ManagedRuleGroupStatement:              Name: AWSManagedRulesCommonRuleSet              VendorName: AWS              RuleActionOverrides:                - Name: CrossSiteScripting_BODY                  ActionToUse:                    Count: {}                - Name: GenericLFI_BODY                  ActionToUse:                    Count: {}        - Name: !Sub ${ProjectName}-regional-webacl-reblock          Priority: 15          RuleLabels:            - Name: reblock          Statement:            AndStatement:              Statements:                - OrStatement:                    Statements:                      - LabelMatchStatement:                          Key: awswaf:managed:aws:core-rule-set:CrossSiteScripting_Body # Pay very careful attention to the casing, it's NOT CrossSiteScripting_BODY                          Scope: LABEL                      - LabelMatchStatement:                          Key: awswaf:managed:aws:core-rule-set:GenericLFI_Body  # Pay very careful attention to the casing, it's NOT GenericLFI_BODY                          Scope: LABEL                - NotStatement:                    Statement:                      RegexMatchStatement:                        FieldToMatch:                          UriPath: {}                        RegexString: ^.*\/(?:uri1|uri2|uri3)$ # Note that this regex is supposed to (according to AWS support that I received) match on the whole field, not just part of it.  Also notice that you don't specify the leading forward slash and the trailing slash with common regex flags.  If you need lowercase, then use a TextTransformation to do that.                        TextTransformations:                          - Priority: 0                            Type: NONE          Action:            Block: {}          VisibilityConfig:              CloudWatchMetricsEnabled: true              MetricName: !Sub ${ProjectName}-regional-webacl-reblock-metric              SampledRequestsEnabled: true  regionalWebAclLoggingConfiguration:    Type: AWS::WAFv2::LoggingConfiguration    Properties:      LogDestinationConfigs:        - !GetAtt regionalWebAclLogGroup.Arn      ResourceArn: !GetAtt regionalWebAcl.Arn      RedactedFields:        - SingleHeader:            Name: authorization        - QueryString: {} # If you use an auth query string parameter, redact the whole query string      LoggingFilter:        DefaultBehavior: DROP        Filters:          - Behavior: KEEP            Requirement: MEETS_ANY            Conditions:              - ActionCondition:                  Action: BLOCK          - Behavior: KEEP            Requirement: MEETS_ANY            Conditions:              - ActionCondition:                  Action: COUNT  regionalWebAclLogGroup:    Type: AWS::Logs::LogGroup    DeletionPolicy: Delete    UpdateReplacePolicy: Delete    Properties:      LogGroupName: !Sub aws-waf-logs-${ProjectName}-regional-webacl      RetentionInDays: 30Outputs:  regionalWebAclArn:    Value: !GetAtt regionalWebAcl.Arn  regionalWebAclLogGroupArn:    Value: !GetAtt regionalWebAclLogGroup.ArnPlease note that the rule names and rule labels are identical in the characters used, but not in the casing, so you cannot simply copy and paste a rule name from this documentationRegarding RegEx testing, they recommended this website in PCRE2 mode: https://regex101.com/CommentShareBrandon Milleranswered 2 months ago"


Question: "Problem with AWS Elemental MediaLive Workflow Wizard, Error OutputDataBackground failed to send file for URL [mediapackage]"

"I am using AWS media live workflow wizard for creating the channel and OTT live feeds. It has been successully created. I am using OBS software for sending RTMP push to Media Live. Connection also established. But HLS ingest point is giving below error.Cannot load https://c75a7e79204e539d.mediapackage.us-east-1.amazonaws.com/out/v1/070112deeb2945b4b135134ce269fc73/CMAF_HLS/index.m3u8HTTP response code:0This might be a CORS issue, consider installing Allow-Control-Allow-Origin Chrome ExtensionI did try installing chrome extension for CORS, disbale firewall, switched to different browsers.But error is same.In the cloudwatch notification for channel, I am getting alertsOutputDataBackground failed to send file for URL [mediapackage://LiveChan1/channel_emp_240p30_20220805T200336_00298.ts],after [3] attempts, error [failed to write output to mediapackage]FollowComment"

Answer: 0There was a problem in the IAM role created for media live. It was not having permission to access to media package for transferring files. After modifying IAM role for media live solved the issue.CommentShareAbhinavanswered 10 months ago


Question: Account details

"We have a running main account with where I would like to customize the details like company name, address etc.Does this have a possible impact on the running services or can I customize this without worrying ?FollowComment"

Answer: "0Yes, the change will not affect existing services.CommentShareEXPERTRiku_Kobayashianswered 23 days ago0Changing this info on the account settings page is just an administrative edit (like contact details, company name, address, tax settings, even account name or password...) - these don't have impact on the underlying running services and resources, or the worloads that you have created within the account.The only admin change that could potentially impact your running services would be... if you changed your payment method in the billing settings in some way, and if you then missed your monthly bill payment, and then left that bill unpaid for over a month, your account could get suspended until you'd pay your bill... in that case you could lose access to your running services, or they could get terminated over time.Other than that - if you simply made admin changes in your account to update any contact info or similar, that has does not affect your running resources.CommentShareEXPERTNataliya_Ganswered 23 days ago"


Question: Subscribe-Success message is not getting received into SQS from SNS topic while integrating AWS Marketplace SaaS product with Contract pricing

"I am trying to integrate our SaaS product with AWS Marketplace. Our product has been published to limited. We have also created a SQS and subscribed the Subscription Notification topic to the SQS. However, after making a test purchase, I am not receiving any subscribe-success message to queue.FollowComment"

Answer: "0Hello,There could be a few reasons due to which the “Subscribe-Success message” is not being received to SQS from an SNS Topic:SQS Access Policy permission issues: Please ensure you have set Principal to be the Amazon SNS service. For more details on the same, please refer to the below attached document:[+] https://docs.aws.amazon.com/sns/latest/dg/subscribe-sqs-queue-to-sns-topic.html#SendMessageToSQS.sqs.permissionsUser permissions to appropriate topic and queue actions : Here there are two ways to control access to a topic or queue:Add policy to IAM user or group:Please refer to the sample below:{"Statement": [    {"Effect": "Allow",      "Action": "sns:Publish",      "Resource": "arn:aws:sns:us-east-2:1234567890**:MyTopic"    }  ]}{"Statement": [    {"Effect": "Allow",      "Action": [        "sqs:ReceiveMessage",        "sqs:DeleteMessage"      ],      "Resource": [        "arn:aws:sqs:us-east-2:1234567890**:MyQueue1",        "arn:aws:sqs:us-east-2:1234567890**:MyQueue2"      ]    }  ]}Adding a policy to a topic or queue:Please refer to the sample below:{  "Statement": [    {      "Effect": "Allow",      "Principal": {        "AWS": "1111222233**"      },      "Action": "sns:Publish",      "Resource": "arn:aws:sns:us-east-2:<Account-number>**:MyTopic"    }  ]}{"Statement": [    {"Effect": "Allow",      "Principal": {"AWS": "1111222233**"      },      "Action": [        "sqs:DeleteMessage",        "sqs:ReceiveMessage"      ],      "Resource": [        "arn:aws:sqs:us-east-2:<Account-number>**:MyQueue"      ]    }  ]}You can refer to this document[+] for more details on the same[+] https://docs.aws.amazon.com/sns/latest/dg/subscribe-sqs-queue-to-sns-topic.html#SendMessageToSQS.iam.permissionsCommentShareSUPPORT ENGINEERrePost-User-4843893answered 9 months ago"


Question: Viewing broker host information

"I created a Kafka cluster with 2 Brokers per AZ, and when I click on the View client information button, it only lists 3 zookeeper and broker hosts. Isn't it supposed to have 6 in total ?Here are the host names of the brokersb-6.********.kafka.us-east-2.amazonaws.com:9092b-1.**********.kafka.us-east-2.amazonaws.com:9092,b-4.**********.kafka.us-east-2.amazonaws.com:9092 I am assuming it is missing b-2,b-3 and b-5Thanks,JasonEdited by: jcoelho on May 2, 2019 6:55 AMFollowComment"

Answer: "0Hi Jason,The brokers listed are bootstrap brokers. It is not necessary to list all the brokers in the cluster as bootstrap brokers, and 3 are sufficient for producers/consumers to contact the cluster. So we don't list all the brokers there.ThanksCommentShareAWS-User-0635593answered 4 years ago"


Question: Is there a way to Connect a raspberry pi to a aws react app?

"Hi, I'm wondering if there is a way to connect a raspberry pi vea token to an Aws react app that I can then use to call out information on the pi. Then when I want to change something on the pi I can have the website send that to the pi?FollowComment"

Answer: "0Hi - Not very sure on your overall requirements on what is getting changed on the PI but one option is to use it as an IOT device which listens to a mqtt topic. The app which is react in this case can publish to the same IOT topic and then pi which is subscribed to that topic can read the payload and do the required changes. You can also you use device shadow which can make a device’s state available to apps and other services whether the device is connected to AWS IoT or not.Referenceshttps://docs.aws.amazon.com/iot/latest/developerguide/topics.htmlhttps://docs.aws.amazon.com/iot/latest/developerguide/iot-device-shadows.htmlYou can also check https://aws.amazon.com/iot-device-management/ which can help to register, organize, monitor, and remotely manage connected devices at scaleCommentShareEXPERTAWS-User-Nitinanswered 9 months ago"


Question: How do I check what is using my Free Tier Amazon Elastic Block Store (EBS)?

"I have started a project on AWS free tier, with a project size of ~1 Gb, but when I look at my /billing/home/freetier report page, it says my current usage is 25 Gb, and forcasted to hit 54 Gb. Free tier comes with 30 Gb/mo so I am trying to figure out how it has gotten to 25Gb already.In my AWS console, when I run du -sh it says I've only used ~4.6 GbFilesystem      Size  Used Avail Use% Mounted on/dev/root        32G  4.6G   27G  15% /tmpfs           507M     0  507M   0% /dev/shmtmpfs           203M  885k  202M   1% /runtmpfs           5.3M     0  5.3M   0% /run/lock/dev/xvda15     110M  5.5M  104M   5% /boot/efitmpfs           102M  4.1k  102M   1% /run/user/1000FollowComment"

Answer: "1Your "Snapshot ID snap-027e114..." would be the snapshot backing the AMI used to create the instance.  If you used a public AMI rather than one of your own, then that snapshot would likewise be public and not listed among the ones you own.In Billing you said it shows "26.161 GB" - do you mean 26.161 GB-mo?  Yes it will steadily increase through the month.  If you have a 30GB volume then over a month you will be charged 30 GB-mo, but a days-worth of it each day.  If you really do have only a single 30 GB volume then you would be billed roughly 1 GB-mo per day and you wouldn't go over 30 GB-mo each month.CommentShareEXPERTskinsmananswered 4 months ago0With EBS, you pay for the space that you have provisioned for the volume(s), not what is used by the OS.CommentShareEXPERTkentradanswered 4 months agoCasey 4 months agoApologies as I am new to all this AWS stuff, but what do you mean by "the space that you have provisioned for the volume(s)"?And why does it increase every day even when I'm not working on the project?Sharekentrad EXPERT4 months agoSo, if you allocated a 30GB volume but only stored 4GB of files on the volume, you are charged for the amount that is allocated to the volume, 30GB. Are you creating snapshots of the volumes, that is an additional cost. You can drill down on this on the EC2 Dashboard in the Console, look for Volumes and Snapshots in the left menu. Also, look at the Billing Dashboard, Cost and Usage Reports.ShareCasey 4 months agoThank you @kendtrad, looking in the Console, I have the one Volume with a size of 30 GiB, which shows a Snapshot ID snap-027e114... But when I look at Snapshots, there are none listed in the table.In my Billing Dashboard, under AWS Service Charges > Elastic Compute Cloud > US West (Oregon) > EBS it shows the 26.161 GB. It has been steadily increasing there since Thursday (when it was at 15 GB) which is why I am concerned.Are you saying it will stop at 30 GB because my volume size is 30 GB? And therefore remaining within the free tier limits?Share"


Question: Configure Letsencrypt ssl certificate for AWS Lightsail instance

"Namaste,We have a domain say  [example.com]  hosted in server x from last 3 years. There we configured letencrypt and it is working fine.Recently we are in to AWS cloud and we are created one lightsail instance and couple of subdomains say [sd1.example.com] and [sd2.example.com] are pointed to AWS static ip using DNS A record in server x . After that we are added letsencrypt certificate as per this tutorial https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-using-lets-encrypt-certificates-with-wordpress The ceritificate installation went correctly and got the congratulations message with expiry date in CLI as mentioned in tutorial.The TXT records are added at both DNS zone of server x and also tried with creating DNS zone within lightsail. Tried with rebooting apache and whole instance also.But, still browser not able to validate SSL and showing invalid certificate notice.Where we are missing?Edited by: SaraghsoftTechnologies on Oct 6, 2020 8:41 PMFollowComment"

Answer: "0This is solved by using the method mentioned in this link https://docs.bitnami.com/aws/apps/wordpress/administration/generate-configure-certificate-letsencrypt/Edit: https://docs.bitnami.com/aws/apps/wordpress-pro/administration/generate-configure-certificate-letsencrypt/Edited by: SaraghsoftTechnologies on May 31, 2021 2:19 AMCommentShareSaraghsoftTechnologiesanswered 3 years ago"


Question: Where data stored if I use Glacier Flexible Retrieval  class?

"I have some data in Glacier Flexible Retrieval , and I want to clear those data, please tell me how to do this.I have searched for the Internet. However, I only find CLI command to delete Glacier vaults onlyFollowComment"

Answer: 1Accepted AnswerHiS3 Glacier Flexible Retrieval is a Storage Class within Amazon S3. You can use normal S3 commands for delete object operations as shown below:https://docs.aws.amazon.com/AmazonS3/latest/userguide/delete-objects.htmlAbove reference is when an object is stored in S3 Glacier Flexible Retrieval.  Glacier vault has its own set of APIs.CommentSharerePostfanclubanswered a year ago


Question: DL AMI GPU TensorFlow 2.9.1 (Ubuntu 20.04) on p3.8xlarge not using GPUs

"This AMI comes with tensorflow, but I had to add tensorflow-gpu.  When I test what we have:print (tf.version)print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))I get:2.9.1Num GPUs Available:  4And running the canonical examples shows a GPU being used:c = tf.matmul(a, b)Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0Executing op MatMul in device /job:localhost/replica:0/task:0/device:GPU:0My code is implemented to do distributed training (and I've tested without that too), but the result is the same: nvidia-smi shows basically no GPU usage.  It's also very slow (training on 50,000 image for semantic segmentation):|   0  Tesla V100-SXM2...  On   | 00000000:00:1B.0 Off |                    0 || N/A   39C    P0    49W / 300W |  15238MiB / 16384MiB |      0%      Default ||                               |                      |                  N/A |+-------------------------------+----------------------+----------------------+|   1  Tesla V100-SXM2...  On   | 00000000:00:1C.0 Off |                    0 || N/A   40C    P0    49W / 300W |  15112MiB / 16384MiB |      0%      Default ||                               |                      |                  N/A |+-------------------------------+----------------------+----------------------+|   2  Tesla V100-SXM2...  On   | 00000000:00:1D.0 Off |                    0 || N/A   39C    P0    52W / 300W |  15112MiB / 16384MiB |      0%      Default ||                               |                      |                  N/A |+-------------------------------+----------------------+----------------------+|   3  Tesla V100-SXM2...  On   | 00000000:00:1E.0 Off |                    0 || N/A   40C    P0    54W / 300W |  15112MiB / 16384MiB |      0%      Default |Does anyone have any idea what I should look at?  Or - and I have looked - are there examples out there of this working?GPU-Util at 0% is not right.  On my workstation with a Quadro GV100 I run it to 97% and 30G memory usage.https://www.tensorflow.org/guide/distributed_trainingmirrored_strategy = tf.distribute.MirroredStrategy()FollowComment"

Answer: "0Hello,Thank you for using AWS DLAMI.Looking at the above query, I understand that while using DLAMI for TensorFlow 2.9.1 it was observed that GPU utilization was zero and it was using CPU after adding tensorflow-gpu explicitly.From the complete scenario shared, I assume that the tensorflow-gpu is added but eventually it is using the tensorflow-cpu, but again due to limited visibility on your setup and code used I cannot be sure enough to claim that. To further understand the issue more in depth, I'd recommend you to reach to AWS Premium Support by creating a support case so that the engineer can investigate the root cause of the issue.But prior to that, I'd highly recommend using DL container on SageMaker instead using DLAMI on EC2. Along with that you use  SageMaker distributed training libraries which are available only through the AWS deep learning containers for the TensorFlow. Additionally SageMaker also provides Bring you container, where the users can built their own container as per their requirements for the use case.If using SageMaker also does not help overcome the issue, then as mentioned above please reach out to AWS Premium Support so that we can better assist you and help you overcome the issue.Reference:[1] TensorFlow with Horovod - https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-horovod-tensorflow.html[2] Amazon SageMaker Distributed Training Libraries - https://docs.aws.amazon.com/sagemaker/latest/dg/distributed-training.html[3] https://github.com/aws/sagemaker-tensorflow-extensions[4] Use TensorFlow with Amazon SageMaker - https://docs.aws.amazon.com/sagemaker/latest/dg/tf.html[5] Open a support case with AWS using the link: https://console.aws.amazon.com/support/home?#/case/createCommentShareSUPPORT ENGINEERKartik_Ranswered 9 months ago"


Question: IS AWS BATCH IS AVAILBLE WITH FREE TIRE

"I am beginner in aws, just want to know the aws batch is free? and to create sample job what will be prerequisite?FollowComment"

Answer: "0Hello,According to AWS Batch pricing:There is no additional charge for AWS Batch. You pay for AWS resources (e.g. EC2 instances, AWS Lambda functions or AWS Fargate) you create to store and run your application. You can use your Reserved Instances, Savings Plan, EC2 Spot Instances, and Fargate with AWS Batch by specifying your compute-type requirements when setting up your AWS Batch compute environments. Discounts will be applied at billing time.If you have AWS Free Tier account, you can try it for free.Here's the documentation to get started: Getting Started with AWS BatchCommentShareMickael-AWSanswered 10 months ago"


Question: unable to install psycopg2 in glue JUPYTER notebook

"DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-supportDefaulting to user installation because normal site-packages is not writeableLooking in indexes: https://aws:****@amazon-590186200215.d.codeartifact.us-east-1.amazonaws.com/pypi/aws-glue/simple/User for amazon-590186200215.d.codeartifact.us-east-1.amazonaws.com:FollowComment"

Answer: "0If you are using a Sagemaker notebook attached to Glue dev endpoint with PySpark kernel, please use the following method to update the libraries.Update the dev endpoint using aws glue update-dev-endpoint --cli-input-json file://./input.json where input.json looks like{       "EndpointName": "YOUR_DEV_ENDPOINT_NAME",       "CustomLibraries": {           "ExtraPythonLibsS3Path": "s3://my-bucket/library1.zip,s3://my-bucket/library2.zip,s3://my-bucket/library3.zip"       },       "UpdateEtlLibraries": true  }You can then verify the new list of libraries using following Python code:import osos.listdir('.')Please note that as of writing this, Sagemaker Jupyter notebooks running PySpark kernel now come with following Python version3.6.12 (default, May 18 2021, 22:47:55)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]CommentShareSUPPORT ENGINEERManu_Ganswered a year ago"


Question: Can't get Partitions to work with my Glue Data Catalog

"I have S3 files that are uploaded to a single bucket.  There is no folders or anything like that, its just 1 file per hour uploaded to this bucket.I run a Glue ETL job on these files, do some transformations, and insert the data into a Glue Data Catalog stored in a different bucket.I can then query that Glue Data Catalog with Athena, and that works.What I would like to do is store the files in the S3 folder of the Data Catalog as YEAR/MONTH/DAY, using partitions.  Even though the SOURCE data is just files uploaded every hour with no partitions, I want to store them in the Data Catalog WITH partitions.  So I extracted the YEAR, MONTH, DAY from the files during Glue ETL, and created columns in my Data Catalog table accordingly and marked them as partitions:Partition 1 YEARPartition 2 MONTHPartition 3 DAYThe proper values are in these columns, and I have verified that.After creating the partitions I rand MSCK REPAIR TABLE on the table, and it came back with "Query Ok."I then ran my Glue ETL job.  When I look in the S3 bucket I do not see folders created.  I just see regular r-part files.  When I click on the Table Schema it shows the columns YEAR, MONTH, DAY marked as partitions, but when I click on View Partitions it just shows:year                  month                     dayNo partitions foundWhat do I need to do? These are just CSV files.  I can't control the process that is uploading the raw data to S3, it is just going to store hourly files in a bucket.  I can control the ETL job and the Data Catalog.  When I try to query after creating the partitions and running MSCK REPAIR TABLE, there is no data returned.  Yet I can go into the Data Catalog bucket and pull up one of the r-part files and the data is there.FollowComment"

Answer: "0Accepted AnswerI believe I got this figured out.  I was missing a step, where I had to add additionalOptions to my sink.  As explained here:https://medium.com/analytics-vidhya/add-new-partitions-in-aws-glue-data-catalog-from-aws-glue-job-79b0442b17af# parameter "enableUpdateCatalog" tells the aws glue job to update #the glue data catalog  as the new partitions are createdadditionalOptions = {"enableUpdateCatalog": True}# define the partition keys additionalOptions["partitionKeys"] = ["year", "month","day"]After doing this, I saw that the partitions (nested directories) were being created.  Now I am having some other isuses with querying, but I will post that as a separate question on here.CommentSharebfeenyanswered a year ago"


Question: Bug Report - AWS SES - Email as case sensitive

"I was working in AWS SES for the last days, and I found a strange behavior on the identity validation and the email sending function (related to case-sensitive email).Basically, as I'm working on sandbox, the recipient email must be validated.So, I created an identity and I successfully validated the email "user.name@example.com" (it's an example).After that, I tried to send an email using AWS SDK for JavaScript, with nodemailer (using SES as provider) to the email: "User.Name@example.com" and I received the following message: "Email address is not verified. The following identities failed the check in region XX-XXXX-X: User.Name@example.com".To verify if the case sensitive was the problem, I tried again using the recipient email as "user.name@example.com" and it works!Is it a bug or this is the expected behavior?I guess that it could be a strange behavior because emails are not case-sensitive.RequestID: 09427677-c13b-44d1-9bdc-be9941697413Thank you,Lucas.FollowComment"

Answer: "0Great catch, Lucas.This is expected behavior by AWS.  As noted on https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses.html for email verification, email addresses are case sensitive and the email cannot be used unless the case matches what was verified.The example AWS gives is sender@EXAMPLE.com vs sender@example.com.CommentSharejsoncanswered a year ago"


Question: "HTTP API GW -> (WAF) -> ALB, cannot pick up source IP"

I have an HTTP API GW that connects to a private ALB via VPC Link.But i cannot make WAF understand the forwarded HTTP header that APIGW setsforwarded: for=someip;host=somehost;proto=httpsFrom what i understand WAF wants a CSV type of input in the header it reads for IP and uses the first one and the documentation states that it's usually X-Forwarded-ForIs there any way of making WAF understand the format that HTTP API GW is sending to ALB?FollowComment

Answer: "0The WAF attached to the ALB which is behind API Gateway does not recognize the source IP of the client. One approach would be to front CloudFront before API Gateway and have AWS WAF on CloudFrontAlternatively you could use HTTP API GW -> WAF -> NLB -> ALB.Or Switching to port base routing as opposed to path based routing and changing from ALB to NLB.CommentSharePiyushMattooanswered a month agoalexander a month agoI tried placing a CF in front of the GW (which is the cleaner solution i agree), but for the life of me I could not make it workFollowed several guides but i only ended up with"< x-cache: Error from cloudfront"Route53 -> CF -> custom domain in my HTTP API GWAnyone had similar issues?Share"


Question: Maximum length MQTT payload

"Hello AWS team!I'm using AWS IoT Core with an ESP32.When using MQTT test client with this JSON everything works as expected:{"pattern1": { "name":"Hello", "phases": {"position": "1", "speed":"1", "accel" : "0.2", "delay":"0"}},"pattern2": { "name":"World", "phases": {"position": "1", "speed":"1", "accel" : "0.2", "delay":"0"}}}The problem is that I want to send a lot of objects in this JSON, probably 50 or more. When I test it with 3 objects it doesn't work{"pattern1": { "name":"Smooth", "phases": {"position": "1", "speed":"1", "accel" : "0.2", "delay":"0"}},"pattern2": { "name":"Yanking", "phases": {"position": "1", "speed":"1", "accel" : "0.2", "delay":"0"}},"pattern3": { "name":"Yanking", "phases": {"position": "1", "speed":"1", "accel" : "0.2", "delay":"0"}}}I realized that if I reduce the number of words then I'm able to send it, like this:{"1": { "n":"S", "p": {"p": "1", "sp":"1", "a" : "0.2", "d":"0"}},"2": { "n":"Y", "p": {"p": "1", "s":"1", "a" : "0.2", "d":"0"}},"3": { "n":"Y", "p": {"p": "1", "s":"1", "a" : "0.2", "d":"0"}},"4": { "n":"Y"}}I read that the MQTT payload limit is 128kb, but those 3 objects are smaller than 128kb.On the ESP32 side, the callback function is triggered as expected. But, when the payload is too big it is not triggering.Is this an AWS problem related to being able to send those larger payloads?The AWS region is sa-east-1Thanks a lot in advance!FollowComment"

Answer: "1Hi juanGomez. Although IoT Core can handle messages up to 128KB, practical limits, in regards to RAM and buffer sizes on the device, are usually hit first.Assuming you're using AWS IoT Embedded C SDK libraries, please check the buffer size you are passing into MQTT_Init():https://aws.github.io/aws-iot-device-sdk-embedded-C/202012.00/libraries/standard/coreMQTT/docs/doxygen/output/html/mqtt_init_function.html.If your message is bigger than that buffer size, you will have a problem. Although your failed message doesn't look very big.Are you calling MQTT_Publish() to publish? Are you checking the return value?https://aws.github.io/aws-iot-device-sdk-embedded-C/202012.00/libraries/standard/coreMQTT/docs/doxygen/output/html/mqtt_publish_function.html.This call may be failing. Hence the message is perhaps never sent, and that may be why you don't get the callback.On the cloud side, you can enable AWS IoT logging in CloudWatch to get greater visibility:https://docs.aws.amazon.com/iot/latest/developerguide/configure-logging.htmlCommentShareEXPERTGreg_Banswered 3 months ago0Thanks a lot for the help.Problem was that MQTT_MAX_PACKET_SIZE was set to 256 bytes.I changed it to 65k bytes using setBufferSize() function and it solved the issue.Thanks a lotCommentSharejuanGomezanswered 3 months agoGreg_B EXPERT3 months agoGlad to hear it. Please don't be shy about accepting answers. We all like a few points.65KB is a big buffer by the way. I suspect you probably didn't need to go that big.Share0You are correct about the max payload of an MQTT message being 128KB, and if you were to run up against this limit, you would receive a CLIENT_ERROR with reason of PAYLOAD_LIMIT_EXCEEDED (see references below).https://docs.aws.amazon.com/iot/latest/developerguide/topicdata.htmlhttps://docs.aws.amazon.com/general/latest/gr/iot-core.html#message-broker-limitsI am able to successfully publish the 2nd example you've given above...Are you receiving a payload limit exception error? I suspect there is some other reason you are running into this issue.Please accept answer if at all helpful!CommentShareDarren Ranswered 3 months ago0Hi,I am facing problem to build HTTPS API for AWS IoT core. and Facing  problem with GPRS through Arduino. How can I solve this?CommentSharerePost-User-2143012answered 2 months ago"


Question: AWS CDK Codepipeline post step lambda invoke action being incorrectly added parallel to a pre step lambda invoke action

"Hello,I'm using the AWS CDK to define our infrastructure. We use 3 accounts: Security account, Dev and Prod account. My Security account holds common resources and the Code Pipeline that deploys the infra to Dev and Prod accounts. I created a Lambda function that i need to run before and after the Dev stage (which deploys the resources to the Dev account) and also for the Prod stage, which does the same but for the Prod account.This function basically makes a http call to a Google Cloud function to block and unblock some IoT Devices, based on the parameters i pass to it as part of the UserParameters object inside the event object it receives after being invoked. To avoid issues when updating the underlying infra, I was planning on blocking them as a 'Pre' step for both my stages and unblocking them as a 'Post' step for both my stages. There is also a Manual approval step as a 'Post' step between the Dev and Prod stages.The problemThe problem i'm having is that even though i put the call to the lambda action that blocks the devices as a 'Post' step for both my prod and dev stages, it is always added right next to the call that blocks the devices, so it is being treated as if it where a 'Pre' step. The manual approval step previously mentioned is correctly placed as a 'Post' step. Attached is a screen capture where you can see these two lambda invoke action steps right next to each other, the first correctly treated as a 'Pre' step but not the second one, that was defined as a 'Post' step.Source code where I define my pipeline:import { AwsInfrastructureStage } from "./stage";import { getEnv } from "../config/env-config";import * as cdk from "aws-cdk-lib";import {  CodePipeline,  CodePipelineSource,  ManualApprovalStep,  ShellStep,} from "aws-cdk-lib/pipelines";import { Construct } from "constructs";import { DeploymentStack } from './stacks/deployment-satck';import { LambdaInvokeStep } from "./lambdaInvokeStep";export class AwsInfrastructureStack extends cdk.Stack {  constructor(scope: Construct, id: string, props?: cdk.StackProps) {    super(scope, id, props);    const deploymentStack = new DeploymentStack(this, 'DeploymentStack', { env: props?.env });    const pipeline = new CodePipeline(this, 'Pipeline', {      pipelineName: 'AwsInfrastructurePipeline',      synth: new ShellStep('Synth', {        input: CodePipelineSource.gitHub('NicolasGorga/aws-infrastructure', 'develop'),        commands: ['npm ci', 'npm run build', 'npx cdk synth']      }),      crossAccountKeys: true,    });    let envConfig = getEnv("dev");    const blockIotDevicesDevStep = new LambdaInvokeStep(`${envConfig.env}BlockIotDevicesStep`, {      actionName: `${envConfig.env}BlockIotDevices`,      lambda: deploymentStack.blockDevicesLambda,      userParameters: {        blockIotDevicesCloudFunction: envConfig.blockIotDevicesCloudFunction,        block: true,        tokenSecretName: "MAINTENANCE_ACTION_TOKEN",        stageEnv: envConfig.env,      },    });    const unblockIotDevicesDevStep = new LambdaInvokeStep(`${envConfig.env}UnblockIotDevicesStep`, {      actionName: `${envConfig.env}UnblockIotDevices`,      lambda: deploymentStack.blockDevicesLambda,      userParameters: {        blockIotDevicesCloudFunction: envConfig.blockIotDevicesCloudFunction,        block: false,        tokenSecretName: "MAINTENANCE_ACTION_TOKEN",        stageEnv: envConfig.env,      },    });    const manualApprovalStep = new ManualApprovalStep("Approval before prod deployment");    const devStage = pipeline.addStage(      new AwsInfrastructureStage(this, envConfig.env, {        env: {          account: envConfig.accountId,          region: envConfig.region,        },        envConfig      }),      {        pre: [blockIotDevicesDevStep],        post: [unblockIotDevicesDevStep, manualApprovalStep]      }    );    envConfig = getEnv('prod');    const blockIotDevicesProdStep = new LambdaInvokeStep(`${envConfig.env}BlockIotDevicesStep`, {      actionName: `${envConfig.env}BlockIotDevices`,      lambda: deploymentStack.blockDevicesLambda,      userParameters: {        blockIotDevicesCloudFunction: envConfig.blockIotDevicesCloudFunction,        block: true,        tokenSecretName: "MAINTENANCE_ACTION_TOKEN",        stageEnv: envConfig.env,      },    });    const unblockIotDevicesProdStep = new LambdaInvokeStep(`${envConfig.env}UnblockIotDevicesStep`, {      actionName: `${envConfig.env}UnblockIotDevices`,      lambda: deploymentStack.blockDevicesLambda,      userParameters: {        blockIotDevicesCloudFunction: envConfig.blockIotDevicesCloudFunction,        block: false,        tokenSecretName: "MAINTENANCE_ACTION_TOKEN",        stageEnv: envConfig.env,      },    });    pipeline.addStage(      new AwsInfrastructureStage(this, envConfig.env, {        env: {          account: envConfig.accountId,          region: envConfig.region,        },        envConfig      }), {        pre: [blockIotDevicesProdStep],        post: [unblockIotDevicesProdStep],      }    );  }}Source Code where I define the custom LambdaInvokeStep:import * as pipelines from "aws-cdk-lib/pipelines";import { IStage } from "aws-cdk-lib/aws-codepipeline";import { LambdaInvokeAction, LambdaInvokeActionProps } from "aws-cdk-lib/aws-codepipeline-actions";export class LambdaInvokeStep extends pipelines.Step implements pipelines.ICodePipelineActionFactory {    constructor(        id: string,        private readonly invokeActionProps: LambdaInvokeActionProps,    ) {        super(id);    }        produceAction(stage: IStage, options: pipelines.ProduceActionOptions): pipelines.CodePipelineActionFactoryResult {        stage.addAction(new LambdaInvokeAction(this.invokeActionProps));        return { runOrdersConsumed: 1 };    }}I have tried everything but the LambdaInvokeStep that is supposed to be a 'Post' step gets always added as a 'Pre' step to be executed in parallel with the LambdaInvokeStep that is supposed to be the only 'Pre' step.Thanks in advance, really looking forward for any input you guys have!!Enter image description hereFollowComment"


Question: I want to create a folder in the shared top directory of FSx.

"I want to create a folder in the shared top directory of FSx.Flow Powershell commands using ssm's AWS-RunPowerShellScript.However, the executing user is ssm-user.This user is not an active directory user and access is denied.The New-Item command does not set credentials.Is it possible to create a folder in the FSx shared top directory by running the powershell command from ssm?FollowComment"

Answer: "1A similar question was asked here and has two different approaches on how to solve issues like this.Fetching AD User credentials from Secrets ManagerGranting the AD Computer object permission to execute the commandSee the answer section for more details.https://repost.aws/questions/QU0nNAbP6mTNiszw8JqzX1rQ/about-the-user-who-executes-ssmCommentShareJoeD_AWSanswered a year agoAWS-User-0207697 a year agoThank you for your response.The New-Item command does not allow optional credentials.Therefore, I would like to give permissions to the computer object.But I don't know how to do that.If you know how to do that, please let me know.Thank you so much.Share"


Question: Create a Glue table (Presto/Athena view) for analyzing ALB logs using CloudFormation or Terraform

"I am trying to create a Glue/Athena table using CloudFormation or Terraform for analyzing ALB logs. I can easily do it using AWS Console following this guide https://docs.aws.amazon.com/athena/latest/ug/application-load-balancer-logs.htmlHowever I can't find any recipe how to do it using CloudFormation or Terraform. I was trying to follow this example ( https://stackoverflow.com/a/56347331 ) but it just creates a Presto view, not using any regex to parse anything in S3 bucket.How can I achieve it? Could anyone please point some example or documentation?FollowCommentRiJo a year agoHi Aleksei,See if these templates could be of any help to you. You definitely need to customize it for your requirement.https://github.com/garystafford/athena-glue-quicksight-demo/blob/master/cloudformation/smart-hub-athena-glue.ymlhttps://gist.github.com/gcchaan/bd4f74034aa2e10b869dc7653cba2807ThanksRimpal JohalShare"

Answer: "0You could start from the Athena Cloudformation documentation page.Look at the Named Query  construct, there is also an example.If you are looking to just create the same table as for the link you mentioned in the question it should look like the code snippet below.You can also see the cloudformation template from the Athena Workshop to see how the Named Query construct has been used in it.Hope this helps.AthenaNamedQuery:    Type: AWS::Athena::NamedQuery    Properties:      Database: "swfnetadata"      Description: "A query that selects all aggregated data"      Name: "MostExpensiveWorkflow"      QueryString: >                    CREATE EXTERNAL TABLE IF NOT EXISTS alb_logs (                                type string,                                time string,                                elb string,                                client_ip string,                                client_port int,                                target_ip string,                                target_port int,                                request_processing_time double,                                target_processing_time double,                                response_processing_time double,                                elb_status_code int,                                target_status_code string,                                received_bytes bigint,                                sent_bytes bigint,                                request_verb string,                                request_url string,                                request_proto string,                                user_agent string,                                ssl_cipher string,                                ssl_protocol string,                                target_group_arn string,                                trace_id string,                                domain_name string,                                chosen_cert_arn string,                                matched_rule_priority string,                                request_creation_time string,                                actions_executed string,                                redirect_url string,                                lambda_error_reason string,                                target_port_list string,                                target_status_code_list string,                                classification string,                                classification_reason string                                )                                ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'                                WITH SERDEPROPERTIES (                                'serialization.format' = '1',                                'input.regex' =                             '([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*):([0-9]*) ([^ ]*)[:-]([0-9]*) ([-.0-9]*) ([-.0-9]*) ([-.0-9]*) (|[-0-9]*) (-|[-0-9]*) ([-0-9]*) ([-0-9]*) \"([^ ]*) (.*) (- |[^ ]*)\" \"([^\"]*)\" ([A-Z0-9-_]+) ([A-Za-z0-9.-]*) ([^ ]*) \"([^\"]*)\" \"([^\"]*)\" \"([^\"]*)\" ([-.0-9]*) ([^ ]*) \"([^\"]*)\" \"([^\"]*)\" \"([^ ]*)\" \"([^\s]+?)\" \"([^\s]+)\" \"([^ ]*)\" \"([^ ]*)\"')            LOCATION 's3://your-alb-logs-directory/AWSLogs/<ACCOUNT-ID>/elasticloadbalancing/<REGION>/';CommentShareEXPERTFabrizio@AWSanswered a year ago"


Question: Urgent! Amplify studio is not opening in the AWS Console

"Hello AWS Team,I am building an application in the AWS Amplify studio.But now the Amplify Studio is not opening in the AWS console.I tried everything from clearing the cache to browser change but nothing else.Whatever I have done is not coming. I am not able to work anymore.Here is the error from browser console. I am not sure if it relevant.  But seems like because of this error, I am not able to open the studio.Uncaught TypeError: Cannot read properties of undefined (reading 'githubToken')    at F.render (main.js:1:4215133)    at zo (vendors~main.js:59:71428)    at Uo (vendors~main.js:59:71227)    at zs (vendors~main.js:59:112369)    at ku (vendors~main.js:59:98325)    at Cu (vendors~main.js:59:98253)    at Su (vendors~main.js:59:98116)    at mu (vendors~main.js:59:95103)    at vendors~main.js:59:44772    at t.unstable_runWithPriority (vendors~main.js:67:3768)Image/Screenshot:https://ibb.co/k1h7nnVFollowComment"

Answer: "0HelloPlease check if you have any network issues, If you do not have network issues try accessing from AWS CLI,Keep an eye on AWS status https://status.aws.amazon.com/ if everything is fine try opening AWS Support ticket AWS premier support can help you.Thank YouGKCommentShareGKanswered a year ago"


Question: How to revoke permission to a VPC Endpoint Service and/or force deletion of a VPC Endpoint Service when there is an inbound VPC Endpoint from another account?

"Currently I am unable to delete a VPC Endpoint Service due to an inbound VPC Endpoint from another account which I don't control.Is there any way to a) revoke the Endpoint permissions and make the Endpoint connection go away, or b) force deletion of the VPC Endpoint Service in my account?FollowComment"

Answer: "0Accepted AnswerSelf-answering this one... just realized you can 'Reject' a VPC Endpoint even after it's been accepted.CommentShareAlexRanswered 6 months agomasuther 3 months agoJust a clarification for anyone in the future who is reading this -- even after "Rejecting" all connections in the AWS console, it will still not allow you to delete the VPC Endpoint Service.In order to forcibly delete the VPC Endpoint Service, you will need to use the AWS CLI with the command aws ec2 delete-vpc-endpoint-service-configurations --service-ids <service_id_to_delete>.Share"


Question: What trust role/policy to assign to create a lambda Post Confirmation Trigger in Cognito?

"Hello,I have an existing cognito user pool. I want a lambda function to execute when a user signs up and confirms their email address. For this, I have used the PostConfirmation lambda trigger.However, I am seeing a Role Exception on trying to create this trigger, both through serverless framework and through the AWS console.Following is the exception:Could somebody please help with what Role/Policy needs to be attached in order to create the trigger? I have read this documentation Update to IAM Role Trust Policy Behavior too but no luck.I have followed this documentation to create the lambda function with the post confirmation trigger in serverless: https://www.serverless.com/framework/docs/providers/aws/events/cognito-user-pool#using-existing-poolsI have also tried assigning the following role to the lambda function:Resources:    CognitoRole:      Type: AWS::IAM::Role      Properties:        RoleName: CognitoSignupRole                AssumeRolePolicyDocument:          Version: '2012-10-17'          Statement:            - Effect: "Allow"              Principal:                Service: "cognito-idp.amazonaws.com"              Action: "sts:AssumeRole"However, this throws an error stating:The role defined for the function cannot be assumed by Lambda.What trust policy needs to be assigned here so the post confirmation trigger gets created?I have already created the following trust policies while creating the user pool:const unauthenticatedRole = new iam.Role(      this,      'CognitoDefaultUnauthenticatedRole',      {        assumedBy: new iam.FederatedPrincipal(          'cognito-identity.amazonaws.com',          {            StringEquals: {              'cognito-identity.amazonaws.com:aud': identityPool.ref,            },            'ForAnyValue:StringLike': {              'cognito-identity.amazonaws.com:amr': 'unauthenticated',            },          },          'sts:AssumeRoleWithWebIdentity'        ),      }    );    unauthenticatedRole.addToPolicy(      new iam.PolicyStatement({        effect: iam.Effect.ALLOW,        actions: ['mobileanalytics:PutEvents', 'cognito-sync:*'],        resources: ['*'],      })    );    const authenticatedRole = new iam.Role(      this,      'CognitoDefaultAuthenticatedRole',      {        assumedBy: new iam.FederatedPrincipal(          'cognito-identity.amazonaws.com',          {            StringEquals: {              'cognito-identity.amazonaws.com:aud': identityPool.ref,            },            'ForAnyValue:StringLike': {              'cognito-identity.amazonaws.com:amr': 'authenticated',            },          },          'sts:AssumeRoleWithWebId)FollowComment"

Answer: "1Since it is the users registered in the user pool that invoke Lambda, the following trust policy may be necessary.https://docs.aws.amazon.com/ja_jp/cognito/latest/developerguide/role-trust-and-permissions.htmlCommentShareEXPERTRiku_Kobayashianswered 2 months agoshwetajoshi0654423 2 months agoHello,Thank you so much for your response. I have already assigned this trust policy to the User Pool while creating it. I am not sure why this error is still being thrown.I am editing the question to include the Cognito User Pool policies for authenticated and unauthenticated users.Share"


Question: DynamoDB On Demand Provisioned Cost Optimization

"Customer reached out looking for options to cut their monthly Dynamo spend. Majority of their spend is WriteRequestUnits and I recommended (if it fit with their usage patterns) consider provisioned capacity, enable auto-scaling, then consider a capacity reservation.Customer feels that their workload is too unpredictable and they have varying throughput needs so provisioned might be more expensive than on-demand.What guidance can we provide customers for cost savings using on-demand capacity?They mentioned deleting PITR for non-critical tables, possibly using AWS Backup to story last 2-3 days of data vs PITR which saves 30 days of data but that seems to be limited in terms of savings.Thanks!FollowComment"

Answer: "0Accepted AnswerProvisioned capacity mode is an option when you have predictable application traffic and you use the autoscaling to adjust the capacity in response to changing traffic. However you will get throttled if the request rate exceeds the provisioned capacity. Note that it is very unlikely to operate at 100% utilization of the provisioned capacity to be cost efficient. You can use the ReturnConsumedCapacity parametercin the request to see in the response exactly how much capacity the operation consumed.Given that the customer has unpredictable request pattern, the On-Demand capacity mode will be more appropriate. And note that a newly created table with On-Demand capacity mode can serve up to 4000 write request units or 12000 read request units or a combination of the two. With On-Demand capacity mode, the requests can burst up to double the previous peak on the table. Note that throttling can occur if the requests exceed double the previous peak within 30 mins. It may be possible that the traffic rate may be different and if your spikes are too high (more than double) than the default capacity or the previously achieved peak request rate, then one solution is to go with pre-warming the tables to the expected peak capacity as per the spike you anticipate.Pre-warming the table can be achieved by following steps when you currently on On-Demand capacity mode.Switch to Provisioned Capacity ModeWait for 24 hours(You can switch between the capacity modes once every 24 hours)Set the Write throughput to the desired peak value or 40000. [ You will incur cost of this high value of throughput until you switch back to On-Demand (few mins) ]Switch to On-Demand capacity mode. This should sustain the provisioned throughput capacity values.Here is a blog that describes how one of our customer was able to achieve a 90% cost saving by opting for On-Demand capacity mode and they too had a spiky workload.CommentShareAWS-User-6438840answered 2 years ago"


Question: Please add TDL ".com.tw" to Route53

"We have a ".com.tw" domain we wish to transfer to Route53, but ".com.tw" does not appear to be in the list of supported top level domains. Would it be possible to add it?The purpose of transferring to route53 is to renew the domain name more efficiently through route53.FollowComment"

Answer: "0Hello,Hello There,Currently, the NEW TLD request are need to create a Feature Request via Customer Support (CS)/ Technical support (TS). I would suggest you to open a case with the CS for the same. There is no ETA as AWS does not publicize roadmap items, but you can check in periodically to see if this domain extension is available by reviewing these AWS documentation pages:Domains that you can register with Amazon Route 53 -- https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/registrar-tld-list.html -- [1]What's New with AWS? -- https://aws.amazon.com/new -- [2]AWS blogs -- https://aws.amazon.com/blogs/aws/  [3]That said, one way to use R53 (for better integration with AWS services, for example) without actually transfering the domain, is to update the name servers in your current registrar with R53 name servers. You can refer to the process here. [4]https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/migrate-dns-domain-in-use.html -- [4]CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 9 months ago"