Question: SES and Cognito timeout on Lambda

"HiI've some problems using the SES and Cognito services in some Lambda Functions (Runtime NodeJs 12.x).A lot of times the lambda go on timeout around 10 times before giving me a successful response.I also use another service (RDS w/AuroraDB) but this one never gave me problems.I think the way I use them is correct because, after the first tries, the lambda reach the end.//SES CODE//DECLARATIONconst ses = new AWS.SES({region: "eu-central-1"});//{...}//SES REQUESTtry{    const sesRes = await ses.sendRawEmail(emailParams).promise(); //<- GET STUCK HERE    console.log();    return {        isBase64Encoded: false,        statusCode: 204,        headers:{            'Access-Control-Allow-Origin' : '*',        },        body: null     };}catch (err){    return {       isBase64Encoded: false,       statusCode: 401,       headers:{           'Access-Control-Allow-Origin' : '*',       },       body: JSON.stringify({           message: "FINAL DB error",           error: err})       };    }}//COGNITO CODE//DECLARATIONconst cognitoPool = new Aws.CognitoIdentityServiceProvider({region: "eu-central-1"});// {..}//COGNITO REQUESTtry{    await cognitoPool.adminCreateUser(params).promise(); //<- GET STUCK HERE    await cognitoPool.adminAddUserToGroup({          GroupName: userGroup,          Username: userInfo.email,          UserPoolId: process.env.USERPOOL,    }).promise();}catch(err){    return {       isBase64Encoded: false,       statusCode: 401,       headers:{            'Access-Control-Allow-Origin' : '*',       },       body: JSON.stringify({             error: err.code,       })     };  }FollowComment"

Answer: "0The problem was the subnet usedCommentSharedogemistanswered 3 years ago0Hi, i´m facing the same problem, what was the problem with the sub net?CommentShareTobbeanswered 3 years ago"


Question: Unable to remove AWS CLI V1 from Windows10 Pro?

"I'm unable to remove aws cli v1 from my Windows10 Pro system. There is no entry in "Programs and Features", so I have no uninstaller to invoke. I've removed it from the all the known disk locations. I've run pip3, and it says that it has no awscli installed.Yet when I invoke aws --version from an (elevated) command prompt, I get this:aws-cli/1.16.309 Python/2.7.18 Windows/10 botocore/1.13.45  I want to do this so that I can install and run V2. I jettisoned Python 2.7x ages ago.What am I missing? How do I get the old version of aws-cli off my system?FollowComment"

Answer: 0Sigh -- I found the answer. I'm leaving this here in case it bites anybody else.I apparently installed this forever ago using pip (instead of pip3).I therefore needed to run pip uninstall awscli in order to remove it.CommentShareSomervilleTomanswered 2 years ago


Question: SecurityHub member accounts stuck in "Enabling in process" for over a day

"I delegated management responsibility of Security Hub from my Control Tower management account to another account and, using a user with admin privileges in that account, enabled Security Hub, auto-enrolled new accounts, and clicked "Enroll" to enroll all existing accounts in my organization. For over 24 hours, all of those accounts have been stuck in a "Enabling in process" status. Is there anything I can do to resolve this? I feel like it shouldn't take this long to enroll member accounts.FollowCommentEly_K a year agoWhat region were you seeing this behavior in? If you have an AWS Support account, filing a ticket would likely be the best bet here. This isn't expected behavior.Share"

Answer: 0Do you have any Service Control Policies (SCPs) that may be blocking that action? Can you make sure that the prerequisites are met that are mentioned within the below link:https://docs.aws.amazon.com/securityhub/latest/userguide/designate-orgs-admin-account.htmlCommentSharebullittbirantanswered 7 months ago


Question: I have questions on Kubernetes ENI allocation

"Via https://github.com/aws/amazon-vpc-cni-k8s, I found how ENI allocation based on instance type.It described it by using m4.4xlarge as an example.For example, a m4.4xlarge node can have up to 8 ENIs, and each ENI can have up to 30 IP addresses. ( https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html ).If the number of current running Pods is between 0 to 29, ipamD will allocate one more eni. And Warm-Pool size is 2 eni * (30 -1) = 58If the number of current running Pods is between 30 and 58, ipamD will allocate 2 more eni. And Warm-Pool size is 3 eni * (30 -1) = 87I did calculate it further below. Please correct me if I am wrong.Q1) m4.4xlarge will have maximum pods will be 203 and Warm-pool will have 232.Is that right?Q2) If the pods will be over 203, what will be happened? Does new instance(m4.4xlarge) will be launching?Q3) Why this calculation get started from '0(zero)' pod? pleasePodsWarm-PoolCalculation0 - 29582eni * (30 - 1)30 - 58873eni * (30 - 1)...146 - 1742037eni*(30 - 1)175 - 2032328eni * (30 - 1)Thank youFollowComment"

Answer: "0Accepted AnswerIt is now possible to have greater control over how many ENI's are grabbed by the CNI.On the other hand, Kubernetes itself has a supported limit of 100 pods per node. Making some of the larger instances with lots of available addresses less attractive. While the pod per node limit IS configurable, I would not increase it without a very good reason.  This implies that the best instances sizes are between 2xlarge-4xlarge, at least in terms of address allocation. Larger sizes may be better in terms performance, but you will not get any more useful addresses.https://kubernetes.io/docs/setup/cluster-large/No more than 100 pods per nodeQ1) a m4.4xlarge node can have up to 8 ENIs, and each ENI can have up to 30 IP addresses. ( https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html ). This is 240 addresses maximum for pods. The warm pool is controlled by "WARM_ENI_TARGET", which is defaults to 1. This will imply 30 addresses in the warm pool (the number available to an ENI). However, at maximum addresses, there will be no warm pool left. The warm pool target is configurable by en ENV variable, and the algorithm has been tweaked recently. I would test this out to verify the actual numbers if there is concern.Q2) Nothing is done automatically.  You can run out of addresses in a K8 cluster.  You may want to look into "cluster autoscaling" which will launch new hosts upon address exhaustion (or pod placement failure to be more accurate).Q3) One address allocated to the ENI is considered the Primary, and it used for routing traffic out of the worker node. This is why you lose an address per ENI.CommentShareMODERATORAWS-User-7304085answered 4 years ago"


Question: head_object request on S3 object after restore still shows old storage class

I'm trying to dynamically determine the storage class of an object that was restored from GLACIER to STANDARD. But when i make this request from boto3's head_object i still keep getting the old storage class of the object. I've verified the restore completion from the url (version:null)FollowComment

Answer: "0When you restore an object to from Glacier Flexible Retrieval (GFC) or Glacier Deep Archive (GDA) to S3 Standard, a temporary copy is stored in S3 standard and will remain there for the number of days you specified when you restored the object. The storage class will remain in Glacier, but will be made available for you to GET as you would any other S3 object. To determine if an object stored is available for GET using the S3 api, you can use the head-object api call which will tell you if the restore is on-going and when the restored object will expire out of S3 Standard. If there is no expiry time, the object must first be restored.Restoring an archived objectCommentSharesigtrap5answered 8 months ago"


Question: SageMaker Serverless Inference metrics not recorded/displayed

Sagemaker serverless inference metrics are not shown in the UI:The container is huge 103860px in hight and if scrolled down it shows "No widget on this dashboard."Data appears in cloudwatch though.Seems to be an issue on AWS side here...FollowComment

Answer: "0Hi Felix, can you check if you're using the right time range?Also, in your second screenshot, I'm seeing 7 metrics, can you try selecting those metrics and see if your graph is populated? See before and after choosing the metrics on CW -CommentShareDurga_Sanswered 23 days agoFelix 21 days agoOh my bad cloudwatch looks fine. The dashboard in sagemaker itself does not show anything though: If I scroll down I only see "No widget on this dashboard." (updated original question)Share"


Question: Greengrass for data processing and ML model training

Is it possible to train and deploy ML models in Greengrass? Or is Greengrass limited to inference while training is done using SageMaker in cloud?FollowComment

Answer: "0Accepted AnswerAs a native service offering, Greengrass has support for deploying models to the edge and running inference code against those models. Nothing prevents you from deploying your own code to the edge that would train a model, but I suspect you wouldn't be able to store it as a Greengrass local resource for later inferences without doing a round trip to the cloud and redeploying to GG.CommentShareRyan_Banswered 4 years ago"


Question: Pausing a server

"Hi,We're trying to launch a Collibra DQ instance via the following link:https://aws.amazon.com/marketplace/pp/prodview-ejyn7foj6lsnu#pdp-supportNow I was wondering something. Is it possible that after launching, we can pause the server when it is not being used. So that we can limit the costs of launching such an instance? Because the way I see it, you'll pay for the idle hours just as much as the running hours.Thanks in advance!FollowComment"

Answer: "0Accepted AnswerYes, you can stop and start your instance. When it is stopped, you are not paying for compute. You will still have a cost for the EBS volumes.You can also automate this using the Instance Scheduler on AWS.CommentShareEXPERTkentradanswered 3 months agorePost-User-8875557 3 months agoThanks! One follow up: Does it store everything when going offline, for example does it keep all the files and data. Or do we have to restore this when starting the instance up again.Sharekentrad EXPERT3 months agoIf you are using EBS volumes (most likely), everything is preserved.SharerePost-User-8875557 3 months agoThanks, sounds perfect!Share"


Question: Why is PSI not offering AWS Sys Ops Associate certification?

Why is PSI not offering AWS Sys Ops Associate certification? I would like to take this cert and looks like only Pearson VUE is offering this. I would prefer PSI over PearsonVUE anyday.Thank youFollowComment

Answer: "0Most likely because that exam contains practical part, where you need to use provided AWS account, so I suppose that  PSI doesn't have tech possibility for that.CommentShareA-Shevchenkoanswered a year ago"


Question: Public Endpoint IP addresses static or dynamic?

"I was asked to provide the IP address of my Transfer Family SFTP ( public endpoint ) so that on their side they can make an outbound rule in their firewall.I thought this IP address was dynamically assigned to the endpoint, since i see many mentions that for whitelisting its required a static IP (EIP).If its dynamic is it possible to determine a IP range (that was their suggestion), without having to use a static IP (EIP)?ThanksEdit: whatismyipaddress.com mentions that is likely a static IPFollowComment"

Answer: "1When you create SFTP endpoint into VPC I believe you must provide EIP(s) for it. Details are shown in this blog post. https://aws.amazon.com/blogs/storage/use-ip-whitelisting-to-secure-your-aws-transfer-for-sftp-servers/CommentShareKalluanswered a year agodotmindlabs a year agoHi, thanks. I was trying to avoid to have to redeploy the cloudformation, since originally it seems to have been deployed as Public Endpoint type. I get 3 different addresses from nslookup, but i am afraid if i provide them with this (or even a range/subnet) they can be altered anytime in the future. So this means that for customers to be able to whitelist, the only solution seems to be with VPC if i undestood correctly.ShareKallu a year agoYou're right. Public endpoint IPs can change. Here is the summary of different endpoint types.https://aws.amazon.com/premiumsupport/knowledge-center/aws-sftp-endpoint-type/Share1Can you use a VPC endpoint with internet-facing access?According to the above blog, if you choose a VPC endpoint with internet-facing access you can attach Elastic IP addresses to the endpoint. These can be AWS-owned IP addresses or your own IP addresses (BYOIP). Elastic IP addresses attached to the endpoint don't change.https://docs.aws.amazon.com/transfer/latest/userguide/create-server-in-vpc.html#create-internet-facing-endpointCommentShareDennis_Aanswered a year ago"


Question: AWS Instance Scheduler

"Hello everyone!Hope you can help me with customizing a schedule on AWS Instance Scheduler.I successfully configured an AWS Instance Scheduler to start and stop my instances on Monday to Friday from 7:00 AM to 8:00 PM and this scheduler is working as expected.I currently have 5 instances provisioned and tagged to successfully run and stop based on the schedule mentioned above, but one from these 5 instances is required to also run during weekends on a different hour schedule e.g. Saturday from 7:00 AM to 5:00 PM and Sunday from 10:00 AM to 5:00 PM.I tried modify this from DynamoDB by creating another item under ConfigTable e.g. Weekend1 with period 07:00 to 17:00 and Weekend2 with period 10:00 to 17:00 and tagged the specific instance that will run on weekends but it does not run the tagged instance based on the weekend schedule.Hoping you can guide me on the right way to configure a different schedule.Thank you.FollowComment"

Answer: "0Hello Bdacebedo,I understand that you want to schedule the start and stop of your instances but there is one instance that you want to start/stop on weekends. You are currently using the AWS instance scheduler and it is working perfectly fine. However, you would like to customize it based on your scenario.You are in the right direction of creating a custom schedule in AWS instance scheduler[1]. To add a schedule period for your instance you will modify the DynamoDB table and create two more period for your schedule [2]. Below is a sample JSON code that can be used to set schedule period  for Saturday from 7:00 AM to 5:00 PM:{  "type": {    "S": "period"  },  "name": {    "S": "sat-7-5"  },  "begintime": {    "S": "7:00"  },  "endtime": {    "S": "16:59"  },  "weekdays": {    "SS": [      "sat"    ]  }}You will do the same when creating the period for Sunday. Then modify your schedule and add the periods you created. For the period described above the schedule will look as follows:{  "type": {    "S": "schedule"  },  "name": {    "S": "schedule-name"  },  "timezone": {    "S": "UTC"  },  "periods": {    "SS": [      "sat-7-5"    ]  }}When more than one period is used in a schedule, the Instance Scheduler will apply the appropriate start action when at least one of the period rules is true. For more information, refer to Period Rules.References:How to use Instance SchedulerSchedule definitionsCommentShareCebianswered a year ago"


Question: Amazon CloudFront - Collapsed request answered with Cache-Control: no-store

"Scenario:There has been no previous requests on an object.CloudFront behavior is default (CacheOptimized).Several requests enough to make request collapsing reached to one Edge LocationOrigin server responded with Cache-Control: no-storeHow does Amazon CloudFront react?I think it will send all other requests to origin, but wanted to confirm.FollowComment"

Answer: 0Accepted AnswerCloudFront will collapse the request for traffic spike scenario regardless of no-store and/or no-cache directive.https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/RequestAndResponseBehaviorCustomOrigin.html#request-custom-traffic-spikesIn the logs you will see something like "NonS3CollapsedHit" for something that's normally a MISS/uncacheable.CommentShareEXPERTEJanswered 3 years ago


Question: Why Cost intelligence dashboard is only visible to owner not other users ?

"Hi,I have created CID,CUDO and KPI dashbaords and all 3 are working well. But as an owner only i can able to see all of them in dashbaord section. CID dashboard is not visible to users like admin,reader and i'm unable to debug. Please help.Thanks!FollowComment"

Answer: "1I can share the dashboard with other users, etc. as per this document, have you completed the setup?https://docs.aws.amazon.com/quicksight/latest/user/sharing-a-dashboard.htmlCommentShareEXPERTRiku_Kobayashianswered 19 days agorePost-User-3825728 18 days agoThanks Riku. Sharing worked for this.ShareRiku_Kobayashi EXPERT18 days agoIf this answer leads to a resolution, please approve the answer for the betterment of the community.Sharerathmyle 17 days agoIn the FAQ section of the instructions for CID Dashboard deployments on AWS Well-Architected Labs site, there are 2 links for instructions on how to share your dashboards.   https://wellarchitectedlabs.com/cost/200_labs/200_cloud_intelligence/faq/"You can share dashboards and visuals with specific users or groups in your account or with everyone in your Amazon QuickSight account. Or you share them with anyone on the internet. You can share dashboards and visuals by using the QuickSight console or the QuickSight API. Access to a shared visual depends on the sharing settings that are configured for the dashboard that the visual belongs to. To share and embed visuals to your website or application, adjust the sharing settings of the dashboard that it belongs to. For more information, see the following:"Granting individual Amazon QuickSight users and groups access to a dashboard in Amazon QuickSightGranting everyone in your Amazon QuickSight account access to a dashboardShare"


Question: Why doesn't cloudformation support ZipFile for golang lambdas?

"It's really painful that CloudFormation won't let me declare a lambda as a ZipFile:Resource handler returned message: "Invalid request provided: The ZipFile attribute is not supported for the runtime 'go1.x'"Why should it care that the content of the .zip is go?  It's just an executable.The problem is there's no clean workaround.  I was going down a path of uploading the .zip separately but the problem there is if the filename (s3 key) doesn't change then CloudFormation won't think it needs an update.  I can generate a random file name (s3 key) every time but that has to be passed into using --parameter-overrides which also is a little messy.  It's sort of compounded by the problem that the aws sdk for go has no support for packaging stacks directly from go.I guess this is the way it is, but go definitely is a second class citizen here, so my main question is will it always be this way?FollowComment"

Answer: "0The ZipFile attribute in CloudFormation only works with scripted language runtimes nodejs and python, since CloudFormation does not / cannot do the "compile" step for you. The clean way is to use S3Key as you already stated corectly. And then specify new versions with S3ObjectVersion.AWS SAM automates these steps for you. Another tool worth a look for compiling/handling Lambdas with CloudFormation is the CDK (golang alpha module).CommentShareSoenke Ruempleranswered a year ago"


Question: Customer with Multiple Users Needs Concurrent Access to Application and Database

"Customer who is on-prem and also working with Azure to host some Windows 10 remote desktop workloads.They were very interested in Workspaces, especially as it works very well with Active Directory.They wanted to know if Workspaces has a similar capabilities as Windows 10 Enterprise multi-session - it seems like it allows for concurrent sessions with several users:https://docs.microsoft.com/en-us/azure/virtual-desktop/windows-10-multisession-faqThey are looking to be able to host a single MS SQL database as well as a 3rd party application, and access it via remote desktops for each of their 10-15 users.However, they seemed averse to hosting it on an EC2 or RDS instance and using RDP to access it via WorkSpaces - specifically they "don't want to use a terminal server", and would rather stream the application to their users which then accesses the database (I'm not sure where the database would be, it seems like it would have to be hosted somewhere though).I was considering AppStream 2.0 to be the best service to turn to for this use case, but was wondering if there is a better solution, possibly more in line with what they are looking at with Windows 10 Enterprise multi-session.FollowComment"

Answer: "0Accepted AnswerWorkSpaces are mapped 1:1 to users. Each WorkSpace provides a persistent Desktop to a single user (think of it like your personal cloud laptop in AWS.. your data and user profile persists).AppStream still has the 1:1 mapping between streaming instances and users. However, it's focused on delivering applications to users and offers some additional features (for example, SAML federation which means users can log in via Okta, Azure AD, etc.). AppStream is also generally a bit more elastic in its pricing since you're paying per hour each streaming instance is running and you can autoscale your fleet capacity (whereas WorkSpaces requires some upfront commitment per user). Another important consideration is that AppStream applications must support a Windows Server environment.Bottom line: if the customer is looking for a complete Desktop experience, look at WorkSpaces. If they're interested in delivering a single application to users and that application can run on Windows Server, look at AppStream.CommentShareEXPERTSpencerAtAWSanswered 3 years ago"


Question: Migrate contact flow to another instance

"I am not able to create contact flow because of that below error message. But when i tried to do export and import using AWS connect i was able to save it. I dont want to publish it. I just want to replicate the import functionality using boto3.error message: {'message': 'Action with identifier 1c3f7cf2-c278-495f-bf06-12747abd0b54 has invalid resource: arn:aws:connect:us-west-2::instance//contact-flow/299cdb4c-54de-4e65-bef0-ffe40a29587d, Path: Actions[2]'}desribe_contact_flow:{'Arn': 'arn:aws:connect:us-west-2::instance//contact-flow/98990c59-b1da-46aa-a05f-d4c42ff0d367', 'Id': '98990c59-b1da-46aa-a05f-d4c42ff0d367', 'Name': 'Assigned Prompt Test', 'Type': 'CONTACT_FLOW', 'State': 'ACTIVE', 'Description': 'this is a contact flow with prompt block', 'Content': '{"Version":"2019-10-30","StartAction":"938223b8-9b22-445e-9918-7eb0cb3790ab","Metadata":{"entryPointPosition":{"x":40,"y":40},"ActionMetadata":{"d2a337a4-a222-43f6-9404-b182a1406f45":{"position":{"x":702.4,"y":203.2}},"938223b8-9b22-445e-9918-7eb0cb3790ab":{"position":{"x":244,"y":76.8},"parameters":{"PromptId":{"displayName":"Beep.wav"}},"promptName":"Beep.wav"},"1c3f7cf2-c278-495f-bf06-12747abd0b54":{"position":{"x":460.8,"y":335.2},"parameters":{"ContactFlowId":{"displayName":"hls-prompt2_Copy"}},"ContactFlow":{"text":"hls-prompt2_Copy"}}}},"Actions":[{"Parameters":{},"Identifier":"d2a337a4-a222-43f6-9404-b182a1406f45","Type":"DisconnectParticipant","Transitions":{}},{"Parameters":{"PromptId":"arn:aws:connect:us-west-2::instance/prompt/45543cca-b8fa-4f03-93ba-b47e7de96ae3"},"Identifier":"938223b8-9b22-445e-9918-7eb0cb3790ab","Type":"MessageParticipant","Transitions":{"NextAction":"d2a337a4-a222-43f6-9404-b182a1406f45","Errors":[{"NextAction":"1c3f7cf2-c278-495f-bf06-12747abd0b54","ErrorType":"NoMatchingError"}]}},{"Parameters":{"ContactFlowId":"arn:aws:connect:us-west-2::instance//contact-flow/299cdb4c-54de-4e65-bef0-ffe40a29587d"},"Identifier":"1c3f7cf2-c278-495f-bf06-12747abd0b54","Type":"TransferToFlow","Transitions":{"NextAction":"d2a337a4-a222-43f6-9404-b182a1406f45","Errors":[{"NextAction":"d2a337a4-a222-43f6-9404-b182a1406f45","ErrorType":"NoMatchingError"}]}}]}', 'Tags': {}}FollowComment"

Answer: "0Describe-Contact-Flow will throw an exception when ran against an un-published contact flow. You can publish the flow with a new name and retain the saved flow, then run DeleteContactFlow on the published version after your export/imports are complete.CommentShareawslewisanswered 10 days agorePost-User-5070253 10 days agoHi Awslewis, thank you for answering this question. However my goal is to replicate the save functionality in amazon connect UI in api using boto3. Like for instance the contact flow still has errors and not ready to be publish, I want the user to have the capability to save it. Right now, i only see create_contact_flow in boto3, this method automatically publish the contact flow. I am looking for a solution that to save. Hope it makes senseShareawslewis 10 days agoThe API actions specific to Contact Flows (DescribeContactFlow, UpdateContactFlowContent, CreateContactFlow, etc) do not currently support Saved state contact flows, however, there are plans to introduce this support in the future.Share"


Question: "Cloudformation: update stack error  Service: ElastiCache, Status Code: 400"

"Hi,Im using cdk to to create a stack with a redis cluster       const subnets = vpc.privateSubnets.map(subnet => subnet.subnetId);       const subnetGroups = new CfnSubnetGroup(            this,            'redisSubnets',            {                description: 'redis-subnets-${environmentName}',                subnetIds: subnets            }        );        new RedisLogs(            this,            'redisClusterLogs',            environmentName        );        const logsDeliveryConfiguration: CfnCacheCluster.LogDeliveryConfigurationRequestProperty = {            logFormat: 'json',            logType: 'slow-log',            destinationDetails: {                cloudWatchLogsDetails: {                    logGroup: `/redis/${environmentName}`                }            },            destinationType: 'cloudwatch-logs'        };        const clusterProps: CfnCacheClusterProps = {            cacheNodeType: 'cache.t3.micro',            engine: 'redis',            numCacheNodes: 1,            autoMinorVersionUpgrade: true,            vpcSecurityGroupIds: [securityGroup.securityGroupId],            cacheSubnetGroupName: subnetGroups.ref,            clusterName: `cluster-${environmentName}`,            logDeliveryConfigurations: [logsDeliveryConfiguration],            engineVersion: "6.2"        };        this.redisCluster = new CfnCacheCluster(            this,            'redis-cluster',            clusterProps        );        this.redisCluster.addDependsOn(subnetGroups)cloudformation fails with this error into resource redisSubnets:Resource handler returned message: "No modifications were requested. (Service: ElastiCache, Status Code: 400, Request ID: xxxxxxxx)" (RequestToken: xxxxxxx, HandlerErrorCode: GeneralServiceException)Im using the same code for develop, certification environments, only in test environment causes this errorThis stack is in a bigger stack with several nested stacks like ecs, rds and another services, curiously, redis stack not have any changesFollowComment"

Answer: "0Hello,Greetings of the day!! Thank you for contacting AWS.I understand you are getting the following error "Resource handler returned message: "No modifications were requested." while creating a stack with Redis cluster via CDK and though the code works well for production environment, it is failing for the test environment where no changes were made to the Redis stack.To answer your question, we require details that are non-public information. Please open a support case with AWS using the following link https://console.aws.amazon.com/support/home#/case/createThank you and have a nice day!!CommentShareSUPPORT ENGINEERAditi_Banswered 7 months ago0I had this when I changed subnets in my VPC. I have added new ACL to make my subnets private, so I had to change fromCommentSharecapajanswered 2 months ago"


Question: Pattern for streaming changes from RDS

Is there a best practice/pattern around integrating a central RDS database (postgres) with other systems by streaming its modifications similar to Dynamo DB streams?FollowComment

Answer: 0Accepted AnswerYou can simply integrate database to stream changes to Kinesis of MSK. The best practices would surround across the no. of tables you like to stream and what are the use cases. I have done a similar configuration for making data available to multiple consumers as well as generating real-time alerts for a specific use case. The stream will be near real-time and does not require major code implementation. See the reference architecture : https://aws.amazon.com/blogs/database/use-the-aws-database-migration-service-to-stream-change-data-to-amazon-kinesis-data-streams/CommentShareAWS-User-3964373answered 3 years ago


Question: update glue trigger via CDK code

"Hi team,I'm trying to use CDK customResource to update EventBatchingCondition for a glue trigger( as this is not supported natively by cloudFormation)this my code :    new AwsCustomResource(this, "updateEventBatching", {      policy: AwsCustomResourcePolicy.fromSdkCalls({        resources: AwsCustomResourcePolicy.ANY_RESOURCE,      }),      onCreate: {        service: "Glue",        action: "updateTrigger",        parameters: {          Name: myGlueTrigger.name, //The name of the trigger to update.          TriggerUpdate: {            EventBatchingCondition: {              BatchSize: "20",              BatchWindow: "900",            },          },        },        physicalResourceId: PhysicalResourceId.of(          "updateEventBatching_id"        ),      },      onUpdate: {        service: "Glue",        action: "updateTrigger ",        parameters: {          Name: myGlueTrigger.name, //The name of the trigger to update.          TriggerUpdate: {            EventBatchingCondition: {              BatchSize: "20" ,              BatchWindow: "300",            },          },        },        physicalResourceId: PhysicalResourceId.of("updateEventBatching_id"),      },    });I followed this article to grabe the service name, action, and parameters :https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Glue.html#updateTrigger-propertywhen I try to deploy I have this error :not sure what if it's about the service action is not correct ?also I'm not sure what I should put in physicalResourceId parameter in this case, I just put a static stringnode_modules\aws-cdk-lib\aws-iam\lib\policy-statement.js:1"use strict";var _a;Object.defineProperty(exports,"__esModule",{value:!0}),exports.Effect=exports.PolicyStatement=void 0;const jsiiDeprecationWarnings=require("../../.warnings.jsii.js"),JSII_RTTI_SYMBOL_1=Symbol.for("jsii.rtti"),cdk=require("../../core"),group_1=require("./group"),principals_1=require("./principals"),postprocess_policy_document_1=require("./private/postprocess-policy-document"),util_1=require("./util"),ensureArrayOrUndefined=field=>{if(field!==void 0){if(typeof field!="string"&&!Array.isArray(field))throw new Error("Fields must be either a string or an array of strings");if(Array.isArray(field)&&!!field.find(f=>typeof f!="string"))throw new Error("Fields must be either a string or an array of strings");return Array.isArray(field)?field:[field]}};class PolicyStatement{constructor(props={}){this.action=new Array,this.notAction=new Array,this.principal={},this.notPrincipal={},this.resource=new Array,this.notResource=new Array,this.condition={},this._principals=new Array;try{jsiiDeprecationWarnings.aws_cdk_lib_aws_iam_PolicyStatementProps(props)}catch(error){throw process.env.JSII_DEBUG!=="1"&&error.name==="DeprecationError"&&Error.captureStackTrace(error,this.constructor),error}for(const action of[...props.actions||[],...props.notActions||[]])if(!/^(\*|[a-zA-Z0-9-]+:[a-zA-Z0-9*]+)$/.test(action)&&!cdk.Token.isUnresolved(action))throw new Error(`Action '${action}' is invalid. An action string consists of a service namespace, a colon, and the name of an action. Action names can include wildcards.`);this.sid=props.sid,this.effect=props.effect||Effect.ALLOW,this.addActions(...props.actions||[]),this.addNotActions(...props.notActions||[]),this.addPrincipals(...props.principals||[]),this.addNotPrincipals(...props.notPrincipals||[]),this.addResources(...props.resources||[]),this.addNotResources(...props.notResources||[]),props.conditions!==void 0&&this.addConditions(props.conditions)}static fromJson(obj){const ret=new PolicyStatement({sid:obj.Sid,actions:ensureArrayOrUndefined(obj.Action),resources:ensureArrayOrUndefined(obj.Resource),conditions:obj.Condition,effect:obj.Effect,notActions:ensureArrayOrUndefined(obj.NotAction),notResources:ensureArrayOrUndefined(obj.NotResource),principals:obj.Principal?[new JsonPrincipal(obj.Principal)]:void 0,notPrincipals:obj.NotPrincipal?[new JsonPrincipal(obj.NotPrincipal)]:void 0}),errors=ret.validateForAnyPolicy();if(errors.length>0)throw new Error("Incorrect Policy Statement: "+errors.join(`                                                                                ^Error: Action 'glue:UpdateTrigger ' is invalid. An action string consists of a service namespace, a colon, and the name of an action. Action names can include wildcards.    at new PolicyStatement (C:\xxxx\node_modules\aws-cdk-lib\aws-iam\lib\policy-statement.js:1:1371)    at new AwsCustomResource (C:\xxxx\node_modules\aws-cdk-lib\custom-resources\lib\aws-custom-resource\aws-custom-resource.js:1:4109)    at new CdkGlueEdwLoadStack (C:\xxxxxx\lib\cdk-glue-edw-load-stack.ts:634:5)    at Object.<anonymous> (C:\xxxxx\bin\index.ts:115:1)    at Module._compile (node:internal/modules/cjs/loader:1105:14)    at Module.m._compile (C:\xxxxx\node_modules\ts-node\src\index.ts:1056:23)    at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)    at Object.require.extensions.<computed> [as .ts] (C:\xxxxx\node_modules\ts-node\src\index.ts:1059:12)    at Module.load (node:internal/modules/cjs/loader:981:32)    at Function.Module._load (node:internal/modules/cjs/loader:822:12)FollowComment"

Answer: "0Accepted AnswerI ended up by adding explicitly an I am policy :used AwsCustomResourcePolicy.fromStatements instead of AwsCustomResourcePolicy.fromSdkCallsnew AwsCustomResource(this, "resourceId", {      policy: AwsCustomResourcePolicy.fromStatements([        new iam.PolicyStatement({          effect: iam.Effect.ALLOW,          actions: ["glue:UpdateTrigger"],          resources: [            `myGlueTrigger_ARN`,          ],         }),      ]),      onCreate: {           ......           ......but if someone has any idea why the following doesn't work thanks for sharing :policy: AwsCustomResourcePolicy.fromSdkCalls({        resources: AwsCustomResourcePolicy.ANY_RESOURCE,      }CommentShareJessanswered 10 months ago0I tested the below policy in a Custom Resource sample:policy: AwsCustomResourcePolicy.fromSdkCalls({        resources: AwsCustomResourcePolicy.ANY_RESOURCE,      }The policy worked on my end without any issue.The below error which you were seeingError: Action 'glue:UpdateTrigger ' is invalid. An action string consists of a service namespace, a colon, and the name of an action. Action names can include wildcards.was due to an extra space after 'glue:UpdateTrigger  '. We can see this extra space in the error message as well.I was getting the same error on my end when I added that extra space.policy: cr.AwsCustomResourcePolicy.fromStatements([        new iam.PolicyStatement({          effect: iam.Effect.ALLOW,          actions: ["glue:UpdateTrigger "],............Removing the space should fix the error.CommentShareSUPPORT ENGINEERRuchir_Sanswered 10 months agoEXPERTFabrizio@AWSreviewed 9 months ago"


Question: unable to create eks cluster using REST API

"I need to create a eks cluster using REST API/ansible but ansible community.aws.aws_eks_cluster dosen't have kubernetesNetworkConfig parameter where we can specify serviceIpv4Cidr while creating cluster.I used the below link to create a cluster but unable to pass the authorization credentials in the POST rest api request,https://docs.aws.amazon.com/eks/latest/APIReference/API_CreateCluster.htmluri:url: "https://eks.us-east-1.amazonaws.com/clusters"Have the below auth paramsaws_access_key: "{{ aws_access_key1 }}"aws_secret_key: "{{ aws_secret_key1 }}"X-Amz-Security-Token: "{{ security_token1 }}"Need to use it using the URI ansible module ( need to use in headers: in url for authentication)Have to pass something like below:headers:Authorization: Bearer "{{a}}"Failing with Unable to determine service/operation name to be authorizedAccessDeniedException,FollowComment"

Answer: "1Accepted AnswerHello,When you make API requests to AWS, you have to sign the requests so that AWS can identify who sent them. You will have to sign the requests with your AWS access key (consists of the Access Key ID and Secret Access Key).Signature Version 4 (SigV4) is the process to add authentication information to AWS API requests sent by HTTP(s). Please refer this document for more information on SigV4.Please refer Signing AWS requests with Signature Version 4 documentation to understand how to sign your AWS requests with SigV4.Once the signing process is complete, you'll need to pass your signature in the Authorization header when making the POST /clusters request.Please take a look at this sample python code provided in the AWS documentation that explains how to make a POST request to the DynamoDB API using the above mentioned sigV4 signing.It is recommended to use AWS SDKs for performing API requests to AWS services as they handle the signature calculation process for you, so you do not have to manually complete the signing process.I understand that achieving the above steps through Ansible code is a challenging task. Therefore, I have suggested an alternate approach below.Alternate approach:The eksctl (official CLI for Amazon EKS) can be leveraged in your use-case to perform the cluster creation without the need for the REST API approach.You can create a jinja2 template for the eksctl cluster configuration file and run eksctl create cluster -f cluster.yaml command using the Ansible command module to create your cluster.I hope this helps!CommentShareSUPPORT ENGINEERVenkat Penmetsaanswered 9 months agoEXPERTToni_Sreviewed 9 months agoToni_S EXPERT9 months agoTerraform also has an extensive support for EKS.SharerePost-User-0286276 9 months agoHi @Venkat Penmetsa, Thanks for responding back,i would like to know if we can send a request to aws api services like below to GET/POST request in the below format (curl,GET /resource HTTP/1.1Host: server.example.comAuthorization: Bearer mF_9.B5f-4.1JqMExample GET:curl -H "$(oauth2l header --json /data/xxxx.json cloud-platform userinfo.email)" -H "Content-Type: application/json" "https://compute.googleapis.com/compute/v1/projects/xxxx/regions/us-central1/routers"Could you provide a curl request to aws services (for example to get the list of clusters),is  AWS API requests unfortunately not straight forward to implement in curl? and mandatory to use the AWS SDKs for performing API requests to AWS services and use SigV4?ShareVenkat Penmetsa SUPPORT ENGINEER9 months agoAlthough it is possible to make requests to AWS services without using SDKs, it is difficult to achieve AWS SigV4 authentication with shell commands alone.You can write a custom ansible module that can perform the request signing process. Check out this code (https://github.com/ansible-collections/amazon.aws/blob/main/plugins/module_utils/urls.py) to understand how ansible aws modules perform the SigV4 signing to make API requests to AWS.Share"


Question: Lex Response Cards in Amazon Connect Chat

"Hello;Does amazon connect chat, support Lex response cards?I could not make it work. Is there any specific configuration?Here is my lex response:"dialogAction": {"type": "Close","fulfillmentState": "Fulfilled","message": {"contentType": "PlainText","content": "Here are your appointments"},"responseCard": {"version": 0,"contentType": "application/vnd.amazonaws.card.generic","genericAttachments": [{"title": "Appointment 1","subTitle": "OPEN 30 MIN"}, {"title": "Appointment 2","subTitle": "OPEN 30 MIN"}, {"title": "Appointment 3","subTitle": "Surgery - Confirmed"}, {"title": "Appointment 4","subTitle": "OPEN 30 MIN"}, {"title": "Appointment 5","subTitle": "Surgery - Pre PA/NP Visit"}]}}And this is what I get in chat websocket:{"content": "{"AbsoluteTime":"2021-02-02T11:20:51.138Z","Content":"Here are your appointments","ContentType":"text/plain","Id":"002bda83-00dc-43d1-b4a9-c9a4078ebf22","Type":"MESSAGE","ParticipantId":"6364965d-64ab-4d75-a36c-ad79de978156","DisplayName":"BOT","ParticipantRole":"SYSTEM","InitialContactId":"5fdf5185-8ef4-40ca-a81e-558207be70a0","ContactId":"5fdf5185-8ef4-40ca-a81e-558207be70a0"}","contentType": "application/json","topic": "aws/chat"}ThanksEdited by: bbolek on Feb 2, 2021 3:24 AMEdited by: bbolek on Feb 2, 2021 3:25 AMEdited by: bbolek on Feb 2, 2021 3:27 AMEdited by: bbolek on Feb 8, 2021 1:15 AMIs there any response to this?ThanksFollowComment"

Answer: "1I do not think you can use Lex response cards with the default Connect chat UI. However some interactive messaging templates have recently been released. https://docs.aws.amazon.com/connect/latest/adminguide/interactive-messages.htmlCommentSharerobmooresanswered 2 years ago0Accepted AnswerPlease take a look at this blog article. It details how to achieve this step-by-step: https://aws.amazon.com/blogs/contact-center/easily-set-up-interactive-messages-for-your-amazon-connect-chatbot/CommentShareFernando_Fanswered 2 years ago0Hello,Is there any response to this?ThanksCommentSharebbolekanswered 2 years ago0Thanks man,This resolved my issue.Have a great dayCommentSharebbolekanswered 2 years ago"


Question: Terraform Plan UI & EU-West-1 issues

"Hello,Per the attached, when I ran the terraform plan for AZ EU-West-1, I get that error.Kindly assist.Oluwasegun OjeyinkaFollowComment"

Answer: "1Hi Ojeyinka,how many subnets are you passing in the aws_db_subnet_group resource?Make sure you pass at least 2 subnetsresource "aws_db_subnet_group" "default" {  name       = "main"  subnet_ids = [aws_subnet.frontend.id, aws_subnet.backend.id]  tags = {    Name = "My DB subnet group"  }}Hope it helps, and if it does I 'd appreciate answer to be accepted so that community can benefit for clarity when searching for similar issues, thanks!CommentShareEXPERTalatechanswered 3 months agoOluwasegun 3 months ago@alatech, this is my entire code block RDS subnet script:resource "aws_db_subnet_group" "ACS-rds" {name       = "acs-rds"subnet_ids = var.private_subnetstags = merge(var.tags,{Name = "ACS-database"},)}create the RDS instance with the subnets groupresource "aws_db_instance" "ACS-rds" {allocated_storage      = 20storage_type           = "gp2"engine                 = "mysql"engine_version         = "5.7"instance_class         = "db.t2.micro"name                   = "daviddb"username               = var.db-usernamepassword               = var.db-passwordparameter_group_name   = "default.mysql5.7"db_subnet_group_name   = aws_db_subnet_group.ACS-rds.nameskip_final_snapshot    = truevpc_security_group_ids = var.db-sgmulti_az               = "true"}Sharealatech EXPERT3 months agoHi, what is the content of private_subnets? It must be an array, can you share?Share0I think there is an error in the subnet you are trying to create in the RDS Subnet Group.please check if the subnets configured in the subnet group are 2AZ or more.if it is configured with more than 2AZ, please check if each subnet is in an available AZ.The following command will tell you if it is available or notaws ec2 describe-availability-zones --region eu-west-1CommentSharetakakunianswered 3 months ago0@takakuni, Thank you for the response.This is my entire code blockresource "aws_db_subnet_group" "ACS-rds" {name       = "acs-rds"subnet_ids = var.private_subnetstags = merge(var.tags,{Name = "ACS-database"},)}create the RDS instance with the subnets groupresource "aws_db_instance" "ACS-rds" {allocated_storage      = 20storage_type           = "gp2"engine                 = "mysql"engine_version         = "5.7"instance_class         = "db.t2.micro"name                   = "daviddb"username               = var.db-usernamepassword               = var.db-passwordparameter_group_name   = "default.mysql5.7"db_subnet_group_name   = aws_db_subnet_group.ACS-rds.nameskip_final_snapshot    = truevpc_security_group_ids = var.db-sgmulti_az               = "true"}CommentShareOluwasegunanswered 3 months ago0Hi @Oluwasegun,I believe the part that corresponds to var.private_subnets contains the subnet IDs in list form.The subnet group requires at least two Availability Zones to be in use by a group of subnets.From the error code, I think var.private_subnets consists only of eu-west-1c.Try the following command for the subnet ID to check the Availability Zone.aws ec2 describe-subnets --subnet-ids YOUR_SUBNET_ID --query [Subnets[].AvailabilityZone,Subnets[].SubnetId]CommentSharetakakunianswered 3 months ago"


Question: How to pass the nested json(internal structure) in Quick Sight using manifest file

"We have a JSON which is like this { Id: "12", Id_type: "xyz","Details":[{"Timestamp": "2022-10-21"}]}. This is our format. When I update the mainfest file with the foldr name it is taking only Id and ID_type in filters. It was not taking the internal one which is Details. Can I get the solution for this, how to get the internal structure data as well into the filters.FollowComment"


Question: Glue delays since last night

"Hi team, we are getting some unusual errors in Glue since last night.  Jobs are taking longer to start (Startup time does not show up for some time - like 3 minutes) also jobs continue to run continuously - even ones which were short-run jobs.  We noticed AWS made a recent update to the dashboard, so we are wondering if there was an update made to Glue 3.0 recently.  Has anyone heard about issues with Glue?Seeing this message: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources (edited)FollowComment"

Answer:
"0Hello,I understand that you are facing some unusual errors in Glue such as “Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources“ and jobs are taking longer to start and run.Please note that the error “Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources“ usually are transient issue in Glue which are observed if the resources are unavailable in the backend to execute the job leading the job to timeout or running for long.

This error occurs in scenarios if they is an ongoing AWS issue due to which it does not have enough resources on their end to run your Glue job. These issue usually get resolved in some time and on retrying the Jobs again in a couple of hours or next day the issue no longer happens.

Hence, this issue does not seem to be because of recent update to the dashboard. You can also check out the below link to understand some scenarios that might be resulting in Glue Job to run for a longer time:
https://repost.aws/knowledge-center/glue-job-running-longCommentShareSUPPORT ENGINEERrePost-User-6739329answered 2 months ago"


Question: "AWS GLue Spark job: Found duplicate column(s) in the data schema and the partition schema: `day`, `month`, `year`"

"Glue spark job is failing with the error message: AnalysisException: Found duplicate column(s) in the data schema and the partition schema: day, month, year. And my actual parquet data file in S3 includes these partition columns as well.Code snippet:dyf= glueContext.create_dynamic_frame.from_catalog(database = db,table_name = tbl, additional_options={"catalogPartitionPredicate":"year>=2022 and month>=10 and day>=1"},transformation_ctx="dyf")dyfdrop = dyf.drop_fields(paths=["year", "month", "day"])dyfdrop.toDF().printSchema()dyfdrop.toDF().show()S3 Path:s3://bucket/f1/f2/tbl/year=2022/month=10/day=1/Partition column names in glue catalog: "year", "month", "day"I am reading from Glue catalog by filtering partition column using above code.I have also tried to drop partition columns and still it didnt work.Please help me to fix this in order to read these files.ThanksFollowComment"

Answer: "1Hello,As the partition columns are also written in the schema of the Parquet files, because of this when we read the data using DynamicFrame and perform some Spark action to the created DynamicFrame it fails with below error:AnalysisException: Found duplicate column(s) in the data schema and the partition schema:In order to fix this error, the ideal way is to fix the underlying Parquet files by re-writing them after dropping the partition columns.However, as a workaround in this scenario, you can consider to update the Partition Column names manually in the Glue Data Catalog. Kindly refer [1] for more details. Then, you would be able to perform the action. By doing this, the partition column name read by the DynamicFrame will be referred from the Glue Data Catalog (as you are creating DynamicFrame using 'from_catalog()' method).Thank you!Reference:[1] https://docs.aws.amazon.com/glue/latest/dg/console-tables.html#console-tables-detailsCommentShareSUPPORT ENGINEERArushi_Janswered 7 months ago"


Question: Continuous child workflow

"HelloWe are designing a long-running workflow. We have a "parent" workflow that only does setting up and cleanup, the majority of work is done in child WF.Complete task may run for up to a year, so we thinking to implement child workflow to continue ascontinuous WF (https://docs.aws.amazon.com/amazonswf/latest/awsflowguide/continuous.html) continuing as new after completing each "chunk", but keep the parent WF running (setting timeout to one year as max allowed)We have tested the approach and it seems to work. The parent WF seems to be unaware of child restarting (which is exactly what we need) and history stays small.Is there any drawback to this approach? Anything we should consider?Thank you,AkbarFollowComment"

Answer: "0This works as designed :). This is a commonly used pattern for a parent to watch the child which calls continue as new periodically.CommentSharemfateevanswered 4 years ago0The tricky part is updating workflow code when you have open workflows.Use WorkflowContext.isImplementationVersion (https://github.com/aws/aws-swf-flow-library/blob/master/src/main/java/com/amazonaws/services/simpleworkflow/flow/WorkflowContext.java) to perform non backward compatible changes to workflow code without breaking running workflows.CommentSharemfateevanswered 4 years ago0Thanks a lot Maxim!CommentShareAlex2answered 4 years ago0Hello!Is there any documentation describing how "Continue as new" works in details?The issue we are facing is that the WF waits on tasks that are delivered via signal, processes tasks and then restarts itself. Sometimes seems we are skipping a task while restarting. So we would like to understand details of restarting to implement it correctly.Also is there any recommended practices on processing incoming tasks that can come in "burst" ie sometimes we can get multiple tasks while the first one is still being processed.Thank youCommentShareAlex2answered 4 years ago0As the WF actual restart happens when everything is completed, is there a way to execute a code right before WF restarts?Specifically we would like to pass the parameters to the "new" WF at the time of actual restart ie when everything is completedEdited by: Alex2 on Mar 10, 2019 6:21 PMCommentShareAlex2answered 4 years ago0Is there any documentation describing how "Continue as new" works in details?The issue we are facing is that the WF waits on tasks that are delivered via signal, processes tasks and then restarts itself. Sometimes seems we are skipping a task while restarting. So we would like to understand details of restarting to implement it correctly.There are two parts of the picture. The service and the client behavior. From the service point of view continue as new is the same as completion, but with a new run of a workflow created atomically with the same id. The Java client expects all the code in the workflow function complete (using an enclosing TryFinally to ensure that) and then requests continue as new. Calls to a self client are not causing immediate continue as new execution, they just store arguments for the next run. These arguments are used later when the workflow function is done.Also is there any recommended practices on processing incoming tasks that can come in "burst" ie sometimes we can get multiple tasks while the first one is still being processed.There are multiple strategies depending on your requirements. For example the signal handler can add the task to the queue field of the workflow and the workflow function gets  tasks from that queue and executes them. So if tasks come while processing they are just added to the queue.As the WF actual restart happens when everything is completed, is there a way to execute a code right before WF restarts?  Specifically we would like to pass the parameters to the "new" WF at the time of actual restart ie when everything is completedYou can use a TryFinally that wraps the rest of your workflow execution logic and call continue as newCommentSharemfateevanswered 4 years ago0Thank you very much, Maxim!Can you please elaborate on how to "use a TryFinally that wraps the rest of your workflow execution" in presence of signal handler?We ended up implementing it as follows:Implement custom workflow worker down to AsyncDecider overriding getContinueAsNewOnCompletion to supply parameters at that point.Signal handler just start a new task and puts the result promise to a list that AsyncDecider has access to.CommentShareAlex2answered 4 years ago0You are correct that wrapping the workflow function in TryFinally is not going to help if a signal was received after the workflow function completion. But I think calling self client from the signal in this case still has to override the continue as new arguments.In the new version of the Flow Framework (github.com/uber/cadence-java-client) I found a much better solution for this race condition. The basic idea is that if a signal is received after the workflow function completion the workflow code is rolled back and the signal is delivered before it is completed. This way having a try finally in the workflow code would provide such guarantee. The other nice property of the new framework that the workflow code is synchronous, so normal try/catch can be used.CommentSharemfateevanswered 4 years ago0We noticed an issues with the approach: even though the parent WF has child policy of "Terminate", if we terminate parent after the child continued as new, child is not terminated.Is it possible to enforce child termination in this scenario?Thank youCommentShareAlex2answered 4 years ago0Weird. Sounds like a server side bug. I'm not sure what is the best way to report it as this forum apparently is not monitored by the AWS folks.CommentSharemfateevanswered 4 years ago0More details:The child WF gets terminated, but after it continues as new again!CommentShareAlex2answered 4 years ago"


Question: Workmail not working

"Hi,I have configured Workmail in Ireland Region.  I am getting the following message.You have unverified domains. Resolve this issue before you can use the domain in your email addresses.The domain is hosted in Route 53. I can see the entries present correctly.Could someone please help me in resolving this.Thanks,KamalFollowComment"

Answer: "0Accepted AnswerHi,Thank you for using WorkMail. The error message you received is to complete your domain set-up. After adding a domain in WorkMail you need to add domain records to prove you own the domain and configure mail routing for that domain. It also includes domain records to make mail delivery more secure.More details on what each record is for can be found here: https://docs.aws.amazon.com/workmail/latest/adminguide/add_domain.htmlKind regards,RobinCommentShareMODERATORrobinkawsanswered a year ago"


Question: Account names not showing in some dashboards

"We have successfully deployed the CID plus TAO and COD. We wish to have account names showing so followed option Option3-A on https://wellarchitectedlabs.com/cost/200_labs/200_cloud_intelligence/cost-usage-report-dashboards/dashboards/code/0_view0/ and created the account_map view. The Organization-Data-CID Lambda has run successfully a number of times.The account names are only showing on the TAO and COD dashboards, not CUDOS and Cloud Intelligence.Is anybody able to advise what we've missed please?!Thank you.FollowComment"

Answer: "1Hi, It appears you already have the right steps to follow.Few additional things to review here:Can you confirm you have created the view account_map in Athena based on organizational data and that it's not empty? If the account names are showing for TAO and COD, then it means the table account_map is not empty but this is worth checking.Could you ensure the account IDs you are checking are valid and not suspended/terminated?Finally, have you tried to refresh the CUDOS SPICE related datasets in QuickSight?CommentShareEXPERTJonathan_Danswered 5 months ago0Accepted AnswerSuccess! We followed the documentation to create the map BUT we unknowingly ran it against the optimization_data database. Running this against the cid_cur has fixed it :)CommentShareSiTheAwsGuyanswered 5 months ago0Hi, thanks for your response.We created the account_map and all the accounts within the organisation are showing in there as we'd expect.All accounts are valid.We have manually refreshed the data by invoking the Lambda and it has also run on its schedule.CommentShareSiTheAwsGuyanswered 5 months ago"


Question: Postgres database upgrade failed (11.6 -> 13.7)

"Postgres database upgrade failed (11.6 -> 13.7) with error:pg_restore: while PROCESSING TOC:pg_restore: from TOC entry 6052; 1259 720106 INDEX unq_nickname_ci dbapg_restore: error: could not execute query: ERROR:  function public.unaccent(unknown, text) does not existLINE 3: SELECT public.unaccent('public.unaccent', $1)  -- schema-qua...^HINT:  No function matches the given name and argument types. You might need to add explicit type casts.After the first failed upgrade, the index and function were deleted, so they don't exist in the catalog, but subsequent upgrades still ends with the same error. Do you have any idea how to proceed?FollowComment"

Answer: "0Hi there,Can you run this command and see which targets are available for your current version:aws rds describe-db-engine-versions--engine postgres--engine-version 11.6--query "DBEngineVersions[].ValidUpgradeTarget[].{EngineVersion:EngineVersion}" --output textVersion 11.6 is currently not available on AWS so you may have to upgrade via the CLI, if you're not already. Follow this guide to ensure all requirements are met - https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_UpgradeDBInstance.PostgreSQL.html#USER_UpgradeDBInstance.PostgreSQL.MajorVersionThe command to upgrade is seen when you expand the AWS CLI toggle as seen below:aws rds modify-db-cluster--db-cluster-identifier mydbcluster--engine-version new_version--allow-major-version-upgrade--no-apply-immediatelyCommentShareSUPPORT ENGINEERBrandonanswered a month ago"


Question: Including thing identity in messages published to message broker

"So my understanding is that when an IoT device publishes and/or subscribes to the MQTT topics related to its shadow that AWS will ensure that the certificate on the device is allowed to access that particular shadow/topics.  Furthermore, the shadow topics have the name of the device embedded in the topics ($aws/things/ <thingName> /shadow/delete).With these two things, and assuming the device's private key hasn't been compromised, this ensures that the only device able to pub/sub on those topics is the one assigned the certificate and also proves the device's identity to the system.My question is if there is any similar functionality available for non-shadow MQTT topics.  For example, say I want to publish messages to /mySensor/ <thingName> /temperature, is there any way to 1) automatically include the thingName 2) enforce that the thingName is only accessible by the thing that possess the proper certificate?An alternative would be to publish to a topic not specific to a device (ex: /sensors/temperature) but have the system inject the device identities in the message that ends up on the topic.FollowComment"

Answer: 0Of course 10 minutes after posting I find the answer on my own but figured I'd post it here for anyone else looking:It turns out the IoT thing IAM policies can leverage variables to enforce the described functionality. See https://docs.aws.amazon.com/iot/latest/developerguide/thing-policy-variables.htmlCommentSharenakanswered 4 years ago


Question: Is there a way to right click a selection of mesh and send to a specific layer?

Follow

Answer: "0Accepted AnswerAs far as I know, the layer system is not supported anymore. All slices are per default added to the main layer so you can keep track of the objects added to a level in the level list, but you cant change the layers. Not 100% sure about that because I am mostly coding, but you might wanna look into the slice system on how to create hierarchies and simultaneous work on one level with your team.SharerePost-User-5460328answered 6 years ago0Can I pay you to teach me how to use Lumberyard for 1 hour ?SharerePost-User-0522404answered 6 years ago0Best for you would be, to check the samples provided by Amazon. See how they do things and try to rebuild that in your own game/level.If you have specific questions, people here are glad to answer them for you for free! ;)I am not sure what I can teach you that you could not learn by yourself.SharerePost-User-5460328answered 6 years ago"


Question: CDK Lambda Deployment

"Hello,So I'm using CDK to build my infrastructure. I have created a Lambda Function in my CDK and another folder that holds the Lambda code. I used the following commands to deploy it to my AWS account in the command line:npm run build && cdk synthcdk bootstrapcdk deployEverything worked fine the first time. However when I make any changes to my Lambda code and re-use these commands again my lambda code is not modified on the console. Seems like its not deploying the code because there is no changes in the CDK. How do I deploy my modified lambda code then? Can't find any documentation on that.FollowComment"

Answer: "2Accepted AnswerSince CDK uses CloudFormation to deploy its resources, any updates will need to trigger a CloudFormation update so that the code will be updated. As a test, I used the following CDK Lambda Code. Please see below:const bucket = s3.Bucket.fromBucketName(this, 'bucket', '<Bucket_Name>');      const lambdaFunc = new lambda.Function(this, 'lambdaFunc', {          runtime: lambda.Runtime.PYTHON_3_9,          code: new lambda.S3Code(bucket, "hello.py.zip", '<S3_Object_Version_Number>'),          handler: 'lambda.lambda_handler',    });Since I am leveraging S3 as the source of my code, any changes to the code will require the following steps:Upload the Code to S3Obtain the Version Number in the S3 console in the Versions tabAdd the Version Number to the third parameter in the lambda.S3Code() functionrun the cdk deploy commandAfter the Stack updated successfully, I was able to confirm that my Lambda code changed in the Lambda Console. Please see the test results below:===Original:def handler(event, context):    print('request: {}'.format(json.dumps(event)))    return {        'statusCode': 200,        'headers': {            'Content-Type': 'text/plain'        },        'body': 'Hello, CDK! You have a hit {}\n'.format(event['path'])    }Updated:def handler(event, context):    print('request: {}'.format(json.dumps(event)))    return {        'statusCode': 200,        'headers': {            'Content-Type': 'text/plain'        },        'body': 'Hola, CDK! Tienes un éxito {}\n'.format(event['path'])    }===Since there are various ways to source the Code in CDK for Lambda Constructs, a code snippet and CDK version will help me in provide the appropriate update steps for your use case.CommentShareGovind Kumaranswered a month ago2I noticed that you mentioned that the Lambda code is in another file. My apologies for missing this detail earlier.As a test, I deployed a CDK Stack that leveraged the Lambda construct. Please see the CDK code below: const lambdaFunc = new lambda.Function(this, 'lambdaFunc', {          runtime: lambda.Runtime.PYTHON_3_9,          code: lambda.Code.fromAsset('lambda'),          handler: 'hello.handler',    });  }The Lambda code will need to be in the in the CDK App directory but does not have to be in the bin nor the lib directory. In my example, the handler function is defined in the hello.py file that is in the lambda folder in my app.Once my stack was deployed, I made the changes to the hello.py file and, then, ran the cdk deploy command. CDK was able to detect that the file changed and made the update to the Lambda function via CloudFormation. Please see the test results below:===Original:def handler(event, context):    print('request: {}'.format(json.dumps(event)))    return {        'statusCode': 200,        'headers': {            'Content-Type': 'text/plain'        },        'body': 'Hola, CDK! Tienes un éxito {}\n'.format(event['path'])    }Updated:def handler(event, context):    print('request: {}'.format(json.dumps(event)))    return {        'statusCode': 200,        'headers': {            'Content-Type': 'text/plain'        },        'body': 'Hello, CDK! You have a hit {}\n'.format(event['path'])    }===CommentShareGovind Kumaranswered a month ago"


Question: SES won't tag links with unique identifiers

"Hello,I'm trying to uniquely add identifiers to links, so that I can track Click events separately.I'm following what described in question 5 ("Q5. Can I tag links with unique identifiers?") of the Amazon SES email sending metrics FAQs, which suggests to add ses:tags:<a ses:tags="product:book;genre:fiction;subgenre:scifi;type:newrelease;"     href="http://www.amazon.com/…/">New Releases in Science Fiction</a>The problem is that, upon reception of the email, the ses:tags do not appear in the email received by recipients.Even by inspecting the source of the email delivered, it is clear that these tags have been stripped before delivery (still not clear to me if the stripping has been done by SES or maybe Gmail).(To give additional context: generic Open and Click tracking work fine with my current setup. Only ses:tags are not showing up).Can anyone help please?FollowComment"


Question: Client VPN slowness

"I have Client VPN set up on AWS, which connects to a business web app on EC2. I access it from a Windows 10 computer using OpenVpn GUI.The web app runs very slowly at times, where a page takes 5-10 seconds to load. Previously, when it was accessed directly and not through the VPN, the web app loaded pages in under 1 second.Is there any way to improve the speed of Client VPN?FollowComment"

Answer: "0It may due to the internet connectivity speed and latency between your Windows PC and the Web EC2 if there is large among of data to be transferred in order to be load up a page.You can consider trying with another internet connection.There is also another option that you can consider is running AWS workspaces (https://aws.amazon.com/workspaces/) such that your desktop client is close enough to the Web EC2 and you just send the screen output instead of the page content to your Windows client.CommentShareGaryKo_AWSanswered a year ago0The number of seconds are unlikely caused by latency, even with very sub-optimal routing. This looks more like a timeout issue on some of the resources. I recommend that you look into the developer tools of your browser. Many have support for recording the network traffic. Check what is happening during the 5-10 seconds and look at the destination URLs. Maybe your web application is loading fonts or JavaScript from other web pages. Check the reachability of that URLs with and without ClientVPN. I guess you will find an indication here to decide on the next steps.CommentShareEXPERTAndreas Seemuelleranswered a year ago"


Question: HyperV on Cluster migration support

Helo everyone!We have a hyperV cluster running and I need to migrate VMs to AWS.I saw AWS MS documentation and it tells that supports:1- Standalone HyperV2- SCVMM that manages HypeV3- HyperV servers managed by SCVMMBut I haven´t found any information about support of HyperV CSV Cluster. Is it supported? Any docs related?Thanks!ThiagoFollowComment

Answer: "0Hi,Posting in case someone look for information.I did run the script on the cluster node and the process to replicate the clustered VMs to AWS worked smoothly.Regards,ThiagoCommentSharethiagoandradeanswered 4 years ago"


Question: Resolution for Fatal error when using DMS for on-going replication from RDS Postgres to S3

"I attempted to configure my RDS postgres instance for CDC using [Setting up an Amazon RDS PostgreSQL DB instance as a source][1]   I configured the DMS source endpoint to use a specific slotName.  First using a slot that already existed in the DB and then using a slot I created using pg_create_logical_replication_slot.  After successfully testing the endpoint to ensure connectivity, I started the replication task with was to load existing data followed by on-going changes.  Both times, the replication failed with a fatal error such as the following and no data (even the existing data that should have loaded) gets replicated.Last Error Stream Component Fatal error. Task error notification received from subtask 0, thread 0 [reptask/replicationtask.c:2822] [1020101] Error executing source loop; Stream component failed at subtask 0, component st_0_KVPUHIZBICJJRNGCC32HP5EFGGLIZLRF2YSRJ6Y ; Stream component 'st_0_KVPUHIZBICJJRNGCC32HP5EFGGLIZLRF2YSRJ6Y' terminated [reptask/replicationtask.c:2829] [1020101] Stop Reason FATAL_ERROR Error Level FATALWhat is the configuration I'm missing to enable CDC from RDS Postgres v11.8?  I'm configuring DMS to use the RDS master user.  A migration existing data only DMS task succeeds on this instance (if the slotName configuration is removed).The relevant CloudWatch log entries appear to be these2021-02-03T22:03:05 [SOURCE_CAPTURE  ]I:  Slot has plugin 'test_decoding'  (postgres_test_decoding.c:233)2021-02-03T22:03:05 [SOURCE_CAPTURE  ]I:  Initial positioning requested is 'now'  (postgres_endpoint_capture.c:511)2021-02-03T22:03:05 [SOURCE_CAPTURE  ]E:  When working with Configured Slotname, user must specify LSN [1020101]  (postgres_endpoint_capture.c:517)2021-02-03T22:03:05 [TASK_MANAGER    ]I:  Task - W6AUC5OI3DNFMFUQ6ZDEYYNZ3NBSABQK3HFD2WQ is in ERROR state, updating starting status to AR_NOT_APPLICABLE  (repository.c:5101)2021-02-03T22:03:05 [TASK_MANAGER    ]E:  Task error notification received from subtask 0, thread 0 [1020101]  (replicationtask.c:2822)2021-02-03T22:03:05 [TASK_MANAGER    ]E:  Error executing source loop; Stream component failed at subtask 0, component st_0_KVPUHIZBICJJRNGCC32HP5EFGGLIZLRF2YSRJ6Y ; Stream component 'st_0_KVPUHIZBICJJRNGCC32HP5EFGGLIZLRF2YSRJ6Y' terminated [1020101]  (replicationtask.c:2829)2021-02-03T22:03:05 [TASK_MANAGER    ]E:  Task 'W6AUC5OI3DNFMFUQ6ZDEYYNZ3NBSABQK3HFD2WQ' encountered a fatal error  (repository.c:5194)2021-02-03T22:03:05 [SORTER          ]I:  Final saved task state. Stream position , Source id 0, next Target id 1, confirmed Target id 0, last source timestamp 0  (sorter.c:803)Other items in the log appear to be informational to track progress.[1]: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.RDSPostgreSQL.CDCFollowComment"

Answer: "1Accepted AnswerPlease check if below information can help,rds.logical_replication is enable.If using Existing Slot we will need to configure below for a Task.-->CLI -aws dms create-replication-task--replication-task-identifier <<task_identifier>>--source-endpoint-arn <<SRC_ARN>>--target-endpoint-arn <<Target_ARN>>--replication-instance-arn <<RepInstance_ARN>>--migration-type cdc--table-mappings <<path_to_mapping_json>>--cdc-start-position  "0/57D86A50" <<LSN need to specified as Start Position>>--replication-task-settings <<Task_Setting_File>>-->Add Slot name in source end points connection attribute within AWS DMSIf don't wanna use existing replication slot,AWS DMS will automatically create one based on task identified, we don't need to create.Verify usingselect * from pg_replication_slots;Additional details for referencehttps://aws.amazon.com/premiumsupport/knowledge-center/dms-postgresql-fail-slots-in-use/https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.RDSPostgreSQLCommentShareDeepak Mahtoanswered 2 years ago"


Question: Using Lightsail with Cloudfront

"I've previously used Cloudfront with S3 and also with WordPress sites hosted on Lightsail. I'm now trying to set up a Cloudfront distribution for standard websites (non WordPress) hosted on Lightsail, (without a Load Balancer). I note that the Cloudfront distribution seems to hit the 'default' Lightsail page (This can be the Plesk default page if you're using Plesk or the default website - if you have set one as default). There doesn't seem to be any way of targeting a second or third site hosted on the Lightsail instance with a Cloudfront distribution. Does this mean I can only have one Cloudfront distribution for a single site on a Lightsail instance or has anyone come across a workaround?FollowComment"

Answer: "0The answer is to amend the CloudFront distribution to Whitelist HTTP headers. You can then choose to forward the Host header from CloudFront to your origin.CommentShareGTecanswered 3 years agorePost-User-2742014 4 months agohow do you do this in lightsail? edit the way you do this with lightsail is in your distribution's Directory and file overrides, you need to specify path to directories to NOT CACHE. files in those directories seem to recieve full http headers and for webhooks shouldnt be cached anyways.Share"


Question: Data Lake on AWS - implementation still current?

"I'm investigating and deploying https://docs.aws.amazon.com/solutions/latest/data-lake-solution/welcome.htmlLooking at the GitHub repo https://github.com/aws-solutions/aws-data-lake-solution it looks like this solution is somewhat outdated.It also contains a broken link to the underlying documentation: https://github.com/aws-solutions/aws-data-lake-solution from this page: https://docs.aws.amazon.com/solutions/latest/data-lake-solution/deployment.htmlBefore I invest a lot of time in this solution the question then is if this solution is still up-to-date and is it still worthwhile to deploy and explore it for a production use case? Alternatively, what is a comparable and current solution to look at? Thanks in advance for advice.FollowComment"

Answer: "0Accepted AnswerHi - I would recommend that you take a look at the DDK (https://github.com/awslabs/aws-ddk) if you want to create data pipelines using CDK constructs or if you want a pre-packaged data lake in CloudFormation you could check out the SDLF (https://github.com/awslabs/aws-serverless-data-lake-framework/).  Both have workshops that should quickly get you up to speed!CommentShareNickanswered 10 months ago0Great! I missed the DDK. That will be awesome to use! Thanks.CommentSharePeter Lorentanswered 10 months ago0Hi, Have a look at https://aws.amazon.com/lake-formation/CommentShareNuno_Qanswered 10 months ago"


Question: API Gateway - Gateway response - HTTP API

"With HTTP APIs, is there a way to customise the response sent back in errors from the API GW itself - i.e. is there an equivalent to Gateway Responses in REST APIs?FollowComment"

Answer: 0Accepted AnswerChanging API Gateway errors is currently not supported. You can use response parameters  to transform the HTTP response from a backend integration before returning the response to clients. You can modify headers or the status code of a response before API Gateway returns the response to clients.CommentShareEXPERTUrianswered 2 years agoEXPERTFaraz_AWSreviewed 2 years ago


Question: SSM sendcommand does not return any messages at lambda function

"Hello there!I'm a very beginner engineer and really new to AWS and web application systems. I have some questions about SSM send command functions.I tried to run EC2 by lambda function using SSM sendcommand. But there are no errors and no response from sendcommand. Besides, it didn't send any log to CloudWatch. It worked from System Manager console so I think IAM policies are fine. The lambda uses javascript and AWS SDK for JavaScript v2. Please tell me how to solve this problem.. Thank you in advance for your kind help!This is the program which I tried to run on EC2,#!/bin/bash    echo test >> test.txt  echo test  And this is index.js.const AWS = require("aws-sdk");    const SSM = new AWS.SSM({ region: process.env.REGION });  const REMOTE_WORKING_DIR = '/home/ec2-user/environment/test';    exports.handler = async (event) => {      try{          let command = 'sh test.sh';                    let params = {              DocumentName: 'AWS-RunShellScript',              InstanceIds: \["**********"],              Parameters: {                  commands: \\[command\],                  workingDirectory: \\[REMOTE_WORKING_DIR\]              },                            CloudWatchOutputConfig: {                  CloudWatchOutputEnabled: true              },                            TimeoutSeconds: 3600          };                    console.log(params);                    await SSM.sendCommand(params, function(err, data){              if(err){                  console.log("error", err);              }else{                  console.log("success", data);              }          });      }catch(e){          console.log("ERROR");          console.log(e);      }            return;  };  Edited by: yoshi09japan on Feb 7, 2021 6:09 PMFollowComment"

Answer: "0I found this was caused by not waiting the sendCommand.This works fine.        //Modified these lines          // await SSM.sendCommand(params, function(err, data){          //     if(err){          //         console.log("error", err);          //     }else{          //         console.log("success", data);          //     }          // });            //this works fine          const sendCommandResult = await SSM.sendCommand(params).promise();          console.log(sendCommandResult);  CommentShareyoshi09japananswered 2 years ago"


Question: Quicksight working with calculated fields & percentages

"Hi, I have the following dataset and I am really struggling to find the right calculation to get the result I am looking for:-issueIdteammonthYearworkCategorypointsME-44846TeamAApr-2023Engineering3ME-45314TeamAApr-2023BAU4ME-45739TeamAApr-2023Engineering3ME-43757TeamADec-2022Engineering3ME-43760TeamADec-2022BAU3ME-43762TeamADec-2022BAU2ME-44256TeamADec-2022BAU2ME-44257TeamBApr-2023BAU2ME-44331TeamBApr-2023Engineering3ME-44571TeamBApr-2023Engineering3ME-44583TeamBApr-2023BAU2ME-44816TeamBDec-2022BAU2ME-44830TeamBDec-2022BAU2ME-44832TeamBDec-2022BAU2ME-44835TeamBDec-2022Engineering0I have a calculated field (sum(max({points},[{issueId}))) which sums the points per team.  But I actually want to convert the points into a percentage based on the workCategory, team and month. ie Team A in Apr-2003 spent 60% BAU/40% Engineering which I can plot on a graph with the month as the x-axis.I'd then like to sum the averages across teams to get a total percentage split on Engineering/BAU.Is this even possible?FollowComment"


Question: Redshift - How to grant user permission to SELECT from a view without granting access to the underlying external table

"Hi,I have external tables in an external schema(datashare). I have created views off these tables in a separate schema.I'm looking to grant a user access to only the views, and not the underlying tables.Is this at all possible?For example, when the user tries to read from the view thats pointing to the external table, they get error "ERROR: permission denied for schema external_schema".However, running GRANT USAGE ON SCHEMA external_schema TO user;gives the user SELECT access to both the view and the underlying external table, which is what I want to avoid.Thanks!FollowComment"

Answer: "0Accepted AnswerHello,Thank you for reaching out. Your understanding is right that views created on external tables for users who do not have access to the underlying tables.For a user to access the view, they needed to be granted USAGE permission on the external schema. This is currently a limitation and we have a feature request in place to address this concern. However, we do not have an ETA for the feature at this point of time.I request you to follow below blogs for information on new features.[1] What's new: https://aws.amazon.com/redshift/whats-new/[2] Blog: https://aws.amazon.com/blogs/aws/category/database/amazon-redshift/We apologize for the inconvenience.CommentShareSUPPORT ENGINEERAWS-User-4315904answered 10 months ago"


Question: EC2 instance rebooted unexpectedly

"Hello,Instance ID: i-0582a28a46a656e64 in ap-northeast-1 rebooted unexpectedly at 2023-02-15 22:15:35 UTC.$ journalctl --list-boots-2 f2c39f848342459da7d2a916638583dd Wed 2021-09-01 01:19:10 UTC—Mon 2021-10-18 07:16:17 UTC-1 bc0fca97974644549d4db7f7400f8191 Mon 2021-10-18 07:16:22 UTC—Wed 2023-02-15 22:06:00 UTC 0 2821ef65e52940e298c6fa400aef02b7 Wed 2023-02-15 22:15:35 UTC—Thu 2023-02-16 01:34:03 UTCI checked CloudTrail but could not find any event that looked relevant.Can someone from Amazon check the cause of this reboot?Thank you.FollowComment"

Answer: "0Hello,It would be recommended to open up a case with AWS Support if at all possible regarding the mentioned EC2 instance so that AWS Support Engineers are able to review your resources in-depth. In general, there is a wide variety of causes for reboots per this knowledge center article.The instance failed one or both of its status checks.The underlying hardware hosting your instance was faulty and Amazon EC2 restarted the instance to move it to new, healthy hardware.Scheduled maintenance occurred on your instance that required a reboot.A user or application inside your server rebooted the instance.A kernel bug. If you encounter a kernel bug, be sure that your kernel is fully updated.Resolutions and investigative steps are located within the article. This article also exists for Windows instances specifically.CommentShareKameron_Banswered 4 months agorePost-User-8825234 4 months agoThank you for the response. However, I could not reach the technical support since my organization is in a basic support plan. Regarding the general causes, I did not find any scheduled event in the console's Events page, the instance's kernel was updated to the latest (release: 226.531.amzn2), and the server was not rebooted from inside according to the system/audit log.Is it not possible to know if there was any underlying hardware issue, like in these previous questions?https://repost.aws/questions/QUHuTCxSGCRlihWPCyBpqWMQ/ec-2-instance-unexpectedly-reboots-itselfhttps://repost.aws/questions/QUJTK5pEciRBeQjfagKZvXDw/ec-2-instance-rebooted-by-itselfShare"


Question: XML interpret one struct as an array

"I've been trying this for a week but I'm starting to give up - I need some help understanding this. I have an S3 bucket full of XML files, and I am creating a pyspark ETL job to convert them to Parquet so I can query them in Athena.Within each XML file, there is an XML tag called ORDER_LINE. This tag is supposed to be an array of items, however in many files, there is only one item. XML does not have the concept of arrays, so when I pass this into my ETL job, Glue interprets the field as a Choice type in the schema, where it could either be an array or a struct type. I need to coerce this into an array type at all times. Here's a list of everything I've tried:Using ResolveChoice to cast to an array. This doesn't work because a struct can't be casted to an arrayDoing ResolveChoice to "make_struct", then the Map.apply() step to map the field where if "struct" has data, transform it to [struct]. This doesn't work and the Map docs hint that it does not support the python map function for arrays.Converting the dynamic frame to a data frame, and then using pyspark withColumn(when(struct.isNotNull, [struct]).otherwise(array)) functions to convert the struct to an array, or make the array the main object, depending on which one is not null. This doesn't work because Glue is inferring the schema in the structs, and the fields in the structs are in a different order, so while all the fields in the schema are the same, Spark can't combine the result because the schema is not exactly the same.Converting to data frame, then using a pyspark UDF to transform the data. This worked on a small dev sample set, but failed on the production dataset. The error message was extremely cryptic and I wasn't able to find the cause. Maybe this could work but I wasn't able to fully understand how to operate on the data in pyspark.Trying to use the "withSchema" format_option when creating the dynamic frame from XML. The intention is to define the schema beforehand, but running this gives an error:com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` out of VALUE_TRUE token at [Source: (String)" [...] (through reference chain: com.amazonaws.services.glue.schema.types.StructType["fields"]->java.util.ArrayList[0]->com.amazonaws.services.glue.schema.types.Field["properties"])So my question is, how do I make the XML data source for Glue interpret a tag as always an array, instead of a Choice, or how do I combine them? Even StackOverflow failed me here, and the forum post https://forums.aws.amazon.com/thread.jspa?messageID=931586&tstart=0 went unanswered.Here's a snippet of my pyspark code:import sysimport jsonfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom awsglue.gluetypes import (    StructType,    Field,    StringType,    IntegerType,    ArrayType,)from awsglue.context import GlueContextfrom awsglue.job import Job## @params: [JOB_NAME]args = getResolvedOptions(    sys.argv,    [        "JOB_NAME",        "source_bucket_name",        "target_bucket_name",    ],)sc = SparkContext()glueContext = GlueContext(sc)spark = glueContext.spark_sessionjob = Job(glueContext)job.init(args["JOB_NAME"], args)source_bucket_name = args["source_bucket_name"]target_bucket_name = args["target_bucket_name"]schema = StructType(    [        [fields removed as they are sensitive]        Field(            "ORDER_LINE",            ArrayType(                StructType(                    [                        Field("FIELD1", IntegerType(), True),                        Field(                            "FIELD2",                            StructType([Field("CODE", StringType(), True)]),                            True,                        ),                        Field(                            "FIELD#",                            StructType([Field("CODE", StringType(), True)]),                            True,                        ),                        [fields removed]                    ]                )            ),            True,        ),    ])datasource0 = glueContext.create_dynamic_frame.from_options(    "s3",    {"paths": [f"s3://{source_bucket_name}"]},    format="xml",    format_options={        "rowTag": "ORDER",        "withSchema": json.dumps(schema.jsonValue()),    },    transformation_ctx="datasource0",)[more steps after this]FollowComment"

Answer: "1Accepted Answerhi again,I have been testing the suggested solution (just using your AWS forums xml example data). If you add the Databrics XML library to glue then you can let the library to infer your schema (it does well in my case with two files, one containing the array and the another without it):from pyspark.sql import SparkSessionfrom pyspark.sql.types import *df =spark.read.format('xml').options(rowTag='indices').load('s3_dir')df.show()df.printSchema()If you want to declare the schema in advance it will be similar to this.my_new_schema = StructType([    StructField('index', ArrayType(StructType([        StructField('indexname', StringType()),        StructField('indexsymbol', StringType())    ])))])df = spark.read.format('xml').options(rowTag='indices').load('s3_dir', schema = my_new_schema)You would need to add to the Glue Job parameters Conf the following in order to add the library to Glue.spark.jars.packages = com.databricks:spark-xml_2.12:0.13.0BestsCommentShareacmanjonanswered a year agoEXPERTsedat_salmanreviewed 3 days agoEXPERTFabrizio@AWSreviewed a year agoEelviny a year agoNice, I'll give this a go! I've not really gone outside of the options given by pyspark, and I can't quite figure out how to input spark.jars.packages = com.databricks:spark-xml_2.12:0.13.0. I tried adding it as a command line argument --conf spark.jars.packages=com.databricks:spark-xml_2.12:0.13.0 but my CDK app rejects this as --conf is a reserved parameter that shouldn't be set. How are you adding it?Shareacmanjon a year agoThe official way to go is to use the approach of uploading the libraries to S3 and then reference them on --extra-jars parameter.Look at the documentation here: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.htmlAnd you can look for a CloudFormation template here: https://github.com/aws-samples/amazon-deequ-glue/blob/master/src/template.yamlSorry but I am not an expert on CDK, hopefully you have that parameter there.ShareEelviny a year agoThanks @acmanjon, got it working in CDK! The options are basically the same there, and it has the added bonus that I can add the JAR as an asset and it will handle uploading and referencing the JAR. However I've hit another snag, when using spark XML, Glue seems to lose the information it needs to handle job bookmarks, so running the job a subsequent time does the whole run again. Will keep searchingShare1Hi,Could you paste the 4rd option error? I think it should be the way to go.Another option is to use Databricks XML library. If you specify the schema forehand it should work.https://github.com/databricks/spark-xmlBestsCommentShareacmanjonanswered a year agoEelviny a year agoThanks! I forgot to mention that I'm using pyspark (I'm not a java expert by any means). I'll add sample code to my question.Here's the UDF code I tried:schema = ArrayType(    StructType(        [            StructField("DY_AMOUNT", IntegerType(), True),            StructField(                "PRODUCT_DRAGER",                StructType([StructField("CODE", StringType(), True)]),                True,            ),            StructField(                "PRODUCT_SOORT",                StructType([StructField("CODE", StringType(), True)]),                True,            ),            StructField("PU_AMOUNT", IntegerType(), True),            StructField("RC_EQUIVALENT_DY_AMOUNT", IntegerType(), True),            StructField("RC_EQUIVALENT_PU_AMOUNT", IntegerType(), True),            StructField("RC_EQUIVALENT_SCHEDULED", IntegerType(), True),            StructField("SCHEDULED", IntegerType(), True),        ]    ))struct_to_array = udf(    lambda order: order.array if order.array else [order.struct], schema)map0 = dataframe0.withColumn("ORDER_LINE", struct_to_array(col("ORDER_LINE")))`` `ShareEelviny a year agoAnd the error:[task-result-getter-2] scheduler.TaskSetManager (Logging.scala:logWarning(69)): Lost task 32.0 in stage 2.0 (TID 189) (172.34.232.204 executor 7): org.apache.spark.api.python.PythonException: Traceback (most recent call last):  [...]  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/types.py", line 449, in toInternal    return self.dataType.toInternal(obj)  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/types.py", line 641, in toInternal    raise ValueError("Unexpected tuple %r with StructType" % obj)ValueError: Unexpected tuple 2 with StructTypeat org.apache.spark.api.python.BasePythonRunner$ReaderIterator.handlePythonException(PythonRunner.scala:517)at org.apache.spark.sql.execution.python.PythonUDFRunner$$anon$2.read(PythonUDFRunner.scala:84)[...]Share0@acmanjon's solution is the most elegant way of solving the problem, with the Spark XML library handling the inference of the struct as an array. However, that method doesn't use Glue's create_dynamic_frame to create the dataframe, meaning that Glue bookmarks don't work. So, I present the second option: the string hack.By using ResolveChoice to cast the struct/array as a string, we can use a combination of pyspark's UDF and from_json functions to take the serialized string and convert simply to a uniform schema. It's not pretty or performant, but solves the immediate issue.import sysfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom pyspark.sql.functions import col, udf, from_jsonfrom pyspark.sql.types import (    StructType,    StructField,    StringType,    IntegerType,    ArrayType,)from awsglue.context import GlueContextfrom awsglue.dynamicframe import DynamicFramefrom awsglue.job import Job## @params: [JOB_NAME]args = getResolvedOptions(    sys.argv,    [        "JOB_NAME",        "source_bucket_name",        "target_bucket_name",    ],)sc = SparkContext()glueContext = GlueContext(sc)spark = glueContext.spark_sessionjob = Job(glueContext)job.init(args["JOB_NAME"], args)source_bucket_name = args["source_bucket_name"]target_bucket_name = args["target_bucket_name"]datasource0 = glueContext.create_dynamic_frame.from_options(    "s3",    {"paths": [f"s3://{source_bucket_name}"]},    format="xml",    format_options={"rowTag": "ORDER"},    transformation_ctx="datasource0",)# Resolve ambiguous data fields; casting to correct valuedatachoice0 = ResolveChoice.apply(    frame=datasource0,    choice="cast:string",    transformation_ctx="datachoice0",)schema = ArrayType(    StructType(        [...put your schema here]    ))# In string form, look to see if the string is in a square bracket [, indicating an array, if not add them@udf(returnType=StringType())def struct_to_array(order):    if order:        return f"[{order}]" if order[:1] != "[" else order    # Handle case where "array" is empty    return "[]"map0 = datachoice0.toDF().withColumn(    "ORDER_LINE", from_json(struct_to_array(col("ORDER_LINE")), schema))fromdataframe0 = DynamicFrame.fromDF(map0, glueContext, "fromdataframe0")[...continue with the rest of your ETL]CommentShareEelvinyanswered a year ago"


Question: AWS DMS from On-Prem Informix to AWS MYSQL

"I'm unable to authenticate/connect to local Informix server from AWS DMS endpoint.**Here is the Error: **Test Endpoint failed: Application-Status: 1020912, Application-Message: Cannot connect to DB2 LUW Server Network error has occurred, Application-Detailed-Message: RetCode: SQL_ERROR SqlState: 08001 NativeError: -1336 Message: [unixODBC][IBM][CLI Driver] SQL1336N The remote host "Informix_server" was not found. SQLSTATE=08001Which source Engine in Endpoint, I need to use to connect to Informix Server?FollowComment"

Answer: 1Here is a list of databases that DMS supports as a source - https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.Sources.htmlInformix is not a supported source database.CommentShareEXPERTIndranil Banerjee AWSanswered 8 months ago


Question: AWS App Runner Stuck

"HiI've tried starting up an app runner service using a custom Docker container, and it seems to be stuck. My log messages are as follows:08-21-2021 10:51 PM [AppRunner] Successfully created pipeline for automatic deployments.08-21-2021 10:51 PM [AppRunner] Service status is set to OPERATION_IN_PROGRESS.08-21-2021 10:51 PM [AppRunner] Service creation started.Unfortunately, these were created 40 minutes ago at the time of writing and I can't see any update from here. I tried using the aws console to delete the running service, but I get the following error message:An error occurred (InvalidStateException) when calling the DeleteService operation: Service cannot be deleted in the current state: OPERATION_IN_PROGRESS.I can't do anything, e.g. update env variables, use the console to delete etc. Is there anything else I can do as it looks stuck. Has anyone else had this issue before?SimmyFollowCommentrePost-User-4420190 8 months agoI have the same issue but it's been stuck for 6 days!Share"

Answer: "0Ah it finally failed out enabling me to delete it just after I posted. I think that role provided doesn't have access to the container registry, so that's a learning point for me.CommentShareSimmy1234567890answered 2 years ago"


Question: "Athena: Reference left-hand row in right-hand subquery? (correlated subquery, CROSS JOIN LATERAL)"

"I have a usecase where I need to support recursive relationship fetching, with a possible LIMIT clause on each depth of relationship.Something like:SELECT ... FROM artists LIMIT 5for artist in artists:    SELECT ... FROM albums WHERE artist_id = artist.id LIMIT 3    for album in albums:        SELECT ... FROM tracks WHERE album_id = albums.id LIMIT 2In Postgres/MySQL etc, you can do this with LATERAL/CROSS APPLY.select *from  lateral (    select ...    from `Artist`    limit 5  ) as `Artist`,  lateral (    select ...    from `Album`    where `Artist`.`ArtistId` = `Album`.`ArtistId`    limit 3  ) as `Album`Athena says that it supports CROSS JOIN LATERAL:https://docs.aws.amazon.com/athena/latest/ug/engine-versions-reference.htmlHowever if you try to use this with a correlated subquery, it wigs outIs there any way to emulate the behavior of LATERAL/CROSS APPLY with the variant of SQL that Athena V2 supports?FollowComment"


Question: VMC on AWS Bill of materials

"Hi Gurus,I am working on setting up VMC on AWS. Can anybody please advise with Bill of materials(minimum required template) for setting up VMC on AWS?ThanksJoFollowComment"

Answer: "0Hello,This planning guide from VMware has useful pointers: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vmc-aws/preparing-for-vmware-cloud-on-aws.pdfCommentShareAWS-ADolganovanswered a year ago0Another useful reference, VMware Cloud on AWS - Pricing.CommentShareEXPERTkentradanswered a year ago"


Question: Get AWS textrack to ignore logo part when scanning

I have a AWS textrack service that keeps reading the address from the logo when scanning how do I get it to ignore that part of the documentFollowComment

Answer: 0This AWS blog post https://aws.amazon.com/blogs/machine-learning/process-text-and-images-in-pdf-documents-with-amazon-textract/ is specifically targeted at the use case of removing visuals from pdf documents. There is a link to a Juypter notebook.CommentShareleftbrainstuffanswered a month ago


Question: delivery rate drops from 500 to 50 MPS every day after 9 PM

our cx have noticed that the delivery rate dropped enormously to 50MPS(normally 500mps) after 9pm everyday.They are making inquiries that if Singapore forbids sending promotional messages after 9pm everyday.Would anyone kindly tell meFollowComment