Question: Scale testing or simulation very large AWS environments

"How do I simulate very large AWS account(s) with 70k VMs or large K8 clusters?FollowCommentChris_G EXPERT7 months agoCan you provide more context in your question?  What are your goals and what are you trying to achieve? When you say 70k VMs, are you referring to EC2 instances?  Its very hard to give you an answer without more details.SharerePost-User-8005690 7 months agoHi Chris, thanks for responding. Yes, these are EC2 instances. I'm trying to simulate a large enterprise AWS footprint to test some of my scenarios for scale such as enumerating the resources in the account, reading the metadata of each resource programmatically. Basically you can think of combinations of accounts across various resources that a large enterprise would be create/deploy. Let me know if additional information is needed.SharerePost-User-8005690 7 months agoanyone else?ShareChris_G EXPERT7 months agoI unfortunately don't have a great answer when it comes to that scale.  There are service quotas and availability zone specific resource limitations that may prevent tests like that from being possible.  You could definitely write your own scripts to execute CloudFormation, CDK, or Terraform constructs to build such an environment, but with service quotas and resource limitations, it may take a significant amount of time and resources.Share"


Question: Glue Studio Job - How to add the file name or path as a Source Key output or partition

"I would like to add the source file path and name as a partition to the output of a Glue Studio UI designed job. What, if any, context or session values are available? How does one reference them in the UI?I am seeking to avoid editing the graphically generated script, if possible though not ruling that option out.FollowComment"

Answer: "0Accepted AnswerUnfortunately, this is not available out of the box via Glue Studio UI. Such custom use cases require editing the code as you mentioned and will disable the automatic generation of code graphically.CommentShareSUPPORT ENGINEERManu_Ganswered a year ago"


Question: Static website in S3 not working with HTTPS

"I have a static website in S3 and it only works with HTTP. http://testing9598.s3-website-eu-west-1.amazonaws.com I can see the index.html content but if I use the HTTPS variant, it won't load. I've basically created the S3 without any policy and just given public access. Then I followed this guide https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-https-requests-s3/ without using AWS Route 53 and used AWS Cloud Front with the option for SSL Certificate as Default CloudFront Certificate (*.cloudfront.net). Can someone guide me on what am I missing?FollowComment"

Answer: 0The problem relied in me using the S3 endpoint instead of the CloudFront one.CommentShareeliasaws67answered 3 years ago


Question: Issues consuming AWS API Gateway API

"HiI have a simple HTTP API created using AWS API Gateway that uses a lambda integration to returns some data.I have also configured it with a custom DN using route53 (CNAME)Recently I have been getting the following error when making a call to the endpointError: Hostname/IP does not match certificate's altnames: Host: xxxxxx. is not in the cert's altnames: DNS:*.execute-api.eu-west-2.amazonaws.comCan anyone help with why this is happening? I have setup a certificate for my custom domain using AWS certificate manager as well so its all AWS services, but for some reason its just stopped working?ThanksAndrewFollowComment"

Answer: "0I am weirdly getting this issue intermittently, when I make a call to the API in a browser I get the following error:This server could not prove that it is api.xxxx.co.uk; its security certificate is from *.execute-api.eu-west-2.amazonaws.com. This may be caused by a misconfiguration or an attacker intercepting your connection.Then it goes away and it works again? HUH? Any ideas?AWS Support, I am using all your services for this but for some reason there seems to be an issue, can someone please help?CommentShareandrewkew-qcanswered 3 years ago0Forget it I found someone on stackoverflow with the same problem.Seems to be down to the route53 record. Instead of a CNAME to the API invoke URL it needs to be an ALIAS A record to the region URL of the APII have detailed the solution herehttps://stackoverflow.com/a/63173982/966911Problem solved.CommentShareandrewkew-qcanswered 3 years ago"


Question: Textract not extracting all the tables

"I am using laravel and jscript for extracting image dataMy single page image has 7 tables, textract extracting only first 5 table. 6th and 7th table is not extracting and returning table data as line block. Is there any restriction for table extraction. How can i read label and its value from line block.Please help me , i am stuck in this issueFollowComment"

Answer: "0Hi,Thanks for reaching out to us regarding your issue. In order to answer the best way possible to your query, can you please reach out to your customer support and send example of such a document. It will help us find the best solution for your case.Thanks for your understandingCommentShareCyprien_Danswered 5 months ago"


Question: MTLS availability in all regions for API gateway.

"Our team has a service that will be deployed in all regions, commercial, china and ADC. Our solution depends on API gratway MTLS which we came to find out is not available for some regions like MEL, the public documentation says its available.How can I confirm availability for all regions? And what would be a good alternative that is available in all regions?FollowComment"


Question: MPEG-DASH stream using VP9 codec

"HiI want to create DASH streams using VP9.But when I used MediaConvert to transcode an input mp4 (avc codec) to output DASH (VP9 codec), I'm not able to play the stream.  I've successfully created DASH streams with AVC codec before and they play fine.   Is there something I need to do differently for VP9?Any help is appreciated.Thanks!FollowComment"

Answer: "0HiThank you for your questions. May I know the following?Which player did you use for testing the output stream?Could you send me your aws account ID, mediaconvert job ID and region in "private message"?RegardsCommentShareawsmicmicanswered 2 years ago0HiOne more thing to remind, if you use s3 as streaming origin, please be reminded to enable the CORS in your S3 bucket: https://docs.aws.amazon.com/AmazonS3/latest/userguide/enabling-cors-examples.htmlRegardsMichaelCommentShareawsmicmicanswered 2 years ago0Thank you Michael.  CORS is already enabled on the S3 bucket.  As an experiment, I uploaded a VP9 encoded WebM (from public test source) and I'm able to stream it.  But when I try to create a new VP9 Dash stream , I'm not able to stream it.  I tried converting different inputs too, like mp4/AVC -> Dash/VP9, but in all these instances, I'm not able to stream the content.  I think the media convert step must be the cause, but I'm not sure what change I need to make in the convert job.CommentShareTestStreameranswered 2 years ago0HiWe apologize for the inconvenience while using MediaConvert. We have identified the issue and are working to fix it. Unfortunately we cannot give a timeline on when this will be available. We will inform you as soon as we can. At this moment, please avoid to use $dt$ name modifier for VP9 codec. Instead, you can either choose to use $Number$ or $Time$ in name modifier.RegardsMichaelCommentShareawsmicmicanswered 2 years ago0Thank you Michael!  Removing the $dt$ modifier resolved my issue.Thanks again for the quick reply!CommentShareTestStreameranswered 2 years ago"


Question: Cannot restart EC2 instance

HelloI can't re-start my EC2 instance. It was unexpectedly found in a stopped state and cannot be restarted.Instance: i-08f3186ff42xxxxxxAvailability zone: us-east-1bAWS forum chat for a similar problem  (threadID=151576) suggested un-attaching and re-attaching the volume.I unattached the volume: vol-038ebe6e8bdxxxxxx I cannot re-attach it - the option is greyed-out. Trying to Force Detach Volume results in error:'xxx: Volume xxxx is in the 'available' state'.thanks for any adviceFollowComment

Answer: "0Will try Azure, thanks.CommentSharecharlesmianswered 4 years ago0Hello,I can see that the EBS volume vol-038ebe6e8bdxxxxxx is, unfortunately, showing the 'error' status. The 'error' status indicates that the underlying hardware related to your EBS volume has failed, and the data associated with the volume is unrecoverable [1].If you have an EBS snapshot of the volume, you can restore that volume from your snapshot.If you don't have an EBS snapshot of the volume, create a new EBS volume, and then restore the data to it using the manual backup solution you prefer.It’s a best practice to keep backups of your EC2 resources, including EBS volumes. For more information, see Best Practices for Amazon EC2 [2].Note: You are not billed for volumes with a status of error.Please accept our apologies for the inconvenience that this has caused to you.Links:[1] https://aws.amazon.com/premiumsupport/knowledge-center/ebs-error-status/[2] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-best-practices.htmlCommentShareAWS-User-3892169answered 4 years ago"


Question: Network bandwidth performance

We have deployed 2 ec2 instances in same availability zone i.e. r5.2xlarge as per this instance features it has capability of up 10 GB network performance I'm confuse about its bandwidth calculation. Can someone give clarity on below points.When it says up to 10 GB network performance is that mean it will give constant speed of 10GBPS.?If I want to check maximum network bandwidth limit between 2 ec2 instances (Linux) how we can measure?Thanks in advance as this may help us for long term.FollowComment

Answer: "1Hello,The available network bandwidth of an instance depends on the number of vCPUs that it has. For example, an m5.8xlarge instance has 32 vCPUs and 10 Gbps network bandwidth, and an m5.16xlarge instance has 64 vCPUs and 20 Gbps network bandwidth. However, instances might not achieve this bandwidth; for example, if they exceed network allowances at the instance level, such as packet per second or number of tracked connections. How much of the available bandwidth the traffic can utilize depends on the number of vCPUs and the destination. For example, an m5.16xlarge instance has 64 vCPUs, so traffic to another instance in the Region can utilize the full bandwidth available (20 Gbps). However, traffic to another instance in a different Region can utilize only 50% of the bandwidth available (10 Gbps).Typically, instances with 16 vCPUs or fewer (size 4xlarge and smaller) are documented as having "up to" a specified bandwidth; for example, "up to 10 Gbps". These instances have a baseline bandwidth. To meet additional demand, they can use a network I/O credit mechanism to burst beyond their baseline bandwidth. Instances can use burst bandwidth for a limited time, typically from 5 to 60 minutes, depending on the instance size. You could also consider scaling horizontally, by adding an instance. This would also give you an opportunity to use multiple availability zones and remove a single point of failure.An instance receives the maximum number of network I/O credits at launch. If the instance exhausts its network I/O credits, it returns to its baseline bandwidth. A running instance earns network I/O credits whenever it uses less network bandwidth than its baseline bandwidth. A stopped instance does not earn network I/O credits. Instance burst is on a best effort basis, even when the instance has credits available, as burst bandwidth is a shared resource. R5 is a memory optimized instance.Regarding bandwidth on the EC2 instances: Whilst some instance types can clearly deliver 10Gbps or more of bandwidth you can't use all of this for internet-bound traffic. You should expect to see the following limits on internet egress bandwidth per instance:Single Flow from one instance - 5GbpsMultiple Flows from one instance - 5GbpsIn many scenarios, multiple flows from an instance can hit much higher limits (e.g. 25Gbps to S3, or over VPC Peering links), but for Internet egress there is a 5Gbps cap per instance. You can find this in the public documentation here:Network traffic outside a cluster placement group (e.g. to the Internet) is limited to 5 Gbps (full duplex)Refer below for Network bandwidth limitation- https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-network-bandwidth.htmlR5 instance limitation with respect to EBS bandwidth & Network performance- https://aws.amazon.com/ec2/instance-types/r5/CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 9 months agoAWS-User-6176623 9 months agoThanks for your answer. But when we say up to 10 GBPS what would it's baseline performance?ShareUser-5519327 3 days agoYour copy/paste of the official documentation and original source mention that the burstable instances have baseline bandwidth. Where can I find the actual values for baseline bandwidth for the general purpose instances?And what is the way to check or monitor Network Bandwidth credits balance. I can't seem to find CloudWatch metric for that (unlike CPU credits).Could you please help with this?Share0Network bandwidth may be affected by traffic from other processes.  One possible option is to use the HPC stack (https://aws.amazon.com/hpc/ see information on Elastic Fabric Adapter which has bandwidth upto 100GB/s). On this one can run MPI bandwidth measurements (http://mvapich.cse.ohio-state.edu/benchmarks/ or https://github.com/intel/mpi-benchmarks), to get some idea of what can be achieved, though your software stack will also impact this.CommentSharebakwmsanswered 9 months ago"


Question: one user two resource groups

we have two resource groups that swap resource allocation at a certain time at night and in the morning.We have asked the dba to place a certain user in the high resource allocation resource group at night and the high resource resource group in the morning so it always has high resource.  They instead added the user to both resource groups.FollowComment

Answer: "0Accepted AnswerAre you using Manual WLM configuration, or Auto? I recommend using the Automatic WLM which makes sure that the cluster is fully utilized and gives you additional benefits such as automatic concurrency management and query priorities. (https://docs.aws.amazon.com/redshift/latest/dg/automatic-wlm.html)In any case, in your situation, the order in which the queries are routed to the right queue is determined by the order in which they are defined in the configuration. So it will use the first match in that order.If you need to disambiguate between the two queues, you can use query groups. That way, you have control from your application over which queue to run in by setting the right query group.https://docs.aws.amazon.com/redshift/latest/dg/automatic-wlm.html#wlm-auto-defining-query-queues-query-groupsCommentShareAWS-User-0226321answered 3 years ago0got kicked out before finishing.  is this a good practice and what resources would the user getCommentSharemjmebesanswered 3 years ago0thanks for the reply.  i'm not a dba but i think they are setting things manually.  i think i will try to get them to flip the group for the user based on time of day so there is no ambiguity.thanksagainCommentSharemjmebesanswered 3 years ago"


Question: Can IAM policy be restricted to just temporary sessions (GetSessionToken)?

"I want to create an IAM account that should just be used to create temporary session tokens and ListBucket in S3. The temporary sessions should then also be able to PutObject, GetObject and DeleteObject. I don't think this is possible with GetSessionToken since the temporary session would have the same restrictions as the original account. But I wasn't sure if there was some special principal or other conditions (like checking for a session token) that might make this work?This is my basic policy...{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "AWS": "SOME MAGIC SUGAR HERE???"      },      "Action": [        "s3:PutObject",        "s3:GetObject",        "s3:DeleteObject"      ],      "Resource": "my_s3_bucket_arn/*"    },{      "Effect": "Allow",      "Principal": {        "AWS": "SOME MAGIC SUGAR HERE???"      },      "Action": [        "s3:ListBucket"      ],      "Resource": "my_s3_bucket_arn"    }  ]}Do I need to use AssumeRole here instead so the temporary session gets different permission?Thanks!FollowComment"

Answer: "0Accepted AnswerHello,If I understood correctly, you would like to allow S3 actions only if the requester is using temporary credentials.In this case, you can use the "Null" condition operator [1] to check the existence of the "aws:TokenIssueTime" [2] Global condition key; as stated at the documentation [1], "If the user is using temporary credentials, then the key aws:TokenIssueTime exists and has a value."The same documentation [1] brings an example which "the user MUST NOT be using temporary credentials (the key must not exist) for the user to use the Amazon EC2 API." (Which is the opposite of your case).If you are planning to attach your policy to an IAM user, it should be like:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "s3:PutObject",                "s3:GetObject",                "s3:DeleteObject"            ],            "Resource": "my_s3_bucket_arn/*",            "Condition": {                "Null": {                    "aws:TokenIssueTime": "false"                }            }        },        {            "Effect": "Allow",            "Action": [                "s3:ListBucket"            ],            "Resource": "my_s3_bucket_arn",            "Condition": {                "Null": {                    "aws:TokenIssueTime": "false"                }            }        }    ]}Note: IAM identity-based policies (attached to IAM users, groups or roles) don't have the "Principal" element [3].The above policy is checking if the "aws:TokenIssueTime" IS NOT Null, which means the entity should be using temporary credentials; in this case, the user will be able to call the allowed S3 API actions if it is using temporary credentials obtained using "GetSessionToken" or Assuming a role.I hope this has answered your question!References:[1] IAM JSON Policy Elements: Condition Operators  - Condition Operator to Check Existence of Condition Keys  - https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_Null[2] AWS Global Condition Context Keys - aws:TokenIssueTime - https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-tokenissuetime[3] AWS JSON Policy Elements: Principal  - https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.htmlCommentShareJulioHenriqueanswered 3 years ago0Thanks. That was exactly what I was looking for.CommentShareTedOCanswered 3 years ago"


Question: Does AWS Glue support fixed-byte-length source data?

"Hello.Can AWS Glue read source data file like below?  20220101E00011000AAABBBCCC  20220101E00021000あいCCCThe second record contains Japanese characters, and is the same byte length with the first record.The numbers of characters vary by record.Thank you.FollowCommentrePost-User-2300507 9 months agoIf the numbers of characters are the same by record, I can use Grok pattern like "(?<col0>.{6})(?<col1>.{2})..." when making a data catalog using crawler. But in this case, numbers differ, and bytes are the same. Can anyone tell AWS Glue does support/not support  data set like this?Share"

Answer: 0Accepted AnswerUnfortunately parsing data by Bytes right now is not supported in glue. I observed your data is unstructured and the only way to parse that data is Grok SerDe or Regex SerDe but they parse data by identifying patterns so they are not feasible. I will recommend you to preprocess your data and then load them in glue.unstructured data -> preprocessing using some Custom built parser function(csv) -> S3 -> crawl and createDatabase in glue.CommentShareSUPPORT ENGINEERShubhanswered 9 months agorePost-User-2300507 9 months agoThank you for your reply.Share


Question: Where do I find the URL to my DynamoDB API?

"How am I missing this?  I just started a new project and was able to easily create a DynamoDB table, but I cannot for the life of me find the API URL?  All I see is my table resource name, and my Amazon Resource Name (ARN), which is not very helpful.  I've searched for tutorials or documentation that would explain step 2 for me, but for me step two is accessing my API via Postman, but I cannot do that if I cannot find something as simple as the API URL.  Can someone point me to where in the UI I find this information?Edited by: szaske on Sep 17, 2017 6:33 PMEdited by: szaske on Sep 18, 2017 6:14 AMFollowComment"

Answer: "0Hi szaske,These are the standard endpoints for DynamoDB: http://docs.aws.amazon.com/general/latest/gr/rande.html#ddb_regionClarifying questions:Are you running your own servers?Which platform are you programming on? Java? Android? iOS? Python?CommentShareAWS-User-7129927answered 6 years ago0I think you need to take a look at this page about REST API callshttp://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.LowLevelAPI.htmlCommentSharedariosotoanswered 6 years ago0This was helpful.CommentShareZManswered 4 years ago"


Question: describe-transit-gateways get transit-gateway-ID using its description row

"So, i wanted to get transit-gateway-id using description line (or maybe better to use tag?) that i have added to it when i was creating the transit-gateway. Now im trying to get transit-gateway-id using this command aws ec2 describe-transit-gateways.But im stuck with using --filter, cant really understand if it is even possible.ThanksFollowComment"

Answer: "1Accepted AnswerUpon review of the documentation, it did not appear to be possible to filter by description field or tag.https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/describe-transit-gateways.html--filters (list)One or more filters. The possible values are:options.propagation-default-route-table-id - The ID of the default propagation route table.options.amazon-side-asn - The private ASN for the Amazon side of a BGP session.options.association-default-route-table-id - The ID of the default association route table.options.auto-accept-shared-attachments - Indicates whether there is automatic acceptance of attachment requests (enable | disable ).options.default-route-table-association - Indicates whether resource attachments are automatically associated with the default association route table (enable | disable ).options.default-route-table-propagation - Indicates whether resource attachments automatically propagate routes to the default propagation route table (enable | disable ).options.dns-support - Indicates whether DNS support is enabled (enable | disable ).options.vpn-ecmp-support - Indicates whether Equal Cost Multipath Protocol support is enabled (enable | disable ).owner-id - The ID of the Amazon Web Services account that owns the transit gateway.state - The state of the transit gateway (available | deleted | deleting | modifying | pending ).transit-gateway-id - The ID of the transit gateway.CommentSharetakakunianswered 7 months agoEXPERTAWS-User-Nitinreviewed 7 months agoEXPERTTushar_Jreviewed 7 months agoJoann Babak 7 months agoWelp, yeah, if it is the only possible values for filtering, then sad.Share"


Question: Deployment configuration for ECS Fargate Service

"We have build a tier-1 service and we want to ensure 100% availability during the deployment. Our Service needs 15 tasks to serve 850 tps traffic. We are looking for deployment configuration(1) Desired count is 15 as of nowTo ensure the service is always available I had set minimumHealthyPercent to 100%, but during deployment I had seen there is spike in the unhealthy hosts.(2) what should be the minimumHealthyPercent ?(3) what should be maximumPercent ?(4) Should we modify the health check associated with target group ?FollowComment"

Answer: "2Set minimum to 100% and maximum at 200%, then ECS will scale up 15 new task, once they are ready they will replace the 15 old. No need to modify any health checksCommentShareEvolate-Timanswered 4 months ago"


Question: Regaring scheduling an event bridge trigger

"Hi All,I am having a requirement, which should to the following things:Call an external cliient api from AWS to fetch a file daily once.While fetching a file, the below are the scenarios:-- If fetching of the file from the external api  is failed in the first attempt, then check again in a interval of 15 minutes. Do this check like this for every 15 minutes upto 2 hours. If after 2 hours also, the file fetching is failed from the external api, then log the failure.-- If fetching of the file is sucussful, then stop calling the external api to fetch the file again, for that      day and push that file to store into s3.The Problem:As per inital design, I am using an aws event bridge scheduler to schedule for every 15 minutes to trigger the lambda upto 2 hoursThis lambda will call the external api and keep the file into s3.But one, thing which I am finding a bit hard to figure it out  is that, in case, -- If i  am successful, in retrieving the file from the external api, then how can I ask the aws event      scheduler to stop re-triggering for that day.  ---  If even after 2 hours, I am not able to fetch the file, then how could eventbridge or the lambda can log (because,   ---- 1) event bridge will not know anything about success of failure and it only knows that it. should trigger for every 15 minutes upto 2 hours        2) lambda is not aware, if the window scheduled for 2 hours is completed)Is there any better approach if any one can suggest  for the above.FollowComment"

Answer: "2Accepted AnswerI would use a different approach. Use EventBridge to schedule a daily Step Functions process. The Step Functions workflow will handle the retrieving of the file using a Lambda function. Step Functions has built in Retries. You will set the Interval to 900 seconds (15 minutes) maximum retries to 8 (first try + 8 retries = 2 hours). So a single step in Step Functions will retrieve the file as designed.CommentShareEXPERTUrianswered a year ago0Maybe you could trigger the lambda when the file is uploaded to S3. Then you could use a DynamoDB table to store how long since the last trigger if you want to write some logic around that.https://i.imgur.com/DsRsejU.pngCommentShareAWS-User-1answered a year agoAWS-User-4736214 a year agoMy question, is about the lambda hitting the external api with the event bridge scheduler for every 15 mins, to pull a file from an external api. If the lambda is successful, in retrieving the file, then how can I stop the event bridge scheduler to stop from further asking the Lambda to trigger the external api for that day.  Only if the external api call is failed, then only the even bridge scheduler should re-trigger in the next 15 minutes.Share"


Question: Transfer EC2 and Savings Plan from one AWS Account to another

"Since I am new with AWS, I just come across Organizations in AWS, and just created new OUs and AWS Accounts.Initially, I have an EC2 and a Savings plan under one account since I don't know Organizations then.Now that my main account is the Management Account, I want the EC2 instance and the Savings plan to be transferred to the new AWS Account under a specific OU. This is so that I can easily manage and determine the costs per OU.Is this possible?And if not, will this work:Savings Plan will stay in the Management Account (I can't recreate this one since the plan is already for 3 years);EC2 will be deleted on the Management Account;Recreate a new EC2 on the new AWS Account;with Consolidated Billing, the Savings Plan in the Management Account will take account for the EC2 on the new AWS AccountLet me know your thoughts on this.FollowComment"

Answer: "2Accepted AnswerHello @GGsent,Hope you are doing well.I see that you would like to know how the Savings plans work in a consolidated billing environment.Firstly, I would like to inform you that it is ideal that the Savings plans or Reserved instances contract to be purchased on the management account. This way, you can make sure that they do not go to waste if one of the member account that owns the SP or RI is shutdown or inaccessible for various reasons.On the management account you can enable Reserved Instances and Savings Plan discount sharing [1] for the specific member accounts that you wish the discounts to be applied for. As per the steps provided by the above answers, you will be able to migrate the instance to another member account but also make sure that the SP continues to apply for its usage.However, if you have member accounts that belong to different clients and would like to have different savings plans on different accounts, I recommend that you reach out to the Support team [2] and request them to transfer the SP to one of the member accounts.Please note that the team would only review the transfer of RIs or SPs within an Organisation. They do not support transfer between standalone accounts or from and Organisation to a standalone account.--References --[[1]]https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/ri-turn-on-process.html[[2]]https://console.aws.amazon.com/support/home#/case/createHope you found this information useful.Regards,Harshavardhan GCommentShareHarshaVardhanGanswered a year agoGGsent a year ago+1 for pointing out that it is ideal to have a Savings Plan under the management account. Thank you.ShareNataliya_G EXPERTa year agoPurchasing Savings Plans or Reserved Instances on the Management Payer account is NOT ALWAYS a good idea. We do say that the best practice is that customers should not run any workloads on the Management Payer account. But a lot of customers still do that. And because of this SPs are not used efficiently. It's better to create a dedicated account(s) within the Organization to be used solely for RI/SP management and purchases. This will ensure that SP discount will always apply to accounts and resources in Organization that provide you highest savings (if RI/SP sharing is enabled).Share2Thank you for using re:PostYou can indeed delete the ec2 from the management account and recreate it in your new account. You can also turning on shared reserved instances and Savings Plans discounts: https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/ri-turn-on-process.htmlEricCommentShareEricanswered a year ago1In order to simplify the migration of your EC2 instance, you can make a snapshot and share it with the new account.https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-modifying-snapshot-permissions.html#share-unencrypted-snapshotSavings Plans can be purchased in any account within an AWS Organization/Consolidated Billing family. By default, the benefit provided by Savings Plans is applicable to usage across all accounts within an AWS Organization/consolidated billing family.https://aws.amazon.com/savingsplans/faq/CommentShareEXPERTHernitoanswered a year ago0From the target account, find the AMI using the EC2 console or the AWS CLI. Launch a new instance from the shared AMI on the target account. To import a key pair from a previous account, see Import your own public key to Amazon EC2.CommentShareUsha kumarianswered a year ago"


Question: Does historical complaints percentage ever decrease in the AWS pinpoint?

"We just breached 0.1% warning level for aws pinpoint sending out emails and currently in the 'warning' level.I was reviewing our historic complaints percentage and it seems to only go up even after we take people who complain off the list (we get automated emails from SES about complaints (unless we aren't getting all of them)). Even after taking them off the sending lists, we see our historical complaints percentage stable and it only goes up when new complaint is email to us.What have you done to decrease your historical complaints percentage?FollowComment"

Answer: "0Accepted AnswerOk, this is what I got so far:in cloudwatch, I got my 'sent' parameter to csv so I could see 'compelete' number of emails sent, which was 59,944.Then I went to my email and looked at all complaints I received there. 6Which matches the rate displaed (historic complaints rate) 0.11% if you round up.So I assume my rate will go down as I keep sending more emails without receiving a single complaint.CommentShare7ruthanswered 8 months ago"


Question: Why is AWS::Lambda::EventInvokeConfig's limit for MaximumRetryAttempts 2?

"The documentation [1] states that MaximumEventAgeInSeconds can be up to 6 hours long. However, because MaximumRetryAttempts only goes up to 2 retries, the end result for an asynchronous invocation's age can only be about 3 minutes in practice. How can I use more of that age in seconds limit with only 2 retries?[1] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventinvokeconfig.html#cfn-lambda-eventinvokeconfig-maximumretryattemptsFollowComment"

Answer: "0When you invoke a function asynchronously, the event is added to an internal queue and then the Lambda service gets the messages from the queue and invokes the function. It will try to invoke the function up to a maximum of 3 times (2 retries - configurable) and if it fails, it will send the event to a DLQ (if configured).For different reasons, it may be that the event will remain in the queue and will not be handled immediately. In that case you can configure the max event age (up to the max of 6 hours, which is the default). If the event remains in the queue for longer that specified, it wil be dropped.CommentShareEXPERTUrianswered a year agoMetaSkills a year agoI am sorry, this is not an answer to the question. At best, perhaps your hand wave "different reasons" was an attempt. Could you please expand and re-read my question:Share0Thanks, I appreciate the help. Sadly this is the answer (throttling) I was dreading and it means I can not do an easy exponential backoff  [1] [2] for myself and the AMQP replacement with EventBride -> Lambda (w/Destinations) system I am building.If Destinations allowed me to have MaximumRetryAttempts more than 2 I could get something similar (up to 6 hours) to the service-level behavior for my client code. Building code like (https://bitesizedserverless.com/bite/async-lambda-function-retries-with-backoff-and-jitter/) is not a solution. All we need is more attempts.system errors (500-series), Lambda returns the event to the queue and attempts to run the function again for up to 6 hoursJust to make sure, when you say "system" you mean AWS service, not my client side 500 errors right?CommentShareMetaSkillsanswered a year agoskinsman EXPERTa year agoYes, AWS service, like described here - https://aws.amazon.com/premiumsupport/knowledge-center/lambda-troubleshoot-invoke-error-502-500/.Share0I know it's confusing because there are two types of retries.  Uri correctly pointed out that the age limit in the queue and retry limit are for different reasons, it's not about those retries "using that limit".As in https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html, retries (max 2) are for when your function returns an error.  The age limit is for this situation where many more than 2 retries may occur:If the function doesn't have enough concurrency available to process all events, additional requests are throttled. For throttling errors (429) and system errors (500-series), Lambda returns the event to the queue and attempts to run the function again for up to 6 hours. The retry interval increases exponentially from 1 second after the first attempt to a maximum of 5 minutes. If the queue contains many entries, Lambda increases the retry interval and reduces the rate at which it reads events from the queue.CommentShareEXPERTskinsmananswered a year ago"


Question: "I have a sagemaker account setup in US-WEST-2 region. I have successfully trained and deployed my model; however, how can I chose to deploy the endpoint in a different region?"

"Assume I have a sagemaker account in one region where I have all my code, data and model artifacts etc..I have been able to successfully deploy these models using the estimator.deploy() API after calling the estimator.fit() method.However, what If I dont want to deploy the model in the same region as my sagemaker account and want to deploy the endpoint in another region, how can I do that?FollowCommentVictor Ghadban EXPERT2 months agoWith AWS Model registry you can share models accross accounts.  You can register your model in the region you choose then deploy it from there.Share"

Answer: 0You can specify the region using either AWS SDK for Python (Boto3) or SageMaker Python SDK. Amazon SageMaker documentation provide a step by step guide for each one of them.You needSageMaker region nameSageMaker execution roleYour frameworkmodel data pathCommentShareZmnakoanswered 2 months ago


Question: Scheduled maintance for Amazon Lightsail instances

"Hello all,I got a message from AWS that there is a maintenance window for Friday 10 Feb 14:00 GMT for two hours.How can I imagine this? Is my instance really offline for 2 hours? That can't be, can it?Do I have any possibilities to postpone the whole thing until the evening?Here is the mail from AWS.*Dear Amazon Lightsail customer,One or more of your Amazon Lightsail instances will be undergoing maintenance in a 2-hour window starting Fri, 10 Feb 2023 14:00:00 GMT.The maintenance will cause the following Lightsail instance(s) to be unavailable in the eu-central-1 region and then restart:amzls-custom-eu-central-1-2020-10-17-15-58-50-0xxxxxxxxxIf you have any questions or concerns, please contact the AWS support team on the community forums and via AWS Premium Support at: http://aws.amazon.com/supportSincerely,Amazon Lightsail TeamAmazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc, 410 Terry Ave. North, Seattle, WA 98109-5210*Thank you for your helpFollowComment"


Question: AWS Lambda outbound connection timeout

"Our team has created a Lambda Function URL to manage background tasks in a queuing fashion. The Lambda essentially schedules the kick off of these tasks by simply calling a RESTful API request to and external endpoint still hosted on AWS. However, we are getting outbound connection timeout to the external endpoint (and basically anywhere) time to time and then unable to send outbound requests for a few hours, eg: after sending 35 requests in 2 hours we are unable to connect for about 3hours. I suspect this being caused by some sort of limit or quota being reached within VPC configuration but I cannot make any sense of this as we don't get any meaningful warning or error. Removing the VPC does solve the timeout problem.A bit more details about the setup: Lambda has got an RDS attached where it stores the state and updates of last runs and the Lambda is configured with VPC to get access to the RDS instance with the outbound rules such that allows all protocol on all ports. The frequency of the outbound API calls is 5mins with a very light payload, which normally gets a response within 50ms before timeout happens.What are we doing wrong with the VPC config and shouldn't we get some warning/emails when quotas/limit are reached?ThanksFollowComment"

Answer: "0Accepted AnswerIt's odd that this works and then doesn't; then works again.From your question I want to clarify: It sounds like the Lambda function is attached to a VPC - so I'm going to go with that assumption.When you associate your Lambda function with a VPC you nominate the subnets that it should be attached to. From time to time your Lambda function may run in a different back-end context (i.e. different physical server; maybe even a different Availability Zone) and at that time it will use a different subnet.Is it possible that your subnets are not created "equal"? As in, there's a subnet that's working correctly; but another that isn't? It would be easy to test: Start an EC2 instance in each subnet and try the REST API call to see what result you are getting.If all is well, I'd suggest a creating a support case to troubleshoot more.CommentShareEXPERTBrettski-AWSanswered 3 months ago0If your function is attached to a VPC as implied by your question, make sure that all subnets the function is attached to has a route the NAT Gateway for 0.0.0.0/0.CommentShareEXPERTUrianswered 2 months ago0Hi,So the issue was indeed with the network configuration of the VPC.  Th VPC we have for the AWS Lambda has 3 availability zones and the Subnets was configured by adding both the 3 private and 3 public subnets, which as I understand is wrong since it's the NAT which translates the private IPs and there is no need to add the public subnets (the Lambda Function only connects to the RDS instance).Thank you for your  help, really appreciated!CommentSharerePost-User-8467086answered 2 months ago"


Question: AWS IQ Experts Fee Charges

"Hi,Recently I have taken a service from AWS IQ , and the proposal of the expert cost me 75$.I have some business credits in my AWS Activate account.  Now, where this money will get deducted, from my Credit card attached to my account or from my credits?Eagerly awaiting someone's reply.ThanksFollowComment"

Answer: 0Credits cannot be used for AWS IQ as described in the following document.https://aws.amazon.com/iq/experts/faq/?nc1=h_lsQ. Can AWS Promotional Credits be used on IQ?Customers can not use credits to pay experts on IQ. Customers may use credits for eligible resources that are spun up in their AWS environment. Learn more about AWS Promotional Credits.CommentShareEXPERTRiku_Kobayashianswered 2 months agoEXPERTkentradreviewed 2 months ago


Question: How can software code get online sampling rules?

"https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-dotnet-configuration.htmlThe configuration method for .NET apps only shows how to grab rules from a local JSON file.But how can an app query and grab sampling rules already defined beforehand in AWS console?And beyond that, if the online sampling rules are adjusted, do they get dynamically propagated to all existing related app systems?FollowComment"

Answer: "0Hi,As long as your application has X-Ray SDK instrumented, the SDK will periodically check X-Ray backend to fetch the centralized sampling rule that you defined in X-Ray console.CommentShareaws-luanswered 2 years ago0But how do we grab specific sampling rules? We have multiple systems, tiers, environments and they won't be operating with the same sampling rate.CommentShareicelavaanswered 2 years ago0Hello,If you would like to create a custom solution involving manually retrieving your sampling rules, you can call the GetSamplingRules API: https://docs.aws.amazon.com/xray/latest/api/API_GetSamplingRules.htmlThis will return all sampling rules you've created on the X-Ray console, and you can select the one that you'd like to apply.CommentShareAWS-User-2079090answered 2 years ago0For ASP.NET Core app, the Startup.Configure() method registers the app's main segment name.app.UseXRay("APPSERVICENAME", this.Configuration);A sampling rule with Service name equaling that app name must be created for it to be fetched and applied automatically.HOWEVERMy sample infrastructure is multi-tier; there is another back-end web app that receives HTTP requests from the front-end web app. The back-end service has its own sampling rule. But on inspecting the traces, it appears only the front-end sampling rule was applied. Is it because the back-end is considered a sub-segment and thus slaved to the front-end's sampling rule?{"Id": "1-61adbe98-2dbb90b3efdd86e20c536db3","Duration": 0.544,"LimitExceeded": false,"Segments": [{"Id": "9ec8aad6f0dfe2d6","Document": {"id": "9ec8aad6f0dfe2d6","name": "XRayBackApi-exp","start_time": 1638776472.237228,"trace_id": "1-61adbe98-2dbb90b3efdd86e20c536db3","end_time": 1638776472.752401,"parent_id": "2ac704e5eba6c989","http": {"request": {"url": "http://localhost:8081/api/Tier","method": "GET","client_ip": "::1"},"response": {"status": 200}},"aws": {"xray": {"sdk_version": "2.10.1","sdk": "X-Ray for .NET Core"}},"annotations": {"System": "XRay-exp"},"service": {"runtime": ".NET Core Framework","runtime_version": "3.1.21"}}},{"Id": "a70b7cc33c3e6d9a","Document": {"id": "a70b7cc33c3e6d9a","name": "XRayFrontWeb-exp","start_time": 1638776472.217652,"trace_id": "1-61adbe98-2dbb90b3efdd86e20c536db3","end_time": 1638776472.761628,"http": {"request": {"url": "http://localhost:8080/Home/MultiTierResponsiveness","method": "GET","user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36 Edg/95.0.1020.53","client_ip": "::1"},"response": {"status": 200}},"aws": {"xray": {"sampling_rule_name": "XRayFrontWeb-exp","sdk_version": "2.10.1","sdk": "X-Ray for .NET Core"}},"annotations": {"System": "XRay-exp"},"service": {"runtime": ".NET Core Framework","runtime_version": "3.1.21"},"subsegments": [{"id": "2ac704e5eba6c989","name": "localhost","start_time": 1638776472.219333,"end_time": 1638776472.752914,"http": {"request": {"url": "http://localhost:8081/api/Tier","method": "GET"},"response": {"status": 200}},"namespace": "remote"}]}}]}CommentShareicelavaanswered a year ago0Tested a separate PHP-Laravel front-end tier which does not have sampling rule and omitted the Sampled=1 parameter from the X-Amzn-Trace-Id HTTP header for back-end web requests. With this setup, the back-end can apply its own sampling rule.{"Id": "1-61adc749-e0aa74be2a8ed62ace626b04","Duration": 1.5,"LimitExceeded": false,"Segments": [{"Id": "561ad6c8c7cb840f","Document": {"id": "561ad6c8c7cb840f","name": "XRayLaravel-exp","start_time": 1638778697.660007,"trace_id": "1-61adc749-e0aa74be2a8ed62ace626b04","end_time": 1638778699.089339,"http": {"request": {"url": "http://localhost:9091/traceapi","method": "GET","client_ip": "127.0.0.1"},"response": {"status": 200}},"annotations": {"System": "XRay-exp","Framework": "Laravel 8.68.1"},"subsegments": [{"id": "54675e8f2c067792","name": "laravel boot","start_time": 1638778697.589603,"end_time": 1638778697.677686,"subsegments": [{"id": "e7c3d03023f9d693","name": "route matching","start_time": 1638778697.677683,"end_time": 1638778697.681337,"annotations": {"route": "AppHttpControllersXRayController@trace"}}]},{"id": "91a295beee84cb18","name": "request handled","start_time": 1638778697.681333,"end_time": 1638778699.089067,"metadata": {"info": {"controller": "AppHttpControllersXRayController@trace"}},"subsegments": [{"id": "4dd5a4d45a805237","name": "XRayController","start_time": 1638778697.688244,"end_time": 1638778699.085528},{"id": "eefd7b032b90997b","name": "View traceapi","start_time": 1638778699.086454,"end_time": 1638778699.087839}]}]}},{"Id": "6b5b0eba10878dc1","Document": {"id": "6b5b0eba10878dc1","name": "XRayBackApi-exp","start_time": 1638778697.717974,"trace_id": "1-61adc749-e0aa74be2a8ed62ace626b04","end_time": 1638778699.083292,"parent_id": "4dd5a4d45a805237","http": {"request": {"url": "http://localhost:8081/api/Tier","method": "GET","user_agent": "GuzzleHttp/7","client_ip": "::1"},"response": {"status": 200}},"aws": {"xray": {"sampling_rule_name": "XRayBackApi-exp","sdk_version": "2.10.1","sdk": "X-Ray for .NET Core"}},"annotations": {"System": "XRay-exp"},"service": {"runtime": ".NET Core Framework","runtime_version": "3.1.21"}}}]}So it does appear that back-tier sub-segments are slaved to the parent segment sampling rule.CommentShareicelavaanswered a year ago"


Question: "When AppFlow writes parquet to S3, can it maintain the source datatypes or not?"

"In the AppFlow UI, it appears to indicate that maintaining the source datatypes is possible via the option underneath parquet:However, in documentation (https://docs.aws.amazon.com/appflow/latest/userguide/s3.html), it states:If you choose Parquet as the format for your destination file in Amazon S3, the option to aggregate all records into one file per flow run will not be available. When choosing Parquet, Amazon AppFlow will write the output as string, and not declare the data types as defined by the source.These two sources conflict with each other. The behavior I am seeing is what the documentation describes and all data is being written as string type. I am trying to determine if this is intended or a bug. If the latter, I can open a support ticket.FollowComment"

Answer: "0Hi there, As mentioned When choosing Parquet, Amazon AppFlow will write the output as string and not declare the data types as defined by the source. That means it will not declare any data as per source code but will write the data as string and do not declare any other data type. I hope it would be clear now. If you still have queries please feel free to reach us via case. Thank you!CommentShareSUPPORT ENGINEERManish_Aanswered 6 months agotjtoll 6 months agoWhat is the option "Preserve source data types in Parquet format" for? Trying to understand if I can keep source data types somehow.Share"


Question: "Installation of traefik, prometheus, grafana on ec2 instance"

"I have to install Traefik, Prometheus, and Grafana on the "ubuntu ec2 instance" using the docker-compose file, can anyone provide a link or give supportFollowComment"


Question: Why my instance suddenly cannot be accessed via SSH?

"In the past, I was able to connect to my instance using SSH, and even used VS Code remote-ssh to connect without any issues. However, today I suddenly cannot connect anymore. When I try to connect using Terminal, I get a message saying the connection is refused. I checked the security rules for my instance, and my IP address is already added to the inbound rules, so it shouldn't be blocked by the firewall. What should I do?FollowComment"

Answer: "0Hi, both status checks of my instance show normal, and there is no indication of abnormalities. However, the website still cannot be accessed, which is somewhat strange. It seems that AWS customer service requires payment to assist with this issue.CommentSharerePost-User-8082621answered 2 months agoRiku_Kobayashi EXPERT2 months agohttps://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-console.htmlHave you tried checking the logs using the "instance console output" method?SharerePost-User-8082621 2 months agoHi, I tried.There is nothing error on it.Starting Postfix Mail Transport Agent...[  OK  ] Started Postfix Mail Transport Agent.[   16.383767] cloud-init[1444]: Cloud-init v. 23.1.1-0ubuntu018.04.1 running 'modules:config' at Sun, 09 Apr 2023 12:35:29 +0000. Up 16.25 seconds.[  OK  ] Started Apply the settings specified in cloud-config.Starting EC2 instance hibernation setup agent...[  OK  ] Started EC2 instance hibernation setup agent.[  OK  ] Reached target Multi-User System.[  OK  ] Reached target Graphical Interface.Starting Update UTMP about System Runlevel Changes...Starting Execute cloud user/final scripts...[  OK  ] Started Update UTMP about System Runlevel Changes.[   17.266236] cloud-init[2026]: Cloud-init v. 23.1.1-0ubuntu018.04.1 running 'modules:final' at Sun, 09 Apr 2023 12:35:30 +0000. Up 17.10 seconds.[   17.278597] cloud-init[2026]: Cloud-init v. 23.1.1-0ubuntu0~18.04.1 finished at Sun, 09 Apr 2023 12:35:30 +0000. Datasource DataSourceEc2Local.  Up 17.25 seconds[  OK  ] Started Execute cloud user/final scripts.[  OK  ] Reached target Cloud-init target.2023/04/09 12:35:30Z: Amazon SSM Agent v3.1.1732.0 is running2023/04/09 12:35:30Z: OsProductName: Ubuntu2023/04/09 12:35:30Z: OsVersion: 18.04Ubuntu 18.04.6 LTS ip-172-31-13-57 ttyS0ip-172-31-13-57 login:ShareRiku_Kobayashi EXPERT2 months agoI see from the logs that the SSM agent is installed.If that's the case, try logging in with session manager and checking the sshd status, etc.The Session Manager can be used in the following ways.https://docs.aws.amazon.com/ja_jp/systems-manager/latest/userguide/session-manager.htmlShare0Has your IP address changed, etc.?Check your IP address to be sure.It is recommended to check network ACLs and security groups afterwards.https://repost.aws/knowledge-center/ec2-linux-resolve-ssh-connection-errorsIf this does not solve the problem, you can also try to connect using Session Manager.https://docs.aws.amazon.com/ja_jp/systems-manager/latest/userguide/session-manager.htmlCommentShareEXPERTRiku_Kobayashianswered 2 months ago0How can I check the firewall settings?I have checked my security group, and the settings seem normal:sgr-050eaa7e60aa4aa89 IPv6 HTTPS TCP 443 ::/0 ––sgr-02fb68a313da3f4bc IPv4 SSH TCP 22 0.0.0.0/0 ––sgr-02634c3a46d3011ce IPv4 HTTPS TCP 443 0.0.0.0/0 ––sgr-0d58092cd29d7b455 IPv4 Custom TCP TCP 34210 0.0.0.0/0 runcloud–sgr-0aa5912a1d4962712 IPv6 HTTP TCP 80 ::/0 ––sgr-0751e7b1cb5ff144e IPv4 HTTP TCP 80The worst thing is that after I restarted the instance, even the website inside cannot be accessed anymore, and it seems that AWS charges a fee to help fix the problem.CommentSharerePost-User-8082621answered 2 months ago0Doesn't sound like a problem with security groups, etc.Are there any abnormalities in the EC2 status?https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-system-instance-status-check.htmlThe following documents are common troubleshooting steps to take when you cannot connect to an instance.https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-console.htmlFollow the instructions in the "Instance console output" section and check the logs.If sshd is stopped for some reason, the instance must be restarted.CommentShareEXPERTRiku_Kobayashianswered 2 months ago0The Session Manager can be used in the following ways.https://docs.aws.amazon.com/ja_jp/systems-manager/latest/userguide/session-manager.htmlAfter logging in with Session Manager, check the status of sshd with the following command.sudo systemctl status sshdIf it is stopped, restart sshd with the following command.sudo systemctl restart sshdCommentShareEXPERTRiku_Kobayashianswered 2 months ago"


Question: IAM permissions for AWS Backup Lock in governance mode

"HelloAccording to what we can read in the documentation (Vaults locked in governance mode can have the lock removed by users with sufficient IAM permissions), I would like to create a user account that will have permissions to remove the lock while ensuring that none of the other administrator accounts have such permissions.How should I configure permissions on the privileged account and on the other administrator accounts?https://docs.aws.amazon.com/aws-backup/latest/devguide/vault-lock.htmlFollowComment"

Answer: 0Short answer is to restrict which can "backup:DeleteBackupVaultLockConfiguration"https://docs.aws.amazon.com/service-authorization/latest/reference/list_awsbackup.html#:~:text=DeleteBackupVaultLockConfigurationProbably want to layer an Organizational SCP with DENY with condition ArnNotEquals for the arn of the user who you allow to delete the vault lock.CommentSharerePost-User-1635376answered 6 months ago


Question: AWS Batch Jobs on EKS Cluster

"Hi Team,I'm trying to setup aws batch on eks cluster.Created EKS cluster in default vpc in public subnets  and no node groupsAfter EKS Active. Created a Compute Environment its validCreated Job QueueCreated Job DefinitionAll are validBut it's in runnable state not getting to running state   or succeeded stateAny suggestions on this issueFollowComment"

Answer: "0Hi,It might be for different reasons, insufficient resources, limit reached....Can you check the links below :https://docs.aws.amazon.com/batch/latest/userguide/troubleshooting.html#job_stuck_in_runnablehttps://aws.amazon.com/premiumsupport/knowledge-center/batch-job-stuck-runnable-status/Regards.CommentShareMed Agouanswered 5 months ago"


Question: Objects used in queries

"Hello, is there any way to find out the list of all the tables (including external tables) used in the sqls executed in redshift. What I am looking is if there is any view or table which says this query has accessed these tables, if there is such view/table it would be easy to extract the info I am looking for. Otherwise I would need to go query history and manually get the objects accessed in each query.FollowComment"

Answer: "0Accepted AnswerHi nmakb,I believe you can construct query or view for what you're looking by basing a query off of a UNION of  query system log table pg_catalog.stl_scan jointed to a few other table for all table in Redshift storage UNIONEed with a query of svl_s3query smilarly joined to a few other tables to get the query text and the table and schema name.Basically stl_scan tracks all table scan operators against table in Redshift storage for in all query plans and svl_s3query effectively does the same because all Spectrum sub-queries are single table in nature and produce equivalent scan results returned to the slices from the Spectrum layer.I hope this helps you develop the result set you're looking for.Regards,-KurtCommentShareklarsonanswered 4 years ago"


Question: What's really meaning of AWS Glue Error: ResourceNumberLimitExceededException

"Hi everyone,Recently, I faced the error ERROR - Failed to run aws glue job, error: An error occurred (ResourceNumberLimitExceededException) when calling the StartJobRun operation: Reached the limit while starting jon run while starting a run for job with name: my-job-name. while trying to start glue job via Airflow.After I made some researches on internet, I found this might cause from "AWS Quotas Limit"  but I don't really know which resource that I have exceeded?, error message and documentation (https://docs.aws.amazon.com/glue/latest/webapi/API_StartJobRun.html#API_StartJobRun_Errors)  didn't provide exact information that I can use it to open the request for escalating my quotas limit.Additional Information: Currently I have all 47 glue jobs.Please help.FollowComment"

Answer: "0Accepted AnswerHi,Can you check the AWS Service Quota for "Max concurrent job runs per account"? It might be 0 right now as there is an issue ongoing with several accounts were this limit sets to 0. And this then prevents any job from runninghttps://console.aws.amazon.com/servicequotas/home/services/glue/quotas (make sure it is the correct region)CommentShareLukasanswered 10 months agocodecollarV2 10 months agoFor my situation this solution is work, Thanks!SharecodecollarV2 10 months agoIn additional, you should checkout the quotas that should not be equal 0 e.g. max task dpus per account, and then request AWS support to set it back to defualt value, the problem should be resolved.Share0Please, check if somehow you are not exceeding some service quota for Glue. See that Max jobs per trigger is 50. You mentioned 47 jobs, which is really close and potentially your code could maybe is trying to trigger more than once? Just a guessing, without access to code to take a look.Some of them are soft limits that you can ask for an increase. If you believe there's some potential issue, please open a ticket to get support.CommentShareIbrahim Cesaranswered 10 months agocodecollarV2 10 months agoFor my situation the solution in first comment fitted, Thanks for kindly answer.Share0I'm getting the same issue, all of the glue jobs that used to run normally before failed to run today.Max concurrent job runs per account is 0, also Max task dpus per account is 0, what happend?CommentSharerePost-User-0106131answered 10 months agoLukas 10 months agoAWS had an issue affecting multiple accounts. Sadly this was not visible on any status page. We only learned about it because the limit did set to 0 for us and kept on being set to 0 every two hours after we requested the limit increase (and got an auto accept)Share"


Question: Server with multiple rooms

"Hello,I'm setting up GameLift, but I'm having some trouble wrapping my head around matchmaking and having servers that have multiple matches going on at the same time.I've made a card game where 2 players can play against each other in a room. Each server could hold lets say 200 rooms.I have an idea in my head on how this could work, but not sure if it would.Two players enter the matchmaking queue and get paired up. GameLift then finds a server with room for 2 players and sends the client the details. The clients connect to that server and are thrown into a room based on some id or key or something (not sure if GameLift could generate this?).FollowComment"

Answer: "0GameLift would allow you to:Match make 1v1 game sessions based on provided player attributes. Note: GameLift does not store player data for you, so you would need to track and update ELO rankings in something else assuming ranked matches are important to you.Each player would open a matchmaking ticket with the attributes required for your matchmaking rulesetFind open server processes to host the match and or scale a new server if required.Player's 'see' their tickets fufilled, ticket contains player session details to connect on.Thinks to note:GameLift currently has a restriction that a server can run only one game session (if running hosted servers in GameLift). You can pack multiple game servers per instance but currently a single server cannot host multiple concurrent game sessions.GameLift hosted servers are much cheaper if you can target Linux and Spot instances (ie tolerate some low interruption percentage)If your server requirements are lightweight, GameLift Realtime could be a better fit for you or the new AWS GameSparks service (note: there is not a confirmed launch date).CommentSharerePost-User-5332620answered 2 years ago0Thanks for the information!So it looks like I'd need to update my game server to not host multiple rooms, but instead host a single match of 2 players.The current server build holds X rooms and each room holds 2 players. The server then iterates over the rooms and updates/handles messages/ect.So instead of rooms I'd just have 2 players in a single session. But a linux instance could handle multiple sessions.Does that sound right?CommentSharerePost-User-6200020answered 2 years ago0[quote="JamesM_Aws, post:4, topic:11089"]50 processes per instance is the limitation at the time of this writing. We have plans to increase this limitation in the future.[/quote]@REDACTEDUSERPlease also considering adding support to support multiple rooms/games per process. It's important for certain genres - It may be not possible to spawn N processes for a memory reason, but you can spawn fewer processes and multiple rooms per process (using async / thread internally).CommentSharerePost-User-6332773answered 2 years ago0Hi @REDACTEDUSERYup, your understanding is correct. You can concurrently run up to 50* game server processes on a single instance, and with each process handling 2 players, you could have up to 100 players playing on a single instance, provided that you have optimized you server processes well and/or selected a sufficiently powerful instance type to run 50 processes.Once you have that set up, all you need to do is to call gamelift::StartMatchmaking for each player, and gamelift will automatically place 2 players in an available process for you in any of your active fleets' instances, and will give you the ip/port for your 2 clients to connect to the server process.If you choose not to use gamelift's matchmaking service, you can group up players by yourself and call gamelift::StartGameSessionPlacement to place game session on the cheapest/least-latency instance in your fleets.* 50 processes per instance is the limitation at the time of this writing. We have plans to increase this limitation in the future.CommentSharerePost-User-7108006answered 2 years ago"


Question: How can I consume subscribed Lake Formation data assets using Glue or EMR?

"ProblemI'd like to process my data using Spark.How can I consume Lake Formation data assets that I'm subscribed to, using Glue ETL Jobs or EMR?ContextI created following domains: Marketing, Sales and Reporting.Project in the Reporting domain is suppose to consume a data from Marketing and from Sales domains in order to create a report. The report is going to be published as a data product in Reporting domain.In Reporting domain I created a "reporting project" with Data Lake Producer capability. Then I subscribed "reporting project" to Sales and Marketing assets (Lake Formation tables).As a project owner of "reporting project" I can see assets that I subscribed to in my SUB db. I can query those assets with a help of Athena run from DataZone portal in the context of my Project. I can also create reporting table in my PUB db that I can publish later in the Data Catalog.After successful subscription, the authorization to my Sales and Marketing Lake Formation tables is done via DataZone project roles (owner, contributor, viewer).FollowComment"

Answer: "0Grant permissions: Ensure that the IAM roles associated with your Glue or EMR services have the necessary Lake Formation permissions to access the data. You can do this through the Lake Formation console by granting the necessary permissions (SELECT, INSERT, DELETE, ALTER, DROP) to the IAM roles for the specific databases and tables.Use Glue Catalog: Your Glue or EMR jobs should be configured to use the AWS Glue Data Catalog as the metastore. This way, the jobs will be able to access the table metadata that's managed by Lake Formation.Use Lake Formation API: When reading data in your Spark jobs, you should use the Lake Formation API to get a temporary, pre-signed URL that your job can use to read the data. In Spark, you can use spark.read.format("parquet").load(preSignedUrl) to read the data.Access Control: It's worth noting that Lake Formation enforces fine-grained access control. This means that even though your Glue or EMR job might be able to see the table metadata, the job might not be able to access the underlying data if the IAM role associated with the job doesn't have the necessary permissions.import boto3# Initialize a session using AWS SDKsession = boto3.Session(region_name="us-west-2")  # specify the correct region# Initialize the Lake Formation clientlakeformation = session.client("lakeformation")# Get the pre-signed URL for a data locationresponse = lakeformation.get_data_lake_settings()# Check if the response is successfulif response['ResponseMetadata']['HTTPStatusCode'] == 200:    # Extract the pre-signed URL    pre_signed_url = response['DataLakeSettings']['PreSignedUrl']# Read data using Sparkdf = spark.read.format("parquet").load(pre_signed_url)CommentShareEXPERTsedat_salmananswered 13 days ago0Thanks for the answer.Sorry for the inaccurate description of the problem. The question was asked in the context of the new Amazon DataZone service and its data asset subscription model.I think we've already found a solution.DataZone for each project creates three roles (project owner, contributor, viewer).datazone-usr-o-proj-MyProjectIddatazone-usr-c-proj-MyProjectIddatazone-usr-u-proj-MyProjectIdThose roles obtain SELECT permission on Lake Formation tables that the project is subscribed to.In case of project with DataLake Producer capability, the roles get assigned policy that enables them to submit and run glue jobs => https://docs.aws.amazon.com/datazone/latest/userguide/Identitybasedroles.htmlThe policy is defined with the condition:            "Condition": {                "ForAnyValue:StringEquals": {                    "aws:ResourceTag/datazone:projectId": "proj-MyProjectId"                }            }So to be able to process data with glue we need to for glue job:select IAM role create by data zone project, e.g. datazone-usr-c-proj-MyProjectIdput our jobs script in the bucket created by data zone project, e.g.: s3://datazone-proj-MyProjectId-...add resource tag to the job with our project id: "datazone:projectId"= "proj-MyProjectId"Looking at the project roles policies, as of now, it's possible to process data with Glue but unfortunately not with EMR.What would say if we extend policies to include EMR related permissions?CommentShareirmeanswered 12 days ago"


Question: CognitoUserSession is not valid because idToken is null

I have been following the guidance from this link https://docs.amplify.aws/lib/auth/emailpassword/q/platform/js/ and when I get to Amplify.Auth.signIn(); it continues to throw the error CognitoUserSession is not valid because idToken is null. Is cognito supposed to pass an IdToken back or am I supposed to request one before executing Ampligy.Auth.signin()?FollowComment

Answer: "0Assign openid scope to your user pool to get ID Token.The Amazon Cognito authorization server redirects back to your app with access token. Because openid scope was not requested, Amazon Cognito doesn't return an ID token.https://docs.aws.amazon.com/cognito/latest/developerguide/authorization-endpoint.htmlCommentSharePravoanswered 9 months ago"


Question: Health Check Fail - App Runner .NET 6

"Hello, I'm facing a problem when trying to upload a .NET 6 API to the App Runner service, I do all the settings and create the service.Through the Logs, they follow a little of its progress, it starts successfully, builds an application successfully and at the verification stepthe Health Check, it stays +- 20 min until it fails.I tried changing the port to 5000, 5001 and 8080 but none were successful. Please could someone help me?FollowComment"


Question: Messages passed visibility timeout still "in flight"

"I am occasionally seeing a problem that I cannot explain with Fifo queues.In the console and on the aws-cli I see messages in-flight that are way passed their visibility timeouts.They stay there for hours.It seems to be the same issue this person posted:https://forums.aws.amazon.com/thread.jspa?messageID=520092Looking at my process output, all messages seem to have been processed without issue so I don't know what the remaining in flight means.I am using boto3 with python2 to interact with the queues.Thank youFollowComment"

Answer: "0Accepted AnswerHi there!What you're describing sounds more like there are a few messages that are repeatedly failing to process for some reason. If you don't have a redrive policy (https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html) defined for a FIFO queue, any messages that can't be processed will cycle back and forth between visible and invisible indefinitely.If you're looking at metrics like ApproximateNumberOfMessagesNotVisible, it could look like it's staying elevated consistently, giving the impression that messages are staying invisible for a long time, when in truth there are several poison pills fluctuating back and forth.What does ApproximateAgeOfOldestMessage look like for your queue?RobinCommentShareAWS-User-3060309answered 4 years ago"


Question: Serverless file processing

"Hi, I am building an architecture following the serverless pattern, scenario file comes to the S3 folder, S3 event triggers Lambda to validate & process the file...  do we need a staging folder in S3 to perform our business operations? i am thinking to read the file from S3 landing folder where files arrives and perform the validation & business operation to move the files to duplicate or complete folder... in this journey does it makes sense to have a staging folder other than landing folder?Which one will be suitable to trigger lambda from S3?S3 events --> LambdaS3 events --> SQS --> Lambdathank youFollowComment"

Answer: "0It depends on your use case.If this is more of a pre-processing than it can be beneficial to move all the valid files to the final destination and all the invalid files in another folder.Moving the files gives you the visibility which files are processed and which not.CommentShareMarcoanswered a year ago0I would definitely have one folder/prefix where files are delivered to; then processed files are put somewhere else. If you put the processed files back in the original folder then the Lambda will (maybe, but likely) be triggered again. To avoid this you'll have to have some logic in the Lambda function that detects when it is processing a new file; or you'll have to rely on file extensions. Much easier to avoid this situation (and potentially having a circular loop which invokes many Lambda functions) but using a different folder.CommentShareEXPERTBrettski-AWSanswered a year ago0I worked on a similar architecture in the past. We had coupe of issues I would like to highlight with the S3 >> Lambda.For lambda updates/deployment, we had to wait for off hours. If any files were delivered that time, we had to copy again those to initiate the pipeline.Same for Lambda failure, we had to find the file/object and copy again to initiate the pipeline.With S3 >> SQS >> Lambda,For Continuous Delivery, during deployment the unprocessed messages (files) will be available at SQS for the set retention time. No, manual effort needed.SQS activate retries and dequeue, again failures available at given time.If at all you want to decouple the file upload and start or a delayed start. You can have source team/system delivering the file and start the process only after the message to SQS. Modern data pipeline platform offers the ability to notify after successful event.Back to the first part of the question, yes having different folders would help. We had Staging, Processed and Failure.CommentSharesrcsakthivelanswered a year ago"


Question: ECS fails to remove a task from the load balancer target group?

"We recently migrated our service to ECS, and we’ve seen a pattern of errors like this a few times:Our CPU usage is low, so as part of normal autoscaling, ECS starts to reduce the number of tasks by 1ECS claims to have deregistered 1 target and be draining connections90 seconds later (after the deregistration delay) ECS stops the taskImmediately after the task is stopped, a flood of load balancer 502s happens, all directed at just one IP. We suspect that this is the IP of the task that was removed and stopped, but somehow not removed from the ELB target groupWe don’t have any long-lived connections, so the 90 second deregistration delay should be long enough for the task to finish processing its requests before it’s stopped.It seems that the task selected for removal isn’t actually removed from the ALB target group, even though the ECS logs include messages indicating that it is. The logs include* service \[our-service\] deregistered 1 targets in target-group \[our-target-group\]followed by* service \[our-service\] has begun draining connections on 1 tasks.Events like this happen rarely (definitely not every time we scale down) but frequently enough to notice. Does anybody have ideas about why these errors might be happening, or how to get more information about what is going on? Thanks in advance.FollowComment"

Answer: "0It's due to the load balancer stickiness , it should be set up by adding the flag for it.CommentShareshivam_awsanswered 3 months ago"


Question: AWS Glue Mongodb Atlas Connection Works For Crawler But Not For ETL Job?

"Hi community,I am trying to perform an ETL job using AWS Glue.Our data is stored in MongoDB Atlas, inside a VPC.Our AWS is connected to our MongoDB Atlas using VPC peering.To perform the ETL job in AWS Glue I have first created a connection using the VPC details and the mongoDB Atlas URI along with the password and username. The connection is used by the AWS Glue crawlers to extract the schema to AWS Data Catalog Tables.This connection works!However, I am then attempting to perform the actual ETL job using the following pySpark code:#My Temp Variablessource_database="d*********a"source_table_name="main_businessessource_mongodb_db_name = "main"source_mongodb_collection = "businesses"glueContext.create_dynamic_frame.from_catalog(database=source_database,table_name=source_table_name,additional_options = {"database": source_mongodb_db_name,"collection":source_mongodb_collection})However the connection times out and for some reason mongodb atlas is blocking the connection from the ETL job.It's as if the ETL Job is using the connection differently than the crawler does. Maybe the ETL Job is not able to run the job inside our AWS VPC that is connected to the MongoDB Atlas VPC (VPC Peering is not possible?).Does anyone have any idea what might be going on or how I can fix this?Thank you!FollowCommentsausborn 2 months agoHave you checked for differences in the IAM policies attached to ETL job role and the crawler role?SharerePost-User-0675162 2 months agoHi Sausborn,I have given the crawler and the glue job the same role and so the same IAM policies...Yet the crawler is able to connect to the database with the given roles but the glue job fails.Share"

Answer: "0Accepted AnswerAre you adding the connection to the Glue job?It's not enough to reference the table since the Glue job needs to know the connection VPC details before startingCommentShareGonzalo Herrerosanswered 2 months agorePost-User-0675162 2 months agoHi Gonzalo,I have reference the VPC details when I created the connection. (VPC id, subnet and security group)When clicking on the connection, I can see that both the crawler and the glue jobs reference this connection. The crawler is successfully connecting but the glue job is not.Which other way is there to reference the connection VPC details from within the glue job ?ShareGonzalo Herreros 2 months agoMake sure the connection is added in the job connection listSharerePost-User-0675162 2 months agoThank you :D That helped.Share"


Question: Wordpress Website is not working

"Hello everyone, I have a website on WordPress that uses lighsail as an instance. However recently we have been getting an error that the website can't be reached and I can't access the admin dashboard even. The website is called wonderedhub.com. And for the record, we have not been using the website for a while and we redirect from it to a new domain. So we don't what might be the problem as no one edited anything in it.Any help is really appreciated!FollowCommentyensteveAWS 2 months agoHi there! If you haven't taken a look into these yet, here are some troubleshooting links that may help with this issue.https://wordpress.org/documentation/article/faq-troubleshooting/https://repost.aws/knowledge-center/lightsail-instance-failed-status-checkShare"

Answer: "1It looks like your domain is good, DNS is hosted in Cloudflare and is responding.  I would check in the AWS/Lightsail console that this public IP address for your website is still accurate.  It could possible be that you need to update this A record if there was an IP change on your Lightsail instance.CommentShareAWS-User-0357409answered 2 months agoFatima-Hk 2 months agoThank you for answering! My lightsail instance indeed has this IP as public, so I don't think there is a need to change this record.Share0have  your problem solved? if yes please help me. I am facing same kind of issusethankuCommentShareJunaidanswered 3 days ago"


Question: Restoring the editor layout still an issue on multi-monitor setup with disparate resolutions

"RE: Lumberyard 1.18 Release Notes / Fixes / MiscellaneousRestoring the editor layout is no longer an issue on multi-monitor setups with disparate resolutions and sizes.https://docs.aws.amazon.com/lumberyard/latest/releasenotes/lumberyard-v1.18-fixes.htmlThis is still an issue for me, as my editor layout will not restore correctly. The sizing of docked windows is not saved when I close and reopen the editor and I have to resize my docked windows with every launch. Please confirm this as a known issue or let me know if you need more information. Thanks!Follow"

Answer: "0Accepted AnswerHi! We have recorded an issue for this in our backlog. Thanks for the feedback!SharerePost-User-5018585answered 4 years ago0@REDACTEDUSERSharerePost-User-4357073answered 4 years ago0Sorry for the delays @REDACTEDUSERSharerePost-User-9965251answered 4 years ago0@REDACTEDUSERAny update?SharerePost-User-4357073answered 4 years ago0@REDACTEDUSERAny update?SharerePost-User-4357073answered 4 years ago0@REDACTEDUSERThank you. If you need any additional information, screenshots, etc. just let me know.SharerePost-User-4357073answered 4 years ago"


Question: Getting error in adding role policy to service linked role

"resource "aws_iam_service_linked_role" "AWSServiceRoleForLexV2" {aws_service_name = "lexv2.amazonaws.com"tags = local.common_tags}resource "aws_iam_role_policy" "lex2_policy" {name = "backend_bot_lex2_policy"role = aws_iam_service_linked_role.AWSServiceRoleForLexV2.idpolicy = jsonencode({"Version" : "2012-10-17","Statement" : [{"Effect" : "Allow","Action" : "logs:CreateLogGroup","Resource" : "arn:aws:logs:eu-west-2:${var.aws_accountId}:"},{"Effect" : "Allow","Action" : ["logs:CreateLogStream","logs:PutLogEvents"],"Resource" : ["arn:aws:logs:eu-west-2:${var.aws_accountId}:log-group:"]}]})}ErrorError putting IAM role policy backend_bot_lex2_policy: ValidationError: The specified value for roleName is invalid. It must contain only alphanumeric characters and/or the following: +=,.@_-│ status code: 400,FollowComment"

Answer: "0In resource "aws_iam_role_policy", you should place .name instead of .id for the role: aws_iam_service_linked_role.AWSServiceRoleForLexV2**.name**.However, note that you can't modify an Amazon Lex V2 service-linked role (only the description can be updated). Check this document for more info: https://docs.aws.amazon.com/lexv2/latest/dg/using-service-linked-roles.htmlCommentSharejoahnaanswered a year ago"


Question: Amazon SageMaker Built-in algorithms and Spot checkpointing

"Which Amazon SageMaker built-in algorithms support checkpointing? In the documentation it says that:SageMaker built-in algorithms and marketplace algorithms that do not checkpoint are currently limited to a MaxWaitTimeInSeconds of 3600 seconds (60 minutes).However, in the algorithms I don't find any pointer to "checkpoint" or "spot". Can you help me out?FollowComment"

Answer: "0Accepted AnswerThis is the best resource that I've found to clarify this:https://aws.amazon.com/blogs/aws/managed-spot-training-save-up-to-90-on-your-amazon-sagemaker-training-jobs/Built-in algorithms: computer vision algorithms support checkpointing (Object Detection, Semantic Segmentation, and very soon Image Classification). As they tend to train on large data sets and run for longer than other algorithms, they have a higher likelihood of being interrupted. Other built-in algorithms do not support checkpointing for now.Also:Please note that TensorFlow uses checkpoints by default. For other frameworks, you’ll find examples in our sample notebooks and in the documentation.CommentShareAWS-User-3911429answered 3 years ago"


Question: Amazon CloudFront Communication wordpress how ?

"Have question about CDN Amazon CloudFront. Example im have installed wordpress on my hosting server and on my hosting server now is style.css file. When im use CDN Cloudfront this style.css will load from amazon not from my hosting server and if im wanna to make change on style.css if im make this change on hosting server, will change style.css file on amazon server ? and how amazon will know im change .css file ?FollowComment"

Answer: "2Accepted AnswerThis will depend on the cache (expiration) settings. From the Amazon CloudFront user guide:Typically, CloudFront serves a file from an edge location until the cache duration that you specified passes—that is, until the file expires. After it expires, the next time the edge location gets a request for the file, CloudFront forwards the request to the origin to verify that the cache contains the latest version of the file.By default, each file automatically expires after 24 hours, but you can change the default behavior in two ways: (a) CloudFront TTL settings or (b) Cache-Control headerIn case you need to remove a file manually from CloudFront edge caches before it expires, you can invalidate the file (e.g. CSS style sheet file). More information about Invalidation is available here: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.htmlCommentSharejoahnaanswered 8 months ago"


Question: S3 integration Oracle RDS adding option to existing option group

"Hi Gurus,I currently have an Oracle RDS instance running wih 12.1.0.2.v8 DB engine version running in North Virginia with a custom option group,I read that we can integrate s3 with Oracle RDS by adding option in an option group.Can i add the option to my existing custom option group or do i need to upgrade the instance to the latest db engine version to use it?ThanksFollowComment"

Answer: "1Accepted AnswerThere is no version requirement for the new S3_INTEGRATION option. You can simply add this option to your existing group. Additional information is in the documentation at https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/oracle-s3-integration.htmlCommentShareAWS-User-9938760answered 4 years ago1Hi Nagarajan,I did this activity just today and I could set this option.Here is what I did:Go to RDS service consoleSelect the DB instance for which I want this feature addedIn the "Connectivity & security" tab, scrolled down to the bottom. I could see "Manage IAM Roles" section in there.selected "custom role" rds-s3-integration-role (I created this) and attached.Hope this helps!CommentShareRakeshDevanswered 4 years ago0Hi Team,Based on the documentation at https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/oracle-s3-integration.html, i have started working on the Oracle RDS to S3 integration.But looks this given option (Connectivity & security tab, in the Manage IAM roles section) is not available in AWS Console. I don't find option to add the role to the Oracle DB instance.Even i have tried to do this via AWS CLI and got the error likeInvalid choice: 'add-role-to-db-instance', maybe you meant:** add-role-to-db-cluster*AWS Cli command used:aws rds add-role-to-db-instance ^--db-instance-identifier new-scripts-qa-db ^--feature-name S3_INTEGRATION ^--role-arn arn:aws:iam::<acct #>:role/rds-s3-roleKindly help me to resolve this issue.CommentShareRNagarajan4answered 4 years ago"


Question: Using human loop configuration of Amazon Augmented AI with Amazon Textract's Expense Analysis API

"I am using Textract's StartExpenseAnalysis and GetExpenseAnalysis API to analyze multipage documents.These are the documents that I had followed for reference -https://docs.aws.amazon.com/textract/latest/dg/api-async-roles.htmlhttps://docs.aws.amazon.com/textract/latest/dg/async-analyzing-with-sqs.htmlWe want to integrate Amazon Augmented AI's human review process in our flow. I have gone through the following document to understand the human review process using the analyze document API -https://aws.amazon.com/blogs/machine-learning/using-amazon-textract-with-amazon-augmented-ai-for-processing-critical-documents/However, when I tried to use the same "HumanLoopConfig" with the Expense Analysis API, I got this error  -Parameter validation failed: Unknown parameter in input: "HumanLoopConfig"I am attaching the screenshots of the error for referenceIs the human review process supported for expense analysis API ?Or is there some issue with the way I am using the "HumanLoopConfig" in the API call ?If the process is supported, is there any documentation that I can review ?FollowComment"

Answer: "0Hi,Thanks for using Textract service! We understand your need to integrate with A2I but unfortunately Expense analysis APIs currently not support A2I yet. Please consider using AnalyzeDocument API to integrate with A2I.You can find the APIs here and supported parameters. If HumanLoopConfig is not in the request parameter list, it means not supported yet.Thank you!CommentShareAWS-Shayneanswered 6 months ago"


Question: EMR workspace cluster change

"Hello,From the new interface how do we attach a cluster to an existing Workspace? In the old UI we could do this by clicking on the workspace but now clicking on it tries to start it. Thanks in advanceFollowComment"


Question: What is the total number of EFS volumes that can be created in a VPC?

"I need to figure out how many EFS volumes can actually be created within a VPC. I'm aware that 1000 volumes can be created within an account, and this quota is adjustable. However, based on EFS mount target quotas it sounds like only 400 mount targets can be created within a VPC.I'm currently using 3 availability zones per EFS  (3 mount targets per filesystem). Does this quota mean that I will only be able to launch 133 EFS volumes within a VPC?Looking for a solution that allows us to scale to as many EFS volumes as possible within a VPC.FollowComment"

Answer: "2I'm assuming the reason for so many file systems is for isolation of files. Have you taken a look at Access Points for EFS? This might help you to leverage fewer file systems and still be able to isolate use cases: https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.htmlCommentShareEXPERTRob_Hanswered a year agoAWS-User-3835952 a year agoYes, you're correct. My plan is to isolate compute and storage for each customer via Lambda and EFS.Using Access Points does seem like a possible solution. I was using an IAM policy to restrict the access to the filesystem, and it looks like I can do something similar with access points. This does have a few drawbacks though:Data isn't truly isolated, is enforcing a root directory enough?Would lose the ability to encrypt the data with distinct keys per customerCustomers would share IOPS which would force me to use Max/IO instead of General Purpose performance which is slowerSharerePost-User-8743013 6 months agoAccess points are limited to 120 per file system. So this would only help if you add complex logic to organize which access point of which EFS volume should be used for each case.The alternative way to scale would be to create more VPCs so that you can have more EFS volumes with mount points, but then to connect them you'd need some complex VPC peering set up.Share0To my understanding you are correct - if you are using 3 availability zones for each filesystem you'll be limited to 133 filesystems before hitting the mount quota limit (400) which cannot be increased.You might want to consider if Amazon FSx for Ontap better meets your requirementsCommentShareEXPERTPeter_Ganswered a year agoAWS-User-3835952 a year agoThanks for the suggestion but Amazon FSx for Ontap doesn't seem to fit the use case based on price.SharerePost-User-8743013 6 months agoNoting: FSx seems to have a default quota of 100 Filesystems per AWS account. You can request quota increases, though it is not clear to what limit. So this may be more restrictive than the 133 EFS volumes per VPC.Share"


Question: cdk deployment error: RuntimeError: Cannot read properties of undefined (reading 'bindToGraph')

"I am trying to do a cdk deployment, where I am just copying the example from the documentation. See example below (https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_stepfunctions/StateMachineFragment.html#aws_cdk.aws_stepfunctions.StateMachineFragment) from aws_cdk import (    # Duration,    Stack,    aws_stepfunctions as sfn,    aws_stepfunctions_tasks as tasks, ) from constructs import Construct class MyJob(sfn.StateMachineFragment):    def __init__(self, parent, id, *, job_flavor):        super().__init__(parent, id)        choice = sfn.Choice(self, "Choice").when(sfn.Condition.string_equals("$.branch", "left"), sfn.Pass(self, "Left Branch")).when(sfn.Condition.string_equals("$.branch", "right"), sfn.Pass(self, "Right Branch"))        # ...        self.start_state = choice        self.end_states = choice.afterwards().end_states          def start_state(self):        return self._start_state    def end_states(self):        return self._end_statesclass CdkStateMachineFragmentStack(Stack):    def __init__(self, scope, id):        super().__init__(scope, id)        # Do 3 different variants of MyJob in parallel        parallel = sfn.Parallel(self, "All jobs").branch(MyJob(self, "Quick", job_flavor="quick").prefix_states()).branch(MyJob(self, "Medium", job_flavor="medium").prefix_states()).branch(MyJob(self, "Slow", job_flavor="slow").prefix_states())        sfn.StateMachine(self, "MyStateMachineFragmentTest",            definition=parallel        )When deploying, I get the following error:Traceback (most recent call last):File "app.py", line 10, in <module>CdkStateMachineFragmentStack(app, "CdkStateMachineFragmentStack",File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/jsii/_runtime.py", line 111, in callinst = super().call(*args, **kwargs)File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/cdk_state_machine_fragment/cdk_state_machine_fragment_stack.py", line 34, in initsfn.StateMachine(self, "MyStateMachineFragmentTest",File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/jsii/_runtime.py", line 111, in callinst = super().call(*args, **kwargs)File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/aws_cdk/aws_stepfunctions/init.py", line 6871, in initjsii.create(self.class, self, [scope, id, props])File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/jsii/_kernel/init.py", line 336, in createresponse = self.provider.create(File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/jsii/_kernel/providers/process.py", line 363, in createreturn self._process.send(request, CreateResponse)File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/jsii/_kernel/providers/process.py", line 340, in sendraise RuntimeError(resp.error) from JavaScriptError(resp.stack)RuntimeError: Cannot read properties of undefined (reading 'bindToGraph')I tried already what is suggested in the following stack overflow post [https://stackoverflow.com/questions/70553737/cannot-read-properties-of-undefined-reading-bindtograph] getting a different error:Traceback (most recent call last):File "app.py", line 10, in <module>CdkStateMachineFragmentStack(app, "CdkStateMachineFragmentStack",File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/jsii/_runtime.py", line 111, in callinst = super().call(*args, **kwargs)File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/cdk_state_machine_fragment/cdk_state_machine_fragment_stack.py", line 34, in initparallel = sfn.Parallel(self, "All jobs").branch(MyJob(self, "Quick", job_flavor="quick").prefix_states()).branch(MyJob(self, "Medium", job_flavor="medium").prefix_states()).branch(MyJob(self, "Slow", job_flavor="slow").prefix_states())File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/.venv/lib/python3.8/site-packages/jsii/_runtime.py", line 111, in callinst = super().call(*args, **kwargs)File "/home/workspace/development/cdk/cdk_test/cdk_state_machine_fragment/cdk_state_machine_fragment/cdk_state_machine_fragment_stack.py", line 18, in initself.start_state = choiceAttributeError: can't set attributeCould you support me?Thanks,PabloFollowComment"

Answer: "0The error message "RuntimeError: Cannot read properties of undefined" is typically encountered when trying to access a property or method of a variable that has not been defined. In order to resolve this issue, you can add the @property decorator above the start_state and end_states methods, this will ensure that the methods are treated as properties and can be accessed without encountering the undefined error.The second error message "AttributeError: can't set attribute" occurred because you had defined methods and attributes with the same name, namely "start_state" and "end_states". In Python, methods are attributes too, and having an attribute and method with the same name can cause naming conflicts. To fix this issue, you can simply rename the attributes to something else, you could use "_start_state" and "_end_states".You can find both fixes below, these changes will ensure that your code runs smoothly:from aws_cdk import (    # Duration,    Stack,    aws_stepfunctions as sfn,    aws_stepfunctions_tasks as tasks, )from constructs import Constructclass MyJob(sfn.StateMachineFragment):    def __init__(self, parent, id, *, job_flavor):        super().__init__(parent, id)        choice = sfn.Choice(self, "Choice").when(sfn.Condition.string_equals("$.branch", "left"), sfn.Pass(self, "Left Branch")).when(sfn.Condition.string_equals("$.branch", "right"), sfn.Pass(self, "Right Branch"))        # ...        self._start_state = choice        self._end_states = choice.afterwards().end_states    @property    def start_state(self):        return self._start_state    @property    def end_states(self):        return self._end_statesclass CdkStateMachineFragmentStack(Stack):    def __init__(self, scope, id):        super().__init__(scope, id)        # Do 3 different variants of MyJob in parallel        parallel = sfn.Parallel(self, "All jobs").branch(MyJob(self, "Quick", job_flavor="quick").prefix_states()).branch(MyJob(self, "Medium", job_flavor="medium").prefix_states()).branch(MyJob(self, "Slow", job_flavor="slow").prefix_states())        sfn.StateMachine(self, "MyStateMachineFragmentTest",            definition=parallel        )CommentShareMaha_Eanswered 3 months ago"


Question: how to enable route propagation in the routing tables

"I hope all is well. I am working on a project to move our infrastructure to terraform and have the Infrastructure as Code. Route propagation is not working in our test account however I copied everything that we have in our main account.I know there is no limitation for routings and it should be a technical config issue.3 VPCs (Prod, Dev, Stage) are going to route traffic to the internet through 1 VPC (DMZ) and 1 IGW in the DMZ. In the main account the routing is working fine but in the test account the propagation does not change to enabled ("yes").FollowComment"

Answer: "0Thank you for reaching out to AWS,I understand you would like to enable route propagation to automatically propagate routes to the route tables within your VPC. This would remove the need to manually enter VPN routes to your route tables. You may enable Route Propagation via the Amazon VPC Console or through AWS CLI.In the Amazon VPC Console:Navigate to Route TablesSelect the Route Table, Choose Actions, and Edit Route PropagationEnable the check Box for the Virtual Private Gateway and then SaveIn AWS CLI:Using this command will enable route propagationaws ec2 enable-vgw-route-propagation —route-table-id ROUTE-TABLE-ID —gateway-id GATEWAY-IDThe provided link demonstrates these steps for each on how to enable Route Propagation.https://docs.aws.amazon.com/vpc/latest/userguide/WorkWithRouteTables.html#EnableDisableRoutePropI hope this helps! Please let me know if I can be of further assistance.CommentShareJulio_Aanswered 10 months ago"