Question: Redshift ML / SageMaker - Deploy an existing model artifact to a Redshift Cluster

"Is it possible to deploy an existing model artifact from SageMaker to Redshift ML?For example, with an Aurora ML you can reference a SageMaker endpoint and then use it as a UDF in a SELECT statement.Redshift ML works a bit differently - when you call CREATE MODEL - the model is trained with SageMaker Autopilot and then deployed to the Redshift Cluster.What if I already have a trained model, can i deploy it to a Redshift Cluster and then use a UDF for Inference?FollowComment"

Answer: "0Accepted AnswerAs of January 30 2021, you can't deploy an existing model artifact from SageMaker to Redshift ML directly with currently announced Redshift ML preview features. But you can reference  sagemaker endpoint through a lambda function and use that lambda function as an user defined function in Redshift.Below would be the steps:Train and deploy your SageMaker model in a SageMaker Endpoint.Use Lambda function to reference sagemaker endpoint.Create a Redshift Lambda UDF referring above lambda function to run predictions.CommentShareManash Debanswered 2 years ago"


Question: Amazon s3  SDK issue: XML document structures must start and end within the same entity (s3 client sdk)

"Running into an issue on presto with the underlying s3 client (HUDI table in glue)If you see the stacktrace the issue is being caused by the underlying s3 client barfing on unable to parse XML.I have seen various flavors of this issue with regards to the s3 file listing where folks claim it was due to incorrect permissions, URL encoding, or some other issue -  I have verified that permissions are correct and it's not a URL encoding issue.Here is why:The same presto query works fine when I run between 1 - 9 concurrent threads for the same queryThe moment I go to 10 concurrent queries - exact same query, exact same data, 7+ of the 10 queries fail with the exceptionAny subsequent run for the next few minutes with even 1 thread fails with the same exceptionIt appears to be a s3 client connection management issue /sdk issue.  I increased the # of s3 connections available to the SDK but that didn't help. Any guidance on addressing this would be appreciated.com.facebook.presto.spi.PrestoException: Error checking path :s3://[REDACTED PATH]/.hoodie_partition_metadata, under folder: s3://[REDACTED PATH]at com.facebook.presto.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:128)at com.facebook.presto.hive.util.ResumableTasks.safeProcessTask(ResumableTasks.java:47)at com.facebook.presto.hive.util.ResumableTasks.access$000(ResumableTasks.java:20)at com.facebook.presto.hive.util.ResumableTasks$1.run(ResumableTasks.java:35)at com.facebook.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)at java.base/java.lang.Thread.run(Thread.java:829)Caused by: org.apache.hudi.exception.HoodieException: Error checking path :s3://[REDACTED PATH]/.hoodie_partition_metadata, under folder: s3://[REDACTED PATH]at org.apache.hudi.hadoop.HoodieROTablePathFilter.accept(HoodieROTablePathFilter.java:188)at com.facebook.presto.hive.util.HiveFileIterator.lambda$getLocatedFileStatusRemoteIterator$0(HiveFileIterator.java:103)at com.google.common.collect.Iterators$5.computeNext(Iterators.java:639)at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)<Snipped>at com.facebook.presto.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:40)at com.facebook.presto.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:121)... 7 moreCaused by: com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandlerat com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:178)at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:335)at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:118)at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:107)at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31)at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)<Snipped>at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5445)at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5392)at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5386)at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:928)at com.amazonaws.services.s3.AmazonS3Client.listNextBatchOfObjects(AmazonS3Client.java:1000)at com.amazonaws.services.s3.AmazonS3Client.listNextBatchOfObjects(AmazonS3Client.java:977)at com.facebook.presto.hive.s3.PrestoS3FileSystem$1.computeNext(PrestoS3FileSystem.java:540)at com.facebook.presto.hive.s3.PrestoS3FileSystem$1.computeNext(PrestoS3FileSystem.java:530)at com.google.common.collect.AbstractSequentialIterator.next(AbstractSequentialIterator.java:75)at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)at com.google.common.collect.Iterators$ConcatenatedIterator.hasNext(Iterators.java:1332)at com.facebook.presto.hive.s3.PrestoS3FileSystem.getFileStatus(PrestoS3FileSystem.java:350)at org.apache.hadoop.fs.HadoopExtendedFileSystem.getFileStatus(HadoopExtendedFileSystem.java:384)at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)at org.apache.hudi.common.io.storage.HoodieWrapperFileSystem.exists(HoodieWrapperFileSystem.java:461)at org.apache.hudi.exception.TableNotFoundException.checkTableValidity(TableNotFoundException.java:44)at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:116)at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:103)at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:89)at org.apache.hudi.hadoop.HoodieROTablePathFilter.accept(HoodieROTablePathFilter.java:149)... 24 moreCaused by: org.xml.sax.SAXParseException: XML document structures must start and end within the same entity.at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204)at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:178)at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)at java.xml/com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1471)<Snipped>at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1216)at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:164)... 61 moreFollowComment"


Question: Which service should I use to authorize client applications running on desktops?

"Hello, I have a client application written in Java that is running as a desktop app. I want it to communicate with my API. The problem is I want to prevent unauthorized clients from to connecting the API. At first, I was considering using some kind of API keys, but then I thought that maybe IoT core will be a better option. I could authorize clients with certificates.What do you think, would that be the best choice?FollowComment"

Answer: "1Accepted AnswerUsing AWS IoT Core to authenticate desktop apps is definitively an option. You can use X509 certificates which can be stored in the OS keychain, and you can also use the certificate to obtain temporary STS credentials in order to directly access AWS services in case MQTT protocol is not sufficient. With AWS IoT Core you can also leverage additional services like Device Defender, Registry, Fleet Indexing, which are not available when using API Gateway. Another main difference is that with AWS IoT Core you do not need to maintain a PKI infrastructure to manage the certificates.You would still need to solve how the app obtains its unique certificate, one option being using Fleet provisioning.CommentShareEXPERTMassimilianoAWSanswered a month agoKrzysztof A a month agoThanks, I think this is exactly what I'm looking for.Share1What about using Cognito for user authentication management?Simple authentication processes can be created with Cognito.https://aws.amazon.com/jp/cognito/CommentShareEXPERTRiku_Kobayashianswered a month agoKrzysztof A a month agoWouldn't that mean, that I have to implement login in my client application? I want to use it similarly to an IoT device, so I know it's a certain device, but I don't want to require user to log into it.ShareRiku_Kobayashi EXPERTa month agoIndeed, with Cognito, you need to implement a login function.API Gateway can also be used for certificate authentication, so please review the following document to determine if it is appropriate for your use case.https://aws.amazon.com/jp/blogs/compute/introducing-mutual-tls-authentication-for-amazon-api-gateway/Share"


Question: AWS S3 upload using python

"I'm trying to learn how to upload data to AWS S3 using python.  I've partially coded a script that does upload files and directories, but I was wondering if there was anyway I could have someone check it out and tell me what was good or bad about the script.  It works, but I want to do more with it.  Is that possible in this forum?  I'm not sure I want to post the script here.  Maybe I could email it or send it in a message to some experienced AWS python developers?FollowCommentjschwar313 a year agoMaybe I should just ask my questions.  For example, I see that an option for the TransferConfig has max_concurrency as an option, which would take a number for the number of threads to use.  Does this mean we don't need to create a pool for the threads?  Does AWS do that?Sharejschwar313 a year agoHow do we put the logs from the various threads to CloudWatch or save them if it's true we don't have to create a pool?Sharejschwar313 a year agoI'd love to be able to get a message to AWS about their documentation.  It's pretty bad.  I don't expect examples matching what I want to do, but I have decades of experience in unix, mainframe and windows development and I've worked in over 20 computer languages, but this documentation is pretty poor.  The more I read, the more questions I have and I have no way of getting them answered except here.  There is no training that will answer them, either, that I know of.  It's like AWS is saying that they shouldn't be the source of documentation, but the internet should be the source.Sharejschwar313 a year agoIf AWS is listening to this, I'll give an example of their poor documentation.  Here's a link:  https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/boto3.html  On this page, it describes boto3.set_stream_logger which has a parameter of level.  Nowhere does it say what the options are for that parameter and this is done throughout their documentation.  Like I said, I have a lot of experience and I have checked out documentation from many different companies.  This is pretty bad documentation.Share"

Answer: 0Accepted AnswerMy understanding is that the treading means that you can upload multiple files at the same time to improve efficiency if the source system is capable of running in that manner.For CloudWatch there would will only be entries if you have enable CloudTrail to log to CloudWatch.This would monitor management and data level actions and wouldn't matter if multi-tread or multi-part from what i've seen.CommentShareRobin Fordanswered a year ago0https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-examples.html should help youCommentShareDidier_AWSanswered a year ago0I work with PHP more than Python but AWS provide very useful SDKs for the majority of their services: https://aws.amazon.com/developer/language/python/CommentShareDaniel Craigieanswered a year ago0Here is an s3 upload example from the doc. Here is a working example from one of the POCs. You can check these to know how it's done against what you have. Hope this helps!CommentShareKunal_Ganswered a year ago


Question: Amazon Connect outbound Caller ID set to private

Have not seen anything related to creating a private outbound caller ID and having the number masked.Is a work around available for this to mask the number as Private/Anonymous?FollowComment

Answer: "1We are needing this functionality as well, for a good reason too, to protect the users of the service. There are some legitimate use cases for this across the health industry, emergency services, debt collection agencies etc.One suggestion that was provided, which may be ok for some, is grabbing a pool of DDIs, and randomising which one is used as the outbound CLID, then change them out every few months.Unfortunately, in our case, this is not going to work, so we are a bit stuck for now.CommentShareNixNZConnectFanaticanswered 10 months ago0Accepted AnswerThis isn't possible and from what I have seen AWS are unlikely to change this, as they seem to be big on avoiding spam, either emails, sms or calls, coming from within AWS.Is there a good reason to have the number set to private? Can you not just claim an additional number and then set a contact flow that says that the number is not active, or some other message?CommentShareledgeanswered a year ago0I would reach out to support for this request as that seems like a perfectly reasonable request.davidCommentSharedmaciasanswered a year ago"


Question: CloudFormation stack template failing to receive 1 resource signal during creation of EC2 instance

"Hello everyone,I am trying to deploy a MATLAB VM using the official stack deployment method on the MATLAB Help Centre (through the provided GitHub repository). However, when I try to do this, the EC2 instance creation always fails with "Failed to receive 1 resource signal(s) within the specified duration". This whole process apparently takes only 10-20 minutes and I have tried waiting for 3 hours by increasing the timeout period but I still get CREATE_FAILED followed by rollback. What could possibly be happening here and how could I go about resolving it?FollowComment"

Answer: "0The first place I would look is the network configuration of the subnet. You will need to provide a way for the cfn-signal to communicate to the CloudFormation endpoints. Here is an article on how to implement a VPC endpoint, Signaling AWS CloudFormation WaitConditions using AWS PrivateLink.CommentShareEXPERTkentradanswered a year ago"


Question: ECS task role chaining limitation to 1 hour

A customer is running into an error when trying to take the current task role credentials to assume a different role beyond 1 hour. They run into this error message:"ClientError: An error occurred (ValidationError) when calling the AssumeRole operation: The requested DurationSeconds exceeds the 1 hour session limit for roles assumed by role chaining."I can't seem to find guidance on how a customer should configure the task role credentials duration beyond 1 hour so that assume role chaining can also extend beyond 1 hour. Is that possible?FollowComment

Answer: "0Accepted AnswerI am just working on a container that ran into exactly the same issue. I made an assume role and added the credentials directly to the invocation of a third party lib. In the moment where the credentials timed out, all calls failed.I found this merge request https://github.com/boto/botocore/pull/1313!Now, I created a profile with the EcsContainer source:[profile crossaccount]role_arn="${CROSS_ACCOUNT_ROLE_ARN}"credential_source=EcsContainerI pass this profile to the 3rd party lib instead. Under the hood, this profile is registered in the credentials provider chain which solves the issue.Best regardsCommentSharechristiananswered 3 years ago"


Question: Guardrail: Deny access to AWS based on the requested AWS RegionInfo - how to customize the Guardrail SCP??

"Hello if you use the Region deny option in AWS Control Tower ist set the Guardrail: Deny access to AWS based on the requested AWS RegionInfo. In this Guardrail the SCP is missing the global Service "Artifact" in the SCP Part "Resource": "*",       "Effect": "Deny",       "NotAction": [....How can i customize this SCP?FollowComment"

Answer: "2Hi, I believe the best way to do that is with your own custom SCP deployment rather than use the Region Deny setting in Control Tower, as it can't be modified. You can use the same template that Control Tower uses via this link. And deploy it via your own processes, which may use Customizations for Control Tower, Account Factory for Terraform or other infrastructure as code process.CommentShareJimmy_Manswered 10 months agoAlex 10 months agoI opened a case and the customer support created a request at the internal team but for now just custom SCP or deactivation as workaround is possible.Share"


Question: aws lex bot api

"I'm trying to export my lex bot using aws lexv2 api, I used this api url:https://runtime-v2-lex.us-east-1.amazonaws.com/bot/botname/alias/botaliasname/user/userid/textH: Content-Type : application/jsonbut I got this error :{"Output": {"__type": "com.amazon.coral.service#UnknownOperationException"},"Version": "1.0"}any idea how to fix it ?FollowComment"


Question: Integrating Glue Catalog with EMR on EKS Managed Endpoint for EMR Studio

"In EMR Studio, when attaching an EMR Virtual cluster to the notebook, Glue catalog is not accessible. Some common errors on trying to access Glue are:"Hive support is required to ...""Table or view not found ..."Adding enableHiveSupport() to the spark statement does not seem to work either. For example:spark = SparkSession \    .builder \    .appName("Python Spark SQL Hive integration example") \    .enableHiveSupport() \    .getOrCreate()What configuration is required when connecting from the notebook kernel to make Glue catalog accessible to the notebook?FollowComment"

Answer: "0Accepted AnswerFor EMR Studio to connect to EMR on EKS, a managed endpoint needs to be created. This managed endpoint needs to be configured to use Hive as the catalog and to point to Glue. Use the following CLI to configure the managed endpoint that is able to connect to Glue as the catalog:aws emr-containers create-managed-endpoint \--type JUPYTER_ENTERPRISE_GATEWAY \--virtual-cluster-id ${virtclusterid} \--name virtual-emr-endpoint \--execution-role-arn ${role_arn} \--release-label ${emr_release_label} \--certificate-arn ${certarn} \--region ${region} \--configuration-overrides '{    "applicationConfiguration": [      {        "classification": "spark-defaults",        "properties": {          "spark.hadoop.hive.metastore.client.factory.class": "com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory",          "spark.sql.catalogImplementation": "hive"        }      }    ]  }'See the following documentation for more information about the various flags and description: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-create-eks-cluster.htmlCommentShareMatthew Tananswered 2 years ago"


Question: How to pass environment variables in sagemaker tuner job

"Sagemaker training jobs support setting environment variables on-the-fly in the training job: "Environment": {       "string" : "string"    },https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.htmlI did not find an equivalent for the tuner jobs:https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateHyperParameterTuningJob.htmlAccording to my testing, the SagemakerTuner in the python SDK simply ignores the environment variables set in the passed estimator.Is there any way to pass environment variables to the training jobs started by a tuner job programmatically, or is that currently unsupported?FollowComment"

Answer: "1Accepted AnswerThanks for raising this. Yes, as you point out the Environment collection is not supported in the underlying CreateHyperparameterTuningJob API and therefore the SageMaker Python SDK can't make use of it when running a tuner.As discussed on the SM Py SDK GitHub issue here, you might consider using hyperparameters instead to pass parameters through to the job?If you specifically need environment variables for some other process/library, you could also explore setting the variables from your Python script (perhaps to map from hyperparam to env var?).Or another option could be to customize your container image to bake in the variable via the ENV command? For example to customize an existing AWS Deep Learning Container (framework container), you could:Use sagemaker.image_uris.retrieve(...) to find the base image URI for your given framework, version, region, etc. You'll need to authenticate Docker to this registry as well as your own Amazon ECR account.Create a Dockerfile that takes this base image URI as an arg and builds FROM it, something like this exampleAdd the required ENV commands to bake in the (static) environment variables you needdocker build your custom container (passing in the base image URI as a --build-arg), upload it to Amazon ECR, and use in your SageMaker training job.CommentShareEXPERTAlex_Tanswered 6 months ago"


Question: "instances are stuck in 'Stop' state, can you start them?"

"Hi, we have 1 EC2 instances that are stuck in the 'stop' state for 1 hour now, and we cannot start them.i-027743f1aeac6a2c7Can you help?thanks!Alexander TungFollowComment"

Answer: "0Thank you , problem sloved.CommentShareTungAlexanderanswered 2 years ago"


Question: Unable to send SMS through AWS SNS?

While sending SMS from backend we are receiving "SMS SENT" in our terminal but when we checked it in the AWS SNS(Text Messaging) graph all our SMS requests have failed. We would like to know what could be the reason for failure of messages as it was working fine until few days now the failure rate is 100%.FollowComment

Answer: 0I recommend that you open a support case.CommentShareEXPERTUrianswered 8 months ago


Question: Workspaces Group Policy configuration template for WSP

Does Workspaces Streaming Protocol (WSP) have any GPO templates similar to PCoIP?Like the ability to disable USB drive redirection through the AWS Workspace client?FollowComment

Answer: "0Accepted AnswerWSP WorkSpaces does have GPO templates. You can install it from a running Windows WorkSpace, make a copy of the wsp.admx and wsp.adml files in the C:\Program Files\Amazon\WSP directory.Currently you can only control:Configure Printer Support for WSP Enable or Disable ClipboardRedirection for WSP Enable or Disable Video-In Redirection for WSPEnable or Disable Audio-In Redirection for WSP Disable Time ZoneRedirection for WSP Enable or Disable Smart Card Redirection for WSPEnable or Disable Disconnect Session on Screen Lock for WSPCommentShareEXPERTDzung_Nanswered 2 years ago"


Question: "No MFA device is assigned, still being requested to enter MFA Code"

"Hi,one of our IAM users, started getting MFA Code entry screens on login, although no MFA device is configured in IAM/users/Security Credentials/Assigned MFA device.What else do I need to check?FollowComment"

Answer: "1Is your account part of a cross acount setup or organisation. Could be configured for the aws account?CommentShareAWS-Useranswered 10 months agorePost-User-1949278 10 months agoNo other IAM user is requested to provide an MFA code. I have verified the roles and do not see an MFA setting there.Share0Hi rePost-User-1949278,I remember I had a similar issue a while ago. It was like a "ghost" MFA configuration. I recommend installing the AWS Cli (step-by-step in case you need it) and "digging" the user information from the CLI. Here are some commands that might help:aws iam list-mfa-devices --user-name $YOUR_USERIf the command above retrieves any information like the following structure:{      "MFADevices": [        {            "UserName": "YOUR_USER",             "SerialNumber": "arn:aws:iam::$account-number:mfa/YOUR_USER",              "EnableDate": "YYYY-MM-DDTHH:MIMSS"        }    ]}You may delete the MFA bind to the the account with the aws iam delete-virtual-mfa-device --serial-number $VIRTUAL_DEVICE_ARN command.I hope this helps.Best regards, Gabriel BartholomeuCommentShareGabriel Bartholomeuanswered 10 months agorePost-User-1949278 10 months agoI tried this and interestingly the MFA device list was empty. I reset the password for this user and it doesn't ask for MFA code now. Maybe it was a temp bug on AWS for a brief time.Share"


Question: Configure PHP Laravel client to send messages from an AWS Workmail Account

"Hello, Community.I have an AWS Workmail Organization with one user (myuser@mydomain.com).I have a PHP Laravel app and I want to configure .env file so my app can send messages using myuser@mydomain.com email account. How can I accomplish this without using SES?Thanks in advanced.Regards,FollowComment"

Answer: "0Hi,You should be able to use the WorkMail SMTP end-point. documentation. Do note that WorkMail is a mail and calendaring solution that is not intended for automated mailing and violation of its terms can lead to being blocked from sending mail. For automated mailings its recommended to use SES.Kind regards,RobinCommentShareMODERATORrobinkawsanswered a year agoAliantek a year agoThanks, Robin.So, How Can I configure SES to send messages using my WorkMail Account (myuser@mydomain.com) from PHP Laravel client? Because I see that the parameter asks for a SMTP user name. So, How Can I use my WorkMail email account as username?Thanks in advanced.Sharerobinkaws MODERATORa year agoHi,You will need to configure this in SES by verifying your email address in the SES console.Kind regards,RobinShare"


Question: How to overcome document (pdf) size limit with Comprehend

"Hello community,I am attempting to implement the tutorial found in the Kendra documentation (p. 245) to create an intelligent search tool, and the first step after storing the data in S3 is leveraging AWS Comprehend entities analysis. I'm using my own data instead of the tutorial, to test a real-world use case, and I'm finding the file size limit to be quite ridiculous* (1MB cap) on pdf or word doc's according to the documentation and the error I first got when attempting for the first time - "SINGLE_FILE_SIZE_LIMIT_EXCEEDED, etc. 1048576 bytes allowed for ONE_DOC_PER_FILE format".I put an asterisk next to ridiculous because I suppose this is relative, but I would tend to believe that most real-world applications have documents that are larger. Not to mention what the limits are for other operations, like most other asynchronous operations. I'm someone who has some practical programming experience with ML in python, so when attempting to look at the possible work-arounds or solutions a couple of things came to mind -Use the CreateEntityRecognizer API along with python/boto3 SDK - not sure this would work or be any different, according to documentation it appears this would fall under custom entity recognitionDo my own portion of solutioning in python and use something like a tokenizer - If I'm doing this I might as well do most of my work outside of leveraging any AWS ML platform...The KISS approach: simply "chunk" up my pdf doc's so that they are all less than the 1MB cap, ensuring to keep context intact while doing soAny thoughts, comments, or suggestions are appreciated.Thanks!FollowComment"

Answer: "1Amazon Comprehend limits are documented here https://docs.aws.amazon.com/comprehend/latest/dg/guidelines-and-limits.html and indeed for entity detection documents cannot be bigger than 1MB in size. I would suggest you take your latest approach, that is splitting documents in 1MB max chunks and perform entity detection on those. When building the Kendra index you can then aggregate the entities for each chunk and associate them to the original document.CommentShareEXPERTMassimilianoAWSanswered a year agoIMTheJuggernautX a year agoThat list bit about associating all the "bits" of the broken up documents together when using Kendra was something I was concerned about, so I think that answers that - with Comprehend it seems the ends shall justify the means, if you will. I'll continue testing and see what happens. Thanks for your answer!*Just wanted to update, the next roadblock is that although Comprehend will accept PDF files, it won't actually produce any metadata with them because they aren't UTF-8 formatted, which I found out the hard way after looking at the "output" file. So now I have to add an extra step to all of this and convert any pdf to UTF-8.Share0According to https://docs.aws.amazon.com/comprehend/latest/dg/guidelines-and-limits.html#limits-custom-entity-recognition, the max document size for PDF and Word documents should be 50MB and 5MB, respectively, and the max document size for UTF-8 encoded plain-text documents is 1MB.CommentSharednlenAWSanswered 10 months ago"


Question: NLB preserving client IP addresses in combination with NACL having source CIDR constraint

"I have a VPC with two subnets, each subnet containing an EC2 instance accessible via port 80. There is a NACL associated with both subnets restricting inbound traffic to a certain source CIDR outside of AWS. An internet-facing NLB is configured to route traffic to the instances via instance id.If "preserve client ip addresses" is disabled, everything works fine, requests originating from the correct CIDR are reaching port 80. But if it is enabled, my requests are timing out.A solution is to add a rule to the NACL allowing inbound traffic from the VPC itself. This is in line with what the documentation says. But I don't understand why this is required only when preserving client ip addresses. It is preserving the source ip address, it should be covered by the original NACL.I guess the answer is something like "... because Hyperplane", but I would like to have a deeper understanding.FollowComment"

Answer: "0Does a security group associated with the EC2 instances permit traffic from the external source CIDR?If the instance security group rules permit access from the VPC CIDR but not the external source CIDR, requests will succeed when client IP preservation is disabled (because the instances see requests coming from the NLB) and fail when client IP preservation is enabled (because the instances see requests coming from the external source CIDR).CommentShareMikeJanswered 8 months ago0The security group associated with the instances is allowing port 80 from 0.0.0.0/0. I am even able to bypass the LB and talk to the instances directly from the external source CIDR.CommentSharerandyanswered 8 months ago"


Question: Why doesn't Appsync use DAX for caching rather than ElasticCache for Redis?

"Behind Appsync is DynamoDB and I think that DAX should be use for caching. However, from the Appsync document, it says that ElasticCache for Redis is sued for caching in Appsync. Can anyone explain why is that?FollowComment"

Answer: "-1Hello,AWS AppSync’s server-side data caching capabilities backed by Amazon ElastiCache for Redis, makes data available in a high speed, in-memory cache, improving performance and decreasing latency.If you want to connect AppSync with DynamoDB, you can use Unit Resolvers which provide a direct resolver connection to DynamoDB. Unit resolvers run against the Database, thereby providing a singular response and request mapping template using Apache Velocity Template Language (VTL).A request will be taken as input, and then outputs a JSON document with instructions for the resolver.To learn more about Unit Resolvers, work through the how to configure resolvers developer guide.Further more:Caching within AppSync works at two levels:Caching across the full API:This means, If the data is not in the cache, it is retrieved from the data source and populates the cache until the time to live (TTL) expiration. All subsequent requests to the API are returned from the cache. This means that data sources aren’t contacted directly unless the TTL expires.Caching per resolver level (including pipeline resolvers):With this setting, each resolver must be explicitly opted in for it to cache responses.Note: This is important as the other GraphQL endpoints don’t require caching in AppSync and with this situation, you can explicitly cache at the database level i.e. DynamoDB DAX.This reduces the need to directly access data sources, whether that be Lambda, DynamoDB, Serverless Aurora, HTTP or more.For certain endpoints you may look to utilise AppSync caching. The AppSync API resolves to DynamoDB through Lambda for its data, and you'd use DAX as a cache sitting in front of the database. You can utilise DAX to cache at the database level here.I hope this helps.CommentShareLungaanswered a year agohai a year agoIt does not explain why AppSync using ElasticCache rather than DAX? Note that behind AppSync is DynamoDB.Share"


Question: Redis is stuck in the create status for cluster and in the delete for node

"I tried to upgrade the Redis engine to version 6.2, and on the first failed attempt I had connectivity issues on my side, so I started it again, and when I saw the same error for node 1, I tried to delete it, and now it's stuck. I am using CloudFormation to deploy a Redis cluster and there seems to be some kind of conflict. Could you help me?FollowComment"


Question: IDP authentication and map users in IAM Management account .

"We have recently migrated to IAM identity center in favour of user in a group in security account assuming roles of target accounts.The experience has been great so far. love the UI to instantly switch beetween the accounts etc.we are also using AzureAD integration for IDP . but with this we have to wait for sometime for the SCIM protocol to complete the groups to be mapped (or sent to) to AWS.this limits and slows the process provisioning.we would like to see if its possible , just use the Authentication part of the IDP and then create the userclaim as a member of IAM groups in IAM identity center ( may be by using the email address).This would solve most of the use case . and also speeds up provisioning and decouples creating groups in AzureAD for this purpose and hence the Application Group assignments as well and then SCIM.Thanks.FollowComment"

Answer: "0Hello,I hope you're doing well.Thank you for reaching out to us with your concern.From your query, it seems that you have recently migrated to IAM identity center and you have mentioned that the issue is you have to wait for sometime for the SCIM protocol to complete the groups to be mapped (or sent to) to AWS and you would like to see if its possible to  just use the Authentication part of the IDP and then create the userclaim as a member of IAM groups in IAM identity centerI would suggest you to refer the following documentTutorial: Azure AD SSO integration with AWS Single-Account Access - https://learn.microsoft.com/en-us/azure/active-directory/saas-apps/amazon-web-service-tutorialAs you mentioned that you would like to see if its possible to  just use the Authentication part of the IDP and then create the userclaim as a member of IAM groups in IAM identity center.I would like to inform you that the role mapping is pre-defined and the attribute which comes as part of SAML assertion defines what role is supposed to be assumed. Hence , the use-case you are trying to achieve seems to be not possible.Moreover, as I have only limited information, I would need more data to understand your use case much better. To understand your use case further, I would request you raise a technical support case ticket, where we can go deep dive into the use case and suggest you actions  based on your environment resources.CommentShareSUPPORT ENGINEERrePost-User-0125464answered 2 months ago"


Question: EC2 instance reboot/outage?

"HiDid anyone else have an reboots/outages in us-east-1 couple of hours ago? 5:29 AM UTCmy instance has rebooted with no other log entries inside OS, and no events in CloudTrailmy InstanceId (if that helps) i-0a4f0f23503d0dd5ais it possible to see such events anywhere?ie https://status.aws.amazon.com/ doesn't show any warnings/errorsThanks,ConstantinFollowComment"

Answer: "0Helloif you have the cloudwatch agent, you might be able to see more details about the instance OSand you could also create a cloudwatch alarm to notify you immediately if there is a status changehttps://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-system-instance-status-check.html#types-of-instance-status-checkswith this, you might be able to go to the OS system logs and find something, cloudwatch agent logs, or cloudtrailhope this helpsRTCommentSharerttanswered 4 years ago"


Question: IPv6 support for Cloudwatch Logs

The CloudWatch logs endpoints are all IPv4-only (e.g. logs.eu-west-2.amazonaws.com). This makes it difficult to e.g. export logs from a CloudWatch agent running on an IPv6-only EC2 instance. Is there a roadmap for supporting IPv6 on these endpoints?FollowComment

Answer: "0Hi ,Here is the list of the services that suports ipv6https://docs.aws.amazon.com/general/latest/gr/aws-ipv6-support.htmlCloudwatch is not in the list but  you can keep track of this page.CommentShareAWS-User-7653869answered 6 months ago"


Question: Can we connect to the instance (via ssh or other means) where a Triton Sagemaker endpoint is deployed?

"I am deploying a Triton server endpoint on Sagemaker and I want to ssh into the instance where the endpoint is running for debugging purposes. I can't find a way to identify the instance (e.g. find the instance id) and connect to it. I saw this repo, https://github.com/aws-samples/sagemaker-ssh-helper#inference, but it seems it only works with estimator objects on Sagemaker. I was wondering if there is a way to do the same thing with Sagemaker client objects: botocore.client.SageMaker. I am using the Sagemaker client to create the endpoint with the Triton image on aws ECR.FollowComment"

Answer: "1That SSH Helper library/sample is likely still the approach you want to take - just may need some extra hacking since it's built around the high-level SageMaker Python SDK as you mentioned. The SMPySDK is open source and uses the same service APIs as boto3 does under the hood, so it's just a matter of figuring out how the SSH connectivity works and replicating it in your low-level API calls.Per the SSH Helper Readme, to use the helper with an endpoint you need to:Add the helper library as a dependency to the model, which equates to including it under a /code folder of your model.tar.gz (see re-pack decision here and repack_model implementation here in SMPySDK)Edit your inference.py to import and set up the SSH helper library.On the automated side, from the SSHModelWrapper source code it looks like the only modification the class makes to the Model object is to add some environment variables to the model definition:env.update({'START_SSH': str(self.bootstrap_on_start).lower(),                    'SSH_SSM_ROLE': self.ssm_iam_role,                    'SSH_OWNER_TAG': user_id,                    'SSH_LOG_TO_STDOUT': str(self.log_to_stdout).lower(),                    'SSH_WAIT_TIME_SECONDS': f"{self.connection_wait_time_seconds}"})...So if you're creating and deploying your model objects via boto3, I believe you should be able to get an equivalent setup by doing the same steps:Ensuring your model.tar.gz contains the SSH helper library code and an appropriate inference.py under the code/ subfolderSetting the Environment variables the SSH helper library expects when calling CreateModel.The fastest way to debug/get this working may be to create a temporary model+endpoint using the SM Python SDK and then inspect the created model.tar.gz and use DescribeModel / DescribeEndpointConfig / DescribeEndpoint to fully understand what configuration you need to replicate. (To clear up a misconception I've heard in the past: Yes, you can import a pre-trained model.tar.gz bundle using SM Python SDK Model object... You don't need to start from an Estimator and run a training job from scratch)If you're not using the ML framework containers (for example you're using a built-in SageMaker algorithm, or a JumpStart model, or a from-scratch custom container instead of the AWS-provided frameworks for e.g. PyTorch, TensorFlow, etc), then your serving stack might not support an inference.py script bundle in which case things are a bit more complicated: You'd need to bake the SSH library into the container image itself and edit the serving stack to make sure it gets initialized.CommentShareEXPERTAlex_Tanswered 3 months ago"


Question: "Instance stuck in "shutting down" state, need forced termination"

"An instance in our VPC is stuck in a "shutting down" state. Posting for help to force terminate it, as per the suggestion in https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesShuttingDown.html#instance-stuck-terminating.Instance id: i-0d8f74b6ecaa8d936Actions taken: Deleted a CloudFormation stack that created this instance. Waited for the instance to stop, and it appears stuck. The CloudFormation stack can't be recreated properly until the instance is removed.Edited by: matt-coleman on Mar 14, 2019 9:30 PMFollowComment"

Answer: 0Looks like the instance has finally shutdown and terminated.It would be nice if users were able to signal an instance as being "stuck" in order to trigger some sort of forced removal.CommentSharematt-colemananswered 4 years ago0Problem solved itself after a few hours.CommentSharematt-colemananswered 4 years ago


Question: CUDOS - Trusted Advisor Dashboard New Updated Version Failing to Deploy

"Hello,Is anyone having this issue where the Trusted Advisor dashboard deployment is failing with this error -Error: error waiting for CloudFormation Stack (XXXXXXXXXXXXXXXXXXXXX) update: failed to update CloudFormation stack (UPDATE_ROLLBACK_COMPLETE): ["Received response status [FAILED] from custom resource. Message returned: Athena query failed: line 219:1: Column 'resource' cannot be resolved.I checked in Athena, the tableta_organizational_view_reports does not have this column nameFollowComment"

Answer: "0Hello, yes we observe that on previous versions of CID deployment toolPlease do following:Udate your CFN stack parameter CidVersion to 0.2.12Retry TAO deployCommentShareIakovanswered 4 months ago"


Question: FlexMatch maxLatency expansion based on player attribute

"Hi there,We're looking into adding a "max latency" setting on the client, so that someone is only matched on a server that's under their specified max ping.Eg. "I only want to get into a match which will be placed on a server under 100ms ping to me"Currently we're using expansions to relax the built-in maxLatency parameter over time, starting at 100ms and going up around 200ms over a few minutes (to prioritise games with lower latency, but still allow players to match with those further away from them eventually). This is good for players which don't mind a laggier match, but we've heard from others who'd rather queue for longer in case others in their region start searching than be placed on a server that's far away.Is there a way base the expansion values on attributes that the player has passed up themselves? Even if there needs to be an attribute per step (eg. maxPingStepOne: 120, maxLatencyStepTwo: 160)?In the meantime, it seems like we'd basically need to calculate which regions are under the player's desired max latency, and pass that white list of regions up as a player attribute, then do a set intersection to only match players with one of those regions in common (similar to what we do for platforms).It's a bit of a weird one, because this sort of needs to be done at the matchmaking level rather than the queue level, but the queue is the real source of truth for what regions we have fleets available in.I might go ahead and implement the set intersection anyway, but am interested if there's a more elegant solution.Edit: An additional question! Although we're only relaxing our latency rule's maxLatency to 200ms, we're still seeing players from the other side of the world matched together. This seems to be possible because they each have some regions in common under 200ms, and so pass the maxLatency check. The problem is that the match will then be placed on a server, and the queue doesn't seem to take into account the value of maxLatency at whatever expansion they were matched, so just picks a fleet with the lowest average latency to all players. If there are multiple players in one region then the average latency to that region can be pulled down enough to place on a fleet there, even though that fleet was over someone else's maxLatency. I've now set our queue's latency policy to max 200ms, which is our "last" expansion, though there could still be edge cases where players are matched before they've hit the last expansion.I could potentially set up the queue to spend a few seconds searching for a fleet below each expansion's latency rule maxValue (there should always be available game sessions on all fleet regions), though feels like there could be some edge cases I haven't thought of so am reluctant to do something more complex like that.Cheers,GeordieFollowComment"

Answer: 0Did you ever receive any direction on official support for this?  Was looking to see if there was any support for this exact situation.ThanksCommentSharerePost-User-4882916answered 3 years ago


Question: Action Required: Irregular activity in your AWS account.

"I have a  problem, AWS reported suspicious activity and sent me a bill for $4200. I have never used AWS since I signed up 2 years ago. Yesterday, I spent the day with the help desk to delete the log groups, models etc... My bill is blocked because my credit card is no longer valid. The call center told me to put my new valid card which will be charged $4200 and then recredited by them. But I think it's risky. I want to leave my expired card which prevents a charge. I don't know who to do about this bill. (AWS recognizes that my account has been hacked)FollowComment"

Answer: "1Hello,I highly recommend you take a look to this guide: What do I do if I notice unauthorized activity in my AWS account? and also reach out to AWS Support for Billing.CommentShareMickael-AWSanswered 10 months agorePost-User-4322630 10 months agoThank you but I have already secured my account. I never use it.  My question is: how do I get out of it regarding the bill (which is not due by me because my account was hacked)ShareMickael-AWS 10 months agoYou need to reach out to AWS Support so they can address the Billing issue.Share"


Question: aws glue job failed

"Apr 17, 2022, 9:18:07 PM Pending execution2022-04-17 15:48:28,534 main WARN JNDI lookup class is not available because this JRE does not support JNDI. JNDI string lookups will not be available, continuing configuration. java.lang.ClassNotFoundException: org.apache.logging.log4j.core.lookup.JndiLookup at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.logging.log4j.util.LoaderUtil.loadClass(LoaderUtil.java:173) at org.apache.logging.log4j.util.LoaderUtil.newInstanceOf(LoaderUtil.java:211) at org.apache.logging.log4j.util.LoaderUtil.newCheckedInstanceOf(LoaderUtil.java:232) at org.apache.logging.log4j.core.util.Loader.newCheckedInstanceOf(Loader.java:301) at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:95) at org.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:114) at org.apache.logging.log4j.core.config.DefaultConfiguration.<init>(DefaultConfiguration.java:55) at org.apache.logging.log4j.core.layout.PatternLayout$Builder.build(PatternLayout.java:430) at org.apache.logging.log4j.core.layout.PatternLayout.createDefaultLayout(PatternLayout.java:324) at org.apache.logging.log4j.core.appender.ConsoleAppender$Builder.<init>(ConsoleAppender.java:121) at org.apache.logging.log4j.core.appender.ConsoleAppender.newBuilder(ConsoleAppender.java:111) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.createBuilder(PluginBuilder.java:158) at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:119) at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:813) at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:753) at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:745) at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:389) at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:169) at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:181) at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:446) at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:520) at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:536) at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:214) at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:146) at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41) at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194) at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:597) at org.apache.spark.metrics.sink.MetricsConfigUtils.<clinit>(MetricsConfigUtils.java:12) at org.apache.spark.metrics.sink.MetricsProxyInfo.fromConfig(MetricsProxyInfo.java:17) at com.amazonaws.services.glue.cloudwatch.CloudWatchLogsAppenderCommon.<init>(CloudWatchLogsAppenderCommon.java:62) at com.amazonaws.services.glue.cloudwatch.CloudWatchLogsAppenderCommon$CloudWatchLogsAppenderCommonBuilder.build(CloudWatchLogsAppenderCommon.java:79) at com.amazonaws.services.glue.cloudwatch.CloudWatchAppender.activateOptions(CloudWatchAppender.java:73) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:81) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383) at org.apache.spark.network.util.JavaUtils.<clinit>(JavaUtils.java:41) at org.apache.spark.internal.config.ConfigHelpers$.byteFromString(ConfigBuilder.scala:67) at org.apache.spark.internal.config.ConfigBuilder$$anonfun$bytesConf$1.apply(ConfigBuilder.scala:235) at org.apache.spark.internal.config.ConfigBuilder$$anonfun$bytesConf$1.apply(ConfigBuilder.scala:235) at org.apache.spark.internal.config.TypedConfigBuilder$$anonfun$transform$1.apply(ConfigBuilder.scala:101) at org.apache.spark.internal.config.TypedConfigBuilder$$anonfun$transform$1.apply(ConfigBuilder.scala:101) at org.apache.spark.internal.config.TypedConfigBuilder.createWithDefault(ConfigBuilder.scala:143) at org.apache.spark.internal.config.package$.<init>(package.scala:121) at org.apache.spark.internal.config.package$.<clinit>(package.scala) at org.apache.spark.SparkConf$.<init>(SparkConf.scala:716) at org.apache.spark.SparkConf$.<clinit>(SparkConf.scala) at org.apache.spark.SparkConf.set(SparkConf.scala:95) at org.apache.spark.SparkConf$$anonfun$loadFromSystemProperties$3.apply(SparkConf.scala:77) at org.apache.spark.SparkConf$$anonfun$loadFromSystemProperties$3.apply(SparkConf.scala:76) at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:221) at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:428) at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:428) at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732) at org.apache.spark.SparkConf.loadFromSystemProperties(SparkConf.scala:76) at org.apache.spark.SparkConf.<init>(SparkConf.scala:71) at org.apache.spark.SparkConf.<init>(SparkConf.scala:58) at com.amazonaws.services.glue.SparkProcessLauncherPlugin$class.getSparkConf(ProcessLauncher.scala:41) at com.amazonaws.services.glue.ProcessLauncher$$anon$1.getSparkConf(ProcessLauncher.scala:78) at com.amazonaws.services.glue.ProcessLauncher.<init>(ProcessLauncher.scala:84) at com.amazonaws.services.glue.ProcessLauncher.<init>(ProcessLauncher.scala:78) at com.amazonaws.services.glue.ProcessLauncher$.main(ProcessLauncher.scala:29) at com.amazonaws.services.glue.ProcessLauncher.main(ProcessLauncher.scala)FollowComment"

Answer: "1Hi - Is this warning message impacting your Glue job operation? As a part of  the patch for "Apache Log4j2 Issue (CVE-2021-44228)", the vulnerable JNDI lookup classes were removed from the Log4j jars, which is the reason for the class not found warnings. These warnings can be safely ignored.Snippet from the security bulletin:We've taken this issue very seriously, and our world-class team of engineers has fully deployed the Amazon-developed Java hot patch available here to all AWS services. The hot patch updates the Java VM to disable the loading of the Java Naming and Directory Interface (JNDI) class, replacing it with a harmless notification message, which mitigates CVE-2021-44228 and CVE-2021-45046.  We will shortly complete our deployment of the updated Log4j library to all of our servicesCommentShareEXPERTGokulanswered a year ago0Hi,your question mentions the job failing. Is that so?In that case you should look in the logs for additional errors, to try to get a better understanding of the logs you can enable AWS Glue Job Insights, to try to get some more specific information on the exact error causing the job failure.If the only issue is the warning error, as mentioned in the previous answer this can be ignored.CommentShareEXPERTFabrizio@AWSanswered a year ago"


Question: AMI Customizations disappear mysteriously after boot

"Over the past few weeks, I've been seeing a very strange issue.  At first I thought I must have been missing a step in my AMI creation process, but last night it happened again.  Here's what we did and experienced:Used a RHEL-7 AMI as a base, and we installed our own codebase and libraries on there.  From that we created an AMI.   As we've discovered more dependencies and updated the codebase, we've updated the AMI to include the new stuff.Last night then, I booted up an instance from the AMI. We ssh'd in (via AWS SSM Session Manager) and checked and everything was working. Randomly about 40 minutes later it stopped pinging into our pingdom dashboard.In trying to investigate: we couldn't start an AWS SSM session. Instead we had to ssh in via the old jump host.On the instance intself -- the ssm-agent is not even present, not running.  The custom config files (crontab and scoutdaemon) are no longer even on the system!  The instance just looks like some plain RHEL from a couple of months ago, not the one we booted last night.Has anyone seen this happen before?  What's the underlying cause / what am I doing wrong?This reduces our confidence in the stability of thisFollowComment"

Answer: "0Hello seemant,Could you please share the specific instance IDs, the region in question, the times and dates related to each change (at least approximate if you do not know the exact times, and also please include the time zone), the AMI IDs with the steps that you have taken all chronologically described? We do not have visibility into the instances internally, nor we can see into the AMIs when they are not public, but we might be able to help you to pinpoint the issue if we have more specific information. Further, if you have a Premium Support plan, you can create a new support case to have this all reviewed within the SLA.Thank you for your understanding.Regards,awstomasCommentShareAWS-User-3892169answered 3 years ago0Thank you. I'll close this post for now as I'm no longer generating custom AMIs at the momentCommentShareseemantanswered 3 years ago"


Question: Glue Jobs - G2 worker type - executor not allocating!

"Hi, I am using G2x type worker, during data write operation glue job allocating only 5 executors and not allocating more executors which makes process getting so long. How to resolve this issue.FollowComment"

Answer: "0Hello, AWS Glue has a great feature that you can take advantage of, it will optimize costs and resource usage by dynamically scaling the number of workers up and down throughout the job run.Please Note:  Auto Scaling is available for your AWS Glue ETL and streaming jobs with AWS Glue version 3.0 or later.This means over a job run, when more executors are requested by your Spark application, more workers will be added to the cluster allowing the process being done to be faster. When the executor has been idle without active computation tasks, the executor and the corresponding worker will be removed.Auto scaling feature is just below "Worker Type" option on Job Detail page.https://docs.aws.amazon.com/glue/latest/dg/auto-scaling.htmlCommentShareMuzi_Manswered 16 days agoFabrizio@AWS EXPERT15 days agoThis is a great explanation of the autoscaling feature, and I think it is important to note may help cost optimizing jobs that like in the case of the question cannot scale out and leave executors idles.Need more details on check to be done to understand why the current job would not scale over 5 executor during the write phase.Share0Hi,there might be many different reasons why the job is not allocating more executors, mostly reducing to the fact that your job is going write only a limited number of files and for this only 5 executors are needed, a couple of example why this is happening could be:Number of executors depends on the number of workers you have defined for the job, G2 nodes have 1 executor per worker, you might not have enough workers ( hypotetical, because you did not specify the configuration of the job)there is data skew and only a limited number of partitions are created so to write those partition only 5 executors are needed by sparkthere might be a repartition or coalesce operation to limit the number of files that needs to be write outif the job is actively repartitioning the data then you might want to increase the number of partitions; here you will have a tradeoff between faster performance of writes and size of the files.depending on the AWS Glue job version (v2.0 / v3.0 / v 4.0) and thus the spark version, there might be different techniques to salt the data in case of data skew.Looking at the Spark history server logs , may help troubleshoot more the reasons why this is happening. Some resource on Spark history Server below:AWS Glue Documentation: Monitoring jobs using the Apache Spark web UIAWS Glue Workshop: Enabling Spark UI and Monitoringhope this helpsCommentShareEXPERTFabrizio@AWSanswered 15 days agorePost-User-7121930 15 days agoWe are running the job with 50 G2.x worker nodes , basically the data would be 1250 partitions and total size 120 GB data, likely 5 billion rows.what i observed, initial data reading  from catalog consumes 42/50 ips from subnet however during writing the data to s3 , the max executor consumption between 5-10 only. we enabled the auto scaling, from the job metrics its says required executor equal to no of partitions 1250, but only between 5-10 are active executors . as per the the available worker nodes looks 40 idle.There are no repartition/ coalesce operation when the writing data.we use glue 3.0 version.Share0Thanks for your inputs, i have added salting and added--config on glue job parameters ex."--conf": "spark.executor.memory=28g","--conf": "spark.driver.memory=26g",i have enabled the continuous logs , as below details i found ..  why these failure logs are happening...WARN ExecutorTaskManagement: executor status Success(FAILED) for 48 g-ba0411327eea7bbe2bc63a063ca1de8b7d166064INFO TaskGroupInterface: getting status for executor task g-35aa087036b77f0b0b89c45368add14850cefcbbWARN ExecutorTaskManagement: executor status Success(FAILED) for 27 g-35aa087036b77f0b0b89c45368add14850cefcbbINFO TaskGroupInterface: getting status for executor task g-dcd8e66de97f52d19293f5caa5328ca1c39f2ac2WARN ExecutorTaskManagement: executor status Success(FAILED) for 33 g-dcd8e66de97f52d19293f5caa5328ca1c39f2ac2INFO TaskGroupInterface: getting status for executor task g-4fce6339dba7ec32535a30f5a9525005f0c25a28WARN ExecutorTaskManagement: executor status Success(FAILED) for 12 g-4fce6339dba7ec32535a30f5a9525005f0c25a28INFO TaskGroupInterface: getting status for executor task g-83a793602062e370813da0bc9c4f7813312dab07WARN ExecutorTaskManagement: executor status Success(FAILED) for 15 g-83a793602062e370813da0bc9c4f7813312dab07INFO TaskGroupInterface: getting status for executor task g-9839e7940677dda3c42efc4654452a422639709fWARN ExecutorTaskManagement: executor status Success(FAILED) for 8 g-9839e7940677dda3c42efc4654452a422639709fINFO TaskGroupInterface: getting status for executor task g-16e4047fe0b73994937621a3e830b0ceae8c3a07WARN ExecutorTaskManagement: executor status Success(FAILED) for 42 g-16e4047fe0b73994937621a3e830b0ceae8c3a07INFO TaskGroupInterface: getting status for executor task g-0bf4442d874990951edc71a0d569cfbc8d3dff5fWARN ExecutorTaskManagement: executor status Success(FAILED) for 36 g-0bf4442d874990951edc71a0d569cfbc8d3dff5fINFO TaskGroupInterface: getting status for executor task g-e76afa0b7974c1ecb0870b052af7135fed1878d3WARN ExecutorTaskManagement: executor status Success(FAILED) for 21 g-e76afa0b7974c1ecb0870b052af7135fed1878d3INFO TaskGroupInterface: getting status for executor task g-7369001b1d0764133377e477ea5f9fb8e416715dWARN ExecutorTaskManagement: executor status Success(FAILED) for 18 g-7369001b1d0764133377e477ea5f9fb8e416715dINFO TaskGroupInterface: getting status for executor task g-c76a64979a1b31f2f3c1dbb752b62914b6468f51WARN ExecutorTaskManagement: executor status Success(FAILED) for 24 g-c76a64979a1b31f2f3c1dbb752b62914b6468f51INFO TaskGroupInterface: getting status for executor task g-f76484471d4ca8fa8048e1de3e1924371bb30cddWARN ExecutorTaskManagement: executor status Success(FAILED) for 35 g-f76484471d4ca8fa8048e1de3e1924371bb30cddINFO TaskGroupInterface: getting status for executor task g-51b7f66b831d9f65237d9b0e55de0e72d6e729bdWARN ExecutorTaskManagement: executor status Success(FAILED) for 41 g-51b7f66b831d9f65237d9b0e55de0e72d6e729bdINFO TaskGroupInterface: getting status for executor task g-40da826f314851e71d3b9e812da4982b00114a19WARN ExecutorTaskManagement: executor status Success(FAILED) for 7 g-40da826f314851e71d3b9e812da4982b00114a19INFO TaskGroupInterface: getting status for executor task g-fe46afa0be1ebfb5f75a92bae9ec49ad4d860f7fWARN ExecutorTaskManagement: executor status Success(FAILED) for 17 g-fe46afa0be1ebfb5f75a92bae9ec49ad4d860f7fINFO TaskGroupInterface: getting status for executor task g-91ed08b48433369880860643b88de2d75a1de509WARN ExecutorTaskManagement: executor status Success(FAILED) for 44 g-91ed08b48433369880860643b88de2d75a1de509INFO TaskGroupInterface: getting status for executor task g-e66dcec9a3608c4c2e8feb78db6d0981200a7706WARN ExecutorTaskManagement: executor status Success(FAILED) for 23 g-e66dcec9a3608c4c2e8feb78db6d0981200a7706INFO TaskGroupInterface: getting status for executor task g-6c924ff6f6c421447de5d37850da61730ed8636dWARN ExecutorTaskManagement: executor status Success(FAILED) for 38 g-6c924ff6f6c421447de5d37850da61730ed8636dINFO TaskGroupInterface: getting status for executor task g-a5158ab1b4b9f4490c2b513e27b69900c99b61efWARN ExecutorTaskManagement: executor status Success(FAILED) for 47 g-a5158ab1b4b9f4490c2b513e27b69900c99b61efINFO TaskGroupInterface: getting status for executor task g-3a9aa260fccbd23980d17a1b4af790c6d6c960baWARN ExecutorTaskManagement: executor status Success(FAILED) for 26 g-3a9aa260fccbd23980d17a1b4af790c6d6c960baINFO TaskGroupInterface: getting status for executor task g-3c3272f213f8b2b480069bbda405e86beb0895d7WARN ExecutorTaskManagement: executor status Success(FAILED) for 11 g-3c3272f213f8b2b480069bbda405e86beb0895d7INFO TaskGroupInterface: getting status for executor task g-38cbb4ae9009e9ec921540567e74170496986fb2WARN ExecutorTaskManagement: executor status Success(FAILED) for 32 g-38cbb4ae9009e9ec921540567e74170496986fb2INFO TaskGroupInterface: getting status for executor task g-ee91ed94060467a290da4eeaaac5e07a7c9bbb04WARN ExecutorTaskManagement: executor status Success(FAILED) for 14 g-ee91ed94060467a290da4eeaaac5e07a7c9bbb04INFO TaskGroupInterface: getting status for executor task g-89ae7c53fc8f7ed049a097e6982808b56690004eWARN ExecutorTaskManagement: executor status Success(FAILED) for 29 g-89ae7c53fc8f7ed049a097e6982808b56690004eINFO TaskGroupInterface: getting status for executor task g-cf97b33b99d5c80417373fa7f4e06ca39226fe78WARN ExecutorTaskManagement: executor status Success(FAILED) for 20 g-cf97b33b99d5c80417373fa7f4e06ca39226fe78INFO TaskGroupInterface: getting status for executor task g-f6e35d669143a6570c44b1d5dfad5cee39c3d5aaWARN ExecutorTaskManagement: executor status Success(FAILED) for 46 g-f6e35d669143a6570c44b1d5dfad5cee39c3d5aaCommentSharerePost-User-7121930answered 15 days agoGonzalo Herreros 14 days agoI would advise you not to touch any memory settings. SparkUI https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html will tell you what are those errors (they should also appear in the log but probably are much harder to find)Share"


Question: How to fully uninstall aws client VPN

"So I'm trying to use AWS VPN Client on mymacOS Monterey version 12.4Macbook Pro 16 inch 2019When downloading a .ovpn from "Client VPN endpoints" I cannot seem to connect to the VPN. When I try to connect, I keep getting this from the client"Aws VPN Client is trying to install a new helper tool." in which it asks me for my credentials which I give it. Then immediately afterwards I get"AWS VPN Client Helper Tool is required to establish the connection."I've reinstalled the software about 8 times.Anytime I try to use the "Uninstall AWS VPN Client" I get "Uninstalled failed" with no additional information.So I manually delete the files, I run this command to do additional cleanupsudo /bin/launchctl remove /Library/LaunchDaemons/com.amazonaws.acvc.helper.plistsudo rm /Library/LaunchDaemons/com.amazonaws.acvc.helper.plistsudo rm /Library/PrivilegedHelperTools/com.amazonaws.acvc.helperI feel like im missing something obvious. It was working fine on my computer until recently. I've confirmed on another computer that the certs work fine. So if there is additional cleanup I can do, let me know.So I was wondering if this is a known issue or just me? I looked around and couldn't find any information in the FAQ or troubleshooting of it.FollowComment"

Answer: "0Hello,Have you looked at the below troubleshooting instructions for MacOS?https://docs.aws.amazon.com/vpn/latest/clientvpn-user/macos-troubleshooting.htmlThe other thing is, you could try Tunnelblick VPN client and see if that works properly if you have issues with the current client.https://docs.aws.amazon.com/vpn/latest/clientvpn-user/macos.htmlCommentShareEXPERTTushar_Janswered 10 months ago"


Question: AWS File Transfer Logical directory Structure

"Hi All,We are trying to setup simple directory structure in S3 bucket for each user when they login to AWS file transfer SFTP server.${transfer:UserName}/folder1${transfer:UserName}/folder2We have Active directory group A access added to File Transfer server. So only group A users will able to access file transfer server.As soon as user login to SFTP, user should be able to see both child directory under his/her home directory and transfer files to respective directory.Please advise how to achieve this?FollowComment"

Answer: "0Hello,Greetings of the day!I can see that you need guidance regarding setting up permission for a user to access Transfer for SFTP.I would like to mention that you can control a user's access to AWS Transfer Family resources by using an AWS Identity and Access Management (IAM) policy. An IAM policy is a statement, typically in JSON format, that allows a certain level of access to a resource. You use an IAM policy to define what file operations that you want to allow your users to perform and not perform. You can also use an IAM policy to define what Amazon S3 bucket or buckets that you want to give your users access to. To specify these policies for users, you create an IAM role for AWS Transfer Family that has the IAM policy and trust relationship associated with it.Please refer below documents as they provide detailed information regarding providing access.[+] Managing access controls - https://docs.aws.amazon.com/transfer/latest/userguide/users-policies.html[+] Creating a session policy for an Amazon S3 bucket - https://docs.aws.amazon.com/transfer/latest/userguide/users-policies.html#users-policies-session[+] https://docs.aws.amazon.com/transfer/latest/userguide/logical-dir-mappings.html[+] https://aws.amazon.com/blogs/apn/designing-a-multi-tenant-sftp-server-with-aws-transfer-family/[+] https://aws.amazon.com/blogs/storage/simplify-your-aws-sftp-structure-with-chroot-and-logical-directories/Further, I request you to open a support case if you need any further assistance.CommentShareSUPPORT ENGINEERRohan_Sanswered 7 months agoUser-07051983 7 months agoHi Rohan,Thanks for the reply. I went through the document and added access policy and IAM role as well but it is not working as expected.We have setup AWS file transfer server with AWS directory service (connected to Microsoft AD) authentication. As per use case, once user login to sftp, user should be able to see two directory within their own folder. {username}/folder1 {username}/folder2I have setup below Access policy and IAM policy (attached to S3)create-access CLI:aws transfer create-access --home-directory-type LOGICAL --home-directory-mappings '[{"Entry":"/folder1","Target":"/bucket_name/${transfer:UserName}/folder1" },{ "Entry": "/folder2", "Target":"/bucket_name/${transfer:UserName}/folder2"}]' --role arn:aws:iam::account_id:role/iam_role --server-id s-1234567876454ert --external-id S-1-2-34-56789123-12345678-1234567898-1234access policy was created successfully.Below IAM role is attached to S3 bucket and file-transfer server.{"Version": "2012-10-17","Statement": [{"Action": ["s3:ListBucket","s3:GetBucketLocation"],"Resource": ["arn:aws:s3:::bucket_name"],"Effect": "Allow","Sid": "ReadWriteS3"},{"Action": ["s3:PutObject","s3:GetObject","s3:DeleteObject","s3:DeleteObjectVersion",Share"


Question: Issue with Remote Desktop Licensing Manager on AWS Microsoft AD

"Hey all, I am trying to setup Remote Desktop Licensing Manager on AWS but i am running into this error with the Manager "The system cannot determine if the license server is member of TSLS group on Active Directory Domain Services (AD DS) because the AD DS cannot be contacted."Now I added the Instanced to the domain and then added the instance to the TSLS group in AD. for whatever reason I am stuck here. All ports are open that are needed. is there something in AWS managed AD that i need to change?FollowComment"


Question: Retrieving CPUUtilization for terminated instances

"I am trying to pull some metrics from CloudWatch programmatically (for example, CPUUtilization). How would I accomplish this if I wanted to go back 6+ months and also get the same metrics for all terminated (and running) instances in that time period?FollowComment"

Answer: "0Accepted AnswerCloudWatch Metrics retain all running/terminated instances metrics, here is the timelineWhat is the retention period of all metrics?CloudWatch launched High Resolution Custom Metrics on July 26, 2017. This enables you to publish and store custom metrics down to 1-second resolution. Extended retention of metrics was launched on November 1, 2016, and enabled storage of all metrics for customers from the previous 14 days to 15 months. CloudWatch retains metric data as follows:Data points with a period of less than 60 seconds are available for 3 hours. These data points are high-resolution custom metrics.Data points with a period of 60 seconds (1 minute) are available for 15 daysData points with a period of 300 seconds (5 minute) are available for 63 days Data points with a period of 3600 seconds (1 hour) are available for 455 days (15 months)Data points that are initially published with a shorter period are aggregated together for long-term storage. For example, if you collect data using a period of 1 minute, the data remains available for 15 days with 1-minute resolution. After 15 days this data is still available, but is aggregated and is retrievable only with a resolution of 5 minutes. After 63 days, the data is further aggregated and is available with a resolution of 1 hour. If you need availability of metrics longer than these periods, you can use the GetMetricStatistics API to retrieve the datapoints for offline or different storage.https://aws.amazon.com/blogs/aws/amazon-cloudwatch-update-extended-metrics-retention-user-interface-update/CommentShareMODERATORAWS-User-8855869answered 3 years ago"


Question: AWS devicefarm - how to disable assistive touch

"Hello AWS Team,I am seeing my tests are failing due to the Assistive touch icon is appearing exactly at the link where I need to Tap to Sign Out and Tests are failing. Is there a way I can set the DesiredCapability to disable it?I am seeing this from last week.I am using TestNG with Java, with Appium 1.9.0. and device iPhone 8 Plus (iOS 12).ThanksChittaFollowComment"

Answer: "0Hello,There is no way to disable assistive touch using a desired capability now. Please share the run url where you saw this issue and we will fix that device.CommentShareanand-awsanswered 4 years ago0Thank you for the reply.I am sharing the URL - https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/310b1ec6-ea89-4644-98f6-f3142813344a/runs/f62fb764-6d45-4840-8ca5-c0e189752a19/jobs/00000please check if you mean this URL or I need to provide some other URL.Thanks for all your help.CommentSharecbeheraanswered 4 years ago0Thanks for letting us know.I've disabled that phone until this can be addressed, if you see the issue again, feel free to let us know.Scott.CommentShareAWS-User-1182145answered 4 years ago0Thank you very much.CommentSharecbeheraanswered 4 years ago"


Question: Delegate subdomain to Route53 to serve static website on S3

"Hi all,Here is my situation: I bought a domain domain.ai on gandi. I would like to serve the subdomain admin.domain.ai through Route 53, to hit an S3 bucket, with HTTPS.I have:1 question: Do I really need Cloudfront to have an HTTPS connection or Route53 to an S3 bucket is enough?1 problem: I followed the guide [1]. A few hours after doing it, admin.domain.ai seemed to hit AWS DNS servers but, with nslookup admin.domain.ai, I received:Non-authoritative answer:Name:admin.domain.aiAddress: 18.203.172.170Could you help me on this?[1] https://docs.aws.amazon.com/fr_fr/Route53/latest/DeveloperGuide/CreatingNewSubdomain.htmlEdited by: AlexandreSicara on Jun 24, 2019 12:13 AMEdited by: AlexandreSicara on Jun 24, 2019 12:14 AMFollowComment"

Answer: "0My answer after a few days:You do need Cloudfront to establish an HTTPS connection with your bucket.To make every request go to admin.domain.ai, here is the path of the request:a. The request hits Gandi (because the domain domain.ai is handled by Gandi)b. Gandi has to redirect it to the AWS servers. For this to happen, you need to add the Name Servers given by Route53:admin 1800 IN NS ns-1194.awsdns-21.org.admin 1800 IN NS ns-1916.awsdns-47.co.uk.admin 1800 IN NS ns-287.awsdns-35.com.admin 1800 IN NS ns-988.awsdns-59.net.Bonus: To have HTTPS connection, here is the process I followed:Go to AWS ACMRequest a certificate for admin.domain.aiAsk for a DNS validationIn Gandi, I added the record:XXXXXXXXXXXXXXXXXX.admin 1800 IN CNAME YYYYYYYYYYY.ZZZZZZZ.acm-validations.aws.Where the X, Y and Z values were given by AWS ACM.The certificate was promptly issued and I added it to my Cloudfront distribution.CommentShareAlexandreSicaraanswered 4 years ago"


Question: Site Access to the ec2 is getting site can't be reached

Access to the ec2 is getting site can't be reachedI have installed the jenkins in ec2 linux. Jenkins is up and running . however when I trying to access the ec2 I am getting the site cant be reached. It is taking too long. I have set the inbound security rule as custom with port 8069FollowComment

Answer: "0Hello,I recommend you go to your ec2 instance,and type netstat -an to see if your server is listening to that port for connection.if you didn't open connection for that port, it will not listen to it.I believe the default Jenkins port is 8080?CommentShareteduri90answered 6 months ago"


Question: How can EC2 maintain IP addresses in a multi-AZ failover situation?

"A vendor provides a service to a single AWS region. A client of that vendor has an EC2 instance residing in a single AZ. The vendor's service is tied to an IP address for licensing purposes.Now let's say that the AZ or specific EC2 fails.I know that you can set auto scaling groups to spin up replacements for failed instances, but the IP address is going to be different and thus the vendor's license scheme will be a blocker.I think I recall the ability to reserve a specific IP for EC2, but haven't touched that to-date. But that being said, can you use that reserved IP address across AZs?So the question is, in the event of a failure of an EC2 instance in AZ1, can I have a specific IP address be picked up for a backup EC2 instance in AZ2?FollowComment"

Answer: "0Accepted AnswerYou can use an Elastic IP address for this. It can be assigned across a region. To re-assign the Elastic IP address when an instance is 'lost', you can use CloudWatch Events.CommentShareEXPERTLeeAtk@AWSanswered 3 years agouv28 2 months agoI see that Elastic IP is region specific and cannot be moved to a different region. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html.Share"


Question: How to see who called me on AWS Connect ? How to create contacts ?

"We started using AWS Connect and had a few questions from the end users and wanted to see if anyone had any solutions to thiswhen someone calls me on AWS Connect, CCP is available, but I'm on a different screen /busy and didn't get to answer the call, but the app only says 'the customer ended the call. how can I see who actually called me?FollowComment"

Answer: 0You can customize CCP by using the "Amazon Connect Streams API".I thought we could use this one and customize it to display the phone number.https://github.com/amazon-connect/amazon-connect-streamsCommentShareEXPERTRiku_Kobayashianswered 2 months ago0The way to do this as scale is to do a custom CCP for sure. The other option that would be available out of the box is to use contact search to find who called.DavidCommentSharedmaciasanswered a month ago


Question: [Announcement] Amazon Elastic File System (Amazon EFS) is now available in the AWS Asia Pacific (Jakarta) Region

"Customers in the AWS Asia Pacific (Jakarta) Region can now use Amazon Elastic File System (Amazon EFS).Amazon EFS is a simple, serverless, set-and-forget, elastic file system that makes it easy to set up, scale, and cost-optimize file storage in the AWS Cloud. It is built to scale on demand to store petabytes of data without disrupting applications, growing and shrinking automatically as you add and remove files, eliminating the need to provision and manage capacity to accommodate growth.For more information, visit the Amazon EFS product page.FollowCommentNguyen_M SUPPORT ENGINEERa year agoThis is an announcement migrated from AWS Forums that does not require an answerShare"


Question: AWS Site-to-Site VPN authentication failing for Customer Gateway behind NAT device

"We are creating an AWS Site-to-Site VPN connection between a client's on-premise network and our AWS VPC. The client receives an authentication error when attempting to establish a connection (using a pre-shared key).In order to debug this, we ran strongSwan on an EC2 instance to be able to inspect the logs and traffic. While doing this, we could see that they were attempting to connect from IP address 1 (e.g. 1.0.0.1) but using IP address 2 (e.g. 1.0.0.2) as an ID. When we setup strongSwan to authenticate against IP address 2 (e.g. 1.0.0.2), the connection was established successfully. We have since learned that IP address 1 (e.g. 1.0.0.1) is their NAT device, and IP address 2 (e.g. 1.0.0.2) is their customer gateway device.To my question: how can I setup the AWS Site-to-Site VPN connection and customer gateway so that they can be authenticated successfully? If I create the customer gateway with IP address 1 (e.g. 1.0.0.1, NAT device) they can connect but can't authenticate. If I create the customer gateway with IP address 2 (e.g. 1.0.0.2, customer gateway device) they can't connect at all.FollowCommentTushar_J EXPERTa year agoIf you can specify the CGW make and model and exact error seen in the logs that would be helpfulShare"

Answer: "0Hello Rudi,If you have NAT device in front of the CGW then you must use the Public IP of the NAT device. It is documented hereInternet-routable IP address (static) of the customer gateway device's external interface - The public IP address value must be static. If your customer gateway is behind a network address translation (NAT) device that's enabled for NAT traversal (NAT-T), use the public IP address of your NAT device, and adjust your firewall rules to unblock UDP port 4500.As for the PSK - are you using auto-generated PSK? you can replace it, instrcutions are hereTo change the IKE pre-shared keyYou can modify the tunnel options for the Site-to-Site VPN connection and specify a new IKE pre-shared key for each tunnel. For more information, see Modifying Site-to-Site VPN tunnel optionsCommentShareEXPERTTushar_Janswered a year ago"


Question: Rekognition search faces API endpoint

"Hi everyone!Currently, I've accomplished detecting all the faces from a collection and then generating sub-galleries of each subject with all their photos associated with the ruby SDK '~> 1.65'To do this, I've indexed the faces of all photos within a collection, list all the faces (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#list_faces-instance_method), then grabbing each face_id recognized and search the faces related to that face_id (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#search_faces-instance_method), and delete the face id used to do the API call and all the returned ones to tell when a new detected subject starts and ends.My issue is that the search faces API returns different results depending on which face id param you are doing the request with. For example, if there are 10 faces ids detected that belong to a person (1, 2, 3, ..., 10), the search faces call with face id = 1 param, should return the faces id (2, 3, 4, ..., 10) but if you continue to do this with the other face ids this is not always the case with some scenarios where the search faces call with face id = 3 has returned a subset of the previously mentioned like just (4, 5, 6).Is there any other way to achieve this to prevent this kind of "error"? if not, this is a real concern for us because it depends on the order in which we call the search faces with different face ids, and sometimes it seems like there is more than 1 subject detected with almost the same photos when in reality it's the same person.Thanks in advance!FollowComment"

Answer: "0Hi, thanks for your reply!All the mentioned above happened with the matching threshold set to 90%, but even after I've changed it to 95% it keeps behaving the same wayCommentShareAWS-User-5694207answered a year agoAWS-User-8651363 a year agoAlso no change when the threshold is under 80% ? Perhaps the similarity between some faces is under the default thresholdShare0Hello, did you notice any correlation with the face matching threshold ? By default it's 80%https://docs.aws.amazon.com/rekognition/latest/APIReference/API_SearchFaces.htmlCommentShareAWS-User-8651363answered a year ago0Hi,we are happy to work with you to investigate this issue. Can you open a case using this link and share the case ID here. We can then engage directly to troubleshoot.CommentShareAWS-litaoanswered a year ago"


Question: Add reader endpoint for DB clusters when creating AWS::SecretsManager::SecretTargetAttachment

"When creating a DB cluster (ie: Aurora Mysql) with autorotated secrets manager credentials, and attaching connection data using AWS::SecretsManager::SecretTargetAttachment, it is missing the replicas read-only endpoint.It would be nice to include it under some readersHost key.Is it possible? Where should I ask for it?FollowComment"


Question: Payment successful and payment past due

What does this error mean when making payment and the card is not charged automatically on an account.This is the error message when after making paymentYou have 1 payment past dueYou successfully competed paymentPlease advise.FollowComment

Answer: "0As shown in the following document, if you check the Payment Due tab and nothing is overdue, you have made a normal payment and are good to go.https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/manage-making-a-payment.html#view-payment-infoCommentShareEXPERTRiku_Kobayashianswered 2 months ago"


Question: AWS slow upload on tiny files

"Hi there,I upload on a regular basis static websites on S3 buckets.I have a new website where this upload is very very very slow: it can take up to 1 hour to upload a ~30MB bundle of 1500 to 2000 files, mainly .js (i.e. no large image or video).The upload first indicates a correct speed and ~2min remaining, then it just slows and seems very lost in the estimated remaining time :)It does not happen on other projects, the bucket in the same region (eu-west-3), I activated the Transfer acceleration without any substantial difference and I tried from 3 different networks.Any idea ?!ThanksFollowComment"

Answer: "0Hello,This could be a Client config, see "max_concurrent_requests" in https://docs.aws.amazon.com/cli/latest/topic/s3-config.html. The SDK will have similar configuration options, like https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3.html.Hope this helped!//CarlCommentShareCarl Ostrandanswered a year agoVincent a year agoHi Carl, thanks for your answer.I am actually not using any CLI or sdk, I just move the files from my computer into the bucket through the browser (Chrome). I haven’t tried using another brother tho, I’ll edit this post if this makes a difference !Share"


Question: "Forbid IAM user to assume role with every tag, except defined ones"

"Hi,I want to forbid IAM user (for example: support-IAM ) to authenticate to the EC2 instance user ( for example : root-EC2 ), but still be able to authenticate as other EC2 instance user ( for example: support-EC2 ) , for that, as i understood i need to forbid this user to set --tags with value root-EC2 (or in my case, everything except support-EC2) while using command aws sts assume-role, but how can i do it?P.S im using AWS session managerJoannFollowCommentAWS-User-5530564 a year ago$PSStyle.OutputRendering = 'PlainText'Share"

Answer: "0Accepted AnswerOkay, i suppose i found the answer myself, i probably need to add something like            "Condition": {                "StringEquals": {                    "iam:ResourceTag/SSMSessionRunAs": "${aws:username}"                }            }to the "Trust relationship" in the Role. Then each user will need to provide exact the same Tag while assuming the role as their IAM username is, and by using this separation i can separate users in EC2 instance.CommentShareJoann Babakanswered a year ago"


Question: AWS Lambda 403 and weird errors everywhere

"I'm unable to use AWS Lambda and to find a reasonable explanation.Dashboard shows error box with empty messageCreate form shows a spinner for 10 seconds and then stops and nothing happensAPI returns {"message": null}The user has enough permissions to use LambdaI can just ask if someone faced something similar, since support doesn't want to answer my ticket.ThanksFollowCommentalatech EXPERT4 months agoIs your lambda code shareable here? Otherwise can you confirm the code returns a correct message? Does it happen even with a simple hello world?SharerePost-User-2776693 4 months agoThere's no code, It happens just by accessing "AWS Lambda > Functions", and also when creating a Function from scratch or just using the "Hello World" blueprint example.Also, tried creating a brand new user (non-root) with all the permissions and still the same issue.Sharealatech EXPERT4 months agoThe user you are logged in with, what IAM permissions has associated? Does this happen even with an user with full admin access? Have you tried to other regions? If so, the only reason left is that either disturbances in the region your account is created or something aws support must look into.Share"

Answer: "0Accepted AnswerThanks for all the answers.Luckily this issue has now been resolved. Turns out my account was compromised (maybe me logging in from another country) and contacting support (though a hidden chat) unlocked the resources and solved the issue.Thanks.CommentSharerePost-User-2776693answered 4 months agojonsully1dev 3 months agoAny chance you could point me in the direction of that hidden chat?? I'm having the exact same problem, both error in the browser and deploying from a serverless.js app 🙃Share-1Based on the error the user does not have the right permissions. Try providing the user/role either of the following 2 AWS Managed policies based on your requirement.The AWSLambda_ReadOnlyAccess policy grants read-only access to Lambda, Lambda console features, and other related AWS services. This policy was created by scoping down the previous policy AWSLambdaReadOnlyAccess.The AWSLambda_FullAccess policy grants full access to Lambda, Lambda console features, and other related AWS services. This policy was created by scoping down the previous policy AWSLambdaFullAccess.https://docs.aws.amazon.com/lambda/latest/dg/security_iam_troubleshoot.htmlCommentShareDebasis Mitra AWSanswered 4 months ago"


Question: Can I access RDS and elasticache redis through AWS fargate task role?

"I tried to access the aws service rds and elasticache redis through fargate's task role.I connected the full access of the service to the task role, but it was not connected, so I allowed the subnet with the security group setting, and the connection was possible.Is it possible to access only after setting the original role and security group?Or did I not set the role properly?FollowComment"

Answer: "0Hi AWS-User-3005727,If I understood your question correctly, it's if you can enable access to RDS from a Fargate task without modifying the RDS Security Group?Supposing that's your quesiton, the answer would be NO, since RDS is always protected by a Security Group, there is no way any Fargate task role, by itself, can bypass that security group.What you could do (but I wouldn't recommend it from a Security Standpoint) is in the RDS Security group add a rule to Allow all Inbound from your Private Subnet's CIDR, that way by default any instance that lives in the subnet will be able to access the database.I hope this answers your question, if it does please mark it as accepted answer, otherwise please create a new question with a more detailed question (hopefully with a diagram) to see how to best help you.May you have happy holidays!CommentSharePablo Guzmananswered 6 months ago"


Question: Autoscaling with python

"Due to security requirements autoscaling scale out and scale in is notallowed.  the requirement is that the instances that are added to the targetgroup and then removed cannot be terminated.  The plan is to create EC2instances and stop the instances.  when an instance needs to be added to thetarget group due to metric being exceeded (CPU over 60%) then the existinginstances is started and then added to the target group.  then when themetric falls below the CPU metric then the instance is removed from thetarget group and then stopped.I have created python  script that will start the stopped instance and addedthe instance to the target group when the metric is exceeded. I have apython script that will remove the instances, decrement the desiredcapacity, and shutdown the instance when the metric falls below the setmetric.where i am lost is figuring out how to trigger these python scripts to scaleout and scale in.  I am looking through cloudwatch and don't see anything tohelp me.  the autoscaling group does not have seem to have the setting tojust point to a lambda function when scaling out and scaling in.is this something that just is not doable?FollowComment"

Answer: "1If the only requirement is to retain the instances on scale in. You can leverage instance termination protection in auto scaling. This would allow you to take adavantage of all the features of auto scaling, and still retain the instances. https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-termination.htmlIf you absolutely cannot use auto scaling, you can leverage EventBridge to invoke a Lambda to run your python script that will interact with EC2. https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-eventbridge/CommentShareEXPERTRob_Hanswered a year agoSteven_Normoyle a year agothank you very much.  i will try your suggestionShare0In CloudWatch, create an Alarm on the Auto Scaling Group metric such as CPUUtilization. For the Notification, create or select an SNS topic. Then subscribe your Lambda function to this topic.CommentShareEXPERTkentradanswered a year agoSteven_Normoyle a year agothank you very much  i will try your suggestionShare"


Question: Amazon Workspace slow performance

The Amazon workspace getting slow on a LAN connection and at the same time its working fine on a WiFi connection. What is the issue? Connection Health Check in that region was ACCEPTABLE 160ms.FollowComment

Answer: 0I have checked the same thing from different locations and from different networks but the issue was the same. I have changed the client version from 64 bits to 32 bits and now it's working fine. It's a bit strange but it's working.CommentShareSandipkumar Patelanswered 4 months ago


Question: SageMaker "Uploading" phase grew from 2min -> 3h with no changes to artifact size

"Hi!As of a few days ago, the "Uploading" phase of my SageMaker training jobs jumped from 2 minutes to 3+ hours. The size of my artifacts did not change, but I did enable check-pointing (although this shouldn't affect the zipping and S3 upload of a different directory taking place).Is there any way to see what SageMaker is doing during that time? I have set sagemaker_container_log_level = 10 (debug), but no additional logs are published. (and I assume that anything after Training will not be logged.FollowCommentyann_stoneman 4 months agoHmm... have you also checked the training job's metrics in CloudWatch? If so, maybe you could add any interesting findings about that to your question https://docs.aws.amazon.com/sagemaker/latest/dg/training-metrics.htmlShare"


Question: Troubleshoting HTTPS on Elastic Beanstalk

"I used to have HTTPS working on my Beanstalk environment but in the attempt of allowing access to another service I messed up something and I can't get it working again. ATM my configuration is:Elastic BeanstalkAn active listener on port 443 with the appropriate certificateA process on port 80, Health check path: /EC2 and Security groups:One security group allowing inbound from 443, 80 and 27017Outbound all ports all IPsWhat I think I messed up:How to determine the correct origin for the security groups?Also, is there anything wrong on this setup? I read many guides, I am not going to terminate the https connections like explained in this guidehttps://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-https-configuration/simply because I haven't done it before and it was working. Do I really need it?thanksMFollowComment"

Answer: "0Yes I opened port 80/443/27017 on a security group that I attached to the instances, not sure where else they need to be opened?Do I need a security group for the load balancer and a separate one for the instances?Can you clarify this point please?CommentSharerePost-User-3116220answered 4 months agoEvolate-Tim 4 months agoYes the load balancer has it's own security group as well as the instances.Share0So you have opened up access in the security group for your load balancer on port 80/443/27017, have you also made sure that this security group has been added to the security group on the instances running your beanstalk application?CommentShareEvolate-Timanswered 4 months ago"