Question: S3 Access point Internet.

"Hi, i have created a S3 bucket so i can perform firmware  OTA update.To get the file - object must be,  in someway public inside the bucket.I want to keep the S3 bucket private and if its possible to use an access point to get the new update object / file.So i have created a user / access point / bucket with the following permissions.User Policy  - UpdateUserOTA:{"Version": "2012-10-17","Statement": [{"Sid": "VisualEditor0","Effect": "Allow","Action": "s3:","Resource": ["arn:aws:s3:eu-central-1:***:accesspoint/test-ap","arn:aws:s3:eu-central-1:*****:accesspoint/test-ap/"]}]}Inside the AccessPoint i have select No  :  Block all public access.Access Point Policy - test-ap :{"Version": "2012-10-17","Statement": [{"Sid": "APGet","Effect": "Allow","Principal": {"AWS": "arn:aws:iam:::user/UpdateUserOTA"},"Action": "s3:GetObject","Resource": "arn:aws:s3:eu-central-1::accesspoint/test-ap/object/UpdateUserOTA/*"}]}Bucket Policy  - updatebucket123:{"Version": "2012-10-17","Statement": [{"Sid": "Statement1","Effect": "Allow","Principal": {"AWS": "arn:aws:iam:::user/UpdateUserOTA"},"Action": "s3:GetObject","Resource": "arn:aws:s3:::updatebucket123/UpdateUserOTA/","Condition": {"StringEquals": {"s3:DataAccessPointAccount": ""}}}]}In the end i want to have a  :            url : https://test-ap-.s3-accesspoint.eu-central-1.amazonaws.com/o/s*.json,so i can download my update file from a device, that is connected to internet.What  am i doing wrong and i can't get access for my file through the browser for example?Thanks.FollowComment"

Answer: "0Hello, are you familiar with S3 presigned URLs? That is a way to generate a temporary URL that allows users with that URL to download an object. It is a good option for your use case. You create the URL with an expiration date and time then share it with the users who need it. For more information, see the documentation hereCommentShareGrantAWSanswered 4 months agoAWS-User-9603158 4 months agoHi, thanks.Yes i am familiar with them. I have tried them and they work fine.The problem is that they are very long up to 1Kbyte or more.My communication method is  a sim card, so i will consume a lot of bytes.I would like to know if the method described above is possible to work.The hole point is to keep the bucket private.Share0If you don't need any authentication and just want to serve up the files from the S3 bucket without making the bucket or objects public, use a CloudFront distribution in front of the bucket. See: Amazon S3 + Amazon CloudFront: A Match Made in the Cloud.CommentShareEXPERTkentradanswered 4 months agoAWS-User-9603158 3 months agoThanks, I will try it.Share0Dear Mr. Kentrad,I have created the CloudFront distribution in front of the bucket and i can access the files from my board.Now i would like more security.So i have created following this example :  How to configure Cloudfront Presign URLshttps://www.youtube.com/watch?v=NTOCzsn7b4A, a protected folder with the key.During the HTTPS connection i send the the url of the CloudFront bucket, the CA Root certificate to get the object in the non protected folder.My question is how can i access the file  without a Presigned URL inside  the protected folder?I have the following structure from ESP32:/**@brief HTTP configuration*/typedef struct {const char                  url;                /!< HTTP URL, the information on the URL is most important, it overrides the other fields below, if any */const char                  host;               /!< Domain or IP as string /int                         port;                /!< Port to connect, default depend on esp_http_client_transport_t (80 or 443) */const char                  username;           /!< Using for Http authentication */const char                  password;           /!< Using for Http authentication /esp_http_client_auth_type_t auth_type;           /!< Http authentication type, see esp_http_client_auth_type_t */const char                  path;               /!< HTTP Path, if not set, default is / */const char                  query;              /!< HTTP query */const char                  cert_pem;           /!< SSL server certification, PEM format as string, if the client requires to verify server /size_t                      cert_len;            /!< Length of the buffer pointed to by cert_pem. May be 0 for null-terminated pem /const char                  client_cert_pem;    /!< SSL client certification, PEM format as string, if the server requires to verify client /size_t                      client_cert_len;     /!< Length of the buffer pointed to by client_cert_pem. May be 0 for null-terminated pem /const char                  client_key_pem;     /!< SSL client key, PEM format as string, if the server requires to verify client /size_t                      client_key_len;      /!< Length of the buffer pointed to by client_key_pem. May be 0 for null-terminated pem /const char                  client_key_password;      /!< Client key decryption password string /size_t                      client_key_password_len;   /!< String length of the password pointed to by client_key_password /const char                  user_agent;         /!< The User Agent string to send with HTTP requests /esp_http_client_method_t    method;                   /!< HTTP Method /int                         timeout_ms;               /!< Network timeout in milliseconds /bool                        disable_auto_redirect;    /!< Disable HTTP automatic redirects /int                         max_redirection_count;    /!< Max number of redirections on receiving HTTP redirect status code, using default value if zero/int                         max_authorization_retries;    /!< Max connection retries on receiving HTTP unauthorized status code, using default value if zero. Disables authorization retry if -1/http_event_handle_cb        event_handler;             /!< HTTP Event Handle /esp_http_client_transport_t transport_type;           /!< HTTP transport type, see esp_http_client_transport_t /int                         buffer_size;              /!< HTTP receive buffer size /int                         buffer_size_tx;           /!< HTTP transmit buffer size */void                        user_data;               /!< HTTP user_data context /bool                        is_async;                 /!< Set asynchronous mode, only supported with HTTPS for now /bool                        use_global_ca_store;      /!< Use a global ca_store for all the connections in which this bool is set. /bool                        skip_cert_common_name_check;    /!< Skip any validation of server certificate CN field */esp_err_t (*crt_bundle_attach)(void conf);      /!< Function pointer to esp_crt_bundle_attach. Enables the use of certificationbundle for server verification, must be enabled in menuconfig /bool                        keep_alive_enable;   /!< Enable keep-alive timeout /int                         keep_alive_idle;     /!< Keep-alive idle time. Default is 5 (second) /int                         keep_alive_interval; /!< Keep-alive interval time. Default is 5 (second) /int                         keep_alive_count;    /!< Keep-alive packet retry send count. Default is 3 counts */struct ifreq                if_name;            /!< The name of interface for data to go through. Use the default interface without setting */} esp_http_client_config_t;In the not protected folder i use :const char                  *url;const char                  *cert_pem;Thanks.CommentShareAWS-User-9603158answered 3 months ago"


Question: [solution] how to solve problem "ubuntu has been blocked by the current security policy "

"https://repost.aws/questions/QU9PJl7BvYTE6fEvKsa-GouQ#ANJDSwe-nfT5e--OEfw6tIQwThe link above is my previous question.I finally solved this problem.Those who experienced the same problem as me would have tried to initialize by following the link below.https://docs.aws.amazon.com/deepracer/latest/developerguide/deepracer-ubuntu-update-preparation.htmlFor those who have experienced similar problems, aws told me that the "shell script" file will run automatically if I run the link above.When booted into the usb iso file, the "shell script" actually worked, but it did not initialize.To solve this problem, only the contents of the factory reset file must be included in the Flask usb drive.The link below contains what I explained.https://www.youtube.com/watch?v=f_GD7RcMsR0In aws, in order to avoid this problem, it would be better to correct that part in the link below and add related photos.https://docs.aws.amazon.com/deepracer/latest/developerguide/deepracer-ubuntu-update-preparation.htmlFollowComment"

Answer: "0Thanks for the follow up and glad you got it sorted, I'll review the documentation and see if there's anything we can change to make this clearer when creating the USB boot disk about which files go where.CommentShareDavidFSmithanswered a year ago"


Question: Tool for Data Dictionary (similar to SQL Doc)

"A customer is building a data lake solution, and will leverage RedShift. He wants a tool for "Data Dictionary" (to show relations between data, columns, and catalogue the data in a "dictionary"), and mentioned SQL Doc does this. Do we offer anything similar?FollowComment"

Answer: "0Accepted AnswerHere are 23 data dictionary tools, along with their supports and notable featuresData Dictionary ToolsThe customer data lake would be in S3, 2) the catalog in Glue, 3) they have to create a Redshift external table as mentioned External table redshift spectrum and then query the data from S3 using Redshift. It is also an effective way to join datawarehouse with the data lake. Once the data lake metadata is present in Redshift the customer can use any of the supported data-dictionary-tools.CommentShareKunal_Ganswered 5 years ago"


Question: "Configure public key authentication for new user, when password authentication is disabled"

"In an EC2 instance, with Amazon Linux AMI, "PasswordAuthentication" is disabled in "sshd_config" file, and "PubkeyAuthentication" is enabled, by default.Incase "Linux System admin user" creates new Linux user on the EC2 instance, how the new Linux user ssh to the system? Because password authentication is already disabled.And if the new Linux user generates public key in his local machine, and try to place it, in the EC2 instance, he could not place. Because currently both Public key authentication and password authentication will not work.How to overcome this scenario?FollowComment"

Answer: "0When Linux system admin create a new user, ask them to create a SSH key for new user and share the private keys with respective user. Then the private key can be used to login to the instance as new user.CommentShareSijoJose-AWSanswered a year ago"


Question: To manage bucket encryption using condition in CloudFormation template

"Hi AWS, I am trying to impose a condition on S3 BucketEncryption property whether it should be customer managed (SSE-KMS) or AWS managed key (SSE-S3). The code for the template is:# version: 1.0AWSTemplateFormatVersion: "2010-09-09"Description: Create standardized S3 bucket using CloudFormation TemplateParameters:  BucketName:    Type: String    Description: "Name of the S3 bucket"  KMSKeyArn:    Type: String    Description: "KMS Key Arn to encrypt S3 bucket"    Default: ""  SSEAlgorithm:    Type: String    Description: "Encryption algorithm for KMS"    AllowedValues:      - aws:kms      - AES256Conditions:  KMSKeysProvided: !Not [!Equals [!Ref KMSKeyArn, ""]]Resources:  S3Bucket:    Type: 'AWS::S3::Bucket'    DeletionPolicy: Retain    UpdateReplacePolicy: Retain    Properties:      BucketName: !Ref BucketName      PublicAccessBlockConfiguration:        BlockPublicAcls: true        BlockPublicPolicy: true        IgnorePublicAcls: true        RestrictPublicBuckets: true      BucketEncryption:         ServerSideEncryptionConfiguration:         - !If          - KMSKeysProvided          - ServerSideEncryptionByDefault:              SSEAlgorithm: !Ref SSEAlgorithm              KMSMasterKeyID: !Ref KMSKeyArn            BucketKeyEnabled: true          - !Ref "AWS::NoValue"When I am selecting the SSEAlgorithm as AES256 I am receiving this error Property ServerSideEncryptionConfiguration cannot be empty. I know KMSMasterKeyID should not be present when the SSEAlgorithm is of AES256 type but I am confused how to get rid of this error.Please help.FollowComment"

Answer: "1Accepted AnswerHello,I hope you're doing well!You are right. KMSMasterKeyID should not be present when the SSEAlgorithm is of AES256 type. So, you should check SSEAlgorithm type. Not KMSKeysProvided. Please check the updated following example.# version: 1.0AWSTemplateFormatVersion: "2010-09-09"Description: Create standardized S3 bucket using CloudFormation TemplateParameters:  BucketName:    Type: String    Description: "Name of the S3 bucket"  KMSKeyArn:    Type: String    Description: "KMS Key Arn to encrypt S3 bucket"    Default: ""  SSEAlgorithm:    Type: String    Description: "Encryption algorithm for KMS"    AllowedValues:      - aws:kms      - AES256Conditions:  KMSKeysProvided: !Not [!Equals [!Ref KMSKeyArn, ""]]Conditions:  AES256: !Equals [!Ref SSEAlgorithm, "AES256"]Resources:  S3Bucket:    Type: 'AWS::S3::Bucket'    DeletionPolicy: Retain    UpdateReplacePolicy: Retain    Properties:      BucketName: !Ref BucketName      PublicAccessBlockConfiguration:        BlockPublicAcls: true        BlockPublicPolicy: true        IgnorePublicAcls: true        RestrictPublicBuckets: true      BucketEncryption:         ServerSideEncryptionConfiguration:         - !If          - AES256          - ServerSideEncryptionByDefault:              SSEAlgorithm: !Ref SSEAlgorithm            BucketKeyEnabled: true          - ServerSideEncryptionByDefault:              SSEAlgorithm: !Ref SSEAlgorithm              KMSMasterKeyID: !Ref KMSKeyArn            BucketKeyEnabled: trueI hope this to be helpful.CommentShareDaein_Canswered 2 months ago0The working code:# version: 1.0AWSTemplateFormatVersion: "2010-09-09"Description: Create standardized S3 bucket using CloudFormation TemplateParameters:  BucketName:    Type: String    Description: "Name of the S3 bucket"  KMSKeyArn:    Type: String    Description: "KMS Key Arn to encrypt S3 bucket"    Default: ""  SSEAlgorithm:    Type: String    Description: "Encryption algorithm for KMS"    AllowedValues:      - aws:kms      - AES256Conditions:  AES256: !Equals [!Ref SSEAlgorithm, "AES256"]Resources:  S3Bucket:    Type: 'AWS::S3::Bucket'    DeletionPolicy: Retain    UpdateReplacePolicy: Retain    Properties:      BucketName: !Ref BucketName      PublicAccessBlockConfiguration:        BlockPublicAcls: true        BlockPublicPolicy: true        IgnorePublicAcls: true        RestrictPublicBuckets: true      BucketEncryption:         ServerSideEncryptionConfiguration:         - !If          - AES256          - ServerSideEncryptionByDefault:              SSEAlgorithm: !Ref SSEAlgorithm            BucketKeyEnabled: true          - ServerSideEncryptionByDefault:              SSEAlgorithm: !Ref SSEAlgorithm              KMSMasterKeyID: !Ref KMSKeyArn            BucketKeyEnabled: trueCommentShareArjun Goelanswered 2 months agoEXPERTalatechreviewed 2 months ago"


Question: Application properties

"Hello,I have a question regarding application properties.Successfully set up a CodePipeline that pulls content of a repository from github, builds the java app with CodeBuild and deploys it on Elastic Beanstalk.Would like to modify the build and include an extra step to replace the application.properties file before creating the jar.Would like to specify this step in the buildspec.yml that currently looks like this:pre_build:commands:- echo "Copying application.properties to CodeBuild..."- cp -f /root/xyz-api/config/application.properties ./src/main/resources/build:commands:- mvn testpost_build:commands:- mvn packageUnfortunately the pre_build phase fails. Where can we store the application.properties file to be accessible by CodeBuild (so the pre_build would swap the file before running maven test and package?) We are planning to have a separate application.properties file for dev and prod environments.Bw,IanFollowComment"

Answer: "0How about placing them in S3?Then you can use "aws s3 cp" to copy the files.For example, you might execute the following command.aws s3 cp s3://example-s3/application.properties ./src/main/resources/If this answer leads to a resolution, please approve the answer for the betterment of the community.CommentShareEXPERTRiku_Kobayashianswered 2 months ago"


Question: Using IAIK PKCS#11 Wrapper with AWS CloudHSM

"Hi,I'm trying to issue RSA key pairs on AWS CloudHSM with a JAVA application using IAIK PKCS#11 Wrapper and JVM JCE Provider and I'm facing some difficulties. Indeed, for now, the only way to generate a keypair is providing minimal templates for both public and private key:RSAPublicKey rsaPublicKeyTemplate = new RSAPublicKey();RSAPrivateKey rsaPrivateKeyTemplate = new RSAPrivateKey();// ModulusrsaPublicKeyTemplate.getModulusBits().setLongValue(new Long(keyLength));// Public expbyte[] publicExponentBytes = {0x01, 0x00, 0x01};rsaPublicKeyTemplate.getPublicExponent().setByteArrayValue(publicExponentBytes);All other tries I made for customizing fail with the PKCS#11 error CKR_TEMPLATE_INCONSISTENT, for example when I try to make the generated key pair persistent:rsaPublicKeyTemplate.getToken().setBooleanValue(Boolean.TRUE);rsaPrivateKeyTemplate.getToken().setBooleanValue(Boolean.TRUE);It seems that the HSM does not understand the templates I provide...Has someone here even succeeded in using IAIK Provider or is AWS CloudHSM not compatible with this provider?Then, are we forced to use AWS JCE Provider?Thanks in advance for taking time to read my questionsEricEdited by: EricT on Jan 11, 2021 3:23 AMEdited by: EricT on Jan 11, 2021 6:51 AMFollowComment"

Answer: "0Hi everyone,For all that are here coming from Google or anything else, the error CKR_TEMPLATE_INCONSISTENT can be easily solved by carefully looking at supported attributes :)In my case, it was because I tried to build a RSA/ECDSA template with a subject and start/end dates attributes which are not part of the list: https://docs.aws.amazon.com/cloudhsm/latest/userguide/pkcs11-attributes.htmlÉricCommentShareEricTanswered 2 years ago0To avoid CKR_TEMPLATE_INCOSISTENT error, build RSA/ECDSA templates only with supported attributesCommentShareEricTanswered 2 years ago"


Question: Error deploying Django application that serves static files from an S3 Bucket

"So the deployment process was going smoothly and well for a long time, with no errors, no failures, no nothing.However, suddenly the following error starts to appear:botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the DeleteObject operation: Access Denied  Error while running '$ python manage.py collectstatic --noinput'.I changed literally nothing to my AWS S3 configuration, and never experienced such a problem, any help??FollowComment"

Answer: "0The error points to be not having access to delete some thing from the s3 bucket. There can be two things:-The application is trying to delete some object from the bucket, but it cant because there is no permission to do so, you can update the role policy and add permissions.The policy was changed and delete permission was removed, which will again require adding back the permission.CommentShareRishianswered 6 months ago"


Question: DNS Error: SERVFAIL

"I have a domain with rout53 configured to a Gmail, mail server using MXThe setup has been working for a year but just recently am unable to send and receive emails, sent emails bounce back with an error: DNS Error: DNS type 'mx' lookup of getaidpharma.com responded with code SERVFAILHow do I resolve this?FollowCommentAWS-User-0357409 2 months agoCould you possibly share how you have your records setup in Route53.  Redact any information if need to!Share"

Answer: "1Accepted AnswerHi, it looks like NS records for the domain are missing. I would advise checking your DNS registrar configuration and verifying that Route53 name servers associated with your DNS zone are set for your domain. You can get those by following this documentation - https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/GetInfoAboutHostedZone.htmlCommentShareJaroslav-AWSanswered 2 months ago0Currently, NS records are available, I have not made changes since its setup about a year ago, and its been working until about a week ago.Attached is the NS recordsCommentSharerePost-User-5122answered 2 months agoJaroslav-AWS 2 months agoYou have to check at the DNS registrar, where domain name was purchased from, not DNS zone settings.Share0Thanks for the information; I found out the NS at the registrar has changed, but I am not sure why and when this happened. However, I have updated the NS at the hosted zone and verified if it's working via mxtoolbox.com, but I still get a failed response.CommentSharerePost-User-5122answered 2 months agoJaroslav-AWS 2 months agoYou need to give it some time to replicate across the Internet. How long depends on TTL settings and caching. I can now see proper NS and MX records.Share"


Question: Access denied error while creating Data Filter in Lake Formation

"Hi,I am signed in with a userId which has IAM Administrator access and DataLake Administrator access. The userId also has permissions to table columns set up in Lake Formation. However when creating a Data Filter on a table using console, I get following error message.Error creating data cell filter: data-filter-1. Not authorized to create filter.Any suggestions to resolve the issue will be greatly appreciated.FollowCommentKeshav_B a year agoError is encountered in Singapore region, tested in N.Virginia and works ok in that region.Share"

Answer: "0The governed tables, transaction support, row-level security, and storage optimizations features for Lake Formation are available in the following AWS Regions.US East (N. Virginia) (us-east-1)US East (Ohio) (us-east-2)US West (Oregon) (us-west-2)Europe (Ireland) (eu-west-1)Asia Pacific (Tokyo) (ap-northeast-1)Data filters are not available in Singapore region yet and that's the reason you're seeing the error.CommentShareSamuelanswered a year agoEXPERTFabrizio@AWSreviewed a year ago"


Question: "FIDO SSH support (ed25519-sk, ecdsa-sk)"

"I have a ed25519-sk ssh key that I would like to add as an SSH key for AWS CodeCommit. Pasting the key into the upload form yields the error message Invalid public key uploaded. I assume this is because it is a format that's only existed for a few years. OpenSSH has supported FIDO ssh keys since https://www.openssh.com/txt/release-8.2. Verbose logging of an ssh connection to git-codecommit.us-east-1.amazonaws.com:22 shows it as an acceptable host key algorithm:host key algorithms: rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.comCould CodeCommit be updated to accept this key type? Or does it already and I'm doing something wrong?FollowComment"


Question: Redshift Serverless - Create workspace fails with no free IP address available

I have created a VPC as follows with 4091 free IP addressesAnd then trying to create a redshift serverless workspace:And getting this error:FollowComment

Answer: "2Accepted AnswerThat appears to be an unclear error message since clearly you have enough IPs available, but it also says "Each subnet should be in a different Availability Zone" From the screenshot it appears you are using eu-central-1, which has 3 AZs, but you have 4 subnets selected. Based on the information provided, it is likely that you can avoid the error by only selecting one subnet per AZ.CommentShareEXPERTThaddeus_Wanswered a month ago0yes, one need to create 3 AZ/subnet. The error message is not right. Thanks.CommentSharerePost-User-7270290answered a month ago"


Question: HTTP API PROXY INTEGRATION_NETWORK_FAILURE Error

"Hi,Here is my setup:my.domain.com -> API GW Custom Domain -> HTTPAPI -> Route (/api/{+proxy}) -> VPCLink -> ALB -> HTTPS Listener -> TargetGroup (Type: Instance) -> ECS Fargate ServiceHTTPAPI integration has the following parameter mapping: path -> overwrite -> /$request.path.proxy (I want to get rid of "api" part in the url)when I make below request I got 503 errors time to time:https://my.domain.com/api/otherPathI have enabled access logs on HTTPAPI but they show very limited information. ALB logs are sent to S3 bucket so it is almost impossible to track request.Why do I get 503 errors ?Access log from HTTP API:{    "requestId": "req_id=",    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",    "sourceIp": "**.**.**.**",    "requestTime": "09/Nov/2022:18:07:24 +0000",    "requestTimeEpoch": "1668017244",    "httpMethod": "GET",    "path": "/otherPath",    "status": "503",    "protocol": "HTTP/1.1",    "responseLength": "33",    "domainName": "my.domain.com",    "basePathMatched": "(none)",    "error_Message": "Service Unavailable",    "error_responseType": "INTEGRATION_NETWORK_FAILURE",    "integrationErrorMessage": "-",    "integration_Error": "-",    "integrationStatus": "200",    "integration_Status": "-",    "integration_IntegrationStatus": "200",    "integrationLatency": "9002"}Please help,ThanksFollowCommentgzub a month agoI have the same problem, with intermittent 503s and an integration latency of 9000 and a similar setup...  Did you get a resolution?Share"


Question: AWS Personalize "Input is not a valid Avro schema"

"Hello,I try to build a CDK to setup an AWS Personalize environment. For me it look good, the Avro schema was copied from a tutorial of AWS. What is wrong with my Schema?//AWS PErsonalize Init    const cfnDatasetGroup = new personalize.CfnDatasetGroup(this, 'MyCfnDatasetGroupLAb62s', {      name: 'pinpoint-recommendations-clothing',                   });        const cfnSchema = new personalize.CfnSchema(this, 'MyCfnSchema', {      name: 'pinpoint-recc-schema-clothing',      schema: this.getPErsonalizeCsvSchemaJSON.toString(),               });    const cfnDataset = new personalize.CfnDataset(this, 'MyCfnDataset', {      datasetGroupArn: cfnDatasetGroup.attrDatasetGroupArn,      datasetType: 'datasetType',      name: 'pinpoint-recc-dataset-clothing',      schemaArn: cfnSchema.attrSchemaArn,          datasetImportJob: {                        dataSource: {dataLocation: personalizeBucket.s3UrlForObject.toString()},        jobName: 'pinpoint-importjob-clothing',        //roleArn: 'roleArn',      },    });The method behind is:private getPErsonalizeCsvSchemaJSON() {        let content = {      "type": "record",      "name": "Interactions",      "namespace": "com.amazonaws.personalize.schema",      "fields": [          {              "name": "USER_ID",              "type": "string"          },          {              "name": "ITEM_ID",              "type": "string"          },          {              "name": "TIMESTAMP",              "type": "long"          }      ],      "version": "1.0"    }  return content  }And the error message is : "Input is not a valid Avro schema (Service: Personalize, Status Code: 400FollowComment"

Answer: "0To get a string representation of a map object in TypeScript/JavaScript, you need to use JSON.stringify(this.getPErsonalizeCsvSchemaJSON).this.getPErsonalizeCsvSchemaJSON.toString() returns [object Object]CommentShareEXPERTMassimilianoAWSanswered 5 months agoEXPERTChris_Greviewed 5 months ago"


Question: email abuse report

"Hello!I am using AWS for the first time ever and got this email. I have an EC2 instance and using Amazon SES for transactional emails for my website. Not sure what exactly I need to do. Any help/direction would be much appreciated. I searched on re:Post but couldn't find a similar post.We've received a report(s) that your AWS resource(s)AWS ID: XXXX XXXXXX     Region: us-west-2    EC2 Instance Id: XXXXXXXXXXXXAWS ID: XXXX XXXXXX     Region: us-west-2    Network Interface Id: XXXXXXXXXXXXXXXXXXhas been implicated in activity which resembles attempts to access remote hosts on the internet without authorization.If you're unaware of this activity, it's possible that your environment has been compromised by an external attacker, or a vulnerability is allowing your machine to be used in a way that it was not intended.Log Extract:<<<This is an email abuse report about the IP address XX.XX.XX.XX generated at Mar 10 19:32:53You get this email because you are listed as the official abuse contact for this IP address.**The following intrusion attempts were detected:Mar 10 19:32:53 arwen fail2ban.filter[4731]: INFO [proftpd] Found XX.XX.XX.XX - 2023-03-10 19:32:53Mar 10 19:32:54 arwen fail2ban.filter[4731]: INFO [proftpd] Found XX.XX.XX.XX - 2023-03-10 19:32:54Mar 10 19:32:55 arwen fail2ban.filter[4731]: INFO [proftpd] Found XX.XX.XX.XX - 2023-03-10 19:32:55Mar 10 19:32:55 arwen fail2ban.actions[4731]: NOTICE [proftpd] Ban XX.XX.XX.XX**FollowComment"

Answer: "1(Have you opened an AWS Abuse case?)It appears that your EC2 instance has been flagged for potentially attempting to access remote hosts on the internet without authorization. This could be due to unauthorized access by an external attacker or a vulnerability that is allowing your machine to be used in an unintended way.The log extract you received shows that there have been multiple intrusion attempts detected from the IP address XX.XX.XX.XX, which has triggered a fail2ban filter that has banned that IP address. This suggests that your instance may have been compromised, and it is important to investigate this further to ensure the security of your AWS environment.To address this issue, you should take the following steps:Review your EC2 instance logs: Start by reviewing the logs for your EC2 instance to identify any unusual or suspicious activity. This can help you determine if your instance has been compromised and what steps you need to take to address the issue. You can access the logs from the EC2 console or via SSH.Scan your instance for malware and vulnerabilities: Use a tool like AWS Inspector or a third-party vulnerability scanner to scan your instance for malware and vulnerabilities. This can help you identify any security risks that need to be addressed.Secure your instance: Once you have identified any vulnerabilities or malware on your instance, take steps to secure it. This may involve updating software and patching vulnerabilities, implementing access controls, and disabling unused services and ports.Contact Amazon SES support: If you are using Amazon SES for transactional emails, you should contact Amazon SES support to report the abuse complaint and investigate any potential unauthorized use of your account.Report the abuse complaint: Finally, you should report the abuse complaint to the appropriate authorities, including AWS abuse and your internet service provider (ISP), if applicable.By taking these steps, you can address the issue and ensure the security of your AWS environment.CommentShareAWS_Guyanswered 3 months ago"


Question: unlock Elastic IP address from my account

"Hello,Could you please unlock the EIP 54.77.254.244 from my account?"The address with allocation id eipalloc-XXXX cannot be released because it is locked to your account. Please contact AWS Support to unlock it"The error has been occurring as a root or any iam account. I've already contacted support but they redirected me to forum.Thank you!FollowComment"

Answer: "0I have been in the same situation (almost). I did this workaround: create a new EC2 free tier, attach the IP, stop the instance, detach the IP, release it, and then delete the EC2. In my case, always work in that wayCommentShareAWS-User-8785608answered a year agoAWS-User-4565300 a year agoIt didn't helped.SharerePost-User-9338007 8 months agoIt does not work. My Elastic ip is for VPC. Any way to contact support to fix their own bug?  BTW, I only have basic support plan.Share0I was in the same issue having requested a reverse DNS record before you could set them with the API. We also have a basic support plan so I couldn't open a regular support case to have the address unlocked.I requested to have the address unlocked at https://support.console.aws.amazon.com/support/contacts#/rdns-limits and they did it within 12 hours.CommentSharerePost-User-1082690answered a month ago"


Question: CDK Stck Failed to publish one or more assets Access Denied

"Hi All,In My BuildProject/BuildSpec (in my STG Account), I run this command :cdk deploy --require-approval neverit gives me this error :myStack: deploying...[0%] start: Publishing e988sdsf934da0d45effe675sdscb946f3e1sds68:current[0%] check: Check s3://cdk-hnb65dds-assets-xxxxxxxx-cregion/assets/e9882ab1236873df4sdfeffe67sdfc8ce13bsdff3e1d6sdf8d68.zipCall failed: listObjectsV2({"Bucket":"cdk-hnsd59fds-assets-xxxxxxxx-region","Prefix":"assets/e98ssdfsd87dsffsdffdsfcc8sdsdfdd6141fsdd68.zip","MaxKeys":1}) => Access Denied (code=AccessDenied)[33%] fail: Access Denied[33%] start: Publishing c24b999656e4fe6c609c31dfadffbcdfdfc2c86df:current[33%] check: Check s3://cdk-hnb659fds-assets-xxxxxxxx-cregion/assets/c24b999656e4fe6c609c31dfadffbcdfdfc2c86df.zipCall failed: listObjectsV2({"Bucket":"cdk-hnb659fds-assets-xxxxxxxx-cregion","Prefix":"assets/c24b999656e4fe6c609c31dfadffbcdfdfc2c86df.zip","MaxKeys":1}) => Access Denied (code=AccessDenied)[66%] fail: Access Denied[66%] start: Publishing werer56e4fe6c609c3ewrd17a4d9c3afwr6b8c2wer:current[66%] check: Check s3://cdk-hnb659fds-assets-xxxxxxxx-cregion/assets/werer56e4fe6c609c3ewrd17a4d9c3afwr6b8c2wer.zipCall failed: listObjectsV2({"Bucket":"cdk-hnb659fds-assets-xxxxxxxx-cregion","Prefix":"assets/werer56e4fe6c609c3ewrd17a4d9c3afwr6b8c2wer.zip","MaxKeys":1}) => Access Denied (code=AccessDenied)[100%] fail: Access Denied ❌  myStack failed: Error: Failed to publish one or more assets. See the error messages above for more information.    at publishAssets (/usr/local/lib/node_modules/aws-cdk/lib/util/asset-publishing.ts:27:11)How can I give CDK stack running from BuildSpec permission to publish assets?I already added this policy to my codeBuild service role, but still same issue :{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "s3:GetObject*",                "s3:PutObject",                "s3:PutObjectAcl",                "s3:getBucketLocation"            ],            "Resource": [                "arn:aws:s3:::cdk*"            ]        }    ]}also had this error :ser: arn:aws:sts::xxxxxx:assumed-role/codebuild-mybp-service-role/AWSCodeBuild-d1acsd11-4sad7-9sada6834ffsadbs is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:region:xxxxxxxx:function:myStack-CustomCDKBucketDeployment-l5dzxcszxA7assa because no identity-based policy allows the lambda:InvokeFunction action (Service: AWSLambda; Status Code: 403; Error Code: AccessDeniedException; Request ID: eedf2-03dfdf3-4ddsfd7-bfdg7-2dfsdff5c2dfgd0; Proxy: null)not sure which lamda  he wants to invoke here and why? what are the right permissions for thisThank you!!FollowComment"

Answer: "1Without further context around the run which failed on Lambda, it is difficult to speculate, however based on the naming, it looks like it may be part of the stack which you are trying to deploy, as it seems to match the style of CDK-generated names.As for the permissions, CDK normally creates a deployment role in your account as part of the bootstrapping process. I have included a link to the details of this role in the bootstrapping template (1). One potential option here is to specify this role in your CDK CLI call by passing the -r or --role-arn parameter with that role (2). If you are experiencing this issue in an account or region that has not been bootstrapped, then you may just need to bootstrap the account in order to have this deployment role (3)I hope this helps! If you need more assistance with this, I would encourage you to open a support case with CDK so that we can have a more hands-on look at the current configuration, and can provide more concise advice, securely and one-on-one.(1) https://github.com/aws/aws-cdk/blob/master/packages/aws-cdk/lib/api/bootstrap/bootstrap-template.yaml#L362-L465(2) https://docs.aws.amazon.com/cdk/v2/guide/cli.html#cli-ref : AWS CDK Toolkit (cdk command) - Toolkit reference(3) https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html : BootstrappingCommentShareSUPPORT ENGINEERWayne_Ganswered a year ago0Accepted AnswerI added to my Build project service  role the following missing permissions and it works :1 - S3:listObjects2- AWSLambdaRole to have the permission to lambda:InvokeFunctionCommentShareJessanswered a year ago"


Question: Impossible to stop a bugged EC2 instance

"Hello,I tried to stop my crashed instance using both command and interface and it does not workHere is what it shows :An error occurred (InternalError) when calling the StopInstances operation (reached max retries: 4): An internal error has occurredThanks and best regards,FollowComment"

Answer: "0Accepted AnswerCheck "AWS Health Dashboard". I have the same issue and I can red there:We are investigating an issue that impacts the availability of some EC2 instances in the eu-west-2 region. Your affected EC2 instances are listed in the “Affected resources” Tab.And:Some instances in a single Availability Zone are currently impaired or have lost power in the EU-WEST-2 Region. The root cause is a thermal event within a data center in the affected Availability Zone that we are working to resolve. Some instances may also be experiencing network connectivity issues in the affected Availability Zone. Elastic Load Balancing has shifted traffic away from the Availability Zone. All multi-AZ databases have failed away from the affected Availability Zone, however single-AZ database will remain affected until we see full recovery. We do not yet have a an ETA for full recovery but expect it to be more than an hour. We will provide further guidance as soon as we have it. For customers that are able to fail away from the affected Availability Zone, we recommend doing so.I dont know what to do, because I can even create image from the unreachable instance :(CommentShareaiacanswered a year ago0Is your instance on EU_WEST-2 (london)?CommentShareaiacanswered a year ago0Yes it is.CommentSharePascalanswered a year ago0OK thanks aiac, I think we just have to wait...CommentSharePascalanswered a year ago0Any updates? We are also experiencing this issue. Our affected EC2 server was terminated.CommentSharerePost-User-0364260answered a year ago0Also experiencing this exact problem on EU-West-2.Instance went offline during the EU-West-2 outage, and since then has not been able to stop, start or restart.  Have tried force stopping, it doesn't make any difference.Update: it is now online again.CommentShareAlanG-BSanswered a year ago"


Question: Tape Gateway - Understanding upload buffer

"Hi!We were testing Tape Gateway and it came to a moment that upload buffer disk filled and it took like 7 hours to get empty. In this 7 hours, no data was uploaded to AWS. The upload resumed when the disk got empty. It went from 100% usage to 0%, and then started uploading again.https://ibb.co/P6qnHH2https://ibb.co/F4QRSq4https://ibb.co/VgTcPhyhttps://ibb.co/dDgTV4VI would understand if the upload buffer disk stops receiving files to get uploaded the files it already has, but why would it stop uploading?Is there any explanation for it?Thanks you in advance!FollowComment"

Answer: "0Accepted AnswerHi,The Tape Gateway stops uploading to AWS during the "Pass Through" state.Here are the Tape Gateway state transitions that occur when your Upload Buffer is set too low and the Upload Buffer runs out of capacity.state: "Available" - normal tape gateway operationsstate: "Pass Through" - when the upload buffer runs out of capacity (normally because the Upload Buffer is set too low). During this phase NO data is uploaded to AWS and the data currently in the Upload Buffer is processed going from 100% to 0%. During the Pass Through status, the data maintained locally becomes out of sync with AWS stored data.state: "Bootstrapping" - the volume re-establishes synchronization with in AWS, so that it can resume the record (log) of changes to the volume, and re-enable CreateSnapshot functionality. During Bootstrapping, writes to the volume are recorded in upload buffer.state: "Available" - normal tape gateway operationsMore Info here:Link: https://docs.aws.amazon.com/storagegateway/latest/userguide/ManagingLocalStorage-common.htmlHope this helps.-randyCommentShareRandyTakeshitaanswered 4 years ago"


Question: Share a Gateway cert between gateways?

"Is it possible to have a single, pre-defined gateway certificate to share between gateways?  I understand this is not ideal from a security setup, but it would make provisioning new gateways onto IoT Wireless much easier.When creating a gateway in the web console the only option provided is to create a new gateway certificate, but I was hoping it would be possible to define using the CLI or API and pre-load it onto a gateway.Also, a second question - are the server trust certs the same for all gateways?While also in the web console provisioning a gateway, the CUPS & LNS cert download appears to not actually create new certs, which I assume means they are common across all gateways for my CUPS & LNS endpoints?FollowComment"

Answer: "1Accepted AnswerHi,Q1)you can use API https://docs.aws.amazon.com/iot-wireless/2020-11-22/apireference/API_AssociateWirelessGatewayWithCertificate.html or CLI https://docs.aws.amazon.com/cli/latest/reference/iotwireless/associate-wireless-gateway-with-certificate.html for that purpose. As you mentioned, it's not optimal from security standpoint.See this example for automation: https://github.com/aws-samples/aws-iot-core-lorawan/tree/main/automationQ2) Server trust certs are for authentication cloud endpoints for LNS and CUPS. As long as gateways connect to the same endpoint, the server trust certs are the same. Practically that means that you can use the same one for the same account and region combination.CommentShareAndrei Sanswered a year agoAWS-User-9404308 a year agoExcellent thank you.  I managed to also find the AWS CVM repo (https://github.com/awslabs/aws-iot-certificate-vending-machine) as well as a slightly simpler approach based on CVM here: https://github.com/t04glovern/serverless-cvmShare1As of now  https://docs.aws.amazon.com/iot-wireless/2020-11-22/apireference/API_AssociateWirelessGatewayWithCertificate.html will not allow cert to be shared.This is rule is there because we consider each Gateway as an individual client to IoTWireless service.CommentShareYujie Chenanswered 5 months ago"


Question: AWS Backups : EC2 and EBS ?

"AWS Backups : When adding an EC2 resource, do I have to add the corresponding EBS volume as well  ? Is it enough to check EC2 resource only ?FollowComment"

Answer: "0If you get AWS Backup to back up an EC2 instance, there's no need to separately back up its EBS volumes.  When you back up an EC2 instance, AWS Backup will protect all EBS volumes attached to the instance, and it will attach them to an AMI that stores all parameters from the original EC2 instance except for two (Elastic Inference Accelerator and user data script).CommentShareEXPERTskinsmananswered a year ago"


Question: How is bounce rate calculated?

AWS encourages customers to keep their bounce rates below 5%. Does 5% mean 5% of the total number of emails a customer can send per day? If not how then is it calculated. I need the answer to be explained as simply as possible. Thanks in advance.FollowComment

Answer: "0Accepted AnswerHello,Quoting from our FAQ'sQ5. Over what period of time is my bounce rate calculated?We don't calculate your bounce rate based on a fixed period of time, because different senders send at different rates. Instead, we look at a representative volume—an amount of email that represents your typical sending practices. To be fair to both high- and low-volume senders, the representative volume is different for each user and changes as the user's sending patterns change.Q6. Can I calculate my own bounce rate by using the information from the Amazon SES console or the GetSendStatistics API?No. The bounce rate is calculated using representative volume. Depending on your sending rate, your bounce rate can stretch farther back in time than the Amazon SES console or GetSendStatistics can retrieve. In addition, only emails to non-verified domains are considered when calculating your bounce rate. However, if you regularly monitor your bounce rates using those methods, you should still have a good indicator that you can use to catch problems before they get to levels that cause us to place your account under review or pause your account's ability to send email.You can refer the below guide for more details.Amazon SES Sending Review Process FAQs - Amazon SES Bounce FAQ -https://docs.aws.amazon.com/ses/latest/DeveloperGuide/e-faq.html#e-faq-bnRegards,Gaurav @ AWSCommentShareAWS-User-7751712answered 4 years ago"


Question: Aurora Serverless V2 repeatedly fails over because LSN couldn't advance

"We recently migrated our production workload to Aurora MySQL and we've had the primary writer instance fail, triggering a fail over. The fail over triggers errors in our application as the connection pool attempts to maintain it's connection to the now defunct instance. The error in the logs is:Attempting to restart the database with zero downtime because the last sequence number (LSN) couldn't advance.Is this a misconfiguration or issue with Aurora Serverless V2? Searching for this error yielded no results.FollowComment"

Answer: "1This error message, "Attempting to restart the database with zero downtime because the last sequence number (LSN) couldn't advance," suggests that the primary instance was unable to write new transactions to the database, leading to the failover.If your application is generating a high volume of write operations that the database can't keep up with, you might see this error. You might need to adjust your workload, or consider scaling up your Aurora Serverless v2 capacity.CommentShareEXPERTsedat_salmananswered 23 days ago"


Question: Highly Available

"what is aws managed services,give examples. Are all managed services highly availabe, scalable,and elastic?FollowCommentBrettski-AWS EXPERTa year agoThat sounds a lot like an assignment question. Could you be more specific about your requirements?Share"

Answer: "0Hi,It appears you are referring to services managed by AWS, as opposed to the service "AWS Managed Services". Based on your tags, you are specifically looking for compute-related services as well. Services that are managed by AWS do not require you to configure servers. Lambda, Batch, Auto Scaling, Aurora, and RDS are all services managed by AWS where you do not have to worry about the underlying hardware, and are provided as a service. As these services are created with the Well Architected Framework/ AWS best practices in mind, they are all intended to be scalable, elastic, and highly available depending on your use case, which is further clarified below.https://aws.amazon.com/products/compute/?nc2=h_ql_prod_cpCommentSharekedkormeanswered 10 months ago"


Question: V2: Use Thing name in MQTT topic for triggering Lambda

"I want to be able to send messages to specific Things and have them trigger a Lambda on the Thing. I can get it working for a single Thing by hardcoding the Thing name in the YAML when I create the component:lambdaFunction:    ...    componentLambdaParameters:      eventSources:        - topic: print-job/MyThing          type: IOT_CORE  But I haven't been able to find a way to make the topic dynamic. I thought print-job/${AWS_IOT_THING_NAME} was working when I first tried it, but I think I just misunderstood how components' configurations are updated and it wasn't actually applied because I didn't have reset: \[""] in the deployment. I also tried print-job/{system:thingName} without any luck.Is this possible to do this without using wildcards or creating a separate deployment for each Thing? And feel free to tell me if what I'm trying to do doesn't actually make sense.I'd rather not use a wildcard in the topic because each Thing only needs to receive the messages sent to the topic for that Thing. It could just discard the other messages, but that seems like a waste of resources.If there's no way to do this using LambdaManager, I can use a pinned Lambda and have it subscribe to the topic. I just wanted to check to see if there's an easier/cleaner way first.FollowComment"

Answer: "0Kinda trying the same here: https://forums.aws.amazon.com/thread.jspa?threadID=336516&tstart=0I don't think there is a way unless your lambda is pinned and "manually" subscribes to its corresponding IoT Core topicCommentShareQuantumLoveanswered 2 years ago0Hi knad,Thank you for posting your question.Unfortunately there isn't an easier way to use a dynamic topic as you described.You could try either of the following:Use a wildcard in the topic.Use a pinned lambda and subscribe to it's corresponding topic.You could also explore creating a custom component https://docs.aws.amazon.com/greengrass/v2/developerguide/create-components.html instead of a lambda. You could then leverage the environment variables available to the component https://docs.aws.amazon.com/greengrass/v2/developerguide/component-environment-variables.htmlPlease let me know if either of these options work for you.Thanks,Navya.CommentShareNavyaKuchibhotla-AWSanswered 2 years ago"


Question: UE4 on Gaemelift: What are the best practices for securing and separating server only code from client - Updating Databases etc?

"What is the best practice for securing and separating a UE4 server's ability—hosted via Gamelift—to update databases and perform other tasks that only the server should see and have access to?The methods we have come up with are:With an Auth Token to an internal API: utilize Pre-processor directives so that these functions and tokens are never even shipped with the client. (The downside to this is that most of our team is allergic to formal code, so we are not sure how UE4 handles segregated, pre-processor directives that are Blueprint callable functions. Will this cause problems if the Blueprint UFunction ends up being removed on the client?)Same as 1, but have the servers make a GET request on startup to receive dynamically generated and cycled auth tokens.What method should we be pursuing to secure our UE4 server's ability to modify databases etc?FollowComment"

Answer: "0Hey - sorry for the delay! GameLift usually doesn't recommend managing auth through the game server or client. Does your game have a corresponding back end service that can handle this for you?Some docs on the topic:https://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-architecture.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift_quickstart_customservers_designbackend.htmlCommentShareAlanZ_AWSanswered 9 months agorePost-User-1475035 8 months agoYes, we do have a back end service; however, we are trying to figure out how to add an extra level of security check to make sure that ONLY the game server is the one making the call. Someone recommended checking if the call is coming from within the same VPC https://repost.aws/questions/QU0MPwSTJGQhKDcl9Zw1e_zQ/aws-game-lift-server-best-solution-for-generating-and-rotating-api-keys-for-aws-server-authentication but I'm not sure if Gamelift can leverage that or not. The problem with UE4 is that the client and server code is in the same build, which can be separated via directives.Share"


Question: Python bases AWS Lambda function with pysftp module not working

"I have a python based Lambda function that suppose to log into a SFTP server and downlaod a file.To do so I use a python module named pysftp.I uploaded the pysftp module (and its dependencies) I installed on my machine with the Lambda function source code file to the AWS Lambda directory, but when I tested the function I got the following error:"[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': cannot import name 'asn1' from 'cryptography.hazmat.bindings._rust' (unknown location)Traceback (most recent call last):"I tried to search about the error and some people say the problem is having native libraries in the installed modules files that are installed differently on my windows OS than the AWS OS.The propsed solution was to use a Docker, but I am not familiar with it at all so I wanted to ask if it is in fact a problem involving native libraries and different OS's, and if so how do I implement the Docker solution or any other solution?FollowComment"

Answer: "1Accepted AnswerHi,Have a look at this: https://repost.aws/knowledge-center/lambda-python-package-compatibleEssentially it mentions yo specify platform flag when installing dependencies, as certain libraries, if installed locally from windows, will conflict as they are expected with Linux binaries.Hope it helps ;)CommentShareEXPERTalatechanswered a month agoTamirm a month agoIt worked, thank you very much.I had to install some of the dependencies for the AWS platform using pip and upload them with the rest of the modules and the source code.Share0AWS Lambda doesn't provide the cryptography library for Python so you have to package the library with your Lambda function, which means you have to package/compile it somewhere outside Lambda and then import it into Lambda. I recommend you package the libraries inside the Amazon Linux-based EC2 instance as the versions will be compatible. I have faced a similar issue and got it fixed by getting all the library files inside the Amazon Linux-based EC2 instance.Using Docker is another good solution to ensure that you have the same environment on both your local machine and the Lambda environment. Essentially, you will create a Docker container that replicates the Lambda environment, including the necessary dependencies such as pysftp and cryptography. You can then use this container to run your Lambda function, which should resolve the dependency issues.CommentShareManuanswered a month ago"


Question: Can I count custom object in single image with Amazon Rekognition Custom Labels?

I need to count custom object in single image using Amazon Rekognition Custom Labels.For example I have image that containing multiple balls inside. I need to count how many balls in the image?Can I do it with Amazon Rekognition Custom Labels?ThanksFollowComment

Answer: "0Check this blog that explains how a store adjusts its workforce based on the number of people detected using Amazon Rekognition. Around the middle of the blog there is a lambda function that uses amazon Rekognition to "count number of people in the picture".CommentShareDavid Canswered 24 days ago0Hi,Thanks for your interest in Rekognition Custom Labels! This is one of the common use cases for Custom Labels. If you have a set of images that contains a specific type of balls you can annotate them using the Custom Labels console. After that you can submit them for training and will get a model tailored to the specific use case. With the model, you can further run real time inference to get the ball detected and thus counting them.Check out the documentation https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/understanding-custom-labels.html#tm-object-localization for more information.Thank you!CommentShareaws-renjunanswered 11 days ago"


Question: Question regarding Lightsail/Plesk and snapshots

"Hello, I am going through the process of upgrading one of my LightSail instances.In order to do this I created a snapshot from the original then created a new instance.However I noticed something strange, and maybe my lack of knowledge in AWS is to blame.I cannot gain access to plesk in the new instance I created from the snapshot. Plesk is asking for a username/pw yet the original did not have one set up.I only gained access to the original instance by using the following command: sudo plesk login | grep -v internal:8.I would then use the url provided and it gave me access, no need for username/pw.Yet on this new instance (which I assume should be a straight copy) is forcing a username/pw, as well as sending me to the ip address of the old instance.How do I gain access to the plesk on my new instance?Any insight into this issue would be greatly appreciated, as of right now I have no way of accessing plesk for the new instance.FollowComment"


Question: Using Terraform with AWS SSO accounts

"Hi,I'm using ControlTower to deploy my infrastructure with multiple accounts.Now I want to use terraform but authentication is currently done via SSO and I have noticed that terraform isn't able to handle these kind of credentials.Which is the best way to use terraform with SSO ? I think the only way at the moment is to use a dedicated user in a separated account.thank you guys,Ax.FollowComment"

Answer: "2If you create a profile through SSO using the AWS CLI with aws configure sso [1], you should be able to then use this profile name within Terraform, either by setting it within the Terraform provider [2], or by setting the AWS_PROFILE environment variable in the shell before you run terraform apply.[1] https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#sso-configure-profile-auto[2] https://registry.terraform.io/providers/hashicorp/aws/latest/docs#shared-configuration-and-credentials-filesCommentShareEdanswered a year ago1The existing answers are good, I'm just going to give you a warning about actually using Terraform + AWS SSO... The role that is used to create resources in AWS will be the owner of certain resources (like your EKS cluster). This situation is worse with KMS keys! if your SSO administrator deletes and recreates the permissions set that Terraform uses, you could be left with KMS keys that you have no access to. I ran into this! The AWS Identity Center permissions sets got refactored, and the KMS keys that were created via the terraform-aws-modules/eks module were left 'orphaned'. Not even the root account can recover these. I need to contact AWS Support, but since this is not a production account I do not have it on a paid support plan. (AWS Support, unlike Billing, is not consolidated under an AWS Organization, it's a per-account charge)CommentSharerePost-User-2162580answered 24 days ago0Just authenticate and export the AWS_PROFILE, terraform will automatically use the temporary credentials you obtained stating ss login in the CLI.To operate on different account do as usual, using multiple profiles aliases.CommentShareE Gabriellianswered 5 months ago0If you use ControlTower you need to deploy a custom role, moreover the use of AWS_PROFILE in team is a mess because you must have the same profile name set on each local machine that run that code. Also in a pipeline could be a problem.CommentSharelpisciottaanswered 5 months ago"


Question: Random Queries with [ErrorCode: INTERNAL_ERROR_QUERY_ENGINE] Amazon Athena experienced an internal error

"HiI'm testing a lambda function that executes batches of small athena queries against an S3 glue table.When I execute the function code locally as my admin it schedules the queries and they never error, when the function executes them perhaps 1 in 30 fail (each query is the same with different partition constraints, changing the date range of files to search).I have even set the code to execute the identical query again if the original fails, in the majority of cases (not 100% of them but most) the 2nd execution succeeds, with no change in permissions or the query itself.If I select the failed query from the athena console and re-run it, it executes without error.If anyone from AWS support can look at the error, here are two identical queries in us-east-1 that failed and then succeeded...042735b7-31e6-4ac4-90ef-913ac8c5201f FAILED04a415cd-3f01-42b8-a05c-8b344d06085e SUCCEEDEDRegardsMatthewFollowComment"

Answer: "0Accepted AnswerCould you please share the exact error with which the query fails?based on the few information it could be that when the lambda function executes the query it submits more queries that your quota allow as explained here.If this is the case you could be able to resolve the issue either:requesting a quota increase (could not work if later you continue to submit more queries than the quota)throttling the number of queries that your lambda can run in parallel.set up a retry mechanismhope this helpsCommentShareEXPERTFabrizio@AWSanswered 24 days agorePost-User-4872743 24 days agoThanks, that was more or less the entire message, i had previously hit the quota exceeded messages, when this happens you get a meaningful error. I have asked for a quota increase but this particular message was not quota based (or the error was not saying quota), even when only one process was running I was getting random instances of these. That was on the 10th, however yesterday this behaviour stopped, the same processes running yesterday didn't fail randomly in the same way. My gut is something was failing behind the scenes and AWS have resolved the issue in this region. I did spend a day adding in retry mechanisms and reducing the batch sizes of the queries to make my process more tolerant to things like this.Share"


Question: URGENT Action Required - Your certificate renewal

"Received an email for Certificate Renewal stating the link has been forwarded for approval, but i have not received the link in my email ID,Can you let me know in which email ID it is forwarded?It would be helpful if i get an answer at the earliest as i have to renew the same before 24th July 2022.Regards,Smrithi Kumar IyerFollowComment"

Answer: "1Hello Smrithi,For ACM certificate renewal, the ACM performs the MX record lookup and/or it will send to the domain owner register email. Below are the some common email which uses mostly.administrator@your_domain_namehostmaster@your_domain_namepostmaster@your_domain_namewebmaster@your_domain_nameadmin@your_domain_nameYou can get the email address which are pointed to you domain certificate from the ACM console.AWS Certificate Manager>> Certificates>> choose_the_certificate and under the domain look for Registered owners, which will show up the list of emails where the link is shared.Also, one more quick way to check the email address for the domain certificate using Whois tool and look for "Registrant Contact" and Email row{+} Example consider for domain amazon.com:-https://www.whois.com/whois/amazon.comhere Email is hostmaster@amazon.comNote:- If you found your email in that list and not received then it might blocked by spam filters or lost in transit. Click on "To resend validation email" tab to re-generate again.https://docs.aws.amazon.com/acm/latest/userguide/request-domain-validation-email-for-renewal.html[+] Using cli to resend-email for certification.$ aws acm resend-validation-email --certificate-arn arn:aws:acm:region:account:certificate-authority/CA_ID/certificate/certificate_ID --domain subdomain.example.com --validation-domain example.comRegards:CKCommentShareChethan SEanswered a year ago"


Question: Unable to delete Managed AD Directory

"HelloI'm trying to delete unused managed AD directory I get error message:"Cannot delete the directory because it still has authorized applications. : RequestId: 8303df88-c4c2-4a3a-bdb1-de8305377158"When viewing apps and services on the directory in question, all applications have status disabled.I would appreciate any help on how to delete this.MarwenFollowComment"

Answer: "0can u try running this command and replace <directory-id> with the ID of your AWS managed AD directory. This command will return a list of all the authorized applications that have been granted access to your directory. Maybe some of them u dont see in the AWS Console?aws ds list-applications --directory-id <directory-id> If you have a large number of applications, you can use the --output parameter to customize the output format. For example, you can use the following command to display the output in a table format:aws ds list-applications --directory-id <directory-id> --output tableCommentShareNikoanswered 3 months agoMarwen 3 months agoI couldn't find the sub command list-applications :/ I didn't find it in here https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ds/index.html#cli-aws-dsHowever, i tried this command and  I got an empty arrayaws ds describe-conditional-forwarders --directory-id <directory-id>ShareNiko 3 months agohow about this - https://docs.aws.amazon.com/cli/latest/reference/ds/describe-trusts.html ?ShareMarwen 3 months agoSame thing always an empty arrayShareNiko 3 months agodo you AWS SSO (identity center) enabled by any chance?ShareMarwen 3 months agoNo it's disabled..ShareShow 1 more comment0Kindly check through this steps to delete your directory- https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_delete.htmlCommentSharerePost-User-5076019answered 3 months agoMarwen 3 months agoI did . A lot of these services are not available in my region.. and I have everything disabled in the consoleShare"


Question: Elasticcache scaling up takes time

"Hi Team,I have configured dynamic scaling for elastic cache for 80% memory usage, adding shards.Configurations :1- 3 primary2- 2 replica each3. cache.r6g.largeWhen it's triggering up, AWS elastic cache modifying the cluster is taking up. Sometimes more than an hour for 14GB of instance.The impact is when modifying timeout is happening in the application.Is there any way to avoid this ?FollowComment"


Question: AWS ECS containers having random difficulties connecting to AWS RDS

"Hello, we are running containers in ECS and using host/url to specify the database for a Rails app. It works on 80% of the instances, and in some other instances, the system complains of a socket connection to mysql error:A Mysql2::Error::ConnectionError occurred in #:Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)I can't figure out why this is very random, connection capacity in the RDS is not even at 10%. It's always triggered with the ELB check, randomly. Any direction is appreciated, thanks!FollowComment"

Answer: "0Hello, dependency health check is not a best practices as it can flag your task as irresponsive even in case of an other problem on the database. cf https://aws.amazon.com/builders-library/implementing-health-checks/Does you code has a fallback on the local socket in case of connection error on the url of the DB ?There are several factors that can make connection to fail on the DB: the number of allowed connections, or the network bandwith that are different dependent of the instance size : https://aws.amazon.com/rds/instance-types/CommentShareallamandanswered 6 months ago0Hi,Thank you for contacting us.Based on the error message we could see that there is some local database dependency in the application due to which it it trying to connect to the database in localhost rather than RDS. If there are no other congestion noted from the container side, please check from the application point to understand more on this.Thank you.CommentSharePrasobh S Panswered 6 months ago"


Question: Unable to copy Elasticache for redis backup

"I have followed those instructions step by step: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups-exporting.htmlHowever I have the following error:An error occurred (InvalidParameterValue) when calling the CopySnapshot operation: Elasticache was unable to validate the authenticated user has access on the S3 bucket ...The bucket is in the same region of the backup.This is my bucket configuration:{    "LocationConstraint": "eu-central-1"}{    "Version": "2012-10-17",    "Id": "xxxxxxxx",    "Statement": [        {            "Effect": "Allow",            "Principal": {                "Service": "eu-central-1.elasticache-snapshot.amazonaws.com"            },            "Action": [                "s3:PutObject",                "s3:GetObject",                "s3:ListBucket",                "s3:GetBucketAcl",                "s3:ListMultipartUploadParts",                "s3:ListBucketMultipartUploads"            ],            "Resource": [                "arn:aws:s3:::my-bucket-name/*",                "arn:aws:s3:::my-bucket-name"            ]        }    ]}This is the snapshot{    "Snapshots": [        {            "SnapshotName": "my-snapshot-name",            "CacheClusterId": "xxxxxxxx-xxx",            "SnapshotStatus": "available",            "SnapshotSource": "manual",            "CacheNodeType": "cache.t2.micro",            "Engine": "redis",            "EngineVersion": "5.0.3",            "NumCacheNodes": 1,            "PreferredAvailabilityZone": "eu-central-1c",            "CacheClusterCreateTime": "xxxxxxx",            "PreferredMaintenanceWindow": "mon:02:30-mon:03:30",            "Port": 6379,            "CacheParameterGroupName": "default.redis5.0",            "CacheSubnetGroupName": "internal",            "VpcId": "xxxxx",            "AutoMinorVersionUpgrade": true,            "SnapshotRetentionLimit": 7,            "SnapshotWindow": "00:00-02:00",            "NodeSnapshots": [                {                    "CacheNodeId": "0001",                    "CacheSize": "33 MB",                    "CacheNodeCreateTime": "xxxxxx",                    "SnapshotCreateTime": "xxxxxx"                }            ],            "ARN": "arn:aws:elasticache:eu-central-1:000000000:snapshot:my-snapshot-name",            "DataTiering": "disabled"        }    ]}FollowComment"

Answer: "0Hi AWS-User-7081016,Just thought of checking if the step 3(Step 3: Export an ElastiCache backup) of the documentation is adhered. AWS expects the user who performs the operation to have the below permission. I'm assuming you have the below set of permission to your IAM user through which the export operation is attempted.{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Action": ["s3:GetBucketLocation","s3:ListAllMyBuckets","s3:PutObject","s3:GetObject","s3:DeleteObject","s3:ListBucket"],"Resource": "arn:aws:s3:::*"}]}CommentShareKarthikeyan Venkatramananswered a year agoAWS-User-7081016 a year agoHello, I have S3 full accessShare0Have you tried to enable the ACL to grant Elasticache access right according to the documentation?CommentShareblacktulip11answered 9 months ago"


Question: Nginx is not being served on EC2 instance Public IP

"I have installed nginx on ec2 instance and added in bound security rules for port 80, 22 and 443 but when I try to browse the public IP I got "This site can’t be reached" error but curl localhost is giving response in terminal. I have tried to change the region from US East (N. Virginia) to Asia Pacific (Mumbai) as well but it didn't help me out. I have registered in aws for the very first time and using free tier. Looking forward for the help. Thank youFollowComment"

Answer: "0Is there a route to the internet gateway on the subnet where the EC2 is located?Do the subnet's network ACLs allow the required communication?Does the security group allow the required IP addresses with inbound rules?For additional troubleshooting, check the following documents.https://repost.aws/knowledge-center/ec2-instance-hosting-unresponsive-websiteCommentShareEXPERTRiku_Kobayashianswered a month ago"


Question: how does inference work in multi model endpoint in sagemaker?

"based on the docs provided here, https://docs.aws.amazon.com/sagemaker/latest/dg/create-multi-model-endpoint.html. i created a multi model endpoint and invoked it as documented here https://docs.aws.amazon.com/sagemaker/latest/dg/invoke-multi-model-endpoint.html. I'm getting a Invalid model exception and the message => "model version is not defined" . my set up is , i have created two models , say modelOne and modelTwo.tar.gz file , and both models have their own custom script/inference.py file with following directory structure.when we send request to a multi model endpoint, does sagemaker uncompresses the tar.gz file specified in the request, as in my case both models have save directory structure and same model.pth files inside the tar files , is it getting mixed up and not sure which one to invoke?inference.pyimport torchimport osdef model_fn(model_dir, context):    model = Your_Model()    with open(os.path.join(model_dir, 'model.pth'), 'rb') as f:        model.load_state_dict(torch.load(f))    return modeldirectory structuremodel.tar.gz/|- model.pth|- code/  |- inference.py  |- requirements.txt  response = runtime_sagemaker_client.invoke_endpoint(                        EndpointName = "my-multi-model-endpoint",                        ContentType  = "text/csv",                        TargetModel  = "modelOne.tar.gz",                        Body         = body)FollowComment"


Question: Error in place design stage [DRC HDPR-6] Logic illegally placed:

"Hi ,I am creating a project with a pblock drawn in SLR2 and when i implement it , I am getting the following errors in the place design stage :ERROR: (DRC HDPR-6) Logic illegally placed: Cell 'WRAPPER_INST/SH/SH/RC_AXISTR_REG_SLICE/inst/axisc_register_slice_0/gen_AB_reg_slice.payload_a_reg(186)' is placed at site 'SLICE_X159Y249' which belongs to reconfigurable Pblock 'pblock_SH'.  This cell is not part of the reconfigurable logic assigned to this Pblock, and should not be placed at this siteERROR: (DRC HDPR-12) A Partition Pblock cannot reference more than one partition: The Pblock 'pblock_SLR2' references more than one partition instance. The Pblock references both partition instance 'WRAPPER_INST/CL' and partition instance 'WRAPPER_INST/SH'.I have created the Pblock through following commands in xdc(cl_pnr_usr.xdc) filecreate_pblock pblock_SLR2resize_pblock (get_pblocks pblock_SLR2) -add {CLOCKREGION_X0Y11:CLOCKREGION_X5Y14}resize_pblock (get_pblocks pblock_SLR2) -add {CLOCKREGION_X0Y10:CLOCKREGION_X2Y10}set_property PARENT pblock_CL (get_pblocks pblock_SLR2)Can anyone please suggest what is going wrong which is producing this error?FollowComment"

Answer: "0Hi Prigish,We are checking on this and we will get back to you soon.Thanks,Sunil.CommentShareawssunilanswered 4 years ago0Hello all,I think we need more information to investigate this issue. First, if we could obtain the constraints file(s) that define these elements that would be helpful.Specific questions to ask are:Where do you intend the element in question (payload_a_reg(186)) to reside? Is the register supposed to be a part of the static pblock, or a different RP pblock?Pblock_SH is not indicated in the constraints you listed, so we would need to know about this as well.OK, if you could provide this information we would appreciate it.Best regards,ScottCommentShareScottXilinxanswered 4 years ago0Hi Scott,You can find the constraints (given by us) file through the following link:https://drive.google.com/open?id=10B-RmWEuVcFMWdoXb59eZJawUu30oaZdWe have not given any constraints for pblock_SH.Those are automatically generated by AWS scripts.CommentShareprigishanswered 4 years ago0Hi Scott,We faced another problem too during floorplanning in SLR2.We gave following commands in the xdc file :create_pblock pblock_SLR2resize_pblock (get_pblocks pblock_SLR2) -add {CLOCKREGION_X0Y10:CLOCKREGION_X5Y14}set_property PARENT pblock_CL (get_pblocks pblock_SLR2)During the place design stage, it throws the following error :(DRC HDPR-14)Nested Partition Pblocks must have a common ancestor Pblock: The Pblock 'pblock_CL' and Pblock 'pblock_BA452_top_0' do not have a common ancestor for the HD.RECONFIGURABLE partition 'WRAPPER_INST/CL'.(DRC HDPR-66) Reconfigurable Pblock must not overlap other pblocks.: HD.RECONFIGURABLE Pblock 'pblock_CL' and Pblock 'pblock_BA452_top_0' overlap.  Please re-floorplan Pblocks to ensure that reconfigurable pblocks don't have overlap with other rm or static pblocks.Can you please take a look into this issue too please?Currently we are using only the below mentioned portion of SLR 2:resize_pblock (get_pblocks pblock_SLR2) -add {CLOCKREGION_X0Y11:CLOCKREGION_X5Y14}resize_pblock (get_pblocks pblock_SLR2) -add {CLOCKREGION_X0Y10:CLOCKREGION_X2Y10}Can you please suggest what needs to be done to overcome this?CommentShareprigishanswered 4 years ago0Hello,Thank you for providing the constraints, that helps. I think for the first error, this looks to be valid, and that your reconfigurable partition boundaries are overlapping the AWS static partition. I am not sure of those boundaries, so was wondering if you could provide a complete set of constraints excluding the timing ones. You can do that with the following command:“write_xdc -exclude_timing netlist_constraints.xdc”.This should give us the entire PBLOCK structure.For the second error, this may be due to a known issue with parent/child blocks and snapping mode. The workaround for that issue is to set the SNAPPING_MODE to ON. Can you try that to see if it helps with your design?OK, thank you,ScottCommentShareScottXilinxanswered 4 years ago0Thanks for the response Scott.Actually I was able to get through the first error.There was a "register slice 0" mentioned in my XDC file which was causing the trouble because it was being referenced both by pblock_CL and pblock_SH .So I changed the name in the XDC file and that problem got solved.Regarding the second problem, I will try your recommendation(SNAPPING mode ON) and will update you with the results.ThanksPrigishCommentShareprigishanswered 4 years ago"


Question: How to detect drifts of Cloudformation Custom Resource?

"Hello, I create one Cloudformatin resource where I am creating custom resources by following:-AWS::CloudFormation::CustomResourceNow I made a few manual changes in created resource and when I am checking the drift of CloudFormation template, It is showing InSync,So how can I check the drift of custom resources?FollowCommentIhtesham a year agoPlease share your custom resource cloudformation code and also write in detail what changes have you made specifically.Share"

Answer: "0This feature isn't currently available, you can only detect drift on supported resource types. So you'd have to handle this outside of CloudFormation, using a custom solution.CommentShareEXPERTRaphaelanswered a year ago"


Question: Route 53 Alias To API Gateway Not Routing Traffic

"Hi everyone,I recently followed multiple tutorials from both AWS and Youtube to make sure I set up everything correctly. To summarize - I've set up a custom domain name for an api in API Gateway, and I've also configured some Cloudwatch Logs so I know when a request hits the Api. My Route53 has the correct record (using A-Alias type) to route api.mywebsite.com to the API Gateway domain name as seen in the API Gateway console. However, when I try to see if api.mywebsite.com resolves correctly, it does not. In addition, it does not log anything to Cloudwatch, suggesting that it's not even routing traffic.Things I've tried -adding different routes to the API Gateway configuration.modifying the custom domain name mapping to have an empty path so that api.mywebsite.com by itself is accepted by API Gateway, theoretically.changing the ACM certificate used by the custom domain to match the name of api.mywebsite.com, character for character.adding a $default route in API Gateway that will accept any path.I feel like I'm reaching the end of my rope here. I apologize in advance if these forums are not the best place for such questions.FollowComment"

Answer: "1I feel very silly - it turns out that in my computer browser typing in api.mywebsite.com directed to http://api.mywebsite.com, which failed, as compared to my mobile browser, which directed it to https://api.mywebsite.com, which worked.Closing this threadCommentShareapokellypseanswered 3 years ago0Another interesting thing is that my mobile browser successfully hits the backend through api.mywebsite.com, but on my desktop Chrome and Firefox hitting api.mywebsite.com fails.CommentShareapokellypseanswered 3 years ago"


Question: Connecting OpenSearch to QuickSight

"I'm trying to connect AWS OpenSearch to QuickSight and export data to SPICE, but I get ELASTICSEARCH_CURSOR_NOT_ENABLED exception.I was not able to find how to enable cursor for opensearch distro 1.1region:eu-west-1timestamp:1649003264494requestId:95b8a137-9839-453c-affc-f50c0e32ae4esourceErrorCode:ELASTICSEARCH_CURSOR_NOT_ENABLEDsourceErrorMessage:The domain monei-prod has not enabled cursor.FollowComment"

Answer: "1I've been able to enable cursors using an undocumented feature: sending the settings to the query endpoint instead of the _cluster/settings:PUT _opendistro/_sql/settings{  "persistent" : {    "opendistro.sql.cursor.enabled" : true  }}{  "acknowledged" : true,  "persistent" : {    "opendistro" : {      "sql" : {        "cursor" : {          "enabled" : "true"        }      }    }  },  "transient" : { }}works.After this it is possible to ingest data to SPICE from QS.CommentShareraul_romayanswered a year ago0Hi - I believe this is due to "The ElasticSearch/OpenSearch domain doesn't have SQL cursors enabled ("opendistro.sql.cursor.enabled" : "true")." (1). The Cursor is one of the settings you can configure on your cluster (2). You can enable it using Kibana or CLI as well.(1) https://docs.aws.amazon.com/quicksight/latest/user/errors-skipped-rows-during-import.html(2) https://opendistro.github.io/for-elasticsearch-docs/docs/sql/settings/CommentShareEXPERTGokulanswered a year ago0I don't understand how this can be the accepted answer, that setting is removed from OpenSearch since version 1.0.https://github.com/opensearch-project/sql/pull/75It is not working in OpenSearch v1.2 either.CommentShareraul_romayanswered a year ago"


Question: MSCK REPAIR TABLE behaves differently when executed via Spark Context vs Athena Console/boto3

"I have a Glue ETL job which creates partitions during the job    additionalOptions = {"enableUpdateCatalog": True, "updateBehavior": "LOG"}     additionalOptions["partitionKeys"] = ["year", "month", "day"]I don’t have it Update the Data Catalog because doing so changes all my Table Data Types.  So after I am done, the way I get the Data Catalog updated with the correct partition information is to run MSCK REPAIR TABLE.  If I do this inside the Glue ETL job using the Spark Context like so:   spark.sql("use gp550_load_database_beta")   spark.sql("msck repair table gp550_load_table_beta").show()The following happens:Serde Properties of my table are updated with “serialization.format : 1”Table Properties are updated with:EXTERNAL : TRUEspark.sql.partitionProvider : catalogALL Data Types in my table are set to “String” with a comment of “from deserializer”Basically it makes a mess.If I instead run MSCK REPAIR TABLE from boto3, or if I manually run it from Athena Console, then there are no issues.  No Serde Properties are changes, no table properties, no data types are changed, it simply adds my partitions to the Data Catalog like I want.  I do like so in my ETL job:    client = boto3.client('athena')    sql = 'MSCK REPAIR TABLE gp550_load_database_beta.gp550_load_table_beta'    context = {'Database': 'gp550_load_database_beta'}    client.start_query_execution(QueryString = sql,                                  QueryExecutionContext = context,                                 ResultConfiguration= { 'OutputLocation': 's3://aws-glue-assets-977465404123-us-east-1/temporary/' })Why does it behave so differently?  Is it because somehow I need to tell Spark to work with HIVE?  I had thought that since I already had a spark context it would be easy to use that to kick off the MSCK REPAIR TABLE, but obviously I was surprised at the result!FollowComment"


Question: "Aurora serverless, RDSDataService, executeStatement() API, " Unexpected key 'formatRecordsAs' found in params" error"

"Hello, I'm trying to add formatRecordsAs parameter to sqlParams for Aurora serverless RDSDataService (according to RDSDataService SDK) and getting "Unexpected key 'formatRecordsAs' found in params".Here is my code sample:let sqlParams = {        secretArn: storeArn.Value,        resourceArn: dbArn.Value,        database: dbSchema.Value,        formatRecordsAs: 'JSON'      };sqlParams.sql = `select ...`;let data = await RDS.executeStatement(sqlParams).promise();Note that without "formatRecordsAs" parameter all works just fine. I double checked with RDSDataService SDK docs and didn't find any inconsistency in my code.What I might possibly missing here???Thank you,Victor.FollowComment"

Answer: "0Hi, if you are using a lambda function, then the sdk version in use is 2.1055 at the time of this answer (https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html), and in this version 'formatRecordAs' parameter isn't supported.As a workaround, you should create a lambda layer with a more recent sdk version (v3 is already available).Hope this helps :)CommentShareAlexisanswered 10 months ago"


Question: Route 53: Transfer Domain (.com.mt)

Are there any plans to support .com.mt for domains as we would like to transfer our domain over. Thanks for you time!FollowComment

Answer: "0Greetings from AWS..!!I understand that you would like to transfer the “.com.mt” domain to Route 53.Please note that Route 53 Domains is currently directly accredited with .com, .net, and .org registries. All other TLDs are provided via third party registrar associates. To improve our level of service and support, we are actively working to become directly accredited with all other TLDs. In addition to improving our existing offering, we will become accredited with other registries to expand our TLD catalog. This work is currently in progress. I have raised a feature request with the service team on your behalf for adding support for ".com.mt" domains.While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our AWS Blogs - https://aws.amazon.com/blogs/aws/ and what's new on AWS - https://aws.amazon.com/new/  for any new feature announcements.CommentShareSuchita_Janswered a year ago"


Question: EFS Client Connections Metric Fluctuations

"Hello,I have 3 EC2 instances, each mounted to the same EFS file system. For the file system metric "Client connections", I would expect to have a constant value of 3.However, when viewing this graphed metric on the "Sum" statistic, there are periodic fluctuations. Most of the time the value is 3, but sometimes can drop to 2, then sometimes go up to 4.Please help me understand why this is happening. I followed the documentation for mounting but perhaps something went wrong somehow? I'm concerned there is some intermittent connection issue that may impact performance/reliability.Any suggestions on how I should further troubleshoot?Thanks for reading.FollowComment"

Answer: "0Metrics are a "point in time" measurement. The number you're seeing is true for an instant - when the measurement is taken.My guess here (and it is a guess) is that there are times when one of your instances has disconnected (there will be timeouts, reboots, kernel updates, many things that can cause a brief disconnect) and during that time there will be one less connection to EFS; and that may be when the metric is calculated.Similarly, an unexpected drop in a connection and a subsequent reconnection will see a situation where EFS thinks the old connection is still alive (it will timeout eventually) as well as the new connection. In that case you'll see four connections for a short period.However: It's impossible to tell without detailed logs from the operating system and similarly detailed logs from EFS. Our support team can assist with troubleshooting the EFS side so I'd recommend that you raise a support case with them to dive deeper.From a reliability and performance perspective I would enable detailed filesystem and network logging on your instances (temporarily) to try and determine if and why connections are dropping.CommentShareEXPERTBrettski-AWSanswered 10 months ago"


Question: "Route53, ACM seem not validating certificate"

"Hi,We registered a domain using AWS and now we are trying to configure ACM and Route53 in order to use it. The certificate is stuck in "Pending Validation" status even if the validation CNAMEs are present the Route53 hosted zone.The hosted zone (and other resources) has been created later (not at the moment of the domain registration) with Terraform using the working code from a previous project. It has been created with different name servers than what I see in the console at the domain details page. Do name servers have to be the same?In order to test the DNS, we added a CNAME record to route test.<domain-name>.com to google.com but it doesn't work. This test works instead with another domain (<domain-name>.live) that is also registered in AWS and managed via Route53.We also tried to run an nslookup on <domain-name>.com and test.<domain-name>.com but it looks like this domain doesn't exist.I already had the same issue with the previous domain because it hasn't been confirmed and then suspended, so my supposition is that there's something wrong with the domain registration process, also because this time the domain has been activate without email confirmation. Is it normal?How can we check if everything is okay with the domain registration, that the domain is active and validate the certificate?Thank you.EDIT:When I run nslookup.io with the root of the *.live domain that is working I can see the name servers:But when I try with the new one, the name servers don't show up:FollowComment"

Answer: "1I will answer your questions point wise:The hosted zone (and other resources) has been created later (not at the moment of the domain registration) with Terraform using the working code from a previous project. It has been created with different name servers than what I see in the console at the domain details page. Do name servers have to be the same?Yes, the name servers have to be same. When you create a hosted zone, Route 53 assigns a set of four name servers to the hosted zone. If you delete a hosted zone and then create a new one, Route 53 assigns another set of four name servers. Typically, none of the name servers for the new hosted zone match any of the name servers for the previous hosted zone. If you don't update the domain configuration to use the name servers for the new hosted zone, the domain will remain unavailable on the internet.Depending upon which hosted zone you want to use, you would need to update the name servers of the domain under "Registered Domains" section in Route 53 console to the same name servers of the hosted zone you want to use. Please refer to https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-name-servers-glue-records.html for more information on how to add or change name servers for a domain.How can we check if everything is okay with the domain registration, that the domain is active and validate the certificate?To check the status of a domain registered with Route53, please follow the steps at https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-view-status.html.Once you correct the name server mismatch for your domain and add the ACM generated CNAME records to the relevant hosted zone for your domain, the certificate should pas the domain validation. To learn more about DNS validation, check our public documentation at https://docs.aws.amazon.com/acm/latest/userguide/dns-validation.html.Note: Please note if a domain validation is not successful within 72 hours of a certificate request, the certificate will fail with a status of "Validation_Timed_Out". Refer: https://docs.aws.amazon.com/acm/latest/userguide/troubleshooting-timed-out.html.CommentShareAWS-User-4837236answered 2 months agobraejay99 2 months agoYour solution to step one helped solve my issue! Thank you for sharing with the community!Share"


Question: "Golang SDK: IAM check whether a user is root user name or not programmatically, I am using listusers to get al list of all users"

In Golang SDK V2 IAM service https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/iamHow can check whether a user is root user name or not programmatically.I am using listusers to get al list of all users and except for root user I want to do some action.Also checked "credential reports" that doesn't helpFollowComment

Answer: 0You can use the STS get-caller-identity to see who you are and then make a decision.CommentShareEXPERTkentradanswered 8 months agorePost-User-8566734 8 months agosts get-caller-identity can only give the identity of the user who calls the api and not for another user.some example scenario I am quoting belowI am calling the api as user01 and want to know the identity of user890808Share0Any solution available for the problem I had shared?CommentSharerePost-User-8566734answered 7 months ago


Question: OpenSearch cross-cluster search and autotune

"IHAC would like to enable cross-cluster search, but the docs limit that to clusters created after June 2020. How to understand the limitation? Is that something we can work around? Creating a new cluster for their 100B documents is going to be quite painful.What a given autotune actually does? Right now it’s just lines on a graph, and sometimes the graph gets worse afterwards.What would it take to get shard allocation control? Not being able to control shard routing has been a real painpoint for them when allocation goes wrong.Similar, what’s the recommended way to kick a bad node from the cluster?Really appreciate your help!FollowComment"


Question: S3 CRR and objects locks

"A customer is building a DR architecture, using S3 CRR to copy its data to another region. All the infra is replicated (at small scale) in the DR region.The source S3 bucket is accessed by a SFTP server doing read/write operations. So there is  another SFTP server in the DR region, pointing to the CRR bucket, ready to take over if DR occurs.Question:Is there any kind of locking on the CRR done by the CRR process, preventing access from the DR SFTP server to the CRR bucket ?FollowComment"

Answer: "0Accepted AnswerThe S3 Consistency model applies, with no object locks between writes. See more details here: https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel.CommentShareEXPERTLuca_Ianswered 3 years ago"


Question: Add Firebase dynamic link TXT in route 53

I am trying to add firebase TXT to my route 53 to verify it... but still the status shows INSYNC../change/C0784828XK9MDNWLBYHE...FollowComment


Question: SES inbound SPF/DKIM verification

"Hello,I am curious if inbound emails verify SPF/DKIM. If SES doesn't do this, does SES pass back the following metadata for our app to do it ourselves:messages envelope addressIP addressThanks!FollowComment"

Answer: "1Accepted AnswerYes, Amazon SES does verify SPF and DKIM for incoming emails by default.When SES receives an incoming email, it automatically checks if the email's SPF and DKIM signatures are valid. If the signatures are valid, SES marks the email as authenticated and adds some additional headers to the email, which include information about the authentication results.These headers can be used by your application to verify that the email was sent by an authorized sender and that the message wasn't tampered with during transmission.Here are some examples of headers that SES adds to incoming emails:Authentication-Results: contains information about the results of the SPF and DKIM checks.X-SES-Spam-Verdict: contains information about whether the email was marked as spam or not.X-SES-Virus-Verdict: contains information about whether the email contained a virus or not.In addition to the authentication headers, SES also provides information about the message envelope and the IP address of the sender. This information can be accessed through the Amazon SES API or the Amazon SES console.To retrieve this information, you can use the "GetRawMessageContent" API action, which returns the raw, MIME-formatted content of an email message. You can then extract the message envelope and IP address information from the email headers.References:Amazon Simple Email Service (SES) - https://aws.amazon.com/ses/Authenticating Email with DKIM in Amazon SES - https://docs.aws.amazon.com/ses/latest/dg/send-email-authentication-dkim.htmlAuthenticating Email with SPF in Amazon SES - https://docs.aws.amazon.com/ses/latest/dg/send-email-authentication-spf.htmlAmazon SES API v2 Reference - https://docs.aws.amazon.com/ses/latest/APIReference-V2/Welcome.htmlCommentShareNIHAR-AWSanswered 2 months ago"


Question: AWS WAFv2 how to use httpRequest.httpVersion in rules

"I am wondering if it is possible to check HTTP version and incorporate them into WAF rules.For example - check httpRequest.httpVersion = 'HTTP/1.1' then add a label, the label would then be used in other rules.FollowComment"

Answer: "0AWS WAFv2 does not currently offer a direct configuration option for HTTP version.However, we can leverage CloudFront to achieve this functionality.CloudFront acts as the initial service that receives the request and adds the following headers:CloudFront-Forwarded-Proto – Contains the protocol of the viewer's request (HTTP or HTTPS).CloudFront-Viewer-Http-Version – Contains the HTTP version of the viewer's request.CloudFront-Viewer-TLS – Contains the SSL/TLS version, the cipher, and information about the SSL/TLS handshake that was used for the connection between the viewer and CloudFront.Reference: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/adding-cloudfront-headers.htmlCommentShareAbheeshanswered 3 months ago-2Hello Hieuvu,Yes! it's possible to create a rule with the matching condition to match {Key:Vlaue} "httpVersion":"HTTP/1.1".  To create a rule, please check Single header.https://docs.aws.amazon.com/waf/latest/developerguide/waf-rule-statement-fields-list.html#waf-rule-statement-request-component-single-headerTo label, use the action count and add the custom label to the request post that we are good to use the label match condition next.To label: https://docs.aws.amazon.com/waf/latest/developerguide/waf-rule-label-add.htmlLabel match condition example: https://docs.aws.amazon.com/waf/latest/developerguide/waf-rule-label-match-examples.htmlCommentShareChethan SEanswered 10 months agoHieuVu 10 months agoHi Chethan,I do not believe httpVersion is apart of the request header and therefore can not using the suggested match to a single header.Share"