Question: Elastic Beanstalk Management console down ?

"Hi,It seems like the elastic beanstalk management console is down.It is stuck on the loading page.I'm in Europe.Does anyone has the same problem ?Is there a place where we can monitor incidents or give direct feedback on such cases ?CheersJeremyFollowComment"

Answer: "0Here is some code I got from the console :moment.min.js:6 Deprecation warning: moment().lang() is deprecated. Use moment().localeData() instead.e @ moment.min.js:6jquery.min.js:2 Uncaught Error: [$parse:lexerr] http://errors.angularjs.org/1.8.1/$parse/lexerr?p0=Unexpected%20next%20character%20&p1=s%200-0%20%5B%E2%80%99%5D&p2=%E2%80%99http%3A%2F%2F%E2%80%99%20%2B%20environment.cname%20%7C%20linky%3A%E2%80%99_blank%E2%80%99at angular.js:99at Ob.throwError (angular.js:15876)at Ob.lex (angular.js:15799)at q.ast (angular.js:16000)at Nb.getAst (angular.js:17364)at Nb.parse (angular.js:17347)at f (angular.js:17505)at angular.js:14066at Array.map (<anonymous>)at h (angular.js:14066)DevTools failed to load source map: Could not load content for https://d3e2aud6m9sl41.cloudfront.net/gzip/Bower-angular-ui-select/dd0eec436b07f8f0b1d16579d972662d/angular-ui-select/dist/select.min.js.map: HTTP error: status code 403, net::ERR_HTTP_RESPONSE_CODE_FAILURECommentShareJezAndNicoanswered 2 years ago0Same problem here. I am also in the Europe (Paris) region.This problem occurs on my computer but a colleague loging in with my credentials doesn't have the error.CommentShareJGGanswered 2 years ago0My friend near Nancy has the same problem.I'm near Metz.I don't think it's related to location. Perhaps the web browser ?We tried Chrome and Firefox.CommentShareJezAndNicoanswered 2 years ago0We have also tried Firefox, Chrome and Safari. As well as 3 Mac OS & 1 Windows.It works on one Mac but not the other 2 nor the Windows.CommentShareJGGanswered 2 years ago0Hi.Exact same problem for me and my colleague, both in two different cities in France. Switching my Firefox from French to English seems to fix this problem. Thanks for the workaround idea ! Hope this will get fixed soon.CommentSharekogakanswered 2 years ago0I've found a workaround by switching the browser language to English.It may be caused by a ’ in a link in the french interface which can't be parsedEdited by: JGG on Sep 24, 2021 7:01 AMCommentShareJGGanswered 2 years ago"


Question: no basic auth credentials - error pushing docker image to ECR

"I'm struggling pushing a docker image to amazon registry (ECR) through my aws codepipeline. Despite I'm able to successfully login into the registry docker push fails with error no basic auth credentialsHere the buildspec.yml used by my codepipelineversion: 0.2phases:  install:    runtime-versions:      java: corretto8  pre_build:    commands:      - echo -e "[INFO] \t\t In the pre_build phase..."      - echo -e "[INFO] \t\t Logging in to Amazon ECR..."      - aws --version      - aws ecr get-login-password --region eu-central-1 | docker login --username AWS --password-stdin <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com      - cat ~/.docker/config.json      - REPOSITORY_URI=<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library  build:    commands:      - echo -e "[INFO] \t\t In the build phase..."      - echo -e "[INFO] \t\t Listing build folder directory..."      - ls -lrtha      - which docker      - ./build_script.sh      - docker build --build-arg JAR_FILE=build/libs/*.jar -t javaapp-aws:0.0.1 .      - docker tag <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1  post_build:    commands:      - echo -e "[INFO] \t\t In the post_build phase..."      - echo -e "[INFO] \t\t Build completed"      - docker images      - docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1      - printf '{"ImageURI":"%s"}' <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1 > imagedefinitions.jsonartifacts:  files: imagedefinitions.jsonHere the logs from the build phase[Container] 2022/10/28 21:08:38 going inside waitForAgent[Container] 2022/10/28 21:08:38 Waiting for agent ping[Container] 2022/10/28 21:08:41 Waiting for DOWNLOAD_SOURCE[Container] 2022/10/28 21:08:41 Phase is DOWNLOAD_SOURCE[Container] 2022/10/28 21:08:41 finished waitForAgent[Container] 2022/10/28 21:08:41 inside CopySrc[Container] 2022/10/28 21:08:41 CODEBUILD_SRC_DIR=/codebuild/output/src628607031/src[Container] 2022/10/28 21:08:41 finished CopySrc[Container] 2022/10/28 21:08:41 YAML location is /codebuild/readonly/buildspec.yml[Container] 2022/10/28 21:08:41 No commands found for phase name: install[Container] 2022/10/28 21:08:41 Setting HTTP client timeout to higher timeout for S3 source[Container] 2022/10/28 21:08:41 Processing environment variables[Container] 2022/10/28 21:08:41 Selecting 'java' runtime version 'corretto8' based on manual selections...[Container] 2022/10/28 21:08:41 Running command echo "Installing corretto(OpenJDK) version 8 ..."Installing corretto(OpenJDK) version 8 ...[Container] 2022/10/28 21:08:41 Running command export JAVA_HOME="$JAVA_8_HOME"[Container] 2022/10/28 21:08:41 Running command export JRE_HOME="$JRE_8_HOME"[Container] 2022/10/28 21:08:41 Running command export JDK_HOME="$JDK_8_HOME"[Container] 2022/10/28 21:08:41 Running command for tool_path in "$JAVA_8_HOME"/bin/* "$JRE_8_HOME"/bin/*; do tool=`basename "$tool_path"`;  if [ $tool != 'java-rmi.cgi' ];  then   rm -f /usr/bin/$tool /var/lib/alternatives/$tool \    && update-alternatives --install /usr/bin/$tool $tool $tool_path 20000;  fi;done[Container] 2022/10/28 21:08:44 Moving to directory /codebuild/output/src628607031/src[Container] 2022/10/28 21:08:44 Configuring ssm agent with target id: codebuild:e2e7f1b3-7716-4764-90cf-8ca556103f3a[Container] 2022/10/28 21:08:44 Successfully updated ssm agent configuration[Container] 2022/10/28 21:08:44 Registering with agent[Container] 2022/10/28 21:08:44 Phases found in YAML: 4[Container] 2022/10/28 21:08:44  INSTALL: 0 commands[Container] 2022/10/28 21:08:44  PRE_BUILD: 6 commands[Container] 2022/10/28 21:08:44  BUILD: 8 commands[Container] 2022/10/28 21:08:44  POST_BUILD: 5 commands[Container] 2022/10/28 21:08:44 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED[Container] 2022/10/28 21:08:44 Phase context status code:  Message: [Container] 2022/10/28 21:08:44 Entering execCommands[Container] 2022/10/28 21:08:44 Entering phase INSTALL[Container] 2022/10/28 21:08:44 Phase complete: INSTALL State: SUCCEEDED[Container] 2022/10/28 21:08:44 Phase context status code:  Message: [Container] 2022/10/28 21:08:44 Entering phase PRE_BUILD[Container] 2022/10/28 21:08:44 Running command echo -e "[INFO] \t\t In the pre_build phase..."[INFO]       In the pre_build phase...[Container] 2022/10/28 21:08:44 Running command echo -e "[INFO] \t\t Logging in to Amazon ECR..."[INFO]       Logging in to Amazon ECR...[Container] 2022/10/28 21:08:44 Running command aws --versionaws-cli/1.25.1 Python/3.8.10 Linux/4.14.281-212.502.amzn2.x86_64 exec-env/AWS_ECS_EC2 botocore/1.27.1[Container] 2022/10/28 21:08:44 Running command aws ecr get-login-password --region <ACCOUNTID> | docker login --username AWS --password-stdin <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.comWARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded[Container] 2022/10/28 21:08:45 Running command cat ~/.docker/config.json{    "auths": {        "<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com": {            "auth": "HEREISSTOREDTHELONGAUTHTOCKEN"        }    }}[Container] 2022/10/28 21:08:45 Running command REPOSITORY_URI=<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library[Container] 2022/10/28 21:08:45 Phase complete: PRE_BUILD State: SUCCEEDED[Container] 2022/10/28 21:08:45 Phase context status code:  Message: [Container] 2022/10/28 21:08:45 Entering phase BUILD[Container] 2022/10/28 21:08:45 Running command echo -e "[INFO] \t\t In the build phase..."[INFO]       In the build phase...[Container] 2022/10/28 21:08:45 Running command echo -e "[INFO] \t\t Listing build folder directory..."[INFO]       Listing build folder directory...[Container] 2022/10/28 21:08:45 Running command ls -lrthatotal 52K-rw-rw-r-- 1 root root   14 Oct 28 09:06 version.txt-rw-rw-r-- 1 root root   34 Oct 28 09:06 settings.gradle-rwxrwxr-x 1 root root 1.2K Oct 28 09:06 release.sh-rw-rw-r-- 1 root root   25 Oct 28 09:06 info.txt-rw-rw-r-- 1 root root 2.7K Oct 28 09:06 gradlew.bat-rwxrwxr-x 1 root root 5.7K Oct 28 09:06 gradlew-rwxrwxr-x 1 root root  120 Oct 28 09:06 get_tag.sh-rwxrwxr-x 1 root root  920 Oct 28 09:06 build_script.sh-rw-rw-r-- 1 root root  912 Oct 28 09:06 build.gradle-rw-rw-r-- 1 root root    0 Oct 28 09:06 application.yml-rw-rw-r-- 1 root root  692 Oct 28 09:06 README.md-rw-rw-r-- 1 root root  113 Oct 28 09:06 Dockerfile-rw-rw-r-- 1 root root  293 Oct 28 09:06 .gitignoredrwxr-xr-x 3 root root   18 Oct 28 21:08 srcdrwxr-xr-x 3 root root   21 Oct 28 21:08 gradledrwxr-xr-x 4 root root  278 Oct 28 21:08 .drwxr-xr-x 3 root root   17 Oct 28 21:08 ..[Container] 2022/10/28 21:08:45 Running command which docker/usr/local/bin/docker[Container] 2022/10/28 21:08:45 Running command ./build_script.sh[INFO]       Building component image for javaapp-aws, version 0.0.1[INFO]       Building java...Downloading https://services.gradle.org/distributions/gradle-7.2-bin.zip..........10%...........20%...........30%...........40%...........50%...........60%...........70%...........80%...........90%...........100%Welcome to Gradle 7.2!Here are the highlights of this release: - Toolchain support for Scala - More cache hits when Java source files have platform-specific line endings - More resilient remote HTTP build cache behaviorFor more details see https://docs.gradle.org/7.2/release-notes.htmlStarting a Gradle Daemon (subsequent builds will be faster)> Task :compileJava> Task :processResources NO-SOURCE> Task :classes> Task :bootJarMainClassName> Task :bootJar> Task :jar SKIPPED> Task :assemble> Task :compileTestJava NO-SOURCE> Task :processTestResources NO-SOURCE> Task :testClasses UP-TO-DATE> Task :test NO-SOURCE> Task :check UP-TO-DATE> Task :buildBUILD SUCCESSFUL in 1m 3s3 actionable tasks: 3 executed[Container] 2022/10/28 21:09:49 Running command docker build --build-arg JAR_FILE=build/libs/*.jar -t javaapp-aws:0.0.1 .Sending build context to Docker daemon  38.05MBStep 1/5 : FROM openjdk:8-jdk-alpine8-jdk-alpine: Pulling from library/openjdke7c96db7181b: Pulling fs layerf910a506b6cb: Pulling fs layerc2274a1a0e27: Pulling fs layerf910a506b6cb: Verifying Checksumf910a506b6cb: Download completee7c96db7181b: Verifying Checksume7c96db7181b: Download completee7c96db7181b: Pull completef910a506b6cb: Pull completec2274a1a0e27: Verifying Checksumc2274a1a0e27: Download completec2274a1a0e27: Pull completeDigest: sha256:94792824df2df33402f201713f932b58cb9de94a0cd524164a0f2283343547b3Status: Downloaded newer image for openjdk:8-jdk-alpine ---> a3562aa0b991Step 2/5 : ARG JAR_FILE ---> Running in 5f2f64973a5fRemoving intermediate container 5f2f64973a5f ---> 3323b6a04548Step 3/5 : COPY ${JAR_FILE} app.jar ---> 5f2e1367eabeStep 4/5 : ENTRYPOINT ["java","-jar","/app.jar"] ---> Running in d75a0dcd812bRemoving intermediate container d75a0dcd812b ---> 69e20eacbcbeStep 5/5 : EXPOSE 8080 ---> Running in acb0e7e5dcaeRemoving intermediate container acb0e7e5dcae ---> 275d152d55fcSuccessfully built 275d152d55fcSuccessfully tagged javaapp-aws:0.0.1[Container] 2022/10/28 21:09:55 Running command docker tag javaapp-aws:0.0.1 <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1[Container] 2022/10/28 21:09:55 Phase complete: BUILD State: SUCCEEDED[Container] 2022/10/28 21:09:55 Phase context status code:  Message: [Container] 2022/10/28 21:09:55 Entering phase POST_BUILD[Container] 2022/10/28 21:09:55 Running command echo -e "[INFO] \t\t In the post_build phase..."[INFO]       In the post_build phase...[Container] 2022/10/28 21:09:55 Running command echo -e "[INFO] \t\t Build completed"[INFO]       Build completed[Container] 2022/10/28 21:09:55 Running command docker imagesREPOSITORY                                                             TAG            IMAGE ID       CREATED                  SIZE<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws   0.0.1          275d152d55fc   Less than a second ago   142MBjavaapp-aws                                                            0.0.1          275d152d55fc   Less than a second ago   142MBopenjdk                                                                8-jdk-alpine   a3562aa0b991   3 years ago              105MB[Container] 2022/10/28 21:09:55 Running command docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1The push refers to repository [<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws]5c8eac8a352b: Preparingceaf9e1ebef5: Preparing9b9b7f3d56a0: Preparingf1b5933fe4b5: Preparingno basic auth credentials[Container] 2022/10/28 21:09:55 Command did not exit successfully docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1 exit status 1[Container] 2022/10/28 21:09:55 Phase complete: POST_BUILD State: FAILED[Container] 2022/10/28 21:09:55 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1. Reason: exit status 1[Container] 2022/10/28 21:09:55 exiting execCommands[Container] 2022/10/28 21:09:55 Expanding base directory path: .[Container] 2022/10/28 21:09:55 Assembling file list[Container] 2022/10/28 21:09:55 Expanding .[Container] 2022/10/28 21:09:55 Expanding file paths for base directory .[Container] 2022/10/28 21:09:55 Assembling file list[Container] 2022/10/28 21:09:55 Expanding imagedefinitions.json[Container] 2022/10/28 21:09:55 Skipping invalid file path imagedefinitions.json[Container] 2022/10/28 21:09:55 Phase complete: UPLOAD_ARTIFACTS State: FAILED[Container] 2022/10/28 21:09:55 Phase context status code: CLIENT_ERROR Message: no matching artifact paths foundWhat is strage is that the login command works as per above log. Paste here a snipped to focus on[Container] 2022/10/28 21:08:44 Running command aws --versionaws-cli/1.25.1 Python/3.8.10 Linux/4.14.281-212.502.amzn2.x86_64 exec-env/AWS_ECS_EC2 botocore/1.27.1[Container] 2022/10/28 21:08:44 Running command aws ecr get-login-password --region <ACCOUNTID> | docker login --username AWS --password-stdin <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.comWARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeededbut at the end get error[Container] 2022/10/28 21:09:55 Running command docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1 The push refers to repository [<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws] 5c8eac8a352b: Preparing ceaf9e1ebef5: Preparing 9b9b7f3d56a0: Preparing f1b5933fe4b5: Preparing no basic auth credentialsany suggestion?tksI've also tried locally on my laptop but got the same issue I'm experiencing within the pipeline.Attached to codebuild there are the following policiesEC2InstanceProfileForImageBuilderECRContainerBuildsAmazonEC2ContainerRegistryPowerUserFollowComment"

Answer: "0Your logs indicate this log output:[Container] 2022/10/28 21:08:44 Running command aws ecr get-login-password --region <ACCOUNTID>I would expect the region here, not the <ACCOUNTID>. Also, for that version of the AWS CLI, I use the aws ecr get-login command. While it says it is deprecated, I am still using it with that AWS CLI version. I also send the output to /dev/null to hide the output of the command in the logs.$(aws ecr get-login --no-include-email --region ${AWS_DEFAULT_REGION}) > /dev/null 2>&1CommentSharejimmyrayanswered 7 months ago"


Question: Authenticode certificate in AWS KMS

"Hi, we would like to use AWS Key Management Service to sign our binaries with our Authenticode certificate (both normal and EV).Is there a way to import our own key into KMS? We can import our own symmetric key, but to sign stuff we need asymmetric keys.FollowComment"

Answer: "0Hello,unfortunatly only symmetric keys are supported with imported key material.Imported key material is supported only for symmetric encryption KMS keys in AWS KMS key stores, including multi-Region symmetric encryption KMS keys. It is not supported on asymmetric KMS keys, HMAC KMS keys, or KMS keys in custom key stores.See also: https://docs.aws.amazon.com/kms/latest/developerguide/importing-keys.htmlCommentShareHeikoMRanswered 6 months ago"


Question: "Problem with reading jpg image in Sagemaker, imageclassification algorithm"

"Dears,I created a sagemaker job but I see the following error:[01/06/2023 11:29:21 ERROR 140033245599552] Customer Error: imread read blank (None) image for file: /opt/ml/input/data/train/s3://dbimgraces/output/af/af05.jpgAs far as I can see from my label file, the location is fine and the image also looks fine. Is any problem with the imread itself?FollowCommentArun Lokanatha 5 months agoCan you provide more information on how you are creating the estimator and passing the data. Looks like the data path the script is reading is wrong.Sharegreencomputing 5 months ago@arun tnx for your comment. I am not sure what you mean by estimator but I pass the data via S3 location with S3 data type S3prefix and s3 data distribution type:FullyReplicate. Actually, two channel are created for training/validation and two more channels for lables for training and validation.Share"

Answer: "0Not sure if you are using a sample code or writing on your own. If you can share the details, that would be helpful.If you are using Image Classification - MXNet, the SageMaker Image Classification algorithm supports both RecordIO (application/x-recordio) and image (image/png, image/jpeg, and application/x-image) content types for training in file mode, and supports the RecordIO (application/x-recordio) content type for training in pipe mode. However, you can also train in pipe mode using the image files (image/png, image/jpeg, and application/x-image), without creating RecordIO files, by using the augmented manifest format.Reference : https://docs.aws.amazon.com/sagemaker/latest/dg/image-classification.htmlCommentShareEXPERTAWS-User-Nitinanswered 5 months agogreencomputing 4 months agoDear @aws-user-Nitin, tnx for your comment. No I use  Image Classification - MXNet, the SageMaker Image Classification algorithm itself. I havent impelement any coding yet. I also tested with Pipe and (application/x-recordio and application/x-image ) and it still do not work. You can contact me for more demonstation via skype/teams greencomputinguae at g mail  dot c@mwith leave type for channel emtpy as optional:ClientError: Unable to initialize the algorithm. ContentType for channel 'train_lst' is empty. Please set content type for channel 'train_lst'. (caused by KeyError) Caused by: 'train_lst', exit code: 2with setting type for channel application/x-image:ClientError: ContentType must be specified for train channel., exit code: 2By setting to application/x-recordioClientError: Invalid RecordIO format. Please make sure that the RecordIO files are not corrupted. , exit code: 2By setting to application/x-image:ClientError: Invalid RecordIO format. Please make sure that the RecordIO files are not corrupted. , exit code: 2It is strange, if it mentions the field is optional, then why user force to set it?Share"


Question: Insufficient privileges for accessing data in S3 when running a lambda function to create a Personalize dataset import job

"I am trying to create a lambda function to automate the creation of a dataset import job in Personalize. I followed this guide: https://docs.aws.amazon.com/personalize/latest/dg/granting-personalize-s3-access.html#attaching-s3-policy-to-roleand kept getting the same error saying "Insufficient privileges for accessing data in S3".Here are the steps I took:Add AmazonPersonalizeFullAccess to my IAM userCreate a personalizeLambda role with 4 policies:AmazonS3FullAccessCloudWatchLogsFullAccessAmazonPersonalizeFullAccessAWSLambdaBasicExecutionRoleThis didn't work with the error above so I added this policy:PersonalizeS3BucketAccessPolicyCustom:{"Version": "2012-10-17""Id": "PersonalizeS3BucketAccessPolicyCustom","Statement": [    {        "Sid": "PersonalizeS3BucketAccessPolicy",        "Effect": "Allow",        "Action": [            "s3:*"        ],        "Resource": [            "arn:aws:s3:::<bucket-name>",            "arn:aws:s3:::<bucket-name>/*"        ]    },    {        "Effect": "Allow",        "Action": "sts:AssumeRole",        "Resource": [            "arn:aws:s3:::<bucket-name>",            "arn:aws:s3:::<bucket-name>/*"        ]    },    {        "Effect": "Allow",        "Action": "lambda:InvokeFunction",        "Resource": [            "arn:aws:lambda:<region>:<id>:function:create-personalize-model*",            "arn:aws:lambda:<region>:<id>:function:create-personalize-dataset-import-job"        ]    }]}Create a bucket policy in the S3 bucket that has the dataset files:{"Version": "2012-10-17","Id": "PersonalizeS3BucketAccessPolicy","Statement": [    {        "Effect": "Allow",        "Principal": {            "AWS": "arn:aws:iam::<id>:role/personalizeLambda",            "Service": "personalize.amazonaws.com"        },        "Action": "s3:*",        "Resource": "arn:aws:s3:::jfna-personalize"    },    {        "Effect": "Allow",        "Principal": {            "AWS": "arn:aws:iam::<id>:role/personalizeLambda",            "Service": "personalize.amazonaws.com"        },        "Action": "s3:*",        "Resource": "arn:aws:s3:::jfna-personalize/*"    }]}I still get the same error no matter how many times I've followed the guide. I would really appreciate it if someone could help figure out what I'm missing or did wrong.FollowComment"

Answer: "2Accepted AnswerAs noted in another answer it's not totally clear at what point in the chain you're getting the permission error here, but I notice you didn't mention your Personalize execution role.Your IAM user permissions should be mostly irrelevant (so long as you have access to see what's going on in Personalize and S3 and edit your Lambda).Your Lambda function will run under an execution role (personalizeLambda per the description) and will need access to whatever Personalize actions it needs to call. specifically, personalize:CreateDatasetImportJob sounds like the main one.At the point your Lambda calls CreateDatasetImportJob, it will need to pass a Personalize execution role (roleArn in API, role_arn in Python boto3).Personalize needs permission to assume this role (i.e. it should have personalize.amazonaws.com in the trust policy)This role is the one that needs access to S3, because Personalize assumes it to read your data from S3.I believe your Lambda role will also need iam:PassRole permission on the Personalize one, to permit your function to request actions under the Personalize role.From the description, it sounds like you might be forgetting to set up your Personalize role with S3 access, or just re-using the Lambda role ARN in that slot without adding the extra trust policy to allow Personalize service to use it as well as Lambda... But I could be wrong.You can find more info about this in the Setting up permissions and Importing bulk records sections of the Personalize Developer Guide.CommentShareEXPERTAlex_Tanswered a year agoAnna_G a year agoI agree with everything that Alex_T said.Just one more thought: I think you already did it from your comment above, but it is worth double checking that you added the S3 bucket policy to the bucket you are using to allow Personalize to read data from it ([Giving Amazon Personalize access to Amazon S3 resources] (https://docs.aws.amazon.com/personalize/latest/dg/granting-personalize-s3-access.html))ShareAWS-User a year agoThank you everyone for your help! I figured that I had all the steps correctly but was using the same IAM role for Lambda and for the role_arn in the Lambda function. As I separated these two, it worked for me. Thank you very much for all your suggestions!Share1Is the error coming from Lambda? The Lambda execution role will need permission to access S3 if it is trying to perform S3 operations. Does this help? https://aws.amazon.com/premiumsupport/knowledge-center/lambda-execution-role-s3-bucket/CommentShareguyanswered a year ago1For sure your lambda exécution has not enough privilegeEither your user is admin your lambda execution role need to have a policy to redefine the access level of executionCan you share or have a look at your lambda role policies ?CommentShareOmid Eidivandianswered a year ago"


Question: How to pass EMR Serverless PySpark entryPointArguments as variable

"I have an EMR Serverless PySpark job I am launching from a step function. I am trying to pass arguments to SparkSubmit from the entryPointArguments in the form of variables set in the beginning of the step function i.e. today_date, source, tuned_parameters, which I then use in the PySpark code.I was able to find a partial solution in this post here  however I am trying to pass variables from the step function and not the hardcoded argument i.e.. "prd".  "JobDriver": {          "SparkSubmit": {            "EntryPoint": "s3://xxxx-my-code/test/my_code_edited_3.py",            "EntryPointArguments": ["-env", "prd", "-source.$", "$.source"]          }        }Using argparse I am able to read the first argument "-env" and it is successfully returning "prd", however I am having troubles figuring out how to pass a variable for the source argument.FollowComment"

Answer: 0Managed to find an answer and explained the solution here:https://stackoverflow.com/questions/75572165/how-to-pass-emr-serverless-pyspark-entrypointarguments-as-variable/75601454#75601454CommentSharegeorge_ognyanovanswered 3 months ago


Question: Not Able to reimport SSL certificate in AWS certificate manager

I have got SSL certificate from GoDaddy and trying to reimport into certificate manager but it showing me error while trying i have used many documents from aws or other services still not able to put in right fields.I have.crt.pem.chained.crt.bundle.crtWhich one is need to be in respective field specificallyFollowComment

Answer: "0Did you get any errors when you tried to import?If you are out, would it be possible for you to share it with us?The following documents will help you troubleshoot by error message.https://repost.aws/knowledge-center/acm-import-troubleshootingCommentShareEXPERTRiku_Kobayashianswered 22 days agorePost-User-3502845 22 days agothis is the error which im getting while trying to add in CERTIFICATE PRIVATE KEY sectionThe certificate private key provided is not in a valid PEM format.ShareRiku_Kobayashi EXPERT22 days agoThat error is an error that occurs when the PEM format is not a valid PEM format as stated in the URL destination I shared.Please review the following documents and convert to a valid PEM format before registering.https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_server-certs.html#server-certificate-troubleshootingShare"


Question: Is Athena Iceberg is ~10 slower comparing to usual external table?

"We have time series data. And we always need only the last item for certain ID.When data is collected we usually extracting a single record by it's ID.We have about 5*10^8 records per day in ~1K batches. Monthly size of data is ~100TB compressed.I was thought that Iceberg will be a perfect candidate if we will use MERGE INTO for each new portion of data.However, after basic benchmark I figured out that Iceberg is about ~10x slower than usual external table. For the sake of simplicity I minimized my benchmark as much as I can. And it seams like my assumption is correct even with 12-rows-table.Here is the benchmark**Iceberg **CREATE  TABLE my_test (   my_partition bigint,   my_data string) PARTITIONED BY (  my_partition)LOCATION    's3://my-data/iceberg'TBLPROPERTIES (     'table_type' = 'ICEBERG',    'format'='parquet')insert into my_test values (1, 'Some data'),(1, 'Some data'),(1, 'Some data'),(1, 'Some data'),(1, 'Some data'),(1, 'Some data'),(1, 'Some data'),(1, 'Some data'),(1, 'Some data'),(2, 'Some data'),(2, 'Some data'),(2, 'Some data'),(2, 'Matching data')-- Run time: 3.586 secselect count(*) from my_test-- Run time: 8.293 secselect count(*)from my_testwhere my_partition = 2-- Run time: 7.485 secselect *from my_testwhere my_partition = 2 and my_data = 'Matching data'-- Run time: 7.766 sec**External table **CREATE EXTERNAL TABLE my_test_non_ice (   my_data string) PARTITIONED BY (     my_partition int)STORED AS PARQUETLOCATION 's3://ma-data/parquet'insert into my_test_non_ice values ( 'Some data', 1),( 'Some data', 1),( 'Some data', 1),( 'Some data', 1),( 'Some data', 1),( 'Some data', 1),( 'Some data', 1),( 'Some data', 1),( 'Some data', 1),( 'Some data' ,2),( 'Some data' ,2),( 'Some data' ,2),( 'Matching data' ,2)-- Run time: 1.422 secselect count(*) from my_test_non_ice-- Run time: 1.087 secselect count(*)from my_test_non_icewhere my_partition = 2-- Run time: 1.18 secselect *from my_test_non_icewhere my_partition = 2 and my_data = 'Matching data'-- Run time: 790 msAs you can see, every single query is much faster. The last one (the most interesting for me) is x10 slower.Our goal is to extract a single record in a 3 digits milliseconds time. Is it achievable at all with Iceberg tables?To me, counting 12 rows for 8+ seconds is a kind of strange.Any suggestions about what I'm doing wrong will be highly appreciated.FollowComment"

Answer: "1Accepted AnswerAn "ICEBERG" table works with snapshots and metadatas files.Globally, it distributes the files differently than the standard EXTERNAL tables, even duplicates the data, and thanks to a set of metadatas in json+avro knows how to query the data correctly. (so there is more complexity to read all these files than reading a simple EXTERNAL TABLE through a simple Parquet Serde (that only read the Parquet file internal metadata for queries))With a single example dataset, this is not too noticeable but you have to take into account that the use of S3 storage can quickly increase depending on the process with ICEBERG.If you perform a MERGE that applies many updates, and some deletes, you can quickly have the disk space on S3 increase by a factor, and the performance will be even less efficient (because data will be split into multiples snapshots/files).(However using the OPTIMIZE command ... can then bring the performance back to the initial state of the ICEBERG table, by combining files into a new one, and update metadata to ignore old files)There are advantages (much simplification of maintenance, query partitioned data without making explicit conditions on these partitions, ...) and disadvantages (less efficient performance in reading, storage space that increases rapidly, ...) to ICEBERG.I have not managed to see an automatic disk space optimization at work using VACUUM and OPTIMIZE personally though, I just know that it uses less files afterwards and that deleting files that are no longer used does not break my iceberg table.If ever the goal is to try to gain some performance, I would advise to set the compression of the PARQUET differently... But this would probably not change everything.(also if you already perform multiples delete/insert on your table, try an OPTIMIZE to fix performances)CommentShareDarkCenobyteanswered 4 months ago"


Question: Inexplicable error messages.

"I am new to AWS.  I tried installing AWS CLI and ran AWS configure where I entered my keys, etc.But when I type:  aws ssm help, or any service help, or just aws help, I get the following:'DOSKEY' is not recognized as an internal or external command,operable program or batch file.'more' is not recognized as an internal or external command,operable program or batch file.I have not typed either "DOSKEY" nor 'more' so I do not understand the error message.the AWS CLI version I installed is:  aws-cli/2.7.21 Python/3.9.11 Windows/10 exe/AMD64 prompt/offI am using windows 10.Can anyone help me?  All I want to do is run a python program I have put out in my S3 bucket from the CLI.FollowCommentPavithra B 4 months agoDid you find the answer ?Share"

Answer: "0Hello,I would say try the below steps for troubleshooting and also refer to the Windows section troubleshooting to find the steps.Let me know if it helps.Link- https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html -- [1]Also, refer to this link for S3 specific commands and best practices.Link- https://docs.aws.amazon.com/cli/latest/userguide/cli-services-s3-commands.html#using-s3-commands-prereqs -- [2]Hope this helps. Please click Accepted answer if the answer helped you.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 10 months ago"


Question: Call External Number while another call is on hold

"Hello;We are using CCP to handle calls in aws connect. When an agent gets a call, sometimes s/he needs some more information. And gets the call on-hold. And wants to make a call to external number.When s/he clicks Number pad, s/he can see numbers, but there is no call button. Only End call button. Agent can make a call to quick connect. But it is not always best solution as we cannot record every number as quick connect.Is there any way to call external numbers without registering them as quick connect first?Thank you..FollowComment"

Answer: "0Accepted AnswerFrom my end, I can also only see the End call button when I click on the Number pad directly during a call. Seems like a bug.. However, I found a workaround.Click on Quick connectsIn the Quick connects page, click on the Number pad button at the bottom of the screenThe Call button will now be enabled after entering any external phone numberClick CallCommentSharejoahnaanswered a year agobbolek a year agoHello Joahna, thanks for your response. Yes I was able to make a call with your method.I also created a ticket to AWS (https://github.com/amazon-connect/amazon-connect-streams/issues/557)Thank you again.Have a great dayShare"


Question: How to Get RootFolderId By UserId in DescribeUser Amazon Workdocs API?

"I have UserId, and I need the root folder id for that particular user.What I have done is, used the DescribeUser API of AmazonWorkdocsAnd provided id in UserIds Query.This API is working fine in postman.But when I'm using the same for My project (NetCore)it returns{{"Message": "Invalid SigV4 credentials."}}The code is provided below.var userRequest = new ESBMessage();                    userRequest.LocalPath = String.Format("/api/v1/users");                    userRequest.Http.Method = "GET";                    //userRequest.Http.Query.Add("organizationId", OrganizationId);                    userRequest.Http.Query.Add("userIds", userId);// UserId contains id of user                    SetAuthenticationHeader(userRequest);                    var driveList = _client.CallRestApi(userRequest, GetRequestURI(), true);FollowComment"

Answer: 0Accepted AnswerEncode UserIdCommentSharerePost-User-3543171answered 4 months ago


Question: Snapshot of EBS volume created from snapshot

"Is next snapshot of new volume incremental or full? For example:Create a new volume from existing snapshot.Create a snapshot from new volume.Is this snapshot incremental or full?It could be incremental because the volume was derived form a prior snapshot.Or it could be full because each new volume creates a new history.If incremental, then system keeps an incremental history of branches.If full, each creation of a new volume from a prior history creates a new, separate historyFollowComment"

Answer: "0Hi,In short, the first snapshot of each EBS volume is always full and the subsequent snapshots on that same volume are incremental [1]. I will illustrate this using your own example.Your question: "Create a new volume from existing snapshot. Create a snapshot from new volume. Is this snapshot incremental or full?"Answer: This snapshot is full.I hope that you find this answer useful. Please feel free to contact us if you need any further assistance.Links:[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSSnapshots.html#how_snapshots_workRegards,awstomasCommentShareAWS-User-3892169answered 4 years ago0Thanks for the quick helpful reply. steveCommentShareSteven A Frankanswered 4 years ago"


Question: FIRST send 'HTTP-like' command ?

"--Premise--(1) I want to send a command from a server on AWS to a machine tool connected to the company's LAN (within the Shimowada factory). (Automatic control)(2) There is a machine tool command sending server (RS232C (BSC procedure), Ether (sending commands without HTTP headers to machine tools) interfaces on the company LAN.(3) When the above command sending server receives an HTTP-like command in AWS (I think it's easy to implement), it sends the command to the machine tool via RS232C or Ether.ーーQuestionー In the above, the AWS server is expressed as "HTTP-like" to prevent primitive implementations such as sockets. However, I think that normal HTTP always sends a request from the client side and returns a response from the server side for the first time. Is there an implementation method (method) that is easy to implement and that commands are issued first from the server?I don't want to use sockets as much as possible.FollowComment"


Question: Codepipeline or API to automate the task of obtaining EC2 IP's after deployment

"I have a bunch of resources that get stood up as part of a regular deployment, i.e. worker EC2's. I need their IP's and to populate them into a config (But this is a post-deployment process) and it can become long/monotonous as its trivial work.Is there any api calls or any features in AWS I can use to automate this? I'm fine with shell/bash/python.Something in my head is to create a pipeline job that runs after the deployment pipeline, it queries or "scans" for the scope-named EC2's and then retrieves the IP's. Stores them for reference and then appends this to a config file I store in code commit. Possible?FollowComment"

Answer: "2You can use the AWS CLI - https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.htmlor SDKs in the language of your choice. For example, the Python SDK would be - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_instancesCommentShareEXPERTIndranil Banerjee AWSanswered 10 months ago"


Question: How to enable VPC Endpoint setting about multiple private subnet per AZ

I set one vpc and two private subnet in same AZ.I set ssm endpoint about one private subnet but I can't set ssm endpoint about different private subnet.I want to set ssm endpoint about two private subnet.How can I set ssm endpoint about two private subnet?FollowComment

Answer: "0You can deploy one SSM endpoint in one of the subnets in the same AZ, as described in https://docs.aws.amazon.com/vpc/latest/privatelink/concepts.html#architecture-diagramNote that clients in the other private subnet would still be able to talk to SSM through the endpoint. And by default, each interface endpoint can support a bandwidth of up to 10 Gbps per Availability Zone and automatically scales up to 100 Gbps. A bit more details are available here: https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html#considerations-interface-endpointsCommentShareweidianswered 8 months ago0You can specify more than one subnet in different Availability Zones to help ensure that your interface endpoint is resilient to Availability Zone failures.CommentShareEXPERTTushar_Janswered 8 months ago"


Question: AWS SSM adds \t and removes ' ocassionally

"very weird thing happening in AWS SSM Parameter Input. The pasted input below gets modified in AWS SSM Automation Documents. Can anyone tell me why this happens? It breaks my script and i can't work with this.here's my codeschemaVersion: "2.2"description: "Check HANA DB and stop it LINUX"mainSteps:- action: "aws:runShellScript"  name: "HANA_DB_STOP"  inputs:    runCommand:        - '#### errocodes'        - '#### 0   =   SCRIPT RAN OK'        - '#### 10  =   SCRIPT FAILED'        - ''        - '# stop hana db'        - 'echo "Stopping Hana DB"'        - 'sudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function Stop'        - ''        - '# check hana db state'        - 'echo "Checking if Hana DB is running."'        - 'HANADBSTATUS=`sudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function GetProcessList`'        - 'sleep 20'        - 'if [[ "$HANADBSTATUS" =~ "GRAY" ]]'        - 'then'        - '    echo "Hana DB is stopped."'        - '    exit 0'        - 'else'        - 'i=1'        - 'while [[ ! "$HANADBSTATUS" =~ "GRAY" ]] && [[ "$i" -lt 11 ]]'        - ' do'        - '     echo "Warning: HANA DB is running. Checking 10 times with 20 second intervalls until script aborts. This is check $i." '        - '     sudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function Stop'        - '     sleep 20'        - '     # check db state again'        - '     HANADBSTATUS=`sudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function GetProcessList`'        - '     ((i++))'        - '     if [ "$i" = 10 ]'        - '         then'        - '             echo "Error: retried $i-Times. Couldnt stop DB. Exiting Script."'        - '             echo "Script aborts with Error 0"'        - '             exit 10'        - '     fi'        - ' done'        - ' if [[ "$HANADBSTATUS" =~ "GRAY" ]]'        - '    then'        - '        echo "Hana DB is stopped."'        - '        exit 0'        - ' fi'        - 'fi'This is the output AWS gives meschemaVersion: '2.2'description: Check HANA DB and stop it LINUXmainSteps:  - action: 'aws:runShellScript'    name: HANA_DB_STOP    inputs:      runCommand:        - '#### errocodes'        - "#### 0\t=   SCRIPT RAN OK"        - "#### 10\t=   SCRIPT FAILED"        - ''        - '# stop hana db'        - echo "Stopping Hana DB"        - sudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function Stop        - ''        - '# check hana db state'        - echo "Checking if Hana DB is running."        - HANADBSTATUS=`sudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function GetProcessList`        - sleep 20        - 'if [[ "$HANADBSTATUS" =~ "GRAY" ]]'        - then        - '    echo "Hana DB is stopped."'        - '    exit 0'        - else        - i=1        - 'while [[ ! "$HANADBSTATUS" =~ "GRAY" ]] && [[ "$i" -lt 11 ]]'        - "\tdo"        - "\t\techo \"Warning: HANA DB is running. Checking 10 times with 20 second intervalls until script aborts. This is check $i.\" "        - "\t\tsudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function Stop"        - "\t\tsleep 20"        - "\t\t# check db state again"        - "\t\tHANADBSTATUS=`sudo /usr/sap/hostctrl/exe/sapcontrol -nr 00 -function GetProcessList`"        - "\t\t((i++))"        - "\t\tif [ \"$i\" = 10 ]"        - "\t\t\tthen"        - "\t\t\t\techo \"Error: retried $i-Times. Couldnt stop DB. Exiting Script.\""        - "\t\t\t\techo \"Script aborts with Error 0\""        - "\t\t\t\texit 10"        - "\t\tfi"        - "\tdone"        - "\tif [[ \"$HANADBSTATUS\" =~ \"GRAY\" ]]"        - '    then'        - '        echo "Hana DB is stopped."'        - '        exit 0'        - "\tfi"        - fiI know why \t gets added, but i still don't understand why and how it removes the ' in my scriptsFollowComment"

Answer: "0Hello,I am an AWS technical support engineer.I did raise this query with our engineering team and we would need more details to debug further.As our customer's data privacy is paramount to us. We are unable to ask your confidential data on repost. I would request you to create a Support Case with us to assist further.[+] Creating support cases and case management - https://docs.aws.amazon.com/awssupport/latest/user/case-management.htmlCommentShareSUPPORT ENGINEERSunilanswered 6 months ago"


Question: Is there any way to modify (change instance type) of my Windows Reserved EC2 Instance?

I have a t2.medium Windows based reserved instance. Now I want to upgrade it to t2.Large. As far as i have checked one can only change instance type of Linux reserved instances. Is there any way i can do it for my Windows based reserved instance?When i go to submit modification request it doesn't display any option to change instance type. Only Availability Zone modification is allowed.ThanksFollowComment

Answer: 0According to the documentation you cannot change a Reserved Instance if it's running windows.CommentShareRobert Loveanswered a month ago


Question: Purpose of Guardduty

"Hi Team,Im aware Guardduty is used for threat detection based on the API calls.Im struck where not all logs are appearing in the Guardduty.I have a control tower setup with organization enabled where all accounts are enabled with guard duty. Im not sure where im missing.Can someone explain where im lackingRegards,VijayFollowComment"

Answer: 0Try the following documentation:https://dev.to/aws-builders/configuring-security-services-with-aws-organizations-part-1-getting-started-3bcehttps://dev.to/aws-builders/configuring-security-services-with-aws-organizations-part-2-aws-guardduty-4g5CommentSharevtjeananswered 2 months agoVijaya Gokula Vasan 2 months agoI have aldready enabled Guard duty in an audit account and made that a delegated account.Now i dont see the logs or insights for that account in my audit account. Need to know what type of resource logs should be enabled to get the logs of all accounts.Share


Question: Get rawQueryString from ALB

"API Gateway is able to provide the rawQueryString from a request going to the API Gateway and then to the lambda, but I cannot find this attribute in the load balancer.For example, same request to API Gateway and ALBAPI Gateway proxy event{  rawQueryString: 'filters[siteId][$in]=85ff70e0-2870-4cf7-a693-f62f4367daa&filters[siteId][$in]=321',  queryStringParameters: {    'filters[siteId][$in]': '85ff70e0-2870-4cf7-a693-f62f4367daa,321'  }}ALB event forwarding to Lambda:{  queryStringParameters: { 'filters[siteId][$in]': '321' }}I am not able to find this in the documentation in the load balancer:https://docs.aws.amazon.com/elasticloadbalancing/latest/application/lambda-functions.html#multi-value-headersWhy is ALB modifying the request and the query string? I would expect to get the same request passed to the lambda and have the same raw query parameters.FollowComment"

Answer: "0Are you sure your API gateway is not doing some mapping for the response?ALBs does not modify the HTTP Requests received from clients - it adds additional headers to the existing request (e.g. X-Forwarded-For, X-Forwarded-Port, X-Forwarded-Proto, X-Amzn-Trace-Id ), and it can make forwarding decision based on the HTTP Request method, HTTP Header, Path, Query String or Source IP address values . But it does not modify any of the existing headers or the body of the request. Even while using "forward to" rule the parameters should be passed just like the redirection rule.However using a mapping template to override an API's request and response parameters and status codes, so check your API Gateway configuration and specifically the response from API gateway.https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-override-request-response-parameters.htmlCommentSharerePost-User-2054795answered 3 months agorePost-User-5709058 3 months agoWe are not modifying anything in the API gateway request. The property rawQueryString is given to the lambda from the API Gateway.This doesn't happen in the ALB. The lambda does not get the raw query parameters.Share0Looking at the query, I would like you to share some more information on the setup and the use case you are trying to achieve.I understand that, you were able to successfully pass the query string parameters from the API to the Lambda function when you tried the setup as below.API gateway -> Lambda. Am I right?And Is your current setup or the setup you are trying to achieve is as below:API gateway -> ALB -> Lambda? If yes, please confirm.If your setup is something different than the above one's, please share more information on the same.Meantime, you can also have at some of the discussions below related to the Client -> ALB -> Lambda in the documentation:"If requests from a client contain headers with multiple values or contains the same header multiple times, or query parameters with multiple values for the same key, you can enable support for multi-value header syntax. After you enable multi-value headers, the headers and query parameters exchanged between the load balancer and the Lambda function use arrays instead of strings. If you do not enable multi-value header syntax and a header or query parameter has multiple values, the load balancer uses the last value that it receives."[+]https://docs.aws.amazon.com/elasticloadbalancing/latest/application/lambda-functions.html#multi-value-headers[+]https://stackoverflow.com/questions/60997930/serverless-alb-event-does-not-have-query-params-as-a-list-with-multivaluequerystCommentShareSUPPORT ENGINEERZacchae_Janswered 3 months ago"


Question: Issue with copying RDS Multi AZ Cluster snapshot to different region

"Hello,Is there a way to copy an RDS Multi AZ Cluster snapshot to a different region?When I try using the cli I get the error: "An error occurred (InvalidParameterCombination) when calling the CopyDBClusterSnapshot operation: You can't copy a cross-Region Multi-AZ DB cluster snapshot."aws rds --profile ${PROFILE} copy-db-cluster-snapshot --region ${DEST_REGION_2} --source-db-cluster-snapshot-identifier ${SNAP} --target-db-cluster-snapshot-identifier ${SNAP}The UI gives almost the same error.Thanks!FollowComment"

Answer: "0Thank you.  Is there any way to know if items on that list will be supported and when?CommentSharerePost-User-1339259answered 3 months agokentrad EXPERT3 months agoThis is something to discuss with your AWS Solution Architects and/or Technical Account Manager.Share0Assuming this is Aurora:Could it be the combination of source and destination regions? See: Cross-Region snapshot copying.Also, you cannot copy across regions and accounts with one step.CommentShareEXPERTkentradanswered 3 months ago0The cluster is a mysql db.The source region is us-east-1 and the destination is us-west-2.CommentSharerePost-User-1339259answered 3 months ago0Looks like copying a snapshot of a RDS Multi-AZ DB Cluster is not supported. Limitations for Multi-AZ DB clusters.Multi-AZ DB clusters don't support the following features:* Copying a snapshot of a Multi-AZ DB clusterCommentShareEXPERTkentradanswered 3 months ago"


Question: A browser launched from EC2 Win Server running in private subnet cannot open web site hosted by IIS on EC2 Win Server in public subnet.

"My simplified network architecture is as follows:VPC has a public and private subnets.Public Subnet is connected to Internet Gateway.Private subnet is connected to Internet through NAT.Public EC2 Win Server has public IP4: 13.22.45.23 and private IP4: 10.0.10.15. The security group allows incoming traffic from private EC2 Web server on ports 80, 1433 and 433.Security group of private EC2 Win Server allows incoming traffic from the public EC2 Web server on ports 80, 1433 and 433.Public website  https://MyWebSite.com/MyWebApplication has binding to port 443 and hostname associated with EC2 Win Server's public IP4 address. The hostname is registered in AWS Route 53, i.e. MyWebSite.com is associated with 13.22.45.23.What I tried:Test #1 - Successfully opened website in my laptop typing  https://MyWebSite.com/MyWebApplication  in Chrome browser. This proved that website was working and was accessible from Internet.Test #2 - RDC'd to private EC2 Win Server and successfully opened random Internet websites in Chrome browser. This proved that private EC2's Internet connection works OK. Note, private EC2 is connected to Internet through NAT installed in public subnet.Test #3 - RDC'd to private EC2 Win Server and successfully opened Default website (port:80) hosted by public EC2 Win Server IIS. I opened it using private IP address of public EC2 instance, i.e. typing http://10.0.10.15 in Chrome. This proved that HTTP requests go through OK from private to public subnet.Test #4 - RDC'd to private EC2 Win Server and tried to open https://MyWebSite.com/MyWebApplication. The browser displayed a message 'This site can’t be reached. MyWebSite.com took too long to respond'Test #5 - Tried the same test #4 above but using public IP4 address of the public EC2, i.e. using URL https://13.22.45.23/MyWebApplication. The result was the same like in test #4.Test #6 - Tried the same test #4 above but using private IP4 address of the public EC2, i.e. using URL https://10.0.10.15/MyWebApplication. The result was the same like in test #4.The problem is the https://MyWebSite.com/MyWebApplication exposes Web API endpoints that I need to call from the private subnet.  Because all calls failed, I did the above connection tests.I don't understand why I can open https://MyWebSite.com/MyWebApplication from outside of my VPC but can't from the private subnet. In fact, as I mentioned above, I can can open any https websites from the private subnet except MyWebsite.com. I wonder what is special about it?I must admit that the real VPC looks more complex than I described in the beginning. It has Load Balancers associated with Cloud Watch, Event Bridge, Private Link and API Gateway. None of the load balancers have listeners  on port 443 though.Can anyone please advise any directions where should I look at for a solution ?Thank you.FollowComment"

Answer: "0Based on the information provided, it seems that the issue is related to the routing of traffic between the private and public subnets in your VPC. Specifically, it appears that traffic from the private subnet is not being properly routed to the public subnet where the Web API hosted on your public EC2 instance is located.One possible solution to this issue could be to check the routing table for the private subnet to ensure that it is properly configured to route traffic to the public subnet. Additionally, you should check the security group rules for the private and public subnets to ensure that they are configured to allow traffic between the subnets on the necessary ports (443 in this case).Another possible issue could be the load balancer, check the listeners associated with it, and make sure it's not blocking port 443 traffic.You should also check your Network ACLs (NACLs) for the public and private subnets to ensure that they are allowing traffic between the subnets on the necessary ports.It's also worth checking the VPC peering connections and make sure it's not blocking the traffic.CommentShareVarun Mittalanswered 4 months agorePost-User-4543988 4 months agoThank you, Varun. Have checked everything that you advised: route tables, SG's, load balancer listeners and targets, ACL's, Private Link Service Endpoint. Also, I did Reachability Analyzer successful test between two EC2 instances. All looks correct but connection still does not work.I've done another test:I created a new website in Public EC2 IIS with binding to port 443 but with empty  hostname. I tried to reach the new website from private EC2  using Public EC2 private IP4 address, i.e. https://10.0.10.15/MyNewWebSite. This worked.I still don't understand why  https://MyWebSite.com/MyWebApplication does not work in private EC2.It looks like hostname MyWebSite.com associated with public EC2 IP4 address is the problem. But I don't understand why.Share0Try running Reachability Analyzer, int's in the VPC page on the AWS console, you can use it to analyze the path between the two EC2 instances and see where it's blocked.CommentShareDror Helperanswered 4 months agorePost-User-4543988 4 months agoThank you Dror Helper. Have used Reachability Analyzer between Public and Private instances through port 443 and the Reachability Status is Reachable.Share"


Question: Cloudwatch event rule triggered by AWS ECR event

"Hello.I am trying to setup CI/CD with AWS codepipeline and now I am stuck with pipeline autostart.Looks like cloudwatch does not detect ECR events so does not start a pipeline.Target and role configured correctly, but in access advisor for role I don`t see any role invocations.Region us-west-2.Here is event pattern that I use:{"detail": {"eventName": ["PutImage"],"requestParameters": {"imageTag": ["service.develop.latest"],"repositoryName": ["repository"]}},"source": ["aws.ecr"]}I can see PutImage events in cloudtrail but this rule does not work. Any help appreciated, thanks.FollowComment"

Answer: 0Magically fixed itself. Looks like an internal AWS problem.CommentShareteamfighteranswered 4 years ago


Question: AWS SSM Patch Manager

"I have question around AWS SSM Patch manger custom Patch baseline. I create a custom patch baseline for Windows servers and add to Patch group, so far good. I tried to use this custom patch baseline in Maintenance Window task, Couldn't find anything. Only option for Run_Command is AWS-RunPatchBaseline which is default, not the custom that I create.FollowComment"

Answer: "1Hello, you are correct -- when you use the document AWS-RunPatchBaseline, you target managed nodes using instance IDs, resource tags, or resource groups. The SSM Agent on each instance makes it's own determination for which baseline it should use based on tags added to itself.You can either set the baseline as default for the OS or you would want to add Patch Group tags to the instance and the appropriate baseline.More information can be found in this documentation topic:https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-patch-patchgroups.htmlCommentShareErik_Wanswered a year ago0Yes I did. Looks like answer is -- When you run AWS-RunPatchBaseline, you can target managed nodes using their ID or tags. SSM Agent and Patch Manager then evaluate which patch baseline to use based on the patch group value that you added to the managed node. If this is true, I am good with this set up.CommentShareAWS-User-8123009answered a year agoGK a year agoGood hear thatShare0HelloDid you Set the patch baseline as default https://docs.aws.amazon.com/systems-manager/latest/userguide/set-default-patch-baseline.htmlThank YouGKCommentShareGKanswered a year ago"


Question: Create another forum for Lumberyard unrelated Topics (e.g. Amazon GameLift)

"I would suggest to create another Amazon forum for Lumberyard unrelated topics like e.g. Amazon GameLift.The forum head picture says Lumberyard by amazon. On the otherside the URL is https://forums.awsgametech.com/. So I get it, the AWS GameTech division is maintaining both, Lumberyard and completely separate AWS cloud related projects. Just from the user perspective, there is little overlap.FollowComment"

Answer: "0Thanks for the feedback.I will bring it to the attention of the GameLift and Lumberyard teams.CommentSharerePost-User-5332620answered 3 years ago0I agree, as it seems GameLift related topics are frequently related to projects built with game engines other than Lumberyard (this was the same in the previous forums iirc). Although they are tagged as GameLift, still feels as if the engine is somewhat undercut by this.CommentSharerePost-User-2743711answered 3 years ago0I agree with guys we need separate forum, what will be mainly oriented on game engine itself.Aslo it's will be nice if you LyTeam create something like unity store, but for LY's where we can upload or download free/pay assest and gemsCommentSharerePost-User-2514050answered 3 years ago0This is just my opinion, but I am not sure if there are enough unrelated topics to warrant a separate forum. However, I do agree that the forum head picture shouldn't say lumberyard, perhaps a AWS Game Tech logo would be more appropriate.CommentSharerePost-User-5100160answered 3 years ago0Sure @REDACTEDUSERCommentSharerePost-User-2743711answered 3 years ago"


Question: Transit Gateway attachment cost to VPC and subnets

"As perhttps://aws.amazon.com/transit-gateway/pricing/"Price per AWS Transit Gateway attachment" i.e. 5 or 7c /hrhttps://docs.aws.amazon.com/vpc/latest/tgw/tgw-vpc-attachments.html"When you attach a VPC to a transit gateway, you must specify one subnet from each Availability Zone to be used by the transit gateway to route traffic. Specifying one subnet from an Availability Zone enables traffic to reach resources in every subnet in that Availability Zone."Does this mean for a VPC with 3 AZs/Subnets , customers would pay 3 x 7 = 21 c/hr for a VPC ?FollowComment"

Answer: 0Accepted AnswerThe TGW attachment (tgw-attach-xxxxxxxxx) irrespective of the number of AZs for which it is configured is charged on the unique attachment id  not the individual AZs. So no difference if you have 1AZ versus when you have 3AZs.CommentSharesumitawsanswered 3 years ago


Question: API Gateway websockets don't stay alive

"My websocket connection works fine until the keepalive timeout triggers on the client, and then it closes despite messages being sent successfully in both directions.  I have tried with both .NET ClientWebSocket and python websockets.  Does API Gateway not handle websocket pings?  Are there any settings related to this?  I could set the client keepalive really high but that seems wrong in the case of actual network issues.Playing around with the python websocket, the connection closes as soon as it sends a ping:import asyncioimport websocketsasync def ws():    print('connecting...')    async with websockets.connect('wss://my-websocket-url',            ping_interval = 1, ping_timeout = 5) as socket:        print('connected')        message = await socket.recv()        print(f"< {message}")        print('exiting')asyncio.run(ws())If no messages are received, the above code exits after 1 second with:asyncio.streams.IncompleteReadError: 0 bytes read on a total of 2 expected bytesThe above exception was the direct cause of the following exception:websockets.exceptions.ConnectionClosed: WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reasonThis implies to me that AWS is responding to the ping with something weird?  Interestingly, if I don't recv() at all then the connection does not fail.  Why would receiving messages conflict with websocket pings?Edited by: F. Spiral on Apr 3, 2019 12:21 PMEdited by: F. Spiral on Apr 3, 2019 12:25 PMEdited by: F. Spiral on Apr 3, 2019 12:29 PMFollowComment"

Answer: "0I have discovered that this happens in the us-west-2 region but not in us-east-1.  Those are the only regions I tried.  Steps to reproduce:Create a new lambda function (python 3.7), accept all the defaults and publish it.Create a new API Gateway websocket API.Route the $connect function to your lambda function.Deploy the API.Run my python code from above using your websocket URL.For me, in us-west-2 the connection reliably closes after ping_interval seconds but in us-east-1 it stays open.  Am I just confusing myself or is something misconfigured on Amazon's side?Edited by: F. Spiral on Apr 3, 2019 2:00 PMCommentShareF. Spiralanswered 4 years ago0Well, this problem seems to have fixed itself for now.  This was happening for about a day and then it suddenly started working fine.CommentShareF. Spiralanswered 4 years ago0This seems to be happening again, but now I'm using C# and System.Net.Websockets.ClientWebSocket.  The KeepAliveInterval doesn't seem to matter this time.  The websocket is disconnecting with status Aborted after 30 seconds of no activity, even if KeepAliveInterval is set to 5 seconds.  How do I keep an API Gateway websocket alive without inventing random data to send?  Is anyone else using ClientWebSocket and having this problem?CommentShareF. Spiralanswered 3 years ago0Okay, this latest issue was actually my fault - apparently if you have a timeout on a ReceiveAsync() or SendAsync() then it aborts the whole websocket, by design, so my 30s timeout was making it seem like a keepalive issue.  False alarm!CommentShareF. Spiralanswered 3 years ago"


Question: [Redshift] How to find table ID associated with DDL statement

How can I find the table id modified by a DDL statement in Redshift? STL_DDLTEXT doesn't have a column associated with table id. Any help would be appreciated.FollowComment

Answer: "0from https://medium.com/@alex_handley/aws-redshift-handy-query-to-get-all-table-create-statements-for-a-schema-fcc0eaac03c1 something like the following should provide that information. Note that user generated schema's are not in pg_table_def search path by defaultSET SEARCH_PATH TO <something>;SELECT ddlFROM admin.v_generate_tbl_ddlJOIN pg_table_def ON (admin.v_generate_tbl_ddl.schemaname = pg_table_def.schemaname ANDadmin.v_generate_tbl_ddl.tablename = pg_table_def.tablename)WHERE admin.v_generate_tbl_ddl.schemaname = '<something>'GROUP BY admin.v_generate_tbl_ddl.tablename, ddl, "seq"ORDER BY admin.v_generate_tbl_ddl.tablename ASC, "seq" ASC;from https://github.com/awslabs/amazon-redshift-utils/blob/master/src/AdminViews/v_generate_tbl_ddl.sql will provide you with detailed table information including distkey, sortkey, constraints, not null, defaults, etc.CommentShareleftbrainstuffanswered 4 months ago"


Question: The best AWS service for sending SMS via REST API calls

I would like to use an AWS service. I have several thousand phone numbers. This application must be able to send WEB REST API calls to a service (AWS) to send SMS. I had thought of SNS. Is this the right service for sending SMS to non verified destination phone numbers?  Thank youFollowComment

Answer: "1SNS is the right service where you can make an API call to publish a message to a topic and subscribers to that topic can be SMS clients. However, you will need to go through a sandbox phase before getting full access to the service. See Mobile text messaging (SMS).CommentShareEXPERTkentradanswered a year ago1Hi.You can use SNS to send SMS messages to unidentified users.There is a caution. The following pages provide notes on sending unsolicited messages within best practices. Please comply with this.https://docs.aws.amazon.com/ja_jp/sns/latest/dg/channels-sms-best-practices.htmlAnd how do you manage thousands of phone numbers?In addition to simply sending, you may be able to take advantage of Amazon Pinpoint for scenarios that require segmentation or workflow.CommentShareEXPERTiwasaanswered a year ago"


Question: "Network Load Balancers, Elastic IPs, and Fargate Services + Tasks?"

"Hi,I'm new to AWS and trying to get some of our on-prem VMs migrated over to Fargate.  From what I've read, I need to use a Network Load Balancer since I want to setup an Elastic IP on the NLB which can be delegated via an A record from our on-prem DNS (Route 53 isn't an option right now).In ECS, I have a Task Definition for my container, which runs Nginx, listening on TCP 80 and 443. Also in ECS, I've setup a Cluster and two Services; One Service for TCP 80, and another Service for TCP 443.  Each of these services have at least one Task defined which spins up the Nginx container.When I define the NLB, I create two Listeners/Target Groups. One for TCP 80 and the other for TCP 443. When defining the Listeners, I assign the EIP to each listener.  The NLB has an A record associated with it named something like my-net-lb-xxxxxxx.elb.us-east-1.amazonaws.com. There are also two public IP addresses associated with the A record but here's where it gets weird.If I do an nslookup on NLB A record, I only see one of the EIPs attached to it. Likewise, I can point a browser at that EIP and I can reach the Fargate Task. The second IP seems to be orphaned somewhere. It's setup as a listener, but doesn't show up anywhere except under the other AZ for the NLB. Pointing a browser at that orphaned IP goes nowhere.I was expecting a Round Robin DNS setup on the NLB A record so if one of the EIPs is unroutable because a Task is down, the client has a chance to refresh and roll-over to the other IP address.I must be missing something. I've tried finding specific documentation on this setup but everything I've found so far is either incomplete, or so old it's irrelevant.Thanks for any insight, and apologies this is so longFollowComment"

Answer: "0Accepted AnswerThere are 2 types of DNS for NLB - regional and zonal. you should use regional DNS names and AWS will automatically resolve to zonal DNS in a round robin fashion; the reason is if you are using zonal DNS and if there is AZ level outage (unlikely but possible) there wont be any failover therefore its recommended to use Regional DNS of NLB for connectivity. See more information on this topic hereIf you are using a single NLB deployed across multiple Availability Zones, you can use the Availability Zone-specific DNS names provided by the NLB in your DNS configuration. For example, suppose you have a workload, www.example.com, that operates behind an NLB in us-west-2a, us-west-2b, and us-west-2c. It has a Regional AWS provided DNS A record of my-load-balancer.elb.us-west-2.amazonaws.com, as well as zonal records for each Availability Zone such as us-west-2a.my-load-balancer.elb.us-west-2.amazonaws.com.CommentShareEXPERTTushar_Janswered 7 months ago0x90 7 months agoThanks for that answer and the link for the DNS explanation! The second EIP showed up in DNS a couple days later. I'm not clear why the second EIP took so long to propogate.Share"


Question: cloudwatch bill with lambda is high. Any insights into why this is the case and how to control costs?

"A customer is concerned about their CloudWatch bill.They are using lambda, and logging to CloudWatch.Reviewing their costs, it seems like the following metric is causing the high bill:"GetMetricData"Both "CW:GMD-Metrics ($)" and "CW:MetricMonitorUsage"Has anyone else seen high bills rates with these metrics and lambda?Any insights are appreciated.FollowComment"

Answer: 0Accepted AnswerIf you need a indepth analysis of what is happening we need to do a    API Caller Analysis for GetMetricData. One thing to keep in mind is that GetMetricData is billed by number of metrics requested and not by API calls.1 ) You need to identify the resources2)  look at what calls are being made3) then Metrics vs requested per callDoes the customer using any monitoring service (Say - Data  Dog) .I would suggest to open a support case . So that engineer will be able to  go deep dive to the API calls and will give you the exact metricsCommentShareAneesh_Vanswered 4 years ago


Question: Copy data from EFS to EFS in another AWS account.

"Hi Everyone,I'm looking for some guidance on copying EFS data from one AWS account to another AWS account.From what what I understand from reading other posts, I need to choose the source as NFS and the destination as EFS.How do I go about referencing the EFS share in the destination location? It won't show in the drop-down list as it resides in another AWS account which makes sense.Do I need to create a peering connection between the two VPC's? Or is there something else i'm missing?My agent is activated via a public endpoint if that matters any.Thanks for any help provided,Craig.FollowComment"

Answer: "1Hi Craig,VPC peering is not needed in this case.Given a source account A, and a destination account B, you can:1- Run an EC2 agent in account A, in the AWS Region where your source EFS resides, in an Availability Zone that has a mount target for your EFS.2- Activate the agent against account B, in the destination region (if you are using the AWS Management Console, log into account B, and provide the IP of your agent when performing activation).3- In account B, in the destination region, create a task with a source NFS location, using your agent, and specifying the source EFS address. The destination would be an EFS location.Regards,OlgaEdited by: olga-aws on Jul 29, 2020 11:50 AMCommentShareAWS-User-9583871answered 3 years agodpapageo 20 days agoOne question here,Did you set up an endpoint in the destination subnet? If yes how did you make it visible during the agent creation in the source subnet?Share0It all makes sense now :)Thank you so much Olga. I was successfully able to replicate my EFS share over to the second account using the advice you posted.Thank you very much for all your help.Craig.CommentSharecraigthreadgoldanswered 3 years ago"


Question: Can you validate an ACM public certificate using a domain record in a Route 53 private hosted zone?

"I'm trying to create an ACM public certificate and then validate it using a domain record in a Route 53 private hosted zone, but the setup isn't working. I don't see an option to specify the private hosted zone ID in the validation request. Is it possible to validate an ACM public certificate using a domain record in a Route 53 private hosted zone?FollowComment"

Answer: "0Accepted AnswerIt's not possible to validate an ACM public certificate using a domain record in a Route 53 private hosted zone.When you request an ACM public certificate using DNS validation, ACM provides a CNAME record that you must add to your DNS configuration to validate your ownership of the domain. Because anyone can create a private DNS zone and put records on it under any domain name, being able to make a change in a private DNS zone doesn't prove public ownership of the domain.For more information on DNS validation, see DNS validation in the ACM user guide.CommentShareEXPERTgavinmcanswered 5 years ago"


Question: Problems using the GameLift SDK with custom protobuf version

"I am trying to implement the latest C++ GameLift SDK with my custom game engine and ran into some problems because I use protobuf for my own networking and the versions are different.There is a USE_SYSTEM_PROTOBUF make setting which I am using to provide my own version of it, the problem is that it is a different/newer version and the included generated header file is not compatible with it.What I need to fix this is the original sdk.proto file from which these are generated.Also I am including GameLift into my own cmake scripts using ExternalProject_Add and doing this sioclient/sio_client.h which is included from Network.h can not be found because the include path that is used does include the "sioclient" part.For now it seems to be working, after extracting my own version of sdk.proto from the headers and changing the path, but this will just break again with any update.So I guess what I'd like to see for a next version is: Fixed include path and sdk.proto as part of the SDK and the proto header and cpp files generated from it at build time.FollowComment"

Answer: 0Apologies for the inconvenience and thanks for the feedback.I have passed this on to the GameLift service team for their attention.CommentSharerePost-User-5332620answered 4 years ago


Question: Update DynamoDB data is not automatically updating IOS App with AWS Amplify Datastore and API

"Hello,I have created an IOS App using AWS Amplify. I have the app configure with Amplify with all the necessary AWS Plugins. I am having an issue with updates to DynamoDB not updating in the IOS App. I am using Amplify.DataStore.observeQuery to check for any changes to the database. If I change some data in the "content (datastore)" of Amplify Studio the data on the IOS App does update. My issue is when I update the data in DynamoDB, the IOS app will not update until I run Amplify.DataStore.clear() and then Amplify.DataStore.observeQuery() functions. I am sending data to DynamoDB using MQTT with a wifi module and I would like to see real time results of the data being sent to DynamoDB on the IOS App without manually pressing any buttons. I believe I am close, but I must be missing something to get it automatically update instead of manual pressing buttons to run the clear and observeQuery functions. Feel free to add comments to this post with any questions. Thank you all in advance.FollowComment"


Question: Instance is unresponsive after "cloning" it from the good working instance

"I have the instance 'i-01' in us-west2. This is a Windows Server 2016 one. It works fine for years. I can access it via RDP, via Connect in AWS Console. It has SSM and Cloud Watch Agent services installed and running.I created AMI ami-01 from it and launched a new instance i-02 in the same region from this ami-01. But:One of the 2 Status Checks is Instance reachability check failedI can't RDP to itI can't connect via Session Manager in AWS ConsoleI can't connect via EC2 Serial Console. It gives me Still waiting for meta-data accessibility...What I tried to do:Reboot i-02Reboot the initial i-01 to verify it will be accessible after rebooting. And it is.Remove i-02 and create a new instance from ami-01. The same result - inaccessible.Copy ami-01 to us-east1 region and create the instance there. The same result - inaccessible.Run CLI  command aws describe-instances and compare JSONs for both i-01 and i-02. No significant changes other than IP addresses and IDs.The question is: How can I understand what is going on inside this instance and why it's inaccessible?FollowComment"

Answer: "1Accepted Answer"Waiting for the metadata service"CommentShareEXPERTkentradanswered a year agoAlex a year agoAccepted this answer. The reason was the static IP address on Network Interface. Configuring it to get IP address from DHCP fixed the issue.Share0Hello,I'd like to suggest to first make sure you upgrade all AWS PV Drivers, ENA and/or NVMe and upgrade to Ec2Launch v2 (1), then perform sysprep (2) to create a new reusable AMI. Drivers would help so that the new Instance is compatible with any Instance type, while sysprep will remove unique identifiers so the AMI can be deployed anywhere.Keep in mind that sysprep would disjoin the Instance if it is part of any domain, so you will be required to have local credentials to log back in and join it back to the Domain.I hope this helps. Good luck!References(1) Migrate to latest generation instance types - https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/migrating-latest-types.html(2) How can I use Sysprep to create and install custom reusable Windows AMIs in Amazon EC2? - https://aws.amazon.com/premiumsupport/knowledge-center/sysprep-create-install-ec2-windows-amis/CommentShareSUPPORT ENGINEERJosueAWSanswered a year ago"


Question: Best AWS Service to Put PySpark / ML Code into Production

"I am very new to MLOps and seeking guidance on how to put a piece of ML python code into production on AWS.Here are my requirements:Model is run once a weekPick up very large data files from S3Models are stored in S3 and will need to be loaded from there as wellData processing is done in PySparkProcessed data is then converted into Pandas for further processing before being fed into modelsModels are typically SkLearn or XGBoostModel results should be stored back in S3Any idea of what service is best for this? Again, I have very limited experience outside of writing ML models in python, so anything that is simple where I am not managing or worrying about infrastructure is preferable. However, I will do what I must to put the code into production.FollowComment"

Answer: "0From what you have described, Amazon SageMaker is your best bet. It is a managed service (meaning you dont have to worry about managing the platform) and has built-in support for feature engineering (data processing) in PySpark through SageMaker processing. You can find an example here. It natively integrates with S3, which allows you to fetch data from S3 at runtime, and it also saves model artefacts to an S3 bucket (you can specify which bucket and prefix). Commonly used frameworks/models like XGBoost and Sklearn are supported and you can run your model training tasks using SageMaker managed training. Finally, to make the code production ready, you can use SageMaker Pipelines, which is the MLOps tool that can take care of moving your code to the production environment.CommentShareWajahat_Aanswered 2 months agoAlbz 2 months agoThank you, I will investigate SageMaker Pipelines. Why would you not recommend Amazon EMR, is it overkill for this use case?ShareWajahat_A 2 months agoIf your intent is to train a machine learning model, SageMaker is a better fit as it provides tooling for all steps of the model development lifecycle. You have access to features like SageMaker Model Monitoring (to monitor your model in production), SageMaker Debugger, and a number of model inference options.ShareAlbz a month agoI'm starting to dig into SageMaker pipelines and watch tutorial videos. Before I get too far, I want to ask... is it possible to create a highly custom modeling pipeline? For example, I have my own custom cross validation class, where hundreds of models are trained and tested, and my own functions which stops training when very niche custom criteria are hit. I'd like to save this class as a pickled object which is then picked up in production from S3 and fed the same set of features.Is SageMaker Pipelines still the best route for me? Any suggestions would be appreciated, including if I should rethink how I'm doing things.Share"


Question: FTP user name & password not work when using FileZilla

"when I try to log in to FileZilla by using credentials that I created in FTP in an EC2/Lightsail (Ubuntu) instance, it was not connected. otherwise, I use a keypair, it will work.Please give a reason and give a proper procedure for creating FTP Server by using an EC2/Lightsail.Thank You in AdvanceFollowComment"

Answer: "0which package are you using for ftp on lightsail instance - vsftpd? or anotherplease check the configuration firstfor example if the package is vsftpdUncomment the line local_enable=YES to allow local users to log in.Uncomment the line write_enable=YES to allow users to write to the server.Uncomment the line chroot_local_user=YES to lock users in their home directories.CommentShareEXPERTsdtslmnanswered 2 months agoPavithran 2 months agoi'm using vsftpd packageSharePavithran 2 months agoI tried this, but not work :(Share0Did you activate FTP passive mode:  see https://support.tigertech.net/filezilla-passiveCommentShareDidier_AWSanswered 2 months agoPavithran 2 months agoIt's already in a passive modeShare0Did you properly follow: https://repost.aws/knowledge-center/lightsail-set-up-ftpCommentShareDidier_AWSanswered 2 months agoPavithran 2 months agois it necessary for an SSL certificate configure vsftpd?Share0is it necessary for an SSL certificate configure vsftpd?CommentSharePavithrananswered 2 months ago0SSL is separate from userid/ password. If you're happy with the working keypair, you can skip this.CommentShareDidier_AWSanswered 2 months ago0Thanks for your support.Actually, I made a mistake when creating an instance. After making some changes in security groups, it works.CommentSharePavithrananswered 2 months ago"


Question: Reusing voucher after canceling exam

"I scheduled my Cloud Practitioner exam a few weeks ago. I wasn't ready to take the exam so I decided to use my two rescheduling opportunities. It's scheduled online for this Friday, but I just had an emergency and won't be able to actually be home to take it. Since I used my two reschedules, would I be able to cancel the exam and reuse the same voucher I initially used when I schedule it again?FollowCommentNelliT-AWS 2 months agoPlease reach out to Pearson VUE, so they can help you reschedule and keep your voucher.Pearson VUE Customer Support (https://home.pearsonvue.com/aws/contact)Share"

Answer: "0I think that there is no problem as long as the voucher does not exceed the deadline as described in the document below.https://aws.amazon.com/certification/policies/before-testing/?nc1=h_lsPlease note: You can reschedule your exam up to 24 hours before your scheduled exam time. Each exam appointment can only be rescheduled twice. If you need to reschedule your exam a third time, you will need to cancel your exam and schedule it for a more convenient date. If you paid for your exam using a voucher, verify the voucher expiration date prior to modifying your existing registration. Exam voucher expiration dates cannot be extended.If this answer leads to a solution, please approve the answer to improve the communityCommentShareEXPERTRiku_Kobayashianswered 2 months ago"


Question: IoT Protobuf support SQL problems

"I'm thrilled that AWS IoT now supports Protobuf decoding within Rules - it's going so save us a bunch of expensive Lambda function calls to decode simple protobuf messages.However, I'm stuck trying to understand the implementation.I've followed the instructions to the T, but am stuck on the SQL query inside the IoT rule.  Unfortunately it's not very well documented, including what appears to be a missing ' in the example:SELECT VALUE decode(encode(*, 'base64'), 'proto', '<BUCKET NAME>, '<FILENAME>.desc', '<PROTO_FILENAME>', 'Person') FROM 'test/proto'Shouldnt it be '<BUCKET NAME>' with the trailing parentheses?Anyway, if my incoming payload looks like this:{  "PayloadData": "CAEQZBj0HA==",   "foo":"bar"}What is the correct SQL query to decode the payload value (which is a valid Base64 encoded Protobuf message).I've tried:SELECT VALUE decode(PayloadData, 'proto', 'example-proto-registry', 'abc123.desc', 'abc123', 'Uplink') FROM 'test/proto'But that is not working as expected... Is this syntax correct? I'm not seeing any messages flowing to the republish topic that I defined in the rule, nor any errors flowing to the error action republish topic either.Looking through the logs I do see two errors, but can't seem to overcome them - I've tried several versions of the query and several version of the Role policy (including no restrictions on any S3 bucket with *)Error 1:Seems to be an issue with the policy..."reason": "Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: *****; S3 Extended Request ID: ******; Proxy: null)",    "details": "decode(..., 'proto', 'example-proto-registry', 'abc123.desc', 'abc123', 'Uplink') failed while retrieving S3 object metadata"Current Policy for the Rule use to test Protobuf - you can see I've granted all access and it's still failing.{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": [                "cloudwatch:PutMetricData",                "firehose:PutRecord",                "s3:GetObjectVersionTagging",                "s3:GetStorageLensConfigurationTagging",                "s3:GetObjectAcl",                "s3:GetBucketObjectLockConfiguration",                "s3:GetIntelligentTieringConfiguration",                "s3:GetObjectVersionAcl",                "s3:GetBucketPolicyStatus",                "s3:GetObjectRetention",                "s3:GetBucketWebsite",                "dynamodb:PutItem",                "s3:GetJobTagging",                "s3:GetMultiRegionAccessPoint",                "s3:GetObjectAttributes",                "s3:GetObjectLegalHold",                "s3:GetBucketNotification",                "s3:DescribeMultiRegionAccessPointOperation",                "s3:GetReplicationConfiguration",                "s3:PutObject",                "s3:GetObject",                "kinesis:PutRecord",                "s3:DescribeJob",                "s3:GetAnalyticsConfiguration",                "s3:GetObjectVersionForReplication",                "s3:GetAccessPointForObjectLambda",                "s3:GetStorageLensDashboard",                "es:ESHttpPut",                "s3:GetLifecycleConfiguration",                "s3:GetAccessPoint",                "s3:GetInventoryConfiguration",                "s3:GetBucketTagging",                "s3:GetAccessPointPolicyForObjectLambda",                "s3:GetBucketLogging",                "s3:GetAccelerateConfiguration",                "s3:GetObjectVersionAttributes",                "s3:GetBucketPolicy",                "sqs:SendMessage*",                "s3:GetEncryptionConfiguration",                "s3:GetObjectVersionTorrent",                "sns:Publish",                "s3:GetBucketRequestPayment",                "s3:GetAccessPointPolicyStatus",                "s3:GetObjectTagging",                "cloudwatch:SetAlarmState",                "s3:GetMetricsConfiguration",                "s3:GetBucketOwnershipControls",                "iot:Publish",                "s3:GetBucketPublicAccessBlock",                "s3:GetMultiRegionAccessPointPolicyStatus",                "s3:GetMultiRegionAccessPointPolicy",                "s3:GetAccessPointPolicyStatusForObjectLambda",                "s3:GetBucketVersioning",                "s3:GetBucketAcl",                "s3:GetAccessPointConfigurationForObjectLambda",                "s3:GetObjectTorrent",                "s3:GetMultiRegionAccessPointRoutes",                "s3:GetStorageLensConfiguration",                "s3:GetAccountPublicAccessBlock",                "s3:GetBucketCORS",                "s3:GetBucketLocation",                "s3:GetAccessPointPolicy",                "s3:GetObjectVersion"            ],            "Resource": "*"        }    ]}Error 2:Appears to be an issue with the syntax."details": "Function 'Decode' failed to execute for rule 'protocoder'. Invalid parameters. Expected decode(*, $scheme, $[scheme parameters])"What am I doing wrong??FollowCommentAWS-User-9404308 5 months agoAdding more detail here.  The docs state "Make sure you grant AWS IoT Core access to read the FileDescriptorSet from S3." but are not clear on where that policy needs to be set. On the Rule that is executing the query? Or in Manage > Security > Policies (at the IoT data plane level)?Additionally, using the example policy from the docs:{"Version": "2012-10-17","Statement": [{"Sid": "Statement1","Effect": "Allow","Principal": {"Service": "iot.amazonaws.com"},"Action": "s3:Get*","Resource": "arn:aws:s3:::example-proto-registry/abc123.desc"}]}Returns an error saying: "Policy document is Malformed: Has prohibited field Principal".Would really appreciate some assistance on this!Share"

Answer: "0In case anyone else experiences the same issues, here's what worked for me:Create your Protobuf definitions (.proto) and .desc files locally using the instructionsCreate a dedicated S3 bucket to store those filesCreate a Bucket Policy, inside the S3 bucket, not in the Rule you will create in step 4, that gives permission for IoT to access those files, and list each Resource object (.desc) explicitly.  See policy below:Create an IoT Rule and with the correct SQL syntax. See two examples below:#4 IoT Rule SQLIf your protobuf payload is coming into IoT already Base64 encoded, for example:{"PayloadData":"CAMQKhiTFQ=="}Use the following SQL, where PayloadData is the key containing your Base64 encoded Protobuf payload.SELECT VALUE decode(PayloadData, 'proto', 'example-proto-registry', 'abc123.desc', 'abc123', 'Uplink') FROM 'test/proto'#3 Bucket Policy Example{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "Statement1",            "Effect": "Allow",            "Principal": {                "Service": "iot.amazonaws.com"            },            "Action": "s3:Get*",            "Resource": "arn:aws:s3:::example-proto-registry/abc123.desc",            "Resource": "arn:aws:s3:::example-proto-registry/def456.desc"        }    ]}CommentShareAWS-User-9404308answered 5 months ago"


Question: Deploy Microsoft Defender for Endpoint on Amazon Linux 2

"Hey Team,I am unable to deploy Deploy Microsoft Defender for Endpoint on Amazon linux 2 EC2 instance. I follow the steps mentioned in the document https://learn.microsoft.com/en-us/microsoft-365/security/defender-endpoint/linux-install-manually?view=o365-worldwide.Can someone please help me on this?OS Release version:NAME="Amazon Linux"VERSION="2"ID="amzn"ID_LIKE="centos rhel fedora"VERSION_ID="2"PRETTY_NAME="Amazon Linux 2"ANSI_COLOR="0;33"CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"Shell Installation output:[root@ip-172-31-76-111 ec2-user]# sudo yum install yum-utils -yLoaded plugins: extras_suggestions, langpacks, priorities, update-motdamzn2-core                                                                                | 3.7 kB  00:00:00Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest versionNothing to do[root@ip-172-31-76-111 ec2-user]# sudo yum-config-manager --add-repo=https://packages.microsoft.com/config/rhel/7.2/prod.repoLoaded plugins: extras_suggestions, langpacks, priorities, update-motdadding repo from: https://packages.microsoft.com/config/rhel/7.2/prod.repograbbing file https://packages.microsoft.com/config/rhel/7.2/prod.repo to /etc/yum.repos.d/prod.reporepo saved to /etc/yum.repos.d/prod.repo[root@ip-172-31-76-111 ec2-user]# sudo rpm --import http://packages.microsoft.com/keys/microsoft.asc[root@ip-172-31-76-111 ec2-user]# sudo yum install mdatp -yLoaded plugins: extras_suggestions, langpacks, priorities, update-motdNo package mdatp available.Error: Nothing to do[root@ip-172-31-76-111 ec2-user]# sudo yum --enablerepo=packages-microsoft-com-prod install mdatpLoaded plugins: extras_suggestions, langpacks, priorities, update-motdNo package mdatp available.Error: Nothing to doFollowComment"

Answer: "1Hello,Thank you for reaching out to us.I understand that you are trying to deploy ‘Microsoft Defender for Endpoint’ on Amazon Linux 2 and in the process you are seeing the message “No package mdatp available”. You are looking for assistance on the same.I simulated your case in our lab environment by launching EC2 Instances on both 64-bit(x86) and 64-bit(Arm) architectures. In my test results, I found that the instance on 64-bit(Arm) arch had the issue “No package mdatp available” although the repository “packages-microsoft-com-prod“ was present in the output of the command “yum repolist”, while the issue was not seen on the instance launched on 64-bit(x86) arch. This suggests that the repository does not have the ‘mdatp’ package for 64-bit(Arm) architecture.Further, I tried to find a repository suitable for 64-bit(Arm) arch but I could not find any. In the Microsoft documentation[*], I am able to see the Supported Linux server distributions and x64 (AMD64/EM64T) and x86_64 versions to deploy 'Microsoft Defender for Endpoint', and the 64-bit(Arm) versions are not explicitly listed. You may refer further details here:[*] System requirements: https://learn.microsoft.com/en-us/microsoft-365/security/defender-endpoint/microsoft-defender-endpoint-linux?view=o365-worldwideTherefore, I recommend AWS alternatives to 'Microsoft Defender for Endpoint' such as Amazon GaurdDuty[1] or Amazon Inspector[2] for instances on 64-bit(Arm) arch. However, if there is a need for you to use 'Microsoft Defender for Endpoint', then you need to install it on the instances with supported architecture[*].To check the architecture, run the command below on your EC2 Instance:#uname -mI hope this helps.Have a good day ahead!References:[1] Amazon GaurdDuty: https://docs.aws.amazon.com/guardduty/latest/ug/what-is-guardduty.html[2] Amazon Inspector: https://docs.aws.amazon.com/inspector/latest/user/what-is-inspector.htmlCommentShareMeenakshianswered 4 months ago0After running the "yum repolist", I am seeing the similar outputBut when I run the command "yum install mdatp -y", I am seeing "No package mdatp available".CommentShareSrikanth Achutunianswered 4 months agoCheriyanM 4 months agoYou can do a yum find using the command  : sudo yum search "mdatp" if all the repos are configured and accessible. It should return results similar to belowLoaded plugins: extras_suggestions, langpacks, priorities, update-motd===================================================== N/S matched: mdatp ======================================================mdatp.x86_64 : Microsoft Defender (Production)Name and summary matches only, use "search all" for everything.Another option is to disable the repo using command : yum-config-manager --disable packages-microsoft-com-prod   and re -run the steps as per installation guide.Share0HiPlease validate if the repos have been configured properly. The link has more info on managing packages on Linux 2. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/managing-software.htmlIf you run "yum repolist", it should show the following similar output :ThanksCommentShareCheriyanManswered 4 months ago"


Question: How can I optimize performance when uploading large files to S3 across multiple AWS Regions?

"We need the ability to upload large files—between 500 MB and 1 GB per file—to Amazon S3 in under a minute. The files are already compressed, and the files come from our application's end users, who are located across continental United States.We've already looked at using multiple AWS Regions so that we can send uploads to the same Region that the end user is already in, but that didn't improve performance enough. We also looked at activating S3 Transfer Acceleration, but the S3 Transfer Acceleration was slower than going over the public internet in several locations.What's the best way to optimize performance when uploading large files to S3 across multiple Regions?FollowComment"

Answer: "0Accepted AnswerTo upload a 1 GB file to S3 in less than 1 minute, your application's end users need to achieve an average upload rate of 17 MB per second. Assuming that criteria can be met for your use case, you'll need to plan for a transfer method that can reach an upload rate of 17 MB per second.When you upload large files to S3, it's a best practice to use multipart uploads. Using multipart uploads allows you to break down a larger file into smaller parts for quicker upload speeds. For more information, see How can I optimize performance when I upload large files to Amazon S3?Note: A multipart upload requires that a single file is uploaded in not more than 10,000 distinct parts. You must be sure that the chunksize that you set balances the part file size and the number of parts. For example: If the file size is 1 GB and your average upload rate is 500 KB per second for each TCP connection, then you'd want to configure at least a 34-part upload with a part size that's equal to or smaller than 30 MB.CommentShareMarco_Lanswered 3 years ago"