Question: Greengrass v2 Pkcs11Provider with TPM - not connecting after install

"OverviewI am trying to use a TPM2.0 HSM with Pkcs11Provider component to do manual Greengrass v2 provisioning.After following the steps to install, I am seeing errors in the greengrass.log and unsure how to interpret.What I've TriedI made my way through the 5 steps here, and have a key and cert stored in the TPM:$ sudo ./tpm2_ptool listobjects --label greengrass- CKA_CLASS: CKO_PRIVATE_KEY  CKA_ID:  - '64653534386238323765613433653632'  CKA_KEY_TYPE: CKK_RSA  CKA_LABEL: greenkey  id: 1- CKA_CLASS: CKO_PUBLIC_KEY  CKA_ID:  - '64653534386238323765613433653632'  CKA_KEY_TYPE: CKK_RSA  CKA_LABEL: greenkey  id: 2- CKA_CLASS: CKO_CERTIFICATE  CKA_ID:  - '64653534386238323765613433653632'  CKA_LABEL: greenkey  id: 3$ sudo p11tool --login --list-privkeys 'pkcs11:manufacturer=Infineon;token=greengrass;pin-value=123456'Object 0:URL: pkcs11:model=SLB9670%00%00%00%00%00%00%00%00%00;manufacturer=Infineon;serial=0000000000000000;token=greengrass;id=%64%65%35%34%38%62%38%32%37%65%61%34%33%65%36%32;object=greenkey;type=privateType: Private keyLabel: greenkeyFlags: CKA_PRIVATE; CKA_NEVER_EXTRACTABLE; CKA_SENSITIVE; ID: 64:65:35:34:38:62:38:32:37:65:61:34:33:65:36:32I generated the cert in AWS from a CSR before storing in the TPM.I downloaded the Pkcs11Provider JAR from here which was v2.0.4 on Nov 21, 2022.I then ran the command sudo -E java -Droot="/home/user/greengrass/v2" -Dlog.store=FILE -jar ./GreengrassInstaller/lib/Greengrass.jar --trusted-plugin ./aws.greengrass.crypto.Pkcs11Provider-latest.jar --init-config install.yaml --component-default-user ggc_user:ggc_group --setup-system-service true.  This is the content of the config file:$ cat ~/install.yaml ---system:  certificateFilePath: "pkcs11:object=greenkey;type=cert"  privateKeyPath: "pkcs11:object=greenkey;type=private"  rootCaPath: "/home/user/rootCA.pem"  rootpath: "/home/user/greengrass/v2"  thingName: "MyThing"services:  aws.greengrass.Nucleus:    componentType: "NUCLEUS"    version: "2.7.0"    configuration:      awsRegion: "us-west-2"      iotRoleAlias: "GreengrassV2TokenExchangeRoleAlias"      iotDataEndpoint: "<redacted>-ats.iot.us-west-2.amazonaws.com"      iotCredEndpoint: "<redacted>.credentials.iot.us-west-2.amazonaws.com"  aws.greengrass.crypto.Pkcs11Provider:    configuration:      name: "tpm2_pkcs11"      library: "/usr/local/lib/libtpm2_pkcs11.so"      slot: 1      userPin: "123456"Not sure if it matters, but I linked the certificate created via CSR to the Thing in AWS IoT Core dashboard, and gave the certificate the usual policies for access.  The Thing already existed since I had previously created with automatic provisioning using tokens in environment vars.  I did not delete the Thing, but deleted the automatically-created cert when I linked the new one.Error LogThis is the error I see when Greengrass starts up:$ sudo tail -n 100 greengrass/v2/logs/greengrass.log2022-11-22T14:45:17.246Z [INFO] (pool-2-thread-16) com.aws.greengrass.lifecyclemanager.GenericExternalService: generic-service-shutdown. {serviceName=aws.greengrass.Nucleus, currentState=STOPPING}2022-11-22T14:45:17.274Z [INFO] (aws.greengrass.Nucleus-lifecycle) com.aws.greengrass.lifecyclemanager.GenericExternalService: service-set-state. {serviceName=aws.greengrass.Nucleus, currentState=STOPPING, newState=FINISHED}2022-11-22T14:45:17.376Z [INFO] (pool-2-thread-8) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: Initializing PKCS11 provider with configuration. {configuration=name=tpm2_pkcs11library=/usr/local/lib/libtpm2_pkcs11.soslot=1, serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW}2022-11-22T14:45:17.915Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW, newState=INSTALLED}2022-11-22T14:45:17.921Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=INSTALLED, newState=STARTING}2022-11-22T14:45:17.925Z [INFO] (pool-2-thread-16) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}2022-11-22T14:45:17.927Z [INFO] (pool-2-thread-16) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}2022-11-22T14:45:17.929Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=STARTING, newState=RUNNING}2022-11-22T14:45:17.933Z [INFO] (main-lifecycle) com.aws.greengrass.lifecyclemanager.GenericExternalService: service-set-state. {serviceName=main, currentState=INSTALLED, newState=STARTING}2022-11-22T14:45:17.939Z [INFO] (pool-2-thread-11) com.aws.greengrass.lifecyclemanager.GenericExternalService: generic-service-finished. Nothing done. {serviceName=main, currentState=STARTING}2022-11-22T14:45:17.944Z [INFO] (main-lifecycle) com.aws.greengrass.lifecyclemanager.GenericExternalService: service-set-state. {serviceName=main, currentState=STARTING, newState=FINISHED}2022-11-22T14:45:18.890Z [ERROR] (pool-2-thread-6) com.aws.greengrass.mqttclient.MqttClient: Error subscribing. {topic=$aws/things/MyThing/jobs/$next/namespace-aws-gg-deployment/get/accepted}java.util.concurrent.CompletionException: software.amazon.awssdk.crt.mqtt.MqttException: Error during getting mqtt connection builderat java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:989)at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137)at com.aws.greengrass.mqttclient.AwsIotMqttClient.connect(AwsIotMqttClient.java:234)at com.aws.greengrass.mqttclient.AwsIotMqttClient.subscribe(AwsIotMqttClient.java:153)at com.aws.greengrass.mqttclient.MqttClient.subscribe(MqttClient.java:403)at com.aws.greengrass.mqttclient.WrapperMqttClientConnection.subscribe(WrapperMqttClientConnection.java:73)at com.aws.greengrass.deployment.IotJobsClientWrapper.SubscribeToDescribeJobExecutionAccepted(IotJobsClientWrapper.java:198)at software.amazon.awssdk.iot.iotjobs.IotJobsClient.SubscribeToDescribeJobExecutionAccepted(IotJobsClient.java:599)at com.aws.greengrass.deployment.IotJobsHelper.subscribeToGetNextJobDescription(IotJobsHelper.java:504)at com.aws.greengrass.deployment.IotJobsHelper.subscribeToJobsTopics(IotJobsHelper.java:463)at com.aws.greengrass.deployment.IotJobsHelper.lambda$setupCommWithIotJobs$5(IotJobsHelper.java:339)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:750)Caused by: software.amazon.awssdk.crt.mqtt.MqttException: Error during getting mqtt connection builderat com.aws.greengrass.mqttclient.MqttClient.lambda$new$0(MqttClient.java:180)at com.aws.greengrass.mqttclient.MqttClient.lambda$getNewMqttClient$16(MqttClient.java:766)at com.aws.greengrass.mqttclient.AwsIotMqttClient.establishConnection(AwsIotMqttClient.java:256)... 14 more2022-11-22T14:45:18.906Z [WARN] (pool-2-thread-6) com.aws.greengrass.deployment.IotJobsHelper: No connection available during subscribing to Iot Jobs descriptions topic. Will retry in sometime. {ThingName=MyThing}software.amazon.awssdk.crt.mqtt.MqttException: Error during getting mqtt connection builderat com.aws.greengrass.mqttclient.MqttClient.lambda$new$0(MqttClient.java:180)at com.aws.greengrass.mqttclient.MqttClient.lambda$getNewMqttClient$16(MqttClient.java:766)at com.aws.greengrass.mqttclient.AwsIotMqttClient.establishConnection(AwsIotMqttClient.java:256)at com.aws.greengrass.mqttclient.AwsIotMqttClient.connect(AwsIotMqttClient.java:234)at com.aws.greengrass.mqttclient.AwsIotMqttClient.subscribe(AwsIotMqttClient.java:153)at com.aws.greengrass.mqttclient.MqttClient.subscribe(MqttClient.java:403)at com.aws.greengrass.mqttclient.WrapperMqttClientConnection.subscribe(WrapperMqttClientConnection.java:73)at com.aws.greengrass.deployment.IotJobsClientWrapper.SubscribeToDescribeJobExecutionAccepted(IotJobsClientWrapper.java:198)at software.amazon.awssdk.iot.iotjobs.IotJobsClient.SubscribeToDescribeJobExecutionAccepted(IotJobsClient.java:599)at com.aws.greengrass.deployment.IotJobsHelper.subscribeToGetNextJobDescription(IotJobsHelper.java:504)at com.aws.greengrass.deployment.IotJobsHelper.subscribeToJobsTopics(IotJobsHelper.java:463)at com.aws.greengrass.deployment.IotJobsHelper.lambda$setupCommWithIotJobs$5(IotJobsHelper.java:339)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:750)ConclusionAny advice for diagnosing this is appreciated!Edit 1 - SoftHSMAs noted in the comments, I tried using softhsm rather than TPM functions, just to make sure the Pkcs11Provider plugin is working.  I was still having some issues with that.  I am pretty sure I have the slot: correct for the Pkcs11Provider config.$ sudo pkcs11-tool --module /usr/local/lib/softhsm/libsofthsm2.so --list-slotsAvailable slots:Slot 0 (0x69906b7d): SoftHSM slot ID 0x69906b7d  token label        : greengrass  token manufacturer : SoftHSM project  token model        : SoftHSM v2  token flags        : rng, login required, PIN initialized, token initialized, other flags=0x20  hardware version   : 2.6  firmware version   : 2.6  serial num         : 2337b990e9906b7dSlot 1 (0x1): SoftHSM slot ID 0x1  token state:   uninitialized  $ sudo pkcs11-tool --module /usr/local/lib/softhsm/libsofthsm2.so --list-objects --slot 1771072381 --loginLogging in to "greengrass".Please enter User PIN: Certificate Object, type = X.509 cert  label:        ID:         1771072381Private Key Object; RSA   label:      greenkey  ID:         1771072381  Usage:      decrypt, sign, unwrapPublic Key Object; RSA 2048 bits  label:      greenkey  ID:         1771072381  Usage:      encrypt, verify, wrap  $ sudo cat greengrass/v2/config/effectiveConfig.yaml  | grep "slot\|pkcs"  certificateFilePath: "pkcs11:object=greenkey;type=cert"  privateKeyPath: "pkcs11:object=greenkey;type=key"      name: "softhsm_pkcs11"      slot: 1771072381$ sudo tail -f greengrass/v2/logs/greengrass.log2022-12-02T21:32:38.826Z [INFO] (pool-2-thread-14) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: Initializing PKCS11 provider with configuration. {configuration=name=softhsm_pkcs11library=/usr/local/lib/softhsm/libsofthsm2.soslot=1771072381, serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW}2022-12-02T21:32:39.431Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=NEW, newState=INSTALLED}2022-12-02T21:32:39.438Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=INSTALLED, newState=STARTING}2022-12-02T21:32:39.442Z [INFO] (pool-2-thread-13) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}2022-12-02T21:32:39.444Z [INFO] (pool-2-thread-13) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=pkcs11}2022-12-02T21:32:39.446Z [INFO] (aws.greengrass.crypto.Pkcs11Provider-lifecycle) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: service-set-state. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=STARTING, newState=RUNNING}...2022-12-02T21:32:41.162Z [ERROR] (pool-2-thread-7) com.aws.greengrass.mqttclient.MqttClient: Error subscribing. {topic=$aws/things/MyThing/jobs/$next/namespace-aws-gg-deployment/get/accepted}...2022-12-02T21:32:41.173Z [WARN] (pool-2-thread-7) com.aws.greengrass.deployment.IotJobsHelper: No connection available during subscribing to Iot Jobs descriptions topic. Will retry in sometime. {ThingName=MyThing}...2022-12-02T21:32:41.206Z [ERROR] (pool-2-thread-8) com.aws.greengrass.security.provider.pkcs11.PKCS11CryptoKeyService: Private key must be a PKCS11 private type, but was key. {serviceName=aws.greengrass.crypto.Pkcs11Provider, currentState=RUNNING}FollowComment"

Answer: "3Accepted AnswerI used strace to determine that the AWS Pkcs11Provider was looking in the wrong location for the TPM2 PKCS11 sqlite3 database:So this explains the CKR_OPERATION_NOT_INITIALIZED error in greengrass.log.   The database at the path above is empty, and thus has no matching entries for slots, tokens, and objects that the TPM is tracking...The AWS plugin needed to be instructed to look in a different location where the TPM2 database actually lives, which can be done via environment variable (and probably other ways too):CommentSharettnickbanswered 6 months agoJoeAtAWS 6 months agoGreat detective work! Thank you for sharing your findings.For others who stumble across this, here is TPM2 initialization documentation which defines the search order for the store: https://github.com/tpm2-software/tpm2-pkcs11/blob/master/docs/INITIALIZING.mdShareGreg_B EXPERT6 months agoYeah really great stuff ttnickb!Share0Hi ttnickb. You'll get that error if the slot ID in the Pkcs11Provider component configuration is wrong. I think you may be using the slot index instead of the slot ID. For example, here's my setup:pi@raspberrypi4:~ $ sudo pkcs11-tool --module /usr/lib/arm-linux-gnueabihf/softhsm/libsofthsm2.so --list-slotsAvailable slots:Slot 0 (0x20f1b2b2): SoftHSM slot ID 0x20f1b2b2  token label        : greengrass  token manufacturer : SoftHSM project  token model        : SoftHSM v2  token flags        : login required, rng, token initialized, PIN initialized, other flags=0x20  hardware version   : 2.4  firmware version   : 2.4  serial num         : 6e6640c0a0f1b2b2  pin min/max        : 4/255Slot 1 (0x1): SoftHSM slot ID 0x1  token state:   uninitializedpi@raspberrypi4:~ $ sudo pkcs11-tool --module /usr/lib/arm-linux-gnueabihf/softhsm/libsofthsm2.so --list-objects --login --pin 1234Using slot 0 with a present token (0x20f1b2b2)Certificate Object; type = X.509 cert  label:      greengrass  subject:    DN: CN=AWS IoT Certificate  ID:         deadPrivate Key Object; RSA   label:      greengrass  ID:         dead  Usage:      decrypt, sign, unwrapThe correct value to use in my case is 552710834 (0x20f1b2b2), not 0.CommentShareEXPERTGreg_Banswered 6 months agottnickb 6 months agoThanks Greg.  I installed pkcs11-tool and tried the command you showed.  It seems to confirm I have the slot correct as 1 but maybe I am mis-interpreting the output?$ sudo pkcs11-tool --module /usr/local/lib/libtpm2_pkcs11.so --list-slotsAvailable slots:Slot 0 (0x1): greengrass  token label        : greengrass  token manufacturer : Infineon  token model        : SLB9670  token flags        : rng, login required, PIN initialized, token initialized  hardware version   : 1.16  firmware version   : 7.62  serial num         : 0000000000000000ShareGreg_B EXPERT6 months agoOK. Did you test your new certificate as a file-based credential before swapping to PKCS? If not, I suggest you swap back to file-based credentials, to know you have a good certificate and key, that the cloud and device agree on. Then go back to PKCS, importing the proven credentials.If your PKCS configuration was wrong (slot, label etc), you should see more errors than the one you reported. I tested on my RPi; with the wrong slot ID, I get the error you report, but many other errors too.Sharettnickb 6 months agoHi Greg, yeah I actually took the approach of provisioning automatically and then storing the key and cert into the TPM.  This was following the steps described under "Configure hardware security on an existing core device" here.This was how I confirmed the cert stored in the TPM was correct:$ sudo ./tpm2_ptool objmod --id 3 --key 17 | cut -d " " -f2 | xxd -r -p | openssl x509 -inform DER -out fromtpm.crt  # extracts cert$ diff fromtpm.crt ~/greengrass/v2/thingCert.crt          # outputs nothing, retcode=0 (no diff)The --id 3 above comes from here:$ sudo ./tpm2_ptool listobjects  --label greengrass- CKA_CLASS: CKO_PRIVATE_KEY  CKA_ID:  - '32643332333762346634656638613633'  CKA_KEY_TYPE: CKK_RSA  CKA_LABEL: greenkey  id: 1- CKA_CLASS: CKO_PUBLIC_KEY  CKA_ID:  - '32643332333762346634656638613633'  CKA_KEY_TYPE: CKK_RSA  CKA_LABEL: greenkey  id: 2- CKA_CLASS: CKO_CERTIFICATE  CKA_ID:  - '32643332333762346634656638613633'  CKA_LABEL: greenkey  id: 3   ###### Use this ID to extract cert ######I did see more errors when I changed the slot ID in the Pkcs11Provider config to something else.ShareMichaelDombrowski-AWS EXPERT6 months agoDid you try using slot 0? That's what the output of the pkcs tool said: Slot 0Sharettnickb 6 months agoI tried SoftHSM since that worked for @Greg_B. A slot and token are initialized, with AWS-provided key and cert:$ sudo pkcs11-tool --module /usr/local/lib/softhsm/libsofthsm2.so --list-objects --loginUsing slot 0 with a present token (0x7ec8556f)Logging in to "greengrass"....Private Key Object; RSA   label:      greenkey  ID:         2127058287  Usage:      decrypt, sign, unwrapCertificate Object, type = X.509 cert  label:      greenkey  ID:         2127058287I see a GG error still:...PKCS11CryptoKeyService: Private key must be a PKCS11 private type, but was key.ShareShow 5 more comments"


Question: Mail bounces back with "No Such User Here"

"Problem solved. I don't see a way to delete my question.Edited by: kurupted on May 24, 2019 6:11 PMFollowComment"

Answer: 0okay.CommentShareSasanapuri-AWSanswered 4 years ago


Question: RDS Blue/Green Documentation Discrepancy/Unclear

"The documentation for RDS blue/green deployments indicates the following limitation:MySQL versions 8.0.13 and lower have a community bug that prevents RDS from supporting them for blue/green deploymentsI'm running a MySQL database in RDS on version 5.7.33, yet my Action menu still shows the option to "Create Blue/Green Deployment" for this instance. Is the limitation/bug only referring to MySQL version 8 databases, or would version 5.7 also be impacted by this bug? I don't want to perform a required update only for it to fail, or even worse, to lose data in a production environment.FollowCommentphilaws MODERATOR2 months agoWhen you do choose to upgrade to 8.0, just choose a more recent minor version and skip 8.0.13 entirely.Share"

Answer: "0Accepted AnswerHelloMySQL versions 8.0.13 and lower means that the bug is affected in the MySQL versions from 8.0.x, until MySQL versions 8.0.13.If this answer has led to a solution, please approve the answer for the betterment of the community.CommentShareVetrivelanswered 2 months ago"


Question: SES Error "...Maximum sending rate exceeded..."

"I have set up SES and DKIM and I sent a few emails via a PHP script to myself. The last time I ran the script I got the following error:Email not sent. SMTP Error: data not accepted.SMTP server error: DATA END command failed Detail: Throttling failure: Maximum sending rate exceeded. SMTP code: 454I have only been sending emails to myself and I need to be able to do this quite a lot in order to develop my website.Edit: I went to  Amazon SES > Account dashboard and under 'Daily email usage' I can see have sent 21,428 emails. I am worried about this as I have only knowingly sent about 5 emails. Either an error has occurred (possibly me programming) or perhaps someone has accessed my account. I would really appreciate some help sorting this out as it is a bit worrying.FollowComment"

Answer: "0I have just received a warning from GitHub saying:GitGuardian has detected the following SMTP credentials exposed within your GitHub account.So, I suspect those credentials were detected and used to send emails from my account. I therefore deleted all my SES verified identities from Amazon SES. I will make sure not to make that mistake again.CommentSharerePost-User-8454414answered 8 months ago"


Question: Access Control in Secrets Manager for Federated Users

"My scenario:I have my users in Azure AD. This is connected to AWS Single Account SSO into an AWS Account using IAM SAML IDP (PS: we are not using AWS SSO Service). We are using AWS Secrets Manager and want to store per user secret using a secret name path (eg /usersecrets/<azure_ad_username>/<secret_name>When the users login using Azure AD auth, they automatically assume the IAM Role attached. I would like to do the following:Requirement1:Allow users to list secrets, create secrets and get secret value for any secret which has a name /usersecrets/<azure_ad_username>/* (here the azure_ad_username is what AWS session sees when the assume role to login)Deny access to any secret unless the request is coming from Federated user (i.e local IAM users in AWS account should not be able to see any secret in path /usersecrets/<azure_ad_username>/*Requirement2:In addition to the federates Azure AD users, I also want to allow a EC2 Instance Role to be able to Get/List/Describe any secret. This EC2 role is in same AWS account where secrets are and is attached to all Windows Servers. This IAM role is to allow SSM Run commands to execute on these Windows machines and fetch the secrets values (eg, to get the secret of a user and create a local windows user with same name and password as it is in secret manager using powershell.Questions:Can you help with some sample IAM Policy for the role or the secret manager resource policy I can use to meet both the requirements?FollowComment"

Answer: "1You can use the concept of a Token Vending Machine (see here to create dynamic policies for users when you federate it.For the EC2 Role, see examples here: https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access_examples.htmlCommentShareRodney Lesteranswered a year agoEXPERTBrettski-AWSreviewed a year ago"


Question: Perform UPSERT in S3 with awswrangler library

"Hi,I am trying to perform an upsert of an inceberg table.The script below creates a table with raw data stored in parquet format in an S3 bucket.Then it creates an empty iceberg table to be populated and eventually updated.When trying to insert data, it fails, please see error further down.The script:import pandas as pdimport awswrangler as wrimport boto3database = "test"iceberg_database = "iceberg_mid_sized_demo"bucket_name = "test-bucket"folder_name = "iceberg_mid_sized/raw_input"path = f"s3://{bucket_name}/{folder_name}"session = boto3.Session()glue_client = session.client('glue')try:    glue_client.create_database(DatabaseInput={'Name': database})    print('Database created')    except glue_client.exceptions.AlreadyExistsException as e:    print("The database already exists")# Create external table in input parquet files. create_raw_data_table_query = """CREATE EXTERNAL TABLE test.raw_input(  op string,   id bigint,   name string,   city string)ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS PARQUETLOCATION 's3://test-bucket/iceberg_mid_sized/raw_input/'tblproperties ("parquet.compress"="SNAPPY");"""create_raw_data_table_query_exec_id = wr.athena.start_query_execution(sql=create_raw_data_table_query, database=database)# create iceberg tables databasetry:    glue_client.create_database(DatabaseInput={'Name': iceberg_database})    print('Database created')    except glue_client.exceptions.AlreadyExistsException as e:    print("The database already exists")# Create output Iceberg table with partitioning. Replace the S3 bucket name with your bucket namecreate_output_iceberg_query = """CREATE TABLE iceberg_mid_sized_demo.iceberg_output (  id bigint,  name string,  city string  ) LOCATION 's3://test-bucket/iceberg-mid_sized/iceberg_output/' TBLPROPERTIES (  'table_type'='ICEBERG',  'format'='parquet')"""create_iceberg_table_query_exec_id = wr.athena.start_query_execution(sql=create_output_iceberg_query, database=iceberg_database)primary_key = ['id']wr.s3.merge_upsert_table(delta_df=val_df, database='iceberg_mid_sized_demo', table='iceberg_output', primary_key=primary_key)This last line returns the following traceback and error:ArrowInvalid                              Traceback (most recent call last)/var/folders/y8/11mxbknn1sxbbq7vvhd14frr0000gn/T/ipykernel_17075/2358353780.py in <module>      1 primary_key = ['id']----> 2 wr.s3.merge_upsert_table(delta_df=val_df, database='iceberg_mid_sized_demo', table='iceberg_output', primary_key=primary_key)/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/s3/_merge_upsert_table.py in merge_upsert_table(delta_df, database, table, primary_key, boto3_session)    111     if wr.catalog.does_table_exist(database=database, table=table, boto3_session=boto3_session):    112         # Read the existing table into a pandas dataframe--> 113         existing_df = wr.s3.read_parquet_table(database=database, table=table, boto3_session=boto3_session)    114         # Check if data quality inside dataframes to be merged are sufficient    115         if _is_data_quality_sufficient(existing_df=existing_df, delta_df=delta_df, primary_key=primary_key):/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/_config.py in wrapper(*args_raw, **kwargs)    448                 del args[name]    449                 args = {**args, **keywords}--> 450         return function(**args)    451     452     wrapper.__doc__ = _inject_config_doc(doc=function.__doc__, available_configs=available_configs)/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/s3/_read_parquet.py in read_parquet_table(table, database, filename_suffix, filename_ignore_suffix, catalog_id, partition_filter, columns, validate_schema, categories, safe, map_types, chunked, use_threads, boto3_session, s3_additional_kwargs)    969         use_threads=use_threads,    970         boto3_session=boto3_session,--> 971         s3_additional_kwargs=s3_additional_kwargs,    972     )    973     partial_cast_function = functools.partial(/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/s3/_read_parquet.py in read_parquet(path, path_root, path_suffix, path_ignore_suffix, version_id, ignore_empty, ignore_index, partition_filter, columns, validate_schema, chunked, dataset, categories, safe, map_types, use_threads, last_modified_begin, last_modified_end, boto3_session, s3_additional_kwargs, pyarrow_additional_kwargs)    767     if len(paths) == 1:    768         return _read_parquet(--> 769             path=paths[0], version_id=versions[paths[0]] if isinstance(versions, dict) else None, **args    770         )    771     if validate_schema is True:/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/s3/_read_parquet.py in _read_parquet(path, version_id, columns, categories, safe, map_types, boto3_session, dataset, path_root, s3_additional_kwargs, use_threads, pyarrow_additional_kwargs)    538             use_threads=use_threads,    539             version_id=version_id,--> 540             pyarrow_additional_kwargs=pyarrow_args,    541         ),    542         categories=categories,/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/s3/_read_parquet.py in _read_parquet_file(path, columns, categories, boto3_session, s3_additional_kwargs, use_threads, version_id, pyarrow_additional_kwargs)    480             source=f,    481             read_dictionary=categories,--> 482             coerce_int96_timestamp_unit=pyarrow_args["coerce_int96_timestamp_unit"],    483         )    484         if pq_file is None:/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/s3/_read_parquet.py in _pyarrow_parquet_file_wrapper(source, read_dictionary, coerce_int96_timestamp_unit)     41         try:     42             return pyarrow.parquet.ParquetFile(---> 43                 source=source, read_dictionary=read_dictionary, coerce_int96_timestamp_unit=coerce_int96_timestamp_unit     44             )     45         except TypeError as ex:/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/pyarrow/parquet.py in __init__(self, source, metadata, common_metadata, read_dictionary, memory_map, buffer_size, pre_buffer, coerce_int96_timestamp_unit)    232             buffer_size=buffer_size, pre_buffer=pre_buffer,    233             read_dictionary=read_dictionary, metadata=metadata,--> 234             coerce_int96_timestamp_unit=coerce_int96_timestamp_unit    235         )    236         self.common_metadata = common_metadata/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/pyarrow/_parquet.pyx in pyarrow._parquet.ParquetReader.open()/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()ArrowInvalid: Parquet magic bytes not found in footer. Either the file is corrupted or this is not a parquet file.I have also tried to run the script replacing the following line:wr.s3.merge_upsert_table(delta_df=val_df, database='iceberg_mid_sized_demo', table='iceberg_output', primary_key=primary_key)with thesemerge_into_query = """MERGE INTO iceberg_mid_sized_demo.iceberg_output tUSING test.raw_input sON t.id = s.idWHEN MATCHED AND s.op = 'D' THEN DELETEWHEN MATCHED THEN UPDATE SET t.city = s.cityWHEN NOT MATCHED THEN INSERT (id, name, city) VALUES (s.id, s.name, s.city);"""merge_into_query_id = wr.athena.start_query_execution(sql=merge_into_query,                                                     database="iceberg_mid_sized_demo",                                                    workgroup='wgname'                                                     )however, now I am getting:---------------------------------------------------------------------------InvalidRequestException                   Traceback (most recent call last)/var/folders/y8/11mxbknn1sxbbq7vvhd14frr0000gn/T/ipykernel_17075/2112489404.py in <module>      1 merge_into_query_id = wr.athena.start_query_execution(sql=merge_into_query,      2                                                      database="iceberg_mid_sized_demo",----> 3                                                     workgroup='athena3'      4                                                      )/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/_config.py in wrapper(*args_raw, **kwargs)    448                 del args[name]    449                 args = {**args, **keywords}--> 450         return function(**args)    451     452     wrapper.__doc__ = _inject_config_doc(doc=function.__doc__, available_configs=available_configs)/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/athena/_utils.py in start_query_execution(sql, database, s3_output, workgroup, encryption, kms_key, params, boto3_session, max_cache_seconds, max_cache_query_inspections, max_remote_cache_entries, max_local_cache_entries, data_source, wait)    494             encryption=encryption,    495             kms_key=kms_key,--> 496             boto3_session=session,    497         )    498     if wait:/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/athena/_utils.py in _start_query_execution(sql, wg_config, database, data_source, s3_output, workgroup, encryption, kms_key, boto3_session)    101         ex_code="ThrottlingException",    102         max_num_tries=5,--> 103         **args,    104     )    105     return cast(str, response["QueryExecutionId"])/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/awswrangler/_utils.py in try_it(f, ex, ex_code, base, max_num_tries, **kwargs)    341     for i in range(max_num_tries):    342         try:--> 343             return f(**kwargs)    344         except ex as exception:    345             if ex_code is not None and hasattr(exception, "response"):/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)    389                     "%s() only accepts keyword arguments." % py_operation_name)    390             # The "self" in this scope is referring to the BaseClient.--> 391             return self._make_api_call(operation_name, kwargs)    392     393         _api_call.__name__ = str(py_operation_name)/opt/anaconda3/envs/data_analysis/lib/python3.7/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)    717             error_code = parsed_response.get("Error", {}).get("Code")    718             error_class = self.exceptions.from_code(error_code)--> 719             raise error_class(parsed_response, operation_name)    720         else:    721             return parsed_responseInvalidRequestException: An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: line 5:31: mismatched input '.'. Expecting: '='How do you perform UPSERT of Athena tables?ThanksFollowComment"

Answer: "1Accepted AnswerThe following query provided the solution (make sure athena engine version 3 is used):merge_into_query = """MERGE INTO "iceberg_mid_sized_demo"."iceberg_output" tUSING "test"."raw_input"  sON (t.id = s.id)WHEN MATCHED AND s.op = 'D' THEN DELETEWHEN MATCHED THEN UPDATE SET city = s.cityWHEN NOT MATCHED THEN INSERT (id, name, city) VALUES (s.id, s.name, s.city);"""merge_into_query_id = wr.athena.start_query_execution(sql=merge_into_query,database="iceberg_mid_sized_demo",workgroup='athena3')CommentShareAWS-learneranswered 4 months ago"


Question: Programmatic RBAC in MWAA apart from default roles

"Hello,we would like to add roles and permissions programmatically to MWAA. Currently, this is only possible via API, which is denied for all endpoints. Is there a workaround to create roles and attach permissions to them in a programmatic way?This is required for our case to make MWAA able to serve multiple tenants independently without having multiple MWAA instances.Thank you!Best, ChristophFollowComment"

Answer: "1Accepted AnswerHi!As with most things in Airflow, you can perform this via a DAG direct to the DB.  Here is an example that will duplicate an existing role (if the new role doesn't exist) and then assign a user to that role by username.  You can then modify the new role for the specific permissions you wish.from airflow.decorators import dag, taskfrom datetime import datetimeimport osfrom airflow import settingsfrom sqlalchemy import textDAG_ID = os.path.basename(__file__).replace(".py", "")NEW_ROLE = "My New Role"SOURCE_ROLE = "Viewer"USER_NAME = "MyUsername"SQL_QUERY = """DO $$DECLARE    new_role_name CONSTANT VARCHAR(64) := '{0}'; -- new role name    source_role_name CONSTANT VARCHAR(64) := '{1}'; -- role to copy from    user_name CONSTANT VARCHAR(64) := '{2}'; -- user to assign role to    new_role_id integer;    source_role_id integer;    new_user_id integer;BEGIN    IF NOT EXISTS (SELECT id from ab_role WHERE name = source_role_name) THEN        RAISE EXCEPTION 'Role "%" does not exist.', source_role_name;    else        SELECT id from ab_role WHERE name = source_role_name INTO source_role_id;        RAISE INFO 'Source role ID is %', source_role_id;        IF NOT EXISTS (SELECT id from ab_role WHERE name = new_role_name) THEN            RAISE INFO 'Creating role "%"...', new_role_name;            INSERT INTO ab_role(name) VALUES(new_role_name);            SELECT id from ab_role WHERE name = new_role_name INTO new_role_id;            RAISE INFO 'New role ID is %', new_role_id;            INSERT INTO ab_permission_view_role                (permission_view_id, role_id)            SELECT a.permission_view_id, new_role_id AS role_id FROM ab_permission_view_role AS a WHERE a.role_id=source_role_id;        else            RAISE WARNING 'Role "%" exists...skipping create', new_role_name;            SELECT id from ab_role WHERE name = new_role_name INTO new_role_id;        END IF;        IF EXISTS (SELECT id from ab_user WHERE username LIKE user_name limit 1) THEN            SELECT id from ab_user WHERE username LIKE user_name limit 1 INTO new_user_id;            RAISE INFO 'Assigning role to user id %...', new_user_id;            INSERT INTO ab_user_role(user_id, role_id) VALUES(new_user_id, new_role_id);        else            RAISE WARNING 'Could not find user "%"...skipping role assignment', user_name;        END IF;    END IF;END$$ LANGUAGE plpgsql;"""@task()def execute_sql_fn(sql):    try:        session = settings.Session()        result = session.execute(text(sql)).all()        return result    except Exception as e:        print(e)        return None    @dag(    dag_id=DAG_ID,    schedule_interval=None,         start_date=datetime(2022, 1, 1),    )def sql_dag():    t = execute_sql_fn(SQL_QUERY.format(NEW_ROLE,SOURCE_ROLE,USER_NAME))my_sql_dag = sql_dag()Cheers!CommentShareJohn_Janswered 10 months ago"


Question: AWSEBInstanceLaunchWaitCondition: WaitCondition timed out

"Hi,I'm in a trouble on Beanstalk with Cloudformation.I have a Cloudformation template, which creates AWS::ElasticBeanstalk::ApplicationVersion, ConfigurationTemplate and Environment, it's always completed fine.Then, executing the template mentioned above seems to try stacking another template automatically, whose name is "awseb-e-...-stack", which ends in failure.Events of "awseb-e-...-stack" shows "CREATE_FAILED" because "WaitCondition timed out. Received 0 conditions when expecting 1" for "AWSEBInstanceLaunchWaitCondition".I looked into the template for the stack and come to see "AWSEBInstanceLaunchWaitCondition" was waiting for "AWSEBAutoScalingGroup" however, I have no idea at all to solve the problem.Can anyone kindly suggest any clue, or share experiences/resolution on similar issues? Any help would be greatly appreciated.Thank you.(part of descriptions on the template for "awseb-e-...-stack": )    ...        "AWSEBAutoScalingGroup": {      "Properties": {        ...        "LaunchConfigurationName": {          "Ref": "AWSEBAutoScalingLaunchConfiguration"        },        "MinSize": "1"      },      "DependsOn": "AWSEBBeanstalkMetadata",  // the dependency resource was successfully created      "Type": "AWS::AutoScaling::AutoScalingGroup",      "Metadata": {        ...      }    }        ...        "AWSEBInstanceLaunchWaitCondition": {      "Properties": {        "Handle": {          "Ref": "AWSEBInstanceLaunchWaitHandle"        },        "Count": "1",        "Timeout": "900"      },      "DependsOn": "AWSEBAutoScalingGroup",      "Type": "AWS::CloudFormation::WaitCondition"    },        ...FollowComment"

Answer: "0Resolved.The cause was the failure of the health check from ELB to my app, it prevented completing of "AWSEBAutoScalingGroup" creation, getting rid of the problem on the health check led me to the successful execution of the template.CommentShareiizukatanswered 4 years ago"


Question: "FastAPI on Cloud9, Localhost Not Accessible"

uvicorn main:app --reload gave me the link 'http://127.0.0.1/8000' as you can see below:But clicking it throws this:How can I access to http://127.0.0.1/8000 like I can on my desktop so that I can have access to swagger?FollowComment

Answer: "0Hi, thank you for reaching out.Cloud9 application preview feature does not support insecure HTTP endpoints. You would need to switch to HTTPS to access websites served from the environment.CommentShareshcherbavanswered a year ago"


Question: Elasticache Endpoint migration stuck on 'Migrating' status

I've successfully initiated Endpoint migration for Elasticache Redis Cluster. After a few hours status is still set on 'Migrating' I'm debating if I should stop migration manually and wanted to find out more why it didn't completeFollowComment


Question: How Can we Move index date from opensearch to s3 before deleting the retention policy of that index

How Can we Move index date from opensearch to s3 before deleting the retention policy of that index by automated way ?FollowComment

Answer: 0You can configure S3 snapshot repository for backups as documented here - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-snapshots.htmlAnother tutorial that can be helpful to configure ISM (Index State Management) and snapshot - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ism-tutorial.html--SydCommentShareSydanswered a month ago


Question: "Send from"  email Address in Email Activity of Journey

"Hi,Is it possible to make the "Send From" Email address dynamic based on the endpoint [for eg. I am change the send from address based on the end points salesperson. I want to do this using APIs]regards,MahavirFollowComment"

Answer: "0Hi Mahavir,I think you could use AWS SES and programmatically define a source when sending emails. See references:https://docs.aws.amazon.com/ses/latest/APIReference/API_SendEmail.htmlhttps://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/ses-examples-sending-email.htmlHope it fits your case ;)CommentShareEXPERTalatechanswered 2 months agorePost-User-1786472 2 months agoHi Alatech,Thanks for you response. We are using the Journey Email Activity in Pinpoint to send emails. Under this we can define a default 'send from' email for each journey but not based on the end point attributes. Looking to solve this issue.Share0Hi Alatech,Thanks for you response. We are using the Journey Email Activity in Pinpoint to send emails. Under this we can define a default 'send from' email for each journey but not based on the end point attributes. Looking to solve this issue.CommentSharerePost-User-1786472answered 2 months agoalatech EXPERT2 months agoOh apologize then. Not very familiar with that product, but would this help? https://docs.aws.amazon.com/pinpoint-email/latest/APIReference/API_SendEmail.html It is kinda of similar of what I suggested above but for pinpoint. Or are you looking for something in the AWS console?Share"


Question: AWS WAF to block automated bot from stealing content.

"CiaoI am trying to use bot control to block bot and content scrapper from stealing content on client blog. After setting up WAF i realized some bot still are still stealing content and google news not working.  I am using an ALB.If i turn on Anonymous IP List, it blocks the bot but also block VPN and Proxies.Please how can i go about it, so it doesn't block service i need and use, but block automated bot from stealing content.Best RegardsFollowComment"

Answer: "0Hello,You may want to check out some of our WAF Bot Control example rules in the documentation.  One specific rule I think you are looking for is Explicitly Allow Verified Bots.  Here is the documentation to set up this rule: https://docs.aws.amazon.com/waf/latest/developerguide/waf-bot-control-example-allow-verified-bots.htmlHere is a list of example Bot Control Rules you can implement: https://docs.aws.amazon.com/waf/latest/developerguide/waf-bot-control-examples.htmlCommentShareEXPERTChris_Ganswered a year ago"


Question: Bad AMI - Passing off as Ubuntu 22.04?

"I would report this to AWS but for the life of me I can't figure out how to without having to pay for a support subscription (that I don't need).Personally I've found a (not nice) workaround for now, but this should probably be fixed.ami-084b0ab87791797baaws ec2 describe-images --region eu-west-1 --image-ids ami-084b0ab87791797ba{"Images": [{"Architecture": "x86_64","CreationDate": "2023-04-21T19:43:03.000Z","ImageId": "ami-084b0ab87791797ba","ImageLocation": "aws-marketplace/ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20230325-84dfee58-b88e-45ac-b133-db810fba9d60","ImageType": "machine","Public": true,"OwnerId": "679593333241","PlatformDetails": "Linux/UNIX","UsageOperation": "RunInstances","ProductCodes": [{"ProductCodeId": "7v6zoqj7fqa0yxobp2glr6kps","ProductCodeType": "marketplace"}],"State": "available","BlockDeviceMappings": [{"DeviceName": "/dev/sda1","Ebs": {"DeleteOnTermination": true,"SnapshotId": "snap-0e31fd25610ed18db","VolumeSize": 8,"VolumeType": "gp2","Encrypted": false}},{"DeviceName": "/dev/sdb","VirtualName": "ephemeral0"},{"DeviceName": "/dev/sdc","VirtualName": "ephemeral1"}],"Description": "Canonical, Ubuntu, 22.04 LTS, amd64 jammy image build on 2023-03-25","EnaSupport": true,"Hypervisor": "xen","ImageOwnerAlias": "aws-marketplace","Name": "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20230325-84dfee58-b88e-45ac-b133-db810fba9d60","RootDeviceName": "/dev/sda1","RootDeviceType": "ebs","SriovNetSupport": "simple","VirtualizationType": "hvm","BootMode": "legacy-bios","DeprecationTime": "2025-04-21T19:43:03.000Z"}]}Looks like an Ubuntu 22.04 image right? Wrong.Search for ami-084b0ab87791797ba in AWS market place and its:Causal AI Platform (CAP) for AI MCBy: DeepCyber Latest Version: 3.1The only give away in the API response (same via boto3 btw) is the product-code.Given that Ubuntu server is free and this image isn't.... that seems a little sneaky at best to me as any scripts selecting the latest Ubuntu image using the API will likely get stung. Luckily you have to subscribe to it first to be able to use so unlikely to do any real harm I suppose.Anyone know how to get this reported/ corrected?FollowCommentAhmet Akan a month agoThank you for reporting this issue. We have an internal ticket open to investigate this.Share"

Answer: 0I think the only way to do this is to provide feedback to AWS support as I think it would be difficult to manipulate from our side.CommentShareEXPERTRiku_Kobayashianswered a month ago0I'm not inclined to have to pay for a support subscription just to be able to report bugs to AWS. Doesn't seem like a great model that for encouraging a better cloud platform...CommentSharehamzaanswered a month ago0The Marketplace seller has provided an updated AMI with a corrected name and this AMI has been marked as deprecated:~ aws ec2 describe-images --region eu-west-1 --image-ids ami-084b0ab87791797ba | grep DeprecationTime"DeprecationTime": "2023-04-26T22:41:00.000Z"CommentShareGustavanswered a month ago


Question: "AWS Amplify: connect to external, publicly available S3 bucket"

"Is there a way to read and download files from an externally hosted, publicly available S3 bucket via the Amplify framework?FollowComment"

Answer: "0Yes, you just need to use the url or arnCommentSharejthornes-awsanswered 10 months ago"


Question: "Without ECR being enabled in AWS account at Organization Level, what's the impact to SageMaker Studio?"

"I have a customer already standardized Artifactory as the centralized image registration. They disabled ECR service at Org level. Now we want to understand the potential impact on customer's day-2-day use of SageMaker Studio as a platform to support their full ML lifecycle.(If customer only use built-in SageMaker algorithm or framework and use prebuilt SageMaker container images)Especially when customer trying to deploy the model to endpoint, does that need ECR service to be enabled in customer account?FollowComment"

Answer: "1Accepted AnswerHi, yes, if you're restricting the user to only built-in algorithms and frameworks, and prebuilt images for Studio, you should be able to use it seamlessly (to deploy endpoints as well). That said, it severely restricts the data scientist from using custom images that could be built to their needs and packages, or bringing their own container for machine learning.CommentShareDurga_Sanswered a year ago"


Question: Unsupported case of DataType

"ERROR MESSAGE: An error occurred while calling o518.pyWriteDynamicFrame. Unsupported case of DataType: com.amazonaws.services.glue.schema.types.StringType@235d3a6f and DynamicNode: integernode.I utilize glue catalog and use s3 as my DB. I'm uploading CSV files and processing them with glue jobs, however I continue to get the error message **"An error occurred while calling o518.pyWriteDynamicFrame. Unsupported case of DataType: com.amazonaws.services.glue.schema.types.StringType@235d3a6f and DynamicNode: integernode.Datatype". **I've updated and recreated the glue tables and crawlers multiple times as well as the schema to match data types but I still get the same error message. The glue job only succeeds when I update the schema to have string as the only data type but the data type needs to be int for integers when querying in Athena so this will cause an issue down stream. Can anyone help?FollowCommentR6 a year agoI had the same problem. Not sure about the cause but I was able to circumvent the error by usingformat="parquet"instead offormat="glueparquet"in my glueContext.write_dynamic_frame.from_options() callThis is probably slower, but that's not a problem in my case.Share"

Answer: "0To troubleshoot, Inspect your columns with the string data type and convert any string to integer.Refer to the considerations and limitations about using SQL queries in Athena.If the problem persists, please contact AWS Support. For more information on how to gather troubleshooting information, check reference link 3.Reference links:https://docs.aws.amazon.com/athena/latest/ug/other-notable-limitations.htmlhttps://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-common.htmlhttps://docs.aws.amazon.com/glue/latest/dg/troubleshooting-contact-support.htmlCommentShareMatthew_Ranswered 9 months ago"


Question: AWS Monitron sensor access

How can we access the AWS Monitron sensor data on local network using the gateway. We  would like to use this sensor data to take some action locally in the network. Is there any API or commands by which we can access the sensor data. If yes please share relevant links or AWS services which can be used to obtain this data.FollowComment

Answer: "1Hi,There are two ways to access your raw Amazon Monitron data outside of Amazon Monitron.You may want access your data on an ongoing basis, so that you can use it elsewhere. In that case, you can configure Amazon Monitron to automatically add your data to a Kinesis stream. From there, you can port it to various destinations, including Amazon S3 and Lambda. This process requires configuration, and that configuration requires an understanding of Kinesis Data Streams. However, once you have all the elements arranged to your satisfaction, you can keep your data streaming automatically.Or you may want to access your data once in a while, just to gain a clear understanding of what kind of data you are storing and analyzing on AWS. In that case, you can ask AWS support to manually copy your data to Amazon S3. This process requires less configuration, but it cannot be automated. It only gives you the data that Amazon Monitron has accumulated up until now, in one chunk.https://docs.aws.amazon.com/Monitron/latest/admin-guide/data-download-monitron.htmlCommentShareAWS-User-7653869answered 7 months ago0Instructions for configuring the Monitron data export via Kinesis can be found below and there is a new reference architecture diagram showing how to use this data source and connect it with other downstream systems.https://docs.aws.amazon.com/Monitron/latest/admin-guide/monitron-kinesis-export.htmlhttps://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/industrial-data-lake-for-predictive-maintenance-using-amazon-monitron-and-amazon-kinesis-ra.pdfCommentShareEXPERTBishr_Tanswered 7 months ago"


Question: What is the best practice and downtime for migrating an existing AWS environment to use the Transit Gateway?

"What is the best practice and downtime for migrating an existing AWS environment to use the Transit Gateway?For example, if there are currently separate VPCs, each with access to the Internet, which will be migrating to use the Transit Gateway with traffic sent through a central egress VPC to the Internet, what is the best practice for migrating this environment and the downtime involved? Thanks!FollowComment"

Answer: "0Accepted AnswerYou can accomplish this migration with zero downtime.Steps:Setup the TGW, attach the desired VPCs and configure the TGW route tables. (Don't update the VPC route tables yet)Setup the central egress VPC to the Internet(Optional) Create a test VPC, attach it to the TGW and test that the central egress functionality works as expectedChange the old VPC route tables and point 0.0.0.0/0 to the TGW instead of wherever it points today. The trick here is that you cannot create a second 0.0.0.0/0 entry. (Therefore create 0.0.0.0/1 and 128.0.0.0/1 and point them both to the TGW, then delete the old 0.0.0.0/0 route, create the new 0.0.0.0/0 entry and last but not least delete the two x/1 entries.)(Optional) Remove any VPC peering that you no longer need.CommentShareEXPERTChrisE-AWSanswered 3 years ago"


Question: HIVE_BAD_DATA: Error parsing field value '2022-12-14T06:51:14.710Z'

"I face this problem when trying to query from Athena, the data is stored in S3 bucket. If I exclude the timestamp column from SELECT statement, it can still be queried. Can anyone suggest a fix for this problem? Changing the access_at field in log record maybe difficult since the task is migrating logs data from RDS.HIVE_BAD_DATA: Error parsing field value '2022-12-14T06:51:14.710Z' for field 13: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]the table format is as belowCREATE EXTERNAL TABLE `user_report`(  `type` int COMMENT 'from deserializer',  `system_id` int COMMENT 'from deserializer',  `id` string COMMENT 'from deserializer',  `company_id` int COMMENT 'from deserializer',  `user_id` int COMMENT 'from deserializer',  `token` char(255) COMMENT 'from deserializer',  `device_type` tinyint COMMENT 'from deserializer',  `app_version` char(255) COMMENT 'from deserializer',  `session_cnt` int COMMENT 'from deserializer',  `requested_cnt` int COMMENT 'from deserializer',  `scheduled_cnt` int COMMENT 'from deserializer',  `rescheduled_cnt` int COMMENT 'from deserializer',  `canceled_cnt` int COMMENT 'from deserializer',  `access_at` timestamp COMMENT 'from deserializer')PARTITIONED BY (  `created_hour` string)ROW FORMAT SERDE  'org.openx.data.jsonserde.JsonSerDe'STORED AS INPUTFORMAT  'org.apache.hadoop.mapred.TextInputFormat'OUTPUTFORMAT  'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat'LOCATION  's3://demo-kinesis-athena/'TBLPROPERTIES (  'has_encrypted_data'='false',  'projection.created_hour.format'='yyyy/MM/dd/HH',  'projection.created_hour.interval'='1',  'projection.created_hour.interval.unit'='HOURS',  'projection.created_hour.range'='2018/01/01/00,NOW',  'projection.created_hour.type'='date',  'projection.enabled'='true',  'storage.location.template'='s3://demo-kinesis-athena/${created_hour}',  'transient_lastDdlTime'='1671014054')FollowComment"

Answer: "3Accepted AnswerMy suggestion would be to define the column as text as you are unable to convert before storing the timestamp.You could use something similar to the below query to parse it into a date or use the CAST functionselect date_parse(substr('2022-12-14T06:51:14.710Z',1,24),'%Y-%m-%dT%H:%i:%s.%fZ') CommentShareAnanth Tirumanuranswered 5 months agoEXPERTFabrizio@AWSreviewed 5 months ago"


Question: Route 53 DNS Firewall - Wildcard / CNAME issue

"We're using DNS firewall and using an allowed list acceptable outbound domains.  In some cases where there are multiple subdomains like sec.gov so we use a wildcard *.sec.gov.  www.sec.gov is a CNAME record which returns www.sec.gov.edgekey.net. and the subsequent lookup of that fails unless we add that to our allowed list as well.  Makes sense.  There have a few domains where the target of the CNAME has changed and we can only update after the DNS lookup has failed and we are alerted.  I'm just wondering if there is any way to be proactive with this, by somehow allowing the *.sec.gov to cover any CNAME it may encounter.FollowComment"

Answer: "1Given that you have no control over the target DNS and you want to have a very strict allow-list of things that you can connect to you're going to need to set something up that verifies the validity of your allow-list against the firewall.For example: Have a list (probably external to DNS Firewall) that has all of the site/DNS records that are allowed. Look each one up. If it is a CNAME, find the record that is returned and add that to the list and therefore to the firewall as well.You could do this manually, but far better to automate it.Note that this isn't going to be perfect. Even if you run it at one minute intervals you may have some clients blocked as changes occur. And it's entirely possible that the web pages that users are reaching are going to have other sites that they link to so there is a larger problem here. Restrictive allow-lists are always going to be a lot more work to maintain than is originally thought.CommentShareEXPERTBrettski-AWSanswered a year ago"


Question: Step Functions Local with Boto3

"Hello,I am working with AWS Step Functions and we are exploring the testing options. AWS Step Functions local seems like the way to go. I have tested it out locally as outlined in this guide, which works well. That requires using the AWS CLI to interact with SFN local. We are a Python shop so being able to continue using boto3 for interactions with AWS would be great. Is there anyway to use boto3 to query the local SFNs? I haven't seen any documentation about this and I've tried but the latest boto3 (1.24.71) doesn't seem to know about endpoints.ThanksFollowComment"

Answer: "1Accepted AnswerHi,I believe you can specify the endpoints as followsimport boto3client = boto3.client('stepfunctions', endpoint_url='http://localhost:8083')https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.htmlCommentSharehayao-kanswered 9 months agoacrookes-fe 9 months agoThat did it. Thank you!Share0I don't think there is a way and I must say that I do not understand why you need it. We are talking about testing the state machine. So all you need to do is create the input JSON and invoke it locally. You are not running integration tests locally which require some other service to invoke the state machine.CommentShareEXPERTUrianswered 9 months agoacrookes-fe 9 months agoThe thought around using Python is that we already use it for our automated testing framework and it would be nice to not have to introduce another paradigm. With boto3 support we could start Step Functions Local and then ,in Python, provide the input to the local state machine and then inspect the output to ensure it conforms to our expectations. If anything goes wrong we could use the standard errors and patterns our team is already familiar with to signify a failed test. This can certainly occur by calling out to the AWS CLI from Python but that is less than ideal.If this still doesn't fit a pattern that makes sense to you could you outline how you would go about structuring automated testing of Step Functions?Share"


Question: Adding Template Data when sending Custom Verification Emails

"Hi,So when sending an email, I want to be able to provide data so that the template body (template content) containing the html code can utilize the curly brackets {{}} so that it gets replaced with the provided data.I know this can easily be done when sending a templated email by providing TemplateData  when using the API call, but I want to do the same when sending a custom verification email when using the API call: SendCustomVerificationEmail.Does anyone know if this can be done at all? Or are you completely restricted with a pre-defined template when using Email Verification Templates?FollowComment"

Answer: "1Accepted AnswerYou could add your template before sending the email using the API call CreateCustomVerificationEmailTemplate (https://docs.aws.amazon.com/ses/latest/APIReference/API_CreateCustomVerificationEmailTemplate.html), then refer to that template when calling SendCustomVerificationEmail.If you are not using the template again, you can delete it after sending the email using DeleteCustomVerificationEmailTemplate, but make sure the email has been sent and is not being sent asynchronously else you might delete the template before its being used!CommentShareRollerblade7answered a year agoCorey-Lightmare a year agoAhh perfect! Yeah I didn't think of that. That'll work, I'll just make sure to wait for the callbacks. Cheers.SharerePost-User-1872572 a year ago@Rollerblade7 Does this mean one has to interpolate TemplateData programatically before creating the Template and then send it calling SendCustomVerificationEmail ?ShareCorey-Lightmare a year agoYes. So when updating the Custom Verification Email Template, it takes the 'TemplateContent' as an entire string, so I just make sure to format the string in code with the data I want in it before I make the request to update the email template. Then when it finishes updating, I then immediately send the verification email using that updated template.To format the TemplateContent string, I make sure to include the {0} symbols within the html string. You essentially have to implement the TemplateData behaviour yourself for custom verification email templates.Share"


Question: Getting ERR_TOO_MANY_REDIRECTS after create load balancer and SSL

"Hi, I have an instance on AWS Lightsail with Wordpress which was working OK with domain(Route53). I created a load balance, attached the instance, installed a SSL Certificate, and updated the Route53 with the ALB dns name, and after that I've been getting ERR_TOO_MANY_REDIRECTS at the browser. I've been reading in forums how to stop the redirection and I've applied the recommendations at files .htaccess, wp-config.php, httpd.conf, and bitnami.conf but none of them have helped me out.Is there a way to know where the redirection is happening?Any help is more than welcome!Thanks!Edited by: Fernando-developer on Nov 24, 2021 7:27 AMEdited by: Fernando-developer on Nov 24, 2021 7:32 AMFollowComment"

Answer: "0Hi, an update here. I created another instance and killed the previous one and problem solved. I had to copy over my website content.CommentShareFernando-developeranswered 2 years ago"


Question: AWS S3 tar.gz compressed files issue after download from S3

"Hello,I would like to check with you guys here in AWS forum the issue that I discovered a while ago. Basically I'm trying to test our backup recovery process. Below is my simple flow of our backing up process.Backup Flowec2 -> uploads backup files compressed using tar.gz to S3 -> S3(backup bucket).NOTE: I used AWS CLI for moving files from ec2 to s3.However when I tried to download the tar.gz file from S3, considering that on the AWS Console S3 file list it's has a correct file extension of tar.gz e.g. backup1.tar.gz BUT after the download, I've noticed that my backup file turns to backup1.tar (not tar.gz) so when I tried to decompress the backup its not usable or readable etc.I'm not sure if it's on my OS level error but overall it will make a huge confusion to the user if the backup process is working fine or not.FollowComment"

Answer: "0Why don't you use native AWS tool for creating snapshots? Why you make the process of backup your EC2 instance so complex?)Moreover you can automate all this by using any backup vendors tools. For example, Cloudberry Backup Server Edition.CommentShareAlViKoanswered 6 years ago0I'm having the same problems...I did aws s3 sync some/local/folder s3://my-special-bucket/some/local/folderand some of the files that got uploaded were .tar.gz files. Well, AWS decided that those should receive the metadata Content-Type: applicaton/x-tar instead of Content-Type: application/x-gzip. Highly aggravating. I had to write  a script to list every file in my bucket and then fix the metadata for any file with the .tar.gz extension.CommentShareDavidZemonanswered 4 years ago0I have yet to see anyone explain how to download a tar.gz from an S3 bucket without AWS changing the format to a .tar and changing the config of the files. I need help to figure out how to down load a large (1.7 GB) tar.gz file.CommentShareDSimonanswered 3 years ago0Hello All,I have the same issue, anyone has any solution or suggestion?CommentShareanjalitrambadiaanswered 3 years ago0Hi anjalitrambadia, DSimon,We took a quick look over the sync issues and wanted to get some more information.If you wouldn't mind creating an issue on the AWS-CLI's Github (https://github.com/aws/aws-cli/), that will help us better track this. Including the version of your aws cli (aws --version) and a brief set up steps for reproduction will be really useful.Thanks,AWS-CLI teamEdited by: nateprewitt on May 12, 2020 6:11 PMCommentSharenateprewittanswered 3 years ago0You should use --content-encoding "application/x-compressed-tar" option during S3 uploading:aws s3 cp  --content-encoding "application/x-compressed-tar".......CommentSharedastafurovanswered 2 years ago"


Question: event bridge event visualization

"event bridge event visualization : is there a way to see the event sent to an event bridge and also get ack , nack ?FollowComment"

Answer: "0You can add a CloudWatch logs target to a rule that catches all the events. You will then be able to see the events in CloudWatch Logs.Not sure what you mean with the get back Ack/Nack. Get from where?CommentShareEXPERTUrianswered a year agorePost-User-0364151 a year agoNot cloudwatch logs , some other source or where the events are stored or queued within event bridge.Ack / back from event bridge saying it has received the event / msg - one way is that when we putevent we get if its success or failureShareUri EXPERTa year agoYou do not have visibility into the internal works of EventBridge. You have some metrics that tell you what happened.You always get an ACK (200 OK) when EventBridge receives the message, otherwise you should retry sending it. The PutEvent API is synchronous. If it return 200 OK, it means the message was received.SharerePost-User-0364151 a year agoso basicallywe configure belowarchiveDLQmetric on  event bridgelogs of the put event from codeand create our own visualization so that we have a detailed pictureShare"


Question: Does WordPress work on T4G.nano (ARM) instance?

"I have followed the steps for Amazon Linux 2023 4 times now (terminating and starting new).  They simply do not work on T4G instance.  Is this not supported?  Are the instructions incomplete?https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/hosting-wordpress-aml-2023.htmlWhen I try to follow step:In a web browser, type the URL of your WordPress blog (either the public DNS address for your instance, or that address followed by the blog folder). You should see the WordPress installation script. Provide the information required by the WordPress installation. Choose Install WordPress to complete the installation. For more information, see Step 5: Run the Install Script on the WordPress website.I getThis site can’t be reachedec2-###-##-###-##.compute-1.amazonaws.com refused to connect.Try:Checking the connectionChecking the proxy and the firewallERR_CONNECTION_REFUSEDI have already created a security group to allow inbound 22, 443, and 80.FollowComment"

Answer: 0Accepted AnswerFinally got it working by not installing full lamp and self signed cert first.  Did not see way to delete this question.CommentShareKisnardOnlineanswered a month ago


Question: How to create simple Athena table from simple JSON data?

"I have simple JSON log data like:{   "referer":"https://example.org",  "data": {    "datum1": "value1",    "datum2":"value2",    "ts":2381738917239  }}How do I get that stuff easily in the Athena so I can type:select * from mytable where data.ts > '2022-06-01 and datum1 = 'filter'The documention is really comprehensive (spent a couple of hours browsing it), but I am really not that interesting about knowing all the SerDe details, I just need to get this simple thing going and preferrably fast. Like in Snowflake (took a minute or two to do it).BR, JoniFollowComment"

Answer: "1I was able to get a crawler to crawl the data and write a quick query in Athena to be able to accomplish your use case effectively. Please take a look and try it out. If you have any questions, please let me know.Step 1: I mocked up data for 10 different dates and moved the data in S3Step 2: I created a crawler to crawl the S3 location and ran the crawler. It created a table with 2 columns referer and data.Step 3: I ran the athena query below to get the dates > 9/4/2022SELECT referer, data.datum2, data.ts, data.datum1, date_format(from_unixtime(data.ts),'%Y-%m-%d') FROM "raw_db"."nested_json" where date_format(from_unixtime(data.ts),'%Y-%m-%d') > '2022-09-04'limit 10;CommentShareAnanth Tirumanuranswered 8 months ago"


Question: Cannot login to WordPress Account hosted on LightSail

I have been locked out of my WordPress admin portal which is set up on a AWS LightSail instance. When I enter the password given to me by sshing into the instance and using the admin portal for the domain linked to the instance it tells me the information is incorrect. When I try to reset the password through the terminal it does not fix anything and still gives all the same errors. I have tried everything I can find to regain access to the site to no avail. I also noticed that all of the drafted pages not meant to be published quite yet have all been published.FollowComment

Answer: "1Were you able to log in before?In that case, it may be possible to restore and use a snapshot of LightSail, if one is left.It is also possible to change the password of a WordPress user directly by executing SQL statements, if it is possible to access the database used by WordPress.However, work with databases can be dangerous and should be done with caution.A sample SQL statement is provided below.UPDATE wp_users SET user_pass = MD5('New Password') WHERE user_login = 'WordPress Admin User';CommentShareEXPERTRiku_Kobayashianswered 14 days agoFLPolyCon 14 days agoI have been able to login before, the instance has been running for over half a year with no issues and only recently and suddenly stopped allowing me to login. How would I go about running from a snapshot without completely messing up my website as I do not wish to cause any damage to the site.ShareRiku_Kobayashi EXPERT14 days agoIt is possible to create a new LightSail from a snapshot using the method in this document.https://lightsail.aws.amazon.com/ls/docs/en_us/articles/lightsail-how-to-create-instance-from-snapshotBy the way, just to confirm, is the WordPress database running inside LightSail?In that case, you can restore LightSail from the snapshot, but if you have a WordPress database outside of LightSail using RDS or other means, you will need to restore the RDS from the snapshot.ShareFLPolyCon 13 days agoIt appears I do not have any snapshots to recover from as I never took a manual one and the automatic was turned off. Is there a way to fix my login without restarting the entire instance or remaking it from a backup?ShareRiku_Kobayashi EXPERT13 days agoIf we can't modify the password from the WordPress screen, the only way is to change the database directly.Share1Hello,You can modify the WordPress password through an SSH session and the use of the WordPress CLI.  The Lightsail console provides a web-ssh client you can open in 1-click.Once in the instance, you need to execute the following command:Command:sudo wp user update user --user_pass=YourNewPasswordGoesHere --skip-emailExpected output:Success: Updated user 1.If you have other user(s) and want to change their password, you need to supply the correct name.  In that case, please replace "UserName" and "YourNewPasswordGoesHere" with their respective values.sudo wp user update UserName --user_pass=YourNewPasswordGoesHere --skip-emailThe above would literally change the password for your default user, which is named user.  The new password would be "YourNewPasswordGoesHere" so please replace that with your chosen new password.I hope this resolves your issues.  Please advise if you are still blocked.-AndyCommentShareAWS-Andyanswered 8 days agoAWS-Andy 8 days agoIf you have additional wordpress user issues you may be able to address them through the wp cli.  The relevant "wp user" command is documented here: https://developer.wordpress.org/cli/commands/user/ShareFLPolyCon 8 days agoWhen I run the command, switching out the "YourNewPasswordGoesHere" for a new password, it seems to just run indefinitely without ever actually updating the password.ShareAWS-Andy 7 days agoI have not encountered an issue with WP CLI hanging before so I can't speak to that.  Have you tried using the "wp user..." commands to do other actions?  You may want to try creating an entirely new admin user and see if that works.  If it does, you could then delete other user(s).If you omit user_pass it'll generate a strong password for you automatically.wp user create newName newEmail@example.com --role=administrator --user_pass=YourNewPasswordSuccess: Created user 2.At that point I'd also check your user list and see if you have extras to delete.Share"


Question: Access Parameter Store Values in EventBridge Input Transform

"I currently have an input transformer that is modifying an event and forwarding on certain values to an API Destination. I am hoping that it is also possible to access some values in parameter store, so that I can also add these values to my transformed event.Is this possible?FollowComment"

Answer: 0I do not believe that is possible.  You may be able to hardcode the parameter name as part of the input transformer but to actually lookup the parameter would have to be done by the target/destination.Other option is to create your own schema that includes the value needed already:https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-schema.htmlCommentSharealekse_wanswered a year ago


Question: Unable to activate AWS Genomics CLI

"I would like to use the AWS Genomics CLI, but have failed to activate it both locally and on an EC2 instance. I do not have permissions to alter my IAM roles, and my IT team has tried to setup my account to work based on the info provided in the Setup documentation. However, I have had no luck. I have tried with both existing VPC and S3 bucket, as well as without. Any help is appreciated. I would like to use AWS's resources! I am pasting one error below.agc account activate2022-06-17T18:19:28+02:00 𝒊  Activating AGC with bucket '' and VPC ''Bootstrapping CDK... [--o-] 51s                                                                                2022-06-17T18:20:21+02:00 ✘   ⏳  Bootstrapping environment aws://272554863871/us-east-2...2022-06-17T18:20:21+02:00 ✘  Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit: creating CloudFormation changeset...2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:43 PM | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack | Agc-CDKToolkit User Initiated2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:50 PM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | Agc-CDKToolkit User Initiated2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::ECR::Repository  | ContainerAssetsRepository 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role        | FilePublishingRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter   | CdkBootstrapVersion 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role        | CloudFormationExecutionRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role        | LookupRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::S3::Bucket       | StagingBucket 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role        | ImagePublishingRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role        | FilePublishingRole Resource creation Initiated2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:55 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role        | LookupRole Resource creation Initiated2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:56 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role        | CloudFormationExecutionRole Resource creation Initiated2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:56 PM | CREATE_FAILED        | AWS::S3::Bucket       | StagingBucket cdk-agc-assets-272554863871-us-east-2 already exists2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:56 PM | CREATE_FAILED        | AWS::IAM::Role        | FilePublishingRole Resource creation cancelled2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:56 PM | CREATE_FAILED        | AWS::IAM::Role        | ImagePublishingRole Resource creation cancelled2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:56 PM | CREATE_FAILED        | AWS::IAM::Role        | LookupRole Resource creation cancelled2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:56 PM | CREATE_FAILED        | AWS::IAM::Role        | CloudFormationExecutionRole Resource creation cancelled2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:56 PM | CREATE_FAILED        | AWS::SSM::Parameter   | CdkBootstrapVersion Resource creation cancelled2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:57 PM | CREATE_FAILED        | AWS::ECR::Repository  | ContainerAssetsRepository Resource creation cancelled2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:19:57 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | Agc-CDKToolkit The following resource(s) failed to create: [ImagePublishingRole, FilePublishingRole, CdkBootstrapVersion, LookupRole, StagingBucket, CloudFormationExecutionRole, ContainerAssetsRepository]. Rollback requested by user.2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:20:10 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role        | CloudFormationExecutionRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:20:10 PM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter   | CdkBootstrapVersion 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:20:10 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role        | FilePublishingRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:20:10 PM | DELETE_SKIPPED       | AWS::S3::Bucket       | StagingBucket 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:20:10 PM | DELETE_IN_PROGRESS   | AWS::ECR::Repository  | ContainerAssetsRepository 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  0/12 | 6:20:10 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role        | LookupRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  1/12 | 6:20:10 PM | DELETE_COMPLETE      | AWS::IAM::Role        | ImagePublishingRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  2/12 | 6:20:11 PM | DELETE_COMPLETE      | AWS::SSM::Parameter   | CdkBootstrapVersion 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  3/12 | 6:20:11 PM | DELETE_COMPLETE      | AWS::IAM::Role        | FilePublishingRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  4/12 | 6:20:11 PM | DELETE_COMPLETE      | AWS::ECR::Repository  | ContainerAssetsRepository 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  5/12 | 6:20:11 PM | DELETE_COMPLETE      | AWS::IAM::Role        | CloudFormationExecutionRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  6/12 | 6:20:12 PM | DELETE_COMPLETE      | AWS::IAM::Role        | LookupRole 2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit |  7/12 | 6:20:13 PM | ROLLBACK_COMPLETE    | AWS::CloudFormation::Stack | Agc-CDKToolkit 2022-06-17T18:20:21+02:00 ✘  2022-06-17T18:20:21+02:00 ✘  Failed resources:2022-06-17T18:20:21+02:00 ✘  Agc-CDKToolkit | 6:19:56 PM | CREATE_FAILED        | AWS::S3::Bucket       | StagingBucket cdk-agc-assets-272554863871-us-east-2 already exists2022-06-17T18:20:21+02:00 ✘   ❌  Environment aws://272554863871/us-east-2 failed bootstrapping: Error: The stack named Agc-CDKToolkit failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE2022-06-17T18:20:21+02:00 ✘      at waitForStackDeploy (/Users/Moneill/.agc/cdk/node_modules/aws-cdk/lib/api/util/cloudformation.ts:311:11)2022-06-17T18:20:21+02:00 ✘      at processTicksAndRejections (node:internal/process/task_queues:96:5)2022-06-17T18:20:21+02:00 ✘      at prepareAndExecuteChangeSet (/Users/Moneill/.agc/cdk/node_modules/aws-cdk/lib/api/deploy-stack.ts:376:26)2022-06-17T18:20:21+02:00 ✘      at /Users/Moneill/.agc/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:575:242022-06-17T18:20:21+02:00 ✘      at async Promise.all (index 0)2022-06-17T18:20:21+02:00 ✘      at CdkToolkit.bootstrap (/Users/Moneill/.agc/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:572:5)2022-06-17T18:20:21+02:00 ✘      at initCommandLine (/Users/Moneill/.agc/cdk/node_modules/aws-cdk/lib/cli.ts:342:12)2022-06-17T18:20:21+02:00 ✘  2022-06-17T18:20:21+02:00 ✘  The stack named Agc-CDKToolkit failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE2022-06-17T18:20:21+02:00 ✘   error="exit status 1"Error: an error occurred invoking 'account activate'with variables: {bucketName: vpcId: publicSubnets:false customTags:map[] subnets:[] amiId:}caused by: exit status 1FollowComment"


Question: EC2 Dashboard says I have NO INSTANCES

"I only login to AWS every 6 months or so. I'm running -- and have been -- two instances for the past several years. I get billed -- and have been -- every month. The instances show on the billing, but the EC2 Instances Dashboard says I have no instances. I know they're both running, but why don't they show in my Dashboard?FollowComment"

Answer: "0Crikey, I forgot to make sure I was looking in the proper Zone. Problem solved. Doh!CommentShareJeff Scottanswered 3 years ago"


Question: Java Spring Framework + Cognito; Cognito works locally fails with deploymen

"I have a Spring framework based application; When I run the app locally with callback/redirect url as localhost:8080 , the flow of signing in and being redirected back to the home page works. However when I deploy using CodeStar after attempting to sign in and updating the callbackurl/redirect url to the correct hostname the browser get a timeout error , and DevTools in chrome reports failed.edit: the same happens when i test in the Cognito UI using the hosted UI link.if it helps development.furrerindustries.com, I can also provide any other data that would be of assistance.Forgive me if I have committed a forum crime , I tried searching for this in the forum and could not find anything.Edited by: AdamFurrer on Dec 10, 2020 9:59 AMFollowComment"

Answer: 0Misconfiguration AWS Support helped withCommentShareAdamFurreranswered 2 years ago


Question: details of IAM role which is attached to ec2 instances with cloudshell

"I want view details of IAM Role to how many instances it is attached to with Cloudshell, cli which commands should use give example.lets assume I have IAM Role TestRole I want to know to how ec2 instances TestRole is attached to.FollowComment"

Answer: "1Accepted AnswerYou will need to run this against all of your active regions to get a full count.aws ec2 describe-instances \    --output text \    --region us-east-1 \    --query Reservations[*].Instances[*].InstanceId \    --filters Name=iam-instance-profile.arn,Values=$(aws iam list-instance-profiles-for-role \        --role-name TestRole \        --query InstanceProfiles[*].Arn \        --output text) | wc -lCommentShareEXPERTkentradanswered a year agoAWS-User-6176623 a year agoThanks for saving so much time man. last query  want view ec2 instances with no Iam role attached.Sharekentrad EXPERTa year agoaws ec2 describe-instances --query 'Reservations[*].Instances[?IamInstanceProfile==null].InstanceId' --output textShareAWS-User-6176623 a year agoThanks again Forgot to tell I removed last thing " | wc -l " once again that this command too working great......aws ec2 describe-instances --output text --region us-east-1 --query Reservations[].Instances[].InstanceId --filters Name=iam-instance-profile.arn,Values=$(aws iam list-instance-profiles-for-role --role-name TestRole --query InstanceProfiles[*].Arn --output text)Share1I am a newbie and did a little bit of research. I found this. See if it helps.Find instance profiles linked to role.https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/list-instance-profiles-for-role.html.Then find the instance profile and instance Id associationshttps://docs.aws.amazon.com/cli/latest/reference/ec2/describe-iam-instance-profile-associations.htmlCommentShareSandeepVudataanswered a year ago"


Question: Routing network traffic between two EC2 instances in the same subnet to a firewall appliance in another VPC

"Hi Team,I have an application VPC with two private Subnets in the same Availability Zone. Subnet A contains multiple EC2 instances. Subnet B is a transit gateway subnet that connects to a firewall VPC that contains a firewall appliance to analyze and control network traffic. In this example the firewall VPC will be a hub and spoke model. To enhance security I would like all traffic that goes between EC2 instances inside Subnet A to be routed to the firewall VPC for inspection. The firewall appliance would need to remain inside the firewall VPC. Is this type of configuration possible?FollowComment"

Answer: "1Accepted AnswerNo, this is not possible. You can insert inspection appliances between subnets, but not within the same subnet. More details here: https://aws.amazon.com/blogs/aws/inspect-subnet-to-subnet-traffic-with-amazon-vpc-more-specific-routing/Can you create different subnets for different kind of workloads?Also, you may want to consider a multi-AZ deployment for resiliency. Especially if this is for production workloads.CommentShareACanswered a year agoEXPERTBrettski-AWSreviewed a year ago0Good evening,Could this be a possible solution for you?Traffic Mirroring is an Amazon VPC feature that you can use to copy network traffic from an elastic network interface of Amazon EC2 instances. You can then send the traffic to out-of-band security and monitoring appliances for:Content inspectionThreat monitoringTroubleshootingThe security and monitoring appliances can be deployed as individual instances, or as a fleet of instances behind a Network Load Balancer with a UDP listener. Traffic Mirroring supports filters and packet truncation, so that you only extract the traffic of interest to monitor by using monitoring tools of your choice.https://docs.aws.amazon.com/vpc/latest/mirroring/what-is-traffic-mirroring.htmlhttps://aws.amazon.com/blogs/aws/new-vpc-traffic-mirroring/CommentSharesharpedgewvanswered a year ago"


Question: Upgrade NVIDIA drivers Sagemaker notebook instances

"Hi, Is it possible to upgrade the NVIDIA drivers on the g4dn series of instances on Sagemaker? They currently have 510 which has a max cuda of 11.6. Pytorch 2 is soon to be released and will only support >=CUDA 11.7.  Without an upgrade these instances won't be able to use pytorch 2. What is the correct command to run to upgrade these?ThanksFollowComment"


Question: EC2 image builder creating AMIs with encrypted volumes

"I am trying to create EC2 AMI images using EC2 pipeline builder. Need to host these AMIs on AWS marketplace for distribution.Issue is - the AMI that is created automatically creates a root volume that is encrypted. AWS Marketplace doesnt allow AMIs with encrypted volumes.I have tried disabling encryption on the volumes in EC2 pipeline builder recipe, but doesnt helpLooking for suggestions that will help me create AMI with unencrpted root volumes.FollowComment"

Answer: "0Accepted AnswerThank you for the response Shashank! Looks like my AWS account had default EBS encryption enabled, so every EC2 + EBS that was created was getting encrypted. Had to disable this global setting to get an unecrypted EBS and thus an AMIhttps://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.htmlCommentShareaditya_kotharianswered 5 months agoShashank Abhishek 5 months agoyes thats another option we have by default enabled.. Great to hear that you are good now..Share0Are you using custom AMI , generally when we use custom AMIs we encrypt the associated volumes , can you please check your custom AMI and if thats encrypted the image builder will create new AMI based on that image itself which results with encryption.CommentShareShashank Abhishekanswered 5 months ago"


Question: Identitystore ListUsers and ListGroups API results in UnknownOperationException

"Hi,Reproduced in 2 environments:ADConnector->AWS SSOMicrosoftAD->AWS SSOProvisioning method is ADSyncTested with latest versions of boto3 and AWS cli both.When I call ListGroups / ListUsers and other API methods of IdentityStore, I'm facing different errors:When ListUsers / ListGroups without Filters - getting error from the server that I must add filters although this argument was deprecated:self._client.list_users(IdentityStoreId=self._identity_store_id)2023-01-29 15:36:20,909 urllib3.connectionpool [DEBUG] Resetting dropped connection: identitystore.us-east-1.amazonaws.com2023-01-29 15:36:21,351 urllib3.connectionpool [DEBUG] https://identitystore.us-east-1.amazonaws.com:443 "POST / HTTP/1.1" 400 63*** botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ListUsers operation: Filter is requiredYou can see in the logs that a request has been made and it's not the SDK that blocks me from listing users without filters.Second issue is when listing group memberships, getting UnknownOperationException - (the UserId is correct and works with DescribeUser API call).This reproduces both in ListGroupMemberships and ListGroupMembershipsForMember:self._client.list_group_memberships_for_member(IdentityStoreId=self._identity_store_id, MemberId={'UserId': 'REDACTED'})2023-01-29 15:39:42,371 urllib3.connectionpool [DEBUG] https://identitystore.us-east-1.amazonaws.com:443 "POST / HTTP/1.1" 400 38*** botocore.exceptions.ClientError: An error occurred (UnknownOperationException) when calling the ListGroupMembershipsForMember operation:I have only read access to this AWS account.Thanks.FollowComment"

Answer: "0It seems that the issue with the ListUsers and ListGroups methods is that they require filters, which are now mandatory, and the deprecated argument has been removed.As for the ListGroupMemberships and ListGroupMembershipsForMember methods, the error seems to be related to an "UnknownOperationException".Additionally, it is possible that these issues are related to permissions, and it may be necessary to check the IAM roles and policies attached to the AWS account being used.CommentShareDivyam Chauhananswered 4 months agoohad 4 months agoI'm sorry but it's seems that you answered the question only to gain points here.The filters argument is no longer required.Also, I can guarantee that I have sufficient permissions as this issue does not reproduce on many other environments (some with Active Directory and some aren't) - with the same permissions (and DescribeUser/DescribeGroup working perfectly).Share"


Question: Security Update in AWS Bitnami WordPress

Can advise how do I update the security patch for individual AWS Lightsail Bitnami WordPress?FollowComment

Answer: "0First, you want to login to your WordPress admin account. Once you have logged in click on Dashboard > Updates. You will be taken to the WordPress dedicated update page in the admin area where you can view all updates that are available.https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-tutorial-launching-and-configuring-wordpressCommentShareAWS-User-2776000answered 9 months ago"


Question: Why is Route 53 showing me a price of 0.00 for a .ch domain transfer?

"I'm trying to transfer a .ch domain to Route 53, but while the .ch TLD is in the list of supported domains (under Europe) and is also detailed here, Route 53 says the price is US$0.00.Is this a bug or is it actually free, since the domain isn't up for renewal until February, 2023?FollowComment"

Answer: "0This is not a bug, AWS does not charge to transfer ".ch" domain.You can find the price for transfer/ registration/ ownership change of AWS supported domains in https://d32ze2gidvkk54.cloudfront.net/Amazon_Route_53_Domain_Registration_Pricing_20140731.pdfCommentShareGGanswered 7 months ago"


Question: I can't access some web site with curl command on several Ubuntu server in EC2

"I can't access some web site with curl command on several Ubuntu server in EC2curl command is timeout.I checked followingDNS by dig : OKtraceroute : OKIP black list : OKcurl for other web site : OKsecurity : OKFirewall : OKSo, I don't know what else I need to check.FollowComment"

Answer: "1I wonder if some sites are blocking because of a lack of reverse DNS for the elastic IP? You might want to try this procedure, Create a reverse DNS recordCommentShareEXPERTkentradanswered a month ago0When I changed elastic IP to other, I can access with curl.So, it looks an elastic IP has an issue...But I still don't understand root cause.My IP is not in IP black list. And I think usually web site doesn't stop access from normal IP address.So, I hope to support to find root cause.CommentShareAWS-User-2922947answered a month ago"


Question: Redshift MV issue – mv_version != NULL

"Hey guys,I stumble upon this error while creating Materialized Views.Every second dbt job on Redshift cluster gets failed due to this error.Database Error in model stg_uploads (models/staging/stg_uploads.sql)  Assert  DETAIL:      -----------------------------------------------    error:  Assert    code:      1000    context:   mv_version != NULL -    query:     4441711    location:  refresh.cpp:592    process:   padbMAIN [pid=20121]    -----------------------------------------------Error message doesn't seem to be very informative.Can anybody explain what could be the potential cause and what should I do to get rid of this error?FollowCommentAnusha_C a year agoIs the possible for you to share the SQL/Code you used for the MV. You can get it based on the below query:select * from STL_QUERYTEXTwhere query = 44417114441711 is the queryID from the error message you posted. If you can't find records for that queryID, it would mean that logs for that query have been purged, in which case pick a latest query with this issueShare"


Question: Some addresses do not receive SES verification emails

"I added the address I want to be the sender on the SES page, and there is an address that I did not receive the verification email when verifying the email address, and I am in trouble because I do not know the reason why I can not receive it no matter how many times I resend it. The target address is an address that can be sent and received by a mailer etc. without any problem. The domain after @ of the unreceivable address is the same domain. What kind of problem can you think of?FollowComment"

Answer: "0AWS send you verification email by using from address "no-reply-aws@amazon.com". There could be below mentioned possible issue:If there is any filtering at recipient end on the basis of above mentioned email address, domain or IP address. Please cross verifyCheck spam boxIf domain (related to email address that you want to verify) is verified in SES and SES MX is being used for incoming emails then check if there is any active rule set which is blocking the emails.Are you facing this issue with some specific email addresses of a specific domain or with overall domain. For example able to verify abc@example.com but not able to verify xyz@example.com OR you are not able to verify any address of example.comThis will help us to troubleshoot further. If facing this issue with whole domain then we should check the MX and if there are any specific filters on the basis of From email address and domain.If still facing same issue then share the below mentioned information and open support case with AWSEmail address where you are not getting verification linkTime stamp when you tried recentlyCommentShareSumit Bhardwajanswered a year ago"


Question: How do I install the unlimited strength policy for JCE on AWS Workspaces?

"So I'm using AWS Workspaces as my development environment, and Java lives in C:\Program Files\Java\jdk1.8.0 but I can't copy any data into that drive.  I need to install the JCE Unlimited Strength Cypto Policy which requires me to copy a couple of jars into $JAVA_HOME\lib\security, but I can't.  So how is it done?FollowComment"

Answer: 0I figured it out.  I have to run Cmd using Run As Administrator and it'll let me copy the files then.  Derp!CommentSharechubbsondubsanswered 3 years ago


Question: Linux aws ec2  and Ubuntu t2.micro instances failed to update and install any software

I  have launched today two Linux ec2 t2.micro instances and one Ubuntu t2.micro instances in us-west-2 region. All there did not either update the software or install java-11-openjdk. I have sent the complain to aws and they just gave me a like to the aws re:Post site.  It was not my problem. I launched dozen of ec2 instances before and everything was OK. My home network was working fine. I was connected to the instances. The inbound and outbound security rules were set to accept HTTP traffic (TCP port 80 was open). FollowComment

Answer: "0Please make sure of below 2 things.First of all you need confirm if you have launched EC2 instances in public or private subnet ?If its in public subnet, verify route table associated with the subnet has route towards Internet Gateway.DestinationTarget0.0.0.0/0igw-idIf it is in private subnet then make sure the route table associated with the subnet has route towards NAT Gateway.CommentSharerePost-VivekPophaleanswered 9 months ago"


Question: install agent in my locaL PC

i need to install agent in my pc win 10 64bit 8gb 500ssdfor backup  to aws s3can i do?FollowComment

Answer: "1One option is to use the AWS CLI and the S3 sync command to automatically copy files to S3. The nice thing about the sync command is that it will compute a delta since last run, meaning that any new files or updated files will be picked up instead of re-uploading every single file.For example, if I wanted to automatically backup all of my photos from my local machine, I would run the following command:aws s3 sync $HOME/Pictures/ s3://mybucket/Pictures/ --include "*.jpg"The above command will "sync" my local Pictures directory into provided S3 bucket URI, and will include any files under that directory that have the .jpg filename suffix.Then if you wanted to automate the process, you could execute the above command as a scheduled task.CommentShareEXPERTDavid Maloneanswered 10 months ago"