Question: SSM Patch Manager Scan Error: "InsecureRequestWarning: Unverified HTTPS request is being made"

"Hello everyone,I'm using ssm patch manager Scan task (AWS-RunPatchBaseline) once a week on some EC2 instances, some Ubuntu 20 and some AmazonLinux2.Everything was fine since the 16th of January, as the task would complete successfully for every instance. Starting from the 23th of January, the task fails on every Ubuntu instance with the following error on the standardError result:"/var/log/amazon/ssm/patch-baseline-operations/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning)\nfailed to run commands: exit status 1".Fun fact: the same identical warning is also there on the AmazonLinux instances, but the command does not fail with exit code 1.I don't think anything changed on those Ubuntu instances in the meantime, but, maybe some unattended upgrades.I already tried the last SSM-agent available version 3.2.532.0, and re-do the Scan, but the outcome is the same.What could it be the problem, and how could it be to solved?I leave here the ssm-agent logs, thanks in advance.2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [EngineProcessor] Done2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [CommandProcessorWrapper] received plugin: PatchLinux result from Processor2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] received reply for RunCommandResult aws.ssm.aeb48a17-5e94-4c8a-be85-ef64685132dc.i-0b1836a486824d20d with message id 1219c4ea-3561-49ad-952a-5f3336a36f1e2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] Got reply msg Id 1219c4ea-3561-49ad-952a-5f3336a36f1e for RunCommandResult aws.ssm.aeb48a17-5e94-4c8a-be85-ef64685132dc.i-0b1836a486824d20d, starting reply thread2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] started reply processing - 1219c4ea-3561-49ad-952a-5f3336a36f1e2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] Sending reply {  "additionalInfo": {    "agent": {      "lang": "en-US",      "name": "amazon-ssm-agent",      "os": "",      "osver": "1",      "ver": ""    },    "dateTime": "2023-02-01T13:52:42.872Z",    "runId": "",    "runtimeStatusCounts": {      "Failed": 1,      "Skipped": 1    }  },  "documentStatus": "InProgress",  "documentTraceOutput": "",  "runtimeStatus": {    "PatchLinux": {      "status": "Failed",      "code": 1,      "name": "aws:runShellScript",      "output": "/usr/bin/python3\n/usr/bin/python2.7\n/usr/bin/python2\n/usr/bin/python\n/usr/bin/apt-get\nReading package lists...\nBuilding dependency tree...\nReadingstate information...\npython3-apt is already the newest version (1.6.6).\nThe following packages were automatically installed and are no longer required:\n  linux-aws-5.4-headers-5.4.0-1085 linux-headers-4.15.0-197\n  linux-headers-4.15.0-197-generic linux-headers-5.4.0-1085-aws\n  linux-image-4.15.0-197-generic linux-image-5.4.0-1085-aws\n  linux-modules-4.15.0-197-generic linux-modules-5.4.0-1085-aws\n  linux-modules-extra-4.15.0-197-generic\nUse 'apt autoremove' to remove them.\n0 upgraded, 0 newly installed, 0 to remove and 55 not upgraded.\nUsing python binary: 'python3'\nUsing Python Version: Python 3.6.9\n02/01/2023 14:52:35 root [INFO]: Downloading payload from https://s3.dualstack.eu-south-1.amazonaws.com/aws-patch-manager-eu-south-1-c52f3f594/patchbaselineoperations/linux/payloads/patch-baseline-operations-1.100.tar.gz\n02/01/2023 14:52:35 root [INFO]: Attempting to import entrance file os_selector\n02/01/2023 14:52:36 root [INFO]: Running with snapshot id = 2b11b799-73e8-4567-9dc6-9e8c40101338 and operation = Scan\n02/01/2023 14:52:36 botocore.credentials [INFO]: Found credentials in shared credentials file: /var/lib/amazon/ssm/credentials\n02/01/2023 14:52:36 root [INFO]: Instance Id: i-0b1836a486824d20d\n02/01/2023 14:52:36 root [INFO]: Region: eu-south-1\n02/01/2023 14:52:36 root [INFO]: Product: Ubuntu18.04\n02/01/2023 14:52:36 root [INFO]: Patch Group: \n02/01/2023 14:52:36 root [INFO]: Operation type: Scan\n02/01/2023 14:52:36 root [INFO]: Snapshot Id: 2b11b799-73e8-4567-9dc6-9e8c40101338\n02/01/2023 14:52:36 root [INFO]: Patch Baseline: {'accountId': '237742590236', 'baselineId': 'pb-04615267655bfc0f4', 'name': 'InxUbuntu', 'globalFilters': {'filters': []}, 'approvalRules': {'rules': [{'filterGroup': {'filters': [{'key': 'PRODUCT', 'values': ['*']}, {'key': 'SECTION', 'values': ['*']}, {'key': 'PRIORITY', 'values': ['Required', 'Important']}]}, 'complianceLevel': 'UNSPECIFIED', 'enableNonSecurity': False, 'approveAfterDays': 0, 'approveUnti\n---Output truncated---\n----------ERROR-------\n/var/log/amazon/ssm/patch-baseline-operations/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning)\nfailed to run commands: exit status 1",      "startDateTime": "2023-02-01T13:52:34.232Z",      "endDateTime": "2023-02-01T13:52:42.870Z",      "outputS3BucketName": "",      "outputS3KeyPrefix": "",      "stepName": "",      "standardOutput": "/usr/bin/python3\n/usr/bin/python2.7\n/usr/bin/python2\n/usr/bin/python\n/usr/bin/apt-get\nReading package lists...\nBuilding dependency tree...\n----------------------------------------- LIST OF LOTS OF APT PACKAGES--------------------------------------dev.amd64', 'installedTime': 1654041792.6328669, 'state': 'InstalledOther'}, 'dsniff.amd64:2.4b1+debian-28.1~build1': {'id': 'dsniff.amd64', 'installedTime': 1616790243.063162, 'state': 'InstalledOther'}, 'eatmydata.amd64:105-6': {'id': 'eatmydata.amd64', 'installedTime': --output truncated--",      "standardError": "/var/log/amazon/ssm/patch-baseline-operations/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings\n  InsecureRequestWarning)\nfailed to run commands: exit status 1"    }  }}2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] successfully sent reply message id: 30104615-d7cc-4428-a4ca-ffb807869dac2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [EngineProcessor] [BasicExecuter] [aeb48a17-5e94-4c8a-be85-ef64685132dc] Executer closed2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [EngineProcessor] execution of aws.ssm.aeb48a17-5e94-4c8a-be85-ef64685132dc.i-0b1836a486824d20d is over. Removing interimState from current folder2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [EngineProcessor] [BasicExecuter] [aeb48a17-5e94-4c8a-be85-ef64685132dc] mast** listener stopped on path: /var/lib/amazon/ssm/i-0b1836a486824d20d/channels/aeb48a17-5e94-4c8a-be85-ef64685132dc2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] received message through control channel 0c3f8974-f99c-4cc7-8903-a23481a9acd32023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] Processing AgentMessage: MessageType - agent_job_reply_ack, Id - 0c3f8974-f99c-4cc7-8903-a23481a9acd32023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] received ack id 1219c4ea-3561-49ad-952a-5f3336a36f1e for message id 0c3f8974-f99c-4cc7-8903-a23481a9acd32023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] ended reply processing - 1219c4ea-3561-49ad-952a-5f3336a36f1e2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] received message through control channel 61b0e48d-417c-44b0-96d5-1e53236f95af2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] Processing AgentMessage: MessageType - agent_job_reply_ack, Id - 61b0e48d-417c-44b0-96d5-1e53236f95af2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] received ack id 2687a09b-afb7-46cc-bb13-8e85d4aa1d61 for message id 61b0e48d-417c-44b0-96d5-1e53236f95af2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] ended reply processing - 2687a09b-afb7-46cc-bb13-8e85d4aa1d612023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] received message through control channel c25083f2-6323-4a90-8c26-23677122c48f2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] Processing AgentMessage: MessageType - agent_job_reply_ack, Id - c25083f2-6323-4a90-8c26-23677122c48f2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] received ack id 30104615-d7cc-4428-a4ca-ffb807869dac for message id c25083f2-6323-4a90-8c26-23677122c48f2023-02-01 14:52:42 INFO [ssm-agent-worker] [MessageService] [MGSInteractor] ended reply processing - 30104615-d7cc-4428-a4ca-ffb807869dac```FollowComment"

Answer: "0Accepted AnswerBy the way, just for let you know, this issue has gone on the Scannings starting from today, or, at least, I did realise just from today.CommentShareFrancescoanswered 3 months ago0The error message is related to a warning from the python library "urllib3" regarding unverified HTTPS requests. This is likely because the Python installation on the Ubuntu instances is missing the required certificates to verify the HTTPS connection. The error only occurs on the Ubuntu instances because the AmazonLinux instances likely have the necessary certificates installed.To resolve this issue, you could try updating the CA certificates on the affected Ubuntu instances, which would allow them to verify the HTTPS connections. You can do this by running the following command:sudo apt-get install ca-certificatesAfter the installation, restart the ssm-agent and run the Scan task again to verify if the error still occurs.CommentShareDivyam Chauhananswered 4 months ago0Thanks for the answer Divyam, I tried to install ca-certificates on one of the instances but it was already installed and at the last version.Also tried to restart the ssm agent and to re-do the scan, but the result was the same.CommentShareFrancescoanswered 4 months ago0Hello, any other ideas? Is there any way to open an issue on the SSM agent project, or is there a dedicated project to the AWS-RunPatchBaseline Document ?CommentShareFrancescoanswered 3 months ago"


Question: AWS - FTP Solution

"I am new to AWS and looking for guidance to design a FTP solutionInfrastructure :A zipped file (encrypted) plus a checksum file  will be available on FTP server in Data Center 1 (once daily at around 1 am). Data Centre 1 cannot be reached via Public internet.But it has a connectivity to Data Centre 2 via MPLS.Datacentre 2 has Direct Connect Link Set up with AWS Ireland.Requirement:Get the zipped file from on premise server in DC!, and perform following : DEcrypting, perform check on checksum and DEcompression.Store the flat files (from zip file) in AWS London region in S3. These files will be required for 12 months and then deleted.  These flat files wont be accessed frequently and will be saved for audit purposes.Only need to run the SFTP operation once on daily basisPre ReqsFirewall ports will be openedNo agent can be installed on any of the On Premise serverBackup / DR solution required as wellWhat is the best way to achieve this. I thought of using Lambda function but how will network side of things work. Can Lambda function be able to reach to FTP server in DC1 which is sitting behind a firewall. Can all the above operations (checksum, decrypt and decompression) be performed using Lambda function. We can create separate Lambda function for each operation.or to use EC2 instance and get node.js installed.FollowComment"

Answer: "0You can create a Lambda function to access resources in your VPC, and hence on-prem resources if the networking/routing etc is configured.See https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.htmlAs long as your VPC in AWS is routable to DC1, You can create a Lambda to pull the file and perform the other operations like decompression, decryption, etc.CommentShareEXPERTMatt-Banswered 9 months agoEXPERTBrettski-AWSreviewed 9 months agoROberoi 9 months agoThanks for your response. Can this solution we used for file sizes in the region 40-50MB. There will be just one 40-50MB file and a checksum file which needs fetching from FTP server. Will VPC have a unique static IP address, as this IP address will need to be whitelisted.Share"


Question: How can I log into an alb-oidc-okta enabled site using curl?

"we configured an Application Load Balancer to authenticate users using OIDC with Okta IDP. Now users  can successfully log into forntend app by Browser. I secured api side with the same approch. Now FE can make rest API calls with session cookie through alb. Now I want to let external server Applicaton to call the API? For example How can I call this rest api using curl? Is there a way to get bearer token from ALB and request curl with that token? Or is there any other approch to access Api programmatically?  Please help, Thank you :)FollowComment"


Question: CloudFormation with SageMaker LifeCycleConfig without leaving the instance running

"I have a use case with SageMaker in which I want to create a notebook instance using CloudFormation.  I have some initialization to do at creation time (clone a github repo, etc.).  That all works fine.  The only problem is that I would like to do this ahead of time in a set of accounts, and there doesn't appear to be any way to leave the newly-created instance in a Stopped state.  A property in the CFT would be helpful in this regard.I tried using the aws cli to stop the instance from the lifecycle create script, but that fails as shown in the resulting CloudWatch logs:An error occurred (ValidationException) when calling the StopNotebookInstance operation: Status (Pending) not in ([InService]). Unable to transition to (Stopping) for Notebook Instance (arn:aws:sagemaker:us-east-1:147561847539:notebook-instance/birdclassificationworkshop).Interestingly, when I interactively open a notebook instance, open a terminal in the instance, and execute a "stop-notebook-instance" command, SageMaker is happy to oblige.  I would have thought it would let me do the same in the lifecycle config.  Unfortunately, SageMaker still has the notebook in the Pending state at that point, so "stop" is not permitted.Are there other hooks or creative options anyone can provide for me?FollowComment"

Answer: 0Accepted AnswerOne solutions will be to create a CFN custom resource backed by lambda.You can configure to run this resource only when the notebook resource completed. and use the lambda function to stop the notebook using one of our SDKs.CommentShareAWS-User-7953554answered 5 years ago


Question: What should be use list down deployed application on ECS

"My sprint boot application will get deployed on AWS ECS, before that I need to keep few api ready using ECS client.What should be use if need to list down the all deployed applications on ECS. I am quite confuse between listTaskDefinition, listTask or listservices. Has anyone used ECSClient api already? Please provide guidance over here. Also the response of these methods has taskArns(), seems it will be name of deployed resources?FollowComment"

Answer: "0You want to use the ListServices() API which will list all the services in your cluster.From the service, you can find the tasks running for it. You can also go with ListTasks() which will list all the tasks in the cluster, then you can work your way back to the service it belongs to.I use boto3/python though, not Java, but the arguments to the API should be the same.CommentShareJohnPrestonanswered 7 months ago"


Question: Amazon Lex Language support

When will Amazon Lex support the French language?FollowComment

Answer: "0Any clue on this? Even not a given date but an estimate would be interesting... is it something we can hope for in the coming months? We're impatiently waiting :)CommentShareSnooGrapes1518answered 3 years ago0Thank you for your interest. We just launched 4 new locales on Amazon Lex. Please refer to the link below.https://aws.amazon.com/about-aws/whats-new/2020/11/amazon-lex-adds-language-support-for-french-spanish-italian-and-canadian-french/Edited by: AnubhavAWS on Nov 12, 2020 11:21 PMCommentShareAWS-User-0691449answered 3 years ago0Thank you for delivering the Canadian French language support.Now with its regional area support, is there a AWS plan to provide the AWS Lex from Canada (Central)ca-central-1?Note for those clients that require the data to reside in Canada.Thank you again for your continous supportSylvainCommentShareMersAWSanswered 3 years ago0AWS LexV2 is now available in ca-central-1. Ref: https://docs.aws.amazon.com/general/latest/gr/lex.htmlNote: ca-central-1 region is supported only in LexV2 and not LexV1Amazon Lex added support for French and Canadian French languages. Please refer to the link below.https://aws.amazon.com/about-aws/whats-new/2020/11/amazon-lex-adds-language-support-for-french-spanish-italian-and-canadian-french/CommentShareShreyBanswered a year ago"


Question: About DMS migration from  ms-sql to dynamodb

"hlw Sir,I am doing the migration process From ms-sql on prime server to AWS dynamodb but after the migration of 400 tables it shows fail. the reason is primary key so please help me and tell me the process how can i migrate the data without create any primary key..It is not migration ms-sql tables which doesn't have primary key to aws dynamodb using DMS. If there any work around to achieve that.FollowComment"

Answer: "0Hi there,I think your question is whether or not AWS DMS support migration of MS-SQL database tables which does not have primary keys. From their documentation, yes you can migrate from MS-SQL Server containing tables that have no primary keys. Since DynamoDB uses a composite primary key you will have to identify two attributes that will form this primary key from your source tables when preparing your data for migration. Please refer to the following walk-through on how to migrate data from SQL Server to DynamoDB.I hope this helpsCommentShareCebianswered a year ago"


Question: I get the follow error with my lambda function: Calling the invoke API action failed with this message: Network Failure

"I have set up a VPC with outbound traffic. Any help would be massively appreciatedSee below for the code:import osimport timefrom datetime import datetimefrom selenium import webdriverfrom selenium.webdriver.chrome.options import Optionsimport boto3def lambda_handler(event, context):    options = Options()    options.binary_location = '/opt/headless-chromium'    options.add_argument('--headless')         # Add your AWS S3 bucket name    S3_BUCKET_NAME = "webscrapebucketxxxxxx22"    # Initialize the S3 client    s3 = boto3.client("s3")    # Set the dictionary of website URLs you want to screenshot    urls = {  "Google": "http://www.google.com"    }        # Set the path to your browser driver    driver_path = "/opt/chromedriver"     # Create a new instance of the browser driver with headless mode    chrome_options = Options()    chrome_options.add_argument("--headless")    driver = webdriver.Chrome(executable_path=driver_path, chrome_options=options)        # Define a function to take a screenshot and save it to a file    def take_screenshot(url, name):        # Navigate to the website        driver.get(url)                # Wait for 30 seconds to ensure that the page is fully loaded        time.sleep(30)                # Get the current date as a string in the format YYYY-MM-DD        now = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")                # Generate the filename using the website name and the current date        filename = f"{name}_{now}.png"                # Save the screenshot to a file in the /tmp directory        filepath = os.path.join("/tmp", filename)        driver.save_screenshot(filepath)                # Return the filepath for logging purposes        return filepath            def upload_to_s3(filepath):        key = os.path.basename(filepath)        s3.upload_file(filepath, S3_BUCKET_NAME, key)        print(f"Uploaded {filepath} to s3://{S3_BUCKET_NAME}/{key}")                    # Loop through the dictionary of website URLs and take screenshots    for name, url in urls.items():        filepath = take_screenshot(url, name)        upload_to_s3(filepath)        print(f"Screenshot of {name} saved to {filepath}")FollowComment"

Answer: "0Thanks for the quick replies! I originally didnt use a VPC but I was getting the same error which is why I tried the VPC approach.If the VPC isn't required I'll remove it.See below for the cloudwatch log snip, could this be an issue with the version of selenium or chrome webdriver I have used?CommentSharerePost-User-8194477answered a month agoRiku_Kobayashi EXPERTa month agoThe error message "Chrome failed to start ~" appears in the log of the image.This error can be fixed by adding the following options.options.add_argument('--no-sandbox')Share0First, I do not see any reason to attach your function to a VPC. It does not seem you are actually accessing any private resources in your VPC.If you do need the VPC for some reason, your function must be attached to a private subnet with a route to a NAT Gateway or to VPC endpoints to different AWS services you need to access (e.g., S3 in your case).CommentShareEXPERTUrianswered a month ago0If Lambda is configured in a VPC, check the route table of the subnet where Lambda is configured to see if there is a route to the NAT Gateway.Also verify that the security group's outbound rules allow it and that the network ACLs allow the necessary communications.https://docs.aws.amazon.com/lambda/latest/dg/troubleshooting-networking.htmlCommentShareEXPERTRiku_Kobayashianswered a month ago0You may want to check if the Lambda function is configured with the correct security group and network ACL settings that allow outbound traffic.Another thing to check is if your VPC has a NAT gateway or NAT instance set up for internet connectivity. Without a NAT, your Lambda function would not be able to communicate with resources outside of your VPC.You can also check the AWS CloudWatch logs for your Lambda function to see if there are any additional error messages or clues as to why the network call failed.CommentSharehashanswered a month ago"


Question: why does the whisper action on the polly console only work for the Mathew voice?

I recently tried the Polly console set to Neural and SSML and keyed in some text and tried several voices with the Whisper action and it works for the Mathew voice but does not work for any other one.If its set to Normal instead of Neural the whisper does work for other voices but does not sound good. Is there a reason the whisper only works for that one voice using the Neural setting?FollowCommentAWS-User-3907265 a year agoCan anyone who works on Polly at Amazon answer this question? It's been almost a week since I posted this. Thanks.Share


Question: Upgrading from Aurora 1.22.2 to 2.10.2 [URGENT]

"Hello.I have made an update in my Aurora mysql databases, after that, the read connection started to fail and to reboot.This cycle of failures and restarts occurs when tried to run the next query:LOAD DATA FROM s3 's3://xxxxxxx' INTO TABLE temp_table FIELDS TERMINATED BY '|' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 LINES (`id`)Log from server./grover/runtime/grover-volume.h:1028: Assertion failed: m_lsnManager != NULL: Stack trace:<inline> (in Grover::Runtime::GroverVolume::getLsnManager() const at /local/p4clients/pkgbuild-fkx7d/workspace/src/GroverRuntime/./grover/runtime/grover-volume.h:1028)/rdsdbbin/oscar/bin/mysqld() [0x19b1123] (in groverPartitionAllocateContiguousLsns at /local/p4clients/pkgbuild-fkx7d/workspace/src/GroverRuntime/grover/runtime/grover-csd.cpp:1458)/rdsdbbin/oscar/bin/mysqld(_Z24log_allocate_grover_lsnsmPKmPP20_hgrover_write_statebb+0x153) [0x181a933]/rdsdbbin/oscar/bin/mysqld(_Z35mtr_log_reserve_and_write_to_groverP5mtr_t+0xc1b) [0x182801b]/rdsdbbin/oscar/bin/mysqld(_ZN5mtr_t6commitEv+0xcb) [0x16bddcb]/rdsdbbin/oscar/bin/mysqld(_Z21dict_hdr_flush_row_idv+0xcb) [0x183b70b]/rdsdbbin/oscar/bin/mysqld(_Z12row_ins_stepP9que_thr_t+0x5f7) [0x17284b7]/rdsdbbin/oscar/bin/mysqld(_Z15que_run_threadsP9que_thr_t+0x4a7) [0x1702c07]/rdsdbbin/oscar/bin/mysqld(_Z12que_eval_sqlP11pars_info_tPKcmP5trx_t+0x197) [0x1703657]/rdsdbbin/oscar/bin/mysqld(_ZN8S3Stream20persistS3FileHistoryEmP5trx_t+0xe3) [0x1248ec3]/rdsdbbin/oscar/bin/mysqld(_ZN8S3Stream22persistS3StreamHistoryEv+0x73) [0x1248ff3]/rdsdbbin/oscar/bin/mysqld(_ZN8S3Stream4readEPhm+0x11f) [0x123b9bf]/rdsdbbin/oscar/bin/mysqld(_my_b_s3_read+0xd3) [0x1156e33]/rdsdbbin/oscar/bin/mysqld(_my_b_get+0x2f) [0x14b798f]/rdsdbbin/oscar/bin/mysqld(_ZN9READ_INFO9next_lineEv+0x407) [0x126ef07]/rdsdbbin/oscar/bin/mysqld(_Z10mysql_loadP3THDP12sql_exchangeP10TABLE_LISTR4ListI4ItemES8_S8_15enum_duplicates14LoadFileSource+0xa9f) [0x1272a3f]/rdsdbbin/oscar/bin/mysqld(_Z30mysql_execute_command_internalP3THDb+0x1ba3) [0xfa5c43]/rdsdbbin/oscar/bin/mysqld(_Z21mysql_execute_commandP3THDb+0x12b) [0xfaa3ab]/rdsdbbin/oscar/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x3ab) [0xfab2eb]/rdsdbbin/oscar/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x1017) [0xfac497]/rdsdbbin/oscar/bin/mysqld(_Z10do_commandP3THD+0x147) [0xfad107]/rdsdbbin/oscar/bin/mysqld(_ZN22OscarSchedulerConsumer7consumeEjjb+0x2a7) [0x10a15a7]/rdsdbbin/oscar/bin/mysqld(_ZN22OscarSchedulerConsumer5startEv+0x10f) [0x10a17af]/rdsdbbin/oscar/bin/mysqld(_ZN22OscarSchedulerConsumer11drain_queueEPv+0x4b) [0x10a19eb]/rdsdbbin/oscar/bin/mysqld(pfs_spawn_thread+0xd7) [0x18dd437]/lib64/libpthread.so.0(+0x72ab) [0x40001982a2ab]/lib64/libc.so.6(+0xd5e5b) [0x4000212f2e5b]03:14:59 UTC - mysqld got signal 6 ;This could be because you hit a bug. It is also possible that this binaryor one of the libraries it was linked against is corrupt, improperly built,or misconfigured. This error can also be caused by malfunctioning hardware.Attempting to collect some information that could help diagnose the problem.As this is a crash and something is definitely wrong, the informationcollection process might fail.key_buffer_size=16777216read_buffer_size=262144max_used_connections=9max_threads=400thread_count=9connection_count=9connection_count=6It is possible that mysqld could use up tokey_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 227952 K bytes of memoryHope that's ok; if not, decrease some variables in the equation.Trying to get some variables.Some pointers may be invalid and cause the dump to abort.Query (401900cebae5): LOAD DATA FROM s3 's3://xxxxxx' INTO TABLE temp_table FIELDS TERMINATED BY '|' ENCLOSED BY '"' LINES TERMINATED BY ' ' IGNORE 1 LINES ( `id`)Connection ID (thread ID): 18Status: NOT_KILLEDThe manual page at http://dev.mysql.com/doc/mysql/en/crashing.html containsinformation that should help you find out what is causing the crash.aurora backtrace compare flag : 1Writing a core file2022-02-16T03:15:11.316356Z 0 [Warning] The syntax '--secure-auth' is deprecated and will be removed in a future release2022-02-16T03:15:11.316576Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.2022-02-16T03:15:11.316700Z 0 [Note] /rdsdbbin/oscar/bin/mysqld (mysqld 5.7.12) starting as process 19993 ...2022-02-16T03:15:11.325755Z 0 [Warning] InnoDB: Setting innodb_checksums to OFF is DEPRECATED. This option may be removed in future releases. You should set innodb_checksum_algorithm=NONE instead.2022-02-16T03:15:11.325795Z 0 [Note] InnoDB: Started in read only mode2022-02-16T03:15:11.325803Z 0 [Note] InnoDB: PUNCH HOLE support available2022-02-16T03:15:11.325807Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins2022-02-16T03:15:11.325810Z 0 [Note] InnoDB: Uses event mutexes2022-02-16T03:15:11.325814Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier2022-02-16T03:15:11.325817Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.112022-02-16 03:15:11 0x400027042160[IB_LOG_LEVEL_INFO]:Initializing buffer pool, size = 89.7G (srv0start.cc:4406)2022-02-16T03:15:11.325881Z 0 [Note] InnoDB: == Add SYNC_FAST DDL...2022-02-16T03:15:11.326091Z 0 [Note] InnoDB: Number of pools: 12022-02-16T03:15:11.326217Z 0 [Note] InnoDB: Not using CPU crc32 instructions2022-02-16T03:15:11.326224Z 0 [Note] InnoDB: Disabling background log and ibuf IO write threads.2022-02-16T03:15:11.326944Z 0 [Note] InnoDB: Initializing buffer pool, total size = 89.7432G, instances = 16, chunk size = 5.60895G2022-02-16 03:15:11 0x400027042160 Attempt to open data volume as READER(buf0oscar.cc:799)(buf0buf.cc:2411)(buf0buf.cc:2411)(buf0buf.cc:2411)(buf0buf.cc:2411)(buf0oscar.cc:799)(buf0oscar.cc:799)(buf0oscar.cc:799)2022-02-16T03:15:13.952531Z 0 [Note] InnoDB: Completed initialization of buffer pool2022-02-16T03:15:13.952807Z 0 [Note] InnoDB: page_cleaner coordinator priority: -202022-02-16T03:15:13.952815Z 0 [Note] InnoDB: page_cleaner worker priority: -202022-02-16T03:15:13.952825Z 0 [Note] InnoDB: page_cleaner worker priority: -202022-02-16T03:15:13.952825Z 0 [Note] InnoDB: page_cleaner worker priority: -202022-02-16T03:15:13.974660Z 0 [Note] InnoDB: Highest supported file format is Barracuda.2022-02-16T03:15:14.009425Z 0 [Note] InnoDB: 5.7.12 started; log sequence number 02022-02-16T03:15:14.009612Z 0 [Note] Plugin 'FEDERATED' is disabled.220216 3:15:14 server_audit: Audit STARTED.Found DAS config file, trying to load DAS switcher from DAS config file.2022-02-16 03:15:14 70369398759776:[DAS][INFO]: Calculated persistence threads 8aurora_enable_das:0220216 3:15:14 server_audit: server_audit_incl_users set to ''.220216 3:15:14 server_audit: server_audit_excl_users set to ''.2022-02-16T03:15:14.018244Z 0 [Warning] Read-only servers cannot create temp tables in InnoDB. Changing default engine for temp tables to MyISAM.2022-02-16T03:15:14.018252Z 0 [Warning] Read-only servers cannot create intrinsic temp tables in InnoDB. Changing default engine for temp tables to MyISAM.2022-02-16T03:15:14.019683Z 0 [Note] Skipping generation of SSL certificates as options related to SSL are specified.2022-02-16T03:15:14.020167Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.2022-02-16T03:15:14.020229Z 0 [Note] Server hostname (bind-address): '*'; port: 33062022-02-16T03:15:14.020250Z 0 [Note] IPv6 is available.2022-02-16T03:15:14.020258Z 0 [Note] - '::' resolves to '::';2022-02-16T03:15:14.020373Z 0 [Note] Server socket created on IP: '::'.2022-02-16T03:15:14.281071Z 0 [Note] /rdsdbbin/oscar/bin/mysqld: ready for connections.Version: '5.7.12' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)% Total % Received % Xferd Average Speed Time Time Time CurrentDload Upload Total Spent Left Speed0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0100 11 100 11 0 0 11000 0 --:--:-- --:--:-- --:--:-- 11000----------------------- END OF LOG ----------------------FollowComment"

Answer: "0Accepted AnswerThis version of RDS has a bug with the queryLOAD DATA FROM S3The AWS technicians contacted us and informed us thatThe fix was use de load data of mysql, creating the file on runtimeCommentShareHely Palenciaanswered a year ago0HiI understand you want to upgrade from Aurora 1.22.2 to 2.10.2 and you are getting a cycle of errors.Please correct me if I misunderstood.Reviewing the logs you posted ,your cluster hits a bug .I would recommend that you patch your cluster to the latest version to avoid future crashes.I would also recommend that if the issue persists open a case with AWS Support for further assistance [2].I hope the was helpful!Reference[1]https://docs.aws.amazon.com/AmazonRDS/latest/AuroraMySQLReleaseNotes/AuroraMySQL.Updates.3010.html[2]https://us-east-1.console.aws.amazon.com/support/home/?nc2=h_ql_cu&region=us-east-1#/CommentShareKaraboanswered a year ago"


Question: PARQUET argument is not supported when loading from file system

"In the AWS documentation is states I should be able to COPY parquet files into Redshift. (https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/) I am having a dog of a time making this work.After creating a role with S3ReadOnlyAccess, AWSGlueFullAccess, and AWSAthenaFullAccess, I trusted the role to the user account of Redshift. I then added the IAM role to the Redshift Cluster.However after running the COPY command I received the error:PARQUET argument is not supported when loading from file systemI am attempting to load it from an S3 location. Any idea why this error is being thrown? Is PARQUET not supported as an Importable file into Redshift from S3, or am I not giving the role the correct access to something?Any help is appreciated.Thanks,Stephen.Edited by: roland23571113 on Jun 8, 2018 6:47 PMEdited by: roland23571113 on Jun 11, 2018 1:32 PMI have attached a pointer to the documentation I read online. This does not work on my US-WEST-2 redshift cluster. https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/Edited by: roland23571113 on Jun 12, 2018 10:50 AMAfter discussing with the aws architect it appears the role settings are correct:AWSS3ReadOnlyAWSGlueFullControlAWSAthenaFullControlAssigning trust of the Redshift user account to the role was done correctly.Adding the role to the AWS Cluster was done correctly.The copy command was the only thing done incorrectly.The trick is that the S3 bucket path must point to a folder containing the .parquet file, not the file itself.  Any file found in the folder is assumed to be uploaded as a parquet file.copy <table_name>from "<S3 Folder Path>IAM_ROLE '<role string>'FORMAT AS PARQUETFollowComment"

Answer: 0Answered by AWS Architect Review.CommentShareroland23571113answered 5 years ago


Question: CloudShell environment CREATION_FAILED

"Unable to start the environment. To retry, refresh the browser or restart by selecting Actions, Restart AWS CloudShell. System error: Environment was in state: CREATION_FAILED. Expected environment to be in state: RUNNING. To retry, refresh the browser or restart by selecting Actions, Restart AWS CloudShell.I opened a support ticket with billing and they said they can't help and to post here on repost.FollowComment"

Answer: "0Have you tried these suggestions? - https://stackoverflow.com/questions/68970459/unable-to-start-the-environment-to-retry-refresh-the-browser-or-restart-by-selCommentShareEXPERTskinsmananswered 4 months agorePost-User-9481576 4 months agoHi skinsman.  Yes, that was the first and second thing I tried.  I came over to repost and searched and have not found any answers that solve this.  I put in the support ticket due to that and they referred me back to repost, so I have made a new post.Shareskinsman EXPERT4 months agoIf you have a support plan I'd push back on Support to resolve it if you're stuck.Share"


Question: Implementing message node in lex

"Hi, I have been looking at different bot builder where they have different node types as part of their bot flow. One of the node is of type "message" where you can just add that as part of the conversation.So for if a user adds a message (sent by bot)) e.g. in between asking question using slots/enitities, how can we implement in lex.Is that a new intent? Or do we need to create a lambda to handle it? Wondering what is the best way to handle it using lex. thanks.FollowCommentKris a year agoCan you provide an example conversation demonstrating the behavior you want? That may help us understand your goal and allow us to offer some ideas.In this context, what is a "message"? Does a "message" require a response from the user in order to proceed? Is it the final outcome of a bot conversation? Is it something else?Share"


Question: AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unable to build AWSGlueClient: com.amazonaws.SdkClientExcep

"AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unable to build AWSGlueClient: com.amazonaws.SdkClientException: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.)Getting this error when defining spark views in AWS Glue local development using Docker image. Below is a function that I have defined and trying to call in the PySpark code ,which results in this error.def incrementalTransform(glueContext, dfc) -> DynamicFrameCollection:    input = dfc.select(list(dfc.keys())[0]).toDF()    target = dfc.select(list(dfc.keys())[1]).toDF()    targetWithColumns = (        target.withColumn("rownum", lit(0))        .withColumn("partition_year", year("create_time"))        .withColumn("partition_month", month("create_time"))        .withColumn("partition_day", dayofmonth("create_time"))    )    inputView = input.createOrReplaceTempView("inputView")    targetView = targetWithColumns.createOrReplaceTempView("outputView")    r1 = spark.sql("(select * from inputView)")    r2 = spark.sql("(select * from targetView)")    result = spark.sql("(select * from inputView) UNION All (select * from targetView)")    r1.show()    r2.show()    result.show()FollowComment"

Answer: "0Hello,Looking at the error Unable to find a region via the region provider chain it appears that your code/script is unable to retrieve the AWS region specification from the available configuration files or environment within the docker container.As per this article , the below command is used while starting the docker containerdocker run -it -v ~/.aws:/root/.aws:ro --name glue_jupyter amazon/aws-glue-libs:glue_libs_1.0.0_image_01 bashThis will mount the .aws folder from your local filesystem onto the docker container. Usually, this .aws folder has the region specification set in a .config file. A sample config file looks like belowcat ~/.aws/config[default]region = us-east-1You can find more about the aws configuration files hereCommentShareSUPPORT ENGINEERChiranjeevi_Nanswered a year ago"


Question: Subscriber not receiving MQTT Publish from Lambda function

"Hi,I am looking for some guidance on what I could be doing incorrectly with my AWS Lambda function. For context, I am publishing a MQTT message from an ESP32-S3 to a specified topic (e.g. client/url/get). Upon receiving a message to that topic I trigger my Lambda function that in turn publishes a pre-signed URL to a topic that my device is subscribed to (e.g. client/url/get/accept). I am able to trigger my Lambda function and a pre-signed URL is published to the "client/url/get/accept" topic when I view my MQTT Test Client, but I am not receiving any feedback on my device in regards to that publish. I can manually publish to the "client/url/get/accept" topic from the MQTT Test Client and am able to see that publish from my device but no luck with my AWS Lambda publish.For my Lambda function I am using aws-sdk's "AWS.IotData" to publish to my topic.Any help would be appreciated, please let me know if you need any more information.FollowCommentGavin_A 5 months agoCan you update your question with the IoT Policy that is attached to the clients principal (e.g. certificate)?Share"

Answer: "0There are a few possible issues that could be causing the problem you're experiencing with your AWS Lambda function:MQTT protocol issues: It's possible that there is a problem with the MQTT protocol itself, such as a problem with the quality of service (QoS) level or with the way the messages are being sent and received.AWS IoT issues: There could be a problem with your AWS IoT configuration, such as an issue with the policies or permissions that are set up for your device or with the MQTT broker itself.Lambda function issues: It's also possible that there is an issue with your Lambda function code or with the way it is configured. For example, there may be a problem with the way you are using the AWS SDK to publish the message, or there may be a problem with the way the message is being formatted or sent.To troubleshoot the problem, here are a few steps you can try:Check the logs: Check the logs for your Lambda function to see if there are any error messages that could indicate the source of the problem.Test the MQTT connection: Use a tool like MQTT.fx to test the MQTT connection between your device and the broker, to see if the connection is working correctly.Check the AWS IoT configuration: Check the policies and permissions for your device and make sure that they are set up correctly. Also check the configuration of the MQTT broker to ensure that it is set up correctly.Review your Lambda function code: Review your Lambda function code to make sure that you are using the AWS SDK correctly and that the message you are publishing is formatted correctly.CommentShareMounickanswered 5 months ago0If you can publish to your device manually from the MQTT test console, then the issue is likely with the way your code is publishing the message. It sounds like the Lambda code is sending some data to the 'client/url/get/accept' topic since you can see that data in the test console. Some things you can check:Are you publishing the message as json from the Lambda function?Look at the AWS IoT and Lambda logs and in CloudwatchAdd some logging/print statements into your Lambda function so that you can see the output of each step from the codeCan you add code to the esp32 device to dump whatever is received from the socket to the screen?If not already, check out the sample code for Device SDK v2 basic Publish/Subscribe hereCommentShareofftraillabsanswered 5 months ago"


Question: Create ECS service using existing load balancer with existing target group

I'm using the AWS console to create an ECS service (using fargate) in an existing cluster.In the second step of the wizard (configure network) I choose an existing application load balancer.The "container to load-balance" section shows my container to add.When I click "add to load balancer" it initially shows the "production listener port" and "target group name" dropdowns showing "create new".When I select an existing target group in the dropdown this grays out (disables) the "production listener port" dropdown.When clicking the "next step" button validation complains the "production listener port" is not filled in (validation message: "please select a listener").Which isn't possible because the control is disabled.First selecting a listener port in the wizard and switching to an existing target group after that doesn't remedy the situation as choosing an existing target group blanks out and disables the "production listener port" dropdown causing the same problem when clicking the "next step" button.How to get the container to register in an existing target group?UpdateThe target group is an empty IP address group.Interestingly it does work if the target group is not empty (the "production listener port" is then filled with 443:HTTPS) but an empty target group (even of the correct type) clears the "production listener port".ReproductionCreate a new target group of type "IP address" leaving other default settings. Do not register any targets in this group.Next add this target group to a (new or existing) load balancer (for testing I added the group to an existing load balancer with a single source IP address filter e.g. 192.168.1.1/32 so it doesn't disrupt normal operations).When creating a new ECS service select a task definition that has a container with an exposed port 80.On the second screen "Configure Network" choose the VPC and subnets and under "Load balancing" pick "Application load balancer".Select the load balancer to which you added the empty target group.Now click "Add to load balancer" next to the container. This shows the "Production listener port" and "Target group name" dropdowns both initially set to "Create new".Choosing the empty target group disables the "Production listener port" dropdown and clears it.If the target group is not empty the "Production listener port" is automatically filled with the correct port.FollowCommentMichael_F EXPERTa year agoWhat kind of target group configuration do you have? The target group needs to be "IP address" -- a target group that is configured to route to instance IDs will not work.Sharejporcelli1120 9 months ago+1 I find this confusing. The documentation for network ELBs references routing to targets by instance so I find this to be confusing. Am I missing something? Also I have target groups targeting both instance and IP type associated with a network load balancer so..?https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.htmlShare

Answer: "0Same problem here, this is really annoyingCommentShareTcheikovskianswered a year ago"


Question: Is there a way to specify language while detecting text in AWS Textract?

"I am trying extract some texts from a document as key-value pairs using Textract boto3 client, but the doc contains some text in regional languages (Hindi in this case). Is there any parameter to mention only 'Engish' as a language for text extraction and ignore any non-english characters. TIA.FollowComment"

Answer: "0Currently English, Spanish, Italian, Portuguese, French, and German are supported languages in Amazon Textract.Please refer the below page for more information on language (under quotas)https://docs.aws.amazon.com/textract/latest/dg/limits-document.htmlCommentShareSanjay Aggarwalanswered 2 months ago"


Question: "Beanstalk: eb deploy erroring out with Invalid length for parameter EnvironmentName, value: 25, valid range: 4-23"

"I have a Beanstalk environment which has been running since more than a year now. It's running Python 3.6 running on 64bit Amazon Linux, which is now deprecated and I need to migrate to Amazon Linux 2 separately.Out of nowhere, I am not able to do ***eb deploy ***to my environment anymore. It throws an error sayingERROR: Parameter validation failed:Invalid length for parameter EnvironmentName, value: 25, valid range: 4-23The environment name has never changed after it was created. Why is this error all of a sudden?My awsebcli version is 30.20.2FollowCommentJoshua_B 10 months agoUser found solution to the problem on a different platform. See their post on Stack Overflow: https://stackoverflow.com/questions/70914749/beanstalk-eb-deploy-erroring-out-with-invalid-length-for-parameter-environmentnShare"

Answer: 0User found solution to the problem on a different platform. See their post on Stack Overflow: https://stackoverflow.com/questions/70914749/beanstalk-eb-deploy-erroring-out-with-invalid-length-for-parameter-environmentnCommentShareJoshua_Banswered 10 months ago


Question: Cloudwatch Canary: How to set User Agent for Python Selenium script?

"According to this documentation I should be able to set the user agent of my CloudWatch Canary using:add_user_agent(user_agent_str), ex: "synthetics_webdriver.add_user_agent('MyApp-1.0')"I have tried this, and my code that attempts this looks like the following:from aws_synthetics.selenium import synthetics_webdriver as webdriver..userAgentString = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36"..webdriver.add_user_agent(userAgentString)driver = webdriver.Chrome()The Canary passes the test, but the user agent is never set (it defaults to the standard user agent).  Has anyone had any success changing the user agent for a Canary, and if so, how?  Thank you!FollowComment"


Question: FlexMatch large match backfill not working

"Hello,I am having trouble getting flexmatch working, when I initiate the first matchmaking request everything goes well a session is created, the placement works and I get a player session in the created game session. But this only works as long as I have available game session slots, every time I create a matchmaking request it tries to create a new game session, so I end up with game sessions with 1 player. I have back fill set to automatic and yet it does not place players in existing game sessions with empty slots. Even after a game session is completely empty with nobody inside of it, it still won't assign the player to the server so I end up with servers empty or with only 1 player.I don't know what else to do, I read that UpdateGameSession has to be implemented but nowhere can I find documentation on what I am supposed to do in this function, is there an AWS call to make to tell GameLift that it was handled ?EDIT: So I modified my RuleSet, that had a max count of 100 to less than 40, when it's not running Large match mode it works fine, is this a bug of the large match algorithm ?Thanks,MaxFollow"

Answer: "0Accepted AnswerGood Afternoon Max,Apologies for the long lapse in communication here. We were able to locate and fix an issue that seems related to the problem you were encountering. Please let us know if you attempt the above testing and encounter this issue again. If you do, providing the same information you did in this last response would be great, as I can see it was very useful to the engineer who last investigated.SharerePost-User-8556151answered 3 years ago0@REDACTEDUSERIt seems like you did you get regular matchmaking working how you liked. The only difference between regular and backfill, is that in backfill you are providing some pre-filled teams; the same rules are still applied.Theres a lot that could be going on, so its worth checking if:Your matchmaking rules could be too restrictive meaning that no players can be backfilled into the game session. I would look at your rules and expansions and ensure players trying to backfill could be placed in an active game session.Your incoming ticket rate is too low for your expansions/relaxations or timeout values. Incoming player rate is important to bear in mind, you may need much more generous values when debugging/developing.Wrt to UpdateGameSession I believe you are talking about the onUpdateGameSession callback rather than the actual UpdateGameSession.This is a callback from GameLift to let you know that the game sessions is being modified. Its recommend that you should log the update reason here to aid troubleshooting, but theres no callback to make to say you handled it as its just providing information.The expectation is that the server owns the matchmaking data, so you should be mapping it some internal state so your server can handle player disconnects etc. Typically servers keep their own internal map of players to connected states etc.For example:void Server::OnUpdateGameSession(Aws::GameLift::Server::Model::UpdateGameSession updateGameSession){        Aws::GameLift::Server::Model::GameSession gameSession = updateGameSession.GetGameSession();        Aws::GameLift::Server::Model::UpdateReason updateReason = updateGameSession.GetUpdateReason();        log("OnUpdateGameSession with GameSession : %s, UpdateReason : %s, and backfillTicketId : %s",            gameSession.GetGameSessionId(), Aws::GameLift::Server::Model::UpdateReasonMapper::GetNameForUpdateReason(updateReason), updateGameSession.GetBackfillTicketId());         if (updateReason == Aws::GameLift::Server::Model::UpdateReason::MATCHMAKING_DATA_UPDATED)        {            // pull out new player data into internal representations        }        if (updateReason == Aws::GameLift::Server::Model::UpdateReason::BACKFILL_CANCELLED ||            updateReason == Aws::GameLift::Server::Model::UpdateReason::BACKFILL_FAILED ||            updateReason == Aws::GameLift::Server::Model::UpdateReason::BACKFILL_TIMED_OUT)        {        // as automatic backfill failed, may want to take manual action etc.        }    }}But as you report this is specifically with large matchmaking, it may be worth providing your matchmaking config id, rule set id etc and a time range where you made requests that didn't work as you expected, and I can get the GameLift service team to take a look.If you have it, it would also be useful to provide your server SDK version and language used.SharerePost-User-5332620answered 3 years ago0Thank you very much for the detailed reply, I understand now what the callback is supposed to do.I don't have any restrictions on my ruleset as there is no skill or any other metric to place players, the goal is here is just to fill the game sessions and create new ones if no player slot is available, so I just send the same skill for every player.Regarding the large match ids here they are :Config id : arn:******Queue id : arn:******Rule set id : TestProjectBatchingRuleSet (could not find an ARN for it)First matchmaking ticket id that goes well : 32db0260-e574-4aa3-9ccf-a0ceaf9db49d ( "StartTime": "2020-06-27T12:00:19.361000+02:00" )Second matchmaking ticket id that never gets filled :  6dbcdb54-194c-4f1d-9364-03e8462f8660 ( "StartTime": "2020-06-27T12:00:47.456000+02:00" )I am using the latest GameLift sdk 04_16_2020 and C++.Thanks,MaxSharerePost-User-4519063answered 3 years ago0Hello,Just checking in to know if there is indeed an internal issue as this is a blocker for us and we cannot use GameLift if we don't get it working.BestMaxSharerePost-User-4519063answered 3 years ago0Apologies for the delay in getting back to you but I was out for a while.I didn't see any other open issues with GameLift large match support currently so there doesn't seem to be a problem with the service (that I know of).I pinged the GameLift service team with your details so they can take a look and will let you know if theres an update.Hopefully you read about the differences here wrt to building rules for large match support: https://aws.amazon.com/blogs/gametech/large-match-support/ (I can't look at your ruleset, but the GameLift can)Also in future, if you have a urgent/blocking issue consider open an issue with AWS support (esp if you get to running live games). Forums support is always best effort and can be delayed at times (we do love to help but theres always a lot going on).Apologies again for the late followup here.SharerePost-User-5332620answered 3 years ago0Hi!Apologies for the lag time, the holiday in the US delayed getting to this. For the second match there are a couple of things which look a little strange.It looks like as part of the second matchmaking ticket you included a player session which was actually already included in another ticket (auto-backfill-81d3592f-b544-4806-9385-0156b5f979f0). That ticket was attempting to backfill the match with 1/100 players, but was cancelled because the previous ticket was created with that player id (we cancel the backfill ticket to avoid a player being 'placed' into both games).The second matchmaking ticket timed out, which was because it attempted to create a new game with the player referenced above instead of backfilling into an existing match. The fleet did not have an available game session to place in, so after several attempts to place the ticket timed out.It looks like when you are triggering the startmatchmaking for that player it is clobbering your autobackfill process. What you should make sure of is that you arent submitting a new matchmaking request  on behalf of a player that is already in a game, that is what causes the collision. That would prevent the timeout as well because matchmaking would not try to create a new game.SharerePost-User-1112179answered 3 years ago0Hello @REDACTEDUSERI have tried everything with very simple commands and I still cannot get backfill to work with large match.I issue the following commands :aws gamelift start-matchmaking --configuration-name TestProjectMatchMaker --players PlayerId=Test01,PlayerAttributes={skill={N=1000}}aws gamelift start-matchmaking --configuration-name TestProjectMatchMaker --players PlayerId=Test02,PlayerAttributes={skill={N=1000}}Only Test01 gets a game session.Test01 has ticket id 7e9e343d-a985-40cd-9505-7a1351aab11cTest02 has ticket id b9d830b6-b085-4656-94eb-6cc5049f5c05If I do the exact same thing but without large match it works fine.Best,MaxSharerePost-User-4519063answered 3 years ago"


Question: How can I use CloudWatch or any other mechanism to know my vCPU usage?

I need to know my vCPU usage across all my accounts and regional deploymentFollowComment

Answer: 0https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.htmlCommentSharerePost-User-5208427answered 2 months ago


Question: Where to place the Amazon Kendra metadata files for Amazon S3 data source?

I have the following structure in Amazon S3 under my bucket:-/kendra-analysis/     - ead0d0af-e769-4c99-97f7-04e68d1cf8b2.txt-/kendra-analysis-meta/     - ead0d0af-e769-4c99-97f7-04e68d1cf8b2.txt.metadata.jsonMy data source configuration in Amazon Kendra has the following:Data source bucket : <Bucket_Name>Metadata prefix : kendra-analysis-meta/Include prefixes : kendra-analysis/It seems like my metadata is not picked up. What am I missing?FollowComment

Answer: "0Accepted AnswerIn this case, the Amazon Kendra index is looking in:/kendra-analysis-meta/kendra-analysis/ead0d0af-e769-4c99-97f7-04e68d1cf8b2.txt.metadata.jsonbecause it mirrors the prefix of the first order object underneath the specified metadata prefix.See: https://docs.aws.amazon.com/kendra/latest/dg/s3-metadata.htmlCommentShareEXPERTJDBanswered 3 years ago"


Question: Custom link preview for aws domain

"Hi there,I just set up an s3 site but cannot figure out how to create a link image preview. I am used to using Yoast through WordPress but can't seem to find anything similar for AWS. Any help you can give would be greatly appreciated!FollowComment"

Answer: "0I suspect what you're asking is how to create Open Graph meta tags for an S3-hosted site that socials will look for.You would need to create these in the HTML page you're linking to, and you might look to generate these with a tool like https://www.opengraph.xyz/Blog here generally describing how open graph tags work: https://ahrefs.com/blog/open-graph-meta-tags/CommentSharePeteWilcockanswered a year ago"


Question: Amazon IoT Device advisor test examples ?

"Hello AWS team, Greeting from EUWe have been getting our hands dirty with using AWS IOT. While exploring the service we came across device advisor test. Can you please provide us with a  some sample MQTT Client that we can use to test out the device advisor. Most of our test are failing and we would like to see an example which passes the test.We are very inclined towards migrating to AWS IoT.https://docs.aws.amazon.com/iot/latest/developerguide/device-advisor-tests.htmlRegardsJZFollowComment"

Answer: "2Hi JZ. Please check the links on this page: https://docs.aws.amazon.com/iot/latest/developerguide/iot-sdks.htmlThe V2 AWS IoT Device SDKs for C++/Java/Python/JavaScript are all qualified against AWS IoT Core Device Advisor. Also, AWS IoT Device Client (which uses the AWS IoT Device SDK V2 for C++) is compatible.  So any of the V2 SDK demo programs, or the Device Client executable, will pass the MQTT Connect test case for example. Whether they will pass other test cases depends on what the particular program does. Since AWS IoT Device Client publishes and subscribes, it will pass the entire AWS IoT Core Qualification Program suite (if you restart it at each test case).BTW, you can see the AWS IoT Device SDK V2 for C++ regression testing against Device Advisor in the CI:https://github.com/aws/aws-iot-device-sdk-cpp-v2/blob/main/.github/workflows/ci.ymlhttps://github.com/aws/aws-iot-device-sdk-cpp-v2/tree/main/deviceadvisorCommentShareEXPERTGreg_Banswered 7 months ago"


Question: AWS Data Transfer Costs with content download from website - S3

"Hello Every one,I'm starting with AWS by trying to build a personal blog with downloadable content from S3 (PDF and images mostly for now) .I'm concerned at this point by data transfer costs when users download this content.Is my concern legitimate ? is it better to avoid downloadable content in this case if it will incur unpredictable costs ?Thanks in advanceFollowComment"

Answer: "1Thanks for choosing AWS and Amazon S3 for your blog. This recent announcement from late 2021 might be of interest as you consider how to architect your blog: https://aws.amazon.com/blogs/aws/aws-free-tier-data-transfer-expansion-100-gb-from-regions-and-1-tb-from-amazon-cloudfront-per-month/ . The free tier out to internet from Amazon S3 and data out from Amazon CloudFront both were greatly increased, which should mean significant savings for you and other customers. If you expect a lot of reads from Amazon S3, then considering S3 Standard or S3 Intelligent-Tiering as your storage class are good options as each has no retrieval fees.CommentShareawsrwxanswered a year agoEXPERTNataliya_Greviewed a year agoNataliya_G EXPERTa year ago+1 consider using CloudFront in this case (where your S3 buckets with content will sit behind it), as it basically provides triple-benefits: better performance, security AND costShare"


Question: Error message: Additional Verification Required / Certificate renewal failed despite proper Route53 setup for verification

"I have a cert that is pending renewal. Hosted zones supported are all located in Route 53 so it supposed to be a breeze. My website is small and is definitely not really expected to be in Alexa top 1000.In cert's page on AWS Certificate Manager I observe the "Pending validation" for some domains and one domain is Failed. The failed domain is in .ru zone, I do not know if it matters. Nevertheless, other domains are in .com and .art zones. Help says to post in this forum.FollowCommentandreiko a year agoHi Onkami,I found your post because I was having a similar issue with my .ru domain.Looks like Russian and Belorussian domains have been banned from ACM by a third-party certificate authority that AWS relies on: https://brenik.livejournal.com/7426031.htmlShare"

Answer: "0Hi There,I understand that you are getting an error "Additional verification required to request certificates for one or more domain names in this request." (.ru domain zone) when validating the domain name to issue the public certificate via ACM.Please note this error could be the results of one of two things.1.When the certificate contains a domain that ranks within the Alexa top 1000 websites.[1]2.Or as of March 10, 2022 ACM has restrictions to issue new certificates for ".ru" domains until further notice.I see in your case you will no longer be able to issue or renew certificates as your domain falls under the following domains..RU.BYБел - BelarusРф - Russian Federation.moscow.москва - Moscow.SU - Soviet Union(http://ru.com/) .RU.COM.РУС.RU.NETAll Amazon certificates for these domains will remain functional until expiration, but will not be renewable and no new certificates from these domains will be issued.The only workaround that would work in your scenario would be to obtain a certificate from a third party that can issue a certificate for your domain, and import the certificate into ACM [2]I trust the above information is helpful to you.References: ==============[1] https://docs.aws.amazon.com/acm/latest/userguide/troubleshooting-failed.html#failed-additional-verification-required[2] https://docs.aws.amazon.com/acm/latest/userguide/import-certificate.htmlPlease note that I personally value your feedback, please accept this answer if you find it helpful to you.CommentShareMfaneloanswered a year ago"


Question: Requesting Clarification on Basic Concepts

"TL;DR at the bottom.I am having some trouble getting the basic concepts of fleets, instances and game sessions down.I was originally under the impression that a fleet was just one physical/virtual machine, with the capacity of hosting as many simultaneous game sessions as the specific hardware would allow.In this way, if I calculated that I could run ~30 game sessions on a single c4.8xlarge machine on a specific region that would mean that if I wanted to boot up an additional session I would need to launch a new fleet (and wait a long time for it to be ready).This didn't seem to make a lot of sense, until I came across the gamelift FAQ which specified that flexmatch may "trigger additional server instances" when trying to find a game for a client. This really confused me as I don't think this would mean waiting for a new fleet to launch every time we hit that limit.So does this mean that a fleet can have many instances, which are each a physical/virtual machine with the resources specified when creating the fleet? Would the potential amount of game sessions then be scaled by the usage limits of that type of fleet (2 for me in this case, allowing for up to 60 simultaneous games on that fleet)? And does this mean that flexmatch would automatically take care of starting up a new instance if needed?Thanks in advance for any help!TL;DRAre fleets single machines with the defined specs or a collection of machines with those defined specs, and if so does flexmatch automatically boot up additional instances if needed?FollowComment"

Answer: "0Some basic concepts:A queue can have 1 or more fleets; fleets can be in different regions, mix of Spot/OnDemand instances etc.A fleet can have 1 or more instances.An instance can have 1 or more processes running. Each process can host a game session.FlexMatch finds players and groups them together in a new game session. It will won't create new capacity directly for you.To autoscale your fleet (to scale up/down based on demand), use target tracking rules: https://docs.aws.amazon.com/gamelift/latest/developerguide/fleets-autoscaling-target.htmlTo fix a fleet capacity you can use min, max and desired to manually scale up/down: https://docs.aws.amazon.com/gamelift/latest/developerguide/fleets-capacity-limits.htmlHope that makes sense.CommentSharerePost-User-5332620answered 4 years ago0That is perfectly clear. I now know how to structure my fleets better. Thank you very much!CommentSharerePost-User-0366499answered 4 years ago"


Question: DKIM signing DISABLED for multiple domains

"hi,I'm running an EC2 instance to host a number of  wordpress websites.for each site I'm hosting I've added the domain details to AWS Simple Email Service and added the relevant CNAME records to the DNS at the registrar.within a few hours of adding them, each domain is verified and everything seems fine.then, a few days ago, i got an email from AWS saying"We detected that the DNS records required for the DKIM setup of example.com are no longer present in your DNS settings."this happened first to one domain, and then others the next day.nothing has changed in the DNS records for these domains, in some cases the DKIM settings have been there for well over a year.i now have 6 domains where i've received emails saying the DKIM is being disabled... i've checked and the relevant CNAME exists for each of them.any suggestions for what could be going wrong?cheersDavidFollowCommentEvelynDee a year agoI am experiencing the same issue; WordPress. Were using the SES Offload light plugin. And after I receive the disabled message I'm receiving some spam spoofing from the domain. And then some time later I'll receive a re-enabled notification from AWS. Maybe it's some kind of exploit?Share"

Answer: "0Did you resolve this. Had exactly the same message from AWS ("We detected that the DNS records required for the DKIM setup of ... are no longer present in your DNS settings."). Checked and CNAME entries are there and SES is still working.CommentSharedeanflyeranswered a year agodogstar10 a year agoin the end it resolved itself... the problem for me was with the domain name registrar... apparently they'd experienced a number of DDOS attacks and had partially blocked a large range of IP addresses which seemed to include AWS servers...after a few days to a week they all  became verified again ....sorry for the delay in writing back, hope that helps if you're still experiencing difficulties...Share"


Question: Transit Gateway attachmet

"Hello,Is there a way to filter direct connect and VPN traffic with AWS firewall without using transit gateway?ThanksFollowCommentSanjeev Gopal 7 months agoAre you looking to filter traffic between the VPN and DX, or between the VPC/Internet and the VPN/DX?Share"

Answer: "0For the North-South inspection TGW is mandatory, see below section from this blogNorth-South: Centralized on-premises egress & ingress via Transit Gateway and Transit VIF/Direct Connect gateway/AWS Site-to-Site VPNLet’s expand the previous model and add inspection for North-South traffic between AWS VPC and on-premises via AWS Transit Gateway. AWS Transit Gateway can connect to your on-premises via AWS Direct Connect or via AWS Site-to-Site VPN.A key requirement for this model is to connect AWS Direct Connect using Transit VIF to AWS Transit Gateway. In case of VPN to on-premises, AWS Site-to-Site VPN can also be used and must be established to AWS Transit Gateway as per Figure 9.CommentShareEXPERTTushar_Janswered 7 months ago"


Question: Syncing external mongodb database info with neptune database

"I have data in mongodb and I want to use some of the properties saved in that database to start and maintain a graph database using amazon neptune. I can use aws dms to get the initial batch of data, but how about maintaining and updating the data as it changes in mongodb? Essentially I need to keep the neptune database synced up with the mongodb database because mongodb remains my primary database. As a simple example, consider that mongodb contains, among other things, users' lists of subscribers. I want this info available for graph queries in neptune, but changes to that info are saved in mongodb primiarily so they need to be forwarded to neptune to keep neptune current. Is there a best-practice for doing something like this? The bulk loader and dms options implicitly assume that I'd be inserting large amounts of data at a time, but these may be simple small changes like one user subscribing to another user. What's the best/correct/recommended way to go about this?FollowComment"

Answer: 0You can use the Kafka Change Data Capture handler: https://www.mongodb.com/docs/kafka-connector/master/sink-connector/fundamentals/change-data-capture/Connect that to our Managed Service for Kafka: https://docs.aws.amazon.com/msk/latest/developerguide/what-is-msk.htmlThen use a Kafka Worker in Lambda to apply the necessary modification in Neptune: https://docs.aws.amazon.com/lambda/latest/dg/with-msk.htmlCommentShareRodney Lesteranswered a year ago


Question: Amazon Access Control system hardware (RFID/Biometrics)

"VentureBeat (https://venturebeat.com/) featured article on AWS access control system hardware 2 years ago, but no information is available from the link.https://devices.amazonaws.com/please share pointers to AWS access control system hardware and supporting software sdk/resources if any.Thank you so much.FollowCommentmaneeshsit 7 months agoWaiting for responses on this. Thank youShare"


Question: "No details available about JEPs, Patches and merged OpenJDK projects"

HiCan any one help me out to get missing details under web page https://docs.aws.amazon.com/corretto/latest/corretto-17-ug/patches.html.Its mentioned "This section lists all the patches applied to OpenJDK for Amazon Corretto. We also provide links to the issues in the OpenJDK project." . But no data provided.FollowComment


Question: "Does SQS reach "unlimited capacity" instantaneously, or do I need to spread messages (ramp up) over some time?"

"The Quotas related to messages documentation page for SQS says:Standard queues support a nearly unlimited number of API calls per second, per API action (SendMessage, ReceiveMessage, or DeleteMessage).Can I suddenly send a lot of messages (from many separate clients at the same time) to a queue that was previously not getting any traffic? Or do I need to ramp up the number of messages per second in some kind of progressive way?For example:Can I send no message to a queue for 23 hours and 59 minutess and then send (thousands? dozens of thousands? hundreds of thousands?) messages per second during one minute?Or do I need to spread those messages over a longer time, maybe ramping up over half an hour before I can reach max throughput?FollowComment"

Answer: "1A single Amazon SQS message queue can contain an unlimited number of messages. However, there is a 120,000 quota for the number of inflight messages for a standard queue and 20,000 for a FIFO queue. There is also a throughput quota with SQS - By default, FIFO queues support up to 3,000 messages per second with batching or up to 300 messages per second (300 send, receive, or delete operations per second) without batching. If you require higher throughput, you can enable high throughput mode for FIFO on the Amazon SQS console, which will support up to 30,000 messages per second with batching, or up to 3,000 messages per second without batching. Refer to https://aws.amazon.com/sqs/faqs/ and is a useful resource for additional information about SQS.CommentShareTim_Aanswered a year agoEXPERTBrettski-AWSreviewed a year ago0Yes it is possible. I have not seen any issues with receiving messages all of the sudden.CommentShareSowndarya_Ganswered a year ago0There will be a limit to how many messages your producers can reliably send and I would recommend testing this. However, SQS distributes your queue over a number of SQS instances to support scale and resilience.From AWS BlogIn order to make SQS scalable, durable, and reliable, messages are stored in a fleet that consists of thousands of servers in each AWS Region. Within a region, we save three copies of each message, taking care to distribute the messages across storage nodes and Availability Zones. In addition to this built-in redundant storage, SQS is self-healing and resilient to host failures & network interruptions.From SQS FAQYou can scale the amount of traffic you send to Amazon SQS up or down without any configurationFrom Basic Amazon SQS architectureredundantly stores the messages across multiple Amazon SQS serversCommentShareMatt_Sanswered a year agoEXPERTBrettski-AWSreviewed a year ago"


Question: Codeguru - Only Security checks/scans

"Is it possible to use codeguru only for performing security scans/checks. Though codeguru provides other scans, I am interested in using codeguru only for security scans. Does AWS provide such facility?FollowComment"

Answer: "1Hi,In your scenerio, I'd recommend to use the exclude and suppress function as the codeguru reviewer will run as a one feature. I've added reference below which you may have seen already but just in case.https://docs.aws.amazon.com/codeguru/latest/reviewer-ug/recommendations.htmlhttps://docs.aws.amazon.com/codeguru/latest/reviewer-ug/recommendation-suppression.htmlCommentShareAWS-CSinghanswered 4 months ago0Yes, it is possible to use AWS CodeGuru to perform only security scans on your code. AWS CodeGuru provides security-related recommendations based on best practices and best practices from the industry, as well as checks for known security vulnerabilities in your code.You can use CodeGuru's security review feature to scan your codebase and identify potential security issues, such as SQL injection vulnerabilities, cross-site scripting (XSS) vulnerabilities, and sensitive data exposure. CodeGuru analyzes the code using a combination of static and dynamic analysis and generates a report that includes a detailed description of the issue, the affected code, and the recommended remediation.Additionally, CodeGuru also integrates with other AWS security services like Amazon Inspector, which allows you to perform security assessments on the running instances.To use CodeGuru for security scans you will need to enable and configure the security review feature for your repository in CodeGuru. You can also configure the settings to scan only specific parts of your codebase and exclude certain files or directories.In summary, AWS CodeGuru provides the facility to perform security scans and checks, and you can use it to focus on the security scans specifically.CommentSharejayamaheizenberganswered 4 months ago"


Question: My VPN is instable

"We have a lot of connexion with my vpn connection, can you help me please to find a root cause below the détails:-----Message d'origine-----De : Amazon Web Services, Inc. no-reply-aws@amazon.com Envoyé : dimanche 23 avril 2023 04:03 À : a.peuriere@ceccli.com Objet : Important notice about your AWS Account regarding VPN connections [AWS Account: 918873256167]Hello AWS VPN Customer,You're receiving this message because you have at least one VPN Connection in the eu-west-3 Region, for which your VPN Customer Gateway is not using both tunnels. This mode of operation is not recommended as you may experience connectivity issues if your active tunnel fails.The VPN Connection(s) which do not currently have both tunnels establishedare:vpn-05dcf0abc284b803evpn-053709db6f969409dYou can obtain the VPN Connection configuration recommendations for several types of VPN devices from the AWS Management Console [1]. On the "Amazon VPC" tab, select "VPN Connections". Then highlight the VPN Connection and choose "Download Configuration".For Static VPNs, enabling both tunnels may lead to asymmetric routing, which will need to be permitted by your Customer Gateway Device. For configuration options, please refer to your vendor specific documentation.This notice will be sent regularly until both tunnels are established. For assistance in establishing your second tunnel, or to opt-out of these notifications, please contact AWS Premium Support [2].[1]https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://console.aws.amazon.com%26source%3Dgmail-ima&source=gmail-imap&ust=1683184020000000&usg=AOvVaw3jRnTKUOaI22uCt388Wwyvp&ust=1682820160000000&usg=AOvVaw0dOysG0DDn42Ob48uFTfOH[2]https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://aws.amazon.com/support%26source%3Dgmail-ima&source=gmail-imap&ust=1683184020000000&usg=AOvVaw2o470b8MpUfvOBmoa3v-bOp&ust=1682820160000000&usg=AOvVaw3aA635NK99E4R_F4qvuQKkSincerely,The AWS VPN TeamAmazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc., 410 Terry Ave. North, Seattle, WA 98109-5210.FollowComment"

Answer: 0It is possible that something is missing from the customer gateway.What model of customer gateway do you have?CommentShareEXPERTRiku_Kobayashianswered a month ago


Question: Update command with join syntax

"I am migrating from MS Sqlserver to Postgrasql with Babelfish. I cannot get the update commands in my stored procedures to work. I've used Compass and it passes. I have built the following simple procedure that works just fine on MSSql and not in Babelfish.CREATE TABLE #table1 (keyID int NULL,Retail money NULL,Cost money NULL,GP numeric(20,8) NULL)CREATE TABLE #table2 (keyID int NULL,Retail money NULL,Cost money NULL,GP numeric(20,8) NULL)INSERT INTO #table1 (keyID, Retail, Cost, GP)Values(1,10,5,1),(2,20,10,1),(3,30,15,1),(4,40,20,1)INSERT INTO #table2 (keyID, Retail, Cost, GP)Values(1,0,0,0),(2,0,0,0),(3,0,0,0),(4,0,0,0)UPDATE #table2SET Retail = a.Retail,Cost = a.Cost,GP = (a.Retail-a.Cost)/a.Retail * 100FROM #table2 bINNER JOIN #table1 a on a.keyID = b.keyIDI get the message:Msg 33557097, Level 16, State 1, Line 35missing FROM-clause entry for table "a"What is the proper syntax?FollowComment"

Answer: "0You have hit a known issue that is on the roadmap. In the meantime, please rewrite your UPDATE statement as per below:UPDATE #table2SET Retail = a.Retail, Cost = a.Cost, GP = (a.Retail-a.Cost)/a.Retail * 100FROM #table2 b, #table1 aWHERE a.keyID = b.keyIDImportant: make sure you are running the latest available version of Aurora PostgreSQL and Babelfish, which is this case is APG 14.5 and BBF 2.2. You can check the version through e.g. sqlcmd with:SELECT CAST( SERVERPROPERTY('BabelfishVersion') AS VARCHAR )CommentShareStefan Karlssonanswered 6 months ago"


Question: ECS Service Connect Health Check

"We are using ECS Service Connect via a CloudFormation template using the ServiceConnectConfiguration property in the ECS Service configuration. This all works fine. However, we have noticed that when we scale out, the new instance immediately starts receiving traffic once it is registered to Service Connect / CloudMap. However, it still take another 2 minutes or so for the container to be ready to accept traffic due to its boot up and initialization needs.I see that CloudMap allows for a Health Check on the Service (ServiceDiscovery.Service.HealthCheckConfig). But the configuration for the ServiceConnectConfiguration.Services property does not allow the definition of a Health Check. I also do not see a way to create the Service directly in CloudMap (ServiceDiscovery::Service) so that I can enable the health check, then map that into the Service Connect settings of the ECS Service defintion.So how can I enable the CloudMap Service Health Check in combination with the ECS Service "Service Connect" configuration? Or is there another / better way to enforce the not routing traffic to the container instance until it is ready instead of immediately upon registration?FollowComment"

Answer: "4ECS service registration with AWS Cloud Map doesn't support health checks by default. However, you can use Application Load Balancer (ALB) in combination with ECS Service to ensure that the traffic is only routed to the healthy containers.the official documentationhttps://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-load-balancing.htmlHere you can find how toCreate an Application Load Balancer (ALB). You can do this using the AWS::ElasticLoadBalancingV2::LoadBalancer resource in your CloudFormation template.Configure the ALB target group with the appropriate health check settings. Use the AWS::ElasticLoadBalancingV2::TargetGroup resource in your CloudFormation template.Set the ECS service's loadBalancers property to associate the ECS service with the ALB target group. In this way, ECS will only route traffic to the healthy tasks behind the ALB.Update your ECS service configuration in the CloudFormation template to use the ALB. You need to add the loadBalancers property to your ECS service configuration:MyECSService:  Type: AWS::ECS::Service  Properties:    ...    LoadBalancers:      - TargetGroupArn: !Ref MyTargetGroup        ContainerName: MyContainer        ContainerPort: 80    ...Ensure that your tasks have a health check configured. You can use the HEALTHCHECK command in your Dockerfile or configure the health check in your task definition.By using ALB with ECS, the traffic will only be routed to the healthy containers, as the ALB will perform the health checks and only route traffic to the targets that are deemed healthy.CommentShareEXPERTsdtslmnanswered 2 months agoTigerSC95 2 months agoThank you for the detailed response. We have traditionally used an ALB with health checks and path based routing, that all worked fine. However, we want to use Service Connect / Cloud Map to avoid routing through the ALB. All of our microservices currently leverage HTTP discoverability via Cloud Map / Service Connect. We do not want to go back to using an ALB for internal traffic between microservices. So, is your answer that you cannot configure Health Checks with Service Connect nor can you configure Cloud Map Service first and then associate it with Service Connect on the ECS instances?Share1Accepted AnswerAfter a lot of research and trial/error, I have determined that you can NOT use Cloud Map Service health checks combined with ECS Service Connect. By using a container health check with the Cloud Map Service, the new container instance in a Cloud Map service would be unhealthy until it passes the Container Health Check and therefore it cannot receive traffic until it is ready. I have proven that this part works, but you lose ECS Service Connect discoverability / naming.Here are some references I found for this:https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-discovery.html (see "Service discovery considerations" section and bold "HealthCheckCustomConfig" item toward bottom of list.)https://docs.aws.amazon.com/cloud-map/latest/api/API_HealthCheckCustomConfig.html (describes how Custom Health Check will be leveraged)The problem specifically lies in the DiscoveryName property (AWS::ECS::Service ServiceConnectConfiguration.Services.DiscoveryName). Per https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-service-serviceconnectservice.html,  "The discoveryName is the name of the new AWS Cloud Map service that Amazon ECS creates for this Amazon ECS service". This is the key, because CloudFormation always tries to CREATE the service and fails if the name already exists. For this to work, the CloudFormation (or API/SDK) build/deployment logic would have to change to only create the service name if it doesn't exist - otherwise link/refer to the existing Cloud Map Service. The alternative is to allow the definition of the Health Check in the ServiceConnectConfiguration.Services object.If you could just link to an existing Cloud Map Service, then the steps below would work (BUT CURRENTLY DO NOT):Create the Cloud Map Namespace (AWS::ServiceDiscovery::HttpNamespace)Create Cloud Map Service (AWS::ServiceDiscovery::Service) -> specify a HealthCheckCustomConfig containing FailureThreshold onlyCreate a Task Definition (AWS::ECS::TaskDefinition) -> add Container HealthCheck object in ContainerDefinitions (Health Check details are specific to your particular needs)Create ECS Service (AWS::ECS::Service) that uses Task Definition from above -> add ServiceConnectConfiguration object and provide DiscoveryName = Name or ARN of Cloud Map Service you created aboveIf you define the Health Check in the ServiceConnectConfiguration.Services object, then the single step below would work :Create ECS Service (AWS::ECS::Service) -> add ServiceConnectConfiguration object and provide NameSpace, DiscoveryName and specify a Health Check ConfigIf either of these (or both) were to be implemented, we could stop directing traffic to instances that are not ready in ECS Service Connect.CommentShareTigerSC95answered a month agochessels 12 days agoHad the same problem. Your research saved me a lot of time. Thanks!Share"


Question: Route Selection in Transit Gateway

"If we've got a Transit Gateway configured and we have multiple VPN connections into the TGW (from multiple routers).  I'd like to know how the TGW will select the return route for packets.For example if we have two VPN connections (two CGWs) into the TGW and a packet arrives from RTR1, how does the TGW select the route to send the packets back down to the on-prem network?We are using BGP.  We have two route tables in play.  One of the route tables has the VPC associated to it and the on-prem network in the "propagations" tab.  Our 2nd route table has the VPN connections associated to it and the VPC attachment is in the propagations.  The on-prem subnet is being advertised from both on-prem routers.We are not using ECMP at this point.FollowComment"

Answer: "0Accepted AnswerHi there,Since you're using BGP routing based VPN, please have different Local Preference and different AS PATH Pre-pending for each of the BGP neighbor.Local Preference helps on premise Customer Gateway to choose the tunnel to exit from the local network while exporting routes with AS PATH pre-pending makes a particular path 'less preferred'/'secondary' for the peer(s) receiving the route.eg)   AS Number on CGW: 65270VPN-ABCTunnel 1 - Local Preference: 500 | AS PATH : 65270Tunnel 2 - Local Preference: 400 | AS PATH : 65270 65270VPN-XYZTunnel 3 - Local Preference: 300 | AS PATH : 65270 65270 65270Tunnel 4 - Local Preference: 200 | AS PATH : 65270 65270 65270 65270The above configuration would make Customer Gateway and AWS VPN to choose the routing tunnel in the following manner based on tunnel availability:Tunnel 1 > Tunnel 2 > Tunnel 3 > Tunnel 4I hope you find this information useful.CommentShareAWS-User-7027861answered 3 years ago"


Question: How can I process flat files with a footer record in AWS Glue?

"I'm trying to use AWS Glue to process a flat file that has header information in the first row and footer information in the last row. The file has 10 data columns, but the footer has only two columns (the number of records in the file and the file origin).What's the best way to process this type of file in AWS Glue?FollowComment"

Answer: "0Accepted AnswerYou can use the following PySpark script in AWS Glue to process flat files like the one you've described:sample_df_1=sc.textFile('temp.txt')sample_df_1.collect()['1,2,3,4,5,6,7,8,9,10', 'A,B,C,D,E,F,G,H,I,K', 'foot,er']hdr=sample_df_1.first()sample_df_2=sample_df_1.filter(lambda l:l != hdr)sample_df_2.collect()['A,B,C,D,E,F,G,H,I,K', 'foot,er']final_df=sample_df_2.map(lambda l:l.split(',')).filter(lambda l: len(l) > 2)j=final_df.toDF()j.show()+---+---+---+---+---+---+---+---+---+---+| _1| _2| _3| _4| _5| _6| _7| _8| _9|_10|+---+---+---+---+---+---+---+---+---+---+|  A|  B|  C|  D|  E|  F|  G|  H|  I|  K|+---+---+---+---+---+---+---+---+---+---+CommentShareSundeepanswered 2 years ago"


Question: Update boto3 version

"I need to use a newer boto3 package for AWS Glue Python3 shell job (Glue Version: 1.0). I included the a wheel file in S3: boto3-1.13.21-py2.py3-none-any.whl under Python Library Path. I did a print on boto3.version and observed that Glue Python Shell job still uses 1.9.203 even if i see the following log: "Successfully installed boto3-1.13.21 botocore-1.16.26 docutils-0.15.2 jmespath-0.10.0 python-dateutil-2.8.1 s3transfer-0.3.3 six-1.15.0 urllib3-1.25.10". Is there any way to overwrite the boto3 package version for Glue Python Shell job?Edited by: blastia on Aug 28, 2020 6:54 PMEdited by: blastia on Sep 1, 2020 5:19 AMFollowComment"

Answer: "2You can upgrade the boto3 version with below steps.Upload boto3 wheel file to your S3 bucket. Boto3 wheel file is available in pypi.org. (https://pypi.org/project/boto3/#files)Configure your Glue Python shell job with specifying the wheel file S3 path in 'Python library path' in the job configuration.Insert below codes at the beginning of your python script. (The print statements can obviously be omitted)import syssys.path.insert(0, '/glue/lib/installation')keys = [k for k in sys.modules.keys() if 'boto' in k]for k in keys:    if 'boto' in k:       del sys.modules[k]import boto3print('boto3 version')print(boto3.__version__)Then you can import boto3 and start scripting with newer boto3.For example, you can use Athena ListDataCatalogs which is not available in default boto3 yet.athena = boto3.client("athena")res = athena.list_data_catalogs()Edited by: NoritakaS-AWS on Oct 22, 2020 9:59 PMCommentShareAWS-User-6283168answered 3 years agoGonzalo Herreros a year agoThe last version of botocore that can run on 3.6 is 1.23.10 https://pypi.org/project/boto3/1.23.10/#filesShare0Hi,We got AWS Glue Python Shell working with all dependency as follows. The Glue has awscli dependency as well along with boto3AWS Glue Python Shell with InternetAdd awscli and boto3 whl files to Python library path during Glue Job execution. This option is slow as it has to download and install dependencies.Download the following whl filesawscli-1.18.183-py2.py3-none-any.whl https://pypi.org/project/awscli/#filesboto3-1.16.23-py2.py3-none-any.whl https://pypi.org/project/boto3/#filesUpload the files to s3 bucket in your given python library pathAdd the s3 whl file paths in the Python library path. Give the entire whl file s3 referenced path separated by commaAWS Glue Python Shell without Internet connectivityReference:  AWS Wrangler Glue dependency build https://github.com/corvuslee/public/blob/master/awswrangler_glue.mdWe followed the steps mentioned above for awscli and boto3 whl filesBelow is the latest requirements.txt compiled for the newest versionscolorama==0.4.3docutils==0.15.2rsa==4.5.0s3transfer==0.3.3PyYAML==5.3.1botocore==1.19.23pyasn1==0.4.8jmespath==0.10.0urllib3==1.26.2python_dateutil==2.8.1six==1.15.0Download the dependencies to libs folderpip download -r requirements.txt -d libsMove the original main whl files also to the lib directoryawscli-1.18.183-py2.py3-none-any.whl https://pypi.org/project/awscli/#filesboto3-1.16.23-py2.py3-none-any.whl https://pypi.org/project/boto3/#filesPackage as a zip filecd libs zip ../boto3-depends.zip *Upload the boto3-depends.zip to s3 and add the path to Glue jobs Referenced files pathNote: It is Referenced files path and not  Python library pathPlaceholder code to install latest awcli and boto3 and load into AWS Python Glue Shell.import os.pathimport subprocessimport sys# borrowed from https://stackoverflow.com/questions/48596627/how-to-import-referenced-files-in-etl-scriptsdef get_referenced_filepath(file_name, matchFunc=os.path.isfile):for dir_name in sys.path:candidate = os.path.join(dir_name, file_name)if matchFunc(candidate):return candidateraise Exception("Can't find file: ".format(file_name))zip_file = get_referenced_filepath("awswrangler-depends.zip")subprocess.run(["unzip", zip_file])# Can't install --user, or without "-t ." because of permissions issues on the filesystemsubprocess.run(["pip3 install --no-index --find-links=. -t . *.whl"], shell=True)#Additonal code as part of AWS Thread https://forums.aws.amazon.com/thread.jspa?messageID=954344sys.path.insert(0, '/glue/lib/installation')keys = [k for k in sys.modules.keys() if 'boto' in k]for k in keys:if 'boto' in k:del sys.modules[k]import boto3print('boto3 version')print(boto3.version)Check if the code is working with latest AWS CLI APIThanks,SarathEdited by: SarathMohan on Nov 23, 2020 10:32 AMEdited by: SarathMohan on Nov 23, 2020 10:35 AMEdited by: SarathMohan on Nov 23, 2020 10:41 AMCommentShareSarathMohananswered 3 years ago0In case anyone else ends up on this thread, Glue v2.0 simplified this process drastically with the addition of the --additional-python-modules parameter. Seehttps://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#aws-glue-programming-python-libraries-glue-20https://docs.aws.amazon.com/glue/latest/dg/reduced-start-times-spark-etl-jobs.htmlThe following job parameter made the Athena client functions in boto3 v1.17.12 available without any need to add extra code modifying the python path."--additional-python-modules", "botocore>=1.20.12,boto3>=1.17.12"CommentSharerluckeyanswered 2 years ago0This doesn't work for some reason...After importing the newly updated boto3 library, and checking the version:print(boto3.__version__)"1.17.9" is printedBut when I try to access the list_data_catalogs() method:res = athena.list_data_catalogs()I receive the following error: "AttributeError: module 'boto3' has no attribute 'list_data_catalogs'"Edited by: g-crmf on Feb 23, 2021 11:43 AMEdited by: g-crmf on Feb 23, 2021 11:43 AMCommentShareg-crmfanswered 2 years ago"


Question: DocumentationPart Property Cloudformation example

I am having hard time understanding how to pass Property in TypeAWS::ApiGateway::DocumentationPart.Here is the sample CF template:-  DocumentationPart:    Type: AWS::ApiGateway::DocumentationPart    Properties:      Location:        Type: "RESOURCE"        Path: "/path"      RestApiId: !Ref RestApiGateway      Property: "{\"description\":\"Testing\"}"The Property is wrong and errors out. Any idea how do i pass it in my CF template? AWS docs have no example showing how to pass it.FollowComment

Answer: "0Accepted AnswerHi!  Good question, this is slightly confusing.  There's no "Property" under Properties for AWS::ApiGateway::DocumentationPart.Try it with "Properties": https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-documentationpart.htmlCommentSharejsoncanswered a year agoAWS Forum a year agoThey should correct the documentation. @AWS guys, if you are reading this please correct the example mentioned in above linked doc!Share"


Question: The following resource(s) failed to create: [mcLoggingBucket].

"I am trying to generate a Valheim game server using this guide: https://aws.amazon.com/blogs/gametech/hosting-your-own-dedicated-valheim-server-in-the-cloud/However, when trying to create the Stack in CloudFormation, I get the below error:"Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting (Service: Amazon S3; Status Code: 400; Error Code: InvalidBucketAclWithObjectOwnership; Request ID: 51R10FQCBW6BCSC9; S3 Extended Request ID: p2+/TRD2zyg4WtFDht/oETooFwJkZkZm0MINYLPK+qQ9fw78R2un1E1araLGUym3cpGe023uUdg=; Proxy: null)"Here is the code for that resource from the YAML file (@=a)mcLoggingBucket:    Type: AWS::S3::Bucket    Properties:       BucketEncryption:        ServerSideEncryptionConfiguration:          - ServerSideEncryptionByDefault:              SSEAlgorithm: 'aws:kms'              KMSM@sterKeyID: KMS-KEY-ARN      PublicAccessBlockConfiguration:         BlockPublicAcls: True        BlockPublicPolicy: True        IgnorePublicAcls: True        RestrictPublicBuckets: True      AccessControl: LogDeliveryWriteFollowComment"

Answer: 1Accepted AnswerThis error is related to the bucket being recognized as a “Bucket Owner Enforced” bucket during creation. ACLs such as “LogDeliveryWrite” are not able to be enabled when a bucket is in this state. This template can only work on buckets with ObjectWriter or BucketOwnerPreffered object ownership currently.You can consider using the below options1.Edit the template to specify a new object ownership rule2.Remove the “LogDeliveryWrite” line and allow S3 to handle this itself via bucket policy.=== Template  ObjectOwnership: ObjectWriter (also tested) | ObjectOwnership: BucketOwnerEnforced (Returns the same error)- ===  OwnershipControls:    Rules:      - ObjectOwnership: BucketOwnerPreferredAdd this section to your template and that should resolve your issueCommentSharemadhuri_danswered 23 days ago