Question: Amazon Athena error on querying DynamoDB exported data

"BackgroundWe've configured an export to s3 from dynamodb using the native dynamodb s3 export, and ION as the format output.After this, we've created a table in AthenaCREATE EXTERNAL TABLE export_2022_07_01_v4 (  `_PK` STRING,  URL STRING,  Item struct<    `_PK`:string,    URL:string  >  )ROW FORMAT SERDE 'com.amazon.ionhiveserde.IonHiveSerDe'WITH SERDEPROPERTIES (  "ignore_malformed_ion" = "true")STORED AS IONLOCATION '...';Querying this works all right for small simple queries, but attempting to produce a full output withUNLOAD (  SELECT Item.URL FROM "helioptileexports"."export_2022_07_01_v4"  WHERE Item.URL IS NOT NULL ) to '...'  WITH (format = 'TEXTFILE')Results in this errorHIVE_CURSOR_ERROR: Syntax error at line 1 offset 2: invalid syntax [state:STATE_BEFORE_ANNOTATION_DATAGRAM on token:TOKEN_SYMBOL_OPERATOR]This query ran against the "helioptileexports" database, unless qualified by the query. Please post the error message on our forum  or contact customer support  with Query Id: f4ca5812-1194-41f1-bfda-00a1a5b2471bQuestionsIs there a way to make Athena more tolerant of formatting errors on specific files? As shown in the example, we are attempting without success to use ignore_malformed_ion. Is there anything beyond that that can be done?Is this a bug on DynamoDB ION export process?Is there any mechanism or logging to identify the files which have the malformed data and remove them?FollowCommentShantanu Oak a year agoTry "ion.ignore_malformed" instead of "ignore_malformed_ion"Share"


Question: Anyone using EFS for 20+TB?

"We are currently utilizing EBS volume that we serve up via NFS from EC2 instances and looking to replace that setup with EFS (and using IA via lifecycles).  These volumes are a combination of GP2 (active data) and SC1 (less active data).Most of the AWS examples in their docs about using EFS reference smaller amounts of data than we have (ie, XXX GB).  Clearly EFS is highly scalable well beyond TB...supporting up to PB-scale.Can anyone share any real experiences using EFS with...Many TB (ie, 20-100) ...my guess is at this scale it might make sense to use multiple EFS volumesRunning with at-rest encryptionRunning snapshot/recovery-point backups to local region every 2 hours (with AWS Backup?)Storing some amount of snapshots/recovery-points out of the regionThanks!FollowComment"

Answer: "0Accepted AnswerHi, thanks for reaching out. I'm on the EFS service team, and can confirm that EFS file systems support PiB-scale, and many customers are using EFS and Backup successfully within and above the size range of 20TiB-100TiB that you mentioned.Based on your current usage of both EBS GP2 and SC1 it sounds like your use case would be a great candidate for EFS Lifecycle Management, which moves less active data to our EFS Infrequent Access storage class at a 92% lower cost. On average, customers that use this capability achieve blended storage cost of $0.08/GiB-mo or less.EFS supports both encryption at rest and encryption-in-motion. You can enable the former at file system creation time with a single click, and the latter is a per-connection option when using our EFS mount helper utility.For backups, you can configure your AWS Backup policy to run as often as every hour by using a custom cron expression in your backup rule configuration. Your first backup will make a fully copy of your file system, and subsequent backups will be incremental-only. You can also configure your backup policy to store some or all backups in another AWS Region.Last, it's unlikely that you need to create multiple file systems to store your data, since each EFS file system elastically scales to PB-scale, and supports up to 35K IOPS and several GB/s of aggregate throughput.CommentShareAWS-User-4040733answered 3 years ago0Hi smaybs,EFS scales to PB and you can use a policy to move older data to Infrequently Accesses storage class.Hope that helps.Stefan RadtkeAWS Solution Architect ProfessionalEdited by: StefanRadtke on Jul 23, 2020 2:09 AMEdited by: StefanRadtke on Jul 23, 2020 8:07 AMRemoved reference to Qumulo.CommentShareStefanRadtkeanswered 3 years ago"


Question: SAP HANA of AWS

"Hello,We have to configure HANA HSR for DR scenario .  Its a lean DR setup with primary HANA of more memory size compared to secondary.Is pacemaker a requirement for HANA system replication for lean DR scenarioThanksFollowComment"

Answer: "1It depends on your DR RTO (Disaster Recovery - Recovery Time Objective) - how fast you want the system to be brought up ? if it is RTO < 5 mins, then yes, pacemaker will be required to automatically resize the secondary HANA, and convert it to primary node. If RTO is more lenient, you can consider doing other alternatives like manual take over and resizing.CommentShareferrymulanswered 8 months ago"


Question: "AWS EKS E0508 12:27:50.996279    2751 describe.go:612] Unable to construct reference to '&core.Pod &  selfLink was empty, can't make reference"

"i am getting the error while deploying the microservice i am proving the start and end for the ERROR, please provide the valuable finding if possible,ERROR START:E0508 12:27:50.996279    2751 describe.go:612] Unable to construct reference to '&core.Pod{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}ERROR END:Ready:false, RestartCount:0, Image:"922250982392.dkr.ecr.ap-south-1.amazonaws.com/reg-rep-ms:22.2.2-0.0", ImageID:"", ContainerID:""}}}}': selfLink was empty, can't make referenceFollowComment"

Answer: 0is image ref correct?CommentSharemyronix88answered a month ago


Question: Java sdk for aws marketplace view subscription

"Hi, I am looking for marketplace API to view purchased subscriptions?FollowComment"

Answer: "0Hi,Currently, there is no API reference to view or manage Marketplace subscriptions. The only way to view active subscriptions in the AWS Marketplace subscription manager is, by signing in to the AWS Marketplace console, and then choosing Manage subscriptions - https://aws.amazon.com/premiumsupport/knowledge-center/view-marketplace-subscriptions/Additionally, as per our documentation, the AWS Marketplace Catalog API service provides an API interface to manage AWS Marketplace for your AWS organization or AWS account. For approved sellers, you can manage your products programmatically, including the self-service publishing capabilities on the AWS Marketplace Management Portal. For private marketplace administrators, you can manage your private marketplace programmatically. For more information on the same, please refer to the following documentation - https://docs.aws.amazon.com/marketplace-catalog/latest/api-reference/welcome.htmlCommentShareSUPPORT ENGINEERrePost-User-0230995answered 6 months ago"


Question: DQDL for Time-series or Data Quality/Insights Report For Sagemaker or Custom report for Data Quality for Times-series data

"AWS provides AWs Glue Data Quality, powered by DQDL. Is there a DQDL example for Time-series sensor data. It also offers "Data Quality and Insights Report in Sagemaker under Data Wrangler, but that is also not a great way, IMHO. Is there a tool in AWS that can provide custom report for Data Quality for Timeseries data? - Asking because the timeseries data is simple, usually having following fields, [Device, Timestamp, Sensorname, Value] where the sensors can be unevenly spaced timeseries etc. Too many options but nothing complete.FollowComment"


Question: Can i execute an Athena saved query from lambda?

"I have a saved query in Athena, that i want to run from Lambda.As per: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.htmlIt seems that the SDK can only execute a query if you provide the SQL.(needs to be node.js in lambda)Is there a way to:var params = {QueryName: '<saved query name>, /* required */},athena.startQueryExecution(params, function(err, data) {if (err) console.log(err, err.stack); // an error occurredelse     console.log(data);           // successful response});FollowComment"

Answer: "1Accepted AnswerI generally use the following steps.get all query ID'sget the query details for each query IDcheck if its the right one & do somethingCode:Get Athena client/objectclient = boto3.client('athena')# Get all the saved queries in Athenaresponse = client.list_named_queries()# Get the named query IDs from the responsenamed_query_IDs = response['NamedQueryIds']# Go through all the query ID, to find the delete & create queries we need to runfor query_ID in named_query_IDs:     # Get all the details of a named query using its ID    named_query = client.get_named_query(        NamedQueryId=query_ID    )        # Get the query string & query name of the query    querystring = named_query['NamedQuery']['QueryString']    queryname = named_query['NamedQuery']['Name']        # If its a create query, add it to the list of create queries    # We also replace the '/subfolder' string in the query with the folder structure for the current month    if 'create_linked_' in queryname:        new_query = querystring.replace('/subfolder', currentmonth)        create_query_strings.append(new_query)            # If its a delete query, add it to the list of delete queries to execute later    if 'delete_linked_' in queryname:        delete_query_strings.append(querystring)CommentShareMODERATORAWS-User-4217071answered 4 years ago1Please check out the section from ** AWS Well-Architected Labs** ,CREATE LAMBDA FUNCTION TO RUN THE SAVED QUERIEShttps://www.wellarchitectedlabs.com/cost/300_labs/300_splitting_sharing_cur_access/4_lambda_function/CommentShareSwwapnilanswered a year ago"


Question: DeepLens unable to register through PC or direct through device

"I have purchased a HW v1.1 DeepLens and I am unable to register the device either through the PC or directly through the device using micro HDMI / Monitor / HBU breakout port with wired keyboard and wired mouse.  For PC, no LAN, No VPN and no blocking from Firewall, but no recognition or registry.  For connect direct to device, three screens scroll within 1 second at start-up and then the device goes dormant, monitor goes into hibernate because of no output signal from the device.  I am preparing to return three DeepLens devices and give up on this initiative.  Any support is appreciated.FollowCommentbloktimeDev a year agoI have the exact same problem. I am very disappointed in the complete lack of support or documentation from AWS about thisShare"


Question: How many days is normal for the DNS not updating? I'm on day 2

"Hi, I updated my DNS records yesterday so I could use an email sender (domain dns verification) and know it can sometimes take hours, ive seen it take hours, minutes, but never more than 1 night of sleep. Am I doing something wrong or is amazon's DNS just really slow? How long should I give it? Im waiting 2 days on mailchimp and twilio's email company and now even a new domain I registered is waiting to be verified, via dns, by Lightsail. IDK why its taking so long.FollowCommentGabriel - AWS 9 months agoA few more details would be helpful to diagnose the issue:Did you update the DNS records in Amazon Lightsail or Route 53?Do you have other records that are working successfully on that same DNS zone?Where did you register the domain?Is the registrar correctly configured to point to the NS servers on that DNS zone? (Please note that each DNS zone has different NS servers so if you re-created the DNS zone you'll have to update them again with your registrar).Share"

Answer: "0You can refer to this link to see possible reasons why your new DNS settings haven't taken effecthttps://docs.aws.amazon.com/Route53/latest/DeveloperGuide/troubleshooting-new-dns-settings-not-in-effect.htmlhttps://docs.aws.amazon.com/Route53/latest/DeveloperGuide/troubleshooting-domain-unavailable.htmlCommentShareEXPERTAWS-User-Nitinanswered 9 months agorePost-User-2742014 9 months agoIt wasn't any of these, for anyone that finds this answer, the answer is your DNS settings are being set in Lightsail if you have a DNS zone attached to your lightsail. Making the changes there basically instantaneously worked faster than any DNS resolution ive ever seen.Share"


Question: Regarding sorting in s3 objects

"Hi,I am sending data from IOT core to AWS kinesis firehose to s3 using AWS iot rule. I am able to get all data in s3 bucket. The problem i am facing is that data received in s3 objects are stored in random manner. How to solve this? or is there any other ways?FollowComment"

Answer: "0To collect your data in a meaningful way, you can use the dynamic partitioning capability in Kinesis Data Firehose. Typically, you'd use a time-series based partition key for IoT ingested sensor data. When your data is delivered to S3, you'll have a prefix for your bucket and it can include year, month, day, even hour.https://docs.aws.amazon.com/firehose/latest/dev/dynamic-partitioning.html"ExtendedS3DestinationConfiguration": {  "BucketARN": "arn:aws:s3:::my-logs-prod",  "Prefix": "customer_id=!{partitionKeyFromQuery:customer_id}/     device=!{partitionKeyFromQuery:device}/     year=!{partitionKeyFromQuery:year}/      month=!{partitionKeyFromQuery:month}/      day=!{partitionKeyFromQuery:day}/      hour=!{partitionKeyFromQuery:hour}/"  }CommentShareJason_Wanswered 4 months agokushal 4 months agoHi @jason WThanks for your comments..Actually I am doing the same that is used dynamic partitioning for s3 bucket. This part is ok. problem is data received in the objects of bucket are comes in random manner. for example 2nd packet comes first and 9th packet comes second and so. since i have to plot using these data, it is important to me that it comes in proper sequence. I have checked sequence in iot core and it is correct but in S3, it is not proper.Share0The object names created by Firehose are random, but you can query the content of the files using Amazon Athena.You can follow this tutorial https://docs.aws.amazon.com/athena/latest/ug/getting-started.html to setup Athena and query your data source.You would likely need to adapt the tutorial with a SerDe matching your file contents.  Refer to https://docs.aws.amazon.com/athena/latest/ug/supported-serdes.html for a list of supported Serializers/Deserializers.As an alternative to Firehose + S3, you can use AWS IoT Analytics. Check this workshop for a guided how-to. https://catalog.workshops.aws/aws-iot-immersionday-workshop/en-US/aws-iot-analytics/lab20-settingupanalyticsCommentShareEXPERTMassimilianoAWSanswered 4 months ago0Hi,Thanks for the reply.problem is not that object name created is random. actually data in object is random and we want to get that data in actual sequence or sort it in some way.Scenario is that we have 10 devices that is sending data to iot core. I want to send data in sorted manner to S3.  I am using firehose and data stored in object using partition key but in random order.If i use iot analytics, all data stored in one object instead of 10 data in 10 files.what to do in this scenario?CommentSharekushalanswered 4 months ago"


Question: Pass additional information to Lambda triggered by S3EventNotification

"I have an architecture were a client uploads an image to an S3 bucket and this upload triggers a Lambda function through an S3 event notification.This Lambda does some pre-processing on this image and then saves it on another bucket under the client's folder. However, this Lambda function does not know who the user is and thus cannot save the image in the user's folder .If the Event Notification was able to pass some extra information, like user ID, to the Lambda this could be done.Is there a way to do this or an alternative solution?FollowComment"

Answer: "0When the client uploads the image to S3 add a metadata to it which contains some user information. A sample of how to add a metadata is demonstrated in the link belowhttps://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/s3-example-upload-bucket-item-with-metadata.htmlNow in your Lambda you can read the metadata object using s3.head_object(Bucket=bucket, Key=key)CommentShareDebasis Mitra AWSanswered 5 months ago"


Question: Bug report: AWSEC2-ConfigureSTIG fails on AL2 if /tmp contains an item with a long user/group like webapp

"The state association will fail with this error:  "The staging directory is not currently owned by the root account. Exiting."This is caused by the shell script not expecting a single space in between user and group.  For example, Elastic Beanstalk runs as user webapp and group webapp.  This causes entries created in /tmp to look like this in ls  (notice the space between webapp and webapp:drwxr-xr-x   2 webapp webapp    6 Sep 18 01:00 uploadsThe script itself is looking for entries looking like:drwx------ 2 root   root      6 Sep 19 17:13 tmp.ZU79vTNUjsNotice the multiple spaces between root and root.Not using /tmp from webapp is the only fix I have found so far.  Having spent many hours on this, I hope this may help someone else.FollowComment"

Answer: "0The problematic line is this one:(ls -l /tmp/ | grep "$stagingDir" | cut -d' ' -f 3,4 | grep -E -q "^(\s*)root\s*root\s*$") || { echo The 'staging directory is not currently owned by the root account. Exiting.'; exit 1; }The cut command is cutting the user and group apart in two different field when the delimiter becomes  a single space, breaking the following grep command.CommentShareDanielanswered 9 months ago"


Question: VPN being used for malicious attacks

Hello!I am a very novice customer and normally do not deal with VPN. However a couple of times now incidents have been identified where our team VPN has been used in probing/brute force attacks. For reference we allow BYoD and the VPN is used mainly for WorkDocs/Workmail access.I have asked users to scan their devices for malicious soft to stop the attacks. However I need assistance with two issues:-how do I identify exactly which of my users' devices is the source of issueis there a way to configure my VPN to prevent it from allowing similar brute force attacks from being carried out in the future?Appreciate any assistance in advance.FollowComment

Answer: "0Not sure if you have tighten the firewall rules and security groups. To avoid it in future make sure you have open the required ports on firewall, along with security group.you should check your LAN/office network too to make sure any malicious machine can not connect to network and should be quarantine immediately from rest of your network.Check AWS best practice to avoid such incidents in future.you might need to do some hard work.CommentShareSachinanswered 3 months ago"


Question: ContainerOverrides parameter

"Hi,I am using Constant JSON text to define the parameter for my Batch target.I am having problem overriding the vcpus for the target job. I am aware the parameter  naming should be in camel case but I having problem overriding the cpus.For example, I want to override the vcpus to 6, but below will run the batch job but fail to override the vcpus.{     "ContainerOverrides": {  "VCpus" : 6, "Memory" : 2000,  "Command": ["run_job.sh"]}}Thanks in advance.FollowComment"

Answer: 0I was close. It's Vcpus.CommentSharehopanswered 3 years ago


Question: Can't launch any aws-cloudhsm-pkcs11-examples because of C_Initialize error

"Hello,Can't launch any aws-cloudhsm-pkcs11-examples. C_Initialize() returns error code 5 (CKR_GENERAL_ERROR). What is missing?I'm trying to integrate CloudHSM/PKCS11 library into existing C++ application. I started by studying official examples from https://github.com/aws-samples/aws-cloudhsm-pkcs11-examples.git.However no example is successful and stuck on C_Initialize() call. It returns error code 5 (CKR_GENERAL_ERROR). Tested on AWS Ubuntu 16.04 instance. What should be done to avoid this error?$ src/encrypt/aes_gcm --pin ...Failed to connect socketFailed to initialize$ make test0% tests passed, 16 tests failed out of 16$ gdb src/encrypt/aes_gcm78          rv = funcs->C_Initialize(&args);79          if (rv != CKR_OK) {(gdb) p rv$4 = 5FollowComment"

Answer: 0My guess it that I somehow missed CloudHSM client config: https://docs.aws.amazon.com/cloudhsm/latest/userguide/install-and-configure-client-linux.htmlCommentShareDmitrijsanswered 3 years ago


Question: "data loading to s3 in csv format  is adding line breaker in rows, randomly to one column"

"we are fetching my data from pentaho , through snowflake, loading to s3. the data in pentaho is accurate ,but while it travels to s3 bucket , for a column,  some portion of the data is randomly divided and send to next row, the data doesn't any new line (\n), for delimiter we use | as a symbolFollowComment"


Question: SES denied production access without a reason

"I am working in a college research organization and developing health-related apps for youths.We are applying for SES production access because we would like to send health assessment reports to our users.We are plan to send 1-100 emails daily and only send emails when our users request it. We had a detailed plan on handling bounce and complaints.We got denied without a reason and AWS support told us they will no longer respond to our case.(CASE 9697186941)We are not asking to increase the limit, all we want is to remove out from the sandbox environment.I understand that there are people abusing the system but I feel this is unfair to us.I've tried to use the AWS workmail but I don't see an SMTP option there.I also tried to set up an SMTP server on EC2, but AWS block port 25.We try to stick with AWS if possible, does anyone have suggestions?Thank youFollowComment"


Question: Setting desired battery level

"We are trying test our application in AWS Devicefarm. We have created Appium Java Testng tests, which would run on devicefarm real devices. We want to verify the battery usage when we use the app. We are planning to achieve this by comparing the battery levels at start and end of test. We are able to extract the device battery level using Appium, but in most cases the device appears to be in charging state. Is there a way to set the battery level in device farm or pick up devices which are not in charging state.FollowComment"


Question: can i send bulk email messages?

"hi, I have 5000+ of my customer's email addresses. can I send a promotional email using Amazon SES?tnpscrockFollowComment"

Answer: "1Accepted AnswerHi!  Yes, this is possible via Amazon SES.I would also check out Amazon Pinpoint (similar to SES) for your promotional email needs.Keep in mind with SES, there are limitations such as # of messages sent in a 24-hour period and only sending to verified emails if your account is still in SES Sandbox mode: https://docs.aws.amazon.com/ses/latest/dg/request-production-access.html.  You may need to move to Production Mode to fully support your use case.  However, AWS will review the request to ensure SES is not being used to send unsolicited or malicious content.CommentSharejsoncanswered a year agoNathaniel Ng a year agoFYI there's info on how to monitor your sender reputation here (in particular the bounce rates and complaint rates): https://docs.aws.amazon.com/ses/latest/dg/monitor-sender-reputation.htmlShare0thanks for your supportCommentShareAWS-User-8809155answered a year ago"


Question: Need guidance with PBR Material work flow

"Hello, so I was playing around with the material editor justrecently. I did watch and read the tutorial on making and applying textures toobjects. However was that the workflow for PBR materials? Please forgive me if Idon’t quite understand the difference between PBR and regular textures, ifthere is any. I won’t be using Substance Designer for my project(s). I amactually using Quixel’s Megascans and an application they have called MegascansStudio to blend materials together. Well I have an albedo map, an AO map(whatever that is), displacement map, Gloss map, Normal map and a Roughness map.These are all of the files they give me when I export my texture. I’m actuallyapplying the texture to the terrain.In the material editor which I bring up by pushing the “M”key, there are options to import a diffuse map, normal map, specularity map,and I think a detail map. Besides maybe the normal map, I’m not sure what elseI am supposed to do with those other files I got from creating my material inMegascans. Can anybody help me understand what I am supposed to do and how toget my material into the engine?Follow"

Answer: "0Accepted AnswerThis is going to be lengthy, so bear with me :).LY documentation:Docs on texturesTerminology (Lumberyard specific):Albedo = Diffuse, where the diffuse map has no shadows or highlights but only the raw surface color information (Albedo as a term was introduced so no one would mix it up with the old diffuse maps, which were allowed to have shadows and highlights in them)Glossiness = 1 - Roughness (meaning they depict the same physical property only inversed) Glossiness goes into the alpha channel of the normal map, which is done to save memoryThe key is to name the file <whatever>_ddna.tif so LY recognizes the special format.Roughness is not used by LY.Specularity: Try to avoid maps for that unless you are trying to create a metal material. Chose a solid color around 58 to 80 greyNow let me shamelessly link to my tutorial on texturing terrain because terrain has one difference to normal PBR materials in Lumberyard in that you will want to use highpassed textures as described in the video:Terrain texturing tutorialNow for PBR vs. "regular" texturing: It's really simple, earlier textures had too look good and then looked good for exactly the light setup they were created for. PBR textures as the name says have to be accurate to look good, but then look good for any light condition. Regular textures were really just images whereas PBR textures hold data such as the albedo of a material or its microsurface texture. This means it's now easier to create realistic looking materials because you don't have to be an artist anymore but simply know which material has which value. The goto guide on PBR is probably by Marmorset.Marmorset Guides on PBRThere are two major workflows: Spec/Gloss and Metal/Roughness. LY uses Spec/Gloss which means you need:Albedo, with Spec/Gloss it tends to be called "Diffuse", again just a naming thingGlossiness (How "not rough" is a material at a certain spot?)Normal map (Always needed, together with Glossiness makes or breaks a material)Specular map (Only needed for metals or materials with parts of metal. Otherwise use uniform grey with values around 58 to 80Hope that serves as guide. PBR is a huge spectrum of knowledge so condensing it into a single post is difficult.SharerePost-User-0887237answered 6 years ago0Well that really clears things up on my end. Thanks so much for the links and the video tutorial. It will help immensely. I certainly need to read up more on materials.SharerePost-User-7372643answered 6 years ago0Nicely done @Ihatenames! :)SharerePost-User-5738838answered 6 years ago"


Question: Athena how to query different table versions.

"I have a workflow that retrieves data and stores it on S3 bucket (Database snapshot). For the first time that the workflow ran, it created one folder for each table. So for example let's say I had table products, then it created a folder for that table. Now inside of this folder there will be a subfolder called version_0. Every time the workflow runs again, it will fetch the data, and create a version_1, version_2, so forth.On Athena, when I query this table, it will give me data from the latest version created. How can I access for example version_0 through the query?Following this documentation , I tried to run the following:select count(*) from products FOR SYSTEM_VERSION AS OF 0But I'm getting the following error:To be honest don't even know if following this documentation that I sent above is the way to go or if there is some other way.I also looked into the Table properties going into AWS Glue -> Tables -> products table and then I see that there is a field TableVersion. I can click on the "Compare Versions" button and it will give me a side by side between version 0 and version 1 (the current versions I have for now).ThanksFollowCommentAnanth Tirumanur 10 months agoThe version parameter is the bigint snapshot ID associated with a governed table version.Share"

Answer: "0Hello, Firstly, Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. If you are seeing only the data from the latest version in Athen that means you only have the latest version of data in the underlying S3 folder. As per your description it seems that the new versions is overwriting the old version in S3.Secondly, if you are able to compare versions in Glue that means that a new version of table is getting added to glue catalog each time data is unloaded into S3.Lastly, if you are looking for an ability to query all the versions of your table snapshot. You have to unload and store the data of each new version as a new folder under products (example version_1, version_2 etc ) and  add the same to glue catalog. You can consider using glue crawler [1] for adding the new partitions to the table.————————Reference:[1] https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html================ Have a nice day!CommentShareSUPPORT ENGINEERArunanswered 10 months agorePost-User-4108613 10 months agoHi Arun,Thanks for answering. I did what you mentioned, (create a crawler to add the new partitions to the table). But the result was just 2 different tables (1 table for each version I guess), and each table starts with a name "part_00000_c5e9..." . Also the link you sent is quite general, could you please specify the documentation that actually aggregates the different partitions/versions into one table?Thank youShare"


Question: xvdca=ephemeral0 - 25 being created without reason

"So we are creating AMIs for our instances and there are references to  xvdca=ephemeral0, 1, 2 and so onThis is causing issues with Lambda functions we are running, any ideas why it is doing this?See image for examples under block storage for the AMIAlso worth adding, it is not doing it to all AMIs that are being created, but is happening to most of themFollowCommentrePost-User-9605189 4 months agoWorth adding that the volume type for that volume is EBSShare"


Question: Single Logout using SAML authentication option

"I've been able to get single sign on (SSO) working between my custom app (using Amazon Connect Streams library) and Connect, but have not found any documentation for configuring single logout (SLO). I have configured my custom application for SLO, so that when a user clicks a logout button from my app, they are logged out from the IdP and redirected back to the sign in page. The problem is, this does not log the user out of CCP. In the agent reports, they are still in the Available state (or whatever state they were in when the user logged out from my custom app).Does Connect support SLO? Or does the user have to logout of CCP, then logout from my custom app?FollowComment"

Answer: 0Set the state then goto the logout page:See here: https://github.com/aws/amazon-connect-streams/issues/13CommentShareDanBloyanswered 4 years ago0Thank you.Using this as a workaround for logout now:https://github.com/aws/amazon-connect-streams/issues/13#issuecomment-385498723CommentSharedaguecianswered 4 years ago


Question: Kinesis Video WebRTC Storage /joinStorageSession 500

When calling /joinStorageSession I get a response of {"message":"An internal failure has occurred !"} and a 500 status code on about half of requests. The other requests work as expected. This occurs both in my own test application and in the demo application here. The relative code for the linked demo is the joinSession function within the master.js file. I was wondering if this is a known issue with the webRTC ingestion preview mode and if there's anything I could do to get more consistent performance.FollowComment


Question: "Brotli support with Amplify, how?"

"I'm usiong amplify for a vue3 app, I have webpack building both gz files and br files.  The gz file are serving but the br files will not  I have added the file extensions in the amplify redirect: </^[^.]+$|.(?!(gz|br|css|gif|ico|jpg|js|png|txt|svg|woff|woff2|ttf|map|json)$)([^.]+$)/>but brotli is not being served, any ideas on why and how to fix this?gzip and brotli are both on in the cloudfront policy > cache > compression support for the "managed-amplify"FollowComment"


Question: Amazon Location Service - Esri Imagery - How to add streets

"I'm attempting to evaluate Amazon Location Service as a replacement for MapBox.The available map styles are here: https://forums.aws.amazon.com/forum.jspa?forumID=394&start=0This page mentions "Esri map styles that are not listed in this section are currently not supported."Is there a way to overlay streets on the "Esri Imagery" (aka Satellite Imagery) map?Notes:I've been using this maplibre-gl-js-react AWS sample as a guide: https://github.com/aws-samples/amazon-location-samples/tree/main/maplibre-gl-js-reactInstructions for creating custom styles are here: https://github.com/aws-samples/amazon-location-samples/tree/main/create-custom-map-styleUnfortunately I'm not able to get that code to work and I seem to be at a dead end there.However, it is not clear to me whether I would be able to pull a streets layer onto the "Esri Imagery"FollowComment"

Answer: "0Hello, thank you for the question!Yes, it is possible to overlay streets on Esri Imagery. The recommended way would be to use a layer that is tailor made to be overlayed over raster basemap for roads, streets, and such.However, with custom map styles, you can add a vector tile layer over satellite imagery and only overlay the streets.An example blog is here: https://aws.amazon.com/blogs/mobile/create-a-custom-map-style-with-amazon-location-service/CommentShareSwagata Prateekanswered a year ago"


Question: Can i still upload files to S3 using the AWS SDK when i restrict access with an CloudFront distribution?

"I want my users to only access files via the cloudfront distribution. I've found this article that explains how to do that.My question is though; If i restrict access like that, will my web application still be able to upload files to amazon s3 buckets directly or does it have to use cloudfront for that as well?Thank youFollowComment"

Answer: "3Accepted AnswerAssuming your web application is running in EC2/ECS, you can create an IAM Role with permissions to access the Bucket and attach it to your EC2 Instance(s).  The Role will authorise SDK calls made from the instance (assuming the correct permissions are defined) without you having to create/manage access keys.As long as your Buckets access Policy/ACLs are configured to allow access from the account/role your web application will be able to continue using the SDK as standard.Note: IAM Roles & Bucket ACLs are notoriously tricky to get right so I strongly advise getting the configuration right on a UAT environment first.CommentShareDaniel Craigieanswered a year agoViridios a year agowhat is an UAT environment?ShareMG a year agoUAT means User Acceptance Testing environment. So Daniel just mentioned that you should always test your solution before going live with it.Share1Like Daniel Craigine wrote, of course, you can use SDK to upload new files to this S3 Bucket.Origin Access Identity (OAI) is the way how the CloudFront is authorized to get objects from S3 and you need to allow OAI permission in Bucket Policy as is explained in the article you mention. But you can still add your application permission to PutObjects to this S3 Bucket in the IAM Role used by your application.I hope it is clear right now :)CommentShareMGanswered a year ago"


Question: aws configure sso issue

aws configure ssoSSO start URL [None]: <MyURL>SSO Region [None]: eu-west-1SSL validation failed for https://portal.sso.eu-west-1.amazonaws.com/assignment/accounts [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)I get above error while configuring aws sso and not able to resolve it.FollowComment

Answer: "0Hi,I understand that you're trying to configure AWS CLI to use AWS Single Sign-On. But, you are receiving the CERTIFICATE_ VERIFY_FAILED error message while doing so.Please note that the cause for this issue is addressed in the following documentation:Troubleshooting AWS CLI errors - I get a "[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed" error. - https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html#tshoot-certificate-verify-failedWhen you use an AWS CLI command, you receive a "[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed" error message. This is caused by the AWS CLI not trusting your proxy's certificate due to factors such as your proxy's certificate being self-signed, with your company set as the Certification Authority (CA) (in case this is the configuration on your end). This prevents the AWS CLI from finding your companies CA root certificate in the local CA registry.To fix this, instruct the AWS CLI where to find your companies .pem file using the ca_bundle configuration file setting, --ca-bundle command line option, or the AWS_CA_Bundle environment variable.https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-config-ca_bundlehttps://docs.aws.amazon.com/cli/latest/userguide/cli-configure-options.html#cli-configure-options-ca-bundleIf you're using an older version of CLI, I'd also recommend updating to a newer version and testing.[] What is the AWS Command Line Interface? - About AWS CLI version 2 - https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html#welcome-versions-v2The AWS CLI version 2 is the most recent major version of the AWS CLI and supports all of the latest features. Some features introduced in version 2 are not backported to version 1 and you must upgrade to access those features. There are some "breaking" changes from version 1 that might require you to change your scripts. For a list of breaking changes in version 2, see 'Breaking changes – Migrating from AWS CLI version 1 to version 2' documentation below.[] Breaking changes – Migrating from AWS CLI version 1 to version 2 - https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.htmlIf you did want to install the latest of AWS CLI version 2, please follow the guide below:[] Installing or updating the latest version of the AWS CLI - https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.htmlPlease let us know if you run into any further issues or questions and we'll be glad to assist!CommentShareSUPPORT ENGINEERSumukhi_Panswered a year agoDilip Bharti a year agoI tried with aws configure sso --ca-bundle C:\rootCAs.pem but it gives me same error.SSL validation failed for https://portal.sso.eu-west-1.amazonaws.com/assignment/accounts [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)aws --versionaws-cli/2.4.9 Python/3.8.8 Windows/10 exe/AMD64 prompt/offShare"


Question: how to run an inference in sagemaker pipeline?

"based on the docs here https://github.com/aws-samples/sagemaker-pipelines-callback-step-for-batch-transform/blob/main/batch_transform_with_callback.ipynb, a separate pipeline is created to perform a batch transform within sagemaker pipeline. the example utilizes a lambda and sqs to achieve this.  couldn't the batch transform job can simply be part of the training pipeline? once the model is trained, and added to model registry, one should be able to query the registry and get the latest model and run a batch transformation job on that, without the callback set up in the docs, right? any examples of running a batch transform job directly from a training pipeline?FollowComment"

Answer: "1Accepted AnswerHi,The solution can be various and it depends on what you are trying to achieve. In general, I believe it is a good idea to build a generic pipeline and utilize parameters for different jobs. The image below shows a typical ML pattern with stages.You can use condition steps to orchestrate Sagemaker jobs, more information with code examples are below:https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-pipelines/tabular/abalone_build_train_deploy/sagemaker-pipelines-preprocess-train-evaluate-batch-transform.htmlHope it helps,CommentShareJadyanswered 5 months agoclouduser 5 months ago@Jady - thank you again. I went through both of the docs and it helped. based on your second comment/docs, all of the processing step , training step , model creation, register, transform ... , all are defined and executed as part of a single pipeline. based on my use case, i would like to have separate pipelines for some of these steps, let's say one pipeline for processing and training, then another pipeline for model creation and registering , and another for testing and so forth. is that possible and if yes, how would one tie these pipelines/artifacts from one pipeline to another ? apologize for adding more question, but promise this is the last thing i'm wonderingShare1Hi,To schedule the batch transformation job with the latest approved model, using a Lambda with an EventBridge rule is (as of today) the best way to achieve their use-case - please find the custom project template for batch inference you could leverage here:https://github.com/aws-samples/sagemaker-custom-project-templates/tree/main/batch-inferenceIn addition you could also add a Transform Step at the end of their SageMaker Pipelines to automatically rely on BatchTransform to achieve their inference once training is done. Reference is here:https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html#step-type-transformThanks,CommentShareJadyanswered 5 months agoclouduser 5 months ago@Jady - thanks. I went through the samples you have provided and looks straightforward. looks like adding a transformer and transformer step , in my current notebook should work. also, is there a way i can extend this, meaning if i copy paste this part of the code to my each notebook, it will work. but i also want to be able to reuse this code in different projects/pipeline. so that i don't have to copy this to every notebook. do i set up a different pipeline alltogether , just for this and utilize the parameters that i can set at the runtime for my different experiments , but how do i connect my training pipeline with my batch testing pipeline , in this case. or is there a different way or better way to do thisShare1Hi,I am glad you find the answer is useful.I have some suggestions which might help, though I I did not connect multiple sagemaker pipelines together.Please check AWS EventBridge -The EventBridge event can be configured to trigger a sagemaker pipeline state changes, reference is here: https://docs.aws.amazon.com/sagemaker/latest/dg/automating-sagemaker-with-eventbridge.html#eventbridge-pipelineEventBridge event can trigger you consequent sagemaker pipeline, reference: https://docs.aws.amazon.com/sagemaker/latest/dg/pipeline-eventbridge.html#pipeline-eventbridge-scheduleAlternatively, you may use a Lambda as last step of pipeline to triggers next pipeline but you will have overhead of managing the lambda.Thanks,CommentShareJadyanswered 5 months agoclouduser 5 months agothanks again.Share"


Question: Unable to create S3 Storage Gateway in N. Virginia

For Support: In the screen after inputting the IP address of the custom EC2 instance and click on "Next" I get a "Page Cannot be Displayed" page. Other Regions are OK. Please fix.FollowComment

Answer: "0Hello,It looks that you are encountering activation issues after providing the IP address of the EC2 instance of storage gateway. I would suggest checking below :Verify that your workstation is able to connect to port 80 of the Storage Gateway EC2 instance. To narrow down, you can choose the option to  "Use default settings" when launching the EC2 instance.Verify the EC2 instance is able to connect to storage gateway endpoints on port 443.Verify that you are using the latest AMI ID for the storage gateway EC2 instance. You can get the latest AMI ID using the following command or launching the EC2 instance from the storage gateway wizard.aws --region us-east-2 ssm get-parameter --name /aws/service/storagegateway/ami/FILE_S3/latestPlease refer this article for further troubleshooting.  Thank you.CommentShareSuryaanswered 6 months ago0Indeed the browser was not able to see the Storage Gateway. Fixed, thanks.CommentShareEric-9462223answered 6 months ago"


Question: scp command difference for Ubuntu EC2 in a VPC

I'm attempting to scp to an Ubuntu EC2 inside a VPC and getting timed out.Established facts:I can ssh into the VPC itself - the keypair works and the instance subnet is open to the internet.The folder I'm attempting to transfer to on the EC2 has permissions 700The command I'm running is: scp -i mykey.pem dumbtest.txt ubuntu@ec2-<my-ip>.compute-1.amazonaws.com:/home/ubuntuAre there additional steps I need to take to scp into EC2's on a VPC?FollowComment

Answer: "0Hello,Regarding SCP'ing into the VPC, I believe you are referring to SCPing into the EC2 instance.As per my understanding, the command you are using is a Pull command from the on-premise via the internet.Have you verified the security groups and NACL settings to exactly what are you allowing in the rules. What is the error that you are getting once you try to copy the folder ? Is it starting and then stopping or you re getting a failure/denied error ?Additionally, Is it possible/Have you tried to zip/compress the folder and then try to copy the same ? You can use gzip to compress the folder and then copy.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered a year ago"


Question: Where can I find Babelfish release notes or recent features?

"I'm looking to find out about recent changes to Babelfish, and find changes by version. I'm running Aurora PostgreSQL-compatible edition.FollowComment"

Answer: "1Accepted AnswerDetailed release notes for Amazon Aurora, including Babelfish changes, can be found here:https://docs.aws.amazon.com/AmazonRDS/latest/AuroraPostgreSQLReleaseNotes/AuroraPostgreSQL.Updates.htmlA simple table of features by Babelfish version can be found here:https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/babelfish-compatibility.supported-functionality-table.htmlIf you'd like to keep up-to-date with new Babelfish features, you can subscribe to the AWS What's New. There is an RSS feed:https://aws.amazon.com/new/Finally- Babelfish is an open source project. If you're looking for release notes from the open source project, Babelfish for PostgreSQL, release notes can be found at the project website.CommentSharelightanswered a year ago"


Question: EC2 Instance gets Unresponsive frequently and SSH connections fail

"I am using an EC2 instance (AP-South-1) inside a VPC with no inbound connections, except those routed by an ELB. The server was working fine since the creation around 2 months ago. Except that since last night, the server gets unresponsive (status check was 1/2 this morning). And now when I SSH into the instance after reboot, I lose the connection in a few minutes. This keeps happening every time I reboot and reconnect. I checked all logs on the server, everything seems to be fine there. CPUS usage is minimal too (less than 20%). I have set up alarms for CPU usage and status check, nothing gets triggered when the instance gets unresponsive.FollowComment"

Answer: "0You don't say what instance type you're using so my guidance is a little vague.First, try using the EC2 Serial Console to gain access to the instance to see why it is becoming unresponsive.Second, if you're using a T-series instance (which runs on CPU credits) check to see if you have exhausted the CPU credit balance. If this is the case I'd suggest using Unlimited mode (which will cost extra in this case); a larger T-series instance; or a non-T-series instance (which doesn't use CPU credits).Finally, if you are continuing to have difficulties definitely create a support case and have our team assist you.CommentShareEXPERTBrettski-AWSanswered a year agoAWS-User-0919530 a year agoHi, I am using a c5.2xlarge instance. I don't think it is a CPU credit related issue. Can you please help me understand what could be the issue hereShareBrettski-AWS EXPERTa year agoMy suggestion above stands: Our support team can help you troubleshoot this issue.  To me it sounds like there is some sort of process on the instance which is causing this and your best path is to determine what that is.Share"


Question: How to concatenate character strings in step function for AWS Glue parameters?

"HI All,I have a step function to call an AWS Glue job with below step definition:    "glue-redshift-call": {      "Type": "Task",      "Resource": "arn:aws:states:::glue:startJobRun.sync",      "Parameters": {        "JobName": "GlueJobRedshiftStoredProcCa-meJ8P3QMSC5p",        "Arguments": {          "--SQLStatement": "call dw_merge_vendor('TNG', 'ASENTINEL')"        }      },      "Next": "post-processing"    },As shown above, SQLStatement parameter is hardcoded as call dw_merge_vendor('TNG', 'ASENTINEL') in this. But I need to make it dynamic, parameter based. As mentioned in Step Function new features Blogpost, I tried below syntax to make it dynamic, but it's not working. Would you be able to suggest the correct syntax if I am doing something wrong here?"Arguments": {  "--SQLStatement": "States.Format('call dw_merge_vendor('{}', '{}'),  ', $.param1, $.param2)"}FollowComment"

Answer: "0Accepted AnswerThere are two issues with what you did. The first is that when you reference other variables or intrinsic functions, you need to append .$ to the key. The second is that when you need to include ' in the string, you need to escape them using \\. It should look like this:"Arguments": {"--SQLStatement.$": "States.Format('call dw_merge_vendor({}, {}),', $.param1, $.param2)"}If you need the params enclosed in ', you should use the following:"Arguments": {"--SQLStatement.$": "States.Format('call dw_merge_vendor(\\'{}\\', \\'{}\\'),', $.param1, $.param2)"}CommentShareEXPERTUrianswered 3 years ago"


Question: Using PartiQL with DAX client for dynamodb

"If we use only PartiQL for Dynamodb calls, do the calls get automatically routed via DAX because we are using DAX cache also in front of some of our tables.Basically how to use PartiQL along with DAXThanksFollowComment"

Answer: "2Accepted AnswerDAX Clients do not yet provide support for PartiQL API's. Unfortunately I do not have an ETA on when it will be released.CommentShareEXPERTLeeroy Hannigananswered a year ago-2The usage of PartiQL and Dax are two different use cases. One is for easier exploratory queries and one is for improved performance. If the users that are using the PartiQL interface feel any issue with the performance of some of their queries, it is a good opportunity to work with them on performance such as a query instead of a scan, adding secondary indexes, or using Dax.This may look as not answering the question, however, from my experience using DynamoDB for many years, it can indicate that you are not using it in an optimal way. It is easy to take a tool and try to use it for all problems, in the same way, disregarding the benefits of each tool or feature. DynamoDB evolved over the year to cover more of the features that you could only get in a relational database such as Oracle or MySQL. Features such as Secondary Index, Point-in-time-recovery, SQL interface, and cache. The two latter are not related in any way and they are serving different needs as I tried to highlight above. It is fine to want to get the SQL-like query to go through the cache, but I believe that you can get better cost-performance in other ways. I suggest that you ask why do I want to support that and then see if you can get the result in a different (better) way.The indication from the product team that they don't support this combination indicates that they also don't see this as a high-priority feature. If you think differently, please explain the use case and we can examine the alternatives, or decide that it should be implemented earlier.CommentShareMLGuyanswered a year ago"


Question: How can I set max_standby_archive_delay setting in RDS?

"Hi all,We're using Postgres 9.6 on RDS and have a master and replica.  When we query the replica we occasionally get a message stating 'ERROR: canceling statement due to conflict with recovery'.One way to deal with this is by setting the 'max_standby_archive_delay setting' on the replica to slow down the replication process .  I believe there's another way to change system settings in RDS.  I don't think 'alter system' commands work on RDS due to superuser permissions but I can't swear to that.Has anyone come changed that setting for their RDS replication before?  If so, how did you do it?Thanks!FollowComment"

Answer: "0Hello larryq,I understand that you are getting the following error "ERROR: canceling statement due to conflict with recovery" and in order to mitigate that you would like to set the paraameter “max_standby_streaming_delay” and “max_standby_streaming_delay”.In order to change the value of the concerned parameters in RDS, you will require to change the same in the parameter group associated to the instance. If you are using default parameter group then you will require to create a custom parameter group and modify the values as per required for the parameters “max_standby_streaming_delay” and “max_standby_streaming_delay”.To create a custom parameter group, you may consider to refer the document:[+] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html#USER_WorkingWithParamGroups.CreatingFurther please note, once you create the parameter group and attach it to the instance, you will require reboot in order for the parameter to be effective.For more information, please find the reference link here:[+] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.DBInstance.Modifying.htmlMoving ahead I would like few more suggestions which you might consider for the issue that you are facing:    (1) Set the parameter "hot_standby_feedback" to 1 at replica.              -       This makes the primary aware about the dependent( contention) queries running at replica.               -       This also prevents VACUUM from removing recently-dead rows and so cleanup conflicts do not occur. If this is applied, it will delay cleanup of dead rows on the primary, which may result in undesirable table bloat. However, the cleanup situation will be no worse than if the standby queries were running directly on the primary server, and you are still getting the benefit of off-loading execution onto the standby.  However, if you want to set the  "max_standby_archive_delay" and “max_standby_streaming_delay” to a higher value, you may consider, but then again if the query is implemented more than the delay, then it will error out. If you know the query time then you may consider to set it accordingly.    (2) Update the "max_standby_archive_delay" and "max_standby_streaming_delay" to value ('T') accordingly at replica.              -       This would allow the queries at replica to run for longer duration (but still within the limit of 'T' beyond which it gets conflicted).              -       But setting 'T' to a way too high value may lead to replica lag.                         You may also consider the following parameter:    (3) Update the "vacuum_defer_cleanup_age" at the primary.                -       This prevents dead rows to be cleaned up as quickly as they normally would be, which in-turn allows more time for queries to execute before they are canceled on the standby, without having to set a high max_standby_streaming_delay. However it is difficult to guarantee any specific execution-time window with this approach, since vacuum_defer_cleanup_age is measured in transactions executed on the primary server.                 Please note, all the parameters mentioned above are dynamic and if the custom parameter is already associated to it, then the dynamic parameter doesn’t require a reboot whereas static parameter requires a reboot.That being said, we strongly recommend you to test the same in the staging environment before moving the same to production.Please let us know if you have any queries regarding the same.Thank you and Have a great day.CommentShareAWS-Prashantanswered 3 years ago"


Question: Domain stuck in "Processing"

"Hi,After what seems to have been a successful update to R20200127 on a 7.1 Elastic Search domain the domain is now stuck in processing for our a week.Can AWS please fix this.Greetz LeonFollowComment"

Answer: "0Hello,If the cluster is stuck in Processing state, It would be suggested to raise a Support case AWS ES so that they find and fix if there is any workflow that is stuck and is leading to the cluster being in Processing state.CommentShareAWS-User-3829235answered 3 years ago0Looks like theres an open issue with ES domains not being able to complete loading or processing. We don't need to pay for technical assistance in order to report bugs so that AWS team can fix them.CommentSharefoxyanswered 3 years ago0Even while stuck in "processing" I was able to schedule the update to patch release R20200302. After this update was done my domain is now in "Active" state again. Go figure.But I agree with foxy. It would be nice I one could raise issues with the ElasticSearch Service team when this happens. Because I feel it's a problem of the service and customers have no way of fixing these issues themselves. Unlike EC2 where you could for instance just launch a new instances when a instance fails.CommentShareleonblcanswered 3 years ago0The above gets even more annoying if you're under Basic Support and can't do anything about the Processing state. I'll open a new thread on that.CommentShareyi2022answered 3 years ago0Hi ,I have de same issue , i change my ES node  instance type and the cluster status is stuck in "processing" por 24 hour and i see that not deploy blue/green occur becouse de count nodes is the same .What can I do to fix this.Region: us-east-1Domain Name: plink-transactionsCommentShareMariuanswered 2 years ago0Hi ,I have de same issue , i change my ES node  instance type and the cluster status is stuck in "processing" por 24 hour and i see that not deploy blue/green occur becouse de count nodes is the same .What can I do to fix this.Region: us-east-1Domain Name: plink-transactionsCommentShareMariuanswered 2 years ago0Still broken....it's an absolute disgrace that AWS expect YOU to pay to notify them of THEIR bugs. Fix your broken service then I'll consider paying for support.Edited by: JoshFreemanViion on Feb 1, 2021 4:21 AMCommentShareJoshFreemanViionanswered 2 years ago0Today, AWS development team confirmed increased error rates impacting Create and Modify operations for Elasticsearch Domains in us-east-1 region but their dashboard doesn't report this.Logging this one here.CommentShareaws-devopsanswered 2 years ago0Seeing domains stuck in "Loading" in us-east-1 and ca-central-1 all day today.Domains are taking an unusually long time to delete.Creating new domains does not work at all.CommentShareRistoanswered 2 years ago0Create and Modify operations were experiencing increased error rates and latencies.The issue has been resolved and the service is now operating normally.Resolved in 15.5 hrsCommentShareaws-devopsanswered 2 years ago0Hi,I got this problem yesterday (2021-04-06) because I added the EBS storage size to max limit and it's still in processing status after more than 24 hours. if you go to edit domain, the status of processing is in storage configuration (Storage configuration is Processing), data node is not in the list of instances and the status of the cluster is red. my account is basic how can i fix the problemregion: us-east-1ES version: 7.4Edited by: indraaarmy on Apr 6, 2021 7:55 PMCommentShareindraaarmyanswered 2 years ago0Hi,My Domain Stuck in Processing status again, but health status is green.region: us-east-1ES version: 7.4Edited by: indraaarmy on May 21, 2021 7:41 AMCommentShareindraaarmyanswered 2 years ago0same issue here, AWS expects you to buy a support plan to troubleshoot such issues which is totally unacceptable. my cluster is stuck in pending state form past 48 hours without support subscription i don''t know how am i going to resolve this issue?CommentShareadithyaSanswered 2 years ago0There are several reasons why a cluster can be stuck in a processing state and none of them is due to a bug. This issue almost always affects clusters which do not follow the Elasticsearch Best Practice and the most common culprit is lack of resources due to too much load on the cluster.This load can be due to:a. Making configuration changes when the cluster is too busy. A cluster can be busy because of client requests or just because it is constantly relocating shards.b. There are too many shards in the cluster given the available CPU cores and heap in your chose instance type. The workflow can timeout if there are too many shards to migrate. Ideally a node should have less than 1000 shards or have up 25 shards per GB of heap. The same can happen if the shards are too big.c. Making concurrent configuration changes. AWS Elasticsearch uses other services like KMS for encryption, S3 for snapshots, IAM for permissions, etc. All these services need to know which is the current configuration version of Elasticsearch to point to. Making many changes at once with a poorly resourced cluster is bound to break the workflow somehow.These are the major causes of broken workflows. Not a bug.CommentSharempumsanswered 2 years ago0That is a load of rubbish @mpumsThe AWS ElasticSearch is an absolute joke, it's utter useless. Want to restart it? No chance. Want to know why it's stuck processing? Go through 32 different trial and error and you'll find out it's none of them.I had an ES that was running fine, its literally there just to be connected to and doesn't hold any data. It's practically 0% Storage usage, it's 1-2% cpu/memory usage, it has 2 shards, it's private, it gets about 3 connections a day and for some f*cking random ass reason it crashed 4 days ago. FINE for weeks, then poof random node crash.... When it's basically sat idle..I've since moved everything to a custom EC2 because a monkey can run ES better than AWS can.CommentShareJoshFreemanViionanswered 2 years ago0Same thing. Completely down an out right now because of thisCommentSharegunnertechanswered 2 years ago"


Question: Suddenly no permissions to do anything anymore in EC2

"Hi everyone.I have the root EC2 account and from one day to the next I cannot do anything anymore in EC2, this started happening end of April :All the IAM users under that root account had the same problem. I cannot even go and manage IAM users anymore, I litterally have permissions to do nothing anymore.But today one of the IAM user under the root account could access everything, he can see and manage all the VMs ....I'm very confused about this and how to solve it.All error messages say that I should contact my admin. But this root account is part of the large organization I work for and I have no indication who is the exact admin for me, so that's my main problem. Is this something inside my org that happened ? I read that some permissions changed in AWS early 2023 and access could be lost if you don't move to a new time of permissions or sth ...FollowComment"

Answer: 0It looks like your account has been locked. Open up a Account and billing support case.CommentShareEXPERTkentradanswered 20 days ago


Question: Problem in accessing AWS lambda functions

"In the past week, I made a couple of lambda functions and integrated them with Amazon Lex. At that point in time, everything was running smoothly ranging from the execution to logs being generated in the cloud watch console. But today when I checked the lambda dashboard then I couldn't find any function!!There is an error coming up and after inspecting the same in the browser, I can see that from the backend itself the functions object is null whereas I had around 4 to 6 active lambda functions in the account. This is weird because I haven't touched any one of them recently. What's more astonishing is that the lex bot with which I integrated the lambda has no lambda function associated with it now.Previously I used one lambda function as the dialogCodeHook in order to validate the slots, but now the checkmark for the lambda initialization has been removed automatically.Is this a temporary issue or is it with me only? Please help me to get things back into the place they were before. If the previous functions can't be restored then it would be a big problem for me. (Also I can't make a new function). I hope this thing gets solved as early as possible because there is no other way for seeking technical help/assistanceFollowComment"

Answer: "0Hi - There can be few possibilities here.Your permission has changed orSome admin or someone removed resources. orYou are not in correct regionPlease contact your administrator and also verify your roles. Would also suggest to reach out to support team to investigate further.CommentShareEXPERTAWS-User-Nitinanswered 6 months agorePost-User-4833170 6 months agoThanks for your reply @nitin. But in my case, none of the 3 possibilities are there. I am in the correct region as initially ie US East-1 (North Virginia) and apart from me, no one has access to my account (no administrator is there).I tried switching to different regions but the problem is the same. I am not able to view the previous functions as well as can't make new functions. By the way, did you notice that in the initial screenshot that I shared, the page says "Last fetched 53 years ago". I am sure 53 years ago even internet wasn't in existence. Really don't know why that's happening.Can you tell me how can I raise a support ticket? Because as of now I know that for Basic users, AWS doesn't provide technical support. If that is the case, then it is very disappointing because even the basic users have some important use cases of AWS.Share0Oh, just got the access to my account back. Actually it was due to a false compromise alert that led to my account suspension. A little chat with the support team sorted the thing outCommentSharerePost-User-4833170answered 6 months ago"


Question: Amazon Inspector v2 ECR enhanced scans are not finding expected vulnerabilities

"We have turned on Amazon Inspector v2 (15-day trial) and we are scanning both EC2 instances and ECR images (continual scanning).  We have container images that are based on the following .NET 6.0 runtime, which itself uses a Debian 11 (bullseye-slim) base image:mcr.microsoft.com/dotnet/aspnet:6.0When scanned by Inspector v2 enhanced scanning, there are no vulnerabilities found ("Image is actively being scanned, no vulnerabilities are currently found").  However, when we scan the image locally using docker scan (which runs on Snyk engine), we can see that there are 38 vulnerabilities found, including 1 critical, 2 high and 35 low vulnerabilities.Similarly, when scanning a Debian 10 (buster-slim)-based image (e.g. mcr.microsoft.com/dotnet/aspnet:5.0), we see that some vulnerabilities are reported by enhanced scanning in Inspector v2, while considerably more are found when running docker scan locally on the image.Why is Amazon Inspector v2 not reporting on these apparent vulnerabilities?FollowComment"


Question: "EC2 instance was terminated, with new instance running, appears to be a problem with DB"

"For some reason, out of nowhere AWS autoscale decided to shut down our EC2 instance and a new one was launched, but now, even if our Backend is up and running on an ECS cluster container and the DB is running on RDS, there seems like our DB is unreachable, meaning on my website/application appears that the DB is offline.I have admin privileges, but I did not setup AWS, I just suppose to 'maintain' it and although im not completely clueless I'm not expert in the field.I'm unable to figure out what causing our issue. Security groups, and VPC are all setup correctly, havent touched it, all that got sorted automatically with the new autoscale group/template launch.If anyone could direct me or suggest some basic troubleshooting options(even though I feel like i checked most of them but still) that would help, would be much appreciated.Thanks,CsabaFollowComment"

Answer: "0Accepted AnswerOkay if ever someone to run into such, the problem was that at EC2 Load Balancing/Target Groups were not associated with the new instance.CommentSharecsaba-kovacsanswered 3 months ago"


Question: AWS Free Tier usage limit - hit 85% limit after a month

"Hello,I am extremely new to the AWS world.  I just finished a personal project where I used S3, EC2 and RDS.  Though I haven't done much on the website.  I deployed my website on Oct 04 and Got a notice in Nov 03, sayin that I have used up 85%  of my free tier.  I also turned off the project on Nov 3rd as well.  I also created the project/website utilzing django and REACT.It would be greatly appreciated if some one can give me some info or direct me to the correct sources to understand this problem bettter and why/how did i already use up 85% of the free tier in a barely a month.Thank you in advance whoever replies!  It is greatly appreicated!FollowComment"

Answer: "1The alerts you're receiving are per-service, so this means you've used 85% of the free tier allowance for this one service. The free tier for EC2, for example, is 750 hours per month, which is just over enough hours to run one instance for the whole month. Assuming you had these instances running continuously from October 4th, it would make sense that you had used 85% of your free tier allowance for EC2 for the month. This resets every month, so you will have 750 hours to use every month for the first 12 months.More information on the free tier is available here: https://aws.amazon.com/free, and you can view your current free tier usage in the console here: https://us-east-1.console.aws.amazon.com/billing/home#/freetierCommentShareEdanswered 6 months ago0Hi, as described in https://aws.amazon.com/free/ you get an allowance per month, so it's probably just saying you used 85% of your month's allowance, but you get more the next month (for 12 months).  The monthly 750 hours for EC2 and for RDS for example each allow for a single instance running 24x7 for just over 31 days.CommentShareEXPERTskinsmananswered 6 months ago"


Question: Joining an AWS Managed Microsoft AD to an existing domain

"Hi,Im new to AWS MM AD. We have an amazon direct connect to connect our on-premise and AWS VPCs. I have a few questions.can we create an AWS managed microsoft AD that has the same domain name as our existing?can we create an aws manage microsoft AD and join it to our existing microsoft AD?can we create an aws managed microsoft AD in the same VPC as our on premise EC2 instance of Microsoft AD?In managing aws managed microsoft AD do we need a jump machine to do that?FollowComment"

Answer: "0Accepted AnswerQ) You mean to say its another domain from my existing on-prem?Ans: If you are asking this for trust creation then the answer is yes, create a trust between AWS Managed AD and your on-prem AD but on-prem AD and AWS AD should have different names.Q) By the way is autojoin a feature only for AWS managed AD? If I have an EC2 instance with AD role inside can it be able to use autojoin to domain feature?Ans: If you want the feature of autojoin and use services like WorkDocs, WorkSpaces for on-prem AD(AD on EC2) please create an AD connector for this AD and you will have all these features. Please refer the below mentioned articles for details and pricinghttps://docs.aws.amazon.com/directoryservice/latest/admin-guide/directory_ad_connector.htmlhttps://aws.amazon.com/directoryservice/other-directories-pricing/CommentShareRobin-Panswered 2 years ago0can we create an AWS managed microsoft AD that has the same domain name as our existing?can we create an aws manage microsoft AD and join it to our existing microsoft AD?No, AWS Manage Microsoft AD is provided as a single domain AD forest that, as the name implies, is fully managed by AWS.  We retain Domain Admin rights and do not grant permissions that would allow you create your own domain controllers.  Instead we encourage you to create a Trust between the managed domain and your on premise domain.  In order to create a trust the domain names can not conflict.  Therefore you should not create an AWS Managed Microsoft AD domain that has the same name as your existing domain.can we create an aws managed microsoft AD in the same VPC as our on premise EC2 instance of Microsoft AD?YesIn managing aws managed microsoft AD do we need a jump machine to do that?The most common solution is to join a Windows computer to the domain and use the RSAT tools as you would your on premise domain.  Or if you have a Trust setup you could even manage both domains from one computer.CommentShareJoeD_AWSanswered 2 years ago0You mean to say its another domain from my existing on-prem?By the way is autojoin a feature only for AWS managed AD? If I have an EC2 instance with AD role inside can it be able to use autojoin to domain feature?CommentSharejanusbarinananswered 2 years ago0So in an AWS managed MS AD we do not have the enterprise or domain admin rights? Can we even create a user that will be a member of domain admin?CommentSharejanusbarinananswered 2 years ago"


Question: Laravel installation error: "No application encryption key has been specified."

"Hi, I have deployed a laravel application to elestic beanstalk here:http://tomrainey.us-east-2.elasticbeanstalk.com/but I get the following error message:"Your app key is missingGenerate your application encryption key using php artisan key:generate."I'm totally new to this, so please be specific. Is this something I have to edit on my local machine or can I do itdirectly on my AWS installation?Thanks.FollowComment"

Answer: 0Execute the following command in Elastic Beanstalk EC2 to create the key.sudo php artisan key:generatesudo php artisan cache:clearThe command will set APP_KEY to .env in the Laravel directory.CommentShareEXPERTRiku_Kobayashianswered 12 days ago


Question: How to run JVM based HFT application on Graviton 3 CPUs

"Thinking of creating a High Frequency Transaction Trading system to test my Algo Trading Strategies. Infrastructure will be AWS EC2 Graviton 3 instance (C7g) + AWS Linux2 + AWS Corretto JVM runtime OR OpenJDK GraalVM distribution. Why Graalvm because its Polyglot and reduces the contextual switch (Marshaling/Un-Marshaling  ) between Data Structures of different programing languages. AWS EC2 Graviton 3 coming with more than 100 virtual cores and more than 100 MB of L1 + L2 + L3 cache. Pre-compiled native ARM CPU instructions will be saved in the CodeCache. Data Analytics will be done by Apache Spark 3 and code will JIT aware (Mostly in Scala and R).   Data will be populated from SSDs and processed in RAM.QuestionsWill Amazon Corretto or GraalVM capable of generating executable native  instructions and interpreting towards  ARM based Graviton CPU.Amazon Corretto is a flavor of OpenJDK. Does the Project GraalVM already merged with Amazon Corretto JVM. Can I replace Java on Truffle - Mete Circular JIT as C2 compiler of Amazon Corretto JVM.Where I can refer guides or whitepapers related to Amazon Corretto supporting OpenJDK JEPs and projects.Which super super quick programing language as a choice to write my Algo trading business logic. Expecting a nanosecond latency from the time of a signal enters the Ethernet port of a microprocessor and returns back the result.Better Suggestions and questions are always appreciated.FollowCommentNati_C a year agoPlease note that Graviton3 has 64 cores, with no hyper-threading (i.e. not "100 vCPUs")To minimize network latency, I'd recommend looking at a kernel bypassing frameworks (DPDK for ENA, or libfabric for EFA), and it's integration with the language of your choosingI'd recommend reaching out to the Corretto team at: https://github.com/correttoShare"


Question: Autoscaling windows ec2 and autojoin to domain

"Hi,Is it possible to have an autoscaling for windows ec2 instances and auto join to domain at the same time?Thanks!FollowComment"

Answer: "0Accepted AnswerHello There,Thank you for contacting AWS.Yes, EC2 instances in auto scaling can be joined to domain automatically. Please refer the below mentioned articles for more detailshttps://aws.amazon.com/blogs/security/how-to-configure-your-ec2-instances-to-automatically-join-a-microsoft-active-directory-domain/ThanksCommentShareRobin-Panswered 2 years agogarshasp005 a year agoWhat if the Active Directory is self-hosted on the EC2 instance. Then how to automate this process?Share"


Question: I'm having an issue with a high load of 502 from an ELB

"So I have constant build issues with my frontend deployment, and it happens to be that is throwing, in some random manner, 502 errors from my ELB.I can't see any kind of error happening in my servers, so I guess it has to do with the ELB service directly.Do you have any idea what can I do next?FollowComment"

Answer: "0Hi ,There could be many reasons for 502 error.Please go through this troubleshooting page : https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-troubleshooting.html#http-502-issuesIf already done, please let us know. We can perform deep-dive on your issue.CommentShareUchiha_Madaraanswered a year ago0Hi thank you for your response, I've checked that list and nothing I see there applies to my issue.I'm getting random 502 errors, and right now for example, they are very difficult to handle.CommentSharefreynoldsanswered a year ago"


Question: Play notification sound for every new message

Can we play a notification sound for every new message both at client and agent side in AWS Chatbot. Do we have some inbuilt capability in Amazon CCPFollowComment


Question: Redshift external schema

"Redshift external schema won't show tables from AWS Glue.using AWS crawler tables names are pulled from S3 bucket, tables are listed in Glue - Data Catalog tables but when external schema is created using Glue Database (which is created during above process), it doesn't have any table.Below option is enabled, network traffic passes through VPC. If network traffic passes through VPC do we need to enable and VPC Redshift endpoint or glue endpoint.Enhanced VPC routing -Enabling this option forces network traffic between your cluster and data repositories through a VPC, instead of the internet.FollowCommentrePost-User-2969389 a year agohow are you managing permissions, do you use Lake formation. I dont see a reason why the table is not visible unless there is no proper ACLs setupShare"

Answer: "1I assume you intend to use Redshift Spectrum and the S3 bucket is in the same region as your cluster. If you enable enhanced VPC routing, the subnets where your Redshift cluster can run are private, and there is not a route from them to a NAT Gateway, you'll need VPC endpoints for Glue and S3. You also need to consider the egress rules on the security group and the permissions of the IAM role associated to your cluster.CommentShareEXPERTTasioanswered a year ago"


Question: What exactly happenned after deploying in RoboMaker

"Hi,I follow the develop guide of RoboMaker and I have done Deploy Robot Application https://docs.aws.amazon.com/robomaker/latest/dg/gs-deploy.html. I install greengrass in my rasberry pi3 and my RoboMaker Dashboard shows Succeed on the status of deployments. I can't tell what really happened after deployment.I am new to greengrass and IoT core. Here are my question:Do RoboMaker deploy a ros system (hello-world-robot.armhf.tar.gz) through greengrass after deployment?Is there a ros node (hello_world_robot) runing inside greengrass after deployment?Thank you so much for any explanation or suggestion.FollowComment"

Answer: "0Hi there,When you deploy to your physical robot using AWS RoboMaker the following things happen:Greengrass executes an AWS RoboMaker lambda function on the device to install and start the new robot application.The lambda function downloads and uncompresses the robot application bundle from S3.The lambda function starts ros using the launch file and package specified by the AWS RoboMaker robot application resource.The robot application is now running on your device. If you can ssh into your device you can use the ros command line tools to inspect the topics, messages, and services which are running.Please let us know if you have any additional questions.CommentShareAWS-User-1271166answered 4 years ago0Hi Andrew,Thank you so much for your answer.  Turn out, I also need to install ROS into my local device.I thought Greengrass will deploy an isolated ROS into my local device.After installing ROS, I can find the robot application (by rosnode, rostopic).Here are other questions I have:Even my local device don't have ROS, the deployment still show succeed. Is the logic correctly?Also, Greengrass runs very quietly, is there any log I can check about the deploy detail?How could l kill the ROS master that is started by Greengrass?I could use rosnode kill to stop other nodes (like /rotate), but my system re-spawns new ROS master after I use rosnode kill or kill pid.Thank you so much for your help!Best regards,RobinCommentShareNexCOBOTanswered 4 years ago0Hi Robin,We bundle ROS with your robot application and start it on the robot as part of the Greegrass deployment. When you SSH into the robot you’ll need to source the ROS install location in order to use the CLI tools.You can set up your environment to mirror how we run the bundle by sourcing each overlay. If you look in the ‘overlays.json’ file of your bundle you’ll see the overlays in the order they should be sourced. The deployment agent downloads the overlays and puts them into directories matching their sha256. You will have to figure out the order in which you should source the overlays. Once you’ve done that you can execute the following, for each overlay, in order:BUNDLE_CURRENT_PREFIX=<path to overlay directory> source <path to overlay directory>/setup.shThe overlay folders should be under: /tmp/roboMakerDeploymentPackage/Once you’ve done that your environment matches the environment under which we run the bundle, so you can execute roslaunch, rosrun, or other ROS tools.More info about overlays: https://github.com/colcon/colcon-bundle/blob/master/BUNDLE_FORMAT.md#overlay-archivesWe are adding documentation and tooling to make this setup easier.The Greengrass logs are available in CloudWatch. You can also configure file system logging and the log files will be stored under greengrass-root/ggc/var/log on the core device.https://docs.aws.amazon.com/greengrass/latest/developerguide/greengrass-logs-overview.htmlWe don’t expect the ROS master to re-spawn after being killed. Can you elaborate more on the steps you are taking to kill it?Thank you,AndrewCommentShareAWS-User-1271166answered 4 years ago"


Question: Modeling DynamoDB

"Dear Team,We are building an application which has documents uploaded to S3 and its metadata ingested to Dynamo DB.  We have a requirement to search on the Metadata from UI.  There will be approx 7 attributes on which the User can search the metadata and all of them will be optional.  The requirement is only to search on metadata.  No full text search requirement on the document.Wanted to check if we would be able to use Dynamo DB for this purpose or we need to use search tools like OpenSearch ?Please let me know if more information is required, i can update the Description.Regards,Dhaval MehtaFollowCommentAWS-User-9438600 a year agoCan you provide more details on what kind of these attributes are and what is probability of them expanding in future. Opensearch is great but you will still have to configure/reconfigure the clusters among other things of maintenance.Share"

Answer: "0Accepted AnswerIf you need to search using more than hmm 2 arguments then you should go for OpenSearch. With small amount of arguments you can try to make more indexes, but with 7 it is no go because you will need to have index per configuration of parameters and it will be expensive to store and search.CommentShareMGanswered a year ago"


Question: Need Information About DMS Instance Low Memory Thresholds

"In order to create effective high memory CloudWatch Alarms for DMS Instances, an effort has been made to observe the behaviour when a DMS Replication Tasks is "Stopped" with and Out of Memory Error.Observed Behavior.Set DMS Instance machine class to something small.Start many "Full Load and Ongoing" Migration Tasks simultaneously.Eventually, one or more tasks will transition to a stopped state with the "Last Error" being "Out of Memory."This indicates that there must be a high memory threshold that the instances use to monitor memory. Replication Tasks seem to be stopped with an "Out of Memory" error, while the instance continues and is available. This makes sense in an effort to not crash the dms instance, some tasks are taking out and placed in a "stopped" state.I have two questions regarding the observance (please correct if observation is not correct).Is there any information about which task(s) are chosen as the victim when a memory threshold is reached?In order to create a low available memory alert, I would like to stay below the dms instances low memory threshold, is there any information relating to this threshold?FollowComment"

Answer: "1Accepted AnswerHello,You may review blog which talks about how to setup cloudwatch alarm including low memory.https://aws.amazon.com/blogs/database/setting-up-amazon-cloudwatch-alarms-for-aws-dms-resources-using-the-aws-cli/About the first question, all task not necessary will go into low memory at the same time, I do not think there is any mechanism or method to choose which task to stop, whenever any task reached to point where DMS replication instance can't give more memory it will stop/crash.Hope this will be helpfiul.ThanksCommentShareJitenderanswered 2 months agoRoss Bush 2 months agoThanks for the replyShare"


Question: <myorg>.awsapps.com/start stopped loading yesterday

"Beginning sometime on Sunday, March 19, 2023, my org's IAM Identity Center login page has not loaded.  It just shows a blank screen. I don't recall making any changes that would break single sign on.  However, I don't see any reporting of an outage.Is anyone else experiencing something similar?Is there any remediation I can use to get the org functional again?Update:Sometimes, but not always, I see the message: "We were unable to load the list of your applications.Try again."It came back online briefly, but now is failing again.FollowComment"

Answer: "0I am not aware of any events that may have impacted the IAM Identity Center page and I have confirmed that I am able to access my org's page.  Have you tested it with another browser or device?CommentShareconor_canswered 2 months agorePost-User-2410793 2 months agoI have tried with several other browsers.  It came back online briefly this evening, but now seems to be down again.  Sometimes, but not always, I see the message "We were unable to load the list of your applications.Try again"Share0It looks like my Eero home router was blocklisting the URL due to parental controls.  When I switch to tethering from my cell phone provider the problem goes away.CommentSharerePost-User-2410793answered 2 months ago"


Question: Elastic Graphics Quota exceeded

"Hello,I am receiving this error when using terraform to provision a t3.medium with eg1.medium Elastic Graphic:Error launching source instance: ElasticGpuLimitExceeded: Your quota allows for 0 more Elastic Graphics accelerators in your account. You requested at least 1. I can't find limits in the limit section nor any quotas. I have terminated all instances that use Elastic Graphics and still cannot create any new ones. Please help.FollowComment"

Answer: 0You can request an increase to your Elastic Graphics accelerators quota through the Service Quotas dashboard at https://console.aws.amazon.com/servicequotas/homeYou will be looking for the quota at Service Quotas > AWS services > Amazon Elastic Compute Cloud (Amazon EC2) > Number of Elastic Graphics acceleratorsCommentSharebenduttanswered a year ago0The issue is relating to the fact that the usage is appearing as not available and isn't actually the quota limit. I am working with support on this.CommentShareAWS-User-8781036answered a year ago


Question: Route table not routing to Site-to-Site VPN's Inside Ipv4 CIDR

"I have a VPC with private subnet (NAT) that has a routing table wich redirects traffic of a given IP range(Data center) to a vgw(virtual private gateway), then I have this site-to-site vpn configured with this vgw and a customer gateway, on its static routes I also had the IP range for the Data center. But can't seem to get my ec2 running ubuntu to traceroute to the corresponding VPN's Inside Ipv4 CIDR when trying to reach Data center's range.What could be wrong? VPN tunnels are up so even if I couldn't reach the Data Center, it should at least hop on the VPN IP address.Thanks in advance for any ideas!FollowComment"

Answer: "0This issue just started for us today with our Site-to-Site VPNs. We can still ping the outside IP addresses of the two tunnels. I went through every setting of our Firewall and the AMS VPC configurations to make sure nothing had changed. I suspect this will turn out to be an issue, which is more global in nature (at least for the AWS zone we are located in). I'm glad to see this post was first on the forum list when I came to check if others were experiencing this today.CommentShareAWS-User-7294684answered a year agoMatt a year agoWell, we have several other vpns on this and two other regions working consistently (or they seem to, which is frightening :)), but just now we've realized we've been getting many of these "Your VPN Connection XXXXXXXXXXXX in the XXXXX Region had a momentary lapse of redundancy as one of two tunnel endpoints was replaced" so, who knows... For now we gave up on the vpn we were setting up. We´ll look at other options to encrypt the connection. Thanks!!Share0Well, we have several other vpns on this and two other regions working consistently (or they seem to, which is frightening :)), but just now we've realized we've been getting many of these "Your VPN Connection XXXXXXXXXXXX in the XXXXX Region had a momentary lapse of redundancy as one of two tunnel endpoints was replaced" so, who knows... For now we gave up on the vpn we were setting up. We´ll look at other options to encrypt the connection. Thanks!!CommentShareMattanswered a year ago0When working with static routed VPN there are few best practices you must keep in mind:AWS only uses one tunnel inside a AWS VPN as active and the other tunnel is expected to be standby. This tunnel is chosen randomly.Your customer gateway device must maintain symmetric traffic flow. If you do not have any monitoring mechanism to do so keep 1 tunnel shut and bring the other one up only when primary tunnel fails.Use dynamic routed VPN, this will leverage BGP to insure you have symmetric path.Please open a support case to investigate more. You can refer some documentation here:https://docs.aws.amazon.com/vpn/latest/s2svpn/VPNRoutingTypes.html#routing-vpn-tunnel-updateshttps://aws.amazon.com/premiumsupport/knowledge-center/vpn-configure-tunnel-preference/CommentShareAzeem Ayazanswered a year ago"


Question: Cannot connect from Glue to RDS postgres

"Hello, i'm trying to connect from Glue to postgres,when testing the connection i'm getting the following error"Failed to test connection  due to FAILED status" ,cloudwatch showing the following error"ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.ERROR StatusLogger Unrecognized format specifier [level]ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.ERROR StatusLogger Unrecognized format specifier [logger]ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.ERROR StatusLogger Unrecognized format specifier [msg]ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.ERROR StatusLogger Unrecognized format specifier [n]ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern. ERROR StatusLogger Unrecognized format specifier [level] ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern. ERROR StatusLogger Unrecognized format specifier [logger] ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern. ERROR StatusLogger Unrecognized format specifier [msg] ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern. ERROR StatusLogger Unrecognized format specifier [n] ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern."*** i can connect to the database from my laptop, also the inbound &outbound have rules referencing the SG.FollowCommentrePost-User-5990915 a month agoI'm having the exact same issue. Self referencing inbound rule for all TCP ports allowed and all outbound traffic allowed  for the security group. Have tried to rule out pretty much anything that I can think of.Very frustrating as the "Failed to test connection connection-name-here due to FAILED status" error message and the cryptic logs posted above when trying to test the connection aren't really of any help.Share"

Answer: "0Hi,I have the same issue here. I several glue connections from RDS Postgres in the past, I have never faced any issue. Today, I tried to create a connection, I get the same error message.CommentShareMelfanianswered a month ago0I got the same error with RDS PostgreSQL engine version 14.6.But it successfully connected when I changed RDS PostgreSQL engine version to 13.10.CommentSharePatisanswered 24 days ago0In CloudWatch go to Log groups and check if you have a log group with a name like /aws-glue/testconnection/output/<connectionName>that might give some more information.(The link in testconnection sends you to /aws-glue/testconnection/error/<connectionName>)We had the same error in the error logs, however we did have information in the output logs of the connection tests.For us that gave a slightly clearer error message that we solved by changing our Database authentication method from "Password and IAM database authentication" to "Password authentication"CommentShareM_Mooiweeranswered 20 days ago0Seeing this issue as well.Looking into the output logs like @M_Mooiweer recommended provided this additional insight.Check that your connection definition references your JDBC database with correct URL syntax, username, and password. The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driverSome other posts indicate that this may be related to scram vs md5 password encryption between Postgresql 13 and 14 so I explored this a bit further.The JDBC Driver version 42.2.0+ should support scram. Per this doc, Glue 3.0-and-up support compatible drivers, but I'm not sure which version of Glue, if any, would be used when testing a connection.Uploading a .jar of a known-compatible driver (works locally) to S3 and specifying it in the connection also does not appear to change the outcome.Other folks have mentioned downgrading from postgresql 14 to 13, but I'm unsure if that's a sustainable solution in all cases.CommentSharerePost-User-6672643answered 17 days ago0I was able to use the correct driver.Glue seems to use a postgresql driver incompatible with Postgresql14 by defaultDownload the latest version from  https://jdbc.postgresql.org/download/  and make the .jar available on s3The custom driver .jar uploaded to a s3 bucket will only be used if the JDBC driver class name is overriddenSo make sure the Jdbc Driver class name is not empty , but it contains the default value:  org.postgresql.DriverTesting connections  is not (yet) supported, this will throw the following exception:com.amazonaws.glue.jobexecutor.commands.exception.CommandExecutorException: com.amazonaws.services.glue.exceptions.InvalidInputException: Testing connections with custom drivers is not currently supported.Data crawling works for me now. I assume other Glue jobs will work accordingly.CommentShareJeroenanswered 4 days ago"


Question: Chime meeting events notifications

"Hi, I'm working on test project to get meeting event notification from Chime. Is there any way to achieve this purpose using Java  straightforwardly?  And is there any sample projects of adding meeting event?FollowCommentHemant Chugh 2 months agofollow below linkhttps://docs.aws.amazon.com/chime-sdk/latest/dg/using-events.htmlShareChengHe 2 months agoI had read that documents but chime sdk doesn't support Java. Is there any way using Java to approach it?Share"