Question: [AWS Greengrass] Core device fails to connect to AWS IoT Core over Charles proxy.

"I am trying to connect my Core device (Raspberry Pi 3 Model B) over Charles Proxy.I set up the Charles proxy on my laptop.I updated the Nucleus configuration by deployment.{  "networkProxy": {    "proxy": {      "url": "http://192.168.0.38:8888"    }  }Added the Charles Root Certificate to the core device's Amazon root CA certificate file as it is described in Enable the core device to trust an HTTPS proxy.Actual result: The device cannot connect to AWS IoT Core.2023-04-04T12:44:19.588Z [INFO] (pool-2-thread-12) com.aws.greengrass.mqttclient.AwsIotMqttClient: Connecting to AWS IoT Core. {clientId=rpi_3_B}2023-04-04T12:44:22.102Z [ERROR] (Thread-4) com.aws.greengrass.mqttclient.AwsIotMqttClient: Unable to connect to AWS IoT Core. {clientId=rpi_3_B}software.amazon.awssdk.crt.mqtt.MqttException: socket is closed.        at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:140)2023-04-04T12:44:22.110Z [ERROR] (pool-2-thread-12) com.aws.greengrass.mqttclient.MqttClient: Error when publishing from spooler. {}java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: socket is closed.        at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)        at com.aws.greengrass.mqttclient.MqttClient.runSpooler(MqttClient.java:659)        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)        at java.base/java.lang.Thread.run(Thread.java:829)Caused by: software.amazon.awssdk.crt.mqtt.MqttException: socket is closed.        at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:140)Unfortunately, I cannot attach .csv or .chls files.Any idea why AWS IoT Core is terminating the connection?FollowComment"

Answer: "1Accepted AnswerAWS IoT Greengrass uses TLS mutual authentication to connect to AWS IoT Core: https://docs.aws.amazon.com/greengrass/v2/developerguide/device-auth.html#:~:text=Communication%20between%20core%20devices%20and%20client%20devices%20and%20between%20devices%20and%20AWS%20IoT%20Core%20or%20AWS%20IoT%20Greengrass%20must%20be%20authenticated.%20This%20mutual%20authentication%20is%20based%20on%20registered%20X.509%20device%20certificates%20and%20cryptographic%20keys..TLS mutual auth prevents introspecting proxies from working by design, for security.You must use a transparent proxy which does not intercept and modify the TLS connection.Additionally, you have set the proxy for "http" and not "https". Use "https" if the connection to the proxy itself should use TLS.Cheers,MichaelCommentShareEXPERTMichaelDombrowski-AWSanswered 2 months agoVadzim 2 months agoI understand. Thank you for such a quick response!Share"


Question: Access Documents hosted behind the VPN via google viewer

"We have a use-case. Our website is hosted on AWS and have rules at cloud front that states requests with specific IP range should be pass through CDN and documents will serve.Now, there are certain documents that user view under google docs viewer say, https://docs.google.com/viewer?url=https://{{siteUrl}}/{{documentPath}} . As documents are hosted on same site which is behind the VPN, user not able to view the documents.Any solution on how we can enable the view of documents, that will be requested by docs.google.com or how how we can allow docs.google.com in aws cloudfront/alb.ThanksFollowCommentBrettski-AWS EXPERTa year agoIt's not clear from your question where the VPN comes into the picture. Are the users connection via VPN? If so, how are they connecting to CLoudFront because it is accessible via public IP addresses. If so, why not enable split-tunnel for Google Docs?Share"

Answer: "0Hello,Is this AWS VPN/CVPN or a 3rd party VPN on an EC2 that is being used ? Where is the VPN located ?Check this link out which looks like similar to your use-case and was recently launched (Whats new).Link- https://aws.amazon.com/about-aws/whats-new/2022/02/amazon-cloudfront-managed-prefix-list/CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 10 months ago"


Question: NoClassDefFoundError when trying to use Hive/JDBC from Zeppelin

"Using EMR 6.1.0 -> Hadoop 3.2.1, Hive 3.1.2, Spark 3.0.0 and Zeppelin 0.9.0, I installed the jdbc interpreter using:sudo bash bin/install-interpreter.sh -n jdbc --artifact org.apache.zeppelin:zeppelin-jdbc:0.9.0-preview2(The --artifact flag was required, otherwise the interpreter would not install).Using the Zeppelin web interface (accessed as localhost:8890 using ssh forwarding), the hive/jdbc interpreter and a simple SELECT statement throws an error (copied below).I'm not sure what is required to get the setup to recognize PropertiesUtil.  Any steps to get the jdbc hive interpreter working in Zeppelin would be appreciated.  I have posted to the Zeppelin forums as well, but this may be specific to EMR.java.lang.NoClassDefFoundError: org/apache/zeppelin/util/PropertiesUtilat org.apache.zeppelin.jdbc.JDBCInterpreter.createConnectionPool(JDBCInterpreter.java:469)at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:485)at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:505)at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:706)at org.apache.zeppelin.jdbc.JDBCInterpreter.internalInterpret(JDBCInterpreter.java:881)at org.apache.zeppelin.interpreter.AbstractInterpreter.interpret(AbstractInterpreter.java:47)at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:110)at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:684)at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:577)at org.apache.zeppelin.scheduler.Job.run(Job.java:172)at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:130)at org.apache.zeppelin.scheduler.ParallelScheduler.lambda$runJobInScheduler$0(ParallelScheduler.java:39)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.ClassNotFoundException: org.apache.zeppelin.util.PropertiesUtilat java.net.URLClassLoader.findClass(URLClassLoader.java:382)at java.lang.ClassLoader.loadClass(ClassLoader.java:418)at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)at java.lang.ClassLoader.loadClass(ClassLoader.java:351)... 15 moreEdited by: AADC on Sep 15, 2020 9:39 PMFollowComment"

Answer: 0--artifact org.apache.zeppelin:zeppelin-jdbc:0.9.0-preview1 needed to be installed instead of preview2.CommentShareAADCanswered 3 years ago


Question: How to Enable Fleet Autoscaling in Gamelift?

"So we have  a game that we are soon going to roll out for beta testing and would like to know how to enable the autoscaling feature? Right now, Under scaling section for the fleet in Frankfurt region, I am shown that I can have only 1 server and when I enable the autoscaling feature, the desired number of servers under scaling limits turns to 0 and no player session is able to be activated throwing FleetExceptionError. So as a team planning to launch a battle royale game, for us we don't mind the max number of services we need as we expect exponential growth in players once we roll out the game, But this available servers=0 is a concern we would like to know how to address. Any help will be much appreciated.FollowComment"

Answer: "0Hi,From the screenshot posted above, you should first uncheck the "Disable all scaling policies" to enable auto-scaling.If you expect to have many sessions, you should crank up the max number of instances. GameLift cannot set the number of instances outside the min/max range, regardless of whether the number of instances is set manually or through auto-scaling: https://docs.aws.amazon.com/gamelift/latest/apireference/API_UpdateFleetCapacity.htmlHope that helps.CommentShareJuneHoP_AWSanswered 6 months agoiamarjunchandra 6 months agoHi, When I uncheck the Disable all scaling policies, I am getting Fleet Exception Error. Also how to set more than 1 instances through autoscaling when the available server instances for my our aws account in frankfurt is 1 for the specific instance type we are choosing?Share"


Question: Issues with CORS and time-shifted playback

"Hello!I've been working on getting MediaPackage and CloudFront working for HLS distribution and so far it's been great! After a slight change (having our encoder count from 0 instead of current time) I recently started to notice that I get cors errors whenever I add the "?start=1" argument onto my distribution URL for Time Shifted Playback. Has anyone else run into this issue, or perhaps is there an AWS Team Member that might have some insight? Is there a better way to add DVR functionality to a live stream than this configuration?FollowComment"

Answer: "0The CORS error is likely not the issue. I have seen some clients report a CORS error on a 404. The request is returning a 404 error because the start time is invalid. The value of the parameter needs to be the actual start time in epoch, or ISO 8601 format. Please read through the documentation for the start/end time formats:https://docs.aws.amazon.com/mediapackage/latest/ug/time-shifted.htmlRequesting "?start=1" means you're requesting content from back in 1970, which will not work. If you use a valid start time, and you have enabled time-shifted playback on the endpoint, that should take care of this.CommentShareAWS-User-8967459answered 4 years ago0Thank you for the quick reply Saurav!Interesting, we tried to setting Timecode Configuration in MediaLive to zerobased which we assumed would allow for "?start=1" as the first frame in the stream starts at 00:00:00. So going forward we should assume that when we want DVR functionality the URL is going to have to be appended with the time the stream started regardless of the timecode?CommentSharejerryfleuryanswered 4 years ago0Hi jerryfleury,jerryfleury wrote:So going forward we should assume that when we want DVR functionality the URL is going to have to be appended with the time the stream started regardless of the timecode?That is correct, the only way to address a section of content in your catchup window a proper timestamp in either epoch or ISO 8601 is required for the start (and optionally end) time.CommentShareThijsBanswered 4 years ago0Thank you for the help everyone!CommentSharejerryfleuryanswered 4 years ago"


Question: "SageMaker Studio PyTorch 1.8 kernel has no PyTorch, Numpy, or Matplotlib module"

"I'm working with SageMaker studio with the following options:kernel: PyTorch 1.8 Python 3.6 GPU optimized.instance: ml.g4dn.xlargeWhen running import torch numpy, matplotlib or PIL, I'm getting the No module named 'X' error. No matter when using pip install in a cell above, it will not be imported. Is this a problem only I am encountering with the new PyTorch 1.8 kernel? It also happens with the CPU-optimized version. However, PyTorch 1.6 kernel does not throw an error.When running conda list, I get the output without any of the previously mentioned modules.FollowCommentZmnako a year agoNot sure really why it is not installed.Share"

Answer: "0I have followed your explanation to recreate the No module named 'X'. After instance lunch, I could import numpy, matplotlib and PIL but could not import torch because torch is not initially installed.I followed these steps to install and import torch:Shut down your instance completely.Attach the kernel to your notebook againLunch your ml.g4dn.xlarge instance againpip3 install torchimport torchCommentShareZmnakoanswered a year agoSiouxDMA a year agoThanks for your answer. Why is it not initially installed, while it is the PyTorch image?ShareAlex_T EXPERTa year agoThanks for raising this @SiouxDMA - I was also able to reproduce and agree it looks unexpected, will follow up internally to see what's going on and what can be done about itShare0No module named 'X' should not be the expected behavior. Thanks for reporting this issue and sorry for the inconvenience it caused.Is your use case flexible to use other PyTorch versions prior to 1.8? If yes, please try other versions.If PT1.8 is the only choice, please try following workaround for unblocking(while service team is working on the fix).Two options:1 - executed following in a notebook cell# switch the python execution to /usr/local/bin/python in the kernel.json file. !sed 's|^ *"python",|  "/usr/local/bin/python",|g' /usr/local/share/jupyter/kernels/python3/kernel.json>/tmp/kernel.json; cp -f /tmp/kernel.json /usr/local/share/jupyter/kernels/python3/kernel.json;2 - directly execute following shell command in kernel image specific terminal(not the global terminal).sed 's|^ *"python",|  "/usr/local/bin/python",|g' /usr/local/share/jupyter/kernels/python3/kernel.json>/tmp/kernel.json; cp -f /tmp/kernel.json /usr/local/share/jupyter/kernels/python3/kernel.json;`The above command is only needed once per kernel gateway app.After above, please restart the kernel. You can verify using following command in a notebook cell. The  '/usr/local/bin/python' should be shown as python executable.import sysprint(sys.executable)CommentShareEdward Sunanswered a year ago"


Question: m6a.24xlarge instance keep rebooting every 2 to 3 hours

"Hi, I have the linux centos instance running on m6a.24xlarge, it reboots every 2 to 3 hours. If i change the instance type and run it on m6a.12xlarge it works fine, no reboot. What could be the reason of m6a.24xlarge keep rebooting? I checked the logs inside the VM, there is no indication or reason of its reboot. Let me know if somebody saw similar behavior or know how this could happen. ThanksFollowComment"

Answer: "0Hello,Did you check the status checks using CloudWatch metric? Is it failing instance/system status checks before reboot?Have you checked the performance metrics like cpu, memory, network in and network out usage of the instance. Do you observe any resource crunch?Have you checked the cloud trail logs. Are you able to view the “Rebootinstances” API call to check if the reboot is triggered from any scheduler?Please refer the below link which addresses various causes that could trigger unexpected reboot:https://repost.aws/knowledge-center/ec2-instance-automatic-reboot-causeCommentShareSree_Canswered 14 days ago"


Question: Add Launch Script after Lightsail Creation?

"My lightsail instance needs a launch script, but I can only find documentation on setting those up when first creating the server.Where does the launch script live on the server? I'd love to just ssh in, edit it, and be done.FollowComment"

Answer: "0Accepted AnswerI did a bit of poking around ... looks like you can just put your startup script in /etc/rc.local and it will run.Make sure you make /etc/rc.local executable (chmod a+x /etc/rc.local).CommentShareDavid Ganswered 2 years ago0What blueprint are you using?There are various ways to accomplish a launch script, but it depends on the blueprint.CommentShareDavid Ganswered 2 years ago0I'm using MEAN.CommentSharebaudotanswered 2 years ago0Baudot,Thanks for using Lightsail!Here is a doc about launch scripts: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/lightsail-how-to-configure-server-additional-data-shell-scriptThe doc does not explicitly state this but it is important to note that the launch script is only executed on initial launch of the instance, not on every launch. For that reason there is no way to configure the launch script after the instance is initially created.If you require software to run on every launch of the instance, you should use the os-dependent initialization software such as 'initd' on linux and startup programs on Windows.I hope this helps!DonleyCommentSharedonleyatawsanswered 2 years ago0David - thanks, that worked.I had to do few more debugging steps to get my script running properly.In the end the script ended up looking like:#!/bin/sh(cd /full/directory/path/to/my/app/full/path/to/init_command 1>/var/log/my_app_name/my_app_name.log 2>&1 &)...and that did it.If anyone else runs into a similar hurdle, hopefully this helps.CommentSharebaudotanswered 2 years ago"


Question: Ubuntu instance lost connection and crash

"Hi,I am a beginner of AWS service,I've deployed some service on my instance, but I found that these service down sometimes and SSH can't connect to it.So I restart the instance and check the syslog. Here is what I found in log.   "eth0: Could not set DHCPv4 address: Connection timed out"Can anyone help me to find out what reason caused this issue ?BTW, here is my instance ID if it's needed.  i-05c242d2ca45d58efFollowComment"

Answer: 0This question was asked on AWS forums. Please refer to the webpage: https://forums.aws.amazon.com/message.jspa?messageID=874873  See if it helps.CommentShareSandeepVudataanswered a year ago


Question: Contradictory information on CPU-usage on EC2 t2.micro

"Hello community.I am using EC2, to be more specific a t2.micro virtual machine with Amazon Linux as OS.I noticed some performance issues so I checked RAM- and CPU-usage. In case of RAM the AWS instance monitoring wouldn't give me any information, but using the Linux command "top" the VM (virtual machine) showed very low RAM-usage, so it should be fine. The issue with the CPU turned out to be more complicated, since the AWS instance monitoring showed me usage of up to 12.5% (= 1/8), whereas the VM itself reported 100%, again using the Linux command "top". Further the VM reports having 1 CPU without specifying wether it is a processor, a core or a vCPU.It would be very important to know which value is the correct one for coming up with a solution for the performance issues. Does anyone have an idea?It would also be interesting to know how this gap came to be. I was thinking about a quad core processor having 8 vCPUs and the AWS monitor showing me the whole processor (which I cannot use), while the VM takes the vCPU as maximum, but that would be kind of silly.Thanks for your ideas.FollowComment"

Answer: "2Part of what you're seeing is because your t2 instance is burstable, so it accumulates CPU credits for bursts of very high performance.  100% CPU in CloudWatch equates to max burst CPU. So if you've exhausted your credits then despite your VM running at 100% of its current capacity, CloudWatch will show a much lower percentage.CommentShareEXPERTskinsmananswered 8 months ago"


Question: RuntimeError: CUDA runtime implicit initialization on GPU:0 failed. Status: device kernel image is invalid

"The environment I'm using is:aws p4dn.24xlarge instance (NVIDIA Ampere A100 GPU )cuda 10.1tensorflow 2.3.0python 3.6.9I get an error when I run the following. What is the reason?tensorflow.test.is_gpu_available()2022-01-23 07:56:08.088849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:10:1c.0 name: A100-SXM4-40GB computeCapability: 8.0 coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 39.59GiB deviceMemoryBandwidth: 1.41TiB/s 2022-01-23 07:56:08.088936: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2022-01-23 07:56:08.089013: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10 2022-01-23 07:56:08.089030: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10 2022-01-23 07:56:08.089046: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10 2022-01-23 07:56:08.089059: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10 2022-01-23 07:56:08.089074: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10 2022-01-23 07:56:08.089090: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7 2022-01-23 07:56:08.092700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0 Traceback (most recent call last): File "", line 1, in File "/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func return func(*args, **kwargs) File "/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/framework/test_util.py", line 1563, in is_gpu_available for local_device in device_lib.list_local_devices(): File "/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/client/device_lib.py", line 43, in list_local_devices _convert(s) for s in _pywrap_device_lib.list_devices(serialized_config) RuntimeError: CUDA runtime implicit initialization on GPU:0 failed. Status: device kernel image is invalidFollowComment"

Answer: "1AWS team has released TF 2.3.2 DLCs with CUDA 11.0 specifically to target the p4d.24xlarge instance type, because of the compatibility issues with drivers and CUDA versions required to work with p4d instances.I assume that since this hasn’t come in through the SageMaker channels, this isn’t about a SageMaker job that you are running.I recommend that you should try to use this image (or plug in any region they need into the image URI)763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.3.2-gpu-py37-cu110-ubuntu18.04to use the p4d.24xlarge instance type, which may help to avoid CUDA related issues.It is possible that the CUDA issue you see is unrelated to this change to CUDA 11.0, but I strongly suspect this is caused by an incompatibility between the CUDA version on the image and the GPU architecture used by p4d instances.Cheers !!CommentShareSUPPORT ENGINEERStewyn_Canswered a year ago"


Question: XRay and AwsParamStorePropertySource

"I am trying to enable X-Ray on our application. At first we were only getting information for Dynamo, but then I realized I needed to include the instrumentors for both v1 and v2 versions of the dependencies in maven. Now my problem is that I am getting tons of errors because segments aren't created for initialization tasks.10:46:35.602 [main] ERROR o.s.c.a.p.AwsParamStorePropertySourceLocator - Fail fast is set and there was an error reading configuration from AWS Parameter Store:Failed to begin subsegment named 'AWSSimpleSystemsManagement': segment cannot be found.10:46:35.645 [main] ERROR o.s.boot.SpringApplication - Application run failedI found an article https://docs.aws.amazon.com/xray/latest/devguide/scorekeep-startup.html that seemed like it would give me what I wanted. The problem is that in that article the person has access to the code that is calling into the AWS SDK. In my case during start up we are calling to get a parameter store. This all happens outside of my direct control so I cannot easily wrap the call in a begin and endSegment call.I am using Java 11, Springboot, and Maven.I could just ignore the errors or change the logging level of that particular class to mask the issue, but that feels hacky. Is there another way to solve this? Any examples anywhere of someone solving it?FollowComment"

Answer: "0Hi Bryce,If there aren’t any other issues apart from segments not being created for initialization tasks, it is fine to ignore the errors or change the logging level of that class. It is expected that segments aren't created for such initialization tasks.CommentShareJon_Lanswered 10 months agoBryce Griner 10 months agoI did think about that. The problem was that apart from initialization tasks, there were also some off-thread issues where segments weren't getting created. I was getting spammed pretty heavily in the logs from those. I ended up opting to leave out the version 1 dependencies and just moved whatever I could to v2 APIs, which are instrumented properly. Maybe if I get some time to fix the off-thread issues, I might add them back in again.Share"


Question: "Is it possible to speed up deletion of a VPC attached lambda, within a CloudFormation stack?"

"When deploying a stack (with CDK) containing a lambda attached to a VPC, then destroying the stack, it takes around 45 minutes to delete a lambda attached to a VPC.I believe this is because the elastic network interface associated with the lambda takes this long to remove, but as I am only testing deployment the eni should not be in use.Is this a known issue and can anything be done to improve the experience around this?FollowComment"

Answer: "0I haven't timed this lately but I know prior to the improved Lambda VPC networking using AWS Hyperplane rolled out 2019/2020, it could take several hours for an ENI to be cleaned up and deleted.  Supposedly it's improved now ... I guess 45 mins is an improvement! :)One workaround I heard of with CloudFormation was using a Custom Resource to force deletion of the ENI.  Something to watch out for with this is that ENIs are shared across Lambda execution environments with the same security group & subnet combination, so best to use a unique security group per Function to avoid deleting an ENI in use by someone else.CommentShareEXPERTskinsmananswered 5 months agoDrew-PUSH 5 months agoThe strange thing is I am creating the security group as part of the same stack and the ENI is only used for the single lambda, so I don't see why the ENI needs to be kept for so long after deletion.I think I had seen something similar with the custom resource, so I might give this a try. Though deleting the ENI early via the AWS Console doesn't seem to be possible, even after CloudFormation has deleted the lambda...Share"


Question: AWS BACKUP does not backup RDS instance

"Hi,I am using AWS Backup to create a snapshot of my Aurora Postgres cluster + instance. In Resource assignments I added arn of cluster and arn of the instance. But backup is only doing a backup of a cluster? So I have one cluster restored but with 0 instances. Why AWS Backup does not backup my RDS instance?ThanksFollowComment"

Answer: "1Accepted AnswerIt is expected behavior when Aurora cluster is restored from AWS Backup console. AWS Backup restores your Aurora cluster; it does not create or attach an Amazon RDS instance to your cluster. This behavior is similar to the restore-db-cluster-from-snapshot command [1]. As, this action only restores the DB cluster, not the DB instances for that DB cluster. You must invoke the CreateDBInstance action to create DB instances for the restored DB cluster, specifying the identifier of the restored DB cluster in DBClusterIdentifier. To create the DB instance you have to run the following command as discussed in Step 14 of our documentation in [2].For Linux, macOS, or Unix:aws rds create-db-instance --db-instance-identifier sample-instance \          --db-cluster-identifier sample-cluster --engine aurora-mysql --db-instance-class db.r4.largeFurther, you can restore the Aurora Cluster from RDS console, which provides the option to create the DB instance when restoring the Cluster. The backups taken via AWS Backup service will also be listed in the RDS console, and you can select the snapshot and restore with the required options.There is a feature request open for this already,However, at this point, I am unable to provide an ETA if or when the product team incorporates this feature in their pipeline going forward. However, please refer to these links [3,4] for our latest updates and feature releases.References:[1]restore-db-cluster-from-snapshot -https://docs.aws.amazon.com/cli/latest/reference/rds/restore-db-cluster-from-snapshot.html[2] Use the AWS Backup Console to restore Aurora recovery pointshttps://docs.aws.amazon.com/aws-backup/latest/devguide/restoring-aur.html#aur-restore-console[3] Featured Announcements -https://aws.amazon.com/new/[4] AWS News Blog -https://aws.amazon.com/blogs/aws/tag/announcements/CommentShareSSanswered 3 months agomyronix88 3 months agoYes, just found out. Thank you. It's not convenient, but good that ticket is opened for that.Share0Im not sure but i feel like you need to configure AWS backup to take a snapshot of each individual instanceCommentShareAWS_Guyanswered 3 months agomyronix88 3 months agoNo, it just doesn't see an instance.Share"


Question: Amazon Connect and Facebook Messenger integration

"I follow this https://aws.amazon.com/fr/blogs/contact-center/adding-digital-messaging-channels-to-your-amazon-connect-contact-center/ and this https://github.com/amazon-connect/amazon-connect-message-streaming-examples to be able to add a canal to my Amazon Connect instance.There have been a couple of times that the message entered into a Facebook page (FB Messenger) has triggered a call to AWS API Gateway. The logs show it to me. But, without my understanding why, 99.99% of the time, no call is made between Facebook and our API (AWS).So I need help to troubleshoot my problem.Thanks for helping me if you have already done this Connect and FB integration.FollowComment"

Answer: "0Accepted AnswerHello. The problem is solved. The bug comes from my misunderstanding of how FB applications work and a lack in the documentation of the blog and the Github page. After having done all the procedure, it is necessary to create roles for test users in the FB application. A text mentions it on the app page, but I hadn't noticed it. Thank you again for your help.CommentShareLuc Paquetanswered a year agofaddi a year agoGreat, glad you could fix it.Share0The message streaming samples create three Lambdas for you. You can find them when browsing through the Lambda Service.When the names are by standard they should start with "ChatMessageStreamingExamp-...." and are divided in three functions:healthCheckFunctionoutboundMessageFunctioninboundMessageFunctionIf you do not receive he facebook messages in Amazon Connect you should check the logs of the inboundMessageFunction, if you cannot reply, check the outboundMessageFunction.In case you do not see anything or nothing good enough to go further, check the Lambda Functions Configuration.Under Environment variables you do see three settings:CONTACT_TABLE - this is the DynamoDB which keeps a record of the contactsDEBUD_LOG - this sets the logging behaviour. By default this is false and you can change it to "true" to get more log outputs. Hence you fixed your issue, make sure to set it back to false to ensure you do not create not required logs.FB_SECRET - this keeps the Secrets Manager ARN where the Function get the Facebook secrets to communicate with the Facebook APIs. Make sure that this is correct by checking the Secret Manager ARN.Things you may want to check:ensure the correct secrets are stored in the Secrets Managerensure the correct Secrets Manager ARN is in all the Lambda functionsensure your Lambda functions do have access to that secretsCommentSharefaddianswered a year ago0First, thank you for your response.Here is more information.The problem is that nothing happens on the AWS side when someone enters a message in the Facebook page (via FB Messenger). I can confirm that as soon as we use tools like Postman, Powerautomate or even the test tool on the FB Dev page, we see the API call arrive and then the Lambda Inbound function trigger.So it seems obvious to me that there is a problem on the FB side. I followed the procedure to be a valid DEV for this app from FB side. (You're a verified admin on this app.).Do you have a way to troubleshoot what is happening on the FB side?FYI, I deleted everything from the app in FB Dev to redo it and I have the same problem. I also tried with another FB page without success. Thank you again for your help.CommentShareLuc Paquetanswered a year ago0Hi Luc,from the description it sounds that either the trigger is not being sent off or the call to the API Endpoint is not arriving.For both you can double check the settings on the Facebook Developer page for your site in question.Access Key of the page being setup properlyVerify that the Webhook for that Page has been set and it's activated for messages in the hookVerify the Callback URL for the webhook including the Auth token to be match. Also test the callback URL with a tool like Postman an see if you reach the Lambda function through Cloudwatch LogsIf that appears all fine and a manual trigger using Postman work I'd suggest to reach out to the Facebook support to gather some logs from that side.CommentSharefaddianswered a year ago"


Question: How to define API Gateway to Eventbridge integration?

"I am building an API Gateway (v1)  integration such that posts to a given endpoint should route the body of the POST to my custom eventbus in Eventbridge. I have created the gateway endpoints, event bus etc but I am struggling with the definition of the integration.I am doing this in Terraform, which basically wraps the AWS API for PutIntegration and I cannot seem to figure out the correct format of the request-parameters map required by AWS.since eventbus payloads have a specific structure, I assume I need to build the map to construct that payload. I also saw a post about needing a custom X-Amz-Target header as well.Are there any AWS documented examples of just how to build this integration & mapping?My attempts invariably lead to an error response along the lines of:Error updating API Gateway Integration: BadRequestException: Invalid mapping expression specified: Validation Result: warnings : [], errors : [Invalid mapping expression specified: tagmodernization-dev-us-west-2-eventbus-information-reporting, Invalid mapping expression specified: integration.request.body.Entries[0].EventBusName]mapping variations I have tried include:integration.request.body.EventBusNameintegration.request.body.Entries[0].EventBusName]EventBusNameI realize I can achieve a similar goal using VTL with the request templates capability, but I am still unclear on the output format of the mapping anyway.FollowComment"

Answer: "0Accepted AnswerThis blog may provide you guidance on how to accomplish this - https://aws.amazon.com/blogs/compute/capturing-client-events-using-amazon-api-gateway-and-amazon-eventbridge/CommentShareEXPERTIndranil Banerjee AWSanswered a year agorePost-User-8075085 a year agoI was trying to achieve this via request-parameters which does not accept values as described in the linked blog. However, I will pivot to using request templates as described in the blog as I think it will give us more flexibility in the future.update... marking this as accepted.Just one warning to others, velocity templates seem to only function well on leaf node of json object.thus $elem.entry as shown only works because the contents are a string not a json body.   if you are not delimiting, expect to see velocity render  a string of name value pairs  - eg { a= b, c=d} from input of {"a":"b","c":"d"}Share"


Question: can't receive emails on work accounts

"I am unable to receive emails, but can sent them.I have checked and reentered the MX records, waiting over 72 hours, but still nothing.Account is not suspended, domain is still active. What else can I do to fix this?FollowComment"

Answer: "0I would suggest creating a support ticket to troubleshoot this issue. However, one think to check is to ensure that you can resolve the MX records publicly. If so, definitely check in with the support team.CommentShareEXPERTBrettski-AWSanswered a year ago0Hi,I’m sorry to hear you’re experiencing problems with receiving mail. There are a things you can verify to ensure your incoming mail is working correctly.First verify that you MX record only contains references to WorkMail. No other services are listed and if there are make sure these have a lower priority than WorkMail (higher number).Secondly you can check the AWS SES console to verify that your inbound rules are correct. Verify that the active rule contains a WorkMail action and if there are other actions listed ensure they work.If that is all correct, do you receive bounce messages when sending to your email address? If you do these can also contain hints on why the delivery fails.Kind regards,RobinCommentShareMODERATORrobinkawsanswered a year ago"


Question: Cancel everything

I could not figure this out 10 years ago!  Make it stop!  I have paid the "overdue" amount that I had no idea that was even accruing...just cancel everything!FollowComment

Answer: "0Go through the process to close your account.CommentShareEXPERTkentradanswered a month agorePost-User-1511704 a month agoFound out that closing the account wasn't enough...all instances must be terminated to not be charged.  The first year is free and I'm afraid it was just all too confusing to me, I never used it.  Today I learned I had a threat of a suspended account...after 108 months of not paying.  I actually tried to pay but I couldn't even do that correctly and only paid for 1 month.  I was beyond frustrated but finally got someone to chat with me and talk me through shutting it all down. They are looking into why it never suspended the account after 108 months of never having paid once.Share"


Question: Mediapackage Asset Jitter

"Hello! We have some assets on Mediapackage (with DRM) where in some point of playing the video, we have a jitter/glitchy. But playing the source is normal. Another wired thing is reacreating the asset, with the exactly same source and config, it's perfect. Also comparing the manifest output from the asset, they are exactly the same, but the first one has jitter and the recreated doesn't. Any ideia where I should start investigating ?FollowComment"

Answer: "0The first step would be to identify the cause of the jitter/glitchy. You can do this by checking the system logs or performance metrics of your Mediapackage, as well as any other relevant system data that might provide clues as to why the jitter/glitchy is occurring. You may also want to check the output of the asset itself to see if there are any differences between the original and recreated assets, such as incorrect settings or encoding errors. Once you identify the cause, you can then take steps to resolve the issue.CommentShareDivyam Chauhananswered 4 months ago0Can you clarify your workflow? It is unclear what is being compared.Is the input to MediaPackage stored in S3? Is it an HLS or DASH asset?Is MediaPackage being used to provide the DRM encryption? What DRM?Are you using CDN to pull from MediaPackage and deliver to the players?Perhaps you can test pulling through MediaPackage without the DRM encryption.Compare MediaPackage output vs CDN output for both unencrypted and encrypted tests.Also, you may want to review the MediaPackage CloudWatch logs to see if there are errors indicated.CommentShareMike-MEanswered 4 months ago"


Question: Where to create finegrained access policies?

We are moving away from account assuming roles of a target accounts to IAM identity center .I would like to understand what is the best way to manage the AWS policies hereas per Quotas mentioned iam identity center we can attach at most one inline policy.and also the inlinepolicy can not be more than 10KBhowever we can create as many customer managed policies but at most we can attach 10.we have a futuristic view of attaching these customer managed policies to a permission set that might hit the 10 limit soon because of how much finegrained access we want to create per team.is this the best place to manage them in IAM identity center. or should we use the role created in the target account and attach more policies if needed .what is the best practice here.FollowComment

Answer: "1Hello,Firstly, note that whenever we assign an SSO user with a permission set to an AWS account present in your Organization, AWS SSO creates an IAM role in that assigned account with the policies attached to the permission set. Depending upon the permission policies attached to that permission set, your SSO user will access the resources present in your account using that IAM role.Secondly, I am referencing the limits of permissions policies that can be attached to SSO permission set:By default, number of AWS managed policies that can be attached to a permission set is 10. However, this can be raised upto 20 as we describe in our AWS docs here - "AWS Identity and Access Management (IAM) sets a quota of 10 managed policies per role. To take advantage of this quota, request an increase to the IAM quota Managed policies attached to an IAM role in the Service Quotas console for each AWS account where you want to deploy the permission set."Note - The maximum limit for attaching a managed policy to an IAM role or user is 20.https://aws.amazon.com/premiumsupport/knowledge-center/iam-increase-policy-size/The number of custom permission policy (inline policy) that can be attached to a permission set is 1 and this is a hard limit. Also, the maximum size of inline policy per permission set is 10,240 bytes and the same cannot be increased further.Thirdly, as for comparing IAM inline policy vs Identity Center's quota, note that with IAM service, you can add as many inline policies as you want to an IAM user, role, or group. But the total aggregate policy size (the sum size of all inline policies) per entity cannot exceed the following limits:User policy size cannot exceed 2,048 characters.Role policy size cannot exceed 10,240 characters.Group policy size cannot exceed 5,120 characters.While in reference to Identity Center the character limit for inline policy per permission set is 10,240 bytes.Also, I would like to mention that there is already a feature request in place with the Identity Center Development team to support multiple inline policies in SSO permission sets.  Kindly note that AWS take customer’s feedback very seriously, as this allows AWS to better prioritize new features/improvements down the road. Please be assured that I will perform my due diligence in ensuring that your case is used as supporting evidence.Unfortunately, it would not be possible for me to furnish you with an ETA on this feature since support engineers are not made privy to development road-maps and do not have visibility into the development timelines. Therefore, I would request you to monitor our products announcement page and AWS News Blog periodically for updates regarding announcements for AWS services[+] What's New at AWS:https://aws.amazon.com/new/[+] AWS Blog:https://aws.amazon.com/blogs/aws/Having said that, as it is not possible to attach multiple inline policies in AWS SSO permission set, I would kindly request you to add all the IAM policies in a single inline policy and attach it to the SSO permission set. However, if your inline policy has reached the limit of 10,240 bytes then I'm afraid to inform you that the only possible workaround is to cut-down the characters in the inline policy to be within the limit of 10,240 bytes. You can also parse through your custom policy (either inline or customer managed) and consolidate several actions by using a “” ( for example: ec2:Describe, ec2:List*, ec2:Get*).Finally, coming to your query of usage and best practices, with Identity Center, you get the benefits of the following -Identity Center provides users in this identity source with a personalized user portal from which they can easily launch multiple AWS accounts or cloud applications.Inside the IAM Identity Center console, you can leverage application assignments to provide single sign-on access to many SAML 2.0 business applications, including Salesforce, Box, and Microsoft 365.Users sign in to the portal using their corporate credentials or with credentials they set up in AWS SSO. Once they sign in, they have one-click access to all applications and AWS accounts that you have previously authorized.You can read more about fine grained permissions and assignments here -https://aws.amazon.com/iam/identity-center/features/https://d1.awsstatic.com/events/Summits/reinvent2022/SEC207_Simplify-your-existing-workforce-access-with-IAM-Identity-Center.pdfhttps://docs.aws.amazon.com/singlesignon/latest/userguide/abac.htmlhttps://d1.awsstatic.com/events/aws-reinforce-2022/IAM336_AWS-SSO-Every-organization-can-centrally-manage-access.pdfCommentShareSUPPORT ENGINEERYash_Canswered 3 months agoAWS-Everett 3 months agoI agree with everything that Yash_C says in their highly comprehensive write up. I would also like to add that there are other layers of permissions that can be leveraged to reduce IAM and IAM Identity Center policies such as Service Control Policies (SCP) and resource-based policies, see https://aws.amazon.com/blogs/security/iam-policy-types-how-and-when-to-use-them/. SCPs are useful for organization-wide controls like regions and AWS services that are denied, see https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps_examples.html. Here are example of resource-based policies for Amazon S3 -> https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html.Share"


Question: Fail to create Endpoints in SageMaker--JNI and NoClassDefFound error

"I was trying to deploy a model(logged as a Mleap model in Databricks and saved in a s3 bucket) to SageMaker, and got stuck at the Endpoint creation:"The primary container for production variant [xxx] did not pass the ping health check. Please check CloudWatch logs for this endpoint."In the log I found the following block repeating over and over again until some time later the creating Endpoint process just stopped and the status turned 'failed' in the SageMaker UI:Error: A JNI error has occurred, please check your installation and try againException in thread "main" java.lang.NoClassDefFoundError: org/eclipse/jetty/util/thread/ThreadPool#011at java.lang.Class.getDeclaredMethods0(Native Method)#011at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)#011at java.lang.Class.privateGetMethodRecursive(Class.java:3048)#011at java.lang.Class.getMethod0(Class.java:3018)#011at java.lang.Class.getMethod(Class.java:1784)#011at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)#011at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.util.thread.ThreadPool#011at java.net.URLClassLoader.findClass(URLClassLoader.java:382)#011at java.lang.ClassLoader.loadClass(ClassLoader.java:424)#011at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)#011at java.lang.ClassLoader.loadClass(ClassLoader.java:357)#011... 7 moreGot sigterm signal, exiting.I coded in Python in Databricks and as far as I could tell this is a Java error so I have no idea what went wrong, anyone has any experience deploying an 'outside' model to SageMaker?  Any help or tips would be much appreciated!FYI: my whole s3 setup, ECR setup were in us-west-2(Oregon); the integration of AWS IAM role and Databricks Role are properly set up; in my s3,  I could see the Databricks distributed filesystem in one bucket and the trained and pickled model in another; the docker image that is supposed to hold the model is successfully registered in ECR. I also tried to change the instance type under 'Production variants' in the Endpoint creation settings, I set it to the same instance type (ml.m5.large) as the one I used to initiate the Databricks runtime cluster but it did not seem to work.Update: I successfully trained, logged and deployed a sklearn model, but still have the same issue with spark ML model; for the container, I used a image built by mlflow:mlflow sagemaker build-and-push-containerEdited by: ShumZZ on Sep 29, 2019 3:12 PMEdited by: ShumZZ on Oct 2, 2019 2:42 PMFollowComment"

Answer: "0Hi ShumZZ,I'm assuming your model container is built through Mleap's Databrick runtime integration. From the code base (https://github.com/combust/mleap/tree/master/mleap-databricks-runtime-fat), it seems that the  underlying implementation is in Scala, which would require JNI bindings to interact with your Python code.Have you tried running your model container locally? If the error persists when in your local environment, I would suggest reaching out to the Mleap community for assistance. To run your container locally, please follow the commands inSageMaker documentation https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.htmlThank you very much for trying out Amazon SageMaker! Please let us know if you have additional questions.Best Regards,YijieCommentShareAWS-User-9619180answered 4 years ago0Hi Yijie,Thanks so much for the reply! I followed your advice and saw indeed that the local test also failed with the same JNI error...Yet I am a bit confused since I actually used Mlflow's CLImlflow sagemaker build-and-push-containerto build the container. I am quite new to all these concepts (this is actually the first time I've ever worked on an end-to-end ML project), so correct me if I am wrong, should I reach out to Mlflow community instead of Mleap? Or is it the case that under the hood the container is indeed built through Mleap's Databrick runtime integration? Any help/ clarifications/ advices would be much appreciated:)Links that might be useful but I do not quite understand...https://github.com/mlflow/mlflow/blob/master/mlflow/sagemaker/cli.pyhttps://github.com/mlflow/mlflow/blob/master/mlflow/models/docker_utils.pyCommentShareShumZZanswered 4 years ago0After some discussion with the Mlflow community, we confirmed the bug, where Java dependencies are not correctly installed in the docker image that Mlflow uses by default. I posted a bug report (https://github.com/mlflow/mlflow/issues/1906) on Github and a temporary fix has been provided by @smurching (https://github.com/mlflow/mlflow/pull/1913).Edited by: ShumZZ on Oct 10, 2019 12:04 PMCommentShareShumZZanswered 4 years ago"


Question: Intermittent issues connecting from a AWS Client VPN to a VPC Peering connection?

"Hello, I'm seeing something weird and am looking for suggestions of places I should continue to look.VPC A (10.1.0.0/16) is peered to VPC B (10.91.0.0/16) via a VPC Peering Connection. This connection is working perfectly for instances in VPC A's Subnet A (10.1.4.0/22).We have an AWS Client VPN Endpoint in VPC A:Client VPN IP Range: 10.90.0.0/22Target Subnet Association:  Subnet A (10.1.4.0/22)Authorization Rules to allow access to 10.1.0.0/16 and 10.91.0.0/16VPN Route Table set to 10.1.0.0/16 and 10.91.0.0/16Now for the weird part / question. The VPN is only able to connect to Instance B sometimes! I've disconnected and reconnected from the Client VPN Endpoint 21 times, and it's worked for 16 of those attempts:The VPN IPs for working attempts: 10.90.1.2 (x6), 10.90.1.3, 10.90.1.34 (x7), 10.90.1.35The VPN IPs for failed attempts: 10.90.0.2 (x3), 10.90.0.37 (x2)So this leads me to notice that the AWS Client VPN seems to only be able to access the VPC Peering connection when it gives the client VPN an IP address in the 10.90.0.0/24 sub-range!?We don't have any networking in VPC that does anything regarding the 10.90.x.x range so I believe this is an issue on the client-side with the AWS official VPN client. Nothing appears to be different with the local routing between when it works and doesn't work.Any help would be greatly appreciated! Thanks!FollowComment"

Answer: "0Accepted AnswerThe issue seems to have resolved itself! 🫠CommentSharetdooner-gcfanswered a month ago0Can not see any obvious error above, but details on VPC B are bit limited. Have you followed these steps when configuring this https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/scenario-peered.htmlCommentShareKalluanswered 2 months agotdooner-gcf 2 months agoThanks for taking a look. VPC B is maintained by a PaaS company and we have no visibility into it. However, my understanding is that all the traffic it sees should be coming from the subnet IPs (10.1.4.0/22).And yeah, I did all the steps in that article above.Share"


Question: Why set X-Amz-Target header for EventBridge integration?

"Why do I need to set the header X-Amz-Target as 'AWSEvents.PutEvents' while integrating API Gateway with EventBridge? I never had to do similar for Firehose, S3, SQS, etc..The Action is already set to PutEvents so why the target header is required again? Please clarify.Thanks,SheenFollowComment"

Answer: "0Hello:I believe you've already gotten a response via twitter, but the TL;DR is that not all AWS services behave the same. Some services do require you to explicitly map the X-Amz-Target header and others will work with the action header specified by API Gateway when configuring the integration.Feedback from your twitter thread will be taken into account as we look at adding AWS service integrations to HTTP APIs.Regards,BobCommentShareEXPERTAWS-User-0395676answered 3 years ago"


Question: Failed to create a node group for an EKS cluster: You've reached your quota for maximum Fleet Requests for this account

"I tried to create an EKS cluster by terraform but the plan failed with the below error msg.[AsgInstanceLaunchFailures: You've reached your quota for maximum Fleet Requests for this account. Launching EC2 instance failed.]I don't believe the root cause of this failure is a quota issue because there are not many EC2 instances within the region (ap-northeast-2). So I tried below to find what the real problem is but still on the same page. Please advise what I should do, thanks.Fyr, the IAM user has administrator permission.Run the same terraform code with the different account (all settings were same, only account (not IAM) was changed to my private one)Result: Success to buildConclusion: No issues with the code (network configuration is included)Run the same terraform code in the different region (tried ap-east-1)Result: Success to buildChanged an instance type from t3.medium to m4.large (the other settings were same)Result: Failed to build with the same errorFollowCommentamtoya 9 months agoI am too getting the same error. I am using ap-southeast-2 region and instance_type=t3.xlarge.Sharedevops-stchealth 8 months agoSame error. It's new AWS account is new with very few EC2. Something else is wrong when done via TF automation or eksctl or manual. I need a solution to this.unexpected state 'CREATE_FAILED', wanted target 'ACTIVE'. last error: 2 errors occurred:│ * eks-managed-ondemand-20220916000118777000000009-90c1a1cc-c40a-4d69-60fa-d40ad3479549: AsgInstanceLaunchFailures: You've reached your quota for maximum Fleet Requests for this account. Launching EC2 instance failed.Share"

Answer: "0Hi.If you didn´t try to double-check your EC2 quota, I guess visiting https://console.aws.amazon.com/servicequotas/home will show you.Another thing to pay attention to if the node group that terraform wants to create is using the usual EC2 or Spot because the service quotas have specific values for each Spot EC2 family.Hope you find out the solution soon. :)CommentShareTrombinianswered 9 months agohyunie 9 months agoHi,I checked the limit is 640 vCPUs and our account is using 18 vCPUs only in total so seems it's not limitation issue.In addition, the terraform code doesn't have any configuration about spot. When I've tried same code on the different region, usual EC2 instances were launched, not spot instances.Is there any other fact can cause the above error msg - You've reached your quota for maximum Fleet Requests for this account?Share"


Question: Unable to stop EC2 instance

"There was a massive operational issue for my availability zone ("degraded performance for some EBS volumes in a single Availability Zone (USE1-AZ2) in the US-EAST-1 Region"). After 3 hours, AWS announced that it is fixed. The problem is that I can no longer stop my instance (id: i-5c4a4cb2) as it's always stuck at "Stopping..." even after I attempted to force stop from both the CLI and the Dashboard. Any help would be appreciated in restarting my instance.FollowComment"

Answer: "0Same issue...3 instances stuck at stopping...can't detach volumes as well.CommentSharebalikpinoyanswered 2 years ago0Same issue...3 instances stuck at stopping...can't detach volumes as well.CommentSharebalikpinoyanswered 2 years ago0Same issue here.Can't force stop and its our secondary prod db server, causing massive outage issues across our systems.Hopefully we can get a resolution soon.CommentShareunixguru2kanswered 2 years ago0the same is happening to me!!! aws support, please help us!CommentSharengcsmanswered 2 years ago0seems to be resolved for us guys.we are assuming the fix is still in propagation for all accounts.for aws status:[01:15 AM PDT] We can confirm that the performance has returned to normal levels for the majority of the affected EBS volumes within the affected Availability Zone (USE1-AZ2). Starting at 12:12 AM PDT, we saw recovery slow down some affected EBS volumes as well as some degraded performance for a small number of additional volumes in the affected Availability Zone. We have investigated the root cause and mitigations are underway to complete the performance recovery of the affected EBS volumes. We continue to work towards full resolution for all affected EBS volumes. In some cases, customers may be experiencing volume state transition delays, which we expect to clear up once volumes have fully recovered.CommentShareunixguru2kanswered 2 years ago0Finally fixed for us. Was able to restart my instance. Thanks.CommentSharevincentlaanswered 2 years ago"


Question: SSO set up error

we created a SSO account and I just added one application to it (PagerDuty) but each time I try opening the application we get a 400 The was a bad request. I checked the configurations and they are all correct and mappings are set correctly. What might be the problem?FollowComment


Question: EKS adot to AWS Managed Prometheus remote write in another account

"I followed the tutorial here (https://docs.aws.amazon.com/eks/latest/userguide/deploy-deployment.html) and it works well when the prometheus remote write is on the same account than the EKSNow I have a separate account for EKS and Managed Prometheus, so I need to assume a role to be able to write to the prometheus remote write. I used this yaml https://raw.githubusercontent.com/aws-observability/aws-otel-community/master/sample-configs/operator/collector-config-amp.yamlAnd I modified the extensions/sigv4auth like this :apiVersion: opentelemetry.io/v1alpha1kind: OpenTelemetryCollectormetadata:  name: my-collector-ampspec:  mode: deployment  serviceAccount: adot-collector  podAnnotations:    prometheus.io/scrape: 'true'    prometheus.io/port: '8888'  config: |    extensions:      sigv4auth:        assume_role:          arn: "arn:aws:iam::1234567890:role/prometheus_remote_write_assumerole"        region: "us-west-2"        service: "aps"Got that error :Error: failed to get config: invalid configuration: extension "sigv4auth" has invalid configuration: bad AWS credentials2022/09/08 13:57:32 application run finished with error: failed to get config: invalid configuration: extension "sigv4auth" has invalid configuration: bad AWS credentialsFollowComment"

Answer: "0Assuming you are using ADOT for EKS add-ons, we expect the next release shortly, and the error messaging here will be improved, providing you more context to resolve your issue.CommentSharehsueheanswered 7 months ago"


Question: Firehose only delivers 99% of records to s3

"I have set up a firehose (direct put) to deliver messages to a s3 bucket.Unfortunately, regardless of the number of message to be delivery, Firehose always delivers n-500 messages.In screenshot attached, we can see what happens when attempting to deliver 500000 records, it gets stuck at 4995000. Same happens when I attempt to deliver 200000 messages, it gets stuck at 199500I have replaced the firehose, change the instances, even tried to do the same from different aws accounts, but the result is the same.Anyone who could shed some light on this would be very much appreciated.Sincerley,Josep MariaEdited by: JosepMaria on Sep 27, 2020 5:53 AMEdited by: JosepMaria on Sep 27, 2020 5:57 AMFollowComment"

Answer: "0Modify code by adding :"initialPositionon":"START_OF_FILE",CommentShareJosepMariaanswered 3 years ago"


Question: Aws SSO User Portal Branding/Ui Options

We are using AWS SSO to authenticate our users to management console and certain saml applications. We were curious if it was possible to add a custom logo to both the saml application and the overall user portal. I cant find any documentation on how to make any kind of UI changes to the default user portal. Any answers would be greatly appreciated.FollowComment

Answer: "1Customization in SSO is limited to customizing the SSO URL. Another way of customizing is Integrating SSO as an application in your application gallery. For example, as referred in https://aws.amazon.com/about-aws/whats-new/2021/03/new-aws-sso-gallery-app-simplifies-azure-ad-set-up-with-aws/CommentShareCheriyanManswered a year ago"


Question: EC2 Windows instance stops automatically

"One of our EC2 windows (Windows 2022 Data Center Edition) instance randomly stops without any manual interference.It's a c6a.8xlarge instance type running at Mumbai location. The server is used for RDP purpose. Whenever users complain they are kicked out of their sessions, we mostly see that it is in "Stopped" mode in the dashboard.This is really impacting our production.How do we resolve this issue?FollowComment"

Answer: 0I would find out who/what was stopping the instance. See this article to get this information from CloudTrail: How do I use AWS CloudTrail to track API calls to my Amazon EC2 instances?.CommentShareEXPERTkentradanswered 2 months ago


Question: Aws cloudhsm pkcs11

Is it possible To use opensc pkcs11 cli tool to create keys inside aws cloudhsmFollowComment

Answer: "0To use PKCS11 , customer has to integrate PKCS11 library from Client SDK 3 or 5. PKCS11 is generally used by application running on Python , Rust go etc.Below link does mention opensc compatibility with PKCS11.https://en.wikipedia.org/wiki/List_of_applications_using_PKCS_11customer needs to follow methods and mechanism mentioned inhttps://docs.aws.amazon.com/cloudhsm/latest/userguide/pkcs11-mechanisms.htmlCommentSharerePost-User-3546897answered 19 days ago"


Question: What is the best way to approach for quick support from AWS?

"We are unable to access ec2 windows machine via RDP. We tried with generating and decrypting password with the ssh-key for the machine. But still not working. I have opened two tickets for the same issue. I am expecting a quick response from AWS support.Severity should be high, but it is setting with general inquire. Where I choose System impaired.FollowComment"

Answer: "0Hi. There are different levels of support depending on the support plan you choose. https://aws.amazon.com/premiumsupport/plans/Developer support gives you the option to choose System Impaired severity, that's the reason you can only choose general guidance.You could use EC2 Rescue: https://aws.amazon.com/premiumsupport/knowledge-center/ec2rescue-windows-troubleshoot/ to figure out why you can't connect.CommentSharePeter Graingeranswered 2 months ago"


Question: Salesforce to quicksight connection fails because of OAuth expiration even it's set not to expire

"Hello,since a couple days all of my datasets in Quicksights with a connection to Salesforce won’t refresh anymore.The error message states that the OAUTH token is not valid anymore but so far I haven’t found a solution to refresh it.In Quicksights useremanual it says that you can’t edit a Saleforce connection.In one of my attempts to fix this problem I recreated the data source and loged in to Saledforce again. That worked for a couple days until the OAUTH token expired again.I also set the following settings in Salesforce OAUTH settings:Permitted Users:  All users may self-authorizeIP Relation: Relex IP restrictionsRefresh Token Policy: Refreshtoken is valid until revokedFollowComment"


Question: We want to understand the pricing of sending SMS using SNS and what is Variable Pricing for Amazon SNS SMS Notifications to Rest Of World (non-US)?

What is Variable Pricing for Amazon SNS SMS Notifications to Rest Of World (non-US) and how this is calculated?FollowComment

Answer: "0This is available in the AWS website and   FAQsThe pricing depends on whether you want to create a dedicated long code (10 digit number for US) or a short code (4 digit number for US) and how many SMSs you send. you can calculate the cost for each country you need to send to by looking up each country. There are specific restrictions to send messages to Singapore, US and India. Check those out to comply with regulations and incur the best cost.NOTE. This pricing is only  indicative. The price not fixed as AWS has to contract this service to a telecom service provider. As per their T&C, they can change the price with a short notice or even without notice. I would suggest to send test messages and observe the cost of usage - It takes 24 hours for the SMS usage report to be available in the S3 bucket.NOTE: If you are an enterprise customer, please contact your TAM.CommentShareAnanth Tirumanuranswered 9 months agoShehan Jayawardane 7 months agohow does cost for SMS for each country charged ? per SMS?Share"


Question: ETL using AWS Glue

"We are doing ETL operation from Redshift to Redshift using AWS Glue. We need to do incremental operation. We need to identify latest modified records from source and process only new and latest modified records(recent updates) from source to avoid reprocessing of millions of records.We have following key columns in source:ID(primary key column),CreatedDate(timestamp in which record has created),UpdatedDate(timestamp in which record has updated. Initially, it will be null)Is there any possibility to process only latest records, instead on re-processing all the records.Much appreciated for possible solutions.FollowComment"

Answer: "1One of the options to identify incremental data is to create a view on top of the source table where the create or update timestamp is greater than the last job run time. You can create the job metadata table to capture the last job run time or any other information based on your requirement to process incremental data.Since both source and target clusters are Redshift, you may want to explore Data Sharing, which is available in the Ra3 instance type and will allow you to read data directly from the source cluster. In that case, you do ELT in the source cluster and share live data with the target cluster. For additional information, please refer tohttps://aws.amazon.com/redshift/features/data-sharing/https://docs.aws.amazon.com/redshift/latest/dg/datashare-overview.htmlCommentShareNita_Sanswered a year ago0Unless I am missing something, I would not use AWS Glue for a Redshift to Redshift ETL process. Glue would be great for file based (likely parquet) data in S3. I would write a Redshift stored procedure and use a Step Function to parameterize and schedule it. For some info on how to incrementally process in Redshift, see this article: https://docs.aws.amazon.com/redshift/latest/dg/t_updating-inserting-using-staging-tables-.htmlCommentSharetjtollanswered a year ago0AWS Glue jobs support bookmarks. You need to have a compound key with those key columns (I'd try only with ID and UpdatedDate first) and use it as job bookmark key in your ETL job.CommentShareEXPERTTasioanswered a year ago0Is there a reason to ETL from one cluster to another?If your clusters are using RA3 nodes, using data sharing between the clusters will remove any ETL requirements as you will get fast access to up to date, transactionally consistent data without needing to move data between your clusters.If there are additional processing needed, you can use native stored procedures on top of datasharing to accomplish the task.https://aws.amazon.com/redshift/features/data-sharing/CommentShareJaganswered a year ago"


Question: SMS delivery to specific country

"Presently, our account is configured to send SMS worldwide. How can we change it to deliver to a specific country?FollowComment"

Answer: 0It is currently not possible to limit the delivery to specific countries with IAM conditions or a similar mechanism. What you can do instead is to restrict it on the application level.CommentShareben-from-awsanswered 6 months ago


Question: Lightsail CDN using Lighsail ALB Distribution with custom domain

"Hi,I have the following infrastructure:LightSail CDN --- (distribuition) > Lightsail ALB --> Lightsail ServersEverything works as expected except my custom domain from CDN which changes to Load Balancer DNS name once someone hits my custom DNS. For example: My domain is pointing to CDN DNS Name using approved certificate and enabled custom domain. So the network behaves like that:Mydomain (mysample.com) -> Lighsail CDN ( cdn-sample123.cloudfront.net) -> Lighsail Load Balancer (lb-sample.elb.amazonaws.com) -> Lightsail Instances.The problem is when I hit "mysample.com". My web site is indeed loaded but the address changes to lb-sample.elb.amazonaws.com. The consequence as you can expect is the browser complaning about certificate: the certificate is issued to mysample.com and the URL address is lb-sample.elb.amazonaws.com.How can I prevent CDN change the address to the load balancer and keep my custom DNS instead?FollowComment"

Answer: "0Hi Jasinski,I understand you access your website the address changes to “lb-sample.elb.amazon.aws.com” and does not stay as the custom address `mysample.com. You can prevent the change by creating an SSL/TLS certificate for your distribution, follow the documentation with the guided steps[1].[1] https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-create-a-distribution-certificateCommentShareLwazianswered a year agoJasinski a year agoHi Lwazi, thanks by the answer. I already have ths SSL/TLS certificate ate my CDN. I think the issue is that my ALB is also SSL/TLS configured and probably this is changing the URL from my CDN custom domain.Share0The issue was that my ALB was SSL/TLS configured also. So once CDN was redirecting to ALB, the ALB also changed to enforce SSL. To solve this I've created a new load balancer withou SSL/TLS and redirected my CDN to this LB.CommentShareJasinskianswered a year ago"


Question: Lughtsail distributions and logs

I'm new to lightsail. I'm considering to enable distribution of my website (cloudfront CDN) but I can't find nowhere informations about what happens to mi apache logs.At present I have scripts running periodically that browse logs in near real time and I'm worry that I will lose this functionality.There is any activity log on my server or any other way I can access CDN logs?ThanksFollowComment

Answer: "0Hello,With the current information you have provided about processing Apache logs with LightSail in real-time, I believe this will help.With LightSail, you are able to install Apache and browse the logs similarly to your current configuration.For Linux these logs are located at: /var/log/httpdYou can also run Lambda functions in real-time/near real-time to process these log files and send them to an S3 bucket for archival purposes. Below are links to Lambda, S3, and LightSail to get you started on a solution that will work for you.Lambda: https://docs.aws.amazon.com/lambda/index.htmlUsing Lambda with S3: https://docs.aws.amazon.com/lambda/latest/dg/with-s3.htmlCommentShareAnthony_Lanswered 10 months agoSUPPORT ENGINEERAWS_SamMreviewed 10 months ago"


Question: Can we generate a custom URL to access the s3 bucket object lifetime?

"Hello guys, can we generate a custom URL or any other way through which we can store the URL in DynamoDB so we can access that object anytime, or if I grant all the permission to specify bucket or grant access? Is there any solution for thisFollowComment"

Answer: "0Hi,Yes, there is a way to do that by using the Pre-Signed URL.(https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html) you can also verify TTL (time to live) for the same URL. However, I would never suggest following this process for private AWS access. instead of that use a bucket policy for DynomoDB and S3 to access the s3 bucket (A secure way to access it). For public uses always attached TTL for security.CommentShareVISIanswered 10 months ago0You can do that in several ways and choose a right option based on the consumers of the object, data sensitivity and access controls you would like to have.Some resources that could help you -https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-https-requests-s3/https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.htmlhttps://aws.amazon.com/premiumsupport/knowledge-center/read-access-objects-s3-bucket/https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html (Short Living URL - Max. 12 hours life time at this moment)CommentShareRajesh Sanswered 10 months ago"


Question: "In RDS Custom for SQL, can users or admins assume roles such as sysadmin?"

Curious if there are restrictions for RDS Custom SQL ServerFollowComment

Answer: 1You can have SA permissions to SQL Server on RDS Custom.CommentShareVikas_Ganswered 9 months ago0Accepted AnswerRDS custom gives the maximum permission - which is a Sysadmin (SA) permission within the database and also an Administrative access to the Operating System. (Both are not possible in the standard RDS instance.)CommentSharelightanswered 9 months ago


Question: AWs trigger EventBatchingCondition/BatchWindow is not optional

"Hi team,I have a glue workflow :trigger (type = "EVENT") => trigger a glue job (to take data from S3 and push them to MySQL RDS)I configured the glue Triggering criteria to kickoff the glue job after 5 events were received.in the console it says :Specify the number of events received or maximum elapsed time before firing this trigger.Time delay in seconds (optional)on AWS documentation it says also it's not required :BatchWindowWindow of time in seconds after which EventBridge event trigger fires. Window starts when first event is received.Type: IntegerValid Range: Minimum value of 1. Maximum value of 900.Required: NoSo I want only my trigger to be triggered only and only after 5 events are received and not depending on: Time delay in seconds (optional).actually, the Time delay in seconds (optional) is set to 900 by default and my job is started after 900s even if there are no 5 events received.that's not the behaviour we want. We want ONLY the job to be started after x events are received.I tried via the console to edit the trigger and remove the 900s for the Time delay in seconds (optional) input but I can't save it until I put a value on it.it says it's optional but it doesn't seem to be.is there a workaround to make the trigger not take account of Time delay in seconds (optional)? and only be launched when it received x events and nothing else.right now the behaviour I have is that my job is triggered after 900s, we want to eliminate this case and let the job be triggered only and only if there is x event received and nothing else.how can I make the  Time delay in seconds (optional) input optional, because now the console forces me to put a value in there?thank you.FollowComment"

Answer: "0Hello,Firstly as mentioned in [1], BatchSize is required and BatchWindow is optional. If BatchWindow is omitted, the window defaults to 900 seconds, which is the maximum window size.Secondly, as mentioned on glue console under Time delay in seconds “The time delay starts when the first event is received”.Therefore, Glue trigger expects all the events to happen with in a maximum of 900 seconds from the start of first event.————————Reference:[1] https://docs.aws.amazon.com/glue/latest/dg/starting-workflow-eventbridge.html================ Thank you and have a nice day!CommentShareSUPPORT ENGINEERArunanswered a year ago"


Question: Which account's API quota is used when a role is assumed?

"If user A and user B assume the same role and both users are making a large number of AWS API requests as the assumed role, are the AWS API quotas / rate limits shared?FollowComment"

Answer: "1The API rate limits are applied to account to which the calling credentials belong.So if user A from account X assumes role B in account Y and then makes API calls it is account Y where the limits are applied.CommentShareEXPERTBrettski-AWSanswered a year agolzap 7 months agoThere is an answer that says the opposite, if I read this correctly. Can you elaborate please?ShareBrettski-AWS EXPERT7 months agoI don't know what else to add here: The limits are applied to the account where the credentials that are calling the API belong to. If you assume a role in another account (let's call that account "M") then the limits are applied to account M, not the original account where you assumed the role from.Share1Limits are applied account-wise. In the the case described above, the limitation is applied to the account to which gets the API Call.CommentShareJapanTakeAWSConsoleanswered a year agolzap 7 months agoThere is an answer that says the opposite, if I read this correctly. Can you elaborate please?Share"


Question: What services are active on account?

How do I find my active services on Lightsails?FollowComment

Answer: "1I am assuming you would like to find out what services e.g. Apache, IIS is running on Linux/Windows Lightsail server.  You can SSH or RDP to the server to find out.===Extracted from Lightsail FAQ (https://aws.amazon.com/lightsail/faq/ )===Lightsail offers a 1-click secure connection to your instance's terminal right from your browser, supporting SSH access for Linux/Unix-based instances and RDP access for Windows-based instances. To use 1-click connections, launch your instance management screens, click Connect using SSH or Connect using RDP, and a new browser window opens and automatically connects to your instance.CommentShareronald_lanswered 4 months agoEXPERTChris_Greviewed 4 months ago"


Question: Maximum throughput for incrementing an atomic counter in DynamoDB with On-Demand Capacity Throughput default quotas

"I would like to know, what the maximum throughput in DynamoDB for incrementing an atomic counter would be.Consider a table with just an id and a counter attribute, so that the overall item size is less than 1K.I would use something like this UpdateExpression for incrementing the counter:SET voteCounter = if_not_exists(voteCounter, :startValue) + :incValueTaking into account the default Throughput Default Quotas for On-Demand Capacity of 40.000 write request units, i would expect to get 40.000 increments per second in the best case. Best case would mean, no other operations would happen on my DynamoDB.As this is a rather extreme use case i could imagine, that the maximum throughput of 40.000 updates per second can only be reached, if updates are distributed on lots of different items and in my use case throughput would be significantly lower. Therefore i am curious, whether 40.000 increments per second is possible.P.S.:I am aware that there are approaches for Scaling Distributed Counters but i am just curious for the raw numbers, which are realistic for the use case as descibed above.FollowComment"

Answer: "0Accepted AnswerSee https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html particularly the 'Isolate Frequently Accessed Items' section."If your application drives consistently high traffic to a single item, adaptive capacity might rebalance your data such that a partition contains only that single, frequently accessed item. In this case, DynamoDB can deliver throughput up to the partition maximum of 3,000 RCUs or 1,000 WCUs to that single item’s primary key."These per-item limits apply to both OnDemand and Provisioned billing modes.CommentShareAWS-User-3966479answered 3 years ago"


Question: "AWS Glue pyspark, AWS OpenSearch and 429 Too Many Requests"

"Hi forum,I'm on AWS and trying to write ~ 1.2mio documents from an AWS Glue 2.0 job Python / pyspark job to an OpenSearch 1.2 "t3.small.search"/SSD cluster.The issue I'm facing is that after a while I'm facing "429 Too Many Requests":org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: [PUT] on [MY/doc/_bulk] failed; server[https://SOME_ENDPOINT_HERE] returned [429|Too Many Requests:]From what I understand and read so far this is pretty much about configuration, throttling down indexing requests on the client side giving the server more time to process queued requests. And that's what I tried but somehow the config on the Hadoop connector does not work for me.Already tried to send smaller batches of documents to the ElasticSearch and increased retry wait time: set 'es.batch.size.entries' to 100 and 'es.batch.write.retry.wait' to 30s:df \    .write \    .mode('overwrite') \    .format('org.elasticsearch.spark.sql') \    .option('es.nodes', 'SOME_ENDPOINT_HERE') \    .option('es.port', 443) \    .option('es.net.ssl', 'true') \    .option('es.net.http.auth.user', 'SOME_USER_NAME_HERE') \    .option('es.net.http.auth.pass', 'SOME_PASS_HERE') \    .option('es.nodes.wan.only', 'true') \    .option('es.nodes.discovery', 'false') \    .option('es.resource', 'SOME_NAME_HERE') \    .option('es.index.auto.create', 'true') \    .option('es.mapping.id', 'SOME_FIELD_HERE') \    .option('es.write.operation', 'index') \    .option('es.batch.size.entries', '100') \    .option('es.batch.write.retry.policy', 'simple') \    .option('es.batch.write.retry.count', '-1') \    .option('es.batch.write.retry.limit', '-1') \    .option('es.batch.write.retry.wait', '30s') \    .save()Already set logging for 'org.elasticsearch.hadoop.rest' logger to DEBUG level:Bulk Flush #[12653715211658247214404]: Sending batch of [34000] bytes/[1000] entriesBulk Flush #[12653715211658247214404]: Response receivedBulk Flush #[12653715211658247214404]: Completed. [1000] Original Entries. [1] Attempts. [1000/1000] Docs Sent. [0/1000] Docs Skipped. [0/1000] Docs Aborted.From what I understand the Hadoop-Connector is sending batches of 1000 documents, not the 100 from my config. Further I can not see any wait time.My actual setup on AWS is:Spark: 2.4.3Python: 3.7OpenSearch: 1.2Elasticsearch Hadoop: 7.13.4 (elasticsearch-spark-20_2.11-7.13.4.jar)Any hints or ideas on my setup?Many Thanks,MatthiasFollowComment"

Answer: "1i'd rather increase the batch size to reduce the overall number of requests to Opensearch. You also may want to increase refresh timeout.https://aws.amazon.com/ru/premiumsupport/knowledge-center/opensearch-indexing-performance/On the other point, t3.small cluster is really small so you might need to use different type of instancesCommentShareAlex_Tanswered 10 months agomatze79 10 months agoHi, thanks for your quick reply @Alex_T.Updated to t3.small cluster and the indexing of the 1.2 million records was immediately successful with AWS Glue in 7 minutes. :) So was a useful hint with upscaling cluster instances.For those who are interested: I also asked the fine people at Elastic about this:https://discuss.elastic.co/t/aws-es-hadoop-and-429/310124The guys over there mentioned that the option "es.batch.size.entries" is not respected under any circumstances. For my use case for example I enabled PySpark's overwrite mode in AWS Glue:df.write.mode('overwrite')...Before new documents are indexed the index is first emptied here. Turns out there is no config option in the elastic-hadoop module for that initial delete, so I always saw "1000 records" in my logs. Maybe this will help someone later.Thanks again.Best,MatthiasShare"


Question: Jupyter kernel dies on SageMaker notebook instance when running join operation on large DataFrames using pd.merge

"I am running a large pandas merge join operation on a jupyter notebook running on SageMaker notebook instance ml.t3.large i.e 8 gb of memory.import pandas as pd        df1 = pd.DataFrame({                         'ID': [1, 2, 3],                        'Name': ['A','B','C'],                        ....                      })    df1.shape    (3000000, 10)        df2 = pd.DataFrame({                        'ID': [],                        'Name': [],                        ....                      )}        df2.shape    (50000, 12)                                      # Join data        df_merge = pd.merge(                         df1,                         df2,                         left_on = ['ID','Name'],                         right_on = ['ID','Name'],                         how = 'left'                       )When I run this operation, the kernel dies within a minute or so. How can I optimize this operation for memory efficiency?FollowCommentAshleyMiller EXPERT4 months agoUsually a kernel will die for one of two reasons: 1) runs out of memory, 2) a bug in the code or a library. Try running this with a subset of your dataset and see if it runs to completion without error. This would eliminate the possibility of a bug. Then choose an instance type with more memory (ml.t3.xlarge has 16GB or RAM) and see if that is enough memory for your dataset.Share"

Answer: "0Hello,A SageMaker kernel could die due to resource utilisation load or an issue within the code or a third-party library.Please check the system resource utilisation in order to ensure that the operations are running at appropriate load levels.To check SageMaker notebook instance resources, enter the following commands in the Notebook terminal:To check memory utilisation:free -hTo check CPU utilisation:topTo check disk utilisation:df -hIf you see high utilisation of CPU, memory, or disk utilisation, please try these solutions:Restart the notebook instance and try again.Review your SageMaker notebook instance type to verify that it's properly scoped and configured for your jobs.=> If a resource crunch is observed, kindly switch to a larger instance type and check if the issue gets fixed.[+] https://aws.amazon.com/premiumsupport/knowledge-center/sagemaker-troubleshoot-connectivity/You may also refer to the following documentation to check for CPU, memory and Disk utilisation : https://aws.amazon.com/premiumsupport/knowledge-center/open-sagemaker-jupyter-notebook/#:~:text=High%20CPU%20or%20memory%20utilizationPlease try running the code separately and execute rows one by one if possible to identify the issue at a more granular level and check if everything works fine with respect to the code.Additionally, please try the following to fix the issue:Close the active sessions and clear the browser cache/cookies.When you have a large number of active sessions, the kernel might take longer to load in the browser.Open the SageMaker Notebook in a different browser. Check if the kernel connects successfully.Restart your notebook instance.I hope this helps!CommentShareVedanshi_Aanswered 4 months ago"


Question: How does data flow as you scale up Kinesis Data Steams and add KPU's to Kinesis Data Analytics.

"I have an event processing architecture that's basically KPL Producer -> Kinesis Data Streams(Raw) -> KDA(Flink Aggregation) -> Kinesis Data Streams(Agg) -> Lambda -> Timestream .  With a basic setup of 10 Raw Shards, 2 KPU's on the Flink App, and 5 Agg Shards(On Demand) and maybe 100 requests per second, my stack processes the data with maybe a delay of minute going through the whole workflow to Timestream.  As I pre-scale things up for a much higher workload(that won't be arriving for several days), I notice that my current delay goes from 1 minute to 30 minutes to arrive in Timestream.  The question I need to understand is where in my workflow is the latency being added.  My current stack is now at 200 Raw Shards, 25 KPU's, and 15 Agg Shards(On Demand).  I know the delay is most likely in Flink as I can see bursts of records being written out every 30 minutes using the incoming record count of my Agg KDS but my question is why?  I know that it will clear up when my requests per second go thru the roof but I always wondered what underneath the hood is occurring in Flink to add this latency.FollowComment"

Answer: 0Accepted AnswerDo you have any idle Kinesis shards? This would make the Apache Flink application watermark stall because it cant progress. This can be avoided by setting SHARD_IDLE_INTERVAL_MILLIS in the Kinesis Consumer config.CommentShareflomairanswered 9 months agosmithatlanta 9 months agoI'm sure I do.  I see that this setting has been deprecated with the ".withIdleness" on the watermarking strategy.  Thank you for pointing me in the right direction.Share


Question: EC2 Instance status check failure and the server doesn't respond

"An EC2 instance suddenly stopped working, even though we did not change anything on the server.The website hosted on the server is no longer accessible. Also, we cannot connect to the server via SSH or with the EC2 instance connection from AWS.When we restart the instance, the instance status is "Running", the system check is successful, but the instance status check failed.When we try to connect to the server via SSH, we get a connection timeout on port 22.When we try to connect to the server via the EC2 instance connection we get this error message : "Failed to connect to your instanceEC2 Instance Connect is unable to connect to your instance. Ensure your instance network settings are configured correctly for EC2 Instance Connect. For more information, see Set up EC2 Instance Connect at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-connect-set-up.html."We also have other servers running on the same subnet, they are running fine. Therefore, I think it is unlikely that it is due to the setting of the VPC, subnet or routing tables.The following image shows the system log of the instance.No error message is displayed in the whole log and the last entry of the log is always the login promptWe have tried the following things to fix the error. However, none of them helped and after restarting the instance, the log file shown above was always displayed.We followed the instruction "Retrieve the system logs" on the AWS-Support site clickWe have looked at the resource monitor and all values are far below our maximum.We have changed the server to different and better instance types.We have restored several backups. All were created well before the error but it did not change anything.We were able to access the server using the EC2 Serial Console. We can enter username and password. Unfortunately, there is no user for the server that has a normal password. You can only connect with the private key. Therefore, the Serial EC2 console is not useful for us.FollowComment"

Answer: "0The causes of instance status check failure are as described below.Do any of these seem likely?This can occur, for example, when EBS is used up to its capacity limit.If you are running databases, etc. in EC2, it is rare that you will run out of EBS capacity.https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-system-instance-status-check.html#types-of-instance-status-checksFailed system status checksIncorrect networking or startup configurationExhausted memoryCorrupted file systemIncompatible kernelCommentShareEXPERTRiku_Kobayashianswered a month ago"


Question: modify SES config with CDK for existing Cognito user pool

"Hi Team,I'm configuring my existing cognito user pool to use SES, by adding this block of code to my existing CDK code that define the user pool:.....email: cognito.UserPoolEmail.withSES({    sesRegion: 'us-east-1',    fromEmail: 'noreply@myawesomeapp.com',    fromName: 'Awesome App',    replyTo: 'support@myawesomeapp.com',    **sesVerifiedDomain**: 'myawesomeapp.com',  }),..........but when I deploy the stack it failed with this error:there is already a construct with name : my-construct-idis not possible to change the SES config for an existing user pool?it seems to try to create a new  user pool when i tried to deploy not changing the existing one.Thank you!FollowComment"

Answer: "0Hello,The error “there is already a construct with name” occurs when you have two constructs with the same construct Id in the same cdk project.I’m assuming you have created a user pool earlier and now to configure it with SES, you have created one more construct with the same construct Id to define theseemail: cognito.UserPoolEmail.withSES({    sesRegion: 'us-east-1',    fromEmail: 'noreply@myawesomeapp.com',    fromName: 'Awesome App',    replyTo: 'support@myawesomeapp.com',    **sesVerifiedDomain**: 'myawesomeapp.com',  }),However, you can just add these properties to the user pool construct which already exists in your cdk code, as that would update that existing user pool with the above email configurations.const userPool = new cognito.UserPool(this, 'userpool', {      userPoolName: 'my-user-pool',      selfSignUpEnabled: true,      signInAliases: {        email: true,      },      autoVerify: {        email: true,      },      standardAttributes: {        givenName: {          required: true,          mutable: true,        },        familyName: {          required: true,          mutable: true,        },      },      passwordPolicy: {        minLength: 6,        requireLowercase: true,        requireDigits: true,        requireUppercase: false,        requireSymbols: false,      },      accountRecovery: cognito.AccountRecovery.EMAIL_ONLY,      removalPolicy: cdk.RemovalPolicy.RETAIN,      email: cognito.UserPoolEmail.withSES({            sesRegion: 'us-east-1',            fromEmail: 'noreply@myawesomeapp.com',            fromName: 'Awesome App',            replyTo: 'support@myawesomeapp.com',          }),    });If this is not the case, then do check for constructs with the same construct Id in your cdk project and rename any one of them.Please note that, if the user pool was created through console then you would have to use CDK Custom Resources to add the email configurations and pass the parameters you wish to update to the “UpdateUserPool” API call handled by that custom resource. Kindly refer to the custom resource documentation [1] and API reference documentation [2] to know more about the same.If the above suggestions don’t suffice your use case, please feel free to elaborate more on this issue.[1] https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources-readme.html#custom-resource-examples[2] https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_UpdateUserPool.htmlCommentShareAbhishek_Tanswered 6 months ago"


Question: Macie job not able to find mobile no. in s3 bucket

"I created a Macie Job for identifying mobile numbers(+91 based) and used all the AWS managed identifiers in the job, yet the job was not able to find/highlight the mobile numbers in the data. Any reason for the same?FollowComment"

Answer: "1Accepted AnswerThe Macie job will identify sensitive data depending on the identifier that has been used here. For eg: Lets assume that the mobile numbers are of a particular geographic location (India) , then you will have to validate whether the managed identifiers used are for Indian phone numbers are not. If there are no such identifiers in the search list of Managed Identifiers then you will have to create a custom managed identifier instead of using managed ones. You can use regex pattern  to match the pattern the +91 based mobile numbers, then the job will be able to precisely create findings .For eg. Create a custom identifier for +91 mobile number ^((+91)?|91)?[789][0-9]{9}, and attach this custom data identifier instead of AWS managed identifier and run the job.CommentSharetatahari_awsanswered 20 days ago"


Question: Silent Install Appstream Client

Is there a silent install switch for "AppStreamClient.exe"?  ThanksFollowComment

Answer: 0The Enterprise Deployment Tool (bottom right of the page)  for AppStream 2.0 contains an MSI that you can use via standard software deployment mechanisms to deploy to client computers.  There is also group policy templates as well for configuration.CommentShareRobert_Fanswered 10 months ago