Question: Greengrass Stream Manager | Missing Messages When Retrying Kinesis Export

"IntroI'm verifying the retry capability of aws.greengrass.StreamManager == 2.1.1 and noticed some messages was missing during retries.SetupDiagram below depicts what I'm trying to do:I'm running greengrass on a device that has internet access through ethernet. My custom component is appending to a StreamManager's message stream ~400 bytes of data every 15 seconds. The message stream is configured with kinesis export like so:kinesis_export = ExportDefinition(            kinesis=[KinesisConfig(identifier="KinesisExport" + self.modbus_data_stream_name, kinesis_stream_name=kinesis_stream_name, batch_size=5)]        )client.create_message_stream(            MessageStreamDefinition(                name=self.modbus_data_stream_name, strategy_on_full=StrategyOnFull.OverwriteOldestData, export_definition=kinesis_export            )        )The aws.greengrass.StreamManager is running version 2.1.1 and all default configurations.On my PC, I'm running a script using boto3 to continuously read data from kinesis. As the program is written, whenever it reads a record from Kinesis, it prints two things: a) the current timestamp and b) the timestamp when the data was generated from the device. The script is attached to the end of the post.ExperimentationNominal RunI ran my device with ethernet connected in parallel with the script on PC and the log appears as shown below.A few observations that's consistent with my expectations:Each Record has ~ 5 messages (because batch size = 5)Time gap between each message is ~15 seconds (because messages appended to stream every 15 seconds)Records are hitting Kinesis every ~75 seconds (which I believe came from batch size [5 msgs] x append rate [15 second/msg] )The format of the log is such:Found record for MyDevice in Kinesis at [time when PC received record in YY-MM-DD_hh:mm:ss][time when message was created on device in YY-MM-DD_hh:mm:ss]Fault InjectionThen I unplugged the ethernet from the device, effectively disconnected the device from the internet.As I expected, the PC no longer receives any record from the cloud during this time.ResumeI then re-connected the ethernet to the device. ~4 min later, I observed a surge of records hitting the PC program that has message time consecutive to when the internet was disconnected, which means the previously failed kinesis exports are being retried (Good!).However, I'm also observing large time gaps (relative to 15 seconds expected time gap) between the retried messages indicating that some messages are missing.In the log I captured below, the internet was disconnected at ~22-12-14_09:33:xx and resumed at ~22-12-14_10:28:xxI marked the time gaps with !!! manually for better visibilityI noticed the following gaps in the device timestamps:a. 22-12-14_09:34:54 to 22-12-14_09:37:39 - 2 minutes 45 seconds gap = 11 messages missingb. 22-12-14_09:39:55 to 22-12-14_09:41:25 - 1 minute 30 seconds gap = 6 messages missingc. 22-12-14_09:44:56 to 22-12-14_09:51:27 - 6 minutes 31 seconds gap = 26 messages missingFound record for MyDevice in Kinesis at 22-12-14_09:32:2722-12-14_09:31:23Found record for MyDevice in Kinesis at 22-12-14_09:32:2722-12-14_09:31:38Found record for MyDevice in Kinesis at 22-12-14_09:32:2722-12-14_09:31:53Found record for MyDevice in Kinesis at 22-12-14_09:32:2722-12-14_09:32:08Found record for MyDevice in Kinesis at 22-12-14_09:32:2722-12-14_09:32:24Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:32:39Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:32:54Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:33:09Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:33:24Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:33:39Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:33:54Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:34:09Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:34:24Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:34:39Found record for MyDevice in Kinesis at 22-12-14_10:32:54!!! 22-12-14_09:34:54Found record for MyDevice in Kinesis at 22-12-14_10:32:54!!! 22-12-14_09:37:39Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:37:54Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:38:10Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:38:25Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:38:40Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:38:55Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:39:10Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:39:25Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:39:40Found record for MyDevice in Kinesis at 22-12-14_10:32:54!!! 22-12-14_09:39:55Found record for MyDevice in Kinesis at 22-12-14_10:32:54!!! 22-12-14_09:41:25Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:41:40Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:41:55Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:42:10Found record for MyDevice in Kinesis at 22-12-14_10:32:5422-12-14_09:42:25Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:42:40Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:42:55Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:43:10Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:43:25Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:43:40Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:43:56Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:44:11Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:44:26Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:44:41Found record for MyDevice in Kinesis at 22-12-14_10:32:55!!! 22-12-14_09:44:56Found record for MyDevice in Kinesis at 22-12-14_10:32:55!!! 22-12-14_09:51:27Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:51:42Found record for MyDevice in Kinesis at 22-12-14_10:32:5522-12-14_09:51:57ContinueI let the PC program keep running for another 5 hours thinking if the StreamManager would retry the missing messages again, but I did not see any trace of them.QuestionBased on my observation, aws.greengrass.StreamManager does not reliably retransmit all previously failed-to-export data to Kinesis. Did I misunderstand the capability of StreamManager? Or is this a bug?Kinesis Read Scriptimport logging as loggerfrom botocore.exceptions import ClientErrorimport boto3from datetime import datetimeclass KinesisStream:    """Encapsulates a Kinesis stream."""    def __init__(self, kinesis_client):        """        :param kinesis_client: A Boto3 Kinesis client.        """        self.kinesis_client = kinesis_client        self.name = None        self.details = None        self.stream_exists_waiter = kinesis_client.get_waiter('stream_exists')    def get_records(self, max_records):        """        Gets records from the stream. This function is a generator that first gets        a shard iterator for the stream, then uses the shard iterator to get records        in batches from the stream. Each batch of records is yielded back to the        caller until the specified maximum number of records has been retrieved.        :param max_records: The maximum number of records to retrieve.        :return: Yields the current batch of retrieved records.        """        try:            response = self.kinesis_client.get_shard_iterator(                StreamName=self.name, ShardId=self.details['Shards'][0]['ShardId'],                ShardIteratorType='LATEST')            shard_iter = response['ShardIterator']            record_count = 0            while record_count < max_records:                response = self.kinesis_client.get_records(                    ShardIterator=shard_iter, Limit=10)                shard_iter = response['NextShardIterator']                records = response['Records']                logger.info("Got %s records.", len(records))                record_count += len(records)                yield records        except ClientError:            logger.exception("Couldn't get records from stream %s.", self.name)            raise    def describe(self, name):        """        Gets metadata about a stream.        :param name: The name of the stream.        :return: Metadata about the stream.        """        try:            response = self.kinesis_client.describe_stream(StreamName=name)            self.name = name            self.details = response['StreamDescription']            logger.info("Got stream %s.", name)        except ClientError:            logger.exception("Couldn't get %s.", name)            raise        else:            return self.detailsdef loads_compressed_data(compressed_data: bytes) -> dict:    import zlib    import json    return json.loads(zlib.decompress(compressed_data).decode())def read_kinesis(DeviceThingName):    while True:        for records in kinesis.get_records(10):            if records:                for r in records:                    data = loads_compressed_data(r['Data'])                    if data["device_id"] == DeviceThingName:                        print(f"Found record for {DeviceThingName} in Kinesis at {datetime.now().strftime('%y-%m-%d_%H:%M:%S')}")                        ts = data['measurement_ts']                        measurement_ts = datetime.fromtimestamp(ts).strftime("%y-%m-%d_%H:%M:%S")                        print(f"{measurement_ts}")# Remember to update credential using aws-cli configuresession = boto3.Session(profile_name="ecotec-dev", region_name='us-west-2')client = session.client('kinesis')kinesis = KinesisStream(client)# need to describe kinesis before getting recorddetails= kinesis.describe("raw_data")print(f"details {details}")read_kinesis("Moxa-UC-8112A-TBBCB1098353")Edit Regarding ochoanel's comment on 12/15/2022Thanks for your follow-up and suggestions.Changes I Made for Better DebugI added the read_message API to my device code to confirm that my message is successfully written to the StreamManger persistent files.Here's a snippet of code I added:The measurement_ts is the timestamp of when the message is created.I also changed the rate of append_message to every 5 seconds instead of 15, just to speed up debug.Logs CollectedI started my custom component and the PC program ~12:30, then I unplugged the ethernet ~12:33:20 for ~10 minutes.The following is a part of the device log (I'm not able to paste it all here due to text limit)Based on the logs, my messages are indeed hitting the StreamManager.However, I'm still seeing missing messages on my PC program. Which could be caused by any of the following:StreamManager did not export the data to KinesisMy PC program did not read all data that reached KinesisI'm honestly suspecting No.2 to be the reason... I'm working on a new tool to replace my PC program.FollowComment"

Answer: "1Accepted AnswerHey ictwist,Thanks for the detailed explanation. We will have to look further into this issue to determine if it is a bug on StreamManager or not. For us to debug the issue, could you provide:Anything you see on the logs between and around the times there are gaps(If possible and you are not exposing any private/proprietary information) provide the code on your component (you posted a few lines but having the full code could help us try to reproduce it on our end more easily)It seems like your component is reading from a controller using modbus (this is just a guess based on modbus_data_stream_name), is there a chance the component fails to post new messages to the stream manager? (seeing what the custom component is doing would help). One thing you could do in the meantime is add on your custom component a call to client.read_messages and log what it returns. When a stream message gets appended to a stream manager stream, it gets stored into disk before being exported to kinesis, that way we could tell if there is one issue with:An issue with the custom component failing to post messages to the local stream manager stream on the deviceMessages not being saved by the stream manager to diskFailures exporting the messages to KinesisCommentShareochoanelanswered 6 months agoictwist 5 months agoThank for your follow-up. please see my response in Edit section of the original post. I think the problem might have been my PC program.Shareictwist 5 months agoJust to follow up, I was able to verify the data in Kinesis in a different way - using lambdas and cloudWatch - and I found all the "missing" messages. I will continue to soak test StreamManager but you may treat this post as a false alarm at the moment.Shareochoanel 5 months agoNice to hear that, and happy to help. Keep us posted if you find any issues. Good luck!ShareMichaelDombrowski-AWS EXPERT5 months ago@ictwist, your script to read from Kinesis only read a single shard in Kinesis. You may have configured more than 1 shard or autoscaling which may create more than one shard. Stream manager will write to shards randomly to distribute the load and increase maximum throughput. I'd suspect this is the problem given that you were able to find the data using other means.Shareictwist 4 months ago@MichaelDombrowski-AWS You're correctShare"


Question: Is it not possible download all files from one S3 bucket?

"Hi,just started on AWS.Created a S3 bucket with couple of files (png & mov) earlier.Now I wanted to download all files, butdownload button only available if I select one file only.Am I not able to download all files in the bucket at once?Thanks,AxelFollowComment"

Answer: "2Hello,to download an entire s3-bucket/all files you have to use the cli with the s3 sync command:aws s3 sync s3://my_bucket_xyBut for that you need to install the awscli on your local computer and also create an access key for your user to connect from your cli to AWS:Install AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.htmlConfigure CLI to connect to your AWS Account: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.htmlAfterwards you should be able to use the as s3 sync command.A faster approach is probably by directly using the AWS Cloudshell (the little cli-symbol on topright of your aws console) and run the s3 sync command there into a specific folder:aws s3 sync s3://my_bucket_xy /bucketcontent... then zipping the downloaded content.zip -r zipped-bucketcontent.zip /bucketcontent... then download the zip directly in the AWS Cloudshell to your local computer -> Actions -> Download File -> Enter the filepath of zipped-bucketcontent.zipfor more infos see here: https://docs.aws.amazon.com/cloudshell/latest/userguide/multiple-files-upload-download.htmlCommentShareHeikoMRanswered 6 months ago"


Question: Block read access to a branch in a CodeCommit Repository

"I have created a repository in CodeCommit. It has one master branch and a few other branches such as module-1, module-2, module-3, module-4, and module-5. I have created 3 groups(org-1, org-2, org-3) and each group has 10 users. Now I am trying to figure out to achieve the following level of restriction.module-1 - org-1 users only - view and modifymodule-2 - org-2 users only - view and modifymodule-3 - org-3 users only - view and modifymodule-4 - org-1 users and org-2 users only - view and modifymodule-5 - all group users can access - read-onlyI am trying to enforce policies but can not achieve this. I can block write access successfully, but when it comes to blocking read access for a particular branch and making a branch invisible for a certain user group, cannot achieve that.  Can you please help me with this scenario? Thanks in advance.FollowComment"

Answer: "0Hello,Review the IAM actions for CodeCommit [1] and you will see access control at the branch level is limited to write actions only.  Unfortunately, read actions are limited to just the repository and cannot get more granular.  If you wish to control read access, it will be necessary to break out the branches to separate repositories.[1] https://docs.aws.amazon.com/service-authorization/latest/reference/list_awscodecommit.htmlCommentShareJustin Ganswered 9 months ago"


Question: Documentation on encryption context in contradiction with the behaviour?

"The AWS documentation on encryption context ( https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context ) states that:"When an encryption context is provided in an encryption request, it is cryptographically bound to the ciphertext such that the same encryption context is required to decrypt (or decrypt and re-encrypt) the data. If the encryption context provided in the decryption request is not an exact, case-sensitive match, the decrypt request fails."In our case we have an SQS queue with encryption enabled and a lambda function triggered by messages arriving to the queue. In CloudTrail events for GenerateDataKey the encryption context contains the key "aws:sqs:arn" as expected:"encryptionContext": {    "aws:sqs:arn": "arn:aws:sqs:eu-west-1:accountnr:queuename"},However, Decrypt events in CloudTrail contain a very different encryption context:"encryptionContext": {    "aws:lambda:FunctionArn": "arn:aws:lambda:eu-west-1:accountnr:function:functionname"}So, the contexts do not contain the same key in both cases, even though you would expect so, based on the above quoted sentence from AWS documentation. Also you would expect decrypting to have failed due to differing context but it seems to work just fine.Presumably I cannot use the same KMS key policy condition for readers and writers in this case to verify that "aws:sqs:arn" contains a specific value since the Decrypt context does not contain such key (?).Did I misunderstand the documentation sentence or why does it seem to work differently?FollowComment"

Answer: "0Can you check to see if there is another CloudTrail event at a later time for decrypt using the encryption context you expect? It could be that you are also encrypting lambda environment variables and that is the event you are viewing.SQS has a data key reuse period - The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again. An integer representing seconds, between 60 seconds (1 minute) and 86,400 seconds (24 hours). The default is 300 (5 minutes).You won’t see the decrypt from SQS right after the GenerateDataKey, since it still has the key cached locally based on the data key reuse period.CommentShareAWS-User-1849807answered 4 years ago0You are right, I should have checked the resource ARNs of the CloudTrail Decrypt events. The KMS key of the decrypt events with an unexpected encryption context is not the KMS key we use for SQS encryption. I was able to find a Decrypt event that has the expected encryption context (refers to our SQS queue) and the resource ARN of the correct KMS key. The majority of the decrypt events are of the wrong kind (i.e. decrypting environment variables), so every event I checked happened to be one of those. Thank you for your help!Edited by: tsillan on Aug 8, 2019 11:57 PMEdited by: tsillan on Aug 8, 2019 11:59 PMCommentSharetsillananswered 4 years ago"


Question: Unable to receive email using Amazon Work Email but can send

I am attempting to receive email for schedulingcoordinator@carbtest.org but emails never arrive even when replying to one sent from that address. The Domain was purchased through Route 53 and I believe everything is et up right.I can send emails fine but do not receive any bounce messages or errors. Nothing ever arrives. Exisitng answers did not resolve my issue.FollowComment

Answer: "0Hi,I'm sorry to hear you're experiencing problems with receiving mail. I took a look your domain and the MX records look correct. I checked with the service team if they could do a quick look at your organization. They found your domain to be missing its inbound rule in SES. This means when we receive a mail for your domain we do not know where to deliver it.You can correct this by adding your domain again in the WorkMail console. This will trigger a workflow to correct any problems with the required settings for a domain. You do not have to remove domain first.Kind regards,RobinCommentShareMODERATORrobinkawsanswered 3 months agorePost-User-0835898 3 months agoRe adding the domain resolved the issueShare"


Question: Redis connection timeout...

"Hi,Trying to train a deep racer using the vanilla github setup (including redis setup in training_worker.py). Redis sets up fine every time but I get a redis connection error when the simulation complete the first episode of 20 runs. Sagemaker start and Robomaker errors logs below. I have searched high and low to find out what is going wrong and no luck - any assistance appreciated!Excerpt of Sagemaker cloudwatch logs on start up related to redis:22:14:50 Invoking script with the following command:22:14:50 /usr/bin/python training_worker.py --RLCOACH_PRESET deepracer --aws_region us-east-1 --s3_bucket sagemaker-us-east-1-158249709041 --s3_prefix rl-deepracer-sagemaker-190413-22044422:14:51 211:C 13 Apr 2019 22:14:51.317 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo22:14:51 211:C 13 Apr 2019 22:14:51.317 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=211, just started22:14:51 211:C 13 Apr 2019 22:14:51.317 # Configuration loaded22:14:51 211:M 13 Apr 2019 22:14:51.319 * Running mode=standalone, port=6379.22:14:51 211:M 13 Apr 2019 22:14:51.319 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.22:14:51 211:M 13 Apr 2019 22:14:51.319 # Server initialized22:14:51 211:M 13 Apr 2019 22:14:51.319 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.22:14:51 211:M 13 Apr 2019 22:14:51.319 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.22:14:51 211:M 13 Apr 2019 22:14:51.319 * Ready to accept connections22:14:56 Redis server started successfully!22:14:57 Uploaded IP address information to S3: 172.31.27.217Excerpt from robomaker cloud watch logs where it bombs is below (note I've customized some of the messages):22:17:27 Step No=7.00 Step Reward=8.1870 On track?=True Dist from centre=0.02 Frame Progress=0.0000 Total Progress=6.6870 To Finish=93.31 X, Y, Z=2.17, 0.60, 0.0022:17:27 Action sent to Ros/Gazebo...22:17:27 Message recieved from Ros/Gazebo... on_track=True22:17:27 Step No=8.00 Step Reward=8.1870 On track?=True Dist from centre=0.02 Frame Progress=0.0000 Total Progress=6.6870 To Finish=93.31 X, Y, Z=2.19, 0.60, 0.0022:21:49 Traceback (most recent call last):22:21:49 File "/home/robomaker/workspace/bundle-store/b56d2ca1-05a5-4aa6-b6d2-84b6766f0d8b/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 484, in connect22:21:49 sock = self._connect()22:21:49 File "/home/robomaker/workspace/bundle-store/b56d2ca1-05a5-4aa6-b6d2-84b6766f0d8b/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 541, in _connect22:21:49 raise err22:21:49 File "/home/robomaker/workspace/bundle-store/b56d2ca1-05a5-4aa6-b6d2-84b6766f0d8b/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 529, in _connect22:21:49 sock.connect(socket_address)22:21:49 TimeoutError: Errno 110 Connection timed outFurther down the error log...22:21:49 File "/home/robomaker/workspace/bundle-store/b56d2ca1-05a5-4aa6-b6d2-84b6766f0d8b/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 489, in connect22:21:49 raise ConnectionError(self._error_message(e))22:21:49 redis.exceptions.ConnectionError: Error 110 connecting to 172.31.27.217:6379. Connection timed out.22:21:50 ================================================================================REQUIRED process agent-9 has died!22:21:50 process has died pid 133, exit code 1, cmd /home/robomaker/workspace/bundle-store/b56d2ca1-05a5-4aa6-b6d2-84b6766f0d8b/opt/install/deepracer_simulation/lib/deepracer_simulation/run_rollout_rl_agent.sh __name:=agent __log:=/home/robomaker/.ros/log/8bfee468-5e39-11e9-a84a-0242a9fe0103/agent-9.log.22:21:50 log file: /home/robomaker/.ros/log/8bfee468-5e39-11e9-a84a-0242a9fe0103/agent-9*.log22:21:50 Initiating shutdown!22:21:50 ================================================================================22:21:51 agent-9 killing on exitEdited by: axb2035 on Apr 15, 2019 2:53 AMEdited by: axb2035 on Apr 15, 2019 2:56 AMFollowComment"

Answer: "0So here three possibilities I thought of after reading up about redis:It's a permissions problem between sagemaker/robomaker.The robomaker machine doesn't have the right information to connect to redis.The information transfer to redis breaches some default redis setting.Any further ideas/suggestions or info or thoughts on the theory above?Thanks in advance!Edited by: axb2035 on Apr 16, 2019 3:01 PMCommentShareaxb2035answered 4 years ago0As it's a timeout, it's likely to be a security group issue with the robomaker instance unable to connect to that port. The default script in https://github.com/awslabs/amazon-sagemaker-examples/blob/master/reinforcement_learning/rl_deepracer_robomaker_coach_gazebo/rl_deepracer_coach_robomaker.ipynb does setup all the network routes for you. Something could be going wrong in the assumptions that script makes. It tries to do the routing through a VPC and you should be able to look at that configuration. You could try to setup two tiny EC2 instances inside the VPC and see if they can communicate on the ports.As for why it's timing out after steps are being made, robomaker (ROS in the background) uses Redis to sync the two tensorflow neural networks so it can do clipped proximal policy optimisation. The robomaker instance is doing a bunch of training on one network, and then sends it to Sagemaker through Redis (maybe S3 as well) to do the policy training. So robomaker doesn't immediately need Redis so it tries to a couple steps and episodes before the timeout is triggered.CommentSharecrr0004answered 4 years ago"


Question: Use built-in Target Based Auto Scaling of Fleet when an outside service (not custom) is managing sessions and matchmaking

"Use case: Trying to implement GameLift managed fleet (✔) with built-in auto-scaling (❌) when using another service initially for session management and matchmaking.TL;DR:  I'm looking for effectively a Server SDK CreateAndActivateGameSessionFromMe() call that will allow GameLift to know a session is active so that it can do Target Based Auto-Scaling based on % Available Sessions.As phase 1 of implementing GameLift, we are looking to keep the existing service for session management and matchmaking.  I have this working on a Fleet with manual scaling.  The sessions all register with the existing (non-custom) service and sessions are found and joined via mechanisms on that service.  GameLift Fleet very efficiently keeps the target number of servers and game processes per server running.I would like to add the Target Based Auto-Scaling but I don't think it's possible without abandoning the other service entirely.  This might be an option in the long run, but being able to just add the auto-scaling would speed up the initial GameLift adoption.GameLift Target Based Auto-Scaling is based off available sessions.  This seems then to require at the very least the use of the Game Session Management layer in addition to Infrastructure.  Looking at https://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-sdk-server-api-interaction-vsd.html.If there is a mechanism where the loaded game process upon activation by the other service could register with GameLift as active, it would allow use of the Target Based Auto-Scaling for % available sessions.  But I can't find a way to do that.CreateGameSession() as shown in the docs and the Server API interaction diagram linked above tells GameLift to find an available session in the Fleet.  I can't specify a session to activate.  I'm looking for effectively a Server SDK CreateAndActivateGameSessionFromMe() call.The Server SDK call ActivateGameSession() will return an error about the process not having an associated Game Session ID if I call it not in response to CreateGameSession()/onActivateGameSession().Is there an option or do I need to implement custom auto-scaling by monitoring the game sessions and then modifying desired count in the fleet to auto (but not GameLift auto)-scale?FollowComment"

Answer: "2Accepted AnswerHi,GameLift AutoScaling supports:Target-based autoscaling (on PercentAvailableGameSessions metric)andRule-based autoscaling (on a variety of different metrics)See https://docs.aws.amazon.com/gamelift/latest/apireference/API_PutScalingPolicy.html for more information.However, all of these metrics depend on GameSessions / PlayerSessions on the fleet being registered with the GameLift Service. If you are unable to do that, then it will not be possible to make use of GameLift Autoscaling.Currently, there is no way to Create a GameSession & Activate it from the GameLift Server SDK alone. You are correct that, at minimum, you must make use of the CreateGameSession() -> ActivateGameSession() workflow in order to create the necessary metrics (e.g. PercentAvailableGameSessions) to make autoscaling work.In summary, you will not be able to make use of GameLift Autoscaling if you are integrating without calling one of GameLift's session placement APIs (i.e. CreateGameSession, StartGameSessionPlacement, StartMatchmaking), and will likely have to go with your proposed custom auto-scaling approach.CommentShareNathanDanswered 7 months agoEXPERTToni_Sreviewed 6 months agomjans71 7 months agoThanks for your response.  This saves me time trying to make something work that won't work :)Share"


Question: Define AWS database to use in Custom SQL?

"I am creating a dataset in AWS Quicksight using custom SQL which I prepare/test in Athena. However, unless I define each join/table "databasename".table, the QS custom SQL fails. I have tried the below but it has failed. Is it possible to instruct the query to fun against a specific DB at the beginning of the query?USING AwsDataCatalog."databasename"FollowComment"

Answer: "0Thank you for posting the question and the detailed explanation on what you are expecting.At present, it is not possible to instruct a query to run against a specific database at the beginning of the query. I have raised a feature request with the service team on your behalf. While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our What's New and Blog pages for any new feature announcements.CommentShareSUPPORT ENGINEERSanket_Kanswered a year ago"


Question: Install the latest python (3.8) on EC2

"I'm trying to install the latest python version (today = 3.8) onto Amazon EC2 Ubuntu 18.04I don't understand why doing sudo apt-get install python3 I only have 3.6.8Using other than Ubuntu 18.04 but Amazon AMI 2, I have 3.7.4I haven't found anything else on the internet discussing about it. I'd like to use the latest python version available.Any idea how I should do to install this latest version on my EC2 Ubuntu ?ThanksFollowComment"

Answer: "0Accepted AnswerHello @Eric75,I understand you are trying to run the latest version of python (3.8) on an Ubuntu AMI.Can try "sudo apt-get install python3.8"?When you run it use "python3.8 function.py"Hope this is helpful.CommentSharepiboawsanswered 4 years ago"


Question: S3 contains new html file but browser shows old html

"Hello,I have a site example.com that I deliver through cloudfront and an s3 bucket. I accessed a page called example.com/login.html on my browser, then updated the login.html file in a text editor, deleted the original login.html on s3, uploaded the new login.html to s3, cleared my browser cache and refreshed the page, but the browser showed the old login.html file.I checked the view page source and the old html was there.When I downloaded the login.html from s3 it was the new login.html file.I checked this on multiple browsers, and they all rendered the old html.How can s3 contain the new html file but the browsers show the old html?FollowComment"

Answer: "1Accepted AnswerDoes your website use CloudFront for CDN?If used, CloudFront may cache html.Try deleting the cache using the method in the following document.https://repost.aws/knowledge-center/cloudfront-clear-cacheCommentShareEXPERTRiku_Kobayashianswered a month agoJeff F a month agoIt does; thank you this workedShare"


Question: AppSync authorization

"I currently have APIs which I am currently implementing using AWS Appsync and Cognito as authorizer. I have a couple of APIs that should be used without authentication (before the user registration) and I thought about using API KEYS with appsync for that purpose ()But I noticed that: "API keys are configurable for up to 365 days, and you can extend an existing expiration date for up to another 365 days from that day"So the key should last 2 years maximum (https://docs.aws.amazon.com/en_us/appsync/latest/devguide/security.html#api-key-authorization) . Can you please suggest us the best way to manage the api key change? My app clients are ios/android apps using amplify framework, and I would like to avoid to create a release dedicated to the api key change every 1/2 year.My questions:is API keys the best way to address this use case?how to effectively manage the API keys renewal process?FollowComment"

Answer: "0Accepted AnswerIf you don't want to manage API Key rotation, use Cognito Unauth Roles and IAM auth on AppSync. Easily configured using the Amplify CLI and the GraphQL Transform: https://aws-amplify.github.io/docs/cli-toolchain/graphql#public-authorization# public authorization with provider overridetype Post @model @auth(rules: [{allow: public, provider: iam}]) {  id: ID!  title: String!}CommentShareawsedanswered 4 years ago"


Question: RDS DB Instance Backup

"For DB instances we taken regular auto back up, &  checked AWSHealth dashboard it show DB optculturedbv2rr  last update current date, but cloud not found his sanpshot,Cloud you please figure out me.FollowComment"

Answer: 0AWS Support can help you resolve such specific issue that requires investigation into your account. Please submit a support case from within your AWS account. Thanks.CommentShareGovardhanan_Ranswered 8 months ago


Question: New top-level domain suggestion

Please add yoga as a new top-level domain. I own a domain with godaddy but not able to move that to AWS. Thanks!FollowComment


Question: instance is stopping for about 25 mins

"After we have received email from amazon about scheduled retirement date, we decided to stop and start my ec2 instance.After clicking stop, it is still stuck on the stopping status for a long time about 25 mins. Shall we wait for the instance to stop? or do we required to do anything else?We don't want the data to be lost.FollowComment"

Answer: 0we did force stop and everything looks normal now.CommentSharethamaraiselvamanswered 4 years ago


Question: Why SES supports disrespect customers?

"I sent request to production access, discribed all information following SES Dodcs, verified domain, spend my time and what i got?Deny, because my SES usage can make "negative impact", i have more than 800k followers in Instagram, 130k on youtubue and SES support telling me, that i can make negative impact.I've read a lot of negative comments about SES production access, but when i tried it by myself, it's really terrible. How new users can start use SES, if SES support dont give a chance?FollowComment"

Answer: "0First of all for testing ses you can use test email with sandbox using verified identities. For production access, you need to share proper answer for below questionaries to AWS.What is the nature of your business, and how do you plan to use Amazon SES to meet the needs of your business?How do you collect the email addresses that are on your mailing list?How do your subscribe and unsubscribe processes work?How do you handle bounces and complaints?What type of email (for example, transnational notifications, marketing content, or system notifications) do you plan to send with Amazon SES?What is the URL for SES service?But remember after production access given , if the bounce rate and complaint rate goes above the limit then your ses will be disabled.CommentSharesouravanswered a year ago"


Question: FlexMatch long matchmaking time

"We are using the following matchmaking ruleset with a standalone configuration{    "name": "MatchmakingRuleset",    "ruleLanguageVersion": "1.0",    "algorithm": {        "backfillPriority": "high",        "batchingPreference": "random",        "strategy": "exhaustiveSearch"    },    "playerAttributes": [        {            "name": "tournamentDefinitionID",            "type": "string"        },        {            "name": "skill",            "type": "number",            "default": 0        }    ],    "teams": [        {             "name": "player",            "minPlayers": 0,            "maxPlayers": 1,            "quantity": 4        }    ],    "rules": [        {            "name": "SameTournament",            "description": "Match players to play in the same tournament type",            "type": "comparison",            "operation": "=",            "measurements": ["flatten(teams[*].players.attributes[tournamentDefinitionID])"]        }    ]}Since minPlayers is 0 for all teams, that means that when there are no other players matchmaking at the moment and no backfilling tickets, players can get a successful match with just their own ticket, in which event we create a new game, and the player enters this game alone. We then call startMatchBackfill with the same configuration and repeat the process until the game is full.This mean that at any given moment, any submitted ticket "should" always result in a MatchmakingSucceeded callback, so theoretically there should be no waiting for matchmaking.For some reason we are experiencing wait times of 5-10 seconds from when we call startMatchmaking until our matchmaking event listener lambda is called (wired through EventBridge), is there any way to speed this up? Is this a technical constraint? Is FlexMatch waiting for more tickets to enter the pool, and is there a way to configure this?FollowComment"

Answer: "0Hello, did you root out the possibility of lambda cold start? Or can you provide some ticket ids & rough time & region so we can take a closer look?CommentSharerePost-User-5046723answered a year ago0Had a very busy couple of weeks here, thanks for responding and we'll appreciate if you'll take a look.Ticket ID:DefaultTournamentDefinition-3--7caa6778-e1d9-4d0a-a246-cc3b77a75a4cRegion:eu-central-1unix seconds of startMatchmaking returned success1646420747our lambda was invoked by event bridge for "MatchmakingSearching" and "MatchmakingSuccess" at unix seconds1646420755Regarding the possibility of a lambda cold start, I'm not seeing why a cold start would take 8 seconds in our case. Also, we can have a ticket take this amount of time followed by multiple tickets that take this amount of time. Also there are no long running / network operations happening in the matchmaking event listener's global scope.CommentSharerePost-User-7390418answered a year ago0Just tried using SNS -> SQS -> Lambda for matchmaking notifications with similar results. Always a delay of at least about 5 seconds before receiving even the first "MatchmakingSearching" event after calling startMatchmaking.CommentSharerePost-User-7390418answered a year ago0Hey @REDACTEDUSERP61064647CommentSharerePost-User-4754183answered a year ago"


Question: Is there a solution for multi-user Notebook on SageMaker?

Is there a solution for multi-user Notebook on Studio Notebook or Notebook Instances? Eg if we want several developers to interact on the same notebook at the same timeFollowComment

Answer: "0Accepted AnswerNotebook instances are not connected to the user. So if two users has the same access rights they will see and will be able to access the same instance (even in the same time).The issue is - Jupyter Notebook is not ready for that, both users will have the same privileges, no tracking who did what, ... And working on the same notebook on the same time - basically they will overwrite each other saves.I had a need for similar thing (pair programming - data scientist and software engineer) - the only viable solution we were able to find was desktop sharing (like TeamViewer, ...)CommentShareAWS-User-3214020answered 3 years ago0SageMaker Studio now supports team-based sharing and real-time collaboration. You can create shared spaces to access, read, edit, and share the same notebook in real time. For more information refer Next Generation SageMaker Notebooks – Now with Built-in Data Preparation, Real-Time Collaboration, and Notebook AutomationCommentSharemarancsanswered 3 months ago"


Question: ECS Fargate service with load balancer not working

"I have been stuck on this for 3 days.I have a fargate service running in ECS. It is running a gunicorn app on port 80. I can see from the logs, the server starts:[2023-04-04 20:51:39 +0000] [1] [INFO] Listening at: http://0.0.0.0:80 (1)I have a load balancer, listening at port 80 with a target group.My issue is I cannot seem to access to load balancer. Get the following:<!doctype html><title>404 Not Found</title>
Not Found

The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
I can confirm that:My load balancer is exposed to the internet.My security group for ECS has inbound rule to allow traffic from my load balancer.I must have tried about 15 times. I cannot seem to make it work.FollowComment"

Answer: "1Accepted AnswerI believe that given that the http "404" error is an application error is not related to load balancer or other network configuration. If you are getting 404 error you are actually reaching the application, however the application do have the web page you are trying to display. My suggestion is to double check your docker image configuration, you are probably having some issue on how you build you image.To make the debugging faster, you could run the container in your local environment, login into your container and verify that you are getting the same error performing the same API call. For example:wget http://localhost:80If you are getting the same error is going to be clear that the issue is within the docker image itselfCommentShareSimone Tallevianswered 2 months agoEXPERTTushar_Jreviewed 2 months agorePost-User-4895466 2 months agoYou're absolutely right. 👏Share"


Question: Clarification on EC2 pricing

"Hi all,I'd like to use an instance with lots of vCPUs to benchmark a specific test I'm running locally.Let's assume I chose a compute instance, like c5.9xlarge at 1.818$/hour (or even more vCPUs like c5.24xlarge).If I create the instance, launch it, download and install some software (like gcc) and use it for one hour, and stop it, I will only pay 1.818 USD ?Do I need to pay if the instance is down ? If not, how long will it be kept ?Thanks for your help.FollowComment"

Answer: "0Hi martinjean,I believe if you do on-demand pricing you will only be charged the hour, but you will still be charged for any EBS volumes backing the VM (even if the VM is stopped).See: https://aws.amazon.com/premiumsupport/knowledge-center/ebs-charge-stopped-instance/ThanksCommentSharepaul10answered 4 years ago0Thanks Paul,So if I delete my instance, I'll only be charged for one hour I guess.CommentSharemartinjeananswered 4 years ago0Correct.CommentSharepaul10answered 4 years ago"


Question: AWS MSK Serverless CloudFormation support

"I'm interested in AWS MSK Serverless, seems exactly what i was interested in. However, so far i only see to create a serverless MSK cluster using the console. Is there some CloudFormation support for MSK serverless available or if not will it be coming soon?FollowComment"

Answer: "0Currently, there is no CloudFormation support for MSK serverless.CloudFormation Coverage Roadmap is available on GitHub, but MSK Serverless is currently in "Researching" status.https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/projects/1?card_filter_query=mskIt will be some time before it is supported.CommentSharehayao-kanswered a year ago"


Question: EBS. Size increasing

"Hi, I have a instance EC2 associate with a EBS volumen, this instance was stopped but the used disk size keeps increasing.I don't understand why the used volume size keeps increasing if the instance was stopped.Thanks.FollowCommentBrettski-AWS EXPERT2 months agoWhere are you seeing the EBS volume size increasing? Is that in the console?SharerePost-User-9133120 2 months agoIn Billing Dashboard ,  View your bill.Thanks.ShareAWS-User-2332965 2 months agoProbably the volume is being used over the network.Look at the security group rules to check what ports allow traffic. To confirm, you could also block all security rules and observe.Share"


Question: Neptune supports Neo4j GraphQL?

Neptune can use openCypher.. Neo4j GraphQL? Is this Library in use?https://neo4j.com/product/graphql-library/FollowComment

Answer: "0The best way to use GraphQL with an Amazon Neptune backend is to leverage AWS AppSync.  With AppSync, you don't have to manage an Apollo server and you can define your GraphQL schema and resolvers within the AppSync service.  We don't yet have the ability to automatically generate the resolvers for Neptune, so you'll have to manually create a resolver using AWS Lambda and the appropriate graph query using either Gremlin or openCypher (1).(1) https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-lambda-resolvers.htmlThere are some great examples of how to use AppSync with Neptune here:https://github.com/dabit3/cdk-appsync-neptunehttps://github.com/aws-samples/aws-appsync-calorie-tracker-workshophttps://github.com/jkahn117/aws-appsync-session-manager-neptuneCommentShareTaylor-AWSanswered a year ago"


Question: DynamoDB BillingMode in Lambda

"Hi there,We're creating DynamoDB tables from inside Lambda functions, and we'd like to use on-demand billing. However, the current version of the SDK is 2.290.0, and support for the DDB billing mode attributes was introduced in 2.365.0.What are the update plans for the lambda runtime environment SDK? Is this something that will arrive anytime soon, or do we need to look for workarounds (bonus points if anyone has a good workaround handy)?Thanks, and best regards,DennisFollowComment"

Answer: "0Facing same issue with some SNS api callsI'm going to have to include node_modules/aws-sdk which has latest SDK version in my deployment package from build server.Such a shame the Lamda SDK is so far behind!!CommentShareMark Lloydanswered 4 years ago0Okay, the NodeJS 10 runtime has been released, which runs on 2.437.Still, the slow releasing of the runtimes is generally an issue, making new features unavailable to Lambda users for months (it took 8 months to go from 2.290 to 2.437). Sure, I can publish the newer version with my code (or in a layer), but AFAIK that means a larger download, making cold starts much slower, and thus more expensive.Edited by: DennisDK on May 20, 2019 1:17 AMCommentShareDennisDKanswered 4 years ago"


Question: Invalid AWSPhone on AppSync

We are creating a record on AppSync with the following valid phone number: +65 8051 2187 but are returned with an invalid 'AWSPhone' value. Wouldn't the AWSPhone validation needs an update?FollowComment

Answer: "1I’m having the same issue. AppSync is returning an error when a field is of type AWSPhone and has correct payload.Here is the error message returned:"Variable 'phone' has an invalid value. Unable to parse +13451112222 as a valid phone number."The phone number +13451112222 is valid (I put 1112222 here to hide real one), and passes validation on many other phone validation libraries. Is just that the area code 345 is from Cayman Islands.CommentShareAWS-User-2047561answered a year ago"


Question: Pushing to SQS through VPC Endpoint from lambda in private VPC fails

"Hello!I have setup a private VPC where some lambda functions reside. Now I would like one of the lambdas to send messages to a SQS queue. For that I have created a VPC Endpoint to SQS.The policy for the VPC endpoint allows everything for every user.The policy for the SQS queue allows all SQS operations on the queue from everywhere.The policy on the lambda function allows all SQS operations.The Security Group of the VPC Endpoint allows all traffic on all protocols.Everything is on eu-central-1.The lambda that I want to send a messate to SQS times out here at the send_message call:def send_message():sqs_client = boto3.client("sqs", region_name="eu-central-1")_message = {"key": "value"}_  _response = sqs_client.send_message(_      _QueueUrl="https://sqs.eu-central-1.amazonaws.com/<user-id>/<queue-name>",_      _MessageBody=json.dumps(message),_      _DelaySeconds=12_  _)_  _print ("Response: ", response)_  (Of course I have substituted the user-id and queue-name with the actual ones).I have also tried all the DNS addresses that VPC Endpoint provides, but none work.What have I missed?FollowComment"

Answer: "0Hi,You should also have a look at the security groups for the Lambda function, and the Network ACL for the subnets that the function resides in.CommentShareAWS-User-5927523answered 2 years ago0As of what I have understood, you are not required to add specific routes to the VPC Endpoint in the private subnets to get it working. Also, the Security Group is allowing all traffic on all protocols and ports.The security group of the lambda function is allowing all outbound traffic.CommentShareDevoMarkyanswered 2 years ago0Oh, I had to set an inbound rule on the SG for the Lambda function that allows all traffic on all protocols and ports for the same SG that the VPC Endpoint had.CommentShareDevoMarkyanswered 2 years ago"


Question: Instance stalled "Stopping" on degraded hardware for > 45 min

"Hi!We received an "Amazon EC2 Instance Retirement" message a few hours ago, following the recommended course I've issued a "Stop" action on instance i-13748c7e  "User initiated (2019-04-18 00:20:01 GMT)"The instance appears to have stalled in the "Stopping" state (now over 45 minutes) - understandable if the underlying hardware is failing.Would it be possible to help instance i-13748c7e to come to a full stop? Thanks!FollowComment"

Answer: 0Is there some magic about posting? The instance is now stopped (at the 50-minute mark :-) Thanks!CommentSharefoursevenelevenanswered 4 years ago


Question: Automated streaming integration and multiple requests for SageMaker endpoint

"A data scientist  is looking to host a Tensorflow model in SageMaker and process low volume streaming event data (~2-3 per second) to collect inferences about each event. Data scientist is looking at having the SageMaker inference model plugged in as a Kinesis Data Analytics Application but Kinesis Data Analytics currently only supports SQL or Flink.One option to set up an ECS or Lambda service to consume data from Kinesis or SNS and invoke the SageMaker inference endpoint per message, but  if there is a more automated and optimal solution available for these kind of workflows.It is not possible to pass multiple requests currently to a SageMaker endpoint, yet Tensorflow models tend to perform much better on batches of data rather than multiple single invocations so some windowing would be beneficial. Ideally the client would want to react to an inference within 10-15 seconds of the event being processed so an S3 based batch approach is probably too slow.Is there anything you can recommend for handling this sort of workload?FollowComment"

Answer: "0Accepted AnswerTo build integration between SageMaker endpoints and Kinesis Data Application use this blog - https://aws.amazon.com/blogs/architecture/realtime-in-stream-inference-kinesis-sagemaker-flink/. It help  to setup serverless service to invoke the SageMaker inference endpoint.To  use batching. The Tensorflow documentation mentions the following:This link mentions that you can include multiple instances in your predict request (or multiple examples in classify/regress requests) to get multiple prediction results in one request to your Endpoint.This link mentions that you can configure SageMaker TensorFlow Serving Container to batch multiple records together before performing an inferenceYou would still have to handle the logic internally in ECS/Lambda to control how many records you consume from your stream in one batch, but at least you will be able to infer on the whole batch on the SageMaker endpoint end based on the above.CommentShareAnamaria_Tanswered 3 years ago"


Question: Site to Site VPN Issue

"I have configure VPN on-prem firewall to AWS site to site so my on-prem firewall showing both tunnel up but in My AWS showing status is down and IPSEC is up, please advise, we have poc. your prompt support will be highly appreciatedFollowCommentemned a year agoA bit more details on the error would be helpful.Share"

Answer: "0Hello, I see you are having issues with your Site-to-Site VPN connection.With AWS site to site VPN, when the status on your AWS console shows IPSEC up and Tunnel Down, this is an indication that IPSEC has been successfully established between the two peers. However, since this is a dynamic (BGP) VPN, the tunnel will come up only if BGP session is established.There are a number of issues that can affect BGP session establishment which include but not limited to the following: IKE security associations and the BGP peer IPs to mention some.To try resolve the issue, verify the BGP[1] configurations such as peer IP, ASN are correct or not. I have referenced documentation to help troubleshoot [2] [3][4] your issue since you have not mentioned a specific one .Be sure to check the Traffic Selectors encryption domain and confirm that is grants the BGP per IPs.Verify that your device has rules allowing BGP traffic, TCP on port 179 inbound and outbound to the AWS tunnel inside IPs.Also have a look at the status of the BGP and logs from your device, which helps analyse any errors on BGP.You can monitor your VPN connection using CloudWatch which will help monitor the state of your tunnel[5].You may also monitor the connections of your tunnel using AWS Health events, which you can configure to monitor what happens when you try to connect Site-to-Site[6].References:[1] https://docs.aws.amazon.com/vpn/latest/s2svpn/cgw-dynamic-routing-examples.html[2] https://aws.amazon.com/premiumsupport/knowledge-center/vpn-cgw-vpg-traffic/[3] https://aws.amazon.com/premiumsupport/knowledge-center/vpn-tunnel-instability-inactivity/[4] https://docs.aws.amazon.com/vpn/latest/s2svpn/Troubleshooting.html[5] https://docs.aws.amazon.com/vpn/latest/s2svpn/monitoring-cloudwatch-vpn.html[6] https://docs.aws.amazon.com/vpn/latest/s2svpn/monitoring-vpn-health-events.htmlCommentShareAntonioanswered a year agorePost-User-5722005 a year agoDear Support,Thanks for your reply, i am think to change Dynamic to Static routing, will it help me, also when i am trying to change the dynamic to static route its not showing me the option so could you please advise me how and from where i can change the DYnamic to Static route. ThanksShare"


Question: AppStream 2.0 Upgrade .NET Version on Image

"I have an application that needs .NET 4.8 in order to run. I have installed the prerequisite version of .NET on the ImageBuilder, but when I go to take a snapshot of that and use the image for a fleet its still showing as the previous 4.7 version installed on the base platform image. Is there something special that I need to do when snapshotting the ImageBuilder to make sure it includes the installed .NET version?Registry Editor for Fleet using snapshotted imageRegistry Editor for Image Builder**Edit When I launch to the Desktop instead of directly into an application, then launch the application it seems to be picking up on the correct .NET version now.ThanksMikeFollowComment"


Question: Environment stuck - Can't terminate - Please help

"Hello all.One of my enviroments is stuck. I tried to terminate it but it had a coupled DB on it, so it entenred on a "No data" state. Now I can't terminate because it says there is a coupled DB.I already deleted the DB but still says there is a DB coupled. I try to decouple and it says "Enviroment must be on a ready state".Everything I try it goes back to "Enviroment must be on a ready state" and if I try to terminate it says that there is a Coupled DB on it. It seems like a loop.I am not sure where I can find the ID of my enviroment but I guess this is the one: e-2r73inr7qpFollowComment"

Answer: "0Please raise a support ticket for this issue - re:Post is a great place to ask "how" and "why" questions but we don't have access to customer environments to troubleshoot.CommentShareEXPERTBrettski-AWSanswered a year ago0I would like to avoid paying 30 USD to (support subscription) to open a ticket. If there is a free way to do this please clarify with me.Thanks in advanceCommentShareJoseMuriloanswered a year ago0I would try doing Actions -> "Rebuild environment". And then maybe you can Terminate it after the rebuild.CommentSharejohnthussanswered a year ago0Thanks for the awner but I already tried "Rebuild environment" but it ends up saying that the eniroment must be in a ready state. It is like a loop hole, I can´t bring it online, I try to terminate and it says there is a coupled DB, I try to decouple the DB but it says the enriment it not ready, I try to rebuild and it says it is not ready, I am stuck.Will it be charged on my account even if the environment has no use?CommentShareJoseMuriloanswered a year ago"


Question: The problem when make ec2

"We have only one region ec2, Korea.but when we make ec2,It is not working with this message, "This account is currently blocked and not recognized as a valid account. Please contact aws-verification@amazon.com if you have questions."what is problem?FollowComment"

Answer: 1This is already answered here. Please check:https://repost.aws/questions/QUwnXvvsQyRzut3QE3ct3qOg/account-is-currently-blocked-when-trying-to-create-ec-2-instanceCommentShareSandeepVudataanswered a year ago


Question: Connect to Workspaces through Proxy Server

"I setup a Workspace which I would like to protect it by whitelisting IP. After I added my public IP (IP X) to IP Access Control Group, I am able to connect to my Workspace.I have a proxy server built using shadowsocks on Ubuntu. I connect my computer to this proxy server using ClashX, and get my public IP (IP Y).When I connect to the proxy server, I have to whitelist both IP X and IP Y before I can connect to Workspace. If I only whitelist only either X or Y, I will not be able to connect to Workspace.What could be the reason? Thanks.FollowComment"

Answer: "1Qinjie-There are two types of traffic for WorkSpaces.Streaming traffic(TCP and UDP ports 4172, 4175) - This needs to be proxy bypassed. Doesnt support proxying.Authentication traffic(TCP 443) - This can be proxied.When you are talking about IP Control Groups, you are talking only about authentication. You also need to make sure the streaming traffic is unblocked.I would first try WebClient https://clients.amazonworkspaces.com/webclient to check whether Public IP X and Public IP Y works with different IP Control Groups before testing Thick Clients.Hope it helps,-arun.CommentShareArun_PCanswered 7 months agoArun_PC 7 months agoAlso, checkout the post related to enabling IP control groups https://repost.aws/questions/QUyVhLbB_rQf6rGAfsKpf89Q/workspaces-ip-access-control-groupsShareqinjie 7 months agoThank you, ArunShare"


Question: Custom domain in Lightsail container service is not working

"I have created a new lightsail container service. Pushed the image, and created a deployment.It has given me a public domail url, this is working fine.I am trying to assign a custom domain. For this, under the custom domains tab I have created a new certificate for API.MYDOMAINHERE.COM. It asked me to verify that I am the owner of the domain by entering the CNAME record name and value, which I have done in my godaddy.com account. After about 2 hours I can now see that the certificate validation completed and then I attached the certificate. The certificate status is showing as valid, in use.At the top of the screen, below the public domain url, I can now see the custom domain url: API.MYDOMAINHERE.COMHowever, when I open the url in browser it says that: This site can’t be reachedThe public domain url that it provided by default (example....us-west-2.cs.amazonlightsail.com) works fine.What could be the issue?FollowComment"

Answer: "0Accepted AnswerI had not configured the CNAME pointing the public address to the custom domain. Once I configured that, it worked.CommentShareAWS-User-2948322answered a year ago0Please check your HTTPs vs HTTP. Check which port you are using. If you are using HTTPS you will need to have that certificate also uploaded into AWS ACM most likely.Further reading if HTTPS custom domain:https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-enabling-distribution-custom-domainsCommentShareYoungCTO_quibskianswered a year ago"


Question: "Multi-model, Multi-container and Variants - what are the possible combinations?"

"This question is mostly for educational purposes, but the current SageMaker documentation does not describe whether these things are allowed or not.Lets suppose I have:a XGBoost_model_1 (that needs a XGBoost container)a KMeans_model_1 and a KMeans_model_2 (both require a KMeans container)1. Here's the first question - can I do the following:create a Model with InferenceExecutionConfig.Mode=Direct and specify two cointainers (XGBoost and KMeans with Mode: MultiModel)That would enable the client:to call invoke_endpoint(TargetContainer="XGBoost") to access the XGBoost_model_1to call invoke_endpoint(TargetContainer="KMeans", TargetModel="KMeans_model_1") to access the KMeans_model_1to call invoke_endpoint(TargetContainer="KMeans", TargetModel="KMeans_model_2") to access the KMeans_model_2I don't see a straight answer in the documentation whether combining Multi-Model containers with Multi-container endpoint is possible.2. The second question - how does the above idea work with ProductionVariants. Can I create something like this:Variant1 with XGBoost serving XGBoost_model_1 having a weight of 0.5Variant2 with a Multi-container having both XGBoost and KMeans (with a MultiModel setup) having a weight of 0.5So that the client could:call invoke_endpoint(TargetVariant="Variant2", TargetContainer="KMeans", TargetModel="KMeans_model_1") to access the KMeans_model_1call invoke_endpoint(TargetVariant="Variant2", TargetContainer="KMeans", TargetModel="KMeans_model_2") to access the KMeans_model_2call invoke_endpoint(TargetVariant="Variant1") to access the XGBoost_model_1call invoke_endpoint(TargetVariant="Variant2", TargetContainer="XGBoost") to access the XGBoost_model_1Is that combination even possible?If so, what happens when the client calls the invoke_endpoint without specifying the variant? For example:would invoke_endpoint(TargetContainer="KMeans", TargetModel="KMeans_model_2") fail 50% of the time (if it hits the right variant then it works just fine, if it hits the wrong one it would most likely result with a 400/500 error ("incorrect payload")?FollowComment"

Answer: "0Well, I've checked that myself.Turns out NONE of these combinations are possible. :)Multi-model + Multi-container is NOT possibleVariants + Multi-container is NOT possibleVariants + Multi-model is NOT possibleIn all cases, you get a corresponding error while invoking create_endpoint_configuration:Multiple ProductionVariants is currently not supported when a Model uses a Direct InferenceExecutionMode.Direct InferenceExecutionMode is not supported when a Container uses MultiModel mode.MultiModel mode is not supported with the current model specification.CommentShareAverage Stackoverflow Enjoyeranswered a year ago"


Question: How to provide shared link for a resource in Amazon Workdocs through rest api?

"As far as I know, we can add resource permissions to resource on basis of recipient id.But how to create a shared link and give different level of access like View, Edit or Public and also set link expiry time programmatically/through rest api?I know that we can give access levels through UI but could not find a way to do it programmatically.FollowCommentkamlau 8 months agoThank you for posting your question to re:Post about WorkDocs APIs.WorkDocs currenly provides supported REST APIs documented here: https://docs.aws.amazon.com/workdocs/latest/APIReference/Welcome.htmlAnything that is not documented is not supported yet. We are prioritizing APIs to be published on an ongoing basis. Creating a share programmatically falls into this category of API. We will consider your request as a data point for our prioritization.Thank you.Share"

Answer: -1Adding Kam Lau to answer your question.CommentShareShrutianswered 8 months ago


Question: Upgrading MSK (Kafka) brokers machine type took more than an hour

I upgraded   3 brokers in MSK from Kafka.t3.small --> to large. The operation to upgrade took 1 hour 20 minutes which is extremely high. Why should it take that long just to upgrade without running anything at all on those brokers?Please fix this.FollowComment


Question: EBS IO2 IOPS volumes are exceeding 64000 set limit

"I have a t4g.xlarge instance type for my production server and have set 200 GiB and IO2 64000 IOPS limit, but am being hit 83000 IOPS and being charged $1200-$1300, so please suggest to me how to get the cost down suggest me a way to optimize it.FollowCommentKallu 3 months agoNot sure if I understood your setup right, but t4g.xlarge and EBS volume with 64000 provisioned IOPS just doesn't make sense. I don't think your instance can do even close to that, meaning you are paying a lot for IO performance you can not utilize. Baseline performance of t4g.xlarge is 4000IOPS (16kb) and at maximum it can burst 15700 for 30min/day.https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.htmlSharerePost-User-8174665 3 months agowe are being charged approx $4900 on 83000 IOPS, how can we sort things by bringing the price down in mumbai region.Share"

Answer: "1A t4g.4xlarge has a maximum capability of 15700 IOPS so setting it to 64000 would be a waste of IOPS as you cannot use that many IOPs on the instance type.  You can find a chart of maximum IOPS supported per instance here:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.html#currentWhat you should check is all of your EBS volumes to see what they are configured like. If you're being charged for 83000IOPS it's because you have disks configured to use that many IOPS as EBS for IO1/2 charges for storage, and configured IOPS.  Take a look in the EC2 console at your volumes and see what disks are IO1/IO2 or GP3 and what their IOPS are configured at to understand where you're being charged from for the 83000 IOPS.CommentShareEXPERTRob_Hanswered 3 months ago"


Question: AWS Amplfy push Error Command failed with exit code 4294967295

"Hello, I am having a problem with amplify i already changed my .csproj local files but the cloud .csproj is not updating and also I am looking for resources about an error and could not find anything about itCommand failed with exit code 4294967295: dotnet lambda package --framework netcoreapp3.1 --configuration Release --output-package C:\KY\Documents\Projects\GreenProjects\GreenProjects-BE\amplify\backend\function\InstallersFunction\dist\latest-build.zipAmazon Lambda Tools for .NET Core applications (5.3.0)Executing publish command... invoking 'dotnet publish', working folder 'C:\Users\KY\Documents\Projects\GreenProjects\GreenProjects-BE\amplify\backend\function\InstallersFunction\src\bin\Release\netcoreapp3.1\publish'... dotnet publish --output "C:\Users\KY\Documents\Projects\GreenProjects\GreenProjects-BE\amplify\backend\function\InstallersFunction\src\bin\Release\netcoreapp3.1\publish" --configuration "Release" --framework "netcoreapp3.1" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-x64 --self-contained false... publish: Microsoft (R) Build Engine version 17.2.2+038f9bae9 for .NET... publish: Copyright (C) Microsoft Corporation. All rights reserved.... publish:   Determining projects to restore...... publish:   Restored C:\Users\KY\Documents\Projects\GreenProjects\GreenProjects-BE\amplify\backend\function\InstallersFunction\src\InstallersFunction.csproj (in 304 ms).... publish: C:\Program Files\dotnet\sdk\6.0.310\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(267,5): error NETSDK1005: Assets file 'C:\Users\KY\Documents\Projects\GreenProjects\GreenProjects-BE\amplify\backend\function\InstallersFunction\src\obj\project.assets.json' doesn't have a target for 'netcoreapp3.1'. Ensure that restore has run and that you have included 'netcoreapp3.1' in the TargetFrameworks for your project. [C:\Users\KY\Documents\Projects\GreenProjects\GreenProjects-BE\amplify\backend\function\InstallersFunction\src\InstallersFunction.csproj]Failed to create application packageFollowComment"


Question: looking for automation to Delete S3 files with specific Extension and date

"I have Automation to delete files with specific range of dated files, but looking for Automation if we have any on Deleting s3 files with specific Extensions and data together which are 60 days old.FollowComment"

Answer: "3Lifecycle is a great mechanism within S3 to automatically delete files based on certain criteria. You can specify a filter by using object size, object key prefix, one or more object tags, or a combination of filters.So one way would be to tag your objects when they are created/uploaded and apply the lifecycle policy.https://docs.aws.amazon.com/AmazonS3/latest/userguide/intro-lifecycle-rules.htmlAnother way would be utilize S3 Inventory report. Identify objects that you want to delete and leverage SDKs/API's as show below to submit request for multiple deletes.https://docs.aws.amazon.com/AmazonS3/latest/userguide/delete-multiple-objects.htmlhttps://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.htmlYou can also use Athena queries to identify the objects from s3 inventory.https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-inventory-athena-query.htmlCommentSharerePostfanclubanswered a year ago0Accepted AnswerI built this and working for my requeirment#!/bin/bashSINCE_today=date --date '-13 weeks -1 days' +%F 2>/dev/null || date -v '-2w' -v '+2d' +%Fbucket=POC_DEV_Bucketaws s3api list-objects --bucket "$bucket" --query 'Contents[?LastModified < '"$SINCE_today"']' --output text > 90days_old.txt && grep -i ".json" 90days_old.txt > s3file_with_extension_with_daysold.txt && cat s3file_with_extension_with_ZERO_daysold.txt && awk '{$1= ""; print $2}' s3file_with_extension_with_daysold.txt |  xargs -I {} aws  s3 rm s3://"$bucket"/{}CommentShareRGADAanswered a year ago0I have Lifecycles enabled, but some bucket which has huge data in terabytes need to be deleted by using particular extension like .json,.csv,.html with 90 days old.so wondering is there a lambda or some automation to do that before I needed to work on new codeCommentShareRGADAanswered a year ago"


Question: Is it possible to modify the statusCode returned by API Gateway when access is denied by a lambda authorizer?

"A customer is migrating an API from Apigee to API Gateway. Currently their API returns 401 Unauthorized when a request is denied because the token is invalid. They want to maintain same status code when possible to avoid having to ask their partners to modify their code to handle different status code.By default API Gateway returns a 403 Forbidden when the request is denied by the Lambda Authorizer. I know that is possible to provide custom messages based on $context. authorizer.key, but have not found a way to change the statusCode returned by API Gateway.FollowComment"

Answer: "0Accepted AnswerYou can use Gateway Response to modify the status code, body, and headers of the response. Choose the 'Unauthorized' gateway response type and configure as desired.CommentShareAWS-User-3193127answered 5 years ago"


Question: Cross account assume role inside organization

"Hi,I have an AWS organization with multiple account (root_account, account_1, account_2, ... account_n). What I'm trying to do is, starting from an user in root_account, be able to assume role in any of the others organization accounts (account_x).In every account_x I've created a role account_example_role with the following trust relationship:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::<root_account_id>:role/root_example_role"            },            "Action": "sts:AssumeRole"        }    ]}while in root_account I have the role root_example_role with the following plicy:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "sts:AssumeRole"            ],            "Resource": [                "arn:aws:iam::<account_1>:role/account_example_role",                "arn:aws:iam::<account_2>:role/account_example_role",                ...                "arn:aws:iam::<account_x>:role/account_example_role"            ]        }    ]}and this seems to work, but it's hard to maintain.I was wondering if there is a way to specify an organization wide role arn inside the root_example_role policy, something like this:"Resource": "arn:aws:organizations::<root_account_id>:organization/o-<org-ID>/role/account_example_role"so that it's no longer needed to update the root_example_role policy for every new account in the organization.FollowComment"

Answer: "0I wonder if a Global condition would work in this situation. Something like this:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "sts:AssumeRole"            ],            "Resource": [                "arn:aws:iam::*:role/account_example_role"            ].            "Condition": { "StringEquals": { "aws:ResourceOrgID": "${aws:PrincipalOrgID}"  }  }        }    ]}CommentShareEXPERTkentradanswered a month agoGigitsu a month agoThank you for your answer, but it doesn't seems to work :(I get this error:This policy does not grant any permissions. To grant access, policies must have an action that has an applicable resource or condition. For details, choose Show remaining Learn moreAnd if I open the Show more menu I find this warning:aws:ResourceOrgID   One or more conditions do not have an applicable action.It seems that ResourceOrgId (https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-resourceorgid) isn't available for every actions, but should work for sts actions.ShareGigitsu a month agoThis is the policy I tried:{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "sts:AssumeRole",                "sts:TagSession"            ],            "Resource": "arn:aws:iam::*:role/account_example_role",            "Condition": { "StringEquals": { "aws:ResourceOrgID": "${aws:PrincipalOrgID}"  }  }        }    ]}Share0IAM roles are linked to a particular AWS account an not to the whole organization. So you still need to mention each account's role arn in the root_example_role policyCommentShareChandra Y Manswered a month agoGigitsu a month agoHi, thank you for you answer, but I don't fully get it. I know that roles are bound to an account and not to an organization, but so are other resources too and I am still capable to grant permissions to a particular resource in an organization using conditions (es. PrincipalOrgID).What's the difference with roles?Share"


Question: How to use preferred username to login a user?

I am trying to use forcealiasatribute by which user can login through preferred username.Email is the only attribute required for sign up but in Amplify.Auth.SignUp() it is asking for username.But I want that username should be a random "sub"(example:"151af027-eede-423f-b17e-3d94908717s2").Now(This is happening):Username: entered through Amplify.Auth.SignUp() function and it must be unique.(I want this should be a random string like below sub and not entered by user)Sub: 151af027-eede-423f-b17e-3d94908717s2 it is also unique.Preferred username: updated through attribute update function it is also unique.I WANT THIS:Username:151af027-eede-423f-b17e-3d94908717s2Preferred username: updated through attribute update function(it should be unique)FollowComment


Question: Lightsail - Upgrade MySQL from 8.0.28 to 8.0.32

"I have a High availability MySQL database (8.0.28) in Lightsail which I would like to upgrade to 8.0.32 but I can't seem to find any option to do this. Do I need to create a new database, manually export/import from old to new and switch over?Many thanks.FollowComment"

Answer: "0From: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-changing-preferred-maintenance-and-backup-windows"Minor version upgrades, and other database maintenance tasks, are performed automatically during the preferred maintenance window for your database. The preferred maintenance window is a 30-minute window selected at random from an 8-hour block of time for each AWS Region. It occurs on a random day of the week. Database backups are performed during the preferred backup window. The preferred backup window is a 30-minute window selected at random from an 8-hour block of time for each AWS Region. It also occurs on a random day of the week."It will be automatically upgraded during a maintenance window.CommentShareAWS-User-0986812answered 3 months ago"


Question: How to solve the situation that the compilation process of fpga program is terminated without error？

"Hello, I am currently using Vitis hls2021.2 to compile an FPGA program. The instance I use is t2.2x. My problem is that there is no error during the compilation process, But my compilation log shows that the compilation is terminated when the fifth task is performed. And before the termination, the log has no output for nearly two hours[05:01:29] Phase 1 Physical Synthesis Initialization[05:03:00] Phase 4.1.1.2 BUFG Replication[05:03:00] Phase 4.1.1.3 Post Placement Timing Optimization[05:08:04] Phase 4.1.1.4 Replication[05:11:06] Phase 4.2 Post Placement Cleanup[05:11:06] Phase 4.3 Placer Reporting[05:11:06] Phase 4.3.1 Print Estimated Congestion[05:11:37] Phase 4.4 Final Placement Cleanup[05:24:17] Finished 4th of 6 tasks (FPGA logic placement). Elapsed time: 01h 13m 56s [05:24:17] Starting logic routing..[05:26:19] Phase 1 Build RT Design[05:29:51] Phase 2 Router Initialization[05:29:51] Phase 2.1 Fix Topology Constraints[05:29:51] Phase 2.2 Pre Route Cleanup[05:30:22] Phase 2.3 Global Clock Net Routing[05:43:32] Phase 2.4 Update Timing[05:47:05] Phase 2.5 Update Timing for Bus Skew[05:47:05] Phase 2.5.1 Update Timing[05:48:36] Phase 3 Initial Routing[05:48:36] Phase 3.1 Global Routing[05:52:09] Phase 3.2 Update Timing[05:57:44] Phase 4 Rip-up And Reroute[05:57:44] Phase 4.1 Global Iteration 0I don't understand if this is caused by me not selecting the f1 instance? Because my personal understanding is that the generation of the xclbin file has nothing to do with having fpga or not. I'd like some help with generating the error, I'll add later if detailed code is neededFollowComment"

Answer: "0Dear customerThank you so much for your interest in AWS. To eliminate the possibility of memory not sufficient on the instance, can you please try your run on a large instance and see if you still see the issue.ThanksCommentShareAWS-User-7905434answered a year ago"


Question: EFS Throughput  not exceeding 1MB

"Good day,I hope you can assist with a question around EFS throughput that I am seeing on our AWS services.I have setup an EFS filesystem which I have mounted on 2 x EC2 instances. Originally, I had it configured as burstable throughput.When looking at the MeteredIOBytes metric, the maximum throughput never exceeds 1.05M. I thought it might be because I am not using much space on EFS and as I understand it, when on burstable throughput, the throughput increases as you use more GBs.I then tried changing the EFS to provisioned throughput of 5MB, but even after doing this, the MeteredIOBytes metric never exceeds 1.5MB.In both burstable and provisioned mode, when comparing the maximum MeteredIOBytes with PermittedThroughput, the MeteredIOBytes is way below Permitted.I am only using 3GB of storage space on EFS.Is there a setting I am missing to increase the throughput beyond 1.5MB?ThanksFollowComment"

Answer: "0PermittedThroughput is the maximum amount of throughput that a file system can drive. For file systems in the Provisioned Throughput mode, if the amount of data stored in the EFS Standard storage class allows your file system to drive a higher throughput than you provisioned, this metric reflects the higher throughput instead of the provisioned amount. For file systems in Bursting Throughput mode, this value is a function of the file system size and BurstCreditBalance. Because the MeteredIOBytes metric is less than the PermittedThroughput metric, the application is not driving the file system hard enough to reach the PermittedThroughput. Please review the Performance section of the Amazon EFS user guide for ideas on how to improve performance - https://docs.aws.amazon.com/efs/latest/ug/performance-tips.html. Here are a few tips: 1) increase the number of threads accessing the EFS file systems (e.g. increase the degree of parallelism), use larger average I/O sizes, use NFSv4.1 and a Linux distro with the latest kernel version.  If you want to verify that the file system is capable of achieving more throughput than you can drive with your application, use a synthetic tool like dd, IOR, FIO, with lots of threads and a large block size (e.g. 1 MiB). Refer to the Amazon EFS tutorial for tips on how to achieve better performance with Amazon EFS - https://github.com/aws-samples/amazon-efs-tutorial/tree/master/performance.CommentShareDarryl_Oanswered a year ago0Thanks for the useful response.I am mounting from Amazon Linux 2 instances and using the efs helper. I tried a few of the DD tools to test throughput, and interestingly, the output from DD and load while DD is running, shows 100MB/s+ throughput so it seems the throughput is definitely available, however the Cloudwatch graphs are only showing peaks of 1.05MB during the times I was running the DD tests which is what is misleading to me.Thanks again.CommentShareClaytonAanswered a year ago"


Question: Copying a file in S3 o ECS Container

How do we copy the file in S3 bucket to a particular location inECS Container ? (eg:/usr/local/tomcat/webapps/root/js).I am downloading the image from ECR and creating a container out of it. But I need to copy the custom config-local.js to a particular location in ECS. I am trying to do it via environment command. But it throwing errors. Please help me as I could not find much information for the same in documentation .FollowComment

Answer: "0Have you tried using volumes as mentioned under the Volumes section of this document from docker - https://docs.docker.com/cloud/ecs-integration/Also look at this - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/efs-volumes.htmlCommentShareEXPERTIndranil Banerjee AWSanswered 7 months agorePost-User-1738073 7 months agoI have tried adding my file to be copied into a folder in efs and I was able to attach it to a container.But that is not what I am looking for. I want to copy a particular file onto a specific destination inside the container(/usr/local/tomcat/webapps/root/js). Copying to efs and attaching the volume did not solve my problem.Appreciate your response. Thanks in advance!!Share0Hello there.I would recommend having a docker volume that you can declare very easily if you use docker-compose and have a sidecar that will deal with downloading/creating configuration files for you in this/these docker volume(s). Especially if you do not need the files to be shared across different containers, which would require you to use NFS/AWS EFS.I have written a sidecar container that can do all of this for you, which you can get from hereI have numerous examples of using it that can be found in labs (GitHub)The "instructions" on files to download / generate files using it in the docs as well.The configuration itself can be stored in either of these. I usually like to store it base64 encoded in SSM, but if the file gets too large, you can use S3 instead.To simplify the deployment of your service with these two, I'd recommend using ECS Compose-X which will take care of your volumes configuration, IAM permissions, and access to S3, etc.This might be one of the simplest examples I have to demonstrate both.Hope this helps.CommentShareJohnPrestonanswered 7 months ago"


Question: Domain verification in Amazon SES

"Hello there,I've accessed my legacy TXT records via a csv file and they look like this:"Type,Name,ValueTXT,_amazonses.xxx@domain.com ,TXT,xxx@domain.com,amazonses:""xxx", "domain.com" = private info.And as for the NAME record this is the error Im getting from the cpanel:"The DNS label must contain only the following characters: a-z, A-Z, 0-9, -, and _."Can someone please help?thanks!FollowComment"


Question: Oracle APEX 22.2 on RDS

Does anyone know if AWS is planning to provide newer versions of Oracle APEX? The latest version available per the product page https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.Oracle.Options.APEX.html is 21.1 which is well over a year old now.FollowCommentJD 8 months agoIt looks like this question was asked and answered a few weeks ago. https://repost.aws/questions/QUBIkH5h_cRv2vA8bgkwsTFQ/when-will-apex-21-2-or-22-1-be-available-in-rds-oracleShare

Answer: "1Its not possible to share the ETA as it depends on several factors. Nonetheless, please keep an eye on [+] for news on Apex on RDS Oracle.https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.Oracle.Options.APEX.htmlCommentShareD-Raoanswered 8 months ago"


Question: When will DocumentDB be available in Bahrain region?

Is there anyone who can shed some light on the availability time of DocumentDB in Bahrain? One of our customers would like to migrate their on-premises cluster there ASAP. Any information will be appreciated.FollowComment

Answer: 0I do not have any ETA on when DocumentDB will available in Bahrain region. But I urge you to keep an eye on the following.RDS blog: https://aws.amazon.com/blogs/aws/category/amazon-rds/RDS updates: https://aws.amazon.com/rds/whats-new/RDS Forum: https://forums.aws.amazon.com/forum.jspa?forumID=60CommentShareVetrivelanswered a year ago


Question: AWS Elasticsearch in VPC - how do we upload data or send a POST request from outside of VPC.

"Kibana dashboard is accessible via nginx proxy. How do we push data from an app directly to the ES ? I couldn't sign the http request with AWS credentials nor with EC2 instance profile of the nginx server. Still times out as it is in a VPC.Second - Do we need to do VPC peering with the EKS cluster, if the app runs in EKS kubernetes cluster in order to push data to ES, which is in it's own VPC ?FollowComment"

Answer: "0If your cluster is accessible from outside the VPC using a NGINX proxy, why not configure the proxy to allow POST requests as well?If your app is in another VPC then VPC peering may be the answer.CommentShareEXPERTBrettski-AWSanswered 2 months agodevops-stchealth 2 months agoYes, we did VPC peering. nginx proxy for ES cluster and kibana dashboard works but it takes to the root and I couldn't upload any data until VPC peering was done. Not sure if I was missing something.Share"


Question: Free Tier - Data Warning?

"I am using just one EC2 free tier linux instance.I check the data stored on the linux instance with df -h command, and it reports 4.2GB used of 30g available.However, I received a Data warning email saying 27GB was used, nearing the 30GB limit.*edit - and I can confirm I only have one volume too, of 30GBI am sure there is only one instance running, so something seems quite odd to me.Advice appreciated. Thanks.FollowComment"

Answer: "3EBS volumes have a free tier of 30GB per month for the first 12 months of your account. The volumes are charged by allocated space and not used space. So if your instance has 30GB of EBS storage allocated, then you are using 30GB of EBS even if only 4.2GB of the 30 is used. The 30GB is per month, so the fact that you received a warning of 27GB toward the end of the month, indicates that you will use the full amount (either the full amount, just under or just over) in the coming days. Given that you stated you have 30GB allocated, this makes sense that you will use the full amount.CommentShareEXPERTShlomo Dubrowinanswered a month agoEXPERTRiku_Kobayashireviewed a month ago"


Question: GameSparks Cloud Code - How to get current backend time (preferably time since epoch)

"Hi,I want to get current time as milliseconds to store in DynamoDB, along with some player data acquired from the client. So far create a new Event and use some of the below approaches:const timeInEpoch = ${now};const timeInEpoch = Date.now();const timeInEpoch = new Date().getTime();But it seems to me that none of the above code works, when I try the event with Test Harness, it return error result as:Is there a way to retrieve this information? Thanks a lot!FollowComment"

Answer: "1Accepted AnswerHi Niles!Here's how you can get the current epoch, in seconds and milliseconds:const epoch = new Date('1970-01-01T00:00:00Z');const currentDate = new Date();const timestamp_ms = currentDate - epoch;const timestamp = Math.floor(timestamp_ms / 1000);GameSparks().Logging().Info("Current timestamp, in seconds: " + timestamp);CommentShareJoris_Danswered 2 months agoNiles a month agoThanks a lot, Joris, It worked now!Share"


Question: Is it possible to get a SQS subscription's ARN using CDK?

I'm using CDK to setup an SNS topic and configure a subscription to an SQS queue. However I've noticed that the addSubscription method of the Topic class is void and as such does not return anything after the subscription is added.Is there a way to get the subscription ARN using the CDK constructs as I'd like to output the subscription ARN as a CloudFormation output?FollowComment

Answer: "0There is no direct way via CDK to get the subscription ARN. Neither CfnTopic nor CfnSubscription return the subscription ARN or provide a way to retrieve it.A helpful way to determine whether a value is available in CDK is this:Does the API provide it? Looking at the sns subscribe api call, it doesn't return the arn.Does CloudFormation's AWS::SNS::Subscription provide it? Looking at the doc, there are no attributes on this resource type.This is a good indicator that CDK doesn't support it either, which we see in the CfnTopic and CfnSubscription docs.You could use the list-subscriptions api in your CDK code to find the arn and store it in a CfnOutput.An alternate approach you could take is to use a lambda-backed custom resource to create the subscription, use the list-subscriptions API to find the arn for the new subscription, and return the arn. But this seems like the "long way around the barn."CommentShareMike_Oanswered a year agomjsztainbok a year agoThe Subscribe API call does return the subscription ARN though. From https://docs.aws.amazon.com/sns/latest/api/API_Subscribe.html:Response ElementsThe following element is returned by the service.SubscriptionArnThe ARN of the subscription if it is confirmed, or the string "pending confirmation" if the subscription requires confirmation. However, if the API request parameter ReturnSubscriptionArn is true, then the value is always the subscription ARN, even if the subscription requires confirmation.Type: StringShare"


Question: make julia notebooks work in SageMaker

"I want to run a Jupyter notebook in SageMaker with a Julia kernel. There is very little documentation about this. There is this:https://d1.awsstatic.com/whitepapers/julia-on-sagemaker.pdf?did=wp_card&trk=wp_cardI followed all the instructions, and Julia shows up in the JupyterLab launcher; but when I run it, Julia 1.17.1 shows up as the kernel and then dies. It appears to be trying, but then gives up and says "No Kernel" instead of "Julia 1.17.1" in the status line.If I run the R kernel, all goes well. If I run the Julia kernel (which shows up in the list of available kernels!), I get the following error message:Connection failedA connection to the notebook server could not be established.The notebook will continue trying to reconnect.Check your network connection or notebook server configuration.FollowComment"

Answer: "1Sorry this is not a full working solution, but too much for a comment & hopefully will still be useful to you:(Assuming you're talking here about SageMaker Notebook Instances rather than SageMaker Studio, same as the whitepaper; that you're trying to install the latest version of Julia from conda rather than v1.0.3 explicitly specified in the whitepaper, currently 1.7.1)The first thing to point out is that you should be able to debug this via the notebook's logs: Either click the "View logs" link on the notebook's detail page in Amazon SageMaker console, or look in the Amazon CloudWatch console for log group /aws/sagemaker/NotebookInstances, stream {YOUR-NBI-NAME}/jupyter.log.Following through the whitepaper instructions myself (on an notebook-al2-v1 notebook instance), the errors I saw preventing the kernel from loading were like:ArgumentError: Package IJulia not found in current path:- Run `import Pkg; Pkg.add("IJulia")` to install the IJulia package.Looking at ~/.local/share/jupyter/kernels/julia-1.7/kernel.json, I saw the created kernel was defined as follows:{  "display_name": "Julia 1.7.1",  "argv": [    "/home/ec2-user/anaconda3/envs/julia/bin/julia",    "-i",    "--color=yes",    "--project=@.",    "/home/ec2-user/anaconda3/envs/julia/share/julia/packages/IJulia/e8kqU/src/kernel.jl",    "{connection_file}"  ],  "language": "julia",  "env": {},  "interrupt_mode": "signal"}If we run julia from within the julia conda environment, using IJulia works no problem... However, if you source activate JupyterSystemEnv from the terminal - you can still run /home/ec2-user/anaconda3/envs/julia/bin/julia from the terminal but the interpreter will not think the IJulia package is installed... I think this is closer to what the above kernel definition is doing (NBI JupyterServer itself runs in this system conda env).I tried a couple of hacky solutions:Simply start Julia in JupyterSystemEnv as above and Pkg.add("IJulia") to install it thereCopy the setup you'll see in the R kernel, /home/ec2-user/.local/share/jupyter/kernels/ir - where kernel.json points to a run.sh script which first activates the target conda environment and then runs the interpreterkernel.json (with a different display name & file path to visually confirm JupyterLab has picked the new one up):{  "display_name": "Julia Fix",  "argv": [    "/home/ec2-user/.local/share/jupyter/kernels/julia-fix/run.sh",    "{connection_file}"  ],  "language": "julia",  "env": {},  "interrupt_mode": "signal"}run.sh (remember to chmod +x this file to avoid 500 server errors as Jupyter can't execute it)#!/bin/bashsource activate julia/home/ec2-user/anaconda3/envs/julia/bin/julia -i --project=@. /home/ec2-user/anaconda3/envs/julia/share/julia/packages/IJulia/e8kqU/src/kernel.jl $1HOWEVER, unfortunately both approaches yield a ZMQStream Invalid Signature error, which looks to me like this open IJulia issue. I see speculation there that Julia isn't playing nice with conda, but am not deep enough with Julia to know if that's really the root cause or how best to mitigate it if so.Maybe you could install Julia itself outside of conda (conda deactivate in terminal to check you're in base environment) and, if still getting the signature error, use a run.sh kernel script to ensure Jupyter also runs the command outside of conda rather than in the JupyterSystemEnv? Or perhaps there's some other cause for the signature issue that can be resolved while keeping conda environments set up...You could also look into the SageMaker Studio Custom Image Samples, instead of Notebook Instances, since Studio kernels are isolated by full container images rather than conda? There is an example image there for Julia (v1.5), although it's not been updated in some time so of course could have some issues of its own.CommentShareEXPERTAlex_Tanswered a year agovodoaldus a year agoThis is super helpful, Alex T. I spent all morning yesterday on the question we are discussing here, and it appears you also ran into trouble, although you got a lot further than I did. Thank you. I will try some of this again with your instructions.Sharevodoaldus a year agoThen I spent the afternoon trying to install this custom SageMaker docker image:https://github.com/aws-samples/sagemaker-studio-custom-image-samples/tree/main/examples/jupyter-docker-stacks-julia-imageEven though I followed all the instructions, I was not able to upload the image into SageMakerdocker push XXXXX.dkr.ecr.us-west-2.amazonaws.com/smstudio-custom:julia-datasciencejust kept "Retrying ..." -- so, all to say, this has been very frustrating.Sharevodoaldus a year agoI currently run Julia notebooks in CoCalc. They have great customer service at CoCalc, but I was hoping AWS could speed up processing times. I will keep trying and post a solution here if I find one. In the meantime, if anybody has gotten Julia/Jupyter notebooks to work on AWS SageMaker, please post instructions.ShareAlex_T EXPERTa year agoSorry only just saw your comments! That "Retrying..." error is odd - I've not seen it before, and seems like you were just not able to push container image to ECR - so would be local docker + Amazon ECR permissions to investigate, rather than anything SageMaker-specific. May be worth a separate question with any other debug information you have?If there's a chance it might be related to your local internet connectivity (pulling all the source layers and then pushing the image up to us-west-2 cloud), you could also try doing those docker steps from your SageMaker Notebook Instance terminal?Share0Note that Alex_T found a (hacky) solution to the ZMQStream problem here:https://www.repost.aws/questions/QUM2y8-rjDS1Wp5LUkdLMhyA/uncaught-exception-in-zmq-stream-callback-trying-to-run-jupyter-notebook-with-julia-kernel-in-sage-makerand got the Jupyter notebook with Julia kernel to work in SageMaker.CommentSharevodoaldusanswered a year ago"