Question: Unable to update dashboards with cid-cmd

"Just upgraded to cid-cmd 0.2.12 Beta and when trying to run cid-cmd update it always says"No updates available or dashboard(s) is/are broken, use --force to allow selection"With "cid-cmd status" it always shows that my dashboards are up to date which is not the case:? [dashboard-id] Please select installation(s) from the list: (Use arrow keys)  » Trends Dashboard (arn:aws:quicksight:eu-west-1:xxxxxxxxxxxxxx:dashboard/trends-dashboard, healthy, up to date)    KPI Dashboard (arn:aws:quicksight:eu-west-1:xxxxxxxxxxxxxx:dashboard/kpi_dashboard, healthy, up to date)    Cost Intelligence Dashboard (arn:aws:quicksight:eu-west-1:xxxxxxxxxxxxxx:dashboard/cost_intelligence_dashboard, healthy, up to date)    Trusted Advisor Organizational View (arn:aws:quicksight:eu-west-1:xxxxxxxxxxxxxx:dashboard/ta-organizational-view, healthy, up to date)    CUDOS Dashboard (arn:aws:quicksight:eu-west-1:xxxxxxxxxxxxxx:dashboard/cudos, healthy, up to date)    Compute Optimizer Dashboard (arn:aws:quicksight:eu-west-1:xxxxxxxxxxxxxx:dashboard/compute-optimizer-dashboard, healthy, up to date)If I hit ENTER on one of them a Python exception is thrown:Dashboard status:   Name (id): Trends Dashboard (trends-dashboard)   Status: up to date   Health: healthy Traceback (most recent call last):   File "/home/luis/.virtualenvs/aws-cid-cmd/bin/cid-cmd", line 8, in <module>     sys.exit(main())   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/click/core.py", line 1130, in __call__     return self.main(*args, **kwargs)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/click/core.py", line 1055, in main     rv = self.invoke(ctx)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/click/core.py", line 1657, in invoke     return _process_result(sub_ctx.command.invoke(sub_ctx))   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/click/core.py", line 1404, in invoke     return ctx.invoke(self.callback, **ctx.params)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/click/core.py", line 760, in invoke     return __callback(*args, **kwargs)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func     return f(get_current_context(), *args, **kwargs)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/cid/cli.py", line 28, in wrapper     res = func(ctx, **kwargs)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/cid/cli.py", line 135, in status     ctx.obj.status(dashboard_id, **kwargs)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/cid/common.py", line 164, in wrap     return func(self, *args, **kwargs)   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/cid/common.py", line 451, in status     dashboard.display_status()   File "/home/luis/.virtualenvs/aws-cid-cmd/lib/python3.10/site-packages/cid/helpers/quicksight/dashboard.py", line 116, in display_status     cid_version = self.deployedTemplate.cid_version AttributeError: 'NoneType' object has no attribute 'cid_version'Finally, when I try to force the update with cid-cmd update --force --recursive I get an error when trying to download a quicksight template from an external AWS account:? [dashboard-id] Please select installation(s) from the list: CUDOS Dashboard (arn:aws:quicksight:eu-west-1:xxxxxxxxxxxxxx:dashboard/cudos, healthy, up to date) CRITICAL - Error: An error occurred (AccessDeniedException) when calling the DescribeTemplate operation: User: arn:aws:sts::xxxxxxxxxxxxxx:assumed-role/AWSReservedSSO_YYYYYYYYYYYY_ae9320745c28c490/user is not authorized to perform: quicksight:DescribeTemplate on resource: arn:aws:quicksight:us-east-1:223485597511:template/cudos_dashboard_v3 with an explicit deny in a service control policy - Cannot find cudos_dashboard_v3 in account 223485597511.The assumed role has permissions for quicksight:* and no SCP are denying access to that service on my OU or account.FollowComment"

Answer: "0Hi there,Thank you for posting the question and the detailed explanation. To answer your question, a screen-share session would be needed and will require details that are non-public information. Can you please open a support case with AWS using the following link and we will be happy to help you.Thank you!CommentShareSUPPORT ENGINEERSanket_Kanswered 3 months ago"


Question: bootstrap failure due to requirement of arm64 version of numpy required for r6 instances?

"Was trying to upgrade to the latest r6 instances from r5s and ran into an issue with installing numpy in our bootstrap script via pip.Found this post that is related to my issue.  Was anyone able to resolve this without building your own wheel file of the arm64 version of numpy?EC2/EMR Cluster ConfigRelease label:   emr-6.5.0Instance Type:   r6gd.8xlargeSnippet of the bootstrap#!/bin/bash# python versionpyv="$(python3 -V 2>&1)"echo "Python version: $pyv"# misc code to link up the requirements.txtecho "`date -u` install python dependencies"#Install Python depssudo python3 -m pip install wheelsudo python3 -m pip install -r requirements.txtrequirements.txtboto3==1.18.46Cython==0.29.24pandas==1.3.3numpy==1.21.2Log Output+ echo 'Python version: Python 3.7.10' ...+ echo 'Thu Sep 29 22:16:46 UTC 2022 install python dependencies'+ sudo python3 -m pip install wheelWARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.  WARNING: The script wheel is installed in '/usr/local/bin' which is not on PATH.  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.+ sudo python3 -m pip install -r job-requirements.txtWARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.  ERROR: Command errored out with exit status 1:   command: /bin/python3 /usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /mnt/tmp/pip-build-env-yy928eo_/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'cython >= 0.29' 'numpy==1.14.5; python_version<'"'"'3.7'"'"'' 'numpy==1.16.0; python_version>='"'"'3.7'"'"'' setuptools setuptools_scm wheel       cwd: None  Complete output (866 lines):  WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.  Ignoring numpy: markers 'python_version < "3.7"' don't match your environment  Collecting cython>=0.29    Using cached Cython-0.29.32-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (1.8 MB)  Collecting numpy==1.16.0    Downloading numpy-1.16.0.zip (5.1 MB)  Collecting setuptools    Downloading setuptools-65.4.0-py3-none-any.whl (1.2 MB)  Collecting setuptools_scm    Downloading setuptools_scm-7.0.5-py3-none-any.whl (42 kB)  Collecting wheel    Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)  Collecting packaging>=20.0    Downloading packaging-21.3-py3-none-any.whl (40 kB)  Collecting tomli>=1.0.0    Downloading tomli-2.0.1-py3-none-any.whl (12 kB)  Collecting typing-extensions    Downloading typing_extensions-4.3.0-py3-none-any.whl (25 kB)  Collecting importlib-metadata; python_version < "3.8"    Downloading importlib_metadata-4.12.0-py3-none-any.whl (21 kB)  Collecting pyparsing!=3.0.5,>=2.0.2    Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)  Collecting zipp>=0.5    Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)  ..._configtest.c:1:10: fatal error: Python.h: No such file or directory       #include <Python.h>                ^~~~~~~~~~      compilation terminated.      failure.      removing: _configtest.c _configtest.o      Traceback (most recent call last):        File "<string>", line 1, in <module>        File "/mnt/tmp/pip-install-tl9eju6y/numpy/setup.py", line 415, in <module>          setup_package()        File "/mnt/tmp/pip-install-tl9eju6y/numpy/setup.py", line 407, in setup_package          setup(**metadata)        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/core.py", line 171, in setup          return old_setup(**new_attr)        File "/usr/lib/python3.7/site-packages/setuptools/__init__.py", line 165, in setup          return distutils.core.setup(**attrs)        File "/usr/lib64/python3.7/distutils/core.py", line 148, in setup          dist.run_commands()        File "/usr/lib64/python3.7/distutils/dist.py", line 966, in run_commands          self.run_command(cmd)        File "/usr/lib64/python3.7/distutils/dist.py", line 985, in run_command          cmd_obj.run()        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/command/install.py", line 62, in run          r = self.setuptools_run()        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/command/install.py", line 36, in setuptools_run          return distutils_install.run(self)        File "/usr/lib64/python3.7/distutils/command/install.py", line 556, in run          self.run_command('build')        File "/usr/lib64/python3.7/distutils/cmd.py", line 313, in run_command          self.distribution.run_command(command)        File "/usr/lib64/python3.7/distutils/dist.py", line 985, in run_command          cmd_obj.run()        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/command/build.py", line 47, in run          old_build.run(self)        File "/usr/lib64/python3.7/distutils/command/build.py", line 135, in run          self.run_command(cmd_name)        File "/usr/lib64/python3.7/distutils/cmd.py", line 313, in run_command          self.distribution.run_command(command)        File "/usr/lib64/python3.7/distutils/dist.py", line 985, in run_command          cmd_obj.run()        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/command/build_src.py", line 148, in run          self.build_sources()        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/command/build_src.py", line 165, in build_sources          self.build_extension_sources(ext)        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/command/build_src.py", line 322, in build_extension_sources          sources = self.generate_sources(sources, ext)        File "/mnt/tmp/pip-install-tl9eju6y/numpy/numpy/distutils/command/build_src.py", line 375, in generate_sources          source = func(extension, build_dir)        File "numpy/core/setup.py", line 423, in generate_config_h          moredefs, ignored = cocache.check_types(config_cmd, ext, build_dir)        File "numpy/core/setup.py", line 47, in check_types          out = check_types(*a, **kw)        File "numpy/core/setup.py", line 281, in check_types          "install {0}-dev|{0}-devel.".format(python))      SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.      ----------------------------------------  ERROR: Command errored out with exit status 1: /bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/mnt/tmp/pip-install-tl9eju6y/numpy/setup.py'"'"'; __file__='"'"'/mnt/tmp/pip-install-tl9eju6y/numpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /mnt/tmp/pip-record-paofd9vx/install-record.txt --single-version-externally-managed --prefix /mnt/tmp/pip-build-env-yy928eo_/overlay --compile --install-headers /mnt/tmp/pip-build-env-yy928eo_/overlay/include/python3.7m/numpy Check the logs for full command output.  ----------------------------------------ERROR: Command errored out with exit status 1: /bin/python3 /usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /mnt/tmp/pip-build-env-yy928eo_/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'cython >= 0.29' 'numpy==1.14.5; python_version<'"'"'3.7'"'"'' 'numpy==1.16.0; python_version>='"'"'3.7'"'"'' setuptools setuptools_scm wheel Check the logs for full command output.FollowComment"

Answer: "0Hello,Thank you for reaching out to us. I understand that you are facing issues while installing numpy module from requirements.txt file.Looking at the error, I can see that the pip is trying to install packages from libraries.h and failed to locate the header file. This can be seen in the error logs as well:SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.You can try installing the -devel package for python3.7:# yum instal python3-devel[Similar Errors][+] https://stackoverflow.com/questions/21530577/fatal-error-python-h-no-such-file-or-directory[+] https://askubuntu.com/questions/526304/cannot-compile-python-h-perhaps-you-need-to-install-python-devpython-develI sincerely hope this helps. If the suggestions above do not help resolve the issue, we might need to troubleshoot based on your configurations. Could you please create a support case instead so we may discuss details on your resource configurations?Please do not post any sensitive information over re:Post since this is a public platform.As always, feel free to reach back with any further questions or concerns in the meantime!CommentShareRoopa_Kanswered 8 months ago"


Question: AWS DMS - 38 character - Limit

"AWS DMS is not pulling data from column which has 38 or more chars, we have also tried adding the transmission rule which changes the datatype to string and INT8.Is there is any other setting, which I have to add to make this work. Please let me know the solution. Thanks in advance.FollowCommentvtjean a month agoWhich is the source and destination that AWS DMS is using?Share"

Answer: 0Please try DMS 3.5.0 which was recently released. It includes new datatype support and may address your issue. More info: https://aws.amazon.com/blogs/database/new-features-in-aws-dms-3-5-0/CommentShareFranke-AWSanswered a month ago


Question: Conect services Ecs con  AWS Cloud Map

How can I configure an AWS Cloud Map to connect ecs service tasks which are automatically raised and destroyed when I push a repo?How can I see the id of a service?FollowComment

Answer: "3Create a namespace in AWS Cloud MapOpen the AWS Cloud Map console (https://console.aws.amazon.com/cloudmap/).Choose "Create namespace" and enter a namespace name and description.To discover the tasks within your application, you can use the AWS SDK or CLI to query the AWS Cloud Map service based on the namespace and service name you've configured. For more information and examples, refer to the AWS Cloud Map documentation: https://docs.aws.amazon.com/cloud-map/latest/dg/what-is-cloud-map.htmlCommentShareEXPERTsdtslmnanswered a month ago"


Question: Android app crashes when access token expires with no network connection during Amplify API call

"I am using the Flutter version of Amplify SDK for authentication and REST API for an Android app. The authentication is based on AWS user pool. The REST API is based on API Gateway and Lambda function with the access restricted to authenticated users only. Everything works fine except that if the network is disconnected when the access token expires, making an API call with Amplify.API.post the Android app will crash with the following exception:  "FATAL EXCEPTION: OkHttp Dispatcher". It appears that this happens not only for the flutter version but also the Android version of the Amplify SDK. Is it possible to make the SDK throw this exception instead of crashing the app? To reproduce the issue, simply follow the Amplify Flutter tutorial for Authentication (https://docs.amplify.aws/lib/auth/getting-started/q/platform/flutter/), and for REST API (https://docs.amplify.aws/lib/restapi/getting-started/q/platform/flutter/). When creating the REST API, please choose "Restrict API access" for  "Authenticated users only". By default, the access token expires in 60 minutes and then you can turn on Airplane mode and make an API call to observe the app crash.FollowComment"


Question: Maximum number of connections from user+IP exceeded

"Hi there,We desperately need some help with the following error:"Maximum number of connections from user+IP exceeded (mail_max_userip_connections=5)"We run workmail for our business. Our employees access have set up access to workmail email accounts on multiple devices (primarily Apple devices). When we're on the run, we work from the phone or laptop. When we're stationary, we can use laptops and workstations ...I for example, access my email account from my iMac, iPad and from my MacBook. Maning that I've set up an email client in Mac Mail on all devices, to access the same WorkMail email account. Everything use to work fine until about a week ago, when we started getting the following error "Maximum number of connections from user+IP exceeded (mail_max_userip_connections=5)". Other employees are experiencing similar issues.I'm guessing from what I've researched, this is an email server setting, related to IT Security controls, etc. It basically limits how many connections can be established from the same IP ... In this instance though it is preventing us from carrying out work.Is it possible to increase this setting from "5" to say "30". If so, How do I go about it in WorkMail / AWS (yes, have searched far and wide, with no help)?Any help greatly appreciated !FollowComment"

Answer: "0From the error message you're seeing, it seems you are using the IMAP protocol on all these devices. We generally recommend you use the Exchange protocols (ActiveSync for iOS devices, and EWS for OSX) to access mail if your device supports it as those do a better job handling multiple connections for the same mailbox.When you configure mail on your device, you should be choosing the "Exchange" account type. See our guides to setting this up:https://docs.aws.amazon.com/workmail/latest/userguide/connect_mac_mail.htmlhttps://docs.aws.amazon.com/workmail/latest/userguide/mobile-client.htmlCommentShareAWS-User-7321420answered 3 years ago0I have been also suffering from this error since around Sep 8th.I can't use Exchange in my application, so I reduced the connection environment.I think the global settings on the WorkMail have changed recently.CommentShareBsBabaanswered 3 years ago0The limit has been increased to 10 concurrent connections from the same IP address. Please note that making more connections doesn't necessarily give you a better experience.CommentShareJohnAWSanswered 3 years ago0Hi Dan,As long as I don't lose any emails, I will try this.Cheers,SteveCommentShareSteve00001answered 3 years ago0Hi JohnK,Increasing the limit ('m assuming this is a global setting that has been tinkered with lately on the workman server end?), appears to have fixed some of our (extensive) connectivity issues. Although, the problem appears to have resurfaced in the last day or two ...I will try Dan's suggestion RE: connecting via ActiveSync (or Exchange) instead of IMAP, but I need to perform relatively extensive research first - to understand impact (on existing email clients, locally archived emails, etc). What's the drive to minimise connections from the same IP address (for IMAP only?). Is it security-driven? In either case, 5 or 10 connections from a single IP address may still be a little low - especially for use cases where administrators or office managers may administer (and access) multiple email accounts.CommentShareSteve00001answered 3 years ago0Any idea if this limit can be raised ?i have 3 devices myself connecting behind a NAT firewall and other members of my team are also at the same location.I do not use outlook or activesync on any of my devicesKenCommentSharekenpooleanswered 2 years ago"


Question: access my buckets in s3 through secret manager key without logging any crenditials in neither cli nor code.

"Hi there,I am developing an Node.js Application. Where I want that only IAM role have permission to access S3 Bucket.I need to do that without Secret key and access ID of IAM User. We store our Keys in Secret Manager.  I want to access my buckets in s3 through secret manager key without logging any crenditials in neither cli nor code.any other way to access these in sdk?FollowComment"

Answer: "0Hi,If I understand your question correctly, you want to know how you can access S3 without using credentials (Access key/secret key).You can use AssumeRole API in your SDK, an example is available here : https://docs.aws.amazon.com/IAM/latest/UserGuide/example_sts_AssumeRole_section.htmlIf you have your credentials stored in secrets manager you can retrieve them without specifying them in your code or CLI. You will use AWS SDK for secrets manager API to specify the variables that stores your credentials. Refer to an example here : https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/manage-credentials-using-aws-secrets-manager.htmlAs you have a Node.js application, you can also consider deploying it with AWS amplify that has a storage plugin to access S3 buckets : https://docs.amplify.aws/lib/storage/getting-started/q/platform/js/CommentShareHarshil_Sanswered 8 months ago"


Question: "12 months free tier not used at all, possible to get an extension?"

"It's possible to get a renewal at a later time?I didn't have time to use it and I would like to use it later when I could.I know it's somewhat unreasonable, I shouldn't have registered when I had not the time to use it.Thank you in advance.FollowComment"

Answer: "1Accepted AnswerHi,No, you have to create a new account with new email or new credit card, free tier cannot be extended, unfortunately.Hope it clarifiesCommentShareEXPERTalatechanswered 2 months agorePost-User-7827436 2 months agoOh, thank you, then I'll try with another email/card when I'll needShare"


Question: Removing personal information content on Cloudfront

We hve found content with personal information about users of our organization hosted on Cloudfront and indexed by Google. It's not hosted on any of our accounts. We would like to get that content removed. I can't seem to find any way to report the file on AWS.What would be the best course of action on our part to get that file removed from Cloudfront ?FollowComment

Answer: 0I would open up a ticket with AWS Support.CommentShareEXPERTkentradanswered 7 months ago


Question: Could not deploy Greengrass group

"Hi,I have followed the documentation for AWS IoT Greengrass. While deploying the greengrass group, I am getting the following error. I am using Raspberry Pi as Greengrass core.[2019-02-06T14:35:57.317+01:00][INFO]-Greengrass Root: /greengrass[2019-02-06T14:35:57.317+01:00][INFO]-Greengrass Write Directory: /greengrass/ggc[2019-02-06T14:35:57.317+01:00][INFO]-Group File Directory: /greengrass/ggc/deployment/group[2019-02-06T14:35:57.317+01:00][INFO]-Default Lambda UID: 111 GID: 115[2019-02-06T14:35:57.318+01:00][INFO]-===========================================[2019-02-06T14:35:57.322+01:00][INFO]-The current core is using the AWS IoT certificates with fingerprint: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx[2019-02-06T14:35:57.323+01:00][WARN]-worker process info: /greengrass/ggc/packages/1.7.0/var/worker/processes[2019-02-06T14:35:57.324+01:00][WARN]-worker process info: /greengrass/ggc/packages/1.7.0/var/worker/processes[2019-02-06T14:35:57.329+01:00][INFO]-Reloading registry[2019-02-06T14:35:57.331+01:00][INFO]-The current core is using the AWS IoT certificates with fingerprint: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx[2019-02-06T14:35:57.553+01:00][INFO]-Deployment agent connected to cloud[2019-02-06T14:35:57.553+01:00][INFO]-Started Deployment Agent and listening for updates[2019-02-06T14:35:57.553+01:00][INFO]-Started Deployment Agent and listening for updates[2019-02-06T14:35:57.553+01:00][INFO]-Trying to subscribe to topic $aws/things/SmartFoodGroup_Core-gda/shadow/update/delta[2019-02-06T14:35:57.58+01:00][INFO]-Subscribed to : $aws/things/SmartFoodGroup_Core-gda/shadow/update/delta[2019-02-06T14:35:57.58+01:00][INFO]-Trying to subscribe to topic $aws/things/SmartFoodGroup_Core-gda/shadow/get/accepted[2019-02-06T14:35:57.609+01:00][INFO]-Subscribed to : $aws/things/SmartFoodGroup_Core-gda/shadow/get/accepted[2019-02-06T14:42:56.194+01:00][INFO]-Start to process new message from the cloud.[2019-02-06T14:42:56.194+01:00][INFO]-Deployment (Group ID: xxxxxxxxxxxxxxxxxxxxxxxxx, Deployment ID: xxxxxxxxxxxxxxxxx, Deployment Type: NewDeployment) is in pending state.[2019-02-06T14:42:56.196+01:00][INFO]-Existing group config does not contain a group arn. Skipping check to perform reset due to update in group id[2019-02-06T14:42:56.196+01:00][INFO]-End processing new message from the cloud.[2019-02-06T14:42:56.196+01:00][INFO]-Received deploymentId xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx of type NewDeployment for group xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx[2019-02-06T14:42:56.196+01:00][INFO]-Updating status to InProgress of deployment xxxxxxxxxxxxxxxxxxxxxxxx for group xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx[2019-02-06T14:42:57.905+01:00][INFO]-Executing the group Downloading step[2019-02-06T14:43:21.401_01:00][ERROR]-Greengrass deployment error: unable to start deployment downloader. Error while downloading. Error while downloading the Group definition file Error while downloading, filename: /tmp/greengrass/gro$[52;1H[2019-02-06T14:43:21.401_01:00][ERROR]-Greengrass deployment error: unable to execute download step in deployment. Error while downloading. Error while downloading the Group definition file Error while downloading, filename: /tmp/greeng$[2019-02-06T14:43:21.401_01:00][ERROR]-Deployment xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx of type NewDeployment for group xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx failed error: Error while downloading. Error while downloading the Group definit$[2019-02-06T14:43:21.401_01:00][INFO]-Updating status to Failure of deployment xxxxxxxxxxxxxxxxxxxxxxxxxxx for group xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx[2019-02-06T14:43:21.46+01:00][INFO]-Updating shadow for deployment: xxxxxxxxxxxxxxxxxxxxxxxxxx of group: xxxxxxxxxxxxxxxxxxxxxxx[2019-02-06T14:43:21.489+01:00][INFO]-Start to process new message from the cloud.[2019-02-06T14:43:21.489+01:00][INFO]-End processing new message from the cloud.Thanks,SantoshFollowComment"

Answer: "0Hi,I have also added proxy details to the config.json file. But, the deployment is in "In Progress" status for a very long time.PFB the runtime.log.[2019-02-07T14:22:38.508+01:00][INFO]-Greengrass Root: /greengrass[2019-02-07T14:22:38.508+01:00][INFO]-Greengrass Write Directory: /greengrass/ggc[2019-02-07T14:22:38.508+01:00][INFO]-Group File Directory: /greengrass/ggc/deployment/group[2019-02-07T14:22:38.508+01:00][INFO]-Default Lambda UID: 111 GID: 115[2019-02-07T14:22:38.509+01:00][INFO]-===========================================[2019-02-07T14:22:38.513+01:00][INFO]-The current core is using the AWS IoT certificates with fingerprint: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx[2019-02-07T14:22:38.514+01:00][WARN]-worker process info: /greengrass/ggc/packages/1.7.0/var/worker/processes[2019-02-07T14:22:38.515+01:00][WARN]-worker process info: /greengrass/ggc/packages/1.7.0/var/worker/processes[2019-02-07T14:22:38.517+01:00][INFO]-Using proxy http://myproxy.com:80[2019-02-07T14:22:38.52+01:00][INFO]-Reloading registry[2019-02-07T14:22:38.523+01:00][INFO]-The current core is using the AWS IoT certificates with fingerprint: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxThanks,SantoshCommentShareSantoshPandaanswered 4 years ago0Can anyone help me out here?Every deployment is in "In Progress" state.The log is same as the previous thread.Thanks,SantoshCommentShareSantoshPandaanswered 4 years ago0Hi SantoshPanda,Every deployment shows "In Progress" on the console? Have you ever been able to get a deployment to work? You said you setup a proxy. Do you know if certain ports are restricted on your Internet connection? Even if you setup a proxy Greengrass still uses port 8883.https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-core.html#alpn-network-proxyYou can test if your ports are restricted by typing:curl -v https://greengrass.iot.us-west-2.amazonaws.com:8883You should see:Connected to greengrass.iot.us-west-2.amazonaws.com (+ip+) port 8883 (#0)The final message will be that there are bad certificates because none were specified in the command.Thanks,KeagenCommentShareKR-AWSanswered 4 years ago"


Question: Activity workflow threw ActivityTaskTimedOut but process is still running

"Hi,We currently have long running processes/jobs that are being executed via workflows. A workflow implementation calls the jobs to run. The call and the job runs asynchronously. By default, we have set the activity workflow to time-out after an hour.We recently saw an issue where if a job has been running for more than an hour, SWF throws an ActivityTaskTimedOutException in our workflow implementation. We expected that this should also cause the long running job to stop, but what happens is that only the workflow implementation catches this exception but job still continued running in the background.My question is, why did this not terminate the long running process? Is this an issue on how swf times-out processes, or do we need to have an implementation on our end that checks these time-out exceptions and terminate the process? If it is the latter, is there any guidance on how we should implement this?We badly need some help on this as we have been searching for a solution on this for some time now.Thank you!FollowComment"

Answer: "0Accepted AnswerJava doesn't provide a method to terminate a thread that executes an activity. So the actual activity implementation is not stopped on the timeout. It is the responsibility of the implementation to ensure that timeouts are not exceeded.It is recommended to heartbeat from a long running activity to ensure timely timeout if the worker goes down. It also helps with the activity cleanup as the heartbeat method throws an exception if an activity is already timed out or was canceled. See "Activity Heartbeat" section from the  https://docs.aws.amazon.com/amazonswf/latest/awsflowguide/errorhandling.html#test.cancellation.resources .CommentSharemfateevanswered 2 years ago0Hi Maxim, thanks for the response! Looks like we will have to do some implementation to fix this.I'm curious, by "Java doesn't provide a method to terminate..." did you mean that the underlying issue is within how Java language manages threads, or that the Java version for swf currently does not support terminating threads that executes an activity. Because if it is the latter, we might just have to wait and see if swf will have an update in the future that would support thisCommentSharedjonathananswered 2 years ago0It is a Java language issue: https://www.baeldung.com/java-thread-stopCommentSharemfateevanswered 2 years ago0I see, thanks for this!CommentSharedjonathananswered 2 years ago"


Question: Limitation on AWS CloudWatch metrics insights

"Hello, Is it true that AWS CloudWatch Metrics Insights can't query linked accounts for AWS organizations?FollowComment"

Answer: "0Accepted AnswerHello,Yes your understanding is absolutely correct unfortunately, at present AWS CloudWatch Metrics Insights can't query linked accounts for AWS organizations. To summarize it is not possible to run a CloudWatch Metric insight query on all linked accounts in an AWS Organization.As per my understanding there is no workaround. Apologies for any inconvenience caused due to this.CommentShareSUPPORT ENGINEERNirmal_Kariaanswered a year ago"


Question: help! Ssh can not be connected to SELinux

"Since the server was attacked, I started selinux, causing me to ssh not connect to the server now, prompt /bin/bash: Permission deniedEc2 server centos Now the console can't enter the server to shut down selinux, my files are not backed up, help, thank you very muchEdited by: handsomehaitao on Apr 2, 2019 10:52 PMFollowComment"

Answer: "0I'm assuming you used CentOS from the marketplace and thus cannot mount the volume as a secondary drive to another instance to undo your change, nor do you have a snapshot. (Otherwise, do one of those).It appears to can have AWS remove the marketplace code from a snapshot (https://forums.aws.amazon.com/message.jspa?messageID=541676). I would recommend doing it through a support ticket. Once complete, you can create a new volume that you can mount on a working system, disable SELinux, and reattach to the original instance.CommentSharersrcnoanswered 4 years ago"


Question: Inventory and CPU / memory utilization report

"I have been fiddling around with trying to export information on a couple of topics, but I have not had a good experience with Cloudwatch or AWS inventory.first, I want to export a list of my EC2 instances to an spreadsheet or cv by instance typesecond, I want to be able to see the average and Max CPU and memory utilization across my fleet for the last 30 days.  I want this data to be readable by each instance, not a aggregated graph with a bunch of lines.  how can I get this data out to look at?FollowComment"

Answer: "3You could create a comma separated file using the following command:aws ec2 describe-instances --query "Reservations[*].Instances[*].{Instance:InstanceId,Type:InstanceType,Name:Tags[?Key=='Name']|[0].Value,Status:State.Name}" --region eu-central-1 | jq -r '.[] | map({Instance,Type,Name,Status}) | (first | keys_unsorted) as $keys | map([to_entries[] | .value]) as $rows | $keys,$rows[] | @csv'Note that this is per region!This uses jq which can be installed using the instructions here:https://stedolan.github.io/jq/download/For the statistics you can also use a command line:aws cloudwatch get-metric-statistics --metric-name CPUUtilization --namespace AWS/EC2 --start-time 2021-11-19T08:00:00Z --end-time 2021-12-19T08:00:00Z --period 86400 --statistic Average --region eu-central-1Note that this is per region! This is for the whole fleet in a region. You can get this per InstanceId/InstanceType/ImageId/AutoScalingGroupName but that would require multiple calls. Here I used one month of data to fetch and specified I only want one datapoint per day at 8:00 (period=86400)Regards JaccoCommentShareJaccoPKanswered a year ago1You can utilize AWS Resources Groups & Tag Editor to get list of EC2 instances and export them.Resource Groups ScreenshotAs far as exporting metrics to CSV, this can't be done without some scripting. Fortunately for you there is a readymade solution documented on AWS official website and available to give it a try. If you know python you might be able to tweak it to your requirements. Check out Publish Amazon CloudWatch metrics to a CSV fileCommentSharehameedullahanswered a year ago"


Question: Using MSK as trigger to a Lambda with SASL/SCRAM Authentication

"Hi,I have set up a MSK cluster with SASL/SCRAM authentication. I have stored the username and password in a secret using AWS Secrets Manager. Now I am trying to set the topic in the MSK cluster as an event source to a Lambda function. In order to do so, I am following this documentation: https://aws.amazon.com/blogs/compute/using-amazon-msk-as-an-event-source-for-aws-lambda/However the above documentation is for unauthenticated protocol. So I tried to add the authentication and the secret. I also added a policy in the execution role of the Lambda function that lets it read the secret value:"Version": "2012-10-17","Statement": [    {        "Effect": "Allow",        "Action": [            "secretsmanager:*"        ],        "Resource": [            "arn:aws:secretsmanager:****:*******:secret:*"        ]    },    {        "Effect": "Allow",        "Action": "secretsmanager:ListSecrets",        "Resource": "*"    }]}When I am trying to add the trigger, I am getting the error:An error occurred when creating the trigger:Cannot access secret manager value arn:aws:secretsmanager:::secret:*******. Please ensure the role can perform the 'secretsmanager:GetSecretValue' action on your broker in IAM. (Service: AWSLambda; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: ****; Proxy: null)I am not able to understand this error since I have included in the policy all the Actions from "secretsmanager" on all the Resources in my account. Can someone help?FollowComment"

Answer: "0Hi,Thanks for your reply. Since I would like the MSK Cluster have public access, I cannot have clusters hosted in private subnets inside the VPC. I do have a NAT Gateway running in the Public subnet. I am able to produce and consume messages to the topic in this cluster from terminal, but when trying to redirect the message to the Lambda function as a payload, I am facing the above mentioned problem.CommentShareAWS-User-6509089answered a year agoTasio EXPERTa year agoAre you deploying the Lambda function with VPC access? Is that the same VPC where the MSK cluster is running?Share0Since Lambda must have access to your VPC to use Amazon MSK as a trigger, you need to make sure that, if the subnets you chose are private, there is connectivity to a NAT Gateway or to a Secrets Manager VPC Endpoint to access Secrets Manager.CommentShareEXPERTTasioanswered a year ago"


Question: Errors when installing Aws Replication Agent

I have checked connection to S3 endpoint with telnet and sucessfully. But got this error when run this .py file?Anyone ever face this error. Help meOracle Linux 6.2Python 3.5FollowComment

Answer: "0Hi ThereCheck that there is no VPC Endpoint Policy that might be blocking access from a specific vpc, iam role, region, etc.https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html#edit-vpc-endpoint-policy-s3CommentShareEXPERTMatt-Banswered 8 months agorePost-User-2684239 8 months agoI have migrated 4 servers with this VPC Endpoint and the Endpoint policy full acess too.Share0Thank you for helping me. After I install openssl and uprage python 3.7.14, i faced an other errors : urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate . I have faced this errors with Windows servers too and fixed it follow this link https://www.logicata.com/blog/ssl-certificate_verify_failed-error-when-trying-to-install-awsreplicationwindowsinstaller-exe-in-windows/ Now i am trying with Linux but still snot fixed. As I think, it's the same as the second reason you mentioned. Cant install packages certifi or something like this in servers. Do you have any idea?Try to set an environment variable pointing to the AWS CA pem file.Download the CA pem file from https://www.amazontrust.com/repository/AmazonRootCA1.pemcurl https://www.amazontrust.com/repository/AmazonRootCA1.pem --output /example_path/AmazonRootCA1.pemSet the env variable to the path of the .pem fileexport AWS_CA_BUNDLE =/example_path/AmazonRootCA1.pemCommentShareEXPERTMatt-Banswered 8 months agorePost-User-2684239 8 months agoIt doesn't work. I think need other ways to point to the certShare0OK, can you also tell me are you behind any sort of proxy/firewall that might be inspecting https traffic?  I see the CERTIFICATE_VERIFY_FAILED error at the end of the log.Possible cause: The AWS CLI doesn't trust your proxy's certificatePossible cause: Your configuration isn't pointing to the correct CA root certificate locationSee https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html#tshoot-certificate-verify-failedCommentShareEXPERTMatt-Banswered 8 months ago0Thank you for helping me. After I install openssl and uprage python 3.7.14, i faced an other errors : urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate . I have faced this errors with Windows servers too and fixed it follow this link https://www.logicata.com/blog/ssl-certificate_verify_failed-error-when-trying-to-install-awsreplicationwindowsinstaller-exe-in-windows/Now i am trying with Linux but still snot fixed.As I think, it's the same as the second reason you mentioned. Cant install packages certifi or something like this in servers. Do you have any idea?CommentSharerePost-User-2684239answered 8 months ago0You can also manually download the Amazon Root CAs from here. https://www.amazontrust.com/repository/CommentSharerePost-User-2656217answered a month ago"


Question: Why my sagemaker training job slower than notebook from studiolab.sagemaker.aws?

"I run neural network tensorflow train on studiolab. and I got:Epoch 145/40001941/1941 - 10s - ... - 10s/epoch - 5ms/stepthen I try to make a train job with script_mode with ml.c5.xlargeestimator = TensorFlow(entry_point='untitled.py',                       source_dir='./training/',                       instance_type='ml.c5.xlarge',                       instance_count=1,                       output_path="s3://sagemaker-[skip]",                       role=sagemaker.get_execution_role(),                       framework_version='2.8.0',                       py_version='py39',                       hyperparameters={...},                       metric_definitions=[...],                       script_mode=True)and its got:Epoch 19/40001941/1941 - 49s - ... - 49s/epoch - 25ms/stepWhy is it 5 times slower than studiolab notebook? Is it because instance type?FollowComment"

Answer: "0May I know which instance type you are using for training locally on your notebook instance. Including factors that influence training performance, hardware spec of the training node is very critical. You might be either getting bottlenecked on CPU, Storage or Memory.  See here for more detailsCommentShareEXPERTHrushi_Ganswered a year ago0I get the same issue when using the Sagemaker SDK (Tensorflow estimator) vs training using the Sagemaker with jupyter notebook.Sagemaker SDK (Tensorflow estimator) is much slower (3X slower) with exactly the same: compute power, model and data.CommentSharerePost-User-8560043answered 16 days ago"


Question: Billing and stored artifacts

"My site is a Jekyll generated static html site, and looking at the billing console for the month of July so far under Amplify I see:$0.023 per GB build artifacts stored  -  8.313 GigaBytes  -  $0.19The total size of the generated site should be well under 8.3gb, it's really just html and images. Not that it's a huge bill currently, but I definitely don't want it to pile up over time for no good reason. This is only two days into the month (last actual build was in June).Are we storing artifacts from previous builds? Is there a way to delete the older builds so I'm not being charged for it? Since there's no way to view the actual contents of S3 is there any way to get some kind of visibility into what this is?FollowComment"

Answer: "0All build artifacts are historically preserved, you can delete artifacts from old builds to remove them from your build artifact storage bill by using the AWS CLI. The following amplify command would delete the artifact history for jobs 1-3, copy/paste this in your command line:for i in {1..3}; do aws amplify delete-job --app-id add your app id here --branch-name add your branch name here --job-id $i; doneYou can get your app's Id and branch name by clicking on your hosted domain in the Amplify Console, the URL will start with: branchName.appIdYou can get a build's job Id by clicking on a branch and then clicking the "View build history" button in the upper right, the job Id is the build number, like 1, 2, 3, etc. Update the command's {1..3} with {startingJob..endingJob}, this will delete all build artifacts in that range.CommentShareAWS-User-4109112answered 4 years ago0That makes sense. Are the old builds kept indefinitely, or are they cleaned up after some amount of time?CommentSharepettazzanswered 4 years ago"


Question: Where can I find a comprehensive list of the billing codes used by cost explorer?

I am running down costs with cost explorer and codes are used for the services and usage type such as:USE2-Storage.SSD.200USE2-Storage.HDD.12.RC20I am routinely searching the web for an explanation to these codes. I particularly can't find the second code shown above anywhere. I know it is FSx but what is the RC20? Is there a link I can bookmark that comprehensively explains these?FollowComment

Answer: 0You can use the AWS pricing API or SDK to get information about the attributes for the services used in those billing codes:Here's is a link providing an example for Amazon FSx:https://www.saisci.com/aws/amazon-fsx-aws-pricing-api-all-attribute-names-and-values/CommentSharevtjeananswered 2 months ago


Question: Recommended method to Scan FSx for malware

"Does AWS scan the FSx filesystem for malware? If not, what is the recommended method of scanning the FSx filesystem for malware files uploaded to the shares?Should we map a drive to the fsx file system from a windows server and then scan the drive?Any other ways to do this? Thanks.FollowComment"

Answer: "0Hi BrianSei,Amazon FSx for Windows File Server offers file storage accessible via the industry-standard SMB protocol, so you can access your file system by mapping a network drive from an EC2 Windows instance with an antivirus solution, and scan the mapped network drive periodically. You can also use the antivirus solution on every client instance that accesses the file system to ensure scanning during the initial file write itself.Thank you,Amazon FSx teamCommentShareAWS-User-9252611answered 3 years ago0Amazon FSx for Windows File Server really could use a native anti-malware solution similar to what competitor public clouds offer.  Please add this as a feature request:)CommentShareDan Adamsanswered 5 months ago"


Question: Usage Plan / Rate limit / x-api-key

We currently require x-api-key header to be passed. This is how we control usage plan and limitOne customer of ours said that they cannot pass header parameters and they can only do urlencoded form parameters.My question is how can we enforce usage plan / limiting if x-api-key will be coming from form parameters?FollowComment

Answer: "0Hello:You can provide the key as the result of a lambda authorizer which will have access to headers, url parameters and query string, but not body parameters.https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-api-key-source.htmlRegards,BobCommentShareEXPERTAWS-User-0395676answered 4 years ago0Bob,Thank you for taking time to reply.I am the same person asking on this thread: https://forums.aws.amazon.com/thread.jspa?messageID=887871&#887871I could have asked both my questions in one thread but they are not related so I created a thread for each of my question.so based on this:  https://forums.aws.amazon.com/thread.jspa?messageID=887871&#887871I will be doing authorization in the Integration level... meaning I won't be having the Authorizer configured anymorehow am I going to enforce x-api-key without an authorizer?CommentShareJohnson965answered 4 years ago0Hello:Unfortunately there isn't currently an option that would support your use case. Apologies for the inconvenience.Regards,BobCommentShareEXPERTAWS-User-0395676answered 4 years ago"


Question: Glue + SageMaker Pip Packages

"My customer is looking to use Glue dev endpoints along with a SageMaker notebook. What I've noticed is that in Glue, a package, in this case scipy, will be listed as 1.4.1, but this will or won't match what you get in a sagemaker notebook dependent on kernel.conda_python3:!pip show scipyName: scipyVersion: 1.1.0Summary: SciPy: Scientific Library for PythonHome-page: https://www.scipy.orgAuthor: NoneAuthor-email: NoneLicense: BSDLocation: /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packagesRequires: Required-by: seaborn, scikit-learn, sagemakerconda_tensorflow_p36:!pip show scipyName: scipyVersion: 1.4.1Summary: SciPy: Scientific Library for PythonHome-page: https://www.scipy.orgAuthor: NoneAuthor-email: NoneLicense: BSDLocation: /home/ec2-user/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packagesRequires: numpyRequired-by: seaborn, scikit-learn, sagemaker, KerasIs there some sort of best practice to use a kernel that corresponds directly to what's installed on Glue?Separate not very useful question. I wasn't able activate the venv that Jupyter notebooks do via shell. Is it using a venv? How come I can't find the right activate script?FollowComment"

Answer: "0Accepted Answerconda_python3 and conda_tensorflow_p36 are local kernels on the SageMaker notebook instance while the Spark kernels execute remotely in the Glue Spark environment.Hence you are seeing different versions. The Glue Spark environment comes with 1.4.1 version of scipy. So when you use the PySpark (python) or Spark (scala) kernels and you will get the 1.4.1 version of scipy.If you use the default LifeCycle script that Glue SageMaker notebooks already come with, the connectivity to the Glue Dev endpoint should be in place. Note that the Glue SageMaker notebooks has a tag called 'aws-glue-dev-endpoint' that is used to identify which Glue Dev endpoint that particular notebook instance communicates with.The Spark kernels cannot be replicated via the python shell. Those kernels relay Spark commands via the Livy service to Spark on the Glue Dev endpoint using a Jupyter module called Sparkmagic.Ref: https://github.com/jupyter-incubator/sparkmagicCommentShareAWS-User-4669487answered 3 years ago"


Question: Can't decrypt Aurora MySQL Kinesis activity stream. (error:InvalidCiphertextException)

"I'm stuck developing an application using AWS SDK for Go, and I'm unable to decrypt the Kinesis activity stream records coming from my Aurora MySQL db.Following these steps to decrypt the record payloads, as I try to decrypt the record data key, I get error:InvalidCiphertextExceptionI'm currently trying to decrypt the record data key (in order to decrypt the payload) using my KMS key, via this call:kmsSession.Decrypt(&kms.DecryptInput{CiphertextBlob: []byte(record.Key)})With record.key being the data encryption key from the record I'm reading, as a string, parsed from the whole record.As far as I know, for this usecase there is no need to specify the Encryption Context, or my KMS keyID (although I tried it anyway, getting the same error).I'm opening the kms session as an user with AdministratorAccess policy group rights, and the user is in the Key Users and Key Administrator list for this KMS key.Also, does the record data key needs to be base-64 decoded?Any help is appreciated. Thanks!FollowComment"


Question: "Data sources, integrations and exporting"

"Hello there,so I've been using SageMaker Canvas for quite a few machine learning models now (lots of them are simple two-category model types, some of them are a time forecasting type of models). Because I am farely new to machine learning technologies and found Sagemaker Canvas few months ago I am not able to implement some serious machine learning algorithms through a SageMaker Studio. Canvas seems convenient and very simple for use but I am looking for a way of implementing a solution for my use case. I have an use case of having a automated machine learning model that could train on a latest dataset from a existing MySQL database.One idea for example is of automatic importing CSV files from a existing S3 bucket which would hold the latest datasets of data in a CSV file that some API would on specific time period export the data from a existing database in CSV format and upload to the S3 so that the Sagemaker Canvas could retrieve those files automatic and the training of a models would be automated as well. But I could not find any of the existing implementions of a similar solutions as well I think in this version, Canvas does not support any of those functionalities.It would be also a convenient if I could target SageMaker Canvas through a AWS SDK Javascript V3 client for importing datasets & exporting predictions, but I could not find any of the documentation as well. I have found a re:Post question regarding to SageMaker Canvas integrations possibilities, but the answer provided regarding to the SageMaker Autopilot API seems useless for my usecase. Are there any possibilities of targeting Sagemaker via API? As well are there any plans of provided several new data sources for importing the data sets (the ones QuickSight support - especially MySQL data source could be much appreciated)?I appreciate any information in advance, best regardsFollowComment"

Answer: "0Hello @MK, thanks for re:posting this question, and glad to see that you're starting your ML journey with SageMaker Canvas! :DAs you have had the possibility to test and discover so far, Canvas is meant to be a UI-driven ML workflow enabler for business analysts and developers, with little to no knowledge about ML. Therefore, at this stage, we do not provide an API or an SDK to connect and use Canvas capabilities. Sorry about that! This situation might change in the future, so stay tuned.Since that's the situation, I suggest you keep using Canvas for your "Exploratory data analysis" and your "Model experimentation" via Preview Models and Quick builds. I can tell you how you could automate some of these capabilities externally from SageMaker Canvas, with two capabilities:Create and automate your ETL workflow with AWS Glue DataBrew:AWS Glue DataBrew provides a UI interface for tabular data analysis and transformation, plus it supports a plethora of sources including JDBC connector to MySQL servers! Once you have defined your transformation (maybe even quality rules) you can create a Recipe job that can be scheduled to your preference, so that you have your ETL process automated. Learn how to automate/schedule Glue Databrew in this blog post.Train (and deploy) your model with SageMaker AutoPilot:SageMaker AutoPilot uses the same AutoML technology as SageMaker Canvas. This means that all you need to do is just to point to a specific S3 bucket where your CSV file is stored, and then call an API to get a model trained! You can set-up a Lambda function / StepFunction workload that triggers on a new file generated in the S3 bucket output of the recipe job above, which triggers an AutoPilot job. The output will be available either in SageMaker Studio or you can also describe the AutoML job to get the best candidate and deploy it using the information in InferenceContainers and calling, in order:create_model()create_endpoint_config() - a suggestion: use Serverless endpoint to save money!create_endpoint()Hope this helps!CommentSharedgallitellianswered a year ago"


Question: GameLift Hosting for existing game

"How can I migrate my existing game to Amazon GameLift completely?To put it in an example, let's have a Ludo Game, now if I want to use FleetIQ I know I have to make few changes to my code and call 3-5 API RegisterGameServer, UpdateGameServer, DeRegisterGameServer, etc to use the Fleet IQ feature.Now if I want to use full-fledge GameLift as my hosting solution, how many changes I will have to make at the code or API level in my existing code/game assuming I have a matchmaking process in place in my code itself and it does not have any AWS or GameLift related logic/code/process inside.FollowComment"

Answer: "1You can check out GameLift Realtime Server, which supports managed game hosting using NodeJS.You'd essentially provide GameLift with 1 or more javascript files, with hook definitions on what happens when player connects, when player sends a message, how to handle each message type, etc., and gamelift will take care of setting up the server process for you. See a sample code: https://docs.aws.amazon.com/gamelift/latest/developerguide/realtime-script.htmlThen, you'll need to setup game clients using a C# client SDK to connect to the server using IP Address/DNS Name and port. See: https://docs.aws.amazon.com/gamelift/latest/developerguide/realtime-client.html. You can also write your client in other languages like JavaScript, others on the forum have done this before, see: https://forums.awsgametech.com/t/gamelift-realtime-server-rangeerror-max-payload-size-exceeded/11071/4There is no officially supported NodeJS server SDK yet, (it's a popularly requested feature and is in our backlog), but you could SSH into a Realtime Server fleet instance and find the source code to see how Realtime server uses NodeJS to communicate with GameLift's onbox service.CommentSharerePost-User-7108006answered 2 years ago0Hi @REDACTEDUSERThese docs should answer your question directly:https://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-sdk-server.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-sdk-client.htmlThese docs may also be helpful:https://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-sdk-interactions.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-sdk-server-api-interaction-vsd.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/integration-testing-local.htmlIf you are using Lumberyard, Unity or Unreal, there are specific docs for those:https://docs.aws.amazon.com/gamelift/latest/developerguide/integration-engines-unity-using.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/integration-engines-setup-unreal.htmlhttps://docs.aws.amazon.com/gamelift/latest/developerguide/game-client-intro.htmlThere are also some good GitHub examples with step-by-step guide for you to follow to create an end-to-end game on GameLift:https://gamelift-persistent-workshop.workshop.aws/en/https://aws-samples.github.io/aws-gamelift-sample/en/intro/https://github.com/aws-samples/aws-gamelift-and-serverless-backend-sampleLet me know if you see any gaps in the documentations or if any parts are confusing to you, and we will fix them. If you have any other questions after going through these, feel free to ask!CommentSharerePost-User-7108006answered 2 years ago0To answer your question more directly... you'll probably need to do the following main things:Download and build the GameLift server SDK: https://aws.amazon.com/gamelift/getting-started/Include the Server SDK binaries into your game server, and setup calls such as initSDK(), processReady(), processEnding() etc. As well as defining the behaviors for when state changes, e.g. onStartGameSession, AcceptPlayerSession() etc. See: https://docs.aws.amazon.com/gamelift/latest/developerguide/gamelift-sdk-server-api-interaction-vsd.htmlSetup your game backend to call GameLift AWS SDK APIs such asStartGameSessionPlacement, StartMatchmaking, DescribeGameSession, DescribePlayerSession etc. DO NOT let your client assume your developer credentials and call GameLift directly for security purposes.You need to setup a CloudFormation stack containing GameLift builds, fleets, queues, etc.: https://github.com/aws-samples/aws-gamelift-and-serverless-backend-sampleSince you already have your own Matchmaking, you'll need to use queue placement to find the right server for your player. To do that, you need to setup a SNS notification to receive events when game session is placed for your players: https://docs.aws.amazon.com/gamelift/latest/developerguide/queue-notification.htmlYou need to configure your client to call your game backend to call GameLift to get the game session connection information, e.g. DescribeGameSessionPlacement returns the IP/port for your client to connect to the server processThere are other minor things that I've omitted/missed, but these should be bulk of the work.CommentSharerePost-User-7108006answered 2 years ago0I have my backend (GameServer) in Nodejs and the Server SDK is supported only for C and C#, so I think I won't be able to take the advantage of GameLift hosting. :slight_smile:CommentSharerePost-User-6659457answered 2 years ago0Thank you James for your Prompt response.CommentSharerePost-User-6659457answered 2 years ago"


Question: Reusing C# Lambda functions

I have a few lambdas written in C#. There are some functions which can be reused among these lambdas. How can I achieve that?FollowComment

Answer: "1Accepted AnswerYou can use the lambda code just as any other C# code, you can share base classes and references to common functionality, either by directly referencing the projects or by using NuGet packages.If all of your lambda functions are in the same repository extract the common code to classes/functions in a base project and reference that code from your Lambda functions or publish a NuGet package with the common code to be used by your Lmabda functions.CommentShareDror Helperanswered a year ago1Have a look at Lambda Layers - I think it's what you're looking for.CommentShareEXPERTBrettski-AWSanswered a year ago0It does not list C# in the list of languages.CommentSharewonderful worldanswered a year agoBrettski-AWS EXPERTa year agoThere are specific instructions for deploying C# layers in the documentation.Share"


Question: "Amazon Rekognition: When I create a new Stream Processor with S3Destination definition, this field is ignored"

"I followed the documentation https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CreateStreamProcessor.htmlThen I created a new Stream Processor from this JSON:{      Name: myStreamProcessor,      Input: {        KinesisVideoStream: {          Arn: 'arn:aws:kinesisvideo:eu-west-1:xxxxxxxxxxxx:stream/test-video-stream/xxxxxxxxxxxx'        }      },      Output: {        KinesisDataStream: {          Arn: 'arn:aws:kinesis:eu-west-1:xxxxxxxxxxxx:stream/ds-video-rekognition-test'        },        S3Destination: {          Bucket: 'matched-faces-test-bucket'        }      },      RoleArn: 'arn:aws:iam::xxxxxxxxxxxx:role/recognition-stream-processor-test',      Settings: {        FaceSearch: {          CollectionId: 'test-faces-test',          FaceMatchThreshold: 85.5        }      }    };But when I describe the Stream Processor:....  Output: {    KinesisDataStream: {      Arn: 'arn:aws:kinesis:eu-west-1:570379525661:stream/ds-video-rekognition-unai'    },    S3Destination: undefined  },....S3Destination is undefined!🤷🤷🤷I tried with aws-cli and with aws-skd for JavaScript with the same result.Thanks,FollowComment"

Answer: "0Hello,Thanks for reaching out to us. The create-stream-processor request should either contain Kinesis data stream or Amazon S3 bucket location under the request parameter Output but not both. If the stream processor is being created for label detection, S3Destination should be used for Output. If the stream processor is being created for face search, KinesisDataStream should be used instead. Since your request is for face search, only KinesisDataStream  should be provided in the request. Please check the doc https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CreateStreamProcessor.html#rekognition-CreateStreamProcessor-request-Output for more details.CommentShareAkash Yeleswarapuanswered a year ago"


Question: Amazon Route 53 weighted routing "stickiness"

"Customer want to start redirecting a small % of the traffic to AWS and the rest keep it on-prem, so they can learn and improve the platform. They will gradually increase the traffic until it reach 100%. How can they be assured using just Route 53 with weighted routing policy that the users that get directed to AWS always (while they are on the same IP) goes to the AWS platform and not to on-prem on the next DNS resolution?. They need to keep the sessions of the user on the first destination it was resolved. Does Route 53 weighted routing makes any sort of stickiness like other 3rd-party Load Balancers are able to do?.FollowComment"

Answer: "0Accepted AnswerAmazon Route 53 Weighted routing does not take the IP of the client into account.  It's pretty much random.   So, we don't have that ability directly.  That said, you can potentially do what you want using Geolocation, Latency-Based Routing or Geoproximity (GeoProximity requires TrafficFlow right now).So, instead of using weights, you can move one continent or country at a time.   Because that is based on client/resolver source IP, it will stay consistent as long as your config is consistent.CommentShareEXPERTgavinmcanswered 3 years agoEXPERTHernitoreviewed a year ago"


Question: Amazon VPC ENI plugin upgrade breaks readiness checks

"When upgrading the Amazon VPC ENI plugin from v1.11.2-eksbuild.1 to v1.12.2-eksbuild.1 two things get broken and need to be set again.kubectl set env daemonset aws-node -n kube-system ENABLE_POD_ENI=truekubectl patch daemonset aws-node -n kube-system \  -p '{"spec": {"template": {"spec": {"initContainers": [{"env":[{"name":"DISABLE_TCP_EARLY_DEMUX","value":"true"}],"name":"aws-vpc-cni-init"}]}}}}'fixes the problem, but in the meantime it leaves readiness checks (and potentially other things) broken.Why are those setting changes undone in this upgrade?  Previous upgrades were fine.  This is a blocker for doing this in production, we can't afford the downtime.  Why would you deliberately disable the whole purpose of the plugin?The alternative of running those on a loop during the whole process is not very clever at all.  Doing it manually is not feasible, even though we're fairly small with only about 20 clusters.  We use terraform to manage resources.FollowComment"


Question: AWS Control Tower in GovCloud

We are moving our product into AWS GovCloud to support a FedRAMP initiative. We traditionally utilizing Control Tower to support our private tenant environments and create new organizations/accounts as well as other controls. What is AWS's recommendation to do the same without Control Tower in the GovCloud environment?FollowComment

Answer: 0Try this: https://github.com/awslabs/compliant-framework-for-federal-and-dod-workloads-in-aws-govcloud-usAnd review this: https://aws.amazon.com/solutions/implementations/compliant-framework-for-federal-and-dod-workloads-in-aws-govcloud-us/Might be useful information in there.CommentSharebingobangoanswered 2 years ago


Question: MLNX_OFED with AmazonLinux

Hello. Does anyone know how to install MLNX_OFED on an instance running Amazon Linux? The Mellanox download website does not have a specific package so I tried with RHEL7.6 but it didn't work. Thanks.FollowComment

Answer: "0When you say "it didn't work," can you elaborate? What didn't work? Were there any errors? What were they?I assume you are using Amazon Linux 2, correct? Or are you using the older Amazon Linux AMI?Is your Amazon Linux instance updated?CommentSharetdcamanswered 4 years ago0Hi. Thanks. The instance is updated and running Amazon Linux 2. The error message is simply:"Current operation system is not supported!" and appears just a few seconds after starting the installation with 'mlnxofedinstall.'Edited by: afernandezody on Jul 25, 2019 7:16 AMCommentShareafodyanswered 4 years ago0Since the Mellanox products are centered around network and Infiniband drivers, it is highly likely that they are built against very specific kernel versions.  RHEL and Amazon Linux do not use the same kernels, so it is not surprising that the drivers would not install.You might contact Mellanox support, to find out their roadmap for providing support/packages for Amazon Linux.CommentShareAWS-User-6280256answered 4 years ago"


Question: Production Access

"Hi everyone.Sorry in advance if my question is silly. I made a request to move out of the sandbox and gain production access. The service says that you get an initial response within 24h for these kind of requests, but 3-4 days have already passed and i haven't gotten any response so far.Can you please advise me on how to proceed? I cant find any place where i can contact the service directly.Thanks in advanceFollowComment"


Question: SQS List - Messages available Sort is not working

"SQS List - Messages available Sort is not workingWhen sorting high to low , the actual SQS with the highest value is not at the top. Instead it seems to be a very random list .Anyone else see this problem? Am I missing something?ThanksFollowComment"

Answer: "0Are you talking about the AWS console?  When you list all the queues and sort by the "Messages available" column?  That should sort correctly, it does for me.  It's possible you're seeing a timing issue though as messages go between available and in flight.CommentShareEXPERTskinsmananswered 2 months agoRodney Lester 2 months agoIt's also possible you are not using a FIFO queue?SharerePost-User-2441365 2 months agoDefinitely not timing. I have over 2,000 queues. Can only page 100 at a time but some have 2,000 or more messages available but when sorting H-L I am presented with queues of just 60 availableShare"


Question: RDS "Restore from S3"  Stuck in "Creating" Status

"I am unable to do a "Restore from S3" operation, following the documentation here:https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Importing.htmlI have tried both Aurora and RDS MySQL, and they both get stuck in "Creating" status.  I can connect to the instance, and see that the imported data is there.  I have examined the logs on the RDS MySQL instance, and see no errors.  How can I get unstuck from "Creating" status?FollowComment"

Answer: "0Hello,Try running with:@with_norecovery = 0and see if that works for you. Setting it to 1 leaves it in a state where you can then restore additional transaction logs etc, or set up replication. If you want the database usable, you need to leave it in an online state, which setting it to 0 should do.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 9 months agoS Evett 9 months agoI don't think MySQL has an equivalent parameter to SQL Server's @with_norecovery.Share0I found the cause.  In the source database from which the backup was taken, there was a user "admin"@"localhost".  When I was choosing the DB create parameters, I accepted the default master user name which was:  "admin"@"localhost".  This conflict caused the restore to fail on creating the master user and then retry, apparently in an infinite loop.  I solved the problem simply by choosing a different name for the master user.CommentShareS Evettanswered 9 months ago"


Question: Babylon JS or Three JS for Sumerian

"As per the new AWS guidelines, its required to move existing scenes to Babylon.JS-AWS, no mention of ThreeJS, what is the recommendations by AWS if we need to develop new scenes, should we use Babylon or Three JS?FollowComment"

Answer: "0Great question! Our open source Amazon Sumerian Hosts JavaScript library on GitHub supports both BabylonJS and Three.js. So you can feel free to use either. We tend to recommend BabylonJS for the following reasons:BabylonJS offers the BabylonJS Editor which provides visual editing capabilities very similar to what Sumerian developers were used to with the Sumerian editor.We provide a plug-in for the BabylonJS Editor that makes it easy to deploy your BabylonJS scenes to AWS Amplify with just a few clicks, much like the Sumerian editor's one-click publish feature.We currently have a broader and clearer set of example applications for BabylonJS than we do for Three.js. You can see videos of those examples in action in this YouTube playlist. You'll find the code for those examples in the GitHub repository linked above.CommentShareKrisanswered 3 months ago"


Question: Connectivity issues across all ec2 related resources

"Any ec2 instance I create on my account in us-west-2, and place in a security group that DOES allow inbound SSH and all ICPM.  I'm only able to connect once via SSH, and for a very short period of time before being disconnected. I then become disconnected and I'm no longer able to connect or even ping the hosts. I've tested this across multiple different ec2 instances and created multiple security groups (all allowing inbound SSH).  The current ec2 instance I'm testing on has the ID of i-0fc771d3daa5baec6, this instance is in my default VPC (vpc-97ff5ef2) which does have an internet gateway (igw-534c8236). I'm wondering if there's some type of network issue up stream in my account, maybe my vpc settings are incorrect. Has anyone else seen similar issues? It's really odd because resources are available and I can connect to them, then suddenly I can't without changing anything.Also is there a way to reset my entire AWS account back to default, where it will resemble a newly created account?FollowComment"

Answer: "0My account was suspended in the past and I had recently submitted payments (several days ago) to bring it to current. Apparently one of the last payments was still pending  and the AWS Services team my account stuck a state of isolation, which was responsible for the connectivity issues.CommentSharecmananswered 4 years ago"


Question: Attach existing EBS Volumes (including root /os volume) to a new EC2 Windows instance as part of ASG

"I have a requirement. I have an Auto Scaling Group which has 2 or 3 EC2 instances. This setup is provisioned using Terraform.When one or more instances become unhealthy & get terminated, new instances are provisioned in their place. However, these instances get new EBS Volumes. But I want to reuse / attach the EBS volumes attached earlier to the terminated or unhealthy EC2 instances. I tried & searched a lot, but didn't get any good results. Can anyone please help me achieve this? I thought of using UserData, but I believe it runs with EC2 in Started state and hence cannot attach. From Terraform, I am not sure if this is possible. I even checked for Termination LifeCycle Hooks, but no luck. Please help.I saw a similar question, but the posted answer didn't seem very helpful to me.https://repost.aws/questions/QUfWhvtTJBRmuPdiK2W-FEWQ/how-are-ebs-volumes-in-an-unhealthy-instance-handled-when-a-new-instance-is-created-by-auto-scalingFollowComment"

Answer: "1It's best to separate your own data from the OS and instance configuration.  For OS & config you can bootstrap new ASG instances via UserData / cfn-init, or ideally as @takakuni said creating a golden (rich) AMI.  Your own data could be on a 2nd EBS volume you detach from old instances and attach to new in the same AZ, or the options @takakuni mentioned.CommentShareEXPERTskinsmananswered 3 months agoVenkatesh 3 months agoThank you @takakuni and @skinsman for your prompt response. However, my golden AMI is already there generated by Packer. Now, I need to attach the existing EBS volumes for other drives (say D, E, F etc) from the terminating /unhealthy instance to the new instance which are part of the ASG. I need to know how to identify these EBS volumes and how to attach them to the new instance. Any links / samples with detailed steps will be really helpful. I was thinking of using Termination LifeCycle Hooks, but I don't know how to call custom shell/powershell scripts from Terminating hook. I don't want to use a Lambda.Share1Unfortunately, I think it will be difficult.I think it is better to make golden AMI for the root /os volume as much as possible, and to save the data to Fsx for Windows or S3 if you want to share it after the instance is stopped.CommentSharetakakunianswered 3 months ago0The process for a launch hook is as follows; termination hooks are similar.  If a hook is active for instance launch:instance enters Pending: Wait state.AS (AutoScaling) sends info & token to the hook's notification target.AS waits until you tell it to continue or timeout ends (1 hour default).You can now perform custom setup etc. on the instance.If you need more time you can make the default wait time longer or record a heartbeat.AS puts instance in Pending: Proceed then InService states.That assumes hook returned "CONTINUE". If it returned "ABANDON" then the instance will instead be terminated.If you want to run something on an instance when its termination lifecycle hook is triggered you have to listen on the instance somehow, e.g. poll an SQS queue or wait for SNS to invoke an http endpoint.However on launch, if you know that a launch hook is in place, you can more simply:make sure the instance is configured with ASG name & hook namerun some setup stuffquery instance ID from metadataCall complete-lifecycle-action passing the above data in; if you have the instance ID you don't need the hook event token.CommentShareEXPERTskinsmananswered 3 months agoVenkatesh 3 months agoHi @skinsman, Thank you for your detailed response. However, I am looking for implementation details, preferably using Terraform.Share0Is it possible to call a custom script (say, Shell or PowerShell) from a Termination Lifecycle Hook which will tag the volumes and make them available? Then, can I have another Launching Lifecycle Hook which will allow me to execute a custom script (shell / powershell) to pick up these volumes and attach them to the new instance? Is this possible? If yes, please provide some detailed steps (with links). If not, what are the alternatives? Please helpCommentShareVenkateshanswered 3 months ago"


Question: How to run batch files using CodeDeploy appspec file hooks on windows ec2 instance?

"I am trying to run batch files to stop/start console apps within a windows ec2 instance upon deployment using CodeDeploy. For testing, I wrote a batch script to run a program that produces a simple text file. This batch script is listed under the ApplicationStart hook of my appspec.yml file. For some reason the text file is not being produced. My assumption is that the script isn't being run. Not sure on how to resolve this issue.appspec.yml:version: 0.0os: windowsfiles:  - source: /    destination: /source/apphooks:  ApplicationStart:    - location: .\application_start.bat       timeout: 300application_start.batSTART C:\source\app\ConsoleApp1\bin\Release\ConsoleApp1.exeI checked the script after deploying it and was able to run it manually just fine. What might I be missing?FollowCommentrePost-User-1532704 2 months agoI am also running through the free tier version of the ec2 and code deploy. Could this have caused an impact on the batch files not running?Share"

Answer: 0Here's a link to the CodeDeploy's Windows Deployment tutorial: https://docs.aws.amazon.com/codedeploy/latest/userguide/tutorials-windows.htmlCommentShareMichael Choianswered 2 months ago


Question: Hosted zone not configured properly for workmail

"Hi I recently transfered my domain to route 53 but am having an issue when setting up Workmail. I went through the setup process, and made it to a screen with a list of records that need to be created in my hosted zone. On this screen there is an alert with the following message:Amazon Route 53 hosted zone is not configured correctlyYou have a Route 53 hosted zone  for this domain, but you need to make it authoritative. To do that, see the Route 53 documentation .Additionally there are records listed on this page that are labeled as missing even though I have clearly created them in my hosted zone. There are a couple that are also in pending state. I tried looking online about making a domain authoritative but can not find anything of value. I also tried looking at the provided documentation, which was telling me to update my dns records, which are already updated.Any help would be greatly appreciated.Thank youFollowComment"

Answer: "1Hi.I also took a look at your domain configuration. Robin's point about multiple hosted zones is likely to be the problem. You can investigate it as follows:Run the following command:$ nslookup -type=NS <YOUR DOMAIN>It will return 4 name servers that are currently configured to be responsible (authoritative) for your domain name.Now, go back to the domain details page in WorkMail console and click "Route 53 hosted zone" link in the message. It should take you to the matching Route 53 hosted zone in your account. Check the value of "NS" records in that hosted zone. Does that match the 4 name servers that you see in the output above? If not, you have two options:a) Make this hosted zone the authoritative one for your domain name by updating the name servers for your domain. For that, you can refer to Route 53 documentation: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/MigratingDNS.htmlb) Find which hosted zone your actual NS records (in nslookup output) refers to. Go to that hosted zone and add the required DNS records. You may want to delete the other hosted zone to avoid any future conflicts.Do the NS records in your hosted zone match nslookup output? Then we need further investigation. Please take a screenshot of your Route 53 hosted zone and send me as a private message.Best regards,Murat from Amazon WorkMailCommentShareAWS-User-7632138answered 2 years ago0Hi,I'm sorry to hear you're experiencing problems with verifying your domain. I took a look at your account and checked the added custom domain. While the domain looks correctly configured with the correct NS and SOA records, I'm unable to resolve the required WorkMail records. It looks like there is some configuration error on the Route 53 side, but I cannot look into that.Do you have multiple Route 53 hosted zones for your domain?Kind regards,RobinCommentShareMODERATORrobinkawsanswered 2 years ago0murat-aws wrote:Hi.I also took a look at your domain configuration. Robin's point about multiple hosted zones is likely to be the problem. You can investigate it as follows:Run the following command:$ nslookup -type=NS <YOUR DOMAIN>It will return 4 name servers that are currently configured to be responsible (authoritative) for your domain name.Now, go back to the domain details page in WorkMail console and click "Route 53 hosted zone" link in the message. It should take you to the matching Route 53 hosted zone in your account. Check the value of "NS" records in that hosted zone. Does that match the 4 name servers that you see in the output above? If not, you have two options:a) Make this hosted zone the authoritative one for your domain name by updating the name servers for your domain. For that, you can refer to Route 53 documentation: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/MigratingDNS.htmlb) Find which hosted zone your actual NS records (in nslookup output) refers to. Go to that hosted zone and add the required DNS records. You may want to delete the other hosted zone to avoid any future conflicts.Do the NS records in your hosted zone match nslookup output? Then we need further investigation. Please take a screenshot of your Route 53 hosted zone and send me as a private message.Best regards,Murat from Amazon WorkMailYou guys were fully correct, and i think this thread my stray away from workmail related questions now.I had initially got my domain with google, and I configured an ecs instance with one of my AWS accounts lets say Account A. I dont remember the exact details but I had initially setup route 53 to point to this domain and I had copied the name servers from route 53 into google. Later I decided to transfer the domain to AWS from google onto a separate account lets say Account B. I started running into a bunch of issues, with the root cause that you had both outlined in your answers.So to fix, I wanted to make Account B have the default hosted zone for this domain. I deleted the hosted zone from Account A. On Account B I recorded the NS for the domain, and edited the NS record set in the hosted with the NS for the registered domain. Note that in the hosted zone details section the name servers are different, and are populated with the NS that the hosted zone was originally given. Could this pose any issue? It will not let me edit these values. Additionally doing an nslookup on my domain (cntautomation.com) gives me SERVFAIL after deleting the hosted zone in Account A. I have since deleted and tried re creating a hosted zone for this domain, but am having little luck.CommentShareambringoanswered 2 years ago0I guess I needed to change the registered domains to match the hosted zones rather than changing hosted zones domains to match the registered domains. Everything is working properly now I think. Thank you very much guysCommentShareambringoanswered 2 years ago0ambringo wrote:I guess I needed to change the registered domains to match the hosted zones rather than changing hosted zones domains to match the registered domains. Everything is working properly now I think. Thank you very much guysIndeed, you needed to update your name servers in your domain registrars to match with your hosted zone. With Route 53 you get randomly assigned name servers, so matching the other way around is not feasible.Glad that it worked out for you.Best,MuratCommentShareAWS-User-7632138answered 2 years ago"


Question: I want consolidated application logs running on AWS ECS with microservices architecture

"I want consolidated  logging of all Front End, Back End and ALB/Network logs for application  which is running on AWS ECS fargate with microservices architecture. Which one will be good out of Datadog or Splunk or ELK or AWS Centralized Logging or Prometheus / Grafana.FollowComment"

Answer: "0I would definitely make sure that you collect all metrics on AWS CloudWatch. For non-custom metrics, these are very cheap and integrate everywhere. Whether you visualize these in CW or in Grafana, doesn't matter. I'd recommend to stick as much as possible to a single source of truth when it comes to metrics.As for logs, with ECS + FireLens you can send your logs to CW Logs pretty much any destination (fluentbit powa!).But considering that the ALB logs will go to S3, I'd recommend FireHose / DataStream to log things in the same bucket / different buckets, same format. That way you can at least query the data with strong correlations with Athena for example.With 1 FireHose per source account and a central storage location, you can also log things in multi-accounts without any issues pretty easily. We have done / are in the process of doing just that: monitoring account, aggregating / accessing all others to provide near real-time view of what's happening across all our services, with each applications teams considered to be their own tenants.CommentShareJohnPrestonanswered a year ago"


Question: "MySQL RDS 'max_connections_per_hour' error, what are my options. This is not the 'max_connections' parameters, but per hour."

"I'm getting the following error for a DB I have. (db.t2.micro)“1226 (42000): User ‘?????????’ has exceeded the ‘max_connections_per_hour’ resource (current value: 85)”Note that this is not the typical 'max_connections' parameter.I do not see the parameter under my.sql.user (as if it doesn't exist), but I am getting the error nonetheless.Most of the solutions that I read about include upgrading the instance, which I am willing to do but for my current application it would be overkill. Most of my open and close connections are simple api request.85 connections in an hour doesn't seem like much, there has to be another solution.FollowComment"

Answer: "0MAX_CONNECTIONS_PER_HOUR is a MySQL per-user parameter.  To remove this, you can use:ALTER USER user@host MAX_CONNECTIONS_PER_HOUR 0;CommentShareEXPERTChris_Ganswered 9 months agorePost-User-2370401 9 months agoThis fixed it. It's weird, the USER table does not have this parameter visible, and the mysql feedback from running the query is '0 rows affected', but it did work.ShareChris_G EXPERT9 months agoYou can check this with SELECT User, max_connections, max_user_connections FROM mysql.user;.  The 0 rows affected is normal, this happens with any alters to users.  Can you mark this answer correct if it answered your question?Share"


Question: ontap - actual SSD capacity

"When I'm  trying to check the  actual SSD storage behind ONTAP FSx it show less capacity then expected:New ONTAP FSx 4TB - no data, only svm and volume are created.Fsx.....::> vserver show-aggregates                                        AvailableVserver        Aggregate      State         Size Type    SnapLock Type-------------- -------------- ------- ---------- ------- --------------svm01          aggr1          online      3.36TB ssd     non-snaplockWhat I'm missing ? Does it reserve some SSD space for other operations ?Thanks--AlexFollowComment"

Answer: "2Accepted AnswerHi Alex,There are some overheads in ONTAP which mean you don't get the full raw capacity of the disk as available space in an aggregate.Have a read of the link below, which explains various overheads and sizing considerations for NetApp ONTAP.https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/Space_UsageCommentShareTom-Banswered a year agoAlexLevin a year agoThanks, Tom  it make sense.I wish that this 15% difference is visible and announced  during FSx creation ...Share"


Question: Typescript - Types for SQS messages in lambda function

"In lambda function, which type/interface is the event when you get an SQS?. For example:export async function handler(event: any): Promise<string> {const records: any[] = event.Records;return "done";}I set the event type as any because I didn't find a type in "aws-sdk". Or there is not an official type?FollowComment"

Answer: "1Accepted AnswerI know this question is one year old already, and the solution might exist elsewhere as well, but I was able to easily type an SQS even using the unofficial @types/aws-lambda package. You can install it with npm install --save @types/aws-lambda.Then your handler could look like the following:import { SQSHandler, SQSEvent } from "aws-lambda";export const handler: SQSHandler = async (event: SQSEvent) {const records: any[] = event.Records;       .... // do work herereturn;}CommentSharegautieranswered 3 years ago"


Question: bandwidth consuptions and notification

"Hi, I am quite new to aws, thus might not sound appropriate at times:I have a simple requirement:I have a instance under lightscale and it allows me 1.5 TB of usage every month. The instance was setup on 5th January. now all I need is:At any point if I need to check the total bandwidth usage so far of the instance, how can I do that. I have seen things under instances Metrics/Incoming network traffic, but it shows all bars and no total sum figure for me to calculate of how much more I can use or is used. I understand that I will have to make a sum of incoming and outgoing for this, but I need to get to the figures to make a sum.I want to setup a notification for the usage (inbound and outbound traffic) to come to me on my email from time to time, e,g. I want an email to come to me every week with the total usage done so far for current monthFollowComment"

Answer: "1No, sorry, that's not yet available. It's been requested.Personally, I'd like to see total bandwidth utilization for instances & distributions ... basically anything that has a bandwidth cap of some sort.CommentShareDavid Ganswered a year ago1Lightsail has instance health metric alarms for Incoming and Outgoing Traffic available individually. However there is no metric currently available that combines the two traffic metrics together. I will take the feedback here to the team.See more about configuring alarms for instance metrics here: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-adding-instance-health-metric-alarmsThanks for using LightsailCommentShareAWS-User-7183595answered a year ago0There is currently no out of the box way to send you alarms but you can use a  following script to trigger an email to you on certain thresholds. Please see the article and script and see if you can tweak it to your usecase.https://aws.amazon.com/premiumsupport/knowledge-center/ec2-instances-calculate-network-bandwidth/CommentSharehameedullahanswered a year ago"


Question: Proofpoint blocking AWS SES shared IP addresses

"We're using the SES shared IP address pool.Since ~ 10 pm (Pacific) Friday, it seems that email service provider Proofpoint is blocking at least 2 of the SES shared IPs, 54.240.27.52 and 54.240.27.27, rejecting all SES email with "554 Blocked - see Blocked - see https://ipcheck.proofpoint.com/?ip=[ipaddr]"The SES documentation says that AWS manages the reputation of the shared IP addresses. How do we escalate this issue to AWS so AWS can either remove the Proofpoint blocked IP addresses from use and/or reach out to Proofpoint? Outside of purchasing dedicated IP addresses, any solutions?FollowComment"

Answer: "1Apart from the solutions you have indicatedProofpoint provides a self-service portal for email senders to request delisting from their blocklist. You can visit the Proofpoint IP and Domain Reputation Center at https://ipcheck.proofpoint.com/ to request delisting for your SES IP address. However, it is important to note that this process may take some time, and there is no guarantee that your request will be approved.CommentShareEXPERTsdtslmnanswered a month ago0The challenge with the dedicated SES pool is that other users might be impacting your SES operation.It is highly recommended to have at least 2 dedicated IP addresses setup for SES with proper SPF and DKIM records to avoid being impacted by other users of a shared pool.AWS does a very good job monitoring reputation but due to the short lifed nature of systems sending SPAM this can not always be detected fast enough.CommentSharerePost-User-5310971answered a month ago"


Question: AWS Cognito keys rotation frequency

"I am using Cognito JWKS URI https://cognito-idp.<Region>.amazonaws.com/<userPoolId>/.well-known/jwks.json to get public keys.In this service, it is a simple passthrough of the data from this URL to the client.I want to add a cache there. In the documentation, it is stated that keys are sometimes rotated but I cannot find any information on how often this happens.Is there any more conclusive information on how often these keys are rotated?FollowComment"

Answer: 0I don't think that is documented anywhere.We used Cognito for years and can't remember them ever being rotated. I would say it's safe to cache for at least 24hours or even more. But it's very hard to say.CommentShareJimmyDqvanswered 2 months ago


Question: OpenShift 4.12 UPI installation in AWS using terraform with predefined infrastructure

"Hi,We have a client that wants a very restricted installation of OpenShift 4.12 in AWS with predefined VPC, Security Groups... and doesn't want to grant some permissions to the installer account, for example:iam:PassRoleec2:AuthorizeSecurityGroupIngressec2:CreateSecurityGroupec2:RevokeSecurityGroupIngressThe problem of <Iam:PassRole = resource "*" is that they want to make it more granular on the specified resource.We are struggling with that because the resources (instance profiles) are created dynamically during installation so we don't know how to target them beforehand.And there's also the Security Group problem, we can't figure out how to target predefined SGs in the terraform files. But we are also close.Any help will be welcome :)ThxFollowComment"


Question: Is there an API call I can make to multiple VPCs that will return the network firewalls associated with it?

"I'm in progress of creating a custom config rule.I need to confirm upon creation of VPCs (doesn't have to be immediate) that there's a Network Firewall attached to the VPC.However, the issue is when I review the AWS documentation for boto3/cli calls to the API here https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-vpcs.htmlThe describe-vpcs method doesn't return the Network Firewalls. I see that the describe-firewalls action does; however, I'm wondering if there's a way to call an API to determine if there's a network firewall associated with a vpc fro ma vpc standpoint. Example, listing all vpcs in account and metadata within them that INCLUDES the network firewall.If there's no way then at least I know there's no better solution. But I want to confirm this before moving on to network-firewall api call;asFollowComment"

Answer: "0One option is to use the describe-vpc-endpoints call with these filters:vpc-endpoint-type = GatewayLoadBalancervpc-id = <The ID of the VPC you want to check>tag = Name=AWSNetworkFirewallManaged,Values=trueIf you get a result, it means there's an AWS Network Firewall in the VPC. It however doesn't imply that it's being used. You'll need to check the subnet route tables for that.CommentShareSanjeev Gopalanswered 7 months ago0Hello,We dont have direct options to check the VPC associated with AWS Network Firewall. We will be able to see it from VPC route table whether they have routes to Network Firewall endpoint to inspect the traffic. However, If you have multiple VPCs that are connected via Transit Gateway  and you have one AWS Network Firewall as centralized Egress inspection, you will not see all VPC  route tables pointing to AWS Network Firewall Endpoints.Other Options:-Assumptions:- All VPCs are connected via AWS Transit Gateway and Centralized Egress inspection.you can check all VPC's route table has default route (0.0.0.0/0-- TGW )on the Transit gateway route table, ensure default route 0.0.0.0/0 points to Inspection VPC TGW attachment (Inspection VPC --where you host AWS Network Firewall)In the Inspection VPC's route table, where you have TGW ENI, check for default route 0.0.0.0/0 points to AWS Network Firewall Endpoint.Please refer this blogCommentShareAWS-User-7653869answered 7 months ago"