Question: "Glue connection to Redshift Successful, but Glue job timed out"

"Hi, i have setup a connection from Glue to redshift using the connection page. I tested it and it was successful.then i created Visual ETL job and chosen the above-mentioned connection from the dropdown list.When i ran the job, the connection timed out and gave the below error:2023-05-16 18:16:52,896 ERROR [main] glueexceptionanalysis.GlueExceptionAnalysisListener (Logging.scala:logError(9)): [Glue Exception Analysis] {    "Event": "GlueETLJobExceptionEvent",    "Timestamp": 1684261012891,    "Failure Reason": "Traceback (most recent call last):\n       File \"/tmp/rds_order_to_redshift\", line 275, in <module>\n    transformation_ctx=\"AmazonRedshift_node3\",\n   File \"/opt/amazon/lib/python3.6/site-packages/awsglue/dynamicframe.py\", line 802, in from_options\n     format_options, transformation_ctx)\n  File \"/opt/amazon/lib/python3.6/site-packages/awsglue/context.py\", line 331, in write_dynamic_frame_from_options\n format, format_options, transformation_ctx)\n   File \"/opt/amazon/lib/python3.6/site-packages/awsglue/context.py\", line 347, in write_from_options\n     sink = self.getSink(connection_type, format, transformation_ctx, **new_options)\n   File \"/opt/amazon/lib/python3.6/site-packages/awsglue/context.py\", line 310, in getSink\n     makeOptions(self._sc, options), transformation_ctx)\n  File \"/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py\", line 1305, in __call__\n answer, self.gateway_client, self.target_id, self.name)\n   File \"/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py\", line 111, in deco\n     return f(*a, **kw)\n  File \"/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py\", line 328, in get_return_value\n     format(target_id, \".\", name), value)\npy4j.protocol.Py4JJavaError: An error occurred while calling o89.getSink.\n: java.sql.SQLException: [Amazon](500150)  Error setting/closing connection: Connection timed out.\n\t at com.amazon.redshift.client.PGClient.connect(Unknown Source)\n\t at com.amazon.redshift.client.PGClient.<init>(Unknown Source)\n\t at com.amazon.redshift.core.PGJDBCConnection.connect(Unknown Source)\n\t at com.amazon.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)\n\t at com.amazon.jdbc.common.AbstractDriver.connect(Unknown Source)\n\t at com.amazon.redshift.jdbc.Driver.connect(Unknown Source)\n\t at com.amazonaws.services.glue.util.JDBCWrapper$.$anonfun$connectionProperties$5(JDBCUtils.scala:973)\n\t at com.amazonaws.services.glue.util.JDBCWrapper$.$anonfun$connectWithSSLAttempt$2(JDBCUtils.scala:925)\n\t at scala.Option.getOrElse(Option.scala:121)\n\t at com.amazonaws.services.glue.util.JDBCWrapper$.$anonfun$connectWithSSLAttempt$1(JDBCUtils.scala:925)\n\t at scala.Option.getOrElse(Option.scala:121)\n\tat com.amazonaws.services.glue.util.JDBCWrapper$.connectWithSSLAttempt(JDBCUtils.scala:925)\n\t at com.amazonaws.services.glue.util.JDBCWrapper$.connectionProperties(JDBCUtils.scala:969)\n\t at com.amazonaws.services.glue.util.JDBCWrapper.connectionProperties$lzycompute(JDBCUtils.scala:739)\n\t at com.amazonaws.services.glue.util.JDBCWrapper.connectionProperties(JDBCUtils.scala:739)\n\t at com.amazonaws.services.glue.util.JDBCWrapper.getRawConnection(JDBCUtils.scala:752)\n\t at com.amazonaws.services.glue.RedshiftDataSink.<init>(RedshiftDataSink.scala:41)\n\t at com.amazonaws.services.glue.GlueContext.getSink(GlueContext.scala:1059)\n\t at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\t at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\t at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\t at java.lang.reflect.Method.invoke(Method.java:498)\n\t at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\n\t at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)\n\t at py4j.Gateway.invoke(Gateway.java:282)\n\tat py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\n\t at py4j.commands.CallCommand.execute(CallCommand.java:79)\n\tat py4j.GatewayConnection.run(GatewayConnection.java:238)\n Caused by: com.amazon.support.exceptions.GeneralException: [Amazon](500150) Error setting/closing connection: Connection timed out.\n\t ... 28 more\nCaused by: java.net.ConnectException: Connection timed out\n\t at sun.nio.ch.Net.connect0(Native Method)\n\tat sun.nio.ch.Net.connect(Net.java:482)\n\tat sun.nio.ch.Net.connect(Net.java:474)\n\tat sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647)\n\tat sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:107)\n\tat com.amazon.redshift.client.PGClient.connect(Unknown Source)\n\tat com.amazon.redshift.client.PGClient.<init>(Unknown Source)\n\tat com.amazon.redshift.core.PGJDBCConnection.connect(Unknown Source)\n\tat com.amazon.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)\n\tat com.amazon.jdbc.common.AbstractDriver.connect(Unknown Source)\n\tat com.amazon.redshift.jdbc.Driver.connect(Unknown Source)\n\tat com.amazonaws.services.glue.util.JDBCWrapper$.$anonfun$connectionProperties$5(JDBCUtils.scala:973)\n\tat com.amazonaws.services.glue.util.JDBCWrapper$.$anonfun$connectWithSSLAttempt$2(JDBCUtils.scala:925)\n\tat scala.Option.getOrElse(Option.scala:121)\n\tat com.amazonaws.services.glue.util.JDBCWrapper$.$anonfun$connectWithSSLAttempt$1(JDBCUtils.scala:925)\n\tat scala.Option.getOrElse(Option.scala:121)\n\tat com.amazonaws.services.glue.util.JDBCWrapper$.connectWithSSLAttempt(JDBCUtils.scala:925)\n\tat com.amazonaws.services.glue.util.JDBCWrapper$.connectionProperties(JDBCUtils.scala:969)\n\tat com.amazonaws.services.glue.util.JDBCWrapper.connectionProperties$lzycompute(JDBCUtils.scala:739)\n\tat com.amazonaws.services.glue.util.JDBCWrapper.connectionProperties(JDBCUtils.scala:739)\n\tat com.amazonaws.services.glue.util.JDBCWrapper.getRawConnection(JDBCUtils.scala:752)\n\tat com.amazonaws.services.glue.RedshiftDataSink.<init>(RedshiftDataSink.scala:41)\n\tat com.amazonaws.services.glue.GlueContext.getSink(GlueContext.scala:1059)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\n\tat py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)\n\tat py4j.Gateway.invoke(Gateway.java:282)\n\tat py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\n\tat py4j.commands.CallCommand.execute(CallCommand.java:79)\n\tat py4j.GatewayConnection.run(GatewayConnection.java:238)\n\tat java.lang.Thread.run(Thread.java:750)\n",    "Stack Trace": [        {            "Declaring Class": "get_return_value",            "Method Name": "format(target_id, \".\", name), value)",            "File Name": "/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py",            "Line Number": 328        },        {            "Declaring Class": "deco",            "Method Name": "return f(*a, **kw)",            "File Name": "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py",            "Line Number": 111        },        {            "Declaring Class": "__call__",            "Method Name": "answer, self.gateway_client, self.target_id, self.name)",            "File Name": "/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py",            "Line Number": 1305        },        {            "Declaring Class": "getSink",            "Method Name": "makeOptions(self._sc, options), transformation_ctx)",            "File Name": "/opt/amazon/lib/python3.6/site-packages/awsglue/context.py",            "Line Number": 310        },        {            "Declaring Class": "write_from_options",            "Method Name": "sink = self.getSink(connection_type, format, transformation_ctx, **new_options)",            "File Name": "/opt/amazon/lib/python3.6/site-packages/awsglue/context.py",            "Line Number": 347        },        {            "Declaring Class": "write_dynamic_frame_from_options",            "Method Name": "format, format_options, transformation_ctx)",            "File Name": "/opt/amazon/lib/python3.6/site-packages/awsglue/context.py",            "Line Number": 331        },        {            "Declaring Class": "from_options",            "Method Name": "format_options, transformation_ctx)",            "File Name": "/opt/amazon/lib/python3.6/site-packages/awsglue/dynamicframe.py",            "Line Number": 802        },        {            "Declaring Class": "<module>",            "Method Name": "transformation_ctx=\"AmazonRedshift_node3\",",            "File Name": "/tmp/rds_order_to_redshift",            "Line Number": 275        }    ],    "Last Executed Line number": 275,    "script": "rds_order_to_redshift"}FollowComment"

Answer: "0Accepted AnswerI've fixed this. my ETL job moves data from RDS to Redshift. Since the redshift connection alone works, and , the RDS connection alone works. I suspected that Redshift should allow RDS to talk to it.So I've added the security group of the RDS into the inbound rules of the Redshift security group.CommentSharerePost-User-1993102answered 14 days ago"


Question: Transit Gateway connection from customer gateway using Palo Alto without BGP

"A customer needs to connect his site to a Transit Gateway (which will be connected to several VPCs) using a Palo Alto firewall on premise, following this standard architecture (https://docs.aws.amazon.com/en_pv/vpc/latest/tgw/transit-gateway-isolated-shared.html). Key aspect is they want to connect without using BGP.As described in this AWS doc (https://docs.aws.amazon.com/vpc/latest/adminguide/GenericConfigNoBGP.html), it should be possible to do so in environments with other manufacturers. However, Palo Alto is not listed as one of the devices with a template for not using BGP.The Palo Alto documentation (https://docs.paloaltonetworks.com/prisma/prisma-access/prisma-access-panorama-integration/secure-your-public-cloud-deployment-with-prisma-access/onboard-aws-vpcs) describes how to connect to such transit gateway using either static routes, default routes or BGP routing, so I assume it would be possible to do so.Anybody has faced a similar requirement from a customer and could confirm or deny my proposal?FollowComment"

Answer: 0Accepted AnswerThe Palo Alto's own documentation seems to have a good guide how to make the static route VPN connection. It even includes guide how to setup "tunnel monitor" that will failover the connection if the primary VPN connection goes down.The BGP is used to dynamically advertise networks between AWS and customer. Plus it provides this automatic failover without need to configure it separately.I have not implemented this static route Palo Alto setup but previously I have administrated BGP based Palo Alto connections to AWS. So for me the guide made sense and I don't see any reasons why it wouldn't work.But as said without the BGP they will need to maintain the static routes both on AWS VPC and Palo Alto side.CommentShareEXPERTToni_Sanswered 4 years ago


Question: Updating channel configurations via the SDK

"Hi there,One of the more challenging parts of the MediaLive API for me thus far has been the task of updating a Channel.Say I want to take an existing channel and make a request to add another type of output to it. This means I will need to build another OutputGroup in my code, and also traverse the existing configuration to update AudioDescription objects, VideoDescription objects, and OutputDestinations in order to make an acceptable "updateChannel" request against the API.Is there a simpler way of using the API to make high-level changes, such as "add an output" or "remove an output"?FollowComment"

Answer: "0Accepted AnswerchrisatmbAt this time the SDK (and API) does not have a shortcut option to update a channel with a new output group or new output to an existing output group. As you've seen, the current method is to perform a describeChannel to obtain the current channel configuration for the channel, update the config, and then push the new complete, updated channel config with the updateChannel option to the channel.RegardsCommentShareAWS-User-5437070answered 2 years ago"


Question: Multiple EKS audit logs files

"I have enabled EKS audit logs, in cloud logs there are two files getting generated for audit logs alone, each getting written with logs in parallel.Why there are two files generated, is there any difference between them.FollowComment"

Answer: 0Two log streams are created in parallel when you enabled Amazon EKS audit logs. They are named similar to: kube-apiserver-audit-<random-string>. There are two because Amazon EKS runs two API servers for each EKS cluster to maintain availability zone level redundancy."EKS runs a minimum of two API server nodes in distinct Availability Zones (AZs) within in AWS region." Source: https://aws.github.io/aws-eks-best-practices/reliability/docs/controlplane/CommentShareAWS-Harrisonanswered a month ago


Question: Custom port possible with Transfer for SFTP?

"We are migrating an existing on-prem SFTP server to AWS Transfer for SFTP, however the old server was setup to only accept connections on port 2222. We are hoping to make a seamless transfer to AWS behind the scenes without having users need to update anything (we use user/pass auth), but from what I can see, only port 22 is possible...Is it possible to use a custom port or listen on multiple ports (e.g. 2222 for legacy users, 22 for go-forward users)?Thanks,T.FollowComment"

Answer: "0Accepted AnswerHi,You can set up to use a custom port by fronting a server's VPC endpoint with an NLB. Here's a post on "lift and shift" of SFTP servers that provides details on how you can set up this architecture (under "IP address portability" section step 4): https://aws.amazon.com/blogs/storage/lift-and-shift-migration-of-sftp-servers-to-aws/Thanks,SmithaCommentShareAWS-User-9826533answered 3 years ago0Ah that makes total sense - thank you!CommentShareTully-BNSanswered 3 years ago"


Question: How to avoid upload failure due to account logging off overnight?

"I'm uploading one folder with 10k files (3TB) through https://console.aws.amazon.com/s3/. It estimates that it will take 2-3 days. However, at some point during the night the account is logged off and the upload interrupted. When I log in again, it does not offer me to continue the upload. When I try to reupload the files, it starts from the beggining, overwirting the files already uploaded and not skipping them. Do you have any suggestions?FollowComment"

Answer: "1Use the AWS CLI and IAM User access keys to copy the files. Attach a least privilege policy to the user that only allow actions in that bucket.CommentShareEXPERTkentradanswered 4 days agoEXPERTChris_Greviewed 4 days ago0Wow, this is way too complex and long. There must be a way to skip already uploaded files or resume upload through the console...Anyways, I'm trying your suggestion but I'm stuck in this step: https://docs.aws.amazon.com/singlesignon/latest/userguide/get-started-connect-id-source-ad-idp-specify-user.htmlI can't find the manage sync in actions. I only have Customize AWS URL portal and Change Identity Source.CommentSharerePost-User-1696235answered 6 hours ago"


Question: Corrupt data landing in S3 via aws_s3.query_export_to_s3 extension

"When using aws_s3.query_export_to_s3, on Aurora Postgres, version 11, data is reliably and predictably corrupted at certain positions of the payloads that land in s3.It seems that when data falls over a certain byte length, it is reset and resent mid-stream to s3.We have been able to reproduce this positively, as well as reproduce a negative as well; ensuring that data falls at powers of 2 on even byte lengths seems to prevent this issue from occurring.As the length of data increases, as does the frequency of the corruption. For small data sets, it may never surface.You will see upon running this SQL that it occurs once in 100,000 records with 128_9 byte length records, twice for 256_9, and 4 times for 512+9, and so-on, where each record looks like {"A":""}\n where \n is a new line control character, amounting to 9 extra bytes on top of the value in the JSON objects.Has anyone else ever seen anything like this happening with S3?Here's a link to the SQL file:https://gist.github.com/gordol/85226ac917de7984010c507591078792FollowComment"

Answer: "0To be clear, this happens every 10MB.For example, in the 256 byte value variant, the 39569th line fails to fully send and truncates on the 240th byte.The overhead for the JSON is 9 bytes for {"A": ""} where data goes in the quotes.We can easily do the math on this, using the line numbers, plus the payload size, plus the json overhead.((39568 * (256_9)) _ 240) / 1024 / 1024 = 10Same for the 128byte variant... The 76539th row fails and data truncates on the 54th byte of the 76539th row.((76538 * (128_9) ) _ 54) / 1024 / 1024 = 10It's easily reproducible...Edited by: GordoL on Nov 4, 2020 3:50 PMCommentShareGordoLanswered 3 years ago0This is resolved by upgrading from Aurora Postgres 3.1.0/11.6 to 3.3/11.8. The issue is not present in the latest versionCommentShareGordoLanswered 3 years ago"


Question: How to give a user access only to a specific group of cloudwatch logs

"HelloI have created a user and I want to give him the permission to access only a specific group of cloudwatch logs.For this, I have assigned the following strategy:{    "Version": "2012-10-17",    "Statement": [        {    "Effect": "Allow",            "Action": [                "logs:Describe*",                "logs:Get*",                "logs:List*",                "logs:StartQuery",                "logs:StopQuery",                "logs:TestMetricFilter",                "logs:FilterLogEvents"            ],            "Resource": "arn:aws:logs:<aws-region>:<accountId>:log-group:/ecs/copro*"        }    ]}When the user tries to access cloudwatch this is the error message he gets :User: arn:aws:iam::<accountId>:user/poreck is not authorized to perform: logs:DescribeLogGroups on resource: arn:aws:logs:<aws-region>:<accountId>:log-group::log-stream: because no identity-based policy allows the logs:DescribeLogGroups actionI understand that the action logs:DescribeLogGroups is not applicable to a specific resource. Because if in the strategy I replace the value of the **Resource **field with "*", the user has access to all the log groups and I don't want that.My question is to know if there is a way to bypass this blocking by modifying the strategy. Or if there is a simple external solution that consists in retrieving these specific log groups.Thanks for any helpSincerelyFollowCommentDmitry Balabanov a year agoCould you please elaborate how that particular user wants to access the logs? Via the AWS Management Console? Via AWS CLI? AWS SDK for a programming language?Share"

Answer: "0The ARN for CloudWatch Log Groups follows this pattern:arn:aws:logs:us-east-1:123456789012:log-group:/loggroupname:*Note the last :*That references each log stream. Please try modifying your policy as such:{    "Version": "2012-10-17",    "Statement": [        {    "Effect": "Allow",            "Action": [                "logs:Describe*",                "logs:Get*",                "logs:List*",                "logs:StartQuery",                "logs:StopQuery",                "logs:TestMetricFilter",                "logs:FilterLogEvents"            ],            "Resource": "arn:aws:logs:<aws-region>:<accountId>:log-group:<full-log-group-name>:*"        }    ]}CommentShareAWS-Harrisonanswered a month ago"


Question: Can I use eslint as code formatter

"Hi,I installed eslint and set up .eslintrc. it works fine when I use it from terminal.But, it seems to be in error if I set "npx eslint --fix" to code formatter in Preference.When I save files in the project, "SAVING..." button is appeared next to "Run" button, after a while it turns to "ALL CHANGES SAVED" but nothing changed even if the code violate eslint's rule.Any suggestions would be greatly appreciated.FollowComment"

Answer: "0Accepted Answercd doesn't work because cloud9 gives a relative file path which is wrong after cd.Tryf=$(readlink -f "$file");cd $(dirname "$f");  npx eslint --fix $f  CommentShareharutyunanswered 4 years ago0This can be caused by missing "$file" argument to eslint, or the correct version of npx not being on PATHto debug the issue you can add&> ~/eslint-logto the end of the command to see the output in eslint-logFor me the commandPATH=~/.nvm/versions/node/v10.15.3/bin:$PATH npx eslint --fix "$file" works.CommentShareharutyunanswered 4 years ago0Hi harutyun,Thank you for your reply.I checked the output from eslint by the way that you post, PATH seems to be OK but eslint can not find plugin.I checked current directory with followin code formatter settingpwd > ~/test.logthen I got  following result$ cat test.log/home/ec2-user/environmentI have several project directory under ~/environment, so it seems that npx was issued outside of node project directory.But, it is hard to solve.following 2 cmd does not work because code formatter setting will be sanitized(cd dirname $file; npx eslint --fix $file)(cd dirname $file&& npx eslint --fix $file)Then env does not support "-C" in amazon linux, following setting will cause "invalid option" errorenv -C dirname $file npx eslint --fix $filedoes anyone has any suggestions to this problem?CommentSharesogoanswered 4 years ago0Hi harutyun,Thanks again, your solution works perfectly for me.CommentSharesogoanswered 4 years ago0this gives me the following error:Error running code formatter for javascript: f=$(readlink not found, please check your project settingsi've tried some other things, but basically the error it shows always the same error.... not found.i've managed to run this in my command line, just fine;file=/home/ec2-user/environment/git/llinkid/web/js/components/curriculum/goalsTab/index.js; f=$(readlink -f "$file");cd $(dirname "$f"); npx eslint --fix $fit seems like it crashes whenever i add npx in the command of the formatter. i managed to test everything else that seems fine.edit: for some reason, closing the browser and opening c9 again fixed it...Edited by: Stefanvds on Oct 22, 2019 10:58 PMCommentShareStefanvdsanswered 4 years ago0Thanks so much, this worked for me as well!CommentSharecarlschmidtanswered 4 years ago"


Question: is there any boto3 API available to share the appstream image between the AWS accounts.

we have the appstream image in one account and we want to share this appstream image across the other aws accounts we have under the root OU.is there any boto3 API available to share the appstream image between the AWS accounts?FollowComment

Answer: "2Yes, you can use update_image_permissions.https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/appstream.html#AppStream.Client.update_image_permissionsCommentShareJeremy_Sanswered a year agoAWS-User-2038220 a year agothanks for sharing it. i tested it, it is working fine. but with this, i can share the image to only one account. is there is any way to share the image with multiple accounts at a time.ShareJeremy_S a year agoThe method only takes one account ID. You will have to loop or iterate through and make the call with each account IDShareJeremy_S a year agoFor example here is some pseudocode:accounts = [11111111111111, 22222222222222, 33333333333333]client = boto3.client('appstream')def share_image(account):client.update_image_permissions(Name='string',SharedAccountId=account,ImagePermissions={'allowFleet': True|False,'allowImageBuilder': True|False})for account in accounts:share_image(account)Share1You can use the AppStream 2.0 UpdateImagePermissions API (Boto3 doc) to add permissions for another AWS account ID to access the image, either with image builder permissions, fleet permissions, or both. This allows you to share an image within the same region with another AWS account.You can learn more about the specific permissions in the Administrator Guide - Administer your Amazon AppStream 2.0 Images.Hope this helps.CommentShareEXPERTMuraliAtAWSanswered a year agoAWS-User-2038220 a year agohi Murali,thanks for sharing it. i tested it, it is working fine. but with this, i can share the image to only one account. is there is any way to share the image with multiple accounts at a time.Share"


Question: AWS Robomaker Simulation error

"Hey all, I am trying run a simulation from my own package in Gazebo. Bu getting the following error.... logging to /home/ubuntu/.ros/log/ebb4b9b0-f856-11ea-9922-028fb57b8d38/roslaunch-ip-172-31-20-103-3021.logChecking log directory for disk usage. This may take awhile.Press Ctrl-C to interruptDone checking log file disk usage. Usage is <1GB.Traceback (most recent call last):File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/init.py", line 308, in mainp.start()File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/parent.py", line 268, in startself._start_infrastructure()File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/parent.py", line 217, in _start_infrastructureself._load_config()File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/parent.py", line 132, in _load_configroslaunch_strs=self.roslaunch_strs, verbose=self.verbose)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/config.py", line 451, in load_config_defaultloader.load(f, config, verbose=verbose)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 750, in loadself._load_launch(launch, ros_config, is_core=core, filename=filename, argv=argv, verbose=verbose)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 722, in _load_launchself._recurse_load(ros_config, launch.childNodes, self.root_context, None, is_core, verbose)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 686, in _recurse_loadval = self._include_tag(tag, context, ros_config, default_machine, is_core, verbose)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 95, in callreturn f(*args, **kwds)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 627, in _include_tagdefault_machine, is_core, verbose)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 686, in _recurse_loadval = self._include_tag(tag, context, ros_config, default_machine, is_core, verbose)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 95, in callreturn f(*args, **kwds)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 589, in _include_taginc_filename = self.resolve_args(tag.attributes['file'].value, context)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 183, in resolve_argsreturn substitution_args.resolve_args(args, context=context.resolve_dict, resolve_anon=self.resolve_anon)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 370, in resolve_argsresolved = _resolve_args(resolved, context, resolve_anon, commands)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 383, in _resolve_argsresolved = commands[command](resolved, a, args, context)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 151, in _findsource_path_to_packages=source_path_to_packages)File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 197, in _find_executablefull_path = _get_executable_path(rp.get_path(args[0]), path)File "/usr/lib/python2.7/dist-packages/rospkg/rospack.py", line 207, in get_pathraise ResourceNotFound(name, ros_paths=self._ros_paths)ResourceNotFound: gazebo_rosROS path [0]=/opt/ros/kinetic/share/rosROS path [1]=/home/ubuntu/environment/robot_ws/install/cr3_moveit_config/shareROS path [2]=/home/ubuntu/environment/robot_ws/install/cr3/shareROS path [3]=/opt/ros/kinetic/shareFollowComment"

Answer: "0Hi, sorry you're having an error. It looks like it's unable to find the "gazebo_ros" package on the system. Can you provide me some information? Are you including "<depend> gazebo_ros </depend>" in your package.xml file? Are you running colcon bundle to create your simulation application?If the answer to those questions is yes, can you provide me a list of steps you followed to build, bundle and run your simulation? I need more details in order to help you.CommentShareAWS-User-1923695answered 3 years ago0Hi yes. I did add the depend code in package file and it is working now. Thanks a lot.CommentShareAdit1394answered 3 years ago"


Question: general log vs audit log

"A customer was asked by an auditor to enable general logs (general_log) in RDS Aurora MySQL. Recently we had an issue with this cluster and we enabled audit logs (server_audit_logs).After checking some sources it looks like if the audit log has QUERY, QUERY_DCL, QUERY_DDL, QUERY_DML enabled it should have the same info as the general log.Is it recommended to use Audit log instead of general log in Aurora MySQL?FollowComment"

Answer: "0Accepted AnswerIt is. General log can be very heavy, and will profile everything, which is pretty hard on write performance. Audit log is specifically designed for auditing purpose, and is lightweight.More info on audit log can be found here.CommentShareAWS-User-8357187answered 3 years ago"


Question: How to resolve the issue with the disk space of EC2 Linux machine?

"Hello,I probably face a basic issue.I created a Linux EC2 machine with only 8 Gb of disk space.Output of 'df' command:Filesystem     1K-blocks    Used Available Use% Mounted on/dev/xvda1       8065444 5786008   2263052  72% /Tried to go to AWS console to increase the attached volume.In the Description tab, I see:Root device type: ebsRoot device: /dev/sda1Block device: /dev/sda1Clicking on '/dev/sda1' volume ID, I was able to increase its size to 24 GiB.However, when logging in via SSH, I still see only 8 Gb via 'df'.Tried to reboot the machine, no change.I noticed /dev/sda1 and /dev/xvda1 are not the same. But I have no idea how to apply the change to my disk.Thanks for any hint.MaxFollowComment"

Answer: "0Hello Max,The changes that you have made on the console will reflect only at the block level. You can run the command "lsblk" to see the new size on the instance that you have configured on the console.lsblk -> This command lists the information about all available or the specified block devicesAfter you increase the size of an EBS volume, you must use file system–specific commands to extend the file system to the larger size.The steps to extend your filesystem is well documented on this link below:[1] Extending a Linux file system after resizing a volume  - Identifying the file system for a volume - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/recognize-expanded-volume-linux.html#identify-linux-file-systemMoving on, there is no difference between sda1 and xvda1, they are exactly same. When you run your EC2 instance on a Xen hypervisor the xvd* are simply the Xen disk storage devices, therefore they appear instead of sda, etc.CommentShareManikandananswered 3 years ago0Thanks a lot.I was able to solve the problem, using your advice and the article you mentioned.Max.CommentSharemaxkholmyanskyanswered 3 years ago"


Question: AWS Glue Git integration miss tags

"Hello.An AWS Glue job JSON code on an AWS CodeCommit repository created by AWS Glue Git integration feature doesn't contain tags property. So, a new Glue job created with the JSON code needs to be set tags later.Why is the feature exclude tags property?How can I set tags on many Glue jobs efficiently?Thank you.FollowComment"

Answer: "0Hello,I understand that when you tried to pull the job details from the CodeCommit, it was not fetching the associated tags with it.I have replicated this behaviour on my end and I could see that when the job details were pushed, the corresponding json file of the glue job did not have the tags in it. Therefore, as per the current mechanism of version control in Git seems like the data pertaining to tags is not pushed to the git repository.You will have to manually add the tags to your Glue jobs. If the number of jobs is more then you can write a simple script involving boto3 where in you can recursively get the name of each job from batch_get_jobs and then make a tag_resource api call.The batch_get_jobs api call will fetch all the jobs of your account and you can store this in a variable. Once that is done, you can run a loop to recursively tag the jobs using tag_resource api call. You can get the usage using the documentation I have provided earlier.CommentShareSUPPORT ENGINEERChaituanswered 4 days agorePost-User-2300507 4 days agoThank you for your answer and instruction.I will use tag_resource API call for now.Is there any chance that tags will also be pushed to a repository in the near future?ShareChaitu SUPPORT ENGINEER11 hours agoI currently do not see any feature request to implement this but please keep an eye on the https://aws.amazon.com/new/ to know more about the latest updates on the AWS services.Share"


Question: DVTErrorPresenter: Unable to load simulator devices in Device Farm

"Hello everyone, I'm getting this error to try run ios tests on device farm:DVTErrorPresenter: Unable to load simulator devices in Device FarmUsed tools:Appium version: 1.22.0ios version: 15.xLink of run: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-2#/mobile/projects/a5325ea5-f090-4ac3-aabb-dcec96a31cd4/runs/7aba46fc-69db-4e2f-8170-93cb0115639d/jobs/00000Logs:[DeviceFarm] export TEST_PLATFORM=ios[DeviceFarm] export EXECUTION_ENVIRONMENT=AWS[DeviceFarm] export SLACK_SOS_FILE=/tmp/slackMSG.tmp[DeviceFarm] rm -f $SLACK_SOS_FILE[DeviceFarm] echo "Start Appium TestNG test"Start Appium TestNG test[DeviceFarm] java -Djdk.tls.client.protocols=TLSv1.2 -javaagent:dependency-jars/aspectjweaver-1.9.5.jar -Dallure.results.directory=/tmp/allure-results org.testng.TestNG -d $DEVICEFARM_LOG_DIR/test-output testng.xmlWARNING: An illegal reflective access operation has occurredWARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/private/tmp/scratchXJtgqt.scratch/test-packagebB21R6/dependency-jars/groovy-2.5.8.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operationsWARNING: All illegal access operations will be denied in a future release[INFO ] ***oo try to write fileName: /tmp/slackMSG.tmp text: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/a5325ea5-f090-4ac3-aabb-dcec96a31cd4/runs/7aba46fc-69db-4e2f-8170-93cb0115639d [com.mach.core.util.FileUtil.writeFile(FileUtil.java:44)] [13:26:27,817] [0] [INFO ] ***oo capabilities from java: Capabilities {appPushTimeout: 1000, automationName: XCUITest, deviceName: 00008110-001A75102604801E, newCommandTimeout: 420, platformName: iOS, platformVersion: 15.1, updatedWDABundleId: com.facebook.WebDriverAgent..., useNewWDA: true} [com.mach.core.config.driver.AppiumDriverFactory.getIOSCapabilities(AppiumDriverFactory.java:165)] [13:26:27,963] [146] [ERROR] ***** check the capabilities, e: [com.mach.core.config.driver.AppiumDriverFactory.getIOSDriver(AppiumDriverFactory.java:122)] [13:27:20,945] [53128] org.openqa.selenium.SessionNotCreatedException: Unable to create a new remote session. Please check the server log for more details. Original error: An unknown server-side error occurred while processing the command. Original error: Unable to launch WebDriverAgent because of xcodebuild failure: xcodebuild failed with code 65xcodebuild error message:2022-02-02 13:27:10.270 xcodebuild[1191:8716]  DVTErrorPresenter: Unable to load simulator devices.Domain: DVTCoreSimulatorAdditionsErrorDomainCode: 3Failure Reason: The version of the CoreSimulator framework installed on this Mac is out-of-date and not supported by this version of Xcode.Recovery Suggestion: Please ensure that you have installed all available updates to your Mac's software, and that you are running the most recent version of Xcode supported by macOS.--CoreSimulator is out of date. Current version (757.5.0) is older than build version (776.3.0).Domain: DVTCoreSimulatorAdditionsErrorDomainCode: 3--2022-02-02 13:27:10.270 xcodebuild[1191:8716]  iOSSimulator: [SimServiceContext sharedServiceContextForDeveloperDir:error:] returned nil (Error Domain=DVTCoreSimulatorAdditionsErrorDomain Code=3 "CoreSimulator is out of date. Current version (757.5.0) is older than build version (776.3.0)." UserInfo={NSLocalizedDescription=CoreSimulator is out of date. Current version (757.5.0) is older than build version (776.3.0).}). Simulator device support disabled.Command line invocation:    /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild test-without-building -project /usr/local/avm/versions/1.22.0/node_modules/appium/node_modules/appium-webdriveragent/WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -derivedDataPath /tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL -destination id=00008110001A75102604801E IPHONEOS_DEPLOYMENT_TARGET=15.1 GCC_TREAT_WARNINGS_AS_ERRORS=0 COMPILER_INDEX_STORE_ENABLE=NOUser defaults from command line:    IDEDerivedDataPathOverride = /tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL    IDEPackageSupportUseBuiltinSCM = YESBuild settings from command line:    COMPILER_INDEX_STORE_ENABLE = NO    GCC_TREAT_WARNINGS_AS_ERRORS = 0    IPHONEOS_DEPLOYMENT_TARGET = 15.12022-02-02 13:27:12.088 xcodebuild[1191:8717]  DVTAssertions: Warning in /Library/Caches/com.apple.xbs/Sources/DVTiOSFrameworks/DVTiOSFrameworks-19114/DTDeviceKitBase/DTDKRemoteDeviceData.m:373Details:  (null) deviceType from 00008110001A75102604801E was NULL when -platform called.Object:   <DTDKMobileDeviceToken: 0x7fbc90f8c120>Method:   -platformThread:   <NSThread: 0x7fbc8ec124e0>{number = 3, name = (null)}Please file a bug at https://feedbackassistant.apple.com with this warning message and any useful information you can provide.2022-02-02 16:27:19.723982-0500 WebDriverAgentRunner-Runner[417:15887] Running tests...2022-02-02 16:27:19.726461-0500 WebDriverAgentRunner-Runner[417:15887] Unable to load configuration data from specified path /; error: The file “System” couldn’t be opened.2022-02-02 16:27:19.726518-0500 WebDriverAgentRunner-Runner[417:15887] XCTRunner Arguments: (    "/var/containers/Bundle/Application/3B1C00BC-8B46-437E-923A-C3A20F6DA262/WebDriverAgentRunner-Runner.app/WebDriverAgentRunner-Runner")2022-02-02 16:27:19.726621-0500 WebDriverAgentRunner-Runner[417:15887] XCTRunner Environment: {    "CA_ASSERT_MAIN_THREAD_TRANSACTIONS" = 0;    "CA_DEBUG_TRANSACTIONS" = 0;    "CFFIXED_USER_HOME" = "/private/var/mobile/Containers/Data/Application/3CE450E6-4DEF-46F3-BB9C-ACB104BA7E97";    HOME = "/private/var/mobile/Containers/Data/Application/3CE450E6-4DEF-46F3-BB9C-ACB104BA7E97";    LOGNAME = mobile;    "MJPEG_SERVER_PORT" = "";    "MTC_CRASH_ON_REPORT" = 1;    NSUnbufferedIO = YES;    "OS_ACTIVITY_DT_MODE" = YES;    PATH = "/usr/bin:/bin:/usr/sbin:/sbin";    "RUN_DESTINATION_DEVICE_ECID" = 7447061612232734;    "RUN_DESTINATION_DEVICE_NAME" = PDX640550096;    "RUN_DESTINATION_DEVICE_PLATFORM_IDENTIFIER" = "com.apple.platform.iphoneos";    "RUN_DESTINATION_DEVICE_UDID" = 00008110001A75102604801E;    SHELL = "/bin/sh";    "SQLITE_ENABLE_THREAD_ASSERTIONS" = 1;    TMPDIR = "/private/var/mobile/Containers/Data/Application/3CE450E6-4DEF-46F3-BB9C-ACB104BA7E97/tmp/";    "UPGRADE_TIMESTAMP" = 1643837138604;    USER = mobile;    "USE_PORT" = 8100;    "WDA_PRODUCT_BUNDLE_IDENTIFIER" = "com.facebook.WebDriverAgentRunner";    XCTestBundlePath = "PlugIns/WebDriverAgentRunner.xctest";    XCTestConfigurationFilePath = "";    XCTestSessionIdentifier = "88DCC108-B122-4D1C-84E4-DA839C9021CD";    "XPC_FLAGS" = 0x0;    "XPC_SERVICE_NAME" = "UIKitApplication:com.apple.test.WebDriverAgentRunner-Runner[9ca9][rb-legacy]";    "__CF_USER_TEXT_ENCODING" = "0x1F5:0:0";}2022-02-02 13:27:19.643 xcodebuild[1191:8639] [MT] IDETestOperationsObserverDebug: 6.524 elapsed -- Testing started completed.2022-02-02 13:27:19.643 xcodebuild[1191:8639] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start2022-02-02 13:27:19.643 xcodebuild[1191:8639] [MT] IDETestOperationsObserverDebug: 6.524 sec, +6.524 sec -- endTest session results, code coverage, and logs:/tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL/Logs/Test/Test-WebDriverAgentRunner-2022.02.02_13-27-13--0800.xcresultTesting failed:WebDriverAgentRunner:WebDriverAgentRunner-Runner (417) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: The test runner exited with code 1 before checking in. If you believe this error represents a bug, please attach the result bundle at /tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL/Logs/Test/Test-WebDriverAgentRunner-2022.02.02_13-27-13--0800.xcresult))** TEST EXECUTE FAILED **Testing started. Make sure you follow the tutorial at https://github.com/appium/appium-xcuitest-driver/blob/master/docs/real-device-config.md. Try to remove the WebDriverAgentRunner application from the device if it is installed and reboot the device.Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'System info: host: 'ip-192-168-10-5.us-west-2.compute.internal', ip: 'fe80:0:0:0:14c0:e6ff:c23e:5cb3%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.16', java.version: '12.0.2'Driver info: driver.version: IOSDriver...FollowComment"

Answer: "1Accepted AnswerI saw the same issue when I was using appium 1.19.0. The root cause is DEVICEFARM_WDA_DERIVED_DATA_PATH in the test spec file. 1.19 and up should use DEVICEFARM_WDA_DERIVED_DATA_PATH_V6, take a look at your test spec and cross reference the default on on device farmCommentShareAWS-User-2317604answered a year agoAWS-User-1239661 a year agoThank You so much @AWS-User-2317604 , we were working 1 week in this issue and we didn't find solutions. Thank you really for your valuable answer!Share"


Question: dysfunction of my wordpress site

"my wordpress site is hosted on route 53 but for some time my site does not work anymore. i tried to find solutions but nothing works. could someone explain and help me to find a solutionFollowCommentChris_G EXPERT10 months agoHello, are you aware of how your site is set up?  Route 53 is just the DNS.  Where is the wordpress code hosted?ShareChris_G EXPERT10 months agoHello, can you provide your updates in comments and not add additional answers?  This is a Q&A site, not a forum.Share"

Answer: "0Hello,It looks like your DNS is working fine on Route 53.  It appears the issue is where your site is hosted (EC2, Lightsail, etc.).  You need to verify that your instance is online and is using the same IP that you have set in your Route 53 DNS.  You should then validate that the web server (httpd/nginx) is online as well.CommentShareEXPERTChris_Ganswered 10 months ago"


Question: Can I restore specific data from S3 if it was deleted after non-authorized usage (versioning not active)?

"Hello. First of all, this is my first time in this forum and I'm not an expert user. During november 2021 someone hacked my account and the billings rised up from USD$1,5 (I usually payed that) to USD$32.000 in just 1 month. I was forced to block my payment methods. The thing is I talked to a lot of people asking for support and finally (because there was no valid payment method) Amazon had to close my account and deleted almost everything I had in there.I need to restore some specific data, but I don't know if Amazon has something like a "backup of the backup". Is there a last chance to recover what I've lost??Thanks in advance.Regards!FollowComment"

Answer: "0It depends on when exactly your account is being closed.Within 90-day Post-Closure Period, you may restore the data from your closed account.I recommend you to raise a support ticket to reopen your account https://aws.amazon.com/premiumsupport/knowledge-center/reopen-aws-account/CommentShareRichardFananswered a year ago0Hello Richard! Thanks for your answer. I've made what the link you sent me describe. But I recover just part of the information. I don't know what happened with the rest of the data. Does Amazon can selectively erase data??CommentShareRodrigo Mendezanswered a year ago"


Question: Does Global Accelerator share points of presence with CloudFront?

"It is my understanding that CloudFront can also be used (with caching fully disabled) to improve TCP connection "strength" between EC2 instances and far-away users on weak internet connections by letting users connect to one of many closer points of presence with lower latency.Does Global Accelerator share points of presence with CloudFront? In other words, does Global Accelerator have edge servers in all of the AWS Edge Locations listed in CloudFront's marketing materials, or only in AWS Regions?FollowComment"

Answer: "0Accepted AnswerAWS Global Accelerator operates at AWS edge locations and supports application endpoints in multiple AWS Regions. Look at the Region Table [1] for the full list of supported AWS Regions. Global Accelerator shares points of presence with CloudFront, but is present only in some of the AWS edge locations listed in CloudFront’s product details page.[1] https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/CommentShareAWS-User-5708333answered 4 years ago"


Question: Aurora Serverless MySQL - Downtime during scaling event

"We are currently testing an Aurora Serverless v1 database with a Tomcat application server. The JDBC connection pool is managed by Tomcat. We have setup the database to use forced scaling (using the default timeout). As expected, we have noticed that connections with running queries (that don't complete within timeout period) are killed off during the scaling event with remaining connections being maintained. However, occassionaly, we notice that all connections to the database are dropped. The database cannot be accessed from any method (application, MySQL workbench or MySQL command line). The database does recover the connectivity after a few minutes. In this instance, the logs do not show that the database has restarted. It appears to be network connectivity issue.Has anyone else seen or experienced this issue?FollowComment"

Answer: "0Hi there,I understand that you are using forced scaling for the database and occasionally you notice that all connections to the database are dropped. There are basically 2 methods of scaling used by Aurora Serverless v1, mainly Autoscaling and Manual scaling.During autoscaling, AuroraServerless v1 resets the EngineUptime metric. The value of the reset metric value doesn't mean that seamless scaling had problems or that Aurora Serverless v1 dropped connections. It is simply the starting point for uptime at the new capacity. For more information regarding metrics, see reference [1].When it does need to perform a scaling operation, Aurora Serverless v1 first tries to identify a scaling point, a moment when no queries are being processed. Aurora Serverless v1 might not be able to find a scaling point for the following reasons:Long-running queriesIn-progress transactionsTemporary tables or table locksTo increase your Aurora Serverless v1 DB cluster's success rate when finding a scaling point, it is recommended that you avoid long-running queries and long-running transactions. See reference [2] for more information on working with Aurora Serverless v1.In order to scale Aurora Serverless v1 DB cluster capacity manually, you can follow the steps provided in the documentation provided in reference [3].For more detailed information when using Aurora Serverless v1, see reference [4].Please also take note that Aurora Serverless cannot be accessed through MySQL Workbench because an Aurora Serverless DB cluster cannot be assigned a public IP address, you can only access an Aurora Serverless DB cluster only from withing a Virtual Private Cloud (VPC) based on the AWS VPC service. This is explained thoroughly on the documentation provided in reference [5].References:[1] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/MonitoringAurora.html[2] https://aws.amazon.com/blogs/database/best-practices-for-working-with-amazon-aurora-serverless/[3] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.setting-capacity.html[4] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v1.how-it-works.html[5] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html#aurora-serverless.limitationsCommentShareAsenathianswered a year ago"


Question: Congito - Disabling password reuse (last x passwords)

"We have many downstream client requests to prevent the reuse of passwords when a user changes their passwords in our applications. We are using Cognito User Pools to manage all users and store no credentials at all in our system databases.Is there any way to configure parameters on a user account to disallow them from using the same password that has previously been used?We don't want to make the push to Federated auth (to allow the clients to manage this on their side) because a lot of smaller customers don't necessarily have the means to set up an identity provider or an IT team to manage it or push to MFA for the same reason.Any help with this functionality or if there is a known open issue that we can track, that would be appreciated!FollowComment"

Answer: "1Accepted AnswerHi there,I understand that you would like to set password reuse policies and would like to know if there is a way to configure parameters on a user account to disallow them from using the same password that has previously been used.Regretfully, at the moment, Cognito does not support preventing users from re-using the same password.By design, Cognito does not transfer and store user's password to Cognito actually, instead it uses Secure Remote Password (SRP) protocol for password verification for authentication. You can also find more SRP information here[1]. For security reasons, Cognito only transfers the SRP password verifier, instead of actual password of a user under the hood. Hence, Which is the reason it does not store the history of previous password of a user and compare them against the new one that the user is trying to change.A workaround to apply this restriction, would be implementing your own policy verification when calling Change Password API [2] in your code, or saving the password hash as a custom attribute and then compare it with the new one. Whatever options would involve storing passwords and this is not in line with security best practices, so I would recommend you to check your company’s security policies before doing anything.please keep an eye on the What's New Page[3] and AWS Blogs[4] to know more about future releases. Additionally, you can also monitor Cognito Document history page[5], where new Cognito feature release are announced.I hope you find this information helpful.Have a great day ahead.=========Resources[1] SRP: https://en.wikipedia.org/wiki/Secure_Remote_Password_protocol[2] Change Password API: https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ChangePassword.html[3] What's New: https://aws.amazon.com/new/[4] AWS Blogs: https://aws.amazon.com/blogs/aws/[5] Document History for Amazon Cognito - https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-document-history.htmlCommentShareMfaneloanswered a year ago"


Question: Disable CORs on EC2 instance

"Hi I have been going crazy the last 2 days wondering how CORs headers got into my EC2 dockerized API response, and I broke everything down and came to a conclusion it must be coming from amazon.After banging my head wondering how amazon could modify my response, even though I would assign the cors headers myself to try and override theirs, they would take precedence.Then I stumbled into this article that explains everything:https://docs.aws.amazon.com/AWSEC2/latest/APIReference/cors-support.htmlFor simple GET and POST requests amazon EC2 ALWAYS returns Access-Control-Allow-Origin:* which makes it possible to do CORs requests.I specifically do not want any CORs on my site and do not want to have to remove headers set by amazon without my permission... - is there anyway to turn the automatically added CORs headers off??if not maybe a way that I could at least change the access control origin to only my domain, because the current structure means I have to remove amazons response header and add my own, if they are going to add it for me, they may aswell take my input....FollowComment"

Answer: "1Hi! The documentation you linked explains that EC2 API has the CORS header you mentioned. The EC2 API means the AWS API to (in example)  start/stop instances or create/terminate them. So if you build a application that interacts directly with AWS resources you won't be blocked by CORS. This doesn't include any webserver you may have hosted inside an EC2 instance.If you have a Website hosted in EC2 with a direct public IP, AWS does not modify the responses that your server answers. If this is your case you need to look deeper into your application/services to see what service is adding the headers (Check the apache/nginx configuration for instance if you use those services).If you have other services in front of your EC2 instances, like an API Gateway, you need to check the documentation on those services to see how CORS may be implemented in it.I hope this answer helps you! Feel free to expand on your scenario to provide more context to help you or create a support ticket to AWS Support to enroll further help!CommentSharePablo Guzmananswered 8 months agosecurity 8 months agoHi thanks for your help, its much appreciated!I think something just made more sense, what you just said about having an EC2 with a direct public IP!So I initially picked up on these CORs headers hosting my webserver DNS via cloudflare, so the public IP address is not public!Today I was testing with my original EC2 domain (ec2alaska.amazon.eu.com) which is not associated with cloudflare, to try and see if cutting them out would resolve anything! But the CORs headers persisted which is why I thought it must be coming from the EC2 instance!Sharesecurity 8 months ago\continued But like you said, the public IP of that EC2 domain is still being proxied behind cloudflare so amazon must be modifying the headers because of this?So based on my webserver not having a public IP, through cloudflare, is it safe to say that these Access Control Headers are being added by Amazon? Is there any way I could stop this from happening as I was unable to override these CORs headers being set, not from my express server origin, nor at my NGINX proxy. I didnt really want to modify at a cloudflare worker level either as I want my code to be self-reliant at this stage!SharePablo Guzman 8 months agoNo, AWS does not modify the HTTP headers of a EC2 hosted web application that's not being fronted by CloudFront or API Gateway.I would advice to SSH/RDP into the EC2 instance and use a curl-like command to query your webserver and check the existence of the headers, if they exists that means those headers are part of your application stack.Share0Thanks for your help Pablo!My test involved making a simpler version of my app on HTTP and deploying it alone on both my local PC (dev env) and my EC2 instance. Cloudflare proxy was switched off for my EC2 IP and I tested using the public IP to make sure EC2 was at front.I can confirm the Access-Control-Allow_Origin:* header only appears in my EC2 responses, even with direct IP access!In my local dev environment, no access control headers are received whatsoever. No response modification is happening...On my IP EC2 instance, access control allow origin: * is added to both POST and GET requests.Additionally, if I make a preflight cross origin request - this OPTIONs is returned!!I think this confirms our suspicions that the EC2 "API" document I initially stated, is actually relevant to EC2 instance response modification and these responses confirm this...Please do your own due diligence and let me know if anyone gets different results.CommentSharesecurityanswered 8 months ago"


Question: How can Ec2 Instance in private subnet access internet via vpc endpoint?

"I connected to ec2 intance in private subnet via Bastion Host.Then I created a vpc gateway endpoint to reach s3. I assigned the required role to ec2 and the connection to s3 is done. My question is, how can Ec2 instance in private subnet  dowload something from the internet?FollowComment"

Answer: "0Accepted AnswerHi, regarding your comment "Ec2 instance in private subnet was able to install some programs while I had vpcendpoint, even though it was not nat gateway", was this Amazon Linux, and were the packages you installed part of the Linux distribution or third-party?  Amazon Linux is hosted in S3 so you don't need outbound internet access to update or install stuff from there, just S3 access as you've found.  See https://aws.amazon.com/premiumsupport/knowledge-center/ec2-al1-al2-update-yum-without-internet/.CommentShareEXPERTskinsmananswered a year agorePost-User-3058098 a year agoyum update, yum install git... When I used these commands, it performed the download process, but it cannot ping. I guess, as you said, these are the packages belonging to the Linux distribution.Share0Check out this document about setting up a NAT Gateway.Instances in a private subnet do not have public IP addresses so they need a either a NAT Gateway or NAT instance to securely connect to the internet. VPC Endpoints are a way for you to securely connect to AWS Services, here's a list of currently supported services. Essentially it allows you to connect your EC2 to S3 (among other services) without traversing the public internet. They aren't used for giving EC2 instances public internet access. That's where a NAT Gateway or Instance comes into play.CommentShareAWSJoeanswered a year agorePost-User-3058098 a year agoI know what you say, that's why I asked this question. Today, a Ec2 instance in private subnet was able to install some programs while I had vpcendpoint, even though it was not nat gateway.Share"


Question: Putting ALb-NLB-ALB route for requests is giving 502 for application

"We had a primary ALB listening to all out apps mapped through R53 records. Now we have listener rule crunch as ALB doesn't support more rules above 100. So we had been proposed a solution where we can put a NLB under primary ALB and then secondary ALB under NLB. So flow will be:Requests--->R53--->ALB1--->NLB--->ALB2--->AppsALB1 has a default rule which allows unmatched requests to pass through to NLB and then ultimately to ALB2 where new rules are evaluated.Rule configuration at ALB1 is: Default rule --Forwardto-->Rule at NLB: TCP-443 listener rule --ForwardTo--> ALB2 TG with fargate application ipBut we're seeing intermittent 502 responses on primary ALB while testing. We are not seeing any 502 logging on ALB2. So possibly NLB is ending it as we have seen multiple TArget reset count happening at NLB in metrics. Also, nothing is getting logged in application logs.We did another testing where we directly routed traffic to ALB2 through R53, we didn't see any 502 responses there.Any suggestion, how to go about the debugging it?I can give more details if needed.FollowComment"

Answer: "0Hello,Please refer the link: https://aws.amazon.com/premiumsupport/knowledge-center/elb-alb-troubleshoot-502-errors/ for  the troubleshooting of 502 errors in detail.CommentShareAWS-User-0271691answered 10 months ago"


Question: How to create AWS Cognito userpool/users by script?

How to create Cognito userpool  (if not exists) and add users (create/update) to the pool using script (re-enterable)?I'd like to provide the script to customer to avoid manual creation etcFollowComment

Answer: "1In addition to the above, you can also Importing users into user pools from a CSV file.There is another, more dynamic way to import the users via Lambda, Importing users into user pools with a user migration Lambda trigger.CommentShareEXPERTkentradanswered 2 months ago0HI Oleg,I would use CDK  to leverage infrastructure as code. In particular, you can look at below construct to create a Cognito User pool: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.UserPool.html.To create users in that pool programmatically there are 3 options:Still use CDK: https://github.com/awesome-cdk/cdk-userpool-userUse CLI: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cognito-idp/admin-create-user.htmlUse SDK: https://www.npmjs.com/package/@aws-sdk/client-cognito-identityHope it helps, and if does I'd appreciate if answer can be accepted, so that community can benefit for clarity when searching for similar questions, thanks ;)CommentShareEXPERTalatechanswered 2 months ago"


Question: Use our own CA cert with Greengrass aws.greengrass.FleetProvisioningByClaim.jar

"Hi, I've create and registered a CA certificate in IoT core.  When I use this for greengrass fleet provisioning it fails withjava.lang.RuntimeException: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failedThe documentation is not very clear about how to use it with Greengrass.  But the existence of "Automatic certificate registration" when doing "Register CA certificate" suggests it should work and I'm missing something.  Any advice much appreciated.FollowComment"

Answer: "0UPDATE - Looked into this a bit more, seems like you're trying to use your own CA because you'd register your own client certificates signed by that CA to use for your devices, in which case, the Greengrass aws.greengrass.FleetProvisioningPlugin will not help, you can use this plugin if you're okay using the Amazon Root CA and let IoT Core generate client certificates for you from the claim certificate and provisioning template. If that doesn't work for you, you may be looking for JITP/JITR options IoT Core provides  https://docs.aws.amazon.com/iot/latest/developerguide/jit-provisioning.html  and https://docs.aws.amazon.com/iot/latest/developerguide/auto-register-device-cert.htmlCommentShareshagupta-awsanswered a year agoncarn a year agoThanks @shagupta-aws for the information.  If we bake the claim cert and key into lots of devices and then it is deleted or revoked (accidently) is there a way to restore it if it's an Amazon issued certificate?  I guess that was my only reason for using a custom CA.Share0Hi @ncarn,Thanks for using Greengrass. It's hard to tell just by the error message why it failed, but common reasons for this issue are using the incorrect IoT data endpoint or the root CA certificate, could you tell us what configuration options you provided while installing Greengrass using the aws.greengrass.FleetProvisioningByClaim plugin? https://docs.aws.amazon.com/greengrass/v2/developerguide/fleet-provisioning.html As described here, please ensure that you're using the ATS IoT data endpoint and placing the Amazon Root CA certificate on the device you're trying to provision. Did you see the private key and certificate files downloaded on the device, if not, were there any other failure logs in greengrass.log file? It will be helpful if you can share the logs too.You could also try enabling monitoring for connection failures in CloudWatch to get logs in your AWS account https://docs.aws.amazon.com/iot/latest/developerguide/cloud-watch-logs.html It might not provide information for client side issues but will make troubleshooting easier overall.CommentShareshagupta-awsanswered a year ago"


Question: WAF managed rules blocking SES incoming email notifications

"I use SES incoming email for a discussion groups feature.  When an email arrives, SES is configured to notify an SNS topic, which in turn POSTs to an HTTPS endpoint on my backend via a CloudFront distribution.This stopped working some time between Dec 5 and 24: the POST from SNS was showing as 403 Forbidden / Error in the CloudFront logs.After some trial and error, I discovered that by turning off the AWS WAF WindowsManagedRuleSet on my CloudFront distribution, I could fix the problem.Has anyone else seen this?  It seems like an obvious false positive if AWS WAF rules are blocking AWS SESS notifications.FollowComment"

Answer: "0Hello, Chris,I found this AWS document on mitigation of false positives in WAF (1) and how to override rules in the group to identify which is causing the issue.  I see 6 rules in the Windows Operating System managed rule in my own account.  You can set individual rules to "count" one at a time to identify which may be preventing your SNS topic from posting.  See the documentation link for details.If you can identify the rule causing the issue and don't mind leaving it in "count", you can go that route.  Otherwise, if you have a CloudFront request Id (x-amz-cf-id), this can be investigated further via a support case.** RESOURCES **AWS Managed Rules for AWS WAF - https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups.htmlCommentShareSUPPORT ENGINEERRon_Hanswered a year ago"


Question: Sync to me-south-1 from eu-west-1 error!

"Hi,I would like to move the S3 bucket from eu-west-1 (Paris) to me-south-1 (Bahrain), but the AWS cli gives and error:fatal error: An error occurred (IllegalLocationConstraintException) when calling the ListObjectsV2 operation: The me-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.Is this my bad, or yet not supported?Thanks,FollowComment"

Answer: "0Hi,I was able to successfully run the aws s3 sync command from a source bucket in eu-west-1 to a target bucket in me-south-1. Here are the steps I performed.I first created a bucket in eu-west-1:aws s3api create-bucket --bucket xxxx-paris --region eu-west-1  --create-bucket-configuration LocationConstraint=eu-west-1It was successfully created and returned:{    "Location": "http://xxxx-paris.s3.amazonaws.com/"}I created a file called hello.txt and uploaded it to the newly created bucket in eu-west-1:aws s3 cp hello.txt s3://xxxx-paris/hello.txtI then created a bucket in me-south-1:aws s3api create-bucket --bucket xxxx-paris2 --region me-south-1 --create-bucket-configuration LocationConstraint=me-south-1It was successfully created and the following was returned in the terminal:{    "Location": "http://xxxx-paris2.s3.amazonaws.com/"}I then ran the aws s3 sync command to sync the contents of the bucket in eu-west-1 to the bucket in me-south-1:aws s3 sync s3://xxxx-paris s3://xxxx-paris2 --source-region eu-west-1 --region me-south-1And then it shows that it succesfully copied over the file:copy: s3://xxxx-paris/hello.txt to s3://xxxx-paris2/hello.txtI wasn't able to reproduce, but maybe if you try the above steps in your environment and see if that works, you can narrow down what your original problem might have been.Also, to help debug the issue, add the --debug to the aws s3 sync command.Hope this helps.-randyCommentShareRandyTakeshitaanswered 4 years ago0Thanks, adding source & destinaction region solved the issue.aws s3 sync s3://site.com s3://site-bahrin --source-region eu-west-1 --region me-south-1CommentShareBinMardodanswered 4 years ago"


Question: Boto3  - S3 - Downloads Occasionally failing

"Hi Team,I have written a python program that copies videos from S3 bucket to an EC2 machine, videos that are more than 100 MB in size. While the program runs fine most of the time, but sometimes the download of the file fails with the error, wherein the name of the file is changed to video_name+<hash_characters>.After doing wee bit of research I found out the boto3 library of python, downloads the file temp with a hash name and then replaces it with the name of the file once the download is completed.Below is the log when the download fails :: ERROR/ForkPoolWorker-3] Task api.service.push_to_async_report_generation[25fdce51-a067-4893-bf4c-22013368e59f] raised unexpected: FileNotFoundError(2, 'No such file or directory')Traceback (most recent call last):File "/usr/local/lib/python3.8/dist-packages/celery/app/trace.py", line 451, in trace_taskR = retval = fun(*args, **kwargs)File "/usr/local/lib/python3.8/dist-packages/celery/app/trace.py", line 734, in protected_callreturn self.run(*args, **kwargs)File "/home/ubuntu/src/api/service.py", line 100, in push_to_async_report_generationimage_path = bucket.download_file(Kwargs["file_name"])File "/home/ubuntu/src/utils/s3_services.py", line 81, in download_fileself.bucket.download_file(file_name, os.path.join('./static',file_name))File "/usr/local/lib/python3.8/dist-packages/boto3/s3/inject.py", line 244, in bucket_download_filereturn self.meta.client.download_file(File "/usr/local/lib/python3.8/dist-packages/boto3/s3/inject.py", line 170, in download_filereturn transfer.download_file(File "/usr/local/lib/python3.8/dist-packages/boto3/s3/transfer.py", line 307, in download_filefuture.result()File "/usr/local/lib/python3.8/dist-packages/s3transfer/futures.py", line 106, in resultreturn self._coordinator.result()File "/usr/local/lib/python3.8/dist-packages/s3transfer/futures.py", line 265, in resultraise self._exceptionFile "/usr/local/lib/python3.8/dist-packages/s3transfer/tasks.py", line 126, in callreturn self._execute_main(kwargs)File "/usr/local/lib/python3.8/dist-packages/s3transfer/tasks.py", line 150, in _execute_mainreturn_value = self._main(**kwargs)File "/usr/local/lib/python3.8/dist-packages/s3transfer/download.py", line 571, in _mainfileobj.seek(offset)File "/usr/local/lib/python3.8/dist-packages/s3transfer/utils.py", line 367, in seekself._open_if_needed()File "/usr/local/lib/python3.8/dist-packages/s3transfer/utils.py", line 350, in _open_if_neededself._fileobj = self._open_function(self._filename, self._mode)File "/usr/local/lib/python3.8/dist-packages/s3transfer/utils.py", line 261, in openreturn open(filename, mode)FileNotFoundError: [Errno 2] No such file or directory: './static/example.mp4.F5e1dAFEIt seems like the S3transfer package is trying to open the file before it gets downloaded.Any help related to the topic is appreciated!Thanks!FollowComment"

Answer: 0This issue doesn't appear to be related to Storage Gateway.  You should tag your question with S3 to get the correct eyes on your issue.CommentShareMike-HAWSanswered a year ago0The tag may be "Simple Storage Service" that you needCommentShareMike-HAWSanswered a year ago


Question: Amazon Textract amazon-textract-response-parser library with python throws "ValidationError: {'_schema': ['Invalid input type.']}" on expense analysis textract response

"Hi,I tried following the instructions at https://aws.amazon.com/fr/blogs/machine-learning/announcing-expanded-support-for-extracting-data-from-invoices-and-receipts-using-amazon-textract/ to parse the Textrcat response from a call to get_expense_analysis API (aynchronous API call on a pdf file).I get a response from the API which seems to be valid json and compliant with the Textract Expense API documentation.But, when I execute the following code that I copied from the articlefrom trp.trp2_expense import TAnalyzeExpenseDocument, TAnalyzeExpenseDocumentSchemat_doc = TAnalyzeExpenseDocumentSchema().load(out)# out is the json output from the analyse expense API call (tried both text and json dictionnary form to be sure)I get following error---------------------------------------------------------------------------ValidationError                           Traceback (most recent call last)/tmp/ipykernel_7538/447086103.py in <cell line: 2>()      1 from trp.trp2_expense import TAnalyzeExpenseDocument, TAnalyzeExpenseDocumentSchema----> 2 t_doc = TAnalyzeExpenseDocumentSchema().load(out)~/anaconda3/envs/python3/lib/python3.8/site-packages/marshmallow/schema.py in load(self, data, many, partial, unknown)    717             if invalid data are passed.    718         """--> 719         return self._do_load(    720             data, many=many, partial=partial, unknown=unknown, postprocess=True    721         )~/anaconda3/envs/python3/lib/python3.8/site-packages/marshmallow/schema.py in _do_load(self, data, many, partial, unknown, postprocess)    902             exc = ValidationError(errors, data=data, valid_data=result)    903             self.handle_error(exc, data, many=many, partial=partial)--> 904             raise exc    905     906         return resultValidationError: {'_schema': ['Invalid input type.']}It is supposed to work out of the box as documented in the article.I am using Python 3.8.12 and successfully installed amazon-textract-response-parser-0.1.30 botocore-1.24.46 marshmallow-3.14.1What am I missing (I can share the pdf or the output of the API call if needed)  ?Any help welcome.Christian.FollowComment"

Answer: "1Can you file a ticket against the repository? https://github.com/aws-samples/amazon-textract-response-parser/issuesIdeally add the JSON causing the issue, that will help to debug.Thank you.CommentShareMartin Schade _AWS_answered a year agoc_ds a year agoHi Martin. Done. Thanks !Share"


Question: Ec2 Linux instance stuck on boot fetching keys from metadata server

"Hi,Hopefully someone can help. Just rebooted our of our Ec2 instances and it is now stuck on boot.The server log shows where it is stuckWelcome to Oracle Linux Serverip_local_port_range: prefer different parity for start/end values.Starting udev: %G  [ OK  ]Setting hostname <hostname>:  [  OK  ]Checking filesystemsChecking all file systems.[/sbin/fsck.ext4 (1) -- /] fsck.ext4 -a /dev/xvde1 /dev/xvde1: clean, 82719/655360 files, 1144008/2621184 blocks[  OK  ]Remounting root filesystem in read-write mode:  [  OK  ]Mounting local filesystems:  [  OK  ]Enabling /etc/fstab swaps:  [  OK  ]Entering non-interactive startupCalling the system activity data collector (sadc)... Bringing up loopback interface:  [  OK  ]Bringing up interface eth0:  Determining IP information for eth0... done.[  OK  ]Starting auditd: [  OK  ]Starting system logger: [ OK ]Fetching list of trusted keys from metadata server...Any pointers would be much appreciated as it won't continue through the rest of the boot sequence until it fetches the public key from the metadata server.CheersPJFollowComment"

Answer: "0Found that the issue was down to the network configuration.I eventually found the issue by stopping the instance and detaching the boot volume, I then mounted it onto another EC2 instance and drilled down into the logs.CommentSharePeejay996answered 3 years ago"


Question: How do I reduce "CW:GMD-Metrics" costs?

"I've noticed a cost labeled as "CW:GMD-Metrics" in my AWS Cost Explorer. I believe this is related to Amazon CloudWatch custom metrics, specifically the ingestion and storage of granular metric data.I'm looking for advice on how to drill down and find optimization opportunities to reduce these costs.Here's what I've considered so far:Evaluating the necessity of each custom metric.Adjusting the metric resolution.Optimizing metric storage and retention periods.I'd appreciate any further suggestions, especially on how to identify the major cost contributors and best practices for optimizing CloudWatch custom metrics usage. If anyone has had experience reducing "CW:GMD-Metrics" costs or has any insights, please share!Thank you in advance for your help!FollowComment"

Answer: 0You are on the right track . first step to understand what you were charged for and why -Review your AWS Cost and Usage reports to understand your CloudWatch charges. Look for charges for the following services.Custom metrics:MetricStorage <region>-CW:MetricMonitorUsageCloudWatch metric API calls:<API Name> <region>-CW:RequestsGetMetricData <region>-CW:GMD-Requests/MetricsCloudWatch alarms: Unknown <region>-CW:AlarmMonitorUsageUnknown <region>-CW:HighResAlarmMonitorUsageCloudWatch dashboards:DashboardHour DashboardsUsageHour(-Basic)CloudWatch Logs:PutLogEvents <region>-DataProcessing-BytesPutLogEvents <region>-VendedLog-BytesHourlyStorageMetering <region>-TimedStorage-ByteHrsCloudWatch Events:AWSEvents PutEvents <region>-Event(-64K-Chunks)|RequestOnce you understand what you were chargedPlease refer to the Resolution section in the Link : https://aws.amazon.com/premiumsupport/knowledge-center/cloudwatch-understand-and-reduce-chargesCommentShareAneesh_Vanswered a month ago


Question: Include request_datetime from S3 file in Athena table - table made by Glue

"In my early steps on the S3, Glue and Athena services, I managed to create a table with Glue, with respect some parquet files in an S3 bucket. Every 30 minutes a new parquet file, with same  columns, is added to a folder in S3.The first time that Glue created the table there was a request_datetime column (additional to the data of the parquet files) with the datetime of the push to the S3 folder and in this way I could filter in Athena for rows with respect to the request datetime, so it was helpful.Some hours later, I cleaned my S3 bucket and wanted to start clean, but this time the table created from Glue didn't include this column.So the question is:How can I include this column with this information about the request_datetime?The request_datetime is part of the filename that is pushed to the s3 every 30 minutes but I think it is also available from the request data.Thanks in advance for any assistance :)FollowComment"


Question: Lightdail instance retirement scheduled - instance has become unreachable.

"Hi! I have been running a Lightsail instance for a couple of years without problem. Two days ago the instance became unreachable. I've had a note in 'Scheduled Changes' about a lightsail instance retirement scheduled for 10th March. It does say in this description that 'Your instance may already be unreachable'.I can't get in to the server via Plesk or SSH, rebooting does nothing. I detatched and reattached the static IP which did nothing.I tried taking a snapshot - but that has been 'Snapshotting...' for a while with no result.I have just tried to stop the instance in order to start it again, but it seems to be frozen in Status: Stopping.I have tried all I can to get back in to this instance, is there anyone out there who can help at this point?Many thanks.FollowComment"

Answer: "0The stop eventually worked, and on restarting the instance now works.CommentShareStudioAutomataanswered 2 years ago"


Question: Unable to delete AWSServiceRoleForApplicationAutoScaling_DynamoDBTable role

"Hi,I can't tell if it is a IAM or a DynamoDB related issue, but I once have a DynamoDB table called "Person" which doesn't exist any more and when I try to delete the AWSServiceRoleForApplicationAutoScaling_DynamoDBTable service role it displays "deletion failed - Service linked role is still in use - Resource ARNs - eu-west-3 - table/Person/index/id-index".I suppose there is a dangling reference somewhere...Any advice ?Thanks!FollowComment"

Answer: "0Accepted AnswerHiI see that you are facing errors while deleting service linked role AWSServiceRoleForApplicationAutoScaling_DynamoDBTable.As we can see in the error "table/Person/index/id-index” is still using the role. You can delete a service-linked role only after first deleting the related scalable resources.You should deregister the scalable target “table/Person/index/id-index” [1]  and also delete the scaling polices[2]After clearing all the dependencies you can the delete the service-linked role in concern.Please refer https://docs.aws.amazon.com/autoscaling/plans/userguide/aws-auto-scaling-service-linked-roles.html#delete-service-linked-role for more information.Update the thread if you continue to face any issues.[1] https://docs.aws.amazon.com/cli/latest/reference/application-autoscaling/deregister-scalable-target.html[2] https://docs.aws.amazon.com/cli/latest/reference/application-autoscaling/delete-scaling-policy.htmlCommentShareAWS-User-4025440answered 4 years ago0Yes, that was exactly the point! The autoscaling policy for the index in question was still here despite the table was deleted.According to the links you provided and using the CLI to delete the corresponding autoscaling resources, I can now get rid of the role.ThanksCommentSharebl75answered 4 years ago0Hello,We are facing a similar issue, but IAM won't give away the name of the resources still in use. Cloudtrail also does not seem to be helpful in identifying those resources.This is on the back of tearing down a CDK stack that involved a global DynamoDB table.Thank you very much in advance!CommentSharerePost-User-5910300answered 6 months ago"


Question: lambda connection timed out while accessing SQS

"We are using Lamda and passing SQS que as environment variable and The lambda is in a VPC in Public subnet with Internetgateway attached, the role attached to the lambda has SQS Full access, but we are getting the error as below ""errorMessage": "Unable to execute HTTP request: Connect to sqs.ap-south-1.amazonaws.com:443 [sqs.ap-south-1.amazonaws.com/10.0.0.69, sqs.ap-south-1.amazonaws.com/10.0.0.39, sqs.ap-south-1.amazonaws.com/10.0.0.53] failed: connect timed out","errorType": "com.amazonaws.SdkClientException",FollowComment"

Answer: "0To access SQS from a Lambda in a VPC, you must either create a VPC endpoint, create a NAT gateway, or manually set a public IP address in the Lambda's ENI.If you are on a public subnet, it would be easy to set up a VPC endpoint.https://docs.aws.amazon.com/ja_jp/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-sending-messages-from-vpc.html#create-vpc-endpoint-for-sqsCommentShareEXPERTRiku_Kobayashianswered 2 months agorePost-User-6676408 2 months agoBut even in Public subnet with IGW and Private With NAT still getting the errorShareRiku_Kobayashi EXPERT2 months agoLambda does not have a public IP address when placed on a public subnet.So please set up a VPC endpoint for SQSShare0The answer from Riku_Kobayashi is correct but I'd like to comment on something else in the question:The endpoint name is sqs.ap-south-1.amazonaws.com but the error you're getting shows that it is resolving to private IP addresses (10.0.0.69, 10.0.0.39, 10.0.0.53) and that's curious. Do you have anything in your VPC which is acting as a DNS server that is trying to send your traffic to some other endpoint?CommentShareEXPERTBrettski-AWSanswered 2 months ago"


Question: Glue ETL Job with external connection to Redshift - filter then extract?

"I have been attempting to ETL a data set from redshift with glue for data lake consumption.  The redshift data set is very large and I only want to extract the last x days data each job run.  When I setup the job the filter comes after the applymapping and before the resolve choice dataframes.  When doing so the redshift query appears on the cluster as essentially a "Select *" .  It seems the dataframe wants to load the entire redshift table into Glue then filter it, which is expensive and eventually fails.  Is there a way to filter the datasource before the dataframe?FollowComment"

Answer: 0Accepted AnswerI have a workaround for the pushdown predicate using databricks Redshift driver. This  requires some custom coding in Glue but have worked flawlessly in the past. My sample code is https://github.com/saunakc/etl-microservice-datalake/blob/master/src/glue/unload-table-part.py for reference.CommentShareAWS-User-9244242answered 5 years ago


Question: White Box Tool Feedback

"Hey everyone,The 1.25 release of Lumberyard introduced the White Box Tool (an Editor feature for creating simple level geometry while prototyping).There’s a lot left to do but we wanted to get what we had in front of people as early as possible and would love to hear your feedback. Please tell us what you like/don’t like, what you feel is missing and would like to see added or any other issues/bugs/ideas.We’re trying to gather as much feedback as possible and will be using it to help drive future development.Thank you very much for your time! We look forward to hearing from you!Cheers,TomFollow"

Answer: "0Hi @REDACTEDUSER💡 My Idea is For the future, Please Lumberyard team develop this tool a lot, I think include the possibility of making 3D Characters and Animation Rigs  and Unwrap Map 3D Character Texture as well with White Box Tool :star_struck: 😍I saw white box tool in this video 👇 It is Good for now 😎https://www.youtube.com/watch?v=nV2tm19FmYISharerePost-User-8224505answered 3 years ago0Everything is fine, but the boolean of the operation are just necessary...Such as addition, subtraction of geometry, etcThank you, you're cool!SharerePost-User-0396919answered 3 years ago0Thanks very much for your feedback @REDACTEDUSERYes boolean operations are awesome and we would love to add support for them in future. We have a few other bits and pieces we'd like to polish in Sketch Mode but as soon as that's done adding support for intersection/union/substraction is very much on the list!Thanks for the confirmation that this is an important feature!Thanks again for checking out the tool.Cheers!TomSharerePost-User-0765265answered 3 years ago0If it continues to improve. It can compete with Blender. Since it is modeling in realtime like eevee does in Blender.SharerePost-User-2061097answered 3 years ago0Hi @REDACTEDUSERThank you very much for the feedback. The plan is definitely for the tool to continue improving! :slight_smile:Whether we reach the sophistication of a tool like Blender is another story. Our focus is on the tool being used for simple level design primarily. We do know however there's still a ton of useful functionality we're missing. The plan was (and still is!) to get this out early and then build it in the open with the engagement and participation of our customers and community.Any thoughts or ideas you have or pain points you've experienced with the current tool please let us know.Thanks again!TomSharerePost-User-0765265answered 3 years ago0I finally got home and I worked very hard with this tool. I would suggest working out the accuracy. That is, I want to say that when we do some kind of action with the geometry, we must see by how many centimeters or meters or degrees we shift, unfold or scale the polygon or edge or vertex. This is necessary to achieve high accuracy rates. Then light geometry will be very easy to do. At the moment, you can focus on the eye, that is, approximate indicators are obtained. And this is not very convenientSharerePost-User-0396919answered 2 years ago0Hey @REDACTEDUSERThank you very much for the feedback, we really appreciate it!Improved indicators around distance/measurement is a brilliant suggestion and something we're actively thinking about. We're trying to figure out the best way to build the tech to support such a system (there's some interestings ideas being kicked around :slight_smile:).One option to use short-term is to enable grid snapping - all the editing that happens with the White Box Tool actually uses the same Manipulators under the hood as the new Viewport Interaction Model. Grid snapping should help, the only pain is you have to enter/leave Component Mode to enable/disable it (we're looking at addressing that soon too) but if you're going to stay in snapping mode for a while it's quite useful.Thanks again for the feedback, it really helps! Any other suggestion/problems/ideas please let us know! :slight_smile:Cheers,TomSharerePost-User-0765265answered 2 years ago0Hello! I recorded a little gif asking what was really missing in white box. That is the exact coordinates. GIF is just a demonstration of what is needed. I use Modo software. I guess I only have two requests. This add sizes to any boolean of the operation and do bulow operations, but they are not necessary. You can do without themAdd precision first. It is important. And then it may be possible to completely abandon such programs as 3d max, modo, maya, blender, of course, for modeling and prototyping levelsREMOVEDUPLOADSharerePost-User-0396919answered 2 years ago0What would also be cool is some adding and subtracting functionality between different whitebox entities. This is pretty typical of constructive solid geometry. And if you wanted to you could throw in some useful functions like "hollow" function which turns a block into a hollow cube of a certain thickness. I've found these functions very useful for prototyping levels with rooms.SharerePost-User-9359627answered 2 years ago0Thanks very much for the gif @REDACTEDUSERWe have plans to add new submodes that allow more precise control over vertices/faces/edges etc.. using our existing manipulators. In the first preview version we've released we unfortunately didn't get to that as we wanted to focus on ideas around what we call 'Sketch Mode'. We wanted to minimise context switching and clicks to provide an immediate form of editing (just click and drag).It would be great for more precise cases to be able to drop down to a more advanced mode where we show a full manipulator that you can control just as you want like in the gif. It's very much on the roadmap :slight_smile: Other things are winning out on the priority list at the moment but I can assure you we'll get back to it!SharerePost-User-0765265answered 2 years ago0You're totally right - constructive solid geometry (CSG) is super cool and we have plans to integrate a form of it into White Box. Boolean operations are incredibly useful for level prototyping and we would love to add it. Most likely we'll have a sub-mode in White Box to support this with more contrained shapes to ensure the CSG operations perform efficiently. It's likely a little ways off but continues to be a much requested feature so we will be looking at it for sure! :slight_smile: In the meantime thanks for your patience and do keep giving is feedback, it really helps! Thanks!SharerePost-User-0765265answered 2 years ago0I have another suggestion. Of course you don't need whitebox to be like, some feature full modeler or anything, but it would be useful if there was a basic way to assign UV maps. Like a basic tool to assign certain polygons to positions on the texture. Perhaps a separate window pops up and you can have a basic cube unwrap and the ability to move around and cut edges. There's a similar tool in Godot for basically UV mapping polygons.SharerePost-User-9359627answered 2 years ago0Awesome suggestion, that has come up before too. It's something that's super useful and is a must for a tool like this. It's 100% on the list :) Apologies it's not going to be around the next corner but it hopefully won't be too long to wait. Thanks for your patience!SharerePost-User-0765265answered 2 years ago0Boolean operations - especially, or at least  subtraction -should be definitely the next no. 1 thing on that list.I also agree that UV tools would be useful.(OT/BTW: Is UV manipulation in general already exposed to runtime manipulation? I.e. to move textures on the mesh around in RT?)SharerePost-User-5016482answered 2 years ago0For sure! :slight_smile: It's on the list and something we'll be investigating soon. Thanks for letting us know this is important to you.I'm afraid it isn't with White Box but it might be possible to do with some other Lumberyard components (maybe with some tweaks to the Mesh Component?) but I'm not sure off of the top of my head sorry.Thanks again!TomSharerePost-User-0765265answered 2 years ago0[quote="tom-hh, post:16, topic:9153"]I’m afraid it isn’t with White Box but it might be possible to do with some other Lumberyard components (maybe with some tweaks to the Mesh Component?) but I’m not sure off of the top of my head sorry.[/quote]Oh, I didn't mean with White Box anyways. That's why I marked it with OT and with White Box it  wouldn't make too much sense I think.But I think I've read something about this is the manual over the weekend regarding mesh-components.  So I think this is indeed already possible. Not 100% sure, but quite.SharerePost-User-5016482answered 2 years ago0Awesome got you! Best of luck, I hope you can find the answer you're looking for :)SharerePost-User-0765265answered 2 years ago0I think support for adding edge loops would be greatSharerePost-User-7073252answered 2 years ago0Thanks for the feedback @REDACTEDUSERThere's some rather limited support for something approximating edge-loops by using hiding edges in Sketch Mode and then restoring different edges in Restore Mode. You can't yet define new edges on their own though. If you have an example or link to show exactly what you'd like to do that would be super helpful.Thanks!TomSharerePost-User-0765265answered 2 years ago0https://youtu.be/ZZ7_-uTqoYo?t=70I think this video from 1:10 to 1:25 shows it well.SharerePost-User-7073252answered 2 years ago0Hi everyone, This is great feedback. Don't be afraid to keep giving suggestions for the white box tool!Keep them coming! Thanks!SharerePost-User-4369365answered 2 years ago0Another big thing is if scaling works as expected, currently the white box always scales uniformlySharerePost-User-7073252answered 2 years ago0After some time with WBT I really miss multiselection.Extruding multiple faces at once really would save a lot of time.A lot of things are impossible without it.Mirror and merge would also be helpful and should be easy to implement. Bevel would be  great, too.SharerePost-User-5016482answered 2 years ago0Aah I see exactly what you mean, thanks for the link! :)SharerePost-User-0765265answered 2 years ago0Aah this is tricky... I don't think we will support non-uniform scale for the White Box Tool as it makes editing really difficult. The Manipulators you use to interact with the tool operate in the local space of the Entity/Component and if you introduce non-uniform scale they won't behave as expected. We've encountered lots of issues with non-uniform scale in the past (especially when it comes to Entity hierarchies) and to be honest in my opinion it's best to just avoid it and instead use the available editing operations. That's my take anyway :PSharerePost-User-0765265answered 2 years ago0Hi @REDACTEDUSERThanks for the feedback! Multi-selection is a great shout and it should be quite easy to do. Mirror is a really nice feature as well. Merge is trickier (gets into Boolean operations) but it's something we do have planned for the future.Thanks again for checking out the tool and giving us feedback! Much appreciated :slight_smile:Cheers!TomSharerePost-User-0765265answered 2 years ago0This feedback is great. If you are using the WBT often, DM me and if you want, we can do a Q&A watch session with you so I can get feedback from you.I am apart of the Lumberyard Research Team.ThanksSharerePost-User-4369365answered 2 years ago0add(sentence):After creating the geometry - centering the pivot relative to the geometrySharerePost-User-0396919answered 2 years ago"


Question: Entitlement Server

I am a student and trying to do some practice. Does anyone know why I keep getting the error below while using the Redhat free-tier eligible?"Unable to read consumer identityThis system is not registered with an entitlement server. You can use subscription manager to register"FollowComment

Answer: "0Red Hat's recommendation for the This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. message on RHEL5 thru 8 is at https://access.redhat.com/solutions/291813  I notice from your screenshot that you are running RHEL9, but I expect the solution given should still work.The other error failed to download metadata for repo might be resolved by the above, or could be a separate issue. As the problem is that curl is timing out when talking to rhui.ca-central-1.aws.ce.redhat.com and as this is a Red Hat URL then it may be a fault on their side.It could still be something in the setup on the EC2. Start by verifying that outbound port 443/tcp is open in the security group associated with your EC2, and that the routing table and Internet Gateway (and possibly NAT Gateway) are all setup correctly.From the command line can you curl anything else, e.g. curl -k https://www.google.com ?These past questions might be worth reading even if they are not directly relevant to the problem at hand  https://repost.aws/questions/QUOqV95gnxSxOVNCmukuGo5w/unable-to-hit-rhel-package-repos https://repost.aws/questions/QUAyhZTQacQUy47Zki46B8VA/rhel-9-us-east-1-arm64-repos-having-issues-again-2-28-2023CommentShareRWCanswered 13 days ago"


Question: Flow of Presigned URL

"Want to understand the flow of Presigned URL, who assign that to app, likeApp request -> AWSApp request -> APPdoes it go to AWS at any point to get the URLPlease helpFollowComment"

Answer: "0I think a little bit of context could be useful to help you. I'm guessing that you are using S3 Presigned URLs  to serve static content in a web/mobile app? If so, usually your backend will call (through the AWS SDK) the S3 API to create the presigned URL, and it will deliver that URL to your frontend. The frontend will then call that URL to download the content protected by the presigned URL.If that wasn't your case, feel free to add more context in order to help you!CommentSharePablo Guzmananswered 8 months ago"