Question: Elastic beanstalk database migrations no longer running

"I am hosting a django site via elastic beanstalk and I have a 01_migrate.sh file in .platform/hooks/postdeploy in order to migrate model changes to a postgres database on Amazon RDS: #!/bin/shsource /var/app/venv/staging-LQM1lest/bin/activatepython /var/app/current/manage.py migrate --noinputpython /var/app/current/manage.py createsupython /var/app/current/manage.py collectstatic --noinputThis used to work well but now when I check the hooks log, although it appears to find the file there is no output to suggest that the migrate command has been rani.e. previously I would get the following even if no new migrations:2022/03/29 05:12:56.530728 [INFO] Running command .platform/hooks/postdeploy/01_migrate.sh2022/03/29 05:13:11.872676 [INFO] Operations to perform:  Apply all migrations: account, admin, auth, blog, contenttypes, home, se_balance, sessions, sites, socialaccount, taggit, users, wagtailadmin, wagtailcore, wagtaildocs, wagtailembeds, wagtailforms, wagtailimages, wagtailredirects, wagtailsearch, wagtailusersRunning migrations:  No migrations to apply.Found another file with the destination path 'favicon.ico'. It will be ignored since only the first encountered file is collected. If this is not what you want, make sure every static file has a unique path.Whereas now I just get:Found another file with the destination path 'favicon.ico'. It will be ignored since only the first encountered file is collected. If this is not what you want, make sure every static file has a unique path.```I dont know what has occurred to make this change. Of potential relevance is that eb deploy stopped being ablke to find the 01_migrate.sh file so I had to move the folder and its contents .platform/hooks/postdeploy/01_migrate.sh up a to the parent directory and then it became able to find it again.FollowComment"


Question: Why aren't the HTTP headers passed from API Gateway to Step Functions?

"I have an endpoint set up on API Gateway with integration to a step function - the integration is working well and my function is executed. However I have a need to access the headers on the initial request to the api gateway (as they need to be passed on to an API call made by one of the steps in the step function) I've added the http headers to the API Gateway Method Request and also done this in the HTTP Headers section of the Integration Request, then in the mapping template I have#set($inputbody = $input.json('$')){   "method": "$context.httpMethod",   "input": "$util.escapeJavaScript($inputbody)",   "stateMachineArn": "MyStateMachineARN",   "headers": {        #foreach($param in $input.params().header.keySet())        "$param": "$util.escapeJavaScript($input.params().header.get($param))"        #if($foreach.hasNext),#end        #end    }}When I test this I see the headers in the logs after the request body has been transformed - before it executes the step functionrequest body after transformations: {   "method": "POST",   "input": "{\"surname\":\"TESTSURNAME\"}",   "stateMachineArn": "MyStateMachineARN",   "headers": {                "HeaderA": "ValueA"        ,                "HeaderB": "ValueB"                    }}But in the step functions I am struggling to see the headers - the input I can see at the start of the execution is only{   "surname::"TESTSURNAME"}I have inputPath set to $ and the same for the payload.All the suggestions I've found online point to the mapping template but I can not get it to work - any ideas what I'm doing wrong?FollowCommentEduardo a year agoThe input to the Step Functions is only the string in the input property, so you will need to compose the input with both the body and the headers. At the core, API Gateway is calling StartExecution.Share"

Answer: "1You should modify your mapping template to include the headers as part of the Input param as this is the only field which is passed to the Step Functions execution.Something like this (did not actually try it so there may be some syntax mistakes):#set($inputbody = $input.json('$')){   "method": "$context.httpMethod",   "input": {        "$util.escapeJavaScript($inputbody)",       "headers": {          #foreach($param in $input.params().header.keySet())          "$param": "$util.escapeJavaScript($input.params().header.get($param))"          #if($foreach.hasNext),#end          #end      }   }   "stateMachineArn": "MyStateMachineARN"}CommentShareEXPERTUrianswered a year agoNaveen Reddy Marthala a year agoi have a similar problem. i haven't been able to pass headers from method-request to integration request without including them in the body(https://www.repost.aws/questions/QUKKPMq8TeQ5OTeuhAnu8G5g/pass-headers-from-clients-via-method-request-to-integration-request-with-body-intact-for-rest-api-in-aws-api-gateway). does this pass the body as is to the integration request including the headers?Share"


Question: Can mfa-delete be enabled with external IdP users?

"If this is true, how do I obtain the device serial?You are able to add mfa devices to identity center accounts in the sso portal, although I am unsure what these are used for. I neither see this in the identity center dashboard or see a cloudtrail entry for the device registration.FollowComment"

Answer: "-1Hi there! Good question, and the simple answer is yes, you can enable mfa-delete with external IdP users, but you need to configure some additional steps. I've tried to break this down into some steps you can test/implement.First, you need to connect your external IdP to AWS IAM Identity Center (successor to AWS Single Sign-On) using SAML 2.0 protocol1. This will allow your users to sign in to the AWS access portal with their existing credentials from the external IdP.Second, you need to provision your users and groups from the external IdP into IAM Identity Center before you can assign them permissions to AWS accounts or applications. You can do this manually or automatically using SCIM.Third, you need to enable mfa-delete on your S3 buckets using the AWS CLI or the API. You also need to associate your buckets with an AWS service that supports MFA delete, such as Elastic Load Balancing or CloudFront.Fourth, you need to obtain the device serial number and the authentication code from your external IdP when you perform any operation that requires MFA delete, such as deleting a version or changing the versioning state of a bucket. You can use the AWS CLI or the API to pass these parameters along with your security credentials.Let me know if you have any issues with this, or if it helps you then please accept my answer after you've tried it out - it would be much appreciated!Good luck :)CommentSharerePost-User-1144959answered 15 days agoDT-AWSForums 15 days agoThank you for the response. I am at step 3: in order to enable mfa-delete, mfa is required for the request. The CLI reference states the format is "<SERIAL> <MFA Token>". For an IAM user, this would presumably be the arn, but I am unsure what the format is for a device registered in the SSO portal as I see no references to it in the mangement portal, or via CLI. The docs refer to a MFA devices tab in the Identity Center portal, but I do not see that.Share"


Question: Datasync EFS to S3 not working

"Hi AWS team,I am trying to following https://aws.amazon.com/premiumsupport/knowledge-center/datasync-transfer-efs-s3/ to transfer data from EFS to S3 but currently I am unable to select S3 as destination location. The only options available are  EFS & SMB.Not sure why this is happening. I have tried to create locations separately but even then S3 location is not present for destination.Datasync agent is created as EC2 & activated. I have tried with & without endpoints but still unable to find S3 when selecting destination location.Attached are few screenshots.Regards,VikasEdited by: vikassachdeva on Apr 6, 2020 5:52 AMFollowComment"

Answer: "0Thanks for reaching out. For EFS to S3 transfers, the source location needs to be defined as NFS. Could you please double check whether you defined the source as NFS (vs EFS)?Regards,OlgaAWS DataSyncCommentShareAWS-User-9583871answered 3 years agoNathan Zook--Carrum Health a year agoThis advice is out of date.  There is currently no problem selecting an EFS drive as a source location.  Now, if your policy requires encryption in transit, that is another issue...Share0Thanks Olga,Yes that was the mistake, I selected EFS instead of NFS.It works with NFS.Regards,VikasCommentSharevikassachdevaanswered 3 years ago"


Question: Not able to see ElasticMapReduceEditors-Livy security group

"Hi,I have been trying to connect EMR notebook to Git for last 2days. I am following steps in below linkhttps://aws.amazon.com/premiumsupport/knowledge-center/link-emr-notebook-git-repository/I am following exactly same steps until settings for ElasticMapReduceEditors-Livy security group are mentioned. I do not see any security group with name ElasticMapReduceEditors-Livy, hence notebook is not able to connect to git. Could someone please help?Regards,DeeptiFollowComment"

Answer: "0Please check Step 4, make sure you don't miss "In the Software configuration section, be sure that you select a configuration that includes Apache Spark, Apache Hadoop, and Apache Livy.".  An EMR notebook uses Apache Livy to communicate with the cluster via a proxy using TCP Port 18888. By creating custom security groups with rules tailored to your environment. The default EC2 security group for the master instance is associated with the master instance in addition to the cluster's security groups for the master instance. Group Name: ElasticMapReduceEditors-Livy.See -> Specifying EC2 security groups for EMR Notebooks - https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-security-groups.htmlCommentShareAWS-User-8719617answered a month ago"


Question: "The Windows host for EC 2 has Port 3389 open, but the console has been prompted to open 3389"

"My new virtual machine has been disconnected, do you have a solution? Thank youFollowComment"

Answer: 0Did you open the port in the security group? Do you have a NACL blocking the port?Where are you trying to access the host from?Look at the following document with some possible solutionshttps://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/troubleshoot-connect-windows-instance.html#rdp-issuesCommentShareYair Carelanswered 5 months ago


Question: "Elasticsearch 7.7 Update by March 29, 2023 comment"

"I was looking at the verison info for our ElasticSearch version 7.7 and there was a comment in the version info page that read "Update by March 29, 2023". Does that simply mean this version will no longer be supported, or that it will cease to function on that date?FollowComment"

Answer: "0AWS currently does not have an End of Life policy for OpenSearch/ElasticSearch versions. However, we always recommend our customers to upgrade to the latest version to benefit from new features, bug fixes and performance improvements. As for the end of life, if and when AWS will plan to end support for a specific version, AWS will provide initiation via emails regarding the same and also provide enough notice period (12-18 months) for customers to be able to upgrade their domains.CommentShareRoianswered 10 days ago"


Question: S3 return 403 error when generating Whatsapp Image Preview Links (opengraph tag)

"Our website uses S3 for media hosting (only images), with its own url (midia.xxxxxx.com.br) and certificate in "flexible" mode, mediated by Cloudflare (which acts as a primary DNS and proxy, we do not put our certificate on S3 because we use let's encrypt at source). Everything works fine, except for one detail: when we try to share a link in the Whatsapp application, the images offered by the opengraph tag cannot be embedded in the post. Whatsapp receives a 503 code. This does not happen with any other social network. For the rest, everything is ok.Access to the bucket is completely public and the images can be opened normally in any browser. Browsing is properly guided by ssl (albeit forcedly, by Cloudflare), and everything else in media management and loading works fine... Does anyone have any idea how to solve this?FollowComment"

Answer: "0Hello Jeff,Thank you for your question.The issue you are experiencing is a 403 error with the images when sharing your website image link in WhatsApp. The 403 Forbidden error can be resolved by opening the IAM console and checking the permissions to ensure permissions are granted for the s3:PutObject or s3:PutObjectAcl on the bucket.Since the other social media platforms are working with the S3 object, this appears to be a server-side configuration error with WhatsApp. Restart your device, refresh the application, and check if other users are experiencing similar problems with WhatsApp.Documentation:https://aws.amazon.com/premiumsupport/knowledge-center/s3-403-forbidden-error/CommentSharelatrunda_sanswered 9 months ago"


Question: Access denied by aws

"I did my hands on of remove group and then to fix by attaching policy in my aws account and it denied now everything my name admin group everything it is deniedAccess deniedYou don't have permission to iam:ListGroups. To request access, copy the following text and send it to your AWS administrator. Learn more about troubleshooting access denied errors.User: arn:aws:iam::033071476259:user/sagarikaService: iamAction: ListGroupsOn resource(s): arn:aws:iam::033071476259:group/Context: no identity-based policy allows the iam:ListGroups actionthis is what it is showing i treid a lot of thing but it is not fixing please help me outFollowComment"

Answer: "1Hi,It looks like user sagarika doesn't have enough permissions to list groups and perform some actions on IAM. To fix that, log in as a user with enough privileges, go to the IAM console and configure sagarika properly (according to what sagarika needs, making sure sagarika has the correct permissions, including ListGroups). You can use root for that. It is not a best practice to use root for daily operations, so try to avoid using root for daily operation, but you can use root to recover other users' access or perform specific tasks. In the link below, you'll see the second point is "Restore IAM user permissions."Tasks that require root user credentials: https://docs.aws.amazon.com/accounts/latest/reference/root-user-tasks.htmlIn other words... using root or other user with enough privileges, you have to make sure sagarika has an IAM policy that covers all actions sagarika has to perform, including ListGroups. I'd recommend to review all tasks that sagarika should do, not only ListGroups. I also recommend to use an AWS Managed Policy (those already created, pre-built) instead of creating a new one. In the link below you'll see there are some for IAM (IAMAccessAdvisorReadOnly, IAMAccessAnalyzerFullAccess, IAMAccessAnalyzerReadOnlyAccess, IAMFullAccess, IAMReadOnlyAccess, IAMSelfManageServiceSpecificCredentials, IAMUserChangePassword, IAMUserSSHKeys.) Take a look at them and select the best one for sagarika.AWS managed policies: https://docs.aws.amazon.com/aws-managed-policy/latest/reference/policy-list.html.Have a great day!!!CommentShareSergioAanswered 17 days agorePost-User-3218389 17 days agoi am not getting you the things are not going right every access is denied i tried to troubleshoot but the things i need to trouble shoot are not available even please help meShareSergioA 15 days agoHi, please, the first step is to get sagarika access back. For that, follow the instructions provided in the previous post. Use root or a user with admin privileges and change the policies assigned to sagarikaShare0Hi, please, the first step is to get sagarika access back. For that, follow the instructions provided in the previous post. Use root or a user with admin privileges and change the policies assigned to sagarikaCommentShareSergioAanswered 16 days ago"


Question: SageMaker Autopilot not generating Model Explainability (SHAP) report

"I have run an experiment on SageMaker studio and clicked on the best-trained model when the experiment finished. The right-click option gives me the option "Open in trail details". The new tab (named "Describe Trial Components") has the option/tab "Model Explainability" but does not have any data (SHAP values). Instead of the explainability data for the best model, it has the following instructions for a sample notebook:Model explainability can be generated when running feature attribution during training.You can view a sample notebook to get started.Isn't the Model Explainability data automatically created for the best-trained model? Is there an option or config  I'm missing?FollowComment"

Answer: "0After much reading and online research, the solution was to shut down SageMaker Studio (File -> Shut Down -> Shut Down All). For some reason, I had to shut it down twice for the model explainability report to show up. Probably some pending updates of SageMaker Studio were not done on my behalf.CommentShareStavrosanswered 8 months ago"


Question: Private EC2 Instance not able to connect to NAT Gateway

HiThis very much been there done that kind of situation. However it is not working for me. Created EC2 instance in public and private subnet. My NAT gateway is not reachable from private EC2 instance so simple ping to google would also not work. I went through  this scenario multiple times and everything seems correct. Please help.Thank you in advance.FollowComment

Answer: 0There are detail troubleshooting steps for this scenario in the following Knowledge center article:https://aws.amazon.com/premiumsupport/knowledge-center/ec2-internet-connectivity/CommentShareEXPERTTushar_Janswered 6 months ago0Yes tried everything. I am thinking it is an issue with the way my home network works or AMI has issue. it randomly worked before and now despite of trying everything does not work. Even if I open to and fron all it is not working. thanks for the help.CommentSharerePost-User-7572499answered 6 months ago


Question: Unknown system variable using SET parameter in LOAD DATA FROM S3 expression

"I'm trying to fill Aurora MySQL DB from CSV in an S3 bucket using the manual Loading data into an Amazon Aurora MySQL DB cluster from text files in an Amazon S3 bucket.In general, everything works fine, but when I’m using SET to modify data from CSV, I can’t do it, looks like SET parameter doesn’t work at all.Running this queryLOAD DATA FROM S3 's3://mybucket/source_data.csv'INTO TABLE ordersFIELDS TERMINATED BY ';'LINES TERMINATED BY '\n'IGNORE 1 LINES(order_id, @order_name)SET  order_name = UPPER(@order_name);I get Unknown system variable 'order_name' error.Moreover there are strange syntax issues highlighted by my IntelliJ IDEA, on the other hand you can see that SET is fine with the LOAD DATA INFILECan I use SET as it described in the manual, so what am I doing wrong with it?FollowComment"

Answer: "1I would like to inform you, that you are using the order_name as the user variable, which can be the reason for this error.LOAD DATA FROM S3 's3://mybucket/source_data.csv'INTO TABLE ordersFIELDS TERMINATED BY ';'LINES TERMINATED BY '\n'IGNORE 1 LINES(order_id, @order_name)SET  order_name = UPPER(@order_name);As per the AWS documentation [+]:col_name_or_user_var, ... – Specifies a comma-separated list of one or more column names or user variables that identify which columns to load by name. The name of a user variable used for this purpose must match the name of an element from the text file, prefixed with @. You can employ user variables to store the corresponding field values for subsequent reuse.For example, the following statement loads the first column from the input file into the first column of table1, and sets the value of the table_column2 column in table1 to the input value of the second column divided by 100.LOAD DATA FROM S3 's3://mybucket/data.txt'INTO TABLE table1(column1, @var1)SET table_column2 = @var1/100;SET – Specifies a comma-separated list of assignment operations that set the values of columns in the table to values not included in the input file.For example, the following statement sets the first two columns of table1 to the values in the first two columns from the input file, and then sets the value of the column3 in table1 to the current time stamp.LOAD DATA FROM S3  's3://mybucket/data.txt'INTO TABLE table1(column1, column2)SET column3 = CURRENT_TIMESTAMP;Hence, with the LOAD DATA INFILE you are using the columns, as I do not have access to your data, I will request you to check the above syntax and accordingly use the prefix “@”, with order_name. And in case you are still facing issue, I will suggest you to open a AWS Support case for appropriate troubleshooting.[+] Loading data into an Amazon Aurora MySQL DB cluster from text files in an Amazon S3 bucket - Parameters - https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html#AuroraMySQL.Integrating.LoadFromS3.Text.ParametersCommentShareSUPPORT ENGINEERKrishna_Banswered 3 months ago"


Question: Fleet provisioning and lambda triggering topics

"Hi,Sorry for bothering this forum with my silly questions.I got the fleet provisioning thanks to the help I received here.  Now I want to merge my deployed lambdas and those fleet provisioned devices.This document( https://d1.awsstatic.com/whitepapers/Designing_MQTT_Topics_for_AWS_IoT_Core.pdf ) clearly states that it is a good ideato include the thing name in the topics sent and received by a thing. I agree.Is there a way to use parameters when defining pubsubTopics in component configuration/recipe?  I tried {iot:thingName} but to no avail.IMO this is something required to be able to use fleet provisioning. I work on a project where we could end up with 1000's of devices, I don't thinkit is a good idea to trigger all the device where a lambda is deployed, have them check the payload to check it they are targeted, and only then perform the action.Cheers,FrançoisFollowComment"

Answer: "0Hi François,No question is a silly question :) Greengrass v2 is still relatively new and we expect there to be some bumps along the way as customers use it. Your questions help us identify those bumps in the software and documentation that we can hopefully smooth over.There are two things to consider with regard to the MQTT pub/sub topics. First, from the device to IoT core, authorization for pub/sub topics are controlled by your IoT policy. Here you can specify an IoT policy that only allows the device to publish to topics with the {iot:thingName} in it.Second, on the device you can further define which topics a particular component is allowed to publish on. Just because you authorize a component to publish/subscribe on some topic, it does not mean that your IoT policy associated with device certificate allows the device to publish/subscribe on that topic.At this time, the component authorization policy does not support variables like {iot:thingName}. If you do not know the exact topics that the recipe needs to use, you would need to specify "*" to allow the component to pub or sub on any topic. However you can restrict your IoT policy so that it can only pub/sub on your restricted topics with the thingName in it.Please see  https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-iot-core-mqtt.html for more info.-robCommentShareRobanswered 2 years ago0Hi Rob,Thank you for the reply.I do understand that what an IoT thing can do wrt to AWS access, is ultimately controlled by policies attached  to the certificate associated with the  thing.(Although I am not clear how versioning is applied). For fleet provisioned devices, that policy is referenced in the fleet provisioning template and is automaticallyattached to the certificate generated upon provisioning.I also see now how resources for iot:Connect/Publish/Subscribe can restrict the use of topics containing   ${iot:Connection.Thing.ThingName}.but (you knew it was coming ;)My question is about topics used to trigger lambdas on a Greengrass IoT Core device. Those topics are defined in the " default configuration"/update  part of the recipe. According to the doc, a variable {iot:thingName} can be used but only in the lifecycle  part of the recipe. So no way to insert the thing name in the triggering topic.Ideally, the Greengrass IoT Core would process triggering topic for variables and instantiate them  before actually deploying the component. And then subscribe to those (as long as the policy associated with the certificate allows it ). (I guess it already does that for the lifecycle part)For example,  if you want to  perform a "reconfigure network" on a fleet provisioned device, you could deploy a lambda that reacts to a topic with a payload containing your SSID and passphrase. Ideally, your trigger would be something like com.example/MyThingName/network/configure  I do not see how to do that currently, instead I have to use a very generic topic likecom.example/network/configure  And add the think name in the payload to avoid re-configuring all the devices. In the process I have to broadcast my SSID and passphraseOr, did I miss something?Regards,FrançoisCommentShareAWS-User-0691707answered 2 years ago0Hi François,You have not missed anything. It is not possible to put the thing name into the pubsub topics that your lambda subscribes to unless you create a deployment with unique configuration for each device.If you moved away from lambda to a native GG V2 Component, then your component can subscribe to a topic with its thing name (which is provided in environment variables) since the subscription is driven by code instead of by configuration. The accessControl policy would need to include a wildcard for the thing name, but that is supported for the MQTT over IPC access controls.Cheers,MichaelCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago"


Question: Doubt about cloudwatch metrics

"Hi Everyone, please help me to understand the following situationI'm looking at my usage report and I can't understand why I'm being charged 0.10 for the 100.294 metricsconsidering, that my number of metrics is below 10,000 (1800 metrics), where does 100.294 come from?Thanks!cloudwatchmetriccost$0.10 per metric-month for the next 240,000 metrics100.294 Metrics$10.03$0.30 per metric-month for the first 10,000 metrics1800 Metrics$540.0FollowComment"

Answer: "0If I understood correctly, for that particular month, there were 1900.294 metrics in total. With the first 1800 you crossed 10000 since you started using Cloudwatch. Those 1800 were charged @ $0.3. The remaining 100.294 are charged @ $0.1 since you crossed 10000.This is the only possible explanation in my view.CommentShareSandeepVudataanswered a year ago0Hi, do you use both custom metrics and detailed monitoring? Is the count above only one of the 2 or the sum of both?FYI, both detailed monitoring and custom metrics count towards tiers - therefore if you look at only one of the 2, you might get the impression that you have less than 10,000 metrics in the first tier, but that could be e.g. 1,800 custom metrics and 8,200 detailed monitoring metrics in it for example.Can you please share more details on what filters got you those numbers?CommentShareJscanswered a year ago"


Question: Workspaces billing options - where to choose?

"Hello,I wish to create a workspaces bundle for employees.  Some are part time and some are full time.  The part time folks will use an hourly workspace VDI while the full time folks will use a monthly workspace VDI.Where is the option to choose between the two cost options when creating the workspace?TIAFollowComment"

Answer: 0Found the answer - Running modehttps://aws.amazon.com/premiumsupport/knowledge-center/workspace-modify-running-mode/Hope this helps others.ThanksCommentShareBESCanswered 3 years ago


Question: How to extract request headers for APIGW using CloudWatch Logs?

"I have created a Rest APIGW and created a log group in Cloud Watch and configured a logs in log format.Now, making a get call to APIGW. In headers I am sending TransactionID. How to capture this header in Logs. Please find below attached screenshot.I am able to capture rest of the logs. Is there any context variable to extract headers in log format?FollowComment"

Answer: "0In order to see Headers in APIGW logs you need to follow below stepsGo to your APIGW IDGo to stages ===> logs/tracingEnable CloudWatch logs with INFO level logging and you must Enable Log full requests/responses dataAfter configuring like above , you will be able to see the headers in APIGW execution logsAttaching some references:[1] --- https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html#:~:text=User%20Guide.-,Set%20up%20CloudWatch%20API%20logging%20using%20the%20API%20Gateway%20console,-To%20set%20up[2] --- https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-cloudwatch-logs/CommentShareAWS-User-9727102answered 9 months agorePost-User-9491665 9 months agoThanks for your comment.Actually I am trying to extract custom headers using Custom access logging. How to achieve this?Share"


Question: Query timestream from a lambda function on another account

I want to be albe to get data from a timestream table from another account's lambda function. Is it possible? Is there any guide? I could not find anything...FollowComment

Answer: 0The Lambda function will need to assume a role in the Timestream account. The role in the Timestream account will need to have a trust policy that allows this and an execution role/policy that allows access to Timestream. See: How do I configure a Lambda function to assume an IAM role in another AWS account?.CommentShareEXPERTkentradanswered 2 months ago


Question: How to install our new server vaccine file(program) on your server?

We are planning to purchase a new server vaccine program and install it on your server that we are currently using. Please tell me how to install it.Please give me detailed information including whether the file can be transferred directly and where to paste the file. etc. Thank you.FollowComment

Answer: 0Can you please provide some more details around what is it you are trying to install on your AWS servers? A link to the website of the software you are planning to install would be helpfulCommentShareEXPERTIndranil Banerjee AWSanswered 9 months ago


Question: Ingesting IoT Data to Timestream with Timestamp

"I wish to ingest data to Timestream with a custom timestamp from IoT. These timestamps will often be in the past -- but well within the retention period. Here are a couple sample messages:{  "time": "2020-09-03T17:50:07.790000",  "Battery Cranking Voltage": "12.035839999999999"}{  "time": "2020-09-03T17:42:28.770000",  "Electric Energy Out": "6798.6"}Note the time property is important and related to the measurement. Also note that the measurement changes with each message.  AND there are a large and flexible number of measurements.Setting an IoT Rule/Action to send this to Timestream will succeed in creating measure_names and _values, but with the WRONG timestamp.  The records are stamped with the INGEST time not the value from the time field.I can "fix" that by changing the NAME of the 'time' property to, say 'timestamp,' and parsing that field with time_to_epoch.  BUT, that will create extraneous records in Timestream where the measure_name is 'timestamp' and the value is the same as the time.So I'm left with a choice between doubling the size of my database or not having useful timestamps.Is there another way?---- EDIT ----To be more detailed, here is another run:example messages published on topic vt/cvra/teleTester`{"timestamp": "2020-10-01 19:50:36.050","Road Speed": "2.0"}{"timestamp": "2020-10-01 19:50:34.147","Gear Position": "3.0"}`IoT Rule SQL:SELECT * FROM 'vt/cvra/+/cardata'TImestamp value field:${time_to_epoch(timestamp, "yyyy-MM-dd HH:mm:ss.SSS")}Units : MILLISECONDSResults in tImestream:device_id   measure_value::varchar     measure_name      timeteleTester2020-10-01 19:50:36.050timestamp2020-10-01 19:50:36.050000000teleTester2.0Road Speed2020-10-01 19:50:36.050000000teleTester2020-10-01 19:50:35.050timestamp2020-10-01 19:50:35.050000000teleTester0.0Road Speed2020-10-01 19:50:35.050000000teleTester3.0Gear Position2020-10-01 19:50:34.147000000teleTester2020-10-01 19:50:34.147timestamp2020-10-01 19:50:34.147000000See how there are extra rows with the measure of timestamp?I'm looking to suppress that.FollowComment"

Answer: "0Accepted AnswerI did manage to work around this issue.  You simply cannot pass time in the payload.  You can use a timestamp from the payload, but it will creat unnecessary storage by also adding the property as a measure. I worked around it by passing epoch millis on the topic and extracting the timestamp from there. See http://iot.awsworkshops.com/amazon-timestream/lab131a-iotandtimestream/CommentShareAWS-User-2277746answered 2 years ago"


Question: NVIDIA Driver installation on g5.xlarge instance not working

"I have been trying to set up a g5.xlarge (windows server 2019) instance to run some tests on but I'm having difficulty with the NVIDIA driver installation. I followed this page: https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/install-nvidia-driver.html (Option 2), on how to install the driver correctly and from checking the device manager I can see the NVIDIA A10G card under display adapters. All the program files seem to be there as well. The device manager says teh device is working correctly and the Events log shows it installed the driver.I noticed I can't open the NVIDIA Control Panel, use NDI 5 Studio Monitor (Gives an error about OpenGL Shaders not supported), or find the GPU from my GC app which detects the GPUs on system to use for rendering if you prefer that. To me this would indicate that the driver isn't actually installed correctly because no applications seem to be able to find or use it. However, I was able to run GPU optimization commands into powershell (https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/optimize_gpu.html) and it worked just fine which I was not expecting since I thought it wasn't actually installed correctly.We have set up multiple other EC2 instances using g4dn and following the same installation process everything is working just fine. I need to specifically test the new g5 stuff with our products but like I said, I can't seem to get the GPU to work at allWould anyone have an idea as to why I can't use the GPU for anything and get it to start working?FollowComment"

Answer: "0HelloWe just got a g5-4xlarge running using the exact windows image that we used for a g4dn -4xlarge., with driver.Seems to working with openGL.hope that helps a little.-lpCommentShareluciopanswered 2 years ago0Turns out I need more disk space available. I extended the amount of disk space but had to move some stuff around to get enough space for it. The lack of error message from NVIDIA threw me for a loop :)CommentShareswhitesellanswered 2 years ago"


Question: AWS Greengrass Module 4 Error In Subscriber Device

"Hi all,I´m working on AWS greengrass guides and experienced some problems during the Testing Communications section in module 4 (https://docs.aws.amazon.com/es_es/greengrass/latest/developerguide/test-comms.html), I´m not able to receive the message on the subscriber terminal window. The publisher works fine and publish the messages without problem, but once I run the subscriber I get the following:Now we persist the connectivity/identity information...Now proceed to the connecting flow...2019-02-07 12:03:24,920 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Initializing MQTT layer...2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Registering internal event callbacks to MQTT layer...2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - MqttCore initialized2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Client id: HelloWorld_Subscriber2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Protocol version: MQTTv3.1.12019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Authentication type: TLSv1.2 certificate based Mutual Auth.2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Configuring certificates...Trying to connect to core at fe80::ba27:ebff:fe9e:9e9a:88832019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Configuring endpoint...2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Performing sync connect...2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Performing async connect...2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Keep-alive: 600.000000 sec2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Event consuming thread started2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.mqtt_core - DEBUG - Passing in general notification callbacks to internal client...2019-02-07 12:03:24,928 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Filling in fixed event callbacks: CONNACK, DISCONNECT, MESSAGE2019-02-07 12:03:25,960 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Cleaning up before stopping event consuming2019-02-07 12:03:25,960 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Event queue cleared2019-02-07 12:03:25,960 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Stopping network I/O thread...2019-02-07 12:03:25,960 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Network thread stopped2019-02-07 12:03:25,960 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Event callbacks cleared2019-02-07 12:03:25,960 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Event consuming thread stopped2019-02-07 12:03:25,960 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Waiting for event consumer to completely stop2019-02-07 12:03:25,970 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Exiting dispatching loop...2019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.mqtt_core - DEBUG - Event consumer stoppedError in connect!Type: <class 'socket.error'>Error message:Trying to connect to core at 192.168.137.46:88832019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Configuring endpoint...2019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Performing sync connect...2019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Performing async connect...2019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Keep-alive: 600.000000 sec2019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Event consuming thread started2019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.mqtt_core - DEBUG - Passing in general notification callbacks to internal client...2019-02-07 12:03:25,980 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Filling in fixed event callbacks: CONNACK, DISCONNECT, MESSAGE2019-02-07 12:03:26,160 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Starting network I/O thread...2019-02-07 12:03:26,160 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Produced [connack] event2019-02-07 12:03:26,170 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Dispatching [connack] event2019-02-07 12:03:26,170 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - No need for recovery2019-02-07 12:03:26,170 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Invoking custom event callback...2019-02-07 12:03:26,180 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Performing sync subscribe...2019-02-07 12:03:26,180 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Adding a new subscription record: 'hello/world/pubsub' qos: 02019-02-07 12:03:26,180 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Filling in custom suback event callback...2019-02-07 12:03:26,180 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Produced [suback] event2019-02-07 12:03:26,188 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Dispatching [suback] event2019-02-07 12:03:26,188 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Invoking custom event callback...2019-02-07 12:03:26,188 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - This custom event callback is for pub/sub/unsub, removing it after invocation...2019-02-07 12:03:46,161 - AWSIoTPythonSDK.core.protocol.connection.cores - DEBUG - stableConnection: Resetting the backoff time to: 1 sec.2019-02-07 12:03:55,486 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Produced [disconnect] event2019-02-07 12:03:55,486 - AWSIoTPythonSDK.core.protocol.connection.cores - DEBUG - backOff: current backoff time is: 1 sec.As you can see the program stays at "Resetting the backoff time to: 1 sec" forever, I have deleted the last devices and cfreated new ones, also re-deploy the group configuration but the problem persists.Thank you very much for any helpFollowComment"

Answer: 0CommentShareHanselpucteranswered 4 years ago0The problem was solved by removing the single quotes ('     ') in the topic parameter in Publisher and SubscriberCommentShareHanselpucteranswered 4 years ago


Question: Copilot pipeline using image from ECR and triggered by GitHub push

"I have copilot working fine when triggered by a git push to GitHub. But when I try to use an image from a private ECS repository that I built separately, it fails to pull in the changed source code from GitHub. It is triggered by the push to GitHub and deploys just fine but with outdated code. What could I be missing?FollowComment"


Question: Disable the video recording while interacting with the device.

"We are testing OTT based App, so most of the video content are DRM protected. So to test these content we need to disable the video recording for testing. Please let us know the procedure to disable to recording while interacting with the device.FollowComment"


Question: How do I install the RPM Fusion repo on Amazon Linux 2022?

"As per these instructions I’m trying to install the RPM Fusion repo on an Amazon Linux 2022 container spun up in Docker, but it doesn’t work:bash-5.1# dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpmLast metadata expiration check: 0:38:10 ago on Wed Jun 15 10:01:47 2022.rpmfusion-free-release-34.noarch.rpm             14 kB/s |  11 kB     00:00    Error:  Problem: conflicting requests  - nothing provides system-release(34) needed by rpmfusion-free-release-34-1.noarch(try to add '--skip-broken' to skip uninstallable packages)I understood that Amazon Linux 2022 was Fedora 34, so this should work, right?FollowComment"


Question: Encountered unsupported property UpdatePolicy on ElastiCache::ReplicationGr

I'm getting "Encountered unsupported property UpdatePolicy" on AWS::ElastiCache::ReplicationGroup. According to  https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-updatepolicy.html#cfn-attributes-updatepolicy-useonlineresharding it should be supported.FollowComment

Answer: "0Whoops, I was incorrectly putting the UpdatePolicy under "Properties". It should be a top-level field, like so:RedisCluster:Type: AWS::ElastiCache::ReplicationGroupUpdatePolicy:UseOnlineResharding: !Ref UseOnlineReshardingProperties:...CommentSharemariainesanswered 2 years ago"


Question: Latency in GET requests

"Hello.I wrote a code in Python that extracts data from the FTX exchange using their API.I am running the code in an AWS instance (free plan, t2.micro), located very closeby to the servers of the exchange.The code is essentially an infinite loop. At each step, it sends 3 'GET' requests, elaborates the response, and then goes to the next step.For the first few hundred iterations, the latency (defined below at the end of the post) for each block of three requests is of the order of 0.3seconds. After some time, it starts to grow up, reaching values from 2 to 5 seconds. In my local computer, located in the US, the latency is pretty constant at 1 second.There are no ratelimits in the FTX API for GET requests, so I should not expect any limit from the server. Is AWS limiting the rate of GET requests that I can make?I am trying to understand the origin of this extra-latency. To do so, I have monitored the https data traffic with tcpdump and I have modified the python script so that it stops as soon as it experiences a latency > 2 seconds. In this way, I can isolate the last packets in the tcpdump output and try to understand the origin of the delay.However, I really don't know how to read the output (I uploaded it here https://pastebin.com/tAhcicPU).Can anyone help me to understand the origin of the latency?104.18.33.31.443 is the IP of FTX server172.31.9.8 is the IP of the machine where my code runsDefinition of latency used here: I post the relevant part of the code where I compute the latencylatency=0for pair in pairList:  # pairList  = ['BTC/USD','ETH/BTC','ETH/USD']    api=requests.get(f'https://ftx.com/api/markets/{pair}/orderbook?depth={20}')latency+=api.elapsed.total_seconds()return latencySo, it is the total sum of the latency returned by the requests.get for each request.FollowComment"

Answer: 0T-series instances use a concept of "burstable" performance where the instances has a number of CPU credits; when busy these are used; when idle credits build up. If there are no credits then the performance of the instance returns to a (lower) baseline. What you might be seeing is that you're running out of CPU credits. You can monitor the CPU credits so that might be worthwhile checking.If this is causing the latency you might consider other instance types that do not use CPU credits (which is every other instance type) or you can enable Unlimited mode - noting that both of these solutions will have extra costs; and if you're using Unlimited mode all the time it it's best to consider a non-burstable instance type.CommentShareEXPERTBrettski-AWSanswered 8 months ago


Question: Reboot stuck at Reached target Shutdown for two instances

"Can someone assist rebooting the following instances?i-098ec4b6233be75a0i-0c10d51f5d89f9f4cUpdate: Screenshot shows that both instances are stuck at:Booting from Hard Disk 0...I rebooted 4 total instances that are identical from the same AMI and two of them (above) are failing to complete boot-up. Please assist.Much obliged,AlFollowCommentAJ a year agoComon guys, this isn't a problem that I should need paid technical support for. Something is wrong with the boot process and I haven't tampered with it.  Two instances of the same AMI booted and two failed. Can anyone look at this?ShareAJ a year agoThe two failed instances are in the "Running" state, so I am being billed for two instances that are not actually running.Share"

Answer: "0I stopped/started the instances and I was able to connect. Don't know if they resumed the prior reboot command or what, so I attempted to reboot again and they completed this time. A big mystery. Doesn't instill confidence.So, the solution Amazon provides us is the Microsoft mantra, reboot/restart/reboot/restart/re<verb>/re<verb>/re<verb>... until the thing starts working again.  The future is here, people, and it requires frequent starts!CommentShareAJanswered a year ago"


Question: Custom name for a sample application

Is it possible to give a custom name to the sample application used in a Cloud Formation template. I'm using the AWS provided sample app with the S3Key: java-sample-app.zip. When I deploy it using my template the name I see contains the name of the stack + the application name + an ID:Some parts of my template:CustomerApplication5:    Type: AWS::ElasticBeanstalk::Application    DeletionPolicy: Retain    Properties:      ApplicationName: CustomerApplication5      Description: Test Application  CustomerApplication5Version:    Type: AWS::ElasticBeanstalk::ApplicationVersion    DeletionPolicy: Retain    Properties:      ApplicationName:         Ref: CustomerApplication5      Description: Sample Application      SourceBundle:         S3Bucket: !Sub "elasticbeanstalk-samples-${AWS::Region}"        S3Key: java-sample-app.zipMyEB:    Type: AWS::ElasticBeanstalk::Environment    DeletionPolicy: Retain    Properties:      ApplicationName:        Ref: CustomerApplication5      EnvironmentName: customerApp5      Description: EB Test environment      TemplateName:        Ref: MyEBConfigurationTemplate      VersionLabel:        Ref: CustomerApplication5VersionFollowComment

Answer: 0You can change the property  " ApplicationName" under  resource CustomerApplication5 properties and that should help . For exampleCustomerApplication5:    Type: AWS::ElasticBeanstalk::Application    DeletionPolicy: Retain    Properties:      ApplicationName: -------------  could be replaced with what you wanted in application name.CommentShareEXPERTAWS-User-7407181answered 7 months ago


Question: cant download the AWS OpenSearch dashboard as either PDF or PNG

Created Opensearch dashboard and need to download as PDF. After i click the reporting tab next to "EDIT" Generate Reports appears with Download PDF and PNG. But after i click the PDF OR PNG It keeps refreshing and cant download.FollowComment

Answer: "0Hi there, you mention that you cannot download the reports from the dashboard as either PDF or PNG. It could be possible that the domain is breaking the reporting feature.Have you tried to use CSV reports and reformat them outside of OpenSearch as desired. Please find the below steps to generate CSV report from Kibana.1. Save a search. 2. Create a Visualization from the saved search.3. Click on "Inspect" in the top right corner of the Kibana/Dashboard page. 4. Then you will get an option to Download as CSV (Export:Raw CSV or Formatted CSV).5. After downloading the CSV file you can use a site like Convertio[1] to convert the CSV file to PDF as you desiredReferences:[1] https://convertio.co/csv-pdf/CommentShareAntonioanswered a year agorePost-User-3620943 a year agoWhats the diffrent between raw csv and formatted csv? seems like there is no differenceShareAntonio a year agoOne is comma separated and the other can be in a different format , like in a tabular form.Share"


Question: Bucket Compression to Reduce Data Transfer Out?

"Dear AWS Communities,I would like to transfer my 32TB of csv.gz files from S3 Standard out to internet.However, the cost is gigantic for an individual like me.I am wondering is there any way to perform bucket-level compression to further reduce the size of the bucket?If so, could you provide some instructions to do so?Much appreciatedFollowComment"

Answer: 0Accepted AnswerWould a Snowball Edge work for your use case? I ran it through the pricing calculator and it cut the price in half. Not sure you will be able to achieve that with more compression.CommentShareEXPERTkentradanswered 12 days ago


Question: DMS Service roles....

"For setting up the DMS replication instance, the following two roles need to be created as per the AWS [https://docs.aws.amazon.com/dms/latest/userguide/security-iam.html#CHAP_Security.APIRole] -  dms-vpc-role and dms-cloudwatch-logs-role. However, can the service roles be prefixed with some characters e.g. XYZ-dms-vpc-role and XYZ-dms-cloudwatch-logs-role or does it have to be exactly the same as outlined in the documentation?FollowComment"

Answer: "1Accepted AnswerIf you will try to create the DMS replication instance without the dms-vpc-role you will get the following error:The IAM Role arn:aws:iam::<accountId>:role/dms-vpc-role is not configured properly.So to answer your question is yes, your AIM Roles need to have the exact same names. In the page you linked there are the instructions on how to create those roles https://docs.aws.amazon.com/dms/latest/userguide/security-iam.html#CHAP_Security.APIRoleCommentShareSimone Tallevianswered 2 months ago1Those roles names are mandatory: the DMS service code tries to assume them with this exact name with code close to https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role.htmlSo, if you use another modified name, it will fail because the role name (or ARN which can be derived)  of the assume by DMS will not exist.CommentShareDidier_AWSanswered 2 months ago"


Question: AWS Glue Connection with Secrets Manager Issue

"I have a Glue Job for migrating data from a Postgres database in RDS to a S3 bucket in Parquet format and a Crawler that connects to Postgres to infer the table schema. Previously, the Glue Connection used by both was configured to authenticate to Postgres via Username and Password, however now I would like it to authenticate via credentials stored in Secrets Manager instead. After updating the Glue Connection to use Secrets Manager, the Glue Job is failing with the following error:2022-12-14 14:58:07,092 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(73)): Error from Python:Traceback (most recent call last):  File "/tmp/parquet-job.py", line 25, in <module>    database=glue_source_database, table_name=table, transformation_ctx="Datasource")  File "/opt/amazon/lib/python3.6/site-packages/awsglue/dynamicframe.py", line 787, in from_catalog    return self._glue_context.create_dynamic_frame_from_catalog(db, table_name, redshift_tmp_dir, transformation_ctx, push_down_predicate, additional_options, catalog_id, **kwargs)  File "/opt/amazon/lib/python3.6/site-packages/awsglue/context.py", line 186, in create_dynamic_frame_from_catalog    makeOptions(self._sc, additional_options), catalog_id),  File "/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 1305, in __call__    answer, self.gateway_client, self.target_id, self.name)  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 111, in deco    return f(*a, **kw)  File "/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py", line 328, in get_return_value    format(target_id, ".", name), value)py4j.protocol.Py4JJavaError: An error occurred while calling o75.getCatalogSource.: java.util.NoSuchElementException: None.getat scala.None$.get(Option.scala:349)at scala.None$.get(Option.scala:347)at com.amazonaws.services.glue.util.DataCatalogWrapper.$anonfun$getJDBCConf$1(DataCatalogWrapper.scala:218)at scala.util.Try$.apply(Try.scala:209)at com.amazonaws.services.glue.util.DataCatalogWrapper.getJDBCConf(DataCatalogWrapper.scala:209)at com.amazonaws.services.glue.GlueContext.getGlueNativeJDBCSource(GlueContext.scala:487)at com.amazonaws.services.glue.GlueContext.getCatalogSource(GlueContext.scala:320)at com.amazonaws.services.glue.GlueContext.getCatalogSource(GlueContext.scala:185)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)at py4j.Gateway.invoke(Gateway.java:282)at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)at py4j.commands.CallCommand.execute(CallCommand.java:79)at py4j.GatewayConnection.run(GatewayConnection.java:238)at java.lang.Thread.run(Thread.java:750)This is the code for the Glue Job script:import sysfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom awsglue.context import GlueContextfrom awsglue.job import Jobargs = getResolvedOptions(sys.argv, ['JOB_NAME', 'TABLE_NAME', 'PARQUET_LOCATION', 'PARQUET_TABLE_NAME'])sc = SparkContext()glueContext = GlueContext(sc)spark = glueContext.spark_sessionjob = Job(glueContext)job.init(args['JOB_NAME'], args)glue_source_database = "postgres-db"table = args['TABLE_NAME'].replace("-", "_")output_s3_path = args['PARQUET_LOCATION'] + \    "/" + args['PARQUET_TABLE_NAME']Datasource = glueContext.create_dynamic_frame.from_catalog(    database=glue_source_database, table_name=table, transformation_ctx="Datasource") # the script fails at this lineprint("Items Count:  ", Datasource.count())Datasource.printSchema()Transform = ApplyMapping.apply(frame=Datasource, mappings=[    ("id", "int", "id", "int"),    ("name", "string", "name", "string")],    transformation_ctx="Transform")DF = Transform.toDF()formatted_path = output_s3_path.replace("_", "-")DF.write.mode('overwrite').format(    'parquet').save(formatted_path)job.commit()I have confirmed that the IAM role for AWS Glue has permission to access my secret. The Crawler is running successfully using the same connection as the Glue Job. I also tried reverting the connection to use Username and Password, and the job succeeded. Is there anything that I am missing? Could this possibly be a bug with AWS Glue?FollowComment"

Answer: "1Hi,I see that you are facing the “NoSuchElementException: None.get” error when using Secrets Manager credentials over the database connection. Unfortunately getCatalogSource and getCatalogSink currently do not support Secret Managers connection which is resulting in this exception. There is already a feature request for the same. However, we do not have an ETA as to when this feature would be implemented. However as a workaround you can either use boto3 to retrieve credentials from secrets manager or use the username/password to your connection options.I hope the above information helps.Thank you.CommentShareSUPPORT ENGINEERSaiIndira_Chanswered 6 months agokyleh 6 months agoThat makes sense. Hopefully that feature will be implemented soon. Thank you!ShareLukas Gust 6 months agoI get the same error when using from_options. Is it a similar issue?connection_options = {        "useConnectionProperties": "true",        "dbtable": table,        "connectionName": connectionName,    }return glueContext.create_dynamic_frame.from_options(        connection_type='sqlserver',        connection_options=connection_options    )Share"


Question: Endpoint better usage in Amazon Pinpoint

"Hello,I am getting into using Amazon Pinpoint to send emails via campaigns and I have noticed that the biggest cost I have is the addition of endpoints (proportionally the cost of sending is zero).I currently load the endpoints via an API call (updateUserEndpoint) every time I send a campaign.Is it possible to reduce and curb these costs by combining Amazon Pinpoint with another service such as Amazon S3 (or others) ?Can endpoints be saved on some Amazon service to reduce MTA costs ?FollowComment"

Answer: "0I don't understand this question. There is no cost for loading endpoints. The ways to load endpoints into Pinpoint include directly through the API, as a custom event in the events API, or through an upload of CSV/JSON. You can add a file to s3 and have lambda load it as well. There are ways to daisy chain services. https://github.com/aws-samples/communication-developer-services-reference-architectures/blob/master/README.md#Amazon-S3-Triggered-Endpoint-ImportsCommentShareAnaAtAWSanswered 3 months ago"


Question: Word Press AWS Light Sail Bitnami Increasing PHP Max Input Vars

"Hi All,I have tried several times to edit my Wordpress theme  via vis php.ini file with no success.I am accessing the file via:cd /otp/bitnami/php/etclssudo sunano php.iniOnce in the file I go to the section: PHP Max Input Vars  and edit it from (default) 1000 and have tried to raise it to 3000 even though it saves in the php.ini file it does not reflect in my theme and I receive a fatal error each time I try to finalize the isntall of my wordpress theme.Any assistance would be apprecaited.FollowComment"

Answer: 0Accepted AnswerSolved:SHH Open in browsersudo chmod 777 /opt/bitnami/php/etc/php.inisudo -inano /opt/bitnami/php/etc/php.iniControl +w = search “max_input_vars” enter and change check line start without “;”Control + x yes after entersudo /opt/bitnami/ctlscript.sh restartphp -i | grep max_input_varsCommentSharerePost-User-9569334answered a month ago


Question: IoT Analytics Lambda enhancementt

"I'm trying to figure out how Lambda enhancement in the IoT analytics pipeline functions as it operates much differently then a normal lambda connected to IoT gateway.  I can't seen to extract my incoming JSON data so I can manipulate in in Lambda as I do with normal lambdas  outside IoT Analytics.also if I want to output the event object from my incoming JSON payload in lambda from IoT gateway I can use a simple:console.log(event);  and thenvar latitude = event.lat; //works properlyFrom cloudwatch I see my data output as:INFO{Reefer_ID: 34988,size: 128,color: 'green',lat: 45.695406,lon: -121.884298}But I cant extract the event object or its members from lambda from IoT gateway when using IoT analytics.  I noticed the event object is slightly different which may be the issue.  The IoT analytics event object from console.log in lambda is:INFO[ { Reefer_id: 1235,size: 128,color: 'green',lat: 45.695406,lon: -121.884298 } ]How do I assign the object members to a variable so that I can manipulate them?thanksEdited by: SteveB123 on Nov 24, 2019 12:53 PMEdited by: SteveB123 on Nov 24, 2019 3:04 PMEdited by: SteveB123 on Nov 24, 2019 3:05 PMFollowComment"

Answer: "0Accepted AnswerYes in this pseudocode example, you would access latitude withmessage.latinside the anonymous function.CommentShareRyan_Banswered 4 years ago0Hi SteveB,The integration between IoT Analytics pipeline and Lambda supports batching of messages. The default batch size is 1 when using the IoT Analytics console. This is why you see a list of one message instead of the same message forwarded by IoT Core.To work with batched messages sent by IoT Analytics, you need to iterate through the received event array. An example:let messagesReturnedToIoTAnalytics = [];event.forEach(message => {  // do some stuff to `message`  // ...code...  // then add to returned list  messagesReturnedToIoTAnalytics.push(message);});return messagesReturnedToIoTAnalytics;Hope this helps,RyanCommentShareRyan_Banswered 4 years ago0Hi Ryan,It definitely does help.  I was going to look at the Python example from the user guy and try to work backwards so this helps.  I will work on this today and Ill leave this open in case I get stumped again in the next couple of days, but this definitely helps.  thanks!CommentShareSteveB123answered 4 years ago0Hi Ryan,It definitely does help.  I was going to look at the Python google maps example from the user guide and try to work backwards so this helps.  I will work on this today and I'll leave this open in case I get stumped again in the next couple of days, but this definitely helps.which art of your pseudo code relates to the incoming ''event.member', in my case (event.lat)is it message.lat'  ?  since your not defining 'message' I'm assuming this is really the incoming event object?thanksEdited by: SteveB123 on Nov 25, 2019 9:02 AMCommentShareSteveB123answered 4 years ago0RyanB you are a genius, you have solved my problem after i've spend more then 6 hours on i,t to no avail.  I would have had to work back from the Python example to even get a clue thatwe had to parse batched messages to extract the event objects members.Do you know where is this documented?  I'm looking through the IoT Analytics documentation and I don't see this issue listed anywhere.  I'm wondering what AWS documentation explains this issue on another AWS service?  Thanks againSteveCommentShareSteveB123answered 4 years ago0I will pass this feedback on to the service team. This behavior is documented in the IoT Analytics user guide, under the Lambda pipeline activity: https://docs.aws.amazon.com/iotanalytics/latest/userguide/pipeline-activities.html#aws-iot-analytics-pipeline-activities-lambdaThe codefor e in event:shows in Python that an iterator is needed to act on individual messages. We'll see about making it more clear that this is required even for a batch of one message.RyanCommentShareRyan_Banswered 4 years ago"


Question: "Selenium - Send keys not working to '//input[contains(@type, 'file')]' with LocalFileDetector"

"I'm trying to sendKeys to a file handler in chrome and am getting: org.openqa.selenium.UnsupportedCommandException: unknown command: unknown command: session/1b651f6fc5776ba353569d3b7e423552/se/fileI am using the LocalFileDetector:        driver.fileDetector = LocalFileDetector()This works locally as well as on other testing platforms. Is this not supported in Browser Testing(Device Farm)?Command: [1b651f6fc5776ba353569d3b7e423552, uploadFile {file=UEsDBBQACAgIAFqqTlQAAAAAAAAA .......FollowComment"

Answer: "0Hello,I understand that you are getting an error when trying to SendKeys to file handler, using Selenium, in Device Farm Chrome Browser.The se/file endpoint is not part of the W3C specification, so it will not work by default. It is likely that older versions of Selenium server handles this implementation detail on it's own, but we do not use Selenium server at all.We recommend updating to W3C enabled client and see if it resolves the issue.Here's a relevant stack post:https://stackoverflow.com/questions/58296774/ruby-selenium-webdriver-3-142-6-unable-to-upload-file-due-to-seleniumwebdrivCommentShareSUPPORT ENGINEERIsha_Kanswered a year ago"


Question: Lambda function is not getting invoked if I call it inside a cronjob method or in a scheduler

"Hi,I have a cron job method-@Scheduled( cron = "* * * * * *", zone = "Asia/Calcutta" )    public void  fetchUnOcredFilesAndTriggerLambda()  { {   ...                 int totalNoOfPages = invokeLambdaFunctionAsync.lambdaOcr(fileModel.getAbsolutePath(), fileModel.getFileName(), fileModel.getId(), destination, fileModel);   ..             }the lambdaOcr() method does some minor processing and then calls the invokeAsync(req) method-https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/basics-async.htmlpublic int lambdaOcr(String filePath, String fileName, Long fileId, String destination, FileModel fileModel) throws Exception {........             lambda.invokeAsync(req);........}The problem I am facing here is the lambda.invokeAsync(req); is not getting invoked if I use the above cron job. I checked the console cloud watch logs and saw that no logs are coming. BUT If I try to invoke by creating the main method and passing the hard-coded inputs inside the fetchUnOcredFilesAndTriggerLambda() method to call the invokeLambdaFunctionAsync.lambdaOcr(<-hard-coded inputs->), the Lambda function gets invoked and I get the correct output. But I dont want to hard code the inputs and want to invoke the invokeLambdaFunctionAsync.lambdaOcr() using the cron job. But with the cron job, Lambda function is simply not getting invoked.I also tried to replacing @Scheduled( cron = " * * * * ", zone = "Asia/Calcutta" ) with @Scheduled( fixedDelay = 30000 ) but not getting any results. Please can anyone tell me where I am going wrong? How to use the cron job to invoke the Lambda function? I am doing this in Spring boot and Java and new to AWS Lambda. Any help appreciated...thanks.FollowComment"

Answer: "0Hello;I am not an expert on java, but I can advise you to use AWS EventBridge rules to schedule your lambda functions.  (https://us-west-2.console.aws.amazon.com/events/home?region=us-west-2#/rules)CommentSharebbolekanswered a year ago"


Question: Debugging DNS lookup of MariaDB instance.

"Hi. My serverless instance in AWS can reach my MariaDB database (RDS) and things work fine but things don't work if I try to connect from my local computer. The problem appears and disappears randomly, It works some days and it doesn't work on other days.Here is the result of ping to MYDB.eu-west-1.rds.amazonaws.comPING MYDB.eu-west-1.compute.amazonaws.com (52.209.116.0) 56(84) bytes of data.---MYDB.eu-west-1.compute.amazonaws.com ping statistics ---13 packets transmitted, 0 received, 100% packet loss, time 12448msHere is what it looks like if I connect with PMA in a docker container:I tried to reboot the DB.Is there something else I should try?FollowComment"

Answer: 0How many connections per second are being made from the EC2 instance to the RDS instance? You may be running into a DNS quota issue where the single EC2 instance is sending too many packets too quickly to resolve the RDS DNS entry.Info about this: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html#vpc-dns-limitsCommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 10 months agojohanmattsson 10 months agoThank you for your answer. The number of connections is very close to zero at the moment.Things are working again today but I assume I will run into the DNS issue again some day.Share


Question: Transit Gateway Logs - Bug report

I created an Athena table for vpc flow logs according to the documentation. However I noticed an inconsistency when comparing TG flow logs to other VPC flow logs.For other VPC flow logs the column account_id contains my account id (int) and interface_id contains eg 'eni-0eb0ac3356dee8030'.For TG logs account_id contains the string 'TransitGateway' and the interface_id contains my account id.To me this seems like a bug. Especially since it will mess up all tables that expect an int for account_id...FollowComment

Answer: 0Hi ThereQuerying TGW flow logs with Athena is not currently supportedSee https://docs.aws.amazon.com/vpc/latest/tgw/tgw-flow-logs.htmlYou can search TGW flow logs using CloudwatchSee https://docs.aws.amazon.com/vpc/latest/tgw/working-with-flow-logs.html#search-flow-log-recordsCommentShareEXPERTMatt-Banswered 7 months ago


Question: Cannot connect remotely to serverless Aurora MySQL

"I have recently created a serverless Aurora MySQL DB in our account.  I am attempting to connect to it remotely via mysql-workbench in order to setup the schema, etc.  However, I cannot seem to establish the connection.  As this document (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html ) recommends connecting via mysql command line following creation, I assume either command line or mysql-workbench are the recommended/common approaches.I am attempting to follow the steps under the 'A DB Instance in a VPC Accessed by a Client Application Through the Internet' section of this document: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.Scenarios.html#USER_VPC.Scenario4The associate Aurora security group appears to allow the 3306 port through, but I am still unable to connect: Your connection attempt failed...What am I missing to allow these connections please?FollowComment"

Answer: 0Short answer: Aurora Serverless is not accessible via a public IP - it is one of the documented limitations of Aurora serverless: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html#aurora-serverless.limitationsEither access via an EC2 instance within the VPC and/or potentially investigate ssh tunneling to provide access through such an EC2 instance to a local machine as a work around.CommentShareskidvdanswered 4 years ago


Question: How to store the Athena Query Results in DynamoDB table?

"Hello, recently working on a project that includes querying with Athena. What I wanted to achieve is to store the Athena results in a DynamoDB table with some meta-data.FollowComment"

Answer: "0Hi,One approach you can take is to run the Athena Query with the ResultConfiguration set to send the results to an S3 bucket. You can then create a lambda function that triggers on new s3 objects in this bucket and then reads in the results, adds the meta-data and then inserts the data into DynamoDB.-randyCommentShareRandyTakeshitaanswered 4 years ago0Speaking to why I want this is I have some Webpage that includes charts and whenever a user visits that webpage some API calls happen and Athena queries run on S3 data to fetch the data and then load that result data in charts and show it to the user. The data stored in S3 changes every week so let's say within a week if any user that visits the webpage sees the same data on charts no matter how many times he/she visits the webpage. So querying again & again the same data is not a cost-effective way when we already know the output would be the same within a week. So what I wanted to do is store the results in DynamoDB table a single time with a TTL & then check that if there is data in DynamoDB then directly fetch the data & perform some other functions.Will this be a good approach to use DynamoDB in such a use case?CommentSharemrrizvianswered 4 years ago0Hi,Yes, that sounds like a reasonable implementation for your use case.-randyCommentShareRandyTakeshitaanswered 4 years ago0Hello, Thanks for the answer and suggestions.CommentSharemrrizvianswered 4 years ago"


Question: Cannot create ECS service with existing load balancer

"Hi.I am trying to launch a service in ECS with my Docker image.I have successfully started it as a task, but without a domain name. Tried accessing it only by IP.Now I am trying to create it as a service. Again without an attached domain name, it works. However to attach a domain name I need to use ELB and that is when it becomes complicated.Step 1 config.:Launch type: FARGATENo. of tasks: 1Deployment type: Rolling update.The rest are by default.Step 2:Cluster VPC: existing oneSubnets: privateSecurity groups: created one for ECSAuto-assign public IP: disabledLoad balancer type: ApplicationLoad balancer name: choosing existing oneContainer name port: existing one on port 80Enable service integration: checkedand choosing a configured service from there.and when I click next step, the field "Container name port" becomes red with an error "Please configure a container with elb.".And that is when I do not know what to do next.I have checked the rules in ELB and there is one configured on port 80, which points the target group by the host name.The target group exists. However it does not have any targets. Which seems to be fine.In Route53 a private zone is created and the domain name is being created automatically when I try to create a service. So this also seems to work.The IAM role for the service has following policies attached:AWSApplicationDiscoveryServiceFullAccessAmazonECSTaskExecutionRolePolicyElasticLoadBalancingReadOnlyReadSecretsI have tried addingElasticLoadBalancingFullAccessAmazonRoute53FullAccessbut without any luck.Can you please advice what else to check that my be causing this issue?FollowComment"

Answer: "0I have ended with creating a new load balancer for my ECS service.And it worked totally fine for a few days until I have decided to update the task definition. Result: same error again on 2nd step - "Please configure a container with elb.".I have tried googling this error, but without any luck. This error clarification would be nice.CommentShareAlexASFASDASDASDanswered 4 years ago0Solution: is to click the button "Add to load balancer" and configure the parameters there, trying to re-use existing configurations.After a reply from the support, maybe the problem was that when you create a load balancer for the first time for your service, something is pre-configured, while on the subsequent attempts, you have to do it manually.CommentShareAlexASFASDASDASDanswered 4 years ago"


Question: Route 53 Hosted Zone not appearing outside of AWS after removal of DNS zone from lightsail and adding back to Route 53

"I built a POC using the Wordpress offering with Lightsail. As part of that, we migrated a domain name from Route 53 to Lightsail's DNS.  The site was accessible via wordpress/lightsail. When the POC was complete, we decided not to move forward with Lightsail and deleted the instance.The domain reappeared in AWS Route 53 with a SOA, and 4 NS records.  If I run the "test record" feature in Route 53\hosted zone, I get "no error."  Route 53 does not let you delete NS record so I am stuck with what is there.If I query my domain NS via DnsChecker.org or MxToolBox, I get no response for NS or SOA. I can't get to my domain from the Internet. This has been two weeks or appox 14+ days.  Any ideas?FollowComment"

Answer: "0You would have to first REGISTER your domain. This is where you tell your registrar of those NS records. You don't have to use AWS Route53 for this, but that might be the most straight forward option if you don't already have one. Process is explained in https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.htmlCommentShareKalluanswered 2 months ago0The domain has been registered through Route 53 for months. We have 24 registered and I had to get special approval to go over 20.I even deleted the hosted zone and recreated it.CommentShareIRP-100answered 2 months ago"


Question: Programatic way of Creating DynamoDB Global Table

When We create a dynamoDB global table by API interface using AWS Golang SDK or using AWS CLI interface it always creates it using dynamoDB 2017 version..  But 2017 version has a limitation of adding new replication region with the already data present in other regions. It mandates tables to be empty.. So we are looking for a programmatic way of creating dynamoDB Global table with 2019 version which doesn't have that problem in 2017 version..  Please do let us know is this supported ? And how to achieve this.. Thank you..FollowComment

Answer: "3Accepted AnswerTo create a V2019 of DynamoDB Global Table, you need to use the CreateTable API and NOT the CreateGlobalTable API.When DynamoDB moved to v2019 it aligned with the table class, which is why you are seeing the creation of v2017 using the global table class.Once you create the table, you call UpdateTable and specify the ReplicaUpdatesaws dynamodb create-table \    --table-name custT \    --attribute-definitions AttributeName=pk,AttributeType=S AttributeName=sk,AttributeType=S \    --key-schema AttributeName=pk,KeyType=HASH AttributeName=sk,KeyType=RANGE \    --billing-mode PAY_PER_REQUESTaws dynamodb wait table-exists \    --table-name custTaws dynamodb update-table \--table-name custT \--replica-updates '[{"Create": {"RegionName": "eu-west-2"}}]'CommentShareEXPERTLeeroy Hannigananswered 4 months agoEXPERTBrettski-AWSreviewed 4 months agorePost-User-7795558 4 months agoHi Leeroy,Thank you so much for the solution. It unblocked me..Best RegardsNagappaShareLeeroy Hannigan EXPERT4 months agoThat's good to hear Nagappa, please feel free to mark the answer as accepted.SharerePost-User-7795558 4 months agoAccepted Now.. Thank you ..Share0I just created a new table and replicas which are the 2019 version. What version of the SDK and CLI are you using? Which region is the primary DynamoDB table in?CommentShareEXPERTBrettski-AWSanswered 4 months agorePost-User-7795558 4 months agoThank you for the reply.. Did you create through AWS Console ?  I am using aws-sdk-go-v2 SDK .. Also you can use aws CLI  to create table and to create global table with two replicas .. When i do create createGlobalTable dynamoDB version becomes 2017..aws --versionaws-cli/2.8.10 Python/3.10.8 Darwin/21.6.0 source/arm64 prompt/offgithub.com/aws/aws-sdk-go-v2/service/dynamodb v1.17.7ShareBrettski-AWS EXPERT4 months agoI used the AWS CLI: aws-cli/2.4.18 Python/3.8.8 Darwin/21.6.0 exe/x86_64 prompt/offWhich regions are you using? I created my primary table in ap-southeast-2 and replicas in us-east-1 and eu-west-1.Share0Thank you for the reply.. Did you create through AWS Console ?  I am using aws-sdk-go-v2 SDK .. Also you can use aws CLI  to create table and to create global table with two replicas .. When i do create createGlobalTable dynamoDB version becomes 2017..CommentSharerePost-User-7795558answered 4 months ago"


Question: How to stop a EMR Notebook execution after a error in a cell

"Hi there,Do you know how to stop a notebook on cell error?It seems that the normal behavior is to continue on error, but I want to stop execution after the first error or fail,Maybe a notebook flag, or editor flag?We are using emr run notebook api, and we think that one year before it used to stop on error cell but not anymore.ThanksFollowComment"

Answer: "0Hello,Thanks for writing on re:PostThe functionality you have explained here is available when using the PySpark Kernel within a Jupyter Notebook.When raising an error within a cell, if that error is flagged in the run time, the notebook will not continue to run the following cells even when using the Run All Option.When an error is met, the notebook will output the information of this error and will stop executing.I hope the above information helped in addressing your concernThanksCommentShareSUPPORT ENGINEERMudasser_Aanswered 7 months agormis 7 months agoThanks!You're right, we don't have any problem with python3 kernel, just with pyspark.How do you flag them in runtime?Can you do that from emr start emr notebook api  or    step function start emr notebook execution?We raise exceptions inside cells and doesn't stop notebook executions,ThanksShare"


Question: VAT recovery on AWS invoices

"We are a UK based company and receive VAT invoices in US Dollars from AWS which include input VAT.The invoices include in the footer details of the invoicing entity  - AMAZON WEB SERVICES EMEA SARL and AMAZON WEB SERVICES EMEA SARL, UK BRANCH.There is a VAT reg number noted for the UK Branch.It therefore appears that we are being billed by a VAT registered UK branch of AWS EMEA SARL. Are we therefore able to reclaim the input VAT?FollowComment"

Answer: "0Hi there,I'd like to try & point you in the right direction to find an answer to your question.I looked into your concern & located this FAQ page that may help you with your inquiry: https://aws.amazon.com/legal/aws-emea/.If this link isn't quite right or if you have any further questions, please reach out to our Billing team for direct assistance by creating a support case here: http://go.aws/support-center.Best regards,— Aimee K.CommentShareEXPERTAWS Support - Aimeeanswered 25 days ago"


Question: Contact Lens - Raw/Unredacted File Deletion?

"Hello,My goal is to perform a recording with contact lens to redact voice/chat files and **never ** save the original files. For compliance reasons, I do not want this information in our systems if i can help it. It's not clear in the documentation, at least to me, that the original files in the S3 bucket from a recording will need to exist.I was able to complete the post-analysis redaction and limit what an agent can see when viewing a contact. They are unable to listen/see the unredacted conversation and only have access to redacted information. Great! From the Connect web page, this all seems to work fantastically.Now, to what is not so great. The S3 bucket still has the original file in the S3 bucket with the unredacted information. Yuk! I don't want to store any sensitive data, if possible, in my S3 bucket.The documentation seems to be a bit confusing on this matter:  (https://github.com/awsdocs/amazon-connect-admin-guide/blob/master/doc_source/enable-analytics.md#enable-redaction).Option #1: Generate both redacted and original transcripts and audio.Option #2: Generate redacted transcript only, and both redacted and original audio.Based on the options in the redaction, it seems to me to suggest that "original audio" will always exist?Down the way a bit when discussing using dynamic contact attributes we have the following options.Option #1: NoneOption #2: RedactedOnlyOption #3: RedactedAndOriginalHmmmm. "RedactedOnly" you say?I went ahead and setup a dynamic attribute and passed this to the recording and analytics behavior block thinking that I would only get the redacted information after post-analysis has been completed. ie: Delete the original file after it's done it's job.Am i missing something? Gotcha's not documented? Oversight on fundamentals?FollowComment"

Answer: "1Accepted AnswerHi pjn!While we do always automatically store an original recording (the RedactedOnly option you are seeing is for the transcript), you can immediately delete or expire this S3 Lifecycle Policies.CommentShareEXPERTAWS-samfredanswered 3 months agopjn 3 months agoHello!Thank you. I'm not sure what happened originally when i tested where i thought deleting the original audio also deleted the redacted. I tested again and that was not the case, so this helps significantly.Share0You may also need to consider what happens if you do require an audit on the raw original recordings. This may not be the case for your specific workload, but customers asked for the "source of truth" to be able to get access to the original information, as needed.CommentShareMagdaanswered 3 months agopjn 3 months agoAgreed. We do not have those stipulations today but will keep that in mind if ever asked.Share"


Question: Some mails rejected when using Workmail

"Hello,Since a few days, some mails are being rejected :Action : failedDiagnostic-Code: smtp; 550 5.2.0 Mail rejete. Mail rejected. ofr_506 [506]Status: 5.2.0OrDiagnostic-Code: smtp; 554 5.7.1 Command rejectedStatus: 5.7.1OrDiagnostic-Code: smtp; 550 4.4.7 Message expired: unable to deliver in 840 minutes.<421 4.4.2 Connection timed out>Status: 4.4.7This happens on multiple organizations we manage.I can't figure out why. This never happened before.Thanks for your help !AlexandreFollowComment"

Answer: "0Hi Alexandre,I'm sorry to hear you're experiencing with rejected emails. Can you tell me where you're experiencing this? Is in inbound mail or outbound mail.If its inbound email you can check your DMARC settings as WorkMail recurrently started to enforce this. https://docs.aws.amazon.com/workmail/latest/adminguide/inbound-dmarc.htmlKind regards,RobinCommentShareMODERATORrobinkawsanswered 4 years ago0Hi,No, this is for outbound mails.Thank you,CommentShareCalexoanswered 4 years ago0Hi Alexandre,In that case it seems that the problem might be on the receiving end. The error codes should explain why WorkMail was unable to deliver the email.Kind regards,RobinCommentShareMODERATORrobinkawsanswered 4 years ago0Whats is strange is that there is suddenly many errors.There's not much details.For example :Technical report:Reporting-MTA: dsn; a6-235.smtp-out.eu-west-1.amazonses.comAction: failedFinal-Recipient: rfc822; jm-nat.nagel@wanadoo.frDiagnostic-Code: smtp; 550 5.2.0 Mail rejete. Mail rejected. ofr_506 [506]Status: 5.2.0Thank you,Edited by: Calexo on Nov 27, 2019 8:06 AMCommentShareCalexoanswered 4 years ago0Hello,One more customer, on another Organization, reports me a outbound delivery issue.There is clearly an issue.Thanks,CommentShareCalexoanswered 4 years ago0Hi Alexandre,I'm sorry to inform you but in these cases WorkMail cannot fix the problem. Its the receiving end not accepting the message.I can ask the service team to check if the sending server is blacklisted but this is closely monitored so that should never happen. If you can PM me the sending users email address and destination I will ask the service team to take a look.Kind regards,RobinCommentShareMODERATORrobinkawsanswered 4 years ago0OK,I understand, thank you.I send you a PM with more detailed informations.AlexandreCommentShareCalexoanswered 4 years ago0receiving problemCommentShareCalexoanswered 4 years ago0I've got the same problem. During one week 3 different email addresses (including gmail.com) gave me:Action: failedFinal-Recipient: rfc822; XXXXXXXX@gmail.comDiagnostic-Code: smtp; 550 4.4.7 Message expired: unable to deliver in 10 minutes.<220 2.0.0 Ready to start TLS>Status: 4.4.7I sent these emails 2-3 times, but every time I received the same error. Now I don't know what I can do more, only change AWS Workmail to another service.CommentSharepavelsevanswered a year ago"


Question: Has the option to add a gateway to IoT Core disappeared?

"Currently trying to get started with IoT Core for my business, and all the documentation tells me how easy it is to add a LoRaWAN gateway via the Console, and how I can just click "Get Started" followed by "Add Gateway". However, there is simply no "Add Gateway" option. I have sifted through every single link there is on the sidebar - with and without the "New Console Experience" option checked - and every single piece of documentation that I can lay my hands on, to try and find where this option is. What can I possibly be missing?The closest thing that I can find - Add a Device - keeps asking whether my device is running Linux or Windows and which SDK I would like to download. Surely there is no need for all that just for the sake of adding a gateway that's in my hand as we speak?FollowComment"

Answer: "2Accepted AnswerHi. What region are you operating in? AWS IoT Core for LoRaWAN is not available in all regions. I suspect that's the problem. You should have LPWAN devices under Manage in your menu if you've selected a supported region. Supported regions:https://docs.aws.amazon.com/general/latest/gr/iot-core.html#iot-wireless_regionIf you are in a region that supports it, then when you click the link AWS IoT Core for LoRaWAN Intro page in the documentation, it will take you to the LPWAN devices menu section of the New console experience. And this has the Get started button leading to the Add gateway button.CommentShareEXPERTGreg_Banswered a year agorePost-User-3758034 a year agoThanks Greg, I'm surprised that that's the solution but it has now allowed me to add a gateway after switching to EU West 1 (Dublin) from EU West 2 (London)!Share"


Question: How fast can glue ETL convert data to parquet?

Do we have any benchmark number onhow fast glue ETL convert data to parquet?like 1 DPU can process 1GB raw data in X minutesI want to get a baseline number so I can get idea if the ETL job runs normal or has problem.also to estimate the DPUs I should use for my data conversion task.ThanksFollowComment

Answer: "0Accepted AnswerIt really depends on how your data is structured. If it's 1 GB file, then it's going to not benefit from Glue being able to fan out. If it's 1024 1MB files, then you're going to see the benefits. Also, it will depend on the block size of the Parquet to allow for optimal I/O (See tip #5 here https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/).I could only find some information on how to tune your DPUs optimally. The example given was 428 Gzipped JSON files converting to parquet files.https://docs.aws.amazon.com/glue/latest/dg/monitor-debug-capacity.htmlCommentShareAWS-User-9793820answered 4 years ago"


Question: Why don't I get en event with binlog information when I create a snapshot?

"I want to take a snapshot of an RDS (MySQL) database in order to start a replication slave. According to https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Replication.MySQL.html, I first need to take a snapshot, which should then generate an event showing the binlog position. I have taken the snapshot, but there's no event anywhere - is there something else I need to do - activate something, change an option, ...?FollowComment"

Answer: 0Accepted AnswerThe event will be emitted for the restored cluster not the source cluster.Once you restore that you should see the event on the new instance.-PhilCommentShareMODERATORphilawsanswered 2 years ago


Question: User expiration policy Last Login.

"Hello to the entire community, I need to create an IAM policy for my organization that reviews the last login and if more than 45 days have passed, the account is deactivated.Does anyone have an idea how to do it and help me.FollowComment"

Answer: "0Accepted AnswerHey there, I would look to write a lambda function that triggers on an eventbridge schedule that enumerates iam users where last login >= 45 days and status = active and disable them.I would use python myself and the boto3 library.You’ll have to create and grant lambda role IAM user modify accessHere’s a link to something that’s already been written. May save you a job implementing this.https://serverlessrepo.aws.amazon.com/applications/arn:aws:serverlessrepo:us-east-1:062052777684:applications~disabling-access-for-inactive-IAM-usersCommentShareGary Mcleananswered a month ago"


Question: ECS Fargate Log Configuration - How to set the retention policy?

"I have been using ECS Fargate to launch containers that are designed to execute some tasks and terminate once done. In the task definition, I have given awslogs as the logDriver and awslogs-create-group set to true so that each time it creates a log group in CloudWatch.But I notice in CW that the retention policy of the log groups is set to 'Never Expire', and over time there are many log groups that simply exist even though the cluster behind it are terminated.How can I ensure that the log groups are deleted as soon as the Fargate cluster/task is terminated?FollowComment"

Answer: "0Accepted AnswerHi, Good questionI think you can use put-retention-policy--log-group-name <value>--retention-in-days <value>[--cli-input-json <value>][--generate-cli-skeleton <value>]https://docs.aws.amazon.com/cli/latest/reference/logs/put-retention-policy.htmlCommentShareSrianswered a year agoM S Puranik a year agoThanks! So there is no way that I can set this policy from task definitions?ShareSri a year agoI think It can only be done through CloudWatch. Docker has the following options and I can't seem to find anything related to retentionmax-size - The maximum size of the log before it is rolled. A positive integer plus a modifier representing the unit of measure (k, m, or g). Defaults to 20m.max-file - The maximum number of log files that can be present. If rolling the logs creates excess files, the oldest file is removed. A positive integer. Defaults to 5.compress - Toggle compression of rotated log files. Enabled by default.https://docs.docker.com/config/containers/logging/local/Share0Actually, you can create LogGroup yourself and attach it to the Task Definition like here if you are using CloudFormation.Then in LogGroup, you can simply set retention and another benefit is that LogGroup will be removed together with CloudFormation stack.CommentShareMGanswered a year ago"


Question: ALB with Cloudfront

"Hi everyone,I set up Cloudfront - ALB - web server (Dotnet Nuke DNN CMS)Cloudfront got the domain of the front end, the ALB a domain for the communication between cloudfront and ALB, behind the ALB is the web server.The problem is now, the index page works fine, no errors. But the website now contains hyperlinks of the "internal" domain of the ALB that I use for the communication between ALB and Cloudfront. What we need is of course hyperlinks of the public domain facing to the client. Is it possible to tell the server from the ALB or Cloudfront that the domain is not the internal but the public domain? Maybe we have to set this on the webserver, but unfortunately I'm not a DNN developer, so I don't know what to tell the developer what to do.Thank you and bestFollowComment"

Answer: "2Accepted AnswerBrettski@AWS is right about the relative links as far as rendering HTML is concerned. But even if you convert all of those to relative links, you may face other problems. If your CMS sends emails with links (e.g. email confirmations), sets webhooks or participates in OAUTH2 flows for SSO, it's going to have to create absolute links back to itself. That means that it will need to know its correct hostname.There may be a setting in your CMS where you can specify this. But, if there isn't or you don't, it probably infers its hostname from the Host HTTP header that the browser sends in its request. The problem is that where the browser is setting Host: www.your-site.com, that is hitting Cloudfront which in turn queries the CMS. The query from Cloudfront will have something like Host: load-balancer.your-site.com so that's what the CMS infers as its host name.I haven't tried this myself yet but I think that this article describes the solution you need.CommentShareRitchieanswered a year agoPalleAle a year agoExactly what I was looking for, I needed to disable all caching features. Now the website looks good behind Cloudfront. Awesome, thanks a lot!Share0It sounds like the web server is producing "absolute" links to itself rather than "relative" links. This means that the raw HTML coming from the web server contains things like  rather than .The easiest thing to do (although it won't seem that way) is to change the web server to produce the relative links. You could also solve this by using some sort of proxy "in front" of the web server but that will introduce extra costs and additional layers that could fail.CommentShareEXPERTBrettski-AWSanswered a year ago"


Question: getting DNS_PROBE_FINISHED_NXDOMAIN with aws amplify and route 53

"getting DNS_PROBE_FINISHED_NXDOMAIN with AWS amplify and route 53 with an externally registered domain.I'm hosting my app on Amplify, I have gottten the cloudfront address and added it to route 53 in a newly created Hosted zone. I then added the new nameservers in my domain providers portal.When I visit the site i get the error : "DNS_PROBE_FINISHED_NXDOMAIN". I've tried clearning cookies, caches, resetting router - everything but it doesn't worki tested the records on route53 and it found no errors.Any help would be much appreciatedFollowComment"

Answer: "0How do you get the public IP for your site. If it is automatically then you have to change the IP for your A record. If you have a static IP release it and reassign it.Please try to apply the followming:You need to reset Chrome flags, disable VPN and antivirus, check local hosts file and your domain's DNS, and restart your computer.CommentSharerePost-User-7136343answered 3 months ago"