Question: Cloudfront can not serve s3 website via custom origin deployed via Serverless Framework

"Hi, I am setting up CI/CD pipeline for the frontend of an application developed in REACT, everything is working as expected.Build is builtbuild is being pushed to s3site can be visited via s3 static site urlWhat's not working is when the s3 site is set up via CloudFront as the custom origin. I'm attaching my code for creating a bucket and cloudfront distribution.Note: the custom domain works fine as well (it to redirects to the error that cloudfront distribution domain returns, i.e. 504 ERROR)Error message is in the end of the questionType: AWS::CloudFront::DistributionProperties:  DistributionConfig:    Aliases:      - ${param:deploymentUrl}    CustomErrorResponses:      - ErrorCode: 404        ResponseCode: 200        ResponsePagePath: /index.html        ErrorCachingMinTTL: 10    DefaultCacheBehavior:      AllowedMethods:        - GET        - HEAD      TargetOriginId: ${self:custom.cloudfrontOriginId}      ForwardedValues:        QueryString: 'false'        Cookies:          Forward: none      ViewerProtocolPolicy: redirect-to-https      CachePolicyId: 658327ea-f89d-4fab-a63d-7e88639e58f6 # CachingOptimized    DefaultRootObject: index.html    Enabled: 'true'    HttpVersion: 'http2'    Origins:      - DomainName: ${param:s3BucketName}.s3-website-${self:provider.region}.amazonaws.com        CustomOriginConfig:          HTTPPort: 80          HTTPSPort: 443          OriginProtocolPolicy: https-only        Id: ${self:custom.cloudfrontOriginId}    ViewerCertificate:      AcmCertificateArn: ${param:certificateArn}      MinimumProtocolVersion: 'TLSv1.2_2021'      SslSupportMethod: 'sni-only'Type: AWS::S3::BucketProperties:  BucketName: ${param:s3BucketName}  AccessControl: PublicRead  WebsiteConfiguration:    IndexDocument: index.html    ErrorDocument: index.htmlType: AWS::S3::BucketPolicyProperties:  Bucket:    Ref: Bucket  PolicyDocument:    Statement:      - Sid: PublicReadGetObject        Action:          - s3:GetObject        Effect: Allow        Principal: '*'        Resource: arn:aws:s3:::${param:s3BucketName}/*The request could not be satisfied.CloudFront attempted to establish a connection with the origin, but either the attempt failed or the origin closed the connection. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.Generated by cloudfront (CloudFront)Request ID: ...FollowComment"

Answer: "1The static website server provided by the S3 service for a bucket is http only, no support for https.  Your custom origin config says https-only, so it won't be able to contact the origin.The best way to do this is disable the bucket's website and public access and instead of using a custom origin, use an S3 origin secured by an Origin Access Identity.  That's a special CloudFront principal that you grant permissions to in your bucket policy.Then for best practice enable all the Block Public Access settings on the bucket.  Bonus points for doing that at the account level too and never making any of your buckets public!CommentShareEXPERTskinsmananswered 10 months ago"


Question: lustre client does not update properly on ubuntu 22

"Typically when I update my ubuntu servers I have to reinstall the lustre client per the documentation or the FSx lustre volume no longer mounts.I am using this documentation:https://docs.aws.amazon.com/fsx/latest/LustreGuide/install-lustre-client.htmlThis time, as expected, the mount failed after an OS update and I followed the doc above for ubuntu 22. When I try to mount the fsx volume, I get this error:mount.lustre: mount fs-xxxxxxxxxxx.fsx.us-east-2.amazonaws.com@tcp:/xxxxx at /mcb failed: No such deviceAre the lustre modules loaded?Check /etc/modprobe.conf and /proc/filesystemsThe filesystem is available and mounts to amazon linux servers. The fragility of the ubuntu lustre client is an ongoing issue for me.FollowComment"

Answer: "0Accepted AnswerHi, are you on Ubuntu 22 kernel version 5.19? FSx Lustre has not released support for version 5.19 yet.CommentSharerePost-User-9323546answered 24 days agoLinda 24 days agoThanks, is there a way to know when a new kernel is or is not supported? The AWS documentation: https://docs.aws.amazon.com/fsx/latest/LustreGuide/install-lustre-client.htmlgives no indication that 5.19 is not supported. It says: "The Lustre client on Ubuntu 22.04 requires kernel 5.15.0-1015-aws or later..."ShareAaron_B 23 days agoFor client kernel compatibility, you can reference the compatibility matrix at the end of the installation page:https://docs.aws.amazon.com/fsx/latest/LustreGuide/install-lustre-client.html#lustre-client-matrixIf your compute instance isn't running the Linux kernel specified in the installation instructions, and you can't change the kernel, you can build your own Lustre client. Additional details:https://docs.aws.amazon.com/fsx/latest/LustreGuide/install-lustre-client.htmlhttps://wiki.lustre.org/Compiling_LustreShare"


Question: Criteria for choosing which regions to use on a multi-region setup for improved reliability

"Based on the recent outage at us-east-1, where we have most of our AWS resources, the company has taken the path to start transitioning into a multi-region architecture, but picking which extra region(s) to use is not a matter of chance and the decision must be thoughtfully taken.Right now the first criteria to consider is cost as most US-based regions are cheaper for example than the one in Sao Paulo.Latency for our customers (distributed across LATAM and US) might be worthwhile considering, but right now everything is working from Virginia. For major disasters maybe replicating to a different country, or perhaps even continent could be the best for geographic isolation, but at a cost.How do you actually decide which regions to use? How to weigh different factors? Is there an actual guide for such?FollowComment"

Answer: "4There was a recent blog post on this:"What to Consider when Selecting a Region for your Workloads"https://aws.amazon.com/blogs/architecture/what-to-consider-when-selecting-a-region-for-your-workloads/Hope this will help you identify the criteria that you could use.CommentShareDennis_Aanswered a year ago1From the blog post above, you will likely mainly consider latency (some regions such as Stockholm are cheap to use but would significantly impact your US/SA userbase's experience) and cost as you mentioned.Cost though is something you can tune and optimise by carefully consider what you need to replicate (this will be based on your risk modelling). For example, if you go for active/active and plan to do old-fashioned backups, you can keep them in the cheapest region. Traffic flows can be optimised to avoid going too much "cross border" (ie make sure once an user's request enters into a region, it stays there).I know you haven't directly requested it, but if you're planning things now, I suggest to do it with this in mind: https://aws.amazon.com/builders-library/static-stability-using-availability-zones/ - most of Amazon/AWS are designed based on this principle and generally limiting your dependency on control planes or changes in general during a failure will make everything better.CommentShareGiorgio@AWSanswered a year ago"


Question: from the web application to how can I run the Python script in the Ubuntu terminal?

"Is there a way to run the python script terminal and get the output EC2 instance?As an example, I get two values from the web app as input. With these two values as arguments, a Python script runs on the terminal.Python3 add.py value1 value2What AWS services are required?Is there a way to get these values to be sent to a web application?What is the best way to run or trigger the Python script on the EC2 instance from the web app?It's challenging for me to figure this out.FollowComment"

Answer: "0If I understand your question correctly: You have a web application and you want to send two values to a Python script; have it perform some operation and return a result.While you could do this with EC2 I wouldn't recommend it - you would have a whole instance running all the time even when your web application isn't requesting any results.Instead, consider using Lambda. Create a Lambda function using Python then publish that function using Lambda function URLs - you pass the two values to the URL and it executes the Lambda function, returning the appropriate value.CommentShareEXPERTBrettski-AWSanswered a year agosakthivel a year agoBrettski, I know it's not recommended for simple tasks, but the actual application is image processing.The S3 bucket folder name is the argument value. It takes a minimum of two to three hours to complete.I have tested the script manually and it works well. I want to automate it on the EC2 instanceThanksShareBrettski-AWS EXPERTa year agoThanks for expanding on the question. In that case, I would use SQS: Send the job details to a SQS queue; have EC2 instances scale based on the length of the queue (that way you can run zero instances when there are no jobs; and you can scale to more instances if necessary); then your script on the EC2 instances reads the job details from the SQS queue. Don't forget to set the message visibility timeout to several hours. Note that you can create the SQS message as a trigger from S3 when the object is uploaded.Share"


Question: AppSync did't return Highlight response with opensearch resolver

"Hi have the following schema,example:type Demo { id: ID! code: String name: String         title: String highlight: String }and my query resolver template looks like this,{     "version":"2017-02-28",     "operation":"GET",     "path":"/index/_search",     "params":{         "headers":{},         "queryString":{},         "body":{         "query": {           "bool": {             "filter": {               "term": {                 "code": $util.toJson($context.arguments.code)               }             },             "must": [               {                 "bool": {                   "should": [                     {                       "match_phrase_prefix": {                         "title": {                           "query": $util.toJson($context.arguments.title)                         }                       }                     },                     {                       "match_phrase_prefix": {                         "name": {                           "query": $util.toJson($context.arguments.name)                         }                       }                     }                   ]                 }               }             ]           }         },         "highlight": {           "fields": {             "name": {}           }         }         }     } }all results are getting fine except highlightDoes anyone have any idea about this? I want to get highlighting responses in the same query.FollowComment"


Question: AWS CloudWatch Log Insights Generate URL

"Is it possible to deterministically generate a URL for CloudWatch Log Insights given the log group, start time, and query (text area that allows fields, sorts, filters, etc...)? The URL is extremely long and does not have standard URL encoding from what I can tell.There are plenty of suggestions in the below Stack Overflow post about the topic but I was curious if there was a more official guide to doing this?https://stackoverflow.com/questions/60796991/is-there-a-way-to-generate-the-aws-console-urls-for-cloudwatch-log-group-filtersFollowComment"

Answer: "2Accepted AnswerIn addition to the suggestions in the Stack Overflow post, here is some further info on how the Logs Insights URLs are generated that may help you.The deep link of Logs Insights query is generated in three steps:Step 1. Define the queryDetails in a JSON structure {        editorString: '',        end: '',        queryId: '',        source: '',        start: '',        timeType: '',        tz: '',        unit: ''    };Step 2. It uses jsurl (https://www.npmjs.com/package/jsurl) to encode the JSON object logs-insights?queryDetail=<encoded js url>Step 3. It uses following map to replace the special characters inside logs-insights?queryDetail=<encoded js url> {  '!' = '$21',  '#' = '$23',  '%' = '$25',  '&' = '$26',  // Spaces are encoded as +. That's what GWT uses when we do not manually  //   encode our routes, so it is backwards compatible with customer links  //   generated prior to this manual encoding process.  ' ' = '$2B',  ':' = '$3A',  ';' = '$3B',  '=' = '$3D',  '?' = '$3F',}Step 4. add prefix https://console.aws.amazon.com/cloudwatch/home?region=<region>#logsV2:To get more details, you can use some example URL and reverse the steps and see what the original JSON structure looks like.CommentShareSUPPORT ENGINEERJustin_Banswered a year ago"


Question: AWS WorkSpaces Windows Bundle choice

"Hello,  I'm looking to deploy WorkSpaces for my company.  We primarily have two types of users, average office type and developers.   The developers use Visual Studio, eclipse and or IntelliJ applications.What would be the lowest recommended bundle type for these two user types?TIA---KennyFollowComment"

Answer: "0Hello Kenny,I have had a look at the Visual Studio 2019 System Requirements [1] and I believe that the best option for your developers is the "Performance" type while for the other users the "Value" or "Standard" type [2].Having said that, if you are going to use the Windows bundle my suggestion is to start with testing the "Standard" type and increase or decrease the Compute Type later as per your needs [3].Thank you,VasosAWSReferences:[1] https://docs.microsoft.com/en-us/visualstudio/releases/2019/system-requirements[2] https://aws.amazon.com/workspaces/pricing/[3] https://docs.aws.amazon.com/workspaces/latest/adminguide/modify-workspaces.htmlCommentShareAWS-User-5546426answered 3 years ago0VasosAWS,  Thanks for the reply.   Will start testing.--KennyCommentShareBESCanswered 3 years ago"


Question: Deploying ECS cluster + task with Javascript SDK

"I'm trying to deploy an ECS Cluster with a task to run a docker container with a web application. I found the methods to create the cluster, registering the task definition and running it, but I can't find how to choose the instance type that I want to be used and also the disk size.In this post ( https://dev.to/raphaelmansuy/deploy-a-docker-app-to-aws-using-ecs-3i1g ) the person is making a deploy from the AWS Console and he is able to choose the instance type and disk type and size.Is it possible to do the same using the Javascript SDK ( https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/ECS.html ). I can't find where.If it's not possible, do you suggest me some other strategy to make this work? Basically what I need is to create the whole stack triggered from an admin page. May be I trigger the process from a lambda function or directly from the back-end of my application.Thanks.FollowCommentcbevilaqua a year agoAs I can understand now, this option does really does not exist and we must change the memory and cpu requirements for our application, doesn't matter the EC2 instance type, please correct me if I'm wrong. Thanks!Share"

Answer: "0Accepted AnswerHi,About the post screenshots - If you look at the main ECS cluster creation screen, when you choose EC2, it says it will create for your an Autoscaling Group of EC2 instances, then, at the Task creation step, you'll be choosing the EC2 provider to deploy the tasks on the EC2 AutoScaling Group.If you have to reproduce the same setup from the JS SDK, first you need to create an AutoScaling Group from the EC2 APIThen you create a Capacity Provider from the ECS API and reference the Autoscaling Group ARN previously created.CommentShareJulienanswered a year agocbevilaqua a year agothank you so muchShare"


Question: AWS GLUE - java.sql.SQLIntegrityConstraintViolationException: Duplicate entry Error

"I was doing a POC using Glue to Migrate data from RDS MySql to RDS Postgres. I have created Connectors to both source and target, and a crawler which connected to source. Then  created a job and tried to migrate data with out any transformation and started getting java.sql.SQLIntegrityConstraintViolationException: Duplicate entry errorjava.lang.reflect.Constructor.newInstance(Constructor.java:423)\n\tat com.mysql.cj.util.Util.handleNewInstance(Util.java:192)\n\tat com.mysql.cj.util.Util.getInstance(Util.java:167)\n\tat com.mysql.cj.util.Util.getInstance(Util.java:174)\n\tat com.mysql.cj.jdbc.exceptions.SQLError.createBatchUpdateException(SQLError.java:224)\n\tat com.mysql.cj.jdbc.ServerPreparedStatement.executeBatchSerially(ServerPreparedStatement.java:385)\n\tat com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchInternal(ClientPreparedStatement.java:435)\n\tat com.mysql.cj.jdbc.StatementImpl.executeBatch(StatementImpl.java:794)\n\tat org.apache.spark.sql.jdbc.glue.GlueJDBCSink$.saveBatch(GlueJDBCSink.scala:400)\n\tat org.apache.spark.sql.jdbc.glue.GlueJDBCSink$.$anonfun$saveTable$4(GlueJDBCSink.scala:77)\n\tat org.apache.spark.sql.jdbc.glue.GlueJDBCSink$.$anonfun$saveTable$4$adapted(GlueJDBCSink.scala:77)\n\tat org.apache.spark.sql.jdbc.glue.GlueJDBCSink$.savePartition(GlueJDBCSink.scala:261)\n\tat org.apache.spark.sql.jdbc.glue.GlueJDBCSink$.$anonfun$saveTable$3(GlueJDBCSink.scala:77)\n\tat org.apache.spark.sql.jdbc.glue.GlueJDBCSink$.$anonfun$saveTable$3$adapted(GlueJDBCSink.scala:76)\n\tat org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2(RDD.scala:1020)\n\tat org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2$adapted(RDD.scala:1020)\n\tat org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2278)\n\tat org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)\n\tat org.apache.spark.scheduler.Task.run(Task.scala:131)\n\tat org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)\n\tat org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)\n\tat org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat ****java.lang.Thread.run(Thread.java:750)\nCaused by: java.**sql.SQLIntegrityConstraintViolationException: Duplicate entry '00002b0b-1a34-3319-b003-fb073fb8248d' for key 'transformer_fragment.PRIMARY'\n\tat ******com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:117)\n\tat com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)\n\tat com.mysql.cj.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:637)\n\tat com.mysql.cj.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:418)\n\tat com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1092)\n\tat com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1040)\n\tat com.mysql.cj.jdbc.ServerPreparedStatement.executeBatchSerially(ServerPreparedStatement.java:357)\n\t... 19 more\n","Accumulator Updates":[{"ID":130,"Update":"105","Internal":false,"Count Failed Values":true},{"ID":132,"Update":"0","Internal":false,"Count Failed Values":true},{"ID":139,"Update":"0","Internal":false,"Count Failed Values":true},{"ID":140,"Update":"1","Internal":false,"Count Failed Values":true},{"ID":141,"Update":"0","Internal":false,"Count Failed Values":true},{"ID":142,"Update":"0","Internal":false,"Count Failed Values":true},{"ID":143,"Update":"1135","Internal":false,"Count Failed Values":true},{"ID":144,"Update":"0","Internal":false,"Count Failed Values":true},{"ID":145,"Update":"1","Internal":false,"Count Failed Values":true}]},"Task Info":{"Task ID":13,"Index":5,"Attempt":1,"Launch Time":1655403050036,"Executor ID":"1","Host":"172.31.22.88","Locality":"NODE_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1655403050162,"Failed":true,"Killed":false,"Accumulables":The following are the values I have used while creating the job,Type : SparkGlue Version -  Glue 3.0 Supports spark3.1, Scala2, Python3Language - Python3Worker Type - G.1XAutomatically Scale the number of workers - FalseRequested number of workers - 2Generate Job Insights - TrueJob Bookmark - EnableNumber of retries - 1job timeout - 2880Really appreciate any help !FollowComment"


Question: Sent Notification when EBS volume snapshot permission changed to public

"Hi Team,Could you please help me to send an alert when EBS volume snapshot permission changed to public.We don't want others to create volume from the snapshot.FollowComment"

Answer: 3You can use AWS Config to detect non-compliant and notify via Amazon SNS. Using "ebs-snapshot-public-restorable-check" managed rules seems to fit your purpose.ebs-snapshot-public-restorable-check - AWS ConfigHere is an example of a security group.Receive notifications for a non-compliant AWS resource using AWS ConfigCommentSharejhashimotoanswered 5 months agoEXPERTkentradreviewed 5 months agorePost-User-7169480 4 months agoThanks @jhashimoto and kentrad for quick answerSharerePost-User-7169480 3 months ago@All with below configuration for existing non-complaint resources not getting alert. But if the new resource is created and it's compliance changed to not-compliance then only receiving the  alert. Is there a way to get alert for exiting non-complaint resources under config ruleReceive notifications for a non-compliant AWS resource using AWS ConfigShare


Question: Can Data Pipelines be used for running Spark jobs on EMR 6.5.0?

"Hi,I have a problem in that I make heavy use of EMRs, and I orchestrate their use with Data Pipeline - multiple daily runs are automated and EMRs are launched and terminated on conclusion.However, I'd now like to make use of EMR 6.X.X releases via Data Pipelines, rather than EMR 5.X.X releases I'm currently using.  This is for two main reasons:Security compliance: the latest EMR 6.X.X release have less vulnerabilities than the latest EMR 5.X.X releasesPerformance/functionality: EMR 6.X.X releases perform much better than EMR 5.X.X releases for what I'm doing, and have functionality I prefer to useHowever...the current documentation for Data Pipeline says the following regarding EMR versions:AWS Data Pipeline only supports release version 6.1.0 (emr-6.1.0).Version 6.1.0 of EMR was last updated on Oct 15, 2020...it's pretty old.Now, if I try and use an EMR version > 6.1.0 with Data Pipeline, I get the issue that has already been raised here i.e. during initial EMR cluster bring up via the Data Pipeline, there's a failure that renders the cluster unusable.  It looks like a malformed attempt to create a symbolic link to a jar by one of the AWS scripts:++ find /usr/lib/hive/lib/ -name 'opencsv*jar'+ open_csv_jar='/usr/lib/hive/lib/opencsv-2.3.jar/usr/lib/hive/lib/opencsv-3.9.jar'+ sudo ln -s /usr/share/aws/emr/goodies/lib/emr-hadoop-goodies-3.2.0.jar /mnt/taskRunner/oncluster-emr-hadoop- goodies.jar+ sudo ln -s /usr/share/aws/emr/goodies/lib/emr-hive-goodies-3.2.0.jar /mnt/taskRunner/oncluster-emr-hive-goodies.jar+ sudo ln -s /usr/lib/hive/lib/opencsv-2.3.jar /usr/lib/hive/lib/opencsv-3.9.jar /mnt/taskRunner/open-csv.jarln: target â€˜/mnt/taskRunner/open-csv.jarâ€™ is not a directoryCommand exiting with ret '1'So - I guess my questions are:Is there a way to work around the above so that Data Pipelines can be used to launch EMR 6.5.0 clusters for Spark jobs?If there isn't, is there a different way of automating runs of EMR 6.5.0 clusters, other than writing my own script and scheduling that to bring up the EMR cluster and add the required jobs/steps?Thanks.FollowComment"

Answer: "0With this banner in the console, "Please note that Data Pipeline service is in maintenance mode and we are not planning to expand the service to new regions. We plan to remove console access by 05/12/2023", I would focus development of new workloads in Managed Apache Airflow. Using this operator, you can kick off a dag that launches an EMR cluster and terminates it one job is complete, similar to what we used to do in data pipeline.CommentShareEmananswered a month ago"


Question: VPN tunnel switching setting

"Local firewall and VPC set up VPN,the ip of tunnel 1 is first prority and ip of tunnel 2 is second prority in the Local firewallWhen there is a problem with the ip of tunnel 1,AWS will failover to the tunnel 2 to keep communicate with local firewall.But when tunnel 1 is restored, AWS will not switch back to tunnel 1 to communicate with the firewall,but will continue to remain in tunnel 2.How to set up when tunnel 1 is restored, aws can also switch back to tunnel 1 tocommunicate with the local firewallFollowCommentTushar_J EXPERT8 months agoAre you using Static route based VPN or BGP based Dynamic VPN?Share"

Answer: "0Accepted AnswerHi,Simply put, the VPN tunnel is randomly chosen by AWS and is called the preferred tunnel. If your AWS VPN connection (static route type) has an active/active configuration (both tunnels are up), you cannot configure your preferred specific tunnel in AWS to send traffic.But, for dynamic AWS VPN connections (Active/Active), you can set the customer gateway device to prefer one VPN tunnel over the other by leveraging the order of preference criteria:Advertising more specific prefix over preferred tunnelAdvertising shorter ASPATH over the preferred tunnelSetting lower MED values over the preferred tunnelHope this helpsCommentSharejcvipanswered 8 months ago0Hello,See this Knowledge center article:https://aws.amazon.com/premiumsupport/knowledge-center/vpn-configure-tunnel-preference/Note also below from our documentationWe recommend that you use BGP-capable devices, when available, because the BGP protocol offers robust liveness detection checks that can assist failover to the second VPN tunnel if the first tunnel goes down. Devices that don't support BGP may also perform health checks to assist failover to the second tunnel when needed.CommentShareEXPERTTushar_Janswered 8 months ago"


Question: S3 lifecycle rule / ExpiredObjectDeleteMarker': True failed on chalice application

"HiInside a chalice application, I have created a class for handling S3 bucket. I add a method to add a default lifecycle rule which will delete all objects after 1 day. (quite useful for a dev environment)def put_default_lifecycle_rule(self, id_name='default_lifecycle_rule'):   try:        s3 = boto3.resource('s3')        bucket_lifecycle_configuration = s3.BucketLifecycleConfiguration(self._bucketname)        response = bucket_lifecycle_configuration.put(            LifecycleConfiguration={                'Rules': [                    {                        'Expiration': {                            'Days': 1                        },                        'ID': id_name,                        'Status': 'Enabled',                        'Filter': {'Prefix': ''},                        'NoncurrentVersionExpiration': {                            'NoncurrentDays': 1                        },                        'AbortIncompleteMultipartUpload': {                            'DaysAfterInitiation': 1                        }                    }                ]            }        )    except Exception as e:        logging.error(e)        raise e    return responseWhen I add "ExpiredObjectDeleteMarker': True" in expiration block, I got an XML error (malformed).Official documentation says to put this in this block.Any idea ?Thanks guysFollowComment"

Answer: "1Accepted AnswerHi there!You cannot add the ExpiredObjectDeleteMarker configuration within the same expiration rule where you specify a Days or Date value.Try having two separate rules for the LifeCycleConfiguration. Something like this:response = bucket_lifecycle_configuration.put(            LifecycleConfiguration={                'Rules': [                    {                        'Expiration': {                            'Days': 1                        },                        'ID': 'Rule01',                        'Status': 'Enabled',                        'Filter': {'Prefix': ''},                        'NoncurrentVersionExpiration': {                            'NoncurrentDays': 1                        },                        'AbortIncompleteMultipartUpload': {                            'DaysAfterInitiation': 1                        }                    },                    {                        'Expiration': {                            'ExpiredObjectDeleteMarker': True                        },                        'ID': 'Rule02',                        'Status': 'Enabled',                        'Filter': {'Prefix': ''}                    }                ]            }        )CommentShareFabio_Lanswered a year agoJack-Angel-369 a year agoHi )))Yes I should have thought about that.This is the solutionThanks FabioShare0The documentation for S3.BucketLifecycleConfiguration.put() states:ExpiredObjectDeleteMarker (boolean) --Indicates whether Amazon S3 will remove a delete marker with no noncurrent versions. If set to true, the delete marker will be expired; if set to false the policy takes no action. This cannot be specified with Days or Date in a Lifecycle Expiration Policy.-- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.BucketLifecycleConfiguration.put(emphasis mine)Specifying  'Expiration': { 'Days': 1  } causes current versions to expire after one day.The 'NoncurrentVersionExpiration': {                             'NoncurrentDays': 1                         }block specifies that non-current versions are deleted one day after Expiration.CommentShareChandler Howellanswered a year agoJack-Angel-369 a year agoHi thanks for the replyYes sure this is the code I wrote. The question is about the expired object delete marker)))I have read the doc before writing the code.With the AWS console I can set up the lifecycle rule for expired object delete markers but not on code...When some buckets can't be delete with the console, the only solution is to put a lifecycle rule for deletingcurrent versionsnon current versionsexpired object delete markersaccording to AWS documentation...So the issue remainsThanks for the helpShare"


Question: TLS certificate still pending even though domain and nameservers correct in route 53

"Hello,I am trying to obtain a valid tls certificate through ACM, however, the certificate is still pending. I registered my domain (let's use example.com as the example) in Route 53, and the 4 name servers under route 53 > registered domains > example.com > hosted zone details matches the 4 name servers in my hosted zone under route 53 > hosted zone > example.com - under the NS record. Also, I have 3 other records under r 53 > hosted zone > example.com, them being an A record which routes to my s3 bucket, an SOA record which routes to the first name server of the NS record, and a CNAME record with 'record name' of 'www.example.com' and routing to 'example.com'.I transferred this domain from another host a month ago, and aws support told me yesterday that the name servers hadn't transferred over correctly, so yesterday I fixed the name servers in route 53 > registered domains > example.com > hosted zone details to the name servers that were within r 53 > hosted zone > example.com - under the NS record. Also, I don't think it took 48 hours for the dns settings to update because they are already updated under route 53 > registered domains > example.com > hosted zone details.So domain and name servers are all correct, and the aws documentation here says that if all checks out it should take 30 min max to issue a valid certificate. However, it has been almost 24 hours and the certificate for example.com is still pendingFollowComment"

Answer: "1Accepted AnswerYou need to make sure the CNAME records have been created in order for ACM to validate the Domain. Within the ACM certificate, it will define the CNAME record thta needs creating.You have not mentioned you have created this CNAME record!Also make sure the domain registrar records point to the name servers for this domain.CommentShareGary Mcleananswered a month agoJeff F a month agoOh ok I see. I manually added the CNAME records to r 53 > hosted zones > example.com yesterday and now it works!Question though - Do I need only one hosted zone for example.com, or do I need a hosted zone both for example.com and www.example.com ? I'm asking because within the hosted zone r 53 > hosted zones > example.com I now have 3 CNAME records :'record name' is www.example.com with 'value/route to' being example.comrandomcharacters1.example.com routing to randomcharacters2.tftwdmzmwn.acm-validation.aws.randomcharacters3.www.example.com routing to randomcharacters4.tftwdmzmwn.acm-validation.aws.In the hosted zone for r 53 > hosted zones > www.example.com I only have one CNAME record, and I think this CNAME record auto-populated somehow. It just happens to be one of the CNAME records from example.com:randomcharacters3.www.example.com routing to randomcharacters4.tftwdmzmwn.acm-validation.aws.Share"


Question: Failure in Cloudformation template [ CommandRunenr] while running CLI command for Cloudtrail

"Hi Guys,I am trying to run CLI command to update a CloudTrail  but stack is getting failed. Requirement is to apply advanced data events to existing CloudTrail.Please find below details of CF template:CF templateAWSTemplateFormatVersion: 2010-09-09Resources:UpdateTrail:Type: AWSUtility::CloudFormation::CommandRunnerProperties:Role: ec2-role-nameSubnetId: subnet-XXXXXXXXXLogGroup: log-group-nameCommand: aws cloudtrail put-event-selectors --trail-name XXXX --region XXXX --advanced-event-selectors....ErrorResource handler returned message: "Either the command failed to execute, the value written to /command-output.txt was invalid or the Subnet specified did not have internet access. The value written to /command-output.txt must be a non-empty single word value without quotation marks. Check cloud-init.log in the LogGroup specified for more information."CLI commandaws cloudtrail put-event-selectors --trail-name XXXX --region XXXX--advanced-event-selectors'[{"Name": "S3EventSelector","FieldSelectors": [{ "Field": "eventCategory", "Equals": ["Data"] },{ "Field": "resources.type", "Equals": ["AWS::S3::Object"] },{ "Field": "eventName", "Equals": ["PutObject","DeleteObject"] },{ "Field": "resources.ARN", "StartsWith": ["arn:aws:s3:::XX","arn:aws:s3:::XX"] }]}]'Note : Command runs successfully in CLI.pre-requisites for commandRunner is installed.Also,  Subnet specified does have internet access.I sense, it might be the issue with command format or may be something else.Any assistance would be appreciated.ThanksFollowComment"

Answer: "0Hello, as indicated here, you must write the command's output to a reserved file called /command-output.txt. Now, per the code, the above error can occur if SSM PutParameter failed due to invalid value in /command-output.txt.I tested the template in my lab and noticed PutParameter event in CloudTrail failed with the following error:"1 validation error detected: Value at 'value' failed to satisfy constraint: Member must have length greater than or equal to 1.Then, observed the following error in a log named i-****/cloud-init-output.log in the CloudWatch log group that I configured using the LogGroup property in AWSUtility::CloudFormation::CommandRunner resource type.aws: error: argument --event-selectors is requiredContents of /command-output.txt =As seen above, contents of the /command-output.txt is empty.This --event-selectors argument error occurs if the session uses AWS CLI v1 and is not seen in AWS CLI v2. Observed the Amazon Linux AMI used by CommandRunner does not have the latest version of the AWS CLI installed at present, therefore we will need to update the AWS CLI on the instance to successfully execute the above put-event-selectors command. I used the following snippet to fix the issue:....Resources:  CommandRunner:    Type: 'AWSUtility::CloudFormation::CommandRunner'    Properties:      Command: >-        curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip";        unzip awscliv2.zip;        sudo ./aws/install;        aws --version;        aws cloudtrail put-event-selectors --trail-name XXX --region XXXX --advanced-event-selectors '[ { "Name": "S3EventSelector", "FieldSelectors": [ { "Field": "eventCategory", "Equals": ["Data"] }, { "Field": "resources.type", "Equals": ["AWS::S3::Object"] }, { "Field": "eventName", "Equals": ["PutObject","DeleteObject"] }, { "Field": "resources.ARN", "StartsWith": ["arn:aws:s3:::XX","arn:aws:s3:::XXX"] } ] } ]' --output text > /command-output.txt      Role: yourrole      LogGroup: log-group-nameHope it helps!CommentShareSUPPORT ENGINEERSreshtha_Kanswered 6 months ago"


Question: A certificate for this thing could not download

"Hi, I am a new user for IOT.When I try to built my first thing, I can create the certificate. However, I could not download the "A certificate for this thing", because it was blocked by Windows.message: xxxxxx-certificate.pem.crt was blocked because this type of file can harm your device.I try to disable the Windows Security but without improvement.Please help to teach me.Thank you.GordonFollowComment"

Answer: "0Hi,It could be operating system specific error. You may be able to choose either Save anyway or Discard.  If that is not the case, use aws CLI to generate certificate and attach to thing.Hope this helps,CommentShareAWS-User-3247576answered 2 years ago0Hi,The other method could not work. I will try to use aws CLI to generate the cert.Thank you.GordonCommentSharehorseHKanswered 2 years ago"


Question: Custom Entity Recognition Job Not Working with .txt files

"I am trying to run a custom entity recognition job on a couple text files. I trained the recognizer using PDF annotation, but I am sending .txt files into the job.So, I am getting this error: DOCUMENT_CORPUS_SIZE_LESS_THAN_MINIMUM: Document corpus size is less than the minimum requirement: 5120 bytes.It is true, the text files in my input folder are only 1.2 KB, but I am not sure how to proceed from here. I tried changing the options to "ONE_DOC_PER_LINE", but that gave another error saying that is unsupported for semi-structured data.UPDATE: I added extra text to the file, and it went through, but the entity recognition is very poor. Before this, I was submitting PDFs to the analysis job, and it was working well. However, I need to submit text documents because I want to eventually create an endpoint to use for this job. What do I do?FollowComment"

Answer: 0This was a bug in our validation logic which did not allow for a total input corpus size of less than 5kb for a semi-structured trained model. This issue is being addressed and should be deployed worldwide by 07/29.CommentSharednlenAWSanswered 10 months ago


Question: Is this the right way to backup my ec2 instance? (+some questions)

"Hi, I am an AWS EC2 free-tier user and I would like to know if the way I backup is a correct or not.So, the way I currently backup isMake a snapshot from an ebs volumeMake an AMI from the snapshotMake a new instance from that AMITerminate the pervious instance and detach the volume from the instance as wellAttach the new instance with a new volume to my domain(www.blahblah)Change my domain public ip to the new instance public ip.I have done my backup this way but I am not sure if the way I do it is correct or not. Please let me know if there is anything that I am missing.+I have some few more questions other than the procedure of the ebs snapshot backup. My questions areHow to check how much volume is left in the current ebs storage? (I want to check for the free space in my ebs. I have heard that we need to size up the ebs volume when there are more files and more snapshots in the volume. 'df -hT /dev/xvda1' command didn't show the ebs volume that are free.)Do I get a notification when my ebs storage is almost full?Do I need to stop my instance when I am making a snapshot? (For the current backup, I didn't stop my instance when I took a snapshot and It was fine. but on the internet, people seem to say it is better to stop the instance before taking a snapshot. so I'm confused.)Is using S3 mandatory to store ebs snapshots? (I'm currently not using S3 but I have seen people storing ebs snapshots in their S3 storage. so I'm not sure if If I need to store them in S3.)Thanks!FollowComment"

Answer: "0Take a look at this blog for how to automate backing up your EBS volumes: Automating Amazon EBS snapshot management using Data Lifecycle Manager.You don't need to create the AMI and boot up a new instance unless your are trying to restore the volume (steps 2-6).You have to use the OS/Filesystem commands to determine usage, like 'df'.No, not by default. You can install the CloudWatch agent to monitor OS level resources and get alerts, see: Collect metrics and logs from Amazon EC2 instances and on-premises servers with the CloudWatch agentNo, no need to stopS3 is used by the EBS service, you can't control this or see the bucket where your snapshots are stored.CommentShareEXPERTkentradanswered 5 months agorePost-User-9594951 5 months agoHi, Thank you for the quick response.I am a bit confused by "You don't need to create the AMI and boot up a new instance unless your are trying to restore the volume (steps 2-6).".If I don't create an AMI from the snapshot, how do I backup my current instance?In snapshot action, we havecreate volumecreate imagemanage fast snapshot restore etcIf I don't create an image(AMI) to backup, should I create a volume from that snapshot and attach it to the current instance? Or should I do the fast snapshot restore? (but I have heard that fast snapshot restore is expensive though...)Thanks!SharerePost-User-9594951 5 months agoI got it, I created a volume from the snapshot and it worked out fine.Thanks!!Share0Adding some information to question (3):Do you need to stop your instance before doing a snapshot? No, you don't. However the snapshot process is not aware of the state that the filesystem on the disks is in. Nor is it aware of the state that any application data might be in.For example, if the operating system is in the middle of writing a file but has not yet updated the file allocation table (or whatever the equivalent is for that particular operating system) and you take a snapshot right then the filesystem in the snapshot is in a state which isn't (for want of a better word) "stable". That means when you restore from snapshot the new volume will be in the same state. Most operating systems will recover from this gracefully but you might lose the contents of the file. Maybe. It depends.If you're running a database on the instance and the database is in the middle of committing a transaction and you take a snapshot at that exact moment then the database will be in an unknown state in the snapshot. Same thing applies - the database engine might automatically fix that problem on the next startup (on the new volume); or it might not. And the data might be lost. Or not. Again, it depends.So it is safer to stop your instance to do a snapshot? Yes. If the filesystem is relatively stable but you're running a database then you might want to stop write operations to the database while you're doing the snapshot - that could be done at an application level or within the database. That way you know that the database is fully written to disk before the snapshot is taken.CommentShareEXPERTBrettski-AWSanswered 5 months ago"


Question: Lumberyard and Steam

"Hey,How does it work for if i make a game and decide to host the game on steam? When i host, it on there, they host the Multiplayer servers. Would that be allowed? How would it work with how Lumberyard is free, but you use aws to host the servers.Follow"

Answer: "0According to Lumberyard's FAQ:---Quote----Q. Can my Lumberyard game connect to services like Steamworks, Xbox Live, PSN, Apple Game Center, Google Play Games, or console social services?A. Yes. Your game may read and write data to platform services and public third-party game services for player save state, identity, social graph, matchmaking, chat, notifications, achievements, leaderboards, advertising, player acquisition, in-game purchasing, analytics, and crash reporting.Q. Can my game use an alternate web service instead of AWS?A. No. By “alternate web service” we mean any non-AWS web service that is similar to or can act as a replacement for Amazon EC2, Amazon Lambda, Amazon DynamoDB, Amazon RDS, Amazon S3, Amazon EBS, Amazon EC2 Container Service, or Amazon GameLift. You can use hardware you own and operate for your game servers.Q. Is it okay for me to use my own servers?A. Yes. You can use hardware you own and operate for your game. It’s also ok for your users to connect peer-to-peer, and for users to host personal servers (as long as they’re not operating as a service provider business).---End Quote----So you can connect to Steamworks, but you can't use Steam as host service for your multiplayer game, it is not allow because it's operating as a service provider business.Hope I understand your question correctly.SharerePost-User-6460635answered 5 years ago0The steam part is an issue, as your forced to pay for the servers while hosing your game on there, so if you are putting your game on there, then you would end up paying for aws and steam. Steam its not like its optional as its part of the take they take and its already 30-40% they take. Steam can not be defined by an alternaitive web service as its just the multi player servers and you cant use them unless you have a game sold on there. I would like to get input from a LY Staff on this rather than a quote....SharerePost-User-1354821answered 5 years ago0Hi @REDACTEDUSERSharerePost-User-0479726answered 5 years ago"


Question: unable to start cloudshell

"Unable to start the environment. To retry, refresh the browser or restart by selecting Actions, Restart AWS CloudShell. I am getting that error even after following the instruction and I am using ohio region. I alread tried using other regions with cloudshell like tokyo but still facing this issue. I am also using my root account and still the same issue.FollowCommentAWS-User-2148182 a year agoI have the same problem.ShareMaulik_Makadiya a year agoAnother thread already running with the same problemhttps://repost.aws/questions/QUufc-elFvSvqYlbJw95RI6Q/unable-to-start-the-environment-to-retry-refresh-the-browser-or-restart-by-selecting-actions-restart-aws-cloud-shellSharerePost-User-5406173 3 months agoI can't use cloudshell, same problem.Share"

Answer: "0Make sure there are no Service Control Policies blocking the account from using CloudShell. The policy simulator may help narrow down what is blocking if it is a permissions issue. https://policysim.aws.amazon.com/CommentShareEXPERTkentradanswered a year agoAWS-User-2148182 a year agoPolicy simulator shows I should have access to CloudShell but I don't have.Share0The two regions you mentioned are supported refer to AWS CloudShell endpoints and quotas. If this was a permission error I would expect to see a different error. To be sure check that your current permissions includes necessary permissions, for example, AWSCloudShellFullAccess.This could be browser related. CloudShell supports below so check that your browser meets these requirementsGoogle ChromeLatest three major versionsMozilla FirefoxLatest three major versionsMicrosoft EdgeLatest three major versionsApple Safari for macOSLatest two major versionsCommentShareRoBanswered a year ago0I'm having the same issue even though I'm using Root User and I don't think it's a browser problem because another root user can be used normally. I tried to simulate and the permission column said allowed both CloudShell Create & StartEnvironment. How should I do?Thank you very much.CommentSharePlAwAnSaIanswered a year ago-1So i was able to resolve this issue with the help of AWS repost user @Maulik_Makadiya.Few things to try to create CloudShell environment:Time Synchronization: Make sure Your machine time is accurate. It means its correct based on world time. did you try from another machine to see if its working there? may be any time sync related issue?Check in different regions..check AWSCloudShellFullAccess policy to ensure it has below JSON data.{    "Version": "2012-10-17",    "Statement": [        {            "Action": [                "cloudshell:*"            ],            "Effect": "Allow",            "Resource": "*"        }    ]}Check in different browser to see if it works in different browser or not. https://docs.aws.amazon.com/cloudshell/latest/userguide/troubleshooting.htmlDid you deleted cloudshell home directory or something? Try after resetting home directory. But it can DELETE all your data exists in home directory. https://docs.aws.amazon.com/cloudshell/latest/userguide/vm-specs.html#deleting-home-directorycheck if any DENY policy created for it. remove that.It is possible your account is not 100% verified. Try this: Create a CloudFront distribution if you get the below error it confirms your account is unverified or if you can create 2 distributions and can't create the 3rd one.Your account must be verified before you can add new CloudFrontresources. To verify your account, please contact AWS Support(https://console.aws.amazon.com/support/home#/ ) and include thiserror message.Click the support linkNavigate to:Support / New case / Service limit increaseLimit type:CloudFront DistributionsIn Requests select:Limit: Web Distributions per AccountNew limit value: <TYPE_YOUR_NEW_VALUE_HERE>MY CASE: In my case, I had 2 distributions, wanted to create 3rd, but couldn't. So I have put as <TYPE_YOUR_NEW_VALUE_HERE> a number 10.Note: If nothing works choose last option as your last resort to confirm your account is verified.hope it helps.CommentShareAWS-User-3586868answered a year ago"


Question: How to : Multiple core devices and subscriptions across all core devices

"Hi,What I am trying to achieve is having a group of multiple core devices, where the lambda functions gets executed, and these devices will have to send and receive messages across these devices.these are my questions regarding the same,Can I have a group with only core devices?In the above mentioned case, the same lambda functions will be running on all the cores, so in this case, how to create the subscriptions as the target and source will be the lambda functions?Please advise on the same.Thanks.Edited by: mockingBird on Jan 11, 2019 12:22 AMFollowComment"

Answer: "0Hi MockingBird,No. You can have a group with only a core. However, you can only have one core per group.If you launch multiple groups you would setup a subscription from Lambda -> IoT Cloud and from IoT Cloud -> Lambda for each group.Can you tell me a bit more about your use case and what you're trying to do?Thanks,KeagenCommentShareKR-AWSanswered 4 years ago0Hi Keagen,Thanks for your reply.What we are trying to achieve is, we have a set of sensors, and we want to crunch the data there itself,  that means we prefer to execute the lambda in each of these device, and all these devices should communicate each other to share some data for their processing, that means all these devices need to be in a group.So could you please suggest a proper way to do the sameThanks,CommentSharemockingBirdanswered 4 years ago0Some more clarification neededCommentSharemockingBirdanswered 4 years ago0Hi mockingBird,Currently there isn't an easy way to communicate from core A to core B without going through the IoT Cloud. You can make core A a device for core B and vice versa. However, MQTT from device to core is only QoS 0. You'll need to handle missing/lost messages in your application.Please let me know if you have any other questions.Thanks,KeagenCommentShareKR-AWSanswered 4 years ago0Thanks a lot KaeganCommentSharemockingBirdanswered 4 years ago"


Question: Index in Lightsail Object Storage

Is there a way to enable static hosting with --index-document in Lightsail object storage?FollowComment

Answer: "0Hi, thanks for using Lightsail. I'm sorry to say that, currently, Lightsail object storage does not support this feature.LeslieCommentShareAWS-Leslieanswered 2 years ago"


Question: Custom DNS for MSK

"Hi, how can I "mask" an MSK endpoint with a custom domain? I'm trying to find a solution that won't cause SSL issues.Thanks!FollowComment"

Answer: "1[Updated as this answer was incorrect]CommentShareAnanth Tirumanuranswered 4 months agoEdbE 4 months agoCan you elaborate, at which point this provides connectivity with MSK/Kafka?Share0Accepted AnswerCustom domain names are not supported at this point.The only work around possible would involves non-data PLAINTEXT access for bootstrapping (fetch metadata request) and regular SSL for data-in-transfer encryption.Update advertised listeners to have port 9094 for PLAINTEXT protocolDefine R53 A-record with custom domain name resolved to an NLB with targets to all brokers to port 9092 (PLAINTEXT)a. create certificates in ACM and deploy on NLB listenersb. define NLB listeners as TLS to terminate TLS connection at NLBThis will let you use custom domain name with TLS traffic.Limitation of this approach is that it won't work with SASL (SCRAM, IAM), because listeners don't have SASL implementation mechanisms.CommentShareEdbEanswered 4 months ago"


Question: g5.48xlarge instance launch failed

"Hi, I am trying to launch a g5.48xlarge instance. But it kept saying my configuration was not supported.My configuration is listed below:Region: US-WEST-2 (Oregon)AMI: I choose one Ubuntu AMI from the "Community AMI" from verified provider and the AMI explicitly says it support G5 instance.Networking: I just choose the default existing groupStorage: I tried GP3, GP2, IO1, IO2 (requiring 2200 GB)Can anyone help me here? Also, I have encountered issues in launching P4 instance and cannot find any tutorial documentation. It is good to provide templates for each instance, so people know what should choose and what should not.Thanks.FollowComment"


Question: I can not sign into my AWS console

"I can not log into my AWS console AWS keeps sending a verification code to an old number I no longer have access to. What can I do?FollowCommentD G 9 days agoJust checking base with you about the answer provided.Let me know if you have any issues with this, or if it helps you then please accept my answer after you've tried it out - it would be much appreciated! Good luck :)Share"

Answer: "0Open up a case here: https://support.aws.amazon.com/#/contacts/aws-account-support/CommentShareEXPERTkentradanswered 12 days agoEXPERTBrettski-AWSreviewed 11 days ago0One is this a personal account? If so you could work to attain root privileges.https://docs.aws.amazon.com/signin/latest/userguide/troubleshoot-forgot-root-password.htmlOnce you have that you have "root domain" on the account in question.However just like those that messaged above. It is FAR better to sign in to submit.It's much faster to get support via the console, please click the button below to sign in and submit your requestSign in to submit your billing and account support requestLet me know if you have any issues with this, or if it helps you then please accept my answer after you've tried it out - it would be much appreciated! Good luck :)CommentShareD Ganswered 10 days ago"


Question: S3 sync command fails with "Connection broken" messages during reliable connection

"When attempting to download from S3 to a Windows 11 system using the AWS CLI s3 sync command, some files fail with the following error message:download failed: s3://bucketname/filename to D:\bucketname\filename ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))There were no connectivity issues during the transfer and other files from the same bucket during the same sync session are written as expected.aws-cli/2.7.13 Python/3.9.11 Windows/10 exe/AMD64 prompt/offWindows Terminal, Version: 1.13.11432.0Microsoft Windows 11 Enterprise, 10.0.22000 N/A Build 22000FollowCommentrePost-User-8683112 a year agoFor reference, similar behavior was reported by other users in Stack Overflow four years ago: https://stackoverflow.com/questions/51310901/aws-cli-s3-sync-error-an-existing-connection-was-forcibly-closed-by-the-remoteSharerePost-User-8683112 a year agoUsing the same sync command syntax, hardware, bucket, files, destination, and host OS, I was able to work-around the defect by running the AWS CLI from Ubuntu on WSL. The AWS client on Ubuntu/WSL was:aws-cli/2.7.12 Python/3.9.11 Linux/5.10.16.3-microsoft-standard-WSL2 exe/x86_64.ubuntu.20 prompt/offConsidering anti-virus as a possible cause per the Stack Overflow in the previous comment, the same anit-virus was running at the time both the Windows and WSL AWS CLIs were executing. The host OS was running the default Microsoft Defender Antivirus distributed with Microsoft Windows.Share"


Question: Transfer files (1GB to 2G) from web url to S3 on a schedule

My customer has a use case to access file from a web url periodically and upload to S3. This is currently implemented using lambda with schedule trigger and is working well for smaller files. But larger files need higher memory settings in lambda. Is there a way to directly upload to S3 without storing in memory or are there better serverless implementation options for this use case?FollowComment

Answer: "0Accepted AnswerI've previously used a very small container running in ECS for this. ECS has scheduling in-built, so you simply set it the schedule appropriately, and it runs regularly thereafter.The container I used was built on Alpine, and used wget to pull the files I needed.CommentShareEXPERTAndrew L.answered 3 years ago"


Question: DNS Zone TXT Records limited to 255 characters - need more

"Good day,New to Amazon Lightsail here. Migrating my instance off another vendor. Instance, firewall, static ip - all went ok. Now setting up my DNS Zone. One of my DNS TXT records is about 400 characters long. It is a google-generated DKIM record which contains key-value pairs including a public key.When I try to add the record in the DNS Zone Editor, an error pops up: "Each line must be between 0 and 255 characters and contain only printable ASCII characters". But since the 400 characters contain a public key, I don't think I should break it up. Is there any solution to this?Thanks -RicardoEdited by: RigaDorje on Jun 17, 2019 7:42 PMEdited by: RigaDorje on Jun 17, 2019 7:43 PMFollowComment"

Answer: "0Accepted AnswerStrings in SPF and TXT records should be no longer than 255 characters. However to get around this limitation, per RFC 4408 a TXT or SPF record is allowed to contain multiple strings, which should be concatenated together by the reading application.-- https://kb.isc.org/docs/aa-00356CommentShareDavid Ganswered 4 years ago0Hi David G,Thanks for the information and the reference link. I broke up the long string by making several lines each less than 255 chars (separated using the Enter key) -- trusting that the reading application will reassemble it back correctly. The editor saved my record.I then used Mxtoolbox's DKIM Lookup tool https://mxtoolbox.com/dkim.aspx to query the DNS record. It interpreted the DKIM record correctly.Thank you! -RicardoCommentShareRigaDorjeanswered 4 years ago"


Question: AWS charges for non-existent service

"Hi,on my bill I see high costs for a service called "Elastic Cloud (Elasticsearch Service) (sold by Elastisearch Inc)".I searched for a service with that name, but it seems, there is no such thing. I searched extensively. There are similar things like "CloudSearch" or "Elastic * Cloud", but I don't have anything there. If I open CloudSearch in any region it just displays "You have no search domains in *".Is there a hidden menu somewhere?Thanks,STefanFollowComment"

Answer: "1The "sold by ElasticsearchInc" makes me think this is from the AWS Marketplace. If you to the Marketplace (you can search for it in the AWS Console search bar) it should list your subscriptions.CommentShareEXPERTShlomo Dubrowinanswered 3 months ago0Thanks a lot. I just cancelled 8 active subscriptions that I didn't know of. :)CommentSharerePost-User-8067226answered 3 months agoShlomo Dubrowin EXPERT3 months agoAwesome, glad I could help.Share"


Question: "Does the Lambda , AWS Active MQ support Message Groups ?"

I want to use a single Active MQ  Queue and have it invoke lambda functions.  I want to enforce order but also have parallelism.   Active MQ has a concept of "Message Groups" which sounds perfect.    Does the AWS lambda support Message Groups and will invoke different instance of lambda synchronously with different message groups as required  based on load ?FollowComment

Answer: "0To answer my own question,  the AcitveMQ  Message Group is irrelevant as the documentation says"The Lambda service limits the concurrency to one execution environment per queue." seehttps://aws.amazon.com/blogs/compute/using-amazon-mq-as-an-event-source-for-aws-lambda/As such there will never be more than one lambda function being executed at one time as part of a synchronous call, as no no matter whether the Message Group is specified there will not be any opportunity to increase the amount of concurrent processing.CommentShareAWS-User-1467335answered a year ago"


Question: AWS IoT Policy - using * in combination with a text string or variable

"When using * in combination with a variable or text string the resource is not working as expected.When using a allow effect for action iot:Connect on a resource"arn:aws:iot:*:*:client/${iot:Connection.Thing.ThingTypeName}-*"I'am expecting to be able to connect using a client id based on the thing name and a string seperated by a dash.Eg. "MyThingName-client1" or "MyThingName-abc" based on the condition that the thing name is "MyThingName".The behaviour i experiance is that i can not connect using the wildcard in combination with variable or a string.A full reproducing example is shown below.  I can connect using only the thing name bu not using the thing name dash any string.{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Action": [        "iot:Connect"      ],      "Resource": [        "arn:aws:iot:*:*:client/${iot:Connection.Thing.ThingTypeName}",        "arn:aws:iot:*:*:client/${iot:Connection.Thing.ThingTypeName}-*"      ]    },    {      "Effect": "Allow",      "Action": [        "iot:Receive"      ],      "Resource": [        "*"      ]    },    {      "Effect": "Allow",      "Action": [        "iot:Publish"      ],      "Resource": [        "arn:aws:iot:*:*:topic/${iot:Connection.Thing.ThingTypeName}/input/${iot:Connection.Thing.ThingName}",        "arn:aws:iot:*:*:topic/${iot:Connection.Thing.ThingTypeName}/event/${iot:Connection.Thing.ThingName}"      ]    },    {      "Effect": "Allow",      "Action": [        "iot:Subscribe"      ],      "Resource": [        "arn:aws:iot:*:*:topic/${iot:Connection.Thing.ThingTypeName}/output/${iot:Connection.Thing.ThingName}"      ]    }  ]}FollowComment"

Answer: "0Thanks for using AWS IoT. As our docs(https://docs.aws.amazon.com/iot/latest/developerguide/thing-policy-variables.html) say, to use thing-policy variables these two things must be true.ClientId must match with ThingName.Certificate must be associated with the Thing (specified in the clientId).A certificate can be associated with more than one thing so we depend on device giving the thing name while connecting to IoT core in clientId field. This is needed even if certificate is attached to a single thing because if the certificate is attached to another thing in its lifetime (maybe accidentally), we don't want existing devices to break as policy variables cannot be resolved if we cannot identify the thing uniquely.We have a feature request in our backlog to support multiple client ids. We cannot give exact dates but we are working hard to get the features out based on the priorities.CommentShareAWS-User-3247576answered 3 years ago0I had overlooked the fact that the thing name should be equal to the client id.Thanks!CommentSharesavnikanswered 3 years ago"


Question: I got Instance status checks fail in EC2

I got Instance status checks fail in EC2 ubuntu 20.04when I check syslog log will be show  as below before status fail pls help to find root cause and how to fix itApr 18 13:46:19 ip-172-31-28-165 systemd-networkd[18468]: ens5: ConfiguredApr 19 07:07:01 ip-172-31-28-165 systemd-networkd[18468]: ens5: FailedFollowComment

Answer: 0I just launched a Ubuntu 20.04 instance with no issues. From your logs I see that ens5 failed which is the enhanced networking interface - are you trying to add secondary interfaces? Or do something else?For troubleshooting: I'd start with the base image and then work forwards until you get to the command/module that fails.CommentShareEXPERTBrettski-AWSanswered a year ago


Question: Table & Pivot Table Header Formatting No Longer Working

"Has anyone experienced existing analyses formatting of the headers is no longer in place and using the configuration options under edit for headers no longer alters the table headers? This is very recent, the analyses were formatted for background color and text wrapping. Neither are in working now even with the settings in place. Efforts to re-format cause no change onscreen. :(FollowComment"


Question: how do i delete a lambda@edge that isn't attached to a cloudfront?

"hiI only have two cloudfront distributions in my account, neither of them reference my edge lambda, and yet when I try to delete it I get this error message:Lambda was unable to delete arn:aws:lambda:us-east-1:???:function:???:1 because it is a replicated function. Please see our documentation for Deleting Lambda@Edge Functions and Replicas.The cloudfront it was associated with was deleted over a week ago and I am still getting this error when I try to delete.How can I disassociate it from cloudfront when the cloudfront it was attached to is already gone?FollowCommentperekrestov a year agoI have the same error.There are no associated CloudFront distributions for this Lambda for at least several months.It's 100% true, I've checked using https://former2.com/ - there are no associations.But I'm not able to delete the Lambda@Edge function.Share"

Answer: "0Are you sure you have checked all the behaviors of your CloudFront distributions, to make sure the function is not associated anywhere?If you are confident that is the case, and you've followed all the instructions in the documentation: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-delete-replicas.html then I'd recommend opening a case with AWS support to investigate further.CommentSharePaul_Lanswered a year agocctommyvn a year agoI only have 2 origins in this account so there aren't a lot of places this could hide, i'm fairly sure i've checked everywhere it could be and it's not there, and it's certainly had more than a few days to figure itself out.I don't have a support plan tho, so i'm not sure how to contact support about this. I certainly don't want to subscribe to a support plan just to tidy this up. ?Do you know how I raise this without one?Share0Because the lambda@edge are replicated to edge location, you need to wait for a while.Detache it from CloudFront distribution and wait for a while for AWS to remove replicates.Normally it should take no more that few minutes.But I also had a situation when I was not able to remove it even after days, then I recommend to contact support and they will remove it for yourCommentShareMGanswered a year ago"


Question: Where is my April Badge?

AWS Repost had a special badge on offer for 10 unanswered questions with one accepted.I met the criteria but never recieved the badge. Anyone else had this issue/know whats  happened to the badges?FollowComment

Answer: 0Accepted AnswerCongratulations on meeting the criteria and winning the badge. The badge will be granted soon.CommentShareEXPERTIsraa-Nanswered 17 days agoEXPERTsdtslmnreviewed 3 hours agoGary Mclean 17 days agoThank you very much for your response. Look forward to receiving it.Share


Question: Stop RDS db.r6g.2xlarge charges after changing to Serverless V2

"Hi, I am new to AWS. I set up an AWS Aurora database and my initial instance class was  Memory optimized classes (includes r classes) - db.r6g.2xlarge. After three days, I realized my usage was too low and the instance class was too expensive. I, therefore, decided to change to Serverless V2 but my bill is still adding db.r6g.2xlarge charges. How can I stop these charges from accruing?Kindly assist.FollowComment"

Answer: "0Hi there!It'd be best to open a case in the Support Center to have this investigated on an account level.Please reach out to our Account and Billing Support team to review the charges linked to your db.r6g.2xlarge instance and also help confirm if this is still running and accruing any unintended charges.You can use this link to quickly visit the Support Center: https://console.aws.amazon.com/support/home.Regards,Kraig E.CommentShareEXPERTAWS Support - Kraiganswered 7 months ago"


Question: Upstream timed out 110: Connection timed out on elastic beanstalk

"I have a node.js server hosted on Amazon Linux 2/5.4.9 elasticbeanstalk. This API server is running fine for the past year. But lately I found that the server was in severe condition and had to be restarted first so that things got better. In the logs file I get lots of error codes like the following:2022/12/27 16:42:07 [error] 23659#23659: *7403 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.31.24.78, server: , request: "GET /url-my-api HTTP/1.1", upstream: "http://127.0.0.1:8080/url-my-api", host: "api.mydomain.com", referrer: "https://mydomain.com/"I've never done any updates. Is there a solution that I should do?Additional information:Instance types : t2.smallScaling cooldown : 180sServers live 24 hoursFollowComment"

Answer: "0Are you using cloudwatch for any metric monitoring of your elastic beanstalk environment or servers? Have you also tried to integrate x-ray to determine further latency problems?CommentShareCloud_Ganswered 5 months agotoyotaintercom 5 months agoI haven't used it and don't know how to use itShare0Hello,Are you using nginx and configured proxy_pass with upstream server. If so you have some issue on resolving DNS.After a time, the upstream is not reachable and you have to restart to have access again. Is that the issue you have ?https://distinctplace.com/2017/04/22/nginx-upstream-timed-out/CommentShareIsmahelanswered 4 months ago"


Question: "Lightsail instance stops loading Wordpress site, requiring instance reboot"

"Wordpress website (bitnami package) running on Lightsail instance stops displaying after 2-3 days, requiring a restart.  Bitnami offers a built-in diagnostic tool which states that ports 22, 80 and 443 are not accessible remotely at the time this issue persists. Lightsail instance networking/firewall settings show that the ports in question are enabled.  Rebooting the Lightsail instance resolves the issue, but the problem repeats itself after 2-3 days.  We have to keep restarting the server every couple days to keep the website live.  Would like to find a reasonable solution.FollowCommentrePost-User-7512181 11 days agoI have same problem Any solution?Share"


Question: How to update a WebACL in WAFV2 using either PHP SDK or AWS CLI without losing important configuration

"When working with WAFV2 and making a call to GetWebACL the CustomResponse configuration is missing.If this configuration is subsequently used in a call to UpdateWebACL then the CustomResponse is lost.This appears to be a serious bug that would potentially cause undetected loss of configuration as the response from both API calls is successful.The API documentation states:To modify a web ACL, do the following:1) Retrieve it by calling GetWebACL2) Update its settings as needed3) Provide the complete web ACL specification to UpdateWebACLhttps://docs.aws.amazon.com/aws-sdk-php/v3/api/api-wafv2-2019-07-29.html#updatewebaclFor example a WAFV2 Rule with the following configuration:{   "Name":"RateLimit-3000",   "Priority":8,   "Statement":{      "RateBasedStatement":{         "Limit":3000,         "AggregateKeyType":"IP"      }   },   "Action":{      "Block":{         "CustomResponse":{            "ResponseCode":429,            "CustomResponseBodyKey":"TooManyRequests"         }      }   },   "VisibilityConfig":{      "SampledRequestsEnabled":true,      "CloudWatchMetricsEnabled":true,      "MetricName":"RateLimit-3000"   }}Is returned from an API call to GetWebACL as this:{   "Name":"RateLimit-3000",   "Priority":8,   "VisibilityConfig":{      "MetricName":"RateLimit-3000",      "CloudWatchMetricsEnabled":true,      "SampledRequestsEnabled":true   },   "Action":{      "Block": { }   },   "Statement":{      "RateBasedStatement":{         "AggregateKeyType":"IP",         "Limit":3000      }   }}If that configuration is then passed back to an API call to UpdateWebACL then the CustomResponse in the Block Action is removed.Is this a known bug or is there another way to correctly update a WebACL without loss of configuration?FollowCommentkumo-hiyori 2 months agoI would open a support ticket and bring to service team's attention.ShareEsteban_S 2 months agoExcellent question and the right place to put it!I am breaking down the answer in 4 easy steps:First of all, I suggest that you try the procedure below on a test/non-production WEBACL to get familiarized with the process.Even though it easy to enable custom response code using AWS WAFv2 web console, I understand that is some cases it can be convenient using AWS CLI to change the WAF WebACL programmatically.For testing purposes I disable custom response created with the AWS WAFv2 web console before starting and I'm going to enable custom response using AWS CLI. I'm going to change some parameters to highlight the difference with the custom response code generated using the web console.[1] First of all, we need to get some additional information from our WebACL such as theWebACL ID and the Locktoken. Please run the following command as shown below:$ aws wafv2 list-web-acls --scope REGIONALYou should get an output as shown below:“…{…“WebACLs”:[{“Name”: “ACL-NAME”“Id”:”123456789-idid-1234-1234-id12345678910”“Description”: “description”,“LockToken”: “id345678-1234-1234-idid-1234567891011”,“ARN”: “arn:aws:wafv2:region:awsaccount:regional/webacl/ACLNAME/ARN”},…”ShareEsteban_S 2 months ago[2] Copy the Web ACL ID and the LockToken.AWS WAF returns a token to your GET and LIST requests, to mark the state of the entity at the time of the request. To make changes to the entity associated with the token, you provide the token to operations like UPDATE and DELETE, with this you can add the “CustomResponse”. AWS WAF uses the token to ensure that no changes have been made to the entity since you last retrieved it. If a change has been made, the update fails with a WAFOptimisticLockException. If this happens, perform another get, and use the new token returned by that operation. So, make sure that you get the latest LockToken before running aws wafv2 update-web-acl.This is what happens when you don’t use the proper LockToken:“…User$ aws wafv2 update-web-acl –name ACLNAME –scope REGIONAL –default-action Allow={} –id 123456789-idid-1234-1234-id12345678910 –lock-token id345678-1234-1234-idid-1234567891011 –visibility-config SampleRequestEnabled=true,CloudWatchMetricsEnabled=true,MetricName=TestWebAcl-Metric –cli-input-json file://directory/WebAclJsonFile.jsonAn Error occurred (WAFOptimisticLockException) when calling the UpdateWebACL operation: AWS WAF couldn’t save your changes because someone changed the resource after you started to edit it. Reapply your changes.…”ShareEsteban_S 2 months ago[3] Once you get the Web ACL ID and the LockToken, you will need to run aws wafv2 update-web-acl using the following cli-skeleton. Modify the file accordingly and save it as “.json”==================================================={"Name": "<WebACL_Name>","Scope": "REGIONAL","Id": "123456789-idid-1234-1234-id12345678910 ","DefaultAction": {"Allow": {}},"Rules": [{Rule statement},"Action": {"Block": {"CustomResponse": {"ResponseCode": <StatusCode_selected>,"CustomResponseBodyKey": "<CustomResponseBodyName>","ResponseHeaders": [{"Name": "<CustomHeader(Optional)>","Value": "<CustomHeader_Value>"}]}}},"VisibilityConfig": {"SampledRequestsEnabled": true,"CloudWatchMetricsEnabled": true,"MetricName": "<Metric_name for the rule>"}}],"VisibilityConfig": {"SampledRequestsEnabled": true,"CloudWatchMetricsEnabled": true,"MetricName": "<Metric Name for the WebACL>"},"LockToken": "Latest LockToken gotten using "aws wafv2 list-web-acls --scope REGIONAL" ","CustomResponseBodies": {"<CustomResponseBodyName>": {"ContentType": "TEXT_HTML","Content": "HTML_Code/PlainText or Json"}}}ShareEsteban_S 2 months ago[4] In order to update the WebACL run "aws wafv2 update-web-acl", using the saved “.json” file, including your CustomResponse, as below:aws wafv2 update-web-acl --name <WebACL_Name> --scope REGIONAL --default-action Allow={} --id <WebACL ID> --lock-token <Latest LockToken> --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=<Metric_Name_for_WebACL> --cli-input-json file://<path>/Filename.jsonOnce you run the command, you should get the next LockToken. This confirms that your changes were successfully applied. See:“…User$ aws wafv2 update-web-acl –name ACLNAME –scope REGIONAL –default-action Allow={} –id 123456789-idid-1234-1234-id12345678910 –lock-token id345678-1234-1234-idid-1234567891012 –visibility-config SampleRequestEnabled=true,CloudWatchMetricsEnabled=true,MetricName=TestWebAcl-Metric –cli-input-json file://directory/WebAclJsonFile.json{“NextLockToken”: “id345678-1234-1234-idid-12345678910id”}…“After the above , feel free to check the WAF web console to confirm that our changes were applies successfully.Additional information is available at:Custom responses for Block actionshttps://docs.aws.amazon.com/waf/latest/developerguide/customizing-the-response-for-blocked-requests.htmlUpdateWebACLhttps://docs.aws.amazon.com/waf/latest/APIReference/API_UpdateWebACL.htmlShare"

Answer: "0Accepted AnswerThanks for all the suggestions. After contacting AWS Support, this issue has now been solved by updating both PHP SDK and AWS CLI to their latest versions. Hopefully this answer will help others as using an older version of either PHP SDK or AWS CLI will still work but using them will be destructive and cause configuration to be inadvertently lost.CommentSharePeteranswered 2 months ago0Hi,Thank you for contacting AWS Re:post .I understand you have some clarifications regarding updating a WebACL in WAFV2 using either PHP SDK or AWS CLI without losing important configurationLooking at the issue, This would require further analysis and troubleshooting. I would suggest you to open a case with the WAF premium support team and provide details such as:--> WAF configuration--> Any supported Logs (Access logs)Thanks and have a nice day!CommentShareSUPPORT ENGINEERAnsh_C answered 2 months ago"


Question: Allow alternative login

"Often people don't have personal AWS accounts and do not want their re:Post profile to be a hostage of their current employment! Please, allow alternative authentication! And maybe linking AWS accounts (more than one).FollowComment"

Answer: "5Would suggest the team to add social media logins as well.While the team is working to add more login methods, one alternative is to create a new, personal AWS account, and leveraging the 1 year free tier one might be getting some additional AWS knowledge that might help them professionally down the road.CommentShareJason_Sanswered a year agoJon Bonso a year agoI think this is a good workaround. Yes, having a new login method is pretty convenient but underneath that convenience lies the risk of having bogus profiles and lots of spam in this new platform.Sharelocalghost a year agoHow would one add their cert badges from Credly if the emails don't match?Share5Thank you for your suggestion for more flexible login/account capabilities. We hear your feedback on this, and this enhancement is already on our priority list -- please stay tuned for more around this request. We appreciate your feedback and continued participation in the re:Post community!Kind Regards,Monica L.CommentShareSUPPORT ENGINEERMonica_Lluisanswered a year ago4Hi, using an account to log in ensures that you are a customer of AWS and have accepted terms and conditions.  I understand that this is the reason behind requiring an account.The concern about loosing your profile when switching jobs sounds reasonable, my advice is to create a personal free account and leverage the free tier to practice.About login with social media (once you have initially authenticated with your AWS account), could be a good request for enhancement. I'll channel this request internally.Hope my answer helps. Please remember to choose an accepted answer.CommentShareDario_Ganswered a year agozachjonesnoel a year agoCertainly having a personal account and using that for re:Post makes sense that way you will also get use AWS env for some hands-on experience.Share0It is now possible to register for re:Post using an Amazon Builder ID.CommentShareEdanswered 6 months ago"


Question: CodePipeline - how to pass and consume multiple artifacts across CodeBuild Steps?

"Hello.  I know I can pull multiple Source actions (e.g. 2 GitHub repos) into a CodeBuild step by adding multiple Source Artifacts. It would seem that i should follow the same pattern (e.g. $CODEBUILD_SRC_DIR__<secondary_artifact_name>) where individual CodeBuild actions are producing and consuming artifacts. In my case, my consuming step needs the following inputs:Source input (primary source, to get my buildspec.yml)The output of a prior CodeBuild step (a couple of files).I am successful in setting up the Input/Output artifacts in my pipeline and validating that the zipped files are placed into the pipeline s3 bucket. Its all good up until that point.However, when I run the consuming step and troll the file system I do not see anything like $CODEBUILD_SRC_DIR__<secondary_artifact_name>.Is this scenario supported? if so, how do I reference the secondary artifacts in my CodeBuild buildspec?Thanks DaveFollowComment"

Answer: "0It is supported and the name you're looking for is correct. So I think you just have a small error.Here's an example (CDK Typescript): https://github.com/awslabs/aws-greengrass-labs-component-for-the-things-stack-lorawan/blob/main/cicd/lib/cicd-stack.tsThe output from the build stage is passed as an extra input to the deploy stage. Both of which are CodeBuild actions:        const buildAction = new codepipeline_actions.CodeBuildAction({            actionName: `${Names.PREFIX_DASH}-build`,            project: buildProject,            input: source,            outputs: [build]        });        const deployAction = new codepipeline_actions.CodeBuildAction({            actionName: `${Names.PREFIX_DASH}-deploy`,            project: deployProject,            input: source,            extraInputs: [build],            outputs: [deploy],            environmentVariables: {                "GREENGRASS_CORE_NAME": { value: context.greengrassCoreName }            }        });And the buildspec for the deploy stage, that consumes the output from the build stage: https://github.com/awslabs/aws-greengrass-labs-component-for-the-things-stack-lorawan/blob/main/cicd/deployspec.yaml      # Get the recipe file generated by the build stage      - RECIPE_FILE=$(ls $CODEBUILD_SRC_DIR_Build/aws.greengrass.labs.*.json)CommentShareEXPERTGreg_Banswered a year agoDaveC a year agoHey Greg - many apologies for not touching base back sooner.  I saw your answer but was getting some weirdness in the portal.My scenario is specific to using CodeBuild within CodePipeline. There are some constraints with this that are not applicable with CodeBuild so I dont think this example will work for me.ShareGreg_B EXPERTa year agoDave, the example above is indeed CodeBuild within CodePipeline.Share0How about caching in AWS CodeBuild. A cache can store reusable pieces of your build environment and use them across multiple builds. Your build project can use one of two types of caching: Amazon S3 or local.CodeBuild also supports secondaryArtifacts. These artifacts use the secondary-artifacts block of the buildspec file that is nested inside the artifacts block.CommentShareRoBanswered a year agoDaveC a year agoHey RoB - thanks, I looked into caching and that appears to be an optimization for a single codebuild step - i need a normal workflow with separate producing and consuming steps.  As for secondary artifacts, I'm using that feature - it just doesn't behave the way I would expect.Share"


Question: Amplify + Nextjs 12: CloudFront error 503 on API routes

"I've deployed a Nextjs v12 app on Amplify but I got error 503 on nextjs API routes:ISSUEThe Lambda function associated with the Cloudfront distribution is invalid or doesn't have the required permissions. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.If you provide content to customers through Cloudfront, you can find steps to troubleshoot and help prevent this error by reviewing the Cloudfront documentation.MY TRIESI tried to add permissions for lambda functions to the Amplify role but it doesn't work. Also can't find a policy specific to lambda@edge.I can see on the linked Cloufront the lambda@edge.I found different documentation on this issue but the only solution proposed was to downgrade Nextjs to v11, something I wan to avoid.No issues logged in the Amplify deploy:2023-03-21T18:42:58 [INFO]: Deployed the following resources to your account:2023-03-21T18:42:58 [INFO]: - CloudFront Domain ID: xxx2023-03-21T18:42:58 [INFO]: - SSR Lambda@Edge: xxx2023-03-21T18:42:58 [INFO]: - API Lambda@Edge: xxx2023-03-21T18:42:58 [INFO]: - Image Optimization Lambda@Edge: xxx2023-03-21T18:42:58 [INFO]: - S3 Bucket: xxx2023-03-21T18:42:59 [INFO]: Deployment completeSPECSI'm using Amplify 'web dynamic', Nextjs v12, prisma (I do 'npx prisma generate' in the build phase).FollowComment"


Question: "Using a ConditionExpression in UpdateItem, without granting Write permission to the attribute used in the ConditionExpression"

"I have a DynamoDB database where I'm trying to lock down access to a table (MyTable) to only allow certain updates to be performed by certain users. So the table has items with attributes such as Name, Email, ItemStatus, PrivateAttr1, PrivateAttr2 etc. UserA is allowed to update the Name,  Email etc. but not ItemStatus, PrivateAttr1, PrivateAttr2 etc.To achieve this, UserA belongs to a group to which I have applied a number of IAM policies including one which allows "UpdateItem" with the following Condition:"Condition": {"ForAllValues:StringEqualsIfExists": {"dynamodb:Attributes": ["Name","Email"]}}This all works correctly - I'm using the C++ SDK - and allows UserA to change ONLY the Name and Email attributes but without allowing them to update ItemStatus, Attr1 and Attr2.What I want to do now is only allow this change based on a specific value of the "ItemStatus" attribute. But this particular user is not allowed to CHANGE the ItemStatus attribute. But I want them to be able to only update other attributes if the ItemStatus attribute (as set by another user) is a certain value (say 2).So I have added a ConditionExpression "ItemStatus = :status" and added a AttributeValue of (:status = 2). But instead of getting a failure along the lines of "Conditional Request failed", I am getting a permissions failure: "User: .../UserA is not authorized to perform: dynamodb:UpdateItem on resource: ....:table/MyTable"It appears that in order to allow the UpdateItem expression to include a ConditionExpression based on the ItemStatus, I need to include ItemStatus in the "UpdateItem" IAM Policy - surely this is not the case, as it also allows UserA to be able to edit the ItemStatus, which I DON'T want to allow.Thanks in advance for any pointers on how to achieve this!FollowComment"

Answer: "1Hi,What you are trying to achieve is not currently possible in a single operation with DynamoDB, as ConditionExpression is not a read operation, its a write operation. As you have blocked this user from UpdateItem on that attribute, they will be unable to do a conditional check on it. You would need to read the item first, making your assertion on the client side before updating.Let me know if you would like me to raise a Feature Request with the team for this functionality.CommentShareEXPERTLeeroy Hannigananswered a year agoLiam a year agoHi Leeroy, many thanks for the quick reply. That makes sense, although I do think that Condition Expression should be treated as a "read" operation even when it's part of a "Write" operation. So yes, please do raise a feature request - and if you can, I'd be grateful if you could let me know how I could keep track of whether that functionality is going to be included! In the meantime I'll find a way around this by doing a "read" first, but ideally I'd be able to do it all as part of the same request. Thanks!ShareLeeroy Hannigan EXPERTa year agoI can raise the FR for you, consider that done. But unfortunately there is no way to track requests. Rest assured the team implement requests based on demand and customer needs.You can keep up with the latest feature releases on the AWS Portal https://aws.amazon.com/new/Share"


Question: "API Gateway/OpenAPI 3.0 deploy warning, what is ComposedSchema?"

"Hello all!I am trying to use the allOf keyword to inherit attributes of one schema model into another as such:providerIdentityExpansion:      description: ID of the Provider Identity with fields possibly expanded.      allOf:        - $ref: '#/components/schemas/providerIdentityNoExpansion'      properties:        oft_confused_with:          oneOf:          - $ref: '#/components/schemas/oft_confused_with'          - $ref: '#/components/schemas/oftConfusedWithExpansionArray'      title: ProviderIdentityExpansion      type: objectI previously just duplicated the model, but changed the names (one with expansion, one without). But using allOf, when I use swagger-cli to bundle all OpenAPI 3.0 Spec docs together into one file and deploy on API Gateway, it responds with following warning:"Unsupported model type 'ComposedSchema' in 200 response to method 'GET /providers/{id}/identity'. Ignoring.",        "Unsupported model type 'ComposedSchema' in 200 response to method 'GET /providers/{id}/identity/children'. Ignoring."Is anyone familiar with the model type ComposedSchema, and how to utilize the allOf keyword without triggering this error in AWS?FollowComment"


Question: Java Deployment across multiple machines - best practice

"So I have a few server farms running Glassfish/Payara and we are moving off letting developers on the box's for numerous reasons and we want to automate the deployment ..The WAR file can reside on S3 or an EFS that all servers have access to so trying to think of the best way.  The process would simply be they develop on a staging box, confirm things work, then say go live.   With Bash and SSH I was able to cycle through the machines but had to put the servers in an array, and just removed from the target group, deployed and added, but we are also moving to more autoscaling groups, and the servers should really be able to be terminated and new ones just start from the launch template, deploy the latest version (as it maybe old), then put in the target group so I won't have the ability to do this in an array.So, I thought about in bash someway to query where the tag was X, get the IP's and cycle though but that still seems a bit legacy.  I have SSM up and in place, so trying to think of what the developers can do when there done, have a WAR file that needs to go to X number of servers and just say go.   From there, it goes to server 1, 2, X amount and naturally sequential so they all don't get pulled out at once.The other thought I used for some basics is code-deploy and have them just commit the WAR file.  That would trigger the copy down, using the webhooks we could do a script after to say run this deploy now which I think maybe easier and cleaner all based on tags, etc.   The only question I have on this path is that if the group says have 5 servers, 1 fail's and gets terminated, when the group starts a new one from the template, the code would be outdated, so how do I ensure on startup that its the latest codebase.Thank you for all the help and thoughts in this.FollowComment"

Answer: "1Accepted AnswerHi, I think this is the solution when using CodeDeploy for newly launched instances: https://docs.aws.amazon.com/codedeploy/latest/userguide/integrations-aws-auto-scaling.htmlCommentShareDidier_AWSanswered 15 days agolraymond 14 days agoPerfect, exactly what I was needing!Share"


Question: MediaLive not respecting RTMP Bitrate

"I am using medialive and mediaconnect to restream srt streams to multiple destinations.I realized that the RTMP output is not respecting the bitrate specified and even sometimes is not respecting the resolution.Is there a special setting to enable for it to output with the specified resolution and bitrate?Current setttings:Width : 1920  Height: 1080 Codec: H264Rate Control: CBR 6 500 000Frame Rate: Specified 30000 / 1001 ProgressiveGOP Size : 60 Frames B-Frames 0Scaling Default AFD Passtrough--> Received stream : 1920x1080 or 1280x720 / 1.3 to 1.6 mbps** EDIT/UPDATE**I tried to create a brand new channel from a workflow, selecting twitch as a destination but replacing the endpoint with my own RTMP destination. The only output option was 1080p@30/4.5mbps output.Didn't change anything except the endpoint. Start the flow.And I'm still receiving 1.5mbps video.I was hoping to avoid rolling my own encoder to forward streams to our clients, but if there's no way to control the output of medialive, I don't see how I can achieve it witihin AWS.FollowCommentB-Wilkinson 10 months agoA few comments. Not sure why you have 0 B frames. B frames are needed for efficient compression. Either way not the problem you are having.I am almost certain that you did these tests with static content. RTMP generally ramps itself down when content is static such as color bars and a slate.I would define all aspect of the Media Live encode and not allow "same as source". Pixel shape should be 1 to 1.Semi curious how are you getting SRT into Media Live. The only way I can think of is through an SRT listen in Media Connect. Media Live does not accept SRT.ShareTechSol 10 months agoYes, sorry I was unclear. SRT in from mediaconnect, then vpc input from mediaconnect to medialive.I stream the same content from my computer to the RTMP destination, I get 1080p and 6.5mbps.I stream out of mediaconnect with SRT to my machine, I get 1080p.But when I do mymachine->mediaconnect->medialive->RTMP 3rd party provider.  I get ~1.5mbps and sometimes 1280x720.Also, I tried with static content and moving text and the infamous twitch sync footage you can find somewhere on the internet.As for the B frames, it's been a habit to remove them to lower the lookahead as it would require the full GOP to be downloaded before being displayed. Not sure in this case it changes much though.Share"


Question: Getting error while connecting EKS cluster error: You must be logged in to the server (Unauthorized)

"Hi All,I have create a EKS cluster . and i am trying to connect this cluster from my local machine .I am getting this error while executing kubectl command .Required your support .]# kubectl describe -n kube-system configmap/aws-autherror: You must be logged in to the server (Unauthorized)FollowComment"

Answer: "0Accepted AnswerDid you update your kubeconfig ?https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.htmlStep 3After you create your cluster with eksctl, you want to update your Kubeconfig withaws eks update-kubeconfig --region region-code --name my-clusterCommentShareRolyanswered 3 months agorePost-User-1891481 3 months agoThis file is updated .still facing same issue .aws eks update-kubeconfig --region eu-west-1 --name  vikas1Updated context arn:aws:eks:eu-west-1:912603790734:cluster/vikas1 in /root/.kube/configShare0Are you logged into AWS as well ? can you issue ?aws s3 ls.?CommentShareRolyanswered 3 months ago"


Question: "aws swf register-domain" output argument

"My use case  is region build services chain reaction plugin - aws cli.  My objective is to reduce the number of tasks to automate.  I'd like to execute one command and receive feedback about success or failure to enable workflow continuation or do retry enforcement.I am using aws swf register-domain for its intended purpose and I'd like to understand if it is at all possible to invoke a response from the command execution other than none.  I look longingly at the --output argument which, according to documentation, allows one to determine an output format.   Does this imply that using the argument enables one to invoke a json response from the execution of a sequence that looks like this aws --region xyz-northeast-1  swf register-domain --name unique-domain-name --workflow-execution-retention-period-in-days 90 --output json ?FollowComment"


Question: getBucketNotificationConfiguration not returning filters

"Hi,We put the topic configuration using 'putBucketNotificationConfiguration' but when we tried to get it back with getBucketNotificationConfigurationIt returns everything but the Filters and there are filters, I can see those on the web and when creating the event we pass the topic config with the filters.Here is what we send to the putBucketNotificationConfiguration as topic configuration{"Id": "my-id","TopicArn": "arn:aws:sns:eu-west-1:account-number:my-id","Events": ["s3:ObjectCreated:*"],"Filter": {"Key": {"FilterRules": [{"Name": "prefix","Value": "data-lake/scada_mdb"}]}}}And this is what we get back{"Id": "my-id","Events": ["s3:ObjectCreated:"],"Event": "s3:ObjectCreated:","Topic": "arn:aws:sns:eu-west-1:account-number:my-id"}As you can see the filters are missing and for us this is the most crucial part. Any reason those are not coming back?We know for sure that the bucket is configured correctly because we can see the notification working as expected but we need to be able to get the right configuration back.FollowComment"

Answer: "0My bad, I was running 'getBucketNotification' instead of 'getBucketNotificationConfiguration' but as I was promisifying I didn't notice >.<CommentSharealietors-sennenanswered 2 years ago"