Question: Route 53 How to delete ns and soa record?

"Hello Community. I am trying to set up an account on google work space. They send me here at AWS for domain verify. I think i just need TXT record but I created additional name server (NS) and  (SOA) records in the Route 53 hosted zone, and i can not delete the existing NS and SOA records. What i need to do?FollowComment"

Answer: "1if your domain is in Route53 then hosted zone must min contain SOA and NS record. You dont need to delete them. They are necessary for domain to function and support your TXT record.So just create your TXT record as google said, and dont worry about NS/SOA record.CommentShareMuhammadanswered 5 months ago0Would you please reset or clear all my set up please. Or woul you show me how to clear all set up please thank you so muchCommentSharerePost-User-0120123answered 5 months agoVM 5 months agoif you're looking to delete the Public hosted zone, you can reference this link: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/DeleteHostedZone.htmlShare0You cannot delete the NS and SOA type records-https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-deleting.htmlTo read more about the NS and SOA record types, you can reference the link given below:https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/SOA-NSrecords.htmlCommentShareVManswered 5 months ago"


Question: Textract Confidence for Query Results

"Hi,when I use queries in textract I get the confidence score back. I wonder what this confidence implies.Is it the confidence that the question is answered correctly? Is it a mix between the confidence of the line, word and question?Thanks and kind regards,OaulFollowComment"

Answer: "1Confidence that the answer is correct for the given query. Includes confidence of this being a match for the query and confidence in the value.CommentShareMartin Schade _AWS_answered a year ago0Hi, I´m back and my research don't support that the query confidence takes into the confidence of the value.As an example I have a png that I analyse.When I analyse it via the following:response = textract.analyze_document(Document={'S3Object': {            'Bucket': S3Bucket,            'Name': S3key,            }            },         HumanLoopConfig={            "FlowDefinitionArn":"arn:aws:sagemaker:eu-central-1:392047662260:flow-definition/onekey",            "HumanLoopName":"onekey2",            "DataAttributes" : {                "ContentClassifiers":["FreeOfPersonallyIdentifiableInformation","FreeOfAdultContent"]            }         },    FeatureTypes=["QUERIES"],    QueriesConfig={        "Queries": [{            "Text": "Blub",            "Alias": "Blub"        }]    })I get the following response:{'DocumentMetadata': {'Pages': 1}, 'Blocks': [{'BlockType': 'PAGE',   'Geometry': {'BoundingBox': {'Width': 1.0,     'Height': 1.0,     'Left': 0.0,     'Top': 0.0},    'Polygon': [{'X': 9.166517763292134e-17, 'Y': 0.0},     {'X': 1.0, 'Y': 1.6361280468230185e-16},     {'X': 1.0, 'Y': 1.0},     {'X': 0.0, 'Y': 1.0}]},   'Id': '23dddda1-597b-4f50-a3b7-8f694a68bc25',   'Relationships': [{'Type': 'CHILD',     'Ids': ['8007bbfe-e149-46a8-983f-a1d851819349',      'caa128eb-48d9-47fd-911c-fe8bfcd11bce',      '90694f79-c3ea-49c5-a008-a16b3ee107f3',      'c894d2b7-cb06-49eb-8b26-a5188236107b',      'fcea87fe-9be9-4f1d-a56b-a2536c61c855']}]},  {'BlockType': 'LINE',   'Confidence': 99.37574005126953,   'Text': 'Name: Paul Spöring',   'Geometry': {'BoundingBox': {'Width': 0.5787070989608765,     'Height': 0.0934426411986351,     'Left': 0.09132613986730576,     'Top': 0.22022898495197296},    'Polygon': [{'X': 0.09132613986730576, 'Y': 0.22022898495197296},     {'X': 0.6700332760810852, 'Y': 0.22022898495197296},     {'X': 0.6700332760810852, 'Y': 0.31367161870002747},     {'X': 0.09132613986730576, 'Y': 0.31367161870002747}]},   'Id': '8007bbfe-e149-46a8-983f-a1d851819349',   'Relationships': [{'Type': 'CHILD',     'Ids': ['b585bb68-2ee8-4f14-a3b7-b8255b57272a',      '8aab4266-4eaa-4dba-b8bb-7624cb02e3ed',      'ff606403-3987-4b8b-8a5b-16c1d40adb09']}]},  {'BlockType': 'LINE',   'Confidence': 99.62093353271484,   'Text': 'Alter: 20',   'Geometry': {'BoundingBox': {'Width': 0.2573024034500122,     'Height': 0.07728662341833115,     'Left': 0.08906695991754532,     'Top': 0.3895558714866638},    'Polygon': [{'X': 0.08906695991754532, 'Y': 0.3895558714866638},     {'X': 0.3463693857192993, 'Y': 0.3895558714866638},     {'X': 0.3463693857192993, 'Y': 0.46684250235557556},     {'X': 0.08906695991754532, 'Y': 0.46684250235557556}]},   'Id': 'caa128eb-48d9-47fd-911c-fe8bfcd11bce',   'Relationships': [{'Type': 'CHILD',     'Ids': ['26853a02-3da9-4c13-ae60-f583c9d4cfb4',      'd8bfac6e-17ad-44fc-a7ca-1982b29943b4']}]},  {'BlockType': 'LINE',   'Confidence': 82.59257507324219,   'Text': 'Blub: you',   'Geometry': {'BoundingBox': {'Width': 0.43706580996513367,     'Height': 0.11816448718309402,     'Left': 0.09128420799970627,     'Top': 0.560679018497467},    'Polygon': [{'X': 0.09128420799970627, 'Y': 0.560679018497467},     {'X': 0.5283499956130981, 'Y': 0.560679018497467},     {'X': 0.5283499956130981, 'Y': 0.6788434982299805},     {'X': 0.09128420799970627, 'Y': 0.6788434982299805}]},   'Id': '90694f79-c3ea-49c5-a008-a16b3ee107f3',   'Relationships': [{'Type': 'CHILD',     'Ids': ['ccb5d25f-dbf6-47b4-a0e4-29f4c90fe5e1',      '71e21630-0ca9-400f-a050-e3c5168c5174']}]},  {'BlockType': 'WORD',   'Confidence': 99.62495422363281,   'Text': 'Name:',   'TextType': 'PRINTED',   'Geometry': {'BoundingBox': {'Width': 0.19461767375469208,     'Height': 0.07422882318496704,     'Left': 0.09132613986730576,     'Top': 0.2239169031381607},    'Polygon': [{'X': 0.09132613986730576, 'Y': 0.2239169031381607},     {'X': 0.28594380617141724, 'Y': 0.2239169031381607},     {'X': 0.28594380617141724, 'Y': 0.29814571142196655},     {'X': 0.09132613986730576, 'Y': 0.29814571142196655}]},   'Id': 'b585bb68-2ee8-4f14-a3b7-b8255b57272a'},  {'BlockType': 'WORD',   'Confidence': 98.95515441894531,   'Text': 'Paul',   'TextType': 'PRINTED',   'Geometry': {'BoundingBox': {'Width': 0.12700314819812775,     'Height': 0.07847694307565689,     'Left': 0.30101263523101807,     'Top': 0.22022898495197296},    'Polygon': [{'X': 0.30101263523101807, 'Y': 0.22022898495197296},     {'X': 0.428015798330307, 'Y': 0.22022898495197296},     {'X': 0.428015798330307, 'Y': 0.29870593547821045},     {'X': 0.30101263523101807, 'Y': 0.29870593547821045}]},   'Id': '8aab4266-4eaa-4dba-b8bb-7624cb02e3ed'},  {'BlockType': 'WORD',   'Confidence': 99.54711151123047,   'Text': 'Spöring',   'TextType': 'PRINTED',   'Geometry': {'BoundingBox': {'Width': 0.23034201562404633,     'Height': 0.09066428989171982,     'Left': 0.4396912455558777,     'Top': 0.22300733625888824},    'Polygon': [{'X': 0.4396912455558777, 'Y': 0.22300733625888824},     {'X': 0.6700332760810852, 'Y': 0.22300733625888824},     {'X': 0.6700332760810852, 'Y': 0.31367161870002747},     {'X': 0.4396912455558777, 'Y': 0.31367161870002747}]},   'Id': 'ff606403-3987-4b8b-8a5b-16c1d40adb09'},  {'BlockType': 'WORD',   'Confidence': 99.47355651855469,   'Text': 'Alter:',   'TextType': 'PRINTED',   'Geometry': {'BoundingBox': {'Width': 0.16495245695114136,     'Height': 0.07728662341833115,     'Left': 0.08906695991754532,     'Top': 0.3895558714866638},    'Polygon': [{'X': 0.08906695991754532, 'Y': 0.3895558714866638},     {'X': 0.2540194094181061, 'Y': 0.3895558714866638},     {'X': 0.2540194094181061, 'Y': 0.46684250235557556},     {'X': 0.08906695991754532, 'Y': 0.46684250235557556}]},   'Id': '26853a02-3da9-4c13-ae60-f583c9d4cfb4'},  {'BlockType': 'WORD',   'Confidence': 99.76831817626953,   'Text': '20',   'TextType': 'PRINTED',   'Geometry': {'BoundingBox': {'Width': 0.07841669768095016,     'Height': 0.07549438625574112,     'Left': 0.26795268058776855,     'Top': 0.3907521665096283},    'Polygon': [{'X': 0.26795268058776855, 'Y': 0.3907521665096283},     {'X': 0.3463693857192993, 'Y': 0.3907521665096283},     {'X': 0.3463693857192993, 'Y': 0.4662465453147888},     {'X': 0.26795268058776855, 'Y': 0.4662465453147888}]},   'Id': 'd8bfac6e-17ad-44fc-a7ca-1982b29943b4'},  {'BlockType': 'WORD',   'Confidence': 99.90613555908203,   'Text': 'Blub:',   'TextType': 'PRINTED',   'Geometry': {'BoundingBox': {'Width': 0.15235087275505066,     'Height': 0.07806506007909775,     'Left': 0.09128420799970627,     'Top': 0.560679018497467},    'Polygon': [{'X': 0.09128420799970627, 'Y': 0.560679018497467},     {'X': 0.24363507330417633, 'Y': 0.560679018497467},     {'X': 0.24363507330417633, 'Y': 0.638744056224823},     {'X': 0.09128420799970627, 'Y': 0.638744056224823}]},   'Id': 'ccb5d25f-dbf6-47b4-a0e4-29f4c90fe5e1'},  {'BlockType': 'WORD',   'Confidence': 65.27902221679688,   'Text': 'you',   'TextType': 'HANDWRITING',   'Geometry': {'BoundingBox': {'Width': 0.2605644464492798,     'Height': 0.10577575862407684,     'Left': 0.26778554916381836,     'Top': 0.5730677247047424},    'Polygon': [{'X': 0.26778554916381836, 'Y': 0.5730677247047424},     {'X': 0.5283499956130981, 'Y': 0.5730677247047424},     {'X': 0.5283499956130981, 'Y': 0.6788434982299805},     {'X': 0.26778554916381836, 'Y': 0.6788434982299805}]},   'Id': '71e21630-0ca9-400f-a050-e3c5168c5174'},  {'BlockType': 'QUERY',   'Id': 'c894d2b7-cb06-49eb-8b26-a5188236107b',   'Relationships': [{'Type': 'ANSWER',     'Ids': ['fcea87fe-9be9-4f1d-a56b-a2536c61c855']}],   'Query': {'Text': 'Blub', 'Alias': 'Blub'}},  {'BlockType': 'QUERY_RESULT',   'Confidence': 98.0,   'Text': 'you',   'Geometry': {'BoundingBox': {'Width': 0.26047903299331665,     'Height': 0.10400000214576721,     'Left': 0.26646706461906433,     'Top': 0.5720000267028809},    'Polygon': [{'X': 0.26646706461906433, 'Y': 0.5720000267028809},     {'X': 0.5269461274147034, 'Y': 0.5720000267028809},     {'X': 0.5269461274147034, 'Y': 0.6759999990463257},     {'X': 0.26646706461906433, 'Y': 0.6759999990463257}]},   'Id': 'fcea87fe-9be9-4f1d-a56b-a2536c61c855'}], 'HumanLoopActivationOutput': {'HumanLoopArn': 'arn:aws:sagemaker:eu-central-1:392047662260:human-loop/onekey2',  'HumanLoopActivationReasons': ['ConditionsEvaluation'],  'HumanLoopActivationConditionsEvaluationResults': '{"Conditions":[{"Or":[{"And":[{"ConditionType":"ImportantFormKeyConfidenceCheck","ConditionParameters":{"ImportantFormKey":"Blub","ImportantFormKeyAliases":[],"KeyValueBlockConfidenceLessThan":70.0,"WordBlockConfidenceLessThan":70.0},"EvaluationResult":false},{"ConditionType":"ImportantFormKeyConfidenceCheck","ConditionParameters":{"ImportantFormKey":"Blub","ImportantFormKeyAliases":[],"KeyValueBlockConfidenceGreaterThan":0.0,"WordBlockConfidenceGreaterThan":0.0},"EvaluationResult":false}],"EvaluationResult":false},{"ConditionType":"MissingImportantFormKey","ConditionParameters":{"ImportantFormKey":"Blub","ImportantFormKeyAliases":null},"EvaluationResult":true}],"EvaluationResult":true}]}'}, 'AnalyzeDocumentModelVersion': '1.0', 'ResponseMetadata': {'RequestId': 'bc5b9bac-9e6b-4b4b-8686-127526be99e8',  'HTTPStatusCode': 200,  'HTTPHeaders': {'x-amzn-requestid': 'bc5b9bac-9e6b-4b4b-8686-127526be99e8',   'content-type': 'application/x-amz-json-1.1',   'content-length': '7318',   'date': 'Fri, 16 Sep 2022 11:26:37 GMT'},  'RetryAttempts': 0}}As you can see, the confidence of the query is 98%, but the confidence of the value is just 65%.CommentSharegoodvirusanswered 9 months ago"


Question: DFS and AWS Directory Service

"On the following thread, it says that AWS Directory Service (Microsoft AD) can be used for DFS Replication.https://forums.aws.amazon.com/thread.jspa?threadID=244580Nowhere in the documentation does this get mentioned for FSx. Can we use it for FSx, or must be have a Windows server for this purpose?Thanks,Paul.FollowComment"

Answer: "0Hi asdflkjh,Yes, Amazon FSx fully supports the use of DFS Replication and DFS Namespaces with your file systems. We include a guide in our documentation for setting up multi-AZ file system deployments using DFS Replication and DFS Namespaces (https://docs.aws.amazon.com/fsx/latest/WindowsGuide/multi-az-deployments.html). We also include guides in our documentation for scaling out performance (https://docs.aws.amazon.com/fsx/latest/WindowsGuide/scale-out-performance.html and https://docs.aws.amazon.com/fsx/latest/WindowsGuide/scale-out-read.html), and for grouping multiple file systems into a common namespace (https://docs.aws.amazon.com/fsx/latest/WindowsGuide/group-file-systems.html).Thank you,Amazon FSx teamCommentShareAWS-User-9252611answered 4 years ago"


Question: Hyper-V cannot be installed: The processor does not have required virtualization capabilities.

"Hello every body, i subscribed for Docker Engine - Enterprise for Windows Server 2019 and followed this videohttps://www.youtube.com/watch?v=7eObt3MSzWw&ab_channel=CloudInfrastructureServicesbut i encountered the the below error :Success Restart Needed Exit Code      Feature ResultFalse   Maybe          Failed         {}Install-WindowsFeature : A prerequisite check for the Hyper-V feature failed.Hyper-V cannot be installed: The processor does not have required virtualization capabilities.Please advise.FollowComment"

Answer: "0Hello,On Amazon EC2 virtualised instances, nested virtualisation is not available. However, you can use Windows Server Containers without Hyper-V isolation. To use those, you can continue your installation with ignoring this error code.If you need to use a Windows Server container with Hyper-V isolation specifically, you'd have to use a bare metal instance type. That'd will allow you to enable the Hyper-V feature set.CommentSharemediouni-manswered 2 months agorePost-User-1955331 2 months agoThanks for the reply, but i need to run Linux and windows containers together, is that possible without the  nested virtualization?Sharemediouni-m 2 months agoIf you need to run Linux and Windows containers on the same Windows host, then you can use WSL2, which does require Hyper-V.Note that Microsoft only supports WSL2 on Windows Server for development and testing purposes: https://techcommunity.microsoft.com/t5/itops-talk-blog/using-wsl-2-on-windows-server-2022-to-run-linux-containers/ba-p/3624745 . As such, that configuration is not endorsed for production.You might want to consider running Linux containers on a separate instance, notably through Amazon ECS instead.Share"


Question: Json Playlist as Medialive inputs

"Hello Team.I'm looking to build our music channel using MediaLive but as inputs, we need to use a JSON playlist URL because we don't want to use OBS or software like it as input resource. Can you let me know the way to build that solution?FollowCommentAWS-User-7272408 a year agoCan anyone give me an idea?Share"


Question: Kinesis return wrong error when using PRODUCER_TIMESTAMP for end timestamp that is outside of retention period

"** I'm trying to retrieve a video stream from Kinesis that is outside of the retention period** either by using  KinesisVideoArchivedMedia or GetClip or event through the Kinesis download media clip UI (which uses the getClip API).when I'm using SERVER_TIMESTAMP I get the correct error message which is: InvalidArgumentException End timestamp XXXXXXXXXXX is outside of the stream retention period.However when I'm trying to get the same stream using PRODUCER_TIMESTAMP I get the wrong error: ResourceNotFoundException No fragments found in the stream for the clip request.although the video existed before it reached the retention period, I wish to get InvalidArgumentException error so I could present a proper error message to my clients. when using PRODUCER_TIMESTAMP I have no way to tell whether the video never existed or it is reached the retention period.Is there any other way to solve this issue and continue using PRODUCER_TIMESTAMPThank you.FollowComment"


Question: Emergent Snapshot Request: Databases found to still be awaiting snapshot.

RDS event / Log showsEmergent Snapshot Request: Databases found to still be awaiting snapshot.FollowCommentPanneer Thangavel a year agoRDS status shows backupShare

Answer: "0Hello,I understand the DB instance shows as backup and from the RDS event, the message "Emergent Snapshot Request: Databases found to still be awaiting snapshot" is generated.The Emergent Snapshot Request are automated system snapshots which will be initiated by the RDS in order to provide up-to-date snapshot data for point in time restore actions that might be requested.  These snapshots will be requested by the automated health monitors we have in place when for example:An instance fail-over was initiated, upon completion of such a fail-over a new snapshot will be taken.A restore of a database was performed, which replaced an existing database.Recovery model of a database has changedIf native backup and restore is being performed on the DB Instance, it is expected that  Emergent Snapshot will be initiated after the native backup and restore. If the DB Instance status still shows as backup, I would advise creating a support case to further investigate on the DB Instance .CommentShareSUPPORT ENGINEERSomto_Manswered a year ago"


Question: Transit gateway & VPC peering - IP forwarding

A customer has the below set-up:Branch Subnets----(Layer 2)----> Data Center Subnets ------(VPN Tunnel via AWS Transit Gateway)--------->Transit VPC(AWS)------(VPC Peering)-----> Citrix VPC(AWS)-----(VPC Peering)----> Prod VPC(AWS)Some constraints:Prod VPC and Data Center Subnets have overlapping CIDRs as few servers have IP address affinity.Direct VPN connectivity between branches and Citrix VPC would not be possible immediatelyProblem - They need to get printing working from Citrix VPC EC2 instances to the printers in Branches. What would be recommended implementation approach for IP forwarding for printing to work knowing the above constraints?FollowComment

Answer: "0Accepted AnswerA few observations:Transitive Routing: As VPC Peering doesn't support transitive routing, the path from the Citrix VPC to the Transit VPC over VPC Peering cannot be used to establish connectivity. Unfortunately you comment that "2. Direct VPN connectivity between branches and Citrix VPC would not be possible immediately" rules out one solution approach. Instead you might want to look into connecting the Citrix VPC to the TGW to have a path between Citrix VPC and branches.Overlapping CIDRs: You need to clarify what "Prod VPC and Data Center Subnets have overlapping CIDRs" means here. If a host in the Citrix VPC needs to talk to endpoints in the Prod VPC and Data Center Subnet under the same IP address, you'll need to look into NAT. If not, more specific routes might be sufficient.CommentShareEXPERTChrisE-AWSanswered 3 years ago"


Question: EC2 Instances not connecting to cli or Putty agent

"hello! i have a free tier AWS account for a year almost, for learning AWS.my account is active, but i cant connect to any instances from any AWS services.i also cant find any means to contact aws customer service or support.any Help, please!thanks!FollowComment"

Answer: "1Do you have an error messages that you can provide? Without that, there are many reasons that you can't connect. Check that your DNS is working; that you can connect on port 22 to some other target; and then on port 22 to your instance. Is the instance running? Have you tried restarting it?To contact support, there is a link in the console at the top-right of the window (click on the question mark): https://support.console.aws.amazon.com/CommentShareEXPERTBrettski-AWSanswered 9 months agoEXPERTChris_Greviewed 9 months ago0thanks Bret!i did but the issue is the same.since i have the free tier, i can't contact any technical support on AWS.but if there is a way to get technical support, unknown to me, any assistance helps.thanks!CommentSharerePost-User-8562115answered 9 months agoBrettski-AWS EXPERT9 months agoThis forum is best-effort and we're all volunteers here. We also have no visibility into your AWS account so we have to rely on what you tell us in order to troubleshoot. When you say "I did" - what did you do? What was the result? What error messages are you getting?Share"


Question: How to host an multi region api in aws?

"I want to host an REST based Api in multi region. to do so, I am assuming i will have to create resources in each region. like api gateway, lambdas or anything related to the api.  i want to create a new domain and based on the traffic or health check i can route the request to each regions api gateway endpoint. also for the security purposes, i need to create certain firewall rules. what resources do i need to create, like a domain name i suppose and firewall rules, any load balancing and besides domain name, what resources would be global that i need to create in each region?FollowComment"

Answer: "2Thanks for checking. Some of these resources and reference architecture will provide good background from a serverless standpointhttps://docs.aws.amazon.com/architecture-diagrams/latest/multi-region-api-gateway-with-cloudfront/multi-region-api-gateway-with-cloudfront.htmlhttps://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/aws-reference-architecture-multi-region-cloudfront-api-gateway-ra.pdfThis is a good re:Invent presentation : https://d1.awsstatic.com/events/reinvent/2019/REPEAT_1_Best_practices_for_building_multi-region,_active-active_serverless_applications_SVS337-R1.pdfhttps://aws.amazon.com/solutions/implementations/multi-region-application-architecture/These are also some good references you can referCreating a Multi-Region Application with AWS Serviceshttps://aws.amazon.com/blogs/architecture/creating-a-multi-region-application-with-aws-services-part-1-compute-and-security/https://aws.amazon.com/blogs/architecture/creating-a-multi-region-application-with-aws-services-part-2-data-and-replication/https://aws.amazon.com/blogs/architecture/creating-a-multi-region-application-with-aws-services-part-3-application-management-and-monitoring/Security specific areas that can be referredhttps://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/SecurityAndPrivateContent.htmlhttps://docs.aws.amazon.com/apigateway/latest/developerguide/security-best-practices.htmlhttps://docs.aws.amazon.com/whitepapers/latest/best-practices-api-gateway-private-apis-integration/best-practices-api-gateway-private-apis-integration.pdfCommentShareEXPERTAWS-User-Nitinanswered 8 months ago1You have some options to implement what you are trying to.If you want to implement the API in just one region but you have API consumers in other regions, you can use an Edge Optimized API Gateway end-point. That adds a Cloudfront distribution in front of the API Gateway, thereby ensuring that clients can get into the AWS network from the POP nearest to them, thereby reducing the latency of API calls. - https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-api-endpoint-types.htmlIf the Edge Optimized API Gateway does not work for you, you will need to look at implementing multi-region Regional API Gateway endpoints. You will then have to either shard your API consumers, so every region has to maintain data only for its own local consumers, or if that does not work for you, then you will have to replicate the data across all the regions. You also have to make sure that your Infrastructure as Code and CI/CD pipelines deploy your infrastructure and code in every region whenever changes are needed.If you go with the sharding kind of approach, then every region can have its own regional end-point which is exposed to the consumers of that regional endpoint only.If you don't want to shard consumers but want them to be able to connect to any of the regional end-points, then you can use one of two approaches - Cloudfront Origin Failover or Route53 routing policiesCloudfront origin failover can work if you have two regions, a primary and a secondary. It however works only for GET, HEAD and OPTIONS and does not support POST or PUT requests etc. - https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/high_availability_origin_failover.htmlWith Route53 routing policies, you can choose Geolocation routing or Geoproximity routing or Latency routing policies - https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.htmlCommentShareEXPERTIndranil Banerjee AWSanswered 8 months ago"


Question: Why can't I install python logging library on Linux2 instance

"Just started a new instance to run my python3 script. I need several libraries which I can install with pip3 (pip3 install requests runs well) but I can't get logging library installed.I have this output:$ pip3 install loggingDefaulting to user installation because normal site-packages is not writeableCollecting logging  Using cached logging-0.4.9.6.tar.gz (96 kB)    ERROR: Command errored out with exit status 1:     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_n141mbi/logging/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_n141mbi/logging/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-pxlnpi1y         cwd: /tmp/pip-install-_n141mbi/logging/    Complete output (48 lines):    running egg_info    creating /tmp/pip-pip-egg-info-pxlnpi1y/logging.egg-info    writing /tmp/pip-pip-egg-info-pxlnpi1y/logging.egg-info/PKG-INFO    writing dependency_links to /tmp/pip-pip-egg-info-pxlnpi1y/logging.egg-info/dependency_links.txt    writing top-level names to /tmp/pip-pip-egg-info-pxlnpi1y/logging.egg-info/top_level.txt    writing manifest file '/tmp/pip-pip-egg-info-pxlnpi1y/logging.egg-info/SOURCES.txt'    Traceback (most recent call last):      File "<string>", line 1, in <module>      File "/tmp/pip-install-_n141mbi/logging/setup.py", line 13, in <module>        packages = ["logging"],      File "/usr/lib64/python3.7/distutils/core.py", line 148, in setup        dist.run_commands()      File "/usr/lib64/python3.7/distutils/dist.py", line 966, in run_commands        self.run_command(cmd)      File "/usr/lib64/python3.7/distutils/dist.py", line 985, in run_command        cmd_obj.run()      File "/usr/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 297, in run        self.find_sources()      File "/usr/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 304, in find_sources        mm.run()      File "/usr/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 535, in run        self.add_defaults()      File "/usr/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 571, in add_defaults        sdist.add_defaults(self)      File "/usr/lib64/python3.7/distutils/command/sdist.py", line 226, in add_defaults        self._add_defaults_python()      File "/usr/lib/python3.7/site-packages/setuptools/command/sdist.py", line 135, in _add_defaults_python        build_py = self.get_finalized_command('build_py')      File "/usr/lib64/python3.7/distutils/cmd.py", line 298, in get_finalized_command        cmd_obj = self.distribution.get_command_obj(command, create)      File "/usr/lib64/python3.7/distutils/dist.py", line 857, in get_command_obj        klass = self.get_command_class(command)      File "/usr/lib/python3.7/site-packages/setuptools/dist.py", line 768, in get_command_class        self.cmdclass[command] = cmdclass = ep.load()      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2461, in load        return self.resolve()      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2467, in resolve        module = __import__(self.module_name, fromlist=['__name__'], level=0)      File "/usr/lib/python3.7/site-packages/setuptools/command/build_py.py", line 16, in <module>        from setuptools.lib2to3_ex import Mixin2to3      File "/usr/lib/python3.7/site-packages/setuptools/lib2to3_ex.py", line 13, in <module>        from lib2to3.refactor import RefactoringTool, get_fixers_from_package      File "/usr/lib64/python3.7/lib2to3/refactor.py", line 19, in <module>        import logging      File "/tmp/pip-install-_n141mbi/logging/logging/__init__.py", line 618        raise NotImplementedError, 'emit must be implemented '\                                 ^    SyntaxError: invalid syntax    ----------------------------------------ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.I can't understand why this happens.Anybody has an idea how to install logging ?ThanksFollowComment"

Answer: 1Accepted AnswerAccording to the python docs (https://docs.python.org/3.9/library/logging.html#module-logging) logging is part of the standard Python3 library. You shouldn't need to install it with pip.CommentShareChris_Panswered a year agoEXPERTBrettski-AWSreviewed a year agoAWS-User-3646434 a year agoMy mistake! Thanks :)Share


Question: How to connect my node.js backend application with aws elasticache redis cluster?

"I have a redis cluster after using that in my node.js backend code it is giving 'ENOTFOUND' error, my backend deployed in ECS docker.can anyone please help me on why i am getting this error?FollowCommentDakota Lewallen 5 months agoAre you using route 53 at any point in the setup? I ran into similar issues in the past where my ECS containers were using only the private hosted zone of my domain. So I had to duplicate my public records into my private zone and that resolved it.SharerePost-User-1977484 5 months agoCan you update your post with code, which shows the way that you are connecting to redis?SharerePost-User-7272412 5 months agoThis is my code on node.js;var redis = require("redis");////const redis = require('redis');// const client = redis.createClient({//  host:'redis endpoint url',//      port:6379,// });// client.on("error", function (error) {//  console.error("Error encountered:", error);// });// client.on("connect", function (data1) {//  console.log("Redis Connection Established");// })errors:Error : Error: Redis connection in broken state: connection timeout exceeded.Share"

Answer: "0connection timeout means a connectivity issue. You need to make sure that NACL and Security Group allows ECS to connect to the Redis.In case when your Redis and ECS are in different VPCs, you also need to configure a peering and routing between them.CommentShareOleksii Bebychanswered 5 months agorePost-User-7272412 5 months agoecs security group is added in inbound rule of redis security group, but still same issue is persiting, am i writing code wrong way?This is my code on node.js;var redis = require("redis");////const redis = require('redis');// const client = redis.createClient({//  host:'redis endpoint url',//      port:6379,// });// client.on("error", function (error) {//  console.error("Error encountered:", error);// });// client.on("connect", function (data1) {//  console.log("Redis Connection Established");// })error :errno: 'ENOTFOUND',Share"


Question: javav2: EBT worker hangs during connection establishment to a Dynamodb

"Is there anything that prevents communication between an EBT worker and a Dynamodb?I've been trying it the whole week, but without success...The setup looks like thisan EBT Web front-end accepts requests and records them in a DynamoDB. The corresponding unique ID (uid) is placed in an SQS queue. Here everything is fine.an EBT worker backend which receives the uid from the SQS. This works fine so far. Now the EBT worker should update the status in the DynamoDB. But I just can't manage to establish a connection to the DynamoDB. The needed env. vars in question are exported (access key and secret key)DynamoDbClientBuilder ddbBuilder = DynamoDbClient.builder(); // also tried with many other DynamoDbClientBuilders like Default, Instance etc. etc. etc.DynamoDbClient ddb = ddbBuilder.region(Region.EU_CENTRAL_1).build()... here it hangs forever. No exception, no null value return. Nothing at all. Just hangsBut if I do the same thing (on the same eb instance with the same webapp user account) on the console everything is fine, for example with ...eb sshsudo -u webapp bashaws dynamodb list-tables --region eu-central-1this shows the table in question. And I can also access the values belonging to the uid as expected. In other words: the needed credentials are there and they are valid.I'm really running out of ideas of now...Can anyone help here and give me a hint? I would be grateful even if someone only can tell this is simply not possible at the moment, for example because of a known bug.thank you - HansEdited by: der-Hans-der-kanns on Oct 7, 2020 2:14 PMEdited by: der-Hans-der-kanns on Oct 7, 2020 2:17 PMFollowComment"

Answer: 0fixed by a newer version of the awssdk (updated from v2.10.86 to 2.15.5)and here is a dbclient builder code snippet I have successfully used in the end ...AwsCredentialsProvider credentialsProvider = DefaultCredentialsProvider.create();  DynamoDbClientBuilder ddbBuilder = DynamoDbClient.builder().region(EU_CENTRAL_1).credentialsProvider(credentialsProvider);              DynamoDbClient ddb = ddbBuilder.region(Region.EU_CENTRAL_1).credentialsProvider(credentialsProvider).build();CommentShareder-Hans-der-kannsanswered 3 years ago


Question: Data Wrangler: Data Flow: Export to S3 using Jupyter Notebook

"When I have created a Data flow using data wrangler and when I am trying to Export to export to S3 using Jupyter Notebook and when I am running the notebook, I am getting the below mentioned error every time when creating a processing job:Error:  An error occurred (ResourceLimitExceeded) when calling the CreateProcessingJob operation: The account-level service limit 'ml.m5.4xlarge for processing job usage' is 0 Instances, with current utilization of 0 Instances and a request delta of 2 Instances. Please contact AWS support to request an increase for this limit.Please provide me with the solution for this. I have increased the service quota for running apps and Notebook instance also but then also same issue arises.FollowComment"

Answer: "1When you export a data flow to S3, you're starting a SageMaker processing job that will run the script and store data in S3.So, increase your account limit for Processing Job instances, for the instance type ml.m5.4xlarge (instead of running apps/notebook instances).CommentShareDurga_Sanswered 8 months agoEXPERTAlex_Treviewed 8 months ago"


Question: How can I prevent an Amazon SQS message from invoking my Lambda function more than once?

"I configured my AWS Lambda function to process messages in an Amazon Simple Queue Service (Amazon SQS) queue. Now, some of my valid Amazon SQS messages are received multiple times, up to the maxReceiveCount. How do I stop duplicate Lambda function invocations for the same Amazon SQS message?FollowCommentAWS-User-9438600 a year agoIs your sqs FIFO or regular sqs?Share"

Answer: "6There may be different reasons for this:First, SQS standard queues are "at least once delivery" so it may be that the same message will be delivered more than once. It is important to have your processing to be idempotent so that further invocations will not have bad affect on the system.Another option might be a short visibility timeout. If the visibility timeout is too short, the message may become visible again before the function finishes handling it. The recommended value for the visibility timeout is 6 * function duration.It may also be that your function ended with an exception. In this case the messages will not be deleted from the queue and will be delivered again.CommentShareEXPERTUrianswered a year ago0Short descriptionLambda supports at-least-once message delivery. In some cases, the retry mechanism might send duplicates of the same message. Amazon SQS then sends the messages to your dead-letter queue, if you've configured one.Duplicate Lambda invokes for the same Amazon SQS message can happen for one of the following reasons:Your function returns an error or times out.The Lambda service fails to delete a message from the Amazon SQS queue after a successful batch before the visibility timeout expires.The Lambda service sent the event to but failed to receive an acknowledgement from the function.An intermittent issue caused Amazon SQS to return the same message and it's polled again by the Lambda service.The sum of the batch window and the function duration is longer than your Amazon SQS queue visibility timeout. The SQS visibility timeout must be at least six times the total of the function timeout and the batch window timeout.To resolve these issues, turn on Report Batch item failure in your Lambda function's SQS trigger. Then, create a modular function code that iterates through the batch, processes, and deletes successful and duplicate messages. The function stores the messageID of the successful messages in a DynamoDB table and then verifies that the message was processed earlier.Note: Be sure to consider the cost for making API calls to Amazon DynamoDB.ResolutionCreate a DynamoDB tableOpen the DynamoDB console.Choose Create table.In the Create DynamoDB table screen, set the following values:For Table, enter ProcessedRecordsUnder Primary key, for Partition key, enter RecordsSet the data type to StringEnter other settings as needed for your use case. Then, choose Create.** Create a Lambda function**Important: The Lambda function code must be idempotent. For idempotency best practices and example function logic, see How do I make my Lambda function idempotent?After you have created a DynamoDB table, create a Lambda function.In your function:Add an execution role to allow the **dynamodb:Query **and **dynamodb:PutItem **actions.Report batch item failures (Using Lambda with Amazon SQS) in the SQS trigger to identify and skip duplicate messages in the batch.In the following example, the function logic converts the message body to upper case. It is written under process_message(...) method:import boto3dynamodb_client = boto3.client('dynamodb')DDB_TABLE = 'ProcessedRecords'# Validates if the message is already processed in previous invokes.# @input string message_id## @param message_id used to query the message from DynamoDB# @return Booleandef is_duplicate_message(message_id):    return dynamodb_client.query(        TableName='ProcessedRecords',        Select='COUNT',        KeyConditionExpression='Records = :Records',        ExpressionAttributeValues={            ':Records': {'S': message_id}        }    )["Count"] != 0   # Processes the message body to upper case.   # @input string body   #   # @param body to be processed   # @return uppercase body   def process_message(body):       return body.upper()   # Put the message to the DynamoDB Table.   # @input string batch_item_success   #   # @param batch_item_success of the message to put.   # @return Boolean   def push_to_dynamoDB(batch_item_success):          for message_id in batch_item_success:              response = dynamodb_client.put_item(                               TableName = DDB_TABLE,                                Item={ 'Records': {'S':message_id}                                }                                                                      )          return True   # processor function iterating through messages in the event.   # @input dict Records   #   # @param Records to be processed   # @return Boolean   def iterate_records(Records):              batch_item_failures = []              batch_item_success = []              for record in Records:                         message_id = record["messageId"]                     print("Message: " + message_id)                     if is_duplicate_message(message_id):                               print("Message duplicate: " + message_id)                            continue                             try:                             process_message(record["body"])                             batch_item_success.append(message_id)                          except:                               batch_item_failures.append({"itemIdentifier": message_id})               push_to_dynamoDB(batch_item_success)              return batch_item_failures   def lambda_handler(event, context):           return iterate_records(event["Records"])Related informationWhy is my Lambda function retrying valid Amazon SQS messages and placing them in my dead-letter queue? CommentShareEXPERTChristine_Wanswered a year agoEXPERTValerie_Breviewed a year ago0Hi Christine,assuming you're not hitting the limits of an SQS fifo queue, that would be the easiest solution.One limit is that the fifo version only supports up to 300 API calls per second, per API method (SendMessage, ReceiveMessage, or DeleteMessage) [https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html].if you're able to use batch operations, you can exceed those limits.If you're eventually hitting the limits, you need to manually do a deduplication [https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagededuplicationid-property.html] but that requires some implementation effort.Kind regardsCommentShareTillanswered a year ago"


Question: EMR bootstrap script with pip numpy installation fails on r6+ instances

"I recently tested moving from r5 to r6 instance fleets for our PySpark script. It has a dependency on numpy and pandas that is installed via pip in a bootstrap script, along with a few other dependencies for communicating with s3:#!/bin/bash -xeecho "---------------------------------------------------------"echo "using python version:"python3 --versionecho "initial python packages (sudo python3 -m pip list):"sudo python3 -m pip listecho "---------------------------------------------------------"echo "install python3-dev development tools"sudo yum -y install python3-develecho "---------------------------------------------------------"echo "installing python dependencies"sudo python3 -m pip install -U pipecho "pip installed/updated"sudo python3 -m pip install -U setuptoolsecho "setuptools installed"sudo python3 -m pip install \    cloudpickle==1.6.0 \    boto3==1.21.7 \    fsspec==2022.2.0 \    s3fs==0.4.2echo "aws dependencies installed (boto3, cloudpickle, fsspec, s3fs)"# sudo python3 -m pip install \#     pandas==1.1.5 \#     numpy==1.16.5# echo "pandas + numpy installed"sudo python3 -m pip install pandas==1.2.5echo "pandas installed"echo "final python packages (sudo python3 -m pip list):"sudo python3 -m pip listThis runs without failure on r5 instances, and numpy is available in the python environment as expected.When allowing {r6, r6g) instance types, the bootstrap script fails with the following message:      _configtest.c:1:10: fatal error: Python.h: No such file or directory       #include <Python.h>                ^~~~~~~~~~      compilation terminated.      failure.      removing: _configtest.c _configtest.o      Traceback (most recent call last):        File "<string>", line 36, in <module>        File "<pip-setuptools-caller>", line 34, in <module>        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/setup.py", line 419, in <module>          setup_package()        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/setup.py", line 411, in setup_package          setup(**metadata)        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/core.py", line 171, in setup          return old_setup(**new_attr)        File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 155, in setup          return distutils.core.setup(**attrs)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 148, in setup          return run_commands(dist)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 163, in run_commands          dist.run_commands()        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands          self.run_command(cmd)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command          cmd_obj.run()        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/command/install.py", line 62, in run          r = self.setuptools_run()        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/command/install.py", line 36, in setuptools_run          return distutils_install.run(self)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/command/install.py", line 670, in run          self.run_command('build')        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command          self.distribution.run_command(command)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command          cmd_obj.run()        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/command/build.py", line 47, in run          old_build.run(self)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/command/build.py", line 135, in run          self.run_command(cmd_name)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command          self.distribution.run_command(command)        File "/usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command          cmd_obj.run()        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/command/build_src.py", line 148, in run          self.build_sources()        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/command/build_src.py", line 165, in build_sources          self.build_extension_sources(ext)        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/command/build_src.py", line 322, in build_extension_sources          sources = self.generate_sources(sources, ext)        File "/mnt/tmp/pip-install-np4ypx_v/numpy_833906a79d1d4dfeb4789de3524fd268/numpy/distutils/command/build_src.py", line 375, in generate_sources          source = func(extension, build_dir)        File "numpy/core/setup.py", line 423, in generate_config_h          moredefs, ignored = cocache.check_types(config_cmd, ext, build_dir)        File "numpy/core/setup.py", line 47, in check_types          out = check_types(*a, **kw)        File "numpy/core/setup.py", line 281, in check_types          "install {0}-dev|{0}-devel.".format(python))      SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.      [end of output]    note: This error originates from a subprocess, and is likely not a problem with pip.error: legacy-install-failureÃ— Encountered error while trying to install package.â•°â”€> numpynote: This is an issue with the package mentioned above, not pip.hint: See above for output from the failure.Note, this bootstrap script attempts to address the problem from the error message by installing python-devel via yum before running the pip install.FollowComment"

Answer: "0I suspect you need arm64 version of numpy, probably might have to build it as wheel file and then add it to the EMR clusterCommentShareAlex_Tanswered a year agoAWS-User-9649900 a year agoahhhhhhhhhh. good thinking, that makes sense. and, that solution is reasonable, but it also means that i would have to fully commit my instance fleet to the AMD architecture if it's coming in via my Spark --py-files (wheels) parameter.Share"


Question: Website down and don't know why

"Hi...running a WP site on AWS for 2 months, and it's now down.When I try to access via Chrome, I receive the followong message:"This site can’t be reached xxxxxxxx.com took too long to respond.Try:Checking the connectionChecking the proxy and the firewallERR_CONNECTION_TIMED_OUT"I've rebooted the site to no avail.Please let me know if you can help. Thanks!JonFollowComment"

Answer: "0Are you running this on EC2 or Lightsail? And when you rebooted the site was that just a restart, or an actual stop & then start? If it was the latter then the public IP address & its DNS name will almost certainly have changed.Which address(es) does your security group (instance firewall for Lightsail) allow inbound access on ports 22, 80 & 443? If it was originally setup to be limited to your IP then has that changed - depending on your ISP your home IP address can change periodically, and obviously your IP is different at home/at work/on VPN/on cellular. Try opening it up to 0.0.0.0/0 just while you're troubleshooting, to see if that helps.If you can connect to it alright but the site if still going offline then look at resource usage. There will be a limited amount of information in AWS Console, to get more details information look at installing CloudWatch Agent on EC2 or Lightsail.CommentShareRWCanswered 13 days agoComedy Writing for Revenue Teams 13 days agoThanks for your reply.  This was part - but not the root cause of - my problem.Share0Hello,It sounds like you may have load exceeding the sustainable capacity of your instance.  Lightsail instances do allow periods of burst capacity, but if the instance remains in that higher load zone for too long it will reduce performance, potentially causing the issues you describe.  Please see if this documentation helps you confirm or rule out bursting as your issue.Bursting documentation: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-viewing-instance-burst-capacityIf bursting is the cause, your best bet is likely increasing the instance size.  This guidance explains the process of creating a snapshot of your existing instance and then using the snapshot to create a larger one.  https://lightsail.aws.amazon.com/ls/docs/en_us/articles/how-to-create-larger-instance-from-snapshot-using-consoleCommentShareAWS-Andyanswered 13 days ago"


Question: Unable to resolve "client intended to send too large body" using extensions

"Hi folks,I have a dockerized Django app running on Elastic Beanstalk environment that allows users to upload images. I'm seeing a bunch of "client intended to send too large body" errors on image upload. I looked around on the internet and learned that this issue is caused by Nginx and the solution is to increase the client_max_body_size. Since Nginx is managed by Beanstalk, I am trying to update the limit via .ebextensions. I'm using Amazon Linux AMI and hence I did the following:- Created a directory structure as follow: .platform/nginx/conf.d/elasticbeanstalk- Created a file 01_body_size.conf with the content below:client_max_body_size 10M;I bundled up the .platform directory with my Dockerrn.aws.json, I can see the file being copied to /etc/nginx/conf.d/elasticbeanstalk/ on the EC2 instance managed by Beanstalk. Given the following entry in /etc/nginx/nginx.conf- include conf.d/elasticbeanstalk/*.conf;everything should work but it isn't and I'm still seeing the error in /var/log/nginx/error.log complaining that it got a request that was too large from the loadbalancer. I think I'm missing some small details and would love some help to figure this out.FollowComment"

Answer: 0This is resolved.CommentShareNishant-showflooranswered 3 years ago


Question: AWS SES - authentication failure with vadesecure / open-exchange

"Hello,On 17th Nov we started to get authentication failure reports (DMARC) from recipients / customers which are using services of vadesecure / open-exchange.I think open-exchange is mail provider and vadesecure provide some technology for mail filtration.We have DKIM records for aws ses domain and DMARC policy which state that failed authentication should got to junk. (soft)It was working fine until 17th November. We did not have any recent changes to mail content, domain or aws ses configuration.We contacted customer (using domain pointed to open-exchange MX servers) which is not tech savy.Because that emails are core of product we are offering their failure is not great.On 21st i send email to open-exchange but there is no answer from them.Today i contacted vadesecure but their support said that abuse team is processing only requests from their online form WHICH require to verify ip address of sender e.g. (AWS SES IP) which is obviously not possible.We do not have any paid support from AWS and it was never required for anything for far.We only get report like that:Feedback-Type: auth-failureUser-Agent: mtabuilder/1.0Version: 1Original-Mail-From: XXXXXXX0@amazonses.comOriginal-Envelope-Id: XXXXXXXXXXXXXXAuthentication-Results: oxsus-vadesecure.net; dmarc=fail (p=quarantine) header.from=XXXXXX <messages@XXXXXX.com>Auth-Failure:Arrival-Date: Sat, 19 Nov 2022 17:50:01 +0000Source-IP: 54.240.8.195Reported-Domain: XXXXReceived: from a8-195.smtp-out.amazonses.com ([54.240.8.195]) by oxsus1nmtai02p.internal.vadesecure.com with ngmta id a6942fe6-17290db34cc8b7bf; Sat, 19 Nov 2022 17:50:00 +0000On 18th i got similar reports for different product which is using infusion soft for marketing emails but again from vadesecure. there we have DKIM authentication for domains too.We do not have issues with any other major mailbox provider (gmail, microsoft, yahoo and etc).Having different providers failing because of vadesecure filtering indicate that probably vadesecure has something special .. which is more strict than most of the world.DMARC policy which we have for all domains isv=DMARC1; p=quarantine; sp=quarantine; pct=100; adkim=r; aspf=r; rua=mailto:XXXXX; ruf=mailto:XXXXX; fo=1;Any advices are appreciated.FollowComment"

Answer: "0Any update? I have started to receive the same messages for our outbound emails; also no changes in our MX config, and we've had a 100% score on mxtoolbox for over a year. Were you able to resolve this?CommentShareAppCollScottanswered 13 days ago"


Question: SSM Document Timeout in CDK

"I am writing an automation for systems manager using CDK.I would like to call a Systems Manager Run Command when a CloudWatch alarm enters the alarm state, via EventBridge.I was able to achieve this using the CfnDocument and CfnRule constructs in CDK.However, my Run Command shell script is long running, and can time out after 3600 seconds.Is there any way that I can increase the timeout of the document when invoking from EventBridge?I know you can increase the timeout when calling Run Command from the console, but I need this to be automated. I've also tried adding 'timeoutSeconds' to the long running steps in my SSM runbook, but it still times out after 3600 s of execution.FollowCommentAWS-User-2604211 10 months agoI think I figured it out. In your ssm document, in the mainsteps section, you add the timeoutSeconds parameter to the inputs of the runcommand, like so:mainSteps:- name: LongScript  action: aws:runShellScript  inputs:  - runCommand: "sleep 10000"    timeoutSeconds: 11000Share"


Question: Billing - AWS Landing Zone has SPP discounts?

May I know if the SPP discounts are available on the  AWS landing zone? Can you elaborate on their differences?Thanks!FollowComment

Answer: "0Hello there,I understand you would like to know if AWS Landing Zone has discounts for Solution Provider Program.Please note that you may contact your Account Manager regarding queries on billing and discounts.CommentSharerePost-User-5157190answered 2 months ago"


Question: Best way to have a snapshot-based QuickSight histogram

"I have a cron job that exports a list of number of messages per user frequencies such as:10 * 15 * 28 * 3(Which means: 8 users have posted 3 messages)I'd like to have a histogram chart which displays last snapshot's data (ie. bar, mean, avg).I was thinking of TimeStream, but it actually accumulate.What is the recommended storage for this usecase?FollowComment"


Question: Running short-lived tasks on AWS Lightsail Containers

"Hello!I wish to run some short lived tasks on AWS Lightsail Containers. The container boots, performs some work which may take anywhere from a handful of seconds to serveral hours, and then it exits once complete.Is AWS Lightsail Containers suitable for this workload? In the documentation I got the impression that it was intended for long-lived HTTP services that do not terminate.Thanks,LouisFollowComment"

Answer: "0AWS App Runner is more suitable for the use case you specified.CommentShareJason_Sanswered a year agomreferre EXPERTa year agoI am not sure App Runner would be the best choice for this. It sounds more like a batch-y type of workload and App Runner doesn't work like that (it's a strict request/response service where if there is no explicit http request it just "shuts down" - so to speak)ShareAWS-User-0928316 a year agoThanks for the comment Jason, but I'm not looking for recommendations, I'm looking for information on specifically Lightsail. Thank youShare0Really it comes down to pricing. With Lightsail you are paying month-to-month regardless of usage. If you used something serverless like Fargate, or as Jason mentioned, AppRunner, you are paying by usage.CommentShareJohn Voldanswered a year agoAWS-User-0928316 a year agoHi John. I'm not interested in pricing here, I'm looking for information on whether Lightsail Containers supports workloads that are intended to exit. i.e. Is it possible to have Lightsail not restart a container once it exits successfully as that would in this case be the intended behaviour. Thank youShareJohn Vold a year ago"Is AWS Lightsail Containers suitable for this workload?" - Yes."In the documentation I got the impression that it was intended for long-lived HTTP services that do not terminate." - No?https://aws.amazon.com/blogs/aws/lightsail-containers-an-easy-way-to-run-your-containers-in-the-cloud/"Under the hood, Lightsail is powered by Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (RDS), Application Load Balancer, and other AWS services..." - Lightsail is just an added administration layer on top of existing services.Share0Hi -To directly address your question on Lightsail, generally when we approach the issue, it's two factors that is important (regardless of solutions you choose), among other things (like security, flexibility and access to other AWS resources).cost. Lightsail is a Virtual Private Server. What is means is you pay a fixed price for the amount of resources you provision regardless of your node utilization. In your case this is very important. It's a moot question to ask whether Lightsail is suitible for your use case because Llightsail for sure will be suitable for the usecase as long as you provisioned enough nodes to handle peak load. However generally if you consistently have low node utilization, you are leaving money on the table so you might want to visit other services.  Sorry I might not have elaborated more in detail in my previous answer but additional services such as AWS Batch (with Fargate) or even a queue based solution might be a better fit and deliver better cost structure for what you need.scalability. Almost all AWS solutions can be architected to scale. However, serverless solutions such as Fargate will make the tasks simpler.CommentShareJason_Sanswered a year ago"


Question: Upgrade ecs-agent on Windows 2016 ECS Optimized AM

"I'm using the Windows 2016 ECS Optimized AMI, my current EC2 hosts displays a warning that the latest ecs-agent is not installed on the host.  Recommended action is to spin up a new EC2 host to have the latest version installed.I performed this action today and the ecs-agent is the same version as the previous EC2 host.Is there a method to upgrade to the latest ecs-agent?  I'm receiving an issue while attempting to use gMSA and I would like to rule out the ecs-agent version as the cause.FollowComment"

Answer: 0migrated to the 2019 ECS Optimized in order to upgrade to latest ECS Agent and required GMSA abilityCommentSharentxarchanswered 3 years ago


Question: Amazon linux 2 on-prem in vmware network doesn't configure it self.

"I want to test amazon linux 2 in our on-premise vmware environment. I was able to deploy the ovf with the seed iso mounted, it did read the files in the iso, as it changed the password of the ec2-user, and applied some other configs. But network config isn't applied.I added a network-config file also, and it's beeing read, as per the logs, but not applied.I see this in the could-init.log :Mar 12 14:08:18 cloud-init[4512]: stages.py[DEBUG]: applying net config names for {'version': 1, 'config': [{'subnets': [{'netmask': '255.255.255.240', 'address': '192.168.128.6', 'type': 'static', 'dns_nameservers': ['192.168.126.3', '192.168.126.4'], 'gateway': '192.168.128.1', 'dns_search': ['domain.com']}], 'type': 'physical', 'name': 'eth0'}]}Mar 12 14:08:18 cloud-init[4512]: __init__.py[DEBUG]: no interfaces to renameMar 12 14:08:18 cloud-init[4512]: stages.py[INFO]: Applying network configuration from ds bringup=False: {'version': 1, 'config': [{'subnets': [{'netmask': '255.255.255.240', 'address': '192.168.128.6', 'type': 'static', 'dns_nameservers': ['192.168.126.3', '192.168.126.4'], 'gateway': '192.168.128.1', 'dns_search': ['domain.com']}], 'type': 'physical', 'name': 'eth0'}]}Mar 12 14:08:18 cloud-init[4512]: util.py[DEBUG]: Reading from /etc/os-release (quiet=False)Mar 12 14:08:18 cloud-init[4512]: util.py[DEBUG]: Read 212 bytes from /etc/os-releaseMar 12 14:08:18 cloud-init[4512]: stages.py[ERROR]: Unable to render networking. Network config is likely broken: No available network renderers found. Searched through list: ['eni', 'sysconfig', 'netplan']Mar 12 14:08:18 cloud-init[4512]: main.py[DEBUG]: [local] Exiting. datasource DataSourceNoCloud [seed=/dev/sr0][dsmode=net] not in local mode.My meta-data file :local-hostname: amzlinux-testnetwork-interfaces: |  auto eth0  iface eth0 inet static  address 192.168.128.6  network 192.168.128.0  netmask 255.255.255.240  broadcast 192.168.128.15  gateway 192.168.128.1my network-config file :version: 1config:  - type: physical    name: eth0    subnets:      - type: static        address: 192.168.128.6        netmask: 255.255.255.240        gateway: 192.168.128.1        dns_nameservers:          - 192.168.126.3          - 192.168.126.4        dns_search:          - domain.comI was able to manually configure the network and get into the machine, but still I'd like to be able to seed the config at first boot.Edited by: rossnick on Mar 12, 2020 8:18 AMEdited by: rossnick on Mar 12, 2020 8:19 AMFollowCommentRon 6 months agoHas there been any updates yet?Share"

Answer: "0This used to work for me fine, but is broken now using thehttps://cdn.amazonlinux.com/os-images/2.0.20200304.0/hyperv/amzn2-hyperv-2.0.20200304.0-x86_64.xfs.gpt.vhdx.zipimage. Seems like a regression somewhere. Just following the instructions here:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/amazon-linux-2-virtual-machine.htmlresults in a network interface that is not configured and tries to use dhcp. I get the same error messages in /var/log/messages.Edited by: wynnw on Mar 16, 2020 11:39 AMEdited by: wynnw on Mar 16, 2020 12:07 PMCommentSharewynnwanswered 3 years ago0Thanks, that was the instructions I followed. The meta-data is followed, at least a little bit, since the hostname is set, and the user-data also, since the password is set for the ec2-user.Let's hope it'll resolve in a future releaseCommentSharerossnickanswered 3 years ago0I was having the same problem with amzn2-vmware_esx-2.0.20200722.0-x86_64.xfs.gpt.ova, but when I used the same seed iso with amzn2-vmware_esx-2.0.20190612-x86_64.xfs.gpt.ova it worked perfectly.Failed attempt with 20200722cloud-init.log:Sep 04 21:45:03 cloud-init[6391]: stages.py[INFO]: Applying network configuration from ds bringup=False: {'version': 1, 'config': [{'subnets': [{'control': 'auto', 'dns_nameservers': ['10.2.5.21', '9.9.9.9'], '_orig_eni_name': 'eth0', 'broadcast': '10.2.5.255', 'netmask': '255.255.255.0', 'address': '10.2.5.77', 'type': 'static', 'gateway': '10.2.5.21'}], 'type': 'physical', 'name': 'eth0'}]}cloud-init.log:Sep 04 21:45:03 cloud-init[6391]: stages.py[ERROR]: Unable to render networking. Network config is likely broken: No available network renderers found. Searched through list: ['eni', 'sysconfig', 'netplan']Successful attempt with 20190612cloud-init.log:Sep 04 21:55:04 cloud-init[6405]: stages.py[INFO]: Applying network configuration from ds bringup=False: {'version': 1, 'config': [{'subnets': [{'control': 'auto', 'dns_nameservers': ['10.2.5.21', '9.9.9.9'], '_orig_eni_name': 'eth0', 'broadcast': '10.2.5.255', 'netmask': '255.255.255.0', 'address': '10.2.5.77', 'type': 'static', 'gateway': '10.2.5.21'}], 'type': 'physical', 'name': 'eth0'}]}cloud-init.log:Sep 04 21:55:04 cloud-init[6405]: util.py[DEBUG]: Writing to /etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [644] 249 bytesCommentShareAWS-User-1515530answered 3 years ago0What's mentioned here is a real issue in cloud-init, see below for a possible fix.https://forums.aws.amazon.com/thread.jspa?threadID=330409&tstart=0CommentShareadamkonrad-bgsanswered 3 years ago0This is sadly still broken with the latest 2.0.20210126.0 release.The documentation in the README.cloud-init is broken, as is the document here:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/amazon-linux-2-virtual-machine.htmlThe linked reply with a diff to fix cloud-init does look like it would work, but there's no way to patch this at boot time that I can see. I'm really stuck trying to figure out how to work around this, and I'm still stuck loading a two year old image and then upgrading it. Really frustrating - I've tried all sorts of yaml config variants for the seed.iso with no luck.CommentSharewynnwanswered 2 years ago0And finally it's fixed in 2.0.20210721.2.Make sure to use dns_nameservers and dns_search, not dns-nameservers/dns-search.CommentSharewynnwanswered 2 years ago"


Question: How to integrate WAF with an EC2 instance?

"I was looking into using AWS's Web Application Firewall, but it can only be used by an elastic load balancer or a CloudFront distribution. How might I go about implementing a WAF with my EC2 that is serving my website?FollowComment"

Answer: "0Accepted AnswerNot supported directly, currently needs to be behind ALB or CloudFront.  Here is a tutorial that walks through protecting EC2 instances.https://docs.aws.amazon.com/waf/latest/developerguide/tutorials-ddos-cross-service.htmlCommentShareStarmananswered 5 years ago"


Question: Codepipeline deployed by CDK fails on the first run

"Hi, I am writing CodeCommit and CodePipeline project with CDK.But I am facing an issue:When the pipeline is created for the first time, it starts automatically and always fails at the build stage with an error.If I restart the pipeline from the Management Console, it runs without any errors.First run -> fail at the build stageRestart pipepine manually -> build successThe error message isError calling startBuild: User: arn:aws:sts::111111111111:assumed-role/blabla-XXXXXXX/nnnnnn is not authorized to perform: codebuild:StartBuild on resource: arn:aws:codebuild:ap-northeast-1:111111111111:project/SampleBuildProject because no identity-based policy allows the codebuild:StartBuild action (Service: AWSCodeBuild; Status Code: 400; Error Code: AccessDeniedException; Request ID: xxx; Proxy: null)I confirmed that the role had 'startBuild' action.I think it happens because the pipeline deployment is still running when the first build runs.Is there any solution to avoid this?Thanks!FollowComment"


Question: Running NextJS on ECS + Fargate

"Hi! Would like to know if it is possible to deploy a NextJS app with SSR on AWS ECS + Fargate? After deploying I face a 500 server error, and googling returns a similar issue here: https://github.com/aws-amplify/amplify-js/issues/8049Would love to know if anyone has experience with this. Thanks!FollowCommentMohamed_B MODERATOR9 months agocan you check the container logs to see what's causing the 500 server error?Share"

Answer: "0Hello,The Amplify framework currently does not provide support for SSR with Next JS on ECS Fargate.Please track the github issue 8049 that you've posted in the question for future updates about the feature request.Please feel free to provide detailed explanation of your use-case and code-snippets on the Amplify project issues page to get direct advice from Amplify developers.CommentShareSUPPORT ENGINEERVenkat Penmetsaanswered 9 months ago"


Question: Unable to get notification for ACM Soon Expiring Cert

"Dear all,I need some helps on the topic.I input one 30 days self signed cert into acm.I also set sns topic and eventbridge and subscribed to the sns topic.But unable to get notifications from the rule trigger.FollowComment"

Answer: 0Have you set up this document?https://repost.aws/knowledge-center/acm-certificate-expirationPlease check the status of EventBridge and AWS Config to see if it is configured correctly.EventBridge also has matching rules and records them in CloudWatch metrics when an event is triggered.This can be seen from the EventBridge monitoring screen.CommentShareEXPERTRiku_Kobayashianswered 13 days ago


Question: update security custom header

"Hi team,I want to modify a custom header value in a CF origin with lambda. this lambda should read the value from the secret manager and update the value of the custom header in Cf accordingly.is this achievable via a lambda function?Thank you!FollowComment"

Answer: "1Hi,This is a use case which could be solved using Lambda@Edge. The blog post How to enhance Amazon CloudFront origin security with AWS WAF and AWS Secrets Manager demonstrates a similar scenario, adding a custom header with a value from Secrets Manager. One area to pay attention to is the selection of the CloudFront event used to trigger your Lambda function. Some events can be used to trigger the function on every request while others may only execute the function when CloudFront forwards a request to the origin, i.e. when the requested object is not in the CloudFront cache.CommentShareEdward_Sanswered a year ago"


Question: Docker: Stderr:fatal error: runtime: out of memory on AWS Elasticbeanstalk

"Each time I am trying to build and deploy my React Project on AWS Elastic Beanstalk using Docker, I am getting this error:Stderr:fatal error: runtime: out of memoryThe project is the basic react project and nothing more, serving it via nginx. What might be the reason of this error? And how do I resolve it. Please note, I have emptied my S3 bucket and have restarted the appServer too, to try and resolve any unnecessary memory or cache use. And I am able to successfully build and run this file locally. The logs are as follows:Step 1/9 : FROM node:alpine as builder ---> eb56d56623e5Step 2/9 : WORKDIR '/app' ---> Using cache ---> 41d7415dae07Step 3/9 : COPY package.json . ---> Using cache ---> 95877ba9972cStep 4/9 : RUN npm install ---> Using cache ---> c366758de80fStep 5/9 : COPY . . ---> 5697eace7031Step 6/9 : RUN npm run build ---> Running in 52cc1a3e6ac0> react-project-frontend@0.1.0 build> react-scripts --openssl-legacy-provider buildCreating an optimized production build...2022/03/03 13:14:45.346275 [WARN] failed to execute command: docker build -t aws_beanstalk/staging-app /var/app/staging/, retrying...2022/03/03 13:14:45.425369 [INFO] Running command /bin/sh -c docker build -t aws_beanstalk/staging-app /var/app/staging/2022/03/03 13:14:47.001398 [ERROR] An error occurred during execution of command [app-deploy] - [Docker Specific Build Application]. Stop running the command. Error: failed to build docker image: Command /bin/sh -c docker build -t aws_beanstalk/staging-app /var/app/staging/ failed with error exit status 2. Stderr:fatal error: runtime: out of memoryFollowComment"

Answer: "0Does node:alpine have npm installed by default? Maybe you need to install npm before running your npm install command?CommentShareJohn Voldanswered a year agoHarjyot Bagga a year agoYes. Node alpine has npm installed by default. Also, my docker file is successfully built on my local system.Share0Hello  From Step2 to Step 4 is using cache try to clear them and re-runCommentShareralagarasananswered a year ago"


Question: Inspector2 dashboard issue in the delegated admin account

"Inspector2 is enabled in our aws org for few accounts. in the delegated Inspector2 account the dashboard displays 0 critical and 0 total findings, which is not true as there are findings in other accounts. Also the AWS accounts with most critical findings, ECR repositories/container images with most critical findings lists are empty too. if I goto all findings, its all there. any ideas about why they are not shown in the dashboard?FollowComment"

Answer: "0anyone else with the same issue? ideas, hints?CommentSharefpganswered a year ago0all of a sudden it started working so I think it was a product issueCommentSharefpganswered a year ago"


Question: How could I avoid Billing?

"Hi,I Stopped the instances  2 months ago and I already get a billing for the service.  As long as I understand there is NO billing at least when the instance is stoppedThis are my instances and the current statusHow could I avoid the billing if I dont using this instances?Thank youFollowComment"

Answer: "2Hello,You are billed for the EBS volume storage that you are using when the instances are stopped.  You are also billed for any elastic IPs that are being used.  See the link in the notice above the table in your screenshot: Avoiding Unexpected ChargesCommentShareEXPERTChris_Ganswered 9 months ago"


Question: Is there a way to query a Microsoft SQL Server programmatically in a python job shell in AWS Glue?

"We have been able to connect to a Microsoft SQL Server DB using both Glue's DynamicFrame and Spark's own JDBC write option due to the Glue connection option. However, considering the transactional nature of the data, we wanted to switch this to sending data programmatically using a python library, thus allowing us to move from Glue ETL Job to Python Shell Job. Our initial option was pyodbc, however, due to our inability to integrate the required driver with Glue, we were unsuccessful. Another option that we looked at was pymssql. Our experience with connecting to Microsoft SQL server using pymssql was seamless. However, it was restricted to Python 3.6 and we were unable to import it with Python 3.9. Our requirement for Python 3.9 was due to the boto3 version compatibility with the said python version as we are also trying to access the boto3 redshift-data client in the same script. Having considered all of the above, is it possible to query Microsoft SQL Server using a python library with Python 3.9 in AWS Glue Python Shell Job?FollowComment"

Answer: "0HiYou can use the AWS SDK for pandas - these a tutorial on how to connect to MS SQL Server here: https://aws-sdk-pandas.readthedocs.io/en/stable/tutorials/007%20-%20Redshift%2C%20MySQL%2C%20PostgreSQL%2C%20SQL%20Server%2C%20Oracle.htmlHTHNickCommentShareNickanswered 3 months agoHamza Ehsan Khan 3 months agoHi Nick,Firstly thanks so much for responding.Secondly, when connecting to my SQL Server using wr.sqlserver.connect(), I get an error similar to when I used pyodbc: pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not found (0) (SQLDriverConnect)")Could you please guide further on this?I have used a glue catalog connection that we have used before to connect to server using spark.jdbc and glue dynamic frame.ShareNick 3 months agoThere are some specific instructions if you want to use the AWS SDK for pandas with MS SQL (https://aws-sdk-pandas.readthedocs.io/en/stable/install.html#notes-for-microsoft-sql-server) - its to keep the package size down for customers that don't choose this data source.Share"


Question: Auth.currentAuthenticatedUser() does not return Cognito User.

"Auth.currentAuthenticatedUser() only returns  google token like so:{    "id": "us-east-1:xxxxx",    "name": "xxx",    "token": "xxxxxx"}const googleToken = response.credential;    const googleData = parseToken(googleToken);    console.log('googleData:', googleData);    const { email, given_name, exp } = googleData;    Auth.federatedSignIn(      'google',      {        token: googleToken,        expires_at: exp      },      {        name: given_name      }    ).then((response) => {      console.log('federatedSignIn res', response);      // Use the Auth.currentAuthenticatedUser() method to check if the user is already authenticated      Auth.currentAuthenticatedUser()        .then((cognitoUser) => {          console.log('cognitoUser:', cognitoUser);          const cognitoToken =            cognitoUser.signInUserSession.accessToken.jwtToken;          // Use the Cognito token to authenticate requests to your backend API          console.log('cognitoToken:', cognitoToken);        })FollowComment"


Question: How to deliver DNS log from Route 53 to on-premises ELK

"Hi guys!I receive a requirement that needs to deliver public DNS query log from Route 53 to their existing ELK (Elasticsearch, Logstash, Kibana) located in on-premises. Although I recommended that they use CloudWatch Log, OpenSearch in AWS but they insist on leveraging their existing ELK, so I have to find solution to meet this requirement.If you know the answer, please help me.Thanks!FollowComment"

Answer: "0Accepted AnswerYou could configure your Route53 logs to be sent to a Kinesis Data Firehose delivery stream, with the destination of the stream set to your ELK setup. Since you have an on-prem ELK, you would have to use an HTTP endpoint to receive this data - https://docs.aws.amazon.com/firehose/latest/dev/create-destination.html#create-destination-httpCommentShareChamalanswered 6 months agoSteven 6 months agothank you so muchShare"


Question: "AWS Glue says it supports python 3.7 for AWS Glue 2.0 but in reality, only python 3.6 for Python Shell ELT"

"According AWS Glue release, AWS Glue says it supports python 3.7 for AWS Glue 2.0 but in reality, only python 3.6.Here's the steps that I did. I attached snapshots for additional reference.Open AWS Glue/ ELT/ JobsSelect Python Shell script editorCreate jobOnly Python 3.6 for Python Shell.https://docs.aws.amazon.com/glue/latest/dg/release-notes.htmlFollowCommentFabrizio@AWS EXPERT8 months agoPlease see my answer with the launch of Glue Python shell support for Python 3.9Share"

Answer: "1I found the answer, that python shell only supports 3.6. Reference link: https://repost.aws/questions/QU7EmDfBcESiWaFTxZQKaAow/glue-py-shell-python-versionCommentSharerePost-User-5408781answered 10 months ago0hello,We recently launched support for python 3.9 in Glue python shells and simplified the way it works with additional libraries.For detailed information please have a look at this blog post.hope this helpsCommentShareEXPERTFabrizio@AWSanswered 10 months ago"


Question: Reprocessing failed DDB streams

"We've been using DDB streams as an event source for lambda functions serving our analytics pipeline.Additionally, we configured the lambda to send the failed records metadata to a DLQ in case the processing fails after a given number of retries - by doing so, we get the event metadata rather than the actual failing records. Once we fix the underlying issue we currently have no easy way to reprocess the failing records.Looked at KCL but it requires Java to be installed. As we use Python, we find it somewhat strange to install Java just for the sake of getting the impacted DDB records. To me, the data that's sent to the DLQ should include the impacted records.Would appreciate any easy way to fetch the impacted records given the metadata sent in such a case.FollowComment"

Answer: "1Should be possible in Python using boto3.client('dynamodbstreams')First, you should get shard iterator using metadata of failed record(s) and then using that shard iterator you get records themselves.CommentShareA-Shevchenkoanswered 2 years agoohadbenita 2 years agoThanks for this answer, I was wondering whether there's an option using AWS native service to get this information. If no other answers come up I'll mark is as the answer.Share1Yes, you don't get the impacted records, but a shard iterator.You need to call get-shard-iterator, and then call getRecords to get the actual impacted records.In response to your questionWould appreciate any easy way to fetch the impacted records given the metadata sent in such a case.Two Reasons why this is not given by AWS out-of-the box:Reason1:SQS message size has a limit, your DDB record update size may exceed the message limit. Hence by delivering the shard information which is of fixed size, AWS internally didn't had to worry about actual message size limitations.Reason2:Above is the approach to get the impacted records. You might ask Why can't AWS DDB or lambda deliver the impacted records to DLQ?, to answer this, we need to understand what DDBTrigger and shard is ?Since, DDB trigger is providing you the realtime updates to table data using kinesis behind the scenes(shard mechanism), and if particular read from shard failed, then it's delivered to dlq and continue with next data in the shard.So within a shard, let's say your record Id=ABC was updated multiple times(v0,v1,v2,v3 be the versions), and if some udpate(v1) from DDB failed processing in your lambda, they are sidelined to DLQ. It's possible that next updated to same record Id=ABC, were succesffully processes later on by the lambda. Let's assume your lambda is writing to S3 (eventually overriding older versions). In this scenarios,v0 was written to S3, as lambda was able to process succesffulyv1's updates got delivered to dlq since lambda failedv2 was written to S3, as lambda was able to process succesffulyv3 was written to S3, as lambda was able to process succesffulyNow, you need a processor over the dlq (shard reader + getRecords) to read and identify whether you should write to S3 or not. In this case you should ignore and delete the dlq message, since v3 has been written by lambda later on.Since, lambda failed on one of the shard iterator, and your business logic in lambda can be doing anything with the data, it delivers that shard iterator to the DLQ and not the messages.Hope this information helps you in understanding the problem.Let me know if anything is unclear.CommentSharemtkanswered a year agosmosek a year agoThis is clear, but unfortunately it means you can't process messages after 24 hours have passed, because they're no longer in the stream. Right?Share1Would it work in your case to use a Lambda function to consume (and enrich) the records from the SQS dead letter queue? You could write the processing logic in Python and only activate the Lambda function once you have recovered from the incident and are ready to process the backlog from the SQS queue.https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.htmlCommentShareSteffen Hausmannanswered a year ago"


Question: aws patch manager baseline issue

"when I apply patches I get this erros  which I bold mark/usr/bin/python3/usr/bin/apt-getReading package lists...Building dependency tree...Reading state information...python3-apt is already the newest version (2.0.1ubuntu0.20.04.1).0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded.Using python binary: 'python3'Using Python Version: Python 3.9.1603/21/2023 06:13:45 root [INFO]: Downloading payload from https://s3.dualstack.ap-south-1.amazonaws.com/aws-ssm-ap-south-1/patchbaselineox/payloads/patch-baseline-operations-1.105.tar.gz03/21/2023 06:13:46 root [INFO]: Attempting to import entrance file os_selector03/21/2023 06:13:46 root [INFO]: Running with snapshot id =  and operation = Scan03/21/2023 06:13:46 botocore.credentials [INFO]: Found credentials in shared credentials file: ~/.aws/credentials03/21/2023 06:13:46 root [ERROR]: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009dab947dec531ad doesn't match the credentialsTraceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 126, in _get_snapshot_infopatch_snapshot = _get_snapshot_with_client(ssm_client, instance_id, snapshot_id, baseline_override)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 434, in _get_snapshot_with_clientreturn ssm_client.get_deployable_patch_snapshot_for_instance(File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 276, in _api_callreturn self._make_api_call(operation_name, kwargs)File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 586, in _make_api_callraise error_class(parsed_response, operation_name)botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009vbhjkab94bnmad doesn't match the credentials03/21/2023 06:13:46 root [INFO]: Unable to retrieve snapshot with default ssm client, retry with fallback ssm client03/21/2023 06:13:46 botocore.credentials [INFO]: Found credentials in shared credentials file: ~/.aws/credentials03/21/2023 06:13:46 root [ERROR]: Error loading entrance module.Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 126, in _get_snapshot_infopatch_snapshot = _get_snapshot_with_client(ssm_client, instance_id, snapshot_id, baseline_override)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 434, in _get_snapshot_with_clientreturn ssm_client.get_deployable_patch_snapshot_for_instance(File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 276, in _api_callreturn self._make_api_call(operation_name, kwargs)File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 586, in _make_api_callraise error_class(parsed_response, operation_name)botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009dab9ghjkl531ad doesn't match the credentialsDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 102, in _get_snapshot_info_with_fallback_ssm_clientpatch_snapshot = _get_snapshot_with_client(fallback_ssm_client, instance_id, snapshot_id, baseline_override)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 434, in _get_snapshot_with_clientreturn ssm_client.get_deployable_patch_snapshot_for_instance(File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 276, in _api_callreturn self._make_api_call(operation_name, kwargs)File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 586, in _make_api_callraise error_class(parsed_response, operation_name)botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009dab947dec531ad doesn't match the credentialsDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_startup_entrance.py", line 203, in executeexit(   entrance_module.execute(*argv))File "/var/log/amazon/ssm/patch-baseline-operations/os_selector.py", line 54, in executecommon_os_selector_methods.fetch_snapshot(operation_type, instance_id, region, reboot_option, document_step,File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 280, in fetch_snapshotsnapshot_info = _get_snapshot_info(instance_id, snapshot_id, region, baseline_override_dict)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 136, in _get_snapshot_inforeturn _get_snapshot_info_with_fallback_ssm_client(instance_id, region, snapshot_id, baseline_override)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 108, in _get_snapshot_info_with_fallback_ssm_clientraise PatchManagerError("GetDeployableSnapshotForInstance had access denied and no metadata credentials were available",patch_common.exceptions.PatchManagerError: ('GetDeployableSnapshotForInstance had access denied and no metadata credentials were available', 145)03/21/2023 06:13:46 root [ERROR]: ('GetDeployableSnapshotForInstance had access denied and no metadata credentials were available', 145)Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 126, in _get_snapshot_infopatch_snapshot = _get_snapshot_with_client(ssm_client, instance_id, snapshot_id, baseline_override)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 434, in _get_snapshot_with_clientreturn ssm_client.get_deployable_patch_snapshot_for_instance(File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 276, in _api_callreturn self._make_api_call(operation_name, kwargs)File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 586, in _make_api_callraise error_class(parsed_response, operation_name)botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009dab947dec531ad doesn't match the credentialsDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 102, in _get_snapshot_info_with_fallback_ssm_clientpatch_snapshot = _get_snapshot_with_client(fallback_ssm_client, instance_id, snapshot_id, baseline_override)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 434, in _get_snapshot_with_clientreturn ssm_client.get_deployable_patch_snapshot_for_instance(File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 276, in _api_callreturn self._make_api_call(operation_name, kwargs)File "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 586, in _make_api_callraise error_class(parsed_response, operation_name)botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009dab947dec531ad doesn't match the credentialsDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_startup_entrance.py", line 203, in executeexit(   entrance_module.execute(*argv))File "/var/log/amazon/ssm/patch-baseline-operations/os_selector.py", line 54, in executecommon_os_selector_methods.fetch_snapshot(operation_type, instance_id, region, reboot_option, document_step,File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 280, in fetch_snapshotsnapshot_info = _get_snapshot_info(instance_id, snapshot_id, region, baseline_override_dict)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 136, in _get_snapshot_inforeturn _get_snapshot_info_with_fallback_ssm_client(instance_id, region, snapshot_id, baseline_override)File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 108, in _get_snapshot_info_with_fallback_ssm_clientraise PatchManagerError("GetDeployableSnapshotForInstance had access denied and no metadata credentials were available",patch_common.exceptions.PatchManagerError: ('GetDeployableSnapshotForInstance had access denied and no metadata credentials were available', 145)03/21/2023 06:13:46 root [ERROR]: write() argument must be str, not NoneTraceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 126, in _get_snapshot_infoFile "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 434, in _get_snapshot_with_clientFile "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 276, in _api_callFile "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 586, in _make_api_callbotocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009dab947dec531ad doesn't match the credentialsDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 102, in _get_snapshot_info_with_fallback_ssm_clientFile "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 434, in _get_snapshot_with_clientFile "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 276, in _api_callFile "/var/log/amazon/ssm/patch-baseline-operations/botocore/client.py", line 586, in _make_api_callbotocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetDeployablePatchSnapshotForInstance operation: Instance Id i-009dab947dec531ad doesn't match the credentialsDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/var/log/amazon/ssm/patch-baseline-operations/common_startup_entrance.py", line 203, in executeFile "/var/log/amazon/ssm/patch-baseline-operations/os_selector.py", line 54, in executeFile "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 280, in fetch_snapshotFile "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 136, in _get_snapshot_infoFile "/var/log/amazon/ssm/patch-baseline-operations/common_os_selector_methods.py", line 108, in _get_snapshot_info_with_fallback_ssm_clientpatch_common.exceptions.PatchManagerError: ('GetDeployableSnapshotForInstance had access denied and no metadata credentials were available', 145)During handling of the above exception, another exception occurred:Traceback (most recent call last):File "<stdin>", line 181, in <module>File "/var/log/amazon/ssm/patch-baseline-operations/common_startup_entrance.py", line 210, in executeFile "/var/log/amazon/ssm/patch-baseline-operations/common_startup_entrance.py", line 149, in abortTypeError: write() argument must be str, not NoneFollowComment"

Answer: 0I  find  soluction mv .aws/credentials file in other folder like bewlow commandmv .aws/credentials  /tmpCommentShareRamanswered 2 months ago


Question: Amazon Connect - Contact search Issue

"Hi Team, we are working on Amazon Connect for Contact flow development. Till two weeks before we can able to view the Contact search report for our dialed calls. But without any change, reports are not showing for our dialed calls and also for past records too. We need a way forward steps how to resolve this issue to view the Contact search report.We need to know whether we have missed any basic steps to view the Contact search report. If there is any kindly share us the proper structure to resolve this issue.FollowComment"

Answer: "1In order to access to the Contact search reports, you need the required permissions.You can check the Search for contacts documentation for a guide on how Contact search works. More info about the required permissions here (Contact search section).CommentShareigsalvaranswered 11 days agoArunPB 11 days agoHi @igsalvar, thanks for your information. I've enabled "CallCenterManager" on "security profile" and also all the permissions on CL too.But my concern is Contact search is not showing any of the data for the dialed call. Contact search is just blank for all user. I've enabled all the permission listed on "security profile" and also selected all on CL permission for viewing the data on "Contact Search", But still there is no data viewing on "Contact Search".Share0I leave you this troubleshooting guide: How do I troubleshoot issues with contact trace records or applying filters in Amazon Connect?.Sometimes the Restrict contact access option can cause this type of problems, you can try tu turn it off. Check the section Overlapping security profiles in the guide for more details.CommentShareigsalvaranswered 10 days ago0To be sure, you see the call activity in Cloudwatch, right?CommentSharedmaciasanswered a day ago"


Question: Handling S3 events. Race condition?

"Is there a possibility of race conditions with S3 events? Does the event fire before the object is fully PUT? We are getting occasional "Unexpected end of JSON input" on the files that we are processing when handling the event. But upon manual inspection the file is valid and is not broken. Just wondering if possibly the PUT event is fired before the file is fully stored, and when we try to read it back in the event handler, we get a partial file back.FollowComment"


Question: Specify Individual Instance In Trust Policy Of IAM Role

"The following trust policy is the default trust policy for an EC2 instance role.{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "sts:AssumeRole"            ],            "Principal": {                "Service": [                    "ec2.amazonaws.com"                ]            }        }    ]}Is it possible to limit this trust policy to allow the role to only be attached to a specific instance? I know that it would be possible to only grant the IAM permissions to a user to pass this role to a specific instance but I would also like to limit the scope of this role to a specific instance at the same time.FollowComment"

Answer: "0Accepted AnswerI could not find a way to do this and this article confirms this, How to use trust policies with IAM roles:Several customers have asked if it’s possible to design a trust policy for an IAM role such that it can only be passed to a specific Amazon EC2 instance. This isn’t directly possible. You cannot place the Amazon Resource Name (ARN) for an EC2 instance into the Principal of a trust policy, nor can you use tag-based condition statements in the trust policy to limit the ability for the role to be used by a specific resource.The only option is to manage access to the iam:PassRole action within the permission policy for those IAM principals you expect to be attaching IAM roles to AWS resources. This special Action is evaluated when a principal tries to attach another IAM role to an AWS service or AWS resource.CommentShareEXPERTkentradanswered a year ago"


Question: AWS-SDK configuration error In Angular application running on EC2 with instance profile role

"We have an angular application running on an EC2 instance with an instance profile role attached.Ideally configuring aws-sdk secret and access key is not required. Because of instance profile role.But AWS-SDK configure is failing, and showing the error: "Missing credentials in the config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1"Note: We cannot use API Gateway in the project that is the reason we have aws-sdk installed in the angular app itself.FollowComment"

Answer: 0You still need to add an IAM role and add attach the required policies to enable your app communication with SDK.CommentShareAkbar Majeedanswered 10 months ago