Question: Glue Interactive vs SageMaker Processing?

"Greetings! I'm a data scientist working in SageMaker notebooks. I'd appreciate an explanation about when should I use Glue Interactive and not SageMaker Processing jobs.To my understanding, they are very similar and I can't differentiate them.Thank you!FollowComment"

Answer: "2Hello! It depends on what you are trying to achieve.Let us just talk about notebooks first - Sagemaker notebook (or even Glue notebook) is quite efficient for quick prototyping and analysis of data. For example, if you just want to make certain charts from a CSV or do quick data wrangling etc. then the Notebook is often the preferred choice.  The notebook is also fantastic for documenting algorithms. The interactivity helps process step by step and to change your processing along based on the data that you would see. From a tooling perspective, the Glue notebooks provide the data engineer ability to run Jupyter notebok or Zeppelin notebook.  SageMaker notebook is the tool preferred by data scientists and Machine learning engineers and provides the Jupyter notebook interface.Sagemaker provides multiple computing options including ability to choose EC2 instances. in SageMaker Processing you can customize the execution environment, as you could provide a Docker imageA Glue job is typically built for executing ETL jobs in a Spark based/Python serverless job that executes in a cluster of nodes to parallel process data in multiple nodes. AWS Glue is a serverless data integration platform that makes combining, preparing, and finding data for application development, machine learning, and analytics a breeze. It delivers all of the features required for data integration, allowing you to begin analyzing and putting your data to use in minutes rather than months. To make data integration simpler, AWS Glue offers both code-based and visual interfaces. The AWS Glue Data Catalog allows users to quickly locate and retrieve data. With just limited clicks in AWS Glue Studio, ETL (extract, transform, and load) developers and data engineers can graphically construct, execute, and monitor ETL processes. AWS Glue DataBrew allows data analysts and scientists to visually enhance, clean, and standardize information without writing codes. AWS Glue scans your data sources, recognizes data types, and recommends schemas for storing your data. It produces the code needed to conduct your data transformations and loading operations automatically. AWS Glue makes it simple to perform and manage hundreds of ETL processes, as well as to mix and duplicate data across numerous data stores using SQL.CommentShareAnanth Tirumanuranswered 9 months agoEXPERTFabrizio@AWSreviewed 8 months agorePost-User-5941005 9 months agoThank you for the elaborate answer. I'll clarify the scenario. I'm a data scientist working in a Jupyter notebook, and I need some data cleaning done beforehand. Should I choose SageMaker Processing or Glue Interactive? Thanks!Share1Accepted AnswerI would suggest that you use Sagemaker processing for the data cleansing and preparation. I have led projects where all the data cleansing, preparation and model build and testing have been done in Sagemaker and the data scientists love the tool.CommentShareAnanth Tirumanuranswered 9 months ago"


Question: Athena "Generate Create Table DDL" call results in a Java Exception

"Hi,I'm getting an java exception when describing the ddl for a table that I have. The table's ddl uses a '-' for one of its keys. I believe this is a bug.Here is an example of a record:{"updated_at": "2019-01-17T00:51:17Z", "tenant_guid": "123456708", "payload": {"2017": {"UpdatedAt": "2019-01-17T00:51:17Z", "Members": [{"Summary": [{"DataID": "Members", "WithDocs": 0, "Total": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "Documents": [{"Summary": [{"DataID": "Documents", "Acquired": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "CodeActions": [{"Summary": [{"DataID": "Code Actions", "Suggested": 0}], "Accepted": [{"Percent": 0, "ActualValue": 0}], "Rejected": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "ManAdd": [{"Percent": 0, "ActualValue": 0}]}], "CodesLinkedToClaim": [{"Summary": [{"DataID": "Codes Linked to a Claim", "Accepted_Added": 0}], "Linked": [{"Percent": 0, "ActualValue": 0}], "NotLinked": [{"Percent": 0, "ActualValue": 0}]}]}, "2017-2018": {"UpdatedAt": "2019-01-17T00:51:17Z", "Members": [{"Summary": [{"DataID": "Members", "WithDocs": 0, "Total": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "Documents": [{"Summary": [{"DataID": "Documents", "Acquired": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "CodeActions": [{"Summary": [{"DataID": "Code Actions", "Suggested": 0}], "Accepted": [{"Percent": 0, "ActualValue": 0}], "Rejected": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "ManAdd": [{"Percent": 0, "ActualValue": 0}]}], "CodesLinkedToClaim": [{"Summary": [{"DataID": "Codes Linked to a Claim", "Accepted_Added": 0}], "Linked": [{"Percent": 0, "ActualValue": 0}], "NotLinked": [{"Percent": 0, "ActualValue": 0}]}]}, "2017-2019": {"UpdatedAt": "2019-01-17T00:51:17Z", "Members": [{"Summary": [{"DataID": "Members", "WithDocs": 0, "Total": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "Documents": [{"Summary": [{"DataID": "Documents", "Acquired": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "CodeActions": [{"Summary": [{"DataID": "Code Actions", "Suggested": 0}], "Accepted": [{"Percent": 0, "ActualValue": 0}], "Rejected": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "ManAdd": [{"Percent": 0, "ActualValue": 0}]}], "CodesLinkedToClaim": [{"Summary": [{"DataID": "Codes Linked to a Claim", "Accepted_Added": 0}], "Linked": [{"Percent": 0, "ActualValue": 0}], "NotLinked": [{"Percent": 0, "ActualValue": 0}]}]}, "2018": {"UpdatedAt": "2019-01-17T00:51:17Z", "Members": [{"Summary": [{"DataID": "Members", "WithDocs": 0, "Total": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "Documents": [{"Summary": [{"DataID": "Documents", "Acquired": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "CodeActions": [{"Summary": [{"DataID": "Code Actions", "Suggested": 0}], "Accepted": [{"Percent": 0, "ActualValue": 0}], "Rejected": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "ManAdd": [{"Percent": 0, "ActualValue": 0}]}], "CodesLinkedToClaim": [{"Summary": [{"DataID": "Codes Linked to a Claim", "Accepted_Added": 0}], "Linked": [{"Percent": 0, "ActualValue": 0}], "NotLinked": [{"Percent": 0, "ActualValue": 0}]}]}, "2018-2019": {"UpdatedAt": "2019-01-17T00:51:17Z", "Members": [{"Summary": [{"DataID": "Members", "WithDocs": 0, "Total": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "Documents": [{"Summary": [{"DataID": "Documents", "Acquired": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "CodeActions": [{"Summary": [{"DataID": "Code Actions", "Suggested": 0}], "Accepted": [{"Percent": 0, "ActualValue": 0}], "Rejected": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "ManAdd": [{"Percent": 0, "ActualValue": 0}]}], "CodesLinkedToClaim": [{"Summary": [{"DataID": "Codes Linked to a Claim", "Accepted_Added": 0}], "Linked": [{"Percent": 0, "ActualValue": 0}], "NotLinked": [{"Percent": 0, "ActualValue": 0}]}]}, "2019": {"UpdatedAt": "2019-01-17T00:51:17Z", "Members": [{"Summary": [{"DataID": "Members", "WithDocs": 0, "Total": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "Documents": [{"Summary": [{"DataID": "Documents", "Acquired": 0}], "Reviewed": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "WithoutOpportunity": [{"Percent": 0, "ActualValue": 0}]}], "CodeActions": [{"Summary": [{"DataID": "Code Actions", "Suggested": 0}], "Accepted": [{"Percent": 0, "ActualValue": 0}], "Rejected": [{"Percent": 0, "ActualValue": 0}], "Remaining": [{"Percent": 0, "ActualValue": 0}], "ManAdd": [{"Percent": 0, "ActualValue": 0}]}], "CodesLinkedToClaim": [{"Summary": [{"DataID": "Codes Linked to a Claim", "Accepted_Added": 0}], "Linked": [{"Percent": 0, "ActualValue": 0}], "NotLinked": [{"Percent": 0, "ActualValue": 0}]}]}}, "dashboard_type": "scout_progress"}Your query has the following error(s):FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException: Error: : expected at the position 1025 of 'string:string:struct<2017:struct<UpdatedAt:string,Members:array<struct<Summary:array<struct<DataID:string,WithDocs:int,Total:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,Documents:array<struct<Summary:array<struct<DataID:string,Acquired:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,CodeActions:array<struct<Summary:array<struct<DataID:string,Suggested:int>>,Accepted:array<struct<Percent:int,ActualValue:int>>,Rejected:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,ManAdd:array<struct<Percent:int,ActualValue:int>>>>,CodesLinkedToClaim:array<struct<Summary:array<struct<DataID:string,Accepted_Added:int>>,Linked:array<struct<Percent:int,ActualValue:int>>,NotLinked:array<struct<Percent:int,ActualValue:int>>>>>,2017-2018:struct<UpdatedAt:string,Members:array<struct<Summary:array<struct<DataID:string,WithDocs:int,Total:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,Documents:array<struct<Summary:array<struct<DataID:string,Acquired:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,CodeActions:array<struct<Summary:array<struct<DataID:string,Suggested:int>>,Accepted:array<struct<Percent:int,ActualValue:int>>,Rejected:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,ManAdd:array<struct<Percent:int,ActualValue:int>>>>,CodesLinkedToClaim:array<struct<Summary:array<struct<DataID:string,Accepted_Added:int>>,Linked:array<struct<Percent:int,ActualValue:int>>,NotLinked:array<struct<Percent:int,ActualValue:int>>>>>,2017-2019:struct<UpdatedAt:string,Members:array<struct<Summary:array<struct<DataID:string,WithDocs:int,Total:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,Documents:array<struct<Summary:array<struct<DataID:string,Acquired:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,CodeActions:array<struct<Summary:array<struct<DataID:string,Suggested:int>>,Accepted:array<struct<Percent:int,ActualValue:int>>,Rejected:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,ManAdd:array<struct<Percent:int,ActualValue:int>>>>,CodesLinkedToClaim:array<struct<Summary:array<struct<DataID:string,Accepted_Added:int>>,Linked:array<struct<Percent:int,ActualValue:int>>,NotLinked:array<struct<Percent:int,ActualValue:int>>>>>,2018:struct<UpdatedAt:string,Members:array<struct<Summary:array<struct<DataID:string,WithDocs:int,Total:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,Documents:array<struct<Summary:array<struct<DataID:string,Acquired:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,CodeActions:array<struct<Summary:array<struct<DataID:string,Suggested:int>>,Accepted:array<struct<Percent:int,ActualValue:int>>,Rejected:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,ManAdd:array<struct<Percent:int,ActualValue:int>>>>,CodesLinkedToClaim:array<struct<Summary:array<struct<DataID:string,Accepted_Added:int>>,Linked:array<struct<Percent:int,ActualValue:int>>,NotLinked:array<struct<Percent:int,ActualValue:int>>>>>,2018-2019:struct<UpdatedAt:string,Members:array<struct<Summary:array<struct<DataID:string,WithDocs:int,Total:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,Documents:array<struct<Summary:array<struct<DataID:string,Acquired:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,CodeActions:array<struct<Summary:array<struct<DataID:string,Suggested:int>>,Accepted:array<struct<Percent:int,ActualValue:int>>,Rejected:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,ManAdd:array<struct<Percent:int,ActualValue:int>>>>,CodesLinkedToClaim:array<struct<Summary:array<struct<DataID:string,Accepted_Added:int>>,Linked:array<struct<Percent:int,ActualValue:int>>,NotLinked:array<struct<Percent:int,ActualValue:int>>>>>,2019:struct<UpdatedAt:string,Members:array<struct<Summary:array<struct<DataID:string,WithDocs:int,Total:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,Documents:array<struct<Summary:array<struct<DataID:string,Acquired:int>>,Reviewed:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,WithoutOpportunity:array<struct<Percent:int,ActualValue:int>>>>,CodeActions:array<struct<Summary:array<struct<DataID:string,Suggested:int>>,Accepted:array<struct<Percent:int,ActualValue:int>>,Rejected:array<struct<Percent:int,ActualValue:int>>,Remaining:array<struct<Percent:int,ActualValue:int>>,ManAdd:array<struct<Percent:int,ActualValue:int>>>>,CodesLinkedToClaim:array<struct<Summary:array<struct<DataID:string,Accepted_Added:int>>,Linked:array<struct<Percent:int,ActualValue:int>>,NotLinked:array<struct<Percent:int,ActualValue:int>>>>>>:string' but '-' is found.This query ran against the "rascout" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: a66ffc54-81a3-4ef5-b2b2-0e33befb0b0f.FollowComment"

Answer: "0did you create a table in Athena ?Athena does not support "-" however, datacatalog shared by other services like glue,emr and redshift which might not have same  validation hence you may get above error.CommentShareShivananswered 4 years ago0The table was created with glue. Should Athena support "-" for consistency???CommentSharewtruong424answered 4 years ago0Hi,Athena table, view, database, and column names cannot contain special characters, other than underscore (_).Link : https://docs.aws.amazon.com/athena/latest/ug/tables-databases-columns-names.htmlTo work with special characters within the statements we can try to use backtics for the columns or table names. Example:We can use backtics to enclose table, view, or column names that begin with an underscore. For example:CREATE TABLE _myunderscoretable (_id string,_indexstring,…Similarly for the select statement as well we can use backtics when having a “-“ (or any other special character). Example:SELECT * FROMfpa-devLIMIT 10Regarding the describe table behavior with tables having special characters within the nested columns you can use column mapping within the SerDe properties of the table. We can handle forbidden characters with the help of mapping. Please refer the below blog for using the mapping parameter within the serve properties:https://aws.amazon.com/blogs/big-data/create-tables-in-amazon-athena-from-nested-json-and-mappings-using-jsonserde/I hope this helps but in case you have any doubts or any queries/issues when using Athena that need more in-depth analysis you can definitely contact AWS Support. And we will be glad to assist as always.Have a AWSome Day :)CommentShareAWS-User-4025440answered 4 years ago"


Question: Oracle RDS upgrade to 12.2 not supported

"Hi,I need to upgrade our development database to 12.2 (to make use of new json functionality).Attribute                   Current value                                      New valueDB engine version12.1.0.2.v14 (Oracle 12.1.0.2.v14)12.2.0.1.ru-2019-01.rur-2019-01.r1(Oracle 12.2.0.1.ru-2019-01.rur-2019-01.r1)This is a t2.micro instance. With an option group including Apex (v5.14), Apex_dev and utl_mail. When I attempt the modification I get this message:RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.micro, Engine=oracle-se2, EngineVersion=12.2.0.1.ru-2019-01.rur-2019-01.r1, LicenseModel=license-included. For supported combinations of instance class and database engine version, see the documentation. (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: 36639790-1ee2-4907-8368-4fadc8f1fef5)Please adviseThanksRalphFollowComment"

Answer: "1Howdy Ralph!The "micro" instance types are no longer supported with RDS Oracle as of Oracle Database 12.2. You will need to scale to a db.t2.small or similar before you will be able to upgrade. To see a full list of supported instance classes please refer to our documentation.https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Oracle.html#Oracle.Concepts.InstanceClassesHope this helps. Thanks!MichaelCommentShareAWS-User-9938760answered 4 years ago0Turns out micro instances are not supported with v12.2. You need minimum small. This is a shame as they cost much more.CommentShareRalph99answered 4 years ago0Unfortunately. But not a huge surprise. I believe Micro instances were always documented as not meeting Oracle's configuration requirements and only for use for limited testing. 12.2 probably had just enough increase in resource requirements to cause performance to fall off a cliff on the memory constrained Micros.Edited by: HalTemp on Mar 22, 2019 5:07 AMCommentShareHalTempanswered 4 years ago"


Question: Windows instance takes ~18 minutes to restart

"Hi all, my Windows Server 2019 m5d.2xlarge instance previously restarted quickly (~4min) but since mid-Sept 2021 it is taking exactly 18 minutes to restart. When I do "ping -t [IP]" the server is completely absent during that time, as if it were rebooting & chkdsk'ing just like old times.I created a replacement fresh instance in mid-Sept 2021, Windows Server 2022 m5d.4xlarge based on the AWS Windows base AMI, and it also took a very long time to reboot nearly 30 minutes. I am now running my production workload on this instance so I have not rebooted it, but I'm concerned that it takes so long. My Win2019 instance is still around as a backup & I'm experimenting with it to make reboots normal again -- any ideas?I have tried:stopped all non-essential services and set to Manualreviewed the eventvwr logs (it shows silence for ~18 min between typical last-event and normal-startup-event)run "fsutil dirty query c:" but it's cleanran "Sysprep /generalize /oobe /mode:vm" (which failed with unspecified error, but there was no change in behavior)did a "hard reboot" from Ec2 console (made reboot take longer, and Windows eventvwr complained about it later)I have already installed all Win updatesAny ideas would be appreciated, thank you!FollowComment"

Answer: "0Replying to my own message -- the problem was our 12,000 local Windows accounts, which previously (before Sept 2021) hadn't been a problem but now were causing a 20-30min dead hang on reboot in my environment. Everything was rebuilt from scratch and the hangs returned when the user accounts were added in. We dropped most of the accounts & the problem went away.Also, don't run the Sysprep command -- it nuked an instance.CommentSharedoubleshoanswered 2 years ago"


Question: Ec2 instance in private subnet can send outbound traffic without NAT gateway

"Hi Team,Could you please let me know if Ec2 instance in private subnet can send outbound traffic without NAT gateway.Regards,Rajesh BFollowComment"

Answer: "0For private subnets, access to the public Internet requires a NAT gateway or NAT instance.Other outbound traffic (e.g., communications within a VPC) can communicate without a NAT gateway.Access to AWS services (e.g. S3) requires a VPC endpoint or NAT gateway.CommentShareEXPERTRiku_Kobayashianswered 2 months ago0You can use NAT instance, see the difference between the two solutions:NAT Gateway Vs NAT Instance:https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-comparison.htmlYou can also deploy Squid Proxy for internet outgoing access control.Reference: https://aws.amazon.com/es/blogs/security/how-to-set-up-an-outbound-vpc-proxy-with-domain-whitelisting-and-content-filtering/CommentShareEXPERTTushar_Janswered 2 months ago0It depends what you mean by "outbound".  Previous answers have covered IPv4 traffic going directly from your subnet to the general internet or to AWS services.  Also IPv6 traffic may use an egress-only IPv6 gateway to reach the internet from a private subnet.  And outbound traffic could go via direct-connect or site-to-site VPN to go through an on-prem proxy on the way to the internet or just to reach on-prem servers.CommentShareEXPERTskinsmananswered 2 months ago"


Question: can't connect to my instance

"I'm using AWS RDP instance.In this instance RDP network setting, I manually changed my IP address through network adapter properties and change (TCP/ipv4) and click on apply.after that,I suddenly exit from RDP instance.then I try to reconnect again but I can't able to connect.I reboot my instance multiple time,I redownload my remote desktop file, but didn't work anythingit didn't connect to RDP so definitely no password issue.I got error messageFollowComment"

Answer: "0Hello,If the instance is in a subnet that uses an AWS NAT Gateway, make sure that you're connecting to the instance's private IP address. The instance is unreachable on the public IP address behind the AWS NAT Gateway.Verify that port 3389 isn't blockedVerify that port 3389 isn't blocked by your AWS security group, Network ACL, OS firewall, or antivirus software.Confirm you're using the correct firewall and network configurationOpen a PowerShell window from your local device, and then run the following command:=> Test-NetConnection -ComputerName yourdomain.exampleco.com -Port 33892.    In the output of the preceding command, verify that TcpTestSucceeded is True. This indicates that the EC2 instance is responding over the network and you can reach the required RDP port. For more information, see Test-NetConnection on the Microsoft website.For EC2 checks, refer the below link:https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/troubleshoot-connect-windows-instance.html#rdp-issues -- [1]CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 10 months ago0solved....I can't explain but, I'm from non technical field so its very hard to understand me. but after one day of research I solved this problem.first I actually remove my default public IP.(it's trick, search on google/YouTube) then I add a new elastic IP.then I add one new private IP through network interface option.then finally I bind these newly created two IP (elastic + private) together using elastic IP association setting.so now my default IP is already removed.so now I can access my instance through new elastic IP. which is now working like my public IP.that's it DONE....my all inbound and outbound rules are defaults setting. I never need to edit/change it.CommentSharerePost-User-3411932answered 10 months ago"


Question: Unable to delete ebs volumes

"Hi there,I have three volumes in my AWS account that are not attached to any instance but I can't delete them. I can not attach them or detach them (neither force detachment).How could I delete them?FollowComment"

Answer: 0Please read this similar post and see if it helps.https://repost.aws/questions/QUE_vyhnlySMaS_ZQyrue2Ww/cant-delete-ebs-volume-not-even-with-the-force-optionCommentShareSandeepVudataanswered a year ago


Question: cpu utilization in millicore units instead of percentage

I want to check the current cpu utilization in millicore units instead of percentage and the maximum cpu utilization that a pod can allow me (in millicore units). How can I check that?FollowComment


Question: can't view dynamodb table from EC2 ssh session

"I ssh into my EC2, my caller-identity is assume role. and i try to display dynamodb table test (created by user bob)usingaws dynamodb scan --table-name test--region us-east-1. I got error:An error occurred (AccessDeniedException) when calling the Scan operation: User: arn:aws:sts::0********:assumed-role/ssm/i-****** is not authorized to perform: dynamodb:Scan onresource: arn:aws:dynamodb:us-east-1:0******:table/test, how can i switch user in EC2 ssh session to bob, so I can view the table, or there is better way to display dynamodb table in EC2 ssh session? thanksFollowComment"

Answer: "1To get access to your Amazon DynamoDB service in your local account, you need to update your Amazon EC2 Instance Profile by adding a IAM policy allowing at least the Action 'dynamodb:Scan'.More information about:Amazon EC2 Instance ProfileExample policy for access to Amazon DynamoDBCommentSharekunztanswered a year ago0In your description, its not detailed how IAM is designed for your usecase. So its hard for me to comment on a better way. But based on your description, you might be able to go to the instance role (in IAM) corresponding to the EC2 instance (the i** in the ARN User: arn:aws:sts::0****:assumed-role/ssm/i-**) where you are ssh-ed and then add an inline permissions policy (in IAM) which is identical to the DynamoDB related permission policy associated with the user bob (in IAM).CommentShareNiket_Chhajedanswered a year ago"


Question: How do we unzip a file in S3 bucket using C#.NET

"There is a ZIP file my S3 bucket, I just want to Un-ZIP using C#.NETFollowComment"

Answer: "1The Amazon S3 SDK offers you a way to download a file in-memory. By using the ResponseStream property of the response object, you obtain access to the downloaded object data. Refer to the documentation about downloading objects from S3.The System.IO.Compression.ZipArchive has a constructor accepting a stream as an input parameter. You can use this constructor to create an object for working with the archive in-memory.Here is an example code snippet that should give you an idea. Please note that this code is not production-ready and should not be used as-is.var client = new AmazonS3Client();var request = new GetObjectRequest { BucketName = "your-bucket", Key = "file.zip" };using var response = await client.GetObjectAsync(request);using var zip = new ZipArchive(response.ResponseStream, ZipArchiveMode.Read);foreach (var entry in zip.Entries){    using var stream = entry.Open();    // stream will contain decompressed data, do whatever you want with it}UPDATEAs mentioned in the comment below, you should be aware of the caveats of this approach. You might need to copy the ResponseStream to a MemoryStream first. Also, you have to ensure the zip file itself and its extracted content fit in the available memory. Otherwise, your application will terminate with an OutOfMemoryException.CommentShareDmitry Balabanovanswered a year agoChris M a year agoIf you don't want to create a temporary file, this is the correct approach, but with a caveat. There is a possibility the backing stream will have to be hydrated into memory in order to jump to different entries. If the file is very large, there is a chance for OutOfMemoryException. Only you know the about system resources available, but if the file is over - say 100MB, you might consider writing it to disk first. On disk, it's possible to use random access to move backward and forward within the stream using buffering. Network streams simply don't have this option; they are forward only.Share0Hello,You'll need to download it first and then unzip - and upload it back if you need it in the same s3 bucketCommentShareAWS-ADolganovanswered a year agoAWS-User-8673818 a year agoHi, Thanks for your reply.I cannot download it to local hard disk, it should be run on memory.....do we need to use here lamda functions?ShareByron_G a year agoYes, a lambda function will work. You need to be aware of any sizing requirements, and not exceed memory of the lambda.Share0Is there any code snippet please, I did not find the answers in internet.I just need to unzip the file in S3 bucket on the fly in c#, no physical storageCommentShareAWS-User-8673818answered a year ago0I have a similar question here: https://repost.aws/questions/QUh855MWiuRJerSN9fGVjVuQ/aws-s-3-get-object-not-working-getting-corrupt-file. My code downloads the zip file from S3 but is corrupt, and if is corrupt I can't unzip it. Please help!CommentShareNicolasAbo17answered a year ago"


Question: EMR Serverless jar configuration

"I'm working on EMR Serverless for validating of data located in s3 using deequ library. But I am Unable to do that , I got this error...Traceback (most recent call last):File "/tmp/spark-f2b9d6f8-9bb9-4879-a398-1f67f9ec5e70/app3.py", line 179, in <module>.addConstraintRule(UniqueIfApproximatelyUniqueRule()) File "/home/hadoop/environment/lib64/python3.7/site-packages/pydeequ/suggestions.py", line 81, in runresult = self._ConstraintSuggestionRunBuilder.run()File "/usr/lib/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line 1322, in callFile "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 190, in decoFile "/usr/lib/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/protocol.py", line 328, in get_return_valuepy4j.protocol.Py4JJavaError: An error occurred while calling o177.run.: com.amazon.deequ.analyzers.runners.MetricCalculationRuntimeException: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 6.0 failed 4 times, most recent failure: Lost task 0.3 in stage 6.0 (TID 19) ([2600:1f18:2d85:5e03:ba20:a78b:a26c:61ab] executor 1): java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field org.apache.spark.sql.execution.aggregate.HashAggregateExec.aggregateExpressions of type scala.collection.Seq in instance of org.apache.spark.sql.execution.aggregate.HashAggregateExechow could I resolve this errorFollowComment"

Answer: "0Hello,Thank you for raising this question on re:Post.From the stacktrace shared I can see it is a ClassCastException during serialization, which points to some incompatibility of the classes for serialization in the spark application. However, it is not enough to clearly identify the root cause here. Please help us with the following so that we can assist you further on thisAre you able to run this successfully on EMR on EC2 cluster?Are you able to run a test without deequ to confirm EMR serverless job is working as expected without this dependency?Please share how you are adding the additional deequ libraries to the runtime serverless environment. Your start-job-run command should have the details on this.Please share reproduction steps, including a link to download the deequ library if it is publicly available.CommentShareSUPPORT ENGINEERKrishnadas Manswered a month agorePost-User-6281491 a month agoThank You very much for the response...Yes I have successfully run this on the EMR cluster.yes. it's working without deequ--conf spark.jar=https://repo1.maven.org/maven2/com/amazon/deequ/deequ/2.0.1-spark-3.2/deequ-2.0.1-spark-3.2.jar --conf spark.jars.packages=com.amazon.deequ:deequ:2.0.1-spark-3.24 ?Share"


Question: Code Obfuscation in Jupyter/Databricks Notebooks

"As we are having reusable functions in spark ETL notebooks in AWS EMR - Jupyter, we have to obfuscate/ secure the source code present in a particular notebook.By default, ETL notebooks provide the notebooks view access to group members within the service/ workspace account so that code can easily be interpreted by others.please advise the best way to obfuscate/ secure the code present in the notebooksFollowComment"

Answer: 0Currently you can use conditions in your identity-based policy to control access to  EMR notebooks based on tags.Please refer to our public documentation below for multiple examples of controlling access to EMR Notebookshttps://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-fine-grained-cluster-access.html#emr-managed-notebooks-tags-examplesCommentShareSujay_Manswered a year ago


Question: Restoring point in time - Aurora Mysql DB gives an error:  Yo don't neet to set storage size

"Loged in aws console, for my Aurora Mysql DB cluster I select restoring point in time option:Set instance nameClick "Restore to point in time" buttonThen I get this error: "The requested DB instance will be a member of DB cluster. Yo don't neetd to set a storage size"There is nothing in this process where I can set the storage sizeIf I restore by snapshot it works fine.FollowComment"


Question: Amazon Lightsail traffic limits

"Currently on the $3.5 plan but I aim to scale up.What kind of traffic limits do we expect on this plan?How many visitors per day would be the limit?How many concurrent users at any time for website and for database?If I'm expecting 100,000 or more views per day what kind of configuration do I need to do when needed?FollowComment"

Answer: "0The answer to your question is a very definitive "It Depends".  :)It depends on what apps are you are running, how much activity each user engages in, what kind of traffic you are serving, etc.Lightsail instances aren't intended for high performance activity ... mostly lower traffic sites with the occasional burst of activity.CommentShareDavid Ganswered 3 years ago0Its a static website, but will have a few emdedded youtube videos. and some forms that the user would fill, the form data will be saved in a database.CommentShareAmaze4Everanswered 3 years ago0Sounds like you're well within the scope of Lightsail.The external videos won't count against your traffic.I would suggest going with a slightly higher tier instance and see how it goes.  If performance turns out to be a problem, you can always snapshot your instance, create a larger instance from the snapshot, and reassign the static IP.davidCommentShareDavid Ganswered 3 years ago0Ok that's good to hear,I should have put it like 100,000 + visitors per day. So here's my plan you2x  USD10 instances of Lightsail1x Managed Database (Don't know if this resides on a separate light sail instance)1x Load BalancerI hope this doesn't sound overkill.Are you aware how many instances can a load balancer take, that way I can keep adding light sail intances to it once the limit is about to reach.My only issue now is bandwidth, more users would cross the bandwidth limit pretty fast, and then I'll be in the pay as you go range, which is pretty pricey.CommentShareAmaze4Everanswered 3 years ago0Amaze4Ever wrote:I hope this doesn't sound overkill.IMO, your plan is a bit overkill, at least for a new site.My philosophy has always to use the KISS method (Keep It Simple Stupid) (no offense intended).Start with a single $10 instance + basic database and see how it performs.If the performance isn't sufficient, then consider using a load balancer.If you KNOW what kind of data transfer you're going to experience (using metrics from a current implementation), then maybe it's not overkill.Are you aware how many instances can a load balancer take, that way I can keep adding light sail intances to it once the limit is about to reach.I've never actually used the load balancer feature. I'm not sure if there is a limit.My only issue now is bandwidth, more users would cross the bandwidth limit pretty fast, and then I'll be in the pay as you go range, which is pretty pricey.Keep in mind that the limitation is on data transfer ... not users. If you design your site well, you can limit the amount of traffic that is served directly from your instance.  Use content distribution networks (CDN), host your static content on S3 and use cloudfront to distribute the load, etc.CommentShareDavid Ganswered 3 years ago0Thank you for the wonderful answers.I agree with you with the KISS principle.However I was thinking if I just start with 1 server and the system is live, how to I scale it without down time then?I mean lets say if with a single server a threshold is reached where I now need to scale up, then I would probably need to add another instance and load sharing. This would result in downtime wouldn't it.Or is there a way to increase resources on the fly without affecting downtime?I would have liked the system to autoscale on load however am not familiar with AWS technology as yet to identify the components needed to get this done.Thanks again!CommentShareAmaze4Everanswered 3 years ago0Is your traffic estimates based on experience or what you hope the traffic to be?If this is a new site, then I would start small and ramp up as needed.If you're migrating a site, where you know the traffic, then set it up the way you are describing.CommentShareDavid Ganswered 3 years ago"


Question: How do I reserve IP addresses for a specific purpose?

Is there a way to reserve a subset of IP addresses within a subnet's CIDR range so that no one else can use them except for their intended purpose?FollowComment

Answer: "0Accepted AnswerYes, you can create an elastic network interface for each of the IP addresses that you don't want to be used.CommentShareEXPERTJPanswered 3 years ago"


Question: Kubernetes version and the AMI version to be at the same level ?

""Can someone kindly inform me if, in a Kubernetes environment (EKS , when upgrading from Kubernetes version 1.22 to 1.23, is it mandatory to update the AMI (version 1.22.12-20220926) of the nodes to the latest version, or can I continue using it without any issues even if a node restarts?"FollowComment"

Answer: "0Hello,You can upgarde your eks version from 1.22 to 1.23 with current ami then, after the control plane upgrade you must upgrade the node group version to 1.23. As to further upgrade cluster to any Kubernetes version both the managed nodes and Fargate nodes in your cluster are required to be same as your control plane's version.You can check this doc 'https://docs.aws.amazon.com/eks/latest/userguide/update-cluster.html' for detailed information on eks upgrade process.CommentSharesanjusanswered 5 days ago"


Question: Aurora upgrade 2 to 3 / MySql 5.7 to 8.0: potential bug in pre-check validation

"We believe we have noticed some weird behavior in AWS upgrade pre-checks for Aurora 2 to 3 / Mysql 5.7 to 8.0.We believe it is related to the AWS-specific rule "There must be no queries and stored program definitions from MySQL 8.0.12 or lower that use ASC or DESC qualifiers for GROUP BY clauses," though we are not breaking this rule.Our findings: (SP1)DELIMITER $$CREATE PROCEDURE sp_1 ()BEGIN    SELECT st.name as hdervascferef, max(st.status)    FROM SetupTableName st    GROUP BY st.name;END$$DELIMITER ;Produces a precheck error:{      "level": "Error",      "dbObject": "trax.sp_1",      "description": "Obsolete procedure - trax.sp_1. Contains depreciated keywords."},But: (SP2)DELIMITER $$CREATE PROCEDURE sp_2 ()BEGIN    SELECT st.name as hdervasferef, max(st.status)    FROM SetupTableName st    GROUP BY st.name;END$$DELIMITER ;Does not produce an error.The only difference here is that the alias, hdervascferef  from SP1, an arbitrary string containing substring asc, while the alias hdervasferef alias from SP2 has the 'c' removed and thus does not contain substring asc, and so there is no error.We are running into this on many stored procedures because we have many tables with a column named hasChilds, which has the asc substring, and thus prevents these SPs from passing the pre-check. We have found removing instances of letters asc from an SP causes the pre-check to pass, but this is not a viable option for us, as the use of the hasChilds column in our stored procedures is vital to their function.Replication steps :Add these two SPs into our AWS Aurora instance with engine = 5.7.mysql_aurora.2.07.2follow instructions at AWS RDS MySql Testing an Upgradeverify failed prechecks for SP1 but not SP2We would appreciate any help / guidance that can be provided!tl;drI think we are being flagged for the "There must be no queries and stored program definitions from MySQL 8.0.12 or lower that use ASC or DESC qualifiers for GROUP BY clauses," due to our query having a group by and the sub string 'asc' in it, despite not actually breaking the rule, and this prevents us from upgrading our Aurora instances because we fail the pre-check.FollowComment"

Answer: "0I suggest creating a support ticket for this issue, they will be able to take a look at what was happening and help with the remediation.CommentShareYUTAKA_Hanswered a year ago"


Question: "When Delete Stack, Run Script Before Terminating EC2 Instance"

"When I choose CloudFormation -> Select Stack Name -> Actions -> Delete Stack, I would like a cleanup script to automatically run on each Windows instance before they are terminated, such as delicensing a Tableau Server.  Does anyone have any tips on doing this?  Please let me know.FollowComment"

Answer: "0I asked AWS Support, and they came back with the following reply:Thank you for reaching out to AWS Support.I understand that you would like your Windows instance deployed by CloudFormation to run a cleanup script just before the termination through the stack.Unfortunately there is no native way in CloudFormation to run scripts on an EC2 instances before the stack is deleted (instance is terminated).However, there are several options that can allow you do this:OpsWorks Stack implementation:In case you are familiar with Chef, you may consider deploying OpsWorks Stacks through CloudFormation. OpsWorks Stacks lifecycle events can handle running specific commands on instances registered under your OpsWorks Stack.These commands however, need to specified through Chef recipes.This solution may provide you with a native execution of your commands, although in case you are not familiar with Chef, it may not be the best solution for you.To deploy OpsWorks Stack's elements through CloudFormation, you will need the following resoruce types:Stack: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-opsworks-stack.htmlLayer: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-opsworks-layer.htmlInstance: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-opsworks-instance.htmlGeneral information about OpsWorks stacks can be found here:https://docs.aws.amazon.com/opsworks/latest/userguide/workingstacks.htmlAuto Scaling implementation:Instead of defining specific instances, you can use Auto Scaling Group deployed through CloudFormation, which will launch your instances for you. Auto Scaling has a feature called Lifecycle hooks, which can be used to pause instance action (launch/termination) in order to do some other action through another service. You could configure your Lifecycle hook which will, on instance termination, report to an SNS topic and this SNS topic can trigger Lambda function which can run System Service Manager run command action on the instance.I believe this solution would suit your need better as it will set the instance to Termination:Wait status while the SSM run command is executed. I understand that this scenario will require configuration of several other services, that is why I would suggest to test configuring the services manually and then setup the CloudFormation template with the required configuration.The resources you will need for this scenario are as follows:ASG: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.htmlLifecycleHookSpecificationList: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-autoscaling-autoscalinggroup-lifecyclehookspecification.htmlASG Launch Configuration: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-launchconfig.htmlSNS: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sns-topic.htmlSubscribtion: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sns-subscription.htmlLambda: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.htmlSSM: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-document.htmlDocumentation links to this scenario:ASG lifecycle hooks: https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.htmlSNS: https://docs.aws.amazon.com/sns/latest/dg/welcome.htmlLambda: https://docs.aws.amazon.com/lambda/latest/dg/welcome.htmlSSM: https://docs.aws.amazon.com/systems-manager/latest/userguide/rc-sns-notifications.html#rc-send-notificationsSimilar scenario is also describe on the diagram attached, although instead of SNS, CloudWatch is used to trigger Lambda function.CommentShareMikaelSikoraanswered 4 years ago0I asked AWS Support the same question, and posted the reply in this thread.CommentShareMikaelSikoraanswered 4 years ago"


Question: Health Checks to AWS API Gateway fail when API keys enabled (x-api-key)

"Route 53 Health Checks are impossible to fully enable to AWS API Gateway when the api on the gateway has API keys enabled. Why? Because the request to the gateway must send the "x-api-key" header with a known value.To my knowledge, at best you can configure an HTTP (not HTTPS) health check to the path desired. This request will be responded with a 301 redirect. The 301 is accepted as "healthy" by the default rules of a Route 53 Health Check.If you were to configure an HTTPS health check, there would be no 301 redirect. Instead you would immediately receive a 403 Forbidden. This would fail the health check.FollowComment"

Answer: "0Hi DalePh,Thanks for your feedback. Yes, unfortunately there is no way to pass in headers at this time. For now the workaround of using HTTP or TCP health check type is the only option.thanksCommentShareAWS-User-2990809answered 6 years ago"


Question: cant delete hosted zone

"I am working on a static website and made a mistake with dnssec signing in route53.At one point I could access my website from the internet.But tried to secure the connection to eliminate warning message that site may be insecure.The hosted zone is broken but I can't delete no matter what I try.Now even the unsecure static page will not show.Errors belowError occurredBefore you can delete a hosted zone, you must first delete all resource record sets except the NS and SOA resource record sets for the zone apex.(HostedZoneNotEmpty 400: The specified hosted zone contains DNSSEC Key Signing Keys and so cannot be deleted.)Hmmm… can't reach this pageCheck if there is a typo in  www.xxx.com.Search the web for XXXXXIf spelling is correct, try running Windows Network Diagnostics.DNS_PROBE_FINISHED_NXDOMAINFollowComment"

Answer: "0If you have DS record then reduce the TTL time 0. If you do not have any DS record then go "Customer manaed CMK details" page click the link to Manage key. Select schedule key actions under the "Key actions" drop down menu. Minimun days 7, after the 7-day you can delete this key.If you have DS record, please follow the instructions under the "Disable DNSSEC signing". Please remember Key-signing keys (KSKs) has:Switch to advanced viewSwitch to default viewUnder your registered domain (Domain Registration) "DNSSEC status", click the "Manage keys" and delete or remove any key you have.(Dashboard > Domain > xxx.com > DNSSEC > Manage keys)I hope this will help you.CommentSharerePost-User-7136343answered a month ago"


Question: My putty connection times and xrdp doesn't connect anymore

"Hello,yesterday night my server was working. It was a bit a laggy/slow but working. Today i rebooted the server 3 times and still won't let me login with my password in Putty and xrdp stopped working. I need help please. Its the connection timed out error...how do i fix this? I literally didn't do anything and it stopped woorking....ThanksFollowComment"


Question: AWS Transfer Family client returns an error with no function list_profiles in Lambda but code runs fine locally

"When trying to grab the profiles from the AWS Transfer family client, it returns an error saying 'Transfer' object has no attribute 'list_profiles'. However, the same code works fine when I run it locally. It's only in AWS Lambda does it fail. I thought it might've been a permissions issue but the role has full access to Transfer Family. It also has no issues running other functions like describe_server or list_tags_for_resource so it's not like the client is somehow missing.client = boto3.client(service_name = 'transfer', region_name = region)response = client.list_profiles()By the way, I tested it with other profile functions like describe_profile and the same issue with the attribute not existing happens. Here's the documentation I was following: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/transfer.html#Transfer.Client.describe_profileFollowComment"

Answer: 0Accepted AnswerThe included SDK in the Lambda runtime takes sometime to update to the latest version. I suggest that you include your own version of the SDK and try again.CommentShareEXPERTUrianswered 8 months ago


Question: Not able to access AWS lambda service or ec2 service

"Hi, I have followed steps provided by AWS account team and was not able to view or use AWS lambda service or ec2 service. There appears the issues with the account. However, any public URLs I cannot view and when I ping the url I get 'unknown host'.FollowComment"


Question: Lightsail backup issue - cost

"Because of plesk license issue by instance , we have to put two sites in one instance. if i upgrade to 8gb memory 160gb diskthe cost for backup is high. Our harddisk usage is only  <15gbHow to reduces backup cost?FollowComment"

Answer: "1Hello,Thank you for using Amazon Lightsail. You may already be aware of this, but referring to the Amazon Lightsail FAQ (https://aws.amazon.com/lightsail/faq/), under the question "Q: What do Lightsail snapshots cost?", the following is explained:"Lightsail snapshots cost $0.05 USD/GB-month for both instance snapshots and for disk snapshots. This means if you take a snapshot of your 30 GB SSD instance and keep it for a month, you pay $1.50 USD at the end of the month.When you take multiple successive snapshots of the same instance, Lightsail automatically cost-optimizes your snapshots. For each new snapshot you take, you're changed only for the part of the instance that changed. In the example above, if your instance only changes by 2 GB, your second instance snapshot costs only $0.10 USD per month."So, only your first snapshot would increase in cost (vs a smaller instance size). Any subsequent snapshots of the instance will be cost-optimized and you would only be charged for the difference between snapshots. If you have multiple snapshots, you would not pay $8/mo for every snapshot ($0.05 USD/GB-month X 160 GB), unless you make 160GB of changes between snapshots.The way to reduce backup cost would be to simply store less data on your instance. A few ways you can accomplish this are by offloading static files to Lightsail buckets, or by using a Lightsail Database instead of an on-instance database.I hope this helps!-MaxCommentShareMaxwellanswered a year ago0I know that. issue is i do not need 160gb disk space in 8GB ram instance. If i do snapshot on it i pay more. And i am using 10% of diskspace.CommentSharealexkanswered a year agoMaxwell a year agoI understand. I will take your feedback back to the team.Amazon EC2 provides more control over how much storage your instance has, as you can attach an EBS volume of any size to an EC2 instance. Lightsail provides several pre-defined bundle sizes to simplify the user experience and cost, which can mean that we do not offer configurations like this.Thank you for your feedback!-MaxShare0Database backup i just have to do cron job to export sql every day to  google drive.CommentSharealexkanswered a year ago"


Question: Roles - changing tag depending on user assuming the role

"Hi,My first question - https://repost.aws/questions/QUS8M4w0jkS8iV6EzPmaRmag/ssh-key-managment-for-multiple-accountsIm trying to use "AWS system manager" - "session manager".As i was advised in my previous question, to be able to login into the EC2 instances located in multiple accounts, i will need to do something similar to https://aws.amazon.com/blogs/mt/vr-beneficios-session-manager/But the problem is that i need to have for each IAM user their own user in EC2 instance, as i found out, i need to pass tag "SSMSessionRunAs" with the value of the username to witch im login in.  But if i will use "group" roles (roles assigned for multiple users), they will be authenticating with the same user in EC2 instance, which will not work for me.Does that mean, that in my case i will need to create a role for each IAM user? or i can change tag of the role depending on the user assuming this role?Thank you very much.JoannFollowComment"

Answer: "0Accepted AnswerI can set tags, while assuming the role ---> https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html , so when user assumes the role, user also sets the tag, with value of user in EC2 instance.CommentShareJoann Babakanswered a year ago"


Question: "AWS User moved from US to AU, has USD-AUD & service fee question regarding fee surcharge scaling"

"As an AWS User, I permanently moved from the United States to Australia. My Bill is approximately $26.00 USD, which is $38.00 AUD. For the purposes of this question, we assume this exchange this exact rate is precise and have no questions regarding it. The Bill also has an AWS Service Fee surcharge for $0.01 USD, which is assumed to be $0.01 AUD.Will this AWS Service Fee surcharge ever not be $0.01 AUD? Does this charge scale or will it always, 100% or the time, be $0.01 AUD?Side note: can you tell I’ve used Jira a lot?FollowComment"

Answer: "0Hi,I've found this article which may answer your question: https://aws.amazon.com/premiumsupport/knowledge-center/non-us-currency/. If it's not quite specific enough, please feel welcome to reach out to our Billing Support team via a support case.Additionally, you may find this article helpful if you were thinking of changing the currency you're currently billed in: https://aws.amazon.com/blogs/aws/new-set-preferred-payment-currency-for-your-aws-account/.NicolaCommentShareEXPERTAWS Support- Nicolaanswered 4 months ago"


Question: New to AWS: What will be the best service to use to repeatedly ping an API endpoint? (And ideally stay in free tier)

"As I mentioned, I'm new to AWS and trying to plan out a personal project. I have an API that I want to get near-real-time updates from, but there is no notification service for that API (yet). My goal is to ping it at the rate limit to detect updates, and when certain updates come through, I want to ingest the data from those updates and store them. I have two current thoughts:EC2 instance to ping endpoint -> SQS message when I find the certain updates I need  -> lambda for data transformation (and another API call) -> store in dynamodb for later accessLimitations:If updates to this API come more often than the request rate limit, then I will need to make requests from multiple instances/different IP addressesTo stay up to date, this would require 24/7 EC2 uptime, which I imagine will be expensive and I would like to not spend much on this personal projectAlternate Idea would be the same process, but all API calls would come from lambda functions (called from the EC2 instance) to get more API requests and more data in the same time. Getting around the API rate limit to stay up to date on the data I'm looking forLimitations:24/7 EC2 uptime plus significantly more Lambda calls. This would be even more expensiveI would appreciate advice from anyone who has suggestions for other services or practices I could use to keep this as close to free-tier as possible!FollowComment"

Answer: "1Accepted AnswerDepending on some numbers using a Lambda might be worth it. Assumptions:A month has 2629800 seconds.You can use the cheapest Lambda for checks (128 MB), which costs 0.000001666675 per second + 0.0000002 per call in the cheapest regions (official docs).Tu is the time you take to detect updates.Tc is the time between checks (lambda executions).Your monthly cost for using Lambdas will be Tu * 0.000001666675 * (2629800 / Tc) +  (2629800 / Tc) *  0.0000002 =  (4.383Tu + 0.526)/TcYou don't have much control over Tu, although you can optimize the function to be quicker. Let's assume you're using something like Python and each check takes 200 ms. Now you have: Cost = 1.4/TcIf you accept making checks every second, we are talking about 1.4 USD / month, which is cheaper than any on demand EC2 instance in the cheapest regions. If you need quicker checks, having a very small EC2 instance such as the t4g.nano turned on 24/7 costs about 3 USD per month, again, in the cheapest regions. So it all depends on your definition of near-real-time updates.CommentShareCarlos Dominguesanswered a month agorePost-User-2167464 a month agoI appreciate your advice! This is very helpful!Share1You may want to use the Amazon EventBridge service to create a rule to schedule the execution of the Lambda function, which pings the API to fetch the data. Amazon EventBridge is a managed service for building event-driven applications. It costs only $1.00/million invocations per month after the initial 14 million free invocations per month. So it is very cheap than running and maintaining an EC2 instance.You can also use AWS Step Functions to orchestrate the Lambda functions to build a workflow.For more information, refer https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-run-lambda-schedule.htmlCommentSharezafar_khananswered a month ago"


Question: Cannot select options to "GRUB install devices" prompt

"Hi all,I was trying the tutorial in setting up an environment and building applications (https://docs.aws.amazon.com/robomaker/latest/dg/gs-build.html) when I encountered this problem. Specifically, after creating the environment, when I tried to download the 'Hello World' sample, the following showed up after a while:The GRUB boot loader was previously installed to a disk that is no longer present, or whose unique identifier has changed for some reason. It is importantto make sure that the installed GRUB core image stays in sync with GRUB modules and grub.cfg. Please check again to make sure that GRUB is written to the  appropriate boot devices.If you're unsure which drive is designated as boot drive by your BIOS, it is often a good idea to install GRUB to all of them.Note: it is possible to install GRUB to partition boot records as well, and someappropriate partitions are offered here. However, this forces GRUB to use the blocklist mechanism, which makes it less reliable, and therefore is not recommended.  1. /dev/nvme0n1 (51539 MB; ???)    3. none of the above  2. - /dev/nvme0n1p1 (51538 MB; /) (Enter the items you want to select, separated by spaces.) GRUB install devices:Nothing I typed worked; all it does is go into a new line after I press the ENTER key every time without any sort of response. Things I've tried:'1', then ENTER key'1. /dev/nvme0n1', then ENTER key'1.', then ENTER key'1 2', then ENTER key'1. /dev/nvme0n1 (51539 MB; ???)', then ENTER keyDoes anyone else have this problem / encountered this? This sounds like a ridiculously simple problem to solve, but I can't seem to find out how.FollowComment"

Answer: "0Hello,Could you please provide the following details:IDE URLAWS Region you were operating in,Instance type, andSoftware suiteThanks.CommentShareAWS-User-5010218answered 3 years ago0I was able to go around the issue (more on it below).I deleted that environment alreadyus-east-2 (Ohio), subnet-1f179453Not sure what this is (still a newbie), but is EC2 what you're looking for?ROS KineticI (kind of) fixed the issue by first doing an apt update and upgrade before doing anything else in new environment instances, which will trigger the GRUB install devices prompt but in a GUI instead, and that allowed me to select the partition to install. Sometimes when I did that it would tell meCould not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable) E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?which goes away after I leave it alone for some time (around 5-15 minutes).CommentSharekhaiyichinanswered 3 years ago0Thank you. Yes, I meant the EC2 instance. We tried to follow the steps and reproduce the issue, but we couldn't. Good to know that it is working for you. Please let us know if you have any further questions.CommentShareAWS-User-5010218answered 3 years ago"


Question: Referencing variables in Step Functions

"In my particular case I am trying to reference a value from the originating event that kicks off my step function the body of an SES email.For example, this is the originating input event:{   "eventType":"connected",   "GatewayEui":"abcd12345",   "timestamp":"2022-06-14T19:40:17.511Z"}And the SES Send Email node is configured like so:{  "ConfigurationSetName": "test",  "Source": "noreply@abc.abc",  "Destination": {    "ToAddresses": [      "johndoe@abc.def"    ]  },  "Message": {    "Subject": {      "Data": "Device disconnected"    },    "Body": {      "Html": {        "Data": "$.GatewayEui is disconnected",        "Charset": "UTF-8"      ....This doesn't work - but notice I am trying to get the GatewayEui and reference it in the body of the email, but confusingly the only way I can figure out how to reach it is as as a JSON key/value.  But all I want is the value.How can I achieve this?Also, two HUGE feature requests:Please let us add in an example event JSON into the visual workflow so we can inspect how it would be referenced in particular nodes while we're editing the flow.Please add a debug node!  Which would let us log specific branches of a flow and not others.FollowComment"

Answer: 0Have you tried using States.Format intrinsic function - https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.htmlCommentShareEXPERTIndranil Banerjee AWSanswered a year ago


Question: Anomalies in Cognito User Pool settings

The problem is the reset of the pre sign-up cognito trigger lambda function(Node.js) . The trigger disappears from the selected one and is reset to 'None'.  I tried look into CloudTrail to see if anyone or anything is changing this setting but couldn't find anything. Any ideas on the places to look for logs to see what is the reason for this setting reset?  It happened in both prod and stage account twice already. Maybe is related to Amplify somehow?FollowComment

Answer: "1Accepted AnswerHi Daniel,The issue you're experiencing is most likely related to the bug described here - https://github.com/aws-amplify/amplify-cli/issues/8997. Please follow the GitHub issue for further updates on the fix and workaround. Thanks!CommentShareAmplificationanswered a year agoDaniel Gaina a year agoThank you for sharing this with me. I can see is the exact problem i have. I will monitor for further updates on the fix and workaround. Million thanks! 🙏Share0Hi,Do you have any automation (CloudFormation, CLI, API calls) that update the user pool using UpdateUserPool APIs? You can look in CloudTrail logs for UpdateUserPool events and inspect the input.The behavior of this API is that all inputs need to be provided or otherwise default values will be used, the correct way to do that is to first call DescribeUserPool API, edit/add the settings that you would like to change then pass the entire payload to UpdateUserPool API call.CommentShareEXPERTMahmoud Matoukanswered a year ago"


Question: RDS stucked when installing Postgis Extension (2.3.7)

"Hey,I have setup an RDS Aurora cluster with PostgreSQL 9.6.9 (I also tried 9.6.11). I am trying to run "CREATE EXTENSION postgis" on it. But it simply does not do anything (Even after waiting for 20 minutes). Installing other extensions is fine. Is there anything I can do to get it resolved?Regards,ThomasFollowComment"

Answer: 0Can be closed. The issue was i was running an Aurora cluster with a Writer only. Adding a reader fixed the issue instantly.CommentShareAlien2150answered 4 years ago0The issue was i was running an Aurora cluster with a Writer only. Adding a reader fixed the issue instantly!CommentShareAlien2150answered 3 years ago


Question: How to provision CloudFront distribution with ACM certificate in CloudFormation?

"I am trying to achieve the following.First, deploy a CloudFormation stack to region ap-southeast-2 containing:An S3 bucketA CloudFront distribution using the S3 bucket as originA CNAME in Route53 under a pre-existing HostedZone pointing to the CloudFront distributionThen, I want to deploy another CloudFormation stack to region us-east-1 with:An ACM SSL certificate for the domain (CNAME) registered in point 3, above.Then, I want the certificate created in point 4, above, to be associated with the CloudFront distribution created in point 2, above, as the Viewer Certificate.I cannot set the CloudFront Viewer Certificate in the first CloudFormation template because it does not exist yet. It will only exist after the second stack is created.Is there a way to achieve this with CloudFormation? Is there anything I can do with CloudFormation StackSets to achieve this?FollowComment"

Answer: "0Yes, you can use StackSets to do this in conjunction with using Conditions in the template. Basically, set a condition based on the current region and create the resources for that region. You can define that the stacks are created sequentially with us-east-1 first then ap-southeast-2. See: Stack set operation options (see Region concurrency),CommentShareEXPERTkentradanswered a year agoGabriel Oliveira a year agoOK, but creating them sequentially is not enough. After CloudFormation creates the second stack with the ACM certificate, it needs to go back to the first stack and assign that new ACM certificate to the CloudFront distribution that was created in that first stack. How does your solution achieve that?Also, the link you provided takes me back to this very same question in re:Post.Sharekentrad EXPERTa year agoIf fixed the link. You will have to run the template in us-east-1 first so you have the cert. You could run it first in ap-southeast-2 and have a condition on whether to setup HTTPS. Then go and create the cert in us-east-1 and then go back and update the stack with the new cert information in ap-southeast-2.ShareGabriel Oliveira a year agoRight, kentrad. But that would be a procedural solution. If I am deploying the templates via CLI, I would have to perform 3 steps:Deploy the first stackDeploy the second stackUpdate the first stackEven if points 1 and 2 could be performed together as a StackSet, I would still have two separate steps, for it would require me to trigger an update to link the certificate with CloudFront in the end.I reckon CloudFormation should offer a fully declarative solution for this problem, and the circular dependency should be resolved by AWS.Sharekentrad EXPERTa year agoI think what is missing in your scenario is when you create a certificate, there is currently no way to specify  where to install it in CloudFormation for AWS::CertificateManager::Certificate. So, there is no way to create the this circular dependency.Share"


Question: Deploy lambdas in parallel with CodeBuild

"I have a CodePipeline configured with my single GitHub repo to trigger build & deploy on any commit.My repo contains dozens of individual lambda functions.The pre_build step does a full clean & restore. Then in the build step, I have a long list of commands like this:- dotnet build -c Debug- cd folder1- dotnet lambda deploy-function --configuration Debug --region $Region --config-file aws-lambda-tools-test.json- cd ../folder2- dotnet lambda deploy-function --configuration Debug --region $Region --config-file aws-lambda-tools-test.json- cd ../folder3- dotnet lambda deploy-function --configuration Debug --region $Region --config-file aws-lambda-tools-test.jsonMy question is, is there any way for me to execute these deployments in parallel?FollowComment"

Answer: "1Accepted AnswerCodeBuild batch support a matrix build where a build matrix defines tasks with different configurations that run in parallel. CodeBuild creates a separate build for each possible configuration combination.Consider extending your pipeline to include CodeDeploy which supports deployment groups for grouping AWS Lambda functions in a CodeDeploy deployment.CommentShareRoBanswered a year agomaxmoore14 a year agoMatrix build (and the other options outlined in that linked doc) is what I was looking for. Thanks.Share-3I would suggest to pick CodeBuild compute type to support the required parallelism, and then use a Makefile with parallel execution.CommentShareAlex_Panswered a year ago"


Question: Can we set to  Password never expires With EC Instance Windows Server 2016 using puppet script

"Can we set to  Password never expires With EC Instance Windows Server 2016 using puppet script?As per AWS documentation, came to know that With Windows Server 2016 and later, Password never expires is disabled for the local administrator. With Windows Server 2012 R2 and earlier, Password never expires is enabled for the local administrator.FollowComment"

Answer: "1Hi thereYes you can  set the local user account password to be never expired on Windows Server 2016 and later. By following these steps .1 RDP to your windows instance.2 start puppet cmd3 Type the following command   wmic UserAccount where "Name='username'" set PasswordExpires=Falseand press Enter key. Replace the username with your administrator. When it shows "Property(s) update successful", the user’s password expiration is disabled. To check that you are successfully change your password never expired type the following command:                 net user administratorNow you can see that your password expires NEVER.CommentShareNosiphiwoanswered a year ago"


Question: aws greengrass lambda function connection

"Hello, there was a problem connecting the Greengrass Lambda functionhttps://docs.aws.amazon.com/ko_kr/greengrass/v1/developerguide/configs-core.html This is the tutorial I was working on, but there was a problem when distributing the lambda function in the last stage.Deployment failed. And error message is Deployment 42c9ea0c-ac1d-44ce-bcbd-b7e7ca7d0163 of type NewDeployment for group 1e3d2692-8e85-499c-9f70-316a51233e3b failed error: The following cgroup subsystems are not mounted: devices, memory.How can i fix this?FollowComment"

Answer: "0The cgroup memory must be mounted and enabled to run Lambda functions. You can find more information in the Greengrass documentation about Supported platforms and requirements.The Greengrass dependencies checker can verify if your device has the dependencies to run Greengrass this includes also required cgroups.You can find information howto enable cgroups in the documentation for your Linux distribution.KR,PhilippCommentSharePhilipp Sanswered a year ago0Only cgroup v1 is supported for Greengrass lambdas. You must disable cgroup v2 in order to use cgroup v1. You can also work around this by simply using the no-container mode of Greengrass so that cgroups aren't required.To disable cgroup v2, see: https://github.com/aws-samples/aws-greengrass-samples/issues/41#issuecomment-974845635CommentShareEXPERTMichaelDombrowski-AWSanswered a year ago"


Question: %additional_python_modules failure in glue notebook

"while additional_python_modules work in Glue version 4.0, but since I cannot change the glue version in glue notebook (Glue notebook supports 2.0 and 3.0 only), the whl file pointing to s3 path fails to import in glue notebook. Its a deceiving message when it says it actually used the whl file. Can you fix that in glue notebook or remove it from %help. I learnt it the hard way.FollowComment"

Answer: "0Hello,I see that you are unable to import the python package into your Glue notebook. You tried to install the dependency from whl file stored in s3 bucket. I have tried it on my test environment and I was able to import my module using whl file stored in s3 bucket. Here are the steps that I followed:For testing purpose, I uploaded this whl file into my s3 bucket.Next, I started a Glue notebook and then I used a magic to give the s3 bucket path of my wheel file. The magic command was something like this:%additional_python_modules <my-s3-uri>I executed this and then my session got created. When I tried to import psycopg2 module, it was working fine for me.Therefore, I request you to verify the following things on your end:Do check whether or not you have followed the above steps.Please do note that once the session is created, it is not possible to run the magic command. You will have to stop and then restart the notebook to run a magic (%additional_python_modules in your case). These magic commands will only run at the time of session creation.Confirm whether the whl file that you are trying to install is compatible with the Glue version you are using.If you are still facing the same issue, then please share me the steps you were trying to follow along with the error that you were facing.CommentShareSUPPORT ENGINEERChaituanswered 3 months agorePost-User-0701661 3 months agoThanks for the response, I tried importing the psycopg2 package that you provided in the link and it work as expected. However when I tried to import Django package from official website downloaded from https://files.pythonhosted.org/packages/89/86/59e237f7176cfc1544446914fa329fd560bb8fce46be52dd7af5dc7c54f9/Django-4.1.7-py3-none-any.whlIt wont work, See the screenshot attached.I am wondering if there is any difference in packaging ?BTW same Django package works perfectly fine in glue 4.0Share0Screenshot attachedCommentSharerePost-User-0701661answered 3 months agoChaitu SUPPORT ENGINEER3 months agoThe django python whl file that you are trying to use is not supported by Glue 3.0 because the Django 4.1.7 supports python version >=3.8 but the python version in Glue 3.0 is 3.7. This is the reason why it was not working in Glue 3.0 and as the Glue 4.0 has python version 3.10, it worked there. Refer -  https://docs.aws.amazon.com/glue/latest/dg/release-notes.htmlThus, please try to download this file - https://files.pythonhosted.org/packages/57/12/da22535f809b8c06c8d58eaf236ec8683ffd4e1dc4eced175b174e6446fa/Django-3.2.18-py3-none-any.whl and then use it in your Glue notebook, it should work fine. I have tested this on my end as well.Share0Hey Chaitu, Thank you for the response. Appreciate it. However as an end user had the notebook errored out while loading the library, I would have saved much time and pain that i wasted in troubleshooting. I am sure as you won't expect customers to be aware of these nuances by expecting them to read all the documentation. Could have been avoided by failing fast.Now this has opened a pandora box.I have to downgrade all my glue notebooks from 4.0 to 3.0 so that we maintain same package versions and compatibility in dev notebooks and prod glue jobs. Please pass this feedback to the maintainers.CommentShareSunilanswered 3 months agoChaitu SUPPORT ENGINEER3 months agoI will pass on the feedback as suggested by you. So, you could have tried it with the normal Glue job set to version 3.0 and then observed the "error logs" for the job. It will show the error message if it fails to install the python library. In fact, I was able to find this out by the same way. Having said that, it could have been much easier & time saving for you if the error was displayed on the notebook itself. I will convey this to the relevant team.Share"


Question: "My automations are tagged in system manager AWS, but I can't find them"

""StartAutomationExecution":https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_StartAutomationExecution.htmlThe "StartAutomationExecution" call supports tagging, but I could not find a way to read it from the "GetAutomationExecution" and "DescribeAutomationExecutions" calls either.What is the best way to retrieve the tag I put on the automation that was executed?FollowComment"

Answer: "0If you want to find the tag key/value, you can use the ListTagsForResources that lists tags for a specified resource like this:{    "ResourceType": "Automation",    "ResourceId": "EXAMPLE_ID"}You can to retrieve the execution with the tag key using DescribeAutomationExecutions like this:{    "Filters":        [            {                "Key": "TagKey",                "Values": [ "EXAMPLE_KEY" ]            }        ]}CommentShareJoe Kanganswered 6 months agocombii 6 months agoThanks for the response. Let's imagine you have a user executing the automation and you are tagging the automation with his username. If you didn't know what the username tag of every automation was, how would you find out?ShareJoe Kang 6 months agoInteresting, what would have been useful is if DescribeAutomationExecutions could filter by tagValue. Then you could have filtered by the username value.Share"


Question: S3 Bucket Default Storage Class

Is it possible to configure a default storage class for a S3 bucket other than Standard. For example can I configure the default storage class of a bucket to Intelligent Tiering so that any objects which are PUT to the bucket will be stored in Intelligent Tiering unless explicitly set otherwise in the initial PUT? This would save the transition cost of each object.FollowComment

Answer: "0Unfortunately the current behavior of S3 is that everything that every object put on S3 that has no explicit storage class set will be stored with S3 Standard Storage class: https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html#sc-howtosetI recommend that you reach out to your AWS Account Representative to address these need. They will make sure that your use case will be added to our list of customer demands which will influence our planning for future development and prioritization.CommentShareEXPERTAndreas Seemuelleranswered a year ago0There's no way to set a "default" storage class, but to add on to @Adreas_S's comment, you can specify the storage class when creating/uploading a new object. For example, when creating objects using the PUT Object, POST Object, and Initiate Multipart Upload APIs, you add the x-amz-storage-class request header to specify a storage class. If you don't add this header, Amazon S3 uses Standard, the default storage class.Same holds true if you happen to be uploading objects via the AWS Command line you can use the --storage-class INTELLIGENT_TIERING parameter. For example:aws s3 cp <source file> s3://<destinationBucket> --storage-class INTELLIGENT_TIERINGThis is all outlined in Andrea's link above.CommentShareAWSJoeanswered a year ago"


Question: Not working DNSSEC on hosted zones

"I create DS record with the route 53 keys data in Domain register .cl, its works and validate the keys, but when i test DNSSEC for the hosted zones in route 53, i recieve No DS records found for norcomsys.cl in the cl zone, the register are insecure?FollowComment"

Answer: "0According to DNSVIZ (https://dnsviz.net/d/norcomsys.cl/dnssec/), DNSSEC is enabled and working correctly for norcomsys.clThe .cl TLD is returning DS record for norcomsys.cl$ dig norcomsys.cl @cl2-tld.d-zone.ca DS +short9037 13 2 8F772DDB3DC7AABFE4094D0F9110E39EBED14778953F59321F6D82C8 D58B5728Google DNS which performs DNSSEC validation, is returning RRSIG when resolving norcomsys.cl$ dig norcomsys.cl @8.8.8.8 +dnssec +short186.67.10.50A 13 2 300 20220603091147 20220603070647 62283 norcomsys.cl. K6yz3hyKboDYYr9METvaSS0nJd2kld4OqGh9wRNosar2qwM+e4BeC8Ml lJjhFVJZO/FLam8JF0/Q73VJ/cgLAg==How are you checking for DS record for  norcomsys.cl?CommentShareWernerWanswered a year ago0There was a problems with nic.cl, its take some time to update and validate the DS record, when they validate the DS record, the problems was resolve.CommentSharerePost-User-1548718answered a year ago"


Question: How can I tell if my Notebook instance is frozen?

I am running a Sagemaker Notebook instance. How can I tell if my Notebook is frozen or just taking a long time? I am using a 24xlarge and querying from Athena in parallel and it seems to be stuck on the same query for a long time. How can I tell if I need more Memory or more VCPUs?FollowComment

Answer: "0Accepted AnswerHi there,Greetings for the day!I understand that you wanted to know how can you determine if you need more VCPU or memory when your SageMaker Notebook Instance is frozen or just taking a long time?I’d like to inform you that If your Sagemaker Notebook instance is taking a long time, you can check if it is frozen or still running by monitoring the CPU and memory usage. If the CPU usage is low or zero, it may be frozen.If the CPU usage is high but the memory usage is low, you may need more VCPUs and If the memory usage is high, you may need more memory.You can check it via SageMaker Notebook Instance terminal:To see the memory and CPU information in detail , kindly follow the below instructions:-[1] Start Your Notebook Instance[2] Go to the  Jupyter Home Page[3] Right hand side ,Click on DropDown Option “New”[4] Select “Terminal”.In the Jupyter terminal, Run the below commands to see the information of Memory and CPUs.[+] To see the memory information:$ free -h=> output of “free -h” will provide the information of total memory, used memory, free memory, shared memory etc in human readable form.[+] To see the CPU information, you can run any of the commands:$ mpstat -u=> Output of “mpstat -u” consists of different fields like %guest, %gnice, %steal etc.%stealShow the percentage of time spent in involuntary wait by the virtual CPU or CPUs while the hyper‐visor was servicing another virtual processor.%guestShow the percentage of time spent by the CPU or CPUs to run a virtual processor.%idleShow the percentage of time that the CPU or CPUs were idle and the system did not have an out‐standing disk I/O request.many more.. You can find more detail about the each field of mpstat command by visiting the manual page of it. To see the manual page of mpstat command, use “$ man mpstst”.Along with “mpstat -u”, you can also try the below listed commands to get information about the cpu:$ lscpu$ cat /proc/cpuinfo$ topAdditionally, You can also check the cloudwatch logs for any errors or warnings that may indicate the cause of the issue. Most of the time, Cloud watch logs helps to find out the root cause of the issue.You can find the CloudWatch logs under CloudWatch → Log Groups → /aws/sagemaker/NotebookInstances -> Notebook Instance NameBased on the analysis , you can select different Notebook Instance type, You can find more detail of SageMaker Instance Here [1].I request you to kindly follow the above suggested workarounds.If you have any difficulty or run into any issue, Please reach out to AWS Support[+] (Sagemaker), along with your issue/use case in details and we would be happy to assist you further.[+] Creating support cases and case management - https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-casehttps://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-caseI hope this information would be useful to you.Thank you!REFERENCES:[1] https://aws.amazon.com/sagemaker/pricing/CommentShareSUPPORT ENGINEERrePost-User-2736031answered a month agoEXPERTRiku_Kobayashireviewed a month ago0How about SSH connection to Sagemaker Notebook to check CPU and memory load?If the resulting load is too high, we will need to consider changing the specifications.CommentShareEXPERTRiku_Kobayashianswered a month ago"


Question: Open connections to Timestream after credential expires

"Lets say I open a connection to Timestream, from a long-running service. The connection potentially stays open for multiple days.Meanwhile, the connection was opened by authority of a temporary cert.. which expires a few hours in.What happens to the connection?Does it stay valid, or does it get auto-terminated somehow?FollowComment"

Answer: "0When you establish a connection to AWS Timestream using temporary credentials, the connection itself will remain active as long as it's being used, even if the credentials have expired. AWS Timestream does not automatically close the connection when the credentials expire, but the expired credentials will cause any further requests to fail with an "AccessDenied" or a similar error.CommentShareEXPERTsedat_salmananswered 2 months agorePost-User-7935554 2 months agoThe first part of your post made me feel relieved. The second part made me anxious again :DWhat do you mean, specifically, by "further requests"?Do you mean any brand new connections, or... ?Share"


Question: DELETE OLD LOG FILES ON AWS RDS

"Is there a way to delete old log files and old backup logs from AWS RDS ? I am getting an alert "Storage size 200 GB is approaching the maximum storage ".On the ON-Premise databases, we would go on the database server and delete old files, what is the solution provided by AWS RDS, OracleFollowComment"

Answer: "1hi,One of the options is to use RMAN tasks on Amazon RDS to backup and move the file to Amazon S3. The procedure to do so is here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.Oracle.CommonDBATasks.RMAN.html.  RMAN can be used to backup archived redo logs to Amazon S3. You can create a script that connects to the RDS instance and uses RMAN to backup the logs to S3. You can then schedule this script to run periodically using the operating system's scheduler (e.g., cron on Linux)ThanksArunCommentShareArunanswered 3 months ago"


Question: SSH connection to a newly created Ubuntu EC2 instance fails

"Hello,I launched an Ubuntu EC2 instance from AWS console, and set the security rule to permit SSH connection from anywhere.  I chose Ubuntu from "Quick Start" menu as the screen capture shown below.When I tried to connect to it from my laptop via SSH, it failed with "Connection timeout". I also tried  the "connect" to the instance button on AWS console, but it told that the instance was somehow unreachable.I was trying to connect like this:ssh -i "$HOME/.ssh/my-key.pem" ubuntu@<pulibc-ip-here>.ap-northeast-1.compute.amazonaws.comWas there any configuration or procedure I was missing?I did just the same way as I successfully connected about a half year ago, so I guess there might have been a change on AWS.FollowComment"

Answer: "3Accepted AnswerHi,Have you seen the following troubleshooting information in the EC2 User Guide? https://docs.aws.amazon.com/en_us/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.htmlCheck the section that talks specifically about "Connection timed out": https://docs.aws.amazon.com/en_us/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html#TroubleshootingInstancesConnectionTimeoutIt's not clear to me whether you've successfully connected to this instance or another instance half a year ago? If you connected to this instance, have you stopped and then started your instance since you last connected to it? When you stop your instance, the public IPv4 address that Amazon EC2 automatically assigned to the instance on launch or start is lost. When you start your instance, a new public IPv4 address is assigned. Check that this IP address appears in your connection command.I hope the above helps!Kind regards,MarilynCommentShareMarilynAWSanswered 10 months ago0Hi,It turned out that I mistakenly deleted the internet gateway attached to the VPC of that instance. So re-creating an internet gateway and attaching it to the VPC again solved the connection issue.By the way, I meant that I could connect other instances a half year ago, not the one I deal with now. But they were inside the same VPC.The provided link reminded me to refer the route of VPC, that was the cause of this issue.Thank you for letting me know :-)Best wishes,RyosukeCommentSharepathtrkanswered 10 months agoMarilynAWS 10 months agoGlad you're sorted now! :)Share"


Question: max_wal_senders invalid setting

"Hi guys,We have a single Postgresql 10 rds instance replicating to another instance and haven't changed any default parametersWe had a brief outage on Sunday, looking at the event logs on the main server we see the following:Sun, 10 Mar 2019 13:11:11 GMT   Recovery of the DB instance has started. Recovery time will vary with the amount of data to be recovered.Sun, 10 Mar 2019 13:15:27 GMT   The parameter max_wal_senders was set to a value incompatible with replication. It has been adjusted from 10 to 15.Sun, 10 Mar 2019 13:16:01 GMT   Recovery of the DB instance is complete.Sun, 10 Mar 2019 13:16:28 GMT   The parameter max_wal_senders was set to a value incompatible with replication. It has been adjusted from 10 to 15.Sun, 10 Mar 2019 13:16:31 GMT   DB instance restartedI'm not entirely sure from this what the cause of the outage was... looking at the default parameters our max_wal_senders is set to 10 stillDoes anyone have any ideas how a) i can work out what happened? and b) if 10 is actually a valid value for that setting?thanks!DanFollowComment"

Answer: "0Message: "The parameter max_wal_senders was set to a value incompatible with replication. It has been adjusted from 10 to 15.","Please be assured the above message is a known message when an RDS instance is restarted, this event is harmless and has no performance impact on the instance. Kindly note that the internal team is aware of this event occurrence and is working on rolling out a fix.Hence, currently, there is no requirement of setting up the above parameter and you can safely ignore this Warning message until the internal team rolls out a fix at the backend.Unfortunately, I do not have a valid ETA for when the fix will be rolled out by internal team, however please be assured that this message is harmless and it's level is "Warning" and no action is required from your end.CommentShareAWS-User-9317441answered 4 years ago0ok, thanks for thatCommentSharedanturnanswered 4 years ago"


Question: Can't create DMS Instance with The Replication Subnet Group already exists

"Hello!I've got 'SYSTEM ERROR MESSAGE:The Replication Subnet Group 'default-vpc-c2c6c6a5' already exists.' when I tried to create a new DMS Instance. I tried to recreate IAM role dms-vpc-role, tried to create another VPC. I'm using almost fresh AWS account so I don't think that I have already cracked it.Edited by: ZnoWM on Feb 16, 2020 5:34 PMFollowComment"

Answer: "0HiI just had the same problem when creating a new replication instance in AWS console. First time I hit the create button I had another error, I fixed it, hit create button again and then I had the same error 'Subnet Group already exists'.I went to the 'Subnet groups' page (in DMS) and manually deleted the existing subnet group. Then the creation went just fine.Seems like the subnet group creation is not rolled back when an error occurs.CommentSharemasoanswered 3 years ago0Maso's solution worked for me too.CommentSharekyorgyanswered 2 years ago"


Question: How to use postgres rest api in AWS

"Hi Team,we are currently using the postgrest api (https://postgrest.org/en/stable/) in AWS EC2 instance .Do we have any service offered by AWS.  Please do let me know which service have to used  ?  we are planning to use it on Production not sure which service is reliable and provided by AWS.FollowComment"

Answer: "0Hi,You can use postgree REST API, refer to the below article.https://medium.com/the-dev-caf%C3%A9/creating-a-serverless-rest-api-with-node-js-aws-lambda-api-gateway-rds-and-postgresql-303b0baac834CommentSharelearn2skillsanswered a year ago0Hi,There are some factors I think you can consider before choose the service to be used. One of example, number of expected traffic , traffic pattern, scaleability and security. You can consider some of services combination:In high level overview with assumption you will handle huge write and read traffic:Api management (for security, canary deployment) + Load balancing (service to distribute traffic) + compute layer (to run Postgrest).For Api management you can use = API GatewayFor Load balancing you can use = ALBFor compute layer you can use = EC2+auto scaling group, EKS, or Lambda. Since i read that Postgrest has docker official image also = https://hub.docker.com/r/postgrest/postgrest/.For caching you can use = ElasticCacheFor message queing you can use = SQSFor security protection such as to handle from SQL injection can use = AWS WAF integrated with Api gateway or ALBFor connection pooling you can use = RDS ProxySample solution combination can be like this:Api gateway + ALB + EC2 and auto-scaling groupAPI gateway + ALB + EKS with KarpenterApi gateway + ALB + LambdaPlease read this reference to help you decide if you want to choose serverless or kubernetes: https://aws.amazon.com/architecture/serverless/serverless-or-kubernetes/.Then, for frequent data which required sub mili latency you can enable caching to some API endpoint Api gateway + Elastic cache. And you can also consider to push some write request into SQS for queue before handling it to Postgrest. For security you can integrate AWS WAF to protect api gateway or ALB. And finally to improve your api you can consider use RDS Proxy to help with connectoin pooling (multiplexing) management.Please alo do due diligence of above services and feature availability in your region: https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/Thank you hope it can give some insight for you.CommentSharerePost-User-7694765answered 5 months ago"


Question: Get Customer Input block throws error when using Lex as customer input

"I am trying to configure lex bot in AWS Connect. I have created a demo Lex bot that is working fine in Lex but after integrating it with Aws Connect in the Get Customer Input block in the contact flow, it throws an error. I have checked the error in CloudWatch and you can see the error following:{     "Results": "Error",     "ContactId": "contactId",     "ContactFlowId": "COntactFlowId",     "ContactFlowName": "TalkToOctaveAgent2",     "ContactFlowModuleType": "GetUserInput",     "Timestamp": "2022-03-25T05:30:08.252Z" }I can't recognize where this error comes from. I have set lex intents in the Get Customer Input block with proper names as in Lex. Please let me know how I can resolve this error.FollowComment"

Answer: 0You might find this checklist useful to troubleshoot whether you have followed all the pre-requisiteshttps://docs.aws.amazon.com/connect/latest/adminguide/amazon-lex.htmlCommentShareKishananswered a year ago


Question: Enabling S3 Encryption-at-rest on a go-forward basis with s3fs

"Hi,We have some buckets (have been around for a while, approx 200GB+ data) and we want to turn on encryption-at-rest using SSE-S3 (the most "transparent" way) https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-encryption.htmlThe S3 buckets are mounted to our Linux VMs using S3FShttps://github.com/s3fs-fuse/s3fs-fusewhich has support for this (seems fairly transparent)So, it seems like the way this works is that you can only enable this on files on a go-forward basis so the older files that already exist will not be in encrypted-at-rest (which is ok, we can backfill this later)Has anybody tried to do this before using this combo?  If we mount the bucket using s3fs with -o use_sse option, what will happen as the files will be half-and-half?Will it "just work"?  s3fs will be mounted with -o use_sse and it will be able to handle files that are BOTH the old way (not encrypted-at-rest) and the newer files (encrypted-at-rest) ... we can then start backfilling the older files and we have time or will this fail catastrophically the minute we mount the s3 bucket :(Is the solution to just start a new bucket and do the SSE-S3 and then just start moving the files over (we have done this before in terms of having code in our application check for a file in multiple buckets before giving up)Of course, we will test all this stuff, just wanted to ask a quick question in case we are worried about this too much and if this is a "no big deal" or "be very careful"Thanks!FollowComment"

Answer: 0Initial testing seems to indicate that SSE-S3 is really transparent and S3FS does not even need to be configured differently (no need for  -o use_sse).  S3FS can handle both encrypted and non-encrypted files (same bucket) with no problemsCommentShareAWS-User-3488042answered a year ago


"Hi all,I run a simple query on Athena using Athena engine version 3.create table if not exists "test"."total_customer" as select *from "A_customer" union allselect *from "B_customer"And I faced this error:HIVE_PATH_ALREADY_EXISTS: Target directory for table 'test.total_customer' already exists: s3://xxxx/total_customer/2022/11/27/tables/ef02d72d-75e2-4b14-b53f-95d591094cfa. You may need to manually clean the data at location 's3://xxxx/total_customer/2022/11/27/tables/ef02d72d-75e2-4b14-b53f-95d591094cfa' before retrying. Athena will not delete data in your account.The weird thing is it works perfectly when I use UNION instead of UNION ALL. So I'm not sure if this is Athena engine version 3 error or the query error.Thank you guys for taking a look at my issue.FollowComment"

Answer: 0Thank you for reporting this and sorry for the inconvenience. There is a fix that will go out for this shortly.CommentShareTheoanswered 6 months agorePost-User-2631176 6 months agoThank Theo for supporting. Waiting for your reply.Share