Question: Creating EC2 Ingress rule in C#

"I'm trying to create an ingress rule in C# and I'm getting an error at runtime.Here's the relevant code:`        ///////////BEGIN Set Vars//////////////////////        ///////////////////////////////////////////////        Amazon.EC2.AmazonEC2Client ec2Client = new Amazon.EC2.AmazonEC2Client();        Amazon.EC2.Model.AuthorizeSecurityGroupIngressRequest secRequest = new        **Amazon.EC2.Model.AuthorizeSecurityGroupIngressRequest();        Amazon.EC2.Model.IpPermission ipPerm = new Amazon.EC2.Model.IpPermission();        Amazon.EC2.Model.IpRange ipRange = new Amazon.EC2.Model.IpRange();        List<Amazon.EC2.Model.IpPermission> ipRangeList = new List<Amazon.EC2.Model.IpPermission>();        ///////////////////////////////////////////////        ///////////END Set Vars////////////////////////        ///////////////////////////////////////////////        ///////////////////////////////////////////////        ///////////BEGIN IP Range//////////////////////        ///////////////////////////////////////////////        ipRange.CidrIp = "5.5.5.10/32";        ipRange.Description = "My new IP rule";        ipRangeList.Add(ipPerm);        ///////////////////////////////////////////////        ///////////END IP Range////////////////////////        ///////////////////////////////////////////////        ///////////////////////////////////////////////        ///////////BEGIN IP Perms//////////////////////        ///////////////////////////////////////////////        ipPerm.IpProtocol = "tcp";        ipPerm.ToPort = 3389;        ipPerm.FromPort = 3389;        ipPerm.Ipv4Ranges.AddRange((IEnumerable<Amazon.EC2.Model.IpRange>)ipRangeList);        ///////////////////////////////////////////////        ///////////END IP Perms////////////////////////        ///////////////////////////////////////////////`If I just try to add ipRange as a range to ipPerm, the precompiler complains that it needs to be type of List<Amazon.EC2.Model.IpPermission>.When I use the code above and cast it to List<Amazon.EC2.Model.IpPermission>, the precompiler gets happy, but I get a runtime error:**  Message=Unable to cast object of type 'System.Collections.Generic.List1[Amazon.EC2.Model.IpPermission]' to type 'System.Collections.Generic.IEnumerable1[Amazon.EC2.Model.IpRange]'.Source=System.Private.CoreLibStackTrace:at System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void* toTypeHnd, Object obj)at AWSFirewall.Program.Main(String[] args) in C:\Users\SeanMcCown\source\repos\AWSFirewall\Program.cs:line 44**FollowComment"

Answer: "1You are trying to cast incompatible types, so the .NET runtime generates an exception. You want to add a list of IpPermission objects into a list of IpRange objects, which is not possible.Please refer to the AWS SDK documentation about Updating security groups.There, you can find a code snippet that shows you how to add an ingress rule to a security group:async Task AddIngressRuleAsync(IAmazonEC2 eC2Client, string groupID, string ipAddress, int port){  // Create an object to hold the request information for the rule.  // It uses an IpPermission object to hold the IP information for the rule.  var ingressRequest = new AuthorizeSecurityGroupIngressRequest { GroupId = groupID };  ingressRequest.IpPermissions.Add(new IpPermission  {    IpProtocol = "tcp",    FromPort = port,    ToPort = port,    Ipv4Ranges = new List<IpRange> { new IpRange { CidrIp = ipAddress } }  });  // Create the inbound rule for the security group  var responseIngress = await eC2Client.AuthorizeSecurityGroupIngressAsync(ingressRequest);  Console.WriteLine($"\nNew RDP rule was written in {groupID} for {ipAddress}.");  Console.WriteLine($"Result: {responseIngress.HttpStatusCode}");}You can then use this method to update the security group with your desired permissions:var ec2Client = new Amazon.EC2.AmazonEC2Client();var sgId = "your-security-group-id";var cidr = "5.5.5.10/32";var port = 3389;await AddIngressRuleAsync(ec2Client, sgId, cidr, port);CommentShareDmitry Balabanovanswered a year ago"


Question: Aurora Read replicas for failover instead of multi - AZ

"Hi, I am new to AWS and new to databases. While reading the Aurora FAQs:"Q: How can I improve upon the availability of a single Amazon Aurora database?You can add Amazon Aurora Replicas. Aurora Replicas in the same AWS Region share the same underlying storage as the primary instance. Any Aurora Replica can be promoted to become primary without any data loss and therefore can be used for enhancing fault tolerance in the event of a primary DB Instance failure. To increase database availability, simply create 1 to 15 replicas, in any of 3 AZs, and Amazon RDS will automatically include them in failover primary selection in the event of a database outage."Does this mean there is no concept of multi-AZ with Aurora like with other Amazon RDS DBs ? and I can just use read replicas for availability as well as performance purposes ?FollowComment"

Answer: "0Accepted AnswerMulti-AZ is very much applicable with Amazon Aurora. With Aurora compute and storage is decoupled. Storage will always get replicated across 3 AZs, regardless of you compute configuration. When you create read replicas, you can spread them across multiple AZs as well.Read replicas in Aurora can be used not only for reads, but they also serve HA purpose. Although keep in mind that the readers will lag by several miliseconds (which is acceptable for vast majority of customers). This is a significant improvement over regular RDS multi-AZ HA, which only works as a standby, and doesn't allow reads.So, to summarize: Multi-AZ HA works on both compute and storage layers with Aurora, and the secondary nodes can serve as HA as well as read replicas.CommentShareAWS-User-8357187answered 3 years ago"


Question: Instance fails to start due to "insufficient capacity" error

"Over the past 2 days, I have been trying to start one of my instances, and repeatedly get "insufficient capacity" errors.  I checked the support article regarding this, and none of the cases match my issue.  I am not launching one, just trying to start an existing one.  And as I mentioned, it has been a problem for over two days now (i.e. the "wait a few minutes" solution isn't working either).  I also have tried starting this one when no other instance of mine is running, so I don't think it's a resource issue regarding me having too many instances running.It is only one particular instance having this problem.  The others all start fine.  I need this instance to be able to start so that I can perform tasks on it related to our customers.Is there any other solution besides the ones already given in the Troubleshooting section?FollowCommentChris_G EXPERT10 months agoWhat instance type are you trying to use?ShareBrettski-AWS EXPERT10 months agoAlso: What region and availability zone?Share"

Answer: "1This error doesn't mean you might have too many resources running. It means AWS doesn't have enough capacity for that particular instance type in that particular availability zone. So as long as they don't have enough capacity for it in that availability zone, you won't be able to start it. You can ask AWS but I am guessing it's not that straightforward to add more capacity and those kind of things can take time. Since you have already tried starting it multiple times at different intervals, here's what I can suggest as a workaround.Change your instance type and try to start it again. Like I said, that instance type isn't available so you may have better luck changing the instance type and then try to start it again. Hopefully, the other instance type will have enough capacity. You can find a different instance type which has enough resources to support your workload. Take a look at the list: https://aws.amazon.com/ec2/instance-types/If you absolutely need to use the same instance type, then one workaround would be to create an AMI from your instance. And then re-launch the instance from your AMI in a different availability zone. There is no guarantee it would be work. But you can try your luck in a different availability zone.NOTE: Before you do any of those, make sure to create a backup of your volume or an AMI of your instance to be on the safe side!CommentSharetahatahiranswered 10 months ago0Thanks tahatahir.  I'll try changing the instance type.  I see m1 isn't even listed as an instance type anymore, so maybe that could be my issue.  Almost like an out-of-support thing.KenCommentShareKenanswered 10 months agotahatahir 10 months agom1 is a previous generation instance type. Should still be available. you would likely have to select previous generation instance types in the console. regardless, i would suggest perhaps trying to switch to one of the current generation instance types. you should be able to find one with enough resources for your use caseShare0What region and availability zone?N. Virginia, and US East 1C.What instance type are you trying to use?m1.small.  It is a community AMI, as I needed an older Linux version with older system libraries (specifically, an older glibc version), and Amazon's AMIs are too "new".KenCommentShareKenanswered 10 months ago"


Question: How can I avoid UnicodeDecodeError when trying to upload binary files to a local SAM lambda with API Gateway?

"HelloI'm trying to run a SAM instance locally containing an API Gateway and a Lambda written in python.  My goal is to POST images to my API so that I can upload them to an S3 bucket where they can be publicly served.If I do HTTP operations using JSON everything works fine, but when I try to POST a binary type like 'image/jpeg' I get the following error:2023-03-18 09:06:27  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)UnicodeDecodeError while processing HTTP request: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte2023-03-18 09:06:34 127.0.0.1 - - [18/Mar/2023 09:06:34] "POST /media HTTP/1.1" 502 -I've tried adding BinaryMediaTypes to my template.yaml and create a fully defined CloudFormation but I still get this error.Here's my code:https://github.com/caelumvox/blog-api-snipWould anyone know how to get this working locally?  Thank youFollowComment"


Question: Does Amazon WorkDocs support document tagging?

"I was wondering if Amazon WorkDocs supports document tagging, or if that is on the development radar, so that teams can get away from folder structures and move towards filtering based on tags.FollowComment"

Answer: "0Today, it is not possible to tag files and folders within WorkDocs. You can use WorkDocs API, CreateLabels to add labels to files and folders. However, they are not visible from the web client.CommentShareTaka_Manswered 8 months ago"


Question: Validate path param in AWS API Gateway through OpenAPI spec

"I created a POST API with path parameter and request body through openAPI spec."/v2/{lang}/subject": {  "post": {    "parameters": [        {          "name": "lang",          "in": "path",          "required": true,          "type": "string"        },        {          "in": "body",          "name": "subject",          "required": true,          "schema": {            "$ref": "#/definitions/subject"          }        }      ],    "requestBody": {      "content": {        "application/json": {          "schema": {            "$ref": "#/components/schemas/subject"          }        }      },      "required": true    },    "x-amazon-apigateway-request-validator" : "all",    .....   }I want to validate the path param and request body at APIGW. Unfortunately, 1 test case is failing.If I try accessing API with missing lang, ex: https://apigwsampleurl.com/v2/subject, AWS APIGW is throwing Missing Authentication Token error.How this case should be handled through openapi configuration?FollowComment"

Answer: "0Hello,I understand that you have created POST API with path parameter and request body validation using OpenAPI definition. You tested by sending API request with missing {lang}, to validate the path parameter, however it is not validating properly. Please correct, if I am misunderstanding.API Gateway currently does not support request’s path parameter validation. If the schema includes "required: true”, basic request validation only verifies if the parameter is included. If you require that only specific paths should exist, please create the resource for those paths separately.https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-method-request-validation.htmlAs a workaround, you can also consider implementing request validation logic, using Lambda authorizer or at the backend API integration itself.API returns 403 “Missing Authentication Token” error, if request is made to a resource that doesn't exist, eg: /v2/subject. https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-authentication-token-errors/We already have a feature request open with API Gateway service team, to support path parameter validation. I have added this post to the feature request. While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our What's New (https://aws.amazon.com/new/) and Blog (https://aws.amazon.com/blogs/aws/) pages for any new feature announcements.Please let us know if you have any further questions or concerns.CommentShareSUPPORT ENGINEERIsha_Kanswered a year ago"


Question: CloudFormation reference the userId that is running the Template

"My usecase consists of a user launching a cloudformation template via Service Catalog to create an ec2 instance.  I would like the CFN template to create a tag key called "Owner" and assign the userid who launched the CFN to the tag value.I use this "Owner" tag in a custom policy to provide permissions based on this condition: "Condition": {"StringEquals": {"aws:ResourceTag/Owner": "${aws:userid}"}The policy part works great, if I manually add the Owner tag to the resource, ec2 in this case.  Just not sure how to automatically assign the "Owner" tag value in the cloudformation template.In case it matters, the userId I'd like to assign is an federated SSO user.Many thanks in advance.FollowComment"

Answer: "0there is no pseudo parameter for the launcher of the stack. The only way this can be achieved currently is utilising a custom resource with lambda. This blog post shows a lambda being used for a similar purpose with service catalog where is inputs the userid into dynamo, you would need to use the output of the lambda into the input for your iam policyCommentShareEXPERTPeter_Ganswered a year ago0Just saw this question because I had the same issue with Service Catalog, is sad that SC have very limited feautres, would like to be more easy get information about the user issuing the provisioning from withing the cloud formation because I was using the solution to instead deploy a resource like ec2, to run a cloudformation template using launch constraint to include the user in a group where the user would have more permissions but is not easy to do that.Service Catalog is missing a settings for quotas for example, would like to limit how many provisioned products a user can have. Automate all of this using launch constraing is not easy.About the blog post, the problem with that is, if you are using a launch constraing  you will not get the user id from service catalog that started the product, you will get the service catalog principal id.The solution that comes to my mind after a lot of research.Use a lambda function to receive events from service catalog and check for events related to provisiong a product, I check and there you can have the id of the user, save the information in dynamo db. Getting information from some events you can get a information that is common on the cloudformation events, or you can use the pseu parameter stack id, compare it with the one saved from sc event api on dynamo db to check which user issued the provisioning command.Based on that you can properly tags your products, or in my case, you can use a lambda function to compute that and check if a user have a already provisioned product and cancel the operation.Is not much pratical becaus everything would be easily solved if Service Catalog have these features.CommentShareedipofanswered 8 months ago"


Question: Change projected attributes to existing GSI without delete and recreate in DynamoDB?

Is it possible to change projected attributes in an existing GSI in-place without delete and recreate?FollowComment

Answer: "1Accepted AnswerUnfortunately not, to change projected attributes you must create a new index, this remains true even if you would only like to add an extra attribute to INCLUDE projection.CommentShareEXPERTLeeroy Hannigananswered 2 months agoAWS-User-5132967 2 months agoThank you!Share0GSIs are read-only. You have to change the item in the base table and have it propagate into the GSI.CommentSharejzhunteranswered 2 months ago"


Question: How to manage backup of each service in AWS Batch

"There is a requirement to implement a backup that manages Aurora, S3, EFS, and ECR in AWS Batch in a certain project.How should I implement it in AWS Batch after considering the backup method and restore method of each service?Please tell me how to implement it.FollowComment"

Answer: "0AWS Batch is service to run the batch computing workloads. You just need to create docker image of your script and can run a job on batch job. that job will execute the code as mentioned in your docker image and get finish once done.So to use AWS Batch for backup, you need to write custom script which can take backup and create docker image of that script. later you can submit job in AWS Batch to execute that script.Looking at the query, it seems you would like have backup of other services like Aurora, S3, EFS, and ECR. It seems AWS Backup https://docs.aws.amazon.com/prescriptive-guidance/latest/backup-recovery/aws-backup.html might fit for your use case.CommentShareSUPPORT ENGINEERAshish_Tanswered a year ago"


Question: Crontab missing from most recent AWS Linux AMI

I have installed al2022-ami-2022.0.20220419.0-kernel-5.15-x86_64 on a m5.2x. I went to install a user cron and the crontab binary is missing. This is a default linux binary. The system cron seems to be fine (though I haven't tested it) but user crons are not working.ec2-user $ crontab -l -bash: crontab: command not found ec2-user $ which crontab /usr/bin/which: no crontab in ... ec2-user $ ls /usr/bin/crontab ls: cannot access '/usr/bin/crontab': No such file or directoryThis is fine in previous versions with the default install. Here it is with a default version of amzn2-ami-kernel-5.10-hvm-2.0.20220406.1-x86_64-gp2 also installed on an m5.2xec2-user $ which crontab/usr/bin/crontabFollowComment

Answer: "1Hello,Thank you for your post. I launched a test instance using the al2022-ami-2022.0.20220419.0-kernel-5.15-x86_64 AMI and verified your report. The cronie package, which provides the crontab command, is not included in this AMI.# rpm -qa |grep -i croncrontabs-1.11-24.20190603git.amzn2022.noarchI also checked the previous version of the AL2022 AMI and confirmed that the cronie package was not present there either. I would suggest you manually install the cronie and cronie-anacron packages that are both available from the amazonlinux repository.yum install cronie cronie-anacronNote, you can also make use of user data scripts[1] to install these packages at the time of launching instances.In the meantime, I have opened an internal ticket with the Amazon Linux team to report that these packages are not included in the AL2022 images.References:[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.htmlCommentShareSUPPORT ENGINEERAWS_SamManswered a year ago"


Question: "Received abuse warning from an Elastic IP, no instances match that IP.  How can we track it down?"

"Hello.  We recently received an abuse report saying an Elastic IP of ours was being used to send a malicious payload.  We do indeed have that IP, but it isn't mapped or associated with anything (perhaps it was at one point.  If so, it was before I was on the team).  We dug through everything we could to find any instance using this IP and found nothing.  No EC2 instances.  Nothing in load balancers.  I checked in every region, just in case.  Has anyone come across something similar?  If so, how did you track it down?  Can we find what might be using the IP?  We see the network interface mapped to it, but it's associated with a VPC used by a number of instances (none of which are the IP in the report).Otherwise, if we can't find the IP being used anywhere, I'm assuming it's safe to just get rid of it, correct?  We do not currently have a dedicated AWS person, so any help is appreciated.  Thank you so much for any assistance you can provide.Edit to add: Is it possible someone is spoofing the IP? The gateway says it was created in 2016 and I do not know the last time that IP was associated with an EC2 instance (if there's a way of finding that out).FollowComment"

Answer: "0In the EC2 console, under "Elastic IPs" you can check if the EIP has an "Association ID".  If not then it isn't in use and should be deleted - you get charged for unused EIPS so BTW that would appear on your bill.  If it does have an Association ID you can also see the associated instance ID or NAT Gateway etc.CommentShareEXPERTskinsmananswered 3 months agodmoczulski 3 months agoThank you.  Just before your answer I discovered the IP is a NAT Gateway, not a network device associated with an instance.  So now I need to figure out if I can find out which of our instances is initiating the malicious call.  I can find the internal NAT IP and the IP it's trying to hit in a CloudWatch log, but I'm not sure if the entry directly above it is the instance that initiated it?  I need to research more how the logs work.Thank you for the answer.Share"


Question: Tape Gateway on Amazon S3 Standard Storage

A customer would like to use the Tape Gateway and store 20% of data to S3 Standard Storage and 80% of data to Glacier. Is this possible using only the Tape Gateway?FollowComment

Answer: "0Accepted AnswerWith Tape Gateway virtual tapes are first created in the Virtual Tape Library, which sits on S3. Once the backup application ejects the tape, the tape gets archived into Virtual Tape Storage, which sits on Glacier or Glacier Deep Archive based on the customer's configuration.In other words, if the customer would like to keep 20% of their tapes available (i.e. in S3) and 80% archived (i.e. in Glacier or Glacier Deep Archive), that is certainly possible if the backup application ejects/archives tapes accordingly.CommentShareDaniela_Banswered 3 years ago"


Question: Unable to use launch template with AWS Batch Compute Environment

"I am trying to use a launch template with a AWS Batch Compute Environment, but I keep getting this error:INVALID - CLIENT_ERROR - You are not authorized to use launch template: Batch-lt-f51c5077-bb43-3a1a-b8eb-e7b971b2f39bI have verified that both the batch service role and the instance profile have the appropriate permissions. I have been unable to find any more information about this error online.  The compute environment works fine without a launch template.  The launch template that it is referencing is not the launch template I specified, but a copy of it created by the batch service role (I understand this is normal)Can anyone assist me ?FollowComment"

Answer: "1Accepted AnswerIs your account part of an AWS Organization? There's a chance that a service control policy is limiting permissions. See https://aws.amazon.com/premiumsupport/knowledge-center/batch-invalid-compute-environment/ for more detailCommentShareChris Pollardanswered 9 months agocarpejoe 9 months agoYes it is, I will review thisSharecarpejoe 9 months agoThis was it! There was an org policy that required certain tags, I needed to include these tags in the launch template to be propagated to the instanceShare"


Question: Why am i getting charged?

I use my AWS  for learning but getting charged. I have free tier  subscription and follow best practice to stop servers when not in use. Please help!FollowComment

Answer: "0Hi,I suggest to follow below resources so you do not risk to get charged:https://repost.aws/knowledge-center/free-tier-chargeshttps://repost.aws/knowledge-center/stop-future-free-tier-chargesHope it helps ;)CommentShareEXPERTalatechanswered a month ago"


Question: Best practices for migrating MySQL RDS to Aurora Serverless

I am looking for best practices on the process of migrating MySQL RDS to Aurora Serverless while minimizing any downtime.FollowComment

Answer: 1Accepted AnswerThe best approach at the moment is to use a 2 step approach to migrate from MySQL to Aurora Serverless.Step 1 - Is migrate to Aurora provisioned through a read replicahttps://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Migrating.RDSMySQL.Replica.htmlStep 2 - Is migrate from Aurora provisioned to Aurora Serverless from a snapshothttps://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.restorefromsnapshot.htmlCommentShareDerek_Yuillanswered 4 years ago


Question: Ec2 instance goes down every day

"I have hosted my Ec2 instance in Mumbai Region, Which goes down every day for few hours. Giving Host Error 524. (Billing section is working fine)FollowComment"

Answer: "1According to Wikipedia a 524 error is a CloudFlare timeout error. Have you tried connecting directly to your instance when this happens?Without more information this is pretty difficult to troubleshoot. What do the logs say around the time you're having this error? What to the CloudWatch metrics show? Is the instance reaching 100% CPU? If it is a T-series instance s it running out of CPU credits? Is it running out of memory? Is there a spike in network or disk activity?CommentShareEXPERTBrettski-AWSanswered a year agoAWS-User-8570116 a year ago@Brettski@AWSPasted the error response here.https://paste.pics/1b62db3e837c642b529e4c45e19720b9Below are the details of matrices: (I am not much technical can you please suggest what is causing the issue)https://paste.pics/GOUP2As a basic website is hosted,so no traffic, I am not sure why high resources is been used.ShareBrettski-AWS EXPERTa year agoThe smoking gun here is the "CPU credit balance" (very last graph). You're using a T-series instance which has burstable performance; when the CPU is busy it draws down on its credit balance until it gets to zero. At that point you get baseline performance which (depending on the size of the instance) may be very slow - and if other tasks are using the CPU (which it appears they are) then you'll not get the response you require.Solutions: Use a bigger T series instance (it has a higher credt balance). Choose "unlimited" for your T series instance (that will cost you a little extra but only when you run out of credits). Use a non-T series instance (because they don't have burstable performance - they run at the same speed all the time).Finally: Look at the process table on your instance (and the logs) to find out what is consuming all of the CPU. Is it web traffic? Is it something else? Fix whatever it is that is causing that; or use the solutions above.Share"


Question: Lambda Netcore 3.1 why cloudsearchdomain search  hang when it is deployed?

"hello,i have a lambda net core 3.1 +apigateway that must launch a search on a cloudsearchdomain.i use this code :using (var client = new AmazonCloudSearchDomainClient(searchServiceConfig)){....LambdaLogger.Log("launch search");Task<SearchResponse> task = client.SearchAsync(request); task.Wait();                                        //// hang here LambdaLogger.Log("parse result");/// this message don't appear in the cloudwatch logSearchResponse response = task.Result;LambdaLogger.Log("after parse result");totalResult = response.Hits.Found;when i test this code :in a windows app : OKwith Mock Lambda Test Tools : OKdeployed in lambda on aws infrastruture : KO the code don't pass the task.Wait() instruction.any idea for unblock this problem ?FollowComment"

Answer: 0it was not a problem with async method. it was a problem with vpcCommentShareWANAOJDanswered 3 years ago


Question: Copy/Paste is Not working b/n workspaces-local computer

"Hello,We have created workspaces (Windows Server 2016), and copy/paste b/n the workspace and the local computer is not working.It is not restricted by policy. It is not working from the workspace client and from the web as well.I tried to create a policy to allow copy/paste too, and still not working.Can someone help here, is there some workaround, is this a known issue ?Best Regards,FollowComment"

Answer: "0Hello, copy and paste is enabled by default unless specifically disabled via group policy. Which client and OS are you using? Please ensure that you have updated to the latest client (5.6 on Windows and Mac).Please note that copy and paste is supported for text only. If you are trying to copy a file via copy and paste (such as you can do using Remote Desktop) this is not supported in WorkSpaces.Additionally, if there's latency between your connection and the WorkSpace, there could be a delay as the data needs to be transferred to the WorkSpace, however on most occasions this should be quick.Please attempt the following:Connect to the WorkSpace and open notepad.Open notepad on the local computer and type some text. Copy it and paste it on the same notepad to ensure the clipboard and pasting is working as expected.Attempt to paste on the WorkSpace directly on the notepad application.If this still doesn't work, ensure that the clipboard redirection policy (for either PCOIP or WSP depending on what you are running) is either not defined or enabled in both directions https://docs.aws.amazon.com/workspaces/latest/adminguide/group_policy.html#gp_clipboardPlease note, with clipboard redirection enabled on the WorkSpace, if you copy content that is larger than 890 KB from a Microsoft Office application, the application might become slow or unresponsive for up to 5 seconds.If the issue still occurs, you will need to take a look at the client logs in %LOCALAPPDATA%\Amazon Web Services\Amazon WorkSpaces\logs (Windows) or ~/Library/"Application Support"/"Amazon Web Services"/"Amazon WorkSpaces"/logs (Mac). When you connect to a workspace, you will see a new log entry under the PCOIP or WSP folder. The log created in this folder will register the clipboard events. For example, in my WSP based WorkSpace I can see the following events:2023/01/12 18:44:01.513 [INF] viewer.ClipboardMonitor - Getting dcv/text clipboard data2023/01/12 18:44:01.517 [INF] viewer.ClipboardChannel - Sending dcv/text clipboard data of 5 bytes (id 1, error "NONE")When I disconnect, I see the following:2023/01/12 18:44:13.026 [DBG] viewer.ClipboardChannel - <SESSION> Transport closed2023/01/12 18:44:13.026 [INF] viewer.ClipboardChannel - Clipboard channel disconnected2023/01/12 18:44:13.026 [DBG] viewer.ClipboardMonitor - Remove ClipboardChannel viewer.ClipboardChannelYou can take a look at this log and see if any error messages are shown regarding Clipboard events.Please feel free to open a new support case so that an engineer can troubleshoot those logs for you if needed.Have a great day ahead!CommentShareSUPPORT ENGINEERFrancisco_Lanswered 5 months ago"


Question: Unable to launch Appstream instances. login attempts fail

"We have a number of fleets already set up and those are working fine but one of the newer fleets that I have created has an issue when new users try to launch and log into Appstream instances that belong to the said fleet. The password that is used is correct but they sometimes get an incorrect password error and then it pops up "An unknown error occurred (1909)". The strange part about it all is that I encountered a similar problem when I first launched and logged in to an instance on the fleet. It took about four or five attempts before it worked. Since that successful sign in, I only encounter the above error occasionally but it works most of the time. Unfortunately, the same doesn't seem to be happening for the newer users that have been assigned access to it.So my questions are, why is this happening in the first place and why is it allowing few users that have signed in previously but none of the new users?FYI, we use Okta as our identity provider.Any advice you may be able to provide would be most appreciated.Thanks and regards,DiemFollowCommentDiem D a year agoUpdate: To try and reach the bottom of this issue, we tried using an older 2012 server image with the fleet and that seems to be working alright. It seems that there is a problem with the way the new images have been created that is causing this issue.  Could it be that we are using a domain user as a template user for the image that might be causing this? I used my own domain account as the template user and I find that I can log in after maybe two or three attempts but most new users that attempt to log in fail to do so.Share"

Answer: "1Hi Diem -The error codes that you see on the password prompt from AppStream 2.0 are coming directly from Active Directory. According to Microsoft's documentation, error code 1909 refers to "The referenced account is currently locked out and may not be logged on to."Since Okta and Active Directory can be their own independent information sources, it's possible for a user to login to Okta successfully but fail on the Active Directory login. Can you validate the users Active Directory object to confirm it isn't locked? Since you mentioned taking multiple attempts before it succeeds, one or more of your domain controllers might not be synched, which could be what's causing the issue sporadically instead of consistently. You may need to connect to the individual domain controller(s) to validate.It's also recommended to set the sites for the AppStream 2.0 IP ranges to the domain controllers that are (ideally) in the same VPC/AZs as the AppStream 2.0 environment - barring this, to the domain controllers with the lowest latency.Hope this helps.MuraliCommentShareEXPERTMuraliAtAWSanswered a year agoDiem D a year agoThanks for your comment Murali.To answer your question, having checked our logs, we can confirm that DCs are properly synced and that the user's account wasn't locked out on any of the servers.Appreciate your recommended solution, but it doesn't explain why the other fleets are working normally apart from this one. Besides, the IP address range is already covered in sites and services and they do exist in the same AZ with the domain controller.ShareMuraliAtAWS EXPERTa year agoStrange that somehow Windows is returning the 1909 error code then. Can you try providing the user access to RDP into an AppStream 2.0 instance? You can try doing it to an domain-joined image builder from an EC2 instance. Curious to see if that allows login immediately, or if it's a similar experience for this specific user.Also, have you opened a support case? AWS Support would also be able to see the AppStream 2.0 logs for potentially more data.ShareDiem D a year agoThanks for your input Murali. Apologies for the late response.Coincidentally when I try to start up the image builder that I normally use, I see a notification, namely "DOMAIN_JOIN_INTERNAL_SERVICE_ERROR": The account already existsI've created a new Image builder because of the error and we've managed to connect an ordinary user to the instance once we allowed RDP access. They connected using their AD account without any issue on the first go.I will try rebuilding the image and if that fails the next step will be to contact AWS Support.Really appreciate your feedback on this though.Many thanks.DiemShare0Diem,Is the fleet domain joined? At what stage are you getting this error? Is it at the Okta login page or is it after that? If the user is in Active Directory, you may want to check whether account is locked out or not. I would check the user account status, password policy for Bad password attempts and Account Lockout Duration both at Okta and Active Directory(if you are using AD).CommentShareArun_PCanswered a year agoDiem D a year agoThanks for your reply Arun.To answer your questions, yes, the fleet is domain joined. This happens past the Okta login page. The URL on the login page is an amazon one so I think it is an Amazon password challenge. This is what it looks like "https://appstream2.eu-west-2.aws.amazon.com/#/streaming?reference=fleet..."I've confirmed with a user's account that was used to sign on that no bad password attempts have been registered for it on AD and Okta shows SSO success on Amazon Appstream 2.0, so nothing to indicate that any user error is involved.Anything else you can think of regarding why this may be happening? It's weird because, after the first attempt, it doesn't bring up an error. It just presents the password prompt again. Then after a number or attempts, if at all, the 1909 error pops up.Share"


Question: "Is it allowed to send data, copied from s3 to lightsail, to internet?"

"'AWS Service Terms' document:66.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public Internet or other destinations or excessive data processing through load balancing Services as described in the Documentation), and if you do, we may throttle or suspend your data services or suspend your account.So if I copy files from s3 to lightsail disk and then send it to internet from lightsail, is it allowed?Edited by: paltus on Jul 9, 2019 9:17 AMFollowComment"

Answer: "0If someone interesting, I get an answer from amazon:Q: is it allowed?A: This is allowed.CommentSharepaltusanswered 4 years ago0Hi paltus,Why would you need to copy files from S3 to a Lightsail instance in this case and not serve your data directly from S3?BestGeorgeCommentShareAWS-User-4674287answered 4 years ago0S3 to INTERNET costs $0.09/Gb;S3 to lightsail free (probably; however uses lightsail free transfer quota) ; lightsail to INTERNET 1Tb free (for cheapest plan). So if you need to restore handreds Gbs or Tb of data from s3 (backup for example), it can help save significal money.Edited by: paltus on Jul 9, 2019 11:00 AMCommentSharepaltusanswered 4 years ago"


Question: Pushing registry keys across accounts daily

"Hello everyone,I am trying to push a set of reg keys to non prod and prod accounts. we spin up new machines daily and i'd like to make sure the keys are pushed. What would be the best way to do this?FollowComment"

Answer: "1Hi THereHere is a guide that shows you how to push registry keys to windows instances using Systems Manager Run Commandhttps://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/automate-adding-or-updating-windows-registry-entries-using-aws-systems-manager.htmlAnother easy way is to use EC2 Instance UserData to run a command at at launch.  You can run the "reg add" command to add the key when the instance is launched.  Seehttps://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ec2-windows-user-data.htmlhttps://learn.microsoft.com/en-us/windows-server/administration/windows-commands/reg-addCommentShareEXPERTMatt-Banswered 8 months agojoeyfine 8 months agoSo i was hoping for something to run daily in system manager like my maintenance windows for patching (say like a lambda that kicks off a job to push the keys every 4 hours). I dont want to push the both non pro and prod keys to a new instance when they are spun up in Kubernetes.Sharejoeyfine 8 months agoI already do a manual push of the keys via run command. I was hoping to automate it out completely to run every so many hours.ShareMatt-B EXPERT8 months agoHiYes you can do this with EventBridge.  See https://docs.aws.amazon.com/systems-manager/latest/userguide/monitoring-systems-manager-targets.htmlYou can use EventBridge to kick off a Run Command on a schedule, or you can use an event like When a new node is created in an Auto Scaling group, a Run Command target action could turn on the web server role or install software on the node.Share"


Question: Which version of MongoDB driver supports 'ssl_ca_certs' parameter in URI?

"Hi,I want to connect to DocumentDb using Spring Data in a Spring Boot 2 Java application. I've added the query parameter ssl_ca_certs to the MongoDB URI but when I start my application it fails because ssl_ca_certs is not known as a parameter in the URI.org.mongodb.driver.uri : Unsupported option 'ssl_ca_certs' in the connection string 'mongodb://(host):27017/(database)?ssl=true&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false&ssl_ca_certs=rds-combined-ca-bundle.pem'.Which version of java MongoDb driver supports the ssl_ca_certs parameter?FollowComment"

Answer: "0The latest version of the Java MongoDB driver is compatible with Amazon DocumentDB. To use the rds-combined-ca-bundle.pem cert from your Java application, you must create a trust store with the CA certificate, import the trust store locally on your client machine and then set the trust store from your application. For instructions on how to do this, refer to  https://docs.aws.amazon.com/documentdb/latest/developerguide/connect_programmatically.html#connect_programmatically-tls_enabledCommentShareAWS-User-8311923answered 3 years ago"


Question: KMS Key policy ignored over IAM Role

"I have a "poweruseraccess" policy applied to a "Developer" role in my account that is used by multiple users. This role allows access to AWS resources, as such anyone with this role can encrypt/decrypt using keys in KMS. I want to restrict encryption/decryption actions on a particular kms key. For this, I added a deny section to the default kms policy on this specific key as below. This denies encrypt/decrypt action to any principal except if their userid is the root (12345) or specific role AROAADMINROLE (account admins), AROALAMBDAROLE (captures assumerole)  and an IAM user AIDAMYIAMUSER. In spite of this explicit deny section, the users with Developer role are still able to encrypt/decrypt with the key. Can someone please help me figure out the issue?Similar policies works for restricting our S3 bucket access. I followed this article for building the policies. https://aws.amazon.com/premiumsupport/knowledge-center/explicit-deny-principal-elements-s3/ . It's the same principle for below policy by using wildcards and StringNotLike in conditions.KMS policy{    "Id": "my-key-consolepolicy",    "Version": "2012-10-17",    "Statement": [        {            "Sid": "Enable IAM User Permissions",            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::12345:root"            },            "Action": "kms:*",            "Resource": "*"        },        {            "Sid": "Allow access for Key Administrators",            "Effect": "Allow",            "Principal": {                "AWS": "arn:aws:iam::12345:user/my_iam_user"            },            "Action": [                "kms:Create*",                "kms:Describe*",                "kms:Enable*",                "kms:List*",                "kms:Put*",                "kms:Update*",                "kms:Revoke*",                "kms:Disable*",                "kms:Get*",                "kms:Delete*",                "kms:TagResource",                "kms:UntagResource",                "kms:ScheduleKeyDeletion",                "kms:CancelKeyDeletion"            ],            "Resource": "*"        },        {            "Sid": "ExplicitDenyEncryptDecryptAccess",            "Effect": "Deny",            "Principal": "*",            "Action": [                "kms:Encrypt",                "kms:Decrypt"            ],            "Condition": {                "StringNotLike": {                    "aws:userid": [                        "12345",                        "AROAADMINROLE",                        "AROAADMINROLE:*",                        "AIDALAMBDAROLE:*",                        "AIDALAMBDAROLE",                        "AIDAMYIAMUSER:*",                        "AIDAMYIAMUSER"                    ]                }            }        },        {            "Sid": "Allow attachment of persistent resources",            "Effect": "Allow",            "Principal": {                "AWS": [                    "arn:aws:iam::12345:user/my_iam_user",                    "arn:aws:iam::12345:role/my_lambda_role"                ]            },            "Action": [                "kms:CreateGrant",                "kms:ListGrants",                "kms:RevokeGrant"            ],            "Resource": "*",            "Condition": {                "Bool": {                    "kms:GrantIsForAWSResource": "true"                }            }        }    ]}Edited by: swatic on Aug 20, 2019 10:49 AMFollowComment"

Answer: "0Accepted AnswerHi,It appears that you just have a typo and are missing the Resource for you Deny Condition block.Your policy should be changed to:        {            "Sid": "ExplicitDenyEncryptDecryptAccess",            "Effect": "Deny",            "Principal": "*",            "Action": [                "kms:Encrypt",                "kms:Decrypt"            ],            "Resource": "*" ,            "Condition": {                "StringNotLike": {                    "aws:userid": [                        "12345",                        "AROAADMINROLE",                        "AROAADMINROLE:*",                        "AIDALAMBDAROLE:*",                        "AIDALAMBDAROLE",                        "AIDAMYIAMUSER:*",                        "AIDAMYIAMUSER"                    ]                }            }        },Hope this helps!-randyCommentShareRandyTakeshitaanswered 4 years ago"


Question: API Gateway Websocket - payload quotas

"Hello folks,hope all of you are well!I'm here in order to ask if someone know differences betweenWebSocket frame sizeMessage Payload SizeI've found these field on this web page https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#apigateway-account-level-limits-table: here is reported the AWS WebSocket quotas and limits, but is not clear to me what is the difference between " WebSocket frame size" and "Message Payload Size".Could you provide some examples?Cheers.FollowComment"

Answer: "2Frames are defined by the WebSocket RFC. This is the way in which clients and servers exchange data. When you send a message it is encoded in a frame. If the message is > 32KB, it must be encoded in multiple frames.CommentShareEXPERTUrianswered a year agom_piffari a year agoThanks for RFC reference.And how AWS treat frames >32KB? I found in AWS API Gateway quotas that max payload is 128Kbytes. I create a test web socket gateway and I send over it a packet composed exactly of 128Kbytes, and the communication was ok.So, AWS provide and integrated logic in order to handle multiple frame? Logic used in both direction (in and out from gateway)?ShareUri EXPERTa year agoIf I am not mistaken, the RFC covers that as well, i.e., how to break a large message to multiple frames.Share"


Question: I can't find my EC2 instance in my dashboard

"My friend created an EC2 instance and shared it with me (I received an invitation email from AWS when he shared) and I have been using it to host my blog. However today when I logged into AWS using the same email that my friend shared the EC2 with me, I cannot see that EC2 anymore. I have looked in all US regions. I am still able to ssh to the EC2, I just can't see it in the dashboard. What should I do to show the EC2 in my dashboard? ThanksFollowComment"

Answer: "2Rather than hunting through regions, you could try EC2 Global View in the EC2 dashboard or via https://console.aws.amazon.com/ec2globalview/home.  If you still can't see it are you sure you're logged into the right account?CommentShareEXPERTskinsmananswered a year ago"


Question: The image cannot be displayed because it contains errors

"Hi,I'm working with saving and displaying images in and S3 bucket.I am on a mac and the images show fine on the mac.I am able to upload many images to the bucket and then I can display them using a pre-signed URL. All good...But then I have some other varied images such as .jpg that I see fine on the mac and seem to upload OK however do not display from s3 using pre-signed URL. When viewed in Mac Safari or chrome or Firefox I get the broken image symbol. Firefox also says:The image "https://xxxxxxxxxx" cannot be displayed because it contains errors"Someone suggested that possibly the original file creation might have been strange in someway and the Mac might be able to interpret the image however S3 cannot do this successfully. Possibly this might be a cross platform Windows / Mac / Linux image issue?Test: I took one of the .jpg images that did not show up from S3 - and I opened  it in preview  on the Mac and exported it also as .jpg under a different name. Then I uploaded this new Version add this did seem to fix the problem because it now she displays correctly from s3.However for what I'm doing I do not want to have to export every image and resave it - in order to go to S3.Q: Does anybody have any solutions as to why I am getting some Errors when trying to display images from S3? Any ideas how to fix this?Quick update - in Mac terminal I tried : file -I ~/Desktop/test.jpg and surprisingly it came back as = image/heic even though the file had .jpg suffix.... Any idea how to get s3 to read "heic files"?Thanks daveFollowComment"


Question: PostGIS and Aurora compatibility: experience feedback

"Customer would like to move his PostgreSQL to Aurora PostgreSQL with PostGIS.As per AWS forum, this should work as RDS PostgreSQL documentation is supported and Aurora doesn't list it as incompatible.I quickly checked I could load the postgis extension in Aurora but I don't have extensive experience with this.Are there any drawbacks, tips with Aurora + PostGIS ?FollowComment"

Answer: "0Accepted AnswerYes Aurora Postgres also supports PostGiS. We maintain extensions parity as much as possible between RDS Postgres and Aurora Postgres. Please give it a try and let me know if you have any problems.CommentShareAWS-User-3438484answered 5 years ago0We are running Aurora Serverless v1 with Postgis extension, mainly for data storage, and haven't noticed problems with extension itself. Instead there has been a bunch of problems with db-connections when inserting data to DB with FME.CommentSharemikkokanswered a year ago"


Question: instance is unhealthy yet doesn't get replaced

"I have a load balanced + auto scaled .NET Windows application configured with 2 hosts minimum. But when 1 of the hosts is "unhealthy", it doesn't get replaced automatically. I have to kill it and then it will be replaced.The load balancer also knows it is unhealthy. So why the disconnect with Elastic Beanstalk?Any ideas what could be configured wrong here?And FWIW, I'm pretty sure this used to work.FollowComment"

Answer: "0Accepted AnswerHello,So I get that when the ELB healthchecks fails for the instances, the instances are not replaced. By default the ASG healthcheck type is set to EC2 in Beanstalk. So even if the ELB healthcheck fails, the instances wont be replaced.In order to replace the instances based on ELB healthchecks, we would have to set the HealthCheckType to ELB using the .ebextions config file:Resources:  AWSEBAutoScalingGroup:    Type: AWS::AutoScaling::AutoScalingGroup    Properties:      # Example syntax of a property that takes in the actual value      HealthCheckType: ELB      HealthCheckGracePeriod: 600More info:https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environmentconfig-autoscaling-healthchecktype.htmlCommentShareKaushikM-AWSanswered 4 years ago0Awesome, thank you!CommentSharefoxcubgamesanswered 4 years ago"


Question: How to pull a list of items from a DynomoDB table without using primary key in Go.

"Below is a single item in JSON, there are hundreds more.  The account_name and ami_name are the partition key and sort key.  I want to be able to query a list based on the ami_name and filter that list by the environment value, in this case, "test".  I have an example of what I am trying so far.package mainimport (    "context"    "fmt""log"    "github.com/aws/aws-sdk-go-v2/aws"    "github.com/aws/aws-sdk-go-v2/config"    "github.com/aws/aws-sdk-go-v2/service/dynamodb"    "github.com/aws/aws-sdk-go-v2/service/dynamodb/types")func main() {    cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"))    if err != nil {        log.Fatalf("unable to load SDK config, %v", err)    }tableName   := "aws-webhooks-table-test"// account_name := "aws-computer-test"ami_name     := "win2019core"    environment := "test"    svc := dynamodb.NewFromConfig(cfg)    out, err := svc.Query(context.TODO(), &dynamodb.QueryInput{        TableName:              aws.String(tableName),        KeyConditionExpression: aws.String("ami_name = :ami_name"),FilterExpression: aws.String("environment = :environment"),        ExpressionAttributeValues: map[string]types.AttributeValue{            ":ami_name":  &types.AttributeValueMemberS{Value: ami_name},":environment":  &types.AttributeValueMemberS{Value: environment},        },        ExpressionAttributeNames: map[string]string{            "ami_name": ami_name,        },}{  "account_name": {    "S": "aws-computer-test"  },  "ami_name": {    "S": "win2019core"  },  "account_id": {    "S": "7666666666"  },  "aws_region": {    "S": "us-east-1"  },  "environment": {    "S": "test"  },  "lob": {    "S": "computer"  },  "payload_url": {    "S": "https://codebuild.us-east-1.amazonaws.com/webhooks?t=eyJlbmNyeXBlotsoflettersKPIkeo"  },  "project_name": {    "S": "aws-computer-test-win2019-ami-project"  },  "repo_url": {    "S": "https://git.nylcloud.com/Cloud-Team/packer-aws-win2019core"  },  "secret": {    "S": "dq7bHlotsofletterZKPIkeo"  }}FollowComment"

Answer: "2In DynamoDB you must always provide the Partition Key when reading items, unless using Scan operation which will read every item in the table and is best to be avoided if possible.For your use-case you would need to create a Global Secondary Index (GSI) and specify ami_name as the partition key. Optionally you can use environment as your sort key.Now you would target the GSI and use a Query operation to read the data (with partition key only):    out, err := svc.Query(context.TODO(), &dynamodb.QueryInput{        TableName:              aws.String(tableName),        IndexName:              aws.String(indexName),        KeyConditionExpression: aws.String("ami_name = :ami_name"),FilterExpression: aws.String("environment = :environment"),        ExpressionAttributeValues: map[string]types.AttributeValue{            ":ami_name":  &types.AttributeValueMemberS{Value: ami_name},    ":environment":  &types.AttributeValueMemberS{Value: environment},        },        ExpressionAttributeNames: map[string]string{            "ami_name": ami_name,        },}If you choose to use environment as sort-key on your index then you would include that in your KeyConditionExpression and remove the FilterExpression.CommentShareEXPERTLeeroy Hannigananswered 7 months ago-2Yes you can use the pagination method which uses 'Scan' operation, however this operation will read every item in the table and is best to be avoided if possible due to two reasons :Increase in cost.Time consuming.Therefore you must always provide the Partition Key when reading items.CommentShareSUPPORT ENGINEERrePost-User-6470877answered 7 months ago"


Question: Zeppelin is not printing a Flink DataStream

"I'm running this code in a Zeppelin notebook (via Kinesis Analytics Studio), and it is not printing anything:env.fromElements("a", "b", "c").print()env.execute()FollowComment"

Answer: "0If you are using %flink.pyflink or %flink as your interpreters, you will need to use the ZeppelinContext to visualize the results within the notebook.%flink// create a DataStreamval dataStream = senv.fromElements("Alice", "Bob", "John")// interpret the insert-only DataStream as a Tableval inputTable = stenv.fromDataStream(dataStream)z.show(inputTable, streamType="update")For more PyFlink specific examples, see Query your data streams interactively using Kinesis Data Analytics Studio and Python.CommentShareJeremyanswered 7 months ago"


Question: Which is the default region "aws s3" connects by default from EC2?

"We were trying to setup an GatewayEndpoint for S3 in eu-central-1.After setting up the endpoint, route and security groups, the command "aws s3 ls" failed, but "aws s3 ls --region eu-central-1" was successful.Why is that? To which region is the request sent by default if not specifying the region explicitly?FollowComment"

Answer: 0Accepted AnswerThe default region depends on the value set for the environment variable AWS_DEFAULT_REGION or the value set at the time when aws configure was used to add the access key and the secret key.CommentShareEXPERTagardezianswered 3 years ago


Question: Textract using FIFO SNS topics and FIFO SQS queues

"Hi,Can Textract be used with FIFO SNS topics and FIFO SQS queues? Or do they have to be standard topics and queues?Thanks!FollowComment"

Answer: "0Hello,Customers should be able to use FIFO as well as standard SNS topics for asynchronous Textract API completion notifications.Let us know if you have any more questions.CommentShareMarkoceAtAWSanswered 2 years ago0Upon further review, Textract does not support SNS FIFO currently. We would need to pass a message de-duplication ID as well as a message group ID with the publish request which we do not.CommentShareMarkoceAtAWSanswered 2 years ago"


Question: Why is SubscribeToShard not available for the AWS SDK for .NET?

"I'm working with clients who are locked in to using the AWS SDK for .NET. They are reading records from a Kinesis stream, and I suggested they use SubscribeToShard method because it worked so well in the SDK for Python. But it didn't seem to work in .NET and the above documentation page gives a 404 page. Furthermore, the API documentation for AmazonKinesisClient does not list the method, even though it is referenced in the RegisterStreamConsumer method which seems to be available.Is there any reason why this particular method is not available in the SDK for .NET?FollowComment"

Answer: 1The SubscribeToShard method requires HTTP/2 and we don't currently have HTTP/2 supported integrated into the AWS SDK for .NET.CommentShareNormanswered a year ago


Question: AWS Glue Postgres JDBC Data Source Failing with NullPointerException

"I am writing a Spark Job in Java for execution on AWS Glue. It attempts to connect to a Postgres database using the glueContext.getSource() method. It is failing with the following NullPointerException:2023-04-20 14:57:01,183 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(94)): Exception in User Classjava.lang.NullPointerExceptionat com.amazonaws.services.glue.util.JDBCWrapper$.$anonfun$apply$6(JDBCUtils.scala:914)at scala.collection.MapLike$MappedValues.$anonfun$foreach$3(MapLike.scala:252)at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:788)at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:230)at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:461)at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:787)at scala.collection.MapLike$MappedValues.foreach(MapLike.scala:252)at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:58)at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:49)at scala.collection.mutable.MapBuilder.$plus$plus$eq(MapBuilder.scala:25)at scala.collection.immutable.DefaultMap.$plus(DefaultMap.scala:40)at scala.collection.immutable.DefaultMap.$plus$(DefaultMap.scala:38)at scala.collection.immutable.MapLike$$anon$2.$plus(MapLike.scala:101)at scala.collection.immutable.MapLike.$anonfun$$plus$plus$1(MapLike.scala:87)at scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156)at scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)at scala.collection.immutable.Map$Map3.foreach(Map.scala:192)at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156)at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154)at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)at scala.collection.TraversableOnce.$div$colon(TraversableOnce.scala:150)at scala.collection.TraversableOnce.$div$colon$(TraversableOnce.scala:150)at scala.collection.AbstractTraversable.$div$colon(Traversable.scala:104)at scala.collection.immutable.MapLike.$plus$plus(MapLike.scala:87)at scala.collection.immutable.MapLike.$plus$plus$(MapLike.scala:86)at scala.collection.immutable.MapLike$$anon$2.$plus$plus(MapLike.scala:101)at com.amazonaws.services.glue.util.JDBCWrapper$.connectionProperties(JDBCUtils.scala:956)at com.amazonaws.services.glue.util.JDBCWrapper.connectionProperties$lzycompute(JDBCUtils.scala:739)at com.amazonaws.services.glue.util.JDBCWrapper.connectionProperties(JDBCUtils.scala:739)at com.amazonaws.services.glue.util.JDBCWrapper.tableDF(JDBCUtils.scala:865)at com.amazonaws.services.glue.util.NoCondition$.tableDF(JDBCUtils.scala:87)at com.amazonaws.services.glue.util.NoJDBCPartitioner$.tableDF(JDBCUtils.scala:173)at com.amazonaws.services.glue.JDBCDataSource.getDynamicFrame(DataSource.scala:1088)at com.amazonaws.services.glue.DataSource.getDynamicFrame(DataSource.scala:101)at com.amazonaws.services.glue.DataSource.getDynamicFrame$(DataSource.scala:101)at com.amazonaws.services.glue.AbstractSparkSQLDataSource.getDynamicFrame(DataSource.scala:725)at com.amazonaws.services.glue.DataSource.getDataFrame(DataSource.scala:118)at com.amazonaws.services.glue.DataSource.getDataFrame$(DataSource.scala:118)at com.amazonaws.services.glue.AbstractSparkSQLDataSource.getDataFrame(DataSource.scala:725)My code for connecting and building a Data Frame is as follows:scala.collection.mutable.Map<String, String> optionsMap = new scala.collection.mutable.HashMap<>();optionsMap.put("url", "jdbc:postgresql://[hostname]:5432/postgres");optionsMap.put("dbtable", "public.test");optionsMap.put("user", "postgres");optionsMap.put("password", "Passw0rd!123");JsonOptions jsonOptions = new JsonOptions(optionsMap);DataSource source = super.getGlueContext().getSource(getConfiguration().getFormat().toLowerCase(),jsonOptions,"","");I have confirmed that my job has an IAM role with sufficient permissions to read RDS. Does anyone have any suggestions?FollowComment"

Answer: "0Accepted AnswerThis ended up being a hidden null option input. The username and password options, pulled from a separate class, were not parsing correctly, and therefore returning nulls. This ended up causing a NullPointerException deep in the Glue code.So if you get this NPE, make sure that all of the option inputs to the JsonOptions map are non-null.CommentShareDonCampbellanswered a month ago0I'm not sure if that "JsonOptions(optionsMap)" is correctly handling the configuration map.I've always passed a json string to build JsonOptions, like in the documentation:https://docs.aws.amazon.com/glue/latest/dg/glue-etl-scala-apis-glue-gluecontext.html#glue-etl-scala-apis-glue-gluecontext-defs-getSourceCommentShareGonzalo Herrerosanswered a month ago"


Question: RDS Sql Server support for 15.0.4198.2

When will AWS support 15.0.4198.2 for RDS?FollowComment

Answer: "0Hello,I understand you want to know when the engine version 15.0.4198.2 for SQL Server will be available on RDS SQL Server.RDS SQL Server currently does not support CU15 15.0.4198.2 for SQL Server 2019 and there is no ETA for when this version will be released and available. I would recommend that you refer to the link https://aws.amazon.com/new/ for latest feature releases and announcements as when engine version is available, it will be announced in the What's New with AWS page.Currently, the latest version for SQL Server 2019 is CU12 15.00.4153.1 https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SQLServer.html you can manually upgrade to the latest version of SQL Server 2019 via modifying the DB Instance https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.Upgrading.html#USER_UpgradeDBInstance.Upgrading.ManualCommentShareSUPPORT ENGINEERSomto_Manswered a year ago"


Question: AWS RDS Mssql performance issues.

"I have an AWS RDS Mssql and On-premise Mssql, Both servers have 4 CPUs and 16 Memory. Both servers have the same data in the table about 700k records.When the exec query selects 1000 records first.AWS RDS Mssql: ~2.4 secondsOn-premise Mssql: ~ 0.5 seconds.Why is AWS RDS Mssql too slower than On-premise Mssql?FollowComment"

Answer: "1This could be due to latency from the client. What is the distance between the on-prem client and SQL versus the on-prem client and RDS MSSQL?   Also, run the query on both platforms and record the query stats to determine exactly where the latency is.   Does the query involve any indexes? Are these indexes present on both platforms? Have you checked the query plan of each platform for any differences.. Are statistics up to date? Finally, is the performance of the storage the same with regard to IOPS and throughput..  perhaps there is more caching of data on-prem versus in RDS.  I hope these ideas will help you to locate the reason for the difference in query speed.CommentShareAlan-Cranfield-awsanswered 3 months agorePost-User-6212217 3 months agoYes, I think that. I tried:Increase max server memory to the max ->  it did not changeIndexing it -> Execute time same as the on-prem.Other than that I try to install Mssql on the Ec2 and have the same config (4 CPUs and 16 Memory). And It has the performance same as the on-prem.Share1Great. So, from you answer above I understand that you now have the same execution time as on-prem after updating the indexes.   And a SQL Server on EC2 instance that you launched also has the same performance.   Looks like you're good.CommentShareAlan-Cranfield-awsanswered 3 months ago0Also, I do not think that your RDS instance matches the specs you mention.CommentShareMODERATORphilawsanswered 3 months ago"


Question: GGV2: Unable to run docker containers: docker.sock - permission denied

"This was working just fine a month ago, but now:=> When Greengrass tries to install docker images with "docker load -i [...]" I get this error:2021-04-14T15:24:51.673Z [WARN] (Copier) xxxxxxxxx: stderr. Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/images/load?quiet=1: dial unix /var/run/docker.sock: connect: permission denied. {scriptName=services.xxxxxxxxx.lifecycle.Install.Script, serviceName=xxxxxxxxx, currentState=NEW}2021-04-14T15:24:51.676Z [WARN] (pool-2-thread-17) xxxxxxxxx: shell-runner-error. {scriptName=services.xxxxxxxxx.lifecycle.Install.Script, serviceName=xxxxxxxxx, currentState=NEW, command=["docker load -i /greengrass/v2/packages/artifacts/xxxxxxxxx/1...."]}I tried:reverting back to Nucleus 2.0.3 but I seem to get the same problem.The only thing that solves it, is to make the docker.sock world-writable... but that is not going to production.More info:Greengrass is running as root.I can run these commands myself with no problem in a shellprivileged containers I spin up can access the docker.sock with no problemThis happens both in my arm and amd64 devicesI don't know how to check in what user does Greengrass try to run the docker load command, but I assume it is its own user.Has anyone experienced something similar?I feel silly asking this question because it was working before, but I did not change anything so I am confused.FollowComment"

Answer: "0Accepted AnswerHi QuantumLove,the error was from your "Install" step. Can you set"RequiresPrivilege": truelike you did in step "Run" for "Install" as well?Thank you!CommentShareAWS-User-zhaoylinanswered 2 years ago0This is a sample component recipe:{"RecipeFormatVersion": "2020-01-25","ComponentName": "xxxxxxx","ComponentVersion": "1.5.0","Manifests": [{"Platform": {"os": "linux","architecture": "amd64"},"Lifecycle": {"Install": {"Setenv": {"ARCH": "amd64"},"Script": "docker load -i {artifacts:path}/artifact-${ARCH}.tar"},"Run": {"Setenv": {"ARCH": "amd64"},"Script": "docker run --rm --privileged --name xxxxxxx xxxxxxx:${ARCH}-1.5.0","RequiresPrivilege": true}},"Selections": ["all"]},{"Platform": {"os": "linux","architecture": "arm"},"Lifecycle": {"Install": {"Setenv": {"ARCH": "arm32v7"},"Script": "docker load -i {artifacts:path}/artifact-${ARCH}.tar"},"Run": {"Setenv": {"ARCH": "arm32v7"},"Script": "docker run --rm --privileged --name xxxxxxx xxxxxxx:${ARCH}-1.5.0","RequiresPrivilege": true}},"Selections": ["all"]}],"Lifecycle": {"Install": {"Script": {"all": "docker load -i {artifacts:path}/artifact-${ARCH}.tar"}},"Run": {"Script": {"all": "docker run --rm --privileged --name xxxxxxx xxxxxxx:${ARCH}-1.5.0"},"RequiresPrivilege": {"all": true}}}}CommentShareQuantumLoveanswered 2 years ago"


Question: Read AWS Bucket with Python

"Greetings,I am trying to read JSON data in an AWS Bucket using Python.I don't get any errors, but don't return any data in Python script??Any suggestions to resolve this. would be appreciated. ~/.aws/configimport os, sysimport http, httptools, aiohttpimport urllib, urllib3import boto, boto3from boto3 import Sessionfrom boto.s3.connection import S3Connection#from boto import OpenSSLfrom boto.s3 import bucketimport glob, os, jsonimport pandas as pdimport numpy### SET Directoryjson_dir = 'losses-costs/2022_12_23/France/'### SELECT Filesjson_pattern = os.path.join(json_dir, '*.json')file_list = glob.glob(json_pattern)### IMPORT filesdfs = []for file in file_list:    with open(file) as f:        json_data = pd.json_normalize(json.loads(f.read()))        json_data['site'] = file.rsplit("/", 1)[-1]    print(f) #print(df)There is null output, the folder has four files??Any assistance, appreciated.ThanksFollowComment"

Answer: "1When using Boto you need to initialize an S3 client.Then you use that client to get the object.You then need to convert the object to a usable format.Look at this link for samples on how to use Boto3 with S3 buckets. Specifically look at the Get an object from a bucket sample.https://docs.aws.amazon.com/code-library/latest/ug/python_3_s3_code_examples.htmlCommentShareYair Carelanswered 5 months ago0Hi,I see you are importing several boto related libraries, but you script doesn't actually make use of them as far as I can tell. Based on the script you shared, I believe you need to add the code that lists the objects in your bucket. Once you do that, you can continue to develop your code to load each file into memory, or download it to disk, etc.Here is a basic example using boto3 which should list the objects in your bucket.import boto3session = boto3.Session( aws_access_key_id='<your_access_key_id>', aws_secret_access_key='<your_secret_access_key>')s3 = session.resource('s3')my_bucket = s3.Bucket('<your_bucket_name>')for my_bucket_object in my_bucket.objects.all():    print(my_bucket_object.key)CommentSharemicahanswered 5 months ago"


Question: AWS Amplify: quasar build command not found

"I am facing weird issue for deploying a branch on AWS Amplify service, I have a build script which has a specific line called quasar build -m spa. This line works well for other branch deployments but for this current one it gets deployed once but re-deployments is creating the issue.FollowComment"


Question: Roles defined in Redshift

"Hello,There are several people in our group that create their own redshift database in a provisioned cluster. I created a database in that cluster as well. Now I want to define roles in database and attach RLS (Row level security) policy to the roles I create. After creating first role I got error that role already exists.How do I see what roles are already created in a Redshift database?Can I see info about the roles - created/updated date and any policies attached to the roles in Redshift?Is it possible to use same roles across different Redshift databases in same cluster and databases across different clusters?FollowComment"

Answer: "3To fetch the list of roles and the role owner you can use the below query:SELECT role_name,role_owner FROM svv_roles;Use SVV_RLS_POLICY to view a list of all row-level security policies created on the Amazon Redshift cluster.SELECT * FROM svv_rls_policy;Use SVV_RLS_RELATION to view a list of all relations that are RLS-protected. For example the below query should help you find the relation name, database name , schema name etc.SELECT datname, relschema, relname, relkind, is_rls_on FROM svv_rls_relation ORDER BY relname;Below are some links which would help you understand more about the tables & their respective columns.https://docs.aws.amazon.com/redshift/latest/dg/r_SVV_RLS_RELATION.htmlhttps://docs.aws.amazon.com/redshift/latest/dg/r_SVV_RLS_POLICY.htmlhttps://docs.amazonaws.cn/en_us/redshift/latest/dg/r_SVV_ROLES.htmlCommentShareSrikant Dasanswered 8 months ago"


Question: OnStartGameSession Not Triggering

"Hello,We're using a fresh Third Person Template in Unreal Engine 5, setup everything from this documentation link https://docs.aws.amazon.com/gamelift/latest/developerguide/integration-engines-setup-unreal.htmlAfter uploading the build to Amazon GameLift and creating a fleet from it, we get 2 clients to start matchmaking (our matchmaking rules require 2 players atm), the matchmaking ticket is stuck on PLACING and it's because the game session in the fleet is stuck in Activating state.Added prints inside the OnStartGameSession lambda and it seems that it's not printing in the server logs.We're using flexmatch with a queue pointing to an alias that is pointing to the fleet.What could the issue be?Thanks in advance.ElieFollowComment"

Answer: "0Hi,The GameLift team needs more information to debug why a game session could be stuck in ACTIVATING. I recommend cutting a customer ticket to GameLift with the following information:AWS Account Id, region, fleetId, gameSessionId, queue name, and a time frame for when the issue occurred.Thanks!CommentShareJuneHoP_AWSanswered 6 months agoTolis 6 months agoShould they also try to remotely access the GameLift instance to check what's going on as describe here https://docs.aws.amazon.com/gamelift/latest/developerguide/fleets-remote-access.html ?Share"


Question: [S3] The bandwidth of directly downloading a s3 media file by different client at the same time?

"Dear Sirs,I've tried to directly download a public s3 file via browser, the bandwidth it's about 16~24 mbps. And with CDN, the download bandwidth is over 200 mbps.Above case is one file with one download client.But, My target is to progressive download a s3 media file from different locations at the same time.Is there an official statement about the average bandwidth of directly downloading a single s3 media file from multiple client (100~200) in different regions?Thanks for your time.WellFollowComment"

Answer: "0Ever heard of CloudFront? I don't know the exact bandwidth of S3, but CloudFront limits are here:https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-limits.htmlCommentSharerePost-User-3196201answered a year ago"


Question: Cors Issue when connecting React Application to AWS Lambda using API Gateway HTTP API

"I am writing an HTTP API and have run into cors issues. I have tried a bunch of different things but have ultimately starting doing the following.Having a separate lambda function for the OPTIONS route to my api (my api only has one POST method).Sending back the following headers back in both my POST method lambda and my OPTIONS lambdaheaders: {            "Access-Control-Allow-Origin": "https://www.linkanimage.com, http://www.linkanimage.com, http://localhost:3000",            "Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, OPTIONS",            "Access-Control-Allow-Headers": "Content-Type",            "Content-Type": "text/plain"}Lastly I added the following config to the API Gateway corshttps://ibb.co/KNZQghTMy problem is - the OPTIONS cors response returns a 200 but the actual POST request still returns a CORS error:Access to fetch at 'https://api.linkanimage.com/uploadImage' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.FollowComment"

Answer: "1Are you testing this in the browser or in a different client, such as Postman? Something that usually gets me in these situations is that if you're explicitly adding the CORS headers to the response, the headers aren't added to the response if the function fails before reaching the return statement. Note that CORS is a security feature built in to browsers, so test the API with Postman, CURL or some other client to check if it is a CORS error or not.Another tip is to ensure you have an understanding of your functions' failure modes, so that you can log errors for debugging and fail gracefully.CommentShareNiklasanswered a year agopranchan a year agoThings seem to work with postman - this appears to be a cors issue - it only appears in the browserShareNiklas a year agoIf you know that you're attaching the right CORS headers, it probably means that your Lambda function code fails before reaching the return statement, leading to a 5xx response being returned, which doesn't contain the right CORS headers.Share0So, I don't know your exact architecture but;If you will serve the API and front-end on the same domain in production, and have this error while working local React app:You do not need to enable CORS on the API GW.Simply you can use your relative API path in the React as '/getUserProfile' instead of 'https://api.linkanimage.com/getUserProfile' and put proxy:'https://api.linkanimage.com' in the webpack config file. This will proxy everything and you will have no CORS issues.If you want to serve the API and front end under different domain names, then:You will be doing cross-site requests so, you have to enable CORS in API GW for every methodYou can enable cors with default values and if you want you can limit the  Access-Control-Allow-Origin to the necessary domains for more strict policies.PS.1 Regarding file upload, your content-type should be "multipart/form-data" not "text/plain"PS.2 Again, if you are using JSON for your regular (not upload) API calls, your content-type will be "application/json"CommentShareETanswered a year ago"


Question: Copy FSx across Account and across Region

"Hi Team,I need to move and sync a folder from FSx storage from AWS Account 1 AWS Regions A, to FSx AWS Account 2 AWS Regions B. How should I set up DataSync for this transfer? If DataSync is not suitable for this task what other AWS services can I use?FollowComment"

Answer: "0Take a look at this: How to replicate Amazon FSx for Windows File Server data across AWS Regions.CommentShareEXPERTkentradanswered 5 months ago0Note that since April 2021 you can also use cross-Region and cross-account backup support to copy backups across AWS Regions to implement cross-Region business continuity, and across AWS accounts to protect their backups from accidental user activity, account compromises, and ransomware.https://aws.amazon.com/blogs/storage/cross-region-and-cross-account-backups-for-amazon-fsx-using-aws-backup/CommentShareNikhil Srivastavaanswered 5 months ago"


Question: how to automatically deploy a code to ecs in AWS pipeline

"hi, how to automatically deploy a new code to ECS in AWS pipeline? Source & Build stages in pipeline work well, but Deploy stage cant be executed, it runs for an hour, then finishes with error (timeout). I need to manually stop docker container on EC2 (ECS), after that Deploy stage will work. I m using Free tier, is there any limitation for codepipeline and ecs? Maybe there is a problem with Task Placement.FollowCommentAWS-User-4253191 a year agoI found where was a problem. Problem was in Dynamic port mapping, i removed a port, and Deploy stage worked. Before it was 80 port for host. How can i configure a port to be 80 always?Share"

Answer: "0This is a question that has a lot of valid answers.You could look into letting AWS Copilot deal with with the majority of that for rather simple / straightforward use-cases.You can also look at AppRunner potentially, if your use-case fits what it can cater for.We use AWS CodePipeline + CodeBuild (and for my personal work make use of free tier as much as I can hehe) to build and deploy. The deployment of services to ECS is done with CloudFormation. The Templates for the ECS tasks / services is generated from docker compose definition files using ECS Compose-X (See examples and labs here). We use the ansible playbook here to setup everything for multi (or single) account deployments, then the rest is up to what kind of services you want to deploy and how to build them. Advantage of ECS Compose-X is that is supports compose syntax so you can run your usual docker-compose up/down commands to do local testing and create additional files specifically for AWS.We deploy services multiple times a day following this pattern and the devs can focus on writing code instead of dealing with infra.CommentShareJohnPrestonanswered a year ago0There is no restriction in free tier.Please check the reason for the error in the Deploy stage and the reason for stopping the ECS task.This will help you solve the problem.CommentSharezizianswered a year ago0We have similar question. We run ECS on EC2. Our containers are basically just runtime and access the website code on mount volume on the EC2 instances. I use CodeDeploy ECS Blue/Green deployment and it ran successfully to deploy task definition, but it is not able to deploy new website code in the shared mount volume. ECS AppSpec.yml doesn't have copy files to destination sort of thing like EC2/on-premises AppSpec.yml. We have to use Ansible to deploy new website code at this moment, but we can't restart those runtime containers and stop traffic. Is that a way of CodeDeploy Blue/ Green deployment to do so? I also tried CodeDeploy on EC2/ on-premises, but it failed to stop traffic on dynamic ports and it requires code deploy agent to be installed which should not be in ECS CodeDeploy.CommentSharerePost-User-9660871answered a year ago"


Question: Is it possible to restrict user login in AWS by IP Whitelisting?

Is it possible to restrict the user to login in AWS by IP Whitelisting? I was wondering if it possible using IAM policies or is there any other way to do it. Thanks in advance to anyone that will enlighten me.FollowCommentBrettski-AWS EXPERT16 days agoCould you provide more detail on "login to AWS"? Do you mean logging into the console? Or logging into an instance that is running in AWS? Or something else?Share

Answer: "1This looks like it may be what you are looking for https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_aws_deny-ip.htmlCommentShareRWCanswered 17 days agorePost-User-5927665 17 days agois there any rule that can prevent a user to sign in IF their IP is not in the WHITELIST?ShareRWC 17 days agoThat's what the linked example is showing - actions are denied when the source IP address is not in the list.SharerePost-User-5927665 17 days agobut I tested it, I can still login.Share0Have you looked at Console Private Access?CommentShareRodney Lesteranswered 15 days ago"


Question: How to ingest data from oracle on premise to SnowFlake

"Hi, i need to ingest data from Oracle on premise to SnowFlake, what would be the right way to do it?, using DMS? or which other options can i use?, if i have to use DMS, how to do that ingest?, thanksFollowComment"

Answer: 0AWS DMS does not support SnowFlake as a target.https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.Targets.htmlAnother possible approach is to set the DMS target to S3.I believe it is possible to load data from SnowFlake after importing data into S3.https://docs.snowflake.com/en/user-guide/data-load-s3-create-stageCommentShareEXPERTRiku_Kobayashianswered 8 days agoWilli5 8 days agoThanks RikuShare


Question: What are necessary permissions to query S3 from Athena + Spark?

"I'm trying to play with the new Athena enhancement that provides support for spark notebooks.I have data in S3 that follows a partition scheme like this:s3://my_bucket/path1/p1=1642/p2=431/p3=2023-02-02 00:00:00/*.parquet"I want to read it into a spark data frame by prefix. Something like:df = spark.read.option("recursiveFileLookup","true").parquet('s3://my_bucket/path1/p1=1642/p2=431/*')When I try that, I get an error that "path doesn't exist"Even if I put the full path to a .parquet file, I still get an error related to the p3 partition:pyspark.sql.utils.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: fdatehb=2023-02-02 00:00:00I even tried to simply read a file from S3 using straight python:`def read_file(bucket, key, encoding="utf-8") -> str:file_obj = io.BytesIO()bucket.download_fileobj(key, file_obj)wrapper = io.TextIOWrapper(file_obj, encoding=encoding)file_obj.seek(0)return wrapper.read()`And I still get errors related to permissions - which indicates to me that I have to setup some kind of relationship between Athena and the source bucket. I can already query this same data using "traditional" Athena (SQL), so it must be something additional (?)But I can't find any documentation about this.Who can point me in the right direction?FollowComment"

Answer: "0Hello,Have you set the region or try using the regional endpoint from here https://docs.aws.amazon.com/general/latest/gr/s3.html ?Other topic to check - how are you authenticating?https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html#Authentication_propertiese.g. fs.s3a.aws.credentials.providerThird question is there any s3 resource based policy / ACL in place for the bucket?And one more thing - is it S3 encrypted with KMS bucket or not? If encrypted, you need to check KMS key attached policy.CommentShareTodor Todorovanswered 4 months ago0When you configure the Athena WorkGroup for Spark, you select a role to use.The role needs to have permissions to read those s3 files.CommentShareGonzalo Herrerosanswered 3 months ago"


Question: IoT SQL function `get_dynamodb` fails unexpectedly

"Execution of the get_dynamodb function of IoT SQL fails with a strange error message when calling it with the four parameters specified in the documentation https://docs.aws.amazon.com/iot/latest/developerguide/iot-sql-functions.html.My rule definition equalsSELECT    *,    get_dynamodb("myTableName", "devEui", WirelessMetadata.LoRaWAN.DevEui, "myRoleArn") AS contextFROM 'myTopic'WHERE    message.WirelessMetadata.LoRaWAN.FPort = 10    OR message.WirelessMetadata.LoRaWAN.FPort = 32I have configured fine-grained logging for AWS IoT Core and see the following error message in CloudWatch{logLevel: ERRORreason: ExternFunctionExceptionstatus: Failuretimestamp: 2021-12-29 07:12:10.436details: Function 'GetDynamodb' failed to execute for rule 'XXX'. get_dynamodb takes either 4 or 6 arguments, but 4 given. Expected get_dynamodb(tableName, partitionKeyName, partitionKeyValue, roleArn) or get_dynamodb(tableName, partitionKeyName, partitionKeyValue, sortKeyName, sortKeyValue, roleArn).eventType: RuleExecution}Is there something I'm missing or is the IoT Core SQL function documentation outdated?FollowComment"

Answer: "0Accepted AnswerI solved this by fixing a typo in the query. Namely, the partitionKeyValue argument missed the message prefix. The error message was a bit misleading as it stated that I had called the function with four arguments even though one of these was undefined.CommentShareAWS-User-2752247answered a year ago"


Question: working examples for  python GreengrassCoreIPCClientV2().get_configuration()

"Google gives me nothing, when I try to look for working python examples ofGreengrassCoreIPCClientV2() followed by get_configuration()I cant follow the API reference well enough to figure out what to do withReturn type:GetConfigurationResponse??FollowComment"

Answer: 2Accepted AnswerHi. It's like this (for a component with config parameters foo and bar):        response = ipc_client.get_configuration()        foo = response.value['foo']        bar = response.value['bar']CommentShareEXPERTGreg_Banswered 2 months ago1Thanks! Much appreciated.CommentSharerePost-User-8057778answered 2 months ago


Question: An error occurred while calling o280.pyWriteDynamicFrame

"run ID: jr_1a42fdc73c7bbb7fee44654edf3dd44eb9f2bdb94e2c364d217c5501366580ccThis Job loads the data from S3 files to Staging table in Redshift and subsequently getting loaded into Main Transaction table.ThanksBirajEdited by: birajamishra on Aug 17, 2018 7:52 AMFollowComment"

Answer: 0post details error message from cloud watch . default glue temp credential expire after 1 hours hence use iam role if job would run more than hours.https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-redshift.htmlCommentShareShivananswered 5 years ago0I have this same issue.CommentShareSioanswered 4 years ago


Question: DMS CDC with Oracle - "Cannot retrieve Oracle archived Redo log destination ids"

"I believe I have solved my own problem, but given the hours of hair pulling I went through, I'm posting here for the benefit of others.I was setting up an Oracle to Aurora migration for a demo and I wanted to get Change Data Capture working. The DMS task continuously failed with the following error visible in the console:"Cannot retrieve Oracle archived Redo log destination ids; Failed to set stream position on context"Interestingly nothing useful logged in CloudWatch despite log level set to the max.I went through a lot of different theories, but the one that seems to fit is that given this is a static demo environment, Oracle has never actually got round to archiving any log files since it was configured for ARCHIVELOG mode. Perhaps DMS requires at least one archived log??Finally I discovered this Oracle command: "alter system switch logfile;"   I believe this forces an archive. After this the CDC task just started working.FollowComment"

Answer: "2Accepted AnswerThank you for sharing to others. Yes, anytime DBA set up in archivelog mode on the DB, first thing to do is run multiple times "alter system switch logfile" and validate "archive log list" by ensuring the archive log number prior to confirming that DB is on archivelog mode. Additional good practices, ensure archive log directory has sufficient space to generate log files, log file retention period is set by a shell script or other processes.CommentShareMODERATORAWS-User-8855869answered 3 years ago0Thanks! Helped me to get replication task working properly.CommentSharedanv006answered a year ago"


Question: Duplicate an existing serverless app infra(built in the console management) using terraformer

Hello !I have already an AWS infrastructure built manually. Now i want to practise infrastructure as code so i used a tool called terraformer to extract the existing infra as code. But i can not use this code to create an infra in an other AWS account using terraform command. Could you help me please? when i run terraform apply i have the error : can not read state fileFollowCommentchaitanya a year agoif you want to load existing infra use terraform import command.don't touch the state file at any case.Share

Answer: "1Hey - the terraform state file is unique to each deployment. It sounds like your new deployment is trying to reach the old state file.I'd recommend going through the HashiCorp AWS Terraform tutorials and learning the product, before using another tool to extract IaC from an existing account.CommentShareJBanswered 8 months ago"


Question: there's no option to change node.js version

"hello.My node app has a dep which is not compatible with the node version >= 12.19.0(ref: https://github.com/prisma/prisma-client-js/issues/907)So I need to fix the version of 12.18.0 or lower.I looked into the guides https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_nodejs.container.html?tag=brevis-20 , but my console does not show such an option. https://user-images.githubusercontent.com/1215767/99962519-ac7aa400-2dd3-11eb-9085-4e9edb9d970b.pngcan anyone guide me to change the node version?thanks.FollowComment"

Answer: 0resolved by specifying node version in package.jsonCommentSharekyuwooanswered 3 years ago


Question: "HIVE_CURSOR_ERROR: Failed to read Parquet file, when i using Athena engine version 3 in my workgroup"

"When I change the version of athena to v3 in the workgroup I am having the error HIVE_CURSOR_ERROR: Failed to read Parquet file. When I using Athena engine v2 the query works normally. I'm just running a table preview, I didn't change anythingSELECT * FROM {table} limit 10;FollowComment"

Answer: "0I'm experiencing the same thing.I have a parquet file that reads fine in athena engine 3 that is not loading.I downgraded the engine version to 2 in the athena workgroup and it works fine.Is this a bug in Athena?CommentSharekadeanswered 25 days ago0I'm also experiencing the same thing. Downgrading to v2 helps. If someone from AWS is interested, I can provide file/config/queries which is causing issues.CommentSharetobiaszanswered 15 days ago0Experiencing the same issue. I uploaded a lot of parquet files and it fails to query over new + old combined.I inspected the schema with parquet-tools and they are the exact same, so no issue there.Downgrading to v2 also helped us, but why?CommentSharerePost-User-0184675answered 6 days ago"


Question: what is "running tasks" in enhanced monitoring of aurora MYSQL?

what is "running tasks" in enhanced monitoring of aurora MYSQL?FollowComment


Question: Error running TestNG tests for apk file with different Android versions

"Hi,I am trying run a Appium script written in TestNG for a Android Mobile application in device farm.When I run the test in device farm devices which are running lower version of Android like 6 and 7 run the tests successfully. However the devices runnning OS version 8 and 9 fail.All the devices fail with the same errorIt is impossible to create a new session because 'createSession' which takes HttpClient, InputStream and long was not found or it is not accessible Build info: version: '3.12.0', revision: '7c6e0b3', time: '2018-05-08T14:04:26.12Z' System info: host: 'ip-172-31-13-65', ip: '172.31.13.65', os.name: 'Linux', os.arch: 'amd64', os.version: '3.13.0-139-generic', java.version: '1.8.0_151' Driver info: driver.version: AndroidDriverI have attached screenshot as well.Any help on this would be highly appreciatedEdited by: UserDeviceFarm on Jan 10, 2019 3:21 AMFollowComment"

Answer: "0Hello,Could I get a run URL of where your tests are failing so that I can look deeper into this?ThanksCommentShareAWS-User-1182145answered 4 years ago0Hi Scott,Find below the URL of the testshttps://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/866204ae-2f1b-4345-b960-144f88e118d7/runs/bb879082-d3ff-442e-925b-feb50203dc36CommentShareUserDeviceFarmanswered 4 years ago0Hi Scott,Please find the URL of the testshttps://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/866204ae-2f1b-4345-b960-144f88e118d7/runs/bb879082-d3ff-442e-925b-feb50203dc36Could you please look into this issue and let me know. I am trying to complete the execution on Android devices but because of the above issue I am not able to proceed further.CommentShareUserDeviceFarmanswered 4 years ago0Any update on this ?Thank you for your help in advance.CommentShareUserDeviceFarmanswered 4 years ago0HiI pulled your Appium server logs for both android 7 and android 8 devices.Looks like you are using webview in your tests.The weview is not getting loaded on Android 8+ devices.autoWebview: trueOn the failed devices (8 and 9) it is not able to fetch web viewGetting process name for webview2019-01-29 22:35:55:373 - [debug] [ADB] Getting connected devices...2019-01-29 22:35:55:471 - [debug] [ADB] 1 device(s) connected2019-01-29 22:35:55:472 - [debug] [ADB] Running '/opt/dev/android-sdk-linux/platform-tools/adb' with args: ["-P",5037,"-s","HT79C1A06060","shell","ps"]2019-01-29 22:35:55:755 - [debug] [AndroidDriver] Returning process name: 'unknown'2019-01-29 22:35:55:756 - [debug] [AndroidDriver] Found webviews: ["WEBVIEW_unknown"]On passed devices (6 and 7)Getting process name for webview2019-01-29 22:35:14:106 - [debug] [ADB] Getting connected devices...2019-01-29 22:35:14:237 - [debug] [ADB] 1 device(s) connected2019-01-29 22:35:14:237 - [debug] [ADB] Running '/opt/dev/android-sdk-linux/platform-tools/adb' with args: ["-P",5037,"-s","0117113b985a0301","shell","ps"]2019-01-29 22:35:14:507 - [debug] [AndroidDriver] Parsed pid: '31825' pkg: 'io.ionic.myplacev3' from2019-01-29 22:35:14:507 - [debug] [AndroidDriver]     USER      PID   PPID  VSIZE  RSS   WCHAN              PC  NAME2019-01-29 22:35:14:508 - [debug] [AndroidDriver]     u0_a1581  31825 3119  3336420 192196 SyS_epoll_ 0000000000 S io.ionic.myplacev32019-01-29 22:35:14:508 - [debug] [AndroidDriver] Returning process name: 'io.ionic.myplacev3'2019-01-29 22:35:14:509 - [debug] [AndroidDriver] Found webviews: ["WEBVIEW_io.ionic.myplacev3"]2019-01-29 22:35:14:573 - [debug] [AndroidDriver] Available contexts: ["NATIVE_APP","WEBVIEW_io.ionic.myplacev3"]2019-01-29 22:35:14:577 - [debug] [AndroidDriver] Connecting to chrome-backed webview context 'WEBVIEW_io.ionic.myplacev3'Can you try the following thingsYou have hardcoded platform version as 8.1 in ur tests. can you remove this from your test codeJust wanted to double confirm that you are actually testing a web view ?Meanwhile we will investigate from our side on the possible rootcauseCommentShareAWS-User-6552639answered 4 years ago0HiWe have identified that this is an issue on our side (related to adb version we are using). I will discuss with the team and post and ETA for the fixCommentShareAWS-User-6552639answered 4 years ago0The fix for this has now been deployed to all devices.Please reopen or create a new post if you face the issue againCommentShareAWS-User-6552639answered 4 years ago"


"Dear All:I have a VPC with its respective CIDR1 (with 3 subnets) which is attached to a Transit Gateway. Then another CIDR2 (1 subnet) is added to said VPC. Is it possible that this new CIDR can be attached to the same TGW? Finally, is it possible that a VPC that has 2 CIDRs can be attached to the same transit gateway?I await your comments, thank you very much, best regards.FollowCommentTushar_J EXPERT7 months agoHello, Did this work for you?Share"

Answer: "0Yes, this should work just fine. If you propagate the VPC attachment in the TransitGateway route table it should automatically install the route for the VPC secondary CIDR as well.CommentShareEXPERTTushar_Janswered 7 months ago"