Question: emr serverless custom image

"Hi,We are comparing the emr/eks and emr serverless offer. We are trying to achieve the tpc-ds perf benchmark through (https://github.com/aws-samples/emr-on-eks-benchmark). We are not able to do it on emr/serverless as some pre requiste are on os part like /opt/tpcds-kit/tools  (https://github.com/aws-samples/emr-on-eks-benchmark/blob/main/docker/benchmark-util/Dockerfile) .Is there any way to set os/binaries pre requisite on serverless workers to achieve this use case ?BR,AbdelFollowComment"

Answer: "1Hi, Thanks for writing to re:Post.I Understand that you want help in running benchmarks for EMR Serverless using TPC-DS.The below listed steps should assist you in running the benchmark!Below are the step to generate 300Gb data and run TPC-DS test.Source code - https://github.com/aws-samples/emr-on-eks-benchmarkPull public spark benchmark docker image and SSH into docker imagedocker pull ghcr.io/aws-samples/eks-spark-benchmark:3.1.2docker run -it --user root ghcr.io/aws-samples/eks-spark-benchmark:3.1.2  bashInstall AWS CLI version 2 within docker imageRun following commands:apt updateapt install curl unzip -ycurl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip " -o "awscliv2.zip "unzip awscliv2.zip./aws/installRun 'aws --version' to verify AWS CLI is installed successfully.Configure AWS CLIRun 'aws configure'=> The following example shows sample values. Replace them with your own values as described in the following sections.$ aws configureAWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLEAWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEYDefault region name [None]: us-west-2Default output format [None]: json[+] https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-configPrepare tools.tar.gz for dependencycd /opt/tpcds-kit/chown -R hadoop:hadoop toolstar -czvf tool.tar.gz toolsCopy following files to S3 location using 'aws s3 cp' commandaws s3 cp /opt/tpcds-kit/tool.tar.gz s3://<your-bucker>/aws s3 cp /opt/spark/examples/jars/eks-spark-benchmark-assembly-1.0.jar s3://<your-bucker>/Uploaded jar and tool kit tar in s3 would be required to call when starting benchmark jobs in next steps.Generate TPCDS benchmark data and put it S3.aws emr-serverless start-job-run --application-id "<application id>" --execution-role-arn "<ARN of emr-serverless-job-role>" --region <region> --job-driver '{"sparkSubmit": {"entryPoint": "s3://<your-bucket>/eks-spark-benchmark-assembly-1.0.jar","entryPointArguments": ["s3://<your-bucket>/input-data/","/home/hadoop/environment/tools", "parquet","300","200","true", "true", "true"],"sparkSubmitParameters": "--class com.amazonaws.eks.tpcds.DataGeneration --archives s3://<your-bucket>/tools.tar.gz#environment --conf spark.driver.cores=4 --conf spark.driver.memory=10g --conf spark.executor.cores=4 --conf spark.executor.memory=10g --conf spark.executor.instances=27"}}' --configuration-overrides '{"monitoringConfiguration": {"s3MonitoringConfiguration": {"logUri": "<s3://<your-bucket>/logs/>"}}}'Run TPCDS Benchmark test with the data generated in S3aws emr-serverless start-job-run --application-id "<application id>" --execution-role-arn "ARN of emr-serverless-job-role" --region <region> --job-driver '{"sparkSubmit": {"entryPoint": "<s3://<your-bucket>/eks-spark-benchmark-assembly-1.0.jar>","entryPointArguments": ["s3://<your-bucket>/input-data/","s3://<your-bucket>/output/","/home/hadoop/environment/tools", "parquet","300","3","false","q1-v2.4","true"],"sparkSubmitParameters": "--class com.amazonaws.eks.tpcds.BenchmarkSQL --archives s3://<your-bucket>/tools.tar.gz#environment --conf spark.driver.cores=4 --conf spark.driver.memory=10g --conf spark.executor.cores=4 --conf spark.executor.memory=10g --conf spark.executor.instances=47"}}' --configuration-overrides '{"monitoringConfiguration": {"s3MonitoringConfiguration": {"logUri": "<s3://<your-bucket>/logs/>"}}}'In above steps please update S3 location, region as per your environment. Please ensure EMR Serverless job execution role have required permissions on the S3 buckets for reading dependency files and writing data.I hope you find this information helpful.Thank you and have a good rest of your day!CommentShareSUPPORT ENGINEERMudasser_Aanswered 8 months agoEXPERTFabrizio@AWSreviewed 8 months ago"


Question: MSK Serverless Bootstrap Server Endpoint does not resolve

I followed this tutorial to provision a serverless MSK cluster.I am not able to connect to the cluster.This troubleshooting guide recommends testing the connection via telnet:telnet bootstrap-broker port-numberI receive this response:telnet: could not resolve abc.c2.kafka-serverless.us-west-2.amazonaws.com/9098: Name or service not knownDoes anyone know how I can connect a client to serverless MSK?Note:I've followed the same steps with a provisioned cluster. These steps seem to work with a provisioned cluster. Perhaps the docs are just wrong.FollowComment

Answer: "1Hello,I understand that you are experiencing following error when running telnet command to troubleshoot your MSK serverless endpoint:telnet: could not resolve abc.c2.kafka-serverless.us-west-2.amazonaws.com/9098: Name or service not knownI would request you to please check your telnet command and run the command again to confirm your network connectivity.Telnet command syntax:telnet <endpoint> <port number>From the error message, I can see you are passing port number along with your endpoint (abc.c2.kafka-serverless.us-west-2.amazonaws.com/9098) due to which you are receiving above error. Correct telnet command syntax is shared below:telnet abc.c2.kafka-serverless.us-west-2.amazonaws.com 9098[+] Troubleshooting your Amazon MSK cluster - Unable to access cluster from within AWS: Networking issues - https://docs.aws.amazon.com/msk/latest/developerguide/troubleshooting.html#networking-troubleCommentShareSUPPORT ENGINEERRajat_Canswered a month ago"


Question: AWS Control Tower - Ownership account

"Hello,I am trying to figure something out that I noticed during my deployment of AWS Control Tower. There seems to be something different then I expected in the account page of the Control Tower landing page.The accounts like Log Archive and Audit all have their ownership to AWS Control Tower, you can see this via de GUI on the Control Tower Landing page.However, I used the account factory to spawn an account in a new OU (Shared Services). This account is called the Network account. The ownership of this account is set to: "Self"My question really is: Why is it set to "Self", what influences this decision?FollowComment"

Answer: "1After deploying Control Tower, you can enroll additional accounts. These accounts are self-managed under the defined Organization, and the email provided in their creation is the full owner of the account.Suggest to review the Multi-Account Whitepaper https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/organizing-your-aws-environment.html  andControl Tower user guide https://docs.aws.amazon.com/controltower/latest/userguide/account-factory.htmlCommentShareVladanswered 9 months ago"


Question: S3 bucket behind VPC needing custom SSL cert

Hi ThereI have gotten a S3 bucket to work behind VPC Endpoint and custom url. I want to change the SSL cert the bucket is using without using cloudfront. We are planning to expose the VPC IP address via our gateway vm it is currently working just the SSL certificate is still showing up s3.amazon one. Is it possible to change the cert?FollowComment

Answer: "0The short answer is no. First because S3 static websites feature do not support SSL. This is a total different feature than using S3 as a regular object storage, in which you do not use HTTP protocol to GET HTML web pages or other static content, you use API calls to S3 API which are different endpoints (and they are TLS with s3.amazon certificate). So, when you deploy a VPC Endpoint (it doesn't matter Network or Gateway) you are accessing the S3 API not the feature which supports WebSite hosting which has a totally different endpoint and cannot be accessed using VPC Endpoints for this reason.So, if you want to access your website form a private IP you cannot use VPC Endpoints. If you want to expose a public website with a custom SSL certificate, the best approach is to use CloudFront (pay per use and you remove proxy management).I hope to have helped you to clarify your question.Best,CommentShareJuanEn_Ganswered 7 months ago"


Question: Are Rate-Based Rules for AWS WAF Global or per Edge

"The article Protect Web Sites & Services Using Rate-Based Rules for AWS WAF discusses how to define a Rate-based Rule to control when IP addresses are added to and removed from a blacklist.Are these rate limits evaluated globally, across all CloudFront edges that have WAF deployed, or are they evaluated locally, in each edge location individually?FollowComment"

Answer: 0Accepted AnswerIt's done globally.  Aggregated across all POPs.CommentShareAWS-Tinoanswered 6 years ago


Question: NET::ERR_CERT_COMMON_NAME_INVALID with Cloud front

"I have an S3-hosted static website and a Cloud Front distribution. THere is an A record in Route 53 pointing to the Cloud Front distribution. When I created the distribution, I accepted the default setting for the certificate and I selected the origin from the list of s3 buckets.. I did not enter a certificate ID or request a certificate. When I attempt to browse to the site with Chrome, I get NET::ERR_CERT_COMMON_NAME_INVALID.Since there is apparently no way to get information about the default certificate, it is impossible to compare its common name with the domain,FollowComment"

Answer: 1Hello!You have to create a custom SSL certificate for your domain with ACM (Amazon Certificate Manager) or import into ACM a certificate you already own. You then have to configure the certificate in your distribution settings. Follow the instructions here: https://aws.amazon.com/premiumsupport/knowledge-center/install-ssl-cloudfront/In order to use a custom domain you have to set up an alternate domain name (CNAME) for your distribution. Here are the instructions: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CNAMEs.htmlCommentSharemasseanswered 7 months ago


Question: Dynamodb estimation for capacity reservation

"Hi Team,My customer is planning to buy dynamodb capacity reservations and wanted to make sure he is purchasing correct amount of capacity.Is there any standard/reference document for estimating the right capacity for reservation based on historical usage? We can consider the average values but just wanted to make sure is there any best practice document for correct estimates.Thank you!FollowComment"

Answer: 1Accepted AnswerAverages are usually good as you don't want to over reserve and pay for more than you use. There are efforts to create a tool to help with capacity recommendations however I have no ETA on that.CommentShareEXPERTLeeroy Hannigananswered 9 months agoJoseph 9 months agoThank you!Share


Question: KMS decrypt error inside Lambda in VPC network timeout

"I have a lambda which needs to be exposed to the internet and also needs outgoing internet network access.This lambda needs to decrypt some environment variables on startup (KMS). This encryption fails sometimes.This lambda is running in a VPC with 2 private subnets and 1 public. The private subnets have a routing table to NAT-gateway.The lambda has required KMS permissions.  Also I configured a VPC endpoint for KMS.However I get frequently network timeout errors to KMSINFO Decrypt error: Error: read ECONNRESET at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) { errno: -104, code: 'ECONNRESET', syscall: 'read', name: 'TimeoutError', '$metadata': { attempts: 7, totalRetryDelay: 5866 } }Any suggestions are more then welcome.Thanks in advance.FollowComment"

Answer: "0Accepted AnswerIs the subnet where Lambda is located a private subnet?Or is it a public subnet?Do you have KMS VPC endpoints set up on public subnets as well?Lambdas in the VPC do not have a global IP address even if they are located on a public subnet, so you need to set up a VPC endpoint.Alternatively, you can also communicate by manually setting a public IP address in the Lambda's ENI.Otherwise, make sure that the required security groups are set up for the KMS VPC endpointsCommentShareEXPERTRiku_Kobayashianswered 2 months agoFilip Vercauteren 2 months agoThe lambda is on both a private and public subnet. I don't have KMS VPC endpoints setup on the public subnets only on private subnets. I will try that and see whether this solves the problem. Thanks !Share"


Question: Nice DCV movie quality question.

"Hi,In NiceDCV, when you play a movie, the movie quality will be reduced with some block noise.When you stop the movie, the movie quality will return to normal.Is it possible to force the movie quality to stay at the full quality during video playback?(It would be nice if there was an option to easily switch between the two, even at the expense of fps and response time.dcv server : centos 7 (+ nvidia gpu)dcv client : windows 10FollowComment"

Answer: "0Hi,are you using the TCP based or QUIC/UDP based protocol for movie streaming for best performance?Which version of DCV do you use?Did you try setting best quality in the Connection Settings -> Streaming Mode?Block noise seem to indicate you are hitting bandwidth limits so your fps will probably drop when increasing the quality.Best regards,KarstenCommentShareKarstenNISPanswered 2 years ago0Hi,are you using the TCP based or QUIC/UDP based protocol for movie streaming for best performance?I tried both, and both produced block noise.Which version of DCV do you use?server 2021.1.10557-1gl 2021.1.937-1xdcv 2021.1.392-1client 2021.1-7713Did you try setting best quality in the Connection Settings -> Streaming Mode?Yes, streaming mode is the best quality.I also tried changing various settings by referring to the following web site,but I could not completely remove the block noise.https://www.ni-sp.com/support/nice-dcv-tips-and-tricks/regards.szkCommentShareszkanswered 2 years ago"


Question: Amazon Connect Dropping Inbound Calls Instead of Transferring to External Phone Number

"Hello --This was a problem earlier this year but I thought AWS had fixed it.  There was something going on with  Amazon Connect and the servers out in Virginia, I believe?  From the notifications that I received AWS corrected the issues and my company was able to receive inbound calls again.  Now, most of the inbound calls to our AWS 800 phone line and our DID phone line are being dropped instead of getting transferred to our virtual receptionist that is a private phone line for our company.According to the Cloud Watch logs there are no errors with Amazon Connect, and it is not our virtual receptionist as that is working.  When I call our AWS 800 phone number or our AWS DID phone number I receive the standard greeting and am then transferred to our virtual receptionist where I can select an extension and be connected to our staff.  When a customer outside of our company dials in on the 800 number or the DID number they receive the greeting but instead of being transferred externally to our virtual receptionist the call simply ends.  Some customers have reported hearing a message that says there are call restrictions and their call cannot be completed.Due to the major issues with the AWS Virginia servers recently reported I suspect the Amazon Connect team needs to work on the affected servers and provide AWS customers with more support until these problems are properly resolved.FollowComment"

Answer: "0Accepted AnswerI have to imagine this has happened sporadically, right? I've not seen such an issue, but I generally don't build Connect out of the East region. I suspect a support ticket is your only real way forward.davidCommentSharedmaciasanswered a year agoAWS-User-8580712 a year agoI think I may have resolved the issues actually.  It was a combination of service provider failures and no notifications or alerts by them.  Earlier this year we were using US west coast servers and those were not handling AWS Connect calls at all.  The east coast servers have been the most reliable when using new and experimental AWS products and services.Share"


Question: Open Search Cross Cluster Replication

"Understanding Active Passive Cross Cluster replication for AWS Open Search[1] is pretty straightforward. You have a leader index in the active cluster that replicates everything to the follower index in the passive cluster. In case there is a DR and we witness a region failure, we can continue to process read operations through the follower index on the passive cluster.However, the documentation provided does not go through the following two points.How do we handle write operations that come during the time the active cluster is down. Is it possible to write on the passive cluster during this time and the primary would catch up when it comes back online or only read is possible during this time, and we need to batch process all the writes once active cluster comes back up ?When active cluster comes back online, does Open Search automatically starts redirecting search queries back to active cluster instead of the passive cluster ?In addition, in case we wish to test the DR failover setup for Open Search, is it possible to simulate active cluster failure and test ?[1] https://docs.aws.amazon.com/opensearch-service/latest/developerguide/replication.htmlFollowComment"

Answer: "0Hi. That's a great question. Below is the clarification :Currently, replication under OpenSearch works in active-passive model. The follower will not take any write traffic and is only open for search. Yes, once the leader cluster is back online, the writes have to be made on the leader(/active) cluster.Each cluster (leader and follower) will have separate endpoint and OpenSearch doesn't redirect the search traffic. Search traffic will continue on the configured search endpoint at the client.Inducing failure directly on the managed service nodes is not possible.Please follow the steps for the failover onto the follower cluster:Terminate the connection between follower and leader cluster (Connection APIs)Trigger stop on all the follower indices that were getting replication: https://opensearch.org/docs/latest/replication-plugin/get-started/#stop-replicationOnce replication is stopped on all the indices, the follower indices should be able to take write traffic.Let us know, If you have any further questions.CommentShareSUPPORT ENGINEERVarun_Sanswered a year agoAWS-User-0334767 a year agoThanks for your help Varun.Share"


Question: Authenticating with StreamManager in python Docker container

"In the developer guide for Greengrass V2, it says that to use inter-process communication, a docker container must have the following environment variables set.AWS_REGIONSVCUIDAWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENTAWS_CONTAINER_AUTHORIZATION_TOKENAWS_CONTAINER_CREDENTIALS_FULL_URIhttps://docs.aws.amazon.com/greengrass/v2/developerguide/run-docker-container.htmlI have written a simple Docker container that simply installs the stream manager sdk, and attempts to create a StreamManagerClient instance. The Dockerfile looks like this:FROM python:3.9-alpineENV PYTHONUNBUFFERED=1WORKDIR /workdirRUN apk add git && pip install git+https://github.com/aws-greengrass/aws-greengrass-stream-manager-sdk-pythonCOPY test_connect.py test_connect.pyCMD [ "python", "test_connect.py" ]The python script test_connect.py looks like this:from stream_manager import StreamManagerClientdef main():    client = StreamManagerClient()    print(f"Connected to client: {client}")if __name__ == "__main__":    main()    The "Run" Lifecycle step in my component recipe looks like this:docker run -v /greengrass/v2:/greengrass/v2 -e AWS_REGION=$AWS_REGION -e SVCUID=$SVCUID -e AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT=$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT -e AWS_CONTAINER_AUTHORIZATION_TOKEN=$AWS_CONTAINER_AUTHORIZATION_TOKEN -e AWS_CONTAINER_CREDENTIALS_FULL_URI=$AWS_CONTAINER_CREDENTIALS_FULL_URI --rm stream-test-dockerWhen this component runs, the python script fails with the following error:Connection error while connecting to server: [Errno 111] Connect call failed ('127.0.0.1', 8088)Traceback (most recent call last):  File "/workdir/test_connect.py", line 10, in <module>    main()      File "/workdir/test_connect.py", line 5, in main    client = StreamManagerClient()  File "/usr/local/lib/python3.9/site-packages/stream_manager/streammanagerclient.py", line 112, in __init__    UtilInternal.sync(self.__connect(), loop=self.__loop)  File "/usr/local/lib/python3.9/site-packages/stream_manager/utilinternal.py", line 39, in sync    return asyncio.run_coroutine_threadsafe(coro, loop=loop).result()  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 440, in result    return self.__get_result()  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result    raise self._exception  File "/usr/local/lib/python3.9/site-packages/stream_manager/streammanagerclient.py", line 140, in __connect    self.__reader, self.__writer = await asyncio.wait_for(  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 481, in wait_for    return fut.result()  File "/usr/local/lib/python3.9/asyncio/streams.py", line 52, in open_connection    transport, _ = await loop.create_connection(  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1056, in create_connection    raise exceptions[0]  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1041, in create_connection    sock = await self._connect_sock(  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 955, in _connect_sock    await self.sock_connect(sock, address)  File "/usr/local/lib/python3.9/asyncio/selector_events.py", line 502, in sock_connect    return await fut  File "/usr/local/lib/python3.9/asyncio/selector_events.py", line 537, in _sock_connect_cb    raise OSError(err, f'Connect call failed {address}')ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 8088)I have confirmed that the relevant environment variables are being mapped into the container, as is my greengrass root dir "/greengrass/v2". This is the same error returned by any other non-greengrass client on the system (i.e. if I run the same python script outside of the docker container). I have also tried setting the StreamManager config STREAM_MANAGER_AUTHENTICATE_CLIENT to "false", with no luck (though I don't believe this should be the case, as the container is running via Greengrass and has all the recommended environment variables set; the documentation was quite vague on this ).The only successful workaround I was able to find was to set --network=host in my docker run command, but this is not ideal, as there should be no need to operate this particular component's container in host networking mode aside from this. Is this possibly the only way?So in summary, my questions:What is the appropriate method for interacting with StreamManager inside a greengrass-managed docker container?What is the correct StreamManager configuration value for STREAM_MANAGER_AUTHENTICATE_CLIENT when using a Greengrass-managed docker container (as opposed to a docker container or other process unrelated to Greengrass).FollowComment"

Answer: "0Accepted AnswerHi,Stream Manager binds to the localhost which is probably the reason why the client inside the container is not able to connect to the SM server.I suggest you try connecting via the gateway of the network used by your container. If you are using the default network, this is most probably the network named bridge.You can list the available networks by running docker network lsFor instance my available networks are:NETWORK ID          NAME                DRIVER              SCOPE  9d81c010d50a        bridge              bridge              local  d1fdeb535122        host                host                local  72f2a273972a        none                null                local  Now you should find the network your container is running in.To do that run docker inspect --format='{{json .NetworkSettings.Networks}}' <NAME_OF_THE_CONTAINER>For me it is bridge:{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"9d81c010d50a3e2b2d175e68bcdac6edb6c440677af479dc4256f612f6d4146c","EndpointID":"702afd9f1cab8d837baf9e0917a3a28fc7131e2b92acc8124500e2d05a178f58","Gateway":"240.10.0.1","IPAddress":"240.10.0.2","IPPrefixLen":24,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:f0:0a:00:02","DriverOpts":null}}  Ie, my container is running with bridge network which is also the default. The gateway of the bridge network should be 172.17.0.1 by default, we can confirm that using docker network inspect bridgeOnce you have the gateway address. You need to configure the Stream Manager client to talk over the gateway.Modifying your example:from stream_manager import StreamManagerClient     def main():      client = StreamManagerClient(host='172.17.0.1')      print(f"Connected to client: {client}")     if __name__ == "__main__":      main()    To learn more about docker networking: https://docs.docker.com/engine/tutorials/networkingcontainers/You can also get the address outside the container by running HOST_GATEWAY_ADDRESS=$(ip addr show | grep "\binet\b.*\bdocker0\b" | awk '{print $2}' | cut -d '/' -f 1) and then injected the variable into the container to use in code.ThanksCommentSharesushantAtAWSanswered 2 years ago0Thank you for the detailed reply! That makes much more sense.CommentSharetraviprossanswered 2 years ago"


Question: Softphone failed errors

"Hi,Posting this again as we are having the same issues even when people are already working at home.We are currently bombarded with the "Softphone call failed" error and is significantly affecting our business operations.These are the errors but sometimes it would just a blank page.1.) Softphone call failedFailed to establish softphone connection. Try again or contact your administrator with thefollowing.Browser unable to establish media channel withturn: 18.233.213.226:3478?transport=udpturn: 18.233.213.230:3478?transport=udp2.) Softphone call failedFailed to establish softphone connection. Try again or contact your administrator with the following:Browser connected but failed handshake with wss://rtc.connect-telecom.us-east-1.amazonaws.com/LilyRTC3.) Real-time communication errorFailed to establish your softphone connection. Try again or contact your administrator with the following:Call failed due to a browser-side WebRTC issue.We had this problem when we were still working in the office and now everyone when working at home. We never had any issues with our previous platform in regards to inbound and outbound traffic. Any help is appreciated.Edited by: realcapone on May 4, 2020 7:52 AMFollowComment"

Answer: "0My first question would be is the client browser going through a VPN or internal network as this to me indicates that a firewall is blocking this as most of the signalling is via port 443 (https) which is usually allowed. But the UDP port (3478) need to be specifically opened.CommentSharem66rusanswered 3 years ago0No VPN, everyone is at their own internal network for the moment.As for opening of the 3478 port. This has been taken cared of. And even back when we were in the office but was still receiving the same set of errors. Thank you.CommentSharerealcaponeanswered 3 years ago"


Question: Get Glue Data Catalog Name/ID

How do I get the name or ID of the/my data catalog in Glue?FollowComment

Answer: "0Hello,AWS Glue Catalog ID is the AWS account ID. This is mentioned on our AWS Docs here [1] as well -Each AWS account has a single Data Catalog in an AWS Region with the 12-digit account ID as the catalog ID.Additionally, in regards to querying the Glue Data Catalog metadata, you can leverage SQL queries in Athena as mentioned here on our AWS Docs [2].To obtain AWS Glue Catalog metadata, you query the information_schema database on the Athena backend.References:[1]https://docs.aws.amazon.com/glue/latest/dg/glue-specifying-resource-arns.html#data-catalog-resource-arns[2] https://docs.aws.amazon.com/athena/latest/ug/querying-glue-catalog.htmlCommentShareSUPPORT ENGINEERYash_Canswered a year ago"


Question: Best Practice to Redrive events from a Lambda DLQ

"We have a Lambda Function triggered from EventBridge events. If the lambda fails it will write the records to a DLQ. What is the best practice to redrive those messages back to the lambda. Note, we can not use the built in redrive method bc that will only work with bc that is only to send the messages to another SQS queue (not a lambda). My only thought is to build a 2nd "redrive" lambda that pulls the message off the queue and sends them 1 by 1 to the original lambda. Also of note, we don't want this process to be automatic, whatever caused the failures needs to be addressed before we manually redrive the events.FollowComment"

Answer: "0Have you considered Lambda Destinations? With Destinations, you can move messages to SNS and then fan out to SQS and send automated alerts to concerned teams. When the issue is fixed, the messages can then be picked up from the SQS and moved to the EventBridge.CommentShareSatrajit Basuanswered 6 months agoAndrew Harmon 6 months agoSo my concern with this is that A) multiple things may subscribe to the event and some of those may be successful. So I don't want those other things to trigger again if the event is placed back on EventBridge. And B), this requires an SNS and another SQS resource (for each lambda i setup this way). That sounds like a lot of additional resources just to redrive events back to the lambda. SQS supports this nicely with the redrive features, thinks that land on the DLQ can easily be sent back to the main queue. With Lambda DLQ, it would be great if you could redrive the events back to the lambda internal queue, but that feature just doesn't exist (yet). Its just interesting that AWS offers this DLQ feature on lambdas but no real examples or best practices on how to utilize it.Share"


Question: Troubleshoot document access denial

I am getting this message when I open my document URL. Help! Cannot figure out what I have done wrong - it worked prior to this.This XML file does not appear to have any style information associated with it. The document tree is shown below.<Error>AccessDenied<Message>Access Denied</Message><RequestId>VFN87XHBQMBPTT0M</RequestId><HostId>ZwsItvtUmYrz+zoSQD+4NNKULFqPAafW56yvumX9piRjV22hpyieax5gXGpo18ai/zBxk9yF8Ik=</HostId></Error>FollowComment

Answer: 0Can you post the document URL here as well so we can take a further check? Thanks.CommentShareBruceanswered 10 months ago


Question: "SNS SMS intermittent failure on specific MNC in sweden (Telia, Tele2)"

"HiSince the 29th of september we've been receiving intermittent failures when sending sms. The content in sms are similiar and the api calls the same.The only two mnc's that has failed is 1 and 7 in Sweden. We mostly send sms to carriers in Sweden. Tough mnc 1 and 7 (Telia and Tele2 respectively) stands for about half the amount of sms we send.Example error:{"notification": {"messageId": "35f21fc4-a893-5c45-8689-460f1f92be24","timestamp": "2021-09-27 12:10:41.446"},"delivery": {"mnc": 1,"numberOfMessageParts": 3,"destination": "+4670...","priceInUSD": 0.21768,"smsType": "Transactional","mcc": 240,"providerResponse": "Unknown error attempting to reach phone","dwellTimeMs": 385,"dwellTimeMsUntilDeviceAck": 3573},"status": "FAILURE"}Edited by: misterorange on Sep 30, 2021 1:47 AMEdited by: misterorange on Sep 30, 2021 1:48 AMFollowComment"

Answer: 0Amazon has responded that it is due to our sms containing links and some nordic operators has begun blocking sms with links in them.To get pass this we were told to create a template of the contents that they will send to the operators.CommentSharemisterorangeanswered 2 years ago


Question: Access lambda in custom VPC from public API Gateway

"I have a lambda function HTTP REST API which is exposed to the public via API Gateway endpoint. This works just fine if the lambda is assigned to the default VPC. But I need to have a custom VPC to make outbound calls via specific IP. However, as soon as I assign the lambda to the custom VPC, the REST API is no longer accessible via API Gateway. How can I make the lambda in custom VPC accessible again?FollowComment"

Answer: "1Lambda functions, regardless of their VPC setting, are invoked using the Lambda service public API. This means that from the point of view of API Gateway it doesn't matter what the VPC setting is.You did not mention what the error is, but maybe the function is accessible but it doesn't have proper access to the internet configured. For example: did you assign the function to a private subnet? Do you have a NAT Gateway configured in a public subnet in that VPC? Do you have a routing table with 0.0.0.0/0 pointing to the NAT Gateway in that private subnet?CommentShareEXPERTUrianswered a year agoAlan Oberto a year agoAgree with Uri, the API, with the proper IAM or Resources permissions will be able to invoke the lambda, no matter which VPC you place it. However, if the lambda is places in a subnet with no proper routing to internet or AWS services (using vpc links) there is no way you can communicate with external dependenciesShare0Accepted AnswerI had sorted this out some time ago: I accidentally had attached the public subnet directly to the VPC. It should only be accessible through the NAT-Gateway. So yes, it was a subnet misconfiguration.CommentShareAWS-User-2258024answered a year ago0Right now about half of my calls to the API Gateway are successful. It seems that some condition during the start of the lambda container determines whether it works or times out. If I call the API Gateway in short intervals the result stays the same. If I wait for a longer time, allowing the lambda container to shut down, there is a 50:50 chance that it will work for the next call.The logs (with API Gateway logging turned on) show no clue of what is going on. Does anyone have an idea about what to look for?CommentShareAWS-User-2258024answered a year agoAWS-User-5225285 a year agoI'm currently getting my head around using API Gateway with lambda in VPC when I came across your comment. I found in the AWS knowledge centre some information on allowing internet access to lambda, and noticed it mentioned "An Amazon VPC-connected Lambda function randomly selects an associated subnet when making requests. All subnets that your function uses should have the same configuration to prevent random errors caused by Lambda using a misconfigured subnet." This sounds like it could have been your issue, and would explain the 50/50 issue (sometimes used one subnet, sometimes the other)Share0When I call a function in my lambda via API Gateway it simply times out. This is also reflected in the CloudWatch logs: They show a START event followed by an END stating that the task timed out. I created my VPC according to instructions here: https://digitalvarys.com/how-to-assign-a-static-ip-to-the-aws-lambda-function/Does that show a correct way to do it? And how do I assign a subnet to a lambda?CommentShareAWS-User-2258024answered a year ago0Ok, now I have it partially working: sometime it works, sometimes it doesn't (times out). How can this be?CommentShareAWS-User-2258024answered a year ago0I'm currently getting my head around using API Gateway with lambda in VPC when I came across your comment. I found in the AWS knowledge centre some information on allowing internet access to lambda, and noticed it mentioned "An Amazon VPC-connected Lambda function randomly selects an associated subnet when making requests. All subnets that your function uses should have the same configuration to prevent random errors caused by Lambda using a misconfigured subnet." This sounds like it could have been your issue, and would explain the 50/50 issue (sometimes used one subnet, sometimes the other.) Hope this info helps. Link: https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/CommentShareAWS-User-5225285answered a year ago"


Question: KNN search on opensearch using .Net

"I have an index with KNN filed in opensearch with this schema.schema = {"settings": {"index": {"knn": True,}},"mappings": {"properties": {"question": {"type": "text"},"ques_embedding": {"type": "knn_vector","dimension": 384,}}},}I need to perform a knn field search using .Net.Can you please guide me on how to perform that? All the examples I could find are searches on textual fields.FollowComment"

Answer: "0Hello,To perform knn field search using .Net, you can utilise the "OpenSearch .NET clients". I have provided a sample below:Firstly I have created an index with the below setting:PUT my-index{  "settings": {    "index.knn": true  },  "mappings": {    "properties": {      "my_vector1": {        "type": "knn_vector",        "dimension": 2      },      "my_vector2": {        "type": "knn_vector",        "dimension": 4      }    }  }}Then I have ingested some sample data using _bulk request as shown below:POST _bulk{ "index": { "_index": "my-index", "_id": "1" } }{ "my_vector1": [1.5, 2.5], "price": 12.2 }{ "index": { "_index": "my-index", "_id": "2" } }{ "my_vector1": [2.5, 3.5], "price": 7.1 }{ "index": { "_index": "my-index", "_id": "3" } }{ "my_vector1": [3.5, 4.5], "price": 12.9 }{ "index": { "_index": "my-index", "_id": "4" } }{ "my_vector1": [5.5, 6.5], "price": 1.2 }{ "index": { "_index": "my-index", "_id": "5" } }{ "my_vector1": [4.5, 5.5], "price": 3.7 }{ "index": { "_index": "my-index", "_id": "6" } }{ "my_vector2": [1.5, 5.5, 4.5, 6.4], "price": 10.3 }{ "index": { "_index": "my-index", "_id": "7" } }{ "my_vector2": [2.5, 3.5, 5.6, 6.7], "price": 5.5 }{ "index": { "_index": "my-index", "_id": "8" } }{ "my_vector2": [4.5, 5.5, 6.7, 3.7], "price": 4.4 }{ "index": { "_index": "my-index", "_id": "9" } }{ "my_vector2": [1.5, 5.5, 4.5, 6.4], "price": 8.9 }After this, I have used the below sample code to perform knn search. Kindly, modify the code based on your requirement: using OpenSearch.Net ;using OpenSearch.Client;using OpenSearch.Net .Auth.AwsSigV4;namespace vs;class Program{    public static void Main(string[] args)    {        // Create a client with custom settings        var uri = new Uri("https://search-xxxx.region.es.amazonaws.com ");        var connection = new AwsSigV4HttpConnection();        var settings = new ConnectionSettings(uri, connection);        var client = new OpenSearchLowLevelClient(settings);        Console.WriteLine("knn query search sample");        var searchResponse = client.Search <StringResponse>("my-index",            @"{                ""size"": 2,                ""query"": {                    ""knn"": {                        ""my_vector2"": {                            ""vector"": [2, 3, 5, 6],                            ""k"": 2                            }                        }                    }                }");        Console.WriteLine(searchResponse.Body);    }}Dependencies:Opensearch.Net NuGet: https://www.nuget.org/packages/OpenSearch.NetOpenSearch.Client NuGet: https://www.nuget.org/packages/OpenSearch.Client/OpenSearch.Net.Auth.AwsSigV4 NuGet: https://www.nuget.org/packages/OpenSearch.Net.Auth.AwsSigV4Reference:k-Nearest Neighbor (k-NN) search in Amazon OpenSearch Service - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/knn.htmlOpenSearch .NET clients - https://opensearch.org/docs/latest/clients/dot-net/OpenSearch .NET sample program - https://opensearch.org/docs/latest/clients/OpenSearch-dot-net/#sample-programCommentShareSUPPORT ENGINEERAdesh_Janswered a month ago"


Question: Can I run S3DistCp on EMR Serverless?

"Is there a way to utilize EMR Serverless to run S3DistCp? Looking at the base Docker images, I can see that the s3-dist-cp command is included in the Hive image. How can I submit a job run that runs it? Is this even supported - or planned to be supported in the future?ThanksFollowComment"

Answer: "0S3DistCp uses mapreduce framework to copy the data in distributed manner. In EMR Serverless, only Tez as execution engine is supported.Refhttps://docs.aws.amazon.com/emr/latest/ReleaseGuide/UsingEMR_s3distcp.htmlCommentShareShaileshanswered 4 months ago"


Question: Trouble with uploads

"I am a new user and really don't have enough experience to trouble shoot minor issues.  I recently created a web-page, which I have full access to make alterations.  I made two changes, uploaded those changes to my storage bucket, checked to see if they uploaded properly.  Everything lined up, but when I checked the web-page, those changes did not take.  It has now been four days, and I have cleared my cache, but still there is nothing.  Can you help me to understand what is wrong?FollowCommentalatech EXPERT3 months agoCan you elaborate if is a s3 website or are you using more things such as CloudFront?ShareSanjeeb 3 months agoIf I understood correctly, you hosted a static website using AWS S3. If yes, please download the content and see whether the changes are their or not. If not, you can try to delete and make S3 empty and then upload your new website content. Also if it is not working, set up an error page and see whether it is pointing to error page or not. Also please check whether you set up your static website properly or not. See the link - https://www.youtube.com/watch?v=H83xYoC3z4AShareTravis A Williams 3 months agoSanjeeb It is a s3.  My download function is not available.  It may be an accessibility issue.  I have edited out all the restrictions, but I can't perform this simple task.  I deleted a whole file, but the page remained the same.  My site is set as an index.html, and my error is also set as an index.html.ShareAlberto Fernandes 3 months agoYou is using SPA (angular or react)?Share"


Question: Bastion host service from AWS that supports multi cloud environments.

A customer is looking for bastion host service from AWS that supports multi cloud environments.Please add your opinions on below:Normal EC2 can be provisioned as bastion host.Amazon EC2 Systems Manager - is there any blocker for multi-cloud environment?FollowComment

Answer: "0Accepted AnswerI would reframe the question about the function of the bastion host. If the goal is to provide secure, access-controlled, audited accesses to remote hosts, then the use of Session Manager (or other features of Systems Manager) can accomplish those goals.One of the primary benefits of using Session Manager is that it obviates the need to open up SSH or RDP, which reduces an environment's attack surface and simplifies the infrastructure by eliminating a bastion setup.See this blog for details (although it pre-dates Session Manager): https://aws.amazon.com/blogs/mt/replacing-a-bastion-host-with-amazon-ec2-systems-manager/.CommentShareAWS-User-0560630answered 3 years ago"


Question: "Permission denied (publickey,gssapi-keyex,gssapi-with-mic)- Windows 11"

"Hi all,I am trying to connect to ec2 machine (default linux) while on Windows 11 as my work machine.I am receiving the following on my CLI:C:>ssh -i C:\AWS\pizza-key.pem ec2-user@44.208.4.50Load key "C:\AWS\pizza-key.pem": Permission deniedec2-user@44.208.4.50: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).Please suggest.ThanksFollowCommentrePost-User-6570472 9 months agoI'm seeing this same issue with some instances. The instance typically works fine for a while and oftentimes never has the issue, but then I get this error. I'm not sure what is happening to my instances - it seems like a bug. A couple weeks ago it seemed like most of my instances had this issue after a couple days of ssh ing into them without problems.SharerePost-User-5600275 2 months agoI am trying to connect to a CentOS EC2 instance from my WIN 11 machine and am facing the same issue you have reported.Have been trying for the last 2 days but no luck. kindly share your inputs if you were able to resolve it.$ ssh -i "CentOSStream9.pem" centos@ec2-xx-xx-xx-xxx.us-east-2.compute.amazonaws.comcentos@ec2-xx-xx-xx-xxx.us-east-2.compute.amazonaws.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).Share"


Question: Using Timestream as a dataset for QuickSight (Domain: ap-southeast-2)

"Hi, I'm trying to use my Timestream DB as a dataset for QuickSight BI. Both of them are in the ap-southeast-2 domain, but Timestream is not on the list. (Please find the attached screenshot).The only way I can make this work is by changing the domain to us-east-1,2. Is there any solution to this??FollowComment"

Answer: "0HI,As of till date , Quicksight doesnt have a direct connection with Timestream DB. However you can leverage Athena(Serverless SQL Service) in between to Integrate Timestream  and Quicksight.Once you go to Athena you can create a connector(Via Lambda) to your data source Timestream DB . Then you can make Athena as the source for your Quicksight for Dashboards and reporting. This Feature of Athena is called as federated queryRef Flow Architecture:-Timestream ---> Athena(Federated Query via Lambda Connectors) ----> QuicksightFew useful links for your reference :-Lambda Connector Template for Timestream in Athena (Github Source) - https://github.com/awslabs/aws-athena-query-federation (Go to athena-timestream)Documentation - https://docs.aws.amazon.com/athena/latest/ug/connectors-timestream.htmlReference Blog - https://aws.amazon.com/blogs/big-data/query-any-data-source-with-amazon-athenas-new-federated-query/CommentSharerePost-User-8827802answered 8 months ago0Hello User-1450233,Just to clarify, Timestream as a direct datasource in QuickSight is not yet available in ap-southeast-2 and ap-northeast-1 regions. This is something we are working on.However, in all other regions where Timestream is available (refer pricing page), you can use Timestream as a direct datasource in QuickSight.While we are working on this, you can consider alternative approaches like the one suggested above.Stay tuned for updates.RajuCommentShareEXPERTRajuAWSanswered 8 months ago"


Question: Strategies for RDS MySQL migration to Aurora

"RDS has automatically upgraded us to MySQL 8.0.30, and Aurora currently only supports migration from 8.0.28.  AWS Support says they can't provide a timeline for 8.0.30 support, and recommends either using DMS (which seems very lacking for this type of migration) or dumping the database and recreating it (difficult for a database in the hundreds of gigabytes range.)Does anyone have any good strategies for migrating a database of this size to Aurora?The answer here -- https://repost.aws/questions/QUM2j4BPEQS5CBHVu4QbOLCA/migrate-rds-my-sql-8-0-28-to-aurora-my-sql -- doesn't work, since you can't create an Aurora read replica for unsupported MySQL versions...FollowComment"

Answer: "0Hello,Unfortunately, there is NO in-place method available for migrating from RDS MySQL to Aurora for version 8.0.25 and above. Hence, the work-around available currently is to utilize DMS[1] or performing a logical dump[2] as suggested by the AWS support. Alternatively, you can consider waiting for the official support for migrating from MySQL to aurora, you can check the site[3] for any news about latest releases.[1] https://aws.amazon.com/blogs/database/migrating-from-mysql-to-amazon-aurora-using-aws-sct-and-aws-dms/[2] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Exporting.NonRDSRepl.html#MySQL.Procedural.Exporting.NonRDSRepl.PrepareSource[3] https://aws.amazon.com/new/CommentShareSUPPORT ENGINEERSanketh Saliananswered 6 months ago"


Question: DX connect traffic via the non associated region caveats

"A customer is looking to establish a connectivity between their onPrem DC in Oslo, Norway to our Stockholm region.Currently the only DX provider mentioned here is DigiPlex Ulven, Oslo, Norway and that has associated region as Frankfurt. They prefer to use a DX provider in Norway. Now the question is:Will using this provider add extra latency via Direct connect gateway when talking to workload in Stockholm region?Any other approach/options that can be considered here ?FollowComment"

Answer: "0Accepted Answeryou can check out the FAQ hereQ. If I use Direct Connect gateway, does my traffic to the desired AWSRegion go via the associated home AWS Region?No. When using Direct Connect gateway, your traffic will take theshortest path from your Direct Connect location to the destination AWSRegion and vice versa regardless of the associated home AWS Region ofthe Direct Connect location that you are connected at.Note in direct connect resiliency recommendations we do recommend for them to be location redundant (in two direct connect locations).As per alternatives, they could use VPN over the internet, but if they are concerned for latency this probably doesn't fit, or use a partner to backhaul from their location to another Direct Connect location.CommentShareEXPERTScottanswered 3 years ago"


Question: CDK for ECS - Subnet and Architecture type

"HI AllI have 2 questions on how to use CDK to deploy ECS using python.How do we set subnet on the cluster via CDK? The VPC is straightforward using the vpc parameter, however i could not find the param for subnet.How do we set the architecture(x64 or ARM), using the CDK ?  While creating the task I need to set the architecture to a specific value as I am getting the following error: standard_init_linux.go:228: exec user process caused: exec format errorThank you for your help.RegardsFollowComment"

Answer: "1Accepted AnswerYou can specify specific subnets for your EC2 container instances and/or your AWSVPC network configuration (if you are using Fargate or AWSVPC networking for your tasks/services).For EC2 instances, the options you can pass to add_capacity (for EC2) are here - you want the vpc_subnets property.For task networking, you can specify VPC subnets by specifying the vpc_subnets property to your Ec2Service or FargateService declarations.If you want to run your task on Graviton2 (arm64 architecture), you don't need to declare the architecture of an ECS Task except on Fargate. At this writing, CDK does not yet have Level 2 construct support for architecture configurations in Task Definitions. You would have to use an Escape Hatch to modify the underlying L1 construct (CfnTaskDefinition in particular) to set the runtime_platform property.If you're running your task on an EC2 instance, you just need to make sure that the architecture of your EC2 instance matches the architecture of your container image - or, better still, create a multi-architecture container image.CommentShareEXPERTMichael_Fanswered a year agotkansara a year agoHI MichaelThank you for the response."You don't need to declare the architecture of an ECS Task except on Fargate. "Regarding the above mentioned line, what I understand is that instead of using the  ecs.FargateTaskDefinition, I should be using the ecs.CfnTaskDefinition.  Is that what I understand  ?Here is what I am currently using:task_definition = _ecs.FargateTaskDefinition(self, "TaskDefinition",cpu=2048,memory_limit_mib=4096,execution_role=ecs_role,task_role = ecs_role,)ThanksThanksShareMichael_F EXPERTa year agoYou can still use ecs.FargateTaskDefinition, but you'll need to use the escape hatch to get to the underlying ecs.CfnTaskDefinition and add the property. The documentation I linked above explains how. See "Modifying the AWS CloudFormation resource behind AWS constructs".Share1Hi Michael,Thanks a bunch for your help. I was able to resolve this.For everyone else:runtime_platform_property = _ecs.CfnTaskDefinition.RuntimePlatformProperty(            cpu_architecture="ARM64",            operating_system_family="LINUX"        )cfn_task_defn = task_definition.node.default_childcfn_task_defn.runtime_platform = runtime_platform_propertyThank you.CommentSharetkansaraanswered a year ago"


Question: Problems with 'LOAD DATA LOCAL INFILE' migrating to RDS

"We host a customers database on an ubuntu server running mysqld  Ver 10.1.33-MariaDB-1~xenial we are trying to get migrated to RDS. The customer has a job to update their database and the use the load local file parameter. This does not seem to be working on RDS. We have been researching and trying different options and now looking to see if the community can help. Below is the command that is throwing the error: EXEC('LOAD DATA LOCAL INFILE ''' + @DataFile1 + ''' INTO TABLE sh_phone_order FIELDS TERMINATED BY ''\t'' ENCLOSED BY ''^'' LINES TERMINATED BY ''\r\n'';') AT MYSQL_NEW_USER_MAGENTO_STAGINGThis is the error they receive:Executed as user: CLAWNET\SQL2agent. Could not execute statement on remote server 'MYSQL_NEW_MAGENTO_STAGING'. [SQLSTATE 42000] (Error 7215)  OLE DB provider "MSDASQL" for linked server "MYSQL_NEW_MAGENTO_STAGING" returned message "[MySQL][ODBC 5.3(a) Driver][mysqld-5.6.10]The used command is not allowed with this MySQL version". [SQLSTATE 01000] (Error 7412)  Could not execute statement on remote server 'MYSQL_NEW_MAGENTO_STAGING'. [SQLSTATE 42000] (Error 7215)  OLE DB provider "MSDASQL" for linked server "MYSQL_NEW_MAGENTO_STAGING" returned message "[MySQL][ODBC 5.3(a) Driver][mysqld-5.6.10]The used command is not allowed with this MySQL version". [SQLSTATE 01000] (Error 7412).  The step failed.The RDS server has localfile set to 1. We are not sure what else to check to get this working.Does anyone know what we might need to change to allow this?Edited by: tonywebscale on Apr 21, 2020 12:18 PMFollowComment"

Answer: "0I had a little trouble understanding how this example ever worked, but I think I understand after some research. If I read it correctly, there is a Linked Server from a Microsoft SQL Server to a MySQL Server. You then tell Microsoft SQL Server to pass the string "Load Data Local Infile" to the MySQL Server and execute it local to the MySQL Server. But the entire point of Load Data Local Infile is to read data from the client and send it to the server, so it can't be executed on the MySQL Server directly. BUT, a little searching around suggests that the ODBC driver can indeed intercept and execute this command locally.Sooo, I think the problem is that you need to be setting ENABLE_LOCAL_INFILE = 1 on the client side of the ODBC connection from SQL Server to the RDS MySQL database. You can probably just add this to the connect string. See https://stackoverflow.com/questions/55058843/mysql-load-data-local-infile-not-working-after-driver-update-from-5-to-8 for more discussion.Another thing to consider is that the ODBC driver used in the Microsoft SQL Server installation is a version that doesn't support ENABLE_LOCAL_INFILE. But before tackling if there is a way to update the driver, make sure the connection string is trying to enable local infile.CommentShareHalTempanswered 3 years ago"


Question: S3Transfer class ProgressPercentage writes frequently

"I'm uploading a large file and I have it configured to use 1gb parts in the upload.  I just recently added the ProgressPercentage class to S3Transfer and it's writing much more frequently than I expect.  I would expect it to write once per part.  Why is there 262k between values of what it's uploading?  I'm using the standard class posted here, but I only modified how it writes.EDIT: I modified my script to only print when it's an even percentage, so it will print at 0.00, 1.00, etc.  Still, why is it doing this?https://boto3.amazonaws.com/v1/documentation/api/latest/_modules/boto3/s3/transfer.html2022-11-21 07:14:42,774 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3580887040 / 92093203968.0 (3.89%)2022-11-21 07:14:42,852 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3581149184 / 92093203968.0 (3.89%)2022-11-21 07:14:42,930 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3581411328 / 92093203968.0 (3.89%)2022-11-21 07:14:42,972 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3581673472 / 92093203968.0 (3.89%)2022-11-21 07:14:43,179 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3581935616 / 92093203968.0 (3.89%)2022-11-21 07:14:43,240 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3582197760 / 92093203968.0 (3.89%)2022-11-21 07:14:43,334 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3582459904 / 92093203968.0 (3.89%)2022-11-21 07:14:43,428 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3582722048 / 92093203968.0 (3.89%)2022-11-21 07:14:43,459 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3582984192 / 92093203968.0 (3.89%)2022-11-21 07:14:43,475 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3583246336 / 92093203968.0 (3.89%)2022-11-21 07:14:43,662 - aws_pc_backup - [CRITICAL] - Progress for C:\Temp_Download\C_VOL-b031.spf  3583508480 / 92093203968.0 (3.89%)FollowComment"

Answer: "0Why wasn't this post shared with the Storage topic group?CommentSharejschwar313answered 6 months ago0I see what's happening.  The S3TransferConfig has a parm called io_chunksize that's set to 256 kb.  I guess that would be configurable.  I'll try that.  I might leave it the way it is, though.  Here's where I found that:https://boto3.amazonaws.com/v1/documentation/api/latest/_modules/boto3/s3/transfer.htmlCommentSharejschwar313answered 6 months ago"


Question: Can you edit a Contact Lens Rule name after you've created it?

I have made some Contact Lens rules and am trying to edit them after. I am able to edit the conditions of the Rules but not the name of the Rule. How would I do that?FollowComment

Answer: "0Hello,You can not currently edit the rule name in created rules for Contact Lens. You should create a new one changing the rule name while configuring the new one.On the other hand, contact your AWS account team if you are really interested on this feature to further explain your case, we appreciate your feedback.Regards.CommentShareJoan_Banswered a year ago"


Question: The network interface can't be deleted. Reason: Network interface is currently in use.

"Hello everyone, I have been working on a personal project and encountered a strange issue today. I am unable to clean up my resources properly. After finishing my work, I deleted EC2 instances, but I'm facing difficulty in deleting Security Groups, non-default VPCs, Subnets, and Network ACLs. Most of them indicate that I have associated network interfaces, and I've tried to delete them, but they can't be removed. I have gone through the AWS documentation to delete the interfaces using the UI, but it shows that they are still in use. I have also released elastic IPs, but to no avail. I would appreciate any help or suggestions you may have. Thank you in advance!FollowComment"

Answer: "01. Log in to the AWS Management Console and navigate to the EC2 service.2. From the EC2 console, select "Network Interfaces" from the sidebar menu.3. Locate the network interface you want to delete and check the "Attachment Information" column to see if it's attached to an EC2 instance.4. If the network interface is attached to an EC2 instance, select the network interface from the list, choose "Actions", and then "Detach Network Interface".5. Before detaching the network interface, make sure to stop any processes or services running on the EC2 instance that might be using the interface. Otherwise, the detachment process will fail.6. Once the network interface is detached, try to delete it again. If the deletion still fails, you may need to contact AWS support for further assistance.CommentShareMauliksinhanswered 2 months agolou 2 months agoThanks for your response, but the issue remains unresolved.Share0Hi,You can also try the script in this article: https://repost.aws/knowledge-center/troubleshoot-dependency-error-delete-vpc which can help identify dependencies with the ENI.Hope it helps ;)CommentShareEXPERTalatechanswered 2 months agolou 2 months agoThanks for your prompt response . I tired it, but I couldn't resolve the issue.Share"


Question: Creating ecs service from ecs-cli

"Hi,I would like to create a new ecs service instance from ecs-cli, with the parameters specified below:Launch type: FARGATEService name: ecs-service-01Number of tasks: 1Cluster VPC: vpc-0000000Subnets: subnet-0000000Security Groups: sg-00000000 (default)Auto-assign public IP: ENABLEDI had a look at options available from the URL below, but it seems like the command only set launch-type and VPC from the above specified options.https://docs.aws.amazon.com/en_us/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-service-create.html#cmd-ecs-cli-compose-service-create-optionsCould someone show me if it is possible to set all of the 7 options shown above from ecs-cli command?FollowComment"

Answer: "0I found the solution here:https://docs.aws.amazon.com/en_us/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-ecsparams.htmlThe necessary params can be set using ecs-params.yml, with the option below:-------snippet-----------awsvpc_configuration:subnets:- "subnet ID 1"- "subnet ID 2"security_groups:- "security group ID"assign_public_ip: ENABLEDCommentSharemonocomanswered 2 years ago"


Question: eb platform create fails with Ruby SDK deprecated error

"When trying to create a custom ElasticBeanstalk platform that uses Python3.10.5, I keep running across this error:[2022-07-01T05:50:06.466Z] INFO  [5419]  - [CMD-PackerBuild/PackerBuild/PackerBuildHook/build.rb] : Activity execution failed, because: Version 2 of the Ruby SDK will enter maintenance mode as of November 20, 2020. To continue receiving service updates and new features, please upgrade to Version 3. More information can be found here: https://aws.amazon.com/blogs/developer/deprecation-schedule-for-aws-sdk-for-ruby-v2/  'packer build' failed, the build log has been saved to '/var/log/packer-builder/Python3.10_Ubuntu:1.0.8-builder.log' (ElasticBeanstalk::ExternalInvocationError)caused by: Version 2 of the Ruby SDK will enter maintenance mode as of November 20, 2020. To continue receiving service updates and new features, please upgrade to Version 3. More information can be found here: https://aws.amazon.com/blogs/developer/deprecation-schedule-for-aws-sdk-for-ruby-v2/  'packer build' failed, the build log has been saved to '/var/log/packer-builder/Python3.10_Ubuntu:1.0.8-builder.log' (Executor::NonZeroExitStatus)I'm not sure how to get around it, as none of my actual code for this uses ruby at all.I have tried to SSH into the packer build box and run gem install aws-sdk to get the latest version, however the problem above  still persists.I'm really unsure of what to do at this point. Any advice?FollowComment"

Answer: "0Hello,The simplest path to upgrade to version 3 from 2 is;gem 'aws-sdk', '~> 3'If your gem-file uses the bundlers aws-sdk-core or aws-ask-resources, you will need to update these bundlers in your gem file, as well as any ruby require statements that call them:Gem file#gem 'aws-sdk-core', '~> 2'#gem 'aws-sdk-resources', '~> 2'gem 'aws-sdk', '~> 3'Requires#require 'aws-sdk-core'require 'aws-sdk'Please see Upgrading from Version 2 to Version 3 of the AWS SDK for Ruby for additional informationCommentShareUgochukwu_Ianswered a year agojbrace a year agoHi, as I have mentioned in my post, I did upgrade to the latest versions, however the files that are complaining about it are endemic to the default packer AMI provided by Amazon (AMI ID: ami-0428124429e0d6ae2).This means that every example listed on the https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/custom-platforms.html help page does not work. The AMI in question is provided directly by amazon, and is automatically selected when running eb platform create.The default version of ruby on that instance is also insufficient to run the latest version of the aws-sdk.Share0Hello,Thanks for the feedback. I have reproduced this issue with the same AMI "ami-0428124429e0d6ae2" in us-east-1 and got the same error. This points to issues with the AMI.Activity execution failed, because: Version 2 of the Ruby SDK will enter maintenance mode as of November 20, 2020. To continue receiving service updates and new features, please upgrade to Version 3. More information can be found here: https://aws.amazon.com/blogs/developer/deprecation-schedule-for-aws-sdk-for-ruby-v2/  'packer build' failed, the build log has been saved to '/var/log/packer-builder/customplatform:1.0.0-builder.log' (ElasticBeanstalk::ExternalInvocationError)caused by: Version 2 of the Ruby SDK will enter maintenance mode as of November 20, 2020. To continue receiving service updates and new features, please upgrade to Version 3. More information can be found here: https://aws.amazon.com/blogs/developer/deprecation-schedule-for-aws-sdk-for-ruby-v2/  'packer build' failed, the build log has been saved to '/var/log/packer-builder/customplatform:1.0.0-builder.log' (Executor::NonZeroExitStatus)In order to troubleshoot this further, please create a support case as we require information that is not public. This would enable us reach out to the Elastic Beanstalk service team.CommentShareUgochukwu_Ianswered a year ago"


Question: Chime recall function

"In the workplace, typing and sending incorrect messages can make the other person feel that you are very unprofessional. I hope to have a recall message function to ensure effective communication. It also can be convenient for the sender to reorganize or make new decisions.FollowComment"


Question: Host website on AWS using existing domain name registered at Register.com

"I have an existing domain name registered at Register.comI would like to experiment with hosting a website on AWS using this domain name.Typically when changing hosts I would change the DNS settings at Register.com to use new host DNS.I'm not sure how to accomplish that here.   I have skimmed tutorial for creating a static website but that has domain registered in Route 53....I see I can create a Hosted Zone in Route 53, but I'm not clear if I use the dns entries that are generated there and populate them on Register.com or....Any input or links on how to have AWS hosted website use my existing domain name on Register.com would be appreciated.FollowComment"

Answer: "0I'm going to assume that your domain is hosted by register.com rather than Route 53.For the most part, who hosts the domain (that is, who is responsible for the servers that respond to the DNS queries) isn't particularly important and doesn't have to be related to where your website is being hosted. So if your DNS is currently hosted at register.com you can use that without using Route 53.You don't say how you're hosting your website in AWS but in general, it's going to be as easy as creating an A record in your DNS and pointing it at the IP address of the EC2 instance that is hosting your website; or creating a CNAME record and pointing that to the name of the load balancer that you're using. The instructions for Route 53 will be much the same.The edge case here is if you want to use the "apex record" in your domain. Say you own example.com - it's straightforward to create a record for www.example.com and have that point to an EC2 instance or load balancer as above. But if you want to use example.com (the domain itself) as a record then that's a little tricky and is generally easier if your domain is hosted in Route 53.CommentShareEXPERTBrettski-AWSanswered 4 months ago"


Question: Why is there this error message on my domain?

"hi I can't transfer this same domain to a webflow site. How do I go about for this. The links I found online leads me to purchases, which I don't see on my dashboard. So I'll have to remove this wordpress site of https://pytch.media and direct them to https://pytch.webflow.io/FollowComment"

Answer: "0Hello,Can you further elaborate on what you are doing? Tried accessing the site and I'm face with:Fastly error: unknown domain: pytch.media. Please check that this domain has been added to a service.Details: cache-qpg1250-QPGTried doing a curl to https://pytch.media/ and I'm seeing that the certificate is not matching:* Server certificate:*  subject: CN=t.ssl-351-default.ssl.fastly.netSeems it is seeing the Fastly certificate.CommentShareTimothy_Manswered a year agorePost-User-4215423 a year agoHi the error message has disappered now. However, now I can't transfer this same domain to a webflow site. How do I go about for this. The links I found online leads me to purchases, which I don't see on my dashboard. So I'll have to remove this wordpress site of https://pytch.media and direct them to https://pytch.webflow.io/Share0I'm seeing a slightly different, but still certificate related issue.Specifically that this site is using a Let's Encrypt certificate, which has expired as of Tuesday, June 28, 2022 at 2:16:36 PM Central Daylight Time.Updating the certificate should resolve the issues here.CommentShareGreg_Hanswered a year ago0Hi the error message has disappered now. However, now I can't transfer this same domain to a webflow site. How do I go about for this. The links I found online leads me to purchases, which I don't see on my dashboard. So I'll have to remove this wordpress site of https://pytch.media and direct them to https://pytch.webflow.io/CommentSharerePost-User-4215423answered a year ago"


Question: [Greengrass][IDT][Raspi] Deployment did not reach COMPLETED

"Hello,I'm trying to run the IDT test suite on my Raspi 4B with the Raspi OS and a Greengass Core Software v2.7.0 running on it. I launch the tests from a Windows PC with a IDT v4.5.8, connected with Putty through SSH.My Greengrass Core Software is PREINSTALLED. I also run the test HSM with PKCS11 provider, as I generated key and certificate with the TPM connected to my Raspi.I do not manage to pass the tests that involve deployments: I always get Deployment did not reach COMPLETED even for the Empty Deployment used for cleaning up.Here is an example for a part of the suite that tests a local deployment.[Logs from test_manager.log]time="2022-09-27T12:57:37+02:00" level=info msg=12:57:37.231 [localdeployment] [idt-bf5893219b34898de77c] [INFO] com.aws.greengrass.testing.resources.AbstractAWSResourceLifecycle - Created GreengrassDeployment in GreengrassV2Lifecycletime="2022-09-27T12:57:37+02:00" level=info msg=12:57:37.233 [localdeployment] [idt-bf5893219b34898de77c] [INFO] com.aws.greengrass.testing.features.DeploymentSteps - Created Greengrass deployment: 3c3703c6-feb1-404a-86fe-317282a02b15time="2022-09-27T12:57:37+02:00" level=info msg=12:57:37.233 [localdeployment] [idt-bf5893219b34898de77c] [INFO] greengrass/features/localdeployment.feature - line 12: 'the Greengrass deployment is COMPLETED on the device after 180 seconds'time="2022-09-27T13:00:37+02:00" level=info msg=13:00:37.455 [localdeployment] [idt-bf5893219b34898de77c] [ERROR] greengrass/features/localdeployment.feature - Failed at step: 'the Greengrass deployment is COMPLETED on the device after 180 seconds'time="2022-09-27T13:00:37+02:00" level=info msg=java.lang.IllegalStateException: Deployment idt-bf5893219b34898de77c-gg-deployment did not reach COMPLETEDtime="2022-09-27T13:00:37+02:00" level=info msg=at com.aws.greengrass.testing.features.DeploymentSteps.deploymentSucceeds(DeploymentSteps.java:272)time="2022-09-27T13:00:37+02:00" level=info msg=at ?.the Greengrass deployment is COMPLETED on the device after 180 seconds(classpath:greengrass/features/localdeployment.feature:12)13:00:37.463 [localdeployment] [idt-bf5893219b34898de77c] [INFO] greengrass/features/localdeployment.feature - line 13: 'I verify greengrass-cli is available in greengrass root'time="2022-09-27T13:00:37+02:00" level=info msg=13:00:37.463 [localdeployment] [idt-bf5893219b34898de77c] [INFO] greengrass/features/localdeployment.feature - line 14: 'I create a local deployment with components'time="2022-09-27T13:00:37+02:00" level=info msg=13:00:37.463 [localdeployment] [idt-bf5893219b34898de77c] [INFO] greengrass/features/localdeployment.feature - line 16: 'the local Greengrass deployment is SUCCEEDED on the device after 120 seconds'time="2022-09-27T13:00:37+02:00" level=info msg=13:00:37.464 [localdeployment] [idt-bf5893219b34898de77c] [INFO] greengrass/features/localdeployment.feature - line 17: 'the aws.greengrass.LocalHelloWorld log on the device contains the line "Hello World!!" within 20 seconds'time="2022-09-27T13:00:37+02:00" level=info msg=13:00:37.959 [localdeployment] [idt-bf5893219b34898de77c] [INFO] com.aws.greengrass.testing.features.DeploymentSteps - Cleaning up component through an empty deploymenttime="2022-09-27T13:00:38+02:00" level=info msg=13:00:38.394 [localdeployment] [idt-bf5893219b34898de77c] [INFO] com.aws.greengrass.testing.resources.AbstractAWSResourceLifecycle - Created GreengrassDeployment in GreengrassV2Lifecycletime="2022-09-27T13:01:12+02:00" level=info msg=13:01:12.903 [localdeployment] [idt-bf5893219b34898de77c] [INFO] com.aws.greengrass.testing.features.DeploymentSteps - Cleaning up component through an empty deploymenttime="2022-09-27T13:01:13+02:00" level=info msg=13:01:13.269 [localdeployment] [idt-bf5893219b34898de77c] [INFO] com.aws.greengrass.testing.resources.AbstractAWSResourceLifecycle - Created GreengrassDeployment in GreengrassV2Lifecycletime="2022-09-27T13:02:13+02:00" level=info msg=13:02:13.456 [localdeployment] [idt-bf5893219b34898de77c] [WARN] com.aws.greengrass.testing.features.DeploymentSteps - Empty deployment did not reach COMPLETED[Logs from greengrass.log (file from the core device tested)]2022-09-27T10:57:44.221Z [INFO] (Thread-3) com.aws.greengrass.deployment.ShadowDeploymentListener: Added shadow deployment job. {ID=arn:aws:greengrass:eu-west-2:548876126010:configuration:thing/MyGreengrassCoreV2:17}2022-09-27T10:57:56.652Z [INFO] (pool-2-thread-12) com.aws.greengrass.deployment.DeploymentService: Received deployment in the queue. {deploymentId=arn:aws:greengrass:eu-west-2:548876126010:configuration:thing/MyGreengrassCoreV2:17, DeploymentType=SHADOW, serviceName=DeploymentService, currentState=RUNNING}2022-09-27T10:57:56.653Z [INFO] (pool-2-thread-12) com.aws.greengrass.deployment.DeploymentService: Received deployment document in queue. {document={"deploymentId":"3c3703c6-feb1-404a-86fe-317282a02b15","schemaDate":"2021-05-17","deploymentName":"idt-bf5893219b34898de77c-gg-deployment","configurationArn":"arn:aws:greengrass:eu-west-2:548876126010:configuration:thing/MyGreengrassCoreV2:17","creationTimestamp":1664276257193,"components":{"aws.greengrass.Cli":{"version":"2.7.0"},"aws.greengrass.Nucleus":{"version":"2.7.0"}},"failureHandlingPolicy":"ROLLBACK","requiredCapabilities":[],"componentUpdatePolicy":{"timeout":60,"action":"NOTIFY_COMPONENTS"},"configurationValidationPolicy":{"timeout":60}}, serviceName=DeploymentService, currentState=RUNNING}[...]2022-09-27T10:58:07.731Z [INFO] (pool-2-thread-5) com.aws.greengrass.componentmanager.ComponentManager: prepare-package-start. {packageIdentifier=aws.greengrass.Cli-v2.7.0}2022-09-27T10:58:07.732Z [DEBUG] (pool-2-thread-5) com.aws.greengrass.componentmanager.ComponentStore: finding-package-recipe. {packageRecipePath=/greengrass/v2/packages/recipes/rQVjcR-rX_XGFHg0WYKAnptIez3HKwtctL_2BKKZegM@2.7.0.recipe.yaml}2022-09-27T10:58:07.778Z [DEBUG] (pool-2-thread-5) com.aws.greengrass.componentmanager.ComponentManager: downloading-package-artifacts. {packageIdentifier=aws.greengrass.Cli-v2.7.0}2022-09-27T10:58:07.817Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain: Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.core.internal.interceptor.HttpChecksumRequiredInterceptor@14e72a8, software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor@5198ce, software.amazon.awssdk.awscore.eventstream.EventStreamInitialRequestInterceptor@1f24246]. {}2022-09-27T10:58:07.819Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.request: Sending Request: DefaultSdkHttpFullRequest(httpMethod=GET, protocol=https, host=greengrass-ats.iot.eu-west-2.amazonaws.com, port=8443, encodedPath=/greengrass/v2/components/arn%3Aaws%3Agreengrass%3Aeu-west-2%3Aaws%3Acomponents%3Aaws.greengrass.Cli%3Aversions%3A2.7.0/artifacts/JdZfpOZVO9z8GzOhHm01lU3WQaCWO2RkfzJ_k2LT51I%3D/aws.greengrass.cli.client.zip, headers=[amz-sdk-invocation-id, User-Agent], queryParameters=[]). {}2022-09-27T10:58:07.823Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: Connecting to greengrass-ats.iot.eu-west-2.amazonaws.com/18.133.211.72:8443. {}2022-09-27T10:58:07.873Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: socket.getSupportedProtocols(): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1, SSLv3, SSLv2Hello], socket.getEnabledProtocols(): [TLSv1.3, TLSv1.2]. {}2022-09-27T10:58:07.873Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: TLS protocol enabled for SSL handshake: [TLSv1.2, TLSv1.1, TLSv1, TLSv1.3]. {}2022-09-27T10:58:09.355Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.net.SdkSslSocket: created: greengrass-ats.iot.eu-west-2.amazonaws.com/18.133.211.72:8443. {}2022-09-27T10:58:09.573Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.request: Received successful response: 200. {}2022-09-27T10:58:09.574Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.requestId: x-amzn-RequestId : 622cc548-944a-e387-164c-e6c9889efe42. {}2022-09-27T10:58:09.575Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.requestId: x-amz-id-2 : not available. {}2022-09-27T10:58:10.033Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: Connecting to evergreencomponentmanageme-artifactbucket7410c9ef-1t7hs9n3e7a5l.s3.eu-west-2.amazonaws.com/52.95.148.90:443. {}2022-09-27T10:58:10.156Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: socket.getSupportedProtocols(): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1, SSLv3, SSLv2Hello], socket.getEnabledProtocols(): [TLSv1.3, TLSv1.2]. {}2022-09-27T10:58:10.156Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: TLS protocol enabled for SSL handshake: [TLSv1.2, TLSv1.1, TLSv1, TLSv1.3]. {}2022-09-27T10:58:10.685Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.net.SdkSslSocket: created: evergreencomponentmanageme-artifactbucket7410c9ef-1t7hs9n3e7a5l.s3.eu-west-2.amazonaws.com/52.95.148.90:443. {}2022-09-27T10:58:10.903Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.net.SdkSslSocket: closing evergreencomponentmanageme-artifactbucket7410c9ef-1t7hs9n3e7a5l.s3.eu-west-2.amazonaws.com/52.95.148.90:443. {}2022-09-27T10:58:10.932Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain: Creating an interceptor chain that will apply interceptors in the following order: [software.amazon.awssdk.core.internal.interceptor.HttpChecksumRequiredInterceptor@14e72a8, software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor@5198ce, software.amazon.awssdk.awscore.eventstream.EventStreamInitialRequestInterceptor@1f24246]. {}2022-09-27T10:58:10.934Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.request: Sending Request: DefaultSdkHttpFullRequest(httpMethod=GET, protocol=https, host=greengrass-ats.iot.eu-west-2.amazonaws.com, port=8443, encodedPath=/greengrass/v2/components/arn%3Aaws%3Agreengrass%3Aeu-west-2%3Aaws%3Acomponents%3Aaws.greengrass.Cli%3Aversions%3A2.7.0/artifacts/JdZfpOZVO9z8GzOhHm01lU3WQaCWO2RkfzJ_k2LT51I%3D/aws.greengrass.cli.client.zip, headers=[amz-sdk-invocation-id, User-Agent], queryParameters=[]). {}2022-09-27T10:58:10.937Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: Connecting to greengrass-ats.iot.eu-west-2.amazonaws.com/18.133.211.72:8443. {}2022-09-27T10:58:11.880Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: socket.getSupportedProtocols(): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1, SSLv3, SSLv2Hello], socket.getEnabledProtocols(): [TLSv1.3, TLSv1.2]. {}2022-09-27T10:58:11.881Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: TLS protocol enabled for SSL handshake: [TLSv1.2, TLSv1.1, TLSv1, TLSv1.3]. {}2022-09-27T10:58:13.389Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.net.SdkSslSocket: created: greengrass-ats.iot.eu-west-2.amazonaws.com/18.133.211.72:8443. {}2022-09-27T10:58:13.626Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.request: Received successful response: 200. {}2022-09-27T10:58:13.627Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.requestId: x-amzn-RequestId : 49c34118-9025-6ef7-4def-489354e1c4e3. {}2022-09-27T10:58:13.628Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.requestId: x-amz-id-2 : not available. {}2022-09-27T10:58:13.706Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: Connecting to evergreencomponentmanageme-artifactbucket7410c9ef-1t7hs9n3e7a5l.s3.eu-west-2.amazonaws.com/52.95.148.90:443. {}2022-09-27T10:58:13.785Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: socket.getSupportedProtocols(): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1, SSLv3, SSLv2Hello], socket.getEnabledProtocols(): [TLSv1.3, TLSv1.2]. {}2022-09-27T10:58:13.786Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory: TLS protocol enabled for SSL handshake: [TLSv1.2, TLSv1.1, TLSv1, TLSv1.3]. {}2022-09-27T10:58:14.292Z [DEBUG] (pool-2-thread-5) software.amazon.awssdk.http.apache.internal.net.SdkSslSocket: created: evergreencomponentmanageme-artifactbucket7410c9ef-1t7hs9n3e7a5l.s3.eu-west-2.amazonaws.com/52.95.148.90:443. {}2022-09-27T10:59:57.647Z [DEBUG] (idle-connection-reaper) software.amazon.awssdk.http.apache.internal.net.SdkSslSocket: shutting down output of greengrass-ats.iot.eu-west-2.amazonaws.com/18.133.211.72:8443. {}[...]2022-09-27T11:00:38.493Z [INFO] (Thread-3) com.aws.greengrass.deployment.ShadowDeploymentListener: Added shadow deployment job. {ID=arn:aws:greengrass:eu-west-2:548876126010:configuration:thing/MyGreengrassCoreV2:18}2022-09-27T11:00:42.020Z [INFO] (pool-2-thread-12) com.aws.greengrass.deployment.DeploymentService: Canceling current device deployment. {DeploymentId=arn:aws:greengrass:eu-west-2:548876126010:configuration:thing/MyGreengrassCoreV2:17, serviceName=DeploymentService, currentState=RUNNING}2022-09-27T11:00:42.022Z [DEBUG] (pool-2-thread-12) com.aws.greengrass.deployment.DeploymentStatusKeeper: Storing deployment status. {DeploymentId=arn:aws:greengrass:eu-west-2:548876126010:configuration:thing/MyGreengrassCoreV2:17, DeploymentStatus=CANCELED}2022-09-27T11:00:42.023Z [INFO] (pool-2-thread-4) com.aws.greengrass.deployment.DeploymentService: deployment-task-execution. Cancelled package download due to received interrupt. {deploymentId=3c3703c6-feb1-404a-86fe-317282a02b15, serviceName=DeploymentService, currentState=RUNNING}2022-09-27T11:00:42.138Z [INFO] (pool-2-thread-12) com.aws.greengrass.deployment.DeploymentStatusKeeper: Stored deployment status. {DeploymentId=arn:aws:greengrass:eu-west-2:548876126010:configuration:thing/MyGreengrassCoreV2:17, DeploymentStatus=CANCELED}I'm not sure which part of the logs is relevant, don't hesitate to ask for more logs.The thing is that I see no error from the tested device point of view, the deployment is launched, but after some time the device exits from whatever was going on (see end of part 2/3 of greengrass.log above) and almost a minute later a cancel command is given by the tester that considers that the deployment is not completed in time.I'm completely new to AWS so I may have missed something obvious, I'll take any insight on my issues.PS : the time in the logs from the tested device is 2h late (an explanation of the errors?) even though the time of my Raspi (given by the "date" command) is the right one...FollowCommentsatvemulAWS 8 months agoHelloThanks for the logs you have provided. For now, what you have provided suffice.In the greengrass.log, can you trace the status of the deployment with Id '548876126010'. Please confirm it reached RUNNING status, before the cancellation was attempted.If the deployment was still in progress, then for various reasons, the device is needing more time for the deployment to happen. You can use the "Timeout multiplier" feature to increase the actual wait times. The usage is documented herePlease let me know your findings.ShareTheFrenchIntern 8 months agoHello, thanks for the reply.The deployment execution is started according to these logs, few lines below the part 1/3 of greengrass.log that I've already posted:2022-09-27T10:57:57.018Z [INFO] (pool-2-thread-4) com.aws.greengrass.deployment.DeploymentService: deployment-task-execution. Starting deployment task. {Deployment service config={ComponentToGroups={}, dependencies=[], GroupMembership={thinggroup/MyGreengrassGroupV2=null}, GroupToRootComponents={thing/MyGreengrassCoreV2={}, thinggroup/MyGreengrassGroupV2={}}, runtime={ProcessedDeployments={}}, version=0.0.0}, deploymentId=3c3703c6-feb1-404a-86fe-317282a02b15, serviceName=DeploymentService, currentState=RUNNING}I don't think that is a question of timing because there is a delay between the last action in the deployment and the order of shutting down (see 2 last lines of greengrass.log part 2/3, nothing happens between 10:58:14 and 10:59:57) and again another ~40s delay before the tester decides to cancel the deployment.What I find even weirder is that logs indicate that the EmptyDeployment (used for cleaning up) is executed and its status is marked as SUCCEEDED after some time while the test manager indicates "Empty deployment did not reach COMPLETED"...ShareTheFrenchIntern 8 months agoI found an error occuring before (during the cloudComponent test) that may be related and that I was not able to solve neither. It involves the credendials provider.2022-09-27T10:55:43.390Z [DEBUG] (pool-2-thread-17) com.aws.greengrass.tes.CredentialRequestHandler: Received response from cloud: {"message":"Access Denied"}. {iotCredentialsPath=/role-aliases/GreengrassCoreTokenExchangeRoleAlias/credentials, statusCode=403}2022-09-27T10:55:43.391Z [ERROR] (pool-2-thread-17) com.aws.greengrass.tes.CredentialRequestHandler: TES responded with status code: 403. Caching response. {"message":"Access Denied"}. {iotCredentialsPath=/role-aliases/GreengrassCoreTokenExchangeRoleAlias/credentials}2022-09-27T10:55:43.393Z [ERROR] (pool-2-thread-17) com.aws.greengrass.tes.CredentialRequestHandler: Error in retrieving AwsCredentials from TES. {iotCredentialsPath=/role-aliases/GreengrassCoreTokenExchangeRoleAlias/credentials, credentialData=TES responded with status code: 403. Caching response. {"message":"Access Denied"}}2022-09-27T10:55:43.393Z [DEBUG] (pool-2-thread-17) com.aws.greengrass.componentmanager.builtins.S3Downloader: get-bucket-location. task failed and will be retried. {task-attempt=4, componentIdentifier=com.aws.HelloWorld, artifactUri=s3://idt-87dd63247a106c50bff2-gg-component-store/greengrass/components/artifacts/aws-greengrass-testing-features-cloudcomponent.zip}software.amazon.awssdk.core.exception.SdkClientException: Failed to fetch credentialsSharesatvemulAWS 8 months agoI am sorry. I should have asked for the status of 'COMPLETED' for the deployment with id 548876126010. Can you try to find in the logs, what is the last status reported, before the cancellation was requested.As far as the errors related to 'GreengrassCoreTokenExchangeRoleAlias/credentials' is concerned, since this is case of pre-installed greengrass, the expectation is that the 'GreengrassCoreTokenExchangeRoleAlias' has the right role with policy attached. You can find some documentation related to this role here. You can see the one highlighted note about s3 permissions attached to the policy. The issue could be in that area.For non-preinstalled use case, IDT Greengrass Testsuite offers a way to configure custom role alias as documented here. That can also give additional insights.ShareTheFrenchIntern 8 months agoI do not see any mention that the deployment reached COMPLETED. I tried to increase the timeout as you suggested but it only makes more obvious the delays that I described in one of my earlier comments.ShareShow 5 more comments"

Answer: "0I'm providing here raw results from a group test I ran.It was the "component group". The Raspi passed the first part of the CloudComponent test and the LocalComponent test but failed at the second part of the CloudComponent test.You can see in my logs that I have sometimes troubles with my PKCS#11 provider, in general after some time it manages to connect. But maybe it is the root cause of my issues with deployments.I noticed that in the failed deployment case there was a lot of "Waiting for dependencies", as if the device had troubles loading services, it may be the reason it failed, but in this case, I don't know how to solve this issue.There is the same surprising behavior that I mentioned above, at 07:55:38 (in greengrass.log), the deployment stops, it seems to be successsful, at 07:57:08 connections are closed and 30" later an empty deployment is launched to clean the whole thing up.I also find extremely difficult to debug all of this because some errors seem to pop up randomly. I managed to pass this morning the "mqtt" test for example, but this afternoon, it failed while I had not changed a thing.If someone has any explanation on these puzzling issues, I would be very grateful.CommentShareTheFrenchInternanswered 8 months agosatvemulAWS 8 months agoHelloI have gone through the logs, and for the Cloud Component failure, it appears like an issue of timing. By using timeout multiplier, you can give the test run more time to succeed. I see that the test component in fact executed, but appears the test flow could not validate it before it failed.As far as the reliability goes, there are three aspects involved.The test flow, which is fixed runs a series of actions on the device and Cloud APIs, and validates expected results. This aspect can be influenced by adding time out multipler. This is because depending on the device and it's network setup, it might need more time.The Cloud API informs device(Greengrass) for any deployment related actions. Here, the region you are using to test the device, can play a role.The device with Greengrass installed on it, responds to the deployment actions, and installs and runs the components. In this aspect, the device capability and the network setup can play a role.ShareTheFrenchIntern 8 months agoThanks for your reply.I ran the same test with a timeout multiplier = 2 but to me it does not change anything. At 08:01:27 everything is closing down even if there still 4 minutes left by the tester for the deployment to be completed. The logs can be found here https://drive.google.com/drive/folders/1e7tatn-s-cyIkHVF1NfgzjRn2GSZAQhU?usp=sharingAnd if it happens to be linked with the device or network capability, do you have an idea why some deployments do pass anyway?For example, in the CloudComponent test, if I understand it right, the first deployment is targeted on the thing while the second is on the group. It seems to me that the difference is minor, however the first succeeds whereas the second does not.ShareTheFrenchIntern 8 months agoAs for the region, I would ideally use eu-west-3 due to my location, but it was unavailable, can't remember the reason, so I chose eu-west-2 instead: do you think it could play a significant role ?ShareTheFrenchIntern 8 months agoDid you manage to find any explanation on the issue?ShareAWS-IoT-GG-Rafe 8 months agoHelloThis is Rafe from IoT Greengrass team, we are still working on your case. At the meantime, can u provide us the account id you are using for the test run you listed before - https://drive.google.com/drive/folders/1e7tatn-s-cyIkHVF1NfgzjRn2GSZAQhU?We need the account id and region to check that problematic deployment context from the database.ThanksRafeShareShow 5 more comments"


Question: How to remove emails from AWS SES queue

"Hello, Is it possible to remove emails from Remaining sends in AWS SES queue.FollowComment"

Answer: 0HelloYou could use PurgeQueue method against queue used by SES (https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_PurgeQueue.html)CommentShareAleksandr_Panswered 10 months ago


Question: DynamoDB Update Deep Nested Attributes

"I have the following data structure with multi-level Map columns/items:{"pk": {    "S": "cf633075-e808-4e33-9fc1-08301b54384b"},"active": {    "N": "1"},"delta": {    "N": "-0.9371997798455771"},"dte": {    "N": "0.01893752246685426"},"hedge_positions": {  "M": {    "151e9c9f-aa5d-4f5d-9826-74011da03137": {      "M": {        "active": {          "N": "1"        },        ...      }    },    "1bed59c9-6f73-48ae-8b65-62fd1f10ddad": {      "M": {        "active": {          "N": "1"        },        ....      }    },    "24cf6015-780e-408d-b0e1-935ae2c7199d": {      "M": {        "active": {          "N": "1" //Update this to 0        },        ...      }    }  }}I now wish to update the value of "active" for one of the "hedge_positions" to 0. The hedge_positions is a Map, containing unique Maps.In Javascript, I have built the following update, but I am recieving the error "The provided expression refers to an attribute that does not exist in the item".const AWS = require('aws-sdk');const dynamoDb = new AWS.DynamoDB({region: process.env.AWS_REGION});var hID = "24cf6015-780e-408d-b0e1-935ae2c7199d";var params = {            TableName: Positions,            Key: {                pk: {"S": positionId }            },            UpdateExpression: "SET #hp.#id.#a = :a",            ExpressionAttributeValues: {":a": {"N": "0"}},            ExpressionAttributeNames: {"#hp": "hedge_positions", "#id": hID, "#a": "active"}};await dynamoDb.updateItem(params).promise();I believe this to be because I am not specifying that the "GUID" is itself a Map. I have tried to set the UpdateExpression to "SET #hp.#id.M.#a = :a" but to no avail. What have I set incorrectly in the UpdateExpression?FollowComment"

Answer: "1I tested this in my own account and it works with no issue, returning the NEW_IMAGE in the response to ensure the update was successful:const AWS = require('aws-sdk');const dynamoDb = new AWS.DynamoDB({region: 'eu-west-1'});var hID = "24cf6015-780e-408d-b0e1-935ae2c7199d";var params = {    TableName: "cust1",    Key: {        pk: { "S": "123" },        sk: { "S": "abc" }    },    UpdateExpression: "SET #hp.#id.#a = :a",    ExpressionAttributeValues: { ":a": { "N": "0" } },    ExpressionAttributeNames: { "#hp": "hedge_positions", "#id": hID, "#a": "active" },     ReturnValues: "ALL_NEW"};dynamoDb.updateItem(params).promise().then(res=>console.log(JSON.stringify(res))).catch(err=>console.log(err));Item before:{ "pk": "123", "sk": "abc", "active": 1, "delta": -0.9371997798455771, "dte": 0.01893752246685426, "hedge_positions": {  "151e9c9f-aa5d-4f5d-9826-74011da03137": {   "active": 1  },  "1bed59c9-6f73-48ae-8b65-62fd1f10ddad": {   "active": 1  },  "24cf6015-780e-408d-b0e1-935ae2c7199d": {   "active": 1  } }}Item After:{ "pk": "123", "sk": "abc", "active": 1, "delta": -0.9371997798455771, "dte": 0.01893752246685426, "hedge_positions": {  "151e9c9f-aa5d-4f5d-9826-74011da03137": {   "active": 1  },  "1bed59c9-6f73-48ae-8b65-62fd1f10ddad": {   "active": 1  },  "24cf6015-780e-408d-b0e1-935ae2c7199d": {   "active": 0  } }}CommentShareEXPERTLeeroy Hannigananswered 9 months agoEXPERTChris_Greviewed 9 months ago"


Question: Unable to update Wordpress on Lightsail

"Hi Team,i have successfully built a Wordpress site on a personal Lightsail account, and post development i was able to package it and the DB, and setup it up on a new instance on the clients AWS account and transferred all the files and the db, the site works completely fine.Except i am unable to update the Wordpress core. I have tried updating the file permissions and nothing has worked, the error i get is"The update cannot be installed because some files could not be copied. This is usually due to inconsistent file permissions.: wp-admin/includes/update-core.php"What i doubt is the problem is with the user role that triggers the update or the write access to these folders, so i tried to do the update through the wp-cli and i got the same error, even though i ran it sudo.all the files have the permission of 664/644 and dir 775/755. and the file ownership is bitnami:daemon.The plugin/theme updates are working fine, the media uploads without an issues, only the core update im having an issue.I will put out a short decription on how to move from lightsail to another account based lightsail, once i can rectify this.Regards,ShivaFollowComment"