Question: Redshift removes a backslash from string stored in a varchar field

"Hi Good People,Redshift and Aurora or plain PostgreSQL works a little bit differently in terms of storing backslashes in a varchar field. Redshift removes one backslash in distinction from Aurora which is inserting string as it is. Is it OK?Minimal example:On Redshift: select '\\' as test from test_table limit 1;returns:\On Aurora or pgsql:select '\\' as test from test_table limit 1;returns:\\I would like to store JSON in varchar field in RS. Problems start when JSON contains backslash sign to escape JSON's token:{"field":"text\""}In result, \ is removed from stored value changing the value that has been requested to be stored, eventually corrupting JSON which cannot be deserialized.CREATE TABLE test_table(  varchar_field varchar);insert into test_table_1 (varchar_field) VALUES ('{"field":"text\""}');select * from test_table;returns:{"field":"text""}which is not proper JSON but a proper JSON were inserted.For the same example, Aurora returns a proper JSON:{"field":"text\""}Is this desired behavior?How can I prohibit removing backslashes?Is somewhere described how storing texts in RS different from other DBs?I'm inserting data to RS via insert statement, it won't be doable in short term solution to use COPY instead. How can I deal with it? It's highly possible that I'm missing something crucial here but I stuck for a few hours and cannot find a solution by myself :)RS Cluster Version: 1.0.7562RS Driver version: 1.2.16.1027Best regards,pbyEdited by: pby on May 17, 2019 7:49 AMEdited by: pby on May 17, 2019 8:03 AMFollowComment"

Answer: "0Accepted AnswerHi pby,I suspect the difference in the behavior may be tied to the base PostgreSQL version for each the test cases you demonstrate.Redshift is a single version derivative of a PostgreSQL v8.0.2 fork as documented here: https://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.htmlAurora versions are derivatives of many PostgreSQL versions documented here: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Updates.20180305.htmlPostgreSQL will be whatever version it is.I suspect the escape handling in your string literals is an artifact of the base PostgreSQL version, i.e. Redshift and PosgreSQL v 8.0.2 will likely do the same thing.I hope this helps,-KurtCommentShareklarsonanswered 4 years ago0Hi klarson,Thank you. You right.I checked on postgres v8.0.2 and, in fact, '' is handled differently than with postgres 10.Best regards,pbyCommentSharepbyanswered 4 years ago"


Question: AWS CDK - how can I retrieve the ID of my primary network interface?

In AWS CDK I am creating an EC2 instance. CDK automatically creates a default primary network interface for this instance. How can I access this network interface's id? I'd need this in the output.FollowComment

Answer: "0The challenge here is that the ENI identifier isn't known until it is created which happens during stack creation, not when CDK is executin (which is when the CloudFormation template is being written).Why do you need the identifier? What are you trying to do?CommentShareEXPERTBrettski-AWSanswered a year agoTheBlob a year agoI know that the the actual ENI identifier isn't known during execution - that's OK. I'd need the value as an Output.Why do I need it? I want to use the identifier to attach secondary IP addresses to it via CLI .Share"


Question: .metal instance question

Do .mental instances keep accruing compute charges when they are shutdown?FollowComment

Answer: "2Accepted AnswerFrom the docs: "When you stop an instance, we shut it down. We don't charge usage for a stopped instance, or data transfer fees, but we do charge for the storage for any Amazon EBS volumes. Each time you start a stopped instance we charge a minimum of one minute for usage. After one minute, we charge only for the seconds you use."There is no guarantee that when you stop an instance when you restart you will be running on the same hardware. Any NVMe "ephemeral" storage is not available after an instance is stoppedCommentShareEXPERTPeter_Ganswered a year agoEXPERTBrettski-AWSreviewed a year ago"


Question: Cloudfront distribution when using Amplify Studio

"I have an application deployed using Amplify Studio with custom domain, cloudfront distribution and a TLS certificate. Everything works well. I do not know why the cloudfront page does not list the distribution. I can see being configured in the hosted zone A record. Any clues?Thank you.FollowComment"

Answer: 0Because that AWS managed Cloudfront distribution and it exist in AWS Account.You can follow below if you want to deploy your own managed Cloudfront distrohttps://docs.amplify.aws/cli/hosting/hosting/#amazon-s3-and-amazon-cloudfrontCommentShareMuhammadanswered 4 months ago


Question: Block suspicious(specific) user-agent using AWS WAF

"I want to block all requests with suspicious user-agent to access APIGW. For example, how can I make  rule to block all requests  with user-agent as "python-requests/2.27.1" or other version of python-requestsFollowComment"

Answer: 0Accepted AnswerThe simplest way to do this is to create a string match condition and then check the User-Agent header against that condition.CommentShareEXPERTBrettski-AWSanswered a year ago


Question: Kinesis Firehose Dynamic Partitoning with inline jq expression

"Following the aws blog post : Dynamic Partitioning.So I have a firehose delivery stream configured to convert api calls into parquet output  to s3 - and now partitioned according to the value of a field from within my data - 'playtype'.However, in some rows the value in this partioning field is null/empty and this causes those rows to error. Therefore I want to adjust the jq expression for the dynamic partioning key to substitute a default value for those null cases. Having trouble with the syntax though.JQ expression specified for the field is .playtypeSo I have tried things like:.playtype // "some-hardcoded-default".playtype | if . == null or . == "" then "some-hardcoded-default" else . endThese evalute as valid jq in jqplay - but in practice i.e. in firehose are returning a 'jq syntax error'. How can i specify this logic correctly? i.e. partition on this field but if it the value is null or empty then use 'some-hardcoded-default'.Thankyou!FollowCommentAWS-User-4356880 a year ago(.playtype | if . == null or . == "" then "some-hardcoded-default" else . end)put it inside the brackets , it will work .Share"

Answer: "1Accepted AnswerHi there, please try as per above user suggestion in the comments and let us know the results. jqplay is currently blocked on our end so couldn't check the validity. I believe having brackets for each expression should fix the issue. However if that doesn't work, I would recommend you to resort to using a transformation Lambda function like mentioned here: https://docs.aws.amazon.com/firehose/latest/dev/dynamic-partitioning.html in the case where field is not present in the incoming data. That way you would have a lot more control on the fields.CommentShareSUPPORT ENGINEERHarshith_Manswered a year agoAWS-LDD a year agoAWS-User-4356880 is correct! Many thanks. I had used a transformation lambda function as you also suggest and this works too but the former was a much simpler, quicker way to go.Share"


Question: Update Node to LTS

"According to DF documentation, "Device Farm supports all Appium server versions 1.6.5 and above." Appium 1.21 & 1.22 requires at least Node v12, but the current Node version in DF is set to v10. How do I update the version or set it to a specific version? I tried "n lts" in the yaml file, but got "mkdir: cannot create directory ‘/usr/local/n’: Permission denied".There are fixes in 1.21/1.22 that we need for mobile testing.FollowComment"

Answer: "0Accepted AnswerThanks for reaching out. Here are commands you can add to your test spec file in order to upgrade your Node version:[code]export NVM_DIR=$HOME/.nvm. $NVM_DIR/nvm.shnvm install --lts[/code]CommentSharesimsjon at AWSanswered 2 years ago0Can someone please provide info on how to update Node.js in DF?CommentShareMatthewSanswered 2 years ago0Thanks, that works!CommentShareMatthewSanswered a year ago"


Question: DeepAR vs DeepVAR confusion

"I understand that sagemaker Deepar is a deep learning-based model designed for time series forecasting and it is primarily used for univariate time series forecasting.However, it can incorporate dynamic features, which are time-varying covariates or additional time series that provide extra information to improve the forecasting accuracy. My question is: Are the patterns and relationships between the dynamic variables factored into DeepAR algorithm, and hence making it suitable for applications where the interactions between variables are important?What will be the main difference betwen DeepAR and DeepVAR then?Many thanks.FollowComment"

Answer: "0DeepAR and DeepVAR are booth deep learning models designed for time series forecasting, and they can handle multiple related variables simultaneously. This difference is in their underlying architecture and modelling approach.The Amazon SageMaker DeepAR forecasting algorithm is a supervised learning algorithm for forecasting scalar (one-dimensional) time series using recurrent neural networks (RNN). Classical forecasting methods, such as autoregressive integrated moving average (ARIMA) or exponential smoothing (ETS), fit a single model to each individual time series. They then use that model to extrapolate the time series into the future.In many applications, however, you have many similar time series across a set of cross-sectional units. For example, you might have time series groupings for demand for different products, server loads, and requests for webpages. For this type of application, you can benefit from training a single model jointly over all of the time series. DeepAR takes this approach. When your dataset contains hundreds of related time series, DeepAR outperforms the standard ARIMA and ETS methods. You can also use the trained model to generate forecasts for new time series that are similar to the ones it has been trained on.https://docs.aws.amazon.com/sagemaker/latest/dg/deepar.htmlDeepVAR: DeepVAR, on the other hand, is specifically designed for multivariate time series forecasting. It considers the interdependencies and cross-correlations among multiple variables. DeepVAR is suitable when the relationships between variables are important and need to be explicitly modeled. This model is particularly useful in domains where multiple related variables influence each other, such as macroeconomic forecasting, supply chain management, and portfolio risk analysis.DeepAR is based on an auto-regressive recurrent neural network model, specifically designed for multivariate time-series modeling producing accurate probabilistic forecasts.  DeepVAR captures both the temporal dynamics within each variable and the cross-series interactions. DeepVAR is based on the concept of vector autoregression a statical model commonly used in econometrics.CommentSharesqavianswered 6 days ago"


Question: TGW Route Table entries

"I just saw a session about Advanced Architecture with TGW where we show an example of a TGW with 4 attachments (2 VPCs, 1 DGW, 1 VPN). All attachments are associated with the single TGW route table and propagate their routes to it. Nevertheless the RT only shows 3 entries. The presenter says that’s due to DX is leveraged as primary connection to onPrem here and VPN only as a backup.In my understanding of path selection behavior DX propagated routes are preferred over VPN propagated routes. So what is the reason that the RT of the TGW does not show all 4 entries?I’ve built that setup in a lab environment and can confirm that you only see 3 entries there - VPN is missing. Only when you failover to VPN, the DX propagated routes are not shown anymore in the TGW RT.FollowComment"

Answer: "0Accepted AnswerThis is a "normal" behavior that you can also see with physical router. If multiple routes to the same destination are received, they only place the one with the highest priority in their routing table.On a high level this is due to the decoupling of routing information base (rib) and forwarding information base (fib). The rib takes care of the path selection, e.g. calculates the best path for each known destination (based on different rules and algorithms) and places only this one in the fib, so if there is a packet received that needs to be forwarded the router only needs to look up the next hop in the fib and forwards the packet without the need to calculate the best path every time.I guess it's just the same for TGW route table, I hope someone here can confirm that :-)CommentShareSebastian_Ganswered 2 years ago"


Question: Patch Manager vs Maintenance Window

"Hello,Can someone please show some light on me with the difference between Patch Manager vs Maintenance Window?FollowComment"

Answer: "1Patch ManagerPatch Manager, where you create Patch Policies, which will allow SSM to scan and install patches to managed instances [1], Patch Manager can automate this process with both security related updates and other type of updates.Please check this document on how to use Patch Manager [2]You can use Patch Manager to apply patches for both operating systems and applications. (On Windows Server, application support is limited to updates for applications released by Microsoft.) You can use Patch Manager to install Service Packs on Windows nodes and perform minor version upgrades on Linux nodes.Maintenance WindowMaintenance window helps you define the schedule in SSM for when to perform a task on your managed node such as patching an OS, install drivers or software [3]Please check this document and video on how to create a Maintenance Window [4] [5]you can schedule actions on numerous other AWS resource types, such as Amazon Simple Storage Service (Amazon S3) buckets, Amazon Simple Queue Service (Amazon SQS) queues, AWS Key Management Service (AWS KMS) keys, and many more.Each maintenance window has a schedule, a maximum duration, a set of registered targets, and a set of registered tasks. You can add tags to your maintenance windows when you create or update them.To conclude, Patch manager automate the task of scanning and installing an update on the nodes while Maintenance window help you to create a schedule for when to perform action.References:https://docs.aws.amazon.com/systems-manager/latest/userguide/patch-manager.htmlhttps://docs.aws.amazon.com/systems-manager/latest/userguide/patch-manager-console.htmlhttps://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-maintenance.htmlhttps://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-maintenance-working.htmlhttps://www.youtube.com/watch?v=1r3YU2KoK5sCommentShareGovind Kumaranswered 20 days agoAWSUser 20 days agoVery Well explained.Share-2The following document is not a patch manager, but is a use case for State Manager and Maintenance Windows.https://docs.aws.amazon.com/systems-manager/latest/userguide/state-manager-vs-maintenance-windows.htmlTo begin with, I don't think Patch Manager and Maintenance Windows are comparable.The patch manager is an image of a maintenance window or state manager that has the task of patching and distributing packages cut out as a function.CommentShareEXPERTRiku_Kobayashianswered 20 days ago"


Question: Api Gateway with AWS_IAM authorizer | 403 Forbidden

"I have an OCR service hosted in the below setupAPI Gateway -> LambdaI have enabled AWS_IAM Authorization on API Gateway, and for input data like JSON it's working as expectedHowever, when I choose to enable binary media types and try to upload Image to the same setupThe request is denied with status code 403 and error message as,{"message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details ...[TRUNCATED]"}I do know that within API Gateway, there is the conversion of Headers and request body for binary inputs, which I believe is the main cause for this issue.But this must be a standard setup and I must be missing something here.Appreciate your help.Update: this is not working only from POSTMAN, It works seamlessly via python requests and AWS4AuthFollowComment"

Answer: 0This sounds like a problem in: X-Amz-Signature.I suggest you review the following documentation:https://docs.aws.amazon.com/general/latest/gr/signature-v4-troubleshooting.html#signature-v4-troubleshooting-canonical-errorsCommentShareMaxCloudanswered a year ago


Question: Error when trying to launch new project in editor on Lumberyard 1.25

"I just installed lumberyard and when into the project configuration and created a new project (Requiem) and that screen showed me that it built the project and everything; however, when I go into the editor, I get an error from the Asset Processor, picture of the error message is below. I have noticed that in the Bin64vc142 there is no Gem.Requiem.something.dll file either. So it looks to be that Gem was not compiled and could not find a way to manually compile that Gem like with VS2019 or command lineREMOVEDUPLOADFollow"

Answer: "0I just tested this, at least with 1.25 you should be able to build from command line and it should build what is necessary in minutes instead of taking 2 hours to build everything.Run these commands in your dev directory from command line.Configure the project:lmbr_waf configureBuild all for VS2019:lmbr_waf build_win_x64_vs2019_profile -p all --progressSharerePost-User-6034607answered 3 years ago0when i run "lmbr_waf build_win_x64_vs2019_profile -p all --progress" i get a build error:e:\Games\Requiem\lumberyard\dev\Code\Framework\AzQtComponents\AzQtComponents/Components/AutoCustomWindowDecorations.h(15): fatal error C1083: Cannot open include file: 'QObject': No such file or directoryButtonDivider.cppe:\Games\Requiem\lumberyard\dev\Code\Framework\AzQtComponents\AzQtComponents/Components/ButtonDivider.h(16): fatal error C1083: Cannot open include file: 'QFrame': No such file or directoryBuild failed<++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>Compilation failed - File: 'AutoCustomWindowDecorations.cpp.9426794.obj', Module: 'AzQtComponents', Configuration: 'win_x64_vs2019|profile', error code 2Input Files:   'e:\Games\Requiem\lumberyard\dev\Code\Framework\AzQtComponents\AzQtComponents\Components\AutoCustomWindowDecorations.cpp'Output Files:  'e:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\AzQtComponents\AzQtComponents\Components\AutoCustomWindowDecorations.cpp.9426794.obj'Command:       'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\bin\HostX64\x64\CL.exe /bigobj /nologo /WX /MP /Gy /GF /Gm- /fp:fast /Zc:wchar_t /Zc:forScope /Gd /Zc:inline /wd4530 /wd4577 /FS /std:c++17 /Zc:__cplusplus /experimental:external /external:W0 /wd4193 /Ox /Ob2 /Ot /Oi /Oy- /GR /showIncludes /MD /W4 /wd4127 /wd4251 /wd4351 /Z7 /Ie:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\AzQtComponents\AzQtComponents /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\AzQtComponents\AzQtComponents /Ie:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\AzQtComponents /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\AzQtComponents /Ie:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\AzToolsFramework /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\AzToolsFramework /Ie:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\AzFramework /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\AzFramework /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\AzFramework\Platform\Windows /Ie:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\GridMate /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\GridMate /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\GridMate\Platform\Windows /Ie:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\AzCore /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\AzCore /Ie:\Games\Requiem\lumberyard\dev\Code\Framework\AzCore\Platform\Windows /IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\ATLMFC\include /IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\include /IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um /IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt /IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared /IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um /IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt /IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt /external:IE:\Games\Requiem\lumberyard\3rdParty\rapidjson\rapidjson-1.1.0\include /external:IE:\Games\Requiem\lumberyard\3rdParty\rapidxml\1.13-modified.1\include /external:IE:\Games\Requiem\lumberyard\3rdParty\Lua\5.1.1.9-az /external:IE:\Games\Requiem\lumberyard\3rdParty\zstd\1.35-pkg.1\lib /external:IE:\Games\Requiem\lumberyard\3rdParty\zstd\1.35-pkg.1\lib\common /external:IE:\Games\Requiem\lumberyard\3rdParty\zstd\1.35-pkg.1\lib\compress /external:IE:\Games\Requiem\lumberyard\3rdParty\zstd\1.35-pkg.1\lib\decompress /external:IE:\Games\Requiem\lumberyard\3rdParty\lz4\r128-pkg.3\lib /external:IE:\Games\Requiem\lumberyard\3rdParty\zlib\1.2.8-pkg.3\include /DPRODUCT_SKU_default /D_WIN32 /D_WIN64 /D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING /D_ENABLE_EXTENDED_ALIGNED_STORAGE /D_PROFILE /DPROFILE /DNDEBUG /DAZ_ENABLE_TRACING /DAZ_ENABLE_DEBUG_TOOLS /DEXE_VERSION_INFO_0=1 /DEXE_VERSION_INFO_1=25 /DEXE_VERSION_INFO_2=0 /DEXE_VERSION_INFO_3=0 /DQT_QML_DEBUG /DAZ_QT_COMPONENTS_EXPORT_SYMBOLS /D_MT /D_DLL /DPLATFORM_SUPPORTS_AWS_NATIVE_SDK /DUSE_WINDOWS_DLL_SEMANTICS /DRESOURCE_COMPILER /DFORCE_STANDARD_ASSERT /D_CRT_SECURE_NO_DEPRECATE=1 /D_CRT_NONSTDC_NO_DEPRECATE=1 /DWIN32 /DWIN64 /DNOMINMAX /DLY_METRICS_BUILD_TIME=0 /DUSE_ZSTD e:\Games\Requiem\lumberyard\dev\Code\Framework\AzQtComponents\AzQtComponents\Components\AutoCustomWindowDecorations.cpp /c /Fo e:\Games\Requiem\lumberyard\dev\BinTemp\win_x64_vs2019_profile\Code\Framework\AzQtComponents\AzQtComponents\Components\AutoCustomWindowDecorations.cpp.9426794.obj'Output:AutoCustomWindowDecorations.cppe:\Games\Requiem\lumberyard\dev\Code\Framework\AzQtComponents\AzQtComponents/Components/AutoCustomWindowDecorations.h(15): fatal error C1083: Cannot open include file: 'QObject': No such file or directory<++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>SharerePost-User-3292008answered 3 years ago0@REDACTEDUSERHave you run Setup Assistant and made sure you have all the requirements? This looks like it's having trouble finding Qt.There's a good, current set of installation instructions  here:https://docs.aws.amazon.com/lumberyard/latest/welcomeguide/wg-setup-intro.htmlParticularly this page for Setup Assistant.https://docs.aws.amazon.com/lumberyard/latest/welcomeguide/wg-setup-assistant.htmlMake sure you have green checks for everything on the Install Required SDKs page. Also important to make sure the Path at the top of the Required SDKs page correctly points to your third party folder. If you are coming from an older version of Lumberyard, it's possible that Qt was upgraded and needs to be re-downloaded.SharerePost-User-6034607answered 3 years ago0[quote="microAMZN, post:4, topic:9213"]ake sure you have green checks for everything on the Install Required SDKs page. Also important to make sure the Path at the top of the Required SDKs page correctl[/quote]Yes I did for the items that I had checked. I did some digging and the command "lmbr_waf build_win_x64_vs2019_profile -p all --progress" required the Qt framework is only copied when you have selected "compile the engine and asset pipeline" which I did not select.SharerePost-User-3292008answered 3 years ago0Ahh, sorry, I should have thought of that. Glad you worked it out!SharerePost-User-6034607answered 3 years ago"


Question: Get spot instance cost by costExplorer get-cost-and-usage cli

"I created a spot instance, and after a day, I use costExplorer get-cost-and-usage to get real cost of this instance.But I got the cost is the same with on demand pricing.I'm starting to doubt my instance is spot or not.I can see my instance "Lifecycle" is "Spot" in ec2 console.And I can see this instance in "Spot Requests" list.If we confirm the instance is spot, so my question is whether costExplorer api can get spot instance cost and how to get it?Thanks,MingtongFollowComment"

Answer: 0Hi - I think this blog provides a very good overview of what you are looking for.  https://aws.amazon.com/blogs/compute/diving-deep-into-ec2-spot-instance-cost-and-operational-practices/. See under "Cost management tools" options. It also points out some best practices of using tags.CommentShareEXPERTAWS-User-Nitinanswered 7 months ago


Question: CouchDB in managed blockchain

In managed hyperledger..  Does the member node run couchdb or can we add it somehow?FollowComment

Answer: 0I just had a conversation with AWS support team. and they only supports leveldb in ManagedBlockchain.They do have a feature request for the couchDB. so hopefully they do support sooner than laterCommentShareykimanswered 4 years ago0Now Amazon Hyperledger fabric supports node configuration in both LevelDB and CouchDB to save state of ledger at nodes. https://docs.aws.amazon.com/managed-blockchain/latest/APIReference/API_NodeConfiguration.htmlCommentShareDDanswered 7 months ago


Question: Two accounts are being shown as both enrolled in one view and not enrolled in another. Why?

"I have 3 accounts in a "Privileged" OU: ExternalDevelopment, Production, and Shared. All were made with the Account Factory.In the CT Account view, ExternalDevelopment and Production show as enrolled, and Shared doesn't.In the CT OU view, all three accounts show as Not Enrolled.ScreenshotAny explanation?FollowComment"

Answer: "0HelloIn CT the existing accounts doesnt automatically enrolle the accounts, Those accounts must meet requirements for th CT, Please check the link https://aws.amazon.com/blogs/architecture/field-notes-enroll-existing-aws-accounts-into-aws-control-tower/CommentShareGKanswered a year ago"


Question: Maximum number of user/group to access a QuickSight dashboards

Is there a limit to number of user/group to access a QuickSight dashboards?FollowComment

Answer: "2Amazon QuickSight has limits on the number of users and groups that can be created within an account, but there are no specific limits on the number of users or groups that can access a single dashboard. However, the practical limit for users or groups accessing a dashboard will depend on the total number of users and groups that you can create within your account, which is subject to the QuickSight service quotas.the default quotas for Amazon QuickSight are:*** Standard Edition:**Maximum 500 users per AWS accountMaximum 500 groups per AWS account*** Enterprise Edition:**Maximum 2,000 users per AWS accountMaximum 2,000 groups per AWS accountPlease note that these quotas are subject to change, and you can request a quota increase from AWS Support if needed.CommentShareEXPERTsdtslmnanswered 2 months ago"


Question: KMS key per application or per service

"I’m trying to figure out the best approach when deploying applications and encrypting them with KMS. All deployments will be done via code.Is it best practice to have a key per service?Would it be overkill to have a key per application, per service?example :One Key Per-Application•Application – SWG-Archive•Key: Alias/SWG-Archive•One key to encrypted EBS , S3, File gatewayMany keys per application•Application – SWG-Archive•Key:oAlias/SWG-Archive/S3oAlias/SWG-Archive/ebsoAlias/SWG-Archive/file_garewayWhich is the best approach.FollowComment"

Answer: "0Accepted AnswerHi,I don't think there is a 'one size fits all' answer here.  I think you need to think about: -Account strategy - are you separating by environment, application, business unit, scope of external accreditation (e.g. PCI-DSS)Who's developing / administering the applications (one team, multiple teams etc.) and whether it's being done via shared tooling, separate tooling etc.How highly you want to separate your dataI wouldn't personally suggest you create a central KMS account and then share the keys out from there in a large environment.  It's complex from a permissions point of view and in very large environments you could end up with throttling issues.You also want to think about the costs of KMS.  There's a cost per key, and also the way you use KMS in services like S3 can significantly impact the cost of KMS API usage.There's an AWS best practice guide to KMS that might be useful to you here.I'd be inclined to go for one key per application, rather than a shared key per service, or multiple keys per application, unless you're specifically trying to do something different with the data and want to limit the rights to a certain area (e.g. archive) using both IAM permissions as well as limits on what can be done with the keys, yuor application uses different classifications of data that you need to treat differently (different algorithms, being backed by a HSM etc.) or using different purposes (e.g. encrypt / decrypt versus sign/verify).MarkCommentSharemarkrossatosanswered a year agoAWS-User-3628866 a year agoThank you for your very detailed answer, the question was very open-ended but you have helped a lot. The links you provided for further reading were very useful as well :) .Share"


Question: WordPress files on LightSail

Can I see and access WordPress files on LightSail without using the console?FollowComment

Answer: "2Hi THereYou can connect to your Linux and Unix Lightsail instances using a third-party SSH client, and connect to your Windows instances using a third-party RDP client.See https://lightsail.aws.amazon.com/ls/docs/en_us/articles/understanding-ssh-in-amazon-lightsailCommentShareEXPERTMatt-Banswered 8 months agoEXPERTBrettski-AWSreviewed 8 months ago0The above guidance is helpful if you are looking to access your WordPress instance file system from outside the console.  If you are interested in viewing & managing site content, you may find the guidance here helpful: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-tutorial-launching-and-configuring-wordpress#signing-in-to-the-administration-dashboardSpecifically, you can follow the guidance to find your WP admin password, and login to the admin dashboard.  From there, you can manage your posts and media.CommentShareAWS-Andyanswered 8 months agorePost-User-2381584 7 months agoCould you please help  on how i can make my WordPress htaccess file writable because ssl plugin has a trouble with the file but i can't see where is httaccess file is locatedShare0You may also want to ssh into the instance and edit one or both of the following if you are running into trouble enabling HTTPS:sudo nano /opt/bitnami/apache2/conf/bitnami/bitnami.confsudo nano /opt/bitnami/apache2/conf/bitnami/bitnami-ssl.confThere's also a tutorial specifically for enabling HTTPS w/ WordPress in Lightsail: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-enabling-https-on-wordpress.CommentShareAWS-Andyanswered 7 months ago"


Question: Code pipeline failed with no error message - we are adding openSearch to dynamoDB

pipeline failed...Drill in...Drill in...    (Update_Complete) and no error and no data on any of the tabs.FollowComment


Question: IVS can't create Recording Configuration

"Hi All,Please help me, when I created Recording Configuration in IVS (Amazon Interactive Video Service) show error:"This recording configuration cannot be used."" There was an error while creating this recording configuration. It cannot be associated with any channels and can be safely deleted."My region is: Europe (Ireland) eu-west-1Thank everybody.FollowComment"

Answer: "1You may first want to validate that your AWS account has the proper permissions and roles to create IVS Recording Configurations in eu-west-1, as well as S3 buckets. Some corporate IT orgs restrict employee AWS account service access to as needed and block usage of other services.If account permissions, are not the issue, then select the eu-west-1 region in the AWS console, go to the IVS service page. From the menu on the left select Recording Configurations. See if you can create the configuration there. Then add the Recoding config to an existing IVS channel (within same AWS region) or select it when creating a new IVS channel within the same region.CommentShareMike-MEanswered a year agoAWS-User-7754781 a year agoThank you for your advisor.Best regard.Share"


Question: Password Expired

"Greetings,I am new to AWS EC2 and the forum. I recently created an instance and logged in but its been a few weeks and was unable to log back in. I was unable do to a dialog box that popped up saying that the password expired.  I tried creating a new instance. No luck. I tried to run the AWSSupport-ResetAccess and it seemed to have executed successfully but I went back to my instance and I get the same error. It is impossible and so it seems I will not be able to use my free trial.  Thanks in advance.FollowComment"

Answer: 0I resolved my issue in the short term. I will update if I can find out what settings to modify on BitDefender.Turned off Bit defender.  This stopped the "password expired" dialog. But a new issue arose..."Login attempt failed".  I wasn't sure why this was happening because I know the credentials were right.I had to explicitly type "domain\user" in the user dialog after selecting "More Options"  despite the exact same user\domain being listed as the default.CommentShareAlienVanswered 2 years ago0I resolved my issue in the short term. I will update if I can find out what settings to modify on BitDefender.Turned off Bit defender.  This stopped the "password expired" dialog. But a new issue arose..."Login attempt failed".  I wasn't sure why this was happening because I know the credentials were right.I had to explicitly type "domain\user" in the user dialog after selecting "More Options"  despite the exact same user\domain being listed as the default.CommentShareAlienVanswered 2 years ago0I resolved my issue in the short term. I will update if I can find out what settings to modify on BitDefender.Turned off Bit defender.  This stopped the "password expired" dialog. But a new issue arose..."Login attempt failed".  I wasn't sure why this was happening because I know the credentials were right.I had to explicitly type "domain\user" in the user dialog after selecting "More Options"  despite the exact same user\domain being listed as the default.CommentShareAlienVanswered 2 years ago0I resolved my issue in the short term. I will update if I can find out what settings to modify on BitDefender.Turned off Bit defender.  This stopped the "password expired" dialog. But a new issue arose..."Login attempt failed".  I wasn't sure why this was happening because I know the credentials were right.I had to explicitly type "domain\user" in the user dialog after selecting "More Options"  despite the exact same user\domain being listed as the default.CommentShareAlienVanswered 2 years ago


Question: Amazon Transcribe Python API: Event handler processes audio only after the stream ends

"I am sending streaming audio data from web browser as a blob via websocket. In backend, I am using Django Channels' AsyncWebsocketConsumer to receive it, then send it to Amazon Transcribe and attempt to reply to the browser with the transcript in near real-time.I am able to send the transcription for a single blob, but couldn't send it in a streaming manner. I see the event handler is triggered only if the stream ends, which led me to think, maybe I am not using the API correctly.What I have tried so far, apart from the code below:Created a separate stream for each audio chunk. The error amazon_transcribe.exceptions.InternalFailureException: An internal error occurred. is thrown.Used asyncio.gather function to group stream.input_stream.send_audio_event function and handler.handle_events function. The handler is not invoked.Used asyncio.create_task(handler.handle_events) to create a non-blocking task for the handler. The handler is not invoked and also it didn't wait for 15 seconds. The task got completed immediately.stream_client = TranscribeStreamingClient(region="us-west-2")class AWSTranscriptHandler(TranscriptResultStreamHandler):    def __init__(self, transcript_result_stream):        self.channel_layer = get_channel_layer()        self.channel_name = "test"        super().__init__(transcript_result_stream)    async def handle_transcript_event(self, transcript_event: TranscriptEvent):        results = transcript_event.transcript.results        for result in results:            if not (result.is_partial):                for alt in result.alternatives:                    await self.channel_layer.send(                        self.channel_name,                        {"type": "send_transcript", "message": alt.transcript},                    )class ChatConsumer(AsyncWebsocketConsumer):    async def connect(self):        await self.accept()        await self.send(            text_data=json.dumps(                {"type": "connection_established", "channel_name": self.channel_name}            )        )        self.stream = await stream_client.start_stream_transcription(            language_code="en-US",            media_sample_rate_hz=48000,            media_encoding="ogg-opus",        )        self.handler = AWSTranscriptHandler(self.stream.output_stream)    async def disconnect(self, close_code):        await self.stream.input_stream.end_stream()    async def receive(self, text_data=None, bytes_data=None):        if bytes_data:            await self.stream.input_stream.send_audio_event(audio_chunk=bytes_data)            await self.handler.handle_events()    async def send_transcript(self, event):        await self.send(            text_data=json.dumps({"type": "transcript", "message": event["message"]})        )FollowComment"


Question: LimitExceededException when creating a Global Accelerator on brand new account

I just created a new AWS account under our existing organization and am attempting to deploy our infrastructure to it (using Pulumi).  It includes a Global Accelerator which is unable to be provisioned and throws the following exception:LimitExceededException: Limit Exceeded for the number of Accelerators that can be created for account XXXXXXXXXThe account is empty and has 0 accelerators associated with it.  I also tried to create one from the AWS web UI and experienced the same result.FollowComment

Answer: "1It's worth checking your account's service quotas (https://us-east-1.console.aws.amazon.com/servicequotas/home/services/globalaccelerator/quotas) to see if any of those are set to 0, and request an increase if necessary.CommentShareEdanswered 10 months ago"


Question: In command prompt they showing connection error.

"Is this compulsory that we have to use linux for EC2 connection, because in window its showing connection error..can anybody help to do this task?I am new on AWS so i didn't understand.FollowComment"

Answer: "0Hello,We can connect to EC2 instance from windows source machine. Please check this link for guidance.Hope this helps.CommentShareSUPPORT ENGINEERVinay_Sanswered 2 months ago"


Question: Textract AnalyseId not working / available on Lambda

"Is it possible that the new Textract AnalyzeID function is not available in Lambda yet? In both Lambda and SAM I get this error:Textract.analyzeID is not a functionconst AWS = require('aws-sdk');const Textract = new AWS.Textract();exports.handler = async (event, context) => {      Textract.analyzeID(params, function(err, data) {        ...      });};When I run the similar code in an empty project locally it works, making me wondering if its something to do with the Lambda/SAM container?FollowComment"

Answer: "1Accepted AnswerI suspect that you are using an older version of the AWS SDK that doesn't have the function you're trying to use. Make sure your Lambda function is packaged with an appropriate AWS SDK version. You can find some instructions here: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-layer-aws-sdk-latest-version/CommentShareEXPERTBrettski-AWSanswered a year agopauldulong a year agoAah excellent, that solved the problem! Thank you so muchShare"


Question: All Instances are hanging after creation

"Hi everyoneI have a really weird issue here.When I create instances (from AMI) they get hung once I log in via SSH; sometimes even immediately once I try to connect.After the hanging, I can't connect to that instance anymore, even after reboot or stop.I tried Ubuntu 18.4LTS, Debian 9, Amazon Linux all are the same hanging. I also tried with bigger RAM and Storage but no luck.Troubleshooting I've done:Different Datacenter (N. Virginia, N. California)Different OSDifferent RAM(up to 4 GB) and Disk Storage(up to 30 GB)Different Security Groups (even any to any)Different IAM Role (Administrator Role)Reviewed the gateway and the routesPlease adviseThanks!FollowComment"

Answer: "0Accepted AnswerHello,I would like to inform you that the issue with your account has now been resolved. You should now be able to connect to your instances. Please let me know if you experience any further issues.Regards,awstomasCommentShareAWS-User-3892169answered 4 years ago0Hello,I am sorry to hear about the issue that you are experiencing.Please note that the reason why you are unable to connect to your instance is caused by an administrative issue with your account. I have created an internal ticket for our customer service team to remedy this situation. I will inform you as soon as I receive an update.Regards,awstomasCommentShareAWS-User-3892169answered 4 years ago0Thank you guys,It's finally workingCommentShareKINGSABRIanswered 4 years ago"


Question: Identity Center use across Regions

"HiI have a single AWS Account, with EC2 instances deployed in multiple Regions. I just set up IAM Identity Center in one Region using Azure AD as my Directory source. I set up one click login access to my EC2 instances via Fleet Manager as per the following post - https://aws.amazon.com/blogs/security/how-to-enable-secure-seamless-single-sign-on-to-amazon-ec2-windows-instances-with-aws-sso/.This process worked in the home Region, and I was able to connect to an EC2 instance with my Azure AD Credentials. When I switch to another Region and click on Connect with Remote Desktop, I'm not provided with the SSO option.While I can execute other functions in other Regions with my SSO account, is there a limitation in Fleet Manager where it doesn't recognise my SSO credentials?ThanksMickFollowComment"

Answer: "1Hello Mick,The behaviour you have pointed out is an expected one. As you are trying to connect RDP into instances using Fleet manager on one-click authentication with SSO, the authentication will be done in the region where IAM Identity Center(SSO) is enabled. Therefore, you will not be able to view the option to select authentication type as SSO in the regions where SSO was not enabled. On this note, please allow me to remind you that IAM Identity Center is only supported in one AWS region at a time.[1]If you're authenticated to the AWS Management Console using AWS IAM Identity Center, Fleet Manager integrates with AWS IAM Identity Center so you can connect to your instances without providing additional credentials. Fleet Manager supports AWS IAM Identity Center authenticated RDP connections in the same AWS Region where you enabled AWS IAM Identity Center[2]Reference[1] AWS IAM Identity Center (successor to AWS Single Sign-On) Region availability - IAM Identity Center Region data - https://docs.aws.amazon.com/singlesignon/latest/userguide/regions.html#region-data[2] Connect using Remote Desktop - https://docs.aws.amazon.com/systems-manager/latest/userguide/fleet-rdp.htmlCommentShareDivya_Aanswered 6 months ago"


Question: Is it possible to give all privileges of a specific user to a specific user in aws rds oracle?

There are users who want to keep accessing tables created by a specific user in the future.So is it possible to give full access to a specific user's table to another specific user?And can you give the user admin privileges?FollowComment

Answer: "0Accepted AnswerHi there,Object privilege management is the same either on-premise oracle or RDS oracle.There is no such privilege that grants access to tables of another user (especially that will be created in future). Rather we can create a role and grant the required object privileges to the role and then grant the role to the another user, but for future tables you need to grant the previlies once there are created.So we better way is to use a dynamic query to grant the privileges to the role and run/schedule the execution after objects are created on first user. You can refer belowlink for dynamic query:https://community.oracle.com/tech/developers/discussion/966693/grant-access-to-schemaYes granting DBA role/any other admin role will work BUT it is NOT recommended process in terms of security.Thank you!CommentShareJosephanswered 8 months ago0If the user has been granted the object privilege with the WITH GRANT OPTION, the user can give the object priviledge to another user.If the user has been granted the system priviledges CREATE USER, ALTER USER and DROP USER with the WITH ADMIN, the user can give the system priviledges to another user.Details are in the following document :https://docs.oracle.com/cd/F19136_01/sqlrf/GRANT.html#GUID-20B4E2C0-A7F8-4BC8-A5E8-BE61BDC41AC3And also you can use the DBA role on RDS for Oracle. The user granted the DBA role can give any priviledges to another user, but some priviledges are limited from DBA role. The details of the limitations are in the following.https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Oracle.Concepts.limitations.html#Oracle.Concepts.dba-limitationsCommentShareAWS-User-5505188answered 8 months ago"


Question: Api to add new subdomain to amplify app .

"There are some issues with the createDomainAssociation API that is provided by the AWS SDK amplify.I am not able to associate the subdomain to amplify the app using createDomainAssociation API.I Found another API to associate subdomain updateDomainAssociation.For using updateDomainAssociation API, there are concerns that this may negatively affect the already associated subdomains if the request fails.Question: If there is an API that we can use to add the subdomain to the amplify app.Question: When using updateDomainAssociation, if we add already associated subdomains along with a new subdomain in the updateDomainAssociation API request, does this API re-associate all subdomains, or does this only update/add/remove whatever is there in the request.FollowComment"


Question: Traffic doesnt flow whe using ALB as a target of NLB

"I have a client's requirement to use static public ip's for our applications, instead the ALB dynamic ip's.For this escenario we have an NLB with EIP as static ip and is usingALB as a target of NLB. For testing purposes, we create a EC2 with a simple apache, NLB and ALB has listeners on port 80 and everything its working, we can use de DNS name in a browser and we have the apache Welcome page.But, if we tried to use https its not working. For this last escenario we have the following configuration:NLB:* Listener on TCP 443* ALB as a target group for NLB* AZ's matches between NLB and ALB* NLB is internet-facingALB* Listener on https* Certificate configuration using aws certificate manager* Rule to target by default Apaches EC2* ALB is internalWe test the ALB with telnet and curl from another EC2 instance and is responding with the two commands, but when we used the NLB dns name, nothing happend.We used the same subnets an AZ from the escenario with the port 80, what are we missing?Best regardsFollowComment"

Answer: "0The next troubleshooting step I would try is to assign the target group directly to the NLB - skip the ALB just for the time being to ensure that the NLB is operating correctly.If that still doesn't work, please create a support ticket - from this forum we have very limited (close to no) visibility of customer environments but the support team can help.Another question has been asked which references this question and provides some other context; my other answer my therefore be better: https://repost.aws/questions/QU94k_s2LyRd-hJRloOlzAwQCommentShareEXPERTBrettski-AWSanswered a year ago"


Question: Intermittent application certificate problem

"I run the nagios nrpe server on a Lightsail Debian instance. This needs a custom port opening, TCP 5666.Last year, without warning communications between the nrpe client on a remote host and the nrpe server running on the Lightsail instance started to fail.Further investigation revealed that the remote client and the Lightsail instance were still talking on the custom port, but both sides of the connection were now reporting that the SSL handshake was failing.When I disabled SSL on the nrpe server running on the Lightsail instance everything worked fine.No updates had been made to either the remote client machine or the Lightsail instance that the nrpe server is running on. The same remote client machine could still successfully connect using SSL to multiple other nrpe servers not hosted on the AWS network and local SSL connections to the nrpe server on Lightsail also worked fine.In summary the only failed connections to the nrpe server on Lightsail were from remote clients using SSL.The next day when I checked, everything had mysteriously started working again with successful remote SSL connections to the nrpe server on the Lightsail instance.Today, exactly the same thing started happening again.Is this some wierdness on the AWS network? Has anyone else experienced this?Edited by: EIC-BTC on May 25, 2021 9:01 AMFollowComment"

Answer: 0It turns out that the problem was on my providers network. Not AWS..CommentShareEIC-BTCanswered 2 years ago


Question: Using 'aws:RequestTag' in policy not working as expected

"We've created an IAM policy designed to allow creation of S3 buckets from the Service Catalog, but deny from S3 console unless a specific tag is used.For that intent we build the policy as follow:{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": "s3:CreateBucket",            "Resource": "arn:aws:s3:::*",            "Condition": {                "StringEquals": {                    "aws:RequestTag/TrustTag": "TrustValue"                }            }        },        {            "Sid": "VisualEditor1",            "Effect": "Allow",            "Action": [                "s3:Get*",                "s3:List*",                "s3:Put*"            ],            "Resource": [                "arn:aws:s3:::*",                "arn:aws:s3:::*/*"            ]        }    ]}The policy is not working as it blocks the bucket creation even if the tag is included.Is there another way to do this?FollowComment"

Answer: "1Accepted AnswerThe following document shows that there is no "RequestTag" in the "CreateBucket" condition key, so I don't think it is possible to restrict creation by tag.https://docs.aws.amazon.com/ja_jp/service-authorization/latest/reference/list_amazons3.htmlAs far as I know, there is no other way.CommentShareEXPERTRiku_Kobayashianswered 21 days agoMartin Trejo 20 days agoThank you! We changed the condition to this:{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "VisualEditor0",            "Effect": "Allow",            "Action": "s3:CreateBucket",            "Resource": "arn:aws:s3:::*",            "Condition": {                "Bool": {                    "aws:ViaAWSService": "true"                }            }        },        {            "Sid": "VisualEditor1",            "Effect": "Allow",            "Action": [                "s3:Get*",                "s3:List*",                "s3:Put*"            ],            "Resource": [                "arn:aws:s3:::*",                "arn:aws:s3:::*/*"            ]        }    ]}Share"


Question: Delete key material is greyedout

"I have one customer managed key, I try to delete, but can only schedule deletion for 7 days later, and the delete key material option is greyed out. how to immediately delete this key? otherwise, i have to pay for 7days for KMS usage.I also find there is one AWS managed key,"AWS/secretmanager" how to delete this one as well? thanks.FollowComment"

Answer: "1Hi, Good questionBecause it is destructive and potentially dangerous to delete a KMS key, AWS KMS requires you to set a waiting period of 7 – 30 days. The default waiting period is 30 days.As per https://docs.aws.amazon.com/kms/latest/developerguide/deleting-keys.htmlAbout the waiting periodHowever, the actual waiting period might be up to 24 hours longer than the one you scheduled. To get the actual date and time when the KMS key will be deleted, use the DescribeKey operation. Or in the AWS KMS console, on detail page for the KMS key, in the General configuration section, see the Scheduled deletion date. Be sure to note the time zone.During the waiting period, the KMS key status and key state is Pending deletion.* A KMS key pending deletion cannot be used in any cryptographic operations.* AWS KMS does not rotate the key material of KMS keys that are pending deletion.After the waiting period ends, AWS KMS deletes the KMS key, its aliases, and all related AWS KMS metadata.Use the waiting period to ensure that you don't need the KMS key now or in the future. You can configure an Amazon CloudWatch alarm to warn you if a person or application attempts to use the KMS key during the waiting period. To recover the KMS key, you can cancel key deletion before the waiting period ends. After the waiting period ends you cannot cancel key deletion, and AWS KMS deletes the KMS key.SriCommentShareSrianswered a year ago0In regards to the AWS Managed Keys, as they are managed by AWS for use with AWS services, you do not have the ability to delete them.   You are not charged for them either.CommentShareAllison_Panswered a year ago0The way to think about the waiting period is as a feature. Key deletion can impact all services and data associated with that key. Therefore, the delay allows you to protect yourself from accidental or malicious deletion, and undo the deletion within the wait period. As Allison mentioned, you are not charged for keys that are scheduled for deletion.CommentShareTushar Aanswered a year ago0Thanks for your question. Just wanted to add that the "delete key material" option is available only for KMS keys with imported key material. You can't delete key material from any other type of KMS key.CommentSharejuneblanswered a year ago"


Question: Grant roles to newly created oracle db user

"Hi all,Currently I´m doing some research on the aws rds oracle instance related to how the roles work here. According to what I have found i cannot simply add the DBA role to a recently created user for them to administer the instance with me. My question is, how can I grant roles to recently created users to have the same access as the master user that created the db or at least, similar to the db_owner role in other SQL instances.RegardsFollowComment"

Answer: "1Accepted AnswerWhen you create an Amazon RDS for Oracle database instance, the default master user is created and granted the maximum user permissions on the DB instance with some limitations.For RDS Oracle, this user comes with the below privileges and roles:1 - System Privileges: ALTER DATABASE LINK, ALTER PUBLIC DATABASE LINK, DROP ANY DIRECTORY, EXEMPT ACCESS POLICY, EXEMPT IDENTITY POLICY, GRANT ANY OBJECT PRIVILEGE, RESTRICTED SESSION, EXEMPT REDACTION POLICY2- Database Roles: AQ_ADMINISTRATOR_ROLE, AQ_USER_ROLE, CONNECT, CTXAPP, DBA, EXECUTE_CATALOG_ROLE, RECOVERY_CATALOG_OWNER, RESOURCE, SELECT_CATALOG_ROLEIf you wish to have another user with the same privileges, you can grant the above-mentioned privileges and roles the new user.SQL> GRANT ALTER DATABASE LINK, ALTER PUBLIC DATABASE LINK, DROP ANY DIRECTORY, EXEMPT ACCESS POLICY, EXEMPT IDENTITY POLICY, GRANT ANY OBJECT PRIVILEGE, RESTRICTED SESSION, EXEMPT REDACTION POLICY TO  newuser;Furthermore, if you wish to grant individual privileges to a user you can use "rdsadmin.rdsadmin_util.grant_sys_object" as follows:beginrdsadmin.rdsadmin_util.grant_sys_object(p_obj_name  => '<OBJECT_NAME>',p_grantee   => '<USER>',p_privilege => '<PRIVILAGE>');end;For details on master user privileges, please refer to the following documentations.[+]Master user account privileges - https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.MasterAccounts.html[+]Granting SELECT or EXECUTE privileges to SYS objectshttps://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.Oracle.CommonDBATasks.System.html#Appendix.Oracle.CommonDBATasks.TransferPrivileges[+]How do I manage user privileges and roles in my Amazon RDS for Oracle DB instance? - https://repost.aws/knowledge-center/rds-oracle-user-privileges-rolesCommentShareSUPPORT ENGINEERYASHanswered 2 months ago"


Question: Web server down

"Our web servers (three separate websites) keep crashing and our web developer thinks this is some kind of issue with the main control panel on Amazon, however, there are no issues with our account health. Any insight into what the issue could be would be greatly appreciated!FollowCommentbwhaley EXPERT3 months ago... you need to provide more detail if you want help. Nobody will be able to assist based on what you've stated so far.Share"

Answer: "0Hi there, i would highly recommend logging a support case to get AWS support to assist where they can -https://docs.aws.amazon.com/awssupport/latest/user/case-management.htmlCommentShareEXPERTGreg Vintonanswered 3 months ago0You need to go to the webser server log path. Check the files that were updated recently and ask your developer to check the Errors. It should have a clue there.CommentShareBiswaanswered 3 months ago"


Question: AWS setting how many long-polling concurrent requests Node.js can handle?

"I'm setting up a new application for long-polling messages with interval of 10 sec from AWS sqs. I've tried to test it. And after 50 users that waiting their requests latency start growing and reach 15 seconds and reached 30 second with 150 users. Is it something wrong with my code or aws/node have some type of setting for it?const port = process.env.PORT || 3001;const express = require('express');const app = express();const AWS = require('aws-sdk');AWS.config.update({region: 'eu-west-1'});const MD5 = function(d){<md5function>}const sleep = (waitTimeInMs) => new Promise(resolve => setTimeout(resolve, waitTimeInMs));const SQS = new AWS.SQS({  region: 'eu-west-1'});const LONG_POLL_TIMEOUT = 10;async function checkQueue(req, res) {  const {version, token} = req.params;  const auth = req.query.auth;  if (!isTokenValid(token, auth)) {    await sleep(LONG_POLL_TIMEOUT * 1000);    res.send()  } else {    getUpdateMessage(version, token, res);  }}function getUpdateMessage(version, token, res) {  const urlParams = {    QueueName: `_version-queue-${version}-${token}`  };  SQS.getQueueUrl(urlParams, (urlErr, urlData) => {    if (urlErr) {      res.status(204).send();    } else {      const messageParams = {        QueueUrl: urlData.QueueUrl,        WaitTimeSeconds: LONG_POLL_TIMEOUT,      };      SQS.receiveMessage(messageParams, (err, data) => {        if (err) {          res.status(204).send();        } else {          if (data.Messages) {            res.send(data.Messages[0].Body);            SQS.deleteMessage({              QueueUrl: urlData.QueueUrl,              ReceiptHandle: data.Messages[0].ReceiptHandle            }, (err1, data) => {              if (err1) {              }            });          } else {            res.send();          }        }      });    }  });}function isTokenValid(token, auth) {  // check against tokens for last 14 days  let dayNumber = Math.ceil(Date.now() / (24 * 3600 * 1000));  for (let i = 0; i < 14; i++) {    const stringToHash = `<string>`;    if (MD5(stringToHash) == auth) {      return true;    }    dayNumber--;  }  return false;}app.use(function(req, res, next) {  res.header("Access-Control-Allow-Origin", "*");  next();});app.get('/versions/:version/long_poll_updates/:token', function (req, res) {  checkQueue(req, res);});app.get('/check', function (req, res) {  res.send('I\'m ok!');});app.use((req, res) => {  res.status(404).send("Sorry, that route doesn't exist. Have a nice day :)");});app.listen(port, () => {  console.log('Server running at http://127.0.0.1:' + port + '/');});CPU Utilisation was less then 10 percent.FollowComment"

Answer: "0Here's the detailed documentation:https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/node-configuring-maxsockets.htmlQuote:When using the default of https, the SDK takes the maxSockets value from the globalAgent. If the maxSockets value is not defined or is Infinity, the SDK assumes a maxSockets value of 50.You are most likely observing this setting kicking in.CommentShareKubaanswered 4 years ago0Thank you, this setting solved the problem!CommentSharese0gaanswered 4 years ago"


Question: can NOT see Athena "data source connector" db in QuickSight

"I created a Athena data source connector, which can query dynamodb data successfully.However, when I tried to create a new dataset in QuickSight with Athena, it can NOT see my 'data source connector'? Do I miss anything? Thanks!FollowComment"

Answer: "1Thanks for the reply!** Select the "AWSLambdaRole" and click "Attach Policy" ** solved my problem.CommentShareJLanswered a year ago0Hi,to be able to use the connector you need to grant the QuickSight Role permission to invoke the AWS Lambda function used by the connector.Please, look at this blog post for further information.Also, for QuickSight related questions we have launched a fully dedicated community; in case you want to check there to see if there are further answer to your questions.hope this helps,CommentShareEXPERTFabrizio@AWSanswered a year agoFabrizio@AWS EXPERTa year agoI see you were able to fix this error and move forward, based on another question.Share"


Question: verify post-installation scripts run in AWS ParallelCluster

"I uploaded a shell script to install conda and a specific environment post-configure on my compute nodes in AWS ParalellCluster. Either there were errors in the installation script or ParallelCluster had issues executing the script.I didn't find any obvious answers in /var/log/cfn-init-cmd.log or /var/log/cfn-init.log. Below is the script that I was attempting to run:#!/bin/bash# script for setting up conda environment and installation of openMM on AWS ParallelCluster# this script is called in the configuration script for pcluster during creation of the virtual cluster# pull down miniconda and install itwget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh/bin/bash Miniconda3-latest-Linux-x86_64.sh -b -p /shared/miniconda3source ~/.bashrc# install mamba to speed up package search, download, and installation# use "-y" flag to allow for silent installationconda install -c conda-forge mamba -y# create the openMM environment by referencing the .yaml file in shared directorymamba env create -f /shared/miniconda/openmm8.yaml -yand below is my configuration script for generating the ParallelCluster:HeadNode:  InstanceType: c5.2xlarge  Networking:    SubnetId: subnet-xxxxxxxx    AdditionalSecurityGroups:      - sg-xxxxxxxx  Ssh:    KeyName: mertz_key  LocalStorage:    RootVolume:      VolumeType: gp3  Iam:    AdditionalIamPolicies:      - Policy: arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCoreScheduling:  Scheduler: slurm  SlurmQueues:    - Name: modbind      AllocationStrategy: lowest-price      ComputeResources:        - Name: modbind          Instances:            - InstanceType: g4dn.metal          MinCount: 0          MaxCount: 10          DisableSimultaneousMultithreading: true          Efa:            Enabled: true      Networking:        SubnetIds:          - subnet-xxxxxxxx        PlacementGroup:          Enabled: true        AdditionalSecurityGroups:          - sg-xxxxxxxx      ComputeSettings:        LocalStorage:          RootVolume:            VolumeType: gp3      CustomActions:        OnNodeConfigured:          Script: >-            s3://parallelcluster-10552b48cfa2e9aa-v1-do-not-delete/modbind-cluster-setup.sh    - Name: spot      AllocationStrategy: lowest-price      ComputeResources:        - Name: spot          Instances:            - InstanceType: g4dn.metal          MinCount: 0          MaxCount: 10          DisableSimultaneousMultithreading: true          Efa:            Enabled: true      ComputeSettings:        LocalStorage:          RootVolume:            VolumeType: gp3      CapacityType: SPOT      Networking:        SubnetIds:          - subnet-xxxxxxx        PlacementGroup:          Enabled: true        AdditionalSecurityGroups:          - sg-xxxxxxx      CustomActions:        OnNodeConfigured:          Script: >-            s3://parallelcluster-10552b48cfa2e9aa-v1-do-not-delete/modbind-cluster-setup.sh  SlurmSettings: {}Region: us-east-1Image:  Os: centos7DirectoryService:  DomainName: modulus.ad.com  DomainAddr: ldaps://modulus.ad.com  PasswordSecretArn: >-    arn:aws:secretsmanager:us-east-1:xxxxxxxxxxxx:secret:PasswordSecret-modulus-AD-xxxxxx  DomainReadOnlyUser: cn=ReadOnlyUser,ou=Users,ou=MODULUS,dc=modulus,dc=ad,dc=comSharedStorage:  - Name: Efs0    StorageType: Efs    MountDir: /shared    EfsSettings:      FileSystemId: fs-xxxxxxxxFollowCommentAshish_T SUPPORT ENGINEER2 months agoOther way to narrow down the issue of install script would be :Launch an parallel cluster without post install for a while. Once it is up, SSH into head node and run the scripts command manually as root user. If the commands runs successfully and exitcode is 0, we can be sure that there is no issue with commands.As there is nothing in cfn-init logs which can say anything about issue. In that case there can be resource configuration issue as well.i would request you to raise a support case with resource details. So this can be troubleshoot in right direction.Share"

Answer: "0Have you taken a look at /var/log/parallelcluster/clustermgtd on the HeadNode? While protected mode may not be your exact problem, this doc may help direct your debugging efforts.CommentShareChris Pollardanswered 2 months ago0Chris, thanks for the suggestion. I looked at clustermgtd and there is nothing that I can see that would indicate that the script was read during cluster configuration:2023-03-30 16:37:12,798 - [slurm_plugin.clustermgtd:main] - INFO - ClusterManager Startup2023-03-30 16:37:12,798 - [slurm_plugin.clustermgtd:_get_config] - INFO - Reading /etc/parallelcluster/slurm_plugin/parallelcluster_clustermgtd.conf2023-03-30 16:37:12,799 - [slurm_plugin.clustermgtd:set_config] - INFO - Applying new clustermgtd config: ClustermgtdConfig(_config=<configparser.ConfigParser object at 0x7f1f391953d0>, region='us-east-1', cluster_name='modbind-AD-modstore1', dynamodb_table='parallelcluster-slurm-modbind-AD-modstore1', head_node_private_ip='10.0.0.195', head_node_hostname='ip-10-0-0-195.ec2.internal', _boto3_retry=1, _boto3_config={'retries': {'max_attempts': 1, 'mode': 'standard'}}, loop_time=60, disable_all_cluster_management=False, heartbeat_file_path='/opt/slurm/etc/pcluster/.slurm_plugin/clustermgtd_heartbeat', boto3_config=<botocore.config.Config object at 0x7f1f33df3160>, logging_config='/opt/parallelcluster/pyenv/versions/3.9.16/envs/node_virtualenv/lib/python3.9/site-packages/slurm_plugin/logging/parallelcluster_clustermgtd_logging.conf', disable_ec2_health_check=False, disable_scheduled_event_health_check=False, health_check_timeout=180, health_check_timeout_after_slurmdstarttime=180, disable_all_health_checks=False, launch_max_batch_size=500, update_node_address=True, fleet_config={'modbind': {'modbind': {'Api': 'create-fleet', 'CapacityType': 'on-demand', 'AllocationStrategy': 'lowest-price', 'Instances': [{'InstanceType': 'g4dn.metal'}], 'Networking': {'SubnetIds': ['subnet-659e6f4a']}}}, 'spot': {'spot': {'Api': 'create-fleet', 'CapacityType': 'spot', 'AllocationStrategy': 'lowest-price', 'Instances': [{'InstanceType': 'g4dn.metal'}], 'Networking': {'SubnetIds': ['subnet-659e6f4a']}}}}, run_instances_overrides={}, create_fleet_overrides={}, terminate_max_batch_size=1000, node_replacement_timeout=1800, terminate_drain_nodes=True, terminate_down_nodes=True, orphaned_instance_timeout=120, protected_failure_count=10, insufficient_capacity_timeout=600.0, disable_nodes_on_insufficient_capacity=True, hosted_zone='Z06359702HT9JMZIE8TK6', dns_domain='modbind-ad-modstore1.pcluster.', use_private_hostname=False, compute_console_logging_enabled=True, compute_console_logging_max_sample_size=1, compute_console_wait_time=300, worker_pool_size=5, worker_pool_max_backlog=100)2023-03-30 16:37:14,333 - [botocore.credentials:load] - INFO - Found credentials from IAM Role: modbind-AD-modstore1-RoleHeadNode-1I8XZ48CF2G092023-03-30 16:37:14,904 - [slurm_plugin.clustermgtd:_get_config] - INFO - Reading /etc/parallelcluster/slurm_plugin/parallelcluster_clustermgtd.conf2023-03-30 16:37:14,989 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Managing cluster...2023-03-30 16:37:15,277 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Current compute fleet status: RUNNING......2023-03-30 16:47:15,331 - [slurm_plugin.clustermgtd:_get_config] - INFO - Reading /etc/parallelcluster/slurm_plugin/parallelcluster_clustermgtd.conf2023-03-30 16:47:15,333 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Managing cluster...2023-03-30 16:47:15,619 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Current compute fleet status: RUNNING2023-03-30 16:47:15,619 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Retrieving nodes info from the scheduler2023-03-30 16:47:20,720 - [slurm_plugin.clustermgtd:_get_ec2_instances] - INFO - Retrieving list of EC2 instances associated with the cluster2023-03-30 16:47:20,797 - [slurm_plugin.clustermgtd:_perform_health_check_actions] - INFO - Performing instance health check actions2023-03-30 16:47:20,797 - [slurm_plugin.clustermgtd:_maintain_nodes] - INFO - Performing node maintenance actions2023-03-30 16:47:20,797 - [slurm_plugin.clustermgtd:_maintain_nodes] - INFO - Following nodes are currently in replacement: (x0) []2023-03-30 16:47:20,798 - [slurm_plugin.clustermgtd:_terminate_orphaned_instances] - INFO - Checking for orphaned instanceAny other ideas?CommentShareblakemanswered 2 months ago0Other way to narrow down the issue of install script would be :Launch an parallel cluster without post install for a while. Once it is up, SSH into head node and run the scripts command manually as root user. If the commands runs successfully and exitcode is 0, we can be sure that there is no issue with commands.As there is nothing in cfn-init logs which can say anything about issue. In that case there can be resource configuration issue as well.i would request you to raise a support case with resource details. So this can be troubleshoot in right direction.CommentShareSUPPORT ENGINEERAshish_Tanswered 2 months ago"


Question: Sparse index utilisation without $exists

"https://docs.aws.amazon.com/documentdb/latest/developerguide/functional-differences.html#functional-differences.sparse-index is causing me problems migrating an existing MongoDB code-base that makes use of sparse indexes - some of the queries are now unindexed and therefore take minutes/hours rather than milliseconds.Is there a plan to fix this compatibility bug any time soon, or do I need to track down every query that relies on a sparse index and fix it?Thanks,TomFollowComment"

Answer: "0For now, the recommended action is to track the slow queries by enabling Profiling and update the queries to include $exists.CommentShareMihai Aanswered a year ago"


Question: Cloudfront cost by distribution

"I would like to know how can I get the billing of AW Cloudfront by distribution ?With the new billing console, I can't get the old CSV by ressource for the previous month.FollowComment"

Answer: "1Have a look at the Cost And Usage Report which allows you to publish your cost and billing data to an Amazon S3 bucket , which you can later query , using Athena .The following example query will return the CloudFront cost , by distribution , including usage (TBs) :select  "line_item_usage_type",  "line_item_resource_id" as distribution,  "product_from_location",  "product_to_location",  sum("line_item_usage_amount")/1024 as "Usage (TBs)",  sum("line_item_unblended_cost") as costfrom DATABASE.TABLEwhere  "line_item_product_code" = 'AmazonCloudFront'group by  "line_item_usage_type",  "line_item_resource_id",  "product_from_location",  "product_to_location"order by  sum("line_item_unblended_cost") desIn addition , you may look at the Query Library for additional queries that you could use (CloudFront or other services).All reports can be downloaded as a CSV file.CommentSharedotanpanswered 10 months ago0You can view this with a nice visualization that is also customizable by deploying the Cloud Intelligence Dashboards.Have a look at an example, click on the Data Transfer Summary tab, and scroll all the way to the bottom to see the CloudFront section where you can find visualizations by distribution.Click here to deploy the CIDs into your own account.CommentShareAaron Edellanswered 10 months ago0Take a look at Cost Explorer for this exercise.  Applying a unique "Name" tag to each CloudFront distribution, and activating that "Name" tag using these instructions would provide visibility into costs per CloudFront Distribution.  Once tagged, the cost data would only be moving forward, not retroactive.CommentSharegrelon-AWSanswered 10 months ago"


Question: The Assertion of the Response is not encrypted and the SP require it

We  are trying to configure as a Cloud application Centra Guardicore in AWS SSO and we are seeing the following error message:The assertion of the Response is not encrypted and the SP require itDoes AWS SSO send over all the assertions encrypted or is there a way to configure this to be sent over encrypted?FollowComment

Answer: "1Accepted AnswerUnfortunately, the current AWS implementation of SAML 2.0 federation does not support encrypted SAML assertions. However, the traffic between the customer's systems and AWS is transmitted over an encrypted (TLS) channel.I was able to see an existing feature request to add support encrypted SAML assertions which I have +1'd on your behalf. While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our What's New and Blog pages for any new feature announcements.CommentShareSUPPORT ENGINEERMichael_Hanswered a year ago"


Question: "Not able to load data using neptune bulk load- Credential should be scoped to a valid region, not 'us-east-1 error shows"

"When I Submit api call get initiate successfully with status 200 ok using this command.Command:awscurl -X POST --service neptune-db -H 'Content-Type: application/json' --region us-east-2 https://omdena-kyield-database-1.cluster-c4brigvg3m9m.us-east-2.neptune.amazonaws.com:8182/loader -d'{"source" : "s3://kyieldgraphdboutput/Unsaved/2022/12/13/4a873928-9910-47b0-85ca-de593ace4f4a.csv","format" : "csv","iamRoleArn" : "arn:aws:iam::959061167427:role/NeptuneLoadFroms3","region" : "us-east-2","failOnError" : "FALSE"}'Output:{"status" : "200 OK","payload" : {"loadId" : "b4627762-11c1-4aa8-8bbe-3fb70ef850ac"}}But when able to check Monitor the Loading Process using this command then I am able to get this error.Command:awscurl --service neptune-db  'https://omdena-kyield-database-1.cluster-c4brigvg3m9m.us-east-2.neptune.amazonaws.com:8182/loader/b4627762-11c1-4aa8-8bbe-3fb70ef850ac'Error:{"code":"AccessDeniedException","requestId":"e5a7a210-fd31-4263-90e7-c2154be1e00e","detailedMessage":"Credential should be scoped to a valid region, not 'us-east-1'. "}requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://*****:8182/loader/33aaf4f9-54b5-4f7b-8b39-b00cfb379397Can anyone please help me with this ?FollowComment"

Answer: 0On this command here:awscurl --service neptune-db 'https://omdena-kyield-database-1.cluster-c4brigvg3m9m.us-east-2.neptune.amazonaws.com:8182/loader/b4627762-11c1-4aa8-8bbe-3fb70ef850ac'You still need to add the appropriate region to sign the request:--region us-east-2So in total:awscurl --service neptune-db 'https://omdena-kyield-database-1.cluster-c4brigvg3m9m.us-east-2.neptune.amazonaws.com:8182/loader/b4627762-11c1-4aa8-8bbe-3fb70ef850ac' --region us-east-2CommentShareTaylor-AWSanswered 5 months ago


Question: AWS GLUE job changing from STANDARD to FLEX not working as expected

"Hi, I'm having an issue with the new FLEX feature.In our company, we are trying to save costs when running GLUE Jobs (we are validating our product fit). The same day that FLEX was released we tried it. Since then I'm not able to make it work. I thought that only checking that Flex checkbox would suffice but I think I'm doing something wrong. The jobs are now running as before (without that checkbox checked) and they are running 100% OK. Simply put, we are reading from an RDS SQL Server Table, doing basic ETL processes, and storing it in an S3 bucket in CSV format. Also, I don't think there's an issue with Job timeout since it is set to 60 minutes and the job takes bearly a couple of minutes to failThe failed job status shows:Glue version: 3.0Start-up time: 16 secondsExecution time: 6 minutes 25 secondsTimeout: 45 minutesWorker type: G.1XNumber of workers: 10Execution class: FLEXMax capacity: 10 DPUsThe success job status is the same but:Execution class: STANDARDIn the job monitor we read:An error occurred while calling o87.getDynamicFrame. Job 0 cancelled because SparkContext was shut down caused by threshold for executors failed after launch reached. Note: This run was executed with Flex execution. Check the logs if run failed due to executor termination.in cloudwatch logs part of the output error:An error occurred while calling o90.getDynamicFrame.\n: org.apache.spark.SparkException: Job 0 cancelled because SparkContext was shut down\n\tat org.apache.spark.scheduler.DAGScheduler.$anonfun$cleanUpAfterSchedulerStop$1(DAGScheduler.scala:1130)\n\tat org.apache.spark.scheduler.DAGScheduler.$anonfun$cleanUpAfterSchedulerStop$1$adapted(DAGScheduler.scala:1128)\n\tat scala.collection.mutable.HashSet.foreach(HashSet.scala:79)\n\tat org.apache.spark.scheduler.DAGScheduler.cleanUpAfterSchedulerStop(DAGScheduler.scala:1128)\n\tat org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onStop(DAGScheduler.scala:2703)\n\tat org.apache.spark.util.EventLoop.stop(EventLoop.scala:84)\n\tat org.apache.spark.scheduler.DAGScheduler.stop(DAGScheduler.scala:2603)\n\tat org.apache.spark.SparkContext.$anonfun$stop$12(SparkContext.scala:2111)\n\tat org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1419)\n\tat org.apache.spark.SparkContext.stop(SparkContext.scala:2111)\n\tat org.apache.spark.SparkContext.$anonfun$new$39(SparkContext.scala:681)\n\tat org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:214)\n\tat org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$2(ShutdownHookManager.scala:188)\n\tat scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)\n\tat org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1996)\n\tat org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$1(ShutdownHookManager.scala:188)\n\tat scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)\n\tat scala.util.Try$.apply(Try.scala:213)\n\tat org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:188)\n\tat org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:178)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:750)\n\tat org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:914)\n\tat org.apache.spark.SparkContext.runJob(SparkContext.scala:2238)\n\tat org.apache.spark.SparkContext.runJob(SparkContext.scala:2259)\n\tat org.apache.spark.SparkContext.runJob(SparkContext.scala:2278)\n\tat org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:477)\n\tat org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:430)\n\tat org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:47)\n\tat org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:3733)\n\tat org.apache.spark.sql.Dataset.$anonfun$head$1(Dataset.scala:2762)\n\tat org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3724)\n\tat org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)\n\tat org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:232)\n\tat org.apache.spark.sql.execution.SQLExecution$.executeQuery$1(SQLExecution.scala:110)\n\tat org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:135)\n\tat org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)\n\tat org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:232)\n\tat org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:135)\n\tat org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:253)\n\tat org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:134)\n\tat org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:772)\n\tat org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:68)\n\tat org.apache.spark.sql.Dataset.withAction(Dataset.scala:3722)\n\tat org.apache.spark.sql.Dataset.head(Dataset.scala:2762)\n\tat org.apache.spark.sql.Dataset.take(Dataset.scala:2969)\n\tat com.amazonaws.services.glue.JDBCDataSource.getLastRow(DataSource.scala:1089)\n\tat com.amazonaws.services.glue.JDBCDataSource.getJdbcJobBookmark(DataSource.scala:929)\n\tat com.amazonaws.services.glue.JDBCDataSource.getDynamicFrame(DataSource.scala:953)\n\tat com.amazonaws.services.glue.DataSource.getDynamicFrame(DataSource.scala:99)\n\tat com.amazonaws.services.glue.DataSource.getDynamicFrame$(DataSource.scala:99)\n\tat com.amazonaws.services.glue.SparkSQLDataSource.getDynamicFrame(DataSource.scala:714)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\n\tat py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)\n\tat py4j.Gateway.invoke(Gateway.java:282)\n\tat py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\n\tat py4j.commands.CallCommand.execute(CallCommand.java:79)\n\tat py4j.GatewayConnection.run(GatewayConnection.java:238)\n\tat java.lang.Thread.run(Thread.java:750)\n","Stack Trace":[{"Declaring Class":"get_return_value","Method Name":"format(target_id, \".\", name), value)","File Name":"/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py","Line Number":328},{"Declaring Class":"deco","Method Name":"return f(*a, **kw)","File Name":"/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py","Line Number":111},{"Declaring Class":"__call__","Method Name":"answer, self.gateway_client, self.target_id, self.name)","File Name":"/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py","Line Number":1305},{"Declaring Class":"getFrame","Method Name":"jframe = self._jsource.getDynamicFrame()","File Name":"/opt/amazon/lib/python3.6/site-packages/awsglue/data_source.py","Line Number":36},{"Declaring Class":"create_dynamic_frame_from_catalog","Method Name":"return source.getFrame(**kwargs)","File Name":"/opt/amazon/lib/python3.6/site-packages/awsglue/context.py","Line Number":185},{"Declaring Class":"from_catalog","Method Name":"return self._glue_context.create_dynamic_frame_from_catalog(db, table_name, redshift_tmp_dir, transformation_ctx, push_down_predicate, additional_options, catalog_id, **kwargs)","File Name":"/opt/amazon/lib/python3.6/site-packages/awsglAny help would be much appreciated,Agustin.FollowComment"

Answer: "0Hello Agustin,Have you retired the flex job and was it the same behaviour?If it fails every time, I believe it would be better if you could raise a support ticket. That would give better visibility on the resources and the Support can check through on what could be causing the issue.Based on the doc https://aws.amazon.com/blogs/big-data/introducing-aws-glue-flex-jobs-cost-savings-on-etl-workloads/, it is recommended that you use flex jobs for pre-prod, testing or not urgent tasks.Even if retry doesn’t work, please raise a support ticket. That would give a better visibility on the resources and cause of the issue.Thank you!CommentShareSUPPORT ENGINEERManish Kumar_Nanswered 9 months agoJorge Vidinha 2 months agoHello Manish , maybe you can help with the following question https://repost.aws/questions/QUHPzEaPxoTJuil56o2_SoSQ/glue-jupyter-job-execution-classApreciateThanksShare0We require jobrun id to check internal logs for a specific job run. To further help with this issue, could you please create a support ticket with jobrun id of both successful and failed run.CommentShareSachin_Kanswered 9 months ago"


Question: Need guidance on which GPU sagemaker instance is suitable for my huggingface model

"We want to use huggingface whisper large hindi model on amazon sagemaker instance, https://huggingface.co/vasista22/whisper-hindi-large-v2. The model requires atleast 16GB GPU Memory, and we want to use it along with other hugging face models so we might need additional gpu memory. We are facing Resource Limit Exceeded error. Please let us know which resource to use, and please increase my limit accordingly.FollowComment"

Answer: "0Hi,When deploying large models, the ideal situation is that you can fit the model on a single GPU. This is the best option with respect to performance as it eliminates the overhead of communication between GPU devices. For some models it is simply impossible to fit them on a single GPU due to model size. For other models, they may fit on a single GPU, but it may be more cost effective to partition the model across multiple cheaper GPUs.In your case, you may consider an instance like g4dn.xlarge if using for inference. Details for using GPU can refer official document.Hope it helps.CommentShareTina Qiananswered a month ago"


Question: Basic S3 Security for Web App

"I have an S3 bucket on US-East that is public - it hosts a Web site that includes HTML and Javascript functions that run in the browser.I want some of these Javascript functions to send data to a different S3 bucket that will accept this data (essentially CSV data or JSON) and store it. It's essentially custom analytics from the users actions.What I don't want is for any other person to be able to send data to that bucket except the application itself as hosted on the original S3 instance.Since the data will not be readable, just writeable my only concern is not theft of data (it's meaningless analytics) but instead purposeful pollution of data by malicious people.I'm unsure how to do this. In general I'm assuming the following:The source to my web app will be viewable as all source is and thus the address of the analytics bucket is exposed, and perhaps even passwords.  However, I'm also assuming that the receiving bucket itself is configured such that it properly rejects any effort to send data to it that's not coming from code hosted on the host S3 bucket. I also want to assume it's doing this in a way where easy header spoofs, etc. can't bypass the write access permissions on the S3 bucket.Any thoughts on where to read how to do this? Everything I've seen so far is not hitting the nail on the head such that I understand it and find it fairly infallible.FollowComment"

Answer: "2In order to secure S3 access from a frontend web application you really need some type of server-side authentication.  As you astutely point out, there's no way to really "hide" any credentials that might be used to authenticate your application and thereby place a limit who or what can write to your S3 bucket.  You can't even really use IP address because it's FE, and the IP address will just represent the computer where the browser is running.  Because you can't really establish a reliable identity (or principal if we're being technical) the only way to write to the bucket directly from your webapp is to make it public.  Once it's public, there's not really an effective way to limit "who" writes to it.So I think the solution is to make the bucket NOT public and limit "who" can access it by introducing server-side authentication.  One way to do this would be to require logins to your site with something like Cognito.  Then you'd create an API Gateway backed by a Lambda to accept the JSON and/or CSV payloads and write them to the S3 bucket.  You would lock down the S3 bucket so that only the lambda could write to it, and you'd add a Cognito authorizer to the API Gateway to make sure the data that's coming in is generated by an actual user on the website.The real sort of philosophical issue here is that it seems like you are not interested in the users, only in the website.  But you can't really authenticate a FE website automatically without exposing your credentials, so you need some form of user interaction.  An acceptable solution might be to piggy-back on user-logins via Cognito as described above.CommentShareMiguelUTanswered 8 months agoEXPERTkentradreviewed 8 months agoBenSawyerDev 8 months agoThis makes sense. You see this same problem with Google Analytics which is widely used on open frontend sites with no authentication -- people report polluted data that has made it into their GA Analytics logs. I wasn't sure if there was a way to otherwise lockdown the receiving bucket such that it wouldn't accept data that came from elsewhere but I suspect it's still too easy to spoof something where there isn't otherwise some sort of credentialed user.ShareMiguelUT 8 months agoYes -- FE is very difficult to secure it turns out!  If you found this answer helpful it would really benefit me if you'd mark it as "Accepted"!Share"


Question: Server Error During Password Recovery via Workspaces Windows Client

"Hello, I'm trying to recover a password for a workspace I created previously. I'm using Amazon Workspaces Windows Desktop client and the password recovery form falls into a loop of the "Server Error There was an error with your request. Please try again later." message. I'm pretty sure that my username is correct.The username contains uppercase letters. I also tried typing username with lowercase letters but it did not work.Restarting the workspace through the AWS web console did not resolve the issue.Please kindly assist me.Thanks,IsmetFollowComment"

Answer: "0Are you using Microsoft Active Directory that you manage for your WorkSpaces's enabled Directory? Likely, you'll need RSAT Active Directory Users and Computers to find the user and reset the password.CommentShareEXPERTStevieStetsanswered 10 months agoidumlupinar 10 months agoDo we have to provision an EC2 to instance? Or can we use our home computer, join a local vm to the domain and use it instead? (BTW, yes we use AD Standard edition and directory/workspace both located in London region)Share"


Question: EC2 Session manager is not loading from inside AWS workspaces.

"EC2 Session manager is not loading for linux instances when trying to access it from inside AWS workspaces station. It shows blank black screen although I'm able to connect to EC2 session manager from my personal system. Please guide.Also, RDP connection for windows is working fine from AWS workspaces.FollowComment"


Question: Message does not get delivered for a very long time

"I recently observed a message that seemed to be stuck in a FIFO queue for 30+ minutes before being delivered for the very first time. The queue was not under heavy load, and the visibility timeout of the message should have been on the order of seconds.If there are any AWS admins around, happy to provide the queue name and message ID for further investigation.FollowComment"

Answer: "0After some more digging into our application logs, it turned out that this particular message was stuck behind another message with the same Message Group ID, that was taking a very long time to process.CommentSharetesruanswered 4 years ago"


Question: Redshift Accepts IGNORE NULLS outside of expression

We've noticed placing INGORE NULLS inside and outside the expressions for FIRST_VALUE and LAST_VALUE yields the same semantic result. According to the docs it should be inside the expression. Is this expected behaviour or should Redshift only accept IGNORE NULLS inside the expression.SELECT LAST_VALUE(venuename) IGNORE NULLS    OVER (        PARTITION BY venuestate        ORDER BY venueseats        ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW    ) AS last_valFROM venueyields same result asSELECT LAST_VALUE(venuename IGNORE NULLS)    OVER (        PARTITION BY venuestate        ORDER BY venueseats        ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW    ) AS last_valFROM venueFollowComment

Answer: 0Indeed as per documentation it should be placed inside but since it is working either-ways then it would be really nice if you can use the Provide feedback link on the bottom of documentation page and help us make it better.CommentShareMilindOkeanswered a year ago


Question: Can AWS CloudFormation template access s3 from any region other than its current region?

"I currently use AWS CloudFormation template to create resources in my users' account. Now, my cloudformation template is in an s3 bucket in region us-east-1 and my use case is that my if my users choose any region other than us-east-1 during creating the stack, then it is failing with the following error:"Error occurred while GetObject. S3 Error Code: AuthorizationHeaderMalformed. S3 Error Message: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-central-1'".Please help me with some solution here. Thanks.FollowComment"

Answer: "3Hi there,I understand that you want to create a CloudFormation stack using a template that is stored in your S3 bucket sitting in different region. Even though S3 is a global service which means you can be able to access its objects anywhere regardless of where it was created from. For example: if your S3 is created in us-east-1 you should be able to access its content even if you are in eu-central-1 given that you have necessary permissions to do so. However, things differ when you want to create a stack using a template stored in S3. To successfully create the ClodFormation stack, the template must be accessible to the user who is creating the stack and the stack must be created in the same region where the S3 bucket is. For more information on how to create a stack please refer to this documentation.I hope you find this information helpful.CommentShareCebianswered a year ago"


Question: TO MIGRATE TO AMAZON SES API VERSION 4

About a year and half ago i mistakenly unsubscribed my client email address from my email list. I realised this mistake later  and since all efforts to resolve this error have failed.  This text pop ups anytime i try contacting my client ( MIGRANT TO AMAZON SES API VERSION 4 ) Please anyone willing to help can contact me . I can send some screenshots in order to give you an IDEA of the problem.No problem at all about " TERMS AND CONDITIONS OF SERVICE "FollowComment


Question: how to version step functions for ML?

"Hi,Step Functions can be used to create ML workflows. What is the best practice to version the code creating those workflows? boto3 code in CodeCommit? Something else?CheersOlivierFollowComment"

Answer: "0Accepted AnswerA Step Functions state machine usually doesn't come alone and typically relies on other resources such as Lambda, EC2, DynamoDB, etc. You might want to package these dependent artifacts/resources altogether within a version otherwise you might have a state machine that doesn't fully work (eg, state machine version doesn't match Lambda version). I guess the simplest way to achieve this is to provision these resources together as code (eg, CDK or CloudFormation) and store them in a Git repo. You could then use Git tags for versioning.CommentShareMarcilio_Manswered 4 years ago"