Question: Has the model for Content Moderation in Stored video been updated?

"HelloI use Content moderation API on stored videos. I noticed that on a video that was run in August with min confidence of 60, some moderation labels were returned. The same video was run a few days ago and did not return any label until the min confidence passed was 40. The labels returned were also at different timestamps. I could not find any documentation on model improvement release for anything later than November 2021. Will you please let me know how the model improvements are released and what is a good place to look for up to date news?FollowComment"

Answer: "0Hello,Could you please confirm that you noticed the change by comparing the same video submitted in Aug 2022 and Oct 2022?Sharing more information would help us to look further into the issue, e.g.Video job idExamples of how the timestamp has been changed for labels in the detection resultsConfidence delta for labelsWe also have this release note for improving Rekognition Video content moderation on Jan 2022: https://aws.amazon.com/about-aws/whats-new/2022/01/amazon-rekognition-content-moderation-video/Thank you!CommentShareAWS-User-0631881answered 7 months ago0Hi.Yes, I can confirm that it was the same video. I stored the video in a database only if content moderation label was detected and not empty.Unfortunately, I did not save the resultant labels of Content moderation from Rekognition from August. I delete the queue after receiving the results too, so I think job id will not help. I have saved all the results from October, but I am not sure if that will help. Please let me know what else can be done.CommentShareAWS-User-3217907answered 7 months ago"


Question: How to handle NULL with AWS Glue bookmark

I have a table of 30GB in size I am running an etl with an aws-glue job that copies the table to an s3 bucket. I try to bookmark using the combination of a couple of columns as the bookmark key. Some of the columns have rows with null values. An error occurred while calling o97.getDynamicFrame. Incorrect DATETIME value: 'null'. I would like to ask if there is any way to give the does column a default value.The other alternative was moving the entire table without bookmark which I don't think is efficient.FollowCommentTasio EXPERT6 months agoWhere is that origin table stored?Share

Answer: "0Hello,For this use case, the bookmark keys are used at data consuming side, and per documented at [1], the create_dynamic_frame.from_catalog just takes the column names for the "jobBookmarkKeys".There is no option to give a default value to a column when its value is null.However, there is workaround to this.If your original table is stored in an RDBMS system, then you can add a computed column, which has same value as the original column, and has a default value where the original is null.Then in your glue job, you can use the computed column as part of bookmark keys.Hope it helps.=========Reference:[1] - https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.htmlCommentShareThi_Nanswered 6 months agoEXPERTTasioreviewed 6 months ago"


Question: Do Cloud Intelligence Dashboards Work on Orgs and Individual Accounts?

I am going through the Well-Architected Labs on Cloud Intelligence and wanted to find out if it is possible to use those on both an org with many accounts vs. a single account (and what would be different in the implementation.)FollowComment

Answer: "1It is possible to use CUR based  Cloud Intelligence Dashboards like CUDOS, KPI dashboard with individual accounts. Implementation will be the same the only difference is Cost and Usage Report (CUR) should be created in that single accountCommentShareYuriy Prykhodkoanswered 10 months ago0I'm using it under my organization account, not sure if it works with individual, but if I have to guess I would say yes.Cheers!-DCommentSharedominguezdanielanswered a year agorePost-User-3054073 a year agoWhere CID  should be deployed? Management or child account?ThanxShare"


Question: Error establishing SSH connection to your instance. Try again later.

Error establishing SSH connection to your instance. Try again later.FollowComment

Answer: "0Sumit Kumar,Could you kindly post snapshot of your errors?     Firstly, check your EC2 is in proper state, i.e. running and accessible.   Try to check the EC2 status in AWS console and connect to it using instance connect or via System Manager.If it is accessible from AWS console, try to explore the following reasons.Key pair mismatch.  You are using different keypair to login EC2.  Need use the same keypair when you specified during launched the EC2 instance.Security group Issue:    The default security group does not allow incoming SSH traffic from Internet. Check the security group settings and ensure that it allows incoming traffic on port 22 from your own IP address or add 0.0.0.0 for any IP address.Public DNS name or IP address.  Especially when you stop the instance for a while and start again,  the DNS name or IP may change.Network ACLs.   Checked whether there is network ACL configued to block the access in VPC.  You can enable VPC Flow to check the error messages in CloudWatch Logs.CommentSharecharliezhanganswered 4 months agoSaddam Hossain 4 months agoYour all requirements are fulfilled. But can't loginShare0Your all requirements are fulfilled. But can't loginCommentShareSaddam Hossainanswered 4 months ago"


Question: AWS Lambda not able to connect to public RDS instance

I have a public RDS instance running postgres. I can connect to it from my local host. The same function fails when I try to run it on lambda. The error I get is***could not translate host name <RDS_HOST_NAME> to address: Name or service not known\n"There is no VPC associated with the lambda function. I can connect using psq -h <RDS_HOST_NAME> from my machine. Could you please suggest on how to resolve this?FollowCommentAWS-ADolganov a year agoAre you using the public RDS endpoint as host name? (....<region>.rds.amazonaws.com)Sharetaskmonk a year agoYes..  it is <dbname>.<...>.ap-south-1.rds.amazonaws.comShare

Answer: "0I had a similar problem, which I solved with the following steps.I created a new RDS instance in the management console.At this time, I specified that a new VPC, subnet group, and security group be created as well.In this case, a rule was created with the source IP "90.149.145.179/32" as an inbound rule for the security group.In this state, I could still connect from the local PC, but not from Lambda.When I changed the source IP to 0.0.0.0.0/0, I was able to connect from Lambda.CommentSharemoritalousanswered a year ago"


Question: Error While querying in Athena query editor.

"Dears,I have a file in S3, and while querying got the following error:  Need your suggestion to get this fixed.**Qry **:SELECT * FROM "de-dup-db"."input_data" limit 10;**Error **:SYNTAX_ERROR: line 1:8: Column 'name' cannot be resolvedThis query ran against the "de-dup-db" database, unless qualified by the query. Please post the error message on our forum  or contact customer support  with Query Id: c7dde2fc-69be-4814-831f-71448486d4cbRegards,KeshavaFollowComment"

Answer: "0Can you execute SHOW CREATE TABLE de-dup-db.input_data; to validate that the column is called name and that there are no additional characters or encodings?This error is typically caused when the column does not exist.  If you think it should exist, there may be a misspelling or extra character due to encoding.CommentShareEXPERTChris_Ganswered a year agoAWS-User-4000301 a year agoI Executed the above script, got an error as : "Queries of this type are not supported"Following are the columns:name, id, age, salaryShareAWS-User-4000301 a year agoAlso these tables are created via AWS Glue, hence data source should be AwsDataCatalog.ShareChris_G EXPERTa year agoIn the Athena Query Editor, can you click the 3 dots next to the table and click "Generate Table DDL"?ShareAWS-User-4000301 a year agoEven that's not coming up Chris. Error thrown out. Gave all the relevant data lake permissions on databse, tables as well as on the columns. Am observing this for the first time.Share"


Question: How to control per user per account permissions with IAM identity center?

"I am struggling with IAM Identity center.I want to make sure user Y can only assume power user role when accessing account Z.It is not clear to me how I can achieve that when all permissions sets are assigned on an account level and not a user level.I have the following permissions sets assigned to an accountThe console says that I can assigned permissions to a groupBut when I start assigning permissions sets, they are assigned to ACCOUNTS only. So there is no way to say user X can only be PowerUser but not Administrator when accessing the account YHere is the stack overflow questions (that doesn't have an answer)https://stackoverflow.com/questions/74417061/how-to-manage-user-roles-with-aws-iam-identity-centerFollowComment"

Answer: "0Hi, you should understand the two core components of the AWS IAM Identity Center service.Core ComponentsPermission SetA permission set is a template you create and maintain that defines a collection of one or more IAM policies. Permission sets simplify the assignment of AWS account access for users and groups in your organization.You can think that a permission set is a reusable role with proper permissions, which can be used in several AWS accounts in the same AWS Organization.Account AssignmentAn account assignment is the task of assigning a permission set for a specific AWS account to multiple users or groups.AnswerYou can create an account assignment for the PowerUser permission set of the AWS account to user X.ReferencesPermission setsAccount AssignmentCommentShareEXPERTposquit0answered 7 months ago"


Question: Redshift UNLOAD questions

"I’ve checked the documentation and searched the Internet (thought my GooFu is weak here ) but I can’t find an answer to my customer’s questions below – is there some information somewhere I’m missing?, alternatively can you shed light on their questions? - These are questions around the UNLOAD statement:  When using the UNLOAD, how are various column types converted to strings? Specifically how are Booleans, dates, and timestamps (with and without timezones) converted?  Any gotchas around UNLOAD encryption (AES256-GCM) with third-party decryption libraries/tools? Do we have sample code/tool usage?  What are the line endings for manifest files? Does the final line have one?  Is there a resource listing ways in which the UNLOAD statement may fail?Any insights will be gratefully received.FollowComment"

Answer: "0Accepted AnswerPlease see below answers to your questions:Since unload uses SELECT statement to get data, the data type conversions are like what happen in normal SELECT output shown on screen. Customer can perform type casting in SELECT statement used with UNLOAD for conversions needed.As per Redshift documentation[1] UNLOAD does not support Amazon S3 server-side encryption with a customer-supplied key (SSE-C).Manifest file created by UNLOAD is a JSON file, so JSON format is used and JSON notations are used for beginning and end of records in it.Refer Redshift usage notes[1] has some of failure conditions in UNLOAD.Apart from above mentioned points you are recommended to check the UNLOAD documentation[3] for more details.[1] http://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD.html[2] http://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD.html#unload-usage-notes[3] http://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD_command_examples.htmlCommentShareAWS-User-2213912answered 6 years ago"


Question: Weather attributes used in Amazon Forecast

"Amazon Forecast can use Amazons own weather information when creating predictors (if you provide geo coordinates or zip codes). I'd like to know what weather attributes are specifically considered. For example, does it use humidity and percent cloud cover? I can't find a list of what weather attributes are considered so I'm not sure if I should be providing them myself in a related dataset.Thank you!FollowComment"

Answer: "0In my experience (which is limited, but matches what I've heard from others in the domain), raw weather signals need careful feature engineering to produce usefully-predictive inputs for human behaviour forecasting (like retail sales, etc).The Amazon Forecast Weather Index uses built-in, proprietary, featurization to tackle this effectively on your behalf - optimized across a range of use cases. As you found, the exact choices are not currently documented.So my suggestion would be to start out with just the Weather Index enabled and not worry about bringing or choosing custom weather attributes.If you have weather data available and find reason to think some particular weather feature important to your domain isn't being captured well enough (or just want to try your favourite weather feature out) - then can experiment with adding your feature(s) to explore the impact on model accuracy. The model explainability (feature importance) scores can also help you get an idea of how strongly your proposed features are driving the model.CommentShareEXPERTAlex_Tanswered 4 months ago"


Question: Unable to delete IAM Role

"I am trying to delete some bunch of IAM Roles based on conditions like LastUsedDate > 90days, LastCreateDate>90 days etc. In this process some IAM Roles could not be deleted because of : "Cannot delete entity, must remove roles from instance profile first". When I try to get "listInstanceProfilesForRole", I dont get Instance Profiles, I am getting an empty array. If so then why am I not able to delete IAM Role. Please suggest.FollowComment"

Answer: "0Hi,From the question I understand that you are trying to delete an IAM role but are getting an error that states "Cannot delete entity, must remove roles from instance profile first". When you try to run the command “list-instance-profiles-for-role” you do not get any instance profiles in the output.After testing in my account I was not able to replicate this issue. When running the command “list-instance-profiles-for-role” I was able to find the instance profile associated to the IAM role. I then ran the command “remove-role-from-instance-profile” and once that was completed I ran the “delete-role” command with no errors. In order to better troubleshoot this issue I would recommend creating a support case with IAM and providing the specific role name that is experiencing the issue.I hope you have a great rest of your day!CommentShareSUPPORT ENGINEERPatrick_Vanswered a year ago"


Question: SES sending templated email: how to get result.

I am trying to find out how to receive the results of a templated email merge.  We are recording every email sent for tracking purposes and need to have the subject and body AFTER the template merge occurs.FollowComment

Answer: "0Hi, thank you for reaching out. I wanted to confirm my understanding with regards to your question.Can you please confirm if you are trying to merge SES templates and wish to understand what the merged template looks like?Additionally, feel free to reach out to our Support with a couple of email Message IDs for review. This will help us get more clarity into the template used.For more details on SES Email Templates feel free to refer to the following guide,[+] https://docs.aws.amazon.com/ses/latest/dg/send-personalized-email-api.htmlCommentShareSUPPORT ENGINEERRohit_SCanswered 5 months ago0Yes, we are trying to get access to the subject and body of the email post merge.  In this fashion we can not only see what was sent but also build "send again" functionality.CommentSharerePost-User-4161349answered 5 months ago"


Question: IP Metrics for SES dedicated IPs from Microsoft SNDS are missing in CloudWatch

"I have multiple dedicated IPs in my SES setup and sending works good so far. I'd like to analyse the Microsoft SNDS reputation data that should be available according to the Developer Guide (https://docs.aws.amazon.com/ses/latest/dg/snds-metrics-dedicated-ips.html).Looking into CloudWatch under Metrics > All Metrics and selecting SES only gives me the option to select Account Metrics. I guess this is where the IP Metrics mentioned in the guide should be available.Is there any additional setup needed to enable the IP Metrics in CloudWatch for SES?FollowCommentmadlen_lf 7 months agoHi taifoon, did you ever find out why it wasn't working? Currently facing the same issue :(Share"


Question: Best practices for S3 File Storage Gateway for storing the backups

"IHAP that would like to use S3 File Storage Gateway for storing the backups, and map S3 bucket as the local share. Is there any best practices around this? Main objective is to protect the traffic, and securely map this to the on-premises/local computer as local share.FollowComment"

Answer: "1Accepted AnswerHi there,Refer to the following blog with detailed instructions.We recommend to use end points over Direct Connect to address any latency. You will see detailed reference architectures below.  Let me know if you have any follow up questions.https://aws.amazon.com/blogs/architecture/connect-amazon-s3-file-gateway-using-aws-privatelink-for-amazon-s3/CommentShareSreenathanswered a month agoEXPERTJonathan_Dreviewed a month ago1Some of the best practices are:Make sure you deploy your storage gateway closer to your client systems on which you will mounting the file shareHave enough network bandwidth between your storage gateway and AWS Endpoints for both data & control plane communicationIf you have direct connect we highly recommend to use VPC endpoints for Storage Gateway & S3 (for S3 file type gateways) or have virtual public interface created on your direct connect. Refer the link: https://docs.aws.amazon.com/filegateway/latest/files3/using-dx.htmlIf you are creating SMB file shares and for AD user authentication, make sure storage gateway is joined into Domain using Active Directory servers which are closer to Storage GatewayAllocate root & cache disks from high performance datastore disks for optimum performanceBefore deploying, ensure that network & firewall port requirements are in place. Please see here: https://docs.aws.amazon.com/filegateway/latest/files3/Resource_Ports.htmlWe do not recommend having multiple sources of data write to S3 bucket prefix where the file share is configured and storage gateway uploading the dataFor all other best practices, refer here: https://docs.aws.amazon.com/filegateway/latest/files3/best-practices.htmlCommentShareRRP_AWSanswered a month ago"


Question: Glue job s3 file not found exception

"Hi, I'm trying to run a simple glue job to convert a set of csv files to parquet. I'm seeing the following error in the error logs when running the job.Caused by: java.io.FileNotFoundException: No such file or directory 's3://.../*.csv'The glue script was entirely generated by aws and was not modified by me. So why am I getting this error?Thanks,DavidEdited by: davidwbr on Oct 24, 2017 10:18 AMFollowComment"

Answer: 0It turned out to be an IAM s3 permissions problem.CommentSharedavidwbranswered 6 years ago0Same error on object of public bucket !!CommentShareTusharPanswered 5 years ago


Question: AWS Redshift Maintenance Announcement (January 9th - March 24th 2022)

"Major Version(01/19/2022)We will be patching your Amazon Redshift clusters during your system maintenance window in the coming weeks. The timing of the patch will depend on your region and maintenance window settings. You can view or change your maintenance window settings from the AWS Management Console. After your cluster has been patched, the new cluster version will be Build 1.0.34934. Please contact us at redshift-feedback@amazon.com if you have any questions.This version includes the following new features and improvements:• Amazon Redshift: The database users that are automatically created when using GetClusterCredentials API to get temporary credentials will now be created with "IAM:" prefix in the database.• Amazon Redshift: User names from this version are treated as case sensitive when surrounded by double quotes irrespective of the value of the configuration parameter "enable_case_sensitive_identifier".• Amazon Redshift: Customers can use ST_Intersection to calculate the intersection of two geometries.• Amazon Redshift: Automatically discovering spatial reference system id (SRID) during COPY of shapefiles when .prj file next to .shp file exists.• Amazon Redshift: Customers can combine two sketches that appear in separate columns into one.• Amazon Redshift: Added support for AUTO table property for distribution style in CTAS.• Amazon Redshift: Customers can use St_GeoHash function to return the geohash value of an input point.• Amazon Redshift: Amazon Redshift ML now supports unsupervised training with K-Means clustering.• Amazon Redshift: Added support for converting interleaved sort keys to compound sort keys or no sort key with Alter Sortkey command.Additionally, the following fixes are included:• With the performance improvement for the numeric datatype, this resolves a sig11 issue with all customers using drivers that send numeric to Redshift server in binary mode like .Net driver.Minor Versions(02/07/2022)We will be patching your Amazon Redshift clusters during your system maintenance window in the coming weeks. The timing of the patch will depend on your region and maintenance window settings. You can view or change your maintenance window settings from the AWS Management Console. After your cluster has been patched, the new cluster version will be Build 1.0.35480. Please contact us at redshift-feedback@amazon.com if you have any questions.In addition to what is included in the major version, this version includes the following new features and improvements:• Amazon Redshift: Public Preview of Redshift Streaming Ingestion for Kinesis Data Streams (KDS) pulls data from a KDS stream into a Redshift Materialized View (MV) in near real-time with high throughput. Please note: Preview features are not to be used to run production workloads. To get started see the Redshift documentation.Additionally, the following fixes are included:• Fixed a rare situation where with Materialized View auto refresh enabled, external functions cause Redshift cluster instability.(02/15/2022)We will be patching your Amazon Redshift clusters during your system maintenance window in the coming weeks. The timing of the patch will depend on your region and maintenance window settings. You can view or change your maintenance window settings from the AWS Management Console. After your cluster has been patched, the new cluster version will be Build 1.0.35649. Please contact us at redshift-feedback@amazon.com if you have any questions.In addition to what is included in the major version, this version includes the following new features and improvements:• Miscellaneous fixes(03/03/2022)We will be patching your Amazon Redshift clusters during your system maintenance window in the coming weeks. The timing of the patch will depend on your region and maintenance window settings. You can view or change your maintenance window settings from the AWS Management Console. After your cluster has been patched, the new cluster version will be Build 1.0.36224. Please contact us at redshift-feedback@amazon.com if you have any questions.In addition to what is included in the major version, this version includes the following new features and improvements:• Miscellaneous fixes(03/03/2022)We will be patching your Amazon Redshift clusters during your system maintenance window in the coming weeks. The timing of the patch will depend on your region and maintenance window settings. You can view or change your maintenance window settings from the AWS Management Console. After your cluster has been patched, the new cluster version will be Build 1.0.36905. Please contact us at redshift-feedback@amazon.com if you have any questions.In addition to what is included in the major version, this version includes the following new features and improvements:• Miscellaneous fixesFollowComment"

Answer: "0If looking for different release notes, please, refer to the answer to the  question Where to find AWS Release Notes?List to other maintenance posts (they can be searched with the phrase "AWS Redshift Maintenance Announcement") below:AWS Redshift Maintenance Announcement (March 2nd - April 4th 2022)CommentShareEXPERTFabrizio@AWSanswered a year ago"


Question: NVME disk charges

"Hi All,I am using i3.2xlarge instances. I want to know about cost for nvme disk that are available by default with i3 type of instance for high performance of IO. If anyone knows please let me know.FollowComment"

Answer: 0Are you able to use this to check ?https://calculator.aws/#/createCalculator/EC2CommentShareRajesh Sanswered a year ago


Question: insert dataframe to an ICE BERG table in pyspark script

"I wrote the following code in a pyspark shell of Glue to create a table as an ICE BERG and insert a pandas DataFrame into it. The code executes successfully but the output is empty (there is no data in the ice berg table when run query in Athena ). There is no issue and error when I run this  parquet_dyf.errorsCount(). Can anyone help me in this regard?sql_query = f"""CREATE TABLE {table_name}({col_name_types})PARTITIONED BY (partition_0)LOCATION '{s3_location_dh}{table_name}'TBLPROPERTIES ('table_type' = 'ICEBERG', 'format'='parquet','write_target_data_file_size_bytes' = '536870912' )"""get_query_results(executeQuery(sql_query,dhSouce,outputConfig))    # insert the dataframe into the Iceberg table    if not df.empty:        s3_path = f"{s3_location_dh}{table_name}/data/{partition_date}.parquet"        wr.s3.to_parquet(df=df, path=s3_path)        parquet_dyf = glueContext.create_dynamic_frame_from_options('s3',{'path':[s3_path]}, format='parquet')        glueContext.write_dynamic_frame.from_catalog(frame=parquet_dyf,database=database_name,table_name=table_name)FollowCommentGonzalo Herreros a month agoHow about parquet_dyf.count(), does it give you a non zero number? I guess the write command completed without error?Share"

Answer: "0Hi,It seems like iceberg table supports only few frameworks which includes spark, hive , flink etc.. click here for more details.I tried different ways to write the pandas data frame to iceberg table, but it creates a different parquet file which isn't a part of iceberg table, so when I query it in Athena it won't show me the results for that iceberg table.However, with spark data frame it works fine, I am able to add the partitions into iceberg table. I would suggest you to convert your pandas data frame to spark data frame before writing it to iceberg tables.sparkDF=spark.createDataFrame(pandasDF) sparkDF.show()To append a dataframe to an Iceberg table, use append:sparkDF.writeTo("prod.db.table").append() click here for more details.In case if there is a dependency on using only python programming language for data manipulation, then you can think of using PyIceberg:PyIceberg is a Python implementation for accessing Iceberg tables, without the need of a JVM. You can easily perform your queries in python here..When you are running a glue job in glue console, then you can follow the below glue script to write to iceberg table, using glue catalog.-You can read it directly from s3 and then can load the data frame to the cataloged iceberg table pointing to an s3 path (I created the table in Athena, then used glue job to insert data).Athena DDL:CREATE TABLE iceberg_db.iceberg_table (  col1 string,   col2 string,   col3 string,   col4 int, PARTITIONED BY (`col3`)LOCATION 's3://iceberg-bucket/target/'TBLPROPERTIES (  'table_type'='iceberg',  'format'='parquet',  'write_target_data_file_size_bytes'='536870912');Make sure to add the necessary conf parameters in the glue job, here in AWS doc..glue script:import sysfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom awsglue.context import GlueContextfrom pyspark.sql import SparkSession, Row, DataFrameimport pandas as pdfrom awsglue.dynamicframe import DynamicFrameCollectionfrom awsglue.dynamicframe import DynamicFramefrom awsglue.job import Job  sc = SparkContext.getOrCreate()glueContext = GlueContext(sc)spark = glueContext.spark_sessionjob = Job(glueContext)#reading into pandas dataframe    pandasDF=pd.read_csv("s3://bucket-namea/folder/file.csv")print(pandasDF)#converting to spark dataframesparkDF=spark.createDataFrame(pandasDF) sparkDF.show()#write dataframe to iceberg tableglueContext.write_data_frame.from_catalog(    frame=dataframe,    database="iceberg_db",    table_name="iceberg_table"    )job.commit()-Now if you want to manually create a pandas dataframe and then load it to iceberg table in s3, using catalog.import sysfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom awsglue.context import GlueContextfrom pyspark.sql import SparkSession, Row, DataFrameimport pandas as pdfrom awsglue.dynamicframe import DynamicFrameCollectionfrom awsglue.dynamicframe import DynamicFramefrom awsglue.job import Job  sc = SparkContext.getOrCreate()glueContext = GlueContext(sc)spark = glueContext.spark_sessionjob = Job(glueContext)students = {    'col1':['value1','value2'],    'col2' :['value1', 'value2'],    'col3':['value1','value2'],    'col4':['value1','value2']}pandasDF = pd.DataFrame(students)print(pandasDF)#Create Spark DataFrame from PandassparkDF=spark.createDataFrame(pandasDF) sparkDF.show()#write dataframe to iceberg tableglueContext.write_data_frame.from_catalog(    frame=sparkDF,    database="iceberg_db",    table_name="iceberg_table"    )job.commit()CommentSharerePost-User-6395508answered a month ago"


Question: How to latest version  Id of object stored in s3 in cloudformation/serverless?

How to latest version  Id of object stored in s3 in cloudformation/serverless? I need this to set configuration version of aws:appconfig:deployment in my serverless file. I don't want to hardcover version id.FollowComment

Answer: 0You may want to look into AWS::CloudFormation::CustomResource.  This will allow you to execute a Lambda function to find the latest version of an object in S3.CommentShareEXPERTChris_Ganswered 9 months ago


Question: Cost reduction Tips in Shield Advanced

How do i reduce my data transfer out cost in AWS shield Advanced?Any heads up to follow?FollowComment

Answer: "0Hi,All of the Data Transfer Out pricing for resources protected by AWS Shield Advanced are explained here - https://aws.amazon.com/shield/pricing/If you can provide more detail about your workload, it would be helpful in terms of offering advice.CommentSharemicahanswered 5 months ago0Hi,You'll need to run an estimation on the data transfer cost to understand what are the resource consuming the most and benefits of using Shield Advanced.Use this reference to do the same - https://aws.amazon.com/blogs/aws-cloud-financial-management/estimate-your-aws-waf-and-aws-shield-advanced-cost/Once  you know how to estimate the cost to protect your resources with AWS Shield Advanced, you will be able to balance more efficiently between the extra protection and services brought by this new technology and the cost incurred by this feature.CommentShareAWS-CSinghanswered 4 months ago"


Question: Greengrass v2 how to subscribe MQTT message from IoT Core

"Hi,Publishing works.  IoT Core receives MQTT message from this program.  But I can't get to receive MQTT message from IoT Core.I have read below document and forum.  The exact sample program in the document.  It finishes the program right away.  And doesn't wait to receive message from IoT core since they are not synchronized.  I saw that I have to use queue.  But I can't get to work.  Could you help me?https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-iot-core-mqtt.htmlhttps://forums.aws.amazon.com/thread.jspa?threadID=334561&tstart=0I also tried that I made two ipc_client for publish and ipc_client2 to subscribe just in case.------------------------------Publish works, Subscribe doesn't----------------------------import queueimport osimport jsonimport datetimeimport timeimport randomimport awsiot.greengrasscoreipc.client as clientfrom awsiot.eventstreamrpc import Connection, LifecycleHandler, MessageAmendmentfrom awscrt.io import (ClientBootstrap,DefaultHostResolver,EventLoopGroup,SocketDomain,SocketOptions,)from awsiot.greengrasscoreipc.model import (IoTCoreMessage,QOS,SubscribeToIoTCoreRequest,PublishToIoTCoreRequest)class IPCUtils:def connect(self):elg = EventLoopGroup()resolver = DefaultHostResolver(elg)bootstrap = ClientBootstrap(elg, resolver)socket_options = SocketOptions()socket_options.domain = SocketDomain.Localamender = MessageAmendment.create_static_authtoken_amender(os.getenv("SVCUID"))hostname = os.getenv("AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT")print(hostname)connection = Connection(host_name=hostname,port=8033,bootstrap=bootstrap,socket_options=socket_options,connect_message_amender=amender,)self.lifecycle_handler = LifecycleHandler()connect_future = connection.connect(self.lifecycle_handler)TIMEOUT = 10connect_future.result(TIMEOUT)return connectionipc_utils = IPCUtils()connection = ipc_utils.connect()ipc_client = client.GreengrassCoreIPCClient(connection)class StreamHandler(client.SubscribeToIoTCoreStreamHandler):def init(self):super().init()def on_stream_event(self, event: IoTCoreMessage) -> None:      message = str(event.message.payload, "utf-8")      print("MESSAGE RECEIVED: ")      print(message)      queue.put(message)  def on_stream_error(self, error: Exception) -> bool:      # Handle error.      print('Error ---!')      return True  def on_stream_closed(self) -> None:      pass  dt=datetime.datetime.now().isoformat(timespec='seconds')message = {"deviceid":"ggc", "timestamp": dt , "Temperature":random.randint(20,37)}request = PublishToIoTCoreRequest()request.topic_name = "ggc/topic"request.payload = json.dumps(message).encode('utf-8')request.qos = QOS.AT_LEAST_ONCEoperation = ipc_client.new_publish_to_iot_core()operation.activate(request)future = operation.get_response()TIMEOUT=10future.result(TIMEOUT)queue = queue.Queue()request = SubscribeToIoTCoreRequest()request.topic_name = "ggc/#"request.qos = QOS.AT_LEAST_ONCEhandler = StreamHandler()operation = ipc_client.new_subscribe_to_iot_core(handler)future = operation.activate(request)future.result(TIMEOUT)i=0while queue.empty():i=i+1print(i)time.sleep(1)print(queue.get())queue.join()FollowComment"

Answer: "0Hi,Can you please provide the logs from both your component and Greengrass? You may also want to enable debug logging in the Nucleus by updating its logging config level through a deployment (look at the logging section of https://docs.aws.amazon.com/greengrass/v2/developerguide/greengrass-nucleus-component.html#greengrass-nucleus-component-configuration). With debug logs, you will see if we are performing the subscribe request.Also verify your IoT policy to make sure that it does allow you to subscribe to the topic which you're attempting to subscribe to.Cheers,Michael DombrowskiCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago0Hi,Are you trying to subscribe to the message that you publish here on the "ggc/topic" topic? If so, try swapping the order of the publish and subscribe requests, so that the program subscribes before it publishes.Best regards,Finn ThompsonCommentShareAWS-User-2970134answered 2 years ago0No, I want to exchange date between greengrass and IoT core/aws cloud using same ggc/topic.So that when I use "test" on IoT core, I can see what I am receiving from GGC and when I publish from IoT core, GGC should subscribe message from IoT core.GGC-->Publish_to_IoT core (ggc/topic)---->IoT copre  (first 10-15 seconds)GGC<--Subscribe_to_IoT core (ggc/topic)<---IoT core (After I made sure that IoT core test received MQTT message from GGC, then I manually publish MQTT message from IoT core test page.)So the order is okay with the way I wrote.   Publish first.   Then subscribe.Eventually I would like to publish and subscribe messages async repeatedly.Publish---every 1sec, GGC upload IoT data to IoT core-->Lambda-->DynamoDBSubscribe --- When needed, GGC receives some data or command from Lambda or program in EC2.My recipe is in case there might be some mistakes.ComponentConfiguration:DefaultConfiguration:accessControl:aws.greengrass.ipc.mqttproxy:HelloWorld:pubsub:1:policyDescription: "Allows access to publish to ggc/topic."operations:- "*"resources:- "ggc/topic"CommentSharejx2900answered 2 years ago0Can you please enable debug logging and provide the Greengrass logs?Thank you,Michael DombrowskiCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago0Thank you, Michael,I put "*" in recipe like below and  I am being able to publish message.   But I don't see this accessControl in effectiveConfig.yaml.  I don't know why.  Maybe I am deploying it locally.My greengrass.log updated over last night and it doesn't have the log when I ran the program yesterday.I will try to enable debug logging in the Nucleus on next Thusday and I will then post all proper logs.I just wanted to reply since I have something else to take care of next few days.ComponentConfiguration:DefaultConfiguration:accessControl:aws.greengrass.ipc.mqttproxy:HelloWorld:pubsub:1:policyDescription: "Allows access to publish to ggc/topic."operations:- "*"resources:- "ggc/topic"CommentSharejx2900answered 2 years ago0The effective config file is only written when Greengrass starts and stops, so if you haven't restarted Greengrass then it would not necessarily reflect the current state of the configuration. You can use the CLI or the local debug console to view the current configuration of components.Cheers,Michael DombrowskiCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago0Hi Michael,Thank you for the information.   I attached greengrass.log and HelloWorld.log.<1> I added aws.greengrass.Nucleus component with aws.greengrasl.Cli in a deployment to Core Device GreengrassCore2-177b2fdecec from aws cloud console.  I hope that this is one that you mentioned to enable Nucleus.<2> I stopped and restarted greengrass core using systemctl.   This added HelloWorld configuration into effectiveConfig.yaml.But it seems like that HelloWorld doesn't subscribe but HelloWorld can publishes with the same topic "gcc/topic".How do I retrieve the message?  Is it in def on_stream_event like below?class StreamHandler(client.SubscribeToIoTCoreStreamHandler)::def on_stream_event(self, event: IoTCoreMessage) -> None:message = str(event.message.payload, "utf-8")How can I retrieve message using queue from a class StreamHandler?  I tried below and didn't work.class StreamHandler(client.SubscribeToIoTCoreStreamHandler):def init(self):super().init()self.q = queue.Queue() # <------ def on_stream_event(self, event: IoTCoreMessage) -> None:        message = str(event.message.payload, "utf-8")        self.queue.put(message)   # <------  ipc_client2 = client.GreengrassCoreIPCClient(connection)request2 = SubscribeToIoTCoreRequest()request2.topic_name = "ggc/topic"request2.qos = QOS.AT_LEAST_ONCEhandler = StreamHandler()operation2 = ipc_client2.new_subscribe_to_iot_core(handler)future2 = operation2.activate(request2)future2.result(TIMEOUT)i=0while handler.q.empty():  # <------i=i+1print(i)time.sleep(1)print(handler.q.get())   # <------CommentSharejx2900answered 2 years ago0Hi,Your reply does not contain the logs, only the effectiveConfig file.Yes when a message is published to the topic which you have successfully subscribe to the code which you put in the on_stream_event handler will be called.Please review the example code provided in https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-iot-core-mqtt.html#ipc-operation-subscribetoiotcore-examples and use the StreamHandler from this code example exactly as it is shown.What is ipc_client2 in your code? You should not open multiple connections to Greengrass. Use a single connection for all operations.Edited by: MichaelDombrowski-AWS on Feb 25, 2021 10:07 PMCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago0I added logs this time.  But when I attached effectiveConfig.txt, I saw that the icon of log file changed from image of clip to image of file.  This might have caused the problem. Anyway, here are the logs.Okay, I will delete on ipc_client2 and have just one.It looks like I subscribed once.HelloWorld.log LineLine 4:  This is message the I published from HelloWorld.py.But I got stream_error at line 5.And I am publishing different messages multiple times from IoT Core with the same topic name "ggc/topic".  This is what I want to subscribe at ggc from IoT core.   But I am not able to subscribe message from IoT core.Edited by: jx2900 on Feb 25, 2021 10:48 PMI tried the sample code. But it ends too quickly before subscribe.  Someone asked the same question and you mentioned about using queue and while. So I am trying to use the queue and while.CommentSharejx2900answered 2 years ago0Everything seems fine. You should print what the actual error is, not just "Error!". The function is provided with the error as an input.You should also change how you are reading from the queue. Do not check if the queue is empty or not. Simply call queue.get(). That call will block until there is something to be read out of the queue.Please also upgrade your Python dependency on awsiotsdk to version 1.5.7.Your code also mixes "q" and "queue" please correct these as parts of the code are referencing different variables which may or may not exist.Edited by: MichaelDombrowski-AWS on Feb 25, 2021 10:54 PMCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago0Thank you, Michael,Greengrass core is dead again.  So I will have to re-install it to get the result.To upgrade awsiotsdk to 1.5.7, is the below okay in recipe?"Manifests": [{"Lifecycle": {"install": {"Timeout": 1000,"Script": "pip3 install awsiotsdk==1.5.7"I corrected q and queue.  Sorry, I mixed them up while I was trying different things with a lot of unresolved questions.I will reinstall ggc and get the actual error and report back to you.Edited by: jx2900 on Feb 26, 2021 6:52 AMI was able to re-install ggc and up and running.But I don't get any message any more.  No error either.   Stream_handler doesn't get error nor message anymore.  IoT core gets message from HelloWorld.py.   What am I doing wrong?awsiotsdk is now 1.5.7.CommentSharejx2900answered 2 years ago0Since I do not see any logs from your component, please use this guide to disable Python's output buffering: https://docs.aws.amazon.com/greengrass/v2/developerguide/troubleshooting.html#python-component-no-log-outputCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago0Hi,Please find the attached files of sample recipe and artifact that achieves what you're trying to do. Note that the code provided is only a sample and needs refactoring.This code publishes message to the cloud only once but logs the subscribed messages infinitely from the queue since I tried to keep it similar to your earlier code.To test this, go to AWS IoT Console  Test->MQTT Test client -> Publish to a topic -> topic = ggc/topic and message-> "Any message" . You should find the message sent in PubSub.logSample output in the /greengrass/v2/logs/PubSub.log2021-02-26T18:08:48.794Z [INFO] (Copier) Subscribe: stdout. ['{\n  "message": "Hello from AWS IoT console"\n}']. {scriptName=services.PubSub.lifecycle.Run.script, serviceName=PubSub, currentState=RUNNING}2021-02-26T18:09:04.863Z [INFO] (Copier) Subscribe: stdout. ['{\n  "message": "Hello from Greengrass"\n}']. {scriptName=services.PubSub.lifecycle.Run.script, serviceName=PubSub, currentState=RUNNING}2021-02-26T18:09:20.344Z [INFO] (Copier) Subscribe: stdout. ['{\n  "message": "Hello again!"\n}']. {scriptName=services.PubSub.lifecycle.Run.script, serviceName=PubSub, currentState=RUNNING}Thanks,SaranyaEdited by: saranya-aws on Feb 26, 2021 10:30 AMEdited by: saranya-aws on Feb 26, 2021 10:41 AMCommentShareSaranyaanswered 2 years ago0Thank you very much for the sample program, Saranya,It finally worked!   I even did copy and pasted to HelloWord component which I had already deployed locally with its configuration.  And this worked.  So the recipe was okay.I guess that there was only problem in my program which was using print(queue.get()) ---- Using while and print, HelloWorld.log didn't log stdout from print.  When I replaced your  logger.info(queue.get()) to print(queue.get()) like below.  Then it didn't print the subscribed message to the HelloWorld.log.#logger.info(queue.get())print(queue.get())   <-------------------thisHope these are my last questions.Why doesn't print work?   You are using stdout for logger and print is stdout.  So they should both work.I want to publish data every 1sec.   And at the same time, I want to subscribe message without missing to subscribe.  How can I do that?  Should I make two operations under same ipc_utils and run them with two different threads?Note: I am using Raspy Linux.  So I had to delete two lines from the recipe or it failed to deploy.  Otherwise, it is beautiful to see that ggc is subscribing.architecture: amd64Name: 64-bit x86_64 - LinuxCommentSharejx2900answered 2 years ago0Please read https://docs.aws.amazon.com/greengrass/v2/developerguide/troubleshooting.html#python-component-no-log-outputYou do not need to use multiple threads, however you can choose to use multiple threads. If you just rewrite the while loop you can have it publish something every second in there. Or you can go ahead and create a separate thread to handle the publishing.CommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago0Thank you, Michael,I've read all troubleshooting sections but couldn't find this page in Japanese.  In fact, when first I clicked the URL, it jumped to a Japanese page of troubleshooting.  The page didn't even have a word "python".   I refreshed the cache but the result was same in Japanese.  But when I changed the language to English, the proper page appeared.I used the below and it worked!print(handler.q.get(),file=sys.stdout, flush=True)Thank you for your explanation.  Now I think that I understand how the class object works for publish and subscribe.Thank you very much, Michael and Saranya!CommentSharejx2900answered 2 years ago0Hi.  Thank you for posting this example.  This is something I want to do as well.  The part I'm interested in is using the cert that is already installed on the gg core device to authenticate to the IoT Core and listen to MQTT messages.  It appears this is accomplished by means of a "MessageAmmendment" (right ?).  Is this something that is possible using the java API ?  I couldn't find any documentation about a class with that name.CommentSharewilliamrmayanswered 2 years ago0Hi,For Java you should really use the Greengrass IPC SDK, see https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-iot-core-mqtt.html#ipc-operation-publishtoiotcore for information about publishing and subscribing with MQTT through Greengrass.Cheers,MichaelCommentShareEXPERTMichaelDombrowski-AWSanswered 2 years ago"


Question: DNS input for owner verification GoDaddy

"Hi,I have to verify the ownership of my domainname on godaddy so that i can activate my emailadress.They are asking me to put in a TXT name and value TXT so that they can verify me as the owner. where is this TXT space so that I can input the info.ThanksFollowComment"

Answer: "0It sounds like you are hosting your domain on GoDaddy, and need to add a TXT record there in order to verify ownership.In that situation, please refer to the GoDaddy documentation for adding a TXT record to your GoDaddy hosted domain [1].If I misunderstood your question, and you are hosting the domain in Amazon Route53, please refer to the Route53 documentation for adding a TXT record as a simple record in Route53 [2].If this answer helps, please consider accepting the answer to help others with the same issue in the future[1] https://www.godaddy.com/help/add-a-txt-record-19232[2] https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-values-basic.htmlCommentShareGreg_Hanswered 2 months ago"


Question: Inconsistent "The Lambda Function Returned An Error."

"I have an AWS Lambda (node.js) that returns a value, in this case, an NPI number.  Per the AWS documentation (https://docs.aws.amazon.com/connect/latest/adminguide/connect-lambda-functions.html#verify-function), I am returning a JSON flat object:function successResponse(npi, callback) {const npiResponse = {NPI: npi};console.log(`Response back to client: ${JSON.stringify(npiResponse)}`);callback(null, npiResponse);}If I look at CloudWatch log for this response, I see:2020-03-27T16:45:03.026Zafe0c304-bac7-4a03-8692-81b57fb9082bINFOResponse back to client: {    "NPI": 1986799264}However, in the AWS Connect log, I see:{    "Results": "The Lambda Function Returned An Error.",    "ContactId": "ca7db64a-7284-409d-80bf-db44ffe0ae2f",    "ContactFlowId": "arn:aws:connect:us-east-1:611631033586:instance/ffb670d1-4977-42e6-a6f5-af43bec29887/contact-flow/73f1fc14-f26d-4067-aaed-f15982226dff",    "ContactFlowModuleType": "InvokeExternalResource",    "Timestamp": "2020-03-27T16:44:59.355Z",    "ExternalResults": {        "callCenterStatus": "Open"    },    "Parameters": {        "FunctionArn": "arn:aws:lambda:us-east-1:611631033586:function:provider-npi-dev",        "Parameters": {            "NPI": "1986799264"        },        "TimeLimit": "3000"    }}The response is rather simple, but I inconsistently get an Lambda error. Thoughts on what I am doing wrong?FollowComment"

Answer: "0The issue was that the Lambda entry within the contact flow has its own timeout value that supersedes the timeout value of the Lambda being called. I was suffering a timeout error when my execution time exceded the default timeout value of 3 seconds.  Unfortunately, the AWS Connect log will not tell you that it was a timeout issue (or anything of significance), just that the Lamda invocation suffered an error.CommentSharePerryHoekstraanswered 3 years ago"


Question: Which connection method when using SageMaker Notebook through VPC Interface Endpoint?

"Hi, I see in this page of documentation https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-interface-endpoint.html that:"You can connect to your notebook instance from your VPC through an interface endpoint in your Virtual Private Cloud (VPC) instead of connecting over the internet. When you use a VPC interface endpoint, communication between your VPC and the notebook instance is conducted entirely and securely within the AWS network."How would customer interact on their laptop with the UI of a notebook instance sitting in a VPC?FollowComment"

Answer: "0Accepted AnswerIf you are trying to access from within VPC, you'll have a direct connection. Otherwise, you'll need a configuration in place, such as Amazon VPN or AWS Direct Connect, to connect to your notebooks. Here is the blog post where we tried to explain how to set up AWS PrivateLink for Amazon SageMaker notebooks: https://aws.amazon.com/blogs/machine-learning/direct-access-to-amazon-sagemaker-notebooks-from-amazon-vpc-by-using-an-aws-privatelink-endpoint/CommentShareAWS-User-7081172answered 5 years ago"


Question: AWS CloudFormation CloudWatch Alarm Issue

"I'm creating a stack for CW alarm for mem_percent_used disk utilization metrics. It worked when I tried creating an alarm manually. It doesn't work after I created an alarm using CF. I have noticed that ami id, device, and fstype are missing in my CF script. How do I supply these values? Also, if I have 50 instances can I have single alarm to monitor the disk usage metrics?FollowCommentrePost-User-5062687 a year agoany pointers?Share"

Answer: "0ami id, device, and fstype are missing in my CF scriptThose values are the 'dimensions' of the metric: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#DimensionA metric is a unique combination of:NamespaceMetricNameDimension(s) (optional)Unit (optional)"Optional" here means that not all metrics will have them, but the alarm configuration has to exactly match the metric configuration.  So if a metric has no unit and 3 dimensions; the alarm must have no unit and the exact same 3 dimensionsI have 50 instances can I have single alarm to monitor the disk usage metrics?You can't create a single alarm to track all 50 individual instances; but you can aggregation_dimensions to the CWAgent config.  This makes it so you'll have 51 metrics.  Each instance will push 2 datapoints at a timeTo its unique instance metric (where its instanceID is one of the dimensions)To a shared metric with the aggregation_dimensionsIf these instances are in an ASG you can add something like this to the 'metrics' block of the config file:        "aggregation_dimensions": [            [                "AutoScalingGroupName"            ]        ],https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-common-scenarios.html#CloudWatch-Agent-aggregating-metricsYou can then make an alarm based on the 'min' or 'max' statistics, so that if any individual instances datapoint within that shared metric is above/below your threshold the alarm will trigger.  There isn't any way to disaggrigate the datapoints at that point to see which instance caused it, so you can use something like a search expression or the metric explorer to search through the 50 individual metrics and figure out which one is high.https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.htmlhttps://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.htmlCommentShareShahad_Canswered 10 months ago"


Question: Redirect domain old to new domain automatically

"How i can assign my old domain to new domain automatically?, can anybody help me please?.example case :example.com (old)example1.com(new)My Goal :if someone access web example.com, their will be assign to example1.com automatically.FollowComment"

Answer: "1If you are using an application load balancer, host-based routing is recommended.https://aws.amazon.com/premiumsupport/knowledge-center/elb-configure-host-based-routing-alb/?nc1=h_lsCommentSharehayao-kanswered a year agorePost-User-0218697 a year agoYesterday, I try for change rules on load balancer. And It work like you said, thank you for the hint :).Share1Another method would be to use s3 bucket redirection with Cloudfront, and an ACM certificate.You need to setup an s3 bucket named "example1.com" (the website that you want to redirect to). to do that, create the bucket => go to properties => scroll all the way down to "static website hosting" => set static site hosting to enabled, hosting type to redirect requests for an object, hostname to "example1.com", protocol to none.Create an ACM certificate for your domain. go to ACM => request a certificate, put in the FQDN of the website that you want to redirect to (example1.com). you will also need to validate the certificateCreate a cloudfront distribution. enter your origin (This will be the value that you get from s3 as the bucket website endpoint. ex: bucket-name.s3-website-us-west-2.amazonaws.com), change the viewer protocol policy to "Redirect HTTP to HTTPS", give an alternate CNAME record (www.example1.com, example.com), choose your SSL certificate that you created in ACM, create distribution.Go to your hosted zone in route 53, create an alias A record pointing to the cloudfront distribution that you created.here are some resources that I find helpful:https://simonecarletti.com/blog/2016/08/redirect-domain-https-amazon-cloudfront/https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-serve-static-website/CommentSharelandonbabayanswered a year ago0A simple Domain CNAME can accomplish this. A set up documentation from GoDaddy here. Respective DNS hosting Orgs/Domain Registrars will provided similar documentation.https://www.godaddy.com/help/add-a-cname-record-19236CommentShareAWS-User-2036647answered a year ago"


Question: Is cost of public AMI same as private?

"I have both, public and private AMIs stored in my account.Is there any difference in cost or they are charged under the same plan?FollowComment"

Answer: "0Accepted AnswerHi Shantanu Oak,The charges associated with AMIs do not depend on whether they are public or private. Whether it is public or private does not matter, it depends on the AMI’s storage category for the root device. AMIs are categorized as either backed by Amazon EBS or backed by instance store - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html#storage-for-the-root-device. With AMIs backed by instance store, you're charged for instance usage and storing your AMI in Amazon S3. With AMIs backed by Amazon EBS, you're charged for instance usage, EBS volume storage and usage, and storing your AMI as an EBS snapshot.You can also check out the contents of this doc for some more information: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ami-billing-info.htmlI hope this helps. Let me know if you have any questions.Warm regards,awsamyCommentShareamz99answered 2 years ago"


Question: System status check failed and reboot and stop are not working

System and instance status checks are failing since 6AM ESTI tried rebooting with no effect.  Then I tried stopping the instance which resulted in the Instance State changing to Stopping.  After 15+ minutes in Stopping state I tried to "Force stop instance" but it is still in the Stopping state.Instance ID: i-057392402dcfe0f21FollowComment

Answer: 1Accepted AnswerWell it finally move to the Stopped state about 2 hours after I initially tried to stop it.  Then it started with no issues (with a new IP).CommentShareAneranswered a year ago


Question: specify metric in static routes

"Hello,Would it be possible to specify metric in static routes?We have setup a Site-to-Site VPN with an external customer gateway.According to the documentation at https://docs.aws.amazon.com/vpn/latest/s2svpn/VPNRoutingTypes.html"On a Site-to-Site VPN connection, AWS selects one of the two redundant tunnels as the primary egress path. This selection may change at times, and we strongly recommend that you configure both tunnels for high availability, and allow asymmetric routing."Our amazon side has two tunnels with equal priority (metric).The remote (customer) side is a Fortigate firewall. Unfortunately, the customer has specified different metrics on their tunnels, making one on the tunnels act as "primary" and the other acting as "secondary".This discrepancy between the two sides, amazon being asymmetric vs. customer being fixed primary/secondary is a problem, we don't have traffic flowing smoothly in the tunnels.Is there an option on the amazon side to set a corresponding metric on the tunnels, making them equivalent to the customer's settings, i.e. primary/secondary ?Thanks,Sot.FollowComment"

Answer: "4For the purposes of this answer I've assumed you're running a VPC-attached Virtual Private Gateway.The documentation is correct - you can't specific a metric as the tunnel selection is within the VPN service and therefore either tunnel could be used.Another option is to use Transit Gateway; that allows for both tunnels on the same VPN connection to be used simultaneously (using ECMP) or for you to configure two different tunnels and advertise different routes and metrics for the tunnels using BGP. It is a more complex setup but it does give you greater flexibility.CommentShareEXPERTBrettski-AWSanswered a year agoEXPERTHernitoreviewed a year agostsimb a year agoHello @brettski, thank you very much for your anwer.Our customer doesn't want to use BGP, that's why we're using static routing.If we used a Transit Gateway, would we have an option to set the metric per tunnel and avoid ECMP (because the customer wants priorities, not equal cost paths) ?Share0Agree with what Brettski replied, that TGW gives your customer more flexibility and control.However, if static routing is the only option, your customer won't be able to select one tunnel over another.I challenge you to dive a bit deeper into the reasons why:BGP isn't an optionthere is the desire to privilege one tunnel over another.CommentSharenico_awsanswered a year ago"


Question: Alternative to monthly bill payment option in India

"AWS has discontinued the auto-bill payment option recently, due to a recent change in government guidelines in India.The manual tracking and monthly bill payment mode carry the risk of missing one or two bill payments due to any unpredictable circumstances.  As a result of which you run into the risk of losing your account and all the resources and data in it.I have been seeking an alternative solution in order to cover my account against such vulnerabilities. Any such solution already available to address this issue will be greatly helpful.The following alternatives come to my mind, not sure why anything like that is not provided by Amazon.A digital wallet to cover one/two months' billing expenseAuto clearance of minimum payment to keep an account activeAn upfront payment for a certain amount of storage, similar to what is already there for reserved compute resources.FollowComment"

Answer: "0Hello,Thanks for sharing your feedback regarding alternative bill payment options, we're always trying to improve our services and appreciate your input!I have passed your thoughts and ideas on to our Payments team for review.Regards,-- Ben G.CommentShareEXPERTAWS Support - Benanswered a year ago"


Question: How can I get out of the local console of the AWS storage gateway?

"I have implemented AWS storage gateway with linux kvm in on-promise.It was possible to access only locally, so when I directly connected to the console, aws transit gateway appliance mode, which is a local console, came out.I wanted to get out of that mode because there was something I wanted to set up in the Linux console.I looked it up and it said that I can exit by pressing 'ctrl' + ']', but '^]' appears and I can't get out.How can I get out of that mod?FollowComment"

Answer: "0Accepted AnswerIf you are using AWS Storage Gateway appliance and hosted on KVM hypervisor, I would suggest you to refer the AWS Doc [1] on how to connect appliance console from kvm command line. You can come out of the console by entering "X" if you are in main logged in prompt.However, if you are looking for appliance's Linux OS level prompt (latest version is built on AL2 version) that is not possible since it is AWS managed and developed AMI image and we strongly recommend not to make  any changes within the OS which will lead to issues with storage gateway appliance.The we have already described in our public documentation [2]. You should treat the Storage Gateway appliance as a managed embedded device, and should not attempt to access or modify its installation in any way. Attempting to install or update any software packages using methods other than the normal gateway update mechanism (for example, SSM or hypervisor tools) may cause the gateway to malfunction.Refer:[1] https://docs.aws.amazon.com/filegateway/latest/files3/accessing-local-console.html#MaintenanceConsoleWindowKVM-common[2] https://docs.aws.amazon.com/filegateway/latest/files3/MaintenanceManagingUpdate-common.htmlCommentShareRRP_AWSanswered 9 months ago0Hi, adding to above answer, if you are looking to come out of the AWS Appliance console after exiting the configuration session using X, you would need to provide the Escape character. You would see an escape character when you are connected to the VM using virsh console <vmname>, for example like Escape character is ^].  Depending on the keyboard language settings, you might need to use different combination of keys. Please refer here for additional details. Thank you.CommentShareSuryaanswered 9 months ago"


Question: AWS SSO Google Workspace IDP SCIM

"Hi all!I've been able to configure AWS SSO to with with Google Workspace as it's identity provider using this guide - https://aws.amazon.com/blogs/security/how-to-use-g-suite-as-external-identity-provider-aws-sso/.I saw that Google isn't a fully supported external identity provider, meaning that it doesn't support automatic provisioning of users/groups from Google Workspace into AWS SSO.When will the automatic provisioning feature be available for Google Workspace?In the meantime, what alternatives are there for this feature? I have came across and checked this project https://github.com/awslabs/ssosync,However, It is no longer available or supported.Any help will be much appreciated :) ,DaveFollowCommentchristiangda 2 months agoHi Dave, I understand your frustration, I pass through the same contributing to the awslabs/ssosync project. So, that is why I build this other one https://github.com/slashdevops/idp-scim-sync from scratch, maybe this work for your requirements until AWS have a new better solutionShare"

Answer: "1Right now that SSO Sync on Github is the best option if you want to automatically synchronize groups from your Google Workspaces to AWS SSO.We also have a workshop that demonstrates how the SSO Sync can be used to setup integration between AWS SSO and Google Workspaces.CommentShareEXPERTToni_Sanswered a year agoDave a year agoI have tried deploying the SSO Sync Github project however, I't seems that this is a dead project for a few reasons.The last commit was a year ago.The README link to "AWS Serverless Application Repository" that supposedly enables the deployment of SSO Sync doesn't work.There's actually an open issue that many people aren't able to access it and the reason seems to be that the related account was deleted.Following the manual instructions just doesn't work, there are multiple bugs.From the workshop link you provided it's also stated in the intro that -"AWS Single Sign-On (SSO) currently does not support Google Workspace as an identity provider for automatic provisioning of users and groups, or the ssosync application, available on the AWS Serverless Application Repository."Share"


Question: "Some PDFs uploaded to S3 are given the Content-Type 'Image', and then won't open in a browser"

"I am using a Wordpress plugin called Media Cloud, which allows me to upload files into my CMS, which pass through to an S3 bucket. I'm not sure whether this is happening at the plugin level or the S3 level, but sometimes, when the PDF arrives in my bucket, the metadata shows the Content-Type as image/png, and not application/pdf.This means that the PDF won't open in a browser, because the browser is trying to treat it as an image. Does anyone know why this might happen, and if there is a rule I can set to force all pdfs to have the PDF metadata?FollowComment"

Answer: 0The content-type metadata is set while uploading the file. Have you checked how your CMS processes and identifies files? It sounds like the error happens while the CMS is uploading it into S3. The default behaviour when uploading to S3 depends on the AWS SDK that your CMS uses.If you can't fix this on the CMS side you could consider creating your own little Lambda function that is triggered when file is uploaded and it then checks if the content-type in metadata matches the file extension. If it doesn't it could update it automatically. Check out how to trigger Lambda based on S3 in our documentation.CommentShareEXPERTToni_Sanswered a year ago


Question: EMR Hive read/write performance issues when using S3 as storage layer

"We recently migrated part of our Big Data platform on Cloudera to AWS EMR. We are seeing significant read/write performance degradation using Hive with the storage layer as S3. Performance is acceptable when we copy all the data from S3 onto HDFS local block storage on the EMR cluster. However, this has increased our overall costs on the platform.Any suggestions/tips to improving EMR Hive performance on S3 would be appreciated.FollowComment"

Answer: "0I understand that you are experiencing performance degradation using Hive with the storage layer as S3. Given S3 is a storage layer that lives outside of your VPC, it will incur additional network traffic adding latency and cost. Having said that I hope the following will help you minimize the performance degradation if not preventing it.Please have a look at the Operational differences and considerations which details the drawbacks of having versioned buckets and mitigation by updating the S3 bucket Lifecycle policy delete them frequently in the "/tmp" directory.Enable Hive EMRFS S3 optimized committer to take advantage of the performance enhancements built into the EMRFS library for S3 on EMR.Please check your CloudTrail for S3 503 "Slow Down" errors for your Hive jobs, if you do have them please follow the recommendations in the Knowledge Center article on the same.In worst cases the above recommendation may still not yield any performance improvements, the case I'm referring here is regarding the S3 (internal) partitions which should not be confused with the Data Partitioning. S3 limit is 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per partitioned prefix. If there are too many objects in your Bucket which your Queries in Hive are reading, it may trigger GetObject requests for each of these objects, resulting in 503 "Slow Down", in most cases retrying will help however extreme cases even retrying may not succeed as you may have many worker nodes requesting objects at the same time. You will have to reach out to AWS Support to get that increased for your bucket, which should get you better performance for S3 read/write.I believe the above will help you improve the performance for Hive on S3, please reply to this thread if you have any follow up questions.CommentShareSUPPORT ENGINEERKrishnadas Manswered a year ago"


Question: How to Clone instance EC2

"If I clone an instance, will the new instance contain all the programs, folders, and files that the original instance has?FollowComment"

Answer: "2You can't clone an EC2 as such (at least not like you can e.g. clone a VM in vSphere), but you can take a snapshot of it, create an AMI from the snapshot, and then provision a new EC2 instance from that AMI.This is for the system disk (or root disk), additional disks can be snapshotted and then you can choose to "Create volume from snapshot".And yes, everything that is on the original EC2 instance will be on the new one too.CommentShareRWCanswered 18 days agoEXPERTalatechreviewed 18 days ago0follow steps as per below article. Try in test environment first ...https://docs.bitnami.com/aws/faq/administration/clone-server/CommentShareHemant Chughanswered 17 days ago0You can create an Amazon Machine Image (AMI) of your instance, and then use the AMI to launch instances that are identical to the original instance. Here are the instructions in the EC2 User Guide: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-ebs.htmlCommentShareMarilynAWSanswered 16 days ago"


Question: Greengrass v2 add component

I need to create a new greengrass v2 component and remove old ones. Is it better to update the current greengrass core device with the new component or need to cleanup and create a new core device with the new component?FollowComment

Answer: "3Accepted AnswerThere should be no need to create a new core device. You can add the new component to your deployment and remove any that are no longer needed and the deployment will take care of getting the existing core device(s) that are associated with that deployment synced up with your changes. I actually used this very method while creating and updating a demo I was working on recently.Keep in mind that this action may interrupt processes running on your existing device, so if it's production critical plan around this.CommentShareJeff_Banswered 2 years ago0If you want to get started using Greengrass v2, be sure to check out http://greengrassworkshop.com/ where we take you through on setting up component and deploying it.CommentShareSyed Rehananswered a year ago"


Question: Savings plan and reserve instance

"on one single instance family that has an existing Savings plan, can i implement reserve instance. (two savings plan on one single instance family)FollowComment"

Answer: "1Accepted AnswerWhen AWS will send the bill, it will apply the RI to one instance and then apply the savings plan to other instances but not the one where RI was already applied. The answer in short is no. Makes sense?CommentShareteknivoranswered 5 months agorePost-User-8174665 5 months agosuper good, thank you boss.Share0You can. As per AWS doc, "Savings Plans apply to your usage after the Amazon EC2 Reserved Instances (RI) are applied."https://docs.aws.amazon.com/savingsplans/latest/userguide/sp-applying.htmlReserved Instances should be purchased to reserve the capacity from AWS, the compute capacity that you purchase via RI will always be there for you to use. Once RI is applied to your instance, Savings Plan will apply to the remaining instances, even if they are of the same family.CommentShareteknivoranswered 5 months agorePost-User-8174665 5 months agoHi teknivor, thank you the response.so my question is that if we purchased RI on 1 instanceand we applied 1 of saving plan (from our compute or ec2 saving plan) on the same instance so will it WORK OR NOT?Share"


Question: AWS Managed Elasticsearch 6.8 Backup and Restoration

"Hi Team,I have a requirement of taking AWS Managed Elasticsearch 6.8 manual Backup and Restore it. Can I increase the Storage of Elasticsearch if i do it will there downtime.FollowComment"

Answer: "0Accepted AnswerHi,Not sure if your question mentioning backup / restore and increase in storage has some corelation; in which case you might want to clarify it further. Hence answering the question about increase in storage:Increasing storage of elasticsearch cluster would trigger a blue / green deployment[1] which could cause minimal downtime. Also since the number of data nodes are doubled during the activity, it could cause strain on the master nodes since they have more nodes to manage as well as possibly increase search and indexing latency[2]. So you should normally carry out this activity during lean hours.When you modify the cluster configuration to change the storage, you can run an analysis (shown towards the end of the page) to confirm that the change triggers a blue / green deploymentReferences:[1] - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-configuration-changes.html#bg[2] - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-configuration-changes.html#initiateCommentShareSydlg...answered 7 months agolg...EXPERTFabrizio@AWSlg...reviewed 7 months agolg...rePost-User-2738569 7 months agoThanks for answeringShare"


Question: updated reward function to speed up but not working as expected.

"HiI updated reward_function to speed up using params['steps'] but not working as expected.The reward_function is this=====================================def reward_function(params):'''Example of rewarding the agent to follow center line'''reward=1e-3# Read input parameters  track_width = params\['track_width']  distance_from_center = params\['distance_from_center']  steering = params\['steering_angle']  speed = params\['speed']  all_wheels_on_track = params\['all_wheels_on_track']  steps = params\['steps']  progress = params\['progress']    # Total num of steps we want the car to finish the lap, it will vary depends on the track length  TOTAL_NUM_STEPS = 300    if distance_from_center >=0.0 and distance_from_center <= 0.03:      reward = 1.0    if not all_wheels_on_track:      reward = reward -1  else:      reward = reward + progress        # add speed penalty  if speed < 1.0:      reward *=0.80  else:       reward += speed        **# Give additional reward if the car pass every 100 steps faster than expected**   **if (steps % 50) == 0 and progress > (steps / TOTAL_NUM_STEPS) :**      **reward += 10.0**    return float(reward)  ==================================================There was no lines with bold in previous model.I trained for 5 hours but the result is slower than before.with Previous model, I could get 2124 seconds to run the Empire City track but with the new model after training for 5 hour, I am getting 2427 seconds.Other Action space and hyperparameters are same.Anybody knows why the result is slower than before?FollowComment"

Answer: "0Hi,You could insert some debug code to ensure it is getting applied, e.g. print("Fast reward bonus given").However, looking at your code, I think the issue is that progress is a value from 0-100, where as your steps / TOTAL_NUM_STEPS gives a fraction 0-1. So progress will nearly always be greater than that, and so your bonus is probably being given regardless of actual performance.I'm also not sure that a bonus of 10 every 50 steps would be enough to encourage the model to favour those actions. It would also depend on your discount rate, e.g. if you're using the default discount rate of 0.9 then that only equates to 10 steps look ahead, so your bonus would be invisible to most policy training updates.Finally, you need to be careful of overfitting your model to the track. When that happens, lap times can get worse as the model learns the safer actions/track position to ensure more stability. So that might explain why your model got slower, even if your reward wasn't correctly being applied.LyndonCommentShareETAGGELanswered 4 years ago"


Question: "When querying athena using aws sdk, is it necessary to save results in s3?"

"I tried to do a select query on Athena using aws sdk.In the sample, using resultconfiguration, it was stored in s3 and then queried.However, it seems that the console showed the select result without saving the query result.So I skipped resultconfiguration and ran aws sdk, and the following error occurred.An output location is required either through the Workgroup result configuration setting or as an API input.So I think that to query athena from aws sdk, the result must be saved in s3. Is this correct?If not, please tell me how should I configure the aws sdk.FollowComment"

Answer: "1Accepted AnswerHello,Amazon Athena automatically stores query results and metadata information for each query that runs in a query result location that you can specify in Amazon S3.If necessary, you can access the files in this location to work with them. You can also download query result files directly from the Athena console.To determine the location that Athena will use you need a combination of a workgroup and client-side settings.The client-side settings are based on how you run the query:If you run the query using the Athena console, the Query result location entered under Settings in the navigation bar determines the client-side setting.If you run the query using the Athena API, the OutputLocation parameter of the StartQueryExecution action determines the client-side setting.If you use the ODBC or JDBC drivers to run queries, the S3OutputLocation property specified in the connection URL determines the client-side setting.To use the AWS CLI to identify the query output location and result files, run the aws athena get-query-execution command, as in the following example.aws athena get-query-execution --query-execution-id <your-query-id>Note: When you run a query using the API or using the ODBC or JDBC driver, the console setting does not apply.For more detailed information, refer to Finding Query Output Files in S3.I hope this helps.CommentShareLungaanswered a year agojoker a year agoDoes it work even if I don't specify the OutputLocation parameter in StartQueryExecution?It doesn't work in my case, so I'm wondering if I need to save the result to s3 when using the api.Share"


Question: AWS Instance no longer contactable

"Dear Sir/Madam,My AWS instance is no longer contactable. I have recreated the HTTP/HTTPS and SSH firewall rules and ensured my public IP is listed.I have checked the public elastic IP address is correct.The instance appears to have booted based on screenshot (serial console login is not possible on my instance type)The problem occurred when (as far as I know) no changes were being made to the system.The startup log is below.Many thanks for your help.Welcome to CentOS Linux 7 (Core)![    6.646736] systemd-sysv-generator[373]: stat() failed on /etc/rc.d/init.d/jexec: No such file or directory[  OK  ] Stopped Switch Root.[  OK  ] Stopped Journal Service.Starting Journal Service...[  OK  ] Stopped target Switch Root.Mounting POSIX Message Queue File System...[  OK  ] Created slice system-serial\x2dgetty.slice.[  OK  ] Started Dispatch Password Requests to Console Directory Watch.Starting Read and set NIS domainname from /etc/sysconfig/network...[  OK  ] Created slice system-selinux\x2dpol...grate\x2dlocal\x2dchanges.slice.[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.[  OK  ] Reached target rpc_pipefs.target.[  OK  ] Set up automount Arbitrary Executab...ats File System Automount Point.[  OK  ] Created slice User and Session Slice.[  OK  ] Reached target Slices.[  OK  ] Started Forward Password Requests to Wall Directory Watch.[  OK  ] Reached target Paths.[  OK  ] Stopped target Initrd File Systems.[  OK  ] Listening on Delayed Shutdown Socket.[  OK  ] Created slice system-getty.slice.Starting Create list of required st... nodes for the current kernel...Mounting Debug File System...[  OK  ] Reached target Local Encrypted Volumes.Mounting Huge Pages File System...[  OK  ] Stopped File System Check on Root Device.Starting Remount Root and Kernel File Systems...Starting Apply Kernel Variables...[  OK  ] Reached target RPC Port Mapper.Starting Setup Virtual Console...[  OK  ] Stopped target Initrd Root File System.[  OK  ] Listening on udev Control Socket.[  OK  ] Listening on udev Kernel Socket.[  OK  ] Mounted Debug File System.[  OK  ] Mounted POSIX Message Queue File System.[  OK  ] Mounted Huge Pages File System.[  OK  ] Started Journal Service.[  OK  ] Started Read and set NIS domainname from /etc/sysconfig/network.[  OK  ] Started Create list of required sta...ce nodes for the current kernel.[  OK  ] Started Remount Root and Kernel File Systems.[  OK  ] Started Apply Kernel Variables.[  OK  ] Started Setup Virtual Console.Starting udev Coldplug all Devices...Starting Configure read-only root support...Activating swap /swap...Starting Create Static Device Nodes in /dev...Starting Flush Journal to Persistent Storage...[    7.774831] systemd-journald[398]: Received request to flush runtime journal from PID 1[  OK  ] Started Create Static Device Nodes in /dev.[  OK  ] Started udev Coldplug all Devices.[  OK  ] Started Flush Journal to Persistent Storage.[  OK  ] Reached target Local File Systems (Pre).Starting udev Kernel Device Manager...[  OK  ] Started Configure read-only root support.Starting Load/Save Random Seed...[  OK  ] Reached target Local File Systems.Starting Preprocess NFS configuration...Starting Import network configuration from initramfs...[  OK  ] Started Load/Save Random Seed.[  OK  ] Started Preprocess NFS configuration.[  OK  ] Started udev Kernel Device Manager.[    8.092198] input: PC Speaker as /devices/platform/pcspkr/input/input4[    8.113229] piix4_smbus 0000:00:01.3: SMBus base address uninitialized - upgrade BIOS or use force_addr=0xaddr[  OK  ] Started Import network configuration from initramfs.Starting Create Volatile Files and Directories...[    8.317801] [TTM] Zone  kernel: Available graphics memory: 1939310 kiB[    8.332058] [TTM] Initializing pool allocator[  OK  ] Found device /dev/ttyS0.[  OK  [    8.373158] Adding 4194300k swap on /swap.  Priority:-2 extents:4 across:335358284k SSFS] Started Create Volatile Files and Directories.[    8.399536] [TTM] Initializing DMA pool allocator[  OK  ] Activated swap /swap.[    8.427533] cryptd: max_cpu_qlen set to 1000[    8.436055] [drm] fb mappable at 0xF0000000[    8.436056] [drm] vram aper at 0xF0000000[    8.436056] [drm] size 33554432[    8.436057] [drm] fb depth is 16[    8.436057] [drm]    pitch is 2048[    8.496630] fbcon: cirrusdrmfb (fb0) is primary device[    8.515080] ppdev: user-space parallel port driver[    8.530430] AVX2 version of gcm_enc/dec engaged.[    8.530431] AES CTR mode by8 optimization enabled[    8.533264] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni)[    8.533291] alg: No test for __generic-gcm-aes-aesni (__driver-generic-gcm-aes-aesni)[    8.703061] Console: switching to colour frame buffer device 128x48[    8.727500] cirrus 0000:00:02.0: fb0: cirrusdrmfb frame buffer device[  OK  ] Reached target Swap.Starting Security Auditing Servi[    8.793774] [drm] Initialized cirrus 1.0.0 20110418 for 0000:00:02.0 on minor 0ce...[    8.808114] EDAC sbridge:  Ver: 1.1.2[    8.867523] type=1305 audit(1682408275.896:4): audit_pid=526 old=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1[  OK  ] Started Security Auditing Service.Starting Update UTMP about System Boot/Shutdown...[  OK  ] Started Update UTMP about System Boot/Shutdown.[  OK  ] Reached target System Initialization.[  OK  ] Listening on D-Bus System Message Bus Socket.Starting Initial cloud-init job (pre-networking)...[  OK  ] Started Daily Cleanup of Temporary Directories.[  OK  ] Reached target Timers.[  OK  ] Listening on RPCbind Server Activation Socket.[  OK  ] Reached target Sockets.[  OK  ] Reached target Basic System.Starting Authorization Manager...Starting NTP client/server...[  OK  ] Started irqbalance daemon.Starting Login Service...[  OK  ] Started D-Bus System Message Bus.Starting Dump dmesg to /var/log/dmesg...Starting Process Monitoring and Control Daemon...Starting GSSAPI Proxy Daemon...Starting RPC bind service...[  OK  ] Started Login Service.[  OK  ] Started RPC bind service.[  OK  ] Started NTP client/server.[  OK  ] Started Authorization Manager.[  OK  ] Started Dump dmesg to /var/log/dmesg.[  OK  ] Started GSSAPI Proxy Daemon.[  OK  ] Reached target NFS client services.[  OK  ] Reached target Remote File Systems (Pre).[  OK  ] Reached target Remote File Systems.[  OK  ] Started Process Monitoring and Control Daemon.[   10.304853] cloud-init[595]: Cloud-init v. 19.4 running 'init-local' at Tue, 25 Apr 2023 07:37:57 +0000. Up 10.27 seconds.[  OK  ] Started Initial cloud-init job (pre-networking).[  OK  ] Reached target Network (Pre).Starting LSB: Bring up/down networking...[FAILED] Failed to start LSB: Bring up/down networking.See 'systemctl status network.service' for details.[  OK  ] Reached target Network.Starting MySQL Server...Starting Dynamic System Tuning Daemon...Starting Sendmail Mail Transport Agent...Starting The Apache HTTP Server...Starting Initial cloud-init job (metadata service crawler)...[  OK  ] Started Sendmail Mail Transport Agent.Starting Sendmail Mail Transport Client...[  OK  ] Started Sendmail Mail Transport Client.[   12.895411] cloud-init[871]: Cloud-init v. 19.4 running 'init' at Tue, 25 Apr 2023 07:37:59 +0000. Up 12.78 seconds.[   12.989232] cloud-init[871]: ci-info: ++++++++++++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++++++++++[   13.034113] cloud-init[871]: ci-info: +--------+------+-----------------------------+-----------+-------+-------------------+[   13.069618] cloud-init[871]: ci-info: | Device |  Up  |           Address           |    Mask   | Scope |     Hw-Address    |[   13.095540] cloud-init[871]: ci-info: +--------+------+-----------------------------+-----------+-------+-------------------+[   13.157977] cloud-init[871]: ci-info: |  eth0  | True | fe80::4f6:39ff:fe19:ee8a/64 |     .     |  link | 06:f6:39:19:ee:8a |[   13.185821] cloud-init[871]: ci-info: |   lo   | True |          127.0.0.1          | 255.0.0.0 |  host |         .         |[  OK  ] Started Dynamic System Tuning Daemon.[   13.227876] cloud-init[871]: ci-info: |   lo   | True |           ::1/128           |     .     |  host |         .         |[   13.244949] cloud-init[871]: ci-info: +--------+------+-----------------------------+-----------+-------+-------------------+[   13.279694] cloud-init[871]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++[   13.299077] cloud-init[871]: ci-info: +-------+-------------+---------+-----------+-------+[   13.321298] cloud-init[871]: ci-info: | Route | Destination | Gateway | Interface | Flags |[   13.339736] cloud-init[871]: ci-info: +-------+-------------+---------+-----------+-------+[   13.363740] cloud-init[871]: ci-info: |   9   |  fe80::/64  |    ::   |    eth0   |   U   |[  OK  ] Started The Apache HTTP Server.[   13.387090] cloud-init[871]: ci-info: |   12  |   ff00::/8  |    ::   |    eth0   |   U   |[   13.403810] cloud-init[871]: ci-info: +-------+-------------+---------+-----------+-------+[  OK  ] Started Initial cloud-init job (metadata service crawler).Starting Permit User Sessions...[  OK  ] Reached target Network is Online.Starting Notify NFS peers of a restart...Starting Crash recovery kernel arming...Starting System Logging Service...Starting OpenSSH server daemon...[  OK  ] Reached target Cloud-config availability.Starting Apply the settings specified in cloud-config...[  OK  ] Started Permit User Sessions.[  OK  ] Started Notify NFS peers of a restart.[  OK  ] Started Command Scheduler.[  OK  ] Started Getty on tty1.[  OK  ] Started Serial Getty on ttyS0.[  OK  ] Reached target Login Prompts.[  OK  ] Started System Logging Service.[  OK  ] Started OpenSSH server daemon.[   14.429972] cloud-init[1001]: Cloud-init v. 19.4 running 'modules:config' at Tue, 25 Apr 2023 07:38:01 +0000. Up 14.29 seconds.[  OK  ] Started Apply the settings specified in cloud-config.Starting Execute cloud user/final scripts...[   14.938029] cloud-init[1083]: Cloud-init v. 19.4 running 'modules:final' at Tue, 25 Apr 2023 07:38:01 +0000. Up 14.83 seconds.[   15.050137] cloud-init[1083]: Cloud-init v. 19.4 finished at Tue, 25 Apr 2023 07:38:02 +0000. Datasource DataSourceEc2Local.  Up 15.04 seconds[  OK  ] Started Execute cloud user/final scripts.[  OK  ] Started Crash recovery kernel arming.CentOS Linux 7 (Core)Kernel 3.10.0-1160.76.1.el7.x86_64 on an x86_64ip-172-31-20-0 login:FollowComment"

Answer: "0I changed the instance type from t2.medium to t3.medium in order to allow a serial connection to debug further. This fixed the issue and the server was online again, without needing the serial interface. The system log seems to suggest the server was no longer assigned a private IP address at boot time.No other changes were made other than changing the instance type.Changing back to a t2.medium does not make the error reappear.Can anyone shed any light on what might have caused this issue and how to avoid it in the future?CommentSharemundo1979answered a month ago"


Question: Input stitchin with one audio track for all video

"Hi all,I have 3 input video file, i need to merge them together, and this is not a problem but i need to insert an audio track on whole video output after merge, is it possible with a single job?For example 4 assets:intro.mp4content.mp4outro.mp4song.mp3I need to merge intro,content, outro in one video and insert song.mp3 as soundtrack on whole video in one jobsany ideas?FollowComment"

Answer: "1Accepted AnswerHiRegarding your use case, it is almost impossible to do it in ONE job. Here is the explanation.Let's make the following assumption. The duration of the video and song as follow.input 1- intro.mp4 - 10 secondsinput 2 - content.mp4 - 10 secondsinput 3 - outro.mp4 - 10 secondssong.mp3 - 30 secondsYou can use "external file" for audio for each input (https://docs.aws.amazon.com/mediaconvert/latest/ug/create-selectors.html). In theory, you can use "offset" to adjust the start time of the audio for input switch. i.e.input 1- intro.mp4 - 10 seconds, audio start at 00:00:00input 2 - content.mp4 - 10 seconds, audio start at 00:00:10input 3 - outro.mp4 - 10 seconds, audio start at 00:00:20song.mp3 - 30 secondsHowever, because of the frequency of audio, it is almost impossible to make a seamless switch even using offset. Therefore, the most simple and perfect solution would be 1) create a job to merge all three videos, 2) create another job to merge the song to the output video of job1.RegardsMichaelCommentShareawsmicmicanswered 2 years ago0Thx Michael, I was hoping that there was another way to merge video and global audio track. Thx for support!Edited by: Raziss on May 13, 2021 10:02 PMCommentShareRazissanswered 2 years ago"


Question: AWS: Using TLS-certificate without private key

"For using an API from the dutch chamber of commerce, It's mandatory to install a TLS-certificate from the Dutch government. Only when the certificate is installed, It's possible to make authorized API-requests. The chamber of commerce has published an install guide here: https://developers.kvk.nl/support/documentation/manual-tls-certificateWe want to use this API in an application that's hosted with AWS amplify as backend with React as frontend. The most obvious approach was using AWS certificate manager, because it's integrated with Amplify. The problem we encounter using the certificate manager is that It's mandatory to provide a private key in PEM format to import a certificate (see picture). The thing is, there are no private keys given since the certificates are from the governments, and the manual does not describe what needs to be done in our case.(https://i.stack.imgur.com/uBXRl.png)We tried to contact the chamber of commerce but our response was that it was not in their reach and we should try it elsewhere. Our only option as we see now is that we need to host a dedicated server to redirect the api requests and responses, but that would defeat our point of being serverless with Amplify.FollowComment"


Question: CloudFormation Updating AppRunner Service

"Is it possible after the AppRunner is created from CloudFormation to modify the same AppRunner service using CloudFormation? (in azure the scripts everytime that are executed check if the resource is created and then update it, instead of deleting it). Or I will have to use CDK/SDK to update an already created AppRunner service?FollowComment"

Answer: "0When you look through the CloudFormation documentation for this resource (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apprunner-service.html) you'll find that each property defines what happens when you change it.  Some are "Update requires: No interruption", some "Update requires: Replacement".CommentShareEXPERTskinsmananswered 3 months ago"


Question: AWS IoT for DynamoDB

"I connected my device (ESP32) to the IoT core and I'm receiving messages subscribing to a topic in the MQTT test, but my messages don't arrive in DynamoDB, could someone more experienced help me? I've already reviewed the rules, roles and attached the appropriate policies, but I can't find the error.help me, pleaseFollowComment"

Answer: "2Accepted AnswerHi,please check CloudWatch logs for details about your rule execution (logging must be enabled for AWS IoT Core).Your payload has temperature and humidity as keys. What is your primary key in DynamoDB? Every primary key identifies every item uniquely. Temperature or humidity can have the same value over time and cannot be considered to be unique.To generate a unique key from your SQL statement you could use something likeSELECT newuuid() as prim_key, * FROM 'mytopic'Cheers,PhilippCommentSharePhilipp Sanswered 9 months agorePost-User-9750032 9 months agoI configured IoT to be monitored on CloudWatch and I didn't find any errors.I changed the as suggested, SQL: SELECT newuuid() as prim_key, *but still no items identified in the table.I'm using dynamodbv2.ShareGreg_B EXPERT9 months agoHi. How to enable logging: https://docs.aws.amazon.com/iot/latest/developerguide/configure-logging.html. This will then create log streams in a CloudWatch log group called AWSIoTLogsV2. Rule execution logs you should see: https://docs.aws.amazon.com/iot/latest/developerguide/cwl-format.html#log-rules-rule-ex.This tutorial should steer you in the right direction for the rule, payload enrichment and DynamoDB (even though it's the V1 action, the principles are similar): https://docs.aws.amazon.com/iot/latest/developerguide/iot-ddb-rule.htmlSharePhilipp S 9 months agoWhat is the primary key of you DynamoDB table?Do you see in CloudWatch that your rule is triggered? If yes, what is the result when the rule is being triggered?SharerePost-User-9750032 9 months agoI created a group of logs as suggested, I didn't see any occurrence, I added the variable "Tempo" and set it as my primary key and apparently it solved my problem, thank you all for your attention, very grateful to you.Share0I added the variable "Time" to my message and set it as my partition key and finally I have data in my table.CommentSharerePost-User-9750032answered 9 months ago"


Question: IAM Policy Conditions

"Hi,Just like to know in general, does IAM allow conditions where the value is a concatenation of > 1 variable? Eg. Specifically:"Condition": {"ForAllValues:StringEquals": {"dynamodb:LeadingKeys": ["${www.amazon.com:user_id}${aws:PrincipalTag/myCustomTag}"]The above is for DynamoDB, but in general is concatenation allowed?Thank youFollowComment"

Answer: "0Could you please elaborate on what is the intent or what are you trying to achieve exactly with the policy? This will help answer the question more accurately."ForAllValues:StringEquals" in a condition simply means that all values have to be present (true) for the condition to take place, whereas for example "ForAnyValue:StringEquals" would accept any of the values given for the condition to take effect.More info: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_multi-value-conditions.htmlHope this helps.CommentShareZadusPlaceanswered a year ago0Thank you for your reply. Further to your question, since the IAM policies are just a matching mechanism, all I was interested was whether IAM allows substitutions of more than 1 variable in a conditional evaluation. What I had there was just an example: "${www.amazon.com:user_id}${aws:PrincipalTag/myCustomTag}". So if www.amazon.com:user_id="JOHN", and aws:PrincipalTag/myCustomTag="NEW YORK", will IAM concatenate this to "JOHNNEW WORK" and evaluate this?CommentSharerePost-User-8196003answered a year ago0So in this case, yes, however, because your operator is "ForAllValues:StringEquals" it will take 2 things for the condition to be active: that the userID is John, and also, that the PrincipalTag = NEWYORK. So both have to be true or present before the condition becomes "active" (whether it is for an Allow or Deny).I believe in your use-case you would need 2 condition operators within the condition, something like this for example:  "Condition": {            "NotIpAddress": {                "aws:SourceIp": [                    "192.0.2.0/24"                ]            },              "StringNotLike": {                 "aws:RequestTag/Env": [                        "Dev"                    ]            }In the example above, it means the NopIPAddress operator AND the RequestTag operators both have to be "valid" before the condition is applied, meaning, the IP is not from the subnet 192.0.2.0/24, and also, that the RequestTag is not a DEV env, then the condition will applyCommentShareZadusPlaceanswered a year ago"