Question: Best practices for hundreds/thousands of keys/usage plans?

"I am working with an API Gateway system to give clients access to data.  I am currently having each user get their own Usage Plan and API Key: my reasoning for not having one Usage Plan is that I do not want one user to lock out another by making a high burst of requests.  I am aware of AWS limits (300 usage plans per region,  500 api keys per region https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html) which have me concerned about how I will add users when my Usage Plan/API Key limit is reached.I am aware these limits are able to be raised by contacting Amazon, but is there another way I should be doing this?  I don't want to reach a point where I can't add more users and am stuck.FollowComment"

Answer: "0my reasoning for not having one Usage Plan is that I do not want one user to lock out another by making a high burst of requests.API Keys in the same usage plan do not share usage data and would not affect other keys in the same plan so this would not be a concern.That being said, we can not currently support 100s of thousand of API keys. This may change in the future, but at this time you would eventually not be able to create new API keys.Regards,BobCommentShareEXPERTAWS-User-0395676answered 4 years ago0ThanksCommentShareColinWithMothrasanswered 4 years ago"


Question: Call a redshift stored procedure with dynamic parameter from boto3 lambda

"I have a lambda that needs to call a stored proc in redshift with a parameter value not known at runtime. I have narrowed the code down to this in the call:response = rs_client.execute_statement(            ClusterIdentifier='my_cluster_name',            Database='my_db_name',            DbUser='my_user'',            Sql='call my_schema.this_is_my_proc()',            Parameters=[{'name': 'in_param', 'value': 'this is the parameter'}],            StatementName='Test of SP call'        )The above works fine with the creds (I have anonymised them) and with no parameter and also a hardcoded parameter in the Sql argument i.e. Sql=call my_schema.this_is_my_proc('this is hardcoded)' so that's not the issue. I also tried passing in the parameter as a formatted string e.g.param = 'this is the parameter'stmnt = f'call my_schema.this_is_my_proc({param})',The message I get back from the lambda Test execution is:An error occurred (ValidationException) when calling the ExecuteStatement operation: Parameters list contains unused parameters. Unused parameters: [in_param]Is it possible to call Redshift stored procs that have parameters from a lambda?TIAFollowComment"

Answer: "0Accepted AnswerHi,as mentioned in the documentation the parameter should be typed in the sql statement in the format :parameter.following the doc your code should look like:response = rs_client.execute_statement(            ClusterIdentifier='my_cluster_name',            Database='my_db_name',            DbUser='my_user'',            Sql='call my_schema.this_is_my_proc(:in_param)',            Parameters=[{'name': 'in_param', 'value': 'this is the parameter'}],            StatementName='Test of SP call'        )hope this helps,CommentShareEXPERTFabrizio@AWSanswered a year ago"


Question: UDF stored in S3 unable to run as a glue job

"Trying to use a UDF for converting a column to uppercaseUDF is in the S3 bucket .zip file which is accessed by the Glue jobpy script of UDF for reference attachedISSUE:The UDF is not storing the data in the expected S3 bucket nor it is reading the data from the S3 bucket providedimport sysfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom awsglue.context import GlueContext# Import the UDF code from Amazon S3spark.sparkContext.addPyFile("s3://**********")from mylibrary import udf_uppercase# Create a Spark contextsc = SparkContext()# Create a Glue contextglueContext = GlueContext(sc)spark = glueContext.spark_session# Define the AWS Glue jobjob = Job(glueContext)job.init("uppercase_testUDF", args)# Read data from S3datasource0 = glueContext.create_dynamic_frame.from_options(    "s3",     {"paths": ["s3://***********"]},    format="json")# Apply the UDF to the datatransformed_data = datasource0.toDF().withColumn("column_name", udf_uppercase(datasource0["column_name"]))transformed_data.show()Write the transformed data back to S3glueContext.write_dynamic_frame.from_options(    frame=transformed_data,    connection_type="s3",    connection_options={        "path": "s3://*****************",        "format": "json"    },    transformation_ctx="test_uppercase_bookmark")# Run the AWS Glue jobjob.commit()FollowComment"

Answer: "0Your UDF file should be at the root level of your zip file , if you are not making a py package .Instead of importing like from mylibrary import udf_uppercase ,  just use import udf_uppercase given that udf_uppercase.py is your UDF file nameTry to use the Python Library Path text box in the job properties tab. give the full path like :s3://BucketName/utils/utils.zipRefer :https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.htmlCommentShareKoustavCanswered 4 months ago0That should work (assuming that your file is named mylibrary.py) so that import can work.The issue I see there is that you convert to DataFrame, add a column (still DataFrame) and then try to pass it to write_dynamic_frame which is intended for DynamicFrame.You should receive an error like this:TypeError: frame_or_dfc must be DynamicFrame orDynamicFrameCollection. Got <class 'pyspark.sql.dataframe.DataFrame'>BTW, I guess that's just an example, it's much more efficient if you don't use udfs and do upper case using a SQL function.CommentShareGonzalo Herrerosanswered 3 months ago"


Question: AWS Glue Notebook issue when running SQL script

"I am following the steps outlined in the link below:https://aws.amazon.com/blogs/big-data/introducing-native-delta-lake-table-support-with-aws-glue-crawlers/(1) No issue with Query Delta Lake tables using Amazon Athena, able to query out the data.(2) Issue AWS Glue Notebook:(a) created IAM role named "AWSGlueServiceRoleDefault" which includes(i) AmazonS3FullAccess, which is AWS managed(ii) AWSGlueServiceRole, which is AWS managed(iii) PassRolePolicy, which is Customer InlinePassRolePolicy as below:Following the instructions from the link, which I used the IAM role ("AWSGlueServiceRoleDefault") I created above, the first part of the Python code is able to run as shown below:But the SQL portion throws me a list of Py4JJavaError:Even the below Python code which tries to retrieve information about the table generates the same error as below:The above error is repeatable, which can be viewed here:https://justpaste.it/1zxzzWould appreciate if you could help me to see which portion is creating the error, and how to remedy it. Thanks.FollowComment"

Answer: "0Accepted AnswerThe libraries for the Delta format are not enabled by default.The job needs a parameter: --datalake-formats=deltaFrom a notebook you need to pass it as config before you run any other cell (once the session is started, it won't take effect)%%configure{    "--datalake-formats": "delta"}CommentShareGonzalo Herrerosanswered 2 months agorePost-User-3769185 2 months agoSplendid.... your short and concise answer saves me lots of hours of frustration to find out exactly what is the issue. Thank you very much.SharerePost-User-3769185 2 months agoTake note above also needs to add IAM:PassRole policy, which is not instructed in the first link, else it would have issue relating to IAM:PassRole.Share"


Question: OAuth redirect URL

"Hi,I am trying to connect to a 3rd party software via APIs to get a Bearer token.I need a redirect URL for this.  What is the easiest way to do this?I thought through the API Gateway but when I started looking for information I came acrossAmazon Cognito https://docs.aws.amazon.com/cognito/latest/developerguide/authorization-endpoint.htmlAre either of those the best way to proceed? Or is there another way? I am sorry if this is a verysimple problem but the more documentation I read the more confused I became.Please advise I am using lambdas written in Python.Thank you for your helpFollowComment"

Answer: "0Hello,Based on the limited info provided, I can distinguish two parts to this question; one being the authentication (where tokens kick in) and another part of access (where we can use the API Gateway along with the lambda function)If you are looking to generate a token via an IdP, Cognito can be used, which in its ecosystem with User Pool and an App Client will enable you to retrieve Access and ID Tokens upon successful authentication. You can refer the below link to learn more about using tokens with Cognito.[+] Using tokens with user poolsYou can in turn use these tokens via an API Gateway to grant access to your own API or lambda function. Additionally,API gateway can be used to access a protected API through lambda authorisers as well if that suits more to your use case. Links to integrate these can be found below:[+] Use API Gateway Lambda authorizers[+] Control access to a REST API using Amazon Cognito user pools as authorizer[+] Accessing resources with API Gateway and Lambda after sign-inThat being said, to further look into this issue more closely, if you would like to do a resource based troubleshooting, please raise a support case with AWS for further information. If a support case has already been created please be assured that we will get back to you and assist you in the best way possible.https://console.aws.amazon.com/support/home#/case/createCommentSharerePost-User-2616680answered a month ago"


Question: o110.pyWriteDynamicFrame error in AWS Glue

"Hi there,I used the same script for another table and got the error in AWS Glue "An error occurred while calling o110.pyWriteDynamicFrame. Exception thrown in awaitResult:" I have checked the script a few times and re-created the logic from scratch too.Could anyone help me to understand where the issue is?Script is belowimport sysfrom awsglue.transforms import *from awsglue.utils import getResolvedOptionsfrom pyspark.context import SparkContextfrom awsglue.context import GlueContextfrom awsglue.job import Job@params: [TempDir, JOB_NAME]args = getResolvedOptions(sys.argv, ['TempDir','JOB_NAME'])sc = SparkContext()glueContext = GlueContext(sc)spark = glueContext.spark_sessionjob = Job(glueContext)job.init(args['JOB_NAME'], args)@type: DataSource@args: [database = "csvdatafroms3 - customer.io - metrics", table_name = "customer_io", transformation_ctx = "datasource0"]@return: datasource0@inputs: []datasource0 = glueContext.create_dynamic_frame.from_catalog(database = "csvdatafroms3 - customer.io - metrics", table_name = "customer_io", transformation_ctx = "datasource0")@type: ApplyMapping@args: [mapping = [("event_id", "string", "event_id", "string"), ("workspace_id", "int", "workspace_id", "int"), ("delivery_id", "string", "delivery_id", "string"), ("metric", "string", "metric", "string"), ("created_at", "timestamp", "created_at", "timestamp"), ("reason", "string", "reason", "string"), ("link_id", "int", "link_id", "int"), ("link_url", "string", "link_url", "string")], transformation_ctx = "applymapping1"]@return: applymapping1@inputs: [frame = datasource0]applymapping1 = ApplyMapping.apply(frame = datasource0, mappings = [("event_id", "string", "event_id", "string"), ("workspace_id", "int", "workspace_id", "int"), ("delivery_id", "string", "delivery_id", "string"), ("metric", "string", "metric", "string"), ("created_at", "timestamp", "created_at", "timestamp"), ("reason", "string", "reason", "string"), ("link_id", "int", "link_id", "int"), ("link_url", "string", "link_url", "string")], transformation_ctx = "applymapping1")@type: SelectFields@args: [paths = ["workspace_id", "reason", "event_id", "metric", "delivery_id", "created_at", "link_url", "link_id"], transformation_ctx = "selectfields2"]@return: selectfields2@inputs: [frame = applymapping1]selectfields2 = SelectFields.apply(frame = applymapping1, paths = ["workspace_id", "reason", "event_id", "metric", "delivery_id", "created_at", "link_url", "link_id"], transformation_ctx = "selectfields2")@type: ResolveChoice@args: [choice = "MATCH_CATALOG", database = "redshiftschemaall", table_name = "dev_customerionew_metrics", transformation_ctx = "resolvechoice3"]@return: resolvechoice3@inputs: [frame = selectfields2]resolvechoice3 = ResolveChoice.apply(frame = selectfields2, choice = "MATCH_CATALOG", database = "redshiftschemaall", table_name = "dev_customerionew_metrics", transformation_ctx = "resolvechoice3")@type: ResolveChoice@args: [choice = "make_cols", transformation_ctx = "resolvechoice4"]@return: resolvechoice4@inputs: [frame = resolvechoice3]resolvechoice4 = ResolveChoice.apply(frame = resolvechoice3, choice = "make_cols", transformation_ctx = "resolvechoice4")@type: DataSink@args: [database = "redshiftschemaall", table_name = "dev_customerionew_metrics", redshift_tmp_dir = TempDir, transformation_ctx = "datasink5"]@return: datasink5@inputs: [frame = resolvechoice4]datasink5 = glueContext.write_dynamic_frame.from_catalog(frame = resolvechoice4, database = "redshiftschemaall", table_name = "dev_customerionew_metrics", redshift_tmp_dir = args["TempDir"], transformation_ctx = "datasink5")job.commit()FollowComment"

Answer: 0Did you fix this?CommentSharedinganswered 2 years ago


Question: Invoke lamda async from Aurora PostgreSQL

"A customer has setup Aurora PostgreSQL and any time a CDC occurs they need to trigger lambda function. e.g like in this Blog https://aws.amazon.com/blogs/database/capturing-data-changes-in-amazon-aurora-using-aws-lambda/However there doesnt seem to invoke lambda from PostgreSQL..?If there's none, how else have any of your customers accomplished this requirement?FollowComment"

Answer: "0Accepted AnswerCalling Lambda from Aurora PostgreSQL is not yet supported. Rather than calling a lambda function in a trigger, you can consider consuming the change stream from PostgreSQL in a lambda function. See: https://aws.amazon.com/blogs/database/stream-changes-from-amazon-rds-for-postgresql-using-amazon-kinesis-data-streams-and-aws-lambda/Think about the architecture and volume. Do they want to see every change, or only changes on a certain table?How many changes per minute are they expecting?If they are looking for the all changes or a high volume of changes, they may prefer using a small EC2 instance and a CDC tool like Debezium which can consume CDC records from PostgreSQL and publish to various targets. See:  https://debezium.io/blog/2020/02/25/lessons-learned-running-debezium-with-postgresql-on-rds/or: https://aws.amazon.com/blogs/apn/how-behalf-met-its-streaming-data-scaling-demands-with-amazon-managed-streaming-for-apache-kafka/CommentShareAWS-User-0257939answered 3 years ago0Accepted AnswerThis feature was launched in December 2020: https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-aurora-postgresql-integrates-with-aws-lambda/CommentShareigizotovanswered 2 years ago"


Question: PrivateLink to S3 from Azure

A customer is running DataBricks on Azure and want to read/write data to S3 without traversing the public internet to S3. What are the possible options for traffic to not traverse over public internet?FollowComment

Answer: "0Accepted AnswerVPN or use Direct Connect + express route + intermediary router (partner) to create private connectivity from azure to Amazon VPC, then just setup a privatelink endpoint for S3 and use whatever mechanism in Azure to point <region>.s3.amazonaws.com -> the privatelink endpoint IPsCommentShareEXPERTScottanswered 2 years ago"


Question: Using HuggingFace in Sagemaker Studio as part of a project

"TLDR: if we are trying to use a HuggingFaceProcessor/Estimator in a Sagemaker Studio project, what are the requirements for the train.py file in terms of how it refers to the assembled training data, and where it should store the results of the operations it performs( e.g. compiled model, datae etc.)FULL DETAILSSo our high level goal is to be able to deploy some kind of non-XGB model from a sagemaker studio project, given that the templates provided are all XGB.  As outlined in an earlier question we'd started with TensorFlow, but since our TensorFlow model wraps a HuggingFace model we thought let's try something even simpler, just a HuggingFace model using the HuggingFaceProcessor.So following docs on HuggingFaceProcessor and a HuggingFace Estimator example we started to adjust the abalone (project template) pipeline.py to look like this (full code can be provided on request):    # processing step for feature engineering    hf_processor = HuggingFaceProcessor(        role=role,         instance_count=processing_instance_count,        instance_type=processing_instance_type,        transformers_version='4.4.2',        pytorch_version='1.6.0',         base_job_name=f"{base_job_prefix}/frameworkprocessor-hf",        sagemaker_session=pipeline_session,    )    step_args = hf_processor.run(        outputs=[            ProcessingOutput(output_name="train", source="/opt/ml/processing/train"),            ProcessingOutput(output_name="validation", source="/opt/ml/processing/validation"),            ProcessingOutput(output_name="test", source="/opt/ml/processing/test"),        ],        code=os.path.join(BASE_DIR, "preprocess.py"),        arguments=["--input-data", input_data],    )    step_process = ProcessingStep(        name="PreprocessTopicData",        step_args=step_args,    )    # training step for generating model artifacts    model_path = f"s3://{sagemaker_session.default_bucket()}/{base_job_prefix}/TopicTrain"    hf_train = HuggingFace(entry_point='train.py',                            source_dir=BASE_DIR,                            base_job_name='huggingface-sdk-extension',                            instance_type=processing_instance_type,                            instance_count=processing_instance_count,                            transformers_version='4.4',                            pytorch_version='1.6',                            py_version='py36',                            role=role,                          )      hf_train.set_hyperparameters(       epochs=3,       train_batch_size=16,       learning_rate=1.0e-5,       model_name='distilbert-base-uncased',    )                               step_args = hf_train.fit(        inputs={            "train": TrainingInput(                s3_data=step_process.properties.ProcessingOutputConfig.Outputs[                    "train"                ].S3Output.S3Uri,                content_type="text/csv",            ),            "validation": TrainingInput(                s3_data=step_process.properties.ProcessingOutputConfig.Outputs[                    "validation"                ].S3Output.S3Uri,                content_type="text/csv",            ),        },    )Finding that pushing to master doesn't provide any feedback on issues arising from pipeline.py, we realised that trying to get the pipeline from a notebook was a better way of debugging these sorts of changes, assuming one remembered to restart the kernel each time to ensure changes to the pipeline.py file was available to the notebook.So using the following code in the notebook we worked through a series of issues trying to bash the code into shape such that it would compile:from pipelines.topic.pipeline import get_pipelinepipeline = get_pipeline(    region=region,    role=role,    default_bucket=default_bucket,    model_package_group_name=model_package_group_name,    pipeline_name=pipeline_name,)We needed to change the default processing and training instance types to avoid a "cpu" unsupported issue:    processing_instance_type="ml.p3.xlarge",    training_instance_type="ml.p3.xlarge",and add a train.py script:from transformers import AutoTokenizerfrom transformers import TFAutoModelForSequenceClassificationtokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")model = TFAutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased", num_labels=18)import pandas as pdimport tensorflow as tffrom sklearn.model_selection import train_test_splitfrom transformers import (    DistilBertTokenizerFast,    TFDistilBertForSequenceClassification,)DATA_COLUMN = 'text'LABEL_COLUMN = 'label'MAX_SEQUENCE_LENGTH = 512LEARNING_RATE = 5e-5BATCH_SIZE = 16NUM_EPOCHS = 3NUM_LABELS = 15if __name__ == "__main__":    # --------------------------------------------------------------------------------    # Tokenizer    # --------------------------------------------------------------------------------    tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased')    def tokenize(sentences, max_length=MAX_SEQUENCE_LENGTH, padding='max_length'):        """Tokenize using the Huggingface tokenizer        Args:            sentences: String or list of string to tokenize            padding: Padding method ['do_not_pad'|'longest'|'max_length']        """        return tokenizer(            sentences,            truncation=True,            padding=padding,            max_length=max_length,            return_tensors="tf"        )    # --------------------------------------------------------------------------------    # Load data    # --------------------------------------------------------------------------------    from keras.utils import to_categorical    from sklearn.preprocessing import LabelEncoder    labelencoder_Y_1 = LabelEncoder()    yy = labelencoder_Y_1.fit_transform(train_data[LABEL_COLUMN].tolist())    yy = to_categorical(yy)    print(len(yy))    print(yy.shape)    train_dat, validation_dat, train_label, validation_label = train_test_split(        train_data[DATA_COLUMN].tolist(),        yy,        test_size=0.2,        shuffle=True    )    # --------------------------------------------------------------------------------    # Prepare TF dataset    # --------------------------------------------------------------------------------    train_dataset = tf.data.Dataset.from_tensor_slices((        dict(tokenize(train_dat)),  # Convert BatchEncoding instance to dictionary        train_label    )).shuffle(1000).batch(BATCH_SIZE).prefetch(1)    validation_dataset = tf.data.Dataset.from_tensor_slices((        dict(tokenize(validation_dat)),        validation_label    )).batch(BATCH_SIZE).prefetch(1)    # --------------------------------------------------------------------------------    # training    # --------------------------------------------------------------------------------    model = TFDistilBertForSequenceClassification.from_pretrained(        'distilbert-base-uncased',        num_labels=NUM_LABELS    )    optimizer = tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE)    model.compile(        optimizer=optimizer,        loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),    )However we are now stuck on this error when trying to get the pipeline from a notebook.<ipython-input-3-be38b3dda75f> in <module>      7     default_bucket=default_bucket,      8     model_package_group_name=model_package_group_name,----> 9     pipeline_name=pipeline_name,     10 )     11 # !conda list~/topic-models-no-monitoring-p-rboparx6tdeg/sagemaker-topic-models-no-monitoring-p-rboparx6tdeg-modelbuild/pipelines/topic/pipeline.py in get_pipeline(region, sagemaker_project_arn, role, default_bucket, model_package_group_name, pipeline_name, base_job_prefix, processing_instance_type, training_instance_type)    228                     "validation"    229                 ].S3Output.S3Uri,--> 230                 content_type="text/csv",    231             ),    232         },/opt/conda/lib/python3.7/site-packages/sagemaker/workflow/pipeline_context.py in wrapper(*args, **kwargs)    246             return self_instance.sagemaker_session.context    247 --> 248         return run_func(*args, **kwargs)    249     250     return wrapper/opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)   1059         self._prepare_for_training(job_name=job_name)   1060 -> 1061         self.latest_training_job = _TrainingJob.start_new(self, inputs, experiment_config)   1062         self.jobs.append(self.latest_training_job)   1063         if wait:/opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in start_new(cls, estimator, inputs, experiment_config)   1956         train_args = cls._get_train_args(estimator, inputs, experiment_config)   1957 -> 1958         estimator.sagemaker_session.train(**train_args)   1959    1960         return cls(estimator.sagemaker_session, estimator._current_job_name)/opt/conda/lib/python3.7/site-packages/sagemaker/session.py in train(self, input_mode, input_config, role, job_name, output_config, resource_config, vpc_config, hyperparameters, stop_condition, tags, metric_definitions, enable_network_isolation, image_uri, algorithm_arn, encrypt_inter_container_traffic, use_spot_instances, checkpoint_s3_uri, checkpoint_local_path, experiment_config, debugger_rule_configs, debugger_hook_config, tensorboard_output_config, enable_sagemaker_metrics, profiler_rule_configs, profiler_config, environment, retry_strategy)    611             self.sagemaker_client.create_training_job(**request)    612 --> 613         self._intercept_create_request(train_request, submit, self.train.__name__)    614     615     def _get_train_request(  # noqa: C901/opt/conda/lib/python3.7/site-packages/sagemaker/session.py in _intercept_create_request(self, request, create, func_name)   4303             func_name (str): the name of the function needed intercepting   4304         """-> 4305         return create(request)   4306    4307 /opt/conda/lib/python3.7/site-packages/sagemaker/session.py in submit(request)    608         def submit(request):    609             LOGGER.info("Creating training-job with name: %s", job_name)--> 610             LOGGER.debug("train request: %s", json.dumps(request, indent=4))    611             self.sagemaker_client.create_training_job(**request)    612 /opt/conda/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)    236         check_circular=check_circular, allow_nan=allow_nan, indent=indent,    237         separators=separators, default=default, sort_keys=sort_keys,--> 238         **kw).encode(obj)    239     240 /opt/conda/lib/python3.7/json/encoder.py in encode(self, o)    199         chunks = self.iterencode(o, _one_shot=True)    200         if not isinstance(chunks, (list, tuple)):--> 201             chunks = list(chunks)    202         return ''.join(chunks)    203 /opt/conda/lib/python3.7/json/encoder.py in _iterencode(o, _current_indent_level)    429             yield from _iterencode_list(o, _current_indent_level)    430         elif isinstance(o, dict):--> 431             yield from _iterencode_dict(o, _current_indent_level)    432         else:    433             if markers is not None:/opt/conda/lib/python3.7/json/encoder.py in _iterencode_dict(dct, _current_indent_level)    403                 else:    404                     chunks = _iterencode(value, _current_indent_level)--> 405                 yield from chunks    406         if newline_indent is not None:    407             _current_indent_level -= 1/opt/conda/lib/python3.7/json/encoder.py in _iterencode_dict(dct, _current_indent_level)    403                 else:    404                     chunks = _iterencode(value, _current_indent_level)--> 405                 yield from chunks    406         if newline_indent is not None:    407             _current_indent_level -= 1/opt/conda/lib/python3.7/json/encoder.py in _iterencode(o, _current_indent_level)    436                     raise ValueError("Circular reference detected")    437                 markers[markerid] = o--> 438             o = _default(o)    439             yield from _iterencode(o, _current_indent_level)    440             if markers is not None:/opt/conda/lib/python3.7/json/encoder.py in default(self, o)    177     178         """--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '    180                         f'is not JSON serializable')    181 TypeError: Object of type ParameterInteger is not JSON serializableWhich is telling us that some aspect of the training job (?) is not serializable, and it's not clear how to debug further.What would be enormously helpful is project templates for sagemaker studio showing the use of all the Processors, e.g. HuggingFace, TensorFlow and so on, but failing that we'd be most grateful is anyone could point us to documentation on what the requirements are for the train.py file that we need to specifiy for the HuggingFace Estimator.many thanks in advanceFollowComment"

Answer: "0Hi, Not sure I can get you all the way to solution either but I think here are some more useful tips:Instance type: To my knowledge ml.p3.xlarge does not exist - you'll probably want to look at ml.g4dn.xlarge or ml.p3.2xlargeAvoiding kernel restarts:It is possible to make Jupyter pick up changes you make to local files on-the-fly instead of having to restart the kernel each time. Just add the following lines to the top of your notebook before you run any imports: The autoreload extension will then reload modules each time before running your code.%load_ext autoreload%autoreload 2Framework Processors with Pipelines:As you might already be aware, there were some issues using Pipelines when FrameworkProcessor (which HuggingFaceProcessor, TensorFlowProcessor, etc are built on) was first launched. I believe these should now be fixed, but do require you to be using the pipeline_session syntax - I see you already are, so that's great.Just in case you're seeing any echoes of this, would maybe recommend to try demonstrating pipeline creation first without the processing job (don't necessarily need to have it working properly end-to-end), and then adding the processing job in. Do be aware you might come across some older samples that haven't been updated yet to use PipelineSession, and these may not work properly with your DL Framework Processors. (I think Heiko's sample above might pre-date this).Pipeline JSON error:It's hard to tell what's up for sure because seems like there are some gaps in your code (e.g. is processing_instance_count a plain number or a pipeline parameter? Either way, seems like something is going wrong at the actual pipeline definition stage rather than trying to execute it. I'd suggest to simplify and gradually build up: E.g. hard-coding pipeline parameters to plain values, building the pipeline with just a subset of steps, etc.HF processing and training scripts:Getting your pipeline working should hopefully be largely separate from (and parallelizable with) getting your individual training/processing/inference jobs running as expected. You'll find more information about the requirements for your script in the Hugging Face section of the SageMaker Python SDK doc.I'd maybe point to this example training script for sequence classification, which shows the standard pattern of using argparse and SM_MODEL_DIR to find what local folder your script should save the trained model to. That same repository has many examples showing other features and use cases too... But from a quick check most/all of them don't seem to take data channel inputs. This one shows how input data (local folders) are also passed through the CLI/environment variables to your script.To summarize:Your script receives input and output locations (local folders) through CLI variables with environment variable fallbacks - e.g. --foo and SM_CHANNEL_FOO if you run a job like estimator.fit({"foo": "s3://.../..."})Hyperparameters are also received through CLI and/or the SM_HPS JSON environment variableMake sure to save your model to the SM_MODEL_DIR folderif you want to output metrics, just use print() or logging on the script side and then define regular expressions on the pipeline side, to tell SageMaker how to scrape them from the logsHere is a quick overview from HF, another introductory example, and a really over-engineered sample if you want to see what a very complex use case might look like 🥲 Hope this helps!CommentShareEXPERTAlex_Tanswered 8 months agoregulatansaku 7 months agohi @Alex_T, thanks so much for your detailed response - that's been really helpful.  Sorry for not thanking you sooner - we had to move away from Studio to get something else working as a stop gap, but now we're looking at it again.Thanks to your help and the post from @Heiko we've moved forward to the point of starting to get a train.py working.  We're now stuck on a new point regarding saving DataSets.  For XGB we had been saving the training, test, and validation data into csvs.  All the train.py examples work with load_from_disk operations that expect a DataSet format.  In our preprocess.py we can't load tensorflow or DataSet from datasets, so what would be of great help is an example of a preprocess.py script that worked with DataSets.The particular difficulty we're finding with SageMaker Studio is working out exactly which versions of which libraries are running within these scripts ...Share0Hi thereI'm not 100% sure what caused the error that you're seeing, but since you mentioned that a Hugging Face (HF) Pipeline example could be useful, I wanted to share this project I developed a while ago where we use all the HF component in a Sagemaker Pipeline: https://github.com/marshmellow77/ade-pipeline/tree/main/ade-modelbuild/pipelinesThe original pipeline definition is in teh abalone folder, and the new one in the ade folder (ade = Adverse Drug Event).CheersHeikoCommentShareHeikoanswered 8 months ago"


Question: IAM Policy based on office hours

"Hi,We need create a policy that allow access based on office hours, looking the policies documentation, we found **DateGreaterThan ** and DateLessThan condition operators, but this operators works with specific dates.Any option to works with hours range?Regards!FollowComment"

Answer: 0As far as I’m aware this is not possible. Date and time is only possible.That would be a nice feature.CommentShareGary Mcleananswered 25 days ago


Question: There is no that file!

"I am a new beginner of lumberyard, and I started learning from "Getting Started video". But I found that my file folder doesn't have the file he use in the video.(VIDEO)REMOVEDUPLOAD(MINE)REMOVEDUPLOADIn the video, he has 'C:\Amazon\Lumberyard\1.25.0.0\dev\StarterGame\Textures\HeightMap\FTUE_heightmap_test.tif', but I don't have folder 'HeightMap' inside 'Texture'.Please give me some help!!!Follow"

Answer: "0Accepted AnswerPlease find the updated heightmap file in the video description below the video:REMOVEDUPLOADThis should help you move forward. Please let me know if it doesn't.Find a link to the video here, then click on the download link in the video description or here.Then drag it into your textures folder for your project and let the asset processor find it. Then you should be good to go.SharerePost-User-8914159answered 3 years ago0Hi @REDACTEDUSERSorry to hear about this issue :(I had a quick search and found that the file in question exists up until the 1.24 release and for some reason in 1.25 it is no longer there. I don't know why it was removed but I can try and reach out to the team who'll know tomorrow.Thanks for your patience,TomSharerePost-User-0765265answered 3 years ago0Hi @REDACTEDUSERIs it possible , help me about how to make a water ripple effect component ? I see this component in editor but does not work for 1.24.0 , I see ocean component in 1.24.0 and does work without problem. 🙂Thank you 🙏 🌹https://forums.awsgametech.com/t/where-is-water-ripple-effect-component-in-gem-ly-1-23/8108/6SharerePost-User-8224505answered 3 years ago0Thank you for help! 😀SharerePost-User-8733796answered 3 years ago0@REDACTEDUSERIn version 1.25 we have introduced new StarterGame content. You can find the content in dev\Gems\StarterGame. We are also working on updated tutorial content. That particular file is not part of the new content. You can still learn a bit from the video, you will just have to use another map or generate a terrain.You can use any grayscale tif file as a heightmap or auto generate terrain by selecting Generate Terrain from the tools menu in Terrain Editor. A properties window will open allowing you to set some inputs for the terrain generator. The terrain generator will use those properties and some random seeds to generate a natural looking heightmap that you can refine in the terrain editor and with the terrain tool.Also note the Infinite Ocean component is not included by default when creating a new level. You can select the DefaultLevelSetup entity in the Entity Outliner and add the Infinite Ocean component to it in the Entity Inspector if you would like to see water, as shown in the video.SharerePost-User-6034607answered 3 years ago0@REDACTEDUSERI tried to look at the Water ripple component yesterday. See my reply in you thread.SharerePost-User-6034607answered 3 years ago0OK @REDACTEDUSERCan you explain more and what was the result?Is this a bug or a lack of water ripple effect gem compilation? 😊SharerePost-User-8224505answered 3 years ago0In GSG 007 'Terrain Creation - Import Terrain mega textures', he use a file BMP to make a green ground, but there is no a BMP in my folder. How can I do?SharerePost-User-8733796answered 3 years ago0@REDACTEDUSERThe GSG has been retired and will be replaced with a new guide.  Because the starter game content has been updated, many of the assets referenced in the retired guides not longer exist. You can still work through many of the tutorials, but will have to use alternative assets, possibly provide your own. In this case, you can use any image file, but I have attached the missing asset to this post so you can get an idea what mega terrain textures are supposed to look like.REMOVEDUPLOADSharerePost-User-6034607answered 3 years ago0Where can I see the new guide?SharerePost-User-8733796answered 3 years ago0The guide is currently being created, we do not have a release date.SharerePost-User-6034607answered 3 years ago"


Question: AWS Location Tracker - custom event-bus setting

"I am trying to control which event-bus an AWS Tracker goes onto. I looked at the API Docs and I did not see anything like an "event-bus" as part of it's parameters. In my testing it seems that the Event Bus rule uses the "default" bus every time for an "ENTER" or "EXIT" event. We use various event-bus' per environment. Is there any way I can configure my API call to point to a specific event-bus? Here is a sample snippet I am currently using:var paramsUpdateTracker = {  TrackerName: 'test-tracker',  Updates: [    {      DeviceId: 'fake-device-number',      Position: [-90.22222,41.888888],      SampleTime: new Date,    }  ]};  location.batchUpdateDevicePosition(paramsUpdateTracker, function(err: any, data: any) {  if (err) {    console.log(err, err.stack);  }  else     {    console.log(data);  }});FollowComment"

Answer: "1By default, events generated by AWS services get sent to the default event bus in Eventbridge. The custom event bus will be used for custom code (if you use the Eventbridge APIs to send events from your code). It is however possible to send an event from one bus to another - https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-bus-to-bus.htmlCommentShareEXPERTIndranil Banerjee AWSanswered a year agojim_b a year agoSo it sounds like all ENTER/EXIT events for AWS Location will always be routed to the "default" bus, and there is no way to programmatically put that event on a specific EventBridge bus. I was hoping to avoid a scenario like this, but sounds like we may have no other choice. Just to be clear, what you are suggesting is that all Location events for geofence ENTER/EXIT will go to the default bus, and from there we can route traffic to the proper environment EventBridge bus (dev/qa/production/etc...)?Share0On the last sentence, are you using the same AWS account for all your environments? This is not a recommended practice. Ideally, all environments should be in their own AWS accounts. Sometimes customers do use the same AWS account for lower environments such as Dev and Test, but then use separate accounts for Pre-prod and Prod. At the minimum, production accounts should be separate from non-production accounts. Even if you share the same AWS account between different environments, the workloads should be kept separate. That is however difficult if you are using Eventbridge default event bus because there can be only one default event bus in one AWS account. How would you know if an event on the default event bus is meant for the Dev environment or the Test environment? This needs to be thought through more.CommentShareEXPERTIndranil Banerjee AWSanswered a year agojim_b a year agoWe actually do have a separate account for production and one for lower environments. The account for lower environments includes development/qa/stage. We do currently use EDA for other use cases. The EventBridge API is used which includes the actual name of the target bus. Here is a sample setup:    const entry: EventBridge.PutEventsRequestEntry = {      Source: <domainName>,      DetailType: <eventName>,      EventBusName: <busName>,      Detail: JSON.stringify(request),    };Share0Reacting to Amazon Location Service events with Amazon EventBridge : https://docs.aws.amazon.com/location/latest/developerguide/location-events.htmlTracking using MQTT with Amazon Location Service :  https://docs.aws.amazon.com/location/latest/developerguide/tracking-using-mqtt.htmlCommentShareDeveshanswered a year ago"


Question: Pixel Streaming Hosting on AWS

"Deploying Pixel Streaming application on AWS Windows server, and getting some roadblocks when accessing the app remotely. Locally it loads well but remotely gives this error on both Firefox Mozilla and Google chrome."Starting connecting to the server, Please wait"The logs statesLog file open, 06/16/22 20:08:00LogWindows: Failed to load 'aqProf.dll' (GetLastError=126)LogWindows: File 'aqProf.dll' does not existLogWindows: Failed to load 'VtuneApi.dll' (GetLastError=126)LogWindows: File 'VtuneApi.dll' does not existLogWindows: Failed to load 'VtuneApi32e.dll' (GetLastError=126)LogWindows: File 'VtuneApi32e.dll' does not existLogConsoleResponse: Display: Failed to find resolution value strings in scalability ini. Falling back to default.LogConsoleResponse: Display: Failed to find resolution value strings in scalability ini. Falling back to default.LogPakFile: Display: Found Pak file ../../../pixel/Content/Paks/pixel-WindowsNoEditor.pak attempting to mount.LogPakFile: Display: Mounting pak file ../../../pixel/Content/Paks/pixel-WindowsNoEditor.pak.LogPakFile: PakFile PrimaryIndexSize=26482LogPakFile: PakFile PathHashIndexSize=45743LogPakFile: PakFile FullDirectoryIndexSize=56410LogShaderLibrary: Display: ShaderCodeLibraryPakFileMountedCallback: PakFile '../../../pixel/Content/Paks/pixel-WindowsNoEditor.pak' (chunk index -1, root '../../../') mountedLogShaderLibrary: Display: ShaderCodeLibraryPakFileMountedCallback: pending pak file info (ChunkID:-1 Root:../../../ File:../../../pixel/Content/Paks/pixel-WindowsNoEditor.pak)LogPakFile: OnPakFileMounted2Time == 0.000066LogPlatformFile: Not using cached read wrapperLogTaskGraph: Started task graph with 5 named threads and 11 total threads with 3 sets of task threads.LogStats: Stats thread started at 0.131755LogICUInternationalization: ICU TimeZone Detection - Raw Offset: +0:00, Platform Override: ''LogInit: Display: Loading text-based GConfig....LogPluginManager: Mounting plugin PixelStreamingLogPluginManager: Mounting plugin WmfMediaLogPluginManager: Mounting plugin HardwareEncodersLogPluginManager: Mounting plugin Paper2DLogPluginManager: Mounting plugin AISupportLogPluginManager: Mounting plugin EnvironmentQueryEditorLogPluginManager: Mounting plugin LightPropagationVolumeLogPluginManager: Mounting plugin CameraShakePreviewerLogPluginManager: Mounting plugin GameplayCamerasLogPluginManager: Mounting plugin TemplateSequenceLogPluginManager: Mounting plugin LevelSequenceEditorLogPluginManager: Mounting plugin OodleDataLogPluginManager: Mounting plugin OodleNetworkLogPluginManager: Mounting plugin AnimationSharingLogPluginManager: Mounting plugin SignificanceManagerLogPluginManager: Mounting plugin PixWinPluginLogPluginManager: Mounting plugin PluginUtilsLogPluginManager: Mounting plugin PropertyAccessEditorLogPluginManager: Mounting plugin UObjectPluginLogPluginManager: Mounting plugin AssetManagerEditorLogPluginManager: Mounting plugin FacialAnimationLogPluginManager: Mounting plugin GeometryModeLogPluginManager: Mounting plugin SpeedTreeImporterLogPluginManager: Mounting plugin DatasmithContentLogPluginManager: Mounting plugin VariantManagerContentLogPluginManager: Mounting plugin AlembicImporterLogPluginManager: Mounting plugin GeometryCacheLogPluginManager: Mounting plugin AutomationUtilsLogPluginManager: Mounting plugin ScreenshotToolsLogPluginManager: Mounting plugin BackChannelLogPluginManager: Mounting plugin ChaosClothEditorLogPluginManager: Mounting plugin ChaosClothLogPluginManager: Mounting plugin ChaosEditorLogPluginManager: Mounting plugin PlanarCutLogPluginManager: Mounting plugin GeometryProcessingLogPluginManager: Mounting plugin EditableMeshLogPluginManager: Mounting plugin GeometryCollectionPluginLogPluginManager: Mounting plugin ProceduralMeshComponentLogPluginManager: Mounting plugin ChaosSolverPluginLogPluginManager: Mounting plugin ChaosNiagaraLogPluginManager: Mounting plugin NiagaraLogPluginManager: Mounting plugin PythonScriptPluginLogPluginManager: Mounting plugin CharacterAILogPluginManager: Mounting plugin MotoSynthLogPluginManager: Mounting plugin OpenImageDenoiseLogPluginManager: Mounting plugin PlatformCryptoLogPluginManager: Mounting plugin MagicLeapPassableWorldLogPluginManager: Mounting plugin MagicLeapLogPluginManager: Mounting plugin MLSDKLogPluginManager: Mounting plugin MagicLeapLightEstimationLogPluginManager: Mounting plugin AvfMediaLogPluginManager: Mounting plugin ImgMediaLogPluginManager: Mounting plugin MediaCompositingLogPluginManager: Mounting plugin MeshPaintingLogPluginManager: Mounting plugin TcpMessagingLogPluginManager: Mounting plugin UdpMessagingLogPluginManager: Mounting plugin ActorSequenceLogPluginManager: Mounting plugin MatineeToLevelSequenceLogPluginManager: Mounting plugin OnlineSubsystemNullLogPluginManager: Mounting plugin OnlineSubsystemLogPluginManager: Mounting plugin OnlineSubsystemUtilsLogPluginManager: Mounting plugin LauncherChunkInstallerLogPluginManager: Mounting plugin ActorLayerUtilitiesLogPluginManager: Mounting plugin AndroidPermissionLogPluginManager: Mounting plugin AppleImageUtilsLogPluginManager: Mounting plugin ArchVisCharacterLogPluginManager: Mounting plugin AssetTagsLogPluginManager: Mounting plugin AudioCaptureLogPluginManager: Mounting plugin CableComponentLogPluginManager: Mounting plugin ChunkDownloaderLogPluginManager: Mounting plugin CustomMeshComponentLogPluginManager: Mounting plugin ExampleDeviceProfileSelectorLogPluginManager: Mounting plugin GooglePADLogPluginManager: Mounting plugin LocationServicesBPLibraryLogPluginManager: Mounting plugin MobilePatchingUtilsLogPluginManager: Mounting plugin OculusVRLogPluginManager: Mounting plugin OpenXREyeTrackerLogPluginManager: Mounting plugin OpenXRLogPluginManager: Mounting plugin OpenXRHandTrackingLogPluginManager: Mounting plugin LiveLinkLogPluginManager: Mounting plugin TakesLogPluginManager: Mounting plugin PhysXVehiclesLogPluginManager: Mounting plugin RuntimePhysXCookingLogPluginManager: Mounting plugin SoundFieldsLogPluginManager: Mounting plugin SteamVRLogPluginManager: Mounting plugin SynthesisLogPluginManager: Mounting plugin AudioSynesthesiaLogPluginManager: Mounting plugin WebMMoviePlayerHowever, I'm having an issue:The ice candidates are successfully sent to each peer.The host's .iceConnectionState becomes checkingThe .iceGatheringState of both become completeThe remote's .iceConnectionState becomes failedA console.error appears WebRTC: ICE failed, add a TURN server and see about:webrtc for more detailsI'm using a public google STUN server, which I'm fairly sure is enough.Have tried to install, still doesn't work.Does anyone know why this error is occurring? I appreciate the effortFollowComment"


Question: S3 bandwidth limit

"A customer is managing the cloud infrastructure for a large company. Recently they had an issue of a fairly large chunk of the bill caused by company employees downloading data from S3. The customer is asking how can he restrict:S3 bandwidth used per userAmount of data downloaded per userAFAIK, there are no native options to achieve this. The possible solutions I thought of:Bandwidth can be restricted by using a proxy server on the customer end (capable of SSL interception)Amount of data:can be potentially reported on using customer proxy reporting servicescollect the bucket access logs and send them to ElasticSearch / Splunk. Then use their reporting capabilities to report on top S3 download users. Based on the report results, contact the user to find out the reason.Any other options I am missing?[EDIT]They are using AWS CLI and SDKs to get the data, so CloudFront is less suitable. DX is an option to reduce costs.FollowComment"

Answer: "0Accepted AnswerYou are correct, there is no way to control this natively within AWS. If they are downloading content from S3, I would look at using Direct Connect or CloudFront to reduce the data transfer costs, if possible. These may reduce costs enough that it is no longer a concern.If it still is a concern, I would consider using the access logs to calculate the amount of data transfer by each user/role and then internally-charge the teams. This will give them the responsibility to manage their usage, without imposing a hard limit.CommentShareEXPERTLeeAtk@AWSanswered 3 years ago"


Question: How to configure AWS Client VPN users to have outbound Elastic (fixed) IP when connecting to resources outside of the Client VPN associated VPC?

"A customer wants to know if there is a way to for the client VPN to get a static public IP?The customer needs a static IP to connect to a remote app that uses IP filtering to allow access only from a specific IP.The customer is using this documentation as the starting point: https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/scenario-onprem.html. The want to route all traffic, both from clients and on-premises, through the VPN Interface where they can allowlist specific IPs.FollowComment"

Answer: "0Accepted AnswerJust got this exact question from a customer as well. The customer needs to allowlist their employees ip addresses (on client VPN) for accessing partner data. In this case it looks like you would have to traverse your traffic through NAT Gateway + EIPYou can find an example architecture in this blog, the first example under "Client VPN to Internet"https://aws.amazon.com/blogs/networking-and-content-delivery/using-aws-client-vpn-to-scale-your-work-from-home-capacity/also relatedhttps://aws.amazon.com/blogs/networking-and-content-delivery/introducing-aws-client-vpn-to-securely-access-aws-and-on-premises-resources/CommentShareEXPERTjohn_lanswered 3 years agoelbigbadmo a year agohmmm...didn't work for me.Share0Since the interfaces are being recreated periodically, the attached EIP are released new Public IP Address is assigned. How can we enusre the interface always use the specific EIP for outgoing traffic from AWS VPN Client interface.CommentSharerePost-User-5883452answered a year ago"


Question: EC2 freeze up

"Hi,I am running a ec2 instance with Node Red and InfluxDB.Lately every day or so the instance locks up and freezes, I go to the console and it seems fine.I reboot the instance and then will run a few hours to sometimes more than a day and freeze.Any ideas?Thank youJohanFollowComment"

Answer: "1Hi there,Thank you for replying :DAccording to the logs you have provided everything seems to check out although the nature of your problem “Freezing” are generally symptoms of an instances memory and I/O being overloaded or starving for resources.As a method of solving your problem you can try installing Linux tools like "atop" and log resource utilization every minute or two. If any process is consuming high CPU or memory and causing this issue, you can identify by analyzing "atop" logs from /var/log/atop which will show indications of what may be "starving"  your performance.As guidance I have listed an article below which will provide you insight on how to install  “atop” and monitor resource usage on your instance by assessing atop logs.[1]https://haydenjames.io/use-atop-linux-server-performance-analysis/I trust the information provided above will assist in resolving your query, however, should you have any further questions or concerns, please do not hesitate to reach out to me.CommentShareArtemisFanswered 2 years ago0Hi there,Thank you for posting your question here :)There are multiple reasons that may cause your instance to freeze up from time to timeFrom your description I would advise you to dig through your instances cloud watch logs to try to find out what exactly is going on.The problem you are facing could be due to multiple reasons but it would be good to start off with performance factors which needs to be assessed such as the EBS volumes metrics for resource saturation and even the CPU metrics.As a method of solving your problem I have listed steps below that will guide you in diagnosing and solving the problem you are facingUse a larger size, e.g. t2.small or t2.medium - if the freezing goes away it may be memory-related.Examine CPU Credits usage - if it "freezes" when the CPU credits go low you're overloading the instance. Use a larger one.Examine the OS logs - in /var/log/*. There may be some clues about memory or other problems.Once it "freezes" go to EC2 -> Instances -> Actions -> Instance Settings -> Get System Log and see if you get any clues there.If you are unable to access your logs due to the instance being unresponsive you can mount the instance's volume on a temporary instance for diagnostics.Please follow the below steps:[+] If your primary instance is running, stop it. You cannot perform the rest of this procedure on a running instance.[+] Check the integrity of your volume. File-system corruption that is picked up by the snapshot may render a restored root volume unbootable.[+] Take a snapshot of your volume. It can be easy to corrupt or lose your data in the following procedures. If you have a fresh snapshot, you can always start over in case of a mistake and your data will still be safe.[+] Record the device name that the volume is attached to. You can find this information on the Root device field of the instance's details pane. The value is likely /dev/sda1 or /dev/xvda.[+] Detach the volume from the primary instance.[+] Attach the volume to another (secondary) instance in the same Availability Zone. If your EBS volume is encrypted, you must use a secondary instance that supports Amazon EBS encryption; otherwise, you can use a t2 micro instance for this procedure. If you do not already have a secondary instance, you will need to launch oneIt is important to note the following:The secondary instance you launched must be running when you attach the volume, and you should not reboot the secondary instance while multiple root volumes are attached; booting an instance with multiple root volumes attached could cause the instance to boot to the wrong volume.[+] Log in to the secondary instance with SSH.[+] Identify the device that contains the partition that you want to expand. Use the lsblk command to list all devices and partitions attached to the instance.https://forums.aws.amazon.com/As an example of the output$ lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTxvdf 202:80 0 100G 0 disk└─xvdf1 202:81 0 8G 0 part /mntxvda1 202:1 0 30G 0 disk /[+] If the secondary volume (xvdf in the above example but could be different for you) is not mounted you need to do the following[+] Mount the Volume (Substitute xvdf1 with whatever comes up from the lsblk command above)mount /dev/xvdf1 /mntOnce done you can analyze the contents of the volume.If necessary you can make changes to this volume that will be persistent across reboots. Once done follow the above steps to move the volume back to the original instance. Insure you attach the volume with the device name you took note of above.Alternatively you can utilize a tool created by AWS called EC2 rescue which you can use to troubleshoot problems that may occur at Operating System level. I have included an article below that will guide you with the process.https://aws.amazon.com/premiumsupport/knowledge-center/ec2-instance-boot-issues/Please check if the resources suggested in the above link provides you with assistance regarding your problem. However, if things are still unclear, please feel free to refer with questions that you need to ask to gain further clarity. I trust the information provided above will assist in resolving your query.CommentShareArtemisFanswered 3 years ago0Hi,This is my System Log : i-0f8e88ccfe4aef0bb0 , can you please advise on what may cause the issue , I rebooted the ec2 about 30 min ago and it freedze about 2 hours ago.I write date at a InfulxDB about ever min, that is how I pick it up, the straight line is the period the ec2 were frozen.https://drive.google.com/file/d/1xLruJe5PxeA4t0qv68XievVbuUJmjIbF/view?usp=sharinghttps://drive.google.com/file/d/1LQOnmZr4qoMoua5fLRX9SxpQOn4By024/view?usp=sharingCommentShareJohanScheepersanswered 2 years ago"


Question: aft account customisation resources not created in target account

"AFT Version: 1.9.1terraform version: 0.15.5terraform providers: AWSDescription:-We have deployed control tower and AFT for terraform in a separate AFT account using Terraform, aft version 1.9.1. After deploying aft new account request is working fine,  it is running pipeline for creating the account whenever we add new account request terraform code in our AFT account request repository.But account customisation is not working and even we can't see the state machine for account-provisioning-customization as well as no pipeline for any of the account created for account customisation.When we try to run the aft-invoke-customization step function then we are getting below error.Note: The logs mentions about account creation but the account is already existing and we are making customisation through account-customization.{"Cause": "An error occurred while executing the state 'run_create_pipeline?' (entered at the event id #33). Invalid path '$.Input.account_provisioning.run_create_pipeline': The choice state's condition path references an invalid value.","Error": "States.Runtime","ExecutionArn": "arn:aws:states:us-east-2:<aft-account-id>:execution:aft-account-provisioning-framework:e5c48973-f6fa-4def-beaf-55ca11e33ba2","Input": "{"account_info":{"account":{"id":"<shared-account-id>","email":"shared_acct@email","name":"shared-account","joined_method":"CREATED","joined_date":"2023-03-09 07:51:44.747000+00:00","status":"ACTIVE","parent_id":"ou-38lh-9att8jja","parent_type":"ORGANIZATIONAL_UNIT","type":"account","vendor":"aws"}},"control_tower_event":{},"account_request":{"custom_fields":"{\"group\":\"prod\"}","change_management_parameters":{"change_reason":"Create new ControlPlane account shared-account","change_requested_by":"shared_acct@email.com"},"id":"shared_acct@email.com","control_tower_parameters":{"AccountEmail":"sharedservices-account@email","SSOUserFirstName":"-sharedservices-account","SSOUserLastName":"sharedservices-account","ManagedOrganizationalUnit":"controlplane-ou","AccountName":"shared-account","SSOUserEmail":"shared_acct@email.com@email"},"account_tags":{"Environment":"prod","Owner":"sharedservices-account sharedservices-account","Project":"xyz","Vended":"true","created_by":"sharedservices-account@email"},"account_customizations_name":"shared-customizations"},"account_provisioning":{"run_create_pipeline":"true"},"customization_request_id":"c0bb8f9a-9f82-4c30-a62c-96119a391b53"}","InputDetails": {"Included": true},"Name": "e5c48973-f6fa-4def-beaf-55ca11e33ba2","StartDate": 1679307003825,"StateMachineArn": "arn:aws:states:us-east-2:<aft-account-id>:stateMachine:aft-account-provisioning-framework","Status": "FAILED","StopDate": 1679307036829}To Reproduce:-Steps to reproduce the behavior:Add terraform code in account-customization repository under account_customization_name valued folderRun the Step function with below input{"include": [{"type": "accounts","target_value": ["<target account id>"]}]}FollowComment"

Answer: "0Can you try just doing a 'Release Change' on the account specific Pipeline?I know that doesn't directly address your problem using the Invocation, however this should trigger the deployment of the solution to the account similarly to using the invocation step function.This might tell you if the solution is with the deployment of the customization or if there is just an error triggering the step function.CommentShareRoss Wickmananswered 2 months agorePost-User-2280568 2 months agoThanks for info, the issue is resolved.Though the state tf files were already there in our aft-account-provisioning-customizations repository as mentioned in https://github.com/aws-ia/terraform-aws-control_tower_account_factory/tree/main/sources/aft-customizations-repos/aft-account-provisioning-customizations/terraform, but somereason they were not picked up during AFT deployment.So we just had to make a dummy commit into the repository and that triggered aft to deploy the state machine from our aft-account-provisioning-customizations.Though the current/actual issue is resolved but the I still see a bug ... that AFT during deployment should take the repository content and deploy it, it should not explicitly wait for another commit into the repository to deploy the state machine, if the files are already present.Thanks again for the helpRegardsShare0Hi THereCheck that the ct-aft-account-provisioning-customizations pipeline ran successfully, as that pipeline creates the aft-account-provisioning-customizations state machineCommentShareEXPERTMatt-Banswered 2 months ago"


Question: Allow AssumeRole from an assumed (SSO-generated) role? [Solved]

"This was mostly pilot error; I had the wrong role name!For future reference, this is the trust policy that worked for us:"AssumeRolePolicyDocument": {  "Version": "2012-10-17",  "Statement": [    {      "Sid": "AllowLambdaService",      "Effect": "Allow",      "Principal": {        "Service": "lambda.amazonaws.com"      },      "Action": "sts:AssumeRole"    },    {      "Sid": "AllowTrustedARNs",      "Effect": "Allow",      "Principal": {        "AWS": [          "arn:aws:iam::222222222222:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_Admin_0123456"        ]      },      "Action": "sts:AssumeRole"    }  ]},Below is the original question for context.We have a single-org, multi-account setup. We're using IAM Identity Center SSO on our organizational management account, and we've successfully used it to replace most CLI and web access to our AWS resources.How can we configure a normal IAM Role to trust (i.e., allow sts:AssumeRole from) anyone that can authenticate into an SSO-generated role (on a "leaf" account, not on the org account)?For example, let's say our org account is 111111111111 and our leaf account is 222222222222.The org account has a PermissionSet named 'Admin'. User alice@example.com is assigned to that permission set for the leaf account.As soon as the org account has any assignments against the leaf account, the SSO system uses the PermissionSet (on the org account) to create a Role (on the leaf account). In this case, let's say that it creates arn:aws:iam::222222222222:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_Admin_0123456.When Alice wants to connect to the leaf account, she goes through AWS SSO and ends up with a temporary session within that role; sts get-caller-identity returns something like:{    "UserId": "AROA....:alice@example.com",    "Account": "222222222222",    "Arn": "arn:aws:sts::222222222222:assumed-role/AWSReservedSSO_Admin_0123456/alice@example.com"}We would like to allow Alice to assume a standard IAM role on account 222222222222, e.g., arn:aws:iam::222222222222:role/my-lambda-execution-role.But we're unable to find any way to express this in the trust relationship for the local role. Even if IAM accepts my syntax, Alice always receives:$ aws sts assume-role \    --role-arn 'arn:aws:iam::222222222222:role/my-lambda-execution-role' \    --role-session-name 'mock_lambda'An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::222222222222:assumed-role/AWSReservedSSO_Admin_0123456/alice@example.com is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::222222222222:role/my-lambda-execution-roleThings we've tried:Verified that the AWSReservedSSO_Admin_0123456 role is allowed to do sts:AssumeRole (via the IAM Simulator);Copied the denied user string directly (IAM accepts the change, but Alice still get AccessDenied.)Tried various permutations of the Role ARN without any success. (Honestly don't recall which of these were rejected outright by IAM, vs those that were accepted but still didn't allow Alice to assume the local role.)role/AWSReservedSSO_Admin_0123456assumed-role/AWSReservedSSO_Admin_0123456assumed-role/AWSReservedSSO_Admin_0123456/alice@example.com)both arn:aws:sts::... and arn:aws:iam:... forms for the ARNsTried trusting root along with a Condition to limit incoming roles (similar to this) (accepted, still denied).Other answers we've found tend to fall into one of a few buckets:Only ever use one layer of roles: Doesn't seem possible while still providing per-lambda roles.Configure IAM roles to trust federated users: Seems to duplicate grouping / permissions set on the Identity Center? (Also seems to require specifying individual users even within the federated identity; if so, that's more duplication we'd prefer to avoid.)Problems with ~/.aws/config: This seems to be working fine, since sts get-caller-identity is happy, and we can do most other CLI things successfully.Interestingly, we're able to successfully use the following in our EKS authentication map:    - rolearn: arn:aws:iam::222222222222:role/AWSReservedSSO_Admin_0123456      username: "sso:admin:{{SessionName}}"      groups:        - adminThanks in advance for any ideas / suggestions you might have!FollowComment"

Answer: "0Accepted AnswerAs explained in the (edited) question, I was testing against the wrong role name!The correct role ARN for the trust relationship is everything up to the last slash. So if the output from sts get-caller-identity shows something like:"arn:aws:sts::222222222222:assumed-role/AWSReservedSSO_Admin_0123456/alice@example.com"Then the trust policy should use the truncated ARN:"arn:aws:sts::222222222222:assumed-role/AWSReservedSSO_Admin_0123456"CommentShareAnthony Foianianswered 13 days ago"


Question: EC2 going into Safe Mode at startup

We are not able to take a Remote Desktop of our one of Windows EC2 because EC2 booting into Safe Mode how we can tackle this issue.FollowComment

Answer: 1Accepted AnswerOne of the users has run into the same problem and his query is answered on AWS forums. Please see if the link helpshttps://forums.aws.amazon.com/thread.jspa?threadID=152055CommentShareSandeepVudataanswered a year ago


Question: How to set subtitles as forced subtitles via Mediaconvert

"Hi,we are creating HLS- and DASH-Outputs via MediaConvert.Our source assets provide multiple subtitles(captions) in the ttml-format.Some of these subtitles are forced subtitles and we need to flag those properly in therespective playlists created for HLS/DASH  (mpd/m3u8). We set the destination type of the subtitles in the output settings to WebVTT.Given a main-m3u8 file created by MediaConvert the following lines are refering to the added subtitles:#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="subs_hu",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="hun",URI="hlssubtitles_hu.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="subs_en_US",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="eng",URI="hlssubtitles_en_US.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="forced_subs_hu",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="hun",URI="hlsforced_subtitles_hu.m3u8"For each entry there is a value "FORCED=NO". How can we alter this value for forced subtitles to  "FORCED=YES"?Similarly how do we set this up for DASH-Playlists?Subtitles were added in the following form:   <AdaptationSet mimeType="text/vtt" lang="hu">       <Role schemeIdUri="urn:mpeg:dash:role" value="subtitle"/>       <Label>forced_subs_hu</Label>       <Representation id="11" bandwidth="256">         <BaseURL>dashforced_subtitles_hu.mpd</BaseURL>       </Representation>     </AdaptationSet>According to DASH's ISO/IEC 23009-1:2019we should be able to set the role-value to forced-subtitles.FollowComment"


Question: What service should I use to host my application?

I'm building a new application using Ruby on Rails connecting to a PostgreSQL database.Which AWS service should I use to host my application?FollowComment

Answer: "2Accepted AnswerConsider looking at AWS App Runner. App Runner is the simplest way to run containers on AWS, including Rails containers. And App Runner also supports running in a custom VPC, making it easy to connect your application to an Amazon RDS database. App Runner also automatically scales your application in and out to meet demand, helping you save money.CommentShareEXPERTMichael_Fanswered a year agojplock a year agoWow, this looks just what I was looking for, thanks for the suggestion!ShareJason_S a year agoIt's unclear why you think AppRunner is going to solve your problem of "paying for something running all the time even while developing my app and its getting no traffic". Currently due to the limitation of ECS it cannot scale to 0, though the price is relatively cheap. Also like I already pointed out, there are limitations App Runner has similar to Lambda custom container, specifically on storage. I will recommend you build on Elastic Beanstalk unless you know containers very well and the application does not have strong state / persistence requirements. Even that, you might be better off with Lambda.Sharejplock a year agoThank you for the response.  I like the pay-as-you-pricing model that App Runner provides.  Everything I've read about Beanstalk is I'd have to pay for EC2 instances even if my application isn't receiving any traffic.  I did look at Lambda, but since I'm using Rails for request routing in the application, I'd have to pay for an API Gateway as I don't think I can invoke a Lambda function from a web request without one.  I was looking at using Sidekiq for asynchronous tasks, but I think using an SQS queue and Lambda functions would be a good fit for that use case.Share0Hello -That question depends on what's your experience level is and what setup you already have.However, for many new customers looking for easy to start solutions and setups, have you considered ElasticBeanstalk and RDS?CommentShareJason_Sanswered a year agojplock a year agoI was looking at Beanstalk but I didn't like the idea of paying for something running all the time even while developing my app and its getting no traffic.  Is there anything cheaper?ShareJason_S a year agodepends on your use case, Lambda might be an option though you will complicate the setup if you need to persist data. But generally, for development you might just use a local environment or even a Cloud9, and for testing you can have a small EC2 instance which can be shut down if not using.Share"


Question: Systems Manager Change Manager Notifications not working

"Hello,I have finished the setup of Systems Manager Change Manager, I created the SNS Topic for Template Reviewers, Change Request Approvers and also another one for Notifications of the change request progress.Every time I run a Change Request it says on the Timeline:Failed to publish notifications to SNS.I have tried to use the same Topic of Approvers and Template Reviewers (where I know the emails are being sent correctly) but the error is the same, I would like to know what am I missing ? Does the Topic need a different Access Policy ? Do I need to add a SNS Policy to another role ?Thanks for your insights.FollowComment"

Answer: "1It sounds like SSM does not have permission to publish to SNS.  See Steps 2, 3, and 5 here in the Monitoring Systems Manager status changes using Amazon SNS notifications documentationCommentSharesetheliotanswered a year agoHigher a year agoHello and thanks for your comments, I think SSM has the correct permissions, I made some tests and after changing the SNS Access Policy it worked, but I have to leave the SNS too wide open and I do not like that too much.Share"


Question: Black/blank screen after connecting on new computer

"I have been using Amazon Workspaces Client on a Windows 10 i5 computer for several years without issue.I recently got access to a brand new Windows 10 i7 11th Gen Dell.  I installed the latest Windows 10 client (3.1.2.1844), and much to my surprise, after logging in and connecting, the screen just showed a blank/black window.   It did not work after reboots and several attempts with different screen resolutions.  Both laptops are connected to the same external display (1440p).I downloaded the 2.5.11 client and that worked!  It seems crazy that my brand new computer with a better processor and graphics card doesn't work with Amazon Workspaces.Any ideas?  I don't see a place to download any of the older 3.x versions to try those out.FollowComment"

Answer: "0I had the same problem today. I was using the AWS 3.1.2.1844 Windows client without any problems on an older HP Elitebook. Today I switched to a new HP Spectre and downloaded the AWS 3.1.2.1844 Windows client. When I log into my workspace, I get a black screen (and a very small mouse pointer). Once or twice when trying to log in, if I waited long enough, the icons on the AWS machine appeared, but it was unresponsive to the keyboard or mouse. I closed the AWS client and opened it on my old computer and everything worked fine. Both laptops are Windows 10 (old one is Home, new one is Pro).CommentSharenlkanswered 2 years ago0One additional note. I was able to access the workspace on my new computer through Web Access. I still can't access it with the Windows client.CommentSharenlkanswered 2 years ago0I tried downloading AWS client 2.5.11 (the only older option on the website), and I was able to access my workspace from my new laptop just fine.CommentSharenlkanswered 2 years ago0Adding tags to this postCommentSharenlkanswered 2 years ago0Same problem here, new laptop (Surface Pro 7 i7 if that makes a difference), latest Workspaces Client gives me black screen upon connecting.  Tried multiple Workspaces, all black screen.Web client works fine.  Old Client (v2.5.1) works fine.Latest client on my old laptop (Surface Book 1 i5) works fine.Definitely a problem with the latest client on certain devices.CommentShareAudiowazeanswered 2 years ago0same issue on some computers (also with version 3.1.3.1894)Seem work only with old versionEdited by: markf2020 on Feb 15, 2021 2:58 AMCommentSharemarkf2020answered 2 years ago0I loaded the new Version 3.1.3.1894, and I'm still having the same problem on my new computer. I'll have to go back to Version 2.5.11.CommentSharenlkanswered 2 years ago0Same here.  Disappointing that they released an update that didn't address this whatsoever.CommentShareAudiowazeanswered 2 years ago0Finally, Latest Client (3.1.4.1996) appears to have resolved this issue for meCommentShareAudiowazeanswered 2 years ago03.1.4 resolved it for me too.CommentSharenlkanswered 2 years ago"


Question: EC2 VM doesn't have Internet connection

"I have a testing VPC created long time ago, it's connected to our On-Prem network with a VPN connection. We don't assign public IP in EC2 instances in this VPC and only access them through the VPN.It was working before, but today when I powered up the only EC2 instance in the VPC, I found it lost Internet access. I can still ssh into it with the private IP from On-Prem, but it can't reach any website by curl, I can't ping any public IP including 8.8.8.8 ( DNS resolving is still working ).[root@ip-10-240-46-19 ec2-user]# ping 8.8.8.8PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.^C--- 8.8.8.8 ping statistics ---1 packets transmitted, 0 received, 100% packet loss, time 0ms[root@ip-10-240-46-19 ec2-user]# nslookupwww.yahoo.comServer:         10.240.46.2Address:        10.240.46.2#53Non-authoritative answer:www.yahoo.com   canonical name = atsv2-fp-shed.wg1.b.yahoo.com.Name:   atsv2-fp-shed.wg1.b.yahoo.comAddress: 98.137.246.8[root@ip-10-240-46-19 ec2-user]# ping www.yahoo.comPING atsv2-fp-shed.wg1.b.yahoo.com (98.137.246.8) 56(84) bytes of data.^C--- atsv2-fp-shed.wg1.b.yahoo.com ping statistics ---[root@ip-10-240-46-19 ec2-user]# yum updateLoaded plugins: extras_suggestions, langpacks, priorities, update-motdCould not retrieve mirrorlist http://amazonlinux.us-west-1.amazonaws.com/2/core/latest/x86_64/mirror.list error was12: Timeout on http://amazonlinux.us-west-1.amazonaws.com/2/core/latest/x86_64/mirror.list: (28, 'Connection timed out after 5000 milliseconds')[root@ip-10-240-46-19 ec2-user]# curl -v http://amazonlinux.us-west-1.amazonaws.com/2/core/latest/x86_64/mirror.listTrying 52.219.120.48...TCP_NODELAY setTrying 2600:1fa0:c040:250:34db:78a1::...TCP_NODELAY settcpdump shows there is no response from public server:14:36:42.468736 IP 10.240.46.19.42110 > 52.219.24.33.80: Flags [S], seq 3488750344, win 26883, options [mss 8961,sackOK,TS val 1544269441 ecr 0,nop,wscale 7], length 014:36:43.475306 IP 10.240.46.19.42110 > 52.219.24.33.80: Flags [S], seq 3488750344, win 26883, options [mss 8961,sackOK,TS val 1544270448 ecr 0,nop,wscale 7], length 014:36:45.491327 IP 10.240.46.19.42110 > 52.219.24.33.80: Flags [S], seq 3488750344, win 26883, options [mss 8961,sackOK,TS val 1544272464 ecr 0,nop,wscale 7], length 0I did some troubleshooting:checked security group:There is only one inbound rule to allow TCP 22; one outbound rule to allow "AllAll0.0.0.0/0".Checked VPC routing table, there are three rules:10.240.46.0/24local0.0.0.0/0 -> internet gateway10.0.0.0/8 -> VPN Gateway ( apparent this one still working )Tried to create a new Internet Gateway and swap it, didn't fix it.Tried to create a Nat gateway and changed routing 0.0.0.0/0 to use the Nat Gateway, didn't fix it.removed Nat Gateway, changed it back to use Internet Gateway for 0.0.0.0/0.Created a new EC2 instance in same subnet with a different private IP, same problem.Inside the instance, it shows:Destination     Gateway         Genmask         Flags Metric Ref    Use Iface0.0.0.0         10.240.46.1     0.0.0.0         UG    0      0        0 eth010.240.46.0     0.0.0.0         255.255.255.224 U     0      0        0 eth0169.254.169.254 0.0.0.0         255.255.255.255 UH    0      0        0 eth0Now I run out of idea.Is there some where else I should be looking?Thanks,JackFollowComment"

Answer: "0forgot to mention, also checked VPC ACL,its outbound rule has rule 100 to allow any.CommentSharejackchen858answered 4 years ago0OK will answer the question myself:opened support ticket with AWS and the support answered my question:"In order for your NAT Gateway to work properly, you will have to place it in a public subnet ie. the subnet should have an internet gateway attached to it and you should have subnet specific route tables where the 0.0.0.0/0 on your private subnet is routed to your NAT Gateway and the 0.0.0.0/0 on your public subnet is routed to the Internet Gateway."https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario2.htmlWorked mostly in Azure ( where Internet gateway and NAT gateway were setup directly by Azure Vnet ), I didn't realize how private network should be configured in AWS.CommentSharejackchen858answered 4 years ago0Hi jackchen858,I'm glad that you were able to solve the issue. Please let us know if you need any further help.Regards,awstomasCommentShareAWS-User-3892169answered 4 years ago"


Question: Registers Data Not Updated In Subsequent Requests In ModBus RTU Component

"This issue is in regards to the v2 aws.greengrass.Modbus component. The setup:Core device deployed on Raspberry Pi 4B Linux with Waveshare's RS485 CAN Hat connected to a temperature/humidity sensor (XY-MD02).Local simple Python scripts test fine.AWS IoT MQTT test client initially tests fine on first request but subsequent (and different) requests show the "registers" array in the result does not change although the function_code and operation parameters are all correctly updated. Basically, it appears almost as if the registers data remains somewhere in memory and is returned instead of the real results. It takes calling the request 3-4 times before the correct data in the registers in returned. What is causing this problem?FollowComment"

Answer: "0Hi,Could you provide more logs, configurations and info for this issue? From current symptom, we are not able to detect any root causes.CommentShareDaojinganswered 8 months ago"


Question: What is the best way to work with SQS queues in local environments?

"I am currently working in an environment based on microservices.In my environments in "production". I have some SQS queues with the names:-** pro-update-profile**and for test environments-** test-update-profile**I was thinking of setting up a queue called-** dev-update-profile**for all developers in their local environments, but they would have message crossing problems when developing and testing in their local environments receiving messages from other developers.What is the best way to solve this problem? I could also create a queue for each developer with initial prefixes of their names when starting their local environments automatically, but it would have too many queues in SQS.FollowComment"

Answer: "3Accepted AnswerMy recommendation would be your last option, i.e., create a queue for each developer with their name in the queue name to identify them. I understand that this will create many queues. One way to overcome this would be to create a separate account per developer, or at least per team. This way the number of queues per account will be small.CommentShareEXPERTUrianswered a year agoEXPERTHernitoreviewed a year ago1Probably you can use localstack for local dev environments, it supports SQS in free versionCommentShareA-Shevchenkoanswered a year agoEXPERTHernitoreviewed a year agoMaxCloud a year agoWow, its a great tool!Share0Define "too many queues" - there is no quota on the # of queues in an account.  More fundamentally, using a single account like this with only a naming convention separating dev, test, prod queues is inherently risky; an accident could easily place messages in an incorrect queue.  The solution is to completely separate the environments.  In Azure we would use separate subscriptions/resource groups, in GCP we would use separate projects.  In AWS we use separate accounts.  Take a look at organizations / control tower so you can 'vend' separate accounts for test, dev, or individuals whenever needed.CommentSharekennykanswered a year ago"


Question: Need car battery for DeepRacer. Where to get replacement parts?

"Is there a parts list for the DeepRacer that includes Amazon links to the parts used or where replacements can be obtained? I need a replacement for the car battery and have not been able to find one that will actually attach to the car - the plug types are different. Where can I buy a replacement for either battery? I need the small one, but might as well get spares for everything.Edited by: Gregory G. Pierce on Oct 12, 2019 3:06 PMFollowComment"

Answer: 0https://docs.aws.amazon.com/deepracer/latest/developerguide/deepracer-vehicle-chassis-parts.htmlLeaving this for anyone else in the same boat that is looking to replace missing or damaged components on the racer.CommentShareGregory G. Pierceanswered 4 years ago0You can check the Deep Racer Store from Amazon https://www.amazon.com/b?ie=UTF8&node=32957528011CommentShareMaximiliano_Kanswered 21 days ago


Question: When will g5 instances be available for batch transform?

"I see the new A10 GPU G5 instances are available for sagemaker training and notebook but not Batch transform, when will it be rolled out to this service?FollowComment"

Answer: "0The G5 instances should be available soon for SageMaker transform as well.In the meantime, you could use SageMaker Training Jobs with custom container to do inference as well. Then you can retrieve the predictions results after unzipping the default output file model.tar.gz which is provided by the SM Training Job.Another alternative is to use the G4DN GPU instances with your SageMaker Batch Transform.CommentShareIoananswered 6 months agorePost-User-0853106 6 months agoThanks for the reply! Any estimate on how soon is soon?, G4s unfortunately won't work as we need bfloat16 compatibilityShare"


Question: Unable to add APNS certificate to Pinpoint

"I'm trying to use a universal (Sandbox & Production) APNS certificate with Pinpoint. Upon uploading it I get the following message:BadRequestException: You provided a certificate of type SANDBOX, which cannot be used to create an application of type iOS Production. Please select an application of type SANDBOX or provide a certificate of type iOS ProductionI've checked that I'm creating the universal certificate and exporting it as a .p12. Pinpoint seems to thing it is a SANDBOX certificate. What am I doing wrong?FollowComment"


Question: Devices with Alexa

"Hi,I find the "Alexa for Business Is Now Generally Available" very interesting, and it is a step forward for Alexa users.As far as I understand requires use of Alexa for Business to actually use Alexa-enabled devices. Currently the Amazon homepage doesn't allow me to buy Alexa devices to Norway (and I guess a lot of other countries).As Alexa for Business is generally available, where may I buy such devices to Norway?FollowComment"

Answer: 0Thanks for your interest in Alexa for Business.Alexa for Business is currently only available in the US. You must indeed have an Alexa-enabled device and purchase it through the existing channels.CommentShareMilo Oostergoanswered 5 years ago


Question: VALIDATION FOR INSTANCES CLOUD PRACTICE

"Hi, Im new and excited about this wonderful learning experience. I understand no question is an stupid question . I've been getting a non-validation for my first instance in the CLOUD Practice Lab..My second instance is going thru fine and validations has succeeded . I keep going over the steps for zones and regions correct and no change in the message. SO ONE PLEASE HELP ME ..FollowComment"

Answer: "0Welcome abord!It is a great question. Could you please provide more information, such as the VPC, subnet, key pair, security group, image (AMI) and instance type. I am sorry to ask yoy which platform you are using to create the instance, such AWS or Azure. If possible take a screenshot of your non-validation page, and reply with that.Never give up.CommentSharerePost-User-7136343answered 3 months ago"


Question: how do we version control an aws glue workflow?

how do we version control an aws glue workflow ?FollowComment

Answer: "0Hi,You can use a source control tool (e.g., AWS CodeCommit) for checking in and versioning your AWS Glue ETL source code and AWS Glue Workflow CloudFormation template files. Refer the following two blogs which provide more details regarding the same...Using AWS DevOps Tools to model and provision AWS Glue workflowshttps://aws.amazon.com/blogs/devops/provision-codepipeline-glue-workflows/Implement continuous integration and delivery of serverless AWS Glue ETL applications using AWS Developer Toolshttps://aws.amazon.com/blogs/big-data/implement-continuous-integration-and-delivery-of-serverless-aws-glue-etl-applications-using-aws-developer-tools/Hope this helps!CommentShareamanbajwanswered a year ago"


Question: AWS SAM CLI Error: Error: PythonPipBuilder:ResolveDependencies - {pyyaml==6.0(wheel)}

"Hi, I am keeping running into the same error as mentioned in the title even though I have installed wheel. I have tried sam build --use-container (and there is a template.yaml in the folder), but it caused another error (Error: Docker is unreachable. Docker needs to be running to build inside a container).It would be appreciated if there is anyone who could help me for solving this issue!FollowComment"

Answer: "2Accepted AnswerDo you have docker installed and the docker daemon running on the machine from where you are trying to use sam build --use-container command.Take a look at https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install-windows.htmlStep 3 - Install Docker mentions the following - "Docker is a prerequisite only for testing your application locally and for building deployment packages using the --use-container option. If you don't plan to use these features initially, you can skip this section or install Docker at a later time."So if you are using the --use-container option, then you do need docker to be running on the machine.CommentShareEXPERTIndranil Banerjee AWSanswered 8 months agoMona 8 months agoIt worked. Thank you so much :)Share"


Question: Redshift Federated Query Error 25101

"Hello everyone,I discovered a problem during a Redshift Spectrum SELECT query that works as expected when it's ran inside the RDS (Postgres) DB.The query is like the following:SELECT MIN(column) FROM table;When ran inside the RDS, the result is correct, when ran via Redshift Spectrum it raises the following exception:SQL Error [XX000]: ERROR: Decimal input parse error  Detail:   -----------------------------------------------  error:  Decimal input parse error  code:      25101  context:   str:0.5507626528024723909068  query:     10256  location:  federation_fetchers.cpp:194  process:   query0_121_10256 [pid=8648]  -----------------------------------------------The data type is set to NUMERIC in the RDS DB. Please, notice there is no 0.5507626528024723909068 value in the table.I can't find any similar question anywhere and I would be really grateful if anyone had any advice on this. Thank you in advance!FollowComment"

Answer: "0The easy way to solve this is by moving the table in question to Redshift, as well, via a DMS Replication Task. This works but keep in mind that it changes its data type automatically to NUMERIC(28,6) which means that you may lose some precision. However, I haven't found a better solution for this, I am still open to any suggestions. Posting this answer in case it helps someone else.CommentSharesnikolakisanswered a year ago"


Question: Storage Gateway - Virtual Tape Library *direct* archive to S3 Glacier Deep?

"A customer who is studying the possibility of using Tape Storage Gateway. They have asked for a 120 TB/month write costs estimation. From the FAQs:Q: What Amazon S3 storage classes does Tape Gateway support?A: Tape Gateway supports S3 Standard, S3 Glacier, and S3 Glacier Deep Archive storage classes. Data on your virtual tapes is stored in virtual tape library in Amazon S3 when backup application is writing data to tapes. After you eject tapes from backup application, your tapes are archived to S3 Glacier or S3 Glacier Deep Archive.I understand the data is stored in the S3 virtual tape library and then moved to Glacier Deep Archive (in my case).What's the correct approach?FollowComment"

Answer: "0Accepted AnswerData will be moved to S3 Glacier or S3 Glacier Deep Archive as soon as the tape is ejected. Have the backup software eject the tape as soon as it makes sense to do so. Alternatively, many backup applications integrate directly with S3 Glacier.CommentShareJohn_Panswered 3 years ago"


Question: Sumerian IDE Integration?

"Hi folks!Obviously, Sumerian includes it's own IDE for scripts, HTML files etc, but is there planned integration for third party products such as VSCode? Now that my number of files is increasing, I could really use the additional features to help me keep track of everything. ThanksFollowComment"

Answer: "0Accepted AnswerThere are no official plugins for IDE integration, unfortunately.CommentShareJames_Canswered a year ago"


Question: Aws lightsail - Can I connect a custom subdomain to my container deployment without moving my DNS nameserver to AWS?

"I have a number of other subdomains already set up using squarespace nameservers. I would like to be able to add a new subdomain there without having to port all of my DNS records over to AWS DNS zones.I notice there is a guide for using other nameservers - however, when I follow the instructions, I never see the option to 'edit nameservers' in my lightsail dashboard: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-domain-register-other-dns-service-procedureFollowComment"

Answer: "1HiThis is only possible if you registered your domain through Lightsail not by simply creating a DNS Zone.For a DNS Zone for a domain registered elsewhere, you cannot change the nameservers.I would recommend trying to create a CNAME record, that points directly to the default domain (public domain) of your container service.CommentShareBisinaanswered 11 days ago0You can point a subdomain from your existing registrar to your Lightsail container service https://lightsail-preprod.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-enabling-container-services-custom-domainsThe only case you'd need to move your DNS records is if you wanted the container to host your root/apex domain (example.com vs subdomain.example.com)CommentShareGabriel - AWSanswered 5 days ago"


Question: How to backup AWS opensearch to on-prem

"Hello,I have assigned for backup data Opensearch from aws to local disk.Is it possible to ?Create Opensearch on-premBackup data from AWS to local diskUpload to on-prem OpensearchDo you have medthod how I backup to local?Note: I no have exp about Opensearch beforeFollowComment"

Answer: 0You can use the snapshot method to restore data from AWS cluster to on-prem. I'm outlining the process with the assumption that the snapshot repository used is S3.Configure the cluster in AWS to use S3 as a repository.Take a snapshot of the indices ( all or selected). The first snapshot would take time since its a full backup. Subsequent snapshot would be quicker.Verify that the snapshot process completed successfully.Configure the on-prem cluster to use S3 as a source repository for restore ( preferably in readonly mode to minimize risk of messing with backup)Restore the indices from the snapshotVerify the data on on-prem cluster esp. document count in index.You can find details here - https://opensearch.org/docs/2.5/tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore/--SydCommentShareSydanswered 3 months ago


Question: web URL overlays for streaming

"I'm doing a project where I'm streaming video (locally) using SRT with a hardware encoder. I wanted to know if there was an aws service, that receives a SRT stream, but then allows you to add a web URL address on top of the streamed video (to be used as a graphic overlay)? I don't want a png overlay, I want to be able to put in a live https URL address as my overlay source.Thanks,RFollowComment"

Answer: "1Accepted AnswerThere are several components of the answer to your questions.First, SRT reception: AWS MediaConnect, a stream and ingest and distribution service, does accept streams protected with SRT.https://docs.aws.amazon.com/mediaconnect/latest/ug/protocols.htmlNext, to create an Adaptive Bit Rate stream (ABR), such as HLS, AWS MediaLive would be used. MediaConnect would send the input stream to MediaLive for ABR HLS encoding.https://docs.aws.amazon.com/medialive/latest/ug/creating-hls-output-group.htmlMediaLive offers the ability to use an external HTML5 page as an overly for graphics. This can be turned on and off using a scheduling function within MediaLive. It could be used to display the ULR you ask about.https://docs.aws.amazon.com/medialive/latest/ug/feature-mgi.htmlMediaLive also has the ability to use a static graphic overlay, though does not seem to match you specific need.https://docs.aws.amazon.com/medialive/latest/ug/working-with-image-overlay.htmlMediaLive would need to push the HLS to an origin server, such as AWS MediaPackage or Amazon S3 storage. A CDN, such as Amazon CloudFront would need to frontend the streams, offering access to viewers.CommentShareMike-MEanswered 2 months agorePost-User-9128667 2 months agoThanks for the breakdown. I was able to send my encoded video to MediaConnect via SRT. I'm now sending my Flow to MediaLive, to do the HTML5 overlay. However, I don't see an option to then send the output from MediaLive (video + overlay) via SRT. Is that not possible?Share"