Question: "Error for Training job catboost-classification-model , ErrorMessage "TypeError: Cannot convert 'xxx'' to float"

"When I performed the following AWS tutorial, I got an error when training the model.https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/lightgbm_catboost_tabular/Amazon_Tabular_Classification_LightGBM_CatBoost.ipynbThe error that occurred isUnexpectedStatusException: Error for Training job jumpstart-catboost-classification-model-2022-07-22-07-33-18-038: Failed. Reason: AlgorithmError: ExecuteUserScriptError:ExitCode 1 ErrorMessage "TypeError: Cannot convert 'b'BROOKLYN'' to floatThese are all the files that I have upload in S3 bucket  :Amazon S3 --> Buckets---> R-sandbox-sagemaker--->ml/---> train/and in the train folder 'data.csv' and 'categorical_index.json' are uploaded based on the mentioned tutorial.data point "BROOKLYN" is in the categorical column, its index is already included in the JSON file to tell Catboost that it is categorical data. Data has 55 categorical data columns; only two of them are integers , all other stringCould you give me some advice on how to solve it?Also here all the code and full traceback of the issue:!pip install sagemaker ipywidgets --upgrade –quietimport sagemaker, boto3, jsonfrom sagemaker import get_execution_roleaws_role = get_execution_role()aws_region = boto3.Session().region_namesess = sagemaker.Session()##2.1 Retrieve Training Artifacts-#retrieve the training docker container, the training algorithm source, and the tabular algorithm. Note that model_version="*" fetches the latest model.# Currently, not all the object detection models in jumpstart support finetuning. Thus, we manually select a model# which supports finetuning.from sagemaker import image_uris, model_uris, script_uristrain_model_id, train_model_version, train_scope = "catboost-classification-model", "*", "training"training_instance_type = "ml.m5.xlarge"# Retrieve the docker imagetrain_image_uri = image_uris.retrieve(    region=None,    framework=None,    model_id=train_model_id,    model_version=train_model_version,    image_scope=train_scope,    instance_type=training_instance_type,)# Retrieve the training scripttrain_source_uri = script_uris.retrieve(    model_id=train_model_id, model_version=train_model_version, script_scope=train_scope)# Retrieve the pre-trained model tarball to further fine-tunetrain_model_uri = model_uris.retrieve(    model_id=train_model_id, model_version=train_model_version, model_scope=train_scope)## 2.2 Set Training Parameters# Sample training data is available in this buckettraining_data_bucket = "R-sandbox-sagemaker"training_data_prefix = "ml"training_dataset_s3_path = f"s3://{training_data_bucket}/{training_data_prefix}"output_bucket = sess.default_bucket()output_prefix = "jumpstart-example-tabular-training"s3_output_location = f"s3://{output_bucket}/{output_prefix}/output"from sagemaker import hyperparameters# Retrieve the default hyper-parameters for fine-tuning the modelhyperparameters = hyperparameters.retrieve_default(    model_id=train_model_id, model_version=train_model_version)# [Optional] Override default hyperparameters with custom valueshyperparameters[    "iterations"] = "500"  # The same hyperparameter is named as "iterations" for CatBoostprint(hyperparameters)## 2.3. Train with Automatic Model Tuningfrom sagemaker.tuner import ContinuousParameter, IntegerParameter, HyperparameterTuneruse_amt = Trueif train_model_id == "lightgbm-classification-model":    hyperparameter_ranges = {        "learning_rate": ContinuousParameter(1e-4, 1, scaling_type="Logarithmic"),        "num_boost_round": IntegerParameter(2, 30),        "early_stopping_rounds": IntegerParameter(2, 30),        "num_leaves": IntegerParameter(10, 50),        "feature_fraction": ContinuousParameter(0, 1),        "bagging_fraction": ContinuousParameter(0, 1),        "bagging_freq": IntegerParameter(1, 10),        "max_depth": IntegerParameter(5, 30),        "min_data_in_leaf": IntegerParameter(5, 50),    }if train_model_id == "catboost-classification-model":    hyperparameter_ranges = {        "learning_rate": ContinuousParameter(0.00001, 0.1, scaling_type="Logarithmic"),        "iterations": IntegerParameter(50, 1000),        "early_stopping_rounds": IntegerParameter(1, 10),        "depth": IntegerParameter(1, 10),        "l2_leaf_reg": IntegerParameter(1, 10),        "random_strength": ContinuousParameter(0.01, 10, scaling_type="Logarithmic"),    }## 2.4. Start Trainingfrom sagemaker.estimator import Estimatorfrom sagemaker.utils import name_from_basetraining_job_name = name_from_base(f"jumpstart-{'catboost-classification-model'}-training")# Create SageMaker Estimator instancetabular_estimator = Estimator(    role=aws_role,    image_uri=train_image_uri,    source_dir=train_source_uri,    model_uri=train_model_uri,    entry_point="transfer_learning.py",    instance_count=1,    instance_type=training_instance_type,    max_run=360000,    #hyperparameters=hyperparameters,    output_path=s3_output_location,)# Launch a SageMaker Training job by passing s3 path of the training datatabular_estimator.fit(        {"training": training_dataset_s3_path}, logs=True, job_name=training_job_name    )2022-07-22 07:33:18 Starting - Starting the training job...2022-07-22 07:33:46 Starting - Preparing the instances for trainingProfilerReport-1658475198: InProgress2022-07-22 07:35:06 Downloading - Downloading input data...2022-07-22 07:35:46 Training - Downloading the training image...2022-07-22 07:36:11 Training - Training image download completed. Training in progress..bash: cannot set terminal process group (-1): Inappropriate ioctl for devicebash: no job control in this shell2022-07-22 07:36:14,025 sagemaker-training-toolkit INFO     Imported framework sagemaker_pytorch_container.training2022-07-22 07:36:14,027 sagemaker-training-toolkit INFO     No GPUs detected (normal if no gpus installed)2022-07-22 07:36:14,036 sagemaker_pytorch_container.training INFO     Block until all host DNS lookups succeed.2022-07-22 07:36:14,041 sagemaker_pytorch_container.training INFO     Invoking user training script.2022-07-22 07:36:15,901 sagemaker-training-toolkit INFO     Installing dependencies from requirements.txt:/opt/conda/bin/python3.8 -m pip install -r requirements.txtProcessing ./catboost/tenacity-8.0.1-py3-none-any.whlProcessing ./catboost/plotly-5.1.0-py2.py3-none-any.whlProcessing ./catboost/graphviz-0.17-py3-none-any.whlProcessing ./catboost/catboost-1.0.1-cp38-none-manylinux1_x86_64.whlProcessing ./sagemaker_jumpstart_script_utilities-1.0.0-py2.py3-none-any.whlRequirement already satisfied: six in /opt/conda/lib/python3.8/site-packages (from plotly==5.1.0->-r requirements.txt (line 2)) (1.16.0)Requirement already satisfied: numpy>=1.16.0 in /opt/conda/lib/python3.8/site-packages (from catboost==1.0.1->-r requirements.txt (line 4)) (1.19.1)Requirement already satisfied: scipy in /opt/conda/lib/python3.8/site-packages (from catboost==1.0.1->-r requirements.txt (line 4)) (1.7.1)Requirement already satisfied: matplotlib in /opt/conda/lib/python3.8/site-packages (from catboost==1.0.1->-r requirements.txt (line 4)) (3.4.3)Requirement already satisfied: pandas>=0.24.0 in /opt/conda/lib/python3.8/site-packages (from catboost==1.0.1->-r requirements.txt (line 4)) (1.2.4)Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.8/site-packages (from pandas>=0.24.0->catboost==1.0.1->-r requirements.txt (line 4)) (2.8.2)Requirement already satisfied: pytz>=2017.3 in /opt/conda/lib/python3.8/site-packages (from pandas>=0.24.0->catboost==1.0.1->-r requirements.txt (line 4)) (2021.3)Requirement already satisfied: pillow>=6.2.0 in /opt/conda/lib/python3.8/site-packages (from matplotlib->catboost==1.0.1->-r requirements.txt (line 4)) (8.3.2)Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->catboost==1.0.1->-r requirements.txt (line 4)) (2.4.7)Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.8/site-packages (from matplotlib->catboost==1.0.1->-r requirements.txt (line 4)) (0.10.0)Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->catboost==1.0.1->-r requirements.txt (line 4)) (1.3.2)tenacity is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.Installing collected packages: plotly, graphviz, sagemaker-jumpstart-script-utilities, catboostAttempting uninstall: plotlyFound existing installation: plotly 5.3.1Uninstalling plotly-5.3.1:Successfully uninstalled plotly-5.3.1Successfully installed catboost-1.0.1 graphviz-0.17 plotly-5.1.0 sagemaker-jumpstart-script-utilities-1.0.0WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv2022-07-22 07:36:32,568 sagemaker-training-toolkit INFO     No GPUs detected (normal if no gpus installed)2022-07-22 07:36:32,604 sagemaker-training-toolkit INFO     Invoking user scriptTraining Env:{    "additional_framework_parameters": {},    "channel_input_dirs": {        "model": "/opt/ml/input/data/model",        "training": "/opt/ml/input/data/training"    },    "current_host": "algo-1",    "framework_module": "sagemaker_pytorch_container.training:main",    "hosts": [        "algo-1"    ],    "hyperparameters": {},    "input_config_dir": "/opt/ml/input/config",    "input_data_config": {        "model": {            "ContentType": "application/x-sagemaker-model",            "TrainingInputMode": "File",            "S3DistributionType": "FullyReplicated",            "RecordWrapperType": "None"        },        "training": {            "TrainingInputMode": "File",            "S3DistributionType": "FullyReplicated",            "RecordWrapperType": "None"        }    },    "input_dir": "/opt/ml/input",    "is_master": true,    "job_name": "jumpstart-catboost-classification-model-2022-07-22-07-33-18-038",    "log_level": 20,    "master_hostname": "algo-1",    "model_dir": "/opt/ml/model",    "module_dir": "s3://jumpstart-cache-prod-us-east-1/source-directory-tarballs/catboost/transfer_learning/classification/v1.1.3/sourcedir.tar.gz",    "module_name": "transfer_learning",    "network_interface_name": "eth0",    "num_cpus": 4,    "num_gpus": 0,    "output_data_dir": "/opt/ml/output/data",    "output_dir": "/opt/ml/output",    "output_intermediate_dir": "/opt/ml/output/intermediate",    "resource_config": {        "current_host": "algo-1",        "current_instance_type": "ml.m5.xlarge",        "current_group_name": "homogeneousCluster",        "hosts": [            "algo-1"        ],        "instance_groups": [            {                "instance_group_name": "homogeneousCluster",                "instance_type": "ml.m5.xlarge",                "hosts": [                    "algo-1"                ]            }        ],        "network_interface_name": "eth0"    },    "user_entry_point": "transfer_learning.py"}Environment variables:SM_HOSTS=["algo-1"]SM_NETWORK_INTERFACE_NAME=eth0SM_HPS={}SM_USER_ENTRY_POINT=transfer_learning.pySM_FRAMEWORK_PARAMS={}SM_RESOURCE_CONFIG={"current_group_name":"homogeneousCluster","current_host":"algo-1","current_instance_type":"ml.m5.xlarge","hosts":["algo-1"],"instance_groups":[{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.m5.xlarge"}],"network_interface_name":"eth0"}SM_INPUT_DATA_CONFIG={"model":{"ContentType":"application/x-sagemaker-model","RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"},"training":{"RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"}}SM_OUTPUT_DATA_DIR=/opt/ml/output/dataSM_CHANNELS=["model","training"]SM_CURRENT_HOST=algo-1SM_MODULE_NAME=transfer_learningSM_LOG_LEVEL=20SM_FRAMEWORK_MODULE=sagemaker_pytorch_container.training:mainSM_INPUT_DIR=/opt/ml/inputSM_INPUT_CONFIG_DIR=/opt/ml/input/configSM_OUTPUT_DIR=/opt/ml/outputSM_NUM_CPUS=4SM_NUM_GPUS=0SM_MODEL_DIR=/opt/ml/modelSM_MODULE_DIR=s3://jumpstart-cache-prod-us-east-1/source-directory-tarballs/catboost/transfer_learning/classification/v1.1.3/sourcedir.tar.gzSM_TRAINING_ENV={"additional_framework_parameters":{},"channel_input_dirs":{"model":"/opt/ml/input/data/model","training":"/opt/ml/input/data/training"},"current_host":"algo-1","framework_module":"sagemaker_pytorch_container.training:main","hosts":["algo-1"],"hyperparameters":{},"input_config_dir":"/opt/ml/input/config","input_data_config":{"model":{"ContentType":"application/x-sagemaker-model","RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"},"training":{"RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"}},"input_dir":"/opt/ml/input","is_master":true,"job_name":"jumpstart-catboost-classification-model-2022-07-22-07-33-18-038","log_level":20,"master_hostname":"algo-1","model_dir":"/opt/ml/model","module_dir":"s3://jumpstart-cache-prod-us-east-1/source-directory-tarballs/catboost/transfer_learning/classification/v1.1.3/sourcedir.tar.gz","module_name":"transfer_learning","network_interface_name":"eth0","num_cpus":4,"num_gpus":0,"output_data_dir":"/opt/ml/output/data","output_dir":"/opt/ml/output","output_intermediate_dir":"/opt/ml/output/intermediate","resource_config":{"current_group_name":"homogeneousCluster","current_host":"algo-1","current_instance_type":"ml.m5.xlarge","hosts":["algo-1"],"instance_groups":[{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.m5.xlarge"}],"network_interface_name":"eth0"},"user_entry_point":"transfer_learning.py"}SM_USER_ARGS=[]SM_OUTPUT_INTERMEDIATE_DIR=/opt/ml/output/intermediateSM_CHANNEL_MODEL=/opt/ml/input/data/modelSM_CHANNEL_TRAINING=/opt/ml/input/data/trainingPYTHONPATH=/opt/ml/code:/opt/conda/bin:/opt/conda/lib/python38.zip:/opt/conda/lib/python3.8:/opt/conda/lib/python3.8/lib-dynload:/opt/conda/lib/python3.8/site-packagesInvoking script with the following command:/opt/conda/bin/python3.8 transfer_learning.pyINFO:root:Validation data is not found. 20.0% of training data is randomly selected as validation data. The seed for random sampling is 200.Traceback (most recent call last):  File "_catboost.pyx", line 2167, in _catboost.get_float_featureFile "_catboost.pyx", line 1125, in _catboost._FloatOrNan  File "_catboost.pyx", line 949, in _catboost._FloatOrNanFromStringTypeError: Cannot convert 'b'BROOKLYN'' to floatDuring handling of the above exception, another exception occurred:Traceback (most recent call last):  File "transfer_learning.py", line 221, in <module>ru```FollowCommentM 10 months agoFile "transfer_learning.py", line 221, in <module>run_with_args(args)File "transfer_learning.py", line 182, in run_with_argscat_train, cat_eval = Pool(data=X_train, label=y_train, cat_features=cat_features), Pool(File "/opt/conda/lib/python3.8/site-packages/catboost/core.py", line 628, in initself._init(data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, timestamp, feature_names, thread_count)File "/opt/conda/lib/python3.8/site-packages/catboost/core.py", line 1171, in _initself._init_pool(data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, timestamp, feature_names, thread_count)File "_catboost.pyx", line 3755, in _catboost._PoolBase._init_poolFile "_catboost.pyx", line 3803, in _catboost._PoolBase._init_poolFile "_catboost.pyx", line 3638, in _catboost._PoolBase._init_features_order_layout_poolFile "_catboost.pyx", line 2664, in _catboost._set_features_order_data_pd_data_frameFile "_catboost.pyx", line 2208, in _catboost.create_num_factor_dataFile "_catboost.pyx", line 2169, in _catboost.get_float_feature_catboost.CatBoostError: Bad value for num_feature[non_default_doc_idx=0,feature_idx=2]="BROOKLYN": Cannot convert 'b'BROOKLYN'' to float2022-07-22 07:36:34,967 sagemaker-training-toolkit ERROR    Reporting training FAILURE2022-07-22 07:36:34,967 sagemaker-training-toolkit EShareM 10 months agoExitCode 1ErrorMessage "TypeError: Cannot convert 'b'BROOKLYN'' to floatDuring handling of the above exception, another exception occurred: Traceback (most recent call last):   File "transfer_learning.py", line 221, in <module> run_with_args(args)   File "transfer_learning.py", line 182, in run_with_args cat_train, cat_eval = Pool(data=X_train, label=y_train, cat_features=cat_features), Pool(   File "/opt/conda/lib/python3.8/site-packages/catboost/core.py", line 628, in init self._init(data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, timestamp, feature_names, thread_count)   File "/opt/conda/lib/python3.8/site-packages/catboost/core.py", line 1171, in _init self._init_pool(data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, timestamp, feature_names, thread_count)   File "_catboost.pyx", line 3755, in _catboost._PoolBase._init_pool File "_catboost.pyx", line 3803, in _catboost._PoolBase._init_pool   File "_catboost.pyx", line 3638, in _catboost._PoolBase._init_features_order_layout_pool File "_catboost.pyx", line 2664, in _catboost._set_features_order_data_pd_data_frame File "_catboost.pyx", line 2208, in _catboost.create_num_factor_data   File "_catboost.pyx", line 2169ShareM 10 months agoin _catboost.get_float_feature _catboost.CatBoostError: Bad value for num_feature[non_default_doc_idx=0,feature_idx=2]="BROOKLYN": Cannot convert 'b'BROOKLYN'' to float"Command "/opt/conda/bin/python3.8 transfer_learning.py"2022-07-22 07:36:34,967 sagemaker-training-toolkit ERROR    Encountered exit_code 12022-07-22 07:36:47 Uploading - Uploading generated training model2022-07-22 07:36:47 Failed - Training job failedUnexpectedStatusException                 Traceback (most recent call last)/tmp/ipykernel_13950/1137348215.py in <cell line: 23>()2122 # Launch a SageMaker Training job by passing s3 path of the training data---> 23 tabular_estimator.fit(24         {"training": training_dataset_s3_path}, logs=True, job_name=training_job_name25     )~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)953     ):954         """Train a model using the input training dataset.--> 955956         The API calls the Amazon SageMaker CreateTrainingJob API to start957         model training. The API uses configuration you provided to create the~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/estimator.py in wait(self, logs)1954                 logger.debug(1955                     "Selecting TrainingInput's input_mode (%s) for TrainingInputMode.",ShareM 10 months ago1958                 train_args["input_mode"] = inputs.config["InputMode"]~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/session.py in logs_for_job(self, job_name, wait, poll, log_type)3796                 color_wrap,3797             )-> 3798             if state == LogState.COMPLETE:3799                 break3800~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/session.py in _check_job_status(self, job, desc, status_key_name)33343335         Args:-> 3336             name (str): Name of the Amazon SageMaker batch transform job.33373338         Raises:UnexpectedStatusException: Error for Training job jumpstart-catboost-classification-model-2022-07-22-07-33-18-038: Failed. Reason: AlgorithmError: ExecuteUserScriptError:ExitCode 1ErrorMessage "TypeError: Cannot convert 'b'BROOKLYN'' to floatShareM 10 months agoUnexpectedStatusException: Error for Training job jumpstart-catboost-classification-model-2022-07-22-07-33-18-038: Failed. Reason: AlgorithmError: ExecuteUserScriptError:ExitCode 1ErrorMessage "TypeError: Cannot convert 'b'BROOKLYN'' to floatDuring handling of the above exception, another exception occurred: Traceback (most recent call last):   File "transfer_learning.py", line 221, in <module> run_with_args(args)   File "transfer_learning.py", line 182, in run_with_args cat_train, cat_eval = Pool(data=X_train, label=y_train, cat_features=cat_features), Pool(   File "/opt/conda/lib/python3.8/site-packages/catboost/core.py", line 628, in init self._init(data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, timestamp, feature_names, thread_count)   File "/opt/conda/lib/python3.8/site-packages/catboost/core.py", line 1171, in _init self._init_pool(data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, timestamp, feature_names, thread_count)   File "_catboost.pyx", linShare"

Answer: "0Hello - can you remove the categorical_index.json inside Amazon S3 --> Buckets---> R-sandbox-sagemaker--->ml/---> train/ and place it in the directory above it, i.e. Amazon S3 --> Buckets---> R-sandbox-sagemaker--->ml/. So only data.csv is in Amazon S3 --> Buckets---> R-sandbox-sagemaker--->ml/---> train/. Then re-run. Please let me know if this works.CommentShareAWS-User-9634033answered 10 months agoM 10 months ago@AWS-User-9634033 please see the errorShare0@AWS-User-9634033 I did what ever you suggest now it gets the below error:2022-08-19 14:43:30 Uploading - Uploading generated training model2022-08-19 14:43:30 Failed - Training job failedProfilerReport-1660920003: Stopping---------------------------------------------------------------------------UnexpectedStatusException                 Traceback (most recent call last)/tmp/ipykernel_9559/1137348215.py in <cell line: 23>()     21      22 # Launch a SageMaker Training job by passing s3 path of the training data---> 23 tabular_estimator.fit(     24         {"training": training_dataset_s3_path}, logs=True, job_name=training_job_name     25     )~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/workflow/pipeline_context.py in wrapper(*args, **kwargs)    246             return self_instance.sagemaker_session.context    247 --> 248         return run_func(*args, **kwargs)    249     250     return wrapper~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)   1062         self.jobs.append(self.latest_training_job)   1063         if wait:-> 1064             self.latest_training_job.wait(logs=logs)   1065    1066     def _compilation_job_name(self):~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/estimator.py in wait(self, logs)   2145         # If logs are requested, call logs_for_jobs.   2146         if logs != "None":-> 2147             self.sagemaker_session.logs_for_job(self.job_name, wait=True, log_type=logs)   2148         else:   2149             self.sagemaker_session.wait_for_job(self.job_name)~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/session.py in logs_for_job(self, job_name, wait, poll, log_type)   3851    3852         if wait:-> 3853             self._check_job_status(job_name, description, "TrainingJobStatus")   3854             if dot:   3855                 print()~/anaconda3/envs/python3/lib/python3.8/site-packages/sagemaker/session.py in _check_job_status(self, job, desc, status_key_name)   3389                     actual_status=status,   3390                 )-> 3391             raise exceptions.UnexpectedStatusException(   3392                 message=message,   3393                 allowed_statuses=["Completed", "Stopped"],UnexpectedStatusException: Error for Training job jumpstart-catboost-classification-model-2022-08-19-14-40-03-772: Failed. Reason: AlgorithmError: ExecuteUserScriptError:ExitCode 1ErrorMessage ""Command "/opt/conda/bin/python3.8 transfer_learning.py", exit code: 1CommentShareManswered 10 months ago"


Question: Console stuck in License Terms

"Hello all,I wanted to upgrade my instance from Windows Server 2012 R2 to Windows Server 2016 and the upgrade process is now complete, but the system is stuck in License Terms view, which I can see via the EC2 instance screenshot.The server is not available via RDP connection which it was under Windows Server 2012 R2.Did I do a process incorrectly during the migration and how do I skip past that?EULAID: T1S_SR_1_ED_SRVSTD_V_en-usEdited by: aws1338 on Jan 16, 2019 7:17 AMFollowComment"

Answer: "0The answer is fairly simple - Dont install the Standard edition but the data centre edition. The licence agreement will be checked BEFORE the restart and during the install process in the running instanceCommentShareaws1338answered 4 years ago0We ended up with the same problem, even though I'd selected Datacenter edition for the upgrade. It was fixed by connecting remotely to the registry and putting in place the following registry key and then rebooting.HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OOBE\SetupDisplayedEula, of type REG_DWORD, and value 0x1We did all the work through the Windows Admin Center, but I guess it could be done through remote registry and the AWS Console.CommentSharekylegordonanswered 3 years ago"


Question: AWS Workload Discovery Post Implementation

"So far Workload Discovery is a really nifty tool; however, when I deployed it via CloudFormation I set off the infosec alarms because it created a Internet Gateway, NAT, and EIP's. I didn't see anything related to those topics in the implementation guide. The CloudFormation template created 4 subnets 2 public and 2 private. The public subnets has two routes to the IGW and an s3 endpoint. The private subnet has routes to a NAT and the same s3 endpoint.If this service needs public ip's...internet access is it possible to pivot this traffic over to existing IGW/NAT? In the eyes of infosec they see public IP's so my goal is to try and implement this in our current network design pattern or articulate to them exactly what traffic is being generated from these resources.FollowComment"

Answer: "0Hello,In regards to your query description, below are the points to be noted:The discovery process runs in a private subnet but needs to make calls to ~13 AWS services using the AWS SDK. Not all of these services have private link and even if they did, adding private link for each of these services would add ~$100 per month to the cost of running the solution. As such this means we need to have NAT Gateways so processed running in the private subnet can access the internet. These NAT Gateways must run in public subnets.You cannot currently deploy the solution to an existing VPC. This will be a feature of v2.1.0 that is currently in development (in fact, the feature has already been implemented) but as you can see from our GitHub milestone we are very far from finished development of this version.In regards to the query:is it possible to pivot this traffic over to existing IGW/NAT?Please understand that you could change the cloud formation to use an existing vpc by hardcoding the subnet ids into the template too but then you'd have to maintain a custom version of the installation scripts.You can refer the below GitHub link for the same:https://github.com/aws-solutions/workload-discovery-on-aws/pull/383Please note that you have more concerns and queries regarding this, I would request you to raise a support case with the cloudformation team who is the best one to answer such related queries.Adding on to this, for your reference you can go through the below documents with respect to cloudFormation functionalities:References:Bringing existing resources into CloudFormation management:https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import.htmlImporting existing resources into a stack:https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-existing-stack.htmlResources that support import and drift detection operations:https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-supported-resources.html==================================================CommentShareSUPPORT ENGINEERkalpitabanswered 4 months ago"


Question: Requester pay model on EC2

"I have multiple IAM users attached to my account. I know on S3 I can make them pay for using S3, rather then paying for it myself (https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html) . Is there such an option for EC2 ? What would be the best alternative ?ThanksFollowComment"

Answer: 0Hello!Our Billing experts can discuss this with you in more detail - please create a support case to chat with them directly: http://go.aws/support-center.-RachaelAWSCommentShareEXPERTAWS Support - Raeanswered 3 years ago0Answer from billing support: it is not currently possible.CommentShareKreolanswered 3 years ago


Question: Conversion API not working

"i've tried to change the instance on my conversion api from m5.large to t2.micro because i dont have the money to pay for the m.5 large instance and the costs that come with it. ever since my conversion api gateway isnt working, is this because this instance is not compatible with the gateway, or have i done something wrong?FollowComment"

Answer: "0Accepted AnswerHi - Thanks for checking, can you please check the following?Check if Instance has started and in good standing changing instance type, may be Requirements for new instance type not met or AMI does not support instance typeCheck if Public IPv4 address is releasedCommentShareEXPERTAWS-User-Nitinanswered 2 months agorePost-User-9285719 2 months agois the conversion api stack compatible with the t2 micro instanceShare"


Question: 502 Bad Gateway nginx/1.22.1

Hello. I have deployed IoT cloud web page from Siemens LOGO! LWE software application according instructions included in it. Everything seemed correct  IoT registering successfull and deployment was confirmed. I have got an email address. But error message mentioned above appeared when web address created was tested by web browser. Please could the error message "502 Bad Gateway nginx/1.22.1" give some advice where the problem could be? I only know about default gateway in my router and it is 192.168.68.1.FollowComment


Question: Loogin AWS Console always the same error

"Using Google Chrome, and MAC OS, always when I am login, show me this error, the solution is to reload the page, and works well. I have deleted history, someone knows why is the reason, Thanks.FollowComment"

Answer: "0Does this happen with other browsers?Are you experiencing similar events in, for example, safari?CommentShareEXPERTRiku_Kobayashianswered a month ago"


Question: Cannot save a file with a specific format name and extension in Bucket

"Hello everyone! I'm using a lambda function that saves a log into a bucket but I'm having a problem trying to save  these files with a specific format name and file extension.This is my code:**const createBucket = async (s3, bucket, path, data) => {const putObjectRequest = {// Adds a bucket and key to the body.Bucket: bucket,Key: path,Body: data,ContentType: 'text/plain',};return s3.putObject(putObjectRequest).promise().then(respo => {return respo;}).catch(errs => {throw errs;});}const saveFile = async (bucket, current, date, historyData) => {  const logName = util.format('ekt-%s_%s000000.log', current.toLowerCase(), date);  const savedData = await createBucket(    s3,    bucket.logs,    `${current}/${logName}`,    data  );if (!savedData) {    throw new Error(      `The file hasn't been created: ${current}/${logName}`    );  }  console.log(`The file has been created: ${current}/${logName}`);}**I'm trying to save a file with this format and extension ekt-cp_20230112000000.log but at the end the file is created with this format and extension **ekt-cp_20230112.txt **I have no clue whats going on!FollowComment"

Answer: "0HiI understand you would like to upload a file to s3 using the AWS NodeJS SDK. Below is example code// Load the AWS SDK for Node.jsvar AWS = require('aws-sdk');// Set the region AWS.config.update({region: 'REGION'});// Create S3 service objectvar s3 = new AWS.S3({apiVersion: '2006-03-01'});// call S3 to retrieve upload file to specified bucketvar uploadParams = {Bucket: process.argv[2], Key: '', Body: ''};var file = process.argv[3];// Configure the file stream and obtain the upload parametersvar fs = require('fs');var fileStream = fs.createReadStream(file);fileStream.on('error', function(err) {  console.log('File Error', err);});uploadParams.Body = fileStream;var path = require('path');uploadParams.Key = path.basename(file);// call S3 to retrieve upload file to specified buckets3.upload (uploadParams, function (err, data) {  if (err) {    console.log("Error", err);  } if (data) {    console.log("Upload Success", data.Location);  }});You would call the above code with the following command:node s3_upload.js BUCKET_NAME FILE_NAMEPlease note, you should use the upload API to upload files, and not the createBucket API call.I hope this helps.CommentShareRidwaan_M-Lanswered 2 months agorePost-User-2907139 2 months agoNo, I'm not trying to upload a file. I'm creating a new file with data. The file is created correctly except the name and file extension that I'm defining in my codeShare"


Question: Only one of several virtual nice-dcv sessions may be connected to

"I have an application that requires a single shared centos 7 instance that hosts several user accounts. Various users will remotely log into their respective accounts (sometimes concurrently) using system authentication (their linux passwords) to do work through niceDCV. I have setup a script that will automatically launch several niceDCV sessions for each user that follows this format:sudo dcv create-session --owner <username1> --user <username1> <username1>-sessionsudo dcv create-session --owner <username2> --user <username2> <username2>-sessionAfter running this script, if I run: "sudo dcv list-sessions", I get confirmation that there is a session for both users. If I try connecting with nicedcv client, only one is successful. They other reports the following: "Unable to connect. Check the hostname/address and try again or contact the administrator".Other interesting points:Using "sudo dcv describe-session" I can get extra information about the sessions. The session I can connect to is the one with lowest X display number.This remains true even if a third dcv session is established (only the one with the lowest X display number connects)I can establish two connections in parallel if I follow these steps. :launch one dcv session for the user with the higher X display numberconnect through nicedcv to the user with the higher X display numberlaunch another dcv session for the user with the lower X display numberconnect through nicedcv to the user with the lower X display numberOn a failed connect, the dcv log gets updated (as shown in the anonymized sample below). In this example, when username1 attempts to login, a warning is thrown because it cannot create a connection to username2's session<timestamp> INFO  authenticator - Received authentication request from client <IP><timestamp> INFO  authenticator - Sending authentication result OK to <IP> for user <username1> (2 sessions)<timestamp> INFO  authenticator - Cannot authenticate client <IP> Error reading data from TLS socket: error:00000005:lib(0):func(0):DH lib<timestamp> INFO  authenticator - Requesting token authentication for session <username2>-session using 'system' method<timestamp> WARN  frontend-handler - Cannot create connection from client <IP> to session '<username2>-session': User '<username>' not authorized on any channelIs there a way to correct this behaviour so that when multiple dcv sessions are running they can all be connected to?FollowComment"

Answer: "0Accepted AnswerHi David,did you try adding the session-id to specify which session the DCV client should connect to based on your example with #<username1>-session?So in the client you would enter ip_or_hostname#<username1>-session.Best regards,KarstenCommentShareKarstenNISPanswered 2 years ago0That worked! That quick resolution was super satisfying! Thank you!Edited by: ProlucidDavid on Jan 8, 2021 7:02 AMCommentShareProlucidVitaaDavidanswered 2 years ago"


Question: AWS glue header doesn't work properly

"Hello. I am using aws glue, but the aws glue header does not automatically recognize it.In other words, the header is included in the data and the query does not work properly.The header is Korean, and I tried classfier, but it didn't work. Is there a solution?plus. I use Athena to get the result.FollowComment"

Answer: "0The classifier has an option to enter the column names if your columns are not dynamic.If that would not work, you could try one of the following:Change the column names in the Data Catalog, set the SchemaChangePolicy to LOG, and set the partition output configuration to InheritFromTable for future crawler runs.Create a custom grok classifier to parse the data and assign the columns that you want.CommentShareAnanth Tirumanuranswered 8 months ago"


Question: QuickSight SPICE Refresh failed with permission denied

"In SPICE -> I have one simple query ag select * from <db>.public.<tableName>and when I try to full refresh the spice it give me error assourceErrorCode:500310sourceErrorMessage:Amazon Invalid operation: permission denied for relation <tableName>;sourceErrorState:42501sourceException:java.sql.SQLExceptionsourceType:REDSHIFTFYI: I can perform the same query on query editor, it works fine. I also granted the full permission to my user using GRANT ALL PRIVILEGES ON ALL FUNCTIONS IN SCHEMA public to <userName>;any help?FollowCommentFabrizio@AWS EXPERTa year agojust to confirm: have you checked if the Redshift DataSource is using the User you expect for the connection?Share"

Answer: "0Thank you for posting your query, Please check your Redshift DataSource to confirm if the user you are using in the DataSource and the user that you have granted the privileges are the same. If you still have issues, please raise a ticket with AWS support so we can investigate further.CommentShareSUPPORT ENGINEERChaitanya_Tanswered a year ago"


Question: windows-base:2019-2.0 has wrong Python version

"Reading https://docs.aws.amazon.com/codebuild/latest/userguide/available-runtimes.html, windows-base:2019-2.0 image should have Python 3.10.4, but, if I run a sample instance, I get this:[Container] 2022/09/01 12:34:27 Running command python --versionPython 3.8.3Is this a bug on the image, or the documentation?FollowComment"


Question: connection timeout to SNS URL

"I am trying to allow my application to reach sns through a newly created interface sns vpc endpoint. The application has the same security groups as the sns vpc endpoint and is in the same subnet group specified when endpoint was created. We do not have specific NACL rules that would block the services IP yet when I try and use aws sns publish --region region --topic-arn sns arn --message "test" , we receive the endpoint timeout url. Is there something we may have missed?FollowComment"

Answer: "0Hello,Did you check the SG and allow the same in the SNS for the new application to access ?Are you using the correct IAM role for the application or Does that IAM role have access to SNS ?Link- https://docs.aws.amazon.com/sns/latest/dg/sns-authentication-and-access-control.html -- [1]You can use X-Ray with Amazon SNS to trace and analyze the messages that travel through your application.CommentShareSUPPORT ENGINEERAWS-User-Chiraganswered 9 months agorePost-User-8617247 9 months agoYes, both the web server and the sns vpe endpoint are using the same security group. We do not use xray at this time, maybe that can be another debugging option, though we can see rejected traffic in the Eni vpc flow logs.Share0Can you please create a separate Security Group for the SNS VPC end-point that allows HTTPS (443) inbound from the security group of the machine where your application is running? Also make sure that the VPC endpoint for SNS is in the correct region. As Chirag has mentioned, I would also make sure IAM permissions allow the role of the application to send SNS messages.CommentShareEXPERTIndranil Banerjee AWSanswered 9 months ago"


Question: AWS EC2 Linux Fails during boot(ssh not available) - IMDVS1 Disabled

"I was having the following issue when running an instance with IMDVs1 disabled. My user_data.sh is empty since I am not doing anything special but seems like cloud-init is trying to retrieve the instance-id. Is there a way to update the cloud-init configuration during boot to accept IMDVs2?[  OK  ] Reached target Remote File Systems.[    9.155309] cloud-init[633]: Cloud-init v. 18.5 running 'init-local' at Sat, 05 Mar 2022 23:56:27 +0000. Up 9.12 seconds.[   60.325042] cloud-init[633]: 2022-03-05 23:57:19,050 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: bad status code [401][   61.338391] cloud-init[633]: 2022-03-05 23:57:20,063 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [1/120s]: bad status code [401][   62.345464] cloud-init[633]: 2022-03-05 23:57:21,070 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [2/120s]: bad status code [401][   63.350126] cloud-init[633]: 2022-03-05 23:57:22,075 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [3/120s]: bad status code [401][   64.361486] cloud-init[633]: 2022-03-05 23:57:23,086 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [4/120s]: bad status code [401][   65.366914] cloud-init[633]: 2022-03-05 23:57:24,091 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [5/120s]: bad status code [401][   67.372195] cloud-init[633]: 2022-03-05 23:57:26,097 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [7/120s]: bad status code [401][   69.377789] cloud-init[633]: 2022-03-05 23:57:28,102 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [9/120s]: bad status code [401][   71.384404] cloud-init[633]: 2022-03-05 23:57:30,109 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [11/120s]: bad status code [401][   73.390297] cloud-init[633]: 2022-03-05 23:57:32,115 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [13/120s]: bad status code [401]FollowCommentGuillaume Delacour a year agoThe AMI seems to use an old version of cloud-init that doesn't support IMDSv2; it's supported only starting from version 19.4 (you have 18.5): https://github.com/canonical/cloud-init/blob/main/ChangeLog#L1051Shareaws-lmayorga a year agoSo, how can I check which version of cloud-init I am running. Just by checking the newest ami log?Share"

Answer: 0The issue is solved with a new ami with the newest cloud-init packageCommentShareaws-lmayorgaanswered 10 months ago


Question: How can I use Azure AD credentials for SSH into AWS EC2 Instance?

"I would like to use Azure AD credentials for signing into my AWS Linux EC2.So far, I have managed to setup sso using these steps:https://docs.microsoft.com/en-us/azure/active-directory/saas-apps/amazon-web-service-tutorial#configure-aws-single-account-access-ssoThe steps above are successful at getting a user to the management console.I would like to ssh or use the steps above to directly access an instance if possible.How can I accomplish this?I am in a cloud only environment.We use Azure AD as our only IDP.We do not have Microsoft Active Directory.FollowComment"

Answer: 0Accepted AnswerConsider using AWS Systems Manager Session Manager to access your instances.CommentShareEXPERTkentradanswered a year ago


Question: Target deregistration / unhealthy for ECS loadbalancer

"I have tried creating an ECS cluster using both fargate and EC2 to use one of my prebuilt containers in ECR. I have done this using both the console and terraform code based on this blog: https://medium.com/avmconsulting-blog/how-to-deploy-a-dockerised-node-js-application-on-aws-ecs-with-terraform-3e6bceb48785, but I still face the target deregistration / target unhealthy. My container is on port 8000 for reference. Right now the container is hosted on EC2, but I'd like to use ECS for scalability. Is there a simple and fool proof guide I can follow for this?FollowComment"

Answer: "1Accepted AnswerDoes the application respond to the health check configured in the load balancer? To pass a successful health check the application needs to respond to the ping. This can be impacted by misconfiguration in the port/path fields in the health check for the target group, security groups attached to the ECS task/load balancer, or an application that isn't serving requests. This knowledge center article has some steps to troubleshoot the health checks.CommentShareNoah_Lanswered 4 months ago"


Question: URGENT: AWS Certification Account Name setup; No Last Name (Non-Matching IDs)

"Hi.I am Indian passport holder. I do not have a surname/last name mentioned in my passport. My passport has given name as "Salim Ali Siddiq" and the surname/last name section is blank. Due to this, my US visa and subsequently my California driver's license have the first name as "FNU" and last name as "Salim Ali Siddiq". This is because the US immigration services, needs to have a last name and hence they use my given name as last name and mention the first name as "FNU" which is first name unknown.I see that the certification test needs two ID proofs. I have my driver's license and passport as ID proof. Both have different names. Which name and IDs should I use?In the final AWS certificate, I'd like to have my name as "Salim Ali Siddiq".Please help me out as soon as possible. I'm trying to schedule a test and I have to sort out this issue before I can schedule a test. Your quick and prompt response is highly appreciated.FollowComment"

Answer: "1You will have to contact AWS Training and Certification support. Have you tried reaching out to them?https://support.aws.amazon.com/#/contacts/aws-trainingIf you only have a first name or last name, please contact AWS Training and Certification support to ensure your AWS Certification Account name is acceptable for test day.CommentShareSoumyaanswered 10 months agoSalim Ali Siddiq 10 months agoThank you for the response. Yes, I tried reaching out to them but have not received any response.Share"


Question: Problem instanciating ILA in TCL workflow

"Hello.I am trying to add an ILA to check the behavior of an AXI bus in my design.My design works in simulation but gets stuck on the FPGA. In order to debug this I want to check an AXI bus with an ILA.I used the following document as a reference : https://github.com/aws/aws-fpga/blob/master/hdk/docs/Virtual_JTAG_XVC.mdAnd the "hello_world" design as an example.I instanciated the debug bridge in my top with the following lines :        // Debug Bridge        cl_debug_bridge CL_DEBUG_BRIDGE (            .clk(clk_main_a0),            .S_BSCAN_drck(drck),            .S_BSCAN_shift(shift),            .S_BSCAN_tdi(tdi),            .S_BSCAN_update(update),            .S_BSCAN_sel(sel),            .S_BSCAN_tdo(tdo),            .S_BSCAN_tms(tms),            .S_BSCAN_tck(tck),            .S_BSCAN_runtest(runtest),            .S_BSCAN_reset(reset),            .S_BSCAN_capture(capture),            .S_BSCAN_bscanid_en(bscanid_en)        );I created an ILA using the IP catalog and a board diagram, the board diagram is because the ILA names the input ports probeXX instead of the AXI signal names and with the board diagram I can make a wrapper with the correct names.I added the ILA in the design and got the following errors on synthesis :INFO: [Common 17-349] Got license for feature 'Synthesis' and/or device 'xcvu9p'INFO: [Vivado_Tcl 4-550] User specified maximum number of URAM that can be cascaded is 2CRITICAL WARNING: [filemgmt 20-1741] File 'xsdbm_v3_0_0_in.vh' is used by one or more modules, but with different contents, and may lead to unpredictable results:* bd_a493_xsdbm_0 (/home/user/design/aws-fpga/hdk/common/shell_v04261818/design/ip/cl_debug_bridge/bd_0/ip/ip_0/hdl/verilog/xsdbm_v3_0_0_in.vh)* test_axi_ila_1_ila_0_0 (/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/verilog/xsdbm_v3_0_0_in.vh)Please reset and regenerate these modules to resolve the differences, or synthesize them independently.CRITICAL WARNING: [filemgmt 20-1741] File 'xsdbm_v3_0_0_bs.vh' is used by one or more modules, but with different contents, and may lead to unpredictable results:* bd_a493_xsdbm_0 (/home/user/design/aws-fpga/hdk/common/shell_v04261818/design/ip/cl_debug_bridge/bd_0/ip/ip_0/hdl/verilog/xsdbm_v3_0_0_bs.vh)* test_axi_ila_1_ila_0_0 (/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/verilog/xsdbm_v3_0_0_bs.vh)Please reset and regenerate these modules to resolve the differences, or synthesize them independently.CRITICAL WARNING: [filemgmt 20-1741] File 'xsdbm_v3_0_0_bs_core.vh' is used by one or more modules, but with different contents, and may lead to unpredictable results:* bd_a493_xsdbm_0 (/home/user/design/aws-fpga/hdk/common/shell_v04261818/design/ip/cl_debug_bridge/bd_0/ip/ip_0/hdl/verilog/xsdbm_v3_0_0_bs_core.vh)* test_axi_ila_1_ila_0_0 (/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/verilog/xsdbm_v3_0_0_bs_core.vh)Please reset and regenerate these modules to resolve the differences, or synthesize them independently.CRITICAL WARNING: [filemgmt 20-1741] File 'xsdbm_v3_0_vl_rfs.v' is used by one or more modules, but with different contents, and may lead to unpredictable results:* bd_a493_xsdbm_0 (/home/user/design/aws-fpga/hdk/common/shell_v04261818/design/ip/cl_debug_bridge/bd_0/ip/ip_0/hdl/xsdbm_v3_0_vl_rfs.v)* test_axi_ila_1_ila_0_0 (/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v)Please reset and regenerate these modules to resolve the differences, or synthesize them independently.INFO: Launching helper process for spawning children vivado processesINFO: Helper process launched with PID 17187 ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:5838]ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:5867]ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:5900]ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:5934]ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:9887]ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:9916]ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:9949]ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v:9983]Failed to read verilog '/home/user/design/aws-fpga/hdk/cl/developer_designs/new_synth/ip/bd/test_axi_ila_1/ipshared/6851/hdl/xsdbm_v3_0_vl_rfs.v'INFO: [Common 17-83] Releasing license: Synthesis6 Infos, 17 Warnings, 4 Critical Warnings and 9 Errors encountered.synth_design failedERROR: [Common 17-69] Command failed: Synthesis failed - please see the console or run log file for details    while executing"source -notrace ./synth_${CL_MODULE}.tcl"    invoked from within"if {${cl.synth}} {   source -notrace ./synth_${CL_MODULE}.tcl}"    (file "create_dcp_from_cl.tcl" line 196)INFO: [Common 17-206] Exiting Vivado at Mon May 27 15:52:58 2019...There is a conflict between the sources used by the debug bridge IP and my ILA IP.I followed the instructions in https://github.com/aws/aws-fpga/blob/master/hdk/docs/Virtual_JTAG_XVC.md Embedding Debug Cores in the CL. But I could not find how to solve this.Do you have any suggestion on how to solve this ?Thank you very much. Best Regads.FollowComment"

Answer: "0Hi,Does the aws-fpga hello world example with debug work for you?Have you tried adding debug to the IPI hello world example?For your design, "ILA using the IP catalog and a board diagram", What version of the ILA did you use?  The versions the example use are located:  https://github.com/aws/aws-fpga/tree/master/hdk/common/shell_v04261818/design/ip/What version of the tools are you using?ThanksKrisCommentShareAWS-kristopkanswered 4 years ago0Hello.I can build the hello world example.I used the ILA version 6.2 from the IP catalog. I did use Vivado 2017.4 to generate it.I use Vivado 2018.3 to build the project, several IPs are still from Vivado 2017.4 because they were created in older projects. The version of the CL shell I use is v04261818, HDK ver 1.4.8.I did not try any IPI example or flow.CommentSharerweanswered 4 years ago0Ok, thanks for confirming the hello world plus debug is working.Instead of generating ILA, please try to use the IP (ILA and debug bridge) included in the developer kit.https://github.com/aws/aws-fpga/tree/master/hdk/common/shell_v04261818/design/ipCommentShareAWS-kristopkanswered 4 years ago0Hello,I will try modifying and using the ILAs that were provided.I need to modify them to change the number of ports, as well as their size.I will report on this next week. Thank you.CommentSharerweanswered 4 years ago0Hello,Were you able to resolve this issue ?ThanksCommentShareawsSiddharthanswered 4 years ago0Yes,Using and modifying the existing ILA core did work. Thank you.Regards.CommentSharerweanswered 4 years ago"


Question: Amazon Inspector Public Bucket Access Denied

"We have an 'unauthorised API call' alarm that is being tripped by Amazon Inspector.It's attempting to download windows.zip from an AWS Public Bucket. Here is a snippet of the Cloudwatch log:-    "eventSource": "s3.amazonaws.com",    "eventName": "GetObject",    "awsRegion": "ap-southeast-2",    "sourceIPAddress": "Redacted IP Address",    "userAgent": "[aws-sdk-go/1.44.78 (go1.18.3; windows; amd64) amazon-ssm-agent/]",    "errorCode": "AccessDenied",    "errorMessage": "Access Denied",    "requestParameters": {        "bucketName": "aws-ssm-document-attachments-ap-southeast-2",        "Host": "aws-ssm-document-attachments-ap-southeast-2.s3.ap-southeast-2.amazonaws.com",        "key": "e89/810622359321/AmazonInspector2-InspectorSsmPlugin!d6f98620-d464-4b63-ab7c-e10b41c673c6/20/windows.zip"    },We've setup an Instance role and attached permissions policy as specified here:-https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-minimum-s3-permissions.htmlIn fact, I added GetBucket* and ListBuckets at the Bucket level as well just in case that was the issue.When I look in Systems Manager, Fleet Manager it showed failures invoking AmazonInspector2-ConfigureInspectorSsmPlugin and AmazonInspector2-InvokeInspectorSsmPlugin for the instance in question.We are getting quite a few of these 'Access Denied' errors, but I have allocated the listed permissions. I also checked that my VPC Endpoint Policy does not restrict access.The only issue might be there is a Service Control Policy that is denying access. Has anybody got any other insights as to what might be causing this?FollowComment"

Answer: "0Try the Policy Simulator on the role. In the console, find the role, click on the Simulate button and then set up the simulator for the GetObject and set the Object key. This may tell you what is blocking access or at least eliminate some policies.CommentShareEXPERTkentradanswered 5 months agoAlan111S 5 months agoThanks for your answer. Unfortunately I tried that and the policy is allowed on both the Bucket and the Resource.What was interesting though was my executing the policy simulator triggered the Access Denied alarm for 'GetBucketPolicy ' using my own user but when I checked the results, I have 'Allow' permissions.Share"


Question: Delayed Replication w/ Aurora MySQL

"A customer is interested in implementing Delayed Replication for their Aurora MySQL DB instances as a way to recover quickly from a failed database migration or human error. It looks like this feature was added to RDS for MySQL in August 2018 , but not Aurora MySQL.Is this something that can be implemented with Aurora MySQL?For Aurora MySQL customers looking to have a multi-hour delay between replicas, what is the best path forward? Should Aurora Backtrack be instead positioned here?FollowComment"

Answer: "0Accepted AnswerYes, Aurora backtrack is the solution for the delayed replication however the limit for a backtrack window is 72 hours. Also, there are other limitation.Backtrack Demonstration: https://www.youtube.com/watch?v=20xXnoQA2j4&feature=youtu.be&t=1468The RDS MySQL the replication uses the traditional binary log MySQL replication between the MASTER and Read Replicas.However, in Amazon Aurora replicas share the same underlying storage as the source instance and replication happens on the storage layer (Without binary logs). For a write to be considered durable in Aurora, the primary instance (“master”) sends a redo stream to six storage nodes, two in each availability zone for the storage volume, and waits until four of the six nodes have responded.CommentShareVijayKanswered 4 years ago"


Question: IoT Rule -> Timestream

Does IoT Core batch messages to Timestream or invoke the WriteRecords API for every incoming message? The Timestream "Best Practices" docs recommends batching to optimize data ingestion performance & cost.https://docs.aws.amazon.com/timestream/latest/developerguide/data-ingest.htmlThanks.FollowComment

Answer: "0Hi otl. An AWS IoT rule is evaluated against each incoming message. So if you use the Timestream rule action it will not batch messages. If you are handling a high volume of messages, you may instead consider to use a different rule action to batch the messages in another service before writing into Timestream. This blog explains using Kinesis Data Streams for the batching: https://aws.amazon.com/blogs/database/near-real-time-processing-with-amazon-kinesis-amazon-timestream-and-grafana/If you have control over the device firmware, you might also consider to batch messages on the device side. IoT messages are billed in 5KB increments. Therefore you can reduce costs by putting multiple measurements into a single message up to 5KB (if the timing constraints of your application will allow it): https://docs.aws.amazon.com/wellarchitected/latest/iot-lens/optimizing-over-time.html. The Timestream rule action can only handle a single timestamp per message. So if you batched on the device like this, you would have to use the Lambda rule action to write into Timestream.CommentShareEXPERTGreg_Banswered 7 months ago"


Question: "Using aws S3 in spring boot, while trying to access awsClient I am getting some unexpected error."

"`com.amazonaws.SdkClientException: Unable to unmarshall response (Unexpected character 'd' (code 100) after '<!' (malformed comment?)at [row,col {unknown-source}]: [1,3]). Response Code: 200Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character 'd' (code 100) after '<!' (malformed comment?)at [row,col {unknown-source}]: [1,3]`it is working fine in sts and docker-compose but when I am running it in ocp (OpenShift Container Platform) it is giving me this error.FollowComment"


Question: Default Password for Lightsail Windows Server

"I created a Windows Server on Lightsail 4 days ago and did not change the password nor have I  been able to login to the server yet.When using the AWS Browser RDP I get prompted for the Admin password.When I go below and click on the link to retrieve the default password I get the error "Your Password isn't available yet, try again later".How Do I get the default password for a Windows Instance on lightsail? Does it take more than 4 days to create?thanks in advanceFollowComment"

Answer: "0Hi, @Mark.No, you can get it immediately after a few minutes of creation.If you restore the administrator password from the changed snapshot, you will not be able to get the password. So I'm guessing you changed from the snapshot.In that case, you need the modified password that was configured before the snapshot was taken.I previously wrote the following article about it.https://dev.classmethod.jp/articles/browse-to-windows-instance-that-has-a-lightsail-snapshot-feature/If you want to recover, you may be able to convert Lightsail to EC2 and reset your password with EC2 Rescue.https://aws.amazon.com/jp/premiumsupport/knowledge-center/lightsail-migrate-to-ec2/https://dev.classmethod.jp/articles/ec2-windows-password-reset/If it happens in a blueprint instead of a snapshot, please tell me the region and blueprint you created.CommentShareEXPERTiwasaanswered a year ago"


Question: "Using kms:GetKeyRotationStatus on AWS managed key(ex aws/acm), AWS doesn't grant access to this API call to anyone, including the account owner. Is that correct?"

"I'm testing Java AP with KMS API.When I do kms:GetKeyRotationStatus on AWS managed key "aws/acm", It will fail no matter what permissions are set.AccessDeniedException is displayed.<User: arn:aws:iam::xxxxxxxx:user/zzzz is not authorized to perform: kms:GetKeyRotationStatus on resource: arn:aws:kms:ap-northeast-1:xxxxxxxxxx:key/xxxxxxxxxxxxxxxxxx because no resource-based policy allows the kms:GetKeyRotationStatus action>So, Although the documentation doesn't explain it, using kms:GetKeyRotationStatus on AWS managed key(ex aws/acm), AWS doesn't grant access to this API call to anyone, including the account owner. Is that correct?FollowComment"

Answer: "0Hi, I was able to get the KeyRotationStatus on a aws-managed kms key.Make sure that the user/role used by the JAVA AP is in the same region as the KMS managed key, and that it really has enough permissions. You should then be able to run the following:$aws kms list-keys                                                                                {    "Keys": [        {            "KeyId": "e1xxxxx-e6xx-45xx-xxxx-xxxxxxxxxxxxx",            "KeyArn": "arn:aws:kms:us-west-2:000000000000:key/e1xxxxx-e6xx-45xx-xxxx-xxxxxxxxxxxxx"        }    ]}followed by$ aws kms get-key-rotation-status --key-id e1xxxxx-e6xx-45xx-xxxx-xxxxxxxxxxxxx                                                                                {    "KeyRotationEnabled": true}That key id belongs to an aws-managed KMS key (aws/sns)Hope this helps.CommentShareZadusPlaceanswered a year ago"


Question: Unable to enable enhanced networking on a Centos 7.9 instance.

"Hi,I have Centos 7.9 on a m4.large instance that I want to migrate to m5.large.I'm trying to enable enhanced networking and despite the fact that I followed the documentation again and again, when I issue the command ethtool -i eth0 the answer is still as the following:driver: vifversion:firmware-version:expansion-rom-version:bus-info: vif-0supports-statistics: yessupports-test: nosupports-eeprom-access: nosupports-register-dump: nosupports-priv-flags: noThere is no way to get answer such as driver: enaI enabled enhanced networking on some other instances in the past and it worked just fine however with this one, I'm stuck.Of course, Instance attribute (enaSupport) is set on the instance.Any help would be appreciated.Thank you.FollowComment"

Answer: "0Finally, the problem was not related to the "enhanced networking" as I thought first.I thought it because of the error message I got when I tried to launch the m5.large instance was:Failed to start the instanceThe requested configuration is currently not supported. Please check the documentation for supported configurations.Finally, this error message was displayed because the m5.large instance is not available in the current AZ of my instance.CommentShareg_planteanswered 2 years ago"


Question: Scaling authorization policies with AWS IoT Core

The number of policies that can be attached to a Amazon Cognito identity is 10. This means that our customers are limited to a maximum of 10 devices.I read a solution suggestion to solve this in the below document:https://aws.amazon.com/tr/blogs/iot/scaling-authorization-policies-with-aws-iot-core/Multiple device topics into a single policy.For publish and receive actions (85 characters):arn:aws:iot:us-east-1:123456789012:topic/$aws/things/cheryl-kitchen-light-01/shadow/*But I am using the jitr method. This doesn't seem appropriate to me.on https://docs.aws.amazon.com/general/latest/gr/iot-core.html:Maximum number of policies that can be attached to a certificate or Amazon Cognito identity:  default value is 10 : not adjustableWhat should I do in this situation?FollowComment

Answer: "0Policy variables are a way to scale one IoT policy to many devices:https://docs.aws.amazon.com/iot/latest/developerguide/basic-policy-variables.htmlhttps://docs.aws.amazon.com/iot/latest/developerguide/example-iot-policies.htmlhttps://docs.aws.amazon.com/iot/latest/developerguide/pub-sub-policy.htmlThe blog you reference uses wildcards and partial topic matching to achieve similar.Please also refer to this whitepaper: https://docs.aws.amazon.com/whitepapers/latest/designing-mqtt-topics-aws-iot-core/designing-mqtt-topics-aws-iot-core.htmlAnd particularly: https://docs.aws.amazon.com/whitepapers/latest/designing-mqtt-topics-aws-iot-core/mqtt-design-best-practices.htmlI think this quote may be of interest to you:Consider using Named Shadows to create logical groups of properties. You can create a unique access policy for each Named Shadow, therefore controlling what applications or services can view or update that group of properties. An example of this would be the device management team viewing the firmware, battery health, or WiFi signal strength but not having access to the data being published by the sensors on said devices.Named shadows didn't exist when that blog was published. With named shadows you could use a policy variable for the thing name and reference the shadow name explicitly.CommentShareEXPERTGreg_Banswered a year agoAWS-User-8111104 a year agoI will carefully read the documents you shared. Please allow me to ask a question first. Is your answer compatible with the JITR method?Because in the jitr method, the policy that matches the device is created through the lambda function. I will also use mqtt pub sub instead of shadow.ShareGreg_B EXPERTa year agoYes. The IoT policy created by the Lambda function can include policy variables (and after creating it once, it won't be recreated). Alternatively the Lambda can just attach a pre-existing policy (that has policy variables).Share0As noted in your question, the approach described in the blog post you have referred to does not work with the JITR/JITP process, as it requires knowing the user claiming the device at the time of the device onboarding.A possible solution is two have a 2 step process: first, as the device is powered up and connected to Internet, it gets automatically onboarded in AWS IoT Core via the JITP/JITR process. At this stage, your device will be represented with a thing using a unique device id as thing name and will have a unique certificate. In order to avoid consuming any of the 10 directly attachable certificate policies, you can leverage Thing Group IoT Policies (https://docs.aws.amazon.com/iot/latest/developerguide/thing-groups.html#group-attach-policy) creating a generic policy that uses ioT policy variables and attaching that to the thing group. The device thing is then attached to the thing group. Let's call it unclaimed group.Example policy to be attached to the Thing Group that allows the device to only process claiming messages, ie not being fully functional:{    "Version": "2012-10-17",    "Statement": [      {        "Effect": "Allow",        "Action":["iot:Subscribe"],        "Resource": ["arn:aws:iot:us-east-1:123456789012:topicfilter/claimed/${iot:Connection.Thing.ThingName}"]      },     {        "Effect": "Allow",        "Action":["iot:Publish"],        "Resource": ["arn:aws:iot:us-east-1:123456789012:topic/claimed/${iot:Connection.Thing.ThingName}/confirmed"]      },      {        "Effect": "Allow",        "Action": ["iot:Connect"],        "Resource": ["arn:aws:iot:us-east-1:123456789012:client/${iot:Connection.Thing.ThingName}"]      }    ]}In a second phase a user claims the device via a companion app you are providing. At this point you:create a new Thing according to the naming convention described in the blog post (eg <unique-user-id>-<device-id>)attach the thing to a claimed Thing Group which has a policy allowing the device full operationsattach the existing device certificate to the new Thing you createdsend a message to the device to communicate the new thing namethe device responds back confirming it has gotten the new namethe device reconnects with the new thing name as clientIdIn addition, the CognitoId principal for the user has a policy attached that allows it to connect to IoT Core and communicate with all devices matching the <unique-user-id> prefix. This policy is created when the user creates the account on your platform.Example of the IoT policy attached to the Cognito id principal:{    "Version": "2012-10-17",    "Statement": [      {        "Effect": "Allow",        "Action":["iot:Subscribe"],        "Resource": ["arn:aws:iot:us-east-1:123456789012:topicfilter/<unique-user-id>*"]      },     {        "Effect": "Allow",        "Action":["iot:Publish"],        "Resource": ["arn:aws:iot:us-east-1:123456789012:topic/<unique-user-id>*"]      },      {        "Effect": "Allow",        "Action": ["iot:Connect"],        "Resource": ["arn:aws:iot:us-east-1:123456789012:client/${iot:Connection.Thing.ThingName}"]      }    ]}This said, if your solution is about having a users being able to control multiple devices via an application I would advice to use the approach described in this blog post https://aws.amazon.com/blogs/iot/connecting-home-appliances-with-a-smart-home-solution-built-on-aws-in-the-aws-china-region/ instead of letting the application communicate directly with AWS IoT Core.A similar approach is also adopted by the AWS Smart Product solution https://docs.aws.amazon.com/solutions/latest/smart-product-solution/architecture.htmlCommentShareEXPERTMassimilianoAWSanswered a year ago"


Question: Error on create view

"Hi, i'm trying to upgrade from athena version 2 to 3 and  the following query fails now with the error bellow. I run the select alone and it works fine. Did anything change in the new version that i missed ?FollowCommentalatech EXPERT2 months agoTry check if done breaking change affected you https://docs.aws.amazon.com/athena/latest/ug/engine-versions-reference-0003.html#engine-versions-reference-0003-breaking-changes.Share"

Answer: "0im not sure what breaking change affects it so ended up casting to timestamp(3) without timezoneUNNEST("sequence"(cast(from_iso8601_timestamp('2021-01-01') as timestamp), cast(from_iso8601_timestamp(to_iso8601(current_date + INTERVAL  '1' YEAR)) as timestamp), INTERVAL  '1' DAY)) sequence (day)CommentSharerePost-User-2235301answered 2 months ago"


Question: Is anyone aware of expiry dates of some of the AWS APN programs?

"My team is an AWS APN partner and Here are some of the questions they have:We have recently been approved for AWS Service Delivery Program  for a particular Service, how long is it valid for?We have recently been approved for AWS Competency Program for a particular Competency, how long is it valid for?AWS Service Ready Program as well.I understand that the AWS FTR are valid for 2years and AWS Certifications for 2/3 years depending on which you choose.What about this programs, If anyone can share some information, that would really help.FollowComment"

Answer: "0Please submit a ticket to the AWS Training & Certification customer support team, and they will provide assistance. If you already submitted a case, please allow at least 48 hrs to get a response.CommentShareNelliT-AWSanswered 3 months ago"


Question: Can I limit the type of instances that data scientists can launch for training jobs in SageMaker?

"We want to limit the types of instances that our data scientists can launch for running training jobs and hyperparameter tuning jobs in SageMaker. Is it possible to limit the instance size options available through SageMaker by using IAM policies, or another method? For example: Could we remove the ability to launch ml.p3.16xlarge instances?FollowComment"

Answer: "0Accepted AnswerYes, you can limit the types of instances that are available for your data scientists to launch in SageMaker by using an IAM policy similar to the following one:Note: This example IAM policy allows SageMaker users to launch only Compute Optimized (ml.c5)-type training jobs and hyperparameter tuning jobs.{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "EnforceInstanceType",            "Effect": "Allow",            "Action": [                "sagemaker:CreateTrainingJob",                "sagemaker:CreateHyperParameterTuningJob"            ],            "Resource": "*",            "Condition": {                "ForAllValues:StringLike": {                    "sagemaker:InstanceTypes": ["ml.c5.*"]                }            }        }     ]}CommentShareMichael Hoarauanswered 3 years agoEXPERTTasioreviewed 6 months ago"


Question: can't receive SES email verification.

"Hey,as the title say, my domain is verified, but when i try to verify my mails "exemple@domain.com" everything is well, except the emails verification.FollowComment"

Answer: "1If you have a verified domain identity then you do not need to verify email address identities within the domain. Furthermore, you should not verify email address identities within a verified domain identity because: a) the more specific email address identity supersedes the domain identity when you send messages, and b) email address identities cannot be authenticated which can lead to deliverability issues. Once your domain identity is verified, you can send as any email address within the domain.CommentShareJesse_Tanswered 2 months ago0Hello, following the thread, if I already configured SES with S3 Bucket rule set, and SNS with email verified already, and of course domain verified. What I'm doing wrong that when I'm trying to send and email to user@domainverified.com bounce back with this message?DNS Error: DNS type 'mx' lookup of domainverified.com responded with code NOERROR DNS type 'mx' lookup of domainverified.com had no relevant answers. DNS type 'aaaa' lookup of domainverified.com responded with code NOERROR DNS type 'aaaa' lookup of domainverified.com had no relevant answers. DNS type 'a' lookup of domainverified.com responded with code NOERROR DNS type 'a' lookup of domainverified.com had no relevant answers.I can tell you again that the domain has been verified and all DNS records are correct.Thank you in advanced.Best,CommentSharerePost-User-1773121answered a month ago"


Question: Max connection limit for websocket?

"Hi, is there any explicit limit for websocket max connections?I went through the docs but couldn't find it.The websocket support is super exciting.Looking forward for your reply!FollowComment"

Answer: "0There is no explicit limit for WebSocket max connections.However, we do have these limits that will restrict max connections indirectly:New connections per second per account (across all WebSocket APIs) per region : 500Connection duration for WebSocket APIs : 2 hoursCurrently, all of them are hard limits and cannot be increased. We are planning to allow customers to request limit increases, but we have no set timeline on when that feature will be available.CommentShareAWS-User-3921561answered 4 years ago0Thanks, really glad to hear there is no hard limit on max connections!CommentSharetomodiananswered 4 years ago"


Question: HttpApi invalid error on nested stack

"Im trying to create a nested stack on a cloudformation template, i have declared in the parent application a reference to the http api we are using, and use this api in the child template. When i try to do the build with sam, it throws this error:"E0001 Error transforming template:ApiId must be a valid reference to an 'AWS::Serverless::HttpApi' resource in same template."Parent template declaration:childStack:    Type: "AWS::Serverless::Application"    Properties:      Location: ./child.yaml      Parameters:        ApiId: !Ref ApiReferenceChild template declaration:AWSTemplateFormatVersion: '2010-09-09'Transform: AWS::Serverless-2016-10-31Parameters:  ApiId:    Type: stringGlobals:  Function:    Runtime: !Ref "AWS::NoValue"    Handler: !Ref "AWS::NoValue"    Layers: !Ref "AWS::NoValue"Resources:  lambdaFunctionLogGroup:    Type: 'AWS::Logs::LogGroup'    Properties:      Location: ./parent.yaml      LogGroupName: !Join        - '/'        - - '/aws/lambda'          - !Ref 'TournamentSubscriptionFunction'      RetentionInDays:        !FindInMap [Service, !Ref EnvironmentName, LogRetentionInDays]  lambdaFunction:    Location: ./parent.yaml    Type: AWS::Serverless::Function    Properties:      Description: Image validation for identity verification      FunctionName: !Sub '${EnvironmentName}-lambda'      PackageType: Image      Architectures: ['arm64']      Environment:        Variables:          ExampleVariable      Policies:        - CloudWatchLambdaInsightsExecutionRolePolicy      Events:        Name:          Type: HttpApi          Properties:            Path: /event-client/api/lambda            Method: POST            ApiId: !Ref ApiReference            Auth:              Authorizer: OAuth2Authorizer      VpcConfig:        SubnetIds: !Split          - ','          - Fn::ImportValue: !Sub '${EnvironmentName}-PrivateSubnets'      Tags:        Environment: !Sub '${EnvironmentName}'    Metadata:      DockerTag: nodejs16.x-v1      DockerContext: ../dist/src/client/lambda-route      Dockerfile: DockerfileFollowComment"

Answer: 0According to the docs:This cannot reference an AWS::Serverless::HttpApi resource defined in another template.CommentShareEXPERTkentradanswered 2 months ago


Question: AWS Timestream convert HEX string

"Hi,Is there a way to convert a hex string to an INT  in AWS timestream?  I cant find any functions that are supported to do this.Thanks!MikeFollowComment"

Answer: "1Accepted AnswerHi Mike, there are conversion functions available depending on your hex size:select from_big_endian_64(from_hex('00000000DEEDDEED')) orselect from_big_endian_32(from_hex(‘DEEDDEED’))CommentShareSUPPORT ENGINEERAnushaGanswered 3 months ago"


Question: Add subdomain on existing AWS EC2 instance

"Hi,I'm struggling to make a subdomain to work.What I have done :create an AWS EC2 instance with WHM and CPANEL on domain www.fakename.comsubdomain created in WHM and CPANELcreate the route53 and hosted zone as followedwww.webamazingapps.com. | A | 13.XXX.XXX.183 (elastic IP)www.webamazingapps.com. | NS | ns-796.awsdns-35.net.ns-1744.awsdns-26.co.uk.ns-72.awsdns-09.com.ns-1431.awsdns-50.org.www.webamazingapps.com. | SOA | ns-1744.awsdns-26.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 XXXXXXX 86400commandes.www.webamazingapps.com. | CNAME | 13.XXX.XXX.183 (elastic IP)www.www.webamazingapps.com. | CNAME | 13.XXX.XXX.183 (elastic IP)I have added to my NAMECHEAP domain the following custom DNSns-796.awsdns-35.net.ns-1744.awsdns-26.co.uk.ns-72.awsdns-09.com.ns-1431.awsdns-50.org.if I follow the urlcommandes.webamazingapps.comI've got an ERR_NAME_RESOLUTION_FAILEDI'm stuck, would need some expertise here.Thank you, cheers, MarcEdited by: marcQ on Oct 25, 2019 9:59 PMFollowComment"

Answer: "0Accepted AnswerHello,I have checked on my side your sub-domain name commandes.webamazingapps.com it is resolving.Can you please confirm from your side if you still experiencing the same error you were getting?RegardsMahlwempuCommentSharemahlwempuanswered 4 years ago0Sorry for my response delay. Everything is fine now, I have created a new instance from the scratch and now everything is working fine. Thanks. CheersCommentSharemarcQanswered 4 years ago"


Question: Appstream login disabled

"Hi,My administrator has assigned an app to me through AppStream. However I get the message, "Your login has been disabled. Contact your AppStream 2.0 admin for help." My admin has assured me that the login is not disabled.Could anyone please advise?Thanks in advance.Best regardsVinayakFollowComment"

Answer: "1I can only assume you're referring to User Pools since there is no other information provided. Your administrator will want to double-check if your stack assigned user account is actually disabled.CommentShareEXPERTStevieStetsanswered a year agovinayakgole a year agoThank you Stevie but my admin insists the account has not been disabled.Sharevinayakgole a year agoHello @StevieStets and @Chibuike_A,I have checked with my admin and he has shown me that the account is enabled. Unfortunately I cannot paste a screenshot hereI cannot reset the password, I get the message, "You are not authorized to login"Please AdviseThanks very much for your helpVinayakShare0In addition to StevieStets answer, if your account is in enabled state, I would recommend your administrator to disable and re-enable your account. Also trigger forget password to enable you to reset the password. See Resetting a Forgotten PasswordOnce this is done, then the stack can be reassign the stack to youCommentShareChibuike_Aanswered a year agovinayakgole a year agothank you. Let me as my admin to try the above.Share"


Question: How to failback to Original Writer instance after Upgrade

"Currently i have 1 writer and 2 reader instance in AuroraDB. When I was trying to change the instance write instance type to larger instance, aurora automatically promoted a reader instance to writer. But after the upgrade finish, the original writer instance stayed as reader instance. Is there a way to have a automatic failback to the original writer after the upgrade is completed?FollowComment"

Answer: "0Unfortunately, no. The upgrade is designed to failover first to reduce downtime. The readers and write should be on the similar instance class in Aurora cluster. You can try either of these:Upgrade your new writer now, which would automatically failover to give your original configuration with both instances to new instance sizing.Manually failover the cluster to switch the configuration from console, CLI or API call, and then you can upgrade reader without a failover.CommentShareShivam_Ganswered a year ago"


Question: AWS Batch Succeeded but no logs generated

"Hello Everyone,I have setup AWS Batch Job with AWS RDS Postgresql as the backend. Earlier the jobs I submitted failed due to permission issues. But is fixed now. Earlier I was able to see the Logs and fix the issues. Now the jobs status shows as "SUCCEEDED". but the time it ran is zero and no logs are generated.Can someone guide me with this issue?Thanks in AdvanceViddyaFollowComment"


Question: How can I make one python file run another on Aws

"I want to run multiple python scripts to run one after another These are connected, what is the best way to do it ( want to use Ec2 Instance)FollowComment"

Answer: "0here are several ways to run multiple Python scripts in sequence on an EC2 instance. Here are a few options:Use a shell script: You can create a shell script that calls each Python script in sequence using the python command, and then run that shell script on the EC2 instance. For example, you could create a script called run_scripts.sh with the following contents:#!/bin/bashpython script1.pypython script2.pypython script3.pyThen, you can run this script on your EC2 instance by giving it execute permission (chmod +x run_scripts.sh) and running it (./run_scripts.sh).Use a Python script to run the other scripts: You can create a Python script that calls the other Python scripts in sequence using the subprocess module. For example, you could create a script called run_scripts.py with the following contents:import subprocesssubprocess.run(['python', 'script1.py'])subprocess.run(['python', 'script2.py'])subprocess.run(['python', 'script3.py'])Then, you can run this script on your EC2 instance by running python run_scripts.py.Use a task scheduler: You can use a task scheduler like cron to schedule the execution of each Python script at a specific time. For example, you could create a cron job that runs each script in sequence at midnight every day. To create a cron job, you can run the command crontab -e on your EC2 instance, and then add a line like the following:0 0 * * * python /path/to/script1.py && python /path/to/script2.py && python /path/to/script3.pyThis will run each script in sequence at midnight every day.These are just a few examples of how you can run multiple Python scripts in sequence on an EC2 instance. The best approach for your use case will depend on your specific requirements and constraints.Please let me know if that helpedCommentShareZJonanswered 3 months agorePost-User-7470157 3 months agoBut if we have dependencies like pytorch. How will we install those and if a model that is also in s3 bucket with weights, how we call it in Ec2 instance?ShareZJon 3 months agoInstall dependencies using pip:pip install torchInstall and configure AWS CLI:pip install awscliaws configureCopy model and weights from S3:aws s3 cp s3://your-bucket/path/to/model.pth /local/directory/Load the model in your Python script:import torchmodel = YourModelClass()model.load_state_dict(torch.load('/local/directory/model.pth'))This installs dependencies, sets up AWS CLI, and loads the model from S3 in your Python script on the EC2 instance.Share0Here's an example Python script that accomplishes this:import subprocess# List of Python scripts to execute in orderscripts = ["script1.py", "script2.py", "script3.py"]# Loop through the list and execute each scriptfor script in scripts:    subprocess.run(["python", script])This script uses the subprocess module to execute each Python script in order. The subprocess.run() function runs a command in a subprocess and waits for it to complete. In this case, the command is the Python interpreter with the name of the script as an argument.CommentShareMina Gobrialanswered 3 months ago"


Question: EC2 vs Fargate in ECS/EKS cluster

I want to know what is the major difference between EC2 and Fargate. Why fargate is more expensive it is charge based on minutes. Which one is better to consider for running applicationFollowComment

Answer: "2Accepted AnswerFargate is a managed compute layer by AWS: They manage the underlying host to run the containers into for you.EC2 instances are managed and deployed by yourself, so it is up to you to deal with maintaining these.They both have pros and cons, and I personally much prefer to use Fargate as it is a lot less overhead management, "I just want to have containers running".We have 1000s of containers running on top of Fargate and not having to manage any EC2 hosts is worth the additional costs.Also not having access to the underlying host is, to me, a great bonus security-wise as people don't connect and hack their way into doing things on the hosts they are not supposed to be doing. It's also a lot less worry to patch, update the OS, etc.With ECS you can create capacity providers, which is ECS driving an AutoScaling group so that you don't have to do this. It will provision EC2 instances for you based on the number of services that need to be running.Hope that helps,CommentShareJohnPrestonanswered 3 months ago"


Question: How does EKS decides which node to be removed during node group resizing

"Hi there,For an EKS cluster without auto-scaling enabled, if we scaled down a node group by updating the node group config (decreasing the desired node size). How does EKS decide which node to be removed?If the removing algorithm is just random, is there any way to enforce a specific node to be removed? If we drain and cordon a node to make it unscheduled, will EKS favour this and delete it?Thanks!FollowComment"

Answer: "0Hello, greetings for the day.I see that you are interested in knowing which node (instance) would be taken out (removed) first if you are using nodgroups in EKS cluster.As you might already know that the nodegroups[1] utilize autoscaling groups at the back end of a nodegroup and the selection of the instance that needs to be terminated depends on the autoscaling group termination policies.Suppose you have a nodegroup that has a few instances running and if you decrease the desired size of the nodegroup by 1 then the oldest running instance would be selected to terminate first.  You can verify this yourself by going to the autoscaling group created by EKS for your nodegroups. On the autosclaing group console, under advanced configurations, there is section called as termination policies and here you will see the following three entries:-AllocationStrategy: This comes into picture when there are both spot and on demand instances.  -OldestLaunchTemplate: Which means the instances launched by the older launch template will be terminated first.-OldestInstance: Which means the oldest instance will be terminated first.To answer your question: " If we drain and cordon a node to make it unscheduled, will EKS favour this and delete it?"Answer: No, the instance that gets terminated would depend on the termination policies we have discussed above.Please refer this documentation[2] for more information on what I have shared above.The answer(s) that I have provided are considering the autoscaling group itself and not including the part where the cluster autoscaler[3][4] plays a role.Reference:[1] https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html[2]  https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-termination-policies.html#custom-termination-policy[3] https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md[4] https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscalerCommentShareShabbar_Hanswered 6 months ago"


Question: How to install usbip vhci_hcd drivers on an AWS EC2 Ubuntu Kernel Version

"It took me a while to figure out the problem, but I've been stuck and can't seem to find a solution to this.So basically I want to connect a local USB to a AWS EC2 Instance, and the way to do it on linux is through usbip, but the problem now lies with EC2 Kernel Instances, they dont have the drivers installed by default, and drives need the exact specs to be loaded on the system.Now, I have been trying to manually compile the drivers but I haven't managed to successfully accomplish that, the reason being is that the system does not compile the specific .ko module but at the same time, the other .ko modules installed, are not supported by the kernel.The steps I am following are taken from here on Ubuntu on how to build my kernel: https://wiki.ubuntu.com/Kernel/BuildYourOwnKernelThis is to pull the correct kernel version that is under use from the EC2, the other steps are:ubuntu@:$ sudo cp /boot/config-$(uname -r) .configubuntu@:$ sudo cp /usr/src/linux-headers-$(uname -r)/Module.symvers .ubuntu@:$ sudo make menuconfigubuntu@:$ sudo make modules_prepare && sudo make M=drivers/usb/usbipubuntu@:~$ sudo insmod drivers/usb/usbip/usbip-core.ko && sudo insmod drivers/usb/usbip/vhci-hcd.kobut even though the module compiles, there are no .ko drivers, nor the ones compiled work.Is there a way to make usbip work on AWS on a EC2 instance? Is there another solution to the usbip drivers?FollowComment"

Answer: "0So, I got an answer to this question from someone at AWS Support team. Hope this answer helps someone else in need.You reached out to AWS Support, because you are looking to use usbip client on AWS EC2 instances, but found no required kernel modules (usbip-core, vhci-hcd) available on Ubuntu[18|20] AMIs. You would like our assistance to address this and find the way to have usbip working on AWS EC2 instances.In my test environment in us-east-1 AWS Region, I launched test EC2 instances from the latest AMIs for Amazon Linux 2 (ami-0be2609ba883822ec), and  Ubuntu 20.04 (ami-0885b1f6bd170450c).I found that Amazon Linux 2 has required drivers:uname -r4.14.209-160.339.amzn2.x86_64  modinfo vhci_hcd usbip_core | grep filenamefilename:       /lib/modules/4.14.209-160.339.amzn2.x86_64/kernel/drivers/usb/usbip/vhci-hcd.ko  filename:       /lib/modules/4.14.209-160.339.amzn2.x86_64/kernel/drivers/usb/usbip/usbip-core.ko  But, I didn't find usbip userspace tools after enabled EPEL repository[1], and scanned for package which may have this binary:yum provides */usbip | grep binAnd proceeded with installation of it from a tarball I sourceforge[2]:yum install autoconf libtool libsysfs-devel glib*wget https://downloads.sourceforge.net/project/usbip/usbip/0.1.7/usbip-0.1.7.tar.gztar -xzf usbip-0.1.7.tar.gzcd usbip-0.1.7/src./autogen.sh./configuremake installBy default it's installed in /usr/local/bin which can be added to PATH with profile:echo "export PATH=$PATH:/usr/local/bin" > /etc/profile.d/path.sh && source /etc/profile.d/path.shusbip --helpFor Ubuntu 20.04, I found that the drivers are available in "linux-image-extra-virtual" and I have it installed with 5.4.0-58-generic kernel:uname -r5.4.0-1029-aws  apt updateapt install linux-image-extra-virtualfind /usr/ -iname "vhci*"/usr/lib/modules/5.4.0-58-generic/kernel/drivers/usb/usbip/vhci-hcd.ko  ..  find /usr/ -iname "usbip*"/usr/lib/modules/5.4.0-58-generic/kernel/drivers/usb/usbip  /usr/lib/modules/5.4.0-58-generic/kernel/drivers/usb/usbip/usbip-core.ko  /usr/lib/modules/5.4.0-58-generic/kernel/drivers/usb/usbip/usbip-vudc.ko  /usr/lib/modules/5.4.0-58-generic/kernel/drivers/usb/usbip/usbip-host.ko  ..  Added drivers to load on boot:echo "vhci_hcd" >> /etc/modules-load.d/modules.confecho "usbip_core" >> /etc/modules-load.d/modules.confThen installed usbip:apt install linux-tools-5.4.0-58-genericThen modified grub to boot with 5.4.0-58-generic instead of 5.4.0-1029-aws according to $menuentry_id_option in my /boot/grub/grub.cfggrep -A1000 submenu  /boot/grub/grub.cfg |grep menuentrysubmenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-175c087c-0fe9-4d62-a38d-e5f8a66a5851' {..menuentry 'Ubuntu, with Linux 5.4.0-58-generic' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.0-58-generic-advanced-175c087c-0fe9-4d62-a38d-e5f8a66a5851' {..vim /etc/default/grub#GRUB_DEFAULT=0GRUB_DEFAULT="gnulinux-advanced-175c087c-0fe9-4d62-a38d-e5f8a66a5851>gnulinux-5.4.0-58-generic-advanced-175c087c-0fe9-4d62-a38d-e5f8a66a5851"Then updated grub, rebooted ec2 instance, and verified usbip and loaded modules:update-grub && rebootuname -r5.4.0-58-generic  lsmod | grep vhcivhci_hcd               53248  0  usbip_core             36864  1 vhci_hcd  usbip --helpEdited by: TValley on Dec 31, 2020 12:51 AMCommentShareTValleyanswered 2 years ago"


Question: Problem with accessing EC2 via HTTP

Hi everyone! I am currently facing a problem with EC2. I am unable to access EC2 visa HTTP.Does anyone know why this is happening?FollowCommentoggie 4 months agoI havent been able to access my s3 buckets via console for about 12 hours if it makes any difference... I was able to send an image to a bucket using the cli though.Share

Answer: "0Not enough information to provide a solid answer. But there are a few potential reasons why you may be unable to access your EC2 instance via HTTP:Security Group configuration: Make sure that the security group associated with your EC2 instance has an inbound rule allowing incoming traffic on port 80 (HTTP) from your IP address.Network ACL configuration: Make sure that the Network ACL of the VPC associated with the EC2 instance has an inbound rule allowing incoming traffic on port 80 (HTTP) from your IP address.Firewall on the instance: Check if there is any firewall on the EC2 instance that is blocking incoming traffic on port 80.Application issue: Make sure that the application is running and responding to requests on port 80 on the instance. Try to access the application on the localhost by connecting to the instance using ssh and running curl command.Elastic IP: Make sure that the Elastic IP is associated with the instance.Check the instance's state: Make sure that the EC2 instance is in a running state and that the status checks are passed.By checking the above points, you should be able to identify the cause of the issue and take appropriate actions to resolve it. If none of these steps resolve the issue, it may be helpful to review the EC2's system and instance logs for more information.CommentSharejayamaheizenberganswered 4 months ago"


Question: Instance storage not showing up on ec2 instance (Ubuntu 5.19)

I boot an i3.large instance daily which is supposed to have 475 GB NVMe SSD instance storage. Recently on two occasions the disk did not show up and did not appear in lsblk. Rebooting did not fix it and it was necessary to create a new instance from scratch to access the instance storage. Is there a way to debug or fix this problem permanently?FollowComment

Answer: 0Accepted AnswerThis seems to be a problem with the Amazon firmware described here (https://lore.kernel.org/all/3E46620F-C371-4108-9A74-AA1A624AA9A5@amazon.de/T/)The problem can be confirmed with journalctl -b | grep nvme which will show nvme nvme0: IO queues not created. Under correct operation it should show nvme nvme0: 15/0/0 default/read/poll queues.The problem is triggered by 5.19 kernel and it seems that rolling back the kernel offers a workaround.Kernel downgrade instructionsEdit /etc/default/grub to say GRUB_DEFAULT=saved.Then run sudo update-grub to regenerate the configuration.grep menuentry /boot/grub/grub.cfg to find the menu ids of the old kernel.In my case gnulinux-advanced-cda0c0a7-e64a-4413-85a5-a0235f6f567f>gnulinux-5.15.0-1031-aws-advanced-cda0c0a7-e64a-4413-85a5-a0235f6f567f.Run sudo grub-set-default "gnulinux-advanced-cda0c0a7-e64a-4413-85a5-a0235f6f567f>gnulinux-5.15.0-1031-aws-advanced-cda0c0a7-e64a-4413-85a5-a0235f6f567f"Now reboot and it should use the previous kernel.Confirm the correct kernel has booted by running uname -a. 5.15.0-1031-aws should be visible.CommentShareJamesanswered 3 days ago