Question: What are the permissions required for AWS DMS when using Oracle as the source endpoint?

I have an AWS Database Migration Service (AWS DMS) task that uses Oracle as the source endpoint. What are the minimum permissions required for AWS DMS?

Answer: "I have an AWS Database Migration Service (AWS DMS) task that uses Oracle as the source endpoint. What are the minimum permissions required for AWS DMS?Short descriptionAWS DMS requires permissions to read data from the source database so that it can migrate the data to the target. Depending on your AWS DMS task settings and the type of migration that you're performing, the required permissions vary. See the following minimum required permissions for:Oracle source databases that are AWS managed; that is, Amazon Relational Database Service (Amazon RDS).Oracle source databases that are self-managedOracle source databases that have ongoing replication tasksResolutionOracle source databases that are AWS managed; that is, Amazon RDSFor Oracle databases that are managed by Amazon, for example an Amazon RDS DB instance that runs Oracle, see Working with an Amazon managed Oracle database as a source for AWS DMS.Oracle source databases that are self-managedFor Oracle databases that are self-managed, the following permissions are required:GRANT SELECT on V_$DATABASE to <dms_user>;GRANT SELECT on V_$THREAD to <dms_user>;GRANT SELECT on V_$PARAMETER to <dms_user>;GRANT SELECT on V_$NLS_PARAMETERS to <dms_user>;GRANT SELECT on V_$TIMEZONE_NAMES to <dms_user>;GRANT SELECT on ALL_INDEXES to <dms_user>;GRANT SELECT on ALL_OBJECTS to <dms_user>;GRANT SELECT on ALL_TABLES to <dms_user>;GRANT SELECT on ALL_USERS to <dms_user>;GRANT SELECT on ALL_CATALOG to <dms_user>;GRANT SELECT on ALL_CONSTRAINTS to <dms_user>;GRANT SELECT on ALL_CONS_COLUMNS to <dms_user>;GRANT SELECT on ALL_TAB_COLS to <dms_user>;GRANT SELECT on ALL_IND_COLUMNS to <dms_user>;GRANT SELECT on ALL_LOG_GROUPS to <dms_user>;GRANT SELECT on SYS.DBA_REGISTRY to <dms_user>;GRANT SELECT on SYS.OBJ$ to <dms_user>;GRANT SELECT on DBA_TABLESPACES to <dms_user>;GRANT SELECT on ALL_TAB_PARTITIONS to <dms_user>;GRANT SELECT on ALL_ENCRYPTED_COLUMNS to <dms_user>;GRANT SELECT ANY TRANSACTION to <dms_user>;GRANT SELECT on V_$LOGMNR_LOGS to <dms_user>;GRANT SELECT on V_$LOGMNR_CONTENTS to <dms_user>;GRANT SELECT on V_$LOG to <dms_user>;GRANT SELECT on V_$ARCHIVED_LOG to <dms_user>;GRANT SELECT on V_$LOGFILE to <dms_user>;GRANT SELECT on V_$TRANSACTION to <dms_user>;To expose views, the following permissions are required:GRANT SELECT on ALL_VIEWS to <dms_user>;Note: To expose views, you must also add the following extra connection attribute to the source endpoint: exposeViews=true.To use a pattern that matches the table names in your replication task or to allow the source database user to read tables from a different schema, the following permissions are required:GRANT SELECT ANY TABLE to <dms_user>;To specify a table list in your replication task, the following permissions are required:GRANT SELECT on <schema>.<table > ;Note: Add these permissions for all tables in the table list. To migrate from Oracle RAC, you must grant SELECT permissions on materialized views that have the prefixes gv_$ and v_$.To enable AWS DMS validation for large objects (LOBs), the following permissions are required:GRANT execute on sys.dbms_crypto to <dms_user>;If you use the addSupplementalLogging extra connection attribute, the following permissions are required:GRANT ALTER ANY TABLE to <dms_user>Oracle source databases that have ongoing replication tasksOracle offers two methods for reading the redo logs during ongoing replication, Oracle LogMiner and Oracle Binary Reader. Depending on whether you use LogMiner or Binary Reader, you must grant the source user additional permissions after you grant the required permissions mentioned previously.To use LogMiner, the following permissions are required:GRANT EXECUTE ON dbms_logmnr TO <dms_user>;For Oracle 12c only, the following permissions are required:GRANT LOGMINING TO <dms_user>;To use Binary Reader, the following permissions are required:GRANT SELECT ON v_$transportable_platform  to <dms_user>;Because Binary Reader reads redo and archive logs by using Oracle directories, you must either grant AWS DMS access to create the Oracle directories on the source database:GRANT CREATE ANY DIRECTORY to <dms_user>;Or, if you don't want to allow AWS DMS to create Oracle directories on your source Oracle database, you can precreate the Oracle directories on the redo and archive log paths. Then, grant READ access to the AWS DMS user on those directories:GRANT READ ON DIRECTORY <RedoLog_directory_name> to <dms_user>;GRANT READ ON DIRECTORY <ArchiveLog_directory_name> to <dms_user>;Finally, confirm that the directories are created by querying the ALL_DIRECTORIES table.If the source database uses Oracle ASM, then the following permissions are required:GRANT SYSASM  to <ASM_user>;Note: These permissions are required because reading the redo and archive logs from Oracle ASM uses a function from the Oracle DBMS_DISKGROUP package that requires the SYSASM or SYSDBA privileges.If you use the copyToTempFolder extra connection attribute, then you must create a directory and grant the following permissions:Create directory <bfile_dir> as <copyToTempFolder_PATH>;GRANT READ, WRITE ON DIRECTORY <bfile_dir> TO <dms_user>;For AWS DMS versions 3.1 or later, it's a best practice to add the parallelASMReadThreads extra connection attribute. For more information, see Using Oracle LogMiner or Oracle Binary Reader for Change Data Capture (CDC).Note: AWS DMS supports Oracle Active Data Guard Standby as a source. In Active Data Guard mode, standby databases can be open in read-only mode, so you must grant the previously mentioned permissions on the primary instance. After the permissions are granted on the primary instance, perform a few log switches so that the permissions are replicated to the standby database. For more information, see the Oracle documentation for Forcing log switches.Related informationUsing an Oracle database as a source for AWS DMSFollow"


Question: Why is my Lambda function retrying valid Amazon SQS messages and placing them in my dead-letter queue?

I configured my AWS Lambda function to process messages in an Amazon Simple Queue Service (Amazon SQS) queue. Some of my valid Amazon SQS messages are received multiple times up to the maxReceiveCount and end up in my dead-letter queue.

Answer: "I configured my AWS Lambda function to process messages in an Amazon Simple Queue Service (Amazon SQS) queue. Some of my valid Amazon SQS messages are received multiple times up to the maxReceiveCount and end up in my dead-letter queue.Short descriptionIf your Lambda function is throttled, returns an error, or doesn't respond when reading an Amazon SQS message batch, then the messages return to your queue. After the visibility timeout occurs, your Lambda function receives the message batch again. If your function fails to process valid messages multiple times, then Amazon SQS sends the messages to your dead-letter queue, if you've configured one.To prevent valid messages from being placed in a dead-letter queue, your function code must be idempotent and capable of handling messages multiple times. For more information, see How can I prevent an Amazon SQS message from invoking my Lambda function more than once?ResolutionVerify that your Lambda function's code is idempotentFor idempotency best practices and example function logic, see How do I make my Lambda function idempotent?Verify that your Amazon SQS queue's visibility timeout is at least six times longer than your Lambda function's timeout settingSet your source queue's visibility timeout to at least six times longer than your function's timeout. The extra time allows your function to retry processing a batch if the function is throttled while processing a previous batch.For more information, see Setting the visibility timeout in the Amazon SQS Developer Guide.Note: If your function doesn't receive messages because the queue's visibility timeout isn't long enough, then the messages won't be recorded in your Amazon CloudWatch Logs.Verify that the maxReceiveCount attribute is set to at least five on your source queue's redrive policySet the maxReceiveCount on the source queue's redrive policy to at least five. If your function returns an error, or can't be invoked because it's at maximum concurrency, processing might succeed with additional attempts. A maxReceiveCount of at least five gives your messages more chances to be processed before they're sent to your dead-letter queue.To update the maximum concurrency setting limit, see Configuring maximum concurrency for Amazon SQS event sources.For more information, see How do dead-letter queues work? and avoiding inconsistent message processing in the Amazon SQS Developer Guide.Check the Lambda function for throttling and reserved concurrencyLambda functions are sometimes throttled to protect your resources and downstream applications. Even though Lambda automatically scales to accommodate incoming traffic, your function can still be throttled for various reasons.Use the Lambda console to check the setting for reserved concurrency. If reserved concurrency isn't configured, then the function uses unreserved concurrency. When invocations with functions exceed the unreserved concurrency, throttling occurs.Note: If you configured a function with zero reserved concurrency, then the function is throttled because it can't process any events. Make sure that you increase the value to a number greater than zero.If other functions in the same AWS Region are using the concurrency limit, throttling can occur. Lambda functions experiencing throttling send the SQS messages to the dead-letter queue.Avoid reprocessing all SQS messages in a failed batchAll SQS messages are visible in the dead-letter queue for Lambda functions encountering errors when processing a batch, including messages that Lambda processed successfully. Lambda then retries the entire batch of SQS messages again.To avoid reprocessing all SQS messages in a failed batch, configure your event source mapping with the value ReportBatchItemFailures in the FunctionResponseTypes list. This lets your function return a partial success and reduce the number of retries. For more information, see Reporting batch item failures.Identify and resolve any errors that your Lambda function returnsFollow the instructions in How do I troubleshoot Lambda function failures? Your function automatically removes messages from your queue only if the function doesn't return an error.Related informationHow do I request a concurrency limit increase for my Lambda function?How do I troubleshoot Lambda function throttling with "Rate exceeded" and 429 "TooManyRequestsException" errors?Follow"


Question: Why is my Storage Gateway bill high or increasing?

"I'm using AWS Storage Gateway, and I'm seeing an increase in my billing charges. Why is my Storage Gateway bill high?"

Answer: "I'm using AWS Storage Gateway, and I'm seeing an increase in my billing charges. Why is my Storage Gateway bill high?Short descriptionThere are three main components of Storage Gateway billing:Storage pricing: These are the charges for the storage service that you use with your gateway. The charges can be from using Amazon Simple Storage Service (Amazon S3) or Amazon Elastic Block Store (Amazon EBS).Request pricing: These are the charges for the amount of data that's written to the storage service from your gateway.Data transfer pricing: These are the charges for data transferred out of the storage service to your gateway appliance.If you're using AWS Direct Connect, AWS VPN, or Amazon Elastic Compute Cloud (Amazon EC2) as part of your Storage Gateway setup, then you're charged for the use of those services as well.Tip: Use the AWS Pricing Calculator to get an estimate and review the costs associated with your deployed resources.ResolutionReview the following potential cost drivers for each gateway type:S3 File gatewaysAmazon S3 request charges: Your file gateway uploads files to the S3 bucket, and then it retrieves files from the S3 bucket. This means that the file gateway makes the corresponding S3 requests on your behalf. You're billed for these S3 API request calls according to Amazon S3 pricing.Versioning-enabled buckets: If your file gateway stores files on an S3 bucket with versioning enabled, then your storage costs are higher. This is because every change to a file creates a new version of the object in the S3 bucket.The file gateway performs several incremental uploads to the S3 bucket based on modified data, modified metadata, or multipart uploads of larger files. In a version-enabled bucket, this results in the creation of multiple copies of one object, which increases your storage costs.If you're using a version-enabled bucket and you notice a large amount of data stored in the bucket, consider configuring lifecycle rules that control the number of stored versions of a single object.Before you configure a file gateway with S3 buckets that have versioning or replication enabled, it's a best practice to understand the storage implications for your use case.Amazon FSx File GatewaysStorage types: Amazon FSx file gateways offer storage on hard disk drives (HDD) and solid state drives (SSD). Because SSD storage offers higher performance, it comes at a higher cost. It's a best practice to evaluate your performance needs before deciding which type of storage to use. Note that you can't change the storage type on an existing file system, but you can create a backup and restore it to a different storage type.Volume gatewaysVolume backups: Volume gateways generate and store EBS snapshots of volumes. The Storage Gateway service also integrates with the AWS Backup service to manage the backup and recovery of volumes. Review the following:Your Amazon Data Lifecycle Manager (Amazon DLM) settings for EBS snapshots.Your AWS Backup scheduling and lifecycle rules.Note: You're charged only for the amount of volume data that you store in AWS. You're not charged for the provisioned size of the resource. For example, if you have a provisioned capacity of 100 GB but you use only 10 GB of the storage capacity, then you're charged for the 10 GB of storage.Tape gatewaysTape retrieval and storage class transition requests: When you request data that's stored in virtual tapes, you incur charges based on the amount of data that's being retrieved. Consider the amount of data that's being retrieved from the Amazon S3 Glacier storage class or the S3 Glacier Deep Archive storage class.Additionally, you incur migration charges when you move virtual tapes from the S3 Glacier storage class to the S3 Glacier Deep Archive storage class.Early deletes of virtual tapes: The S3 Glacier and S3 Glacier Deep Archive storage classes have associated charges for early deletes. These charges apply when stored data is deleted before the minimum storage duration. For the S3 Glacier storage class, the minimum duration is three months. For S3 Glacier Deep Archive, it's six months.Note: You're charged only for the amount of virtual tape data that you store in AWS. You're not charged for the provisioned size of the resource. For example, if you have a provisioned capacity of 100 GB, but you use only 10 GB of the storage capacity, then you're charged for the 10 GB of storage.Additional cost considerationsIf you use Amazon Virtual Private Cloud (VPC) to open a secure connection to your gateway, then you incur additional charges. AWS PrivateLink uses Amazon VPC endpoints to activate private connectivity, but you are charged for every hour of use. For more information, see AWS PrivateLink pricing.Follow"


Question: How do I troubleshoot and fix failing health checks for Classic Load Balancers?

"One or more instances behind my Classic Load Balancer are failing health checks. What are some potential causes for this, and how can I fix it?"

Answer: "One or more instances behind my Classic Load Balancer are failing health checks. What are some potential causes for this, and how can I fix it?Short descriptionElastic Load Balancing (ELB) supports Application Load Balancers, Network Load Balancers, and Classic Load Balancers. Instances running on Classic Load Balancers might fail health checks for the following reasons:Connectivity issues between the load balancer and backendConfiguration issues with the applicationNot receiving a "200 OK" response from the backend for the health check requestProblems with SSL, which are causing HTTPS or SSL health checks to failResolutionTo troubleshoot and resolve health check related issues, check the following:Resolve potential connectivity issues between your load balancer and your backendBe sure that the following are true:The backend allows health check traffic. To check your security group rules, see Recommended rules for load balancer security groups. For the backend instance, see Security groups for instances in a VPC.The network access control list (ACL) for the subnet with the load balancer or backend allows health check traffic. To check the ACL rules for the subnet with the load balancer and backend, see Network ACLs for load balancers in a VPC.Instance-level or OS-level firewalls on the backend allow health check traffic. Be sure that the instance or OS level firewalls on the backend are allowing the health check traffic in and out.The resources of the backend instance are not over-utilized. Be sure that memory and CPU utilization are within acceptable limits. If your memory or CPU utilization is too high, add more backends or upgrade the backends to a larger instance type.Any additional resources are allowing health check traffic. The health check request might be forwarded to an additional resource by the backend. If the additional dependency fails to respond or takes too long to respond, the health check times out and fails. Be sure that the additional resource is responding to the health check requests.Resolve potential configuration issues with your applicationThe application is running. For example, if you are running Apache on a Linux instance, use the sudo service httpd status command to confirm that Apache is running.The application is listening on the health check port. To confirm that the server is in a listening state for the health check port, run the netstat -ant command on the server. Check the application configuration port to verify that it is running. To simulate a TCP connection between the load balancer and backend, run the **telnet <backend private IP> <health-check port>**command on the instance. If the output is "Connected," then the backend is listening on the health check port.The application is set to respond to requests with a custom host header. For HTTP and HTTPS health checks, Classic Load Balancers set the header to the private IP address with the primary network interface of the backend, and the user agent to "ELB-HealthChecker/1.0". If the backend only responds to requests with a custom host header or a user agent, then the health check request fails.The application is listening on the primary network interface of the backend. Classic Load Balancers connect to the primary network interface of the backend instance. Be sure that the application is listening on the primary network interface of the backend instance, allowing the backend to send a response for the health check requests.Confirm a "200 OK" response from the backend to a health check requestBe sure that the following are true:The ping path is valid. If the file specified in the ping path is not configured on the backend, the backend might respond with a "404 Not Found" response code, and the health check will fail. Note: The default value for the ping path is /index.html. If you do not have an index.html file on the backend, create one, or change the value of the ping path to a file name on the backend.Redirection is not configured on the backend. Redirection configured on the backend can result in a 301 or 302 response code, resulting in failed health checks. For example, if you have a redirection from HTTP:80 to HTTPS:443 on the backend, then HTTP health checks on port 80 will fail, unless you change the health check to HTTPS and health check port to 443. Note: You can simulate the health check request sent by the load balancer using the curl -Ivk http[s]://<private-IP-address-of-the-backend-instance>:<health-check-port>/<health-check-target-page> command from an instance in the subnet that is associated with the load balancer.Resolve any SSL-related issuesBe sure that the following are true:All protocols and ciphers match. For HTTPS or SSL health checks, the load balancer and the backend should use the same protocol and cipher. Packet captures taken on the backend instance show the ciphers and protocol supported on the load balancer in the client hello sent by the load balancer. The registered backend instance should support one or more matching ciphers and a protocol sent by the load balancer.Server Name Indication (SNI) is not enabled on the backend server. If the health check is HTTPS/SSL and SNI is enabled on the backend instance, the load balancer and backend can't establish the SSL handshake. This is because the backend server sends an RST after the client hello. Disable SNI, or use TCP health checks instead.Related informationRegistered instances for your Classic Load BalancerFollow"


Question: How do I create an AWS Storage Gateway S3 file gateway using Amazon EC2 as the host platform?

I want to create an AWS Storage Gateway to store files on Amazon Simple Storage Service (Amazon S3). How do I create this file gateway using Amazon Elastic Compute Cloud (Amazon EC2) as the host platform?

Answer: "I want to create an AWS Storage Gateway to store files on Amazon Simple Storage Service (Amazon S3). How do I create this file gateway using Amazon Elastic Compute Cloud (Amazon EC2) as the host platform?ResolutionNote: This resolution explains how to create a file gateway with Amazon EC2, but the steps are similar for an on-premises deployment. For more information, see Create and activate an Amazon S3 File Gateway.Open the Storage Gateway console, then choose Create gateway.For Gateway settings, set a gateway name and set a time zone. Then, choose Next.For Gateway options, choose Amazon S3 File Gateway. Then, choose Next.For Platform options, choose Amazon EC2. Then, choose Launch instance. You are redirected to the EC2 console.In the Amazon EC2 console, configure the EC2 instance using the Launch an Instance wizard.Proceed with the steps to connect to the file gateway, activate the file gateway, and then configure local disks.----%3E).Create the file share.Related informationFile Gateway setup requirementsFollow"


Question: How do I transfer the ownership of an Amazon S3 bucket to a different AWS account?

I want to transfer the ownership of Amazon Simple Storage Service (Amazon S3) bucket from one AWS account to a different account.

Answer: "I want to transfer the ownership of Amazon Simple Storage Service (Amazon S3) bucket from one AWS account to a different account.ResolutionYou can't transfer Amazon S3 bucket ownership between AWS accounts because the bucket is always owned by the account that created it. Instead, you can copy Amazon S3 objects from one bucket to another so that you give ownership of the copied objects to the destination account. For instructions, see How can I copy S3 objects from another AWS account?Warning: AWS Support can't transfer bucket ownership or copy Amazon S3 objects or manipulate any configuration options in AWS accounts. Also, you can't separate an AWS account from an Amazon.com account.Follow"


Question: How can I reduce unexpected high costs for Amazon S3?

"The charges for Amazon Simple Storage Service (Amazon S3) usage on my AWS bill are higher than expected, and I want to reduce them."

Answer: "The charges for Amazon Simple Storage Service (Amazon S3) usage on my AWS bill are higher than expected, and I want to reduce them.Short descriptionAmazon S3 charges are based on storage, requests and data retrievals, data transfer (including S3 Transfer Acceleration usage), and data management.To identify where you can reduce your Amazon S3 charges, review your current billing and usage for Amazon S3. Then, review how you incur costs.The following tips can help you reduce your Amazon S3 billing costs:Clean up incomplete multipart uploads.Delete previous versions of objects that you don't need.Review your storage-class transition costs.Review your data retrieval costs.Track the requests to your bucket.Check for changes in the size of your bucket.Review the cost of individual buckets.Understand how your usage relates to your charges.You can also use the following Amazon S3 features to help you investigate billing concerns:Amazon S3 Storage Lens to obtain a single view of usage and activity across your Amazon S3 storage.Amazon S3 storage class analysis to observe data access patterns.ResolutionClean up incomplete multipart uploadsAfter you initiate a multipart upload, Amazon S3 retains all the uploaded parts of the file until you complete or shut down the multipart upload. If you don't complete the multipart upload, then Amazon S3 continues to store the uploaded parts. As a result, you're charged for the storage of uploaded parts.If you use the AWS Command Line Interface (AWS CLI) to shut down the multipart upload, then the operation deletes the incomplete uploaded parts. However, if you use other tools that use the Amazon S3 multipart upload API, then incomplete multipart uploads might leave the uploaded parts.Here are some additional best practices for when you clean up incomplete multipart uploads:Create a lifecycle policy to clean up incomplete multipart uploads after a certain number of days.Use Amazon S3 API calls to list your multipart uploads. Then, shut down multipart uploads that failed or that you want to cancel.Use S3 Storage Lens to determine the number and size of incomplete multipart uploads for buckets in your account.Amazon S3 Storage Lens to identify which S3 buckets contain multipart uploads.Delete previous versions of objects that you don't needImportant: Amazon S3 can't restore any data that you delete from your buckets.If you activated versioning on your bucket, then each object can have multiple versions. Each object version contributes to your storage costs.To review your object versions, use the Amazon S3 console or run the ListObjectVersions API call. Then, delete any object versions that you don't need.Note: In a version-activated bucket, you must specify the object version ID to permanently delete an object version. If you run a simple delete request on an object in a version-activated bucket, then Amazon S3 adds a delete marker to the object. The delete marker becomes the current version of the object, and the actual object becomes the previous version. However, this doesn't erase the object and its versions.You can also use the following methods:A lifecycle rule to manage the current and non-current versions of your objects. However, because you can't reverse permanent deletions in Amazon S3, review the parameters for your lifecycle rule.S3 Storage Lens to determine the number of current versions, previous versions, and delete markers for specific buckets or all buckets in your account.Review your storage-class transition costsYou might incur costs when you transition objects from one storage class to another.For example, when objects transition from S3 Standard to the S3 Standard-IA storage class, you're charged $0.01 for every 1,000 lifecycle transition requests. When objects transition from any storage class to the S3 Glacier Flexible Retrieval storage class, you're charged $0.03 for every 1,000 lifecycle transition requests. For more information and pricing details, see Amazon S3 pricing.If you set up a lifecycle configuration rule that transitions an entire bucket's objects to another storage class, then expect an increase in Amazon S3 charges. To determine if an increase in your charges is because of a lifecycle rule, review your AWS CloudTrail event history. You can also review your Amazon S3 server access logs for PutBucketLifecycleConfiguration calls to identify when the rule was applied.You can also use S3 Storage Class Analysis to determine how much data is retrieved from the different storage classes.Note: You must configure S3 storage class analysis on a per bucket basis.Review your data retrieval costsObjects that are stored in the following storage classes have costs that are associated with data retrievals:S3 Standard-IAS3 One Zone-IAS3 Glacier Flexible RetrievalS3 Glacier Deep Archive storage classesFor example, you retrieve 15 million objects from the S3 Glacier Flexible Retrieval storage class. The total cost of retrieval is based on the bulk retrieval option that you choose.The following example costs are for a standard retrieval:The retrieval request is 15,000,000 / 1000 * $0.05 = $750.The retrieval is 100,000 * $0.01 = $1,000.The total cost is $1750.The following example costs are for an expedited retrieval:The retrieval request is 15,000,000 / 1000 * $10 = $150,000.The retrieval is 100,000 * $0.03 = $3,000.The total cost is $153,000.For objects that you store in S3 Glacier Flexible Retrieval, there's no charge for bulk retrievals.Tip: To help you monitor the different storage requests and data usage, you can also use the S3 Storage Lens dashboard.Track the requests to your bucketYou can monitor requests to your bucket in the following ways:Server access logging. For more information on how to review access logs, see Amazon S3 server access log format.Object-level logging through AWS CloudTrail.Amazon CloudWatch metrics, such as AllRequests and BytesDownloaded.To reduce you costs from requests, you can also use bucket policies or AWS Identity and Access Management (IAM) policies. These policies prevent unauthorized access or limit public access to your buckets.To reduce your costs from data transfer OUT charges, you can use an Amazon Elastic Compute Cloud (Amazon EC2) instance to access your bucket. The Amazon EC2 instance must be in an Amazon Virtual Private Cloud (Amazon VPC) within the same AWS Region as your bucket.Check for changes in the size of your bucketTo isolate storage costs, compare the current size of your bucket with the size of your bucket from a previous month.1.    Open the Amazon S3 console.2.    Choose Buckets, and then select your bucket.3.    Choose the Management tab.4.    Choose Metrics.You can also use the AWS CLI to review your bucket's storage size, or you can monitor your storage metrics with Amazon CloudWatch. To monitor bucket sizing changes on a daily, weekly, and monthly basis, use S3 Storage Lens.Review the cost of individual bucketsTo identify which bucket incurs higher costs, activate the cost allocation tags on your buckets. Then, review your buckets with the AWS Cost Explorer. For more information, see How do I find the cost of my Amazon S3 buckets?Understand how your usage relates to your chargesYour usage report records storage usage in byte hours, and your billing report records storage usage in GB months. To understand how your storage usage relates to your storage charges, convert byte hours into GB months. For an example of this conversion, see Understanding your AWS billing and usage reports for Amazon S3.You can download your Amazon S3 usage report from the AWS Billing and Cost Management dashboard. Or, to obtain detailed storage and billing metrics, create an S3 Storage Lens dashboard for all the buckets in your account.Follow"


Question: How do I use the S3 Storage Lens dashboard to view prefix-level metrics for my Amazon S3 bucket?

I want to use Amazon S3 Storage Lens dashboards to view prefix-level daily usage metrics for my Amazon Simple Storage Service (Amazon S3) buckets.

Answer: "I want to use Amazon S3 Storage Lens dashboards to view prefix-level daily usage metrics for my Amazon Simple Storage Service (Amazon S3) buckets.ResolutionTo view the daily usage metrics at the prefix level, activate the Amazon S3 advanced metrics and recommendations feature on the S3 Storage Lens dashboard. The dashboard includes the following metrics:Storage metrics, such as total storage and object countCost optimization metrics, such as noncurrent version object count and delete marker object countTo view prefix-level metrics, activate the Prefix aggregation option.Note: S3 Storage Lens doesn't publish prefix-level activity metrics. Create an Amazon S3 Storage Lens dashboard with Advanced metrics and Prefix aggregation options activatedOpen the Amazon S3 console.In the navigation pane, under S3 Storage Lens, choose Dashboards.Choose Create dashboard.On the Dashboard page, in the General section, enter a name for the dashboard.Under Metrics selection, choose Advanced metrics and recommendations. Under Advanced metrics and recommendations features, choose Prefix aggregation.  Under Prefix aggregation, configure the following:Choose the minimum prefix threshold size for this dashboard. For example, a prefix threshold of five percent indicates that prefixes that make up five percent or more of the bucket's total storage size are aggregated. The minimum prefix threshold size is one percent. Choose the prefix depth. This setting indicates the maximum number of levels that the prefixes are evaluated at. The prefix depth must be less than 10.Enter a prefix delimiter character to identify each prefix level. The default value in Amazon S3 is the / character, but your storage structure might use other delimiter characters.Choose Create dashboard.After you create the dashboard, it takes up to 48 hours to start publishing the metrics.View the prefix-level metrics on the dashboardIn the Dashboard list, select the dashboard that you created in step 1.Choose the Prefix tab. By default, the dashboard shows metrics for top 10 prefixes. You can increase this number up to 50.Scroll to the Prefixes section to view usage metrics for the Top N prefixes. From the Metrics categories dropdown list, choose Summary and Cost optimization.You can determine the size of listed prefixes from the Total storage and Object count metrics.To view the prefix-level metrics for a particular S3 bucket or prefix, complete the following steps:On the dashboard page in Storage Lens, choose Filters. By default, Accounts, AWS Regions, Storage classes, Buckets, and Prefixes are selected for your dashboard.To filter to a specific S3 bucket or prefix, clear Select all. Then from the dropdown list, choose the options to filter.Choose Apply.Note: To see data for a specific metric, choose the metric from the Metric dropdown list in any section in the S3 Storage Lens dashboard.Related informationCreating and updating Amazon S3 Storage Lens dashboardsFollow"


Question: How can I download binary logs from an Amazon RDS MySQL DB instance?

How can I download binary logs from an Amazon Relational Database Service (Amazon RDS) MySQL DB instance?

Answer: "How can I download binary logs from an Amazon Relational Database Service (Amazon RDS) MySQL DB instance?Resolution1.    Install the mysqlbinlog utility on your client machine, if you haven't already:sudo yum install -y mysql2.    Connect to your RDS DB instance from the MySQL client.3.    Run the show binary logs command on the RDS DB instance, and review the output. You receive an output similar to the following that includes a list of the binary logs that are present on the RDS DB instance:mysql> show binary logs;+----------------------------+-----------+| Log_name                   | File_size |+----------------------------+-----------+| mysql-bin-changelog.068462 |       729 || mysql-bin-changelog.068463 |       578 || mysql-bin-changelog.068464 |       578 || mysql-bin-changelog.068465 |       154 |+----------------------------+-----------+4 rows in set (0.00 sec)3.    Access the MySQL binary logs to download or stream the logs from your RDS DB instance.Related InformationMySQL Database Log FilesHow do I download my Amazon RDS DB instance logs?Follow"


Question: How do I use Amazon RDS to turn on query logging for PostgreSQL?

I want to turn on query logging for my Amazon Relational Database Service (Amazon RDS) for PostgreSQL DB instances.

Answer: "I want to turn on query logging for my Amazon Relational Database Service (Amazon RDS) for PostgreSQL DB instances.Short descriptionTo turn on query logging on PostgreSQL, modify the following parameters that are associated with the DB instance for a customized parameter group:log_statement (on the PostgreSQL website)log_min_duration_statement (on the PostgreSQL website)When you modify log parameters, the DB instance's volume might require more space. If the volume's storage is full, then the DB instance is unavailable. It's a best practice to modify the rds.log_retention_period parameter to clean up unnecessary logs. It's also a best practice to use the Amazon CloudWatch metric FreeStorageSpace to continually monitor your storage consumption. When you see that the storage space is almost full, you can increase the storage space so that the DB instance is always available.ResolutionDepending on the queries you want to log, turn on log_statement or log_min_duration_statement. You don't need to modify both parameters to turn on logging.Modify log_min_duration_statement to set a threshold in milliseconds. This allows you to log all queries that take longer than the set parameter value. For example, suppose you set the log_min_duration_statement value to 500. When this happens, Amazon RDS logs all queries that take longer than half of a second to complete. If you set the value for this parameter to 2000, then Amazon RDS logs all queries that take longer than two seconds to complete. If you set the parameter value to -1, then the parameter is turned off. Amazon RDS doesn't log any queries based on the time to complete. If you set the parameter value to 0, Amazon RDS logs all queries.Note: The log_min_duration_statement parameter doesn't depend on or interfere with the log_statement parameter.Modify the log_statement to control the SQL statements that are logged. The default value is none. You can modify the following values for this parameter:ddl logs all data definition language (DDL) statements such as CREATE, ALTER, and DROP.mod logs all DDL and data modification language (DML) statements such as INSERT, UPDATE, and DELETE.all logs all queries (regardless of the execution time).Note: Regardless of the value that you set for log_statement and log_min_duration_statement, the queries are written to the log once.Turn on query logging on PostgreSQLComplete the following steps to turn on query logging on PostgreSQL:Note: The following example parameter modifications logs all queries that take longer than one second and also logs all schema changes.Open the Amazon RDS console, and choose Databases from the navigation pane.Choose the DB instance that you want to log queries for.Choose the Configuration view to see the details of the DB instance and to see the parameter group associated with the DB instance.Note: When you create a DB instance, the DB instance is associated with the default DB parameter group. Because you can't modify this group, create a custom parameter group to modify the parameter group. When you change the DB instance parameter group (from default to a custom group), you must reboot the DB instance.Choose Parameter group from the Instance configuration pane, and then choose the parameter group that you want to modify.Choose Edit Parameter.In the Filter parameters field, select the parameter that you want to change. For example:Enter log_statement and change the value to ddl.Enter log_min_duration_statement and change the value to 1000. (This value is in milliseconds and equals one second).Choose Save changes.Note: These parameters are dynamic. If your DB instance has a custom parameter group, then you don't need to reboot the DB instance for these parameters to take effect.Confirm that logging is turned onAfter you save the parameter group associated with your DB instance, the status is set to applying. After the parameter group is set in your DB instance, the status is set to in-sync. To confirm that logging is turned on, connect to the DB instance. In this example, confirm that the log includes all DDL statements and all queries that take longer than one second. Then, run commands similar to the following:CREATE TABLE article(article_code bigint, created_at timestamp with time zone, summary text, content text) ;SELECT pg_sleep(2);SELECT generate_series(1,10000000) as test;View query logsComplete the following steps to view the logs:1.    Open the Amazon RDS console.2.    In the navigation pane, choose Databases.3.    Choose your DB instance, and then choose the Logs view.4.    In the Logs & Events tab, choose the most recent log, and then choose View log to see the content of logs. For example:2018-12-19 11:05:32 UTC:172.31.xx.yyy(35708):user@awspostgres:[27174]:LOG: statement: CREATE TABLE article(article_code bigint, created_at timestamp with time zone, summary text, content text) ;2018-12-19 11:10:36 UTC:172.31.xx.yyy(35708):user@awspostgres:[27174]:LOG: duration: 2010.247 ms statement: select pg_sleep(2);2018-12-19 11:11:25 UTC:172.31.xx.yyy(35708):user@awspostgres:[27174]:LOG: duration: 2159.838 ms statement: SELECT generate_series(1,10000000) as test;Note: Make sure that you don't set the previous parameters at values that generate extensive logging. For example, setting log_statement to all or setting log_min_duration_statement to 0 generates a large amount of logging information. This affects your storage consumption. To set the parameters to these values, make sure you do so only for a short time for troubleshooting purposes. Closely monitor the storage space throughout when you're troubleshooting.Related informationRDS for PostgreSQL database log filesFollow"


Question: How can I block emails from specific domains or email addresses in Amazon SES?

I don't want my Amazon Simple Email Service (Amazon SES) identity to receive emails from a certain domain or email address. How can I block those emails?

Answer: "I don't want my Amazon Simple Email Service (Amazon SES) identity to receive emails from a certain domain or email address. How can I block those emails?Short descriptionTo prevent certain domains or email addresses from sending emails to your Amazon SES identity, follow these steps:1.    Create an AWS Lambda function to process incoming emails, evaluate the sender, and then drop the message if the sender matches a specific domain or email address.2.    Create an Amazon SES receipt rule that routes all incoming email to the Lambda function.3.    Check the function's Amazon CloudWatch Logs to verify the emails being blocked.Note: You incur Amazon SES and Lambda charges when you add the Lambda action on the Amazon SES receipt rule.ResolutionCreate an AWS Lambda function1.    Open the AWS Lambda console.2.    In the Region selector on the navigation bar, select the AWS Region that you're using for Amazon SES. The Lambda function must be in the same Region that you're using with Amazon SES. For more information, see Invoke Lambda function action.3.    Choose Create function.4.    Select Author from scratch.5.    For Function Name, enter a name for your function. For example, you can enter "SESReceiptRule".6.    For Runtime, select Node.js 10.x.7.    Under Permissions, expand Choose or create an execution role.8.    For Execution Role, select Create a new role with basic Lambda permissions.9.    Choose Create function.10.    Under Function code, enter the following code:// Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.// SPDX-License-Identifier: MIT-0'use strict';const AWS = require('aws-sdk');exports.handler = (event, context, callback) => {    console.log('Blocking email filter starting');    const sesNotification = event.Records[0].ses;    const messageId = sesNotification.mail.messageId;    const receipt = sesNotification.receipt;    const mail = sesNotification.mail;       // Convert the environment variable into array. Clean spaces from it.    var blockingListString = process.env.blockingList;    blockingListString = blockingListString.replace(/\s/g,'');      var blockingListArray = blockingListString.split(",");    // Check if the mail source matches with any of the email addresses or domains defined in the environment variable    function isListed() {        var length = blockingListArray.length;        for(var i = 0; i < length; i++) {            if (mail.source.endsWith(blockingListArray[i]))                return true;        }        return false;    }    console.log('Processing message:', messageId);        // Processing the message    if (isListed()) {            callback(null, {'disposition':'STOP_RULE_SET'});            console.log('Rejecting messageId: ', messageId, ' - Source: ', mail.source, ' - Recipients: ',receipt.recipients,' - Subject: ', mail.commonHeaders['subject']);    }    else {        console.log('Accepting messageId:', messageId, ' - Source: ', mail.source, ' - Recipients: ',receipt.recipients,' - Subject: ', mail.commonHeaders['subject']);        callback();    }};11.    Under Environment variables, for Key, enter "blockingList".Note: There's no limit to the number of environment variables that you can create as long as the total size of the set doesn't exceed 4 KB. For more information, see Using AWS Lambda environment variables.12.    For the Value of "blockingList", enter a comma-separated list of the email addresses and domains that you want to block. For example, enter "example.com, JohnDoe@example.com".Note: You can edit the list of email addresses and domains without changing the Lambda function code.13.    Choose Save.Create an Amazon SES receipt rule1.    Open the Amazon SES console.2.    In the navigation pane, under Email Receiving, choose Rule Sets.3.    If you want to add the rule to an existing active rule set, skip to step 4. To create a new rule set, choose Create a Rule Set, enter a rule set name, and then choose Create a Rule Set.Note: If you create a new rule set, be sure to select the rule set, and then choose Set as Active Rule Set.4.    Choose View Active Rule Set.5.    Choose Create Rule.Note: You can also choose to update an existing rule with the same values described in the following steps.6.    For Recipients, enter the email addresses or domains associated with your Amazon SES identity.Important: Enter the email address or domain that you want to prevent from receiving emails from specific senders. Don't enter the email address or domain that you want to block emails from. For example, if your Amazon SES identity uses the email address "JaneRoe@example.net" and you want to block emails from "example.com", enter "JaneRoe@example.net".7.    Choose Next Step.8.    Under Actions, for Add action, select Lambda. Then, complete these fields:For Lambda function, select the function that you created.For Invocation type, select RequestResponse.(Optional) Configure SNS topic as required by your use case. Or, you can leave this field blank. For more information on this field, see Lambda Action.9.    Choose Next Step.10.    Under Rule Details, complete these fields:For Rule name, enter a name for your rule.For Enabled, be sure that this checkbox is selected so that the rule is created as an active rule.(Optional) Select Require TLS or Enable spam and virus scanning if required by your use case.For Rule set, verify that your active rule set is selected. An active rule set includes (Active) in the name.For Insert after rule, select the position that you want for your rule within the rule set.11.    Choose Next Step.12.    Choose Create Rule.Note: If you get a Missing Permissions dialog box with the message "SES was unable to access the resource [your Lambda function]," then Amazon SES needs permissions to the function. You can choose Add permissions to configure the required permissions.Check the function's Amazon CloudWatch LogsAfter you set up the Lambda function and Amazon SES rule, follow these steps to verify the emails being blocked:1.    Open the CloudWatch console.2.    From the navigation pane, choose Logs.3.    From the list of log groups, choose your Lambda function's log group. For example, choose /aws/lambda/name_of_your_function.4.    Choose the log stream that you want to check. The log stream displays the email messages and domains being processed by the Lambda function, similar to the following:14:08:25 START RequestId: aa939984-1b9b-11e7-83d2-efc6877bdc9b Version: $LATEST14:08:25 2017-04-07T14:08:25.957Z aa939984-1b9b-11e7-83d2-efc6877bdc9b Blocking email filter starting14:08:25 2017-04-07T14:08:25.958Z aa939984-1b9b-11e7-83d2-efc6877bdc9b Processing message: jc0iurgrtkrsrs7f5pk0rsmf4r3q0poikdjfdi0114:08:25 2017-04-07T14:08:25.959Z aa939984-1b9b-11e7-83d2-efc6877bdc9b Rejecting messageId: jc0iurgrtkrsrs7f5pk0rsmf4r3q0poikdjfdi01 - Source:user@example.com - Recipients: [ 'user@domain.com' ] - Subject: This is an unwanted message14:08:25 END RequestId: aa939984-1b9b-11e7-83d2-efc6877bdc9bRelated informationGetting started with LambdaRecipient-based control using receipt rulesFollow"


Question: How can I use Systems Manager automation to enforce that only IMDSv2 is used to access instance metadata from my Amazon EC2 instance?

I want to use Instance Metadata Service Version 2 (IMDSv2) to access instance metadata from my Amazon Elastic Compute Cloud (Amazon EC2) instance. How do I enforce IMDSv2 on my Amazon EC2 instance?

Answer: "I want to use Instance Metadata Service Version 2 (IMDSv2) to access instance metadata from my Amazon Elastic Compute Cloud (Amazon EC2) instance. How do I enforce IMDSv2 on my Amazon EC2 instance?Short descriptionBy default, you can retrieve instance metadata from a running Amazon EC2 instance using either or both of the following methods:Instance Metadata Service Version 1 (IMDSv1) – a request/response methodInstance Metadata Service Version 2 (IMDSv2) – a session-oriented methodTo require the use of IMDSv2 on an instance, you can run the AWS Systems Manager AWSSupport-ConfigureEC2Metadata Automation document.Important: If you enforce IMDSv2, then IMDSv1 no longer works, and applications that use IMDSv1 might not function correctly. Before enforcing IMDSv2, verify that any applications that use Amazon EC2 metadata are upgraded to a version that supports IMDSv2. For more information about instance metadata, see Configure the instance metadata service.ResolutionPrerequisitesTo run the automation and read the output, you must have ssm:StartAutomationExecution and ssm:GetAutomationExecution permissions.Run the AWSSupport-ConfigureEC2Metadata Automation documentOpen the Systems Manager console, and then choose Automation from the navigation pane.Choose Execute automation.On the Owned by Amazon tab, for Automation document, enter AWSSupport-ConfigureEC2Metadata, and then press Enter.Select the radio button for the AWSSupport-ConfigureEC2Metadata document, and then choose Next.For Execute automation document, choose Simple execution.Note: If you need to run the automation on multiple targets, then choose Rate Control.For Input parameters, specify the following parameters:InstanceId: Enter the ID of the Amazon EC2 instance that you want to configure metadata settings for.HttpPutResponseHopLimit: Leave as 0 to retain the current value, or enter your desired value (1-64).EnforceIMDSv2: Choose required.MetadataAccess Choose enabled.(Optional) For AutomationAssumeRole, choose a role. If a role isn't specified, then Systems Manager Automation uses the permissions of the user that runs the document.Note: To change the target Amazon EC2 instance, the AutomationAssumeRole or user role must have ec2:ModifyInstanceMetadataOptions and ec2:DescribeInstances permissions. For more information about creating the assume role for Systems Manager Automation, see Create a service role for Automation.Choose Execute.You can also use the AWSSupport-ConfigureEC2Metadata document to change other Amazon EC2 instance metadata. For example, you can turn off metadata or change the value of the HttpPutResponseHopLimit  attribute. For more information, see AWSSupport-ConfigureEC2Metadata.Related informationRetrieve instance metadataFollow"


Question: How can I duplicate or clone my RDS for SQL Server user login permissions?

I want to clone my Amazon Relation Database (Amazon RDS) for Microsoft SQL Server user login permissions. How can I do this?

Answer: "I want to clone my Amazon Relation Database (Amazon RDS) for Microsoft SQL Server user login permissions. How can I do this?ResolutionNote: This script doesn't make any changes to new logins unless the user running the script has access to grant those permissions. If the user doesn't have access to grant permissions, then the permissions don't appear in the script. This is because the login used to access the script doesn't have view privileges. In addition, if you try to add the permissions script manually when you don't have grant privileges, the script fails.1.    Copy and run the MSSQL_RDS_Clone_Login script to create a stored procedure in the environment where you want to duplicate the user. You use this stored procedure later to create a duplicate of the user login, database user, server level, and database level permissions.You can create the stored procedure in any user database that's not within the Amazon RDS for SQL Server system database. Replace [DB_NAME] with the name of the user database, according to your use case.The following script creates three stored procedures:DuplicateLogin: Duplicates the login and duplicate database user for those logins in each database containing the login permissions that you're copying from.GrantUserRoleMembership: Duplicates DB user permissions and roles to the new user.DuplicateRDS: Consolidates the results of both the DuplicateLogin and GrantUserRoleMembership stored procedures.2.    After creating the stored procedure, open a new TSQL window and run the stored procedure using the following format.Note: Before running the script, make sure that the results are in text format by pressing CTRL+T on your keyboard.--SQL loginEXEC    DuplicateRDS @NewLogin=[<duplicate_login_name>]         ,@NewLoginPwd = password    ,@LoginToDuplicate = master_login    , @WindowsLogin  = F    ,@DatabaseName=NULL-- Windows loginEXEC    DuplicateRDS @NewLogin=[<domain\duplicate_login_name>]    ,@LoginToDuplicate = master_login    ,@NewLoginPwd = NULL    , @WindowsLogin  = T    ,@DatabaseName=NULL3.    After the script generates, copy the script from the Results tab and run it in a new query window. After the script runs, your SQL Server login generates with similar server and database level permissions as your master login.Note: This example excludes SSISDB permissions for ssis_admin and ssis_logreader. If you require these permissions, provide them separately.ALTER ROLE [ssis_admin] ADD MEMBER [mydomain\user_name]ALTER ROLE [ssis_logreader] ADD MEMBER [mydomain\user_name]GONote: The script out might occur in an unorderly fashion due to temp tables being used in the stored procedure. If this occurs, drop and then recreate the stored procedure to get it into the correct format.4.    If you have orphan users, run the following script. This script checks for orphan users in databases that the script failed removes those orphan users.The following script checks for orphan users:Use [DB_NAME] ; GO exec sp_change_users_login @Action='Report' ;GOThe following script drops orphan users:Use [DB_NAME] ;GOexec sp_revokedbaccess 'username'GO5.    If you don't want to keep the stored procedures after duplicating the login (for example, due to compliance), then run the following script:USE [DB_NAME]GODROP PROCEDURE [dbo].[DuplicateRDS]GODROP PROCEDURE [dbo].[DuplicateLogin]GODROP PROCEDURE [dbo].[GrantUserRoleMembership]GOFollow"


Question: How can I retrieve Amazon EKS control plane logs from CloudWatch Logs?

I'm troubleshooting an Amazon Elastic Kubernetes Service (Amazon EKS) issue and I need to collect logs from the components that run on the EKS control plane.

Answer: "I'm troubleshooting an Amazon Elastic Kubernetes Service (Amazon EKS) issue and I need to collect logs from the components that run on the EKS control plane.Short descriptionTo view the logs in Amazon CloudWatch Logs, you must turn on Amazon EKS control plane logging. You can find EKS control plane logs in the /aws/eks/cluster-name/cluster log group. For more information, see Viewing cluster control plane logs.Note: Replace cluster-name with your cluster's name.You can use CloudWatch Logs Insights to search through the EKS control plane log data. For more information, see Analyzing log data with CloudWatch Insights.Important: You can view log events in CloudWatch Logs only after you turn on control plane logging in a cluster. Before you select a time range to run queries in CloudWatch Logs Insights, verify that you turned on control plane logging.ResolutionSearch CloudWatch InsightsOpen the CloudWatch console.In the navigation pane, choose Logs, and then choose Log Insights.In the Select log group(s) menu, select the cluster log group that you want to query.Choose Run to view the results.Note: To export the results as a .csv file or to copy the results to the clipboard, choose Export results. You can change the sample query to get data for a specific use case. See these example queries for common EKS use cases.Sample queries for common EKS use casesTo find the cluster creator, search for the IAM entity that's mapped to the kubernetes-admin user.Query:fields @logStream, @timestamp, @message| sort @timestamp desc| filter @logStream like /authenticator/| filter @message like "username=kubernetes-admin"| limit 50Example output:@logStream, @timestamp @messageauthenticator-71976 ca11bea5d3083393f7d32dab75b,2021-08-11-10:09:49.020,"time=""2021-08-11T10:09:43Z"" level=info msg=""access granted"" arn=""arn:aws:iam::12345678910:user/awscli"" client=""127.0.0.1:51326"" groups=""[system:masters]"" method=POST path=/authenticate sts=sts.eu-west-1.amazonaws.com uid=""heptio-authenticator-aws:12345678910:ABCDEFGHIJKLMNOP"" username=kubernetes-admin"In this output, IAM user arn:aws:iam::12345678910:user/awscli is mapped to user kubernetes-admin.To find requests that a specific user performed, search for operations that the kubernetes-admin user performed.Example query:fields @logStream, @timestamp, @message| filter @logStream like /^kube-apiserver-audit/| filter strcontains(user.username,"kubernetes-admin")| sort @timestamp desc| limit 50Example output:@logStream,@timestamp,@messagekube-apiserver-audit-71976ca11bea5d3083393f7d32dab75b,2021-08-11 09:29:13.095,"{...""requestURI"":""/api/v1/namespaces/kube-system/endpoints?limit=500";","string""verb"":""list"",""user"":{""username"":""kubernetes-admin"",""uid"":""heptio-authenticator-aws:12345678910:ABCDEFGHIJKLMNOP"",""groups"":[""system:masters"",""system:authenticated""],""extra"":{""accessKeyId"":[""ABCDEFGHIJKLMNOP""],""arn"":[""arn:aws:iam::12345678910:user/awscli""],""canonicalArn"":[""arn:aws:iam::12345678910:user/awscli""],""sessionName"":[""""]}},""sourceIPs"":[""12.34.56.78""],""userAgent"":""kubectl/v1.22.0 (darwin/amd64) kubernetes/c2b5237"",""objectRef"":{""resource"":""endpoints"",""namespace"":""kube-system"",""apiVersion"":""v1""}...}"To find API calls that a specific userAgent made, you can use this example query:fields @logStream, @timestamp, userAgent, verb, requestURI, @message| filter @logStream like /kube-apiserver-audit/| filter userAgent like /kubectl\/v1.22.0/| sort @timestamp desc| filter verb like /(get)/Shortened example output:@logStream,@timestamp,userAgent,verb,requestURI,@messagekube-apiserver-audit-71976ca11bea5d3083393f7d32dab75b,2021-08-11 14:06:47.068,kubectl/v1.22.0 (darwin/amd64) kubernetes/c2b5237,get,/apis/metrics.k8s.io/v1beta1?timeout=32s,"{""kind"":""Event"",""apiVersion"":""audit.k8s.io/v1"",""level"":""Metadata"",""auditID"":""863d9353-61a2-4255-a243-afaeb9183524"",""stage"":""ResponseComplete"",""requestURI"":""/apis/metrics.k8s.io/v1beta1?timeout=32s"",""verb"":""get"",""user"":{""username"":""kubernetes-admin"",""uid"":""heptio-authenticator-aws:12345678910:AIDAUQGC5HFOHXON7M22F"",""groups"":[""system:masters"",""system:authenticated""],""extra"":{""accessKeyId"":[""ABCDEFGHIJKLMNOP""],""arn"":[""arn:aws:iam::12345678910:user/awscli""],""canonicalArn"":[""arn:aws:iam::12345678910:user/awscli""],""sourceIPs"":[""12.34.56.78""],""userAgent"":""kubectl/v1.22.0 (darwin/amd64) kubernetes/c2b5237""...}"To find mutating changes made to the aws-auth ConfigMap, you can use this example query:fields @logStream, @timestamp, @message| filter @logStream like /^kube-apiserver-audit/| filter requestURI like /\/api\/v1\/namespaces\/kube-system\/configmaps/| filter objectRef.name = "aws-auth"| filter verb like /(create|delete|patch)/| sort @timestamp desc| limit 50Shortened example output:@logStream,@timestamp,@messagekube-apiserver-audit-f01c77ed8078a670a2eb63af6f127163,2021-10-27 05:43:01.850,{""kind"":""Event"",""apiVersion"":""audit.k8s.io/v1"",""level"":""RequestResponse"",""auditID"":""8f9a5a16-f115-4bb8-912f-ee2b1d737ff1"",""stage"":""ResponseComplete"",""requestURI"":""/api/v1/namespaces/kube-system/configmaps/aws-auth?timeout=19s"",""verb"":""patch"",""responseStatus"": {""metadata"": {},""code"": 200 },""requestObject"": {""data"": { contents of aws-auth ConfigMap } },""requestReceivedTimestamp"":""2021-10-27T05:43:01.033516Z"",""stageTimestamp"":""2021-10-27T05:43:01.042364Z"" }To find requests that were denied, you can use this example query:fields @logStream, @timestamp, @message| filter @logStream like /^authenticator/| filter @message like "denied"| sort @timestamp desc| limit 50Example output:@logStream,@timestamp,@messageauthenticator-8c0c570ea5676c62c44d98da6189a02b,2021-08-08 20:04:46.282,"time=""2021-08-08T20:04:44Z"" level=warning msg=""access denied"" client=""127.0.0.1:52856"" error=""sts getCallerIdentity failed: error from AWS (expected 200, got 403)"" method=POST path=/authenticate"To find the node that a pod was scheduled on, query the kube-scheduler logs.Example query:fields @logStream, @timestamp, @message| sort @timestamp desc| filter @logStream like /kube-scheduler/| filter @message like "aws-6799fc88d8-jqc2r"| limit 50Example output:@logStream,@timestamp,@messagekube-scheduler-bb3ea89d63fd2b9735ba06b144377db6,2021-08-15 12:19:43.000,"I0915 12:19:43.933124       1 scheduler.go:604] ""Successfully bound pod to node"" pod=""kube-system/aws-6799fc88d8-jqc2r"" node=""ip-192-168-66-187.eu-west-1.compute.internal"" evaluatedNodes=3 feasibleNodes=2"In this example output, pod aws-6799fc88d8-jqc2r was scheduled on node ip-192-168-66-187.eu-west-1.compute.internal.To find HTTP 5xx server errors for Kubernetes API server requests, you can use this example query:fields @logStream, @timestamp, responseStatus.code, @message| filter @logStream like /^kube-apiserver-audit/| filter responseStatus.code >= 500| limit 50Shortened example output:@logStream,@timestamp,responseStatus.code,@messagekube-apiserver-audit-4d5145b53c40d10c276ad08fa36d1f11,2021-08-04 07:22:06.518,503,"...""requestURI"":""/apis/metrics.k8s.io/v1beta1?timeout=32s"",""verb"":""get"",""user"":{""username"":""system:serviceaccount:kube-system:resourcequota-controller"",""uid"":""36d9c3dd-f1fd-4cae-9266-900d64d6a754"",""groups"":[""system:serviceaccounts"",""system:serviceaccounts:kube-system"",""system:authenticated""]},""sourceIPs"":[""12.34.56.78""],""userAgent"":""kube-controller-manager/v1.21.2 (linux/amd64) kubernetes/d2965f0/system:serviceaccount:kube-system:resourcequota-controller"",""responseStatus"":{""metadata"":{},""code"":503},..."}}"To troubleshoot a CronJob activation, search for API calls that the cronjob-controller made.Example query:fields @logStream, @timestamp, @message| filter @logStream like /kube-apiserver-audit/| filter user.username like "system:serviceaccount:kube-system:cronjob-controller"| display @logStream, @timestamp, @message, objectRef.namespace, objectRef.name| sort @timestamp desc| limit 50Shortened example output:{ "kind": "Event", "apiVersion": "audit.k8s.io/v1", "objectRef": { "resource": "cronjobs", "namespace": "default", "name": "hello", "apiGroup": "batch", "apiVersion": "v1" }, "responseObject": { "kind": "CronJob", "apiVersion": "batch/v1", "spec": { "schedule": "*/1 * * * *" }, "status": { "lastScheduleTime": "2021-08-09T07:19:00Z" } } }In this example output, the hello job in the default namespace runs every minute and was last scheduled at 2021-08-09T07:19:00Z.To find API calls that the replicaset-controller made, you can use this example query:fields @logStream, @timestamp, @message| filter @logStream like /kube-apiserver-audit/| filter user.username like "system:serviceaccount:kube-system:replicaset-controller"| display @logStream, @timestamp, requestURI, verb, user.username| sort @timestamp desc| limit 50Example output:@logStream,@timestamp,requestURI,verb,user.usernamekube-apiserver-audit-8c0c570ea5676c62c44d98da6189a02b,2021-08-10 17:13:53.281,/api/v1/namespaces/kube-system/pods,create,system:serviceaccount:kube-system:replicaset-controllerkube-apiserver-audit-4d5145b53c40d10c276ad08fa36d1f11,2021-08-04 0718:44.561,/apis/apps/v1/namespaces/kube-system/replicasets/coredns-6496b6c8b9/status,update,system:serviceaccount:kube-system:replicaset-controllerTo find operations that are made against a Kubernetes resource, you can use this example query:fields @logStream, @timestamp, @message| filter @logStream like /^kube-apiserver-audit/| filter verb == "delete" and requestURI like "/api/v1/namespaces/default/pods/my-app"| sort @timestamp desc| limit 10The preceding example query filters for delete API calls on the default namespace for pod my-app.Shortened example output:@logStream,@timestamp,@messagekube-apiserver-audit-e7b3cb08c0296daf439493a6fc9aff8c,2021-08-11 14:09:47.813,"...""requestURI"":""/api/v1/namespaces/default/pods/my-app"",""verb"":""delete"",""user"":{""username""""kubernetes-admin"",""uid"":""heptio-authenticator-aws:12345678910:ABCDEFGHIJKLMNOP"",""groups"":[""system:masters"",""system:authenticated""],""extra"":{""accessKeyId"":[""ABCDEFGHIJKLMNOP""],""arn"":[""arn:aws:iam::12345678910:user/awscli""],""canonicalArn"":[""arn:aws:iam::12345678910:user/awscli""],""sessionName"":[""""]}},""sourceIPs"":[""12.34.56.78""],""userAgent"":""kubectl/v1.22.0 (darwin/amd64) kubernetes/c2b5237"",""objectRef"":{""resource"":""pods"",""namespace"":""default"",""name"":""my-app"",""apiVersion"":""v1""},""responseStatus"":{""metadata"":{},""code"":200},""requestObject"":{""kind"":""DeleteOptions"",""apiVersion"":""v1"",""propagationPolicy"":""Background""},..."To retrieve a count of HTTP response codes for calls made to the Kubernetes API server, you can use this example query:fields @logStream, @timestamp, @message| filter @logStream like /^kube-apiserver-audit/| stats count(*) as count by responseStatus.code| sort count descExample output:responseStatus.code,count200,35066201,525403,125404,116101,2To find changes that are made to DaemonSets/Addons in the kube-system namespace, you can use this example query:filter @logStream like /^kube-apiserver-audit/| fields @logStream, @timestamp, @message| filter verb like /(create|update|delete)/ and strcontains(requestURI,"/apis/apps/v1/namespaces/kube-system/daemonsets")| sort @timestamp desc| limit 50Example output:{ "kind": "Event", "apiVersion": "audit.k8s.io/v1", "level": "RequestResponse", "auditID": "93e24148-0aa6-4166-8086-a689b0031612", "stage": "ResponseComplete", "requestURI": "/apis/apps/v1/namespaces/kube-system/daemonsets/aws-node?fieldManager=kubectl-set", "verb": "patch", "user": { "username": "kubernetes-admin", "groups": [ "system:masters", "system:authenticated" ] }, "userAgent": "kubectl/v1.22.2 (darwin/amd64) kubernetes/8b5a191", "objectRef": { "resource": "daemonsets", "namespace": "kube-system", "name": "aws-node", "apiGroup": "apps", "apiVersion": "v1" }, "requestObject": { "REDACTED": "REDACTED" }, "requestReceivedTimestamp": "2021-08-09T08:07:21.868376Z", "stageTimestamp": "2021-08-09T08:07:21.883489Z", "annotations": { "authorization.k8s.io/decision": "allow", "authorization.k8s.io/reason": "" } }In this example output, the kubernetes-admin user used kubectl v1.22.2 to patch the aws-node DaemonSet.To find the user that deleted a node, you can use this example query:fields @logStream, @timestamp, @message| filter @logStream like /^kube-apiserver-audit/| filter verb == "delete" and requestURI like "/api/v1/nodes"| sort @timestamp desc| limit 10Shortened example output:@logStream,@timestamp,@messagekube-apiserver-audit-e503271cd443efdbd2050ae8ca0794eb,2022-03-25 07:26:55.661,"{"kind":"Event","verb":"delete","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"],"arn":["arn:aws:iam::1234567890:user/awscli"],"canonicalArn":["arn:aws:iam::1234567890:user/awscli"],"sessionName":[""]}},"sourceIPs":["1.2.3.4"],"userAgent":"kubectl/v1.21.5 (darwin/amd64) kubernetes/c285e78","objectRef":{"resource":"nodes","name":"ip-192-168-37-22.eu-west-1.compute.internal","apiVersion":"v1"},"responseStatus":{"metadata":{},"status":"Success","code":200},"requestObject":{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Background"},"responseObject":{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"ip-192-168-37-22.eu-west-1.compute.internal","kind":"nodes","uid":"518ba070-154e-4400-883a-77a44a075bd0"}},"requestReceivedTimestamp":"2022-03-25T07:26:55.355378Z",}}"Follow"


Question: How do I resolve issues with an Amazon RDS database that is in an incompatible-network state?

"My Amazon Relational Database Service (Amazon RDS) DB instance is in an incompatible-network state. What does this mean, and how can I fix it?"

Answer: "My Amazon Relational Database Service (Amazon RDS) DB instance is in an incompatible-network state. What does this mean, and how can I fix it?Short descriptionAn incompatible-network state indicates one or more of the following is true of the Amazon RDS DB instance:There are no available IP addresses in the subnet that the Amazon RDS DB instance was launched into.The subnet mentioned in the Amazon RDS DB subnet group no longer exists in the Amazon Virtual Private Cloud (Amazon VPC).The RDS DB instance is publicly accessible, but the DNS hostnames and the DNS resolution options are turned off. Both options must be turned on for an RDS DB instance to be publicly accessible.You reached your service quota for elastic network interfaces in the AWS Region that you're using, and you must request a service quota increase.The original elastic network interface of the RDS DB instance was deleted while the instance was stopped. When the RDS DB instance was restarted, the elastic network interface wasn't found.The incompatible-network state of your RDS DB instance could be the result of performing one of the following actions:Modifying the RDS DB instance's class.Modifying the RDS DB instance to use a Multi-AZ deployment.Replacing a host due to a maintenance event.Launching a replacement RDS DB instance.Creating a new RDS DB instance after the quota for the elastic network interfaces has been reached.Restoring from a snapshot backup.Starting an RDS DB instance that was stopped.For more details about why your RDS DB instance is in an incompatible-network state, see Viewing Amazon RDS events.ResolutionThe incompatible-network state means that the Amazon RDS DB instance might still be accessible at the database level, but you can't modify or reboot it. It's a best practice to have a backup (snapshot or logical), in case your RDS DB instance enters an incompatible-network state. Having a backup allows you to restore the data to another RDS DB instance.Open the Amazon RDS console, and then choose Databases from the navigation pane.Choose the RDS DB instance that is in an incompatible-network state, and then note the VPC ID and subnet IDs from the Connectivity & Security pane.Open the Amazon VPC console, and then choose Your VPCs from the navigation pane.Select the VPC that you noted previously, choose Actions, and then choose Edit DNS hostnames. If DNS hostnames is not turned on, then check the box for enable.From Your VPCs, select the VPC that you noted previously, choose Actions, and then choose Edit DNS resolution. If the DNS resolution is not turned on, then check the box for enable.Choose Subnets from the navigation pane. Confirm that all subnets noted previously from the RDS DB instance details section are listed and have available IP addresses.Open the Amazon Elastic Compute Cloud (Amazon EC2) console, and then confirm that you haven't reached the quota for elastic network interfaces. Request a service quota increase if needed.Note: For more information on quotas for network interfaces per AWS Region, see Network interfaces.Note: Your instance will be in the incompatible-network state if RDS can't allocate your IP address under the following circumstances:You're using Amazon RDS on AWS Outposts.You activate the Customer-owned IP (CoIP) address setting.To fix an RDS DB instance that's in an incompatible-network state, verify if you've turned on automated backups for your RDS DB instance. If you did turn on automated backups, then temporarily stop any writes to the database and perform a point-in-time recovery (PITR).If you didn't turn on automated backups, create a new RDS DB instance. Then, migrate the data using AWS Database Migration Service (AWS DMS), or by using a backup and restore tool. For example, mysqldump, pg_dump, expdp / impdp.Important: After an instance enters an incompatible-network state, the RDS DB instance might not be accessible at the DB level to perform a logical backup.Related informationDB instance statusWorking with an Amazon RDS DB Instance in a VPCMy Amazon RDS instance status is stuck in an incompatible-parameters state. How can I fix this?Follow"


Question: Why can't I run AWS CLI commands on my EC2 instance?

Why am I receiving errors while trying to run AWS Command Line Interface (AWS CLI) commands on my Amazon Elastic Compute Cloud (Amazon EC2) instance?

Answer: "Why am I receiving errors while trying to run AWS Command Line Interface (AWS CLI) commands on my Amazon Elastic Compute Cloud (Amazon EC2) instance?Short descriptionWhen running AWS CLI commands on your instance, you might see one of the following error messages:"Unable to locate credentials. You can configure credentials by running 'aws configure'""An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation: You are not authorized to perform this operation""An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials""An error occurred (ExpiredToken) when calling the GetCallerIdentity operation: The security token included in the request is expired"The operation listed in the error varies depending on the operation that you called when the error occurred. In the preceding examples, the errors occurred when calling the DescribeInstances and GetCallerIdentity operations.Note: For communication issues between the AWS CLI and one of the AWS service endpoints, confirm that the DNS resolution and any VPC endpoints work correctly. For more information, see the following:AWS PrivateLink and VPC endpointsHow do I create VPC endpoints so that I can use Systems Manager to manage private EC2 instances without internet access?Amazon DNS serverResolutionVerify that you're running the most recent AWS CLI versionIf you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.Unable to locate credentialsVerify that the AWS CLI is installed and configured correctly.When using AWS Identity and Access Management (IAM) instance profiles, make sure that the IAM role association has completed.An error occurred (UnauthorizedOperation) and (AuthFailure)Make sure that the IAM role or IAM user has the correct permissions to run the relevant commands. For instructions on how to do this, see Why am I receiving the error message "You are not authorized to perform this operation" when I try to launch an EC2 instance?Make sure that the time on your Linux or Windows instance is correct.Make sure that you're using the correct Amazon Simple Token Service (AWS STS) token format. For more information, see Why did I receive the IAM error "AWS was not able to validate the provided access credentials" in some AWS Regions?Make sure that you're using the correct credentials to make the API call. If there are multiple sets of credentials on the instance, credential precedence might affect the credentials that the instance uses to make the API call. Verify the set of credentials that you're using by running the aws sts get-caller-identity command. For more information, see Why is my Amazon EC2 instance using IAM user credentials instead of role credentials?An error occurred (ExpiredToken)Temporary credentials expire at the time interval specified during creation. If the credentials for your IAM role are expired, then obtain a new STS token by  assuming a new IAM role.Related informationHow can I troubleshoot issues with an IAM policy?Follow"


Question: Why is my Storage Gateway activation failing when I try to activate my gateway using a public endpoint?

"I'm trying to activate my gateway on AWS Storage Gateway using a public endpoint. However, the activation is failing. How can I troubleshoot this?"

Answer: "I'm trying to activate my gateway on AWS Storage Gateway using a public endpoint. However, the activation is failing. How can I troubleshoot this?Short descriptionGateway activation using a public endpoint can fail for the following reasons:The gateway VM doesn't meet the minimum system requirements.The gateway's virtual machine (VM) can't reach the service endpoints on port 443.The gateway is not reachable on port 80.ResolutionNote: Activation can fail if the gateway VM or the gateway instance on Amazon Elastic Compute Cloud (Amazon EC2) was already activated. After activation is complete, the gateway stops listening on port 80 and can't communicate with the endpoint. Before you begin troubleshooting, confirm that the gateway VM or instance hasn't been activated before.Troubleshooting a gateway that's hosted on-premisesConfirm that the gateway VM meets the minimum hardware and storage requirementsThe VM must have a minimum of 4 CPUs and 16 GB memory to communicate with the gateway endpoints.The VM's root disk must be at least 80 GB.Storage Gateways have additional requirements and recommendations, depending on the type of gateway:See the hardware requirements for a Volume Gateway.See the hardware requirements for a Tape Gateway.See the hardware requirements for an S3 File Gateway.See the hardware requirements for an FSx File Gateway.Confirm that the gateway VM meets the network requirementsThe gateway VM must listen on TCP port 80 to receive the activation request from the gateway endpoint.The gateway VM must be able to access the endpoint anon-cp.storagegateway.region.amazonaws.com 443 to communicate with AWS.Your firewall and router must allow the required service endpoints for outbound traffic to AWS.Your firewall must allow traffic on port 123 so that your gateway's VM can synchronize with NTP time.Your firewall DNS resolution must allow traffic on port 53.All gateways share a common set of ports, but additional requirements vary based on the type of gateway:See the network requirements for a Volume Gateway.See the network requirements for a Tape Gateway.See the network requirements for an S3 File Gateway.See the network requirements for an FSx File Gateway.Additionally, verify that there are no ongoing SSL inspections or deep packed inspections between your gateway's VM and the Storage Gateway service endpoints. The file gateway software disconnects the connection when there's a deep packet inspection. This is because the software treats deep packet inspections as a man-in-the-middle attack.To check for any ongoing inspections, you can run an OpenSSL command from a VM that's in the same network as your gateway's VM:openssl s_client -connect client-cp.storagegateway.us-east-1.amazonaws.com:443Test the network connectionYou can test your gateway's connection to the required endpoints in the following ways:Run a network connectivity test from your gateway's local VM console.Run a telnet command from a VM that's in the same network as the gateway VM:telnet anon-cp.storagegateway.region.amazonaws.com 443Troubleshooting a gateway that's hosted on Amazon EC2Confirm that the Amazon EC2 gateway instance meets the minimum hardware and storage requirementsThe instance must have a minimum of 4 CPUs and 16 GB memory for the gateway to communicate with the gateway endpoints.The instance's root disk must be at least 80 GB.Note: Best practices for the instance types to use for a gateway are m4.xlarge and m4.2xlarge.Storage Gateways have additional requirements and recommendations, depending on the type of gateway:See the hardware requirements for a Volume Gateway.See the hardware requirements for a Tape Gateway.See the hardware requirements for an S3 File Gateway.See the hardware requirements for an FSx File Gateway.Confirm that the gateway instance meets the network requirementsThe instance's security group must allow inbound traffic from your client or workstation's IP address on TCP port 80.The instance's security group must allow outbound traffic to TCP ports 443, 123, and 53 to communicate with the service endpoints.All gateways share a common set of ports, but additional requirements vary based on the type of gateway:See the network requirements for a Volume Gateway.See the network requirements for a Tape Gateway.See the network requirements for an S3 File Gateway.See the network requirements for an FSx File Gateway.Test the network connectionRun a network connectivity test from your gateway's local VM console.Run a telnet command from an EC2 instance that's in the same network or subnet as the gateway instance:telnet anon-cp.storagegateway.region.amazonaws.com 443Follow"


Question: How can I migrate from an RDS for PostgreSQL to Aurora PostgreSQL?

I want to migrate from an Amazon Relational Database Service (Amazon RDS) for PostgreSQL to Amazon Aurora PostgreSQL-Compatible Edition.

Answer: "I want to migrate from an Amazon Relational Database Service (Amazon RDS) for PostgreSQL to Amazon Aurora PostgreSQL-Compatible Edition.Short descriptionThere are five options for migrating data from your existing Amazon RDS for PostgreSQL database to an Amazon Aurora PostgreSQL-Compatible DB cluster.To migrate from Amazon RDS for PostgreSQL to Aurora PostgreSQL, use one of the following methods depending on your use case:Using a snapshotUsing an Aurora read replicaUsing a pg_dump utilityUsing logical replicationUsing a data import from Amazon S3ResolutionUsing a snapshot to migrate an RDS for PostgreSQLTo create an Aurora PostgreSQL DB cluster, you can migrate a DB snapshot of an RDS for PostgreSQL DB instance. The new Aurora PostgreSQL DB cluster is populated with the data from the original RDS for PostgreSQL DB instance. The RDS for PostgreSQL snapshot must use the same or a lower minor version as the Aurora PostgreSQL.You can migrate the snapshot using the Amazon RDS console or AWS Command Line Interface (AWS CLI). For instructions, see Migrating a snapshot of an RDS for PostgreSQL DB instance to an Aurora PostgreSQL DB cluster.Important:If the DB snapshot is in a different AWS Region than where you want the data located, then copy the DB snapshot to another Region.If the migration is to another AWS account, see How can I share an encrypted Amazon RDS DB snapshot with another account?Using an Aurora read replica to migrate an RDS for PostgreSQLUse an RDS for PostgreSQL DB instance as the basis for a new Aurora PostgreSQL DB cluster by using an Aurora read replica. The Aurora read replica is available for migrating only within the same AWS Region and account. The Aurora read replica option minimizes downtime during a migration. You can promote the new cluster when you have zero (0) replication lag between the primary RDS instance and the Aurora read replica.Creating an Aurora read replica using the Amazon RDS console is available only if the AWS Region offers a compatible Aurora PostgreSQL version. You can create an Aurora read replica for an RDS for PostgreSQL DB instance by using the Amazon RDS console or the AWS CLI. For instructions, see Migrating data from an RDS for PostgreSQL DB instance to an Aurora PostgreSQL DB cluster using an Aurora read replica.Note: The source RDS for PostgreSQL must have sufficient storage capacity to retain the write-ahead logs (WAL) segments while the migration is occurring. For more information, see Preparing to migrate data by using an Aurora read replica.Using a pg_dump utility to migrate an RDS for PostgreSQLPg_dump is a utility for backing up a PostgreSQL database. If your database size is small or you don't have downtime considerations, you can use this option. This method is best suited for migrating a few GB to 500-GB sized databases. Migrating larger databases might require higher outage depending on database size. Use an Amazon Elastic Compute Cloud (Amazon EC2) instance as the host within the same Availability Zone as the target database to mitigate network latency.For instructions, see Importing a PostgreSQL database from an Amazon EC2 instance.For additional pg_dump information, see the official PostgreSQL documentation.Using logical replication to migrate an RDS for PostgreSQLAmazon RDS for PostgreSQL and Aurora PostgreSQL aren't supported as targets for external PostgreSQL physical replication. However, logical replication is one way to reduce overall migration time. You can set a publisher and subscriber model between the RDS for PostgreSQL and Aurora PostgreSQL instance to migrate data with minimal downtime.For more information, seeBest practices for migrating PostgreSQL databases to Amazon RDS and Amazon Aurora and follow the instructions inLogical replication.Using a data import from Amazon S3 to migrate an RDS for PostgreSQLIf your data is stored using Amazon Simple Storage Service (Amazon S3), you can migrate the data to an Aurora PostgreSQL DB cluster instance. For instructions, see Importing data from Amazon S3 into an Aurora PostgreSQL DB cluster.Related informationWorking with parameter groupsFollow"


Question: How do I troubleshoot issues with VPC route tables?

"I have configured my route table, but my Amazon Virtual Private Cloud (Amazon VPC) can't communicate with the destination."

Answer: "I have configured my route table, but my Amazon Virtual Private Cloud (Amazon VPC) can't communicate with the destination.Short descriptionEach subnet in an Amazon VPC is associated with a route table that controls the routing for the subnet. The routing options for your Amazon VPC depend on the gateway or connections that you're using, such as:Public subnetsSubnets using NAT instances or NAT gatewaysSubnets using VPC peering connectionsSubnets using AWS VPNSubnets using AWS Direct ConnectSubnets using gateway VPC endpointsSubnets using virtual interface VPC endpointsResolutionTo identify the source of the issue, check the route tables of the subnets with the resources that are impacted.Public subnetsOpen the Amazon VPC console.In the navigation pane, under Subnets, choose your public subnet.Choose the Route Table view.Confirm that the route table destination has a default route (0.0.0.0/0 for IPv4 and ::/0 for IPv6) that points to an internet gateway.For more information, see Why can't I connect to an Amazon EC2 instance within my Amazon VPC from the internet?Subnets using NAT instances or NAT gatewaysOpen the Amazon VPC console.In the navigation pane, under Subnets, choose your private subnet.Choose the Route Table view, and confirm that the route table has a default route that points to a NAT instance or gateway.Confirm that the NAT device is launched in a public subnet. Then, perform the checks required for public subnets listed in the previous section.Note: If you're using a NAT instance, be sure that you turned off the source destination check.To configure your VPC with IPv6, and to prevent traffic from the internet routing to your instances in a private subnet, use egress-only internet gateways. For more information about configuring an egress-only internet gateway, see Turn on outbound IPv6 traffic using an egress-only internet gateway.For more information about troubleshooting VPC peering connection issues, see Troubleshoot NAT gateways.Subnets using VPC peering connectionsOpen the Amazon VPC console.In the navigation pane, choose Peering Connections, and then choose your peering connection.Confirm that its status is Active.From the navigation pane, choose Subnets, and then choose the subnets of the Amazon VPC that you want to connect using a peering connection.Choose the Route Tables view, and then confirm that the route tables have one of the following:Routes to CIDR with specific subnets.Routes to the entire CIDR of the peered Amazon VPC, including the peering connection noted in step 2.Confirm that the route tables include all the subnets for the peered Amazon VPC.Note: Confirm that there are no VPC peering connection configurations that aren't valid.For more information, see How do I troubleshoot problems establishing communication over VPC peering?Subnets using AWS VPNOpen the Amazon VPC console.In the navigation pane, choose VPN Connections, and then choose the VPN connection.Confirm that the VPN status is available and at least one of the tunnels' status is UP.Note: If you are using a dynamic VPN, be sure that BGP routes are received by AWS VPN. You can turn on route propagation to confirm that the BGP routes are being propagated to the virtual private gateway.Note the virtual private gateway used for this VPN connection.Choose Subnets from the navigation pane, and then select the subnet of the Amazon VPC that you want to connect to the VPN.Choose the Route Table view, then confirm the following:The route destination is your network.The target is the virtual private gateway noted in step 4.For more information, see How do I troubleshoot connection problems between an AWS VPN endpoint and a policy-based VPN?Subnets using AWS Direct ConnectOpen the AWS Direct Connect console.In the navigation pane, choose Virtual Interfaces, and then choose the private virtual interface.Confirm that the BGP status is UP.Note the virtual private gateway used for the private virtual interface.Open the Amazon VPC console.In the navigation pane, under Subnets, select the subnets of the Amazon VPC that you want to connect using AWS Direct Connect.Choose the Route Table view, and then confirm the following:There is a route with the destination of your network.There is a route to a target of the virtual private gateway as noted in step 4.Note: If you are using BGP, be sure that the routes are received by AWS. You can turn on route propagation to confirm that the BGP routes are being propagated to the virtual private gateway.For more information, see Troubleshooting AWS Direct Connect.Subnets using gateway VPC endpointsOpen the Amazon VPC console.In the navigation pane, choose Endpoints, and then choose the endpoint.Confirm that its status is available, and then note the Endpoint ID.In the navigation pane, under Subnets, select the subnet of the Amazon VPC that you want to connect to an AWS service using an endpoint.Choose the Route Tables view, and then confirm the following:There is a route added to the route table with a destination that specifies the prefix list ID of the service.There is a target with the endpoint ID obtained in step 3.Confirm that the VPC endpoint policy allows communication to an AWS service for the resources in the subnets of your Amazon VPC.For more information, see Why can’t I connect to an S3 bucket using a gateway VPC endpoint?Subnets using virtual interface VPC endpointsOpen the Amazon VPC console.In the navigation pane, choose Endpoints, and then choose the endpoint.Choose the Subnets column, and then confirm that an endpoint network interface has been created in the subnet associated with service you want to connect.In the navigation pane, under Endpoints, choose the Policy view.Confirm that the security group allows access to the AWS service.For more information, see Access an AWS service using an interface VPC endpoint.Related informationThe VPN tunnel between my customer gateway and my virtual private gateway is Up, but I am unable to pass traffic through it. What can I do?Configure route tablesWork with VPCsWhy can't I connect to an Amazon EC2 instance within my Amazon VPC from the internet?How can I connect to my Amazon VPC?Follow"