Answers: <p>You will need to write a script that parses and queries this data because as far as I'm aware, there is not a tool or cli function that performs this. Luckily, you can gather all of this information with the CLI. </p>
  
  <ol>
  <li><p><a href="http://docs.aws.amazon.com/cli/latest/reference/organizations/list-accounts.html" rel="noreferrer">List</a> and parse all of the accounts in your org. </p>
  
  <pre><code>aws organizations list-accounts
  </code></pre></li>
  <li><p>For each account, <a href="http://docs.aws.amazon.com/cli/latest/reference/s3api/list-buckets.html" rel="noreferrer">list</a> and parse all of the buckets. </p>
  
  <pre><code>aws s3api list-buckets --query "Buckets[].Name"
  </code></pre></li>
  <li><p>Finally, get the size of each bucket within each account. You can use the same cli command you were before, but be warned that you are going to be listing the individual size of each item within the bucket. You can also use this cli command to get bucket size.</p>
  
  <pre><code>aws s3api list-objects --bucket BUCKETNAME --output json --query "
  [sum(Contents[].Size), length(Contents[])]"
  </code></pre></li>
  </ol>
  
Answers: CC BY-SA 3.0
Answers: 2017-10-04T14:44:48.050Z
Answers: 1. does not list the profile information of the AWS account. I did not have time to regex this from the aws config file where this information is stored so I just hardcoded the values in the script i posted below
Answers: Kyle Steenkamp
Answers: CC BY-SA 3.0
Answers: 2017-10-04T14:00:50.990Z
Answers: 2017-10-04T14:00:50.990Z
Answers: Preston Martin
Answers: <p>Easiest way to see this information nowadays is using S3 Storage Lens.</p>
  <p>S3 dashboard &gt; S3 storage lens (left navigation pane) &gt; default-account-dashboard &gt; Buckets tab</p>
  <p>Scroll down to the &quot;Buckets&quot; section and sort by total storage.</p>
  
Answers: CC BY-SA 4.0
Answers: 2022-12-12T19:26:08.130Z
Answers: 2022-12-12T19:40:49.883Z
Answers: 2022-12-12T19:40:49.883Z
Answers: A. Kendall
Answers: <p>You're going to have to write a script to do this unless there's a tool to do it that I can't find.</p>
  
  <p>If you're working with multiple aws accounts you'll need to be switching between accounts to get the bucket sizes.</p>
  
  <p>To get all of the buckets and their total size you can use '<a href="http://docs.aws.amazon.com/cli/latest/reference/s3api/list-buckets.html" rel="nofollow noreferrer">aws s3api</a>'. Using <code>list-buckets</code> you can get all of the buckets in one account and then you'll be able to list their sizes and try the next account.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-07-16T20:15:56.073Z
Answers: I wrote a Python script with boto3 to do this: https://gist.github.com/rwiggins/b945b5ec1693c1d473a814915adcee8c (note: uses CloudWatch's GetMetricStatistics to pull the bucket sizes efficiently / without summing all object sizes in a bucket)
Answers: Reid
Answers: CC BY-SA 3.0
Answers: 2017-10-04T07:55:17.810Z
Answers: 2017-10-04T07:55:17.810Z
Answers: Briansbum
Answers: <p>None of these show the size of the bucket if Versioning is turned on and massive amounts of data have been "deleted".  In the S3 GUI console, the Metrics tab shows what you are being billed for.</p>
  
  <p>Scraped the following command out of a script found at  <a href="https://serverfault.com/questions/84815/how-can-i-get-the-size-of-an-amazon-s3-bucket">https://serverfault.com/questions/84815/how-can-i-get-the-size-of-an-amazon-s3-bucket</a> .  It works well on Windows.  Substitute <code>yourregion</code> and <code>yourbucketname</code>:</p>
  
  <p><code>aws cloudwatch get-metric-statistics --namespace AWS/S3 --start-time 2019-05-09T00:01:00Z --end-time 2019-05-11T23:59:00Z --period 86400 --statistics Average --region yourregion --metric-name BucketSizeBytes --dimensions Name=BucketName,Value=yourbucketname Name=StorageType,Value=StandardStorage</code></p>
  
  <p>This gives the average size in bytes over the days bounded by the start and end times.  <strong>The result can shock you</strong> if your bucket has Versioning Enabled (active) <em>and</em> you think you have "deleted" massive amounts of data.</p>
  
  <p>To clean up Versioning-enabled buckets, you must set a LifeCycle rule.  You can choose to permanently delete Previous Versions.  You may omit the checkboxes at the Transitions step.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-05-12T23:48:37.400Z
Answers: 2019-05-13T16:52:56.473Z
Answers: 2019-05-13T16:52:56.473Z
Answers: Newbie
Answers: <p>I just had to go through this and wrote/uploaded a script for it in GitHub.  <a href="https://github.com/johnwhumphreys/s3-buckets-by-size/blob/main/get-bucket-sizes.py" rel="nofollow noreferrer">https://github.com/johnwhumphreys/s3-buckets-by-size/blob/main/get-bucket-sizes.py</a></p>
  <p>It's a single python script that will quickly grab the cloudwatch metrics for the sizes for your buckets and rank them by order, showing the top 10.  Runs in a minute or so on our 200 bucket account.</p>
  <p>(no way I could have listed the files we had in any reasonable time frame, so had to find a faster way)</p>
  <p>This also deals w/ <strong>Intelligent Tiering</strong> as of 2023.</p>
  
Answers: CC BY-SA 4.0
Answers: 2023-05-05T00:58:03.007Z
Answers: 2023-05-05T00:58:03.007Z
Answers: John Humphreys
Answers: <p><strong>Resolution 1</strong></p>
  <p>So I solved this with the following script. I originally posted the question just in case there was an easier way that I was not aware of.</p>
  <pre><code>#!/bin/bash
  aws_profile=('profile1' 'profile2' 'profile3');
  
  #loop AWS profiles
  for i in &quot;${aws_profile[@]}&quot;; do
    echo &quot;${i}&quot;
    buckets=($(aws s3 ls s3:// --recursive --profile &quot;${i}&quot; --region your_region | awk '{print $3}'))
  
    #loop S3 buckets
    for j in &quot;${buckets[@]}&quot;; do
    echo &quot;${j}&quot;
    aws s3 ls s3://&quot;${j}&quot; --recursive --human-readable --summarize --profile &quot;${i}&quot; --region your_region | awk END'{print}'
    done
  
  done
  </code></pre>
  <p><strong>Resolution 2</strong></p>
  <p>Using <strong>Dashboards</strong> in <strong>CloudWatch</strong> in the AWS console.</p>
  <p>You can then simply specify all S3 buckets and add the numbers stats to show the storage size metrics.</p>
  <p>This won't cost you plenty of API calls and can be significantly faster depending on the size of the s3 buckets(takes quite awhile to get the size on very large buckets).</p>
  <p><strong>Verdict</strong></p>
  <p>Creating the Dashboard (<strong>Resolution 2</strong>) on each AWS account was the most efficient option <strong>for me</strong> cause it is way quicker for me to log in and grab the metrics manually from each AWS account than to wait for the scripts API calls to finish. :(</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-10-04T14:57:27.603Z
Answers: Why `| awk END'{print}'` ?
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-10-04T15:08:59.587Z
Answers: Using the recursive option shows the size of every folder and file and I only need the output of the total size of the bucket.
Answers: Kyle Steenkamp
Answers: CC BY-SA 4.0
Answers: 2018-07-16T17:38:27.550Z
Answers: You need to select a '1 day' or greater period for the dashboard to display anything
Answers: Jeremy Leipzig
Answers: CC BY-SA 4.0
Answers: 2019-03-06T17:52:16.173Z
Answers: Beware that the solution proposed in Resolution 2 increments your Cloudwatch cost since dashboards have a $3 cost each at the moment. https://aws.amazon.com/cloudwatch/pricing/?nc1=h_ls
Answers: Drubio
Answers: CC BY-SA 4.0
Answers: 2020-03-21T20:14:17.900Z
Answers: What does the `[@]` in `"${buckets[@]}"` mean?
Answers: Joe
Answers: CC BY-SA 4.0
Answers: 2021-07-01T13:22:17.193Z
Answers: @Joe: the `@` in `${buckets[@]}` represents all elements in the array. If you leave off the `[@]` the loop will only work on the first element. See the section "Looping through arrays" here: https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays#:~:text=looping%20through%20arrays
Answers: Russell G
Answers: CC BY-SA 4.0
Answers: 2017-10-04T14:41:30.177Z
Answers: 2022-12-12T20:01:55.010Z
Answers: 2022-12-12T20:01:55.010Z
Answers: Kyle Steenkamp
Body: <p>I have multiple <strong>AWS</strong> accounts and I need to list all <strong>S3</strong> buckets per account and then view each buckets total size. </p>
  
  <p>Currently, I can only view the storage size of a single <strong>S3</strong> bucket with:</p>
  
  <pre><code>aws s3 ls s3://mybucket --recursive --human-readable --summarize
  </code></pre>
  
Comments: CC BY-SA 3.0
Comments: 2017-12-16T10:06:42.720Z
Comments: S3 Select was released few days ago in re:invent 2017 https://www.twitch.tv/videos/206752912
Comments: Abdennour TOUMI
Comments: CC BY-SA 4.0
Comments: 2020-02-05T13:07:42.890Z
Comments: You might want to try this code I wrote to achieve the same thing myself: https://github.com/insperitas/s3-data
Comments: jimmy978654321
Tags: <amazon-s3><aws-cli>
Title: View all AWS S3 buckets and list each buckets storage used
UserDisplayName: Kyle Steenkamp
ThreadText: # View all AWS S3 buckets and list each buckets storage used
  
  (Asked by: username_0 on 2017-10-04)
  
  I have multiple **AWS** accounts and I need to list all **S3** buckets per account and then view each buckets total size.
  
  Currently, I can only view the storage size of a single **S3** bucket with:
  
  ```
  aws s3 ls s3://mybucket --recursive --human-readable --summarize
  
  ```
  
  **Tags:** amazon-s3, aws-cli
  
  > **Comments:**
  > **username_1**: S3 Select was released few days ago in re:invent 2017 https://www.twitch.tv/videos/206752912  
  > **username_2**: You might want to try this code I wrote to achieve the same thing myself: https://github.com/insperitas/s3-data  
  
  ---
  
  ## 6 answers
  
  ### Answer by username_0 on 2017-10-04. Score: 20 | Accepted answer
  
  **Resolution 1**
  
  So I solved this with the following script. I originally posted the question just in case there was an easier way that I was not aware of.
  
  ```
  #!/bin/bash
  aws_profile=('profile1' 'profile2' 'profile3');
  
  #loop AWS profiles
  for i in "${aws_profile[@]}"; do
    echo "${i}"
    buckets=($(aws s3 ls s3:// --recursive --profile "${i}" --region your_region | awk '{print $3}'))
  
    #loop S3 buckets
    for j in "${buckets[@]}"; do
    echo "${j}"
    aws s3 ls s3://"${j}" --recursive --human-readable --summarize --profile "${i}" --region your_region | awk END'{print}'
    done
  
  done
  
  ```
  
  **Resolution 2**
  
  Using **Dashboards** in **CloudWatch** in the AWS console.
  
  You can then simply specify all S3 buckets and add the numbers stats to show the storage size metrics.
  
  This won't cost you plenty of API calls and can be significantly faster depending on the size of the s3 buckets(takes quite awhile to get the size on very large buckets).
  
  **Verdict**
  
  Creating the Dashboard (**Resolution 2**) on each AWS account was the most efficient option **for me** cause it is way quicker for me to log in and grab the metrics manually from each AWS account than to wait for the scripts API calls to finish. :(
  
  > **Comments:**
  > **username_9**: Why `| awk END'{print}'` ?  
  > **username_0**: Using the recursive option shows the size of every folder and file and I only need the output of the total size of the bucket.  
  > **username_10**: You need to select a '1 day' or greater period for the dashboard to display anything  
  > **username_11**: Beware that the solution proposed in Resolution 2 increments your Cloudwatch cost since dashboards have a $3 cost each at the moment. https://aws.amazon.com/cloudwatch/pricing/?nc1=h_ls  
  > **username_12**: What does the `[@]` in `"${buckets[@]}"` mean?  
  > **username_13**: @username_12: the `@` in `${buckets[@]}` represents all elements in the array. If you leave off the `[@]` the loop will only work on the first element. See the section "Looping through arrays" here: https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays#:~:text=looping%20through%20arrays  
  
  ---
  
  ### Answer by username_3 on 2017-10-04. Score: 5
  
  You will need to write a script that parses and queries this data because as far as I'm aware, there is not a tool or cli function that performs this. Luckily, you can gather all of this information with the CLI.
  
  1. [List](http://docs.aws.amazon.com/cli/latest/reference/organizations/list-accounts.html) and parse all of the accounts in your org.
  
     ```
     aws organizations list-accounts
  
     ```
  2. For each account, [list](http://docs.aws.amazon.com/cli/latest/reference/s3api/list-buckets.html) and parse all of the buckets.
  
     ```
     aws s3api list-buckets --query "Buckets[].Name"
  
     ```
  3. Finally, get the size of each bucket within each account. You can use the same cli command you were before, but be warned that you are going to be listing the individual size of each item within the bucket. You can also use this cli command to get bucket size.
  
     ```
     aws s3api list-objects --bucket BUCKETNAME --output json --query "
     [sum(Contents[].Size), length(Contents[])]"
  
     ```
  
  > **Comments:**
  > **username_0**: 1. does not list the profile information of the AWS account. I did not have time to regex this from the aws config file where this information is stored so I just hardcoded the values in the script i posted below  
  
  ---
  
  ### Answer by username_4 on 2022-12-12. Score: 3
  
  Easiest way to see this information nowadays is using S3 Storage Lens.
  
  S3 dashboard > S3 storage lens (left navigation pane) > default-account-dashboard > Buckets tab
  
  Scroll down to the "Buckets" section and sort by total storage.
  
  
  ---
  
  ### Answer by username_5 on 2017-10-04. Score: 2
  
  You're going to have to write a script to do this unless there's a tool to do it that I can't find.
  
  If you're working with multiple aws accounts you'll need to be switching between accounts to get the bucket sizes.
  
  To get all of the buckets and their total size you can use '[aws s3api](http://docs.aws.amazon.com/cli/latest/reference/s3api/list-buckets.html)'. Using `list-buckets` you can get all of the buckets in one account and then you'll be able to list their sizes and try the next account.
  
  > **Comments:**
  > **username_6**: I wrote a Python script with boto3 to do this: https://gist.github.com/rwiggins/b945b5ec1693c1d473a814915adcee8c (note: uses CloudWatch's GetMetricStatistics to pull the bucket sizes efficiently / without summing all object sizes in a bucket)  
  
  ---
  
  ### Answer by username_7 on 2019-05-12. Score: 1
  
  None of these show the size of the bucket if Versioning is turned on and massive amounts of data have been "deleted". In the S3 GUI console, the Metrics tab shows what you are being billed for.
  
  Scraped the following command out of a script found at <https://serverfault.com/questions/84815/how-can-i-get-the-size-of-an-amazon-s3-bucket> . It works well on Windows. Substitute `yourregion` and `yourbucketname`:
  
  `aws cloudwatch get-metric-statistics --namespace AWS/S3 --start-time 2019-05-09T00:01:00Z --end-time 2019-05-11T23:59:00Z --period 86400 --statistics Average --region yourregion --metric-name BucketSizeBytes --dimensions Name=BucketName,Value=yourbucketname Name=StorageType,Value=StandardStorage`
  
  This gives the average size in bytes over the days bounded by the start and end times. **The result can shock you** if your bucket has Versioning Enabled (active) *and* you think you have "deleted" massive amounts of data.
  
  To clean up Versioning-enabled buckets, you must set a LifeCycle rule. You can choose to permanently delete Previous Versions. You may omit the checkboxes at the Transitions step.
  
  
  ---
  
  ### Answer by username_8 on 2023-05-05. Score: 0
  
  I just had to go through this and wrote/uploaded a script for it in GitHub. <https://github.com/johnwhumphreys/s3-buckets-by-size/blob/main/get-bucket-sizes.py>
  
  It's a single python script that will quickly grab the cloudwatch metrics for the sizes for your buckets and rank them by order, showing the top 10. Runs in a minute or so on our 200 bucket account.
  
  (no way I could have listed the files we had in any reasonable time frame, so had to find a faster way)
  
  This also deals w/ **Intelligent Tiering** as of 2023.
  
  
  ---
  
  