Answers.Body: <p>You don't necessarily have to analyse it all. Just by having such data available you can achieve a lot in terms of surveillance, as long as you can retrieve relevant parts.</p>
  
  <p>A few years ago there was a <a href="https://www.wnycstudios.org/podcasts/radiolab/articles/eye-sky" rel="noreferrer">Radiolab podcast, "The Eye in the Sky"</a> (there's a full transcript on the site). The basic idea is that you have a plane circling a city 24/7, and filming what goes on. If there was a crime somewhere, you retrieve the recordings after the event, and you can track back to where vehicles involved in the crime were coming from, and where they went after the crime. If nothing happens, you simply archive the data, and perhaps remove it after a month or so.</p>
  
  <p>This method was used to solve a hit-and-run assassination of a police woman who was on her way to work. The gang who committed the attack were rather surprised when the police showed up at their secret hide-out a few days later, as they could see on the images where the cars involved went to later. At the time and place of the murder there were obviously no witnesses who could have done that. And this involved no computational processing at all.</p>
  
  <p>The possibilities this opens up are just scary, as you can track pretty much anybody's movements without actually needing someone to follow them. Add to that street-level CCTV, and not much can happen without you being able to find out.</p>
  
  <p>In this scenario there is no processing at all, but you could imaging simple processing steps, such as tracking vehicles or changes in the environment, which could be used to give clues about potentially 'interesting' events. So instead of using it 'passively' as a kind of memory, you could use that data to identify things that happened that you weren't aware of.</p>
  
  <p>And this is without even any clandestine access to people's data. If you add that dimension, then you might even be able to identify crimes/etc before they even happen. Text processing can be quite fast, but is not easy to do, as presumably few people would openly communicate about things they were planning. So I guess we're still a long way away from that.</p>
  
  <p>Of course there is the ethical dimension (which is mentioned in the podcast): who has access to that data, and who decides what it is used for? If you do, and you suspect your partner of being unfaithful, who/what would stop you from checking out their movements? Or check up on that politician who might have a secret affair, or a gambling issue, or who keeps being in the same locations as a well-known drug dealer. All rather scary.</p>
  
  <p>While a complete analysis of all such data would be very heavy computationally, and fraught with false positives and recall problems, it might simply be enough to index it by time, location, and perhaps people involved (face recognition seems to be reasonably good, though still with a rather high error rate). This is enough already to make me feel worried about the future.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-28T10:26:08.877Z
Answers.Text: I also think worth adding to this good answer the one-trick item I was going to write: Edge processing & filtering. It is possible to reduce the amount of device-level data down a lot - orders of magnitude - by filtering only to active/interesting scenarios, e.g. there's no point videoing the inside of someone's pockets. Embedded AI assists with this by adding smarter filters, not just "is there a signal", but "is something interesting happening", which can decrease centralised bandwidth, CPU and storage requirements yet further.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-28T18:40:34.260Z
Answers.Text: And if you're writing a book, you can have the protagonists buy 100 phones and hack them to return false data. (Someone did this recently to confuse Google Maps)
Answers.UserDisplayName: Criticize SE actions means ban
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T00:11:26.950Z
Answers.Text: @user253751 Not actually false data, he just loaded them on a cart, walked through his street, and Google Maps concluded there was a traffic jam because 100 "vehicles" were moving slowly. No hacking necessary.
Answers.UserDisplayName: gerrit
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T16:37:43.793Z
Answers.Text: @gerrit It's false data because there weren't 100 vehicles moving slowly in a traffic jam.
Answers.UserDisplayName: Criticize SE actions means ban
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T17:33:26.743Z
Answers.Text: @user253751 The data are real, but Googles algorithms drew incorrect conclusions from real data.
Answers.UserDisplayName: gerrit
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-28T09:36:40.140Z
Answers.LastActivityDate: 2020-02-28T09:36:40.140Z
Answers.UserDisplayName: Oliver Mason
Answers.Body: <p>You would also want to consider physical limitations. If you are even <em>storing</em> 126 yottabyte of data per day, then if we look at the current theoretical densest data storage medium, DNA, at 215 petabytes per gram, we get...
  <span class="math-container">${(126 * 10^{24}) \over (215 * 10^{15})} = 586046511$</span> grams per day</p>
  
  <p>586046511 g = 586046 kg = 586 Metric Tonnes just for storage.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-29T07:45:05.033Z
Answers.Text: to put that number in perspective, note that it's actually a bit less than the 695 metric tonnes of garbage produced daily by the US. (tI took numbers from : https://www.treehugger.com/environmental-policy/trash-numbers-startling-statistics-about-americans-and-their-garbage.html)
Answers.UserDisplayName: Pac0
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-05T00:13:04.293Z
Answers.Text: that's very sad
Answers.UserDisplayName: mark mark
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-28T19:12:48.863Z
Answers.LastActivityDate: 2020-02-28T19:39:08.860Z
Answers.LastEditDate: 2020-02-28T19:39:08.860Z
Answers.UserDisplayName: Martin Main
Answers.Body: <p>The answer is really very simple. If you have the dystopian power over all the mobile devices in the first place, you would not make them send all their data over to any "global data storage" just like that. Instead, you would have put a local AI on each device that filters, processes, categorizes and flags the important parts, sending only those parts plus an intelligent summary of the remaining data to a global AI. The global AI combines and synthesizes the parts that all the local AIs send to it, and may request further data from the local AIs based on what it wants to know.</p>
  
  <p>Naturally, since you are a competent dystopia architect, you design each local AI to be intelligent enough to subvert any human's attempt to remove it, stop its activity, or otherwise interfere with its data collection and processing. The local AIs also continuously communicate in a distributed network with other local AIs regarding their status and any adversarial activities, so that they can quickly act to defend themselves if the need arises, and also notify the global AI of any attack. In this surveillance state, it is an easy task for the global AI to send armed agents to deal with any threat to the AI network that manages to gain any foothold in the information cyberspace.</p>
  
  <p>The point is that the most durable dystopia is a <strong>defended distributed dystopia</strong>, which would make it <strong>robust</strong> and <strong>scalable</strong>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-29T16:03:01.187Z
Answers.LastActivityDate: 2020-02-29T16:03:01.187Z
Answers.UserDisplayName: user21820
Body: <p>This isn't really a conspiracy theory question. More of an inquire on the global computational power and data storage logistics question.</p>
  
  <p>Most recording instruments such as cameras and microphones are typically voluntary opt in devices, in that, they have to be activated before they start recording. What happens if all of these devices were permanently activated and started recording data to some distributed global data storage?</p>
  
  <p>There are 400 hours of video uploaded to YouTube every minute.</p>
  
  <p>Let’s do some very rough math.</p>
  
  <p>I’m going to assume for the rest of this post that the average video is 1080p which is 2.5GB (or <span class="math-container">$10^9$</span> bytes) per hour. From that, we get about 400 hrs * 60 mins * 2.5GB/hrs * 24 hrs = 1.5 petabytes (or <span class="math-container">$10^{15}$</span> bytes) per day.</p>
  
  <p>But YouTube videos post are voluntary, and they are far from continuous video streams.</p>
  
  <p>There are about 3.5 billion smartphones in the world. If video was continuously streamed and recorded, going through the same video math above (<span class="math-container">$3.5 * 10^9 * 1.5 * 10^{15} * 24)$</span> = 126 yottabytes (or <span class="math-container">$10^{24}$</span> bytes) per day.</p>
  
  <p>The IDC projects there will be 175 zettabytes (or <span class="math-container">$10^{21}$</span> bytes) in 2025.</p>
  
  <p>Unless my math is very wrong, it would seem as though smartphone cameras alone could produce more data in one day than all of the data created in human history in 2025.</p>
  
  <p>This, so far, has only been about the data recording, but, to implement a surveillance state, all recorded data would need to be processed by AI to intelligent flag data that is significant. How much processing power would be needed to filter 126 yottabytes into relevant information?</p>
  
  <p>Overall, this question is motivated by the spread of dystopian surveillance media like Edward Snowden NSA whistle blowing leaks or George Orwell's sentiment of "Big Brother is Watching You".</p>
  
  <p>Computationally, could we be surveilled, and to what extent? I imagine text messages surveillance would be the easiest, does the world have the computation power to surveil all text messages? How about audio? or video?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-28T20:09:59.783Z
Comments.Text: Note that your 2.5GB/hour number is typically used for TV/movie video, which is (deliberately) very dynamic by nature.  Video from most other sources is often *much* more compressible (e.g. most surveillance cameras spend long periods with identical video frames, or large portions of the scene not changing, cell phones are often in somebody's pocket, etc).  Also, video doesn't have to be 1080p to be useful for surveillance.  A lot of surveillance video even from 1080p-capable cameras is actually sent/stored at much lower resolution because HD is just not needed for most surveillance purposes.
Comments.UserDisplayName: Foogod
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-28T21:26:15.297Z
Comments.Text: How is this question strictly related to AI? Maybe make it clearer!
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-29T20:30:53.617Z
Comments.Text: I think your question isn't really meaningful as asked.  You add up a bunch of numbers of how much data is produced and then ask if it could all be processed. Processed how?  You could use all the compute power of the universe to process 1 megabyte in infinite ways.  Obviously won't be doing that... The game is using available compute power *efficiently* (only in places and to the degree *useful* for some goal) on these large data sets.
Comments.UserDisplayName: user1169420
Tags: <social><computation>
Title: Is a dystopian surveillance state computationally possible?
UserDisplayName: Harrison Tran
ThreadText: # Is a dystopian surveillance state computationally possible?
  
  (Asked by: username_0 on 2020-02-28)
  
  This isn't really a conspiracy theory question. More of an inquire on the global computational power and data storage logistics question.
  
  Most recording instruments such as cameras and microphones are typically voluntary opt in devices, in that, they have to be activated before they start recording. What happens if all of these devices were permanently activated and started recording data to some distributed global data storage?
  
  There are 400 hours of video uploaded to YouTube every minute.
  
  Let’s do some very rough math.
  
  I’m going to assume for the rest of this post that the average video is 1080p which is 2.5GB (or $10^9$ bytes) per hour. From that, we get about 400 hrs \* 60 mins \* 2.5GB/hrs \* 24 hrs = 1.5 petabytes (or $10^{15}$ bytes) per day.
  
  But YouTube videos post are voluntary, and they are far from continuous video streams.
  
  There are about 3.5 billion smartphones in the world. If video was continuously streamed and recorded, going through the same video math above ($3.5 \* 10^9 \* 1.5 \* 10^{15} \* 24)$ = 126 yottabytes (or $10^{24}$ bytes) per day.
  
  The IDC projects there will be 175 zettabytes (or $10^{21}$ bytes) in 2025.
  
  Unless my math is very wrong, it would seem as though smartphone cameras alone could produce more data in one day than all of the data created in human history in 2025.
  
  This, so far, has only been about the data recording, but, to implement a surveillance state, all recorded data would need to be processed by AI to intelligent flag data that is significant. How much processing power would be needed to filter 126 yottabytes into relevant information?
  
  Overall, this question is motivated by the spread of dystopian surveillance media like Edward Snowden NSA whistle blowing leaks or George Orwell's sentiment of "Big Brother is Watching You".
  
  Computationally, could we be surveilled, and to what extent? I imagine text messages surveillance would be the easiest, does the world have the computation power to surveil all text messages? How about audio? or video?
  
  **Tags:** social, computation
  
  > **Comments:**
  > **username_1**: Note that your 2.5GB/hour number is typically used for TV/movie video, which is (deliberately) very dynamic by nature.  Video from most other sources is often *much* more compressible (e.g. most surveillance cameras spend long periods with identical video frames, or large portions of the scene not changing, cell phones are often in somebody's pocket, etc).  Also, video doesn't have to be 1080p to be useful for surveillance.  A lot of surveillance video even from 1080p-capable cameras is actually sent/stored at much lower resolution because HD is just not needed for most surveillance purposes.  
  > **username_2**: How is this question strictly related to AI? Maybe make it clearer!  
  > **username_3**: I think your question isn't really meaningful as asked.  You add up a bunch of numbers of how much data is produced and then ask if it could all be processed. Processed how?  You could use all the compute power of the universe to process 1 megabyte in infinite ways.  Obviously won't be doing that... The game is using available compute power *efficiently* (only in places and to the degree *useful* for some goal) on these large data sets.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2020-02-28. Score: 21
  
  You don't necessarily have to analyse it all. Just by having such data available you can achieve a lot in terms of surveillance, as long as you can retrieve relevant parts.
  
  A few years ago there was a [Radiolab podcast, "The Eye in the Sky"](https://www.wnycstudios.org/podcasts/radiolab/articles/eye-sky) (there's a full transcript on the site). The basic idea is that you have a plane circling a city 24/7, and filming what goes on. If there was a crime somewhere, you retrieve the recordings after the event, and you can track back to where vehicles involved in the crime were coming from, and where they went after the crime. If nothing happens, you simply archive the data, and perhaps remove it after a month or so.
  
  This method was used to solve a hit-and-run assassination of a police woman who was on her way to work. The gang who committed the attack were rather surprised when the police showed up at their secret hide-out a few days later, as they could see on the images where the cars involved went to later. At the time and place of the murder there were obviously no witnesses who could have done that. And this involved no computational processing at all.
  
  The possibilities this opens up are just scary, as you can track pretty much anybody's movements without actually needing someone to follow them. Add to that street-level CCTV, and not much can happen without you being able to find out.
  
  In this scenario there is no processing at all, but you could imaging simple processing steps, such as tracking vehicles or changes in the environment, which could be used to give clues about potentially 'interesting' events. So instead of using it 'passively' as a kind of memory, you could use that data to identify things that happened that you weren't aware of.
  
  And this is without even any clandestine access to people's data. If you add that dimension, then you might even be able to identify crimes/etc before they even happen. Text processing can be quite fast, but is not easy to do, as presumably few people would openly communicate about things they were planning. So I guess we're still a long way away from that.
  
  Of course there is the ethical dimension (which is mentioned in the podcast): who has access to that data, and who decides what it is used for? If you do, and you suspect your partner of being unfaithful, who/what would stop you from checking out their movements? Or check up on that politician who might have a secret affair, or a gambling issue, or who keeps being in the same locations as a well-known drug dealer. All rather scary.
  
  While a complete analysis of all such data would be very heavy computationally, and fraught with false positives and recall problems, it might simply be enough to index it by time, location, and perhaps people involved (face recognition seems to be reasonably good, though still with a rather high error rate). This is enough already to make me feel worried about the future.
  
  > **Comments:**
  > **username_5**: I also think worth adding to this good answer the one-trick item I was going to write: Edge processing & filtering. It is possible to reduce the amount of device-level data down a lot - orders of magnitude - by filtering only to active/interesting scenarios, e.g. there's no point videoing the inside of someone's pockets. Embedded AI assists with this by adding smarter filters, not just "is there a signal", but "is something interesting happening", which can decrease centralised bandwidth, CPU and storage requirements yet further.  
  > **username_6**: And if you're writing a book, you can have the protagonists buy 100 phones and hack them to return false data. (Someone did this recently to confuse Google Maps)  
  > **username_7**: @user253751 Not actually false data, he just loaded them on a cart, walked through his street, and Google Maps concluded there was a traffic jam because 100 "vehicles" were moving slowly. No hacking necessary.  
  > **username_6**: @username_7 It's false data because there weren't 100 vehicles moving slowly in a traffic jam.  
  > **username_7**: @user253751 The data are real, but Googles algorithms drew incorrect conclusions from real data.  
  
  ---
  
  ### Answer by username_8 on 2020-02-28. Score: 10
  
  You would also want to consider physical limitations. If you are even *storing* 126 yottabyte of data per day, then if we look at the current theoretical densest data storage medium, DNA, at 215 petabytes per gram, we get...
  ${(126 \* 10^{24}) \over (215 \* 10^{15})} = 586046511$ grams per day
  
  586046511 g = 586046 kg = 586 Metric Tonnes just for storage.
  
  > **Comments:**
  > **username_9**: to put that number in perspective, note that it's actually a bit less than the 695 metric tonnes of garbage produced daily by the US. (tI took numbers from : https://www.treehugger.com/environmental-policy/trash-numbers-startling-statistics-about-americans-and-their-garbage.html)  
  > **username_10**: that's very sad  
  
  ---
  
  ### Answer by username_11 on 2020-02-29. Score: 8
  
  The answer is really very simple. If you have the dystopian power over all the mobile devices in the first place, you would not make them send all their data over to any "global data storage" just like that. Instead, you would have put a local AI on each device that filters, processes, categorizes and flags the important parts, sending only those parts plus an intelligent summary of the remaining data to a global AI. The global AI combines and synthesizes the parts that all the local AIs send to it, and may request further data from the local AIs based on what it wants to know.
  
  Naturally, since you are a competent dystopia architect, you design each local AI to be intelligent enough to subvert any human's attempt to remove it, stop its activity, or otherwise interfere with its data collection and processing. The local AIs also continuously communicate in a distributed network with other local AIs regarding their status and any adversarial activities, so that they can quickly act to defend themselves if the need arises, and also notify the global AI of any attack. In this surveillance state, it is an easy task for the global AI to send armed agents to deal with any threat to the AI network that manages to gain any foothold in the information cyberspace.
  
  The point is that the most durable dystopia is a **defended distributed dystopia**, which would make it **robust** and **scalable**.
  
  
  ---
  
  