I've always found this whole "https everywhere" silly. Even when I had a web site with content on it, I didn't want to pay for a certificate, and for https hosting, and I had no content deserving of being encrypted end-to-end.
It doesn't secure anything to add https to sites which don't need it. That's just a cash grab.
Yikes. Thanks for raising awareness of this issue.
@Seth I. Rich. SSL Certificates are free (from letsencrypt) or from Cloudflare so cost isn't an issue. There are lots of advantages to using HTTPS and many disadvantages to not using it.
If you don't secure your site it can be a liability. Do you really want someone injecting scripts, images, or ad content onto your page so that it looks like you put them there? Or changing the words on your page? Or using your site to attack other sites? This stuff happens: on airlines (a lot, and again), in China, even ISPs do it (a lot). And HTTPS prevents all of it. It guarantees content integrity and the ability to detect tampering.
There's a third option on the horizon here: if we have signed HTTP exchanges, local caches become possible again if you're willing to give up privacy but not integrity. There are a couple of Internet-Drafts on this topic, but we also need to be able to tell the browsers to look in the cache before they try poking the origin server. And, of course, none of this will help anyone stuck browsing with IE5 on Windows ME.
Also, Amazon Web Services has its own TLS certificate service that is free and works only on its services.
insecure http was made for cooperatevly sharing information and it is excellent at it. Due to the transfer of money online becoming popular secure https became a good sane default for anyone using money online. Sadly this even further hurts those no-so-well off. There is no reason one should need to have recent crypto installed, an accurate clock, and a reliable connection in order to simply seek static information such that might increase quality of life. Sure, injections are possible, but an open letter delivered is far more useful than a sealed letter lost.
The first two are happening already; the third may be. If you're cool with that, OK. I certainly wouldn't be. I completely agree that this is a harsh penalty to impose on sites that are relatively static and simple or sites that are run by groups that don't have the technical ability to implement https.
HTTPS is a solution–not a good one, I'll grant, but I don't think we have a better one. Well, yes, we do: widespread adoption of IPSec and IPv6. But we can't wait for that.
There is also another solution to the caching issue, but it does affect the privacy of the users. Since you have access to the local machines, you can install your own certificate authority and man-in-the-middle your users with a caching proxy server. The server makes the connection, strips the security, caches the data, and then presents it to the users with using your own CA.
I believe that Apache and Squid can do this.
The above statement from Seth is not true. With services like Let's Encrypt. It is no longer a cash grab and provides people with free certificates with key rotation but I digress.
Harness cURL (or similar software) and download desired websites locally and host the content on a server within the local network.
I am not that well versed in Varnish but I expect it would be possible to set one up on a local network and route all traffic through it so that it returns the cached data to the local network.
Probably not the best solutions, however, the first one would definitely work in the mentioned use case with the second idea being a good candidate for locations with such low issues if they could get their hands on a Raspberry Pi.
Will HTTP/2 not help with this situation?
I can kind of appreciate whats happening, as a European I'm suddenly now not able to read most of the news sites in the US because of where I live! Also till recently I only had a 2mb pipe as I am in the countryside!
Ok so that is different but it is still debilitating!
I love HTTPS everywhere, and I was really thrilled when responsive web design became a thing.
But both developments have a common disadvantage: They exclude old, less capable software and hardware.
This problem seems to be a special case of HTTP's inherent centralisation flaw. Usenet or email wouldn't have had this problem.
So I guess one partial fix is similar. All CDNs have much of the content duplicated around the world. What if there was a dedicated scheme to have shared synchronised local CDN servers, on the network before the satellite hop? So the static data only needs to go over once.
I think Ryan mailed it, they will have to use a proxy server that is a MitM, it takes in https and provides back to the client. That's sounds dodgy, but it is how a lot of corporate networks function. Each client installs the cert from the proxy, and it can then proxy & cache https traffic. Ok so long as the users aren't expecting private collections, but in this case that appears ok.
The comments about whether to use https are a tangent, the web is going that way, we need to make sure the solutions to these sorts of problems are better known and easy to use.
We certainly need encrypted transfer – want to enter your credit card details on an open page? Thought not. The last time I had to do that it was being used within hours.
HTTPS doesn't give quite as much privacy as some people think – the domain name is sent in clear text, necessary because of name-based hosting, where a single server hosts multiple domains. Of course, the actual IP address is also sent in the clear so that intermediate equipment can route the packets, so it's not that huge a leak, but people forget about both of those facts.
But for sure I agree with you about the problems of performance. Pingdom says this Web page weighs in at only 258KBytes, which is well below average (sigh), and only 12 http(s) requests, loading in well under three seconds. My own Web page – https://www.fromoldbooks.org/ – is much worse (800KBytes, 58 requests, 2 seconds), although as a stock image site it's image-heavy.
Some of the worst performance culprits seem to be from Google AdSense and Analytics. Moving some of that into the client could make a big difference to the overall Web experience without compromising on privacy, and Brave has taken some steps in that direction – but at the cost of what some in the ad industry feel to be an unacceptable land-grab.
It'd be a good topic for a browser and Web server symposium or Workshop, to try & improve typical/average Web performance by, say, an order of magnitude in a world where privacy and security matter more than ever before.
This is a real problem, and as a firm HTTPS everywhere advocate, I feel it and know we must deal with it. But I think we can.
CDNs: Sounds like we need more CDN presence closer to Uganda.
BGP fanciness: Sites like Google and Wikipedia should/could get servers closer to Uganda, with better/shorter routes over better links.
Good MITM: The owner/administrator of the clients could choose to trust a cacheing MITM's certificate issuer.
Modern protocols: H/2 and QUIC aim to do more with fewer packets, reducing handshake overhead and the cost of packet loss in those critical early phases. I expect to see that increasingly pay off (for everyone).
Also, I can't imagine that large page-loads (unfortunately, the norm) perform well over satellite links even with HTTP. With latency and packet loss that high, TLS is not uniquely/solely a cause of long load times. I don't see if in your post you discuss what the pre-HTTPS load times were like.
Checking just now, I see CNN International is non-HTTPS, and is very large (1.2 MiB over 207 requests). There's no way that's fast; HTTPS is not the special problem there.
I also recommend that you run Pi-hole on your local network, as it will blackhole useless tracking and advertisement-related servers that would squander your precious (and slow) data.
I'm not sure if you can use Subresource Integrity to load assets over http from an https site. Integrity would be assured (no one could tamper with the assets) but some privacy would be lost (someone could view which assets you were downloading). In some cases, you could use the SHA to download the files from a local cache without needing to go to the source.
The solution for things like schools on satellite links in remote areas might be to use a caching proxy server that unwraps the HTTPs and serves it re-signed with it's own certificate. You'd have to manually install that certificate in the computers, and it might not work due to steps browsers have started taking to prevent that kind of thing being done maliciously.
There's a big exception to this and it's why the problem is off the radar for most people: you can configure a proxy in the browser and it works great. The only thing which broke is doing do without the consent of the client, which means that the people most affected are also the least prepared to deal with it.
Unfortunately, there's not really a great way to deal with this which doesn't open millions of people up to mass surveillance. Google's QUIC protocol is more tolerant of lackey loss but that is far from something you can rely on now. There are some thoughts about shared caching around SRI but those have significant privacy challenges.
I'm just about to manage Satellite connections serving remote communities for the first time — this has been an eye-opener!
You can set up a local caching proxy server which decrypts HTTPS and then re-encrypts it before sending down to the local PC's – It requires all the local PC's to trust the caching server by installing it's certificate as a Trusted Root authority, but without HTTPS you still need to trust the caching server, it's just not formalized.
Technically yes it's a man-in-the-middle but I'd expect the kids in ugandan schools would be happy to make that tradeoff. We're not all Edward Snowden.
How hard this is to do? I can't say as I haven't done it myself, but if it's difficult it seems to me like that is a worthy open source project, and potentially far more useful than service workers. Service workers cache, but only on that one PC, which isn't useful when you're trying to share bandwidth amongst a school full of students.
There's definitely a good case in creating proxy servers to cache content. I was considering a move to Rarotonga a few years back, and the internet there was expensive and quite limited, coming via satellite.
What's I'd consider doing is setting up a squid-cache with SSL-bump. That way you can still be pretty optimal in your use of the scare internet connection and mostly be loading cached resources.
Hey Eric. Thanks for highlighting this issue. QUIC, by Google, might offer an improvement here.
Thanks, Eric, for making me aware of this problem with HTTPS.
Could you elaborate how service workers could mitigate the problem? Assets downloaded over HTTPS can be locally cached by the browser just like with HTTP. What does caching via service workers offer that normal caching could not?
Apart from this, I think it is clear that in such a situation ad-blocking (ideally already at DNS… e.g. with special hosts file) is almost mandatory.
Just as a footnote, there are browsers with ServiceWorker support running on Windows XP – at least Opera (until v36) and also older Chrome versions (not sure about Vivaldi, but possibly too).
Isn't the problem for old systems that protocols like QUIC and HTTP/2 aren't going to be supported?
When it comes to supporting old and constrained connections, I would bet that the size of images and JS matter far more than the HTTPS overhead. I like the suggestion of keeping optional non-SSL domains around, but the challenge there is making sure users know they're available.
Also, with the recent(ish) deprecation of TLS 1.0, there are plenty of SSL configurations out there today that just prevent sites from working in old browsers. If you're using a configuration that relies on Server Name Indication (SNI), you might be blocking even more browsers. Unless I'm misunderstanding, it's basically impossible to serve the "most" secure configurations to only modern clients – you're basically shooting for a balance of "secure enough" and "accessible enough". It's a terrible trade off, to be sure. :( But I also can't fault people for making it.
In those cases, I think local caching servers make sense, but you also have to trust whoever is running that cache.
Is there any way for a server to detect early on that it's coming from a client with low bandwidth? If that's the case, couldn't you configure servers to default to serving smaller (maybe mobile) pages? (Similar to detecting the UA in a request…). I guess, you could have a framework that uses the UA to serve lighter pages to older clients, too, with the understanding that an old browser is likely to be running on old hardware.
The way we secure our communication does not have to be today's HTTPS, though. Encryption is needed just sometimes. On public static sites, it can kind of obscure what are you looking at (e.g., attacker sees you are looking at Wikipedia, but it is not clear what page), but traffic volume analysis can often distinguish between specific pages.
Let's look at some options to make it better. There will be some tradeoffs to privacy, but we will not let attackers to affect traffic in an arbitrary way, as plain HTTP would allow. Thus, we would not make the user more prone to downgrade attacks than with today's HTTPS. Our main point is allowing the caches doing their jobs, maybe a better one than with the current state of the art HTTP caches can do.
Caching proxy: Hey, I have some content for this server cached. See my non-expired approval from the server, signed by the private key of certificate holder. I am allowed to serve you some of the requests. Plus there is the OCSP response, so you know the server's certificate is not revoked. You see, the private key holder indicates there is nothing sensitive in the URL, you can send it to me.
If client or server does not support such a feature, either just because it is not implemented or because they don't want this for a reason, no other party can force the communication to go this way instead of standard TLS.
If the browser chooses not to use this way (e.g., because of user's decision), it can insist on a standard TLS handshake.
None of those suggestions is enough reviewed by others. Furthermore, description of signed caches is quite vague to properly review, because I have prefered to be concise. While I have some security and crypto background, I don't think this should be implemented without any review.
Any change in browsers is likely irelevant for people with Windows XP or something similar. On the other hand, they could be welcome anyway if their usage don't break anything.
If you're using service workers only for the content caching, is that effectively any different to just using cache headers to instruct browsers to cache static content?
Either way the file is only cached internally for the browser (not on the local network). From what I can tell browsers still cache files served over HTTPS even if proxy servers can't.
Any widely adopted solution going forward should not and will not compromise integrity assurance. Meanwhile, Service Workers have high overhead and further increase reliance on scripting. My hope lies in the Peer Web bringing federation of static content and allowing for decentralised infrastructure. Specifically, Dat Protocol is an easy way to allow for p2p propagation of your web content. This would make community internet projects resilient and truly independent.
Just like Orion Edwards, Uwe Trenkner and Mike Healy commented, there's something that doesn't smell right to me about Services Workers.
If you're trying to setup a shared cache, I don't think service workers will be the right choice. From what I understand, they're a mechanism for managing a local cache and custom behaviors on the client, especially useful in case of no or bad connectivity; moreover they only work on a single user agent (a single browser on a single computer) therefore the advantages cannot be shared across a classroom. For this use case, I don't see them improving the situation much from what properly set HTTP caching headers already offer.
Instead, what comes to mind is IPFS which is a secure way to share resource locally (peer-to-peer) in a secure manner. Maybe this sort of use case is the killer-feature of IPFS. Worth looking into!
Cloudflare's Keyless SSL project enables caching whilst maintaining the integrity of the certificate served. So (if my theory is right… bear with me) you could install it on the local cache server as a way of ensuring HTTPS is still present, whilst allowing local caching (I think?).
Use it in conjunction with a SOCKS proxy.
Your point about packet loss got me thinking about HTTP2.
We've been advised to send a larger number of smaller packets with HTTP/2 (as opposed to a small number of larger packets with HTTP/1), but does this negatively impact performance when using satellite internet access like in the example you gave?
See the section "Protecting the oracle" for why keyless SSL won't help in this scenario.


My family had a barbecue today and I was not sure I wanted to go. I asked my dad how my son was doing and he said he was okay, so we had lunch. I went outside with my brother and sister in law and we were talking about this, that and the other. My brother said that we all knew it would happen.
That is when I was told he is in prison for breaking and entering. I think the judge got tired of seeing him; he had his chances and the last Chance was 15 years suspended. I will not put on here what the charges were.
I got a little angry when I was told that it happened a month ago.
People are starting to understand when I say I do not feel like a family member.
I'm sorry to hear about your son; it must be very challenging to deal with his misdemeanours and the stress involved cannot be doing your scleroderma symptoms any good.
I do hope that you are able to cope with the comments from your family.
Thank you. The comments from my family are nothing new so I just close it out. I no longer go there alone and my aunt has learned the truth. And my therapist has done a lot to help me sort all of the rubbish out.
This all started as a child, I now live 70 miles away from my family.
When my mom passed away I thought it would quit, but it didn't.
I have a new therapist; she is great. I will call her Monday and Wednesday. When I go to see my sclero doctor my aunt will be my driver; we have a good time, I always look forward to the drive.
I always tell other people with scleroderma or other diseases to see a therapist; family and friends can blow up in your face simply because a lack of understanding.
I send my blessings to everyone.


text:
From left to right: Viviana Wilkins, Meaghan Carfrey, Nikki Gonzalez
SDSU Team Named Finalists in National Hotel Investment Pitch Competition
Three students from the L. Robert Payne School of Hospitality and Tourism Management examined the finances and feasibility of a project for the "She Has a Deal" competition.
By Cristian Alvarez
Story Keywords: Professional Studies and Fine Arts, Students, Hospitality and Tourism, Student Success, News
An all-female finance team consisting of students in San Diego State University's L. Robert Payne School of Hospitality and Tourism Management (HTM) recently was named a finalist in the annual "She Has a Deal" competition. The nationwide hotel investment education and pitch event serves to help women entering the field of hospitality and tourism in creating a pathway to make hotel development and ownership more obtainable.
The SDSU team — students Nikki Gonzalez, Viviana Wilkins and Meaghan Carfrey — was among three finalists of the competition along with groups representing New York University and Cornell University.
Each year, participants are asked to pitch their ideas for a future hotel development project site. The teams are managed by a panel of judges who are current professionals in the field. Every group is judged based on the financial feasibility, rigor analysis and persuasive ability to invest towards the project. SDSU's team pitched their project towards a potential Hyatt Place Hotel in San Jose.
The winning team earns $50,000 to be utilized as an equity investment for a hotel project. Members of the winning group also receive asset benefits of full ownership, which includes annual cash flows and capital gains, and have the option to participate in the investment process, obtain a role or shadow and be mentored by the founder of TLTsolutions throughout the project's life cycle. In addition, winners receive complimentary registration to attend the Americas Lodging Investment Summit and Asian American Hotel Owners Association Convention.
Tracy Prigmore created the competition in 2019, inspired to help women overcome the barriers within a traditionally male dominated industry.
Gonzalez, who spent all four years a part of the HTM program, said this competition allowed her to get a glimpse into the development and ownership side of hospitality.
"The 'She Has a Deal' competition allowed me to realize that I am more capable of pursuing a career within this area and experience the incredible support of key industry leaders who also believe in this cause," Gonzalez said.
Wilkins mentioned how working alongside Gonzalez and Carfrey has been a meaningful experience while attending SDSU. "We are all hard-working individuals, passionate about the industry and eager to learn new things. Each of us have a unique background and different skills that contribute towards our pitch deck for the competition," she said.
"We never took this competition lightly. We'd spend long days together making sure we understood and got everything for the pitch deck perfect. We didn't have a clue what we were getting ourselves into, but we learned so much and supported each other throughout," Wilkins said. "I was so happy the creators spent time teaching webinars to prepare us. It was such a unique learning experience and inspiring to see how many strong, intelligent women participated."
The competition took place at the Hilton McLean in McLean, Virginia in late March. The finals were scheduled to take place in Washington, D.C. this spring, but have been postponed due to the ongoing COVID-19 pandemic.
American Indian Recruitment Programs Foster College Success
SDSU Designated a First-gen Forward Advisory Institution
Summer Program to Provide Early SDSU Connection to Black Students
2021 Quest for the Best Winners Named
Women in Business Mean Business
The Floor is Hers: A Story of Perseverance
Black Business Society Keeps Eyes on Path to Success
Finding Community Within the Community
Joy in Giving Back
SDSU Student Gets Real About Entrepreneurship
New Faculty Scholar Hits the Ground Running
SDSU Recognized for Excellence in Providing Virtual Student Support

redpajama_set_name: RedPajamaCommonCrawl


The foreign institutional investors might stay away for a bit due to political uncertainties and the time of entry will still be uncertain, Naveen Kulkarni, HoR, Reliance Securities, tells ET Now. Sonata SoftwareNSE -2.28 %, Cyient, Majesco are the top picks among IT midcaps and DCB among midcap private banks.
What is the expectation from Eicher MotorsNSE -0.45 %? We are expecting the margins to contract. The launch cost and increased input cost could hurt the numbers. Is there anything that you are factoring in?
This is a stock that we currently do not have under our coverage but overall from the automobile space, we are not positive on the two-wheeler category as a whole. We are slightly more positive or optimistic on the four-wheeler category. But we see a lot of challenges with regards to channel inventory movement. Second, competitive intensity is continuing to rise. Demand also is not that great. There are much more challenges on the two-wheeler side rather than the four-wheeler side. We are not very optimistic at this juncture on the auto space especially on the two-wheelers.
Would there be a similar view when it comes to passenger vehicles as well? Across categories, one has the impression that other than Tata MotorsNSE 1.29 % because of JLR, even a domestic market leader like Maruti has been hurting on volumes?
Maruti's play is slightly better because the channel inventory is not so high and has been on a decline. There is pressure both on the margin front as well as on the demand scenario. It will take a while for the base to become a little more favourable for the auto companies. That is when optically the growth will start improving, but the real growth rate also has to improve. That will probably take around three to six months. We will be in a little bit of a wait-and-watch mode for even the four-wheeler passenger vehicles. We are quite negative on the commercial vehicle (CV) play. I would say that among passenger vehicles, Maruti is slightly better placed, but even then, we will be in a wait and watch mode.
Let us talk about the overall liquidity. While quite a bit of support came in from the DIIs, FIIs have definitely been steering clear of Indian markets. Do you think that with the big election event around the corner, that kind of a position is likely to continue for the foreign investors?
Yes, it seems more likely the foreign institutional investors may not invest so much, but again it is very difficult to gauge when foreign institutional investors would start looking into the Indian market as a whole.
The global liquidity situation has been tightening for a while and I would believe that this year might be year of little bit of respite but it remains to be seen when that scenario actually changes. To be specific on the India front, I would believe that foreign institutional investors might stay away but it will depend on what the news flow will be on the political front. In a couple of months, we might start getting some of that opinion polls coming through, where if the political scenario emerges as a little more constructive, then we might start seeing some bit of flows coming through.
It is going to be a tricky situation because the news flow might continue to remain volatile. From that perspective, for now, the foreign institutional investors might stay away for a bit and the time of entry will still be uncertain.
Let us understand the outlook as well on the sectors that you are bullish on. What is the earnings growth that you are expecting from IT? Which are the stocks that you like within the space?
IT again has been our favoured sector for a while now. We like both the largecaps and midcaps. From an earnings perspective, we are looking at 12-14% kind of an earnings growth coming through for next year. The stocks that we like in the largecap space are TCS and Infosys. Infosys has been our top pick in the largecap IT space. The stock has done quite well now and so that continues to be one of our bullish picks.
In the midcap space, we like Sonata Software. The results that the company reported last week were pretty decent. The earnings growth has been good. We are looking at an 18% kind of CAGR for the next two years. The earnings growth is likely to remain pretty stable and good for Sonata. So, that is the stock that we like in the midcap space for IT.
Across the midcap category, anything else that you like beside the stocks that you just talked about?
I would say that the IT midcap space looks attractive. The only challenge with the midcap IT space is that the earnings can be a little volatile. In some quarters, there might be misses but what we are seeing right now is a gradual improvement in earnings for the midcap stocks also. So, Sonata Software is the top pick in the midcap IT space. Apart from Sonata, something like Cyient might do better in the forthcoming quarters. We also like Majesco. Overall, the midcap space looks quite interesting to us at this point in time and earnings growth is more likely to pick up in the midcap space from here on.
Within the private banking space, what is catching your eye among the corporate retail lenders ?
Banking as a space has been one of our favoured allocation area. I have been talking about Axis and ICICI BankNSE -1.42 % for a fairly long period and Axis has done pretty well in the last couple of months.
We even continue to like ICICI Bank at current levels, I believe the stock is quite reasonably valued. There is a lot of room for the stock to rerate even from the current levels. The overall operating performance has been pretty good. So ICICI Bank is another name that we like in the private banking space. Among the midcap names what we like is DCB. That is another stock that we like in the midcap space from the banking sector.


Owing to our expertise, we have been able to manufacture, trade and supply our clients a superior range of Neoprene Sheet, which is highly applicable in various sectors such as food industry, refineries, chemical and process units. Manufactured using neoprene rubber and with help of latest techniques under the guidance of experts, this sheet is highly demanded in the market. Reckoned for smooth texture, high strength, light weight, oil & abrasion resistance and durability, this Neoprene Sheet is available in various lengths and thicknesses.


None of the contaminated wheat gluten that led to the US recall of pet food went to manufacturers of food for humans, according to the ingredient's importer.
The Chinese wheat gluten imported by ChemNutra Inc. all went to companies that make pet foods, Stephen Miller, chief executive officer said. Miller declined to identify what companies ChemNutra supplied. Nearly 100 brands of cat and dog foods made with the ingredient, since found to be chemically contaminated, have been recalled.
FDA testing of the wheat gluten has revealed it was contaminated with melamine, a chemical with a variety of industrial uses, including the manufacture of plastic kitchenware.
ChemNutra said it has recalled 792 tonnes of wheat gluten that it shipped to three pet food makers and a single distributor who in turn supplies the pet food industry. The company said the recall applied only to wheat gluten from Xuzhou Anying, one of its three Chinese suppliers of the ingredient.
The importer shipped the product in 25-kilogram paper bags between Nov. 9 and March 8, when it learned the ingredient was suspected as the cause of the pet food problems. ChemNutra said it then quarantined its wheat gluten inventory. Each bag of wheat gluten included content analysis and test results provided by Xuzhou Anying, ChemNutra said.
"The company is particularly troubled that the certificates of analysis provided by the above-named supplier did not report the presence of melamine," ChemNutra said in a statement.


San Jose Medal pendant is available in sterling silver, 14k gold-filled, solid 14k gold, 14k white gold, 18k gold. If you order this item without a chain, a jump-ring is included so you can attach it to your own chain. You can easily add a chain you like from a wide selection of chains available from our "Necklaces & Chains" selection. Chained medals will come gift-boxed.
San Cristobal Medal 1" San Jose Medal 3/4"


Please use this link to purchase locally.
1 pound of our honey in a elegant gift bottle Amazon. Same Honey but purchase via Amazon.
STOP! WAIT this is listed on Amazon not locally via my e-commerce. Amazon charges me a premium please select the local product if you can. I understand you may have Amazon gift dollars that you need to use up. Thank you for understanding!


text:
« Message of the Day: Human Rights, Personal Growth, Economic Opportunity, War, Environment, Disease, Hunger, Population
"Hong Kong tycoon Jimmy Lai sentenced over banned Tiananmen vigil", Reuters »
Issue of the Week: Human Rights, Personal Growth, Economic Opportunity, War, Environment, Disease, Hunger, Population
The Biggest Event In Human History, BBC Radio 4, 1 December 2021
If this doesn't get your attention, what will?
The Biggest Event In Human History.
That's the title of the most recent Reith Lecture on the BBC World Service.
Its the first lecture in a series titled, Living with Artificial Intelligence.
So, the big picture of the topic is obvious.
We'll leave the details to the lecturer. Not the first, last or only word on the subject, to be sure. But one that is current as we write, along with its first follow-up, with two additional follow-ups coming momentarily, in a distinguished context.
You can access them here and listen or read as you please.
The transcript of the first one follows:
Downloaded from www.bbc.co.uk/radio4
THIS TRANSCRIPT WAS TYPED FROM A RECORDING AND NOT COPIES FROM AN ORIGINAL SCRIPT. BECAUSE OF THE RISK OF MISHEARING AND THE DIFFICULTY IN SOME CASES OF IDENTIFYING INDIVIDUAL SPEAKERS, THE BBC CANNOT VOUCH FOR ITS COMPLETE ACCURACY.
BBC REITH LECTURES 2021 – LIVING WITH ARTIFICIAL INTELLIGENCE
With Stuart Russell, Professor of Computer Science and founder of the Center for Human-Compatible Artificial Intelligence at the University of California, Berkeley
Lecture 1: The Biggest Event in Human History
ANITA ANAND: Welcome to the 2021 BBC Reith Lectures. We're at the British Library in the heart of London and as well as housing more than 14 million books, we are also home here to the Alan Turing Institute, the national centre for data science and artificial intelligence. Set up in 2015 it was, of course, named after the famous English mathematician, one of the key figures in breaking the Nazi enigma code therefore saving countless lives. We couldn't really think of a better venue to place this year's Reith Lectures, which will explore the role of artificial intelligence and what it means for the way we live our lives.
Our lecturer has called the development of artificial intelligence "the most profound change in human history," so we've given him four programmes to explain why. He's going to be addressing our fears. He's going to be explaining the likely impact on jobs and the economy and, hopefully, he will answer the most important question of all: who is ultimately going to be in control, is it us or is it the machines?
Let's meet him now. Please welcome the 2021 BBC Reith Lecturer, Professor Stuart Russell.
(AUDIENCE APPLAUSE)
ANITA ANAND: Stuart, it's wonderful that we're going to be hearing from you. I just wonder, actually, when you first became aware of artificial intelligence because for many of us our introduction would have been through sci-fi, so at what point did you think, actually, this will be a real-life career for me?
STUART RUSSELL: So I think it was when my grandmother bought me one of the first programmable calculators, the Sinclair Cambridge programmable, a little tiny, white calculator, and once I understood that you could actually get it to do things by writing these programs, I just wanted to make it intelligent. But if you've ever had one of those calculators you know that there's only 36 keystrokes that you can put in the program, and you can do various things, you can calculate square roots and signs and logs, but you couldn't really make it intelligent with that much. So, I ended up actually borrowing the giant supercomputer at Imperial College, the CDC 6600, which was about as big as this room and far less powerful than what's on your cell phone today.
ANITA ANAND: Well, I mean, this obviously marks you out as very different to the rest of us. We were all writing rude words and turning our calculator upside down.
Can we even measure how fast AI is developing?
STUART RUSSELL: I actually think it's very difficult. Machines don't have an IQ. This is a common mistake that some commentators make is to predict that machine IQ will exceed human IQ on some given date, but if you think about it, so AlphaGo, which is this amazing Go-Playing program that was developed just across the road, is able to beat the human world champion at playing Go but it can't remember anything, and then the Google search engine remembers everything, but it can't plan its way out of a paper bag. So, to talk about the IQ of a machine doesn't make sense.
Humans, when they have a high IQ, typically can do lots of different things. They can play games and remember things, and so it sort of makes sense. Even for humans there's not a particularly good way of describing intelligence, but for machines it makes no sense at all. So, we see big progress on particular tasks. Machine translation, for example, speech recognition is another one, recognising objects and images, these were things that we, in AI, have been trying to do for 50 or 60 years, and in the last 10 years we've actually pretty much solved them. That makes you think that the problems are not insolvable, and we can actually knock them over one by one.
ANITA ANAND: Well, I'm really looking forward to your first lecture. It is entitled The Biggest Event in Human History. Stuart, over to you.
STUART RUSSELL: Thank you, Anita. Thank you to the audience for being here. Thank you to the BBC for inviting me. It really is an enormous and a unique honour to give these lectures. We are at the Alan Turing Institute, named for this man who is now on the 50-pound note. The BBC couldn't afford a real one, so I printed out a fake one.
In 1936, in his early twenties, Turing wrote a paper describing two new kinds of mathematical objects, machines and programs. They turned out to be the most powerful ever found, even more so than numbers themselves. In the last few decades those mathematical objects have created eight of the 10 most valuable companies in the world and dramatically changed human lives. Their future impact through AI may be far greater.
Turing's 1950 paper "Computing Machinery and Intelligence" is at least as famous as his 1936 paper. It introduced many of the core ideas of AI, including machine learning. It proposed what we now call the Turing Test as a thought experiment, and it demolished several standard objections to the very possibility of machine intelligence.
Perhaps less well known are two lectures he gave in 1951. One was on the BBC's Third Programme, but this is going out on Radio 4 and the World Service, so I'll quote the other one, given to a learned society in Manchester. He said:
"Once the machine thinking method had started, it would not take long to outstrip our feeble powers. At some stage therefore we should have to expect the machines to take control."
Let me repeat that: "At some stage therefore we should have to expect the machines to take control."
I must confess that for most of my career I didn't lose much sleep over this issue, and I was not even aware, until a few years ago, that Turing himself had mentioned it.
I did include a section in my textbook, written with Peter Norvig in 1994, on the subject of "What if we succeed?" but it was cautiously optimistic. In subsequent years, the alarm was raised more frequently, but mostly from outside AI.
But by 2013, with the benefit of some time to think during a sabbatical in Paris, I became convinced that the issue not only belonged in the mainstream but was possibly the most important question that we faced. I gave a talk at the Dulwich Picture Gallery in which I stated that:
"Success would be the biggest event in human history and perhaps the last event in human history."
A few months later, in April 2014, I was at a conference in Iceland, and I got a call from National Public Radio asking if they could interview me about the new film Transcendence. It wasn't playing in Iceland, but I was flying to Boston the next day, so I went straight from the airport to the nearest cinema.
I sat in the second row and watched as a Berkeley AI professor, possibly me, played by Johnny Depp, naturally, was gunned down by activists worried about, of all things, super-intelligent AI. Perhaps this was a call from the Department of Magical Coincidences? Before Johnny Depp's character dies, his mind is uploaded to a quantum supercomputer and soon outruns human capabilities, threatening to take over the world.
A few days later, a review of Transcendence appeared in the Huffington Post, which I co-authored along with physicists Max Tegmark, Frank Wilczek, and Stephen Hawking. It included the sentence from my Dulwich talk about the biggest event in human history. From then on, I would be publicly committed to the view that success for my field would pose a risk to my own species.
Now, I've been talking about "success in AI," but what does that mean? To answer, I'll have to explain what AI is actually trying to do. Obviously, it's about making machines intelligent, but what does that mean?
To answer this question, the field of AI borrowed what was, in the 1950s, a widely accepted and constructive definition of human intelligence:
"Humans are intelligent to the extent that our actions can be expected to achieve our objectives."
All those other characteristics of intelligence; perceiving, thinking, learning, inventing, listening to lectures, and so on, can be understood through their contributions to our ability to act successfully.
Now, this equating of intelligence with the achievement of objectives has a long history. For example, Aristotle wrote:
"We deliberate not about ends, but about means. A doctor does not deliberate whether he shall heal, nor an orator whether he shall persuade. They assume the end and consider how and by what means it is attained, and if it seems easily and best produced thereby."
And then, between the sixteenth and twentieth centuries, almost entirely for the purpose of analysing gambling games, mathematicians refined this deterministic view of "means achieving ends" to allow for uncertainty about the outcomes of actions and to accommodate the interactions of multiple decision- making entities, and these efforts culminated in the work of von Neumann and Morgenstern on an axiomatic basis for rationality, published in 1944.
From the very beginnings of AI, intelligence in machines has been defined in the same way:
"Machines are intelligent to the extent that their actions can be expected to achieve their objectives."
But because machines, unlike humans, have no objectives of their own, we give them objectives to achieve. In other words, we build objective-achieving machines, we feed objectives into them, or we specialise them for particular objectives, and off they go. The same general plan applies in control theory, in statistics, in operations research, and in economics. In other words, it underlies a good part of the 20th century's technological progress. It's so pervasive, I'll call it the "standard model."
Operating within this model, AI has achieved many breakthroughs over the past seven decades. Just thinking of intelligence as computation led to a revolution in psychology and a new kind of theory, programs instead of simple mathematical laws. It also led to a new definition of rationality that reflects the finite computational powers of any real entity, whether artificial or human.
AI also developed symbolic computation, that is, computing with symbols representing objects such as chess pieces or aeroplanes, instead of the purely numerical calculations that had defined computing since the seventeenth century.
Also following Turing's suggestion from 1950, we developed machines that learn, that is they improve their achievement of objectives through experience. The first successful learning program was demonstrated on television in 1956. Arthur Samuel's draughts-playing program had learned to beat its own creator, and that program was the progenitor of Deepmind's AlphaGo, which taught itself to beat the human world champion in 2017.
Then in the sixties and seventies, systems for logical reasoning and planning were developed, and they were embodied to create autonomous mobile robots. Logic programming and rule-based expert systems supported some of the first commercial applications of AI in the early eighties, creating an immense explosion of interest in the US and Japan. The first self-driving Mercedes drove on the autobahn in 1987. Britain, on the other hand, had to play catch-up, having stopped nearly all AI research in the early seventies.
Then, in the 1990s, AI developed new methods for representing and reasoning about probabilities and about causality in complex systems, and those methods have spread to nearly every area of science.
Over the last decade, so-called deep learning systems appear to have learned to recognise human speech very well; to recognise objects in images, to translate between hundreds of different human languages. In fact, I use machine translation every year because I'm still paying taxes in France. It does a perfect job of translating quite impenetrable French tax instructions into equally impenetrable English tax instructions. Despite this setback, AI is increasingly important in the economy, running everything from search engines to autonomous delivery planes.
But as AI moves into the real world, it collides with Francis Bacon's observation from The Wisdom of the Ancients in 1609:
"The mechanical arts may be turned either way and serve as well for the cure as for the hurt."
"The hurt," with AI, includes racial and gender bias, disinformation, deepfakes, and cybercrime. And as Bacon also noted:
"Out of the same fountain come instruments of death."
Algorithms that can decide to kill human beings, and have the physical means to do so, are already for sale. I'll explain in the next lecture why this is a huge mistake. It's not because of killer robots taking over the world; it's simply because computers are very good at doing the same thing millions of times over.
All of these risks that I've talked about come from simple, narrow, application-specific algorithms. But let's not mince words. The goal of AI is and always has been general-purpose AI: that is, machines that can quickly learn to perform well across the full range of tasks that humans can perform. And one
must acknowledge that a species capable of inventing both the gravitational wave detector and the Eurovision song contest exhibits a great deal of generality.
Inevitably, general-purpose AI systems would far exceed human capabilities in many important dimensions. This would be an inflection point for civilisation.
I want to be clear that we are a long way from achieving general-purpose AI. Furthermore, we cannot predict its arrival based on the growth of data and computing power. Running stupid algorithms on faster and faster machines just gives you the wrong answer more quickly. Also, I think it's highly unlikely that the present obsession with deep learning will yield the progress its adherents imagine. Several conceptual breakthroughs are still needed, and those are very hard to predict.
In fact, the last time we invented a civilisation-ending technology, we got it completely wrong. On September 11, 1933, at a meeting in Leicester, Lord Rutherford, who was the leading nuclear physicist of that era, was asked if, in 25 or 30-years' time, we might unlock the energy of the atom. His answer was:
"Anyone who looks for a source of power in the transformation of the atoms is talking moonshine."
The next morning, Leo Szilard, a Hungarian physicist and refugee who was staying at the old Imperial Hotel on Russell Square, 10 minutes' walk from here, read about Rutherford's speech in The Times. He went for a walk and invented the neutron-induced nuclear chain reaction. The problem of liberating atomic energy went from "impossible" to essentially solved in less than twenty-four hours.
The moral of this story is that betting against human ingenuity is foolhardy, particularly when our future is at stake. Now, because we need multiple breakthroughs and not just one, I don't think I'm falling into Rutherford's trap if I say that it's quite unlikely we'll succeed in the next few years. It seems prudent, nonetheless, to prepare for the eventuality.
If all goes well, it will herald a golden age for humanity. Our civilisation is the result of our intelligence; and having access to much greater intelligence could enable a much better civilisation.
One rather prosaic goal is to use general-purpose AI to do what we already know how to do more effectively, at far less cost, and at far greater scale. We could, thereby, raise the living standard of everyone on Earth, in a sustainable
way, to a respectable level. That amounts to a roughly tenfold increase in global GDP. The cash equivalent, or the net present value as economists call it, of the increased income stream is about 10 quadrillion pounds or $14 quadrillion. All of the huge investments happening in AI are just a rounding error in comparison.
If 10 quadrillion pounds doesn't sound very concrete, let me try to make this more concrete by looking back at what happened with transportation. If you wanted to go from London to Australia in the 17th century, it would have been a huge project costing the equivalent of billions of pounds, requiring years of planning and hundreds of people, and you'd probably be dead before you got there. Now we are used to the idea of transportation as a service or TaaS. If you need to be in Melbourne tomorrow, you take out your phone, you go tap-tap-tap, spend a relatively tiny amount of money, and you're there, although they won't let you in.
General-purpose AI would be everything as a service, or XaaS. There would be no need for armies of specialists in different disciplines, organised into hierarchies of contractors and subcontractors, to carry out a project. All embodiments of general-purpose AI would have access to all the knowledge and skills of the human race. In principle, politics and economics aside, everyone could have at their disposal an entire organisation composed of software agents and physical robots, capable of designing and building bridges, manufacturing new robots, improving crop yields, cooking dinner for a hundred guests, separating the paper and the plastic, running an election, or teaching a child to read. It is the generality of general-purpose AI that makes this possible.
Now that's all fine if everything goes well. Although, as I will discuss in the third lecture, there is the question of what's left for us humans to do.
On the other hand, as Alan Turing warned, in creating general-purpose AI, we create entities far more powerful than humans. How do we ensure that they never, ever have power over us? After all, it is our intelligence, individual and collective, that gives us power over the world and over all other species.
Turing's warning actually ends as follows:
"At some stage therefore, we should have to expect the machines to take control in the way that is mentioned in Samuel Butler's Erewhon."
Butler's book describes a society in which machines are banned, precisely because of the prospect of subjugation. His prose is very 1872:
"Are we not ourselves creating our successors in the supremacy of the Earth? In the course of ages, we shall find ourselves the inferior race. Our bondage will steal upon us noiselessly and by imperceptible approaches."
Is that the end of the story, the last event in human history? Surely, we need to understand why making AI better and better makes the outcome for humanity worse and worse. Perhaps if we do understand, we can find another way.
Many films such as Terminator and Ex Machina would have you believe that spooky emergent consciousness is the problem. If we can just prevent it, then the spontaneous desire for world domination and the hatred of humans can't happen. There are at least two problems with this.
First, no one has any idea how to create, prevent, or even detect consciousness in machines or, for that matter, in functioning humans.
Second, it has absolutely nothing to do with it. Suppose I give you a program and ask, "Does this program present a threat to humanity?" You analyse the code and indeed, when run, it will form and carry out a plan to destroy humanity, just as a chess program forms and carries out a plan to defeat its opponent. Now suppose I tell you that the code, when run, also creates a form of machine consciousness. Will that change your prediction? No, not at all. It makes absolutely no difference. It's competence, not consciousness, that matters.
To understand the real problem with making AI better, we have to examine the very foundations of AI, the "standard model" which says that:
For example, you tell a self-driving car, "Take me to Heathrow," and the car adopts the destination as its objective. It's not something that the AI system figures out for itself; it's something that we specify. This is how we build all AI systems today.
Now the problem is that when we start moving out of the lab and into the real world, we find that we are unable to specify these objectives completely and correctly. In fact, defining the other objectives of self-driving cars, such as how to balance speed, passenger safety, sheep safety, legality, comfort, politeness, has turned out to be extraordinarily difficult.
This should not be a surprise. We've known it for thousands of years. For example, in the ancient Greek legend, King Midas asked the gods that everything he touch should turn to gold. This was the objective he specified, and the gods granted his objective. They are the AI in this case. And of course, his food, his drink, and his family all turn to gold, and he dies in misery and starvation.
We see the same plot in the Sorcerer's Apprentice by Goethe, where the apprentice asks the brooms to help him fetch water, without saying how much water. He tries to chop the brooms into pieces, but they've been given their objective, so all the pieces multiply and keep fetching water.
And then there are the genies who grant you three wishes. And what is your third wish? It's always, "Please undo the first two wishes because I've ruined the world."
Talking of ruining the world, let's look at social media content-selection algorithms, the ones that choose items for your newsfeed or the next video to watch. They aren't particularly intelligent, but they have more power over people's cognitive intake than any dictator in history.
The algorithm's objective is usually to maximise click-through, that is, the probability that the user clicks on presented items. The designers thought, perhaps, that the algorithm would learn to send items that the user likes, but the algorithm had other ideas.
Like any rational entity, it learns how to modify the state of its environment, in this case the user's mind, in order to maximise its own reward, by making the user more predictable. A more predictable human can be fed items that they are more likely to click on, thereby generating more revenue. Users with more extreme preferences seem to be more predictable. And now we see the consequences of growing extremism all over the world.
As I said, these algorithms are not very intelligent. They don't even know that humans exist or have minds. More sophisticated algorithms could be far more effective in their manipulations. Unlike the magic brooms, these simple algorithms cannot even protect themselves, but fortunately they have corporations for that.
In fact, some authors have argued that corporations themselves already act as super-intelligent machines. They have human components, but they operate as profit-maximising algorithms.
The ones that have been creating global heating for the last hundred years have certainly outsmarted the human race, and we seem unable to interfere with their operation. Again, the objective here, profit neglecting externalities, is the wrong one.
Incidentally, blaming an optimising machine for optimising the objective that you gave it is daft. It's like blaming the other team for scoring against England in the World Cup. We're the ones who wrote the rules. Instead of complaining, we should rewrite the rules so it can't happen.
What we see from these lessons is that with the standard model and mis- specified objectives, "better" AI systems or better soccer teams produce worse outcomes. A more capable AI system will make a much bigger mess of the world in order to achieve its incorrectly specified objective, and, like the brooms, it will do a much better job of blocking human attempts to interfere.
And so, in a sense we're setting up a chess match between ourselves and the machines, with the fate of the world as the prize. You don't want to be in that chess match.
Earlier Anita asked me, "Does everyone in AI agree with me?" Amazingly, not, or at least not yet. For some reason, they can be quite defensive about it. There are many counterarguments, some so embarrassing it would be unkind to repeat them.
For example, it's often said that we needn't put in objectives such as self- preservation and world domination. But remember the brooms: the apprentice's spell doesn't mention self-preservation, but self-preservation is a logical necessity for pursuing almost any objective, so the brooms preserve themselves and even multiply themselves in order to fetch water.
Then there's the Mark Zuckerberg–Elon Musk "smackdown" that was so eagerly reported in the press. Elon Musk had drawn the analogy between creating super-intelligent AI and "summoning the demon."
Mark Zuckerberg replied, "If you're arguing against AI, then you're arguing against safer cars and being able to diagnose people when they're sick." Of course, Elon Musk isn't arguing against AI. He's arguing against uncontrollable AI.
If a nuclear engineer wants to prevent the uncontrolled nuclear reactions
that we saw at Chernobyl, we don't say she's "arguing against electricity." It's not "anti-AI" to talk about risks. Elon Musk isn't a Luddite, and nor was Alan Turing,
even though we were all jointly given the Luddite of the Year Award in 2015 for asking, "What if we succeed?" The genome editors and the life extenders and the
What if we succeed? What then? In the case of AI, how do you propose to retain power, forever, over entities more powerful than ourselves?
One option might be to ban AI altogether, just as Butler's anti-machinists in Erewhon banned all mechanical devices after a terrible civil war. In Frank Herbert's Dune, the Butlerian Jihad had been fought to save humanity from machine control, and now there is an 11th commandment:
"Thou shalt not make a machine in the likeness of a human mind."
But then I imagine all those corporations and countries with their eyes on that 10 quadrillion-pound prize, and I think, "Good luck with that."
The right answer is that if making AI better and better makes the problem worse and worse, then we've got the whole thing wrong. We think we want machines that achieve the objectives we give them, but actually we want something else. Later in the series I'll explain what that "something else" might be, a new form of AI that will be provably beneficial to the human race, as well as all the questions that it raises for our future.
ANITA ANAND: Stuart, thank you very much indeed. Before we open this up to the audience at the Alan Turing Institute, you touched on this chat we had beforehand about whether people agree with you. Can we drill down into that a bit more because you're based at Berkley, Silicon Valley is a stone's throw away.
STUART RUSSELL: Yes.
ANITA ANAND: The majority of people who work in your field, do they regard you as a sage, a Cassandra? I suppose what I'm asking, are you a bit of a Billy No-Mates in Silicon Valley?
STUART RUSSELL: One response is quite understandable, which is I am a machine-learning researcher working at the coalface of AI. It's really difficult to get my machines to do anything. Just leave me alone and let me make progress on solving the problem that my boss asked me to solve. Stop talking about the future. But the problem is, this is just a slippery slope. If you keep doing that, as brain enhancers should also ask:
happened with the climate, I'm sure the people who produce petrol are saying, "Just leave me alone. People need to drive. I'm making petrol for them," but that's a slippery slope.
I do think that there is a sea change in the younger generation of researchers. Five years ago, I would say most people going into machine learning had dollar signs in their eyes, but now they really want to make the world a better place.
ANITA ANAND: Is that sea change enough if we carry on down this slope? You mentioned Chernobyl, I wonder whether you'd go as far as to say that there needs to be a Chernobyl-type event in AI before everyone listens to you?
STUART RUSSELL: Well, I think what's happening in social media is already worse than Chernobyl. It has caused a huge amount of dislocation.
ANITA ANAND: Well, if that's a little bit to chew on, let us chew on it now. Let's take some questions from the floor.
CLAIRE FOSTER-GILBERT: Claire Foster-Gilbert from Westminster Abbey Institute. Thank you very much indeed for your lecture. I wanted to ask you if you had any wisdom to share with us on the kinds of people we should try and be ourselves as we deal with, work with, direct, live with AI?
STUART RUSSELL: I'm not sure I have any wisdom on any topic, and that's an incredibly interesting question that I've not heard before. I'm going to give a little preview of what I'm going to say in the later lecture. The process that we need to have happen is that there's a flow of information from humans to machines about what those humans want the future to be like, and I think introspection on those preferences that we have for the future would be extremely valuable. So many of our preferences are unstated because we all share them.
For example, a machine might decide, okay, I've got this way of fixing the carbon dioxide concentration in the atmosphere to help with the climate, but it changes the colour of the sky to a sort of dirty, green ochre colour. Is that all right? Well, most of us have never thought about our preferences for the colour of the sky because we like the blue sky that we have. We don't make these preferences explicit because we all share them and also because we don't expect that aspect of the world to be changed, but introspecting on what makes a good future for us, our families and the world, and noticing, I think, that actually we all share far more than we disagree on about what that future should be like would be extremely valuable.
I notice that there is now a really active intellectual movement, or even a set of intellectual movements, around trying to make explicit what does human wellbeing mean? What is a good life? And I think it's just in time because for almost all of history in almost all parts of the world the main thing has been how do we not die, and if things go well, that time comes to an end and we actually have a breathing space then if we're not faced with imminent death, starvation, whatever, we have a breathing space to think about what should the future be. We finally have a choice, and we haven't really yet had enough discussion about that.
So that's what I would like everyone to do. If we have a choice, what should the future look like? If you could choose, if you weren't constrained by history or by resources, what would it be?
ANITA ANAND: Let us take a question from this side?
PAUL INGRAM: Paul Ingram soon to start at the Cambridge University Centre for the Study of Existential Risk. Stuart, I wanted to invite you to draw a comparison with another existential risk that you mentioned in your lecture, namely the emergence of splitting of the atom and the potential for nuclear war and the Cold War. We managed to survive, although looking back that was more luck than judgment, do you draw any analogies for the emergence of artificial intelligence?
STUART RUSSELL: I think it is an absolutely fascinating subject. What happened after Leo Szilard had this inspiration, he actually was crossing at the traffic light at South Hampton Row, and I tried walking backwards and forwards across that crossing and I haven't had any inspiration at all.
He realised very soon that this was a bad time to have had this discovery because there was already the beginnings of an arms race with Nazi Germany. He was a refugee. And he figured out how to make a nuclear reactor with all of its feedback control systems to keep the subcritical reaction going. He patented that in 1934 but he kept the patent secret because he did not want it to fall into the wrong hands, but fairly soon the Germans also figured this out.
Otto Hahn, Lise Meitner, were German physicists who were, I think, the first to actually demonstrate a fission reaction, and when it happened in the US, I think Villard and Teller were able to get a fission reaction to happen in their lab, and he went home and wrote in his diary:
"Tonight I felt that the world was headed for grief."
I think we have been incredibly lucky not to have suffered nuclear annihilation, and after the war the United States had a window of complete power and they set up the International Atomic Energy Agency and very strict standards for developing peacetime nuclear power, and that enabled the sharing of designs because we could be sure that the design safety rules would be followed, inspection and regulation and so on, and I think there's a lot of lessons in all of those phases for how we think about AI and a key is not to think of it as an arms race. That's what we're doing right now. We have Putin, we have US Presidents, Chinese, Secretaries, talking about this as if, "We are going to win. We're going to use AI and that will enable us to rule the world," and I think that's a huge mistake.
One is that it causes us to cut corners. If you're in a race, then safety is the last thing on your mind. You want to get there first and so you don't worry about making it safe. But the other is that general purpose or super-intelligent AI would be, essentially, an unlimited source of wealth and arguing over who has it would be like arguing over who has a digital copy of the daily Telegraph or something, right? If I have a digital copy, it doesn't prevent other people from having digital copies and it doesn't matter how many I have, it doesn't do me a lot of good.
So, I think we're seeing, on the corporate side, actually a willingness to share super-intelligent AI technology, if and when it's developed, on a global scale, and I think that's a really good development. We just have to get the governments on board with that principle.
ANITA ANAND: Thank you very much. We have many hands going up but actually, my eye has been caught by one of the fathers of the World Wide Web, the father of the World Wide Web, Tim Berners Lee is with us, and I hope you don't mind, I'm just sort of zeroing on you. Are you optimistic or pessimistic when it comes to the future of AI?
TIM BERNERS LEE: I am hopeful about the power of it, but I think all of these concerns are very real. When things go wrong in terms of social network, the sort of same tipping point happens when people end up getting polarised and afterwards we take the pieces apart, but there are lots of other systems in the world where the world is very connected and some of them are in government, and some of them are in big companies. Some are in, for example, investment companies.
If you're a fast trader, for example, humans need not apply because you have to be too fast. So, we've already got some jobs, and a lot of jobs in banks,
you have to be fast and so therefore it has to be run by AI already. Could it be that we get AI suddenly much more quickly if we build competitive AI systems?
STUART RUSSELL: It might. I would have to say that the whole field of evolutionary computation has been a field full of optimism for a long time. The idea that you could use nature's amazing process of evolving creatures to instead evolve algorithms hasn't really paid off yet. The drawback of doing things that way is that you have absolutely no idea how it works and creating very powerful machines that work on principles you don't understand at all seems to be pretty much the worst way of going about this.
TABITHA GOLDSTAUB: Hello, Stuart. Thank you. I'm Tabitha, the Chair of the government's AI Council. I can't help but ask, what should we be teaching in school?
STUART RUSSELL: I mean, not everyone needs to understand how AI works any more than I need to understand how my car engine works in order to drive it. They should understand what AI can and cannot do presently, and I hope they will understand the need to make sure that when AI is deployed, it's deployed in a way that's actually beneficial.
This is the big change, right, to think not just about how can I get a machine to do X, but what happens when I put a machine that does X into society, into schools, into hospitals, into companies, into governments, what happens, and there's really not much of a discipline answering that question right now.
ANITA ANAND: Let's take some more?
STEPHANIE: Hi, Professor Russell. This is Stephanie here. I'm interested in your views on the role of regulation for artificial intelligence and how we get the balance right between regulating and not constraining innovation, particularly if we do that in a liberal democracy and other countries around the world that are not liberal democracies do not regulate? Thank you.
STUART RUSSELL: I think it depends what you're talking about regulating. I think there are things that we should regulate now, and I'm happy to say that the EU is in the process of doing that, such as the impersonation of human beings by machines. So, that could be, for example, a phone call that you get that sounds exactly like your husband or your wife or one of your children, asking you to send some money or they've forgotten the password for your account or whatever it might be, that's quite feasible to do now. But generally, a machine impersonating a human is a lie and I don't see why we should authorise lies for
commercial purposes, and I'm happy to say the EU is explicitly banning that in the new legislation, and that should be something that is a global agreement.
There are other things we should be very restrictive of, such as deep fakes, material that convinces people that some event happened that didn't actually happen, but the question of safety, how we regulate to make sure that AI systems don't produce disastrous outcomes where humanity loses control, we don't know how to write that rule yet.
ANITA ANAND: One of the phrases you used when you were doing your lecture was, "Good luck with that." I mean, we can't get people to agree on most things, how are you going to agree a framework for this?
STUART RUSSELL: When it's in their self-interest, right, so everyone agrees on TCP/IP, which is the protocol that allows machines to communicate on the internet, because if they don't agree with that the machine at the other end doesn't understand them and so you can't send your message. So, everyone agrees on that protocol because it works. Same with wi-fi and standards for cell phones and all this stuff, so there's a huge process. It's invisible to almost everybody but there are giant committees and annual meetings that go on and on and on, and they argue about the most tiny details of all these standards until they hammer it all out and then that standard is incredibly beneficial.
So, we could do the same thing for how you design AI systems to ensure that they are actually beneficial to humans, but we're not ready to say what the standards should be.
ANITA ANAND: There is one here?
JANE BONHAM CARTER: Jane Bonham Carter. I'm a Liberal Democrat politician but I, for years, worked in television and when TV/radio came along, and that was an intrusion into people's lives in a way that had never existed before, but it covered the ground of what I think you were talking about, which is what people shared. So, can AI not be directed towards a more benign curation, I suppose, is my question?
STUART RUSSELL: Absolutely, and as I said, I think some of the social media companies are genuinely interested. I don't think it's just a window dressing or a self-washing or anything, it's that they are genuinely interested in how their products, which are incredibly powerful, how they can be actually beneficial to people. I can't say very much at the moment but we, among others, are developing research relationships and we're finding openness and willingness
to share data and algorithms so that we can actually understand how to do this right.
It actually turns out to be one of the most difficult questions because if you think about driving, for example, it's difficult but probably not impossible to figure out how we should trade off safety versus getting to your destination, versus politeness to other drivers and so on, but what the algorithms are doing is actually changing our preferences, so it's changing what we want.
The person who first ventures into social media, having never touched it before, might be horrified by the person that they have become 12 months later. But the person 12 months later isn't horrified by themselves, right, they are actually really happy that they're now a diehard ecoterrorist and they're out there doing this, that and the other, and we don't even have a basic philosophical understanding of how to make decisions on behalf of someone who's going to be different when those decisions have impact. Do I help the person achieve what they want now, or do I help the person achieve what they're going to want when I achieve it?
It's a puzzle and philosophers have started writing about it, but we just don't have an answer and so this manipulation of human preferences by social media algorithms is actually getting at the hardest thing to understand in the AI problem, as far as I can see.
ANITA ANAND: Let's take another question here from this row?
STEVE McMANUS: Hi, my name's Steve McManus, a lifelong NHS employee. Arguably you are one of the thought leaders in this field, given also some of the other members of the audience we've got here today; where do you draw your thought leadership on this subject?
STUART RUSSELL: Another good question. I have found, actually, reading outside of my field, reading outside AI, in economics, particularly philosophy, has been enormously useful, although economics has this – it's called "the dismal science," I think that's a bit unfair. It's a very hard problem. It's, in many ways, a lot harder than physics and chemistry, but economists actually do try to think about this question: How should the world be arranged?
I was really shocked going back to read Adam Smith, who's widely reviled as "The Apostle of Greed," and so on and so forth, but actually what Adam Smith says at the beginning of his first book is that:
"It's so obvious to everyone that each of us cares deeply about other people that it hardly merits saying it, but I'm going to say it anyway," and then he says it.
That's the beginning of his first book. So, I've learned a great deal from economists, from philosophers, trying to understand a question that AI is now going to have to answer.
If AI systems are going to be making decisions on behalf of the human race, what does that mean? How do you tell whether a decision is a good or a bad decision when it's being made on behalf of the human race, and that's something that philosophers have grappled with for thousands of years?
ROLY KEATING: Roly Keating from British Library. It's wonderful to have you here. Thank you for the lecture. I was interested in the language and vocabulary of human intellectual life that seems to run around AI, and I'm hearing data gathering, pattern recognition, knowledge, even problem solving, but I think an earlier question used the word "wisdom," which I've not heard so much around this debate, and I suppose I'm trying to get a sense of where you feel that fits into the equation. Is AI going to help us as a species gradually become wiser or is wisdom exactly the thing that we have to keep a monopoly on? Is that a purely human characteristic, do you think?
STUART RUSSELL: Or the third possibility would be that AI helps us achieve wisdom without actually acquiring wisdom of its own, and I think, for example, my children have helped me acquire wisdom without necessarily having wisdom of their own. They certainly help me achieve humility. So, AI could help, actually, by asking the questions, right, because in some ways AI needs us to be explicit about what we think the future should be, that just the process of that interrogation could bring some wisdom to us.
ANITA ANAND: And the final question, apologies if we didn't get to you, so many fantastic questions, but the final one with you?
GILA SACKS: Hi. Gila Sacks. It seems that one of the most scary things about this future is that if individuals feel powerless in the face of machines and corporations, it will be a self-fulfilling prophecy, we will be powerless. So, how can individuals have power in the future that you see playing out, either as consumers or as citizens?
STUART RUSSELL: I wish that the entire information technology industry had a different structure. If you take your phone out and look at it, there are 50 or a hundred corporate representatives sitting in your pocket busily sucking out as much money and knowledge and data as they can. None of the things on your phone really represent your interests at all.
What should happen is that there's one app on your phone that represents you that negotiates with the information suppliers, and the travel agencies and whatever else, on your behalf, only giving the information that's absolutely necessary and even insisting that the information be given back, that transactions be completely oblivious, that the other party retains no record whatsoever of the transaction, whether it's a search engine query or a purchase or anything else.
This is technologically feasible but the way the market has evolved where it's completely the other way around. As individuals, you're right, we have no power. You have to sign a 38-page legal agreement to breathe and that, I really think, needs to change and the people who are responsible for making that change are the regulators.
Just to give a simple example, right, it's a federal crime in the US to make a phone call, a robocall, to someone who is on the federal Do Not Call list. I am on the federal Do Not Call List. I get 15 or 20 phone calls a day from robocalls. When you add that up, that is billions of crimes a day or trillions or crimes every year, trillions of federal crimes occurring, and there hasn't been a single prosecution, as far as I know, this whole year. I think there was one last year where they took one group down, but there is a total failure. We are in the wild west and there isn't a Sheriff in sight. So, as individuals, ask your representatives to do something about it.
We are also responsible, the technologists are also responsible, because we developed the internet in a very benign mindset. I can remember, when I was a computer scientist at Stanford, we could actually map our screens to anybody else's screen in the building and see what was on their screen. We thought that was cool, right? It just never occurred to anyone that that might be not totally desirable. We built technology with just open doors and complete fictitious IDs and all the rest of it. I think on the technology side, allowing real authentication of individual's traceability, responsibility, and then regulations with teeth, would help a great deal.
ANITA ANAND: Well, with thoughts of teeth, robocalls and crowded pockets, I'm afraid we're going to have to leave it there. Next time Stuart is going to be asking: What AI means for conflict and war. That is from Manchester, but for now a big thanks to our audience, to the Alan Turing Institute for hosting us, and, of course, to our Reith Lecturer, Stuart Russell.
END OF TRANSCRIPT
"Remarks by President Biden in Press Conference", The White House
"'This alert is her scream': new system would help locate missing Indigenous women", The Guardian
"How Martin Luther King Jr. Changed His Mind About America", Time Magazine
©2022 Planet Earth Foundation. All Rights Reserved.