text:
The First Ladies of the United States of America who visited India
SharefbsharetwsharepinshareComments (0)
Last updated on - Feb 20, 2020, 16:35 ISTShare fbsharetwsharepinshareComments (0)
01/10Ahead of Melania Trump's grand welcome, many US First Ladies were here too --from Jacqueline Kennedy to Michelle Obama
The President of United States Of America, Donald Trump, along with first lady Melania Trump will be coming to India on February 24 and 25. The Trumps will spend their first day in Ahmedabad and Agra before moving to Delhi for the official reception and bilateral talks. This is also Trump's first state visit to India and also the first standalone visit by a US President to India. The US first lady Melania Trump is likely to visit a Delhi government school on February 25 to understand the "happiness curriculum". Here's taking a look at the first ladies from Jacqueline Kennedy to Michelle Obama, who visited India before Mrs Trump.
Photo Courtesy: US Embassy New Delhi | Flickr
02/10Jacqueline Kennedy, 1962
The former first lady of the United States, Jacqueline Kennedy, wife of the 35th US President, John F. Kennedy, visited India in 1962. Mrs Kennedy came to India for nine days. She had reportedly said that it was a 'dream' for her to visit India. She was welcomed by then Prime Minister Pandit Jawaharlal Nehru. Mrs Kennedy also experienced the Indian culture and joined in the celebrations of Holi before leaving for Pakistan.
03/10Pat Nixon, 1969
The 37th President of the US, Richard Nixon, and his wife Pat Nixon visited India for a day in 1969. Reportedly, Nixon's trip is said to be the shortest trip by an American President. During Pat Nixon's visit to India with her husband, the situation in the country was tense. At that time, then Indian Prime Minister Indira Gandhi and Congress were facing a crisis situation over who would become the President of India after the sudden death of Dr Zakir Hussain.
04/10Rosalynn Carter, 1978
Jimmy Carter, the 39th President of the United States, along with his wife Rosalynn Carter visited India in 1978. Lady Rosalynn was also welcomed with grand pomp and was introduced to Indian culture. While her husband was busy meeting Indian leaders, the former first lady visited a boarding school in Delhi. The then first couple also visited Daulatpur Nasirabad, which is now known as Carterpuri, in Gurugram before leaving for Saudi Arabia. They had reportedly gifted TV sets to the villagers. India's former Prime Minister Atal Bihari Vajpayee welcomed the Carters.
05/10Hillary Clinton, 1995 and 1997
As a part of her 12-day visit to Southeast Asia to address a speech on women's rights and education, former first lady Hilary Clinton, visited India for three days in 1995 without her husband, Bill Clinton. Her daughter Chelsea also accompanied her during her visit. Later, again in 1997, Hilary Clinton came on a visit and attended Mother Teresa's funeral in Kolkata (then known as Calcutta). She also met Canadian PM Jean Chretien's wife Aline Chretien and Sonia Gandhi, at Mother Teresa's funeral mass at Netaji Indoor Stadium in Kolkata, on September 13, 1997.
06/10Laura Bush, 2006
George Bush, the 43rd US President, and first lady Laura Bush visited India in 2006. The first couple spent around 60 hours in India, which is considered as the shortest trip by a US President, after Nixon's 23-hour trip in 1969. However, during her trip, Laura Bush attended a ceremonial reception at Rashtrapati Bhawan, visited Noida's Film City, spent time at a centre run by Mother Teresa's Missionaries of Charity. The two also paid tribute to Mahatma Gandhi at the Mahatma Gandhi memorial at Rajghat in New Delhi.
07/10Michelle Obama, 2010 and 2015
Michelle Obama is considered as one of the most iconic first ladies of America. Michelle Obama visited India in 2010 and 2015. While former President Barack Obama was busy with meetings with Indian diplomats and then PM Manmohan Singh, Michelle Obama was winning hearts. Michelle Obama and President Obama visited a school in Mumbai where the former First Lady flaunted her graceful moves. She certainly floored not just the students but their teachers too. She managed to wow people wherever she went, with her personality, witty answers and Bollywood moves. She visited the National Handicrafts and Handloom Museum in New Delhi and interacted with the students. The couple also met the survivors of the 26/11 terror attack, which happened two years before their visit in 2010. In 2015, Barack and Michelle Obama visited India again and were welcomed by Prime Minister Narendra Modi.
08/10Barack Obama and Michelle Obama, 2015
In 2015, Barack and Michelle Obama visited India again and were welcomed by Prime Minister Narendra Modi. Barack Obama and Michelle Obama also became the first US President and First Lady to be the chief guest at India's Republic Day parade.
09/10Barack Obama and Michelle Obama, 2015
CHOKE OF A JOKE
Builder sells 16 plots without registra... Builder sells 16 plots without registration, RERA slaps Rs 50L fine
Surat: Truck runs over labourers sleepi... Surat: Truck runs over labourers sleeping on footpath, 15 killed
Another runaway bride absconds with jew... Another runaway bride absconds with jewellery worth Rs 94,000
LD Engg draws on green energy to make c... LD Engg draws on green energy to make campus 'aatmanirbhar'
Fake calls: Man loses Rs 10L
Fresh cases in A'bad city drop below 100
{{if user.username || user.id}} {{:user.name}} {{else}} {{:user.name}} {{/if}} {{if user.badge}}
{{for user.badge}} {{if #index < 3}}
{{/if}} {{if user.points}} {{:user.points}} {{:user.pointsNeeded}} more points needed to reach next level. Know more about Times Points {{/if}}
{{if parentusername}} {{:parentusername}} {{/if}} {{if user.location}} {{:user.location}} {{/if}} Just Now {{if user.username || user.id}} •Follow {{/if}}
Reply Flag
Copied from article
Flagged {{if opinion && opinion[0] && opinion[1]}} {{:opinion[0].count}} {{:opinion[1].count}} {{/if}}


text:
» 342 Skidder Trail, Truckee, CA 96161 MLS-20170241
342 Skidder Trail, Truckee, CA 96161 MLS-20170241
Would you like to hear my opinion about 342 Skidder Trail, Truckee, CA 96161, the good, the bad, and the ugly for this property? Shoot me an email, and as Paul Harvey said, "and now you know the rest of the story." I pride myself in being honest and frank and doing right by people. No fluff, honest pros and cons from a down-to-earth guy with over 20 years of experience.
342 Skidder Trail, Truckee, CA 96161
This home was previously listed on 10/31/2005 at a list price of $1,195,000.00. View Property
This home was previously listed on 11/24/2009 at a list price of $850,000.00. View Property
04/05/19 $939,000.00 1626 Deer Path
05/15/19 $819,000.00 7001 Northstar D...
11/22/17 $1,095,000.00 178 Basque
05/23/18 $1,045,000.00 970 Northstar Drive
The property at 342 Skidder Trail, Truckee, CA 96161 is located within in the city of Truckee and the NORTHSTAR HOMES-7NS subdivision. The property was built in 1992 and offers 4 bedrooms 3 bathrooms and 2046 square feet.
The Single Family at 342 Skidder Trail, Truckee, CA 96161 is roughly 9.03 miles from Squaw Valley Ski Resort, 1.3 miles from Northstar Ski Resort, 6.07 miles from Truckee, and 10.49 miles from Tahoe City.
Wonderful and meticulously maintained Northstar getaway on desirable Skidder Trail. The thoughtful and open floor plan, consisting of 4 bedrooms, 3 bathrooms and loft, is perfect for family and entertaining. Recent updates include new kitchen, upgraded bathrooms, high efficiency furnace, carpet, paint, lighting, window coverings and washer/dryer. Large sliders open to a generous deck allowing for gorgeous views of the golf course and Martis Valley. NPOA amenities are excellent.
Address 342 Skidder Trail
Topography Upslope,Gentle
Contact Tristan for questions regarding 342 Skidder Trail, Truckee, CA 96161
The data relating to real estate for sale on this web site comes in part from the Broker Reciprocity Program of the Tahoe Sierra Multiple Listing Service. Real estate listings held by brokerage firms other than Coldwell Banker Residential Brokerage are marked with the Broker Reciprocity logo and detailed information about them includes the name of the listing brokers. This listing courtesy of: Sereno Group. The information being provided is for consumers' personal, non-commercial use and may not be used for any other purpose other than to identify prospective properties consumers may be interested in purchasing. Information deemed reliable but not guaranteed
Please send me more information about the property at 342 Skidder Trail, Truckee, CA 96161
1626 Deer Path
7001 Northstar D...
178 Basque
Sold: Dec. 19 2018
970 Northstar Drive
4000 West Lake B...
West Shore Lake Tahoe, CA, 96141
1580 Washoe Way

redpajama_set_name: RedPajamaCommonCrawl


text:
History is the study of the past. When used as the name of a field of study, history refers to the study and interpretation of the record of humans, families, and societies as preserved primarily through written sources. This is a list of history topics covered on English Wikipedia:

A

 Abkhazia, History of
 Adena culture
 Afghanistan, History of
 Africa, History of
 Age of Discovery
 Age of Enlightenment
 Age of Exploration
 Age of Reason
 Age of Sail
 Agriculture, History of
 Akrotiri and Dhekelia, History of
 Åland, History of
 Albania, History of
 Alfred the Great
 Algeria, History of
 Alternative history
 American history
 American Revolution
 American Samoa, History of
 Ancient China
 Ancient Egypt
 Ancient Greece
 Ancient history
 Ancient India
 Ancient Japan
 Ancient Korea
 Ancient Rome
 Ancient warfare
 Andorra, History of
 Angola, History of
 Anguilla, History of
 Antigua and Barbuda, History of
 Archaeology
 Architecture, History of
 Argentina
 Armenia, History of
 Art history
 Aruba, History of
 Ascension Island, History of
 Asia, History of
 Assyria
 Atomic Age
 Australia, History of
Prehistory
Before 1788
1788–1850
1851–1900
1901–1945
Since 1945
 Austria, History of
 Aviation history
 Axial Age
 Azerbaijan, History of
 Aztecs

B
 Bahamas, History of the
 Bahrain, History of
 Bangladesh, History of
East Pakistan
 Battle
 Babylonia
 Banking, History of
 Belarus, History of
 Belgium, History of
 Bias
 Bibliography of history
 Biography
 Botswana, History of
 Brazil, History of
Colonial
Empire
 Britain, History of
Ancient
Roman
Empire
 Bronze Age
 Business history
 Byzantine Empire

C

 Cambodia, History of
 Cameroon, History of
 Canada, History of
 First Nations
 Carolingian Renaissance
 Carolingian writing
 Carolingian art
 Carolingian architecture
 Carolingian music
 Chad, History of
 China, History of
Xia Dynasty
Shang Dynasty
Zhou Dynasty
Qin Dynasty
Han Dynasty
Three Kingdoms
First Jin Dynasty
Sixteen Kingdoms
Southern and Northern Dynasties
Sui Dynasty
Tang Dynasty
Five Dynasties and Ten Kingdoms period
Liao Dynasty
Song Dynasty
Western Xia Dynasty
Second Jin Dynasty
Yuan Dynasty
Ming Dynasty
Qing Dynasty
Republic of
People's Republic of
 Chile, History of
 Chronology
 Cold War
 Colombia, History of
 Colonialism, History of
 Contemporary history
 Congo, Democratic Republic
 Congo, Republic
 Copper Age
 Croatia, History of
 Crusades
First
Second
Third
Livonian
German
Fourth
Albigensian
Children's
Fifth
Prussian
Sixth
Seventh
Shepherds' 1251
Eighth
Ninth
Aragonese
Shepherds' 1320
Alexandrian
Nicopolis
Northern
Hussite
Varna
Otranto
Ottoman-Habsburg wars
Lepanto
Vienna
 Cuba, History of
Revolution
 Cultural history
 Cyprus, History of
 Czech Republic, History of

D
 Dark Ages (historiography)
 Death of Alexander the Great
 Decolonization
 Denmark, History of
 Digital Age
 Divine Right of Kings

E
 Economic history
 Education, History of
 Egypt, History of
Ancient
Achaemenid
Ptolemaic
Roman
Arab
Ottoman
Muhammad Ali dynasty
Modern
 Eighty Years' War
 History of England
Reformation
Civil War
 Ethiopia, History of
 Europe, History of
European Union
 Environmental history

F
 Falkland Islands, History of
Fashion design, History of
 Finland, History of
 France, History of
Prehistoric
Gaul
Roman Gaul
Frankish Empire
In the Middle Ages
in the Early Modern Period
Revolution
First Republic
First Empire
Bourbon Restoration
July Monarchy
Second Republic
Second Empire
Third Republic
Vichy France
Provisional Government
Fourth Republic
Fifth Republic
 Forest history

G
 Gender history
 Genealogy
 Germany, History of
 Germanic tribes
 Migration period
 Frankish Empire
 Merovingian Dynasty
 Carolingian Dynasty
 Holy Roman Empire
 Early modern history of Germany
 German confederation
 North German Federation
 German Empire
 Weimar Republic
 Nazi Germany
 Partitions of Germany
 East Germany
 West Germany
 Reunification of Germany
 Ghana, History of
 Great Depression
 in Australia
 in Canada
 in France
 in Latin America
 in the Netherlands
 in South Africa
 in the United Kingdom
 in the United States
 Greece, History of
 Guatemala, History of
 Gunpowder warfare

H
 Haiti, History of
 Historical document
 Historical method
 Historical sociology
 Historicism
 Historiography
 History
 History from below
 Holocaust, the
 Holy Roman Empire
 Honduras, History of
 Hong Kong, History of
 Human history
 Hundred Years' War
 Hungary, History of
 Huns

I
 Iceland, History of
 Ideas, History of
 Imperialism
 Inca Empire
 India, History of
Middle kingdoms
Mughal Empire
British Raj
Republic
 Indochina Wars
 Indonesia, History of
 Indus Valley civilization
 Industrial Age
 Industrial Revolution
 Industrial warfare
 Information Age
 The Internet, History of
 Inquisition
 Iran, History of
Revolution
 Ireland, History of
Early history of Ireland
Norman Ireland
 Iron Age
 Italy, History of

J
 Jamaica, History of
 Japan, History of
Ancient
Tokugawa shogunate
Imperial
 Jersey, History of
 Jordan, History of

K
 Kazakhstan, History of
 Kenya, History of
 Korea, History of
Ancient
North
South
Korean War
 Kosovo, History of
 King Arthur

L
 Labor history
 Landscape history
 Laos, History of
 Latvia, History of
 League of Nations
 Lebanon, History of
 Lesotho, History of
 LGBT history
 Liberia, History of
 Libya, History of
 Liechtenstein, History of
 Lithuania, History of
 Local history
 Luxembourg, History of

M
 Malaysia, History of
 Maritime history
 Maya civilization
 Medicine, History of
 Medieval warfare
 Meiji Restoration
 Mesoamerica
 Mesolithic period
 Mesopotamia
 Mexico, History of
 Middle Ages
 Migration
 Military Frontier
 Military history
 Prehistoric warfare
 Ancient warfare
 Medieval warfare
 Early Modern warfare
 Industrial warfare
 Modern warfare
 Minoan civilization
 Modern history
 Mongol Empire
 Mozambique, History of
 Music, History of
 Prehistoric music
 Ancient music
 Early music
 Medieval music
 Renaissance music
 Baroque music
 Common practice period
 Classical
 Romantic music
 20th century music

N
 Napoleonic Wars
 Nationalism
 Natural history
 Naval history
 Neolithic period
 Nepal, History
 New Imperialism
 New Zealand, History of
 Normans
 North America, History of

O
 Oceania, History of
 Oman, History of
 Oral history
 Ottoman Empire

P

 Pakistan, History of
 Palaeography
 Palau, History of
 Paleolithic period
 Palestine, History of
 Panama, History of
 Papua New Guinea, History of
 Paraguay, History of
 Parliamentary procedure, History of
 Pearl Harbor
 People's history
 Periodization
 Persian Empire
 Peru, History of
Viceroyalty
War of Independence
 Philippines, History of
 Philosophy of history
 Poland, History of
Under the Piasts
Under the Jagiellons
First Republic
Partitions
Second Republic
People's Republic
 Political history
 Political science, History of
 Political thought, History of
 Portugal, History of
Empire of
Civil War
 Prehistory
 Prehistoric art
 Prehistoric music
 Prehistoric warfare
 Protohistory
 Pseudohistory
 Psychohistory

Q
 Qatar, History of
 Quantitative history

R

 Rail transport, History of
 Religion, History of
 Renaissance
 Restoration (disambiguation),
Bourbon
English
European
Meiji
Spanish
Swiss
 Revolution,
1848
American
Communist
Cuban
French
Industrial
Russian
Singing (Latvia)
 Road transport, History of
 Romani people, History of
 Romania, History of
 Rome,
Ancient
Empire of
Fall of
Republic of
 Russia, History of
Tsardom
Empire
Revolution (1917)
USSR
post-Soviet Russia
 Rwanda, History of

S
 Science, History of
 Scotland, History of
 Scramble for Africa
 Serbia, History of
 Slavery, History of
 Social history
 Socialism, History of
 Source, Primary
 Source, Secondary
 South Africa, History of
 South America, History of
 South India, History of
 Space Age
 Spain, History of
Empire of
Civil War
 Sport, History of
 Stone Age
 Sweden, History of
 Switzerland, History of
 Syria, History of

T
 Taiping Revolution
 Telecommunication, History of
 Thailand, History of
 Three-age system
 Trade, History of
 Transport, History of
 Turkey, History of

U
 Uganda, History of
 Ukraine, History of
 United States, History of
 Diplomatic history
 Military history
 Technological and industrial history
 Economic history
 Cultural history
 Women's history
 Native American history
 Timeline
 Pre-Columbian era
 Colonial period
 American Revolutionary War
 1776–1789
 1789–1849
 1849–1865
 1865–1918
 1918–1945
 1945–1964
 1964–1980
 1980–1991
 1991–present
 United Kingdom, History of
 Universal history
 Uruguay, History of
 USSR, History of the
 Uzbekistan, History of

V
 Vanuatu, History of
 Vatican City, History of
 Venezuela, History of
 Venice, History of
 Vietnam, History of
American War
 Vikings
 Viking Age

W
 War
 Warfare, Ancient
 Western fashion, History of
 Women's history
 World history
 World War I
 World War II

 Writing, History of

X
 Xinhai Revolution

Y
 Yemen, History of

Z
 Zambia, History of
 Zanzibar, History of
 Zimbabwe, History of

History topics

redpajama_set_name: RedPajamaWikipedia


text:
John M. Flaxman Library
Global News + Events
Historic and Current News Sources
Global News Web Sites
Global Facts & Figures
Flaxman Library
37 S. Wabash, 6th floor
flaxman@saic.edu
Social: Facebook Page Twitter Page Instagram Page
Article Resources
Databases @ Flaxman
News titles @ Flaxman
Off campus? log in using your ARTIC username and password, just like you're logging in to your SAIC email.
US Major Dailies
Current news for the New York Times, Washington Post, Wall Street Journal, Los Angeles Times and the Chicago Tribune. Daily content is available by 8am each day and provides archives back to the 1980s.
LexisNexis (now Nexis Uni)
Articles and transcripts from major newspapers, business magazines, television and radio programs; 1977 to present. Includes New York Times current subscription, text only.
US Major Dailies (ProQuest Historical Newspapers)
Archive of the New York Times, Washington Post, Wall Street Journal, Los Angeles Times and the Chicago Tribune. Dates from the first issue of each title.
Cover-to-cover full text for 20 U.S. & international newspapers, including The Washington Post, The Times (London), etc.
Finds both digital and print subscriptions to journals/newspapers in Flaxman Library
Title begins withTitle equalsTitle contains all wordsISSN equals
Atlantic Monthly
Next: Global News Web Sites >>
Last Updated: Jun 7, 2021 10:56 AM
URL: https://libraryguides.saic.edu/global
Subjects: Current Events, History & Government


text:
Patek Philippe Timepiece
© Patek Philippe
Watch Snob: Can You Be 'Too Young' For A Patek Philippe?
Watch Snob
chrimson
How often do you really need to purge your wardrobe?
Old Enough For A Patek?
Hey Watch Snob,
Love reading your columns. I must say that my respect and love for what a watch is capable of being has increased greatly from reading — and for that, I thank you. Had a question. If one has the means to acquire a truly top-of-the-line piece at a fairly young age, and I do mean something along the lines of a Patek Philippe (which I gratefully say, you introduced me to) — how young is too young to do so?
I want one and can afford one, but I am just shy of 25 and don't want to wear something inappropriate for me age, if that's possible. However, I'm also at that age where I want to make the jump from young adult to adult and buy a truly classy watch. I've never really invested in a good watch simply due to immaturity. And I don't really go out to clubs and such anymore at all so this watch would not be in any drunken danger of that sort.
Would appreciate your insight. Thanks.
I don't think there's any such thing as an arbitrary cut-off for when you're old enough for a Patek and, even if there were, you'd be on the right side of it at 25. If you can afford it and you've evaluated the other options in the category you're considering, why not? You may as well start getting a lifetime of satisfaction out of your watches now. You're old enough to marry, drink, vote, work and serve your country in battle — I think you're entitled to a decent watch.
Who Designed Patek Philippe Neptune 5080?
I am writing to ask what you know about the Patek Philippe 5080, the Neptune. I know only that it was produced in the late '90s and early 2000s; that it was not a commercial success; that it is one of the few stainless steel watches made by the firm (gold and steel as well as gold versions do exist); and that I like the look of the watch.
Do you know who designed the piece? Anything about the styling would be enlightening (the Roman numeral VI and XII have unusual looks). I'm fairly well-versed in the particular movement the 315/190, but any information you would care to share would be appreciated.
Lastly, if you don't know a lot about the watch — after all, it wasn't a very popular piece — perhaps you could share whatever thoughts and impressions you have regarding the Neptune that you might think entertaining.
I ask you, O Snob, because I find little on the 'net and a stony silence from Patek.
An interesting question about a very rarely-heard-of watch — the Neptune was, as you say, not a commercial success for Patek Philippe; it was offered in gold and steel, on strap and bracelet, in mechanical and quartz iterations, but it went unloved — as to why, we can only guess. Full many a flower is born to blush unseen, as the poet Thomas Gray has it.
It simply was not what people wanted from Patek Philippe, I suppose (the bracelet is a little off-the-reservation, for instance, and on a strap it looks a little too much like a Rolex Thunderbird) but for reasons why it was discontinued, you would have to ask the Stern family, or the Invisible Hand (which are becoming closer to the same thing these days, at least as far as Patek Philippe is concerned.)
Patek Philippe Watches
Fit Friday: Kick the Weekend Off With a Free 'Full Body Pump' That Anyone Can DoRead More
Detroit Healthcare Workers 'Lose Themselves' In Eminem's Mom's SpaghettiRead More
The Watch Snob Passes Judgement On Whether It's Ever OK To Modify Your Watch
Don't Sell A Luxury Watch Until You've Read This
Caught In A Love Triangle Between Two Watches
Watch Snob: Is There A Rule For How Much To Spend?
Matching Your Belt to Your Outfit is Easy with These Tips


These beautiful earrings were inspired by design elements of the Minton tile ceiling at Bethesda Terrace Arcade in Central Park. The ceiling is a richly decorated component of the Arcade, which was restored by the Central Park Conservancy in 2007. The earrings are made of solid brass with a non-tarnishing silver finish. Each set is packaged in a gift box. Each earring measures approx. 1 1/2" in length and 1 1/2" wide. The ear wires are hypo-allergenic.


text:
package service

import "context"

// UploadImg upload img
func (s *Service) UploadImg(c context.Context, imgContent []byte, imgName string) (location string, err error) {
	return s.dao.UploadImg(c, imgContent, imgName)
}

redpajama_set_name: RedPajamaGithub


text:
\section{Introduction}
\enlargethispage{\baselineskip}

Deep learning models are vulnerable to adversarial examples -- small imperceptible perturbations to their inputs that lead to misclassification \citep{goodfellow_explaining_2015, szegedy_intriguing_2014}.
To solve this problem, recent works proposed heuristic defenses that are robust to specific classes of perturbations, but many would later be broken by stronger attacking algorithms~\citep{carlini2017adversarial, athalye2018obfuscated, uesato2018adversarial}.
This led the community to both strengthen empirical defenses \cite{kurakin2016adversarial,madry2017towards} as well as build \emph{certified} defenses that provide robustness guarantees, i.e., models whose predictions are constant within a neighborhood of their inputs \citep{wong2018provable,raghunathan2018certified}.
In particular, \emph{randomized smoothing} is a recent method that has achieved state-of-the-art provable robustness~\citep{lecuyer2018certified, li_certified_2019, cohen_certified_2019}.
In short, given an input, it outputs the class most likely to be returned by a base classifier, typically a neural network, under random noise perturbation of the input.
This mechanism confers stability of the output against $\ell_p$ perturbations, even if the base classifier itself is highly non-Lipschitz.
Canonically, this noise has been Gaussian, and the adversarial perturbation it protects against has been $\ell_2$ \citep{cohen_certified_2019,salman2019provably,Zhai2020MACER}, but some have explored other kinds of noises and adversaries as well \citep{lecuyer2018certified,li_certified_2019,dvijotham_framework_2019}.
In this paper, we seek to comprehensively understand the interaction between the choice of smoothing distribution and the perturbation norm.%
\footnote{V2 update: we added results using stability training, semi-supervised learning, and ImageNet pre-training. See \cref{tab:sota}.}
\enlargethispage{\baselineskip}
\begin{enumerate}[nosep]
    \item We propose two new methods to compute robust certificates for additive randomized smoothing against different norms.
    \item We show that, for $\ell_1, \ell_2, \ell_\infty$ adversaries, the optimal smoothing distributions have level sets that are their respective \emph{Wulff Crystals} --- a kind of equilibrated crystal structure studied in physics since 1901 (\citeauthor{wulffZurFrageGeschwindigkeit1901}).
    \item Using the above advances, we obtain state-of-the-art $\ell_1$ certified accuracy on CIFAR-10 and ImageNet. With stability training \citep{li_certified_2019}, semi-supervised learning \citep{carmon_unlabeled_2019}, and pre-training in the fashion of \citet{hendrycks_using_2019}, we further improve CIFAR-10 certified accuracies, with $>30\%$ advantage over prior SOTA for $\ell_1$ radius $\ge 1.5$.
    See \cref{tab:sota}.
    %
    %
    %
    %
    %
    %
    %
    %
    \item Finally, we leverage the classical theory of Banach space \emph{cotypes} \citep{wojtaszczyk1996banach} to show that current techniques for randomized smoothing cannot certify nontrivial accuracy at more than $\Omega(\min(1, d^{\f 1 p - \f 1 2}))$ $\ell_p$-radius, if all one uses are the probabilities of labels when classifying randomly perturbed input.
    %
\end{enumerate}

\begin{table*}[t!]
    \centering
    \begin{tabular}{llrrrrrrrr} \toprule
         ImageNet & $\ell_1$ Radius &  0.5 & 1.0 & 1.5 & 2.0 & 2.5 & 3.0 & 3.5 & 4.0 \\ \midrule
         & Laplace, \citet{teng_ell_1_2019} (\%)  & 48 & 40 & 31 & 26 & 22 & 19 & 17 & 14\\
         & Uniform, Ours (\%) & 55 & 49 & 46 & 42 & 37 & 33 & 28 & 25\\
         & + Stability Training & \bf 60 & \bf 55 & \bf 51 & \bf 48 & \bf 45 & \bf 43 & \bf 41 & \bf 39\\
          \midrule
         CIFAR-10 & $\ell_1$ Radius & 0.5 & 1.0 & 1.5 & 2.0 & 2.5 & 3.0 & 3.5 & 4.0 \\ \midrule
         & Laplace, \citet{teng_ell_1_2019} (\%) & 61 & 39 & 24 & 16 & 11 & 7 & 4 & 3\\
         & Uniform, Ours (\%) & 70 & 59 & 51 & 43 & 33 & 27 & 22 & 18\\
         & + Stability Training & 70 & 60 & 53 & 47 & \bf 43 & 39 & 35 & 28\\
         & + Stability Training, Semi-supervision & \bf 74 & \bf 63 & 54 & \bf 48 & \bf 43 & 38 & 34 & 31 \\
         & + Stability Training, Pre-training & \bf 74 & 62 & \bf 55 & \bf 48 & \bf 43 & \bf 40 & \bf 37 & \bf 33\\
         \bottomrule
    \end{tabular}
    \caption[Caption]{Certified top-1 accuracies of our $\ell_1$-robust classifiers, vs previous state-of-the-art, at various radii, for ImageNet and CIFAR-10.\protect\footnotemark}
    \label{tab:sota}
\end{table*}

\section{Related Works}
\label{sec:relatedWorks}

Defences against adversarial examples are mainly divided into \emph{empirical} defenses and \emph{certified} defenses.

\textbf{Empirical defenses} are heuristics designed to make learned models empirically robust. An example of these are \emph{adversarial training} based defenses  \citep{ kurakin2016adversarial, madry2017towards} which optimize the parameters of a model by minimizing the worst-case loss over a neighborhood around the input to these models~\citep{carlini2017adversarial, laidlaw2019functional, wong2019wasserstein, hu2020improved}.
Such defenses may seem powerful, but have no guarantees that they are not ``breakable''. In fact, the majority of the empirical defenses proposed in the literature were later ``broken'' by stronger attacks \citep{carlini2017adversarial, athalye2018obfuscated, uesato2018adversarial, athalye2018robustness}.

\textbf{Certified defenses} guarantee that for any input $x$, the classifier's output is constant within a small neighborhood of $x$. Such defenses are typically based on certification methods that are either \textit{exact} or \textit{conservative}. Exact methods include those based on Satisfiability Modulo Theories solvers~\citep{katz2017reluplex, ehlers2017formal} or
mixed integer linear programming \citep{tjeng2018evaluating, lomuscio2017approach, fischetti2017deep}, which, although guaranteed to find adversarial examples if they exist,  are unfortunately computationally inefficient.
On the other hand, conservative methods are more computationally efficient, but might mistakenly flag a ``safe'' data point as vulnerable to adversarial examples \citep{wong2018provable, wang2018mixtrain, wang2018efficient, raghunathan2018certified, raghunathan2018semidefinite, wong2018scaling, dvijotham2018dual, dvijotham2018training, croce2018provable, salman2019convex, gehr2018ai2, mirman2018differentiable, singh2018fast, gowal2018effectiveness, weng2018towards, zhang2018efficient}.
However, none of these defenses scale to practical networks.
Recently, a new method called randomized smoothing has been proposed as a \textit{probabilistically} certified defense, whose architecture-independence makes it scalable.

\footnotetext{Unless stated otherwise, these models were trained with noise augmentation. In our replication of \citet{teng_ell_1_2019}, our noise augmentation results matched their adversarial training results.}

\paragraph{Randomized smoothing}

Randomized smoothing was first proposed as a heuristic defense without any guarantees~\citep{liu2018towards, cao2017mitigating}.
Later on, \citet{lecuyer2018certified} proved a robustness guarantee for smoothed classifiers from a differential privacy perspective.
Subsequently, \citet{li_certified_2019} gave a stronger robustness guarantee utilizing tools from information theory.
Recently, \citet{cohen_certified_2019} provided a tight $\ell_2$ robustness guarantee for randomized smoothing, applied by \citet{salman2020blackbox} to provably defend pre-trained models for the first time.
Furthermore, a series of papers came out recently that developed robustness guarantees against other adversaries such as $\ell_1$-bounded \citep{teng_ell_1_2019}, $\ell_\infty$-bounded~\cite{zhang*2020filling}, $\ell_0$-bounded~\cite{levine_robustness_2019,lee_tight_2019}, and Wasserstein attacks~\cite{levine_wasserstein_2019}.
In \cref{sec:main-comp}, we give a more in-depth comparison on how our techniques compare to their results.

\paragraph{Wulff Crystal}
We are the first to relate to adversarial robustness the theory of \emph{Wulff Crystals}.
Just as the round soap bubble minimizes surface tension for a given volume, the Wulff Crystal minimizes certain similar surface energy that arises when the crystal interfaces with another material.
The Russian physicist George Wulff first proposed this shape via physical arguments in 1901 \citep{wulffZurFrageGeschwindigkeit1901}, but its energy minimization property was not proven in full generality until relatively recently, building on a century worth of work \citep{gibbsEquilibriumHeterogeneousSubstances1875,wulffZurFrageGeschwindigkeit1901,hiltonMathematicalCrystallography1903,liebmannCurieWulffScheSatz1914,vonlaueWulffscheSatzFur,dinghasUberGcometrischenSatz1944,burtonGrowthCrystalsEquilibrium1951,herringKonferenzUberStruktur,constableKineticsMechanismCrystallization1968GoogleBooks,taylorUniqueStructureSolutions1975,taylorCrystallineVariationalProblems1978ProjectEuclid,fonseca_muller_1991,brothersIsoperimetricTheoremGeneral1994DOI.orgCrossref,cerfWulffCrystalIsing2006www.springer.com}.

\paragraph{No-go theorems for randomized smoothing} Prior to the initial submission of this manuscript, the only other no-go theorem for randomized smoothing in the context of adversarial robustness is~\citet{zheng2020a}.
However, they are only concerned with a non-standard notion of certified robustness that does not imply anything for the original problem.
Moreover, they show that, under this different notion of robustness, if they are robust for $\ell_\infty$, then the $\ell_2$ norm of the noise must be large on average.
While this provides indirect evidence for the hardness of certifying $\ell_\infty$, it does not actually address the question.
Our result, on the other hand, directly rules out a large suite of current techniques for deriving robust certificates for all $\ell_p$ norms for $p > 2$, for the standard notion of certified robustness.

After the initial submission of this manuscript, we became aware of two concurrent works~\cite{blum2020random,kumar2020curse} that claim impossibility results for randomized smoothing.
\citet{blum2020random} demonstrate that, under some mild conditions, any smoothing distribution for $\ell_p$ with $p > 2$ must have large component-wise magnitude.
This gives indirect evidence for the hardness of the problem, but does not directly show a limit for the utility of randomized smoothness for the robust classification problem, which we do in this work.
\citet{kumar2020curse} demonstrate that certain classes of smoothing distributions cannot certify $\ell_\infty$ without losing dimension-dependent factors.
Our result is more general, as it rules out \emph{any} class of smoothing distributions, and in fact, any smoothing scheme that allows the distribution to vary arbitrarily with the input point.

\section{Randomized Smoothing}

Consider a classifier $f$ from $\mathbb{R}^d$ to classes $\mathcal{Y}$ and a distribution $q$ on $\R^d$.
Randomized smoothing with $q$ is a method that constructs a new, \textit{smoothed} classifier $g$ from the \textit{base} classifier $f$.
The smoothed classifier $g$ assigns to a query point $x$ the class which is most likely to be returned by the base classifier $f$ when $x$ is perturbed by a random noise sampled from $q$, i.e.,
\begin{align}
g(x) &\overset{\mathrm{def}}{=} \argmax_{c \in \mathcal{Y}} q(U_c - x)
\label{eq:smoothed-hard}
\end{align}
where $U_c$ is the decision region $\{x' \in \R^d: f(x') = c\}$, $U_c - x$ denotes the translation of $U_c$ by $-x$, and $q(U)$ is the measure of $U$ under $q$, i.e.\ $q(U) = \Pr_{\delta \sim q}(\delta \in U)$.

\paragraph{Robustness guarantee for smoothed classifiers}

For $p \in [0, 1], v \in \R^d,$ define the \emph{growth function}
\begin{align*}
    \mathcal{G}_q(p, v) \overset{\mathrm{def}}{=}
    \sup_{U \sbe \R^d: q(U) = p} q(U - v),
    \numberthis\label{eqn:growth}
\end{align*}
One can think of $U$ has the decision region of some base classifier.
Thus $\mathcal{G}_q(p, v)$ gives the maximal growth of measure of a set (i.e. decision region) when $q$ is shifted by the vector $v$, if we only know the initial measure $p$ of the set.

Consider an adversary that can perturb an input additively by any vector $v$ inside an allowed set $\mathcal B$.
In the case when $\mathcal B$ is the $\ell_2$ ball and $q$ is the Gaussian measure, \citet{cohen_certified_2019} gave a simple expression for $\mathcal{G}_q$ involving the Gaussian CDF, derived via the Neyman-Pearson lemma, which is later rederived by \citet{salman2019provably} as a nonlinear Lipschitz property.
Likewise, the expression for Laplace distributions was derived by \citet{teng_ell_1_2019}.
(See \cref{thm:gaussianl2} and \cref{thm:laplaceL1} for their expressions.)

Suppose when the base classifier $f$ classifies $x + \delta$, $\delta \sim q$, the class $c \in \mathcal Y$ is returned with probability $\rho = \Pr_{\delta \sim q} (f(x + \delta) = c) > 1/2$.
Then the smoothed classifier $g$ will not change its prediction under the adversary's perturbations if~\footnote{Many earlier works state robustness guarantees in terms of estimates of $p_A = \rho$ of the top class and $p_B$ of the runner up class; however, their implementations are all in the form provided here, as $p_B$ is usually taken to be $1 - p_A$.}
\begin{equation}
    \sup_{v \in \mathcal B} \mathcal{G}_q(1-\rho, v) < 1/2.
    %
    \label{eqn:robustguarantee}
\end{equation}

\section{Methods for Deriving Robust Radii}
Let $q$ be a distribution with a density function, and we shall write $q(x), x \in \R^d$, for the value of the density function on $x$.
Then, given a shift vector $v \in \R^d$ and a ratio $\kappa > 0$, define the \emph{Neyman-Pearson set}
\begin{equation}
    \mathcal{NP}_\kappa \overset{\mathrm{def}}{=} \{x \in \R^d: \kappa q(x-v) \ge q(x)\}.
    \label{eqn:neymanpearsonset}
\end{equation}
Then the Neyman-Pearson lemma tells us that \cite{neyman_ix_1933,cohen_certified_2019}
\begin{align}
    \mathcal{G}_q(q(\mathcal{NP}_\kappa), v) = q(\mathcal{NP}_\kappa - v).
    \label{eqn:NP}
    \tag{NP}
\end{align}
While this gives way to a simple expression for the growth function when $q$ is Gaussian \cite{cohen_certified_2019}, it is difficult for more general distributions as the geometry of $\mathcal{NP}_\kappa$ becomes hard to grasp.
To overcome this difficulty, we propose the \emph{level set method} that  decomposes this geometry so as to compute the growth function exactly, and the \emph{differential method} that upper bounds the growth function derivative, loosely speaking.

\subsection{The Level Set Method}

For each $t > 0$, let $U_t$ be the superlevel set
\begin{align*}
    U_t \overset{\mathrm{def}}{=} \{x \in \R^d: q(x) \ge t\}.
\end{align*}
Then its boundary $\pd U_t$ is the level set with $q(x) = t$ under regularity assumptions.
The integral of $q$'s density is of course 1, but this integral can be expressed as the integral of the volumes of its superlevel sets:
\begin{equation}
    1 = \int q(x) \dd x = \int_0^\infty \mathrm{Vol}(U_t) \dd t.
    \label{eqn:suplevelint}
    \tag{$\bullet$}
\end{equation}
If $q$ has a differentiable density, then we may rewrite this as an integral of \emph{level} sets (\cref{thm:coarea}):
\begin{equation}
    1 = \int_0^\infty \int_{\pd U_t} \f t{\|\nabla q(x)\|_2} \dd x \dd t.
    \label{eqn:levelint}
    \tag{$\circ$}
\end{equation}
\begin{center}
    %
    \includegraphics[width=0.35\textwidth]{graphics/levelset_cropped.pdf}
    %
    %
\end{center}
The graphics above illustrate the two integral expressions (best viewed on screen).
In this level set perspective, the Neyman-Pearson set $\mathcal{NP}_\kappa$ (\cref{eqn:neymanpearsonset}) can be written as
\begin{align*}
    \mathcal{NP}_\kappa &=
        \bigcup_{t > 0} \{x: q(x) = t \text{  and  } q(x - v) \ge t / \kappa\}\\
    &=
    \bigcup_{t > 0} \{\pd U_t \cap (U_{t / \kappa} + v)\}.
\end{align*}
Then naturally, its measure is calculated by
\begin{align}
    q(\mathcal{NP}_\kappa)
    &=
        \int_0^\infty \int_{\pd U_t \cap (U_{t / \kappa} + v)}\f t{\|\nabla q(x)\|_2} \dd x \dd t.
    \label{eqn:psmall}
    \tag{$\vee$}
\end{align}
Similarly, the Neyman-Pearson set can also be written from the perspective of $q(\cdot - v)$,
\begin{align*}
    \mathcal{NP}_\kappa
    &=
        \bigcup_{t > 0}\{x : q(x - v) = t \text{  and  } q(x) \le t \kappa \}\\
    &=
        \bigcup_{t > 0} \{ (\pd U_t + v) \setminus \mathring U_{t\kappa}\},
\end{align*}
where $\mathring U$ is the interior of the closed set $U$.
So its measure under $q(\cdot - v)$ is
\begin{align}
    q(\mathcal{NP}_\kappa - v)
    &=
        \!\int_0^\infty\!\!\!\! \int_{\pd U_t \setminus (\mathring U_{t\kappa} - v)}
         \f{t}{\|\nabla q(x)\|_2} \dd x \dd t.
    \label{eqn:pbig}
    \tag{$\wedge$}
\end{align}
\begin{center}
    \includegraphics[width=0.45\textwidth]{graphics/neyman_pearson_levelset_cropped.pdf}
\end{center}
The graphics above illustrate the integration domains of $x$ in \cref{eqn:psmall,eqn:pbig}.
In general, the geometry of $\pd U_t \cap (U_{t/\kappa} + v)$ or $\pd U_t \setminus (\mathring U_{t\kappa} - v)$ is still difficult to handle, but in highly symmetric cases when $U_t$ are concentric balls or cubes, \cref{eqn:psmall,eqn:pbig} can be calculated efficiently.

\paragraph{Computing Robust Radius}
\cref{eqn:psmall,eqn:pbig} allow us to compute the growth function by \cref{eqn:NP}.
In general, this yields an \emph{upper bound} of the robust radius
\begin{align*}
    %
        &\phantomeq
            \sup \left\{r: \sup_{\|v\|_p \le r} \mathcal{G}_q(1-\rho, v) < 1/2\right\}\\
        &\le
            \sup \left\{r: \mathcal{G}_q(1-\rho, r u) < 1/2\right\}
\end{align*}
for any particular $u$ with $\|u\|_p = 1$.
With sufficient symmetry, e.g. with $\ell_2$ adversary and distributions with spherical level sets, this upper bound becomes \emph{tight} for well-chosen $u$, and we can build a lookup table of certified radii.
See \cref{alg:levelsettable,alg:levelsetcert}.

\begin{algorithm}[tb]
   \caption{Pre-Computing Robust Radius Table
   via Level Set Method for Spherical Distributions Againt $\ell_2$ Adversary}
   \label{alg:levelsettable}
\begin{algorithmic}
   \STATE {\bfseries Input:} Radii $r_1 < \ldots < r_N$
   \STATE Initialize $u=(1, 0, \ldots, 0) \in \R^d$.
   \FOR{$i=1$ {\bfseries to} $N$}
   \STATE Find $\kappa$ s.t.\ $q(\mathcal{NP}_\kappa-r_i u) = 1/2$ (via \cref{eqn:pbig} or \cref{thm:levelsetl2}) by binary search
   \STATE Compute $p_i \gets q(\mathcal{NP}_\kappa)$ via \cref{eqn:psmall} or \cref{thm:levelsetl2}
   \ENDFOR
   \STATE {\bfseries Output:} $p_1 > \cdots > p_N$
\end{algorithmic}
\end{algorithm}

\begin{algorithm}[tb]
    \caption{Certification with Table}
    \label{alg:levelsetcert}
\begin{algorithmic}
   \STATE {\bfseries Input:} Probability of correct class $\rho$
   \STATE {\bfseries Output:} Look up $r_i$ where $p_i \ge 1-\rho > p_{i+1}$
\end{algorithmic}
\end{algorithm}

\begin{figure*}
    \centering
    \includegraphics[width=\linewidth]{graphics/smoothingdist_summary_copy_cropped.pdf}
    \caption{\textbf{Smoothing distributions for which we derive robustness guarantees in this paper.}
    Each box represents a family of distributions that obtain guarantees through similar proofs.
    Text beside each box indicates the name of the family and the $\ell_p$ adversaries against which we have guarantees.
    \emph{Log Convex*} means log convex on the positive and negative half lines, but not necessarily on the whole line.
    The color indicates the basic technique used, among the two proposed techniques in this paper.
    We explicitly list example densities in each box.
    For the robust radii formulas, see \cref{tab:bounds}.
    }
    \label{fig:distribs}
\end{figure*}

\subsection{The Differential Method}

To derive certification (robust radius \emph{lower bounds}) for more general distributions, we propose a \emph{differential method}, which can be thought of as a vast generalization of the proof in \citet{salman2019provably} of the Gaussian robust radius.
The idea is to compute the largest possible \emph{infinitesimal increase in $q$-measure} due to an \emph{infinitesimal adversarial perturbation}.
More precisely, given a norm $\| \cdot \|$, and a smoothing measure $q$, we define
\begin{align}
    \Phi(p) \overset{\mathrm{def}}{=} \sup_{\|v\|=1} \sup_{U \sbe \R^d: q(U) = p} \lim_{r \searrow 0} \f{q(U-rv) - p}r.
    \label{eqn:PhiMaintext}
\end{align}
Intuitively, one can then think of $1/\Phi(p)$ as the \emph{smallest} possible perturbation in $\|\cdot\|$ needed to effect a unit of infinitesimal increase in $p$.
Therefore,
\begin{thm}[\cref{thm:differentialMethodMasterTheorem}]\label{thm:differentialMethodMainText}
The robust radius in $\| \cdot \|$ is at least
\begin{align*}
    R \overset{\mathrm{def}}{=} \int_{1-\rho}^{1/2} \f 1 {\Phi(p)} \dd p,
\end{align*}
where $\rho$ is the probability that the base classifier predicts the right label under random perturbation by $q$.
\end{thm}
By exchanging differentiation and integration and applying a similar greedy reasoning as in the Neyman-Pearson lemma, $\Phi(p)$ can be derived for many distributions $q$ and integrated symbolically to obtain expressions for $R$.
We demonstrate the technique with a simple example below, but much of it can be automated; see \cref{thm:differentialMethodMasterTheorem}.

\begin{exmp}[see \cref{thm:ExpInfL1}]\label{exmp:explinf}
If the smoothing distribution is $q(x) \propto \exp(-\|x\|_\infty/\lambda)$, then the robust radius against an $\ell_1$ adversary is at least
\[R = 2d \lambda (\rho - 1/2),\]
when $\rho$ is the probability of the correct class as in \cref{thm:differentialMethodMainText}.
\end{exmp}
\begin{proof}[Proof Sketch]
By linearity in $\lambda$, we WLOG assume $\lambda = 1$.
By \cref{thm:differentialMethodMainText} and the monotonicity of $\Phi$, it suffices to show that $\Phi(p) = 1/2d$ for $p \ge 1/2d.$
For any fixed $U$ with $q(U) = p$,
\begin{align*}
    \lim_{r \searrow 0} \f{q(U-rv) - p}r
    &= \f{d}{dr} \left.\int_U q(x-rv) \dd x\right|_{r=0}
        \\
    &= \int_U \la v, \nabla q(x)\ra \dd x.
\end{align*}
Note $\nabla q(x) = e_x q(x)$, where $e_x = \sgn(x_{i^*}) e_{i^*}$, $e_i$ is the $i$th unit vector, and $i^* = \argmax_i |x_i|$.
Additionally, the above integral is linear in $v$, so the supremum over $\|v\|_1 = 1$ is achieved on one of the vertices of the $\ell_1$ ball.
So we may WLOG consider only $v = \pm e_i$; furthermore, due to symmetry of $\nabla q(x)$, we can just assume $v = e_1$:
\begin{align*}
    \Phi(p)
        &=
            \sup_{U} \lim_{r \searrow 0} \f{q(U-re_1) - p}r
            %
        =
            \sup_{U}
            \int_U \la e_1, e_x\ra q(x) \dd x,
\end{align*}
where $U$ ranges over all $q(U) = p$.
Note $\la e_1, e_x \ra = 0$ if $i^* \ne 1$, and $\sgn(x_{i^*})$ otherwise.
Thus, to maximize $\lim_{r \searrow 0} \f{q(U-re_1) - p}r$ subject to the constraint that $q(U) = p$, we should put as much $q$-mass on those $x$ with large $\la e_1, e_x \ra$.
For $p \ge 1/2d$, we thus should occupy the entire region $\{x: \la e_1, e_x\ra = 1\}$, which has $q$-mass $1/2d$, and then assign the rest of the $q$-mass (amounting to $p - 1/2d$) to the region $\{x: \la e_1, e_x\ra = 0\}$, which has $q$-mass $1 - 1/d$.
This shows that
\[\Phi(p) = 1/2d, \quad\forall p \in [1/2d, 1-1/2d]\]
as desired.
\end{proof}

\subsection{Comparison of the Two Methods and Prior Works}
\label{sec:main-comp}

We summarize the distributions our methods cover in \cref{fig:distribs} and the bounds we derive in \cref{tab:bounds}.
We highlight a few broadly applicable robustness guarantees:
\begin{exmp}[\cref{thm:iidlogconcaveL1}]\label{exmp:iidlogconcaveL1}
Let $\phi: \R \to \R$ be convex and even, and let  $\mathrm{CDF}_\phi^{-1}$ be the inverse CDF of the 1D random variable with density $\propto \exp(-\phi(x))$.
If $q(x) \propto \prod_i e^{-\phi(x_i)}$, and $\rho$ is the probability of the correct class, then the robust radius in $\ell_1$ is
\begin{align*}
    R = \mathrm{CDF}_\phi^{-1}(\rho)
\end{align*}
and this radius is \emph{tight}.
This in particular recovers the Gaussian bound of \citet{cohen_certified_2019}, Laplace bound of \citet{teng_ell_1_2019}, and Uniform bound of \citet{lee_tight_2019} in the setting of $\ell_1$ adversary.
\end{exmp}

\begin{exmp}[\cref{sec:cubicalExponentialL1,sec:cubicalPowerL1}]
Facing an $\ell_1$ adversary, cubical distributions, like that in \cref{exmp:explinf}, typically enjoy, via the differential method, $\ell_1$ robust radii of the form
\begin{align*}
    R = c(\rho - 1/2)
\end{align*}
for some constant $c$ depending on the distribution.
\end{exmp}

In general, the level set method always gives certificate as tight as Neyman-Pearson, while the differential method is tight only for infinitesimal perturbations, but can be shown to be tight for certain families, like in \cref{exmp:iidlogconcaveL1} above.
On the other hand, the latter will often give efficiently evaluable symbolic expressions and apply to more general distributions, while the former in general will only yield a table of robust radii, and only for distributions whose level sets are sufficiently symmetric (such as a sphere or cube).

For distributions that are covered by both methods, we compare the bounds obtained and note that the differential and level set methods yield almost identical robustness certificates in high dimensions (e.g. number of pixels in CIFAR-10 or ImageNet images).
See \cref{sec:levelset_vs_differential}.

Many earlier works used differential privacy or $f$-divergence methods to compute robust radii of smoothed models \citep{lecuyer2018certified,li_certified_2019,dvijotham_framework_2019}.
In particular, \citet{dvijotham_framework_2019} proposed a general $f$-divergence framework that subsumed all such works.
Our robust radii are computed only from $\rho$; \citeauthor{dvijotham_framework_2019} called this the ``information-limited'' setting, and we shall compare with their robustness guarantees of this type.
While their algorithm in a certain limit becomes as good as Neyman-Pearson, in practice outside the Gaussian distribution, their robust radii are too loose.
This is evident by comparing our baseline Laplace results in \cref{tab:sota} with theirs, which are trained the same way.
Additionally, our differential method often yields symbolic expressions for robust radii, making the certification algorithm easy to implement, verify, and run.
Moreover, we derive robustness guarantees for many more (distributions, adversary) pairs (\cref{fig:distribs,tab:bounds}).
See \cref{sec:fdivCompare} for a more detailed comparison.

\section{Wulff Crystals}
\label{sec:wulffMain}
A priori, it is a daunting task to understand the relationship between the adversary $\mathcal B$ and the smoothing distribution $q$.
In this section, we shall begin our investigation by looking at uniform distributions, and then end with an optimality theorem for all ``reasonable'' distributions.

Let $q$ be the uniform distribution supported on a measurable set $S \sbe \R^d$.
WLOG, assume $S$ has (Lebesgue) volume 1, $\mathrm{Vol}(S) = 1$.
Then for any $v \in \R^d$ and any $p \in [0, 1]$,
\begin{align*}
    \mathcal{G}_q(p, v) = \min\left(1, p + {\mathrm{Vol}((S + v) \setminus S)}\right).
\end{align*}
\begin{wrapfigure}{r}{0.18\textwidth}
\begin{center}
    \includegraphics[width=0.15\textwidth]{graphics/unif_optimal_U.pdf}
\end{center}
\end{wrapfigure}
This can be seen easily by taking $U$ in \cref{eqn:growth} to be a subset of $(S + v) \cap S$ with volume $p$ (or any set of volume $p$ containing $(S + v) \cap S$ if $p \ge \mathrm{Vol}((S + v) \cap S)$) unioned with the complement of $S$.
For example, in the figure here, $U$ would be the gray region, if $U \cap S$ has volume $p$.

If $S$ is convex, and we take $v$ to be an infinitesimal translation, then the RHS above is infinitesimally larger than $p$, as follows:
\begin{align*}
    \lim_{r \to 0} \f{\mathcal{G}_q(p, rv) - p}r &= \lim_{r \to 0} \f{\mathrm{Vol}((S + r v) \setminus S)}r\\
        &= \|v\|_2 \mathrm{Vol}(\Pi_{v} S)
        \numberthis \label{eqn:differentialUniform}
\end{align*}
\begin{wrapfigure}{r}{0.2\textwidth}
\vspace{-20pt}
\begin{center}
    \includegraphics[width=0.2\textwidth]{graphics/leak_cropped.pdf}
\end{center}
\vspace{-10pt}
\end{wrapfigure}
where $\Pi_{v} S$ is the projection of $S$ along the direction $v/\|v\|_2$, and $\mathrm{Vol}(\Pi_{v} S)$ is its $(d-1)$-dimensional Lebesgue measure.
A similar formula holds when $S$ is not convex as well (\cref{eqn:leakage}).
In the context of randomized smoothing, this means that the classifier $g$ smoothed by $q$ is robust at $x$ under a perturbation
$\f{\f 1 2 - p}{\|v\|_2 \mathrm{Vol}(\Pi_{v} S)} v$
when $1/2 - p$ is small, and $p$ is the probability the base classifier $f$ \emph{mis}classifies $x + \delta$, $\delta \sim q$.
Thus, for $r$ small, we have
\begin{align*}
    \sup_{v \in r \mathcal B} \mathcal{G}_q(p, v) \approx p + r \sup_{v \in \mathcal B} \|v\|_2 \mathrm{Vol}(\Pi_{v} S) = p + r\Phi(p),
\end{align*}
with $\Phi$ as in \cref{eqn:PhiMaintext}.
The smaller $\sup_{v \in \mathcal B} \|v\|_2 \mathrm{Vol}(\Pi_{v} S)$ is, the more robust the smoothed classifier $g$ is, for a fixed $p$.
A natural question, then, is: among convex sets of volume 1,
\begin{align*}
    \text{which set $S$ minimizes $\Phi = \sup_{v \in \mathcal B} \|v\|_2 \mathrm{Vol}(\Pi_{v} S)$?}
    \label{eqn:minquestion}
\end{align*}

If $\mathcal B$ is the $\ell_p$ ball, the reader might guess $S$ should either be the $\ell_p$ ball or the $\ell_r$ ball with $\f 1 r + \f 1 p = 1$.
It turns out the correct answer, at least in the case when $\mathcal B$ is a highly symmetric polytope (e.g.\ $\ell_1, \ell_2, \ell_\infty$ balls), is a kind of \emph{energy-minimizing} crystals studied in physics since 1901 (\citeauthor{wulffZurFrageGeschwindigkeit1901}).
\begin{defn}\label{defn:wulff}
The \emph{Wulff Crystal} (w.r.t.\ $\mathcal B$) is defined as the unit ball of the norm dual to $\|\cdot\|_*$, where $\|x\|_* = \EV_{y \sim \mathrm{Vert}(\mathcal B)} |\la x, y \ra|$ and $y$ is sampled uniformly from the vertices of $\mathcal B$~\footnote{When $\mathcal B$ is the $\ell_2$ ball, $\mathrm{Vert}(\mathcal B)$ is the entire boundary.}.
\end{defn}
In fact, Wulff Crystals solve the more general problem without convexity constraint.
\begin{thm}[\cref{thm:wulffOptimalFull}, informal]\label{thm:wulffOptimalMainText}
The Wulff Crystal w.r.t.\ $\mathcal B$ minimizes
\[\Phi = \sup_{v \in \mathcal B} \lim_{r \to 0} \inv r \mathrm{Vol}((S + rv) \setminus S)\]
among all measurable (not necessarily convex) sets $S$ of the same volume, when $\mathcal B$ is sufficiently symmetric (e.g. $\ell_1, \ell_2,\ell_\infty$ balls).
\end{thm}
When $\mathrm{Vert}(\mathcal B)$ is a finite set, the Wulff Crystal has an elegant description as the \emph{zonotope} of $\mathrm{Vert}(\mathcal B)$, i.e.\ the Minkowski sum of the vertices of $\mathcal B$ as vectors (\cref{prop:wulffzonotope}), from which we can derive the following examples.
\begin{exmp}
The Wulff Crystal w.r.t. $\ell_2$ ball is the $\ell_2$ ball itself.
The Wulff Crystal w.r.t. $\ell_1$ ball is a cube ($\ell_\infty$ ball).
The Wulff Crystal w.r.t. $\ell_\infty$ in 2 dimensions is a rhombus; in 3 dimensions, it is a rhombic dodecahedron; in higher dimension $d$, there is no simpler description of it other than the zonotope of the vectors $\{\pm1\}^d$.
\end{exmp}

\begin{center}
    \includegraphics[width=0.3\textwidth]{graphics/wulffcrystalexamples_cropped.pdf}
\end{center}

In fact, distributions with Wulff Crystal level sets more generally maximizes the robust radii for ``hard'' inputs.
\begin{thm}[\cref{thm:wulffOptimalGeneral}, informal]
Let $\mathcal B$ be sufficiently symmetric.
Let $q_0$ be any distribution with a ``reasonable''\footnote{\emph{Reasonable} here roughly means Sobolev, i.e. has weak derivative that is integrable, and this can be further relaxed to \emph{bounded variations}; for details see \cref{thm:wulffOptimalGeneral} and \cref{thm:wulffOptimalGeneralBV}.} and even density function.
Among all ``reasonable'' and even density functions $q$ whose superlevel sets $\{x: q(x) \ge t\}$ have the same volumes as those of $q_0$, the quantity
\[\Phi(1/2) = \sup_{v \in \mathcal B} \sup_{q(U) = 1/2} \lim_{r \searrow 0} \f{q(U-rv) - 1/2}r\]
is minimized by the unique distribution $q^*$ whose superlevel sets are proportional to the Wulff Crystal w.r.t.\ $\mathcal B$.
\end{thm}
This theorem implies that distributions with Wulff Crystal level sets give the best robust radii for those \emph{hard} inputs $x$ that a smooth classifier classifies correctly but only barely, in that the probability of the correct class $\rho = 1/2 + \epsilon$ for some small $\epsilon$.
The constraint on the volumes of superlevel sets indirectly controls the variance of the distribution.
While this theorem says nothing about the robust radii for $\rho$ away from $1/2$, we find the Wulff Crystal distributions empirically to be highly effective, as we describe next in \cref{sec:experiments}.

\section{Experiments}
\label{sec:experiments}

We empirically study the performance of different smoothing distributions on image classification datasets, using the bounds derived via the level set or the differential method, and verify predictions made by the Wulff Crystal theory.
We follow the experimental procedure in \citet{cohen_certified_2019} and further works on randomized smoothing \citep{salman2019provably,li_certified_2019,Zhai2020MACER} using ImageNet \citep{deng_imagenet_2009} and CIFAR-10 \citep{krizhevsky_learning_2009}.

The certified accuracy at a radius $\epsilon$ is defined as the fraction of the test set for which the smoothed classifier $g$ correctly classifies and certifies robust at an $\ell_p$ radius of $\epsilon$.
All results were certified with $N=100,000$ samples and failure probability $\alpha=0.001$.
For each distribution $q$, we train models across a range of scale parameter $\lambda$ (see\ \cref{tab:bounds}), corresponding to the same range of noise variances $\sigma^2 \overset{\mathrm{def}}{=} \EV_{\delta \sim q}[\frac{1}{d}\|\delta\|^2_2]$ across different distributions.
Then we calculate for each model the certified accuracies across the range of considered $\epsilon$.
Finally, in our plots, we present, for each distribution, the upper envelopes of certified accuracies attained over the range of considered $\sigma$.
Further details of experimental procedures are described in \cref{sec:experiments_details}.

We focus on the effect of the noise distribution in this section and only train models with noise augmentation.
In \cref{sec:experiments_details} we also study (1) stability training, and (2) the use of more data through (a) pre-training on downsampled ImageNet \citep{hendrycks_using_2019} and (b) semi-supervised self-training with data from 80 Million Tiny Images \citep{carmon_unlabeled_2019}. As shown in Table \ref{tab:sota}, these techniques further improve upon our results in this section.

\subsection{\texorpdfstring{$\ell_1$ Adversary}{L1 Adversary}}

\begin{figure}[t]
    \centering
    \caption{\textbf{SOTA $\ell_1$ Certified Accuracies.} Certified $\ell_1$ top-1 accuracies for ImageNet (left) and CIFAR-10 (right). For each distribution $q$, we train models across a range of $\sigma^2 \overset{\mathrm{def}}{=} \EV_{\delta \sim q}[\frac{1}{d}\|\delta\|_2^2]$, and at each level of $\ell_1$ adversarial perturbation radius $\epsilon$ we report the best certified accuracy.}
    \begin{subfigure}[t]{0.49\linewidth}
        \caption{ImageNet}
        \includegraphics[width=\linewidth]{graphics/imagenet_certified_accuracies_l1.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.49\linewidth}
        \caption{CIFAR-10}
        \includegraphics[width=\linewidth]{graphics/cifar_certified_accuracies_l1.pdf}
    \end{subfigure}
    \label{fig:l1_certificates}
\end{figure}

As previously mentioned, the Wulff Crystal for the $\ell_1$ ball is a cube.
With this motivation, we explore certified accuracies attained by distributions with cubical level sets.
\begin{enumerate}[nosep]
    \item Uniform, $\propto \ind(\|x\|_\infty \le \lambda)$
    \item Exponential, $\propto \|x\|_\infty^{-j} e^{-\|x/\lambda\|^k_\infty}$
    \item Power law, $\propto (1 + \|x/\lambda\|_\infty)^{-a}$
\end{enumerate}
We compare to previous state-of-the-art approaches using the Gaussian and Laplace distributions, as well as new non-cubical distributions.
\begin{enumerate}[nosep]
    \setcounter{enumi}{3}
    \item Exponential $\ell_1$ (non-cubical), $\propto \|x\|_1^{-j} e^{-\|x/\lambda\|_1^k}$
    \item Pareto i.i.d. (non-cubical), $\propto \prod_i (1 + |x_i|/\lambda)^{-a}.$
\end{enumerate}
The relevant certified bounds are given in \cref{tab:bounds}.

We obtain state-of-the-art robust certificates for ImageNet and CIFAR-10, finding that the Uniform distribution performs best, significantly better than the Gaussian and Laplace distributions (\cref{tab:sota}, \cref{fig:l1_certificates}).
The other distributions with cubic level sets match but do not exceed the performance of Uniform distribution, after sweeping hyper-parameters.
This verifies that distributions with cubical level sets are significantly better for $\ell_1$ certified accuracy than those with spherical or cross-polytope level sets.
See results for other distributions in \cref{sec:additional_results}.

\subsection{\texorpdfstring{$\ell_2$ Adversary}{L2 Adversary}}

\begin{figure}[t]
    \centering
    \caption{CIFAR-10 certified accuracies for $\ell_2$ (left) and $\ell_\infty$ (right) adversaries. For each distribution $q$ we train models across a range of $\sigma^2 \overset{\mathrm{def}}{=} \mathbb{E}[\frac{1}{d}\|\delta\|_2^2]$, and at each level of $\ell_p$ adversarial perturbation radius $\epsilon$, we pick the model that maximizes certified accuracy.}
    \begin{subfigure}[t]{0.49\linewidth}
        \includegraphics[width=\linewidth]{graphics/cifar_certified_accuracies_l2.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.49\linewidth}
        \includegraphics[width=\linewidth]{graphics/cifar_certified_accuracies_linf.pdf}
    \end{subfigure}
    \label{fig:l2_certificates}
    \vspace{-2em}
\end{figure}

The Wulff Crystal w.r.t.\ the $\ell_2$ ball is a sphere, so we explore distributions with spherical level sets (\cref{tab:bounds}):
 \begin{enumerate}[nosep]
     \item Uniform, $\propto \ind(\|x\|_2 \le \lambda)$
     \item Exponential, $\propto\|x\|_2^{-j} e^{-\|x/\lambda\|_2^k}$
     \item Power law, $\propto (1 + \|x/\lambda\|_2)^{-a}$
 \end{enumerate}
We find these distributions perform similarly to, though do not surpass the Gaussian (\cref{fig:l2_certificates}, left).

\subsection{\texorpdfstring{$\ell_\infty$ Adversary}{Linf Adversary}}

The Wulff Crystal for the $\ell_\infty$ ball is the zonotope of vectors $\{\pm 1\}^d$, which is a highly complex polytope hard to sample from and related to many open problems in polytope theory \citep{ziegler_lectures_1995}.
However, we can note that it is approximated by a sphere with constant ratio (\cref{prop:linfWulffCrystalBasic}), and in high dimension $d$, the sphere gets closer and closer to minimizing $\Phi$ (\cref{thm:wulffOptimalMainText}), but the cube and the cross polytope do not (\cref{claim:linfSphereOptimal}).
Accordingly, we find that distributions with spherical level sets outperform those with cubical or cross polytope level sets in certifying $\ell_\infty$ robustness (\cref{fig:l2_certificates}, right).
In fact, in the next section we show that up to a dimension-independent factor, the Gaussian distribution is optimal for defending against $\ell_\infty$ adversary if we don't use a more powerful technique than Neyman-Pearson.

    %
    %

\section{No-Go Results for Randomized Smoothing}

Recall that given a smoothing distribution $q$, a point $x \in \R^d$, and a binary base classifier $U \sbe \R^d$ (identified wth its decision region), the smoothed classifier outputs $\sgn(\rho - 1/2)$ where $\rho = q(U - x)$ is the ``confidence'' of this prediction (\cref{eq:smoothed-hard}).
Randomized smoothing (via Neyman-Pearson) tells us that, if $\rho$ is large enough, then, no matter what $U$ is, a small perturbation of $x$ cannot decrease $\rho$ too much to change $\sgn(\rho - 1/2)$ (\cref{eqn:robustguarantee}).

If all we care about is robustness, then the optimal strategy would set $q$ to be an arbitrarily wide distribution (say, e.g. a wide Gaussian), and the resulting smoothed classifier is roughly constant.
Of course, such a smoothed classifier can never achieve good clean accuracy, so it is not useful.
Thus there is an inherent tension between 1) having to have large enough noise variance to be robust and 2) having to have small enough noise variance to avoid trivializing the smoothed classifier.
In this section, we seek to formalize this tradeoff.
As we'll show, even if we only assume a very weak condition on the accuracy, we can show strong upper bounds on the best robust radius for each $\ell_p$ norm.

In fact, our negative results below will hold for a more general class of smoothing schemes than those in our positive results in previous sections:
In what follows, a \emph{smoothing scheme} for $\R^d$ is any family of probability distributions $\mathcal{Q} = \{ q_x\}_{x \in \R^d}$.
In practice, including in our paper, almost all smoothing schemes are \emph{translational}, that is, there is some base distribution $q$, and for every $x$, the smoothing distribution at $x$ is defined by $q_x (U) = q(U - x)$, for all base classifiers $U \subseteq \R^d$.
The above discussion then motivates the following
\begin{defn}
\label{def:useful}
Let $\| \cdot \|$ be a norm over $\R^d$, and let $\mathcal{Q} = \{q_x\}_{x \in \R^d}$ be a smoothing scheme for $\R^d$.
We say that $\mathcal{Q}$ satisfies \emph{$(\varepsilon, s, \ell)$-useful smoothing} with respect to $\| \cdot \|$ if:
\begin{enumerate}[nosep]
    \item {\bf ($(\varepsilon, s)$-Robustness)} For all $x, y$ with $\| x- y \| < \varepsilon$, if $U \sbe \R^d$ is any set (\emph{read: base classifier}) satisfying $q_x (U) \geq 1/2 + s$, then $q_y (U) \geq 1/2$.
    \item {\bf ($\ell$-Accuracy)} For all $x, y$ with $\| x - y \| \geq 1$, there exists a set (\emph{read: base classifier}) $U \subseteq \R^d$ so that $|q_x (U) - q_y(U)| \ge \ell$.
\end{enumerate}
\end{defn}

\begin{center}
    \includegraphics[width=0.45\textwidth]{graphics/robustness_v_accuracy_cropped.pdf}
\end{center}

We pause to interpret this definition.
Condition (1) indicates how large the certified radii can be for a classifier at any given point $x$, if the smoothed classifier assigns likelihood at least $1/2 + s$ to it; i.e. $(1/2 + s, \varepsilon)$ is a point on the robust radii curve in the style of \cref{fig:radii_vs_rho}.
The goal of the smoothing scheme is to achieve the largest possible $\varepsilon$, for every fixed $s$.
In particular, observe that for $\ell_2$, Gaussian smoothing achieves dimension-independent $\varepsilon$, for every fixed choice of $s$ (\cref{thm:gaussianl2}).
 %

Condition (2) says that the resulting smoothing should not ``collapse'' points: in particular, if $x, y$ are far in norm, then there should be some smoothed classifier that distinguishes them.
We argue that this is a very mild assumption.
For Condition (2) to be satisfied, the $U$ which distinguishes these two points can be completely arbitrary.
Thus, if it is violated for $\ell = o(1)$, the two distributions are indistinguishable by any statistical test in high dimension, implying the impossibility of classifying between $x$ and $y$ after smoothing.

We seek to show that, for constant $s$ and $l$, any $(\varepsilon, s, \ell)$-useful smoothing scheme must have $\varepsilon = o(1)$ for a number of norms, including $\ell_\infty$.
This would imply that any smoothing scheme that satisfying our weak notion of accuracy can only certify a vanishingly small radius, even when the confidence of the classifier is strictly bounded away from $1/2$ by a constant.

\vspace{1em}
\textbf{Randomized Smoothing as Metric Embedding}\ \
A smoothing scheme can be thought of as a mapping from a normed space supported on $\R^d$ to the space of distributions, e.g. each point $x$ is mapped to the distribution $q_x$.
We will show that \cref{def:useful} is roughly equivalent to a bi-Lipschitz condition on this mapping, where the target distributions are equipped with the total variation distance.
Then the existence of a \emph{useful} smoothing scheme is equivalent to whether $(\R^d, \|\cdot\|)$ can be embedded \emph{with low distortion} into the total variation space of distributions.
Classical mathematics has a definitive answer to this question in the form of a geometric invariant, called the \emph{cotype}.
\begin{defn}[see e.g.~\citet{wojtaszczyk1996banach}]
A normed space $T = (X, \| \cdot \|)$ is said to have \emph{cotype} $p$ for $2 \leq p \leq \infty$ if there exists $C$ such that for all finite sequences $x_1, \ldots, x_n \in X$, we have
\[\mathbb{E} \left[ \left\| \sum_{j = 1}^n \sigma_j x_j \right\| \right] \ge C^{-1} \left( \sum_{j = 1}^n \| x_i \|^p  \right)^{1/p},\]
where the $\sigma_j$ are independent Rademacher random variables.
The smallest such $C$ is denoted $C_p (T)$.
\end{defn}
When the underlying space of the normed space $T$ is $\R^d$, John's theorem~\citep{john1948extremum} implies that any norm has cotype $2$ with $C_2 (T) \leq O(d^{1/2})$.
Because $C_2$ lower bounds the distortion of a metric embedding of $T$, by the aforementioned connection with randomized smothing, $C_2$ also limits the usefulness of any smoothing scheme of $T$:

\begin{thm}
    \label{thm:main-lb}
    Let $T$ be any normed space over $\R^d$.
    %
    %
    There exist universal constants $c, K > 0$ so that any $(\varepsilon, s, \ell)$-useful smoothing scheme for $T$ with $s/\ell < c$ must have
    \[\varepsilon \le K \sqrt[4]{s / \ell} \cdot C_2(T)^{-1}.\]
\end{thm}
In particular, it is well-known that $C_2 ((\R^d, \| \cdot \|_p)) = \Omega (\max(1, d^{1/2 - 1/p}))$, for all $p \in [1, \infty]$.
Thus, as an immediately corollary, we get:
\begin{cor}\label{cor:lp-lb}
For the value of $c$ in \cref{thm:main-lb} and for $p \in [1, \infty]$, any $(\varepsilon, s, \ell)$-useful smoothing scheme for $(\R^d, \| \cdot \|_p)$ with $s/\ell < c$ must have
\[\varepsilon \le O(\min(1, d^{-1/2 + 1/p})).\]
\end{cor}
It is easy to see that, up to constants, the Gaussian smoothing scheme achieves equality, and thus is optimal (in terms of dimension dependence), for all $p \in [1, \infty]$.

\paragraph{Discussion}
After \citet{cohen_certified_2019} showed the surprising scalability of Gaussian randomized smoothing to high-dimensional $\ell_2$-robust classification problems, many anticipated that this can be extended to $\ell_\infty$ as well.
One might also hope that, even though it seems like we cannot certify $\ell_2$ radius that grows with input dimension, we could do so for $\ell_1$.
But \cref{thm:main-lb,cor:lp-lb} present a strong barrier to such hopes.
In words:
\begin{center}
\it
\parbox{0.8\linewidth}{
Without using more than the information of the probability $\rho$ of correctly classifying an input under random noise, no smoothing techniques can certify nontrivial robust accuracy at $\ell_\infty$ radius $\Omega (d^{-1/2})$, or at $\ell_2$ or $\ell_1$ radius $\Omega(1)$.
}
\end{center}
Indeed, the $\ell_1$-radii we can obtain nontrivial certified accuracy at are on the same order between CIFAR10 and Imagenet (\cref{fig:l1_certificates}).

However, there are some ways to bypass this barrier.
For one, more information about the base classifier can be collected to produce better robustness certificates.
In fact, \citet{dvijotham_framework_2019} proposed a ``full-information'' algorithm that computes many moments of the base classifier in a convex optimization procedure to improve certified radius, but it is 100 times slower than the ``information-limited'' algorithms we discuss here that use only $\rho$.
It would be interesting to see whether this technique can be scaled up, and whether other methods can leverage more information\!~\footnote{
\citet{lee_tight_2019} also used the decision tree structure of their base classifier to improve $\ell_0$ certification, but the $\ell_0$-adversary does not fall within our framework.}.

Another route is to directly look for better randomized smoothing schemes for multi-class classification.
We formulated our no-go result in the setting of binary classification, and it is not clear whether a similarly strong barrier applies for multi-class classification.
However, current techniques for certification only look at the two most likely classes, and separately reason about how much each one can change by perturbing the input.
Our no-go result then straightforwardly applies to this case as well.

\section{Conclusion}

In this work, we have showed how far we can push randomized smoothing with different smoothing distributions against different $\ell_p$ adversaries, by presenting two new techniques for deriving robustness guarantees, by elucidating the geometry connecting the noise and the norm, and by empirically achieving state-of-the-art in $\ell_1$ provable defense.
At the same time, we have showed the limit current techniques face against $\ell_p$ adversaries when $p > 2$, especially $\ell_\infty$.
Our results point out ways to bypass this barrier, by either leveraging more information about the base classifier or by taking advantage of the multi-class problem structure better.
We wish to investigate both directions in the future.

More broadly, randomized smoothing is a method for inducing stability in a mechanism while maintaining utility --- precisely the bread and butter of differential privacy.
We suspect our methods for deriving robustness guarantees here and for optimizing the noise distribution can be useful in that setting as well, where Laplace and Gaussian noise dominate the discussion.
Whereas previous work \citet{lecuyer2018certified} has applied differential privacy tools to randomized smoothing, we hope to go the other way around in the future.

\newpage

\subsection*{Acknowledgements}
We thank Huan Zhang for brainstorming of ideas and performing a few experiments that unfortunately did not work out.
We also thank Aleksandar Nikolov, Sebastien Bubeck, Aleksander Madry, Zico Kolter, Nicholas Carlini, Judy Shen, Pengchuan Zhang, and Maksim Andriushchenko for discussions and feedback.

redpajama_set_name: RedPajamaArXiv


text:
Some corporate giants back away from Super Bowl ads
http://www.usatoday.com/money/advertising/2003-01-19-sb03-fear-factor_x.htm
http://story.news.yahoo.com/news?tmpl=story&u=/usatoday/20030120/bs_usatoday/4792479
Michael McCarthy USA TODAY
NEW YORK — While some marketers couldn't wait to buy a Super Bowl ad, a growing fear factor is keeping others on the sidelines.
Among the most prominent on the bench for ABC's Jan. 26 telecast are big guns that created some of the Super Bowl's most memorable spots: McDonald's, Nike and Coca-Cola. Once perennial in-game advertisers, they're sitting out for at least a fifth year.
They often now run ads in cheaper pre- or post-game shows, but it's about more than cost. The game's price has always divided the haves and have-nots.
"There's only a handful of advertisers who've been able to afford it since Day One," says Tom DeCabia, executive vice president of media buyer PHD USA.
An ad in the first game in 1967 was a then-pricey $42,000. This year, ABC has gotten up to $2.2 million for 30 seconds of ad time.
That makes the game advertising's ultimate cost/benefit risk, but many who can afford it are held back now by fear that the spotlight is too bright, with a growing number of media and industry observers judging best/worst ads, and that the penalty for failure is too high.
"You can become a hero — or find yourself the butt of jokes," notes Jerry Dow, director of worldwide marketing communications for United Airlines.
Careers are made and broken on the success of these high-profile ads. And the industry is mindful of how Super Bowl advertiser Just for Feet sued Saatchi & Saatchi for $10 million in 1999 for what the retailer essentially alleged was advertising malpractice.
Not everyone buys this hand wringing. Cheryl Berman, chairman and chief creative officer of Leo Burnett USA, who oversaw the famous 1993 "Nothing but Net" McDonald's ad starring Michael Jordan and Larry Bird, thinks the extra attention helps, not hurts, the $245 billion ad industry. "It's fun. We're all superstars that day. People actually talk about advertising."
Why some heavyweights are punting on the game again:
* Nike's 1992 Super Bowl spot with Jordan and Bugs Bunny, by Wieden & Kennedy, Portland, inspired the film Space Jam. But the swoosh will be out for a fifth year, leaving an opening for rival Reebok to return with its first spot since 1994. And Jordan is back — in spots for both Hanes and Gatorade.
Nike now thinks the Super Bowl audience is "too broad" and the timing isn't right for its promotions, says spokeswoman Celeste Alleyne.
* Coke and ad agency McCann-Erickson, New York created one of the game's most popular spots in 1980: Mean Joe Green with a little boy. But even though it just launched a major star-studded campaign for Coke Classic, it will leave the Super Bowl to Pepsi, which will be back in force.
* McDonald's has been out since 1996, though it still runs ads in cheaper pre-game programming, including a new spot this year by DDB Chicago. "We've been very pleased with the pre-kick position," says spokesman Palmer Moody. "It's delivered the audience we like to showcase our brand."
1976 XEROX – Monk Makes Miracle
2013 KIA FORTE – Hot Bots
2020 PRINGLES – Rick and Morty
2020 ROCKET MORTGAGE – Jason Momoa Comfortable
2018 E*TRADE – This Is Getting Old
2017 Kia Super Bowl 51 (LI) TV Commercial "Hero's Journey" Starring Melissa McCarthy