text:
FAA Home ▸ Airports ▸ Planning & Capacity ▸ Non-Federal Program for NavAids and AWOS ▸ Non-Federal Program: Overview
Non-Federal Program: Overview
Non-Federal air navigational aids (NavAids) and Automated Weather Observing Systems (AWOS) are required by law to be operated and maintained to the same standards as Federally owned systems. The Non-Federal Program ensures that:
New non-Federal systems are installed and commissioned according to FAA standards.
Existing non-Federal systems are operated and maintained according to FAA standards.
Non-Federally employed maintenance technicians have sufficient training and competence to meet FAA standards.
Non-Federal systems and technicians are licensed, as necessary, by the Federal Communications Commission (FCC).
New non-Federal systems are designed according to FAA standards.
The NFP enacts policies and procedures to regulate:
The approval process for newly-designed systems.
The life-cycles of those systems.*
The technicians who maintain those systems.
*A "life-cycle" includes installation, commissioning, operation, maintenance, takeover (if applicable), and ultimately, decommissioning.
Non-Federal systems play a significant role in the National Airspace System (NAS). For instance, they facilitate aviation access to hundreds of communities across the U.S. Without non-Federal systems, many of those communities would have little or no aviation access.
Key Elements of the Non Federal Process:
Before a new type/make/model of NavAid or AWOS can be used for non-Federal purposes, it must be approved by the Non-Federal Program.
Owners of non-Federal systems should not install them without Non-Federal Program approval and oversight.
The non-Federally employed technician(s) who will maintain the systems must first receive "verification authority" from the Non-Federal Program.
After a system is installed, the non-Federal technicians must maintain specific records on the system's operations and maintenance.
Non-Federal Program personnel will annually inspect the system's operations and maintenance.
If you don't comply with the above process, FAA may shut down your system.
The Non-Federal Process can be complicated. Contact your Non-Federal Program Liaison for advice and guidance!
Page last modified: May 23, 2018 5:11:20 PM EDT
Plan a non-Federal project
Technician of Record
Information about maintaining non-Federally owned systems, including AWOS
Learn how to request FAA approval for non-Federal use of your system
AWOS Owners
Learn valuable information on buying, operating, & maintaining AWOS
Program Liaisons (PDF)
Non-Federal Quick Reference Guide (PDF)
Guidance & Policy
AWOS-WMSCR FAQ (PDF)
WMSCR: FAA-Approved 3rd Party Service Providers (PDF)
AWOS Type Certification & Modifications
Airport Improvement Program (AIP) grants
Airport Capacity Profiles
Airports GIS Program
GA Airports: National Asset
Joint Civilian/Military (Joint-Use) Airports
National Plan of Integrated Airport Systems (NPIAS)
Non-Federal Program for NavAids and AWOS
List of Non-Federal Systems
Purchasing, Installing, & Operating Non-Federally Owned Systems
Maintaining Non-Federally Owned Systems
Requesting FAA Approval of Systems Intended for Non-Federal Use
Regulations, Advisory Circulars (ACs), & Policy
Buying, Operating, and Maintaining AWOS
Passenger and All-Cargo Statistics
Surplus Property & Military Base Conversions
This page was originally published at: https://www.faa.gov/airports/planning_capacity/non_federal/overview/


text:
Driving in the UK
Events in the UK
Living in England
UK Journal
You are here: Latest Updates » Travel » Europe » United Kingdom » England » England - Merseyside » Liverpool
Museum of Liverpool
Liverpool Cathedral (Anglican)
Liverpool Metropolitan Cathedral of Christ the King (Roman Catholic)
Royal Albert Dock
On the banks of the Mersey a relatively new addition is that of the re-vitalised Albert Dock. Home of a number of small shops, cafes, restaurants as well as the Liverpool branch of the Tate, Albert Dock is a great place to wander in peace and calm away from the hustle of the city only a short distance away.
The centre of the city is mostly full of shops (and offices) but on the periphery are the two iconic cathedrals and, on the water, the Royal Albert Dock, iconic Liver building and museums.
The modern Museum of Liverpool is located on the river adjacent to the Royal Albert Docks. An admission fee is charged.
The Liverpool Chinatown is a small area with a few shops and quite a number of surprisingly trendy restaurants (not only Chinese). Plenty of offices nearby so expect crowds in the weekdays during lunch and dinner.
The Liverpool Cathedral is the slightly less modern cathedral located a short distance across from the Liverpool Metropolitan Cathedral (see below).
The Liverpool Metropolitan Cathedral of Christ the King (Roman Catholic) is a modern cathedral that overlooks the city center. There is parking under the cathedral.
The revitalised riverbank now includes the Royal Albert Dock with a number of restaurants, shops and the Tate Liverpool art gallery.
If you are a Beatles fan then, obviously, you must visit the tiny underground Cavern Club on Mathew Street. This is where the young Beatles played a number of their first gigs. You enter through a small, rather unassuming door on the street to descend quite a large number of stairs. Be aware that a cover charge may be requested at busy times.
Additionally, there are a number of other Beatles tours of the city (including the Magical Mystery Tour bus operated by the Cavern Club]] taking in such sites as Strawberry Fields, Penny Lane, and the birth places of Lennon and McCartney (caution: sing along alert!).
Visitors to Liverpool will find there is a good amount of parking in the town centre. By rail most travel into Liverpool Lime Street station.
For further information, please see:
visitliverpool.com
Wikipedia - Liverpool
Wikitravel - Liverpool

redpajama_set_name: RedPajamaCommonCrawl


text:
Faith Front Page WorldWND EXCLUSIVE
U.N. warring against 'cultural foundations of the West'
After Temple Mount vote, they 'could say Parthenon unrelated to Greeks'
By Bob Unruh
The world was stunned just a few days ago when, as WND reported, the United Nations educational, scientific and cultural division voted that the Jews have no historical connection the Temple Mount.
That despite hard evidence that never has been refuted.
Now, several analysts have pointed out how far-fetched that conclusion is and how it plays into the hands of Islamic supremacists.
"No one would be surprised if, a few years from now, UNESCO, to institutionalize the Islamic presence in the international community, claims that Greeks have nothing to do with the Acropolis and the Parthenon, and that Italy has no historical ties to the Colosseum in Rome."
The comment comes from Maria Polizoidou, a consultant on international and foreign affairs who is based in Greece.
It follows the startling vote from UNESCO that denies all Jewish connection to the Temple Mount and Western Wall, insisting the historically Jewish sites be identified by Muslim names.
Shoshana Bryen, writing for the Gatestone Institute at the time of the vote, said the decision on the Palestinian-backed resolution "seems clearly a response to the expansionist, jihadist aspirations of members of the OIC who sponsored it: Algeria, Egypt, Lebanon, Morocco, Oman, Qatar and Sudan."
The OIC is the powerful Organization of Islamic Cooperation, which controls a 57-nation bloc of Islamic nations.
Polizoidou made clear a radical agenda is under way.
"When the news arrived that UNESCO does not recognize the connection of the Jewish people to Jerusalem's Temple Mount, it brought to mind that the U.N. is the mothership of injustice and radical global Islamization."
She said U.N. members "evidently believe that if they want to transform the Western world into an Islamic colony, first they must bring down the state of Israel."
"If Jerusalem falls into the hands of Islam, the rest of the world will presumably fall. UNESCO's decision is not only nonsensical from a historical perspective (Islam did not even exist at the time of ancient Jerusalem), basically it is also a strategy move against the cultural foundations of the West."
Jagdish Singh, a senior journalist based in New Delhi, joined in Polizoidou's concerns.
"After so many recent votes at UNESCO erasing Judeo-Christian history in favor of Islamist misrepresentation, and now in this fraudulent representation of Judaism's holiest site (a city more than twice as old as Islam) as primarily Muslim, one thing is clear: the sooner democracies leave the U.N., the better."
The claim to sever any connection between Jews and the Temple Mount came on a 24-6 vote. Supporting the move were Brazil, China, Dominican Republic, Egypt, Morocco, Mexico, Mozambique, Nicaragua, Nigeria, Oman, Russia, South Africa, Vietnam and others.
Six "no" votes were from Estonia, Germany, Lithuania, Netherlands, United Kingdom and United States.
Singh, also writing at the Gatestone Institute, said: "It is clear from the history of faiths that the holy scriptures of the Jews are far older than the Islamic ones. … UNESCO is supposed to help preserve the entire world's heritage, but this cultural agency seems to have been hijacked by the leadership and allies of the Palestinians and the Organization of Islamic Cooperation."
Also writing for Gatestone was Denis MacEoin, a distinguished senior fellow who has just finished a large study of Western concerns about Islam.
"Islam believes that it is eternal and had therefore preceded the other two great monotheisms, Judaism and Christianity, even though it was only to become visible to the world through Mohammad in the seventh century…," he wrote. "Lies by UNESCO to rewrite history, erasing all traces of Judaism and Christianity to favor a jihadist Islamic fancy, were already under way in 2015."
He noted at that time Islamists renamed two ancient biblical sites, Rachel's Tomb and the Cave of the Patriarchs, as Islamic sites.
"This is the history of Islam, how it takes over – with both hard jihad (violence) and soft jihad (usurping history, migration [hijrah], political and cultural infiltration), and intimidation (soft jihad) with the threat of hard jihad underneath it."
He added, "Before the U.N., with its authoritarian, anti-democratic voting blocs, finishes eradicating Western, Judeo-Christian civilization, as it is clearly trying to do, it is high time for Western democracies to run, not walk, away."
WND reported when the vote was held the Arab-Muslim denial of historic reality flies in the face of all archeological evidence.
Two weeks ago, it was reported that archaeologists found the battle site where Romans breached Jerusalem's walls. The discovery confirms Josephus's account of the conflict that saw the destruction of Second Temple nearly 2,000 years ago, says the Israeli Antiquities Authority.
During the dig, the archaeologists found the remains of a tower surrounded by scores of stones and boulders fired by Roman catapults at the Jewish forces guarding the wall, the Israel Antiquities Authority said in a statement.
"This is a fascinating testimony of the intensive bombardment by the Roman army, led by Titus, on their way to conquering the city and destroying the Second Temple," the statement said.
Rendition of the Roman destruction of Jerusalem in 70 A.D.
Israel's response was a flat-out rejection.
Israeli Education Minister Naftali Bennett informed UNESCO Director-General Irina Bokova of Israel's decision, Fox News reported.
"Following the shameful decision by UNESCO members to deny history and ignore thousands of years of Jewish ties to Jerusalem and the Temple Mount, I have notified the Israel National Commission for UNESCO to suspend all professional activities with the international organization," Bennett said.
Prime Minister Benjamin Netanyahu issued a statement on Facebook which he titled "The theatre of the absurd continues at the UN." The statement reads:
"Today UNESCO adopted its second decision this year denying the Jewish people's connection to the Temple Mount, our holiest site for over three thousand years.
"What's next? A UNESCO decision denying the connection between peanut butter and jelly? Batman and Robin? Rock and roll?
"Is it any wonder the UN has become a moral farce when UNESCO, the UN body tasked with preserving history, denies and distorts history?"
Lauria Cardoza-Moore, U.N. special envoy for the World Council of Independent Christian Churches and president of Proclaiming Justice to the Nations, said the correct answer now simply is to sever ties to the international body.
"I'm calling on the United States government to immediately rescind their ambassador to the United Nations Educational, Scientific and Cultural Organization and halt all funding to the organization."
WCICC represents over 40 million evangelical Christians worldwide and over 13,000 global PJTN Watchmen.
Bob Unruh
Bob Unruh joined WND in 2006 after nearly three decades with the Associated Press, as well as several Upper Midwest newspapers, where he covered everything from legislative battles and sports to tornadoes and homicidal survivalists. He is also a photographer whose scenic work has been used commercially.
DOJ announces 1st case by election threats task force
Poll shows voters would reject Hillary – for 3rd time


text:
El Consejo Superior de Deportes (CSD) de España es un organismo autónomo, adscrito al Ministerio de Cultura y Deporte, que ejerce directamente las competencias de la Administración General del Estado en el ámbito del deporte. Ejerce las funciones que le otorga la Ley del Deporte de 1990, su Estatuto y el resto del ordenamiento jurídico.

Para el control de las trampas en el deporte y asegurar la buena salud de los deportistas tiene adscrita la Comisión Española para la Lucha Antidopaje en el Deporte, comúnmente conocida como Agencia Estatal Antidopaje.

Historia
El CSD se crea mediante Real Decreto 2258/1977, de 27 de agosto y es directo heredero de la Delegación Nacional de Deportes, creada en 1941, dependiente de la Secretaría General del Movimiento.

Sus competencias esenciales se definieron en la Ley 13/1980, de 31 de marzo, General de la Cultura Física y del Deporte, y más tarde en la norma que la sustituyó: La Ley 10/1990, de 15 de octubre, del Deporte.

Dependencia orgánica
 Ministerio de Cultura (1977-1990).
 Ministerio de Educación y Ciencia (1990-1996).
 Ministerio de Educación y Cultura (1996-2000).
 Ministerio de Educación, Cultura y Deporte (2000-2004).
 Ministerio de Educación (2004-2009).
 Presidencia del Gobierno (2009-2011).
 Ministerio de Educación, Cultura y Deporte (2011-2018).
 Ministerio de Cultura y Deporte (2018- )

Funciones
Según la Ley del Deporte, las competencias del CSD son las siguientes:

Órganos de gobierno

Presidente/secretario de Estado para el Deporte
El presidente del Consejo Superior de Deportes, también conocido como secretario de Estado para el Deporte, es un alto cargo de la Administración General del Estado nombrado nombrado y separado por el Consejo de Ministros, a propuesta del ministro del departamento ministerial al que esté adscrito, y al que le corresponde ejercer la representación y superior dirección del Organismo, preside la Comisión Directiva y ejerce todas las funciones relativas a los asuntos económico-presupuestarios, contratación, recursos humanos, etc.

El presidente es sustituido en los casos de vacante, ausencia y enfermedad por el director general de Deportes.

Lista de presidentes

Comisión Directiva
Es el órgano rector del Consejo Superior de Deportes. Es presidido por el Presidente del CSD y la vicepresidencia es ejercida por el Director General de Deportes. Asimismo, está integrado por representantes de la Administración General del Estado, las comunidades autónomas, las entidades locales, las Federaciones deportivas españolas, del Comité Olímpico Español, del Comité Paralímpico Español, de las ligas profesionales y de las asociaciones de deportistas profesionales.

Lista de directores generales

 Director general de Infraestructuras Deportivas
 Alfonso Luengo Álvarez-Santullano (2011-2012)
 Matilde García Duarte (2009-2011)
 María Dolores Molina de Juan (2008-2009)
 Inmaculada Martín-Caro Sánchez (2004-2008)
 Cristina Torre-Marin Comas (2003)
 Carlos Blanco Bravo (2000-2003)
 Miguel Ángel García Recio (1999-2000)
 Ignacio Ayuso Canals (1996-1999)
 Gonzalo Fernández Rodríguez (1995-1996)
 Eduardo Blanco Pereira (1994-1995)
 José Luis Blanco Velasco (1994)
 Benito Ramos Ramos (1991-1994)
 Tomás Alberdi Alonso (1988-1991)

 Director general de Deportes
 Fernando Molinero Revert (2023- )
 Albert Soler Sicilia (2021-2023)
 Joaquín María de Arístegui Laborde (2020-2021)
 Mariano Soriano Lacambra (2018-2020)
 Jaime González Castaño (2016-2018)
 Óscar Graefenhain de Codes (2015-2016)
 Ana Muñoz Merino (2013-2015)
 David Villaverde Page (2012-2013)
 Matilde García Duarte (2011-2012)
 Albert Soler Sicilia (2008-2011)
 Manuel Fonseca de la Llave (2007-2008)
 Rafael Blanco Perea (2004-2007)
 Guillermo Jesús Jiménez Ramos (2000-2004)
 Eugenio López Álvarez (1999-2000)
 Eduardo Ayuso Canals (1998-1999)
 Santiago Fisas Ayxelá (1996-1998)
 Alfonso Arroyo Lorenzo (1994-1996)
 Manuel Fonseca de la Llave (1993-1994)
 Rafael Cortés Elvira (1987-1993)

Premios y distinciones que otorga el CSD
 Real Orden del Mérito Deportivo
 Premios Nacionales del Deporte

Véase también
 Consejo Superior del Deporte Militar
 Premios Nacionales del Deporte
 Real Orden del Mérito Deportivo

Referencias

Enlaces externos

 Web oficial

Organismos autónomos de España
Entes rectores del deporte en España
Ministerio de Educación de España
Instituciones y organismos con sede en Madrid
Ciudad Universitaria de Madrid
Medallas de Oro al Mérito Social Penitenciario

redpajama_set_name: RedPajamaWikipedia


There have been many strange attempts to avoid a DUI charge including the myth that putting pennies in your mouth tricks a breathalyser and one man who ate his own underwear to soak up the alcohol in his stomach. In that long tradition of half-baked plans, Daniel Vagnini attempted to convince police that he had been assaulted as an explanation for his poor driving.
Police spotted the 22 year old Conneticut native running a stop sign and nearly crashing into another vehicle. The officers tried to follow Vagnini and eventually found him lying next to his car in some nearby woods with tattered clothes, missing his wallet and keys, and showing other signs of an altercation.
Vagnini had struck himself in the face repeatedly, torn his shirt, and thrown his wallet and keys in a nearby river. He tried to explain to officers that while drinking in nearby Hartford he had been assaulted and blacked out. The officers did not take long to see the many holes in the story, including how that would explain the absence of his keys, and arrested Vagnini for reckless driving, driving under the influence, and evading responsibility.
The best way to avoid a DUI charge is by securing a sober driver or taxi after a night of drinking, but if you or anyone you know has been arrested for DUI, contact Bloom Legal at 504-599-9997. We specialize in DUI cases and offer free, no obligation case evaluations.


Though it is simple to order online nowadays you can not beat seeing your kids trying out the bedrooms any kind of. Unfortunately there are not many places where you can see a full array of children's mattresses on display but I actually would always recommend that you take time to find somewhere to find the mattress you are considering before you buy it. This particular way you can examine whether you are happy that your children will be comfortable using ladders or getting in and from the bed.
One 3 rd of your life is spent sleeping; so if a person lives 80 years, 10 years of it was put in sleeping. Now with such duration it is a must to buy a durable and comfortable bed. A person don't want to suffer a backache due to uncomfortable bed for a decade right? Choose a bed and mattress that suits your preferences is an investment in your entire health and well being. Each and every one of us deserves a sound sleeping after a hard days and nights or work.
As far as the utilization of black color is concerned, a lot of designers who are into the modern styles will most definitely choose this among all the other colors available. Even though the traditional design never fades of style; you might want to try the contemporary designs, too. They're sleeker and are more chic looking as well. Indeed, dark bedroom furniture set isn't that hard to find.
Teens love to hide away in their bedrooms. After all, their bedrooms are quite usually the only place they need to call their own. Personalizing a teen's bedroom with teens bedroom furniture is one method to support them assembled a room that suits their individual taste and style.
This is one of the most commonly used bedroom furniture dressers today. Double dressers are short and wide with many drawers – the number of drawers will usually range from six to nine, arranged in rows of twos or threes. A few double dressers come with a dressing mirror attached on top or suspending right above. Should you be looking for high capacity storage, then a double dresser will more than likely do the trick.


text:
Enjoy unforgettable October in Maldives with Soneva
m@ldives 101 views September 16, 2022
Pioneering hospitality group Soneva has announced its spectacular October programme across its award-winning resorts.
At Soneva Fushi and Soneva Jani in the Maldives, the exceptional holiday line-up is designed to delight guests of all ages – with one-of-a-kind family experiences, Michelin-starred guest chefs, rejuvenating wellness treatments, and a star-studded line-up of visiting experts through the ever-evolving Soneva Stars calendar.
As part of their dazzling October programmes, Soneva Fushi and Soneva Jani will also be celebrating 2022's Diwali with unique festival of light-themed experiences for the whole family and special guest appearances – including renowned 'curry queen' Dipna Anand.
Guests at Soneva Fushi this summer can dine with six exceptional visiting chefs from around the world as part of the Soneva Stars roster.
German chef Sascha Stemberg has been at the helm of his family's Michelin-starred Haus Stemberg in Velbert, Germany, since 2015. Winner of the 2013 World Sushi Cup and 'World's Most Creative Sushi Master', Pepi Aneveski, will be showcasing his refined sushi mastery.
Curry queen of the Indian cookery world and co-owner of the famous Brilliant Restaurant in West London's Southall, Dipna Anand will be showcasing some of her grandfather's Punjabi recipes during unforgettable Diwali dinners.
Lisa Goodwin-Allen is the executive chef of the Michelin-starred Northcote restaurant in Lancashire, England, while Chantelle Nicholson is one of the leading female voices in the UK's hospitality industry.
Viktor Beley, brand chef of Moscow's Uhvat restaurant, will be showcasing his original interpretation of traditional Russian cuisine.
In the realm of sports and fitness, six-time NFL Pro-Bowl player Terrel Owens will be hosting a series of workshops for Young Sonevians.
Former English professional rugby player Ben Cohen and his wife Kristina Rihanoff, a world finalist professional ballroom dancer, instructor, choreographer and author, will be helping guests stay in shape with their renowned fitness programmes.
The man behind the global fitness brand The Body Coach and 'The Body Coach TV' YouTube channel, Joe Wicks will be conducting workshops and training sessions for guests.
Todd Woodbridge is best known as one of the world's greatest doubles players in history, while Brazilian former professional footballer 'Naldo' was named the best Bundesliga player in 2017 and was the first defender in Bundesliga history to top this category.
Olympic Swedish swimmer Therese Alshammar will be helping Young Sonevians perfect their stroke while having fun in the water.
With more than two decades' experience as a fitness coach and therapeutic masseur, life-long ultra-endurance athlete Eduardo Sierra will be approaching acute and chronic pain conditions from a multifaceted toolbox combining acupuncture, soft tissue work, osteopathy and the latest functional neurology research at Soneva Soul.
At Soneva Fushi's island observatory, guests can take a fascinating voyage across the heavens with visiting astronomer Dr James O'Donoghue. A planetary astronomer at the Japanese Aerospace Exploration Agency (JAXA), he'll be hosting after-dinner stargazing sessions along with fascinating talks about astronomy throughout the day.
The Den at Soneva Fushi also has a fun-packed programme daily of experiences and activities for younger guests, including eco-friendly arts and crafts workshops, a range of water sports, DJing and mocktail-making classes for teens, movie nights and stargazing workshops, as well as special cooking classes for Diwali celebrations. Older children are also invited to connect with the world around them with the Soneva Academy's immersive modular courses, from exploring the stars to learning about sustainability and the Maldives' marine realm.
View the Soneva Fushi October Programme
Soneva Jani
Soneva's exceptional line-up of visiting chefs will also be going to Soneva Jani in the Maldives' Noonu Atoll.
Sascha Stemberg, Pepi Aneveski, Dipna Anand, Lisa Goodwin-Allen, Chantelle Nicholson and Viktor Beley will all be hosting exclusive dinners at the resort, including at So Engaging, the intimate chef's table that seats just five guests.
Other unforgettable dining experiences during the October calendar include a Champagne sunrise breakfast on the sands of North Beach and a selection of succulent crabs and seafood at our award-winning, open-aired Crab Shack.
A mouth-watering selection of themed dinners showcasing the continent's eclectic cuisine can be found at Asian Kitchen, while diners can explore the heavens with astronomer Dr James O'Donoghue at So Starstruck, the Maldives' only overwater telescope.
A special Diwali celebration dinner with live entertainment will also be taking place on Soneva Jani's South Beach during the festival of lights.
Helping guests stay active this October, fitness experts Joe Wicks, Ben Cohen and Kristina Rihanoff will be conducting their renowned workshops and training sessions.
Former professional football player Ibrahim 'Ibou' Ba will be hosting a Young Sonevians' Football Club, while former professional tennis player Agnieszka Radwańska, known for making intelligent use of the court, will be teaching a series of tennis camps.
Younger guests can also make a splash with Olympic Swedish swimmer Therese Alshammar during a series of swimming sessions.
At Soneva Soul, visiting therapist and clinical chiro practitioner Dr Chet Collins, whose record for relieving and correcting persistent physical pain has resulted in an international reputation as 'Dr Magic Hands', will be helping guests find immediate relief and long-term solutions for their physical needs through a range of treatments.
Guests can also join Soneva Jani's own yoga and meditation masters, Justin and Kamal, for yoga and meditation sessions and learn how to enhance overall wellbeing, stretch and strengthen the body, focus the mind and balance emotions.
The playful escapism of Soneva Jani's luxurious 5.6-kilometre lagoon makes the resort irresistible to experience seekers. A range of complimentary non-motorised water sports, including kayaking, stand-up paddleboarding and snorkelling at the house reef are on offer. Or experience the ocean with complete freedom with some of the latest motorised tech, including eSurfboards and Flyboards, and learn to move through the water like a fish, on the surface and beneath the waves.
A haven for families, The Den at Soneva Jani features a daily roster of exciting, educational and engaging opportunities for both young children and teenagers to create meaningful moments at the barefoot hideaway. Inspired by childhood wonder, The Den is a two-storey awe-inspiring playground for the imagination – and now one of the largest children's clubs in Asia. Featuring a zipline through a cascading waterfall; areas for music, Lego, dressing up and creativity; a soothing sensory room for toddlers and a hidden hangout for teens where mocktails and music set the scene; plus so much more – there's something for all Young Sonevians, both large and small.
Older children and teens can also join the Soneva Academy, which takes learning outside of the classroom with a selection of fascinating immersive courses, including the Marine Life of the Maldives, Mosquitoes and the Adventure of the Corbin, which follows the voyage of an ancient treasure galleon.
View the Soneva Jani Summer Programme
TagsSonevaSoneva FushiSoneva Fushi MaldivesSoneva GroupSoneva JaniSoneva ResortsThe Soneva Group


text:
\section{Appendix}
\input{proofs}

\subsubsection*{Acknowledgments}
\vspace{-.5em}
Lizhen Lin would like to thank Dong Quan Nguyen for very helpful discussions.  Lizhen Lin acknowledges the support from NSF grants IIS  1663870, DMS Career 1654579 and a DARPA grant N66001-17-1-4041.   Bayan Saparbayeva was partially supported by DARPA N66001-17-1-4041.

\clearpage
\bibliographystyle{apalike}

\section{Conclusion and Discussion }\label{sec:conclusion}

We propose a general scheme for solving non-convex optimization on manifolds which yields theoretical guarantees of convergence to a stationary point when the objective function is non-convex. When the objective function is convex, it leads to accelerated  convergence rates for a large class of first order methods, which we show in our numerical examples. One of the interesting future directions  we want to pursue is  proposing accelerated algorithms on statistical manifolds (manifolds of densities or distributions) by employing information-geometric techniques, and applying the algorithms to accelerate convergence and mixing MCMC algorithms.

\section{Introduction}
\label{sec-intro}
Optimization is a near ubiquitous tool used in a wide-range of disciplines including the physical sciences, applied mathematics, engineering and the social sciences. Formally, it aims to maximize or minimize some quantitative criteria, namely, the objective function with respect to some parameters of interest.
In many broad, complex learning
in modern data science, the parameters are naturally defined over
to be on a \emph{manifold}. The emerging field of statistics on manifolds based on Fr\'echet means \citep{rabibook, linclt} can be viewed as one of the notable examples of optimization on general manifolds.

Another example can be found in building scalable recommender systems where extracting a low-rank matrix involves an optimization problem over a Grassmann manifold \citep{boumal}. Recent development in geometric deep learning, where the input or output layer constrained to be on a Riemannian manifold \citep{Lohit2017LearningIR, Huang2017ARN, Huang2017DeepLO}, constitutes another important class of applications. Other applications arise in diverse areas ranging from medical imaging analysis, Procrustes shape matching, dimension reduction, dynamic subspace tracking, and cases involving ranking and orthogonality constraints--among many others. This proliferation of manifold-valued applications demands fundamental development of models, algorithms and theory for solving optimization problems over non-Euclidean spaces.

The current literature on optimization over manifolds mainly focuses on extending existing Euclidean space algorithms, such as  Newton's method \citep{2014arXiv1407.5965S, ring12}, conjugate gradient descent \citep{Edelman1998, Nishimori2008}, steepest descent \citep{steepest}, trust-region methods \citep{Absil2007, NIPS2011_4402} and others.  Many of the objective functions in manifold optimization problems are very complex. One of key challenges for solving such problems lies in the difficulty in verifying the convexity and the degree of convexity of the objective function.
Current approaches cannot adapt to the complexity of the problem at hand in manifold spaces.

We take a major step to address these issues by proposing a general scheme to solve convex and non-convex optimization problems on manifolds using gradient-based algorithms originally designed for convex functions.
 The key idea is to ``convexify'' the objective function by adding a multiple of the squared retraction distance.  The proposed algorithm does not require knowledge of whether the objective function is convex but will automatically converges to an optima if the function is strongly convex. When the objective is non-convex, it achieves rapid convergence to a stationary point. The proposed algorithm is a generalization of Nesterov acceleration \citep{Nesterov2004}, which improves the convergence rate of gradient descent algorithms. Our algorithm (which we call $\mathcal{A}_2$) takes any general existing optimization method (which we call $\mathcal A$), originally designed for convex functions, and converts it into a method applicable for non-convex functions.

Similar schemes have been explored for optimization problems in Euclidean space \citep{pmlr-v84-paquette18a}. Generalizations to arbitrary manifolds, however, require fundamentally novel theoretical development.  In the Euclidean case, the gradient steps are taken towards lines, whereas for the manifold we use the retraction curves which crucially
 affects the result and raises the difficulty in proving convergence. Also for manifolds, it is not trivial to correctly  convexify the 'weakly-convex function', a broad class of non-convex functions on manifolds  we consider which  account for most of the interesting examples of non-convex functions in machine learning. We propose a novel idea to convexify the objective locally with the help of the retraction. Key features of our algorithm include adaptation to the unknown weak convexity of the objective function  and automatic Nesterov acceleration. The proposed algorithm can be used to accelerate a broad class of $\mathcal A$ algorithms including gradient descent as well as parallel optimization approaches \citep[see][]{lizhennips2018}.

Our paper is organized as follows: In Section~\ref{sec:related}, we introduce related work on accelerated optimization algorithms. Next, we present our proposed acceleration algorithm on manifolds in Section~\ref{sec: proposed} and present theoretical convergence results. In Section~\ref{sec-simu}, we consider a simulation study of estimating Fr\'echet means and a real data example using the Netflix prize data set in a matrix completion problem.

\subsection{Proof to Theorem 2}
\label{suppl-proofs}

We first introduce a simple lemma.

\begin{lemma}
Suppose the sequence $\{\alpha_k\}_{k\geq1}$ is produced by $\mathcal{A}_2.$ Then, the following bounds hold for all $k\geq1$
\begin{equation*}
    \frac{\sqrt{2}}{k+2}\leq\alpha_k\leq\frac{2}{k+1}.
\end{equation*}
\end{lemma}

\begin{proof}[Proof of Theorem 2]
The descent condition in
\begin{align}\label{C1}
\begin{split}
    {\rm dist}\big(0_{\bar{\theta}_k}, \partial h(\bar{\theta}_k, \theta_{k-1})\big)&<\kappa_k d_{\mathcal{R}}(\bar{\theta}_k, \theta_{k-1}) \quad \text{and} \\
    \quad h_{\kappa_k}(\bar{\theta}_k, \theta_{k-1})&\leq h_{\kappa_k}(\theta_{k-1}, \theta_{k-1}),
    \end{split}
\end{align}
implies $\{f(\theta_k)\}_{k\geq0}$ are monotonically decreasing.
From this
\begin{equation}\label{T-1}
\begin{split}
    f(\theta_{k-1})&=h_{\kappa}(\theta_{k-1}, \theta_{k-1})\\
                          &\geq h_{\kappa}(\bar{\theta}_k, \theta_{k-1})\\
                          &\geq f(\theta_k)+\frac{\kappa}{2}d_{\mathcal{R}}^2(\bar{\theta}_k, \theta_{k-1}).
\end{split}
\end{equation}
Using condition \eqref{C1}, we apply Lemma 2 with $\vartheta=\theta_{k-1}, \theta=\bar{\theta}_k$ and $\varepsilon=\kappa Kd_{\mathcal{R}}(\bar{\theta}_k, \theta_{k-1});$ hence
\begin{equation*}
    {\rm dist}(0_{\bar{\theta}_k}, \partial f(\bar{\theta}_k))\leq 2\kappa K d_{\mathcal{R}}(\bar{\theta}_k, \theta_{k-1}).
\end{equation*}
Combining the above inequality with \eqref{T-1}, one has
\begin{align}\label{T-2}
\begin{split}
    {\rm dist}^2(0_{\bar{\theta}_k}, \partial f(\bar{\theta}_k))&\leq4\kappa^2K^2d_{\mathcal{R}}^2(\bar{\theta}_k, \theta_{k-1})\\
    &\leq8\kappa_{\max}K^2\big(f(\theta_{k-1}-f(\theta_k)\big).
\end{split}
\end{align}
Summing $j=1$ to $N,$ we can conclude
\begin{align*}
        \min_{j=1, ..., N}\Big\{{\rm dist}^2\big(0_{\bar{\theta}_j}, \partial f(\bar{\theta}_j)\big)\Big\}&\leq\frac{8\kappa_{\max}K^2}{N}\sum^N_{j=1}\big(f(\theta_{j-1})-f(\theta_j)\big)\\
        &\leq\frac{8\kappa_{\max}K^2}{N}\big(f(\theta_0)-f^{*}\big).
\end{align*}

Fix an $v_k\in\partial h_{\kappa}(\Tilde{\theta}_k, \vartheta_k).$ Since the function $f$ is $\kappa_{cvx}(K_1^4K_2^4-R_1)$-strongly convex, the function $h_{\kappa_{cvx}}$ is $\kappa_{cvx} K_1^4K_2^4$-strongly convex.
\begin{align*}
    f(\theta)&+\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\theta, \vartheta_k)\\
   & \geq f(\Tilde{\theta}_k)+\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)\\
  & \quad\quad+\frac{\kappa_{cvx} K_1^4K_2^4}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \theta)+\langle v_k, \mathcal{R}_{\Tilde{\theta}_k}^{-1}\theta\rangle.
\end{align*}
Then
\begin{align*}
    f(\Tilde{\theta}_k) &\leq
    f(\theta)+\frac{\kappa_{cvx}}{2}\big(d_{\mathcal{R}}^2(\theta, \vartheta_k)-K_1^4K_2^4 d_{\mathcal{R}}^2(\Tilde{\theta}_k, \theta)\\
   &\quad\quad -d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)\big)-\langle v_k, \mathcal{R}_{\Tilde{\theta}_k}^{-1}\theta\rangle.
\end{align*}
So for any $\theta\in\mathcal{M}$
\begin{equation*}
\begin{split}
    f(\theta_k)\leq& f(\Tilde{\theta}_k)\\
    \leq&  f(\theta)+\frac{\kappa_{cvx}}{2}\Big(K_1^2\|\mathcal{R}^{-1}_{\theta_{k-1}}\theta-\mathcal{R}^{-1}_{\theta_{k-1}}\vartheta_{k}\|^2- K_1^4K_2^2\|\mathcal{R}^{-1}_{\theta_{k-1}}\Tilde{\theta}_k\\
    &-\mathcal{R}^{-1}_{\theta_{k-1}}\theta\|^2
    \Big)-\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)-\langle v_k, \mathcal{R}_{\Tilde{\theta}_k}^{-1}\theta\rangle.
   \end{split}
\end{equation*}
We substitute $\theta=\mathcal{R}_{\theta_{k-1}}\alpha_k\mathcal{R}^{-1}_{\theta_{k-1}}\theta^{*},$ where $\theta^{*}$ is any minimizer of $f.$ Using convexity of $f$
\begin{equation*}
    f(x)\leq\alpha_k f(\theta^{*})+(1-\alpha_k)f(\theta_k),
\end{equation*}
the stopping criteria ,
\begin{equation}
    \label{C2}
    {\rm dist}\big(0_{\Tilde{\theta}_k}, \partial h_{\kappa_{cvx}}(\Tilde{\theta}_k, \vartheta_k)\big)<\frac{\kappa_{cvx}}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k),
\end{equation}
  i.e. $\|v_k\|<\frac{\kappa_{cvx}}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k),$ and $\vartheta_k=\mathcal{R}_{\theta_{k-1}}\alpha_{k}\mathcal{R}^{-1}_{\theta_{k-1}}\Tilde{\vartheta}_{k-1},$ and $\Tilde{\vartheta}_k=\mathcal{R}_{\theta_{k-1}}\frac{1}{\alpha_k}\mathcal{R}^{-1}_{\theta_{k-1}}\Tilde{\theta}_k,$ one has
  \begin{equation*}
    \begin{split}
        f(\theta_k)\leq&\alpha_k f(\theta^{*})+(1-\alpha_k)f(\theta_k)\\
        &+\frac{\kappa_{cvx}\alpha_k^2}{2}\Big(K_1^2\|\mathcal{R}^{-1}_{\theta_{k-1}}\theta^*-\mathcal{R}^{-1}_{\theta_{k-1}}\Tilde{\vartheta}_{k-1}\|^2\\
        &-K_1^4K_2^2\|\mathcal{R}^{-1}_{\theta_{k-1}}\Tilde{\vartheta}_k-\mathcal{R}^{-1}_{\theta_{k-1}}\theta^*\|^2\Big)\\
        &-\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)+\frac{\kappa_{cvx}}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k)  \|\mathcal{R}_{\tilde{\theta}_k}^{-1}\theta\|\\
  \leq&\alpha_k f(\theta^{*})+(1-\alpha_k)f(\theta_k)\\
        &+\frac{\kappa_{cvx}\alpha_k^2}{2}\Big(K_1^2K_2^2d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})-K_1^2K_2^2d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\Big)\\
        &-\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)+\frac{\kappa_{cvx}}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k)d_{\mathcal{R}}(\tilde{\theta}_k, \theta)\\
  \leq&\alpha_k f(\theta^{*})+(1-\alpha_k)f(\theta_k)\\
        &+\frac{\kappa_{cvx}\alpha_k^2}{2}\Big(K_1^2K_2^2d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})-K_1^2K_2^2d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\Big)\\
        &-\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)\\
        &+\frac{\kappa_{cvx}K_1}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k)\|\mathcal{R}_{\theta_{k-1}}^{-1}\tilde{\theta}_k-\mathcal{R}_{\theta_{k-1}}^{-1}\theta\|\\
     =&\alpha_k f(\theta^{*})+(1-\alpha_k)f(\theta_k)\\
       &+\frac{\kappa_{cvx}\alpha_k^2}{2}\Big(K_1^2K_2^2d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})-K_1^2K_2^2d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\Big)\\
       &-\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)\\
       &+\frac{\kappa_{cvx}\alpha_kK_1}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k)\|\mathcal{R}_{\theta_{k-1}}^{-1}\tilde{\vartheta}_k-\mathcal{R}_{\theta_{k-1}}^{-1}\theta^{*}\|\\
 \leq&\alpha_k f(\theta^{*})+(1-\alpha_k)f(\theta_k)\\
       &+\frac{\kappa_{cvx}\alpha_k^2}{2}\Big(K_1^2K_2^2d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})-K_1^2K_2^2d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\Big)\\
       &-\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)\\
       &+\frac{\kappa_{cvx}\alpha_kK_1K_2}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k)d_{\mathcal{R}}(\tilde{\vartheta}_k, \theta^{*}).
    \end{split}
\end{equation*}
So
\begin{equation}\label{T-3}
\begin{split}
    f(\theta_k)\leq&\alpha_k f(\theta^{*})+(1-\alpha_k)f(\theta_k)\\
                          &+\frac{\kappa_{cvx}\alpha_k^2}{2}\Big(K_1^2K_2^2d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})-K_1^2K_2^2d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\Big)\\
                          &-\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)\\
                          &+\frac{\kappa_{cvx}\alpha_kK_1K_2}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k)d_{\mathcal{R}}(\theta^{*}, \Tilde{\vartheta}_k).
\end{split}
\end{equation}
Set $\mu_k=\frac{1}{k+1}.$ Completing the square yields
\begin{align*}
    -\frac{\kappa_{cvx}}{2}d_{\mathcal{R}}^2(\Tilde{\theta}_k, \vartheta_k)+\kappa_{cvx}\alpha_k\mu_kK_1K_2 d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k)d_{\mathcal{R}}(\theta^{*}, \Tilde{\vartheta}_k)\\
    \leq
    \frac{K_1^2K_2^2\kappa_{cvx}\alpha_k^2\mu_k^2}{2}d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k),
\end{align*}
and subtracting $f^{*}=f(\theta^{*})$ from both sides, we obtain
\begin{equation*}
\begin{split}
    f(\theta_k)-f^{*}&\leq(1-\alpha_k)(f(\theta_{k-1})-f^{*})+\frac{\kappa_{cvx}\alpha_k^2}{2}\Big(K_1^2K_2^2d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})\\
    &\quad-K_1^2K_2^2d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\Big)+\frac{K_1^2K_2^2\kappa_{cvx}\alpha_k^2\mu_k^2}{2}d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\\
    &=(1-\alpha)(f(\theta_{k-1})-f^{*})+\frac{\kappa_{cvx}\alpha_k^2K_1^2K_2^2}{2}d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})\\
    &\quad-\frac{\kappa_{cvx}\alpha_k^2K_1^2K_2^2}{2}(1-\mu_k^2)d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k).
\end{split}
\end{equation*}
So one can obtain
\begin{align*}
    \frac{f(\theta_k)-f^{*}}{\alpha_k^2}+\frac{\kappa_{cvx} K_1^2K_2^2}{2}(1-\mu_k^2)d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\\
    \leq\frac{1-\alpha_k}{\alpha_k^2}(f(\theta_{k-1})-f^{*})+\frac{\kappa_{cvx} K_1^2K_2^2}{2}d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1}).
\end{align*}
Denote $A_k=(1-\mu_k^2).$ Using the equality $\frac{1-\alpha_k}{\alpha_k^2}=\frac{1}{\alpha_{k-1}^2}$ we derive the following recursion
\begin{equation*}
    \begin{split}
         &\frac{f(\theta_k)-f^{*}}{\alpha_k^2}+\frac{\kappa_{cvx} K_1^2K_2^2 A_k}{2}d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k) \\
         &\leq\frac{1-\alpha_k}{\alpha_k^2}(f(\theta_{k-1})-f^{*})+\frac{\kappa_{cvx} K_1^2K_2^2}{2}d_{\mathcal{R}}^2(\theta^*, \Tilde{\vartheta}_{k-1})\\
         &=\frac{f(\theta_{k-1})-f^{*}}{\alpha_{k-1}^2}+\frac{\kappa_{cvx} K_1^2K_2^2}{2}d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_{k-1})\\
         &\leq\frac{f(\theta_{k-1})-f^{*}}{A_{k-1}\alpha_{k-1}^2}+\frac{\kappa_{cvx} K_1^2K_2^2}{2}d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_{k-1})\\
         &=\frac{1}{A_{k-1}}\Bigg(\frac{f(\theta_{k-1})-f^{*}}{\alpha_{k-1}^2}+\frac{\kappa_{cvx} K_1^2K_2^2A_{k-1}}{2}d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_{k-1})\Bigg).
    \end{split}
\end{equation*}
The last inequality holds because $0<A_k\leq1.$
 Iterating $N$  times, we deduce
\begin{align*}
    \frac{f(\theta_N)-f^{*}}{\alpha_N^2}&\leq\frac{f(\theta_N)-f^{*}}{\alpha_N^2}+\frac{\kappa_{cvx} K_1^2K_2^2 A_k}{2}d_{\mathcal{R}}^2(\theta^{*}, \Tilde{\vartheta}_k)\\
    &\leq\frac{\kappa_{cvx} K_1^2K_2^2}{2}d_{\mathcal{R}}^2(\theta^{*}, \theta_0)\prod^N_{k=2}\frac{1}{A_{k-1}}.
\end{align*}
Note that
\begin{equation*}
    \prod^{N}_{k=2}\frac{1}{A_{k-1}}\leq2;
\end{equation*}
thereby with inequality from Lemma 1 we conclude
\begin{align*}
       f(\theta_N)-f^{*}&\leq\frac{\alpha_N^2\kappa_{cvx} K_1^2K_2^{2}}{2}d_{\mathcal{R}}^2(\theta^{*}, \theta_0)\prod^N_{k=2}\frac{1}{A_{k-1}}\\
       &\leq\alpha_N^2\kappa_{cvx} K_1^2K_2^2 d_{\mathcal{R}}^2(\theta^{*}, \theta_0)\\
       &\leq\frac{4\kappa_{cvx} K_1^2K_2^2}{(N+1)^2}d_{\mathcal{R}}^2(\theta^{*}, \theta_0).
\end{align*}
Hence
\begin{equation*}
   f(\theta_N)-f^{*}\leq\frac{4\kappa_{cvx} K_1^2K_2^2}{(N+1)^2}d_{\mathcal{R}}^2(\theta^{*}, \theta_0).
\end{equation*}

\end{proof}

\subsection{Strong convexity of the objective function in estimating the intrinsic Fr\'echet means on the sphere}

We provide a proof that the objective functions in estimating both the intrinsic and extrinsic Fr\'echet means on the sphere in Section 4 is strongly convex.

\begin{proof}
In order to prove the strong-convexity of the intrinsic mean on the sphere $S^n$, we will prove the strong-convexity of the square intrinsic distance function from the point $x_0\in S^n$
\begin{equation*}
    d_g^2(x_0, x)=\arccos^2(x_0^Tx).
\end{equation*}
So for the geodesic from the point $x_1\in S^n$ to the point $x_2\in S^2$
\begin{equation*}
\begin{split}
    \gamma(\lambda)&=\exp_{x_1}\lambda\log_{x_1}x_2\\
    &=\cos\big(\lambda\arccos(x_1^Tx_2)\big)x_1\\
    &\;\;+\sin\big(\lambda\arccos(x_1^Tx_2)\big)\frac{x_2-(x_1^Tx_2)x_1}{\sqrt{1-(x_1^Tx_2)^2}},
\end{split}
\end{equation*}
we need to show following inequality
\begin{align*}
    d_g^2(x_0, \gamma(\lambda))&\leq(1-\lambda)d_g^2(x_0, x_1)+\lambda d_g^2(x_0, x_2)\\
    &\;\;-\frac{\lambda(1-\lambda)\mu}{2}d_g^2(x_1, x_2).
\end{align*}
For the sake of briefness let's use the following notations
\begin{gather*}
d_1=\arccos(x_0^Tx_1), \qquad d_2=\arccos(x_0^Tx_2), \\
d_2=\arccos(x_1^Tx_2).
\end{gather*}

Therefore we have to prove the following inequality
\begin{multline}
\label{strong-con}
    \arccos^2\bigg(\cos(\lambda d_3)\cos(d_1)\\
    +\sin(\lambda d_3)\frac{\cos(d_2)-\cos(d_3)\cos(d_1)}{\sin(d_3)}\bigg)\\
    \leq\quad(1-\lambda)d_1^2+\lambda d_2^2-\frac{\lambda(1-\lambda)\mu}{2}d_3^2.
\end{multline}

Or we should prove the inequality
\begin{align*}
    \arccos^2(x_0^Tx_2)&>\arccos^2(x_0^Tx_1)-2(\log_{x_1}x_0)^T\log_{x_1}x_2\\
    &\;\;+\frac{\mu}{2}\arccos^2(x_1^Tx_2)
\end{align*}
    \begin{align*}
    d_2^2&>d_1^2-2\bigg(d_1\frac{x_0-\cos(d_1)x_1}{\sin(d_1)}\bigg)^T\bigg(d_3\frac{x_2-\cos(d_3)x_1}{\sin(d_3)}\bigg)+\frac{\mu}{2}d_3^2\\
    &=d_1^2-2d_1d_3\frac{\cos(d_2)-\cos(d_3)\cos(d_1)}{\sin(d_1)\sin(d_3)}+\frac{\mu}{2}d_3^2
\end{align*}
The last inequality was checked to hold in Wolfram Mathematica for $d_1,d_2\in[0, \pi/4]$ and $d_3\in\Big[|d_1-d_2|, d_1+d_2\Big],$ where $\mu=1$.

In order to proof the strong-convexity of Fr\'echet function in estimating extrinsic mean on the sphere $S^n$, we will prove the strong-convexity of the square extrinsic distance function from the point $x_0\in S^n$
\begin{equation*}
    d_e^2(x_0, x)=2(1-x_0^Tx).
\end{equation*}
So for the geodesic from the point $x_1\in S^n$ to the point $x_2\in S^2$
\begin{equation*}
\begin{split}
    \gamma(\lambda)&=\exp_{x_1}\lambda\log_{x_1}x_2\\
    &=\cos\big(\lambda\arccos(x_1^Tx_2)\big)x_1+\sin\big(\lambda\arccos(x_1^Tx_2)\big)\frac{x_2-(x_1^Tx_2)x_1}{\sqrt{1-(x_1^Tx_2)^2}},
\end{split}
\end{equation*}
we need to show that
\begin{align*}
    d_e^2(x_0, \gamma(\lambda))&\leq(1-\lambda)d_e^2(x_0, x_1)+\lambda d_e^2(x_0, x_2)\\
    &\;\;-\frac{\lambda(1-\lambda)\mu}{2}d_g^2(x_1, x_2).
\end{align*}

Therefore we have to prove
\begin{multline}\label{strong-con}
    2\bigg(1-\cos(\lambda d_3)\cos(d_1)-\sin(\lambda d_3)\frac{\cos(d_2)-\cos(d_3)\cos(d_1)}{\sin(d_3)}\bigg)\\
    \leq\quad2-2\big((1-\lambda)\cos(d_1)+\lambda\cos( d_2)\big)-\frac{\lambda(1-\lambda)\mu}{2}d_3^2.
\end{multline}

Or we need to show
\begin{multline*}
    2(1-x_0^Tx_2)>\\
    2(1-x_0^Tx_1)-2\big(x_0-(x_0^Tx_1)x_1\big)^T\log_{x_1}x_2+\frac{\mu}{2}\arccos^2(x_1^Tx_2)
\end{multline*}
Thus
    \begin{align*}
    &2\big(1-\cos(d_2)\big)\\
    &\quad\;>2\big(1-\cos(d_1)\big)- 2\big(x_0-\cos(d_1)x_1\big)^T\bigg(d_3\frac{x_2-\cos(d_3)x_1}{\sin(d_3)}\bigg)+\frac{\mu}{2}d_3^2\\
    &\quad=2\big(1-\cos(d_1)\big)-2d_3\frac{\cos(d_2)-\cos(d_3)\cos(d_1)}{\sin(d_3)}+\frac{\mu}{2}d_3^2
\end{align*}
The last inequality was verified Wolfram Mathematica for $d_1,d_2\in[0, \pi/4]$ and $d_3\in\Big[|d_1-d_2|, d_1+d_2\Big],$ where $\mu=1$

\end{proof}

\section{Accelerated algorithms for  optimization on manifolds}
\label{sec: proposed}
\subsection{Weakly convex functions on  manifolds with respect to retraction mapping}

We first define general retraction-based, weakly convex, convex, and strongly convex functions by generalizing from their geodesic-based counterparts . We then prove an important proposition
that can transform a non-convex function into a convex one simply by adding a multiple of the squared retraction-distance to the objective function.

\begin{definition}
A {\it retraction} on a manifold $\mathcal{M}$ is a smooth mapping from its tangent bundle $\mathcal{R}: T\mathcal{M}\rightarrow\mathcal{M}$
with the following properties:
\begin{enumerate}
    \item $\mathcal{R}_{\theta}(0_{\theta})=\mathcal{R}(\theta, 0_{\theta})=\theta,$ where $0_{\theta}$ denotes the zero vector on the tangent space $T_{\theta}\mathcal{M};$
    \item For any point $\theta\in\mathcal R$ the differential $d(\mathcal{R}_{\theta})$ of the retraction mapping at the zero vector $0_{\theta}\in T_{\theta}\mathcal M$ has to be equal to  the identity mapping on $T_{\theta}\mathcal{M},$ that is $d(\mathcal{R}_{\theta}(0_{\theta}))=d\big(\mathcal{R}(\theta, 0_{\theta})\big)={\rm id}_{T_{\theta}\mathcal{M}},$ where ${\rm id}_{T_{\theta}\mathcal{M}}$ denotes the identity mapping on $T_{\theta}\mathcal{M}.$
\end{enumerate}
The exponential map on a Riemannian manifold can be viewed as a special case of the retraction map, and the inverse-exponential map is a special case of the inverse-retraction map. A good choice of retraction map can lead to substantial reduction in computation burden compared to the exponential map. We see an example in Section~\ref{subsec:netflix} on the choice of a retraction map for Grassmannian; Figure~\ref{fig-ret} provides a visualization of a retraction map.
\begin{figure}
\center
\includegraphics[width=.5\linewidth]{test1_gray.png}
\caption{Illustration of a retraction map on a manifold}
\label{fig-ret}
\end{figure}
\end{definition}

We first define the retraction distance function on $\mathcal M$
\begin{equation*}
 d_{\mathcal{R}}(\theta_0, \theta)=\|\mathcal{R}_{\theta_0}^{-1}\theta\|.
 \end{equation*}

Since at  zero the differential of the retraction map is the identity, there is a small enough neighborhood  $D$ of the point $\theta$ where the inverse retraction map $\mathcal{R}_{\theta}^{-1}$ is bi-Lipschitz continuous in $D,$ i.e. $d_{\mathcal{R}}$  satisfies inequalities
\begin{equation*}
    \frac{1}{K_1}d_{\mathcal{R}}(\vartheta_1, \vartheta_2)\leq\|\mathcal{R}_{\theta}^{-1}\vartheta_1-\mathcal{R}_{\theta}^{-1}\vartheta_2\|\leq K_2d_{\mathcal{R}}(\vartheta_1, \vartheta_2),
\end{equation*}
where $\vartheta_1, \vartheta_2\in D,$ and $K_1\geq1,$ and $K_2\geq1.$

In addition, we also require the squared retraction distance function to be {\it $2R_1$-strongly retraction convex} around $\vartheta$--that is, for some $\delta>0$ and constant $0\leq R_1\leq1$ the following inequality holds:
\begin{equation}\label{dR-strongly-convex-2}
    d_{\mathcal{R}}^2(\theta_2, \vartheta)\geq d_{\mathcal{R}}^{2}(\theta_1, \vartheta)+\langle\nabla d_{\mathcal{R}}^2(\theta_1, \vartheta), \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle+R_1d_{\mathcal{R}}^2(\theta_1, \theta_2),
\end{equation}
where $d_{\mathcal{R}}(\theta_i, \vartheta)<\delta,$ $i=1,2.$  Due to the fact that at the zero vector $0_{\vartheta}\in T_{\vartheta}\mathcal{M}$ the differential of $\mathcal{R}_{\vartheta}$  is equal to identity mapping, we can see that in a small neighborhood of $\vartheta$,  the square retraction distance function behaves like the square normal function which is strongly convex.

\begin{definition}
Consider a function $f:\mathcal{M}\rightarrow\bar{\mathbb{R}}$ and a point $\theta$ with $f(\theta)$ finite. The {\it $\mathcal{R}-$subdifferential} of $f$ at $\theta$ is the set
\begin{equation*}
\begin{split}
    \partial f(\theta)=\Big\{v\in T_{\theta}\mathcal{M}: f(\vartheta)\geq f(\theta)+\langle v, \mathcal{R}^{-1}_{\theta}\vartheta\rangle+o\big(d_{\mathcal{R}}(\theta, \vartheta)\big)\\
     \forall \vartheta\in \mathcal{M}\Big\}.
    \end{split}
\end{equation*}
\end{definition}

We now define the notion of convex functions on manifolds with respect to the retraction map.
\begin{definition}
A function $f$ is {\it convex with respect to the retraction} $\mathcal{R}$ if for any points $\theta_1, \theta_2\in\mathcal{M}$ the inequality holds
\begin{equation}
\label{f-convex-2}
    f(\theta_2)\geq f(\theta_1)+\langle v, \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle, \qquad  v\in\partial f(\theta_1).

\end{equation}
\end{definition}

Now we are ready to define one of the most important classes of non-convex functions called  \emph{weakly-convex functions} which constitute many interesting applications of non-convex functions in machine learning.

\begin{definition}
A function $f$ is {\it $\rho$-weakly convex with respect to the retraction} $\mathcal{R}$ if for any points $\theta_1, \theta_2\in\mathcal{M}$ the inequality holds
\begin{equation}
\label{f-weakly-convex-2}
    f(\theta_2)\geq f(\theta_1)+\langle v, \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle-\frac{\rho}{2}d_{\mathcal{R}}^2(\theta_1, \theta_2), \qquad  v\in\partial f(\theta_1).
\end{equation}
\end{definition}

Given the strong retraction convexity of the squared retraction distance (see \eqref{dR-strongly-convex-2}),  we can regularize the weakly convex function $f$ by adding the term $\frac{\kappa}{2}d_{\mathcal{R}}^2(\theta, \vartheta)$ and turn it into a convex function  through the following proposition.
\begin{proposition}
\label{prop1}
Let $d_{\mathcal{R}}$  be a retraction distance that is  strong-retraction convex or  satisfies the inequality \eqref{dR-strongly-convex-2} in the subset $D\subset\mathcal{M}.$
Then the function $f$ is $R_1\kappa$-weakly convex in $D$ if and only if the function
     \begin{equation*}h_{\kappa}(\theta, \vartheta)=f(\theta)+\frac{\kappa}{2} d_{\mathcal{R}}^2(\theta, \vartheta)\end{equation*}
is convex in $D.$
\end{proposition}
\begin{proof}
Let $f$ be $\rho$-weakly convex. Then for any $\theta_1, \theta_2\in D$ and any $\lambda\in[0, 1]$
\begin{align*}
 f(\theta_2)&\geq f(\theta_1)+\langle\partial f(\theta_1), \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle-\frac{R_1\kappa}{2}d_{\mathcal{R}}^2(\theta_1, \theta_2)\\
 &\geq f(\theta_1)+\langle\partial f(\theta_1), \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle+ \frac{\kappa}{2}d_{\mathcal{R}}^{2}(\theta_1, \vartheta)\\
& \quad\quad +\langle\nabla \frac{\kappa}{2}d_{\mathcal{R}}^2(\theta_1, \vartheta), \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle-\frac{\kappa}{2}d_{\mathcal{R}}^2(\theta_2, \vartheta),
\end{align*}
which implies
\begin{equation*}
    h_{\kappa}(\theta_2, \vartheta)\geq h_{\kappa}(\theta_1, \vartheta)+\langle\partial h_{\kappa}(\theta_1, \vartheta), \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle.
\end{equation*}
\end{proof}
For functions defined on an Euclidean space we have a definition of a weakly convex function that is equivalent to \eqref{f-weakly-convex-2}:
\begin{equation}
\label{f-weakly-convex-0}
    f\big(\mathcal{R}_{\vartheta}(\lambda\mathcal{R}^{-1}_{\vartheta}\theta)\big)\leq\lambda f(\theta)+(1-\lambda)f(\vartheta)+\frac{\rho\lambda(1-\lambda)}{2}d_{\mathcal{R}}^2(\theta, \vartheta).
\end{equation}
Over the manifold,  however,  there is no such straightforward equivalence. This is due to the distance function $d_{\mathcal{R}}^2(\vartheta, \theta)$ which does not satisfy the following equality:
\begin{align}
\label{dR-equality}
     d_{\mathcal{R}}^2(\mathcal{R}_{\theta_1}\lambda\mathcal{R}_{\theta_1}^{-1}\theta_2, \vartheta)&\nonumber=\lambda d_{\mathcal{R}}(\theta_2, \vartheta)+(1-\lambda)d_{\mathcal{R}}^2(\theta_1, \vartheta)\\
     &\quad\quad-\lambda(1-\lambda)d_{\mathcal{R}}^2(\theta_1, \theta_2).
\end{align}
Nevertheless in some neighborhood of $\vartheta,$, for some $\delta>0$, the following inequality holds
\begin{equation}\label{dR-strongly-convex-1}
\begin{split}
   d_{\mathcal{R}}^2(\mathcal{R}_{\theta_1}\lambda\mathcal{R}_{\theta_1}^{-1}\theta_2, \vartheta)\leq\lambda d_{\mathcal{R}}^2(\theta_2, \vartheta) +(1-\lambda)d_{\mathcal{R}}^2(\theta_1, \vartheta)\\
   -\lambda(1-\lambda)R_1 d_{\mathcal{R}}^2(\theta_1, \theta_2),
   \end{split}
\end{equation}
where $d_{\mathcal{R}}(\theta_1, \vartheta)<\delta$ and $d_{\mathcal{R}}(\theta_2, \vartheta)<\delta$.

Therefore the function $f$ is {\it $\rho$-weakly convex with respect to the retraction} $\mathcal{R}$ if for any points $\theta, \vartheta\in\mathcal{M}$ such that $\lambda\in[0, 1],$ the approximate secant inequality holds
\begin{equation*}\label{f-weakly-convex-1}
    f\big(\mathcal{R}_{\vartheta}(\lambda\mathcal{R}^{-1}_{\vartheta}\theta)\big)\leq\lambda f(\theta)+(1-\lambda)f(\vartheta)+
    \frac{\rho\lambda(1-\lambda)}{2}d_{\mathcal{R}}^2(\theta, \vartheta),

\end{equation*}
where $d_{\mathcal{R}}(\theta, \vartheta)<\delta.$

\subsection{The acceleration algorithm on manifolds}

In this section, we propose our acceleration algorithms for convex and non-convex functions on manifolds.
We first minimize the convex subproblem of an objective function $f$ for some existing approach $\mathcal{A}$ (such as a gradient descent algorithm) where the objective function is written as
\begin{equation*}
    h_{*}(\vartheta)=\min_{\theta\in \mathcal{M}}\left\{f(\theta)+\frac{\kappa}{2}d_{\mathcal{R}}^2(\theta, \vartheta)\right\},
\end{equation*}
with a positive regularization parameter $\kappa$. Proposition \ref{prop1} ensures the convexity of the subproblem for an appropriate level of regularization.

Therefore, with an existing approach $\mathcal{A}$, we define the {\it proximal operator}
\begin{equation*}
    p(\vartheta)={\rm prox}_{f/\kappa}(\vartheta)=\arg\min_{\theta\in \mathcal{M}}\Big\{f(\theta)+\frac{\kappa}{2}d_{\mathcal{R}}^{2}(\theta, \vartheta)\Big\},
\end{equation*}
where $\vartheta$ is a {\it prox-center.}

To consider optimizing $p(\vartheta)$, we focus on $\mathcal A$ having linear convergence rates. Specifically, a minimization algorithm $\mathcal{A},$ generating the sequence of iterates $(\theta_k)_{k\geq0},$ has a {\it linear convergence rate} if there exists $\tau_{\mathcal{A}, f}\in (0, 1)$ and a constant $C_{\mathcal{A}, f}\in\mathbb{R}$ such that
\begin{equation*}
  f(x_k)-f_{*}\leq C_{\mathcal{A}, f}(1-\tau_{\mathcal{A}, f})^k,
\end{equation*}
where $f_{*}$ is the minimum value of $f.$

There are multiple optimization algorithms on manifolds  with linear convergence rates for strongly-convex functions on manifold. These include gradient descent, conjugate gradient descent, MASAGA \citep{Babanezhad2018MASAGAAL}, RSVRG \citep{Zhang:2016:RSF}, and many others.

For a proximal center $\vartheta$ and a smoothing parameter $\kappa$, we let
\begin{align*}
h_{\kappa}(\theta, \vartheta)=f(\theta)+\frac{\kappa}{2}d_{\mathcal{R}}^2(\theta, \vartheta).
\end{align*}

At the $k$-th iteration, given a previous iterate $\theta_{k-1}$ and the extrapolation term $\tilde{\vartheta}_{k-1},$ we perform the following steps:
\begin{enumerate}
    \item {\bf Proximal point step.}
    \begin{align*}\bar{\theta}_k\approx\arg\min_{\theta\in \mathcal{M}}h_{\kappa}(\theta, \theta_{k-1}).
    \end{align*}
    \item {\bf Accelerated proximal point step.}
    \begin{align*}\vartheta_k=\mathcal{R}_{\theta_{k-1}}\left(\alpha_k\mathcal{R}_{\theta_{k-1}}^{-1}\tilde{\vartheta}_{k-1}\right), &\qquad
        \tilde{\theta}_k\approx\arg\min_{\theta\in\mathcal{M}}h_{\kappa}(\theta, \vartheta_k), \\  \tilde{\vartheta}_k=\mathcal{R}_{\theta_{k-1}}\left(\frac{1}{\alpha_k}\mathcal{R}_{\theta_{k-1}}^{-1}\tilde{\theta}_k\right), &\qquad \frac{1-\alpha_{k+1}}{\alpha_{k+1}^2}=\frac{1}{\alpha_k^2}.
    \end{align*}
\end{enumerate}

One needs a stopping criterion, since we cannot use the functional gap  as a stopping criterion  here as in the convex case.  A stationarity stopping criterion is  adopted which consists of two conditions:
\begin{itemize}
    \item {\bf Descent condition} $h_{\kappa}(\theta, \vartheta) \leq h_{\kappa}(\vartheta, \vartheta);$
    \item {\bf Adaptive stationary condition}  ${\rm dist} \big(0_{\theta},\partial_{\theta} h_{\kappa}(\theta, \vartheta)\big) < \kappa d_{\mathcal{R}}(\theta, \vartheta).$
    \end{itemize}
Here, ${\rm dist}(\cdot, \cdot)$ denotes the standard Euclidean distance on the tangent space.

Recall that a quadratic of the retraction distance  is added to $f$ to make the subproblem convex. So if the weak-convexity parameter $\rho$ is known, then one should set $\kappa>\rho$ to make the problem convex. In this case, it is proven that the number of inner calls to $\mathcal{A}$ for the subproblems
\begin{equation}\label{SubProblem}
    \min_{\vartheta\in\mathcal M}h_{\kappa}(\vartheta, \theta)
\end{equation}
can be bounded by proper initialization point $\vartheta_0:$
\begin{itemize}
    \item if $f$ is smooth, then set $\vartheta_0=\theta;$
    \item if $f=f_0+\psi,$ where $f_0$ is $L$-smooth, then set $\vartheta_0={\rm prox}_{\eta\psi}\big(\mathcal{R}_{\theta}\left(\eta\nabla f_0(\theta)\right)\big)$ with $\eta\leq\frac{1}{L+\kappa}.$
\end{itemize}

However, in general one does not have  knowledge of $\rho.$ Thus we  propose a method that allows algorithm $\mathcal{A}_1$ (Algorithm~\ref{adapt}) to handle the convexity problem adaptively.

Our idea is to let $\mathcal{A}$ run on the subproblem for $T$ predefined iterations, output the point $\bar{\theta}_T,$ and check if a sufficient decrease occurs. If the subproblem is convex, then the aforementioned descent and adaptive stationary conditions are guaranteed. If either of the conditions are violated, then the subproblem is deemed non-convex. In this case, we double the value $\kappa$ and repeat the previous steps

 The tuning parameter $\kappa$ should be  chosen big enough to ensure the  convexity of the subproblems and simultaneously small enough to obtain the optimal complexity by not letting the subproblem deviate too far away from the original objective function. Thus we introduce $\kappa_{cvx}$ as an $\mathcal{A}$- {\it dependent smoothing parameter.} Notice that the linear convergence  rate $\tau_{\mathcal{A}, h_{\kappa}}$  of $\mathcal A$ is independent of the prox-center and varies with $\kappa$. We define $\kappa_{cvx}$ as

\begin{equation*}
    \kappa_{cvx}=\arg\max_{\kappa>0}\frac{\tau_{\mathcal{A}, h_{\kappa}}}{\sqrt{L+\kappa}}.
\end{equation*}

\begin{algorithm}
	\caption{$\mathcal{A}_1$: The Adaptation Algorithm on Manifolds }
	\label{adapt}

{\bf input}
 the point $\theta\in \mathcal{M},$ the smoothing parameter $\kappa$ and the number of iterations $T$\\
	\Repeat { \rm $h_{\kappa}(\bar{\theta}_T, \theta)<h_{\kappa}(\theta, \theta)$ and ${\rm dist}(\partial h_{\kappa}(\theta_T, \theta), 0_{\theta_T})<\kappa d_{\mathcal{R}}(\theta_T, \theta)$}
	{Compute
	\begin{equation*}
    \bar{\theta}_{T}\approx\arg\min_{\vartheta\in\mathcal{M}} h_{\kappa}(\vartheta, \theta)
    \end{equation*}
    by running $T$ iterations of $\mathcal{A}$, using the initialization strategy described below Equation~\eqref{SubProblem}.\\
    {\bf If} $h_{\kappa}(\bar{\theta}_T, \theta)>h_{\kappa}(\theta, \theta)$ or ${\rm dist}(\partial h_{\kappa}(\theta_T, \theta), 0_{\theta_T})>\kappa d_{\mathcal{R}}(\theta_T, \theta)$\\
	{\bf then} go to repeat by replacing $\kappa$ with $2\kappa.$\\

	}
{\bf output} $(\theta_T, \kappa)$
\end{algorithm}

Finally, for an initial estimate $\theta_0\in \mathcal{M},$ smoothing parameters $\kappa_0, \kappa_{cvx},$  an optimization algorithm $\mathcal{A}$, and a stopping criterion based on
a fixed budget $T$ and $S$, we have the following acceleration algorithm, $\mathcal A_2$, for the manifold (Algorithm \ref{Catlalyst2}).

\begin{algorithm}
	\caption{$\mathcal{A}_2$: Acceleration Algorithm on  Manifolds }
	\label{Catlalyst2}
	Initialize $\Tilde{\vartheta}_0=\theta_0,$ $\alpha=1.$\\
	\Repeat{\rm the stopping criterion is \ \ \ ${\rm dist}(\partial f(\bar{\theta}_k), 0_{\bar{\theta}})<\varepsilon$}
{for $k=1, 2, ...$
 \begin{enumerate}
\item compute $(\bar{\theta}_k, \kappa_k)=\mathcal{A}_1(\theta_{k-1}, \kappa_{k-1}, T)$
\item compute $\vartheta_k=\mathcal{R}_{\theta_{k-1}}\left(\alpha_k\mathcal{R}^{-1}_{\theta_{k-1}}\Tilde{\vartheta}_{k-1}\right)$ and apply
      $S_k\log(k+1)$ iterations of $\mathcal{A}_1$ to find
   \begin{equation*}
    \Tilde{\theta}_k\approx\arg\min_{\theta\in\mathcal{M}} h_{\kappa_{cvx}}(\theta, \vartheta_k),
   \end{equation*}
   by using initialization strategy described below \eqref{SubProblem}.
\item Update $\Tilde{\vartheta}_k$ and  $\alpha_{k+1}$:
 \begin{align*}
    \tilde{\vartheta}_k&=\mathcal{R}_{\theta_{k-1}}\left(\frac{1}{\alpha_{k}}\mathcal{R}^{-1}_{\theta_{k-1}}\Tilde{\theta}_k\right),\\
    \alpha_{k+1}&=\frac{\sqrt{\alpha_k^4+4\alpha_k^2}-\alpha_k^2}{2}.
 \end{align*}
\item Choose $\theta_k$ to be any point satisfying $f(\theta_k)=\min\{f(\bar{\theta}_k), f(\Tilde{\theta}_k)\}.$
\end{enumerate}
}
\end{algorithm}

\begin{remark}
Note that there are two sequences  $\left\{\tilde{\theta}_k\right\}$  and $\left\{\bar{\theta}_k\right\}$ in Algorithm $\mathcal A_2$. Since the extrapolation step is designed for the convex case, the second sequence $\{\tilde{\theta}_k\}$ approximates the optimal point with accelerated rate which means that it approaches the optimal point faster than the first sequence $\left\{\bar{\theta}_k\right\}$ above. Intuitively, when the first sequence is chosen it uses the initial algorithm $\mathcal A$ and adapts the smoothing parameter to our objective--implying that the Nesterov step failed to accelerate convergence.
\end{remark}

In the adaptation method $\mathcal{A}_1(\theta_{k-1}, \kappa_{k-1}, T)$, the resulting $\bar{\theta}_k$ and $\kappa_k$ have to satisfy the following inequalities
\begin{align}\label{C1}
    {\rm dist}\big(0_{\bar{\theta}_k}, \partial h(\bar{\theta}_k, \theta_{k-1})\big)&<\kappa_k d_{\mathcal{R}}(\bar{\theta}_k, \theta_{k-1}) \quad \text{and} \\
    \quad h_{\kappa_k}(\bar{\theta}_k, \theta_{k-1})&\leq h_{\kappa_k}(\theta_{k-1}, \theta_{k-1}).
\end{align}
The resulting $\Tilde{\theta}_k,$
needs to satisfy the condition that if the function $f$ is convex, then
\begin{equation}
    \label{C2}
    {\rm dist}\big(0_{\Tilde{\theta}_k}, \partial h_{\kappa_{cvx}}(\Tilde{\theta}_k, \vartheta_k)\big)<\frac{\kappa_{cvx}}{k+1}d_{\mathcal{R}}(\Tilde{\theta}_k, \vartheta_k).
\end{equation}
We then have the following lemma:
\begin{lemma}
\label{lem-2}
Suppose $\theta$ satisfies ${\rm dist}(0_{\theta}, \partial h_{\kappa}(\theta, \vartheta))<\varepsilon,$ and $|\nabla d_{\mathcal{R}}^{2}(\theta, \vartheta)|\leq Kd_{\mathcal{R}}(\theta, \vartheta)$, then the inequality holds:
\begin{equation*}
    {\rm dist}(0_{\theta}, \partial f(\theta))\leq\varepsilon+\kappa Kd_{\mathcal{R}}(\theta, \vartheta).
\end{equation*}
\end{lemma}

\begin{proof}
We can find $v\in\partial h_{\kappa}(\theta, \vartheta)$ with $\|v\|\leq\varepsilon.$ Taking into account $\partial h_{\kappa}(\theta, \vartheta)=\partial f(\theta)+\kappa\nabla d_
{\mathcal{R}}^{2}(\theta, \vartheta)$ the result follows.
\end{proof}
Since we assume retraction distance function $d_{\mathcal{R}}$ is continuous, we can deduce that the vector field $\nabla d_{\mathcal{R}}^{2}(\theta, \vartheta)$ is continuous, so the conditions of Lemma \ref{lem-2} are very mild. Also, as mentioned previously, the square retraction distance function $d_{\mathcal{R}}^2(\cdot, \vartheta)$ acts like a square normal function in a small neighborhood of $\vartheta$.

We define the following retraction-based strongly convex function:
\begin{definition}
A function $f$ is {\it $\mu$-strongly convex with respect to the retraction} $\mathcal{R}$ if for any points $\theta_1, \theta_2\in\mathcal{M}$ and $\mu>0$ the inequality holds
\begin{equation}
\label{f-strongly-convex-2}
    f(\theta_2)\geq f(\theta_1)+\langle v, \mathcal{R}_{\theta_1}^{-1}\theta_2\rangle+\frac{\mu}{2}d_{\mathcal{R}}^2(\theta_1, \theta_2), \;  v\in\partial f(\theta_1).
\end{equation}
\end{definition}

Then we have the following convergence analysis for the acceleration algorithm $\mathcal{A}_2$:

\begin{theorem}
Fix real-valued constants $\kappa_0, \kappa_{cvx}>0$ and the point $\theta_0\in\mathcal{M}.$ Set $\kappa_{\max}=\max_{k\geq1}\kappa_k.$ Suppose that the number of iterations  $T$  is such that $\bar{\theta}_k$ satisfies \eqref{C1}, and $|\nabla d_{\mathcal{R}}^{2}(\theta, \vartheta)|\leq Kd_{\mathcal{R}}(\theta, \vartheta).$ Define $f^{*}=\lim_{k\rightarrow\infty}f(\theta_k).$ Then for any $N\geq1,$ the iterated sequence generated by the
acceleration algorithm satisfies
\begin{equation*}
        \min_{j=1, ..., N}\Big\{{\rm dist}^2\big(0_{\bar{\theta}_j}, \partial f(\bar{\theta}_j)\big)\Big\}\leq\frac{8\kappa_{\max}K^2}{N}\left(f(\theta_0)-f^{*}\right).
\end{equation*}
If in addition the function $f$ is $\kappa_{cvx}(K_1^4K_2^4-R_1)$-strongly convex and $S_k$ is chosen so that $\Tilde{\theta}_k$ satisfies \eqref{C2}, then
\begin{equation}\label{acceleration}
   f(\theta_N)-f^{*}\leq\frac{4\kappa_{cvx} K_1^2K_2^2}{(N+1)^2}d_{\mathcal{R}}^2(\theta^{*}, \theta_0),
\end{equation}
  where $\theta^{*}$ is any minimizer of the function $f.$
\end{theorem}

The detailed proof of this theorem can be found in the Appendix.

\begin{remark}
If the original method $\mathcal A$ has a linear rate of convergence then our method $\mathcal A_2$ also converges to the local minimum for the strongly convex case.  If the knowledge of the strong-convexity is given, then some existing method can achieve optimal linear rate for smooth and convex functions \citep{pmlr-v49-zhang16b}, however, it is overall an extremely difficult to verify convexity of a function on a manifold, and our method adapts to that without requiring the knowledge of the convexity.
Note that our algorithm also applies to the subgradient descent method, where instead of gradient of the function one takes the subdifferential, for non-smooth functions. In this case, for the strongly-convex objective, the subgradient method converges to the optimum with  $O(1/N)$  rate of convergence (see \cite{pmlr-v49-zhang16b}). Thus our  accelerated rate  $O(1/N^2)$ can be considered optimal for strongly-convex functions on the manifold.
\end{remark}

\section{Related work}
\label{sec:related}

\cite{NIPS2017_7072} propose accelerated first-order methods for \emph{geodesically convex optimization} on Riemannian manifolds. This is a direct generalization of Nesterov's original linear extrapolation mechanism to general Riemannian manifolds via a non-linear operator. One drawback of \cite{NIPS2017_7072} is that the accelerated step of their algorithm  involves exact solving of non-trivial implicit equations.

\cite{pmlr-v75-zhang18a} later proposed a computationally tractable accelerated gradient algorithm and a novel estimation sequence for convergence analysis. Our approach is fundamentally different from theirs. We regularize an objective function with a squared retraction distance (see Proposition~\ref{prop1}), solve a sequence of convex subproblems, adapt to the degree of weak convexity of the objective function, and produce accelerated rates for convex objectives.
Even in the convex case, our approach can deal with a much broader class of retraction-based convex functions.

\cite{pmlr-v84-paquette18a} proposes a general scheme called ``Catalyst acceleration'' for solving general optimizations  in  Euclidean space, which has inspired development of  some ideas for our work.  Similar ideas have been explored  for convex functions in Euclidean space in both theory and practice \citep{Lin:2017}. However, optimization problems on manifolds are of fundamentally different nature and require development of  substantially new tools and theory.

There is an interesting line of work on proposing fast  algorithms for stochastic optimization on manifolds \citep[see][]{Zhang:2016:RSF, 2018arXiv181104194Z, pmlr-v89-zhou19a, 6487381} which
employ very different techniques such as minibatching, variance reduction and  utilizing the uncertainty of inputs. Methods like \cite{zhang2018towards} propose optimization methods that are analogous to Nesterov-type algorithms for manifold spaces.

\section{Simulation study and data analysis}\label{sec-simu}
To examine the convergence and acceleration rates of our proposed algorithm, we first apply our method to the estimation of both intrinsic and extrinsic Fr\'echet means on spheres, in which one has the exact optima for comparison in the case of extrinsic mean. We also apply our algorithm to the Netflix movie-ranking data set as an example of optimization over Grassmannian manifolds in the low-rank matrix completion problem.

\subsection{Estimation of intrinsic Fr\'echet means on manifolds}\label{subsec:frechet}
We first consider the estimation problem of Fr\'echet means on manifolds \citep{frechet}.
In this simple example, we have observations $\{x_1,\ldots, x_N\}$ that lie on a sphere $ \mathbb{S}^{d} $ and our goal is to estimate the sample mean:
\begin{align}
\hat\theta=\arg\min_{\theta\in S^d} f(\theta), \hspace{1em} f(\theta) = \sum_{i=1}^n \rho^2(\theta, x_i).
\end{align}
If $\rho$ is the embedded distance metric in the Euclidean space, then there exists a closed form solution $\hat\theta=\sum_{i=1}^N x_i/\| \sum_{i=1}^N x_i\|$, which is the projection of the Euclidean mean $\bar{x}$ onto the sphere \citep{rabibook}.  This is called the \emph{extrinsic mean.} When $\rho$ is taken to be the geodesic or intrinsic distance, $\hat\theta$ is called the \emph{intrinsic mean}.  We will consider estimation of both extrinsic and intrinsic means using our method compared to other optimization techniques.

One simple examples of a retraction map for $\mathbb{S}^d$ is
\begin{equation*}
    \mathcal{R}_{\vartheta}v=\frac{\vartheta+v}{|\vartheta+v|},
\end{equation*}
where $|\cdot|$ is the Euclidean norm in $\mathbb{R}^{d+1}.$
Therefore the inverse retraction has the following expression
\begin{equation*}
    \mathcal{R}_{\vartheta}^{-1}\theta=\frac{1}{\vartheta^T\theta}\theta-\vartheta.
\end{equation*}

We first compare our accelerated method against gradient descent optimization and a Newton-type optimization scheme, DANE \citep{Shamir}, and a Nesterov method, RAGD \citep{zhang2018towards}, adapted for manifolds. For all the experiments in this section,  we optimized the step size of the optimizer  using an Armijo condition backtracing line seach \citep{armijo1966minimization} where we reduce the step size by a factor of $ .95 $ until the difference between the old loss function evaluation and the new one is $ 10^{-5} \times .95$. For our Catalyst algorithm manifold we set the A2 budget to $ S=10 $, the A1 number of iterations to $ T=5 $, and cutoff parameter for A1 is initialized at $ .1 $. For the DANE results, we set the regularization term to $ 1 $, For RAGD we set the shrinkage parameter to $ 1 $. Our synthetic data set is 10,000 observations generated i.i.d from a $100$ dimensional $\mbox{N}(0, I)$ distribution projected onto $ \mathbb{S}^{99} $.

We run each optimization routine for 100 iterations. Figure~\ref{fig:intrinsic_comparison} and Figure~\ref{fig:extrinsic_comparison} shows that our novel accelerated method converges, for an intrinsic mean as well as an extrinsic mean example, to an optima in fewer iterations than the other competing methods, both in terms of the loss function value and the norm of the loss function gradient. Moreover, we can see in the intrinsic mean example, our method is able to obtain a smaller loss function and gradient norm than the competing methods. In the extrinsic mean example, our method obtains a comparable loss function value and MSE between the learned parameter and the closed-form expression of the sample mean with other methods in fewer iterations and obtains a smaller gradient norm than the competing methods.

By explicit calculation we show the objective functions are strongly convex over a neighborhood of any point on the manifold (see the Appendixx for a proof). This is a highly non-trivial task for general objective functions, hence necessitating an adaptive method such as ours. Moreover, in the extrinsic mean example, since we have a closed form expression of the Fr\'echet mean we also show that our optimization approach converges to the true extrinsic mean in terms of mean squared error faster than the other optimization methods.
\begin{figure}
	\centering
	\includegraphics[width=.5\linewidth]{figs/intrinsic_iterations.png}\includegraphics[width=.5\linewidth]{figs/intrinsic_wall_time.png}
	\caption{Intrinsic mean comparison on spheres}
	\label{fig:intrinsic_comparison}
\end{figure}
\begin{figure}
	\centering
	\includegraphics[width=.5\linewidth]{figs/extrinsic_iters.png}\includegraphics[width=.5\linewidth]{figs/extrinsic_wall_time.png}
	\caption{Extrinsic mean comparison on spheres}
	\label{fig:extrinsic_comparison}
\end{figure}
\subsection{Real data analysis: the Netflix example}\label{subsec:netflix}
Next, we consider an application of our algorithm to the Netflix movie rating dataset. This dataset of over a million entries, $ X \in \mathbb{R}^{M \times N} ,$ consists of $M = 17770$ movies and $N = 480189$ users, in which only a sparse subset of the users and movies have ratings. In order to build a better recommendation systems to users, we can frame the problem of predicting users' ratings for movies as a low-rank matrix completion problem by learning the rank-$ r $ Grassmannian manifold $ U \in \mbox{Gr(M, r)} $ which optimizes for the set of observed entries $ (i,j) \in \Omega$ the loss function
 \begin{equation}
		L(U) = \frac{1}{2} \sum_{(i,j) \in \Omega} \left\{ (UW)_{ij} - X_{ij} \right\}^{2} + \frac{\lambda^{2}}{2} \sum_{(i,j) \notin \Omega}(UW)_{ij},
\end{equation}
where $W$ is an $r$-by-$N$ matrix. Each user $ k$ has the loss function $\mathcal{L}(U,k)=\frac{1}{2} \left|  c_k\circ\left(  Uw_k(U)-X_k \right)     \right|^{2}$ , where $\circ$ is  the Hadamard product, $(w_k)^{i}=W_{ik},$ and
\begin{equation*}
\begin{split}
    (c_{k})^i=\begin{cases}
    1, & {\rm if} \ \ \ (i, k)\in \Omega \\
    \lambda, & {\rm if} \ \ \ (i, k)\notin \Omega
    \end{cases},
    \qquad
    (X_{k})^i=\begin{cases}
    X_{ik}, & {\rm if} \ \ \ (i, k)\in \Omega \\
    0, & {\rm if} \ \ \ (i, k)\notin \Omega,
    \end{cases}\\
    w_k(U)=\big(U^T{\rm diag}(c_k\circ c_k)U\big)^{-1}U^T\big(c_k\circ c_k\circ X_k\big).
\end{split}
\end{equation*}
This results in the following gradient
\begin{align*}
    \nabla\mathcal{L}(U, k)&= \big(c_k\circ c_k\circ(Uw_k(U)-X_k)\big)w_k(U)^T\\
    &= {\rm diag}(c_k\circ c_k)(Uw_k(U)-X_k)w_k(U)^T.
\end{align*}

For this problem on Grassman manifolds, we have the retraction map:

\begin{equation}
\mathcal{R}_{V}U = U+V
\end{equation}

and the inverse retraction map:

\begin{equation}
\mathcal{R}_{V}^{-1}U = V - U(U^{T}U)^{-1}U^{T}V
\end{equation}

We look at a comparison of our method against a standard gradient descent method on a subset of the data where we only observe a million ratings ($ \approx 1.5\%$ of the full data set). In this setting we fix the matrix rank $ r=5 $ and the regularization parameter $ \lambda = .01 $. Figure~\ref{fig:parallel_netflix} shows that our accelerated method obtains a smaller loss function value, a smaller identical test set MSE, and nearly identical loss gradient norm faster than RAGD, DANE, or a typical gradient descent approach.

On a large scale, we apply a parallelized version of our accelerated method  and a communication-efficient parallel algorithm on manifolds proposed in \cite[ILEA]{lizhennips2018} on the full Netflix dataset. We randomly distribute the data across 64 processors and run the optimization routine for 200 iterations. In Figure~\ref{fig:parallel_netflix}, again we can see steady acceleration that our method provides in terms of the loss function value across iterations and the loss of gradient norm though ILEA obtains slightly better test set MSE than our method.
\begin{figure}
	\centering
	\includegraphics[width=.5\linewidth]{figs/netflix_wall_time.png}\includegraphics[width=.5\linewidth]{figs/parallel_netflix_iters.png}\\
	\includegraphics[width=.5\linewidth]{figs/netflix_iters.png}\includegraphics[width=.5\linewidth]{figs/parallel_netflix_wall_time.png}
	\caption{Results for the parallel (right) and reduced (left) Netflix example.}\label{fig:parallel_netflix}.
\end{figure}

redpajama_set_name: RedPajamaArXiv


text:
University of St Andrews Research
A critical examination of indices of dynamic interaction for wildlife telemetry studies
Long_2014_JAnE_83_PrePrint.pdf (1.226Mb)
Long, Jed
Nelson, Trisalyn
Webb, Stephen
Gee, Kenneth
Biased random walk
Contact rate
GPS telemetry
Odocoileus virginianus
Sampling resolution
Static interaction
G Geography (General)
Altmetrics DOI Statistics
1. Wildlife scientists continue to be interested in studying ways to quantify how the movements of animals are interdependent – dynamic interaction. While a number of applied studies of dynamic interaction exist, little is known about the comparative effectiveness and applicability of available methods used for quantifying interactions between animals. 2. We highlight the formulation, implementation and interpretation of a suite of eight currently available indices of dynamic interaction. Point- and path-based approaches are contrasted to demonstrate differences between methods and underlying assumptions on telemetry data. 3. Correlated and biased correlated random walks were simulated at a range of sampling resolutions to generate scenarios with dynamic interaction present and absent. We evaluate the effectiveness of each index at identifying different types of interactive behaviour at each sampling resolution. Each index is then applied to an empirical telemetry data set of three whitetailed deer (Odocoileus virginianus) dyads. 4. Results from the simulated data show that three indices of dynamic interaction reliant on statistical testing procedures are susceptible to Type I error, which increases at fine sampling resolutions. In the white-tailed deer examples, a recently developed index for quantifying local-level cohesive movement behaviour (the di index) provides revealing information on the presence of infrequent and varying interactions in space and time. 5. Point-based approaches implemented with finely sampled telemetry data overestimate the presence of interactions (Type I errors). Indices producing only a single global statistic (7 of the 8 indices) are unable to quantify infrequent and varying interactions through time. The quantification of infrequent and variable interactive behaviour has important implications for the spread of disease and the prevalence of social behaviour in wildlife. Guidelines are presented to inform researchers wishing to study dynamic interaction patterns in their own telemetry data sets. Finally, we make our code openly available, in the statistical software R, for computing each index of dynamic interaction presented herein
Long , J , Nelson , T , Webb , S & Gee , K 2014 , ' A critical examination of indices of dynamic interaction for wildlife telemetry studies ' , Journal of Animal Ecology , vol. 83 , no. 5 , pp. 1216-1233 . https://doi.org/10.1111/1365-2656.12198
Journal of Animal Ecology
© 2014 The Authors. Journal of Animal Ecology © 2014 British Ecological Society, This is the accepted version of the following article: A critical examination of indices of dynamic interaction for wildlife telemetry studies Long, J., Nelson, T., Webb, S. & Gee, K. Sep 2014 In : Journal of Animal Ecology. 83, 5, p. 1216-1233, which has been published in final form at http://onlinelibrary.wiley.com/doi/10.1111/1365-2656.12198/pdf
The deer data used in this study were supported by funding from The Samuel Roberts Noble Foundation.


"What is the Schoolyards Program? I never heard of it!" "Common Ground supports School Gardens? Who knew?" "Wait, what's the difference between a garden and a habitat?" "Can you guys come and teach our teachers about how to take their kids outside for learning and not just recess?" These questions frequently come up in our community and as the Schoolyards Program Manager I thought I'd give a little Schoolyards 101 in the hopes of raising awareness about the primary ways Common Ground engages with New Haven Public Schools as well as a preview of some exciting developments coming up this year.
The Schoolyards Program started as the CT School Garden Resource Center in 2012 and while the CT School Garden Resource Center still exists, it is just one component of a three-pronged program, including School Gardens, Schoolyard Habitats, and Professional Development focused on teaching outside and utilizing School Gardens and Habitats for schools, organizations and individuals. We focus our hands-on support of school gardens and schoolyard habitats to New Haven and are in the process of expanding our impact statewide through Professional Development and an exciting new awards program (more on that in a minute).
To date, the Schoolyards Program has partnered with 15 New Haven Public Schools to help build and utilize school gardens on their school sites. Schools apply to be a partner and in the first year receive intensive support for everything from design and installation to help with curriculum integration and developing maintenance plans, in addition to field trips to Common Ground for their students. After their first year, schools receive support tailored to their evolving needs and goals, including work days with our Green Jobs Corps Schoolyards Crew, presentations at faculty meetings, and workshops for teachers. Most schools are focused on growing vegetables and herbs in their gardens and utilize the spaces to teach about the natural versus built environment, plant parts and how plants grow, the water cycle, the food web, health and nutrition. We are piloting the installation of two new school gardens this year with the explicit purpose of serving as refocusing, therapeutic spaces for students struggling with behavior, trauma or other issues.
In addition to school gardens, many of our partner School Garden Schools – 12 schools as of this fall – in New Haven have also gone through the official Schoolyard Habitat certification process in partnership with the Schoolyards Program, Audubon CT, US Fish & Wildlife Service, and the Yale Peabody. Schools design a habitat space in their schoolyard that can serve as both an outdoor classroom for students and as an oasis for birds and pollinators; if you took a tour of Schoolyard Habitats in New Haven you would see butterfly gardens, a wide variety of native plants and trees, logs for seating, nature trails and students and teachers busy studying all the life to be found in these spaces. Schoolyards Program staff provide a suite of experiences for participating schools including professional development focused on utilizing outdoor spaces as classrooms, field trips to Common Ground's flagship Schoolyard Habitat Wetland, and Green Jobs Corps Schoolyards Crew support for habitat installation and maintenance.
This spring we will be launching the first ever statewide CT School Garden Awards Program to recognize superior school garden programs in Connecticut. Schools will be nominated in several categories, including health & wellness, curriculum integration, and community involvement. After an application and vetting process, winners will be announced in Fall 2018, with cash prizes given at an awards event. We are so grateful to FoodCorpsCT, UCONN Extension, and Put Local On Your Tray, who have signed on to sponsor this exciting award program in its inaugural year. Check back in the winter for more details!
Interested in learning more? We have a Kick-Off Schoolyard Habitat Exchange coming up on Tuesday, October 17th from 4-6pm at Fair Haven School. It's free and open to the public. Come take a tour of the Fair Haven Schoolyard Habitat and take a seed saving workshop with Jim Sirch of the Yale Peabody.
Please contact suzannah.holsenbeck@commongroundct.org or kendra.dawsey@commongroundct.org for more information on the Schoolyards Program.


"The media is no one's friend. That goes double for social media." – Frank Sharpe, Fortress Defense.
Friday (9/22/17) morning a Huntington Beach, CA, police officer was involved in a daylight shooting outside of a local 7-11 store. By 9:30am a 26-second video taken by a bystander was uploaded to Twitter.
Within an hour, local and national news stations had run with it, and their comment sections, along with social media, began the usual cop hatred and ridiculous armchair quarterbacking that has been a blight on truth since the advent of the internet. People with nary a clue spewing emotionally-based bile via keyboard.
And there are 10's of thousands of such comments that have been posted. I won't bother dissecting, instead I'll simply point out that this represents a cross-section of the American voting public and potential jury pool.
Media outlets were also doing their best to slander the officer without giving the appearance of taking sides. A slime tactic "news" organizations are quite adroit at.
It's nice that our "expert" at least left himself an out, however, there isn't an ounce of "panic" displayed by the officer at any time. Especially when one is exposed to even more of the event, which we'll get to after the next example.
KCAL starts their coverage off with the copy: It started as a scuffle, but ended with deadly gun fire…", and makes sure they go heavy on the opinions of those who are outraged. They also focus on the right hand of the VCA getting a hold of the officer's magazine, but fail to mention where his left hand is at, engaged a struggle for the officer's gun! They even zoom in on it! [Link at the beginning of paragraph] The officer has to pry his hand away in order to draw his gun.
As usual, the mob is wrong.
Here we see the officer with his Taser deployed and the "poor, innocent victim" refusing to comply and moving towards the officer rapidly. The officer deploys the Taser, and the VCA rips the barbs and wires away, then attacks the officer.
We have multiple failures to comply with orders, a failed tasing, a physical attack, and a hand on the officer's gun during a fight.
Yeah, he got shot. That's what happens when you behave in a way that presents an officer with no other conclusion than you want to murder them – The end.
Happy ending? No. Proper ending? Yes.