text:
★ 'Pale God' by Walker Storz
Posted on January 18, 2018 January 31, 2018 by softcartel
Jamey was dead tired. He was talking to his mother when something inside him snapped. It might as well have been a bone.
He had been sick for a couple years, and nothing much was getting better. If exceptions proved the rule, he was pretty sure his moments of clarity and energy were exceptions. Feeling 'normal,' or not sick, was rare, and was much like being high–he learned not to trust his judgement on those days. Promises would be made that, like curses in the daytime, wouldn't hold when he came back down.
Life had become temporally distorted. His therapist was right when he said that being sick was much like being high, but he didn't know how much Jamey hated being high. At least with pot. Every high was a dissociative nightmare. The narcosis always revealed the screaming dissolution of the universe. Those famed synchronicities of psychedelic trips would sometimes appear, only as if to mock Jamey, as if moments of order were famous, rare creatures dying of pollution by entropy. Being sick had taken him from being a healthy, if angsty, 20 year old to inhabiting a strange fever-dream in which he had the energy and mind of an elderly man, but a still-childish body.
When he looked at his mother's face in a sudden new light, he had the realization that there was no reason for things to get better, and so they probably wouldn't. He was tired of being a parasite, even if he had been made one against his will. Everyone was growing tired of him, he was sure of it. Every time that he fantasized about suicide, the protestations in his mind grew fainter. The fantasies grew more vivid, real, heavy. They were far more frequent than his sexual fantasies. It was as if he was gradually leaving the world of the living behind. He was less and less attached to his flesh.
Later that night Jamey went outside, feeling like he was leaving the world of the corporeal for the last time. He was fairly sure of being unnoticed. He had the keys to his sister's car, an old station wagon. He had sent an email to her with a service that allowed the sending to be delayed, so that she couldn't stop him.
I'm sorry. There's not much I can say. I'm mostly sorry that i'm going to smash up your car. In that light I've left you my debit card and taken the other one that has a little bit on it, for gas etc. The one I left you has that money I got from that gofundme for medical expenses. It's only 800$, I spent some of it. That's half of the cost of this car, I hope you get some insurance money or something.
I just got tired of this shit. Please, please, please don't hate me for this. You can't even imagine. It's not fun. I wish I could be with you and mom and dad. I want to be with the living. I'm taking up everybody's time and money and I'm just not fun anymore. I don't think I deserve this, but I realized that nobody's gonna come along and make things better for me.
Please don't try and figure out my password and go on my computer or my phone. There's nothing horrible or illegal on there but if you go through all of my searches and chats I would just feel very embarrassed, and some of them would probably show me to be a mean and petty person, or just very strange. I know it doesn't make any sense but I can't rest knowing you might do that. Everyone deserves a little privacy.
I really love you. I can't say much more or I might turn around and decide not to do it, but things aren't gonna get better and I can't turn around.
The car–he had come to this out of a quick meditation. He didn't have a gun, or even enough pills of any seriousness to guarantee anything beyond maiming himself inside. Every time he thought about suicide he had the sudden desire to do something that would allow him to die "happy". Not so much a bucket list as a bucket shortlist. His illness had rendered him always-tired, near catatonic, and worst of all, boring. Jamey was determined to die after having felt like he had stuck his finger in a light-socket. If there was divinity he wanted a taste of it.
With a car you could experience this. After all, it was America. He would drive to the horizon and beyond. This was his fantasy. In reality, he had made up his mind to get to some canyons out west, or a mountain road, and just drive off the edge. He was sure that he had at least 24 hours before his parents decided to call the police, and he would take back roads after that point. He would have a car chase before he died. Either he would get away, and die shot into the bottom of a majestic pit, or he would commit suicide by cop. He felt guilty about the people who could be collateral damage, but he had been driven to this. Nobody deserved to have to choose their own death, and he was determined to at least make it memorable. He would not die as a loser, he would not live as a ghost.
A few days earlier, he had broken down and done some oxy with an acquaintance. It was too much effort to call him a friend. Jamey had avoided all drugs for awhile, largely because he had a naive dream that he would attain vigorous good health simply by avoidance of immoral and degenerate behaviors. But he couldn't hold out forever, feeling how he had. They had laid out a few generous lines of "roxy 30s," little blue pills that crushed easy, like soft chalk. When the oxy really hit Paul, he rocked back and forth in insensate pleasure, exclaiming "Money in the bank… feels like money in the bank, ahhh". As Jamey tasted the bitter drip and awaited the high washing over him, he grinned, feeling like crying insanely and joyously. Paul was an idiot, but he was right. This did feel like "money in the bank". Like plenty to the point of overflowing. He had been so poor for feeling recently, and this was like winning the fucking lottery. He looked up at the light. It, too, was like money. Everything was like money-energy, flowing through everything. You just needed to be at the right place at the right time, ready to siphon it into your account.
Things changed quickly, and he could barely even remember feeling that good. It was 4 am and he had made it to somewhere in Pennsylvania, as the black screen of the night began to become translucent, letting a small quantity of light through. He had been speeding only moderately, not wanting a reason to get pulled over or noticed by a policeman. There was nothing illegal in his car, but he knew he seemed unhinged and would not stand up to any kind of questioning or delay without breaking down. This was the kind of grand gesture that had to be done in one continuous motion, or it would fall apart totally. The gas station he pulled in at was fairly small, only two pumps, and lit by a ghostly green-tinged glow coming from inside. Walking inside, Jamey stuffed a container of pills into his pocket out of habit. Normally he couldn't handle stimulants without making his body pay for it later–his illness magnified the crash intensely, but this wasn't a normal time. There would be no crash. He would ride into the sun on a white-hot wave of energy and light. There was no point in worrying about a body that he was jettisoning in his race to the end of the earth. A smile had started to play on his lips as he entered the store, jingling a bell and seemingly startling the man behind the counter out of a light sleep.
"Can I help you?" the man asked nervously. His short brown hair was ruffled, and he appeared to be about Jamey's own age, maybe a little older. Jamey looked at him, scrutinizing him sharper than he meant to, suddenly feeling energy and possibility flowing through him. "30 dollars on pump 1," Jamey said, handing the attendant his card. When he got the card back from the clerk, he went into the bathroom. It was dingy and smelled like antiseptic cleaner and feces. Jamey was too excited to care. He took a 10 mg adderall out of his pocket and crushed it on the back of his phone, snorting it while he took a shit. He took out one of the leftover oxys and did the same. He had snorted heroin while on a toilet one other time, before going to an art opening at his college. It had had a strange sort of thrill, as if covering up a profane act with a somehow even more profane act, or entering an eerie, chthonic world via the bathroom.
He wiped his ass, flushed, and pulled his pants up. Washing his hands, he made use of the mirror, making sure he cleaned any visible powder out of his nostrils. His eyes were almost on fire. They looked fully steeled. He could already feel this shit hitting. He was riding something and it was too late to turn back, but he really didn't care.
Walking out of the bathroom and back into the store was like walking back into a party from the bathroom. This party was pretty empty, but he felt like he could fill it in pretty well. A sudden sense of warmth and hilarity was overtaking him. He grabbed a couple drinks for the road, some bad food, some gum, and asked the attendant for a pack of cigarettes. As he handed over his ID, he could barely hold himself back from telling him everything. I am going to die, he thought, staring again too intently into this kid's green eyes. You can't even imagine! The clerk looked unnerved. You are looking at a man who's going to die. All of these words wanted to burst from his mouth, but most of all he wanted to say that this kid reminded him of his ex. They both had pale, slender faces with angular, almost angry cheekbones. He suddenly wanted to call her up–Sylvie–and tell her everything, wanted to be able to join her, wherever she was at that moment. He was sure she'd welcome him, for some reason. All of the people who he loved in his life seemed to be drawing closer to him as he drew closer to death. He felt a sense of a warm circle closing in. Everyone important was going to be with him in these moments.
If only they could be there in flesh! This was still something though, he felt like he could almost touch them, they were growing so vivid in his mind.
Buoyed by an elation both chemical and saintly, he handed the clerk a few pills with the change that he gave back. "What's your name, kid?" (He didn't know what had come over him, calling someone no younger than him "kid"). But the clerk didn't act angry or surprised. "Evan". "Well, have fun with these, Evan." The "kid" laughed, but turned quickly serious. "Hey, are you sure you're alright?" "I feel like money in the bank, kid." Jamey tossed those words over his shoulder jauntily while he left the store.
He couldn't quite see the sun, and had a superstitious desire to be on the interstate before it peeked over the horizon, so he floored it out of the gas station, being already more reckless than he had planned to be. If I keep at it like this I'll never make it to Colorado. No cops apprehended him this time, but as he drove up the on-ramp, he resolved not to speed that much until he got most of the way there.
The greenish tint on that kid's face was still haunting him a little for some reason, so he put on a CD to push that out of his mind. The sun was coming up, flooding the sky with orange and purple, as the high, melodic bass and drums started spasming neatly. He turned to the passenger's seat, where he had put a few gatorades. Looking at them in the light, he suddenly felt weightless and repulsed to the point of nauseousness. These objects got to continue to be in the world… These bottles were more full of form and weight than he was. He was a rancid ghost at this point, driving what was left of his form into oblivion. All of the sudden he was full of a burning, sickly resentment toward the living, toward the healthy, toward all of the creatures that the sun seemed to be shining on and blessing even now. It was the first time that he questioned whether he was doing the right thing.
The lack of sleep and his sickness and the pills were quite a heady combo, on top of driving an automatic on the highway, and Jamey started to drift off. He had pleasant, vast blue dreams like seas that he would inhabit for what seemed like years, and then he would snap awake, and it had only been seconds. Somehow he didn't lose control of the car through all this. Normally he would've been scared, but this felt incredibly strange and pleasant now that he wasn't attached to living or dying. The highway and surrounding grass seemed to him like a sea that he was navigating, the divider a bank. Sinking into these dreams felt like swimming in little tide pools, not quite connected to the wilder ocean. Finally he just decided to let go.
Jamey was in a parking lot somewhere, lying back, looking up at the sky. The first thing he noticed when he came to is that he felt fine. None of the pain he would expect from a crash, no hangover, just a sense of weightlessness that felt wonderful. It didn't even really feel like he had flesh, since flesh is defined by its knots and its weight and its pain. He sat up and looked around. There didn't seem to be any cars or people in the lot, which was strange, because as far as he could tell, it was a big strip mall in a suburb of some kind. It didn't look like an abandoned mall–the buildings looked nice, new, just empty.
He felt confused and apprehensive. He had been ready for anything, death from drugs or crashing, but what was this? As his mind was churning rapidly, his iPhone vibrated. Sylvie was calling him. He accepted the call.
"Jamey, I just had a dream about you, I need to know you're alright. I know it seems silly…" Her voice was tremulous. "Please just let me know you're okay…" His heart was pounding wildly in his chest. "Sylvie…" escaped his mouth, almost whispered. "Jamey, are you there?" She didn't seem to have heard him. "Jamey, I can't hear you!" she said hysterically. "Jamey, stop FUCKING WITH ME!" She was screaming and sobbing. He said her name over and over again, but she couldn't hear him at all, as far as he could tell. There was a sinking feeling in his stomach as he held the phone up to his ear, listening to her sob, and whirled around, looking for anyone that could help him.
She hung up, and he looked at the phone's screen. He started to stare into the glow intently, as if it was a lake with something under the surface. There was something about it drawing him in. He leaned more and more, and things started to dirty and pixellate and dissolve into a glitchy mess. He kept his eyes on the light of the phone screen and the hum it had. That hum was like money, whirring through the world ceaselessly, making it move. He kept his eyes on it even as he felt himself dissolve and sink into the concrete that just a minute ago was solid, as if fixing his gaze on it could save him, as if it was the sun that he had aimed at all those hours ago.
Posted in FICTIONTagged Walker Storz
Previous Post 'Simians 3' by Walker Storz
Next Post 'Gone' by Walker Storz
1 thought on "★ 'Pale God' by Walker Storz"
Wyatt M	 January 19, 201811:49 am	 Reply
Like money in the bank


Spring Security OAuth2 Client Support for WebFlux. See the sample for how to use it.
Implementation of the Authorization Code Grant. See the sample for how to use it.
If you have feedback on this release, I encourage you to reach out via StackOverflow, GitHub Issues, or via the comments section. You can also ping me @rob_winch , Joe @joe_grandja, or Josh @jzheaux on Twitter.


text:
Results for: Global Warming
Linda Marsa
Linda Marsa, an award winning investigative journalist, is the author of FEVERED, an acclaimed book on climate change and what we can do about it. Read More
Linda Marsa is an award-winning investigative journalist and a contributing editor at Discover who has covered medicine, science, health and the environment for more than two decades. She is a former Los Angeles Times reporter and author of FEVERED: Why a Hotter Planet Will Harm Our Health and How We Can Save Ourselves (Rodale, 2013), which the New York Times called "gripping to read." FEVERED won an honorable mention as best general nonfiction book of the year from the American Society of Journalists and Authors. She has also written for U.S. News & World Report, Aeon, Popular Science, The Daily Beast and Pacific Standard, among numerous others.
A popular speaker, Linda has lectured widely on climate change at major universities, leading environmental organizations, public health institutes and for general audiences, speaking at venues like George Washington University, Emory University, UC Irvine, University of Colorado and the Public Health Institute in Oakland. She has also been an instructor at The Writer's Program at UCLA Extension for more than two decades and was named Teacher of the Year in 1999.

redpajama_set_name: RedPajamaCommonCrawl


In this 6 week remote journey we'll connect each week by phone for our weekly intention and inquiry. We'll be working with a specific spirit remedy (energetics from wildflowers, crystals, gems or trees) each week to support us as we move through any blocks we have (confusion, fear, worry, grief, anger, etc.) that are limiting our power and divine guidance.
These energetics will also aid us in receiving intuitive guidance from our unique soul essence.
Week 1: GROUNDING - We need to feel grounded into our own bodies and Gaia to move through the intensity of these times. To trust the messages that we're receiving and to know that we each have a unique purpose here at this time.
Week 2: PROTECTION - This is particularly important for empaths to practice containing their own energy and encircling themselves from leaking or draining their energy, as well as removing themselves from energy 'vampires'. This is an important practice for all of us these days and pertains to protecting ourselves from the energies and 'information' rooted in fear.
Week 3: FEELINGS - We'll work towards accessing what you need help moving through in this moment, whether it be fear, confusion, resistance, worry, etc. As we access these pent up emotions, we can heal them and move forward.
Week 4: CLARITY - We'll connect to our Higher Soul Self for assistance through our dreams and decision making.
Week 5: INTUITION - We'll work towards our own visioning and psychic guidance and insights.
Week 6: CONNECTION - Pulling all of the intentional work we've done for the past 5 weeks, we'll feel more capable of connecting deeply with ourselves, each other for building community and intimate relationships, as well as a deep meeting with Gaia.
*You'll receive 6 Spirit Remedies as part of this remote journey to work with while we're together and for ongoing support. All of my essences are crafted with permission from the flower, stone or tree that I work with as well as reverence, ceremony and intuitive guidance.
*We'll begin the week of Dec. 19th but the exact Day will be determined collectively. I'd prefer it to be around 6/7 PM EST Mon - Thurs. But again, we'll decided that after registration closes.
*Sign Ups CLOSE Wednesday December 7th so I have time to get your remedies prepared and shipped before we begin.
*You'll receive our call in number and other information after registration closes.
*Energy Exchange - $175 includes 6 spirit remedies, weekly intentions, weekly calls and S/H. Our calls will be recorded if you miss one.
*Please contact me with any questions.


A former Nazi collaborator who served as a labour-camp guard in World War Two has died in Germany aged 95, local media report.
Jakiw Palij, who was thought to be the last Nazi collaborator living in the US, was deported to Germany last year after a lengthy legal battle.
Palij had been stateless since 2003 when a federal judge revoked his US citizenship over his Nazi links.
Reports say he died in a home for the elderly in the town of Ahlen.
Before his deportation, the former guard lived in the Queens borough of New York City.
His presence there, in a particularly diverse borough, became a point of contention when his past emerged.
His home was often picketed by protestors demanding his expulsion.
Palij's deportation case was resolved last year when Germany agreed to take him in - despite him not possessing German citizenship.
In August, the 95-year-old was taken from his home in a wheelchair and flown to Düsseldorf.
At the time Richard Grenell, the US ambassador to Germany, praised President Trump's leadership in helping to resolve the protracted legal dispute.
Palij is believed to have been born in 1923, in an area of Poland that is now in Ukraine.
US authorities alleged he was trained in 1941 by Nazis to work at Trawniki - a camp in German-occupied Poland.
The camp initially served as a holding centre for refugees and Soviet prisoners of war, but was eventually transformed by the SS into a training facility.
The thousands who trained there became known as Trawniki men - tasked with rounding up Jewish civilians from Poland and staffing killing centres at Sobibor, Treblinka and Belzec.
It also housed a forced labour camp where thousands of Jews died - including 6,000 who were killed in a single day in 1943.
US authorities accused Palij of being an armed guard at Trawniki, and said he played an "indispensible role" in ensuring Jews were killed.
Palij himself denied collaborating with the Nazis - telling the New York Times in 2003 he worked for them in fear for his own family's safety. He also denied having ever set foot in a camp himself.
A US court ruled he had assisted in the persecution of prisoners, but did not find him personally responsible for deaths.
The former guard entered the US through Boston in 1949, allegedly under a law meant to help refugees fleeing post-war Europe, the Associated Press report.
He was eventually granted US citizenship in 1957, and held it until his past came to light decades later.
In 2004 he was ordered to leave the country - but ambiguity over his citizenship delayed deportation.
For 14 years US authorities could not persuade Germany, Ukraine or Poland to take him in.
But last year, it was announced the German position had changed.
He was given a residency permit using a technicality decree intended for humanitarian or political purposes, German media report.
However, after Palij's arrival, German officials cast doubt he would ever face prosecution over alleged war crimes - citing lack of evidence.


Jonathan battled his way through 63 competitors to reach the tournament's "feature" bracket, coming in 15th place overall, and most importantly, winning an additional $1,000 for the Alberta Children's Hospital.
November 10, 2014: We are proud to announce that Jonathan Griffith, lawyer at DBB, will be participating in Extra Life, a fundraiser for the Children's Miracle Network Hospitals. So far this year, Extra-Life participants have raised over $5.6 million for the charity.
Extra Life is a 24-hour gaming marathon, which will take place November 11-13, 2014. The event will be hosted in Orlando, Florida, and will be attended by top Extra-Lifers from across Canada and the U.S.
Jonathan has been involved with Extra Life for three years raising over $4,000—consistently ranking as a top fundraiser.
Everyone at DBB wishes Jonathan success with this year's fundraising efforts at Extra Life.


Folks who prefer the extra privacy that the second floor of a home provides should consider placing the new master bedroom in a location that offers a nice view. You may also want to think about adding a balcony if you enjoy the thought of fresh air first thing in the morning.
Bedroom additions offer homeowners a diverse range of opportunities for artificial lighting. Naturally, the type of lighting depends on the theme and overall feel you want for your master bedroom. A small crystal chandelier can provide a significant boost to the elegance of your bedroom thanks to its glittery display. Cove lighting, on the other hand, is ideal for those who prefer a romantic and intimate mood. Standard lighting options with dimmer switches are a good choice for those who crave flexibility.
Many modern homes have started to turn extra space in the master bedroom into a "mini living room". As the master bedroom tends to be quiet and relaxing, many people find that having a small couch or a comfortable chair in the room can offer a relaxing spot for activities like reading or managing the monthly budget on your tablet. If this is a feature that interests you, be sure to leave an extra 150 square feet of space for any furnishings you would like to have.
Steve Silverman is Cape Cod native with a BA in psychology from Bates College. He moved to the Pioneer Valley in 2004 and became a VP with VHI a year later, then bought the 20 employee company in 2013. He and his wife and two teenagers have a farm in Southampton where he spends time growing food and learning new ways to live in harmony with the land.


Established by a person that struggles with GERD herself, the folding MedSlant wedge Pillow is an outstanding alternative for any individual that is seeking remedy for this unpleasant problem and also can also assist with apnea, snoring as well as extra. The room is normally restricted, as well as a lot of times you are entrusted to do your work with the flooring, which can be extremely stressing on your back, particularly when finishing limitless hrs of research. Reviewing wedges are the ideal response to this issue. It's tiny sufficient so it does not use up much space, however it still satisfies of offering assistance so your back does not hurt. When utilizing a normal Pillow your head can sink down right into the Pillow. This can create the bronchial asthma patient not to be able to take a breath however by resting on a bed mattress wedge Pillow your head rises that makes breathing a lot easier.
As I had actually pointed out, we currently had a wedge Pillow. Nevertheless, I have actually found that there were might various other sorts of the wedge and also assistance pillows in the marketplace, with a big range of applications, click.
Along with bed pillows, there are various other items planned for usage outside the house. These various other usages consist of the vehicle traveling, plane traveling, as well as the chair at your work. Once more, they are made for the convenience as well as the assistance of the customer. With a percentage of the research study, you must have little difficulty discovering an item to fit your demands and also rate array. Comfy and also personalized, you're bound to discover the analysis Pillow that's right for you. Choose one up for on your own, your partner, your children, or any individual that takes pleasure in analysis and also viewing TELEVISION in bed. You'll never ever need to prop on your own up with normal pillows once again!


text:
\section{Introduction}
Online convex optimization is a powerful paradigm for sequential decision making \citep{Online:suvery}. It can be viewed as a game between a learner and an adversary: In the $t$-th round, the learner selects a decision $\mathbf{w}_t \in \Omega$, simultaneously the adversary chooses a function $f_t(\cdot): \Omega \mapsto \mathbb{R}$, and then the learner suffers an instantaneous loss $f_t(\mathbf{w}_t)$. This study focuses on the full-information setting \citep{bianchi-2006-prediction}, where the function $f_t(\cdot)$ is revealed to the leaner at the end of each round. The goal of the learner is to minimize the cumulative loss over $T$ periods. The standard performance measure is \emph{regret}, which is the difference between the loss incurred by the learner and that of the best fixed decision in hindsight, i.e.,
\[
\Reg(T)=\sum_{t=1}^T f_t(\mathbf{w}_t) - \min_{\mathbf{w} \in \Omega} \sum_{t=1}^T f_t(\mathbf{w}).
\]

The above regret is typically referred to as \emph{static} regret in the sense that the comparator is time-invariant. The rationale behind this evaluation metric is that one of the decision in $\Omega$ is reasonably good over the $T$ rounds. However, when  the underlying distribution of loss functions
changes, the static regret may be too optimistic and fails to capture the hardness of the problem.

To address this limitation, new forms of performance measure, including \emph{adaptive} regret \citep{Adaptive:Hazan,Hazan:2009:ELA} and \emph{dynamic} regret \citep{zinkevich-2003-online}, were proposed and received significant interest  recently.  Given a parameter $\tau$, which is the length of the interval, the strong version of adaptive regret is defined as
\begin{equation} \label{eqn:strong:adaptive}
\begin{split}
\SAReg(T,\tau) = \max_{[s, s+\tau -1] \subseteq [T]} \left(\sum_{t=s}^{s+\tau -1}  f_t(\mathbf{w}_t)  - \min_{\mathbf{w} \in \Omega} \sum_{t=s}^{s+\tau -1} f_t(\mathbf{w}) \right).
\end{split}
\end{equation}
From the definition, we observe that minimizing the adaptive regret enforces the learner has small static regret over any interval of length $\tau$. Since the best decision for different intervals could be different, the learner is essentially competing with a changing comparator.

A parallel line of research introduces the concept of dynamic regret, where the cumulative loss of the learner is compared against a comparator sequence $\mathbf{u}_1, \ldots, \mathbf{u}_T \in \Omega$, i.e.,
\begin{equation} \label{eqn:dynamic:1}
\DReg(\mathbf{u}_1,\ldots,\mathbf{u}_T) = \sum_{t=1}^T f_t(\mathbf{w}_t)  - \sum_{t=1}^T f_t(\mathbf{u}_t).
\end{equation}
It is well-known that in the worst case, a sublinear dynamic regret is impossible unless we impose some regularities on the comparator sequence or the function sequence \citep{Oinline:Dynamic:Comp}. A representative example is the functional variation  defined below
\begin{equation} \label{eqn:func:var}
V_T = \sum_{t=2}^T \max_{\mathbf{w} \in \Omega} |f_t(\mathbf{w}) - f_{t-1}(\mathbf{w})|.
\end{equation}
\citet{Non-Stationary} have  proved that as long as $V_T$ is sublinear in $T$, there exists an algorithm that achieves a sublinear dynamic regret. Furthermore, under the noisy gradient feedback, a general restarting procedure is developed, and it enjoys $O(T^{2/3}V_T^{1/3})$ and $O(\log T \sqrt{T V_T})$ rates for convex functions and strongly convex functions, respectively. This result is very strong in the sense that these rates are (almost) minimax optimal. However, the restarting procedure can only be applied when an upper bound of $V_T$ is known beforehand, thus limiting its application in practice.

While both the adaptive and dynamic regrets aim at coping with changing environments, little is known about their relationship. This paper makes a step towards understanding their connections. Specifically, we show that the strongly adaptive regret in (\ref{eqn:strong:adaptive}), together with the functional variation, can be used to upper bound the dynamic regret in (\ref{eqn:dynamic:1}). Thus, an algorithm with a small strongly adaptive regret is automatically equipped with a tight dynamic regret. As a result, we obtain a series of algorithms for minimizing the dynamic regret that do not need any prior knowledge of the functional variation.  The main contributions of this work are summarized below.
\begin{compactitem}
\item We provide a general theorem that  upper bounds the dynamic regret in terms of the strongly adaptive regret and the functional variation.
\item For convex functions, we show that the strongly adaptive algorithm of \citet{Improved:Strongly:Adaptive} has a dynamic regret of  $O(T^{2/3} V_T^{1/3} \log^{1/3} T)$, which matches the minimax rate,  up to a polylogarithmic factor.
\item For exponentially concave functions, we propose a strongly adaptive algorithm that allows us to control the tradeoff between the adaptive regret and the computational cost explicitly. Furthermore, we demonstrate that its dynamic regret is $O(\sqrt{T V_T \log T})$, and this is the \emph{first} time such kind of dynamic regret bound is established for exponentially concave functions.
\item Since strongly convex functions with bounded gradients are also exponentially concave, our previous result immediately implies a dynamic regret of $O(\sqrt{T V_T \log T})$, which is also minimax optimal up to a polylogarithmic factor. It also indicates our  bound for exponentially concave functions is almost optimal.
\end{compactitem}

\section{Related Work}
In this section, we give a brief introduction to previous work on static, adaptive, and dynamic regrets in the context of online convex optimization.
\subsection{Static Regret}
The majority of studies in online learning are focused on static regret \cite{Shalev:Primal:Dual,Sparse:Online}. For general convex functions, the classical online gradient descent achieves $O(\sqrt{T})$ and $O(\log T)$ regret bounds for convex and strongly convex functions, respectively \citep{zinkevich-2003-online,ML:Hazan:2007,ICML_Pegasos}.  Both the $O(\sqrt{T})$ and $O(\log T)$ rates are known to be minimax optimal~\citep{Minimax:Regret}. When functions are exponentially concave, a different algorithm, named online Newton step,  is developed and enjoys an $O(\log T)$ regret bound \citep{ML:Hazan:2007}.

\subsection{Adaptive Regret}
The concept of adaptive regret is  introduced by \citet{Adaptive:Hazan}, and later strengthened  by \citet{Adaptive:ICML:15}. To distinguish between them, we refer to the definition of \citet{Adaptive:Hazan} as weakly adaptive regret and the one of \citet{Adaptive:ICML:15} as strongly adaptive regret. The weak version is given by
\[
\WAReg(T)=  \max_{[s, q] \subseteq [T]} \sum_{t=s}^{q} f_t(\mathbf{w}_t) - \min_{\mathbf{w} \in \Omega} \sum_{t=s}^{q} f_t(\mathbf{w}).
\]
To minimize the adaptive regret, \citet{Adaptive:Hazan} have developed two meta-algorithms: an efficient algorithm with $O(\log T)$ computational complexity per iteration and an inefficient one with $O(T)$ computational complexity per iteration. These meta-algorithms use an existing online method (that was possibly designed to have small static regret) as a subroutine.\footnote{For brevity, we ignored the factor of subroutine in the statements of computational complexities. The $O(\cdot)$ computational complexity should be interpreted as $O(\cdot) \times s$ space complexity and $O(\cdot) \times t$ time complexity, where $s$ and $t$ are  space  and time complexities of the subroutine per iteration, respectively.} For  convex functions, the efficient and inefficient meta-algorithms have   $O(\sqrt{T \log^3 T})$ and $O(\sqrt{T \log T)}$ regret bounds, respectively. For exponentially concave functions, those rates are improved to  $O(\log^2 T)$ and $O(\log T)$, respectively. We can see that the price paid for the adaptivity is very small: The rates of weakly adaptive regret differ from those of static regret only by logarithmic factors.

A major limitation of weakly adaptive regret is that it does not respect short intervals well. Taking convex functions as an example, the $O(\sqrt{T \log^3 T})$ and $O(\sqrt{T \log T)}$ bounds are meaningless for intervals of length $O(\sqrt{T})$. To overcome this limitation, \citet{Adaptive:ICML:15} proposed a refined adaptive regret that takes the length of the interval as a parameter $\tau$, as indicated in (\ref{eqn:strong:adaptive}). If the strongly adaptive regret is small for all $\tau <T$, we can guarantee the learner has small regret over any interval of any length. In particular, \citet{Adaptive:ICML:15} introduced the following definition.
\begin{definition} \label{def:strongly:adaptive}
Let $R(\tau)$ be the minimax static regret bound of the learning problem over $\tau$ periods. An algorithm is \emph{strongly adaptive}, if
\[
\SAReg(T,\tau)=O(\poly(\log T) \cdot R(\tau)).
\]
\end{definition}
It is easy to verify that the meta-algorithms of \citet{Adaptive:Hazan} are strongly adaptive for exponentially concave functions,\footnote{That is because (i) $\SAReg(T,\tau) \leq \WAReg(T)$, and (ii) there is a $\poly(\log T)$ factor in the definition of strong adaptivity. } but not for convex functions. Thus, \citet{Adaptive:ICML:15} developed a new meta-algorithm that satisfies $\SAReg(T,\tau)=O( \sqrt{\tau} \log T )$ for convex functions, and thus is strongly adaptive. The algorithm is also efficient and the computational complexity per iteration is $O(\log T)$. Later,  the strongly adaptive regret of convex functions was improved to $O( \sqrt{\tau \log T} )$ by \citet{Improved:Strongly:Adaptive}.
\subsection{Dynamic Regret} \label{sec:dynamic}
In a seminal work, \citet{zinkevich-2003-online} proposed to use the \emph{path-length} defined as
\[
\mathcal{P}(\mathbf{u}_1, \ldots, \mathbf{u}_T)=\sum_{t=2}^T \|\mathbf{u}_t - \mathbf{u}_{t-1}\|_2
\]
to upper bound the dynamic regret. Specifically, \citet{zinkevich-2003-online} proved that for any sequence of convex functions, the dynamic regret of online gradient descent can be upper bounded by $O(\sqrt{T} \mathcal{P}(\mathbf{u}_1, \ldots, \mathbf{u}_T))$. Another regularity of the comparator sequence, which is similar to the path-length, is defined as
\[
 \mathcal{P}'(\mathbf{u}_1, \ldots, \mathbf{u}_T)=\sum_{t=2}^T \|\mathbf{u}_t - \Phi_{t} (\mathbf{u}_{t-1})\|_2
\]
where $\Phi_t (\cdot)$ is a dynamic model that predicts a reference point for the $t$-th round.  \citet{Dynamic:ICML:13} developed a novel algorithm named dynamic mirror descent and proved that its dynamic regret is on the order of $\sqrt{T} \mathcal{P}'(\mathbf{u}_1, \ldots, \mathbf{u}_T)$ . The advantage of $\mathcal{P}'(\mathbf{u}_1, \ldots, \mathbf{u}_T)$  is that when the comparator sequence follows the dynamical model closely, it can be much smaller than the path-length $\mathcal{P}(\mathbf{u}_1, \ldots, \mathbf{u}_T)$.

Let $\mathbf{w}_t^* \in \argmin_{\mathbf{w} \in \Omega} f_t(\mathbf{w})$ be a local minimizer of $f_t(\cdot)$. For  any sequence of $\mathbf{u}_1, \ldots, \mathbf{u}_T \in \Omega$, we have
\[
\begin{split}
& \DReg(\mathbf{u}_1,\ldots,\mathbf{u}_T) =\sum_{t=1}^T f_t(\mathbf{w}_t)  - \sum_{t=1}^T f_t(\mathbf{u}_t) \\
\leq & \DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) = \sum_{t=1}^T f_t(\mathbf{w}_t)  - \sum_{t=1}^T \min_{\mathbf{w} \in \Omega} f_t(\mathbf{w}).
\end{split}
\]
Thus, $\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)$ can be treated as the worst case of the dynamic regret, and there are many work that were devoted to minimizing $\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)$ \citep{Oinline:Dynamic:Comp,Dynamic:Strongly,Dynamic:2016,Dynamic:Regret:Squared}.

When a prior knowledge of $\mathcal{P}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*)$ is available, $\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)$ can be upper bounded by $O(\sqrt{T \mathcal{P}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*)})$ \citep{Dynamic:2016}. If all the functions are strongly convex and smooth, the upper bound can be improved to $O(\mathcal{P}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*))$ \citep{Dynamic:Strongly}. The $O(\mathcal{P}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*))$ rate is also achievable when all the functions are convex and smooth, and all the minimizers $\mathbf{w}_t^*$'s lie in the interior of $\Omega$ \citep{Dynamic:2016}. In a recent study, \citet{Dynamic:Regret:Squared} introduced a new regularity---\emph{squared} path-length
\[
\mathcal{S}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*)=\sum_{t=2}^T \|\mathbf{w}_t^* - \mathbf{w}_{t-1}^*\|_2^2
\]
which could be much smaller than the path-length $\mathcal{P}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*)$ when the difference between successive local minimizers is small. \citet{Dynamic:Regret:Squared} developed a novel algorithm named online multiple gradient descent, and proved that $\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)$ is on the order of $\min(\mathcal{P}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*),\mathcal{S}(\mathbf{w}_1^*, \ldots, \mathbf{w}_T^*))$ for (semi-)strongly convex and smooth functions.

Although closely related, adaptive regret and dynamic regret are studied independently and there are few discussions of their relationships. In the literature, dynamic regret is also referred to as tracking regret or shifting regret \citep{LITTLESTONE1994212,Herbster1998,Herbster:2001:TBL}. In the setting of ``prediction with expert advice'', \citet{Adamskiy2012} have shown that the tracking regret can be derived from the adaptive regret.  In the setting of ``online linear optimization in the simplex'', \citet{Fixed:Share:NIPS12} introduced a generalized notion of shifting regret which unifies  adaptive regret and shifting regret. Different from previous work, this paper considers the  setting of online convex optimization, and illustrates that the dynamic regret can be upper bounded by the adaptive regret and the functional variation.
\section{From Adaptive to Dynamic}
In this section, we first introduce a general theorem that bounds the dynamic regret by the adaptive regret, and then derive specific regret bounds  for convex functions, exponentially concave functions, and strongly convex functions.

\subsection{Adaptive-to-Dynamic Conversion}
Let $\mathcal{I}_1=[s_1, q_1], \mathcal{I}_2 = [s_2, q_2], \ldots, \mathcal{I}_k=[s_k, q_k]$ be a partition of $[1,T]$. That is, they are successive intervals such that
\begin{equation} \label{eqn:intervel}
\begin{split}
s_1=1, \ q_i +1 = s_{i+1},  \  i \in [k-1], \textrm{ and }   \ q_k=T.
\end{split}
\end{equation}
Define the local functional variation of the $i$-th interval as
\[
V_T(i) = \sum_{t=s_i+1}^{q_i} \max_{\mathbf{w} \in \Omega} |f_t(\mathbf{w}) - f_{t-1}(\mathbf{w})|
\]
and it is obvious that $\sum_{i=1}^k V_T(i) \leq V_T$.\footnote{Note that in certain cases, the sum of local functional variation $\sum_{i=1}^k V_T(i)$ can be much smaller than the total functional variation $V_T$. For example, when the sequence of functions only changes $k$ times,  we can construct the intervals based on the changing rounds such that $\sum_{i=1}^k V_T(i)=0$.}
Then, we have the following theorem for bounding the dynamic regret in terms of the strongly adaptive regret and the functional variation.
\begin{thm} \label{thm:1} Let $\mathbf{w}_t^* \in \argmin_{\mathbf{w} \in \Omega} f_t(\mathbf{w})$.  We have
\[
\begin{split}
 \DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \leq  \min_{\mathcal{I}_1,\ldots,\mathcal{I}_k} \sum_{i=1}^k \big( \SAReg(T,|\mathcal{I}_i|) + 2 |\mathcal{I}_i| \cdot V_T(i) \big)
\end{split}
\]
where the minimization is taken over any sequence of intervals that satisfy (\ref{eqn:intervel}).
\end{thm}
The above theorem is analogous to Proposition 2 of \citet{Non-Stationary}, which provides an upper bound for a special choice of the interval sequence. The main difference is that there is a minimization operation in our bound, which allows us to get ride of the issue of parameter selection. For a specific type of problems, we can plug in the corresponding upper bound of strongly adaptive regret, and then choose any sequence of intervals to obtain a concrete upper bound. In particular, the choice of the intervals may depend on the (possibly unknown) functional variation.

Before proceeding to specific bounds, we introduce the following common assumption.
\begin{ass}\label{ass:1} Both the gradient and the domain are bounded.
\begin{compactitem}
\item The gradients of all the online functions are bounded by $G$, i.e., $\max_{\mathbf{w} \in \Omega}\|\nabla f_t(\mathbf{w})\| \leq G$ for all $f_t$.
\item The diameter of the domain $\Omega$ is bounded by $B$, i.e., $\max_{\mathbf{w}, \mathbf{w}' \in \Omega}\|\mathbf{w} -\mathbf{w}'\| \leq B$.
\end{compactitem}
\end{ass}
\subsection{Convex Functions}
For convex functions, we choose the meta-algorithm of \citet{Improved:Strongly:Adaptive} and take the online gradient descent as its subroutine. The following theorem regarding the adaptive regret can be obtained from that paper.
\begin{thm} \label{thm:2}  Under Assumption~\ref{ass:1}, the meta-algorithm of \citet{Improved:Strongly:Adaptive} is strongly adaptive with
\[
\begin{split}
\SAReg(T,\tau) \leq \left(\frac{12 BG}{\sqrt{2}-1} +8 \sqrt{7 \log T + 5} \right) \sqrt{\tau}  = O(\sqrt{\tau \log T}  ).
 \end{split}
\]
\end{thm}
From Theorems~\ref{thm:1} and \ref{thm:2}, we derive the following bound for the dynamic regret.
\begin{cor} \label{cor:convex} Under Assumption~\ref{ass:1}, the meta-algorithm of  \citet{Improved:Strongly:Adaptive}  satisfies
\[
\begin{split}
\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)
\leq & \max\left\{\begin{split}
&  (c +9 \sqrt{7 \log T + 5})  \sqrt{T} \\
&  \frac{(c +8 \sqrt{5} )  T^{2/3} V_T^{1/3} }{\log^{1/6} T}    + 24 T^{2/3} V_T^{1/3} \log^{1/3} T
\end{split} \right.\\
= & O \left( \max\left\{  \sqrt{T \log T} , T^{2/3} V_T^{1/3} \log^{1/3} T  \right\} \right)
\end{split}
\]
where $c=12 BG/(\sqrt{2}-1)$.
\end{cor}

According to Theorem 2 of \citet{Non-Stationary}, we know that the minimax dynamic regret of convex functions is $O(T^{2/3} V_T^{1/3})$. Thus, our upper bound is minimax optimal up to a polylogarithmic factor. The key advantage of the meta-algorithm of  \citet{Improved:Strongly:Adaptive} over the restarted online gradient descent of  \citet{Non-Stationary} is that the former one do not need any prior knowledge of the functional variation $V_T$. Notice that the meta-algorithm of  \citet{Adaptive:ICML:15} can also be used here, and its dynamic regret  is on the order of $\max\left\{  \sqrt{T} \log T, T^{2/3} V_T^{1/3} \log^{2/3} T  \right\}$.
\subsection{Exponentially Concave Functions}
We first provide the definition of exponentially concave (abbr.~exp-concave) functions  \citep{bianchi-2006-prediction}.
\begin{definition} A function $f(\cdot): \Omega \mapsto \mathbb{R}$ is $\alpha$-exp-concave if $\exp(-\alpha f(\cdot))$ is concave over domain $\Omega$.
\end{definition}
Exponential concavity is stronger than convexity but weaker than strong convexity. It can be used to model many popular losses used in machine learning, such as the square loss in regression, logistic loss in classification and negative logarithm loss in portfolio management~\citep{Sto:Exp:Con}.

For exp-concave functions, \citet{Adaptive:Hazan} have developed two meta-algorithms that take the online Newton step as its subroutine, and proved the following properties.
\begin{compactitem}
  \item The inefficient one has $O(T)$ computational complexity per iteration, and its weakly adaptive regret is $O(\log T)$.
  \item The efficient one has $O(\log T)$ computational complexity per iteration, and its weakly adaptive regret is $O(\log^2 T)$.
\end{compactitem}
As can be seen, there is a tradeoff between the computational complexity and the weakly adaptive regret: A lighter computation incurs a looser bound and a tighter bound requires a  higher computation.  In Section~\ref{sec:adaptive:alg}, we develop a unified approach, i.e., Algorithm~\ref{alg:1},  that allows us to trade effectiveness for efficiency explicitly.   Lemma~\ref{lem:ending} indicates  the proposed algorithm has
\[
\left(\lfloor \log_K T \rfloor+1 \right) (K-1)=O\left( \frac{K \log T}{\log K} \right)
\]
computational complexity per iteration, where $K$ is a tunable parameter. On the other hand, Theorem~\ref{thm:adaptive} implies that for $\alpha$-exp-concave functions that satisfy  Assumption~\ref{ass:1}, the  strongly adaptive regret of Algorithm~\ref{alg:1} is
 \[
\left(\frac{(5d+1) \bar{m} + 2}{\alpha} + 5d \bar{m} GB \right) \log T = O\left( \frac{\log^2 T}{\log K}\right)
\]
where $d$ is the dimensionality and $\bar{m}=(\lfloor \log_K T \rfloor + 1)$.

We list several choices of $K$ and the resulting theoretical guarantees in Table~\ref{sample-table}, and have the following observations.
\begin{compactitem}
\item When $K=2$, we recover the guarantee of the  efficient algorithm of \citet{Adaptive:Hazan}, and when $K=T$, we obtain the inefficient one.
  \item By setting $K=T^{1/\gamma}$ where $\gamma>1$ is a small constant, such as $10$, the strongly adaptive regret can be viewed as $O(\log T)$, and at the same time,  the computational complexity is also very low for a large range of $T$.
\end{compactitem}

\begin{table}[t]
\caption{Efficiency  and Effectiveness Tradeoff} \label{sample-table}
\vskip 0.15in
\begin{center}
\begin{small}
\begin{sc}
\begin{tabular}{lll}
\hline
$K$  &Complexity  & Adaptive Regret\\
\hline
$2$         & $O(\log T)$ & $O(\log^2 T)$ \\
$T^{1/\gamma}$   & $O(\gamma T^{1/\gamma} )$ & $O(\gamma \log T)$ \\
$T$             & $O(T)$ & $O(\log T)$ \\
\hline
\end{tabular}
\end{sc}
\end{small}
\end{center}
\vskip -0.1in
\end{table}
According to Definition~\ref{def:strongly:adaptive}, Algorithm~\ref{alg:1} in this paper, as well as the two meta-algorithms of \citet{Adaptive:Hazan}, is strongly adaptive. Based on Theorem~\ref{thm:1}, we derive the dynamic regret of the proposed algorithm.
\begin{cor} \label{cor:exp} Let $K=T^{1/\gamma}$, where $\gamma>1$ is a small constant. Suppose Assumption~\ref{ass:1} holds, $\Omega \subset \mathbb{R}^d$, all the functions are $\alpha$-exp-concave. Algorithm~\ref{alg:1}, with online Newton step as its subroutine, is strongly adaptive with
\[
\begin{split}
\SAReg(T,\tau)
\leq \left(\frac{(5d+1) (\gamma+1) + 2}{\alpha} + 5d (\gamma+1) GB \right) \log T
=O\left(\gamma \log T\right)=O\left( \log T\right)
\end{split}
\]
and its dynamic regret satisfies
\[
\begin{split}
 &\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \\
\leq & \left(\frac{(5d+1) (\gamma+1) + 2}{\alpha} + 5d (\gamma+1) GB +2\right)
  \max\left\{  \log T, \sqrt{T V_T \log T}  \right\} \\
= & O \left( \max\left\{  \log T, \sqrt{T V_T \log T}  \right\} \right).
\end{split}
\]
\end{cor}
To the best of our knowledge, this is the first time such kind of dynamic regret bound is established for exp-concave functions. Furthermore, the discussions in Section~\ref{sec:strongly} implies our upper bound is  minimax optimal, up to a polylogarithmic factor.

\subsection{Strongly Convex Functions} \label{sec:strongly}
In the following, we study strongly convex functions, defined below.
\begin{definition} A function $f(\cdot): \Omega \mapsto \mathbb{R}$ is $\lambda$-strongly convex if
\[
f(\mathbf{y}) \geq f(\mathbf{x}) + \left \langle \nabla f(\mathbf{x}), \mathbf{y} -\mathbf{x} \right \rangle + \frac{\lambda}{2} \|\mathbf{y} -\mathbf{x} \|_2^2,  \  \forall \mathbf{x}, \mathbf{y} \in \Omega.
\]
\end{definition}

It is easy to verify that strongly convex functions with bounded gradients are also exp-concave \citep{ML:Hazan:2007}.
\begin{lemma} \label{lem:strongly} Suppose $f(\cdot): \Omega \mapsto \mathbb{R}$ is $\lambda$-strongly convex and $\nabla f(\mathbf{w}) \leq G$ for all $\mathbf{w} \in \Omega$. Then, $f(\cdot)$ is $\frac{\lambda}{G^2}$-exp-concave.
\end{lemma}
Thus, Corollary~\ref{cor:exp} can be directly applied to strongly convex functions, and yields a dynamic regret of $O(\sqrt{T V_T \log T})$. According to Theorem 4 of \citet{Non-Stationary}, the minimax dynamic regret of strongly convex functions is $O(\sqrt{T V_T})$, which implies our upper bound is almost minimax optimal.

A limitation of Corollary~\ref{cor:exp} is that the constant in the upper bound depends on the dimensionality $d$. In the following, we show that when the functions are strongly convex and online gradient descent is used as the subroutine of Algorithm~\ref{alg:1}, both the adaptive and dynamic regrets are independent from $d$.
\begin{cor} \label{cor:strong:convex} Let $K=T^{1/\gamma}$, where $\gamma>1$ is a small constant. Suppose Assumption~\ref{ass:1} holds, and  all the functions are $\lambda$-strongly convex.  Algorithm~\ref{alg:1}, with online gradient descent as its subroutine, is strongly adaptive with
\[
\begin{split}
\SAReg(T,\tau) \leq  \frac{G^2}{2\lambda} \big(\gamma+1+ (3 \gamma+7)  \log T \big )
= O\left(\gamma \log T\right)=O\left( \log T\right)
\end{split}
\]
and its dynamic regret satisfies
\[
\begin{split}
 \DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)
\leq &\max\left\{\begin{split}
&  \frac{\gamma G^2 }{\lambda} + \left(\frac{5 \gamma G^2}{\lambda } +2 \right) \log T \\
&  \frac{\gamma G^2 }{\lambda} \sqrt{\frac{T V_T}{\log T} }+ \left(\frac{5 \gamma G^2}{\lambda } +2 \right)\sqrt{T V_T \log T}
\end{split} \right.\\
= & O \left( \max\left\{  \log T, \sqrt{T V_T \log T}  \right\} \right).
\end{split}
\]
\end{cor}

\section{An Unified Adaptive Algorithm} \label{sec:adaptive:alg}
In this section, we introduce a unified approach for minimizing the adaptive regret of exp-concave functions, as well as strongly convex functions.

Let $E$ be an online learning algorithm that is designed to minimize the static regret of exp-concave functions or strongly convex functions, e.g.,  online Newton step \citep{ML:Hazan:2007} or online gradient descent \citep{zinkevich-2003-online}. Similar to the approach of following the leading history (FLH) \citep{Adaptive:Hazan}, at any time $t$, we will instantiate an expert by applying the online learning algorithm $E$ to the sequence of loss functions $f_t,f_{t+1},\ldots$, and utilize the strategy of learning from expert advice to combine solutions of different experts \citep{Herbster1998}. Our method is named as improved following the leading history (IFLH), and is summarized in Algorithm~\ref{alg:1}.

Let $E^t$ be the expert that starts to work at time $t$. To control the computational complexity, we will associate an ending time $e^t$ for each $E^t$.  The expert $E^t$ is alive during the period $[t, e^t-1]$. In each round $t$, we maintain a working set of experts $\mathcal{S}_t$, which contains all the alive experts, and assign a probability $p_t^j$ for each $E^j \in S_t$. In Steps~6 and 7, we remove all the experts whose ending times are no larger than $t$. Since the number of alive experts has changed, we need to update the probability assigned to them, which is performed in Steps~12 to 14. In Steps~15 and 16, we add a new expert $E^t$ to $\mathcal{S}_t$, calculate its ending time according to Definition~\ref{def:ending} introduced below, and set $p_t^t = \frac{1}{t}$. It is easy to verify $\sum_{j \in \mathcal{S}_t} p_t^j=1$. Let $\mathbf{w}_t^j$ be the output of $E^j$ at the $t$-th round, where $t \geq j$. In Step~17, we submit the weighted average of $\mathbf{w}_t^j$ with coefficient $p_t^j$ as the output $\mathbf{w}_t$, and suffer the loss $f_t(\mathbf{w}_t)$. From Steps~18 to 25, we use the exponential weighting scheme to update the weight for each expert $E^j$ based on its loss $f_t(\mathbf{w}_{t}^j)$. In Step~21, we pass the loss function to all the alive experts such that they can update their predictions for the next round.

\begin{algorithm}[t]
    \caption{Improved Following the Leading History (IFLH)} \label{alg:1}
    \begin{algorithmic}[1]
    \STATE {\bf Input:} An integer $K$
    \STATE Initialize $\mathcal{S}_0 = \emptyset$.
    \FOR{$t = 1, \ldots, T$}
        \STATE Set $Z_t = 0$
        \LCOMMENT{Remove some existing experts}
        \FOR{$E^j \in \mathcal{S}_{t-1}$} \label{step:1}
            \IF{$e^j \leq t$}\label{step:2}
                \STATE Update $\mathcal{S}_{t-1} \leftarrow \mathcal{S}_{t-1} \setminus \{E^j\}$ \label{step:3}
            \ELSE \label{step:4}
                \STATE Set $Z_t = Z_t + \widehat{p}_t^j$ \label{step:5}
            \ENDIF \label{step:6}
        \ENDFOR
       \LCOMMENT{Normalize the probability}
        \FOR{$E^j \in \mathcal{S}_{t-1}$} \label{step:7}
            \STATE Set $p_t^j = \frac{\widehat{p}_t^j}{Z_t}\left(1 - \frac{1}{t}\right)$ \label{step:8}
        \ENDFOR \label{step:9}
        \LCOMMENT{Add a new expert $E^t$}
        \STATE Set $\mathcal{S}_t = \mathcal{S}_{t-1} \cup \{E^t\}$  \label{step:10}
        \STATE  Compute the ending time $e^t=\mathcal{E}_K(t)$ according to Definition~\ref{def:ending} and set $p_t^t = \frac{1}{t}$ \label{step:11}
        \LCOMMENT{Compute the final predicted model}
        \STATE Submit the solution
        \[
        \mathbf{w}_t = \sum_{E^j  \in \mathcal{S}_t} p_t^j \mathbf{w}_t^j
        \]
         and suffer loss $f_t(\mathbf{w}_t)$  \label{step:12}
        \LCOMMENT{Update weights and expert}
        \STATE Set $Z_{t+1} = 0$ \label{step:13}
        \FOR{$E^j \in \mathcal{S}_t$}
            \STATE Compute $p_{t+1}^j = p_t^j \exp(-\alpha f_t(\mathbf{w}_{t}^j))$ and $Z_{t+1} = Z_{t+1} + p_{t+1}^j$
            \STATE Pass the function $f_t(\cdot)$ to $E^j$ \label{step:14}
        \ENDFOR
        \FOR{$E^j \in \mathcal{S}_t$}
            \STATE Set $\widehat{p}_{t+1}^j = \frac{\widehat{p}_{t+1}^j}{Z_{t+1}}$
        \ENDFOR \label{step:15}
    \ENDFOR
    \end{algorithmic}
\end{algorithm}
The difference between our IFLH and the original FLH is how to decide the ending time $e^t$ of  expert $E^t$. In this paper, we propose the following base-$K$ ending time.
\begin{definition}[Base-$K$ Ending Time] \label{def:ending} Let $K$ be an integer, and the representation of $t$ in the base-$K$ number system as
\[
t= \sum_{\tau \geq 0} \alpha_\tau K^\tau
\]
where $0 \leq \alpha_\tau <K$, for all $\tau \geq 0$. Let $k$ be the smallest integer such that $\alpha_\tau > 0$, i.e.,
\[
k = \min\{\tau:\alpha_\tau > 0\}.
\]
Then, the base-$K$ ending time of $t$ is defined as
\[
\mathcal{E}_K(t)=\sum_{\tau \geq k+1} \alpha_\tau K^\tau + K^{k+1}.
\]
In other words, the ending time is the number represented by the new sequence obtained by setting the first nonzero elements in the sequence $\alpha_0,\alpha_1,\ldots$ to be $0$ and adding $1$ to the element after it.
\end{definition}
 Let's take the decimal system as an example (i.e., $K=10$). Then,
 \[
 \begin{split}
&E_{10}(1)=E_{10}(2)=\cdots =E_{10}(9)=10,\\
&E_{10}(11)=E_{10}(12)=\cdots=E_{10}(19)=20,\\
&E_{10}(10)=E_{10}(20)=\cdots=E_{10}(90)=100.\\
\end{split}
\]
We note that a similar strategy for deciding the ending time was proposed by \citet{Track_Large_Expert}, and a discussion about the difference is given in the supplementary.

When the base-$K$ ending time is used in Algorithm~\ref{alg:1}, we have the following properties.
\begin{lemma} \label{lem:ending}
Suppose we use the base-$K$ ending time in Algorithm~\ref{alg:1}.
\begin{compactenum}
\item For any $t \geq 1$, we have
\[
|\mathcal{S}_t| \leq \left(\lfloor \log_K t \rfloor+1 \right) (K-1)=O\left( \frac{K \log t}{\log K} \right).
\]
\item  For any interval $I = [r, s] \subseteq [T]$, we can always find $m$ segments
\[
I_j = [t_j, e^{t_j}-1], \ j \in [m]
\]
with $m \leq \lfloor \log_K s \rfloor + 1$, such that
\[
 t_1=r, \ e^{t_j}=t_{j+1},  \ j \in [m-1], \textrm{ and } e^{t_m} > s.
\]
\end{compactenum}
\end{lemma}
The first part of Lemma~\ref{lem:ending} implies that the size of $\mathcal{S}_t$ is $O(K \log t/\log K)$. An example of $\mathcal{S}_t$ in the decimal system is given below.
 \[
 \mathcal{S}_{486}=\left\{ \begin{split}
 &481, \ 482, \ \ldots, \ 486, \\
   &410, \ 420, \ \ldots, \ 480,\\
   &100, \ 200, \ \ldots, \ 400
 \end{split} \right\} .
 \]
The second part of Lemma~\ref{lem:ending} implies that for any interval $I=[r, s]$, we can find $O(\log s/\log K)$ experts such that their survival periods cover $I$. Again, we present an example in the decimal system: The interval $[111, 832]$ can be covered by
\[
[111, 119], \ [120, 199], \textrm{ and } [200, 999]
\]
which are the survival periods of experts $E^{111}$, $E^{120}$, and $E^{200}$,  respectively. Recall that $E_{10}(111)=120$, $E_{10}(120)=200$, and $E_{10}(200)=1000$.

Based on Lemma~\ref{lem:ending}, we have the following theorem regarding the adaptive regret of exp-concave functions.
\begin{thm} \label{thm:adaptive} Suppose Assumption~\ref{ass:1} holds, $\Omega \subset \mathbb{R}^d$, all the functions are $\alpha$-exp-concave. If  online Newton step is used as the subroutine in Algorithm~\ref{alg:1}, we have
\[
\begin{split}
 \sum_{t=r}^s f_t(\mathbf{w}_t) - \min\limits_{\mathbf{w} \in \Omega} \sum_{t=r}^s f_t(\mathbf{w})
\leq  \left(\frac{(5d+1) m + 2}{\alpha} + 5d mGB \right) \log T
\end{split}
\]
where $m \leq \lfloor \log_K s \rfloor + 1$.  And thus,
\[
\begin{split}
\SAReg(T,\tau)
\leq \left(\frac{(5d+1) \bar{m} + 2}{\alpha} + 5d \bar{m} GB \right) \log T = O\left( \frac{\log^2 T}{\log K}\right)
 \end{split}
\]
where $\bar{m}=(\lfloor \log_K T \rfloor + 1)$.
\end{thm}
From Lemma~\ref{lem:ending} and Theorem~\ref{thm:adaptive}, we observe that the adaptive regret is a decreasing function of $K$, while the computational cost is an increasing function of $K$. Thus, we can control the tradeoff by tuning the value of $K$.

For strongly convex functions, we have a similar guarantee but without any dependence on the dimensionality $d$, as indicated below.
\begin{thm} \label{thm:strong:convex} Suppose Assumption~\ref{ass:1} holds, and  all the functions are $\lambda$-strongly convex. If  online gradient descent is used as the subroutine in Algorithm~\ref{alg:1}, we have
\[
\begin{split}
 \sum_{t=r}^s f_t(\mathbf{w}_t) - \min\limits_{\mathbf{w} \in \Omega} \sum_{t=r}^s f_t(\mathbf{w})
 \leq \frac{G^2}{2\lambda} \big(m+ (3 m +4)  \log T\big )
\end{split}
\]
where $m \leq \lfloor \log_K s \rfloor + 1$. And thus
\[
\begin{split}
\SAReg(T,\tau)
\leq \frac{G^2}{2\lambda} \big(\bar{m}+ (3\bar{m} +4)  \log T \big)= O\left( \frac{\log^2 T}{\log K}\right)
 \end{split}
\]
where $\bar{m}=(\lfloor \log_K T \rfloor + 1)$.
\end{thm}
\section{Analysis}
We here present the proofs of main theorems. The omitted proofs are provided in the supplementary.

\subsection{Proof of Theorem~\ref{thm:1}}
First, we upper bound the dynamic regret in the following way
\begin{equation} \label{eqn:thm1:1}
\begin{split}
&\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \\
= &  \sum_{i=1}^k \left(\sum_{t=s_i}^{q_i} f_t(\mathbf{w}_t)  - \sum_{t=s_i}^{q_i} \min_{\mathbf{w} \in \Omega} f_t(\mathbf{w}) \right)\\
= & \sum_{i=1}^k \left( \underbrace{\sum_{t=s_i}^{q_i} f_t(\mathbf{w}_t) - \min_{\mathbf{w} \in \Omega} \sum_{t=s_i}^{q_i} f_t(\mathbf{w})}_{:=a_i} +\underbrace{\min_{\mathbf{w} \in \Omega} \sum_{t=s_i}^{q_i} f_t(\mathbf{w})- \sum_{t=s_i}^{q_i} \min_{\mathbf{w} \in \Omega} f_t(\mathbf{w})}_{:=b_i} \right).
\end{split}
\end{equation}

From the definition of strongly adaptive regret, we can upper bound $a_i$ by
\[
\sum_{t=s_i}^{q_i} f_t(\mathbf{w}_t) - \min_{\mathbf{w} \in \Omega} \sum_{t=s_i}^{q_i} f_t(\mathbf{w}) \leq \SAReg(T,|\mathcal{I}_i|).
\]
To upper bound $b_i$, we follow the analysis of Proposition 2 of \citet{Non-Stationary}.
\begin{equation} \label{eqn:thm1:bt:1}
\begin{split}
&\min_{\mathbf{w} \in \Omega} \sum_{t=s_i}^{q_i} f_t(\mathbf{w})- \sum_{t=s_i}^{q_i} \min_{\mathbf{w} \in \Omega} f_t(\mathbf{w})
= \min_{\mathbf{w} \in \Omega} \sum_{t=s_i}^{q_i} f_t(\mathbf{w})- \sum_{t=s_i}^{q_i}  f_t(\mathbf{w}_t^*) \\
\leq & \sum_{t=s_i}^{q_i} f_t(\mathbf{w}_{s_i}^*)- \sum_{t=s_i}^{q_i}  f_t(\mathbf{w}_t^*)
 \leq  |\mathcal{I}_i| \cdot \max_{t \in [s_i,q_i]} \left( f_t(\mathbf{w}_{s_i}^*)- f_t(\mathbf{w}_t^*) \right).
\end{split}
\end{equation}
Furthermore, for any $t \in [s_i,q_i]$, we have
\begin{equation} \label{eqn:thm1:bt:2}
\begin{split}
 &f_t(\mathbf{w}_{s_i}^*)- f_t(\mathbf{w}_t^*)
  =  f_t(\mathbf{w}_{s_i}^*)- f_{s_i}(\mathbf{w}_{s_i}^*) + f_{s_i}(\mathbf{w}_{s_i}^*)  -  f_t(\mathbf{w}_t^*) \\
\leq & f_t(\mathbf{w}_{s_i}^*)- f_{s_i}(\mathbf{w}_{s_i}^*) + f_{s_i}(\mathbf{w}_t^*)  -  f_t(\mathbf{w}_t^*)
 \leq 2 V_T(i).
\end{split}
\end{equation}
Combining (\ref{eqn:thm1:bt:1}) with (\ref{eqn:thm1:bt:2}), we have
\[
\min_{\mathbf{w} \in \Omega} \sum_{t=s_i}^{q_i} f_t(\mathbf{w})- \sum_{t=s_i}^{q_i} \min_{\mathbf{w} \in \Omega} f_t(\mathbf{w})  \leq 2  |\mathcal{I}_i| \cdot V_T(i).
\]

Substituting the upper bounds of $a_i$ and $b_i$ into (\ref{eqn:thm1:1}), we arrive at
\[
\begin{split}
\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)
\leq   \sum_{i=1}^k \left(  \SAReg(T,|\mathcal{I}_i|)+ 2  |\mathcal{I}_i| \cdot V_T(i) \right).
\end{split}
\]
Since the above inequality holds for any partition of $[1,T]$, we can take minimization to get a tight bound.
\subsection{Proof of Corollary~\ref{cor:convex}}
To simplify the upper bound in Theorem~\ref{thm:1}, we restrict to intervals of the same length $\tau$, and in this case $k=T/\tau$. Then, we have
\[
\begin{split}
 & \DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \\
\leq &  \min_{1 \leq \tau \leq T} \sum_{i=1}^k \big( \SAReg(T,\tau) + 2 \tau  V_T(i) \big)
=  \min_{1 \leq \tau \leq T}  \left(  \frac{ \SAReg(T,\tau) T}{\tau} + 2 \tau  \sum_{i=1}^k V_T(i) \right)\\
\leq & \min_{1 \leq \tau \leq T}  \left( \frac{ \SAReg(T,\tau) T}{\tau} + 2 \tau V_T \right).
\end{split}
\]
Combining with Theorem~\ref{thm:2}, we have
\[
\begin{split}
 \DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)
\leq  \min_{1 \leq \tau \leq T}  \left( \frac{(c +8 \sqrt{7 \log T + 5})  T }{\sqrt{\tau}} + 2 \tau V_T \right).
\end{split}
\]
where $c=12 BG/(\sqrt{2}-1)$.

In the following, we consider two cases. If $V_T \geq \sqrt{\log T/T}$, we choose
\[
\tau = \left( \frac{T \sqrt{\log T}}{V_T} \right)^{2/3} \leq T
\]
and have
\[
\begin{split}
 &\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \\
 \leq  &  \frac{(c +8 \sqrt{7 \log T + 5} )  T^{2/3} V_T^{1/3} }{\log^{1/6} T}    + 2 T^{2/3} V_T^{1/3} \log^{1/3} T\\
 \leq & \frac{(c +8 \sqrt{5} )  T^{2/3} V_T^{1/3} }{\log^{1/6} T}    + (2+ 8\sqrt{7}) T^{2/3} V_T^{1/3} \log^{1/3} T.
\end{split}
\]
Otherwise, we choose  $\tau=T$, and have
\[
\begin{split}
 &\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \\
 \leq &(c +8 \sqrt{7 \log T + 5})  \sqrt{T}  + 2 T V_T \\
  \leq & (c +8 \sqrt{7 \log T + 5})  \sqrt{T}  + 2 T \sqrt{\frac{\log T}{T}} \\
\leq & (c +9 \sqrt{7 \log T + 5})  \sqrt{T}.
 \end{split}
\]
\section{Proof of Theorem~\ref{thm:adaptive}}
From the second part of Lemma~\ref{lem:ending}, we know that there exist $m$ segments
\[
I_j = [t_j, e^{t_j}-1], \ j \in [m]
\]
with $m \leq \lfloor \log_K s \rfloor + 1$, such that
\[
 t_1=r, \ e^{t_j}=t_{j+1}, \  j \in [m-1], \textrm{ and } e^{t_m} > s.
\]
Furthermore, the expert $E^{t_j}$ is alive during the period $[t_j, e^{t_j}-1]$.

Using Claim 3.1 of \citet{Hazan:2009:ELA}, we have
\[
\begin{split}
\sum_{t = t_j}^{e^{t_j}-1} f_t(\mathbf{w}_t) - f_t(\mathbf{w}^{t_j}_t) \leq  \frac{1}{\alpha}\left(\log t_j + 2\sum_{t = t_j + 1}^{e^{t_j}-1}\frac{1}{t}\right), \ \forall j \in [m-1]
\end{split}
\]
where $\mathbf{w}^{t_j}_{t_j}, \ldots, \mathbf{w}^{t_j}_{e^{t_j}-1}$ is the sequence of solutions generated by the expert $E^{t_j}$. Similarly, for the last segment, we have
\[
\begin{split}
\sum_{t = t_{m}}^{s} f_t(\mathbf{w}_t) - f_t(\mathbf{w}^{t_{m}}_t) \leq \frac{1}{\alpha}\left(\log  t_{m} + 2\sum_{t =  t_{m} + 1}^{s}\frac{1}{t}\right).
\end{split}
\]

By adding things together, we have
\begin{equation} \label{eqn:bound-1}
\begin{split}
& \sum_{j=1}^{m-1}   \left(\sum_{t = t_j}^{e^{t_j}-1} f_t(\mathbf{w}_t) - f_t(\mathbf{w}^{t_j}_t) \right)  + \sum_{t = t_{m}}^{s} f_t(\mathbf{w}_t) - f_t(\mathbf{w}^{t_{m}}_t)  \\
 \leq & \frac{1}{\alpha}\sum_{j=1}^m \log t_j  + \frac{2}{\alpha} \sum_{t=r+1}^s \frac{1}{t}  \leq  \frac{m + 2}{\alpha} \log T .
 \end{split}
\end{equation}
According to the property of online Newton step \citep[Theorem 2]{ML:Hazan:2007}, we have, for any $\mathbf{w} \in \Omega$,
\begin{equation}  \label{eqn:bound-2}
\sum_{t = t_j}^{e^{t_j}-1} f_t(\mathbf{w}^{t_j}_t) - f_t(\mathbf{w}) \leq 5d \left(\frac{1}{\alpha} +GB \right)\log T, \ \forall j \in [m-1]
\end{equation}
and
\begin{equation}  \label{eqn:bound-3}
\sum_{t = t_{m}}^{s}  f_t(\mathbf{w}^{t_m}_t) - f_t(\mathbf{w}) \leq 5d \left(\frac{1}{\alpha} +GB \right)\log T.
\end{equation}

Combining (\ref{eqn:bound-1}), (\ref{eqn:bound-2}), and (\ref{eqn:bound-3}), we have,
\[
\begin{split}
 \sum_{t=r}^s f_t(\mathbf{w}_t) -  \sum_{t=r}^s f_t(\mathbf{w}) \leq \left(\frac{(5d+1) m + 2}{\alpha} + 5d mGB \right) \log T
\end{split}
\]
for any $\mathbf{w} \in \Omega$.

\section{Proof of Corollary~\ref{cor:exp}}
The first part of Corollary~\ref{cor:exp} is a direct consequence of Theorem~\ref{thm:adaptive} by setting $K=T^{1/\gamma}$.

Now, we prove the second part. Following similar analysis of Corollary~\ref{cor:convex}, we have
\[
\begin{split}
 \DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \leq  \min_{1 \leq \tau \leq T}  \left\{ \left(\frac{(5d+1) (\gamma+1) + 2}{\alpha} + 5d (\gamma+1) GB \right) \frac{ T \log T }{\tau} + 2 \tau V_T \right\}.
\end{split}
\]
Then, we consider two cases. If $V_T \geq \log T/T$, we choose
\[
\tau = \sqrt{ \frac{T \log T }{V_T}} \leq T
\]
and have
\[
\begin{split}
\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)  \leq   \left(\frac{(5d+1) (\gamma+1) + 2}{\alpha} + 5d (\gamma+1) GB +2\right)\sqrt{T V_T \log T} .
\end{split}
\]
Otherwise, we choose  $\tau=T$, and have
\[
\begin{split}
\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \leq & \left(\frac{(5d+1) (\gamma+1) + 2}{\alpha} + 5d (\gamma+1) GB \right)  \log T + 2 T V_T \\
\leq &\left(\frac{(5d+1) (\gamma+1) + 2}{\alpha} + 5d (\gamma+1) GB \right)  \log T + 2 T \frac{\log T}{T}\\
= &\left(\frac{(5d+1) (\gamma+1) + 2}{\alpha} + 5d (\gamma+1) GB +2\right) \log T .
\end{split}
\]

\section{Proof of Theorem~\ref{thm:strong:convex}}
Lemma~\ref{lem:strongly} implies that all the $\lambda$-strongly convex functions are also $\frac{\lambda}{G^2}$-exp-concave. As a result, we can reuse the proof of Theorem~\ref{thm:adaptive}. Specifically, (\ref{eqn:bound-1}) with $\alpha=\frac{\lambda}{G^2}$ becomes
\begin{equation} \label{eqn:strong:convex:1}
\begin{split}
\sum_{j=1}^{m-1}   \left(\sum_{t = t_j}^{e^{t_j}-1} f_t(\mathbf{w}_t) - f_t(\mathbf{w}^{t_j}_t) \right)  + \sum_{t = t_{m}}^{s} f_t(\mathbf{w}_t) - f_t(\mathbf{w}^{t_{m}}_t)    \leq  \frac{(m + 2)G^2}{\lambda} \log T .
 \end{split}
\end{equation}
According to the property of online gradient descent \citep[Theorem 1]{ML:Hazan:2007}, we have, for any $\mathbf{w} \in \Omega$,
\begin{equation}  \label{eqn:strong:convex:2}
\sum_{t = t_j}^{e^{t_j}-1} f_t(\mathbf{w}^{t_j}_t) - f_t(\mathbf{w}) \leq \frac{G^2}{2\lambda} (1+\log T), \ \forall j \in [m-1]
\end{equation}
and
\begin{equation}  \label{eqn:strong:convex:3}
\sum_{t = t_{m}}^{s}  f_t(\mathbf{w}^{t_m}_t) - f_t(\mathbf{w}) \leq \frac{G^2}{2\lambda} (1+\log T).
\end{equation}
Combining (\ref{eqn:strong:convex:1}), (\ref{eqn:strong:convex:2}), and (\ref{eqn:strong:convex:3}), we have,
\[
\begin{split}
 \sum_{t=r}^s f_t(\mathbf{w}_t) -  \sum_{t=r}^s f_t(\mathbf{w}) \leq  \frac{G^2}{2\lambda} \big(m+ (3 m +4)  \log T \big)
\end{split}
\]
for any $\mathbf{w} \in \Omega$.
\section{Proof of Corollary~\ref{cor:strong:convex}}
The first part of Corollary~\ref{cor:strong:convex} is a direct consequence of Theorem~\ref{thm:strong:convex} by setting $K=T^{1/\gamma}$.

The proof of the second part is similar to that of Corollary~\ref{cor:exp}. First, we have
\[
\begin{split}
 \DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \leq & \min_{1 \leq \tau \leq T}  \left\{ \frac{G^2}{2\lambda} \big(\gamma+1+ (3 \gamma+7)  \log T \big) \frac{T}{\tau} + 2 \tau V_T \right\}\\
 \leq &  \min_{1 \leq \tau \leq T}  \left\{  \frac{( \gamma+ 5 \gamma  \log T )G^2 T}{\lambda \tau} + 2 \tau V_T \right\}
\end{split}
\]
where the last inequality is due to the condition $\gamma >1$.

Then, we consider two cases. If $V_T \geq \log T/T$, we choose
\[
\tau = \sqrt{ \frac{T \log T }{V_T}} \leq T
\]
and have
\[
\begin{split}
\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*)  \leq & \frac{\gamma G^2 }{\lambda} \sqrt{\frac{T V_T}{\log T} }+ \frac{5 \gamma G^2}{\lambda } \sqrt{T V_T \log T} +  2 \sqrt{T V_T \log T} \\
=&\frac{\gamma G^2 }{\lambda} \sqrt{\frac{T V_T}{\log T} }+ \left(\frac{5 \gamma G^2}{\lambda } +2 \right)\sqrt{T V_T \log T}  .
\end{split}
\]
Otherwise, we choose  $\tau=T$, and have
\[
\begin{split}
\DReg(\mathbf{w}_1^*,\ldots,\mathbf{w}_T^*) \leq & \frac{( \gamma+ 5 \gamma  \log T )G^2}{\lambda } + 2 T V_T \\
\leq  &  \frac{( \gamma+ 5 \gamma  \log T )G^2}{\lambda } +2 T \frac{\log T}{T} \\
=& \frac{\gamma G^2 }{\lambda} + \left(\frac{5 \gamma G^2}{\lambda } +2 \right) \log T.
\end{split}
\]
\section{Conclusions and Future Work}
In this paper, we demonstrate that the dynamic regret can be upper bounded by the adaptive regret and the functional variation, which implies strongly adaptive algorithms are automatically equipped with tight dynamic regret bounds. As a result, we are able to derive dynamic regret bounds for convex functions, exponentially concave functions, and strongly convex functions. All of these upper bounds are almost minimax optimal and this is the first time that such kind of dynamic regret bound is proved for exponentially concave functions.

The adaptive-to-dynamic conversion  leads to a series of dynamic regret bounds in terms of the functional variation.  As we mentioned in Section~\ref{sec:dynamic}, dynamic regret can also be upper bounded by other regularities such as the path-length. It is interesting to investigate whether those kinds of upper bounds can also be established for strongly adaptive algorithms. Since we derive  dynamic regret from  adaptive regret, we conjecture that adaptive regret is more fundamental, and will try to give a rigorous proof in the future.

redpajama_set_name: RedPajamaArXiv


I've added the fundraising plugin and have created a sidebar with the fundraiser but for some reason the align (float) is off.
You can view it at http://www.reggiecampbell.com. I've noticed that the float is set to left but if I change it to none within firebug it fixes the issue. I've used the below code to add to the CSS but it does not work. Can you tell what I'm doing wrong?
By the way, for some reason there is a $0 option at the bottom and I can't seem to see why that is there. I don't have a blank option of $0 in the Fundraiser.
I couldn't see anything wrong on that page.
Could you provide a screenshot illustrating the problem or describe it in more detail?
Thanks for your patience with this support ticket. Okay, I just created a quick Screenr for you to review to see my issue.
Let me know if you have additional questions, OR if you think you'll need access to my WordPress account.
I noticed you put the float CSS as an inline style. Inline styles are bad for your load time and site's scalability.
That "none" fundraiser is there automatically. Add the following to your CSS to hide it.
Perfect! That worked! I guess the only thing I was missing was the !important item. I didn't know how to do that. As for the inline style you mentioned - bummer! I purchased that theme from Themeforest.net so they must have built it that way.
Case closed! :slight_smile: Thanks for your time!
I'm glad that's working for ya.


text:
MUSIC THERAPY: Picnic in the Park (Los Angeles)
Thursday, December 3, 2020 @ 1:30PM Thu, Dec 3, 2020 @ 1:30PM
Address provided upon booking, Hollywood
Join us for a relaxing afternoon in the park, where we will share stories, songs and pull back the curtain on the songwriting process. Bring your own blanket, picnic, and hangout with us in the park. You will hear 4 voices from different genres, including grammy award winning artists, singing in harmony and sharing their experiences that led to where they are today. You will then be invited to participate in the creation of a brand new song!! We hope inspire you to explore your own, inner childs' creativity!
In order to keep everyone safe, we are staying socially distanced and limiting the number of participants. The entire event will be outdoors.
If you can't make this one, we can do a private group event for you.
Please reach out at: booking@flightofvoices.com
Flight of Voices @ Malibu Wines Hike + Outdoor Concert
Saturday, November 13, 2021 Sat, Nov 13, 2021
Malibu Wine Hikes
Join Flight of Voices for an unforgettable concert experience, featuring incredible Grammy Award winning artists and musicians at a private, Malibu vineyard. You will be led by professional hiking guides through Saddlerock Ranch, so prepare to be immersed in the outstanding natural beauty of the 1100 acre property. The hike is a 1.5 mile loop and will culminate in an intimate hour acoustic performance by Flight of Voices. It's the best way to unplug for an entire afternoon with the perfect combination of mesmerizing nature and outstanding live music!
You will get to sample a "flight" of different voices from different genres, as you sample a "flight" of delicious wines! Guests will receive a bottle of wine to go at the end of the tour. We recommend booking in groups of 4 to experience the full wine flight.
If you don't feel like hiking, don't worry, we have a "Concert Only" ticket option!
Saturday, October 23, 2021 @ 3:30PM Sat, Oct 23, 2021 @ 3:30PM
Malibu Wines, 32111 Mulholland Highway, Malibu Wines, CA
If you don't feel like hiking and would just like to attend the concert, we have an option for that too!
Saturday, September 25, 2021 @ 4:30PM Sat, Sep 25, 2021 @ 4:30PM
Saturday, August 21, 2021 @ 3:30PM Sat, Aug 21, 2021 @ 3:30PM
If you don't feel like hiking and would just like to attend the concert there is an option for that too!
Saturday, July 31, 2021 @ 3:30PM Sat, Jul 31, 2021 @ 3:30PM
If you don't feel like hiking and would just like to attend the concert, we have limited blankets available to accommodate up to 4 people. This includes the 1.5 hour intimate concert experience, reserved blanket seating and a wine flight with 4 different wines paired with 4 different voices!
Flight of Voices: 5 Senses Experience for Patch Coffee's 5th Anniversary
Patch Coffee Co , Lake Forest
Flight of Voices is back in OC for a very special evening of magical music, craft cocktails and connection!
Join us for an unforgettable concert experience, featuring incredible Grammy Award winning artists and musicians to celebrate Patch Coffee's 5th Anniversary.
You will get to sample a "flight" of different voices from different genres, as you enjoy a "flight" of delicious, handcrafted cocktails.
Tickets include the concert, the cocktail flight and seating.
Saturday, June 26, 2021 @ 3:30PM Sat, Jun 26, 2021 @ 3:30PM
Flight of Voices @ Malibu Wines: Hike + Outdoor Concert
Thursday, April 8, 2021 @ 3:30PM Thu, Apr 8, 2021 @ 3:30PM
Malibu Wines
Join Flight of Voices for a one of an unforgettable afternoon through one of California's most beautiful properties. You will be led by professional hiking guides through Saddlerock Ranch. Prepare to be immersed in the outstanding natural beauty of the 1100 acre property. The hike is a 1.5 mile loop and will culminate in an intimate hour acoustic performance by Flight of Voices. Like all Flight of Voices concerts it will be a magical, and unforgettable experience. Guests can bring their own picnic and will receive a bottle of wine to go at the end of the tour!
Flight of Voices Hike and Acoustic Concert @ Malibu Wines
Sunday, November 29, 2020 Sun, Nov 29, 2020
Malibu Wine Hikes , Malibu
Join Flight of Voices on Thanksgiving weekend for a one of a kind musical journey through one of California's most beautiful properties. You will be led by professional hiking guides through Saddlerock Ranch. Prepare to be immersed in the outstanding natural beauty of the 1100 acre property. The hike is a 1.5 mile loop and will culminate in an intimate hour acoustic performance by Flight of Voices. Like all Flight of Voices concerts it will be a magical, and unforgettable experience. Visit www.flightofvoices.com to find about more about them. Guests will receive a bottle of wine to go at the end of the tour.
Virtual Mother's Day Concert Experience
Sunday, May 10, 2020 @ 12:00PM Sun, May 10, 2020 @ 12:00PM
Online On Zoom
This year we had scheduled a special Mother's Day Concert in person at Patch Coffee Co on May 10th, as we are not able to meet in person at this time, we decided we would come together virtually and celebrate all the amazing Mother's around the world through music, community and a flight of cocktails of course!
In this concert, like a flight of wine, where you sample different wines, you will sample 5 incredible voices from different genres. You will be taken on an emotional journey into the hearts and minds of the diverse and highly curated artists, and you will even have an opportunity to participate in creating a song with us...don't worry you won't be asked to sing!
We have limited tickets available. If you are not in a position to pay for a ticket at this time but need some music, laughter and joy in your life please reach out to us, as we have set aside a portion of complimentary tickets and will be issuing them on a first come first served basis.
Alternatively if you are able to support and would like to donate a ticket to someone you can purchase additional tickets as add ons and we will gift them to people in need!
Please note you will be sent out the Zoom link and password once you have purchased your ticket.
Thank you so much for your support and we can't wait to see you on Sunday May 10th 2020!
Flight of Voices First Ever Virtual Concert Experience
As we currently cannot take our wonderful audiences to mountain tops and secret caves to experience our music we have come up with a virtual, online experience which we are very excited to share with you all. So cosy up with us this Saturday April 25th at 6pm PST from the comfort of your home for an intimate, acoustic performance by artists from the Flight of Voices collective on the platform Zoom.
Intimate Concert @ World Famous Village Studios
Saturday, March 21, 2020 @ 8:00PM Sat, Mar 21, 2020 @ 8:00PM
Village Studios, 1616 Butler Ave, Los Angeles, CA
An intimate performance by artists from the Flight of Voices collective, beneath the chandeliers of the Moroccan Ballroom, in the world famous Village Studios. You will be joined by LA locals and fellow travellers to discover some of the best singers, musicians, & multi-instrumentalists in the city.
Every show is different, and totally unique. Like a flight of wine, where you sample different wines, you will sample four incredible voices from different genres, singing, harmonising and playing on each others' songs. We have accompanying musicians supporting the featured solo artists, to create a one-off musical experience. You will be taken on an emotional journey into the hearts and minds of the diverse and highly curated artists, for an unforgettable evening, unlike anything else LA has to offer.
Flight of Voices @ Patch Coffee Co. (Orange County)
Patch Coffee, 29100 Portola Parkway, Lake Forest
An intimate performance by artists from the Flight of Voices collective, at the community driven Patch Coffee Co. You will be joined by OC locals and fellow travellers to discover some of the best singers, musicians, & multi-instrumentalists in the city.
Village Studios , Village Studios, 1616 Butler Ave, Los Angeles, CA
Flight of Voices is a community of incredible solo artists that come together to transform the way both musicians and audiences experience music.


Al Mamlaka (the Kingdom), a news-based TV station from Jordan, will start transmissions on July 16th. Its website will also start on the same day.
The channel is based at the King Hussein Business Park, in Amman, and is operating under a royal decree issued two years ago. The Jordanian government has guaranteed two years of funding for the news channel (Jordanian Dinars 10 million).
The launch means that a proposed third channels from the country's public broadcaster, Jordan Radio & TV Corp (JRTVC) will not now go ahead. Al Mamlaka will be wholly independent.
A statement from Al Mamlaka said Jordanians are expected to witness an "advanced and qualitative leap with more freedom and professionalism", in comparison with other Jordanian broadcasters.


text:
Нашёл 7 млн ответов for 'dota'.
Dota Plus Grandmaster Tier Dota Plus members can now level up heroes to a brand-new Dota Plus members can now purchase a new treasure with shards. This treasure features 12 community-created...
https://en.dota2.com/
Dota Plus Grandmaster Tier Dota Plus members can now level up heroes to a brand-new Grandmaster tier at level 30. In addition to the inherent intimidation factor, players who earn the title of Grandmaster...
https://blog.dota2.com/
Dotabuff is the leading statistics and community website for Dota 2
North America Dota Pro Circuit Upper Division power ranking. The SEA of potential. The new 7.28 patch has brought a new layer of complexity and experimentation to Dota 2 in the form of Aghanim's...
https://www.dotabuff.com/
iCCup — DotA Section
International Cyber Cup is the best place for DotA, DotA2, StarCraft BroodWar and other e-sports gamers. DotA - real classic for real gamers. Only 500 MB HDD. 256 MB RAM.
https://iccup.com/en/dota/
Dota 2 - Home | Facebook
Dota 2. 4,733,726 likes · 2,497 talking about this. Dota is a competitive game of action and strategy, played both professionally and casually by... See more of Dota 2 on Facebook.
https://www.facebook.com/dota2/
Dota 2. 4,735,461 likes · 2,921 talking about this. Dota is a competitive game of action and strategy, played both professionally and casually by... Dota 2 is on Facebook.
https://m.facebook.com/dota2/
DOTA 2 (@dota_2) • Фото и видео в Instagram
Dota 2 is an Action RTS game, developed by IceFrog. Experience the world's biggest esport. Join the most played PC game.
https://www.instagram.com/dota_2/
DOTA 2 (@DOTA2) | Твиттер
Последние твиты от DOTA 2 (@DOTA2). Dota 2 is a multiplayer Action RTS game developed by @valvesoftware.
https://twitter.com/dota2
Reddit Dota 2 (@redditdota2) | Твиттер
Последние твиты от Reddit Dota 2 (@redditdota2). Official twitter account of r/DotA2. Stay tuned for community fueled coverage of everything Dota 2 related. Sainte-Même, France.
https://twitter.com/redditdota2
Dota 2 Wiki
Compare Dota 2/DotA items. Changes from DotA. Help Needed. Dota 2 Wiki is short on editors.
https://dota2.gamepedia.com/


text:
Secret details of Gloucestershire's £500m incinerator to be revealed 'before Christmas'
A hearing set for January has been dropped
Tom Gibbon
A new version of the £500million contract behind Gloucestershire's incinerator is set to be released in the next few days.
Gloucestershire County Council says the 'updated contract' will prove the £100million of savings the Javelin Park burner will create for taxpayers, along with a 'value for money assessment'.
The council was ordered in the summer to reveal crucial information in the contract it had wanted kept secret. A hearing set for January, in which the council was set to appeal that decision by the Information Commissioner's Office, has been dropped.
The council posted on Twitter: "GCC will publish an updated version of Javelin Park contract and value for money assessment by Christmas. With commercial risk for UBB now at an acceptable level, tribunal and FoI processes can be concluded.
The Javelin Park Incinerator is taking shape (Image: Mark Watkins)
"Documents demonstrate the £100m saving for taxpayers, and value for money this project continues to offer, despite delays."
Revealing more details of the contract will mean the public will be able to see what Gloucestershire County Council previously described as commercially sensitive, such as:
The price per tonne agreed with Urbaser Balfour Beatty (UBB)
The price to third parties of electricity generated by the incinerator
The price to all parties for gate fees
Campaigners have argued that without the redacted information the public were unable to judge whether the authority obtained value for money.
Sid Saunders is on hunger strike at Shire Hall
One activist, Sid Saunders, went on hunger strike for two weeks demanding the county council release the redacted details requested through an FOI.
The incinerator is set for its first test runs in March and was connected to the National Grid in November - a moment described as a 'key milestone' by contractors Urbaser Balfour Beatty.
Timeline: Gloucestershire's Javelin Park incinerator
2012: Where it began
In 2012, plans were revealed to build the incinerator at a cost of £500m
2012: The oppostion
Opposition quickly emerged to the plans which would see construction take place near Junction 12 of the M5
March 2013: Planning debacle
Gloucestershire County Council's own planning committee rejects the proposals after an 11-hour meeting
September 2013: Appeal granted
The plans are given permission by the government after an appeal by contractors Urbaser Balfour Beatty
2014-2015: Legal challenges
Stroud District loses a legal challenge against the building of the incinerator
November 2015: It goes ahead
Another vote by the county council means work can go ahead
2015-2016: Alternative schemes
Campaigners continue to press their case for the incinerator not to be built, and propose alternative schemes
January 2017: Contract blunder
The full contract between the council and UBB is released under Freedom of Information laws with much of it redacted. But a blunder means that a simple trick can reveal the redacted parts, and an investigation is launched
2017: Work set to start
Building the incinerator is due to start in the summer, despite ongoing protests, with an opening date of some time in 2019
Summer 2018: Protests continue
While construction of the incinerator continues, protesters refuse to give in their fight. Dad-of-two Sid Saunders, 38, launches a hunger strike on the steps of Shire Hall in a bid for the full unredacted contract to be released.
The Information Commissioner's Office orders Gloucestershire County Council to release more details of the contract, and sets a July deadline for it to be made public.
After initially appealing the ICO's decision, Gloucestershire County Council relents and says an 'updated contract' will be released before Christmas.
The full library of documents relating to the contract are made public, revealing the total cost to be £602million - £102million more than originally expected.
Campaigners against the incinerator call for a full public inquiry to be held into the extra costs.
The first working trials were expected to start at the site.
The first emissions are seen from the incinerator's chimneys as it gears up to become fully active.
Protesters from Extinction Rebellion entered into a 'Mexican standoff' with bin lorry drivers trying to enter the site to dispose of waste.
Gloucestershire police confirm they are reviewing a decision which found no criminal offences had been committed in relation to the Javelin Park waste incinerator.
Are you for or against the incinerator? Join the debate in the comments below.


Would your like to find other researchers that share an interest in your family lines? Please take a few moments to register your surnames of interest. Who knows, you might find that elusive ancestor, or meet a new cousin.
Click on any surname in the list below to see information on the researcher(s) that registered that name.


Scandalous Women: Scandalous Women is 'Excessively Diverting'!
Scandalous Women is 'Excessively Diverting'!
Vic at Jane Austen's World just tagged me with The Excessively Diverting Blog Award. I'm so excited and pleased. Now it is my turn to share the love with some other worthy blogs.
Recipients, please claim your award by copying the HTML code of the Excessively Diverting Blog Award badge, posting it on your blog, listing the name of the person who nominated you, and linking to their blog. Then nominate seven other blogs that you feel meet or exceed the standards set forth. Nominees may place the Excessively Diverting badge in their side bar and enjoy the appreciation of their fellow blogger for recognition of their talent.
Okay, it was really hard to choose but these are my seven most diverting blogs on the Internet. Most of them are history heavy but I have a few that aren't.
History Undressed - One of the most entertaining blogs on the net. Eliza Knight writes about a different aspect of history from clothing to enduring myths every week.
Reading the Past - A blog giving you the best of the most recent historical fiction.
History Hoydens - A group blog featuring historical fiction writers, Lauren Willig, Tracy Grant, Pamela Rosenthal among others.


text:
Charles Mwando Simba (18 October 1936– 12 December 2016) was a Congolese politician and member of UNADEF. He was born in Moba, Tanganyika District. From February 6, 2007, to October 2008, he served in the government of the Democratic Republic of the Congo as Minister of Rural Development. In October 2008, he was appointed as Defense Minister in Adolphe Muzito's cabinet, succeeding Chikez Diemu. In April 2012, he became the First Vice-President of the National Assembly of the Democratic Republic of the Congo, a position he held until his death.

Mwando Simba also served as a minister during the Congo's second republic and was Governor of Grand Kivu for 12 years.

According to the MLC, he held Belgian nationality as of 28 May 2002, as can be seen on page 29,149 of a letter signed by the Belgian king.

References

1935 births
2016 deaths
People from Tanganyika Province
Defence ministers of the Democratic Republic of the Congo
21st-century Democratic Republic of the Congo people

redpajama_set_name: RedPajamaWikipedia