You can save some cash by riding your bike. Check out this calculator to see how much. Even better, if you log your trips in the Pierce Trips Calendar you can win great prizes. Check out the Campaigns page to see what you could win!
If you work at an employer in Pierce County with 100 or more employees who work full-time and are scheduled to begin work between the hours of 6-9am, your worksite is affected by the Commute Trip Reduction Efficiency Act.
Most large worksites have an Employee Transportation Coordinator (ETC) who promotes transportation options. Your ETC can be an important resource for users of all commute options, including cycling. Not only will they have information on the various Pierce Trips campaigns and contests you are eligible for, but the ETC may also know information specific to your worksite including the location of bicycle facilities, nearby bike trails or routes and the contact information for other employees who regularly bicycle. If you are interested in encouraging your employer to provide bicycle amenities, want to know more about your employers CTR program or want to start a bike team for the annual Commuter Contest, your ETC may be able to help.
Employers who do have a transportation program or ETC can elect to partner with Pierce Transit in order to better connect their employees to the wealth of resources offered by the Pierce Trips team and to be eligible for programs including Emergency Ride Home. Visit Pierce Trips' Employer Services Page for more information.
If your employer partners with Pierce Transit and you use a non-drive alone method to get to work, you may be eligible for a free taxi ride home in case of emergency. Visit Pierce Trips' Emergency Ride Home page for more information or contact your employee transportation coordinator.
When you bicycle to work or use another smart commute option, log your trips at the Pierce Trips Calendar to be eligible for great prizes.


text:
Outspoken by Odd
Broadcasters/Presenters
Digital Content Creators
Adam Blair
Portfolio Bio
Seasoned Rugby League legend Adam Blair has an impressive list of accomplishments under his belt.
Having played at an elite level both in New Zealand and Australia, Adam is most well known for his time playing with the New Zealand Warriors.
At just 16 years old, a young Whangarei-born Adam was head hunted and recruited by the Melbourne Storm team, before moving on to represent New Zealand as vice-captain for the Kiwis.
From there, it was back across The Ditch, where he was offered a contract playing for the Wests Tigers, then the Brisbane Bronocs, before returning to NZ to play for the Warriors in 2018.
Early in 2019 the 32-year-old veteran was named as captain of the Māori All-Stars.
Adam is on the cusp of joining the NRL's 300-game club as he heads into his 14th season.
His popularity on the field has grown him a significant following online, as fans tune into his healthy lifestyle, keeping up with his latest news and updates.
As a speaker, Adam gives a raw, honest and articulate account of growing up in small Northland towns and what it took to become a key player in some of Australasia's biggest league teams. He is passionate about helping disadvantaged youth by connecting them with the passion and camraderie of playing team sport.
Adam lives in Auckland with his wife Jess, and their two sons Taika and Harlem.
© Copyright 2020 | powered by MAINBOARD

redpajama_set_name: RedPajamaCommonCrawl


After keeping secrets forces their break up. Josie & Colin must try to move on separately on two different coasts. Until a tragic twist of fate forces Josie to make some very tough decisions. Pitting Ex against Ex. Will those tough decisions and white lies Josie's forced to make. Does true love ever truly die!


text:
Binyamin Appelbaum on how economists rose to rule the world
Binyamin Appelbaum, of The New York Times, talks about his book, The Economists' Hour, which traces the rise of the economists, first in the United States and then around the globe, as their ideas reshaped the modern world, curbing government…
Joel Bakan is professor of law at the University of British Columbia, and an internationally renowned legal scholar and commentator. A former Rhodes Scholar and law clerk to Chief Justice Brian Dickson of the Supreme Court of Canada, Bakan…
100th episode! Unique one-off celebratory album track-by-track with Brian Keneipp and Paul Nugent
We made it to 100 episodes in just over a year. Well done to the whole team. Thanks to the guests, the musicians, the listeners, everyone! How should we celebrate our one-hundredth episode? - it has to be something special…
Martyn Williams - Mountaineer: on a lifetime filled with barely believable adventurers
An incredible episode with a mountaineer turned Transformational Teacher and Coach for living an Enlightening Life. He's a world-record-holding expedition leader, who led expeditions to Everest, the North Pole, and the South Pole. As well as succeeding in business: he…
Charles Eisenstein on the unravelling of civilisation (ft. musical guest Brian Bulger)
Charles Eisenstein is an American public speaker and author. His work covers a wide range of topics, including the history of human civilization, economics, spirituality, and the ecology movement. Key themes explored include anti-consumerism, interdependence, and how myth and narrative…
Sandor Katz on the Art of Fermentation (ft. musical guests The Silent Comedy)
Acclaimed food author and fermentation guru Sandor Katz whose renegade revival of ancient culinary rituals has transformed his relationship with life and death, inspires songwriters Joshua and Jeremiah Zimmerman of The Silent Comedy to create a new track.
Sandor Katz…
Charles van Rees on invasive species (ft. Musical guests Creature Comfort)
Charles van Rees is a conservation biologist and naturalist specializing in freshwater ecology, ornithology, and human-wildlife water conflict. Freshwater ecosystems are a primary focus of his work because of their extremely high levels of threat, important ecosystem services, and multifarious…
Richard Lawrence on the Nine Freedoms
Richard Lawrence is the Executive Secretary of The Aetherius Society for Europe and Africa, which is the oldest UFO contacts organizations in Britain, and possibly the world. He is also a Bishop in The Aetherius Churches. Richard has devoted his…
The legend of Janet Varney (ft. Frances Luke Accord)
Janet Varney is an Emmy-Nominated actor, comedian, writer and producer. In the animation world, she is lucky enough to be the award-winning voice of Korra on Nickelodeon's critically-acclaimed standout hit "The Legend of Korra," a role for which she continues…
Dennis Tirch on treating yourself with compassion
Dr. Tirch is the Founder of The Center for Compassion Focused Therapy, the first clinical training center for Compassion Focused Therapy (CFT) in the United States. Dr. Tirch has been described as one of the country's foremost leaders in compassion…
VOCAL MASTERCLASS with Lisa Paglin and Marianna Brilla
This is a special one-off episode where I've invited revolutionary singing teachers Lisa Paglin and Marianna Brilla back on the show (listen to Episode 58 first), to give advice to singers with vocal problems.
Thank you to these talented artists…
Track-by-Track Commentary on Podsongs Volume 8
A commentary on the last 12 songs created for the Podsongs project, with musicians Jack Stafford, Maurizio Sarnicola, Massimino Voza and Luigi Falcione. Go behind the scenes with the band to hear how they made the songs inspired by…
Catherine Rampell of The Washington Post on what she'd like from her country
Catherine Rampell is an opinion columnist at The Washington Post. She frequently covers economics, public policy, immigration and politics, with a special emphasis on data-driven journalism. She is also an economic and political commentator for CNN, a special correspondent for…
David Holmgren on how to become resilient in an energy descent future
Australian environmental designer, ecological educator and writer. He is best known as one of the co-originators of the permaculture concept with Bill Mollison. His latest project is RetroSuburbia is part manual and part manifesto. The book shows how Australian…
Michael Lennox on dreams and astrology (rerelease)
Dr. Michael Lennox obtained his Masters and Doctorate in Psychology from The Chicago School and his Doctoral Dissertation, Astrology, and Personality, is published by Lambert Academic Publishing. Dr. Lennox is also the author of two books on dreams, Dream Sight…
Rosamund Kissi-Debrah, campaigner for clean air and mother of the only person in the world to have 'officially' died from air pollution
Rosamund Kissi-Debrah is a grassroots campaigner in London, UK raising awareness of asthma and the health problems that can be caused by air pollution. Kissi-Debrah's campaign for clean air followed the death of her 9 year old daughter, Ella Roberta…
Robert Svoboda on Ayurveda
Dr. Robert Svoboda is the first Westerner ever to graduate from a college of Ayurveda and be licensed to practice Ayurveda in India. During and after his formal Ayurvedic training he was tutored in Ayurveda, Yoga, Jyotish, Tantra and other…
Chad Nelsen the surfer campaigning for plastic-free seas
Chad Nelsen is the head of The Surfrider Foundation, a grassroots environmental organization dedicated to protecting oceans and beaches. Its major programmatic work is carried out by sixty local chapters around America. They've more than forty thousand members, as well…
Jeffrey Babcock - the curator of Amsterdam's underground cinema scene
Jeffrey Babcock is an American alternative film curator and cultural activist now in Amsterdam. As a promoter of independent film culture he is currently the programmer of several underground cinema venues throughout the city, where he places the films he…
Gregg Caruso puts the case against free will (ft. Nate Frederick)
Gregg D. Caruso is Professor of Philosophy at SUNY Corning, Visiting Fellow at the New College of the Humanities (NCH London), and Honorary Professor of Philosophy at Macquarie University. He is also Co-Director of the Justice Without Retribution Network housed…
« First ‹ Prev 1 2 … 6 Next › Last »


text:
Explore the Y
About the Mission
Day and Overnight Camps
Youth Potential
Y Achievers
Y Readers
Let's Move! Outside
Y-Guides
Accessible Activities
Equal Membership Access
D.I.G. Committee
DIG Innovation Network
New American Welcome Center
Safety Around Water
Advocacy with the Y
Partner with the Y
Work at the YMCA
About Y Advocacy
Current Issues/Topics
Help Raise Funds
Join a Community Project
Coach Youth Sports
A School Year Unlike Any Other
A Nonprofit Unlike Any Other
DO MORE. BE MORE.
Change Lives with a Career at the Y
MEANINGFUL WORK THAT MATTERS
From inspiring youth in day camp with a sense of belonging, encouraging busy parents with nutrition advice for their family or helping underserved teens accomplish academic success — the Y is more than just a gym, and working here is more than a job. It's a mission to transform lives, especially the young, and encourage all to develop their fullest potential in spirit, mind and body.
The YMCA of Greater Charlotte employs over 4,000 full and part time team members across our 19 locations and two overnight camps. Together we serve over 300,000 men, women and children — regardless of age, income or background. We strive to recruit, develop and retain a talented and diverse workforce that advances the mission of the YMCA, and you'll find a wide variety of career options depending on your individual goals.
More Than a Job
We invite you to explore our current opportunities and learn more about what it means to be a team member at the YMCA of Greater Charlotte. Join us in this amazing work — to develop your skills, realize your leadership potential and discover a career that's so much more than a job!
Explore Career Opportunities
Looking for a great place to work and play? We're hiring in Charlotte! Fill out a job application to join our team and make a positive and lasting impact on the lives of children, adults, and families.
WORKING AT THE Y
YMCA Culture
Lead and Develop
The YMCA is a Great Place to Build Your Career
Multiply the Mission
Since the first YMCA in Charlotte opened on November 11, 1874, we've made it our mission to put Christian principles into practice through programs that build healthy spirit, mind and body for all. If you seek a fulfilling, challenging and rewarding career, the Y is a great place to grow and make a lasting impact on countless lives.
Our Strategic Focus
Inspired by Christian principles, we're committed to growing our reach, resources and impact to serve all. Many people choose to work for the Y to make a real, lasting difference in our members and others who participate in our programs. What they soon discover is that changing others for the better changes themselves. As an employee of the YMCA of Greater Charlotte, you'll have a meaningful opportunity to help influence positive social change within the community you serve. Learn more about our mission.
Exceptional Team Members Drive Our Mission
Our work environment is filled with passionate dreamers, thinkers and doers. At the Y, everyone is a leader. Whether a camp counselor, marketing manager or fitness instructor, we all share a common goal to serve and make our community better. Our inclusive culture is as diverse as the communities we serve, and we celebrate the uniqueness of every individual.
Encouraging Life at Work
Many organizations talk about promoting a work-life balance. At the Y, we believe life doesn't have to begin when work ends. We inspire your life at work. Our free Y memberships encourage health and wellness for our team members and their entire households. And we promote volunteer opportunities, such as allowing our employees to leave work and read to 6-year-olds with Y-Readers, lead arts and crafts at day camp, or teach kids about safety around water. Our culture feels like a large, extended family that genuinely cares for the well-being of one another.
Investing in Our Team
The Y is a rewarding place to work while developing a career and gaining leadership and advancement opportunities. YMCA of Greater Charlotte employs talented and dedicated people to serve our community. Our team members are able to put their experiences, abilities and energy to work while exploring and discovering new talents. We're committed to personal and professional development with support and accountability.
Tools to Help You Succeed
Our devoted Leadership Development Team is dedicated to help you define, create and develop your future here. Aside from mandatory trainings in safety and service as well as technical trainings, we offer leadership development and nationally recognized certification courses — and we provide the time away from work to take them. Our goal is to develop our team members and volunteers, and to promote talent from within our organization.
Career Paths Focusing on Impact
As a Y team member, you'll have the opportunity to build a career that's right for you. Our Individual Development Plan for full-time employees captures opportunities to achieve personal career aspirations and goals, as well as further the mission of the organization. Our plan focuses on three areas of growth and impact at all stages of your career:
Our current vision and the strategic focus related to growth in leadership
Our seven behaviors that guide our service
Your personal career aspirations and goals
Enriching Work and Life
We believe our team members deserve the best. Our Total Rewards package breathes life into work with benefits that help team members grow personally and professionally, with a wide array of programs to move them successfully forward. From little things like discounts on Y programs to big things like a generous matching 403(b) Retirement Savings Plan, the YMCA of Greater Charlotte is committed to helping our people thrive at work and at home. Our comprehensive benefits package determined by employment status includes the following:
FULL-TIME STAFF PART TIME STAFF
Flexible Spending Account Health Care Reimbursement
Flexible Spending Account Dependent Care Reimbursement
Retirement 403(b) Savings Plan
Emergency Travel Assistance
Short-term Disability
Free Y Membership for Household
Free Y Membership for Employee
FITNESS INSTRUCTORS
Discounts on Nearly 20 Athletic Brands
Employee Discounts on Programs
Access to Free Counseling Services
Access and Belonging for All
The Y is made up of people of all ages and from every walk of life working side by side to strengthen communities. We practice inclusion by valuing the diversity of all people within our organization, and in those we serve. We celebrate the uniqueness of every individual. Our diverse team members thrive in a culture where they're encouraged and supported to reach their full potential.
Our Diversity Makes Us Strong
The Y welcomes independent thinkers from a variety of backgrounds who can work together collaboratively with a focus on multiplying our mission. We know different strengths and perspectives encourage creativity and challenge us to create better experiences for our members and our community.
We Believe in Equal Opportunity
We provide equal opportunity in employment to all team members and applicants for employment regardless of race, color, creed, religion, national origin, sex, marital status, sexual orientation, status with regard to public assistance, membership or activity in a local commission, disability, age, or other legally protected status.


Can you believe that I just ____________?
Fill in the blank Friday. Every Friday, I put up a random picture from my week and start the caption. I invite you fill in the blank and finish my caption. Feel free to be funny or cute or just shout out the first thing that comes to mind!
"took one little sip from the jug under the sink and now my arm is disappearing"
ha! that's my secret stash! i would never delete you, brad! my comments never seem to work either. i should look into that.
What Do Margaritas Have to do With Peace?


This message is for the kid, yep you are right this truely is persistance and that is how you get what you want in life.
God, your kid is something else. You must be proud of your genes! On second thought, maybe your wife's genes. Okay both of your genes.
Enjoy the kid! Coz I am!!!


text:
\section{Introduction}\label{sec-intro}

Many large-scale complex systems reach an equilibrium over time. Examples include epidemics, transportation, markets, and supply chains. With no planning, the global performance at this equilibrium can be poor. When a decision-maker can control some parameters of such a system, it can influence the equilibrium that the system converges to. Examples are changing the frequencies of subway lines, or the masking and isolation policies during an epidemic. However, a model for the dynamics of these large-scale complex systems is rarely available. Instead, the decision-maker can only observe the impact of its decisions in real time. It is infeasible to allow the system to converge to equilibrium under each policy since this will waste significant time on suboptimal policies. This introduces a learning problem of controlling such systems with `bandit feedback' \cite{bandit-book}.

Motivated by this, we consider an agent that takes an action from a discrete set of actions at each timestep. There is an underlying system that evolves with time depending on the agent's action. At each timestep, the agent receives a noisy reward as a function of its action \textbf{and} the  `state' of the system. The key aspect of the underlying system is that if we fix the action, it would asymptotically converge to a unique equilibrium as a function of this action. However, waiting too long for the system to converge to a bad equilibrium is costly. We measure the regret of the agent as the difference between the reward at the optimal equilibrium (i.e., for the optimal action) and the accumulated reward of the agent. This introduces a new bandit problem which we name \textit{`Equilibrium Bandits'}.

The reward process in equilibrium bandits is not i.i.d. over time. The expected reward approaches the expected reward at equilibrium and therefore has memory. Hence, stochastic bandit algorithms result in linear regret in $T$. On the other hand, adversarial bandit algorithms also result in linear regret.

In applications such as epidemics and transportation, convergence to equilibrium can take significant time. Therefore, we are interested in the dependence of the regret on the worst-case `approximate convergence time' $\tau_c$ in addition to the horizon $T$. Alternatively, we can think of $\tau_c$ as $T$-dependent.

When estimating the reward at equilibrium, the distance of the current state from equilibrium creates `equilibrium noise'. Hence, we need more consecutive `arm pulls' to weaken the equilibrium noise at the last pull. This conflicts with the averaging required to weaken the i.i.d. reward noise, since averaging over longer periods would have to use states that are too far from equilibrium.

We present the Upper Equilibrium Concentration Bound (UECB) algorithm for equilibrium bandits that builds upon the basic intuition behind the Upper Confidence Bound (UCB) algorithm \cite{bandit-book}. Our main innovation is employing `convergence bounds' to determine the maximum possible reward the agent could get by waiting for the system to converge to equilibrium for a given action. A chosen action in UECB is played consecutively for a full ``epoch''. The epoch length increases with the number of epochs this action has been chosen in the past. Consequently, UECB spends time weakening the equilibrium noise only for promising actions. UECB balances between the i.i.d.\ noise and equilibrium noise by only using a fraction of samples from a given epoch for the reward estimation. We show that UECB achieves $\mathcal{O}(\log(T)+\tau_c\log(\tau_c)+\tau_c\log\log(T))$ expected regret. We also prove a lower bound that shows that UECB is optimal up to logarithmic factors in $\tau_c$.

We detail two real-life problems that can be modeled as equilibrium bandits. The first example is epidemic control, where the agent is the government that chooses a policy that may include lockdowns, link closures, and enforcing masks \cite{bistritz2019controlling}. These policies incur both an operational cost and a health cost by affecting the infection rate. We use the SIS model \cite{SIS1} to formalize the epidemic spread. Our second example is a continuous game where the system consists of multiple players trying to optimize their utility functions using gradient-based learning \cite{mazu}. Specifically, we consider resource allocation games where the policymaker chooses the set of resources available to each player and wishes to maximize the sum of utilities of all players at equilibrium \cite{korilis1999avoiding}. We then simulate these examples and show that UECB performs well based on minimal knowledge of the system that is readily available in practice.

\subsection{Related Work}
There has been plenty of research on multi-armed bandits in the last century, including stochastic bandits, contextual bandits, linear bandits, and adversarial bandits \cite{bandit-book}. Special cases for each of these have been studied in great detail \cite{bandit-1, bandit-2, Auer_adversarial}. There are also works on bandits which deal with a system evolving with time. These often deal with a Markov-chain based stochastic evolution, e.g., restless bandits \cite{Whittle_1, Whittle_2}.

Unlike restless bandits, in equilibrium bandits, an unperturbed system, where the chosen action is fixed, would asymptotically converge to equilibrium as a function of that action. Such behavior is typical to multiagent systems, specifically with many humans in the loop (e.g., epidemics and transportation).  Furthermore, stochastic and adversarial bandit algorithms both give linear regret in equilibrium bandits. We propose the first algorithm, named UECB, that achieves sublinear regret for equilibrium bandits.

Equilibrium bandits can be thought of as a special case of non-stationary stochastic bandits \cite{besbes2014stochastic}, but applying this approach would result in a regret bound of $O(T^\frac{2}{3})$ at best (depending on the convergence rate to equilibrium) since it does not leverage the converging structure of equilibrium bandits.

Another field closely related to our work is that of reinforcement learning (RL) \cite{Sutton-book}. Classically, the RL problem is modeled using Markov decision processes and the aim of the agent is to choose the action at each step that maximizes its cumulative reward. There has been significant research in developing algorithms for RL \cite{Watkins, BVR, policy_grad}. In recent years, there has been progress in developing deep learning-based methods for complex problems such as multi-agent control, robotics, and games\cite{alphago, atari, dec_trans, robotics}. Equilibrium bandits differ from the RL literature in one major aspect: the state in our case evolves in a non-stochastic, converging way. One could model our evolution as a deterministic Markov chain. However, RL algorithms are designed for general problems and typically assume an ergodic Markov chain \cite{Watkins} or the existence of an offline simulator that allows `restarts' of the Markov chain \cite{alphago}. These assumptions do not hold in our case, since our deterministic Markov chain is absorbing and our target is real-time learning which cannot be restarted. Furthermore, in equilibrium bandits, only the reward is observable whereas typical RL assumes that the state is observable as well.

Our work is related to the literature on control and intervention in games, where a manager can tune some parameters in the reward functions of the players \cite{grammatico2017dynamic,parise2020analysis,mguni2019coordinating, Alpcan2009a,bistritz2021online,ratliff2020adaptive}. While our dynamics do not have to stem from a game, games are a key example of a system that converges to equilibrium. From this point of view, our work is the first to provide regret guarantees while learning to control an unknown game.

\section{Problem Formulation}\label{sec-prob_form}

Consider an agent that chooses an action $a_t$ at each time $t$ from the action set $\mathcal{A}=\{1,\ldots,K\}$. The action controls an underlying system that evolves with time and affects the agent's reward. Let $z_t$ be the state of the system at time $t$. We assume that $z_t$ lies in a bounded and closed set $\mathcal{Z}\subset\mathbb{R}^d$. Then we define the `evolution' function $g:\mathcal{A}\times\mathcal{Z}\rightarrow\mathcal{Z}$ and the `reward' function $f:\mathcal{A}\times\mathcal{Z}\rightarrow\mathbb{R}$. We assume that $f(a,z)$ is Lipschitz continuous with parameter $L>0$, as a function of $z$, for all actions $a$. Without loss of generality, we make the following two assumptions to simplify the notation:
$f(a,z)$ is bounded in $[0,1]$ for all $a,z$ and $\mathcal{Z}\subseteq \mathcal{B}_{0.5}(\|\cdot\|)$ where $\mathcal{B}_{0.5}(\|\cdot\|)=\{z\in\mathbb{R}^d\mid\|z\|\leq 0.5\}$. Here $\|\cdot\|$ denotes any compatible norm on $\mathbb{R}^d$. The function $g$ determines the next state of the system based on the current state and the action taken, i.e., $z_{t+1}=g(a_t;z_t)$. The function $f$ determines the agent's expected reward, i.e., $x_t=f(a_t;z_t)$. The noisy reward $y_t$ observed by the agent is given by $y_t= x_t+\eta_t$ where $\eta_t$ is i.i.d.\ subgaussian noise with parameter $\sigma$, i.e., $\mathbb{E}[\eta_t]=0$ and $\mathbb{E}[\exp(\alpha\eta_t)]\leq \exp(\alpha^2\sigma^2/2)$ for all $\alpha\in\mathbb{R}$.  Rewards are typically noisy since the effectiveness of a policy cannot be deduced accurately and is often based on stochastic data.

Motivated by applications such as epidemic control and game control, we make the following assumption on the evolution function:
\begin{assumption}\label{assu-evolution}
The function $g(\cdot;\cdot)$ satisfies the following conditions:
\begin{enumerate}[(a)]
    \item For each action $a$, consider the iteration $z_{t+1}=g(a;z_t)$ for $t>0$. There exists a unique equilibrium corresponding to action $a$, i.e., a $z_a^*$ such that $z_a^*=g(a;z_a^*)$. Furthermore, this equilibrium is a stable point, i.e., $\lim_{t\uparrow\infty} g^{(t)}(a;z)=z_a^*$ for all $z\in\mathcal{Z}$.

    \item When action $a$ is played, the distance of the state $z_t$ from $z_a^*$ decreases. Formally,
    \begin{equation}\label{assumption-2}
        \|g(a;z)-z_a^*\|\leq c(a;z)\|z-z_a^*\|, \; \forall \; a\in\mathcal{A}, z\in\mathcal{Z},
    \end{equation}
    where $0<c(a;z)<1$. We assume that $c(a;z)$ is bounded away from $1$, i.e., $\exists \; \tau_c\geq 1$ s.t.\ $c(a;z)<e^{-\frac{1}{\tau_c}}$ for all $a$ and $z$.
\end{enumerate}
\end{assumption}

Part (a) of the above assumption implies that if the agent keeps the action $a$ fixed, the system will asymptotically converge to the equilibrium state $z_a^*$. In addition, we define $x_a^*\coloneqq f(a,z_a^*)$ as the equilibrium reward for action $a$.

The $\tau_c$ in part (b) is the ``approximate convergence time to equilibrium'', i.e., the timesteps required until the distance of the state from equilibrium is at most a $1/e$ factor of its initial value.

A basic class of functions $g$ satisfying the above assumption are contraction mappings. A contraction mapping has a unique fixed point which is the unique equilibrium point required for part (a). Similarly, the contraction factor is the constant $c(a;z)$. Contraction mappings are commonly found in solutions of many ODEs such as Newton's method \cite{Protter} and for popular policy evaluation schemes such as temporal difference learning \cite{TD0}. In section \ref{sec-appli}, we detail two applications that result in non-contractive mappings but still satisfy the above assumption.

We now return to the problem and the objective of the agent. The agent takes an action $a_t$ at each time $t$. The agent does not observe the underlying state $z_t$ and only observes a noisy version of the reward $f(a_t;z_t)$ at each time $t$. The optimal action $a^*$ is defined as $a^*\coloneqq\argmax_a f(a;z_a^*)$, i.e., the action with the highest equilibrium reward. For simplicity, we assume that this optimal action is unique, i.e., $f(a,z^*_a)<f(a^*,z^*_{a^*})$ for all $a\neq a^*$. Nevertheless, our analysis follows with minor modifications for the case of multiple optimal actions, using the same algorithm. Define the suboptimality gap for each action as $\Delta_a=x^*_{a^*}-x^*_a$. The regret till time $T$ is $R(T)\coloneqq \sum_{t=1}^T \left(f(a^*,z_{a^*}^*) - y_t\right)$.
We want to design an algorithm for the agent that minimizes the expected cumulative regret:
$$\mathbb{E}[R(T)]=\max_a\sum_{t=1}^T f(a;z^*_a)-\mathbb{E}\left[\sum_{t=1}^Tf(a_t;z_t)\right]$$
where the expectation is with respect to the stochastic noise. With multiple equilibria, we cannot guarantee to which one the system would converge. We can then redefine the above regret such that $z^{*}_{a^{*}}$ is the worst equilibrium for action $a$, with no modifications needed in our analysis or algorithm.

With this regret, a good algorithm would find the optimal action as quickly as possible and then commit to it to allow the system to converge to the corresponding equilibrium. This objective is inspired by applications such as epidemics and transportation (studied in detail in section \ref{sec-appli}). In an epidemic, the government is the agent that has to choose the best policy to control the spread. Then, the different policies (e.g., lockdown, masks) are the actions and the underlying state is the fraction of infected individuals. The cost then takes into account the health costs (e.g., deaths and complications) and the operational cost (e.g., treatment and economic implications). This example makes it clear why we wish to maximize the expected cumulative reward and why we wish to commit to the optimal action as quickly as possible.

Our contribution is the novel UECB algorithm, which is presented in the next section. Our main result proves a regret bound for UECB, which we show is optimal up to logarithmic factors.
\begin{theorem}\label{thm-noisy}
For an equilibrium bandit instance satisfying Assumption \ref{assu-evolution}, with a Lipschitz continuous reward function bounded in $[0,1]$ and noise $\eta_t$ that is subgaussian with parameter $\sigma$, the expected cumulative regret of the UECB algorithm (Algorithm \ref{algo-noisy}) is bounded as:
\begin{eqnarray}\label{eqn-thm-noisy}
    \mathbb{E}[R(T)]&=& \mathcal{O}\bigg(\sum_{a\in\mathcal{A},a\neq a^*}\frac{\sigma^2\log(T)}{\Delta_a}+\tau_c\log\left(\tau_c\log\left(\frac{1}{\Delta_a}\right)\right)\nonumber\\
    &&\;\;\;\;\;\;+\tau_c\log\left(\frac{\sigma^2\log(T)}{\Delta_a^2}\right)\bigg),
\end{eqnarray}
where $\Delta_a=x_{a^*}^*-x_a^*$ is the suboptimality gap for action $a$.
\end{theorem}

The dominant term in \eqref{eqn-thm-noisy} depends on $\frac{T}{\tau_c}$ which quantifies how many times we can afford to converge to equilibrium, and is application dependent. For example, $\mathbb{E}[R(T)]=\mathcal{O}(\sqrt{T}\log(T))$ for $\tau_c=\mathcal{O}(\sqrt{T})$, and $\mathbb{E}[R(T)]=\mathcal{O}(\log(T)\log\log(T))$ for $\tau_c=\mathcal{O}(\log(T))$.

Stochastic bandits can be viewed as a special case of equilibrium bandits where the convergence to equilibrium is instantaneous. Equilibrium bandits is a more challenging problem since the rewards are no longer i.i.d. over time. Algorithms like UCB treat the rewards as independent over time and do not account for how far the system is from the equilibrium. Consequently, the UCB algorithm achieves linear regret for equilibrium bandits (Theorem \ref{UCB-linear} in the Appendix).

Equilibrium bandits are a special case of adversarial bandits, where any sequence of rewards is allowed. However, adversarial regret bounds are significantly weaker than our regret bound since they compare to the best action in hindsight. In contrast, our regret resembles the more demanding regret of stochastic bandits, which compares to the ``absolute'' optimal action. Therefore, adversarial bandit algorithms also achieve linear regret for equilibrium bandits.

In our notation, the regret for the adversarial problem would be $$\mathbb{E}[R_{adv}(T)]=\max_a\sum_{t=1}^T f(a;z_t)-\mathbb{E}\left[\sum_{t=1}^Tf(a_t;z_t)\right].$$
The adversarial regret looks at the state sequence $\{z_t\}_{t=1}^T$ as given and ignores the fact that our action sequence $\{a_t\}_{t=1}^T$ impacted the state sequence. In the epidemic control example, this would mean that the adversarial regret tries to find the best action given the number of infections over time, as though these numbers could not be avoided by an agent who would have taken better actions.

\section{UECB Algorithm}
In this section, we present our novel UECB algorithm designed for equilibrium bandits. To provide intuition, we start by analyzing the simpler case where the rewards are not noisy.

\subsection{The Noiseless Case}\label{subsec:noiseless}
Consider the special case where the rewards are not noisy, i.e., $\eta_t=0$ a.s. for all $t$. Hence, the agent directly observes $y_t=x_t=f(a_t;z_t)$. Since there is no noise, it should be possible to find the optimal action in bounded time which results in an expected regret of $\mathcal{O}(1)$.

A naive algorithm would pick each action consecutively a sufficiently large number of times (denoted by $t_{try}$), to allow the system to approach the equilibrium corresponding to this action. The agent will then know the reward at equilibrium for each action with arbitrarily low error, and can then commit to the optimal action. This naive algorithm is actually the default choice in many real-life scenarios. This algorithm can achieve sublinear regret if $t_{try}$ is above a threshold, which depends on $\tau_c$ and the suboptimality gap. Since the suboptimality gap is unknown to the agent, the naive algorithm achieves linear regret in general.

Instead, we propose the ``Upper Equilibrium Confidence Bound'' (UECB) algorithm for the noiseless case based on `convergence bounds'. Suppose the state of the system at time $t$ is $z_t$ and action $a$ is taken for $\ell$ timesteps consecutively after that. Then, using Assumption \ref{assu-evolution}, we have $$\|z_{t+\ell}-z_a^*\|\leq e^{-\frac{\ell}{\tau_c}}\|z_{t}-z^*\|.$$ Using the assumption that $\mathcal{Z}\subseteq \mathcal{B}_{0.5}(\|\cdot\|)$, we have $\|z_{t}-z^*\|\leq 1$. Without this assumption, the only modification required would be to replace $L$ with $2L\max_{z\in\mathcal{Z}}\|z\|$ henceforth in the paper. Now using the Lipschitz property of the reward function, we deduce that
\begin{equation}\label{UECB-intuition}
    f(a;z_{t+\ell})-Le^{-\frac{\ell}{\tau_c}}\leq f(a;z_a^*)\leq f(a;z_{t+\ell})+Le^{-\frac{\ell}{\tau_c}}.
\end{equation}

We assume that the agent knows $\tau_c$, which is a worst-case bound on the actual convergence time. Given this knowledge, $f(a;z_{t+\ell})+Le^{-\ell/\tau_c}$ is the maximum possible reward action $a$ can yield at the equilibrium point corresponding to $a$. As we demonstrate in the applications of Section \ref{sec-appli}, knowing a bound on $\tau_c$ is significantly easier than knowing the system parameters.

In practice, the agent can often observe more than just the reward (e.g., the state $z_t$ or the convergence rate $c(a;z_t)$). With more knowledge, the above bounds can be tightened without affecting our analysis or the UECB algorithm that uses them as input.

It is necessary to play an action consecutively for some time to get an accurate enough estimation of the reward at equilibrium given that action. However, we do not want to always wait for the system to converge as it might waste precious time and incur significant regret. The idea behind the UECB algorithm (Algorithm \ref{algo-noisy}) is to play actions that seem to lead to good equilibria for an increasing number of turns to allow it to reach closer to convergence. This along with the above-mentioned convergence bound serves as the basic intuition behind UECB. Instead of switching an action at every timestep, UECB chooses an action to be played consecutively over a full `epoch'. The epoch length increases with the number of epochs the chosen action has been played before.

 Let $m_{a,n}$ denote the number of epochs action $a$ has been chosen for till the end of epoch $n$, and let $t_n$ denote the total number of timesteps till the end of epoch $n$. Additionally, let $t_{a,n}$ denote the number of timesteps action $a$ has been played till the end of epoch $n$. Let the action in the next epoch $a_{n+1}$ be chosen as $$a_{n+1}=\argmax_{a\in\mathcal{A}}UECB_{a,n}$$ where $UECB_{a,n}$ is defined below. The length of epoch $n+1$ is chosen as $\ell_{n+1}\coloneqq2\rho_2\exp(\rho_1(m_{a_{n+1},n}+1))$, where $\rho_1,\rho_2$ are positive parameters, as explained after Theorem \ref{thm-noiseless}. The agent plays this action for the complete epoch and observes the reward obtained at the last timestep of that epoch $x_{t_{n}+\ell_{n+1}}=x_{t_{n+1}}$, denoted by $\hat{x}_{a,n+1}$. Finally at end of epoch $n+1$, $UECB_{a_{n+1},n+1}$ is updated as follows:
$$UECB_{a_{n+1},n+1}=\hat{x}_{a,n+1}+Le^{-\frac{\ell_{n+1}}{\tau_c}}$$
and $UECB_{a,n+1}=UECB_{a,n}$ for all other actions $a\neq a_{n+1}$.

We now give a bound on the maximum number of times the UECB algorithm chooses a non-optimal action and a bound on the maximum possible regret in the noiseless case.
\begin{theorem}\label{thm-noiseless}
Let $\eta_t=0, a.s.$ for all $t$ (i.e., no noise). Then for an equilibrium bandit instance satisfying Assumption \ref{assu-evolution}, with a Lipschitz continuous reward function bounded in $[0,1]$,
\begin{enumerate}[(a)]
    \item Algorithm \ref{algo-noisy} chooses a suboptimal action, i.e., action other than $a^*$, only for a finite number of turns $\hat{T}$ where $$\hat{T}= \mathcal{O}\left(\sum_{a\in\mathcal{A},a\neq a^*}\tau_c\log\left(\frac{1}{\Delta_a}\right)\right).$$
    \item For all $t>0$,
$$    R(t)= \mathcal{O}\left(\sum_{a\in\mathcal{A},a\neq a^*}\Delta_a\tau_c\log\left(\frac{1}{\Delta_a}\right) + \tau_c\log\left(\tau_c\log\left(\frac{1}{\Delta_a}\right)\right) \right).$$
\end{enumerate}
\end{theorem}

Part (a) of Theorem \ref{thm-noiseless} is based on the maximum number of times a suboptimal action may need to be played consecutively to differentiate it from the optimal action. This gives the maximum number of epochs that may be required for each suboptimal action and hence the number of steps required in the worst case. The first term in the regret bound is obtained by simply multiplying the suboptimality gap for each action. The second term stems from the maximum number of times UECB switches between actions. Switching to a new action resets the convergence of the system to a new equilibrium,  which incurs $\mathcal{O}(\tau_c)$ regret per switch.

We can construct scenarios where any algorithm that achieves sublinear regret would have to play each suboptimal action $a$ at least $\Omega(\tau_c\log(1/\Delta_a))$ times to distinguish it from the optimal action. To see that, consider converging reward sequences that are identical for all actions for the first $\tau_c\log(1/\Delta_a)$ turns and start differing only after. This implies a worst-case lower bound of $\Omega(\tau_c\Delta_a\log(1/\Delta_a))$ for the noiseless case (Theorem \ref{lower-bound} in the appendix).

\sloppy The exponential increase in the epoch lengths is chosen to obtain bound of the form $\mathcal{O}(\tau_c\log(1/\Delta_a))$ in Theorem \ref{thm-noiseless} part (a). Any increasing sequence of epoch lengths will give a finite regret but will not have a better bound orderwise. For example, linearly increasing epoch lengths i.e., $\ell_n\propto m_{a_n,n}$, yield a bound of $\mathcal{O}((\tau_c\log(1/\Delta_a))^2)$. On the other hand, even if epoch lengths grew faster than exponential, e.g., $\ell_n\propto \exp(\exp(m_{a_n,n}))$, we would still obtain a bound of $\mathcal{O}(\tau_c\log(1/\Delta_a))$. Epoch lengths that increase too fast do not do well in practice as they waste precious time on suboptimal actions.

\subsection{The Noisy Case}
We now consider the general case where $\eta_t\neq0$. The UECB algorithm given for the noiseless case cannot be used here as it only considers the final reward observed, which can be very noisy. To deal with the noise, it is necessary to average multiple observations. The estimated expected reward corresponding to the equilibrium point of an action has two kinds of errors - due to the i.i.d. noise and due to the distance from equilibrium (i.e., ``equilibrium noise''). Averaging creates a trade-off between the two errors. For example, averaging over all rewards observed for an action reduces the i.i.d.\ noise but increases the equilibrium noise since early rewards were earned far from equilibrium. Similarly, considering only the last reward has low equilibrium noise but high i.i.d.\ noise.

Hence we propose the UECB algorithm for the noisy case inspired by the popular UCB algorithm \cite{bandit-book}. We have the same epoch-based structure as before, i.e., $\ell_{n+1}\coloneqq 2\rho_2\exp\left(\rho_1(m_{a_{n+1},n}+1)\right)$. For action $a_{n}$, define
$$\hat{x}_{a_{n},n}=\frac{2}{\ell_n}\sum_{t=t_{n-1}+\frac{\ell_{n}}{2}+1}^{t_{n}}y_t,$$
i.e., the average of the second half of the last epoch corresponding to that action. For other actions $a\neq a_n$, $\hat{x}_{a,n}=\hat{x}_{a,n-1}$. Also, for all actions $a$, define $n_a$ as the last epoch action $a$ was played before the end of epoch $n$. Then $\ell_{n_a}$ is the length of the last epoch action $a$ was played before the end of epoch $n$, i.e., $\ell_{n_a}=2\rho_2\exp\left(\rho_1m_{a,n}\right)$. Then for all actions, we define:
$$UECB_{a,n}\coloneqq\hat{x}_{a,n}+\frac{2}{\ell_{n_a}}\frac{L\exp(-\frac{1}{\tau_c}(1+\frac{\ell_{n_a}}{2}))}{1-\exp(-\frac{1}{\tau_c})}+\sqrt{\frac{4\sigma^2}{\ell_{n_a}}\log\left(\frac{2}{\delta_n}\right)},$$
where $\delta_n\coloneqq \frac{1}{t_n^3}.$ Similar to the intuition behind the UCB algorithm, this $UECB_{a,n}$ is defined to ensure that $UECB_{a,n}\geq x^*_a$ with probability of at least $1-\delta_n$. This is proved in Lemma \ref{lemma-part1}. Finally, as before, the action for the next epoch is chosen as follows:
$$a_{n+1}=\argmax_{a\in\mathcal{A}}UECB_{a,n}.$$

\begin{algorithm}[t]
\caption{\label{algo-noisy} UECB Algorithm}

\textbf{Initialization: }Let $m_{a,0}=0$ for all $a\in\mathcal{A}$.

\textbf{Input: } Constant $\tau_c\geq 1, L>0$ and parameters $\rho_1, \rho_2>0$.

\textbf{For epochs $n=1$ to $K$ do}
\begin{enumerate}
    \item Play action $a_n=n$ for $\ell_n=2\rho_2e^{\rho_1}$ timesteps from $t=t_{n-1}+1$ to $t=t_n=t_{n-1}+\ell_n$.
    \item $\hat{x}_{a,n}, m_{a,n}, UECB_{a,n}=UPDATE(y_{t_{n-1}+1},\ldots,y_{t_n})$
\end{enumerate}

\textbf{For epochs $n\geq K+1$ do}
\begin{enumerate}
    \item Choose action $a_n=\argmax_{a\in\mathcal{A}} UECB_{a,n-1}$.
    \item Play action $a_n$ for $\ell_n=2\rho_2\exp(\rho_1(m_{a_n,n-1}+1))$\\ timesteps from $t=t_{n-1}+1$ to $t=t_n=t_{n-1}+\ell_n$.
    \item $\hat{x}_{a,n}, m_{a,n}, UECB_{a,n}=UPDATE(y_{t_{n-1}+1},\ldots,y_{t_n})$
\end{enumerate}
\textbf{End}

\textbf{function $UPDATE(y_{t_{n-1}+1},\ldots,y_{t_n})$}
\begin{enumerate}
    \item $m_{a_n,n}=m_{a_n,n-1}+1$ and $m_{a,n}=m_{a,n-1}$ for $a\neq a_n$.
    \item if (noiseless):
    \begin{enumerate}
        \item $\hat{x}_{a_n,n}=y_{t_n}$ and $\hat{x}_{a,n}=\hat{x}_{a,n-1}$ for $a\neq a_n$.
        \item $UECB_{a,n}=\hat{x}_{a,n}+Le^{-\frac{1}{\tau_c}\ell_{n_a}}$ for $a\in\mathcal{A}$.
    \end{enumerate}
    \item if (noisy):
    \begin{enumerate}
        \item $\hat{x}_{a_n,n}=\frac{2}{\ell_n}\sum_{t=t_{n-1}+\frac{\ell_n}{2}+1}^{t_n}y_t$ and $\hat{x}_{a,n}=\hat{x}_{a,n-1}$ for $a\neq a_n$.
        \item For all actions $a\in\mathcal{A}$, $$UECB_{a,n}=\hat{x}_{a,n}+\frac{2}{\ell_{n_a}}\frac{L\exp(-\frac{1}{\tau_c}(1+\frac{\ell_{n_a}}{2}))}{1-\exp(-\frac{1}{\tau_c})}+\sqrt{\frac{4\sigma^2}{\ell_{n_a}}\log\left(\frac{2}{\delta_n}\right)},$$ where $\delta_n=1/t_n^3$.
    \end{enumerate}
\end{enumerate}
\end{algorithm}

Theorem \ref{thm-noisy} gives a regret bound on the UECB algorithm. Next, we make a few comments on the algorithm and its bound:
\begin{itemize}
    \item The first term in the regret bound, $\mathcal{O}(\log(T)/\Delta_a)$, appears also in the regret bound for the UCB algorithm in stochastic multi-armed bandits and stems from the noisy observations that both scenarios share. The second term in the regret bound, $\mathcal{O}(\tau_c\log(\tau_c))$, also appears in the regret bound for the noiseless case given in Theorem \ref{thm-noiseless}. This second term dominates the first term if convergence to equilibrium takes significant time which is the case in applications.

    \item \sloppy Theorem \ref{lower-bound} gives a lower bound of $\Omega(\log(T)/\Delta_a+\tau_c\Delta_a\log(1/\Delta_a))$ for equilibrium bandits. The first and second terms in this lower bound are obtained using the lower bounds for stochastic and noiseless equilibrium bandits, respectively, both of which are special cases of equilibrium bandits. Hence, UECB is order-wise optimal in $T$ and $\Delta_a$ while being optimal up to logarithmic factors in $\tau_c$.
    \item For each action, UECB only uses the rewards observed during the current epoch. It also uses only the latter half of that epoch. We chose the `half' fraction arbitrarily for simplicity and any other constant fraction yields the same order of magnitude dependencies. Another possible modification is to employ a weighted average over all the past rewards \cite{discounted_ucb}.
\end{itemize}

\section{Regret Analysis}
In this section, we explain our proof strategy by breaking the proof of Theorem \ref{thm-noisy} into lemmas. While the proof generally follows that of UCB \cite{bandit-book}, significant modifications are needed due to the converging nature of the rewards, and the epoch-based structure. In particular, UECB has to balance the tradeoff between the i.i.d. noise and the ``equilibrium noise'', which measures the distance to equilibrium. This tradeoff is unique to our problem.

The first lemma gives a probabilistic bound on $|\hat{x}_{a,n}-x_a^*|$ and motivates the definition of $UECB_{a,n}$. This gives a probabilistic upper bound on how far away our estimate of the equilibrium reward is. The difference between $\hat{x}_{a,n}$ and $x_a^*$ has two terms - one due to the i.i.d. noise and the other due to ``equilibrium noise''.
\begin{lemma}\label{lemma-part1}
The UECB algorithm maintains the following statements:
\begin{enumerate}[(a)]
    \item The inequality
    $$|\hat{x}_{a,n}-x_a^*|\leq\sqrt{\frac{4\sigma^2}{\ell_{n_a}}\log\left(\frac{2}{\delta_n}\right)}+ \frac{2}{\ell_{n_a}}\frac{L\exp(-\frac{1}{\tau_c}(1+\frac{\ell_{n_a}}{2}))}{1-\exp(-\frac{1}{\tau_c})},$$
    holds with probability of at least $1-\delta_n$.
    \item $UECB_{a,n}\geq x_a^*$ with probability of at least $1-\delta_n$.
    \item Define
    \begin{equation*}
        \ell^{(1)}_{a,n}\coloneqq \frac{64\sigma^2}{\Delta_a^2}\log\left(\frac{2}{\delta_n}\right)  \; \textrm{and} \;\;\;\; \ell^{(2)}_{a,n}\coloneqq 2\tau_c\log\left(\frac{8L}{\Delta_a}\right).
    \end{equation*}
    Then given that $\ell_{n_a}\geq \ell^{(1)}_{a,n}$ and $\ell_{n_a}\geq \ell^{(2)}_{a,n}$ we have $\hat{x}_{a,n}\leq x_a^*+\frac{\Delta_a}{2}$ with probability greater than $1-\delta_n$.
\end{enumerate}
\end{lemma}
Part (b) and (c) above are direct implications of part (a) and the definition of $UECB_{a,n}$. The conditions on $\ell_{n_a}$ in part (c) can be easily translated to conditions on $m_{a,n}$, i.e., the number of epochs during which $a$ has been played. Let the corresponding number of epochs be $m_{a,n}^{(1)}$ and $m_{a,n}^{(2)}$ respectively, i.e., $\ell^{(i)}_{a,n}=2\rho_2\exp\left(\rho_1m^{(i)}_{a,n}\right)$ for $i=1,2$. Here $m_{a,n}^{(1)}$ is the number of epochs required for the noise term to be sufficiently small, and $m_{a,n}^{(2)}$ is the number of epochs required for the second term, stemming from the convergence time, to be sufficiently small. A large $m_{a,n}$ implies that $a$ was played consecutively for a higher number of turns which reduces the error due to i.i.d. noise and brings the system closer to equilibrium.

The next lemma shows that with high probability, the $UECB$ algorithm identifies suboptimal arms given that they have been played for a sufficiently large number of times. It also gives an upper bound on the probability of playing a suboptimal action given that the action has been played a sufficient number of times.
\begin{lemma}\label{lemma-part2}
At the end of epoch $n$, if a suboptimal arm $a\neq a^*$ has been played for enough epochs, such that $\ell_{n_a}\geq \max\{\ell^{(1)}_{a,n},\ell^{(2)}_{a,n}\}$ (defined in Lemma \ref{lemma-part1}) then $UECB_{a,n}\leq UECB_{a^*,n}$ with probability at least $1-2\delta_n$. Therefore, under these conditions, the probability that the UECB algorithm plays action $a$ in the $(n+1)$\textsuperscript{th} epoch is bounded by:
$$P\left(a_{n+1}=a\;|\;\ell_{n_a}\geq \ell^{(1)}_{a,n}, \ell_{n_a}\geq \ell^{(2)}_{a,n}\right)\leq 2\delta_n.$$
\end{lemma}

Note that the expected instantaneous loss at time $t$ when action $a$ is taken can be split as follows:
\begin{equation}\label{random-split}
x^*_{a^*}-x_t = (x^*_{a^*}-x^*_{a})+(x^*_{a}-x_t).
\end{equation}
The first term in \eqref{random-split} is the difference in the rewards at equilibrium between the optimal arm and a suboptimal arm. We first bound the regret corresponding to the first term, which depends on the number of times an action is taken multiplied by the suboptimality gap. The next lemma bounds the expected number of times \textit{(and not epochs)} a suboptimal action is played.

\begin{lemma}\label{lemma-part3}
For any suboptimal arm $a\neq a^*$, the expected number of timesteps the UECB algorithm plays $a$ can be bounded as follows:
$$\mathbb{E}\left[\sum_{k=1}^n\ell_{k}I\{a_{k}=a\}\right]=\mathcal{O}\left(\frac{\sigma^2\log(t_n)}{\Delta_a^2}+\tau_c\log\left(\frac{1}{\Delta_a}\right)\right)$$
where $I\{\cdot\}$ is $1$ when $\{\cdot\}$ is true and $0$ otherwise (i.e., an indicator).
\end{lemma}

The second term in (\ref{random-split}) denotes the convergence error, or ``equilibrium noise''. The regret accumulated due to this term can be shown to be $\mathcal{O}(\tau_c)$ times the number of arm switches, which is in turn bounded by the number of epochs where suboptimal actions were played. The next lemma bounds this number.
\begin{lemma}\label{lemma-part4}
For any suboptimal arm $a\neq a^*$, the expected number of epochs the UECB algorithm chooses $a$ can be bounded as follows:
$$\mathbb{E}\left[\sum_{k=1}^nI\{a_{k}=a\}\right]= \mathcal{O}\left(\log\left(\frac{\sigma^2\log(t_n)}{\Delta_a^2}\right)+\log\left(\tau_c\log\left(\frac{1}{\Delta_a}\right)\right)\right).$$
\end{lemma}
This gives rise to the final term in the bound in Theorem \ref{thm-noisy}. Combining Lemma \ref{lemma-part3} with Lemma \ref{lemma-part4} gives us the following final lemma which gives a regret bound at the end of each epoch.

\begin{lemma}\label{lemma-part5}
Let $t_n$ denote the time at the end of epoch $n$, then the UECB algorithm maintains:
\begin{eqnarray*}
\mathbb{E}[R(t_n)]&=& \mathcal{O}\bigg(\sum_{a\in\mathcal{A},a\neq a^*}\frac{\sigma^2\log(t_n)}{\Delta_a}+\tau_c\log\left(\tau_c\log\left(\frac{1}{\Delta_a}\right)\right)\nonumber\\
    &&\;\;\;\;\;\;+\tau_c\log\left(\frac{\sigma^2\log(t_n)}{\Delta_a^2}\right)\bigg).
\end{eqnarray*}
\end{lemma}
The lemma above proves our UECB regret bound, but only at timesteps that are at the end of some epoch. To complete the proof of Theorem \ref{thm-noisy}, we just need to prove a similar regret bound for all $T$. To that end, define $\tau$ as the time at which the last epoch ended, i.e., the ongoing epoch started at $t=\tau+1$. Then we divide $[1,T]$ into two intervals: $[1,\tau]$ and $[\tau+1,T]$. The lemma above gives a bound for the regret accumulated during $[1, \tau]$. The result for Theorem \ref{thm-noisy} is obtained by showing that the regret achieved during $[\tau+1,T]$ is bounded by a constant times the regret achieved during $[1,\tau]$.

\section{Applications}\label{sec-appli}
In this section, we detail two real-life problems that the equilibrium bandits framework can model. The agent is a policymaker who learns the policy that maximizes the collective good. However, the impact of any policy cannot be seen instantly as society interacts in a game-theoretic manner and converges to an equilibrium.

\subsection{SIS Epidemic Model}\label{subsec-SIS}
We consider the Susceptible-Infectious-Susceptible (SIS) model of epidemics \cite{SIS1, SIS2} as the underlining unknown dynamics to be controlled by the decision-maker. In this model, a susceptible individual becomes infected with some probability after contacting an infected individual and remains infected for a random period of time. Once the individual recovers, they return to the susceptible class since the disease does not provide any long-lasting immunity. Examples for diseases that follow the SIS model are influenza, meningitis, and tuberculosis \cite{SIS3}.

Specifically, we consider the networked SIS model with a graph with $M$ nodes and a symmetric weighted adjacency matrix $A$. These nodes can represent communities, cities, or countries. The weight $A_{ij}$ in $A$ is the contact probability between nodes $i$ and $j$. Let $I(i,t)$ be the fraction of infected individuals in node $i$ at time $t$. Then the discretized differential equation is given by:
\begin{equation}\label{sis-eqn-main}
    I(i,t+\Delta t)=I(i,t)+\left(\beta(1-I(i,t))\sum_{j=1}^M A_{ij}I(j,t)-\gamma I(i,t)\right)\Delta t.
\end{equation}
Here $0<\Delta t\ll 1$ is the stepsize for discretization which is assumed to be sufficiently small. $\beta>0$ is the infection rate and $\gamma>0$ is the recovery rate. Let $I(t)$ be the $M$-dimensional vector with the elements $I(i,t)$, then equation (\ref{sis-eqn-main}) can be written as
\begin{eqnarray}\label{sis-eqn-vector}
    I(t+\Delta t)&=&h(\beta,\gamma,A,\Delta t; I(t))\nonumber\\
    &\coloneqq& I(t)+\left(\beta(\mathbbm{1}_M-\textrm{diag}(I(t)))AI(t)-\gamma I(t)\right)\Delta t,
\end{eqnarray}
where $\mathbbm{1}_M$ is the identity matrix of dimension $M$ and $\textrm{diag}(I(t))$ is a diagonal matrix with the elements of the vector $I(t)$.

The decision-maker is the government or the policy-maker. Examples of actions can be the enforcement of masks, advertisements to increase awareness, and different types of lockdowns, e.g., shutting down schools or offices. These actions change the contact patterns between individuals and the rate of infection, which dictate the adjacency matrix and infection rate for this action,  denoted by $A_a$ and $\beta_a$, respectively, for action $a$. $\Delta t$ is the stepsize for the discretization and is typically much smaller than the time step at which the infection rates or the rewards are actually observed, which can range from a few days to a few weeks.
Let $z_t=I(t)$ be the state of the system at time $t$. Then, for action $a$, $$z_{t+1}=g(a;z_t)=h^{(1/\Delta t)}(\beta_a, \gamma, A_a, \Delta t;z_t).$$ The cost function $-f(a;\cdot)$ can be a combination of the operation cost of the policies and the health damages due to the disease.

The government does not know the functions $f$ and $g$. In particular, it is unlikely that the government can estimate the matrix $A_a$ for each action in a large-scale setting. In addition, depending on the resolution, the government may or may not know the fraction of infected individuals in each node,  (e.g., neighborhoods as opposed to individuals). Fortunately, as explained next, UECB only requires weak bounds on the problem parameters to perform very well.

Let $\lambda^{max}_a$ be the maximal eigenvalue of $A_a$. We assume that $\beta_a\lambda^{max}_a>\gamma_a$ for all actions. This assumption implies that a non-zero stable equilibrium point exists for the iteration given by (\ref{sis-eqn-vector}) for all actions. Let this equilibrium be $I_a^*$. The zero vector (i.e., no infections) is an unstable equilibrium in this case. If this assumption were false, i.e., $\beta_a\lambda^{max}_a\leq\gamma_a$, then there exists only one equilibrium point given by the zero vector which is also stable. We only consider actions that satisfy our assumption because, unfortunately, there are often no policies that completely eradicate the epidemic. Next, we show that this $g$ satisfies Assumption \ref{assu-evolution}):

\begin{proposition}\label{sis-prop}
For each action $a$, let $I_a^*$ be the non-zero stable equilibrium of the iteration in (\ref{sis-eqn-vector}), where $\beta_a, \gamma$ and $A_a$ satisfy $\beta_a\lambda_a^{max}>\gamma$ for the maximal eigenvalue of $A_a$, $\lambda_a^{max}$. If $I(t)>0$, then
\begin{equation}\label{sis-bound}
    \|I(t+\Delta t)-I_a^*\|_1\leq \max_{1\leq i\leq M} \left(1-\beta_a\Delta t\sum_{j=1}^MA_{a_{ij}}I(j,t)\right)\|I(t)-I_a^*\|_1,
\end{equation}
where $\|\cdot\|_1$ denotes the $\ell_1$ norm.
\end{proposition}

We just need a bound on $\beta_{a}$ and on the elements of matrix $A_{a}$ for all actions $a$ to employ the bound in \eqref{sis-bound}. If the policymaker does not know the number of infections in each node (but only the ``reward''), then they just need a bound on the number of infections. This bound is easy to deduce due to our assumption that $\beta_a\lambda_a^{max}>\gamma$. Let $\beta_a\sum_{j=1}^MA_{a_{ij}}I(j,t)>\alpha$ for all nodes $i$ and all actions. Then,
\begin{align*}
   \|z_{t+1}-z_a^*\|_1&\leq  \left(\max_{1\leq i\leq M} \left(1-\beta_a\Delta t\sum_{j=1}^MA_{a_{ij}}I(j,t)\right)\right)^{1/\Delta t}\|z_t-z_a^*\|_1 \\
   &\lessapprox e^{-\alpha}\|z_t-z_a^*\|_1.
\end{align*}
Then $c(a;z)$ as defined in equation (\ref{assumption-2}) is approximately equal to $e^{-\alpha}$, which becomes accurate as $\Delta t$ goes to $0$. If the number of infections in each node is known, then better bounds can be obtained.

\subsection{Strongly Monotone Game}
Our second example is deterministic gradient-based learning in continuous games \cite{mazu}. The underlying system in this example is a set of $M$ players. Each player $i\in\mathcal{M}=\{1,\ldots,M\}$ has their own decision variable $z_{i,t}\in\mathcal{Z}_i\subset\mathbb{R}^d$ at time $t$. Here each $\mathcal{Z}_i$ is a convex and compact set. The state variable of the system is just a concatenation of all the decision variables, i.e., $z_t=(z_{1,t},\ldots,z_{M,t})\in\mathcal{Z}_1\times\ldots\times\mathcal{Z}_M\subset\mathbb{R}^{dM}$. Each player has their own utility function, which depends on the decision variables of all the players and is parameterized by the action taken by the agent. At time $t$, if the action taken by the agent is $a_t$, then the utility function for player $i$ is given by $u_i(a_t;z_t)$. Each player seeks to maximize their utility function and can only control their own decision variable. We assume that all players have access to the decision variables of each player and their own utility function, but not to the utility functions of other players. Then at each time $t$, each agent updates their decision variables as follows:
\begin{equation}\label{ex-2}
z_{i,t+1}=z_{i,t}+\alpha h_i(a_t;z_t),
\end{equation}
where $h_i(a;z_t)=\frac{\partial u_i(a;z_t)}{\partial z_{i,t}}$. We have considered a constant step-size $\alpha$ for simplicity; the following results can easily be generalized to a decreasing step-size. The agent observes noisy rewards based on its own reward function: $f(a_t;z_t)$.

We make certain assumptions on the underlying game to ensure that the assumptions for our bandit problem are satisfied. To that end, define $H(a;z)=(h_1(a;z),\ldots,h_M(a;z))$, i.e., the concatenation of all gradients. Then we assume that
$$\langle z'-z, H(a;z')-H(a;z)\rangle\leq -\lambda_a\|z'-z\|_2^2,\; \forall a\in\mathcal{A}.$$
This implies that for all actions, $\mathcal{G}_a=(\mathcal{M}, \{\mathcal{Z}_i\},\{u_i(a;\cdot)\})$ is a strongly monotone game with parameter $\lambda_a>0$. Then for each action $a$, $\mathcal{G}_a$ has a unique pure Nash equilibrium $z_a^*$ \cite{Rosen} which acts as the equilibrium corresponding to that action. For an action $a$, the Nash equilibrium $z^*$ is defined as the decision profile which satisfies: $u_i(a;z_i^*,z_{-i}^*)\geq u_i(a;z_i,z_{-i}^*)$ for all $z_i\in\mathcal{Z}_i$ and for all $i\in\mathcal{M}$. Note that $z_{-i}$ denotes the decision variables for all players except $i$. This ensures that part (a) of Assumption \ref{assu-evolution} is satisfied. We also assume that, for all $a$, $H(a,.)$ is $\beta_{a}$-Lipschitz continuous:
$$\|H(a;z)-H(a;z')\|_2\leq\beta_a\|z-z'\|_2. $$ The next proposition shows that such games satisfy Assumption \ref{assu-evolution}:
\begin{proposition}\label{prop:game}
The iterates given by (\ref{ex-2}), with action $a_t=a$, satisfy:
$$\|z_{t+1}-z_a^*\|_2\leq \sqrt{1-2\lambda_a\alpha+\alpha^2\beta_a^2}\|z_{t}-z_a^*\|_2.$$
\end{proposition}
For a sufficiently small step-size ($\alpha\leq \frac{2\lambda_a}{\beta_a^2}$), this proposition shows that the distance of the state $z_t$ from $z_a^*$ decreases when action $a$ is taken. A sufficiently small step-size can be avoided with a sequence of decreasing step-sizes, which instead would imply that the system satisfies Assumption \ref{assu-evolution} from some $t_0$ onwards.

As a concrete example, consider a resource allocation game \citep{Agrawal,bistritz2021online} where there are $d$ resources and each player's decision variable $z_i=(z_i^1,\ldots,z_i^d)$ denotes how much to use of each resource. The utility function of player $i$ depends on the value it assigns to each resource and the price of each resource. Here, action $a$ by the policymaker means that each player $i$ has access only to the subset of resources $\mathcal{R}_i(a)\subseteq\{1,\ldots,d\}$, so $z_i^{\ell}=0$ for $\ell\notin\mathcal{R}_i(a)$. The reward function for the agent is the sum of utilities of the players $f(a;z)=\sum_{i=1}^{M} u_i(a;z)$.

\begin{figure*}[!h]
\centering
\begin{subfigure}{.3\textwidth}
  \centering
  \includegraphics[width=0.99\linewidth]{New_SIS_noiseless.png}
  \caption{Noiseless}
  \label{fig:epi_noiseless}
\end{subfigure}%
\begin{subfigure}{.3\textwidth}
  \centering
  \includegraphics[width=.99\linewidth]{New_SIS_Noisy.png}
  \caption{Noisy}
  \label{fig:epi_noisy}
\end{subfigure}
\caption{Simulation plots for SIS epidemic control in the (a) noiseless, and (b) noisy cases}
\label{fig_epi}
\end{figure*}
\begin{figure*}[!h]
\centering
\begin{subfigure}{.3\textwidth}
  \centering
  \includegraphics[width=0.99\linewidth]{New_Game_noiseless.png}
  \caption{Noiseless}
  \label{fig:game_noiseless}
\end{subfigure}%
\begin{subfigure}{.3\textwidth}
  \centering
  \includegraphics[width=.99\linewidth]{New_Game_noisy.png}
  \caption{Noisy}
  \label{fig:game_noisy}
\end{subfigure}
\caption{Simulation plots for resource allocation game in the (a) noiseless, and (b) noisy cases.}
\label{fig_game}
\end{figure*}

\section{Simulations}
In this section, we simulate the applications from Section \ref{sec-appli}. Each curve is the average of 100 random realizations, and has been plotted along with the standard deviation region. The randomness in the noiseless case stems from the random initializations.

\subsection{SIS Epidemic}

We simulate a system with $K=4$ actions and $M=10$ nodes. For each action $a$, we generate a random sparse symmetrical matrix $A_{a}$. We use $\gamma=0.01$  for all actions. The values for $\beta_{a}$ for the $4$ actions are $0.011,0.012, 0.013$ and $0.014$ respectively \cite{SIS3}. We use the \textbf{cost} function $f(a;z)\coloneqq w_{0,a}+w_a^Tz,$, where $w_a\in (0,1]^{M}$ is the health cost vector and $w_{0,a}\in(0,1]$ is the operational cost which only depends on the action. Clearly, $f$ is  Lipschitz with $L=1$.

To implement UECB, we only assume that $\beta_a>0.05$ for all actions and that the sum of each row of $A$ is at least $1$, while the actual values are unknown and in $[3,5]$. Additionally, the infected fraction for each node is unknown and only the cost is known.

Figure \ref{fig:epi_noiseless} compares the performance of UECB for the noiseless case with a naive algorithm where each action is played for $t_{try}$ consecutive timesteps in the beginning (see subsection \ref{subsec:noiseless}). Then,  the naive algorithm plays the arm that had the best reward at the end of its epoch for the rest of the timesteps. As expected, our UECB algorithm outperforms the naive algorithm for both small and large $t_{try}$. For small $t_{try}$, we do not give enough time for the system to converge which then commits to a suboptimal action, yielding linear regret. For large $t_{try}$, the system gets close to equilibrium for each action, but wastes time on suboptimal actions. This gives an $O(1)$ regret, but it is still worse than that of UECB. For the noisy case, Figure \ref{fig:epi_noisy} shows that UECB achieves sublinear regret over time while UCB, EXP3 and REXP3 \cite{besbes2014stochastic} do not.

\subsection{Resource Allocation Game}
We consider electricity grids as our resource allocation game \cite{Deng_DSM,MA_DSM}, with $d=10$ resources and $M=1000$ players. The utility function for each player $i$ is defined as
$$u_i(z)=\sum_{\ell=1}^d \left(\gamma_{i,\ell}\log(1+z_i^{\ell})-\zeta_{i,\ell}z_i^{\ell}s_{\ell}\right),$$
where $s_{\ell}=\sum_{i=1}^M z_i^{\ell}$. $\gamma_{i,\ell}$ and $\zeta_{i,\ell}$ are chosen uniformly at random in $[0.8,1]$. It can be easily verified that this function satisfies our assumptions. For each action, the subsets of resources that can be chosen by each player are generated randomly.

We again assume very little knowledge about the system. The agent knows $\alpha$, but only uses bounds on $\lambda_a$ and $\beta_a$. Here, we assume that the agent can observe the current state, since monitoring which player picked what resource is natural in practice. The results, given in Figure \ref{fig_game}, are similar to those from the SIS epidemic scenario.

\section{Conclusions}\label{sec-conc}
In this paper, we presented equilibrium bandits, a new bandit problem, designed to deal with systems that converge to equilibrium over time. The agent can control some parameters of this system that dictate the resulting equilibrium. While the agent only observes the real-time impact of their actions, their aim is to find the set of parameters that give the best performance at equilibrium.

We proposed Upper Equilibrium Concentration Bound (UECB), a new algorithm for equilibrium bandits that assumes very little about the system. The key innovation of UECB is the use of `convergence bounds' which bound how far the system is from the equilibrium at any given point. We proved regret bounds for UECB which are optimal up to logarithmic factors. We showed that two applications, epidemic control and resource allocation games, fall under the framework of equilibrium bandits. We simulated UECB to confirm the theoretical performance guarantees for these applications.

By introducing a new bandit model, our work opens up many new research avenues. An important extension is to be able to learn the evolution system parameters (i.e., $\tau_c$ and $L$) on the fly, instead of using fixed worst-case bounds on these parameters. Another significant extension is to systems that evolve stochastically, which would allow the equilibrium bandits framework to include reinforcement learning algorithms and stochastic gradient-based games.

redpajama_set_name: RedPajamaArXiv


text:
Karmos Viktor (Rákosliget, 1925. január 15. – Budapest, 2013. április 13.) textilmérnök, a volt Habselyem Kötöttárugyár műszaki igazgatója, vezérigazgató-helyettese.

Életútja, munkássága

A Kemény Zsigmond Gimnáziumban érettségizett, majd a Budapesti Műszaki Egyetemen 1950-ben szerzett textil-gépészmérnöki diplomát, abban az évfolyamban, amely a legelső volt a Zilahi Márton vezette Textiltechnológia Tanszék létrejöttét követően.
Első munkahelye a Békéscsabai Kötöttárugyár volt, ahol egy évet töltött üzemmérnöki munkakörben, majd 1951-ben az iparvezetés kinevezte a Selyem- és Gyapjúárugyár (a későbbi Habselyem Kötöttárugyár) főmérnökévé. Itt dolgozott mint főmérnök, majd műszaki igazgató és vezérigazgató-helyettes 1985-ig, nyugállományba vonulásáig.

Ez alatt az idő alatt a gyár hatalmasat fejlődött: a 400 fős kis gyárból 5000 főt foglalkoztató, több telephelyes nagyvállalat lett. Karmos Viktor érdeme, hogy hazánkban vállalatánál honosították meg az 1960–70-es években nagyon korszerűnek számító és népszerű nyloning- és nylonfehérnemű-gyártást, itt helyeztek üzembe Magyarországon először percenként 1000 szemsort kötő lánchurkológépeket és gazdag csipkedíszítések készítésére alkalmas korszerű csipke-raschelgépeket, itt működtek először az országban nagyon modern Fukuhara (Monarch) körkötőgépek. Fiatal munkatársai sokat köszönhettek neki, mert vezetőként nagy segítséget nyújtott ahhoz, hogy pályafutásuk minél sikeresebb lehessen.

Rendszeresen részt vett a Könnyűipari Minisztérium, az Országos Műszaki Fejlesztési Bizottság és más szervek által létrehozott különböző szakmai bizottságok munkájában.

Nyugdíjasként sem szakadt el a kötőipartól: a könnyűipari gépek kereskedelmével foglalkozó Wilhelm Budapest Kft. keretében jelentős szerepe volt abban, hogy a számos új és régi kötő- és konfekcióipari vállalkozás korszerű kötő- ill. varrógépekkel gazdagodott.

Munkáját számos kitüntetés ismerte el: több kormánykitüntetésben és szakmai kitüntetésben részesült. 1974-ben az első között nyerte el az Eötvös Loránd-díjat. Magas színvonalú mérnöki munkásságát a Budapesti Műszaki Egyetem 2001-ben arany diplomával ismerte el.

A Textilipari Műszaki és Tudományos Egyesületnek 1951 óta volt tagja. Évtizedeken át tagja volt a kötő szakosztály vezetőségének és tevékeny munkát végzett a Magyar Textiltechnika, valamint a Kötőipari Szemle rovatszerkesztőjeként is.  A TMTE kötő szakosztályának több cikluson át volt megválasztott küldötte a TMTE küldött-közgyűlésein. Különös érdeme volt, hogy évtizedeken át nagy szorgalommal gyűjtötte a hazai textil- és ruhaiparral összefüggő, a sajtóban megjelenő személyi híreket (kitüntetések, munkahely-változások, halálesetek), ezeket a Magyar Textiltechnikában rendszeresen publikálta is. Ez a gyűjteménye igen jó szolgálatot tett, amikor a TMTE megalakulásának 60. évfordulójára összeállították – és azóta is folyamatosan kiegészítik –  a Szellemi Örökségünk albumot, amelynek anyagát jelentős részben Karmos Viktor gyűjteménye alapozta meg.

2013-ban hunyt el.

Forrás

 Szellemi Örökségünk album. Textilipari Műszaki és Tudományos Egyesület, Budapest

Magyar mérnökök
Magyar textilmérnökök
Textilipar
Rákosmentiek
Eötvös Loránd-díjasok
1925-ben született személyek
2013-ban elhunyt személyek

redpajama_set_name: RedPajamaWikipedia


text:
Isle aux Morts är en ort i Kanada.   Den ligger i provinsen Newfoundland och Labrador, i den östra delen av landet,  km öster om huvudstaden Ottawa. Antalet invånare är .

Terrängen runt Isle aux Morts är huvudsakligen platt, men åt nordost är den kuperad. Havet är nära Isle aux Morts åt sydost. Den högsta punkten i närheten är  meter över havet,  km nordväst om Isle aux Morts. Trakten är glest befolkad. Närmaste större samhälle är Channel-Port aux Basques,  km väster om Isle aux Morts.

Inlandsklimat råder i trakten. Årsmedeltemperaturen i trakten är  °C. Den varmaste månaden är augusti, då medeltemperaturen är  °C, och den kallaste är februari, med  °C. Genomsnittlig årsnederbörd är  millimeter. Den regnigaste månaden är december, med i genomsnitt  mm nederbörd, och den torraste är maj, med  mm nederbörd.

Kommentarer

Källor

Orter i Newfoundland och Labrador


Ehrenreich is referring here to the fancy French lunch she shared with the editor of Harper's—the lunch at which the concept of conducting an undercover investigation of the low-wage workplace was first proposed. What is significant about the line—apart from the fact that it opens the book—is its deadpan quality, its casual division of "circumstances". Ehrenreich begins in one world and journeys to a vastly different one. She and Lewis Lapham occupy a rarefied realm, one of "salmon and field greens" where various ideas are pitched regarding one of Ehrenreich's "more familiar themes—poverty." Ehrenreich does not evade what is fundamentally problematic about her posing as a member of America's bottom class, but instead underlines it, draws attention to it, milks the discrepancy for all the irony it is worth.
Colleen, one of the employees at The Maids, betrays what strikes Ehrenreich—and, by extension, the reader—as a sad complacency. Browbeaten by Ted and rewarded only meagerly for her exhausting work, Colleen shrugs and implies that it's only fair—in other words, such is her lot. Ehrenreich cannot accept that premise, and neither can the reader. Where is the justified rage of the low-wage workforce? Ehrenreich suggests that those sorts of feelings have been slowly drained by the nature of the low-wage workplace. Jobs so tiring, so demeaning, so taxing, leave little energy for fighting back.
In this passage, Ehrenreich describes the way the low-wage workplace crushes the spirit in broad brushtrokes, with nearly cartoonish imagery. One imagines a looming employer holding out a carrot, wielding a heavy stick behind his back. The imagery echoes turn-of-the-century political cartoons—when whole countries could be represented by crying babies, warfare rendered a tussle in the sandbox, abstract notions like prosperity cast as concrete objects. Indeed, Ehrenreich has something of the traditional muckraker in her, and her journalism indulges in the occasional flash of sensationalism. In more ways than one, hers is an avowedly "old-fashioned" approach.
If Ehrenreich's style harkens back to turn-of-the-century agitation, the persona she fashions for herself in Nickel and Dimed is pure 70's New Journalism. Here, the larger thesis of the book is temporarily eclipsed by a confession of fear—fear of the "darker" reaches of the country—and that confession is communicated in a distinctively comical tone. It's almost a Woody Allen moment, Ehrenreich poking fun at her own trepidation, deflating any association her persona might have with "heroic" journalism. She's not a daredevil, and she is sure to remind us. At the same time, while Ehrenreich herself is the true subject of the passage, one cannot help but note the reference to other parts of the U.S.—states just as "American" and therefore, one would think, democratic as, say, Maine or Florida—as though they were vestiges of the Dark Ages. There's a sense of scorn there, but also defeatism—no point bothering with Idaho or Louisiana, as they're already Third World anyway.
Ehrenreich is watching Survivor and catching herself in her own incredulity at it. What is interesting here is Ehrenreich tacitly compares herself to a Survivor contestant. She's struggling to survive in a harsh landscape, but with a safety net always ready to catch her should she fall; what is more, her travails are being recorded for the entertainment of the masses. What makes her better than a game-show participant? Sure, there are social issues at play, a political message to communicate, but Ehrenreich seems here to worry about the validity (artistic or moral) of her enterprise.
"See, I am the vacuum cleaner."
Ehrenreich is watching what she describes as a "disturbing" training video on vacuuming for The Maids. The moment is shot through with absurdist comedy—except it's all real. A man in the video is demonstrating a special vacuum contraption that hooks onto the body; everything is presented as though to third-graders, with the fake smile that is ubiquitous in these environments; and Ehrenreich and her fellow trainees are forced to sit and watch, silently taking in the lessons like automatons. The vacuum contraption renders the employee machine-like—he or she is the vacuum, and, by implication, nothing more (certainly nothing human)—and the training as a whole seems like nothing so much as a brainwashing session. The result is a workforce of robots—made to feel as such and treated as such.
Ehrenreich opens the door in this passage to what could be an entirely different book. Nickel and Dimed is very much a class-oriented treatise; race rarely plays a large role in it. Yet is Ehrenreich's disguise akin to "passing"? Or to donning blackface? In other words, is her hop-scotching from class to class analogous in any way to moving from race to race? On the surface, the answer would seem to be no, as America prides itself on social mobility. Class is fluid, race is fixed. But Ehrenreich argues that class is far more fixed than we think it is, and that being a maid is akin to being an oppressed minority. At the end of the day, perhaps it all boils down to the roles society requires us to play.
In clear and cogent terms, Ehrenreich outlines the tyranny of the low-wage workplace. It is often prohibitively difficult to change jobs, as each new job means such a production. As a result, low-wage employees are left defenseless; they can complain, but at the end of the day they need to hold their jobs. They cannot walk out in a huff, as the consequences are too dire. One wonders if it is all a conscious attempt on the part of employers to keep employees in line, to tamp down mobility; if so, it is the mendacity of it, the way in which employers hide their true intentions, that is the most galling thing of all.
It's a rhetorical question. Throughout Nickel and Dimed, Ehrenreich continually suggests that low-wage employees lose much of their willpower and fighting spirit in the exhaustion and quiet brutality of the workplace. Ehrenreich does not write of steel mills, of employers beating incalcitrant workers, of starvation and widespread disease. She writes instead of hours so long that there is nothing to do in between work shifts but sleep; of work so repetitive and tiring that one must fight to stay alert; of a culture of debasement. The vision is in its own way as depressing as Dickens'.
Back to New Journalism: Ehrenreich herself is the subject here. Yet the issue at stake is very much a social one, and very much related to Ehrenreich's larger thesis. The low-wage workplace is stripping her of her humanism, rendering her a more hostile and short-tempered person, more judgmental, quicker to take offense. It may have done the same to her coworker. Ehrenreich's own performance has begun to affect her; the act bleeds back into reality, and the boundaries are blurred. If "this is not me", who is it? The question may seem formalist, but in this case what it concerns is the very humanity of the workplace. How does one stay "good" in a cruel world? Do adverse circumstances bring out the best in us—or the worst?
Chazelle, Damien. Chainani, Soman ed. "Nickel and Dimed: On (Not) Getting By in America Quotes and Analysis". GradeSaver, 28 July 2009 Web.


I am an on location wedding and portraiture photographer based in Webster, TX serving the greater Houston area (and willing to travel!) I specialize in capturing love + life in its authentic, emotional glory. I would describe my editing style as rich, romantic, intimate. I have a big personality and laugh at everything. My favorite thing is to come away from sessions with clients and hear them tell me how I didn't make them feel nervous or awkward at all! I have a lot of fun during my shoots, so be prepared to say silly things and for me to make very bad jokes!
I have been a creative for as long as I can remember. Between having drawing contests with my sister when I was ten, teaching myself Adobe Photoshop 7.0 when I was twelve and dabbling in art journaling, digital drawing, graphic design and scrap booking. I have always been drawn to photography but didn't become serious about it until High School, when I joined the Yearbook and Newspaper staff and began learning to master photojournalism. I was mentored by professional freelance and photojournalism photographer Mike McLean and have personally spent the last nine years learning and refining my passion.
Lets get personal! I am 23 years old and a fraternal twin. I am obsessed with my flame point Siamese cat Wilson. When I'm not photographing, I'm binge watching Grey's Anatomy or playing Animal Crossing.
If you think we could make some awesome work together, send me a message at nikki@clicknikki.com!


text:
This website uses cookies to help us know a little bit about you, how you use our website and improve the browsing experience. You can find out what cookies we use, what they do and how you can disable them in our Cookie Policy. By browsing this website and closing this message, you consent to the use of cookies in accordance with our Cookie Policy unless you have disabled them.
Skip Navigation - go to content
Footer with further Navigational links
BPL - Bio Products Laboratory
About BPL Group
Welcome to the BPL Group website
At BPL, we are committed to providing a continuous supply of reliable, high quality plasma-derived products worldwide, supporting both healthcare professionals and patients every day.
About BPL
BPL Plasma
Therapeutics US
We are a UK headquartered Group which delivers life-saving treatments around the world for conditions related to immune deficiencies, bleeding disorders and critical care. These treatments are produced through complex transformation of plasma, which is collected from US donors in a number of US-based collection facilities.
Reporting adverse events
© Bio Products Laboratory Ltd. 2020, Registered Office: Dagger Lane, Elstree, Hertfordshire, WD6 3BX, United Kingdom.
Tel: +44 (0)20 8957 2200 Fax: +44 (0)20 8957 2601 Email: info@bpl.co.uk Website: www.bplgroup.com
Registered in England and Wales, No. 7343036
UK/OTH/1117/0005 (1) Date of preparation: Jul 2018


SM launched its 60th Anniversary celebrations by lighting up the iconic Mall of Asia globe, and with it, starting off a series of exciting 60 themed visual attractions, fun experiences, and interactive activities for its customers from May to October in SM malls nationwide.
The SM story began in 1958 when founder Henry Sy, Sr. opened the first Shoemart store in downtown Manila, introducing exciting new merchandising and retailing concepts. At that time, Mr. Sy, who has been honored as the Father of Philippine Retail by the Philippine Retailers Association, thought that if he could sell a pair of shoes to every Filipino, he could be a successful man.
In the 1960s, Shoemart expanded its shoe store chain and became one of the pioneers in the new urban centers in Makati and Cubao.
In the seventies, Shoemart evolved into a department store, becoming SM, which has been synonymous with one-stop shopping excitement. SM positioned itself for growth in the 1980s, expanding its department store chain and venturing in the supermarket and appliance business.
1985 saw the opening of the first SM shopping center, SM City North EDSA. It was a time of a major political and economic crisis, but the mall was an instant success, paving way for the so-called malling phenomenon in the Philippines. Malls became part of the Filipino lifestyle, places where families and friends could shop together, dine together, and have fun together in world class surroundings.
Today, SM has close to a thousand retail stores, and 70 malls in the Philippines, and 7 malls in China.
Through its property arm, SM Prime Holdings, SM has also become a leading developer of residential condominiums in Metro Manila, where residents could live like a star at affordable prices.
Mr. Sy always believed in tourism as a catalyst, and with that, built destination malls like the SM Mall of Asia. Through SM Hotels and Conventions, the company now owns and operates world class hotels, a state of the art arena, convention and exhibition centers, and premier beach and mountain resorts.
At the ceremonial toast of SM's 60th Anniversary launch. From left SM Chairman for Executive Committee Hans T. Sy, SM Retail Vice Chair Herbert Sy, SM Investments Vice Chair Tessie Sy Coson, and SM Hotels and Conventions Corp. President Elizabeth Sy.
In the finance area, the acquisition of Banco de Oro in the 1980s enabled SM to provide better cash management services to its suppliers and retail banking services to its customers. Today, as the country's largest bank in terms of resources, deposits, and loans, BDO has made finance inclusive for millions of Filipinos.
Through SM Foundation, SM reaches out to communities with its educational, health, and livelihood training programs aimed at making farmers self-sufficient. Through SM Cares, SM has undertaken environmental programs by building green spaces in its malls, as well as sustainability efforts, trash to cash recycling, and disaster risk reduction programs. It has also make going around the malls easier for those with difficulties through PWD program.
Marking the 60th Anniversary of SM in the province of Bulacan, SM Malls in Baliwag, Marilao, Pulilan, and San Jose del Monte celebrates the occasion with exciting activities and fantastic treats for its patrons. Shoppers can indulge in shopping fun and sumptuous meals as 60% discount or pay less Php 60 promos are applied to selected items with #AweSM Deals. Dazzling entertainment will also thrill the shoppers with #AweSMDance.
SM City Baliwag will also stir up the milestone happenings with the Iconic 60 live painting of Guhit Pinas, Bulacan on May 25. Spot the hidden freebies and discounts from partner stores using the WallaMe App and win exciting prizes at SM City San Jose del Monte.


large laundry cabinet laundry room cabinet ideas large shelves and free standing laundry room storage ideas laundry room storage.
large laundry cabinet luxury laundry room laundry room vanity cabinet laundry room vanity luxury laundry room vanity large size of laundry room vanity laundry sink and home.
large laundry cabinet laundry cabinets with sink bathrooms amazing large utility sink laundry cabinets utility sink cabinet full size laundry sinks with cabinets laundry utility.
large laundry cabinet large size of laundry room vanity sink cabinet wonderful cabinets bathroom ideas basin bat bathroom.
large laundry cabinet laundry room sink cabinet garage sink laundry room sink cabinet basement sink freestanding utility sink large laundry room sink cabinet.
large laundry cabinet bath sink cabinet interior decor ideas marvelous laundry vanity large size of home depot bathroom cabinets.
large laundry cabinet medium size of decorating laundry room storage options between washer dryer storage cabinet washer dryer storage.
large laundry cabinet designs modern laundry room with wall mounted cabinet contemporary laundry room with painted.
large laundry cabinet laundry room base cabinets large size of room sink base cabinet together with laundry room cabinet laundry room base cabinets.
large laundry cabinet narrow laundry sink large size of trough and cabinet narrow utility sink plastic utility sink stationary narrow ceramic laundry sink.
large laundry cabinet designs modern laundry room with wall mounted cabinet classic white laundry room with.
large laundry cabinet stainless steel laundry room sink utility room sink with cabinet large size of faucets and room.