text:
Partial opening of commercial crab fishery….
F/V Eclipes unloading crab last season in Newport.
Ken Gagne photo
State opens a portion of the coast for commercial Dungeness crabbing, extends recreational opening
NEWPORT – The Oregon Department of Fish and Wildlife (ODFW) and the Oregon Department of Agriculture (ODA) has announced the opening of the commercial crab season from Cape Blanco (just north of Port Orford) to the OR/CA border is set for Dec. 18.
"We have consistently taken a very precautionary approach when opening our crab fisheries," said Caren Braby, ODFW Marine Resources Program Manager. " Recent test results have consistently shown low biotoxin results on the southern end of the state and decreasing levels in ports north of this area indicating they are of excellent quality, safe for consumption and ready for harvest."
The two agencies also announced the immediate opening of the recreational bay and ocean crab fishery from Tillamook Head (just south of Seaside) south to Cape Lookout (just south of Netarts Bay) effective Dec 10. Recent test results met the criteria to remove the shellfish health advisory in this area. All recreational harvest from Tillamook Head north to, and including, the Columbia River remains open. All recreational harvest of Dungeness crab from Cape Lookout to Cape Blanco remains closed due to elevated levels of domoic acid.
"We are excited to be able to open up another section of the coast for recreational crabbing as we see biotoxin levels decrease and stay below alert levels, Braby said. "We hope this trend will continue and allow us to open the remaining areas for recreational crabbing and commercial harvest soon."
Opening of the commercial Dungeness crab season along the other areas of the Oregon coast is still delayed due to concerns about domoic acids levels in the central section of the coast. ODFW will continue to work closely with ODA and the Oregon commercial Dungeness crab industry to test crab in central section of the coast. In close coordination with ODA, fishery managers from Washington and California and the Oregon commercial Dungeness crab industry, ODFW plans to evaluate options for opening the commercial season once additional domoic acid test results are available.
It is recommended that crab always be eviscerated prior to cooking. Evisceration includes removing and discarding the internal organs and gills.
Despite the delay, crab and shellfish products sold in retail markets and restaurants remain safe for consumers.
Domoic acid or amnesic shellfish toxin can cause minor to severe illness and even death. Severe poisoning can result in dizziness, headaches, vomiting and diarrhea. More severe cases can result in memory loss and death. Shellfish toxins are produced by algae and originate in the ocean. Toxins cannot be removed by cooking, freezing or any other treatment. For more information on toxin closures, call ODA's shellfish safety information hotline at (800) 448‐2474 or visit the
ODA shellfish closures web page at:
http://www.oregon.gov/ODA/programs/FoodSafety/Shellfish/Pages/ShellfishClosures.aspx.


The purpose of this is to neutralize any pre-existing odors from detergents and to remove any residue from manufacturing that could react with the activated oxygen. Use the pre-wash in your sink with water, like you would with ordinary dish soap.
Washing your CPAP equipment: Use 1/4 of the included pre-wash bottle with a gallon of water to wash your CPAP equipment (mask, hose, and reservoir), and then rinse thoroughly.


text:
Browse: Naxos American Classics - up to 25% off, Schoenfield (composer)
Offer, Schoenfield - Camp Songs & Ghetto Songs
World Première Recordings
Angela Niederloh (mezzo-soprano), Erich Parce (baritone), Paul Schoenfield (piano) & Morgan Smith (baritone)
Music of Remembrance, Gerard Schwarz
Release Date: 26th Oct 2009
Usually despatched within 1 working day
Offer, Paul Schoenfield - Refractions
James Tocco (piano), Yehuda Hanani (cello), Alexander Fiterstein (clarinet) & Paul Schoenfield (piano)
Release Date: 26th Apr 2010
Offer, American Classics - Paul Schoenfield
Robert Vernon (viola)
Rundfunk-Sinfonieorchester Berlin, BBC Singers, University of Michigan Opera Chorus and Orchestra, Yoel Levi, Avner Itai, Kenneth Kiesler
Release Date: 1st Aug 2005
American Classics - Introducing the World of American Jewish Music
Rare, new and rediscovered American Classics
Release Date: 1st Nov 2004
Offer, American Classics - Klezmer Concertos and Encores
David Krakauer (clarinet and bass-clarinet), Scott Goff (flute), Alberto Mizrahi (tenor), Alicia Svigals (violin), Martha Mooke (electric viola), Pablo Aslan (contrabass)
Barcelona Symphony, National Orchestra of Catalonia, Seattle Symphony Orchestra, Rundfunk-Sinfonieorchester Berlin, Gerard...

redpajama_set_name: RedPajamaCommonCrawl


It is ideally positioned for any of the 3 peaks and the wonderful countryside.
Made the short break seem special from the start.
House was lovely, spacious, clean with everything we needed. Would love to return in the summer, surrounding scenery was beautiful. Later check out time would have been nice, all a bit of a rush on a Sunday morning & cut the weekend short . Otherwise all good, thank you!
It offered warmth and comfort making it ideal for a winter break - with plenty room for storage of wet weather gear. The two storey central feature dining room was a plus as was the two bathrooms and bedroom mix that offered a fine degree of separation at bedtime when occupied by a group. In terms of recommendations perhaps- 1 Some comfortable seating in the dining area would help families to have slightly different space at certain times of the day. 2 Vehicular access and egress is a little tight - particularly for large cars. I would add however that neither of these items spoiled our enjoyment.
In a good location for walking, cycling and sightseeing. Very good village pub within easy walking distance. Carpets could do with updating.
The property is quirky and very well equipped. The information provided both on the Sykes' site and at the house as accurate and useful. Beds were comfortable, everything was spotless, we loved the location (there is road noise during the week from about 5.45am but it didn't worry me and I've never lived anywhere noisy and am not used to it), were lucky enough to see salmon leaping the Foss, enjoyed the food at the local pub and all in all, had a great time. We varied in number from 4 to 8 and there was plenty of space for people to do their own thing. Having somewhere comfortable to sit downstairs would make this even better.
Carpets were a little stained, bulbs were not working, extractor fans did not work in either bathroom, cobwebs lots of small things that we had not come across in other cottages we have stayed at. The situation of the cottage was amazing surrounded by the beautiful Yorkshire countryside, many lovely walks right on your doorstep. The local pub serviced good quality food with a welcome smile but again a little disappointed that the nearest shop was over 3 miles away in Settle when advertised it was less than a mile away. Overall our stay was lovely just a few small things could have made it better.
A lovely big upstairs lounge with plenty of seating and the jacuzzi bath is an added bonus, just the thing after being out walking all day. Stainforth an ideal base for walking and touring. would definitely stay there again.
It was also extremely well equipped in the kitchen and we had everything we required. (Only thing lacking was a larger cafetière, or better still a coffee maker!) The only negative comment ( as a party of 6 adults) would be one of the mattresses in the twin room, but for children it would have been absolutely fine. Big bonus was location for amazing walks and accessibility to pub within walking distance. We loved it and may very well return, either as the same group or with our family. Great holiday let with much to offer.
The barn is very spacious and extremely well equipped making for a very enjoyable and relaxing stay. Lovely quiet village with friendly pub and scenic walks. Perfect!
Very comfortable and had all we needed. One tip - the 4th bedroom is a sofabed in the snug room that has a skylight in it. During the lighter days the room fills up with light very early in the morning which can disturb sleep. If there was a facility of shutting of the light with the blind it would make the cottage perfect.
Beds were comfy and loved the quirky make up of the barn. Great for our family week including 10 to 81 year olds.
If you want a relaxing few days with stunning scenery and a beautiful clean cottage then visit here.
Access to more fire wood and coal would have ensured the property was kept warm as the central heating is not adequate enough for the size of the building and the starter pack of coal and logs only lasted 1 night on a cold and wet weekend in March. I would recommend this property for location and the fantastic food served at the local pub - but only during warmer weather.
This was the second time we have stopped here this year and once again it was brilliant we were a family party of 8 and we planned to have an early Christmas together as due to circumstances we wouldn't be together on the day itself. It was the perfect 5 days to chill out, have a lovely Christmas dinner and a couple of trips to the Craven Heifer (which is under new management and is a 100% better!). The facilities are great, there is ample room for 8 people and there is everything you could wish for in the kitchen. We will definitely be booking again for the new year as we felt so comfortable and relaxed that it was a struggle to go home!
An ideal location for peace & quiet. The village pub is not too far away & the food is lovely.
Stayed for a week, very well equipped place with lovely views, this holiday let stands on its own next to a main road, not busy and very quiet at night, the church across the road chimes on the hour from about 8am but it wasn't annoying, inside is spectacular, 2 stairways, 1 that gave it it's name is from Haworth Church where the Bronte sisters father served, there is a large kitchen also well equipped, there are 3 bedrooms but also a pull out bed under the stairs which can be sectioned off and private, the bed I stayed in was quite soft, I prefer a harder mattress, but all in all a very nice place and well worth the visit.
Beautiful view from first floor lounge large window. Great value for 8 people.
This stone-built, detached barn conversion is set in its own grounds, near the beautiful river on the outskirts of the peaceful village of Stainforth, three miles from Settle.
An unusual and quite spectacular layout, the main sitting room sits high amongst the ceiling beams on a first floor mezzanine overlooking the ground floor.
Accessed by two staircases the room is spacious, extending across most of the first floor and features a central chimney stack containing a woodburner.
Full of character, the cottage combines lovely soft furnishings, a picture window overlooking the Dales countryside, stone walls and a magnificent cathedral ceiling to ensure that this is a perfect place to relax.
The bathroom with Jacuzzi bath and the king-size double bedroom, both boasting lovely views, are also contained on the first floor.
The ground floor has a further two bedrooms served by a shower room, a small kitchen/diner, spacious dining room, summer room, utility and internal access to the garage.
The ground floor snug contains a sofa bed and access to the first floor via an original staircase from Haworth Church, which gave the cottage its name.
This is a fantastic location for family and friends to stay with a lovely local pub a short walk away.
The river and access to many footpaths, including the Pennine Way are very close by and a short drive will enable the more adventurous in the group to attempt the famous and gruelling Three Peaks challenge, a test of stamina and determination.
For a more relaxing day out, there are many sightseeing activities in the area; visit the Hoffman Kiln at Langcliffe, the famous Ribblehead Viaduct and the market towns of Settle and Skipton.
A great location for a holiday in the Yorkshire Dales National Park that will live long in the memory.
Note: This property is next to a road, therefore some associated noise will be heard.


As part of our unique therapist led in home and community support model, our community support workers (CSW's) work in partnership with our Allied Health, Behaviour therapists and families to ensure we deliver the effective and high quality supports to our clients. We are privileged work as a strong multidisciplinary team, who share information and have regular case discussions so that clients are provided holistic intervention and supports.
As each child is unique, in order to provide community supports to families, our clients must first work with our therapy team, to provide an understanding of your child's strengths and abilities so that we can provide a program that results in improvements in quality of life, acquisition of valued skills, and increased community and social inclusion.
Our team will work with you and your child, to ensure we provide the best outcomes in community support.


text:
Empowering youth to build an Olympic future
By Thomas Bach(chinadaily.com.cn)
Zhu Zhenyu, flagbearer of China, holds Chinese national flag at the opening ceremony for the Lillehammer 2016 Winter Youth Olympic Games in Lillehammer, Norway on Feb.12, 2016. [Photo/Xinhua]
As a former athlete, I remember the excitement that accompanied the first nomination to represent my country. Many of the young athletes that will participate in the Winter Youth Olympic Games in Lillehammer, Norway, this week will also share this special moment when the sport they love changes their lives. Hopefully, this experience will be a springboard for many of them for future success in their sport and their life. The Youth Olympic Games connect young athletes to Olympic Values of excellence, friendship, respect and fair-play – values that will serve them well both in their sports and their future careers.
This is exactly what makes the Youth Olympic Games unique. They go beyond the sporting competitions, not only by making the Games an unforgettable learning experience for the universal values of sport but also by building a human legacy. The idea is to engage young people from around the world and inspire them to play an active role in their communities.
Using the power of sport to have a positive impact on his community is what motivated China's Han Yan: He is one of nearly 250 Young Ambassadors and other young leaders from youth programmes associated with the IOC and the Winter Youth Olympic Games Lillehammer 2016. Yan participated at the first edition of the Winter Youth Olympic Games in Innsbruck in 2012 before going on to compete at the Sochi Olympic Winter Games in 2014.
Other Young Ambassadors include coaches, students and young professionals, all aged between 18 and 28 years. Despite their diverse backgrounds, they are driven by their common experience at the Youth Olympic Games to inspire other young people to take responsibility for their own lives and the communities they live in. This youth initiative - and many other similar ones – gives young people the power to change their lives and the lives of others. It is this kind of responsible citizenship that the Youth Olympic Games is creating.
For Lillehammer, it will feel like going back to the future. Through the Winter Youth Olympic Games 2016, the city is taking the legacy of the Olympic Winter Games in 1994 to another level. Many venues used in 1994 – such as the iconic ski jumping arena where the Opening Ceremony will take place – have been modernized, giving them a second life 22 years later, to ensure that Lillehammer's Olympic legacy can inspire a new generation of young athletes. In doing so, Lillehammer is laying the groundwork for the future. The true test of the legacy for Lillehammer and other host cities of the Youth Olympic Games will be the inspiration they bring for the next generation of young athletes and responsible citizens. By investing in youth, these host cities are also laying the foundation for the future success of their societies at large. They are fostering talent and inspiring youth with values that will last a lifetime.
Just like the Youth Olympic Games can be a launch pad for the future careers of these talented young athletes, hosting the Youth Olympic Games can be a springboard for the Winter Games. In Austria, Innsbruck has successfully built on the legacy of the Olympic Winter Games of 1976 by hosting the inaugural Winter Youth Olympic Games in 2012. Lillehammer is pressing the reset button for a new generation of sport enthusiasts in the country 22 years after the Olympic Winter Games of 1994. The Winter Youth Olympic Games Lausanne 2020 will bring the Olympic Games back to Switzerland for the first time since 1948 and into the "Olympic Capital", which is home of the IOC since 1915. What these cities have in common is their recognition of the value that investing in youth can bring to future of their communities - sometimes this means going back to the future. In all cases, it uses the power of sport to make the world a better place.
The author is the President of International Olympic Committee.
Young Olympic Champion 2007-08-14 10:49
Crowds flock to see Olympic venue 2016-02-16 08:02
Olympic team signs up bug spray maker 2016-02-04 07:47
Former Olympic champion poses with family 2016-02-03 06:43
Inspectors to cover all of military
Britons embrace 'Super Thursday' elections
Campaign spreads Chinese cooking in the UK
Trump to aim all guns at Hillary Clinton
Labour set to take London after bitter campaign
Labour candidate favourite for London mayor
Fossil footprints bring dinosaurs to life
Buffett optimistic on China's economic transition
Sinopec opens new industrial platform
Data point to Chinese economy shrugging off sluggishness and stabilizing
China leads way on US adoptions
Season of the locust eaters
Humble bicyclist becomes Beijing nighthawk
Chinese must adapt to UK 'study shock'


text:
NewsProject Drive Sober
Loophole still allows Wisconsin drunk drivers to break the law
Convicted drunk drivers are continuing to skirt the law and not install court ordered ignition interlock devices, four years after an I-TEAM investigation.
By: Marty Hobe
WISCONSIN — See more on this story tonight on Live at 10.
In 2016, nearly half of drivers ordered to install IIDs in their cars did not comply. In 2020, we found there is still a gap of drivers not following court orders.
Wisconsin Department of Transportation number show that 37,906 drivers of the 58,888 ordered to install IIDs registered their device, a compliance rate of about 64 percent. That means up to 20,982 drivers are illegally driving without an IID.
In 2015, one such driver hit and killed Roger Patoka. Tony Garvey, who was just out of jail for this fourth OWI, hit Roger while he was on his motorcycle driving up to a family vacation in Shawano. His wife Paula and their kids were following in the car behind, and were also hit by Garvey.
'She knew then her husband had died': Woman relives tragic OWI crash that killed her husband
"He didn't have the interlock device," she also said. "He wasn't supposed to be driving. It wasn't his car. I mean somebody bought him his car. In his family, somebody bought him a car."
"It was a horrible,horrible night," Paula said. "And watching your kids suffer through all of this, it's hard to this day, and it's four and a half years later."
Garvey was cited for driving without an ordered IID back in 2012.
"They can still go and drive somebody else's car," said state Sen. Van Wanggaard. "But if they drive it, the judge is saying if you drive somebody else's car, you need to have an IID in it. But its not attaching it to that person's license status."
Wanggaard has authored a bill he believes will help with IID compliance, while increasing the penalties for not following court orders.
Current law allows for a maximum penalty of $200 if you're caught without an IID. Wanggaard's bill increases that penalty to $600, and potentially six months in jail. It also attaches the IID requirement to their license status, not the car registered in their name.
"The ultimate goal is compliance," Wanggaard said. "We want people to be compliant with what they're doing, not consume alcohol."
The bill has the support of Mothers Against Drunk Driving. Director of State Government Affairs Frank Harris says the bill also allows offenders to start their interlock time prior to a conviction, which he calls a best practice.
"Upon revocation ... you're supposed to get an occupation license in Wisconsin," Harris said. "And that's time and route restricted and there's no way for law enforcement to enforce it and it does absolutely nothing to change someone's behavior."
Harris added IIDs have stopped more than 3,000 drunk driving attempts in the state of Wisconsin.
How an ignition interlock device works
Wanggaard has introduced versions of this bill in the past, but they failed due to technicalities between state and federal law.
Paula and others affected by drunk driving are tired of waiting for justice for their loved ones.
"That's really sad that its gone this long," Paul said. "It's maddening. It makes me angry."
Last month the bill passed the Senate and now awaits a vote in the Assembly.
Wanggaard's office tells us they believe this version has a better chance in the assembly than previous ones.
Loophole still allows Wisconsin drunk drivers to break the law_snackable
Report a typo or error
Who are my legislators?
Get help with substance abuse


text:
Q: stop refresh of page between hours I have an issue where I want to check the time and do something based on it. For example, from 12 midnight to 1 am, I would like to stop the page from refreshing. How would I accomplish this?
function RefreshPage()
{
    if (refresh == 'Y')
    {
                CheckAndStopAutoRefresh();
        }
}

function CheckAndStopAutoRefresh()
{
    var d = new Date();
        var currentHour = d.getHours(); // 0-23

        if (currentHour >= 0 && currentHour < 1)
        {
        return;

        }
        else
        {
            window.location=window.location; // refresh
        }

}

A: //initialize date object
var d = new Date();
var currentHour = d.getHours(); //note 0-23

//you could use currentHour == 0 but this expression allows for a range
if (currentHour >= 0  &&  currentHour < 1)
 {
     console.log('between 0:00 and 1:00 hrs');
  }
 else { console.log('after 1am'); }

https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Date provides and overview of the date object
I am not too sure what you mean by 'stop refresh' I do not believe it possible to disable refresh/F5, but perhaps if you modify your question or specify your intentions I can take another stab at the question.
EDITED
--edited to reflect updated question.
I would use setInterval(code,millisec,lang)
function CheckAndStopAutoRefresh() {
    //initialize date object
    var d = new Date();
    var currentHour = d.getHours(); //note 0-23

    //you could use currentHour == 0 but this expression allows for a range
    if (currentHour >= 0  &&  currentHour < 1)
     {
         console.log('between 0:00 and 1:00 hrs');
      }
     else { console.log('after 1am'); }
}

setInterval(CheckAndStopAutoRefresh,60000); //60,000 milliseconds in a minute

redpajama_set_name: RedPajamaStackExchange


text:
Papilio albinus est une espèce de lépidoptères (papillons) de la famille des Papilionidae. L'espèce est endémique de l'île de Nouvelle-Guinée et des îles environnantes.

Systématique
L'espèce Papilio albinus a été décrite pour la première fois en 1865 par Wallace dans Transactions of the Linnean society.

Liens externes

Notes et références

Papilionidae
Lépidoptère décrit au XIXe siècle
Lépidoptère d'Océanie

redpajama_set_name: RedPajamaWikipedia


text:
Perverted Bengali's Mind
I trust people's freedom and freedom of speech. I want to say that whatever you want to say without any criminal offense is abusive. But in my opinion, abuse is the last weapon of humble people that they can not understand or apply. It's his mental dilemma that he can not control himself. If someone slaughters him, then let him be cursed in response to his level. My personal beliefs, the Bengali nation fosters the culture of humiliation. They get the most pleasure from neighboring quarrels. Many urban people enjoy the fun and ridicule the villagers. They feel distorted and distorted by the human beings, or they are too small.
Vietnamese soldiers who raped women in Vietnam during the war, after the war, raped women destroyed the child. And they did this because they did not praise the babies as they grew up in the womb. In 1971, many of our mothers who had been raped gave birth to a healthy baby without understanding them. I remember that when we see people of Pakistan praising or distorted and violent tastes around us today. There is a class of Bengali people who laugh at crime like rape. Or try to legitimize rape by rape or rape. To them, Begum Rokeya, Sufia Kamal and Taslima Nasrin are a prostitute. In this case, some of the poets of the poets, who also saw some bad words, sing the same song.
Rape scenes in the dark era of Bangla movie were shown in such a way that sexually disturbed Bangladeshi people saw such a distorted sex. And a generation has seen it grow. However, I think that women and women in street are talking about poor talk or rape with their eyes due to lack of family education. Repeat virtual events of the same event also happen. In this case, to send hi, hello, sex or nude pictures to the inbox or ask for a picture of its nude, to a common beparan boy. And when you do not get it out of the end, some of the * nakhi * gi or some f bardi also slap Even after failing in love or after the breakup of relationships, some of the girls also have to listen to abuse.
Feminism is a fight to get a right, but seeing most of our country's elite feminists, it seems that they only do this by collecting their own sugarcane. There is a great difference in the lives and lives of the marginalized women in rural areas with the demand and life of the urban or upper-middle women. In many cases women are female enemies of rural or lower-level families. The same method does not fit all the way. Again, on the question of rights, in most cases they want equal rights but are not interested in taking equal responsibility. Here another side comes in front, everyone speaks against dowry (I am also against dowry or anti-paan) but in danmohar again do not want to reduce the figure. Many such contradictions are observed in Bengali women.
Years ago, in a newspaper news, a Taliban boy from 5-6 years said in a weapon that women are unlawful, and women are unclean. But the child came to the world from the forbidden illegal system several years ago. What has been done in this case is bleeding or brainwashing. Describing these distorted statements or making various crimes by showing the children of the children of the destined child, brainwashing the mind of the imaginary heaven. Ahmad Shafi alias Tantul Sood, who had sabotaged at Shapla Chattar on 5th May, was from a type of idol, which the Taliban used to hold at least 15 years ago. However, this ruling snake is now directly and indirectly supporting and cooperating. There is no ideological difference between Jamaat Islam and Hefajat Islam. Old wine in that new bottle So it means to indulge themselves, killing themselves on their feet. To cooperate with them means pushing the country towards destruction is to push the country towards destruction.
Ahmad Shafi Gong will give reception to the Prime Minister in the historic Suhrawardy Udyan tomorrow. Actually, who does not want to be praised, but the fact is, how rational it is to get reception from fundamentalist or communal groups. There is a matter of politics in Giv and Tech, so who wants to give it here and who wants to take it is clear. The ruling party wants cadre and hefajat want government recognition and immoral advantage I do not know whether there was any secret negotiation agreement between the two parties, but there was something that did not need to be a rocket crystallist to understand. It may be understood that the poet has to make a comparison of the Master, the land of the railway, the textbook to execute them, and the formation of the Digital Security Act in accordance with the blasphemy law, that is, to understand their support. And in the same way, the country is moving forward a little bit towards the Taliban. And Sheikh Hasina's contribution in creating a communal state of Bangladesh is a little too much to be written in history.
Open Letter to PM Sheikh…
Perverted Bengali Mind


text:
<?php
return array(
	'Flickr\Controller\FlickrController'					=> __DIR__ . '/src/Backend/Controller/FlickrController.php',
	'Backend\Controller\FlickrBackend'			=> __DIR__ . '/src/Backend/Controller/FlickrBackend\Controller\FlickrBackend.php',
	'Backend\Controller\FlickrBackendInsertor'	=> __DIR__ . '/src/Backend/Controller/FlickrBackend\Controller\FlickrBackendInsertor.php',
);

redpajama_set_name: RedPajamaGithub


If you want to serve webpages with ssl certificates that have Subject Alternative Names, and you use FreeIPA, you will need to take a few steps to make this possible. If you got to this page, you probably already know the importance of SAN on a cert.
This document will demonstrate how to get IPA to sign a certificate that has the ever-important SubjectAltName.
You don't even need to have all the SANs in the same domain!
Use a simple passphrase you can remember.
Before you generate the csr, you will need to modify the default openssl.cnf file so it will make a csr with Subject Alternative Names.
In CentOS 7, that file is /etc/pki/tls/openssl.cnf.
You can also include IP.1 = 192.168.1.1 entries.
To be able to sign a certificate in freeipa with whatever SANs you want, you need to have a host entry for each domain.
So manually create the hosts. You can force it; they are just dummy hosts.
Also manually create HTTP service entries for each of those hosts.
I used the web interface for this, because it was easier for me. But everything in freeipa can be done with the cli; I simply haven't done the research for how to make new host objects in FreeIPA on the command line yet.
In the web UI, you can navigate to Identity -> Services -> principal HTTP/storage1.ipa.example.com@IPA.EXAMPLE.COM.
Select the Actions button, and then New Certificate.
Paste the contents of the csr file.
In the web UI, under the section Service Certificate, select the Actions button -> Get certificate. You can copy the text and save it in the terminal.


text:
LG HT953TV Home Cinema System Review - LG HT953TV Review
John Archer | February 7, 2011 11:12 am GMT
Page 1 LG HT953TV Home Cinema System Review
Page 2 LG HT953TV Review
Turning to the system's convenience features, meanwhile, it's good – if not surprising these days – to find that the HT953TV is compatible with HDMI's CEC communications system, ensuring that you can control it using the TV's remote.
Even more nifty is the system's USB Direct Recording feature. This allows you to dump in MP3 form any CDs you're playing in the system's disc tray direct to a connected, compatible MP3 player or USB memory stick. The USB port that makes this function possible can also be used to play MP3, WMA, JPEG and DivX files through the system.
If you're one of the legions of iPod users, don't worry; you haven't been forgotten. For the HT953TV supports iPod docking, complete with iPod control via the system's remote control.
There's more feature friendliness, too, in the shape of 1080p upscaling from the system's DVD deck; an HD AV Sync feature to help you avoid the lip-synch problems sometimes associated with HD TVs and broadcasts; touch-sensitive controls on the main control unit; and a colossal (and quite probably very optimistic) claimed total power output of 1,000W.
Before immersing ourselves in the HT953TV's hopefully groundbreaking surround sound performance, it's worth dwelling for a moment on other aspects of the system's connectivity beyond the USB and iPod ports we've already covered. For these include an optical audio input so that external devices like a PS3 or Sky HD box can make use of the system's audio talents, and the HDMI output necessary to ship the deck's 1080p images to a suitably talented TV.
And so to the moment of truth: does the HT953TV sound as good as it looks? And does its much-vaunted 10.1 system actually deliver the goods?
Starting out in straight 5.1 mode, it's quickly apparent that the HT953TV uses its unusually large size to deliver one of the most potent and dynamic soundstages we've heard from an all-in-one system. Those floorstanding speakers really do manage to produce veritable walls of immersive sound in front of and behind you, yet they do so without compromising the geographical accuracy of spot effects that's so important in creating a convincing film or game soundtrack.
In other words, if a Splicer in Xbox 360 smash Bioshock (which I've strangely found myself playing again lately) is sneaking up from somewhere over your right shoulder, that's exactly where you'll hear them coming from – against a wider spread of rear-channel audio ambience that really locates you in the game's terrifying world with remarkable conviction.
It's worth stating, too, that despite the truly impressive power of the HT953TV's floorstanding speakers, they're carefully calibrated so that they never overwhelm the dialogue emerging – with clarity and richness – from the system's centre speaker.
The size and power of the four main speakers also helps the HT953TV circumvent another common all-in-one system failing: detached bass. The likeably low but well-controlled rumblings of the subwoofer blend nearly seamlessly with the bottom end of the main speakers' range, leaving no obvious bass 'gap'.
John Archer
John Archer has written about, and been immersed in the world of, home entertainment technology for over 20 years. He intends to continue in this vein until he becomes too frail to lift ridiculously b…
Unlike other sites, we thoroughly test every product we review. We use industry standard tests in order to compare features properly. We'll always tell you what we find. We never, ever accept money to review a product. Tell us what you think - send your emails to the Editor.
LG HT953TV Home Cinema System Review Next
LG HT953TV Review


text:
<?php

namespace hipanel\site\helpers;

use Yii;
use yii\base\Behavior;
use yii\web\Response;
use yii\web\UrlRule;

class SEORedirectBehavior extends Behavior
{
    /**
     * @var array
     */
    public $redirectMap = [];

    /**
     * @var int
     */
    public $statusCode = 302;

    public function events()
    {
        return [
            Response::EVENT_BEFORE_SEND => 'beforeSend'
        ];
    }

    public function beforeSend()
    {
        if (Yii::$app->response->statusCode === 404) {
            $pathInfo = Yii::$app->request->pathInfo;
            if (!empty($pathInfo)) {
                foreach ($this->redirectMap as $pattern => $redirect) {
                    $rule = (new UrlRule(['pattern' => $pattern, 'route' => $pathInfo]));
                    if (preg_match($rule->pattern, $pathInfo, $matches)) {
                        Yii::$app->response->redirect('/' . $this->redirectMap[$rule->name], $this->statusCode)->send();
                        Yii::$app->end();
                    }
                }
            }
        }
    }
}


text:
\section{Introduction}\label{sec:intro}

Decision-making and optimization in uncertain environments are ubiquitous tasks faced by human beings, other natural creatures and intelligent machines. For example, animals have to decide on which unknown environment patches to search for food resources, which is crucial for their survival~\cite{Viswanathan2011}.
Such tasks are also common in a pandemic situation, where one needs to decide on whether to adopt new life-saving medicines or vaccines with limited supporting evidence, or to research these medications to understand their pros and cons more thoroughly before decisions are made~\cite{Villar2015}; in terms of testing, to decide which areas or groups to prioritize testing resources without a comprehensive understanding of the infection mechanism when testing capacity is limited~\cite{GrushkaCohen2020}.
Sticking to familiar good solutions can fully exploit the knowledge from past experiences but may lose the opportunity to discover possible better solutions, which is sub-optimal in the long run. On the other extreme, always exploring new territories can increase the chance to search for the best solutions but the excessive exploration may incur a high cost. The root of such decision-making tasks is to strike a balance  between exploration and exploitation~\cite{Sutton2018}.

The multi-armed bandit problem is one of the most classical models to address this issue~\cite{Bubeck2012,Slivkins2019}. In this model, the bandit machine consists of $K$ independent arms; each arm $k\in\{1,2,...,K\}$ will return a reward $x_{k}\in\mathcal{\mathbb{R}}$ drawn from an unknown probability distribution $P_{k}(x_{k})$ once it is pulled by the player.
The task of the player is to choose an arm $a^{t}\in\{1,2,...,K\}$ at time $t$ based on the historical outcomes of the rewards $\{ x^{\tau}_{a^{\tau}} \}_{\tau=0}^{t-1}$, in order to maximize the cumulative reward $R=\sum_{t=0}^{T}x^{t}_{a^{t}}$ for a time period $T$.
We denote the expected reward of arm $k$ as $\mu_{k}=\mathbb{E}_{P_{k}}[x_{k}]$, and define the best arm $k^{*}$ as the arm having the largest expected reward $k^{*}=\text{arg}\max_{k}\mu_{k}$. Maximizing the the cumulative rewards is equivalent to minimizing the cumulative regrets $r = \sum_{t=0}^{T} (\mu_{*} - x^{t}_{a^{t}})$ with $\mu_{*}=\mu_{k^{*}}$ as the mean reward of the best arm, since $\mu_{*}$ is a constant.
From the player's perspective, the regret is not accessible as $\mu_{*}$ is unknown, so the player will use the cumulative reward $R$ as the objective function in practice. However, the cumulative regret $r$ is more useful for theoretically assessing the player's strategy in hindsight.

Despite the simplicity of the MAB model, it captures the essential characteristics of decision-making under uncertainty, in that the rewards returned by each arm is probabilistic and that the player has to devise a strategy to make decisions based on the noisy observations.
There exist a lot of research works on the multi-armed bandit models in the statistics and applied mathematics literature, most of which aim to bound the expected cumulative regrets in the long run. Notably, optimal asymptotic bounds are established for certain policies to play the bandit machines under some appropriate conditions of the reward distributions, which states that as the time period $T$ increases, the expected regrets $\mathbb{E}[r]$ only grows as $O(\log T)$, provided that the gap is sufficiently large between the optimal and sub-optimal arms~\cite{Lai1985}.
The MAB model and the corresponding optimal strategies also receive a lot of attentions in the fields of reinforcement learning and the optimization of black-box functions, since the theoretical understanding established on the former is relevant to the latter~\cite{Sutton2018,Shahriari2016}.

Most existing theoretical studies focus on the expected behaviours of MAB. However, even when a strategy is optimal in the average sense, it may still incur high regrets in some realizations of the processes due to fluctuations of the rewards or the actions of the player (if they are stochastic)~\cite{Audibert2009, Salomon2011}.
In some applications, safety is also of concern in addition to collecting more rewards on average, which may require a more conservative strategy in general. To this end, many risk-averse bandit algorithms were proposed~\cite{Sani2012, Galichet2013}. Most existing research efforts along this line addressed this issue by augmenting or replacing the expected rewards by some risk-aware measures (such as variance), and focused on devising algorithms for the new problems.

More in-depth understanding of the fine-grained probabilistic characteristics of the deviations from the average behaviors including rare events is important for such purposes, but it remains much less explored. Some developments in such directions were made recently by mapping some bandit algorithms onto stochastic differential equations and considering the asymptotic limit~\cite{Wager2021, Fan2021}.
From another perspective, the continual interactions between the player and the bandit machine constitute an interesting stochastic dynamical system, which may exhibit rich emergent behaviours.

In this study, we aim to investigate the probabilistic nature of the regret distribution and shed light on the MAB model as a complex stochastic dynamical system through the lens of the large deviation analysis combined with a path-integral formalism from statistical physics~\cite{Touchette2009,Hertz2016}, and to provide insights on decision-making processes under uncertainty.
Comparing to many existing studies which are based on high-probability bounds and use the asymptotic limit of a long time (i.e. a large number of trials), our analysis can provide more detailed and explicit characteristics of MABs in less explored regimes more difficult to be analyzed, including those rare events with a small probability in a finite time.
We emphasize that our target is not to introduce new bandit algorithm or improve existing methods.

\section{The Model}

\subsection{Problem formulation}
The essential elements of the MAB model have been briefly introduced in Sec.~\ref{sec:intro}. Here we provide more details of the model and clarify the notations that will be used. We consider a fixed time period $0\leq t \leq T$ of the game. At time $t=0$, the player pulls each arm $k\in \{ 1,2, ... ,K \}$ once to warm up the system, and receives the corresponding random reward $x^{0}_{k} \sim P_{k}(x)$ for each arm $k$.
This can be considered as a homogeneous initial exploration step. At each time step $t \in \{1,2,...,T\}$, the player selects only one arm $a^{t} \in \{ 1,2,...,K \}$ among all $K$ options based on the historical observations of the model, and receives a random reward $x^{t}_{a^{t}}$ from the corresponding arm.

We denote $n^{t}_{k}$ as the number of times that arm $k$ has been pulled up to time $t$, and $s^{t}_{k}$ as the total rewards received from arm $k$ up to time $t$. The quantities $n^{t}_{k}$ and $s^{t}_{k}$ satisfy
\begin{align}
n_{k}^{0} & = 1, \label{eq:n0_def} \\
s_{k}^{0} & = x^{0}_{k}, \label{eq:s0_def} \\
n_{k}^{t} & = \sum_{\tau=1}^{t}\delta(a^{\tau},k)+n_{k}^{0}, \label{eq:n_def} \\
s_{k}^{t} & = \sum_{\tau=1}^{t}x_{a^{\tau}}^{\tau}\delta(a^{\tau},k)+s_{k}^{0}, \label{eq:s_def}
\end{align}
where $\delta(\cdot, \cdot)$ is the Kronecker delta function. That is, the action $a^{\tau}$ at time $\tau$ has a contribution to $n^{t}_{k}$ and $s^{t}_{k}$ only if $a^{\tau} = k$ when $\tau \leq t$.
We also adopt the short-hand notations of the vector along the time dimension $s^{0:T}_{k} := [s^{0}_{k}, s^{1}_{k}, ..., s^{T}_{k}]$, and the vector along the arm dimension $\boldsymbol{s}^{t} := [s^{t}_{1}, s^{t}_{2}, ..., s^{t}_{K}]$. Under these notations, the cumulative reward is $R = \sum_{k} s_{k}^{T}$ and the cumulative regret is $r = (T+K) \mu_{*} - \sum_{k} s_{k}^{T}$.

The policy (or algorithm) of a MAB model specifies a strategy of choosing an arm $k = a^{t+1}$ at time $t+1$ based on previous actions $\{ a^{\tau} \}_{\tau \leq t}$ and observations $\{ x^{\tau}_{a^{\tau}} \}_{\tau \leq t}$. Due to the non-interacting nature of the basic MAB model, it is convenient to consider the sum of historical choices and payoffs $\{ n^{t}_{k'}, s^{t}_{k'} \}_{\forall k'}$ to determine which arm to pull at time $t$. In this case, the action $a^{t+1}$ is a function of sum of these historical statistics as $a^{t+1}( \boldsymbol{n}^{t}, \boldsymbol{s}^{t})$.
Since the parameters of the distributions of the arms are unknown, we need to be estimate them based on historical observations. For instance, after pulling the arm at time $t$, the expected reward of arm $k$ can be estimated as
\begin{equation}
\hat{\mu}_{k}^{t} = \frac{s_{k}^{t}}{n_{k}^{t}}, \label{eq:mu_hat_def}
\end{equation}
which will be used by the player for determining which arm to pull in the future.

\subsection{Bandit Algorithms}
A simple strategy for optimizing outcomes from the MAB model is to perform the following two-stage operation (i) allocate a fixed number of trials to each arm $k$ to estimate the expected reward $\hat{\mu}_{k}$ by Eq.~(\ref{eq:mu_hat_def}) (this is a purely exploration phase); (ii) identify the estimated best arm $\hat{k}^{*} = \text{arg}\max_{k}\hat{\mu}_{k}$ and keep pulling it in the future steps (this is a purely exploitation phase).
In order to precisely estimate the arm parameters for the benefit of the second stage, a substantial amount of resources is needed to explore all arms, resulting in low rewards being collected from the non-optimal arms in the first stage. On the other hand, if less resources are spent in the first stage for exploration, then the player has a higher risk of incorrectly identifying the best arm.
Many real life problems are solved by this policy, e.g., in clinical trials, it is usual to perform a predefined amount of experimental trials to choose one medicine or vaccine among several possible options and then deploy the chosen one widely.
Such a policy can be sub-optimal for the purpose of maximizing the cumulative rewards. Many research efforts are devoted to finding a better policy (or algorithm) for MAB which strikes a balance between exploration and exploitation.

\subsubsection{$\epsilon$-greedy Algorithm}
An alternative strategy is to perform exploration and exploitation stochastically in an online fashion. In the $\epsilon$-greedy algorithm~\cite{Sutton2018}, at each time step, the player either chooses the estimated best arm to pull (with probability $1-\epsilon$), or selects a random arm to explore (with probability $\epsilon$). The probability $\epsilon$ controls the amount of exploration, which can vary over time.

\subsubsection{Softmax Algorithm}
Another stochastic algorithm is the softmax method~\cite{Sutton2018}, which pickes an arm at time $t+1$ according to a Boltzmann distribution
\begin{equation}
p_{k}^{t+1} = \frac{ e^{ \beta \hat{\mu}_{k}^{t}} }{ \sum_{j=1}^{K} e^{\beta \hat{\mu}_{j}^{t}} },
\end{equation}
where $\beta$ is the inverse temperature parameter controlling the amount of exploration. For a small $\beta$, it tends to explore more uniformly among different arms, while for a large $\beta$, it tends to exploit the estimated best arm. The inverse temperature can also vary over time.

\subsection{UCB Algorithm}
Despite the simplicity of the above-mentioned stochastic algorithms, they either yield sub-optimal total rewards or require carefully tuning the parameters.
Another important class of bandit algorithms, the upper confidence bounds (UCB) algorithms, stem from more solid theoretical properties and achieve optimal asymptotic total rewards on average~\cite{Auer2002, Bubeck2012, Slivkins2019}. It starts by computing the upper confidence bound (UCB) of the sample mean estimation for each arm $k$ as
\begin{align}
B_{k}^{t} & = \frac{s_{k}^{t}}{n_{k}^{t}} + b^{t} \frac{1}{\sqrt{n_{k}^{t}}}, \label{eq:UCB_def1}
\end{align}
where the parameter $b^{t}$ controls the confidence level, which can vary over time. Since the sample standard deviation of a certain random variable is proportional to $1/\sqrt{n}$ after $n$ independent measurements, Eq.~(\ref{eq:UCB_def1}) represents an upper confidence bound for reward $x_{k}$ after $n^{t}_{k}$ measurements on arm $k$.
The UCB algorithm proceeds by selecting the arm corresponding to highest UCB index $a^{t+1} = \text{arg}\max_{k} B_{k}^{t}$, which is a principle of ``optimism in the face of uncertainty'' (by comparing the best possibilities of all arms in a certain confidence level).

The two terms of the UCB index in Eq.~(\ref{eq:UCB_def1}) represent the effects of exploitation and exploration, respectively. The second term of Eq.~(\ref{eq:UCB_def1}) encourages exploration because if an arm $k$ has a low $n_{k}^{t}$ (low exploration of the arm up to time $t$), then the second term is large for arm $k$, which increases its UCB index and the chance that arm $k$ is being pulled.

If the rewards have bounded range, e.g., $0 \leq x_{k} \leq 1$, it is sufficient to achieve the optimal asymptotic regret bound of $O(\log(T))$ by setting the tuning parameter $b^{t-1}$ in Eq.~(\ref{eq:UCB_def1}) as $b^{t} = c\sqrt{\log(K+t)}$, where $c$ is a parameter tuning the level of exploration; notice that $K+t$ is the total number of arm pulls until time $t$.

\subsection{MABs as Stochastic Dynamical Systems}
Despite the simplicity of the problem formulation, the MABs represent fairly complex decision-making process from the dynamical system perspective, as the random rewards being collected will impact on the decisions in future rounds, and this will in turn impact on the estimate of rewards given by individual arms and hence the reward given by the bandit machine in the future.

To disentangle such complex dynamics, we will adopt methods from statistical physics on stochastic dynamical systems for analyzing MABs. To this end, we consider the probabilistic evolution of the action $\{ a^{t} \}_{1 \leq t \leq T}$ in the form of, given by
\begin{align}
& B_{k}^{t}(s_{k}^{t},n_{k}^{t}) = \frac{s_{k}^{t}}{n_{k}^{t}} + c \sqrt{\frac{\log(K + t)}{n_{k}^{t}}}, \\
& P(a^{t+1}=k) = \frac{e^{\beta B_{k}^{t}(s_{k}^{t},n_{k}^{t})}}{\sum_{j}e^{\beta B_{j}^{t}(s_{j}^{t},n_{j}^{t})}}, \qquad t \geq 0, \label{eq:P_of_a_def}
\end{align}
which is a combination of the softmax and UCB strategies. In the infinite $\beta$ (zero temperature) limit, Eq.~(\ref{eq:P_of_a_def}) reduces to the UCB algorithm. On the other hand, it becomes the traditional softmax algorithm when $c=0$.

In this work, we specify the arm reward distributions as Gaussian distributions,
\begin{equation}
P(x_{k}^{t})=\mathcal{N}(x_{k}^{t}|\mu_{k},\sigma_{k}^{2}). \label{eq:P_of_x_def}
\end{equation}

\section{Statistical-physics Analysis}
We now proceed to analyze the stochastic dynamical process of MABs by adopting methods from statistical physics. The quantity of interest is the probability distribution of the cumulative regret
\begin{align}
P(r) & = \bigg\langle \delta \bigg[ r - \big( (T+K)\mu_{*} - \sum_{k}s_{k}^{T} \big) \bigg] \bigg\rangle_{a^{1:T}, \boldsymbol{x}^{0:T}}, \nonumber \\
& = \int\prod_{t=0}^{T-1}\mathrm{d}\boldsymbol{x}^{t}P(\boldsymbol{x}^{t})\prod_{t=1}^{T} \sum_{a^{t}} P(a^{t}) \nonumber \\
& \qquad \times \delta\big(r+\sum_{k}s_{k}^{T}-(T+K)\mu_{*}\big),  \label{eq:P_of_r_def}
\end{align}
where the average is taken over the dynamical variables $a^{1:T}$ and $\boldsymbol{x}^{0:T}$ with respect to the distributions given in Eq.~(\ref{eq:P_of_a_def}) and Eq.~(\ref{eq:P_of_x_def}).

\subsection{Path-integral Formalism}
We note that in Eq.~(\ref{eq:P_of_a_def}), the variables $s_{k}^{t}$ and $n_{k}^{t}$ also depend on the historical trajectories $a^{1:t}$ and $\boldsymbol{x}^{0:t}$ through Eq.~(\ref{eq:n_def}) and Eq.~(\ref{eq:s_def}), making the average in Eq.~(\ref{eq:P_of_r_def}) highly non-trivial. Borrowing methods from statistical physics, we consider $\{r, \boldsymbol{n}^{0:T},  \boldsymbol{s}^{0:T} \}$ as statistical fields and adopt the path-integral formalism for computing this average.

We first express the delta function in Eq.~(\ref{eq:P_of_r_def}) by its Fourier representation
\begin{equation}
\delta\big(r+\sum_{k}s_{k}^{T}-(T+K)\mu_{*}\big) = \int \frac{d\hat{r}}{2\pi} e^{-\mathrm{i} \hat{r}\big(r+\sum_{k}s_{k}^{T}-(T+K)\mu_{*}\big)},
\end{equation}
and insert the Fourier representation of unities for  $\{ s_{k}^{t},n_{k}^{t} \}$ through the definition in Eq.~(\ref{eq:n0_def})-Eq.~(\ref{eq:s_def}) to Eq.~(\ref{eq:P_of_r_def})
\begin{align}
1 & = \int \frac{\mathrm{d} \hat{n}_{k}^{0} \mathrm{d} n_{k}^{0}}{2\pi} \exp\big[ -\mathrm{i} \hat{n}_{k}^{0} \big( n_{k}^{0} - 1 \big) \big], \\
1 & = \int \frac{\mathrm{d} \hat{s}_{k}^{0} \mathrm{d} s_{k}^{0}}{2\pi} \exp\big[ -\mathrm{i} \hat{s}_{k}^{0} \big( s_{k}^{0} - x_{k}^{0} \big) \big], \\
1 & = \int \frac{\mathrm{d} \hat{n}_{k}^{t} \mathrm{d} n_{k}^{t}}{2\pi} \exp\bigg[ -\mathrm{i} \hat{n}_{k}^{t} \bigg( n_{k}^{t}-1-\sum_{\tau=1}^{t}\delta(a^{\tau},k) \bigg) \bigg], \\
1 & = \int \frac{\mathrm{d} \hat{s}_{k}^{t} \mathrm{d} s_{k}^{t}}{2\pi} \exp\bigg[ -\mathrm{i} \hat{s}_{k}^{t} \bigg( s_{k}^{t}-x_{k}^{0}-\sum_{\tau=1}^{t}x_{k}^{\tau}\delta(a^{\tau},k) \bigg) \bigg],
\end{align}
after which Eq.~(\ref{eq:P_of_r_def}) becomes
\begin{align}
P(r) & = \int\prod_{t=0}^{T}\mathrm{d}\boldsymbol{x}^{t}P(\boldsymbol{x}^{t})\int\prod_{k=1}^{K}\prod_{t=0}^{T}\frac{\mathrm{d}\hat{s}_{k}^{t}\mathrm{d}s_{k}^{t}}{2\pi}\frac{\mathrm{d}\hat{n}_{k}^{t}\mathrm{d}n_{k}^{t}}{2\pi}\frac{\mathrm{d}\hat{r}}{2\pi} \nonumber \\
& \sum_{\{a^{t}\}_{t=1}^{T}} \prod_{t=1}^{T} \bigg[ e^{\beta B_{a^{t}}^{t-1}(s_{a^{t}}^{t-1},n_{a^{t}}^{t-1})} / \sum_{k'=1}^{K}e^{\beta B_{k'}^{t-1}(s_{k'}^{t-1},n_{k'}^{t-1})} \bigg] \nonumber \\
& \times \exp \bigg\{ -\mathrm{i} \hat{r}\big(r+\sum_{k}s_{k}^{T}-(T+K)\mu_{*}\big) \bigg\} \nonumber \\
& \times \exp\bigg\{-\mathrm{i}\sum_{k=1}^{K}\bigg[\hat{s}_{k}^{0}\big(s_{k}^{0}-x_{k}^{0}\big)+\hat{n}_{k}^{0}\big(n_{k}^{0}-1\big)\bigg]\bigg\} \nonumber \\
& \times \exp\bigg\{-\mathrm{i}\sum_{k=1}^{K} \sum_{t=1}^{T}\hat{s}_{k}^{t}\bigg(s_{k}^{t}-x_{k}^{0}-\sum_{\tau=1}^{t}x_{k}^{\tau}\delta(a^{\tau},k)\bigg) \bigg\} \nonumber \\
& \times \exp\bigg\{ -\mathrm{i}\sum_{k=1}^{K} \sum_{t=1}^{T}\hat{n}_{k}^{t}\bigg(n_{k}^{t}-1-\sum_{\tau=1}^{t}\delta(a^{\tau},k)\bigg) \bigg\}. \label{eq:Pr_path_int_1st}
\end{align}
In the second line of Eq.~(\ref{eq:Pr_path_int_1st}), the variables $\{ s_{k}^{t},n_{k}^{t} \}$ are stochastic fields which do not explicitly depend on $a^{1:t}$ and $\boldsymbol{x}^{0:t}$. Instead, such dependencies are expressed through their coupling to the conjugate fields $\{ \hat{s}_{k}^{t}, \hat{n}_{k}^{t} \}$.

Now the average over the disorder variables $\boldsymbol{x}^{0:T}$ in Eq.~(\ref{eq:Pr_path_int_1st}) can be easily perform by using the identity $\int e^{-\mathrm{i}xh}\mathcal{N}(x|\mu,\sigma^{2})dx=e^{-\frac{1}{2}\sigma^{2}h^{2}-\mathrm{i}\mu h}$, such that the distribution of regret $P(r)$ has the form of
\begin{align}
P(r) = \int\prod_{k=1}^{K}\prod_{t=0}^{T}\frac{\mathrm{d}\hat{s}_{k}^{t} \mathrm{d}s_{k}^{t}}{2\pi}\frac{\mathrm{d}\hat{n}_{k}^{t} \mathrm{d} n_{k}^{t}}{2\pi}\frac{d\hat{r}}{2\pi}e^{-\Phi}, \label{eq:Pr_of_Phi}
\end{align}
where $\Phi$ is a stochastic action defined as
\begin{align}
\Phi & = \mathrm{i}\hat{r}\bigg(r+\sum_{k}s_{k}^{T}-(T+K)\mu_{*}\bigg) \nonumber \\
& +\mathrm{i}\sum_{k}\sum_{t=0}^{T}\big(\hat{s}_{k}^{t}s_{k}^{t}+\hat{n}_{k}^{t}(n_{k}^{t}-1)\big)  \nonumber \\
& + \frac{1}{2}\sum_{k}\sigma_{k}^{2}\sum_{t,t'=0}^{T}\hat{s}_{k}^{t}\hat{s}_{k}^{t'}n_{k}^{\min(t,t')} -\mathrm{i}\sum_{k}\mu_{k}\sum_{t=0}^{T}\hat{s}_{k}^{t}n_{k}^{t} \nonumber \\
& - \sum_{t=1}^{T}\bigg\{\log\sum_{k}\exp\bigg[\beta B_{k}^{t-1}(s_{k}^{t-1},n_{k}^{t-1}) +\mathrm{i}\sum_{\tau=t}^{T}\hat{n}_{k}^{\tau} \bigg] \nonumber \\
& \qquad - \log\sum_{k}e^{\beta B_{k}^{t-1}(s_{k}^{t-1},n_{k}^{t-1})}\bigg\}. \label{eq:Phi_def}
\end{align}
See Appendix~\ref{sec:appendix_a} for details of the calculations.
The action function $\Phi$ is reminiscent of a thermodynamic potential, which expresses the stochastic dynamical system through the order parameters $\{ s_{k}^{t}, n_{k}^{t}, \hat{s}_{k}^{t}, \hat{n}_{k}^{t} \} $ of all time steps. Hence, Eq.~(\ref{eq:Pr_of_Phi}) is a discrete-time path integral.

\subsection{Small Noise Limit and Saddle-point Equations}\label{sec:saddle}
An exact expression of Eq.~(\ref{eq:Pr_of_Phi}) is difficult to obtain. To facilitate further analysis, we focus on the limit of low temperature and small arm noise $\beta \to \infty, \sigma_{k} \to 0$, which essentially consider the large deviation of the regret under the UCB algorithm~\cite{Grafke2019}. This is because when the noise is small, the regret $r$ has a high probability to be near its most probable value $r^{\text{mpv}}$; in this case a sufficiently strong departure of $r$ from $r^{\text{mpv}}$ become rare events, which is the focus of large deviation theory in terms of probability~\cite{Touchette2009, Grafke2019}. Although the theory is built on such a small noise limit, we expect some physical properties of the system still hold in the finite noise case, which is another interesting regime of the MAB problem.

In the small noise limit, for a particular value of regret $r$, the path integral in Eq.~(\ref{eq:Pr_of_Phi}) is dictated by the saddle point of the action function $\Phi$~\cite{Grafke2019}, which satisfies $\partial \Phi / \partial y = 0, y \in \{ s_{k}^{t}, n_{k}^{t}, \hat{s}_{k}^{t}, \hat{n}_{k}^{t}, \hat{r} \}$.

To make the notations more compact, we define the following quantities
\begin{align}
B^{t}_{k} & = B^{t}_{k}(s^{t}_{k}, n^{t}_{k}), \\
B^{t}_{s,k} & = \frac{\partial B^{t}_{k}(s^{t}_{k}, n^{t}_{k})}{\partial s^{t}_{k}}, \\
B^{t}_{n, k} & = \frac{\partial B^{t}_{k}(s^{t}_{k}, n^{t}_{k})}{\partial n^{t}_{k}}, \\
\rho_{k}(\boldsymbol{v}) & = \frac{e^{v_{k}}}{\sum_{j=1}^{k} e^{v_{j}}}, \\
h_{k}^{t+1} & = \beta B_{k}^{t}(s_{k}^{t},n_{k}^{t}) + \mathrm{i}\sum_{\tau > t}^{T}\hat{n}_{k}^{\tau},
\end{align}
where $\rho_{k}(\cdot)$ is the softmax operator.

Then the saddle point equations admit the following expressions
\begin{align}
n_{k}^{0} & = 1,  \\
n_{k}^{t} & = 1+\sum_{\tau=1}^{t}\rho_{k}(\boldsymbol{h}^{\tau}),\quad1\leq t\leq T, \label{eq:nkt_forward_1} \\
s_{k}^{t} & = \mu_{k}n_{k}^{t} + \sigma_{k}^{2}\sum_{t'=0}^{T}(\mathrm{i}\hat{s}_{k}^{t'})n_{k}^{\min(t,t')}	,\quad0\leq t\leq T, \label{eq:skt_forward_1} \\
\mathrm{i}\hat{s}_{k}^{t} & = \beta B_{s,k}^{t}\big[\rho_{k}(\boldsymbol{h}^{t+1})-\rho_{k}(\beta \boldsymbol{B}^{t})\big],\; 0\leq t < T, \label{eq:iSkt_backward_1} \\
\mathrm{i}\hat{s}_{k}^{T} & = -\mathrm{i}\hat{r}, \\
\mathrm{i}\hat{n}_{k}^{t} & = \beta B_{n,k}^{t}\big[\rho_{k}(\boldsymbol{h}^{t+1})-\rho_{k}(\beta \boldsymbol{B}^{t})\big]+\mu_{k}\mathrm{i}\hat{s}_{k}^{t} \nonumber \\
& \quad -\sigma_{k}^{2}\sum_{t'>t}^{T}\hat{s}_{k}^{t}\hat{s}_{k}^{t'}-\frac{1}{2}\sigma_{k}^{2}(\hat{s}_{k}^{t})^{2},\quad0\leq t < T, \label{eq:iNkt_backward_1} \\
\mathrm{i}\hat{n}_{k}^{T} & = \mu_{k}\mathrm{i}\hat{s}_{k}^{T}-\frac{1}{2}\sigma_{k}^{2}(\hat{s}_{k}^{T})^{2}, \\
r & = (T+K)\mu_{*}-\sum_{k}s_{k}^{T},
\end{align}
where the last equation is a constraint for the total reward $\sum_{k} s_{k}^{T}$ (we remind that $r$ is a pre-defined parameter instead of a dynamical variable). We remark that the conjugate fields $\{ \hat{s}_{k}^{t}, \hat{n}_{k}^{t}, \hat{r} \}$ are defined on the imaginary axis in the saddle point~\cite{Paga2017, Li2020sensitivity}.

\subsection{Simplification in the Large $\beta$ Limit}\label{sec:sim_saddle}
Further simplification can be made by exploiting the limit $\beta \to \infty$ under consideration. In this limit, the field $h_{k}^{t+1}$ is dominated by $\beta B_{k}^{t}$, so that Eq.~(\ref{eq:nkt_forward_1}) can be approximated as
\begin{equation}
n_{k}^{t} = 1+\sum_{\tau=1}^{t}\rho_{k}(\beta \boldsymbol{B}^{\tau - 1}),\quad1\leq t\leq T, \label{eq:nkt_forward_2}.
\end{equation}

To derive the dynamics of the conjugate order parameters $\{ \hat{s}_{k}^{t}, \hat{n}_{k}^{t} \}_{t<T}$ in this limit, we notice
\begin{align}
\rho_{k}(\boldsymbol{h}^{t+1}) & \approx \rho_{k}(\beta \boldsymbol{B}^{t}) + \sum_{j=1}^{K} \left. \frac{\partial \rho_{k}(\boldsymbol{h}^{t+1})}{\partial h_j^{t+1}} \right|_{ \boldsymbol{h}^{t+1} = \beta \boldsymbol{B}^{t} } \cdot \mathrm{i}\sum_{\tau > t}^{T}\hat{n}_{j}^{\tau} \nonumber \\
& =: \rho_{k}(\beta \boldsymbol{B}^{t}) + \hat{\Delta}\rho^{t+1}(\beta \boldsymbol{B}^{t}) \cdot \sum_{\tau > t}^{T} \mathrm{i} \boldsymbol{n}^{\tau},
\end{align}
where the $K\times K$ matrix $\hat{\Delta}\rho^{t+1}(\beta \boldsymbol{B}^{t})$ has the element $\hat{\Delta}\rho^{t+1}(\beta \boldsymbol{B}^{t})_{kj} = \delta_{jk} \rho_{k}(\beta \boldsymbol{B}^{t}) - \rho_{k}(\beta \boldsymbol{B}^{t})\rho_{j}(\beta \boldsymbol{B}^{t})$.

Under this approximation, the dynamical rules of Eq.~(\ref{eq:iSkt_backward_1}) and Eq.~(\ref{eq:iNkt_backward_1}) become
\begin{align}
\mathrm{i}\hat{s}_{k}^{t} & = \beta B_{s,k}^{t} \sum_{j} \hat{\Delta}\rho^{t+1}_{kj} \sum_{\tau > t}^{T}\mathrm{i}\hat{n}_{j}^{\tau}, \\
\mathrm{i}\hat{n}_{k}^{t} & = \beta B_{n,k}^{t} \sum_{j} \hat{\Delta}\rho^{t+1}_{kj} \sum_{\tau > t}^{T}\mathrm{i}\hat{n}_{j}^{\tau} +\mu_{k}\mathrm{i}\hat{s}_{k}^{t} \nonumber \\
& \quad + \sigma_{k}^{2}\sum_{t'>t}^{T} (\mathrm{i}\hat{s}_{k}^{t}) (\mathrm{i}\hat{s}_{k}^{t'})+\frac{1}{2}\sigma_{k}^{2}(\mathrm{i}\hat{s}_{k}^{t})^{2},
\end{align}
which admit a backward iteration form.

In such a large $\beta$ limit, the stochastic action $\Phi$ \textit{evaluated at the saddle point} has a simple expression
\begin{equation}
\Phi^{*}(r | \sigma_{k}) = \frac{1}{2}\sum_{k}\sigma_{k}^{2}\sum_{t,t'=0}^{T} (\mathrm{i}\hat{s}_{k}^{t}) (\mathrm{i}\hat{s}_{k}^{t'}) n_{k}^{\min(t,t')}.
\end{equation}

We notice that if the variances of all arms are rescaled by a common factor $\gamma$, i.e., $\sigma_{k}^{2} \to \gamma \sigma_{k}^{2}$, then the conjugate order parameters $\{ \hat{s}_{k}^{t}, \hat{n}_{k}^{t}, \hat{r} \}$ evaluated at the saddle point equations change accordingly as $\hat{s}_{k}^{t} \to \hat{s}_{k}^{t}/\gamma, \hat{n}_{k}^{t} \to \hat{n}_{k}^{t}/\gamma, \hat{r} \to \hat{r}/\gamma$, while the order parameters $\{ s_{k}^{t}, n_{k}^{t} \}$ remain unchanged; the stochastic action changes as $\Phi^{*} \to \Phi^{*}/\gamma$.

If we set the variance of arm $k$ as $\sigma_{k}^{2} = \gamma \Tilde{\sigma}_{k}^2$, where  $\Tilde{\sigma}_{k}^2$ is a fixed parameter and $\gamma$ can be varied (common to all arms), then the cumulative regret $r$ follows a large deviation principle as
\begin{align}
P(r|\sigma_{k}^2 = \gamma \Tilde{\sigma}_{k}^2) & \propto \exp\bigg( - \frac{1}{\gamma} I(r; \Tilde{\sigma}_{k}^2) \bigg), \label{eq:Pr_LDP} \\
I(r | \Tilde{\sigma}_{k}^2) & \equiv \gamma \Phi^{*}(r | \sigma_{k}),
\end{align}
where $I(r | \Tilde{\sigma}_{k}^2)$ is the rate function governing the rareness of regret $r$, and the order parameters $\{ s_{k}^{*t}, n_{k}^{*t} \}$ at the saddle point dictates the most probable pathway leading to the regret $r$ being specified~\cite{Grafke2019}.

We remark that such a relation is derived in the limit $\sigma_{k} \to 0, \beta \to \infty$, but we expect some physical pictures can also be extended to cases with a finite noise strength. The approximation in Eq.~(\ref{eq:nkt_forward_2}) essentially neglects some fluctuation of the arm selection noise from the Boltzmann distribution in Eq.~(\ref{eq:P_of_a_def}), which requires that the fluctuation due to a finite $\beta$ in Eq.~(\ref{eq:P_of_a_def}) is much smaller than the fluctuation from finite arm variances $\sigma_{k}^{2}$ (the limit of $\beta \to \infty$ is taken before the limit $\sigma_{k} \to 0$).

We also remark that the saddle point approach introduced above is only a leading-order approximation, and it is possible to derive higher-order corrections for a better accuracy~\cite{Hertz2016}.

\section{Results}
In this section, we report the theoretical results from the above statistical-physics analysis on MABs in different scenarios, and corroborate them by numerical experiments.

We consider a bandit machine with $K=3$ independent arms, where the reward of each arm $k$ follows a Gaussian distribution $x_{k} \sim \mathcal{N}(\mu_{k}, \sigma_{k}^{2})$. The arm distribution parameters are set to be $\boldsymbol{\mu} = [1,2,3], \boldsymbol{\sigma} = \sqrt{\gamma}[1, 1, 1]$, which corresponds to setting $\Tilde{\sigma}_{k} = 1$ in Eq.~(\ref{eq:Pr_LDP}). One technical difficulty is to solve the highly nonlinear saddle point equations with $4K(T+1) + 1$ variables. Hence, we focus on systems with a relatively small number of time steps ($T\sim O(10)$), which already exhibits many interesting dynamical phenomena.

\subsection{The Action Potential}
To solve the saddle-point equations, we adopt an iteration method described in Appendix~\ref{sec:iteration_method} in details.
We found that in some parameter regime, the saddle point equations admit multiple solutions, in which cases we retained the solution with the smallest action potential $\Phi^{*}$.

In Fig.~\ref{fig:Phi_vs_r_vary_gamma_c}, we sketch the resulting rescaled action potential $\gamma \Phi^{*}(r | \sigma^{k})$ for different arm variance magnitude $\gamma$ and different exploration parameter $c$, by fixing $\beta = 10$ and considering $T=20$.
The rescaled potential $\gamma \Phi^{*}(r | \sigma^{k})$ predicted by the theory (black lines) exhibits a peculiar non-convex structure, comprised of 3 convex pieces (in general there are $K$ convex pieces for $K$-armed bandits). This indicates a multimodal structure of the regret distribution $P(r)$~\cite{Audibert2009}.

We also compare the theoretical prediction of the action potential to numerical simulations, obtained by simulating the corresponding MAB for $1 \times 10^9$ trials. The action potential for MABs by simulation is defined as
\begin{equation}
\Phi^{\text{sim}}(r) = - \log \hat{P}(r) - \min_{r} \big[ - \log \hat{P}(r) \big],
\end{equation}
where $\hat{P}(r)$ is the empirical density of the cumulative regret, and we have also subtracted $- \log \hat{P}(r)$ by its minimum. Under this definition, the minimal $\Phi^{\text{sim}}(r)$ is zero, which facilitates an easier comparison to the theory.
We note that $\Phi^{\text{sim}}(r)$ loses the information about the precise value of $\hat{P}(r)$, but still carries the information of the relative strength between the probability densities of different regrets.

The results shown in Fig.~\ref{fig:Phi_vs_r_vary_gamma_c} demonstrate a good match between theory and simulation for small and moderate arm variance level $\sqrt{\gamma}$. The difference between the two approaches becomes more prominent for large $\sqrt{\gamma}$, which is expected since the theory is developed in the small noise limit; but even in these cases, the qualitative trend of $\Phi^{\text{sim}}(r)$ still follows the theoretical prediction.
In both theory and simulations, the action potential is asymmetric and right-skewed, which indicates that the MAB has a higher chance to be unlucky (the regret $r$ is larger than the most probable value $r^{\text{mpv}}$) than lucky (the regret $r$ is smaller than $r^{\text{mpv}}$). This is because being lucky mainly originates from the rarely good outcomes of many rounds from the optimal arm, but being unlucky originates from the rarely bad outcomes of a small number of rounds from the optimal arm at the beginning, which makes it not well explored.  The former occurs with a very small probability, while the latter occurs with a relatively higher probability.
The heavy right tail of the regret distribution indicates a noticeable probability of the total arm rewards being sub-optimal in some realizations in a shot time, despite that the algorithm is optimal in the long run in the average sense~\cite{Audibert2009, Fan2021}.

\begin{figure}
    \centering
    \includegraphics[scale=1.05]{Phi_vs_r_K3_T20_beta10.eps}
    \caption{Rescaled action potential $\gamma \Phi^{*}(r | \sigma_{k})$ as a function of the cumulative regret $r$. The black lines represent the theoretical predictions, while the dash lines with dots are obtained by numerical simulations of $1\times 10^9$ trials. The parameters are $K=3, T=20, \beta=10$. The theory in Sec.~\ref{sec:saddle} and \ref{sec:sim_saddle} predicts a universal rescaled action potential $\gamma \Phi^{*}(r|\sigma_{k}) = I(r|\Tilde{\sigma}_{k})$ in the low noise limit.}
    \label{fig:Phi_vs_r_vary_gamma_c}
\end{figure}

\subsection{Effect of the Exploration Parameter $c$}
The noticeable probability for the system to have a high regret indicates that the system can be trapped into a state of excessive selections of sub-optimal arms (as will be shown in details below), which is partly due to insufficient exploration.
This effect can be alleviated by increasing the exploration parameter $c$, as shown in Fig.~\ref{fig:Phi_and_rmpv_vary_c}(a). It is observed that for a higher $c$, the rate function $I(r|\Tilde{\sigma}_{k})$ attains larger values in the unlucky regime (with large positive regrets), indicating a much lighter tail for $P(r)$ for $r\gg 0$ and hence a much lower probability for more unlucky events.

On the other hand, increasing the exploration parameter $c$ will increase the expected value of the regret, as the chance to explore sub-optimal arms also gets higher. Since it is not straightforward to measure the expected regret in theory since it involves the knowledge of the complete form of the rate function $I(r|\Tilde{\sigma}_{k})$ over the whole domain of $r$, we compute the most probable value of the regret $r^{\text{mpv}}$ for different $c$, as shown in Fig.~\ref{fig:Phi_and_rmpv_vary_c}(b).
It is observed that $r^{\text{mpv}}$ increases very gently at the beginning, and starts to rise more significantly after $c \approx 0.4$. Therefore, the suppression of the right tail of the regret distribution $P(r)$ comes with a price of increasing the most probable regret. A risk-averse MAB algorithm needs to take these effects into account.

\begin{figure}
    \centering
    \includegraphics[scale=1.05]{Phi_and_rmpv_theory_vary_c.eps}
    \caption{(a) The rate function $I(r|\Tilde{\sigma}_{k})$ predicted by the theory in the low noise limit $\beta \to \infty, \gamma \to 0$ as a function of regret $r$ for different exploration parameters $c$. (b) The most probable value of regret $r^{\text{mpv}} = \text{arg}\min_{r} I(r|\Tilde{\sigma}_{k})$ as a function of the exploration parameter $c$.}
    \label{fig:Phi_and_rmpv_vary_c}
\end{figure}

\subsection{Dominant Trajectories}
Another nice feature of the large deviation path-integral approach is that it can give rise to the dominant trajectories leading to a particular regret $r$. When the noise is small, the trajectory leading to the smallest action $\Phi^{*}$ at a particular $r$ dominates the probability density $P(r)$ in the path integral Eq.~(\ref{eq:Pr_of_Phi}).
These dominant trajectories are also called the optimal paths or the instantons~\cite{Grafke2019}. Here, we use the characteristics of the optimal paths derived from the small noise limit to explain the dynamics behaviors of MABs with a finite noise strength, which is of more practical interest.
We also consider a  small time window $T=20$, which already provides many valuable insights.

\subsubsection{Lucky and slightly unlucky events: Conditioned on $r=-8$ and $r=6$}
In Fig.~\ref{fig:traj_small_r}, we consider the trajectories conditioned on a negative regret $r=-8$ and a small positive regret $r=6$, which is considered as a lucky case and a  slightly unlucky case. It is observed that in both cases, the optimal arm $k^{*} = 3$ is mostly chosen by the player after $t=0$, and the trajectory leading to a particular regret $r$ is due to homogeneous deviation of the realized reward $x_{k^{*}}^{t}$ at time $t$ from the its expected value $\mu_{*}$ (as demonstrated in Fig.~\ref{fig:traj_small_r}(a) and (c)). That is, the deviation $x_{k^{*}}^{t} - \mu_{*}$ is almost the same for all $t>0$, which is akin to the ``fluid phase'' of independent random variables $\{ x_i \}$ conditioned by the value of their sum~\cite{Godrche2019}.

The theoretical prediction and numerical simulations match very well, although the arm noise is not small.

\begin{figure}
    \centering
    \includegraphics[scale=1.05]{traj_K3_T20_beta10_c04_s06_small_r.eps}
    \caption{Dominant trajectories at a negative regret $r=-8$ (panel (a) and (b)) and a small positive regret $r=6$ (panel (c) and (d)), revealed by the dynamical evolution of $s_{k}^{t} / n_{k}^{t}$ and $n_{k}^{t}$. The lines represent the trajectories predicted by the theory. The dots with error bars are results from numerical simulations by simulating $1\times 10^9$ trials and keeping the trajectories with cumulative regret $r^{\text{sim}} \in [r, r+0.5)$; the error bars represent one standard deviation of the quantity measured in these trajectories. The parameters are $K=3, T=20, \beta=10, c=0.4, \sqrt{\gamma}=0.6$.}
    \label{fig:traj_small_r}
\end{figure}

\subsubsection{Unlucky events: Conditioned on $r=16$ and $r=24$}\label{sec:traj_r16_r24}
In Fig.~\ref{fig:traj_medium_r}, we consider the unlucky cases with moderate regrets with $r=16$ and $r=24$, which enters the second convex branch of the potential $\Phi^{*}(r)$ shown in Fig.~\ref{fig:Phi_vs_r_vary_gamma_c}(c).
Interestingly, the typical trajectory leading to $r=16$ behaves as follows: the reward from the 3rd arm (the optimal arm) in the initial exploration stage is unluckily a bit lower than that from the 2nd arm (the sub-optimal arm), resulting in a substantial exploitation of the 2nd arm in the subsequent steps. During this process, $n_{2}^{t}$ grows while $n_{3}^{t}$ remains unchanged. Up to a certain point, the exploration term $c\sqrt{\log(K+t)/n_{3}^{t}}$ on the 3rd arm becomes large enough for the UCB index $B_{3}^{t}$ of the 3rd arm to dominate that of the 2nd arm.
After then, the player starts to collect rewards from the 3rd arm and gradually realize that it has a higher expected reward than the more exploited 2nd arm.

Similarly, for $r=24$, the reward from the 3rd arm in the initial exploration step is also unluckily small, and it is even smaller than the case of $r=16$. Therefore, the 2nd arm is being pulled throughout the process. As the gap between the empirical means $\frac{s_{2}^{t}}{n_{2}^{t}} - \frac{s_{3}^{t}}{n_{3}^{t}}$ is large in this case, the exploration term in the 3rd arm is not strong enough to overturn the exploitation of the 2nd arm in such a small time window, which is different from the case of $r=16$.

Similar findings of the typical behaviors of the optimal arm leading to large regrets have also been reported before~\cite{Audibert2009, Fan2021}. Our analytical method complements these studies and provides a much more detailed picture of the dominant trajectories in this regime.

\begin{figure}
    \centering
    \includegraphics[scale=1.05]{traj_K3_T20_beta10_c04_s06_medium_r.eps}
    \caption{Dominant trajectories at regrets $r=16$ (panel (a) and (b)) and $r=24$ (panel (c) and (d)). The parameter setting is the same as those in Fig.~\ref{fig:traj_small_r}.}
    \label{fig:traj_medium_r}
\end{figure}

\subsubsection{Extremely unlucky events: Conditioned on $r=36$ and $r=40$}
In Fig.~\ref{fig:traj_big_r}, we consider the extremely unlucky cases with large regrets with $r=36$ and $r=40$, which enters the third convex branch of the potential $\Phi^{*}(r)$ shown in Fig.~\ref{fig:Phi_vs_r_vary_gamma_c}(c). The resulting phenomena are very similar to the cases in Sec.~\ref{sec:traj_r16_r24}, except that both the 3rd arm and the 2nd arm unluckily yield rewards smaller than the 1st arm (the worst arm). So the excess exploitation of the worst arm leads to such high regrets being considered.

Lastly, we notice that for trajectories conditioned on $r=16$ and $r=36$, numerical simulations exhibit very large fluctuations. In these cases, there exist many trajectories having similar values of $\Phi^{*}$ as the optimal path (e.g., there can be multiple solutions of the saddle point equations with similar potentials). Our theory just picks the one with the smallest $\Phi^{*}$, the most probable trajectory, assuming a vanishing arm noise. However, when the noise is finite, different dominant and sub-dominant trajectories can coexist.

\begin{figure}
    \centering
    \includegraphics[scale=1.05]{traj_K3_T20_beta10_c04_s06_big_r.eps}
    \caption{Dominant trajectories at regrets $r=36$ (panel (a) and (b)) and $r=40$ (panel (c) and (d)). The parameter setting is the same as those in Fig.~\ref{fig:traj_small_r}.}
    \label{fig:traj_big_r}
\end{figure}

\subsection{Exact Results of a Small System}
In this section, we further investigate the nature of multiple solutions of the saddle point solution equations, which is related to the behavior of multiple branches of the action potential $\Phi^{*}(r|\sigma_{k})$. To this end, we consider a small MAB system with $K = 2, T = 1$, and set the arm related parameters as $\boldsymbol{\mu} = [1, 2], \boldsymbol{\sigma} = \sqrt{\gamma}[1, 1]$. For such a small system, the saddle point equations developed in Sec.~\ref{sec:saddle} and Sec.~\ref{sec:sim_saddle} can be straightforwardly reduced to the following self-consistent equation of a single variable $\Delta s^{0} = s_{2}^{0} - s_{1}^{0}$,
\begin{align}
& g(\Delta s^{0}) = - 2 (\mu_{2} - \mu_{1}) \beta \gamma \mathcal{S}(\beta \Delta s^{0}) \big[ 1-\mathcal{S}(\beta \Delta s^{0}) \big] \mathrm{i}\hat{r}(\Delta s^{0})  \nonumber \\
& \qquad \qquad + (\mu_{2} - \mu_{1}) - \Delta s^{0} = 0, \label{eq:g_of_Ds} \\
& \mathrm{i}\hat{r}(\Delta s^{0})  = \frac{(\mu_{2} - \mu_{1}) \big[ \mathcal{S}(\beta \Delta s^{0})-2 \big] + r}{3 \gamma},
\end{align}
where we have defined the sigmoid function $\mathcal{S}(x) := 1/(1+e^{-x})$.

The root $\Delta s^{0*}$ of the equation $g(\Delta s^{0})=0$ (for a given $r$) is a stationary point of $\Phi$. We set $\sqrt{\gamma}=0.4, \beta=10$ and identify all possible solutions for different regrets; the results are shown in Fig.~\ref{fig:small_syst}. For $r < r_{c} \approx 1.9533$, there is only one branch of solution with $\Delta s^{0*} \approx 1 (= \mu_{2} - \mu_{1})$; this corresponds to the ``liquid phase'' where the arm rewards deviate from their expected value homogeneously. For $r > r_{c} \approx 1.9533$, the system develops two additional branches of stationary solutions, where the optimal arm unluckily yields a small reward. The three branches of stationary solutions for large regrets correspond to two local minima (the 1st and 2nd branches in Fig.~\ref{fig:small_syst}) separated by a saddle point (the 3rd branch in Fig.~\ref{fig:small_syst}) in the action potential in the space of order parameters. In the small noise limit, the branch with the smallest potential $\Phi^{*}$ dominates the probability density. In the case with a finite noise strength, trajectories of different branches can coexist.

\begin{figure}
    \centering
    \includegraphics[scale=1.05]{Phi_and_eqofdslhs_K2_T1_beta10_s04_.eps}
    \caption{Behaviors of a small MAB with $K=2, T=1$. We set the parameters as $\sqrt{\gamma} = 0.4, \beta = 10$. (a) The stationary action potential $\Phi^{*}(r|\sigma_{k})$ determined by Eq.~(\ref{eq:g_of_Ds}). (b) The left-hand side of the self-consistent equation $g(\Delta s^{0})$ defined in Eq.~(\ref{eq:g_of_Ds}).}
    \label{fig:small_syst}
\end{figure}

\section{Conclusion}
In summary, we employed the path-integral method from statistical physics to examine the complex dynamics of multi-armed bandits, which are classical prototypical models for understanding decision-making and optimization in uncertain environments and the dilemma between exploration and exploitation.
By mapping the MABs under the UCB algorithm onto stochastic dynamical systems and considering the low temperature and small noise limit, we derived a large deviation formalism for the cumulative regrets of MABs, from which many valuable insights of the regret distribution and the dominant trajectory leading to a certain regret were obtained, revealing the transient dynamics leading lucky, unlucky or even extreme events.

We observed that the action potential of the MABs can be non-convex, indicating the multimodal structure of the regret distribution. We showed that such a multimodal structure is due to the existence of multiple solutions of the saddle point equations when the regret is larger than a threshold. Under the UCB algorithm, the MABs have a much higher chance to be unlucky (the regret is higher than expected) than lucky (the regret is lower than expected), where the chance of being unlucky depends on the strength of exploration. The dominant trajectories leading to high regrets are those where the optimal arm unluckily yields a small reward in the initial exploration stage, such that it is much less exploited in the subsequent time steps.

Despite a relatively small time window being considered, our study uncovered many interesting characteristics of the MABs as complex systems.
Extending the analysis to larger systems and longer time windows requires a sophisticated method to solve the high-dimensional nonlinear saddle point equations, which contributes the construction of new methodology as well as a new understandings on MABs, exploitation-exploration dilemma, as well as the origin and occurrence of extremely unlucky events.
We envisage that the methods developed in this work can provide a valuable tool for analyzing different variants of MABs and other complex stochastic decision-making processes.

\begin{acknowledgments}
We thank David Saad for helpful discussions. B.L. acknowledges support from the Leverhulme Trust (RPG-2018-092), European Union's Horizon 2020 research and innovation programme under the Marie Sk{\l}odowska-Curie Grant Agreement No. 835913 and the startup funding from Harbin Institute of Technology, Shenzhen (Grant No. 20210134).
C.H.Y. is supported by the Research Grants Council of the Hong Kong Special Administrative Region, China (Projects No. EdUHK GRF 18304316, No. GRF 18301217, and No. GRF 18301119), the Dean's Research Fund of the Faculty of Liberal Arts and Social Sciences (Projects No. FLASS/DRF 04418, No. FLASS/ROP 04396, and No. FLASS/DRF 04624), and the Internal Research Grant (Project No. RG67 2018-2019R R4015 and No. RG31 2020-2021R R4152), The Education University of Hong Kong, Hong Kong Special Administrative
Region, China.
\end{acknowledgments}

\section*{Author Declarations}
\subsection{Conflict of Interest}
The authors have no conflicts to disclose.

\subsection{Author Contributions}
\textbf{Bo Li:} Conceptualization (equal); Methodology (equal); Formal Analysis (lead); Software (lead); Writing – original draft (lead).
\textbf{Chi Ho Yeung:} Conceptualization (equal); Methodology (equal); Formal Analysis (supporting); Software (supporting); Writing – original draft (supporting).

\section*{Data Availability Statement}
Data sharing is not applicable to this article as no new data were created or analyzed in this study.

redpajama_set_name: RedPajamaArXiv


The form and issue on the town are the seen item from your guest visitors whenever they see, that Standard Depth Bathroom Vanity image collection is usually your personal drive to boost your home. Standard Depth Bathroom Vanity picture gallery will pamper most people using fantastic patterns which are ready to decide on. Every single impression within Standard Depth Bathroom Vanity pic collection can provide different recommendations, this means upon entering many tricks to decorate your personal property. Standard Depth Bathroom Vanity graphic gallery is often a excellent example of this in the method of obtaining creative ideas given it basically gives you premium variations. By applying that which you see within Standard Depth Bathroom Vanity image stock, you will get property with a calming atmosphere. Basically show your private innovation by applying the initial designs with this awesome Standard Depth Bathroom Vanity graphic collection.
Do not forget consistently to put on the idea you undoubtedly such as, and one of the many photos in such a Standard Depth Bathroom Vanity photograph collection may just be selection. To deliver a loosening ambience to your house, you require a design and theme that is lovely as in Standard Depth Bathroom Vanity graphic collection. You can actually duplicate this trend definitely, or simply employ your creative ideas coming from Standard Depth Bathroom Vanity pic stock partly, contingent on your family needs. You can also unite your kinds of Standard Depth Bathroom Vanity graphic gallery using suggestions that you really already have got. Standard Depth Bathroom Vanity graphic gallery delivering graphics around HIGH DEFINITION top quality, consequently all of them deserving to be able to transfer. In addition to as a possible inspiration, it is also possible to employ photos from Standard Depth Bathroom Vanity snapshot collection to enhance the look with the PC. I really hope, you can get yourself striking creative ideas from Standard Depth Bathroom Vanity snapshot gallery. Satisfy bookmark that Standard Depth Bathroom Vanity picture stock or even site to bring up to date modern information and facts together with thank you meant for watching this blog.
Standard Bathroom Vanity Size Modest Standard Bathroom Vanity Height And Depth E9119639 Standard Bathroom Vanity Depth .
Depth Of Bathroom Vanity Amazing Furniture Ideas For Home Interior Ordinary Standard Bathroom Vanity Depth Australia .
Vanity Cabinet Depth Bathroom Vanity Cabinet Depth Bathroom Vanity Cabinet Dimensions Bathroom Vanity Cabinet Depth Standard .
Surprising Breathtaking Standard Vanity Sizes Bathroom Bathroom Cabinet Depth Bathroom Cabinet Measurements Standard Bathroom Vanity Cabinet .
Bathroom Vanity Cabinet Dimensions Average Standard Bathroom Vanity Cabinet Depth .
Standard Bathroom Vanity Height Standard Vanity Sizes Bathroom Standard Bathroom Vanity Height Standard Bathroom Vanity Height .
How Tall Is A Standard Bathroom Vanity Depth Bathroom Vanity Luxury Double Sink Cabinet Size Standard .


text:
KMD Architects, Taller Global lead design for Ritz-Carlton, Mexico City
Nov 3, 2015 8:41am
The Ritz-Carlton Hotel Company announced plans to open its first hotel in Mexico City in 2019. Upon completion, the 153-room Ritz-Carlton, Mexico City will operate under a management agreement with Thor Urbana Capital and Heldan Hotels & Resorts.
The hotel will be part of the Chapultepec Uno mixed-use project, located at the intersection of Avenida Reforma and Circuito Interior in the heart of the city's financial center. Once completed, the structure will include 58 floors and 1.1 million square feet of space, rendering it one of the tallest buildings in the city.
KMD Architects and Taller Global are leading design and architecture efforts for the project.
FREE HOTEL MANAGEMENT NEWSLETTER
Like this story? Subscribe to Hotel Design!
Hospitality professionals turn to Hotel Design as their go-to news source for the latest products, projects, and trends for hotel interior designers and architects. Sign up today to get news and updates delivered to your inbox and read on the go.
The Ritz-Carlton, Mexico City will occupy floors 36-47 of the building, featuring views of Chapultepec Park. The hotel will include Ritz-Carlton luxury services and amenities including a Club Lounge and spa.
Domestic travel boom sparks Mexico hotel development
Mexican travelers are looking to Canada and Europe for international travel, but strong domestic tourism is the catalyst for developers.
by Jena Tesse Fox Jul 15, 2019 2:23pm
Apple Leisure Group brings AMResorts to Europe
The 162-room Secrets Mallorca Villamil Resort & Spa is open in Mallorca, Spain.


text:
FILM LIBRARY » 2019 » DOCUMENTARY » FILM DETAILS NEXT »
THE WASHING SOCIETY
Director: Lynne Sachs, Lizzie Olesker
Format: Digital (screening) - HDV (shooting)
NEXT 2019 DOCUMENTARY FILM »
2019 DOCUMENTARY FILMS »
Cast: Ching Valdes-Aran, Veraalba Santa, Jasmine Holloway
Crew: Cinematographer: Sean Hanley - Editor: Amanda Katz - Composer: Stephen Vitiello
Email: lynnesachs gmail.com
When you drop off a bag of dirty laundry, who's doing the washing and folding? THE WASHING SOCIETY brings us into New York City laundromats and the experiences of the people who work there. Collaborating together for the first time, filmmaker Lynne Sachs and playwright Lizzie Olesker observe the disappearing public space of the neighborhood laundromat and the continual, intimate labor that happens there. With a title inspired by the 1881 organization of African-American laundresses, THE WASHING SOCIETY investigates the intersection of history, underpaid work, immigration, and the sheer math of doing laundry. Drawing on each other's artistic practices, Sachs and Olesker present a stark yet poetic vision of those whose working lives often go unrecognized, turning a lens onto their hidden stories, which are often overlooked. Dirt, skin, lint, stains, money, and time are thematically interwoven into the very fabric of THE WASHING SOCIETY through interviews and observational moments. With original music by sound artist Stephen Vitiello, the film explores the slippery relationship between the real and the re-enacted with layers of dramatic dialogue and gestural choreography. The juxtaposition of narrative and documentary elements in THE WASHING SOCIETY creates a dream-like, yet hyper-real portrayal of a day in the life of a laundry worker, both past and present.
Lynne Sachs makes films, installations, performances and web projects that explore the intricate relationship between personal observations and broader historical experiences by weaving together poetry, collage, painting, politics and layered sound design. Strongly committed to a dialogue between cinematic theory and practice, she searches for a rigorous play between image and sound, pushing the visual and aural textures in her work with every new project. Between 1994 and 2006, she produced five essay films that took her to Vietnam, Bosnia, Israel, Italy and Germany - sites affected by international war - where she looked at the space between a community's collective memory and her own subjective perceptions. Recently, after 25 years of making experimental documentaries, Sachs learned something that turned her filmmaking upside down. While working on Your Day is My Night (2013) in NYC's Chinatown, she realized that her subjects were performing for the camera rather than revealing something completely candid about their lives. The process of recording guaranteed that some aspect of the project would be artificial. This moved Sachs toward a new type of filmmaking - she invited her subjects to become her collaborators - to work with her to make the film about their lives. This new way of "working with reality" has inspired Sachs to present Your Day is My Night (2013) as well as Every Fold Matters (2016) as live film performances in alternative venues around New York City - including homeless shelters, labor union headquarters and laundromats as well as small non-commercial theaters. In 2017, she completed her newest experimental documentary Tip of My Tongue which was supported by a Guggenheim Fellowship in the Creative Arts and premiered as the closing night film in the Museum of Modern Art's Documentary Fortnight. Sachs has made over 25 films. Her films have screened at the New York Film Festival, the Sundance Film Festival and Toronto's Images Festival amongst others. They have also been exhibited at the Museum of Modern Art, the Whitney, Walker Art Center, Wexner Center for the Arts and other venues nationally and internationally. The Buenos Aires International Festival of Independent Cinema, Festival International Nuevo Cine in Havana and the China Women's Film Festival have all presented retrospectives of Sachs' films. Since 2006, Sachs has collaborated with her partner, filmmaker Mark Street, in a series of playful, mixed-media performances called "The XY Chromosome Project". Lynne holds an MFA in Film from the San Francisco Art Institute, an MA in Cinema from San Francisco State University, and a BA in History from Brown University. She has taught at New York University, Princeton, Hunter College, The New School, and the University of California, Berkeley.
Lizzie Olesker is a writer/director/performer whose plays and performances explore the poetry of the everyday. Lizzie presented her solo piece TINY LIGHTS: Infinite Miniature at the New Ohio Theater and Invisible Dog in Brooklyn. She was a 2013-14 Audrey Fellow with New Georges, with her new play EMBROIDERED PAST about family hoarding and loss of nature. Other plays have been developed and presented at Dixon Place, Brave New World Repertory, Clubbed Thumb, the Cherry Lane Theater and the Public. Published in the Brooklyn Rail and Heinemann Press, she's received support from the Brooklyn Arts Council, New York Foundation for the Arts and the Dramatists Guild. Also worked with the Talking Band (at the Ohio Theater and at Here) and teaches playwriting at Tisch/NYU and the New School.
Website Filmography