text:
Q: Will this spaceship collide with the star? (time dilation) I thought of the above thought experiment and arrive on 2 conflicting conclusions. I can't seem to identify the flaw in my reasoning.
Suppose there is a star 4 light years from earth that has will explode and turn into a white dwarf in 3 years (as measured in the earth frame). A spaceship travels to the star at 86% the speed of light.
According to earth's frame of reference, the journey to the star will take 4.5 years so the star will have turned into a white dwarf.
according to the ship's frame of reference, however, the journey will only take 2.25 years. Moreover, since the star is travelling relative to the ship in its own frame, the event of the star exploding will actually take 6 years. So the spaceship will collide with a white dwarf instead of a star.
I thought this had something to do with simultaneity, but I know that the events must be same in all frames of reference. My collusions imply that collisions occur between different bodies depending on the frame of reference, which can't possibly be true, can it? Where am I going wrong?

A: *

*In the earth frame, there is a star 4 light years from earth,
due to turn into a white dwarf in 3 years.  A spaceship travels to the start at speed $v=\sqrt3/2\approx .866$.  (All speeds are stated as multiples of the speed of light.)

*When the spaceship leaves earth, let's say the earth observer and the spaceship observer both set their clocks to zero.   Here are three events:
A: The spaceship leaves earth.
B: The star becomes a white dwarf.
C: The spaceship reaches the star.

*In the earth frame, these have the following coordinates:

A: $t=0$, $x=0$
B: $t=3$, $x=4$
C: $t=4/v\approx 4.62$, $x=4$

*Lorentz transform to the ship frame:

A: $t'=0$, $x'=0$
B: $t'=6-4\sqrt3\approx -.928$, $x'=8-3\sqrt3\approx 2.804$
C: $t'=4/\sqrt3\approx 2.309$, $x'=0$
As you can see, the order of events in the earth frame is first $A$, then $B$, then $C$ (the ship leaves earth, then the star explodes, then the ship reaches the star).  The order of events in the ship frame is first $B$ (occurring .928 years before $A$), then $A$, then $C$ (the star explodes, then the ship leaves earth, then the ship reaches the star).
So everyone agrees that the star explodes before the ship gets there.  We should have known this even before we calculated because $B$ and $C$ are spacelike separated.
This is essentially an elementary homework problem and I'm mildly ashamed of myself for answering it, but  I want you to see that if you took half the time you've been spending on these questions and spent it learning relativity, you'd easily be able to do this for yourself.

A: It is a synchronization problem. When the two observers synchronize their clocks at earth (Let us assume the traveler accelerates almost instantaneously so it reaches full speed still at earth), the star that is current for the traveler is in the future for the person on earth. So for the traveler, the time left to explode is smaller than otherwise. Imagine the star becomes a red giant before exploding. This means that at synchronization time, it is possible that for the earth observer the star is still in normal stage, but for the traveler it will be already in red giant stage. When they synchronize their clock at earth, far away objects will still be out of synchrony. So the star will explode before the spaceship will collide, for both observers, of course. Both observers will agree on a single event in spacetime, such a the collision between two objects.

A: Rule #1 when you can't figure out what's going on in a special relativity problem:  draw a spacetime diagram.  This is, by far, the best way to understand what's going on in most special relativity paradoxes.  If you want to actually understand what's going on with special relativity, rather than simply learning a few counterintuitive formulas, I highly recommend learning about spacetime diagrams.  Here's a good set of lecture notes from Stanford about how to construct them and how to interpret them.  They're not hard (a high-school student such as yourself should be able to understand them), and they make everything so much clearer.
For your situation, we denote the events as follows:

*

*Event $A$: spaceship passes Earth.  (We'll assume, for simplicity, that it's zipping past Earth rather than accelerating from rest.)

*Event $B$: the star collapses into a white dwarf.

*Event $C$: the spaceship passes the location of the star/white dwarf.

*Event $D$: the "countdown" until the star's collapse begins.  In the Earth's frame, this countdown takes three years.

What we see, when we construct this diagram, is that the events you're concerned about happen in a different order in the spaceship frame.  In the Earth's frame, they happen as follows:

*

*The spaceship passes the Earth (Event $A$).  Simultaneously, the countdown to the star's collapse begins (Event $D$).

*The star collapses (Event $B$.)

*The spaceship passes the location of the star, now a white dwarf (Event $C$.)
But in the spaceship's frame, the events happen in the following order:

*

*The countdown to the collapse begins (Event $D$.)

*The star collapses into a white dwarf (Event $B$.)

*The spaceship passes the Earth (Event $A$.)

*The spaceship passes the location of the star, now a white dwarf (Event $C$.)
In the ship's reference frame, the "countdown" to the collapse does in fact take much longer than the journey from the Earth to the star's location.  We can see this from the diagram — there's much more distance along the $t'$ axis between $D$ and $B$ than there is between $A$ and $C$.  But in the ship's reference frame, the countdown also starts (and ends!) well before the ship passes the Earth.  So there's no paradox with events happening "out of order".

redpajama_set_name: RedPajamaStackExchange


Royal Bank of Scotland reached a $612 million settlement with authorities today to resolve the Libor scandal, but an investigation has revealed something awfully sinister — they instant-message each other not unlike the rest of us (slightly less morally bankrupt) commoners.
A CFTC investigation into the corrupt practices of Royal Bank of Scotland traders who were rigging the global interest rate known as Libor for years has revealed something awfully sinister — they instant-message each other not unlike the rest of us (slightly less morally bankrupt) commoners.
There are more at the Washington Post, or you can look through the entire CFTC report for more.


text:
PSN Mega Discounts sale cuts the price of Witcher 3, Rage 2 and more
Meanwhile, Bioshock: The Collection is the deal of the week.
Deals by	James Pickard, Commerce Editor
A new Mega Discounts promotion has got underway today on the EU PlayStation Store, offering some substantial savings on many top PS4 games.
You can get up to 65 per cent off titles included in the sale, so while there are pages and pages for you to flick through, I've picked out some of the highlights here.
When you're so angry you just have to kick a truck.
First up, Rage 2 is only £31.99. The id and Avalanche shooter has only been out for a little over a month, but Bethesda always seems quite happy to drop the price of their games soon after launch. This could be quite tempting if you want to mindlessly cruise around blowing up stuff.
With its arrival on the Switch imminent, you can get The Witcher 3: Wild Hunt for £7.49 on PS4. That's just the base version of the game, though there's more in there to keep you gripped for many hours to come.
If you do want more of Geralt and his monster-slaying shenanigans, The Witcher 3 Expansion Pass is down to £7.99 too. That includes the Hearts of Stone and Blood and Wine expansions, plus all of the other 16 free add-ons released for the RPG.
Worth it for Minerva's Den alone.
Also worth pointing out (even if it isn't technically in the Mega Discounts' promo) is Bioshock: The Collection for £7.39. This not only bundles the three main games from the series but also all DLC, including the highly-praised Minerva's Den and Burial at Sea episodes.
You can browse everything on offer on the 'Mega Discounts' PSN page. However, here are some other top picks:
Tom Clancy's Ghost Recon Wildlands for £15.99
Thronebreaker: The Witcher Tales for £11.99
Dishonored: Death of the Outsider for £7.39
Tales of Berseria for £9.99
Middle-earth: Shadow of War for £15.99
Need some extra space for all of those new games? Then do take a look at our list of all the best PS4 external hard drives on Jelly Deals!
Buy The Witcher 3: Wild Hunt from Amazon [?]
Rage 2 Review
The Witcher 3: Wild Hunt Review
James Pickard
Commerce Editor

redpajama_set_name: RedPajamaCommonCrawl


The 'death of the nation' is a fallacy. Young people in the UK are turning back to the national in order to rebuild democracy.
An extract from a new collection of essays on generational politics in Britain: Resist! Against a Precarious Future, edited by Ray Filar. The compete book can be downloaded here.
What I'm about to argue shouldn't be controversial. Namely, that the 2014 Scottish independence referendum was the most important political event yet for my generation of young people in Britain. Not only that: it was a harbinger of the kind of politics we can expect for the future, not only in the UK but also Europe-wide. The independence campaign was movement-based, created alliances across political tribes and the vote was direct democracy in action, if not of the radical kind. In a country whose political system has been dying of slow suffocation – with the average voting age rising and increasing numbers turning their backs on the ballot box – it re-engaged the 'missing million', including 80 per cent of under thirty-fives. The majority of these young people voted for independence. In fact, without the country's pensioners, who voted against by 77 per cent, Scotland would now be an independent country. The 'impossible', the end of Britain as we know it, very nearly happened, falling short by a 10 per cent gap.
I was lucky enough to be staying in East Lothian, near Edinburgh, in the three months leading up to Scotland's referendum on 18 September 2014. Nothing prepared me for walking the streets of Edinburgh and Glasgow during that time. This was not a campaign controlled by Alex Salmond, the First Minister of Scotland and leader of the ruling Scottish National Party (SNP), as much of the coverage coming out of the London-based media would have had us believe. The independence movement was a broad alliance, overlapping but not dominated by the 'official' Yes campaign (itself a cross-party body but, as you might expect, heavily steered by the Scottish government). When asked about the party 'breakdown' of the movement, it would be fair to say it was mainly SNP, Green, Scottish Socialist Party and some Labour supporters, but this would be missing the point. A staggering number of people in Scotland took some part in the referendum campaigns, from either side, although the Yes campaign's activist base was far larger. Reflective of the general population, the majority of these 'activists' were not party members and did not self-define along party lines.
I am not going to deal here with the many and complex reasons why, as a young English woman, I voted 'Yes' to Scotland breaking away from the British state. You do not have to agree with my position to recognise that the kind of politics the referendum generated can be placed in a continuum with other movements for democracy and political rejuvenation over the last half decade, movements that have been clearly framed as 'youth politics'. Why not the Scottish referendum? The fact that it carried the spirit of the 2011-2012 wave of social movements, including Occupy, the Indignados and what was then called the Arab Spring, should have been clear from the start, and not simply because a significant number of young people voting 'Yes' in Scotland described their political journey explicitly in those terms.
Crucially, the referendum gave young people the chance to challenge the nature of the democratic system, not just to place an X in a box. Again, we won't dig into the arguments around how deliverable this promise was. Suffice to say that those voting 'Yes' were backing more and deeper democracy. This was most obvious at the national level; they sought to bring power closer to the 5 million people living in Scotland and to boot out the Tories that Scotland never voted in. But the independence movement outside of the SNP also emphasised the opportunities for delivering more radical proposals for local, decentralised decision-making. An important example is Common Weal's 'people's manifesto', a key document for the wider movement, which advocates for policy to be driven by mini-publics, through mechanisms such as citizen assemblies and deliberative polls. If the referendum had been about a narrow nationalistic 'right' to home rule, a silence would now reign over Scotland. Instead, in the fortnight after the 'No' vote, both the Greens and SNP's membership tripled, while Common Weal and the Radical Independence Campaign are realigning their strategies for the May 2015 General Elections. Building a genuinely politically engaged citizenry required only one action: asking a question with genuine consequences for people's lives. It was something that my generation of young people in Britain had never experienced.
When it was announced that sixteen and seventeen year-olds would be given the vote, the first polls confirmed the idea that Scottish teenagers would vote to stay in Britain. Alex Salmond had made a big mistake, the commentators said: of course, young people were against 'nationalism', 'separatism' and the nostalgic, romantic idea of 'freedom from the London yoke'. Even Ian Bell, columnist for the Scottish Herald, the only newspaper in the UK to support independence, published an apologetic letter to 'Young Scotland' after the vote. 'I had assumed that you don't fight a referendum with kids', he admitted, 'But they were, always, the best of us'. Commentators across the board profoundly misunderstood the independence movement, which won over these teenagers, ending with over 90 per cent registering to vote, and 71 per cent voting 'Yes'. I don't believe they would have been brought over if my generation, Generation Y, had not led from the front, using social media and 'boots on the ground', events and direct action, to engage this younger demographic. Back in late 2013, Ipsos Mori brought out a report on what they called the 'Independence Generation'. They found that Generation Y, those born in the 1980s and 1990s, were far more likely to vote for an independent Scotland. By referendum day, nearly a year later, this generation had been joined by the sixteen and seventeen year-olds, newly given the vote, along with those aged between thirty-five and sixty-five. Pensioners, consuming anti-independence newspapers and TV and largely absent from social media, were just too difficult for the independence movement to reach.
Why did this generational dynamic come as a surprise? My theory is that, on a fundamental level, young people like myself are presumed to be 'global' in our outlook and thus uninterested in the 'national'. Anyone under thirty-five has grown up during an era of slow and apparently inevitable dissolution, with the 'nation' weakening along with national democracy. Why would we care about either? During our lives, trans-nationals, NGOs and international governing bodies have expanded their powers, as has the European Union, without strengthening its own forms of democracy. The system of representational democracy has become increasingly unstable and compromised, but nothing has been seriously proposed to replace it. Yet, as we saw with the 2011-2012 social movements, despair at the democratic system can lead to engagement with changing this system. Is it so inconceivable that the same trend might apply to national politics? Yet we find it hard to accept that young people might be interested in redefining and redrawing their national political communities. Rather than seeing this as the next step of endeavouring to claim agency over our national democracies, we at once cry 'nationalism', presuming to find patriotic sentiments of racial and cultural supremacy underlying such a politics.
It is clear that Scotland's young people are not, all of a sudden, majority 'nationalists' in the sense described above. In fact, polling in the week before the vote found that those voting 'No' cited 'national identity' as one of their three main reasons for doing so, while identity did not appear in the polling of those intending to vote 'Yes'. This indicates that a weakening of 'British' identity could have been more of a factor for those supporting Scottish independence than a strengthening of 'Scottish' identity. In turn, this reflects a broader UK-wide trend that has seen younger demographics identifying less strongly with 'Britishness' over the years. Leaving aside Northern Ireland with its sectarian divisions along Republican and Loyalist lines, young people in Scotland and Wales are more likely to identify as only Scottish or Welsh, not British. Perhaps more surprisingly, under thirty-fives in England are more likely to want more powers for England.
Through this framework, the picture of young people's relationship to the nation within the British Isles takes on a new shape. Much has been made of a recent Ipsos Mori report that appeared to find that Generation Y was less supportive of the welfare state than older generations. But the report explicitly warns against such a conclusion, pointing out that there has been a cross-generational fall in support for spending on welfare, while the most marked generational difference is over the statement: 'the creation of the welfare state is one of Britain's proudest achievements'. Eighteen to twenty-five year olds were the least likely to agree with this, while very high levels answered 'don't know'.
This could be read as lack of support for the welfare state, or ambivalence towards the ideas of 'Britishness' and of 'British pride'. Last year, research by the Economist Intelligence Unit into the democratic health of countries across the globe concluded that Britain is 'beset by a deep institutional crisis' with 'trust in government, parliament and politicians at an all-time low'. As the same Ipsos Mori report found, this is particularly true of younger generations, who 'tend to value personal choice more, and traditional institutions less'. To say that this marks Generation Y as a 'selfish' or 'individualistic' generation is to confuse the traditional institutions of the British state with effective, benevolent bodies that accommodate the desires and needs of all British citizens, or indeed that treat the citizens of all nations in any way equally.
To put it bluntly, us 'Millenials', 'Thatcher's children', 'Generation Net', have little concrete experience of Britain as a functioning national community of politics and solidarity. Falling party and trade union memberships have marked our political lifetimes: institutions that once linked politics across borders. We have grown up under a devolution settlement, watching the post-war consensus of the welfare state and public services being dismantled, shrunk, asset stripped or sold off to the private sector. Many young people in Scotland and Wales observe what is taking place in England with horror, and see their national politicians struggling to use their devolved powers to soften the sharper edges of austerity. While Thatcher never dared to touch the health service, the NHS of England and Wales is now divided between state and private service providers, making creeping privatisation one of the key fears pushing people in Scotland to vote in favour of breaking away from Westminster. The drift between the nations of Britain is set to continue, regardless of the 'No' vote in the Scottish referendum.
Over the next decade, it will become increasingly important not to get trapped in a London mentality, which sees movements for self-determination in Britain's regions and nations as 'nationalistic' (meaning patriotic and supremacist) or 'parochial'. British identity is far stronger in London: the capital and Birmingham are the only regions in the UK where over 30 per cent of people identify as only British. A strong feeling of 'Britishness' is now the exception, not the rule, particularly for young people. This does not only concern Scotland, Wales and Northern Ireland, but England too. The think tank IPPR has done a lot of work on the emergence of a politics of Englishness over the last two years and the way in which the Scottish independence referendum has reinforced this trend. The majority of people living in England now want more powers over their country, with polls suggesting younger generations feel slightly more strongly about the need for English democracy. In The Politics of English Nationhood, Michael Kenny explains the rising support for reforming English democracy as linked to 'a return across Europe to forms of belonging associated with historic forms of national and regional identity'. But are young people articulating a stronger sense of identity and 'belonging' than older generations? The general trends common to Generation X and Y might suggest that this is more to do with power and voice: in other words a democratic demand.
It is understandably worrying to the political establishments of our countries that young people's politics may not be as 'global' in outlook as it first may seem. Occupy, certainly in its British manifestation, fitted a particular narrative that can be seen as running parallel to that of neoliberal individualism: that of individuals empowered by technological innovation, in global leaderless networks (just like markets), fostering a kind of borderless form of political identity that left no institutional legacy that could have allowed it to build beyond this. I felt myself to be part of, and am supportive of Occupy, and of course it was (and is) something more than this. The 99 per cent slogan is one of trans-national solidarity, recognising that economic inequality and exploitation, exacerbated and brought into sharp relief by the financial crash, is a global problem requiring a global solution. This is not only a correct analysis, but the only possible moral response to the state of the world today. This does not mean that the 'Umbrella Revolution' in Hong Kong, drawing heavily on student politics and initiated by a group called Occupy Central, cannot advocate global solidarity while focusing its agency on the Chinese and Hong Kong political systems. Would this be any less a movement with a global consciousness and outlook, if it were to advocate the separation of Hong Kong from China as the only way to ensure free and fair elections of nominated and directly elected candidates without interference from the Communist Party of China?
To take a step back, the politics that emerged around Scottish independence should be seen alongside so-called 'networked' and 'globally-minded' social movements like Occupy in its many manifestations. There is less contradiction here than there might at first appear. The independence movement put great emphasis on a 'Yes' vote as the internationalist choice. The removal of Trident was a reason in itself to many voters, but a 'Yes' vote was also positioned as heralding a move away from foreign aggression and militarism. In the longer-term, breaking away from Britain would guarantee that Scotland would remain in the European Union. The internationalist vision of the 'Yes' movement was a powerful rallying call for Generation Y, many of whom were politicised by the 2 million anti-Iraq march in 2003 and its dismissal by the Blair government. Across Britain, Generations X and Y are more pro-European than older demographics, and have arguably more to lose with the prospect of an end to free movement across European borders and the ability to work, live and study on the continent with comparative ease.
In a recent article, the journalist and author Paul Mason calls Scotland and Catalonia 'straws in the wind for the whole of Europe'. He points to recent OECD predictions for the next fifty years, which see the economies of European countries slowing and stagnating unless two main changes occur: mass inward migration to balance ageing populations; and state-directed innovation that 'ideally solves the energy problem'. He argues that big states will find this difficult to do, hence the rationale for small-state separation. 'It is not just that big states are unwieldy', he says. 'Old developed countries such as Britain and Spain have political elites aligned with economic interests that do not favour state-funded innovation, high immigration or sustainable energy'. The SNP's White Paper, their 'blueprint for independence', placed a great deal of emphasis on encouraging skilled migrants to settle, while their economic programme promised state investment and the development of a 'new green economy'. Much of the wider independence movement went further in proposals for more radical environmental policy and the further opening of Scottish borders. Whether you agree with the 'Yes' campaign or not, that it presented a long-term vision for the future cannot be denied. It is not surprising that young people should vote for the promise of a politics that is vocal about combatting long-term problems and preventing future crises, which they may see in their lifetimes.
Much fun has been poked at supporters of Scottish independence through comparisons between Alex Salmond and Nigel Farage (both are strong, charismatic figures, both want state autonomy and the repatriation of powers). However, many 'Yes' voters saw independence as a way to break free from a country defined by UKIP's vision of the future. UKIP, an anti-immigration party, reliant on a battalion of grey votes, with its pathological hatred of wind farms and all things 'green', is short-term politics at its most dangerous. The Westminster political class are at once fascinated by UKIP and over-willing to bend over backwards to bring back their older voters, exacerbating a vicious cycle of youth disengagement. The fact that young people are in general repelled by UKIP's politics of ethnic nationalism – reliant on nostalgia, intolerance and the desire for ethnic and cultural purity – is a reason to be hopeful for Britain's future.
A major factor is that Generations X and Y reflect the general trend of a decrease in deference to authority, tradition and hierarchy. We are not searching for a 'lost Britain' that we have never known. It is also a positive rejection, in which the Scottish referendum played a part, of a politics of hatred and fear, one that inevitably arises in times of financial hardship and unease. We can't rely on this forever. The majority of young people in Britain believe they will be 'worse off than their parents'. Under thirty-fives, disproportionately under and unemployed, with squeezed wages, in rented accommodation and leading precarious lives, could yet prove good hunting ground for Nigel Farage. Surely, this group also needs a scapegoat to explain the sudden change in their fortunes, caused by economic and socio-political global factors that are overwhelming in their complexity? In Matthew Goodwin's book Revolt on the Right he identifies UKIP's voter base as built on the 'left-behind' social groups. These are mainly old, white, working-class and lower middle-class males. Only 15 per cent of UKIP members are under forty. Yet young people in Britain today also face uncertainties as to their identity and role in society.
This brings us to the overall vision of an independent Scotland as presented by the movement: that of a 'fairer' and 'more equal' society. Again, we won't deal here with how deliverable this vision was, but only with the kind of political alliance it inspired, between voters from lower income groups, and voters from lower age groups. Under thirty-fives have suffered disproportionately at the hands of the austerity program implemented following the 2008 financial crash, so younger demographics would inevitably be more willing to take risks with the economy in order to combat growing inequality and desperation, a perspective that would be shared with the most vulnerable members of all age groups. Immigrant communities were also more likely to vote Yes. This aligns loosely with the composition of the 'precariat' as set out in Guy Standing's book, The Precariat: The New Dangerous Class. Standing envisions an emerging group with shared interests, spanning the unemployed and under-employed young, the 'graduate without a future', the migrant population and communities formerly belonging to the traditional working class. Standing references Occupy within his analysis of an emerging politics of the precariat, but much of this discourse could equally be applied to the movement for Scottish independence. 'Youth politics' is not simply a question of the attitudes taken by those people who happen to be aged under thirty-five. Young people are leading the demand for a politics of the future, rejecting the current political and economic consensus as not only unfair but also unsustainable. In Britain, this may well continue to take the form of a re-engagement with the political life of the nations and regions, given that the Westminster establishment has otherwise proved disastrously resistant to change.
Growing up under a particular kind of neoliberal globalisation does not mean that under thirty-fives conform to this way of thinking and being. Constructing identities through consumption of global culture, lifestyle choices, sexual orientations and trans-national political movements, may imply that younger demographics have turned their back on all forms of collective solidarity rooted in place or national democracy. But this is only a pragmatic reaction to a political and social landscape where the nation-state's power has weakened. It does not mean that the politics of nationhood has become irrelevant. There is now a vacuum, which many different forces, parties and movements are rushing to fill.
Younger demographics in Britain have so far approached the nation on democratic lines. In Scotland, young people have pushed hard to rejuvenate civil society and democracy in their nation, seeking independence in order to ensure greater solidarity with those living in Scotland and beyond its borders, through a return to the moral compass of social democracy and moving against aggressive foreign policy. The referendum in September 2014 led many young people to recognise their shared interests with migrants and the economically disadvantaged of all ages, whether or not they were aware of this in the framework of 'precarity' or of a 'precarious' class. UKIP approaches the nation on ethnic lines, and addresses the anxiety of an uncertain future by finding scapegoats to blame. The party has thus far not come near the youth vote, despite rising disaffection, disadvantage and resentment amongst younger demographics following the implementation of austerity.
We only need to look across the waters to Europe to see that we can't take this dynamic within Britain for granted. Independence movements that attract young people as supporters and leaders are not necessarily as clear-sighted and inclusive in their approach as the Scottish 'Yes' movement. In fact, it may turn out to have been an exceptional moment. Europe is heading into dangerously unpredictable times. The majority of young people who don't exercise their vote at the ballot box want real change – not the 'switcheroo' of political parties but new forces and movements, constitutional alignments and new states. At the time of writing, Catalonia looks likely to hold an unofficial referendum on independence, again led by young people who have borne the brunt of austerity in Spain, with a 55 per cent youth unemployment rate. Independence movements have emerged or are gaining ground in the Basque country and Venetia in Italy, as well as in Ukraine, to mention some of the most notable. Young people's precarity, under- and un-employment and uncertain future leave us vulnerable to the appeal of the far right and arguments for a 'return to the nation' based on ethnic, rather than democratic lines.
Finally, my generation faces immense challenges that require international cooperation on an unprecedented scale. How we are able to confront these will in part depend on our relationship to the nation-state, whose role is in transition. The Scottish independence referendum showed that, where demands for democratic revival and constitutional change are met with a genuine outlet for expression, young people respond with peaceful, wide-scale and transformative engagement. In Britain, the weakness and instability of the Union, and the detachment of younger demographics from the 'British' identity, should not mean that we ignore or dismiss the politics of nationhood. This will be crucial in how we respond to and influence the volatile politics that is set to reshape Britain and Europe, even if our ultimate aim is a borderless world.
An informative and interesting discussion of the local, Scottish, manifestations of global effects. The discourse of market forces which legitimates the industrial vandalism that afflicts the entire UK (but especially central Scotland) warrants a further piece by this gifted author. I would find her assessment of how Gen Y deconstructs the discourse very interesting.
In Wales, despite an interest in what has been happening in Scotland, there hasn't been anything like the same amount of political engagement over the past couple of years – and the election campaign as a Welsh election simply didn't happen. But with a majority Conservative government short of voices in Wales, the near future would be a great time for sharing the experience of democratic growth the Scots have achieved.
I've put forward a few initial ideas on the blog for how we in Wales might begin to feel our way to a more significant democratic movement and alternative media movement in immediate response to the redoubled austeriarchy and centralism of a majority Conservative government.


text:
The Alabama Gulf Coast Zoo opened in June 1989 in Gulf Shores, Alabama, United States as Zooland Animal Park. It is operated by the Zoo Foundation, Inc., a 501(c)(3) non-profit organization, and relies on admissions, memberships and contributions for its funding. It was the basis of the Animal Planet show The Little Zoo That Could.

The zoo sits on approximately 25 acres and lies about two miles north of the Intracoastal Waterway and is home to more than 300 animals including Barbary lions, leopards, tigers, wolves, bears, monkeys, and macaws. It includes a petting zoo, reptile house, and aviary, as well as daily animal shows in the summer. The zoo opened its new location in early 2020.

History

The Alabama Gulf Coast Zoo started as the private project of Robin Willis and Joey Ward of Gulf Shores, and opened in June 1989 as Zooland Animal Park. The Ward family created the Zoo Foundation Inc. (a 501(c)(3) non-profit organization) in 1991 to operate the zoo, at the same time donating the  that make up the zoo today. An additional , which are still undeveloped, was donated by the Ward family and the Erie Meyer Foundation in 1994.

In July 1997, Hurricane Danny hit Gulf Shores. Then in 1998, Hurricane Georges hit, causing more flood damage. The zoo evacuated all of its animals in both cases, transporting them  inland to higher ground and becoming the first zoo to have a full-scale evacuation during a hurricane. These expensive evacuations, and the repairs following the hurricanes, almost caused the zoo to close in 1999, but fund raising efforts were successful in raising enough money to keep it open. Hurricane Ivan in 2004 caused $500,000 in damage, forcing another evacuation and closing the zoo for 14 months. Several animals were not caught during this evacuation, and were lost. More evacuations resulted from Dennis and Katrina in 2005, but the zoo was able to reopen fairly quickly in October 2005.

The Little Zoo that Could

The flurry of activity around the zoo attracted media attention during Hurricane Ivan, and the zoo was approached about doing a series for Animal Planet. Shooting for the series started in 2005, and the first of thirteen episodes of The Little Zoo that Could aired in February 2006. The series focused on the zoo's efforts to reopen after being battered by three major hurricanes in 2004 and 2005: Hurricane Ivan, Hurricane Dennis, and Hurricane Katrina. A one-hour follow-up on the series aired in February 2007. The zoo announced that they are in the process of making a new show, suspected to air about the same time the new zoo opens.

The future

The local community and businesses have donated money, building supplies, and services in the past. On-going funding through sponsorships, grants, and endowments from major corporations did not exist before 1998, but are now being pursued to ensure the zoo's future.

In 2009, the zoo unveiled a plan to move to new quarters that are farther inland and less susceptible to flooding. The  for the new zoo facilities were donated by Clyde Weir and his daughter Andrea Weir Franklin. The new facilities were scheduled to open in early 2013.
The new zoo reopened on March 11, 2020.

Notes

External links

Zoos in Alabama
Buildings and structures in Baldwin County, Alabama
Protected areas of Baldwin County, Alabama
Tourist attractions in Baldwin County, Alabama
Zoos established in 1989
1989 establishments in Alabama

redpajama_set_name: RedPajamaWikipedia


Rather than sit down to ponder a follow-up to his first official solo album (Neighbor Singing, released Tuesday on Hometapes), Medicine member Brad Laner has chosen to score a film. Eric Green's Beautiful Noise is a documentary about shoegazer music that follows such likely acts as My Bloody Valentine, The Jesus and Mary Chain, The Cocteau Twins, as well as contemporary bands like Serena-Maneesh and Ladytron, who are still incorporating elements of shoegaze into their music.
It remains to be seen if Green can tell us something new about the genre that started a movement (and eventual backlash) of distortion, droning guitars, and muted vocals. Early buzz is positive, and he is said to be focusing on the late-'80s and early-'90s, during the genre's "first wave."
Laner has been commissioned to write the film's soundtrack. His band Medicine is the only American outfit profiled in the film.
Download "Out Cold" by Brad Laner.


If you have read the previous post related to King's newest game, Farm Heroes Super Saga, it seems to be the next popular series of Farm Heroes Saga franchise.
Therefore, just take the time and look the board carefully where the next crop is coming from.
It is caused by, you will get more chance for other crops to move around the map.
Here, you will also have to beat boss battles, Rancid the Raccoon, who throws cans when you try to make a big match.
You can do this by matching crops next to them multiple times.
And, if you can create a T style shape of matches, you will boost the quota of crops adjoining to them.
Moreover, if you can match four crops in a square formation, it will come to a much bigger crop, which will give you more points.
Doing this will allow you to take out the whole objective in one move.
Plus, matching 5 crops in a row will clear all of the crops of the same kind.
On the other side, wind or weather system will also take a big role to change everything during in your performance of matching up those crops on the board.
The wind will also change if you move crops from the left, right, up, or down in each stage.


text:
\section{Introduction} \label{Introduction}

In recent years, reinforcement learning (RL) algorithms have impacted a vast range of real-world tasks, such as robotics \citep{andrychowicz2020learning, kober2013reinforcement}, power management
\citep{xiong2018power}, autonomous control \cite{bellemare2020autonomous}, traffic control \citep{abdulhai2003traffic1, wiering2000multi}, and more \citep{mahmud2018applications,luong2019applications2}. These achievements are possible by the ability of RL agents to learn a behavior policy via interaction with the environment while simultaneously maximizing the obtained reward.

A common family of algorithms in RL is Q-learning \citep[QL]{watkins1992q} based algorithms, which focuses on learning the value-function.
The value represents the expected, discounted, reward-to-go that the agent will obtain. In particular, such methods learn the optimal policy via an iterative maximizing bootstrapping procedure. A well-known property in QL is that this procedure results in a positive estimation bias. \citet{hasselt2010double} and \citet{van2016deep} argue that over-estimating the real value may negatively impact the performance of the obtained policy.
They propose Double Q-learning (DQL) and show that, as opposed to QL, it has a negative estimation bias.

The harmful effects of \emph{any} bias on value-learning are widely known.
Notably, the deadly triad \citep{sutton2018reinforcement,van2018deep} attributes the procedure of bootstrapping with biased values as one of the major causes of divergence when learning value functions. This has been empirically shown in multiple works \citep{van2016deep, van2018deep}. Although this phenomenon is well known, most works focus on avoiding positive-estimation instead of minimizing the bias itself \citep{van2016deep,wang2016dueling,hessel2018rainbow}.

In this work, we tackle the estimation bias in an underestimation setting. We begin, similar to \citet{hasselt2010double}, by analyzing the estimation bias when observing a set of i.i.d.\ random variables (RVs). We consider three schemes for estimating the maximal mean of these RVs. \textit{(i)} The \emph{single-estimator}, corresponding to a QL-like estimation scheme. \textit{(ii)} The \emph{double-estimator}, corresponding to DQL, an estimation scheme that splits the samples into two equal-sized sets: one for determining the index of the RV with the maximal mean and the other for estimating the mean of the selected RV.
\textit{(iii)} The \emph{ensemble-estimator}, our proposed method, splits the data into $K$ sets. Then, a single set is used for estimating the RV with the maximal-mean whereas all other sets are used for estimating the mean of the selected RV.

We start by showing that the ensemble estimator can be seen as a double estimator that unequally splits the samples (\cref{prop:proxy}). This enables, for instance, less focus on maximal-index identification and more on mean estimation. We prove that the ensemble estimator under-estimates the maximal mean (\cref{lemma:EE underestimation}). We also show, both theoretically (\cref{prop:split_ratio}) and empirically (\cref{fig:MSE as func of r}), that in order to reduce the magnitude of the estimation error, a double estimator with equally-distributed samples is sub-optimal.

Following this theoretical analysis, we extend the ensemble estimator to the multi-step reinforcement learning case. We call this extension Ensemble Bootstrapped Q-Learning{} (EBQL). We analyze EBQL{} in a tabular meta chain MDP and on a set of ATARI environments using deep neural networks. In the tabular case, we show that as the ensemble size grows, EBQL{} minimizes the magnitude of the estimation bias.
Moreover, when coupled with deep neural networks, we observe that EBQL{} obtains superior performance on a set of ATARI domains when compared to Deep Q-Networks (DQN) and Double-DQN (DDQN).

\textbf{Our contributions are as follows:}
\begin{itemize}
    \item We Analyze the problem of estimating the maximum expectation over independent random variables. We prove that the estimation mean-squared-error (MSE) can be reduced using the Ensemble Estimator. In addition, we show that obtaining the minimal MSE requires utilizing more than two ensemble members.
    \item Drawing inspiration from the above, we introduce Ensemble Bootstrapped Q-Learning{} (EBQL) and show that it reduces the bootstrapping estimation bias.

    \item We show that EBQL{} is superior to both Q-learning and double Q-learning in both a tabular setting and when coupled with deep neural networks (ATARI).
\end{itemize}

\section{Preliminaries} \label{Preliminaries}
\subsection{Model Free Reinforcement Learning}

A reinforcement learning \cite{sutton2018reinforcement} agent
faces a sequential decision-making problem in some unknown or partially known environment. We focus on environments in which the action space is discrete.
The agent interacts with the environment as follows. At each time $t$, the agent observes the environment state $s_t \!\in\! \calig{S}$ and selects an action $a_t \!\in\! \calig{A}$. Then, it receives a scalar reward $r_t \!\in\! \mathbb{R}$ and transitions to the next state $s_{t+1} \!\in\! S$ according to some (unknown) transition kernel $P(s_{t+1} | s_t, a_t)$.
A policy $\pi:\calig{S}\to\calig{A}$, is a function that determines which action the agent should take at each state and the sampled performance of a policy, a random variable starting from state $s$, is denoted by
\begin{align}
    R^\pi (s) = \sum_{t=0^\infty} \gamma^t r_t | s_0 = s, a \sim \pi (s_t) \, \label{eq:reward-to-go}.
\end{align}
where $\gamma \in [0, 1)$ is the discount factor, which determines how myopic the agent should behave.
As $R^\pi$ is a random variable, in RL we are often interested in the expected performance, also known as the policy value,
\begin{align*}
    v^\pi (s) = \mathbb{E}^\pi \sbrac{\sum_{t=0}^\infty \gamma^{t} r(s_{t}) | s_0 = s}  \, .
\end{align*}
The goal of the agent is to learn an optimal policy $\pi^*\in\mathop{\mathrm{argmax}}_\pi v^\pi(s)$. Importantly, there exists a policy $\pi^*$ such that $v^{\pi^*}(s)=v^*(s)$ for all $s\in\calig{S}$ \citep{sutton2018reinforcement}.

\subsection{Q-Learning}
One of the most prominent algorithms for learning $\pi^*$ is Q-learning \citep[QL]{watkins1992q}. Q-learning learns the Q-function, which is defined as follows
\begin{equation*}
    Q^\pi(s,a) = \mathbb{E}^\pi \sbrac{\sum_{t=0}^\infty \gamma^tr_t(s_t,a_t) |s_0=s, a_0=a} \, .
\end{equation*}
Similar to the optimal value function, the optimal Q-function is denoted by $Q^*(s,a)= \max_\pi Q^\pi (s,a), \ \forall (s,a) \!\in\! \calig{S}\!\times\! \calig{A}$.
Given $Q^*$, the optimal policy $\pi^*$ can obtained by using the greedy operator $\pi^*(s) = \mathop{\mathrm{argmax}}_a Q^*(s,a)\ \forall s \in \calig{S}$.

QL is an off-policy algorithm that learns $Q^*$ using transition tuples $\brac{s, a, r, s'}$. Here, $s'$ is the state that is observed after performing action $a$ in state $s$ and receiving reward $r$. Specifically, to do so, QL iteratively applies the optimal Bellman equation \citep{bellman1957markovian}:
\begin{equation}\label{eqn: bellman}
    Q(s,a) \leftarrow (1-\alpha)Q(s,a) + \alpha (r_t + \gamma \max_{a'}Q(s',a')) \, .
\end{equation}
Under appropriate conditions, QL asymptotically converges almost surely to an optimal fixed-point solution \cite{tsitsiklis1994asynchronous}, i.e. $Q_t(s,a) \xrightarrow{t\rightarrow \infty} Q^*(s,a) \ \forall s\in\calig{S},a\in\calig{A}$.

A known phenomenon in QL is the over-estimation of the Q-function. The Q-function is a random variable, and the optimal Bellman operator, \cref{eqn: bellman}, selects the maximal value over all actions at each state. Such estimation schemes are known to be positively biased \citep{smith2006optimizer}.

\subsection{Double Q-Learning}
A major issue with QL is that it overestimates the true Q values. \citet{hasselt2010double, van2016deep} show that, in some tasks, this overestimation can lead to slow convergence and poor performance.
To avoid overestimation, Double Q-learning \citep[DQL]{hasselt2010double} was introduced. DQL combines two Q-estimators: $Q^A$ and $Q^B$, each updated over a unique subset of gathered experience.
For any two estimators $A$ and $B$, the update step of estimator $A$ is defined as (full algorithm in the appendix \cref{alg:DQL}):
\begin{align}
    & \hat{a}^*_A = \mathop{\mathrm{argmax}}_{a'} Q^A(s_{t+1}, a')\nonumber\\
    & Q^A(s_t,a_t) \gets (1-\alpha_t)Q^A(s_t,a_t)
    \nonumber\\
    &\qquad\qquad\qquad +\alpha_t \brac{r_t + \gamma Q^B\left(s_{t+1},\hat{a}^*_A\right)}. \label{eq:DQL update}
\end{align}
Importantly, \citet{hasselt2010double} proved that DQL, as opposed to QL, leads to underestimation of the Q-values.

\section{Related Work} \label{related work}

\textbf{Q-learning stability.} Over the last three decades, QL has inspired a large variety of algorithms and improvements, both in the tabular setting \citep{strehl2006pac, kearns1999finite, ernst2005tree, azar2011speedy} and in the function approximation-based settings \citep{schaul2015prioritized,bellemare2017distributional,hessel2018rainbow,jin2018q,badia2020agent57}.
Many improvements focus on the estimation bias of the learned Q-values, first identified by \citet{thrun1993issues}, and on minimizing the variance of the target approximation-error \citep{anschel2017averaged}.
Addressing this bias is an ongoing effort in other fields as well, such as economics and statistics \cite{smith2006optimizer, thaler2012winner}.
Some algorithms \citep{hasselt2010double, zhang2017weighted} tackle the QL estimator inherent bias by using double estimators, resulting in a \textit{negative} bias.

RL algorithms that use neural-network-based function approximators (deep RL) are known to be susceptible to overestimation and oftentimes suffer from poor performance in challenging environments.
Specifically, estimation bias, a member of the \textit{deadly triad} \citep{van2018deep}, is considered as one of the main reasons for the divergence of QL-based deep-RL algorithms.

In our work, we tackle the estimation bias by reducing the MSE of the next-state Q-values. While EBQL{} is negatively biased, we show that it better balances the bias-variance of the estimator. We also show that the bias magnitude of EBQL{} is governed by the ensemble size and reduces as the ensemble size grows.

\textbf{Ensembles.}
One of the most exciting applications of ensembles in RL \citep{osband2018randomized, lee2020sunrise} is to improve exploration and data collection. These methods can be seen as a natural extension of Thompson sampling-like methods to deep RL.
The focus of this work complements their achievements. While they consider how an ensemble of estimators can improve the learning process, we focus on how to better train the estimators themselves.

\section{Estimating the Maximum Expected Value}
As mentioned in \cref{Preliminaries}, QL and DQL display opposing behaviors of over- and under-estimation of the Q-values. To better understand these issues, and our contribution, we take a step back from RL and statistically analyze the different estimators on i.i.d.\ samples.

Consider the problem of estimating the \textit{maximal expected value} of $m$ independent random variables $\brac{X_1, \dots ,X_m}\triangleq X$, with means  $\mu \!=\! \brac{\mu_1, \dots ,\mu_m}$ and standard deviations $\sigma \!=\! \brac{\sigma_1, \dots ,\sigma_m}$.
Namely, we are interested in estimating
\begin{align} \label{eq:max_exp_X}
    \max_a \mathbb{E} \sbrac{X_a} = \max_a \mu_a \triangleq \mu^*\enspace.
\end{align}
The estimation is based on i.i.d.\ samples from the same distribution as $X$. Formally, let $S = \cbrac{S_a}_{a=1}^m$ be a set of samples, where $S_a=\cbrac{S_a(n)}_{n=1}^{N}$ is a subset of $N$ i.i.d.\ samples from the same distribution as $X_a$. Specifically, this implies that $\mathbb{E}\sbrac{S_a(n)} = \mu_a$ for all $n\in \sbrac{N_a}$.
We further assume all sets are mutually independent.

Denote the empirical means of a set $S_a$ by $\hat{\mu}_a(S_a) =
\frac{1}{N_a}\sum_{n \in [N_a]} S_a(n)$.
This is an unbiased estimator of $\mu_a$, and given sufficiently many samples, it is reasonable to approximate $\hat{\mu}_a(S_a) \approx \mu_a$. Then, a straightforward method to estimate \eqref{eq:max_exp_X} is to use the \textit{Single-Estimator} (SE):
\begin{align*}
\hat{\mu}_{\text{SE}}^* \triangleq \max_a \hat{\mu}_a(S_a) \approx \max_a \mathbb{E} \sbrac{\hat{\mu}_a(S_a)} = \mu^*~.
\end{align*}
As shown in \cite{smith2006optimizer} (and rederived in \cref{proof:se_unbiased} for completeness), this estimator is positively biased.
This overestimation is believed to negatively impact the performance of SE-based algorithms, such as QL.

To mitigate this effect, \citep{hasselt2010double} introduced the \textit{Double Estimator} (DE). In DE, the samples of each random variable $a\in[m]$ are split into two disjoint, equal-sized subsets $S_a^{(1)}$ and $S_a^{(2)}$, such that $S_a^{(1)} \cup S_a^{(2)} = S_a$ and $S_a^{(1)} \cap S_a^{(2)} = \emptyset$ for all $a\in\sbrac{m}$. For brevity, we denote the empirical mean of the samples in $S_a^{(j)}$ by $\hat{\mu}^{(j)}_a \triangleq\hat{\mu}_a(S^{(j)}_a)$. Then, DE uses a two-phase estimation process:
In the first phase, the \textbf{index} of the variable with the maximal expectation is estimated using the empirical means of $S^{(1)}$, $\hat{a}^* = \mathop{\mathrm{argmax}}_a \hat{\mu}^{(1)}_a$.
In the second phase, the \textbf{mean} of $X_{\hat{a}^*}$ is estimated using $S^{(2)}_{\hat{a}^*}$, $\hat{\mu}^*_{\text{DE}} = \hat{\mu}^{(2)}_{\hat{a}^*}$.
The resulting estimator is negatively biased, i.e., $\mathbb{E} \sbrac{\hat{\mu}^*_{\text{DE}}} \leq \mu^*$ \citep{hasselt2010double}.

\subsection{The Ensemble Estimator}\label{subsec: ensemble estimator}
In this section, we introduce the \textit{Ensemble Estimator} (EE), a natural generalization of DE to $K$ estimators.
We take another step forward and ask:
\begin{center}
    \textit{How can we benefit by using $K$ estimators rather than 2?}
\end{center}

In the DE, two sources are affecting the approximation error. One type of error rises when the wrong index $\hat{a}^*$ is selected (inability to identify the maximal index). The second, when the mean is incorrectly estimated.
While increasing the number of samples used to identify the index reduces the chance of misidentification, it results with fewer samples for mean estimation. Notably, the DE na\"ively allocates the same number of samples to both stages. As we show, the optimal ``split" is not necessarily equal, hence, in an attempt to minimize the total MSE, a good estimator should more carefully tune the number of samples allocated to each stage.

Consider, for example, the case of two independent random variables $(X_1,X_2) \sim \brac{\calig{N}(\mu_1, \sigma^2), \calig{N}(\mu_2, \sigma^2)}$.
When the means $\mu_1$ and $\mu_2$ are dissimilar, i.e., $\abs{\mu_1-\mu_2}/\sigma \gg 1$, the chance of index misidentification is low, and thus, it is preferable to allocate more samples to the task of mean estimation.
On the other extreme, as $\abs{\mu_1-\mu_2}/\sigma \rightarrow 0$, the two distributions are effectively identical,
and the task of index identification becomes irrelevant. Thus, most of the samples should be utilized for mean estimation.
Interestingly, in both these extremities, it is beneficial to allocate more samples to mean estimation over index identification.

As argued above, minimizing the MSE of the estimator can be achieved by controlling the relative number of samples in each estimation phase. We will now formally define the ensemble estimator and prove that it is equivalent to changing the relative allocation of samples between index-estimation and mean-estimation (\cref{prop:proxy}). Finally, we demonstrate both theoretically and numerically that EE achieves lower MSE than DE, and by doing so, better balances the two sources of errors.

Ensemble estimation is defined as follows:
While the DE divides the samples into two sets, the EE divides the samples of each random variable $a\in[m]$ into $K$ equal-sized disjoint subsets, such that $S_a = \bigcup_{k=1}^K S_a^{(k)}$ and $S_a^{(k)}\! \cap S_a^{(l)} = \emptyset$, $\forall k\neq l \in \sbrac{K}$.
We further denote the empirical mean of the $a^{th}$ component of $X$, based on the $k^{th}$ set, by $\hat{\mu}_a^{(k)}$.
As in double-estimation, EE uses a two-phase procedure to estimate the maximal mean.
First, a single arbitrary set $\tilde{k}\in[K]$ is used to estimate the \textbf{index} of the maximal expectation,
$\hat{a}^* = \mathop{\mathrm{argmax}}_a \hat{\mu}_a^{(\tilde{k})}$.
Then, EE utilizes the remaining ensemble members to jointly estimate
$\mu_{\hat{a}^*}$, namely,
\begin{align*}
    \hat{\mu}_{\text{EE}}^* &\triangleq
    \frac{1}{K-1}\sum_{j \in [K] \backslash \tilde{k}}\hat{\mu}_{\hat{a}^*}^{(j)} \\
    &\approx \max_a \mathbb{E} \sbrac{\frac{1}{K-1}\sum_{j \in [K] \backslash \tilde{k}} \hat{\mu}_a^{(j)}

    =\mu^*~.
\end{align*}
\cref{lemma:EE underestimation} shows that by using EE, we still maintain the underestimation nature of DE.
\begin{restatable}{lemma}{SEbias}
\label{lemma:EE underestimation}
Let $\calig{M} =\mathop{\mathrm{argmax}}_a \mathbb{E} \sbrac{{X_a}}$
be the set of indices where $\mathbb{E}[X_a]$ is maximized.
Let $\hat{a}^*\in\mathop{\mathrm{argmax}}_a \hat{\mu}^{\brac{\tilde{k}}}_a$ be the estimated maximal-expectation index according to the $\tilde{k}^{th}$ subset.
Then
$\mathbb{E} \sbrac{\hat{\mu}_{\text{EE}}^*}  = \mathbb{E} \sbrac{\mu_{\hat{a}^*}} \leq \max_a \mathbb{E} \sbrac{X_a}$.
Moreover, the inequality is strict if and only if $P\brac{\hat{a}^* \notin \calig{M}} > 0$.
\end{restatable}
The proof can be found in \cref{proof:lemma1}.
In addition, below, we show that changing the size of the ensemble is equivalent to controlling the partition of data between the two estimation phases. As previously mentioned, the split-ratio greatly affects the MSE, and therefore, this property enables the analysis of the effect of the ensemble size $K$ on the MSE.

\begin{restatable}[The Proxy Identity]{proposition}{proxyIdentity}\label{prop:proxy}
 Let $S^{(1)},\dots,S^{(K)}$ be some subsets of the samples of equal sizes $N/K$ and let $\tilde{k}\in\sbrac{K}$ be an arbitrary index. Also, denote by $\hat{\mu}_{\text{EE}}^*$, the EE that uses the $\tilde{k}^{th}$ subset for its index-estimation phase.
Finally, let $\hat{\mu}_{\text{W-DE}}^*$ be a DE that uses $S^{(\tilde{k})}$ for its index-selection and all other samples for mean-estimation; i.e., $\hat{a}^* = \mathop{\mathrm{argmax}}_a \hat{\mu}^{(\tilde{k})}_a$ and $\hat{\mu}_{\text{W-DE}}^*=\hat{\mu}_{\hat{a}^*}\brac{\bigcup_{j \in [K]\backslash \tilde{k}} S^{(j)}}$. Then, $\hat{\mu}_{\text{EE}}^* = \hat{\mu}_{\text{W-DE}}^*$.
\end{restatable}

We call the weighted version of the DE that uses $1/K$ of its samples to the index estimation W-DE. The proxy identity establishes that W-DE is equivalent to the ensemble estimator.
Specifically, let $N_1$ be the number of samples used in the first phase of W-DE (index estimation) and say that the MSE of the W-DE is minimized at $N_1 = N_1^*$. Then, by the proxy-identity, the optimal ensemble size of the EE is $K \approx N/N_1^*$. Thus, the optimal split-ratio of W-DE serves as a \textit{proxy} to the number of estimators to be used in EE.

To better understand the optimal split-ratio, we analyze the MSE of the EE. To this end, we utilize the proxy identity and instead calculate the MSE of the W-DE with $N_1\approx N/K$ samples for index-estimation. Let $X$ be a vector with independent component of means $\brac{\mu_1, \dots, \mu_m}$ and variances $\brac{\sigma_1^2, \dots, \sigma_m^2}$. Assume w.l.o.g.\ that the means are sorted such that $\mu_1 \ge \mu_2 \ge \dots \ge \mu_m$. Then, the following holds (see derivations in Appendix \ref{Ensemble estimator MSE}):
    \begin{align*}
    &\mathrm{bias}(\hat{\mu}^*_{\text{W-DE}})=
    \sum_{a=1}^m \brac{\mu_a-\mu_1} P(\hat{a}^*=a), \\
    &\mathrm{var}(\hat{\mu}^*_{\text{W-DE}})
    =\sum_{a=1}^m \brac{ \frac{\sigma_a^2}{N-N_1}+ \mu_a^2 } P(\hat{a}^*=a) \\
    &\qquad\qquad\qquad- \brac{\sum_{a=1}^m \mu_a P(\hat{a}^*=a)}^2.
    \end{align*}

    As $\mu_1$ is assumed to be largest, the bias is always negative; hence, EE underestimates the maximal mean.
    Furthermore, we derive a closed form for the MSE:
    \begin{align} \label{eq:mse}
    \mathrm{MSE}&\brac{\hat{\mu}^*_{\text{W-DE}}}  \nonumber\\
    &= \sum_{a=1}^m\brac{\frac{\sigma_a^2}{N-N_1} + (\mu_1-\mu_a)^2}P\brac{\hat{a}^*=a}.
    \end{align}
We hypothesize that in most cases, the MSE is minimized for $K>2$.
As \eqref{eq:mse} is rather cumbersome to analyze, we prove this hypothesis in a simpler case and demonstrate it numerically for more challenging situations.
\begin{restatable}{proposition}{propSplitRatio}\label{prop:split_ratio}
    Let
    $X=(X_1,X_2)\sim \calig{N}\brac{(\mu_1,\mu_2)^T,\sigma^2I_2}$ be a Gaussian random vector
    such that $\mu_1\ge\mu_2$
    and let $\Delta=\mu_1-\mu_2$. Also, define the signal to noise ratio as $\mathrm{SNR}=\frac{\Delta}{\sigma/\sqrt{N}}$ and let $\hat{\mu}^*_{\text{W-DE}}$ be a W-DE that uses $N_1$ samples for index estimation. Then, for any fixed even sample-size $N>10$ and any $N_1^*$ that minimizes $\mathrm{MSE}(\hat{\mu}^*_{\text{W-DE}})$, it holds that
    \begin{enumerate*}
        \item[(1)] \label{large_snr}As $\mathrm{SNR} \to \infty$, $N_1^*\to 1$
        \item[(2)] \label{small_snr}As $\mathrm{SNR} \to 0$, $N_1^*\to 1$
        \item[(3)] \label{any_case_EE_better}For any $\sigma$ and $\Delta$, it holds that $N_1^*<N/2$.
    \end{enumerate*}
\end{restatable}
The proof can be found in \cref{proof:prop2}. Note that a similar analysis can be done for sub-Gaussian variables, using standard concentration bounds instead of using $\Phi$. However, in this case, we have to bound $P(\hat{a}^*=a)$ and can only analyze an upper bound of the MSE.

\begin{figure}[t]
\begin{center}
\centerline{\includegraphics[width=0.9\columnwidth]{figures/mse_as_func_of_R.png}}
\caption{The MSE values as function of the split-ratio for two Gaussians with different mean-gaps $\Delta=\mu_1-\mu_2$ and $\sigma^2=0.25$. Stars mark values in which the minimum MSE is achieved. As expected from \cref{prop:split_ratio}, the optimal split-ratio is always smaller than $N/2$. The black arrows show the trend of the optimum as $\Delta$ increases. Notably, the optimal split-ratio increases from $0$ to $0.4$ and then decreases back to $0$, with alignment to the $\mathrm{SNR}$ claims.
}
\label{fig:MSE as func of r}
\end{center}
\end{figure}

\begin{figure}[t]
\begin{center}
\centerline{\includegraphics[width=0.9\columnwidth]{figures/optimal_r_as_func_of_delta_num.png}}
\caption{
Numerical calculation of the optimal sample split-ratio as function of normalized gap for 2, 4 and 6 Gaussians with means uniformly spread over the interval $\Delta\!=\!\mu_{max}\!-\!\mu_{min}$.
\label{fig:optimal r as func of delta}
\end{center}
\end{figure}

\Cref{prop:split_ratio} yields a rather intuitive, yet surprising result. From the first claim, we learn that when $X_1$ and $X_2$ are easily distinguishable (high $\mathrm{SNR}$), samples should not be invested in index-estimation, but rather allocated to the mean-estimation. Moreover, the second claim establishes the same conclusion for the case where $X_1$ and $X_2$ are indiscernible (low $\mathrm{SNR}$). Then, $X_1$ and $X_2$ have near-identical means, and samples are better used for reducing the variance of any of them. The most remarkable claim is the last one; it implies that that the optimal split ratio is \textit{always} smaller than half. In turn, when $N$ is large enough, this implies that there exists $K>2$ such that the MSE of EE is strictly lower than the one of DE.
To further demonstrate our claim for intermediate values of the $\mathrm{SNR}$, \cref{fig:MSE as func of r} shows the MSE as a function of $N_1$ for different values of $\Delta=\mu_1-\mu_2$ (and fixed variance $\sigma^2=0.25$).

In \cref{fig:optimal r as func of delta} we plot the value of $N_1$ that minimizes the MSE as a function of the normalized-distance of the means, $\frac{\Delta}{\sigma\sqrt{m}}$, for a different number of random variables $m \in \cbrac{2,4,6}$.
The means are evenly spread on the interval $\Delta$, all with $\sigma^2\!=\!0.25$.
\cref{fig:optimal r as func of delta} further demonstrates that using $N_1<\frac{N}{2}$ can reduce the MSE in a large range of scenarios. By the proxy-identity, this is equivalent to using ensemble sizes of $K>2$.
Therefore, we expect that in practice, EE will outperform DE in terms of the minimal MSE (MMSE).

\section{Ensemble Bootstrapped Q-Learning}

In \cref{subsec: ensemble estimator}, we presented the ensemble estimator and its benefits. Namely, ensembles allow unevenly splitting samples between those used to select the index of the maximal component and those used to approximate its mean.
We also showed that allocating more samples to estimate the mean, rather than the index, reduces the MSE in various scenarios.

\begin{figure*}[!ht]
    \centering
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{figures/Correct_Action_rate_mu_-0_2.png}
        \caption{}
        \label{subfig:neg_mu}
    \end{subfigure}%
    ~

       \quad
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{figures/Correct_Action_rate_mu_0_2.png}
        \caption{}
        \label{subfig:pos_mu}
    \end{subfigure}
    ~

      \quad
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{figures/Correct_Action_rate_averaged_across_MDPs.png}
        \caption{}
        \label{subfig:avg_mu}
    \end{subfigure}
    \caption{Correct-action rate from state $A^i$ as function of episode in the Meta Chain MDP.
    \cref{subfig:neg_mu} shows the results of a specific chain-MDP where $\mu_i=-0.2<0$, where in \cref{subfig:pos_mu}, $\mu_i=0.2>0$.
    In \cref{subfig:avg_mu}, the average of 6 symmetric $\mu$-values is presented. All results are averaged over 50 random seeds.}
    \label{fig:results_MCMDP}
\end{figure*}

Although the analysis in \cref{subsec: ensemble estimator} focuses on a statistical framework, a similar operation is performed by the QL agent during training when it bootstraps on next-state values. Specifically, recall that $R^{\pi}(s, a)$ is the reward-to-go when starting from state $s$ and action $a$ (see \cref{eq:reward-to-go}). Then,
denoting $\brac{X_1, \dots, X_m} = \brac{R^{\pi}(s_{t+1}, a_1), \dots, R^{\pi}(s_{t+1}, a_m)}$ and $\brac{\mu_1, \dots, \mu_m} = \brac{Q^{\pi}(s_{t+1}, a_1), \dots, Q^{\pi}(s_{t+1}, a_m)}$, the next-state value used by QL is determined by $\max_a \mu_a$.
As the real $Q$-values are unknown, DQL uses two $Q$ estimators $\brac{\mu_1^{(1)}, \dots, \mu_m^{(1)}} = Q^A$ and $\brac{\mu_1^{(2)}, \dots, \mu_m^{(2)}} = Q^B$ in a way that resembles the DE -- the index is selected as $a^*=\mathop{\mathrm{argmax}}_a \mu_a^{(1)}$ and the value as $\mu^{(2)}_{a^*}$.

Our method, Ensemble Bootstrapped Q-Learning{} (EBQL) presented in \cref{alg:EBQL}, can be seen as applying the concepts of EE to the QL bootstrapping phase by using an ensemble of $K$ Q-function estimators. Similar to DQL, when updating the $k^{th}$ ensemble member in EBQL{}, we define the next-state action as $\hat{a}^* = \mathop{\mathrm{argmax}}_a Q^k (s_{t+1}, a)$. However, while DQL utilizes two estimators, in EBQL{}, the value is obtained by averaging over the remaining ensemble members $Q^{EN \backslash k} = \frac{1}{K-1} \sum_{j \in [K]\backslash k} Q^j(s_{t+1}, \hat{a}^*)$. Notice how when $K=2$ (two ensemble members) we recover the same update scheme as DQL, showing how EBQL{} is a natural extension of DQL to ensembles. In practice, the update is done using a learning rate $\alpha_t$, as in DQL (\cref{eq:DQL update}).

\renewcommand{\algorithmiccomment}[1]{#1}
\begin{algorithm}[t]
  \caption{Ensemble Bootstrapped Q-Learning{} (EBQL)}
  \label{alg:EBQL}
\begin{algorithmic}
  \STATE {\bfseries Parameters:} learning-rates: $\cbrac{\alpha_t}_{t\ge1}$
  \STATE {\bfseries Initialize:} Q-ensemble of size $K: \cbrac{Q^i}_{i=1}^K$ , $s_0$
  \FOR{$t= 0, \dots , T$}
    \STATE Choose action $a_t=\mathop{\mathrm{argmax}}_{a}
    \sbrac{\sum_{i=1}^K Q^i(s_t,a)}$
    \STATE $a_t = \text{explore}(a_t)$  \hfill\COMMENT{//e.g. $\epsilon$-greedy}
    \STATE $s_{t+1}, r_t \gets \text{env.step}(s_t,a_t)$
    \STATE Sample an ensemble member to update: $k_t \!\sim\! \calig{U}\!\brac{\sbrac{K}}$
    \STATE $\hat{a}^* = \mathop{\mathrm{argmax}}_a Q^{k_t}(s_{t+1},a)$

    \STATE $Q^{k_t}(s_t,a_t) \gets (1-\alpha_t)Q^{k_t}(s_t,a_t)$\\  $\qquad\qquad\qquad + \alpha_t\brac{r_t + \gamma Q^{EN \setminus k_t}(s_{t+1},\hat{a}^*)}$
  \ENDFOR
  \STATE \textbf{Return} $\cbrac{Q^i}_{i=1}^K$
\end{algorithmic}
\end{algorithm}

\begin{figure}[t]
\begin{center}
\centerline{\includegraphics[width=.975\columnwidth]{figures/toy_mdp.png}}
\caption{The Meta-Chain-MDP, a set of chain-MDPs differing by the reward at state $D$. Each episode the agent is randomly initialized in one of the MDPs. The variance $\sigma=1$ is identical across the chain MDPs.}
\label{fig:MCMDP}
\end{center}
\end{figure}

\begin{figure}[t]
\begin{center}
\centerline{\includegraphics[width=0.9\columnwidth]{figures/Q_bias_averaged_across_MDPs.png}}
\caption{
Estimation bias of the optimal action in state $A^i$ as a function of time in the meta-chain MDP. Due to the zero-initialization of the Q-tables, both DQL and EBQL{} are positively biased for a short period of time.
}
\label{fig:Estimation bias}
\end{center}
\end{figure}

\section{Experiments} \label{Experimints}
In this section, we present two main experimental results of EBQL{} compared to QL and DQL in both a tabular setting and on the ATARI ALE \citep{bellemare2013arcade} using the deep RL variants. We provide additional details on the methodology in \cref{appendix:implementation}.

\subsection{Tabular Experiment - Meta Chain MDP}\label{exp: tabular}

We begin by analyzing the empirical behavior of the various algorithms on a \emph{Meta Chain MDP} problem (\cref{fig:MCMDP}).
In this setting, an agent repeatedly interacts with a predetermined set of $N$ chain-MDPs. All chain-MDPs share the same structure, and by the end of the interaction with one MDP, the next MDP is sampled uniformly at random.

For each chain-MDP $i\in [N]$, the agent starts at state $A^i$ and can move either left or right.
While both immediate rewards are 0, transitioning to state $C^i$ terminates the episode.
However, from state $B^i$, the agent can perform one of $m$ actions, all transitioning to state $D^i$ and providing the same reward $r (D^i) \sim \mathcal{N}(\mu_i, \sigma^2)$, and the episode is terminated.
The set of chain MDPs is a symmetrical mixture of chains with positive and negative means. While a negative $\mu_i$ amplifies the sub-optimality of QL due to the over-estimation bootstrapping bias, we observe that positive $\mu_i$ presents the sub-optimality of DQL under-estimation bias.

This simple, yet challenging MDP serves as a playground to examine whether an algorithm balances optimism and pessimism. It also provides motivation for the general case, where we do not know if optimism or pessimism is better.

\textbf{Analysis:} We present the empirical results in \cref{fig:results_MCMDP}. In addition to the results on the meta chain MDP, we also present results on the single-chain setting, for both positive and negative reward means $\mu_i$.

As expected, due to its optimistic nature, QL excels in sub-MDPs where $\mu_i > 0$ (\cref{subfig:pos_mu}).
The optimistic nature of QL drives the agent towards regions with higher uncertainty (variance).
As argued above, we observe that in this case, DQL is sub-optimal due to the pessimistic nature of the estimator.
On the other hand, due to its pessimistic nature, and as shown in \citet{hasselt2010double}, DQL excels when $\mu_i < 0$. We replicate their results in \cref{subfig:neg_mu}.

Although QL and DQL are capable of exploiting the nature of certain MDPs, this may result in catastrophic failures. On the other hand, and as shown in \cref{subfig:avg_mu}, EBQL{} excels in the meta chain MDP, a scenario that averages over the sub-MDPs, showing the robustness of the method to a more general MDP.
Moreover, we observe that as the size of the ensemble grows, the performance of EBQL{} improves.

\textbf{Estimation Bias:} In addition to reporting the performance, we empirically compare the bias of the various learned Q-functions on the meta chain MDP. The results are presented in \cref{fig:Estimation bias}. As expected QL and DQL converge to $0^+$ and $0^-$ respectively.
Interestingly, the absolute bias of EBQL{} decreases and the algorithm becomes less pessimistic as the ensemble size $K$ grows. This explains the results of \cref{fig:results_MCMDP}, where increasing the ensemble size greatly improves the performance in chain MDPs with $\mu_i>0$. To maintain fairness, all Q-tables were set to zero, which can explain the positive bias of EBQL{} in the initial episodes.

\begin{figure}[t]
\begin{center}
\centerline{\includegraphics[width=0.9\columnwidth]{figures/normalized_scores}}
\centerline{\includegraphics[width=0.9\columnwidth]{figures/median_normalized_scores}}
\caption{\textbf{Normalized scores:} We present both the mean and median normalized scores over the 11 environments (\cref{tab: atari 50m steps}) with 5 seeds per environment. Curves are smoothed with a moving average over 5 points. In addition to DQN and DDQN, we provide a comparison against Rainbow \citep{hessel2018rainbow}. Rainbow represents the state-of-the-art non-distributed Q-learning-based agent.
}
\label{fig: normalized scores}
\end{center}
\end{figure}

\begin{table*}[t]
    \centering
    \caption{\textbf{ATARI:} Comparison of the DQN, DDQN and EBQL{} agents on 11 random ATARI environments. Each algorithm is evaluated for 50m steps over 5 random seeds. We present the average score of the final policies.}
    \label{tab: atari 50m steps}
    \begin{tabular}{c||c|c||c|c||c}
        \backslashbox{\textbf{Environment}}{\textbf{Algorithm}} & \textbf{Human} & \textbf{Random} & \textbf{DQN} & \textbf{DDQN} & \textbf{EBQL{}} \\
        \hline\hline
        \textbf{Asterix} & 8503.3 & 210.0 & 4603.2 & 5718.1 & \textbf{22152.5} \\
        \hline
        \textbf{Breakout} & 31.8 & 1.7 & 283.5 & 333.4 & \textbf{406.3} \\
        \hline
        \textbf{CrazyClimber} & 35410.5 & 10780.5 & 93677.1 & 111765.4 & \textbf{127967.5} \\
        \hline
        \textbf{DoubleDunk} & -15.5 & -18.6 & -15.8 & -18.6 & \textbf{-10.1} \\
        \hline
        \textbf{Gopher} & 2321.0 & 257.6 & 3439.4 & 6940.5 & \textbf{21940.0} \\
        \hline
        \textbf{Pong} & 9.3 & -20.7 & 17.4 & 19.9 & \textbf{21.0} \\
        \hline
        \textbf{PrivateEye} & \textbf{69571.3} & 24.9 & 78.1 & 100.0 & 100.0 \\
        \hline
        \textbf{Qbert} & 13455.0 & 163.9 & 5280.2 & 6277.1 & \textbf{14384.4} \\
        \hline
        \textbf{RoadRunner} & 7845.0 & 11.5 & 27671.4 & 40264.9 & \textbf{55927.5} \\
        \hline
        \textbf{Tennis} & -8.9 & -23.8 & -1.2 & -14.5 & \textbf{-1.1} \\
        \hline
        \textbf{VideoPinball} & 17297.6 & 16256.9 & 104146.4 & 230534.3 & \textbf{361205.1}
    \end{tabular}
\end{table*}

\subsection{Atari}\label{experiments: atari}

Here, we evaluate EBQL{} in a high dimensional task -- ATARI ALE \citep{bellemare2013arcade}. We present EBQL's performance when $K=5$ (ensemble size).
Here, we compare to the DQN \citep{mnih2015human} and DDQN \citep{van2016deep} algorithms, the deep RL variants of Q-learning and Double-Q-Learning respectively.

While there exists a multitude of extensions and practical improvements to DQN, our work focuses on learning the Q-function at a fundamental level.
Similarly to what is shown in Rainbow \citep{hessel2018rainbow}, most of these methods are orthogonal and will thus only complement our approach.

In order to obtain a fair comparison, we \emph{do not} use the ensemble for improved exploration (as was done in \citealt{osband2016deep,osband2018randomized,lee2020sunrise}). Rather, the action is selected using a standard $\epsilon$-greedy exploration scheme, where the greedy estimator is taken on the average Q-values of the ensemble. All hyper-parameters are identical to the baselines, as reported in \citep{mnih2015human}, including the use of target networks (see \cref{appendix:implementation} for pseudo-codes).

\textbf{Analysis:} We test all methods on 11 randomly-chosen environments. Each algorithm is trained over 50m steps and 5 random seeds. To ensure fairness, for DQN and DDQN we report the results as provided by \citet{dqnzoo2020github}. The complete training curves over all environments are provided in \cref{appendix:experimental results}.
The numerical results are shown in \cref{tab: atari 50m steps}. In addition to the numerical results, we present the mean and median normalized scores in \cref{fig: normalized scores}.

When comparing our method to DQN and DDQN, we observe that, in the environments we tested, EBQL{} consistently obtains the \textit{highest} performance. In addition, PrivateEye is the only evaluated environment where EBQL{} did not outperform the human baseline. Remarkably, despite EBQL{} not containing many improvements (e.g., prioritized experience replay \citep{schaul2015prioritized}, distributional estimators \citep{bellemare2017distributional}, and more \citep{hessel2018rainbow}), it outperforms the more complex Rainbow in the small-sample region ($< 20$ million steps, \cref{fig: normalized scores}).
Particularly in the Breakout and RoadRunner domains (\cref{appendix:experimental results}), EBQL{} also outperforms Rainbow over all 50m training steps.

\section{Summary and Future Work} \label{Summary}

In this work, we presented Ensemble Bootstrapped Q-Learning, an under-estimating bias-reduction method. We started by analyzing the simple statistical task of estimating the maximal mean of a set of independent random variables. Here, we proved that the ensemble estimator is equivalent to a weighted double-estimator that unevenly splits samples between its two phases (index-selection and mean-estimation). Moreover, we showed both theoretically and empirically that in some scenarios, the optimal partition is not symmetrical. Then, using the ensemble estimator is beneficial.

Based on this analysis, we propose EBQL, a way of utilizing the ensemble estimator in RL. Our empirical results in the meta chain MDP show that, compared to QL (single estimator) and DQL (double estimator), EBQL{} dramatically reduces the estimation bias while obtaining superior performance. Finally, we evaluated EBQL{} on 11 randomly selected ATARI environments. To ensure fairness, we compared DQN, DDQN and EBQL{} without any additional improvements \citep{hessel2018rainbow}. In all the evaluated environments, EBQL{} exhibits the best performance, in addition to outperforming the human baseline in all but a single environment. Finally, EBQL{} performs competitively with Rainbow in the small-samples regime, even though it does not utilize many of its well-known improvements.

Ensembles are becoming prevalent in RL, especially due to their ability to provide uncertainty estimates and improved exploration techniques \citep{osband2018randomized,lee2020sunrise}. Although ensembles are integral to our method, to ensure a fair evaluation, we did not utilize their properties for improved exploration. These results are exciting as they suggest that by utilizing these properties, the performance of EBQL{} can be dramatically improved.

We believe that our work leaves room to many interesting extensions. First, recall that
EBQL{} fixes the ensemble size at the beginning of the interaction and then utilizes a single estimator to estimate the action index. However, in \cref{subsec: ensemble estimator}, we showed that the optimal split ratio depends on the distribution of the random variables in question.
Therefore, one possible extension is to dynamically change the number of ensemble members used for the index-estimation, according to observed properties of the MDP.

Also, when considering the deep RL implementation, we provided a clean comparison of the basic algorithms (QL, DQL and EBQL). Notably, and in contrast to previous work on ensembles in RL, we did not use ensembles for improved exploration. However, doing so is non-trivial; for exploration needs, previous work decouples the ensemble members during training, whereas our update does the opposite.

redpajama_set_name: RedPajamaArXiv


text:
Train derailment in eastern Iowa closes highway, but no injuries reported
U.S. Highway 67 was closed Friday morning after a train derailed in the eastern Iowa city of LeClaire.
Train derailment in eastern Iowa closes highway, but no injuries reported U.S. Highway 67 was closed Friday morning after a train derailed in the eastern Iowa city of LeClaire. Check out this story on desmoinesregister.com: https://www.desmoinesregister.com/story/news/2020/01/03/train-derails-le-claire-friday-morning/2805389001/
Philip Joens, Des Moines Register Published 12:56 p.m. CT Jan. 3, 2020 | Updated 1:54 p.m. CT Jan. 3, 2020
A derailed train in eastern Iowa forced the closure of a highway Friday morning, but no injuries were reported.
The incident occurred in LeClaire, a city of 3,900 located along the Mississippi River about 14 miles northeast of Davenport.
The derailment occurred at 11 a.m. and a chemical leak was discovered, according to a news release from the Scott County Emergency Management Agency. A hazmat team was sent to the site and the leak was mitigated, according to the release.
Canadian Pacific Railway owns the track and was operating the train, Canadian Pacific spokesman Andy Cummings said in an email.
No injuries or fatalities were reported as a result of the incident, according to Cummings and the news release.
Ryan Burchett, the owner of Mississippi River Distilling Co. in LeClaire, said he noticed the train traveling at a high rate of speed shortly before witnessing the derailment. The downtown area where the derailment occurred is next to a river levee lined with restaurants, boat ramps and the Buffalo Bill Museum just west of the train tracks.
A train derailed on the morning of Friday, Jan. 3, 2020, in downtown LeClaire near a Mississippi River levee. The city is located northeast of Davenport. (Photo: Ryan Burchett/Special to the Register)
An area on the east side of the tracks where the derailed train cars rested Friday is routinely used to host festivals and gatherings, according to Burchett.
"It's pretty crazy the levee was where the cars were," Burchett said. "This area of town is used for town gatherings. It's the main draw."
U.S. Highway 67 in LeClaire was closed in both directions between Interstate Highway 80 and County Road F33 because of the derailment, according to the Iowa Department of Transportation.
Philip Joens covers breaking news for The Des Moines Register. He can be reached at 515-284-8184 at pjoens@registermedia.com or on Twitter @Philip_Joens.
Your subscription makes our journalism possible. Subscribe today at DesMoinesRegister.com/Deal.
Read or Share this story: https://www.desmoinesregister.com/story/news/2020/01/03/train-derails-le-claire-friday-morning/2805389001/


IEEE J. Sel. Top. Quantum Electron., vol. 23, no. 2, 2000108 (2017).
Copyright © 2017 IEEE - All Rights Reserved. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
In this paper, we will present the development of a compact LED-based optical gas sensing system in the ultraviolet-C spectral region. This includes the design of the LED heterostruc- ture emitting near 226 nm, the development of an LED chip, and the implementation into a gas sensing system capable of detecting nitrogen oxide concentrations in the ppm range.
AlGaN-based, gas sensing, light emitting diode, nitrogen oxide, spectrally pure, ultraviolet.


Sunday Bloody Sunday was one of two songs on the Some Time In New York City album to be inspired by the Troubles in Northern Ireland.
The song was specifically about Bloody Sunday, which occurred in Derry on 30 January 1972. Twenty-six unarmed civil rights protesters and bystanders were shot by soldiers of the British Army. Fourteen men died as a result of the shootings, five of whom were shot in the back.
Sunday Bloody Sunday was the last song to be composed for Some Time In New York City. John Lennon had written The Luck Of The Irish in late 1971, but felt compelled to write a more specific response following the massacre.
Most other people express themselves by shouting or playing football at the weekend. But me, here I am in New York and I hear about the 13 people shot dead in Ireland, and I react immediately. And being what I am, I react in four-to-the-bar with a guitar break in the middle.
Lennon took the liberty of identifying himself as a spokesman for the republican cause, with lines such as "You claim to be a majority/Well you know that it's a lie/You're really minority/On this sweet emerald isle/When Stormont bans our marches/They've got a lot to learn." No matter that he had never lived in Ireland, and was newly settled in New York after leaving England for good.
Both Lennon and Paul McCartney sympathised with Irish republicanism. Liverpool had a strong Irish immigrant population, and both families had ancestors who emigrated to England from the country.
Lennon's attitude to violent struggle remained uncertain in 1972. Four years previously, on Revolution and Revolution 1, he was unsure whether to be counted in or out of destruction. By 1971, however, he opened Power To The People with the lines "Say you want a revolution/We'd better get it on right away".
In September 1971, four months prior to Bloody Sunday, Lennon explained his difficulty in reconciling pacifism with the violence of the Irish republican struggle.
I understand why they're doing it, and if it's a choice between the IRA or the British army, I'm with the IRA. But if it's a choice between violence and non-violence, I'm with non-violence. So it's a very delicate line... Our backing of the Irish people is done, really, through the Irish Civil Rights, which is not the IRA. Although I condemn violence, if two people are fighting, I'm probably gonna be on one side or the other, even though I'm against violence.
Lennon and Ono donated their royalties for Sunday Bloody Sunday and The Luck Of The Irish to the civil rights movement in Ireland and New York.
Sunday Bloody Sunday was one of Phil Spector's densest productions on the album, with the Wall Of Sound used to its full effect to turn the song into a powerful rocker. It also featured a false ending which faded back in, a trick The Beatles had previously used on Strawberry Fields Forever and the stereo version of Helter Skelter.
Why no demo or out takes of this great track?
Great lennon vocal. Use to listen to it before football gamescto get psyched up.


text:
Program being developed for teen accused of trying to kill stepmother
A treatment program is being developed for a Lacombe teenager accused of trying to kill his stepmother.
Jul. 24, 2012 7:28 p.m.
Lawyers for the 17-year-old boy, whose name is withheld under provisions of the Canada Youth Justice Act, have entered a plea of not guilty because he was not criminally responsible due to his mental state.
The boy, who remains in custody at Calgary Young Offenders Centre, was arrested during the early evening of April 12 in relation to a violent outburst at his family's home.
He was subsequently charged with attempted murder, aggravated assault, using a knife to commit an offence and unlawful possession of a 12-gauge shotgun.
In Red Deer provincial court on Tuesday, Red Deer lawyer Luke Kurata said he and co-counsel John McNaughton do not take issue with the facts.
The defence team had previously submitted a report by Calgary psychiatrist John Naylor, whose findings will be used to support the contention that their client has been grappling with mental health issues.
Kurata urged the court to move forward with the matter as quickly as possible.
"He needs treatment. To keep him sitting (in remand) another month may not be in his best interest," said Kurata.
The accused and his lawyers return to Red Deer youth court on Aug. 9, when it is anticipated that he will be committed for treatment.
Westerner will likely break overall attendance record
Stabbing accused wants to stay in jail


text:
<?php

namespace Google\Service\ServiceManagement;

class Endpoint extends \Google\Model
{
  /**
   * @var bool
   */
  public $allowCors;
  /**
   * @var string
   */
  public $name;
  /**
   * @var string
   */
  public $target;

  /**
   * @param bool
   */
  public function setAllowCors($allowCors)
  {
    $this->allowCors = $allowCors;
  }
  /**
   * @return bool
   */
  public function getAllowCors()
  {
    return $this->allowCors;
  }
  /**
   * @param string
   */
  public function setName($name)
  {
    $this->name = $name;
  }
  /**
   * @return string
   */
  public function getName()
  {
    return $this->name;
  }
  /**
   * @param string
   */
  public function setTarget($target)
  {
    $this->target = $target;
  }
  /**
   * @return string
   */
  public function getTarget()
  {
    return $this->target;
  }
}

// Adding a class alias for backwards compatibility with the previous class name.
class_alias(Endpoint::class, 'Google_Service_ServiceManagement_Endpoint');

redpajama_set_name: RedPajamaGithub


text:
Lemmon est une municipalité américaine située dans le comté de Perkins, dans l'État du Dakota du Sud. Selon le recensement de 2010, elle compte . La municipalité s'étend sur .

Fondée en 1907, la ville doit son nom au rancher .

Démographie

Notes et références

City au Dakota du Sud
Comté de Perkins (Dakota du Sud)


Bouquet consists of preserved red carnation, dried flowers (lavender, wheat), baby's breath and unique fillers.
The meaning of carnation in Greek means 'Heavenly Flowers'. The carnation symbolises eternal love.
A PRESERVED carnation which is everlasting would certainly mean infinite eternal love.
What better way to show that kind of love for your mother this Mother's Day by getting her this bouquet?


We first wrote about the shop when it was preparing to open. Now, it's a thriving local business offering eco-friendly goods to an enthusiastic community. We revisited the store and founder Katerina Bogatireva to learn how things are going and what's coming up.
What has the Bushwick community's response been to the store?
It's been very warm and welcoming — it feels like a very tight knit community. Every person that walks through the door has been an inspiration to me and I constantly learn from my customers. It's been tremendous fun getting to know them.
What has been your favorite part of running the shop so far?
My interactions with people. Connecting and witnessing how people truly are willing to travel to shop responsibly.
What's your favorite product in the store?
Bulk tofu — it was difficult to source responsibly, but we did it!
How has your vision for Precycle evolved since we last spoke?
I've learned a lot from the experience and our customers and coworkers. It's been very inspiring and validating. I am working on bringing Precycle to more neighborhoods in NYC. I'm ready to push forward now more than ever.
We're excited about the Earth Day 2019 celebration in Union Square on April 23. Hope to see some familiar faces there!
PS: We're partnering with Precycle to give away a $100 gift card to one lucky winner, so you can experience the shop's bounty first-hand. Head to our Instagram to learn how to enter!
All photos by Marina Williams.