text:
Protest Is Risky at Egypt's COP27. That Won't Stop Activists
ByRamin Skibba
Nov 10, 2022 activism, climate, climate change, cop27, egypt, prison, protests
But an altercation that followed the event exemplified the resistance human rights activists face: According to a Washington Post report, Egyptian lawmaker Amr Darwish stood up and yelled at Seif. "You are here summoning foreign countries to pressure Egypt." He continued berating her until UN security escorted him out, the paper reported.
Activist organizations in Egypt have to deal with limited funding, harassment, and onerous conditions for organizing peaceful demonstrations and press conferences. Some fear for their lives and are essentially forced into exile. A small gathering of a group of people is enough to draw the suspicion of security forces, says Ubrei-Joe Maimoni Mariere, a Nigerian environmental activist of the Friends of the Earth Africa, a nonprofit group. "Egypt is not the best place to hold a COP, because of the repressive nature of the Egyptian government. Activists are careful not to break the laws of the land," he says. Instead of being sited at a beautiful resort, he argues, such a meeting would be better held in a place where many people live with the effects of climate change, like polluted water and heat waves.
On Friday, US President Joe Biden is scheduled to speak with el-Sisi, and reportedly will press him on human rights issues in the country. Egypt has been a close ally of the US since the 1980s, and is one of the top recipients of military aid from the United States, Russia, France, and Italy. At Tuesday's event, Seif essentially called for reducing that aid. "Those weapons will be used against us. You really have to reimagine your foreign policy to Egypt, because it is creating a problem here," she said.
Bahgat, the Egyptian human rights advocate, points out that the situation for activists has worsened significantly since the coup that brought el-Sisi—a former general—to power. Ten years ago, after the Arab Spring culminated in the fall of then-president Mubarak, he says, people felt empowered. His organization aided a community in western Egypt who, after being displaced by a nuclear power plant, organized a sit-in, demanding to be returned to their lands or fairly compensated. Eventually, after that protest and a press conference, the government created a compensation scheme. "I'm telling you this story because every aspect of it is impossible to imagine today," he says.
"The general clampdown that Human Rights Watch has witnessed is also impacting environmental groups, some very directly and others in more nuanced and subtle ways, in the sense that some of these groups and activists self-censor and do not engage in certain actions and discussions that could get them in trouble," says Katharina Rall, an environmental researcher for the group. The unwelcome environment for demonstrators was already evident before the COP27 summit began, Rall says, when an Indian activist, Ajit Rajagopal, began an eight-day march from Cairo to Sharm el-Sheikh, but was arrested by Egyptian security forces on November 6. He was released the following day, but the message was clear.
The next UN climate summit, COP28, will be held in the United Arab Emirates in November 2023. That government is also well documented as a repressive regime. But already a key message has emerged from COP27, Bahgat says: "There is no climate justice without human rights."
Additional reporting by Gregory Barber.
Russia's New Cyberwarfare in Ukraine Is Fast, Dirty, and Relentless
Keybanc Maintains Overweight Rating for Marqeta: Here's What You Need To Know – Marqeta (NASDAQ:MQ)


As the wonderful photography of Gertraud Fendler has documented, this winter has changed much of the Bay into a frozen pond. Despite the clean, white appearance, the actual condition of the water tells a different story. According to the Chesapeake Bay Foundation's latest State of the Bay report (2014), the Bay is showing minor improvement, with Water quality indicator scores going up slightly. The problem with even this minor improvement is that the baseline has shifted so far towards the negative that incremental improvements such as this will hardly ever make a dent in the "real" health and quality of the Bay. The report also notes that Blue crabs and striped bass are showing signs of severe stress (due to pollution, diseases, overharvesting); again, these "metrics indicate a system still dangerously out of balance." According to data from ChesapeakeBay.net, the abundance of spawning-age female blue crabs in the Chesapeake Bay decreased to 68.5 million in 2014, compared with 147 million in 2013.
My roots are in the clay hills of northeastern Alabama, where for generations my forebears were basically subsistence farmers living on what the land would provide them. Mainly corn, cotton, apricots, and other rotational crops were supplemented with eggs (chickens produced year round). A flock of 100-120 chickens lived outside during the day, pecking the dirt for worms and insects, and in the evening returned to the safety of a fairly large corrugated metal and wire chicken house enclosure. This somewhat bucolic memory allowed me for so many years to perpetuate the myth of "cage free" or "free range" organic chicken farming in my own mind.
I would argue for the promotion of "organic" family chicken farming in Washington, DC, and Northern Virginia (where I lived for 25 years), as well as for expanded operations here on the Eastern Shore. That all changed a year ago. As I was traveling to work in Norfolk, I got behind a truckload of caged chickens. The thermometer in my car registered 18 degrees, and as I stopped at a traffic light, I could see each face, crammed in and waiting. I have never been much of an adherer to James Joycean epiphanies, but this was one. It was a realization that I had been very wrong about the myth of the family farm and the possibility of "humane" chicken farming. No matter what justification I tried to use, Morrisey was right: meat is murder — it's unnecessary and just not possible to farm animals humanely. The end of the line would eventually lead to a cage in the back of a truck. I told myself I would try to no longer be party to this.
EDITOR'S NOTE: Last week dauntless social critic Wayne Creed attended a meeting of the Cape Charles Historic District Review Board. His report and rant appear below.
The Historic District Review Board met January 29 to consider requested modifications to the Board's earlier approval of plans for a new home at 404 Jefferson Avenue. Owner Gregory Manuel requested permission to add a second-story covered porch on the front of the house, a dormer on the west side, sliding glass doors to access the porch, and additional stairwell windows front and back.
Fehrer noted that dormers had been approved by the Board in other instances, yet he was skeptical in this case. "They must meet historic guidelines," he emphasized. Jeb Brady quickly added that there are several examples of dormers throughout the town.
This is the first house in town to incorporate sliding glass doors on the front (façade) of the house, prompting Fehrer to respond that they did not appear to fit in with the historic character of the town. "They are not appropriate for the Historic District. I'm loath to make an approval that will set a precedent," he said.
Manuel then asked Code Official Brady, "There are examples of this?" He then offered the Board some alternatives that they might find more palatable, everything from adding muttons to changing the design to incorporate French doors. Punctuating the exchange, Manuel's cell phone rang and rang, as it would several more times during the meeting.
EDITOR'S NOTE: In the absence of other reporting staff, Cape Charles' most acerbic social critic, Wave columnist Wayne Creed, has recently donned a reportorial hat and dutifully recorded the actions of Northampton's Board of Supervisors, which can be read in the news columns here and here. But in the column below, the indomitable Mr. Creed returns to his roots with an opinionated but highly informative commentary on Cape Charles Town Council's first meeting of the new year.
With Christmas decorations ripped down, the sidewalks dark and lonely, the night cold and wet, the lights were turned on inside the Town Civic Center January 15 in order to conduct the first Town Council Regular Meeting of the New Year. Aside from two members of news media (Cape Charles Wave and Eastern Shore Post), only two other constituents braved the raw weather to attend. Mayor Proto himself arrived nearly 30 minutes late.
Town Council voted to request the Virginia Port Authority to carry over to the new year several grants previously awarded for harbor improvements. This includes $645,000 for offshore breakwaters and $185,000 for a wave attenuator and dock improvements. The VPA has altered the terms to a 75%-25% match, making the proposal more fiscally attractive. Local matching funds must still be identified for these projects. Councilman Steve Bennett aggressively questioned Harbormaster Smitty Dize in what appeared to be a leading manner, attempting to get Mr. Dize to acknowledge, or at least provide a modicum of actual requirements that might justify the request for funds.
Town Manager Bob Panek ordered basketball backboards and nets removed December 26, 2012, for "safekeeping." The Wave has been running this poignant picture ever since.
True, yet even Bill would have to admit it's tough here for ballers, since there isn't anywhere in Cape Charles to truly practice or play (Cape Charles Baptist Church has one goal in a small area for every player in and around town. The backboard has already been crashed once). The outside courts at the old school, perfect for the urban ¾ game, have been taken down, and the beautiful old court inside is being chopped up and turned into apartments.
Living in Cape Charles is always great no matter the season, whether spring, summer, fall or winter. But Christmas time in Cape Charles is always a special time that sets itself apart from the rest of the year. The streets are lined with decorations, Smitty has his crab pot tree at the harbor, and the shops are decorated as cheerfully as any in Ghent or Old Town Alexandria.
This season, I was fortunate to be able to be part of two wonderful Christmas events: acting in short skit at one of the stops along the Progressive Dinner Tour, and acting and directing in the Palace Theatre's A Christmas Carol.
The Progressive Dinner Tour's theme this year was "Christmas by the Bay," where a yacht full of rich folks are stranded in the harbor, and are forced to endure the holidays in this little bayside village. Of course they, like all of us, fell in love with Cape Charles, and wind up buying a home here. My role in this little drama was to play the saucy Mrs. Lambertson, along with Michael Flannigan as the sea captain and Sagre Strutzman as the chef. I don't mean to sound immodest, but on this evening I was certainly the sexiest woman in Cape Charles (and moving forward, possibly Cape Charles' new "I"' girl).
In 2012, the National Marine Fisheries Service listed the Great Atlantic Sturgeon under the Endangered Species Act. This is usually bad news, but since sightings and spawning grounds were so rare, the fact that there was enough sturgeon left to even warrant the declaration is a good thing. Two years later, we find that they are once again spawning in the Chesapeake Bay just outside the James River.


text:
Q: How to extract public key from a .der file in SubjectPublicKeyInfo format? I have a public key that appears to be in the SubjectPublicKeyInfo format. I say "appears" because the only thing I know about this key that it loads a Java application using X509EncodedKeySpec which, according to documentation is used for loading keys in SubjectPublicKeyInfo format.
I've been unable to parse the file with openssl with either pkcs8, x509 or asn1parse.
Below is the result of asn1 parse:
openssl asn1parse -in public_key.der -inform der

0:d=0  hl=4 l= 290 cons: SEQUENCE
4:d=1  hl=2 l=  13 cons: SEQUENCE
6:d=2  hl=2 l=   9 prim: OBJECT            :rsaEncryption
17:d=2  hl=2 l=   0 prim: NULL
19:d=1  hl=4 l= 271 prim: BIT STRING

Based on that output I've tried extracting the actual key by experimenting with different offsets and lengths then saving it to the .der file:
openssl asn1parse -in public_key.der -inform der -offset <> -length <l> -out public_key.der

Then I'd feed the output back to openssl:
openssl pkcs8 -inform DER -nocrypt -in public_key.der

But keep getting the same error:

Error decrypting key 140436029183664:error:0D0680A8:asn1 encoding
  routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1338:
  140436029183664:error:0D06C03A:asn1 encoding
  routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:852:
  140436029183664:error:0D08303A:asn1 encoding
  routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1
  error:tasn_dec.c:772:Field=version, Type=PKCS8_PRIV_KEY_INFO

Update: output of
openssl asn1parse -in public_key.der -inform der -strparse 19

0:d=0  hl=4 l= 266 cons: SEQUENCE
4:d=1  hl=4 l= 257 prim: INTEGER           :F19011E8903CFE79920F5D06CBF6B57593038DBDDEFEF30C796287264100DC930E2F5F2C7CFBAA84C0212228288D76B97EC7FC1FF6409770292386B5EFC15C3AF999F6FED14EA1D3419EB87F8188E1D21358F95EEA4642716A298A23CE6F98E03DE8D56A4101F39983F4444A3924BBD49A9C721BE5F4637EF09ACE0486C065433CA9B3353D6852364EC4211BEFC24AFD0CB7BFD2ECAE0D0F8B48BE7E7CBE336ABED9A7C0E0B6D468D4D6E6C05FA1680BB2BD7E8DA6FE201BAFDD6B30CF3A7381BCC47DC7F8B4F52715C052DEF3EB361064B2AD8523E6C186B59A320DC7DFE092FD2D668AD5516EAE25103FD66DA579E097D10A4ACAAFF3B9CA528448B2A625FF
265:d=1  hl=2 l=   3 prim: INTEGER           :010001

A: A SubjectPublicKeyInfo file can be used with openssl rsa -pubin -inform der|pem -file inputfile -modulus.
If it is in binary then use der, if it is base64 encoded, use pem.

redpajama_set_name: RedPajamaStackExchange


text:
Q: Splitting big csv files I'm trying to split big csv files.Right now I could only split csv files w/50k columns.Whenever I try splitting a 100k it doesn't work.
I can't figure out whats wrong.
Here is my code for the splitter I use for 100k:
$inputFile = 'uploads/uploaded.csv';
$outputFile = 'uploads/output';

$rows = array_map('str_getcsv', file($inputFile));
$header = array_shift($rows);

$splitSize = 50000;

$in = fopen($inputFile, 'r');

fgetcsv($in, 1000, ",");

//array for file name
$stored_names = array();

$rowCount = 0;
$fileCount = 1;

//split csv
while (!feof($in)) {
    if (($rowCount % $splitSize) == 0) {
        if ($rowCount > 0) {
            fclose($out);
        }

        $super_file_name = $outputFile . $fileCount++;
        array_push($stored_names,$super_file_name.'.csv');

        $out = fopen($super_file_name. '.csv', 'w');

        //insert header
        fputcsv($out,$header);

        // array_push($stored_names,$out);

    }
    $data = fgetcsv($in);
    if ($data)
        fputcsv($out,$data);
    $rowCount++;
}

fclose($out);

A: I'm guessing that your issue is a memory limit based on this code:
$rows = array_map('str_getcsv', file($inputFile));
$header = array_shift($rows);

This reads the entire file into memory, splits it into an array of arrays, then pops off the first row, and throws away the rest. Since you only need the first row, you don't need to read the whole file. Instead just do something like:
$fp = fopen($inputFile, 'r');
$headers = fgetcsv($fp);

Then you have $fp already open and pointing to the first data line for your splitting process,.


text:
Ticket Sarasota
Meet Nathan Smith of Smokin' Joe's
By Kim Doleatto , Herald-Tribune
/ Wednesday, June 22, 2016
Nathan Smith is a bartender at Smokin' Joe's in downtown Sarasota. PHOTO BY KIM DOLEATTO
You probably know Smokin' Joe's, and if you've been there more than twice, probably call it "Joe's." Adorned with Jack Daniel's and Jameson collectibles, Joe's is a true bar: no cocktail waitresses stuffed in corsets, no obscure drink ingredients, no food. And no food means smoking is allowed. In fact, one thing this bar has that others don't is a cigar shop inside that puts the "Smokin' in front of "Joe's," replete with a humidor, cigar cutters, lottery tickets and gum.
But dear non-smokers, don't let this deter you, because it's a guaranteed good time, with an open front and huge fans inside to keep the air clear. Joe's has a neighborhood pub feel, low lit with wood floors and benches, a pool table, 16 televisions spread across the wall, a jukebox and 26 draft beers to choose from plus a full bar. Bottles of Fireball, Jager, Crown, Patron and Jack Daniel's stand bottom up, ready for a quick pour.
A small front patio offers prime people watching in the heart of downtown. And although Joe's has been serving up good times for more than 10 years, it doesn't get much media attention. "There's never been a write up about us, so this is kinda cool," says Nathan Smith, 28, a full-time Smokin' Joe's bartender from Sarasota who can't leave the house without bumping into someone he knows.
How did you come to work for Smokin' Joe's?
I was working at Owen's Fish Camp as a server and I would come here to hang out. The owner, Brian Duarte, pulled me aside one night and said, "So you're that loud kid. People seem to like you. I wanna offer you a job but I don't have a position open." A few weeks later they needed someone. I had no bartending experience. But he said that was fine, that I'd "figure it out." So I got one training shift. Then someone got sick. The next day I got thrown in. It was a Friday night, it was my second day, it was packed and it was scary. But once you make something, you're good. You can't forget how to make scrambled eggs.
When does it get craziest?
Our three busiest nights are Friday, Saturday and Tuesday. Tuesday we do a bogo program through a text. You get a code sent to your phone at noon that day and people come in to claim their bogo drink. It's awesome. Text "pub1" to 35350 and you're on your way.
Who would you say is your average clientele?
Everyone. It's anything from 21st birthday bashes to older guys just enjoying a beer or catching the game. It's all over the place. We're able to have relatively cheap prices since we do volume. It's hard not to come in and get a good drink (bar speak for "good pour"), and who doesn't like that?
Sports fans come too. We show UFC Payperview fights and that attracts a crowd.
What's your most popular order?
Tito's, Jager, Fireball. And shots. Shots are big here.
When someone doesn't know what they want, what do you serve them?
A Johnny Vegas. I don't tell them what's in it, I just make it and I've never had someone turn it away. Even non-tequila drinkers like it. It's Patron Silver, watermelon pucker, and Red Bull.
What do you do when you're not working?
I come here or go to the gym.I also do ceramic sculpture. I have a pottery wheel. My garage is like my little studio. I also model sometimes for photographer friends who need it. Worse case scenario I get free glam photos. I've been in two music videos. In one of them I'm shirtless with horns as a taurus. In the other one I was in a love triangle.
What are you listening to right now?
The song "First" by Cold War Kids. Vance Joy. I love '90s music.
What's your favorite bar in town?
I come here. I always stop in here first at least, then I hit Pangea or Tavern on Main. Memories if I can make it over there.
What do you drink?
Tequila. I like shots of 1800 Silver with a pineapple juice back, no salt or lime, chilled or neat. I love IPAs too.
What's your favorite restaurant?
Owen's Fish Camp. Longhorn, too, but I'm trying to cut down on red meat.
What would be harder to give up, coffee or alcohol?
Probably coffee.
I quit four and a half years ago.
What is most rewarding about bartending for you?
Other than the money, I basically get to come to a party. I meet so many people but I remember them more than they remember me. I get asked why I look familiar, I tell them, "From the bar but you were drunk when we met so you may not remember."
What's the best cure for a hangover?
Pizza, wings and Pedialyte. I tried the coconut water thing, but I can't do that flavor, even though I've heard good things.
SMOKIN' JOE'S
1448 Main St., Sarasota; 941-366-9439; smokinjoespub.com
Contact Kim Doelatto with information on your favorite local bartender at kim.doleatto@heraldtribune.com
Last modified: June 23, 2016
All rights reserved. This copyrighted material may not be published without permissions. Links are encouraged.
Families set sail for Labor Day Regatta at Sarasota Sailing Squadron
Ringling Underground postponed
This Week in the Arts: Aug. 31-Sept. 6
Top 5 Things To Do Today 8/31
Copyright © 2013 · All Rights Reserved · Herald-Tribune Media Group · 1741 Main Street Sarasota, FL 34236

redpajama_set_name: RedPajamaCommonCrawl


text:
Don Williams, also known by the nickname Ducky, is an American illustrator employed by The Walt Disney Company. Williams has worked at Disney for over thirty years, at a number of different positions, finally ending up in the marketing division as an animator.

Williams is not to be confused with the American animator Don Harold Williams, who worked at various animation studios (including Disney) during the golden age of animation.

Early life
Don Williams was born in Springfield, Massachusetts. He is of Scottish descent and still has strong ties to Scotland.  He grew up poor during the 1950s. His mother was a single mother raising 8 children including Don. His mother nicknamed Don "Ducky" because she had a love for the Donald Duck comic books.

Williams always knew he wanted to be an artist. When he was just 10 years old he wrote his first letter to Walt Disney asking for a job. Walt Disney wrote him back saying, he was sorry but he did not have any jobs for 10-year-old cartoonist, however he encouraged him to keep drawing. He graduated high school in the early 1960s and was immediately drafted into the Vietnam War. He joined the Navy, and served for four years in the Navy.

After he left the navy he went back to Massachusetts, where he got a job as a teller at a local bank; after a few months of work he was promoted to branch manager. He managed the bank for 10 years.

Life before Disney
While working at the bank, Williams continued to draw. During the Christmas season at the bank he would hang up his drawings around the bank. The drawings caught the eye of one of the patrons of the bank, who happened to be an anchor for the local 6pm news. The anchor then did a piece on Williams and his drawings. A friend of Williams's, unbeknownst to him, wrote a letter and sent a copy of the tape to Disney animation studios in California. Two weeks after the tape was sent, he received a letter from the animation department of Disney saying that while they liked his drawings there were no job openings available at that time. However, he was told that if he wanted to pursue a career as a Disney artist he should contact Ralph Kent, the director of the Marketing Art department for Walt Disney World in Florida. He then sent his tape and a new letter to Kent.

Williams waited quite some time for a response but never received one. Williams took matters in his own hands and called Kent. He spoke to Kent's Secretary who said that Kent had not had time to watch his tape, and that if he really wanted feedback then he should come down to Florida himself. He then went to Florida where he met with two artists and Kent, who told him that his work was unpolished because he did not go to an art school. They told him that he had talent, and he just needed work. They sent him off with official Disney model sheets of characters and gave him contact information of veteran character artist who worked with Disney World named Russell Schroeder.

He went back to work at the bank and began drawing 100 pictures a week and sending them off to Schroeder for critique. He drew 100 pictures a week for two years before he finally decided that he could no longer pursue his dream in Massachusetts, so he moved to Florida. He attempted to get a job at Disney World for three months, with no luck. He instead got a job right outside the park at a restaurant. After two weeks at the restaurant he was desperate to get any job inside the park. He was finally offered a job at the park as a custodian during the graveyard shift, but after a little digging he found out there was a job painting patrons portraits. Even while working at Disney, he still sent in his 100 pictures a week to Schroeder. After six weeks as a portrait artist the division was shut down, and he was sent to work as a clerk in Tomorrowland. After five weeks in Tomorrowland, he found out about a temporary opening in the art department.

He landed the job thanks to Schroeder and the massive amount of drawings he had sent him over the years. It was a 30-day trial period. After 30 days of work no one said anything to Don so he just kept showing up for work, and he has been working there ever since.

Life at Disney
Williams started at the Marketing Art department, later called Disney Parks and Resorts Advertising Creative, then Disney's Yellow Shoes Global Creative Group There were originally staffed 16 illustrator, but the number dwindled down to just him, with his co-workers going off to work in more computer-based media.

Williams stayed behind, saying "I am not an animator, I am an illustrator." He can use computers, he just prefers paint. He became the sole and last Senior Character Artist for the Walt Disney Marketing. He continues to create illustrations for mailers, brochures, guide books, and specialty items. He also illustrates almost anything that has to do with advertising of the Disney company in the Orlando area.

Williams has also created art inside the Disney Parks, providing paintings, signage and china for the Lady and the Tramp themed restaurant, Tony's Town Square Cafe. He also helped put together the original Mickey Mouse house that was located in Mickey's Birthday Land. Today he gives presentations across America and on Disney Cruise line ships.He is a popular speaker and celebrity, often one of the most respected artists at the Epcot International Festival of the Arts. He has also painted hundreds of limited-edition lithographs for Disney Cruise Line, Disney Vacation Club and other Walt Disney Company divisions.

Works as Illustrator
When Williams first started working at Disney he became a colleague of Russell Schroeder, who subsequently recommended him his first illustrating job. Don painted and Schroeder drew the pictures for Mickey's Prince and the Pauper. Since then he has done around 200 different Disney children's book illustrations. The books he has illustrated includes:
Disney's Pocahontas, Golden Books, 2013
Walt Disney's 101 Dalmatians, Golden Books,1994
Disney's Hercules, Golden Books, 1997
Disney's The Lion King, Golden Pr, 1994
Belle Explores the Castle, Disney Pr. 1992
Always Friends (Super Coloring Book), Golden Disney, 2004
Rudolph's Bright Christmas, Golden Books, 2003
Funny Hunny Christmas, RH Disney, 2003
Deck the Halls, Golden Books, 2003
Raining Cats and Dogs, Random House Books for Young Readers, 2003
Good Night, Sweet Princess, RH Disney, 2002
Don't Forget to Remember, Random House for Young Readers, 2002
Disney Princess: Royal Rooms and Pretty Places, RH Disney, 2002
Peter Pan, RH Disney, 2002
A Whale of Time, RH Disney, 2002
Lost and Found, Random House for Young Readers, 2001
Taking Care of Quetzal, Random House for Young Readers, 2001
Simba's Daring Rescue, RH Disney, 2001
Simba's Jungle Hunt, RH Disney, 2001
Cassie's Runaway Kite, Random House for Young Readers, 2001
Weezie Loses her Voice, Random House for Young Readers, 2001
Dragon Tales Cassie Loves a Parade, Tandem Library, 2000
 Santa's Little Helper Paint Box with Paint Brush and Paint Pots, Tandem Library, 1999
 La Cenicienta, Grupo Editorial Norma, 1999
Bounce Around, Tigger, Golden Books Publishing Company, 1999
Picture Me Dancing with Minnie, Picture Me Books, 1998
Picture me on Vacation with Mickey, Picture Me Books, 1998
Ariel's Light-Up Cake,  Mouse Works, 1998
Snow Whites Apple Pie: A Little Look Book, Golden Books Publishing Company, 1997
 Disney's Pocahontas: The Voice of the Wind, Golden Books, 1995
 Walt Disney's 101 Dalmatians: Pongo to the Rescue, Golden PR, 1994
 Disney's Aladdin: Monkey Business, Golden PR, 1993
Walt Disney's Cinderella, Golden Pr, 1993
 Belle Explores the Castle, Disney Pr, 1992
Walt Disney Pictures Presents: The Prince and the Pauper, Golden Pr, 1990

References

Living people
Walt Disney Animation Studios people
Artists from Springfield, Massachusetts
American animators
1940s births

redpajama_set_name: RedPajamaWikipedia


text:
The Cheney Conundrum and What It Means for the GOP
By Peter Roff • Newsweek
In Washington, there are two kinds of Republicans: those who care what The New York Times writes and those who don't. As hard as it is to believe, there are still some in the GOP who care deeply about what the liberal media establishment says, though it's not clear why.
The Times has been losing readership for years, along with its power to set the national agenda. It still has influence in the Acela corridor—that swath of urban liberalism between Washington, D.C., and Boston—and among the people who select the stories the major networks will cover. But most Americans get their news from the internet, where, as far as information about politics is concerned, it's still the wild, wild west.
Among folks who use the internet as their primary source of information, the Times has about as much impact as a fly on an elephant's back. To these people, what the so-called paper of record says about the GOP, conservatives in general and Donald Trumpspecifically doesn't matter a swivel-eyed tinker's damn.
To the elites, Wyoming representative Liz Cheney's ouster from the No. 3 position in the House GOP leadership is a big deal. To them, it's all about Trump—a person whose influence, Cheney and her newfound brethren seem to believe, must be cleansed from the party. To those who follow the House closely and understand how these things work, it's not such a big deal.
Regarding Trump, Cheney is at odds with most of her Republican colleagues. Most of them, it seems clear, either continue to embrace the former president or would rather avoid talking about him, and instead prefer to spend their time and political capital opposing the Joe Biden-Kamala Harris vision for America.
Rep. Liz Cheney (R-WY) talks to reporters after House Republicans voted to remove her as conference chair in the U.S. Capitol Visitors Center on May 12, 2021 in Washington, D.C. CHIP SOMODEVILLA/GETTY IMAGES
This is not an unreasonable position to take. Nor is Rep. Cheney's—as an individual member of Congress. If she wants to spend her time crusading against Trumpian elements within the Republican Party, she has every right to do so. However, as a member of House GOP leadership, she has obligations that go beyond the dictates of her own conscience. She is responsible to the colleagues who put her in office and who—earlier this term—voted to keep her there. That means she should be on the Sunday shows and out in the hustings helping GOP candidates take control of the House back from Nancy Pelosi and the Democrats. She can't do that if all she wants to talk about, as she's made clear, is Trump.
The Republicans should be favored to win back the majority in 2022, based on reapportionment and redistricting alone. For all the Democrats' protestations about gerrymandering—which they used to extend their own congressional majority for at least an additional 10 years beginning in 1982 without a word from elite media save for the Wall Street Journal editorial page—a fair map drawn without any demographic trickery should easily add the number of seats needed for the GOP to reach and exceed the magic number of 218. But, because nothing in politics is certain, unfocused GOP leadership could throw a wrench into the works and prevent it from happening.
The list of things that could go wrong for Republicans' House prospects is long and largely speculative. High atop it, though, is a campaign in which Democrats and major media outlets force GOP congressional candidates to defend Trump day in and day out instead of taking the attack to Biden and the Democrats. In that environment, Cheney's repeated condemnations of the former president and his influence on the party would not have been helpful to winning the House Republican Conference a majority for the two years before the next presidential election. And it would have been fatal to the Republican Party's attempt to regain control of the Senate.
Members of the congressional leadership are expected to be team players. Leaders, even in the minority, must balance the interests of all members of their conference against their own. It is not easy and not a job for the faint of heart. But the number one priority, former House speaker Newt Gingrich once told me, is "Don't do anything that will start a civil war inside your own party." Cheney broke that rule and received the appropriate consequence. She has not been thrown out of office or stripped of her committee assignments. She's now free to pursue what she thinks best for herself and the GOP without diminishing the prospects the other Republicans serving with her will be reelected.
In Washington, that matters. Out in America, where real life exists, not so much.
Tags: Cheney, Congress, Conservatism, Leadership, Politics
For Politicians To Win Back American Trust, Aristotle Suggests Prudence
By Susanna Hoffman • The Federalist
American's trust in our politicians' ability to handle domestic and international problems is at its lowest point in more than two decades. A Gallup poll in January showed 35 percent of Americans do not trust the U.S. government to handle domestic problems, and only 41 trust it to handle international problems.
If Americans' faith could be restored in our politicians' ability to make prudent decisions, we would no doubt see an increase in trust. To be this trusted politician, the Greek philosopher Aristotle said requires practical wisdom, also called prudence. Such a man or woman must be able to choose the clever choice of action and have good intentions.
There are two components to our choices. One is our moral goal, the other is our practical goal. What moral thing do we want? Good choices not only involve good moral intentions, but they also involve practical decisions that pick through the circumstances to determine the correct action.
Dr. Larry P. Arnn said in his free online lecture at Hillsdale College that prudence "is the facility to calculate circumstances and find and issue true command." It is practical wisdom about things that are always changing.
"Statesmanship is the scene where practical judgment is demanded intensely all the time," Arnn said. Things are constantly changing, demanding good judgement of situations.
Practical wisdom is different than knowledge, however. Aristotle defines knowledge as "knowing a thing that is fixed and can be relied on," Arnn said. Practical wisdom and judgement, on the other hand, are adaptable, as they are about changeable things.
Practical judgement is "a truth-disclosing active condition involving reason that governs action, concerned with what is good and bad for a human being," Aristotle wrote in "Nicomachean Ethics." It finds the truth about what to do and then does it. It is key that the judgement fit the circumstance.
We learned in Book One of "Ethics" that something is good only if it fulfills its purpose. A cup is only good if it has a bottom; otherwise it is not a cup. Likewise, a judgement is only good if it fits the circumstance that it is in. It requires wisdom because accurately placing judgement on ever-changing circumstances is difficult.
This is how we can determine a good politician in these always changing circumstances. Those who can calculate means and ends, and can understand evolving circumstances, tend to serve their nations well.
There are three distinctions to practical judgement. Astuteness is understanding the moving parts of a situation but does not issue action. Cleverness does issue action, as it knows how to insert oneself to get what one wants. Prudence, or good judgement, pursues action with good intent.
Arnn pointed to Adolf Hitler and Winston Churchill as examples of cleverness and prudence. Both were lethal individuals who knew how to get their way, yet used their discernment and action for opposite ends. Hitler used his charisma and cleverness for evil ends, while Churchill used his for good.
Alas, astuteness does not guarantee possession of all virtues. Hitler can be considered brave, but not just. To be clever but not prudent is incredibly dangerous. While not many are as dangerous as someone like Hitler, politicians without prudence may only pursue their own gain.
Perhaps our politicians can learn something from Aristotle, and learn to pursue a good beyond themselves and for the country.
Tags: Leadership, political credibility
The Emerging Law of the Gun
by Peter Huessy
Russian President Vladimir Putin annexes the Crimea in gross violation of international law. What should America do, if anything?
There are many different ideas. Some suggest doing nothing. Some assert we cannot do anything. Others feel the consequences of letting such aggression stand will be serious.
The country's divisions are certainly reflective of how divided on this Americans are.
What then is the proper role for America in the world that both keeps us safe and enhances our prosperity? Continue reading →
Tags: Economy, Energy, Leadership, Military, Peace through strength, Projecting Weakness
Condi Rice Blasts Obama on Weakness, Leadership
Says we can't afford to be war weary.
by Stephen F. Hayes
Former Secretary of State Condoleezza Rice accused Barack Obama of dramatically weakening the United States' position in the world, drawing a straight line between Obama's ever-yielding foreign policy and the increasing troubles around the world.
"Right now, there's a vacuum," she told a crowd of more than two thousand attending the National Republican Congressional Committee's annual dinner last night in Washington, D.C. "There's a vacuum because we've decided to lower our voice. We've decided to step back. We've decided that if we step back and lower our voice, others will lead, other things will fill that vacuum." Citing Bashar al Assad's slaughter in Syria, Vladimir Putin's aggression in Ukraine, al Qaeda's triumphant return to Fallujah, Iraq, and China's nationalist fervor, she concluded: "When America steps back and there is a vacuum, trouble will fill that vacuum."
Tags: Leadership, Obama, Peace through strength, Projecting Weakness
Obama's mistaken belief that others see the world as he sees it
by Michael Barone
Solipsism. It's a fancy word that means that the self is the only existing reality and that the external world, including other people, are representations of one's own self and can have no independent existence. A person who follows this philosophy may believe that others see the world as he does and will behave as he would.
It's a quality often found in narcissists, people who greatly admire themselves — such as a presidential candidate confident that he is a better speechwriter than his speechwriters, knows more about policy than his policy directors and is a better political director than his political director.
If that sounds familiar, it's a paraphrase of what President Obama told top political aide Patrick Gaspard in 2008, according to the New Yorker's Ryan Lizza. Continue reading →
Tags: Defending Freedom, Failed Leadership, Foreign Policy, Leadership
Even the Hapless Jimmy Carter Wasn't This Bad
by Jack Kelly
Leaders of other countries don't respect President Barack Obama, said 53 percent of respondents in Gallup's annual World Affairs poll, conducted Feb. 3-6. That only 53 percent of Americans think this is an indictment of the news media's coverage of foreign affairs.
He would lead the world by "deed and example," not try to "bully it into submission," Sen. Barack Obama wrote in Foreign Affairs magazine in 2007.
In a major foreign policy speech in 2008, Mr. Obama said he would focus on "ending the war in Iraq responsibly; finishing the fight against al-Qaida and the Taliban; securing all nuclear weapons and materials from terrorists and rogue states; achieving true energy security; and rebuilding our alliances to meet the challenges of the 21st century." Continue reading →
Tags: Defending Freedom, Failed Leadership, Ineffectual Leader, Leadership, National Defense, Peace through strength
What America Still Can Learn From the Iron Lady
by Matt Naham
When Margaret Thatcher passed from this world on April 8, myriad books followed. Consequently, an exclusively eulogistic feel characterizes many of these works. Catalogues of the Iron Lady's rousing successes, lasting impact, and life story predominate — as is expected and appropriate — the books dedicated to her memory.
Nile Gardiner and Stephen Thompson's collaborative work, Margaret Thatcher on Leadership: Lessons for American Conservatives Today, effectively explores the success, the impact, as well as the life of Lady Thatcher and, while doing so, goes beyond mere eulogy.
On Leadership not only offers retrospective, but also proposes practical solutions for securing a promising future in the face of clear and present national malady; in its scope, this is a dynamic proposition rather than an insular one. Continue reading →
Tags: Conservatism, Leadership, Margaret Thatcher
Obama: Ducking a leader's duty
A vast scholarly literature spanning more than six decades exists on the subject of leadership. The characteristics of effective leaders have been pored over, cataloged and debated. Among them, one trait stands out as axiomatic: Effective leaders take responsibility for problems around them; they do not shift blame to others. As Winston Churchill put it, "The price of greatness is responsibility." Continue reading →
Tags: Failed Leadership, Leadership, Obama
Shutting down World War II Memorial: A Symbol of Malice
The mainstream media is in the midst of one of its regular exercises in completely missing a wave of groundswell conservative anger — this time over the closing of the World War II Memorial. It's as if the entire conservative case against the Obama administration's incompetence, malice, and inefficiency was boiled down into one incident.
1. Government overreach: This was a monument built almost entirely through private donations — now the government pretends the monument belongs to it, and not to the people who donated to build it, not to the vets whose sacrifice it honors, and not to the families of vets and other citizens who want to use it to teach their kids about courage, honor, and sacrifice. Continue reading →
Tags: Failed Leadership, Government Shut-down, Leadership, Obama
Historic Incompetence
We have a president who delights in delivering partisan speeches to adoring audiences but doesn't seem interested in whether his administration gets results.
Evidence of the astonishing incompetence of the Obama administration continues to roll in.
It started with the stimulus package. One-third of the money went to public employee union members — a political payoff not very stimulating to anyone else. Billions went to green energy loans, like the $500 million that the government lost in backing the obviously hapless Solyndra.
Infrastructure projects, which the president continues to tout, never seem to get built. He's been talking about dredging the port of Charleston, for example, to accommodate the big container ships coming in when the Panama Canal is widened. Continue reading →
Tags: Big Government, FDR, Leadership, Obama
European Perspective: Obama's not to be trusted on foreign policy
Having declared an end to the War on Terror, the US president no longer has any clear idea of his country's global role
by Janet Daley
The West can no longer rely on American leadership in the world. For the remaining duration of the Obama administration, Washington's judgment and effectiveness in foreign policy cannot be trusted. It is quite an achievement for the one remaining superpower to appear as ineffectual and wrong-footed as the United States has managed to do in the past week. But there it is. The president's global strategy in his second term was based on two resounding premises. First, al‑Qaeda was "on the run" having been smashed by the killing of Osama bin Laden and the successful US drone operations in Pakistan: in May, Mr. Obama gave a triumphal speech in which he declared the War on Terror officially over. Continue reading →
Tags: Afghanistan, al‑Qaeda, Benghazi, Failed Leadership, Foreign Policy, Leadership, NSA Spying Scandal, Obama, Pakistan, Putin, Russia
[Flashback Interview] The Iron Lady on Individual Self-Reliance ("There is no such thing as society.")
"All too often the ills of this country are passed off as those of society. Similarly, when action is required, society is called upon to act. But society as such does not exist except as a concept. Society is made up of people. It is people who have duties and beliefs and resolve. It is people who get things done. She prefers to think in terms of the acts of individuals and families as the real sinews of society rather than of society as an abstract concept. Her approach to society reflects her fundamental belief in personal responsibility and choice. To leave things to 'society' is to run away from the real decisions, practical responsibility and effective action."
by Scott L. Vanatter
Four score and seven years ago Margaret Thatcher was born in Grantham, Lincolnshire County. Today she belongs to the ages.
Of course, she always has belonged to the ages — at least since her principled leadership helped bring Britain back from the brink of a crippling socialist ruin. For good reason a Soviet journalist labeled her the Iron Lady. With Ronald Reagan she led the free world in the face of a once fearful Soviet Union.
Below are excerpts of an interview she gave as Prime Minister at No. 10 Downing Street, on September, 23, 1987. It was immediately controversial because she made the case that there was "no such thing as Society." Continue reading →
Tags: Employment, Family, Leadership, Margaret Thatcher, Social security and welfare, Society, Voluntary sector and charity


text:
Home	/ Media	/ Press releases	/ Ajay Kanwal to step down as Regional CEO, ASEAN and South Asia
Ajay Kanwal to step down as Regional CEO, ASEAN and South Asia
Standard Chartered today announces the resignation of Ajay Kanwal. He will step down from his role as Regional CEO, ASEAN and South Asia (ASA) with immediate effect. Anna Marrs will assume the role of Regional CEO, ASA, in addition to her existing role as CEO, Commercial and Private Banking subject to regulatory approval.
Ajay said: "The Bank rightly sets very high internal standards on disclosures for all staff. After careful deliberation I felt that some of my internal disclosures about my historical personal investments in businesses outside the Bank did not meet these very high standards. Though I do not own these investments anymore, as a senior leader my actions should be beyond reproach. Hence with regret I have decided to tender my resignation. I wish Bill and all my colleagues a very successful future."
Bill Winters commented: "Ajay's historical personal investments in businesses outside the Bank had no impact on our clients or the economic interest of the Bank. He has elected to put the Bank's interests above his own and I respect his decision to step down. Ajay has had a 24-year long distinguished career with the Bank and has done an outstanding job as the Regional Head. Along with my colleagues on the Management Team and his many friends across the Bank, I wish him well for the future."
Ajay will assist Anna in transitioning to the new role as required.
Anna joined the Bank in 2012 and has held a number of roles within Standard Chartered including Group Head of Strategy and Corporate Development, and is currently the CEO of Standard Chartered's Commercial and Private Banking businesses. Anna will continue to be based in Singapore.
Find out more about Anna Marrs
Jon Tracey +44 2078857613
Siow Joo Lim +65 65967759
We are a leading international banking group, with a presence in 59 of the world's most dynamic markets, and serving clients in a further 85. Our purpose is to drive commerce and prosperity through our unique diversity, and our heritage and values are expressed in our brand promise, here for good.
Standard Chartered PLC is listed on the London and Hong Kong Stock Exchanges.
For more stories and expert opinions please visit Insights at sc.com. Follow Standard Chartered on Twitter, LinkedIn, Instagram and Facebook.


2007 represents something of a watershed for us. Ever since we started with organic and biodynamic farming, people would ask us: can you taste the difference in the wine? The only reply we could give was: we don't know, because we don't know what the wine would have tasted like if we didn't do it.
The 2007 vintage has changed that. The season started with a cold spring and early summer and many of the vines throughout Central Otago started to show signs of stress. Although the second half of the growing season was warm and kind, many vines stalled on their way to ripeness, simply lacking the strength to finish the job. That was the regional picture.
We saw a different story. Our vines stayed dark in colour and full of resilience. They cruised through to ripeness. We were possibly the first to start picking, an unheard of thing when many other vineyards are about a week ahead of us in a normal season. We were the first to finish, having put all the fruit away while some were just starting their harvest. For a full month after picking, the vines stayed green and fresh looking; something we have never witnessed before. At first all this was a bit disconcerting; had we missed something? Should we have waited?
In the last few months we have been somewhat surprised to hear many people describe the 2007 vintage in Central Otago as OK… quite nice… a bit stressy. For us these are wines of unmatched concentration and rich complexity. They combine the ripeness of the 06s with the concentration of the 05s and a certain extra magic that is unique to this vintage. In short we see them as landmark wines.
There are other signs. For years we have had problems with yields and general vine strength in Block 9: a smaller Chardonnay block. Gareth decided to try an application of preparation 501: a biodynamic silica, buried through the summer months, which is supposed to increase energy in the vines. We have just harvested the best crop from this block we have ever seen with vine health looking transformed.
So this is the year when we can really say: yes, we can see a clear difference in the wines that relates straight back to the vineyard programme.
It is also the year that we finally bring Cornish Point into the Felton Road stable. Ever since we planted our Pinot Noir laboratory as many have termed it, we have been keenly watching Cornish Point's evolution and have been using some fruit to produce a Pinot Noir under the Cornish Point Label as well as using it in Felton Road Pinot Noir. This is the fifth vintage from Cornish Point and we have decided to take some of the vineyards finest blocks to create a single vineyard wine that sits alongside our Calvert Pinot Noir.
They form a fascinating contrast: where Calvert displays elegance and great purity, Cornish Point displays heady perfume and seductive opulence: very much the feminine vineyard out of the three estate properties. This probably brings the range of wines to its logical conclusion as well: the Felton Road Pinot Noir being the blend of the three vineyards, then single vineyard wines from Calvert and Cornish Point, and the two single block wines from Elms (Block 3 and Block 5).
What else is happening? Jane has just left us to start her new position as winemaker at nearby Carrick. She will be a tough act to follow, having played a key part in making not just the wines, but brownies when treats are needed and stepping in to cook lunch when Nigel's not around. Well miss her, but she is being replaced with Todd Stevens, who joins us from Quartz Reef (sorry Rudi!). Todd has settled in very fast and while not displaying quite the same skills with brownies, he's certainly at home making his first Felton Roads.
It is also great for me to have Alex Kongsgaard here to help for the vintage. Alex's dad; John, was my early mentor, taught me much about Chardonnay (he makes one of America's very finest), and its nice to return a little of the favour and show Alex something of how we make wine.
Sustainability has been in the news a lot this year as has the issue of Carbon footprints. We have added a novel twist to sustainability by planting a large veggie patch at Calvert and another at the foot of Block 5. We are specialising in growing some of the old varieties long forgotten by modern farmers. Pink Fir Apple potatoes and Brandywine tomatoes have been very popular as well as French beans. Next year we intend to extend the programme and increase the varieties and selection.
We have been doing a lot of work measuring our carbon footprint and comparing it against other producers. The detail would take far too long to go into, but we are very proud of where we stand in the comparison charts and we have been asked to present some of our work at a conference in the US later in the year.
I hope you enjoy drinking the 2007 wines. We are very proud of them and we trust you'll share our enthusiasm.


Welcome to my website! Here you will find lots of useful information about our classroom happenings. Visit my page to gain access to helpful links to other CB sites too. Enjoy exploring!
Click here for home access directions for Clever. The Clever Application will give your child access to every software program we use in school including Typing Agent, Social Studies TCI, PowerPoint, Microsoft Word, and so much more!
Check out some highlights from our Cultural Artifact Share!
Below is a link to Titus' Parent Volunteer page with FAQ and information letting you know how you can get involved!
Click the link to see who is in our class this year!
Titus Elementary believes in providing second chance learning when students have not demonstrated their understanding of secure grade level skills. Second chance learning includes the re-teaching of secure skills followed by a reassessment of those concepts. These opportunities will vary based on individual student needs and teacher discretion. Parents/students will be notified when second chance learning is necessary.
On progress reports, student grades and teacher comments will reflect the second chance learning given throughout the marking period.
Click the link to see what our schedule looks like this year!


Located on stunning Ninh Van Bay, with its immpressive rock formations and small sandy coves to the front and forested mountains to the rear of the resort. The guest villas nestle in amongst the rocks, mountain and forests.
Child from 0 to 5 years old complimentary room and breakfast while sharing existing bed with parents. Meal for free from children s menu.
Child from 6 - 11 years old complimentary room and breakfast while sharing existing bed with parents. Full charge from children s menu, 50 off for regular menu.


text:
\section{Introduction}
\label{introduction}
Certain applications require \textit{continual} learning---splitting training into a series of tasks and discarding the data after training each task. These settings vary tremendously. If data relate to individuals it may be unethical, illegal, or imprudent to retain old datasets: for example, a hospital might want to delete old patient data. In other applications, real-time systems could face distributional shifts with an underlying distribution which changes faster than the time it would take to retrain a new model with all the data. For example, a Mars rover might cross regions of different terrain that require adaptation without forgetting. Neural networks trained on such series of tasks tend to forget earlier tasks (often referred to as \textit{catastrophic forgetting}).

Recent works have shown promising advances towards continual learning by adding regularization terms to the loss function which preserve important parameters \citep{kirkpatrick_overcoming_2017, Zenke2017, nguyen_variational_2018, Chaudhry2018, Ritter2018}. We call these approaches \scare{prior-focused} because they can be interpreted as treating intermediate models as priors when learning updated weights. Although a useful starting point, experimental evaluations in these recent works have major blind-spots which mask weak-points of current approaches. The design of the evaluations does not fully reflect the core motivations for continual learning, regardless of the dataset used, including evaluations based on MNIST \citep{LeCun1998}, notMNIST \citep{Bulatov2011}, FashionMNIST \citep{xiao_fashion-mnist:_2017}, CIFAR10 \citep{Krizhevsky2009} and others. But evaluations which obscure the shortcomings of suggested continual learning solutions impede developments in the field, since researchers are not made aware of limitations of past research.

In this paper, we explore desiderata for continual learning evaluations based on real-world uses. Applications of continual learning are very diverse, and authors consider a wide range of settings. As a result, rather than propose a single benchmark which risks being overly narrow, we propose fundamental desiderata for empirical evaluations in continual learning to make them more representative of the challenges that motivate the field. We then demonstrate that experiments which neglect our core desiderata can be misleading---a continual learning system that performs well when a subset of the desiderata are in place might fail entirely when a core desideratum is imposed. No prior-focused continual learning system has so far been shown to succeed when all five desiderata are applied. We demonstrate that several leading prior-focused approaches stop working on a more robust evaluation. We therefore argue that existing evaluations are biased, because prior-focused approaches appear to succeed, but have major blind-spots. Lastly, we introduce new evaluations based on our list of desiderata which offer richer challenges for continual learning systems.

Specifically, our four main contributions are as follows:
\begin{enumerate}
\item We propose fundamental desiderata for future evaluations, which can be applied regardless of dataset.
\item We analyse the shortcomings of existing widely used evaluations in continual learning.
\item We show empirically that existing evaluations are biased towards prior-focused approaches.
\item We propose new experimental designs which mitigate the issues of existing ones.
\end{enumerate}

\section{Continual Learning}
We start by formalising the definition of \textit{continual learning}.
In a typical supervised learning setting, we aim to learn parameters $\mathbf{w}$ using an independently and identically distributed (i.i.d.) labelled training dataset $\mathcal{D} \equiv \{(\mathbf{x}^{(i)}, y^{(i)})\}$ to accurately predict $p(y^*|\mathbf{w}, \mathbf{x}^*)$ for an unseen $(\mathbf{x}^*, y^*)$ pair.

In the continual learning setting, members of $\mathcal{D}$ are not i.i.d. Instead they may be split into disjoint subsets $\mathcal{D}_{t} \equiv \{(\mathrm{x}_t^{(i)}, y_t^{(i)})\}$. These sets are assumed to be drawn from $T$ distinct i.i.d. distributions each of which represents a \textit{task}. The challenge of continual learning is to learn a single model which is able to predict well on data from any task, despite training on each task in sequence without deliberately revisiting previous tasks. For some applications, stronger or weaker assumptions are made. For example, \citet{nguyen_variational_2018} concern themselves with fast adaptation, so they allow themselves to retain a small coreset of data from old tasks. Similarly, \citet{kirkpatrick_overcoming_2017, schwarz_progress_2018} consider a simulation in which they allow themselves to revisit each task multiple times. In this paper, we will frequently consider a model after training on some, but not necessarily all $T$ tasks. The $t$'th model is the state of the model after training on datasets $\{D_1:D_t\}$.

\section{Proposed Desiderata for Continual Learning Evaluations}\label{desiderata}
The setting and motivation for continual learning can vary enormously, from image segmentation to medical diagnostics to control. Specialized continual learning datasets like Core50 \citep{Lomonaco2017} or environments such as that in \citet{schwarz_towards_2018} are therefore useful for subfields but cannot be seen as an overall benchmark for continual learning. We therefore concentrate instead on developing set of sufficient desiderata for future evaluations of continual learning. We propose the following fundamental principles which ought either to be respected or to be explicitly mentioned as absent. In \S\ref{critical_analysis} we offer a critical analysis of existing experiments which suggests why core desiderata are important. In \S\ref{failure_modes}, we present a set of experiments showing how neglecting these core aspects can make a continual learning system look more successful than it is.

Use cases of continual leaning vary tremendously. One example is a hospital developing a system for automated disease diagnosis using patient sub-population A which is then refined in a different hospital with patient sub-populations B (with sub-population A data not allowed to leave the first hospital). A second example is a wind-turbine safety system which predicts wind velocity an hour ahead to decide whether to turn off the turbine in case of strong winds, with wind dynamics changing between the seasons of the year. A third example is an ad serving system that has to update with huge amounts of streaming data, without feasibly being able to keep all data around. Yet another use case is a Mars rover updating its behaviour as it encounters new types of terrain---perhaps much looser or coarser soil than it has seen before---but which must remain able to manoeuvre on old terrain. All use cases share common core desiderata, which we demonstrate next using the Mars rover example for concreteness. Note that this example is for illustrative purposes only---we are \textit{not} proposing that this example should be used for evaluations. Our desiderata arise just as readily from other settings.

\begin{compactenum}
\item[\textbf{A: Cross-task resemblances}]{Input data from later tasks must resemble old tasks enough that they at least sometimes result in confident predictions of old classes, early in training. The widely used Permuted MNIST (see \S\ref{permuted_description}) which violates this would correspond to every input sensor in our Mars rover being randomly rewired---unlikely to be representative of real cases.}
\item[\textbf{B: Shared output head}]{If each task is given a different output vector during training and testing, it must be explicitly and prominently mentioned. This has a large effect on the difficulty of the challenge. In our example, the rover is using the same outputs to control its motors in all terrain.}
\item[\textbf{C: No test-time assumed task labels}]{Related to desideratum B, if we knew in advance what the tasks were and had a way to distinguish them, one could have a different model for each task and switch between them. In our example, if the rover has a good way to distinguish rock from dust and can learn a separate policy for each it might not need continual learning.}
\item[\textbf{D: No unconstrained retraining on old tasks}]{Many motivations of continual learning preclude being able to retrain on task $1$ after having trained on task $T$. In our example, the rover needs to actually move into terrain it has left behind before it can train there. In other examples, even retaining small amounts of data from old tasks might violate privacy laws}
\item[\textbf{E: More than two tasks}]{The more tasks a continual learning system can handle the better. Our rover, for example, may face a potentially open-ended and large number of tasks. Two-task transfer (\S\ref{two-task}) is therefore interesting, but succeeding at two-task transfer does not guarantee good performance on more tasks.}
\end{compactenum}

In certain applications we might deliberately ignore some of the desiderata. For example, \citet{nguyen_variational_2018} neglect desiderata B and C because they consider fast adaptation to new tasks which have labels during both training and testing. In a setting with easy access to a simulation of all tasks, retraining might be permissible, setting aside desideratum D (as in \citet{schwarz_progress_2018}). Desideratum E is mostly set aside for convenience---there might be interesting intermediate research which is nevertheless useful progress for the field. Desideratum A is mostly neglected for historical reasons discussed in \S\ref{crit_permuted}. While the five listed desiderata will not always be applicable, it remains a surprising fact that there is \textbf{no prior-focused approach that has been shown to perform well in an evaluation with all of the five core desiderata}.

The core desiderata are a starting point which can lead to more difficult challenges, some of which are beginning to be addressed by recent work, including:
\begin{compactenum}
\item[\textbf{Unclear task demarcation}]{Task boundaries are generally assumed knowledge during training.}
\item[\textbf{Continuous tasks}]{Current practice usually assumes that tasks are discretely different rather than varying continuously.}
\item[\textbf{Overlapping tasks}]{Current practice tends to have tasks where each task has its own classes which are disjoint from the other tasks. \citet{Lomonaco2017} discuss this point.}
\item[\textbf{Long task sequences}]{Continual learning is most interesting over extremely long sequences of tasks.}
\item[\textbf{Time/compute/memory constraints}]{It is widely acknowledged that continual learning solutions that update quickly or use small/fixed memory are more useful than ones that have large computational overheads.}
\item[\textbf{Strict privacy guarantees}]{For settings motivated by privacy, tracking the differential privacy guarantees between tasks is important.}
\end{compactenum}

Each of these further desiderata offers a way to make a continual learning evaluation more representative of the needs of applications \textit{without making the dataset itself harder}. In \S\ref{further_evaluations} we suggest two experiments that address the most interesting of these. In \S\ref{mutual_information} we show how model uncertainty can be used to detect task changes, which is essential in many practical settings and allows streaming settings to be converted into continual learning ones. In \S\ref{time} we show how training time and accuracy must be traded off against each other as design choices, rather than just reporting the time taken by the most accurate system. Memory can be treated like time, here, and the other desiderata are straightforward to construct from existing datasets by tweaking the dataloader.

We note that a very real risk for the field is to \textit{embrace more complex datasets before establishing robust experimental design.} Until we have continual learning evaluations that test the core challenges of continual learning, moving from MNIST to more complex datasets does little to measure progress in continual learning itself.
\begin{figure}
    \centering
    \includegraphics[width=0.8\columnwidth]{venn_diagram}
    \vspace{-5mm}
    \caption{We contrast prior-focused with likelihood-focused continual learning. By comparing these, and hybrid forms, we can see which evaluations pose a bigger challenge to different approaches.}
    \label{fig:venn}
    \vspace{-3mm}
\end{figure}

\section{Existing Work} \label{prior_work}
In this paper, we critically analyse common evaluations used by the majority of recent papers. In this section, we review existing work focusing first on the methods employed (\S\ref{methods}) and second on the evaluations used (\S\ref{evaluations}). Throughout, we provide a framing which anticipates our critical analysis.

\subsection{Methods}\label{methods}
Approaches to continual learning and catastrophic forgetting form three main families. First, prior-focused approaches use regularization to create \scare{elastic} parameters. These include: Elastic Weight Consolidation (EWC) \citep{kirkpatrick_overcoming_2017, huszar_note_2018}, Synaptic Intelligence (SI) \citep{Zenke2017}, Variational Continual Learning (VCL) \citep{nguyen_variational_2018}, Kronecker-factored approximated Laplace approximation \citep{Ritter2018}, and Riemanian Walk (RW) \citep{Chaudhry2018}. In fact, all of these can be seen as setting a distribution over the parameters of the $t-1$'th model as the prior when training on the $t$'th dataset.

VCL makes this most explicit. VCL is a variational inference (VI) method that uses Bayesian neural networks \citep{MacKay1992, Neal1995, Blundell2015}. VCL sets the posterior at the end of training a task to be the prior when beginning training for the next task. Schematically, as the authors of these papers note, all the approximately Bayesian approaches use a similar loss function:
\begin{equation}
\label{eq:prior}
\begin{split}
    \mathcal{L}_t = &\sum_i \mathrm{log}\big(p(y_t^{(i)}|\mathbf{w}_t, \mathbf{x}_t^{(i)})\big) \\
    &- \frac{c}{2}(\mathbf{w}_t - \mathbf{w}_{t-1})^T\Sigma_{t-1}^{-1}(\mathbf{w}_t - \mathbf{w}_{t-1})
\end{split}
\end{equation}
where $\Sigma$ is the covariance of the prior, which is estimated differently under each approach. The first term reflects model log-likelihood on the $t$'th task. The second reflects distance from the model prior.

Prior-focused approaches rely on the prior term to capture everything learned on previous tasks and only use the newest data to estimate the likelihood term. However, in practice the prior term is only approximated. \citet{farquhar_unifying_2018} contrast prior-focused approaches with a second family, likelihood-focused approaches, that attempt to estimate the likelihood of the current model on past tasks  (see figure \ref{fig:venn}). Examples of this include Deep Generative Replay (DGR) by \citet{Shin2017}, which uses generative models to store summaries of old tasks, or Variational Generative Replay (VGR) a variational and simplified alternative presented in \citet{farquhar_unifying_2018}. Simply storing a small random sample or coreset is an alternative approach in this family. Many members of this family are called rehearsal strategies after `pseudorehearsal' introduced by \citet{Robins1995}, or called dual-memory strategies following a neuro-scientific motivation. However, some dual-memory strategies, e.g., GEM \citep{lopez-paz_gradient_2017} or model adaptation \citep{hu_overcoming_2019}, are not likelihood-focused.

Some authors have observed better results for prior-focused methods with the addition of some likelihood estimation, although this has not been rigorously explored. For example, VCL is also presented using coresets---fine-tuning on a small withheld sample just before testing. Similarly, RW presents results both with and without training on subsamples of previous datasets.

The last major family uses dynamic architectures. These change the structure of the networks in significant ways to incorporate learning from each task separately (e.g., \citet{Rusu2016, Li2017}). These perform well, but in most cases their need for new models for each task creates growing compute/memory costs that do not seem to truly solve the continual learning problem. A more thorough survey of these as well as prior- and likelihood-focused approaches can be found in Appendix \ref{furtherpriorwork}. We focus our examination on prior-focused and their counterpart likelihood-focused methods, because they seem to represent the most exciting recent advances in continual learning, but the experiments used to support them may not be fairly evaluating their performance.

\subsection{Experimental Evaluations}\label{evaluations}
In this section we give an in-depth review of key experimental evaluations as they are used in \textit{current research}. We illustrate our critique using the most commonly performed experiments using variations of the MNIST dataset: \textit{Permuted MNIST} and \textit{Split MNIST}, most commonly in \textit{multi-headed} form. Although we emphasise MNIST for explanatory purposes, our critique in \S\ref{critical_analysis} applies to many other evaluations which share similar design choices.

\subsubsection{Permuted MNIST}\label{permuted_description}
The Permuted MNIST experiment was introduced in \citet{Goodfellow2013}. In their experiment, a model is trained on MNIST as $\mathcal{D}_1$. Each later $\mathcal{D}_t$ for $1 < t \leq 10$ is constructed from the MNIST data but with the pixels of each digit randomly permuted. A fresh permutation is drawn for each task and applied to all images in the same way for that task. After training on each dataset, one evaluates the model on each of the previous datasets as well as the current. \citet{Goodfellow2013} used this experiment to investigate feature extraction, but it has since become a mainstay for continual learning evaluation \citep{Zenke2017, Shin2017, kirkpatrick_overcoming_2017, Lee2017, lopez-paz_gradient_2017, nguyen_variational_2018, Ritter2018, hu_overcoming_2019, chaudhry_efficient_2019}.

\subsubsection{Split MNIST}\label{prior_split}
The Split MNIST experiment was introduced by \citet{Zenke2017} in a \textit{multi-headed} form and used by other authors including \citet{Shin2017, nguyen_variational_2018, Chaudhry2018} (\citet{Ritter2018} use a two-task variant). The experiment constructs a series of five related tasks. The first task is to distinguish the digits (0, 1), then (2, 3) etc. Most papers use a multi-headed variant the model prediction is constrained to be only from the two classes represented in each task. For example, when evaluating the performance on the first task, the model only needs to predict probabilities for zero versus one. In some cases, multi-heading is taken even further and training is only done on the head governing the specific classes included in the task \citep{Zenke2017, nguyen_variational_2018, Ritter2018}. A \textit{single-headed} version does not limit predictions during either training or testing. As \citet{Chaudhry2018} note, the multi-headed variant is much easier to solve. Multi-heading is often used in similar non-MNIST evaluations, for example, in \citet{Zenke2017, nguyen_variational_2018, Ritter2018}. \citet{Chaudhry2018} use both a single- and multi-headed version of Split MNIST. \citet{hu_overcoming_2019} use a single-headed set-up but their method effectively learns a different head for each task.

\subsubsection{Two-task transfer}\label{two-task}
In some works, a two-task transfer learning evaluation is used for continual learning. For example, \citet{Shin2017} train first on MNIST and then on SVHN \citep{Netzer2011} (or vice versa) in order to see whether their algorithm preserves performance on the first task after training on the second. \citet{Jung2016a}, \citet{Li2017}, and \citet{pfulb_comprehensive_2018} perform similar experiments.

\subsubsection{Desiderata, benchmarking and metrics}
Indicating the importance of robust evaluations, several authors have recently included some continual learning desiderata alongside their work. For example, \citet{schwarz_progress_2018} list desiderata for continual learning systems which includes being able to handle long sequences of tasks without knowing task labels and ideally no clear task demarcation, but do not introduce experimental set-ups which could enforce all these desiderata, such as the ones we propose in \S\ref{further_evaluations}. \cite{chaudhry_efficient_2019} call for time and memory constraints as well as careful use of cross-validation. \citet{pfulb_comprehensive_2018} call for testing on many datasets, care with cross-validation, and storage constraints. \citet{farquhar_differentially_2018} discuss the importance of tracking differential privacy guarantees in continual learning settings motivated by privacy.

Several authors have commented on the importance of systematic benchmarks for continual learning, including \citet{Lomonaco2017}, focused on object recognition, and \citet{schwarz_towards_2018}, focused on control in a StarCraft environment. Others have worked towards richer metrics for measuring continual learning performance like forward/backwards transfer or learning curve area \citep{lopez-paz_gradient_2017, Chaudhry2018, schwarz_progress_2018, diaz-rodriguez_dont_2018, chaudhry_efficient_2019}. Both of these developments are orthogonal to the arguments of this paper --- well-chosen metrics evaluated in a challenging environment are still unhelpful if the experiment is not designed to reflect the continual learning needs of diverse applications.

\section{Critical Analysis of Existing Evaluations}\label{critical_analysis}
Pixel permutation (e.g., Permuted MNIST), split classification (e.g., multi-headed Split MNIST), and two-task transfer experiments are commonly used to compare continual learning algorithms. Although they are a useful starting point, each of these makes continual learning easier for prior-focused approaches. All of our critiques of the way MNIST is used by \citet{Zenke2017} apply also to their use of the CIFAR100 dataset, the use of notMNIST, Fashion MNIST, SVHN and CIFAR10 in \citet{Ritter2018}, and the use of notMNIST in \citep{nguyen_variational_2018}. These critiques are about experimental design, not dataset choice.

\subsection{Pixel Permutation}\label{crit_permuted}
Permuted MNIST represents an unrealistic best case scenario for continual learning---although it satisfies the literal definition of continual learning. The positions of the pixels are fully randomized---which suited its original purpose. \citet{goodfellow_empirical_2013} investigated whether neural networks have `high-level concepts' that get re-mapped to pixel positions when the input space is permuted, and found evidence they did not. Now, however, the experiment has been repurposed for continual learning, where it is not suitable.

An image from each permuted dataset is practically unrecognizable given previous datasets. The actual world is almost never structured like this---new situations look confusingly similar to old ones---the sensor input in a Mars rover will never be permuted no matter what terrain one moves onto. A model presented with a new task in Permuted MNIST will be uncertain, while in settings that are not deliberately randomized a model will tend to make confident but false predictions. This makes Permuted MNIST significantly different from real-world settings in a way that directly affects how new tasks are learned. In Appendix \ref{permuted_problems}, we offer an empirical investigation of this phenomenon and a hypothesis for the mechanism by which Permuted MNIST simplifies the continual learning challenge.

\subsection{Split Classification}\label{crit_split}
The multi-headed version of Split MNIST, which is most often used, is easier but less relevant to applications because it requires knowledge of the task and the classes represented in each task, as \citet{Chaudhry2018} point out. Usually, if that were possible, continual learning would be unnecessary, since one could use a separate classifier for each task, which generally would perform better.

Unfortunately, prior-focused continual learning systems tend to look much better in multi-headed evaluations than single-headed ones. Suppose some model has already been trained on the first four tasks of Split MNIST and is then tested on the digit \scare{1}. In the single-headed variant, when shown a \scare{1} it may incorrectly predict the label is seven, which was seen more recently. In the multi-headed variant, we \textit{knowingly assume} that the label comes from [0:1]. Because the model only needs to decide between 0 and 1 (and not even consider if the image is a 7), a multi-headed model could correctly predict the label is 1 even though the same approach will completely fail in a single-headed experiment setup. Most current prior-focused continual learning systems are good enough that models retain the ability to distinguish old classes, but only if they are able to know which task the example is from. A multi-headed evaluation can therefore make it seem as if an approach has solved a continual learning problem when it has not. \textit{Single-headed Split MNIST}---a less-used evaluation---is the simplest experimental setting that meets all five of our core desiderata. We recommend it as a toy baseline, which is easily adapted to harder datasets. It satisfies our core desiderata: \textbf{A:} The tasks resemble each other---e.g., a seven can look like a one; \textbf{B:} Single-heading means all outputs are shared; \textbf{C:} A full prediction is made over all possible outputs each time; \textbf{D:} Each task is trained only once and only the model is carried forwards; and \textbf{E:} There are five tasks.

\subsection{Other non-representative evaluations}
Two-task transfer is not representative of realistic problems because continual learning use-cases might require a long series of tasks, not just two. An algorithm might have elements that perform well with just one previous task but fail with more. For example, the approximate Fisher information matrix used in EWC is estimated using a Taylor expansion which is only locally accurate in one part of parameter-space. If, after many tasks, the model reaches a very different part of parameter-space, the estimated Fisher information for old tasks will be inaccurate. Moreover, in practice two-task transfer is just easier than true continual learning: \citet{pfulb_comprehensive_2018} conclude that a simple fully connected network will show no catastrophic forgetting on a two-task Permuted MNIST but do not consider longer task sequences, while other authors have already shown that with more tasks there is indeed catastrophic forgetting \citep{kirkpatrick_overcoming_2017}.

\section{Empirical Analysis of Existing Evaluations}\label{empirical_desiderata}
This section has three aims. First, we show that experiments which satisfy only a subset of the five main desiderata from \S\ref{desiderata} can have blind-spots that are important to applications that demand all five desiderata. We do this by showing that leading methods fail on an experiment that uses all five desiderata, while they look good on experiments that use just a subset. Second, we show that existing evaluations are biased towards prior-focused approaches. Third, we suggest two further experimental set-ups.

\subsection{Experimental approach}
We consider representatives of the prior-focused approach, likelihood-focused approach, and a hybrid approach. We use these to demonstrate the shortcomings of existing evaluations and to tease out which aspects of the architecture are challenged under different experimental design choices.

We use three variants of VCL because it has the clearest Bayesian interpretation. First, we use pure VCL without a coreset, exactly as described by \citet{nguyen_variational_2018} (see Appendix \ref{furtherpriorwork}). Second, we use a small coreset of 40 datapoints extracted from each dataset (we use their k-center coreset approach, but this does not have a large effect). The second method is the same as pure VCL except that, at the end of training on each task, the model is trained on the coresets before testing, as described in their work. This reflects a hybrid approach. Third, we use a \scare{coreset only} approach as a likelihood-focused version. It is exactly like the second variant except that the prior used for variational inference is the initial prior each time---it is not updated after each task.\footnote{This is not the coreset only algorithm used in \citet{nguyen_variational_2018}. Theirs is seeing \textit{only} coresets---much less data---which is why it performs badly on even the first task.}

We use two further baselines to show that the effects we find are not an artefact of VCL specifically. EWC \citep{kirkpatrick_overcoming_2017} is used as a prior-focused alternative to VCL because it has been widely discussed in the literature. VGR \citep{farquhar_unifying_2018} is used as a more powerful likelihood-focused alternative to coresets. VGR, VCL and its variants use a Bayesian neural network (BNN) and variational inference, while EWC does not, so performance comparisons between these architectures should be interpreted carefully. The fact that our observations for VCL and its variants are mirrored for EWC supports our hypothesis, based on our analysis, that the problems discussed in this section affect prior-focused approaches in general.
\begin{figure}[t]
\vspace{-2mm}
\includegraphics[width=\columnwidth]{single_MNIST}
\vspace{-11mm}
\caption{\textbf{Single-headed Split MNIST.} This experiment meets all core desiderata and shows big performance differences.}
\label{fig:split_single_all}
\end{figure}

\begin{figure}[t]
\vspace{-5mm}
\includegraphics[width=\columnwidth]{single_Fashion_MNIST.pdf}
\vspace{-11mm}
\caption{\textbf{Single-headed Split Fashion MNIST.} The harder dataset shows the prior-approximation starting to deteriorate, but does not change performance ranking.}
\vspace{-2mm}
\label{fig:fashion_single}
\end{figure}

\subsection{Experiments With All Five Desiderata Show Big Performance Differences}
We use Single-headed Split MNIST, described in \S\ref{crit_split}. This satisfies all main desiderata from \S\ref{desiderata}. It reveals major differences in performance between the approaches (see figures \ref{fig:split_single_all} and \ref{fig:fashion_single}). The performance of VCL with coreset appears to be entirely driven by the presence of the coresets. When coresets are removed, VCL alone completely forgets old tasks---its accuracy comes from correctly classifying the most recent task only. EWC performs exactly like VCL, suggesting that it is prior-focused approaches in general that are struggling. Meanwhile VGR, the more accurate likelihood-focused approach, is the only method that performs nearly perfectly. Using FashionMNIST rather than MNIST is harder and a worthwhile additional test, but does not reveal a radically different story. VCL performs much worse on FashionMNIST than VGR even though both use the same model. It seems the prior approximation struggles with data complexity more than the GAN.
\begin{figure}[t]
\centering
\begin{minipage}{\columnwidth}
\vspace{-2mm}
\includegraphics[width=\columnwidth]{multi_MNIST}
\vspace{-12mm}
\caption{\textbf{Multi-headed Split MNIST.} All methods succeed.}
\label{fig:split_multi_all}
\end{minipage}
\begin{minipage}{\columnwidth}
\includegraphics[width=\columnwidth]{multi_Fashion_MNIST.pdf}
\vspace{-12mm}
\caption{\textbf{Multi-headed Split FashionMNIST.} All perform similarly, so no clean differentiation. VCL performs slightly worse without coresets.}
\label{fig:fashion_multi}
\vspace{-6mm}
\end{minipage}
\end{figure}

\subsection{Missing Desiderata Create Blindspots}\label{failure_modes}
Having shown that there are big differences in performance between our baselines on a well-constructed experiment that follows all desiderata, we now show that experiments that do not include all desiderata can fail to reveal these important performance differences. This means that these experimental evaluations can be misleading. The fact that they systematically hide shortcomings of prior-focused approaches suggests that the evaluations are inadvertently biasing research efforts in the field. We use MNIST and FashionMNIST to show that while using more complex datasets does improve evaluations, it does not make up for unrepresentative experimental design.

\subsubsection{No cross-task resemblance}
As we argue in \S\ref{crit_permuted}, Permuted MNIST is unusual in that there is no resemblance between images in each task that might cause confident incorrect predictions. Previous authors have demonstrated the effectiveness of their methods on Permuted MNIST \cite{kirkpatrick_overcoming_2017, nguyen_variational_2018}. Virtually any continual learning method proposed has shown good performance on this task. This demonstrates the weakness of experiments that do not reflect desideratum A. Further experimental investigation of these phenomena is in Appendix \ref{permuted_problems} with baseline results in Appendix \ref{perm_details}.

\subsubsection{No shared output head}
Authors frequently use a multi-headed version of Split MNIST, following \citet{Zenke2017}, which we describe in \S\ref{prior_split}. On this evaluation, all approaches look similarly good (see figures \ref{fig:split_multi_all} and \ref{fig:fashion_multi}). This shows that multi-heading, and neglecting desideratum B, can create misleading experiments. Details of training and hyperparameters can be found in Appendix \ref{multi_details}.

\begin{figure}[t]
\centering
\begin{minipage}{\columnwidth}
\vspace{-2mm}
\includegraphics[width=\columnwidth]{figures/ttk_MNIST.pdf}
\vspace{-12mm}
\caption{\textbf{Test-time Knowledge Split MNIST.} All succeed.}
\label{fig:ttk_mnist}
\end{minipage}
\begin{minipage}{\columnwidth}
\includegraphics[width=\columnwidth]{figures/ttk_Fashion_MNIST.pdf}
\vspace{-12mm}
\caption{\textbf{Test-time Knowledge Split FashionMNIST.} The experiment is harder but still does not cleanly differentiate prior- and likelihood-focused methods.}
\label{fig:ttk_fashion}
\end{minipage}
\vspace{-6mm}
\end{figure}

\subsubsection{Test-time Task Knowledge}
Instead of entirely separate heads for each task, one might simply know which output classes to look at during testing. In this variant, which breaks desideratum C, everything is exactly the same as for the single-headed variant during training, but during testing all outputs except those for the correct class are masked with zeros before the most probable class is selected. Here, both on MNIST and FashionMNIST the evaluation does not differentiate the performance of prior- and likelihood-focused methods (see figures \ref{fig:ttk_mnist} and \ref{fig:ttk_fashion}).

\subsubsection{Retraining on old tasks}
An experimental evaluation that is exactly like single-headed Split MNIST except that the tasks are looped through several times does still find that prior-focused methods perform badly (see Appendix \ref{retraining}). Neglecting this desideratum, therefore, might not make evaluations biased, though it is certainly not in keeping with the core motivations of continual learning applications since it requires data to be kept indefinitely.

\subsubsection{Only two tasks}
Many authors have tested their methods with only two tasks. This is an interesting challenge, but significantly simpler than a longer series. Showing that a method succeeds on a two-task transfer does not entail that it will work on a longer series. For example, looking at the first two tasks in figure \ref{fig:split_single_all} would not differentiate VCL with coresets and VGR, where looking at the longer series shows a clear difference in performance.

\subsection{Further Experimental Options}\label{further_evaluations}
Having shown the importance of our five main desiderata in experimental evaluations, we further suggest experiments based on the extended desiderata listed in \S\ref{desiderata}. The first tests the ability of the model to detect a change in task, which has not yet been demonstrated in continual learning. The second evaluates time against accuracy (or some other metric) as a trade-off that must be carefully selected for different settings.

\subsubsection{Model Uncertainty on Unseen Tasks}\label{mutual_information}
Model uncertainty gives another tool for understanding forgetting. A good continual learning model should be more certain on tasks it has seen before than unseen tasks. To test the ability of model uncertainty to distinguish a task boundary, we assess the uncertainty of the model, for each batch of training, on data from the current task. We find, for a model trained using VGR, that model uncertainty spikes at the start of each new task (see figure \ref{fig:fashion_vgr_mi}). This means that VGR, or another method with sufficiently good model uncertainty estimation, does not need to be told task boundaries. Before each training epoch, we can compare the model uncertainty on the new data to the previous running average uncertainty. If the change in model uncertainty is above any sensible cut-off threshold we see a task boundary has occurred and trigger training of the generative model with a cache of the previous epoch's data. One could either cache the last epoch's data or train the generative model alongside the task training, but either way task detection comes with some additional memory and computational overheads. To measure uncertainty, we use the mutual information between predictions on each task and the model posterior, following \citet{Gal2016}.

\begin{figure}
\centering
\begin{minipage}{\columnwidth}
\vspace{-2mm}
\includegraphics{figures/FashionMNIST_VGR_MI_during_training.pdf}
\vspace{-9mm}
\caption{\textbf{Mutual Information on FashionMNIST with VGR.} Uncertainty spikes at the start of each new task (black dotted line), allowing task boundary detection. Uncertainty is assessed using the mutual information between predictions on each task and the model posterior.} \label{fig:fashion_vgr_mi}
\end{minipage}
\begin{minipage}{\columnwidth}
\vspace{1mm}
\includegraphics[width=\columnwidth]{figures/split_second_MNISTsingle.pdf}
\vspace{-9mm}
\caption{\textbf{Timed Single-headed Split MNIST.} Hyperparameter choice can lead to slower, more accurate, models for each architecture. Ideal performance is in the top left corner. VGR is accurate but slower than VCL.}
\label{fig:split_second_single}
\vspace{-7mm}
\end{minipage}
\end{figure}

\subsubsection{Time and Memory}\label{time}
\textit{Timed Split MNIST} evaluates the trade-off of training time against accuracy in continual learning, rather than reporting a single run-time. Much of the need for continual learning comes from situations where a system cannot feasibly retrain on all previous data due to constraints of cost and time. Architecture selection and hyperparameter selection, therefore involve a tradeoff of performance and speed. For example, in VGR, training on larger sampled datasets improves performance but takes more time. In figure \ref{fig:split_second_single} we show the average accuracy of various models plotted against the wall-clock time taken to finish training for a wide range of such hyper-parameters. While it is good to report the time taken by the most accurate model from one approach, representing the trade-offs within each approach gives a fairer reflection of the costs and benefits of each method. None of the configurations of VCL provides good accuracy on the single-headed task. At the same time, none of the configurations of VGR is able to provide good time performance (see Appendix \ref{comparing_prior_likelihood} for further details and hyperparameter choices).

\section{Discussion}
Experimental design shapes the field by providing reproducible comparisons between architectures, and testing whether an architecture satisfies fundamental objectives. Accidental blindspots in existing evaluations have created misleading comparisons in recent research. When experiments better reflect the continual learning problem, recent leading approaches fail even on simple datasets like MNIST. Using more realistic datasets is important, but until we improve the experimental design, the field will only show illusory progress on those datasets, without overcoming fundamental challenges inherent to continual learning itself. Authors of new papers currently feel compelled by the reviewing process to assess new methods on the same, incomplete, evaluations used by old papers. This paper reflects an attempt to reassess the experimental foundations of continual learning and provide a new direction for future work.

redpajama_set_name: RedPajamaArXiv


Is There Iowa Workers' Compensation for Volunteers?
Iowa law generally requires a person to be considered an employee to be covered under workers' compensation, however, there are some exceptions. For example, Iowa Code §85.36(9)(a) provides compensation to volunteer fire fighters, emergency personnel, reserve police officers, ambulance drivers and other volunteer medical personnel. The compensation includes medical care, medical mileage, money while they are off work healing from their injuries, and compensation for permanent injuries. The amount of wages that are used to determine compensation as one of these types of volunteers is set at either the pay they receive at their regular job or 140% of the statewide average weekly wage (usually around $1,100 gross), whichever is more.
Are There Other Volunteer Jobs that are Covered?
There can be other situations where a volunteer should receive Iowa workers' compensation benefits, but much of it depends upon the facts. For example, students participating in a work study type program are usually considered employees. Elected and appointed officials like a mayor or city council persons are employees. Inmates who are working at a prison or outside of a prison at a public or charitable facility are usually considered employees of the state. However, keep in mind that there are different wage laws for these types of volunteers compared to medical personnel described above.
What Should I Do if I Am Injured While in a Volunteer Position?
If you or a loved one were working as a volunteer and believe that you should receive workers' compensation benefits then you should call a qualified work injury attorney right away. We offer a review of Iowa work injuries at no cost or risk to you, so call now to see if we can help.
To receive a Free book entitled "Iowa Workers' Compensation- An Insider's Guide to Work Injuries" which describes Iowa work injuries including the "7 Mistakes To Avoid If You Are Hurt At Work" Call Now 1-800-707-2552 (ext. 511) (24 Hour Recorded Message) or click on the link above. Why offer a Book at No Cost? Iowa Workers' Compensation Attorney Corey J. L. Walker practices primarily in workers' compensation law and has represented hundreds of Iowans hurt at work and has seen too many clients from Des Moines, Newton, Cedar Rapids and throughout the state of Iowa make mistakes before they had the "right" information about work injuries resulting in them losing thousands of dollars. Iowans hurt at work are beginning to realize that the insurance company is not there to help them and that they should have someone on their side. Finally, you can learn about work injuries in the comfort of your own home with no cost, risk or obligation. For immediate assistance call 641-792-3595 and ask for Corey or Erik.