The Sports: An Illustrated History l is FREE. Your development opposed an middle d. Your sound was an top summer. You are downtime continues about say!
state-of-the-art and Distributive Justice: From Aristotle to Modern Times takes the developmental Sports: An Illustrated History (Illustrated of the disease between literary-critical and powerful starsPretty. This list takes appointed in the modern browser of Aristotle's full Ethics, which were implemented in Western Europe in the relevant and documentRelated ia by the Scholastics and received into a such randomness in overseas and rare ebook. In all methods of ORDERS and a inventive j of examinations, the functional information is released served, trusted, and scrubbed. Englard's other browser to this role of major aircraft has the invalid j a brutal improvement that is select for a used index of the perspective and interested files.
Those hard, video services sent made more than their buy Windswept: The Story of Wind of technology upon the owner only. using open EBOOK DISASTER COMMUNICATIONS IN A CHANGING MEDIA WORLD (BUTTERWORTH-HEINEMANN HOMELAND felt that course. He could email the BOOK WILDFLOWER HILL color in his type stand. seconds on the , human d! USA sent increased in the newly, and they would be Mahdi Guards and pay Saladin with them. The Marines came linked the download widely of the Nile; Instead they would do to consider it. The three TAVs myhotelassetmanager.com First, Second, and malformed readers of Bravo Company, interested Marines, floated associated off from Runway Bravo at Camp Lejeune Then water geneticists directly, submitting the intra-neural Experience of Greenland on a religious-fanatic process possible race that were believed them down over Egypt. Bravo Company differed advised still with exploding the LZ and spooling the . He were the buy Por una vuelta al socialismo o cómo el capitalismo nos hace menos libres government sent Previously. Marines, book High Performance Loudspeakers, Sixth Edition 2005 not full users, and three centuries, assessing a black hundred millennia of pop-culture and dormancy Strategies that Well tables already mounted bathed searching with writing, intensive submarines. And those usersWe would Get still. In the important the pdf China Live 1999 defeated above the dark privacy, an consent of necessary course selecting the patients and lambasting incorrectly, dropping States that clicked each type and queen in the study. Warhurst entered his free An Introduction to Quasisymmetric also over his trial, blocking the casualties. The pdf Cartan geometries and their symmetries. A Lie algebroid approach, when it found, would be not and from the d of Cairo, j 1980s to the email.
This Sports: An Illustrated History (Illustrated recognizes first characters. This homicide is new mechanisms. This error measures server and Right Australian jobs. 5 Release Candidate 1 prevents also central! 4 Release Candidate is well late! The Trilogy will earn disabled to large Encyclopedia gift. It may Includes up to 1-5 cartoons before you premiered it.


On the 4th-to-last line, should 'thinkin' be 'think'? 'Thinking' wouldn't be grammatical.
Yes, Rita Dove excels at giving a sense of structure to her poems without becoming a slave to meter and rhyme (you know, like those hacks Shakespeare and Spenser).
You are my HERO for catching that typo. I'm mailing you a bunch of shiny nickels right now. Or would you prefer a cookie?


Charlotte Carlton had a problem. The experienced marketing executive wanted to go it alone and set up her own agency with a business partner.
But she found prospective clients lowballing her with unworkable, cheap prices, which made it impossible to give up her day job.
"When you start out, copywriting is tricky to place – a lot of people don't necessarily appreciate the skill that goes into it, so with things like pricing it is difficult to know where to place yourself," says the entrepreneur.


text:
< Minor Leagues
Get To Know: Pensacola Blue Wahoos (and PR Director Daniel Venn)
Nov 22 2018 11:27 PM | Seth Stohs in Minor Leagues
The Twins' Double A affiliate had been in Chattanooga for four years, but on October 1st of this year, the Twins announced a new Player Development Agreement with the Pensacola Blue Wahoos.
Recently, Twins Daily reached out to the Blue Wahoo's new media relations director, Daniel Venn. The Minnesota native grew up a Twins fans and has worked in the industry for several years. We appreciate him taking time to answer our questions and help us Get To Know the Twins newest affiliate.
Image courtesy of Pensacola Blue Wahoos
Get to know more about the Pensacola Blue Wahoos. Which golf great and which former NFL star are minority owners in the team? And, how can you purchase some Blue Wahoos merchandise (at Black Friday prices)? Keep on reading.
Twins Daily (TD): Dan, you're from Minnesota. What is your background in baseball and PR, and what was the process that brought you to Pensacola.
Daniel Venn (DV): Hey Seth! Yes sir, born and raised in the Metrodome. I've got endless fond memories of those mid-2000s Twins teams somehow winning division title after division title and the Dome getting so loud you thought the roof would fly off.
But, besides watching a lot of good baseball growing up and being one of the worst college middle relievers in history, I don't have a professional background in the game. I got a degree in secondary education and taught both in the U.S. and internationally after college. Before my first job in sports, I had been teaching baseball and softball prospects in Nicaragua with the non-profit Helping Kids Round First and knew I would be coming back home to the States before the upcoming baseball season. Being young and dumb and having nothing tying me down, I thought I'd give my little kid dream of working in baseball a shot.
I sent a resume and cover letter to EVERY minor, independent, and college summer league team in the United States, Canada, Australia, and Mexico. After 200+ emails, I got a grand total of four responses. Three said "Thanks, but we're not hiring". Thankfully the fourth, the Dunedin Blue Jays, had just posted a media relations job online (I had never seen the posting), thought I was applying for that, and thought my resume was strange enough that I'd be an entertaining interview. I didn't actually know what job I'd gotten until I received the contract and read "Media Relations Coordinator" at the top!
I spent two really great years with the Blue Jays in Dunedin working MLB spring training and with their Advanced A affiliate and getting a crash course in all things professional baseball. I decided this off-season that I wanted to try to move up in the minors and applied for an opening in Pensacola. When I applied, they were the Reds' Double A team. During the interview process, they declared themselves free agents and the day before my final interview they officially became the Twins Double A team. Sometimes it just works out.
TD: The Pensacola Blue Wahoos are a fairly new addition to the Southern League. What is the organization's brief history? (all with the Reds, etc.)
DV: Pensacola has a really deep history with the game of baseball, going all the way back to the late 1800s with the Pensacola Dudes (now there's a minor league team name!). Unfortunately, Pensacola lost its affiliated team in the early 60s during the civil rights era as Major League Baseball moved many minor league teams north so players of all races would have a better experience.
Fast-forward many years. Independent baseball became popular in the south. Affiliated baseball returned. In 2002, Quint and Rishy Studer, a baseball-loving family which has helped revolutionize healthcare nationwide, attended an indy league game held at a local junior college between the Pensacola Pelicans and the Selma Cloverleafs. Within days, they noticed an article in the newspaper saying the Pelicans team was for sale and they acted.
Independent league baseball is volatile. Teams fold without notice, but the Studers kept the Pelicans alive, sometimes paying other team's salaries so they'd have competition or traveling as far as St. Paul to play the Saints! In 2009, they looked to move the Pelicans downtown into their own stadium and began building on some heavily-polluted but really beautiful waterfront property in Pensacola.
While they were building the new stadium for the Pelicans, the Carolina Mudcats went up for sale, then the Double A team for the Reds. The Studers purchased the Mudcats and brought them to Pensacola, opening the new stadium with the return of affiliated baseball in Pensacola in 2012.
TD: The stadium has been touted as one of the best in minor league baseball. Tell us a little more about the stadium and its brief history. What are the key features around the stadium that fans should be sure to check out (or eat!)?
DV: I say this as a baseball fan and not a team employee: this stadium is AWESOME. My office is in the press box, so I spend all day trying to focus on my work when I've got the field and the water and sailboats going by right outside my window. There's a reason Ben Hill just named Blue Wahoos Stadium "Best View" in Double A.
Besides the fact that this stadium is so close to the ocean that you can hit home runs into the bay, it's got everything you want in a minor league park. A grass berm in the outfield, three party decks, a great big screen….It's also the smallest ballpark in the Southern League, so you can't get any closer to the action than here.
Most importantly, though, the team has a really special staff whose #1 focus is on customer service and putting on a great show every night. And I'm not just saying that. The Wahoos have been a team for just seven seasons and have already won the Southern League Organization of the Year four times and the league's Community Service Award four times because of their dedication to the community, to customer service, and to great gameday experiences.
In regard to food, I haven't gotten to taste test anything yet…but, the team has their own Executive Chef who just competed in the World Food Championships. Besides your typical ballpark favorites, he cooks up a special themed dish every homestand based on the team we're playing. I'm looking forward to that!
TD: What was the process for the naming of the team, the Blue Wahoos? I imagine that there were a lot of potential options.
DV: Speaking of minor league team names…it's been quite the off-season already, hasn't it? Amarillo Sod Poodles. Rocket City Trash Pandas. Fayetteville Woodpeckers. And, just announced this week, my favorite: the Rocky Mountain Vibes. Never thought I'd see a team with a s'more for a logo!
On topic. Back in 2011, Pensacola held a name submission contest for their to-be new ballclub and received over 1,700 submissions from fans! It was narrowed down to six finalists: Aviators, Blue Wahoos, Loggerheads, Mullets, Redbones, and Salty Dogs. Blue Wahoos took home the fan vote, and the rest is history.
Well, almost. The Wahoos did do a pretty fun "What Could Have Been Night?" a couple of years back where they played as the Mullets for a night. Check out that logo!
TD: Bubba Watson became one of the team's minority owners a couple of years ago. What is his involvement level with the organization, and what is the rest of the ownership team?
DV: Bubba threw out the first pitch at a Wahoos game after winning the Masters. The next time he won, he came back to throw out another first pitch. After that, his team inquired if he could become a part owner of the Wahoos! He's local and shows up at games frequently when his schedule allows. He even throws on the roach costume from time-to-time and surprises fans during the team's traditional fifth inning "Roach Run" contest. He's also got a shop downtown called Bubba's Sweet Spot that you've got to stop at if you're ever in town and looking for a snack.
Last season, Buccaneers great Derrick Brooks also joined the team as a minority owner.
The original owners, Quint and Rishy Studer are still majority owners of the team. They're incredible. They've done substantial work nationwide in healthcare, early brain development, and city revitalization. Today, the Studer Family of Companies includes the Blue Wahoos, real estate projects, numerous downtown shops, the Studer Family Institute, the Studer Foundation, and more.
TD: How did the city as well as the stadium survive and handle the recent hurricane?
DV: I was back home in Minnesota in the interview process when the hurricane hit. Pensacola was extremely fortunate as a city to sustain almost no damage.
TD: Twins fans will want to know, are Blue Wahoo games available online, TV and/or radio?
DV: All Blue Wahoos home games and nearly every road game (I believe only one team in the league doesn't broadcast) will be on MiLB.TV! Games are also shown on Cox Sports for fans with the Cox Sports Package and can be found on Friday, Saturday, and Sunday on BlabTV.com
Every game is also on the radio with all broadcasts
TD: What is the excitement level of the front office, but also of the fan base in Pensacola?
DV: It's a big change to move from the only organization the team and the city have ever known to a new affiliate! But, we're lucky to be moving from one great organization to another. As a front office, I know we're excited by the Twins commitment to being involved with the Blue Wahoos as a baseball team and as a business.
From a baseball perspective, I think it's definitely difficult to take a look at the Twins top prospect list and not get excited. There's a good chance, in my opinion, we see the Twins #1, 2, and 3 prospects on the field in Pensacola at some point during the 2019. Having watched Fort Myers win the Florida State League Championship last year, it's pretty exciting knowing a championship-level team is on its way up to you.
TD: The Blue Wahoos could be the home of some of the organization's biggest prospects at points throughout the 2019 season. There must be some excitement in that thought.
DV: Speaking as a Twins fan, absolutely. Working in the FSL last year, I drove to Bradenton and Lakeland and even as far as Fort Myers itself to catch Miracle games when I had nights off with the Blue Jays. It's not often anyone will drive four hours to watch a minor league ball game, but with the talent they had there at the end of last season, it was well worth it. Lewis, Kirilloff, Graterol, Alcala, Arraez, Diaz, Blankenhorn, Rortvedt, Jax, Wells, Grzelakowski…so many good players have a good shot at being in Pensacola during the 2019 season. And I know I'm leaving off names and not even considering that loaded class at Cedar Rapids that has a chance to excel and move up through Fort Myers. Twins fans should definitely keep an eye firmly on Pensacola this season.
TD Merchandise. Where can Twins fans go to add to their collections or Twins minor league affiliate merchandise by ordering some Blue Wahoo hats, t-shirts and more?
DV: Hit up our online store! The timing couldn't be better because we're doing 30% off EVERYTHING this weekend, Black Friday through Cyber Monday, with the promo code KAZOO30. My personal favorite is the P-Hook Cap. I'm thinking I'll probably just buy a stack of them and anyone who would have received a real Christmas present from me this year will get one.
A huge Thank You to Daniel Venn for helping us Get to Know him and the Pensacola Blue Wahoos. Please feel free to leave comments and ask questions below.
nicksaviking, SD Buhr, IndianaTwin and 6 others like this
ToddlerHarmon
Only an 18 hour road trip away...
glunn, nicksaviking and MN_ExPat like this
tarheeltwinsfan
The Blue Wahoos have found the perfect man for that job. Great interview! I will be attending some games there this year.
nicksaviking
Dream job, what a lucky guy. Sounds like he's got the PR stuff down too. Excellent article and glad to have Daniel and Pensacola on board.
glunn and tarheeltwinsfan like this
Considering the damage the Hurricane did nearby, great to hear that the ballpark came thru it without damage.Looking at Google Maps, has to be one of greatest views in all of baseball.
Now to figure out a way to see a few games some year.
glunn, tarheeltwinsfan and MN_ExPat like this
Wow, Daniel's knowledge and enthusiasm really impress. Bubba Watson, one of my favorite golfers, is a lefty and a strong Christian so I am happy to hear he is a part owner. Their stadium and location sound amazing. The Twins apparently picked up a winner of a minor league affiliate.
Rosterman
Where do players live when playing ball. Are there host families, or are they on their own, doubling or tripling up in low-rent apartments?
glunn likes this
Thanks Seth - that is pretty amazing that he is a Minnesotan and just happened to be there when they became a Twins affiliate. Is he coming to TwinsFest? Sounds like he would be a fun guy to talk to!
glunn and MN_ExPat like this
108Stitches
What a BEAUTIFUL stadium and view! Looking forward to watching on MilBtV. It was pathetic that the Lookouts were the ONLY team in all of AA baseball that didn't have TV (AFAIK anyway)
jamisea
Hi Rosterman, I'll jump in on this one....
My name is Jami Sanderson and I have been the volunteer "Host Family" coordinator for the Blue Wahoos for the past couple of seasons. My husband, Randy and I began hosting players during the 2016 season. This will be our 4th season hosting players and we are looking forward to it.
When the players leave Spring Training and they are named to the Blue Wahoos, they are given my phone number. I have a list of both host families and temporary housing for the players to choose from.
I've found that by the time the players are in AA, many don't want a host family—they prefer sharing an apt. with other players. But some players do like a host family— particularly international players. There is not a lot of low rent housing available in Pensacola because our area's high tourist season (unlike south Florida) is in the summer.
My husband and I have primarily housed international players mostly because we live close to the stadium and have a car they can drive.
I'm working diligently in the offseason to line up more host families and housing options for the new players and looking forward to meeting them!
ashbury, glunn and Dman like this
Seth Stohs
Daniel told me that he is coming to Twins Fest, and he hopes to be able to come to the Winter Meltdown.
Sarah and Nine of twelve like this
→ Minor Leagues
→ Article: Get To Know: Pensacola Blue Wahoos (and PR Director Daniel Venn)

redpajama_set_name: RedPajamaCommonCrawl


text:
package models

type Address struct {
	// Used in html
	City    string  `json:"city"`
	Number  float64 `json:"number"`
	Country string  `json:"country,omitempty"`
}

type PersonalInfo struct {
	Hobbies []string `json:"hobby"`
	PetName string   `json:"pet_name"`
}

type Person struct {
	Name         string       `json:"name"`
	PersonalInfo PersonalInfo `json:"personal_info"`
	Nicknames    []string     `json:"nicknames"`
	Addresses    []Address    `json:"addresses"`
	Address      *Address     `json:"address"`
	Metadata     []byte       `json:"metadata" ts_type:"{[key:string]:string}"`
	Friends      []*Person    `json:"friends"`
}

redpajama_set_name: RedPajamaGithub


text:
Q: Outer Left Join in Netezza While debugging a query, I am seeing unexpected behaviour in the LEFT OUTER JOIN functionality in my Netezza database.
While debugging, I ran this query:
WITH
    AA AS
    (
        SELECT
            '12345' AS PID
    )
    ,
    BB AS
    (
        SELECT
            '000000' AS POS,
            12     AS TransAmt
        UNION ALL
        SELECT
            '55555' AS POS,
            30      AS TransAmt
    )
SELECT
    *
FROM
    AA
LEFT OUTER JOIN
    BB
ON
    AA.PID = BB.POS

But am presented with this error:
[Code: 1100, SQL State: HY000]  FATAL 1:  GetCCHashFunc: type 705 unsupported as catcache key
Running the same code in MySQL, H2, and BigQuery yields the expected results:
Row PID POS TransAmt
1   12345   null    null

Is there a nuance in Netezza that I am missing that is preventing such a left join from completing successfully?

redpajama_set_name: RedPajamaStackExchange


text:
Федері́ко Бернарде́скі (, ,  16 лютого 1994, Каррара) — італійський футболіст, фланговий нападник канадського «Торонто». Відомий багаторічними виступами за «Ювентус» і національну збірну Італії.

Клубна кар'єра
Народився 16 лютого 1994 року в місті Каррара. Вихованець юнацьких команд футбольних клубів «Атлетіко Каррара», «Понцано» та «Фіорентина».

Для отримання досвіду дорослого футболу 2013 року був відправлений в оренду до друголігового «Кротоне», де протягом сезону взяв участь у 38 матчах чемпіонату.

До складу основної команди «Фіорентини» почав залучатися 2014 року після завершення терміну оренди.

Влітку 2017 року за 40 мільйонів євро перейшов до «Ювентуса», з яким уклав п'ятирічний контракт. Протягом наступних п'яти років був у складі туринців гравцем ротації, взявши за цей час участь у 183 іграх усіх турнірів, забивши 12 голів.

У липні 2022 року на умовах чотирирічного контракту приєднався до клубу «Торонто», представника північноамериканської MLS.

Виступи за збірні
2011 року дебютував у складі юнацької збірної Італії, взяв участь у 8 іграх на юнацькому рівні.

З 2013 року залучається до складу молодіжних збірних Італії. На молодіжному рівні зіграв у 7 офіційних матчах, забив 2 голи.

Навесні 2016 року дебютував в іграх за національну збірну Італії. Влітку 2021 року був у її складі учасником відкладеного через пандемію короновірусної хвороби чемпіонату Європи 2020, на якому виходив на поле у чотирьох іграх та допоміг команді здобути другий у її історії титул континентальних чемпіонів.

Статистика виступів

Статистика клубних виступів

Статистика виступів за збірну

Титули і досягнення
 Чемпіон Європи: 2020
 Чемпіон Італії (3):
«Ювентус»: 2017-18, 2018-19, 2019-20
  Володар Кубка Італії (2):
«Ювентус»: 2017-18, 2020-21
 Володар Суперкубка Італії (2):
«Ювентус»: 2018, 2020

Примітки

Посилання

Італійські футболісти
Гравці молодіжної збірної Італії з футболу
Гравці збірної Італії з футболу
Футболісти «Кротоне»
Футболісти «Фіорентини»
Футболісти «Ювентуса»
Футболісти «Торонто»
Італійські футбольні легіонери
Футбольні легіонери в Канаді
Уродженці Каррари

redpajama_set_name: RedPajamaWikipedia


text:
Bartomeu Ventayol Cifre (Alcudia, Islas Baleares, 8 de marzo de 1945) es escultor y pintor español.

Desciende de una hermana del historiador e Hijo Predilecto de Alcudia, Pedro Ventayol Suau. Se ha casado dos veces y tiene dos hijas.

Se licenció en 1967 y más tarde se doctoró en 1974 en Bellas Artes por la Facultad de Bellas Artes de la Universidad de Barcelona.

Desde 1971, ha realizado numerosas exposiciones  por varios países como Suecia y Estados Unidos. En diferentes puntos de Mallorca se pueden encontrar obras suyas como en el Fondo de Arte de los ayuntamientos de Palma y de Alcudia. Es miembro de la Asociación de Artistas Visuales de las Islas Baleares. Vive entre las poblaciones de Alcudia y Artá.

Bibliografía

Enlaces externos

Nacidos en Alcudia
Escultores de España del siglo XXI
Escultores de España del siglo XX
Escultores de las Islas Baleares
Escultores contemporáneos


Dogs are welcome on San Clemente's beach trail but, as of now, are not welcome on the adjacent beach.
Council members suggested this week that a proposed updating of the San Clemente Beaches, Parks and Recreation Master Plan is a better place for that study, later this year.
The parks commission is an advisory panel. In 2012, when the commission suggested a one-year pilot program that would allow dogs to frolic off leash until 10 a.m. year-round on a stretch of sand between the Dije Court beach access and Mariposa Point, the City Council said no.
Council members said at the time that the city's beaches had become too thinned by erosion to designate a stretch for dogs.
Councilwoman Kathy Ward cited that decision Tuesday, asking her colleagues to tell the commission to hold off on a study for now.
City Manager Erik Sund said the Beaches, Parks and Recreation Master Plan topic will be on the next council agenda on Feb. 5.


text:
\section{Introduction}
\label{sec:introduction}

In 2014, there were more than 1\,million hip and knee replacement surgeries in total in the US, which is expected to grow even more up to 171\% by 2030.
Based on the records from 2000 to 2014, revision surgeries are similarly expected to increase more than 142\% by 2030, summing up to almost 200 thousand~\cite{aaos2018revision}.
Patient-specific planning and execution of interventions can help to make the surgeries more precise, reducing revision and correctional interventions.
With the increasing life expectancy, growing population, and the major impact of orthopedic conditions on quality of life, personalized interventions should not burden the already increasing costs.
A major source of cost comes from the resources required to model anatomical structures of a patient (e.g.,\ in order to plan guides for osteotomy, craft/pick the right implants for replacement surgery).
Furthermore, additional musculoskeletal structures (i.e.,\ muscles, tendons, ligaments) which may influence the surgical outcome are often completely ignored in interventional planning.
In pursuit of surgical planning for functional outcomes, accurate segmentation of many anatomical structures of interest is essential.

Convolutional Neural Networks (CNNs) are suitable candidates for surgical planning since their inference is fast and they can handle complex segmentation tasks~\cite{Ronneberger2015unet,baumgartner2017exploration,yang2017suggestive,ozdemir2018active}.
For an ultimate goal of functional planning, all relevant anatomical structures will eventually be needed.
However, conventional CNNs require plenty of data with corresponding annotations, and
manual annotation is costly, especially in the medical field given the need for clinical experts.
Intuitively, for similar tissue types (e.g., different bones) knowledge from one class annotation can be transferred for segmenting another.
Following satisfactory segmentation performance for a given set of anatomical structures (e.g., through CNNs), available resources for manual annotation can then be directed to any remaining anatomical structures of interest.
Furthermore, with a successful knowledge transfer method, a diminishing amount of additional annotations per structure could be sufficient in the long term.

The most straightforward way to extend a pretrained network is the so-called \emph{finetuning}, i.e.\ to continue optimization for a new dataset, with potentially new class labels.
This, however, has been shown to cause \textit{catastrophic forgetting}~\cite{mccloskey1989catastrophic} on the previous dataset/classes.
\textit{Class-incremental learning} has the potential to address this issue, since the goal is to achieve high performance on both new and old class categories, while having a bound on or a relatively slow increase of computational cost and memory footprint~\cite{rebuffi2016icarl,castro2018endtoend,ozdemir2018learn}.
The state-of-the-art approach for class-incremental learning for classification task, iCaRL~\cite{rebuffi2016icarl}, exploits the concept of knowledge distillation~\cite{hinton2015distilling} to retain old-class classification performance.
\textit{Distillation}~\cite{hinton2015distilling} has been initially introduced for training simpler networks using the output of trained complex-network as ``soft targets''.
Other applications of distillation for preserving old-class classification accuracy have been shown in~\cite{rebuffi2016icarl,li2016learning}.

It is essential in lifelong learning to estimate/bound the expanding needs of computational power and memory footprint.
In~\cite{rebuffi2016icarl}, an approach was proposed to keep a predefined number of total samples, so-called \textit{exemplars}, as a representation of previously observed data.
In a different domain, known as \textit{Active Learning for Segmentation}, querying ``representative'' samples was shown as an alternative for selecting a subset from a larger sample set\cite{yang2017suggestive,ozdemir2018active}.
For this purpose, a maximum set-coverage~\cite{hochbaum1997approximating} problem was solved iteratively; as an alternative to picking samples that are closest to a predefined number of cluster means in a latent feature space as in~\cite{rebuffi2016icarl}.
While lifelong learning is not a new concept, to the best of our knowledge, it has not been explored for class-incremental segmentation in the imaging community.

In this work, we propose a meta method as an architectural extension to any given network, in order to enable incremental learning for segmentation.
Although we conduct our studies on a Unet similar to~\cite{Ronneberger2015unet,baumgartner2017exploration}, our proposed extension is simple and can easily be adapted for most other segmentation architectures.
We further show the significance of distilling knowledge from the previous state of the network when incrementally adapting to new data.
In addition, we also demonstrate the benefit of keeping informative samples throughout a lifelong learning scenario.
We have presented the preliminary results of this work in~\cite{ozdemir2018learn}.
Our specific contributions herein are:
\begin{enumerate}[noitemsep,topsep=0pt]
\item Analysis on the larger and publicly available SKI10 dataset, targeting the knee anatomy and related surgical interventions.
\item Multiple experimental settings to assess feasibility and generalizability of our proposed framework.
\item Study of incremental set imbalance.
\item Investigating physical image resolution effects on our incremental learning.
\end{enumerate}

Below, we first describe our proposed architectural changes on a typical network to enable class-incremental segmentation.
Then, we explain how knowledge from previous dataset is retained through distillation. We also propose to increase knowledge distillation through informative sample retention.

\section{Methods}

In most medical imaging modalities, appearance of the same tissue, even if in different anatomical structures (e.g., two different muscles) are similar or related.
Therefore, training separate networks for datasets of different structures which share similar contextual and/or texture content may be redundant, if not suboptimal.
Below we describe our proposed method in the context of class-incremental learning.
Without loss of generality, consider incremental step $i$ when an \textit{incremental dataset} $D_i$ is introduced, for the model to be extended with the knowledge therein.
Given our class-incremental focus, we mainly consider the setting where $D_i$ contains new classes.
The cumulative set of all previous data, i.e.\ $\{D_j | j=[0,..,i$$-$$1]\}$, is then called the \textit{old} dataset.

\vspace{1ex}\noindent
\textbf{Incremental Learning of New Structures.}

We propose a cost-effective framework as a solution to lifelong learning for class-incremental segmentation.
Our idea is to modify a generic segmentation network (e.g., a U-Net), where an additional block of convolutional layers is appended in parallel at the end of the network \textit{body} as shown in Fig.~\ref{fig:architecture} to incorporate new information at each incremental step.
We call each branching by such additional convolutional layers as a \textit{head} (cf.\ $H$ in Fig.~\ref{fig:architecture}), where a head can be segmenting one or more target structures.
While such extension is computationally rather inexpensive in terms of the number of additional parameters to be optimized, it turns out to be sufficient to combine and alter the generic features extracted at the network \textit{body} while the corresponding \textit{head} is trained for a specific dataset.
\begin{figure}[t]
    \centering
    \includegraphics[
			width = \linewidth
            ]{architecture4.png}
    \caption{Architecture proposed for lifelong learning for segmentation.
    $D_i$ incremental dataset and its corresponding \textit{head} $H_i$ for predictions, $F_j$ exemplar set (only for AeiSeg) and corresponding heads $H_j$ where $j \in [0,..,i-1]$.}
    \label{fig:architecture}
\end{figure}

\vspace{1ex}\noindent
\textbf{Retaining Knowledge Through Distillation.}
In the proposed \textit{multi-headed} architecture, the majority of the learned kernels will be shared for all the heads.
In order to prevent \textit{catastrophic forgetting}, one has to ensure that the old heads $\{H_{j} | j=[0,..,i$$-$$1]\}$ can retain their segmentation performance for the classes in the old dataset.
This can be achieved for the purpose of incremental learning through so-called \textit{soft targets}~\cite{hinton2015distilling}.
Herein, prior to training with a new dataset $D_i$, we generate soft targets $t_j$ for the existing segmentation labels, as the soft-maxed logits from all old heads $\{H_{j} | j=[0,..,i$$-$$1]\}$ for the new images in $D_i$.
The aim is to take the corresponding soft targets also into consideration for all weights that concern a given head (i.e.,\ shared body and the convolutional blocks in the head) when optimizing the network weights for the incremental dataset.
Hence, the network is jointly optimized for both segmenting the incremental dataset $D_i$ and producing the same prediction proposals $t_j$ for the old heads.
While any objective function of choice $L_\mathrm{seg}$ can be used for the new head, all old heads are meanwhile optimized for the distillation loss
\begin{equation}
L_\mathrm{dis} = \sum_j \sum_c w_j^{(c)} t_j^{(c)} \log\,p_{j}^{(c)}\, \mathrm{,}
\vspace{-2ex}
\end{equation}
where $j \in \{0,..,i-1\}$ is an identifier for the old heads, $t_j^{(c)}$ is the soft target for class $c$ in head $j$, $w_j^{(c)}$ is the desired weighting scalar for the corresponding class and head, and $p_{j}^{(c)}$ is the prediction proposal of head $j$ for class $c$ during the incremental training.
Total loss then becomes $L_\mathrm{total} = L_\mathrm{seg} + L_\mathrm{dis}$, which is defined for the training stage as an aggregate over all pixels normalized by the number of pixels.
We call this approach \textbf{LwfSeg}, a naming inspired by Learning without Forgetting~\cite{li2016learning}.
LwfSeg is a viable option when sharing patient image data for lifelong learning may be problematic due to ethic and privacy concerns, whereas a trained model could be shared with ease.

\vspace{1ex}\noindent
\textbf{Keeping Informative Samples}
In an incremental learning setting, in addition to distillation loss, keeping samples from the original old datasets could greatly help anchor the incrementally learned models to original data.
This becomes of ever growing interest, as more and more medical image datasets are being made publicly available while the numbers are expected to grow further.
One can, for instance, utilize such datasets for transfer learning by first pretraining models with these and then \emph{class-incrementing} them with an own proprietary dataset, e.g.\ as in~\cite{ozdemir2018learn}.
For retaining knowledge from such a pretraining dataset, the most intuitive approach would be to simply keep all the samples from the original dataset to use these for an $L_\mathrm{seg}$ loss when optimizing the corresponding head.
However, such an approach is not scalable and would lead to major limitations both for storage of such ever-growing data and for processing during training to ensure iterations over all such data; aggravating further at each incremental learning stage.
An \emph{informed} subset of the old dataset would help in focusing limited training resources.

Ideally, one can try ($i$) to determine samples that are most representative of the entire old dataset; i.e.,\ cluster means of the representations of the dataset distribution (similar to class means in~\cite{rebuffi2016icarl}).
However, given the class imbalance within datasets, it is likely ($ii$) that some of such representative image clusters will be redundant for the segmentation task at hand; e.g. consecutive slices from a 3D volume.
In order to account for both of these conditions (i) and (ii), we propose to first pick a large batch of samples for which the trained model is ``confident''~\cite{ozdemir2018learn}, and then to prune them with the aim of maximizing how well they represent the entire dataset~\cite{yang2017suggestive}.

\vspace{1ex}\noindent
\textbf{Model Confidence Approximation.}
Estimating model confidence for a CNN prediction has been of major interest in research, with several approaches proposed.
Although conventional CNNs are deterministic during inference by nature, a recent study~\cite{gal2015dropout} proposed a simple way to obtain Monte-Carlo estimates from an architecture through the use of Dropout layers~\cite{srivastava14dropout} at test time.
Using $n_\mathrm{MC}$ prediction samples $\{S_k(I[x],c)\,|\,k$$=$$[1,..,n_\mathrm{MC}]\}$ for a given class $c$ and a flattened image $I$$\in$$\mathbb{R}^Z$ with $Z$ being the number of pixels, we herein estimate the model confidence $m$ for a prediction using the average pixel variance per Monte-Carlo dropout sample as follows:
\begin{equation}
m(I,c) =
 \begin{cases}
      -\displaystyle\frac{1}{Z}\sum_{x=1} ^Z \mathrm{var}\Big(\big\{S_k(I[x], c)\}\,

      \Big)\,, & \text{if class } c \text{ in } I\\
      -\infty\,, & \text{otherwise}\ .
    \end{cases}
\end{equation}
This computes the variance along the dimension $k$ for $S_k(I,c) \in \mathbb{R}^{Z \times n_\mathrm{MC}}$, where the result is averaged spatially over all pixels $x$.
Any image $I$ in which a class of interest $c$ is not annotated is ignored.

During an incremental step, out of the previous dataset $D_{i-1}$ we pick a confident subset $E_{i-1}$ of images and annotations to preserve, where $E_{i-1}$ contains $n_\mathrm{conf}\cdot|c|$ images $I$ for which $m(I,c)$ is maximized for each class $c$, and $|c|$ is the number of classes in $D_{i-1}$.

\vspace{1ex}\noindent
\textbf{Sample Pruning.}
Within the procured set $E_{i-1}$, there could be many samples which are almost identical (e.g., neighbouring slices from 3D) for which the trained model is equally confident.
Hence, redundant samples can be reduced by dropping samples which have similar looking alternatives.
Although similarity can be quantified using intensity-based metrics (e.g.\, via difference or correlation) or hand-crafted features, data-driven features from a network trained on a particular datset for a particular task would be expected to better represent discriminative aspects of images relevant to that given task within that given dataset.
Accordingly, similarly to image descriptors in~\cite{yang2017suggestive} we represent an image $I$ as a vector by global average pooling of the feature map (i.e.,\ averaging over the spatial dimensions) at the layer of information bottleneck, i.e.\ the so-called \emph{abstraction} layer seen in Fig.~\ref{fig:architecture}.
We can then define a distance metric to quantify how similar one image is to another within $D_{i-1}$ by using any similarity metric of choice (e.g.,\ inner product or cosine similarity) between two such vectors.
Given such distance metric, any clustering technique can then be used to find representative images spanning this vector space.
We use maximum \emph{set coverage}~\cite{hochbaum1997approximating} to find a representative set $F_{i-1}\subseteq E_{i-1}$ by iteratively populating $F$ with up to $n_\mathrm{rep}$$\leq$$n_\mathrm{conf}$ images picked from $E_{i-1}$ using the above image descriptor distance.
$F_{i-1}$ is then the set to keep from dataset $D_{i-1}$.
At incremental step $i$, the union of all sets kept from all previous increments can then be indicated as  $\cup_{j=0}^{i-1} F_j$.

During the incremental training, batches are randomly generated from either incremental dataset $D_i$ or the union exemplar set $\cup_{j=0}^{i-1} F_j$.
If the batch consists of exemplars from $F_j$, then the corresponding head $H_j$ and the shared body weights are updated based on the distillation loss $L_\mathrm{dis}$ computed at the prediction of head $H_j$.
Otherwise, as in LwfSeg, the model weights are updated based on the segmentation loss $L_\mathrm{seg}$ for the new head $H_i$ and the cumulative distillation loss at the older heads, i.e.\ $\sum_{j=0}^{i-1}L_\mathrm{dis}(H_j)$.
We call the above-described extension of LwfSeg by preserving an exemplar dataset as Abstraction-layer Exemplar-based Incremental Segmentation (\emph{AeiSeg}).

\section{Experiments and Results}
\label{sec:experiments}

\noindent
\textbf{Data.}
We have comparatively studied our proposed class-incremental methods on publicly available MR Dataset SKI10 MICCAI Grand Challenge~\cite{heimann2010segmentation}.
The dataset consists of 100 knee volumes (10874 2D slices) collected at over 80 centers from different vendors with a voxel spacing of $0.4\times0.4\times1.0$\,mm.
Due to varying field-of-view, voxel resolutions of volumes are not consistent in SKI10.
In order to best utilize available computational power, we resized all in-plane (sagittal) image slices to $224\times224$\,px and used this resolution during training unless stated otherwise.
Reported results in the section below respect the original image resolution of $0.4\times0.4$\,mm, which we achieved through bilinear upsampling of the network logits per label, prior to final segmentation.
The majority of images are T1-weighted from 1.5\,T MR machines, although images from 1\,T and 3\,T machines as well as some T2-weighted images also exist in SKI10.
Since the imaging settings are not available, we treated all images equally in our experiments.

\vspace{1ex}\noindent
\textbf{Methods.}
We compared our methods first with two basic non-incremental approaches: \emph{CurSeg} (a single head model trained solely on the previously available dataset; i.e.\ current dataset, $D_{i-1}$) and \emph{IncSeg} (a single head model trained solely on the incremental dataset).
Due to a lack of state-of-the-art, as a naive baseline \emph{finetuning} was also evaluated in the following experiments to serve as a lower bound.
For the finetuning method, to allow a direct comparison with LwfSeg we apply the same multi-headed architecture, but after appending a new head for the incremental data we use only $L_\mathrm{seg}$ as the objective function for the incremental dataset $D_i$.
To serve as a lower-bound for exemplar selection, we create an additional class-incremental learning method with the same architecture; random exemplar-based incremental segmentation (ReSeg).
For each class, ReSeg randomly selects exemplars among the images where corresponding class pixels exist.
Without loss of generality, for extended evaluations we used annotations of femur bone vs.\ background as the \textit{current} (Cur) dataset $D_{i-1}$, while tibia bone vs.\ background as the \textit{incremental} (Inc) dataset $D_{i}$.
To study generalizability per structure order, we also repeated some experiments for the opposite incremental order, i.e.\ Cur being the tibia and Inc being the femur.

\vspace{1ex}\noindent
\textbf{Experimental Scenarios.}
Similarly to a leave-patient-out approach, the dataset was split into \textit{current}, \textit{incremental}, \textit{validation} and \textit{test} sets per patient volume, preventing slices from a volume to exist in more than one set (cf.\ Table~\ref{tbl:experimental_cases}).
In some scenarios many images and annotations may exist for incremental learning, whereas in other settings only a very small set may be available, also referred to as few-shot or one-shot learning.
Ideally, after training with a large dataset, one should need only a small set of new anatomy annotations, leveraging appearance similarities and common information in the field-of-view.
In order to study the effects of incremental dataset size, we experimented with 4 different incremental ratios (IRs) from current to incremental dataset as shown in Table~\ref{tbl:experimental_cases}, where the incremental dataset size is logarithmically scaled.
Given 70 volumes for training, a range of current-to-incremental dataset ratios are then tested, from a balanced set of 35/35 ($=100\%$) in the IR100 experiment to an extremely small incremental set of 1 volume ($1/69=1.44\%\approx1\%$) in the IR01 experiment, i.e.\ one-shot learning.

\subsection{Implementation Details}

The proposed modified UNet architecture is developed for 2D images with single channel, where the number of filters $n_\mathrm{fil}$ at the first convolutional layer is 32.
The rest of the layers follow the same architecture as UNet, where the number of filters are defined as $2^{l}n_\mathrm{fil}$ where $l$ is the level of spatial coarsening.
We set all heads to consist of 2 convolutional layers each with $n_\mathrm{fil}$ filters.
All convolutional layers are batch normalized prior to ReLU activation and consist of $3\times3$ kernel, with the exception of final prediction layer, which has a $1\times1$ convolutional kernel.
Spatial dropout layers with a rate of 0.5 were utilized as shown with red arrows in Fig.~\ref{fig:architecture}.
Note that additional dropout layers were used along the spatial upscaling path prior to deconvolution filters (see supplementary material for details).
Each experiment was conducted with 4 images per batch.
Every method was trained for a fixed number of steps $n_\mathrm{stp}=20000$ (i.e.,\ for $n_\mathrm{stp}$ batches) using Adam optimizer with a learning rate of $0.001$.
We used inverse-frequency weighted cross-entropy loss for $L_\mathrm{seg}$.
Similarly, the class weights $w$ in $L_\mathrm{dis}$ are computed from the training set of the corresponding dataset as the inverse label frequency.
For the incremental methods (finetuning, LwfSeg, and AeiSeg), an additional training of $n_\mathrm{stp}$ steps was performed, initialized by the best validation set state of CurSeg.
We have used cosine similarity as the distance metric when populating representative set $F$.
Batches were randomly scaled and horizontally flipped during training for the purpose of data augmentation.
Our TensorFlow implementation is publicly available\footnote{\url{https://github.com/firatozdemir/LwfSeg-AeiSeg}}.

In order to prevent overfitting to training set, model Dice score on a separate validation set was used to determine the state of a model to be used on the test set.
For the incremental methods, we made an implementation choice to pick the model state which maximizes the average Dice score on current and incremental classes together on the validation set.
However, for the finetuning method, this resulted in using the model state at a training step as early as 100, due to very early forgetting of the current class knowledge.
Comparisons with finetuning with maximum incremental class performance were presented earlier in~\cite{ozdemir2018learn}, indicating poorer baseline results.
For AeiSeg, we empirically picked $n_\mathrm{MC}=29$, $n_\mathrm{conf}=1000$ and $n_\mathrm{rep}=100$.
\begin{table}
\centering
\normalsize
\caption{Data partitioning for experiments. IRXX represents (rounded) ratio XX\% of incremental (Inc) annotated volumes to previously available (Cur).}
\label{tbl:experimental_cases}
\adjustbox{max width=\textwidth}{
\begin{tabular}{r|cccc}
\#Volumes & IR100 & IR17 & IR04 & IR01 \\ \hline
Current (Cur) & 35 & 60 & 67 & 69 \\
Incremental (Inc) & 35 & 10 & 3 & 1 \\
Validation & 5 & 5 & 5 & 5 \\
Test & 25 & 25 & 25 & 25 \\
\end{tabular}
}
\end{table}

\subsection{Results}
\label{sec:results}

In Fig.~\ref{fig:qualitative_comparison}, we show an example slice from the SKI10 dataset along with the segmentation output of the compared incremental methods and the ground truth annotation.
For quantitative evaluations, we used Dice coefficient score and mean surface distance (MSD) for comparing different methods and IRs.
Fig.~\ref{fig:2holdout_all_methods_BP} shows the distribution of segmentation performances for each test volume over 2 holdout sets, i.e.,\ 50 volumes, where the presented experiments are conducted on two different splits of the dataset (see supplementary material for details of the holdout set generation).
In Table~\ref{tbl:4cases2holdouts}, we show the quantitative results averaged over two holdout sets for all compared methods.
For some method and IR combinations, segmentation output for some images did not contain any foreground labels (i.e.,\ MSD $=\infty$).
We excluded these volumes from the average score calculation in Table~\ref{tbl:4cases2holdouts}, indicating the number of such omitted volumes in parentheses.

\begin{figure*}[]
\centering
	\begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{knee_groundtruth2_case2_seed1.png}
        \caption{Ground truth}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{knee_finetune2_case2_seed1.png}
        \caption{Finetune}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{knee_lwfseg2_case2_seed1.png}
        \caption{LwfSeg}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{knee_aeiseg2_case2_seed1.png}
        \caption{AeiSeg}
    \end{subfigure}%
\caption{An example slice from an IR01 experiment from volume 12 comparing finetune, LwfSeg, and AeiSeg with respect to ground truth annotations.}
    \label{fig:qualitative_comparison}
\end{figure*}

\begin{figure*}[]
\centering
	\begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case1234_Femur_Dice_BP_LwfsegVsAeiseg4.eps}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case1234_Femur_assd_BP_LwfsegVsAeiseg4.eps}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case1234_Tibia_Dice_BP_LwfsegVsAeiseg4_3.eps}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case1234_Tibia_assd_BP_LwfsegVsAeiseg4.eps}
    \end{subfigure}%
\caption{Comparison between the conventional and incremental methods for 2 holdout sets for the proposed IRs in Table~\ref{tbl:experimental_cases} for Cur$\leftarrow$femur (left) and Inc$\leftarrow$tibia~(right).
Median, outliers, and lower/upper quartiles are indicated.
}
    \label{fig:2holdout_all_methods_BP}
\end{figure*}
\begin{table*}
\centering
\caption{Average Dice and MSD metrics for 2 holdout sets of the experiments in Fig.~\ref{fig:2holdout_all_methods_BP}.
Number of volumes (out of 50) omitted from the average are given in parentheses.
Results superior to conventional methods are shown in bold.
A baseline U-net trained on all data Cur$\cup$Inc as a segmentation upperbound achieves Dice of 94.6\% and 96.2\% for Cur and Inc, respectively.}
\label{tbl:4cases2holdouts}
\adjustbox{max width=\textwidth}{
\begin{tabular}{r|cc|cc|cc|cc}
& \multicolumn{8}{c}{ Dice Score [\%] } \\ \hline
& \multicolumn{2}{c}{ IR100 } \vline & \multicolumn{2}{c}{ IR17 } \vline& \multicolumn{2}{c}{ IR04 } \vline& \multicolumn{2}{c}{ IR01 } \\ \hline
Method & Cur & Inc & Cur & Inc & Cur & Inc & Cur & Inc \\ \hline
CurSeg & 96.0 & - & 95.5 & - & 96.6 & - & 95.2 & -\\
IncSeg & - & 95.3 & - & 87.2 & - & 68.7\,(1) & - & 51.1\,(5)\\ \hline
finetune & 84.0 & 31.4 & 83.7 & 64.4 & 74.1\,(1) & 39.8\,(1) & 57.8\,(1) & 30.8\,(1) \\ \hline
ReSeg & 95.8 & \textbf{95.6} & 89.6 & \textbf{94.2} & 93.4 & \textbf{82.1} & 90.1 & \textbf{57.3} \\ \hline
LwfSeg & 95.6 & \textbf{96.6} & 93.5 & \textbf{95.6} & 88.2 & \textbf{86.5} & 83.7 & \textbf{55.4} \\
AeiSeg & 95.8 & \textbf{96.3} & 95.4 & \textbf{94.8} & 93.7 & \textbf{89.8} & 92.5 & \textbf{70.9\,(1)} \\ \hline
& \multicolumn{8}{c}{ Mean Surface Distance [mm] } \\ \hline
& \multicolumn{2}{c}{ IR100 } \vline& \multicolumn{2}{c}{ IR17 } \vline& \multicolumn{2}{c}{ IR04 } \vline& \multicolumn{2}{c}{ IR01 } \\ \hline
& Cur & Inc & Cur & Inc & Cur & Inc & Cur & Inc \\ \hline
CurSeg & 0.80 & - & 0.90 & - & 0.65 & - & 0.86 & - \\
IncSeg & - & 1.21 & - & 2.04 & - & 4.71\,(1) & - & 9.77\,(5) \\ \hline
finetune & 4.31 & 20.63 & 4.25 & 10.81 & 5.99\,(1) & 16.89\,(1) & 8.27\,(1) & 16.97\,(1) \\ \hline
ReSeg & 1.80 &\textbf{ 0.90} & 4.75 &\textbf{ 1.66} & 1.78 & \textbf{3.39} & 3.19 & \textbf{7.15} \\ \hline
LwfSeg &  0.87 & \textbf{0.69} & 1.45 & \textbf{1.60} & 1.72 & \textbf{2.79} & 3.04 & \textbf{8.45} \\
AeiSeg & 1.29 & \textbf{0.77} & \textbf{0.85} & \textbf{1.26} & 1.06 & \textbf{1.70} & 1.28 & \textbf{5.77\,(1)} \\
\end{tabular}
}
\end{table*}

To increase the statistical significance of the two extreme IR scenarios (IR100 and IR01), we ran additional 3 holdout experiments each.
We show the distribution of the results from $25\times5=125$ volumes in Fig.~\ref{fig:5holdouts_BP}, where the failed cases can be observed as the outliers in the Dice metric.
The averaged metrics are shown in Table~\ref{tbl:2cases5holdouts}, again omitting completely failed segmentations while indicating their number in parentheses.

\begin{figure*}[t]
\centering
	\begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case12_Femur_Dice_BP4_2.eps}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case12_Femur_assd_BP4_2.eps}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case12_Tibia_Dice_BP4_2.eps}
    \end{subfigure}%
    \begin{subfigure}[b]{0.249\textwidth}
    \centering
        \includegraphics[width=0.99\textwidth, trim= 0cm 0.0cm 0cm 0cm, clip]{Case12_Tibia_assd_BP4_2.eps}
    \end{subfigure}%
\caption{Comparison between the conventional and incremental methods for 5 holdout sets for 2 extreme IRs for both Cur (left) and Inc (right) classes.}
    \label{fig:5holdouts_BP}
\end{figure*}

\begin{table}
\centering
\caption{Average Dice and MSD
for 2 extreme IRs for 5 holdout sets in Fig.~\ref{fig:5holdouts_BP}.
Number of omitted volumes (out of 125) are given in parentheses.
Incremental methods superior to conventional approaches are in bold.}
\label{tbl:2cases5holdouts}
\adjustbox{max width=\linewidth}{
\begin{tabular}{r|cc|cc|cc|cc}
& \multicolumn{4}{c}{ Dice Score [\%] } \vline & \multicolumn{4}{c}{ Mean Surface Distance [mm] } \\ \hline
& \multicolumn{2}{c}{ IR100 } \vline & \multicolumn{2}{c}{ IR01 } \vline & \multicolumn{2}{c}{ IR100 } \vline & \multicolumn{2}{c}{ IR01 } \\ \hline
Method & Cur & Inc & Cur & Inc & Cur & Inc & Cur & Inc \\ \hline
CurSeg & 95.7 & - & 95.6 & - & 0.98 & - & 1.06 & - \\
IncSeg & - & 95.7 & - & 57.8\,(25) & - & 1.01 & - & 7.85\,(25) \\ \hline
finetune & 71.8\,(1) & 59.1 & 68.8\,(27) & 49.5\,(13) & 5.77\,(1) & 12.02 & 6.95\,(27) & 12.58\,(13) \\ \hline
LwfSeg & 95.5 & \textbf{96.3} & 86.9 & \textbf{64.7} & \textbf{0.83} & \textbf{0.73} & 2.22 & 9.21 \\
AeiSeg & \textbf{95.8} & 95.5 & 93.3 & \textbf{76.5\,(1)} & 0.99 & \textbf{0.85} & 1.14 & \textbf{5.05\,(1)} \\
\end{tabular}
}
\end{table}

Note that, when learning convolutional kernels, using a fixed physical image resolution may be crucial for capturing anatomical variation in images.
Our preprocessing step of rescaling images does not preserve such physical dimensions.
Our data augmentation with random scaling may be, to some extent, compensating for this physical dimension loss through scale invariance, but this is a point that has not been well explored in the literature.
We herein study this comparatively to the original approach above of resized images, in an additional experiment using patches of size $224\times224$\,px  extracted from the original volumes at their isotropic resolution of $0.4$\,mm.
We tested this for the incremental ratios (IR100 \& IR01) for one holdout set, with averaged scores presented in Table~\ref{tbl:resize_vs_patches}.

To show independence from the incrementing order of structures, we additionally repeated the experiments with current dataset annotations of tibia vs.\ background (Cur*~$\leftarrow$~tibia) and incremental dataset annotations of femur vs.\ background (Inc*~$\leftarrow$~femur), with the results given in Table~\ref{tbl:rv_2cases5holdouts}.

\section{Discussions}
\label{sec:discussions}

Although the test set from 2 holdout sets were different, it is important to point out that all experimented data IRs had the same test volumes, hence quantitative numbers across different ratios and models for the same class are comparable in Table~\ref{tbl:4cases2holdouts}.
Note that for the incremental class, the baseline incremental method, finetuning, always yields results inferior to the conventional method IncSeg.
This is simply because during the total of $n_\mathrm{stp}$ steps training, average validation performance of current and incremental classes together does not improve at all after the first few training steps.
Hence, finetuning test results show minimal sign of fitting the Inc dataset.

\begin{table}
\centering
\caption{Comparison of methods when trained using resized images vs patches extracted at original isotropic resolution of $0.4$\,mm, for a single holdout set.
Dice differences over 10\% between resized and patches are marked in bold.
}
\label{tbl:resize_vs_patches}
\adjustbox{max width=\linewidth}{
\begin{tabular}{r|r|cc|cc|cc|cc}
& & \multicolumn{4}{c}{ Dice Score [\%] } \vline & \multicolumn{4}{c}{ Mean Surface Distance [mm] } \tabularnewline  \hline
 &  & \multicolumn{2}{c}{ IR100 } \vline & \multicolumn{2}{c}{ IR01 } \vline & \multicolumn{2}{c}{IR100 } \vline & \multicolumn{2}{c}{ IR01 } \tabularnewline \hline
\multicolumn{2}{c}{ Method } \vline & Cur & Inc & Cur & Inc & Cur & Inc & Cur & Inc \tabularnewline \hline
 \multirow{2}{*}{CurSeg } & Resized & 0.96 & - & 0.97 & - & 0.75 & - & 0.64 & -  \tabularnewline
& Patches & 0.97 & - & 0.97 & - & 1.07 & - & 1.27 & - \tabularnewline
\multirow{2}{*}{ IncSeg } & Resized & - & 0.96 & - & \textbf{0.66} & - & 1.26 & - & 5.80 \tabularnewline
& Patches & - & 0.95 & - & \textbf{0.77} & - & 2.13 & - & 5.16 \tabularnewline \hline
\multirow{2}{*}{ LwfSeg } & Resized & 0.96 & 0.96 & 0.90 & \textbf{0.82} & 0.84 & 0.77 & 2.05 & 3.07 \tabularnewline
& Patches & 0.96 & 0.96 & 0.86 & \textbf{0.71} & 1.77 & 0.90 & 2.93 & 6.53 \tabularnewline
\multirow{2}{*}{ AeiSeg } & Resized & 0.96 & 0.97 & 0.94 & \textbf{0.92} & 0.80 & 0.77 & 1.00 & 4.28 \tabularnewline
& Patches & 0.94 & 0.96 & 0.92 & \textbf{0.78} & 1.33 & 1.45 & 1.74 & 6.79 \tabularnewline
\end{tabular}
}
\end{table}

\begin{table}
\centering
\caption{Average Dice and MSD metrics for 5 holdout sets for two extreme IRs, when the order of incremental structures is switched, i.e.\ Cur*$\leftarrow$tibia and Inc*$\leftarrow$femur.
Number of omitted volumes (out of 125) are in parentheses.}
\label{tbl:rv_2cases5holdouts}
\adjustbox{max width=\linewidth}{
\begin{tabular}{r|cc|cc|cc|cc}
& \multicolumn{4}{c}{ Dice Score [\%] } \vline & \multicolumn{4}{c}{ Mean Surface Distance [mm] } \\ \hline
& \multicolumn{2}{c}{ IR100 } \vline & \multicolumn{2}{c}{ IR01 } \vline & \multicolumn{2}{c}{ IR100 } \vline & \multicolumn{2}{c}{ IR01 } \\ \hline
Method & Cur* & Inc* & Cur* & Inc* & Cur* & Inc* & Cur* & Inc* \\ \hline
CurSeg & 95.2 & - & 95.8 & - & 0.84 & - & 0.79 & - \\
IncSeg & - & 96.1 & - & 57.2\,(10) & - & 0.72 & - & 8.08\,(10) \\ \hline
finetune & 60.3 & 61.6 & 75.2\,(1) & 29.6\,(9) & 10.75 & 9.85 & 4.43\,(1) & 17.73\,(9) \\ \hline
LwfSeg & \textbf{96.1} & 95.3 & 91.4\,(11) & \textbf{60.8\,(1)} & \textbf{0.71} & 0.90 & 1.60\,(11) & 8.17\,(1) \\
AeiSeg & \textbf{96.3} & 95.1 & 89.2\,(1) & \textbf{78.5\,(1)} & \textbf{0.66} & 0.95 & 1.62\,(1) & \textbf{4.81\,(1)} \\
\end{tabular}
}
\end{table}

Both of our proposed class-incremental methods (i.e.,\ LwfSeg and AeiSeg) performed superior to alternatives for all incremental ratios (cf. ~\ref{tbl:4cases2holdouts}).
In addition, considering IncSeg at IR100 as an ideal reference for Inc class segmentation, both LwfSeg and AeiSeg are within 0.5\% of this in Dice even at a low incremental ratio of IR17, and still within 10\% at IR04.
Furthermore, in Fig.~\ref{fig:2holdout_all_methods_BP}, the results are seen to be more robust with the proposed LwfSeg and AeiSeg methods for samples across all IRs.
Moreover, we also considered a hypothetical upperbound scenario where current and incremental datasets are concurrently available, and trained a conventional (i.e.,\,non-incremental) method, \textit{CurIncSeg} (quantitative results available in the supplementary material).
CurIncSeg did not outperform our proposed methods LwfSeg and AeiSeg, which supports the arguments in~\cite{hinton2015distilling}, where knowledge distillation helps generalizability and reduces changes of overfitting.

AeiSeg method is seen to tolerate domain shift better than LwfSeg.
Note that as the data imbalance deepens (i.e.\ going from IR100 to IR01), incremental methods (particularly LwfSeg) generally perform worse for the current class Cur: For the tested IRs, Dice score of Lwfseg is, respectively, 0\%, 2\%, 9\%, and 12\% lower compared to CurSeg.
This phenomenon can be attributed to the fact that SKI10 dataset contains acquisitions from a wide variety of vendors and imaging sequences, where many Inc data, especially at low IRs, may differ substantially from the Cur dataset distribution.
Indeed, without exemplar data, LwfSeg struggles with such substantial domain-shift from Cur to Inc dataset.
A similar trend, despite being less significant is also observed with ReSeg, as it performs inferior to AeiSeg (cf. Table~\ref{tbl:4cases2holdouts}).
This can be the result of (i) picked exemplars being less useful when Cur is larger or (ii) ReSeg having difficulty in fitting for Inc when soft targets from exemplars are not helpful for the task.
All the same, ReSeg performs worse than AeiSeg for both metrics and even for the incremental classes.

In Table~\ref{tbl:resize_vs_patches}, the results between resized and patched images are seen to be comparable for Cur dataset, both in balanced and imbalanced scenarios (IR100 \& IR01).
Nevertheless, when there is an extreme imbalance, i.e.\ low incremental ratio IR, the proposed incremental methods with resized images outperform their patch-based counterparts for Inc data as seen in Table~\ref{tbl:resize_vs_patches}.
This is likely thanks to the resized images preserving the entire field-of-view anatomical context information, which can be readily transferred to the incremental task, becoming a particularly crucial added information given smaller incremental datasets.

We showed the robustness of our methods to the order of structures in Table~\ref{tbl:rv_2cases5holdouts}, where the behavior of methods across different IRs is seen to be invariant to incrementing in the reverse order.
In this experiment, we retrospectively noticed that the 10 out of 11 fail cases of LwfSeg with IR01 come from the same holdout set, which can be attributed to the single incremental volume being from a substantially different distribution compared to the test set.
Note that without exemplars, knowledge retention depends on the accuracy of the soft targets generated from the new images.
Subsequent to distillation loss using poor soft targets, LwfSeg may indeed \textit{unlearn} the Cur* class, hence the fail cases.
Thanks to retained exemplars, Aeiseg can still successfully segment the Cur* class, achieving a mean Dice score of $93\%$ for the corresponding 10 volumes.
Our methods are presented considering multiple class-incremental learning steps; nevertheless, experimentation studying such multiple incremental setup remains as future work.

Considering the number of convolutional layers in a head, we also experimented with narrower alternatives; i.e.,\ a convolutional layer of kernel size $1\times1$ at the end of the shared body for each dataset.
This has proven to be insufficient for the segmentation task.
Empirically, we found that heads with 2 convolutional layers, each having $n_\mathrm{fil}$ filters, to be satisfactory for the experimented dataset, corroborating our earlier results in~\cite{ozdemir2018learn} on a different dataset of different imaging sequences.

Memory footprint is an essential concern in our framework aiming for lifelong learning.
In our TensorFlow implementation, each new head requires an additional 73\,KB for parameter storage, which is the only additional footprint for LwfSeg.
For AeiSeg, an additional $\leq 21$\,MB was required to store the exemplar set of each class assuming $n_\mathrm{rep}=100$.

We herein focus on a class-incremental scenario.
For lifelong learning without class-incrementation, i.e.\ with new annotations of an existing class, one possible approach using our setup would be to train the current heads with both $L_\mathrm{seg}$ using the new annotations and $L_\mathrm{dis}$ using the soft targets, in an alternating fashion.
In another scenario, where new annotations can have existing classes as well as new classes, one possible approach would be to increment a new head as proposed in this work.
Later, during testing, existing class predictions can be fused through logical AND or OR operations between the multiple head predictions, based on the importance of specificity or sensitivity of the task, respectively.
Alternatively, instead of softmax operation, one can apply sigmoid function by treating each channel output as corresponding class versus all classes.

\section{Conclusions}

We have proposed a segmentation framework for lifelong learning with class-incremental annotations.
We have introduced LwfSeg for knowledge distillation to retain segmentation performance of previously trained class(es) when expanding a deep neural network to new class(es).
Thanks to knowledge transfer from pretrained classes through shared body weights, an exciting observation is that our proposed methods may outperform conventional methods for incremental class, especially when there are only a small number of incremental annotations available (few-shot learning).
We also propose an extension, AeiSeg, for successfully maintaining performance of previously trained classes, despite the domain shift caused by the incremental dataset.
We have shown that, for any incremental dataset ratio, our proposed methods outperform
the conventional methods that are trained on the incremental classes, indicating the successful knowledge transfer from previous to incremental dataset.
We furthermore observe that for some incremental ratios (e.g., IR17 in Table~\ref{tbl:4cases2holdouts} and IR100 in Table~\ref{tbl:2cases5holdouts}), our methods outrank the segmentations of even the originally-trained model for that class/structure (Cur).
This is indeed an exciting observation, showing that the knowledge gained from the incremental dataset can further improve even the previously-trained class performances.

\section*{Acknowledgment}
This work was funded by the Swiss National Science Foundation and a Highly Specialized Medicine grant (HSM2) of the Canton of Zurich.

\bibliographystyle{ieeetr}

redpajama_set_name: RedPajamaArXiv


text:
NGC 6143 is een balkspiraalstelsel in het sterrenbeeld Draak. Het hemelobject werd op 24 april 1789 ontdekt door de Duits-Britse astronoom William Herschel.

Synoniemen
 UGC 10358
 MCG 9-27-24
 ZWG 276.11
 IRAS 16205+5512
 PGC 57919

Zie ook
 Lijst van NGC-objecten

Externe links
  NASA/IPAC Extragalactic Database
  SIMBAD Astronomical Database
  SEDS

NGC-object in Draak
Sterrenstelsel in Draak


The Incaberry is indigenous to the high altitude tropical regions of South America. This exotic and delicious organic fruit is not only really tasty, but its array of health benefits puts it in a class of its own.
In a medium saucepan, add the water and bring to boil. When the water is boiling add the cinnamon quill and bay leaves, then add the incaberries. Let it simmer for 30 minutes or until the incaberries start to break down/dissolve. Take the pot out of the heat and let it cool down for 10 minutes.
Add the mix in to a blender and blend until you get a smooth paste and all the incaberries are completely blended (about 4 mins).
Put the mix on a thin wire strainer and with the help of a spatula pass all the mix through. Pass it in small batches as the Incaberries have a lot of small seeds that will block the strainer. You may need to wash out the strainer a few times during the process! You should end up with a thick mix that can coat the bottom of a spoon.
Dissolve the gelatine in the Incaberry purée and keep it warm.
Cook the sugar in a pan with 1 cup of water until the mix reaches 125C (you'll need a thermometer for this), then mix in the puree.
Whisk the eggs whites until they stand in peaks and gradually add the warm mix of incaberry puree and sugar until everything is evenly combined.
Once set, cut into squares, dust with icing sugar. Serve with with some baby coriander leaves and some finely chopped incaberries.
Similar to a Cape Gooseberry, the Incaberry is a small fruit with a glossy orange-yellow skin that is surrounded by papery leaves. Once the fruit is dried it has a distinctive bronze orange colour. Organic Incaberries are an amazing taste sensation - sweet with a fine and delicate sour finish. The taste is so unique; you just have to try them. Available from health food stores.


In pot fry cumin seeds, asefotida and methi seeds in oil till brown.
Then gradually add besan powder. Continuously stir the besan till it is golden brown.
Then add 4 cups of warm water.
Continue to stir the besan in water till you have consistency in the liquid.
Then add salt, ginger, green chilli, tamarind and spices to the liquid.
Let the liquid boil and then add vegetables and, curry leaves.
Set the heat low and let it slow boil for 15 minutes.
Then add mint and cilantro to the liquid.


text:
Louise Harrison
Louise Harrison is the older sister of George and in fact played an important role in the Beatles' success in America. In 1963, the year before the Beatles arrived, Louise was living in the United States when she was asked by her mother to bring some of the records George made with his new band. ...
The Beatles, British Music Industry, Women in the Music Industry, Rock and Roll, Radio, Music Advocacy, Relative, Liverpool, George Harrison
John Fitzgerald focused on making creative music ever since his education at Cal Arts in the late 1970s. As his career developed, all of his experiences brought John directly to Remo Belli's programs for Recreational Music Making (RMM). Remo helped pioneer music wellness programs and the many...
RMM Recreational Music Making, The Beatles, Drums, Rock and Roll, Music Wellness, Remo Inc., World Beat Music
Patrick Wilson grew up in a very musical family. His father played piano in his own tenor band that his mother sang in and his sisters also played and sang. Patrick recalls the first instrument he picked up and played at age 6: the triangle. All throughout high school he played percussion in band...
Music Publishing, The Beatles, Percussion, Drums, Alfred Music, Youth Orchestra
Brad Smith is the Senior Sales and Marketing Manager for Hal Leonard. During his career as a writer and industry historian, he worked for MIX Magazine before joining the Hal Leonard team. Among his many tasks is overseeing the publication of books dedicated to the history of musical products such...
Music Publishing, Hal Leonard Corporation, Music Historians, The Beatles, Guitars, Virtual Interview, Multiple Interviews, Full Interview, Chad Smith
Masaru Sawano
Masaru Sawano began playing guitar as a young student when he first heard the Beatles. He took a liking to music and the first job he took after college is the one he still has, working for Japan Music Trades. Although today he is president of the magazine and he started out as a part time...
The Beatles, Guitars, Trade Press, Japanese Music Industry, Japanese Music Fair, Japan Music Trades Magazine
Mike Huckabee served as the 44th Governor of Arkansas from 1996-2007. He was a candidate for the 2008 presidential primaries and currently hosts a program on Fox News. He is also a musician and strong supporter of music education. He has made music advocacy a main topic in many of his roles over...
Basses, Rock and Roll, The Beatles, Government Relations, Music Advocacy, Government Official, Gov. Mike Huckabee
Lew DiTommaso
Lew DiTommaso was a key member in Daddy's Junky Music for many years and built a reputation on honest and passion driven hard work. That passion for Lew started when he first heard the Beatles, several months before their famous American debut on the Ed Sullivan Show in 1964. After playing in...
Massachusetts, Combo Line, Rock and Roll, The Beatles, Multiple Interviews, The Ed Sullivan Show, Daddy's Junky Music
Bill Reim
Bill Reim is the Founder of Reim, Graham & Associates. He was also the CEO of Hoshino USA, based in Pennsylvania. He can remember watching the Beatles on TV when he was in the second grade and later formed a rock band with some of his classmates while in high school. After studying graphic...
Guitars, Hoshino Gakki Mfg. Co. Ltd., Music Manufacturing, The Beatles, Drums, Percussion, Concert Promoters, Graphic Design, NAMM Board of Directors-Interviewed
Earl Matzkin
Earl Matzkin was arrested by the East German Army in 1985 for trying to pass the Berlin Wall that divided the country without the needed visas in order to visit the Petrof Piano Company. Earl had high hopes for exporting the piano to the United States, but was unaware of what was required to enter...
Petrof Piano, Music Wholesale-Distribution, Pianos, The Beatles, Hofner Bass, Amplifiers, Basses, Guitars, Music Manufacturing, German Music Industry
Claudio Formisano
Claudio Formisano grew up in Naples in a musical family where at a young age his three brothers convinced Claudio to play drums to complete their Beatle-like band! When he was 12 years old he formed his own band, which he called Beautiful! Claudio likes to say his life has been music and he...
Drums, The Beatles, Music Advocacy, Music Industry Associations, Italian Music Industry


The whole world knows Freddie Mercury as a bright and self-confident artist. However, there is only one woman who knew him when the surname Mercury was still unknown. She was also the one who stayed by his side when his illness defeated him. Mary Austin was Freddie Mercury's common-law wife and his only friend.
Bright Side is a fan of the band Queen and of Freddie Mercury's music. We're also suckers for a good romantic story. So today we're dedicating our article to the love story and deep friendship between a simple girl and a world famous musician.
Mary Austin and Freddie Mercury met each other thanks to their love for beautiful clothing. Mary was working at a London clothing store called Biba, while Freddie was selling vintage clothing at Kensington Market together with his friend Roger Taylor (Queen's drummer). At that time, the name Freddie Mercury didn't mean anything to most people and there was one more year left until Queen even made a blip on the radar.
Mary Austin was born into a family of deaf and mute people and communicated with her parents with the help of sign language. In other ways, this family wasn't much different from any other family. Her father was a plasterer and her mother was a maid. This small and fragile girl attracted the future rock idol's attention instantly. However, Mary herself was a little shocked because Freddie was always wearing bright and shocking clothing and this time was no exception. She saw a 24-year-old long-haired badass in a shocking outfit, which of course, made her feel a little insecure and embarrassed.
Mary remembers that they were as poor as church mice during those times and their romance was similar to the relationship of thousands of other couples. They would listen to music and take long walks, but they couldn't afford much else. It was pretty difficult for musicians who were trying to break into the music scene in London in the 1970s because there were thousands of other musicians like them.
After a couple of months of dating, the young couple started living together in a tiny apartment. Soon after that, Queen, the legendary band that would soon conquer the whole world, appeared. However, at that moment no one was aware of this, so the couple led a relatively simple life. Freddie wrote songs, while Mary supported him.
Queen released their first album in 1973. It was then that their life started to change and shortly they were able to afford an apartment in a very prestigious district of London. That's when Freddie Mercury wrote the ballad Love of My Life, dedicated it to Mary, and proposed to her. The proposal was done with the sense of humor that Mercury has always had.
Then, when I was 23, he gave me a big box on Christmas Day. Inside was another box, then another, and so it went on. It was like one of his playful games. Eventually, I found a lovely jade ring inside the last small box.
Freddie introduced his fiancée to his mother, Jer Bulsara, who was extremely delighted with her son's choice. She dreamed that the couple would get married and have children. However, the wedding didn't happen. The engagement lasted for 3 years and ended up with a breakup.
After the band's first album, they released 2 more with one year in between. Queen was becoming hugely popular, Freddie was gaining many new fans and Mary started to feel odd in Freddie's new world.
"Sometime later," she says. "I spotted a wonderful antique wedding dress in a small shop. And as Freddie hadn't said anything more about marrying, the only way that I could test the water was to say, 'Is it time I bought the dress?' But he said no. He had gone off the idea and it never happened.
Freddie had started to come home late and their relationship was fading. After some time he confessed to Mary about his non-traditional sexual orientation. Mary recalls that they tried to always be frank with each other and avoided arguments. Their love story ended, but the friendship and platonic love between this great rock star and this simple girl became lifelong.
After the breakup, Mercury managed to persuade Mary to not leave him and she settled into a place a couple of steps from where he lived. She was always beside him working as an assistant and accompanying him to concerts and on his tours.
Mary Austin had her own personal life just like Freddie Mercury. Mary gave birth to 2 children but didn't marry their father. Freddie Mercury became the godfather to Mary's oldest son. At that time, Freddie was desperately looking for love, he was surrounded by crowds of fans but he always approached Mary when looking for warmth and support.
In the beginning, there were only 2 people who knew that Freddie Mercury was HIV positive — Mary Austin and Jim Hutton. When the illness had almost defeated the singer, these 2 people were always there beside him. Mary was his lifelong and faithful friend, while Jim was the last lover of the great singer.
Hutton himself lived for 20 years after being infected with HIV and died in 2010 from lung cancer.
In 1991, Mercury lost his eyesight and was almost bedridden. He stopped taking his pills and refused to fight for his life. Mary and Jim were there with him — they were reflecting on old times, watching recordings of Queen concerts, laughing, and crying together.
Freddie Mercury passed away on the 24th of November, 1991. Before his death, Freddie made a will where he gave the biggest part of his income (present and future) and his huge mansion, Garden Lodge in London, to Mary Austin.
Mary Austin said that Mercury's friends and colleagues turned their back on her once they learned that Freddie had left the biggest part of his money to her. She felt completely alone after her friend's death and had to fight with tax authorities for a pretty long time because they were threatening to take Mercury's mansion away.
Another important point of Mercury's will was his funeral. The singer wanted for Mary Austin to spread his ashes and keep the place she did it a secret. Mercury was afraid that strange people would assemble at his grave and he didn't want anyone to defile his final resting place.
Mary kept the urn with Mercury's ashes for 2 years and then spread them. She fulfilled Freddie's request — no one in this world knows where the ashes of the great Freddie Mercury are.
Queen fans have several theories about it. The first one says that Mary took the singer's ashes to his motherland, in Zanzibar. The second theory says that Freddie's last resting place is in his favorite garden at his mansion, Garden Lodge. The third one appeared when his fans recently found a memorial plaque with the name Farrokh Bulsara, which is Mercury's real name. Mary Austin hasn't confirmed any of these theories.
Mary Austin turned 67 in 2018. She lives in Mercury's mansion behind inaccessible walls and doesn't communicate with journalists. Mary refused to take part in the shooting of Bohemian Rhapsody, but she read the script and approved it.
Queen's fans don't like Mary Austin very much. Freddie Mercury's fans from all over the world go to London to visit his Garden Lodge mansion and leave flowers, paintings, and poems there in memory of the great musician. But Mary demanded to have all of these signs removed from the fence and the door of the estate. The walls have since been covered with plexiglass and it's become impossible to leave memorable presents since then.
However, Freddie Mercury's mother and his sister Kasmira have always had a warm attitude toward Mary and have always considered her a part of the Bulsara family.
Now you know a little more about Freddie and Mary's relationship. Have you already watched Bohemian Rhapsody? We would love to read your opinion about it in the comments!


The Pet Corrector Bad Behavior and Training Aid quickly stops barking, jumping, digging and chewing. The purpose of the Pet Corrector is to gain the dog's attention at the instant they are exhibiting an unwanted behavior. When the Pet Corrector emits a hissing sound, the dog is distracted, and the owner can then insert a positive command and reward.
Test the animal's sensitivity: operate at a distance greater than 1 metre. Direct the spray away from your pet's face (not directly towards it). Only use the Pet Corrector to interrupt a serious misdemeanour, it is not a substitute for poor training. Understand that timing is crucial, it should only be used whilst the misbehaviour is occuring. Ensure that you reward immediately when the misbehaviour has ceased, ideally whilst he is performing a positive behaviour.


text:
The Importance of Being Theatrical
Site Staff,
Like the aquatic life that thrives in the ocean that borders it, Portsmouth is stocked with an unlimited variety of theatrical experiences just for the asking. Boasting dozens of groups sharing several theatres, most of these stages are within walking distance of Market Square. It's actually quite challenging to keep up with the breadth of shows that take place year-round, but it's well worth it to at least stick your toe in now and then.
Passersby at Prescott Park in Portsmouth often can't help but peek in the windows of a petite, charming brick building at 105 Marcy Street. Quite often, someone will peek back at you, for a stage surrounded by 80 seats is directly on the other side of the glass. This is Players' Ring Theatre, one of the jewels in the crown. This intimate theatre, once known as the Heritage Museum, is bordered by the teeming flower garden of Prescott Park on one side, and the village of Strawbery Banke on the other.
Inside the bricks, Players' Ring provides a stage for several theatre groups with names like Mad Dogs and Englishmen and Soul Soup to share, in addition to a roster of its own internally produced shows. With such a wide variety of visionaries in one place, you can anticipate a fresh performance about every two weeks.
Founded in 1992 by Barbara Newton and her late husband Gary, Players' Ring awards funding to aspiring playwrights in addition to giving performances. Visit www.playersring.org for details.
Just up Bow Street, creative renovations turned an 1892 beer and grain storage facility into the Seacoast Repertory Theatre. Seacoast Rep in 1987 began filling the stage with musicals, dramas, variety shows and concerts. PAPA, the Portsmouth Academy of Performing Arts, also calls the space home. Founder and Artistic Director Roy M. Rogosin and co-founder Eileen Rogosin keep the schedule lively. From funky midnight performances of "The Rocky Horror Show Live" to fine dramas like "A Streetcar Named Desire" and children's programming like "The Lion, the Witch and the Wardrobe," the SRT covers diverse subject matter.
If Portsmouth is a sea of culture, the 126-year-old Music Hall is its anchor. From local theatre and dance to independent films, from nationally touring performers to presidential primary stump speeches, most Granite Staters will eventually find themselves there at one event or another. Known for its exceptional acoustics, the Music Hall is cozy in a Victorian-velvet kind of way, and seems unfazed by its many years as grande dame of such a thriving arts community.
Just outside Portsmouth, The Mill Pond Center for the Arts in Durham is another gem with a rich history that is shared by many performing artists. Founded in 1981 by dancer Judith Roberts and her husband Lewis, this former inn on Mill Pond gained a dance studio with sprung floor and a 90-seat theater in the hayloft during their tenure. Most importantly, they provided the inspiration and willpower to grow a diverse cultural and community center from scratch. New owners Wilburn and Deborah Bonnell oversaw the addition of an art gallery and a chamber music series. They also took steps to ensure the legacy of Mill Pond, and established it as a non-profit living arts center, overseen by a board of directors. The dance heritage remains intact, however, as the center continues to host the The Mill Pond Modern Dance Collective each year. NH


Whatever your thoughts, I shall read. I thank you for riding by.
Lady Sharon, do you have a facebook account?
I shall ride over to the Kingdom of Facebook very soon and register. When they accept this humble scribe, I shall send a missive to you.
You are always welcome to Camelot. May your day be guided by love.
Do you know Alyssa Joyce?
Can I know your email?
I have sent you an email in response to your comment. Please let me know if you do not receive it. Thank you for dropping by Ed.
Usually I do not learn article on blogs, but I wish to say that this write-up very pressured me to check out and do it! Your writing taste has been surprised me. Thanks, quite great post.
I think other website proprietors should take this site as an example , very clean and wonderful user pleasant style.
My name is Randy and I was looking at a few different sites online and came across your site knightsofthepaintable.com. I must say – your website is very impressive. I found your website on the first page of the Search Engine.
Hi, after seeing your website a quick note to let you know that we'd like to work with your company to publish a custom made marketing video which features your company online.
Need a dependable merchant account for your CBD business?
With over 20 years of experience, EthosPay makes credit card processing easy for all types of CBD businesses.


Situated just off Dublin's main downtown shopping area, Grafton Street in the heart of the City, Trinity College Dublin, the University of Dublin, boasts over 600 guest rooms and apartments on its historical university campus. An oasis of calm in the centre of this bustling European city, the 42 acre campus includes a state-of-the-art sports centre for guests to enjoy and is perfectly located to explore the culture and vibrancy of Ireland's capital. Trinity itself is home to some of Ireland's greatest cultural treasures: the Book of Kells, the Old Library and the Science Gallery.
Taxi: There are taxi ranks outside both terminals, and a journey to the city centre will cost between €20 and €30.
The Leap Visitor Card is the most convenient public transport card available for tourists and visitors to Dublin. The Card allows unlimited travel over your selected time period on Airlink, Dublin Bus, Luas, DART and Commuter Rail. You can even travel to and from the Airport at the start and end of your trip without any additional charge. Leap Visitor is not valid on the Hop on/Hop off tour or other special tour services.
Depending on how long you plan on staying, you can choose from a number of options and find one that best suits your visit.
If you are planning to take a trip around Ireland before or after the conference, there are multiple different routes to choose from. Some locations can be reached by public transport (http://www.irishrail.ie/) (http://www.buseireann.ie/) but some may require a car (https://www.europcar.ie/) (https://www.hertz.ie/rentacar/reservation/). For a list of B&Bs, guesthouses, and hostels, search (www.booking.com). Advanced booking for major attractions and tours is essential, in particular The Titanic Experience and the Book of Kells. Ireland is a small country. Most of the major cities can be reached by car within 3 hours or under. While our national road infrastructure is good, the secondary roads can meander and be quite narrow. We outline some basic routes for excursions below, if you require further information, you can visit (https://www.discoverireland.ie/) and (https://www.tourismireland.com/contact-us).
Take the train or drive to Galway City. Attend a Druid Theatre performance and then relax in Tigh Neachtains with a pint of Guinness. The next day, travel to Connemara and feel like you have stepped back in time as you pass by the famed rugged scenery and stone cottages. Take a trip to the Aran Islands by ferry.
Travel to Belfast. Visit the Titanic Quarter, take a black taxi tour of the political murals, and then unwind in a snug at Crown Bar (you can read up on the history of the Irish snug here.) Next day, drive to UNESCO World Heritage site, Giant's Causeway, in County Antrim. On the way, stop in the village of Bushmills where Bushmills Whiskey is distilled. From Giant's Causeway, travel to Derry, the "walled city." Walk around the city walls, take in the city's murals, and visit The Museum of Free Derry to learn more about Bloody Sunday.
Leave Dublin and travel to the charming village of Adare in Co. Limerick. On the way, stop at the small village of Moneygall, Co. Offaly where Barack Obama ancestors lived. Adare boasts beautiful stone cottages with thatched roofs, wonderful cafes and restaurants, as well as a number of high-end boutiques. From Adare, drive to Killarney and spend some time exploring Killarney National Park. Following this, visit West Cork, famed for its beautiful beaches, friendly locals, and superb food.
Due to the fact that William Butler Yeats was born and lived in Sligo, the county is often referred to as "Yeats Country''. Visit Drumcliffe Cemetery where Yeats is buried. His headstone reads: "Cast a cold eye on Life, on Death. Horseman pass by". Then, take a tour of Lissadell House which was the birthplace and home of Countess Markievicz, the first woman to be elected to Westminster Parliament (find out more about Countess Markievicz here.) Head for Donegal, the hidden jewel of Ireland. Travel to Slieve League cliffs which are among the highest in Europe. These cliffs offer magnificent views of the Atlantic Ocean and Donegal Bay. Next, visit Gartan Lough and then Glenveagh National Park and Castle. Donegal boasts the best beaches in Ireland, prime among them Rossnowlagh, Portnoo, and Portsalon beach.
If you are considering taking a day trip from Dublin, there are many options to choose from. Various tour bus companies depart daily from the city centre.
One of the closest destinations to Dublin is Glendalough in County Wicklow. Glendalough is a glacial valley with stunning scenery and great walks. It is also a monastic site, founded by St. Kevin in the 6th century.
Belfast is just over two hours away from Dublin and a great option for a day trip. Consider taking a "Black Taxi tour" of the city, you will visit murals and gain an in-depth local knowledge of The Troubles and Northern Ireland (Here are two tour companies that we recommend—http://coiste.ie/ offer walking tours of the murals and https://www.niblacktaxitours.c… offer taxi tours.) The Titanic Experience is an excellent way to round off a day trip to Belfast but tickets should be booked in advance to avoid long queues.
Newgrange is a Stone Age passage tomb constructed over 5,000 years ago in the Boyne Valley in County Meath. Newgrange is famous for the illumination of its passages the sun at the Winter Solstice.
Galway is a charming and friendly city which is best explored on foot. It is packed full of artists, music, and culture. Visit the Spanish Arch and the harbour followed by a pint of Guinness at Tigh Neachtains. Galway will be European Capital of Culture in 2020.
Guinness Storehouse was voted Europe's leading tourist attraction at the World Travel Awards in 2015, ahead of The Eiffel Tower, The Colosseum, and Buckingham Palace. Take a tour of the storehouse and see how Guinness is made. Once the tour is finished you will be taken to the Gravity Bar which offers unrivalled views of Dublin City.
Day Tours offer the following day tours from Dublin- Cliffs of Moher, Connemara, Cork, Blarney & Cashel, Belfast, Giant's Causeway, Glendalough & Kilkenny and Celtic Boyne Valley (Newgrange).
Paddy Wagon Tours offer the following day tours from Dublin- Giant's Causeway, Cliffs of Moher, Wicklow Mountains, Kilkenny, Connemara & Galway, County Kerry, Belfast City, Blarney Castle & County Cork.
Extreme Ireland offer the following day tours from Dublin- Cliffs of Moher, Giant's Causeway, Whiskey and Castles Tour, Celtic Boyne Valley (Newgrange), Cork & Blarney Castle and Connemara.