text:
Work with Flexibility
Keep Work Secure
Events & Webinars Resources
Can you recover overwritten files on your hard drive?
Whether the result of a misplaced click or a version control mix-up, overwriting a file can be a daunting experience. Learn how to recover your overwritten files, and prevent it from happening again, with our guide.
Discover Dropbox Backup
We've all been there. You're working on a new file and thoroughly in "the zone". In your haste to save your progress, you speed through the options and hit Save. All seems fine until you revisit the folder later on and a sudden feeling of frustration washes over you—you've just overwritten an important file.
If you're finding yourself in this position right now, don't worry! We're here to help.
In this guide, we'll explain how to recover overwritten files on both Windows PC and Mac. Plus, we'll explain how to prevent this type of file loss from happening again in future.
Recover overwritten files on a Windows PC
Recover overwritten files on a Mac
How did my file get overwritten?
Files can be overwritten for a number of reasons, and in some cases, the circumstances can determine whether or not the original file will be recoverable at all.
Let's be honest, files don't overwrite themselves. If a file has been saved over, there's a pretty good chance that it's the result of human error in some shape or form.
Here are just a few examples of common mistakes that could lead to a file being overwritten:
You've gone to save a new file and accidentally clicked on an existing file before hitting Save
You meant to overwrite a file with a new version, but clicked on the wrong one
You returned to a file, made some changes, and then saved the file—either accidentally or intentionally. It might be that you were just experimenting, such as trying some changes to a digital illustration or video edit, and didn't mean to overwrite your original copy.
File deletion and overwriting of storage space
In addition to manually saving-over a version of a file, the term overwriting can also refer to the overwriting of the storage space itself.
When a file is deleted, it's often not actually removed from the hard drive, but rather the location on the drive where the file is stored is marked as available for overwriting with new data. Once that new data is stored to the same location, recovery of the original overwritten data becomes much more difficult.
Luckily, recovering files that have been overwritten due to human error can be quite simple.
How to recover overwritten files on a Windows PC
In order to recover an overwritten file that was saved locally on your Windows PC, you'll need to have had the File History feature enabled.
Once that's confirmed, simply follow these steps to recover your file:
Open Windows Explorer and open the folder that contained the overwritten file
From anywhere within the folder, right-click and select Properties
Select the Previous Versions tab
Look at the list of versions, each with a date, and look for the version you want
Select the version you'd like to restore
Click Restore
How to recover overwritten files on a Mac
File recovery on Mac devices is much simpler than their Windows counterparts, thanks to built-in systems on MacOS.
Providing the Time Machine feature has been enabled, recovering overwritten files is simple:
Open Time Machine
Navigate to the folder containing the file
Selecting the previous version from the Time Machine vault
What to do next: Backup your system for quick and easy file recovery
Discovering that you have overwritten a file, and then figuring out how to recover it, is a grueling task that most would prefer not to go through more than once.
Thankfully, there is an easier way to keep your files and all of their versions safe—with automatic cloud backup services like Dropbox Backup.
With our helpful, dedicated cloud backup service, you can regularly store a digital snapshot of your devices and their files, which can then be recovered whenever necessary.
You don't have to worry about finding space for all these backups either. We securely store them in our cloud servers, meaning you can access your backups from anywhere. You could even recover your files to a new device, making Dropbox Backup a handy tool for migrating files to a new computer in just a few clicks.
File recovery FAQs
Can you recover an overwritten file?
Yes—you can recover a previous version of an overwritten file using the File History feature in Windows, or the Time Machine tool in MacOS, if they have already been enabled on your device. Otherwise, Dropbox Backup is a cloud-based solution that can help to avoid future file loss, giving you more ways to recover than your device's built-in local backup features.
How do you recover overwritten video files?
You can recover overwritten video files in the same way you would for any other type of file, using the File History feature in Windows, or Time Machine on MacOS.
Can you undo replacing a file?
Unfortunately, the Ctrl+Z undo shortcut doesn't work in Windows File Explorer, nor does the Cmd+Z equivalent in Finder on a Mac. To restore the lost data in a previous version of an overwritten file, you'll need to have had the respective File History or Time Machine feature enabled in advance.
Protect your files from being overwritten today, with Dropbox Backup
Dropbox Backup makes it easy to backup your files to the cloud automatically, so you can quickly and smoothly recover from whatever life throws your way.
Pick up right where you left off.
Learn more about Dropbox Backup
Do more than store
Advance access


text:
Gerhard Anton Riedmann (* 24. März 1925 in Wien; † 9. Februar 2004 in Kematen in Tirol) war ein österreichischer Schauspieler.

Leben

Der Sohn eines Bahnbeamten gehörte zu den Wiener Sängerknaben und erwarb seine Matura an der Wiener Oberrealschule. Er studierte Anfang der 1940er Jahre Violine an der Akademie für Musik, bis er zum Kriegsdienst eingezogen wurde. Von 1945 bis 1946 besuchte er das Max-Reinhardt-Seminar. Er debütierte 1947 im Theater in der Josefstadt. Von 1953 bis 1955 wirkte er an den Münchner Kammerspielen, danach in Zürich, Hamburg und bei zahlreichen Gastspielen.

Bekannt wurde Gerhard Riedmann vor allem als Filmschauspieler. Nach einigen kleineren Rollen gelang ihm 1953 der Durchbruch als Adam in einer Filmadaption der Operette Der Vogelhändler. In zahlreichen Operettenfilmen und Heimatfilmen der 1950er Jahre war Riedmann als unwiderstehlicher Charmeur der Hauptdarsteller, wobei er häufig sein gesangliches Können zeigte. Zu seinen Partnerinnen gehörten unter anderem Waltraut Haas, Renate Holm und Marianne Hold.

In den sechziger Jahren wurde Riedmann vor allem Fernsehschauspieler, wo er wiederum in Operettenverfilmungen seine eindrücklichsten Aufgaben übernahm. Danach war er noch in einigen weniger bedeutenden Nebenrollen zu sehen.

Am 15. Juli 1954 heiratete er in Muggendorf, Gemeinde Wiesenttal die Schauspielerin Eva Probst; von ihr wurde er 1960 geschieden. Aus dieser Ehe ging eine Tochter Eva-Maria (* um 1955) hervor. 1960 heiratete er die Industriellentochter Anita Swarovski und arbeitete als Leiter einer Projektgruppe der Glas- und Optikindustrie im Werk seines Schwiegervaters mit. Riedmann, Vater von insgesamt sechs Kindern, lebte zuletzt mit seiner dritten Ehefrau Gertrud in Kematen (Tirol); dort befindet sich auch sein Grab.

Filmografie

Literatur
 Hermann J. Huber: Langen Müller's Schauspielerlexikon der Gegenwart. Deutschland. Österreich. Schweiz. Albert Langen • Georg Müller Verlag GmbH, München • Wien 1986, ISBN 3-7844-2058-3, S. 829.
 Kay Weniger: Das große Personenlexikon des Films. Die Schauspieler, Regisseure, Kameraleute, Produzenten, Komponisten, Drehbuchautoren, Filmarchitekten, Ausstatter, Kostümbildner, Cutter, Tontechniker, Maskenbildner und Special Effects Designer des 20. Jahrhunderts. Band 6: N – R. Mary Nolan – Meg Ryan. Schwarzkopf & Schwarzkopf, Berlin 2001, ISBN 3-89602-340-3, S. 531 f.

Weblinks

Filmschauspieler
Theaterschauspieler
Darstellender Künstler (Wien)
Person (Tirol)
Österreicher
Geboren 1925
Gestorben 2004
Mann

redpajama_set_name: RedPajamaWikipedia


You are currently browsing the Watered Gardens weblog archives for 'Organizing' tag.
I am currently doing the online bible study called, "Made to Crave."
Tania: Hi Sweetie! Tried to leave a long comment and it was not working. 🙁 Will try again later. Love you!
Robin: Stopping in to say HI! Love you!
» Click HERE to leave a tag!!
I am a daughter of the King, wife to best friend for 23 years, mother of 4 great kids, studier of the Word, School Office Administrator, mentor to others, creative by nature!
Rich, my husband has been an engineer/paramedic/fireman with Lakeside Fire Department for over 25 years.
My 24th Anniversary starts in 0 days.


text:
More Tasty NYC Street Food
Guest Author - Karen Joyce Williams
Here's more New York City Street Food in Part Two of my article on what's good on the streets and how to eat it. There is no ranking here, just a compilation of tasty eats on the streets! For Part One of How to Eat the NEW New York City Street Food click on the link at the bottom of this article.
1. Classic Knishes
Where there are hot dogs, there are knishes, and that's every other block. Yummy, filling and nothing but carbs, the basic street knish is a potato, onion and spices filled flaky square that has either been fried or baked. Served super hot, they are good, inexpensive eats, especially on a cold winter day.
Knish is a Yiddish word that simply means bun, and every ethnic group has its variation: Jamaican beef patties, Latin American empenadas, Polish pierogis, you get the idea. Knishes now are the classic potato, or are filled with kasha (buckwheat groats) or spinach. They are also filled with cheese, coming dangerously close to the Italian calzone. There are also cheese and fruit fillings, which I will admit, is still on my "to try" list.
Upscale delis in NYC sell a round baked variety that can be filled with anything from broccoli to artichokes (okay, maybe not artichokes), but the classic street knish is a large square that has been cooked in oil, called the Coney Island Square. Visit us on a cold winter's dayand grab a knish and a hot cocoa. You'll warm up in no time!
2. Mexican Morsels
There are two ways to eat corn in the city: a grilled ear slathered with butter, mayo, cheese, a zesty combination of spices topped off with a squirt of lime, is one way.
It is a Mexican-derived treat and these babies are everywhere, Queens, Manhattan, Brooklyn the Bronx and Staten Island. Did I mention that they are served on a stick that you eat like ice cream while walking down the street. Frankly, I find this vaguely uncomfortable, as corn on the cob seems like it should be relegated to some rulebook somewhere for people who want to sit down and eat. I could go on about this...the dropping of kernels on kilims for sale at markets, the greasy fingers and mouths that attempt to engage in conversation...but I digress.
A much more discreet and far more acceptable alternative is a styrofoam (when will everyone stop using this poisonous substance for food?) cup full of esquites, sauteed corn kernels in butter or oil and flavored with the pungent herb, epazote, which is as old as the Aztecs and is medicinally used to ease stomach distress.
Served differently in different neighborhoods, there is one place I haven't tried yet in Sunset Park Brooklyn, where esquites are served with a little mayo for dipping, some crumbles of hard tangy cheese, cayenne and the ubiquitous splash of lime.
3. Pupusas
Have you ever had meat or vegetable wrapped in a grain patty? Of course you have, and we just talked about how that combo is prevalent worldwide. Pupusas, however, are corn masa flour patties containing special meat and spice combinations with a Caribbean-Latino bite.
Pupusas are the national food of El Salvador. They are filled and then grilled and served hot. These days they can be filled with sophisticated veggie combinations, cheese and spices, or chicken, beef or fish.
4 & 5. Biryani and Tikkas - complete meals
New York cannot claim Biryanis as its own. The first time I ever tasted the scrumptious Pakistani/Indian chicken and rice dish as street food I was in Niagara Falls at a carnival.
There are the biryanis then there are the tikkas and a host of other dishes that you'll have to discover on your own. We'll just talk about two here: First, biryanis are a little difficult for me to eat standing up.
Chicken, goat or lamb is marinated in spices and yogurt, simmered forever until it falls off of the bone, then is hidden inside of a mound of Basmati rice. People do it. I've seen them. Plastic forks and all. The Biriyani Cart on 46th street and 6th in Manhattan is written about all of the time, so I'm probably not telling you something novel about them. The traffic at that intersection stinks and it's better to walk to the cart than to think you can pop out of your car and grab a meal.
Next are tikkas and they are...confusing! Only because we went to two different vendors for chicken tikka, and they served up entirely different dishes. One was a tandoori (clay oven baked) style chicken, served in a yogurt-y tomato cream sauce. It was darkly spicy and scrumptious! The other tikki was chicken also, only it was boneless breast that had been marinating in something I could not define. It was served in a little checkered paper tray with yogurt and a mound of shredded cabbage and carrots. This dish was really refreshing and finally the muted flavors of coriander and mint incorporated into the dish fnally emerge, turning a dubious expectation on a corner into a really elegant moment.
There are so many food vendors on the corners of New York City! The prices are incredible reasonable (read: cheap) and because business is brisk, everything is always freshly made. New York City is still called the melting pot in some circles, and now you know why. We cook, we eat and then we cook some more. Bring your list of street food to try when you visit NYC and create your own tasting contest with your family or fellow travelers!
How to Eat the NEW New York City Street Food
100 Years of Patince and Fortitude in New York City
Content copyright © 2019 by Karen Joyce Williams. All rights reserved.
This content was written by Karen Joyce Williams. If you wish to use this content in any manner, you need written permission. Contact BellaOnline Administration for details.
Love New York? From New York City to the Adirondacks and beyond, stop in and find out what´s new and why everyone loves NY!
New York Forum
Valentine's Day In New York City
Yolanda Shoshana
Places To Say "I DO" in New York City
The Red and the Black Comes To The Stage
Subscribe for free weekly updates from this New York site.
Follow New York
New York Categories
1,000 Islands and the North Country
Capital & Saratoga Region
Go Green, Buy Vintage
Hudson Valley & Catskills
Shopping & Made in NYS
The Niagara Frontier
Western New York/ Finger Lakes Region

redpajama_set_name: RedPajamaCommonCrawl


text:
Dr Alice Taylor
Reader in Medieval History
alice.taylor@kcl.ac.uk
I joined the department in September 2011 as a Lecturer in Medieval British History. I did my undergraduate and masters degrees in Oxford (2004, 2005) and completed my doctorate there in 2009 under the supervision of Professor Richard Sharpe and Dr John Maddicott. I then held a research fellowship at King's College Cambridge until 2011.
Between 2014 and 2017, I was Co-Investigator on the AHRC project 'Models of Authority: Scottish Charters and the Emergence of Government, 1100-1250', and, between 2017 and 2020, am Principal Investigator of the AHRC project 'The community of the realm in Scotland, 1249-1424: history, law and charters in a recreated kingdom'. I have been a member of the editorial board of Past & Present since 2017, won the Royal Historical Society's Whitfield Prize for my first book (published in 2016), and am a Fellow of the Royal Historical Society.
Medieval states and government (Europe, 11-13th centuries)
Law and legal norms
Feudalism and lordship
Medieval Scotland, 11-15th centuries, particularly its law.
I am currently researching and writing a comparative history of European states and governments in the long twelfth century. My previous research focussed on the form of the medieval state in Scotland, particularly on uncovering its early legal tradition from its later medieval manuscript tradition.
I would be interested in PhD proposals from students wishing to work in any of the above areas. I am currently or have previously supervised students on medieval cartularies, communal law enforcement, universal chronicles and the letters of medieval queens.
I teach modules at levels of the undergraduate course, from survey lectures on aspects of medieval British history, to the cultural and social history of the aristocracy in the twelfth and thirteenth centuries, to a module which uses anthropological and sociological work to understand the medieval past.
I co-convene the Earlier Middle Ages seminar and the European History Seminar, 1150-1500 seminar at the Institute of Historical Research, London. I also co-host the podcast Medieval History for Fun and Profit with my colleague, Alice Rio, a fortnightly show where we attempt to answer questions from members of the public on anything they would like to know about the Middle Ages. I also have contributed to the Times Literary Supplement and have appeared on BBC Radio 4's History of Ideas Series and the long-running radio programme, Making History.
Formalising aristocratic power in royal acta in late twelfth- and early thirteenth-century France and Scotland 01 September 2018
Review of N.F. Shead ed., Scottish Episcopal Acta 1: the twelfth century 02 April 2017
Review of J.C. Holt, Magna Carta 26 January 2017
Review of 'Neville, C.J. and Simpson, G.G., eds, Regesta Regum Scottorum, vol. 4, part 1: The Acts of Alexander III, 1249-1286 (Edinburgh 2013) 01 January 2015
Review of Cynthia J. Neville and Grant G. Simpson (eds), Regesta Regum Scottorum IV, Part 1, The acts of Alexander III, 1249-1286 (Edinburgh University Press 2013) 01 January 2015
Review of T.N. Bisson, The Crisis of the Twelfth Century: Power, Lordship and the Origins of European Government 01 January 2010
Review of Cynthia J. Neville, Land, law and people in Medieval Scotland (Edinburgh University Press, 2010) 01 January 2011
Review of: Saints' Cults in the Celtic World, ed. S. Boardman, J.R. Davies and E. Williamson (Woodbridge 2009) 01 January 2011
Review of: The Cult of Saints and the Virgin Mary in Medieval Scotland 01 February 2016
The Scottish clause in Magna Carta in context: Homage, overlordship and the consequences of peace in the early thirteenth century 01 January 2018
Strand Campus, Strand, London WC2R 2LS
Room C5
Dr Alice Taylor on Royal Historical Society shortlist
Dr Alice Taylor's first book, The Shape of the State in Medieval Scotland, 1124-1290, has been…


My refreshing Green tea and Tangerine mix.
1. A sachet of green tea....2 cals.
Brew the 2 tea bags in hot water for about 5mins to infuse well. Then peel the two Tangerines and remove the seeds. Remove the white pith in the tangerine because they give a bitter taste. Pour the warm tea into the blender then add the tangerine and honey, just blitz it up, pour in a glass and chill in the fridge. please note that i didn't add water to it. Its a very light and refreshing healthy drink and its just 129calories.
If thats too much for you based on your daily calorie need, then try lemons instead. Please try it and let me know what you think. Take care and God bless.


Acute Coronary Syndrome or ACS is really a catch all term used to describe some symptoms consistent with acute myocardial ischemia when the symptoms aren't pathognomonic. Myocardial ischemia is really a situation in which insufficient blood flow is attaining the heart muscle. This is usually due to atherosclerotic plaques building-up in the coronary arteries.
In female patients, older people, and individuals with diabetes there is an increased incidence of atypical presentation. This could translate to other, non-specific, symptoms such as feeling vulnerable or light-headed into a total lack of symptoms.
Analysis of ACS often requires an Electrocardiogram. Level in the ST segment indicates that destruction has occurred for the muscle and that intervention is necessary immediately. In the absence of ST segment elevation, it is more challenging to distinguish between unstable angina and non-ST segment elevation myocardial infarction. Blood tests could be given to consider increases in cardiac enzymes. A second, typical predictor is an improved Creatine Kinase amount. Still another diagnostic device that may be used may be the ACI-TIPI. The ACI-TIPI is really a tough algorithm that uses EKG information and information to offer an estimate of the reality of myocardial infarction.
Aspirin is frequently given on-site by paramedics to reduce clot size. Beta blockers in many cases are administered to lessen the job load on one's heart. Anticoagulants, such as for example heparin, could be implemented to prevent further clots. ACE inhibitors tend to be administered to avoid a few of the heart enlargement. Removing the blockage when possible is critical to patient survival in the event of ST segment elevation myocardial infarction. Generally an angioplasty and stent placement is completed in a hour or two when possible. Doctors also can use intravenous Thrombolytics to break up clots.
For non-ST segment elevation myocardial infarction, the remedies are usually the same, though without the same time constraints.
If an angioplasty is not a viable choice due to current surgery, a bleeding problem, or multiple clogged arteries, coronary artery bypass surgery may be used to restore blood flow. Also visit Check Here.


text:
Season of Docs
Google Season of Docs is an initiative to foster collaboration between the open source community and technical writers. Mentors and writers work together for a period of 3 months (or 6 months for long running projects) to improve their open source project's documentation. The program is typically announced in March and the writer application process begins in May/June.
The WMF's participation is overseen by Wikimedia's Developer Advocacy team.
1 Season of Docs 2020
3 Our Season of Docs History
Season of Docs 2020[edit]
Season of Docs 2020: Information about Season of the Docs 2020 including potential projects.
Resources[edit]
Information for Mentors
Information for Administrators
Our Season of Docs History[edit]
Season of Docs 2019
Contact[edit]
If you have read the Season of Docs 2020 and Participants pages and still have questions for Wikimedia's program administrators or other program applicants, please join us on Zulip in the channel https://wikimedia.zulipchat.com/#narrow/stream/232472-gsod20
Retrieved from "https://www.mediawiki.org/w/index.php?title=Season_of_Docs&oldid=4273890"


text:
\section{Introduction}

In many applications, it is important to have a suitable similarity measure  between objectives, such as  text mining~\cite{DBLP:journals/ldvf/HothoNP05},  image retrieval~\cite{DBLP:conf/cvpr/HoiLLM06}. For example, when using $k$ \emph{nearest neighbor} ($k$-NN) to classify documents, the accuracy will significantly depend on the quality of the similarity measure. In addition, the performance of content based image retrieval also heavily relies on the similarity measure. To find a meaningful similarity measure, online metric learning has been extensively studied for years~\cite{DBLP:conf/nips/XingNJR02,GoldbergerRHS04,LMNNnipsWeinbergerBS05,icml_McFeeL10} due to the scalability of online learning~\cite{DBLP:conf/icml/ZhaoH10,DBLP:conf/icml/ZhaoHJY11,DBLP:journals/jmlr/ZhaoHJ11,HoiWZ14}, where a Mahalanobis distance function is learnt. Specifically, online metric learning approaches try to learn a  linear Mahalanobis distance ~\cite{de2000mahalanobis,LEGOjain2009online,OASISChechik2010,aaaiWuDZMH14}, satisfying a set of constraints. Generally the constraints assign larger similarity scores for pairs of similar instances than dissimilar instances. This is equivalent to learn a linear projection of the data to a feature space where constraints on the training set are better satisfied.  These metric learning algorithms have significantly improved the classification accuracy of $k$-NN and retrieval performance of content-based image retrieval on real-world datasets. As we know, Mahalanobis distance requires the model, i.e. the matrix, to be \emph{positive semi-definite} (PSD). So during the learning process, projections are needed to keep the model in the PSD cone. However, the projections need singular value decompositions with very high time complexity, which makes metric learning not scalable with respect to the dimension of instances.

To solve this issue, online similarity learning has been recently studied~\cite{OASISChechik2010,aaaiWuDZMH14,QamarG09,surveyBelletHS13,DBLP:conf/mm/WuHXZWM13,DBLP:journals/pami/XiaHJZ14}, which does not require its model to be PSD, so it is much more efficient than metric learning. In addition, the similarity metric can even be non-symmetric, so it can be applied to measure similarities of objects from two different feature spaces. Following this principle, similarity learning is studied for classification problems~\cite{QamarG09,IcmlBelletHS12}, and image retrieval tasks~\cite{OASISChechik2010,aaaiWuDZMH14}. Empirical studies show that those online similarity learning algorithms generally are better than or at least comparable with the metric learning algorithms on the classification tasks and image retrieval problems. At the same time, they are more efficient and scalable than metric learning algorithms, since the projection steps are avoided. As a result, online similarity learning is more suitable to large-scale data mining and machine learning tasks. However, these existing online similarity learning algorithms~\cite{OASISChechik2010,crammer2012adaptive} usually try to learn a full matrix during the learning process, which results in high memory and computational cost for high dimensional learning tasks.

Recently, to solve high dimensional metric/similarity learning problems, there are several off-line learning algorithms focused on learning sparse global linear metrics, e.g., SDML~\cite{SDMLQiTZCZ09}, SML~\cite{SMLYingHC09} and GSML~\cite{GSMLHuangYC11}. They are all based on Davis's work~\cite{ITMLdavis2007information} and try to learn a global sparse model by minimizing the Bregman divergence between the model and the identity matrix corresponding to the Euclidean distance, subject to a set of linear constraints. By minimizing Bregman divergence, the learned Mahlanobis matrix usually can be as close to the identity matrix as possible~\cite{SDMLQiTZCZ09}, so that the learnt Mahlanobis matrix can be very sparse, since  the identity matrix is rather sparse. However, these algorithms cannot handle online learning scenario, which is more realistic for many real-world data stream tasks. Motivated by the above observations, we propose a new \emph{Sparse Online Relative Similarity Learning} (\textbf{SORS}) scheme, where the similarity function is a sparse model learnt efficiently and scalably from the training data. Specifically, we propose two types of algorithms: SORS based on recent proximal online gradient descent; and AdaSORS based on state-of-the-art adaptive proximal online gradient descent. The scalability and efficiency of these advanced sparse online learning strategies make the proposed algorithms attractive to large-scale high dimensional real world applications. The contributions of our work are summarized as following: 1) We introduce a new Sparse Online Relative Similarity learning scheme that takes sparsity into consideration; 2) Based on the scheme, we propose several sparse online similarity learning algorithms; 3) We theoretically analyze the proposed algorithm; 4) We apply the proposed algorithms on a set of real-world datasets, where encouraging empirical results are achieved compared with some state-of-the-art similarity/metric learning algorithms.

The rest of this paper is organized as follows. The related work is reviewed in the next section. Section ~\ref{sec:alg} presents our proposed sparse online relative similarity learning approaches. Section ~\ref{sec:exp} conducts an extensive set of experiments by comparison with several state-of-the-art methods, and section ~\ref{sec:con} concludes our paper.

\section{Related Work}
\label{sec:relwork}

\vspace{-0.05in}
\subsection{Similarity/Metric Learning}
\vspace{-0.05in}
Online similarity learning is a group of efficient and scalable machine learning algorithms~\cite{HoiWZ14,aaaiWuDZMH14,MDMLKunapuliS12}. Most existing works in similarity learning rely on learning a Mahalanobis distance, which has been found to be a sufficiently powerful class of metrics that work on many real-world problems. POLA~\cite{POLAShalev-ShwartzSN04}, \emph{Pseudo-metric Online Learning Algorithm}, is the first online Mahalanobis distance learning approach which learns the distance metric $M\succeq 0$ as well as a threshold $b\geq1$. The authors provide a regret bound for this algorithm, under the assumption that the dataset can be separated by some metrics. LMNN ~\cite{LMNNnipsWeinbergerBS05} addresses the distance learning problem by exploring a large margin nearest neighbor classifier LEGO~\cite{LEGOjain2009online}, \emph{LogDet Exact Gradient online}, an improved version of POLA based on LogDet divergence regularization. It features tighter regret bounds, more efficient updates, and automatic semi-definiteness. RDML~\cite{RDMLjin2009regularized}, \emph{Regularized Distance Metric Learning}, is also similar to POLA but is more flexible. At each step $t$, instead of forcing the margin constraint to be satisfied, it performs a gradient descent step. Those methods focus on the symmetric distance: measure the distance between two images $\x_1$ and $\x_2$ by $(\x_1-\x_2)^\top M(\x_1-\x_2)$, where the matrix $M$ must be positive semi-definite.

The proposed online learning scheme is close to the recent work of scalable image similarity learning (OASIS)~\cite{OASISChechik2010,aaaiWuDZMH14}. This method learns bilinear similarity with a focus on large-scale problems. In this model, given two images $\x_1$ and $\x_2$, it measures similarity through $\x_1^\top M\x_2$. The metric $M$ is related to the (generalized) cosine similarity but does not include normalization nor PSD (symmetric positive semi-definite) constraint. In another part, unlike the Mahalanobis distance, it can define a similarity measure between instances of different dimensions. However, the learnt model $M$ is not sparse, which is unsuitable to large scale tasks for high dimension datasets.

\vspace{-0.05in}
\subsection{Sparse Online Learning}
\vspace{-0.05in}
The general online learning algorithms have solid performance on classification problems. However, for large-scale high-dimensional task, they learn a full feature classifier, which may cause high memory demand and heavy computational load. To deal with this limitation, the \emph{Sparse Online Learning}~\cite{LangfordLZ09,DuchiS09,DBLP:conf/icdm/WangWZMH14} has been studied recently. The goal of sparse online learning is to learn a sparse classifier, which only contains limited active features. In current \emph{Sparse Online Learning} studies, there are two types of solutions. One solution follows the general idea of subgradient descent with truncation, such as, FOBOS method~\cite{DuchiS09}, makes an improvement based on the \emph{Forward-Backward Splitting} method to solve the sparse online learning problem. There are two steps in the FOBOS method: (1) an unconstrained subgradient descent step, and (2) an instantaneous optimization step for a trade off between maintaining the result calculated in the first step and minimizing $\ell_{1}$ norm regularization. The optimization goal in the step 2 can be solved by adopting \emph{soft-thresholding} operations, which make truncation on the weight vectors. Under the guidance of this idea, the TG scheme~\cite{LangfordLZ09}, \emph{Truncated Gradient}, is proposed. When there are less than a predefined threshold $\theta$, the TG scheme truncates coefficients every $K$ steps. This demonstrates that truncation on each iteration is too aggressive as each step modifies the coefficients by only a small amount. The second solution of sparse online learning uses the dual averaging method~\cite{Nesterov09}, which studies the regularization structure in an online setting. RDA~\cite{Xiao10}, \emph{Regularized Dual Averaging}, is one representative study, which simultaneously achieves the optimal convergence rates for both convex and strongly convex loss. An extension work~\cite{LeeW12} of RDA algorithm is presented by using a more aggressive
truncation threshold and generates significantly more sparse solutions. However, all the above algorithms are designed for classification problems.

\section{Sparse Online Relative Similarity Learning}
In this section, we will review the problem setting,  propose our algorithm, and provide theoretical guarantees.
\label{sec:alg}

\vspace{-0.05in}
\subsection{Problem Formulation}
\vspace{-0.05in}
Following~\cite{OASISChechik2010}, we would study the problem of online similarity learning. Our goal is to learn a similarity function $S:\R^d\times\R^d\rightarrow \R$ based on a sequence of training triplets $\{(\x_t,\x_t^+, \x_t^-)\in \R^d \times \R^d \times \R^d| t\in[T]\}$  with $[T]=\{1, \ldots, T\}$, where the relevance between $\x_t$ and $\x_t^+$ is greater than that between $\x_t$ and $\x_t^-$. Specifically, we would like to learn a similarity function $S(\x,\x')$ that assigns higher similarity scores to more relevant instances, i.e.,
\begin{eqnarray*}\label{eqn:similarity function}
S(\x_t,\x_t^+) > S(\x_t, \x_t^-), \forall t\in[T]
\end{eqnarray*}

For example, compared with an image $\x_t^-$, $\x_t^+$ is a more similar image with $\x_t$.

For the similarity function, we  adopt a parametric similarity function that has a bi-linear form,
\begin{eqnarray*}\label{eqn:bilinear similarity function}
S_M(\x,\x') = \x^\top M \x'
\end{eqnarray*}
where $M\in\R^{d\times d}$. In order to learn the optimal parameter $M$, we introduce a loss function $\ell(M;(\x_t,\x_t^+,\x_t^-))$ that measures its performance on the $t$-th triplet. One popular loss function is the well-known hinge loss
\begin{eqnarray*}
\ell(M;(\x_t,\x_t^+,\x_t^-))=[1-S_M(\x_t,\x_t^+)+S_M(\x_t,\x_t^-)]_+
\end{eqnarray*}
where $[\cdot]_+=\max(0, \cdot)$. The above loss measures how much the violation of the desired constraint $S_M(\x_t,\x_t^+) \geq S_M(\x_t,\x_t^-)$ is by the similarity function defined by $M$.

Under these settings, an online learning algorithm updates its model in rounds. At each round a new model is estimated, based on the previous one and the current triplet. We denote the models estimate after the $t-1$-th round by $M_t$. A typical online algorithm starts with some initial models, e.g., $M_1=0$. For each round $t$, a triplet $(\x_t,\x_t^+,\x_t^-)$ will be presented to the algorithm. The algorithm can update the model from $M_t$ to $M_{t+1}$, based on the current triplet. The goal of online similarity learning is to minimize the regret of the algorithm defined as
\begin{eqnarray}\label{eqn:regret}
R_T = \sum^T_{t=1}\ell_t(M_t)-\min_M\sum^T_{t=1}\ell_t(M)
\end{eqnarray}
where $\ell_t(M)=\ell(M;(\x_t,\x_t^+,\x_t^-))$. If the regret is sublinear, i.e., $R_T\le o(T)$, it is easy to observe
\begin{eqnarray}
 \sum^T_{t=1}\ell_t(\bar{M}_T)\le \min_M\sum^T_{t=1}\ell_t(M) + o(T)
\end{eqnarray}
where $\bar{M}_T=\frac{1}{T}\sum^T_{t=1}M_t$, so  $\bar{M}_T$ is a good estimate of $\arg\min_M\sum^T_{t=1}\ell_t(M)$. In practice, The final outputted model $M_T$ usually will be used for testing. Please note the online triplets  are generally not provided in advance.

To solve this problem, some authors proposed an online learning scheme, OASIS in~\cite{OASISChechik2010}, by applying the online \emph{Passive Aggressive} (PA) learning strategy~\cite{DBLP:journals/jmlr/CrammerDKSS06}, i.e.,
\begin{eqnarray*}
M_{t+1} = \arg\min_{M}\frac{1}{2}\|M-M_t\|^2_F +C \ell_t(M)
\end{eqnarray*}
where $C>0$ is a trade-off parameter for the regularization term and loss. This algorithm enjoys the following closed-form solution
\begin{eqnarray*}
M_{t+1} = M_t + \tau_t \x_t(\x_t^+-\x_t^-)^\top
\end{eqnarray*}
where $\tau_t=\min(C, \frac{\ell_t(M_t)}{\|\x_t(\x_t^+-\x_t^-)^\top\|^2_F})$, and $\|\cdot\|_F$ is Frobenius norm. This algorithm is only provided with a mistake bound by the authors. However, a careful analysis should give it a regret bound of order $O(\sqrt{T})$.

 Another possible method to solve this problem is based on the \emph{Online Gradient Descent} (OGD) method~\cite{zinkevich2003online}, i.e.,
\begin{eqnarray*}
M_{t+1} = M_t - \eta_t G_t
\end{eqnarray*}
where $G_t=\partial_M  \ell_t(M_t)$. The gradient $G_t$ can be computed as follows
\begin{displaymath}
G_t = \left\{ \begin{array}{ll}
- \x_t(\x_t^+-\x_t^-)^\top & \textrm{if $\ell_t(M_t)>0$}\\
0 & \textrm{else}
\end{array} \right.
\end{displaymath}

This algorithm generally has a regret bound of $O(\sqrt{T})$. However these two algorithms both learn a dense matrix $M$, which may result in high testing time complexity and memory cost. To solve this issue,  we will study sparse online learning techniques to tackle this problem in this paper.

\vspace{-0.05in}
\subsection{Sparse Online Relative Similarity Learning (SORS)}
\vspace{-0.05in}
To make the model sparse, we can add sparse regularization function into the online objective function, i.e.,
\begin{eqnarray}\label{eqn:online-objective}
&&\hspace{-0.4in}M_{t+1}= \arg\min_{M}\Big[\ell_t(M) \hspace{-0.02in}+\lambda r(M) \hspace{-0.02in}+\frac{\|M\hspace{-0.02in}-\hspace{-0.02in}M_t\|^2_F}{2\eta_t}\Big]
\end{eqnarray}
where $r(M)$ is typically set as
\bqs
r_1(M)=\|M\|_1=\sum_{ij}|M_{ij}|
\eqs

This objective function has three terms:
\begin{itemize}
\item the first one is to minimize the loss of new similarity matrix on the current triplet;
\item the second term is to make the new similarity matrix sparse, where $\lambda$ is a sparsity parameter. If $\lambda$ is larger, the model will be sparser.
\item the final term is used to keep the new similarity matrix close to the current one.
\end{itemize}

The reason to use $\|M\|_1$ is that $\|M\|_1$ is a convex upper bound of $\|M\|_0$, which can make the optimal solution sparse, i.e., most of $M_{ij}$ will be zeros. In this way, the computation of $S(\x,\x')$ will be significantly reduced, since $S(\x,\x')=\sum_{ij}M_{ij}\x_i\x'_j$. However, this regularization treats the diagonal elements of $M$ equally with those off diagonal elements, it is clearly insufficient,  since diagonal elements should generally be dense while  those off-diagonal elements should be sparse. The reason is that the diagonal element represents the self-correlation of a feature while the off-diagonal elements represent correlation between features. Self-correlations are always larger while inter correlations are generally sparse in high dimension space. To solve this issue, an alternative is to adopt the off-diagonal $L_1$ norm
\bqs
r_2(M)=\|M\|_{1,\text{off}}=\sum_{i\not=j}|M_{ij}|
\eqs
to pursue a sparse solution while keep the diagonal elements dense.

Given the regularization function, we would solve the online objective function Eq.~\eqref{eqn:online-objective}. However this optimization usually does not have a closed-form solution, which will incur additional computational cost. To tackle it, we propose to linearize the loss function $\ell_t(M)$, to get the following new online objective function
\begin{eqnarray}\label{eqn:goal}
&&\hspace{-0.4in} M_{t+1}\hspace{-0.03in}\nonumber\\
&&\hspace{-0.4in}=\arg\min_{M}\hspace{-0.03in}\Big[\ell(M_t)+\hspace{-0.03in}\langle M-M_t, G_t\rangle  \hspace{-0.03in}+\hspace{-0.03in}\lambda r(M) \hspace{-0.03in}+\hspace{-0.03in}\frac{\|M-M_t\|^2_F}{2\eta_t}\hspace{-0.02in}\Big]\nonumber\\
&&\hspace{-0.4in}=\arg\min_{M}\hspace{-0.03in}\Big[\hspace{-0.03in}\langle M, G_t\rangle  \hspace{-0.03in}+\hspace{-0.03in}\lambda r(M) \hspace{-0.03in}+\hspace{-0.03in}\frac{\|M-M_t\|^2_F}{2\eta_t}\hspace{-0.02in}\Big]
\end{eqnarray}
where $\langle M, G_t\rangle=\Tr(M^\top G_t)$ and $G_t=\partial_M\ell_t(M_t)$.

It is easy to verify that this objective will produce the $t+1$-th iterate as
\bq\label{eqn:SORSproxfunc}
M_{t+1} = \prox_{\eta_t\lambda r}(M_t-\eta_t G_t)
\eq
where $\prox_h(M)=\arg\min_N\Big(h(N)+\frac{1}{2}\|N-M\|^2_F\Big).$
Luckily, when $r_1(M)=\|M\|_1$, we have
\bq\label{eqn:SORSrank1}
\prox_{\eta_t\lambda r_1}(M)=\sign(M)\odot[|M|-\eta_t\lambda]_+
\eq
and when $r_2(M)=\|M\|_{1,\text{off}}$, we have
\bq\label{eqn:SORSrank2}
\prox_{\eta_t\lambda r_2}(M)=\sign(M)\odot[|M| - \eta_t\lambda (1-I)]_+
\eq
where $I$ is the identity matrix. $\odot$ is element-wise product for two matrices. These updates can be computed efficiently.

Finally, we can summarize the proposed  Sparse Online Relative Similarity Learning algorithm in Algorithm~\ref{alg:SORS}.

\begin{algorithm}[htpb]
\caption{\textbf{S}parse \textbf{O}nline \textbf{R}elative \textbf{S}imilarity Learning {\bf(SORS)}} \label{alg:SORS}
\begin{algorithmic}[1]
\STATE {\bf Input}: Regularization parameter $\lambda>0$, learning rates $\eta_t>0$, and regularizer $r(\cdot)$.
\STATE {\bf Initialize}:  $M_1=I$.
\FOR{$t=1,\ldots,T$}
\STATE Receive $(\x_t,\x_t^+,\x_t^-)$;
\STATE Compute $\ell(M_t;(\x_t,\x_t^+,\x_t^-))$;
\STATE $G_t=\partial_M\ell(M_t;(\x_t,\x_t^+,\x_t^-))$;
\STATE $M_{t+1} = \prox_{\eta_t\lambda r}( M_{t}-\eta_t G_t)$;
\ENDFOR
\STATE {\bf Output}: $M_{T+1}$
\end{algorithmic}
\end{algorithm}

\vspace{-0.05in}
\subsection{Adaptive Sparse Online Relative Similarity Learning (AdaSORS)}
\vspace{-0.05in}
To improve the learning performance of relative similarity learning, a second order similarity learning algorithm AROMA is proposed~\cite{DBLP:conf/icml/CrammerC12}. AROMA is based on the well-known Confidence Weighted learning strategy~\cite{DBLP:conf/icml/HoiWZ12}, which does not only use the first order information, i.e., weighted mean of examples, but also the second-order information, i.e., the covariance matrix for all the features, to update the model. Theoretical results and empirical comparisons have shown that AROMA can significantly outperform OASIS in terms of information retrieval performance. However, the model produced by AROMA is still dense.

In this subsection, we would like to adopt the second order information of the data stream to further improve the learning efficiency and efficacy of sparse online relative similarity learning algorithm. Different from AROMA, we will utilize the idea of AdaGrad~\cite{jmlrDuchiHS11,DBLP:conf/aaai/DingZHO15} to incorporate the second order information through adaptive regularization. Specifically, we will maintain a matrix $\Sigma_t= \delta  + H_t$, where $\delta$ is a smooth parameter to keep each element of $\Sigma_t$ invertible and $H_t$ is similar with correlation matrix between features. Specifically, $H_1$ will be initialized as zero matrix, and at the $t$-th iteration, it will be updated using the following rule:
\bqs
H_{t,ij} = \sqrt{H_{t-1,ij}^2+G_{t,ij}^2},\forall i,j\in[d]
\eqs

Given the matrix $\Sigma_t$, we can introduce a norm induced by $\Sigma_t$, which is defined as follow:
\bqs
\|M\|^2_{\Sigma_t} = \sum_{ij} \Sigma_{t,ij} M_{ij}^2
\eqs

Given this norm, we can follow the similar idea of AdaGrad to replace the final term of the  first-order objective function of SORS with this new norm, to get the following updating strategy
\begin{eqnarray}\label{eqn:so-goal}
&&\hspace{-0.4in} M_{t+1}\hspace{-0.03in}=\arg\min_{M}\hspace{-0.03in}\Big[\hspace{-0.03in}\langle M, G_t\rangle  \hspace{-0.03in}+\hspace{-0.03in}\lambda r(M) \hspace{-0.03in}+\hspace{-0.03in}\frac{\|M-M_t\|^2_{\Sigma_t}}{2\eta_t}\hspace{-0.02in}\Big]
\end{eqnarray}
which is considered as the objective of Adaptive Sparse Online Relative Similarity learning algorithm. The unique difference between this objective and SORS is the final proximity term.

If we define a proximal operator as
\bqs
\prox_{h}^{\Sigma_t}(M)=\arg\min_N(h(N)+\frac{1}{2}\|N-M\|^2_{\Sigma_t})
\eqs
then, it is easy to see the solution for AdaSORS is
\bq\label{eqn:AdaSORSproxfunc}
M_{t+1}=\prox_{\eta_t\lambda r}^{\Sigma_t}(M_t-\eta_t G_t./\Sigma_t)
\eq
where $./$ is element-wise division.

When $r$ is set as $r_1=\|M\|_1$, the corresponding proximal operator is
\bq\label{eqn:AdaSORSrank1}
\prox_{\eta_t\lambda r_1}^{\Sigma_t}(M)=\sign(M)\odot[|M|-\lambda\eta_t./\Sigma_t]_+
\eq
and when $r$ is set as $r_2=\|M\|_{1,off}$, the corresponding proximal operator is
\bq\label{eqn:AdaSORSrank2}
\prox_{\eta_t\lambda r_2}^{\Sigma_t}(M)=\sign(M)\odot[|M|-\lambda\eta_t(1-I)./\Sigma_t]_+
\eq
whose time complexities are $O(d^2)$ so that they can be conducted efficiently.

Finally, we can summarize the proposed Adaptive Sparse Online Relative Similarity Learning algorithm in Algorithm~\ref{alg:AdaSORS}.
\begin{algorithm}[!htpb]
\caption{\textbf{Ada}ptive \textbf{S}parse \textbf{O}nline \textbf{R}elative \textbf{S}imilarity Learning {\bf(AdaSORS)}} \label{alg:AdaSORS}
\begin{algorithmic}[1]
\STATE {\bf Input}: Regularization parameter $\lambda>0$, learning rates $\eta_t>0$, smooth parameter $\delta$, and regularizer $r(\cdot)$.
\STATE {\bf Initialize}:  $M_1=I$.
\FOR{$t=1,\ldots,T$}
\STATE Receive $(\x_t,\x_t^+,\x_t^-)$;
\STATE Compute $\ell(M_t;(\x_t,\x_t^+,\x_t^-))$;
\STATE $G_t=\partial_M\ell(M_t;(\x_t,\x_t^+,\x_t^-))$;
\STATE $H_{t,ij} = \sqrt{H_{t-1,ij}^2+G_{t,ij}^2}$, $\forall i,j\in[d]$;
\STATE $\Sigma_t = \delta + H_t$;
\STATE $M_{t+1} = \prox_{\eta_t\lambda r}^{\Sigma_t}( M_{t}-\eta_t G_t./\Sigma_t)$;
\ENDFOR
\STATE {\bf Output}: $M_{T+1}$
\end{algorithmic}
\end{algorithm}

\vspace{-0.05in}
\subsection{Time Complexity Analysis}
\vspace{-0.05in}
The proposed algorithms' computational complexities depend on the number of features ($d$) of an instance. During each online learning step, element-wise operations are involved between $M_t\in\R^{d\times d}$ and $G_t\in\R^{d\times d}$, so the time complexity of SORS is $O(d^2)$ for each learning iteration. AdaSORS needs to deal with matrix $\Sigma_t$ and $H_t$, it consumes double time than SORS: $O(2d^2)$. If the instances are sparse, then generally we can compute and update the matrices $G$, $M$, $\Sigma$, and $H$ more efficiently.

\vspace{-0.05in}
\subsection{Theoretical Analysis}
\vspace{-0.05in}
\label{analysis}
In this subsection, we will provide theoretical upper bounds for the regret of the proposed algorithms defined as follow,
\begin{eqnarray*}
R_T=\sum^T_{t=1}[\ell_t(M_t)+\lambda r(M_t)]-  \sum^T_{t=1}[\ell_t(M_*)+\lambda r(M_*)]
\end{eqnarray*}
where $M_*=\arg\min_M \sum^T_{t=1}[\ell_t(M)+\lambda r(M)]$. This regret is different from the definition in the Eq.~\eqref{eqn:regret}, since this regret also considers the sparsity regularization term. The bound on the regret implies the gap between the cumulative loss of online learning algorithm and the cumulative loss of the best model which can be chosen in hindsight.

Now, we will analyze the theoretical performance of the proposed algorithms for the case that $\ell_t$ is Lipschitz continuous e.g., $\ell_t(M)=[1-\x_t^\top M\x_t^+ +\x_t^\top M\x_t^-]_+$.
\begin{thm}
Let $\{(\x_t,\x_t^+,\x_t^-)\ |\ t=1,\ldots,T \}$ be a sequence of triplets, where $\x_t,\x_t^+,\x_t^-\in\R^d$. Suppose $\ell_t(M)$ is  convex over $\R^{d\times d}$ and $L$-Lipschitz for any $t\in[T]$, if the SORS algorithm is run on this sequence of triplets with $\eta_t=\eta$, then we have the following regret bound,
\begin{small}
\bqs
R_T\le \frac{1}{2\eta}\|M_*\|_F^2 + \frac{\eta}{2} L^2T
\eqs
\end{small}
Furthermore, if we set $\eta=\|M_*\|/(L\sqrt{T})$, then for all $T>0$, we have
\begin{small}
\bqs
R_T\le\|M_*\|_F L\sqrt{T}
\eqs
\end{small}
\end{thm}

{\bf Remark}: This theorem implies the SORS algorithms will suffer a regret of order $O(\sqrt{T})$ and an average of regret of order $O(1/\sqrt{T})$. If $T$ approaches infinity, the average regret vanishes.

\begin{thm}
Let $\{(\x_t,\x_t^+,\x_t^-)\ |\ t=1,\ldots,T \}$ be a sequence of triplets, where $\x_t,\x_t^+,\x_t^-\in\R^d$. Suppose $\ell_t(M)$ is convex over $\R^{d\times d}$ and the gradient of $\ell_t$ with respect $M_t$ is $\nabla_M\ell_t(M_t)=G_t$, if the AdaSORS algorithm is run on this sequence of triplets, then we have the following regret bound
\bqs
R_T\le\frac{1}{2\eta}D_M^2\sum^d_{i=1}\sum^d_{j=1}\|G_{1:t,ij}\|_2+\eta\sum^d_{i=1}\sum^d_{j=1}\|G_{1:t,ij}\|_2
\eqs
where $D_M=\max_t\|M_*-M_t\|_\infty$ and $\|G_{1:t,ij}\|_2=\sqrt{\sum^T_{t=1}G_{t,ij}^2}$.

Furthermore, if $\eta$ is set as $D_\infty/\sqrt{2}$, we have
\bqs
R_T\le\sqrt{2}D_\infty\sum^d_{i=1}\sum^d_{j=1}\|G_{1:t,ij}\|_2
\eqs
\end{thm}
{\bf Remark:} According to~\cite{jmlrDuchiHS11}, this regret is in the same order of the regret by choosing the best fixed $\Sigma$ for the Eq.~\eqref{eqn:so-goal}.

\section{Experiments}
\label{sec:exp}
In this section, we evaluate the proposed approaches on six public high dimensional benchmark datasets to examine their effectiveness.

\vspace{-0.05in}
\subsection{Compared Algorithms}
\vspace{-0.05in}
We compare the following approaches:
\begin{itemize}
  \item \textbf{Euclidean}: The baseline measurement method using the standard Euclidean distance in feature space
  \item \textbf{LMNN}: \emph{Largest Margin Nearest Neighbor} method proposed by~\cite{LMNNnipsWeinbergerBS05}.
  \item \textbf{ITML}: \emph{Information Theoretic Metric Learning} proposed by~\cite{ITMLdavis2007information}.
  \item \textbf{LEGO}: \emph{LogDet Exact Gradient Online} Learning proposed by~\cite{LEGOjain2009online}.
  \item \textbf{RDML}: \emph{Regularized Distance Metric Learning} proposed by~\cite{RDMLjin2009regularized}.
  \item \textbf{AROMA}: \emph{Adaptive Regularization for Weight Matrices} proposed by~\cite{crammer2012adaptive}.
  \item \textbf{OASIS}: It learns a bilinear similarity based on online Passive Aggressive algorithm using triplet instances~\cite{OASISChechik2010}.
  \item \textbf{SORS}: The algorithm described above in Algorithm~\ref{alg:SORS}. \textbf{SORS-I} is the algorithm adopted the  Eq.~\eqref{eqn:SORSrank1} as $\prox$ function in Eq.~\eqref{eqn:SORSproxfunc}. \textbf{SORS-II} is the one adopted the Eq.~\eqref{eqn:SORSrank2}.
  \item \textbf{AdaSORS}: The algorithm described above in Algorithm~\ref{alg:AdaSORS}. \textbf{AdaSORS-I} is the algorithm adopted the  Eq.~\eqref{eqn:AdaSORSrank1} as $\prox$ function in Eq.~\eqref{eqn:AdaSORSproxfunc}. \textbf{AdaSORS-II} is the one adopted the Eq.~\eqref{eqn:AdaSORSrank2}.

\end{itemize}

\vspace{-0.05in}
\subsection{Experimental Datasets and Setup}
\vspace{-0.05in}
To examine the performance, we test all the algorithms on six publicly available high dimensional datasets ``Caltech256''\footnote{\url{http://www.vision.caltech.edu/Image_Datasets/Caltech256}}, ``BBC''\footnote{\url{http://mlg.ucd.ie/datasets/bbc.html}}, ``Gisette''\footnote{\url{http://archive.ics.uci.edu/ml/datasets.html}}, ``Protein'', ``RCV1'', and ``Sector'' from LIBSVM\footnote{\url{http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets}}, as shown in Table~\ref{tb:hd-datasets}. The sparsity in Table~\ref{tb:hd-datasets} means the percentage of empty attribute values.

\begin{table}[!ht]
\vspace{-0.0in}
\centering
\caption{Statistics of Experimental Datasets.\label{tb:hd-datasets}}
\vspace{-0.0in}
\begin{tabular}{|c|c|c|r|r|c|}
\hline
Data Set & Source & Class & \# Feature & \# Size & Sparsity (\%) \\ \hline \hline
Protein & LIBSVM & 3 & 357 & 17,766 & 71.00 \\
Caltech256  & Caltech & 257 & 1,000 & 29,461 & 96.75 \\
Gisette & UCI & 2 & 5,000 & 6,000 & 00.91 \\
BBC  & UCD & 5 & 9,636 & 2,225 & 98.66\\
RCV1 & LIBSVM & 53 & 47,236 & 8,967 & 99.86 \\
Sector & LIBSVM & 105 & 55,197 &  9,619 & 99.70 \\
\hline
\end{tabular}
\vspace{-0.1in}
\end{table}

Prediction for the structure and function of proteins plays an important role in bioinformatics. Protein dataset is used to predict the local conformation of the polypeptide chain~\cite{bibYang04}. This dataset comprises 17,766 protein entries with 3 categories. Each protein entry is represented by 357 features.

Caltech256 is a dataset for image retrieval tasks, which consists of 30,607 images assigned to 257 categories, where each image is represented by 1,000 features. We use the images from the first 10 and 50 classes to construct experimental datasets, which are denoted as ``Caltech10'' and ``Caltech50''.

Gisette is a dataset used for a handwritten recognition problem, which is to separate the confusable digits handwritten number '4' and '9'~\cite{confnipsGuyonGBD04}. The Gisette dataset is released from NIPS 2003 Feature Selection Challenge, which contains 6,000 entries. Each entry is represented by 5,000 features. From Table~\ref{tb:hd-datasets}, we can see that the entry in Gisette dataset is dense.

BBC news article dataset is gathered from BBC news website, which corresponds to stories in five topical areas from 2004-2005~\cite{icmlGreeneC06}. The dataset consists of 2,225 documents in five areas: business, entertainment, politics, sport and tech. Each document is represented by 9,636 features.

The Reuters RCV1 is a text classification dataset~\cite{jmlrLewisYRL04}. This dataset contains documents originally written in five different languages over a common set of 6 categories. Each RCV1 entry is represented by 47,236 features. We use the first 10 categories as our study dataset.

\begin{table*}[!htpb]
\caption{Evaluation of the Learning Algorithms on the Data Sets.}\label{tab:performance}
\renewcommand*\arraystretch{1.0}
\begin{center}
\begin{small}
\begin{tabular}{|l|c|c|c|c|c|c|c|}        \hline
\multirow{1}{*}{ Algorithm}& \multicolumn{3}{c|}{  Protein (\#Class=3, \#Iterations=$10^5$)} & & \multicolumn{3}{c|}{  Caltech50 (\#Class=50, \#Iterations=$10^5$) }\\
\cline{2-4} \cline{6-8}
 &   MAP (\%)         &   Sparsity (\%) & ~Time (s)~ &&    MAP (\%)         &   Sparsity (\%) & ~Time (s)~
\\\hline\hline
Eucl.	&	36.93$\pm$0.01	&	99.9$\pm$0.00	&	0	&	&	10.05$\pm$0.21	&	99.9$\pm$0.00	&	0	\\\hline
LMNN	&	45.91$\pm$0.01	&	0.56$\pm$0.01	&	78088.84$\pm$219.14	&	&	8.07$\pm$1.10 &	5.91$\pm$0.01	&	106884.06$\pm$894.25	\\
ITML	&	37.13$\pm$0.01	&	0.56$\pm$0.01	&	50.00$\pm$0.01	&	&	10.29$\pm$0.20	&	6.83$\pm$0.57	&	1124.88$\pm$26.01	\\
LEGO	&	37.44$\pm$0.01	&	0.56$\pm$0.01	&	60.00$\pm$0.01	&	&	10.15$\pm$0.01	&	5.52$\pm$0.01	&	531.94$\pm$4.62	\\
RDML	&	38.96$\pm$0.01	&	0.58$\pm$0.01	&	108.90$\pm$8.20	&	&	10.10$\pm$0.22	&	7.50$\pm$0.10	&	623.78$\pm$12.13	\\
OASIS	&	43.91$\pm$0.16	&	0.56$\pm$0.01	&	86.51$\pm$15.18	&	&	12.93$\pm$0.20	&	5.94$\pm$0.10	&	673.43$\pm$89.74	\\
AROMA	&	44.58$\pm$0.22	&	0.56$\pm$0.01	&	822.93$\pm$71.38	&	&	13.39$\pm$0.10	&	5.93$\pm$0.12	&	1294.66$\pm$147.32	\\
SORS-I	&	45.66$\pm$0.16	&	1.02$\pm$0.04	&	217.83$\pm$0.54	&	&	13.28$\pm$0.14	&	13.82$\pm$0.14	&	895.79$\pm$27.87	\\
SORS-II	&	45.69$\pm$0.06	&	0.81$\pm$0.03	&	239.62$\pm$1.18	&	&	13.29$\pm$0.18	&	13.83$\pm$0.18	&	958.97$\pm$20.19	\\
AdaSORS-I	&	47.02$\pm$0.41	&	0.97$\pm$0.05	&	400.11$\pm$9.21	&	&	13.82$\pm$0.14	&	14.10$\pm$1.21	&	2552.17$\pm$169.52	\\
AdaSORS-II	&	46.45$\pm$0.39	&	0.99$\pm$0.04	&	423.27$\pm$10.91	&	&	13.83$\pm$0.15	&	14.10$\pm$1.26	&	2993.64$\pm$282.13	\\
\hline\hline
\multirow{1}{*}{ Algorithm}& \multicolumn{3}{c|}{ Gisette (\#Class=2, \#Iterations=$10^5$)} & & \multicolumn{3}{c|}{  BBC (\#Class=5, \#Iterations=$10^5$) }\\
\cline{2-4} \cline{6-8}
 &   MAP (\%)         &   Sparsity (\%) & ~Time (s)~ &&    MAP (\%)         &   Sparsity (\%) & ~Time (s)~
\\\hline\hline
Eucl.	&	62.05$\pm$0.01	&	99.9$\pm$0.00	&	0		&	&	32.91$\pm$0.01	&	99.9$\pm$0.00	&	0	\\\hline
LMNN	&	87.01$\pm$0.24	&	1.75$\pm$0.01	&	78994.68$\pm$141.21		&	&NA	&NA &	NA				\\
ITML	&	62.09$\pm$0.32	&	28.64$\pm$0.01	&	750.82$\pm$23.12		&	&	31.78$\pm$0.47	&	10.86$\pm$1.43	&	109687.28$\pm$633.54	\\
LEGO	&	78.38$\pm$0.36	&	6.45$\pm$0.01	&	5410.54$\pm$30.94	&	&	30.29$\pm$0.01	&	3.44$\pm$0.26	&	31827.25$\pm$59.46	\\
RDML	&	82.49$\pm$0.32	&	20.73$\pm$0.01	&	2090.51$\pm$7.88		&	&	70.79$\pm$2.35	&	83.75$\pm$0.01	&	8189.38$\pm$188.18	\\
OASIS	&	83.49$\pm$0.22	&	4.72$\pm$0.01	&	2316.77$\pm$469.77		&	&	79.36$\pm$0.41	&	61.62$\pm$0.24	&	8881.26$\pm$475.22	\\
AROMA	&	83.57$\pm$1.82	&	3.89$\pm$0.18	&	26097.59$\pm$27.40		&	&	79.68$\pm$0.43	&	60.83$\pm$1.23	&	9723.02$\pm$381.83	\\
SORS-I	&	88.89$\pm$2.49	&	4.01$\pm$1.62	&	16839.32$\pm$101.85		&	&	91.03$\pm$1.12	&	78.67$\pm$3.56	&	90370.71$\pm$519.35	\\
SORS-II	&	89.89$\pm$1.22	&	4.08$\pm$1.61	&	22578.96$\pm$1028.02		&	&	92.20$\pm$1.28	&	78.41$\pm$3.21	&	135376.85$\pm$678.71	\\
AdaSORS-I	&	90.92$\pm$0.71	&	4.40$\pm$1.10	&	31256.19$\pm$1206.14		&	&	92.43$\pm$0.62	&	82.15$\pm$2.68	&	303718.57$\pm$643.43	\\
AdaSORS-II	&	90.99$\pm$0.53	&	4.20$\pm$1.12	&	35471.63$\pm$848.63		&	&	94.09$\pm$0.61	&	81.24$\pm$2.35	&	364976.73$\pm$1485.68	\\
\hline\hline
\multirow{1}{*}{ Algorithm}& \multicolumn{3}{c|}{ RCV1 (\#Class=10, \#Iterations=$10^5$) } & & \multicolumn{3}{c|}{ Sector (\#Class=105, \#Iterations=$10^5$) }\\
\cline{2-4} \cline{6-8}
 &   MAP (\%)         &   Sparsity (\%) & ~Time (s)~ &&    MAP (\%)         &   Sparsity (\%) & ~Time (s)~
\\\hline\hline
Eucl.	&	58.28$\pm$0.01	&	99.9$\pm$0.00	&	0	&	&	19.40$\pm$0.01	&	1$\pm$0.00	&	0	\\\hline
OASIS	&	92.37$\pm$0.01	&	97.14$\pm$0.01	&	30222.26$\pm$792.28	&	&	65.18$\pm$0.01	&	86.98$\pm$0.01	&	196561.51$\pm$1862.72	\\
AROMA	&	92.54$\pm$0.01	&	97.11$\pm$0.01	&	40407.61$\pm$149.85	&	&	65.47$\pm$0.01	&	87.91$\pm$0.01	&	275107.98$\pm$4295.58	\\
SORS-I	&	92.62$\pm$0.01	&	99.84$\pm$0.01	&	71473.62$\pm$1370.12	&	&	66.22$\pm$1.02	&	98.12$\pm$0.01	&	311849.83$\pm$3976.32	\\
SORS-II	&	92.62$\pm$0.01	&	99.84$\pm$0.01	&	100543.28$\pm$4981.78	&	&	66.89$\pm$1.02	&	98.12$\pm$0.01	&	381573.25$\pm$10874.97	\\
AdaSORS-I	&	92.82$\pm$0.01	&	99.64$\pm$0.01	&	148260.37$\pm$3411.35 &	&	67.18$\pm$0.01	&	98.20$\pm$0.01	&	632147.31$\pm$21041.49	\\
AdaSORS-II	&	92.86$\pm$0.01	&	99.64$\pm$0.01	&	162515.32$\pm$9431.53	&	&	67.10$\pm$0.01	&	98.20$\pm$0.01	&	687947.24$\pm$15972.15	\\
\hline
\end{tabular}
\end{small}
\end{center}
\vspace{-0.0in}
\end{table*}

Sector dataset is used for text categorization~\cite{mccallum1998comparison}. The scaled data consists of company web pages classified in a hierarchy of industry sectors. It is collected from web sites belonging to industry companies from various economic sectors. The whole dataset contains 9,619 entries. Each Sector entry is represented by 55,197 features.

For each dataset, data instances from each class are randomly split into training set (70\%) and test set (30\%). To generate a triplet $(\x_t,\x_t^+,\x_t^-)$, $\x_t$ is firstly randomly selected from the whole training set, then $\x_t^+$ is randomly selected from the subset of training set, which consists of the examples with the same class of $\x_t$, finally $\x_t^-$ is randomly selected from the rest of training set, which consists of the examples with different classes of $\x_t$.

To make a fair comparison, all algorithms adopt the same experimental setup. We use cross-validation to select the values of hyper parameters for all algorithms. Specifically, the parameters set by cross validation include: the $\mu$ parameter for LMNN ($\mu\in$\{0.1,0.2,...0.9\}), the $\gamma$ parameter for ITML ($\gamma\in$\{0.1,1,10\}), the regularization parameter $\eta$ for LEGO ($\eta\in$\{0.02,0.08,0.32\}), the $\lambda$ parameter for RDML ($\lambda\in$\{0.01,0.1\}), the scalar parameter $r$ for AROMA ($r\in$\{0.01,0.1,1,10,20...100\}) and the aggressiveness parameter $C$ for OASIS ($C\in$\{0.1,0.08,0.06,0.04...\}). All the evaluation results listed below are achieved by choosing  the parameters using cross validation.

\vspace{-0.05in}
\subsection{Evaluation Measures}
\vspace{-0.05in}
We evaluate the performance of the algorithms, by the sparsity of the model, the precision at top $k$ retrieval results, the mean average precision of retrieval results, and the running time.
\begin{itemize}
  \item The sparsity of a matrix $M$ is defined as Sparsity$= 1-\frac{\|M\|_0}{d^2}$, which is controlled by $\lambda$ in Eq.~\eqref{eqn:goal} and Eq.~\eqref{eqn:so-goal}.
  \item Precision at $top$-$k$  corresponds to the number of relevant results on the first $k$ query results.
  \item The \emph{mean average precision} (MAP) is used to evaluate the accuracy of the retrieval performance.
  \item The training time is also reported for each algorithm to show the computational efficiency.
\end{itemize}

\vspace{-0.05in}
\subsection{Performance Evaluation}
\vspace{-0.05in}
Table~\ref{tab:performance} summarizes the performance of all the compared similarity/metric learning algorithms over all the benchmark datasets, where results for LMNN on ``BBC" ``RCV1" and ``Sector" are not reported since the learning task can not be finished in 168 hours. From the results, we can draw several observations.

Firstly, the sparsity of Euclidean metric is very high, however its MAP performance  is much lower than those of other algorithms for most of cases, which implies the importance of metric learning for improving the information retrieval performance.

Secondly, among the baseline algorithms in comparison, we observe that the offline algorithm LMNN is generally significantly slower than the other online learning algorithms including ITML, LEGO, RDML, and OASIS, although it generally achieves better MAP than the online learning algorithms. This verifies the efficiency and scalability of online learning, which makes them more attractive for large-scale tasks and mining data streams.

Thirdly, among the online learning algorithms in comparison, we observe that the online similarity learning algorithms (OASIS and AROMA) generally achieve better or at least comparable MAP performance compared with online metric learning algorithms (ITML, LEGO, RDML), which indicates the effectiveness of similarity learning techniques, especially that learning similarity function without PSD constraint is effective. Compared with OASIS, AROMA generally achieves better MAP performance on all the datasets, which verifies the effectiveness of introducing second order information to improve the learning efficacy

Finally, comparing with all the baseline algorithms,  the proposed SORS algorithms (SORS, AdaSORS) achieve the highest test MAP performance on all the datasets. This shows that the proposed learning strategy is effective in improving the generalization ability of the learnt model. Secondly, by examining the sparsity of resulting matrix, we observe that SORS algorithms result in the sparsest model than the other baseline algorithms for most of the datasets (Although ITML produces sparser model for Gisette and RDML produces sparser models for Gisetter and BBC, their MAP performance is significantly worse). This implies that the proposed strategy can effectively improve the sparsity of the modal. In addition, compared with SORS-I and SORS-II, AdaSORS algorithms (AdaSORS-I and AdaSORS-II) generally achieve better or comparable MAP, and produce comparably or sparser models.

Moreover, Figure~\ref{fig:MAP-iteration} shows the MAP performance of all the online similarity learning algorithms in comparison over trials. The parameters for these algorithms are selected by cross validation. Specifically, on the dataset ``Protein", the parameters are set as: $C=0.01$ for OASIS, $r=60$ for AROAM,  $\lambda=1.e-6$, $\eta=0.1$ for SORS-I \& SORS-II, and $\lambda=1.e-4$, $\eta=0.1$, $\delta=5$ for AdaSORS-I \& AdaSORS-II . On the dataset ``Caltech50'', the parameters are set as: $C=0.06$ for OASIS, $r=13$ for AROMA, $\lambda=1.e-6$, $\eta=0.1$ for  SORS-I \& SORS-II, and $\lambda=1.e-6$, $\eta=0.1$, $\delta=0.1$ for AdaSORS-I \& AdaSORS-II. On the dataset ``Gisette'', the parameters are set as $C=0.01$ for OASIS,  $r=90$ for AROMA, $\lambda=1.e-6$, $\eta=0.06$ for SORS-I \& SORS-II, and $\lambda=1.e-5$, $\eta=0.06$, $\delta=20$ for AdaSORS-I \& AdaSORS-II. On the dataset ``BBC'', the parameters are set as: $C=0.01$ for OASIS, $r=90$ for AROMA, $\lambda=1.e-6$, $\eta=0.1$ for SORS-I \& SORS-II, and $\lambda=1.e-6$, $\eta=0.1$, $\delta=0.1$ for AdaSORS-I \& AdaSORS-II. If not stated, the parameters for the rest of the figures are set as the same as here.

\begin{figure}[!htpb]
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.55in]{fig_protein_map_iter.eps}
\makebox[2.5in]{(a) Protein}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.55in]{fig_caltech50_map_iter.eps}
\makebox[2.5in]{(b) Caltech50}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.55in]{fig_gisette_map_iter.eps}
\makebox[2.5in]{(c) Gisette}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.55in]{fig_bbc_map_iter.eps}
\makebox[2.5in]{(d) BBC}
\end{subfigure}
\caption{MAP of online similarity learning algorithms as a function of the number of training iteration on four data sets.}\label{fig:MAP-iteration}
\end{figure}

\begin{figure}[!htpb]
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_protein_prec.eps}
\makebox[2.5in]{(a) Protein}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_caltech50_prec.eps}
\makebox[2.5in]{(b) Caltech50}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_gisette_prec.eps}
\makebox[2.5in]{(c) Gisette}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_bbc_prec.eps}
\makebox[2.5in]{(d) BBC}
\end{subfigure}
\caption{Precision$@$k of the online similarity learning algorithms on four data sets.}\label{fig:Prec_k}
\end{figure}

\begin{figure}[!htpb]
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_protein_sparsity_iter.eps}
\makebox[2.5in]{(a) Protein}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_caltech50_sparsity_iter.eps}
\makebox[2.5in]{(b) Caltech50}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_gisette_sparsity_iter.eps}
\makebox[2.5in]{(c) Gisette}
\end{subfigure}
\centering
\begin{subfigure}
\centering
\includegraphics[width=2.65in]{fig_bbc_sparsity_iter.eps}
\makebox[2.5in]{(d) BBC}
\end{subfigure}
\caption{Online sparsity of the online similarity learning algorithms on four data sets.}\label{fig:sparsity-iteration}
\end{figure}

From Figure~\ref{fig:MAP-iteration}, we can observe that generally SORS algorithms outperform OASIS and AROMA, which indicates the effectiveness of introduction of sparsity. In addition, AdaSORS algorithms generally can achieve the best MAP performance, which verifies the effectiveness of using second order information for improving learning efficacy. Overall, these observations are consistent with  Table~\ref{tab:performance}, which again verifies that the introduction of sparse regularization can efficiently remove noisy interactions between features and improve the generalization ability of the learning algorithms.

To further demonstrate the effectiveness of the proposed algorithms, Figure~\ref{fig:Prec_k} summarizes the precision$@k$  performance of the online similarity learning algorithms: OASIS, AORMA, SORS-I, SORS-II, AdaSORS-I and AdaSORS-II on four datasets. It can be  observed that AdaSORS and SORS algorithms overall outperform OASIS and AROMA on most of $k$ (except $k$=1). The reason for this is the fact SORS and AdaSORS algorithms are designed to optimize the AUC performance, which may not guarantee the best precision at top $k$. However, the proposed algorithms still achieve better precision at top $k$ for most of cases. All these observations verify the effectiveness of the proposed algorithms.

Moreover, we also estimate the online sparsity of the models produced by online similarity learning algorithms with respect to iterations on four datasets, illustrated in Figure~\ref{fig:sparsity-iteration}. We can observe the sparsity generally decreases as the number of iteration increases  for all the algorithms, which is due to the fact more and more rank-1 matrices are added to the model. In addition, AdaSORS and SORS algorithms generally achieve much higher sparsity than AROMA and OASIS (except for Gisette, where all the algorithms achieve comparable sparsity) and AdaSORS algorithms overall produce sparser models than SORS, which again verifies the effectiveness of the sparse regularization to reduce memory cost.

\vspace{-0.05in}
\subsection{Testing Efficiency}
\vspace{-0.05in}
To evaluate the testing efficiency, Figure~\ref{fig:runtime} summarizes the query time of the models learnt using OASIS, AROMA, SORS-I, SORS-II, AdaSORS-I, and AdaSORS-II on ``BBC'' for test datasets with varied sizes, for which the sparsity of these models are already shown in the Table~\ref{tab:performance}.
\begin{figure}[!htpb]
\centering
\includegraphics[width=3.1in]{fig_bbc_runtime.eps}
\label{fig:bbc_runtime}
\caption{Comparison of the testing efficiencies of the models from 6 online similarity learning algorithms on ``BBC".}\label{fig:runtime}
\end{figure}

We can observe that AdaSORS and SORS algorithms generally are more efficient than OASIS and AROMA over all the test sets of different sizes, since the models of these four algorithms are much sparser than OASIS and AROMA. In addition, when the number of queries is small, the time cost of non-sparse algorithms (OASIS, AROMA) and sparse algorithms (SORS-I, SORS-II, AdaSORS-I, AdaSROS-II) are not much different; however if the number of queries is very large, sparse algorithms generally are significantly efficient than non-sparse algorithms, e.g., AdaSORS-II method saves 50\% of test time when the size of data is 50K. This makes the proposed algorithms more attractive for large-scale high dimensional real-world applications.

\vspace{-0.05in}
\subsection{Sparsity-MAP Tradeoffs}
\vspace{-0.05in}
Finally, we evaluate the relationship between the MAP performance and the sparsity of the models of the online similarity learning algorithms on ``Caltech10'', which is demonstrated in Figure~\ref{fig:caltech10_map_sparsity_sors}, where the sparsity and MAP of OASIS are constant values.
\begin{figure}[!htpb]
\vspace{-0.1in}
  \centering
  \includegraphics[width=3.1in]{fig_caltech10_map_spa.eps}
  \caption{Tradeoff between MAP and Sparsity of models of online similarity learning algorithms on ``Caltech10'' dataset for 100K iterations.}\label{fig:caltech10_map_sparsity_sors}
\vspace{-0.1in}
\end{figure}

We can observe that when the sparsity is small enough (less than 0.5), the MAPs performance of the proposed SORS and AdaSORS algorithms tend to be highest and stable, while when the sparsity is larger, increasing the sparsity generally leads to rapid deceasing of MAPs. However, the proposed SORS and AdaSORS algorithms allow us to choose a much sparser model, e.g. with around $15$ percentage sparsity, with  better or at least comparable MAP performance compared with OASIS and AROMA. In addition, when the sparsity is extremely high, SORS-II and AdaSORS-II can still achieve much better MAP performance than SORS-I and AdaSORS-I respectively, which indicates the importance of those diagonal elements for learning an effective similarity function.

\section{Conclusion}
\label{sec:con}
This paper presented a \emph{Sparse Online Relative Similarity Learning} (SORS) framework, and four scalable algorithms for similarity learning tasks with high dimensional features. This framework would like to learn a sparse model which can assign similar items with higher similar values.  We theoretically analyze the regret bounds of the proposed algorithms, and conduct a set of experiments by comparing with a number of competing algorithms. Promising empirical results shows that the proposed algorithms can effectively learn sparser model with better or at least comparable MAP performance.

\section*{Acknowledgment}
The work is partially supported by NSFC (No.61472149, No.71401128), the Fundamental Research Funds for the Central Universities (2015QN67), and the SRF for ROCS, SEM. The authors would like to thank Zhenzhong Lan for the fruitful comments on this work.

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


All CANYON TRAILS UNIT 3 PARC homes currently listed for sale in Goodyear as of 04/26/2019 are shown below. You can change the search criteria at any time by pressing the 'Change Search' button below.
"Gorgeous 6 bedroom home available in highly desirable Goodyear, close to the I-10 and multiple shopping. Flowing floorplan is perfect for entertaining, with over 4000 sq ft, a spacious formal living room with soaring ceilings and a separate family area with tile flooring and built-in shelving. Stylish eat in kitchen boasts stainless appliances, granite countertops, an island/breakfast bar and warm, wood cabinetry & pantry. Bed and office with full bath downstairs. Huge master suite includes his"
"WONDERFUL 4 BEDROOM FAMILY HOME WITH LOFT AND 2ND FLOOR SITTING ROOM. DRAMATIC WOOD BURNING LR FIREPLACE. 11 FANS, CHERRY CABINETS AND GRANITE TOPS, STAINLESS KITCHEN APPLIANCES, FRONT LOADING SAMSUNG WASHER AND DRYER WITH PEDESTALS. MAGIC TOUCH KITCHEN FAUCET. MASTER SUITE WITH BALCONY AND EN-SUITE BATH WITH WALK-IN SHOWER, SEPARATE TUB AND WALK-IN CLOSET. BLOCK WALLED BACK YARD WITH SAFETY FENCE AROUND THE POOL WITH A WATERFALL. GRASS PLAY AREA WITH CEMENT CABANA/STORAGE BLDG. PAD COVER"
"Fantastic 4 bed, 2.5 bath home situated in a huge pie-shaped lot! Boasting formal living & dining rooms, spacious loft that easily could be converted as 5th & 6th bedrooms, vaulted ceilings, and cozy family room. Gourmet kitchen has matching appliances, center island for prep & breakfast bar, pantry, and cabinet/counter space. Lovely master bedroom offers full bath with dual sinks, step-in shower, separate tub, and walk-in closet. Property has Solar Panels providing HUGE savings! Bring your ima"
"This incredible home with NEW CARPET, has a chef kitchen with complete granite counter top, an island, gas cook top & cherry cabinets. You will love the two- story living and dinning space, which alternatively can become a game room or party central. One bedroom down for guest and 3 spacious BRs up. New paint and New stove! Hugh lot and RV gate on the side yard for storing space, trail, boat or you can turn backyard into swimming pool and play ground. Walking distance to Copper Trail Elem., Fr"
"MOVE-IN READY!!! Exquisite home built in 2016 with a split floor plan and tile in all the main living areas. The gourmet kitchen features dual ovens, stainless steel appliances, and beautiful quartz countertops. The kitchen opens up into a spacious dining and living room perfect for entertaining guests. The home also has an extra room which can be used for a home office. Master bath features dual sinks, separate shower/tub, and a walk-in closet. This home features a fully landscaped lot tha"
"This beautiful 4 bedroom home is model perfect! The amazing curb appeal is brought out with the stone facade and gated front entry. Once inside, this gorgeous home boasts a neutral color palette, tile flooring throughout with newer carpet in the bedrooms, a large guest bath with dual sinks, and an over-sized laundry room. The beautiful great room floor plan is open and bright, with a kitchen that's designed with the gourmet chef in mind! Rich espresso cabinets, beautiful granite counter-tops, ba"
"Amazing Home built 2016...This home boosts Great Room Concept with alcove for office, den or has many uses.There is a bedroom and full bath downstairs. Upstairs has Loft, 3 Bedrooms, 2 Baths, laundry room. Designer features include built-in stainless steel appliances, painted linen cabinets with 4 1/4'' crown molding and Rockside Brown granite counters. Custom 20x20 tile flooring, two-tone interior paint, 8' interior doors. Backyard is perfect for pool setup with RV gate available."
"This Beautiful two story home has an amazing living and dinning space perfect for party central or play room. New updated paint and carpet through out the entire home. One bedroom down for guest and 3 BRs upstairs. You will love the chef kitchen with complete granite counter top and cherry cabinet. Fully paved spacious and less maintenance backyard. Walking distance to Copper Trail Elem. Easy access to loop 303 & I-10, shopping and entertainment!"
"Grab your swimsuits and head on over to view this beautiful home!! Highly sought after 5 bedrooms and 3.5 baths!! Lot's of space in this house for everyone to enjoy some alone time **bedroom and full bath downstairs** Some features to this beauty are... diagonal tile, staggered cabinets, r/o system, water softener loop, utility sink in garage just to name a few!! Out side you will find your own private oasis with pool and waterfall and built in grill area for all your entertaining! Call for"
"This home is a show stopper! Saying it is updated and elegant throughout is an understatement. You have an entertainers dream setup with a kitchen/great room concept. The kitchen has a walk-in pantry, double ovens and a built-in electric cook-top. The island has ample room for chairs and a separate area perfect for having a more formal sit down meal. The large master bedroom gives you all the feels with an over sized walk-in closet and separate tub/shower with dual sinks in the master bath. The "
"Now this is a fantastic corner lot home you will fall in love with. The first thing you'll notice is the fantastic curb appeal of the home. Beautiful tile flooring through the main walking areas. Simply a fantastic kitchen with breakfast bar, double wall ovens, pantry, island, granite counter tops, and upgraded cabinetry. Separate living & family rooms. Back yard has covered patio and easy to maintain landscaping. Great open floor plan and an extremely well maintained property. Hurry and com"
"This beauty is located on a premium corner lot and has been well taken care of - barely lived in used as 2nd home during the winter months ! 3 bedroom plus double door den - 2 full baths with 2 car garage and great curb appeal - tiled formal living and dining room upon entry - open kitchen to family/great room concept - nice sized kitchen island - pantry - eatin tiled kitchen -stainless steel appliances -master suite offers sliding glass doors - full bath includes separate tub and shower - priva"
"Come see this beautiful T.W. Lewis 4 bed, 2 bath home in Goodyear! Featuring low maintenance front yard, 2 car garage, and charming curb appeal. This property has everything you are looking for! Step inside to discover neutral colors, living/dining room, and a great room style family area with an open kitchen that is equipped with stainless steel appliances and ample cabinetry. Master suite has freshly cleaned carpet, and a full bath with separate tub & shower, as well as a walk-in closet. . Nic"
"Beautiful two story home now available to one lucky buyer! Fantastic front porch ideal for relax at morning or sunset. Gorgeous interior with living & dining areas, neutral paint, tons of natural light, 4 bed, 2.5 bath, and ample loft. Wonderful kitchen equipped with breakfast bar, white appliances, pantry, and plenty of cabinet/counter space. Lovely master suite with fabulous bath and spacious walk-in closet. Large desert style backyard comprised of covered patio and ramada perfect for barbecue"


text:
The Best Flags on the Planet. Guaranteed.
Join the Patriot Club
Get Discounts, Prizes and More!
YES, I WANT IN!
State flag
Military flag
Military Branch History Part 1: The Army
Comprised of a variety of different branches, the United States military has been one of the most storied and influential groups in the history of the world.
Dating back for centuries now, the U.S. Army was the first organized military out of America and has stood the test of time as one of the most important organizations around today.
From the Revolutionary War to the 21st century war on terror, the Army has helped shaped America and the rest of the world into what it is today.
First founded in 1747, a group of American soldiers were first formed as "His Majesty's First Independent Company of American Rangers."
While this unit did in fact serve England and the king, it was the first group founded in the United States and created the important stepping stones moving forward in the military.
In 1775, this Second Continental Congress founded the first true United States Army as they declared their war of Independence from the British.
That year, a ten infantry branch was created within the U.S. Army, comprised of rifleman station throughout Pennsylvania, Maryland, and Virginia.
Under the leadership of George Washington as the first Commander in Chief of the Continental Army, America would go on to win the Revolutionary War in 1783 and gain their independence from England.
Wars of the 1800s
Photo Credit: U.S. Army
While the U.S. established itself over the few decades following the Revolutionary War, the War of 1812 brought them down a couple of notches as brutal attack from England threatened to disrupt their new way of life and freedom.
The United States would come out of this war stronger however and would never face another British attack.
By the mid 1800s, slavery had begun to tear apart the Northern and Southern parts of the country, as the incredibly brutal Civil War broke out between the two sides.
During this war, new ranks and officers were created and appointed to the United States Army, as Ulysses S. Grant would become the very first General of the Army in 1864.
The Civil War also saw the first drafts happen for the military, as the Confederate Army would hold the first one in 1862 with the Union Army waiting until 1863 to hold a draft for their side.
Incredibly defining times for the entire global population, the United States Army was heavily involved in both World War I and World War II.
The Selective Service Draft was founded during the first World War in 1917 and saw roughly four million men drafted into duty.
World War II saw some of the most storied Army Generals make names for themselves, such as Eisenhower, MacArthur, and Patton. These men helped define what the Army would eventually become and led their troops to end the war with Germany.
Vietnam to Now
Modern warfare took a turn during the Vietnam War, as the brutality of this war is still felt today.
Lasting from 1965-1973, this war involved an incredible amount of Army soldiers.
Today, the war on the Middle East still rages on, as the United States Army is the most powerful and highly skilled branch involved in protecting our peace and freedom at home.
We completed the series. So you can enjoy all parts of our Military Branch History Series by going to the pages below and get our top quality US Army Flag here.
Military Branch History Series
Military Branch History Part 2: The Navy
Military Branch History Part 3: Air Force
Military Branch History Part 4: Marine Corps
Military Branch History Part 5: Coast Guard
Join thousands of others receiving our newsletter.
Expanded Mental Healthcare Options Coming for Veterans
What Soldiers Miss about Home
Filed Under: Flag Holidays
Join our newsletter for patriotic news and limited time specials.
Our flags are treated to dry fast & resist sun & chemical deterioration holding the colors strong against ultraviolet rays in the bright sunshine. The red & white stripes make good contrast with the blue background of the white embroidered stars, making the different colors pop from the fabric.
Form + Function
Our nylon flags vary in dimensions & are crafted using heavyweight 200 denier nylon making them both tough & long-lasting. Two rows of superior lock stitching is applied to the stripes, while 4 rows to the fly end of the flag to prevent unraveling & fraying thus improving its lifespan. The stars are also embroidered in densely filled thread count.
Our flags come with a MADE IN USA authentication tag sewn into the label. So you can be sure that the materials, construction, and labor are made in America only. Our flags also arrive in an attractive presentation box making them a perfect gift for friends and family!
What people say about Star Spangled Flags
"The craftsmanship is superb and the colors are vibrant, we couldn't be happier with it."
"Right down to the packaging and shipping box everything was 5 ⭐"
"Great company to deal with. They even sent an email making sure we knew about assembly and how to contact them for information. A definite A+ seller."
"Very pleased with this seller and their product. Actually sent me an email thanking me for buying this product. Arrived before delivery due date."
Premium Quality Flags 100% Made In The USA
Join the PATRIOT CLUB for new videos, discount codes, and more! Join the Patriot Club
Flag FAQs
What Makes Our Flags Different
Star Spangled Flags
For all general inquiries, please contact us here. Sitemap. State Flag FAQs.
FREE Shipping Today on all orders to the U.S.A.


text:
Political players of all party stripes gather for fond farewell to Jim Flaherty
By on April 16, 2021 Comments Off on Political players of all party stripes gather for fond farewell to Jim Flaherty
Ontario MPP Christine Elliott and her sons stand by the casket of her husband, the late former federal finance minister Jim Flaherty, during visitation in Whitby, Ont., on Tuesday, April 15, 2014. THE CANADIAN PRESS/Frank Gunn
Lee-Anne Goodman, The Canadian Press
TORONTO - A gallery of political luminaries from all sides of the partisan divide gathered at a cavernous Toronto cathedral Wednesday to set aside their differences and pay final respects to former finance minister Jim Flaherty.
Mourners donned green scarves, a tribute to Flaherty's Irish heritage, as they entered the downtown St. James Cathedral to remember Flaherty, 64, who died of a heart attack last week in his Ottawa condominium.
"What a sad time this is in the life of our country," said Prime Minister Stephen Harper, who began the proceedings with an upbeat personal tribute to his longtime political confidante.
Directly addressing Flaherty's wife Christine Elliott and the couple's triplet sons, Harper said, "We have lost a partner in politics, but you have lost a partner in life."
Liberal Leader Justin Trudeau, NDP Leader Tom Mulcair and former prime ministers Brian Mulroney, Kim Campbell and John Turner were among those on hand, as were various federal cabinet ministers, including John Baird, Peter MacKay and Chris Alexander.
Harper and his wife Laureen arrived in advance of the cortege, followed by Gov. Gen. David Johnston and his wife Sharon. Harper exchanged pleasantries with Mulroney and shook hands with Ontario Premier Kathleen Wynne as they sat down.
At one point, Mulroney and Turner, famous political rivals, were seen to exchange smiles and nods.
A host of familiar faces from the Ontario legislature — former premier Ernie Eves and Conservative Leader Tim Hudak among them — were also present.
Before the proceedings got underway, a choir assembled behind the altar and an organist filled the chamber with hymnal music as mourners awaited the arrival of the cortege carrying Flaherty's casket.
The entire federal Conservative caucus was invited, as well as all current provincial premiers and finance ministers, as well as living former prime ministers and governors general.
Large white tents were erected on the lawn of the cathedral. Across the street from the church, a phalanx of TV cameras documented the arrival of family, friends and dignitaries.
Onlookers began gathering several hours before the ceremony was scheduled to begin. One man had his patriotism on full display, wearing a white jacket adorned with a red maple leaf and clutching a Canadian flag.
The busy downtown Toronto streets surrounding the church were closed to all vehicular traffic, except for streetcars.
The ceremonial journey of the casket began in Whitby, Ont., and proceeded along Highway 401, down the Don Valley Parkway and through downtown Toronto before its scheduled arrival at the church just before 3 p.m. ET.
Flaherty's death, which came less than a month after his retirement as minister, sent shock waves through the national capital, where flags have been flying at half-mast and the Peace Tower has been bathed in green light.
Hundreds of dignitaries and citizens lined up to pay their respects Tuesday at Whitby's Abilities Centre, which caters to the disabled and able-bodied alike. Harper arrived late in the day for a private viewing.
Labour Minister Kellie Leitch, a Flaherty confidante who rushed to his condo in an attempt to revive her friend and had dinner with him on the eve of his death, also paid her respects.
Leitch was scheduled to deliver a reading at the funeral in what was expected to be one of the emotional high points of the service.
Mark Carney, the former Bank of Canada governor who now heads the Bank of England, was also on hand at the funeral, which took place under tight security just blocks from Toronto's famed financial district, a favourite Flaherty stomping ground.
Toronto Mayor Rob Ford, whose friendship with Flaherty caused the diminutive finance minister some uncomfortable moments in the media spotlight last year, was also attending. Ford and his brother, Doug, also paid their respects in Whitby on Tuesday.
Flaherty's state funeral is the first such honour since 2011, when former NDP leader Jack Layton was laid to rest. State funerals are customarily only given to current or former prime ministers, governors general, sitting cabinet ministers or members of the Royal Family.
Note to readers: This is a corrected story. A previous version mistakenly identified MPP Christine Elliott as an MP.
Political players of all party stripes gather for fond farewell to Jim Flaherty added by Greg Holden on April 16, 2021
View all posts by Greg Holden →
<\/iframe><\/div>"); } })(); var ABDSettings = { cssSelectors: '', enableIframe: "yes", enableDiv: "yes", enableJsFile: "yes", statsAjaxNonce: "7bb829a6cb", ajaxUrl: "index.html" } // Make sure ABDSettings.cssSelectors is an array... might be a string if(typeof ABDSettings.cssSelectors == 'string') { ABDSettings.cssSelectors = [ABDSettings.cssSelectors]; }


I have a Kona (edit - Acacia wood) wood tray at home. Its very light and super durable. Google it and find all sizes available.
I use a piece of leather, approximately 7" by 9". (estimating, too lazy to go get it.) It rolls up, fits right into my pipe bag.
Mine was sent to me as a materials sample from a tannery in MN, so I can't forward you to a place to get exactly the same piece. However, you could go to Distant Drums on eBay and find a piece of Buffalo hide that you like. Here's a square of Veg Tan that you could cut to your own size. They sell many different weights and colors of buffalo, as well as a few different sizes of cut pieces like the one at the link.
Very nice jason.. I'll have to consider one of the small ones.
Never thought about leather. I know it's possible to form leather into a bowl or tray, but I don't know the method. Might have to research that, as leather is easily found in my neoghborhood.
You'd be surprised what you can find at the Goodwill. Ours is full of old/antique serving trays, platters, plates, etc.
Maybe swing in to one and see if anything catches your eye.
That Montessori site has some good tray options.
Smallest ones they have. Might order both, as they're certainly affordable.
Just another plug for a leather mat: Lightweight, easy to pack, durable, classy. You don't need to form it or stiffen it if you have a flat surface you can put it on. It readily becomes a funnel for placing excess tobacco into your tin, jar, or pouch when you're done packing. I love my leather.
Ok, that's my extra .02, I'll leave you alone now.
Would a pleated paper coffee filter meet your needs?
Yeah the leather mat really is a good idear. I probably have some at home too..leftovers from making straps.
I'm a regular at the local antique/thrift stores, I've been on the lookout.
You'd be surprised what you can find at the Goodwill.
They have leather ones on ebay. The one I just grabbed has snaps on the corner so it lays flay or snaps to make a tray. $11. Easy Peasy.
Get a pouch and learn to codger scoop.
I'd really like to find a wooden or light metal tray/bowl/plate that I can take along with me on a daily basis.
You can find anything on fleabay. But wtf for??
Find a good plastic soapdish.
Shutterbugg.. to rub out flakes when I'm on the go and don't have anything else to use.
I went ahead and ordered a fiberglass one from the site Jason linked above.
I got to hang out with some American pipe makers at the pipe show in China last week. Scott Tinker was there too. I picked up a sandblasted big leafy curly maple (i think?) flake plate and a tamper. Very satisfied!
I agree with your choice. Makes sense. Price is right.
Kiel, I've never heard it referred to as a flake plate. Brilliant! And American pipes at show in China?!?
Very cool Flake Plate you got there!!
@bluegrassbrian Sorry my post didn't really help you answer your question. Contacting Scott about a tobacco dish might be the best way to get one. Unfortunately I don't have his contact info, but I'm sure he's up on some insta-face (or one of the many other sites that are censored here) social media.
@nate Yes! This is the second year for the Chengde International Pipe Show. It's kinda neat that a civilian would go through all this work to organize an "international" event. He is a very rich civilian with his own Pipe Museum and a Harley Museum, but it's cool that he's willing to go this far and set up a China show too.
+1 for the mouse pad, thanks pagan. Just picked one up at the dollar store, trimmed it a bit, rolled it up and stuffed it in my pouch. Very nice plate! That would look awesome next to my rack!
Wisdom is not a product of schooling but of the life long attempt to acquire it.
I tend to just use my handkerchief or a spare napkin.
If it fits in my pipe purse, even better.
Ok, nobody's going to bring this up ?
Ha I do often end up using a napkin but I'm trying to secure something reusable and lasting.
Davet... yeah I've got a pipe purse. So what!
I gotta admit, I am with shutterbug. Why the heck would you want to cart around more bric-a-brack? If you like to cube cut your flakes, you can easily prep them then throw them in a baggie to take with you, and if you like to rub them out, just rub them out in the palm of your hand ( ). I do it all the time with no issue.
Mine is just a small ceramic dish I got at Wal-Mart for a dollar or so. Roughly 4x6.
Ok I give up. That picture won't load now.
I have a picture on an 8.5x11 piece of paper that I laminated for home. Works great as I is really pliable and great for pouring excess back into the jar. For the truck I have a little brass change plate I found at an second hand shop that works perfectly.
I love the ritual and hardware and a tray gives me a place to perform said ritual and put it away with all implements in one place.
Tis why I like my ceramic cowboy hat (70's chip and dip tray) with the cork knocker glued in.
Can carry that baby around and it's got plenty of room for the various accoutrement. And room to dry and rub out flakes and such.
We just had some visitors from Italy and I missed out. The guy before me snagged a pretty picture on a piece of leather. He asked me what it was. It's one of the ones with snaps on the corners to pop it up into a tray. No one knew what it was and I had to tell them, and then explain why I knew that. And still he decided to keep it rather than give me the sweet pop up tobacco tray. So it goes.
I have an olive wood plate I bought in Italy. But that can't compete with Kiel's flake plate, really beautiful !
I bought a small square plate from Target for under $2. For on the go I just folded up a piece of paper to use in a pinch.
I use a plate that I turned from a ceader fence slat to dry and rub out my Flakes. I can also put it in my pipe bag when I go out. Actually I have 3 of these. While turning them each is a bit different.
Do a search for leather valet or serving trays. You should be able to find something that fit the bill.
I also know someone who works at a high school and he hooks me up with small bowls that kids make and do not want to take home. I have found that a small clay bowl that has not been glazed works really well for drying tobacco. My theory is that the clay is able to absorb moisture like a paper towel would.
I use a small wooden plates for the office and shop. For travel and something small, why not use the lid of flake tin such as vanilla flake? It's small, and has a lip, I've used lids lots of times when packing a pipe, it's probably the most common thing I use outside a plate.
Here is a link to a leather tobacco valet tray that I make. It's soft enough to bend and pour tobacco in to your pipe.
Basically I use it for rubbing out tobacco or letting it dry.
It's normally available in various sizes, but they have been selling out.
-I just use a dinner plate.
I like the paper hotdog trays you get at the gas station. I keep a stack of around 8-10 of them in my car!
I use the lid of the tin, or the flap of the pouch. I don't see the need for additional apparatus. Maybe I ought to give it a try, even if it's just the lid off a tupperware box.
Thinking about getting this one. You can even get it engraved.
I use a terra cotta planter saucer. Works great for drying tobacco too.
There's a lot of kitchen stuff made of silicone like roll up baking trays, trivets, and drink coasters.
Although not as classy as leather; silicone would be easy to clean and can never catch on fire so might make a good pipe resting spot when outdoors.
I found this spoon rest which looks nice for $1.