text:
Agency Bookings
Blog (ger)
Berlin´s Best
Alternative Berlin
Street Art Berlin
Private Berlin Tour
More Bike Tours
Fahrradtour Berlin on Bike » Privacy Policy
District guides
Privacy Policy 25. May 2018 Martin Wollenberg
An overview of data protection
General information and mandatory information
www.berlinonbike.de
Martin Wollenberg
Knaackstraße 97 in 10435 BERLIN
Postfach 550253 in 10372 BERLIN
Telephone: 030 43 73 99 99
Email: info@berlinonbike.de
Encrypted payments on this website
If you enter into a contract which requires you to send us your payment information (e.g. account number for direct debits), we will require this data to process your payment.
Payment transactions using common means of payment (Visa/MasterCard, direct debit) are only made via encrypted SSL or TLS connections. You can recognize an encrypted connection in your browser's address line when it changes from "http://" to "https://" and the lock icon in your browser line is visible.
In the case of encrypted communication, any payment details you submit to us cannot be read by third parties.
Processing of data (customer and contract data)
We collect, process, and use personal data only insofar as it is necessary to establish, or modify legal relationships with us (master data). This is done based on Art. 6 (1) (b) DSGVO, which allows the processing of data to fulfill a contract or for measures preliminary to a contract. We collect, process and use your personal data when accessing our website (usage data) only to the extent required to enable you to access our service or to bill you for the same.
Collected customer data shall be deleted after completion of the order or termination of the business relationship. Legal retention periods remain unaffected.
Data transferred when signing up for services and digital content
We transmit personally identifiable data to third parties only to the extent required to fulfill the terms of your contract with us, for example, to banks entrusted to process your payments.
Your data will not be transmitted for any other purpose unless you have given your express permission to do so. Your data will not be disclosed to third parties for advertising purposes without your express consent.
Analytics and advertising
This website uses Google Analytics' demographic features. This allows reports to be generated containing statements about the age, gender, and interests of site visitors. This data comes from interest-based advertising from Google and third-party visitor data. This collected data cannot be attributed to any specific individual person. You can disable this feature at any time by adjusting the ads settings in your Google account or you can forbid the collection of your data by Google Analytics as described in the section "Refusal of data collection".
Our websites use the features of Google Analytics Remarketing combined with the cross-device capabilities of Google AdWords and DoubleClick. This service is provided by Google Inc., 1600 Amphitheater Parkway, Mountain View, CA 94043, USA.
This feature makes it possible to link target audiences for promotional marketing created with Google Analytics Remarketing to the cross-device capabilities of Google AdWords and Google DoubleClick. This allows advertising to be displayed based on your personal interests, identified based on your previous usage and surfing behavior on one device (e.g. your mobile phone), on other devices (such as a tablet or computer).
Once you have given your consent, Google will associate your web and app browsing history with your Google Account for this purpose. That way, any device that signs in to your Google Account can use the same personalized promotional messaging.
To support this feature, Google Analytics collects Google-authenticated IDs of users that are temporarily linked to our Google Analytics data to define and create audiences for cross-device ad promotion.
You can permanently opt out of cross-device remarketing/targeting by turning off personalized advertising in your Google Account; follow this link: https://www.google.com/settings/ads/onweb/.
The aggregation of the data collected in your Google Account data is based solely on your consent, which you may give or withdraw from Google per Art. 6 (1) (a) DSGVO. For data collection operations not merged into your Google Account (for example, because you do not have a Google Account or have objected to the merge), the collection of data is based on Art. 6 (1) (f) DSGVO. The website operator has a legitimate interest in analyzing anonymous user behavior for promotional purposes.
For more information and the Google Privacy Policy, go to: https://www.google.com/policies/technologies/ads/.
This website uses Google AdWords. AdWords is an online advertising program from Google Inc., 1600 Amphitheater Parkway, Mountain View, CA 94043, United States ("Google").
As part of Google AdWords, we use so-called conversion tracking. When you click on an ad served by Google, a conversion tracking cookie is set. Cookies are small text files that your internet browser stores on your computer. These cookies expire after 30 days and are not used for personal identification of the user. Should the user visit certain pages of the website and the cookie has not yet expired, Google and the website can tell that the user clicked on the ad and proceeded to that page.
Each Google AdWords advertiser has a different cookie. Thus, cookies cannot be tracked using the website of an AdWords advertiser. The information obtained using the conversion cookie is used to create conversion statistics for the AdWords advertisers who have opted for conversion tracking. Customers are told the total number of users who clicked on their ad and were redirected to a conversion tracking tag page. However, advertisers do not obtain any information that can be used to personally identify users. If you do not want to participate in tracking, you can opt-out of this by easily disabling the Google Conversion Tracking cookie by changing your browser settings. In doing so, you will not be included in the conversion tracking statistics.
Conversion cookies are stored based on Art. 6 (1) (f) DSGVO. The website operator has a legitimate interest in analyzing user behavior to optimize both its website and its advertising.
For more information about Google AdWords and Google Conversion Tracking, see the Google Privacy Policy: https://www.google.de/policies/privacy/.
We use "Google reCAPTCHA" (hereinafter "reCAPTCHA") on our websites. This service is provided by Google Inc., 1600 Amphitheater Parkway, Mountain View, CA 94043, USA ("Google").
reCAPTCHA is used to check whether the data entered on our website (such as on a contact form) has been entered by a human or by an automated program. To do this, reCAPTCHA analyzes the behavior of the website visitor based on various characteristics. This analysis starts automatically as soon as the website visitor enters the website. For the analysis, reCAPTCHA evaluates various information (e.g. IP address, how long the visitor has been on the website, or mouse movements made by the user). The data collected during the analysis will be forwarded to Google.
The reCAPTCHA analyses take place completely in the background. Website visitors are not advised that such an analysis is taking place.
Data processing is based on Art. 6 (1) (f) DSGVO. The website operator has a legitimate interest in protecting its site from abusive automated crawling and spam.
For more information about Google reCAPTCHA and Google's privacy policy, please visit the following links: https://www.google.com/intl/de/policies/privacy/ and https://www.google.com/recaptcha/intro/android.html.
Our website measures conversions using visitor action pixels from Facebook, Facebook Inc., 1601 S. California Ave, Palo Alto, CA 94304, USA ("Facebook").
These allow the behavior of site visitors to be tracked after they click on a Facebook ad to reach the provider's website. This allows an analysis of the effectiveness of Facebook advertisements for statistical and market research purposes and their future optimization.
The data collected is anonymous to us as operators of this website and we cannot use it to draw any conclusions about our users' identities. However, the data are stored and processed by Facebook, which may make a connection to your Facebook profile and which may use the data for its own advertising purposes, as stipulated in the Facebook privacy policy. This will allow Facebook to display ads both on Facebook and on third-party sites. We have no control over how this data is used.
Check out Facebook's privacy policy to learn more about protecting your privacy: https://www.facebook.com/about/privacy/.
You can also deactivate the custom audiences remarketing feature in the Ads Settings section at https://www.facebook.com/ads/preferences/?entry_product=ad_settings_screen. You will first need to log into Facebook.
If you do not have a Facebook account, you can opt out of usage-based advertising from Facebook on the website of the European Interactive Digital Advertising Alliance: http://www.youronlinechoices.com/de/praferenzmanagement/.
Plugins and tools
Payment service providers
Our website accepts payments via PayPal. The provider of this service is PayPal (Europe) S.à.r.l & Cie, S.C.A. (22-24 Boulevard Royal, L-2449 Luxembourg.
If you select payment via PayPal, the payment data you provide will be supplied to PayPal based on Art. 6 (1) (a) (Consent) and Art. 6 (1) (b) DSGVO (Processing for contract purposes). You have the option to revoke your consent at any time with future effect. It does not affect the processing of data previously collected.
(school groups, families, companies, associations, etc.) request now
All Bike Tours in Berlin
Feedback / Guestbook
Contact – Berlin on Bike
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wishAccept Read more

redpajama_set_name: RedPajamaCommonCrawl


text:
How Many Early American Coppers Still Exist? Where Have The Rest Gone?
A total of 156,288,744 large cents were struck and issued between 1793 and 1857, or about the monthly output of cents today. A miniscule number of half cents, only 7,865,226, were struck and issued during these years.
Today, large cents and half cents are available at most coin shops and shows. While there is merit to the thought that you see far fewer coppers today as compared to ten years ago, the fact remains that early American coppers are available to collectors. But how many exist? And what happened to all the others?
First, let's see what happened to these coppers. The same factors that wear out our current coinage were also at work when the early coppers circulated. Wear and tear naturally occurred on them. An unknown number just wore out, passing from hand to hand. In addition, many large and half cents were simply lost, misplaced or dropped. Some of these coppers are found in the ground today, while others turn up in old desks and dressers.
Other cents were exported, never to return to the United States. Many of these coins went to Canada, with some returning years later. Others went to South America. Again, no total number of large cents exported is available, but it is known that the Venezuelan government ordered a million cents from the Mint in 1835.
It is impossible to estimate the number of early coppers lost this way. With virtually no data available for these type losses, perhaps no one can.
Other losses of early coppers can be documented, at least for large cents. Melting is one such loss. One source, the United States Mint, recorded melting a total of 38,386,687 large cents from 1857 until 1953, when separate records for large cent redemptions were stopped.
Of these 38 million large cents, 84,781 were redeemed and melted by the mint after 1930! The last recorded redemption was for 290 large cents in 1953.
Redeeming large cents at the Mint began when the new flying eagle cents were offered to the public, in exchange for old coppers or Spanish silver, in February 1857. Thousands of large cents returned to the Mint that first day and large numbers were redeemed until the start of the Civil War.
Commercial melting of early coppers also occurred. When the intrinsic value of copper exceeded one cent, private melting occurred. Records of private melts are virtually non-existent, but research done by Craig Sholley estimates the number of cents melted at several different times. He estimates that 10-12 million were melted between 1800 and 1820 and a further 32.5 million during the Civil War.
That is 81 – 83 million large cents melted by the Mint and privately, or more than half the total mintage! And this doesn't take into account those lost.
So, how many early American coppers still exist? The number of large and half cents in existence today can not be determined exactly, just as we can not determine exactly how many were melted or lost. It is also impossible to see and count all the large cents that are known. Duplications would occur and come coppers would be missed.
Thus, we must rely on estimates. These estimates are always based on data from a small sample that is extrapolated to cover the entire series of large and half cents. R.H. Williamson was one of the first to present an estimate on the number of surviving early American large cents. In the July 1949 issue of The Numismatist. Williamson suggested that several million large cents still existed (in 1949) in all conditions. He based his estimate on the number of cents he had seen.
In the early 1950s, William Sheldon and Walter Breen intensely studied the cents of 1794 and concluded that 3% still survived. Sheldon and Breen did not suggest their findings applied to all other dates of large cents, but if we were to assume that 3% of the mintage survived for all years, that would mean there were 4.7 million large cents existent.
In the first issue (September 1967) of Penny Wise, the journal of the Early American Coppers club, Warren Lapp "guesstimated" the number of surviving large cents at between 767,000 and 1.57 million, much lower than earlier estimates. Lapp used rarity ratings and data from random purchases by W.E. Johnson, a copper dealer, to reach his conclusions.
More recent estimates have used data accumulated from a large number of collections, long kept records, and eBay. In the late 1990's, Red Henry studied the survivorship of cents dated 1800 – 1807. His conclusions, published in Penny Wise, suggested that cents from this era survived at a rate of about 1%. If extrapolated to the entire large cent series, this would mean about 1.5 million survivors.
Several studies of survivors were published by Ron Manley in Penny Wise in 2001. These included estimates that less than 1% of middle date cents and slightly more than 1% of late date cents had survived. After further study, Manley suggests that only about 570,000 large cents exist today, or about 1/3 of 1% of all those minted!
Bill Eckberg, has been studying half cent survivors. His research, published in Penny Wise in 2000 and 2001, estimates that only 140,000 half cents survive, or a little less than 2% of the total mintage.
Early American coppers were struck to promote commerce in our young nation. They did this admirably, serving as a medium of commerce, as a source of metal, and even as heirlooms and collectibles. Most were melted or lost, but a few survived. When you look at your next early copper, think about what it has been through and marvel at the fact it survived.


Join the UW Madison Chapter for Pengyou Day!
Join us for National Pengyou Day! We will be hosting a discussion with Professor Yin, a professor in the Department of Genetics and Neurology and the director of the Wisconsin China Initiative! The discussion will be followed by brush painting, games, and other activities!


Across the road from the beautiful waters of Lake Macquarie. Centrally located close to major hospitals, minutes from shopping centres, beaches, Glendale sports facility and only 20 minutes from Newcastle CBD.Hire a dinghy, canoe or pedal cat at Warners Bay foreshore during the summer, or take a three hour cruise on beautiful Lake Macquarie. Take a walk along the many cycle tracks around the lake or fish from the jetty. Search below for Lakeside Motor Inn Lake Macquarie reviews, photos, map information and rates.
About this page: Lakeside Motor Inn Lake Macquarie bookings - Last minute rates on Lakeside Motor Inn reservations. Cheap Lakeside Motor Inn rates, last minute deals on rooms (twin, superior and deluxe to luxury suites), group hotel booking, holiday and weekend specials. Compare all online rates at the Lakeside Motor Inn Lake Macquarie.


text:
Republicans are Racists
…And they're not shy about it.
Tag Archives: Mitt Romney
Photos, President Obama, Republicans are Racist, Seriously?
Romney Rally Fail: Put the White Back in the White House!
"Put the White Back in the White House" T-Shirt at Romney / Ryan Rally.
This Getty Images photo from a Romney / Ryan rally in Lancaster, Ohio on Friday has gotten a lot of attention. I wonder why? This Romney supporter not only felt comfortable enough in this crowd to wear a t-shirt saying "Put the White back in the White House" but he wanted to spike the ball by adding a Romney / Ryan sticker to the top of the message.
A Romney spokesperson commented that the shirt was "reprehensible and has no place in this election." That being said, the Right-wing Media leaped to Romnney's rescue claiming that this guy was a Democratic plant and that the photo was taken outside. There is absolutely no evidence of a plant (but what else can you say just weeks before the election) and it was later proven that the photo was indeed taken inside of the event.
This is certainly no surprise, and I'm sure the man got mad props on his wardrobe choice from his fellow Republicans, but what terrible timing. Didn't anyone tell him that the GOP really wants to downplay the race issue until after election day? Guess it doesn't matter to this guy. The important thing to him is getting his message out by preaching to his racist choir.
Tagged GOP, Mitt Romney, politics, Racism, Rally
Mitt Romney, Republicans are Racist
Mitt Romney's Racist Joke At A 1% Fundraiser
As Mitt Romney was giving a speech at the Los Angeles Hispanic Chamber of Commerce yesterday, a video was being leaked from a Romney private fund raiser held in the residence of one of his 1% supporters. Judging by the camera position, someone was being sneaky and getting a recording of a speech that Romney certainly didn't want leaving that room. Well, in this day and age, there is no such thing as complete privacy. Now we can let the real Mitt tell you how he feels.
"My dad, as you probably know, was the governor of Michigan and was the head of a car company. But he was born in Mexico… and had he been born of, uh, Mexican parents, I'd have a better shot at winning this. But he was unfortunately born to Americans living in Mexico. He lived there for a number of years. I mean, I say that jokingly, but it would be helpful to be Latino."
That's right, kids. Romney thinks things would be better for him if he had been born Latino instead of drawing the short straw and being born white, rich beyond most of our wildest dreams, and ruggedly handsome. The poor man has suffered so much, as we all know from the RNC, and all he wants is to be a minority.
Rep. Charlie Gonzalez of San Antonio had the following to say: "It's not about being a Latino that would make Romney a better candidate," Gonzalez said. "Latinos, and all Americans, look at the substance of a candidate's proposals to determine who is the better candidate and who best represents the priorities of their community. That is why upon examining the policies willingly supported by Romney on the economy, health care, and immigration, Latinos overwhelmingly support President Obama."
Now is your time, loyal GOP defenders, to step up once again to declare yourselves innocent. The truth keeps coming out, though. When among their own kind and unaware that any outsiders are listening, even Republican Presidential Candidates are racists.
Tagged current-events, Mitt Romney, politics, Racism, republican presidential candidates
Colin Powell Emails Expose His Opinion on Racism in the GOP
Donald Trump is "the Least Racist Person that You've Ever Met"
Republicans are doomed: How the GOP's cynical election strategy is imploding
The 2016 Democratic Party Debate's Award Winning Racist Tweet
ALTERNET: Donald Trump and the GOP's Demise
Chauvinism
Greg Gutfeld
Republican Candidates
Republicans are Racist
Television Ads
Republicans are Racists – Race Based Politics in the GOP


The energy industry is not the typical manufacturing and distribution environment the AIT team is most associated with, however, we've realized the oil and gas industry is a lot closer than many may think. In the Upstream, we're "developing" geological prospects, "manufacturing" well sites and facilities, and managing output of "production" wells. In the Downstream, we get even more aligned as the extracted oil and gas is refined and manufactured into consumable products like gasoline, diesel, chemicals, motor oil, lubricants and much more. The AIT team has developed a great appreciation for the skills, technology and risks associated with operating in this industry, and we are proud to help companies like ExxonMobil become even better companies!


text:
Gantz pushes to buy third F-35 squadron
The IDF's F-35 "Adir" stealth fighter plane. (Flash 90/Moshe Shai)
(Flash 90/Moshe Shai)
Israel's Defense Minister said the nation currently needs to purchase another squadron of stealth F-35 warplanes from the United States.
By Ebin Sandler, World Israel News
On Monday, Israeli Defense Minister Benny Gantz said that Israel should purchase another batch of F-35s fighter jets, which would be the nation's third squadron of the stealth warplanes.
Israel purchases the aircraft from the United States, and Gantz expressed hope that an agreement for the planes could be concluded while President Donald Trump remains in office. He is set to step down on January 20.
With the United Arab Emirates (UAE) on the verge of acquiring its own fleet of F-35s, Israel is pursuing strategies to maintain its qualitative military edge in the Middle East.
The Trump administration signaled willingness in 2020 to sell F-35s to the UAE, which concluded a peace agreement with Israel last year as part of the Abraham Accords.
In addition to the UAE, Bahrain, Sudan, and Morocco also signed on to establish diplomatic relations with Israel as part of the accords, widely viewed as one of the crowning achievements of Trump and senior advisor Jared Kushner.
Speculation is currently running high that Saudi Arabia will be the next Muslim country in the Mideast to sign a deal with Israel.
Read Iran executes two anti-regime demonstrators
Prior to the Abraham Accords, the U.S. appeared unwilling to permit other allies in the region to purchase F-35s.
While Israel and its new Arab allies have touted benefits related to trade, technology, and tourism, the Abraham Accords also appear designed to forge a unified front against Iran, which operates terror proxies in Lebanon, Syria, Yemen, and the Gaza Strip.
Iran is also believed to be behind attacks in Saudi Arabia and recently captured a South Korean tanker in the Persian Gulf. Analysts believe the Islamic Republic is holding the ship hostage to force South Korea to disgorge $7 billion in funds frozen due to tightening economic sanctions imposed by the Trump administration.
In addition to sponsoring and training terror groups, developing ballistic missiles, and perpetuating civil war in Yemen and Syria, Iran is suspected of developing nuclear weapons.
Israel produced credible evidence in 2018 of Iran's nuclear ambitions, which Jerusalem says has been confirmed by Iran's announcement this week it is enriching uranium to 20% in breach of the 2015 nuclear pact.
Benny GantzF-35IranUnited Arab Emirates
I found a very interesting article on World Israel News! Click to read this: --> https://worldisraelnews.com/gantz-pushes-to-buy-third-f-35-squadron/?ios_app=true


text:
\section{Introduction}

Nonnegative matrix factorization (NMF) aims at factorizing a matrix with
nonnegative entries into the product of two nonnegative matrices.
It has found many applications in various domains which include feature
extraction in image processing and text
mining~\cite{Lee_D_1999_j-nature_learning_ponmf}, audio source
separation~\cite{Smaragdis_P_2014_j-ieee-sig-proc-mag_static_dssunmfuv},
blind unmixing in hyperspectral
imaging~\cite{Berry_W_2007_j-comput-stat-data-anal_algorithms_aanmf,
  BioucasDias_J_2012_j-ieee-j-sel-top-appl-earth-obs-rem-sens_hyperspectral_uogssrba},
and user recommendation~\cite{Hu_Y_2008_p-ieee-icdm_collaborative_fifd}.
See~\cite{Cichoki_A_2009_book_nonnegative_mtf,
  Fu_X_2019_j-ieee-sig-proc-mag_nonnegative_mfsdaiaa,
  Gillis_N_2020_book_nonnegative_mf} for overview papers and books about NMF\@.

Each application gives different interpretations to the factor matrices but the
first factor is often considered as a dictionary of recurring patterns while the
second one describes how the data samples are expanded onto the dictionary
(activation matrix).
The nonnegativity constraint only allows additive combination of the dictionary
elements, yielding meaningful additive and sparse representations of the data.

Computing an NMF generally consists in minimizing a well-chosen objective
function under non-negativity constraints.
A popular choice of objective function is the $\beta$-divergence, which is a
continuous family of measures of fit parameterized by a single parameter $\beta$
that encompasses the Kullback-Leibler (KL) or Itakura-Saito (IS) divergences as
well as the common squared Euclidean
distance~\cite{Cichocki_A_2011_j-entropy_generalized_abdtarnmf}.
In the latter cases, the $\beta$-divergence is a log-likelihood in disguise for
Poisson, multiplicative Gamma and additive Gaussian noise models, respectively.

The classic approach to NMF, and to NMF with the $\beta$-divergence (later referred
to as $\beta$-NMF) in particular, consists in optimizing the two factors
alternately, i.e., using two-block coordinate descent.
Each of the two factors is then updated using Majorization-Minimization (MM), as
described in~\cite{Lee_D_2001_adv_neur_proc_inf_algorithms_nmf,
  Fevotte_C_2011_j-neural-comput_algorithms_nmfbd,
  Yang_Z_2011_j-ieee-trans-nnet_unified_dmalqnmf}: given one of the factors, a
tight upper bound of the objective function is constructed and minimized with
respect to (w.r.t) the other factor.
This results in multiplicative updates that are simple to implement (with no
hyperparameter to tune), have linear complexity per iteration, and that
automatically preserve non-negativity given positive initializations.
By construction, MM ensures monotonicity of the objective function
(non-increasing values), see~\cite{Lange_K_2016_book_mm_oa,
  Sun_Y_2017_j-ieee-trans-sig-proc_majorization_maspcml} for tutorials about
MM\@.
Furthermore, the NMF iterates can be shown to converge to a critical point under
mild assumptions~\cite{Zhao_R_2018_j-ieee-trans-sig-proc_unified_camuarnmf}.

Thanks to its simplicity, the block-descent approach is dominant in matrix
factorization and dictionary learning (using sometimes other block partitions
such as columns or rows~\cite{Cichoki_A_2009_book_nonnegative_mtf,
  Gillis_N_2020_book_nonnegative_mf}) and very few works have addressed joint
(``all-at-once'') optimization of the factors, though the latter approach may be
more efficient.
A notable exception in dictionary learning (real-valued factors, sparse
activations) is~\cite{Rakotomamonjy_A_2013_j-ieee-trans-sig-proc_direct_odlp}.
In this work, the author employs an elegant non-convex proximal splitting
strategy and shows that the joint approach is significantly faster than
alternating methods without altering the quality of the obtained solution.
More recently and using tools from dynamical systems, the authors
in~\cite{Panageas_I_2020_PP_convergence_sosnmfpc} have derived a novel form of
multiplicative update which can run concurrently for each factor matrix at a
given iteration.
Moreover, it has the additional benefit of ensuring the convergence to a local
minimum instead of a mere stationary point.
However, their results are limited to the squared Euclidean distance objective
function.

Inspired by these works, we here explore a joint MM approach to $\beta$-NMF and
compare it to the classic block MM strategy.
Our joint MM relies on a tight majorization of the objective function with
respect to all its variables instead of using blocks of fixed variables.
This results in new multiplicative updates that are simple but more efficient
variants of the classical multiplicative updates.
This is particularly true when considering the squared Euclidean distance and
the KL or IS divergences for which further simplifications occur.

Our results are supported by extensive simulations using datasets with different
sizes arising from various applications (face images, audio spectrogram,
hyperspectral images, song play-counts).
In most scenarii, the proposed joint MM approach leads to a reduction of the
computing time ranging from $10\%$ to $78\%$ without incurring any loss in the
precision of the approximation.

The article is organized as follows: Section~\ref{sec:prelim} states the NMF
optimization problem and summarizes the classic block MM approach.
Section~\ref{sec:jmm} first presents our proposed joint MM method for NMF and
derives the new multiplicative updates.
It then compares the joint and the block MM methods in term of computational
complexity, and discusses the benefit of the joint approach.
Comparative numerical simulations are presented in Section~\ref{sec:simul} and
validate the efficiency of our approach.
Finally, Section~\ref{sec:concl} draws conclusions.

\paragraph*{Notation} $\mathbb{R}_{+}$ is the set of nonnegative real numbers, and
$\nint{1,N}$ is the set of integers from $1$ to $N$.
Bold upper case letters denote matrices, bold lower case letters denote vectors,
and lower case letters denote scalars.
The notation ${[\*M]}_{ij}$ and $m_{ij}$ both stand for the element of $\*M$
located at the $i^{\text{th}}$ row and the $j^{\text{th}}$ column.
For a matrix $\*M$, the notation $\*M \ge 0$ denotes entrywise nonnegativity.

\section{Preliminaries}
\label{sec:prelim}

\subsection{Nonnegative matrix factorization}

We aim at factorizing a $F \times N$ nonnegative matrix $\*V$ into the product
$\*W\*H$ of two nonnegative factor matrices of sizes $F \times K$ and $K \times N$,
respectively.
In most scenarii, the rank value $K$ is chosen such that $FK + KN \ll FN$,
leading to a low-rank approximation of $\*V$.

Giving a desired value for the rank $K$, the factor matrices are obtained by
solving the following optimization problem
\begin{equation}
  \label{eq:nmf}
\min_{\*W, \*H \ge 0 } D_{\beta}(\*V \mid \*W\*H)
\end{equation}
where $D_{\beta}$ is a separable objective function defined by
\begin{equation}
  \label{eq:obj_fct}
  D_{\beta}(\*V \mid \*W\*H) = \sum_{f=1}^{F}\sum_{n=1}^{N} d_\beta(V_{fn}\mid{[\*W\*H]}_{fn}) \, .
\end{equation}
Our measure of fit $d_{\beta}$ is the
$\beta$-divergence~\cite{Cichocki_A_2011_j-entropy_generalized_abdtarnmf} given by
\begin{equation}
  \label{eq:beta-div}
  d_{\beta}(x \mid y) =
  \begin{cases*}
    x\log \frac{x}{y} - x + y          & if $\beta=1$ \\
    \frac{x}{y} - \log \frac{x}{y} - 1 & if $\beta=0$ \\
    \frac{x^{\beta}}{\beta(\beta-1)}+\frac{y^{\beta}}{\beta} - \frac{x y^{\beta-1}}{\beta-1}
    & otherwise.
  \end{cases*}
\end{equation}
The value of $\beta$ is chosen according to the application context and the noise
assumptions on $\*V$~\cite{Fevotte_C_2011_j-neural-comput_algorithms_nmfbd}.
The IS divergence, KL divergence and squared Euclidean distance are obtained
for $\beta= 0,1,2$, respectively.

\subsection{Classic multiplicative updates}
\label{ssec:bmm}

The classic method to solve Problem~\eqref{eq:nmf} consists in a two-block
coordinate descent approach where each block is handled with MM\@.
Namely, it alternately minimizes $(\*W,\*H) \mapsto D_{\beta}(\*V|\*W\*H)$ in $\*W$ and in
$\*H$.

The MM method is a two-step iterative optimization
scheme~\cite{Lange_K_2016_book_mm_oa,
  Sun_Y_2017_j-ieee-trans-sig-proc_majorization_maspcml}.
At each iteration, the first step consists in building a local auxiliary
function $G$ that is minimized in the second step.
The auxiliary function has to be a tight majorizer of the original objective
function $\phi:\mathbb{E}\mapsto\mathbb{R}$ at the current iterate $\tilde{\*x}$, where $\mathbb{E}$ is the
domain of $G$ and $\phi$.
More precisely, it has to satisfy the following two properties:
\begin{align*}
  \left(\forall\*x\in\mathbb{E}\right) \quad
  G(\*x \mid \tilde{\*x}) &\geq \phi(\*x) \\
  G(\tilde{\*x} \mid \tilde{\*x}) &= \phi(\tilde{\*x}) \, .
\end{align*}
These properties ensure that any iterate $\*x$ that decreases the value of $G$
also decreases the value of $\phi$.
Indeed, for a given $\tilde{\*x}$, if we find $\*x$ such that
$G(\*x|\tilde{\*x}) \leq G(\tilde{\*x}|\tilde{\*x})$, then the tight majorization
properties induce the following descent lemma
\begin{equation}
  \label{eq:desc_lemm}
  \phi(\*x) \leq G(\*x \mid \tilde{\*x}) \leq G(\tilde{\*x} \mid \tilde{\*x}) = \phi(\tilde{\*x})
  \, .
\end{equation}

The previous MM scheme can be applied alternately to the minimization of the two
functions $\*W \mapsto D_{\beta}(\*V|\*W\*H)$ and $\*H \mapsto D_{\beta}(\*V|\*W\*H)$.
These two functions are the sum of concave, convex, and constant terms.
An auxiliary function can then be easily derived by using Jensen's inequality
for the convex term and the tangent inequality for the concave term,
see~\cite{Fevotte_C_2011_j-neural-comput_algorithms_nmfbd} and the next section.
This yields the following multiplicative updates
\begin{equation}
  \label{eq:alt_update}
  \begin{aligned}
    \*W &\; \longleftarrow \; \*W.{\left(\frac{\left({(\*W\*H)}^{.(\beta-2)}.\*V\right)\*H^{\top}}
      {\left({(\*W\*H)}^{.(\beta-1)}\right)\*H^{\top}}\right)}^{\gamma(\beta)} \\
    \*H &\; \longleftarrow \; \*H.{\left(\frac{\*W^{\top}\left({(\*W\*H)}^{.(\beta-2)}.\*V\right)}
      {\*W^{\top}\left({(\*W\*H)}^{.(\beta-1)}\right)}\right)}^{\gamma(\beta)}
    \, ,
  \end{aligned}
\end{equation}
where $.$ and $/$ are the element-wise multiplication and division,
respectively, and $\gamma(\beta)$ is a scalar defined as
\begin{equation}
  \label{eq:def_gamma}
  \gamma(\beta) =
  \begin{cases*}
    \frac{1}{2-\beta} & if $\beta \in ]-\infty,1[$ \\
                1 & if $\beta\in[1,2]$ \\
    \frac{1}{\beta-1} & if $\beta\in]2,+\infty[$
  \end{cases*} \, .
\end{equation}
Note that by construction, the matrices $\*W$ and $\*H$ resulting from the
update~\eqref{eq:alt_update} contain only positive coefficients if the input
matrices $\*V$, $\*W$, and $\*H$ are all positive.
The non-negativity of the iterates is thus preserved by the multiplicative
structure of the updates given positive initializations.
Note also that the individual updates of $\*W$ and $\*H$ given
in~\eqref{eq:alt_update} may also be applied several times to fully minimize the
partial functions $\*W \mapsto D_{\beta}(\*V|\*W\*H)$ and $\*H \mapsto D_{\beta}(\*V|\*W\*H)$.
In standard practice, only one sub-iteration is however used, still resulting in
a descent algorithm thanks to the descent lemma~\eqref{eq:desc_lemm}.
The algorithm described by~\eqref{eq:def_gamma} will be referred to as Block MM
(BMM) in the rest of the paper.

\section{Joint Majorization-Minimization}
\label{sec:jmm}

In contrast with the classic alternated scheme presented in
Section~\ref{ssec:bmm}, we develop in this section a joint MM (JMM) approach to
solve Problem~\eqref{eq:nmf}.

\subsection{Construction of the auxiliary function}

In order to apply the MM scheme, we start by looking for a suitable auxiliary
function $G: \mathbb{R}_{+}^{F \times K}\times\mathbb{R}_{+}^{K \times N} \rightarrow \mathbb{R}_{+}$ for the function
$(\*W,\*H) \mapsto D_{\beta}(\*V|\*W\*H)$.
We observe in~\eqref{eq:obj_fct} that $D_{\beta}$ is a sum of $FN$ $\beta$-divergences
between scalars.
Our approach is to construct an auxiliary function for each summand.
Following~\cite{Fevotte_C_2011_j-neural-comput_algorithms_nmfbd}, the
$\beta$-divergence $d_{\beta}$, taken as a function of its second argument, can be
decomposed into the sum of a convex term $\conv{d}_{\beta}$, a concave term
$\conc{d}_{\beta}$, and a constant term $\bar{d}_{\beta}$.
The definitions of these three terms for the different values of $\beta$ are given
in Table~\ref{table:elmt_beta-div}.

\begin{table}[t]
  \caption{Decomposition of $d_{\beta}$ for the different values of $\beta$.}
  \begin{center}
    \setlength\tabcolsep{5.5pt}
    \resizebox{\columnwidth}{!}{\begin{tabular}{cccc}
      \toprule
      $\beta$ & $\conv{d}_{\beta}(v_{fn} \mid {[\*W\*H]}_{fn})$
          & $\conc{d}_{\beta}(v_{fn} \mid {[\*W\*H]}_{fn})$
          & $\bar{d}_{\beta}(v_{fn})$ \\
      \midrule
      \midrule
      $]-\infty,1[\setminus\{0\}$ & $\frac{-v_{fn}}{\beta-1}{[\*W\*H]}_{fn}^{\beta-1}$
                   & $\frac{{[\*W\*H]}_{fn}^{\beta}}{\beta}$
                   & $\frac{v_{fn}^{\beta}}{\beta(\beta-1)}$ \\
      \midrule
      $0$          & $\frac{v_{fn}}{{[\*W\*H]}_{fn}}$
                   & $\log {[\*W\*H]}_{fn}$
                   & $-(\log v_{fn} + 1)$ \\
      \midrule
      $[1,2]$      & $d_{\beta}(v_{fn} \mid {[\*W\*H]}_{fn})$ & $0$ & $0$ \\
      \midrule
      $]2,+\infty[$     & $\frac{{[\*W\*H]}_{fn}^{\beta}}{\beta}$
                   & $\frac{-v_{fn}}{\beta-1}{[\*W\*H]}_{fn}^{\beta-1}$
                   & $\frac{v_{fn}^{\beta}}{\beta(\beta-1)}$ \\
      \bottomrule
    \end{tabular}}
  \end{center}
  \label{table:elmt_beta-div}
\end{table}

Next, we majorize the convex and concave terms of $d_{\beta}$ separately.
The methodology
follows~\cite{Fevotte_C_2011_j-neural-comput_algorithms_nmfbd}, except that none
of the two factors $\*W$ or $\*H$ is treated as a fixed variable.
The convex term $\conv{d}_{\beta}$ is majorized using Jensen's inequality
\begin{align}
  \conv{d}_{\beta}(v_{fn} \mid {[\*W\*H]}_{fn})
  &= \conv{d}_{\beta}\left(v_{fn} \;\middle|\; \sum_{k=1}^{K}w_{fk}h_{kn}\right)
  \nonumber \\
  &= \conv{d}_{\beta}\left(v_{fn} \;\middle|\;
  \sum_{k=1}^{K}\tilde{\lambda}_{fnk}\frac{w_{fk}h_{kn}}{\tilde{\lambda}_{fnk}}\right)
  \nonumber \\
  &\leq \sum_{k=1}^{K}\tilde{\lambda}_{fnk} \conv{d}_{\beta}\left(v_{fn} \;\middle|\;
  \frac{w_{fk}h_{kn}}{\tilde{\lambda}_{fnk}}\right)
  \label{eq:simplif} \\
  &\stackrel{\mbox{\footnotesize def}}{=} \conv{G}_{fn}(\*W,\*H \mid \boldsymbol{\tilde{\mathbf{W}}}, \boldsymbol{\tilde{\mathbf{H}}}) \nonumber
  \, ,
\end{align}
where the coefficient $\tilde{\lambda}_{fnk}$ is defined as
$\tilde{\lambda}_{fnk} = \frac{\tilde{w}_{fk}\tilde{h}_{kn}}{\tilde{v}_{fn}}$ and we
denote $\boldsymbol{\tilde{\mathbf{V}}}=\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}}$ with coefficients $\tilde{v}_{fn}$.
The concave term $\conc{d}_{\beta}$ is majorized using the tangent inequality
\begin{align*}
  \conc{d}_{\beta}&(v_{fn} \mid {[\*W\*H]}_{fn}) \\
  &\leq \conc{d}_{\beta}(v_{fn} \mid \tilde{v}_{fn})
  + \conc{d'}_{\beta}(v_{fn} \mid \tilde{v}_{fn})
  \left({[\*W\*H]}_{fn}-\tilde{v}_{fn}\right) \\
  &\stackrel{\mbox{\footnotesize def}}{=} \conc{G}_{fn}(\*W,\*H | \boldsymbol{\tilde{\mathbf{W}}}, \boldsymbol{\tilde{\mathbf{H}}})
  \, .
\end{align*}
Finally the overall auxiliary function $G$ is given by
\[
  G = \sum_{f=1}^{F}\sum_{n=1}^{N}\left[ \conv{G}_{fn} + \conc{G}_{fn} + \bar{d}_{\beta} \right]
  \, .
\]
By construction, it satisfies the tight majorization properties
\begin{align*}
  &G(\*W,\*H \mid \boldsymbol{\tilde{\mathbf{W}}},\boldsymbol{\tilde{\mathbf{H}}}) \geq D_{\beta}(\*V\mid\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}}) \\
  &G(\boldsymbol{\tilde{\mathbf{W}}},\boldsymbol{\tilde{\mathbf{H}}} \mid \boldsymbol{\tilde{\mathbf{W}}},\boldsymbol{\tilde{\mathbf{H}}}) = D_{\beta}(\*V\mid\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}}) \, ,
\end{align*}
which ensure the descent property in~\eqref{eq:desc_lemm}.

We stress out that the auxiliary function $G$ is a tight joint majorization of
$D_{\beta}$.
This is in contrast with the BMM approach of Section~\ref{ssec:bmm} where two
separate auxiliary functions $G_{\*W}$ and $G_{\*H}$ are built for
$\*W \mapsto D_{\beta}(\*V|\*W\*H)$ and $\*H \mapsto D_{\beta}(\*V|\*W\*H)$, respectively.
A central difference in our JMM approach is the definition of the coefficients
$\{\tilde{\lambda}_{fnk}\}$ which depends on both current iterates
$\boldsymbol{\tilde{\mathbf{W}}}$ and $\boldsymbol{\tilde{\mathbf{H}}}$ and does not lead to a simplification of the term
$w_{fk} h_{kn}/\tilde{\lambda}_{fnk}$ in~\eqref{eq:simplif}.
This is in contrast with BMM, where, say, $\*W$ would be treated as a fixed
variable and the term $w_{fk} h_{kn}/\tilde{\lambda}_{fnk}$ simplifies to
$\tilde{v}_{fn}h_{kn}/\tilde{h}_{kn}$ and leads to close-form minimization of
$G$.
Furthermore, our auxiliary function $G$ is not jointly convex, due to the
bilinear terms $w_{fk} h_{kn}$.
It is however bi-convex, i.e., convex w.r.t $\*W$ (resp., $\*H$) given $\*H$
(resp., $\*W$).

\subsection{Minimization step}

The auxiliary function $G$ does not appear to have a close-form minimizer in
$\*W$ and $\*H$.
Neither is it convex, which makes it difficult to minimize globally.
However, in order to decrease the loss function $D_{\beta}$, it is enough to decrease
$G$ thanks to the descent lemma~\eqref{eq:desc_lemm}.
We therefore search for the points $(\*W,\*H)$ that satisfy the first-order
optimality conditions.
Table~\ref{table:deriv_G} gives the partial derivative of $G$ with respect to
$\{w_{fk}\}$.
The partial derivatives with respect to $\{h_{kn}\}$ are obtained similarly by
permuting the role of $\*W$ and $\*H$.
\begin{table}[t]
  \caption{Partial derivative of $G_{fn}$ with respect to $w_{fk}$ for the
    different values of $\beta$.}
  \begin{center}
    \setlength\tabcolsep{5.5pt}
    \resizebox{\columnwidth}{!}{\begin{tabular}{ll}
      \toprule
      \multicolumn{1}{c}{$\beta$}
      & \multicolumn{1}{c}{$\frac{\partial G}{\partial w_{fk}}(\*W,\*H|\boldsymbol{\tilde{\mathbf{W}}},\boldsymbol{\tilde{\mathbf{H}}})$} \\
      \midrule
      \midrule
      $0$          &
      {$\!\begin{aligned}
        \sum_{n=1}^{N}\left(\frac{h_{kn}}{\tilde{v}_{fn}}
        -v_{fn}\frac{\tilde{w}_{fk}^{2}\tilde{h}_{kn}^{2}}{h_{kn}w_{fk}^{2}\tilde{v}_{fn}^{2}}\right)
      \end{aligned}$} \\
      \midrule
      {$\!\begin{aligned}
        &]-\infty,1[ \\
        &\setminus\{0\}
      \end{aligned}$}
      &
      {$\!\begin{aligned}
        \sum_{n=1}^{N}\left(\tilde{v}_{fn}^{\beta-1}h_{kn}
        - v_{fn}h_{kn}{\left(\frac{w_{fk}h_{kn}\tilde{v}_{fn}}{\tilde{w}_{fk}\tilde{h}_{kf}}\right)}^{\beta-2}\right)
      \end{aligned}$} \\
      \midrule
      $[1,2]$      &
      {$\!\begin{aligned}
        \sum_{n=1}^{N}\left(h_{kn}{\left(\frac{w_{fk}h_{kn}\tilde{v}_{fn}}{\tilde{w}_{fk}\tilde{h}_{kn}}\right)}^{\beta-1}
        -v_{fn}h_{kn}{\left(\frac{w_{fk}h_{kn}\tilde{v}_{fn}}{\tilde{w}_{fk}\tilde{h}_{kn}}\right)}^{\beta-2}\right)
      \end{aligned}$} \\
      \midrule
      $]2,+\infty[$     &
      {$\!\begin{aligned}
        \sum_{n=1}^{N}\left(
        h_{kn}{\left(\frac{w_{fk}h_{kn}\tilde{v}_{fn}}{\tilde{w}_{fk}\tilde{h}_{kn}}\right)}^{\beta-1}
        - v_{fn}\tilde{v}_{fn}^{\beta-2}h_{kn}\right)
        \end{aligned}$} \\
      \bottomrule
    \end{tabular}}
  \end{center}
  \label{table:deriv_G}
\end{table}
As such, searching for the stationary points of $G$ results in a system of
equations for $\{w_{fk}\}$ and $\{h_{kn}\}$ obtained by setting the partial
derivatives to zero.
While this system is hard to solve jointly in $\*W$ and $\*H$, it can be solved
by updating the matrices $\*W$ and $\*H$ alternately.
Doing so results in the following multiplicative updates
\begin{equation}
  \label{eq:jnt_update}
  \begin{aligned}
    \*W &\; \longleftarrow \;
    \boldsymbol{\tilde{\mathbf{W}}}.{\left(
      \frac{\frac{\*V}{{(\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}})}^{.(2-\beta)}}{[\chi_{1,\beta}(\*H,\boldsymbol{\tilde{\mathbf{H}}})]}^{\top}}
           {{(\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}})}^{.(\beta-1)}{[\chi_{2,\beta}(\*H,\boldsymbol{\tilde{\mathbf{H}}})]}^{\top}
           }\right)}^{.\gamma(\beta)} \\
    \*H &\; \longleftarrow \;
    \boldsymbol{\tilde{\mathbf{H}}}.{\left(
      \frac{{[\chi_{1,\beta}(\*W,\boldsymbol{\tilde{\mathbf{W}}})]}^{\top}\frac{\*V}{{(\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}})}^{.(2-\beta)}}}
           {{[\chi_{2,\beta}(\*W,\boldsymbol{\tilde{\mathbf{W}}})]}^{\top}{(\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}})}^{.(\beta-1)}}\right)}^{.\gamma(\beta)}
    \, ,
  \end{aligned}
\end{equation}
where $\gamma(\beta)$ is defined as in~\eqref{eq:def_gamma} and
\begin{align*}
  \chi_{1,\beta}(\*H,\boldsymbol{\tilde{\mathbf{H}}}) &=
  \begin{cases*}
    \frac{\boldsymbol{\tilde{\mathbf{H}}}^{.(2-\beta)}}{\*H^{.(1-\beta)}} & if $\beta \leq 2$ \\
    \*H                                 & if $\beta > 2$ ,
  \end{cases*} \\
  \chi_{2,\beta}(\*H,\boldsymbol{\tilde{\mathbf{H}}}) &=
  \begin{cases*}
    \*H                             & if $\beta < 1$ \\
    \frac{\*H^{.\beta}}{\boldsymbol{\tilde{\mathbf{H}}}^{.(\beta-1)}} & if $\beta \geq 1$ .
  \end{cases*}
\end{align*}
Although our proposed majorization step is performed jointly, we thus still need
to resort to alternate updates of $\*W$ and $\*H$ to minimize $G$.
By doing so and by construction, the JMM updates given by~\eqref{eq:jnt_update}
would coincide with the BMM updates given by~\eqref{eq:alt_update} if we were to
set $\boldsymbol{\tilde{\mathbf{W}}}$ to $\*W$ and $\boldsymbol{\tilde{\mathbf{H}}}$ to $\*H$.
Note however that the JMM updates are conceptually different because the JMM
updates aim at minimizing $G$ given $\boldsymbol{\tilde{\mathbf{W}}}, \boldsymbol{\tilde{\mathbf{H}}}$ while the BMM updates
aim at minimizing $D_{\beta}(\*V|\*W \*H)$ w.r.t $\*H$ given $\*W$, or w.r.t $\*W$
given $\*H$.
Implementation details and further comparisons are given in the next section.

\subsection{Discussion}
\label{ssec:benef_jmm}

\subsubsection{Implementation of the JMM updates}

At every iteration, we need to initialize the updates~\eqref{eq:jnt_update} given
the current iterates $(\boldsymbol{\tilde{\mathbf{W}}}, \boldsymbol{\tilde{\mathbf{H}}}$).
A simple and natural choice is to
\begin{enumerate*}
\item initialize $\*H$ to $\boldsymbol{\tilde{\mathbf{H}}}$,
\item update $\*W$, and
\item update $\*H$.
\end{enumerate*}
Conceptually, steps 2) and 3) need to be applied several times to fully
minimize $(\*W,\*H) \mapsto G(\*W,\*H | \boldsymbol{\tilde{\mathbf{W}}},\boldsymbol{\tilde{\mathbf{H}}})$ (without updating or
recomputing the tilde matrices).
Nevertheless, this does not turn out particularly useful in practice.
Indeed, only the first iteration of~\eqref{eq:jnt_update} results in a
significant decrease of $G$ and we observe that the additional iterations yield
only negligible improvement.
As such, like in traditional NMF practice, we only use one sub-iteration of
$\*W$ and $\*H$ in practice.
By doing so and initializing $\*H$ to $\boldsymbol{\tilde{\mathbf{H}}}$, the BMM and JMM updates of
$\*W$ coincide and only the update of $\*H$ changes (with still a significant
gain in performance).
Note that we have observed empirically that inverting the order of the updates
in~\eqref{eq:jnt_update} (and initializing $\*W$ to $\boldsymbol{\tilde{\mathbf{W}}}$) does not have any
impact on the number of iterations before convergence nor in the quality of the
obtained solutions.

\subsubsection{Special cases}

For the values of notorious importance $0$, $1$ and $2$ of $\beta$, the
multiplicative update~\eqref{eq:jnt_update} can be written in a simpler form as
some of the exponents cancel out and make the corresponding terms vanish.
Similar simplified updates are also available for the classic update
rules~\eqref{eq:alt_update}.
These simplified formulae are shown in Table~\ref{table:simpler_forms} for the
factor matrix $\*H$, where the matrix $\*1$ represents the matrix of dimension
$F \times N$ whose all entries equal $1$.

\begin{table}[t]
  \caption{Simplified updates of $\*H$ for $\beta=0,1,2$.}
  \begin{center}
    \setlength\tabcolsep{5.5pt}
    \resizebox{\columnwidth}{!}{\begin{tabular}{cll}
      \toprule
      $\beta$ & \multicolumn{1}{c}{BMM} & \multicolumn{1}{c}{JMM} \\
      \midrule
      \midrule
      $0$ & $\*H \; \leftarrow\;\*H.{\left(\frac{\*W^{\top}\frac{\*V}{{(\*W\*H)}^{.(2)}}}
            {\*W^{\top}\frac{1}{\*W\*H}}\right)}^{.\frac{1}{2}}$
          & $\*H \; \leftarrow \;\boldsymbol{\tilde{\mathbf{H}}}.{\left(\frac{{\left(\frac{\boldsymbol{\tilde{\mathbf{W}}}^{.2}}{\*W}\right)}^{\top}
          \frac{\*V}{{(\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}})}^{.2}}
            }
            {\*W^{\top}\frac{1}{\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}}}}\right)}^{.\frac{1}{2}}$ \\
      \midrule
      $1$ & $\*H \; \leftarrow \;\*H.\left(\frac{\*W^{\top}\frac{\*V}{\*W\*H}}{\*W^{\top}\*1}\right)$
          & $\*H \; \leftarrow \;\boldsymbol{\tilde{\mathbf{H}}}.\left(\frac{\boldsymbol{\tilde{\mathbf{W}}}^{\top}\frac{\*V}{\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}}}}
            {\*W^{\top}\*1}\right)$ \\
      \midrule
      $2$ & $\*H \; \leftarrow \;\*H.\left(\frac{\*W^{\top}\*V}{\*W^{\top}\*W\*H}\right)$
          & $\*H \; \leftarrow \;\boldsymbol{\tilde{\mathbf{H}}}.\left(\frac{\*W^{\top}\*V}
            {{\left(\frac{\*W^{.2}}{\boldsymbol{\tilde{\mathbf{W}}}}\right)}^{\top}\boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}}}\right)$ \\
      \bottomrule
    \end{tabular}}
  \end{center}
  \label{table:simpler_forms}
\end{table}

\subsubsection{Computational advantages of JMM}
\label{ssec:benef_jmm}

The new update rules~\eqref{eq:jnt_update} are quite similar to the classic
multiplicative updates~\eqref{eq:alt_update} except for the matrices $\boldsymbol{\tilde{\mathbf{W}}}$,
$\boldsymbol{\tilde{\mathbf{H}}}$, and $\boldsymbol{\tilde{\mathbf{V}}}$.
As opposed to BMM, these matrices remain constant and allow for some
computational savings when updating $\*H$ (remember that the BMM and JMM updates
of $\*W$ coincide given our particular implementation choices).
For instance, the computation of $\boldsymbol{\tilde{\mathbf{V}}}$ is performed only once
in~\eqref{eq:jnt_update} whereas in~\eqref{eq:alt_update} the product $\*W\*H$ has
to be computed twice.
Our proposed update hence saves here a matrix product.
Table~\ref{table:nb_op} summarizes the benefit of JMM for the different values of
$\beta$.
We especially emphasize the values $0$, $1$, and $2$ of $\beta$, which enjoy even
larger computational savings thanks to the simplifications shown in
Table~\ref{table:simpler_forms}.
Since these three cases are the most common in practice, the
update~\eqref{eq:jnt_update} brings a very welcome speedup compared
with~\eqref{eq:alt_update}.
It turns out that the largest saving is in the case where $\beta$ is equal to $0$,
i.e., the IS divergence.
For other general values of $\beta$, the JMM updates incur extra divisions
in~\eqref{eq:jnt_update} which mitigate the computational savings.

\begin{table}[t]
  \caption{Computational savings brought by JMM compared to BMM in number of
    operations per iteration.}
  \begin{center}
    \setlength\tabcolsep{5.5pt}
    \begin{tabular}{p{0.2\columnwidth}p{0.2\columnwidth}p{0.2\columnwidth}
        p{0.2\columnwidth}}
      \toprule
               &  Additions &   Multiplications & Divisions \\
      \midrule
      $\beta=0$    & $4FN(K-1)$ & $FN(3+4K)-K(F+N)$ &     $2FN$ \\
      $\beta=1$    &  $FN(K-1)$ &             $FNK$ &      $FN$ \\
      $\beta=2$    &  $FN(K-1)$ &             $FNK$ &       $0$ \\
      \midrule
      $\beta\in]1,2[$     & $FN(K-1)$ & $FNK$ & $\text{$-FN-2KF$}-2KN$ \\
      $\beta\notin[1,2]\cup\{0\}$ & $FN(K-1)$ & $FNK$ & $-FN-KF-KN$            \\
      \bottomrule
      \end{tabular}
  \end{center}
  \label{table:nb_op}
\end{table}

\subsubsection{Failure of heuristic updates}

A common heuristic consists in setting $\gamma(\beta)$ to $1$ in the
update~\eqref{eq:alt_update}, even for values of $\beta$ outside $[1,2]$.
Indeed, it has empirically been observed that it leads to a similar
factorization while decreasing the number of iterations to reach convergence.
Moreover, for values of $\beta$ in $[0,1]$, the authors
in~\cite{Fevotte_C_2011_j-neural-comput_algorithms_nmfbd} have proved that this
heuristic corresponds to a Majorization-Equalization scheme which produces
larger steps than the MM method.
Nevertheless, deriving theoretical support for this heuristic for other values
of $\beta$ is still an open problem.
Setting $\gamma(\beta) = 1$ for all $\beta$ did not lead to similar findings for JMM\@.
While we did observe that the objective function decreases at every iteration
under the heuristic, worse solutions were obtained (i.e., corresponding to
higher values of the objective function in general).
This might be due to the nonconvexity of the JMM auxiliary function $G$, in
which case the Majorization-Equalization principle makes less sense.

\section{Experimental Results}
\label{sec:simul}

We provide in this section extensive numerical comparisons of BMM and JMM using
various datasets (face images, audio spectrograms, song play-counts,
hyperspectral images) with diverse dimensions.

\subsection{Set-up}

\subsubsection{Implementation details}

Our implementation for JMM is sketched in Algorithm~\ref{algo:jmm}.
That of BMM is similar, except for the update of $\*W$ and $\*H$
where~\eqref{eq:alt_update} is used instead of~\eqref{eq:jnt_update}.
The initializations $(\*W^{(0)},\*H^{(0)})$ for BMM and JMM are drawn randomly
according to a half-normal distribution.
Our stopping criterion for both algorithms is based on the relative decrease in
the objective function $D_{\beta}$.
More precisely, the algorithms are stopped when
\begin{equation}
  \label{eq:stop_crit}
  \frac{D_{\beta}(\*V \mid \boldsymbol{\tilde{\mathbf{W}}}^{\top}\boldsymbol{\tilde{\mathbf{H}}})
    -D_{\beta}(\*V \mid \*W^{\top}\*H)}
    {D_{\beta}(\*V \mid \*W^{\top}\*H)}
  \leq \epsilon \; ,
\end{equation}
where $\epsilon$ is a tolerance set to $10^{-5}$, $\*W$ and $\*H$ are the current
iterates of the factor matrices while $\boldsymbol{\tilde{\mathbf{W}}}$ and $\boldsymbol{\tilde{\mathbf{H}}}$ are the previous
ones.
Furthermore, in order to remove the scaling ambiguity inherent to NMF, we
normalize $\*W$ and $\*H$ at the end of each iteration for both BMM and JMM\@.
The normalization consists in dividing each column of $\*W$ by its $\ell_{2}$ norm and
scaling the rows of $\*H$ accordingly.
\begin{algorithm}[t]
  \begin{algorithmic}[1]
    \renewcommand{\algorithmicrequire}{\textbf{Input:}}
    \renewcommand{\algorithmicensure}{\textbf{Output:}}
    \REQUIRE a data matrix $\*V$ and an initial point
    $(\*W^{(0)},\*H^{(0)})$
    \ENSURE an NMF of $\*V$
    \STATE{Initialize $(\*W,\*H)$ to $(\*W^{(0)},\*H^{(0)})$}
    \WHILE{\NOT~\eqref{eq:stop_crit}}
    \STATE{Set $(\boldsymbol{\tilde{\mathbf{W}}},\boldsymbol{\tilde{\mathbf{H}}})$ to $(\*W,\*H)$}
    \STATE{Update $\*W$ using~\eqref{eq:jnt_update}}
    \STATE{Update $\*H$ using~\eqref{eq:jnt_update}}
    \STATE{Normalize the columns of $\*W$ and scale $\*H$ accordingly}
    \ENDWHILE
    \RETURN $(\*W,\*H)$
  \end{algorithmic}
  \caption{Sketch for the JMM algorithm~\label{algo:jmm}.}
\end{algorithm}

\subsubsection{Handling zero values and numerical stability}

The $\beta$-divergence $D_{\beta}(x|y)$ may not be defined when either $x$ or $y$ takes
zero values.
This is for instance the case when $\beta$ is set to $0$ or $1$ due to the quotient
and the logarithm that appear.
As such, we recommend minimizing $D_{\beta}(\*V+\kappa|\*W\*H+\kappa)$ with a small
constant $\kappa$ instead of $D_{\beta}(\*V|\*W\*H)$ for numerical stability.
This first simply amounts to replacing $\*V$ by $\*V + \kappa$ in the previous
derivations.
Then, by treating $\kappa$ as a ${(K + 1)}^{\text{th}}$ constant component like
in~\cite{Fevotte_C_2014_PP_nonlinear_hurnmf}, we may simply replace the
definition of $\boldsymbol{\tilde{\mathbf{V}}}$ by $\boldsymbol{\tilde{\mathbf{V}}} = \boldsymbol{\tilde{\mathbf{W}}}\boldsymbol{\tilde{\mathbf{H}}} + \kappa$ in the
derivation of~\eqref{eq:jnt_update} to obtain a valid MM algorithm.

\subsubsection{Simulation environment}

All the simulations have been conducted in Matlab 2020a running on an Intel
i7-8650U CPU with a clock cycle of 1.90GHz shipped with 16GB of memory.
\footnote{Matlab code will be made publicly available at the time of
  publication.}
In each of the following experimental scenarii, we compare the factorization
obtained by JMM and BMM from $25$ different initializations
$(\*W^{(0)},\*H^{(0)})$.

\subsubsection{Performance evaluation}

We compare BMM and JMM both in terms of computational efficiency (CPU time)
and quality of the returned solutions.
To assess the latter, we use the value of the normalized objective function
$\frac{D_{\beta}(\*V|\widehat{\*W}^{\top}\widehat{\*H})}{FN}$ at the solution
$(\widehat{\*W},\widehat{\*H})$ returned by the NMF algorithms.
We also consider the KKT
residuals~\cite{Fevotte_C_2011_j-neural-comput_algorithms_nmfbd} to measure the
distance to the first order optimality conditions and attest whether the
algorithms reached a stationary point of the criterion $D_{\beta}$.
The KKT residuals are expressed by
\begin{align*}
  \text{res}(\*W)
  &= \frac{\norm{\min\{\*W, [{(\*W\*H)}^{.(\beta-2)}.(\*W\*H-\*V)]\*H^{\top}\}}_{1}}{FK}
  \\
  \text{res}(\*H)
  &= \frac{\norm{\min\{\*H, \*W^{\top}[{(\*W\*H)}^{.(\beta-2)}.(\*W\*H-\*V)]\}}_{1}}{KN}
  \, .
\end{align*}

\subsection{Factorization of face image datasets}

In the context of image processing, NMF can be used to learn part-based features
from a collection of images~\cite{Lee_D_1999_j-nature_learning_ponmf}.
The columns of the matrix $\*V$ correspond to the vectorization of the different
images.
Besides, the factor $\*W$ represents the dictionary of image features, and the
matrix $\*H$ contains the activation encodings.

We compare the BMM and JMM methods on two face images datasets:
\begin{enumerate}
\item the CBCL dataset~\cite{Sung_K_1996_phd_learning_esopr} which contains $2429$
  images of faces of dimensions $19 \times 19$.
  Each image is stored as a column vector of dimension $361$ in the matrix
  $\*V$.
  Similarly to~\cite{Lee_D_1999_j-nature_learning_ponmf}, we preprocessed the
  data by performing a mean centering and contrast (variance) normalization,
  and set $K=49$.
\item the Olivetti dataset from AT\&T Laboratories
  Cambridge~\cite{Samaria_F_1994_p-wacv_parametrisation_smhfi} which contains
  $400$ greyscale images of faces with dimensions $64 \times 64$.
  The corresponding matrix $\*V$ thereby has dimensions $4096 \times 400$.
  We set $K$ to either $5$ or $10$.
\end{enumerate}
We consider the values $0$, $1$, and $2$ for the parameter $\beta$, which
correspond respectively to IS divergence, KL divergence, and squared Euclidean
distance (the latter two being the most common in image processing).

Figures~\ref{fig:cbcl_results} and~\ref{fig:oliv_results_K=5} show the computation
times for $\beta =2$ and $\beta=1$, as well as the values of the normalized objective
function produced by both BMM and JMM on the two datasets, for the $25$ runs.
The same random initialization is used by both methods.
Note that, for the sake of clarity, we use a logarithmic scale for the CPU time.
Moreover, the scale for the objective function is zoomed in and does not start
at zero.
We observe that JMM is always faster than BMM while yielding solutions with a
similar quality in terms of the objective $D_{\beta}$.
The corresponding average CPU time as well as the resulting acceleration ratios
are given in Tables~\ref{table:cbcl_avgtime} and~\ref{table:oliv_avgtime}.
We notice that the computational saving is higher when using the IS divergence,
which confirms our analysis from Section~\ref{ssec:benef_jmm}.
Note that while MM algorithms are prevalent for $\beta$-NMF in general, many other
algorithms have been designed for the specific case $\beta =2$
(quadratic loss)~\cite{Gillis_N_2020_book_nonnegative_mf}.
In that case, some algorithms are notoriously more efficient than BMM,
e.g.,~\cite{Guan_N_2012_j-ieee-trans-sig-proc_nenmf_ogmnmf}.
Though JMM improves on BMM, it may not compete with these other algorithms.
Still, JMM, like BMM, is free of hyper-parameters and very easy to use, and
remains a very convenient option for the general practitioner.

\begin{figure}[!t]
  \centering
  \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{cbcl-kl}
    \caption{Using KL divergence ($\beta=1$).}
  \end{subfigure}
  \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{cbcl-eucl}
    \caption{Using squared Euclidean distance ($\beta=2$).}
  \end{subfigure}
  \caption{Comparative performance using the CBCL face dataset.}
  ~\label{fig:cbcl_results}
\end{figure}

\begin{figure}[!t]
  \centering
  \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{oliv-kl_K=5}
    \caption{Using KL divergence ($\beta=1$).}
  \end{subfigure}
  \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{oliv-eucl_K=5}
    \caption{Using squared Euclidean distance ($\beta=2$).}
  \end{subfigure}
  \caption{Comparative performance using the Olivetti dataset ($K=5$).}
  ~\label{fig:oliv_results_K=5}
\end{figure}

\begin{table}[t]
  \caption{Average CPU times for the CBCL dataset.}
  \begin{center}
    \setlength\tabcolsep{5.5pt}
    \begin{tabular}{cccc}
      \toprule
      & BMM & JMM & Acceleration \\
      \midrule
      $\beta=2$ & 106s & 55s & 48.50\% \\
      $\beta=1$ &  61s & 45s & 24.96\% \\
      $\beta=0$ &  62s &  6s & 91.14\% \\
      \bottomrule
    \end{tabular}
  \end{center}
  \label{table:cbcl_avgtime}
\end{table}

\begin{table}[t]
  \caption{Average CPU times for the Olivetti dataset
    \\\hspace{\textwidth}($K=5/K=10$).}
  \begin{center}
    \setlength\tabcolsep{5.5pt}
    \begin{tabular}{cccc}
      \toprule
      & BMM & JMM & Acceleration \\
      \midrule
      $\beta=2$ &  31s / 55s  & 23s / 36s & 33\% / 35\% \\
      $\beta=1$ &  22s / 40s  & 20s / 34s & 10\% / 16\% \\
      $\beta=0$ & 121s / 229s & 35s / 63s & 71\% / 72\% \\
      \bottomrule
    \end{tabular}
  \end{center}
  \label{table:oliv_avgtime}
\end{table}

Finally, we give in Table~\ref{table:cbcl_kkt} some statistics of the KKT
residuals for the CBCL dataset.
We observe here that both methods converge to a stationary point of $D_{\beta}$.
Actually, an additional significant observation is that in all cases here, both
JMM and BMM return the same solution $(\hat{\*W},\hat{\*H})$ up to
permutation of their columns and up to some round-off errors.
Nevertheless, the trajectory of the iterates may differ.
Similar results are observed for the Olivetti faces dataset.

\begin{table}[t]
  \caption{Statistics of the KKT residuals for CBCL dataset.}
  \begin{center}
    \setlength\tabcolsep{5.5pt}
    \begin{tabular}{cccccc}
      \toprule
      && \multicolumn{2}{c}{BMM} & \multicolumn{2}{c}{JMM} \\
      &&      $\*W$ &      $\*H$ &      $\*W$ &      $\*H$ \\
      \midrule
      \multirow{3}{*}{$\beta=2$}
      &Min. & $4.10^{-3}$ & $8.10^{-6}$ & $4.10^{-3}$ & $8.10^{-6}$ \\
      &Max. & $6.10^{-3}$ & $3.10^{-5}$ & $6.10^{-3}$ & $2.10^{-5}$ \\
      &Med. & $5.10^{-3}$ & $1.10^{-5}$ & $5.10^{-3}$ & $1.10^{-5}$ \\
      \multirow{3}{*}{$\beta=1$}
      &Min. & $2.10^{-2}$ & $2.10^{-4}$ & $4.10^{-2}$ & $3.10^{-4}$ \\
      &Max. & $2.10^{-1}$ & $5.10^{-3}$ & $2.10^{-1}$ & $5.10^{-3}$ \\
      &Med. & $6.10^{-2}$ & $9.10^{-4}$ & $6.10^{-2}$ & $3.10^{-4}$ \\
      \multirow{3}{*}{$\beta=0$}
      &Min. & $3.10^{-1}$ & $1.10^{-3}$ & $2.10^{-1}$ & $4.10^{-4}$ \\
      &Max. & $4.10^{-1}$ & $2.10^{-3}$ & $6.10^{-1}$ & $5.10^{-4}$ \\
      &Med. & $3.10^{-1}$ & $1.10^{-3}$ & $3.10^{-1}$ & $4.10^{-4}$ \\
      \bottomrule
    \end{tabular}
  \end{center}
  \label{table:cbcl_kkt}
\end{table}

\subsection{Factorization of spectrograms}

We now consider NMF of audio magnitude spectrograms.
In this context the factor $\*W$ contains elementary audio spectra with temporal
activations given by
$\*H$~\cite{Smaragdis_P_2014_j-ieee-sig-proc-mag_static_dssunmfuv}.

We generate the spectrogram for two extracts from two different pieces: a piano
recording of the first movement of the sonata K331 from Mozart and the original
recording of the song ``Four on Six'' by Wes Montgomery.
The extracts are $5$-seconds long with a sampling rate of
\SI{44.1}{\kilo\hertz}.
The spectrogram is computed with a Hamming window of length $1024$ (23ms) and an
overlap of $50\%$.
This results in a data matrix $\*V$ of dimensions $513 \times 858$.

In this section, we set $\beta=0$, which is a common value in audio signal
processing~\cite{Fevotte_C_2009_j-neural-comput_nonnegative_mfisdama}, and set
$K$ to either $5$ or $10$.
The results obtained with BMM and JMM for $K=5$ are shown on
Figure~\ref{fig:spectro_res_K=5}.
The average computation times and acceleration ratios are summarized in
Table~\ref{table:spectro_avgtime}.
We observe a tremendous benefit of JMM over BMM in terms of CPU time in this
case.
This confirms our conclusion from Section~\ref{ssec:benef_jmm} that JMM reaches
its highest potential with NMF based on IS divergence.
We observe here again that JMM and BMM return the same solution
$(\hat{\*W},\hat{\*H})$ up to permutation of their columns.

\begin{figure}[!t]
  \centering
   \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{spectro_4on6_K=5}
    \caption{Spectrogram extracted from the song ``Four on Six''.}
  \end{subfigure}
  \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{spectro_K331_K=5}
    \caption{Spectrogram extracted from a recording of the sonata K331.}
  \end{subfigure}
  \caption{Comparative performance with spectrograms
    \\\hspace{\textwidth}($K=5,\beta=0$).}
  ~\label{fig:spectro_res_K=5}
\end{figure}

\begin{table}[t]
  \caption{Average CPU times for two spectrograms
    \\\hspace{\textwidth}($K=5/K=10$).}
    \begin{center}
      \setlength\tabcolsep{5.5pt}
      \begin{tabular}{cccc}
        \toprule
        & BMM & JMM & Acceleration \\
        \midrule
        Spectrogram 1 & 22s / 21s & 6s / 5s & 74\% / 75\% \\
        Spectrogram 2 & 36s / 20s & 9s / 7s & 78\% / 74\% \\
        \bottomrule
      \end{tabular}
    \end{center}
    \label{table:spectro_avgtime}
\end{table}

\subsection{Factorization of song play-counts}

NMF may be used in recommendation systems based on implicit user data.
In this case, the matrix $\*V$ contains information about the interactions of
users with a collection of items.
The factor $\*W$ may extract user preferences while $\*H$ represents item
attributes, see, e.g.,~\cite{Hu_Y_2008_p-ieee-icdm_collaborative_fifd}.

We here consider the
TasteProfile dataset~\cite{Bertin-Mahieux_T_2011_book_million_sd} which contains
counts of songs played by users (e.g., of a music streaming service).
Similarly to~\cite{Gouvert_O_2020_p-icml_ordinal_nmdr} and many other papers using
this dataset, we apply a preprocessing to the original data to keep only users
and songs with more than a given number of interactions (here set to 20).
This results in a large and sparse data matrix $\*V$ of dimensions
$16\,301 \times 12\,118$ with only $1\,176\,086$ nonzero elements, i.e., 0.6\% of the
total number of coefficients.
We set $\beta$ to $1$ (a common choice in recommender systems, because its
corresponds to likelihood of a Poisson model that is natural for count data),
and $K=50$.

Comparative results are displayed in Figure~\ref{fig:tasteprof_res}.
We observe that on average, JMM is $12.80\%$ faster with an average CPU time of
1 hour 38 minutes whereas BMM's average time is equal to 1 hour 58 minutes.
Finally, like in previous scenarii, JMM and BMM return the same solution
$(\hat{\*W},\hat{\*H})$ up to a permutation of their columns.

\begin{figure}[!t]
  \centering
  \includegraphics[width=\linewidth]{tasteprofile_K=50}
  \caption{Comparative performance with TasteProfile
    \\\hspace{\textwidth}($K=50,\beta=1$).}
  ~\label{fig:tasteprof_res}
\end{figure}

\subsection{Factorization of hyperspectral images}

A hyperspectral image is a multi-band image that can be represented by a
nonnegative matrix: each row represents a spectral band while each column
represents a pixel of the image.
Applying NMF to such matrix data allows extracting a collection of individual
spectra representing the different materials arranged in the matrix $\*W$, as
well as their relative proportions given by the matrix $\*H$.

We consider a hyperspectral image acquired over Moffett Field in 1997 by the
Airborne Visible Infrared Imaging Spectrometer~\cite{Aviris_database}.
The image contains $50 \times 50$ pixels over $189$ spectral bands, which leads to a
$\*V$ matrix of dimensions $189 \times 2500$.
We consider $\beta=2$ and $\beta=1.5$.
The latter value in particular was shown to be an interesting trade-off between
Poisson ($\beta=1$) and additive Gaussian ($\beta=2$) assumptions for predicting missing
values in incomplete versions of this dataset,
see~\cite{Fevotte_C_2014_PP_nonlinear_hurnmf}.
The factorization rank $K$ is set to $3$.

Comparative results are given in Figure~\ref{fig:hyperimg_moffet}.
On the top figure corresponding to $\beta$ equals to $2$, we observe that JMM is
faster than BMM with an acceleration of $38.58\%$ on average.
On the other hand, for $\beta=1.5$, we observe that BMM is faster in general (though
not always).
This confirms our analysis of Section~\ref{ssec:benef_jmm}: when $\beta$ is
different from $0$, $1$, and $2$, the benefit of JMM is counterbalanced by the
additional divisions in the general formulae~\eqref{eq:jnt_update}.

\begin{figure}[!t]
  \centering
  \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{hyp_mof_beta2}
    \caption{$\beta=2$.}
  \end{subfigure}
  \begin{subfigure}[b]{\columnwidth}
    \includegraphics[width=\linewidth]{hyp_mof_beta15}
    \caption{$\beta=1.5$.}
  \end{subfigure}
  \caption{Comparative performance with Moffet hyperspectral image ($K=3$).}
  ~\label{fig:hyperimg_moffet}
\end{figure}

\section{Conclusion}
\label{sec:concl}

In this paper, we have presented a joint MM method for NMF\@.
More specifically, we have designed a non-convex tight auxiliary function for
NMF with the $\beta$-divergence.
Alternate minimization of the auxiliary function leads to new multiplicative
updates of the factor matrices.
Despite being a somewhat simple modification of the classic multiplicative
updates, our proposed updates lead to a significant computational speedup,
especially for the Itakura-Saito and Kullback Leibler divergences, and the
squared Euclidean distance.
Fortunately, the three later cases are the three most common cases of $\beta$-NMF\@.
The new updates guarantee the descent property for the objective function.
Moreover, we have observed experimentally that the estimated factors are of the
same quality as the ones returned by the classic block MM scheme.
As a matter of fact, we have noted that both algorithms usually return the same
solutions up to column permutation.
The computational efficiency of the proposed updates has been  demonstrated on
datasets with diverse characteristics.

\section*{Acknowledgment}
The authors acknowledge J{\'e}r{\^o}me Idier and R{\'e}mi Flamary for discussions
related to this work.

\bibliographystyle{IEEEtran}

redpajama_set_name: RedPajamaArXiv


Dr. John Villacis, an ADC allergist and immunologist, says one of the best ways to combat allergies is to be prepared by watching pollen count levels.
"Patients all the time ask me, 'What can I do during the allergy season? How can I better prepare myself?' So one of the tips I usually recommend is to watch your local news," he said.
Many stations in Central Texas offer daily pollen count information, which can help you plan your day.


An empowering declaration for 'You'.
Inspired by the This is 'Me' collection. This Is 'You' continues the story from our bespoke conversational headpieces. This range of ready to wear hair accessory pieces continue the conversation to a wider audience.


text:
Military research chair awarded
In a ceremony held yesterday, Mr. Chris Alexander, Parliamentary Secretary to the Minister of National Defence, presented Colonel Homer Tien, Medical Director of our Tory Regional Trauma Centre at Sunnybrook, with the Canadian Forces Major Sir Frederick Banting Term Chair in Military Trauma Research.
Dr. Tien, who is also an associate scientist at Sunnybrook Research Institute, has served in Bosnia with the Stabilization Force, in Kabul with the International Security Assistance Force, and has deployed multiple times to the Role 3 Multinational Medical Unit (MMU) in Kandahar. He was the first Canadian Forces trauma surgeon to serve at Role 3 MMU in January 2006, and was also the last to serve there in November 2011.
The Military Trauma Research Chair will provide Dr. Tien with funding to continue his research in trauma medicine that will better serve the Canadian Forces (CF) and civilian populations. Past CF research into the management of blood flow and multiple trauma contributed to a 97 percent survival rate for CF soldiers, sailors, airmen and airwomen who were evacuated to the Role 3 Multinational Medical Unit in Kandahar with vital signs.
We are committed to the health of military personnel and have worked with the CF to ensure that ill and injured personnel receive high standard health care. Sunnybrook and our affiliate the University of Toronto are further committed to military and veteran health research through our participation in the Canadian Institute for Military and Veteran Health Research.
Introducing SPARK: The Sunnybrook Program to Access Research Knowledge for Black and Indigenous Medical Students
Sunnybrook researchers awarded more than $6M in CIHR clinical trials funding
INOVAIT invests in the Canadian image-guided therapy sector through its Focus Fund


In this comparison article for Zignature vs Purina Pro Plan, we'll highlight the key differences between these two pet food brands. To properly compare Zignature and Purina Pro Plan, we'll use up-to-date nutritional and price information.
Throughout this Purina Pro Plan vs Zignature comparison, we've utilized average data to make general comparisons. If you'd like to see individual product reviews, visit our Zignature Review Page or Purina Pro Plan Review Page.
There is a relatively insignificant difference between the crude protein content of the two brands. For wet dog foods, Purina Pro Plan typically provides a little more protein (about 3.55% more).
Both brands provide roughly the same amount of crude fat. For wet dog foods, Zignature provides significantly more fat (about 8.57% more).
As you can see, Purina Pro Plan and Zignature guarantee a similar amount of crude fiber. With regards to wet dog food, Purina Pro Plan and Zignature also provide roughly the same amount of crude fiber.
Product Safety: Recall History of Zignature & Purina Pro Plan?
According to our records, Zignature has never been recalled.
If you'd like to calculate the average cost-per-day or cost-pet-year of feeding Zignature or Purina Pro Plan, input your pet's weight into our cost analyzer widget.
Averages price data can identify whether or not there exists a significant price difference between two brands. However, the actual price that you'll pay for Zignature or Purina Pro Plan will depend on your shopping habits.
Pet owners who use our price comparison feature typically save more than 25% off retail prices for Zignature and Purina Pro Plan. In addition, we currently have some great coupons for our readers.


S 12" 31.5" 28" 35.5" 54"
M 12.5" 33.5" 29.5" 37.5" 54"
L 13" 35" 31" 39" 54.5"
XL 13" 36.5" 32.5" 40.5" 55"
For a look that will make his blood burn wear this x-rated solid black jumpsuit in sheer striped solid black stretch fabric. With a mock turtle neck and traditional sleeveless arm openings, it features a side zipper and an attached black stretch fabric waist band that looks like a belt. Its close but not tight fitting bodice makes the most of your assets and leaves literally nothing to the imagination. The pants are fitted casually through the hips and taper their way down to the ankle. Add black stiletto heels and red lipstick and he won't be able to say "no". Hand wash. Drip dry.


all in one bathroom bathroom decor sets.
all in one bathroom all in one shower toilet and sink google search bathroom remodel.
all in one bathroom compact bathrooms main bathroom remodel near me.
all in one bathroom hotel shower bathroom all in one bathroom vanities menards.
all in one bathroom nice all in one bathroom sink and custom bathroom s new of creative bathroom bathroom decor ideas.
all in one bathroom all in one bathroom units prefab bathroom n find complete details about all in one bathroom units prefab bathroom n prefab bathroom bathroom sinks lowes.
all in one bathroom green homes sustainable homes homes solar wind power water systems dome homes shopping home bathroom light fixtures brushed nickel.
all in one bathroom adorable all in one bathroom sink and applied to your home decor shop bathroom bathroom remodel near me.
all in one bathroom china modular bathroom pod with shower and toilet all in one bathroom mirrors lowes.
all in one bathroom kitchen cabinet design flat ideas modern height dimensions all in one bathroom artistic wall decorations gorgeous bathroom decor walmart.
all in one bathroom more info bathroom cabinets for sale.
all in one bathroom full size of kitchen cabinets sale cabinet ideas appliances small white all in bathroom mirrors.
all in one bathroom all in one bathroom units prefab bathroom integrated bathroom suit cabin unit set bathroom remodel near me.
all in one bathroom vanity pro all in one in w bathroom vanity in chestnut with bathroom ideas color.
all in one bathroom small bathroom faucets amazon.
all in one bathroom medium size of bathroom bathroom sinks on top bathroom sinks wall mounted trough sinks for bathroom ideas 2018.
all in one bathroom minimalist all in one magic bathroom cleaner plus to the virtual minimalist party minimalist book orders bathroom cabinets lowes.
all in one bathroom shipping container homes small home living steel containers off the grid self sufficient homestead container house bathroom sink drain parts.
all in one bathroom bath tub and shower all in one bathroom vanities menards.
all in one bathroom bathroom toilet sink shower all in bathroom remodel diy.
all in one bathroom bathroom toilet shower sink all in one room floor drain bathroom mirrors for sale.
all in one bathroom bathroom sink.
all in one bathroom bathroom bathroom light fixtures.
all in one bathroom unique bathroom vanities clearance.
all in one bathroom china modular bathroom pod prefab all in one bathroom bathroom decor sets.
all in one bathroom wood bathroom bathroom remodel diy.
all in one bathroom integrate bathroom pod bathroom sink vanity.
all in one bathroom large size of bathroom affordable bathroom vanities all in one bathroom vanity antique bathroom vanity bathroom bathroom ideas small.
all in one bathroom shower bathroom faucets.
all in one bathroom one piece bathroom sink and built in sink all in one sink and one piece one piece bathroom bathroom cabinets lowes.
all in one bathroom vanity pro all in one bathroom decor pinterest.
all in one bathroom contemporary bathroom ceramic precast for professional use camping bathroom tiny bathrooms and tiny house bathroom bathroom mirrors.
all in one bathroom all in one bathroom units prefab bathroom buy prefab toilet in one bathroom units product on bathroom sink cabinets.


Product prices and availability are accurate as of 2019-04-20 07:17:03 UTC and are subject to change. Any price and availability information displayed on http://www.amazon.com/ at the time of purchase will apply to the purchase of this product.
We are delighted to present the brilliant TrailHeads Race Day Performance Running Cap | The lightweight, quick dry, sport cap for men – 7 Colors.
With so many on offer these days, it is wise to have a make you can trust. The TrailHeads Race Day Performance Running Cap | The lightweight, quick dry, sport cap for men – 7 Colors is certainly that and will be a excellent purchase.
For this reduced price, the TrailHeads Race Day Performance Running Cap | The lightweight, quick dry, sport cap for men – 7 Colors comes widely respected and is a popular choice for most people. TrailHeads have included some excellent touches and this equals great value for money.
Great for running, hiking, tennis and other outdoor activities the TrailHeads Race Day Performance Running Cap is the perfect exercise companion in warm weather. This quick dry, sport cap for men is a one-size-fits-most workout accessory that is useful for exercising and outdoor adventures.
It has an adjustable Velcro closure to fit a variety of head sizes. Reflective accents on the brim and the Velcro tab add an element of safety and visibility to low light workouts or evening walks. This baseball-style cap is very comfortable and lightweight. The light polyester mesh fabric allows air to flow through freely, keeping you from overheating. The dark underbill of the cap reduces glare, sparing your eyes from strong sunlight.
The simple design of the Race Day Running Cap fits with any casual or workout attire. This performance cap is machine washable, delicate cycle, and air dries quickly for hassle-free cleaning. Designed for men and appropriate for some women; one size fits most.
A great running hat, this low profile, unstructured cap is also perfect for golf, tennis, fishing, sailing, daily walks, and more of your favorite summer sports.
LIGHTWEIGHT - A lightweight, sports cap, it weighs only 2 ounces and is available in a variety of colors. The dark under bill keeps your face shaded and reduces glare. It's a hiking and running essential that keeps you comfy without weighing you down. MOISTURE WICKING PERFORMANCE - Quick drying polyester mesh fabric works to keep you dry and comfortable during every workout. The light polyester mesh construction means air flows freely and allows your skin to breathe. ENHANCED VISIBILITY - This reflective running hat features silver accents on the back closure and on the 'sandwich trim' across the brim in addition to a reflective TrailHeads logo on the side. Be safe and be seen while working out! ADJUSTABLE FIT - This one-size-fits-most sports cap for men features an adjustable Velcro closure that delivers a snug, customized, and comfortable fit. Sizing is designed for men but may be appropriate for some women. Machine wash delicate cycle or hand wash in cold water. TRAIL TESTED GUARANTEE: Your complete satisfaction is our goal. If you're not happy with your purchase, let us know and we'll provide a replacement or refund - no fine print, no time limit. Nothing but our commitment to do what it takes to satisfy our customers.


text:
Международное сотрудничество /
Letter to Turkish Embassy
Дата документа: 05.07.2012
Номер документа: 01-06 Л/377
Организация: Профсоюз работников здравоохранения РФ
Направление: Международная работа
Вид документа: Письмо
HEALTH WORKERS UNION OF THE RUSSIAN FEDERATION
№ 01-06 Л/377, 05.07.2012
Dear Ambassador Aydin Adnan Sezgin,
We are writing to you to express our concern over the detention of the President of KESK, Lami Ozgen, as well as Osman ls?i, member of KESK but also Euro-Mediterranean Human rights network executive member and Human Rights association's (IHD) former expert. The first have been released and the later still being jailed.
We would like to protest over the nine women trade unionists who were taken into custody on the morning of 13 February 2012 and as far as we are aware they have not been released. The Health Workers Union of the Russian Federation demands the immediate release of KESK members and leaders who have been detained for more than two months without any legal basis.
This new wave of arrests, officially justified by the anti-terror fight, represents another threat towards Turkish citizens' fundamental freedoms. Along with journalists, lawyers, law makers, academicians, students, trade unionists represent one of the major victims of the present Turkish anti terror-law, which is increasing censorship as well as auto censorship in the matter of freedom of speech, freedom of press and labor rights. Below you will find list of the trade unionists still in jail without any real reason. We would like also to express our concern for the situation of Ismet Arslan, who is in jail while having a skin cancer; we consider that the minimum human standards are not being respected by the Turkish republic.
The Health Workers Union of the Russian Federation strongly condemns the violation of human and trade union rights. We will continue to defend the rights to freedom of association and expression and other fundamental human rights. For this reason, we demand the immediate release of the KESK members who remain in prison and the dropping of all charges against the 15 women who were arrested on 13 February and the arrested trade unionists on June 25th. Finally, we demand the release of the 73 people still arrested.
Mikhail Androchnikov
Vice-President of the Health Worker Union of the Russian Federation
международная работа (2)
зарубежом (2)
предыдущий документ
следующий документ
Приоритеты международной работы
Двустороннее сотрудничество


At Cicerone, we have believed in and have been actively practice sustainable/responsible tourism for years. Our regular guided tours are organised for small groups with a maximum of 12 people.
We are a local company dedicated to caring for our surroundings and offering personalised treatment to our clients. Our guides are local people, they know Granada and its history and love their city, not just for its heritage. We want to transmit this passion to our visitors and that's why some time ago, Cicerone decided to ensure that our groups are as small as possible. This ensures that our clients enjoy a unique experience and at the same time, our activity does not affect the daily functioning of the city and its businesses. That way, we're all winners!
Granada is for everyone, so the locals, visitors and professionals should all live and work alongside each other in the historic centre, always thinking of each other's wellbeing.
Small groups to guarantee that pedestrians and vehicles can pass with ease and to avoid crowds of people and the resulting noise and nuisance.
Our guides are tourism professionals and are trained to give correct, proven information to all our clients. As they're professionals in their knowledge, they are also professional in the way in which they respect our historical and cultural heritage and they'll transmit the importance of this respect to the visitors.
Our routes are all carried out on foot and if we need to use transport to access specific parts of the city, we always use public transport.
We support local businesses and always help our clients to value and encourage the consumption of local produce. We also encourage them to contribute and vary their purchases to prevent favouritism towards certain establishments, including our own.