text:
\section{Image Retrieval with DELF} \label{sec:method}

Our large-scale retrieval system can be decomposed into four main blocks: (i) dense localized
feature extraction, (ii) keypoint selection, (iii) dimensionality reduction and
(iv) indexing and retrieval.
This section describes DELF feature extraction and learning algorithm followed by our indexing and retrieval procedure in detail.

\subsection{Dense Localized Feature Extraction}
\label{sub:dense}
We extract dense features from an image by applying a fully convolutional network (FCN), which is constructed by using the feature extraction layers of a CNN trained with a classification loss.
We employ an FCN taken from the ResNet50~\cite{he2015deep} model, using the output of
the \textsf{conv4\_x} convolutional block.
To handle scale changes, we explicitly construct an image pyramid and apply the FCN for each level independently.
The obtained feature maps are regarded as a dense grid of local descriptors.
Features are localized based on their receptive fields, which can be computed by considering the configuration of convolutional and pooling layers of the FCN.
We use the pixel coordinates of the center of the receptive field as the feature location.
The receptive field size for the image at the original scale is $291\times 291$.
Using the image pyramid, we obtain features that describe image regions of different sizes.

We use the original ResNet50 model trained on ImageNet~\cite{russakovsky2015imagenet} as a baseline, and fine-tune for enhancing the discriminativeness of our local descriptors.
Since we consider a landmark recognition application, we employ annotated datasets of landmark images~\cite{babenko2014neural} and train the network with a standard cross-entropy loss for image classification as illustrated in \figref{fig:train_model}(a).
The input images are initially center-cropped to produce square images and rescaled
to $250\times 250$.
Random $224\times 224$ crops are then used for training.
As a result of training, local descriptors implicitly learn representations that are more relevant for the landmark retrieval problem.
In this manner, neither object- nor patch-level labels are necessary to obtain improved local descriptors.

\begin{figure}[t]
\begin{center}
   \includegraphics[width=0.9\linewidth]{figures/fig2.pdf}
\end{center}
	\vspace{-15pt}
	\caption{The network architectures used for training.}
\label{fig:train_model}
\vspace{-5pt}
\end{figure}

\subsection{Attention-based Keypoint Selection} \label{subsec:attention}

Instead of using densely extracted features directly for image retrieval, we design a technique to effectively select a subset of the features.
Since a substantial part of the densely extracted features are irrelevant to our recognition task and likely to add clutter, distracting the retrieval process, keypoint selection is important for both accuracy and computational efficiency of retrieval systems.

\subsubsection{Learning with Weak Supervision}

We propose to train a landmark classifier with attention to explicitly measure relevance scores for local feature descriptors.
To train the function, features are pooled by a weighted sum, where the weights are predicted by the attention network.
The training procedure is similar to the one described in \secref{sub:dense} including the loss function and datasets, and is illustrated in \figref{fig:train_model}(b), where the attention network is highlighted in yellow.
This generates an embedding for the whole input image, which is then used to train a softmax-based landmark classifier.

More precisely, we formulate the training as follows.
Denote by ${\mathbf{f}_n\in\mathrm{R}^d}, n=1,...,N$ the $d$-dimensional features to be learned jointly with the attention model.
Our goal is to learn a score function $\alpha(\mathbf{f}_n ; \theta)$ for each feature, where $\theta$ denotes the parameters of function $\alpha(\cdot)$.
The output logit $\mathbf{y}$ of the network is generated by a weighted sum of the feature vectors, which is given by
\begin{equation}
	\mathbf{y} = \mathbf{W} \left(
	\sum_{n}\alpha(\mathbf{f}_n ; \theta)\cdot\mathbf{f}_n
	\right),
\end{equation}
where $\mathbf{W}\in\mathrm{R}^{M \times d}$ represents the weights of the final fully-connected layer
of the CNN trained to predict $M$ classes.

For training, we use cross entropy loss, which is given by
\begin{equation}
	\mathcal{L} =
		- \mathbf{y}^*
		\cdot \mathrm{log} \left(
		\frac{
			\mathrm{exp} \left(\mathbf{y}\right)
		}{
			\mathbf{1}^{\text T} \ \mathrm{exp}\left(\mathbf{y}\right)
		}
		\right),
\end{equation}
where $\mathbf{y}^*$ is ground-truth in one-hot representation and $\mathbf{1}$ is one vector.
The parameters in the score function $\alpha(\cdot)$ are trained by backpropagation, where the gradient is given by
\begin{equation}
\begin{split}
\frac
	{\partial \mathcal{L}}
	{\partial \theta}
	=
		\frac
			{\partial \mathcal{L}}
			{\partial \mathbf{y}}
		\sum_{n}
			\frac
				{\partial \mathbf{y}}
				{\partial \alpha_n}
			\frac
				{\partial \alpha_n}
				{\partial \theta}
=
		\frac
			{\partial \mathcal{L}}
			{\partial \mathbf{y}}
		\sum_{n}
			\mathbf{W}
			\hspace{0.03cm}
			\mathbf{f}_n
			\frac
				{\partial \alpha_n}
				{\partial \theta},
\end{split}
\end{equation}
where the backpropagation of the output score $\alpha_n \equiv \alpha(\mathbf{f}_n ; \theta)$ with respect to $\theta$ is same as the standard multi-layer perceptron.

We restrict $\alpha(\cdot)$ to be non-negative, to prevent it from learning negative weighting.
The score function is designed using a 2-layer CNN with a softplus~\cite{dugas2001incorporating} activation at the top.
For simplicity, we employ the convolutional filters of size $1\times 1$, which work well in practice.
Once the attention model is trained, it can be used to assess the relevance of features extracted by our model.

\subsubsection{Training Attention}

In the proposed framework, both the descriptors and the attention model are implicitly learned with image-level labels.
Unfortunately, this poses some challenges to the learning process.
While the feature representation and the score function can be trained jointly by
backpropagation, we found that this setup generates weak models in practice.
Therefore, we employ a two-step training strategy.
First, we learn descriptors with fine-tuning as described in \secref{sub:dense}, and then the score function is learned given the fixed descriptors.

Another improvement to our models is obtained by random image rescaling during attention training process.
This is intuitive, as the attention model should be able to generate effective scores for features at different scales.
In this case, the input images are initially center-cropped to produce square images, and rescaled
to $900\times 900$.
Random $720\times 720$ crops are then extracted and finally randomly scaled with a factor $\gamma \leq 1$.

\subsubsection{Characteristics}

One unconventional aspect of our system is that keypoint selection comes after descriptor extraction, which is different from the existing techniques (\eg, SIFT~\cite{Lowe2004} and LIFT~\cite{yi2016lift}), where keypoints are first detected and later described.
Traditional keypoint detectors focus on detecting keypoints repeatably under different imaging conditions, based only on their low-level characteristics.
However, for a high-level recognition task such as image retrieval, it is also critical to select keypoints that discriminate different object instances.
The proposed pipeline achieves both goals by training a model that encodes higher level semantics in the feature map, and learning to select discriminative features for the classification task.
This is in contrast to recently proposed techniques for learning keypoint detectors, \ie, LIFT \cite{yi2016lift}, which collect training data based on SIFT matches.
Although our model is not constrained to learn invariances to pose and viewpoint, it implicitly learns to do so---similar to CNN-based image classification techniques.

\subsection{Dimensionality Reduction}

We reduce the dimensionality of selected features to obtain improved retrieval accuracy,
as common practice \cite{jegou2012negative}.
First, the selected features are $L_2$ normalized, and their dimensionality is reduced to 40 by PCA, which presents a good trade-off between compactness and discriminativeness.
Finally, the features once again undergo $L_2$ normalization.

\subsection{Image Retrieval System}

We extract feature descriptors from query and database images, where a predefined number of local features with the highest attention scores per image are selected.
Our image retrieval system is based on nearest neighbor search, which is implemented by a combination of KD-tree~\cite{bentley75multidimensional} and Product Quantization (PQ)~\cite{jegou2011product}.
We encode each descriptor to a 50-bit code using PQ, where each 40D feature descriptor is split into 10 subvectors with equal dimensions, and we identify $2^5$ centroids per subvector by $k$-means clustering to achieve 50-bit encoding.
We perform asymmetric distance calculation, where the query descriptors are not encoded to improve the accuracy of nearest neighbor retrieval.
To speed up the nearest neighbor search, we construct an inverted index for descriptors, using a codebook of size 8K.
To reduce encoding errors, a KD-tree is used to partition each Voronoi cell, and a Locally Optimized Product Quantizer \cite{kalantidis2014locally} is employed for each subtree with fewer than 30K features.

When a query is given, we perform approximate nearest neighbor search for each local descriptor extracted from a query image.
Then for the top $K$ nearest local descriptors retrieved from the index, we aggregate all the matches per database image.
Finally, we perform geometric verification using RANSAC~\cite{Fischler1981} and employ the number of inliers as the score for retrieved images.
Many distractor queries are rejected by this geometric verification step because features from distractors may not be consistently matched with the ones from landmark images.

This pipeline requires less than 8GB memory to index 1 billion descriptors, which is sufficient to handle our large-scale landmark dataset.
The latency of the nearest neighbor search is less than 2 seconds using a single CPU under our experiment setup, where we soft-assign 5 centroids to each query and search up to 10K leaf nodes within each inverted index tree.

\section{Introduction} \label{sec:introduction}
Large-scale image retrieval is a fundamental task in computer vision, since it is directly related to various practical applications, \eg, object detection, visual place recognition, and product recognition.
The last decades have witnessed tremendous advances in image retrieval systems---from hand-crafted features and indexing algorithms \cite{Lowe2004,Sivic2003,Philbin07,Jegou2008} to, more recently, methods based on convolutional neural networks (CNNs) for global descriptor learning \cite{arandjelovic2015netvlad,radenovic2016cnn,gordo2016deep}.

\begin{figure}[t]
\begin{center}
   \includegraphics[width=1.02\linewidth]{figures/fig1.pdf}
\end{center}
\vspace{-12pt}
   \caption{Overall architecture of our image retrieval system, using DEep Local Features (DELF) and attention-based keypoint selection.  On the left, we illustrate the pipeline for extraction and selection of DELF.  The portion highlighted in yellow represents an attention mechanism that is trained to assign high scores to relevant features and select the features with the highest scores.  Feature extraction and selection can be performed with a single forward pass using our model.  On the right, we illustrate our large-scale feature-based retrieval pipeline.  DELF for database images are indexed offline. The index supports querying by retrieving nearest neighbor (NN) features, which can be used to rank database images based on geometrically verified matches.}
\label{fig:key_fig}
\vspace{-10pt}
\end{figure}

\iffalse
{\color{red}Such deep global descriptors~\cite{radenovic2016cnn,gordo2016deep} achieve excellent performance in medium-size datasets~\cite{Philbin07,Philbin2008}, even improving upon systems which use local feature-based geometric verification.
In large datasets, on the other hand, a wide variety of imaging conditions is typically observed, which may hinder the performance of global descriptors: cluttered images, occluded objects, with large viewpoint and illumination variations.
CNN-based local features have been proposed for patch-level matching~\cite{han2015matchnet,zagoruyko2015learning,yi2016lift}.
However, these features are not optimized for large-scale recognition and may suffer in this setting.}
\fi

Despite the recent advances in CNN-based global descriptors for image retrieval in small or medium-size datasets~\cite{Philbin07,Philbin2008}, their performance may be hindered by a wide variety of challenging conditions observed in large-scale datasets, such as clutter, occlusion, and variations in viewpoint and illumination.
Global descriptors lack the ability to find patch-level matches between images.
As a result, it is difficult to retrieve images based on partial matching in the presence of occlusion and background clutter.
In a recent trend, CNN-based local features are proposed for patch-level matching \cite{han2015matchnet,zagoruyko2015learning,yi2016lift}.
However, these techniques are not optimized specifically for image retrieval since they lack the ability to detect semantically meaningful features, and show limited accuracy in practice.

\iffalse
Despite the recent advances in CNN-based global descriptors for image retrieval, they still suffer from the lack of ability to find patch-level matches between query and database images; it is difficult to retrieve images based on partial matching in the presence of occlusion and background clutter.
In a different recent trend, CNN-based local features are proposed for patch-level matching \cite{han2015matchnet,zagoruyko2015learning,yi2016lift}.
However, these techniques are not optimized specifically for image retrieval and consequently show limited accuracy in practice.
As a result, image retrieval systems relying on long-standing hand-crafted features, such as SIFT~\cite{Lowe2004} or CONGAS~\cite{buddemeier2012systems}, are still widely used.
\fi

Most existing image retrieval algorithms have been evaluated in small to medium-size datasets with few query images, \ie, only 55 in \cite{Philbin07,Philbin2008} and 500 in \cite{Jegou2008}, and the images in the datasets have limited diversity in terms of landmark locations and types.
Therefore, we believe that the image retrieval community can benefit from a large-scale dataset, comprising more comprehensive and challenging examples, to improve algorithm performance and  evaluation methodology by deriving more statistically meaningful results.

The main goal of this work is to develop a large-scale image retrieval system based on a novel CNN-based feature descriptor.
To this end, we first introduce a new large-scale dataset, Google-Landmarks, which contains more than 1M landmark images from almost 13K unique landmarks.
This dataset covers a wide area in the world, and is consequently more diverse and comprehensive than existing ones.
The query set is composed of an extra 100K images with diverse characteristics; in particular, we include images that have no match in the database, which makes our dataset more challenging.
This allows to assess the robustness of retrieval systems when queries do not necessarily depict landmarks.

We then propose a CNN-based local feature with attention, which is trained with weak supervision using image-level class labels only, without the need of object- and patch-level annotations.
This new feature descriptor is referred to as DELF (DEep Local Feature), and \figref{fig:key_fig} illustrates the overall procedure of feature extraction and image retrieval.
In our approach, the attention model is tightly coupled with the proposed descriptor; it reuses the same CNN architecture and generates feature scores using very little extra computation (in the spirit of recent advances in object detection \cite{ren2015faster}).
This enables the extraction of both local descriptors and keypoints via one forward pass over the network.
We show that our image retrieval system based on DELF achieves the state-of-the-art performance with significant margins compared to methods based on existing global and local descriptors.

\section{Related Work}

There are standard datasets commonly used for the evaluation of image retrieval techniques.
Oxford5k~\cite{Philbin07} has 5,062 building images captured in Oxford with 55 query images.
Paris6k~\cite{Philbin2008} is composed of 6,412 images of landmarks in Paris, and also has 55 query images.
These two datasets are often augmented with 100K distractor images from Flickr100k dataset~\cite{Philbin07}, which constructs Oxford105k and Paris106k datasets, respectively.
On the other hand, Holidays dataset~\cite{Jegou2008} provides 1,491 images including 500 query images, which are from personal holiday photos.
All these three datasets are fairly small, especially having a very small number of query images, which makes it difficult to generalize the performance tested in these datasets.
Although Pitts250k~\cite{torii13visual} is larger, it is specialized to visual places with repetitive patterns and may not be appropriate for the general image retrieval task.

Instance retrieval has been a popular research problem for more than a decade.
See \cite{zheng2016sift} for a recent survey.
Early systems rely on hand-crafted local features \cite{Lowe2004,bay2008speeded,buddemeier2012systems}, coupled with approximate nearest neighbor search methods using KD trees or vocabulary trees~\cite{beis1997shape,Nister}.
Still today, such feature-based techniques combined with geometric re-ranking provide strong performance when retrieval systems need to operate with high precision.

More recently, many works have focused on aggregation methods of local features, which include popular techniques such as VLAD \cite{Jegou2010} and Fisher Vector (FV) \cite{Jegou2012}.
The main advantage of such global descriptors is the ability to provide high-performance image retrieval with a compact index.

In the past few years, several global descriptors based on CNNs have been proposed to use pretrained~\cite{babenko2014neural,tolias2015particular} or learned networks~\cite{arandjelovic2015netvlad,radenovic2016cnn,gordo2016deep}.
These global descriptors are most commonly trained with a triplet loss, in order to preserve the ranking between relevant and irrelevant images.
Some retrieval algorithms using these CNN-based global descriptors make use of deep local features as a drop-in replacement for hand-crafted features in conventional aggregation techniques such as VLAD or FV \cite{yue2015exploiting,uricchio2015fisher}.
Other works have re-evaluated and proposed different feature aggregation methods using such deep local features \cite{babenko2015iccv,kalantidis2015cross}.

CNNs have also been used to detect, represent and compare local image features.
Verdie \etal \cite{verdie2015tilde} learned a regressor for repeatable keypoint detection.
Yi \etal \cite{moo2016learning} proposed a generic CNN-based technique to estimate the canonical orientation of a local feature and successfully deployed it to several different descriptors.
MatchNet~\cite{han2015matchnet} and DeepCompare~\cite{zagoruyko2015learning} have been proposed to jointly learn patch representations and associated metrics.
Recently, LIFT~\cite{yi2016lift} proposed an end-to-end framework to detect keypoints, estimate orientation, and compute descriptors.
Different from our work, these techniques are not designed for image retrieval applications since they do not learn to select semantically meaningful features.

Many visual recognition problems employ visual attention based on deep neural networks, which include object detection~\cite{zhou2016learning}, semantic segmentation~\cite{hong2015decoupled}, image captioning~\cite{xu2015show}, visual question answering~\cite{yang2016stacked}, etc.
However, visual attention has not been explored actively to learn visual features for image retrieval applications.

\section{Conclusion}
We presented DELF, a new local feature descriptor that is designed specifically for large-scale image retrieval applications. DELF is learned with weak supervision, using image-level labels only, and is coupled with our new attention mechanism for semantic feature selection.
In the proposed CNN-based model, one forward pass over the network is sufficient to obtain both keypoints and descriptors.
To properly evaluate performance of large-scale image retrieval algorithm, we introduced Google-Landmarks dataset, which consists of more than 1M database images, 13K unique landmarks, and 100K query images.
The evaluation in such a large-scale setting shows that DELF outperforms existing global and local descriptors by substantial margins. We also present results on existing datasets, and show that DELF achieves excellent performance when combined with global descriptors.

\section{Google-Landmarks Dataset}

Our dataset is constructed based on the algorithm described in \cite{zheng2009tour}.
Compared to the existing datasets for image retrieval~\cite{Philbin07,Philbin2008,Jegou2008}, the new dataset is much larger, contains diverse landmarks, and involves substantial challenges.
It contains $1,060,709$ images from $12,894$ landmarks, and $111,036$ additional query images.
The images in the dataset are captured at various locations in the world, and each image is associated with a GPS coordinate.
Example images and their geographic distribution are presented in \figref{fig:examples} and \figref{fig:geolocation}, respectively.
While most images in the existing datasets are landmark-centric, which makes global feature descriptors work well, our dataset contains more realistic images with wild variations including foreground/background clutter, occlusion, partially out-of-view objects, etc.
In particular, since our query images are collected from personal photo repositories, some of them may not contain any landmarks and should not retrieve any image from the database.
We call these query images {\em distractors}, which play a critical role to evaluate robustness of algorithms to irrelevant and noisy queries.
\begin{figure}[t]
\vspace{3pt}
\begin{center}
\begin{subfigure}{1\linewidth}
   \includegraphics[width=1\linewidth]{figures/database_phileas.jpg} \vspace{-14pt}
   \caption{\footnotesize Sample database images} \vspace{1pt}
\end{subfigure}
\begin{subfigure}{1\linewidth}
   \includegraphics[width=1\linewidth]{figures/query_phileas.jpg} \vspace{-14pt}
   \caption{\footnotesize Sample query images} \vspace{1pt}
\end{subfigure}
\vspace{-8pt}
\caption{Example database and query images from Google-Landmarks. They have a lot of variations and challenges including background clutter, small objects, and multiple landmarks.}
\vspace{-8pt}
\label{fig:examples}
\end{center}
\end{figure}
\begin{figure}[t]
\begin{center}
   \includegraphics[width=1\linewidth]{figures/dataset_geolocation.png}
\end{center}
	\vspace{-15pt}
	\caption{Image geolocation distribution of our Google-Landmarks dataset.  The landmarks are located in 4,872 cities in 187 countries.}
\label{fig:geolocation}
\vspace{-6pt}
\end{figure}

We use visual features and GPS coordinates for ground-truth construction.
All images in the database are clustered using the two kinds of information, and we assign a landmark identifier to each cluster.
If physical distance between the location of a query image and the center of the cluster associated with the retrieved image is less than a threshold, we assume that the two images belong to the same landmark.
Note that ground-truth annotation is extremely challenging, especially considering the facts that it is hard to predefine what landmarks are, landmarks are not clearly noticeable sometimes, and there might be multiple instances in a single image.
Obviously, this approach for ground-truth construction is noisy due to GPS errors.
Also, photos can be captured from a large distance for some landmarks (\eg, Eiffel Tower, Golden Gate Bridge), and consequently the photo location might be relatively far from the actual landmark location.
However, we found very few incorrect annotations with the threshold of $25$km when checking a subset of data manually.
Even though there are few minor errors, it is not problematic, especially in relative evaluation, because algorithms are unlikely to be confused between landmarks anyway if their visual appearances are sufficiently discriminative.

\section{Evaluation} \label{sec:metrics}
Image retrieval systems have typically been evaluated based on mean average precision (mAP), which is computed by sorting images in descending order of relevance and averaging accuracies for individual thresholds.
This evaluation method may be a reasonable choice in the existing datasets such as Oxford5k/105k~\cite{Philbin07}, Paris6k/106k~\cite{Philbin2008} and Holiday~\cite{Jegou2008} because there are no distractors in query (and database) and the sizes of the datasets are small.

However, for the case in large-scale datasets with a lot of queries including distractors, such evaluation method may not be representative since it is important to determine whether each image is relevant to the query or not and ranking irrelevant images is not interesting at all.
In addition, since the scales of scores may be different across query images, it is not straightforward to set a unified threshold to determine the relevance of each image to query, especially in the presence of distractors in dataset and query.

Therefore, we employ precision and recall information instead.
Slightly different from the standard definitions of precision (\textsc{Pre}) and recall (\textsc{Rec}) per image, we consider all query images at the same time as in the following equations because recall is not well-defined for distractor queries:
\begin{align}
\textsc{Pre} = \frac{\sum_{q} | \mathcal{R}^\text{TP}_q |}{\sum_{q} |\mathcal{R}_i |}~~\text{and}~~
\textsc{Rec} = \sum_{q} | \mathcal{R}^\text{TP}_q |,
\label{eq:pr_def}
\end{align}
where $\mathcal{R}_q$ denotes a set of retrieved images for query $q$, and $\mathcal{R}_q^\text{TP} (\subseteq \mathcal{R}_q)$ is a set of true-positive images in the retrieval.

\iffalse
The main application considered in our work is different from that of most existing image retrieval systems.
We discuss two variants of objectives for image retrieval tasks, and describe how we verify the effectiveness of the proposed algorithm in the context of our focus.

\subsection{Ranked vs. Unranked Image Retrieval}
Image retrieval applications can be broadly categorized into two types: (i) ranked retrieval, and (ii) unranked retrieval.
In the first case, systems return a list of results, ranked in descending order of relevance, so even loosely relevant items might be included in this list.
An example of this type of application is web image search, where a user's goal is often to explore potentially similar images to the query.
The performance metric commonly used in this case, mean average precision (mAP), does not penalize returning more results, even if they are incorrect.

In the second type of application, unranked retrieval, a system is expected to return only the items that are relevant for a given query.
The relevance score of each item in the database may not be reported.
Usually, such systems must operate with high precision, in order to avoid exceeding a tolerable false positive rate.
In this setup, it is very important to effectively handle distractor queries, \ie, queries that have no relevant matches in the database, by simply not returning any result.
In this setting, the absolute scores of database images for a given query are important, as only the images with sufficiently high scores should be returned.
An example of this type of application is the automatic annotation of personal photos with landmark labels, where the system should only suggest annotations with high confidence.

\subsection{Performance Assessment}
Our main focus is the unranked image retrieval problem.
For proper evaluation of the target task, we measure the recall achieved at high precision operating points, for example, $\mathrm{Recall}@90\%$ (in short, $\mathrm{R}@90\%$).
To take distractor queries into account, precision and recall measures need to consider all queries together, since recall is not well-defined when evaluating retrieval results for a single distractor query.
More precisely, for a given threshold $\tau$ to accept a retrieval match, denote the number of retrieved results for query $q$ by $R_q$, the number of retrieved true positive results by $R^{TP}_q$, and the number of ground-truth true positive database images by $G_q$.
We compute precision ($\mathcal{P}$) and recall ($\mathcal{R}$) respectively as
\iffalse
\vspace{-25pt}
\begin{multicols}{2}
  \begin{equation}
	\mathrm{P} = \frac{\sum_{q=1}^{Q} R^{TP}_q}{\sum_{q=1}^{Q} R_q}
	\label{eq:precision}
  \end{equation}\break
  \begin{equation}
	\mathrm{R} = \frac{\sum_{q=1}^{Q} R^{TP}_q}{\sum_{q=1}^{Q} G_q},
	\label{eq:recall}
  \end{equation}
\end{multicols}
\fi
\begin{align}
\mathcal{P} = \frac{\sum_{q=1}^{Q} R^{TP}_q}{\sum_{q=1}^{Q} R_q}~~\text{and}~~
\mathcal{R} = \frac{\sum_{q=1}^{Q} R^{TP}_q}{\sum_{q=1}^{Q} G_q}, \nonumber
\end{align}
where $Q$ denotes the number of queries in the dataset.
\fi
\section{Experiments} \label{sec:experiments}

This section mainly discusses the performance of DELF compared to existing global and local feature descriptors in our dataset.
In addition, we also show how DELF can be employed to achieve good accuracy in the existing datasets.

\subsection{Implementation Details}

\paragraph{Multi-scale descriptor extraction}
We construct image pyramids by using scales that are a $\sqrt{2}$ factor apart.
For the set of scales with range from $0.25$ to $2.0$, 7 different scales are used.
The size of receptive field is inversely proportional to the scale; for example, for the $2.0$ scale, the receptive field of the network covers $146 \times 146$ pixels.

\vspace{-10pt}
\paragraph{Training}
We employed landmarks dataset~\cite{babenko2014neural} for fine-tuning descriptors and training keypoint selection.
In the dataset, there are the ``full'' version, referred to as LF (after removal of overlapping classes with Oxf5k/Par6k, by \cite{gordo2016deep}), containing 140,372 images from 586 landmarks, and the ``clean'' version (LC) obtained by a SIFT-based matching procedure \cite{gordo2016deep}, with 35,382 images of 586 landmarks.
We use LF to train our attention model, and LC is employed to fine-tune the network for image retrieval.

\vspace{-10pt}
\paragraph{Parameters}
We identify the top $K (= 60)$ nearest neighbors for each feature in a query and extract up to $1000$ local features from each image---each feature is $40$-dimensional.

\subsection{Compared Algorithms}
DELF is compared with several recent global and local descriptors.
Although there are various research outcomes related to image retrieval, we believe that the following methods are either relevant to our algorithm or most critical to evaluation due to their good performance.

\vspace{-10pt}
\paragraph{Deep Image Retrieval (DIR)~\cite{gordo2016deep}}
This is a recent global descriptor that achieves the state-of-the-art performance in several existing datasets.
DIR feature descriptors are $2,048$ dimensional and multi-resolution descriptors are used in all cases.
We also evaluate with query expansion (QE), which typically improves accuracy in the standard datasets.
We use the released source code that implements the version with ResNet101~\cite{he2015deep}.
For retrieval, a parallelized implementation of brute-force search is employed to avoid penalization by the error from approximate nearest neighbor search.

\vspace{-10pt}
\paragraph{siaMAC~\cite{radenovic2016cnn}}
This is a recent global descriptor that obtains high performance in existing datasets.
We use the released source code with parallelized implementation of brute-force search.
The CNN based on VGG16~\cite{Simonyan15} extracts $512$ dimensional global descriptor.
We also experiment with query expansion (QE) as in DIR.

\vspace{-10pt}
\paragraph{CONGAS~\cite{buddemeier2012systems,neven2008image}} CONGAS is a $40$D hand-engineered local feature, which has been widely used for instance-level image matching and retrieval \cite{aradhye2009video2text,zheng2009tour}.
This feature descriptor is extracted by collecting Gabor wavelet responses at the detected keypoint scale and orientation, and is known to have very similar performance and characteristic to other gradient-based local descriptors like SIFT. A Laplacian-of-Gaussian keypoint detector is used.

\vspace{-10pt}
\paragraph{LIFT}
LIFT~\cite{yi2016lift} is a recently proposed feature matching pipeline, where keypoint detection, orientation estimation and keypoint description are jointly learned.
Features are $128$ dimensional.
We use the source code publicly available.

\subsection{Evaluation}
\label{sub:metrics}
Image retrieval systems have typically been evaluated based on mean average precision (mAP), which is computed by sorting images in descending order of relevance per query and averaging AP of individual queries.
However, for datasets with distractor queries, such evaluation method is not representative since it is important to determine whether each image is relevant to the query or not.
In our case, the absolute retrieval score is used to estimate the relevance of each image.
For performance evaluation, we employ a modified version of precision (\textsc{Pre}) and recall (\textsc{Rec}) by considering all query images at the same time, which are given by
\begin{align}
\textsc{Pre} = \frac{\sum_{q} | \mathcal{R}^\text{TP}_q |}{\sum_{q} |\mathcal{R}_q |}~~\text{and}~~
\textsc{Rec} = \sum_{q} | \mathcal{R}^\text{TP}_q |,
\label{eq:pr_def}
\end{align}
where $\mathcal{R}_q$ denotes a set of retrieved images for query $q$ given a threshold, and $\mathcal{R}_q^\text{TP} (\subseteq \mathcal{R}_q)$ is a set of true positives.
This is similar to the micro-AP metric introduced in \cite{perronnin2009}.
Note that in our case only the top-scoring image per landmark is considered in the final scoring.
We prefer unnormalized recall values, which present the number of retrieved true positives.
Instead of summarizing our result in a single number, we present a full precision-recall curve to inspect operating points with different retrieval thresholds.

\subsection{Quantitative Results}
\figref{fig:large_scale_results} presents the precision-recall curve of DELF (denoted by DELF+FT+ATT), compared to other methods.
The results of LIFT could not be shown because feature extraction is extremely slow and large-scale experiment is infeasible\footnote{LIFT feature extraction approximately takes 2 min/image using a GPU.}.
\begin{figure}[t]
\begin{center}
\includegraphics[width=1\linewidth]{figures/pr_curve_absolute_new2}
\end{center}
\vspace{-15pt}
   \caption{

Precision-recall curve for the large-scale retrieval experiment on the Google-Landmarks dataset, where recall is presented in absolute terms, as in Eq.~\eqref{eq:pr_def}.  DELF shows outstanding performance compared with existing global and local features.  Fine-tuning and attention model in DELF are critical to performance improvement.  The accuracy of DIR drops significantly with query expansion, due to many distractor queries in our dataset.}
\label{fig:large_scale_results}
\vspace{-5pt}
\end{figure}
DELF clearly outperforms all other techniques significantly.
Global feature descriptors, such as DIR, suffer in our challenging dataset.
In particular, due to a large number of distractors in the query set, DIR with QE degrades accuracy significantly.
CONGAS does a reasonably good job, but is still worse than DELF with substantial margin.

To analyze the benefit of fine-tuning and attention for image retrieval, we compare our full model (DELF+FT+ATT) with its variations: DELF-noFT, DELF+FT and DELF-noFT+ATT.
DELF-noFT means that extracted features are based on the pretrained CNN on ImageNet without fine-tuning and attention learning.
DELF+FT denotes the model with fine-tuning but without attention modeling while DELF-noFT+ATT corresponds to the model without fine-tuning but using attention.
As illustrated in \figref{fig:large_scale_results}, both fine-tuning and attention modeling make substantial contributions to performance improvement.
In particular, note that the use of attention is more important than fine-tuning.
This demonstrates that the proposed attention layers effectively learn to select the most discriminative features for the retrieval task, even if the features are simply pretrained on ImageNet.

In terms of memory requirement, DELF, CONGAS and DIR are almost equally complex.
DELF and CONGAS adopt the same feature dimensionality and maximum number of features per image; they require approximately 8GB of memory.
DIR descriptors need 8KB per image, summing up to approximately 8GB to index the entire dataset.

\subsection{Qualitative Results} \label{sub:qualitative}

We present qualitative results to illustrate performance of DELF compared to two competitive algorithms based on global and local features---DIR and CONGAS, respectively.
Also, we analyze our attention-based keypoint detection algorithm by visualization.

\vspace{-5pt}
\paragraph{DELF~vs.~DIR}
\figref{fig:qualitative1} shows retrieval results, where DELF outperforms DIR.
DELF obtains matches between specific local regions in images, which helps significantly to find the same object in different imaging conditions.
Common failure cases of DIR happen when the database contains similar objects or scenes, \eg, obelisks, mountains, harbors, as illustrated in \figref{fig:qualitative1}.
In many cases, DIR cannot distinguish these specific objects or scenes; although it finds semantically similar images, they often do not correspond to the instance of interest.
Another weakness of DIR and other global descriptors is that they are not good at identifying small objects of interest.
\begin{figure}[t]
\begin{center}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_2_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_2_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_2_dir.jpeg}}\\
   \vspace{0.05cm}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_3_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_3_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_3_dir.jpeg}} \\
   \vspace{0.05cm}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_5_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_5_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_5_dir.jpeg}} \\
   \vspace{0.05cm}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_6_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_6_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.2cm]{figures/qualitative/DECODER_bt_DIR_6_dir.jpeg}}\\
   \vspace{0.05cm}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.2cm]{figures/qualitative/DECODER_bt_DIR_1_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.2cm]{figures/qualitative/DECODER_bt_DIR_1_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_1_dir.jpeg}} \\
   \vspace{0.05cm}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_8_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_8_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DECODER_bt_DIR_8_dir.jpeg}}\\
   \vspace{-0.1cm}
\centerline{\footnotesize (a) \hspace{2.3cm} (b) \hspace{2.3cm} (c)}
   \vspace{-0.5cm}
\end{center}
	\caption{Examples where DELF+FT+ATT outperforms DIR: (a) query image, (b) top-1 image of DELF+FT+ATT, (c) top-1 image of DIR.  The green borders denote correct results while  the red ones mean incorrect retrievals.  Note that DELF deals with clutter in query and database images and small landmarks effectively.}
\label{fig:qualitative1}
\vspace{-5pt}
\end{figure}
\begin{figure}[t]
\begin{center}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DIR_bt_DECODER_1_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DIR_bt_DECODER_1_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DIR_bt_DECODER_1_dir.jpeg}}\\
   \vspace{0.05cm}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.2cm]{figures/qualitative/DIR_bt_DECODER_2_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.2cm]{figures/qualitative/DIR_bt_DECODER_2_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DIR_bt_DECODER_2_dir.jpeg}}\\
   \vspace{0.05cm}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DIR_bt_DECODER_3_query.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DIR_bt_DECODER_3_ours.jpeg}}
   \makebox[0.32\linewidth]{ \includegraphics[height=2.0cm, width=0.32\linewidth]{figures/qualitative/DIR_bt_DECODER_3_dir.jpeg}}\\
   \vspace{-0.1cm}
\centerline{\footnotesize (a) \hspace{2.3cm} (b) \hspace{2.3cm} (c)}
   \vspace{-0.6cm}
\end{center}
	\caption{Examples where DIR outperforms DELF+FT+ATT: (a) query image, (b) top-1 image of DELF+FT+ATT, (c) top-1 image of DIR.  The green and red borders denotes correct and incorrect results, respectively.}
	\vspace{-5pt}
\label{fig:qualitative2}
\end{figure}
\begin{figure*}[t]
\begin{center}
   \includegraphics[width=0.323\linewidth]{figures/feat_matches/stanley_park/match-eps-converted-to.pdf}
   \includegraphics[width=0.323\linewidth]{figures/feat_matches/rio/match-eps-converted-to.pdf}
   \includegraphics[width=0.323\linewidth]{figures/feat_matches/louvre/match-eps-converted-to.pdf} \\
   \vspace{0.03cm}
   \includegraphics[width=0.323\linewidth]{figures/feat_matches/antenna/match-eps-converted-to.pdf}
   \includegraphics[width=0.323\linewidth]{figures/feat_matches/st_ives/match-eps-converted-to.pdf}
   \includegraphics[width=0.323\linewidth]{figures/feat_matches/islands/match-eps-converted-to.pdf} \\
   \vspace{-0.5cm}
\end{center}
	\caption{Visualization of feature correspondences between images in query and database using DELF+FT+ATT.  For each pair, query and database images are presented side-by-side.  DELF successfully matches landmarks and objects in challenging environment including partial occlusion, distracting objects, and background clutter.  Both ends of the red lines denote the centers of matching features.  Since the receptive fields are fairly large, the centers may be located outside landmark object areas.  For the same queries, CONGAS fails to retrieve any image.}
	\vspace{-0.2cm}
\label{fig:feat_matches}
\end{figure*}
\figref{fig:qualitative2} shows the cases that DIR outperforms DELF.
While DELF is able to match localized patterns across different images, this leads to errors when the floor tiling or vegetation is similar across different landmarks.

\begin{figure}[!h]
\vspace{0.1cm}
\begin{center}
   \includegraphics[width=0.242\linewidth]{figures/fig3/1.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/1_bs.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/1_ft.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/1_att.jpg}\\
	\vspace{0.05cm}
   \includegraphics[width=0.242\linewidth]{figures/fig3/2.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/2_bs.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/2_ft.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/2_att.jpg}\\
	\vspace{0.05cm}
   \includegraphics[width=0.242\linewidth]{figures/fig3/3.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/3_bs.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/3_ft.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/3_att.jpg}\\
	\vspace{0.05cm}
   \includegraphics[width=0.242\linewidth]{figures/fig3/4.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/4_bs.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/4_ft.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/4_att.jpg}\\
	\vspace{0.05cm}
   \includegraphics[width=0.242\linewidth]{figures/fig3/5.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/5_bs.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/5_ft.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/5_att.jpg}\\
	\vspace{0.05cm}
   \includegraphics[width=0.242\linewidth]{figures/fig3/6.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/6_bs.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/6_ft.jpg}
   \includegraphics[width=0.242\linewidth]{figures/fig3/6_att.jpg}\\
	\vspace{-0.1cm}
\centerline{\footnotesize (a) \hspace{1.7cm} (b) \hspace{1.7cm} (c) \hspace{1.7cm} (d)}
	\vspace{-0.5cm}
\end{center}
	\caption{Comparison of keypoint selection methods. (a) Input image (b) $L_2$ norm scores using the pretrained model (DELF-noFT) (c) $L_2$ norm scores using fine-tuned descriptors (DELF+FT) (d) Attention-based scores (DELF+FT+ATT).  Our attention-based model effectively disregards clutter compared to other options.}
\label{fig:vis_att}
\vspace{-20pt}
\end{figure}

\vspace{-3pt}
\paragraph{DELF vs. CONGAS}
\iffalse
\figref{fig:feat_matches} presents query-database pairs which are successful DELF matches, missed by CONGAS.
DELF's main improvement over CONGAS is in terms of recall: it is able to retrieve many more relevant landmarks compared to CONGAS.
This suggests that DELF descriptors are more discriminative.
We did not observe significant examples where CONGAS outperforms DELF, so we do not include such examples here.

In \figref{fig:feat_matches}, we present feature correspondences by connecting the center of the receptive fields for matching features.
Note that the receptive fields can be quite large.
This explains why some features seem to be localized in undiscriminative regions (\eg, in the ocean or in the sky) -- in these cases, the features are extracted from large patches, which take into account more discriminative regions in its neighborhood.
\fi
The main advantage of DELF over CONGAS is its recall; it retrieves more relevant landmarks than CONGAS, which suggests that DELF descriptors are more discriminative.
We did not observe significant examples where CONGAS outperforms DELF.
\figref{fig:feat_matches} shows pairs of images from query and database, which are successfully matched by DELF but missed by CONGAS, where feature correspondences are presented by connecting the center of the receptive fields for matching features.
Since the receptive fields can be fairly large, some features seem to be localized in undiscriminative regions, \eg, ocean or sky.
However, in these cases, the features take into account more discriminative regions in the neighborhood.

\vspace{-7pt}
\paragraph{Analysis of keypoint detection methods}
\figref{fig:vis_att} visualizes three variations of keypoint detection, where the benefit of our attention model is clearly illustrated qualitatively while the $L_2$ norm of fine-tuned features is marginally different from the one without fine-tuning.

\iffalse
\subsection{Analysis of Fine-tuning and Attention}
\label{sub:annalysis}
To analyze the benefit of fine-tuning and attention model for image retrieval, we compare our full model (\andre{DELF+FT+ATT}) with its variations, DELF+FT and DELF+noFT.
DELF+FT denotes the model with fine-tuning but without attention modeling while DELF+noFT means that extracted features are based on the pretrained CNN on ImageNet without fine-tuning and attention learning.
When our attention model is not used, we employ $L_2$ norm of unnormalized feature descriptor for keypoint selection instead.
As illustrated in \figref{fig:large_scale_results}, both fine-tuning and attention modeling make substantial contribution to performance improvement.
\figref{fig:vis_att} visualizes three variations of keypoint detection, where the benefit of our attention model is clearly illustrated while the $L_2$ norm of fine-tuned features is marginally different from the one without fine-tuning.
\fi

\iffalse
\figref{fig:vis_att} presents a visualization of the different keypoint selection methods, for some images.
Qualitatively, it shows that the $L_2$ norm scores benefit from fine-tuning; scores for image regions without landmarks (\eg, faces, people) are suppressed, and scores for regions containing landmarks are boosted.
A substantial difference can be observed between attention and $L_2$ scores; the attention portion of the network produces high scores for landmark regions while significantly suppressing clutter which is irrelevant for the landmark recognition task.
\fi

\subsection{Results in Existing Datasets}
\label{sub:results}
We demonstrate the performance of DELF in existing datasets such as Oxf5k, Par6k and their extensions, Oxf105k and Par106k, for completeness.
For this experiment, we simply obtain the score per image using the proposed method, and make a late fusion with the score from DIR by computing a weighted mean of two normalized scores, where the weight for DELF is set to 0.25.
The results are presented in \tabref{tab:100k_results}.
We present accuracy of existing methods in their original papers and our reproductions using public source codes, which are very close.
DELF improves accuracy nontrivially in the datasets when combined with DIR, although it does not show the best performance by itself.
This fact indicates that DELF has capability to encode complementary information that is not available in global feature descriptors.%
\begin{table}[t]
\footnotesize
\centering
\caption{Performance evaluation on existing datasets in mAP ($\%$).  All results of existing methods are based on our reproduction using public source codes.  We tested LIFT only on Oxf5k and Par6k due to its slow speed.  (* denotes the results from the original papers.)}
\vspace{-5pt}
\begin{tabular}{@{\hskip0.0pt}l@{\hskip2.5pt}|C{0.7cm}C{0.7cm}C{0.7cm}C{0.7cm}l@{\hskip0.0pt}}
  \toprule
Dataset & Oxf5k & Oxf105k & Par6k & Par106k \\
  \midrule
DIR~\cite{gordo2016deep} & 86.1 &  82.8   &    94.5     &     90.6   \\
DIR+QE~\cite{gordo2016deep} & 87.1 & 85.2 &   95.3    &   91.8   \\
siaMAC~\cite{radenovic2016cnn} & 77.1 & 69.5   & 83.9     & 76.3      \\
siaMAC+QE~\cite{radenovic2016cnn} & 81.7 & 76.6   & 86.2         &  79.8      \\
CONGAS~\cite{buddemeier2012systems} & 70.8 & 61.1 & 67.1 &  56.8  \\
LIFT~\cite{yi2016lift} & 54.0 & -- & 53.6 &  --  \\
\midrule
DIR+QE*~\cite{gordo2016deep} & 89.0 & 87.8 &   93.8     &   90.5   \\
siaMAC+QE*~\cite{radenovic2016cnn} & 82.9 & 77.9   & 85.6         &  78.3      \\
\midrule
DELF+FT+ATT (ours) & 83.8 & 82.6 & 85.0 & 81.7 \\
DELF+FT+ATT+DIR+QE (ours) & {\bf 90.0} & {\bf 88.5} & {\bf 95.7} & {\bf 92.8} \\
  \bottomrule
\end{tabular}
\label{tab:100k_results}
\vspace{-10pt}
\end{table}

\section*{A. Dataset Details}
Our dataset contains 1,060,709 images for 12,894 landmarks, which are located at 4,872 cities in 187 countries as illustrated in  \figref{fig:geolocation}.
\iffalse
More complete geolocations of the images are illustrated in \figref{fig:geolocation}.
\begin{figure}[t]
\begin{center}
   \includegraphics[width=1\linewidth]{figures/DELF_dataset.jpeg}
\end{center}
	\vspace{-15pt}
	\caption{Geolocation distribution of images in our dataset.}
\label{fig:geolocation}
\end{figure}
\fi
The top-20 countries with most landmarks are shown in \figref{fig:landmark_countries}.
The top-10 cities are London (179), New York (174), Paris (133), Berlin (85), Hong Kong (83), San Francisco (82), Rome (81), Moscow (70), Beijing (68), and Prague (67), where the numbers in parentheses denote the number of landmarks.
There are various kinds of landmarks in our dataset, which include tourist attraction (1969), town (1730), structure (1175), building (898), village (657), museum (451), UNESCO World Heritage Site (368), city (315), and park (278).
Note that we only included landmarks that have at least 3 photos.
\begin{figure}[h]
\begin{center}
   \includegraphics[width=1\linewidth]{figures/landmark_countries}
\end{center}
	\vspace{-15pt}
	\caption{Number of landmarks of top-20 countries in our new dataset.}
\label{fig:landmark_countries}
\end{figure}

\iffalse
\section{Large-Scale Experiment with siaMAC}
We present performance of all compared algorithms in our large-scale dataset in \figref{fig:pr_curve}, which includes siaMAC~\cite{radenovic2016cnn} results now.
As illustrated in the figure, siaMAC performs poorly, which is mainly because of distractor queries; it returns spurious results with high scores.
To confirm this hypothesis, we performed an experiment to add distractor queries to the query set of Oxford~\cite{Philbin07} and Paris~\cite{Philbin2008}, and also observed that siaMAC is particularly susceptible to distractor queries in that case.
Note that performance degradation of DIR with Query Expansion (QE) in \figref{fig:pr_curve} is the same reason.
\begin{figure}[t]
\begin{center}
   \includegraphics[width=1\linewidth]{figures/pr_curve_absolute}
\end{center}
	\vspace{-15pt}
	\caption{Precision-recall curve for the large-scale retrieval experiment.  DELF shows outstanding performance compared with existing global and local features.  Fine-tuning and attention model in DELF make significant contribution to performance improvement. Accuracy of siaMAC is very low due to spurious results returned for distractor queries (see \figref{fig:siamac_qualitative} for examples).}
\label{fig:pr_curve}
\end{figure}

To understand low performance of siaMAC better (even compared to DIR), we perform a qualitative evaluation.
According to our observation, siaMAC seems to be overfitted to training datasets, so embedding for unseen object classes is arbitrary.
While queries containing valid landmarks do retrieve correct results, many queries containing objects in particular classes such as dogs, cats, and horses, are matched with database images containing those animals.
Some failure cases for siaMAC are presented in \figref{fig:siamac_qualitative}.
\begin{figure}[t]
\begin{center}
   \makebox[0.49\linewidth]{ \includegraphics[height=2.8cm, width=0.50\linewidth]{figures/sup/siamac/query_1.png}}
   \makebox[0.49\linewidth]{ \includegraphics[height=2.8cm, width=0.47\linewidth]{figures/sup/siamac/db_1.png}}
   \vspace{0.05cm}
   \makebox[0.49\linewidth]{ \includegraphics[height=2.92cm]{figures/sup/siamac/query_2.png}}
   \makebox[0.49\linewidth]{ \includegraphics[height=2.92cm]{figures/sup/siamac/db_2.png}}
   \vspace{0.05cm}
   \makebox[0.49\linewidth]{ \includegraphics[height=2.8cm]{figures/sup/siamac/query_4.png}}
   \makebox[0.49\linewidth]{ \includegraphics[height=2.8cm, width=0.47\linewidth]{figures/sup/siamac/db_4.png}}
\end{center}
	\vspace{-15pt}
	\caption{Interesting failure cases by siaMAC in our large-scale dataset. We present queries (left) and top-1 retrieved images (right) side-by-side.}
\label{fig:siamac_qualitative}
\end{figure}
\fi

\iffalse
\section{Experiment in Traditional Datasets}
We presented results for DELF in Oxford \cite{Philbin07} and Paris \cite{Philbin2008} datasets in Section 5.7 of the main paper.
The results for DIR~\cite{gordo2016deep} and siaMAC~\cite{radenovic2016cnn} in Table 1 of the main paper are slightly different from the numbers in the original references since we used our reproduction results.
All numbers from our reproduction are similar to the ones in the original papers (our reproductions are slightly better than the reported results in the original papers.), and we believe that our own results are more meaningful to show the benefit of DELF when combined with DIR.

To implement the retrieval system by the combination of DELF and DIR, we use a late fusion method; we simply obtain retrieval scores from DELF and DIR per image and compute a weighted mean of two normalized scores, where the weight for DELF is set to 0.25.
\fi

\iffalse
Note that the results for DIR \cite{gordo2016deep} and siaMAC \cite{radenovic2016cnn} in Table 1 of the main paper are slightly different from those in the original references, since we presented the results that were reproduced by us.
We decided to include the reproduced numbers in the main paper, since the exact same code was used to present results which combine DELF and DIR.

\tabref{tab:dir_results} presents a comparison of the results published by \cite{gordo2016deep} and those reproduced by us.
The main reason for the difference between the two is the fact that the released DIR model is based on ResNet-101, while the paper's results \cite{gordo2016deep} are based on ResNet-50.
Our results are generally very close to the published numbers, and better in most cases.

\tabref{tab:siamac_results} presents a comparison of the results published by \cite{radenovic2016cnn} and those reproduced by us.
As can be seen, both results are very similar, and the reproduced results are slightly better in most cases.

\begin{table}
\footnotesize
\centering
\caption{DIR retrieval results: comparing published results against those reproduced by us.}
\vspace{-5pt}
\begin{tabular}{@{\hskip0.4pt}l|cccc@{\hskip1pt}}
  \toprule
Dataset & Oxf5k & Oxf105k & Par6k & Par106k \\
  \midrule
DIR \cite{gordo2016deep} & 84.50 &  81.60   &    91.20      &     86.30   \\
DIR (reproduced) & \bf{86.07} &  \bf{82.75}   &    \bf{94.53}      &     \bf{90.56}   \\
  \midrule
DIR+QE \cite{gordo2016deep} & \bf{89.00} & \bf{87.80} &   93.80     &   90.50   \\
DIR+QE (reproduced) & 87.08 & 85.23 &   \bf{95.30}     &   \bf{91.83}   \\
  \bottomrule
\end{tabular}
\label{tab:dir_results}
\vspace{-0pt}
\end{table}

\begin{table}
\footnotesize
\centering
\caption{siaMAC retrieval results: comparing published results against those reproduced by us.}
\vspace{-5pt}
\begin{tabular}{@{\hskip0.4pt}l|cccc@{\hskip1pt}}
  \toprule
Dataset & Oxf5k & Oxf105k & Par6k & Par106k \\
  \midrule
siaMAC  \cite{radenovic2016cnn} & 77.00 & 69.20   & 83.80         & \bf{76.40}      \\
siaMAC  (reproduced) & \bf{77.14} & \bf{69.46}   & \bf{83.94}         & 76.27      \\
  \midrule
siaMAC+QE  \cite{radenovic2016cnn} & \bf{82.90} & \bf{77.90}   & 85.60         &  78.30      \\
siaMAC+QE  (reproduced) & 81.73 & 76.64   & \bf{86.24}         &  \bf{79.77}      \\
  \bottomrule
\end{tabular}
\label{tab:siamac_results}
\vspace{-0pt}
\end{table}
\fi

\iffalse
\section{Experiment for LIFT}
In the main paper, we presented LIFT~\cite{yi2016lift} results for traditional datasets (Oxford~\cite{Philbin07} and Paris~\cite{Philbin2008}), showing that its performance is weaker than other techniques.
As explained in the main paper, we were not able to run large-scale experiments with LIFT since the released source code is slow, taking on average 2 minutes per image using a GPU for descriptor extraction.
Here, we wanted to briefly mention that this is a known issue with LIFT's code, as can be seen in their source code repository (accessed on March 27, 2017)\footnote{\url{https://github.com/cvlab-epfl/LIFT/issues/2}}.

\section{LIFT Experiments}

In the main paper, we presented LIFT \cite{yi2016lift} results for traditional datasets (Oxford \cite{Philbin07} and Paris \cite{Philbin2008}), showing that its performance is weaker than other techniques.
As explained in the main paper, we were not able to run large-scale experiments with LIFT, since the released source code is slow, taking on average 2 minutes per image using a GPU for descriptor extraction.
Here, we wanted to briefly mention that this is a known issue with LIFT's code, as can be seen in their source code repository (accessed on March 27, 2017): \url{https://github.com/cvlab-epfl/LIFT/issues/2}.
\fi

redpajama_set_name: RedPajamaArXiv


text:
EYES on MAiN a member of Vision Source 406-586-2173 Call Us Appointments
Showcase Your Vision: Melissa Dawn
Eyes on Main is sponsoring local artists in our community by showcasing their work in our Welcome area!
This quarter's featured artist is Melissa Dawn. Her piece, Timeline, is a collagraph with image transfer created in February of 2014. This grid of 54 squares was printed using seven collagraph plates. The blocks were made of matboard, tape, wallpaper, tin foil, ribbon, and paper. They were printed intaglio. Xerox images were then transferred to the completed grid. The images are all of Melissa's face, from the photo taken on the day she was born to a portrait made last year, thus the name Timeline.
Timeline by Melissa Dawn, Collagraph with Image Transfer
Melissa Dawn is currently working on her Bachelor of Fine Arts, with emphases in Printmaking and Sculpture and a minor in Art History at Montana State University. She received her Masters in the Art of Teaching from the University of South Carolina in 1998, and a Bachelor of Arts degree in Dance, Theatre, and Music from the College of Charleston, South Carolina in 1997.
Melissa has participated in a variety of exhibitions over the years, including Vox Populi at the Helen Copeland gallery in 2013, and Upheaval at Montana State University, The Exit Gallery, and the Waller-Yoblonsky Gallery in 2014. Click the link below to read her Artist's Statement, and stop by the office anytime to view Timeline in person!
Melissa Dawn—Artist Statement
Posted In: Showcase Your Vision
Welcome to the New EYES on MAiN Website! Resolve to Take Better Care of Your Eyes
Showcase Your Vision
Good Practices for Safe Night Driving December 28, 2022
Autoimmune Diseases Can Affect Vision December 14, 2022
EYES on MAiN
1425 West Main Street, Suite B
Copyright © 2023 EYES on MAiN. All rights reserved.

redpajama_set_name: RedPajamaCommonCrawl


To many Americans the Zombies always seemed so British, and British-sounding, the same way we think of the Kinks, or the Jam as being particularly British. Despite that, you were much more successful in the United States than you were in Britain. Why do you think that might have been?
I think if you got off on the wrong foot in the UK — remember that we were young — we'd just come out of school, 18 years-old when we first went to Decca Records. We went to the press department. People were so unaware of how important image was. We would have an hour in the press department at Decca Records. We had some incredibly bad photos taken, and those photos still follow us around now, 50 years later. This is a very important part of an artist. This is your image.
Then they started asking us, well what have you been doing. We'd just left school. So they started conjuring up some kind of image of us being academic geeks. That's the kind of image that was put over in the UK. It's so wrong. People like the Rolling Stones were happening and they were pirates and brigands, and we're the academic geeks. We really got off to a very poor start in the UK. I don't think we ever really recovered.
We had more big hit records in America. We only had three big hit records. We did sound quite English. I felt self-conscious if I tried to sing with an American accent. I know that on a lot of rock and blues songs, it's almost impossible to sing them without an American accent. But I was very fortunate that we had two prolific and quite sophisticated writers in the Zombies. So I was singing their songs rather than blues and rhythm & blues classics. So I was free to sing them in their natural accent.
I've had a chance to re-listen to Odessey & Oracle and I've noticed the very English pronunciation of words. I felt self-conscious about singing them with an American accent so I would sing them in an English accent. I suppose it contributes to giving the tracks a very English feel.
Why add Chris White and Hugh Grundy back into the band now? Why weren't they part of earlier revivals, other than the 2008 concerts in England?
Odessey & Oracle is all of our album. It's not Rod and my album, it's all of ours. The reason why they haven't been involved in the past is because in 1967 when the band finished, Chris gave up playing the bass. He was a very successful writer and producer. He went into the record industry on the other side of the business. He's now retired. But neither of them have been playing professionally. Rod and I, in 1999, decided to play six concerts. It wasn't an option to ask them to join us because they weren't playing. They haven't played since 1967. Hugh would play in sort of a semi-pro band, but Chris hadn't picked up a bass in all that time.
We decided to do six shows and we enjoyed it so much we just kept going. We didn't realize that there was this huge interest in the Zombies repertoire. People were constantly asking us at concerts to play Zombies tunes. This didn't start as a new incarnation of the Zombies, it just evolved organically. People asked us to play more Zombies tunes and eventually promoters started billing us as the Zombies, although contractually they were not supposed to do that.
It got to the point three or four years ago when we sat down with the other original members of the band and said, look, this is what's happening. People are asking for more Zombies tunes. It just seems natural, and a very normal progression for us to start touring as the Zombies. This would give exposure not just to the well known songs but to quite obscure material. They were very supportive and said it sounded like a great idea. It was never our idea, when we first got together, to recreate the Zombies.
We were under a lot of pressure when we recorded that album in that we had a very small recording budget. So we rehearsed all of the songs extensively outside of the studio so that we could go into the studio and record. We knew the arrangements and the harmonies. All we had to do is get a performance out of ourselves and to get that onto tape.
It happened that "Time of the Season" was the last song we recorded, the last song that was written for the album. It was actually finished in the morning before we recorded it. Rod and I had a misunderstanding. It had only just been written and I was struggling to get hold of the melody as we were recording it. The rest of the album had been rehearsed in depth and we knew exactly what was happening.
As I was standing there singing "Time of the Season," Rod kept correcting me through the intercom in the control room. Maybe the phrasing wasn't quite right, maybe I didn't get the melody right. I didn't know the song that well. It did start to get a little fiery, and it ended up with me saying listen, if you know the song so well, you come in here and sing it. He very correctly said to me, you're the lead singer in this band, you stand there and get it right. I'm really glad that I did. Half an hour later it was all forgotten.


Cats and dogs may be carnivores, but that has not stopped a surprising number of pet owners from wistfully wishing they could change their furry friends to plant-based diets – or even.
Of 3,673 cat and dog owners surveyed worldwide, a whopping 35 percent expressed interest in veganizing their carnivorous pets.
The aim of the study was to investigate meat prevention among pet owners. and how it affects the food they feed their pets. In fact, the desire to feed pets with a vegan diet was more common, but not exclusive, in vegan people.
Vegans accounted for 212 (5.8 percent) of respondents, of whom 78 percent said they would feed their pet a plant-based diet. In fact, 58 vegans or 27 percent have already done so. A vegetarian also fed his pet with a vegan diet.
"This percentage of 27 percent may sound like a small number," said veterinarian Sarah Dodd of the University of Guelph in Canada, "but if you think of the actual situation number of animals that are affected is enormous and much higher than we expected. "
Cats, on the other hand, are obligatory carnivores. This means that cats can eat plants, but cats can not be healthy without meat diet. It is currently not a good idea to try to feed your cat with a vegan diet. In fact, in the UK, you could bring criminal charges under the Animal Welfare Act that require pet owners to ensure that all their pet needs are met.
Nevertheless, 11 cat owners of the study did so. together with 48 dog owners.
However, there is good news. Although a large number of respondents expressed interest in a vegan pet diet, most of them did not, unless there was vegan pet food that would suit the nutritional needs of pets.
In fact, over half of the omnivores and the surveyed vegetarians said they found it immoral to feed cats and dogs with a plant-based diet.
Those interested in vegan pet diets called concern for the welfare of farm animals as the most prominent concern about feeding their animals, which makes perfect sense.
This reports eating less meat is good for the environment (although this is controversial), your mental health and your physical health. Veganism is increasing worldwide. This could mean that the interest in vegan pet food is also increasing.
"While only a small percentage of pet owners are currently feeding plant-derived foods to their pets, one can say with certainty that interest in diets is likely to increase, says Dodd.
The findings, the researchers say that the plant-based diet of cats and dogs that live with humans needs to be further explored – whether such a diet can work and what health risks may exist 19659003] Until this is discovered, try not killing your kitten from kale smoothies And if you're trying for a vegan dog diet, do not do so without extensive consultation with your veterinarian.
The study was published in PLOS One .


text:
Winnie the Pooh: Pooh Goes Visiting (Dramatised)
Von: A. A. Milne
Gesprochen von: Stephen Fry, Jane Horrocks, Geoffrey Palmer, Judi Dench, Finty Williams, Robert Daws, Michael Wiliams
Kategorien: Kinder-Hörbücher, Literatur & Belletristik
A wonderful adaption of Winnie the Pooh, featuring an all-star cast including Stephen Fry and Dame Judi Dench. Story 1: In which we are introduced. Story 2: In which Pooh goes visiting and gets into a tight place. Story 3: In which Pooh and Piglet go hunting and nearly catch a Woozle. Story 4: In which Eeyore loses a tail and Pooh finds one. Story 5: In which Piglet is entirely surrounded by water. Story 6: In which Christopher Robin gives a Pooh Party, and we say good-bye.
©1998 Trustees of Pooh Properties (P)1998 Hodder Children's Books
Klassiker Geschichten über Spielzeug
Winnie the Pooh: Tigger Comes to the Forest (Dramatised)
Gesprochen von: Stephen Fry, Jane Horrocks, Geoffrey Palmer, und andere
Originalaufnahme
Join Pooh and Piglet as they make a new friend, Tigger, who bounces his way into the Hundred Acre wood, performed by an amazing all-star cast....
schreckliche Synchronisation
Von Maria Am hilfreichsten 19.01.2020
Winnie the Pooh: Piglet Meets a Heffalump (Dramatised)
Gesprochen von: Stephen Fry, Jane Horrocks, Geffrey Palmer, und andere
Join this all-star cast including as they go on adventures and look for heffalumps with Pooh, Piglet and friends in the Hundred Acre Wood.
Winnie the Pooh: The House at Pooh Corner (Dramatised)
Packed full of fun and adventure with Winnie-the-Pooh and all his friends in the Hundred Acre Forest. This is a wonderful dramatisation of The House at Pooh Corner by David Benedictus with music composed, directed and played by John Gould.
Winnie the Pooh: Pooh Invents a New Game (Dramatised)
Gesprochen von: Stephen Fry, Jane Horrocks, Georffrey Palmer, und andere
Enjoy this classic Winnie the Pooh collection, voiced by an amazing all-star cast.
Winnie the Pooh (Dramatised)
Packed full of fun and adventure with Winnie-the-Pooh and all his friends in the Hundred Acre Forest. This is a wonderful dramatisation of Winnie-the-Pooh by David Benedictus with music composed, directed and played by John Gould.
Sehr witziges und liebevoll gemachtes Hörerlebnis
Von Jochen Am hilfreichsten 11.06.2012
The Sherlock Holmes Collection
Von: Arthur Conan Doyle
Gesprochen von: Stephen Scalon
Sherlock Holmes is a fictional detective of the late 19th and early 20th centuries, who first appeared in publication in 1887. He is the creation of Scottish born author and physician Sir Arthur Conan Doyle. A brilliant London-based detective, Holmes is famous for his intellectual prowess, and is renowned for his skillful use of deductive reasoning (somewhat mistakenly - see inductive reasoning) and astute observation to solve difficult cases.
gut verständlich, leider nicht übersichtlich
Von Anonymer Hörer Am hilfreichsten 19.02.2021
Harry Potter and the Philosopher's Stone, Book 1
Sherlock Holmes: The Definitive Collection
Harry Potter and the Goblet of Fire, Book 4
Das sagen andere Hörer zu Winnie the Pooh: Pooh Goes Visiting (Dramatised)


text:
Favorite Scenes
Their 10 Best
Top Fives & Top Tens
In Theaters This Week
IMDB 250 BEST FILMS
Achievement Lists
About Killing Time
Tag Archives: johnny depp
Books, Harry Potter, movie trailers, Movies
Fantastic Beasts: The Crimes of Grindelwald Trailer #2 (2018) *Who Will Change the Future?*
July 21, 2018 sleeplessdave 2 Comments
After an uneven summer of sequel madness, it's good to look forward to franchises that rarely-if ever-disappoint. The Wizarding World is back this fall with its 10th film installment and the second in the Fantastic Beasts cycle. The second trailer for Fantastic Beasts: The Crimes of Grindelwald was unveiled at Comic-Con today and it's absolutely eye-popping. In addition to the continuing adventures of Newt and company, Potterheads will enjoy Easter eggs like Dumbledore as DADA professor, Nicholas Flamel, and The Mirror of Erised. Check out CBR's notes from Warner Brother's panel discussion of the film below.
Jude Law confirmed that this movie will be a "who's side are you on story," where the "wizards within the Wizarding World will be asked whose side they're on. The depths and the darkness of this story are some of the darkest this franchise has gone." Ezra Miller chimed in "Definitely" in confirmation.
Law continued with his take on Dumbledore, "I love the fact that people keep calling him 'young Dumbledore," because I'm 45," he laughed. "…There's a long way to go between this Dumbledore and the one we see here. David Yates our director really let me free to establish him on my own here, but I hope he'll have some familiar traits. He's still mischievous and…he still has this ability to manipulate people."
The panel was then surprise interrupted by Johnny Depp appearing in full costume, in character as Grindelwald giving an ominous speech. "It has been said that I hate the muggles, the no-mag, the can't-spells, I do not hate them," Depp said in character, giving fans an insight into Grindelwald's personality. Depp immediately left the stage after his performance.
Fantastic Beasts: The Crimes of Grindelwald is being directed by David Yates, from a screenplay by J.K. Rowling, and produced by David Heyman. The film Opens in theaters November 16, 2018.
Albus DumbledoreEddie RedmayneFantastic Beasts 2Fantastic Beasts: The Crimes of GrindelwaldFantastic Beasts: The Crimes of Grindelwald Trailer 2Hogwartsjohnny deppjude lawSan Diego Comic-Con 2018SDCC 2018Wizarding World
movie trailers, Movies, Music
City of Lies Trailer #1 (2018) "Who Shot Biggie?"
May 29, 2018 sleeplessdave 1 Comment
City of Lies, the crime drama that follows the investigation behind the unsolved murder of the Notorious B.I.G., aka Biggie Smalls, following the death of Tupac Shakur, has released its first trailer.
Based on the book LAbyrinth by Randall Sullivan, the film stars Johnny Depp as LAPD detective Russell Poole and Forest Whitaker as journalist Jack Johnson. The two team in an attempt to solve the mystery of the rapper's death and the conspiracy to cover up the case.
"Why is this case so important to you?" Johnson asks in the beginning of the trailer. Poole responds, "A murder like that only goes unsolved if the police don't want to solve it."
After discussing some theories regarding Smalls' death, Johnson asserts, "If we could prove the connection between cop and Biggie's murder, it would not only break the LAPD…" Poole finishes the thought: "It would ruin the city."
Brad Furman directed the movie, which was produced by Miriam Segal's Good Films.
City of Lies is set to bow Sept. 7.
*Text from The Hollywood Reporter
Brad FurmanCity of LiesForest Whitakerjohnny deppLabyrinthLAPDMurder of Biggie and TupacNotorious B.I.G.Police CorruptionRapTupac Shakur
Fantastic Beasts: The Crimes of Grindelwald Trailer #1 (2018) *Who Will Stand Against the Darkness?*
March 13, 2018 sleeplessdave 11 Comments
Here's a breakdown of the photo:
Pictured from left to right: Jude Law plays a young Albus Dumbledore, taking on the mantle of one of J.K. Rowling's most beloved characters; Ezra Miller makes a return as the enigmatic Credence, whose fate was unknown at the end of the first film; Claudia Kim appears as a Maledictus, the carrier of a blood curse that destines her ultimately to transform into a beast; Zoe Kravitz plays Leta Lestrange, who had once been close to Newt Scamander but is now engaged to his brother; Callum Turner joins the cast as Newt's older brother, Theseus Scamander, a celebrated war hero and the Head of the Auror Office at the British Ministry of Magic; Katherine Waterston returns as Tina Goldstein, who has been reinstated as an Auror for MACUSA; Eddie Redmayne stars again as wizarding world Magizoologist Newt Scamander, who has now gained fame in the wizarding world as the author of Fantastic Beasts and Where to Find Them; Dan Fogler reprises the role of the only No-Maj in the group, Jacob Kowalski; Alison Sudol reprises the part of Tina's free-spirited sister, Queenie Goldstein, a Legilimens who can read minds; and Johnny Depp returns as the powerful Dark wizard Gellert Grindelwald.
Albus DumbledoreEddie RedmayneFantastic Beasts 2Fantastic Beasts: The Crimes of GrindelwaldFantastic Beasts: The Crimes of Grindelwald TrailerGellert GrindelwaldHogwartsjohnny deppjude lawNewt ScamanderWizarding World
Books, movie review, Movies
Movie Review: Murder on the Orient Express (2017) *The Scenery Is Nice, But The Ride Is Bumpy*
November 10, 2017 sleeplessdave 14 Comments
It's difficult to know how to review Murder on the Orient Express. Should it be judged against the 1970s film? Should it be held against the Agatha Christie novel it adapts? In the end, I decided to judge it on its on merits as a standalone film. To that standard, the picture is well-acted with a great ensemble. Kenneth Branagh does well as the lead, but in the direction and especially the screenplay the film is choppy and almost unbearably expository in its climax. It looks absolutely fantastic, but I doubt Murder is going to inspire a Christie renaissance or have a tremendous impact on the box office. It's not awful, but it's not anywhere near what it should be, and that won't be enough to attract an audience not already in love with the source material.
Continue reading Movie Review: Murder on the Orient Express (2017) *The Scenery Is Nice, But The Ride Is Bumpy* →
Agatha ChristieDaisy RidleyDerek Jacobijohnny deppjudi denchKenneth BranaghLeslie Odom Jr.Michelle PfiefferMurder on the Orient ExpressMurder on the Orient Express Movie ReviewPenélope CruzWillem Dafoe
In Theaters This Week, Marvel Cinematic Universe, Movies
In Theaters This Week (11/10/2017): Murder on the Orient Express, Daddy's Home 2, Three Billboards
November 9, 2017 sleeplessdave 2 Comments
Each Thursday we look at what is going to be coming out in theaters this weekend, show you the trailers for the big releases, predict the box office winner and just generally give you enough of a carrot to pull you through the rest of the work week. This week brings three very different films to theaters, but can any of them knock the Odinson off his throne? (No) Continue reading In Theaters This Week (11/10/2017): Murder on the Orient Express, Daddy's Home 2, Three Billboards →
A Bad Moms ChristmasBox OfficeDaddy's Home 2Daddy's Home 2 TrailerDaisy Ridleyjohnny deppMurder on the Orient ExpressMurder on the Orient Express TrailerThor: RagnarokThree Billboards Outside Ebbing MissouriThree Billboards Outside Ebbing Missouri TrailerWill Ferrell
Luke Cage Season 2 Trailer #1 (2018-Netflix) *Hero Is Your Word, Not Mine.*
My Favorite Scene: The Good Place Season 2 (2017) "The Trolley Problem"
My Favorite Scene: Avatar (2009) "The Hallelujah Mountains"
Killing Time Archives
Killing Time Archives Select Month February 2019 (1) January 2019 (5) August 2018 (4) July 2018 (36) June 2018 (33) May 2018 (37) April 2018 (36) March 2018 (36) February 2018 (36) January 2018 (36) December 2017 (35) November 2017 (37) October 2017 (51) September 2017 (51) August 2017 (41) July 2017 (44) June 2017 (35) May 2017 (39) April 2017 (28) March 2017 (32) February 2017 (30) January 2017 (42) December 2016 (39) November 2016 (44) October 2016 (41) September 2016 (41) August 2016 (37) July 2016 (49) June 2016 (35) May 2016 (42) April 2016 (33) March 2016 (36) February 2016 (29) January 2016 (35) December 2015 (34) November 2015 (37) October 2015 (32) September 2015 (32) August 2015 (32) July 2015 (52) June 2015 (41) May 2015 (43) April 2015 (36) March 2015 (42) February 2015 (38) January 2015 (38) December 2014 (23) November 2014 (40) October 2014 (6) September 2014 (48) August 2014 (70) July 2014 (93) June 2014 (84) May 2014 (77) April 2014 (74) March 2014 (105) February 2014 (92) January 2014 (87) December 2013 (92) November 2013 (107) October 2013 (131) September 2013 (108) August 2013 (124) July 2013 (105) June 2013 (124) May 2013 (111) April 2013 (105) March 2013 (84) February 2013 (54) January 2013 (11) November 2010 (3) October 2010 (9) September 2010 (14) August 2010 (19) July 2010 (16) June 2010 (13) May 2010 (16) April 2010 (16) March 2010 (19) February 2010 (18) January 2008 (5) December 2007 (3) November 2007 (6) October 2007 (6) May 2007 (3) March 2007 (2) February 2007 (6) January 2007 (19) December 2006 (23)
ian mckellan
Star Wars Episode VII: The Force Awakens
the hobbit the desolation of smaug
thorin oakenshield