## Joint Neural Networks for One-shot Object Recognition and Detection

Camilo J. Vargas, Qianni Zhang, Ebroul Izquierdo.

Abstract -This paper presents a novel joint neural networks approach to address the challenging one-shot object recognition and detection tasks. Inspired by Siamese neural networks and state-of-art multi-box detection approaches, the joint neural networks are able to perform object recognition and detection for categories that remain unseen during the training process. Following the one-shot object recognition/detection constraints, the training and testing datasets do not contain overlapped classes, in other words, all the test classes remain unseen during training. The joint networks architecture is able to effectively compare pairs of images via stacked convolutional layers of the query and target inputs, recognising patterns of the same input query category without relying on previous training around this category. The proposed approach achieves 61.41% accuracy for one-shot object recognition on the MiniImageNet dataset and 47.1% mAP for one-shot object detection when trained on the COCO dataset and tested using the Pascal VOC dataset. Code available at https://github.com/cjvargasc/JNN recog and https://github.com/cjvargasc/JNN detection/.

Index Terms -One-shot detection, One-shot recognition, multibox detection, pattern recognition.

## I. INTRODUCTION

T HE automatic recognition, description, classification, and grouping of patterns are relevant tasks in a variety of engineering and scientific disciplines such as biology, psychology, medicine, marketing, security, transport, and remote sensing. Regardless of almost 50 years of research, the design of a general-purpose machine pattern recognizer remains an elusive goal [1][2].

Deep Neural Networks (DNNs) have achieved the stateof-the-art performance on a variety of pattern recognition tasks, most notably visual classification problems. Typically, a very large amount of training samples are required to achieve state-of-art results [3]. In contrast, real world applications often require to achieve high recognition performance with very limited datasets. In those cases the classic deep neural networks fall short, leading to overfitting on the training sets and poor generalizations on the test sets [4]. Thus, the capability to generalize visual concepts from small amounts of data becomes a highly desired characteristic in modern computer vision systems

This paper introduces Joint Neural Networks to tackle the one-shot object recognition and detection tasks. Here, the Joint Neural Network model is trained as a matching network

- C. J. Vargas, Q. Zhang and E. Izquierdo are with the School of Electronic Engineering and Computer Science, Queen Mary University of London, London. UK. E-mail: { c.j.vargascortes, qianni.zhang, ebroul.izquierdo } @qmul.ac.uk

to perform recognition or detection in pairs of images. The proposed approach is tested using input query and target images belonging to categories that remain unseen during the training stage. Such approach is applicable to any field where data availability may be reduced to a single sample, where traditional machine learning and data augmentation approaches fall short.

Two different architectures akin to Siamese neural networks (SNNs) are presented in this paper to tackle the recognition and detection tasks. Both Joint Neural Network architectures rely on the joint layer key concept. The joint layers proposed in this paper are convolutional layers that merge the convolutional filters resulting from the two branches of the Joint Network architectures via concatenation. The outputs of the joint layers are built to match the defined backbone architecture used for the Joint Neural Network.

For recognition, the proposed novel approach is akin to the well known SNN with an AlexNet backbone. Their performance is compared in the experimental section along with several few-shot learning approaches in the context of one-shot recognition. For detection, the proposed recognition approach is adapted inspired by one-stage multi-box detection approaches similar to that introduced by YOLO [5] [6]. The main contributions of this paper are:

- -A scalable framework for object one-shot recognition and detection inspired by Siamese Neural Networks and anchor based one-stage multibox detectors, that additionally do not require support sets or additional training/fine-tuning for unseen classes.
- -The introduction of two novel Joint Neural Networks architectures for one-shot recognition and detection respectively.

## II. RELATED WORK

Previous to modern machine learning approaches, the initial approaches to engage in the one-shot object detection and recognition problems were based on image processing techniques. The most popular image processing based approaches relied on the localisation and description of interest points or keypoints, as it is the case of SIFT [7] and SURF [8] for object detection. Approaches based on keypoint detection and description are often designed to offer feature scale, rotational, and illumination invariance. When applied in realistic scenarios, one of the main withdraws of such approaches is the limited capability of matching feature keypoints under illumination or structural variations in particular when the background scenery is complex.

As the performance of image processing based approaches reached a plateau new machine learning approaches were

introduced to address the one-shot problem. The goal of machine learning algorithms is to discover statistical structures within data, but building modern models such as convolutional neural networks (CNNs) from scratch requires a large amounts of labeled training data [6]. Following [9], the seminal work towards one-shot learning dates back to the early 2000's with work in [10] where the authors developed a variational Bayesian framework for one-shot image classification using the premise that previously learned classes can be leveraged to help forecast future ones when very few examples are available from a given class [10] [11]. However, these approaches are still based on keypoints, inheriting the same drawbacks and the problem remains unsolved for real world scenarios.

Data synthesis and transfer learning are commonly used techniques to support models carrying out the detection or recognition tasks when the labeled training data is limited. Data synthesis focuses on producing artificial data either manually from scratch or using advanced data manipulation techniques to produce novel and diverse training examples. When creating new exemplars from existing instances, the available data must be enough to render novel and diverse instances to produce novel and diverse training examples [12]. In general, the data synthesis approach serves well for cases in which at least a few samples of the target categories are available but its effectiveness is limited for one-shot object recognition/detection.

Transfer learning is one of the most relevant tools in machine learning, proposed to solve the insufficient training data problem. It aims to transfer the knowledge from a source domain to a target domain by relaxing the assumption that the training data and the test data must be independent and identically distributed. This leads to a great positive effect on many domains that are difficult to engage with because of insufficient training data [13][14]. Fine-tuning remains the method of choice for transfer learning with neural networks in which a model is pre-trained on a source domain (where data is often abundant), the output layers of the model are adapted to the target domain, and the network is fine-tuned via back-propagation. This approach was pioneered in [15] by transferring knowledge from a generative to a discriminative model, and has since been generalized with great success [16][17][18]. Traditionally, this method also requires at least a few samples of the target domain categories to be available in order to perform the fine-tuning training process, which becomes a limiting factor when dealing with one-shot detection or recognition.

Recently, a set of approaches for few-shot recognition and suitable for one-shot object recognition have been proposed [19], [20], [21], [22], [23], [24]. The goal of meta-learning is to train a model on a variety of learning tasks, such that it can solve new learning tasks using only a small number of training samples. The work presented in [19] proposes a general and model-agnostic meta-learning algorithm. The algorithm is model-agnostic in the sense that it can be directly applied to any learning problem and model that is trained with a gradient descent procedure. The key idea behind the proposed approach is to train the model's initial parameters such that the model has maximal performance on a new task after the parameters have been updated through one or more gradient steps computed with a small amount of data from that new task.

A set of architectures called Siamese Neural Networks (SNNs) have been also proposed to tackle to one-shot recognition task. A SNN consists of twin neural networks with tied parameters that takes two inputs and join them using an cost function at the top. This function computes a distance metric between resulting the highest-level feature representation on each side. Once trained, SNNs can then capitalise on powerful discriminative features to generalise the predictive power of the network not just to new data, but to entirely new classes from unknown distributions. Furthermore, the shared weights constraint on the twin CNN's guarantees that two extremely similar images could not possibly be mapped by their respective networks to very different locations in feature space given that each network computes the same function [9].

Siamese Neural Networks have been applied for a wide set of tasks including character recognition [9], person reidentification [25], face verification [26] and tracking [27]; given their balance between accuracy and speed [28] for oneshot recognition. Traditionally, Siamese networks were trained with the so called 'contrastive loss' [29] [30]. The main idea here is to construct a Euclidean space where the outputs of the sister networks are close to each other for matching pairs while simultaneously negative pairs are pushed away [31].

Although, most of the work found in the literature is focused on one-shot object recognition, recent research works have started pioneering possible solutions to the challenging oneshot detection problem [32][33] and extending it to fields such as semantic segmentation [34] and visual tracking[35][36]. In contrast to the work presented in this paper, the aforementioned approaches are based on two-stage object detection approaches.

This work introduces an approach inspired by SNNs to tackle the one-shot object recognition problem, that additionally uses concepts from one-stage detectors such as YOLO [6] to extend its application to the one-shot object detection problem. Following the traditional object detection literature case where enough data is available for training, one-stage detectors offer a trade-off between accuracy and speed favorable for real world applications. The novel architecture proposed is based on the idea of allowing the model to Figure out by itself the convolutional activations that lead to pattern matching and locations definition, in contrast to the common encoding comparison at the top of the Siamese sister networks.

## III. PROPOSED APPROACH

The fundamental idea behind the proposed joint neural networks is to allow the convolutional layers to learn the activations that define matches between pairs of images, instead of having a single comparison layer at the top of the architecture as in SNNs.

For this purpose we define joint convolutional layers , which are the backbone of Joint neural networks and can be used within any convolutional neural network architecture. The so called joint layers are convolutional layers like those found

Fig. 1: Proposed joint layer structure.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-1b21d250-51ca-4a54-a3f1-ed434a58cea1.jpg)

in common convolutional neural networks; where, the feature vector taken as input is the result of a concatenation operation between the previous convolutional layers on each branch of the Joint neural network. Figure 1 illustrates the structure of a joint layer. The output of the proposed joint layers is defined to match the architecture used as backbone of the Joint neural network. In this work the AlexNet and DarkNet19 architectures are used to tackle the recognition and detection problems respectively.

In Figure1 f i ( x (1) ) and f i ( x (2) ) are the feature vectors outputted by the i th convolutional layer for the first and second branch respectively. F j ( f i ( x (1) ) ⊕ f i ( x (2) )) is the feature vector outputted by the j th joint layer. And ⊕ is the standard concatenation operation merging the output vectors f i ( x (1) ) and f i ( x (2) ) .

## A. Recognition

When undertaking the one-shot recognition problem the main goal is to perform pattern matching for a pair of images x (1) , x (2) ; outputting a predicted similarity metric P in the same manner as traditional Siamese neural networks. Given that AlexNet is an architecture commonly used as the backbone of SNNs [27] we adopt the same base architecture for demonstration and comparison purposes.

Figure 2 depicts the proposed architecture showing the two branches of the network. The main branch of the architecture at the top of Figure 2 is built over the AlexNet architecture with the additional joint layers that will integrate outputs from the second branch at the bottom, thus supporting information sharing between the network branches. The convolutional layers in the secondary branch of the architecture share weights with the main branch, the same way as sister networks in traditional SNN approaches. Additionally, the last fully connected layer of the architecture consists of a single neuron that will define the similarity metric P . The joint layers will take as input the concatenation of the previous outputs on both branches of the network and keep the AlexNet structure for following layers. Table I shows a detailed description of the architecture layers.

In contrast to the traditional contrastive loss function this work uses the binary cross entropy (BCE) loss function for training the proposed architecture as defined in equation 1. The loss function is defined to comply with the SNN architecture previously defined and the specified one-shot constraints, where, the model is trained to output a similarity measure that is small for matches and big for dissimilar mismatches.

TABLE I: Detailed description of the proposed architecture using AlexNet for one-shot recognition

| Type            | Filters   | Size/Stride   |
|-----------------|-----------|---------------|
| Convolutional 1 | 64        | 11x11/4       |
| Maxpool         |           | 3x3/2         |
| Joint layer 1   | 64        | 3x3/1         |
| Convolutional 2 | 192       | 5x5/1         |
| Maxpool         |           | 3x3/2         |
| Joint layer 2   | 192       | 5x5/1         |
| Convolutional 3 | 384       | 3x3/1         |
| Joint layer 3   | 384       | 3x3/1         |
| Convolutional 4 | 256       | 3x3/1         |
| Convolutional 5 | 256       | 3x3/1         |
| Maxpool         |           | 3x3/2         |
| Linear          | 4096      |               |
| Linear          | 4096      |               |
| Linear          | 1         |               |

$$\text{ctors} \\ \text{cond} \\ \text{ature} \\ x ^ { ( 1 ) } ) & = - \frac { 1 } { N } \sum _ { i = 1 } ^ { N } y _ { i } ( x ^ { ( 1 ) }, x ^ { ( 2 ) } ) \cdot l o g ( p ( x ^ { ( 1 ) }, x ^ { ( 2 ) } ) ) \\ + & ( 1 - y _ { i } ( x ^ { ( 1 ) }, x ^ { ( 2 ) } ) \cdot l o g ( 1 - p ( x ^ { ( 1 ) }, x ^ { ( 2 ) } ) ),$$

where y is the matching label between the inputs x (1) , x (2) , which equals to 1 for matching input pairs and 0 otherwise. p x ( (1) , x (2) ) is the predicted label and N is the batch size.

In order to achieve the one-shot detection task the proposed model is trained with categories ignored during testing. For this purpose the training data loader will select a set of random queries for each epoch with one single query image per batch matched with a random target image that contains the query image 50% of times. Each query and target image is then passed through the model and paired with the ground truth annotations to compute the loss function.

## B. Detection

The presented recognition approach is adapted to perform multi-box one-shot object detection given a query and target images containing a pattern of interest from a category unseen during training. The proposed Joint Neural Network is defined as a multi-box detector based on the Darknet19 architecture [5], taking the query and target images x (1) , x (2) as input and outputting a tensor ˆ P representing the locations where the object of interest may be present in x (2) .

Following [6] we define a S × S grid over the target image x (2) . Each grid cell predicts B anchor window coordinates and confidence score ( x, y, w, h, conf ) ; where, the output predictions are encoded as a S × S × ( B × 5) tensor. In contrast with [6] the category to be detected in the target image x (2) is dictated by the input query image x (1) , so the output tensor does not require additional position to define prediction classes. The confidence scores are trained as intersection over union predictor which will have a higher value when its respective anchor window correctly contains the object of interest.

Fig. 2: Proposed AlexNet-based recognition architecture.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-0cca71eb-0f66-4b91-a058-ac687e086660.jpg)

Fig. 3: Anchor window definition [6]. Where ( b x , b y ) and ( b w , b h ) define the center coordinates and size of the anchor window respectively

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-4fac78f6-e19d-4107-ab8b-b038b77be9fd.jpg)

The approach and notation for defining the anchor windows are inspired by [6] is used to define the anchor windows. For each anchor window in the S × S grid the model yields to five output values t x , t y , t w , t h , and t o . The cell offsets from the top left corner of the image are defined as ( c x , c y ) and the bounding box prior has width and height p w , p h . The resulting predictions are obtained using equations 2 - 6 as illustrated in Figure 3.

$$b _ { x } = \sigma ( t _ { x } ) + c _ { x }$$

$$b _ { y } = \sigma ( t _ { y } ) + c _ { y }$$

$$b _ { w } = p _ { w } e ^ { t _ { w } }$$

$$b _ { h } = p _ { h } e ^ { t _ { h } }$$

$$c o n f = \sigma ( t _ { o } )$$

The proposed model is essentially based on Darknet19 [6], adapted to take two inputs on separated branches. Both branches share weights and are connected by convolutional layers ( joint layers ) at several points on the pipeline, ending in a single convolutional layer encoding the S × S × ( B × 5)

predictions tensor; where S = 14 for the proposed architecture. Figure 4 shows the base network structure for illustrative purposes in accordance with Table 4. The architecture used for experiments slightly differs from Figure 4 following the results presented in section IV-C.

The input sizes for the target and query images are 448 × 448 and 224 × 224 respectively. By adapting the architecture to

TABLE II: Detailed description of the proposed DarkNet19based architecture.

| Type             | Filters   | Size/Stride   | Output   |
|------------------|-----------|---------------|----------|
| Convolutional 1  | 32        | 3x3           | 224x224  |
| Maxpool          |           | 2x2 / 2       | 112x112  |
| Convolutional 2  | 64        | 3x3           | 112x112  |
| Joint conv. 1    | 64        | 3x3           | 112x112  |
| Convolutional 3  | 128       | 3x3           | 112x112  |
| Convolutional 4  | 64        | 1x1           | 112x112  |
| Convolutional 5  | 128       | 3x3           | 112x112  |
| Maxpool          |           | 2x2 / 2       | 56x56    |
| Joint conv. 2    | 128       | 3x3           | 56x56    |
| Convolutional 6  | 256       | 3x3           | 56x56    |
| Convolutional 7  | 128       | 1x1           | 56x56    |
| Convolutional 8  | 256       | 3x3           | 56x56    |
| Maxpool          |           | 2x2 / 2       | 28x28    |
| Joint conv. 3    | 256       | 3x3           | 28x28    |
| Convolutional 9  | 512       | 3x3           | 28x28    |
| Convolutional 10 | 256       | 1x1           | 28x28    |
| Convolutional 11 | 512       | 3x3           | 28x28    |
| Convolutional 12 | 256       | 1x1           | 28x28    |
| Convolutional 13 | 512       | 3x3           | 28x28    |
| Joint conv. 4    | 512       | 1x1           | 28x28    |
| Convolutional 14 | 64        | 1x1           | 28x28    |
| Maxpool          |           | 2x2 / 2       | 14x14    |
| Convolutional 15 | 1024      | 3x3           | 14x14    |
| Convolutional 16 | 512       | 1x1           | 14x14    |
| Convolutional 17 | 1024      | 3x3           | 14x14    |
| Convolutional 18 | 512       | 1x1           | 14x14    |
| Convolutional 19 | 1024      | 3x3           | 14x14    |
| Convolutional 20 | 1024      | 3x3           | 14x14    |
| Convolutional 21 | 1024      | 3x3           | 14x14    |
| Joint conv. 5    | 1024      | 3x3           | 14x14    |
| Convolutional 22 | 1024      | 3x3           | 14x14    |
| Convolutional 23 | B ∗ 5     | 1x1           | 14x14    |

Fig. 4: JNN DarkNet19-based detection architecture.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-2f2cc57d-8c56-4831-8657-7c9bff110d08.jpg)

apply the first pooling layer on the target sister network only, the output sizes for the first 2 convolutional layers become even. This small adaptation makes the concatenation operation and further convolutional layers computation straightforward. In addition,the called joint convolutional layers in Figure 4 are designed to take the concatenation of the previous convolutional layer outputs for each sister network as input and accommodate its output following the DarkNet19 architecture [6] in the same way as the residual residual layer.Finally, the last convolutional layer outputs a 14 × 14 tensor composed of B × 5 filters that will encode the model predictions.

The loss function ℓ used for this work is defined as the weighted sum of losses for localisation and confidence components. The first term being the location loss ℓ loc , deals with how close the predicted box location is to the ground truth annotation; then, the confidence loss term ℓ conf represents how certain the model is that the pattern of interest is located inside a specific anchor box.

$$\ell = \ell _ { l o c } + \ell _ { c o n f }, \quad \ \ \ ( 7 ) \ \frac { \text{eval} } { \ a \ e \ e }$$

$$\ell _ { l o c } = \lambda _ { c o o r d } \sum _ { i = 1 } ^ { S ^ { 2 } } \sum _ { j = 1 } ^ { B } \mathbb { 1 } _ { i j } ^ { o b j } ( y - y _ { p r e d } ) ^ { 2 }, \quad \ ( 8 ) \quad \text{the} \quad \text{the} \quad \text{$\text{$doctools}$}$$

$$\ell.$$

$$\ell _ { l o c } = \lambda _ { c o o r d } \sum _ { i = 1 } ^ { S ^ { 2 } } \sum _ { j = 1 } ^ { B } \mathbb { 1 } _ { i j } ^ { o b j } ( y - y _ { p r e d } ) ^ { 2 }, \quad ( 8 ) \quad \text{between} \quad \text{The} \\ \ell _ { c o n f } = \lambda _ { o b j } \sum _ { i = 1 } ^ { S ^ { 2 } } \sum _ { j = 1 } ^ { B } \left [ \mathbb { 1 } _ { i j } ^ { o b j } c \cdot \log ( c _ { p r e d } ) + \quad \text{template} \\ ( 1 - c ) \cdot \log ( 1 - c _ { p r e d } ) \right ] + \quad \text{Belgal} \\ \lambda _ { n o o b j } \sum _ { i = 1 } ^ { S ^ { 2 } } \sum _ { j = 1 } ^ { B } \left [ \mathbb { 1 } _ { i j } ^ { n o o b j } c \cdot \log ( c _ { p r e d } ) + \quad \text{of one} \\ ( 1 - c ) \cdot \log ( 1 - c _ { p r e d } ) \right ], \quad \text{125} \ c \cdot \\ \text{where} \, S \, is \, the \, size \, of \, the \, grid ; \, B \, is \, the \, number \, of \, \text{anchors} \quad \text{for} \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d} ; \, \text{d}.$$

where S is the size of the grid; B is the number of anchors defined per grid cell; y and y pred are the ground truth and prediction tensors containing the b x , b y , b w , b h values; c and c pred are the ground truth and predicted confidences ( conf ); ✶ obj ij is equals to 1 where the anchor window j at cell i has an Intersection of Union IoU &gt; 0 5 . with the target pattern and

0 otherwise; ✶ noobj ij is ✶ obj ij complement; and λ coord , λ obj , and λ noobj are weights to deal with the relevance of each term in the loss function. In contrast to [5] we use the Binary Cross Entropy function for the confidence terms along with the λ obj , and λ noobj weights to deal with the learning priority when keeping the true positives or discarding false positives. The data loading for learning follows the same protocol as in recognition; where the target has a 50% chance of containing the query pattern.

## IV. EXPERIMENTS

This section presents the performance evaluation of the proposed approach initially for the recognition task using the AlexNet adaptation of the joint neural network architecture and afterwards using the Darknet variation for object detection. The experiments are built over three different datasets: The QMUL-OpenLogo, MiniImagenet, and pascalVOC datasets.

QMUL-OpenLogo Dataset: The OpenLogo dataset introduced in [37] is presented as a challenging benchmark to evaluate logo detection methods under realistic conditions. The dataset contains approximately 27,000 images from more than 300 logo classes captured from different sources. Figure 5 (row 2) presents a small sample set that depicts the variance between the captures and the classes in the dataset.

The OpenLogo dataset exploits existing logo detection datasets; i.e., the dataset contains logo images from several different existing datasets and heterogeneous sources in an attempt to reproduce realistic scenarios [37]. These datasets are: FlickrLogos-27 [38], FlickrLogos-32 [39], Logo32plus [40], BelgaLogos [41], WebLogo-2M [42], Logo-In-The-Wild [43], SportsLogo [44]. To the best of our knowledge, the OpenLogo dataset has not been used by any published work in the context of one-shot recognition or detection.

The dataset is split into 210 categories for training and 125 categories for testing. For recongition tests both input images are cropped, while only the query image is cropped for detection experiments. Again, there is no category overlap between the training and testing sub-sets which guarantees that the embedded CNN has no knowledge of any of the tested query inputs and complies with the one-shot constraints. The categories on the QMUL-OpenLogo dataset are unbalanced, with some of the logos having as little data as 2 instances.

Fig. 5: Subset sample from the MiniImageNet (row 1), QMUL-OpenLogo (row 2), Pascal VOC (row 3), and COCO (row 4) datasets.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-1bdca0ee-43c3-4c7c-b1c5-63f1e2fded89.jpg)

Given that the proposed architecture works with pairs of images it is possible to make enough pairs testing purposes while keeping data balance. Therefore, approximately 20,000 input pairs are formed from the 125 testing categories available with a 50/50 split on matches and mismatches for the unseen testing categories.

MiniImageNet Dataset: The MiniImageNet dataset is derived from the ImageNet [45], first proposed by [46]. MiniImageNet consists of 60,000 84 × 84 images representing 100 classes with 600 instances each. The exact 600 images that comprise each of the 100 classes in miniImageNet can be found at https://goo.gl/e3orz6 and the images can be directly downloaded from https://github.com/yaoyao-liu/ mini-imagenet-tools. The miniImageNet dataset splits 80 classes for training and 20 for testing. Again, the training and testing classes do not overlap following the one-shot problem constraints. Figure 5 presents (row 1) a set sample images from the miniImageNet dataset.

Pascal VOC Dataset: Pascal VOC is a collection of datasets for object detection [47] used for the Pascal Visual Object Classes challenge. The most commonly used combination for benchmarking traditional object detection approaches is using the VOC 2007 trainval and 2012 trainval sets for training and the VOC 2007 test set for validation. These datasets contain 10.000 and 22.500 images respectively, and each image is annotated with one or several labels, corresponding to 20 object categories. Figure 5 (row 3) presents a set of sample images from the VOC dataset.

COCO Dataset: COCO is a dataset that places the question of object recognition in the context of the broader question of scene understanding. The dataset is built from images of complex everyday scenes containing common objects in their natural context. The COCO dataset contains over a hundred thousand annotated images over 80 categories. The object categories in the data include animals, transport vehicles and furniture [48].

## A. Recognition results

In order to evaluate the performance of the proposed approach for the one-shot recognition task results are presented using true positive rates, false positive rates, accuracy, precision, and the ROC curve. In this subsection the proposed JNN based recognition approach is compared with the traditional SNN approach using the AlexNet architecture backbone over the OpenLogo dataset. Additionaly, the proposed approach is compared with few-shot learning methods [19], [20], [21], [22], [23] over the MiniImageNet dataset.

Given that different thresholds over the learned similarity metric P lead to differences in the performance of the binary classifier, a set of 20 thresholds is used over the similarity metric within the range of responses returned by the Siamese neural network in order to compute the ROC curve. The threshold with the best accuracy is latter used for comparison purposes.

The proposed AlexNet variation of the architecture was trained 200 epochs with a 0 005 . learning rate. Each training batch contained 64 pairs of images randomly selected from the training set. The training process was around 9 hours to complete on a GeForce GTX 1080. The proposed approach as well as the Siamese AlexNet architecture used for experiments are implemented using the pythorch library.

Figure 6 shows the performance of the proposed JNN for logo recognition in terms of the ROC curve obtained by thresholding the similarity metric S . For each one of the models, the similarity metric is thresholded in the range of responses of the model to compute and plot the true positive ( TPR ) and false positive ( FPR ) rates. The accuracy, precision, and area under the ROC curve ( AUC ), reported for each method is measured for the threshold with the highest accuracy.

Fig. 6: ROC curve showing the performance of the proposed approach and the standard SNN approach on the QMULOpenLogo dataset.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-5b9730ff-c936-478d-866a-b81fed09a618.jpg)

Table III shows the result scores for the approaches in Figure 6. The proposed JNN shows a significant improvement over the AUC, FPR and accuracy scores when compared with the standard SNN AlexNet approach ( AUC = 0 76 . , FPR = 0 25 . , acc = 0 70 . ).

TABLE III: Performance metrics for the proposed approach and SNN AlexNet.

| Method   |   TPR |   FPR |   acc |   Pr |   AUC |
|----------|-------|-------|-------|------|-------|
| SNN      |  0.67 |  0.44 |   0.6 | 0.59 |  0.65 |
| JNN      |  0.66 |  0.25 |   0.7 | 0.72 |  0.76 |

Table IV shows the accuracy scores for state-of-the-art one-shot learning approaches assessed in miniImageNet for object recognition. The proposed approach outperforms these methods without any fine-tuning or learning associated to the novel tested categories. This means that, compared to fewshot based learning, the presented JNN can fit a wider range of application scenarios, with reasonably accurate prediction outcomes at a significant smaller cost.

TABLE IV: Accuracy measured for the proposed approach compared to state-of-art approaches for one-shot recognition on the miniImageNet dataset.

| Method           |   accuracy (%) |
|------------------|----------------|
| MALM (2017)      |          48.7  |
| ProtoNets (2017) |          49.42 |
| SNAIL (2018)     |          55.71 |
| TADAM (2019)     |          58.5  |
| MLT (2019)       |          61.2  |
| JNN              |          61.41 |

Figure 7 depicts the predicted value P for a few arbitrarily selected patterns from the OpenLogo and MiniImageNet datasets unseen during training.

## B. Detection results

The proposed JNN based detection approach is tested under four experimental settings using the QMUL-OpenLogo, VOC, and COCO datasets. First, we follow the experiment proposed in [32] by training the proposed model in 16 of the 20 classes of the VOC dataset and testing on the additional 4 classes. The model is trained using the VOC 2007 and 2012 train&amp;val set and evaluated using the VOC 2007 test set. Second, we use the experimental approach presented in [49]; where the proposed model is trained in the COCO dataset and tested in the VOC excluding the overlapping classes during training. Third, following [34] the model is trained and tested using the COCO dataset using four different splits for the training and testing classes. Finally, the model is tested on the QMULOpenLogo dataset using a 60/40 split for training and testing classes selected arbitrarily and complying with the one-shot constraints.

The proposed JNN is trained using the SGD optimizer with a learning rate equals to 0.0001, a 0.9 momentum for 160 epochs, with a 16 batch size, over a single GeForce GTX 1080 GPU. All the experiments are performed using an IoU threshold of 0.5.

Table V shows the results for the first experimental setting, along with the train and test class splits for the VOC dataset as in [32]. Under this experimental settings, results show a 13% mAP difference to the top performing approach [49]. However, it should be noted that in contrast to the results presented from the previous approaches we do not pre-train our model. The results table presented in [49] use the pretrained models on the full ImageNet dataset where the test classes may be present, meaning that the one-shot constraint is not in strict use.

The authors in [49] go further, experimenting with one-shot object detection using the COCO and VOC datasets. Following their approach, the second experimental setting presented in this paper for one-shot object detection trains the proposed model on the COCO dataset excluding the VOC classes used for testing. Table VI shows the results compared to the MFCOS approach [49]. Results show a mAP improvement using the JNN based detection JNN based detection approach presented in this paper when compared to the top performing approach [49] in Table V.

In addition, the performance of the JNN is also compared to the state-of-the-art approaches using the experimental approach presented in [34] and [32]. In this case, four different splits of training and testing categories are defined over the COCO dataset. The fully detailed category splits can be found in the first appendix of [34]. Each training split will contain 60 categories leaving 20 for testing. Table VII shows the results of the proposed approach compared to the SiamRPN [34] and coAcoE [32] approaches. Under this experimental setting our approach is down 8.5% mAP when compared with the CoAE approach [32].

Finally, the proposed approach is assessed in the context of one-shot logo detection using the QMUL-OpenLogo dataset. Here, 210 categories are arbitrarily selected for training and 125 for testing in the same way a the recognition experiments in section IV-A. Table VIII shows the average precision

Fig. 7: One-shot recognition results for arbitrarily selected pairs of images from the QMUL-OpenLogo and MiniImageNet datasets.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-375468f0-0c43-4336-9942-798b6548455f.jpg)

TABLE V: Comparison of different one-shot detection methods on VOC in AP (%). Note that the SiamFC [35], SiamRPN [34], CompNet [36], CoAE [32], and M-FCOS+SARM [33] approaches use all classes in their ImageNet pre-trained backbones.

| Method   | Seen classes Unseen classes   |      |      |      |             |       |       |        |      |       |       |      |      |      |            |      |      |           |           |      |      |
|----------|-------------------------------|------|------|------|-------------|-------|-------|--------|------|-------|-------|------|------|------|------------|------|------|-----------|-----------|------|------|
|          | plant                         | sofa | tv   | car  | bottle boat |       | chair | person | bus  | train | horse | bike | dog  | bird | mbiketable |      | mAP  | cow       | sheep cat | aero | mAP  |
| SiamFC   | 3.2                           | 22.8 | 5.0  | 16.7 | 0.5         | 8.1   | 1.2   | 4.2    | 22.2 | 22.6  | 27.8  | 35.4 | 14.2 | 25.8 | 11.7       | 19.7 | 15.1 | 6.8 2.28  | 31.6      | 12.4 | 13.3 |
| SiamRPN  | 1.9                           | 15.7 | 4.5  | 12.8 | 1.0         | 1.1   | 6.1   | 8.7    | 7.9  | 6.9   | 5.1   | 17.4 | 17.8 | 20.5 | 7.2        | 18.5 | 9.6  | 15.9 15.7 | 21.7      | 3.5  | 14.2 |
| CompNet  | 28.4                          | 41.5 | 65.0 | 66.4 | 37.1        | 49.8  | 16.2  | 31.7   | 69.7 | 73.1  | 39.8  | 75.6 | 71.6 | 61.4 | 52.3       | 63.4 | 52.7 | 75.3 60.0 | 47.9      | 25.3 | 52.1 |
| CoAE     | 30.0                          | 54.9 | 64.1 | 66.7 | 40.1        | 54.1  | 14.7  | 60.9   | 77.5 | 78.3  | 46.2  | 77.9 | 73.2 | 80.5 | 70.8       | 72.4 | 60.1 | 83.9 67.1 | 75.6      | 46.2 | 68.2 |
| M-FCOS   | 33.7                          | 58.2 | 67.5 | 72.7 | 40.8        | 48.2  | 20.1  | 55.4   | 78.2 | 79.0  | 48.8  | 76.2 | 74.6 | 81.3 | 71.6       | 72.0 | 61.1 | 74.3 68.5 | 81.0      | 52.4 | 69.1 |
| JNN      | 4.5                           | 44.0 | 38.3 | 51.5 | 2.9         | 43.76 | 13.1  | 39.3   | 45.2 | 57.8  | 72.6  | 58.0 | 75.5 | 64.2 | 71.14      | 12.7 | 43.4 | 64.7 51.0 | 65.2      | 43.5 | 56.1 |

TABLE VI: One-shot object detection performance comparison training on the COCO dataset and testing using the Pascal VOC dataset.

TABLE VII: Evaluation on COCO with respect to mAP score (%) for unseen classes.

| Method   |      |   plant sofa |   tv |   car |   bottle |   boat |   chair |   person |   bus | train     |   horse |   bike | dog       |   bird |   mbike table |   cow |   sheep |   cat |   aero |   mAP |
|----------|------|--------------|------|-------|----------|--------|---------|----------|-------|-----------|---------|--------|-----------|--------|---------------|-------|---------|-------|--------|-------|
| M-FCOS   | 10.3 |         79.1 | 54.2 |  64   |     18.4 |   19.6 |    26.3 |      9.6 |  67.5 | 55.4      |    67.7 |   26.8 | 73.7 50.8 |   26.4 |          11.9 |  84.8 |    75.1 |  69.5 |   28.4 |  45.9 |
| JNN      |  9.5 |         69.3 | 49.8 |  60.3 |      7.2 |   29.1 |    10.1 |      6.7 |  60.3 | 57.2 58.5 |    45.3 |   62.6 | 45.6      |   74.8 |          29   |  70.4 |    55.4 |  54.4 |   88.1 |  47.1 |

TABLE VIII: AP detection results for the top 15 Open-Logo dataset classes and the mAP results for the whole dataset.

| split   |    1 |    2 |    3 |    4 |   Average |
|---------|------|------|------|------|-----------|
| SiamRPN | 15.3 | 17.6 | 17.4 | 17   |      16.8 |
| coAcoE  | 23.4 | 23.6 | 20.5 | 20.4 |      22   |
| JNN     | 12.9 | 16.5 | 10   | 14.4 |      13.5 |

(AP50) results for the top performing 15 unseen classes and the mean average precision (mAP) for the all 125 unseen classes used for testing. Figure 8 show some results of the proposed approach in action.

## C. Ablation study

This paper introduces Joint Neural Networks as an adaptable framework that can be used with any defined backbone architecture to perform the one-shot object recognition or detection tasks, where the joint layers are the key concept used build the Joint Neural Networks. This section explores how the use of joint layers on specific locations on the architecture may affect the performance of the proposed approach.

Figure 4 presented in section III-B shows the proposed convolutional and joint layers used to tackle the one-shot object detection task. In order to test the firstly proposed architecture, Table IX shows the performance of the proposed framework under different joint layer configurations. The test

| Class                 | mAP/AP   |
|-----------------------|----------|
| anz text              | 100.00%  |
| rbc                   | 98.86%   |
| blizzardentertainment | 98.34%   |
| costco                | 93.26%   |
| 3m                    | 90.61%   |
| bosch text            | 90.00%   |
| gap                   | 89.47%   |
| lexus                 | 88.92%   |
| generalelectric       | 83.32%   |
| hp                    | 82.81%   |
| levis                 | 79.36%   |
| airhawk               | 79.17%   |
| danone                | 79.02%   |
| armitron              | 77.73%   |
| google                | 77.66%   |
| all                   | 52.84 %  |

are carried out using the first experimental settings described in section IV-B (Table V).

Each Joint Layer ( JL ) in Figure 4 is depicted as a column in Table IX. The experiments explore the performance of the architecture when adding joint layers sequentialy as well as for groups of two and three layers. Results show that for

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-a571806e-92f5-44f4-8617-97a7e5ee2c13.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-05a43ed8-406e-48da-aca9-a02e9ff54f98.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-d5248dab-a95a-4502-a07c-790b8e3195bb.jpg)

RBC

Fig. 8: One-shot detection results results for arbitrarily selected query and target images from the QMUL-OpenLogo dataset.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00701v1-figure-54a109e2-28fd-497f-946d-92bdb25c0fa0.jpg)

TABLE IX: Ablation study for used joint layers in DarkNet19 architecture for detection

| JL1   | JL2   | JL3   | JL4   | JL5   |   mAP |
|-------|-------|-------|-------|-------|-------|
| ×     |       |       |       |       |  53.3 |
| ×     | ×     |       |       |       |  53.6 |
| ×     | ×     | ×     |       |       |  54.9 |
| ×     | ×     | ×     | ×     |       |  55.2 |
| ×     | ×     | ×     | ×     | ×     |  36.8 |
| ×     |       |       | ×     |       |  52.9 |
|       | ×     |       | ×     |       |  53.9 |
|       |       | ×     | ×     |       |  47.9 |
| ×     | ×     |       | ×     |       |  56.1 |
| ×     |       | ×     | ×     |       |  53.7 |

and one-stage multi-box object detection approaches. The proposed joint layers are the fundamental concept supporting the data sharing between the branches of the presented Joint architectures. This approach is developed for one-shot object recognition and detection tasks where traditional deep learning approaches fall short due to the lack of training data available to build state-of-the-art models.

the one-shot object detection task using the VOC dataset the combination of the joint layers 1, 2, and 4 in DarkNet19 architecture yields the best overall mAP performance (56.1%).

## V. CONCLUSION

This paper presents a novel convolutional neural network based framework named Joint neural networks to tackle the challenging tasks of one-shot object recognition and detection. Joint neural networks are inspired by Siamese neural networks

Results indicate that the proposed approach can compete with the top performing state-of-the-art recognition and detection approaches, and even outperform them under the right experimental settings by using the most basic building blocks of convolutional neural networks. The work presented in this paper leaves open room to experiment with new architectures and joint layer configurations to further improve the state-ofthe-art performance for the one-shot object recognition and detection tasks.

## REFERENCES

[1] A. Jain, R. P. W. Duin, Jianchang Mao. 'Statistical pattern recognition: a review'. IEEE Trans. Pattern Analysis and Machine Intelligence , vol. 22, no. 1, pp. 4-37, Jan 2000.

- [2] Z. Zhao, P. Zheng, S. Xu and X. Wu, 'Object Detection With Deep Learning: A Review', IEEE Trans. Neural Netw. Learn. Syst , vol. 30, no. 11, pp. 3212-3232, Nov. 2019.
- [3] A. Antoniou, A. Storkey, H. Edwards. 'Data augmentation generative adversarial networks', 2017. [Online] Available: https://arxiv.org/abs/ 1711.04340.
- [4] A. Nguyen, J. Yosinski, J, Clune. 'Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images', December 2014. [Online] Available: https://arxiv.org/abs/1412.1897.
- [5] Redmon, J., Divvala, S., Girshick, R., Farhadi, A. 'You only look once: Unified, real-time object detection', In Proc. CVPR , Las Vegas, NV, USA 2016.
- [6] J. Redmon and A. Farhadi, 'Yolo9000: better, faster, stronger', In Proc. CVPR , 2017, pp. 6517-6525.
- [7] Lowe, David G. 'Distinctive Image Features from Scale-Invariant Keypoints', IJCV , 60(2):91-110, November 2004.
- [8] Bay, Herbert, Andreas Ess, Tinne Tuytelaars, and Luc Van Gool. 'Speeded-Up Robust Features (SURF)', Computer Vision and Image Understanding , Volume 110, Issue 3, June 2008.
- [9] G. Koch, R. Zemel, R. Salakhutdinov. 'Siamese neural networks for one-shot image recognition'. ICML Deep Learning workshop , 2015.
- [10] L. Fe-Fei, R. Fergus, P. Perona. 'A Bayesian approach to unsupervised one-shot learning of object categories'. In Proc. ICCV , vol. 2, pp. 11341141, 2003.
- [11] L. Fei-Fei, R. Fergus, P. Perona. 'One-shot learning of object categories'. IEEE Trans. Pattern Analysis and Machine Intelligence , vol. 28, no.4, pp. 594-611, April 2006.
- [12] Nikolenko, S. I. 'Synthetic data for deep learning', 2019. [Online] Available: https://arxiv.org/abs/1909.11512.
- [13] N. Tajbakhsh. 'Convolutional neural networks for medical image analysis: Full training or fine tuning?', IEEE Trans. Med. Imag , vol. 35, no. 5, pp. 1299-1312, May 2016.
- [14] L. Shao, F. Zhu and X. Li, 'Transfer Learning for Visual Categorization: A Survey', IEEE Trans. Neural Netw. Learn. Syst , vol. 26, no. 5, pp. 1019-1034, May 2015.
- [15] R. Girshick, J. Donahue, T. Darrell, and J. Malik. 'Rich feature hierarchies for accurate object detection and semantic segmentation'. Proc. CVPR , 2014.
- [16] M. D. Zeiler and R. Fergus. 'Visualizing and understanding convolutional networks'. In Proc. CVPR , 2014.
- [17] G. E. Hinton and R. R. Salakhutdinov. 'Reducing the dimensionality of data with neural networks'. Science, 313(5786):504-507, 2006.
- [18] Y. Wang, D. Ramanan, and M. Hebert, 'Growing a brain: Fine-tuning by increasing model capacity', in IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit , Jul. 2017, pp. 3029-3038.
- [19] C. Finn, P. Abbeel, and S. Levine. 'Model-agnostic metalearning for fast adaptation of deep networks'. In International Conference on Machine Learning , 2017
- [20] J. Snell, K. Swersky, and R. S. Zemel. 'Prototypical networks for fewshot learning'. In Neural Information Processing Systems , 2017.
- [21] Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 'A simple neural attentive meta-learner'. In International Conference on Learning Representations , 2018.
- [22] B. N. Oreshkin, P. Rodriguez, and A. Lacoste. 'TADAM: Task dependent adaptive metric for improved few-shot learning', May 2018. [Online] Available: https://arxiv.org/abs/1805.10123
- [23] Sun, Q., Liu, Y., Chua, T.S., Schiele, B. 'Meta-transfer learning for few-shot learning'. In Proc. CVPR , 2019.
- [24] H. -G. Jung and S. -W. Lee, 'Few-Shot Learning With Geometric Constraints', IEEE Trans. Neural Netw. Learn. Syst , vol. 31, no. 11, pp. 4660-4672, Nov. 2020.
- [25] E. Ahmed, M. Jones, T. K. Marks. 'An improved deep learning architecture for person re-identification'. In Proc. CVPR , 2015.
- [26] Y. Taigman, M. Yang, M. Ranzato, L. Wolf: Deepface. 'Closing the gap to human-level performance in face verification'. In Proc. CVPR , 2014.
- [27] Z. Zhang, H. Peng, and Q. Wang. 'Deeper and wider siamese networks for real-time visual tracking'. In Proc. CVPR , 2019.
- [28] L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr. 'Fully-convolutional siamese networks for object tracking'. In Proc. ECCV workshop , 2016.
- [29] Bromley, J., Bentz, J. W., Bottou, L., Guyon, I., LeCun, Y., Moore, C., et al. 'Signature verification using a Siamese time delay neural network'. Proc. Advances in Neural Information Processing Systems , 6, pp. 737744, 1994.
- [30] S. Chopra, R. Hadsell, and Y. LeCun. 'Learning a similiarty metric discriminatively, with application to face verification'. In Proc. CVPR , 2005.
- [31] M. Harandi, S. R. Kumar, and R. Nock. 'Siamese Networks: A Thing or Two to Know'. 2017. [Online] Available: https://pdfs.semanticscholar. org/c03c/e09b419b6a15c1228e344a900d8c54bddc78.pdf
- [32] Hsieh, T.I., Lo, Y.C., Chen, H.T., Liu, T.L. 'One-Shot Object Detection with Co-Attention and Co-Excitation''. In Proc. NeurIPS , 2019.
- [33] Xiang Li, Lin Zhang, Yau Pun Chen, Yu-Wing Tai, Chi-Keung Tang. 'One-Shot Object Detection without Fine-Tuning', 2020. [Online] Available: https://arxiv.org/abs/2005.03819.
- [34] Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu. 'High performance visual tracking with siamese region proposal network'. In Proc. CVPR , 2018, Salt Lake City, UT, USA, June 18-22, 2018.
- [35] Miaobin Cen and Cheolkon Jung. 'Fully convolutional siamese fusion networks for object tracking'. In Proc. ICIP 2018, Athens, Greece, October 7-10, 2018, pages 3718-3722.
- [36] Tengfei Zhang, Yue Zhang, Xian Sun, Hao Sun, Menglong Yan, Xue Yang, and Kun Fu. 'Comparison network for one-shot conditional object detection', 2019. [Online] Available: https://arxiv.org/abs/1904.02317.
- [37] Hang Su, Xiatian Zhu, Shaogang Gong. 'Open Logo Detection Challenge'. In Proc. BMVC , 2018.
- [38] Y. Kalantidis, LG. Pueyo, M. Trevisiol, R. van Zwol, Y. Avrithis. 'Scalable Triangulation-based Logo Recognition'. In Proc. ICMR , Trento, Italy, April 2011.
- [39] S. Romberg, L. G. Pueyo, R. Lienhart, and R. Van Zwol. 'Scalable logo recognition in real-world images'. in Proc. ICMR , 2011.
- [40] Simone Bianco, Marco Buzzelli, Davide Mazzini, and Raimondo Schettini. 'Deep learning for logo recognition'. Neurocomputing , 245:23-30, 2017.
- [41] Alexis Joly and Olivier Buisson. 'Logo retrieval with a contrario visual query expansion'. in Proc. ACM Multimedia , Beijing, China, 2009, pp. 581-584.
- [42] Hang Su, Shaogang Gong, and Xiatian Zhu. 'Weblogo-2m: Scalable logo detection by deep learning from the web'. In Proc. ICCV Workshop , 2017.
- [43] Andras T¨ uzk¨, Christian Herrmann, Daniel Manger, and J¨ urgen Beyerer. o 'Open set logo detection and retrieval', 2017. [Online] Available: https: //arxiv.org/abs/1710.10891.
- [44] Yuan Liao, Xiaoqing Lu, Chengcui Zhang, Yongtao Wang, and Zhi Tang. 'Mutual enhancement for detection of multiple logos in sports videos'. In Proc. ICCV , 2017.
- [45] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 'ImageNet: A Large-Scale Hierarchical Image Database'. In Proc. CVPR , 2009.
- [46] Vinyals, Oriol, Blundell, Charles, Lillicrap, Timothy, Kavukcuoglu, Koray, and Wierstra, Daan. 'Matching networks for one shot learning', 2016. [Online] Available: https://arxiv.org/abs/1606.04080
- [47] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. 'The Pascal Visual Object Classes (VOC) Challenge'. IJCV , pages 303-338, 2010.
- [48] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. 'Microsoft COCO: Common objects in context'. In Proc. ECCV . 2014.
- [49] Xiang Li, Lin Zhang, Yau Pun Chen, Yu-Wing Tai, Chi-Keung Tang. 'One-Shot Object Detection without Fine-Tuning', 2020. [Online] Available: https://arxiv.org/abs/2005.03819
Camilo J. Vargas Qianni Zhang Ebroul Izquierdo
Joint Neural Networks for One-shot Object Recognition and Detection
This paper presents a novel joint neural networks approach to address the
challenging one-shot object recognition and detection tasks. Inspired by
Siamese neural networks and state-of-art multi-box detection approaches, the
joint neural networks are able to perform object recognition and detection for
categories that remain unseen during the training process. Following the
one-shot object recognition/detection constraints, the training and testing
datasets do not contain overlapped classes, in other words, all the test
classes remain unseen during training. The joint networks architecture is able
to effectively compare pairs of images via stacked convolutional layers of the
query and target inputs, recognising patterns of the same input query category
without relying on previous training around this category. The proposed
approach achieves 61.41% accuracy for one-shot object recognition on the
MiniImageNet dataset and 47.1% mAP for one-shot object detection when trained
on the COCO dataset and tested using the Pascal VOC dataset. Code available at
https://github.com/cjvargasc/JNN recog and https://github.com/cjvargasc/JNN
detection/