extfieldsofstudy:
Computer Science
Engineering

text:
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation

This paper studies the problem of learning semantic segmentation from image-level supervision only. Current popular solutions leverage object localization maps from classifiers as supervision signals, and struggle to make the localization maps capture more complete object content. Rather than previous efforts that primarily focus on intra-image information, we address the value of cross-image semantic relations for comprehensive object pattern mining. To achieve this, two neural co-attentions are incorporated into the classifier to complimentarily capture cross-image semantic similarities and differences. In particular, given a pair of training images, one co-attention enforces the classifier to recognize the common semantics from co-attentive objects, while the other one, called contrastive co-attention, drives the classifier to identify the unshared semantics from the rest, uncommon objects. This helps the classifier discover more object patterns and better ground semantics in image regions. In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference, hence eventually benefiting semantic segmentation learning. More essentially, our algorithm provides a unified framework that handles well different WSSS settings, i.e., learning WSSS with (1) precise image-level supervision only, (2) extra simple single-label data, and (3) extra noisy web data. It sets new state-of-the-arts on all these settings, demonstrating well its efficacy and generalizability. Moreover, our approach ranked 1st place in the Weakly-Supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data Challenge.

Introduction
Recently, modern deep learning based semantic segmentation models [5,6], trained with massive manually labeled data, achieve far better performance than before. However, the fully supervised learning paradigm has the main limitation of requiring intensive manual labeling effort, which is particularly expensive for annotating pixel-wise ground-truth for semantic segmentation. Numerous efforts are motivated to develop semantic segmentation with weaker forms of supervision, such as bounding boxes [45], scribbles [36], points [3], and image-level labels [46], etc. Among them, a prominent and appealing trend is using only image-level labels to achieve weakly supervised semantic segmentation (WSSS), which demands the least annotation efforts and is followed in this work.
To tackle the task of WSSS with only image-level labels, current popular methods are based on network visualization techniques [80,86], which discover discriminative regions that are activated for classification. These methods use image-level labels to train a classifier network, from which class-activation maps are derived as pseudo ground-truths for further supervising pixel-level semantics learning. However, it is commonly evidenced that the trained classifier tends to over-address the most discriminative parts rather than entire objects, which becomes the focus of this area. Diverse solutions are explored, typically adopting: image-level operations, such as region hiding and erasing [30,69], regions growing strategies that expand the initial activated regions [27,62], and feature-level enhancements that collect multi-scale context from deep features [33,71].
These efforts generally achieve promising results, which demonstrates the importance of discriminative object pattern mining for WSSS. However, as shown in Fig. 1(a), they typically use only single-image information for object pattern discovering, ignoring the rich semantic context among the weakly annotated data. For example, with the image-level labels, not only the semantics of each individual image can be identified, the cross-image semantic relations, i.e., two images whether sharing certain semantics, are also given and should be used as cues for object pattern mining. Inspired by this, rather than relying on intraimage information only, we further address the value of cross-image semantic correlations for complete object pattern learning and effective class-activation map inference (see Fig. 1(b-c)). In particular, our classifier is equipped with a differentiable co-attention mechanism that addresses semantic homogeneity and difference understanding across training image pairs. More specifically, two kinds of co-attentions are learned in the classifier. The former one aims to capture cross-image common semantics, which enables the classifier to better ground the common semantic labels over the co-attentive regions. The latter one, called contrastive co-attention, focuses on the rest, unshared semantics, which helps the classifier better separate semantic patterns of different objects. These two co-attentions work in a cooperative and complimentary manner, together making the classifier understand object patterns more comprehensively.
In addition to benefiting object pattern learning, our co-attention provides an efficient tool for precise localization map inference (see Fig. 1(c)). Given a training image, a set of related images (i.e., sharing certain common semantics) are utilized by the co-attention for capture richer context and generate more accurate localization maps. Another advantage is that our co-attention based classifier learning paradigm brings an efficient data augmentation strategy, due to the use of training image pairs. Overall, our co-attention boosts object discovering during both the classifier's training phase as well as localization map inference stage. This provides the possibility of obtaining more accurate pseudo pixel-level annotations, which facilitate final semantic segmentation learning.
Our algorithm is a unified and elegant framework, which generalizes well different WSSS settings. Recently, to overcome the inherent limitation in WSSS without additional human supervision, some efforts resort to extra image-level supervision from simple single-class data readily available from other existing datasets [35,48], or cheap web-crawled data [18,53,54,70]. Although they improve the performance to some extent, complicated techniques, such as energy function optimization [18,58], heuristic constraints [54], and curriculum learning [70], are needed to handle the challenges of domain gap and data noise, restricting their utility. However, due to the use of paired image data for classifier training and object map inference, our method has good tolerance to noise. In addition, our method also handles domain gap naturally, as the co-attention effectively addresses domain-shared object pattern learning and achieves domain adaption as a part of co-attention parameter learning. We conduct extensive experiments on PASCAL VOC 2012 [10], under three WSSS settings, i.e., learning WSSS with (1) PASCAL VOC image-level supervision only, (2) extra simple singlelabel data, and (3) extra web data. Our algorithm sets state-of-the-art on each case, verifying its effectiveness and generalizability. Our method also ranked 1 st place in the Weakly-supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data (LID) Challenge [72] (LID 20 ), outperforming other competitors by large margins.
Our contributions are three-fold. (1) We address the value of cross-image semantic correlations for complete object pattern learning as well as object location inference, which is achieved by a co-attention classifier that works over paired training samples. (2) Our co-attention classifier mines semantic cues in a more comprehensive manner. In addition to single-image semantics, it mines complimentary supervision from cross-image semantic similarities and differences by the co-attention and contrastive co-attention, respectively. (3) Our approach is general enough to learn WSSS with precise image-level supervision, or with extra simple single-label, or even noisy web-crawled data. It solves inherent challenges of different WSSS settings elegantly, and shows promising results consistently.

Related Work
Weakly Supervised Semantic Segmentation. Recently, lots of WSSS methods have been proposed to alleviate labeling cost. Various weak supervision forms have been explored, such as bounding boxes [9,45], scribbles [36], point supervision [3], etc. Among them, image-level supervision, due to its less annotation demand, gains most attention and is also adopted in our approach.
Current popular solutions for WSSS with image-level supervision rely on network visualization techniques [80,86], especially the Class Activation Map (CAM) [86], which discovers image pixels that are informative for classification. However, CAM typically only identifies small discriminative parts of objects, making it not an ideal proxy ground-truth for semantic segmentation training. Therefore, numerous efforts are made towards expanding the CAM-highlighted regions to the whole objects. In particular, some representative approaches make use of image-level hiding and erasing operations to drive a classifier to focus on different parts of objects [30,34,69]. A few ones instead resort to a regions growing strategy, i.e., view the CAM-activated regions as initial "seeds" and gradually grow the seed regions until cover the complete objects [2,22,27,62]. Meanwhile, some researchers investigate to directly enhance the activated regions on feature-level [31,33,71]. When constructing CAMs, they collect multi-scale context, which is achieved by dilated convolution [71], multi-layer feature fusion [33], saliency-guided iterative training [62], or stochastic feature selection [31]. Some others accumulate CAMs from multiple training phases [23], or self-train a difference detection network to complete the CAMs with trustable information [55]. In addition, a recent trend is to utilize class-agnostic saliency cues to filter out background responses [11,22,31,34,62,69,71] during localization map inference.
Since the supervision provided in above problem setting is so weak, another category of approaches explores to leverage more image-level supervision from other sources. There are mainly two types: (1) exploring simple and single-label examples [35,48] (e.g., images from existing datasets [15,51]); or (2) utilizing near-infinite yet noisy web-sourced image [18,53,54,70] or video [18,32,58] data (also referred as webly supervised semantic segmentation [24]). In addition to the common challenge of domain gap between the extra data and target semantic segmentation dataset, the second-type methods need to handle data noise.
Past efforts only consider each image individually, while only few exceptions [11,53] address cross-image information. [53] simply applies off-the-shelf co-segmentation [25] over the web images to generate foreground priors, instead of ours encoding the semantic relations into network learning and inference. For [11], although also exploiting correlations within image pairs, the core idea is to use extra information from a support image to supplement current visual representations. Thus the two images are expected to better contain the same semantics, and unmatched semantics would bring negative influences. In contrast, we view both semantic homogeneity and difference as informative cues, driving our classifier to more explicitly identify the common as well as unshared objects, respectively. Moreover, [11] only utilizes single image to infer the activated objects, but our method comprehensively leverages the cross-image semantics in both classifier training and localization map inference stages. More essentially, our framework is neat and flexible, which is not only able to learn WSSS from clean image-level supervision, but general enough to naturally make use of extra noisy web-crawled or simple single-label data, contrarily to previous efforts which are limited to specific training settings and largely dependent on complicated optimization methods [18,58] or heuristic constraints [54]. Deterministic Neural Attention. Differentiable attention mechanisms enable a neural network to focus more on relevant elements of the input than on irrelevant parts. With their popularity in the field of natural language processing [7,37,41,47,59], attention modeling is rapidly adopted in various computer vision tasks, such as image recognition [12,21,57,64,73], domain adaptation [65,84], human pose estimation [8,61,78], reasoning among objects [52,87], and image generation [77,82,88]. Further, co-attention mechanisms become an essential tool in many vision-language applications and sequential modeling tasks, such as visual question answering [39,42,76,79], visual dialog [74,85], vision-language navigation [66], and video segmentation [40,60], showing its effectiveness in capturing the underlying relations between different entities. Inspired by the general idea of attention mechanisms, this work leverages co-attention to mine semantic relations within training image pairs, which helps the classifier network learn complete object patterns and generate precise object localization maps.

Methodology
Problem Setup. Here we follow current popular WSSS pipelines: given a set of training images with image-level labels, a classification network is first trained to discover corresponding discriminative object regions. The resulting object localization maps over the training samples are refined as pseudo ground-truth masks to further supervise the learning of a semantic segmentation network. Our Idea. Unlike most previous efforts that treat each training image individually, we explore cross-image semantic relations as class-level context for understanding object patterns more comprehensively. To achieve this, two neural co-attentions are designed. The first one drives the classifier to learn common semantics from the co-attentive object regions, while the other one enforces the classifier to focus on the rest objects for unshared semantics classification.

Co-attention Classification Network
Let us denote the training data as I = {(I n , l n )} n , where I n is the n th training image, and l n ∈{0, 1} K is the associated ground-truth image label for K semantic categories. As shown in Fig. 2(a), image pairs, i.e., (I m , I n ), are sampled from I for training the classifier. After feeding I m and I n into the convolutional embedding part of the classifier, corresponding feature maps, F m ∈ R C×H×W and F n ∈ R C×H×W , are obtained, each with H×W spatial dimension and C channels.
As in [23,31,32], we can first separately pass F m and F n to a class-aware fully convolutional layer ϕ(·) to generate class-aware activation maps, i.e., S m = ϕ(F m ) ∈ R K×H×W and S n = ϕ(F n ) ∈ R K×H×W , respectively. Then, we apply global average pooling (GAP) over S m and S n to obtain class score vectors s m ∈ R K  and s n ∈ R K for I m and I n , respectively. Finally, the sigmoid cross entropy (CE) loss is used for supervision: So far the classifier is learned in a standard manner, i.e., only individual-image information is used for semantic learning. One can directly use the activation maps to supervise next-stage semantic segmentation learning, as done in [22,32]. Differently, our classifier additionally utilizes a co-attention mechanism for further mining cross-image semantics and eventually better localizing objects. Co-Attention for Cross-Image Common Semantics Mining. Our coattention attends to the two images, i.e., I m and I n , simultaneously, and captures their correlations. We first compute the affinity matrix P between F m and F n : where F m ∈ R C×HW and F n ∈ R C×HW are flattened into matrix formats, and W P ∈ R C×C is a learnable matrix. The affinity matrix P stores similarity scores corresponding to all pairs of positions in F m and F n , i.e., the (i, j) th element of P gives the similarity between i th location in F m and j th location in F n . Then P is normalized column-wise to derive attention maps across F m for each position in F n , and row-wise to derive attention maps across F n for each position in F m : where softmax is performed column-wise. In this way, A n and A m store the co-attention maps in their columns. Next, we can compute attention summaries where F m∩n m and F m∩n n are reshaped into R C×W×H . Co-attentive feature F m∩n m , derived from F n , preserves the common semantics between F m and F n and locate the common objects in F m . Thus we can expect only the common semantics l m ∩ l n can be safely derived from F m∩n m , and the same goes for F m∩n n . Such coattention based common semantic classification can let the classifier understand the object patterns more completely and precisely.
To make things intuitive, consider the example in Fig. 2, where I m contains Table and Person, and I n has Cow and Person. As the co-attention is essentially the affinity computation between all the position pairs between I m and I n , only the semantics of the common objects, Person, will be preserved in the co-attentive features, i.e., F m∩n m and F m∩n n (see Fig. 2(b)). If we feed F m∩n m and F m∩n n into the class-aware fully convolutional layer ϕ, the generated class-aware activation maps, i.e., S m∩n m =ϕ(F m∩n m )∈R K×H×W and S m∩n n =ϕ(F m∩n n )∈R K×H×W , are able to locate the common object Person in I m and I n , respectively. After GAP, the predicted semantic classes (scores) s m∩n m ∈ R K and s m∩n n ∈ R K should be the common semantic labels l m ∩ l n of I m and I n , i.e., Person.
Through co-attention computation, not only the human face, the most discriminative part of Person, but also other parts, such as legs and arms, are highlighted in F m∩n m and F m∩n n (see Fig. 2(b)). When we set the common class labels, i.e., Person, as the supervision signal, the classifier would realize that the semantics preserved in F m∩n m and F m∩n n are related and can be used to recognize Person. Therefore, the co-attention, computed across two related images, explicitly helps the classifier associate semantic labels and corresponding object regions and better understand the relations between different object parts. It essentially makes full use of the context across training data.
Intuitively, for the co-attention based common semantic classification, the labels l m ∩ l n shared between I m and I n are used to supervise learning: Contrastive Co-Attention for Cross-Image Exclusive Semantics Mining. Aside from the co-attention described above that explores cross-image common semantics, we propose a contrastive co-attention that mines semantic differences between paired images. The co-attention and contrastive co-attention complementarily help the classifier better understand the concept of the objects. As shown in Fig. 2(a), for I m and I n , we first derive class-agnostic coattentions from their co-attentive features, i.e., The set operation '∩' is slightly extended here to represent bitwise-and.
where σ(·) is the sigmoid activation function, and the parameter matrix W B ∈ R 1×C learns for common semantics collection and is implemented by a convolutional layer with 1×1 kernel. B m∩n m and B m∩n n are class-agnostic and highlight all the common object regions in I m and I n , respectively, based on which we derive contrastive co-attentions: The contrastive co-attention A m\n m of I m , as its superscript suggests, addresses those unshared object regions that are only of I m , but not of I n , and the same goes for A n\m n . Then we get contrastive co-attentive features, i.e., unshared semantics in each images: Compared with the co-attention that investigates common semantics as informative cues for boosting object patterns mining, the contrastive co-attention addresses complementary knowledge from the semantic differences between paired images. Fig. 2(b) gives an intuitive example. After computing the contrastive co-attentions between I m and I n (Eq. 7), , the classifier is required to accurately recognize Table and Cow classes, respectively. When the common objects are filtered out by the contrastive co-attentions, the classifier has a chance to focus more on the rest image regions and mine the unshared semantics more consciously. This also helps the classifier better discriminate the semantics of different objects, as the semantics of common objects and unshared ones are disentangled by the contrastive co-attention. For example, if some parts of Cow are wrongly recognized as Person-related, the contrastive co-attention will discard these parts in F n\m n . However, the rest semantics in F n\m n may be not sufficient enough for recognizing Cow. This will enforce the classifier to better discriminate different objects.
For the contrastive co-attention based unshared semantic classification, the supervision loss is designed as: The set operation '\' is slightly extend here, i.e., ln\lm = ln − ln∩lm.
More In-Depth Discussion. One can interpret our co-attention classifier from a view of auxiliary-task learning [14,43], which is investigated in self-supervised learning field to improve data efficiency and robustness, by exploring auxiliary tasks from inherent data structures. In our case, rather than the task of singleimage semantic recognition which has been extensively studied in conventional WSSS methods, we explore two auxiliary tasks, i.e., predicting the common and uncommon semantics from image pairs, for fully mining supervision signals from weak supervision. The classifier is driven to better understand the cross-image semantics by attending to (contrastive) co-attentive features, instead of only relying on intra-image information (see Fig. 2(c)). In addition, such strategy shares a spirit of image co-segmentation. Since the image-level semantics of training set are given, the knowledge about some images share or unshare certain semantics should be used as a cue, or supervision signal, to better locate corresponding objects. Our co-attention based learning pipeline also provides an efficient data augmentation strategy, due to the use of paired samples, whose amount is near the square of the number of single training images.

Co-Attention Classifier Guided WSSS Learning
Training Co-Attention Classifier. The overall training loss for our co-attention classifier ensembles the three terms defined in Eq. 1, 5, and 9: The coefficients of different loss terms are set as 1 in our all experiments. During training, to fully leverage the co-attention to mine the common semantics, we sample two images (I m , I n ) with at least one common class, i.e., l m ∩l n = 0. Generating Object Localization Maps. Once our image classifier is trained, we apply it over the training data I = {(I n , l n )} n to produce corresponding object localization maps, which are essential for semantic segmentation network training. We explore two different strategies to generate localization maps.
-Single-round feed-forward prediction, made over each training image individually. For each training image I n , running the classifier and directly using its class-aware activation map (i.e., S n ∈ R K×H×W ) as the object localization map L n , as most previous network visualization based methods [23,32,54] done. -Multi-round co-attentive prediction with extra reference information, which is achieved by considering extra information from other related training images (see Fig. 1(c)). Specifically, given a training image I n and its associated label vector l n , we generate its localization map L n in a class-wise manner. For each semantic class k ∈ {1, · · · , K} labeled for I n , i.e., l n,k = 1 and l n,k is the k th element of l n , we sample a set of related images R = {I r } r from I, which are also annotated with label k, i.e., l r,k = 1. Then we compute the co-attentive feature F m∩r n from each related image I r ∈ R to I n , and get the co-attention based class-aware activation map S m∩r n . Given all the class-aware activation maps {S m∩r n } r from R, they are integrated to infer the localization map only for class k, i.e., L n,k = 1 |R| r∈R S m∩r n,k . Here L n,k ∈ R H×W and S (·) n,k ∈ R H×W indicate the feature map at k th channel of L n ∈ R K×H×W and S (·) n ∈ R K×H×W , respectively. '|·|' numerates the elements. After inferring the localization maps for all the annotated semantic classes of I n , we can get L n .
These two localization map generation strategies are studied in our experiments ( §4.5), and the last one is more favored, as it uses both intra-and interimage semantics for object inference, and shares a similar data distribution of the training phase. One may notice that the contrastive co-attention is not used here. This is because contrastive co-attentive feature (Eq. 8) is from its original image, which is effective for boosting feature representation learning during classifier training, while contributes little for localization maps inference (with limited cross-image information). Related experiments can be found at §4.5. Learning Semantic Segmentation Network. After obtaining high-quality localization maps, we generate pseudo pixel-wise labels for all the training samples I, which can be used to train arbitrary semantic segmentation network. For pseudo groundtruth generation, we follow current popular pipeline [20,22,23,31,32,81], that uses localization maps to extract class-specific object cues and adopts saliency maps [19,38] to get background cues. For the semantic segmentation network, as in [20,23,31,32], we choose DeepLab-LargeFOV [5]. Learning with Extra Simple Single-Label Images. Some recent efforts [35,48] are made towards exploring extra simple single-label images from other existing datasets [15,51] for further boosting WSSS. Though impressive, specific network designs are desired, due to the issue of domain gap between additionally used data and the target complex multi-label dataset, i.e., PASCAL VOC 2012 [10]. Interestingly, our co-attention based WSSS algorithm provides an alternate that addresses the challenge of domain gap naturally. Here we revisit the computation of co-attention in Eq. 2. When I m and I n are from different domains, the parameter matrix W P , in essence, learns to map them into a unified common semantic space [44] and the co-attentive features can capture domainshared semantics. Therefore, for such setting, we learn three different parameter matrixes for W P , for the cases where I m and I n are from (1) the target semantic segmentation domain, (2) the one-label image domain, and (3) two different domains, respectively. Thus the domain adaption is efficiently achieved as a part of co-attention learning. We conduct related experiments in §4.2. Learning with Extra Web Images. Another trend of methods [18,24,54,70] address webly supervised semantic segmentation, i.e., leveraging web images as extra training samples. Though cheaper, web data are typically noisy. To handle this, previous arts propose diverse effective yet sophisticated solutions, such as multi-stage training [24] and self-paced learning [70]. Our co-attention based WSSS algorithm can be easily extended to this setting and solve data noise elegantly. As our co-attention classifier is trained with paired images, instead of previous methods only relying on each image individually, our model provides a more robust training paradigm. In addition, during localization map inference, a set of extra related images are considered, which provides more comprehensive and accurate cues, and further improves the robustness. We experimentally demonstrate the effectiveness of our method in such a setting in §4.3.

Detailed Network Architecture
Network Configuration. In line with conventions [23,71,83], our image classifier is based on ImageNet [29] pre-trained VGG-16 [56]. For VGG-16 network, the last three fully-connected layers are replaced with three convolutional layers with 512 channels and kernel size 3×3, as done in [23,83]. For the semantic segmentation network, for fair comparison with current top-leading methods [2,23,31,55], we adopt the ResNet-101 [17] version Deeplab-LargeFOV architecture. Training Phases of the Co-Attention Classifier and Semantic Segmentation Network. Our co-attention classifier is fully end-to-end trained by minimizing the loss defined in Eq. 10. The training parameters are set as: initial learning rate (0.001) which is reduced by 0.1 after every 6 epochs, batch size (5), weight decay (0.0002), and momentum (0.9). Once the classifier is trained, we generate localization maps and pseudo segmentation masks over all the training samples (see §3.2). Then, with the masks, the semantic segmentation network is trained in a standard way [23] using the hyper-parameter setting in [5]. Inference Phase of the Semantic Segmentation Network. Given an unseen test image, our segmentation network works in the standard semantic segmentation pipeline [5], i.e., directly generating segments without using any other images. Then CRF [28] post-processing is performed to refine predicted masks.
Note that above settings are used in traditional WSSS datasets (i.e., §4.1, §4.2, §4.3). Due to the specific task setup in LID 20 [72], corresponding training and testing settings will be detailed in §4.4.

Experiment
Overview. Experiments are first conducted over three different WSSS settings: (1) The most standard paradigm [22,23,55,69] that only allows image-level supervision from PASCAL VOC 2012 [10] (see §4.1). (2) Following [35,48], additional single-label images can be used, yet bringing the challenge of domain gap (see §4.2). (3) Webly supervised semantic segmentation paradigm [24,32,54], where extra web data can be accessed (see §4.3). Then, in §4.4, we show the results in WSSS track of LID 20 , where our method achieves the champion. Finally, in §4.5, ablation studies are made to assess the effectiveness of essential parts of our algorithm. Evaluation Metric. In our experiments, the standard intersection over union (IoU) criterion is reported on the val and test sets of PASCAL VOC 2012 [10]. The scores on test set are obtained from official PASCAL VOC evaluation server.

Experiment 1: Learn WSSS only from PASCAL VOC [10] Data
Experimental Setup: We first conduct experiment following the most standard setting that learns WSSS with only image-level labels [22,23,55,69], i.e., only image-level supervision from PASCAL VOC 2012 [10] is accessible. PASCAL VOC 2012 contains a total of 20 object categories. As in [5,69], augmented training data from [16] are also used. Finally, our model is trained on totally 10,582 samples with only image-level annotations. Evaluations are conducted on the val and test sets, which have 1,449 and 1,456 images, respectively. Experimental Results: Table 1a compares our approach and current topleading WSSS methods (highest mIoU is used for comparison) with image-level supervision, on both PASCAL VOC12 val and test sets. Additionally, we show some segmentation results in Fig. 3. We can observe that our method achieves mIoU scores of 66.2 and 66.9 on val and test sets respectively, outperforming all the competitors. The performance of our method is 87% of the DeepLab-LargeFOV [5] trained with fully annotated data, which achieved an mIoU of 76.3 on val set. When compared to OAA+ [23], current best-performing method, our approach obtains the improvement of 1.0% on val set. This well demonstrates that the localization maps produced by our co-attention classifier effectively detect more complete semantic regions towards the whole target objects. Note that our network is elegantly trained end-to-end in a single phase. In contrast, many other recent approaches including OAA+ [23] and SSDD [55], use extra networks [2,23,55] to learn auxiliary information (e.g., integral attention [23], pixel-wise semantic affinity [55], etc.), or adopt multi-step training [1,69,71].

Experiment 2: Learn WSSS with Extra Simple Single-Label Data
Experimental Setup: Following [35,48], we train our co-attention classifier and segmentation network with PASCAL images and extra single-label images. The extra single-label images are borrowed from the subsets of Caltech-256 [15] and ImageNet CLS-LOC [51], and whose annotations are within 20 VOC object categories. There are a total of 20,057 extra single-label images.  [2], OAA+ [23], and our method.  Table 1b. Our method significantly improves the most recent method (i.e., AttnBN [35]) in this setting by 5.0% and 4.2% in val and test sets, respectively. With the fact that objects of the same category but from different domains share similar visual patterns [35], our co-attention provides an end-to-end strategy that efficiently captures the common, cross-domain semantics, and learns domain adaption naturally. Even AttnBN is specifically designed for addressing such setting by knowledge transfer, our method still suppresses it by a large margin. Compared with the setting in §4.1 where only PASCAL images are used for training, our method obtains improvements on both val and test sets, verifying that it successfully mines knowledge from extra simple single-label data and copes with domain gap well.

Experiment 3: Learn WSSS with Extra Web-Sourced Data
Experimental Setup: We also conduct experiments using both PASCAL VOC images and webly craweled images as training data. We use the web data provided by [54], which are retrieved from Bing based on class names. The final dataset contains 76,683 images across 20 PASCAL VOC classes.
Experimental Results: Table 1c shows the performance comparisons between our method and the previous webly supervised segmentation methods. It shows that our method outperforms all other approaches and sets new state-of-thearts with mIoU score of 67.7 and 67.5 on PASCAL VOC 2012 val and test sets, respectively. Among the compared methods, Hong et al. [18] utilize richer information of the temporal dynamics provided by additional large-scale videos.
In contrast, although only using static image data, our method still outperforms it on the val and test sets by 9.6% and 8.8%, respectively. Compared with Shen et al. [54] which uses the same web data as ours, our method substantially improves it by a clear margin of 3.6% on the test set.  (8), weight decay (0.0005), and max epoch (15). During training, the equivariant attention [67] is also adopted. Once our image classifier is trained, we run the classifier and directly use its class-aware activation map (i.e., S n ) as the object localization map L n .

Ablation Studies
Inference Strategies. Table 2 shows mIoU scores on PASCAL VOC 2012 val set w.r.t. different inference modes (see §3.2). When using the traditional inference mode "single-round feed-forward", our method substantially suppresses Full Model co-attention Lbasic (Eq. 1)+Lco-att (Eq. 5)+L co-att (Eq. 9) 66.2 +contrastive co-attention = L (Eq. 10) basic classifier, by improving mIoU score from 61.7 to 64.7. This evidences that co-attention mechanism (trained in an end-to-end manner) in our classifier improves the underlying feature representations and more object regions are identified by the network. We can observe that by using more images to generate localization maps, our method obtains consistent improvement from "Test image only" (64.7), to "Test images and other related images" (66.2). This is because more semantic context are exploited during localization map inference. In addition, using contrastive co-attention for localization map inference doesn't boost performance (66.2). This is because the contrastive co-attentive features for one image are derived from the image itself. In contrast, co-attentive features are from the other related image, thus can be effective in the inference stage.
(Contrastive) Co-Attention. As seen in Table 4, by only using co-attention (Eq. 5), we already largely suppress the basic classifier (Eq. 1) by 3.8%. When adding additional contrastive co-attention (Eq. 9), we obtain mIoU improvement of 0.7%. Above analysis verify our two co-attentions indeed boost performance.  Table 5, it is easily observed that when increasing the number of related images from 0 to 3, the performance gets boosted consistently. However, when further using more images, the performance degrades. This can be attributed to the trade-off between useful semantic information and noise brought by related images. From 0 to 3 reference images, more semantic information is used and more integral regions for objects are mined. When further using more related images, useful information reaches its bottleneck and noise, caused by imperfect localization of the classifier, takes over, decreasing performance.

Conclusion
This work proposes a co-attention classification network to discover integral object regions by addressing cross-image semantics. With this regard, a co-attention is exploited to mine the common semantics within paired samples, while a contrastive co-attention is utilized to focus on the exclusive and unshared ones for capturing complimentary supervision cues. Additionally, by leveraging extra context from other related images, the co-attention boosts localization map inference. Further, by exploiting additional single-label images and web images, our approach is proven to generalize well under domain gap and data noise. Experiments over three WSSS settings consistently show promising results. Our method also ranked 1 st place in the weakly-supervised semantic segmentation track of LID 20 challenge.


s2fieldsofstudy:
Computer Science
Business

text:
Investigation of Cloud Computing Barriers: A Case Study in Saudi Arabian SMEs

Cloud Computing is arguably the most significant technological development after the Internet. It accelerated technology adoption and gave birth to new business models. More importantly, it is acting as a foundation for new technologies like the Internet of Things (IoT) and Artificial Intelligence (AI). The Cloud Computing paradigm provides a level playing field for Small and Medium-sized Enterprises (SMEs) as they are able to adopt technologies that were not affordable before. Governments, vendors and business support organisations across the world have a plethora of initiatives to encourage SME adoption of Cloud Computing technologies. Despite these initiatives, many SME decision makers are still hesitant to adopt the cloud. This research reports an engagement with 16 information technology (IT) managers working for Saudi manufacturing SMEs. This paper presents their perceived barriers to migrating their applications to cloud services.

INTRODUCTION
'Information and Communication Technology' (ICT) has played a vital role at the organisational level in transforming the 'operational practices' as well as 'electronic services' in both the public and private sectors (Basahel, Yamin, and Drijan, 2016). In the Middle East, Saudi Arabia is one of the largest markets for Enterprise IT products and services. Cloud Computing can be defined as the provision of IT services via the Internet, including storage, software, and computing power. Compared to traditional client server architecture, Cloud Computing provides IT solutions through virtualisation, service-oriented architecture, multi-tenancy and autonomic computing technologies. Accessing services via the cloud is attractive as businesses are invoiced based on consumption and there is little need for workforce to provide onsite support. Cloud Computing has eliminated the need for on site database, software, and server support. In recent years, Small and Medium-sized Enterprises (SMEs) have increased their investment in IT. Despite, the government's push in encouraging IT adoption, Saudi Arabia is still in the early stages of development. According to Amron, Ibrahim, and Chuprat (2017), the implementation of e-Government in Saudi Arabia has been delayed due to the weaknesses in the ICT infrastructure. The legal terms and conditions as well as appropriate resources are factors that delayed the Cloud Computing implementation in Saudi Arabia. It has been found in Basahel, Yamin, and Drijan (2016) that Saudi Arabia's current Revised Cloud Computing Regulatory Framework and strategies are helping them to accomplish quality outcomes in a timely manner. In the context of Saudi Arabian SMEs, many factors and conditions influence the decision to switch from an existing system to the cloud. Karim and Rampersad (2017) suggested that the decision is affected by the lack of knowledge and awareness. Cloud Computing implementation varied in other developed and developing countries, such as East Africa and South Africa (Aloraini and Hammoudeh, 2017). According to Alanezi (2018) Saudi Arabia has identified its productmanufacturing sector as one of the core drivers of the envisaged economic growth, with manufacturing continues to be a major contributor to its Gross Domestic Product (GDP). In 2018, the manufacturing and retail sectors continue to account for nearly half of all enterprises at 452,900 in total. The manufacturing sector accounts for approximately 103,700 enterprises. In Saudi Arabia, around 950,000 SMEs were registered in 2016 providing one million Saudis with employment (Raut et al., 2018). (GASTAT, 2016) predicted that in 2030 the number of SMEs in Saudi Arabia will be increased from 20% to 35%. There have been studies that analysed Cloud Computing in business. In this fast-developing technology, business dynamics changes and there is a need to continuously update research so that appropriate support polices can be in place to address adoption barriers.
This research aims to identify the barriers commonly faced by manufacturing SMEs of Saudi Arabia in adopting or migrating to Cloud Computing, using the qualitative research approach. The majority of previous studies of Saudi Cloud Computing are quantitative with the constructs re-used from the studies undertaken in the West. Since Saudi culture is very different from the west, qualitative research allows a deeper investigation of adoption barriers specific to the Saudi Arabia context. Furthermore, few studies explored decision makers' reasoning concerning Cloud Computing for manufacturing SMEs. SMEs are usually unaware of the risks and barriers that they might face during the transition, and thus, they often face severe consequences and ultimately fail to adopt Cloud Computing appropriately (Amron, Ibrahim, and Chuprat, 2017). Thus, hesitate to adopt Cloud services. The findings will enable the SMEs in the manufacturing sector of Saudi Arabia to gain a better understanding about the challenges of Cloud Computing adoption. This can pave the way for appropriate policy interventions to enable Cloud adoption, and contribute to the Saudi Arabia 2030 Vision.
The structure of this paper is as follows: Section II presents the literature review; Section III explains the methodology; Section IV provides a detailed analysis and interpretation of the interviews and Section V concludes the paper.

LITERATURE REVIEW
Recent views on the barriers to the adoption of Cloud Computing within SMEs of Saudi Arabia are listed in Table 1. Google Scholar, Emerald Insight, Science Direct and Scopus were used to identify the research relevant to cloud computing barriers and SMEs. The search range was limited to 2015-2019. According to a survey by the 'International Data Corporation' (IDC), the most common barrier to the adoption of Cloud Computing that concerns SMEs in Saudi Arabia is security and privacy (Raut et al., 2018). As they are less likely to control and own data, SMEs are usually hesitant to adopt Cloud Computing. Similarly, Al-Ruithe and Benkhelifa (2017), Al-Ruithe, Benkhelifa and Hameed (2017) indicate that the most common concern to the transition to Cloud Computing is privacy and security due to the lack of a physical structure. Almubarak (2017) reflects that the lack of trust of potential cloud services consumers is the main obstacle to adopting Cloud Computing. The work focuses on the fact that SMEs have less control over their data and are usually unaware of who is accessing their confidential data, resulting in trust issues. Al-Ruithe, Benkhelifa, and Hameed (2018) identifies the barrier of compliance. Due to the governmental laws and regulations that pertain to the storage and access of data on the Cloud, Saudi SMEs are hesitant to migrate to Cloud Computing. (AlBar and Hoque, 2019) suggests that availability and reliability are the main barriers that SMEs in Saudi Arabia tend to face, as the design and implementation of a reliable and available generic architecture for Cloud Computing is an open challenge. The research also focuses on the performance of Cloud Computing services. Karim and Rampersad (2017) states that the cost of the services for implementing the new technology is higher than that of the existing systems. Therefore, most of the SMEs in Saudi Arabia are hesitant in adopting Cloud Computing. Alharbi, Atkins, and Stanier (2017) studied five universities of Saudi Arabia, finding that Cloud Computing will soon be popular in the country.
The various barriers have been identified as shown in Figure 1 (arranged in alphabetical order).  (2017) Empirical findings from KSA public sector organisations Up to 70% of KSA organisations in the public sector currently do not adopt any cloud services, as they lack knowledge regarding Cloud Computing. 6 Raut et al. (2018) Through expert opinions and a literature review, 14 critical barriers are identified, and the ISM tool is used to establish the interrelationship between the identified barriers.
An analysis of the barriers reveals the three most significant barriers: lack of top management support (B3), lack of confidentiality (B8), and lack of sharing and collaboration (B2).
7 Al-Badi, Tarhini and Al-Qirim (2018) Qualitative, comprehensive study Three risk types (legal, organisational, and technical) are investigated. Security and privacy are also identified via a comprehensive analysis. 8 Shuaib and et al.
The literature review analysis methodology is used for the examination of key factors and challenges associated with the adoption of Cloud Computing.

METHODOLOGY
This study aims to identify the barriers commonly faced by manufacturing SMEs of Saudi Arabia in adopting or migrating to Cloud Computing. The chosen research method is qualitative. The primary data collection is semi-structured interview. This helps the researcher to assimilate rich data from the participants. The interview instrument is a set of open-ended questions that help the researcher to gather personal opinions regarding Cloud Computing. The semi-structured interviews were focused on identifying the barriers faced by manufacturing SMEs of Saudi Arabia. Questions focus on managers' perceptions of the cloud and its barriers and challenges. Questions such as; what challenges prevented your organisation from adopting the cloud services?.
Semi-structured interviews were conducted with a sample of 16 Information Technology (IT) managers from manufacturing SMEs within Saudi Arabia. The criterion for selecting the interviewees is that they have worked in SMEs in the manufacturing sector for at least three years. All the participants were familiar and aware of the barriers to the adoption of Cloud Computing. The profiles of the respondents are in Table 2.  As the researcher wanted to explore the rich insights, the entire interview conversation was recorded with consent from the respondent. The interview was transcribed for analysis and coded line by line to look for themes and patterns. The interviews were done observing the ethical policy and protocol of the university.
After 16 interviews the themes were recurring and new barriers categories were not identified. At that point, the researcher decided data collection reached saturation and there is no need for further interviews.
The responses were grouped into major themes and tallied based on the number of interviewees who mention them. Figure  2 highlights the ranked barriers. Figure 1 from literature are not present in Figure 2. This shows that the barriers are dynamic in nature. With advances in technology and the pressure to improve, some barriers have less priority in the Saudi manufacturing context. Figure 3.

The summary list of Saudi manufacturing SMEs' barriers from this research is depicted in
The identified barriers are in line with literature and no new emergent barriers were identified. This could be because Cloud Computing has been around for more than 10 years and the technology is becoming mature.

RESULTS AND DISCUSSION
The hindrances confirmed in the interviews were correlated with literature. From the analysis, the challenges and barriers retarding the endorsement of Cloud Computing by Saudi manufacturing SMEs include security, privacy, and lack of knowledge about the cloud, lack of governmental and top management support, provider lock-in, data control, service cost, and sufficient bandwidth.

Theme A: Security
In terms of the adoption of Cloud Computing, security is considered the foremost barrier that is still challenging for manufacturing SMEs in Saudi Arabia. During the interview, the CEO of one of the SMEs in Jeddah indicated that the prime reason behind SMEs' fear of and lack of interest in the implementation of Cloud Computing is security concerns. Cybersecurity threats and issues have adversely affected organisations such as hacking and phishing, whether they are giants or SMEs. Alharthi et al. (2017) supports the statement, according to which the problem of security becomes more complicated in Cloud Computing due to the addition of multiple dimensions, such as 'multi-tenancy'. The study also states that due to the lack of a physical structure, compliance with security is a critical task. Similarly, the CEO of another Jeddah-based manufacturing SME gave his stance on security as a barrier to the transition to Cloud Computing and stated, 'We believe that security is the only reason for the low uptake of the cloud. Alassafi (2018), and Basahel, Yamin and Drijan (2016) support the interpretation, reflecting that Cloud Computing is composed of various deployment and service delivery models that have security limitations with no current standards. Therefore, security is considered the foremost barrier to Saudi Arabian manufacturing SMEs' adoption of Cloud Computing.

Theme B: Lack of Top Management Support
One of the potential barriers to adopting Cloud Computing in Saudi Arabia is the lack of top management support. Management often does not show any concern or interest in the advancement of technology. An IT engineer of an SME in Jeddah stated, 'My manager still thinks that the Cloud is not suitable for our company despite successful implementations of some of our competitors'. The statement reflects the lack of interest in adopting cloud solutions. Almorsy, Grundy and Müller (2016) reflects that the interest of top management highly influences the decision to adopt Cloud Computing. However, if top management does not show any interest or support for the adoption of new technology, it becomes a huge barrier.

Theme C: Data Control
Another barrier that SMEs of Saudi Arabia usually face in adopting Cloud Computing is the control of data. SMEs fear they lose their control of the data in the Cloud. In this regard, the IT manager of an SME located in Dammam stated, 'Well, we just can't accept the fact that, we don't have controls over our data once it is with the service provider'. The findings of El-Gazzar, Hustad and Olsen (2016) reflect that SMEs cannot access and control their data is a potential barrier to the adoption of Cloud Computing in Saudi Arabia.

Theme D: Privacy
Another significant concern of Cloud Computing is privacy and it refers to more than information confidentiality. The data protection and safety of personal information derived from the privacy right through the associated right of self-determination. In the study of Albasri, Mishra and Elbeltagi (2016), the inefficiency in controlling the customers of cloud service is a severe threat towards confidentiality, data integrity, and principles of privacy. The fundamental principles that govern personal data use include fairness, lawfulness, data quality, proportionality, purpose specification, openness, and accountability. These are the basic principles of privacy that are common among legislation of different countries. From the perspective of consumers, Cloud Computing has privacy concerns. As per the survey of Ramachandra, Iftikhar and Khan (2017), around 74% of IT managers responded that the critical challenge that blocks them from using the services of cloud computing are privacy concerns. In this regard, the IT manager of one SME in Jeddah stated, 'Data privacy is so essential; we can't just rely on the cloud'. Martins, Sahandi, and Tian (2016) also supports the argument, reflecting that in Cloud Computing, multiple users share a single infrastructure at the same time, thus sacrificing privacy.

Theme E: Insufficient Bandwidth
Insufficient bandwidth is another critical barrier to Saudi Arabian SMEs' adoption of Cloud Computing. The IT assistant of a Medina-based SME stated, 'The bandwidth speed is still slow in some areas'. Similarly, the findings of El-Gazzar, Hustad and Olsen (2016) reflect that Cloud Computing requires 24*7 connectivity for accessing data. If due to technical issues and the internet is not available, one cannot access data on the cloud. Therefore, bandwidth is one serious barrier to adopting Cloud Computing as a permanent solution. It is supported by Al-Badi, Tarhini, and Al-Qirim (2018) according to which the lack of a secure internet connection is a potential barrier to Saudi Arabian SMEs' adoption of Cloud Computing.

Theme F: Lack of Government Support
Lack of government support is another key barrier to SMEs' adoption of Cloud Computing. SMEs in Saudi Arabia do not receive sufficient funding for the transition (Kalaiprasath, Elankavi and Udayakumar, 2017). The CEO of an SME in Riyadh stated, 'The government is not actually supporting SMEs to adopt the cloud. I do not know if they have funding or are introducing some sort of help'. In the same way, Amron, Ibrahim, and Chuprat (2017) supports the above notion, stating that there is a lack of appropriate governmental funding for cloud adoption, despite the fact that appropriate budgeting in the ICT sector is a potential challenge for SMEs. An IT manager of a Riyadh-based SME stated, 'The Saudi government places strict restrictions on storing data out of the country'. Alresheedi et al. (2018), andAlruithe et al. (2017) support the argument, reflecting that different countries have different laws and regulations about the usage and storage of data on the cloud due to different legal perspectives.

Theme G: Service Cost
One of the biggest concerns that SMEs of Saudi Arabia face is the high cost of services. One of the IT managers of a Makkahbased SME affirmed, 'Within our area, the service cost is high'. Supporting the argument, Amron, Ibrahim, and Chuprat (2017) reflects that due to the high cost of services in Saudi Arabia, it is difficult for SMEs to transition to Cloud Computing, as costs are likely to be higher than those of the existing system. Likewise, another IT manager of a Medina-based SME stated, 'The service cost of the cloud technologies is still high; most providers charge a lot'. Alresheedi et al. (2018), andAlruithe et al. (2017) further support the above notion by reflecting that the cost of services factor makes SMEs hesitant to adopt new technologies, particularly Cloud Computing. High costs affect Saudi SMEs in switching from the existing system.

Theme H: Provider Lock-in
Another significant barrier to the adoption of Cloud Computing is provider lock-in. Provider lock-in usually refers to proprietary technologies that are incompatible with or not required by companies. One of the IT specialists from a Jeddah-based SME stated, 'Being a medium-sized company, we do not want to lock our data with one supplier so if something goes wrong it won't be easy to transition to others'. Similarly, (Khan and Al-Yasiri, 2016) states that vendor lock-in is a potential barrier to the adoption of Cloud Computing because companies are restricted and tied to the same provider even if they do not need that technology. The study shows that this issue is still a big challenge for most of the SMEs of Saudi Arabia.

Theme I: Lack of 'Cloud' Knowledge
Even there is the popularity and enthusiasm of Cloud within the company, it has been identified that there is a shortage of operational skills and cloud knowledge, leading to unsuccessful adoption, barriers, and slower acclimatisation in continuous migration (Jouini, and Rabai, 2019) . Compliance must be considered to fulfil the legal requirements of stored and migrated data. Due to inadequate knowledge about the subject and the protocols, sometimes the legal obligations have not been complied in the fullest. Basahel, Yamin, and Drijan (2016) identified that there is a gap in technical skills that impact on the organisations and they can only use those components that are offered by cloud providers in which they have current knowledge. The interviews confirmed that the lack of Cloud Computing knowledge is one of the barriers that SMEs in Saudi Arabia face in adopting the infrastructure of the cloud. Without expertise and prior knowledge of Cloud Computing, the SMEs cannot migrate their existing systems to the cloud. One of the CEOs of a Jeddah-based SME suggested that 'Lack of appropriate knowledge of the cloud results in severe consequences'. As per Al-Badi, Tarhini, and Al-Qirim (2018), the SMEs' lack of expertise and knowledge of the functionalities of Cloud Computing make it very difficult for them to transition to and adopt the cloud. In the same way, the CEO of a Makkah-based SME stated, 'I believe training is a necessity (for cloud services); the knowledge we have is not enough'. Al-Ruithe, Benkhelifa and Hameed a.b (2017,2018) reflect that the implementation of Cloud Computing requires proper training as well as knowledge of the subject, which the SMEs in Saudi Arabia lack.
Similarly, another IT manager of a Jeddah-based SME stated, 'Support and help should be available from cloud service providers at all times'. Likewise, Beynon-Davies (2005) reflects that the adoption of Cloud Computing requires proper training and expertise, which is a barrier to implementing the cloud.

CONCLUSION
Cloud Computing is one of the most significant emerging technology of this era. However, there are barriers and challenges associated with the transition to Cloud Computing. This study presents an update on the perception of the key barriers that SMEs of Saudi Arabia are facing in the current stage of Cloud Computing. The study found the most common barriers are security, privacy, service cost, data control, insufficient bandwidth, lack of government support, lack of top management support, provider lock-in and lack of cloud knowledge and training. For SMEs in Saudi Arabia, these barriers are critical because Cloud Computing has the potential to unlock new growth opportunities for business. If prepared correctly, SMEs can easily overcome them and implement Cloud Computing in their organisations. The government of Saudi Arabia is playing a significant role in the advancement of ICT in the country. The barriers identified in this study can help the Saudi government as well as SMEs to develop effective strategies for migrating to Cloud Computing.


SENSOR FUSION, GIS AND AI TECHNOLOGIES FOR DISASTER MANAGEMENT

Modern Disaster Management Systems are based on several columns that combine theory and practice, software, and hardware being under technological advance. In all parts, spatial data is key in order to analyze existing structure, assist in risk assessment and update the information after a disaster incident. This paper focus on technological advances in several fields of spatial analysis putting together the advantages, limitations and technological aspects from well-known or even innovative methods, highlighting the huge potential of nowadays technologies for the field of Disaster Risk Management (DRM). A focus then is lying on GIS and Remote Sensing technologies that are showing the potential of high-quality sensors and image products that are getting easier to access and captured with recent technology. Secondly, several relevant sensors being thermal or laser-based are introduced pointing out the application possibilities, their limits, and potential fusion of them. Emphasis is further driven to Machine Learning techniques adopted from Artificial Intelligence that improve algorithms for auto-detection and represent an important step forwards to an integrated system of spatial data use in the Disaster Management Cycle. The combination of Multi-Sensor Systems, new Platform technologies, and Machine Learning indeed creates a very important benefit for the future.

INTRODUCTION
Disasters, either natural or man-made, happen more frequently and are often more intense due to the expansion and densification of urban structures and acceleration of climate change (O´Brien et al. 2006;van Aalst 2006;Bouwer 2011.). Every year, many people either die or are injured by disasters, besides that growing economic damage is needed to be handled. The risk appears in places of a human activity where nature is active and people have settlements (Altan and Kemper 2016.). Risk can be seen as a combination of a natural hazard, which takes an effect on exposed people that experience earthquakes, volcanism, tsunamis, weather-disasters, etc. meanwhile in other places natural disasters often occur unrecognized in areas apart from the population (Altan and Kemper 2016.). Urban sprawl and population growth result in the urbanization of areas that can be threatened by several hazards. People accessing these areas are forced to handle with their livelihoods i.e. farming and social circumstances e.g. volcanic soils like the Pululahua area in Northern Ecuador, which offers high fertility (Altan and Kemper 2016.).
Disasters can be based on climatic basis e.g. storms, floods, droughts, and others, meanwhile, geological disasters would contain earthquakes, tsunamis, volcanic eruptions, landslides, and avalanches. Further, biological disasters as epidemic or pandemic diseases (e.g. the Covid 19 pandemic) or plant diseases that lead e.g. to forest dying are crucial examples. Manmade hazards which should be taken into account are * Contributing author (civil) wars, economic crises or political conflicts (Altan and Kemper 2016.). 1 http://www.un-spider.org/ Modern technologies (hardware) in combination with geospatial data in GIS environment open a wide field for assisting in Disaster Risk Management. Taking into consideration the mitigation, preparation, response and recovery phases of the disaster management cycle (Haigh 2017.) abundant application options are thinkable with nowadays technologies. Crisis Preparedness Plans play a key role in the protection of exposed citizens (Altan and Kemper 2008.) meanwhile disciplines of Photogrammetry, Remote Sensing and Spatial Information Science can contribute in many ways to them. Further, the creation of risk awareness is a key to improve general conditions and measures for disaster risk handling. To be prepared means to have thorough knowledge about the geospatial structure, the mechanism of the disaster, and scenarios to model and train disaster management plans. Geospatial data is one of the basic resources to e.g. detect the critical areas in case of a flood, analyze areas that might be safer than others in case of an earthquake, and to find most suitable places for shelters for homeless people. Looking at the response phase a fast data availability is crucial to successful operation on the ground. It is necessary to guide rescue teams through the situation. Data referencing is, therefore, a fundamental basis to immediately feed the data server of crisis management teams (Altan and Kemper 2016.).
Gathering the abundant methods is needed to guarantee sustainable help to people, whereas interdisciplinary cooperation is highly recommended. The importance of GIS and RS for DRM is surely known within research, but also National and International Institutions or Organizations around the globe work in certain ways with geospatial data like UN SPIDER 1 , the National Entities of Canada (Agrawal 2018.) or the National Unity of Disaster Risk Management in Colombia 2 .
Linking disaster risk management to broader sustainable development goals is now envisaged with the "Sendai Framework for Disaster Risk Reduction" (United Nations Office for Disaster Risk Reduction 2015.). However, this is only possible with the knowledge transfer generated through research and periodic assessments that are effectively communicated to society and governments. DRM decision-makers should be provided with evidence of economic, operational, and strategic benefits of using scientific knowledge and information, which address prevention, mitigation, and response actions (e.g., Altan et al., 2010Altan et al., , 2013. UNDRR's Sendai Framework for Disaster Risk Reduction targets the enhancement of several important aspects in the Disaster Risk Management, which further draws attention to the importance of this topic. By outlining seven clear targets and four priorities for action the framework sets new standards for research and policymakers in the field of Disaster Management.

GIS
Geographic Information System (GIS) opened up new possibilities for data capture, management, analysis and presentation (FAO) working with Geospatial data, which can be described as information bound to certain locations on a coordinate system referring to the Earth 3 . Generally the field of Disaster Risk Management has recognized the usefulness and potential of geospatial analysis using GIS. The growing importance of the findings given by research shows that broad implementation of geospatial analysis can be given during all phases of the disaster management cycle (Haigh 2017.) showing potential for mitigating risk analysis, preventive identification of emergency areas and responsive emergency location and quick responsive mapping. This can be implemented in Preparedness Processes of Evacuation Planning, the Identification of High-Risk Areas, and the Response to provide accurate information on the exact location of an emergency context, as the potential is considerably high (Bala et al. 2017.). The number of papers published towards the GIS-based contributions to DRM shows the growing interest and importance of this method. Mainly Floods and Cyclones are given big importance in recent literature (among others Kienberger and Steinbruch 2005;Abbas S.H. et al. 2009;Khan 2013;Forslund 2016.). Further, the landslide susceptibility (Chen et al. 2018.) or Critical Infrastructure can be seen as examples for GIS focused research in the field of DRM. Having a look at participative approaches in Geoinformatics papers about Community-based DRM (Maskrey 2011;Canevari-Luzardo et al. 2017.) and the inclusion of People as 3 https://dictionary.cambridge.org/dictionary/english/geospatial Sensors for Data Contribution (Laituri and Kodrich 2008.) appear to give a new perspective for further bottom-up strategies in the field. Figure 1: Tsunami Run-up simulation represented as a Map of an area near Istanbul calculated via GIS using land-use data and a DTM (Kemper et.al 2005) 3. SENSORS

Satellite Images
Referring to satellite images as gap-filler is not up to date anymore. Satellite imagery offers a reliable and low-cost alternative to cost-intensive aerial images. Further, multi-spectral information (Landsat TM) gets a more detailed view of aspects, which are visibly hidden to us. Meanwhile the spatial resolution becomes better and the demand for free products is rising, geoprocessing and referencing of big data amounts gets better and faster (Altan and Kemper 2016.). Especially the use of land cover information can give an important extra to Risk Assessments during Preparatory Assessments of an area. They further provide information about the general landscape characteristics of an area and depending on the resolution will be a basis for digitalization works of buildings or roads, which could be realized in a response or mitigation phase (e.g. HOT initiative 4 ) After a disaster strikes a certain area satellite images are one of the first datasets that are available to the disaster response teams. They contribute crucial impressions from the area and can be taken as a first-glance look on the situation, meanwhile, the use for early-warning is limited (Altan and Kemper 2016.) Several research projects (Xueqiao Huang and John R. Jensen 1997; Alpar et al. 2004;Abdel-Aziz Belal et al. 2014.) working with Satellite imagery proof that the potential of this technology is not reached yet, since new satellites with better spatial and temporal resolution is launched every year (e.g. Sentinel satellites).

Radar and Hyperspectral Satellite data
Beside optical satellite sensors, radar, and hyperspectral sensors deliver data for a specific use. Radar can penetrate clouds and can produce a sufficient terrain model. Typically terrain models generated from Lidar or aerial images have a higher density and better accuracy. Therefore, Radar data is not really used for a basic geo-dataset. However, frequently updated radar surveys are able to deliver minor adjustments using interferometric analyses to highlight small changes in the DTM (Tronin 2010.). These anomalies show stress in the geological layers which indicates geological movementsmay be an upcoming earthquake. Radar is a good early warning sensor to indicate geological shock waves. Also after an earthquake, Radar highlights the changes in the terrain, useful information for the rescue teams. Hyperspectral sensors (Niculescu et al. 2010.) have been developed for monitoring environmental and geological structures. Focused on this specific task, they can support disaster management in all parts. Even the geometric resolution is not high, environmental disasters can be well monitored in order to provide early warning and disaster management. As also satellite images are, they can be made accessible for the Disaster Management typically a day after capturing. However, the way of data preprocessing is more complex in order to get the first visible result for inspection.

Aerial nadir images
Aerial nadir images have been and are excellent data sources being frequently used in the past for generating spatial data. During the last 60 years, aerial images have been the most dominant source for mapping and geodata management. Today's digital sensors have replaced the analog technology and enable typically RGB and NIR in one flight and quick access for rapid data update. Medium format mapping cameras with 150 MP can be used in UL aircraft, Trikes, and UASs. This widens the range of use for aerial images due to an enhanced range of GSD (0.1-20 cm). Meanwhile for UASs mid format metric cameras up to 150 MP and found their market to generate Orthophotos and aerial maps in highest resolution and accuracy (Altan and Kemper 2016.).
Nadir aerial images show far more details than satellite images do and the date for a flight can be defined according to the needed resolution and the weather conditions. That gives higher flexibility, resolution and accuracy, especially in 3D mapping applications. Big format digital mapping cameras have resolutions up to 450 MPix (Vexcel UltraCam Eagle Mark 3) that can cover wide areas. Midformat cameras meanwhile also cover three times more than old analog ones did; single cameras with 150 MP, dual stitched 4-bans solutions Systems with 280 MP or setups for wide corridor mapping with a 3 Camera setup at 35.000 pixels cross-track. Besides that, mid-format mapping cameras have changeable lenses that make them extremely flexible for various needs.
All these camera-setups do an excellent job for preparedness mapping in 3D as well as monitoring the Disaster in order to assist the rescue teams.
The new mid format cameras that make use CMOS sensors (e.g. Phase One IXM -RS 150) can capture data also under poor and difficult light conditions and are the first choice for remotely sensed data acquisition in aircraft and UASs. After a mission, the images can be accessed immediately even there is just a core reference. Using direct referencing methods, the image data can be directly be processed and overlayed in a GIS environment to generate rapid information. Inflight processing of images to generate rapid orthophoto on the fly is a technology invented in 2009 5 and since then continuously optimized 6 .

Oblique images
Oblique imaging became very requested within the last years even the way of capturing non-nadir images was already used for many decades. Besides architectural oblique imaging, also for open mining and other special tasks oblique images were used for a long time. For about eight years, there was a higher request for such data, either for homeland-security issues or for internet applications as Bing, Virtual Earth, Google, city planning, and others. With some exceptions, a core georeference to display the data on the approximate place over a map was sufficient. The image quality improved dramatically during the last years and the use of the generated data is not restricted anymore to segments.
Today we can make use of cameras with medium format sensors that capture images with 100-150 MPIX. Not just the higher resolution, also the precise image geometry of such mapping cameras widened the range to use these data also for precise mapping and modeling. The use of the today's captured oblique images is designed for precise 3D City models in order to generate a high dense DSM with an extreme good high accuracy and high redundancy of image data for texturing the facades. For two years, this initiated a strongly growing request for such systems. For the preparedness plan, we can make use of highly accurate oblique cameras system as the OIS-XL4B using 6 PhaseOne cameras each with 100-150 MPix a good way to capture nadir and oblique data in one flight and to extract a high dense city model with all information about the building structures that can assist to generate data for crisis preparedness. In this camera, a nadir 4band system is implemented to get also information as CIR and NDVI (Kemper 2018.). As shown on the figure 2 there is definitely a certain need for such information after e.g. an earthquake or a fire disaster in order to analyze the degree of damage to buildings. For this application, there is no need of a high geometric accurate sensor, also SLC Camera based Oblique Camera system as the OIS-S, which uses Canon Cameras, give great results. From these data, deformation analysis to estimate the remaining building stability can be produced (Kemper 2017.). Also these images can be accessed immediately even with a core reference. Using direct referencing methods, the nadir images data can be already overlaid in a GIS environment and build rapid access information. For the processing of the oblique images, special tools e.g. Photomesh from Skyline Soft or others on a rapid computing environment (e.g. cloud computing) are required.

Thermal Cameras
Thermal cameras measure the emission of longwave infrared emissions which are linked to the surface temperature. Finally, in such data, thermal anomalies are interpreted. The most common is the use of aerial thermal data for heat loss analysis and the determination of urban microclimate. It is also used for environmental monitoring of rivers and lakes/seas as well as an indication for water in the soil or urban structures (Vasel et al. 2012.). That way they can support the database with additional information about the environmental situation and the effect of urban structures. In the monitoring process while detecting changes in the temperature, thermal cameras can be used as early warning sensors, e.g. in the warm-up phases of a geological structure (volcanoes) or environmental disasters. Beside the application for searching people, thermal anomalies can be affected by humidity in constructions (transpiration effects), damaged power lines (e.g. defective Insulators), burning gas tubes, and many other dangerous facts. After a Disaster, Thermal images can help to identify people in the observed area. Thermal data in combination with RGB give further information on the building structure, damages, and potential water intrusion. Even small cracks in concrete can infilter water which itself reduces the thermal emission which is visible in thermal data.
Interpretation of Thermal data are complex due to variation in material emission coefficients, nevertheless context-wise can give good additional information for the crisis managers.
Better nadir the thermal camera only is an additional survey of oblique thermal information. Oblique thermal high-resolution sensor with nine heads (Aero OTS from GGS) enables highresolution nadir and oblique thermal data acquisition. That way, also facades and partly views inside the building are possible and widens the information. A big advantage of thermal sensor ais the usability in day and nighttime. This enables the possibility to fly e.g. during the day in combination with RGB-NIR or hyperspectral and during nighttime in combination with Lidar. Due to the small data volume of thermal images they can be accessed immediately after the flight or already downlinked using modern mobile networks (5G) to a control centre. Using direct referencing methods, the thermal data can be part of the rapid access to GIS.

Lidar
Lidar can generate dense DSM and also DTM using last Pulse information and algorithms to remove artificial objects and vegetation. The height information is received by direct distance measurements and results in an accurate and very dense point cloud. This represents the surface perfectly and builds an excellent data-source for the preparedness plan. Dual-frequency Lidar scanners can measure the depth of water and can be used for bathymetry. This gives important information on changes in the shoreline e.g. after a Tsunami. Lidar supports the disaster management by analyzing changes in the DSM before and after an "event" (Steinle and Bähr 1999.). The advantage of Lidar is that besides rain and clouds, no other weather conditions limit their use. This enables aerial missions at nighttime e.g. in combination with Thermal cameras.
After a mission, the data requires a certain level of preprocessing. Access can be realized roughly about one day after the flight. Innovations are in the development of Lidar systems using RGB laser-beams that enable the generation of colored point clouds that represent height and surface analysis. That way it becomes an image sensor that enables to capture also visible data in the night time. Even not being equal to an aerial image, it widens the surveying time-frame for capturing data dramatically. Small and powerful Lidarsystems can be carried by UAS systems (e.g. AeroSpector) and capture missions of 40' flight time using lidar and high-resolution aerial imaging sensors in combination.

Corona Camera
The GGS Corona Camera is designed for a high frequent RGB Camera beside a daylight filtered UV amplified sensor. It helps to monitor discharge effects on electric powerlines and electric systems to indicate issues that can lead to complete shutdowns or even in the worst case to danger to life or fire hazards.
The combination of these 2 sensors enable the direct overlay of the discharge spikes over an RGB Image

PLATFORMS
As described in the previous part, some platforms are closely linked to the sensor. Satellites travel on discrete orbits and adaptations of the observed area are possible only by rotating the angle of view of the sensor. With the exception of Radar, satellites are depended also on weather conditions. But they are always operational and are able to collect data meanwhile overflying the project area. The resolutions are predefined. Aircraft are far more flexible for carrying various sensors, adopt the resolution by the variation of flight altitude and sensor, and are flexible for flight time. Nevertheless, they need a certain infrastructure and have to be available together with equipment and crew for an aerial survey. Some sensors have been developed for generating ultra-high resolution on Helicopters. This enables closer distance to the objects, lower airspeed, and short take-off and landing area. However, the costs are far higher than for a wing based aircraft.
Figure 4: Monitoring system with 14 sensors for powerline monitoring, adaptable also for disaster mapping (Kemper 2018.) Nevertheless sensors became smaller, lighter and better and also small aircraft as Trikes, Gyrocopters and UAS can be used. Innovative platforms are UAVs for monitoring and mapping small areas. The advantages of such UAVs are that they can fly inside the urban canyons and might monitor the facades very detailed. Some of them are capable to fly inside a building and can assist in searching people. These UAVs can be transported to the project areas easily but you must have certain access to these areas. UAVs have a short endurance and are limited by their payload. But they can operate in areas which are not accessible by aircraft or satellites and work that way complementary. UAVs will surely be more and more part of the disaster management on a detailed level. Today equipped with video live cams using RGB and Thermal IR, they assist in looking inside buildings and behind. Thus, they can continue with the aerial survey where airborne anomalies have been detected.

AI
AI is a striving and actual topic in many scientific fields and methods or concepts of machine learning are applied vividly in research. Concerning the topic of DRM an enhanced analysis of Risk or Post-Event Damages is possible. Further methods from AI were recognized as beneficial for GIS (Egenhofer and Frank 1990.). As the concept of AI in GIS was proposed in the 1990s several concepts of Machine Learning have found their way into Geospatial Analysis (Domshlak et al. 2011.). Machine Learning is often given as the "science of computer modelling of learning processes", which means that it empowers a machine to use given information and theory for the improvement of the machine´s knowledge (Xueqiao Huang and John R. Jensen 1997.) Machine Learning as one of the powerful tools from AI in GIS is discussed in detail by several scientific researchers (Xueqiao Huang and John R. Jensen 1997; Allen et al. 2016;Mojaddadi et al. 2017;Chen et al. 2018.) where it is used for geospatial risk assessments and susceptibility maps. Looking at the advanced machine learning application in Chang et al. 2019. several techniques like Random Forest, Logistic Regression and Support Vector Machine find their way into Geospatial Analysis for a Landslides Susceptibility Mapping in Taiwan. Results reveal, that several of the introduced and conducted techniques can offer a valuable add on to Susceptibility Mapping and Disaster Risk Management in general.
Nowadays research uses these approaches to train computers by giving them a training data set and teach them several rules or indicators, which give it a certain directive. Machine Learning is now able to filter out noise from Twitter data to reduce the amount of information to the relevant one for a Disaster Scenario (Allen et al. 2016.). Further, Machine Learning is useful for the Improvement of the Disaster Communication, as it enhances the appropriation of Response mechanisms and the awareness of local stakeholders (Ighodaro Ogie et al. 2018.). First International Organizations do recognize the high potential of Machine Learning for their Disaster Risk Management Programs, like the World Bank (GFDRR 2018.) Artificial Intelligence plays a crucial role in future Risk analysis and communication with local Managers.

CONCLUSION
Today we have a good variation of sensors, software, and carriers to support Disaster Risk Management. Big technological developments opened up abundant possibilities to support Disaster Risk Management processes capturing, processing, and representing geospatial data in all phases of a disastrous occurrence. Taking into consideration the entrepreneurship working on hardware and platform system e.g. UAV systems and thermal, RGB or oblique sensors, faster data capturing and availability is created, due to smaller and more powerful sensors. Thermal cameras and UV cameras, and in general the sensor combination gathers information together, which is essential for disaster response. Combining these new improvements with the several AI methods, i.e. Machine Learning and GIS software applications a very promising path towards integrated and interdisciplinary Disaster Risk Management. We never have been so flexible in the way of capturing remotely sensed data besides that the data quality improved dramatically. The idea to join different sensors generates new strategies in analyzing the results. There is a big task in the data-fusion and Meta-data management to be done beforehand. We should make use of the technology to support our society in times of crisis and make a pathway for sustainable development and resilient communities.