title: Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training

abstract: Generative dialogue models currently suffer from a number of problems which standard maximum likelihood training does not address. They tend to produce generations that (i) rely too much on copying from the context, (ii) contain repetitions within utterances, (iii) overuse frequent words, and (iv) at a deeper level, contain logical flaws. In this work we show how all of these problems can be addressed by extending the recently introduced unlikelihood loss (Welleck et al., 2019) to these cases. We show that appropriate loss functions which regularize generated outputs to match human distributions are effective for the first three issues. For the last important general issue, we show applying unlikelihood to collected data of what a model should not do is effective for improving logical consistency, potentially paving the way to generative models with greater reasoning ability. We demonstrate the efficacy of our approach across several dialogue tasks.


title: Increasing Robustness to Spurious Correlations using Forgettable Examples

abstract: Neural NLP models tend to rely on spurious correlations between labels and input features to perform their tasks. Minority examples, i.e., examples that contradict the spurious correlations present in the majority of data points, have been shown to increase the out-of-distribution generalization of pre-trained language models. In this paper, we first propose using example forgetting to find minority examples without prior knowledge of the spurious correlations present in the dataset. Forgettable examples are instances either learned and then forgotten during training or never learned. We empirically show how these examples are related to minorities in our training sets. Then, we introduce a new approach to robustify models by fine-tuning our models twice, first on the full training data and second on the minorities only. We obtain substantial improvements in out-of-distribution generalization when applying our approach to the MNLI, QQP, and FEVER datasets.


title: Unsupervised Annotation of Phenotypic Abnormalities via Semantic Latent Representations on Electronic Health Records

abstract: The extraction of phenotype information which is naturally contained in electronic health records (EHRs) has been found to be useful in various clinical informatics applications such as disease diagnosis. However, due to imprecise descriptions, lack of gold standards and the demand for efficiency, annotating phenotypic abnormalities on millions of EHR narratives is still challenging. In this work, we propose a novel unsupervised deep learning framework to annotate the phenotypic abnormalities from EHRs via semantic latent representations. The proposed framework takes the advantage of Human Phenotype Ontology (HPO), which is a knowledge base of phenotypic abnormalities, to standardize the annotation results. Experiments have been conducted on 52,722 EHRs from MIMIC-III dataset. Quantitative and qualitative analysis have shown the proposed framework achieves state-of-the-art annotation performance and computational efficiency compared with other methods.


title: Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks

abstract: Self-supervised pre-training of transformer models has shown enormous success in improving performance on a number of downstream tasks. However, fine-tuning on a new task still requires large amounts of task-specific labelled data to achieve good performance. We consider this problem of learning to generalize to new tasks with few examples as a meta-learning problem. While meta-learning has shown tremendous progress in recent years, its application is still limited to simulated problems or problems with limited diversity across tasks. We develop a novel method, LEOPARD, which enables optimization-based meta-learning across tasks with different number of classes, and evaluate different methods on generalization to diverse NLP classification tasks. LEOPARD is trained with the state-of-the-art transformer architecture and shows better generalization to tasks not seen at all during training, with as few as 4 examples per label. Across 17 NLP tasks, including diverse domains of entity typing, natural language inference, sentiment analysis, and several other text classification tasks, we show that LEOPARD learns better initial parameters for few-shot learning than self-supervised pre-training or multi-task training, outperforming many strong baselines, for example, yielding 14.5% average relative gain in accuracy on unseen tasks with only 4 examples per label.


title: Improving Transformer Models by Reordering their Sublayers

abstract: Multilayer transformer networks consist of interleaved self-attention and feedforward sublayers. Could ordering the sublayers in a different pattern lead to better performance? We generate randomly ordered transformers and train them with the language modeling objective. We observe that some of these models are able to achieve better performance than the interleaved baseline, and that those successful variants tend to have more self-attention at the bottom and more feedforward sublayers at the top. We propose a new transformer pattern that adheres to this property, the sandwich transformer, and show that it improves perplexity on multiple word-level and character-level language modeling benchmarks, at no cost in parameters, memory, or training time. However, the sandwich reordering pattern does not guarantee performance gains across every task, as we demonstrate on machine translation models. Instead, we suggest that further exploration of task-specific sublayer reorderings is needed in order to unlock additional gains.


title: A new tool for predicting the solar cycle: Correlation between flux transport at the equator and the poles

abstract: The magnetic flux cancellation on the Sun plays a crucial role in determining the manner in which the net magnetic flux changes in every solar cycle, affecting the large scale evolution of the coronal magnetic fields and heliospheric environment. We investigate, in this paper, the correlation between the solar magnetic flux cancelled at the equator and the solar magnetic flux transported to the poles by comparing the net amount of magnetic flux in the latitude belt 0${^{\circ}}$-5${^{\circ}}$ and 45${^{\circ}}$-60${^{\circ}}$, estimated using synoptic magnetograms from National Solar Observatory at Kitt Peak, during Solar Cycles 21-24. We find a good correlation between the net flux in the latitude band 0${^{\circ}}$-5${^{\circ}}$ and 55${^{\circ}}$-60${^{\circ}}$ for the Northern as well as for the Southern hemispheres. However, we find a poor correlation if the net flux for the Northern and Southern hemispheres are considered together. In addition, we investigate the correlation between the net flux cancelled at the equator and the strength of solar polar field at cycle minimum, and find a good correlation between the two. We discuss the implication of the correlation of flux transported across the equator and to the poles that has an important bearing in the estimation of the residual polar cap field strength at the cycle minimum. This can be used a predictive tool for estimating the amplitude of subsequent cycles and we use this to estimate maximum smoothed sunspot numbers of 77$\pm$5 and 85$\pm$5 for the Northern and Southern hemispheres, respectively, for the upcoming Solar Cycle 25.


title: Epitaxial stabilization of ultra thin films of high entropy perovskite

abstract: High entropy oxides (HEOs) are a class of materials, containing equimolar portions of five or more transition metal and/or rare-earth elements. We report here about the layer-by-layer growth of HEO [(La$_{0.2}$Pr$_{0.2}$Nd$_{0.2}$Sm$_{0.2}$Eu$_{0.2}$)NiO$_3$] thin films on NdGaO$_3$ substrates by pulsed laser deposition. The combined characterizations with in-situ reflection high energy electron diffraction, atomic force microscopy, and X-ray diffraction affirm the single crystalline nature of the film with smooth surface morphology. The desired +3 oxidation of Ni has been confirmed by an element sensitive X-ray absorption spectroscopy measurement. Temperature dependent electrical transport measurements revealed a first order metal-insulator transition with the transition temperature very similar to the undoped NdNiO$_3$. Since both of these systems have a comparable tolerance factor, this work demonstrates that the electronic behaviors of $A$-site disordered perovskite-HEOs are primarily controlled by the average tolerance factor.


title: A Modular Deep Learning Pipeline for Galaxy-Scale Strong Gravitational Lens Detection and Modeling

abstract: Upcoming large astronomical surveys are expected to capture an unprecedented number of strong gravitational lensing systems. Deep learning is emerging as a promising practical tool for the detection and quantification of these galaxy-scale image distortions. The absence of large quantities of representative data from current astronomical surveys motivates the development of a robust forward-modeling approach using synthetic lensing images. Using a mock sample of strong lenses created upon a state-of-the-art extragalactic catalogs, we train a modular deep learning pipeline for uncertainty-quantified detection and modeling with intermediate image processing components for denoising and deblending the lensing systems. We demonstrate a high degree of interpretability and controlled systematics due to domain-specific task modules trained with different stages of synthetic image generation. For lens detection and modeling, we obtain semantically meaningful latent spaces that separate classes of strong lens images and yield uncertainty estimates that explain the origin of misclassified images and provide probabilistic predictions for the lens parameters. Validation of the inference pipeline has been carried out using images from the Subaru telescope's Hyper Suprime-Cam camera, and LSST DESC simulated DC2 sky survey catalogues.


title: Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

abstract: We introduce an approach for open-domain question answering (QA) that retrieves and reads a passage graph, where vertices are passages of text and edges represent relationships that are derived from an external knowledge base or co-occurrence in the same article. Our goals are to boost coverage by using knowledge-guided retrieval to find more relevant passages than text-matching methods, and to improve accuracy by allowing for better knowledge-guided fusion of information across related passages. Our graph retrieval method expands a set of seed keyword-retrieved passages by traversing the graph structure of the knowledge base. Our reader extends a BERT-based architecture and updates passage representations by propagating information from related passages and their relations, instead of reading each passage in isolation. Experiments on three open-domain QA datasets, WebQuestions, Natural Questions and TriviaQA, show improved performance over non-graph baselines by 2-11% absolute. Our approach also matches or exceeds the state-of-the-art in every case, without using an expensive end-to-end training regime.


title: Knowledge Guided Named Entity Recognition for BioMedical Text

abstract: In this work, we formulate the NER task as a multi-answer knowledge guided QA task (KGQA) which helps to predict entities only by assigning B, I and O tags without associating entity types with the tags. We provide different knowledge contexts, such as, entity types, questions, definitions and examples along with the text and train on a combined dataset of 18 biomedical corpora. This formulation (a) enables systems to jointly learn NER specific features from varied NER datasets, (b) can use knowledge-text attention to identify words having higher similarity to provided knowledge, improving performance, (c) reduces system confusion by reducing the prediction classes to B, I, O only, and (d) makes detection of nested entities easier. We perform extensive experiments of this KGQA formulation on 18 biomedical NER datasets, and through experiments we note that knowledge helps in achieving better performance. Our problem formulation is able to achieve state-of-the-art results in 12 datasets.


title: Synthesis of Feedback Controller for Nonlinear Control Systems with Optimal Region of Attraction

abstract: We propose a framework for synthesizing a feedback control policy that maximizes the region of attraction (ROA) of a closed-loop nonlinear dynamical system. Our synthesis technique relies on stochastic optimization, which involves computation of an objective function capturing the ROA for a feedback control law. We employ a machine learning technique based on deep neural network to estimate the ROA for a given feedback controller. Overall, our technique is capable of synthesizing a controller co-optimizing traditional control objectives like LQR cost together with ROA. We demonstrate the efficacy of our technique through exhaustive experiments carried out on various nonlinear systems.


title: Constructing a Data Visualization Recommender System

abstract: Choosing a suitable visualization for data is a difficult task. Current data visualization recommender systems exist to aid in choosing a visualization, yet suffer from issues such as low accessibility and indecisiveness. In this study, we first define a step-by-step guide on how to build a data visualization recommender system. We then use this guide to create a model for a data visualization recommender system for non-experts that aims to resolve the issues of current solutions. The result is a question-based model that uses a decision tree and a data visualization classification hierarchy in order to recommend a visualization. Furthermore, it incorporates both task-driven and data characteristics-driven perspectives, whereas existing solutions seem to either convolute these or focus on one of the two exclusively. Based on testing against existing solutions, it is shown that the new model reaches similar results while being simpler, clearer, more versatile, extendable and transparent. The presented guide can be used as a manual for anyone building a data visualization recommender system. The resulting model can be applied in the development of new data visualization software or as part of a learning tool.


title: Location Attention for Extrapolation to Longer Sequences

abstract: Neural networks are surprisingly good at interpolating and perform remarkably well when the training set examples resemble those in the test set. However, they are often unable to extrapolate patterns beyond the seen data, even when the abstractions required for such patterns are simple. In this paper, we first review the notion of extrapolation, why it is important and how one could hope to tackle it. We then focus on a specific type of extrapolation which is especially useful for natural language processing: generalization to sequences that are longer than the training ones. We hypothesize that models with a separate content- and location-based attention are more likely to extrapolate than those with common attention mechanisms. We empirically support our claim for recurrent seq2seq models with our proposed attention on variants of the Lookup Table task. This sheds light on some striking failures of neural models for sequences and on possible methods to approaching such issues.


title: A constrained minimum criterion for model selection

abstract: We propose a hypothesis test based model selection criterion for the best subset selection of sparse linear models. We show it is consistent in that the probability of its choosing the true model approaches one and the parameter values of its chosen model converge in probability to that of the true model as the sample size goes to infinity. This criterion is capable of controlling the balance between the false active rate and false inactive rate of the selected model, and it can be applied with other methods of model selection such as the lasso. We also demonstrate its accuracy and advantages with a numerical comparison and an application.


title: Non-stationarity in Esbjerg sea-level returnlevels: Applications in climate change adaptation

abstract: Non-stationary time series modelling is applied to long tidal records from Esbjerg, Denmark, and coupled to climate change projections for sea-level and storminess, to produce projections of likely future sea-level maxima. The model has several components: nonstationary models for mean sea-level, tides and extremes of residuals of sea level above tide level. The extreme value model (at least on an annual scale) has location-parameter dependent on mean sea level. Using the methodology of Bolin et al. (2015)and Guttorp et al. (2014) we calculate, using CMIP5 climate models, projections for mean sea level with attendant uncertainty. We simulate annual maxima in two ways - one method uses the empirically fitted non-stationary generalized extreme-value-distribution (GEV) of 20th-century annual maxima projected forward based on msl-projections, and the other has a stationary approach to extremes. We then consider return levels and the increase in these from AD 2000 to AD 2100. We find that the median of annual maxima with return period 100 years, taking into account all the nonstationarities, in year 2100 is 6.5m above current mean sea level (start of 21stC) levels


title: Rethinking Self-Attention: Towards Interpretability in Neural Parsing

abstract: Attention mechanisms have improved the performance of NLP tasks while allowing models to remain explainable. Self-attention is currently widely used, however interpretability is difficult due to the numerous attention distributions. Recent work has shown that model representations can benefit from label-specific information, while facilitating interpretation of predictions. We introduce the Label Attention Layer: a new form of self-attention where attention heads represent labels. We test our novel layer by running constituency and dependency parsing experiments and show our new model obtains new state-of-the-art results for both tasks on both the Penn Treebank (PTB) and Chinese Treebank. Additionally, our model requires fewer self-attention layers compared to existing work. Finally, we find that the Label Attention heads learn relations between syntactic categories and show pathways to analyze errors.


title: Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering

abstract: Understanding narratives requires reasoning about implicit world knowledge related to the causes, effects, and states of situations described in text. At the core of this challenge is how to access contextually relevant knowledge on demand and reason over it. In this paper, we present initial studies toward zero-shot commonsense question answering by formulating the task as inference over dynamically generated commonsense knowledge graphs. In contrast to previous studies for knowledge integration that rely on retrieval of existing knowledge from static knowledge graphs, our study requires commonsense knowledge integration where contextually relevant knowledge is often not present in existing knowledge bases. Therefore, we present a novel approach that generates contextually-relevant symbolic knowledge structures on demand using generative neural commonsense knowledge models. Empirical results on two datasets demonstrate the efficacy of our neuro-symbolic approach for dynamically constructing knowledge graphs for reasoning. Our approach achieves significant performance boosts over pretrained language models and vanilla knowledge models, all while providing interpretable reasoning paths for its predictions.


title: Comparison of The UV and Optical Fe II Emission in Type 1 AGNs

abstract: We present the kinematical properties of the UV and optical \FeII\ emission gas based on the velocity shift and line width measurements of a sample of 223 Type 1 active galactic nuclei (AGNs) at 0.4 $<$ z $<$ 0.8. We find a strong correlation between the line widths of the UV and optical \FeII\ emission lines, indicating that both \FeII\ emission features arise from similar distances in the broad line region (BLR). However, in detail we find differing trends, depending on the width of \FeII. While the velocity shifts and dispersions of the UV Fe II (\FeUV) and optical Fe II (\FeOPT) emission lines are comparable to each other for AGNs with relatively narrow \FeOPT\ line widths (i.e., FWHM < 3200 \kms; Group A), \FeOPT\ is broader than \FeUV\ for AGNs with relatively broad \FeOPT\ (i.e., FWHM > 3200 \kms; Group B). \FeII\ emission lines are on average narrower than \Hb\ and \MgII\ for Group A, indicating the \FeII\ emission region is further out in the BLR, while for Group B AGNs \FeOPT\ is comparable to \Hb\ and broader than \MgII. While \FeII\ emission lines are on average redshifted ($40\pm141$ \kms\ and $182\pm95$, respectively for \FeUV\ and \FeOPT), indicating inflow, the sample as a whole shows a large range of velocity shifts, suggesting complex nature of gas kinematics.


title: An Overview of Data-Importance Aware Radio Resource Management for Edge Machine Learning

abstract: The 5G network connecting billions of Internet-of-Things (IoT) devices will make it possible to harvest an enormous amount of real-time mobile data. Furthermore, the 5G virtualization architecture will enable cloud computing at the (network) edge. The availability of both rich data and computation power at the edge has motivated Internet companies to deploy artificial intelligence (AI) there, creating the hot area of edge-AI. Edge learning, the theme of this project, concerns training edge-AI models, which endow on IoT devices intelligence for responding to real-time events. However, the transmission of high-dimensional data from many edge devices to servers can result in excessive communication latency, creating a bottleneck for edge learning. Traditional wireless techniques deigned for only radio access are ineffective in tackling the challenge. Attempts to overcome the communication bottleneck has led to the development of a new class of techniques for intelligent radio resource management (RRM), called data-importance aware RRM. Their designs feature the interplay of active machine learning and wireless communication. Specifically, the metrics that measure data importance in active learning (e.g., classification uncertainty and data diversity) are applied to RRM for efficient acquisition of distributed data in wireless networks to train AI models at servers. This article aims at providing an introduction to the emerging area of importance-aware RRM. To this end, we will introduce the design principles, survey recent advancements in the area, discuss some design examples, and suggest some promising research opportunities.


title: A class of two or three weights linear codes and their complete weight enumerators

abstract: In the past few years, linear codes with few weights and their weight analysis have been widely studied. In this paper, we further investigate a class of two-weight or three-weight linear codes from defining sets and determine their weight and complete weight enumerators by application of the theory of quadratic forms and some special Weil sums over finite fields. Some punctured codes of the discussed linear codes are optimal or almost optimal with respect to the Griesmer bound. This paper generalizes some results in \cite{ZhuXu2017,Jian2019}.


title: Effect of Vacancy Creation and Annihilation on Grain Boundary Motion

abstract: Interaction of vacancies with grain boundaries (GBs) is involved in many processes occurring in materials, including radiation damage healing, diffusional creep, and solid-state sintering. We analyze a model describing a set of processes occurring at a GB in the presence of a non-equilibrium, non-homogeneous vacancy concentration. Such processes include vacancy diffusion toward, away from, and across the GB, vacancy generation and absorption at the GB, and GB migration. Numerical calculations within this model reveal that the coupling among the different processes gives rise to interesting phenomena, such as vacancy-driven GB motion and accelerated vacancy generation/absorption due to GB motion. The key combinations of the model parameters that control the kinetic regimes of the vacancy-GB interactions are identified via a linear stability analysis. Possible applications and extensions of the model are discussed.e's comments


title: Resonant spaces for volume preserving Anosov flows

abstract: We consider Anosov flows on closed 3-manifolds preserving a volume form $\Omega$. Following Dyatlov and Zworski (2017) we study spaces of invariant distributions with values in the bundle of exterior forms whose wavefront set is contained in the dual of the unstable bundle. Our first result computes the dimension of these spaces in terms of the first Betti number of the manifold, the cohomology class $[\iota_{X}\Omega]$ (where $X$ is the infinitesimal generator of the flow) and the helicity. These dimensions coincide with the Pollicott-Ruelle resonance multiplicities under the assumption of $\textit{semisimplicity}$. We prove various results regarding semisimplicity on 1-forms, including an example showing that it may fail for time changes of hyperbolic geodesic flows. We also study non null-homologous deformations of contact Anosov flows and we show that there is always a splitting Pollicott-Ruelle resonance on 1-forms and that semisimplicity persists in this instance. These results have consequences for the order of vanishing at zero of the Ruelle zeta function. Finally our analysis also incorporates a flat unitary twist in both, the resonant spaces and the Ruelle zeta function.


title: Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders

abstract: Conditional Text Generation has drawn much attention as a topic of Natural Language Generation (NLG) which provides the possibility for humans to control the properties of generated contents. Current conditional generation models cannot handle emerging conditions due to their joint end-to-end learning fashion. When a new condition added, these techniques require full retraining. In this paper, we present a new framework named Pre-train and Plug-in Variational Auto-Encoder (PPVAE) towards flexible conditional text generation. PPVAE decouples the text generation module from the condition representation module to allow "one-to-many" conditional generation. When a fresh condition emerges, only a lightweight network needs to be trained and works as a plug-in for PPVAE, which is efficient and desirable for real-world applications. Extensive experiments demonstrate the superiority of PPVAE against the existing alternatives with better conditionality and diversity but less training effort.


title: Sequential Recommendation with Dual Side Neighbor-based Collaborative Relation Modeling

abstract: Sequential recommendation task aims to predict user preference over items in the future given user historical behaviors. The order of user behaviors implies that there are resourceful sequential patterns embedded in the behavior history which reveal the underlying dynamics of user interests. Various sequential recommendation methods are proposed to model the dynamic user behaviors. However, most of the models only consider the user's own behaviors and dynamics, while ignoring the collaborative relations among users and items, i.e., similar tastes of users or analogous properties of items. Without modeling collaborative relations, those methods suffer from the lack of recommendation diversity and thus may have worse performance. Worse still, most existing methods only consider the user-side sequence and ignore the temporal dynamics on the item side. To tackle the problems of the current sequential recommendation models, we propose Sequential Collaborative Recommender (SCoRe) which effectively mines high-order collaborative information using cross-neighbor relation modeling and, additionally utilizes both user-side and item-side historical sequences to better capture user and item dynamics. Experiments on three real-world yet large-scale datasets demonstrate the superiority of the proposed model over strong baselines.


title: Learning Koopman Operator under Dissipativity Constraints

abstract: This paper addresses a learning problem for nonlinear dynamical systems with incorporating any specified dissipativity property. The nonlinear systems are described by the Koopman operator, which is a linear operator defined on the infinite-dimensional lifted state space. The problem of learning the Koopman operator under specified quadratic dissipativity constraints is formulated and addressed. The learning problem is in a class of the non-convex optimization problem due to nonlinear constraints and is numerically intractable. By applying the change of variable technique and the convex overbounding approximation, the problem is reduced to sequential convex optimization and is solved in a numerically efficient manner. Finally, a numerical simulation is given, where high modeling accuracy achieved by the proposed approach including the specified dissipativity is demonstrated.


title: Miniature Neutron Spectrometer for Space

abstract: MIDAS is a miniature detector developed with purpose to assess the radiation field parameters near to an astronaut. Part of the device is a spectrometer for fast neutrons. In missions outside the geomagnetic field, fast neutrons are secondary products of the interaction of Galactic Cosmic Ray heavy ions with the materials in the spaceship or even the astronaut body. The Relative Biological Effectiveness of fast neutrons is high. The neutron spectrometer first prototype has been developed, calibrated and used for measuring 252Cf spectra.


title: Performance Analysis on Machine Learning-Based Channel Estimation

abstract: Recently, machine learning-based channel estimation has attracted much attention. The performance of machine learning-based estimation has been validated by simulation experiments. However, little attention has been paid to the theoretical performance analysis. In this paper, we investigate the mean square error (MSE) performance of machine learning-based estimation. Hypothesis testing is employed to analyze its MSE upper bound. Furthermore, we build a statistical model for hypothesis testing, which holds when the linear learning module with a low input dimension is used in machine learning-based channel estimation, and derive a clear analytical relation between the size of the training data and performance. Then, we simulate the machine learning-based channel estimation in orthogonal frequency division multiplexing (OFDM) systems to verify our analysis results. Finally, the design considerations for the situation where only limited training data is available are discussed. In this situation, our analysis results can be applied to assess the performance and support the design of machine learning-based channel estimation.


title: Deep Reinforcement Learning Based Dynamic Trajectory Control for UAV-assisted Mobile Edge Computing

abstract: In this paper, we consider a platform of flying mobile edge computing (F-MEC), where unmanned aerial vehicles (UAVs) serve as equipment providing computation resource, and they enable task offloading from user equipment (UE). We aim to minimize energy consumption of all the UEs via optimizing the user association, resource allocation and the trajectory of UAVs. To this end, we first propose a Convex optimizAtion based Trajectory control algorithm (CAT), which solves the problem in an iterative way by using block coordinate descent (BCD) method. Then, to make the real-time decision while taking into account the dynamics of the environment (i.e., UAV may take off from different locations), we propose a deep Reinforcement leArning based Trajectory control algorithm (RAT). In RAT, we apply the Prioritized Experience Replay (PER) to improve the convergence of the training procedure. Different from the convex optimization based algorithm which may be susceptible to the initial points and requires iterations, RAT can be adapted to any taking off points of the UAVs and can obtain the solution more rapidly than CAT once training process has been completed. Simulation results show that the proposed CAT and RAT achieve the similar performance and both outperform traditional algorithms.


title: Thermal Brownian Motion of Skyrmion for True Random Number Generation

abstract: The true random number generators (TRNGs) have received extensive attention because of their wide applications in information transmission and encryption. The true random numbers generated by TRNG are typically applied to the encryption algorithm or security protocol of the information security core. Recently, TRNGs have also been employed in emerging stochastic computing paradigm for reducing power consumption. Roughly speaking, TRNG can be divided into circuits-based, e.g., oscillator sampling or directly noise amplifying; and quantum physics-based, e.g., photoelectric effect. The former generally requires a large area and has a large power consumption, whereas the latter is intrinsic random but is more difficult to implement and usually requires additional post-processing circuitry. Very recently, magnetic skyrmion has become a promising candidate for implementing TRNG because of their nanometer size, high stability, and intrinsic thermal Brownian motion dynamics. In this work, we propose a TRNG based on continuous skyrmion thermal Brownian motion in a confined geometry at room temperature. True random bitstream can be easily obtained by periodically detecting the relative position of the skyrmion without the need for additional current pulses. More importantly, we implement a probability-adjustable TRNG, in which a desired ratio of 0 and 1 can be acquired by adding an anisotropy gradient through voltage-controlled magnetic anisotropy (VCMA) effect. The behaviors of the skyrmion-based TRNG are verified by using micromagnetic simulations. The National Institute of Standards and Technology (NIST) test results demonstrate that our proposed random number generator is TRNG with good randomness. Our research provides a new perspective for efficient TRNG realization.


title: Hilbert-Kunz function and Hilbert-Kunz multiplicity of some ideals of the Rees algebra

abstract: We prove that the Hilbert-Kunz function of the ideal $(I,It)$ of the Rees algebra $\mathcal{R}(I)$, where $I$ is an $\mathfrak{m}$-primary ideal of a $1$-dimensional local ring $(R,\mathfrak{m})$, is a quasi-polynomial in $e$, for large $e.$ For $s \in \mathbb{N}$, we calculate the Hilbert-Samuel function of the $R$-module $I^{[s]}$ and obtain an explicit description of the generalized Hilbert-Kunz function of the ideal $(I,It)\mathcal{R}(I)$ when $I$ is a parameter ideal in a Cohen-Macaulay local ring of dimension $d \geq 2$, proving that the generalized Hilbert-Kunz function is a piecewise polynomial in this case.


title: Projective and Reedy model category structures for (infinitesimal) bimodules over an operad

abstract: We construct and study projective and Reedy model category structures for bimodules and infinitesimal bimodules over topological operads. Both model structures produce the same homotopy categories. For the model categories in question, we build explicit cofibrant and fibrant replacements. We show that these categories are right proper and under some conditions left proper. We also study the extension/restriction adjunctions.


title: Social Bias Frames: Reasoning about Social and Power Implications of Language

abstract: Warning: this paper contains content that may be offensive or upsetting. Language has the power to reinforce stereotypes and project social biases onto others. At the core of the challenge is that it is rarely what is stated explicitly, but rather the implied meanings, that frame people's judgments about others. For example, given a statement that "we shouldn't lower our standards to hire more women," most listeners will infer the implicature intended by the speaker -- that "women (candidates) are less qualified." Most semantic formalisms, to date, do not capture such pragmatic implications in which people express social biases and power differentials in language. We introduce Social Bias Frames, a new conceptual formalism that aims to model the pragmatic frames in which people project social biases and stereotypes onto others. In addition, we introduce the Social Bias Inference Corpus to support large-scale modelling and evaluation with 150k structured annotations of social media posts, covering over 34k implications about a thousand demographic groups. We then establish baseline approaches that learn to recover Social Bias Frames from unstructured text. We find that while state-of-the-art neural models are effective at high-level categorization of whether a given statement projects unwanted social bias (80% F1), they are not effective at spelling out more detailed explanations in terms of Social Bias Frames. Our study motivates future work that combines structured pragmatic inference with commonsense reasoning on social implications.


title: INSET: Sentence Infilling with INter-SEntential Transformer

abstract: Missing sentence generation (or sentence infilling) fosters a wide range of applications in natural language generation, such as document auto-completion and meeting note expansion. This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context. Solving the sentence infilling task requires techniques in natural language processing ranging from understanding to discourse-level planning to generation. In this paper, we propose a framework to decouple the challenge and address these three aspects respectively, leveraging the power of existing large-scale pre-trained models such as BERT and GPT-2. We empirically demonstrate the effectiveness of our model in learning a sentence representation for generation and further generating a missing sentence that fits the context.


title: Symmetry and Equivalence in Teleparallel Gravity

abstract: In theories such as teleparallel gravity and its extensions, the frame basis replaces the metric tensor as the primary object of study. A choice of coordinate system, frame basis and spin-connection must be made to obtain a solution from the field equations of a given teleparallel gravity theory. It is worthwhile to express solutions in an invariant manner in terms of torsion invariants to distinguish between different solutions. In this paper we discuss the symmetries of teleparallel gravity theories, describe the classification of the torsion tensor and its covariant derivative and define scalar invariants in terms of the torsion. In particular, we propose a modification of the Cartan-Karlhede algorithm for geometries with torsion (and no curvature or nonmetricity). The algorithm determines the dimension of the symmetry group for a solution and suggests an alternative frame-based approach to calculating symmetries. We prove that the only maximally symmetric solution to any theory of gravitation admitting a non-zero torsion tensor is Minkowski space. As an illustration we apply the algorithm to six particular exact teleparallel geometries. From these examples we notice that the symmetry group of the solutions of a teleparallel gravity theory is potentially smaller than their metric-based analogues in General Relativity.


title: CamemBERT: a Tasty French Language Model

abstract: Pretrained language models are now ubiquitous in Natural Language Processing. Despite their success, most available models have either been trained on English data or on the concatenation of data in multiple languages. This makes practical use of such models --in all languages except English-- very limited. In this paper, we investigate the feasibility of training monolingual Transformer-based language models for other languages, taking French as an example and evaluating our language models on part-of-speech tagging, dependency parsing, named entity recognition and natural language inference tasks. We show that the use of web crawled data is preferable to the use of Wikipedia data. More surprisingly, we show that a relatively small web crawled dataset (4GB) leads to results that are as good as those obtained using larger datasets (130+GB). Our best performing model CamemBERT reaches or improves the state of the art in all four downstream tasks.


title: A Bilingual Generative Transformer for Semantic Sentence Embedding

abstract: Semantic sentence embedding models encode natural language sentences into vectors, such that closeness in embedding space indicates closeness in the semantics between the sentences. Bilingual data offers a useful signal for learning such embeddings: properties shared by both sentences in a translation pair are likely semantic, while divergent properties are likely stylistic or language-specific. We propose a deep latent variable model that attempts to perform source separation on parallel sentences, isolating what they have in common in a latent semantic vector, and explaining what is left over with language-specific latent vectors. Our proposed approach differs from past work on semantic sentence encoding in two ways. First, by using a variational probabilistic framework, we introduce priors that encourage source separation, and can use our model's posterior to predict sentence embeddings for monolingual data at test time. Second, we use high-capacity transformers as both data generating distributions and inference networks -- contrasting with most past work on sentence embeddings. In experiments, our approach substantially outperforms the state-of-the-art on a standard suite of unsupervised semantic similarity evaluations. Further, we demonstrate that our approach yields the largest gains on more difficult subsets of these evaluations where simple word overlap is not a good indicator of similarity.


title: Compelling evidence of oscillatory behaviour of hadronic multiplicities in the shifted Gompertz distribution

abstract: Study of charged particle multiplicity distribution in high energy interactions of particles helps in revealing the dynamics of particle production and the underlying statistical patterns, which these distributions follow. Several distributions derived from statistics have been employed to understand its behaviour. In one of our earlier papers, we introduced the shifted Gompertz distribution to investigate this variable and showed that the multiplicity distributions in a variety of processes at different energies can be very well described by this distribution. The fact that the shifted Gompertz distribution, which has been extensively used in diffusion theory, social networks and forecasting has been used for the first time in high energy physics collisions, remains interesting. In this paper we investigate the phenomenon of oscillatory behaviour of the counting statistics observed in the experimental data, resulting from different types of recurrence relations defining the probability distributions. We search for such oscillations in the multiplicity distributions well described by the shifted Gompertz distribution and compare our results with the analysis proposed by G. Wilk et al.


title: Two-Headed Monster And Crossed Co-Attention Networks

abstract: This paper presents some preliminary investigations of a new co-attention mechanism in neural transduction models. We propose a paradigm, termed Two-Headed Monster (THM), which consists of two symmetric encoder modules and one decoder module connected with co-attention. As a specific and concrete implementation of THM, Crossed Co-Attention Networks (CCNs) are designed based on the Transformer model. We demonstrate CCNs on WMT 2014 EN-DE and WMT 2016 EN-FI translation tasks and our model outperforms the strong Transformer baseline by 0.51 (big) and 0.74 (base) BLEU points on EN-DE and by 0.17 (big) and 0.47 (base) BLEU points on EN-FI.


title: Understanding Multi-Head Attention in Abstractive Summarization

abstract: Attention mechanisms in deep learning architectures have often been used as a means of transparency and, as such, to shed light on the inner workings of the architectures. Recently, there has been a growing interest in whether or not this assumption is correct. In this paper we investigate the interpretability of multi-head attention in abstractive summarization, a sequence-to-sequence task for which attention does not have an intuitive alignment role, such as in machine translation. We first introduce three metrics to gain insight in the focus of attention heads and observe that these heads specialize towards relative positions, specific part-of-speech tags, and named entities. However, we also find that ablating and pruning these heads does not lead to a significant drop in performance, indicating redundancy. By replacing the softmax activation functions with sparsemax activation functions, we find that attention heads behave seemingly more transparent: we can ablate fewer heads and heads score higher on our interpretability metrics. However, if we apply pruning to the sparsemax model we find that we can prune even more heads, raising the question whether enforced sparsity actually improves transparency. Finally, we find that relative positions heads seem integral to summarization performance and persistently remain after pruning.


title: Charming loops in exclusive rare FCNC $B$-decays

abstract: Rare $B$-decays induced by flavour-changing neutral currents (FCNC) is one of the promising candidates for probing physics beyond the Standard model. However, for identifying potential new physics from the data, reliable control over QCD contributions is necessary. We focus on one of such QCD contributions - the charming loops - that potentially can lead to difficulties in disentangling new physics effects from the observables and discuss the possibility to gain control over theoretical predictions for charming loops.


title: Second order pressure estimates for the Crank-Nicolson discretization of the incompressible Navier-Stokes Equations

abstract: We provide optimal order pressure error estimates for the Crank-Nicolson semidiscretization of the incompressible Navier-Stokes equations. Second order estimates for the velocity error are long known, we prove that the pressure error is of the same order if considered at interval midpoints, confirming previous numerical evidence. For simplicity we first give a proof under high regularity assumptions that include nonlocal compatibility conditions for the initial data, then use smoothing techniques for a proof under reduced assumptions based on standard local conditions only.


title: Late time cosmic acceleration in modified S\'{a}ez-Ballester theory

abstract: We establish an extended version of the modified S\'{a}ez-Ballester (SB) scalar-tensor theory in arbitrary dimensions whose energy momentum tensor as well as potential are pure geometrical quantities. This scenario emerges by means of two scalar fields (one is present in the SB theory and the other is associated with the extra dimension) which widens the scope of the induced-matter theory. Moreover, it bears a close resemblance to the standard S\'{a}ez-Ballester scalar-tensor theory, as well as other alternative theories to general relativity, whose construction includes either a minimally or a non-minimally coupled scalar field. However, contrary to those theories, in our framework the energy momentum tensor and the scalar potential are not added by hand, but instead are dictated from the geometry. Concerning cosmological applications, our herein contribution brings a new perspective. We firstly show that the dark energy sector can be naturally retrieved within a strictly geometric perspective, and we subsequently analyze it. Moreover, our framework may provide a hint to understand the physics of lower gravity theories.


title: Classification of 3-dimensional complete rectifiable steady and expanding gradient Ricci solitons

abstract: Let $(M,g,f)$ be a 3-dimensional complete steady gradient Ricci soliton. Assume that $M$ is rectifiable, that is, the potential function can be written as $f=f(r)$, where $r$ is a distance function. Then, we prove that $M$ is isometric to (1) a quotient of $\mathbb{R}^3$, or (2) the Bryant soliton. In particular, we show that any 3-dimensional complete rectifiable steady gradient Ricci soliton with positive Ricci curvature is isometric to the Bryant soliton. Furthermore, we show that any $3$-dimensional complete rectifiable expanding gradient Ricci soliton with positive Ricci curvature is rotationally symmetric.


title: A Re-evaluation of Knowledge Graph Completion Methods

abstract: Knowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs. A vast number of state-of-the-art KGC techniques have got published at top conferences in several research fields, including data mining, machine learning, and natural language processing. However, we notice that several recent papers report very high performance, which largely outperforms previous state-of-the-art methods. In this paper, we find that this can be attributed to the inappropriate evaluation protocol used by them and propose a simple evaluation protocol to address this problem. The proposed protocol is robust to handle bias in the model, which can substantially affect the final results. We conduct extensive experiments and report the performance of several existing methods using our protocol. The reproducible code has been made publicly available


title: HighwayGraph: Modelling Long-distance Node Relations for Improving General Graph Neural Network

abstract: Graph Neural Networks (GNNs) are efficient approaches to process graph-structured data. Modelling long-distance node relations is essential for GNN training and applications. However, conventional GNNs suffer from bad performance in modelling long-distance node relations due to limited-layer information propagation. Existing studies focus on building deep GNN architectures, which face the over-smoothing issue and cannot model node relations in particularly long distance. To address this issue, we propose to model long-distance node relations by simply relying on shallow GNN architectures with two solutions: (1) Implicitly modelling by learning to predict node pair relations (2) Explicitly modelling by adding edges between nodes that potentially have the same label. To combine our two solutions, we propose a model-agnostic training framework named HighwayGraph, which overcomes the challenge of insufficient labeled nodes by sampling node pairs from the training set and adopting the self-training method. Extensive experimental results show that our HighwayGraph achieves consistent and significant improvements over four representative GNNs on three benchmark datasets.


title: Semantic Noise Matters for Neural Natural Language Generation

abstract: Neural natural language generation (NNLG) systems are known for their pathological outputs, i.e. generating text which is unrelated to the input specification. In this paper, we show the impact of semantic noise on state-of-the-art NNLG models which implement different semantic control mechanisms. We find that cleaned data can improve semantic correctness by up to 97%, while maintaining fluency. We also find that the most common error is omitting information, rather than hallucination.


title: Efficient Dialogue State Tracking by Selectively Overwriting Memory

abstract: Recent works in dialogue state tracking (DST) focus on an open vocabulary-based setting to resolve scalability and generalization issues of the predefined ontology-based approaches. However, they are inefficient in that they predict the dialogue state at every turn from scratch. Here, we consider dialogue state as an explicit fixed-sized memory and propose a selectively overwriting mechanism for more efficient DST. This mechanism consists of two steps: (1) predicting state operation on each of the memory slots, and (2) overwriting the memory with new values, of which only a few are generated according to the predicted state operations. Our method decomposes DST into two sub-tasks and guides the decoder to focus only on one of the tasks, thus reducing the burden of the decoder. This enhances the effectiveness of training and DST performance. Our SOM-DST (Selectively Overwriting Memory for Dialogue State Tracking) model achieves state-of-the-art joint goal accuracy with 51.72% in MultiWOZ 2.0 and 53.01% in MultiWOZ 2.1 in an open vocabulary-based DST setting. In addition, we analyze the accuracy gaps between the current and the ground truth-given situations and suggest that it is a promising direction to improve state operation prediction to boost the DST performance.


title: Universal averages in gauge actions

abstract: We give a construction of a universal average of Lie algebra elements whose exponentiation gives (when there is an associated Lie group) a totally symmetric geometric mean of Lie group elements (sufficiently closed to the identity) with the property that in an action of the group on a space $X$ for which $n$ elements all take a particular point $a\in{}X$ to a common point $b\in{}X$, also the mean will take $a$ to $b$. The construction holds without the necessity for the existence of a Lie group and the universal average $\mu_n(x_1,\ldots,x_n)$ is a totally symmetric universal expression in the free Lie algebra generated by $x_1,\ldots,x_n$. Its expansion up to three brackets is found explicitly and various properties of iterated averages are given. There are applications to the construction of explicit symmetric differential graded Lie algebra models. This work is based on the second author's minor thesis.


title: Existence of radial bounded solutions for some quasilinear elliptic equations in R^N

abstract: We study the quasilinear equation \[(P)\qquad - {\rm div} (A(x,u) |\nabla u|^{p-2} \nabla u) + \frac1p\ A_t(x,u) |\nabla u|^p + |u|^{p-2}u\ =\ g(x,u) \qquad \hbox{in ${\mathbb R}^N$,} \] with $N\ge 3$, $p > 1$, where $A(x,t)$, $A_t(x,t) = \frac{\partial A}{\partial t}(x,t)$ and $g(x,t)$ are Carath\'eodory functions on ${\mathbb R}^N \times {\mathbb R}$. Suitable assumptions on $A(x,t)$ and $g(x,t)$ set off the variational structure of $(P)$ and its related functional ${\cal J}$ is $C^1$ on the Banach space $X = W^{1,p}({\mathbb R}^N) \cap L^\infty({\mathbb R}^N)$. In order to overcome the lack of compactness, we assume that the problem has radial symmetry, then we look for critical points of ${\cal J}$ restricted to $X_r$, subspace of the radial functions in $X$. Following an approach which exploits the interaction between $\|\cdot\|_X$ and the norm on $W^{1,p}({\mathbb R}^N)$, we prove the existence of at least one weak bounded radial solution of $(P)$ by applying a generalized version of the Ambrosetti-Rabinowitz Mountain Pass Theorem.


title: Spontaneous Generation of delta-sunspots in Convective Magnetohydrodynamic Simulation of Magnetic Flux Emergence

abstract: Observations reveal that strong solar flares and coronal mass ejections tend to occur in complex active regions characterized by delta-sunspots, spot rotation, sheared polarity inversion lines (PILs), and magnetic flux ropes. Here we report on the first modeling of spontaneous delta-spot generation as a result of flux emergence from the turbulent convection zone. Utilizing state-of-the-art radiative magnetohydrodynamics code R2D2, we simulate the emergence of a force-free flux tube in the convection zone that stretches down to -140 Mm. Elevated by large-scale convective upflows, the tube appears on the photosphere as two emerging bipoles. The opposite polarities collide against each other due to the subsurface connectivity, and they develop into a pair of closely-packed delta-spots. The Lorentz force drives the spot rotation and a strong counter-streaming flow of 10 km/s at the PIL in delta-spots, which, in tandem with local convection, strengthens the horizontal field to 4 kG and builds up a highly-sheared PIL. In the atmosphere above the PIL, a flux rope structure is created. All these processes follow the multi-buoyant segment theory of the delta-spot formation, and they occur as a natural consequence of interaction between magnetic flux and turbulent convection, suggesting that the generation of delta-spots and the resultant flare eruptions may be a stochastically determined process.


title: Quasilinear problems without the Ambrosetti-Rabinowitz condition

abstract: We show the existence of nontrivial solutions for a class of highly quasilinear problems in which the governing operators depend on the unknown function. By using a suitable variational setting and a weak version of the Cerami-Palais-Smale condition, we establish the desired result without assuming that the nonlinear source satisfies the Ambrosetti-Rabinowitz condition.


title: Contract Discovery: Dataset and a Few-Shot Semantic Retrieval Challenge with Competitive Baselines

abstract: We propose a new shared task of semantic retrieval from legal texts, in which a so-called contract discovery is to be performed, where legal clauses are extracted from documents, given a few examples of similar clauses from other legal acts. The task differs substantially from conventional NLI and shared tasks on legal information extraction (e.g., one has to identify text span instead of a single document, page, or paragraph). The specification of the proposed task is followed by an evaluation of multiple solutions within the unified framework proposed for this branch of methods. It is shown that state-of-the-art pretrained encoders fail to provide satisfactory results on the task proposed. In contrast, Language Model-based solutions perform better, especially when unsupervised fine-tuning is applied. Besides the ablation studies, we addressed questions regarding detection accuracy for relevant text fragments depending on the number of examples available. In addition to the dataset and reference results, LMs specialized in the legal domain were made publicly available.


title: Effectiveness of self-supervised pre-training for speech recognition

abstract: We compare self-supervised representation learning algorithms which either explicitly quantize the audio data or learn representations without quantization. We find the former to be more accurate since it builds a good vocabulary of the data through vq-wav2vec [1] to enable learning of effective representations in subsequent BERT training. Different to previous work, we directly fine-tune the pre-trained BERT models on transcribed speech using a Connectionist Temporal Classification (CTC) loss instead of feeding the representations into a task-specific model. We also propose a BERT-style model learning directly from the continuous audio data and compare pre-training on raw audio to spectral features. Fine-tuning a BERT model on 10 hour of labeled Librispeech data with a vq-wav2vec vocabulary is almost as good as the best known reported system trained on 100 hours of labeled data on testclean, while achieving a 25% WER reduction on test-other. When using only 10 minutes of labeled data, WER is 25.2 on test-other and 16.3 on test-clean. This demonstrates that self-supervision can enable speech recognition systems trained on a near-zero amount of transcribed data.


title: Can Monolingual Pretrained Models Help Cross-Lingual Classification?

abstract: Multilingual pretrained language models (such as multilingual BERT) have achieved impressive results for cross-lingual transfer. However, due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. In this work, we present two approaches to improve zero-shot cross-lingual classification, by transferring the knowledge from monolingual pretrained models to multilingual ones. Experimental results on two cross-lingual classification benchmarks show that our methods outperform vanilla multilingual fine-tuning.


title: Zero-Shot Fine-Grained Style Transfer: Leveraging Distributed Continuous Style Representations to Transfer To Unseen Styles

abstract: Text style transfer is usually performed using attributes that can take a handful of discrete values (e.g., positive to negative reviews). In this work, we introduce an architecture that can leverage pre-trained consistent continuous distributed style representations and use them to transfer to an attribute unseen during training, without requiring any re-tuning of the style transfer model. We demonstrate the method by training an architecture to transfer text conveying one sentiment to another sentiment, using a fine-grained set of over 20 sentiment labels rather than the binary positive/negative often used in style transfer. Our experiments show that this model can then rewrite text to match a target sentiment that was unseen during training.


title: Generation of massive stellar black holes by rapid gas accretion in primordial dense clusters

abstract: Supernova theory suggests that black holes of a stellar origin cannot attain masses in the range of 50-135 solar masses in isolation. We argue here that this mass gap is filled in by black holes that grow by gas accretion in dense stellar clusters, such as protoglobular clusters. The accretion proceeds rapidly, during the first 10 megayears of the cluster life, before the remnant gas is depleted. We predict that binaries of black holes within the mass gap can be observed by LIGO.


title: Intelligent Reflecting Surface with Discrete Phase Shifts: Channel Estimation and Passive Beamforming

abstract: In this paper, we consider an intelligent reflecting surface (IRS)-aided single-user system where an IRS with discrete phase shifts is deployed to assist the uplink communication. A practical transmission protocol is proposed to execute channel estimation and passive beamforming successively. To minimize the mean square error (MSE) of channel estimation, we first formulate an optimization problem for designing the IRS reflection pattern in the training phase under the constraints of unit-modulus, discrete phase, and full rank. This problem, however, is NP-hard and thus difficult to solve in general. As such, we propose a low-complexity yet efficient method to solve it sub-optimally, by constructing a near-orthogonal reflection pattern based on either discrete Fourier transform (DFT)-matrix quantization or Hadamard-matrix truncation. Based on the estimated channel, we then formulate an optimization problem to maximize the achievable rate by designing the discrete-phase passive beamforming at the IRS with the training overhead and channel estimation error taken into account. To reduce the computational complexity of exhaustive search, we further propose a low-complexity successive refinement algorithm with a properly-designed initialization to obtain a high-quality suboptimal solution. Numerical results are presented to show the significant rate improvement of our proposed IRS training reflection pattern and passive beamforming designs as compared to other benchmark schemes.


title: Stochastic Computing Implemented by Skyrmionic Logic Devices

abstract: Magnetic skyrmion, topologically non-trivial spin texture, has been considered as promising information carrier in future electronic devices because of its nanoscale size, low depinning current density and high motion velocity. Despite the broad interests in skyrmion racetrack memory, researchers have been recently exploiting logic functions enabled by using the particle-like behaviors of skyrmions. These functions can be applied to unconventional computing, such as stochastic computing (SC), which treats data as probabilities and is superior to binary computing due to its simplicity of logic operation. In this work, we demonstrate SC implemented by skyrmionic logic devices. We propose a skyrmionic AND-OR logic device as a multiplier in the stochastic domain and two skyrmionic multiplexer (MUX) logic devices as stochastic adders. With the assist of voltage controlled magnetic anisotropy (VCMA), the precise control of skyrmions collision is not required in the skyrmionic AND-OR logic device, thus improving the operation robustness. In the two MUX logic devices, skyrmions can be driven by Zhang-Li torque or spin orbit torque (SOT). Particularly, we can flexibly regulate the skyrmion motion by VCMA or voltage controlled Dzyaloshinskii-Moriya Interaction (VCDMI) in the SOT case. Furthermore, 3-bit stochastic multiplier and adder are demonstrated by micromagnetic simulations. In addition, simulations in synthetic antiferromagnets (SAF) show that the performance of our skyrmionic logic gates can be optimized through advanced materials. Our work opens up perspective to implement SC using skyrmionic logic devices.


title: Improving BERT Fine-tuning with Embedding Normalization

abstract: Large pre-trained sentence encoders like BERT start a new chapter in natural language processing. A common practice to apply pre-trained BERT to sequence classification tasks (e.g., classification of sentences or sentence pairs) is by feeding the embedding of [CLS] token (in the last layer) to a task-specific classification layer, and then fine tune the model parameters of BERT and classifier jointly. In this paper, we conduct systematic analysis over several sequence classification datasets to examine the embedding values of [CLS] token before the fine tuning phase, and present the biased embedding distribution issue---i.e., embedding values of [CLS] concentrate on a few dimensions and are non-zero centered. Such biased embedding brings challenge to the optimization process during fine-tuning as gradients of [CLS] embedding may explode and result in degraded model performance. We further propose several simple yet effective normalization methods to modify the [CLS] embedding during the fine-tuning. Compared with the previous practice, neural classification model with the normalized embedding shows improvements on several text classification tasks, demonstrates the effectiveness of our method.


title: Relations for Massive Spinors

abstract: Recently introduced massive spinors are written as 2-vectors consisting of two massless spinors with opposite helicities. With this notation a couple of relations between them can be derived easily, entirely avoiding the spinor indices. The high energy limit of three point amplitudes is discussed shortly. Finally we add some comments on recursion relations with massive particles.


title: Rigidity of Steiner's inequality for the anisotropic perimeter

abstract: The aim of this work is to study the rigidity problem for Steiner's inequality for the anisotropic perimeter, that is, the situation in which the only extremals of the inequality are vertical translations of the Steiner symmetral that we are considering. Our main contribution consists in giving conditions under which rigidity in the anisotropic setting is equivalent to rigidity in the Euclidean setting. Such conditions are given in term of a restriction to the possible values of the normal vectors to the boundary of the Steiner symmetral (see Corollary 1.17, and Corollary 1.18).


title: An optical n-body gravitational lens analogy

abstract: Raised menisci around small discs positioned to pull up a water-air interface provide a well controllable experimental setup capable of reproducing much of the rich phenomenology of gravitational lensing (or microlensing events) by $n$-body clusters. Results are shown for single, binary and triple mass lenses. The scheme represents a versatile testbench for the (astro)physics of general relativity's gravitational lens effects, including high multiplicity imaging of extended sources.


title: Un systeme de lemmatisation pour les applications de TALN

abstract: This paper presents a method of stemming for the Arabian texts based on the linguistic techniques of the natural language processing. This method leans on the notion of scheme (one of the strong points of the morphology of the Arabian language). The advantage of this approach is that it doesn't use a dictionary of inflexions but a smart dynamic recognition of the different words of the language.


title: A Dynamic Modelling Framework for Human Hand Gesture Task Recognition

abstract: Gesture recognition and hand motion tracking are important tasks in advanced gesture based interaction systems. In this paper, we propose to apply a sliding windows filtering approach to sample the incoming streams of data from data gloves and a decision tree model to recognize the gestures in real time for a manual grafting operation of a vegetable seedling propagation facility. The sequence of these recognized gestures defines the tasks that are taking place, which helps to evaluate individuals' performances and to identify any bottlenecks in real time. In this work, two pairs of data gloves are utilized, which reports the location of the fingers, hands, and wrists wirelessly (i.e., via Bluetooth). To evaluate the performance of the proposed framework, a preliminary experiment was conducted in multiple lab settings of tomato grafting operations, where multiple subjects wear the data gloves while performing different tasks. Our results show an accuracy of 91% on average, in terms of gesture recognition in real time by employing our proposed framework.


title: Dixmier traces for discrete pseudo-differential operators

abstract: In this paper we provide sharp results for the Dixmier traceability of discrete pseudo-differential operators on $\ell^2(\mathbb{Z}^n)$. In this setting, we introduce a suitable notion of a class of classical symbols which provide a class of Dixmier traceable discrete pseudo-differential operators. We also present a formula for the Dixmier trace of a Dixmier traceable discrete pseudo-differential operator by using the Connes equivalence between the Wodzicki residue and the Dixmier trace.


title: Symmetrical Gaussian Error Linear Units (SGELUs)

abstract: In this paper, a novel neural network activation function, called Symmetrical Gaussian Error Linear Unit (SGELU), is proposed to obtain high performance. It is achieved by effectively integrating the property of the stochastic regularizer in the Gaussian Error Linear Unit (GELU) with the symmetrical characteristics. Combining with these two merits, the proposed unit introduces the capability of the bidirection convergence to successfully optimize the network without the gradient diminishing problem. The evaluations of SGELU against GELU and Linearly Scaled Hyperbolic Tangent (LiSHT) have been carried out on MNIST classification and MNIST auto-encoder, which provide great validations in terms of the performance, the convergence rate among these applications.


title: Gemini: A Functional Programming Language for Hardware Description

abstract: This paper presents Gemini, a functional programming language for hardware description that provides features such as parametric polymorphism, recursive datatypes, higher-order functions, and type inference for higher expressivity compared to modern hardware description languages. Gemini demonstrates the theory and implementation of novel type-theoretical concepts through its unique type system consisting of multiple atomic kinds and dependent types, which allows the language to model both software and hardware constructs safely and perform type inference through multi-staged compilation. The primary technical results of this paper include formalizations of the Gemini grammar, typing rules, and evaluation rules, a proof of safety of Gemini's type system, and a prototype implementation of the compiler's semantic analysis phase.


title: Cooperative Pathfinding based on memory-efficient Multi-agent RRT*

abstract: In cooperative pathfinding problems, no-conflicts paths that bring several agents from their start location to their destination need to be planned. This problem can be efficiently solved by Multi-agent RRT*(MA-RRT*) algorithm, which is still state-of-the-art in the field of coupled methods. However, the implementation of this algorithm is hindered in systems with limited memory because the number of nodes in the tree grows indefinitely as the paths get optimized. This paper proposes an improved version of MA-RRT*, called Multi-agent RRT* Fixed Node(MA-RRT*FN), which limits the number of nodes stored in the tree by removing the weak nodes on the path which are not likely to reach the goal. The results show that MA-RRT*FN performs close to MA-RRT* in terms of scalability and solution quality while the memory required is much lower and fixed.


title: Rigidity and non-existence of spacelike submanifolds with causal mean curvature vector field in spacetimes and the Cauchy problem in General Relativity

abstract: New general results of non-existence and rigidity of spacelike submanifolds immersed in a spacetime, whose mean curvature is a time-oriented causal vector field, are given. These results hold for a wide class of spacetimes which includes globally hyperbolic, stationary, conformally stationary and pp-wave spacetimes, among others. Moreover, applications to the Cauchy problem in General Relativity, are presented. Finally, in the case of hypersurfaces, we also obtain significant consequences in Geometrical Analysis, solving new Calabi-Bernstein and Dirichlet problems on a Riemannian manifold.


title: Positioning of Multiple Unmanned Aerial Vehicle Base Stations in future Wireless Network

abstract: Unmanned aerial vehicle (UAV) base stations (BSs) are reliable and efficient alternative to full fill the coverage and capacity requirements when the backbone network fails to provide such requirements due to disasters. In this paper, we consider optimal UAV-deployment problem in 3D space for a mmWave network. The objective is to deploy multiple aerial BSs simultaneously to completely serve the ground users. We develop a novel algorithm to find the feasible positions for a set of UAV-BSs from a predefined set of locations, subject to a signal-to-interference-plus-noise ratio (SINR) constraint of every associated user, UAV-BS's limited hovering altitude constraint and restricted operating zone constraint. We cast this 3D positioning problem as an l_0 minimization problem. This is a combinatorial, NP-hard problem. We approximate the l_0 minimization problem as non-combinatorial l_1-norm problem. Therefore, we provide a suboptimal algorithm to find a set of feasible locations for the UAV-BSs to operate. The analysis shows that the proposed algorithm achieves a set of the location to deploy multiple UVA-BSs simultaneously while satisfying the constraints.


title: Robust Unsupervised Audio-visual Speech Enhancement Using a Mixture of Variational Autoencoders

abstract: Recently, an audio-visual speech generative model based on variational autoencoder (VAE) has been proposed, which is combined with a nonnegative matrix factorization (NMF) model for noise variance to perform unsupervised speech enhancement. When visual data is clean, speech enhancement with audio-visual VAE shows a better performance than with audio-only VAE, which is trained on audio-only data. However, audio-visual VAE is not robust against noisy visual data, e.g., when for some video frames, speaker face is not frontal or lips region is occluded. In this paper, we propose a robust unsupervised audio-visual speech enhancement method based on a per-frame VAE mixture model. This mixture model consists of a trained audio-only VAE and a trained audio-visual VAE. The motivation is to skip noisy visual frames by switching to the audio-only VAE model. We present a variational expectation-maximization method to estimate the parameters of the model. Experiments show the promising performance of the proposed method.


title: Anti-diffusive, non-oscillatory central difference scheme (adNOC) suitable for highly nonlinear advection-dominated problems

abstract: Explicit non-oscillatory central difference schemes become excessively diffusive when applied to highly nonlinear advection problems where small time steps are necessary to maintain stability. Here, we present a correction to reduce such numerical dissipation for this class of problems. The correction is obtained by selecting the appropriate finite difference approximations for calculating the slopes utilized to reconstruct the solution from the cell averages. The anti-diffusive central scheme does not require any knowledge of the eigenstructure and is fully central. The proposed correction is applied to the widely used Nessyahu-Tadmor scheme to demonstrate the utility of the correction. The stability of the corrected scheme is discussed and the condition for the scheme to become TVD (total variation diminishing) is presented. The corrected scheme is finally tested with a number of test cases and the results are compared with analytical solutions and published results showing the ability of the corrected scheme to effectively resolve sharp discontinuities.


title: Inertial Manifolds and Limit Cycles of Dynamical Systems in $\mathbb R^n$

abstract: We show that the presence of a two-dimensional inertial manifold for an ordinary differential equation in ${\mathbb R}^{n}$ permits reducing the problem of determining asymptotically orbitally stable limit cycles to the Poincare--Bendixson theory. In the case $n=3$ we implement such a scenario for a model of a satellite rotation around a celestial body of small mass and for a biochemical model.


title: Anti-diffusive, Non-oscillatory Central (adNOC) scheme to solve the shallow water equations over an erodible substrate in two dimensions

abstract: Shallow water surface flows commonly entrain sediments, resulting in scouring and/or deposition of the underlying substrate that may strongly influence the pattern of subsequent flow. These coupled phenomena, which can be investigated mathematically with some extension of the shallow water equations, present numerous challenges for numerical methods. Here, we present a straightforward Riemann-solver free approach to solve these equations based on the explicit non-oscillatory central (NOC) scheme that has already been widely applied to hyperbolic conservation laws in other contexts. Our version of the central scheme is second-order accurate in time and space and is used with an anti-diffusive correction to reduce numerical diffusion usually suffered by central schemes. Numerical experiments show that the scheme is accurate and robust for a range of applications from highly dynamic spontaneous dam break over a mobile bed to slowly evolving morphological bed in an alluvial river.


title: Evaluating Voice Conversion-based Privacy Protection against Informed Attackers

abstract: Speech data conveys sensitive speaker attributes like identity or accent. With a small amount of found data, such attributes can be inferred and exploited for malicious purposes: voice cloning, spoofing, etc. Anonymization aims to make the data unlinkable, i.e., ensure that no utterance can be linked to its original speaker. In this paper, we investigate anonymization methods based on voice conversion. In contrast to prior work, we argue that various linkage attacks can be designed depending on the attackers' knowledge about the anonymization scheme. We compare two frequency warping-based conversion methods and a deep learning based method in three attack scenarios. The utility of converted speech is measured via the word error rate achieved by automatic speech recognition, while privacy protection is assessed by the increase in equal error rate achieved by state-of-the-art i-vector or x-vector based speaker verification. Our results show that voice conversion schemes are unable to effectively protect against an attacker that has extensive knowledge of the type of conversion and how it has been applied, but may provide some protection against less knowledgeable attackers.


title: Eddington-inspired-Born-Infeld tensorial instabilities neutralized in a quantum approach

abstract: The recent direct detection of gravitational waves has highlighted the huge importance of the tensorial modes in any extended gravitational theory. One of the most appealing approaches to extend gravity beyond general relativity is the Eddington-inspired-Born-Infeld gravity which is formulated within the Palatini approach. This theory can avoid the Big Bang singularity in the physical metric although a Big Bang intrinsic to the affine connection is still there, which in addition couples to the tensorial sector and might jeopardize the viability of the model. In this paper, we suggest that a quantum treatment of the affine connection, or equivalently of its compatible metric, is able to rescue the model. We carry out such an analysis and conclude that from a quantum point of view such a Big Bang is unharmful. We expect therefore that the induced tensorial instability, caused by the Big Bang intrinsic to the affine connection, can be neutralized at the quantum level.


title: Can Neural Image Captioning be Controlled via Forced Attention?

abstract: Learned dynamic weighting of the conditioning signal (attention) has been shown to improve neural language generation in a variety of settings. The weights applied when generating a particular output sequence have also been viewed as providing a potentially explanatory insight into the internal workings of the generator. In this paper, we reverse the direction of this connection and ask whether through the control of the attention of the model we can control its output. Specifically, we take a standard neural image captioning model that uses attention, and fix the attention to pre-determined areas in the image. We evaluate whether the resulting output is more likely to mention the class of the object in that area than the normally generated caption. We introduce three effective methods to control the attention and find that these are producing expected results in up to 28.56% of the cases.


title: Language Model-Driven Unsupervised Neural Machine Translation

abstract: Unsupervised neural machine translation(NMT) is associated with noise and errors in synthetic data when executing vanilla back-translations. Here, we explicitly exploits language model(LM) to drive construction of an unsupervised NMT system. This features two steps. First, we initialize NMT models using synthetic data generated via temporary statistical machine translation(SMT). Second, unlike vanilla back-translation, we formulate a weight function, that scores synthetic data at each step of subsequent iterative training; this allows unsupervised training to an improved outcome. We present the detailed mathematical construction of our method. Experimental WMT2014 English-French, and WMT2016 English-German and English-Russian translation tasks revealed that our method outperforms the best prior systems by more than 3 BLEU points.


title: Examining the weak cosmic censorship conjecture of RN-AdS black holes via the new version of the gedanken experiment

abstract: Based on the new version of the gedanken experiment proposed by Sorce and Wald, we investigate the weak cosmic censorship conjecture (WCCC) for a Reissner-Nordstr\"{o}m-Anti-de Sitter (RN-AdS) black hole under the perturbation of extra matter fields. Firstly, we propose that the cosmological constant can be effectively derived from the matter fields and its value varies with the matter fields perturbing the black hole. Meanwhile, we assume that the perturbation satisfies the stability condition. This condition means that after a long time of the perturbation, the black hole solution also belongs to the family of the RN-AdS solution. After that, based on both the stability condition and the null energy condition, while using the off-shell variation method, the first-order and the second-order perturbation inequalities are derived respectively when the cosmological constant is considered as a dynamic variable. It is the first time to extend the two perturbation inequalities to contain the term of the press and volume of thermodynamics. Finally, we perform the two perturbation inequalities into testing the WCCC for the RN-AdS black hole under the second-order approximation of the perturbation. It is shown that if the variation of the cosmological constant is caused by the matter fields, while the stability condition and the null energy condition are all satisfied, the black hole cannot be destroyed after the perturbation. In other words, the WCCC for the RN-AdS black hole is valid under the second-order approximation of the perturbation.


title: Hopf algebras arising from partial (co)actions

abstract: In this paper, extending the idea presented by M. Takeuchi in [13], we introduce the notion of partial matched pair $(H,L)$ involving the concepts of partial action and partial coaction between two Hopf algebras $H$ and $L$. Furthermore, we present necessary conditions for the corresponding bismash product $L\# H$ to generate a new Hopf algebra and, as illustration, a family of examples is provided.


title: SLTR: Simultaneous Localization of Target and Reflector in NLOS Condition Using Beacons

abstract: When the direct view between the target and the observer is not available, due to obstacles with non-zero sizes, the observation is received after reflection from a reflector, this is the indirect view or Non-Line-Of Sight condition. Localization of a target in NLOS condition still one of the open problems yet. In this paper, we address this problem by localizing the reflector and the target simultaneously using a single stationary receiver, and a determined number of beacons, in which their placements are also analyzed in an unknown map. The work is done in mirror space, when the receiver is a camera, and the reflector is a planar mirror. Furthermore, the distance from the observer to the target is estimated by size constancy concept, and the angle of coming signal is the same as the orientation of the camera, with respect to a global frame. The results show the validation of the proposed work and the simulation results are matched with the theoretical results.


title: Using LSTMs for climate change assessment studies on droughts and floods

abstract: Climate change affects occurrences of floods and droughts worldwide. However, predicting climate impacts over individual watersheds is difficult, primarily because accurate hydrological forecasts require models that are calibrated to past data. In this work we present a large-scale LSTM-based modeling approach that -- by training on large data sets -- learns a diversity of hydrological behaviors. Previous work shows that this model is more accurate than current state-of-the-art models, even when the LSTM-based approach operates out-of-sample and the latter in-sample. In this work, we show how this model can assess the sensitivity of the underlying systems with regard to extreme (high and low) flows in individual watersheds over the continental US.


title: Question about integral of product of four Hermite polynomials integrated with squared weight

abstract: We found that the integral of four Hermite polynomials integrated with squared weight over the real line generates symmetric polynomials with a beautiful recursive property. We pose a question whether that integral admit an explicit formula or not. The question has a certain interpretation in terms of Hermitian matrices.


title: Analysis of Ownership and Travel Behavior of Women Who Drive Electric Vehicles: The case of Maryland

abstract: This study aims to investigate the contributing socio-demographic characteristics and factors among female EV owners as well as their travel behavior, commuting trip patterns, and purchasing/leasing ownerships. The objective of the study is to recommend public policies to decision makers to prompt gender equity for EV purchase and use by identifying socio-demographic attributes that influence EV travel patterns and behavior.An online survey of EV owners of both sexes was conducted from May 28, 2015, to February 19, 2016. In total, 1,257 EV owners in Maryland completed usable surveys. A set of statistical analysis methods was employed to analyze the data. A set of correlation test, analysis of variance (ANOVA) and multinomial logit model (MNL) were constructed to examine the associations between EV owner characteristics and their EV purchasing/leasing and spatial commuting trip behavior, and the results have been compared to the spatial travel patterns of drivers of internal combustion engine vehicles (ICEVs) in Maryland. The results of this study showed that first, socio-demographic factors, including education and income, played a significant role in preferences attributes of participants for purchasing/leasing an EV and in the commuting travel behavior and pattern and of EV drivers. Second, environmental issues are the main reason for purchasing/leasing EVs, but the EV owners who had longer commutes were more concerned about the price and status of the EV owner and efficiency and performance than were those with shorter commutes. The results of this study improve equity in transportation into the future and could be utilized by transportation authorities, transport investment agencies, and collaborators in developing and improving emerging transportation systems.


title: Coercivity for travelling waves in the Gross-Pitaevskii equation in $\mathbb{R}^2$ for small speed

abstract: In the previous paper, we constructed a smooth branch of travelling waves for the 2 dimensional Gross-Pitaevskii equation. Here, we continue the study of this branch. We show some coercivity results, and we deduce from them the kernel of the linearized operator, a spectral stability result, as well as a uniqueness result in the energy space. In particular, our result proves the non degeneracy of these travelling waves, which is a key step in the classification of these waves and for the construction of multi-travelling waves.


title: Dark photon manifestation in the triplet-like QED processes $\gamma + \ell_i \to \ell^+_j \ell^-_j + \ell_i$, $i\neq j,$ $i=e, \mu,$ $j=e, \mu,\tau$

abstract: The triplet-like QED processes $\gamma + \ell_i \to \ell^+_j \ell^-_j + \ell_i$ with $i\neq j,$ and $i=e, \mu,$ $j=e, \mu,\tau$ has been investigated as the reactions where a dark photon, $A'$, is produced as a virtual state with subsequent decay into a $\ell^+_j \ell^-_j-$-pair. This effect arises due to the so-called kinetic mixing and is characterized by the small parameter $\epsilon$ describing the coupling strength relative to the electric charge $e$. The main advantage of searching $A'$ in these processes is that the background to the $A'$ signal is pure QED. Concerning $A'$, we consider its contribution in the Compton-type diagrams only since, in this case, the virtual dark photon has time-like nature and its propagator has the Breit-Wigner form. So, near the resonance, $A'$ can manifest itself. The contribution of $A'$ in the Borsellino diagrams is negligible since, in this case, the virtual dark photon is space-like, the $A'$ propagator does not peak and the effect is proportional, at least, to $\epsilon^2$. We calculate the distributions over the invariant mass of the produced $\ell^+_j \ell^-_j-$ pair and search for the kinematical region where the Compton-type diagrams contribution is not suppressed with respect to the Borsellino ones. The value of the parameter $\epsilon$ is estimated as a function of the dark photon mass for a given number of events.


title: A unified approach for projections onto the intersection of $\ell_1$ and $\ell_2$ balls or spheres

abstract: This paper focuses on designing a unified approach for computing the projection onto the intersection of an $\ell_1$ ball/sphere and an $\ell_2$ ball/sphere. We show that the major computational efforts of solving these problems all rely on finding the root of the same piecewisely quadratic function, and then propose a unified numerical method to compute the root. In particular, we design breakpoint search methods with/without sorting incorporated with bisection, secant and Newton methods to find the interval containing the root, on which the root has a closed form. It can be shown that our proposed algorithms without sorting possess $O(n log n)$ worst-case complexity and $O(n)$ in practice. The efficiency of our proposed algorithms are demonstrated in numerical experiments.


title: Search for high mass dijet resonances with a new background prediction method in proton-proton collisions at $\sqrt{s} =$ 13 TeV

abstract: A search for narrow and broad resonances with masses greater than 1.8 TeV decaying to a pair of jets is presented. The search uses proton-proton collision data at $\sqrt{s} =$ 13 TeV collected at the LHC, corresponding to an integrated luminosity of 137 fb$^{-1}$. The background arising from standard model processes is predicted with the fit method used in previous publications and with a new method. The dijet invariant mass spectrum is well described by both data-driven methods, and no significant evidence for the production of new particles is observed. Model independent upper limits are reported on the production cross sections of narrow resonances, and broad resonances with widths up to 55% of the resonance mass. Limits are presented on the masses of narrow resonances from various models: string resonances, scalar diquarks, axigluons, colorons, excited quarks, color-octet scalars, W' and Z' bosons, Randall-Sundrum gravitons, and dark matter mediators. The limits on narrow resonances are improved by 200 to 800 GeV relative to those reported in previous CMS dijet resonance searches. The limits on dark matter mediators are presented as a function of the resonance mass and width, and on the associated coupling strength as a function of the mediator mass. These limits exclude at 95% confidence level a dark matter mediator with a mass of 1.8 TeV and width 1% of its mass or higher, up to one with a mass of 4.8 TeV and a width 45% of its mass or higher.


title: Spherical complexities, with applications to closed geodesics

abstract: We construct and discuss new numerical homotopy invariants of topological spaces that are suitable for the study of functions on loop and sphere spaces. These invariants resemble the Lusternik-Schnirelmann category and provide lower bounds for the numbers of critical orbits of SO(n)-invariant functions on spaces of n-spheres in a manifold. Lower bounds on these invariants are derived using weights of cohomology classes. As an application, we prove new existence results for closed geodesics on Finsler manifolds of positive flag curvature satisfying a pinching condition.


title: Interpretable Multiple-Kernel Prototype Learning for Discriminative Representation and Feature Selection

abstract: Prototype-based methods are of the particular interest for domain specialists and practitioners as they summarize a dataset by a small set of representatives. Therefore, in a classification setting, interpretability of the prototypes is as significant as the prediction accuracy of the algorithm. Nevertheless, the state-of-the-art methods make inefficient trade-offs between these concerns by sacrificing one in favor of the other, especially if the given data has a kernel-based representation. In this paper, we propose a novel interpretable multiple-kernel prototype learning (IMKPL) to construct highly interpretable prototypes in the feature space, which are also efficient for the discriminative representation of the data. Our method focuses on the local discrimination of the classes in the feature space and shaping the prototypes based on condensed class-homogeneous neighborhoods of data. Besides, IMKPL learns a combined embedding in the feature space in which the above objectives are better fulfilled. When the base kernels coincide with the data dimensions, this embedding results in a discriminative features selection. We evaluate IMKPL on several benchmarks from different domains which demonstrate its superiority to the related state-of-the-art methods regarding both interpretability and discriminative representation.


title: Notes on observational and radar coordinates for localized observers

abstract: The worldline of a uniformly accelerated localized observer in Minkowski space is restricted in the Rindler wedge, where the observer can in principle arrange experiments repeatedly, and the Cauchy problem for quantum fields in that Rindler wedge can be well defined. However, the observer can still receive the signals sourced by the events behind the past horizon, and coordinatize those events in terms of some kind of observational coordinates. We construct such observational coordinates in some simple cases with the localized observers in Minkowski, de Sitter, and Schwarzschild-like spacetimes, and compare them with radar coordinates for the same observers.


title: TSK-Streams: Learning TSK Fuzzy Systems on Data Streams

abstract: The problem of adaptive learning from evolving and possibly non-stationary data streams has attracted a lot of interest in machine learning in the recent past, and also stimulated research in related fields, such as computational intelligence and fuzzy systems. In particular, several rule-based methods for the incremental induction of regression models have been proposed. In this paper, we develop a method that combines the strengths of two existing approaches rooted in different learning paradigms. More concretely, our method adopts basic principles of the state-of-the-art learning algorithm AMRules and enriches them by the representational advantages of fuzzy rules. In a comprehensive experimental study, TSK-Streams is shown to be highly competitive in terms of performance.


title: Transformation of low-quality device-recorded speech to high-quality speech using improved SEGAN model

abstract: Nowadays vast amounts of speech data are recorded from low-quality recorder devices such as smartphones, tablets, laptops, and medium-quality microphones. The objective of this research was to study the automatic generation of high-quality speech from such low-quality device-recorded speech, which could then be applied to many speech-generation tasks. In this paper, we first introduce our new device-recorded speech dataset then propose an improved end-to-end method for automatically transforming the low-quality device-recorded speech into professional high-quality speech. Our method is an extension of a generative adversarial network (GAN)-based speech enhancement model called speech enhancement GAN (SEGAN), and we present two modifications to make model training more robust and stable. Finally, from a large-scale listening test, we show that our method can significantly enhance the quality of device-recorded speech signals.