title: Why Propagate Alone? Parallel Use of Labels and Features on Graphs

abstract: Graph neural networks (GNNs) and label propagation represent two interrelated modeling strategies designed to exploit graph structure in tasks such as node property prediction. The former is typically based on stacked message-passing layers that share neighborhood information to transform node features into predictive embeddings. In contrast, the latter involves spreading label information to unlabeled nodes via a parameter-free diffusion process, but operates independently of the node features. Given then that the material difference is merely whether features or labels are smoothed across the graph, it is natural to consider combinations of the two for improving performance. In this regard, it has recently been proposed to use a randomly-selected portion of the training labels as GNN inputs, concatenated with the original node features for making predictions on the remaining labels. This so-called label trick accommodates the parallel use of features and labels, and is foundational to many of the top-ranking submissions on the Open Graph Benchmark (OGB) leaderboard. And yet despite its wide-spread adoption, thus far there has been little attempt to carefully unpack exactly what statistical properties the label trick introduces into the training pipeline, intended or otherwise. To this end, we prove that under certain simplifying assumptions, the stochastic label trick can be reduced to an interpretable, deterministic training objective composed of two factors. The first is a data-fitting term that naturally resolves potential label leakage issues, while the second serves as a regularization factor conditioned on graph structure that adapts to graph size and connectivity. Later, we leverage this perspective to motivate a broader range of label trick use cases, and provide experiments to verify the efficacy of these extensions.


title: CNN-DST: ensemble deep learning based on Dempster-Shafer theory for vibration-based fault recognition

abstract: Nowadays, using vibration data in conjunction with pattern recognition methods is one of the most common fault detection strategies for structures. However, their performances depend on the features extracted from vibration data, the features selected to train the classifier, and the classifier used for pattern recognition. Deep learning facilitates the fault detection procedure by automating the feature extraction and selection, and classification procedure. Though, deep learning approaches have challenges in designing its structure and tuning its hyperparameters, which may result in a low generalization capability. Therefore, this study proposes an ensemble deep learning framework based on a convolutional neural network (CNN) and Dempster-Shafer theory (DST), called CNN-DST. In this framework, several CNNs with the proposed structure are first trained, and then, the outputs of the CNNs selected by the proposed technique are combined by using an improved DST-based method. To validate the proposed CNN-DST framework, it is applied to an experimental dataset created by the broadband vibrational responses of polycrystalline Nickel alloy first-stage turbine blades with different types and severities of damage. Through statistical analysis, it is shown that the proposed CNN-DST framework classifies the turbine blades with an average prediction accuracy of 97.19%. The proposed CNN-DST framework is benchmarked with other state-of-the-art classification methods, demonstrating its high performance. The robustness of the proposed CNN-DST framework with respect to measurement noise is investigated, showing its high noise-resistance. Further, bandwidth analysis reveals that most of the required information for detecting faulty samples is available in a small frequency range.


title: Exploring Timbre Disentanglement in Non-Autoregressive Cross-Lingual Text-to-Speech

abstract: In this paper, we study the disentanglement of speaker and language representations in non-autoregressive cross-lingual TTS models from various aspects. We propose a phoneme length regulator that solves the length mismatch problem between IPA input sequence and monolingual alignment results. Using the phoneme length regulator, we present a FastPitch-based cross-lingual model with IPA symbols as input representations. Our experiments show that language-independent input representations (e.g. IPA symbols), an increasing number of training speakers, and explicit modeling of speech variance information all encourage non-autoregressive cross-lingual TTS model to disentangle speaker and language representations. The subjective evaluation shows that our proposed model can achieve decent naturalness and speaker similarity in cross-language voice cloning.


title: Impact of self-association on the architectural properties of bacterial nucleoid proteins

abstract: The chromosomal DNA of bacteria is folded into a compact body called the nucleoid, which is composed essentially of DNA (80%), RNA (10%), and a number of different proteins (10%). These nucleoid proteins act as regulators of gene expression and influence the organization of the nucleoid by bridging, bending, or wrapping the DNA. These so-called architectural properties of nucleoid proteins are still poorly understood. For example, the reason why certain proteins compact the DNA coil in certain environments but make instead the DNA more rigid in other environments is the matter of ongoing debates. Here, we address the question of the impact of the self-association of nucleoid proteins on their architectural properties and try to determine whether differences in self-association are sufficient to induce large changes in the organization of the DNA coil. More specifically, we developed two coarse-grained models of proteins, which interact identically with the DNA but self-associate differently by forming either clusters or filaments in the absence of the DNA. We showed through Brownian dynamics simulations that self-association of the proteins increases dramatically their ability to shape the DNA coil. Moreover, we observed that cluster-forming proteins compact significantly the DNA coil (similar to the DNA-bridging mode of H-NS proteins), whereas filament-forming proteins increase instead significantly the stiffness of the DNA chain (similar to the DNA-stiffening mode of H-NS proteins). This work consequently suggests that the knowledge of the DNA-binding properties of the proteins is in itself not sufficient to understand their architectural properties. Rather, their self-association properties must also be investigated in detail, because they might actually drive the formation of different DNA/protein complexes.


title: Light curves of the hot spots on circular orbits around generic regular black holes related to non-linear electrodynamics with Maxwellian weak-field Limit

abstract: The combined effect of a strong gravitational field and electromagnetic field in the vicinity of a generic regular black hole related to non-linear electrodynamics with Maxwellian weak-field limit on the radiation flux of a hot spot orbiting the regular black hole on the Keplerian disc has been studied. The frequency shift due to the strong gravitational field and magnification of the radiation related to gravitational lensing have been calculated. We compare the flux related to the Maxwellian regular black hole to the flux of the hot spots moving with the same orbital period around standard Schwarzschild and Raissner-Nordstrom black holes to illustrate the role of the effective geometry governing photon motion around the regular black hole.


title: Scattering from spheres made of time-varying and dispersive materials

abstract: Exploring the interaction of light with time-varying media is an intellectual challenge that, in addition to fundamental aspects, provides a pathway to multiple promising applications. Time modulation constitutes here a fundamental handle to control light on entirely different grounds. That holds particularly for complex systems simultaneously structured in space and time. However, a realistic description of time-varying materials requires considering their material dispersion. The combination thereof has barely been considered but is crucial since dispersion accompanies materials suitable for dynamic modulation. As a canonical scattering problem from which many general insights can be obtained, we develop and apply a self-consistent analytical theory of light scattering by a sphere made from a time-varying material exemplarily assumed to have a Lorentzian dispersion. We discuss the eigensolutions of Maxwell's equations in the bulk and present a dedicated Mie theory. The proposed theory is verified with full-wave simulations. We disclose effects such as energy transfer from the time-modulation subsystem to the electromagnetic field, amplifying carefully structured incident fields. Since many phenomena can be studied on analytical grounds with our formalism, it will be indispensable when exploring electromagnetic phenomena in time-varying and spatially structured finite objects of other geometries.


title: Boundaries for Gelfand transform images of Banach algebras of holomorphic functions

abstract: Let $\mathcal{A}$ be a Banach algebra of bounded holomorphic functions on the open unit ball $B_X$ of a complex Banach space $X$. Considering the Gelfand transform image $\widehat{\mathcal{A}}$ of the Banach algebra $\mathcal{A}$, which is a uniform algebra on the spectrum of $\mathcal{A}$, we obtain an explicit description of the Shilov boundary for $\widehat{\mathcal{A}}$ for classical Banach spaces $X$ in the case where $\mathcal{A}$ is a certain Banach algebra, for instance, $\mathcal{A}_\infty (B_X)$, $\mathcal{A}_u (B_X)$ or $\mathcal{A}_{wu} (B_X)$. Some possible application of our result to the famous Corona theorem is also briefly discussed.


title: Semi-supervised Multi-task Learning for Semantics and Depth

abstract: Multi-Task Learning (MTL) aims to enhance the model generalization by sharing representations between related tasks for better performance. Typical MTL methods are jointly trained with the complete multitude of ground-truths for all tasks simultaneously. However, one single dataset may not contain the annotations for each task of interest. To address this issue, we propose the Semi-supervised Multi-Task Learning (SemiMTL) method to leverage the available supervisory signals from different datasets, particularly for semantic segmentation and depth estimation tasks. To this end, we design an adversarial learning scheme in our semi-supervised training by leveraging unlabeled data to optimize all the task branches simultaneously and accomplish all tasks across datasets with partial annotations. We further present a domain-aware discriminator structure with various alignment formulations to mitigate the domain discrepancy issue among datasets. Finally, we demonstrate the effectiveness of the proposed method to learn across different datasets on challenging street view and remote sensing benchmarks.


title: Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling

abstract: Given the claims of improved text generation quality across various pre-trained neural models, we consider the coherence evaluation of machine generated text to be one of the principal applications of coherence models that needs to be investigated. Prior work in neural coherence modeling has primarily focused on devising new architectures for solving the permuted document task. We instead use a basic model architecture and show significant improvements over state of the art within the same training regime. We then design a harder self-supervision objective by increasing the ratio of negative samples within a contrastive learning setup, and enhance the model further through automatic hard negative mining coupled with a large global negative queue encoded by a momentum encoder. We show empirically that increasing the density of negative samples improves the basic model, and using a global negative queue further improves and stabilizes the model while training with hard negative samples. We evaluate the coherence model on task-independent test sets that resemble real-world applications and show significant improvements in coherence evaluations of downstream tasks.


title: Dilepton production rate near the critical temperature of color superconductivity

abstract: We investigate modification of the dilepton production rate by the diquark fluctuations that form well-developed collective modes near the critical temperature of color superconductivity. Through the analysis of the photon self-energy called the Aslamasov-Larkin, Maki-Thompson and density of states terms in the theory of metalic superconductivity, it is shown that the collective mode in the diquark channel affects the photon self-energy significantly and thereby gives rise to an anomalous enhacement of the dilepton production rate in the low invariant-mass region.


title: Inverse analysis of material parameters in coupled multi-physics biofilm models

abstract: In this article we propose an inverse analysis algorithm to find the best fit of multiple material parameters in different coupled multi-physics biofilm models. We use a nonlinear continuum mechanical approach to model biofilm deformation that occurs in flow cell experiments. The objective function is based on a simple geometrical measurement of the distance of the fluid biofilm interface between model and experiments. A Levenberg-Marquardt algorithm based on finite difference approximation is used as an optimizer. The proposed method uses a moderate to low amount of model evaluations. For a first presentation and evaluation the algorithm is applied and tested on different numerical examples based on generated numerical results and the addition of Gaussian noise. Achieved numerical results show that the proposed method serves well for different physical effects investigated and numerical approaches chosen for the model. Presented examples show the inverse analysis for multiple parameters in biofilm models including fluid-solid interaction effects, poroelasticity, heterogeneous material properties and growth.


title: Coarse to Fine: Video Retrieval before Moment Localization

abstract: The current state-of-the-art methods for video corpus moment retrieval (VCMR) often use similarity-based feature alignment approach for the sake of convenience and speed. However, late fusion methods like cosine similarity alignment are unable to make full use of the information from both query texts and videos. In this paper, we combine feature alignment with feature fusion to promote the performance on VCMR.


title: Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution

abstract: In this paper, we introduce a variational Bayesian algorithm (VBA) for image blind deconvolution. Our generic framework incorporates smoothness priors on the unknown blur/image and possible affine constraints (e.g., sum to one) on the blur kernel. One of our main contributions is the integration of VBA within a neural network paradigm, following an unrolling methodology. The proposed architecture is trained in a supervised fashion, which allows us to optimally set two key hyperparameters of the VBA model and lead to further improvements in terms of resulting visual quality. Various experiments involving grayscale/color images and diverse kernel shapes, are performed. The numerical examples illustrate the high performance of our approach when compared to state-of-the-art techniques based on optimization, Bayesian estimation, or deep learning.


title: Same-diff? Conceptual similarities between gauge transformations and diffeomorphisms. Part I: Symmetries and isomorphisms

abstract: The following questions are germane to our understanding of gauge-(in)variant quantities and physical possibility: how are gauge transformations and spacetime diffeomorphisms understood as symmetries, in which ways are they similar, and in which are they different? To what extent are we justified in endorsing different attitudes -- nowadays called sophistication, haecceitism, and eliminativism -- towards each? This is the first of four papers taking up this question. This first paper will discuss notions of symmetries and isomorphisms that will be used in the remaining papers in the series. There are several such notions in the literature and the question of how they mesh with empirical discernibility is a delicate one; even the orthodox view that symmetries are empirically unobservable (even in principle) has recently been challenged by Belot. Focusing on local field theories, I will provide a precise definition of dynamical symmetries in terms of the space of states of the theory at hand. I will then apply the definition to Yang-Mills theories and general relativity and show that these symmetries correspond to automorphisms of `natural' geometric structures: the small diffeomorphisms of the spacetime manifold and the small fiber-preserving diffeomorphisms of a fibered space. Finally, I will show that these automorphisms can be given a passive gloss, since they correspond 1-1 to the coordinate transformations of the underlying manifolds.


title: Same-diff? Conceptual similarities between gauge transformations and diffeomorphisms. Part II: Challenges to sophistication

abstract: The following questions are germane to our understanding of gauge-(in)variant quantities and physical possibility: how are gauge transformations and spacetime diffeomorphisms understood as symmetries, in which ways are they similar, and in which are they different? To what extent are we justified in endorsing different attitudes -- nowadays called sophistication, haecceitism, and eliminativism -- towards each? This is the second of four papers taking up this question, and it is the one that most engages with the metaphysical debates surrounding our understanding of symmetry and equivalence. In this paper, I will provide two desiderata for the application of a treatment of symmetries known as `sophistication' and show that both general relativity and Yang-Mills theory satisfy these desiderata. The first desideratum for symmetries is mathematical, and was shown to hold for general relativity and Yang-Mills in the first paper: (i) that they correspond to the automorphisms of the structured base sets for the models of the two theories. Here I will extend the desideratum to more general theories. The second desideratum is mathematical-physical: (ii) that the general type of structure to which Desideratum (i) refers is axiomatizable and that this axiomatization can be phrased in terms of basic physical predicates of the theory. In the third paper in the series I will provide yet a third desideratum, to deal with an issue that goes beyond the standard debate about sophistication.


title: SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing

abstract: Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning. The SpeechT5 framework consists of a shared encoder-decoder network and six modal-specific (speech/text) pre/post-nets. After preprocessing the input speech/text through the pre-nets, the shared encoder-decoder network models the sequence-to-sequence transformation, and then the post-nets generate the output in the speech/text modality based on the output of the decoder. Leveraging large-scale unlabeled speech and text data, we pre-train SpeechT5 to learn a unified-modal representation, hoping to improve the modeling capability for both speech and text. To align the textual and speech information into this unified semantic space, we propose a cross-modal vector quantization approach that randomly mixes up speech/text states with latent units as the interface between encoder and decoder. Extensive evaluations show the superiority of the proposed SpeechT5 framework on a wide variety of spoken language processing tasks, including automatic speech recognition, speech synthesis, speech translation, voice conversion, speech enhancement, and speaker identification. We release our code and model at https://github.com/microsoft/SpeechT5.


title: Task-Driven Deep Image Enhancement Network for Autonomous Driving in Bad Weather

abstract: Visual perception in autonomous driving is a crucial part of a vehicle to navigate safely and sustainably in different traffic conditions. However, in bad weather such as heavy rain and haze, the performance of visual perception is greatly affected by several degrading effects. Recently, deep learning-based perception methods have addressed multiple degrading effects to reflect real-world bad weather cases but have shown limited success due to 1) high computational costs for deployment on mobile devices and 2) poor relevance between image enhancement and visual perception in terms of the model ability. To solve these issues, we propose a task-driven image enhancement network connected to the high-level vision task, which takes in an image corrupted by bad weather as input. Specifically, we introduce a novel low memory network to reduce most of the layer connections of dense blocks for less memory and computational cost while maintaining high performance. We also introduce a new task-driven training strategy to robustly guide the high-level task model suitable for both high-quality restoration of images and highly accurate perception. Experiment results demonstrate that the proposed method improves the performance among lane and 2D object detection, and depth estimation largely under adverse weather in terms of both low memory and accuracy.


title: Scalar Curvature on some Open 3-Manifolds

abstract: This article is a survey of recent results about scalar curvature and contractible open $3$-manifolds. It is dedicated to the memory of Professor S. S. Chern.


title: Noether charges: the link between empirical significance of symmetries and non-separability

abstract: A fundamental tenet of gauge theory is that physical quantities should be gauge-invariant. This prompts the question: can gauge symmetries have physical significance? On one hand, the Noether theorems relate conserved charges to symmetries, endowing the latter with physical significance, though this significance is sometimes taken as indirect. But for theories in spatially finite and bounded regions, the standard Noether charges are not gauge-invariant. I here argue that gauge-\emph{variance} of charges is tied to the nature of the non-locality within gauge theories. I will flesh out these links by providing a chain of (local) implications: `local conservation laws'${\Rightarrow}$ `conserved regional charges' $\Leftrightarrow$ `non-separability' ${\Leftrightarrow}$ `direct empirical significance of symmetries'.


title: A Dual-Attention Neural Network for Pun Location and Using Pun-Gloss Pairs for Interpretation

abstract: Pun location is to identify the punning word (usually a word or a phrase that makes the text ambiguous) in a given short text, and pun interpretation is to find out two different meanings of the punning word. Most previous studies adopt limited word senses obtained by WSD(Word Sense Disambiguation) technique or pronunciation information in isolation to address pun location. For the task of pun interpretation, related work pays attention to various WSD algorithms. In this paper, a model called DANN (Dual-Attentive Neural Network) is proposed for pun location, effectively integrates word senses and pronunciation with context information to address two kinds of pun at the same time. Furthermore, we treat pun interpretation as a classification task and construct pungloss pairs as processing data to solve this task. Experiments on the two benchmark datasets show that our proposed methods achieve new state-of-the-art results. Our source code is available in the public code repository.


title: Improve Cross-lingual Voice Cloning Using Low-quality Code-switched Data

abstract: Recently, sequence-to-sequence (seq-to-seq) models have been successfully applied in text-to-speech (TTS) to synthesize speech for single-language text. To synthesize speech for multiple languages usually requires multi-lingual speech from the target speaker. However, it is both laborious and expensive to collect high-quality multi-lingual TTS data for the target speakers. In this paper, we proposed to use low-quality code-switched found data from the non-target speakers to achieve cross-lingual voice cloning for the target speakers. Experiments show that our proposed method can generate high-quality code-switched speech in the target voices in terms of both naturalness and speaker consistency. More importantly, we find that our method can achieve a comparable result to the state-of-the-art (SOTA) performance in cross-lingual voice cloning.


title: Control-oriented Modeling of Bend Propagation in an Octopus Arm

abstract: Bend propagation in an octopus arm refers to a stereotypical maneuver whereby an octopus pushes a bend (localized region of large curvature) from the base to the tip of the arm. Bend propagation arises from the complex interplay between mechanics of the flexible arm, forces generated by internal muscles, and environmental effects (buoyancy and drag) from of the surrounding fluid. In part due to this complexity, much of prior modeling and analysis work has relied on the use of high dimensional computational models. The contribution of this paper is to present a control-oriented reduced order model based upon a novel parametrization of the curvature of the octopus arm. The parametrization is motivated by the experimental results. The reduced order model is related to and derived from a computational model which is also presented. The results from the two sets of models are compared using numerical simulations which is shown to lead to useful qualitative insights into bend propagation. A comparison between the reduced order model and experimental data is also reported.


title: Gagliardo-Nirenberg-Sobolev inequalities on planar graphs

abstract: In this paper we study a family of interpolation Gagliardo-Nirenberg-Sololev inequalities on planar graphs. We are interested in knowing when the best constants in the inequalities are achieved. We also analyse the set of solutions of the corresponding Euler-Lagrange equations.


title: Energy method for the Boltzmann equation of monatomic gaseous mixtures

abstract: In this paper, we present an energy method for the system of Boltzmann equations in the multicomponent mixture case, based on a micro-macro decomposition. More precisely, the perturbation of a solution to the Boltzmann equation around a global equilibrium is decomposed into the sum of a macroscopic and a microscopic part, for which we obtain a priori estimates at both lower and higher orders. These estimates are obtained under a suitable smallness assumption. The assumption can be justified a posteriori in the higher-order case, leading to the closure of the corresponding estimate.


title: Confining integro-differential equations originating from evolutionary biology: ground states and long time dynamics

abstract: We consider nonlinear mutation selection models, known as replicator-mutator equations in evolutionary biology. They involve a nonlocal mutation kernel and a confining fitness potential. We prove that the long time behaviour of the Cauchy problem is determined by the principal eigenelement of the underlying linear operator. The novelties compared to the literature on these models are about the case of symmetric mutations: we propose a new milder sufficient condition for the existence of a principal eigenfunction, and we provide what is to our knowledge the first quantification of the spectral gap. We also recover existing results in the non-symmetric case, through a new approach.


title: On homogeneous and oscillating random walks on the integers

abstract: Considering homogeneous and oscillating random walks on the integers, we simplify classical works on recurrence of Spitzer and Kemperman, respectively. Some links with renewal theory are discussed.


title: FedSpeech: Federated Text-to-Speech with Continual Learning

abstract: Federated learning enables collaborative training of machine learning models under strict privacy restrictions and federated text-to-speech aims to synthesize natural speech of multiple users with a few audio training samples stored in their devices locally. However, federated text-to-speech faces several challenges: very few training samples from each speaker are available, training samples are all stored in local device of each user, and global model is vulnerable to various attacks. In this paper, we propose a novel federated learning architecture based on continual learning approaches to overcome the difficulties above. Specifically, 1) we use gradual pruning masks to isolate parameters for preserving speakers' tones; 2) we apply selective masks for effectively reusing knowledge from tasks; 3) a private speaker embedding is introduced to keep users' privacy. Experiments on a reduced VCTK dataset demonstrate the effectiveness of FedSpeech: it nearly matches multi-task training in terms of multi-speaker speech quality; moreover, it sufficiently retains the speakers' tones and even outperforms the multi-task training in the speaker similarity experiment.


title: Alpine Permafrost Modeling: On the influence of topography driven lateral fluxes

abstract: Alpine permafrost environments are highly vulnerable and sensitive to changes in regional and global climate trends. Thawing and degradation of permafrost has numerous adverse environmental, economic, and societal impacts. Mathematical modeling and numerical simulations provide powerful tools for predicting the degree of degradation and evolution of subsurface permafrost as a result of global warming. A particularly significant characteristic of alpine environments is the high variability in their topography and geomorphology which drives large lateral thermal and fluid fluxes. Additionally, harsh winds, extreme weather conditions, and various degrees of saturation have to be considered. The combination of large lateral fluxes and unsaturated ground makes alpine systems markedly different from Arctic permafrost environments and general geotechnical ground freezing applications, and therefore, alpine permafrost demands its own specialized modeling approaches. In this research work, we present a multi-physics permafrost model tailored to alpine regions. In particular, we resolve the ice-water phase transitions, unsaturated conditions, and capillary actions, and account for the impact of the evolving pore volume on fluid-matrix interactions. Moreover, the approach is multi-dimensional, and therefore, inherently resolves fluxes along topographic gradients. Through numerical cases studies based on the elevation profiles of the two prominent peaks of the Zugspitze (DE) and the Matterhorn (CH), we show the strong influence of topography driven thermal and fluid fluxes on active layer dynamics and the distribution of permafrost.


title: Deep-3D Microscope: 3D volumetric microscopy of thick scattering samples using a wide-field microscope and machine learning

abstract: Confocal microscopy is the standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, a confocal microscope is quite expensive compared to traditional microscopes. In addition, the point scanning in a confocal leads to slow imaging speed and photobleaching due to the high dose of laser energy. In this paper, we demonstrate how the advances in machine learning can be exploited to "teach" a traditional wide-field microscope, one that's available in every lab, into producing 3D volumetric images like a confocal. The key idea is to obtain multiple images with different focus settings using a wide-field microscope and use a 3D Generative Adversarial Network (GAN) based neural network to learn the mapping between the blurry low-contrast image stack obtained using wide-field and the sharp, high-contrast images obtained using a confocal. After training the network with widefield-confocal image pairs, the network can reliably and accurately reconstruct 3D volumetric images that rival confocal in terms of its lateral resolution, z-sectioning and image contrast. Our experimental results demonstrate generalization ability to handle unseen data, stability in the reconstruction results, high spatial resolution even when imaging thick ($\sim40$ microns) highly-scattering samples. We believe that such learning-based-microscopes have the potential to bring confocal quality imaging to every lab that has a wide-field microscope.


title: A dark matter solution for the XENON1T electron excess and the galactic center 511 keV line

abstract: The excess of the 511 keV line from the Milky Way galactic bulge, confirmed by the INTEGRAL detector, is a longstanding mystery. The morphology of the line appears to be proportional to the square of the dark matter density, hinting towards a dark matter origin. On the other hand, in 2020, XENON1T has reported an excess of electrons with a recoil energy of $2-4$ keV. We present a model based on a dark matter of a few MeV mass that decays into a pair of pico-charged particles with a lifetime much larger than the age of the Universe. The magnetic field of the galaxy accumulates these relativistic pico-charged particles whose scattering on the electrons can explain the signal reported by XENON1T. The annihilation of the pico-charged particles in the galactic bulge leads to $e^-e^+$ production and therefore to an excess of the 511 keV line. We review the present observational bounds and the strategies to test the model.


title: Analysing the Effect of Test-and-Trace Strategy in an SIR Epidemic Model

abstract: Consider a Markovian SIR epidemic model in a homogeneous community. To this model we add a rate at which individuals are tested, and once an infectious individual tests positive it is isolated and each of their contacts are traced and tested independently with some fixed probability. If such a traced individual tests positive it is isolated, and the contact tracing is iterated. This model is analysed using large population approximations, both for the early stage of the epidemic when the "to-be-traced components" of the epidemic behaves like a branching process, and for the main stage of the epidemic where the process of to-be-traced components converges to a deterministic process defined by a system of differential equations. These approximations are used to quantify the effect of testing and of contact tracing on the effective reproduction numbers (for the components as well as for the individuals), the probability of a major outbreak, and the final fraction getting infected. Using numerical illustrations when rates of infection and natural recovery are fixed, it is shown that Test-and-Trace strategy is effective in reducing the reproduction number. Surprisingly, the reproduction number for the branching process of components is not monotonically decreasing in the tracing probability, but the individual reproduction number is conjectured to be monotonic as expected. Further, in the situation where individuals also self-report for testing, the tracing probability is more influential than the screening rate (measured by the fraction infected being screened).


title: Learning a Compressive Sensing Matrix with Structural Constraints via Maximum Mean Discrepancy Optimization

abstract: We introduce a learning-based algorithm to obtain a measurement matrix for compressive sensing related recovery problems. The focus lies on matrices with a constant modulus constraint which typically represent a network of analog phase shifters in hybrid precoding/combining architectures. We interpret a matrix with restricted isometry property as a mapping of points from a high- to a low-dimensional hypersphere. We argue that points on the low-dimensional hypersphere, namely, in the range of the matrix, should be uniformly distributed to increase robustness against measurement noise. This notion is formalized in an optimization problem which uses one of the maximum mean discrepancy metrics in the objective function. Recent success of such metrics in neural network related topics motivate a solution of the problem based on machine learning. Numerical experiments show better performance than random measurement matrices that are generally employed in compressive sensing contexts. Further, we adapt a method from the literature to the constant modulus constraint. This method can also compete with random matrices and it is shown to harmonize well with the proposed learning-based approach if it is used as an initialization. Lastly, we describe how other structural matrix constraints, e.g., a Toeplitz constraint, can be taken into account, too.


title: Hide-and-seek with cosmic tau neutrinos

abstract: We first revisit the possibility of preserving the original flavor ratio of high energy cosmic neutrino flux by turning on a coupling between these neutrinos and ultra-light dark matter. We discuss the bound that can be set on such a coupling from the recent $\nu_\tau$ observation by ICECUBE and outline the implications of the coupling for the EeV range cosmic neutrino flux to be observed by upcoming neutrino detectors. We then focus on the $3+1$ scheme when the active sterile oscillation length is of order of 1000~km for EeV range cosmic neutrinos. We show that within this scenario, the probability of survival of an active neutrino passing through the Earth can be sizable, despite the fact that the mean free path of the EeV neutrinos is much smaller than the Earth radius. This opens up the possibility to have neutrino events similar to the two anomalous events reported by ANITA.


title: Physical mechanisms underpinning the vacuum permittivity

abstract: Debate about the emptiness of the space goes back to the prehistory of science and is epitomized by the Aristotelian \emph{horror vacui}, which can be seen as the precursor of the ether, whose modern version is the dynamical quantum vacuum. Here, we change our view to \emph{gaudium vacui} and discuss how the vacuum fluctuations fix the value of the permittivity $\varepsilon_{0}$ and permeability $\mu_{0}$.


title: Choice probabilities and correlations in closed-form route choice models: specifications and drawbacks

abstract: This paper investigates the performance, in terms of choice probabilities and correlations, of existing and new specifications of closed-form route choice models with flexible correlation patterns, namely the Link Nested Logit (LNL), the Paired Combinatorial Logit (PCL) and the more recent Combination of Nested Logit (CoNL) models. Following a consolidated track in the literature, choice probabilities and correlations of the Multinomial Probit (MNP) model by (Daganzo and Sheffi, 1977) are taken as target. Laboratory experiments on small/medium-size networks are illustrated, also leveraging a procedure for practical calculation of correlations of any GEV models, proposed by (Marzano 2014). Results show that models with inherent limitations in the coverage of the domain of feasible correlations yield unsatisfactory performance, whilst the specifications of the CoNL proposed in the paper appear the best in fitting both MNP correlations and probabilities. Performance of the models are appreciably ameliorated by introducing lower bounds to the nesting parameters. Overall, the paper provides guidance for the practical application of tested models.


title: Web Search via an Efficient and Effective Brain-Machine Interface

abstract: While search technologies have evolved to be robust and ubiquitous, the fundamental interaction paradigm has remained relatively stable for decades. With the maturity of the Brain-Machine Interface, we build an efficient and effective communication system between human beings and search engines based on electroencephalogram(EEG) signals, called Brain-Machine Search Interface(BMSI) system. The BMSI system provides functions including query reformulation and search result interaction. In our system, users can perform search tasks without having to use the mouse and keyboard. Therefore, it is useful for application scenarios in which hand-based interactions are infeasible, e.g, for users with severe neuromuscular disorders. Besides, based on brain signals decoding, our system can provide abundant and valuable user-side context information(e.g., real-time satisfaction feedback, extensive context information, and a clearer description of information needs) to the search engine, which is hard to capture in the previous paradigm. In our implementation, the system can decode user satisfaction from brain signals in real-time during the interaction process and re-rank the search results list based on user satisfaction feedback. The demo video is available at http://www.thuir.cn/group/YQLiu/datasets/BMSISystem.mp4.


title: Group Identity, Social Learning and Opinion Dynamics

abstract: In this paper, we study opinion dynamics in a balanced social structure consisting of two groups. Agents learn the true state of the world naively learning from their neighbors and from an unbiased source of information. Agents want to agree with others of the same group--in-group identity -- but to disagree with those of the opposite group--out-group conflict. We characterize the long-run opinions, and show that agents' influence depends on their Bonacich centrality in the signed network of opinion exchange. Finally, we study the effect of group size, the weight given to unbiased information and homophily when agents in the same group are homogeneous.


title: Long and Short Fast Radio Bursts are Different from Repeating and Non-repeating Transients

abstract: We collect 133 Fast Radio Bursts (FRBs), including 110 non-repeating and 23 repeating ones, and systematically investigate their observational properties. To check the frequency dependence of FRB classifications, we define our samples with a central frequency below/above 1GHz as subsample I/II. We find that there is a clear bimodal distribution of pulse width for the subsample I. And If we classify FRBs into short FRBs (\emph{s}FRBs) ($<$100 ms) and long FRBs (\emph{l}FRBs) ($>$100 ms) as done for short and long Gamma-Ray Bursts (GRBs), the \emph{s}FRBs at higher central frequency are commonly shorter than those at lower central frequency not only for non-repeating but also repeating \emph{s}FRBs. Secondly, we find that fluence and peak flux density are correlated with a power law relation of $F \varpropto S{^{\gamma}_{p,obs}}$ for both \emph{s}FRBs and \emph{l}FRBs whose distributions are obviously different. Thirdly, the \emph{l}FRBs with isotropic energies ranging from $10^{42}$ to $10^{44}$ erg are more energetic than the \emph{s}FRBs in the $F- DM_{EX}$ plane, indicating that they are two representative types. Finally, it is interestingly note that the peak flux density behaves an independence on the redshift when the distance of the FRBs becomes far enough, which is similar to the scenario of peak flux evolving with redshift in the field of GRBs. We predict that fainter FRBs at higher redshift of $z>2$ can be successfully detected by FAST and SKA in the near future.


title: Ultrafast photonic rainbow with controllable orbital angular momentum

abstract: Increasing any degree of freedom of light beam may open a wide application area of this special light beam. Vortex beam with a dimension of orbital angular momentum (OAM) as a useful light source has been widely applied in many fields. Here, unique multicolor concentric ultrafast vortex beams (MUCU-VBs), which are also named ultrafast photonic rainbow, with controllable orbital angular momentum are firstly generated using cascaded four-wave mixing (CFWM) in an yttrium aluminum garnet (YAG) plate. Up to 9 multicolor concentric annular ultrafast vortex sidebands are generated simultaneously. The topological charges of the sidebands, which are controllable by changing the topological charges of the two input pump beams, are measured and in according with the theoretical analysis very well. The novel MUCU-VBs can be manipulated simultaneously in temporal, spatial, spectral domains and OAM state, which open more than one new degree of freedoms of vortex light beam and will be of wide and special applications, such as multicolor pump-probe experiments, simultaneous microparticle manipulation and exploring, and optical communication. Moreover, the special focusing properties of the multicolor ultrafast sidebands, such as multi-focus of different wavelengths, may further extend their application area.


title: Review of the Classification and Properties of 55 Variable Stars in Scutum

abstract: This study aims to assess the properties and classification of 55 variable stars in Scutum, little studied since their discovery and reported in the Information Bulletin on Variable Stars (IBVS) 985 and update. Using data from previous studies and several astronomical databases, we performed our analysis mainly utilizing a period analysis software and comparing the photometric characteristics of the variables in a Colour-Absolute Magnitude Diagram. For all stars, the variability is confirmed. We discovered new significant results for the period and/or type of 17 variables and highlighted incorrect cross-reference names on astronomical databases for 3 stars. This assessment also identifies 12 cases for which the results from the ASAS-SN Catalog of Variable Stars are systematically not consistent with the original light curves.


title: Filamentary superconductivity of resistively-switched strontium titanate

abstract: SrTiO$_3$, although a wide gap insulator, has long been known to become metallic and superconducting at extremely low doping levels. This has given rise to questions concerning the coexistence or interdependence of metallicity, superconductivity, and the material's polar properties. This issue becomes especially intriguing in conjunction with the observation that filamentary metallicity can be induced by means of resistive switching at conditions well below relevant doping levels for homogeneous metallicity. In this study, we demonstrate that resistive switching can also be employed to generate superconductivity at the superconducting transition temperature of $T_c$ $\approx$ 0.2 K in SrTiO$_3$. By combining local characterization of the conductivity with theoretical analysis, we propose that the superconducting properties are associated with the electro-formation of columnar-like bundles with a diameter of 40-50 nm, consisting of metallic filaments surrounded by polar regions. We provide a theoretical model identifying the coexistence of metallic and polar regions as a prerequisite for the filamentary-like superconductivity observed.


title: Moments of Sunlight Pathlength in Water and Aerosol Clouds from O$_2$ Spectroscopy: Exploitable Parameter Sensitivities

abstract: The last IPCC assessment states that clouds and aerosols remain a challenge in climate prediction with Global Climate Models. Therefore, NASA's 2017 Decadal Survey has made them, along with convection and precipitation, a priority target for future missions under the ACCP banner, now Atmospheric Observing System. Atmospheric science is now more than ever driving renewal in remote sensing to probe clouds and aerosols more accurately and with improved sampling. Here, we bring new results that support Differential Optical Absorption Spectroscopy (DOAS) using oxygen absorption features in the visible/near-IR spectrum. With known concentration and cross-section, the remaining unknown in O$_2$ DOAS is the path the light followed through the absorbing gas. In presence of scattering by cloud or aerosol particles, that path is broken at each interaction. Cumulative pathlength through the gas is a random variable, and its probability distribution function (PDF) will contain desirable information about the clouds or aerosols. Here, we compute statistical moments of the that PDF, showing that mean and variance convey different pieces of information, namely, size and opacity of the medium. We also extend the connection between geometric thickness and mean pathlength in presence of absorption by the scattering particles. We view pathlength moments as intermediate products that can be obtained from O$_2$ spectroscopy, and describe an algorithm to do that. In turn, these moments yield cloud or aerosol profile parameters of interest: layer geometric and optical thicknesses. One normally thinks about active sensors, radars and lidars, for such atmospheric profiling. Here, the profile is parameterized and representative of a horizontal average determined by multiple scattering. However, there are advantages in passive vs active instrumentation, starting with the possibility of imaging over a large swath.


title: Procrastinated Tree Search: Black-box Optimization with Delayed, Noisy, and Multi-Fidelity Feedback

abstract: In black-box optimization problems, we aim to maximize an unknown objective function, where the function is only accessible through feedbacks of an evaluation or simulation oracle. In real-life, the feedbacks of such oracles are often noisy and available after some unknown delay that may depend on the computation time of the oracle. Additionally, if the exact evaluations are expensive but coarse approximations are available at a lower cost, the feedbacks can have multi-fidelity. In order to address this problem, we propose a generic extension of hierarchical optimistic tree search (HOO), called ProCrastinated Tree Search (PCTS), that flexibly accommodates a delay and noise-tolerant bandit algorithm. We provide a generic proof technique to quantify regret of PCTS under delayed, noisy, and multi-fidelity feedbacks. Specifically, we derive regret bounds of PCTS enabled with delayed-UCB1 (DUCB1) and delayed-UCB-V (DUCBV) algorithms. Given a horizon $T$, PCTS retains the regret bound of non-delayed HOO for expected delay of $O(\log T)$ and worsens by $O(T^{\frac{1-\alpha}{d+2}})$ for expected delays of $O(T^{1-\alpha})$ for $\alpha \in (0,1]$. We experimentally validate on multiple synthetic functions and hyperparameter tuning problems that PCTS outperforms the state-of-the-art black-box optimization methods for feedbacks with different noise levels, delays, and fidelity.


title: Optimal Update in Energy Harvesting Aided Terahertz Communications with Random Blocking

abstract: In this paper, we consider an information update system where wireless sensor sends timely updates to the destination over a random blocking terahertz channel with the supply of harvested energy and reliable energy backup. The paper aims to find the optimal information updating policy that minimize the time-average weighted sum of the Age of information(AoI) and the reliable energy costs by formulating an infinite state Markov decision process(MDP). With the derivation of the monotonicity of value function on each component, the optimal information updating policy is proved to have a threshold structure. Based on this special structure, an algorithm for efficiently computing the optimal policy is proposed. Numerical results show that the optimal updating policy proposed outperforms baseline policies.


title: On the Stability of Low Pass Graph Filter With a Large Number of Edge Rewires

abstract: Recently, the stability of graph filters has been studied as one of the key theoretical properties driving the highly successful graph convolutional neural networks (GCNs). The stability of a graph filter characterizes the effect of topology perturbation on the output of a graph filter, a fundamental building block for GCNs. Many existing results have focused on the regime of small perturbation with a small number of edge rewires. However, the number of edge rewires can be large in many applications. To study the latter case, this work departs from the previous analysis and proves a bound on the stability of graph filter relying on the filter's frequency response. Assuming the graph filter is low pass, we show that the stability of the filter depends on perturbation to the community structure. As an application, we show that for stochastic block model graphs, the graph filter distance converges to zero when the number of nodes approaches infinity. Numerical simulations validate our findings.


title: HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media

abstract: We introduce HUMAN4D, a large and multimodal 4D dataset that contains a variety of human activities simultaneously captured by a professional marker-based MoCap, a volumetric capture and an audio recording system. By capturing 2 female and $2$ male professional actors performing various full-body movements and expressions, HUMAN4D provides a diverse set of motions and poses encountered as part of single- and multi-person daily, physical and social activities (jumping, dancing, etc.), along with multi-RGBD (mRGBD), volumetric and audio data. Despite the existence of multi-view color datasets captured with the use of hardware (HW) synchronization, to the best of our knowledge, HUMAN4D is the first and only public resource that provides volumetric depth maps with high synchronization precision due to the use of intra- and inter-sensor HW-SYNC. Moreover, a spatio-temporally aligned scanned and rigged 3D character complements HUMAN4D to enable joint research on time-varying and high-quality dynamic meshes. We provide evaluation baselines by benchmarking HUMAN4D with state-of-the-art human pose estimation and 3D compression methods. For the former, we apply 2D and 3D pose estimation algorithms both on single- and multi-view data cues. For the latter, we benchmark open-source 3D codecs on volumetric data respecting online volumetric video encoding and steady bit-rates. Furthermore, qualitative and quantitative visual comparison between mesh-based volumetric data reconstructed in different qualities showcases the available options with respect to 4D representations. HUMAN4D is introduced to the computer vision and graphics research communities to enable joint research on spatio-temporally aligned pose, volumetric, mRGBD and audio data cues. The dataset and its code are available https://tofis.github.io/myurls/human4d.


title: Mitigation of parasitic losses in the quadrupole resonator enabling direct measurements of low residual resistances of SRF samples

abstract: The quadrupole resonator (QPR) is a dedicated sample-test cavity for the RF characterization of superconducting samples in a wide temperature, RF field and frequency range. Its main purpose are high resolution measurements of the surface resistance with direct access to the residual resistance thanks to the low frequency of the first operating quadrupole mode. Besides the well-known high resolution of the QPR, a bias of measurement data towards higher values has been observed, especially at higher harmonic quadrupole modes. Numerical studies show that this can be explained by parasitic RF losses on the adapter flange used to mount samples into the QPR. Coating several micrometer of niobium on those surfaces of the stainless steel flange that are exposed to the RF fields significantly reduced this bias, enabling a direct measurement of a residual resistance smaller than 5 n$\Omega$ at 2 K and 413 MHz. A constant correction based on simulations was not feasible due to deviations from one measurement to another. However, this issue is resolved given these new results.


title: Fano manifolds with big tangent bundle: a characterisation of $V_5$

abstract: Let X be a Fano manifold with Picard number one such that the tangent bundle T_X is big. If X admits a rational curve with trivial normal bundle, we show that X is isomorphic to the del Pezzo threefold of degree five.


title: On the difficulty of learning chaotic dynamics with RNNs

abstract: Recurrent neural networks (RNNs) are wide-spread machine learning tools for modeling sequential and time series data. They are notoriously hard to train because their loss gradients backpropagated in time tend to saturate or diverge during training. This is known as the exploding and vanishing gradient problem. Previous solutions to this issue either built on rather complicated, purpose-engineered architectures with gated memory buffers, or - more recently - imposed constraints that ensure convergence to a fixed point or restrict (the eigenspectrum of) the recurrence matrix. Such constraints, however, convey severe limitations on the expressivity of the RNN. Essential intrinsic dynamics such as multistability or chaos are disabled. This is inherently at disaccord with the chaotic nature of many, if not most, time series encountered in nature and society. It is particularly problematic in scientific applications where one aims to reconstruct the underlying dynamical system. Here we offer a comprehensive theoretical treatment of this problem by relating the loss gradients during RNN training to the Lyapunov spectrum of RNN-generated orbits. We mathematically prove that RNNs producing stable equilibrium or cyclic behavior have bounded gradients, whereas the gradients of RNNs with chaotic dynamics always diverge. Based on these analyses and insights we suggest ways of how to optimize the training process on chaotic data according to the system's Lyapunov spectrum, regardless of the employed RNN architecture.


title: Solving Large Break Minimization Problems in a Mirrored Double Round-robin Tournament Using Quantum Annealing

abstract: Quantum annealing (QA) has gained considerable attention because it can be applied to combinatorial optimization problems, which have numerous applications in logistics, scheduling, and finance. In recent years, research on solving practical combinatorial optimization problems using them has accelerated. However, researchers struggle to find practical combinatorial optimization problems, for which quantum annealers outperform other mathematical optimization solvers. Moreover, there are only a few studies that compare the performance of quantum annealers with one of the most sophisticated mathematical optimization solvers, such as Gurobi and CPLEX. In our study, we determine that QA demonstrates better performance than the solvers in the break minimization problem in a mirrored double round-robin tournament (MDRRT). We also explain the desirable performance of QA for the sparse interaction between variables and a problem without constraints. In this process, we demonstrate that the break minimization problem in an MDRRT can be expressed as a 4-regular graph. Through computational experiments, we solve this problem using our QA approach and two-integer programming approaches, which were performed using the latest quantum annealer D-Wave Advantage, and the sophisticated mathematical optimization solver, Gurobi, respectively. Further, we compare the quality of the solutions and the computational time. QA was able to determine the exact solution in 0.05 seconds for problems with 20 teams, which is a practical size. In the case of 36 teams, it took 84.8 s for the integer programming method to reach the objective function value, which was obtained by the quantum annealer in 0.05 s. These results not only present the break minimization problem in an MDRRT as an example of applying QA to practical optimization problems, but also contribute to find problems that can be effectively solved by QA.


title: Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans

abstract: Recursive processing is considered a hallmark of human linguistic abilities. A recent study evaluated recursive processing in recurrent neural language models (RNN-LMs) and showed that such models perform below chance level on embedded dependencies within nested constructions -- a prototypical example of recursion in natural language. Here, we study if state-of-the-art Transformer LMs do any better. We test four different Transformer LMs on two different types of nested constructions, which differ in whether the embedded (inner) dependency is short or long range. We find that Transformers achieve near-perfect performance on short-range embedded dependencies, significantly better than previous results reported for RNN-LMs and humans. However, on long-range embedded dependencies, Transformers' performance sharply drops below chance level. Remarkably, the addition of only three words to the embedded dependency caused Transformers to fall from near-perfect to below-chance performance. Taken together, our results reveal Transformers' shortcoming when it comes to recursive, structure-based, processing.


title: Siegel modular forms of degree two and level five

abstract: We construct a ring of meromorphic Siegel modular forms of degree 2 and level 5, with singularities supported on an arrangement of Humbert surfaces, which is generated by four singular theta lifts of weights 1, 1, 2, 2 and their Jacobian. We use this to prove that the ring of holomorphic Siegel modular forms of degree 2 and level $\Gamma_0(5)$ is minimally generated by eighteen modular forms of weights 2, 4, 4, 4, 4, 4, 6, 6, 6, 6, 10, 11, 11, 11, 13, 13, 13, 15.


title: Covariant derivatives for Ehresmann connections

abstract: We deal with the construction of covariant derivatives for some quite general Ehresmann connections on fibre bundles. We show how the introduction of a vertical endomorphism allows construction of covariant derivatives separately on both the vertical and horizontal distributions of the connection which can then be glued together on the total space. We give applications across an important class of tangent bundle cases, frame bundles and the Hopf bundle.


title: TRACK: A python code for calculating the transport properties of correlated electron systems using Kubo formalism

abstract: Exploring the transport properties of different materials brings new avenue for basic understanding of emergent phenomena and practical applications in many different fields. Here, we report a program named as TRACK (TRAnsport properties for Correlated materials using Kubo formalism) which is written in Python 3 for calculating temperature dependent electrical conductivity, electronic part of thermal conductivity, Seebeck coefficient and Lorenz number. In this code, Kubo linear-response formalism is utilized for computing these parameters using both interacting and non-interacting electronic structure methods. The formula for transport coefficients is accordingly modified to obtain the transport parameters under relaxation time approximation using band-theory. The basic inputs of this program are the structural information, dense k-points sampling in the irreducible part of the Brillouin zone and the information of velocity matrix elements, which can be calculated using third-party ab-initio package. TRACK is expected to calculate the transport properties of different class of materials. The code has been benchmarked by performing calculation on three different types of materials namely Vanadium (V), FeSi and LaCoO$_3$, which are metal, semiconductor and Mott insulator, respectively. The temperature dependent behaviour of the transport coefficients for these materials show fairly good agreement with the corresponding experimental data.


title: Building Chinese Biomedical Language Models via Multi-Level Text Discrimination

abstract: Pre-trained language models (PLMs), such as BERT and GPT, have revolutionized the field of NLP, not only in the general domain but also in the biomedical domain. Most prior efforts in building biomedical PLMs have resorted simply to domain adaptation and focused mainly on English. In this work we introduce eHealth, a Chinese biomedical PLM built from scratch with a new pre-training framework. This new framework pre-trains eHealth as a discriminator through both token- and sequence-level discrimination. The former is to detect input tokens corrupted by a generator and recover their original identities from plausible candidates, while the latter is to further distinguish corruptions of a same original sequence from those of others. As such, eHealth can learn language semantics at both token and sequence levels. Extensive experiments on 11 Chinese biomedical language understanding tasks of various forms verify the effectiveness and superiority of our approach. We release the pre-trained model at \url{https://github.com/PaddlePaddle/Research/tree/master/KG/eHealth} and will also release the code later.


title: Chern number matrix of the non-Abelian spin-singlet fractional quantum Hall effect

abstract: While the internal structure of Abelian topological order is well understood, how to characterize the non-Abelian topological order is an outstanding issue. We propose a distinctive scheme based on the many-body Chern number matrix to characterize non-Abelian multicomponent fractional quantum Hall states. As a concrete example, we study the many-body ground state of two-component bosons at the filling faction $\nu=4/3$ in topological flat band models. Utilizing density-matrix renormalization group and exact diagonalization calculations, we demonstrate the emergence of non-Abelian spin-singlet fractional quantum Hall effect under three-body interaction, whose topological nature is classified by six-fold degenerate ground states and a fractionally quantized Chern number matrix.


title: HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism

abstract: Recently, some challenging tasks in multi-agent systems have been solved by some hierarchical reinforcement learning methods. Inspired by the intra-level and inter-level coordination in the human nervous system, we propose a novel value decomposition framework HAVEN based on hierarchical reinforcement learning for fully cooperative multi-agent problems. To address the instability arising from the concurrent optimization of policies between various levels and agents, we introduce the dual coordination mechanism of inter-level and inter-agent strategies by designing reward functions in a two-level hierarchy. HAVEN does not require domain knowledge and pre-training, and can be applied to any value decomposition variant. Our method achieves desirable results on different decentralized partially observable Markov decision process domains and outperforms other popular multi-agent hierarchical reinforcement learning algorithms.


title: A blind spot in undergraduate mathematics: The circular definition of the length of the circle, and how it can be turned into an enlightening example

abstract: We highlight the fact that in undergraduate calculus, the number pi is defined via the length of the circle, the length of the circle is defined as a certain value of an inverse trigonometric function, and this value is defined via pi, thus forming a circular definition. We present a way in which this error can be rectified. We explain that this error is instructive and can be used as an enlightening topic for discussing different approaches to mathematics with undergraduate students.


title: Synchronization and Balancing around Simple Closed Polar Curves with Bounded Trajectories and Control Saturation

abstract: The problem of synchronization and balancing around simple closed polar curves is addressed for unicycle-type multi-agent systems. Leveraging the concept of barrier Lyapunov function in conjunction with bounded Lyapunov-like curve-phase potential functions, we propose distributed feedback control laws and show that the agents asymptotically stabilize to the desired closed curve, their trajectories remain bounded within a compact set, and their turn-rates adhere to the saturation limits. We also characterize the explicit nature of the boundary of this trajectory-constraining set based on the magnitude of the safe distance of the exterior boundary from the desired curve. We further establish a connection between the perimeters and areas of the trajectory-constraining set with that of the desired curve. We obtain bounds on different quantities of interest in the post-design analysis and provide simulation results to illustrate the theoretical findings.


title: Supernova Preshock Neutronization Burst as a Probe of Non-Standard Neutrino Interactions

abstract: Core-collapse supernova (CCSN) provides a unique astrophysical site for studying neutrino-matter interactions. Prior to the shock-breakout neutrino burst during the collapse of the iron core, a preshock $\nu_e$ burst arises from the electron capture of nuclei and it is sensitive to the low-energy coherent elastic neutrino-nucleus scattering (CE$\nu$NS) which dominates the neutrino opacity. Since the CE$\nu$NS depends strongly on the non-standard neutrino interactions (NSI) which are completely beyond the standard model and yet to be determined, the detection of the preshock burst thus provides a clean way to extract the NSI information. Within the spherically symmetric general-relativistic hydrodynamic simulation for the CCSN, we investigate the NSI effects on the preshock burst. We find that the NSI can maximally enhance the peak luminosity of the preshock burst almost by a factor of three, reaching a value comparable with that of the shock-breakout burst. Future detection of the preshock burst will have critical implications on astrophysics, neutrino physics and physics beyond the standard model.


title: A mathematical justification for metronomic chemotherapy in oncology

abstract: We mathematically justify metronomic chemotherapy as the best strategy to apply most cytotoxic drugs in oncology for both curative and palliative approaches, assuming the classical pharmacokinetic model together with the Emax pharmacodynamic and the Norton-Simon hypothesis. From the mathematical point of view, we will consider two mixed-integer nonlinear optimization problems, where the unknowns are the number of the doses and the quantity of each one, adjusting the administration times a posteriori


title: Superconductor-insulator transitions in three-dimensional indium-oxide at high pressures

abstract: Experiments investigating magnetic-field-tuned superconductor-insulator transition (HSIT) mostly focus on two-dimensional material systems where the transition and its proximate ground-state phases, often exhibit features that are seemingly at odds with the expected behavior. Here we present a complementary study of a three-dimensional pressure-packed amorphous indium-oxide (InOx) powder where granularity controls the HSIT. Above a low threshold pressure of ~0.2 GPa, vestiges of superconductivity are detected, although neither a true superconducting transition nor insulating behavior are observed. Instead, a saturation at very high resistivity at low pressure is followed by saturation at very low resistivity at higher pressure. We identify both as different manifestations of anomalous metallic phases dominated by superconducting fluctuations. By analogy with previous identification of the low resistance saturation as a "failed superconductor", our data suggests that the very high resistance saturation is a manifestation of a "failed insulator". Above a threshold pressure of ~6 GPa, the sample becomes fully packed, and superconductivity is robust, with TC tunable with pressure. A quantum critical point at PC~25 GPa marks the complete suppression of superconductivity. For a finite pressure below PC, a magnetic field is shown to induce a HSIT from a true zero-resistance superconducting state to a weakly insulating behavior. Determining the critical field, HC, we show that similar to the 2D behavior, the insulating-like state maintains a superconducting character, which is quenched at higher field, above which the magnetoresistance decreases to its fermionic normal state value.


title: On the classifiation of Landsberg spherically symmetric Finsler metrics

abstract: In this paper, as an application of the inverse problem of calculus of variations, we investigate two compatibility conditions on the spherically symmetric Finsler metrics. By making use of these conditions, we focus our attention on the Landsberg spherically symmetric Finsler metrics. We classify all spherically symmetric manifolds of Landsberg or Berwald types. For the higher dimensions $n\geq 3$, we prove that: all Landsberg spherically symmetric manifolds are either Riemannian or their geodesic sprays have a specific formula; all regular Landsberg spherically symmetric metrics are Riemannian; all (regular or non-regular) Berwald spherically symmetric metrics are Riemannian. Moreover, we establish new unicorns, i.e., new explicit examples of non-regular non-Berwaldian Landsberg metrics are obtained. For the two-dimensional case, we characterize all Berwald or Landsberg spherically symmetric surfaces.


title: Rethinking Point Cloud Filtering: A Non-Local Position Based Approach

abstract: Existing position based point cloud filtering methods can hardly preserve sharp geometric features. In this paper, we rethink point cloud filtering from a non-learning non-local non-normal perspective, and propose a novel position based approach for feature-preserving point cloud filtering. Unlike normal based techniques, our method does not require the normal information. The core idea is to first design a similarity metric to search the non-local similar patches of a queried local patch. We then map the non-local similar patches into a canonical space and aggregate the non-local information. The aggregated outcome (i.e. coordinate) will be inversely mapped into the original space. Our method is simple yet effective. Extensive experiments validate our method, and show that it generally outperforms position based methods (deep learning and non-learning), and generates better or comparable outcomes to normal based techniques (deep learning and non-learning).


title: Neutron transfer in $^{9}$Be + $^{159}$Tb system

abstract: One neutron stripping cross sections ($\sigma_{-1n}$) are measured in $^{9}$Be+$^{159}$Tb system in the energy range E$_{cm}$/V$_{B}$ $\sim$ 0.79 - 1.24 using offline gamma counting technique. The CRC model calculations including the ground state and the 2$^{+}$ resonance state of $^{8}$Be, carried out using the FRESCO code, give a reasonable description of the measured data. In addition, comparisons of reduced 1n-stripping cross sections- $\sigma_{red}$ with $^{9}$Be for different target nuclei (A $\sim$150-200), and $\sigma_{red}$ for $^{9}$Be, $^{6}$Li with $^{159}$Tb target are presented. While no strong target dependence is observed with $^{9}$Be projectile, $\sigma_{red}$($^{9}$Be) is significantly larger than that for $^{6}$Li, which is consistent with the Q-value for transfer reactions and breakup threshold energy of projectiles.


title: Evidence for multiple Ferrel-like cells on Jupiter

abstract: Jupiter's atmosphere is dominated by multiple jet streams which are strongly tied to its 3D atmospheric circulation. Lacking a rigid bottom boundary, several models exist for how the meridional circulation extends into the planetary interior. Here we show, collecting evidence from multiple instruments of the Juno mission, the existence of mid-latitudinal meridional circulation cells which are driven by turbulence, similar to the Ferrel cells on Earth. Different than Earth, which contains only one such cell in each hemisphere, the larger, faster rotating Jupiter can incorporate multiple cells. The cells form regions of upwelling and downwelling, which we show are clearly evident in Juno's microwave data between latitude 60S and 60N. The existence of these cells is confirmed by reproducing the ammonia observations using a simplistic model. This study solves a long-standing puzzle regarding the nature of Jupiter's sub-cloud dynamics and provides evidence for 8 cells in each Jovian hemisphere.


title: The Zwicky Transient Facility Type Ia supernova survey: first data release and results

abstract: Type Ia supernovae (SNe~Ia) in the nearby Hubble flow are excellent distance indicators in cosmology. The Zwicky Transient Facility (ZTF) has observed a large sample of supernovae from an untargeted, rolling survey, reaching $20.8, 20.6, 20.3$ mag in $g$ $r$, and $i$-band, respectively. With a FoV of 47 sq.deg, ZTF discovered $>$ 3000 SNe~Ia in a little over 2.5 years. Here, we report on the sample of 761 spectroscopically classified SNe~Ia from the first year of operations (DR1). The sample has a median redshift $\bar z =$ 0.057, nearly a factor of two higher than the current low-$z$ sample. Our sample has a total of 934 spectra, of which 632 were obtained with the robotic SEDm on Palomar P60. We assess the potential for precision cosmology for a total of 305 SNe with redshifts from host galaxy spectra. The sample is already comparable in size to the entire combined literature low-$z$ anchor sample. The median first detection is 13.5 days before maximum light, about 10 days earlier than the median in the literature. Furthermore, six SNe from our sample are at $D_L < 80$ Mpc, for which host galaxy distances can be obtained in the JWST era, such that we have calibrator and Hubble flow SNe observed with the same instrument. In the entire duration of ZTF-I, we have observed nearly fifty SNe for which we can obtain calibrator distances, key for percent level distance scale measurements.


title: $P$-associahedra

abstract: For each poset $P$, we construct a polytope $A(P)$ called the $P$-associahedron. Similarly to the case of graph associahedra, the faces of $A(P)$ correspond to certain nested collections of subsets of $P$. The Stasheff associahedron is a compactification of the configuration space of $n$ points on a line, and we recover $A(P)$ as an analogous compactification of the space of order-preserving maps $P\to\mathbb{R}$. Motivated by the study of totally nonnegative critical varieties in the Grassmannian, we introduce affine poset cyclohedra and realize these polytopes as compactifications of configuration spaces of $n$ points on a circle. For particular choices of (affine) posets, we obtain associahedra, cyclohedra, permutohedra, and type B permutohedra as special cases.


title: Constraints on Barrow entropy from M87* and S2 star observations

abstract: We use data from M87* central black hole shadow, as well as from the S2 star observations, in order to extract constraints on Barrow entropy. The latter is a modified entropy arising from quantum-gravitational effects on the black hole horizon, quantified by the new parameter $\Delta$. Such a change in entropy leads to a change in temperature, as well as to the properties of the black hole and its shadow. We investigate the photon sphere and the shadow of a black hole with Barrow entropy, and assuming a simple model for infalling and radiating gas we estimate the corresponding intensity. Furthermore, we use the radius in order to extract the real part of the quasinormal modes, and for completeness we investigate the spherical accretion of matter onto the black hole, focusing on isothermal and polytropic test fluids. We extract the allowed parameter region, and by applying a Monte-Carlo-Markov Chains analysis we find that $ \Delta \simeq 0.0036^{+0.0792}_{-0.0145}$. Hence, our results place the upper bound $\Delta\lesssim0.0828$ at 1$\sigma$, a constraint that is less strong than the Big Bang Nucleosynthesis one, but significantly stronger than the late-time cosmological constraints.


title: Label-free detection of single nanoparticles with disordered nanoisland surface plasmon sensor

abstract: We report sensing of single nanoparticles using disordered metallic nanoisland substrates supporting surface plasmon polaritons (SPPs). Speckle patterns arising from leakage radiation of elastically scattered SPPs provides a unique fingerprint of the scattering microstructure at the sensor surface. Experimental measurements of the speckle decorrelation are presented and shown to enable detection of sorption of individual gold nanoparticles and polystyrene beads. Our approach is verified through bright-field and fluorescence imaging of particles adhering to the nanoisland substrate.


title: Three regimes in the tribo-oxidation of high purity copper at temperatures of up to 150 $^\circ$C

abstract: Surface oxidation of high-purity copper is accelerated under tribological loading. Tribo-oxide formation at room temperature is associated with diffusion processes along defects, such as dislocations or grain boundaries. Here, we embark on investigating the additional influence of temperature on the tribo-oxidation of copper. Dry, reciprocating sliding tests were performed with a variation of the sample temperature between 21 - 150 $^\circ$C. Microstructural changes were monitored and analyzed with state-of-the-art electron microscopy techniques. Oxide layer formation through thermal oxidation was observed for 150 $^\circ$C, but not for lower temperatures. As the temperature increases from room temperature up to 100 $^\circ$C, a significantly stronger tribo-oxidation into deeper material layers and an increase in the amount of formed pores and oxides was detected. Up to 75 $^\circ$C, diffusional processes along grain boundaries and dislocation pipes were identified. Starting at 100 $^\circ$C, CuO was detected. Hence, tribological loading significantly alters the CuO formation in comparison with static oxidation. Along with the CuO formation at temperatures >= 90 $^\circ$C, the oxide layer thickness decreased while the friction coefficient increased. The observations broaden our understanding of the elementary mechanisms of tribo-oxidation in high-purity copper. Eventually, this will allow to systematically customize surfaces showing tribo-oxidation for specific tribological applications.


title: Probing a Finite Unified Theory with Reduced Couplings at Future Colliders

abstract: The search for relations among parameters that are renormalization group invariant to all orders in perturbation theory constitutes the basis of the reduction of couplings idea. Reduction of couplings can be achieved in $N=1$ Grand Unified Theories, few of which can become even all-loop finite. We review the basic idea and a resulting theory in which successful reduction of couplings has been achieved, namely the all-loop finite $N = 1$ supersymmetric $SU(5)$ model. We present three benchmark scenarios and investigate their observability at existing and future hadron colliders. The supersymmetric spectrum is found to be beyond the reach of the 14 TeV HL-LHC. In turn, it is found that large parts of the predicted spectrum can be tested at the 100 TeV FCC-hh, but the higher mass regions remain out of reach.


title: Proactive Mobility Management of UEs using Sequence-to-Sequence Modeling

abstract: Beyond 5G networks will operate at high frequencies with wide bandwidths. This brings both opportunities and challenges. Opportunities include high throughput connectivity with low latency. However, one of the main challenges in these networks is due to the high path loss at operating frequencies, which requires network to be deployed densely to provide coverage. Since these cells have small inter-site-distance (ISD), the dwell-time of the UEs in these cells are small, thus supporting mobility in these types of dense networks is a challenge and require frequent beam or cell reassignments. A pro-active mobility management scheme which exploits the trajectory can provide better prediction of cells and beams as UEs move in the coverage area. We propose an AI based method using sequence-to-sequence modeling for the estimation of handover cells/beams along with dwell-time using the trajectory information of the UE. Results indicate that for a dense deployment an accuracy of more than 90 percent can be achieved for handover cell estimation with very low mean absolute error (MAE) for dwell-time.


title: On $(p,q)$-eigenvalues of subelliptic operators on nilpotent Lie groups

abstract: In the article we study the Dirichlet $(p,q)$-eigenvalue problem for subelliptic non-commutative operators on nilpotent Lie groups. We prove solvability of this eigenvalue problem and existence of the minimizer of the corresponding variational problem.


title: An upper bound on the dimension of the Rauzy gasket

abstract: In this note, we give an elementary proof that the Hausdorff dimension of the Rauzy gasket is at most 1.7407, improving upon results of Avila et al. and Fougeron.


title: Divide-and-Conquer Fusion

abstract: Combining several (sample approximations of) distributions, which we term sub-posteriors, into a single distribution proportional to their product, is a common challenge. Occurring, for instance, in distributed 'big data' problems, or when working under multi-party privacy constraints. Many existing approaches resort to approximating the individual sub-posteriors for practical necessity, then find either an analytical approximation or sample approximation of the resulting (product-pooled) posterior. The quality of the posterior approximation for these approaches is poor when the sub-posteriors fall out-with a narrow range of distributional form, such as being approximately Gaussian. Recently, a Fusion approach has been proposed which finds an exact Monte Carlo approximation of the posterior, circumventing the drawbacks of approximate approaches. Unfortunately, existing Fusion approaches have a number of computational limitations, particularly when unifying a large number of sub-posteriors. In this paper, we generalise the theory underpinning existing Fusion approaches, and embed the resulting methodology within a recursive divide-and-conquer sequential Monte Carlo paradigm. This ultimately leads to a competitive Fusion approach, which is robust to increasing numbers of sub-posteriors.


title: Lorentz-boost-driven magneto-optics in a Dirac nodal-line semimetal

abstract: Optical response of crystalline solids is to a large extent driven by excitations that promote electrons among individual bands. This allows one to apply optical and magneto-optical methods to determine experimentally the energy band gap - a fundamental property crucial to our understanding of any solid - with a great precision. Here we show that such conventional methods, applied with great success to many materials in the past, do not work in topological Dirac semimetals with a dispersive nodal line. There, the optically deduced band gap depends on how the magnetic field is oriented with respect to the crystal axes. Such highly unusual behaviour is explained in terms of band-gap renormalization driven by Lorentz boosts.


title: The role of density in the energy conservation for the isentropic compressible Euler equations

abstract: In this paper, we study Onsager's conjecture on the energy conservation for the isentropic compressible Euler equations via establishing the energy conservation criterion involving the density $\varrho\in L^{k}(0,T;L^{l}(\mathbb{T}^{d}))$. The motivation is to analysis the role of the integrability of density of the weak solutions keeping energy in this system, since almost all known corresponding results require $\varrho\in L^{\infty}(0,T;L^{\infty}(\mathbb{T}^{d}))$. Our results imply that the lower integrability of the density $\varrho$ means that more integrability of the velocity $v$ are necessary in energy conservation and the inverse is also true. The proof relies on the Constantin-E-Titi type and Lions type commutators on mollifying kernel.


title: Optimality conditions, approximate stationarity, and applications -- a story beyond Lipschitzness

abstract: Approximate necessary optimality conditions in terms of Fr\'echet subgradients and normals for a rather general optimization problem with a potentially non-Lipschitzian objective function are established with the aid of Ekeland's variational principle, the fuzzy Fr\'echet subdifferential sum rule, and a novel notion of lower semicontinuity relative to a set-valued mapping or set. Feasible points satisfying these optimality conditions are referred to as approximately stationary. As applications, we derive a new general version of the extremal principle. Furthermore, we study approximate stationarity conditions for an optimization problem with a composite objective function and geometric constraints, a qualification condition guaranteeing that approximately stationary points of such a problem are M-stationary, and a multiplier-penalty-method which naturally computes approximately stationary points of the underlying problem. Finally, necessary optimality conditions for an optimal control problem with a non-Lipschitzian sparsity-promoting term in the objective function are established.


title: Momentum-Based Nash Set Seeking over Networks via Multi-Time Scale Hybrid Dynamic Inclusions

abstract: Multi-time scale techniques, such as singular perturbations and averaging theory, have played an essential role in the development of distributed Nash equilibrium-seeking algorithms for network systems. Such techniques intrinsically rely on the uniform asymptotic stability properties of the dynamics that evolve in each time scale of the closed-loop system. When such properties are absent, synthesizing multi-time scale Nash equilibrium-seeking algorithms is more challenging and requires additional regularization mechanisms. In this paper, we investigate the synthesis and analysis of these mechanisms in the context of accelerated pseudogradient flows with time-varying damping in non-cooperative games. Specifically, we introduce a new class of distributed and hybrid Nash set-seeking (NSS) algorithms that synergistically combine dynamic momentum-based flows with coordinated discrete-time resets. The reset mechanisms can be seen as restarting techniques that allow individual players to choose their own momentum restarting policy to achieve better transient performance. The resulting closed-loop system is modeled as a hybrid dynamic inclusion, which is analyzed using tools from hybrid dynamical system's theory. Our algorithms are developed for potential games, as well as for monotone games for which a potential function does not exist. They can be implemented in games where players have access to gradient Oracles with full or partial information, as well as in games where players have access only to measurements of their costs. In the latter case, we use tools from hybrid extremum seeking control to achieve accelerated model-free Nash set seeking.


title: Spherical polar coordinate transformation for integration of singular functions on tetrahedra

abstract: A method is presented for the evaluation of integrals on tetrahedra where the integrand has an integrable singularity at one vertex. The approach uses a transformation to spherical polar coordinates which explicitly eliminates the singularity and facilitates the evaluation of integration limits. The method is also implemented in an adaptive form which gives convergence to a required tolerance. Results from the method are compared to the output from an exact analytical method for one tetrahedron and show high accuracy. In particular, when the adaptive algorithm is used, highly accurate results are found for poorly conditioned tetrahedra which normally present difficulties for numerical quadrature techniques. The approach is also demonstrated for evaluation of the Biot-Savart integral on an unstructured mesh in combination with a fixed node quadrature rule and demonstrates good convergence and accuracy.


title: Proofs, generalizations and analogs of Menon's identity: a survey

abstract: Menon's identity states that for every positive integer $n$ one has $\sum (a-1,n) = \varphi(n) \tau(n)$, where $a$ runs through a reduced residue system (mod $n$), $(a-1,n)$ stands for the greatest common divisor of $a-1$ and $n$, $\varphi(n)$ is Euler's totient function and $\tau(n)$ is the number of divisors of $n$. Menon's identity has been the subject of many research papers, also in the last years. We present detailed, self contained proofs of this identity by using different methods, and point out those that we could not identify in the literature. We survey the generalizations and analogs, and overview the results and proofs given by Menon in his original paper. Some historical remarks and an updated list of references are included as well.


title: Relighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptation

abstract: The modern supervised approaches for human image relighting rely on training data generated from 3D human models. However, such datasets are often small (e.g., Light Stage data with a small number of individuals) or limited to diffuse materials (e.g., commercial 3D scanned human models). Thus, the human relighting techniques suffer from the poor generalization capability and synthetic-to-real domain gap. In this paper, we propose a two-stage method for single-image human relighting with domain adaptation. In the first stage, we train a neural network for diffuse-only relighting. In the second stage, we train another network for enhancing non-diffuse reflection by learning residuals between real photos and images reconstructed by the diffuse-only network. Thanks to the second stage, we can achieve higher generalization capability against various cloth textures, while reducing the domain gap. Furthermore, to handle input videos, we integrate illumination-aware deep video prior to greatly reduce flickering artifacts even with challenging settings under dynamic illuminations.


title: Unwinding toric degenerations and mirror symmetry for Grassmannians

abstract: The most fundamental example of mirror symmetry compares the Fermat hypersurfaces in P^n and P^n/G, where G is a finite group that acts on P^n and preserves the Fermat hypersurface. We generalise this to hypersurfaces in Grassmannians, where the picture is richer and more complex. There is a finite group G that acts on the Grassmannian Gr(n,r) and preserves an appropriate Calabi-Yau hypersurface. We establish how mirror symmetry, toric degenerations, blow-ups and variation of GIT relate the Calabi-Yau hypersurfaces inside Gr(n,r) and Gr(n,r)/G. This allows us to describe a compactification of the Eguchi-Hori-Xiong mirror to the Grassmannian, inside a blow-up of the quotient of the Grassmannian by G.


title: An Approach to Mispronunciation Detection and Diagnosis with Acoustic, Phonetic and Linguistic (APL) Embeddings

abstract: Many mispronunciation detection and diagnosis (MD&D) research approaches try to exploit both the acoustic and linguistic features as input. Yet the improvement of the performance is limited, partially due to the shortage of large amount annotated training data at the phoneme level. Phonetic embeddings, extracted from ASR models trained with huge amount of word level annotations, can serve as a good representation of the content of input speech, in a noise-robust and speaker-independent manner. These embeddings, when used as implicit phonetic supplementary information, can alleviate the data shortage of explicit phoneme annotations. We propose to utilize Acoustic, Phonetic and Linguistic (APL) embedding features jointly for building a more powerful MD&D system. Experimental results obtained on the L2-ARCTIC database show the proposed approach outperforms the baseline by 9.93%, 10.13% and 6.17% on the detection accuracy, diagnosis error rate and the F-measure, respectively.


title: Order Constraints in Optimal Transport

abstract: Optimal transport is a framework for comparing measures whereby a cost is incurred for transporting one measure to another. Recent works have aimed to improve optimal transport plans through the introduction of various forms of structure. We introduce novel order constraints into the optimal transport formulation to allow for the incorporation of structure. We define an efficient method for obtaining explainable solutions to the new formulation that scales far better than standard approaches. The theoretical properties of the method are provided. We demonstrate experimentally that order constraints improve explainability using the e-SNLI (Stanford Natural Language Inference) dataset that includes human-annotated rationales as well as on several image color transfer examples.


title: Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning

abstract: Continual Learning (CL) is an emerging machine learning paradigm that aims to learn from a continuous stream of tasks without forgetting knowledge learned from the previous tasks. To avoid performance decrease caused by forgetting, prior studies exploit episodic memory (EM), which stores a subset of the past observed samples while learning from new non-i.i.d. data. Despite the promising results, since CL is often assumed to execute on mobile or IoT devices, the EM size is bounded by the small hardware memory capacity and makes it infeasible to meet the accuracy requirements for real-world applications. Specifically, all prior CL methods discard samples overflowed from the EM and can never retrieve them back for subsequent training steps, incurring loss of information that would exacerbate catastrophic forgetting. We explore a novel hierarchical EM management strategy to address the forgetting issue. In particular, in mobile and IoT devices, real-time data can be stored not just in high-speed RAMs but in internal storage devices as well, which offer significantly larger capacity than the RAMs. Based on this insight, we propose to exploit the abundant storage to preserve past experiences and alleviate the forgetting by allowing CL to efficiently migrate samples between memory and storage without being interfered by the slow access speed of the storage. We call it Carousel Memory (CarM). As CarM is complementary to existing CL methods, we conduct extensive evaluations of our method with seven popular CL methods and show that CarM significantly improves the accuracy of the methods across different settings by large margins in final average accuracy (up to 28.4%) while retaining the same training efficiency.