title: deepNIR: Datasets for generating synthetic NIR images and improved fruit detection system using deep learning techniques

abstract: This paper presents datasets utilised for synthetic near-infrared (NIR) image generation and bounding-box level fruit detection systems. It is undeniable that high-calibre machine learning frameworks such as Tensorflow or Pytorch, and large-scale ImageNet or COCO datasets with the aid of accelerated GPU hardware have pushed the limit of machine learning techniques for more than decades. Among these breakthroughs, a high-quality dataset is one of the essential building blocks that can lead to success in model generalisation and the deployment of data-driven deep neural networks. In particular, synthetic data generation tasks often require more training samples than other supervised approaches. Therefore, in this paper, we share the NIR+RGB datasets that are re-processed from two public datasets (i.e., nirscene and SEN12MS) and our novel NIR+RGB sweet pepper(capsicum) dataset. We quantitatively and qualitatively demonstrate that these NIR+RGB datasets are sufficient to be used for synthetic NIR image generation. We achieved Frechet Inception Distance (FID) of 11.36, 26.53, and 40.15 for nirscene1, SEN12MS, and sweet pepper datasets respectively. In addition, we release manual annotations of 11 fruit bounding boxes that can be exported as various formats using cloud service. Four newly added fruits [blueberry, cherry, kiwi, and wheat] compound 11 novel bounding box datasets on top of our previous work presented in the deepFruits project [apple, avocado, capsicum, mango, orange, rockmelon, strawberry]. The total number of bounding box instances of the dataset is 162k and it is ready to use from cloud service. For the evaluation of the dataset, Yolov5 single stage detector is exploited and reported impressive mean-average-precision,mAP[0.5:0.95] results of[min:0.49, max:0.812]. We hope these datasets are useful and serve as a baseline for the future studies.


title: Mass and spin of Kerr black holes in terms of observational quantities: The dragging effect on the redshift

abstract: In this work, we elaborate on the development of a general relativistic formalism that allows one to analytically express the mass and spin parameters of the Kerr black hole in terms of observational data: the total redshift and blueshift of photons emitted by geodesic massive particles revolving the black hole and their orbital parameters. Thus, we present concise closed formulas for the mass and spin parameters of the Kerr black hole in terms of few directly observed quantities in the case of equatorial circular orbits either when the black hole is static or is moving with respect to a distant observer. Furthermore, we incorporate the gravitational dragging effect generated by the rotating nature of the Kerr black hole into the analysis and elucidate its non-trivial contribution to the expression for the light bending parameter and the frequency shifts of photons emitted by orbiting particles that renders simple symmetric expressions for the kinematic redshift and blueshift. We also incorporate the dependency of the frequency shift on the azimuthal angle, a fact that allows one to express the total redshift/blueshift along any point of the orbit of the revolving particle for the cases when the black hole is both static or moving with respect to us. These formulas allow one to compute the Kerr black hole parameters by applying this general relativistic formalism to astrophysical systems like the megamaser accretion disks orbiting supermassive black holes at the core of active galactic nuclei. Our results open a new window to implement parameter estimation studies to constrain black hole variables, and they can be generalized to black hole solutions beyond Einstein gravity.


title: Semantic-aligned Fusion Transformer for One-shot Object Detection

abstract: One-shot object detection aims at detecting novel objects according to merely one given instance. With extreme data scarcity, current approaches explore various feature fusions to obtain directly transferable meta-knowledge. Yet, their performances are often unsatisfactory. In this paper, we attribute this to inappropriate correlation methods that misalign query-support semantics by overlooking spatial structures and scale variances. Upon analysis, we leverage the attention mechanism and propose a simple but effective architecture named Semantic-aligned Fusion Transformer (SaFT) to resolve these issues. Specifically, we equip SaFT with a vertical fusion module (VFM) for cross-scale semantic enhancement and a horizontal fusion module (HFM) for cross-sample feature fusion. Together, they broaden the vision for each feature point from the support to a whole augmented feature pyramid from the query, facilitating semantic-aligned associations. Extensive experiments on multiple benchmarks demonstrate the superiority of our framework. Without fine-tuning on novel classes, it brings significant performance gains to one-stage baselines, lifting state-of-the-art results to a higher level.


title: Snowmass 2021 Accelerator Frontier White Paper: Near Term Applications driven by Advanced Accelerator Concepts

abstract: While the long-term vision of the advanced accelerator community is aimed at addressing the challenges of future collider technology, it is critical that the community takes advantage of the opportunity to make large societal impact through its near-term applications. In turn, enabling robust applications strengthens the quality, control, and reliability of the underlying accelerator infrastructure. The white paper contributions that are solicited here will summarize the near-term applications ideas presented by the advanced accelerator community, assessing their potential impact, discussing scientific and technical readiness of concepts, and providing a timeline for implementation.


title: Automating Code Review Activities by Large-Scale Pre-training

abstract: Code review is an essential part to software development lifecycle since it aims at guaranteeing the quality of codes. Modern code review activities necessitate developers viewing, understanding and even running the programs to assess logic, functionality, latency, style and other factors. It turns out that developers have to spend far too much time reviewing the code of their peers. Accordingly, it is in significant demand to automate the code review process. In this research, we focus on utilizing pre-training techniques for the tasks in the code review scenario. We collect a large-scale dataset of real-world code changes and code reviews from open-source projects in nine of the most popular programming languages. To better understand code diffs and reviews, we propose CodeReviewer, a pre-trained model that utilizes four pre-training tasks tailored specifically for the code review scenario. To evaluate our model, we focus on three key tasks related to code review activities, including code change quality estimation, review comment generation and code refinement. Furthermore, we establish a high-quality benchmark dataset based on our collected data for these three tasks and conduct comprehensive experiments on it. The experimental results demonstrate that our model outperforms the previous state-of-the-art pre-training approaches in all tasks. Further analysis show that our proposed pre-training tasks and the multilingual pre-training dataset benefit the model on the understanding of code changes and reviews.


title: DeepAD: A Robust Deep Learning Model of Alzheimer's Disease Progression for Real-World Clinical Applications

abstract: The ability to predict the future trajectory of a patient is a key step toward the development of therapeutics for complex diseases such as Alzheimer's disease (AD). However, most machine learning approaches developed for prediction of disease progression are either single-task or single-modality models, which can not be directly adopted to our setting involving multi-task learning with high dimensional images. Moreover, most of those approaches are trained on a single dataset (i.e. cohort), which can not be generalized to other cohorts. We propose a novel multimodal multi-task deep learning model to predict AD progression by analyzing longitudinal clinical and neuroimaging data from multiple cohorts. Our proposed model integrates high dimensional MRI features from a 3D convolutional neural network with other data modalities, including clinical and demographic information, to predict the future trajectory of patients. Our model employs an adversarial loss to alleviate the study-specific imaging bias, in particular the inter-study domain shifts. In addition, a Sharpness-Aware Minimization (SAM) optimization technique is applied to further improve model generalization. The proposed model is trained and tested on various datasets in order to evaluate and validate the results. Our results showed that 1) our model yields significant improvement over the baseline models, and 2) models using extracted neuroimaging features from 3D convolutional neural network outperform the same models when applied to MRI-derived volumetric features.


title: On the dynamics of shallow ice sheets. Modelling and analysis

abstract: We formulate a model describing the evolution of thickness in a shallow ice sheet lying over a lithosphere. The model is thus governed by a set of variational inequalities that involve nonlinearities in the time derivative and in the elliptic term. Solutions are constructed via a finite difference scheme in time.


title: TMS: A Temporal Multi-scale Backbone Design for Speaker Embedding

abstract: Speaker embedding is an important front-end module to explore discriminative speaker features for many speech applications where speaker information is needed. Current SOTA backbone networks for speaker embedding are designed to aggregate multi-scale features from an utterance with multi-branch network architectures for speaker representation. However, naively adding many branches of multi-scale features with the simple fully convolutional operation could not efficiently improve the performance due to the rapid increase of model parameters and computational complexity. Therefore, in the most current state-of-the-art network architectures, only a few branches corresponding to a limited number of temporal scales could be designed for speaker embeddings. To address this problem, in this paper, we propose an effective temporal multi-scale (TMS) model where multi-scale branches could be efficiently designed in a speaker embedding network almost without increasing computational costs. The new model is based on the conventional TDNN, where the network architecture is smartly separated into two modeling operators: a channel-modeling operator and a temporal multi-branch modeling operator. Adding temporal multi-scale in the temporal multi-branch operator needs only a little bit increase of the number of parameters, and thus save more computational budget for adding more branches with large temporal scales. Moreover, in the inference stage, we further developed a systemic re-parameterization method to convert the TMS-based model into a single-path-based topology in order to increase inference speed. We investigated the performance of the new TMS method for automatic speaker verification (ASV) on in-domain and out-of-domain conditions. Results show that the TMS-based model obtained a significant increase in the performance over the SOTA ASV models, meanwhile, had a faster inference speed.


title: Snowmass White Paper: String Perturbation Theory

abstract: The purpose of this White Paper is to review recent progress towards elucidating and evaluating string amplitudes, relating them to quantum field theory amplitudes, applying their predictions to string dualities, exploring their connection with gravitational physics, and deepening our understanding of their mathematical structure. We also present a selection of targets for future research.


title: PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation

abstract: Despite recent progress of pre-trained language models on generating fluent text, existing methods still suffer from incoherence problems in long-form text generation tasks that require proper content control and planning to form a coherent high-level logical flow. In this work, we propose PLANET, a novel generation framework leveraging autoregressive self-attention mechanism to conduct content planning and surface realization dynamically. To guide the generation of output sentences, our framework enriches the Transformer decoder with latent representations to maintain sentence-level semantic plans grounded by bag-of-words. Moreover, we introduce a new coherence-based contrastive learning objective to further improve the coherence of output. Extensive experiments are conducted on two challenging long-form text generation tasks including counterargument generation and opinion article generation. Both automatic and human evaluations show that our method significantly outperforms strong baselines and generates more coherent texts with richer contents.


title: RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction

abstract: Despite the importance of relation extraction in building and representing knowledge, less research is focused on generalizing to unseen relations types. We introduce the task setting of Zero-Shot Relation Triplet Extraction (ZeroRTE) to encourage further research in low-resource relation extraction methods. Given an input sentence, each extracted triplet consists of the head entity, relation label, and tail entity where the relation label is not seen at the training stage. To solve ZeroRTE, we propose to synthesize relation examples by prompting language models to generate structured texts. Concretely, we unify language model prompts and structured text approaches to design a structured prompt template for generating synthetic relation samples when conditioning on relation label prompts (RelationPrompt). To overcome the limitation for extracting multiple relation triplets in a sentence, we design a novel Triplet Search Decoding method. Experiments on FewRel and Wiki-ZSL datasets show the efficacy of RelationPrompt for the ZeroRTE task and zero-shot relation classification. Our code and data are available at github.com/declare-lab/RelationPrompt.


title: Rough Collisions

abstract: A rough collision law describes the limiting contact dynamics of a pair of rough rigid bodies, as the scale of the rough features (asperities) on the surface of each body goes to zero. The class of rough collision laws is quite large and includes random elements. Our main results characterize the rough collision laws for a freely moving rough disk and a fixed rough wall in dimension 2. Any collision law which (i) is symmetric with respect to a certain well-known invariant measure from billiards theory, and (ii) conserves the projection of the phase space velocity onto the "rolling velocity" is a rough collision law. We also provide a method for explicitly constructing rough collision laws for a broad range of choices of microstructure on the disk and wall. In our introduction, we review past work in billiards, including characterizations of other rough billiard systems, which our results build upon.


title: Knowledge Graph-Enabled Text-Based Automatic Personality Prediction

abstract: How people think, feel, and behave, primarily is a representation of their personality characteristics. By being conscious of personality characteristics of individuals whom we are dealing with or decided to deal with, one can competently ameliorate the relationship, regardless of its type. With the rise of Internet-based communication infrastructures (social networks, forums, etc.), a considerable amount of human communications take place there. The most prominent tool in such communications, is the language in written and spoken form that adroitly encodes all those essential personality characteristics of individuals. Text-based Automatic Personality Prediction (APP) is the automated forecasting of the personality of individuals based on the generated/exchanged text contents. This paper presents a novel knowledge graph-enabled approach to text-based APP that relies on the Big Five personality traits. To this end, given a text a knowledge graph which is a set of interlinked descriptions of concepts, was built through matching the input text's concepts with DBpedia knowledge base entries. Then, due to achieving more powerful representation the graph was enriched with the DBpedia ontology, NRC Emotion Intensity Lexicon, and MRC psycholinguistic database information. Afterwards, the knowledge graph which is now a knowledgeable alternative for the input text was embedded to yield an embedding matrix. Finally, to perform personality predictions the resulting embedding matrix was fed to four suggested deep learning models independently, which are based on convolutional neural network (CNN), simple recurrent neural network (RNN), long short term memory (LSTM) and bidirectional long short term memory (BiLSTM). The results indicated a considerable improvements in prediction accuracies in all of the suggested classifiers.


title: Ultrafast modulation of the molten metal surface tension under femtosecond laser irradiation

abstract: We predict ultrafast modulation of the pure molten metal surface stress fields under the irradiation of the single femtosecond laser pulse through the two-temperature model molecular-dynamics simulations. High-resolution and precision calculations are used to resolve the ultrafast laser-induced anisotropic relaxations of the pressure components on the time-scale comparable to the intrinsic liquid density relaxation time. The magnitudes of the dynamic surface tensions are found being modulated sharply within picoseconds after the irradiation, due to the development of the nanometer scale non-hydrostatic regime behind the exterior atomic layer of the liquid surfaces. The reported novel regulation mechanism of the liquid surface stress field and the dynamic surface tension hints at levitating the manipulation of liquid surfaces, such as ultrafast steering the surface directional transport and patterning.


title: On Modular Cohomotopy Groups

abstract: Let $p$ be a prime and let $\pi^n(X;\mathbb{Z}/p^r)=[X,M_n(\mathbb{Z}/p^r)]$ be the set of homotopy classes of based maps from CW-complexes $X$ into the mod $p^r$ Moore spaces $M_n(\mathbb{Z}/p^r)$ of degree $n$, where $\mathbb{Z}/p^r$ denotes the integers mod $p^r$. In this paper we firstly determine the modular cohomotopy groups $\pi^n(X;\mathbb{Z}/p^r)$ up to extensions by classical methods of primary cohomology operations and give conditions for the splitness of the extensions. Secondly we utilize some unstable homotopy theory of Moore spaces to study the modular cohomotopy groups; especially, the group $\pi^3(X;\mathbb{Z}_{(2)})$ with $\dim(X)\leq 6$ is determined.


title: Parameterized and Exact Algorithms for Class Domination Coloring

abstract: A class domination coloring (also called cd-Coloring or dominated coloring) of a graph is a proper coloring in which every color class is contained in the neighbourhood of some vertex. The minimum number of colors required for any cd-coloring of $G$, denoted by $\chi_{cd}(G)$, is called the class domination chromatic number (cd-chromatic number) of $G$. In this work, we consider two problems associated with the cd-coloring of a graph in the context of exact exponential-time algorithms and parameterized complexity. (1) Given a graph $G$ on $n$ vertices, find its cd-chromatic number. (2) Given a graph $G$ and integers $k$ and $q$, can we delete at most $k$ vertices such that the cd-chromatic number of the resulting graph is at most $q$? For the first problem, we give an exact algorithm with running time $\Oh(2^n n^4 \log n)$. Also, we show that the problem is \FPT\ with respect to the number $q$ of colors as the parameter on chordal graphs. On graphs of girth at least 5, we show that the problem also admits a kernel with $\Oh(q^3)$ vertices. For the second (deletion) problem, we show \NP-hardness for each $q \geq 2$. Further, on split graphs, we show that the problem is \NP-hard if $q$ is a part of the input and \FPT\ with respect to $k$ and $q$ as combined parameters. As recognizing graphs with cd-chromatic number at most $q$ is \NP-hard in general for $q \geq 4$, the deletion problem is unlikely to be \FPT\ when parameterized by the size of the deletion set on general graphs. We show fixed parameter tractability for $q \in \{2,3\}$ using the known algorithms for finding a vertex cover and an odd cycle transversal as subroutines.


title: A note about rational surfaces as unions of affine planes

abstract: We prove that any smooth rational projective surface over the field of complex numbers has an open covering consisting of 3 subsets isomorphic to affine planes.


title: Thick hyperbolic repelling invariant Cantor set and wild attractor

abstract: Let $D$ be the set of $\beta \in (1, 2]$ such that $f_\beta$ is a symmetric tent map with finite critical orbit. For $\beta \in D$, by operating Denjoy like surgery on $f_{\beta}$, we constructed a $C^1$ unimodal map $\tilde{g}_\beta$ admitting a thick hyperbolic repelling invariant Cantor set which contains a wild Cantor attractor. The smoothness of $\tilde{g}_\beta$ is ensured by the effective estimation of the preimages of the critical point as well as the prescribed lengths of the inserted intervals. Furthermore, $D$ is dense in $(1, 2]$, and $\tilde{g}_\beta$ can not be $C^{1+\alpha}$ because the hyperbolic repelling invariant Cantor set of $C^{1+\alpha}$ map has Lebesgue measure equal to zero.


title: Community-Driven Comprehensive Scientific Paper Summarization: Insight from cvpaper.challenge

abstract: The present paper introduces a group activity involving writing summaries of conference proceedings by volunteer participants. The rapid increase in scientific papers is a heavy burden for researchers, especially non-native speakers, who need to survey scientific literature. To alleviate this problem, we organized a group of non-native English speakers to write summaries of papers presented at a computer vision conference to share the knowledge of the papers read by the group. We summarized a total of 2,000 papers presented at the Conference on Computer Vision and Pattern Recognition, a top-tier conference on computer vision, in 2019 and 2020. We quantitatively analyzed participants' selection regarding which papers they read among the many available papers. The experimental results suggest that we can summarize a wide range of papers without asking participants to read papers unrelated to their interests.


title: Grow-up of a Filament Channel by Intermittent Small-scale Magnetic Reconnection

abstract: Filament channel (FC), a plasma volume where the magnetic field is primarily aligned with the polarity inversion line, is believed to be the pre-eruptive configuration of coronal mass ejections. Nevertheless, evidence for how the FC is formed is still elusive. In this paper, we present a detailed study on the build-up of a FC to understand its formation mechanism. The New Vacuum Solar Telescope of Yunnan Observatories and Optical and Near-Infrared Solar Eruption Tracer of Nanjing University, as well as the AIA and HMI on board Solar Dynamics Observatory are used to study the grow-up process of the FC. Furthermore, we reconstruct the non-linear force-free field (NLFFF) of the active region using the regularized Biot-Savart laws (RBSL) and magnetofrictional method to reveal three-dimension (3D) magnetic field properties of the FC. We find that partial filament materials are quickly transferred to longer magnetic field lines formed by small-scale magnetic reconnection, as evidenced by dot-like H{\alpha}/EUV brightenings and subsequent bidirectional outflow jets, as well as untwisting motions. The H{\alpha}/EUV bursts appear repeatedly at the same location and are closely associated with flux cancellation, which occurs between two small-scale opposite polarities and is driven by shearing and converging motions. The 3D NLFFF model reveals that the reconnection takes place in a hyperbolic flux tube that is located above the flux cancellation site and below the FC. The FC is gradually built up toward a twisted flux rope via series of small-scale reconnection events that occur intermittently prior to the eruption.


title: Electromagnetic moments of scandium isotopes and $N=28$ isotones in the distinctive $0f_{7/2}$ orbit

abstract: The electric quadrupole moment of $^{49}$Sc was measured by collinear laser spectroscopy at CERN-ISOLDE to be $Q_{\rm s}=-0.159(8)$ $e$b, and a nearly tenfold improvement in precision was reached for the electromagnetic moments of $^{47,49}$Sc. The single-particle behavior and nucleon-nucleon correlations are investigated with the electromagnetic moments of $Z=21$ isotopes and $N=28$ isotones as valence neutrons and protons fill the distinctive $0f_{7/2}$ orbit, respectively, located between magic numbers, 20 and 28. The experimental data are interpreted with shell-model calculations using an effective interaction, and ab-initio valence-space in-medium similarity renormalization group calculations based on chiral interactions. These results highlight the sensitivity of nuclear electromagnetic moments to different types of nucleon-nucleon correlations, and establish an important benchmark for further developments of theoretical calculations.


title: Nuclear Modified Transverse Momentum Distributions and 3D Imaging in Nuclei

abstract: In this proceeding, we review our recent work in which we performed the first global analysis of nuclear modified Transverse Momentum Distribution Functions (TMDs). We demonstrate for the first time that the global set of TMD experimental data from HERMES, E866, E772, RHIC, ATLAS, and CMS, can be described using a simple model which accounts for the nuclear modifications for the TMDs as a non-perturbative correction. Using this model, we extract the nuclear modified TMDs for the first time.


title: Local Hard-Sphere Poisson-Nernst-Planck Models for Ionic Channels with Permanent Charges

abstract: The main goal of this work is to examine the qualitative effect of ion sizes via a steady-state boundary value problem. We study a one-dimensional version of a Poisson-Nernst-Planck system with a local hard-sphere potential model for ionic flow through a membrane channel with fixed boundary ion concentrations and electric potentials. A complete set of integrals for the inner system is illustrated that delivers information for boundary and internal layers. In addition, a group of simultaneous equations appears in the construction of singular orbits. The research aims to set up a simple formation defined by the profile of permanent charges with two mobile ion species, one positively charged, cation, and one negatively charged, anion. A local hard-sphere potential that depends pointwise on ion concentrations is included in the model to estimate ion-size impacts on the ionic flow. The analysis is built on the geometric singular perturbation theory, particularly on specific structures of this concrete model. For 1:1 ionic mixtures, we first conduct precise mathematical analysis and derive a matching system of nonlinear algebraic equations. We then extend the results by directing on a critical case where the current is zero. Treating the ion sizes as small parameters, we derive an approximation of zero-current fluxes for general values of permanent charge. We then focus on small values of permanent charge to obtain more concrete outputs. We will also examine ion size effects on the flow rate of matter for the zero-current case.


title: Causal Robot Communication Inspired by Observational Learning Insights

abstract: Autonomous robots must communicate about their decisions to gain trust and acceptance. When doing so, robots must determine which actions are causal, i.e., which directly give rise to the desired outcome, so that these actions can be included in explanations. In behavior learning in psychology, this sort of reasoning during an action sequence has been studied extensively in the context of imitation learning. And yet, these techniques and empirical insights are rarely applied to human-robot interaction (HRI). In this work, we discuss the relevance of behavior learning insights for robot intent communication, and present the first application of these insights for a robot to efficiently communicate its intent by selectively explaining the causal actions in an action sequence.


title: Nineteen vortex equations and integrability

abstract: The class of five integrable vortex equations discussed recently by Manton is extended so it includes the relativistic BPS Chern-Simons vortices, yielding a total of nineteen vortex equations. Not all the nineteen vortex equations are integrable, but four new integrable equations are discovered and we generalize them to infinitely many sets of four integrable vortex equations, with each set denoted by its integer order $n$. Their integrability is similar to the known cases, but give rise to different (generalized) Baptista geometries, where the Baptista metric is a conformal rescaling of the background metric by the Higgs field. In particular, the Baptista manifolds have conical singularities. Where the Jackiw-Pi, Taubes, Popov and Ambj{\o}rn-Olesen vortices have conical deficits of $2\pi$ at each vortex zero in their Baptista manifolds, the higher-order generalizations of these equations are also integrable with larger constant curvatures and a $2\pi n$ conical deficit at each vortex zero. We then generalize a superposition law, known for Taubes vortices of how to add vortices to a known solution, to all the integrable vortex equations. We find that although the Taubes and the Popov equations relate to themselves, the Ambj{\o}rn-Olesen and Jackiw-Pi vortices are added by using the Baptista metric and the Popov equation. Finally, we find many further relations between vortex equations, e.g. we find that the Chern-Simons vortices can be interpreted as Taubes vortices on the Baptista manifold of their own solution.


title: MotionAug: Augmentation with Physical Correction for Human Motion Prediction

abstract: This paper presents a motion data augmentation scheme incorporating motion synthesis encouraging diversity and motion correction imposing physical plausibility. This motion synthesis consists of our modified Variational AutoEncoder (VAE) and Inverse Kinematics (IK). In this VAE, our proposed sampling-near-samples method generates various valid motions even with insufficient training motion data. Our IK-based motion synthesis method allows us to generate a variety of motions semi-automatically. Since these two schemes generate unrealistic artifacts in the synthesized motions, our motion correction rectifies them. This motion correction scheme consists of imitation learning with physics simulation and subsequent motion debiasing. For this imitation learning, we propose the PD-residual force that significantly accelerates the training process. Furthermore, our motion debiasing successfully offsets the motion bias induced by imitation learning to maximize the effect of augmentation. As a result, our method outperforms previous noise-based motion augmentation methods by a large margin on both Recurrent Neural Network-based and Graph Convolutional Network-based human motion prediction models. The code is available at https://github.com/meaten/MotionAug.


title: An index theorem for quarter-plane Toeplitz operators via extended symbols and gapped Invariants related to corner states

abstract: In this paper, we discuss index theory for Toeplitz operators on a discrete quarter-plane of two-variable rational matrix function symbols. By using Gohberg-Krein theory for matrix factorizations, we extend the symbols defined originally on a two-dimensional torus to some three-dimensional sphere and derive a formula to express their Fredholm indices through extended symbols. Variants for families of (self-adjoint) Fredholm quarter-plane Toeplitz operators and those preserving real structures are also included. For some bulk-edge gapped single-particle Hamiltonians of finite hopping range on a discrete lattice with a codimension-two right angle corner, topological invariants related to corner states are provided through extensions of bulk Hamiltonians.


title: Time and the Value of Data

abstract: Managers often believe that collecting more data will continually improve the accuracy of their machine learning models. However, we argue in this paper that when data lose relevance over time, it may be optimal to collect a limited amount of recent data instead of keeping around an infinite supply of older (less relevant) data. In addition, we argue that increasing the stock of data by including older datasets may, in fact, damage the model's accuracy. Expectedly, the model's accuracy improves by increasing the flow of data (defined as data collection rate); however, it requires other tradeoffs in terms of refreshing or retraining machine learning models more frequently. Using these results, we investigate how the business value created by machine learning models scales with data and when the stock of data establishes a sustainable competitive advantage. We argue that data's time-dependency weakens the barrier to entry that the stock of data creates. As a result, a competing firm equipped with a limited (yet sufficient) amount of recent data can develop more accurate models. This result, coupled with the fact that older datasets may deteriorate models' accuracy, suggests that created business value doesn't scale with the stock of available data unless the firm offloads less relevant data from its data repository. Consequently, a firm's growth policy should incorporate a balance between the stock of historical data and the flow of new data. We complement our theoretical results with an experiment. In the experiment, we empirically measure the loss in the accuracy of a next word prediction model trained on datasets from various time periods. Our empirical measurements confirm the economic significance of the value decline over time. For example, 100MB of text data, after seven years, becomes as valuable as 50MB of current data for the next word prediction task.


title: False Negative Awareness in Indicator-based Caching Systems

abstract: Distributed caching systems such as content distribution networks often advertise their content via lightweight approximate indicators (e.g., Bloom filters) to efficiently inform clients where each datum is likely cached. While false-positive indications are necessary and well understood, most existing works assume no false-negative indications. Our work illustrates practical scenarios where false-negatives are unavoidable and ignoring them significantly impacts system performance. Specifically, we focus on false-negatives induced by indicator staleness, which arises whenever the system advertises the indicator only periodically, rather than immediately reporting every change in the cache. Such scenarios naturally occur, e.g., in bandwidth-constraint environments or when latency impedes each client's ability to obtain an updated indicator. Our work introduces novel false-negative aware access policies that continuously estimate the false-negative ratio and sometimes access caches despite negative indications. We present optimal policies for homogeneous settings and provide approximation guarantees for our algorithms in heterogeneous environments. We further perform an extensive simulation study with multiple real system traces. We show that our false-negative aware algorithms incur a significantly lower access cost than existing approaches or match the cost of these approaches while requiring an order of magnitude fewer resources (e.g., caching capacity or bandwidth).


title: Semi-toric and toroidal compactifications as log minimal models, and applications to weak K-moduli

abstract: We give a characterization of toroidal (resp., semi-toric) compactifications due to Ash-Mumford-Rapoport-Tai (resp., Looijenga) as log minimal models and apply it to study weak K-moduli compactifications, giving a different proof to a theorem of Alexeev-Engel. We also discuss towards further generalization, in particular revisit Shah-Sterk compactification of moduli of polarized Enriques surfaces to show compatibility with log K-stability.


title: DRAG: Dynamic Region-Aware GCN for Privacy-Leaking Image Detection

abstract: The daily practice of sharing images on social media raises a severe issue about privacy leakage. To address the issue, privacy-leaking image detection is studied recently, with the goal to automatically identify images that may leak privacy. Recent advance on this task benefits from focusing on crucial objects via pretrained object detectors and modeling their correlation. However, these methods have two limitations: 1) they neglect other important elements like scenes, textures, and objects beyond the capacity of pretrained object detectors; 2) the correlation among objects is fixed, but a fixed correlation is not appropriate for all the images. To overcome the limitations, we propose the Dynamic Region-Aware Graph Convolutional Network (DRAG) that dynamically finds out crucial regions including objects and other important elements, and models their correlation adaptively for each input image. To find out crucial regions, we cluster spatially-correlated feature channels into several region-aware feature maps. Further, we dynamically model the correlation with the self-attention mechanism and explore the interaction among the regions with a graph convolutional network. The DRAG achieved an accuracy of 87% on the largest dataset for privacy-leaking image detection, which is 10 percentage points higher than the state of the art. The further case study demonstrates that it found out crucial regions containing not only objects but other important elements like textures.


title: To train or not to train adversarially: A study of bias mitigation strategies for speaker recognition

abstract: Speaker recognition is increasingly used in several everyday applications including smart speakers, customer care centers and other speech-driven analytics. It is crucial to accurately evaluate and mitigate biases present in machine learning (ML) based speech technologies, such as speaker recognition, to ensure their inclusive adoption. ML fairness studies with respect to various demographic factors in modern speaker recognition systems are lagging compared to other human-centered applications such as face recognition. Existing studies on fairness in speaker recognition systems are largely limited to evaluating biases at specific operating points of the systems, which can lead to false expectations of fairness. Moreover, there are only a handful of bias mitigation strategies developed for speaker recognition systems. In this paper, we systematically evaluate the biases present in speaker recognition systems with respect to gender across a range of system operating points. We also propose adversarial and multi-task learning techniques to improve the fairness of these systems. We show through quantitative and qualitative evaluations that the proposed methods improve the fairness of ASV systems over baseline methods trained using data balancing techniques. We also present a fairness-utility trade-off analysis to jointly examine fairness and the overall system performance. We show that although systems trained using adversarial techniques improve fairness, they are prone to reduced utility. On the other hand, multi-task methods can improve the fairness while retaining the utility. These findings can inform the choice of bias mitigation strategies in the field of speaker recognition.


title: Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Input

abstract: The transferability of adversarial examples allows the deception on black-box models, and transfer-based targeted attacks have attracted a lot of interest due to their practical applicability. To maximize the transfer success rate, adversarial examples should avoid overfitting to the source model, and image augmentation is one of the primary approaches for this. However, prior works utilize simple image transformations such as resizing, which limits input diversity. To tackle this limitation, we propose the object-based diverse input (ODI) method that draws an adversarial image on a 3D object and induces the rendered image to be classified as the target class. Our motivation comes from the humans' superior perception of an image printed on a 3D object. If the image is clear enough, humans can recognize the image content in a variety of viewing conditions. Likewise, if an adversarial example looks like the target class to the model, the model should also classify the rendered image of the 3D object as the target class. The ODI method effectively diversifies the input by leveraging an ensemble of multiple source objects and randomizing viewing conditions. In our experimental results on the ImageNet-Compatible dataset, this method boosts the average targeted attack success rate from 28.3% to 47.0% compared to the state-of-the-art methods. We also demonstrate the applicability of the ODI method to adversarial examples on the face verification task and its superior performance improvement. Our code is available at https://github.com/dreamflake/ODI.


title: 1D/2D hybrid Te/Graphene and Te/MoS2: multifaceted broadband photonics and green energy applications

abstract: In this letter, we highlight the enhanced functionalization of the electronic and optical properties in the hybrid heterojunction of 1D Tellurene with 2D monolayer of Graphene and MoS2 in both lateral and vertical geometries, having potential applications in the field of photonics and energy harvesting. The structural geometries of the lateral and vertical assemblies are optimized with a comparative and systematic analysis of the energetics of the different positional placement of the 1D system with respect to the hexagonal 2D layer. The 1D/2D coupling of the electronic structure in this unique assembly enables the realization of the four different types of heterojunctions, viz. type I, type II, Z-scheme and Schottky type, with the band-alignments being entirely dependent upon the stacking geometry of 1D Tellurene with respect to the 2D monolayer. With the static and time-dependent first-principles calculations, we indicate the potential applications of these hybrid systems in broadband photo detection and absorption, covering the full range of Infra-red to visible (IR-Vis) spectrum and in green energy harvesting with an effective separation and migration of photo-generated charge carriers.


title: Are Vision Transformers Robust to Spurious Correlations?

abstract: Deep neural networks may be susceptible to learning spurious correlations that hold on average but not in atypical test samples. As with the recent emergence of vision transformer (ViT) models, it remains underexplored how spurious correlations are manifested in such architectures. In this paper, we systematically investigate the robustness of vision transformers to spurious correlations on three challenging benchmark datasets and compare their performance with popular CNNs. Our study reveals that when pre-trained on a sufficiently large dataset, ViT models are more robust to spurious correlations than CNNs. Key to their success is the ability to generalize better from the examples where spurious correlations do not hold. Further, we perform extensive ablations and experiments to understand the role of the self-attention mechanism in providing robustness under spuriously correlated environments. We hope that our work will inspire future research on further understanding the robustness of ViT models.


title: Conversational Recommendation: A Grand AI Challenge

abstract: Animated avatars, which look and talk like humans, are iconic visions of the future of AI-powered systems. Through many sci-fi movies we are acquainted with the idea of speaking to such virtual personalities as if they were humans. Today, we talk more and more to machines like Apple's Siri, e.g., to ask them for the weather forecast. However, when asked for recommendations, e.g., for a restaurant to go to, the limitations of such devices quickly become obvious. They do not engage in a conversation to find out what we might prefer, they often do not provide explanations for what they recommend, and they may have difficulties remembering what was said one minute earlier. Conversational recommender systems promise to address these limitations. In this paper, we review existing approaches to build such systems, which developments we observe today, which challenges are still open and why the development of conversational recommenders represents one of the next grand challenges of AI.


title: ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Maps

abstract: Pre-trained models (PTMs) have become a fundamental backbone for downstream tasks in natural language processing and computer vision. Despite initial gains that were obtained by applying generic PTMs to geo-related tasks at Baidu Maps, a clear performance plateau over time was observed. One of the main reasons for this plateau is the lack of readily available geographic knowledge in generic PTMs. To address this problem, in this paper, we present ERNIE-GeoL, which is a geography-and-language pre-trained model designed and developed for improving the geo-related tasks at Baidu Maps. ERNIE-GeoL is elaborately designed to learn a universal representation of geography-language by pre-training on large-scale data generated from a heterogeneous graph that contains abundant geographic knowledge. Extensive quantitative and qualitative experiments conducted on large-scale real-world datasets demonstrate the superiority and effectiveness of ERNIE-GeoL. ERNIE-GeoL has already been deployed in production at Baidu Maps since April 2021, which significantly benefits the performance of various downstream tasks. This demonstrates that ERNIE-GeoL can serve as a fundamental backbone for a wide range of geo-related tasks.


title: Time Dependency, Data Flow, and Competitive Advantage

abstract: Data is fundamental to machine learning-based products and services and is considered strategic due to its externalities for businesses, governments, non-profits, and more generally for society. It is renowned that the value of organizations (businesses, government agencies and programs, and even industries) scales with the volume of available data. What is often less appreciated is that the data value in making useful organizational predictions will range widely and is prominently a function of data characteristics and underlying algorithms. In this research, our goal is to study how the value of data changes over time and how this change varies across contexts and business areas (e.g. next word prediction in the context of history, sports, politics). We focus on data from Reddit.com and compare the value's time-dependency across various Reddit topics (Subreddits). We make this comparison by measuring the rate at which user-generated text data loses its relevance to the algorithmic prediction of conversations. We show that different subreddits have different rates of relevance decline over time. Relating the text topics to various business areas of interest, we argue that competing in a business area in which data value decays rapidly alters strategies to acquire competitive advantage. When data value decays rapidly, access to a continuous flow of data will be more valuable than access to a fixed stock of data. In this kind of setting, improving user engagement and increasing user-base help creating and maintaining a competitive advantage.


title: Contrastive Learning with Positive-Negative Frame Mask for Music Representation

abstract: Self-supervised learning, especially contrastive learning, has made an outstanding contribution to the development of many deep learning research fields. Recently, researchers in the acoustic signal processing field noticed its success and leveraged contrastive learning for better music representation. Typically, existing approaches maximize the similarity between two distorted audio segments sampled from the same music. In other words, they ensure a semantic agreement at the music level. However, those coarse-grained methods neglect some inessential or noisy elements at the frame level, which may be detrimental to the model to learn the effective representation of music. Towards this end, this paper proposes a novel Positive-nEgative frame mask for Music Representation based on the contrastive learning framework, abbreviated as PEMR. Concretely, PEMR incorporates a Positive-Negative Mask Generation module, which leverages transformer blocks to generate frame masks on the Log-Mel spectrogram. We can generate self-augmented negative and positive samples by masking important components or inessential components, respectively. We devise a novel contrastive learning objective to accommodate both self-augmented positives/negatives sampled from the same music. We conduct experiments on four public datasets. The experimental results of two music-related downstream tasks, music classification, and cover song identification, demonstrate the generalization ability and transferability of music representation learned by PEMR.


title: Large global solutions of the parabolic-parabolic Keller-Segel system in higher dimensions

abstract: We study the global existence of the parabolic-parabolic Keller-Segel system in $\R^d , d \ge 2$. We prove that initial data of arbitrary size give rise to global solutions provided the diffusion parameter $\tau$ is large enough in the equation for the chemoattractant. This fact was observed before in the two-dimensional case by Biler, Guerra \& Karch (2015) and Corrias, Escobedo \& Matos (2014). Our analysis improves earlier results and extends them to any dimension $d \ge 3$. Our size conditions on the initial data for the global existence of solutions seem to be optimal, up to a logarithmic factor in $\tau$, when $\tau>>1$: we illustrate this fact by introducing two toy models, both consisting of systems of two parabolic equations, obtained after a slight modification of the nonlinearity of the usual Keller-Segel system. For these toy models, we establish in a companion paper [4] finite time blowup for a class of large solutions.


title: Function Field Analogue of Shimura's Conjecture on Period Symbols

abstract: In this paper we introduce the notion of Shimura's period symbols over function fields in positive characteristic and establish their fundamental properties. We further formulate and prove a function field analogue of Shimura's conjecture on the algebraic independence of period symbols. Our results enable us to verify the algebraic independence of the coordinates of any nonzero period vector of an abelian t-module with complex multiplication whose CM type is non-degenerate and defined over an algebraic function field. This is an extension of Yu's work on Hilbert-Blumenthal t-modules.


title: Feature-informed Latent Space Regularization for Music Source Separation

abstract: The integration of additional side information to improve music source separation has been investigated numerous times, e.g., by adding features to the input or by adding learning targets in a multi-task learning scenario. These approaches, however, require additional annotations such as musical scores, instrument labels, etc. in training and possibly during inference. The available datasets for source separation do not usually provide these additional annotations. In this work, we explore transfer learning strategies to incorporate VGGish features with a state-of-the-art source separation model; VGGish features are known to be a very condensed representation of audio content and have been successfully used in many MIR tasks. We introduce three approaches to incorporate the features, including two latent space regularization methods and one naive concatenation method. Experimental results show that our proposed approaches improve several evaluation metrics for music source separation.


title: Geometric morphisms between toposes of monoid actions: factorization systems

abstract: Let M, N be monoids, and PSh(M), PSh(N) their respective categories of right actions on sets. In this paper, we systematically investigate correspondences between properties of geometric morphisms PSh(M) $\rightarrow$ PSh(N) and properties of the semigroup homomorphisms M $\rightarrow$ N or flat-left-N-right-M-sets inducing them. More specifically, we consider properties of geometric morphisms featuring in factorization systems, namely: surjections, inclusions, localic morphisms, hyperconnected morphisms, terminal-connected morphisms, {\'e}tale morphisms, pure morphisms and complete spreads. We end with an application to topos-theoretic Galois theory to the special case of toposes of the form PSh(M).


title: An Empirical Study of Bugs in Eclipse Stable Internal Interfaces

abstract: TThe Eclipse framework is a popular and widely used framework that has been evolving for over a decade. The framework provides both stable interfaces (APIs) and unstable interfaces (non-APIs). Despite being discouraged by Eclipse, application developers often use non-APIs which cause their systems to fail when ported to new framework releases. Previous studies showed that applications using relatively old non-APIs are more likely to be compatible with new releases compared to the ones that used newly introduced non-APIs. Furthermore, from our previous study about the stability of Eclipse internal interfaces, we discovered that there exist 327K stable non-API methods as the Eclipse framework evolves. In the same study, we recommended that 327K stable non-API methods can be used by Eclipse interface providers as possible candidates for promotion to stable interfaces. However, since non-APIs are unsupported and considered to be immature i.e., can contain bugs, to this end there exist a need to first investigate the stable non-APIs for possible bugs before they can be promoted to APIs. In this study, we empirically investigated the stable non-API for possible bugs using Sonarqube software quality tool. We discovered that over 79.8% classes containing old stable non-APIs methods have zero bugs. Results from this study can be used by both interface providers and users as a starting point to analyze which interfaces are well tested and also estimate how much work could be involved when performing bug fixing for a given eclipse release.


title: Co-visual pattern augmented generative transformer learning for automobile geo-localization

abstract: Geolocation is a fundamental component of route planning and navigation for unmanned vehicles, but GNSS-based geolocation fails under denial-of-service conditions. Cross-view geo-localization (CVGL), which aims to estimate the geographical location of the ground-level camera by matching against enormous geo-tagged aerial (\emph{e.g.}, satellite) images, has received lots of attention but remains extremely challenging due to the drastic appearance differences across aerial-ground views. In existing methods, global representations of different views are extracted primarily using Siamese-like architectures, but their interactive benefits are seldom taken into account. In this paper, we present a novel approach using cross-view knowledge generative techniques in combination with transformers, namely mutual generative transformer learning (MGTL), for CVGL. Specifically, by taking the initial representations produced by the backbone network, MGTL develops two separate generative sub-modules -- one for aerial-aware knowledge generation from ground-view semantics and vice versa -- and fully exploits the entirely mutual benefits through the attention mechanism. Moreover, to better capture the co-visual relationships between aerial and ground views, we introduce a cascaded attention masking algorithm to further boost accuracy. Extensive experiments on challenging public benchmarks, \emph{i.e.}, {CVACT} and {CVUSA}, demonstrate the effectiveness of the proposed method which sets new records compared with the existing state-of-the-art models.


title: Type-Driven Multi-Turn Corrections for Grammatical Error Correction

abstract: Grammatical Error Correction (GEC) aims to automatically detect and correct grammatical errors. In this aspect, dominant models are trained by one-iteration learning while performing multiple iterations of corrections during inference. Previous studies mainly focus on the data augmentation approach to combat the exposure bias, which suffers from two drawbacks. First, they simply mix additionally-constructed training instances and original ones to train models, which fails to help models be explicitly aware of the procedure of gradual corrections. Second, they ignore the interdependence between different types of corrections. In this paper, we propose a Type-Driven Multi-Turn Corrections approach for GEC. Using this approach, from each training instance, we additionally construct multiple training instances, each of which involves the correction of a specific type of errors. Then, we use these additionally-constructed training instances and the original one to train the model in turn. Experimental results and in-depth analysis show that our approach significantly benefits the model training. Particularly, our enhanced model achieves state-of-the-art single-model performance on English GEC benchmarks. We release our code at Github.


title: Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning

abstract: Model-agnostic meta-learning (MAML) and its variants have become popular approaches for few-shot learning. However, due to the non-convexity of deep neural nets (DNNs) and the bi-level formulation of MAML, the theoretical properties of MAML with DNNs remain largely unknown. In this paper, we first prove that MAML with over-parameterized DNNs is guaranteed to converge to global optima at a linear rate. Our convergence analysis indicates that MAML with over-parameterized DNNs is equivalent to kernel regression with a novel class of kernels, which we name as Meta Neural Tangent Kernels (MetaNTK). Then, we propose MetaNTK-NAS, a new training-free neural architecture search (NAS) method for few-shot learning that uses MetaNTK to rank and select architectures. Empirically, we compare our MetaNTK-NAS with previous NAS methods on two popular few-shot learning benchmarks, miniImageNet, and tieredImageNet. We show that the performance of MetaNTK-NAS is comparable or better than the state-of-the-art NAS method designed for few-shot learning while enjoying more than 100x speedup. We believe the efficiency of MetaNTK-NAS makes itself more practical for many real-world tasks.


title: MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answering

abstract: Knowledge-based visual question answering requires the ability of associating external knowledge for open-ended cross-modal scene understanding. One limitation of existing solutions is that they capture relevant knowledge from text-only knowledge bases, which merely contain facts expressed by first-order predicates or language descriptions while lacking complex but indispensable multimodal knowledge for visual understanding. How to construct vision-relevant and explainable multimodal knowledge for the VQA scenario has been less studied. In this paper, we propose MuKEA to represent multimodal knowledge by an explicit triplet to correlate visual objects and fact answers with implicit relations. To bridge the heterogeneous gap, we propose three objective losses to learn the triplet representations from complementary views: embedding structure, topological relation and semantic space. By adopting a pre-training and fine-tuning learning strategy, both basic and domain-specific multimodal knowledge are progressively accumulated for answer prediction. We outperform the state-of-the-art by 3.35% and 6.08% respectively on two challenging knowledge-required datasets: OK-VQA and KRVQA. Experimental results prove the complementary benefits of the multimodal knowledge with existing knowledge bases and the advantages of our end-to-end framework over the existing pipeline methods. The code is available at https://github.com/AndersonStra/MuKEA.


title: Search for the correlations between host properties and ${\rm DM_{host}}$ of fast radio bursts: constraints on the baryon mass fraction in IGM

abstract: The application of fast radio bursts (FRBs) as probes to investigate astrophysics and cosmology requires the proper modelling of the dispersion measures of Milky Way (${\rm DM_{MW}}$) and host galaxy (${\rm DM_{host}}$). ${\rm DM_{MW}}$ can be estimated using the Milky Way electron models, such as NE2001 model and YMW16 model. However, ${\rm DM_{host}}$ is hard to model due to limited information on the local environment of FRBs. In this paper, using 17 well-localized FRBs, we search for the possible correlations between ${\rm DM_{host}}$ and the properties of host galaxies, such as the redshift, the stellar mass, the star-formation rate, the age of galaxy, the offset of FRB site from galactic center, and the half-light radius. We find no strong correlation between ${\rm DM_{host}}$ and any of the host property. Assuming that ${\rm DM_{host}}$ is a constant for all host galaxies, we constrain the fraction of baryon mass in the intergalactic medium today to be $f_{\rm IGM,0}=0.78_{-0.19}^{+0.15}$. If we model ${\rm DM_{host}}$ as a log-normal distribution, however, we obtain a larger value, $f_{\rm IGM,0}=0.83_{-0.17}^{+0.12}$. Based on the limited number of FRBs, no strong evidence for the redshift evolution of $f_{\rm IGM}$ is found.


title: Harmonic Pole Placement

abstract: In this paper, we propose a method to design state feedback harmonic control laws that assign the closed loop poles of a linear harmonic model to some desired locations. The procedure is based on the solution of an infinite-dimensional harmonic Sylvester equation under an invertibility constraint. We provide a sufficient condition to ensure this invertibility and show how this infinite-dimensional Sylvester equation can be solved up to an arbitrary small error. The results are illustrated on an unstable linear periodic system. We also provide a counterexample to illustrate the fact that, unlike the classical finite dimensional case, the solution of the Sylvester equation may not be invertible in the infinite dimensional case even if an observability condition is satisfied.


title: Graph Representation Learning with Individualization and Refinement

abstract: Graph Neural Networks (GNNs) have emerged as prominent models for representation learning on graph structured data. GNNs follow an approach of message passing analogous to 1-dimensional Weisfeiler Lehman (1-WL) test for graph isomorphism and consequently are limited by the distinguishing power of 1-WL. More expressive higher-order GNNs which operate on k-tuples of nodes need increased computational resources in order to process higher-order tensors. Instead of the WL approach, in this work, we follow the classical approach of Individualization and Refinement (IR), a technique followed by most practical isomorphism solvers. Individualization refers to artificially distinguishing a node in the graph and refinement is the propagation of this information to other nodes through message passing. We learn to adaptively select nodes to individualize and to aggregate the resulting graphs after refinement to help handle the complexity. Our technique lets us learn richer node embeddings while keeping the computational complexity manageable. Theoretically, we show that our procedure is more expressive than the 1-WL test. Experiments show that our method outperforms prominent 1-WL GNN models as well as competitive higher-order baselines on several benchmark synthetic and real datasets. Furthermore, our method opens new doors for exploring the paradigm of learning on graph structures with individualization and refinement.


title: Covid19 Reproduction Number: Credibility Intervals by Blockwise Proximal Monte Carlo Samplers

abstract: Monitoring the Covid19 pandemic constitutes a critical societal stake that received considerable research efforts. The intensity of the pandemic on a given territory is efficiently measured by the reproduction number, quantifying the rate of growth of daily new infections. Recently, estimates for the time evolution of the reproduction number were produced using an inverse problem formulation with a nonsmooth functional minimization. While it was designed to be robust to the limited quality of the Covid19 data (outliers, missing counts), the procedure lacks the ability to output credibility interval based estimates. This remains a severe limitation for practical use in actual pandemic monitoring by epidemiologists that the present work aims to overcome by use of Monte Carlo sampling. After interpretation of the nonsmooth functional into a Bayesian framework, several sampling schemes are tailored to adjust the nonsmooth nature of the resulting posterior distribution. The originality of the devised algorithms stems from combining a Langevin Monte Carlo sampling scheme with Proximal operators. Performance of the new algorithms in producing relevant credibility intervals for the reproduction number estimates and denoised counts are compared. Assessment is conducted on real daily new infection counts made available by the Johns Hopkins University. The interest of the devised monitoring tools are illustrated on Covid19 data from several different countries.


title: Stability and upper bounds for statistical estimation of unbalanced transport potentials

abstract: In this note, we derive upper-bounds on the statistical estimation rates of unbalanced optimal transport (UOT) maps for the quadratic cost. Our work relies on the stability of the semi-dual formulation of optimal transport (OT) extended to the unbalanced case. Depending on the considered variant of UOT, our stability result interpolates between the OT (balanced) case where the semi-dual is only locally strongly convex with respect the Sobolev semi-norm H1 dot and the case where it is locally strongly convex with respect to the H 1 norm. When the optimal potential belongs to a certain class C with sufficiently low metric-entropy, local strong convexity enables us to recover super-parametric rates, faster than 1 / root n.


title: Recent mass ejection from AGB star W Hya

abstract: We analyse archival ALMA (Atacama Large Millimeter/submillimeter Array) observations of two molecular line emissions, $^{12}$CO(3-2) and $^{29}$SiO(8-7), from oxygen-rich AGB (Asymptotic Giant Branch) star W Hya. Together with results of earlier VLT observations at visible and infrared wavelengths, our results suggest a two-component picture of the morpho-kinematics of the circumstellar envelope (CSE), one stable over time, at the scale of centuries, and the other variable, at the scale of years. The stable component consists of an approximately spherical shell of gas and dust expanding radially to a terminal velocity of $\sim$5 km s$^{-1}$ at a distance of $\sim$30 au from the star. It is found to display comparable features as seen in the CSE of R Dor, a star similar to W Hya. The variable component projects on the plane of the sky over a region confined to the neighbourhood of the star and elongated toward the north. Its very high density and sudden acceleration suggest an interpretation in terms of mass ejection initiated a few years ago. We discuss its properties in relation with earlier observations of dust formation in the same region. Our results offer a picture of the wind of W Hya that differs significantly from the picture that could be suggested by earlier analyses, giving evidence for a mass ejection that had been previously overlooked and underscoring new relations between the dust and gas emissions. They have an impact on the evidence published earlier for the presence of CO masers. They favour an interpretation in terms of convective cell ejections playing the main role in the generation of the nascent wind, the stable component of the CSE being seen as the result of many successive such events occurring in different directions at short time intervals.


title: Effects of dense matter on the tidal deformations of binary neutron star inspirals and gravitational waves

abstract: The role of the dense matter properties on the tidal deformability and gravitational waveforms of binary neutron stars is studied using a set of unified equations of state. Based on the nuclear energy-density functional theory, these equations of state provide a thermodynamically consistent treatment of all regions of the stars and were calculated using functionals that were precision fitted to experimental and theoretical nuclear data.


title: Resonances and Phase Locking Phenomena for Foliation Preserving Torus Maps

abstract: It is well known for experts that resonances in nonlinear systems lead to new invariant objects that lead to new behaviors. The goal of this paper is to study the invariant sets generated by resonances under foliation preserving torus maps. That is torus which preserve a foliation of irrational lines $L_{\theta_{0}}=\{\theta_{0}+\Omega t | t\in\mathbb{R}\}\subset\mathbb{T}^{d}$. Foliation preserving maps appear naturally as reparametrization of linear flows in the torus and also play an important role in several applications involving coupled oscillators, delay equations, resonators with moving walls, etc. The invariant objects we find here, lead to predictions on the behavior of these models. Since the results of this paper are meant to be applied for other problems, we have developed very quantitative results giving very explicit descriptions of the phenomena and the invariant objects that control them. The structure of the phase locking regions for foliation preserving maps is very different than for generic maps of the torus. Indeed, for the sake of completeness, we have developed similar analysis for the case of generic maps of the torus and shown that the objects that appear in foliation preserving maps are quantitatively and qualitatively different from those of generic torus maps. This has consequences in applications.


title: The Qualification of GEM detector and its application to Imaging

abstract: The Gas Electron Multiplier (GEM) is a new age detector, which can handle the high flux of particles. The GEM foil, which is constructed using 50$\mu$m highly insulating foil (Kapton/Apical) coated with 5$\mu$m layers of copper, on both sides, with a network of specifically shaped holes is the major component of these detectors. The European Center for Nuclear Research (CERN) has been the sole supplier of the GEM foils until recently when a few other companies started manufacturing GEM foils under the transfer of technology (TOT) agrement from CERN. Techtra is one such company in Europe which gained a right to use CERN developed technology in order to produce commercially viable GEM foils. Micropack Pvt. Ltd. is another company in India which has successfully manufactured good quality GEM foils. Due to the microscopic structure of holes and dependence on the electric field inside, it becomes essential to study the defect and uniformity of holes along with the electrical property of foils under ambient conditions. In this work we are reporting the tests condition of Techtra GEM foils. We report on the development of a cost effective and efficient technique to study the GEM foils holes geometry, distribution, and defects. We also report on the electrical properties of these foils like leakage current, stability, and discharges. At the detector level, we describe the high voltage (HV) response, gain, uniformity, and stability. The GEMs have been proposed to have a wider applications, so we performed a feasibility study to utilize these for the imaging. We irrediated various objects of varying density with X-rays and reconstructed the images. The reconstructed image shows a good distinction between materials of different densities, which can be very useful in various applications like medical imaging or cargo imaging.


title: Prediction of speech intelligibility with DNN-based performance measures

abstract: This paper presents a speech intelligibility model based on automatic speech recognition (ASR), combining phoneme probabilities from deep neural networks (DNN) and a performance measure that estimates the word error rate from these probabilities. This model does not require the clean speech reference nor the word labels during testing as the ASR decoding step, which finds the most likely sequence of words given phoneme posterior probabilities, is omitted. The model is evaluated via the root-mean-squared error between the predicted and observed speech reception thresholds from eight normal-hearing listeners. The recognition task consists of identifying noisy words from a German matrix sentence test. The speech material was mixed with eight noise maskers covering different modulation types, from speech-shaped stationary noise to a single-talker masker. The prediction performance is compared to five established models and an ASR-model using word labels. Two combinations of features and networks were tested. Both include temporal information either at the feature level (amplitude modulation filterbanks and a feed-forward network) or captured by the architecture (mel-spectrograms and a time-delay deep neural network, TDNN). The TDNN model is on par with the DNN while reducing the number of parameters by a factor of 37; this optimization allows parallel streams on dedicated hearing aid hardware as a forward-pass can be computed within the 10ms of each frame. The proposed model performs almost as well as the label-based model and produces more accurate predictions than the baseline models.


title: Active Visuo-Haptic Object Shape Completion

abstract: Recent advancements in object shape completion have enabled impressive object reconstructions using only visual input. However, due to self-occlusion, the reconstructions have high uncertainty in the occluded object parts, which negatively impacts the performance of downstream robotic tasks such as grasping. In this work, we propose an active visuo-haptic shape completion method called Act-VH that actively computes where to touch the objects based on the reconstruction uncertainty. Act-VH reconstructs objects from point clouds and calculates the reconstruction uncertainty using IGR, a recent state-of-the-art implicit surface deep neural network. We experimentally evaluate the reconstruction accuracy of Act-VH against five baselines in simulation and in the real world. We also propose a new simulation environment for this purpose. The results show that Act-VH outperforms all baselines and that an uncertainty-driven haptic exploration policy leads to higher reconstruction accuracy than a random policy and a policy driven by Gaussian Process Implicit Surfaces. As a final experiment, we evaluate Act-VH and the best reconstruction baseline on grasping 10 novel objects. The results show that Act-VH reaches a significantly higher grasp success rate than the baseline on all objects. Together, this work opens up the door for using active visuo-haptic shape completion in more complex cluttered scenes.


title: The methanol emission in the $J_1-J_0$ A$^{-+}$ line series as a tracer of specific physical conditions in high-mass star-forming regions

abstract: We present results of the investigations of the properties of the methanol $J_1 - J_0$ A$^{-+}$ line series motivated by the recent serendipitous detection of the maser emission in the $14_1 - 14_0$ A$^{-+}$ line at 349~GHz in S255IR-SMA1 soon after the accretion burst. The study includes further observations of several lines of this series in S255IR with the SMA, a mini-survey of methanol lines in the 0.8 mm range toward a sample of bright 6.7~GHz methanol maser sources with the IRAM-30m telescope, and theoretical modeling. We found that the maser component of the $14_1 - 14_0$ A$^{-+}$ line in S255IR decayed by more than order of magnitude in comparison with that in 2016. No clear sign of maser emission is observed in other lines of this series in the SMA observations except the $7_1 - 7_0$ A$^{-+}$ line where an additional bright component is detected at the velocity of the maser emission observed earlier in the $14_1 - 14_0$ A$^{-+}$ line. Our LVG model constrains the ranges of the physical parameters that matches the observed emission intensities. No obvious maser emission in the $J_1 - J_0$ A$^{-+}$ lines was detected in the mini-survey of the 6.7~GHz methanol maser sources, though one component in NGC7538 may represent a weak maser. In general, the maser effect in the $J_1 - J_0$ A$^{-+}$ lines may serve as a tracer of rather hot environments and in particular luminosity flaring events during high mass star formation.


title: Optimal Rejection Function Meets Character Recognition Tasks

abstract: In this paper, we propose an optimal rejection method for rejecting ambiguous samples by a rejection function. This rejection function is trained together with a classification function under the framework of Learning-with-Rejection (LwR). The highlights of LwR are: (1) the rejection strategy is not heuristic but has a strong background from a machine learning theory, and (2) the rejection function can be trained on an arbitrary feature space which is different from the feature space for classification. The latter suggests we can choose a feature space that is more suitable for rejection. Although the past research on LwR focused only on its theoretical aspect, we propose to utilize LwR for practical pattern classification tasks. Moreover, we propose to use features from different CNN layers for classification and rejection. Our extensive experiments of notMNIST classification and character/non-character classification demonstrate that the proposed method achieves better performance than traditional rejection strategies.


title: Self-protecting aqueous lithium-ion batteries with smart ther-moresponsive separators

abstract: Capacity degradation and destructive hazards are two core challenges for lithium-ion batteries at high temperatures, which need to be solved urgently. Adding flame retardants or fire extinguishing agents can only achieve one-time self-protection in case of emergency overheating. Herein, smart self-protecting aqueous lithium-ion batteries were developed using thermoresponsive separators through in-situ polymerization on the hydrophilic separator. The thermoresponsive separator will close the lithium ions transport channel at high temperatures and reopen when the battery cools down; more importantly, the transition is reversible. We studied the lithium salts influence on the thermoresponsive properties of the hydrogels and selected suitable lithium salt (LiNO3) and concentration (1 M) in the electrolyte to achieve self-protection without sacrificing battery performance. In addition, the shut-off temperature can be tuned by adjusting the hydrophilic and hydrophobic moiety ratio in the hydrogel according to actual demands. This self-protecting lithium-ion battery shows promise for smart energy storage devices with safe and extended lifespan.


title: The UV 2175{\AA} Attenuation Bump and its Correlation with PAH Emission at z~2

abstract: The UV bump is a broad absorption feature centered at 2175{\AA} that is seen in the attenuation/extinction curve of some galaxies, but its origin is not well known. Here, we use a sample of 86 star-forming galaxies at z=1.7-2.7 with deep rest-frame UV spectroscopy from the MUSE HUDF Survey to study the connection between the strength of the observed UV 2175{\AA} bump and the Spitzer/MIPS 24 micron photometry, which at the redshift range of our sample probes mid-IR polycyclic aromatic hydrocarbon (PAH) emission at ~6-8 micron. The sample has robust spectroscopic redshifts and consists of typical main-sequence galaxies with a wide range in stellar mass (log(Mstar/Msun) ~ 8.5-10.7) and star formation rates (SFRs; SFR ~ 1-100 Msun/yr). Galaxies with MIPS detections have strong UV bumps, except for those with mass-weighted ages younger than ~150 Myr. We find that the UV bump amplitude does not change with SFR at fixed stellar mass but increases with mass at fixed SFR. The UV bump amplitude and the PAH strength (defined as mid-IR emission normalized by SFR) are highly correlated and both also correlate strongly with stellar mass. We interpret these correlations as the result of the mass-metallicity relationship, such that at low metallicities PAH emission is weak due to a lower abundance of PAH molecules. The weak or complete absence of the 2175{\AA} bump feature on top of the underlying smooth attenuation curve at low mass/metallicities is then expected if the PAH carriers are the main source of the additional UV absorption.


title: Measurement of MHD turbulence properties by synchrotron radiation techniques

abstract: It is well-known that magnetohydrodynamic (MHD) turbulence is ubiquitous in astrophysical environments. The correct understanding of the fundamental properties of MHD turbulence is a prerequisite for revealing many key astrophysical processes. The development of observation-based measurement techniques has significantly promoted MHD turbulence theory and its implications in astrophysics. After describing the modern understanding of MHD turbulence based on theoretical analysis and direct numerical simulations, we review recent developments related to synchrotron fluctuation techniques. Specifically, we comment on the validation of synchrotron fluctuation techniques and the measurement performance of several properties of magnetic turbulence based on data cubes from MHD turbulence simulations and observations. Furthermore, we propose to strengthen the studies of the magnetization and 3D magnetic field structure's measurements of interstellar turbulence. At the same time, we also discuss the prospects of new techniques for measuring magnetic field properties and understanding astrophysical processes, using a large number of data cubes from the Low-Frequency Array (LOFAR) and the Square Kilometre Array (SKA).


title: AdaSplats: Adaptive Splatting of Point Clouds for Accurate 3D Modeling and Real-time High-Fidelity LiDAR Simulation

abstract: LiDAR sensors provide rich 3D information about their surrounding{s} and are becoming increasingly important for autonomous vehicles tasks such as {localization}, semantic segmentation, object detection, and tracking. {Simulation} accelerates the testing, validation, and deployment of autonomous vehicles while {also} reducing cost and eliminating the risks of testing in real-world scenarios. We address the problem of high-fidelity LiDAR simulation and present a pipeline that leverages real-world point clouds acquired by mobile mapping systems. Point-based geometry representations, more specifically splats {(2D oriented disks with normals)}, have proven their ability to accurately model the underlying surface in large point clouds{, mainly with uniform density}. We introduce an adaptive splat generation method that accurately models the underlying 3D geometry {to handle real-world point clouds with variable densities}, especially for thin structures. Moreover, we introduce a {fast} LiDAR {sensor} simulator, {working} in the splatted model, {that leverages} the GPU parallel architecture with an acceleration structure while focusing on efficiently handling large point clouds. We test our LiDAR simulation in real-world conditions, showing qualitative and quantitative results compared to basic splatting and meshing techniques, demonstrating the interest of our modeling technique.


title: Arithmetic of Chatelet surfaces under extensions of base fields

abstract: For Ch\^atelet surfaces defined over number fields, we study two arithmetic properties, the Hasse principle and weak approximation, when passing to an extension of the base field. Generalizing a construction of Y. Liang, we show that for an arbitrary extension of number fields $L/K,$ there is a Ch\^atelet surface over $K$ which does not satisfy weak approximation over any intermediate field of $L/K,$ and a Ch\^atelet surface over $K$ which satisfies the Hasse principle over an intermediate field $L'$ if and only if $[L' : K]$ is even.


title: Dynamic groups in complex task environments: To change or not to change a winning team?

abstract: Organisations rely upon group formation to solve complex tasks, and groups often adapt to the demands of the task they face by changing their composition periodically. Previous research comes to ambiguous results regarding the effects of group adaptation on task performance. This paper aims to understand the impact of group adaptation, defined as a process of periodically changing a group's composition, on complex task performance and considers the moderating role of individual learning and task complexity in this relationship. We base our analyses on an agent-based model of adaptive groups in a complex task environment based on the NK-framework. The results indicate that reorganising well-performing groups might be beneficial, but only if individual learning is restricted. However, there are also cases in which group adaptation might unfold adverse effects. We provide extensive analyses that shed additional light on and, thereby, help explain the ambiguous results of previous research.


title: Thermodynamic symmetry resolved entanglement entropies in integrable systems

abstract: We develop a general approach to compute the symmetry-resolved R\'enyi and von Neumann entanglement entropies (SREE) of thermodynamic macrostates in interacting integrable systems. Our method is based on a combination of the thermodynamic Bethe ansatz and the G\"artner-Ellis theorem from large deviation theory. We derive an explicit simple formula for the von Neumann SREE, which we show to coincide with the thermodynamic Yang-Yang entropy of an effective macrostate determined by the charge sector. Focusing on the XXZ Heisenberg spin chain, we test our result against iTEBD calculations for thermal states, finding good agreement. As an application, we provide analytic predictions for the asymptotic value of the SREE following a quantum quench.


title: EMAKG: An Enhanced Version Of The Microsoft Academic Knowledge Graph

abstract: Scholarly knowledge graphs are valuable sources of information in several research fields. Despite the number of existing datasets related to publications and researchers, resource quality, coverage and accessibility are still limited. This article presents the Enhanced Microsoft Academic Knowledge Graph, a large dataset of information about scientific publications and involved entities, and the methods developed to build it. Data includes geographical information, researchers' collaborative networks and movements between institutions, academic-related metrics, and linguistic features. The dataset merges information from several data sources and has high temporal and spatial 7 coverage, allowing several use cases.


title: Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation

abstract: Scene graph generation is a sophisticated task because there is no specific recognition pattern (e.g., "looking at" and "near" have no conspicuous difference concerning vision, whereas "near" could occur between entities with different morphology). Thus some scene graph generation methods are trapped into most frequent relation predictions caused by capricious visual features and trivial dataset annotations. Therefore, recent works emphasized the "unbiased" approaches to balance predictions for a more informative scene graph. However, human's quick and accurate judgments over relations between numerous objects should be attributed to "bias" (i.e., experience and linguistic knowledge) rather than pure vision. To enhance the model capability, inspired by the "cognitive bias" mechanism, we propose a novel 3-paradigms framework that simulates how humans incorporate the label linguistic features as guidance of vision-based representations to better mine hidden relation patterns and alleviate noisy visual propagation. Our framework is model-agnostic to any scene graph model. Comprehensive experiments prove our framework outperforms baseline modules in several metrics with minimum parameters increment and achieves new SOTA performance on Visual Genome dataset.


title: How Many Data Samples is an Additional Instruction Worth?

abstract: Recently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they are far from state-of-the-art task-specific models. Conventional approaches to improve model performance via creating datasets with large number of task instances or architectural changes in the model may not be feasible for non-expert users. However, they can write alternate instructions to represent an instruction task. Is Instruction-augmentation helpful? We augment a subset of tasks in the expanded version of NATURAL INSTRUCTIONS with additional instructions and find that it significantly improves model performance (up to 35%), especially in the low-data regime. Our results indicate that an additional instruction can be equivalent to ~200 data samples on average across tasks.


title: Interactions between the individual and the group level in organizations: The case of learning and autonomous group adaptation

abstract: Previous research on organizations often focuses on either the individual, team, or organizational level. There is a lack of multidimensional research on emergent phenomena and interactions between the mechanisms at different levels. This paper takes a multifaceted perspective on individual learning and autonomous group formation and adaptation. To analyze interactions between the two levels, we introduce an agent-based model that captures an organization with a population of heterogeneous agents who learn and are limited in their rationality. To solve a task, agents form a group that can be adapted from time to time. We explore organizations that promote learning and group adaptation either simultaneously or sequentially and analyze the interactions between the activities and the effects on performance. We observe underproportional interactions when tasks are interdependent and show that pushing learning and group adaptation too far might backfire and decrease performance significantly.


title: Modeling Dual Read/Write Paths for Simultaneous Machine Translation

abstract: Simultaneous machine translation (SiMT) outputs translation while reading source sentence and hence requires a policy to decide whether to wait for the next source word (READ) or generate a target word (WRITE), the actions of which form a read/write path. Although the read/write path is essential to SiMT performance, no direct supervision is given to the path in the existing methods. In this paper, we propose a method of dual-path SiMT which introduces duality constraints to direct the read/write path. According to duality constraints, the read/write path in source-to-target and target-to-source SiMT models can be mapped to each other. As a result, the two SiMT models can be optimized jointly by forcing their read/write paths to satisfy the mapping. Experiments on En-Vi and De-En tasks show that our method can outperform strong baselines under all latency.


title: A Survey on Brain-Computer Interface and Related Applications

abstract: BCI systems are able to communicate directly between the brain and computer using neural activity measurements without the involvement of muscle movements. For BCI systems to be widely used by people with severe disabilities, long-term studies of their real-world use are needed, along with effective and feasible dissemination models. In addition, the robustness of the BCI systems' performance should be improved so they reach the same level of robustness as natural muscle-based health monitoring. In this chapter, we review the recent BCI related studies, followed by the most relevant applications of BCI systems. We also present the key issues and challenges which exist in regard to the BCI systems and also provide future directions.


title: Partitions, Multiple Zeta Values and the q-bracket

abstract: We provide a framework for relating certain q-series defined by sums over partitions to multiple zeta values. In particular, we introduce a space of polynomial functions on partitions for which the associated q-series are q-analogues of multiple zeta values. By explicitly describing the (regularized) multiple zeta values one obtains as $q\to 1$, we extend previous results known in this area. Using this together with the fact that other families of functions on partitions, such as shifted symmetric functions, are elements in our space will then give relations among (q-analogues of) multiple zeta values. Conversely, we will show that relations among multiple zeta values can be `lifted' to the world of functions on partitions, which provides new examples of functions where the associated q-series are quasimodular.


title: Linear isoperimetric inequality for homogeneous Hadamard manifolds

abstract: It is well known that simply connected symmetric spaces of non-positive sectional curvature admit a linear isoperimetric filling inequality for cycles of dimension greater than or equal to the rank of the space. In this note we extend that result to homogeneous Hadamard manifolds.


title: Unsigned Distance Field as an Accurate 3D Scene Representation for Neural Scene Completion

abstract: Scene Completion is the task of completing missing geometry from a partial scan of a scene. Most previous methods compute an implicit representation from range data using a Truncated Signed Distance Function (T-SDF) computed on a 3D grid as input to neural networks. The truncation decreases but does not remove the border errors introduced by the sign of SDF for open surfaces. As an alternative, we present an Unsigned Distance Function (UDF) as an input representation to scene completion neural networks. The proposed UDF is simple, and efficient as a geometry representation, and can be computed on any point cloud. In contrast to usual Signed Distance Functions, our UDF does not require normal computation. To obtain the explicit geometry, we present a method for extracting a point cloud from discretized UDF values on a sparse grid. We compare different SDFs and UDFs for the scene completion task on indoor and outdoor point clouds collected using RGB-D and LiDAR sensors and show improved completion using the proposed UDF function.


title: On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks

abstract: Capturing aleatoric uncertainty is a critical part of many machine learning systems. In deep learning, a common approach to this end is to train a neural network to estimate the parameters of a heteroscedastic Gaussian distribution by maximizing the logarithm of the likelihood function under the observed data. In this work, we examine this approach and identify potential hazards associated with the use of log-likelihood in conjunction with gradient-based optimizers. First, we present a synthetic example illustrating how this approach can lead to very poor but stable parameter estimates. Second, we identify the culprit to be the log-likelihood loss, along with certain conditions that exacerbate the issue. Third, we present an alternative formulation, termed $\beta$-NLL, in which each data point's contribution to the loss is weighted by the $\beta$-exponentiated variance estimate. We show that using an appropriate $\beta$ largely mitigates the issue in our illustrative example. Fourth, we evaluate this approach on a range of domains and tasks and show that it achieves considerable improvements and performs more robustly concerning hyperparameters, both in predictive RMSE and log-likelihood criteria.


title: Dual-domain Attention-based Deep Network for Sparse-view CT Artifact Reduction

abstract: Due to the wide applications of X-ray computed tomography (CT) in medical imaging activities, radiation exposure has become a major concern for public health. Sparse-view CT is a promising approach to reduce the radiation dose by down-sampling the total number of acquired projections. However, the CT images reconstructed by this sparse-view imaging approach suffer from severe streaking artifacts and structural information loss. In this work, an end-to-end dual-domain attention-based deep network (DDANet) is proposed to solve such an ill-posed CT image reconstruction problem. The image-domain CT image and the projection-domain sinogram are put into the two parallel sub-networks of the DDANet to independently extract the distinct high-level feature maps. In addition, a specified attention module is introduced to fuse the aforementioned dual-domain feature maps to allow complementary optimizations of removing the streaking artifacts and mitigating the loss of structure. Numerical simulations, anthropomorphic thorax phantom and in vivo pre-clinical experiments are conducted to verify the sparse-view CT imaging performance of the DDANet. Results demonstrate that this newly developed approach is able to robustly remove the streaking artifacts while maintaining the fine structures. As a result, the DDANet provides a promising solution in achieving high quality sparse-view CT imaging.


title: Recurrent Neural Networks for Forecasting Time Series with Multiple Seasonality: A Comparative Study

abstract: This paper compares recurrent neural networks (RNNs) with different types of gated cells for forecasting time series with multiple seasonality. The cells we compare include classical long short term memory (LSTM), gated recurrent unit (GRU), modified LSTM with dilation, and two new cells we proposed recently, which are equipped with dilation and attention mechanisms. To model the temporal dependencies of different scales, our RNN architecture has multiple dilated recurrent layers stacked with hierarchical dilations. The proposed RNN produces both point forecasts and predictive intervals (PIs) for them. An empirical study concerning short-term electrical load forecasting for 35 European countries confirmed that the new gated cells with dilation and attention performed best.