title: 2D-MoS2 with Narrowest Excitonic Linewidths Grown by Flow-Less Direct Heating of Bulk Powders

abstract: Developing techniques for high-quality synthesis of mono and few-layered 2D materials with lowered complexity and cost continues to remain an important goal, both for accelerating fundamental research and for applications development. We present the simplest conceivable technique to synthesize micrometer-scale single-crystal triangular monolayers of MoS2, i.e. by direct heating of bulk MoS2 powder onto proximally-placed substrates. Room-temperature excitonic linewidth values of our samples are narrower and more uniform than those of 2D-MoS2 obtained by most other techniques reported in literature, and comparable to those of ultraflat h-BN-capped mechanically exfoliated samples, indicative of their high quality. Feature-rich Raman spectra absent in samples grown or obtained by most other techniques, also stand out as a testament of the high quality of our samples. A contact-growth mode facilitates direct growth of crystallographically-strained circular samples, which allows us to directly compare the optoelectronic properties of flat vs. strained growth from the same growth runs. Our method allows, for the first time, to quantitatively compare the impact of strain on excitonic and Raman peak positions on identically-synthesized 2D-MoS2. Strain leads to average Red-shifts of ~ 30 meV in the A-exciton position, and ~ 2 cm-1 in the E12g Raman peak in these samples. Our findings open-up several new possibilities that expand 2D material research. By eliminating the need for carrier gas flow, mechanical motion or chemical reactions, our method can be either miniaturized for substantially low-cost, high-quality scientific research or potentially scaled-up for mass-production of 2D crystals for commercial purposes. Moreover, we believe this technique can also be extended to other transition metal dichalcogenides and other layered materials.


title: On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer

abstract: Hybrid Autoregressive Transducer (HAT) is a recently proposed end-to-end acoustic model that extends the standard Recurrent Neural Network Transducer (RNN-T) for the purpose of the external language model (LM) fusion. In HAT, the blank probability and the label probability are estimated using two separate probability distributions, which provides a more accurate solution for internal LM score estimation, and thus works better when combining with an external LM. Previous work mainly focuses on HAT model training with the negative log-likelihood loss, while in this paper, we study the minimum word error rate (MWER) training of HAT -- a criterion that is closer to the evaluation metric for speech recognition, and has been successfully applied to other types of end-to-end models such as sequence-to-sequence (S2S) and RNN-T models. From experiments with around 30,000 hours of training data, we show that MWER training can improve the accuracy of HAT models, while at the same time, improving the robustness of the model against the decoding hyper-parameters such as length normalization and decoding beam during inference.


title: Exploring task-based query expansion at the TREC-COVID track

abstract: We explore how to generate effective queries based on search tasks. Our approach has three main steps: 1) identify search tasks based on research goals, 2) manually classify search queries according to those tasks, and 3) compare three methods to improve search rankings based on the task context. The most promising approach is based on expanding the user's query terms using task terms, which slightly improved the NDCG@20 scores over a BM25 baseline. Further improvements might be gained if we can identify more specific search tasks.


title: Overcoming Conflicting Data when Updating a Neural Semantic Parser

abstract: In this paper, we explore how to use a small amount of new data to update a task-oriented semantic parsing model when the desired output for some examples has changed. When making updates in this way, one potential problem that arises is the presence of conflicting data, or out-of-date labels in the original training set. To evaluate the impact of this understudied problem, we propose an experimental setup for simulating changes to a neural semantic parser. We show that the presence of conflicting data greatly hinders learning of an update, then explore several methods to mitigate its effect. Our multi-task and data selection methods lead to large improvements in model accuracy compared to a naive data-mixing strategy, and our best method closes 86% of the accuracy gap between this baseline and an oracle upper bound.


title: A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing

abstract: Abstract Meaning Representations (AMR) are a broad-coverage semantic formalism which represents sentence meaning as a directed acyclic graph. To train most AMR parsers, one needs to segment the graph into subgraphs and align each such subgraph to a word in a sentence; this is normally done at preprocessing, relying on hand-crafted rules. In contrast, we treat both alignment and segmentation as latent variables in our model and induce them as part of end-to-end training. As marginalizing over the structured latent variables is infeasible, we use the variational autoencoding framework. To ensure end-to-end differentiable optimization, we introduce a differentiable relaxation of the segmentation and alignment problems. We observe that inducing segmentation yields substantial gains over using a `greedy' segmentation heuristic. The performance of our method also approaches that of a model that relies on the segmentation rules of \citet{lyu-titov-2018-amr}, which were hand-crafted to handle individual AMR constructions.


title: Quasi-local mass and isometric embedding with reference to a static spacetime

abstract: The mathematical theory of isometric embedding is applied to study the notion of quasilocal mass in general relativity. In particular, I shall report some recent progress of quasilocal mass with reference to a cosmological spacetime, such as the de Sitter or the Anti-de Sitter spacetime, or a blackhole spacetime, such as the Schwarzschild spacetime. This article is based on joint work with Po-Ning Chen, Ye-Kai Wang, and Shing-Tung Yau.


title: Super-Resolution Reconstruction of Interval Energy Data

abstract: High-resolution data are desired in many data-driven applications; however, in many cases only data whose resolution is lower than expected are available due to various reasons. It is then a challenge how to obtain as much useful information as possible from the low-resolution data. In this paper, we target interval energy data collected by Advanced Metering Infrastructure (AMI), and propose a Super-Resolution Reconstruction (SRR) approach to upsample low-resolution (hourly) interval data into higher-resolution (15-minute) data using deep learning. Our preliminary results show that the proposed SRR approaches can achieve much improved performance compared to the baseline model.


title: Nowcasting COVID-19 incidence indicators during the Italian first outbreak

abstract: A novel parametric regression model is proposed to fit incidence data typically collected during epidemics. The proposal is motivated by real-time monitoring and short-term forecasting of the main epidemiological indicators within the first outbreak of COVID-19 in Italy. Accurate short-term predictions, including the potential effect of exogenous or external variables are provided; this ensures to accurately predict important characteristics of the epidemic (e.g., peak time and height), allowing for a better allocation of health resources over time. Parameters estimation is carried out in a maximum likelihood framework. All computational details required to reproduce the approach and replicate the results are provided.


title: Spin Susceptibility in Neutron Matter from Quantum Monte Carlo Calculations

abstract: The spin susceptibility in pure neutron matter is computed from auxiliary field diffusion Monte Carlo calculations over a wide range of densities. The calculations are performed for different spin asymmetries, while using twist-averaged boundary conditions to reduce finite-size effects. The employed nuclear interactions include both the phenomenological Argonne AV8$^\prime$+UIX potential and local interactions that are derived from chiral effective field theory up to next-to-next-to-leading order.


title: Robust Document Representations using Latent Topics and Metadata

abstract: Task specific fine-tuning of a pre-trained neural language model using a custom softmax output layer is the de facto approach of late when dealing with document classification problems. This technique is not adequate when labeled examples are not available at training time and when the metadata artifacts in a document must be exploited. We address these challenges by generating document representations that capture both text and metadata artifacts in a task agnostic manner. Instead of traditional auto-regressive or auto-encoding based training, our novel self-supervised approach learns a soft-partition of the input space when generating text embeddings. Specifically, we employ a pre-learned topic model distribution as surrogate labels and construct a loss function based on KL divergence. Our solution also incorporates metadata explicitly rather than just augmenting them with text. The generated document embeddings exhibit compositional characteristics and are directly used by downstream classification tasks to create decision boundaries from a small number of labeled examples, thereby eschewing complicated recognition methods. We demonstrate through extensive evaluation that our proposed cross-model fusion solution outperforms several competitive baselines on multiple datasets.


title: Unsupervised Dense Shape Correspondence using Heat Kernels

abstract: In this work, we propose an unsupervised method for learning dense correspondences between shapes using a recent deep functional map framework. Instead of depending on ground-truth correspondences or the computationally expensive geodesic distances, we use heat kernels. These can be computed quickly during training as the supervisor signal. Moreover, we propose a curriculum learning strategy using different heat diffusion times which provide different levels of difficulty during optimization without any sampling mechanism or hard example mining. We present the results of our method on different benchmarks which have various challenges like partiality, topological noise and different connectivity.


title: Long Document Ranking with Query-Directed Sparse Transformer

abstract: The computing cost of transformer self-attention often necessitates breaking long documents to fit in pretrained models in document ranking tasks. In this paper, we design Query-Directed Sparse attention that induces IR-axiomatic structures in transformer self-attention. Our model, QDS-Transformer, enforces the principle properties desired in ranking: local contextualization, hierarchical representation, and query-oriented proximity matching, while it also enjoys efficiency from sparsity. Experiments on one fully supervised and three few-shot TREC document ranking benchmarks demonstrate the consistent and robust advantage of QDS-Transformer over previous approaches, as they either retrofit long documents into BERT or use sparse attention without emphasizing IR principles. We further quantify the computing complexity and demonstrates that our sparse attention with TVM implementation is twice more efficient than the fully-connected self-attention. All source codes, trained model, and predictions of this work are available at https://github.com/hallogameboy/QDS-Transformer.


title: Dynamic Contextualized Word Embeddings

abstract: Static word embeddings that represent words by a single vector cannot capture the variability of word meaning in different linguistic and extralinguistic contexts. Building on prior work on contextualized and dynamic word embeddings, we introduce dynamic contextualized word embeddings that represent words as a function of both linguistic and extralinguistic context. Based on a pretrained language model (PLM), dynamic contextualized word embeddings model time and social space jointly, which makes them attractive for a range of NLP tasks involving semantic variability. We highlight potential application scenarios by means of qualitative and quantitative analyses on four English datasets.


title: Numerically modeling stochastic inflation in slow-roll and beyond

abstract: We present a complete numerical treatment of inflationary dynamics under the influence of stochastic corrections from sub-Hubble modes. We discuss how to exactly model the stochastic noise terms arising from the sub-Hubble quantum modes that give rise to the coarse-grained inflaton dynamics in the form of stochastic differential equations. The stochastic differential equations are solved event-by-event on a discrete time grid. We then compute the power spectrum of curvature perturbations that can be compared with the power spectrum computed in the traditional fashion using the Mukhanov-Sasaki equation by canonically quantizing the inflaton fluctuations. Our numerical procedure helps us to easily extend the formalism to ultra slow-roll inflation and study the possibility of primordial black hole formation.


title: On Algebraic Abstractions for Concurrent Separation Logics

abstract: Concurrent separation logic is distinguished by transfer of state ownership upon parallel composition and framing. The algebraic structure that underpins ownership transfer is that of partial commutative monoids (PCMs). Extant research considers ownership transfer primarily from the logical perspective while comparatively less attention is drawn to the algebraic considerations. This paper provides an algebraic formalization of ownership transfer in concurrent separation logic by means of structure-preserving partial functions (i.e., morphisms) between PCMs, and an associated notion of separating relations. Morphisms of structures are a standard concept in algebra and category theory, but haven't seen ubiquitous use in separation logic before. Separating relations are binary relations that generalize disjointness and characterize the inputs on which morphisms preserve structure. The two abstractions facilitate verification by enabling concise ways of writing specs, by providing abstract views of threads' states that are preserved under ownership transfer, and by enabling user-level construction of new PCMs out of existing ones.


title: Robust Correction of Sampling Bias Using Cumulative Distribution Functions

abstract: Varying domains and biased datasets can lead to differences between the training and the target distributions, known as covariate shift. Current approaches for alleviating this often rely on estimating the ratio of training and target probability density functions. These techniques require parameter tuning and can be unstable across different datasets. We present a new method for handling covariate shift using the empirical cumulative distribution function estimates of the target distribution by a rigorous generalization of a recent idea proposed by Vapnik and Izmailov. Further, we show experimentally that our method is more robust in its predictions, is not reliant on parameter tuning and shows similar classification performance compared to the current state-of-the-art techniques on synthetic and real datasets.


title: Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training

abstract: Prior work on Data-To-Text Generation, the task of converting knowledge graph (KG) triples into natural text, focused on domain-specific benchmark datasets. In this paper, however, we verbalize the entire English Wikidata KG, and discuss the unique challenges associated with a broad, open-domain, large-scale verbalization. We further show that verbalizing a comprehensive, encyclopedic KG like Wikidata can be used to integrate structured KGs and natural language corpora. In contrast to the many architectures that have been developed to integrate these two sources, our approach converts the KG into natural text, allowing it to be seamlessly integrated into existing language models. It carries the further advantages of improved factual accuracy and reduced toxicity in the resulting language model. We evaluate this approach by augmenting the retrieval corpus in a retrieval language model and showing significant improvements on the knowledge intensive tasks of open domain QA and the LAMA knowledge probe.


title: Intersection Types and (Positive) Almost-Sure Termination

abstract: Randomized higher-order computation can be seen as being captured by a lambda calculus endowed with a single algebraic operation, namely a construct for binary probabilistic choice. What matters about such computations is the probability of obtaining any given result, rather than the possibility or the necessity of obtaining it, like in (non)deterministic computation. Termination, arguably the simplest kind of reachability problem, can be spelled out in at least two ways, depending on whether it talks about the probability of convergence or about the expected evaluation time, the second one providing a stronger guarantee. In this paper, we show that intersection types are capable of precisely characterizing both notions of termination inside a single system of types: the probability of convergence of any lambda-term can be underapproximated by its type, while the underlying derivation's weight gives a lower bound to the term's expected number of steps to normal form. Noticeably, both approximations are tight -- not only soundness but also completeness holds. The crucial ingredient is non-idempotency, without which it would be impossible to reason on the expected number of reduction steps which are necessary to completely evaluate any term. Besides, the kind of approximation we obtain is proved to be optimal recursion theoretically: no recursively enumerable formal system can do better than that.


title: Loss-analysis via Attention-scale for Physiologic Time Series

abstract: Physiologic signals have properties across multiple spatial and temporal scales, which can be shown by the complexity-analysis of the coarse-grained physiologic signals by scaling techniques such as the multiscale. Unfortunately, the results obtained from the coarse-grained signals by the multiscale may not fully reflect the properties of the original signals because there is a loss caused by scaling techniques and the same scaling technique may bring different losses to different signals. Another problem is that multiscale does not consider the key observations inherent in the signal. Here, we show a new analysis method for time series called the loss-analysis via attention-scale. We show that multiscale is a special case of attention-scale. The loss-analysis can complement to the complexity-analysis to capture aspects of the signals that are not captured using previously developed measures. This can be used to study ageing, diseases, and other physiologic phenomenon.


title: Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

abstract: As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. 1. Due to a lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. 2. Training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNNs called Skip-Connected Self-Recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced in a stereotyped manner by adding self-recurrent connections to spiking neurons, which implements local memory. The network dynamics is enriched by skip connections between nonadjacent layers. Constructed by simplified self-recurrent and skip connections, ScSr-SNNs are able to realize recurrent behaviors similar to those of more complex RSNNs while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. Moreover, we propose a new backpropagation (BP) method called backpropagated intrinsic plasticity (BIP) to further boost the performance of ScSr-SNNs by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP methods optimize intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Based upon challenging speech and neuromorphic speech datasets including TI46-Alpha, TI46-Digits, and N-TIDIGITS, the proposed ScSr-SNNs can boost performance by up to 2.55% compared with other types of RSNNs trained by state-of-the-art BP methods.


title: Multi-Channel Speaker Verification for Single and Multi-talker Speech

abstract: To improve speaker verification in real scenarios with interference speakers, noise, and reverberation, we propose to bring together advancements made in multi-channel speech features. Specifically, we combine spectral, spatial, and directional features, which includes inter-channel phase difference, multi-channel sinc convolutions, directional power ratio features, and angle features. To maximally leverage supervised learning, our framework is also equipped with multi-channel speech enhancement and voice activity detection. On all simulated, replayed, and real recordings, we observe large and consistent improvements at various degradation levels. On real recordings of multi-talker speech, we achieve a 36% relative reduction in equal error rate w.r.t. single-channel baseline. We find the improvements from speaker-dependent directional features more consistent in multi-talker conditions than clean. Lastly, we investigate if the learned multi-channel speaker embedding space can be made more discriminative through a contrastive loss-based fine-tuning. With a simple choice of Triplet loss, we observe a further 8.3% relative reduction in EER.


title: On the Embeddings of Variables in Recurrent Neural Networks for Source Code

abstract: Source code processing heavily relies on the methods widely used in natural language processing (NLP), but involves specifics that need to be taken into account to achieve higher quality. An example of this specificity is that the semantics of a variable is defined not only by its name but also by the contexts in which the variable occurs. In this work, we develop dynamic embeddings, a recurrent mechanism that adjusts the learned semantics of the variable when it obtains more information about the variable's role in the program. We show that using the proposed dynamic embeddings significantly improves the performance of the recurrent neural network, in code completion and bug fixing tasks.


title: AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarization

abstract: Summarization is the task of compressing source document(s) into coherent and succinct passages. This is a valuable tool to present users with concise and accurate sketch of the top ranked documents related to their queries. Query-based multi-document summarization (qMDS) addresses this pervasive need, but the research is severely limited due to lack of training and evaluation datasets as existing single-document and multi-document summarization datasets are inadequate in form and scale. We propose a scalable approach called AQuaMuSe to automatically mine qMDS examples from question answering datasets and large document corpora. Our approach is unique in the sense that it can general a dual dataset -- for extractive and abstractive summaries both. We publicly release a specific instance of an AQuaMuSe dataset with 5,519 query-based summaries, each associated with an average of 6 input documents selected from an index of 355M documents from Common Crawl. Extensive evaluation of the dataset along with baseline summarization model experiments are provided.


title: Adding Interactive Visual Syntax to Textual Code

abstract: Many programming problems call for turning geometrical thoughts into code: tables, hierarchical structures, nests of objects, trees, forests, graphs, and so on. Linear text does not do justice to such thoughts. But, it has been the dominant programming medium for the past and will remain so for the foreseeable future. This paper proposes a novel mechanism for conveniently extending textual programming languages with problem-specific visual syntax. It argues the necessity of this language feature, demonstrates the feasibility with a robust prototype, and sketches a design plan for adapting the idea to other languages.


title: On Construction and Estimation of Stationary Mixture Transition Distribution Models

abstract: Mixture transition distribution time series models build high-order dependence through a weighted combination of first-order transition densities for each one of a specified number of lags. We present a framework to construct stationary transition mixture distribution models that extend beyond linear, Gaussian dynamics. We study conditions for first-order strict stationarity which allow for different constructions with either continuous or discrete families for the first-order transition densities given a pre-specified family for the marginal density, and with general forms for the resulting conditional expectations. Inference and prediction are developed under the Bayesian framework with particular emphasis on flexible, structured priors for the mixture weights. Model properties are investigated both analytically and through synthetic data examples. Finally, Poisson and Lomax examples are illustrated through real data applications.


title: Investigating Saturation Effects in Integrated Gradients

abstract: Integrated Gradients has become a popular method for post-hoc model interpretability. De-spite its popularity, the composition and relative impact of different regions of the integral path are not well understood. We explore these effects and find that gradients in saturated regions of this path, where model output changes minimally, contribute disproportionately to the computed attribution. We propose a variant of IntegratedGradients which primarily captures gradients in unsaturated regions and evaluate this method on ImageNet classification networks. We find that this attribution technique shows higher model faithfulness and lower sensitivity to noise com-pared with standard Integrated Gradients. A note-book illustrating our computations and results is available at https://github.com/vivekmig/captum-1/tree/ExpandedIG.


title: Stabilizing Transformer-Based Action Sequence Generation For Q-Learning

abstract: Since the publication of the original Transformer architecture (Vaswani et al. 2017), Transformers revolutionized the field of Natural Language Processing. This, mainly due to their ability to understand timely dependencies better than competing RNN-based architectures. Surprisingly, this architecture change does not affect the field of Reinforcement Learning (RL), even though RNNs are quite popular in RL, and time dependencies are very common in RL. Recently, Parisotto et al. 2019) conducted the first promising research of Transformers in RL. To support the findings of this work, this paper seeks to provide an additional example of a Transformer-based RL method. Specifically, the goal is a simple Transformer-based Deep Q-Learning method that is stable over several environments. Due to the unstable nature of Transformers and RL, an extensive method search was conducted to arrive at a final method that leverages developments around Transformers as well as Q-learning. The proposed method can match the performance of classic Q-learning on control environments while showing potential on some selected Atari benchmarks. Furthermore, it was critically evaluated to give additional insights into the relation between Transformers and RL.


title: Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary

abstract: The introduction of pre-trained transformer-based contextualized word embeddings has led to considerable improvements in the accuracy of graph-based parsers for frameworks such as Universal Dependencies (UD). However, previous works differ in various dimensions, including their choice of pre-trained language models and whether they use LSTM layers. With the aims of disentangling the effects of these choices and identifying a simple yet widely applicable architecture, we introduce STEPS, a new modular graph-based dependency parser. Using STEPS, we perform a series of analyses on the UD corpora of a diverse set of languages. We find that the choice of pre-trained embeddings has by far the greatest impact on parser performance and identify XLM-R as a robust choice across the languages in our study. Adding LSTM layers provides no benefits when using transformer-based embeddings. A multi-task training setup outputting additional UD features may contort results. Taking these insights together, we propose a simple but widely applicable parser architecture and configuration, achieving new state-of-the-art results (in terms of LAS) for 10 out of 12 diverse languages.


title: On the Slightly Perturbed De Gregorio Model on $S^1$

abstract: It is conjectured that the generalization of the Constantin-Lax-Majda model (gCLM) $\omega_t + a u\omega_x = u_x \omega$ due to Okamoto, Sakajo and Wunsch can develop a finite time singularity from smooth initial data for $a < 1$. For the endpoint case where $a$ is close to and less than $1$, we prove finite time asymptotically self-similar blowup of gCLM on a circle from a class of smooth initial data. For the gCLM on a circle with the same initial data, if the strength of advection $a$ is slightly larger than $1$, we prove that the solution exists globally with $|| \omega(t)||_{H^1}$ decaying in a rate of $O(t^{-1})$ for large time. The transition threshold between two different behaviors is $a=1$, which corresponds to the De Gregorio model.


title: The metric space of limit laws for $q$-hook formulas

abstract: In earlier work, Billey--Konvalinka--Swanson studied the asymptotic distribution of the coefficients of Stanley's $q$-hook length formula, or equivalently the major index on standard tableaux of straight shape and certain skew shapes. We extend those investigations to Stanley's $q$-hook-content formula related to semistandard tableaux and $q$-hook length formulas of Bj\"orner--Wachs related to linear extensions of labeled forests. We show that, while their coefficients are ``generically'' asymptotically normal, there are uncountably many non-normal limit laws. More precisely, we introduce and completely describe the compact closure of the metric space of distributions of these statistics in several regimes. The additional limit distributions involve generalized uniform sum distributions which are topologically parameterized by certain decreasing sequence spaces with bounded $2$-norm. The closure of these distributions in the L\'evy metric gives rise to the space of DUSTPAN distributions. As an application, we completely classify the limiting distributions of the size statistic on plane partitions fitting in a box.


title: A global-local neighborhood search algorithm and tabu search for flexible job shop scheduling problem

abstract: The Flexible Job Shop Scheduling Problem (FJSP) is a combinatorial problem that continues to be studied extensively due to its practical implications in manufacturing systems and emerging new variants, in order to model and optimize more complex situations that reflect the current needs of the industry better. This work presents a new meta-heuristic algorithm called GLNSA (Global-local neighborhood search algorithm), in which the neighborhood concepts of a cellular automaton are used, so that a set of leading solutions called "smart_cells" generates and shares information that helps to optimize instances of FJSP. The GLNSA algorithm is complemented with a tabu search that implements a simplified version of the Nopt1 neighborhood defined in [1] to complement the optimization task. The experiments carried out show a satisfactory performance of the proposed algorithm, compared with other results published in recent algorithms and widely cited in the specialized bibliography, using 86 test problems, improving the optimal result reported in previous works in two of them.


title: Machine learning-based inversion of nuclear responses

abstract: A microscopic description of the interaction of atomic nuclei with external electroweak probes is required for elucidating aspects of short-range nuclear dynamics and for the correct interpretation of neutrino oscillation experiments. Nuclear quantum Monte Carlo methods infer the nuclear electroweak response functions from their Laplace transforms. Inverting the Laplace transform is a notoriously ill-posed problem; and Bayesian techniques, such as maximum entropy, are typically used to reconstruct the original response functions in the quasielastic region. In this work, we present a physics-informed artificial neural network architecture suitable for approximating the inverse of the Laplace transform. Utilizing simulated, albeit realistic, electromagnetic response functions, we show that this physics-informed artificial neural network outperforms maximum entropy in both the low-energy transfer and the quasielastic regions, thereby allowing for robust calculations of electron scattering and neutrino scattering on nuclei and inclusive muon capture rates.


title: Learning Assisted Side Channel Delay Test for Detection of Recycled ICs

abstract: With the outsourcing of design flow, ensuring the security and trustworthiness of integrated circuits has become more challenging. Among the security threats, IC counterfeiting and recycled ICs have received a lot of attention due to their inferior quality, and in turn, their negative impact on the reliability and security of the underlying devices. Detecting recycled ICs is challenging due to the effect of process variations and process drift occurring during the chip fabrication. Moreover, relying on a golden chip as a basis for comparison is not always feasible. Accordingly, this paper presents a recycled IC detection scheme based on delay side-channel testing. The proposed method relies on the features extracted during the design flow and the sample delays extracted from the target chip to build a Neural Network model using which the target chip can be truly identified as new or recycled. The proposed method classifies the timing paths of the target chip into two groups based on their vulnerability to aging using the information collected from the design and detects the recycled ICs based on the deviation of the delay of these two sets from each other.


title: A Bayesian Mixture Modelling of Stop Signal Reaction Time Distributions

abstract: The distribution of single Stop Signal Reaction Times (SSRT) in the stop signal task (SST) as a measurement of the latency of the unobservable stopping process has been modeled with a nonparametric method by Hans Colonius (1990) and with a Bayesian parametric method by Eric-Jan Wagenmakers and colleagues (2012). These methods assume equal impact of the preceding trial type (go/stop) in the SST trials on the SSRT distributional estimation without addressing the case of the violated assumption. This study presents the required model by considering two-state mixture model for the SSRT distribution. It then compares the Bayesian parametric single SSRT and mixture SSRT distributions in the usual stochastic order at the individual and the population level under the ex-Gaussian distributional format. It shows that compared to a single SSRT distribution, the mixture SSRT distribution is more diverse, more positively skewed, more leptokurtic, and larger in stochastic order. The size of the disparities in the results also depends on the choice of weights in the mixture SSRT distribution. This study confirms that mixture SSRT indices as a constant or distribution are significantly larger than their single SSRT counterparts in the related order. This offers a vital improvement in the SSRT estimations.


title: A damaged-informed lung model for ventilator waveforms

abstract: The acute respiratory distress syndrome (ARDS) is characterized by the acute development of diffuse alveolar damage (DAD) resulting in increased vascular permeability and decreased alveolar gas exchange. Mechanical ventilation is a potentially lifesaving intervention to improve oxygen exchange but has the potential to cause ventilator-induced lung injury (VILI). A general strategy to reduce VILI is to use low tidal volume and low-pressure ventilation, but optimal ventilator settings for an individual patient are difficult for the bedside physician to determine and mortality from ARDS remains unacceptably high. Motivated by the need to minimize VILI, scientists have developed models of varying complexity to understand diseased pulmonary physiology. However, simple models often fail to capture real-world injury while complex models tend to not be estimable with clinical data, limiting the clinical utility of existing models. To address this gap, we present a physiologically anchored data-driven model to better model lung injury. Our approach relies on using clinically relevant features in the ventilator waveform data that contain information about pulmonary physiology, patients-ventilator interaction and ventilator settings. Our lung model can reproduce essential physiology and pathophysiology dynamics of differently damaged lungs for both controlled mouse model data and uncontrolled human ICU data. The estimated parameters values that are correlated with a known measure of lung physiology agree with the observed lung damage. In future endeavors, this model could be used to phenotype ventilator waveforms and serve as a basis for predicting the course of ARDS and improving patient care.


title: Learning to Recognize Dialect Features

abstract: Building NLP systems that serve everyone requires accounting for dialect differences. But dialects are not monolithic entities: rather, distinctions between and within dialects are captured by the presence, absence, and frequency of dozens of dialect features in speech and text, such as the deletion of the copula in "He {} running". In this paper, we introduce the task of dialect feature detection, and present two multitask learning approaches, both based on pretrained transformers. For most dialects, large-scale annotated corpora for these features are unavailable, making it difficult to train recognizers. We train our models on a small number of minimal pairs, building on how linguists typically define dialect features. Evaluation on a test set of 22 dialect features of Indian English demonstrates that these models learn to recognize many features with high accuracy, and that a few minimal pairs can be as effective for training as thousands of labeled examples. We also demonstrate the downstream applicability of dialect feature detection both as a measure of dialect density and as a dialect classifier.


title: Stellar occultations enable milliarcsecond astrometry for Trans-Neptunian objects and Centaurs

abstract: Trans-Neptunian objects (TNOs) and Centaurs are remnants of our planetary system formation, and their physical properties have invaluable information for evolutionary theories. Stellar occultation is a ground-based method for studying these small bodies and has presented exciting results. These observations can provide precise profiles of the involved body, allowing an accurate determination of its size and shape. The goal is to show that even single-chord detections of TNOs allow us to measure their milliarcsecond astrometric positions in the reference frame of the Gaia second data release (DR2). Accurated ephemerides can then be generated, allowing predictions of stellar occultations with much higher reliability. We analyzed data from stellar occultations to obtain astrometric positions of the involved bodies. The events published before the Gaia era were updated so that the Gaia DR2 catalog is the reference. Previously determined sizes were used to calculate the position of the object center and its corresponding error with respect to the detected chord and the International Celestial Reference System (ICRS) propagated Gaia DR2 star position. We derive 37 precise astrometric positions for 19 TNOs and 4 Centaurs. Twenty-one of these events are presented here for the first time. Although about 68\% of our results are based on single-chord detection, most have intrinsic precision at the submilliarcsecond level. Lower limits on the diameter and shape constraints for a few bodies are also presented as valuable byproducts. Using the Gaia DR2 catalog, we show that even a single detection of a stellar occultation allows improving the object ephemeris significantly, which in turn enables predicting a future stellar occultation with high accuracy. Observational campaigns can be efficiently organized with this help, and may provide a full physical characterization of the involved object.


title: Multiple-excitation study of the double-resonance Raman bands in rhombohedral graphite

abstract: The double-resonance (DR) Raman process is a signature of all sp2 carbon material and provide fundamental information of the electronic structure and phonon dispersion in graphene, carbon nanotubes and different graphite-type materials. We have performed in this work the study of different DR Raman bands of rhombohedral graphite using five different excitation laser energies and obtained the dispersion of the different DR features by changing the laser energy. Results are compared with those of Bernal graphite and shows that rhombohedral graphite exhibit a richer DR Raman spectrum. For example, the 2D band of rhombohedral graphite is broader and composed by several maxima that exhibit different dispersive behavior. The occurrence of more DR conditions in rhombohedral graphite is ascribed to the fact that the volume of its Brillouin zone (BZ) is twice the volume of the Bernal BZ, allowing thus more channels for the resonance condition. The spectra of the intervalley TO-LA band of rhombohedral graphite, around 2450 cm-1, is also broader and richer in features compared to that of Bernal graphite. Results and analysis of the spectral region 1700-1850 cm-1, where different intravalley processes involving acoustic and optical phonons occurs, are also presented.


title: Improving Classification through Weak Supervision in Context-specific Conversational Agent Development for Teacher Education

abstract: Machine learning techniques applied to the Natural Language Processing (NLP) component of conversational agent development show promising results for improved accuracy and quality of feedback that a conversational agent can provide. The effort required to develop an educational scenario specific conversational agent is time consuming as it requires domain experts to label and annotate noisy data sources such as classroom videos. Previous approaches to modeling annotations have relied on labeling thousands of examples and calculating inter-annotator agreement and majority votes in order to model the necessary scenarios. This method, while proven successful, ignores individual annotator strengths in labeling a data point and under-utilizes examples that do not have a majority vote for labeling. We propose using a multi-task weak supervision method combined with active learning to address these concerns. This approach requires less labeling than traditional methods and shows significant improvements in precision, efficiency, and time-requirements than the majority vote method (Ratner 2019). We demonstrate the validity of this method on the Google Jigsaw data set and then propose a scenario to apply this method using the Instructional Quality Assessment(IQA) to define the categories for labeling. We propose using probabilistic modeling of annotator labeling to generate active learning examples to further label the data. Active learning is able to iteratively improve the training performance and accuracy of the original classification model. This approach combines state-of-the art labeling techniques of weak supervision and active learning to optimize results in the educational domain and could be further used to lessen the data requirements for expanded scenarios within the education domain through transfer learning.


title: On Convergence and Generalization of Dropout Training

abstract: We study dropout in two-layer neural networks with rectified linear unit (ReLU) activations. Under mild overparametrization and assuming that the limiting kernel can separate the data distribution with a positive margin, we show that dropout training with logistic loss achieves $\epsilon$-suboptimality in test error in $O(1/\epsilon)$ iterations.


title: Can images help recognize entities? A study of the role of images for Multimodal NER

abstract: Multimodal named entity recognition (MNER) requires to bridge the gap between language understanding and visual context. While many multimodal neural techniques have been proposed to incorporate images into the MNER task, the model's ability to leverage multimodal interactions remains poorly understood. In this work, we conduct in-depth analyses of existing multimodal fusion techniques from different perspectives and describe the scenarios where adding information from the image does not always boost performance. We also study the use of captions as a way to enrich the context for MNER. Experiments on three datasets from popular social platforms expose the bottleneck of existing multimodal models and the situations where using captions is beneficial.


title: Dual-path Self-Attention RNN for Real-Time Speech Enhancement

abstract: We propose a dual-path self-attention recurrent neural network (DP-SARNN) for time-domain speech enhancement. We improve dual-path RNN (DP-RNN) by augmenting inter-chunk and intra-chunk RNN with a recently proposed efficient attention mechanism. The combination of inter-chunk and intra-chunk attention improves the attention mechanism for long sequences of speech frames. DP-SARNN outperforms a baseline DP-RNN by using a frame shift four times larger than in DP-RNN, which leads to a substantially reduced computation time per utterance. As a result, we develop a real-time DP-SARNN by using long short-term memory (LSTM) RNN and causal attention in inter-chunk SARNN. DP-SARNN significantly outperforms existing approaches to speech enhancement, and on average takes 7.9 ms CPU time to process a signal chunk of 32 ms.


title: Gluon dynamics from an ordinary differential equation

abstract: We present a novel method for computing the nonperturbative kinetic term of the gluon propagator from an exactly solvable ordinary differential equation, whose origin is the fundamental Slavnov-Taylor identity satisfied by the three-gluon vertex, evaluated in a special kinematic limit. The main ingredients comprising the solution are a well-known projection of the three-gluon vertex, simulated on the lattice, and a particular derivative of the ghost-gluon kernel, whose approximate form is derived from a standard Schwinger-Dyson equation. Crucially, the physical requirement of a pole-free answer determines completely the form of the initial condition, whose value is calculated from a specific integral containing the same ingredients as the solution itself. This outstanding feature fixes uniquely, at least in principle, the form of the kinetic term, once the ingredients of the differential equation have been accurately evaluated. Furthermore, in the case where the gluon propagator has been independently accessed from the lattice, this property leads to the unambiguous extraction of the momentum-dependent effective gluon mass. The practical implementation of this method is carried out in detail, and the required approximations and theoretical assumptions are duly highlighted. The systematic improvement of this approach through the detailed computation of one of its pivotal components is briefly outlined.


title: Improving Noise Robustness of an End-to-End Neural Model for Automatic Speech Recognition

abstract: We present our experiments in training robust to noise an end-to-end automatic speech recognition (ASR) model using intensive data augmentation. We explore the efficacy of fine-tuning a pre-trained model to improve noise robustness, and we find it to be a very efficient way to train for various noisy conditions, especially when the conditions in which the model will be used, are unknown. Starting with a model trained on clean data helps establish baseline performance on clean speech. We carefully fine-tune this model to both maintain the performance on clean speech, and improve the model accuracy in noisy conditions. With this schema, we trained robust to noise English and Mandarin ASR models on large public corpora. All described models and training recipes are open sourced in NeMo, a toolkit for conversational AI.


title: Core-Halo Mass Relation in Scalar Field Dark Matter Models and its Consequences for the Formation of Supermassive Black Holes

abstract: Scalar-field dark matter (SFDM) halos exhibit a core-envelope structure with soliton-like cores and CDM-like envelopes. Simulations without self-interaction (free-field case) report a core-halo mass relation $M_c\propto M_{h}^{\beta}$, with either $\beta=1/3$ or $\beta=5/9$, which can be understood if core and halo obey certain energy or velocity scalings. We extend the core-halo mass relations to include SFDM with self-interaction (SI), either repulsive or attractive, and investigate its implications for the gravitational instability and collapse of solitonic cores, leading to supermassive black hole (SMBH) formation. For SFDM parameters that make $\sim$ Kpc-sized cores and CDM-like structure formation on large scales but suppressed on small scales, cores are stable for all galactic halos of interest, from the free-field to the repulsive SI limit. For attractive SI, however, halos masses $M_h\sim (10^{10}-10^{12}) M_\odot$ have cores that collapse to SMBHs with $M_{SMBH}\sim 10^{6}-10^8 M_\odot$, as observations seem to require, while smaller-mass halos have stable cores, for particle masses $m\simeq (2.14\times 10^{-22}-9.9\times 10^{-20})\rm{eV}/c^2$, if the free-field has $\beta=1/3$, or $m = 2.23\times 10^{-21}-1.7\times 10^{-18}\rm{eV}/c^2$, if $\beta=5/9$. For free-field and repulsive cases, however, if previous constraints on particle parameters are relaxed to allow much smaller (sub-galactic scale) cores, then halos can also form SMBHs, for the same range of halo and BH masses, as long as $\beta = 5/9$ is correct for the free-field. In that case, structure formation in SFDM would be largely indistinguishable from Cold Dark Matter (CDM). Such SFDM models might not resolve the small-scale structure problems of CDM, but they would explain the formation of SMBHs quite naturally. Since CDM, itself, has not yet been ruled out, such SFDM models must also be viable (Abbreviated).


title: Deep Learning for Radio-based Human Sensing: Recent Advances and Future Directions

abstract: While decade-long research has clearly demonstrated the vast potential of radio frequency (RF) for many human sensing tasks, scaling this technology to large scenarios remained problematic with conventional approaches. Recently, researchers have successfully applied deep learning to take radio-based sensing to a new level. Many different types of deep learning models have been proposed to achieve high sensing accuracy over a large population and activity set, as well as in unseen environments. Deep learning has also enabled detection of novel human sensing phenomena that were previously not possible. In this survey, we provide a comprehensive review and taxonomy of recent research efforts on deep learning based RF sensing. We also identify and compare several publicly released labeled RF sensing datasets that can facilitate such deep learning research. Finally, we summarize the lessons learned and discuss the current limitations and future directions of deep learning based RF sensing.


title: Learning Guidance Rewards with Trajectory-space Smoothing

abstract: Long-term temporal credit assignment is an important challenge in deep reinforcement learning (RL). It refers to the ability of the agent to attribute actions to consequences that may occur after a long time interval. Existing policy-gradient and Q-learning algorithms typically rely on dense environmental rewards that provide rich short-term supervision and help with credit assignment. However, they struggle to solve tasks with delays between an action and the corresponding rewarding feedback. To make credit assignment easier, recent works have proposed algorithms to learn dense "guidance" rewards that could be used in place of the sparse or delayed environmental rewards. This paper is in the same vein -- starting with a surrogate RL objective that involves smoothing in the trajectory-space, we arrive at a new algorithm for learning guidance rewards. We show that the guidance rewards have an intuitive interpretation, and can be obtained without training any additional neural networks. Due to the ease of integration, we use the guidance rewards in a few popular algorithms (Q-learning, Actor-Critic, Distributional-RL) and present results in single-agent and multi-agent tasks that elucidate the benefit of our approach when the environmental rewards are sparse or delayed.


title: Word2vec Conjecture and A Limitative Result

abstract: Being inspired by the success of \texttt{word2vec} \citep{mikolov2013distributed} in capturing analogies, we study the conjecture that analogical relations can be represented by vector spaces. Unlike many previous works that focus on the distributional semantic aspect of \texttt{word2vec}, we study the purely \emph{representational} question: can \emph{all} semantic word-word relations be represented by differences (or directions) of vectors? We call this the word2vec conjecture and point out some of its desirable implications. However, we will exhibit a class of relations that cannot be represented in this way, thus falsifying the conjecture and establishing a limitative result for the representability of semantic relations by vector spaces over fields of characteristic 0, e.g., real or complex numbers.


title: The profinite completion of the fundamental group of infinite graphs of groups

abstract: Let $(\mathcal{G},\Gamma)$ be an abstract graph of finite groups. If $\Gamma$ is finite, we can construct a profinite graph of groups in a natural way $(\hat{\mathcal{G}},\Gamma)$, where $\hat{\mathcal{G}}(m)$ is the profinite completion of $\mathcal{G}(m)$ for all $m \in \Gamma$. The main reason for this is that $\Gamma$ is finite, so it is already profinite. In this paper we deal with the infinite case, by constructing a profinite graph $\overline{\Gamma}$ where $\Gamma$ is densely embedded and then defining a profinite graph of groups $(\widehat{\mathcal{G}},\overline{\Gamma})$. We also prove that the fundamental group $\Pi_1(\widehat{\mathcal{G}},\overline{\Gamma})$ is the profinite completion of $\Pi_1^{abs}(\mathcal{G},\Gamma)$. This answers Open Question 6.7.1 of the book Profinite Graphs and Groups, published by Luis Ribes in 2017. Later we generalise the main theorem of a paper by Luis Ribes and the second author, proving that if $R$ is a virtually free abstract group and $H$ is a finitely generated subgroup of $R$, then $\overline{N_{R}(H)}=N_{\hat{R}}(\overline{H})$ answering Open Question 15.11.10 of the book of Ribes. Finally, we generalise the main theorem of a paper by Sheila Chagas and the second author, showing that every virtually free group is subgroup conjugacy separable. This answers Open Question 15.11.11 of the same book of Ribes.


title: PEP: Parameter Ensembling by Perturbation

abstract: Ensembling is now recognized as an effective approach for increasing the predictive performance and calibration of deep networks. We introduce a new approach, Parameter Ensembling by Perturbation (PEP), that constructs an ensemble of parameter values as random perturbations of the optimal parameter set from training by a Gaussian with a single variance parameter. The variance is chosen to maximize the log-likelihood of the ensemble average ($\mathbb{L}$) on the validation data set. Empirically, and perhaps surprisingly, $\mathbb{L}$ has a well-defined maximum as the variance grows from zero (which corresponds to the baseline model). Conveniently, calibration level of predictions also tends to grow favorably until the peak of $\mathbb{L}$ is reached. In most experiments, PEP provides a small improvement in performance, and, in some cases, a substantial improvement in empirical calibration. We show that this "PEP effect" (the gain in log-likelihood) is related to the mean curvature of the likelihood function and the empirical Fisher information. Experiments on ImageNet pre-trained networks including ResNet, DenseNet, and Inception showed improved calibration and likelihood. We further observed a mild improvement in classification accuracy on these networks. Experiments on classification benchmarks such as MNIST and CIFAR-10 showed improved calibration and likelihood, as well as the relationship between the PEP effect and overfitting; this demonstrates that PEP can be used to probe the level of overfitting that occurred during training. In general, no special training procedure or network architecture is needed, and in the case of pre-trained networks, no additional training is needed.


title: A set of orbital elements to fully represent the zonal harmonics around an oblate celestial body

abstract: This work introduces a new set of orbital elements to fully represent the zonal harmonics problem around an oblate celestial body. This new set of orbital elements allows to obtain a complete linear system for the unperturbed problem and, in addition, a complete polynomial system when considering the perturbation produced by the zonal harmonics from the gravitational force of an oblate celestial body. These orbital elements present no singularities and are able to represent any kind of orbit, including elliptic, parabolic and hyperbolic orbits. In addition, an application to this formulation of the Poincar\'e-Lindstedt perturbation method is included to obtain an approximate first order solution of the problem for the case of the J2 perturbation.


title: Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation

abstract: Despite significant progress, state-of-the-art abstractive summarization methods are still prone to hallucinate content inconsistent with the source document. In this paper, we propose Constrained Abstractive Summarization (CAS), a general setup that preserves the factual consistency of abstractive summarization by specifying tokens as constraints that must be present in the summary. We adopt lexically constrained decoding, a technique generally applicable to autoregressive generative models, to fulfill CAS and conduct experiments in two scenarios: (1) automatic summarization without human involvement, where keyphrases are extracted from the source document and used as constraints; (2) human-guided interactive summarization, where human feedback in the form of manual constraints are used to guide summary generation. Automatic and human evaluations on two benchmark datasets demonstrate that CAS improves both lexical overlap (ROUGE) and factual consistency of abstractive summarization. In particular, we observe up to 13.8 ROUGE-2 gains when only one manual constraint is used in interactive summarization.


title: Multiparametric Cardiac 18F-FDG PET: Pilot Comparison of FDG Delivery Rate with 82Rb Myocardial Blood Flow

abstract: Myocardial blood flow (MBF) and flow reserve are usually quantified in the clinic with positron emission tomography (PET) using a perfusion-specific radiotracer (e.g. 82Rbchloride). However, the clinical accessibility of existing perfusion tracers remains limited. Meanwhile, 18F-fluorodeoxyglucose (FDG) is a commonly used radiotracer for PET metabolic imaging without similar limitations. In this paper, we explore the potential of 18F-FDG for myocardial perfusion imaging by comparing the myocardial FDG delivery rate K1 with MBF as determined by dynamic 82Rb PET in fourteen human subjects with heart disease. Two sets of FDG K1 were derived from one-hour dynamic FDG scans. One was the original FDG K1 estimates and the other was the corresponding K1 values that were linearly normalized for blood glucose levels. A generalized Renkin-Crone model was used to fit FDG K1 with Rb MBF, which then allowed for a nonlinear extraction fraction correction for converting FDG K1 to MBF. The linear correlation between FDG-derived MBF and Rb MBF was moderate (r=0.79) before the glucose normalization and became much improved (r>0.9) after glucose normalization. The extraction fraction of FDG was also similar to that of Rb-chloride in the myocardium. The results from this pilot study suggest that dynamic cardiac FDG-PET with tracer kinetic modeling has the potential to provide MBF in addition to its conventional use for metabolic imaging.


title: Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?

abstract: Sequence-to-sequence models excel at handling natural language variation, but have been shown to struggle with out-of-distribution compositional generalization. This has motivated new specialized architectures with stronger compositional biases, but most of these approaches have only been evaluated on synthetically-generated datasets, which are not representative of natural language variation. In this work we ask: can we develop a semantic parsing approach that handles both natural language variation and compositional generalization? To better assess this capability, we propose new train and test splits of non-synthetic datasets. We demonstrate that strong existing approaches do not perform well across a broad set of evaluations. We also propose NQG-T5, a hybrid model that combines a high-precision grammar-based approach with a pre-trained sequence-to-sequence model. It outperforms existing approaches across several compositional generalization challenges on non-synthetic data, while also being competitive with the state-of-the-art on standard evaluations. While still far from solving this problem, our study highlights the importance of diverse evaluations and the open challenge of handling both compositional generalization and natural language variation in semantic parsing.


title: Design and Control of SQUEEZE: A Spring-augmented QUadrotor for intEractions with the Environment to squeeZE-and-fly

abstract: This paper presents the design and control of a novel quadrotor with a variable geometry to physically interact with cluttered environments and fly through narrow gaps and passageways. This compliant quadrotor with passive morphing capabilities is designed using torsional springs at every arm hinge to allow for rotation driven by external forces. We derive the dynamic model of this variable geometry quadrotor (SQUEEZE), and develop an adaptive controller for trajectory tracking. The corresponding Lyapunov stability proof of attitude tracking is also presented. Further, an admittance controller is designed to account for changes in yaw due to physical interactions with the environment. Finally, the proposed design is validated in flight tests with two setups: a small gap and a passageway. The experimental results demonstrate the unique capability of the SQUEEZE in navigating through constrained narrow spaces.


title: Cognitive Reflection Test and the Polarizing Force-Identification Questions in the FCI

abstract: The set of polarizing force-identification (PFI) questions in the FCI consists of six items all basically asking only one question: the set of forces acting on a given body. Although it may sound trivial, these questions are among the most challenging in the FCI. In this work involving 163 students, we investigate the correlation between student performance on the set of PFI questions and the Cognitive Reflection Test. We find that for both scores in the FCI as a whole and in the PFI questions, the range of values of the Pearson coefficient at 95\% confidence interval, is suggestive that cognitive reflection may be one of the contributing factors in the student performance in the FCI. This is consistent with the idea that high level of cognitive reflection may help in eliminating seemingly valid choices (misconceptions) in the FCI that are intuitive from everyday experience or "common sense" but otherwise misleading. The ability to activate System 2 in Dual Process Theory, whether from System 1 or right after reading a physics problem, may contribute in narrowing down the set of prospective valid answers in a given physics problem. Complementary to cognitive reflection are other factors associated with deep understanding of physics whose effects are expected to become more evident with the level of difficulty of a set of physics problems. Given two students with the same level of cognitive reflection, the one with deeper understanding of physics is more likely to get the correct answer. In our analysis, the range of correlation coefficient for the set of PFI questions is downshifted with respect to that for the FCI as a whole. This may be attributed to the more challenging nature of the latter compared to a significant fraction of the remaining questions in the former.


title: Differentiate Quality of Experience Scheduling for Deep Learning Inferences with Docker Containers in the Cloud

abstract: With the prevalence of big-data-driven applications, such as face recognition on smartphones and tailored recommendations from Google Ads, we are on the road to a lifestyle with significantly more intelligence than ever before. Various neural network powered models are running at the back end of their intelligence to enable quick responses to users. Supporting those models requires lots of cloud-based computational resources, e.g., CPUs and GPUs. The cloud providers charge their clients by the amount of resources that they occupy. Clients have to balance the budget and quality of experiences (e.g., response time). The budget leans on individual business owners, and the required Quality of Experience (QoE) depends on usage scenarios of different applications. For instance, an autonomous vehicle requires an real-time response, but unlocking your smartphone can tolerate delays. However, cloud providers fail to offer a QoE-based option to their clients. In this paper, we propose DQoES, differentiated quality of experience scheduler for deep learning inferences. DQoES accepts clients' specifications on targeted QoEs, and dynamically adjusts resources to approach their targets. Through the extensive cloud-based experiments, DQoES demonstrates that it can schedule multiple concurrent jobs with respect to various QoEs and achieve up to 8x times more satisfied models when compared to the existing system


title: ANLIzing the Adversarial Natural Language Inference Dataset

abstract: We perform an in-depth error analysis of Adversarial NLI (ANLI), a recently introduced large-scale human-and-model-in-the-loop natural language inference dataset collected over multiple rounds. We propose a fine-grained annotation scheme of the different aspects of inference that are responsible for the gold classification labels, and use it to hand-code all three of the ANLI development sets. We use these annotations to answer a variety of interesting questions: which inference types are most common, which models have the highest performance on each reasoning type, and which types are the most challenging for state of-the-art models? We hope that our annotations will enable more fine-grained evaluation of models trained on ANLI, provide us with a deeper understanding of where models fail and succeed, and help us determine how to train better models in future.


title: Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality

abstract: Byte-pair encoding (BPE) is a ubiquitous algorithm in the subword tokenization process of language models as it provides multiple benefits. However, this process is solely based on pre-training data statistics, making it hard for the tokenizer to handle infrequent spellings. On the other hand, though robust to misspellings, pure character-level models often lead to unreasonably long sequences and make it harder for the model to learn meaningful words. To alleviate these challenges, we propose a character-based subword module (char2subword) that learns the subword embedding table in pre-trained models like BERT. Our char2subword module builds representations from characters out of the subword vocabulary, and it can be used as a drop-in replacement of the subword embedding table. The module is robust to character-level alterations such as misspellings, word inflection, casing, and punctuation. We integrate it further with BERT through pre-training while keeping BERT transformer parameters fixed--and thus, providing a practical method. Finally, we show that incorporating our module to mBERT significantly improves the performance on the social media linguistic code-switching evaluation (LinCE) benchmark.


title: The DKU-DukeECE Systems for VoxCeleb Speaker Recognition Challenge 2020

abstract: In this paper, we present the system submission for the VoxCeleb Speaker Recognition Challenge 2020 (VoxSRC-20) by the DKU-DukeECE team. For track 1, we explore various kinds of state-of-the-art front-end extractors with different pooling layers and objective loss functions. For track 3, we employ an iterative framework for self-supervised speaker representation learning based on a deep neural network (DNN). For track 4, we investigate the whole system pipeline for speaker diarization, including voice activity detection (VAD), uniform segmentation, speaker embedding extraction, and clustering.


title: Octave-Tunable Magnetostatic Wave YIG Resonators on a Chip

abstract: We have designed, fabricated, and characterized magnetostatic wave (MSW) resonators on a chip. The resonators are fabricated by patterning single-crystal yttrium iron garnet (YIG) film on a gadolinium gallium garnet (GGG) substrate and excited by loop-inductor transducers. We achieved this technology breakthrough by developing a YIG film etching process and fabricating thick aluminum coplanar waveguide (CPW) inductor loop around each resonator to individually address and excite MSWs. At 4.77 GHz, the 0.68 square mm resonator achieves a quality factor Q > 5000 with a bias field of 987 Oe. We also demonstrate YIG resonator tuning by more than one octave from 3.63 to 7.63 GHz by applying an in-plane external magnetic field. The measured quality factor of the resonator is consistently over 3000 above 4 GHz. The micromachining technology enables the fabrication of multiple single- and two-port YIG resonators on the same chip with all resonators demonstrating octave tunability and high Q .


title: Learning Fine-Grained Cross Modality Excitement for Speech Emotion Recognition

abstract: Speech emotion recognition is a challenging task because the emotion expression is complex, multimodal and fine-grained. In this paper, we propose a novel multimodal deep learning approach to perform fine-grained emotion recognition from real-life speeches. We design a temporal alignment mean-max pooling mechanism to capture the subtle and fine-grained emotions implied in every utterance. In addition, we propose a cross modality excitement module to conduct sample-specific adjustment on cross modality embeddings and adaptively recalibrate the corresponding values by its aligned latent features from the other modality. Our proposed model is evaluated on two well-known real-world speech emotion recognition datasets. The results demonstrate that our approach is superior on the prediction tasks for multimodal speech utterances, and it outperforms a wide range of baselines in terms of prediction accuracy. Further more, we conduct detailed ablation studies to show that our temporal alignment mean-max pooling mechanism and cross modality excitement significantly contribute to the promising results. In order to encourage the research reproducibility, we make the code publicly available at \url{https://github.com/tal-ai/FG_CME.git}.


title: REMIX: Efficient Range Query for LSM-trees

abstract: LSM-tree based key-value (KV) stores organize data in a multi-level structure for high-speed writes. Range queries on traditional LSM-trees must seek and sort-merge data from multiple table files on the fly, which is expensive and often leads to mediocre read performance. To improve range query efficiency on LSM-trees, we introduce a space-efficient KV index data structure, named REMIX, that records a globally sorted view of KV data spanning multiple table files. A range query on multiple REMIX-indexed data files can quickly locate the target key using a binary search, and retrieve subsequent keys in sorted order without key comparisons. We build RemixDB, an LSM-tree based KV-store that adopts a write-efficient compaction strategy and employs REMIXes for fast point and range queries. Experimental results show that REMIXes can substantially improve range query performance in a write-optimized LSM-tree based KV-store.


title: Boundary problems for three-dimensional Dirac operators and generalized MIT bag models for unbounded domains

abstract: We consider operators of boundary value problems for 3D- Dirac operators in unbounded domains with the uniformly regular boundary. We give effective conditions of self-adjointness of operators under consideration and a description of their essential spectra. We also give applications to operators of the MIT bag problems for unbounded domains


title: Conditional beta and uncertainty factor in the cryptocurrency pricing model

abstract: This research is to assess cryptocurrencies with the conditional beta, compared with prior studies based on unconditional beta or fixed beta. It is a new approach to building a pricing model for cryptocurrencies. Therefore, we expect that the use of conditional beta will increase the explanatory ability of factors in previous pricing models. Besides, this research is also a pioneer in placing the uncertainty factor in the cryptocurrency pricing model. Earlier studies on cryptocurrency pricing have ignored this factor. However, it is a significant factor in the valuation of cryptocurrencies because uncertainty leads to investor sentiment and affects prices.


title: Real-time Non-line-of-Sight imaging of dynamic scenes

abstract: Non-Line-of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. In the past, this method has suffered from the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. While algorithms capable of reconstructing scenes at several frames per second have been demonstrated, real-time NLOS video has only been demonstrated for retro-reflective objects where the NLOS signal strength is enhanced by 4 orders of magnitude or more. Furthermore, it has also been noted that the signal-to-noise ratio of reconstructions in NLOS methods drops quickly with distance and past reconstructions, therefore, have been limited to small scenes with depths of few meters. Actual models of noise and resolution in the scene have been simplistic, ignoring many of the complexities of the problem. We show that SPAD (Single-Photon Avalanche Diode) array detectors with a total of just 28 pixels combined with a specifically extended Phasor Field reconstruction algorithm can reconstruct live real-time videos of non-retro-reflective NLOS scenes. We provide an analysis of the Signal-to-Noise-Ratio (SNR) of our reconstructions and show that for our method it is possible to reconstruct the scene such that SNR, motion blur, angular resolution, and depth resolution are all independent of scene size suggesting that reconstruction of very large scenes may be possible. In the future, the light efficiency for NLOS imaging systems can be improved further by adding more pixels to the sensor array.


title: Remarks on Viscosity Super-Solutions of Quasi-Variational Inequalities

abstract: For Hamilton-Jacobi-Bellman (HJB) equations, with the standard definitions of viscosity super-solution and sub-solution, it is known that there is a comparison between any (viscosity) super-solutions and sub-solutions. This should be the same for HJB type quasi-variational inequalities (QVIs) arising from optimal impulse control problems. However, according to a natural adoption of the definition found in Barles 1985, Barles 1985b, the uniqueness of the viscosity solution could be guaranteed, but the comparison between viscosity super- and sub-solutions could not be guaranteed. This paper introduces a modification of the definition for the viscosity super-solution of HJB type QVIs so that the desired comparison theorem will hold.


title: Venus Observations at 40 and 90 GHz with CLASS

abstract: Using the Cosmology Large Angular Scale Surveyor, we measure the disk-averaged absolute Venus brightness temperature to be 432.3 $\pm$ 2.8 K and 355.6 $\pm$ 1.3 K in the Q and W frequency bands centered at 38.8 and 93.7 GHz, respectively. At both frequency bands, these are the most precise measurements to date. Furthermore, we observe no phase dependence of the measured temperature in either band. Our measurements are consistent with a CO$_2$-dominant atmospheric model that includes trace amounts of additional absorbers like SO$_2$ and H$_2$SO$_4$.


title: Fluid viscoelasticity triggers fast transitions of a Brownian particle in a double well optical potential

abstract: Thermally activated transitions are ubiquitous in nature, occurring in complex environments which are typically conceived as ideal viscous fluids. We report the first direct observations of a Brownian bead transiting between the wells of a bistable optical potential in a viscoelastic fluid with a single long relaxation time. We precisely characterize both the potential and the fluid, thus enabling a neat comparison between our experimental results and a theoretical model based on the generalized Langevin equation. Our findings reveal a drastic amplification of the transition rates compared to those in a Newtonian fluid, stemming from the relaxation of the fluid during the particle crossing events.


title: An Evaluation Protocol for Generative Conversational Systems

abstract: There is a multitude of novel generative models for open-domain conversational systems; however, there is no systematic evaluation of different systems. Systematic comparisons require consistency in experimental design, evaluation sets, conversational systems and their outputs, and statistical analysis. We lay out a protocol for the evaluation of conversational models using head-to-head pairwise comparison. We analyze ten recent models that claim state-of-the-art performance using a paired head-to-head performance (win-loss-tie) on five evaluation datasets. Our findings show that DialoGPT and Blender are superior systems using Bradley-Terry model and TrueSkill ranking methods. These findings demonstrate the feasibility of our protocol to evaluate conversational agents and evaluation sets. Finally, we make all code and evaluations publicly available for researchers to compare their model to other state-of-the-art dialog models.


title: Text Style Transfer: A Review and Experimental Evaluation

abstract: The stylistic properties of text have intrigued computational linguistics researchers in recent years. Specifically, researchers have investigated the Text Style Transfer (TST) task, which aims to change the stylistic properties of the text while retaining its style independent content. Over the last few years, many novel TST algorithms have been developed, while the industry has leveraged these algorithms to enable exciting TST applications. The field of TST research has burgeoned because of this symbiosis. This article aims to provide a comprehensive review of recent research efforts on text style transfer. More concretely, we create a taxonomy to organize the TST models and provide a comprehensive summary of the state of the art. We review the existing evaluation methodologies for TST tasks and conduct a large-scale reproducibility study where we experimentally benchmark 19 state-of-the-art TST algorithms on two publicly available datasets. Finally, we expand on current trends and provide new perspectives on the new and exciting developments in the TST field.


title: Detection of Cyclopropenylidene on Titan with ALMA

abstract: We report the first detection on Titan of the small cyclic molecule cyclopropenylidene (c-C3H2) from high sensitivity spectroscopic observations made with the Atacama Large Millimeter/sub-millimeter Array (ALMA). Multiple lines of cyclopropenylidene were detected in two separate datasets: ~251 GHz in 2016 (Band 6) and ~352 GHz in 2017 (Band 7). Modeling of these emissions indicates abundances of 0.50 +/- 0.14 ppb (2016) and 0.28 +/- 0.08 (2017) for a 350 km step model, which may either signify a decrease in abundance, or a mean value of 0.33 +/- 0.07 ppb. Inferred column abundances are (3-5)E12 cm-2 in 2016 and (1-2)E12 cm-2 in 2017, similar to photochemical model predictions. Previously the C3H3+ ion has been measured in Titan's ionosphere by Cassini's Ion and Neutral Mass Spectrometer (INMS), but the neutral (unprotonated) species has not been detected until now, and aromatic versus aliphatic structure could not be determined by the INMS. Our work therefore represents the first unambiguous detection of cyclopropenylidene, the second known cyclic molecule in Titan's atmosphere along with benzene (C6H6) and the first time this molecule has been detected in a planetary atmosphere. We also searched for the N-heterocycle molecules pyridine and pyrimidine finding non-detections in both cases, and determining 2-{\sigma} upper limits of 1.15 ppb (c-C5H5N) and 0.85 ppb (c-C4H4N2) for uniform abundances above 300 km. These new results on cyclic molecules provide fresh constraints on photochemical pathways in Titan's atmosphere, and will require new modeling and experimental work to fully understand the implications for complex molecule formation.


title: Spitzer Reveals Evidence of Molecular Absorption in the Atmosphere of the Hot Neptune LTT 9979b

abstract: Non-rocky sub-jovian exoplanets in high irradiation environments are rare. LTT 9979b, also known as TESS Object of Interest (TOI) 193.01, is one of the few such planets discovered to date, and the first example of an ultra-hot Neptune. The planet's bulk density indicates that it has a substantial atmosphere, so to investigate its atmospheric composition and shed further light on its origin, we obtained {\it Spitzer} IRAC secondary eclipse observations of LTT 9979b at 3.6 and 4.5 $\mu$m. We combined the {\it Spitzer} observations with a measurement of the secondary eclipse in the {\it TESS} bandpass. The resulting secondary eclipse spectrum strongly prefers a model that includes CO absorption over a blackbody spectrum, incidentally making LTT 9979b the first {\it TESS} exoplanet (and the first ultra-hot Neptune) with evidence of a spectral feature in its atmosphere. We did not find evidence of a thermal inversion, at odds with expectations based on the atmospheres of similarly-irradiated hot Jupiters. We also report a nominal dayside brightness temperature of 2305 $\pm$ 141 K (based on the 3.6 $\mu$m secondary eclipse measurement), and we constrained the planet's orbital eccentricity to $e < 0.01$ at the 99.7 \% confidence level. Together with our analysis of LTT 9979b's thermal phase curves reported in a companion paper, our results set the stage for similar investigations of a larger sample of exoplanets discovered in the hot Neptune desert, investigations which are key to uncovering the origin of this population.


title: Phase Curves of Hot Neptune LTT 9779b Suggest a High-Metallicity Atmosphere

abstract: Phase curve measurements provide a global view of the composition, thermal structure, and dynamics of exoplanet atmospheres. Although most of the dozens of phase curve measurements made to date are of large, massive hot Jupiters, there is considerable interest in probing the atmospheres of the smaller planets that are the more typical end product of planet formation. One such planet is the ultra-hot Neptune LTT 9779b, a rare denizen of the Neptune desert. A companion paper presents the planet's secondary eclipses and day-side thermal emission spectrum; in this work we describe the planet's optical and infrared phase curves, characterized using Spitzer and TESS photometry. We detect LTT 9779b's thermal phase variations at 4.5um, finding a phase amplitude of 358+/-106 ppm and a longitude of peak emission -10 deg +/- 21 deg east of the substellar point. Combined with our secondary eclipse observations, these phase curve measurements imply a 4.5um day-side brightness temperature of 1800+/-120 K, a night-side brightness temperature of 700+/-430 K (<1350 K at 2 sigma confidence), and a day-night brightness temperature contrast of 1110+/-460 K. We compare our data to the predictions of 3D GCMs and to similar observations of hot Jupiters experiencing similar levels of stellar irradiation. Though not conclusive, our measurement of its small 4.5um phase offset, the relatively large amplitude of the phase variation, and the qualitative differences between our target's day-side emission spectrum and those of hot Jupiters of similar temperatures all suggest a super-Solar atmospheric metallicity for LTT 9779b, as might be expected given its size and mass. Finally, we provide a refined ephemeris (P=0.79207022+/-0.00000069 d, T0=2458783.51636+/-0.00027, BJD_TDB) to enable efficient scheduling of future observations to further characterize the atmosphere of this intriguing planet. (abstract abridged)


title: LCFI: A Fault Injection Tool for Studying Lossy Compression Error Propagation in HPC Programs

abstract: Error-bounded lossy compression is becoming more and more important to today's extreme-scale HPC applications because of the ever-increasing volume of data generated because it has been widely used in in-situ visualization, data stream intensity reduction, storage reduction, I/O performance improvement, checkpoint/restart acceleration, memory footprint reduction, etc. Although many works have optimized ratio, quality, and performance for different error-bounded lossy compressors, there is none of the existing works attempting to systematically understand the impact of lossy compression errors on HPC application due to error propagation. In this paper, we propose and develop a lossy compression fault injection tool, called LCFI. To the best of our knowledge, this is the first fault injection tool that helps both lossy compressor developers and users to systematically and comprehensively understand the impact of lossy compression errors on HPC programs. The contributions of this work are threefold: (1) We propose an efficient approach to inject lossy compression errors according to a statistical analysis of compression errors for different state-of-the-art compressors. (2) We build a fault injector which is highly applicable, customizable, easy-to-use in generating top-down comprehensive results, and demonstrate the use of LCFI. (3) We evaluate LCFI on four representative HPC benchmarks with different abstracted fault models and make several observations about error propagation and their impacts on program outputs.


title: Scale-, shift- and rotation-invariant diffractive optical networks

abstract: Recent research efforts in optical computing have gravitated towards developing optical neural networks that aim to benefit from the processing speed and parallelism of optics/photonics in machine learning applications. Among these endeavors, Diffractive Deep Neural Networks (D2NNs) harness light-matter interaction over a series of trainable surfaces, designed using deep learning, to compute a desired statistical inference task as the light waves propagate from the input plane to the output field-of-view. Although, earlier studies have demonstrated the generalization capability of diffractive optical networks to unseen data, achieving e.g., >98% image classification accuracy for handwritten digits, these previous designs are in general sensitive to the spatial scaling, translation and rotation of the input objects. Here, we demonstrate a new training strategy for diffractive networks that introduces input object translation, rotation and/or scaling during the training phase as uniformly distributed random variables to build resilience in their blind inference performance against such object transformations. This training strategy successfully guides the evolution of the diffractive optical network design towards a solution that is scale-, shift- and rotation-invariant, which is especially important and useful for dynamic machine vision applications in e.g., autonomous cars, in-vivo imaging of biomedical specimen, among others.


title: Metastable soliton necklaces supported by fractional diffraction and competing nonlinearities

abstract: We demonstrate that fractional cubic-quintic nonlinear Schr\"odinger equation,characterized by its L\'evy index, maintains ring-shaped soliton clusters ("necklaces") carrying orbital angular momentum. They can be built, in the respective optical setting, as circular chains of fundamental solitons linked by a vortical phase field. We predict semi-analytically that the metastable necklace-shaped clusters persist, corresponding to a local minimum of an effective potential of interaction between adjacent solitons in the cluster. Systematic simulations corroborate that the clusters stay robust over extremely large propagation distances, even in the presence of strong random perturbations.


title: How are gamma-ray burst radio afterglows populated?

abstract: We systematically analyze three GRB samples named as radio-loud, radio-quiet and radio-none afterglows, respectively. It is shown that dichotomy of the radio-loud afterglows is not necessary. Interestingly, we find that the intrinsic durations ($T_{int}$), isotropic energies of prompt gamma-rays ($E_{\gamma, iso}$) and redshifts ($z$) of their host galaxies are log-normally distributed for both the radio-loud and radio-quiet samples except those GRBs without any radio detections. Based on the distinct distributions of $T_{int}$, $E_{\gamma, iso}$, the circum-burst medium density ($n$) and the isotropic equivalent energy of radio afterglows ($L_{\nu,p}$), we confirm that the GRB radio afterglows are really better to be divided into the dim and the bright types. However, it is noticeable that the distributions of flux densities ($F_{host}$) from host galaxies of both classes of radio afterglows are intrinsically quite similar. Meanwhile, we point out that the radio-none sample is also obviously different from the above two samples with radio afterglows observed, according to the cumulative frequency distributions of the $T_{int}$ and the $E_{\gamma, iso}$, together with correlations between $T_{int}$ and $z$. In addition, a positive correlation between $E_{\gamma, iso}$ and $L_{\nu,p}$ is found in the radio-loud samples especially for the supernova-associated GRBs. Besides, we also find this positive correlation in the radio-quiet sample. A negative correlation between $T_{int}$ and $z$ is confirmed to hold for the radio-quiet sample too. The dividing line between short and long GRBs in the rest frame is at $T_{int}\simeq$1 s. Consequently, we propose that the radio-loud, the radio-quiet and the radio-none GRBs could be originated from different progenitors.


title: Refinements of norm and numerical radius inequalities

abstract: Several refinements of norm and numerical radius inequalities of bounded linear operators on a complex Hilbert space are given. In particular, we show that if $A$ is a bounded linear operator on a complex Hilbert space, then $$ \frac{1}{4}\|A^*A+AA^*\| \leq \frac{1}{8}\bigg( \|A+A^*\|^2+\|A-A^*\|^2 +c^2(A+A^*)+c^2(A-A^*)\bigg) \leq w^2(A)$$ and \begin{eqnarray*} \frac{1}{2}\|A^*A+AA^*\| - \frac{1}{4}\bigg\|(A+A^*)^2 (A-A^*)^2 \bigg\|^{1/2} \leq w^2(A) \leq \frac{1}{2}\|A^*A+AA^*\|, \end{eqnarray*} %$$ \frac{1}{4}\|A^*A+AA^*\| \leq \frac{1}{2}w^2(A) + \frac{1}{8}\bigg\|(A+A^*)^2 (A-A^*)^2 \bigg\|^{1/2}\leq w^2(A),$$ where $\|.\|$, $w(.)$ and $c(.)$ are the operator norm, the numerical radius and the Crawford number, respectively. Further, we prove that if $A,D$ are bounded linear operators on a complex Hilbert space, then \begin{eqnarray*} \|AD^*\| \leq \left\| \int_0^1 \left( (1-t) \left(\frac{ |A|^2+|D|^2}{2}\right) +t\|AD^*\|I \right)^2dt \right\|^{1/2} \leq \frac{1}{2}\left\| |A|^2+|D|^2 \right\|, \end{eqnarray*} where $|A|=(A^*A)^{1/2}$ and $|D|=(D^*D)^{1/2}$. This is a refinement of well known inequality obtained by Bhatia and Kittaneh.


title: Model Extraction Attacks on Graph Neural Networks: Taxonomy and Realization

abstract: Machine learning models are shown to face a severe threat from Model Extraction Attacks, where a well-trained private model owned by a service provider can be stolen by an attacker pretending as a client. Unfortunately, prior works focus on the models trained over the Euclidean space, e.g., images and texts, while how to extract a GNN model that contains a graph structure and node features is yet to be explored. In this paper, for the first time, we comprehensively investigate and develop model extraction attacks against GNN models. We first systematically formalise the threat modelling in the context of GNN model extraction and classify the adversarial threats into seven categories by considering different background knowledge of the attacker, e.g., attributes and/or neighbour connections of the nodes obtained by the attacker. Then we present detailed methods which utilise the accessible knowledge in each threat to implement the attacks. By evaluating over three real-world datasets, our attacks are shown to extract duplicated models effectively, i.e., 84% - 89% of the inputs in the target domain have the same output predictions as the victim model.


title: Deriving Time-varying Cellular Motility Parameters via Wavelet Analysis

abstract: Cell migration is an indispensable physiological and pathological process for normal tissue development and cancer metastasis, which is greatly regulated by intracellular signal pathways and extracellular microenvironment (ECM). However, there is a lack of adequate tools to analyze the time-varying cell migration characteristics because of the effects of some factors, i.e., the ECM including the time-dependent local stiffness due to microstructural remodeling by migrating cells. Here, we develop an approach to derive the time-dependent motility parameters from cellular trajectories, based on the time-varying persistent random walk model. In particular, we employ the wavelet denoising and wavelet transform to investigate cell migration velocities and obtain the wavelet power spectrum. The time-dependent motility parameters are subsequently derived via Lorentzian power spectrum. Our analysis shows that the combination of wavelet denoising, wavelet transform and Lorentzian power spectrum provides a powerful tool to derive accurately the time-dependent motility parameters, which reflects the time-varying microenvironment characteristics to some extent.


title: Temporal Reasoning on Implicit Events from Distant Supervision

abstract: We propose TRACIE, a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit events -- events that are not mentioned explicitly in natural language text but can be inferred from it. This introduces a new challenge in temporal reasoning research, where prior work has focused on explicitly mentioned events. Human readers can infer implicit events via commonsense reasoning, resulting in a more comprehensive understanding of the situation and, consequently, better reasoning about time. We find, however, that state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events. To address this, we propose a neuro-symbolic temporal reasoning model, SYMTIME, which exploits distant supervision signals from large-scale text and uses temporal rules to combine start times and durations to infer end times. SYMTIME outperforms strong baseline systems on TRACIE by 5%, and by 11% in a zero prior knowledge training setting. Our approach also generalizes to other temporal reasoning tasks, as evidenced by a gain of 1%-9% on MATRES, an explicit event benchmark.


title: Autoencoder Watchdog Outlier Detection for Classifiers

abstract: Neural networks have often been described as black boxes. A generic neural network trained to differentiate between kittens and puppies will classify a picture of a kumquat as a kitten or a puppy. An autoencoder watch dog screens trained classifier/regression machine input candidates before processing, e.g. to first test whether the neural network input is a puppy or a kitten. Preliminary results are presented using convolutional neural networks and convolutional autoencoder watchdogs using MNIST images.


title: Effective Distant Supervision for Temporal Relation Extraction

abstract: A principal barrier to training temporal relation extraction models in new domains is the lack of varied, high quality examples and the challenge of collecting more. We present a method of automatically collecting distantly-supervised examples of temporal relations. We scrape and automatically label event pairs where the temporal relations are made explicit in text, then mask out those explicit cues, forcing a model trained on this data to learn other signals. We demonstrate that a pre-trained Transformer model is able to transfer from the weakly labeled examples to human-annotated benchmarks in both zero-shot and few-shot settings, and that the masking scheme is important in improving generalization.