title: Parallel Algorithm for Non-Monotone DR-Submodular Maximization

abstract: In this work, we give a new parallel algorithm for the problem of maximizing a non-monotone diminishing returns submodular function subject to a cardinality constraint. For any desired accuracy $\epsilon$, our algorithm achieves a $1/e - \epsilon$ approximation using $O(\log{n} \log(1/\epsilon) / \epsilon^3)$ parallel rounds of function evaluations. The approximation guarantee nearly matches the best approximation guarantee known for the problem in the sequential setting and the number of parallel rounds is nearly-optimal for any constant $\epsilon$. Previous algorithms achieve worse approximation guarantees using $\Omega(\log^2{n})$ parallel rounds. Our experimental evaluation suggests that our algorithm obtains solutions whose objective value nearly matches the value obtained by the state of the art sequential algorithms, and it outperforms previous parallel algorithms in number of parallel rounds, iterations, and solution quality.


title: Probing the Majorana Nature in Radiative Neutrino mass models with the same-sign dilepton final states at future colliders

abstract: Neutrinos in the Standard Model (SM) are considered to be massless which is in contradiction with the evidence from the neutrino oscillation data. These experiments established that at least two SM neutrinos have non-zero masses and that individual lepton numbers are violated. This is strong evidence of new physics beyond the SM that should be responsible for generating non zero mass for the neutrinos. In this work, we study the collider phenomenology of an extension of the SM where neutrinos are generated radiatively at three-loop. We show that the production of same-sign dilepton at lepton colliders (such as FCC-ee and ILC) can be used to probe the Majorana nature of neutrinos in this class of models.


title: On non-exponential cosmological solutions with two factor spaces of dimensions $m$ and $ 1$ in the Einstein-Gauss-Bonnet model with a $\Lambda$-term

abstract: We consider a $(m+2)$-dimensional Einstein-Gauss-Bonnet model with the cosmological $\Lambda$-term. We restrict the metrics to be diagonal ones and find for certain $\Lambda = \Lambda(m)$ class of cosmological solutions with non-exponential time dependence of two scale factors of dimensions $m > 2$ and $1$. Any solutions from this class describes an accelerated expansion of $m$-dimensional subspace and tends asymptotically to isotropic solution with exponental dependence of scale factors.


title: New Algorithms for Functional Distributed Constraint Optimization Problems

abstract: The Distributed Constraint Optimization Problem (DCOP) formulation is a powerful tool to model multi-agent coordination problems that are distributed by nature. The formulation is suitable for problems where variables are discrete and constraint utilities are represented in tabular form. However, many real-world applications have variables that are continuous and tabular forms thus cannot accurately represent constraint utilities. To overcome this limitation, researchers have proposed the Functional DCOP (F-DCOP) model, which are DCOPs with continuous variables. But existing approaches usually come with some restrictions on the form of constraint utilities and are without quality guarantees. Therefore, in this paper, we (i) propose exact algorithms to solve a specific subclass of F-DCOPs; (ii) propose approximation methods with quality guarantees to solve general F-DCOPs; and (iii) empirically show that our algorithms outperform existing state-of-the-art F-DCOP algorithms on randomly generated instances when given the same communication limitations.


title: Efficient Covariance Estimation from Temporal Data

abstract: Estimating the covariance structure of multivariate time series is a fundamental problem with a wide-range of real-world applications -- from financial modeling to fMRI analysis. Despite significant recent advances, current state-of-the-art methods are still severely limited in terms of scalability, and do not work well in high-dimensional undersampled regimes. In this work we propose a novel method called Temporal Correlation Explanation, or T-CorEx, that (a) has linear time and memory complexity with respect to the number of variables, and can scale to very large temporal datasets that are not tractable with existing methods; (b) gives state-of-the-art results in highly undersampled regimes on both synthetic and real-world datasets; and (c) makes minimal assumptions about the character of the dynamics of the system. T-CorEx optimizes an information-theoretic objective function to learn a latent factor graphical model for each time period and applies two regularization techniques to induce temporal consistency of estimates. We perform extensive evaluation of T-Corex using both synthetic and real-world data and demonstrate that it can be used for detecting sudden changes in the underlying covariance matrix, capturing transient correlations and analyzing extremely high-dimensional complex multivariate time series such as high-resolution fMRI data.


title: Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence

abstract: Regularization is typically understood as improving generalization by altering the landscape of local extrema to which the model eventually converges. Deep neural networks (DNNs), however, challenge this view: We show that removing regularization after an initial transient period has little effect on generalization, even if the final loss landscape is the same as if there had been no regularization. In some cases, generalization even improves after interrupting regularization. Conversely, if regularization is applied only after the initial transient, it has no effect on the final solution, whose generalization gap is as bad as if regularization never happened. This suggests that what matters for training deep networks is not just whether or how, but when to regularize. The phenomena we observe are manifest in different datasets (CIFAR-10, CIFAR-100), different architectures (ResNet-18, All-CNN), different regularization methods (weight decay, data augmentation), different learning rate schedules (exponential, piece-wise constant). They collectively suggest that there is a ``critical period'' for regularizing deep networks that is decisive of the final performance. More analysis should, therefore, focus on the transient rather than asymptotic behavior of learning.


title: A Stochastic Derivative Free Optimization Method with Momentum

abstract: We consider the problem of unconstrained minimization of a smooth objective function in $\mathbb{R}^d$ in setting where only function evaluations are possible. We propose and analyze stochastic zeroth-order method with heavy ball momentum. In particular, we propose, SMTP, a momentum version of the stochastic three-point method (STP) \cite{Bergou_2018}. We show new complexity results for non-convex, convex and strongly convex functions. We test our method on a collection of learning to continuous control tasks on several MuJoCo \cite{Todorov_2012} environments with varying difficulty and compare against STP, other state-of-the-art derivative-free optimization algorithms and against policy gradient methods. SMTP significantly outperforms STP and all other methods that we considered in our numerical experiments. Our second contribution is SMTP with importance sampling which we call SMTP_IS. We provide convergence analysis of this method for non-convex, convex and strongly convex objectives.


title: Non-invertible anomalies and mapping-class-group transformation of anomalous partition functions

abstract: Recently, it was realized that anomalies can be completely classified by topological orders, symmetry protected topological (SPT) orders, and symmetry enriched topological orders in one higher dimension. The anomalies that people used to study are invertible anomalies that correspond to invertible topological orders and/or symmetry protected topological orders in one higher dimension. In this paper, we introduce a notion of non-invertible anomaly, which describes the boundary of generic topological order. A key feature of non-invertible anomaly is that it has several partition functions. Under the mapping class group transformation of space-time, those partition functions transform in a certain way characterized by the data of the corresponding topological order in one higher dimension. In fact, the anomalous partition functions transform in the same way as the degenerate ground states of the corresponding topological order in one higher dimension. This general theory of non-invertible anomaly may have wide applications. As an example, we show that the irreducible gapless boundary of 2+1D double-semion (DS) topological order must have central charge $c=\bar c \geq \frac{25}{28}$.


title: Interpretation of the THz-THz-Raman Spectrum of Bromoform

abstract: Nonlinear THz-THz-Raman (TTR) liquid spectroscopy offers new possibilities for studying and understanding condensed-phase chemical dynamics. Although TTR spectra carry rich information about the systems under study, the response is encoded in a three-point correlation function comprising of both dipole and polarizability elements. Theoretical methods are necessary for the interpretation of the experimental results. In this work, we study the liquid-phase dynamics of bromoform, a polarizable molecule with a strong TTR response. Previous work based on reduced density matrix (RDM) simulations suggests that unusually large multi-quanta dipole matrix elements are needed to understand the measured spectrum of bromoform. Here, we demonstrate that a self-consistent definition of the time coordinates with respect to the reference pulse leads to a simplified experimental spectrum. Furthermore, we analytically derive a parametrization for the RDM model by integrating the dipole and polarizability elements to the 4th order in the normal modes, and we enforce inversion symmetry in the calculations by numerically cancelling the components of the response that are even with respect to the field. The resulting analysis eliminates the need to invoke large multi-quanta dipole matrix elements to fit the experimental spectrum; instead, the experimental spectrum is recovered using RDM simulations with dipole matrix parameters that are in agreement with independent ab initio calculations. The fundamental interpretation of the TTR signatures in terms of coupled intramolecular vibrational modes remains unchanged from the previous work.


title: Labor Market Outcomes and Early Schooling: Evidence from School Entry Policies Using Exact Date of Birth

abstract: We use a rich, census-like Brazilian dataset containing information on spatial mobility, schooling, and income in which we can link children to parents to assess the impact of early education on several labor market outcomes. Brazilian public primary schools admit children up to one year younger than the national minimum age to enter school if their birthday is before an arbitrary threshold, causing an exogenous variation in schooling at adulthood. Using a Regression Discontinuity Design, we estimate one additional year of schooling increases labor income in 25.8% - almost twice as large as estimated using mincerian models. Around this cutoff there is also a gap of 9.6% on the probability of holding a college degree in adulthood, with which we estimate the college premium and find a 201% increase in labor income. We test the robustness of our estimates using placebo variables, alternative model specifcations and McCrary Density Tests.


title: Two remarks on sums of squares with rational coefficients

abstract: There exist homogeneous polynomials $f$ with $\mathbb Q$-coefficients that are sums of squares over $\mathbb R$ but not over $\mathbb Q$. The only systematic construction of such polynomials that is known so far uses as its key ingredient totally imaginary number fields $K/\mathbb Q$ with specific Galois-theoretic properties. We first show that one may relax these properties considerably without losing the conclusion, and that this relaxation is sharp at least in a weak sense. In the second part we discuss the open question whether any $f$ as above necessarily has a (non-trivial) real zero. In the minimal open cases $(3,6)$ and $(4,4)$, we prove that all examples without a real zero are contained in a thin subset of the boundary of the sum of squares cone.


title: Sum-of-squares meets square loss: Fast rates for agnostic tensor completion

abstract: We study tensor completion in the agnostic setting. In the classical tensor completion problem, we receive $n$ entries of an unknown rank-$r$ tensor and wish to exactly complete the remaining entries. In agnostic tensor completion, we make no assumption on the rank of the unknown tensor, but attempt to predict unknown entries as well as the best rank-$r$ tensor. For agnostic learning of third-order tensors with the square loss, we give the first polynomial time algorithm that obtains a "fast" (i.e., $O(1/n)$-type) rate improving over the rate obtained by reduction to matrix completion. Our prediction error rate to compete with the best $d\times{}d\times{}d$ tensor of rank-$r$ is $\tilde{O}(r^{2}d^{3/2}/n)$. We also obtain an exact oracle inequality that trades off estimation and approximation error. Our algorithm is based on the degree-six sum-of-squares relaxation of the tensor nuclear norm. The key feature of our analysis is to show that a certain characterization for the subgradient of the tensor nuclear norm can be encoded in the sum-of-squares proof system. This unlocks the standard toolbox for localization of empirical processes under the square loss, and allows us to establish restricted eigenvalue-type guarantees for various tensor regression models, with tensor completion as a special case. The new analysis of the relaxation complements Barak and Moitra (2016), who gave slow rates for agnostic tensor completion, and Potechin and Steurer (2017), who gave exact recovery guarantees for the noiseless setting. Our techniques are user-friendly, and we anticipate that they will find use elsewhere.


title: Identifying Classes Susceptible to Adversarial Attacks

abstract: Despite numerous attempts to defend deep learning based image classifiers, they remain susceptible to the adversarial attacks. This paper proposes a technique to identify susceptible classes, those classes that are more easily subverted. To identify the susceptible classes we use distance-based measures and apply them on a trained model. Based on the distance among original classes, we create mapping among original classes and adversarial classes that helps to reduce the randomness of a model to a significant amount in an adversarial setting. We analyze the high dimensional geometry among the feature classes and identify the k most susceptible target classes in an adversarial attack. We conduct experiments using MNIST, Fashion MNIST, CIFAR-10 (ImageNet and ResNet-32) datasets. Finally, we evaluate our techniques in order to determine which distance-based measure works best and how the randomness of a model changes with perturbation.


title: Langevin Monte Carlo without smoothness

abstract: Langevin Monte Carlo (LMC) is an iterative algorithm used to generate samples from a distribution that is known only up to a normalizing constant. The nonasymptotic dependence of its mixing time on the dimension and target accuracy is understood mainly in the setting of smooth (gradient-Lipschitz) log-densities, a serious limitation for applications in machine learning. In this paper, we remove this limitation, providing polynomial-time convergence guarantees for a variant of LMC in the setting of nonsmooth log-concave distributions. At a high level, our results follow by leveraging the implicit smoothing of the log-density that comes from a small Gaussian perturbation that we add to the iterates of the algorithm and controlling the bias and variance that are induced by this perturbation.


title: On a maximal inequality and its application to SDEs with singular drift

abstract: In this paper we present a Doob type maximal inequality for stochastic processes satisfying the conditional increment control condition. If we assume, in addition, that the margins of the process have uniform exponential tail decay, we prove that the supremum of the process decays exponentially in the same manner. Then we apply this result to the construction of the almost everywhere stochastic flow to stochastic differential equations with singular time dependent divergence-free drift.


title: The Fourier-Mukai transform made easy

abstract: We propose a slightly modified definition for the Fourier-Mukai transform (on abelian varieties) that makes it much easier to remember various formulas. As an application, we give relatively short proofs for two important theorems: the characterization of GV-sheaves in terms of vanishing, due to Hacon; and fact that M-regularity implies (continuous) global generation, due to Pareschi and Popa.


title: Structured Output Learning with Conditional Generative Flows

abstract: Traditional structured prediction models try to learn the conditional likelihood, i.e., p(y|x), to capture the relationship between the structured output y and the input features x. For many models, computing the likelihood is intractable. These models are therefore hard to train, requiring the use of surrogate objectives or variational inference to approximate likelihood. In this paper, we propose conditional Glow (c-Glow), a conditional generative flow for structured output learning. C-Glow benefits from the ability of flow-based models to compute p(y|x) exactly and efficiently. Learning with c-Glow does not require a surrogate objective or performing inference during training. Once trained, we can directly and efficiently generate conditional samples. We develop a sample-based prediction method, which can use this advantage to do efficient and effective inference. In our experiments, we test c-Glow on five different tasks. C-Glow outperforms the state-of-the-art baselines in some tasks and predicts comparable outputs in the other tasks. The results show that c-Glow is versatile and is applicable to many different structured prediction problems.


title: On the Accuracy of Influence Functions for Measuring Group Effects

abstract: Influence functions estimate the effect of removing a training point on a model without the need to retrain. They are based on a first-order Taylor approximation that is guaranteed to be accurate for sufficiently small changes to the model, and so are commonly used to study the effect of individual points in large datasets. However, we often want to study the effects of large groups of training points, e.g., to diagnose batch effects or apportion credit between different data sources. Removing such large groups can result in significant changes to the model. Are influence functions still accurate in this setting? In this paper, we find that across many different types of groups and for a range of real-world datasets, the predicted effect (using influence functions) of a group correlates surprisingly well with its actual effect, even if the absolute and relative errors are large. Our theoretical analysis shows that such strong correlation arises only under certain settings and need not hold in general, indicating that real-world datasets have particular properties that allow the influence approximation to be accurate.


title: Seeing the Wind: Visual Wind Speed Prediction with a Coupled Convolutional and Recurrent Neural Network

abstract: Wind energy resource quantification, air pollution monitoring, and weather forecasting all rely on rapid, accurate measurement of local wind conditions. Visual observations of the effects of wind---the swaying of trees and flapping of flags, for example---encode information regarding local wind conditions that can potentially be leveraged for visual anemometry that is inexpensive and ubiquitous. Here, we demonstrate a coupled convolutional neural network and recurrent neural network architecture that extracts the wind speed encoded in visually recorded flow-structure interactions of a flag and tree in naturally occurring wind. Predictions for wind speeds ranging from 0.75-11 m/s showed agreement with measurements from a cup anemometer on site, with a root-mean-squared error approaching the natural wind speed variability due to atmospheric turbulence. Generalizability of the network was demonstrated by successful prediction of wind speed based on recordings of other flags in the field and in a controlled wind tunnel test. Furthermore, physics-based scaling of the flapping dynamics accurately predicts the dependence of the network performance on the video frame rate and duration.


title: Counting and Segmenting Sorghum Heads

abstract: Phenotyping is the process of measuring an organism's observable traits. Manual phenotyping of crops is a labor-intensive, time-consuming, costly, and error prone process. Accurate, automated, high-throughput phenotyping can relieve a huge burden in the crop breeding pipeline. In this paper, we propose a scalable, high-throughput approach to automatically count and segment panicles (heads), a key phenotype, from aerial sorghum crop imagery. Our counting approach uses the image density map obtained from dot or region annotation as the target with a novel deep convolutional neural network architecture. We also propose a novel instance segmentation algorithm using the estimated density map, to identify the individual panicles in the presence of occlusion. With real Sorghum aerial images, we obtain a mean absolute error (MAE) of 1.06 for counting which is better than using well-known crowd counting approaches such as CCNN, MCNN and CSRNet models. The instance segmentation model also produces respectable results which will be ultimately useful in reducing the manual annotation workload for future data.


title: Spanning Trees and Domination in Hypercubes

abstract: Let $L(G)$ denote the maximum number of leaves in any spanning tree of a connected graph $G$. We show the (known) result that for the $n$-cube $Q_n$, $L(Q_n) \sim 2^n = |V(Q_n)|$ as $n\rightarrow \infty$. Examining this more carefully, consider the minimum size of a connected dominating set of vertices $\gamma_c(Q_n)$, which is $2^n-L(Q_n)$ for $n\ge2$. We show that $\gamma_c(Q_n)\sim 2^n/n$. We use Hamming codes and an "expansion" method to construct leafy spanning trees in $Q_n$.


title: Optimal Timing of Moving Target Defense: A Stackelberg Game Model

abstract: As an effective approach to thwarting advanced attacks, moving target defense (MTD) has been applied to various domains. Previous works on MTD, however, mainly focus on deciding the sequence of system configurations to be used and have largely ignored the equally important timing problem. Given that both the migration cost and attack time vary over system configurations, it is crucial to jointly optimize the spatial and temporal decisions in MTD to better protect the system from persistent threats. In this work, we propose a Stackelberg game model for MTD where the defender commits to a joint migration and timing strategy to cope with configuration-dependent migration cost and attack time distribution. The defender's problem is formulated as a semi-Markovian decision process and a nearly optimal MTD strategy is derived by exploiting the unique structure of the game.


title: A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends

abstract: Deep learning has solved a problem that as little as five years ago was thought by many to be intractable - the automatic recognition of patterns in data; and it can do so with accuracy that often surpasses human beings. It has solved problems beyond the realm of traditional, hand-crafted machine learning algorithms and captured the imagination of practitioners trying to make sense out of the flood of data that now inundates our society. As public awareness of the efficacy of DL increases so does the desire to make use of it. But even for highly trained professionals it can be daunting to approach the rapidly increasing body of knowledge produced by experts in the field. Where does one start? How does one determine if a particular model is applicable to their problem? How does one train and deploy such a network? A primer on the subject can be a good place to start. With that in mind, we present an overview of some of the key multilayer ANNs that comprise DL. We also discuss some new automatic architecture optimization protocols that use multi-agent approaches. Further, since guaranteeing system uptime is becoming critical to many computer applications, we include a section on using neural networks for fault detection and subsequent mitigation. This is followed by an exploratory survey of several application areas where DL has emerged as a game-changing technology: anomalous behavior detection in financial applications or in financial time-series forecasting, predictive and prescriptive analytics, medical image processing and analysis and power systems research. The thrust of this review is to outline emerging areas of application-oriented research within the DL community as well as to provide a reference to researchers seeking to use it in their work for what it does best: statistical pattern recognition with unparalleled learning capacity with the ability to scale with information.


title: Extremal k-packings in compact non-orientable surfaces

abstract: An extremal $k$-packing is a collection of $k$ mutually disjoint metric discs, embedded in a surface, whose radius is maximal for the given topology. We study compact non-orientable surfaces of genus $g\ge 3$ containing extremal $k$-packings.


title: Stochastic Model Predictive Control for Constrained Linear Systems Using Optimal Covariance Steering

abstract: This work develops a stochastic model predictive controller~(SMPC) for uncertain linear systems with additive Gaussian noise subject to state and control constraints. The proposed approach is based on the recently developed finite-horizon optimal covariance steering control theory, which steers the mean and the covariance of the system state to prescribed target values at a given terminal time. We call our approach covariance steering-based SMPC, or CS-SMPC. We show that the proposed approach has several advantages over traditional SMPC approaches in the literature. Specifically, it is shown that the newly developed algorithm can deal with unbounded Gaussian additive noise while ensuring stability and recursive feasibility, and incurs lower computational cost than previous similar approaches. The effectiveness of the proposed CS-SMPC approach is confirmed using numerical simulations.


title: A brute force computer aided proof of an existence result about extremal hyperbolic surfaces

abstract: Extremal compact hyperbolic surfaces contain a packing of discs of the largest possible radius permitted by the topology of the surface. It is well known that arithmetic conditions on the uniformizing group are necessary for the existence of a second extremal packing in the same surface, but constructing explicit examples of this phenomenon is a complicated task. We present a brute force computational procedure that can be used to produce examples in all cases.


title: DeepShift: Towards Multiplication-Less Neural Networks

abstract: The high computation, memory, and power budgets of inferring convolutional neural networks (CNNs) are major bottlenecks of model deployment to edge computing platforms, e.g., mobile devices and IoT. Moreover, training CNNs is time and energy-intensive even on high-grade servers. Convolution layers and fully connected layers, because of their intense use of multiplications, are the dominant contributor to this computation budget. We propose to alleviate this problem by introducing two new operations: convolutional shifts and fully-connected shifts which replace multiplications with bitwise shift and sign flipping during both training and inference. During inference, both approaches require only 5 bits (or less) to represent the weights. This family of neural network architectures (that use convolutional shifts and fully connected shifts) is referred to as DeepShift models. We propose two methods to train DeepShift models: DeepShift-Q which trains regular weights constrained to powers of 2, and DeepShift-PS that trains the values of the shifts and sign flips directly. Very close accuracy, and in some cases higher accuracy, to baselines are achieved. Converting pre-trained 32-bit floating-point baseline models of ResNet18, ResNet50, VGG16, and GoogleNet to DeepShift and training them for 15 to 30 epochs, resulted in Top-1/Top-5 accuracies higher than that of the original model. Last but not least, we implemented the convolutional shifts and fully connected shift GPU kernels and showed a reduction in latency time of 25% when inferring ResNet18 compared to unoptimized multiplication-based GPU kernels. The code can be found at https://github.com/mostafaelhoushi/DeepShift.


title: Properties of mean dimension and metric mean dimension coming from the topological entropy

abstract: In the late 1990's, M. Gromov introduced the notion of mean dimension for a continuous map, which is, as well as the topological entropy, an invariant under topological conjugacy. The concept of metric mean dimension for a dynamical system was introduced by Lindenstrauss and Weiss in 2000. In this paper we will verify which properties coming from the topological entropy map are valid for both mean dimension and metric mean dimension. In particular, we will prove that the metric mean dimension map is not continuous anywhere on the set consisting of continuous maps on both the Cantor set, the interval or the circle. Finally we prove that the metric mean dimension on the set consisting of continuous map on the interval and on the circle is not lower semi-continuous.


title: Generative Imaging and Image Processing via Generative Encoder

abstract: This paper introduces a novel generative encoder (GE) model for generative imaging and image processing with applications in compressed sensing and imaging, image compression, denoising, inpainting, deblurring, and super-resolution. The GE model consists of a pre-training phase and a solving phase. In the pre-training phase, we separately train two deep neural networks: a generative adversarial network (GAN) with a generator $\G$ that captures the data distribution of a given image set, and an auto-encoder (AE) network with an encoder $\EN$ that compresses images following the estimated distribution by GAN. In the solving phase, given a noisy image $x=\mathcal{P}(x^*)$, where $x^*$ is the target unknown image, $\mathcal{P}$ is an operator adding an addictive, or multiplicative, or convolutional noise, or equivalently given such an image $x$ in the compressed domain, i.e., given $m=\EN(x)$, we solve the optimization problem \[ z^*=\underset{z}{\mathrm{argmin}} \|\EN(\G(z))-m\|_2^2+\lambda\|z\|_2^2 \] to recover the image $x^*$ in a generative way via $\hat{x}:=\G(z^*)\approx x^*$, where $\lambda>0$ is a hyperparameter. The GE model unifies the generative capacity of GANs and the stability of AEs in an optimization framework above instead of stacking GANs and AEs into a single network or combining their loss functions into one as in existing literature. Numerical experiments show that the proposed model outperforms several state-of-the-art algorithms.


title: Second-Order Multi-Reference Algebraic Diagrammatic Construction Theory for Photoelectron Spectra of Strongly Correlated Systems

abstract: We present a second-order formulation of multi-reference algebraic diagrammatic construction theory [Sokolov, A. Yu. J. Chem. Phys. 2018, 149, 204113] for simulating photoelectron spectra of strongly correlated systems (MR-ADC(2)). The MR-ADC(2) method uses second-order multi-reference perturbation theory (MRPT2) to efficiently obtain ionization energies and intensities for many photoelectron transitions in a single computation. In contrast to conventional MRPT2 methods, MR-ADC(2) provides information about ionization of electrons in all orbitals (i.e., core and active) and allows to compute transition intensities in straightforward and efficient way. Although equations of MR-ADC(2) depend on four-particle reduced density matrices, we demonstrate that computation of these large matrices can be completely avoided without introducing any approximations. The resulting MR-ADC(2) implementation has a lower computational scaling compared to conventional MRPT2 methods. We present results of MR-ADC(2) for photoelectron spectra of small molecules, carbon dimer, and equally-spaced hydrogen chains (H10 and H30) and outline directions for future developments.


title: A Survey on Biomedical Image Captioning

abstract: Image captioning applied to biomedical images can assist and accelerate the diagnosis process followed by clinicians. This article is the first survey of biomedical image captioning, discussing datasets, evaluation measures, and state of the art methods. Additionally, we suggest two baselines, a weak and a stronger one; the latter outperforms all current state of the art systems on one of the datasets.


title: Local theory of free noncommutative functions: germs, meromorphic functions and Hermite interpolation

abstract: Free analysis is a quantization of the usual function theory much like operator space theory is a quantization of classical functional analysis. Basic objects of free analysis are noncommutative functions. These are maps on tuples of matrices of all sizes that preserve direct sums and similarities. This paper investigates the local theory of noncommutative functions. The first main result shows that for a scalar point $Y$, the ring $O_Y$ of uniformly analytic noncommutative germs about $Y$ is an integral domain and admits a universal skew field of fractions, whose elements are called meromorphic germs. A corollary is a local-global rank principle that connects ranks of matrix evaluations of a matrix $A$ over $O_Y$ with the factorization of $A$ over $O_Y$. Different phenomena occur for a semisimple tuple of non-scalar matrices $Y$. Here it is shown that $O_Y$ contains copies of the matrix algebra generated by $Y$. In particular, there exist nonzero nilpotent uniformly analytic functions defined in a neighborhood of $Y$, and $O_Y$ does not embed into a skew field. Nevertheless, the ring $O_Y$ is described as the completion of a free algebra with respect to the vanishing ideal at $Y$. This is a consequence of the second main result, a free Hermite interpolation theorem: if $f$ is a noncommutative function, then for any finite set of semisimple points and a natural number $L$ there exists a noncommutative polynomial that agrees with $f$ at the chosen points up to differentials of order $L$. All the obtained results also have analogs for (non-uniformly) analytic germs and formal germs.


title: Simply connected Sasaki-Einstein rational homology 5-spheres

abstract: We completely determine which simply connected rational homology 5-spheres admit Sasaki-Einstein metrics.


title: Countering Noisy Labels By Learning From Auxiliary Clean Labels

abstract: We consider the learning from noisy labels (NL) problem which emerges in many real-world applications. In addition to the widely-studied synthetic noise in the NL literature, we also consider the pseudo labels in semi-supervised learning (Semi-SL) as a special case of NL. For both types of noise, we argue that the generalization performance of existing methods is highly coupled with the quality of noisy labels. Therefore, we counter the problem from a novel and unified perspective: learning from the auxiliary clean labels. Specifically, we propose the Rotational-Decoupling Consistency Regularization (RDCR) framework that integrates the consistency-based methods with the self-supervised rotation task to learn noise-tolerant representations. The experiments show that RDCR achieves comparable or superior performance than the state-of-the-art methods under small noise, while outperforms the existing methods significantly when there is large noise.


title: Implicit Background Estimation for Semantic Segmentation

abstract: Scene understanding and semantic segmentation are at the core of many computer vision tasks, many of which, involve interacting with humans in potentially dangerous ways. It is therefore paramount that techniques for principled design of robust models be developed. In this paper, we provide analytic and empirical evidence that correcting potentially errant non-distinct mappings that result from the softmax function can result in improving robustness characteristics on a state-of-the-art semantic segmentation model with minimal impact to performance and minimal changes to the code base.


title: Real-time Approximate Bayesian Computation for Scene Understanding

abstract: Consider scene understanding problems such as predicting where a person is probably reaching, or inferring the pose of 3D objects from depth images, or inferring the probable street crossings of pedestrians at a busy intersection. This paper shows how to solve these problems using Approximate Bayesian Computation. The underlying generative models are built from realistic simulation software, wrapped in a Bayesian error model for the gap between simulation outputs and real data. The simulators are drawn from off-the-shelf computer graphics, video game, and traffic simulation code. The paper introduces two techniques for speeding up inference that can be used separately or in combination. The first is to train neural surrogates of the simulators, using a simple form of domain randomization to make the surrogates more robust to the gap between the simulation and reality. The second is to adaptively discretize the latent variables using a Tree-pyramid approach adapted from computer graphics. This paper also shows performance and accuracy measurements on real-world problems, establishing that it is feasible to solve these problems in real-time.


title: Hangul Fonts Dataset: a Hierarchical and Compositional Dataset for Investigating Learned Representations

abstract: Hierarchy and compositionality are common latent properties in many natural and scientific datasets. Determining when a deep network's hidden activations represent hierarchy and compositionality is important both for understanding deep representation learning and for applying deep networks in domains where interpretability is crucial. However, current benchmark machine learning datasets either have little hierarchical or compositional structure, or the structure is not known. This gap impedes precise analysis of a network's representations and thus hinders development of new methods that can learn such properties. To address this gap, we developed a new benchmark dataset with known hierarchical and compositional structure. The Hangul Fonts Dataset (HFD) is comprised of 35 fonts from the Korean writing system (Hangul), each with 11,172 blocks (syllables) composed from the product of initial consonant, medial vowel, and final consonant glyphs. All blocks can be grouped into a few geometric types which induces a hierarchy across blocks. In addition, each block is composed of individual glyphs with rotations, translations, scalings, and naturalistic style variation across fonts. We find that both shallow and deep unsupervised methods only show modest evidence of hierarchy and compositionality in their representations of the HFD compared to supervised deep networks. Supervised deep network representations contain structure related to the geometrical hierarchy of the characters, but the compositional structure of the data is not evident. Thus, HFD enables the identification of shortcomings in existing methods, a critical first step toward developing new machine learning algorithms to extract hierarchical and compositional structure in the context of naturalistic variability.


title: Machine Learning Methods for Shark Detection

abstract: This essay reviews human observer-based methods employed in shark spotting in Muizenberg Beach. It investigates Machine Learning methods for automated shark detection with the aim of enhancing human observation. A questionnaire and interview were used to collect information about shark spotting, the motivation of the actual Shark Spotter program and its limitations. We have defined a list of desirable properties for our model and chosen the adequate mathematical techniques. The preliminary results of the research show that we can expect to extract useful information from shark images despite the geometric transformations that sharks perform, its features do not change. To conclude, we have partially implemented our model; the remaining implementation requires dataset.


title: A Simplicial Construction for Noncommutative Settings

abstract: In this paper we present a general construction that can be used to define the higher Hochschild homology for a noncommutative algebra. We also discuss other examples where this construction can be used.


title: Gradients of parameterized quantum gates using the parameter-shift rule and gate decomposition

abstract: The parameter-shift rule is an approach to measuring gradients of quantum circuits with respect to their parameters, which does not require ancilla qubits or controlled operations. Here, I discuss applying this approach to a wider range of parameterize quantum gates by decomposing gates into a product of standard gates, each of which is parameter-shift rule differentiable.


title: Convolutional Restricted Boltzmann Machine Based-Radiomics for Prediction of Pathological Complete Response to Neoadjuvant Chemotherapy in Breast Cancer

abstract: We proposed a novel convolutional restricted Boltzmann machine CRBM-based radiomic method for predicting pathologic complete response (pCR) to neoadjuvant chemotherapy treatment (NACT) in breast cancer. The method consists of extracting semantic features from CRBM network, and pCR prediction. It was evaluated on the dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data of 57 patients and using the area under the receiver operating characteristic curve (AUC). Traditional radiomics features and the semantic features learned from CRBM network were extracted from the images acquired before and after the administration of NACT. After the feature selection, the support vector machine (SVM), logistic regression (LR) and random forest (RF) were trained to predict the pCR status. Compared to traditional radiomic methods, the proposed CRBM-based radiomic method yielded an AUC of 0.92 for the prediction with the images acquired before and after NACT, and an AUC of 0.87 for the pretreatment prediction, which was increased by about 38%. The results showed that the CRBM-based radiomic method provided a potential means for accurately predicting the pCR to NACT in breast cancer before the treatment, which is very useful for making more appropriate and personalized treatment regimens.


title: Technical Report of the Video Event Reconstruction and Analysis (VERA) System -- Shooter Localization, Models, Interface, and Beyond

abstract: Every minute, hundreds of hours of video are uploaded to social media sites and the Internet from around the world. This material creates a visual record of the experiences of a significant percentage of humanity and can help illuminate how we live in the present moment. When properly analyzed, this video can also help analysts to reconstruct events of interest, including war crimes, human rights violations, and terrorist acts. Machine learning and computer vision can play a crucial role in this process. In this technical report, we describe the Video Event Reconstruction and Analysis (VERA) system. This new tool brings together a variety of capabilities we have developed over the past few years (including video synchronization and geolocation to order unstructured videos lacking metadata over time and space, and sound recognition algorithms) to enable the reconstruction and analysis of events captured on video. Among other uses, VERA enables the localization of a shooter from just a few videos that include the sound of gunshots. To demonstrate the efficacy of this suite of tools, we present the results of estimating the shooter's location of the Las Vegas Shooting in 2017 and show that VERA accurately predicts the shooter's location using only the first few gunshots. We then point out future directions that can help improve the system and further reduce unnecessary human labor in the process. All of the components of VERA run through a web interface that enables human-in-the-loop verification to ensure accurate estimations. All relevant source code, including the web interface and machine learning models, is freely available on Github. We hope that researchers and software developers will be inspired to improve and expand this system moving forward to better meet the needs of human rights and public safety.


title: Quantum-chemical studies of rutile nanoparticles toxicity I. Defect-free rod-like model clusters

abstract: Using semiempirical PM6 method, the structures of a rod-like [Ti40O124H81]-7 model cluster and of the [Ti40O124H81Cu]-5 ones with Cu2+ coordinated at various sites were optimized. If the relative toxicity of individual Ti centers in rod-like rutile nanoparticles can be evaluated by the electron density transfer to a Cu2+ probe, its maximal values may be ascribed to the pentacoordinated corner and hexacoordinated edge ones with three Ti-OH bonds. However, these centers exhibit the least negative interaction energies which can be compensated by the significantly better accessibility of the corner Ti center in comparison with the remaining ones. The Ti centers with the most negative interaction energy parameters exhibit the lowest extent of the electron density transfer to a Cu2+ probe. The rutile nanoparticles destruction starts at pentacoordinated Ti face centers


title: Graph Attention Memory for Visual Navigation

abstract: Visual navigation in complex environments is inefficient with traditional reactive policy or general-purposed recurrent policy. To address the long-term memory issue, this paper proposes a graph attention memory (GAM) architecture consisting of memory construction module, graph attention module and control module. The memory construction module builds the topological graph based on supervised learning by taking the exploration prior. Then, guided attention features are extracted with the graph attention module. Finally, the deep reinforcement learning based control module makes decisions based on visual observations and guided attention features. Detailed convergence analysis of GAM is presented in this paper. We evaluate GAM-based navigation system in two complex 3D environments. Experimental results show that the GAM-based navigation system significantly improves learning efficiency and outperforms all baselines in average success rate.


title: Singlet-Triplet Energy Gaps of Organic Biradicals and Polyacenes with Auxiliary-Field Quantum Monte Carlo

abstract: The energy gap between the lowest-lying singlet and triplet states is an important quantity in chemical photocatalysis, with relevant applications ranging from triplet fusion in optical upconversion to the design of organic light-emitting devices. The ab initio prediction of singlet-triplet (ST) gaps is challenging due to the potentially biradical nature of the involved states, combined with the potentially large size of relevant molecules. In this work, we show that phaseless auxiliary-field quantum Monte Carlo (ph-AFQMC) can accurately predict ST gaps for chemical systems with singlet states of highly biradical nature, including a set of 13 small molecules and the ortho-, meta-, and para-isomers of benzyne. With respect to gas-phase experiments, ph-AFQMC using CASSCF trial wavefunctions achieves a mean averaged error of ~1 kcal/mol. Furthermore, we find that in the context of a spin-projection technique, ph-AFQMC using unrestricted single-determinant trial wavefunctions, which can be readily obtained for even very large systems, produces equivalently high accuracy. We proceed to show that this scalable methodology is capable of yielding accurate ST gaps for all linear polyacenes for which experimental measurements exist, i.e. naphthalene, anthracene, tetracene, and pentacene. Our results suggest a protocol for selecting either unrestricted Hartree-Fock or Kohn-Sham orbitals for the single-determinant trial wavefunction, based on the extent of spin-contamination. These findings pave the way for future investigations of specific photochemical processes involving large molecules with substantial biradical character.


title: Talagrand's inequality in planar Gaussian field percolation

abstract: Let f be a stationary isotropic non-degenerate Gaussian field on R^2. Assume that f = q * W where q is both C^2 and L^2 and W is the L^2 white noise on R^2. We extend a result by Stephen Muirhead and Hugo Vanneuville by showing that, assuming that q * q is pointwise non-negative and has fast enough decay, the set {f > -l} percolates with probability one when l > 0 and with probability zero if l < 0 or l = 0. We also prove exponential decay of crossing probabilities and uniqueness of the unbounded cluster. To this end, we study a Gaussian field g defined on the torus and establish a superconcentration formula for the threshold T(g) which is the minimal value such that {g > -T(g)} contains a non-contractible loop. This formula follows from a Gaussian Talagrand type inequality.


title: Baryogenesis from axion inflation

abstract: The coupling of an axion-like particle driving inflation to the Standard Model particle content through a Chern-Simons term generically sources a dual production of massless helical gauge fields and chiral fermions. We demonstrate that the interplay of these two components results in a highly predictive baryogenesis model, which requires no further ingredients beyond the Standard Model. If the helicity stored in the hyper magnetic field and the effective chemical potential induced by the chiral fermion production are large enough to avoid magnetic diffusion from the thermal plasma but small enough to sufficiently delay the chiral plasma instability, then the non-vanishing helicity survives until the electroweak phase transition and sources a net baryon asymmetry which is in excellent agreement with the observed value. If any of these two conditions is violated, the final baryon asymmetry vanishes. The observed baryon asymmetry can be reproduced if the energy scale of inflation is around $H_\text{inf} \sim 10^{10}$ - $10^{12}$ GeV with a moderate dependence on inflation model parameters.


title: MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms

abstract: We introduce a large-scale dataset of math word problems and an interpretable neural math problem solver that learns to map problems to operation programs. Due to annotation challenges, current datasets in this domain have been either relatively small in scale or did not offer precise operational annotations over diverse problem types. We introduce a new representation language to model precise operation programs corresponding to each math problem that aim to improve both the performance and the interpretability of the learned models. Using this representation language, our new dataset, MathQA, significantly enhances the AQuA dataset with fully-specified operational programs. We additionally introduce a neural sequence-to-program model enhanced with automatic problem categorization. Our experiments show improvements over competitive baselines in our MathQA as well as the AQuA dataset. The results are still significantly lower than human performance indicating that the dataset poses new challenges for future research. Our dataset is available at: https://math-qa.github.io/math-QA/


title: Combating the Compounding-Error Problem with a Multi-step Model

abstract: Model-based reinforcement learning is an appealing framework for creating agents that learn, plan, and act in sequential environments. Model-based algorithms typically involve learning a transition model that takes a state and an action and outputs the next state---a one-step model. This model can be composed with itself to enable predicting multiple steps into the future, but one-step prediction errors can get magnified, leading to unacceptable inaccuracy. This compounding-error problem plagues planning and undermines model-based reinforcement learning. In this paper, we address the compounding-error problem by introducing a multi-step model that directly outputs the outcome of executing a sequence of actions. Novel theoretical and empirical results indicate that the multi-step model is more conducive to efficient value-function estimation, and it yields better action selection compared to the one-step model. These results make a strong case for using multi-step models in the context of model-based reinforcement learning.


title: Applying Generative Adversarial Networks to Intelligent Subsurface Imaging and Identification

abstract: To augment training data for machine learning models in Ground Penetrating Radar (GPR) data classification and identification, this thesis focuses on the generation of realistic GPR data using Generative Adversarial Networks. An innovative GAN architecture is proposed for generating GPR B-scans, which is, to the author's knowledge, the first successful application of GAN to GPR B-scans. As one of the major contributions, a novel loss function is formulated by merging frequency domain with time domain features. To test the efficacy of generated B-scans, a real time object classifier is proposed to measure the performance gain derived from augmented B-Scan images. The numerical experiment illustrated that, based on the augmented training data, the proposed GAN architecture demonstrated a significant increase (from 82% to 98%) in the accuracy of the object classifier.


title: Assessing The Factual Accuracy of Generated Text

abstract: We propose a model-based metric to estimate the factual accuracy of generated text that is complementary to typical scoring schemes like ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BLEU (Bilingual Evaluation Understudy). We introduce and release a new large-scale dataset based on Wikipedia and Wikidata to train relation classifiers and end-to-end fact extraction models. The end-to-end models are shown to be able to extract complete sets of facts from datasets with full pages of text. We then analyse multiple models that estimate factual accuracy on a Wikipedia text summarization task, and show their efficacy compared to ROUGE and other model-free variants by conducting a human evaluation study.


title: Ultracentral Collisions of Small and Deformed Systems at RHIC: ${U} {U}$, ${d} {Au}$, ${}^{9}{Be} {Au}$, ${}^{9}{Be} {}^{9}{Be}$, ${}^{3}{He} {}^{3}{He}$, and ${}^{3}{He} {Au}$ Collisions

abstract: In this paper, we study a range of collision systems involving deformed ions and compare the elliptic and triangular flow harmonics produced in a hydrodynamics scenario versus a color glass condensate (CGC) scenario. For the hydrodynamics scenario, we generate initial conditions using TRENTO and work within a linear response approximation to obtain the final flow harmonics. For the CGC scenario, we use the explicit calculation of two-gluon correlations taken in the high-$p_T$ ``(semi)dilute-(semi)dilute'' regime to express the flow harmonics in terms of the density profile of the collision. We consider ultracentral collisions of deformed ions as a testbed for these comparisons because the difference between tip-on-tip and side-on-side collisions modifies the multiplicity dependence in both scenarios, even at zero impact parameter. We find significant qualitative differences in the multiplicity dependence obtained in the initial conditions+hydrodynamics scenario and the CGC scenario, allowing these collisions of deformed ions to be used as a powerful discriminator between models. We also find that sub-nucleonic fluctuations have a systematic effect on the elliptic and triangular flow harmonics which are most discriminating in $0-1\%$ ultracentral symmetric collisions of small deformed ions and in $0-10\%$ $\mathrm{d} {}^{197}\mathrm{Au}$ collisions. The collision systems we consider are ${}^{238}\mathrm{U} {}^{238}\mathrm{U}$, $\mathrm{d} {}^{197}\mathrm{Au}$, ${}^{9}\mathrm{Be} {}^{197}\mathrm{Au}$, ${}^{9}\mathrm{Be} {}^{9}\mathrm{Be}$, ${}^{3}\mathrm{He} {}^{3}\mathrm{He}$, and ${}^{3}\mathrm{He} {}^{197}\mathrm{Au}$.


title: A Lightweight Recurrent Network for Sequence Modeling

abstract: Recurrent networks have achieved great success on various sequential tasks with the assistance of complex recurrent units, but suffer from severe computational inefficiency due to weak parallelization. One direction to alleviate this issue is to shift heavy computations outside the recurrence. In this paper, we propose a lightweight recurrent network, or LRN. LRN uses input and forget gates to handle long-range dependencies as well as gradient vanishing and explosion, with all parameter related calculations factored outside the recurrence. The recurrence in LRN only manipulates the weight assigned to each token, tightly connecting LRN with self-attention networks. We apply LRN as a drop-in replacement of existing recurrent units in several neural sequential models. Extensive experiments on six NLP tasks show that LRN yields the best running efficiency with little or no loss in model performance.


title: Diffusion Monte Carlo Study of the O$_2$ Adsorption on a Single Layer Graphene

abstract: Diffusion Monte Carlo (DMC) calculations were performed for an accurate description of the nature of the O$_2$ adsorption on a single layer graphene. We investigated the stable orientation of O$_2$ at a specific adsorption site as well as its equilibrium adsorption energy. At equilibrium adsorption distances, an O$_2$ molecule was found to prefer a horizontal orientation, where the O-O bond is parallel to the graphene surface, to the vertical orientation. However, the vertical orientation is favored at the O$_2$-graphene distances shorter than the equilibrium distance, which could be understood by the steric repulsion between O and C atoms. Contrary to previous DFT calculations, our DMC calculations show that the midpoint of a C-C bond (a bridge site) is energetically preferred for the O$_2$ adsorption to a center of a hexagonal ring (a hollow site). The lowest DMC adsorption energy was found at an intermediate point between a hollow and a bridge site, where the O$_2$ adsorption energy was estimated to be -0.142(4) eV that was in very good agreement with the recently-reported experimental value. Finally, we have found that O$_2$ is very diffusive on the surface of graphene with the diffusion barrier along a bridge-hollow-bridge path being as small as ~ 11 meV.


title: Grammar-based Neural Text-to-SQL Generation

abstract: The sequence-to-sequence paradigm employed by neural text-to-SQL models typically performs token-level decoding and does not consider generating SQL hierarchically from a grammar. Grammar-based decoding has shown significant improvements for other semantic parsing tasks, but SQL and other general programming languages have complexities not present in logical formalisms that make writing hierarchical grammars difficult. We introduce techniques to handle these complexities, showing how to construct a schema-dependent grammar with minimal over-generation. We analyze these techniques on ATIS and Spider, two challenging text-to-SQL datasets, demonstrating that they yield 14--18\% relative reductions in error.


title: Measurement of local optomechanical properties of a direct bandgap 2D semiconductor

abstract: Strain engineering is a powerful tool for tuning physical properties of 2D materials, including monolayer transition metal dichalcogenides (TMD) -- direct bandgap semiconductors with strong excitonic response. Here, we demonstrate an approach for local characterization of strain-induced modification of excitonic photoluminescence in TMD-based materials. We reversibly stress a monolayer of MoSe$_2$ with an AFM tip and perform spatio-spectral mapping of the excitonic photoluminescence in the vicinity of the indentation point. To fully reproduce the experimental data, we introduce the linear dependence of the exciton energy and corresponding photoluminescence intensity on the induced strain. Careful account for the optical resolution allows extracting these quantities with good agreement with the previous measurements, which involved macroscopic sample deformation. Our approach is a powerful tool for the study of local optomechanical properties of 2D direct bandgap semiconductors with strong excitonic response.


title: Analysis of cell size effects in atomistic crack propagation

abstract: We consider crack propagation in a crystalline material in terms of bifurcation analysis. We provide evidence that the stress intensity factor is a natural bifurcation parameter, and that the resulting bifurcation diagram is a periodic "snaking curve". We then prove qualitative properties of the equilibria and convergence rates of finite-cell approximations to the "exact" bifurcation diagram.


title: Local estimates on two linear parabolic equations with singular coefficients

abstract: We treat the heat equation with singular drift terms and its generalization: the linearized Navier-Stokes system. In the first case, we obtain boundedness of weak solutions for highly singular, "supercritical" data. In the second case, we obtain regularity result for weak solutions with mildly singular data ( those in the Kato class). This not only extends some of the classical regularity theory in [AS], [CrZ] and others from the case of elliptic and heat equations to that of linearized Navier-Stokes equations but also proves an unexpected gradient estimate, which extends the recent interesting boundedness result [O]. In the addendum in May 2019, a missing term in Theorem 1.7 and Lemma 3.3 is added. This is due to the use of a formula in a cited reference, which omitted a term. The main conclusion that local solutions of certain linearized Navier-Stokes equation have bounded spatial gradient is intact. This includes bounded local Leray-Hopf solutions to the Navier Stokes equation without condition on the pressure.


title: Constraints on Chameleon f(R)-Gravity from Galaxy Rotation Curves of the SPARC Sample

abstract: In chameleon $f(R)$-gravity, the fifth force will lead to `upturns' in galaxy rotation curves near the screening radius. The location of the upturn depends on the cosmic background value of the scalar field $f_{R0}$, as well as the mass, size and environment of the galaxy. We search for this signature of modified gravity in the SPARC sample of measured rotation curves, using an MCMC technique to derive constraints on $f_{R0}$. Assuming NFW dark matter haloes and with $f_{R0}$ freely varying for each galaxy, most galaxies prefer $f(R)$ gravity to $\Lambda$CDM, but there is a large spread of inferred $f_{R0}$ values, inconsistent with a single global value. Requiring instead a consistent $f_{R0}$ value for the whole sample, models with $\log_{10}|f_{R0}| > -6.1$ are excluded. On the other hand, models in the range $-7.5<\log_{10}|f_{R0}|<-6.5$ seem to be favoured with respect to $\Lambda$CDM, with a significant peak at -7. However, this signal is largely a result of galaxies for which the $f(R)$ signal is degenerate with the core/cusp problem, and when the NFW profile is replaced with a cored halo profile, $\Lambda$CDM gives better fits than any given $f(R)$ model. Thus, we find no convincing evidence of $f(R)$ gravity down to the level of $|f_{R0}| \sim 6 \times 10^{-8}$, with the caveat that if cored halo density profiles cannot ultimately be explained within $\Lambda$CDM, a screened modified gravity theory could possibly provide an alternative solution for the core/cusp problem. However, the $f(R)$ models studied here fall short of achieving this.


title: Discriminative Clustering for Robust Unsupervised Domain Adaptation

abstract: Unsupervised domain adaptation seeks to learn an invariant and discriminative representation for an unlabeled target domain by leveraging the information of a labeled source dataset. We propose to improve the discriminative ability of the target domain representation by simultaneously learning tightly clustered target representations while encouraging that each cluster is assigned to a unique and different class from the source. This strategy alleviates the effects of negative transfer when combined with adversarial domain matching between source and target representations. Our approach is robust to differences in the source and target label distributions and thus applicable to both balanced and imbalanced domain adaptation tasks, and with a simple extension, it can also be used for partial domain adaptation. Experiments on several benchmark datasets for domain adaptation demonstrate that our approach can achieve state-of-the-art performance in all three scenarios, namely, balanced, imbalanced and partial domain adaptation.


title: Identifying Cache-Based Side Channels through Secret-Augmented Abstract Interpretation

abstract: Cache-based side channels enable a dedicated attacker to reveal program secrets by measuring the cache access patterns. Practical attacks have been shown against real-world crypto algorithm implementations such as RSA, AES, and ElGamal. By far, identifying information leaks due to cache-based side channels, either in a static or dynamic manner, remains a challenge: the existing approaches fail to offer high precision, full coverage, and good scalability simultaneously, thus impeding their practical use in real-world scenarios. In this paper, we propose a novel static analysis method on binaries to detect cache-based side channels. We use abstract interpretation to reason on program states with respect to abstract values at each program point. To make such abstract interpretation scalable to real-world cryptosystems while offering high precision and full coverage, we propose a novel abstract domain called the Secret-Augmented Symbolic domain (SAS). SAS tracks program secrets and dependencies on them for precision, while it tracks only coarse-grained public information for scalability. We have implemented the proposed technique into a practical tool named CacheS and evaluated it on the implementations of widely-used cryptographic algorithms in real-world crypto libraries, including Libgcrypt, OpenSSL, and mbedTLS. CacheS successfully confirmed a total of 154 information leaks reported by previous research and 54 leaks that were previously unknown. We have reported our findings to the developers. And they confirmed that many of those unknown information leaks do lead to potential side channels.


title: Optimal basis for the generalized Dicke model

abstract: A methodology is devised for building optimal bases for the generalized Dicke model based on the symmetry adapted variational solution to the problem. At order zero, the matter sector is constructed by distributing $N_a$ particles in all the possible two-level subsystems connected with electromagnetic radiation; the next order is obtained when the states of $N_a-1$ particles are added and distributed again into the two-level subsystems; and so on. In the electromagnetic sector, the order zero for each mode is the direct sum of the Fock spaces, truncated to a value of the corresponding constants of motion of each two-level subsystem; by including contributions of the other modes, the next orders are obtained. As an example of the procedure we consider $4$ atoms in the $\Xi$ configuration interacting dipolarly with two modes of electromagnetic radiation. The results may be applied to situations in quantum optics, quantum information, and quantum computing.


title: How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset

abstract: Program repair is an important but difficult software engineering problem. One way to achieve acceptable performance is to focus on classes of simple bugs, such as bugs with single statement fixes, or that match a small set of bug templates. However, it is very difficult to estimate the recall of repair techniques for simple bugs, as there are no datasets about how often the associated bugs occur in code. To fill this gap, we provide a dataset of 153,652 single statement bug-fix changes mined from 1,000 popular open-source Java projects, annotated by whether they match any of a set of 16 bug templates, inspired by state-of-the-art program repair techniques. In an initial analysis, we find that about 33% of the simple bug fixes match the templates, indicating that a remarkable number of single-statement bugs can be repaired with a relatively small set of templates. Further, we find that template fitting bugs appear with a frequency of about one bug per 1,600-2,500 lines of code (as measured by the size of the project's latest version). We hope that the dataset will prove a resource for both future work in program repair and studies in empirical software engineering.


title: Exact solution of multi-angle quantum many-body collective neutrino flavor oscillations

abstract: I study the flavor evolution of a dense neutrino gas by considering vacuum contributions, matter effects and neutrino self-interactions. Assuming a system of two flavors in a uniform matter background, the time evolution of the many-body system in discretized momentum space is computed. The multi-angle neutrino-neutrino interactions are treated exactly and compared to both the single-angle approximation and mean field calculations. %The time unit chosen is $\mu_0^{-1}=(\frac{G_F}{2\sqrt{2}V})^{-1}$. The mono-energetic two neutrino beam scenario is solved analytically. I proceed to solve flavor oscillations for mono-energetic cubic lattices and quadratic lattices of two energy levels. In addition I study various configurations of twelve, sixteen, and twenty neutrinos. I find that when all neutrinos are initially of the same flavor, all methods agree. When both flavors are present, I find collective oscillations and flavor equilibration develop in the many body treatment but not in the mean field method. This difference persists in dense matter with tiny mixing angle and it can be ascribed to non-negligible flavor polarization correlations being present. Entanglement entropy is significant in all such cases. The relevance for supernovae or neutron stars mergers is contingent upon the value of the normalization volume $V$ and the large $N$ dependence of the timescale associated with oscillations. In future work, I intend to study this dependence using larger lattices and also include anti-neutrinos.


title: Functional limit theorems for the Polya and q-Polya urns

abstract: For the plain Polya urn with two colors, black and white, we prove a functional central limit theorem for the number of white balls assuming that the initial number of black balls is large. Depending on the initial number of white balls, the limit is either a pure birth process or a diffusion. We also prove analogous results for the q-Polya urn, which is an urn where, when picking a ball, the balls of one color have priority over those of the other.


title: Spin and charge correlations across the metal-to-insulator crossover in the half-filled $2d$ Hubbard model

abstract: The $2d$ Hubbard model with nearest-neighbour hopping on the square lattice and an average of one electron per site is known to undergo an extended crossover from metallic to insulating behavior driven by proliferating antiferromagnetic correlations. We study signatures of this crossover in spin and charge correlation functions and present results obtained with controlled accuracy using diagrammatic Monte Carlo in the range of parameters amenable to experimental verification with ultracold atoms in optical lattices. The qualitative changes in charge and spin correlations associated with the crossover are observed at well-separated temperature scales, which encase the intermediary regime of non-Fermi-liquid character, where local magnetic moments are formed and non-local fluctuations in both channels are essential.


title: The MOSDEF survey: a census of AGN-driven ionized outflows at $z = 1.4-3.8$

abstract: Using data from the MOSFIRE Deep Evolution Field (MOSDEF) survey, we present a census of AGN-driven ionized outflows in a sample of 159 AGNs at $1.4 \le z \le 3.8$. The sample spans AGN bolometric luminosities of $10^{44-47} \mathrm{~erg~s}^{-1}$ and includes both quiescent and star-forming galaxies extending across three orders of magnitude in stellar mass. We identify and characterize outflows from the \hbeta, [OIII], \halpha ~and [NII] emission line spectra. We detect outflows in $17\%$ of the AGNs, seven times more often than in a mass-matched sample of inactive galaxies in MOSDEF. The outflows are fast and galaxy-wide, with velocities of $\sim 400-3500 ~\mathrm{km~s}^{-1}$ and spatial extents of $0.3-11.0$ kpc. The incidence of outflows among AGNs is independent of the stellar mass of the host galaxy, with outflows detected in both star-forming and quiescent galaxies. This suggests that outflows exist across different phases in galaxy evolution. We investigate relations between outflow kinematic, spatial, and energetic properties and both AGN and host galaxy properties. Our results show that AGN-driven outflows are widespread in galaxies along the star-forming main sequence. The mass-loading factors of the outflows are typically $0.1-1$ and increase with AGN luminosity, capable of exceeding unity at $L_\mathrm{AGN} \gtrsim 10^{46.3} \mathrm{~erg~s}^{-1}$. In these more luminous sources the ionized outflow alone is likely sufficient to regulate star formation, and when combined with outflowing neutral and molecular gas may be able to quench star formation in their host galaxies.


title: Multitask Text-to-Visual Embedding with Titles and Clickthrough Data

abstract: Text-visual (or called semantic-visual) embedding is a central problem in vision-language research. It typically involves mapping of an image and a text description to a common feature space through a CNN image encoder and a RNN language encoder. In this paper, we propose a new method for learning text-visual embedding using both image titles and click-through data from an image search engine. We also propose a new triplet loss function by modeling positive awareness of the embedding, and introduce a novel mini-batch-based hard negative sampling approach for better data efficiency in the learning process. Experimental results show that our proposed method outperforms existing methods, and is also effective for real-world text-to-visual retrieval.


title: Log-logarithmic Time Pruned Polar Coding

abstract: A pruned variant of polar coding is proposed for binary erasure channels. For sufficiently small $\varepsilon>0$, we construct a series of capacity achieving codes with block length $N=\varepsilon^{-5}$, code rate $R=\text{Capacity}-\varepsilon$, error probability $P=\varepsilon$, and encoding and decoding time complexity $\text{bC}=O(\log\left|\log\varepsilon\right|)$ per information bit. The given per-bit complexity $\text{bC}$ is log-logarithmic in $N$, in $\text{Capacity}-R$, and in $P$; no known family of codes possesses this property. It is also the second lowest $\text{bC}$ after repeat-accumulate codes and their variants. While random codes and classical polar codes are the only two families of capacity-achieving codes whose $N$, $R$, $P$, and $\text{bC}$ were written down as explicit functions, our construction gives the third family. Then we generalize the result to: Fix a prime $q$ and fix a $q$-ary-input discrete symmetric memoryless channel. For sufficiently small $\varepsilon>0$, we construct a series of capacity achieving codes with block length $N=\varepsilon^{-O(1)}$, code rate $R=\text{Capacity}-\varepsilon$, error probability $P=\varepsilon$, and encoding and decoding time complexity $\text{bC}=O(\log\left|\log\varepsilon\right|)$ per information bit. The later construction gives the fastest family of capacity-achieving codes to date on those channels.


title: On Value Functions and the Agent-Environment Boundary

abstract: When function approximation is deployed in reinforcement learning (RL), the same problem may be formulated in different ways, often by treating a pre-processing step as a part of the environment or as part of the agent. As a consequence, fundamental concepts in RL, such as (optimal) value functions, are not uniquely defined as they depend on where we draw this agent-environment boundary, causing problems in theoretical analyses that provide optimality guarantees. We address this issue via a simple and novel boundary-invariant analysis of Fitted Q-Iteration, a representative RL algorithm, where the assumptions and the guarantees are invariant to the choice of boundary. We also discuss closely related issues on state resetting and Monte-Carlo Tree Search, deterministic vs stochastic systems, imitation learning, and the verifiability of theoretical assumptions from data.


title: All-In-One Underwater Image Enhancement using Domain-Adversarial Learning

abstract: Raw underwater images are degraded due to wavelength dependent light attenuation and scattering, limiting their applicability in vision systems. Another factor that makes enhancing underwater images particularly challenging is the diversity of the water types in which they are captured. For example, images captured in deep oceanic waters have a different distribution from those captured in shallow coastal waters. Such diversity makes it hard to train a single model to enhance underwater images. In this work, we propose a novel model which nicely handles the diversity of water during the enhancement, by adversarially learning the content features of the images by disentangling the unwanted nuisances corresponding to water types (viewed as different domains). We use the learned domain agnostic features to generate enhanced underwater images. We train our model on a dataset consisting images of 10 Jerlov water types. Experimental results show that the proposed model not only outperforms the previous methods in SSIM and PSNR scores for almost all Jerlov water types but also generalizes well on real-world datasets. The performance of a high-level vision task (object detection) also shows improvement using enhanced images with our model.


title: All SMILES Variational Autoencoder

abstract: Variational autoencoders (VAEs) defined over SMILES string and graph-based representations of molecules promise to improve the optimization of molecular properties, thereby revolutionizing the pharmaceuticals and materials industries. However, these VAEs are hindered by the non-unique nature of SMILES strings and the computational cost of graph convolutions. To efficiently pass messages along all paths through the molecular graph, we encode multiple SMILES strings of a single molecule using a set of stacked recurrent neural networks, pooling hidden representations of each atom between SMILES representations, and use attentional pooling to build a final fixed-length latent representation. By then decoding to a disjoint set of SMILES strings of the molecule, our All SMILES VAE learns an almost bijective mapping between molecules and latent representations near the high-probability-mass subspace of the prior. Our SMILES-derived but molecule-based latent representations significantly surpass the state-of-the-art in a variety of fully- and semi-supervised property regression and molecular property optimization tasks.


title: Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience

abstract: The ability of overparameterized deep networks to generalize well has been linked to the fact that stochastic gradient descent (SGD) finds solutions that lie in flat, wide minima in the training loss -- minima where the output of the network is resilient to small random noise added to its parameters. So far this observation has been used to provide generalization guarantees only for neural networks whose parameters are either \textit{stochastic} or \textit{compressed}. In this work, we present a general PAC-Bayesian framework that leverages this observation to provide a bound on the original network learned -- a network that is deterministic and uncompressed. What enables us to do this is a key novelty in our approach: our framework allows us to show that if on training data, the interactions between the weight matrices satisfy certain conditions that imply a wide training loss minimum, these conditions themselves {\em generalize} to the interactions between the matrices on test data, thereby implying a wide test loss minimum. We then apply our general framework in a setup where we assume that the pre-activation values of the network are not too small (although we assume this only on the training data). In this setup, we provide a generalization guarantee for the original (deterministic, uncompressed) network, that does not scale with product of the spectral norms of the weight matrices -- a guarantee that would not have been possible with prior approaches.


title: Power Weighted Shortest Paths for Clustering Euclidean Data

abstract: We study the use of power weighted shortest path distance functions for clustering high dimensional Euclidean data, under the assumption that the data is drawn from a collection of disjoint low dimensional manifolds. We argue, theoretically and experimentally, that this leads to higher clustering accuracy. We also present a fast algorithm for computing these distances.


title: A Density Functional Extension to Excited State Mean-Field Theory

abstract: We investigate an extension of excited state mean-field theory in which the energy expression is augmented with density functional components in an effort to include the effects of weak electron correlations. The approach remains variational and entirely time-independent, allowing it to avoid some of the difficulties associated with linear response and the adiabatic approximation. In particular, all of the electrons' orbitals are relaxed state specifically and there is no reliance on Kohn-Sham orbital energy differences, both of which are important features in the context of charge transfer. Preliminary testing shows clear advantages for single-component charge transfer states, but the method, at least in its current form, is less reliable for states in which multiple particle-hole transitions contribute significantly.


title: On Poincar\'e lemma or Volterra theorem about differential forms and cohomology groups

abstract: The Poincar\'{e} lemma (or Volterra theorem) is of utmost importance both in theory and in practice. It tells us every differential form which is closed, is locally exact. In other words, on a contractible manifold all closed forms are exact. The aim of this paper is to present some direct proofs of this lemma and explore some of its numerous consequences. Some connections with Cech-De Rham-Dolbeault cohomologies, $\overline{\partial}$-Poincar\'{e} lemma or Dolbeault-Grothendieck lemma are given.


title: INFaaS: A Model-less and Managed Inference Serving System

abstract: Despite existing work in machine learning inference serving, ease-of-use and cost efficiency remain challenges at large scales. Developers must manually search through thousands of model-variants -- versions of already-trained models that differ in hardware, resource footprints, latencies, costs, and accuracies -- to meet the diverse application requirements. Since requirements, query load, and applications themselves evolve over time, these decisions need to be made dynamically for each inference query to avoid excessive costs through naive autoscaling. To avoid navigating through the large and complex trade-off space of model-variants, developers often fix a variant across queries, and replicate it when load increases. However, given the diversity across variants and hardware platforms in the cloud, a lack of understanding of the trade-off space can incur significant costs to developers. This paper introduces INFaaS, a managed and model-less system for distributed inference serving, where developers simply specify the performance and accuracy requirements for their applications without needing to specify a specific model-variant for each query. INFaaS generates model-variants, and efficiently navigates the large trade-off space of model-variants on behalf of developers to meet application-specific objectives: (a) for each query, it selects a model, hardware architecture, and model optimizations, (b) it combines VM-level horizontal autoscaling with model-level autoscaling, where multiple, different model-variants are used to serve queries within each machine. By leveraging diverse variants and sharing hardware resources across models, INFaaS achieves 1.3x higher throughput, violates latency objectives 1.6x less often, and saves up to 21.6x in cost (8.5x on average) compared to state-of-the-art inference serving systems on AWS EC2.


title: Real zeros of random trigonometric polynomials with pairwise equal blocks of coefficients

abstract: It is well known that the expected number of real zeros of a random cosine polynomial $ V_n(x) = \sum_ {j=0} ^{n} a_j \cos (j x) , \ x \in (0,2\pi) $, with the $ a_j $ being standard Gaussian i.i.d. random variables is asymptotically $ 2n / \sqrt{3} $. On the other hand, some of the previous works on the random cosine polynomials with dependent coefficients show that such polynomials have at least $ 2n / \sqrt{3} $ expected real zeros lying in one period. In this paper we investigate two classes of random cosine polynomials with pairwise equal blocks of coefficients. First, we prove that a random cosine polynomial with the blocks of coefficients being of a fixed length and satisfying $ A_{2j}=A_{2j+1} $ possesses the same expected real zeros as the classical case. Afterwards, we study a case containing only two equal blocks of coefficients, and show that in this case significantly more real zeros should be expected compared to those of the classical case.


title: Threshold-Based Retrieval and Textual Entailment Detection on Legal Bar Exam Questions

abstract: Getting an overview over the legal domain has become challenging, especially in a broad, international context. Legal question answering systems have the potential to alleviate this task by automatically retrieving relevant legal texts for a specific statement and checking whether the meaning of the statement can be inferred from the found documents. We investigate a combination of the BM25 scoring method of Elasticsearch with word embeddings trained on English translations of the German and Japanese civil law. For this, we define criteria which select a dynamic number of relevant documents according to threshold scores. Exploiting two deep learning classifiers and their respective prediction bias with a threshold-based answer inclusion criterion has shown to be beneficial for the textual entailment task, when compared to the baseline.


title: Toeplitz operators on the domain $\{Z\in M_{2\times2}(\mathbb{C}) \mid Z Z^* < I\}$ with $\mathrm{U}(2)\times\mathbb{T}^2$-invariant symbols

abstract: Let $D$ be the irreducible bounded symmetric domain of $2\times2$ complex matrices that satisfy $ZZ^* < I_2$. The biholomorphism group of $D$ is realized by $\mathrm{U}(2,2)$ with isotropy at the origin given by $\mathrm{U}(2)\times\mathrm{U}(2)$. Denote by $\mathbb{T}^2$ the subgroup of diagonal matrices in $\mathrm{U}(2)$. We prove that the set of $\mathrm{U}(2)\times\mathbb{T}^2$-invariant essentially bounded symbols yield Toeplitz operators that generate commutative $C^*$-algebras on all weighted Bergman spaces over $D$. Using tools from representation theory, we also provide an integral formula for the spectra of these Toeplitz operators.


title: Reducing Tail Latency via Safe and Simple Duplication

abstract: Duplication can be a powerful strategy for overcoming stragglers in cloud services, but is often used conservatively because of the risk of overloading the system. We present duplicate-aware scheduling or DAS, which makes duplication safe and easy to use, by leveraging the two well-known primitives of prioritization and purging. To support DAS across diverse layers of a cloud system (e.g., network, storage, etc), we propose the D-Stage abstraction, which decouples the duplication policy from the mechanism, and facilitates working with legacy layers of a system. Using this abstraction, we evaluate the benefits of DAS for two data parallel applications (HDFS, an in-memory workload generator) and a network function (snort-based IDS cluster). Our experiments on the public cloud and Emulab show that DAS is safe to use, and the tail latency improvement holds across a wide range of workloads


title: Busemann functions and semi-infinite O'Connell-Yor polymers

abstract: We prove that given any fixed asymptotic velocity, the finite length O'Connell-Yor polymer has an infinite length limit satisfying the law of large numbers with this velocity. By a Markovian property of the quenched polymer this reduces to showing the existence of Busemann functions: almost sure limits of ratios of random point-to-point partition functions. The key ingredients are the Burke property of the O'Connell-Yor polymer and a comparison lemma for the ratios of partition functions. We also show the existence of infinite length limits in the Brownian last passage percolation model.


title: DiaBLa: A Corpus of Bilingual Spontaneous Written Dialogues for Machine Translation

abstract: We present a new English-French test set for the evaluation of Machine Translation (MT) for informal, written bilingual dialogue. The test set contains 144 spontaneous dialogues (5,700+ sentences) between native English and French speakers, mediated by one of two neural MT systems in a range of role-play settings. The dialogues are accompanied by fine-grained sentence-level judgments of MT quality, produced by the dialogue participants themselves, as well as by manually normalised versions and reference translations produced a posteriori. The motivation for the corpus is two-fold: to provide (i) a unique resource for evaluating MT models, and (ii) a corpus for the analysis of MT-mediated communication. We provide a preliminary analysis of the corpus to confirm that the participants' judgments reveal perceptible differences in MT quality between the two MT systems used.


title: Radial evolution of stochastic heating in low-$\beta$ solar wind

abstract: We consider the radial evolution of perpendicular ion heating due to the violation of magnetic moment conservation caused by Alfven and kinetic Alfven wave turbulence. This process, referred to as stochastic heating, can be quantified by the ratio between the average velocity fluctuations at the ion gyroradius and the perpendicular ion thermal speed $\epsilon \equiv \delta v / v_{t\perp}$. Using 17 years of Helios observations, we constrain how much energy could be dissipated by this mechanism between 0.29 and 0.98 au. We find that stochastic heating likely operates throughout the entire inner heliosphere, but that its radial dependence is steeper than that of empirically derived dissipation rates, being $r^{-2.5}$ compared with $r^{-2.08}$. This difference is significantly increased in fast solar wind streams to $r^{-3.1}$ compared with $r^{-1.8}$.


title: The Response of the Lyman-Alpha Line in Different Flare Heating Models

abstract: The solar Ly$\alpha$ line is the strongest line in the ultraviolet waveband, and is greatly enhanced during solar flares. Here we present radiative hydrodynamic simulations of solar flares under different heating models, and calculate the response of this line taking into account non-equilibrium ionization of hydrogen and partial frequency redistribution. We find that in non-thermal heating models, the Ly$\alpha$ line can show a red or blue asymmetry corresponding to the chromospheric evaporation or condensation, respectively. The asymmetry may change from red to blue if the electron beam flux is large enough to produce a significant chromospheric condensation region. In the Ly$\alpha$ intensity lightcurve, there appears a dip when the change of asymmetry occurs. In thermal models, the Ly$\alpha$ line intensity peaks quickly and then falls, and the profile has an overall red asymmetry, which is similar to the profiles from heating by a soft electron beam. The Ly$\alpha$ profile shows a single red peak at the end of thermal heating, and the whole line is formed in a very small height range.


title: Reinforcement Learning for Mean Field Game

abstract: Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously, the state of every agent moves to the next state, and each agent receives a reward. However, finding an equilibrium (if exists) in this game is often difficult when the number of agents becomes large. This paper focuses on finding a mean-field equilibrium (MFE) in an action coupled stochastic game setting in an episodic framework. It is assumed that the impact of the other agents' can be assumed by the empirical distribution of the mean of the actions. All agents know the action distribution and employ lower-myopic best response dynamics to choose the optimal oblivious strategy. This paper proposes a posterior sampling based approach for reinforcement learning in the mean-field game, where each agent samples a transition probability from the previous transitions. We show that the policy and action distributions converge to the optimal oblivious strategy and the limiting distribution, respectively, which constitute an MFE.


title: Multi-modal Discriminative Model for Vision-and-Language Navigation

abstract: Vision-and-Language Navigation (VLN) is a natural language grounding task where agents have to interpret natural language instructions in the context of visual scenes in a dynamic environment to achieve prescribed navigation goals. Successful agents must have the ability to parse natural language of varying linguistic styles, ground them in potentially unfamiliar scenes, plan and react with ambiguous environmental feedback. Generalization ability is limited by the amount of human annotated data. In particular, \emph{paired} vision-language sequence data is expensive to collect. We develop a discriminator that evaluates how well an instruction explains a given path in VLN task using multi-modal alignment. Our study reveals that only a small fraction of the high-quality augmented data from \citet{Fried:2018:Speaker}, as scored by our discriminator, is useful for training VLN agents with similar performance on previously unseen environments. We also show that a VLN agent warm-started with pre-trained components from the discriminator outperforms the benchmark success rates of 35.5 by 10\% relative measure on previously unseen environments.


title: Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data

abstract: Linguistic Code Switching (CS) is a phenomenon that occurs when multilingual speakers alternate between two or more languages/dialects within a single conversation. Processing CS data is especially challenging in intra-sentential data given state-of-the-art monolingual NLP technologies since such technologies are geared toward the processing of one language at a time. In this paper, we address the problem of Part-of-Speech tagging (POS) in the context of linguistic code switching (CS). We explore leveraging multiple neural network architectures to measure the impact of different pre-trained embeddings methods on POS tagging CS data. We investigate the landscape in four CS language pairs, Spanish-English, Hindi-English, Modern Standard Arabic- Egyptian Arabic dialect (MSA-EGY), and Modern Standard Arabic- Levantine Arabic dialect (MSA-LEV). Our results show that multilingual embedding (e.g., MSA-EGY and MSA-LEV) helps closely related languages (EGY/LEV) but adds noise to the languages that are distant (SPA/HIN). Finally, we show that our proposed models outperform state-of-the-art CS taggers for MSA-EGY language pair.


title: Efficient Forward Architecture Search

abstract: We propose a neural architecture search (NAS) algorithm, Petridish, to iteratively add shortcut connections to existing network layers. The added shortcut connections effectively perform gradient boosting on the augmented layers. The proposed algorithm is motivated by the feature selection algorithm forward stage-wise linear regression, since we consider NAS as a generalization of feature selection for regression, where NAS selects shortcuts among layers instead of selecting features. In order to reduce the number of trials of possible connection combinations, we train jointly all possible connections at each stage of growth while leveraging feature selection techniques to choose a subset of them. We experimentally show this process to be an efficient forward architecture search algorithm that can find competitive models using few GPU days in both the search space of repeatable network modules (cell-search) and the space of general networks (macro-search). Petridish is particularly well-suited for warm-starting from existing models crucial for lifelong-learning scenarios.


title: Anomalies in the Space of Coupling Constants and Their Dynamical Applications II

abstract: We extend our earlier work on anomalies in the space of coupling constants to four-dimensional gauge theories. Pure Yang-Mills theory (without matter) with a simple and simply connected gauge group has a mixed anomaly between its one-form global symmetry (associated with the center) and the periodicity of the $\theta$-parameter. This anomaly is at the root of many recently discovered properties of these theories, including their phase transitions and interfaces. These new anomalies can be used to extend this understanding to systems without discrete symmetries (such as time-reversal). We also study $SU(N)$ and $Sp(N)$ gauge theories with matter in the fundamental representation. Here we find a mixed anomaly between the flavor symmetry group and the $\theta$-periodicity. Again, this anomaly unifies distinct recently-discovered phenomena in these theories and controls phase transitions and the dynamics on interfaces.

categories: hep-th cond-mat.str-el math-ph math.AT math.MP