title: Classification Accuracy Score for Conditional Generative Models

abstract: Deep generative models (DGMs) of images are now sufficiently mature that they produce nearly photorealistic samples and obtain scores similar to the data distribution on heuristics such as Frechet Inception Distance (FID). These results, especially on large-scale datasets such as ImageNet, suggest that DGMs are learning the data distribution in a perceptually meaningful space and can be used in downstream tasks. To test this latter hypothesis, we use class-conditional generative models from a number of model classes---variational autoencoders, autoregressive models, and generative adversarial networks (GANs)---to infer the class labels of real data. We perform this inference by training an image classifier using only synthetic data and using the classifier to predict labels on real data. The performance on this task, which we call Classification Accuracy Score (CAS), reveals some surprising results not identified by traditional metrics and constitute our contributions. First, when using a state-of-the-art GAN (BigGAN-deep), Top-1 and Top-5 accuracy decrease by 27.9\% and 41.6\%, respectively, compared to the original data; and conditional generative models from other model classes, such as Vector-Quantized Variational Autoencoder-2 (VQ-VAE-2) and Hierarchical Autoregressive Models (HAMs), substantially outperform GANs on this benchmark. Second, CAS automatically surfaces particular classes for which generative models failed to capture the data distribution, and were previously unknown in the literature. Third, we find traditional GAN metrics such as Inception Score (IS) and FID neither predictive of CAS nor useful when evaluating non-GAN models. Furthermore, in order to facilitate better diagnoses of generative models, we open-source the proposed metric.


title: Nonregular and Minimax Estimation of Individualized Thresholds in High Dimension with Binary Responses

abstract: Given a large number of covariates $Z$, we consider the estimation of a high-dimensional parameter $\theta$ in an individualized linear threshold $\theta^T Z$ for a continuous variable $X$, which minimizes the disagreement between $\text{sign}(X-\theta^TZ)$ and a binary response $Y$. While the problem can be formulated into the M-estimation framework, minimizing the corresponding empirical risk function is computationally intractable due to discontinuity of the sign function. Moreover, estimating $\theta$ even in the fixed-dimensional setting is known as a nonregular problem leading to nonstandard asymptotic theory. To tackle the computational and theoretical challenges in the estimation of the high-dimensional parameter $\theta$, we propose an empirical risk minimization approach based on a regularized smoothed loss function. The statistical and computational trade-off of the algorithm is investigated. Statistically, we show that the finite sample error bound for estimating $\theta$ in $\ell_2$ norm is $(s\log d/n)^{\beta/(2\beta+1)}$, where $d$ is the dimension of $\theta$, $s$ is the sparsity level, $n$ is the sample size and $\beta$ is the smoothness of the conditional density of $X$ given the response $Y$ and the covariates $Z$. The convergence rate is nonstandard and slower than that in the classical Lasso problems. Furthermore, we prove that the resulting estimator is minimax rate optimal up to a logarithmic factor. The Lepski's method is developed to achieve the adaption to the unknown sparsity $s$ and smoothness $\beta$. Computationally, an efficient path-following algorithm is proposed to compute the solution path. We show that this algorithm achieves geometric rate of convergence for computing the whole path. Finally, we evaluate the finite sample performance of the proposed estimator in simulation studies and a real data analysis.


title: Improving Change Prediction Models with Code Smell-Related Information

abstract: Code smells represent sub-optimal implementation choices applied by developers when evolving software systems. The negative impact of code smells has been widely investigated in the past: besides developers' productivity and ability to comprehend source code, researchers empirically showed that the presence of code smells heavily impacts the change-proneness of the affected classes. On the basis of these findings, in this paper we conjecture that code smell-related information can be effectively exploited to improve the performance of change prediction models, ie models having as goal that of indicating to developers which classes are more likely to change in the future, so that they may apply preventive maintenance actions. Specifically, we exploit the so-called intensity index - a previously defined metric that captures the severity of a code smell - and evaluate its contribution when added as additional feature in the context of three state of the art change prediction models based on product, process, and developer-based features. We also compare the performance achieved by the proposed model with the one of an alternative technique that considers the previously defined antipattern metrics, namely a set of indicators computed considering the history of code smells in files. Our results report that (i) the prediction performance of the intensity-including models is statistically better than that of the baselines and (ii) the intensity is a more powerful metric with respect to the alternative smell-related ones.


title: Deep-Neural-Network based Fall-back Mechanism in Interference-Aware Receiver Design

abstract: In this letter, we consider designing a fall-back mechanism in an interference-aware receiver. Typically, there are two different manners of dealing with interference, known as enhanced interference-rejection-combining (eIRC) and symbol-level interference-cancellation (SLIC). Although SLIC performs better than eIRC, it has higher complexity and requires the knowledge of modulation-format (MF) of interference. Due to potential errors in MF detection, SLIC can run with a wrong MF and render limited gains. Therefore, designing a fall-back mechanism is of interest that only activates SLIC when the detected MF is reliable. Otherwise, a fall-back happens and the receiver turns to eIRC. Finding a closed-form expression of an optimal fall-back mechanism seems difficult, and we utilize deep-neural-network (DNN) to design it which is shown to be effective and performs better than a traditional Bayes-risk based design in terms of reducing error-rate and saving computational-cost.


title: A hybrid model for predicting human physical activity status from lifelogging data

abstract: One trend in the recent healthcare transformations is people are encouraged to monitor and manage their health based on their daily diets and physical activity habits. However, much attention of the use of operational research and analytical models in healthcare has been paid to the systematic level such as country or regional policy making or organisational issues. This paper proposes a model concerned with healthcare analytics at the individual level, which can predict human physical activity status from sequential lifelogging data collected from wearable sensors. The model has a two-stage hybrid structure (in short, MOGP-HMM) -- a multi-objective genetic programming (MOGP) algorithm in the first stage to reduce the dimensions of lifelogging data and a hidden Markov model (HMM) in the second stage for activity status prediction over time. It can be used as a decision support tool to provide real-time monitoring, statistical analysis and personalized advice to individuals, encouraging positive attitudes towards healthy lifestyles. We validate the model with the real data collected from a group of participants in the UK, and compare it with other popular two-stage hybrid models. Our experimental results show that the MOGP-HMM can achieve comparable performance. To the best of our knowledge, this is the very first study that uses the MOGP in the hybrid two-stage structure for individuals' activity status prediction. It fits seamlessly with the current trend in the UK healthcare transformation of patient empowerment as well as contributing to a strategic development for more efficient and cost-effective provision of healthcare.


title: Extreme Multi-Label Legal Text Classification: A case study in EU Legislation

abstract: We consider the task of Extreme Multi-Label Text Classification (XMTC) in the legal domain. We release a new dataset of 57k legislative documents from EURLEX, the European Union's public document database, annotated with concepts from EUROVOC, a multidisciplinary thesaurus. The dataset is substantially larger than previous EURLEX datasets and suitable for XMTC, few-shot and zero-shot learning. Experimenting with several neural classifiers, we show that BIGRUs with self-attention outperform the current multi-label state-of-the-art methods, which employ label-wise attention. Replacing CNNs with BIGRUs in label-wise attention networks leads to the best overall performance.


title: Adaptive Learning Material Recommendation in Online Language Education

abstract: Recommending personalized learning materials for online language learning is challenging because we typically lack data about the student's ability and the relative difficulty of learning materials. This makes it hard to recommend appropriate content that matches the student's prior knowledge. In this paper, we propose a refined hierarchical knowledge structure to model vocabulary knowledge, which enables us to automatically organize the authentic and up-to-date learning materials collected from the internet. Based on this knowledge structure, we then introduce a hybrid approach to recommend learning materials that adapts to a student's language level. We evaluate our work with an online Japanese learning tool and the results suggest adding adaptivity into material recommendation significantly increases student engagement.


title: Effects of noise-induced coherence on the fluctuations of current in quantum absorption refrigerators

abstract: We investigate the effects of noise-induced coherence on average current and current fluctuations in a simple model of quantum absorption refrigerator with degenerate energy levels. We describe and explain the differences and similarities between the system behavior when it operates in the classical regime, where the populations and coherences in the corresponding quantum optical master equation decouple in a suitably chosen basis, and in the quantum regime, where such a transformation does not exist. The differences between the quantum and the classical case are observable only close to the maximum current regime, where the system steady-state becomes non-unique. This allows us to approximate the system dynamics by an analytical model based on a dichotomous process, that explains the behavior of the average current both in the classical and in the quantum case. Due to the non-uniqueness, the scaled cumulant generating function for the current at the vicinity of the critical point exhibits behavior reminiscent of the dynamical first-order phase transition. Unless the system parameters are fine-tuned to a single point in the parameter space, the corresponding current fluctuations are moderate in the quantum case and large in the classical case.


abstract: This paper presents a novel leaderless protocol (FPC-BI: Fast Probabilistic Consensus within Byzantine Infrastructures) with a low communicational complexity and which allows a set of nodes to come to a consensus on a value of a single bit. The paper makes the assumption that part of the nodes are Byzantine, and are thus controlled by an adversary who intends to either delay the consensus, or break it (this defines that at least a couple of honest nodes come to different conclusions). We prove that, nevertheless, the protocol works with high probability when its parameters are suitably chosen. Along this the paper also provides explicit estimates on the probability that the protocol finalizes in the consensus state in a given time. This protocol could be applied to reaching consensus in decentralized cryptocurrency systems. A special feature of it is that it makes use of a sequence of random numbers which are either provided by a trusted source or generated by the nodes themselves using some decentralized random number generating protocol. This increases the overall trustworthiness of the infrastructure. A core contribution of the paper is that it uses a very weak consensus to obtain a strong consensus on the value of a bit, and which can relate to the validity of a transaction.


title: Asymptotic security analysis of discrete-modulated continuous-variable quantum key distribution

abstract: Continuous-variable quantum key distribution (CV QKD) protocols with discrete modulation are interesting due to their experimental simplicity and their great potential for massive deployment in the quantum-secured networks, but their security analysis is less advanced than that of Gaussian modulation schemes. In this work, we apply a numerical method to analyze the security of discrete-modulation protocols against collective attacks in the asymptotic limit, paving the way for a full security proof with finite-size effects. While our method is general for discrete-modulation schemes, we focus on two variants of the CV QKD protocol with quaternary modulation. Interestingly, thanks to the tightness of our proof method, we show that this protocol is capable of achieving much higher key rates over significantly longer distances with experimentally feasible parameters compared with previous security proofs of binary and ternary modulation schemes and also yielding key rates comparable to Gaussian modulation schemes. Furthermore, as our security analysis method is versatile, it allows us to evaluate variations of the discrete-modulated protocols, including direct and reverse reconciliation, and postselection strategies. In particular, we demonstrate that postselection of data in combination with reverse reconciliation can improve the key rates.


title: A note on multiplicative automatic sequences, II

abstract: We prove that any $q$-automatic multiplicative function $f:\mathbb{N}\to\mathbb{C}$ either essentially coincides with a Dirichlet character, or vanishes on all sufficiently large primes. This confirms a strong form of a conjecture of J. Bell, N. Bruin, and M. Coons.


title: Modular properties of surface operators in N=2 SQCD

abstract: We study half-BPS surface operators in N=2 supersymmetric QCD in four dimensions with gauge group SU(2) and four fundamental flavours. We compute the twisted chiral superpotential that describes the effective theory on the surface operator using equivariant localization as well as the Seiberg-Witten data. We then use the constraints imposed by S-duality to resum the instanton contributions to the twisted superpotential into elliptic functions and (quasi-) modular forms. The resummed results match what one would obtain from the description of surface operators as the insertion of a degenerate operator in a spherical conformal block in Liouville CFT.


title: ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems

abstract: Despite remarkable empirical success, the training dynamics of generative adversarial networks (GAN), which involves solving a minimax game using stochastic gradients, is still poorly understood. In this work, we analyze last-iterate convergence of simultaneous gradient descent (simGD) and its variants under the assumption of convex-concavity, guided by a continuous-time analysis with differential equations. First, we show that simGD, as is, converges with stochastic sub-gradients under strict convexity in the primal variable. Second, we generalize optimistic simGD to accommodate an optimism rate separate from the learning rate and show its convergence with full gradients. Finally, we present anchored simGD, a new method, and show convergence with stochastic subgradients.


title: Rearchitecting Classification Frameworks For Increased Robustness

abstract: While generalizing well over natural inputs, neural networks are vulnerable to adversarial inputs. Existing defenses against adversarial inputs have largely been detached from the real world. These defenses also come at a cost to accuracy. Fortunately, there are invariances of an object that are its salient features; when we break them it will necessarily change the perception of the object. We find that applying invariants to the classification task makes robustness and accuracy feasible together. Two questions follow: how to extract and model these invariances? and how to design a classification paradigm that leverages these invariances to improve the robustness accuracy trade-off? The remainder of the paper discusses solutions to the aformenetioned questions.


title: Seeing Convolution Through the Eyes of Finite Transformation Semigroup Theory: An Abstract Algebraic Interpretation of Convolutional Neural Networks

abstract: Researchers are actively trying to gain better insights into the representational properties of convolutional neural networks for guiding better network designs and for interpreting a network's computational nature. Gaining such insights can be an arduous task due to the number of parameters in a network and the complexity of a network's architecture. Current approaches of neural network interpretation include Bayesian probabilistic interpretations and information theoretic interpretations. In this study, we take a different approach to studying convolutional neural networks by proposing an abstract algebraic interpretation using finite transformation semigroup theory. Specifically, convolutional layers are broken up and mapped to a finite space. The state space of the proposed finite transformation semigroup is then defined as a single element within the convolutional layer, with the acting elements defined by surrounding state elements combined with convolution kernel elements. Generators of the finite transformation semigroup are defined to complete the interpretation. We leverage this approach to analyze the basic properties of the resulting finite transformation semigroup to gain insights on the representational properties of convolutional neural networks, including insights into quantized network representation. Such a finite transformation semigroup interpretation can also enable better understanding outside of the confines of fixed lattice data structures, thus useful for handling data that lie on irregular lattices. Furthermore, the proposed abstract algebraic interpretation is shown to be viable for interpreting convolutional operations within a variety of convolutional neural network architectures.


title: Engineering Kernelization for Maximum Cut

abstract: Kernelization is a general theoretical framework for preprocessing instances of NP-hard problems into (generally smaller) instances with bounded size, via the repeated application of data reduction rules. For the fundamental Max Cut problem, kernelization algorithms are theoretically highly efficient for various parameterizations. However, the efficacy of these reduction rules in practice---to aid solving highly challenging benchmark instances to optimality---remains entirely unexplored. We engineer a new suite of efficient data reduction rules that subsume most of the previously published rules, and demonstrate their significant impact on benchmark data sets, including synthetic instances, and data sets from the VLSI and image segmentation application domains. Our experiments reveal that current state-of-the-art solvers can be sped up by up to multiple orders of magnitude when combined with our data reduction rules. On social and biological networks in particular, kernelization enables us to solve four instances that were previously unsolved in a ten-hour time limit with state-of-the-art solvers; three of these instances are now solved in less than two seconds.


title: The water-carbon monoxide dimer: new infrared spectra, ab initio rovibrational energy level calculations, and an interesting intermolecular mode

abstract: Rovibrational energy level calculations using a high-level intermolecular potential surface are reported for H2O-CO and D2O-CO. They predict the ground K = 1 levels to lie about 20 (12) cm-1 above K = 0 for H2O-CO (D2O-CO) in good agreement with past experiment. But the first excited K = 1 levels are predicted to lie about 3 cm-1 below their K = 0 counterparts in both cases. Intensity calculations also indicate that mid-infrared transitions from the K = 0 ground state to this seemingly anomalous excited K = 1 state should be observable. These predictions are strikingly verified by new spectroscopic measurements covering the C-O stretch region around 2200 cm-1 for H2O-CO, D2O-CO, and HOD-CO, and the O-D stretch region around 2700 cm-1 for D2O-CO, HOD-CO, and DOH-CO. The experiments probe a pulsed supersonic slit jet expansion using tunable infrared quantum cascade laser or optical parametric oscillator sources. Discrete perturbations in the O-D stretch region give an experimental lower limit of about 340 cm-1 for D2O-CO, as compared to our calculated binding energy of 368 cm-1. Wavefunction plots are presented to help understand the intermolecular dynamics of H2O-CO. Coriolis interactions are invoked to explain the seemingly anomalous energies of the first excited K = 1 levels.


title: Robust Classification using Robust Feature Augmentation

abstract: Existing deep neural networks, say for image classification, have been shown to be vulnerable to adversarial images that can cause a DNN misclassification, without any perceptible change to an image. In this work, we propose shock absorbing robust features such as binarization, e.g., rounding, and group extraction, e.g., color or shape, to augment the classification pipeline, resulting in more robust classifiers. Experimentally, we show that augmenting ML models with these techniques leads to improved overall robustness on adversarial inputs as well as significant improvements in training time. On the MNIST dataset, we achieved 14x speedup in training time to obtain 90% adversarial accuracy com-pared to the state-of-the-art adversarial training method of Madry et al., as well as retained higher adversarial accuracy over a broader range of attacks. We also find robustness improvements on traffic sign classification using robust feature augmentation. Finally, we give theoretical insights for why one can expect robust feature augmentation to reduce adversarial input space


title: Pressure-controlled interlayer magnetism in atomically thin CrI3

abstract: Stacking order can significantly influence the physical properties of two-dimensional (2D) van der Waals materials. The recent isolation of atomically thin magnetic materials opens the door for control and design of magnetism via stacking order. Here we apply hydrostatic pressure up to 2 GPa to modify the stacking order in a prototype van der Waals magnetic insulator CrI3. We observe an irreversible interlayer antiferromagnetic (AF) to ferromagnetic (FM) transition in atomically thin CrI3 by magnetic circular dichroism and electron tunneling measurements. The effect is accompanied by a monoclinic to a rhombohedral stacking order change characterized by polarized Raman spectroscopy. Before the structural change, the interlayer AF coupling energy can be tuned up by nearly 100% by pressure. Our experiment reveals interlayer FM coupling, which is the established ground state in bulk CrI3, but never observed in native exfoliated thin films. The observed correlation between the magnetic ground state and the stacking order is in good agreement with first principles calculations and suggests a route towards nanoscale magnetic textures by moir\'e engineering.


title: Non-Determinism in Neural Networks for Adversarial Robustness

abstract: Recent breakthroughs in the field of deep learning have led to advancements in a broad spectrum of tasks in computer vision, audio processing, natural language processing and other areas. In most instances where these tasks are deployed in real-world scenarios, the models used in them have been shown to be susceptible to adversarial attacks, making it imperative for us to address the challenge of their adversarial robustness. Existing techniques for adversarial robustness fall into three broad categories: defensive distillation techniques, adversarial training techniques, and randomized or non-deterministic model based techniques. In this paper, we propose a novel neural network paradigm that falls under the category of randomized models for adversarial robustness, but differs from all existing techniques under this category in that it models each parameter of the network as a statistical distribution with learnable parameters. We show experimentally that this framework is highly robust to a variety of white-box and black-box adversarial attacks, while preserving the task-specific performance of the traditional neural network model.


title: Learning Policies from Human Data for Skat

abstract: Decision-making in large imperfect information games is difficult. Thanks to recent success in Poker, Counterfactual Regret Minimization (CFR) methods have been at the forefront of research in these games. However, most of the success in large games comes with the use of a forward model and powerful state abstractions. In trick-taking card games like Bridge or Skat, large information sets and an inability to advance the simulation without fully determinizing the state make forward search problematic. Furthermore, state abstractions can be especially difficult to construct because the precise holdings of each player directly impact move values. In this paper we explore learning model-free policies for Skat from human game data using deep neural networks (DNN). We produce a new state-of-the-art system for bidding and game declaration by introducing methods to a) directly vary the aggressiveness of the bidder and b) declare games based on expected value while mitigating issues with rarely observed state-action pairs. Although cardplay policies learned through imitation are slightly weaker than the current best search-based method, they run orders of magnitude faster. We also explore how these policies could be learned directly from experience in a reinforcement learning setting and discuss the value of incorporating human data for this task.


title: Quarter-plane lattice paths with interacting boundaries: the Kreweras and reverse Kreweras models

abstract: Lattice paths in the quarter plane have led to a large and varied set of results in recent years. One major project has been the classification of step sets according to the properties of the corresponding generating functions, and this has involved a variety of techniques, some highly intricate and specialised. The famous Kreweras and reverse Kreweras walk models are two particularly interesting models, as they are among the only four cases which have algebraic generating functions. Here we investigate how the properties of the Kreweras and reverse Kreweras models change when boundary interactions are introduced. That is, we associate three real-valued weights $a,b,c$ with visits by the walks to the $x$-axis, the $y$-axis and the origin $(0,0)$ respectively. These models were partially solved in a recent paper by Beaton, Owczarek and Rechnitzer (2019). We apply the algebraic kernel method to completely solve these two models. We find that reverse Kreweras walks have an algebraic generating function for all $a,b,c$, regardless of whether the walks are restricted to end at the origin or on one of the axes, or may end anywhere at all. For Kreweras walks, the generating function for walks returning to the origin is algebraic, but the other cases are only D-finite. To our knowledge this is the first example of a quarter-plane model with this property.


title: Broadband X-ray Spectral and Timing Analyses of the Black Hole Binary Candidate Swift J1658.2-4242: Rapid Flux Variation and the Turn-on of a Transient QPO

abstract: We report results from joint NuSTAR, Swift and XMM-Newton observations of the newly discovered black hole X-ray binary candidate Swift J1658.2-4242 in the intermediate state. We observe a peculiar event in this source, with its X-ray flux rapidly decreasing by $\sim$45\% in $\sim$40~s, accompanied by only subtle changes in the shape of the broadband X-ray spectrum. In addition, we find a sudden turn-on of a transient QPO with a frequency of $6-7$~Hz around the time of the flux change, and the total fractional rms amplitude of the power spectrum increases from $\sim$2\% to $\sim$10\%. X-ray spectral and timing analyses indicate that the flux decrease is driven by intrinsic changes in the accretion flow around the black hole, rather than intervening material along the line of sight. In addition, we do not significantly detect any relativistic disk reflection component, indicating it is much weaker than previously observed while the source was in the bright hard state. We propose accretion disk instabilities triggered at a large disk radius as the origin of the fast transition in spectral and timing properties, and discuss possible causes of the unusual properties observed in Swift J1658.2-4242. The prompt flux variation detected along with the emergence of a QPO makes the event an interesting case for investigating QPO mechanisms in black hole X-ray binaries.


title: Risk Analysis Study of Fully Autonomous Vehicle

abstract: Fully autonomous vehicles are emerging vehicular technologies that have gained significant attention in todays research endeavours. Even though it promises to optimize road safety, the proliferation of wireless and sensor technologies makes it susceptible to cyber threats thus dawdling its adoption. The identification of threats and design of apposite security solutions is therefore pertinent to expedite its adoption. In this paper, we analyse the security risks of the communication infrastructure for the fully autonomous vehicle using a subset of the TVRA methodology by ETSI. We described the model of communication infrastructure. This model clarifies the potential communication possibilities of the vehicle. Then we defined the security objectives and identified threats. Furthermore, we classified risks and propose countermeasures to facilitate the design of security solutions. We find that all identified high impact threats emanates from a particular source and required encryption mechanisms as countermeasures. Finally, we discovered that all threats due to an interaction with humans are of serious consequences.


title: Policy Based Inference in Trick-Taking Card Games

abstract: Trick-taking card games feature a large amount of private information that slowly gets revealed through a long sequence of actions. This makes the number of histories exponentially large in the action sequence length, as well as creating extremely large information sets. As a result, these games become too large to solve. To deal with these issues many algorithms employ inference, the estimation of the probability of states within an information set. In this paper, we demonstrate a Policy Based Inference (PI) algorithm that uses player modelling to infer the probability we are in a given state. We perform experiments in the German trick-taking card game Skat, in which we show that this method vastly improves the inference as compared to previous work, and increases the performance of the state-of-the-art Skat AI system Kermit when it is employed into its determinized search algorithm.


title: Defining Quantum Neural Networks via Quantum Time Evolution

abstract: This work presents a novel fundamental algorithm for for defining and training Neural Networks in Quantum Information based on time evolution and the Hamiltonian. Classical Neural Network algorithms (ANN) are computationally expensive. For example, in image classification, representing an image pixel by pixel using classical information requires an enormous amount of computational memory resources. Hence, exploring methods to represent images in a different paradigm of information is important. Quantum Neural Networks (QNNs) have been explored for over 20 years. The current forefront work based on Variational Quantum Circuits is specifically defined for the Continuous Variable (CV) Model of quantum computers. In this work, a model is proposed which is defined at a more fundamental level and hence can be inherited by any variants of quantum computing models. This work also presents a quantum backpropagation algorithm to train our QNN model and validate this algorithm on the MNIST dataset on a quantum computer simulation.


title: Reproducing kernel Hilbert spaces on manifolds: Sobolev and Diffusion spaces

abstract: We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with a number of detailed examples.


title: Consecutive Detecting Arrays for Interaction Faults

abstract: The concept of detecting arrays was developed to locate and detect interaction faults arising between the factors in a component-based system during software testing. In this paper, we propose a family of consecutive detecting arrays (CDAs) in which the interactions between factors are considered to be ordered. CDAs can be used to generate test suites for locating and detecting interaction faults between neighboring factors. We establish a general criterion for measuring the optimality of CDAs in terms of their size. Based on this optimality criterion, the equivalence between optimum CDAs and consecutive orthogonal arrays with prescribed properties is explored. Using the advantages of this equivalence, a great number of optimum CDAs are presented. In particular, the existence of optimum CDAs with few factors is almost completely determined.


title: SpecNet: Spectral Domain Convolutional Neural Network

abstract: The memory consumption of most Convolutional Neural Network (CNN) architectures grows rapidly with increasing depth of the network, which is a major constraint for efficient network training on modern GPUs with limited memory, embedded systems, and mobile devices. Several studies show that the feature maps (as generated after the convolutional layers) are the main bottleneck in this memory problem. Often, these feature maps mimic natural photographs in the sense that their energy is concentrated in the spectral domain. Although embedding CNN architectures in the spectral domain is widely exploited to accelerate the training process, we demonstrate that it is also possible to use the spectral domain to reduce the memory footprint, a method we call Spectral Domain Convolutional Neural Network (SpecNet) that performs both the convolution and the activation operations in the spectral domain. The performance of SpecNet is evaluated on three competitive object recognition benchmark tasks (CIFAR-10, SVHN, and ImageNet), and compared with several state-of-the-art implementations. Overall, SpecNet is able to reduce memory consumption by about 60% without significant loss of performance for all tested networks.


title: The challenges of purely mechanistic models in biology and the minimum need for a 'mechanism-plus-X' framework

abstract: Ever since the advent of molecular biology in the 1970s, mechanical models have become the dogma in the field, where a "true" understanding of any subject is equated to a mechanistic description. This has been to the detriment of the biomedical sciences, where, barring some exceptions, notable new feats of understanding have arguably not been achieved in normal and disease biology, including neurodegenerative disease and cancer pathobiology. I argue for a "mechanism-plus-X" paradigm, where mainstay elements of mechanistic models such as hierarchy and correlation are combined with nomological principles such as general operative rules and generative principles. Depending on the question at hand and the nature of the inquiry, X could range from proven physical laws to speculative biological generalizations, such as the notional principle of cellular synchrony. I argue that the "mechanism-plus-X" approach should ultimately aim to move biological inquiries out of the deadlock of oft-encountered mechanistic pitfalls and reposition biology to its former capacity of illuminating fundamental truths about the world.


title: Temporal-difference learning with nonlinear function approximation: lazy training and mean field regimes

abstract: We discuss the approximation of the value function for infinite-horizon discounted Markov Reward Processes (MRP) with nonlinear functions trained with the Temporal-Difference (TD) learning algorithm. We first consider this problem under a certain scaling of the approximating function, leading to a regime called lazy training. In this regime, the parameters of the model vary only slightly during the learning process, a feature that has recently been observed in the training of neural networks, where the scaling we study arises naturally, implicit in the initialization of their parameters. Both in the under- and over-parametrized frameworks, we prove exponential convergence to local, respectively global minimizers of the above algorithm in the lazy training regime. We then compare this scaling of the parameters to the mean-field regime, where the approximately linear behavior of the model is lost. Under this alternative scaling we prove that all fixed points of the dynamics in parameter space are global minimizers. We finally give examples of our convergence results in the case of models that diverge if trained with non-lazy TD learning, and in the case of neural networks.


title: The 2-Neutrino Exchange Potential with Mixing: A Probe of Neutrino Physics and CP Violation

abstract: The 2-neutrino exchange potential is a Standard Model weak potential arising from the exchange of virtual neutrino-antineutrino pairs which must include all neutrino properties, including the number of flavors, their masses, fermionic nature (Dirac or Majorana), and CP violation. We describe a new approach for calculating the spin-independent 2-neutrino exchange potential, including the mixing of three neutrino mass states and CP violation.


title: Quantum information processing with nuclear spins mediated by a weak-mechanically controlled electron spin

abstract: We propose a scheme to achieve nuclear-nuclear indirect interactions mediated by a mechanically driven nitrogen-vacancy (NV) center in diamond. Here we demonstrate two-qubit entangling gates and quantum-state transfer between two carbon nuclei. When dipole-dipole interaction strength is much larger than the driving field strength, the scheme is robust against decoherence caused by coupling between the NV center (nuclear spins) and the environment. Conveniently, precise control of dipole coupling is not required so this scheme is insensitive to fluctuating positions of the nuclear spins and the NV center. Our scheme provides a general blueprint for multi-nuclear-spin gates and for multi-party communication.


title: Semi-supervised GAN for Classification of Multispectral Imagery Acquired by UAVs

abstract: Unmanned aerial vehicles (UAV) are used in precision agriculture (PA) to enable aerial monitoring of farmlands. Intelligent methods are required to pinpoint weed infestations and make optimal choice of pesticide. UAV can fly a multispectral camera and collect data. However, the classification of multispectral images using supervised machine learning algorithms such as convolutional neural networks (CNN) requires large amount of training data. This is a common drawback in deep learning we try to circumvent making use of a semi-supervised generative adversarial networks (GAN), providing a pixel-wise classification for all the acquired multispectral images. Our algorithm consists of a generator network that provides photo-realistic images as extra training data to a multi-class classifier, acting as a discriminator and trained on small amounts of labeled data. The performance of the proposed method is evaluated on the weedNet dataset consisting of multispectral crop and weed images collected by a micro aerial vehicle (MAV). The results by the proposed semi-supervised GAN achieves high classification accuracy and demonstrates the potential of GAN-based methods for the challenging task of multispectral image classification.


title: On the Commitment Capacity of Unfair Noisy Channels

abstract: Noisy channels are a valuable resource from a cryptographic point of view. They can be used for exchanging secret-keys as well as realizing other cryptographic primitives such as commitment and oblivious transfer. To be really useful, noisy channels have to be consider in the scenario where a cheating party has some degree of control over the channel characteristics. Damg\r{a}rd et al. (EUROCRYPT 1999) proposed a more realistic model where such level of control is permitted to an adversary, the so called unfair noisy channels, and proved that they can be used to obtain commitment and oblivious transfer protocols. Given that noisy channels are a precious resource for cryptographic purposes, one important question is determining the optimal rate in which they can be used. The commitment capacity has already been determined for the cases of discrete memoryless channels and Gaussian channels. In this work we address the problem of determining the commitment capacity of unfair noisy channels. We compute a single-letter characterization of the commitment capacity of unfair noisy channels. In the case where an adversary has no control over the channel (the fair case) our capacity reduces to the well-known capacity of a discrete memoryless binary symmetric channel.


title: Applying Abstract Argumentation Theory to Cooperative Game Theory

abstract: We apply ideas from abstract argumentation theory to study cooperative game theory. Building on Dung's results in his seminal paper, we further the correspondence between Dung's four argumentation semantics and solution concepts in cooperative game theory by showing that complete extensions (the grounded extension) correspond to Roth's subsolutions (respectively, the supercore). We then investigate the relationship between well-founded argumentation frameworks and convex games, where in each case the semantics (respectively, solution concepts) coincide; we prove that three-player convex games do not in general have well-founded argumentation frameworks.


title: Investigation of HIV-1 Gag binding with RNAs and Lipids using Atomic Force Microscopy

abstract: Atomic Force Microscopy was utilized to study the morphology of Gag, {\Psi}RNA, and their binding complexes with lipids in a solution environment with 0.1{\AA} vertical and 1nm lateral resolution. TARpolyA RNA was used as a RNA control. The lipid used was phospha-tidylinositol-(4,5)-bisphosphate (PI(4,5)P2). The morphology of specific complexes Gag-{\Psi}RNA, Gag-TARpolyA RNA, Gag-PI(4,5)P2 and PI(4,5)P2-{\Psi}RNA-Gag were studied. They were imaged on either positively or negatively charged mica substrates depending on the net charges carried. Gag and its complexes consist of monomers, dimers and tetramers, which was confirmed by gel electrophoresis. The addition of specific {\Psi}RNA to Gag is found to increase Gag multimerization. Non-specific TARpolyA RNA was found not to lead to an increase in Gag multimerization. The addition PI(4,5)P2 to Gag increases Gag multimerization, but to a lesser extent than {\Psi}RNA. When both {\Psi}RNA and PI(4,5)P2 are present Gag undergoes comformational changes and an even higher degree of multimerization.


title: Naive probability

abstract: We describe a rational, but low resolution model of probability.


title: Direct Acyclic Graph based Ledger for Internet of Things: Performance and Security Analysis

abstract: Direct Acyclic Graph (DAG)-based ledger and the corresponding consensus algorithm has been identified as a promising technology for Internet of Things (IoT). Compared with Proof-of-Work (PoW) and Proof-of-Stake (PoS) that have been widely used in blockchain, the consensus mechanism designed on DAG structure (simply called as DAG consensus) can overcome some shortcomings such as high resource consumption, high transaction fee, low transaction throughput and long confirmation delay. However, the theoretic analysis on the DAG consensus is an untapped venue to be explored. To this end, based on one of the most typical DAG consensuses, Tangle, we investigate the impact of network load on the performance and security of the DAG-based ledger. Considering unsteady network load, we first propose a Markov chain model to capture the behavior of DAG consensus process under dynamic load conditions. The key performance metrics, i.e., cumulative weight and confirmation delay are analysed based on the proposed model. Then, we leverage a stochastic model to analyse the probability of a successful double-spending attack in different network load regimes. The results can provide an insightful understanding of DAG consensus process, e.g., how the network load affects the confirmation delay and the probability of a successful attack. Meanwhile, we also demonstrate the trade-off between security level and confirmation delay, which can act as a guidance for practical deployment of DAG-based ledgers.


title: Linear Convergence of Variable Bregman Stochastic Coordinate Descent Method for Nonsmooth Nonconvex Optimization by Level-set Variational Analysis

abstract: Large-scale nonconvex and nonsmooth problems have attracted considerable attention in the fields of compress sensing, big data optimization and machine learning. Exploring effective methods is still the main challenge of today's research. Stochastic coordinate descent type methods have been widely used to solve large-scale optimization problems. In this paper, we derive the convergence of variable Bregman stochastic coordinate descent (VBSCD) method for a broad class of nonsmooth and nonconvex optimization problems, i.e., any accumulation of the sequence generated by VBSCD is almost surely a critical point. Moreover, we develop a new variational approach on level sets that aim towards the convergence rate analysis. If the level-set subdifferential error bound holds, we derive a linear rate of convergence for the expected values of the objective function and expected values of random variables generated by VBSCD.


title: Magnetoresistive RAM for error resilient XNOR-Nets

abstract: We trained three Binarized Convolutional Neural Network architectures (LeNet-4, Network-In-Network, AlexNet) on a variety of datasets (MNIST, CIFAR-10, CIFAR-100, extended SVHN, ImageNet) using error-prone activations and tested them without errors to study the resilience of the training process. With the exception of the AlexNet when trained on the ImageNet dataset, we found that Bit Error Rates of a few percent during training do not degrade the test accuracy. Furthermore, by training the AlexNet on progressively smaller subsets of ImageNet classes, we observed increasing tolerance to activation errors. The ability to operate with high BERs is critical for reducing power consumption in existing hardware and for facilitating emerging memory technologies. We discuss how operating at moderate BER can enable Magnetoresistive RAM with higher endurance, speed and density.


title: Bid Optimization by Multivariable Control in Display Advertising

abstract: Real-Time Bidding (RTB) is an important paradigm in display advertising, where advertisers utilize extended information and algorithms served by Demand Side Platforms (DSPs) to improve advertising performance. A common problem for DSPs is to help advertisers gain as much value as possible with budget constraints. However, advertisers would routinely add certain key performance indicator (KPI) constraints that the advertising campaign must meet due to practical reasons. In this paper, we study the common case where advertisers aim to maximize the quantity of conversions, and set cost-per-click (CPC) as a KPI constraint. We convert such a problem into a linear programming problem and leverage the primal-dual method to derive the optimal bidding strategy. To address the applicability issue, we propose a feedback control-based solution and devise the multivariable control system. The empirical study based on real-word data from Taobao.com verifies the effectiveness and superiority of our approach compared with the state of the art in the industry practices.


title: Critical spin fluctuation mechanism for the spin Hall effect

abstract: We propose mechanisms for the spin Hall effect in metallic systems arising from the coupling between conduction electrons and local magnetic moments that are dynamically fluctuating. Both a side-jump-type mechanism and a skew-scattering-type mechanism are considered. In either case, dynamical spin fluctuation gives rise to a nontrivial temperature dependence in the spin Hall conductivity. This leads to the enhancement in the spin Hall conductivity at nonzero temperatures near the ferromagnetic instability. The proposed mechanisms could be observed in $4d$ or $5d$ metallic compounds.


title: Sequential Graph Dependency Parser

abstract: We propose a method for non-projective dependency parsing by incrementally predicting a set of edges. Since the edges do not have a pre-specified order, we propose a set-based learning method. Our method blends graph, transition, and easy-first parsing, including a prior state of the parser as a special case. The proposed transition-based method successfully parses near the state of the art on both projective and non-projective languages, without assuming a certain parsing order.


title: Holographic formulation of 3D metric gravity with finite boundaries

abstract: In this work we construct holographic boundary theories for linearized 3D gravity, for a general family of finite or quasi-local boundaries. These boundary theories are directly derived from the dynamics of 3D gravity by computing the effective action for a geometric boundary observable, which measures the geodesic length from a given boundary point to some centre in the bulk manifold. We identify the general form for these boundary theories and find that these are Liouville like with a coupling to the boundary Ricci scalar. This is illustrated with various examples, which each offer interesting insights into the structure of holographic boundary theories.


title: Nanoscale spatially resolved mapping of uranium enrichment in actinide-bearing materials

abstract: Spatially resolved analysis of uranium isotopes in small volumes of actinide-bearing materials is critical for a variety of technical disciplines, including earth and planetary sciences, environmental monitoring, bioremediation, and the nuclear fuel cycle. However, achieving sub-nanometer scale spatial resolution for such isotopic analysis is currently a challenge. By using atom probe tomography, a three dimensional nanoscale characterization technique, we demonstrate unprecidented nanoscale mapping of uranium isotopic enrichment with high sensitivity across various microstructural interfaces within small volumes (100 nm3) of depleted and low enriched uranium alloyed with 10 wt % molybdenum with different nominal enrichments of 0.20 and 19.75% 235U respectively. The approach presented here can be applied to study nanoscale variations of isotopic abundances in the broad class of actinide-bearing materials, providing unique insights into their origin and thermo-mechanical processing routes.


title: System-theoretic Analysis of Nonlinear Infinite-dimensional Systems with Generalized Symmetries

abstract: Symmetry groups allow to transform solutions of differential equations continuously into other solutions. This property can be used for the observability analysis of infinite-dimensional systems with input and output. In this contribution, we extend an existing approach, which is based on vertical classical symmetries, to non-vertical classical symmetries and generalized symmetries. This makes the approach applicable to a much larger class of systems, since many nonlinear systems do not possess any vertical classical symmetries.


title: HVAC Energy Cost Optimization for a Multi-zone Building via a Decentralized Approach

abstract: It has been well acknowledged that buildings account for a large proportion of the world's energy consumption. However, the energy use of buildings, especially the heating, ventilation and air-conditioning (HVAC), is far from being efficient. There still exists a dramatic potential to save energy through improving building energy efficiency. Therefore, this paper studies the control of HVAC system for multi-zone buildings with the objective to reduce energy consumption cost while satisfying thermal comfort. In particular, the thermal couplings due to the heat transfer between the adjacent zones are incorporated in the optimization. Considering that a centralized method is generally computationally prohibitive for large buildings, an efficient decentralized approach is developed, based on the Accelerated Distributed Augmented Lagrangian (ADAL) method [1]. To evaluate the performance of the proposed method, we first compare it with a centralized method, in which the optimal solution of a small-scale problem can be obtained. We find that this decentralized approach can almost approach the optimal solution of the problem. Further, this decentralized approach is compared with the Distributed Token-Based Scheduling Strategy (DTBSS) [2]. The numeric results reveal that when the number of zones is relatively small (less than 20), the two decentralized methods can achieve a comparable performance regarding the cost of the HVAC system. However, with an increase of the number of zones in the building, the proposed decentralized approach demonstrates better performance with a considerable reduction of the total cost. Moreover, the decentralized approach proposed in this paper demonstrate better scalability with less average computation required.


title: Open quantum systems are harder to track than open classical systems

abstract: For a Markovian open quantum system it is possible, by continuously monitoring the environment, to know the stochastically evolving pure state of the system without altering the master equation. In general, even for a system with a finite Hilbert space dimension $D$, the pure state trajectory will explore an infinite number of points in Hilbert space, meaning that the dimension $K$ of the classical memory required for the tracking is infinite. However, Karasik and Wiseman [Phys. Rev. Lett., 106(2):020406, 2011] showed that tracking of a qubit ($D=2$) is always possible with a bit ($K=2$), and gave a heuristic argument implying that a finite $K$ should be sufficient for any $D$, although beyond $D=2$ it would be necessary to have $K>D$. Our paper is concerned with rigorously investigating the relationship between $D$ and $K_{\rm min}$, the smallest feasible $K$. We confirm the long-standing conjecture of Karasik and Wiseman that, for generic systems with $D>2$, $K_{\rm min}>D$, by a computational proof (via Hilbert Nullstellensatz certificates of infeasibility). That is, beyond $D=2$, $D$-dimensional open quantum systems are provably harder to track than $D$-dimensional open classical systems. Moreover, we develop, and better justify, a new heuristic to guide our expectation of $K_{\rm min}$ as a function of $D$, taking into account the number $L$ of Lindblad operators as well as symmetries in the problem. The use of invariant subspace and Wigner symmetries makes it tractable to conduct a numerical search, using the method of polynomial homotopy continuation, to find finite physically realizable ensembles (as they are known) in $D=3$. The results of this search support our heuristic. We thus have confidence in the most interesting feature of our heuristic: in the absence of symmetries, $K_{\rm min} \sim D^2$, implying a quadratic gap between the classical and quantum tracking problems.


title: Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback

abstract: Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of distributed neural network training. Using 1-bit quantization, signSGD with majority vote achieves a 32x reduction on communication cost. However, its convergence is based on unrealistic assumptions and can diverge in practice. In this paper, we propose a general distributed compressed SGD with Nesterov's momentum. We consider two-way compression, which compresses the gradients both to and from workers. Convergence analysis on nonconvex problems for general gradient compressors is provided. By partitioning the gradient into blocks, a blockwise compressor is introduced such that each gradient block is compressed and transmitted in 1-bit format with a scaling factor, leading to a nearly 32x reduction on communication. Experimental results show that the proposed method converges as fast as full-precision distributed momentum SGD and achieves the same testing accuracy. In particular, on distributed ResNet training with 7 workers on the ImageNet, the proposed algorithm achieves the same testing accuracy as momentum SGD using full-precision gradients, but with $46\%$ less wall clock time.


title: Adiabatic and high-fidelity quantum gates with hybrid Rydberg-Rydberg interactions

abstract: Rydberg blockaded gate is a fundamental ingredient for scalable quantum computation with neutral Rydberg atoms. However the fidelity of such a gate is intrinsically limited by a blockade error coming from a Rydberg level shift that forbids its extensive use. Based on a dark-state adiabatic passage, we develop a novel protocol for realizing a two-atom blockade-error-free quantum gate in a hybrid system with simultaneous van der Waals (vdWsI) and resonant dipole-dipole interactions (DDI). The basic idea relies on converting the roles of two interactions, which is, the DDI serves as one time-dependent tunable pulse and the vdWsI acts as a negligible middle level shift as long as the adiabatic condition is preserved. We adopt an optimized super-Gaussian optical pulse with $k\pi$ ($k\gg 1$) area accompanied by a smooth tuning for the DDI, composing a circular stimulated Raman adiabatic passage, which can robustly ensure a faster operation time $\sim 80ns$ as well as a highly-efficient gate fidelity $\sim0.9996$. This theoretical protocol offers a flexible treatment for hybrid interactions in complex Rydberg systems, enabling on-demand design of new types of effective Rydberg quantum gate devices.


title: No homotopy 4-sphere invariants using ECH = SWF

abstract: In relation to the 4-dimensional smooth Poincar\'e conjecture we construct a tentative invariant of homotopy 4-spheres using embedded contact homology (ECH) and Seiberg-Witten theory (SWF). But for good reason it is a constant value independent of the sphere, so this null-result demonstrates that one should not try to use the usual theories of ECH and SWF. On the other hand, a corollary is that there always exist pseudoholomorphic curves satisfying certain constraints in (punctured) 4-spheres.


title: PNUNet: Anomaly Detection using Positive-and-Negative Noise based on Self-Training Procedure

abstract: We propose the novel framework for anomaly detection in images. Our new framework, PNUNet, is based on many normal data and few anomalous data. We assume that some noises are added to the input images and learn to remove the noise. In addition, the proposed method achieves significant performance improvement by updating the noise assumed in the inputs using a self-training framework. The experimental results for the benchmark datasets show the usefulness of our new anomaly detection framework.


title: A Practical Spectrum Sharing Scheme for Cognitive Radio Networks: Design and Experiments

abstract: Spectrum shortage is a fundamental problem in wireless networks and this problem becomes increasingly acute with the rapid proliferation of wireless devices. To address this problem, spectrum sharing in the context of cognitive radio networks (CRNs) has been considered a promising solution. In this paper, we propose a practical spectrum sharing scheme for a small CRN that comprises a pair of primary users and a pair of secondary users by leveraging the multiple-input and multiple-output (MIMO) technology. In our scheme, we assume that the secondary users take full responsibility for cross-network interference cancellation (IC). We also assume that the secondary users have no knowledge about the primary network, including its signal waveform, frame structure, and network protocol. The key components of our proposed scheme are two MIMO-based interference management techniques: blind beamforming (BBF) and blind interference cancellation (BIC). We have built a prototype of our scheme on a wireless testbed and demonstrated that the prototyped secondary network can coexist with commercial Wi-Fi devices (primary users). Experimental results further show that, for a secondary device with two or three antennas, BBF and BIC achieve an average of 25dB and 33dB IC capability in an office environment, respectively.


title: Non-Markovian Noise Characterization with the Transfer Tensor Method

abstract: We propose simple protocols for performing quantum noise spectroscopy based on the method of transfer tensor maps (TTM), [Phys. Rev. Lett. 112, 110401 (2014)]. The TTM approach is a systematic way to deduce the memory kernel of a time-nonlocal quantum master equation via quantum process tomography. With access to the memory kernel it is possible to (1) assess the non-Markovianity of a quantum process, (2) reconstruct the noise spectral density beyond pure dephasing models, and (3) investigate collective decoherence in multiqubit devices. We illustrate the usefulness of TTM spectroscopy on the IBM Quantum Experience platform, and demonstrate that the qubits in the IBM device are subject to mild non-Markovian dissipation with spatial correlations.


title: Noncommutative LR coefficients and crystal reflection operators

abstract: We relate noncommutative Littlewood-Richardson coefficients of Bessenrodt-Luoto-van Willigenburg to classical Littlewood-Richardson coefficients via crystal reflection operators. A key role is played by the combinatorics of frank words.


title: Distributionally Robust Optimization and Generalization in Kernel Methods

abstract: Distributionally robust optimization (DRO) has attracted attention in machine learning due to its connections to regularization, generalization, and robustness. Existing work has considered uncertainty sets based on phi-divergences and Wasserstein distances, each of which have drawbacks. In this paper, we study DRO with uncertainty sets measured via maximum mean discrepancy (MMD). We show that MMD DRO is roughly equivalent to regularization by the Hilbert norm and, as a byproduct, reveal deep connections to classic results in statistical learning. In particular, we obtain an alternative proof of a generalization bound for Gaussian kernel ridge regression via a DRO lense. The proof also suggests a new regularizer. Our results apply beyond kernel methods: we derive a generically applicable approximation of MMD DRO, and show that it generalizes recent work on variance-based regularization.


title: Identity Connections in Residual Nets Improve Noise Stability

abstract: Residual Neural Networks (ResNets) achieve state-of-the-art performance in many computer vision problems. Compared to plain networks without residual connections (PlnNets), ResNets train faster, generalize better, and suffer less from the so-called degradation problem. We introduce simplified (but still nonlinear) versions of ResNets and PlnNets for which these discrepancies still hold, although to a lesser degree. We establish a 1-1 mapping between simplified ResNets and simplified PlnNets, and show that they are exactly equivalent to each other in expressive power for the same computational complexity. We conjecture that ResNets generalize better because they have better noise stability, and empirically support it for both simplified and fully-fledged networks.


title: Learning with Succinct Common Representation Based on Wyner's Common Information

abstract: A new bimodal generative model is proposed for generating conditional and joint samples, accompanied with a training method with learning a succinct bottleneck representation. The proposed model, dubbed as the variational Wyner model, is designed based on two classical problems in network information theory -- distributed simulation and channel synthesis -- in which Wyner's common information arises as the fundamental limit on the succinctness of the common representation. The model is trained by minimizing the symmetric Kullback--Leibler divergence between variational and model distributions with regularization terms for common information, reconstruction consistency, and latent space matching terms, which is carried out via an adversarial density ratio estimation technique. The utility of the proposed approach is demonstrated through experiments for joint and conditional generation with synthetic and real-world datasets, as well as a challenging zero-shot image retrieval task.


title: Weighted estimates for bilinear fractional integral operators and their commutators on Morrey spaces

abstract: This paper mainly dedicates to prove a plethora of weighted estimates on Morrey spaces for bilinear fractional integral operators and their general commutators with BMO functions of the form $$B_{\alpha}(f,g)(x)=\int_{\mathbb{R}^{n}}\frac{f(x-y)g(x+y)}{|y|^{n-\alpha}}dy,\qquad 0<\alpha<n.$$ We also prove some maximal function control theorems for these operators, that is, the weighted Morrey norm is bounded by the weighted Morrey norm of a natural maximal operator when the weight belongs to $A_{\infty}$. As a corollary, some new weighted estimates for the bilinear maximal function associated to the bilinear Hilbert transform are obtained. Furthermore, we formulate a bilinear version of Stein-Weiss inequality on Morrey spaces for fractional integrals.


title: Graph Neural Networks Exponentially Lose Expressive Power for Node Classification

abstract: Graph Neural Networks (graph NNs) are a promising deep learning approach for analyzing graph-structured data. However, it is known that they do not improve (or sometimes worsen) their predictive performance as we pile up many layers and add non-lineality. To tackle this problem, we investigate the expressive power of graph NNs via their asymptotic behaviors as the layer size tends to infinity. Our strategy is to generalize the forward propagation of a Graph Convolutional Network (GCN), which is a popular graph NN variant, as a specific dynamical system. In the case of a GCN, we show that when its weights satisfy the conditions determined by the spectra of the (augmented) normalized Laplacian, its output exponentially approaches the set of signals that carry information of the connected components and node degrees only for distinguishing nodes. Our theory enables us to relate the expressive power of GCNs with the topological information of the underlying graphs inherent in the graph spectra. To demonstrate this, we characterize the asymptotic behavior of GCNs on the Erd\H{o}s -- R\'{e}nyi graph. We show that when the Erd\H{o}s -- R\'{e}nyi graph is sufficiently dense and large, a broad range of GCNs on it suffers from the "information loss" in the limit of infinite layers with high probability. Based on the theory, we provide a principled guideline for weight normalization of graph NNs. We experimentally confirm that the proposed weight scaling enhances the predictive performance of GCNs in real data. Code is available at https://github.com/delta2323/gnn-asymptotics.


title: Provably Efficient Imitation Learning from Observation Alone

abstract: We study Imitation Learning (IL) from Observations alone (ILFO) in large-scale MDPs. While most IL algorithms rely on an expert to directly provide actions to the learner, in this setting the expert only supplies sequences of observations. We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner. FAIL is the first provably efficient algorithm in ILFO setting, which learns a near-optimal policy with a number of samples that is polynomial in all relevant parameters but independent of the number of unique observations. The resulting theory extends the domain of provably sample efficient learning algorithms beyond existing results, which typically only consider tabular reinforcement learning settings or settings that require access to a near-optimal reset distribution. We also investigate the extension of FAIL in a model-based setting. Finally we demonstrate the efficacy of FAIL on multiple OpenAI Gym control tasks.


title: QuesNet: A Unified Representation for Heterogeneous Test Questions

abstract: Understanding learning materials (e.g. test questions) is a crucial issue in online learning systems, which can promote many applications in education domain. Unfortunately, many supervised approaches suffer from the problem of scarce human labeled data, whereas abundant unlabeled resources are highly underutilized. To alleviate this problem, an effective solution is to use pre-trained representations for question understanding. However, existing pre-training methods in NLP area are infeasible to learn test question representations due to several domain-specific characteristics in education. First, questions usually comprise of heterogeneous data including content text, images and side information. Second, there exists both basic linguistic information as well as domain logic and knowledge. To this end, in this paper, we propose a novel pre-training method, namely QuesNet, for comprehensively learning question representations. Specifically, we first design a unified framework to aggregate question information with its heterogeneous inputs into a comprehensive vector. Then we propose a two-level hierarchical pre-training algorithm to learn better understanding of test questions in an unsupervised way. Here, a novel holed language model objective is developed to extract low-level linguistic features, and a domain-oriented objective is proposed to learn high-level logic and knowledge. Moreover, we show that QuesNet has good capability of being fine-tuned in many question-based tasks. We conduct extensive experiments on large-scale real-world question data, where the experimental results clearly demonstrate the effectiveness of QuesNet for question understanding as well as its superior applicability.


title: Hyperbolic field space and swampland conjecture for DBI scalar

abstract: We study a model of two scalar fields with a hyperbolic field space and show that it reduces to a single-field Dirac-Born-Infeld (DBI) model in the limit where the field space becomes infinitely curved. We apply the de Sitter swampland conjecture to the two-field model and take the same limit. It is shown that in the limit, all quantities appearing in the swampland conjecture remain well-defined within the single-field DBI model. Based on a consistency argument, we then speculate that the condition derived in this way can be considered as the de Sitter swampland conjecture for a DBI scalar field by its own. The condition differs from those proposed in the literature and only the one in the present paper passes the consistency argument. As a byproduct, we also point out that one of the inequalities in the swampland conjecture for a multi-field model with linear kinetic terms should involve the lowest mass squared for scalar perturbations and that this quantity can be significantly different from the lowest eigenvalue of the Hessian of the potential in the local orthonormal frame if the field space is highly curved. Finally, we propose an extension of the de Sitter swampland conjecture to a more general scalar field with the Lagrangian of the form $P(X,\varphi)$, where $X=-(\partial\varphi)^2/2$.


title: On the Evaluation Metric for Hashing

abstract: Due to its low storage cost and fast query speed, hashing has been widely used for large-scale approximate nearest neighbor (ANN) search. Bucket search, also called hash lookup, can achieve fast query speed with a sub-linear time cost based on the inverted index table constructed from hash codes. Many metrics have been adopted to evaluate hashing algorithms. However, all existing metrics are improper to evaluate the hash codes for bucket search. On one hand, all existing metrics ignore the retrieval time cost which is an important factor reflecting the performance of search. On the other hand, some of them, such as mean average precision (MAP), suffer from the uncertainty problem as the ranked list is based on integer-valued Hamming distance, and are insensitive to Hamming radius as these metrics only depend on relative Hamming distance. Other metrics, such as precision at Hamming radius R, fail to evaluate global performance as these metrics only depend on one specific Hamming radius. In this paper, we first point out the problems of existing metrics which have been ignored by the hashing community, and then propose a novel evaluation metric called radius aware mean average precision (RAMAP) to evaluate hash codes for bucket search. Furthermore, two coding strategies are also proposed to qualitatively show the problems of existing metrics. Experiments demonstrate that our proposed RAMAP can provide more proper evaluation than existing metrics.


title: Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networks

abstract: Deep neural networks (DNNs) have become a widely deployed model for numerous machine learning applications. However, their fixed architecture, substantial training cost, and significant model redundancy make it difficult to efficiently update them to accommodate previously unseen data. To solve these problems, we propose an incremental learning framework based on a grow-and-prune neural network synthesis paradigm. When new data arrive, the neural network first grows new connections based on the gradients to increase the network capacity to accommodate new data. Then, the framework iteratively prunes away connections based on the magnitude of weights to enhance network compactness, and hence recover efficiency. Finally, the model rests at a lightweight DNN that is both ready for inference and suitable for future grow-and-prune updates. The proposed framework improves accuracy, shrinks network size, and significantly reduces the additional training cost for incoming data compared to conventional approaches, such as training from scratch and network fine-tuning. For the LeNet-300-100 and LeNet-5 neural network architectures derived for the MNIST dataset, the framework reduces training cost by up to 64% (63%) and 67% (63%) compared to training from scratch (network fine-tuning), respectively. For the ResNet-18 architecture derived for the ImageNet dataset and DeepSpeech2 for the AN4 dataset, the corresponding training cost reductions against training from scratch (network fine-tunning) are 64% (60%) and 67% (62%), respectively. Our derived models contain fewer network parameters but achieve higher accuracy relative to conventional baselines.


title: FOBE and HOBE: First- and High-Order Bipartite Embeddings

abstract: Typical graph embeddings may not capture type-specific bipartite graph features that arise in such areas as recommender systems, data visualization, and drug discovery. Machine learning methods utilized in these applications would be better served with specialized embedding techniques. We propose two embeddings for bipartite graphs that decompose edges into sets of indirect relationships between node neighborhoods. When sampling higher-order relationships, we reinforce similarities through algebraic distance on graphs. We also introduce ensemble embeddings to combine both into a "best of both worlds" embedding. The proposed methods are evaluated on link prediction and recommendation tasks and compared with other state-of-the-art embeddings. While being all highly beneficial in applications, we demonstrate that none of the considered embeddings is clearly superior (in contrast to what is claimed in many papers), and discuss the trade offs present among them. Reproducibility: Our code, data sets, and results are all publicly available online at: http://sybrandt.com/2020/fobe_hobe.


title: Transcribing Content from Structural Images with Spotlight Mechanism

abstract: Transcribing content from structural images, e.g., writing notes from music scores, is a challenging task as not only the content objects should be recognized, but the internal structure should also be preserved. Existing image recognition methods mainly work on images with simple content (e.g., text lines with characters), but are not capable to identify ones with more complex content (e.g., structured symbols), which often follow a fine-grained grammar. To this end, in this paper, we propose a hierarchical Spotlight Transcribing Network (STN) framework followed by a two-stage "where-to-what" solution. Specifically, we first decide "where-to-look" through a novel spotlight mechanism to focus on different areas of the original image following its structure. Then, we decide "what-to-write" by developing a GRU based network with the spotlight areas for transcribing the content accordingly. Moreover, we propose two implementations on the basis of STN, i.e., STNM and STNR, where the spotlight movement follows the Markov property and Recurrent modeling, respectively. We also design a reinforcement method to refine the framework by self-improving the spotlight mechanism. We conduct extensive experiments on many structural image datasets, where the results clearly demonstrate the effectiveness of STN framework.


title: Dynamically Visual Disambiguation of Keyword-based Image Search

abstract: Due to the high cost of manual annotation, learning directly from the web has attracted broad attention. One issue that limits their performance is the problem of visual polysemy. To address this issue, we present an adaptive multi-model framework that resolves polysemy by visual disambiguation. Compared to existing methods, the primary advantage of our approach lies in that our approach can adapt to the dynamic changes in the search results. Our proposed framework consists of two major steps: we first discover and dynamically select the text queries according to the image search results, then we employ the proposed saliency-guided deep multi-instance learning network to remove outliers and learn classification models for visual disambiguation. Extensive experiments demonstrate the superiority of our proposed approach.


title: Search of QCD phase transition points in the canonical approach of the NJL model

abstract: We study the Lee-Yang zeros in the canonical approach to search phase transition points at finite temperature and density in the Nambu-Jona-Lasinio (NJL) model as an effective model of QCD. The canonical approach is a promising method to avoid the sign problem in lattice QCD at finite density. We find that a set of Lee-Yang zeros computed with finite degrees of freedom can be extrapolated to those with infinite degrees of freedom, providing the correct phase transition point. We propose the present method as a useful method for actual lattice simulations for QCD.


title: Enhancing Item Response Theory for Cognitive Diagnosis

abstract: Cognitive diagnosis is a fundamental and crucial task in many educational applications, e.g., computer adaptive test and cognitive assignments. Item Response Theory (IRT) is a classical cognitive diagnosis method which can provide interpretable parameters (i.e., student latent trait, question discrimination, and difficulty) for analyzing student performance. However, traditional IRT ignores the rich information in question texts, cannot diagnose knowledge concept proficiency, and it is inaccurate to diagnose the parameters for the questions which only appear several times. To this end, in this paper, we propose a general Deep Item Response Theory (DIRT) framework to enhance traditional IRT for cognitive diagnosis by exploiting semantic representation from question texts with deep learning. In DIRT, we first use a proficiency vector to represent students' proficiency in knowledge concepts and embed question texts and knowledge concepts to dense vectors by Word2Vec. Then, we design a deep diagnosis module to diagnose parameters in traditional IRT by deep learning techniques. Finally, with the diagnosed parameters, we input them into the logistic-like formula of IRT to predict student performance. Extensive experimental results on real-world data clearly demonstrate the effectiveness and interpretation power of DIRT framework.


title: Explainable Reinforcement Learning Through a Causal Lens

abstract: Prevalent theories in cognitive science propose that humans understand and represent the knowledge of the world through causal relationships. In making sense of the world, we build causal models in our mind to encode cause-effect relations of events and use these to explain why new events happen. In this paper, we use causal models to derive causal explanations of behaviour of reinforcement learning agents. We present an approach that learns a structural causal model during reinforcement learning and encodes causal relationships between variables of interest. This model is then used to generate explanations of behaviour based on counterfactual analysis of the causal model. We report on a study with 120 participants who observe agents playing a real-time strategy game (Starcraft II) and then receive explanations of the agents' behaviour. We investigated: 1) participants' understanding gained by explanations through task prediction; 2) explanation satisfaction and 3) trust. Our results show that causal model explanations perform better on these measures compared to two other baseline explanation models.


title: Computer-aided Detection of Squamous Carcinoma of the Cervix in Whole Slide Images

abstract: Goal: Squamous cell carcinoma of cervix is one of the most prevalent cancer worldwide in females. Traditionally, the most indispensable diagnosis of cervix squamous carcinoma is histopathological assessment which is achieved under microscope by pathologist. However, human evaluation of pathology slide is highly depending on the experience of pathologist, thus big inter- and intra-observer variability exists. Digital pathology, in combination with deep learning provides an opportunity to improve the objectivity and efficiency of histopathologic slide analysis. Methods: In this study, we obtained 800 haematoxylin and eosin stained slides from 300 patients suffered from cervix squamous carcinoma. Based on information from morphological heterogeneity in the tumor and its adjacent area, we established deep learning models using popular convolution neural network architectures (inception-v3, InceptionResnet-v2 and Resnet50). Then random forest was introduced to feature extractions and slide-based classification. Results: The overall performance of our proposed models on slide-based tumor discrimination were outstanding with an AUC scores > 0.94. While, location identifications of lesions in whole slide images were mediocre (FROC scores > 0.52) duo to the extreme complexity of tumor tissues. Conclusion: For the first time, our analysis workflow highlighted a quantitative visual-based slide analysis of cervix squamous carcinoma. Significance: This study demonstrates a pathway to assist pathologist and accelerate the diagnosis of patients by utilizing new computational approaches.


title: TrendNets: Mapping Emerging Research Trends From Dynamic Co-Word Networks via Sparse Representation

abstract: Mapping the knowledge structure from word co-occurrences in a collection of academic papers has been widely used to provide insight into the topic evolution in an arbitrary research field. In a traditional approach, the paper collection is first divided into temporal subsets, and then a co-word network is independently depicted in a 2D map to characterize each period's trend. To effectively map emerging research trends from such a time-series of co-word networks, this paper presents TrendNets, a novel visualization methodology that highlights the rapid changes in edge weights over time. Specifically, we formulated a new convex optimization framework that decomposes the matrix constructed from dynamic co-word networks into a smooth part and a sparse part: the former represents stationary research topics, while the latter corresponds to bursty research topics. Simulation results on synthetic data demonstrated that our matrix decomposition approach achieved the best burst detection performance over four baseline methods. In experiments conducted using papers published in the past 16 years at three conferences in different fields, we showed the effectiveness of TrendNets compared to the traditional co-word representation. We have made our codes available on the Web to encourage scientific mapping in all research fields.


title: Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks

abstract: Natural gradient descent has proven effective at mitigating the effects of pathological curvature in neural network optimization, but little is known theoretically about its convergence properties, especially for \emph{nonlinear} networks. In this work, we analyze for the first time the speed of convergence of natural gradient descent on nonlinear neural networks with squared-error loss. We identify two conditions which guarantee efficient convergence from random initializations: (1) the Jacobian matrix (of network's output for all training cases with respect to the parameters) has full row rank, and (2) the Jacobian matrix is stable for small perturbations around the initialization. For two-layer ReLU neural networks, we prove that these two conditions do in fact hold throughout the training, under the assumptions of nondegenerate inputs and overparameterization. We further extend our analysis to more general loss functions. Lastly, we show that K-FAC, an approximate natural gradient descent method, also converges to global minima under the same assumptions, and we give a bound on the rate of this convergence.


title: Development of a method for measuring rare earth elements in the environment for future experiments with gadolinium loaded detectors

abstract: Demand to use gadolinium (Gd) in detectors is increasing in the field of elementary particle physics, especially neutrino measurements and dark matter searches. Large amounts of Gd are used in these experiments. Therefore, to access the impacts of Gd onto the environments, it is becoming important to measure the baseline concentrations of Gd in the environments. The measurement of the baseline concentrations, however, is not easy due to interferences by other elements. In this paper, a method for measuring the concentrations of rare earth elements including Gd is proposed. In the method, an inductively coupled plasma-mass spectrometry is utilized after collecting the dissolved elements in chelating resin. Results of the ability to detect anomalous concentrations of rare earth elements in river water samples in the Kamioka and Toyama areas are also reported.


title: Interpretable deep Gaussian processes with moments

abstract: Deep Gaussian Processes (DGPs) combine the expressiveness of Deep Neural Networks (DNNs) with quantified uncertainty of Gaussian Processes (GPs). Expressive power and intractable inference both result from the non-Gaussian distribution over composition functions. We propose interpretable DGP based on approximating DGP as a GP by calculating the exact moments, which additionally identify the heavy-tailed nature of some DGP distributions. Consequently, our approach admits interpretation as both NNs with specified activation functions and as a variational approximation to DGP. We identify the expressivity parameter of DGP and find non-local and non-stationary correlation from DGP composition. We provide general recipes for deriving the effective kernels for DGP of two, three, or infinitely many layers, composed of homogeneous or heterogeneous kernels. Results illustrate the expressiveness of our effective kernels through samples from the prior and inference on simulated and real data and demonstrate advantages of interpretability by analysis of analytic forms, and draw relations and equivalences across kernels.


title: Combating Label Noise in Deep Learning Using Abstention

abstract: We introduce a novel method to combat label noise when training deep neural networks for classification. We propose a loss function that permits abstention during training thereby allowing the DNN to abstain on confusing samples while continuing to learn and improve classification performance on the non-abstained samples. We show how such a deep abstaining classifier (DAC) can be used for robust learning in the presence of different types of label noise. In the case of structured or systematic label noise -- where noisy training labels or confusing examples are correlated with underlying features of the data-- training with abstention enables representation learning for features that are associated with unreliable labels. In the case of unstructured (arbitrary) label noise, abstention during training enables the DAC to be used as an effective data cleaner by identifying samples that are likely to have label noise. We provide analytical results on the loss function behavior that enable dynamic adaption of abstention rates based on learning progress during training. We demonstrate the utility of the deep abstaining classifier for various image classification tasks under different types of label noise; in the case of arbitrary label noise, we show significant improvements over previously published results on multiple image benchmarks. Source code is available at https://github.com/thulas/dac-label-noise


title: A closed-form formula for the Kullback-Leibler divergence between Cauchy distributions

abstract: We report a closed-form expression for the Kullback-Leibler divergence between Cauchy distributions which involves the calculation of a novel definite integral. The formula shows that the Kullback-Leibler divergence between Cauchy densities is always finite and symmetric.


title: Using $p$-row graphs to study $p$-competition graphs

abstract: For a positive integer $p$, the $p$-competition graph of a digraph $D$ is a graph which has the same vertex set as $D$ and an edge between distinct vertices $x$ and $y$ if and only if $x$ and $y$ have at least $p$ common out-neighbors in $D$. A graph is said to be a $p$-competition graph if it is the $p$-competition graph of a digraph. Given a graph $G$, we call the set of positive integers $p$ such that $G$ is a $p$-competition the competition-realizer of a graph $G$. In this paper, we introduce the notion of $p$-row graph of a matrix which generalizes the existing notion of row graph. We call the graph obtained from a graph $G$ by identifying each pair of adjacent vertices which share the same closed neighborhood the condensation of $G$. Using the notions of $p$-row graph and condensation of a graph, we study competition-realizers for various graphs to extend results given by Kim {\it et al.}~[$p$-competition graphs, {\it Linear Algebra Appl.} {\bf 217} (1995) 167--178]. Especially, we find all the elements in the competition-realizer for each caterpillar.


title: Soft phonons in the interface layer of the STO substrate can explain high temperature superconductivity in one unit cell FeSe

abstract: Using a microscopic model of lattice vibrations in the STO(001) substrate, an additional 50 mev longitudinal optical (LO) interface mode is identified. The soft mode propagating mainly in the first TiO2 layer ("O chains") has stronger electron - phonon coupling to electron gas in FeSe than a well known 100 mev hard mode. The coupling constant, critical temperature, replica band are calculated. Although there exists a forward in the electron - phonon scattering peak, it is clearly not as sharp as assumed in recent theories (delta function - like). The satellite is broad and its peak appears at frequency much higher than the phonon frequency consistent with observations. Possible relation of the transverse counterpart of the surface LO soft mode with known phonons is discussed.


title: Quantum Robots Can Fly; Play Games: An IBM Quantum Experience

abstract: Quantum Robot is an excellent future application that can be achieved with the help of a quantum computer. As a practical example, quantum controlled Braitenberg vehicles proposed by Raghuvanshi et al. [Proceedings of the 37th International Symposium on Multiple-Valued Logic (2007)] is a mobile quantum system and hence acts as a quantum robot. Braitenberg vehicles are simple circuit robots which can experience natural behaviours like fear, aggression and love etc. These robots can be controlled by quantum circuits incorporating quantum principles such as entanglement and superposition. Complex behaviours can be mimicked by a quantum circuit that can be implemented in a quantum robot. Here we investigate the scheme of Raghuvanshi et al. and propose a new quantum circuit to make the quantum robot fly. We demonstrate one of its application in playing a game. The quantum robot we present here shows the behaviour of `fear' and its movement is deterministic in nature. This phenomenon can be successfully modelled in a game, where it can always avoid accident. The proposed quantum circuit is designed in IBM quantum experience describing the above protocol.


title: Scalable Training of Inference Networks for Gaussian-Process Models

abstract: Inference in Gaussian process (GP) models is computationally challenging for large data, and often difficult to approximate with a small number of inducing points. We explore an alternative approximation that employs stochastic inference networks for a flexible inference. Unfortunately, for such networks, minibatch training is difficult to be able to learn meaningful correlations over function outputs for a large dataset. We propose an algorithm that enables such training by tracking a stochastic, functional mirror-descent algorithm. At each iteration, this only requires considering a finite number of input locations, resulting in a scalable and easy-to-implement algorithm. Empirical results show comparable and, sometimes, superior performance to existing sparse variational GP methods.


title: Compact fiber optical interferometer technique to measure picometer displacements in biological piezoelectric materials

abstract: A simple and robust fiber optical interferometer was developed to non-invasively study the weak piezoelectric effect from thin samples. A biological sample from inter-molt dactyl clubs obtained from the mantis shrimp was used as the test sample. The non-contact technique can measure displacements better than 0.5 picometer for samples subjected to large electric fields. The approach utilizes the phase dependent detection of an oscillating cavity at different frequencies from 0.5 kHz to 2.0 kHz. The piezoelectric constant of the biological samples was calculated from the optical interference fringes and determined to be in the range of 0.3-0.5 pm/V. The noise of 20 fm/Hz^0.5 in the setup is primarily due to thermally associated strains from current flow to the sample electrodes during the measurement.


title: An Empirical Study on Post-processing Methods for Word Embeddings

abstract: Word embeddings learnt from large corpora have been adopted in various applications in natural language processing and served as the general input representations to learning systems. Recently, a series of post-processing methods have been proposed to boost the performance of word embeddings on similarity comparison and analogy retrieval tasks, and some have been adapted to compose sentence representations. The general hypothesis behind these methods is that by enforcing the embedding space to be more isotropic, the similarity between words can be better expressed. We view these methods as an approach to shrink the covariance/gram matrix, which is estimated by learning word vectors, towards a scaled identity matrix. By optimising an objective in the semi-Riemannian manifold with Centralised Kernel Alignment (CKA), we are able to search for the optimal shrinkage parameter, and provide a post-processing method to smooth the spectrum of learnt word vectors which yields improved performance on downstream tasks.


title: Massive primordial black holes in contemporary universe

abstract: The parameters of the original log-normal mass spectrum of primordial black holes (PBH) are approximately adjusted on the basis of existing observational data on supermassive black holes in the galactic centers and the mass distribution of the near-solar mass black holes in the Galaxy. Together with the assumption that PBHs make all or a noticeable mass fraction of the cosmological dark matter, it allows to fix the parameters of the original mass spectrum. The predicted, in this way, the number density of MACHOs is found to be about an order of magnitude below the observed value. A possible resolution of this controversy may be prescribed to the non-isotropic and inhomogeneous distribution of MACHOs or to the modification of the original spectrum, e.g. assuming a superposition of two-maximum log-normal spectra of PBHs. A competing possibility is that MACHOs are not PBHs but dead primordial compact stars.