title: Hessian-Free High-Resolution Nesterov Acceleration for Sampling

abstract: Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed \citep{shi2021understanding}. This work explores the sampling counterpart of this phenonemon and proposes a diffusion process, whose discretizations can yield accelerated gradient-based MCMC methods. More precisely, we reformulate the optimizer of NAG for strongly convex functions (NAG-SC) as a Hessian-Free High-Resolution ODE, change its high-resolution coefficient to a hyperparameter, inject appropriate noise, and discretize the resulting diffusion process. The acceleration effect of the new hyperparameter is quantified and it is not an artificial one created by time-rescaling. Instead, acceleration beyond underdamped Langevin in $W_2$ distance is quantitatively established for log-strongly-concave-and-smooth targets, at both the continuous dynamics level and the discrete algorithm level. Empirical experiments in both log-strongly-concave and multi-modal cases also numerically demonstrate this acceleration.


title: Primary User Emulation and Jamming Attack Detection in Cognitive Radio via Sparse Coding

abstract: Cognitive radio is an intelligent and adaptive radio that improves the utilization of the spectrum by its opportunistic sharing. However, it is inherently vulnerable to primary user emulation and jamming attacks that degrade the spectrum utilization. In this paper, an algorithm for the detection of primary user emulation and jamming attacks in cognitive radio is proposed. The proposed algorithm is based on the sparse coding of the compressed received signal over a channel-dependent dictionary. More specifically, the convergence patterns in sparse coding according to such a dictionary are used to distinguish between a spectrum hole, a legitimate primary user, and an emulator or a jammer. The process of decision-making is carried out as a machine learning-based classification operation. Extensive numerical experiments show the effectiveness of the proposed algorithm in detecting the aforementioned attacks with high success rates. This is validated in terms of the confusion matrix quality metric. Besides, the proposed algorithm is shown to be superior to energy detection-based machine learning techniques in terms of receiver operating characteristics curves and the areas under these curves


title: Assisted music creation with Flow Machines: towards new categories of new

abstract: This chapter reflects on about 10 years of research in AI- assisted music composition, in particular during the Flow Machines project. We reflect on the motivations for such a project, its background, its main results and impact, both technological and musical, several years after its completion. We conclude with a proposal for new categories of new, created by the many uses of AI techniques to generate novel material.


title: Towards Deductive Verification of Control Algorithms for Autonomous Marine Vehicles

abstract: The use of autonomous vehicles in real-world applications is often precluded by the difficulty of providing safety guarantees for their complex controllers. The simulation-based testing of these controllers cannot deliver sufficient safety guarantees, and the use of formal verification is very challenging due to the hybrid nature of the autonomous vehicles. Our work-in-progress paper introduces a formal verification approach that addresses this challenge by integrating the numerical computation of such a system (in GNU/Octave) with its hybrid system verification by means of a proof assistant (Isabelle). To show the effectiveness of our approach, we use it to verify differential invariants of an Autonomous Marine Vehicle with a controller switching between multiple modes.


title: Model Embedding Model-Based Reinforcement Learning

abstract: Model-based reinforcement learning (MBRL) has shown its advantages in sample-efficiency over model-free reinforcement learning (MFRL). Despite the impressive results it achieves, it still faces a trade-off between the ease of data generation and model bias. In this paper, we propose a simple and elegant model-embedding model-based reinforcement learning (MEMB) algorithm in the framework of the probabilistic reinforcement learning. To balance the sample-efficiency and model bias, we exploit both real and imaginary data in the training. In particular, we embed the model in the policy update and learn $Q$ and $V$ functions from the real data set. We provide the theoretical analysis of MEMB with the Lipschitz continuity assumption on the model and policy. At last, we evaluate MEMB on several benchmarks and demonstrate our algorithm can achieve state-of-the-art performance.


title: Weakly-supervised Domain Adaption for Aspect Extraction via Multi-level Interaction Transfer

abstract: Fine-grained aspect extraction is an essential sub-task in aspect based opinion analysis. It aims to identify the aspect terms (a.k.a. opinion targets) of a product or service in each sentence. However, expensive annotation process is usually involved to acquire sufficient token-level labels for each domain. To address this limitation, some previous works propose domain adaptation strategies to transfer knowledge from a sufficiently labeled source domain to unlabeled target domains. But due to both the difficulty of fine-grained prediction problems and the large domain gap between domains, the performance remains unsatisfactory. This work conducts a pioneer study on leveraging sentence-level aspect category labels that can be usually available in commercial services like review sites to promote token-level transfer for the extraction purpose. Specifically, the aspect category information is used to construct pivot knowledge for transfer with assumption that the interactions between sentence-level aspect category and token-level aspect terms are invariant across domains. To this end, we propose a novel multi-level reconstruction mechanism that aligns both the fine-grained and coarse-grained information in multiple levels of abstractions. Comprehensive experiments demonstrate that our approach can fully utilize sentence-level aspect category labels to improve cross-domain aspect extraction with a large performance gain.


title: The Free Electron Gas in Cavity Quantum Electrodynamics

abstract: Cavity modification of material properties and phenomena is a novel research field largely motivated by the advances in strong light-matter interactions. Despite this progress, exact solutions for extended systems strongly coupled to the photon field are not available, and both theory and experiments rely mainly on finite-system models. Therefore a paradigmatic example of an exactly solvable extended system in a cavity becomes highly desireable. To fill this gap we revisit Sommerfeld's theory of the free electron gas in cavity quantum electrodynamics (QED). We solve this system analytically in the long-wavelength limit for an arbitrary number of non-interacting electrons, and we demonstrate that the electron-photon ground state is a Fermi liquid which contains virtual photons. In contrast to models of finite systems, no ground state exists if the diamagentic $\textbf{A}^2$ term is omitted. Further, by performing linear response we show that the cavity field induces plasmon-polariton excitations and modifies the optical and the DC conductivity of the electron gas. Our exact solution allows us to consider the thermodynamic limit for both electrons and photons by constructing an effective quantum field theory. The continuum of modes leads to a many-body renormalization of the electron mass, which modifies the fermionic quasiparticle excitations of the Fermi liquid and the Wigner-Seitz radius of the interacting electron gas. Lastly, we show how the matter-modified photon field leads to a repulsive Casimir force and how the continuum of modes introduces dissipation into the light-matter system. Several of the presented findings should be experimentally accessible.


title: Geometric Singular Perturbation Theory Analysis of an Epidemic Model with Spontaneous Human Behavioral Change

abstract: We consider a model due to Piero Poletti and collaborators that adds spontaneous human behavioral change to the standard SIR epidemic model. In its simplest form, the Poletti model adds one differential equation, motivated by evolutionary game theory, to the SIR model. The new equation describes the evolution of a variable $x$ that represents the fraction of the population using normal behavior. The remaining fraction $1-x$ uses altered behavior such as staying home, social isolation, mask wearing, etc. Normal behavior offers a higher payoff when the number of infectives is low; altered behavior offers a higher payoff when the number is high. We show that the entry-exit function of geometric singular perturbation theory can be used to analyze the model in the limit in which behavior changes on a much faster time scale than that of the epidemic. In particular, behavior does not change as soon as a different behavior has a higher payoff; current behavior is sticky. The delay until behavior changes in predicted by the entry-exit function.


title: Foreground-Background Imbalance Problem in Deep Object Detectors: A Review

abstract: Recent years have witnessed the remarkable developments made by deep learning techniques for object detection, a fundamentally challenging problem of computer vision. Nevertheless, there are still difficulties in training accurate deep object detectors, one of which is owing to the foreground-background imbalance problem. In this paper, we survey the recent advances about the solutions to the imbalance problem. First, we analyze the characteristics of the imbalance problem in different kinds of deep detectors, including one-stage and two-stage ones. Second, we divide the existing solutions into two categories: sampling heuristics and non-sampling schemes, and review them in detail. Third, we experimentally compare the performance of some state-of-the-art solutions on the COCO benchmark. Promising directions for future work are also discussed.


title: Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts

abstract: Accurate estimation of aleatoric and epistemic uncertainty is crucial to build safe and reliable systems. Traditional approaches, such as dropout and ensemble methods, estimate uncertainty by sampling probability predictions from different submodels, which leads to slow uncertainty estimation at inference time. Recent works address this drawback by directly predicting parameters of prior distributions over the probability predictions with a neural network. While this approach has demonstrated accurate uncertainty estimation, it requires defining arbitrary target parameters for in-distribution data and makes the unrealistic assumption that out-of-distribution (OOD) data is known at training time. In this work we propose the Posterior Network (PostNet), which uses Normalizing Flows to predict an individual closed-form posterior distribution over predicted probabilites for any input sample. The posterior distributions learned by PostNet accurately reflect uncertainty for in- and out-of-distribution data -- without requiring access to OOD data at training time. PostNet achieves state-of-the art results in OOD detection and in uncertainty calibration under dataset shifts.


title: Hot gas heating via magnetic arms in spiral galaxies. The case of M 83

abstract: Reconnection heating has been considered as a potential source of the heating of the interstellar medium. In some galaxies, significant polarised radio emission has been found between the spiral arms. This emission has a form of `magnetic arms' that resembles the spiral structure of the galaxy. Reconnection effects could convert some of the energy of the turbulent magnetic field into the thermal energy of the surrounding medium, leaving more ordered magnetic fields, as is observed in the magnetic arms. Sensitive radio and X-ray data for the grand-design spiral galaxy M 83 are used for a detailed analysis of the possible interactions of magnetic fields with hot gas, including a search for signatures of gas heating by magnetic reconnection effects. Magnetic field strengths and energies derived from the radio emission are compared with the parameters of the hot gas calculated from the model fits to sensitive X-ray spectra of the hot gas emission. The available X-ray data allowed us to distinguish two thermal components in the halo of M 83. We found slightly higher average temperatures of the hot gas in the interarm regions, which results in higher energies per particle and is accompanied by a decrease in the energy density of the magnetic fields. The observed differences in the energy budget between the spiral arms and the interarm regions suggest that, similar to the case of another spiral galaxy NGC 6946, we may be observing hints for gas heating by magnetic reconnection effects in the interarm regions. These effects, which act more efficiently on the turbulent component of the magnetic field, are expected to be stronger in the spiral arms. However, with the present data it is only possible to trace them in the interarm regions, where the star formation and the resulting turbulence is low.


title: Two-Dimensional Non-Line-of-Sight Scene Estimation from a Single Edge Occluder

abstract: Passive non-line-of-sight imaging methods are often faster and stealthier than their active counterparts, requiring less complex and costly equipment. However, many of these methods exploit motion of an occluder or the hidden scene, or require knowledge or calibration of complicated occluders. The edge of a wall is a known and ubiquitous occluding structure that may be used as an aperture to image the region hidden behind it. Light from around the corner is cast onto the floor forming a fan-like penumbra rather than a sharp shadow. Subtle variations in the penumbra contain a remarkable amount of information about the hidden scene. Previous work has leveraged the vertical nature of the edge to demonstrate 1D (in angle measured around the corner) reconstructions of moving and stationary hidden scenery from as little as a single photograph of the penumbra. In this work, we introduce a second reconstruction dimension: range measured from the edge. We derive a new forward model, accounting for radial falloff, and propose two inversion algorithms to form 2D reconstructions from a single photograph of the penumbra. Performances of both algorithms are demonstrated on experimental data corresponding to several different hidden scene configurations. A Cramer-Rao bound analysis further demonstrates the feasibility (and utility) of the 2D corner camera.


title: Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs

abstract: We present Graformer, a novel Transformer-based encoder-decoder architecture for graph-to-text generation. With our novel graph self-attention, the encoding of a node relies on all nodes in the input graph - not only direct neighbors - facilitating the detection of global patterns. We represent the relation between two nodes as the length of the shortest path between them. Graformer learns to weight these node-node relations differently for different attention heads, thus virtually learning differently connected views of the input graph. We evaluate Graformer on two popular graph-to-text generation benchmarks, AGENDA and WebNLG, where it achieves strong performance while using many fewer parameters than other approaches.


title: AcED: Accurate and Edge-consistent Monocular Depth Estimation

abstract: Single image depth estimation is a challenging problem. The current state-of-the-art method formulates the problem as that of ordinal regression. However, the formulation is not fully differentiable and depth maps are not generated in an end-to-end fashion. The method uses a na\"ive threshold strategy to determine per-pixel depth labels, which results in significant discretization errors. For the first time, we formulate a fully differentiable ordinal regression and train the network in end-to-end fashion. This enables us to include boundary and smoothness constraints in the optimization function, leading to smooth and edge-consistent depth maps. A novel per-pixel confidence map computation for depth refinement is also proposed. Extensive evaluation of the proposed model on challenging benchmarks reveals its superiority over recent state-of-the-art methods, both quantitatively and qualitatively. Additionally, we demonstrate practical utility of the proposed method for single camera bokeh solution using in-house dataset of challenging real-life images.


title: Eigenvalues of elliptic functional differential systems via a Birkhoff--Kellogg type theorem

abstract: Motivated by recent interest on Kirchhoff-type equations, in this short note we utilize a classical, yet very powerful, tool of nonlinear functional analysis in order to investigate the existence of positive eigenvalues of systems of elliptic functional differential equations. An example is presented to illustrate the theory.


title: An empirical study on using CNNs for fast radio signal prediction

abstract: Accurate radio frequency power prediction in a geographic region is a computationally expensive part of finding the optimal transmitter location using a ray tracing software. We empirically analyze the viability of deep learning models to speed up this process. Specifically, deep learning methods including CNNs and UNET are typically used for segmentation, and can also be employed in power prediction tasks. We consider a dataset that consists of radio frequency power values for five different regions with four different frame dimensions. We compare deep learning-based prediction models including RadioUNET and four different variations of the UNET model for the power prediction task. More complex UNET variations improve the model on higher resolution frames such as 256x256. However, using the same models on lower resolutions results in overfitting and simpler models perform better. Our detailed numerical analysis shows that the deep learning models are effective in power prediction and they are able to generalize well to the new regions.


title: Coulomb collisions in strongly anisotropic plasmas I. Cyclotron cooling in electron-ion plasmas

abstract: The behaviour of a collisional plasma which is optically thin to cyclotron radiation is considered, and the distribution functions accessible to it on the various timescales in the system are calculated. Particular attention is paid to the limit in which the collision time exceeds the radiation emission time, making the electron distribution function strongly anisotropic. Unusually for plasma physics, the collision operator can nevertheless be calculated analytically although the plasma is far from Maxwellian. The rate of radiation emission is calculated and found to be governed by the collision frequency multiplied by a factor that only depends logarithmically on plasma parameters.


title: Prior knowledge distillation based on financial time series

abstract: One of the major characteristics of financial time series is that they contain a large amount of non-stationary noise, which is challenging for deep neural networks. People normally use various features to address this problem. However, the performance of these features depends on the choice of hyper-parameters. In this paper, we propose to use neural networks to represent these indicators and train a large network constructed of smaller networks as feature layers to fine-tune the prior knowledge represented by the indicators. During back propagation, prior knowledge is transferred from human logic to machine logic via gradient descent. Prior knowledge is the deep belief of neural network and teaches the network to not be affected by non-stationary noise. Moreover, co-distillation is applied to distill the structure into a much smaller size to reduce redundant features and the risk of overfitting. In addition, the decisions of the smaller networks in terms of gradient descent are more robust and cautious than those of large networks. In numerical experiments, we find that our algorithm is faster and more accurate than traditional methods on real financial datasets. We also conduct experiments to verify and comprehend the method.


title: Coulomb collisions in strongly anisotropic plasmas II. Cyclotron cooling in laboratory pair plasmas

abstract: The behaviour of a strongly-magnetized collisional electron-positron plasma which is optically thin to cyclotron radiation is considered, and the distribution functions accessible to it on the various timescales in the system are calculated. Particular attention is paid to the limit in which the collision time exceeds the radiation emission time, making the electron distribution function strongly anisotropic. Indeed, these are the exact conditions likely to be attained in the first laboratory electron-positron plasma experiments currently being developed, which will typically have very low densities and be confined in very strong magnetic fields. The constraint of strong-magnetization adds an additional complication in that long-range Coulomb collisions, which are usually negligible, must now be considered. A rigorous collision operator for these long-range collisions has never been written down. Nevertheless, we show that the collisional scattering can be accounted for without knowing the explicit form of this collision operator. The rate of radiation emission is calculated and it is found that the loss of energy from the plasma is proportional to the parallel collision frequency multiplied by a factor that only depends logarithmically on plasma parameters. That is, this is a self-accelerating process, meaning that the bulk of the energy will be lost in a few collision times. We show that in a simple case, that of straight field-line geometry, there are no unstable drift waves in such plasmas, despite being far from Maxwellian.


title: Infrared polarizer based on direct coupling to surface-plasmon polaritons

abstract: We propose a new type of reflective polarizer based on polarization-dependent coupling to surface-plasmon polaritons (SPPs) from free space. This inexpensive polarizer is relatively narrowband but features an extinction ratio of up to 1000 with efficiency of up to 95% for the desired polarization (numbers from a calculation), and thus can be stacked to achieve extinction ratios of 106 or more. As a proof of concept, we experimentally realized a polarizer based on nanoporous aluminum oxide that operates around a wavelength of 10.6 um, corresponding to the output of a CO2 laser, using aluminum anodization, a low-cost electrochemical process.


title: Pressure control using stochastic cell rescaling

abstract: Molecular dynamics simulations require barostats to be performed at constant pressure. The usual recipe is to employ the Berendsen barostat first, which displays a first-order volume relaxation efficient in equilibration but results in incorrect volume fluctuations, followed by a second order or Monte Carlo barostat for production runs. In this paper, we introduce stochastic cell rescaling, a first-order barostat that samples the correct volume fluctuations by including a suitable noise term. The algorithm is shown to report volume fluctuations compatible with the isobaric ensemble and its anisotropic variant is tested on a membrane simulation. Stochastic cell rescaling can be straightforwardly implemented in existing codes and can be used effectively both in equilibration and in production phases.


title: H2 Suboptimal Output Synchronization of Heterogeneous Multi-Agent Systems

abstract: This paper deals with the H2 suboptimal output synchronization problem for heterogeneous linear multi-agent systems. Given a multi-agent system with possibly distinct agents and an associated H2 cost functional, the aim is to design output feedback based protocols that guarantee the associated cost to be smaller than a given upper bound while the controlled network achieves output synchronization. A design method is provided to compute such protocols. For each agent, the computation of its two local control gains involves two Riccati inequalities, each of dimension equal to the state space dimension of the agent. A simulation example is provided to illustrate the performance of the proposed protocols.


title: Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting

abstract: While Graph Neural Networks (GNNs) have achieved remarkable results in a variety of applications, recent studies exposed important shortcomings in their ability to capture the structure of the underlying graph. It has been shown that the expressive power of standard GNNs is bounded by the Weisfeiler-Leman (WL) graph isomorphism test, from which they inherit proven limitations such as the inability to detect and count graph substructures. On the other hand, there is significant empirical evidence, e.g. in network science and bioinformatics, that substructures are often intimately related to downstream tasks. To this end, we propose "Graph Substructure Networks" (GSN), a topologically-aware message passing scheme based on substructure encoding. We theoretically analyse the expressive power of our architecture, showing that it is strictly more expressive than the WL test, and provide sufficient conditions for universality. Importantly, we do not attempt to adhere to the WL hierarchy; this allows us to retain multiple attractive properties of standard GNNs such as locality and linear network complexity, while being able to disambiguate even hard instances of graph isomorphism. We perform an extensive experimental evaluation on graph classification and regression tasks and obtain state-of-the-art results in diverse real-world settings including molecular graphs and social networks. The code is publicly available at https://github.com/gbouritsas/graph-substructure-networks.


title: Regularity of Fluxes in Nonlinear Hyperbolic Balance Laws

abstract: This paper addresses the issue of the formulation of weak solutions to systems of nonlinear hyperbolic conservation laws as integral balance laws. The basic idea is that the "meaningful objects" are the fluxes, evaluated across domain boundaries over time intervals. The fundamental result in this treatment is the regularity of the flux trace in the multi-dimensional setting. It implies that a weak solution indeed satisfies the balance law. In fact, it is shown that the flux is Lipschitz continuous with respect to suitable perturbations of the boundary.


title: Radio and X-ray connection in radio mini-halos: implications for hadronic models

abstract: The radio mini halos (MH) observed in relaxed clusters probe the presence of relativistic particles on scales of hundreds of kpc, beyond the scales directly influenced by the central AGN, but the nature of the mechanism that produces the relativistic electrons is still debated. In this work we explore the connection between thermal and non-thermal components of the ICM in a sample of MH and we study its implications for hadronic models for the origin of the relativistic electrons. We studied the thermal and non-thermal connection by carrying out a point-to-point comparison of the radio and the X-ray surface brightness. We extended the method generally applied to giant radio halos by considering the effects of a grid randomly generated through a Monte Carlo chain. Contrary to what is generally observed for giant radio halos, we find that the mini-halos in our sample have super-linear scaling between radio and X-rays, which suggests a peaked distribution of relativistic electrons and magnetic field. We used the radio and X-ray correlation to constrain the physical parameters of a hadronic model and we compared the model predictions with current observations. Specifically, we focus on a model where cosmic rays are injected by the central AGN and they generate secondaries in the ICM, and we assume that the role of turbulent re-acceleration is negligible. This model allows us to constrain the AGN cosmic ray luminosity in the range $\sim10^{44-46}$ erg s$^{-1}$ and the central magnetic field in the range 10-40 $\mu$G. The resulting $\gamma$-ray fluxes calculated assuming these model parameters do not violate the upper limits on $\gamma$-ray diffuse emission set by the Fermi-LAT telescope. Further studies are now required to explore the consistency of these large magnetic fields with Faraday rotation studies and to study the interplay between the secondary electrons and the ICM turbulence.


title: Corralling Stochastic Bandit Algorithms

abstract: We study the problem of corralling stochastic bandit algorithms, that is combining multiple bandit algorithms designed for a stochastic environment, with the goal of devising a corralling algorithm that performs almost as well as the best base algorithm. We give two general algorithms for this setting, which we show benefit from favorable regret guarantees. We show that the regret of the corralling algorithms is no worse than that of the best algorithm containing the arm with the highest reward, and depends on the gap between the highest reward and other rewards.


title: Strong tunable spin-spin interaction in a weakly coupled nitrogen vacancy spin-cavity electromechanical system

abstract: The long coherence time of a single nitrogen vacancy (NV) center spin in diamond is a crucial advantage for implementing quantum information processing. However, the realization of strong coupling between single NV spins is challenging. Here we propose a method to greatly enchance the interaction between two single NV spins in diamond which are only weakly coupled to an electromechanical cavity. Owing to the presence of a critical point for the linearized electromechanical subsystem, the coupling between a single NV spin and the high-frequency polariton (formed by the mechanical and cavity modes) can be fully decoupled, but the coupling between the single NV spin and the low-frequency polariton is however greatly enhanced. Thus, AC Stark shift of the single NV spin can be measured. With the low-frequency polariton as a quantum bus, a strong coupling between two single NV centers is achievable. This effective strong coupling can ensure coherent quantum-information exchange between two spin qubits in the weakly coupled spin-cavity elecromechanical system.


title: Distribution of entanglement with variable range interactions

abstract: Distribution of quantum entanglement is investigated for an anisotropic quantum XY model with variable range interactions and in the presence of a uniform transverse magnetic field. We report the possibility of \emph{qualitative} growth in the entanglement between distant sites with an increase in the range of interactions that vary either exponentially or polynomially as the distance between the sites increases. Interestingly, we find that such entanglement enhancement is not ubiquitous and is dependent on the factorization points, a specific set of system parameters where the zero-temperature state of the system is fully separable. In particular, we observe that at zero-temperature, when the system parameters are chosen beyond the pair of factorization points, the increments in entanglement length due to variable range interactions are more pronounced compared to the situation when the parameters lie in between the factorization points. By employing the sum of all the bipartite entanglements with respect to a single site, we also show that the shareability of the bipartite entanglements are constrained, thereby establishing their monogamous nature. Furthermore, we note that the factorization points get reallocated depending on the laws of interaction fall-offs and provide an ansatz for the same. We reveal that the temperature at which the canonical equilibrium state becomes entangled from an unentangled one increases with the increase in the range of interactions, thereby demonstrating enhanced robustness in entanglement against temperature in the presence of long-range interactions and only when the system parameters are chosen between the pair of factorization points. We apply an energy-based entanglement witness to provide a justification to the observed robustness with temperature.


title: Applying Social Event Data for the Management of Cellular Networks

abstract: Internet provides a growing variety of social data sources: calendars, event aggregators, social networks, browsers, etc. Also, the mechanisms to gather information from these sources, such as web services, semantic web and big data techniques have become more accessible and efficient. This allows a detailed prediction of the main expected events and their associated crowds. Due to the increasing requirements for service provision, particularly in urban areas, having information on those events would be extremely useful for Operations, Administration and Maintenance (OAM) tasks, since the social events largely affect the cellular network performance. Therefore, this paper presents a framework for the automatic acquisition and processing of social data, as well as their association with network elements (NEs) and their performance. The main functionalities of this system, which have been devised to directly work in real networks, are defined and developed. Different OAM applications of the proposed approach are analyzed and the system is evaluated in a real deployment.


title: Ergoregion instabilities in rotating two-dimensional Bose--Einstein condensates: new perspectives on the stability of quantized vortices

abstract: We investigate the stability of vortices in two-dimensional Bose--Einstein condensates. In analogy with rotating spacetimes and with a careful account of boundary conditions, we show that the dynamical instability of multiply quantized vortices in trapped condensates persists in untrapped, spatially homogeneous geometries and has an ergoregion nature with some modification due to the peculiar dispersion of Bogoliubov sound. Our results open new perspectives to the physics of vortices in trapped condensates, where multiply quantized vortices can be stabilized by interference effects and singly charged vortices can become unstable in suitably designed trap potentials. We show how superradiant scattering can be observed also in the short-time dynamics of dynamically unstable systems, providing an alternative point of view on dynamical (in)stability phenomena in spatially finite systems.


title: Comparing the inversion statistic for distribution-biased and distribution-shifted permutations with the geometric and the GEM distributions

abstract: For a distribution $p:=\{p_k\}_{k=1}^\infty$ on the positive integers, there are two natural ways to construct a random permutation in $S_n$ or of $\mathbb{N}$ from IID samples from $p$--the $p$-biased construction and the $p$-shifted construction. First we consider the case that $p$ is the geometric distribution with parameter $1-q\in(0,1)$. In this case, the $p$-shifted random permutation has the Mallows distribution with parameter $q$. Let $P_n^{b;\text{Geo}(1-q)}$ and $P_n^{s;\text{Geo}(1-q)}$denote the biased and the shifted distributions on $S_n$. The expected number of inversions of a permutation under $P_n^{s;\text{Geo}(1-q)}$ is greater than under $P_n^{b;\text{Geo}(1-q)}$, and under either of these, a permutation tends to have many fewer inversions than it would have under the uniform distribution. For fixed $n$, both $P_n^{b;\text{Geo}(1-q)}$ and $P_n^{s;\text{Geo}(1-q)}$ converge weakly as $q\to1$ to the uniform distribution on $S_n$. We compare the biased and the shifted distributions by studying the inversion statistic under $P_n^{b;\text{Geo}(q_n)}$ and $P_n^{s;\text{Geo}(q_n)}$ for various rates of convergence of $q_n$ to 1. Then we consider $p$-biased and $p$-shifted permutations in the case that the distribution $p$ is itself random and distributed as a GEM$(\theta)$-distribution. In both the GEM$(\theta)$-biased and the GEM$(\theta)$-shifted cases, the expected number of inversions behaves asymptotically as it does under the Geo$(1-q)$-shifted distribution with $\theta=\frac q{1-q}$. Thus, one can consider the GEM$(\theta)$-shifted case as the random counterpart of the Geo$(q)$-shifted case. We also consider another $p$-biased distribution with random $p$ for which the expected number of inversions behaves asymptotically as it does under the Geo$(1-q)$-biased case with $\theta$ and $q$ as above, and with $\theta\to\infty$ and $q\to1$.


title: Structured and Localized Image Restoration

abstract: We present a novel approach to image restoration that leverages ideas from localized structured prediction and non-linear multi-task learning. We optimize a penalized energy function regularized by a sum of terms measuring the distance between patches to be restored and clean patches from an external database gathered beforehand. The resulting estimator comes with strong statistical guarantees leveraging local dependency properties of overlapping patches. We derive the corresponding algorithms for energies based on the mean-squared and Euclidean norm errors. Finally, we demonstrate the practical effectiveness of our model on different image restoration problems using standard benchmarks.


title: Arp 220: New Observational Insights into the Structure and Kinematics of the Nuclear Molecular Disks and Surrounding Gas

abstract: ALMA cycle 3 observations of $^{12}$CO $ J = 3\rightarrow2$, $^{13}$CO $J = 4\rightarrow 3$, SiO J = $8 \rightarrow 7$, and HCN J = $ 5 \rightarrow 4$ are presented. Significant extended emission is detected in $^{12}$CO J $ = 3\rightarrow2$ with a morphology that is indicative of m = 2 tidal features, suggesting gas inflow. In addition, outflow for both nuclei are found in the $^{12}$CO J $ = 3\rightarrow2$. Significant SiO absorption is detected in the western nucleus. HCN that is morphologically distinct from CO is detected in both nuclei. These observations are compared to non-LTE radiative transfer models created using the Line Modeling Engine (LIME) for simple gas dynamics to gain insight into how physical parameters, such as rotational velocity, turbulent velocity, gas temperature, dust temperature, and gas mass can reproduce the observed kinematic and spatial features. The eastern nucleus is found to be best modeled with an inclusion of a temperature asymmetry from one side of the disk to the other. It is also found that the western nucleus is optically thick even in the less abundant species of $^{13}$CO, absorbing significant amounts of continuum radiation.


title: A New Primal-Dual Algorithm for a Class of Nonlinear Compositional Convex Optimization Problems

abstract: We develop a novel primal-dual algorithm to solve a class of nonsmooth and nonlinear compositional convex minimization problems, which covers many existing and brand-new models as special cases. Our approach relies on a combination of a new nonconvex potential function, Nesterov's accelerated scheme, and an adaptive parameter updating strategy. Our algorithm is single-loop and has low per-iteration complexity. Under only general convexity and mild assumptions, our algorithm achieves $\mathcal{O}(1/k)$ convergence rates through three different criteria: primal objective residual, dual objective residual, and primal-dual gap, where $k$ is the iteration counter. Our rates are both ergodic (i.e., on an averaging sequence) and non-ergodic (i.e., on the last-iterate sequence). These convergence rates can be accelerated up to $\mathcal{O}(1/k^2)$ if only one objective term is strongly convex (or equivalently, its conjugate is $L$-smooth). To the best of our knowledge, this is the first algorithm achieving optimal rates on the primal last-iterate sequence for nonlinear compositional convex minimization. As a by-product, we specify our algorithm to solve a general convex cone constrained program with both ergodic and non-ergodic rate guarantees. We test our algorithms and compare them with two recent methods on a binary classification and a convex-concave game model.


title: Bonsai-Net: One-Shot Neural Architecture Search via Differentiable Pruners

abstract: One-shot Neural Architecture Search (NAS) aims to minimize the computational expense of discovering state-of-the-art models. However, in the past year attention has been drawn to the comparable performance of naive random search across the same search spaces used by leading NAS algorithms. To address this, we explore the effects of drastically relaxing the NAS search space, and we present Bonsai-Net, an efficient one-shot NAS method to explore our relaxed search space. Bonsai-Net is built around a modified differential pruner and can consistently discover state-of-the-art architectures that are significantly better than random search with fewer parameters than other state-of-the-art methods. Additionally, Bonsai-Net performs simultaneous model search and training, dramatically reducing the total time it takes to generate fully-trained models from scratch.


title: IsarStep: a Benchmark for High-level Mathematical Reasoning

abstract: A well-defined benchmark is essential for measuring and accelerating research progress of machine learning models. In this paper, we present a benchmark for high-level mathematical reasoning and study the reasoning capabilities of neural sequence-to-sequence models. We build a non-synthetic dataset from the largest repository of proofs written by human experts in a theorem prover. The dataset has a broad coverage of undergraduate and research-level mathematical and computer science theorems. In our defined task, a model is required to fill in a missing intermediate proposition given surrounding proofs. This task provides a starting point for the long-term goal of having machines generate human-readable proofs automatically. Our experiments and analysis reveal that while the task is challenging, neural models can capture non-trivial mathematical reasoning. We further design a hierarchical transformer that outperforms the transformer baseline.


title: Comparing Representations for Audio Synthesis Using Generative Adversarial Networks

abstract: In this paper, we compare different audio signal representations, including the raw audio waveform and a variety of time-frequency representations, for the task of audio synthesis with Generative Adversarial Networks (GANs). We conduct the experiments on a subset of the NSynth dataset. The architecture follows the benchmark Progressive Growing Wasserstein GAN. We perform experiments both in a fully non-conditional manner as well as conditioning the network on the pitch information. We quantitatively evaluate the generated material utilizing standard metrics for assessing generative models, and compare training and sampling times. We show that complex-valued as well as the magnitude and Instantaneous Frequency of the Short-Time Fourier Transform achieve the best results, and yield fast generation and inversion times. The code for feature extraction, training and evaluating the model is available online.


title: Data Augmentation of IMU Signals and Evaluation via a Semi-Supervised Classification of Driving Behavior

abstract: Over the past years, interest in classifying drivers' behavior from data has surged. Such interest is particularly relevant for car insurance companies who, due to privacy constraints, often only have access to data from Inertial Measurement Units (IMU) or similar. In this paper, we present a semi-supervised learning solution to classify portions of trips according to whether drivers are driving aggressively or normally based on such IMU data. Since the amount of labeled IMU data is limited and costly to generate, we utilize Recurrent Conditional Generative Adversarial Networks (RCGAN) to generate more labeled data. Our results show that, by utilizing RCGAN-generated labeled data, the classification of the drivers is improved in 79% of the cases, compared to when the drivers are classified with no generated data.


title: Metrizing Weak Convergence with Maximum Mean Discrepancies

abstract: This paper characterizes the maximum mean discrepancies (MMD) that metrize the weak convergence of probability measures for a wide class of kernels. More precisely, we prove that, on a locally compact, non-compact, Hausdorff space, the MMD of a bounded continuous Borel measurable kernel k, whose reproducing kernel Hilbert space (RKHS) functions vanish at infinity, metrizes the weak convergence of probability measures if and only if k is continuous and integrally strictly positive definite (i.s.p.d.) over all signed, finite, regular Borel measures. We also correct a prior result of Simon-Gabriel & Sch\"olkopf (JMLR, 2018, Thm.12) by showing that there exist both bounded continuous i.s.p.d. kernels that do not metrize weak convergence and bounded continuous non-i.s.p.d. kernels that do metrize it.


title: A Thomassen-type method for planar graph recoloring

abstract: The reconfiguration graph $R_k(G)$ for the $k$-colorings of a graph $G$ has as vertices all possible $k$-colorings of $G$ and two colorings are adjacent if they differ in the color of exactly one vertex. We use a list coloring technique inspired by results of Thomassen to prove that for a planar graph $G$ with $n$ vertices, $R_{10}(G)$ has diameter at most $8n$, and if $G$ is triangle-free, then $R_7(G)$ has diameter at most $7n$.


title: Primal Dual Interpretation of the Proximal Stochastic Gradient Langevin Algorithm

abstract: We consider the task of sampling with respect to a log concave probability distribution. The potential of the target distribution is assumed to be composite, \textit{i.e.}, written as the sum of a smooth convex term, and a nonsmooth convex term possibly taking infinite values. The target distribution can be seen as a minimizer of the Kullback-Leibler divergence defined on the Wasserstein space (\textit{i.e.}, the space of probability measures). In the first part of this paper, we establish a strong duality result for this minimization problem. In the second part of this paper, we use the duality gap arising from the first part to study the complexity of the Proximal Stochastic Gradient Langevin Algorithm (PSGLA), which can be seen as a generalization of the Projected Langevin Algorithm. Our approach relies on viewing PSGLA as a primal dual algorithm and covers many cases where the target distribution is not fully supported. In particular, we show that if the potential is strongly convex, the complexity of PSGLA is $O(1/\varepsilon^2)$ in terms of the 2-Wasserstein distance. In contrast, the complexity of the Projected Langevin Algorithm is $O(1/\varepsilon^{12})$ in terms of total variation when the potential is convex.


title: A Survey of Machine Learning Methods and Challenges for Windows Malware Classification

abstract: Malware classification is a difficult problem, to which machine learning methods have been applied for decades. Yet progress has often been slow, in part due to a number of unique difficulties with the task that occur through all stages of the developing a machine learning system: data collection, labeling, feature creation and selection, model selection, and evaluation. In this survey we will review a number of the current methods and challenges related to malware classification, including data collection, feature extraction, and model construction, and evaluation. Our discussion will include thoughts on the constraints that must be considered for machine learning based solutions in this domain, and yet to be tackled problems for which machine learning could also provide a solution. This survey aims to be useful both to cybersecurity practitioners who wish to learn more about how machine learning can be applied to the malware problem, and to give data scientists the necessary background into the challenges in this uniquely complicated space.


title: Ensemble-based Feature Selection and Classification Model for DNS Typo-squatting Detection

abstract: Domain Name System (DNS) plays in important role in the current IP-based Internet architecture. This is because it performs the domain name to IP resolution. However, the DNS protocol has several security vulnerabilities due to the lack of data integrity and origin authentication within it. This paper focuses on one particular security vulnerability, namely typo-squatting. Typo-squatting refers to the registration of a domain name that is extremely similar to that of an existing popular brand with the goal of redirecting users to malicious/suspicious websites. The danger of typo-squatting is that it can lead to information threat, corporate secret leakage, and can facilitate fraud. This paper builds on our previous work in [1], which only proposed majority-voting based classifier, by proposing an ensemble-based feature selection and bagging classification model to detect DNS typo-squatting attack. Experimental results show that the proposed framework achieves high accuracy and precision in identifying the malicious/suspicious typo-squatting domains (a loss of at most 1.5% in accuracy and 5% in precision when compared to the model that used the complete feature set) while having a lower computational complexity due to the smaller feature set (a reduction of more than 50% in feature set size).


title: Density of States Estimation for Out-of-Distribution Detection

abstract: Perhaps surprisingly, recent studies have shown probabilistic model likelihoods have poor specificity for out-of-distribution (OOD) detection and often assign higher likelihoods to OOD data than in-distribution data. To ameliorate this issue we propose DoSE, the density of states estimator. Drawing on the statistical physics notion of ``density of states,'' the DoSE decision rule avoids direct comparison of model probabilities, and instead utilizes the ``probability of the model probability,'' or indeed the frequency of any reasonable statistic. The frequency is calculated using nonparametric density estimators (e.g., KDE and one-class SVM) which measure the typicality of various model statistics given the training data and from which we can flag test points with low typicality as anomalous. Unlike many other methods, DoSE requires neither labeled data nor OOD examples. DoSE is modular and can be trivially applied to any existing, trained model. We demonstrate DoSE's state-of-the-art performance against other unsupervised OOD detectors on previously established ``hard'' benchmarks.


title: Measurements of top-quark pair single- and double-differential cross-sections in the all-hadronic channel in $pp$ collisions at $\sqrt{s}=13~\mbox{TeV}$ using the ATLAS detector

abstract: Differential cross-sections are measured for top-quark pair production in the all-hadronic decay mode, using proton$-$proton collision events collected by the ATLAS experiment in which all six decay jets are separately resolved. Absolute and normalised single- and double-differential cross-sections are measured at particle and parton level as a function of various kinematic variables. Emphasis is placed on well-measured observables in fully reconstructed final states, as well as on the study of correlations between the top-quark pair system and additional jet radiation identified in the event. The study is performed using data from proton$-$proton collisions at $\sqrt{s}=13~\mbox{TeV}$ collected by the ATLAS detector at CERN's Large Hadron Collider in 2015 and 2016, corresponding to an integrated luminosity of $\mbox{36.1 fb}^{-1}$. The rapidities of the individual top quarks and of the top-quark pair are well modelled by several independent event generators. Significant mismodelling is observed in the transverse momenta of the leading three jet emissions, while the leading top-quark transverse momentum and top-quark pair transverse momentum are both found to be incompatible with several theoretical predictions.


title: Hierarchical, rotation-equivariant neural networks to select structural models of protein complexes

abstract: Predicting the structure of multi-protein complexes is a grand challenge in biochemistry, with major implications for basic science and drug discovery. Computational structure prediction methods generally leverage pre-defined structural features to distinguish accurate structural models from less accurate ones. This raises the question of whether it is possible to learn characteristics of accurate models directly from atomic coordinates of protein complexes, with no prior assumptions. Here we introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes, without using any pre-computed physics-inspired or statistical terms. Our neural network architecture combines multiple ingredients that together enable end-to-end learning from molecular structures containing tens of thousands of atoms: a point-based representation of atoms, equivariance with respect to rotation and translation, local convolutions, and hierarchical subsampling operations. When used in combination with previously developed scoring functions, our network substantially improves the identification of accurate structural models among a large set of possible models. Our network can also be used to predict the accuracy of a given structural model in absolute terms. The architecture we present is readily applicable to other tasks involving learning on 3D structures of large atomic systems.


title: How Secure is Distributed Convolutional Neural Network on IoT Edge Devices?

abstract: Convolutional Neural Networks (CNN) has found successful adoption in many applications. The deployment of CNN on resource-constrained edge devices have proved challenging. CNN distributed deployment across different edge devices has been adopted. In this paper, we propose Trojan attacks on CNN deployed across a distributed edge network across different nodes. We propose five stealthy attack scenarios for distributed CNN inference. These attacks are divided into trigger and payload circuitry. These attacks are tested on deep learning models (LeNet, AlexNet). The results show how the degree of vulnerability of individual layers and how critical they are to the final classification.


title: Communicative need modulates competition in language change

abstract: All living languages change over time. The causes for this are many, one being the emergence and borrowing of new linguistic elements. Competition between the new elements and older ones with a similar semantic or grammatical function may lead to speakers preferring one of them, and leaving the other to go out of use. We introduce a general method for quantifying competition between linguistic elements in diachronic corpora which does not require language-specific resources other than a sufficiently large corpus. This approach is readily applicable to a wide range of languages and linguistic subsystems. Here, we apply it to lexical data in five corpora differing in language, type, genre, and time span. We find that changes in communicative need are consistently predictive of lexical competition dynamics. Near-synonymous words are more likely to directly compete if they belong to a topic of conversation whose importance to language users is constant over time, possibly leading to the extinction of one of the competing words. By contrast, in topics which are increasing in importance for language users, near-synonymous words tend not to compete directly and can coexist. This suggests that, in addition to direct competition between words, language change can be driven by competition between topics or semantic subspaces.


title: An one-factor copula mixed model for joint meta-analysis of multiple diagnostic tests

abstract: As the meta-analysis of more than one diagnostic tests can impact clinical decision making and patient health, there is an increasing body of research in models and methods for meta-analysis of studies comparing multiple diagnostic tests. The application of the existing models to compare the accuracy of three or more tests suffers from the curse of multi-dimensionality, i.e., either the number of model parameters increase rapidly or high dimensional integration is required. To overcome these issues in joint meta-analysis of studies comparing $T >2$ diagnostic tests in a multiple tests design with a gold standard, we propose a model that assumes the true positives and true negatives for each test are conditionally independent and binomially distributed given the $2T$-variate latent vector of sensitivities and specificities. For the random effects distribution, we employ an one-factor copula that provides tail dependence or tail asymmetry. Maximum likelihood estimation of the model is straightforward as the derivation of the likelihood requires bi-dimensional instead of $2T$-dimensional integration. Our methodology is demonstrated with an extensive simulation study and an application example that determines which is the best test for the diagnosis of rheumatoid arthritis.


title: Regularity of Stationary Boltzmann equation in Convex Domains

abstract: Higher regularity estimate has been a challenging question for the Boltzmann equation in bounded domains. Indeed, it is well-known to have "the non-existence of a second order derivative at the boundary" in [15] even for symmetric convex domains such as a disk or sphere. In this paper we answer this question in the affirmative by constructing the $C^{1,\beta}$ solutions away from the grazing boundary, for any $\beta<1$, to the stationary Boltzmann equation with the non-isothermal diffuse boundary condition in strictly convex domains, as long as a smooth wall temperature has small fluctuation pointwisely.


title: Reflection Groups and Rigidity of Quadratic Poisson Algebras

abstract: In this paper, we study the invariant theory of quadratic Poisson algebras. Let G be a finite group of the graded Poisson automorphisms of a quadratic Poisson algebra A. When the Poisson bracket of A is skew-symmetric, a Poisson version of the Shephard-Todd-Chevalley theorem is proved stating that the fixed Poisson subring A^G is skew-symmetric if and only if G is generated by reflections. For many other well-known families of quadratic Poisson algebras, we show that G contains limited or even no reflections. This kind of Poisson rigidity result ensures that the corresponding fixed Poisson subring A^G is not isomorphic to A as Poisson algebras unless G is trivial.


title: 300 GHz generation based on a Kerr microresonator frequency comb stabilized to a low noise microwave reference

abstract: In this letter, we experimentally demonstrate low noise 300GHz wave generation based on a Kerr microresonator frequency comb operating in soliton regime. The spectral purity of a 10GHz GPS-disciplined dielectric resonant oscillator is transferred to the 300GHz repetition rate frequency of the soliton comb through an optoelectronic phase-locked loop. Two adjacent comb lines beat on a uni-travelling carrier photodiode emitting the 300GHz millimeter-wave signal into a waveguide. In an out-of-loop measurement we have measured the 300GHz power spectral density of phase noise to be -88dBc/Hz, -105dBc/Hz at 10kHz, 1MHz Fourier frequency, respectively. The free-running fractional frequency instability at 300GHz is $1 \times 10^{-9}$ at 1 second averaging time. Stabilized to a GPS signal, we report an in-loop residual instability of $2 \times 10^{-15}$ at 1 second which averages down to < $1 \times 10^{-17}$ at 1000 seconds. Such system provides a promising path to the realization of compact, low power consumption millimeter-wave oscillators with low noise performance for out-of-the-lab applications.


title: Derived Representation Schemes and Nakajima Quiver Varieties

abstract: We introduce a derived representation scheme associated with a quiver, which may be thought of as a derived version of a Nakajima variety. We exhibit an explicit model for the derived representation scheme as a Koszul complex and by doing so we show that it has vanishing higher homology if and only if the moment map defining the corresponding Nakajima variety is flat. In this case we prove a comparison theorem relating isotypical components of the representation scheme to equivariant K-theoretic classes of tautological bundles on the Nakajima variety. As a corollary of this result we obtain some integral formulas present in the mathematical and physical literature since a few years, such as the formula for Nekrasov partition function for the moduli space of framed instantons on $S^4$. On the technical side we extend the theory of relative derived representation schemes by introducing derived partial character schemes associated with reductive subgroups of the general linear group and constructing an equivariant version of the derived representation functor for algebras with a rational action of an algebraic torus.


title: Exceptional points in Fermi liquids with quadrupolar interactions

abstract: We show the existence of non-Hermitian degeneracies, known as exceptional points, in the collective mode spectrum of Fermi liquids with quadrupolar interactions. Through a careful analysis of the analytic properties of the dynamic quadrupolar susceptibility, we show that, in the weak attractive region, two stable collective modes coalesce to an exceptional point. We completely characterize this singularity, explicitly showing its topological properties. Experimental signatures are also discussed.


title: Pseudothermalization of the quark-gluon plasma

abstract: In this proceedings contribution I review recent work in kinetic theory which demonstrates that, for system undergoing Bjorken expansion, there exists an attractor in all moments of the one-particle distribution function. I discuss how this attractor emerges in both exact solutions obtained in relaxation time approximation (RTA) and the effective kinetic theory approach to high-temperature quantum chromodynamics (QCD). The QCD effective kinetic theory collisional kernel used includes both elastic (2 <-> 2) and LPM-resummed inelastic (2 <-> 1) contributions. The results obtained indicate that a pseudothermal attractor exists in both RTA and QCD kinetic theory and that their respective attractors can be extended to early times when the system is far from equilibrium. Finally, I discuss how knowledge of the QCD effective kinetic theory attractor can be used to assess different hydrodynamic freeze-out prescriptions used in heavy-ion phenomenology. The results obtained show that improved freeze-out prescriptions such as anisotropic hydrodynamics perform better in conditions corresponding to those generated in high-multiplicity pA and pp collisions, e.g. short lifetime and high inverse Reynolds number.


title: Random tensors, propagation of randomness, and nonlinear dispersive equations

abstract: Abstract. The purpose of this paper is twofold. We introduce the theory of random tensors, which naturally extends the method of random averaging operators in our earlier work arXiv:1910.08492, to study the propagation of randomness under nonlinear dispersive equations. By applying this theory we also solve Conjecture 1.7 in arXiv:1910.08492, and establish almost-sure local well-posedness for semilinear Schr\"{o}dinger equations in spaces that are subcritical in the probabilistic scaling. The solution we find has an explicit expansion in terms of multilinear Gaussians with adapted random tensor coefficients. In the random setting, the probabilistic scaling is the natural scaling for dispersive equations, and is different from the natural scaling for parabolic equations. Our theory, which covers the full subcritical regime in the probabilistic scaling, can be viewed as the dispersive counterpart of the existing parabolic theories (regularity structure, para-controlled calculus and renormalization group techniques).


title: On the Computational Power of Transformers and its Implications in Sequence Modeling

abstract: Transformers are being used extensively across several sequence modeling tasks. Significant research effort has been devoted to experimentally probe the inner workings of Transformers. However, our conceptual and theoretical understanding of their power and inherent limitations is still nascent. In particular, the roles of various components in Transformers such as positional encodings, attention heads, residual connections, and feedforward networks, are not clear. In this paper, we take a step towards answering these questions. We analyze the computational power as captured by Turing-completeness. We first provide an alternate and simpler proof to show that vanilla Transformers are Turing-complete and then we prove that Transformers with only positional masking and without any positional encoding are also Turing-complete. We further analyze the necessity of each component for the Turing-completeness of the network; interestingly, we find that a particular type of residual connection is necessary. We demonstrate the practical implications of our results via experiments on machine translation and synthetic tasks.


title: Building a Collaborative Phone Blacklisting System with Local Differential Privacy

abstract: Spam phone calls have been rapidly growing from nuisance to an increasingly effective scam delivery tool. To counter this increasingly successful attack vector, a number of commercial smartphone apps that promise to block spam phone calls have appeared on app stores, and are now used by hundreds of thousands or even millions of users. However, following a business model similar to some online social network services, these apps often collect call records or other potentially sensitive information from users' phones with little or no formal privacy guarantees. In this paper, we study whether it is possible to build a practical collaborative phone blacklisting system that makes use of local differential privacy (LDP) mechanisms to provide clear privacy guarantees. We analyze the challenges and trade-offs related to using LDP, evaluate our LDP-based system on real-world user-reported call records collected by the FTC, and show that it is possible to learn a phone blacklist using a reasonable overall privacy budget and at the same time preserve users' privacy while maintaining utility for the learned blacklist.


title: Uncovering the Underlying Physics of Degrading System Behavior Through a Deep Neural Network Framework: The Case of Remaining Useful Life Prognosis

abstract: Deep learning (DL) has become an essential tool in prognosis and health management (PHM), commonly used as a regression algorithm for the prognosis of a system's behavior. One particular metric of interest is the remaining useful life (RUL) estimated using monitoring sensor data. Most of these deep learning applications treat the algorithms as black-box functions, giving little to no control of the data interpretation. This becomes an issue if the models break the governing laws of physics or other natural sciences when no constraints are imposed. The latest research efforts have focused on applying complex DL models to achieve a low prediction error rather than studying how the models interpret the behavior of the data and the system itself. In this paper, we propose an open-box approach using a deep neural network framework to explore the physics of degradation through partial differential equations (PDEs). The framework has three stages, and it aims to discover a latent variable and corresponding PDE to represent the health state of the system. Models are trained as a supervised regression and designed to output the RUL as well as a latent variable map that can be used and interpreted as the system's health indicator.


title: Isometric Autoencoders

abstract: High dimensional data is often assumed to be concentrated on or near a low-dimensional manifold. Autoencoders (AE) is a popular technique to learn representations of such data by pushing it through a neural network with a low dimension bottleneck while minimizing a reconstruction error. Using high capacity AE often leads to a large collection of minimizers, many of which represent a low dimensional manifold that fits the data well but generalizes poorly. Two sources of bad generalization are: extrinsic, where the learned manifold possesses extraneous parts that are far from the data; and intrinsic, where the encoder and decoder introduce arbitrary distortion in the low dimensional parameterization. An approach taken to alleviate these issues is to add a regularizer that favors a particular solution; common regularizers promote sparsity, small derivatives, or robustness to noise. In this paper, we advocate an isometry (i.e., local distance preserving) regularizer. Specifically, our regularizer encourages: (i) the decoder to be an isometry; and (ii) the encoder to be the decoder's pseudo-inverse, that is, the encoder extends the inverse of the decoder to the ambient space by orthogonal projection. In a nutshell, (i) and (ii) fix both intrinsic and extrinsic degrees of freedom and provide a non-linear generalization to principal component analysis (PCA). Experimenting with the isometry regularizer on dimensionality reduction tasks produces useful low-dimensional data representations.


title: Novel Randomized Placement for FPGA Based Robust ROPUF with Improved Uniqueness

abstract: The physical unclonable functions (PUF) are used to provide software as well as hardware security for the cyber-physical systems. They have been used for performing significant cryptography tasks such as generating keys, device authentication, securing against IP piracy, and to produce the root of trust as well. However, they lack in reliability metric. We present a novel approach for improving the reliability as well as the uniqueness of the field programmable gated arrays (FPGAs) based ring oscillator PUF and derive a random number, consuming very small area (< 1%) concerning look-up tables (LUTs). We use frequency profiling method for distributing frequency variations in ring oscillators (RO), spatially placed all across the FPGA floor. We are able to spot suitable locations for RO mapping, which leads to enhanced ROPUF reliability. We have evaluated the proposed methodology on Xilinx -7 series FPGAs and tested the robustness against environmental variations, e.g. temperature and supply voltage variations, simultaneously. The proposed approach achieves significant improvement (i) in uniqueness value upto 49:90%, within 0.1% of the theoretical value (ii) in the reliability value upto 99:70%, which signifies that less than 1 bit flipping has been observed on average, and (iii) in randomness, signified by passing NIST test suite. The response generated through the ROPUF passes all the applicable relevant tests of NIST uniformity statistical test suite.


title: Stochastic Activity Networks Templates: Supporting Variability in Performability Models

abstract: Model-based evaluation is extensively used to estimate performance and reliability of dependable systems. Traditionally, those systems were small and self-contained, and the main challenge for model-based evaluation has been the efficiency of the solution process. Recently, the problem of specifying and maintaining complex models has increasingly gained attention, as modern systems are characterized by many components and complex interactions. Components share similarities, but also exhibit variations in their behavior due to different configurations or roles in the system. From the modeling perspective, variations lead to replicating and altering a small set of base models multiple times. Variability is taken into account only informally, by defining a sample model and explaining its possible variations. In this paper we address the problem of including variability in performability models, focusing on Stochastic Activity Networks (SANs). We introduce the formal definition of Stochastic Activity Networks Templates (SAN-T), a formalism based on SANs with the addition of variability aspects. Differently from other approaches, parameters can also affect the structure of the model, like the number of cases of activities. We apply the SAN-T formalism to the modeling of the backbone network of an environmental monitoring infrastructure. In particular, we show how existing SAN models from the literature can be generalized using the newly introduced formalism.


title: Leveraging the Information Contained in Theory Presentations

abstract: A theorem prover without an extensive library is much less useful to its potential users. Algebra, the study of algebraic structures, is a core component of such libraries. Algebraic theories also are themselves structured, the study of which was started as Universal Algebra. Various constructions (homomorphism, term algebras, products, etc) and their properties are both universal and constructive. Thus they are ripe for being automated. Unfortunately, current practice still requires library builders to write these by hand. We first highlight specific redundancies in libraries of existing systems. Then we describe a framework for generating these derived concepts from theory definitions. We demonstrate the usefulness of this framework on a test library of 227 theories.


title: An agent-based self-protective method to secure communication between UAVs in unmanned aerial vehicle networks

abstract: UAVNs (unmanned aerial vehicle networks) may become vulnerable to threats and attacks due to their characteristic features such as highly dynamic network topology, open-air wireless environments, and high mobility. Since previous work has focused on classical and metaheuristic-based approaches, none of these approaches have a self-adaptive approach. In this paper, the challenges and weaknesses of previous methods are examined in the form of a table. Furthermore, we propose an agent-based self-protective method (ASP-UAVN) for UAVNs that is based on the Human Immune System (HIS). In ASP-UAS, the safest route from the source UAV to the destination UAV is chosen according to a self-protective system. In this method, a multi-agent system using an Artificial Immune System (AIS) is employed to detect the attacking UAV and choose the safest route. In the proposed ASP-UAVN, the route request packet (RREQ) is initially transmitted from the source UAV to the destination UAV to detect the existing routes. Then, once the route reply packet (RREP) is received, a self-protective method using agents and the knowledge base is employed to choose the safest route and detect the attacking UAVs. The proposed ASP-UAVN has been validated and evaluated in two ways: simulation and theoretical analysis. The results of simulation evaluation and theory analysis showed that the ASP-UAS increases the Packet Delivery Rate (PDR) by more than 17.4, 20.8, and 25.91%, and detection rate by more than 17.2, 23.1, and 29.3%, and decreases the Packet Loss Rate (PLR) by more than 14.4, 16.8, and 20.21%, the false-positive and false-negative rate by more than 16.5, 25.3, and 31.21% those of SUAS-HIS, SFA and BRUIDS methods, respectively.


title: CC-Light eQASM Architecture Specification

abstract: This document is the specification of the CC-Light instantiation of executable QASM (eQASM), a quantum instruction set architecture (QISA) developed in QuTech targeting to control a seven-qubit superconducting quantum processor. This document can serve as a reference manual for low-level programmers, compiler backend developers, and microarchitecture implementers of eQASM. The design of CC-Light eQASM is under the Apache 2.0 License.


title: Aspherical manifolds, Mellin transformation and a question of Bobadilla-Koll\'{a}r

abstract: In their 2012 paper, Bobadilla and Koll\'ar studied topological conditions which guarantee that a proper map of complex algebraic varieties is a topological or differentiable fibration. They also asked whether a certain finiteness property on the relative covering space can imply that a proper map is a fibration. In this paper, we answer positively the integral homology version of their question in the case of abelian varieties, and the rational homology version in the case of compact ball quotients. We also propose several conjectures in relation to the Singer-Hopf conjecture in the complex projective setting.


title: Inline holographic microscopy through fiber imaging bundles

abstract: Fiber imaging bundles are widely used as thin, passive image conduits for miniaturised and endoscopic microscopy, particularly for confocal fluorescence imaging. Holographic microscopy through fiber bundles is more challenging; phase conjugation approaches are complex and require extensive calibration. This articles describes how simple inline holographic microscopy can be performed through an imaging bundle using a partially coherent illumination source from a multimode fiber. The sample is imaged in transmission, with the intensity hologram sampled by the bundle and transmitted to a remote camera. The hologram can then be numerically refocused for volumetric imaging, achieving a resolution of approximately 6 um over a depth range of 1 mm. The scheme does not require any complex prior calibration and hence is insensitive to bending.


title: A non-conforming dual approach for adaptive Trust-Region Reduced Basis approximation of PDE-constrained optimization

abstract: In this contribution we propose and rigorously analyze new variants of adaptive Trust-Region methods for parameter optimization with PDE constraints and bilateral parameter constraints. The approach employs successively enriched Reduced Basis surrogate models that are constructed during the outer optimization loop and used as model function for the Trust-Region method. Each Trust-Region sub-problem is solved with the projected BFGS method. Moreover, we propose a non-conforming dual (NCD) approach to improve the standard RB approximation of the optimality system. Rigorous improved a posteriori error bounds are derived and used to prove convergence of the resulting NCD-corrected adaptive Trust-Region Reduced Basis algorithm. Numerical experiments demonstrate that this approach enables to reduce the computational demand for large scale or multi-scale PDE constrained optimization problems significantly.


title: Critical Fluctuations in Renewal Models of Statistical Mechanics

abstract: We investigate the sharp asymptotic behavior at criticality of the large fluctuations of extensive observables in renewal models of statistical mechanics, such as the Poland-Scheraga model of DNA denaturation, the Fisher-Felderhof model of fluids, the Wako-Sait\^o-Mu\~noz-Eaton model of protein folding, and the Tokar-Dreyss\'e model of strained epitaxy. These models amount to Gibbs changes of measure of a classical renewal process and can be identified with a constrained pinning model of polymers. The extensive observables that enter the thermodynamic description turn out to be cumulative rewards corresponding to deterministic rewards that are uniquely determined by the waiting time and grow no faster than it. The probability decay with the system size of their fluctuations switches from exponential to subexponential at criticality, which is a regime corresponding to a discontinuous pinning-depinning phase transition. We describe such decay by proposing a precise large deviation principle under the assumption that the subexponential correction term to the waiting time distribution is regularly varying. This principle is in particular used to characterize the fluctuations of the number of renewals, which measures the DNA-bound monomers in the Poland-Scheraga model, the particles in the Fisher-Felderhof model and the Tokar-Dreyss\'e model, and the native peptide bonds in the Wako-Sait\^o-Mu\~noz-Eaton model.

categories: math.PR cond-mat.stat-mech math-ph math.MP


title: A New Run-based Connected Component Labeling for Efficiently Analyzing and Processing Holes

abstract: This article introduces a new connected component labeling and analysis algorithm for foreground and background labeling that computes the adjacency tree. The computation of features (bounding boxes, first statistical moments, Euler number) is done on-the-fly. The transitive closure enables an efficient hole processing that can be filled while their features are merged with the surrounding connected component without the need to rescan the image. A comparison with existing algorithms shows that this new algorithm can do all these computations faster than algorithms processing black and white components.


title: FLUPS: a Fourier-based Library of Unbounded Poisson Solvers

abstract: A Fourier-based Library of Unbounded Poisson Solvers (FLUPS) for 2D and 3D homogeneous distributed grids is presented. It is designed to handle every possible combination of periodic, symmetric, semi-unbounded and fully unbounded boundary conditions for the Poisson equation on rectangular domains with uniform resolution. FLUPS leverages a dedicated implementation of 3D Fourier transforms to solve the Poisson equation using Green's functions, in a fast and memory-efficient way. Several Green's functions are available, optionally with explicit regularization, spectral truncation, or using lattice Green's functions, and provide verified convergence orders from 2 to spectral-like. The algorithm depends on the FFTW library to perform 1D transforms, while Message Passing Interface (MPI) communications enable the required remapping of data in memory. For the latter operation, a first available implementation resorts to the standard all-to-all routines. A second implementation, featuring non-blocking and persistent point-to-point communications, is however shown to be more efficient in a majority of cases and especially while taking advantage of the shared memory parallelism with OpenMP. The scalability of the algorithm, aimed at massively parallel architectures, is demonstrated up to 73 720 cores. The results obtained with three different supercomputers show that the weak efficiency remains above 40\% and the strong efficiency above 30% when the number of cores is multiplied by 16, for typical problems. These figures are slightly better than those expected from a third party 3D Fast Fourier Transform (FFT) tool, with which a 20% longer execution time was also measured on average.


title: Collaborative Pipeline Using Opportunistic Mobile Resources via D2D for Computation-Intensive Tasks

abstract: This paper proposes a mobile pipeline computing concept in a Device-to-Device (D2D) communication setup and studies related issues, where D2D is likely based on millimeter-wave (mmWave) in the 5G mobile communication. The proposed opportunistic system employs a cluster of pipelined resource-limited devices on the move to handle real-time on-site computation-intensive tasks for which current cloud computing technology may not be suitable. The feasibility of such a system can be anticipated as high-speed and low-latency wireless technologies get mature. We present a system model by defining the architecture, basic functions, processes at both system-level and pipeline device level. A pipeline pathfinding algorithm along with a multi-task optimization framework is developed. To minimize the search space since the algorithm may need to be run on resource-limited mobile devices, an adjacency-matrix-power-based graph trimming technique is proposed and validated using simulation. A preliminary feasibility assessment of our proposed techniques is performed using experiments and computer simulation. As part of the feasibility assessment, the impact of mmWave blockage on the pipeline stability is analyzed and examined for both single-pipeline and concurrent-multiple-pipeline scenarios. Our design and analysis results provide certain insight to guide system design and lay a foundation for further work in this line.


title: Effective loop quantum gravity framework for vacuum spherically symmetric space-times

abstract: We develop an effective framework for the $\bar\mu$ scheme of holonomy corrections motivated by loop quantum gravity for vacuum spherically symmetric space-times. This is done by imposing the areal gauge in the classical theory, and then expressing the remaining components of the Ashtekar-Barbero connection in the Hamiltonian constraint in terms of holonomies of physical length $\ell_{\rm Pl}$. The stationary solutions to the effective Hamiltonian constraint can be found exactly, and we give the explicit form of the effective metric in Painlev\'e-Gullstrand coordinates. This solution has the correct classical limit, the quantum gravity corrections decay rapidly at large distances, and curvature scalars are bounded by the Planck scale, independently of the black hole mass $M$. In addition, the solution is valid for radii $x \ge x_{\rm min} \sim (\ell_{\rm Pl}^2 M)^{1/3}$ indicating the need for a matter field, with an energy density bounded by the Planck scale, to provide a source for the curvature in the space-time. Finally, for $M \gg m_{\rm Pl}$, the space-time has an outer and also an inner horizon, within which the expansion for outgoing radial null geodesics becomes positive again. On the other hand, for sufficiently small $M \sim m_{\rm Pl}$, there are no horizons at all in the effective metric.


title: Unsupervised Pansharpening Based on Self-Attention Mechanism

abstract: Pansharpening is to fuse a multispectral image (MSI) of low-spatial-resolution (LR) but rich spectral characteristics with a panchromatic image (PAN) of high-spatial-resolution (HR) but poor spectral characteristics. Traditional methods usually inject the extracted high-frequency details from PAN into the up-sampled MSI. Recent deep learning endeavors are mostly supervised assuming the HR MSI is available, which is unrealistic especially for satellite images. Nonetheless, these methods could not fully exploit the rich spectral characteristics in the MSI. Due to the wide existence of mixed pixels in satellite images where each pixel tends to cover more than one constituent material, pansharpening at the subpixel level becomes essential. In this paper, we propose an unsupervised pansharpening (UP) method in a deep-learning framework to address the above challenges based on the self-attention mechanism (SAM), referred to as UP-SAM. The contribution of this paper is three-fold. First, the self-attention mechanism is proposed where the spatial varying detail extraction and injection functions are estimated according to the attention representations indicating spectral characteristics of the MSI with sub-pixel accuracy. Second, such attention representations are derived from mixed pixels with the proposed stacked attention network powered with a stick-breaking structure to meet the physical constraints of mixed pixel formulations. Third, the detail extraction and injection functions are spatial varying based on the attention representations, which largely improves the reconstruction accuracy. Extensive experimental results demonstrate that the proposed approach is able to reconstruct sharper MSI of different types, with more details and less spectral distortion as compared to the state-of-the-art.


title: Ranking IPCC Models Using the Wasserstein Distance

abstract: We propose a methodology for intercomparing climate models and evaluating their performance against benchmarks based on the use of the Wasserstein distance (WD). This distance provides a rigorous way to measure quantitatively the difference between two probability distributions. The proposed approach is flexible and can be applied in any number of dimensions; it allows one to rank climate models taking into account all the moments of the distributions. By selecting the combination of climatic variables and the regions of interest, it is possible to highlight specific model deficiencies. The WD enables a comprehensive evaluation of climate model skill. We apply this approach to a selected number of physical fields, ranking the models in terms of their performance in simulating them, and pinpointing their weaknesses in the simulation of some of the selected physical fields in specific areas of the Earth.


title: Classical Theta Lifts for Higher Metaplectic Covering Group

abstract: The classical theta correspondence establishes a relationship between automorphic representations on special orthogonal groups and automorphic representations on symplectic groups or their double covers. This correspondence is achieved by using as integral kernel a theta series on the metaplectic double cover of a symplectic group that is constructed from the Weil representation. There is also an analogous local correspondence. In this work we present an extension of the classical theta correspondence to higher degree metaplectic covers of symplectic and special orthogonal groups. The key issue here is that for higher degree covers there is no analogue of the Weil representation, and additional ingredients are needed. Our work reflects a broader paradigm: constructions in automorphic forms that work for algebraic groups or their double covers should often extend to higher degree metaplectic covers.


title: Learning About Objects by Learning to Interact with Them

abstract: Much of the remarkable progress in computer vision has been focused around fully supervised learning mechanisms relying on highly curated datasets for a variety of tasks. In contrast, humans often learn about their world with little to no external supervision. Taking inspiration from infants learning from their environment through play and interaction, we present a computational framework to discover objects and learn their physical properties along this paradigm of Learning from Interaction. Our agent, when placed within the near photo-realistic and physics-enabled AI2-THOR environment, interacts with its world and learns about objects, their geometric extents and relative masses, without any external guidance. Our experiments reveal that this agent learns efficiently and effectively; not just for objects it has interacted with before, but also for novel instances from seen categories as well as novel object categories.


title: Riemann-Roch Polynomials of the known Hyperk\"ahler Manifolds

abstract: We compute explicit formulas for the Euler characteristic of line bundles in the two exceptional examples of Hyperk\"ahler Manifolds introduced by O'Grady. In the Appendix Yalong Cao and Chen Jiang use our formulas to compute the Chern numbers for the 10-dimensional example.


title: Lung Segmentation and Nodule Detection in Computed Tomography Scan using a Convolutional Neural Network Trained Adversarially using Turing Test Loss

abstract: Lung cancer is the most common form of cancer found worldwide with a high mortality rate. Early detection of pulmonary nodules by screening with a low-dose computed tomography (CT) scan is crucial for its effective clinical management. Nodules which are symptomatic of malignancy occupy about 0.0125 - 0.025\% of volume in a CT scan of a patient. Manual screening of all slices is a tedious task and presents a high risk of human errors. To tackle this problem we propose a computationally efficient two stage framework. In the first stage, a convolutional neural network (CNN) trained adversarially using Turing test loss segments the lung region. In the second stage, patches sampled from the segmented region are then classified to detect the presence of nodules. The proposed method is experimentally validated on the LUNA16 challenge dataset with a dice coefficient of $0.984\pm0.0007$ for 10-fold cross-validation.


title: Chaotic-like transfers of energy in Hamiltonian PDEs

abstract: We consider the nonlinear cubic Wave, the Hartree and the nonlinear cubic Beam equations on $T^2$ and we prove the existence of different types of solutions which exchange energy between Fourier modes in certain time scales. This exchange can be considered \emph{chaotic-like} since either the choice of activated modes or the time spent in each transfer can be chosen randomly. The key point of the construction of those orbits is the existence of heteroclinic connections between invariant objects and the construction of symbolic dynamics (a Smale horseshoe) for the Birkhoff Normal Form truncation of those equations.


title: Deep Multimodal Transfer-Learned Regression in Data-Poor Domains

abstract: In many real-world applications of deep learning, estimation of a target may rely on various types of input data modes, such as audio-video, image-text, etc. This task can be further complicated by a lack of sufficient data. Here we propose a Deep Multimodal Transfer-Learned Regressor (DMTL-R) for multimodal learning of image and feature data in a deep regression architecture effective at predicting target parameters in data-poor domains. Our model is capable of fine-tuning a given set of pre-trained CNN weights on a small amount of training image data, while simultaneously conditioning on feature information from a complimentary data mode during network training, yielding more accurate single-target or multi-target regression than can be achieved using the images or the features alone. We present results using phase-field simulation microstructure images with an accompanying set of physical features, using pre-trained weights from various well-known CNN architectures, which demonstrate the efficacy of the proposed multimodal approach.


title: Internal Calibration of the PandaX-II Detector with Radon Gaseous Sources

abstract: We have developed a low-energy electron recoil (ER) calibration method with $^{220}$Rn for the PandaX-II detector. $^{220}$Rn, emanated from natural thorium compounds, was fed into the detector through the xenon purification system. From 2017 to 2019, we performed three dedicated calibration campaigns with different radon sources. We studied the detector response to $\alpha$, $\beta$, and $\gamma$ particles with focus on low energy ER events. During the runs in 2017 and 2018, the amount of radioactivity of $^{222}$Rn were on the order of 1\% of that of $^{220}$Rn and thorium particulate contamination was negligible, especially in 2018. We also measured the background contribution from $^{214}$Pb for the first time in PandaX-II with the help from a $^{222}$Rn injection. Calibration strategy with $^{220}$Rn and $^{222}$Rn will be implemented in the upcoming PandaX-4T experiment and can be useful for other xenon-based detectors as well.


title: Further inequalities for the $\mathbb{A}$-numerical radius of certain $2 \times 2$ operator matrices

abstract: Let $\mathbb{A}= \begin{pmatrix} A & 0 \\ 0 & A \\ \end{pmatrix} $ be a $2\times2$ diagonal operator matrix whose each diagonal entry is a bounded positive (semidefinite) linear operator $A$ acting on a complex Hilbert space $\mathcal{H}$. In this paper, we derive several $\mathbb{A}$-numerical radius inequalities for $2\times 2$ operator matrices whose entries are bounded with respect to the seminorm induced by the positive operator $A$ on $\mathcal{H}$. Some applications of our inequalities are also given.


title: Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks

abstract: Despite its success in a wide range of applications, characterizing the generalization properties of stochastic gradient descent (SGD) in non-convex deep learning problems is still an important challenge. While modeling the trajectories of SGD via stochastic differential equations (SDE) under heavy-tailed gradient noise has recently shed light over several peculiar characteristics of SGD, a rigorous treatment of the generalization properties of such SDEs in a learning theoretical framework is still missing. Aiming to bridge this gap, in this paper, we prove generalization bounds for SGD under the assumption that its trajectories can be well-approximated by a \emph{Feller process}, which defines a rich class of Markov processes that include several recent SDE representations (both Brownian or heavy-tailed) as its special case. We show that the generalization error can be controlled by the \emph{Hausdorff dimension} of the trajectories, which is intimately linked to the tail behavior of the driving process. Our results imply that heavier-tailed processes should achieve better generalization; hence, the tail-index of the process can be used as a notion of "capacity metric". We support our theory with experiments on deep neural networks illustrating that the proposed capacity metric accurately estimates the generalization error, and it does not necessarily grow with the number of parameters unlike the existing capacity metrics in the literature.


title: Tensor Method for Optimal Control Problems Constrained by Fractional 3D Elliptic Operator with Variable Coefficients

abstract: We introduce the tensor numerical method for solving optimal control problems that are constrained by fractional 2D and 3D elliptic operators with variable coefficients. We solve the governing equation for the control function which includes a sum of the fractional operator and its inverse, both discretized over large 3D $n\times n \times n$ spacial grids. Using the diagonalization of the arising matrix valued functions in the eigenbasis of the 1D Sturm-Liouville operators, we construct the rank-structured tensor approximation with controllable precision for the discretized fractional elliptic operators and the respective preconditioner. The right-hand side in the constraining equation (the optimal design function) is supposed to be represented in a form of a low-rank canonical tensor. Then the equation for the control function is solved in a tensor structured format by using preconditioned CG iteration with the adaptive rank truncation procedure that also ensures the accuracy of calculations, given an $\varepsilon$-threshold. This method reduces the numerical cost for solving the control problem to $O(n \log n)$ (plus the quadratic term $O(n^2)$ with a small weight), which is superior to the approaches based on the traditional linear algebra tools that yield at least $O(n^3 \log n)$ complexity in the 3D case. The storage for the representation of all 3D nonlocal operators and functions involved is also estimated by $O(n \log n)$. This essentially outperforms the traditional methods operating with fully populated $n^3 \times n^3$ matrices and vectors in $\mathbb{R}^{n^3}$. Numerical tests for 2D/3D control problems indicate the almost linear complexity scaling of the rank truncated PCG iteration in the univariate grid size $n$.


title: Landau-Khalatnikov-Fradkin transformation in three-dimensional quenched QED

abstract: We study the gauge-covariance of the massless fermion propagator in three-dimensional quenched Quantum Electrodynamics in the framework of dimensional regularization in d=3-2\ep. Assuming the finiteness of the quenched perturbative expansion, that is the existence of the limit \ep \to 0, we state that, exactly in d=3, all odd perturbative coefficients, starting with the third order one, should be zero in any gauge.