title: A Holistic Approach to Interpretability in Financial Lending: Models, Visualizations, and Summary-Explanations

abstract: Lending decisions are usually made with proprietary models that provide minimally acceptable explanations to users. In a future world without such secrecy, what decision support tools would one want to use for justified lending decisions? This question is timely, since the economy has dramatically shifted due to a pandemic, and a massive number of new loans will be necessary in the short term. We propose a framework for such decisions, including a globally interpretable machine learning model, an interactive visualization of it, and several types of summaries and explanations for any given decision. The machine learning model is a two-layer additive risk model, which resembles a two-layer neural network, but is decomposable into subscales. In this model, each node in the first (hidden) layer represents a meaningful subscale model, and all of the nonlinearities are transparent. Our online visualization tool allows exploration of this model, showing precisely how it came to its conclusion. We provide three types of explanations that are simpler than, but consistent with, the global model: case-based reasoning explanations that use neighboring past cases, a set of features that were the most important for the model's prediction, and summary-explanations that provide a customized sparse explanation for any particular lending decision made by the model. Our framework earned the FICO recognition award for the Explainable Machine Learning Challenge, which was the first public challenge in the domain of explainable machine learning.


title: Uncertainties in the $^{18}$F(p,$\alpha$)$^{15}$O reaction rate in classical novae

abstract: Context. Direct observation of gamma-ray emission from the decay of $^{18}$F ejected in classical nova outbursts remains a major focus of the nuclear astrophysics community. However, modeling the abundance of ejected $^{18}$F, and thus the predicted detectability distance of a gamma-ray signal near 511 keV emitted from these transient thermonuclear episodes, is hampered by significant uncertainties in our knowledge of the key $^{18}$F(p,$\alpha$) reaction rate. Aims. We analyze uncertainties in the most recent nuclear physics experimental results employed to calculate the $^{18}$F(p,$\alpha$) reaction rate. Our goal is to determine which uncertainties have the most profound influence on the predicted abundance of $^{18}$F ejected from novae, in order to guide future experimental works. Methods. We calculated a wide range of $^{18}$F(p,$\alpha$) reaction rates using R-Matrix formalism, allowing us to take into account all interference effects. Using a selection of 16 evenly-spaced rates over the full range, we performed 16 new hydrodynamic nova simulations. Results. We performed one of the most thorough theoretical studies of the impact of the $^{18}$F(p,$\alpha$) reaction in classical novae to date. The $^{18}$F(p,$\alpha$) rate remains highly uncertain at nova temperatures, resulting in a factor ~10 uncertainty in the predicted abundance of $^{18}$F ejected from nova explosions. We also found that the abundance of $^{18}$F may be strongly correlated with that of $^{19}$F. Conclusions. Despite numerous nuclear physics uncertainties affecting the $^{18}$F(p,$\alpha$) reaction rate, which are dominated by unknown interference signs between 1/2$^+$ and 3/2$^+$ resonances, future experimental work should focus on firmly and precisely determining the directly measurable quantum properties of the subthreshold states in the compound nucleus $^{19}$Ne near 6.13 and 6.29 MeV.


title: Defending Democracy: Using Deep Learning to Identify and Prevent Misinformation

abstract: The rise in online misinformation in recent years threatens democracies by distorting authentic public discourse and causing confusion, fear, and even, in extreme cases, violence. There is a need to understand the spread of false content through online networks for developing interventions that disrupt misinformation before it achieves virality. Using a Deep Bidirectional Transformer for Language Understanding (BERT) and propagation graphs, this study classifies and visualizes the spread of misinformation on a social media network using publicly available Twitter data. The results confirm prior research around user clusters and the virality of false content while improving the precision of deep learning models for misinformation detection. The study further demonstrates the suitability of BERT for providing a scalable model for false information detection, which can contribute to the development of more timely and accurate interventions to slow the spread of misinformation in online environments.


title: Influence Estimation and Maximization via Neural Mean-Field Dynamics

abstract: We propose a novel learning framework using neural mean-field (NMF) dynamics for inference and estimation problems on heterogeneous diffusion networks. Our new framework leverages the Mori-Zwanzig formalism to obtain an exact evolution equation of the individual node infection probabilities, which renders a delay differential equation with memory integral approximated by learnable time convolution operators. Directly using information diffusion cascade data, our framework can simultaneously learn the structure of the diffusion network and the evolution of node infection probabilities. Connections between parameter learning and optimal control are also established, leading to a rigorous and implementable algorithm for training NMF. Moreover, we show that the projected gradient descent method can be employed to solve the challenging influence maximization problem, where the gradient is computed extremely fast by integrating NMF forward in time just once in each iteration. Extensive empirical studies show that our approach is versatile and robust to variations of the underlying diffusion network models, and significantly outperform existing approaches in accuracy and efficiency on both synthetic and real-world data.


title: SBML2Modelica: integrating biochemical models within open-standard simulation ecosystems

abstract: Motivation: SBML is the most widespread language for the definition of biochemical models. Although dozens of SBML simulators are available, there is a general lack of support to the integration of SBML models within open-standard general-purpose simulation ecosystems. This hinders co-simulation and integration of SBML models within larger model networks, in order to, e.g. enable in silico clinical trials of drugs, pharmacological protocols, or engineering artefacts such as biomedical devices against Virtual Physiological Human models. Modelica is one of the most popular existing open-standard general-purpose simulation languages, supported by many simulators. Modelica models are especially suited for the definition of complex networks of heterogeneous models from virtually all application domains. Models written in Modelica (and in 100+ other languages) can be readily exported into black-box Functional Mock-Up Units (FMUs), and seamlessly co-simulated and integrated into larger model networks within open-standard language-independent simulation ecosystems. Results: In order to enable SBML model integration within heterogeneous model networks, we present SBML2Modelica, a software system translating SBML models into well-structured, user-intelligible, easily modifiable Modelica models. SBML2Modelica is SBML Level 3 Version 2-compliant and succeeds on 96.47% of the SBML Test Suite Core (with a few rare, intricate and easily avoidable combinations of constructs unsupported and cleanly signalled to the user). Our experimental campaign on 613 models from the BioModels database (with up to 5438 variables) shows that the major open-source (general-purpose) Modelica and FMU simulators achieve performance comparable to state-of-the-art specialized SBML simulators. Availability and implementation: https://bitbucket.org/mclab/sbml2modelica


title: Visible-light Phase Curves from the Second Year of the TESS Primary Mission

abstract: We carried out a systematic study of full-orbit phase curves for known transiting systems in the northern ecliptic sky that were observed during Year 2 of the TESS primary mission. We applied the same methodology for target selection, data processing, and light-curve fitting as we did in our Year 1 study. Out of the 15 transiting systems selected for analysis, seven - HAT-P-7, KELT-1, KELT-9, KELT-16, KELT-20, Kepler-13A, and WASP-12 - show statistically significant secondary eclipses and day-night atmospheric brightness modulations. Small eastward dayside hotspot offsets were measured for KELT-9b and WASP-12b. KELT-1, Kepler-13A, and WASP-12 show additional phase-curve variability attributed to the tidal distortion of the host star; the amplitudes of these signals are consistent with theoretical predictions. We combined occultation measurements from TESS and Spitzer to compute dayside brightness temperatures, TESS-band geometric albedos, Bond albedos, and phase integrals for several systems. The new albedo values solidify the previously reported trend between dayside temperature and geometric albedo for planets with $1500<T_{\mathrm{day}}<3000$ K. For Kepler-13Ab, we carried out an atmospheric retrieval of the full secondary eclipse spectrum, which revealed a non-inverted temperature-pressure profile, significant H$_{2}$O and K absorption in the near-infrared, evidence for strong optical atmospheric opacity due to sodium, and a confirmation of the high geometric albedo inferred from our simpler analysis. We explore the implications of the phase integrals (ratios of Bond to geometric albedos) for understanding exoplanet clouds. We also report updated transit ephemerides for all of the systems studied in this work.


title: Insights on the coupling between vibronically active molecular vibrations and lattice phonons in molecular nanomagnets

abstract: Spin-lattice relaxation is a key open problem to understand the spin dynamics of single-molecule magnets and molecular spin qubits. While modelling the coupling between spin states and local vibrations allows to determine the more relevant molecular vibrations for spin relaxation, this is not sufficient to explain how energy is dissipated towards the thermal bath. Herein, we employ a simple and efficient model to examine the coupling of local vibrational modes with long-wavelength longitudinal and transverse phonons in the clock-like spin qubit [Ho(W$_5$O$_{18}$)$_2$]$^{9-}$. We find that in crystals of this polyoxometalate the vibrational mode previously found to be vibronically active at low temperature does not couple significantly to lattice phonons. This means that further intramolecular energy transfer via anharmonic vibrations is necessary for spin relaxation in this system. Finally, we discuss implications for the spin-phonon coupling of [Ho(W$_5$O$_{18}$)$_2$]$^{9-}$ deposited on a MgO (001) substrate, offering a simple methodology that can be extrapolated to estimate the effects on spin relaxation of different surfaces, including 2D materials.


title: Collection and harmonization of system logs and prototypal Analytics services with the Elastic (ELK) suite at the INFN-CNAF computing centre

abstract: The distributed Grid infrastructure for High Energy Physics experiments at the Large Hadron Collider (LHC) in Geneva comprises a set of computing centres, spread all over the world, as part of the Worldwide LHC Computing Grid (WLCG). In Italy, the Tier-1 functionalities are served by the INFN-CNAF data center, which provides also computing and storage resources to more than twenty non-LHC experiments. For this reason, a high amount of logs are collected each day from various sources, which are highly heterogeneous and difficult to harmonize. In this contribution, a working implementation of a system that collects, parses and displays the log information from CNAF data sources and the investigation of a Machine Learning based predictive maintenance system, is presented.


title: Bridging the Gap Between Target Networks and Functional Regularization

abstract: Bootstrapping is behind much of the successes of deep Reinforcement Learning. However, learning the value function via bootstrapping often leads to unstable training due to fast-changing target values. Target Networks are employed to stabilize training by using an additional set of lagging parameters to estimate the target values. Despite the popularity of Target Networks, their effect on the optimization is still misunderstood. In this work, we show that they act as an implicit regularizer which can be beneficial in some cases, but also have disadvantages such as being inflexible and can result in instabilities, even when vanilla TD(0) converges. To overcome these issues, we propose an explicit Functional Regularization alternative that is flexible and a convex regularizer in function space and we theoretically study its convergence. We conduct an experimental study across a range of environments, discount factors, and off-policiness data collections to investigate the effectiveness of the regularization induced by Target Networks and Functional Regularization in terms of performance, accuracy, and stability. Our findings emphasize that Functional Regularization can be used as a drop-in replacement for Target Networks and result in performance improvement. Furthermore, adjusting both the regularization weight and the network update period in Functional Regularization can result in further performance improvements compared to solely adjusting the network update period as typically done with Target Networks. Our approach also enhances the ability to networks to recover accurate $Q$-values.


title: Sigma-Delta and Distributed Noise-Shaping Quantization Methods for Random Fourier Features

abstract: We propose the use of low bit-depth Sigma-Delta and distributed noise-shaping methods for quantizing the Random Fourier features (RFFs) associated with shift-invariant kernels. We prove that our quantized RFFs -- even in the case of $1$-bit quantization -- allow a high accuracy approximation of the underlying kernels, and the approximation error decays at least polynomially fast as the dimension of the RFFs increases. We also show that the quantized RFFs can be further compressed, yielding an excellent trade-off between memory use and accuracy. Namely, the approximation error now decays exponentially as a function of the bits used. Moreover, we empirically show by testing the performance of our methods on several machine learning tasks that our method compares favorably to other state of the art quantization methods in this context.


title: Consensus Multiplicative Weights Update: Learning to Learn using Projector-based Game Signatures

abstract: Cheung and Piliouras (2020) recently showed that two variants of the Multiplicative Weights Update method - OMWU and MWU - display opposite convergence properties depending on whether the game is zero-sum or cooperative. Inspired by this work and the recent literature on learning to optimize for single functions, we introduce a new framework for learning last-iterate convergence to Nash Equilibria in games, where the update rule's coefficients (learning rates) along a trajectory are learnt by a reinforcement learning policy that is conditioned on the nature of the game: \textit{the game signature}. We construct the latter using a new decomposition of two-player games into eight components corresponding to commutative projection operators, generalizing and unifying recent game concepts studied in the literature. We compare the performance of various update rules when their coefficients are learnt, and show that the RL policy is able to exploit the game signature across a wide range of game types. In doing so, we introduce CMWU, a new algorithm that extends consensus optimization to the constrained case, has local convergence guarantees for zero-sum bimatrix games, and show that it enjoys competitive performance on both zero-sum games with constant coefficients and across a spectrum of games when its coefficients are learnt.


title: Optical conductivity of a Dirac-Fermi liquid

abstract: A Dirac-Fermi liquid (DFL)--a doped system with Dirac spectrum--is an important example of a non-Galilean-invariant Fermi liquid (FL). Real-life realizations of a DFL include, e.g., doped graphene, surface states of three-dimensional (3D) topological insulators, and 3D Dirac/Weyl metals. We study the optical conductivity of a DFL arising from intraband electron-electron scattering. It is shown that the effective current relaxation rate behaves as $1/\tau_{J}\propto \left(\omega^2+4\pi^2 T^2\right)\left(3\omega^2+8\pi^2 T^2\right)$ for $\max\{\omega, T\}\ll \mu$, where $\mu$ is the chemical potential, with an additional logarithmic factor in two dimensions. In graphene, the quartic form of $1/\tau_{J}$ competes with a small FL-like term, $\propto\omega^2+4\pi^2 T^2$, due to trigonal warping of the Fermi surface. We also calculated the dynamical charge susceptibility, $\chi_\mathrm{c}({\bf q},\omega)$, outside the particle-hole continua and to one-loop order in the dynamically screened Coulomb interaction. For a 2D DFL, the imaginary part of $\chi_\mathrm{c}({\bf q},\omega)$ scales as $q^2\omega\ln|\omega|$ and $q^4/\omega^3$ for frequencies larger and smaller than the plasmon frequency at given $q$, respectively. The small-$q$ limit of $\mathrm{Im} \chi_\mathrm{c}({\bf q},\omega)$ reproduces our result for the conductivity via the Einstein relation.


title: Be Considerate: Objectives, Side Effects, and Deciding How to Act

abstract: Recent work in AI safety has highlighted that in sequential decision making, objectives are often underspecified or incomplete. This gives discretion to the acting agent to realize the stated objective in ways that may result in undesirable outcomes. We contend that to learn to act safely, a reinforcement learning (RL) agent should include contemplation of the impact of its actions on the wellbeing and agency of others in the environment, including other acting agents and reactive processes. We endow RL agents with the ability to contemplate such impact by augmenting their reward based on expectation of future return by others in the environment, providing different criteria for characterizing impact. We further endow these agents with the ability to differentially factor this impact into their decision making, manifesting behavior that ranges from self-centred to self-less, as demonstrated by experiments in gridworld environments.


title: Examples of symplectic non-leaves

abstract: This paper deals with the following question: which manifolds can be realized as leaves of codimension-1 symplectic foliations on closed manifolds? We first observe that leaves of symplectic foliations are necessarily strongly geometrically bounded. We show that a symplectic structure which admits an exhaustion by compacts with (convex) contact boundary can be deformed to a strongly geometrically bounded one. We then give examples of smooth manifolds which admit a strongly geometrically bounded symplectic form and can be realized as a smooth leaf, but not as a symplectic leaf for any choice of symplectic form on them. Lastly, we show that the (complex) blowup of 2n-dimensional Euclidean space at infinitely many points, both admits strongly geometrically bounded symplectic forms for which it can and cannot be realized as a symplectic leaf.


title: Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions

abstract: Generative adversarial networks (GANs) are among the most successful models for learning high-complexity, real-world distributions. However, in theory, due to the highly non-convex, non-concave landscape of the minmax training objective, GAN remains one of the least understood deep learning models. In this work, we formally study how GANs can efficiently learn certain hierarchically generated distributions that are close to the distribution of real-life images. We prove that when a distribution has a structure that we refer to as Forward Super-Resolution, then simply training generative adversarial networks using stochastic gradient descent ascent (SGDA) can learn this distribution efficiently, both in sample and time complexities. We also provide empirical evidence that our assumption "forward super-resolution" is very natural in practice, and the underlying learning mechanisms that we study in this paper (to allow us efficiently train GAN via SGDA in theory) simulates the actual learning process of GANs on real-world problems.


title: Relative $K$-theory for $C^*$-algebras

abstract: Given C$^*$-algebras $A$ and $B$ and a $^*$-homomorphism $\phi:A\rightarrow B$, we adopt the portrait of the relative $K$-theory $K_*(\phi)$ due to Karoubi using Banach categories and Banach functors. We show that the elements of the relative groups may be represented in a simple form. We prove the existence of two six-term exact sequences, and we use these sequences to deduce the fact that the relative theory is isomorphic, in a natural way, to the $K$-theory of the mapping cone.


title: Single-shot quantum error correction with the three-dimensional subsystem toric code

abstract: We introduce a new topological quantum code, the three-dimensional subsystem toric code (3D STC), which is a generalization of the stabilizer toric code. The 3D STC can be realized by measuring geometrically-local parity checks of weight at most three on the cubic lattice with open boundary conditions. We prove that single-shot quantum error correction (QEC) is possible with the 3D STC, i.e., one round of local parity-check measurements suffices to perform reliable QEC even in the presence of measurement errors. We also propose an efficient single-shot QEC strategy for the 3D STC and investigate its performance. In particular, we numerically estimate the resulting storage threshold against independent bit-flip, phase-flip and measurement errors to be $p_\text{STC} \approx 1.045\%$. Such a high threshold together with local parity-check measurements of small weight make the 3D STC particularly appealing for realizing fault-tolerant quantum computing.


title: The NewHorizon Simulation -- To Bar Or Not To Bar

abstract: We use the NewHorizon simulation to study the redshift evolution of bar properties and fractions within galaxies in the stellar masses range $M_{\star} = 10^{7.25} - 10^{11.4} \ \rm{M}_{\odot}$ over the redshift range $z = 0.25 - 1.3$. We select disc galaxies using stellar kinematics as a proxy for galaxy morphology. We employ two different automated bar detection methods, coupled with visual inspection, resulting in observable bar fractions of $f_{\rm bar} = 0.070_{{-0.012}}^{{+0.018}}$ at $z\sim$ 1.3, decreasing to $f_{\rm bar} = 0.011_{{-0.003}}^{{+0.014}}$ at $z\sim$ 0.25. Only one galaxy is visually confirmed as strongly barred in our sample. This bar is hosted by the most massive disk and only survives from $z=1.3$ down to $z=0.7$. Such a low bar fraction, in particular amongst Milky Way-like progenitors, highlights a missing bars problem, shared by literally all cosmological simulations with spatial resolution $<$100 pc to date. The analysis of linear growth rates, rotation curves and derived summary statistics of the stellar, gas and dark matter components suggest that galaxies with stellar masses below $10^{9.5}-10^{10} \ \rm{M}_{\odot}$ in NewHorizon appear to be too dominated by dark matter to form a bar, while more massive galaxies typically have formed large bulges that prevent bar persistence at low redshift. This investigation confirms that the evolution of the bar fraction puts stringent constraints on the assembly history of baryons and dark matter onto galaxies.


title: The Closer You Look, The More You Learn: A Grey-box Approach to Protocol State Machine Learning

abstract: In this paper, we propose a new approach to infer state machine models from protocol implementations. Our method, STATEINSPECTOR, learns protocol states by using novel program analyses to combine observations of run-time memory and I/O. It requires no access to source code and only lightweight execution monitoring of the implementation under test. We demonstrate and evaluate STATEINSPECTOR's effectiveness on numerous TLS and WPA/2 implementations. In the process, we show STATEINSPECTOR enables deeper state discovery, increased learning efficiency, and more insightful post-mortem analyses than existing approaches. Further to improved learning, our method led us to discover several concerning deviations from the standards and a high impact vulnerability in a prominent Wi-Fi implementation.


title: ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure

abstract: Curvature in form of the Hessian or its generalized Gauss-Newton (GGN) approximation is valuable for algorithms that rely on a local model for the loss to train, compress, or explain deep networks. Existing methods based on implicit multiplication via automatic differentiation or Kronecker-factored block diagonal approximations do not consider noise in the mini-batch. We present ViViT, a curvature model that leverages the GGN's low-rank structure without further approximations. It allows for efficient computation of eigenvalues, eigenvectors, as well as per-sample first- and second-order directional derivatives. The representation is computed in parallel with gradients in one backward pass and offers a fine-grained cost-accuracy trade-off, which allows it to scale. We demonstrate this by conducting performance benchmarks and substantiate ViViT's usefulness by studying the impact of noise on the GGN's structural properties during neural network training.


title: Extracting Dynamical Frequencies from Invariants of Motion in Finite-Dimensional Nonlinear Integrable Systems

abstract: Integrable dynamical systems play an important role in many areas of science, including accelerator and plasma physics. An integrable dynamical system with $n$ degrees of freedom (DOF) possesses $n$ nontrivial integrals of motion, and can be solved, in principle, by covering the phase space with one or more charts in which the dynamics can be described using action-angle coordinates. To obtain the frequencies of motion, both the transformation to action-angle coordinates and its inverse must be known in explicit form. However, no general algorithm exists for constructing this transformation explicitly from a set of $n$ known (and generally coupled) integrals of motion. In this paper we describe how one can determine the dynamical frequencies of the motion as functions of these $n$ integrals in the absence of explicitly-known action-angle variables, and we provide several examples.


title: Dynamics of specialization in neural modules under resource constraints

abstract: It has long been believed that the brain is highly modular both in terms of structure and function, although recent evidence has led some to question the extent of both types of modularity. We used artificial neural networks to test the hypothesis that structural modularity is sufficient to guarantee functional specialization, and find that in general, this doesn't necessarily hold. We then systematically tested which features of the environment and network do lead to the emergence of specialization. We used a simple toy environment, task and network, allowing us precise control, and show that in this setup, several distinct measures of specialization give qualitatively similar results. We further find that in this setup (1) specialization can only emerge in environments where features of that environment are meaningfully separable, (2) specialization preferentially emerges when the network is strongly resource-constrained, and (3) these findings are qualitatively similar across the different variations of network architectures that we tested, but that the quantitative relationships depend on the precise architecture. Finally, we show that functional specialization varies dynamically across time, and demonstrate that these dynamics depend on both the timing and bandwidth of information flow in the network. We conclude that a static notion of specialization, based on structural modularity, is likely too simple a framework for understanding intelligence in situations of real-world complexity, from biology to brain-inspired neuromorphic systems. We propose that thoroughly stress testing candidate definitions of functional modularity in simplified scenarios before extending to more complex data, network models and electrophysiological recordings is likely to be a fruitful approach.


title: Generic differential equations are strongly minimal

abstract: In this manuscript we develop a new technique for showing that a nonlinear algebraic differential equation is strongly minimal based on the recently developed notion of the degree of nonminimality of Freitag and Moosa. Our techniques are sufficient to show that generic order $h$ differential equations with nonconstant coefficients are strongly minimal, answering a question of Poizat (1980).


title: Constraint-based Relational Verification

abstract: In recent years they have been numerous works that aim to automate relational verification. Meanwhile, although Constrained Horn Clauses (CHCs) empower a wide range of verification techniques and tools, they lack the ability to express hyperproperties beyond $k$-safety such as generalized non-interference and co-termination. This paper describes a novel and fully automated constraint-based approach to relational verification. We first introduce a new class of predicate Constraint Satisfaction Problems called pfwCSP where constraints are represented as clauses modulo first-order theories over predicate variables of three kinds: ordinary, well-founded, or functional. This generalization over CHCs permits arbitrary (i.e., possibly non-Horn) clauses, well-foundedness constraints, functionality constraints, and is capable of expressing these relational verification problems. Our approach enables us to express and automatically verify problem instances that require non-trivial (i.e., non-sequential and non-lock-step) self-composition by automatically inferring appropriate schedulers (or alignment) that dictate when and which program copies move. To solve problems in this new language, we present a constraint solving method for pfwCSP based on stratified CounterExample-Guided Inductive Synthesis (CEGIS) of ordinary, well-founded, and functional predicates. We have implemented the proposed framework and obtained promising results on diverse relational verification problems that are beyond the scope of the previous verification frameworks.


title: A Possible Hadronic Origin of TeV Photon Emission from SNR G106.3+2.7

abstract: In our model, the acceleration and propagation of particles from the Bohm-like diffusion region inside the SNR to the Galactic diffusion region outside the SNR are described through nonlinear diffusive shock acceleration (NLDSA). The main content of our NLDSA model is to solve the hydrodynamic equations numerically for gas density, gas velocity, gas pressure as well as the equation for the quasi-isotropic particle momentum distribution. The consequent multi-band non-thermal emission is from two different regions, namely the acceleration region and the escaping region. Our model is capable of explaining the multi-band photon emission via the dominant synchrotron radiation of the electrons accelerated inside the SNR; and the photons with energy of $\gtrsim$ GeV are naturally produced by the protons inside and outside the SNR. Moreover, the photons in the energy range of $\sim 1 - \sim 100$ TeV are due to the interaction of escaped protons with dense molecular clouds. For the photons with energy of $E_\gamma \gtrsim $ 1 GeV from SNR G106.3+2.7, our results here favor a hadronic origin, where the photons in the energy range of $\sim 1$ GeV to $\sim 1$ TeV are produced inside the SNR through proton-proton interaction, while the photons with $E_\gamma \gtrsim 1$ TeV originate from the interaction of escaped protons with a dense molecular cloud.


title: Fundamental tradeoffs between memorization and robustness in random features and neural tangent regimes

abstract: This work studies the (non)robustness of two-layer neural networks in various high-dimensional linearized regimes. We establish fundamental trade-offs between memorization and robustness, as measured by the Sobolev-seminorm of the model w.r.t the data distribution, i.e the square root of the average squared $L_2$-norm of the gradients of the model w.r.t the its input. More precisely, if $n$ is the number of training examples, $d$ is the input dimension, and $k$ is the number of hidden neurons in a two-layer neural network, we prove for a large class of activation functions that, if the model memorizes even a fraction of the training, then its Sobolev-seminorm is lower-bounded by (i) $\sqrt{n}$ in case of infinite-width random features (RF) or neural tangent kernel (NTK) with $d \gtrsim n$; (ii) $\sqrt{n}$ in case of finite-width RF with proportionate scaling of $d$ and $k$; and (iii) $\sqrt{n/k}$ in case of finite-width NTK with proportionate scaling of $d$ and $k$. Moreover, all of these lower-bounds are tight: they are attained by the min-norm / least-squares interpolator (when $n$, $d$, and $k$ are in the appropriate interpolating regime). All our results hold as soon as data is log-concave isotropic, and there is label-noise, i.e the target variable is not a deterministic function of the data / features. We empirically validate our theoretical results with experiments. Accidentally, these experiments also reveal for the first time, (iv) a multiple-descent phenomenon in the robustness of the min-norm interpolator.


title: The absence of superconductivity in the next-to-leading order Ginzburg-Landau functional for Bardeen-Cooper-Schrieffer superconductor

abstract: Shortly after the Gor'kov microscopic derivation of the Ginzburg-Landau (GL) model via a small order parameter expansion in Bardeen-Cooper-Schrieffer theory of superconductivity, the derivation was carried to next-to-leading order in that parameter and its spatial derivatives. The aim was to obtain a generalized GL free energy that approximates the microscopic model better. Since 1960s, multiple works have claimed or implicitly assumed that this extended GL model corresponds to the free energy and has solutions in the form of local minima describing superconductivity, such as vortex solutions. In contrast to this, we prove that this extended GL functional does not represent free energy since it does not have any solutions in the form of minima. Accordingly, it cannot be used to describe superconducting states.


title: Superconducting on-chip spectrometer for mesoscopic quantum systems

abstract: Spectroscopy is a powerful tool to probe physical, chemical, and biological systems. Recent advances in microfabrication have introduced novel, intriguing mesoscopic quantum systems including superconductor-semiconductor hybrid devices and topologically non-trivial electric circuits. A sensitive, general purpose spectrometer to probe the energy levels of these systems is lacking. We propose an on-chip absorption spectrometer functioning well into the millimeter wave band which is based on a voltage-biased superconducting quantum interference device. We demonstrate the capabilities of the spectrometer by coupling it to a variety of superconducting systems, probing phenomena such as quasiparticle and plasma excitations. We perform spectroscopy of a microscopic tunable non-linear resonator in the 40-50 GHz range and measure transitions to highly excited states. The Josephson junction spectrometer, with outstanding frequency range, sensitivity, and coupling strength will enable new experiments in linear and non-linear spectroscopy of novel mesoscopic systems.


title: A Novel Test of Quasar Orientation

abstract: The orientation of the disk of material accreting onto supermassive black holes that power quasars is one of most important quantities that are needed to understand quasars -- both individually and in the ensemble average. We present a hypothesis for determining comparatively edge-on orientation in a subset of quasars (both radio loud and radio quiet). If confirmed, this orientation indicator could be applicable to individual quasars without reference to radio or X-ray data and could identify some 10-20% of quasars as being more edge-on than average, based only on moderate resolution and signal-to-noise spectroscopy covering the CIV 1549A emission feature. We present a test of said hypothesis using X-ray observations and identify additional data that are needed to confirm this hypothesis and calibrate the metric.


title: Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering

abstract: Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neural scene representations are a promising approach to 3D scene understanding. In this work, we propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field parameterized via a neural implicit representation. Rendering a ray from an LFN requires only a single network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or volumetric based renderers in 3D-structured neural scene representations. In the setting of simple scenes, we leverage meta-learning to learn a prior over LFNs that enables multi-view consistent light field reconstruction from as little as a single image observation. This results in dramatic reductions in time and memory complexity, and enables real-time rendering. The cost of storing a 360-degree light field via an LFN is two orders of magnitude lower than conventional methods such as the Lumigraph. Utilizing the analytical differentiability of neural implicit representations and a novel parameterization of light space, we further demonstrate the extraction of sparse depth maps from LFNs.


title: Horospherical invariant measures and a rank dichotomy for Anosov groups

abstract: Let $G=\prod_{i=1}^{r} G_i$ be a product of simple real algebraic groups of rank one and $\Gamma$ an Anosov subgroup of $G$ with respect to a minimal parabolic subgroup. For each $v$ in the interior of a positive Weyl chamber, let $\mathcal R_v\subset\Gamma\backslash G$ denote the Borel subset of all points with recurrent $\exp (\mathbb R_+ v)$-orbits. For a maximal horospherical subgroup $N$ of $G$, we show that the $N$-action on ${\mathcal R}_v$ is uniquely ergodic if $r={rank}(G)\le 3$ and $v$ belongs to the interior of the limit cone of $\Gamma$, and that there exists no $N$-invariant {Radon} measure on $\mathcal R_v$ otherwise.


title: MERLOT: Multimodal Neural Script Knowledge Models

abstract: As humans, we understand events in the visual world contextually, performing multimodal reasoning across time to make inferences about the past, present, and future. We introduce MERLOT, a model that learns multimodal script knowledge by watching millions of YouTube videos with transcribed speech -- in an entirely label-free, self-supervised manner. By pretraining with a mix of both frame-level (spatial) and video-level (temporal) objectives, our model not only learns to match images to temporally corresponding words, but also to contextualize what is happening globally over time. As a result, MERLOT exhibits strong out-of-the-box representations of temporal commonsense, and achieves state-of-the-art performance on 12 different video QA datasets when finetuned. It also transfers well to the world of static images, allowing models to reason about the dynamic context behind visual scenes. On Visual Commonsense Reasoning, MERLOT answers questions correctly with 80.6% accuracy, outperforming state-of-the-art models of similar size by over 3%, even those that make heavy use of auxiliary supervised data (like object bounding boxes). Ablation analyses demonstrate the complementary importance of: 1) training on videos versus static images; 2) scaling the magnitude and diversity of the pretraining video corpus; and 3) using diverse objectives that encourage full-stack multimodal reasoning, from the recognition to cognition level.


title: Aligning Pretraining for Detection via Object-Level Contrastive Learning

abstract: Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning. Such generality for transfer learning, however, sacrifices specificity if we are interested in a certain downstream task. We argue that this could be sub-optimal and thus advocate a design principle which encourages alignment between the self-supervised pretext task and the downstream task. In this paper, we follow this principle with a pretraining method specifically designed for the task of object detection. We attain alignment in the following three aspects: 1) object-level representations are introduced via selective search bounding boxes as object proposals; 2) the pretraining network architecture incorporates the same dedicated modules used in the detection pipeline (e.g. FPN); 3) the pretraining is equipped with object detection properties such as object-level translation invariance and scale invariance. Our method, called Selective Object COntrastive learning (SoCo), achieves state-of-the-art results for transfer performance on COCO detection using a Mask R-CNN framework. Code is available at https://github.com/hologerry/SoCo.


title: Associating Objects with Transformers for Video Object Segmentation

abstract: This paper investigates how to realize better and more efficient embedding learning to tackle the semi-supervised video object segmentation under challenging multi-object scenarios. The state-of-the-art methods learn to decode features with a single positive object and thus have to match and segment each target separately under multi-object scenarios, consuming multiple times computing resources. To solve the problem, we propose an Associating Objects with Transformers (AOT) approach to match and decode multiple objects uniformly. In detail, AOT employs an identification mechanism to associate multiple targets into the same high-dimensional embedding space. Thus, we can simultaneously process multiple objects' matching and segmentation decoding as efficiently as processing a single object. For sufficiently modeling multi-object association, a Long Short-Term Transformer is designed for constructing hierarchical matching and propagation. We conduct extensive experiments on both multi-object and single-object benchmarks to examine AOT variant networks with different complexities. Particularly, our R50-AOT-L outperforms all the state-of-the-art competitors on three popular benchmarks, i.e., YouTube-VOS (84.1% J&F), DAVIS 2017 (84.9%), and DAVIS 2016 (91.1%), while keeping more than $3\times$ faster multi-object run-time. Meanwhile, our AOT-T can maintain real-time multi-object speed on the above benchmarks. Based on AOT, we ranked 1st in the 3rd Large-scale VOS Challenge.


title: Singular Dynamic Mode Decompositions

abstract: This manuscript is aimed at addressing several long standing limitations of dynamic mode decompositions in the application of Koopman analysis. Principle among these limitations are the convergence of associated Dynamic Mode Decomposition algorithms and the existence of Koopman modes. To address these limitations, two major modifications are made, where Koopman operators are removed from the analysis in light of Liouville operators (known as Koopman generators in special cases), and these operators are shown to be compact for certain pairs of Hilbert spaces selected separately as the domain and range of the operator. While eigenfunctions are discarded in the general analysis, a viable reconstruction algorithm is still demonstrated, and the sacrifice of eigenfunctions realizes the theoretical goals of DMD analysis that have yet to be achieved in other contexts. However, in the case where the domain is embedded in the range, an eigenfunction approach is still achievable, where a more typical DMD routine is established, but that leverages a finite rank representation that converges in norm. The manuscript concludes with the description of two Dynamic Mode Decomposition algorithms that converges when a dense collection of occupation kernels, arising from the data, are leveraged in the analysis.

categories: eess.SY cs.LG cs.NA cs.SY math.DS math.FA math.NA


title: Entanglement Wedge Cross Section Inequalities from Replicated Geometries

abstract: We generalize the constructions for the multipartite reflected entropy in order to construct spacetimes capable of representing multipartite entanglement wedge cross sections of differing party number as Ryu-Takayanagi surfaces on a single replicated geometry. We devise a general algorithm for such constructions for arbitrary party number and demonstrate how such methods can be used to derive novel inequalities constraining mulipartite entanglement wedge cross sections.


title: Resonant optical topological Hall conductivity from skyrmions

abstract: We study the high frequency Hall conductivity in a two-dimensional (2D) model of conduction electrons coupled to a background magnetic skyrmion texture via an effective Hund's coupling term. For an ordered skyrmion crystal, a Kubo formula calculation using the basis of skyrmion crystal Chern bands reveals a resonant Hall response at a frequency set by the Hund's coupling: $\hbar\omega_{\text{res}} \approx J_H$. A complementary real-space Kubo formula calculation for an isolated skyrmion in a box reveals a similar resonant Hall response. A linear relation between the area under the Hall resonant curve and the skyrmion density is discovered numerically and is further elucidated using a gradient expansion which is valid for smooth textures and a local approximation based on a spin-trimer calculation. We point out the issue of distinguishing this skyrmion contribution from a similar feature arising from spin-orbit interactions, as demonstrated in a model for Rashba spin-orbit coupled electrons in a collinear ferromagnet, which is analogous to the difficulty of unambiguously separating the d.c. topological Hall effect from the anomalous Hall effect. The resonant feature in the high frequency topological Hall effect is proposed to provide a potentially useful local optical signature of skyrmions via probes such as scanning magneto-optical Kerr microscopy.


title: A simple description of holographic domain walls in confining theories -- extended hydrodynamics

abstract: In the context of theories with a first order phase transition, we propose a general covariant description of coexisting phases separated by domain walls using an additional order parameter-like degree of freedom. In the case of a holographic Witten model with a confining and deconfined phase, the resulting model extends hydrodynamics and has a simple formulation in terms of a spacetime action with corresponding expressions for the energy-momentum tensor. The proposed description leads to simple analytic profiles of domain walls, including expressions for surface tension density, which agree nicely with holographic numerical solutions, despite the apparent complexity of those gravitational backgrounds.


title: Modeling Dense Star Clusters in the Milky Way and Beyond with the Cluster Monte Carlo Code

abstract: We describe the public release of the Cluster Monte Carlo Code (CMC) a parallel, star-by-star $N$-body code for modeling dense star clusters. CMC treats collisional stellar dynamics using H\'enon's method, where the cumulative effect of many two-body encounters is statistically reproduced as a single effective encounter between nearest-neighbor particles on a relaxation timescale. The star-by-star approach allows for the inclusion of additional physics, including strong gravitational three- and four-body encounters, two-body tidal and gravitational-wave captures, mass loss in arbitrary galactic tidal fields, and stellar evolution for both single and binary stars. The public release of CMC is pinned directly to the COSMIC population synthesis code, allowing dynamical star cluster simulations and population synthesis studies to be performed using identical assumptions about the stellar physics and initial conditions. As a demonstration, we present two examples of star cluster modeling: first, we perform the largest ($N = 10^8$) star-by-star $N$-body simulation of a Plummer sphere evolving to core collapse, reproducing the expected self-similar density profile over more than 15 orders of magnitude; second, we generate realistic models for typical globular clusters, and we show that their dynamical evolution can produce significant numbers of black hole mergers with masses greater than those produced from isolated binary evolution (such as GW190521, a recently reported merger with component masses in the pulsational pair-instability mass gap).


title: Bayesian model-selection of neutron star equation of state using multi-messenger observations

abstract: Measurement of macroscopic properties of neutron stars, whether in binary or in an isolated system, provides us a key opportunity to place a stringent constraint on its equation of state. In this paper, we perform Bayesian model-selection on a wide variety of neutron star equation of state using multi-messenger observations. In particular, (i) we use the mass and tidal deformability measurement from two binary neutron star merger event, GW170817 and GW190425; (ii) simultaneous mass-radius measurement of PSR J0030+0451 and PSR J0740+6620 by NICER collaboration, while the latter has been analyzed by joint NICER/radio/XMM-Newton collaboration. Among the 31 equations of state considered in this analysis, we are able to rule out different variants of MS1 family, SKI5, H4, and WFF1 EoSs decisively, which are either extremely stiff or soft equations of state. The most preferred equation of state model turns out to be AP3 (or MPA1), which predicts the radius and dimensionless tidal deformability of a $1.4 M_{\odot}$ neutron star to be 12.10 (12.50) km and 393 (513) respectively.


title: Non-Loudon-Fleury Raman scattering in spin-orbit coupled Mott insulators

abstract: We revisit the theory of magnetic Raman scattering in Mott insulators with strong spin-orbit coupling, with a major focus on Kitaev materials. We show that Kitaev materials with bond-anisotropic interactions are generally expected to show both one- and two-magnon responses. It is further shown that, in order to obtain the correct leading contributions to the Raman vertex operator $\mc{R}$, one must take into account the precise, photon-assisted microscopic hopping processes of the electrons and that, in systems with multiple hopping paths, $\mc{R}$ contains terms beyond those appearing in the traditional Loudon-Fleury theory. Most saliently, a numerical implementation of the revised formalism to the case of the three-dimensional hyperhoneycomb Kitaev material $\beta$-Li$_2$IrO$_3$ reveals that the non-Loudon-Fleury scattering terms actually dominate the Raman intensity. In addition, they induce a qualitative modification of the polarization dependence, including, e.g., the emergence of a sharp one-magnon peak at low energies which is not expected in the traditional Loudon-Fleury theory. This peak is shown to arise from microscopic photon-assisted tunneling processes that are of similar type with the ones leading to the symmetric off-diagonal interaction $\Gamma$ (known to be present in many Kitaev materials), but take the form of a bond-directional magnetic dipole term in the Raman vertex. These results are expected to apply across all Kitaev materials and mark a drastic change of paradigm for the understanding of Raman scattering in materials with strong spin-orbit coupling and multiple exchange paths.


title: Fundamental differences in the radio properties of red and blue quasars: kiloparsec-scale structures revealed by e-MERLIN

abstract: Red quasi-stellar objects (QSOs) are a subset of the quasar population with colours consistent with reddening due to intervening dust. Recent work has demonstrated that red QSOs show special radio properties that fundamentally distinguish them from normal blue QSOs, specifically a higher incidence of low-power radio emission (1.4 GHz luminosities L$_{\rm 1.4} \approx 10^{25}$ - $10^{27}$ W Hz$^{-1}$) that is physically compact when imaged by arcsecond-resolution radio surveys such as FIRST. In this work, we present e-MERLIN imaging of a set of intermediate-redshift ($1.0<z<1.55$), luminous (bolometric luminosities L$_{bol} \approx 10^{46}$ - $10^{47}$ erg s$^{-1}$) red and normal QSOs carefully selected to have radio properties that span the range over which red QSOs show the most divergence from the general population. With an angular resolution $\times25$ better than FIRST, we resolve structures within the host galaxies of these QSOs ($> 2$ kpc). We report a statistically significant difference in the incidence of extended kpc-scale emission in red QSOs. From an analysis of the radio size distributions of the sample, we find that the excess radio emission in red QSOs can be attributed to structures that are confined to galaxy scales ($< 10$ kpc), while we confirm previous results that red and normal QSOs have similar incidences of radio jets and lobes on circumgalactic or larger scales ($> 10$ kpc). Our results indicate that the primary mechanism that generates the enhanced radio emission in red QSOs is not directly connected with the nuclear engine or accretion disc, but is likely to arise from extended components such as AGN-driven jets or winds.


title: Spectroscopic Observations of Obscured Populations in the Inner Galaxy: 2MASS-GC02, Terzan 4, and the 200 km/s stellar peak

abstract: The interpretation of potentially new and already known stellar structures located at low-latitudes is hindered by the presence of dense gas and dust, as observations toward these sight-lines are limited. We have identified APOGEE stars belonging to the low-latitude globular clusters 2MASS-GC02 and Terzan 4, presenting the first chemical element abundances of stars residing in these poorly studied clusters. As expected, the signature of multiple populations co-existing in these metal-rich clusters is evident. We redetermine the radial velocity of 2MASS-GC02 to be -87 +- 7 km/s, finding that this cluster's heliocentric radial velocity is offset by more than 150 km/s from the literature value. We investigate a potentially new low-latitude stellar structure, a kiloparsec-scale nuclear disk (or ring) which has been put forward to explain a high-velocity (V_{GSR} ~200 km/s) peak reported in several Galactic bulge fields based on the APOGEE commissioning observations. New radial velocities of field stars at (l,b)=(-6,0) are presented and combined with the APOGEE observations at negative longitudes to carry out this search. Unfortunately no prominent -200 km/s peak at negative longitudes along the plane of the Milky Way is apparent, as predicted for the signature of a nuclear feature. The distances and Gaia EDR3 proper motions of the high-V_{GSR} stars do not support the current models of stars on bar-supporting orbits as an explanation of the +200 km/s peak.


title: Superconductivity of neutral modes in quantum Hall edges

abstract: Edges of quantum Hall phases give rise to a multitude of exotic modes supporting quasiparticles of different values of charge and quantum statistics. Among these are neutralons (chargeless anyons with semion statistics), which were found to be ubiquitous in fractional quantum Hall matter. Studying and manipulating the neutral sector is an intriguing and interesting challenge, all the more so since these particles are accessible experimentally. Here we address the limit of strongly-interacting neutralons giving rise to neutralon superconductivity, where pairing is replaced by a quarteting mechanism. We discuss several manifestations of this effect, realizable in existing experimental platforms. Furthermore, this superconducting gapping mechanism may be exploited to facilitate the observation of interference of the accompanying charged anyons.


title: Achieving fault tolerance on capped color codes with few ancillas

abstract: Attaining fault tolerance while maintaining low overhead is one of the main challenges in a practical implementation of quantum circuits. One major technique that can overcome this problem is the flag technique, in which high-weight errors arising from a few faults can be detected by a few ancillas and distinguished using subsequent syndrome measurements. The technique can be further improved using the fact that for some families of codes, errors of any weight are logically equivalent if they have the same syndrome and weight parity, as previously shown in [Phys. Rev. A 104, 042410 (2021)]. In this work, we develop a notion of distinguishable fault set which captures both concepts of flags and weight parities, and extend the use of weight parities in error correction from [Phys. Rev. A 104, 042410 (2021)] to families of capped and recursive capped color codes. We also develop fault-tolerant protocols for error correction, measurement, state preparation, and logical T gate implementation via code switching, which are sufficient for performing fault-tolerant Clifford computation on a capped color code, and performing fault-tolerant universal quantum computation on a recursive capped color code. Our protocols for a capped or a recursive capped color code of any distance require only 2 ancillas, assuming that the ancillas can be reused. The concept of distinguishable fault set also leads to a generalization of the definitions of fault-tolerant gadgets proposed by Aliferis, Gottesman, and Preskill.


title: New method for detecting fast neutrino flavor conversions in core-collapse supernova models with two-moment neutrino transport

abstract: Fast-pairwise neutrino oscillations potentially affect many aspects of core-collapse supernova (CCSN): the explosion mechanism, neutrino signals, and nucleosynthesis in the ejecta. This particular mode of collective neutrino oscillations has a deep connection to the angular structure of neutrinos in momentum space; for instance, the appearance of electron neutrinos lepton number (ELN) angular crossings in momentum space is a good indicator of occurrences of the flavor conversions. However, many multi-dimensional (multi-D) CCSN simulations are carried out with approximate neutrino transport (such as two-moment methods), which limits the access to the angular distributions of neutrinos, i.e., inhibits ELN-crossing searches. In this paper, we develop a new method of ELN-crossing search in these CCSN simulations. The required data is the zero-th and first angular moments of neutrinos and matter profile, all of which are available in CCSN models with two-moment method. One of the novelties of our new method is to use a ray-tracing neutrino transport to determine ELNs in the direction of the stellar center. It is designed to compensate for shortcomings of the crossing searches only with the two angular moments. We assess the capability of the method by carrying out a detailed comparison to results of full Boltzmann neutrino transport in 1D and 2D CCSN models. We find that the ray-tracing neutrino transport improves the accuracy of crossing searches; indeed, the appearance/disappearance of the crossings is accurately detected even in the region of forward-peaked angular distributions. The new method is computationally cheap and has a benefit of efficient parallelization; hence, it will be useful for ELN-crossing searches in any CCSN models employed two-moment neutrino transport.


title: The PAU Survey: Measurement of Narrow-band galaxy properties with Approximate Bayesian Computation

abstract: Narrow-band imaging surveys allow the study of the spectral characteristics of galaxies without the need of performing their spectroscopic follow-up. In this work, we forward-model the Physics of the Accelerating Universe Survey (PAUS) narrow-band data. The aim is to improve the constraints on the spectral coefficients used to create the galaxy spectral energy distributions (SED) of the galaxy population model in Tortorelli et al. 2020. In that work, the model parameters were inferred from the Canada-France-Hawaii Telescope Legacy Survey (CFHTLS) data using Approximate Bayesian Computation (ABC). This led to stringent constraints on the B-band galaxy luminosity function parameters, but left the spectral coefficients only broadly constrained. To address that, we perform an ABC inference using CFHTLS and PAUS data. This is the first time our approach combining forward-modelling and ABC is applied simultaneously to multiple datasets. We test the results of the ABC inference by comparing the narrow-band magnitudes of the observed and simulated galaxies using Principal Component Analysis, finding a very good agreement. Furthermore, we prove the scientific potential of the constrained galaxy population model to provide realistic stellar population properties by measuring them with the SED fitting code \textsc{CIGALE}. We use CFHTLS broad-band and PAUS narrow-band photometry for a flux-limited ($\mathrm{i}<22.5$) sample of galaxies up to redshift $\mathrm{z \sim 0.8}$. We find that properties like stellar masses, star-formation rates, mass-weighted stellar ages and metallicities are in agreement within errors between observations and simulations. Overall, this work shows the ability of our galaxy population model to correctly forward-model a complex dataset such as PAUS and the ability to reproduce the diversity of galaxy properties at the redshift range spanned by CFHTLS and PAUS.


title: Ladder resummation of spin 1/2 fermion many-body systems with arbitrary partial-wave content

abstract: We resum the ladder diagrams for the calculation of the energy density $\cal{E}$ of a spin 1/2 fermion many-body system in terms of arbitrary vacuum two-body scattering amplitudes. The partial-wave decomposition of the in-medium two-body scattering amplitudes is developed, and the expression for calculating $\cal{E}$ in a partial-wave amplitude expansion is also given. The case of contact interactions is completely solved and is shown to provide renormalized results, expressed directly in terms of scattering data parameters, within cutoff regularization in a wide class of schemes. $S$- and $P$-wave interactions are considered up to including the first three-terms in the effective-range expansion, paying special attention to the parametric region around the unitary limit.


title: Nonlocal equations with gradient constraints

abstract: We prove the existence and $C^{1,\alpha}$ regularity of solutions to nonlocal fully nonlinear elliptic equations with gradient constraints. We do not assume any regularity about the constraints; so the constraints need not be $C^1$ or strictly convex. We also obtain $C^{0,1}$ boundary regularity for these problems. Our approach is to show that these nonlocal equations with gradient constraints are related to some nonlocal double obstacle problems. Then we prove the regularity of the double obstacle problems. In this process, we also employ the monotonicity property for the second derivative of obstacles, which we have obtained in a previous work.


title: Churn Reduction via Distillation

abstract: In real-world systems, models are frequently updated as more data becomes available, and in addition to achieving high accuracy, the goal is to also maintain a low difference in predictions compared to the base model (i.e. predictive "churn"). If model retraining results in vastly different behavior, then it could cause negative effects in downstream systems, especially if this churn can be avoided with limited impact on model accuracy. In this paper, we show an equivalence between training with distillation using the base model as the teacher and training with an explicit constraint on the predictive churn. We then show that distillation performs strongly for low churn training against a number of recent baselines on a wide range of datasets and model architectures, including fully-connected networks, convolutional networks, and transformers.


title: Noncrossing partition flow and random matrix models

abstract: We study a generating function flowing from the one enumerating a set of partitions to the one enumerating the corresponding set of noncrossing partitions; numerical simulations indicate that its limit in the Adjacency random matrix model on bipartite Erd\"os-Renyi graphs gives a good approximation of the spectral distribution for large average degrees. This model and a Wishart-type random matrix model are described using congruence classes on $k$-divisible partitions. We compute, in the $d\to \infty$ limit with $\frac{Z_a}{d}$ fixed, the spectral distribution of an Adjacency and of a Laplacian random block matrix model, on bipartite Erd\"os-Renyi graphs and on bipartite biregular graphs with degrees $Z_1, Z_2$; the former is the approximation previously mentioned; the latter is a mean field approximation of the Hessian of a random bipartite biregular elastic network; it is characterized by an isostatic line and a transition line between the one- and the two-band regions.


title: Approximating Nash Social Welfare under Binary XOS and Binary Subadditive Valuations

abstract: We study the problem of allocating indivisible goods among agents in a fair and economically efficient manner. In this context, the Nash social welfare-defined as the geometric mean of agents' valuations for their assigned bundles-stands as a fundamental measure that quantifies the extent of fairness of an allocation. Focusing on instances in which the agents' valuations have binary marginals, we develop essentially tight results for (approximately) maximizing Nash social welfare under two of the most general classes of complement-free valuations, i.e., under binary XOS and binary subadditive valuations. For binary XOS valuations, we develop a polynomial-time algorithm that finds a constant-factor (specifically $288$) approximation for the optimal Nash social welfare, in the standard value-oracle model. The allocations computed by our algorithm also achieve constant-factor approximation for social welfare and the groupwise maximin share guarantee. These results imply that-in the case of binary XOS valuations-there necessarily exists an allocation that simultaneously satisfies multiple (approximate) fairness and efficiency criteria. We complement the algorithmic result by proving that Nash social welfare maximization is APX-hard under binary XOS valuations. Furthermore, this work establishes an interesting separation between the binary XOS and binary subadditive settings. In particular, we prove that an exponential number of value queries are necessarily required to obtain even a sub-linear approximation for Nash social welfare under binary subadditive valuations.


title: Structural Monoclinicity and Its Coupling to Layered Magnetism in Few-Layer $\mathrm{CrI_{3}}$

abstract: Using polarization-resolved Raman spectroscopy, we investigate layer number, temperature, and magnetic field dependence of Raman spectra in one- to four-layer $\mathrm{CrI_{3}}$. Layer-number-dependent Raman spectra show that in the paramagnetic phase a doubly degenerated $E_{g}$ mode of monolayer $\mathrm{CrI_{3}}$ splits into one $A_{g}$ and one $B_{g}$ mode in N-layer (N > 1) $\mathrm{CrI_{3}}$ due to the monoclinic stacking. Their energy separation increases in thicker samples until an eventual saturation. Temperature-dependent measurements further show that the split modes tend to merge upon cooling but remain separated until 10 K, indicating a failed attempt of the monoclinic-to-rhombohedral structural phase transition that is present in the bulk crystal. Magnetic-field-dependent measurements reveal an additional monoclinic distortion across the magnetic-field-induced layered antiferromagnetism-to-ferromagnetism phase transition. We propose a structural change that consists of both a lateral sliding toward the rhombohedral stacking and a decrease in the interlayer distance to explain our experimental observations.


title: W-RST: Towards a Weighted RST-style Discourse Framework

abstract: Aiming for a better integration of data-driven and linguistically-inspired approaches, we explore whether RST Nuclearity, assigning a binary assessment of importance between text segments, can be replaced by automatically generated, real-valued scores, in what we call a Weighted-RST framework. In particular, we find that weighted discourse trees from auxiliary tasks can benefit key NLP downstream applications, compared to nuclearity-centered approaches. We further show that real-valued importance distributions partially and interestingly align with the assessment and uncertainty of human annotators.


title: Recurring Planetary Debris Transits and Circumstellar Gas around White Dwarf ZTF J0328$-$1219

abstract: We present follow-up photometry and spectroscopy of ZTF J0328$-$1219 strengthening its status as a white dwarf exhibiting transiting planetary debris. Using TESS and Zwicky Transient Facility photometry, along with follow-up high speed photometry from various observatories, we find evidence for two significant periods of variability at 9.937 and 11.2 hr. We interpret these as most likely the orbital periods of different debris clumps. Changes in the detailed dip structures within the light curves are observed on nightly, weekly, and monthly timescales, reminiscent of the dynamic behavior observed in the first white dwarf discovered to harbor a disintegrating asteroid, WD 1145+017. We fit previously published spectroscopy along with broadband photometry to obtain new atmospheric parameters for the white dwarf, with $M_{\star} = 0.731 \pm 0.023\,M_{\odot}$, $T_{\mathrm{eff}} = 7630 \pm 140\,$K, and $\mathrm{[Ca/He]}=-9.55\pm0.12$. With new high-resolution spectroscopy, we detect prominent and narrow Na D absorption features likely of circumstellar origin, with velocities $21.4\pm1.0$ km s$^{-1}$ blue-shifted relative to atmospheric lines. We attribute the periodically modulated photometric signal to dusty effluents from small orbiting bodies such as asteroids or comets, but are unable to identify the most likely material that is being sublimated, or otherwise ejected, as the environmental temperatures range from roughly 400K to 600K.


title: Chemical signatures of a warped protoplanetary disc

abstract: Circumstellar discs may become warped or broken into distinct planes if there is a stellar or planetary companion with an orbit that is misaligned with respect to the disc. There is mounting observational evidence for protoplanetary discs with misaligned inner discs and warps that may be caused by such interactions with a previously undetected companion, giving us a tantalising indication of possible planets forming there. Hydrodynamical and radiative transfer models indicate that the temperature varies azimuthally in warped discs due to the variable angle at which the disc surface faces the star and this impacts the disc chemistry. We perform chemical modelling based on a hydrodynamical model of a protoplanetary disc with an embedded planet orbiting at a 12$^{\circ}$ inclination to the disc. Even for this small misalignment, abundances of species including CO and HCO$^+$ vary azimuthally and this results in detectable azimuthal variations in submillimetre line emission. Azimuthal variations in line emission may therefore indicate the presence of an unseen embedded companion. Nonaxisymmetric chemical abundances should be considered when interpreting molecular line maps of warped or shadowed protoplanetary discs.


title: Projective Splitting as a Warped Proximal Algorithm

abstract: We show that the asynchronous block-iterative primal-dual projective splitting framework introduced by P. L. Combettes and J. Eckstein in their 2018 Math. Program. paper can be viewed as an instantiation of the recently proposed warped proximal algorithm.


title: Relativistically transparent magnetic filaments: scaling laws, initial results and prospects for strong-field QED studies

abstract: Relativistic transparency enables volumetric laser interaction with overdense plasmas and direct laser acceleration of electrons to relativistic velocities. The dense electron current generates a magnetic filament with field strength of the order of the laser amplitude ($>$10$^5$ T). The magnetic filament traps the electrons radially, enabling efficient acceleration and conversion of laser energy into MeV photons by electron oscillations in the filament. The use of microstructured targets stabilizes the hosing instabilities associated with relativistically transparent interactions, resulting in robust and repeatable production of this phenomenon. Analytical scaling laws are derived to describe the radiated photon spectrum and energy from the magnetic filament phenomenon in terms of the laser intensity, focal radius, pulse duration, and the plasma density. These scaling laws are compared to 3-D particle-in-cell (PIC) simulations, demonstrating agreement over two regimes of focal radius. Preliminary experiments to study this phenomenon at moderate intensity ($a_0 \sim 30$) were performed on the Texas Petawatt Laser. Experimental signatures of the magnetic filament phenomenon are observed in the electron and photon spectra recorded in a subset of these experiments that is consistent with the experimental design, analytical scaling and 3-D PIC simulations. Implications for future experimental campaigns are discussed.


title: Quantum optimization via four-body Rydberg gates

abstract: There is a large ongoing research effort towards obtaining a quantum advantage in the solution of combinatorial optimization problems on near-term quantum devices. A particularly promising platform for testing and developing quantum optimization algorithms are arrays of trapped neutral atoms, laser-coupled to highly excited Rydberg states. However, encoding combinatorial optimization problems in atomic arrays is challenging due to the limited inter-qubit connectivity given by their native finite-range interactions. Here we propose and analyze a fast, high fidelity four-body Rydberg parity gate, enabling a direct and straightforward implementation of the Lechner-Hauke-Zoller (LHZ) scheme and its recent generalization, the parity architecture, a scalable architecture for encoding arbitrarily connected interaction graphs. Our gate relies on onetime-optimized adiabatic laser pulses and is fully programmable by adjusting two hold-times during operation. We numerically demonstrate an implementation of the quantum approximate optimization algorithm (QAOA) for a small scale test problem. Our approach allows for efficient execution of variational optimization steps with a constant number of system manipulations, independent of the system size, thus paving the way for experimental investigations of QAOA beyond the reach of numerical simulations.


title: Galaxy Science with ORCAS: Faint Star-Forming Clumps to AB$\leq$31 mag and r$_e$$\geq$ 0.01"

abstract: The NASA concept mission ORCAS (Orbiting Configurable Artificial Star) aims to provide near diffraction-limited angular resolution at visible and near-infrared wavelengths using laser signals from space-based cubesats as Adaptive Optics beacons for ground-based 8-30 meter telescopes, in particular the 10 meter Keck Telescopes. When built as designed, ORCAS+Keck would deliver images of ~0.01-0.02" FWHM at 0.5-1.2 micron wavelength that reach AB<31 mag for point sources in a few hours over a 5x5" FOV that includes IFU capabilities. We summarize the potential of high-resolution faint galaxy science with ORCAS. We show that the ability to detect optical-near-IR point sources with r_e>0.01" FWHM to AB<31 mag will yield about 5.0x10^6 faint star-forming (SF) clumps per square degree, or ~0.4 per arcsec^2. From recent HST lensing data, the typical intrinsic (unlensed) sizes of SF clumps at z~1-7 will be r_e ~1-80 m.a.s. to AB<31 mag, with intrinsic (unmagnified) fluxes as faint as AB<35-36 mag when searching with ORCAS around the critical curves of lensing clusters imaged with HST and JWST. About half of these SF clumps will have sizes below the ORCAS diffraction limit, and the other half will be slightly resolved, but still mostly above the ORCAS surface brightness (SB) limits. ORCAS will address how galaxies assemble from smaller clumps to stable disks by measuring ages, metallicities, and gradients of clumps within galaxies. ORCAS can monitor caustic transits of individual stars in SF clumps at z>1-2 that have been detected with HST, and those that may be detected with JWST at z>6 at extreme magnifications (mu>10^3-10^5) for the first stars and their stellar mass black hole accretion disks. ORCAS provides a unique opportunity to obtain a statistical census of individual stars at cosmological distances, leveraging the largest telescopes only available on the ground.


title: The Chromatic Quasisymmetric Class Function of a Digraph

abstract: We introduce a quasisymmetric class function associated with a group acting on a double poset or on a directed graph. The latter is a generalization of the chromatic quasisymmetric function of a digraph introduced by Ellzey, while the latter is a generalization of a quasisymmetric function introduced by Grinberg. We prove representation-theoretic analogues of classical and recent results, including $F$-positivity, and combinatorial reciprocity theorems. We also deduce results for orbital quasisymmetric functions. We also study a generalization of the notion of strongly flawless sequences.


title: Counterfactual Explanations Can Be Manipulated

abstract: Counterfactual explanations are emerging as an attractive option for providing recourse to individuals adversely impacted by algorithmic decisions. As they are deployed in critical applications (e.g. law enforcement, financial lending), it becomes important to ensure that we clearly understand the vulnerabilities of these methods and find ways to address them. However, there is little understanding of the vulnerabilities and shortcomings of counterfactual explanations. In this work, we introduce the first framework that describes the vulnerabilities of counterfactual explanations and shows how they can be manipulated. More specifically, we show counterfactual explanations may converge to drastically different counterfactuals under a small perturbation indicating they are not robust. Leveraging this insight, we introduce a novel objective to train seemingly fair models where counterfactual explanations find much lower cost recourse under a slight perturbation. We describe how these models can unfairly provide low-cost recourse for specific subgroups in the data while appearing fair to auditors. We perform experiments on loan and violent crime prediction data sets where certain subgroups achieve up to 20x lower cost recourse under the perturbation. These results raise concerns regarding the dependability of current counterfactual explanation techniques, which we hope will inspire investigations in robust counterfactual explanations.


title: Parametrised topological complexity of group epimorphisms

abstract: We show that the parametrised topological complexity of Cohen, Farber and Weinberger gives an invariant of group epimorphisms. We extend various bounds for the topological complexity of groups to obtain bounds for the parametrised topological complexity of epimorphisms. Several applications are given, including an alternative computation of the parametrised topological complexity of the planar Fadell--Neuwirth fibrations which avoids calculations involving cup products. We also prove a homotopy invariance result for parametrised topological complexity of fibrations over different bases.


title: Emergent Communication of Generalizations

abstract: To build agents that can collaborate effectively with others, recent research has trained artificial agents to communicate with each other in Lewis-style referential games. However, this often leads to successful but uninterpretable communication. We argue that this is due to the game objective: communicating about a single object in a shared visual context is prone to overfitting and does not encourage language useful beyond concrete reference. In contrast, human language conveys a rich variety of abstract ideas. To promote such skills, we propose games that require communicating generalizations over sets of objects representing abstract visual concepts, optionally with separate contexts for each agent. We find that these games greatly improve systematicity and interpretability of the learned languages, according to several metrics in the literature. Finally, we propose a method for identifying logical operations embedded in the emergent languages by learning an approximate compositional reconstruction of the language.


title: Real Time Video based Heart and Respiration Rate Monitoring

abstract: In recent years, research about monitoring vital signs by smartphones grows significantly. There are some special sensors like Electrocardiogram (ECG) and Photoplethysmographic (PPG) to detect heart rate (HR) and respiration rate (RR). Smartphone cameras also can measure HR by detecting and processing imaging Photoplethysmographic (iPPG) signals from the video of a user's face. Indeed, the variation in the intensity of the green channel can be measured by the iPPG signals of the video. This study aimed to provide a method to extract heart rate and respiration rate using the video of individuals' faces. The proposed method is based on measuring fluctuations in the Hue, and can therefore extract both HR and RR from the video of a user's face. The proposed method is evaluated by performing on 25 healthy individuals. For each subject, 20 seconds video of his/her face is recorded. Results show that the proposed approach of measuring iPPG using Hue gives more accurate rates than the Green channel.


title: Robust Resource Allocation for Multi-Antenna URLLC-OFDMA Systems in a Smart Factory

abstract: In this paper, we investigate the worst-case robust beamforming design and resource block (RB) assignment problem for total transmit power minimization of the central controller while guaranteeing each robot's transmission with target number of data bits and within required ultra-low latency and extremely high reliability. By using the property of the independence of each robot's beamformer design, we can obtain the equivalent power control design form of the original beamforming design. The binary RB mapping indicators are transformed into continuous ones with additional $\ell_0$-norm constraints to promote sparsity on each RB. A novel non-convex penalty (NCP) approach is applied to solve such $\ell_0$-norm constraints. Numerical results demonstrate the superiority of the NCP approach to the well-known reweighted $\ell_1$ method in terms of the optimized power consumption, convergence rate and robustness to channel realizations. Also, the impacts of latency, reliability, number of transmit antennas and channel uncertainty on the system performance are revealed.


title: Electron vacancy-level dependent hybrid photoionization of F-@C60+ molecule

abstract: Our previous studies [J. Phys. B 53, 125101 (2020); Euro. Phys. J. D 74, 191 (2020)] have predicted that the atom-fullerene hybrid photoionization properties for X = Cl, Br and I endohedrally confined in C60 are different before and after an electron transfers from C60 to the halogen. It was further found as a rule that the ionization dynamics is insensitive to the C60 level the electron originates from to produce X-@C60+. In the current study, we report an exception to this rule in F@C60. It is found that when the electron vacancy is situated in the C60 level that participates in the hybridization in F-@C60+, the mixing becomes dramatically large leading to strong modifications in the photoionization of the hybrid levels. But when the vacancy is at any other pure level of C60, the level-invariance is retained showing weak hybridization. Even though this case of F@C60 is an anomaly in the halogen@C60 series, the phenomenon can be more general and can occur with compounds of other atoms caged in a variety of fullerenes. In addition, possible experimental studies are suggested to benchmark the present results.


title: Improving Physical Cosmology: An Empiricist's Assessment

abstract: The $\Lambda$CDM cosmology passes demanding tests that establish it as a good approximation to reality, but it could be improved. I present a list of possibly interesting and less well explored things that might yield hints to a better theory.


title: Odds Ratios are far from "portable": A call to use realistic models for effect variation in meta-analysis

abstract: Objective: Recently Doi et al. argued that risk ratios should be replaced with odds ratios in clinical research. We disagreed, and empirically documented the lack of portability of odds ratios, while Doi et al. defended their position. In this response we highlight important errors in their position. Study Design and Setting: We counter Doi et al.'s arguments by further examining the correlations of odds ratios, and risk ratios, with baseline risks in 20,198 meta-analyses from the Cochrane Database of Systematic Reviews. Results: Doi et al.'s claim that odds ratios are portable is invalid because 1) their reasoning is circular: they assume a model under which the odds ratio is constant and show that under such a model the odds ratio is portable; 2) the method they advocate to convert odds ratios to risk ratios is biased; 3) their empirical example is readily-refuted by counter-examples of meta-analyses in which the risk ratio is portable but the odds ratio isn't; and 4) they fail to consider the causal determinants of meta-analytic inclusion criteria: Doi et al. mistakenly claim that variation in odds ratios with different baseline risks in meta-analyses is due to collider bias. Empirical comparison between the correlations of odds ratios, and risk ratios, with baseline risks show that the portability of odds ratios and risk ratios varies across settings. Conclusion: The suggestion to replace risk ratios with odds ratios is based on circular reasoning and a confusion of mathematical and empirical results. It is especially misleading for meta-analyses and clinical guidance. Neither the odds ratio nor the risk ratio is universally portable. To address this lack of portability, we reinforce our suggestion to report variation in effect measures conditioning on modifying factors such as baseline risk; understanding such variation is essential to patient-centered practice.


title: Differentially Empirical Risk Minimization under the Fairness Lens

abstract: Differential Privacy (DP) is an important privacy-enhancing technology for private machine learning systems. It allows to measure and bound the risk associated with an individual participation in a computation. However, it was recently observed that DP learning systems may exacerbate bias and unfairness for different groups of individuals. This paper builds on these important observations and sheds light on the causes of the disparate impacts arising in the problem of differentially private empirical risk minimization. It focuses on the accuracy disparity arising among groups of individuals in two well-studied DP learning methods: output perturbation and differentially private stochastic gradient descent. The paper analyzes which data and model properties are responsible for the disproportionate impacts, why these aspects are affecting different groups disproportionately and proposes guidelines to mitigate these effects. The proposed approach is evaluated on several datasets and settings.


title: Capture and translocation of a rod-like molecule by a nanopore: orientation, charge distribution and hydrodynamics

abstract: We investigate the translocation of rods with different charge distributions using hybrid Langevin Dynamics and Lattice Boltzmann (LD-LB) simulations. Electrostatic interactions are added to the system using the $P^3M$ algorithm to model the electrohydrodynamic interactions (EHI). We first examine the free-solution electrophoretic properties of rods with various charge distributions. Our translocation simulation results suggest that the order parameter is asymmetric during the capture and escape processes despite the symmetric electric field lines, while the impacts of the charge distribution on rod orientation are more significant during the capture process. The capture/threading/escape times are under the combined effects of charge screening, rod orientation, and charge distributions. We also show that the mean capture time of a rod is shorter when it is launched near the wall because rods tend to align along the wall and hence with the local field lines. Remarkably, the \textit{orientational capture radius} we proposed previously for uniformly charged rods is still valid in the presence of EHI.


title: A novel multi-scale loss function for classification problems in machine learning

abstract: We introduce two-scale loss functions for use in various gradient descent algorithms applied to classification problems via deep neural networks. This new method is generic in the sense that it can be applied to a wide range of machine learning architectures, from deep neural networks to support vector machines for example. These two-scale loss functions allow to focus the training onto objects in the training set which are not well classified. This leads to an increase in several measures of performance for appropriately-defined two-scale loss functions with respect to the more classical cross-entropy when tested on traditional deep neural networks on the MNIST, CIFAR10, and CIFAR100 data-sets.


title: Relay Selection and Resource Allocation for Ultra-Reliable Uplink Transmission in Smart Factory Scenarios

abstract: In this paper, a relay-aided two-phase transmission protocol for the smart factory scenario is proposed. This protocol aims at enabling all robots' ultra-reliable target number of uplink critical data transmission within a latency constraint by jointly optimizing the relay selection, resource block (RB) assignment, and transmit power allocation. Such protocol design is formulated as a mixed-integer and strictly non-convex problem where optimization variables are mutual coupling, which is definitely challenging. Instead of conventional methods designed for solving the problem, we leverage the properties of the relative entropy function to equivalently transform the problem without introducing extra constraints. As the packet error probability requirements of each robot under two possible transmission modes are coupled in one overall reliability constraint, the big-M technique is applied to decouple it into two corresponding reliability constraints. One is for direct transmission mode, and the other is for cooperative transmission mode. Moreover, both non-convex penalty (NCP) and quadratic penalty (QP) approaches are utilized to deal with the binary indicator constraints. Based on such penalty methods, a sequence of penalized approximated convex problems can be iteratively solved for sub-optimal solutions. Numerical results demonstrate the efficiency of such two penalty methods from the perspectives of sub-optimal values of total transmit power and convergence rate. Further, the impacts of reliability, the number and location of relays, the number of robots, the target number of data bits on the total power consumption are analyzed.


title: A Universal Quantum Circuit Design for Periodical Functions

abstract: We propose a universal quantum circuit design that can estimate any arbitrary one-dimensional periodic functions based on the corresponding Fourier expansion. The quantum circuit contains N-qubits to store the information on the different N-Fourier components and $M+2$ auxiliary qubits with $M = \lceil{\log_2{N}}\rceil$ for control operations. The desired output will be measured in the last qubit $q_N$ with a time complexity of the computation of $O(N^2\lceil \log_2N\rceil^2)$. We illustrate the approach by constructing the quantum circuit for the square wave function with accurate results obtained by direct simulations using the IBM-QASM simulator. The approach is general and can be applied to any arbitrary periodic function.


title: Layered gradient accumulation and modular pipeline parallelism: fast and efficient training of large language models

abstract: The advent of the transformer has sparked a quick growth in the size of language models, far outpacing hardware improvements. (Dense) transformers are expected to reach the trillion-parameter scale in the near future, for which training requires thousands or even tens of thousands of GPUs. We investigate the challenges of training at this scale and beyond on commercially available hardware. In particular, we analyse the shortest possible training time for different configurations of distributed training, leveraging empirical scaling laws for language models to estimate the optimal (critical) batch size. Contrary to popular belief, we find no evidence for a memory wall, and instead argue that the real limitation -- other than the cost -- lies in the training duration. In addition to this analysis, we introduce two new methods, \textit{layered gradient accumulation} and \textit{modular pipeline parallelism}, which together cut the shortest training time by half. The methods also reduce data movement, lowering the network requirement to a point where a fast InfiniBand connection is not necessary. This increased network efficiency also improve on the methods introduced with the ZeRO optimizer, reducing the memory usage to a tiny fraction of the available GPU memory.


title: Algorithms from Invariants: Smoothed Analysis of Orbit Recovery over $SO(3)$

abstract: In this work we study orbit recovery over $SO(3)$, where the goal is to recover a function on the sphere from noisy, randomly rotated copies of it. We assume that the function is a linear combination of low-degree spherical harmonics. This is a natural abstraction for the problem of recovering the three-dimensional structure of a molecule through cryo-electron tomography. For provably learning the parameters of a generative model, the method of moments is the standard workhorse of theoretical machine learning. It turns out that there is a natural incarnation of the method of moments for orbit recovery based on invariant theory. Bandeira et al. [BBSK+18] used invariant theory to give tight bounds on the sample complexity in terms of the noise level. However many of the key challenges remain: Can we prove bounds on the sample complexity that are polynomial in $n$, the dimension of the signal? The bounds in [BBSK+18] hide constants that have an unspecified dependence on $n$ and only hold in the limit as $\sigma^2 \rightarrow \infty$ where $\sigma^2$ is the variance of the noise. Moreover can we give efficient algorithms? We revisit these challenges from the perspective of smoothed analysis, where we assume that the coefficients of the signal, in the basis of spherical harmonics, are subject to small random perturbations. Our main result is a quasi-polynomial time algorithm for orbit recovery over $SO(3)$ in this model. Our approach is based on frequency marching, which sequentially solves linear systems to find higher degree coefficients. Our main technical contribution is to show that these linear systems have unique solutions, are well-conditioned, and that the error can be made to compound over at most a logarithmic number of rounds. We believe that our work takes an important first step towards uncovering the algorithmic implications of invariant theory.


title: SpikePropamine: Differentiable Plasticity in Spiking Neural Networks

abstract: The adaptive changes in synaptic efficacy that occur between spiking neurons have been demonstrated to play a critical role in learning for biological neural networks. Despite this source of inspiration, many learning focused applications using Spiking Neural Networks (SNNs) retain static synaptic connections, preventing additional learning after the initial training period. Here, we introduce a framework for simultaneously learning the underlying fixed-weights and the rules governing the dynamics of synaptic plasticity and neuromodulated synaptic plasticity in SNNs through gradient descent. We further demonstrate the capabilities of this framework on a series of challenging benchmarks, learning the parameters of several plasticity rules including BCM, Oja's, and their respective set of neuromodulatory variants. The experimental results display that SNNs augmented with differentiable plasticity are sufficient for solving a set of challenging temporal learning tasks that a traditional SNN fails to solve, even in the presence of significant noise. These networks are also shown to be capable of producing locomotion on a high-dimensional robotic learning task, where near-minimal degradation in performance is observed in the presence of novel conditions not seen during the initial training period.


title: Scalable semidefinite programming approach to variational embedding for quantum many-body problems

abstract: In quantum embedding theories, a quantum many-body system is divided into localized clusters of sites which are treated with an accurate `high-level' theory and glued together self-consistently by a less accurate `low-level' theory at the global scale. The recently introduced variational embedding approach for quantum many-body problems combines the insights of semidefinite relaxation and quantum embedding theory to provide a lower bound on the ground-state energy that improves as the cluster size is increased. The variational embedding method is formulated as a semidefinite program (SDP), which can suffer from poor computational scaling when treated with black-box solvers. We exploit the interpretation of this SDP as an embedding method to develop an algorithm which alternates parallelizable local updates of the high-level quantities with updates that enforce the low-level global constraints. Moreover, we show how translation invariance in lattice systems can be exploited to reduce the complexity of projecting a key matrix to the positive semidefinite cone.