title: Resolvent estimates for the one-dimensional damped wave equation with unbounded damping

abstract: We study the generator $G$ of the one-dimensional damped wave equation with unbounded damping. We show that the norm of the corresponding resolvent operator, $\| (G - \lambda)^{-1} \|$, is approximately constant as $|\lambda| \to +\infty$ on vertical strips of bounded width contained in the closure of the left-hand side complex semi-plane, $\overline{\mathbb{C}}_{-} := \{\lambda \in \mathbb{C}: \operatorname{Re} \lambda \le 0\}$. Our proof rests on a precise asymptotic analysis of the norm of the inverse of $T(\lambda)$, the quadratic operator associated with $G$.


title: Exploring Web3 From the View of Blockchain

abstract: Web3 is the most hyped concept from 2020 to date, greatly motivating the prosperity of the Internet of Value and Metaverse. However, no solid evidence stipulates the exact definition, criterion, or standard in the sense of such a buzzword. To fill the gap, we aim to clarify the term in this work. We narrow down the connotation of Web3 by separating it from high-level controversy argues and, instead, focusing on its protocol, architecture, and evaluation from the perspective of blockchain fields. Specifically, we have identified all potential architectural design types and evaluated each of them by employing the scenario-based architecture evaluation method. The evaluation shows that existing applications are neither secure nor adoptable as claimed. Meanwhile, we also discuss opportunities and challenges surrounding the Web3 space and answer several prevailing questions from communities. A primary result is that Web3 still relies on traditional internet infrastructure, not as independent as advocated. This report, as of June 2022, provides the first strict research on Web3 in the view of blockchain. We hope that this work would provide a guide for the development of future Web3 services.


title: Evidence of a new excited charmed baryon decaying to $\Sigma_{c}(2455)^{0,++} \pi^{\pm}$

abstract: We present the study of $\bar{B}^{0} \to \Sigma_{c}(2455)^{0,++} \pi^{\pm} \bar{p}$ decays based on $772\times 10^{6}$ $B\bar{B}$ events collected with the Belle detector at the KEKB asymmetric-energy $e^+e^-$ collider. The $\Sigma_{c}(2455)^{0,++} $ candidates are reconstructed via their decay to $\Lambda_{c}^{+} \pi^{\mp}$ and $\Lambda_{c}^{+}$ decays to $pK^{-}\pi^{+},~pK_{S}^{0},$ and $\Lambda\pi^{+}$ final states. The corresponding branching fractions are measured to be ${\cal B}(\bar{B}^{0} \to \Sigma_{c}(2455)^{0} \pi^{+} \bar{p}) = (1.09 \pm 0.06 \pm 0.07)\times10^{-4}$ and ${\cal B}(\bar{B}^{0} \to \Sigma_{c}(2455)^{++} \pi^{-} \bar{p}) = (1.84\pm 0.11 \pm 0.12)\times 10^{-4}$, which are consistent with the world average values with improved precision. A new structure is found in the $M_{\Sigma_{c}(2455)^{0,++}\pi^{\pm}}$ spectrum with a significance of $4.2\sigma$ including systematic uncertainty. The structure is possibly an excited $\Lambda_{c}^{+}$ and is tentatively named $\Lambda_{c}(2910)^{+}$. Its mass and width are measured to be $(2913.8 \pm 5.6 \pm 3.8)$ MeV/$c^{2}$ and $(51.8\pm20.0 \pm 18.8)$ MeV, respectively. The products of branching fractions for the $\Lambda_{c}(2910)^{+}$ are measured to be ${\cal B}(\bar{B}^{0} \to \Lambda_{c}(2910)^{+}\bar{p})\times{\cal B}(\Lambda_{c}(2910)^{+} \to \Sigma_{c}(2455)^{0}\pi^{+}) = (9.5 \pm 3.6 \pm 1.6)\times 10^{-6}$ and ${\cal B}(\bar{B}^{0} \to \Lambda_{c} (2910)^{+}\bar{p})\times {\cal B}(\Lambda_{c}(2910)^{+} \to \Sigma_{c}(2455)^{++}\pi^{-}) = (1.24 \pm 0.35 \pm 0.10)\times 10^{-5}$. Here, the first and second uncertainties are statistical and systematic, respectively.


title: Language with Vision: a Study on Grounded Word and Sentence Embeddings

abstract: Grounding language in vision is an active field of research seeking to construct cognitively plausible word and sentence representations by incorporating perceptual knowledge from vision into text-based representations. Despite many attempts at language grounding, achieving an optimal equilibrium between textual representations of the language and our embodied experiences remains an open field. Some common concerns are the following. Is visual grounding advantageous for abstract words, or is its effectiveness restricted to concrete words? What is the optimal way of bridging the gap between text and vision? To what extent is perceptual knowledge from images advantageous for acquiring high-quality embeddings? Leveraging the current advances in machine learning and natural language processing, the present study addresses these questions by proposing a simple yet very effective computational grounding model for pre-trained word embeddings. Our model effectively balances the interplay between language and vision by aligning textual embeddings with visual information while simultaneously preserving the distributional statistics that characterize word usage in text corpora. By applying a learned alignment, we are able to indirectly ground unseen words including abstract words. A series of evaluations on a range of behavioural datasets shows that visual grounding is beneficial not only for concrete words but also for abstract words, lending support to the indirect theory of abstract concepts. Moreover, our approach offers advantages for contextualized embeddings, such as those generated by BERT, but only when trained on corpora of modest, cognitively plausible sizes. Code and grounded embeddings for English are available at https://github.com/Hazel1994/Visually_Grounded_Word_Embeddings_2.


title: Relativistic quantum decryption of large-scale neural coding

abstract: Spin-geometrical projections, from the study of the human universe onto the study of the self-organizing brain, are herein leveraged to address certain concerns raised in latest neuroscience research, namely (i) the extent to which neural codes are multidimensional; (ii) the functional role of neural dark matter; (iii) the challenge to classical model frameworks posed by the needs for accurate interpretation of large-scale neural recordings linking brain and behavior. On the grounds of (hyper-)self-duality under (hyper-)mirror supersymmetry, relativistic quantum principles are introduced, whose consolidation, as pillars of a graphical game-theoretical construction, is conducive to (i) the high-precision reproduction and reinterpretation of core experimental observations on neural coding in the self-organizing brain, with the instantaneous geometric dimensionality of neural representations of a spontaneous behavioral state being proven to be at most 16, unidirectionally; (ii) the coexistability of ordinary and dark neural coding for (co-)behavior; (iii) a possible role for spinor (co-)representations, as the latent building blocks of self-organizing cortical circuits subserving (co-)behavioral states; (iv) an early crystallization of pertinent multidimensional synaptic (co-)architectures, whereby Lorentz (co-)partitions are in principle verifiable; (v) the allusion to octonionic dynamics sustained by triality, as candidate dynamics underlying internal states or emotions; and, ultimately, (vi) potentially inverse insights into matter-antimatter asymmetry. New avenues for the decryption of large-scale neural coding in health and disease are being discussed.


title: An integral equation method for the advection-diffusion equation on time-dependent domains in the plane

abstract: Boundary integral methods are attractive for solving homogeneous linear constant coefficient elliptic partial differential equations on complex geometries, since they can offer accurate solutions with a computational cost that is linear or close to linear in the number of discretization points on the boundary of the domain. However, these numerical methods are not straightforward to apply to time-dependent equations, which often arise in science and engineering. We address this problem with an integral equation-based solver for the advection-diffusion equation on moving and deforming geometries in two space dimensions. In this method, an adaptive high-order accurate time-stepping scheme based on semi-implicit spectral deferred correction is applied. One time-step then involves solving a sequence of non-homogeneous modified Helmholtz equations, a method known as elliptic marching. Our solution methodology utilizes several recently developed methods, including special purpose quadrature, a function extension technique and a spectral Ewald method for the modified Helmholtz kernel. Special care is also taken to handle the time-dependent geometries. The numerical method is tested through several numerical examples to demonstrate robustness, flexibility and accuracy


title: Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis

abstract: Alzheimer's Disease (AD) is the most common neurodegenerative disorder with one of the most complex pathogeneses, making effective and clinically actionable decision support difficult. The objective of this study was to develop a novel multimodal deep learning framework to aid medical professionals in AD diagnosis. We present a Multimodal Alzheimer's Disease Diagnosis framework (MADDi) to accurately detect the presence of AD and mild cognitive impairment (MCI) from imaging, genetic, and clinical data. MADDi is novel in that we use cross-modal attention, which captures interactions between modalities - a method not previously explored in this domain. We perform multi-class classification, a challenging task considering the strong similarities between MCI and AD. We compare with previous state-of-the-art models, evaluate the importance of attention, and examine the contribution of each modality to the model's performance. MADDi classifies MCI, AD, and controls with 96.88% accuracy on a held-out test set. When examining the contribution of different attention schemes, we found that the combination of cross-modal attention with self-attention performed the best, and no attention layers in the model performed the worst, with a 7.9% difference in F1-Scores. Our experiments underlined the importance of structured clinical data to help machine learning models contextualize and interpret the remaining modalities. Extensive ablation studies showed that any multimodal mixture of input features without access to structured clinical information suffered marked performance losses. This study demonstrates the merit of combining multiple input modalities via cross-modal attention to deliver highly accurate AD diagnostic decision support.


title: The image of polynomials and Waring type problems on upper triangular matrix algebras

abstract: Let $p$ be a polynomial in non-commutative variables $x_1,x_2,\ldots,x_n$ with constant term zero over an algebraically closed field $K$. The object of study in this paper is the image of this kind of polynomial over the algebra of upper triangular matrices $T_m(K)$. We introduce a family of polynomials called multi-index $p$-inductive polynomials for a given polynomial $p$. Using this family we will show that, if $p$ is a polynomial identity of $T_t(K)$ but not of $T_{t+1}(K)$, then $p \left(T_m(K)\right)\subseteq T_m(K)^{(t-1)}$. Equality is achieved in the case $t=1,~m-1$ and an example has been provided to show that equality does not hold in general. We further prove existence of $d$ such that each element of $T_m(K)^{(t-1)}$ can be written as sum of $d$ many elements of $p\left( T_m(K) \right)$. It has also been shown that the image of $T_m(K)^\times$ under a word map is Zariski dense in $T_m(K)^\times$.


title: Creation of Optical Cat and GKP States Using Shaped Free Electrons

abstract: Cat states and Gottesman-Kitaev-Preskill (GKP) states play a key role in quantum computation and communication with continuous variables. The creation of such states relies on strong nonlinear light-matter interactions, which are widely available in microwave frequencies as in circuit quantum electrodynamics platforms. However, strong nonlinearities are hard to come by in optical frequencies, severely limiting the use of continuous variable quantum information in the optical range. Here we propose using the strong interactions of free electrons with light as a source for optical cat and GKP states. The strong interactions can be realized by phase-matching of free electrons with photonic structures such as optical waveguides and photonic crystals. Our approach enables the generation of optical GKP states with above 10 dB squeezing and fidelities above 90% at post-selection probability of 10%, even reaching >30% using an initially squeezed vacuum state. Furthermore, the free-electron interaction allows for conditional rotations on the photonic state, enabling to entangle a pair of GKP states into a GKP Bell state. Since electrons can interact resonantly with light across the electromagnetic spectrum, our approach may be used for the generation of cat and GKP states over the entire electromagnetic spectrum, from radio-waves to X-rays.


title: FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learning

abstract: Newton-type methods are popular in federated learning due to their fast convergence. Still, they suffer from two main issues, namely: low communication efficiency and low privacy due to the requirement of sending Hessian information from clients to parameter server (PS). In this work, we introduced a novel framework called FedNew in which there is no need to transmit Hessian information from clients to PS, hence resolving the bottleneck to improve communication efficiency. In addition, FedNew hides the gradient information and results in a privacy-preserving approach compared to the existing state-of-the-art. The core novel idea in FedNew is to introduce a two level framework, and alternate between updating the inverse Hessian-gradient product using only one alternating direction method of multipliers (ADMM) step and then performing the global model update using Newton's method. Though only one ADMM pass is used to approximate the inverse Hessian-gradient product at each iteration, we develop a novel theoretical approach to show the converging behavior of FedNew for convex problems. Additionally, a significant reduction in communication overhead is achieved by utilizing stochastic quantization. Numerical results using real datasets show the superiority of FedNew compared to existing methods in terms of communication costs.


title: Epsilon-regularity for the Brakke flow with boundary

abstract: We prove that, if a Brakke flow with boundary is close enough to a stationary half-plane with density one, then it is $C^{1,\alpha}$. Our approach is based on viscosity techniques introduced by Savin in the context of elliptic equations. The same techniques can be used to give a proof of Brakke's (interior) regularity theorem which is alternative to the original one.


title: Comparison of Deep Learning and Deterministic Algorithms for Control Modeling

abstract: Controlling nonlinear dynamics arise in various engineering fields. We present efforts to model the forced van der Pol system control using physics-informed neural networks (PINN) compared to benchmark methods, including idealized nonlinear feedforward (FF) control, linearized feedback control (FB), and feedforward-plus-feedback combined (C). The aim is to implement circular trajectories in the state space of the van der Pol system. A designed benchmark problem is used for testing the behavioral differences of the disparate controllers and then investigating controlled schemes and systems of various extents of nonlinearities. All methods exhibit a short initialization accompanying arbitrary initialization points. FF control successfully converges to the desired trajectory, and PINN executes good controls with higher stochasticity observed for higher-order terms based on the phase portraits. In contrast, combined control failed. Varying trajectory amplitudes revealed that FF, FB, and combined control all fail for unity nonlinear damping gain. Traditional control methods display a robust fluctuation for higher-order terms. For some various nonlinearities, PINN failed to implement the desired trajectory instead of becoming "trapped" in the phase of a small radius, yet FB successfully implemented controls. PINN generally exhibits lower relative errors for varying targeted trajectories. However, PINN also shows evidently higher computational burden compared with traditional control theory methods, with at least more than 30 times longer control time compared with benchmark idealized nonlinear feed-forward control. This manuscript proposes a comprehensive comparative study for future controller employment considering deterministic and machine learning approaches.


title: Prediction of Solar Radiation Based on Spatial and Temporal Embeddings for Solar Generation Forecast

abstract: A novel method for real-time solar generation forecast using weather data, while exploiting both spatial and temporal structural dependencies is proposed. The network observed over time is projected to a lower-dimensional representation where a variety of weather measurements are used to train a structured regression model while weather forecast is used at the inference stage. Experiments were conducted at 288 locations in the San Antonio, TX area on obtained from the National Solar Radiation Database. The model predicts solar irradiance with a good accuracy (R2 0.91 for the summer, 0.85 for the winter, and 0.89 for the global model). The best accuracy was obtained by the Random Forest Regressor. Multiple experiments were conducted to characterize influence of missing data and different time horizons providing evidence that the new algorithm is robust for data missing not only completely at random but also when the mechanism is spatial, and temporal.


title: A Comparative Study of Confidence Calibration in Deep Learning: From Computer Vision to Medical Imaging

abstract: Although deep learning prediction models have been successful in the discrimination of different classes, they can often suffer from poor calibration across challenging domains including healthcare. Moreover, the long-tail distribution poses great challenges in deep learning classification problems including clinical disease prediction. There are approaches proposed recently to calibrate deep prediction in computer vision, but there are no studies found to demonstrate how the representative models work in different challenging contexts. In this paper, we bridge the confidence calibration from computer vision to medical imaging with a comparative study of four high-impact calibration models. Our studies are conducted in different contexts (natural image classification and lung cancer risk estimation) including in balanced vs. imbalanced training sets and in computer vision vs. medical imaging. Our results support key findings: (1) We achieve new conclusions which are not studied under different learning contexts, e.g., combining two calibration models that both mitigate the overconfident prediction can lead to under-confident prediction, and simpler calibration models from the computer vision domain tend to be more generalizable to medical imaging. (2) We highlight the gap between general computer vision tasks and medical imaging prediction, e.g., calibration methods ideal for general computer vision tasks may in fact damage the calibration of medical imaging prediction. (3) We also reinforce previous conclusions in natural image classification settings. We believe that this study has merits to guide readers to choose calibration models and understand gaps between general computer vision and medical imaging domains.


title: Long-term future particle accelerators

abstract: Particle accelerators have enabled forefront research in high energy physics and other research areas for more than half a century. Accelerators have directly contributed to 26 Nobel Prizes in Physics since 1939 as well as another 20 Nobel Prizes in Chemistry, Medicine and Physics with X-rays. Although high energy physics has been the main driving force for the development of the particle accelerators, accelerator facilities have continually been expanding applications in many areas of research and technology. For instance, active areas of accelerator applications include radiotherapy to treat cancer, production of short-lived medical isotopes, synchrotron light sources, free-electron lasers, beam lithography for microcircuits, thin-film technology and radiation processing of food. Currently, the largest and most powerful accelerator is the Large Hadron Collider (LHC) at CERN, which accelerates protons to multi-TeV energies in a 27 km high-vacuum ring. To go beyond the maximum capabilities of the LHC, the next generation of circular and linear particle colliders under consideration, based on radiofrequency acceleration, will require multi-billion investment, kilometric infrastructure and a massive power consumption. These factors pose serious challenges in an increasingly resource-limited world. Therefore, it is important to look for alternative and sustainable acceleration techniques. This review article pays special attention to novel accelerator techniques to overcome present acceleration limitations towards more compact and cost-effective long term future accelerators.


title: What can Speech and Language Tell us About the Working Alliance in Psychotherapy

abstract: We are interested in the problem of conversational analysis and its application to the health domain. Cognitive Behavioral Therapy is a structured approach in psychotherapy, allowing the therapist to help the patient to identify and modify the malicious thoughts, behavior, or actions. This cooperative effort can be evaluated using the Working Alliance Inventory Observer-rated Shortened - a 12 items inventory covering task, goal, and relationship - which has a relevant influence on therapeutic outcomes. In this work, we investigate the relation between this alliance inventory and the spoken conversations (sessions) between the patient and the psychotherapist. We have delivered eight weeks of e-therapy, collected their audio and video call sessions, and manually transcribed them. The spoken conversations have been annotated and evaluated with WAI ratings by professional therapists. We have investigated speech and language features and their association with WAI items. The feature types include turn dynamics, lexical entrainment, and conversational descriptors extracted from the speech and language signals. Our findings provide strong evidence that a subset of these features are strong indicators of working alliance. To the best of our knowledge, this is the first and a novel study to exploit speech and language for characterising working alliance.


title: Semiclassical theory for plasmons in spatially inhomogeneous media

abstract: Recent progress in experimental techniques has made the quantum regime in plasmonics accessible. Since plasmons correspond to collective electron excitations, the electron-electron interaction plays an essential role in their theoretical description. Within the Random Phase Approximation, this interaction is incorporated through a system of equations of motion, which has to be solved self-consistently. For homogeneous media, an analytical solution can be found using the Fourier transform, giving rise to Lindhard theory. When the medium is spatially inhomogeneous, this is no longer possible and one often uses numerical approaches, which are however limited to smaller systems. In this paper, we present a novel semi-analytical approach for bulk plasmons in inhomogeneous media based on the semiclassical (or WKB) approximation, which is applicable when the charge density varies smoothly. By solving the equations of motion self-consistently, we obtain the expressions of Lindhard theory with a spatially varying Fermi wavevector. The derivation involves passing from the operators to their symbols, which can be thought of as classical observables on phase space. In this way we obtain effective (Hamiltonian) equations of motion for plasmons. We then find the quantized energy levels and the plasmon spectrum using Einstein-Brilllouin-Keller quantization. Our results provide a theoretical basis to describe different setups in quantum plasmonics, such as nanoparticles, quantum dots and waveguides.


title: Moment Generating Stirling Numbers and Applications

abstract: In this paper, we investigate certain combinatorial numbers, the \textit{moment generating Stirling numbers}. They are a special case of Hsu's generalized Stirling numbers and satisfy many more properties and combinatorial identities than are known in the general case. As application, we provide the computation of the moments and central moments of the phase type distribution, the recurrence time in Markov chains, the geometric distribution, the negative binomial distribution and of a class of distributions generalizing the negative binomial distribution. All computations can be performed in closed form without recursion. We also present the relationship to the Markov renewal process.


title: Cherrier-Escobar problem for the elliptic Schroedinger-to-Neumann map

abstract: In this paper, we study a Cherrier-Escobar problem for the extended problem corresponding to the elliptic Schroedinger-to-Neumann map on a compact 3-dimensional Riemannian manifold with boundary. Using the algebraic topological argument of Bahri-Coron, we show solvability under the assumption that the extended problem corresponding to the elliptic Schroedinger-to-Neumann map has a positive first eigenvalue, a positive Green function, and also verifies the strong maximum principle.


title: Decentralized adaptive clustering of deep nets is beneficial for client collaboration

abstract: We study the problem of training personalized deep learning models in a decentralized peer-to-peer setting, focusing on the setting where data distributions differ between the clients and where different clients have different local learning tasks. We study both covariate and label shift, and our contribution is an algorithm which for each client finds beneficial collaborations based on a similarity estimate for the local task. Our method does not rely on hyperparameters which are hard to estimate, such as the number of client clusters, but rather continuously adapts to the network topology using soft cluster assignment based on a novel adaptive gossip algorithm. We test the proposed method in various settings where data is not independent and identically distributed among the clients. The experimental evaluation shows that the proposed method performs better than previous state-of-the-art algorithms for this problem setting, and handles situations well where previous methods fail.


title: Exact modulus of continuities for $\Lambda$-Fleming-Viot processes with Brownian spatial motion

abstract: For a class of $\Lambda$-Fleming-Viot processes with Brownian spatial motion in $\mathbb{R}^d$ whose associated $\Lambda$-coalescents come down from infinity, we obtain sharp global and local modulus of continuities for the ancestry processes recovered from the lookdown representations. As applications, we prove both global and local modulus of continuities for the $\Lambda$-Fleming-Viot support processes. In particular, if the $\Lambda$-coalescent is the Beta$(2-\beta,\beta)$ coalescent for $\beta\in(1,2]$ with $\beta=2$ corresponding to Kingman's coalescent, then for $h(t)=\sqrt{t\log (1/t)}$, the global modulus of continuity holds for the support process with modulus function $\sqrt{2\beta/(\beta-1)}h(t)$, and both the left and right local modulus of continuities hold for the support process with modulus function $\sqrt{2/(\beta-1)}h(t)$.


title: Random projections and Kernelised Leave One Cluster Out Cross-Validation: Universal baselines and evaluation tools for supervised machine learning for materials properties

abstract: With machine learning being a popular topic in current computational materials science literature, creating representations for compounds has become common place. These representations are rarely compared, as evaluating their performance - and the performance of the algorithms that they are used with - is non-trivial. With many materials datasets containing bias and skew caused by the research process, leave one cluster out cross validation (LOCO-CV) has been introduced as a way of measuring the performance of an algorithm in predicting previously unseen groups of materials. This raises the question of the impact, and control, of the range of cluster sizes on the LOCO-CV measurement outcomes. We present a thorough comparison between composition-based representations, and investigate how kernel approximation functions can be used to better separate data to enhance LOCO-CV applications. We find that domain knowledge does not improve machine learning performance in most tasks tested, with band gap prediction being the notable exception. We also find that the radial basis function improves the linear separability of chemical datasets in all 10 datasets tested and provide a framework for the application of this function in the LOCO-CV process to improve the outcome of LOCO-CV measurements regardless of machine learning algorithm, choice of metric, and choice of compound representation. We recommend kernelised LOCO-CV as a training paradigm for those looking to measure the extrapolatory power of an algorithm on materials data.


title: Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval

abstract: Our goal in this research is to study a more realistic environment in which we can conduct weakly-supervised multi-modal instance-level product retrieval for fine-grained product categories. We first contribute the Product1M datasets, and define two real practical instance-level retrieval tasks to enable the evaluations on the price comparison and personalized recommendations. For both instance-level tasks, how to accurately pinpoint the product target mentioned in the visual-linguistic data and effectively decrease the influence of irrelevant contents is quite challenging. To address this, we exploit to train a more effective cross-modal pertaining model which is adaptively capable of incorporating key concept information from the multi-modal data, by using an entity graph whose node and edge respectively denote the entity and the similarity relation between entities. Specifically, a novel Entity-Graph Enhanced Cross-Modal Pretraining (EGE-CMP) model is proposed for instance-level commodity retrieval, that explicitly injects entity knowledge in both node-based and subgraph-based ways into the multi-modal networks via a self-supervised hybrid-stream transformer, which could reduce the confusion between different object contents, thereby effectively guiding the network to focus on entities with real semantic. Experimental results well verify the efficacy and generalizability of our EGE-CMP, outperforming several SOTA cross-modal baselines like CLIP, UNITER and CAPTURE.


title: AutoML Two-Sample Test

abstract: Two-sample tests are important in statistics and machine learning, both as tools for scientific discovery as well as to detect distribution shifts. This led to the development of many sophisticated test procedures going beyond the standard supervised learning frameworks, whose usage can require specialized knowledge about two-sample testing. We use a simple test that takes the mean discrepancy of a witness function as the test statistic and prove that minimizing a squared loss leads to a witness with optimal testing power. This allows us to leverage recent advancements in AutoML. Without any user input about the problems at hand, and using the same method for all our experiments, our AutoML two-sample test achieves competitive performance on a diverse distribution shift benchmark as well as on challenging two-sample testing problems. We provide an implementation of the AutoML two-sample test in the Python package autotst.


title: Fermionic-propagator and alternating-basis quantum Monte Carlo methods for correlated electrons on a lattice

abstract: Ultracold-atom simulations of the Hubbard model provide insights into the character of charge and spin correlations in and out of equilibrium. The corresponding numerical simulations, on the other hand, remain a significant challenge. We build on recent progress in the quantum Monte Carlo (QMC) simulation of electrons in continuous space, and apply similar ideas to the square-lattice Hubbard model. We devise and benchmark two discrete-time QMC methods, namely the fermionic-propagator QMC (FPQMC) and the alternating-basis QMC (ABQMC). In FPQMC, the time evolution is represented by snapshots in real space, whereas the snapshots in ABQMC alternate between real and reciprocal space. The methods may be applied to study equilibrium properties within grand-canonical or canonical ensemble, external field quenches, and even the evolution of pure states. Various real-space/reciprocal-space correlation functions are also within their reach. Both methods deal with matrices of size equal to the number of particles (thus independent of the number of orbitals or time slices), which allows for cheap updates. We benchmark the methods in relevant setups. In equilibrium, the FPQMC method is found to have excellent average sign and, in some cases, yields correct results even with poor imaginary-time discretization. ABQMC has significantly worse average sign, but also produces good results. Out of equilibrium, FPQMC suffers from a strong dynamical sign problem. On the contrary, in ABQMC, the sign problem is not time dependent. Using ABQMC, we compute survival probabilities for several experimentally relevant pure states.


title: Searching for Invisible Axion Dark Matter with an 18T Magnet Haloscope

abstract: We report the first search results for axion dark matter using an 18\,T high-temperature superconducting magnet haloscope. The scan frequency ranges from 4.7789 to 4.8094\,GHz. No significant signal consistent with the Galactic halo dark matter axion is observed. The results set the best upper bound of axion-photon-photon coupling ($g_{a\gamma\gamma}$) in the mass ranges of 19.764 to 19.771\,$\mu$eV (19.863 to 19.890\,$\mu$eV) at 1.5$\times|g_{a\gamma\gamma}^{\text{KSVZ}}|$ (1.7$\times|g_{a\gamma\gamma}^{\text{KSVZ}}|$), and 19.772 to 19.863\,$\mu$eV at 2.7 $\times|g_{a\gamma\gamma}^{\text{KSVZ}}|$ with 90\% confidence level, respectively. This remarkable sensitivity in the high mass region of dark matter axion is achieved by using the strongest magnetic field among the existing haloscope experiments and realizing a low-noise amplification of microwave signals using a Josephson parametric converter.


title: Mpemba-like effect protocol for granular gases of inelastic and rough hard disks

abstract: We study the conditions under which a Mpemba-like effect emerges in granular gases of inelastic and rough hard disks driven by a class of thermostats characterized by the splitting of the noise intensity into translational and rotational counterparts. Thus, granular particles are affected by a stochastic force and a stochastic torque, which inject translational and rotational energy, respectively. We realize that a certain choice of a thermostat of this class can be characterized just by the total intensity and the fraction of noise transferred to the rotational degree of freedom with respect to the translational ones. Firstly, Mpemba effect is characterized by the appearance of a crossing between the temperature curves of the considered samples. Later, an overshoot of the temperature evolution with respect to the steady-state value is observed and the mechanism of Mpemba effect generation is changed. The election of parameters allows to design plausible protocols based on these thermostats for generating the initial states to observe the Mpemba-like effect in experiments. In order to obtain explicit results, we use a well-founded Maxwellian approximation for the evolution dynamics and the steady-state quantities. Finally, theoretical results are compared with direct simulation Monte Carlo and molecular dynamics results, and a very good agreement is found.


title: Asymmetric transport computations in Dirac models of topological insulators

abstract: This paper presents a fast algorithm for computing transport properties of two-dimensional Dirac operators with linear domain walls, which model the macroscopic behavior of the robust and asymmetric transport observed at an interface separating two two-dimensional topological insulators. Our method is based on reformulating the partial differential equation as a corresponding volume integral equation, which we solve via a spectral discretization scheme. We demonstrate the accuracy of our method by confirming the quantization of an appropriate interface conductivity modeling transport asymmetry along the interface, and moreover confirm that this quantity is immune to local perturbations. We also compute the far-field scattering matrix generated by such perturbations and verify that while asymmetric transport is topologically protected the absence of back-scattering is not.

categories: math-ph cs.NA math.AP math.MP math.NA physics.comp-ph


title: Optimal tests of genuine multipartite nonlocality

abstract: We propose an optimal numerical test for genuine multipartite nonlocality based on linear programming. In particular, we consider two non-equivalent models of local hidden variables, namely the Svetlichny and the no-signaling bilocal model. While our knowledge concerning these models is well established for Bell scenarios involving two measurement settings per party, the general case based on an arbitrary number of settings is a considerably more challenging task and very little work has been done in this field. In this paper, we applied such general tests to detect and characterize genuine $n$-way nonlocal correlations for various states of three qubits and qutrits. As a measure of nonlocality, we use the probability of violation of local realism under randomly sampled observables, and the strength of nonlocality, described by the resistance to white noise admixture. In particular, we analyze to what extent the Bell scenario involving two measurement settings can be used to determine genuine $n$-way non-local correlations generated for more general models. In addition, we propose a simple procedure to detect genuine multipartite nonlocality for randomly chosen settings with up to 100% efficiency.


title: On the Bug-proneness of Structures Inspired by Functional Programming in JavaScript Projects

abstract: Language constructs inspired by functional programming have made their way into most mainstream programming languages. Many researchers and developers consider that these constructs lead to programs that are more concise, reusable, and easier to understand. However, few studies investigate the implications of using them in mainstream programming languages. This paper quantifies the prevalence of four concepts typically associated with functional programming in JavaScript: recursion, immutability, lazy evaluation, and functions as values. We focus on JavaScript programs due to the availability of some of these concepts in the language since its inception, its inspiration from functional programming languages, and its popularity. We mine 91 GitHub repositories (22+ million LOC) written mostly in JavaScript (over 50% of the code), measuring the usage of these concepts from both static and temporal perspectives. We also measure the likelihood of bug-fixing commits removing uses of these concepts (which would hint at bug-proneness) and their association with the presence of code comments (which would hint at code that is hard to understand). We find that these concepts are in widespread use (1 for every 46.65 LOC, 43.59% of LOC). In addition, the usage of higher-order functions, immutability, and lazy evaluation-related structures has been growing throughout the years for the analyzed projects, while the usage of recursion and callbacks & promises has decreased. We also find statistical evidence that removing these structures, with the exception of the ones associated to immutability, is less common in bug-fixing commits than in other commits. In addition, their presence is not correlated with comment size. Our findings suggest that functional programming concepts are important for developers using a multi-paradigm language, and their usage does not make programs harder to understand.


title: General Law of iterated logarithm for Markov processes: Liminf laws

abstract: Continuing from arXiv:2102.01917v2, in this paper, we discuss general criteria and forms of liminf laws of iterated logarithm (LIL) for continuous-time Markov processes. Under some minimal assumptions, which are weaker than those in arXiv:2102.01917v2, we establish liminf LIL at zero (at infinity, respectively) in general metric measure spaces. In particular, our assumptions for liminf law of LIL at zero and the form of liminf LIL are truly local so that we can cover highly space-inhomogenous cases. Our results cover all examples in arXiv:2102.01917v2 including random conductance models with long range jumps. Moreover, we show that the general form of liminf law of LIL at zero holds for a large class of jump processes whose jumping measures have logarithmic tails and Feller processes with symbols of varying order which are not covered before.


title: SMPL: Simulated Industrial Manufacturing and Process Control Learning Environments

abstract: Traditional biological and pharmaceutical manufacturing plants are controlled by human workers or pre-defined thresholds. Modernized factories have advanced process control algorithms such as model predictive control (MPC). However, there is little exploration of applying deep reinforcement learning to control manufacturing plants. One of the reasons is the lack of high fidelity simulations and standard APIs for benchmarking. To bridge this gap, we develop an easy-to-use library that includes five high-fidelity simulation environments: BeerFMTEnv, ReactorEnv, AtropineEnv, PenSimEnv and mAbEnv, which cover a wide range of manufacturing processes. We build these environments on published dynamics models. Furthermore, we benchmark online and offline, model-based and model-free reinforcement learning algorithms for comparisons of follow-up research.


title: Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes

abstract: Quantization is widely employed in both cloud and edge systems to reduce the memory occupation, latency, and energy consumption of deep neural networks. In particular, mixed-precision quantization, i.e., the use of different bit-widths for different portions of the network, has been shown to provide excellent efficiency gains with limited accuracy drops, especially with optimized bit-width assignments determined by automated Neural Architecture Search (NAS) tools. State-of-the-art mixed-precision works layer-wise, i.e., it uses different bit-widths for the weights and activations tensors of each network layer. In this work, we widen the search space, proposing a novel NAS that selects the bit-width of each weight tensor channel independently. This gives the tool the additional flexibility of assigning a higher precision only to the weights associated with the most informative features. Testing on the MLPerf Tiny benchmark suite, we obtain a rich collection of Pareto-optimal models in the accuracy vs model size and accuracy vs energy spaces. When deployed on the MPIC RISC-V edge processor, our networks reduce the memory and energy for inference by up to 63% and 27% respectively compared to a layer-wise approach, for the same accuracy.


title: MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge

abstract: Autonomous agents have made great strides in specialist domains like Atari games and Go. However, they typically learn tabula rasa in isolated environments with limited and manually conceived objectives, thus failing to generalize across a wide spectrum of tasks and capabilities. Inspired by how humans continually learn and adapt in the open world, we advocate a trinity of ingredients for building generalist agents: 1) an environment that supports a multitude of tasks and goals, 2) a large-scale database of multimodal knowledge, and 3) a flexible and scalable agent architecture. We introduce MineDojo, a new framework built on the popular Minecraft game that features a simulation suite with thousands of diverse open-ended tasks and an internet-scale knowledge base with Minecraft videos, tutorials, wiki pages, and forum discussions. Using MineDojo's data, we propose a novel agent learning algorithm that leverages large pre-trained video-language models as a learned reward function. Our agent is able to solve a variety of open-ended tasks specified in free-form language without any manually designed dense shaping reward. We open-source the simulation suite, knowledge bases, algorithm implementation, and pretrained models (https://minedojo.org) to promote research towards the goal of generally capable embodied agents.


title: Dynamics of a nonminimally coupled scalar field in asymptotically $AdS_4$ spacetime

abstract: We numerically investigate the stability of four-dimensional asymptotically anti-de Sitter ($AdS_4$) spacetime for a class of nonminimally coupled scalar field theories. In particular, we study how the coupling affects the formation of black holes and transfer of energy to different spatial/temporal scales. We conclude by detailing the well-known analogy between the nonminimally coupled scalar-field stress-energy tensor and that of a viscous relativistic fluid, and discuss the limitations of that analogy when it is applied to anisotropic scalar field configurations in asymptotically $AdS$ spacetimes.


title: Intelligent Trading System: Multidimensional financial time series clustering

abstract: Multidimensional time series clustering is an important problem in time series data analysis. This paper provides a new research idea for the behavioral analysis of financial markets, using the intrinsic correlation existing between transactions in the same segment of the financial market to cluster and analyze multidimensional time-series data, so as to obtain different types of market characteristics. In this paper, we propose a multidimensional time series clustering model based on graph attention autoencoder (GATE) and mask self-organizing map (Mask-SOM), based on which we realize multi-step prediction of financial derivatives prices and intelligent trading system construction. To obtain and fully utilize the correlation features between multidimensional financial time series data containing high noise for clustering analysis, constant curvature Riemannian manifolds are introduced in the graph attention autoencoder, and the multidimensional financial time series features captured by the encoder are embedded into the manifold. Following that, the multidimensional financial time series clustering analysis is implemented using Mask-SOM analysis manifold encoding. Finally, the feasibility and effectiveness of the model are verified using real financial datasets.


title: SwarmHive: Heterogeneous Swarm of Drones for Robust Autonomous Landing on Moving Robot

abstract: The paper focuses on a heterogeneous swarm of drones to achieve a dynamic landing of formation on a moving robot. This challenging task was not yet achieved by scientists. The key technology is that instead of facilitating each agent of the swarm of drones with computer vision that considerably increases the payload and shortens the flight time, we propose to install only one camera on the leader drone. The follower drones receive the commands from the leader UAV and maintain a collision-free trajectory with the artificial potential field. The experimental results revealed a high accuracy of the swarm landing on a static mobile platform (RMSE of 4.48 cm). RMSE of swarm landing on the mobile platform moving with the maximum velocities of 1.0 m/s and 1.5 m/s equals 8.76 cm and 8.98 cm, respectively. The proposed SwarmHive technology will allow the time-saving landing of the swarm for further drone recharging. This will make it possible to achieve self-sustainable operation of a multi-agent robotic system for such scenarios as rescue operations, inspection and maintenance, autonomous warehouse inventory, cargo delivery, and etc.


title: Universal co-Extensions of torsion abelian groups

abstract: In [16], a theory of universal extensions in abelian categories is developed; in particular, the notion of Ext-universal object is presented. In the present paper, we show that an Ab3 abelian category which is Ext-small satisfies the Ab4 condition if, and only if, each one of its objects is Ext-universal. We also give a characterization of the co-Ext-universal objects of the category of torsion abelian groups. In particular, we show that such groups are the ones admitting a decomposition $Q\oplus R$, in which $Q$ is injective and $R$ is a reduced group on which each $p$-component is bounded.


title: Distances for Comparing Multisets and Sequences

abstract: Measuring the distance between data points is fundamental to many statistical techniques, such as dimension reduction or clustering algorithms. However, improvements in data collection technologies has led to a growing versatility of structured data for which standard distance measures are inapplicable. In this paper, we consider the problem of measuring the distance between sequences and multisets of points lying within a metric space, motivated by the analysis of an in-play football data set. Drawing on the wider literature, including that of time series analysis and optimal transport, we discuss various distances which are available in such an instance. For each distance, we state and prove theoretical properties, proposing possible extensions where they fail. Finally, via an example analysis of the in-play football data, we illustrate the usefulness of these distances in practice.


title: From random walks on networks to nonlinear diffusion

abstract: Mathematical models of motility are often based on random-walk descriptions of discrete individuals that can move according to certain rules. It is usually the case that large masses concentrated in small regions of space have a great impact on the collective movement of the group. For this reason, many models in mathematical biology have incorporated crowding effects and managed to understand their implications. Here, we build on a previously developed framework for random walks on networks to show that in the continuum limit, the underlying stochastic process can be identified with a diffusion partial differential equation. The diffusion coefficient of the emerging equation is in general density-dependent, and can be directly related to the transition probabilities of the random walk. Moreover, the relaxation time of the stochastic process is directly linked to the diffusion coefficient and also to the network structure, as it usually happens in the case of linear diffusion. As a specific example, we study the equivalent of a porous-medium type equation on networks, which shows similar properties to its continuum equivalent. For this equation, self-similar solutions on a lattice and on homogeneous trees can be found, showing finite speed of propagation in contrast to commonly used linear diffusion equations. These findings also provide insights into reaction-diffusion systems with general diffusion operators, which have appeared recently in some applications.


title: Sparsity of Graphs that Allow Two Distinct Eigenvalues

abstract: The parameter $q(G)$ of a graph $G$ is the minimum number of distinct eigenvalues over the family of symmetric matrices described by $G$. It is shown that the minimum number of edges necessary for a connected graph $G$ to have $q(G)=2$ is $2n-4$ if $n$ is even, and $2n-3$ if $n$ is odd. In addition, a characterization of graphs for which equality is achieved in either case is given.


title: DGMIL: Distribution Guided Multiple Instance Learning for Whole Slide Image Classification

abstract: Multiple Instance Learning (MIL) is widely used in analyzing histopathological Whole Slide Images (WSIs). However, existing MIL methods do not explicitly model the data distribution, and instead they only learn a bag-level or instance-level decision boundary discriminatively by training a classifier. In this paper, we propose DGMIL: a feature distribution guided deep MIL framework for WSI classification and positive patch localization. Instead of designing complex discriminative network architectures, we reveal that the inherent feature distribution of histopathological image data can serve as a very effective guide for instance classification. We propose a cluster-conditioned feature distribution modeling method and a pseudo label-based iterative feature space refinement strategy so that in the final feature space the positive and negative instances can be easily separated. Experiments on the CAMELYON16 dataset and the TCGA Lung Cancer dataset show that our method achieves new SOTA for both global classification and positive patch localization tasks.


title: Analysis of Time- versus Event-Triggered Consensus for a Single-Integrator Multi-Agent System

abstract: It is well known that the employed triggering scheme has great impact on the control performance when control loops operate under scarce communication resources. Various practical and simulative works have demonstrated the potential of event-triggered control to reduce communication while providing a similar performance level when compared to time-triggered control. For non-cooperative networked control systems, analytical performance comparisons of time- and event-triggered control support this finding under certain assumptions. While being well-studied in the non-cooperative setting, it remains unclear if and how the performance relationship of the triggering schemes is altered in a multi-agent system setup. To close this gap, in this paper, we consider a homogeneous single-integrator multi-agent consensus problem for which we compare the performance of time- and event-triggered control schemes analytically. Under the assumption of equal average triggering rates, we use the long-term average of the quadratic deviation from consensus as a performance measure to contrast the triggering schemes. Contrary to the non-cooperative setting, we prove that event-triggered control performs worse than time-triggered control beyond a certain number of agents in this setup. In addition, we derive the asymptotic order of the performance measure as a function of the number of agents under both triggering schemes.


title: Translational Embeddings via Stable Canonical Rules

abstract: This paper presents a new uniform method for studying modal companions of superintuitionistic deductive systems and related notions, based on the machinery of stable canonical rules. Using our method, we obtain an alternative proof of the Blok-Esakia theorem both for logics and for rule systems, and prove an analogue of the Dummett-Lemmon conjecture for rule systems. Since stable canonical rules may be developed for any rule system admitting filtration, our method generalises smoothly to richer signatures. We illustrate this by applying our techniques to prove analogues of the Blok-Esakia theorem (for both logics and rule systems) and of the Dummett-Lemmon conjecture (for rule systems) in the setting of tense companions of bi-superintuitionistic deductive systems. We also use our techniques to prove that the lattice of rule systems (logics) extending the modal intuitionistic logic $\mathtt{KM}$ and the lattice of rule systems (logics) extending the provability logic $\mathtt{GL}$ are isomorphic.


title: Avoid Overfitting User Specific Information in Federated Keyword Spotting

abstract: Keyword spotting (KWS) aims to discriminate a specific wake-up word from other signals precisely and efficiently for different users. Recent works utilize various deep networks to train KWS models with all users' speech data centralized without considering data privacy. Federated KWS (FedKWS) could serve as a solution without directly sharing users' data. However, the small amount of data, different user habits, and various accents could lead to fatal problems, e.g., overfitting or weight divergence. Hence, we propose several strategies to encourage the model not to overfit user-specific information in FedKWS. Specifically, we first propose an adversarial learning strategy, which updates the downloaded global model against an overfitted local model and explicitly encourages the global model to capture user-invariant information. Furthermore, we propose an adaptive local training strategy, letting clients with more training data and more uniform class distributions undertake more local update steps. Equivalently, this strategy could weaken the negative impacts of those users whose data is less qualified. Our proposed FedKWS-UI could explicitly and implicitly learn user-invariant information in FedKWS. Abundant experimental results on federated Google Speech Commands verify the effectiveness of FedKWS-UI.


title: Vertex functions and their flow equations from the 2PI effective action

abstract: By exploiting the convexity of the two-particle-irreducible (2PI) effective action, we describe a procedure for extracting n-point vertex functions. This procedure is developed within the context of a zero-dimensional "quantum field theory" and subsequently extended to higher dimensions. These results extend the practicability and utility of a recent, alternative approach to the functional renormalization group programme [see arXiv:1907.06503; arXiv:2107.12914], and clarify the relationship between the flow equations for coupling parameters and vertices.


title: Thermal conductivity of triangular-lattice antiferromagnet Na2BaCo(PO4)2: Absence of itinerant fermionic excitations

abstract: We present the ultralow-temperature specific heat and thermal conductivity measurements on single crystals of triangular-lattice antiferromagnet Na$_2$BaCo(PO$_4$)$_2$, which was recently argued to host itinerant fermionic excitations, like a quantum spin liquid, above its antiferromagnetic phase transition temperature $T_{\rm N}$ = 0.148 K. In specific heat measurements, we confirm the peaks due to antiferromagnetic ordering when magnetic field $\mu_0 H \leq$ 1 T, roughly consistent with previous work [N. Li $et$ $al.$, Nat. Commun. 11, 4216 (2020)]. However, in thermal conductivity measurements, we observe negligible residual linear term in zero and finite magnetic fields, in sharp contrast to previous report [N. Li $et$ $al.$, Nat. Commun. 11, 4216 (2020)]. At 0.35 K, the thermal conductivity increases with field up to 3 T then saturates, similar to that of another triangular-lattice compound YbMgGaO$_4$, which further shows that the heat is conducted only by phonons with scattering from spins and boundary. Our results clearly demonstrate the absence of itinerant fermionic excitations in the disordered state above $T_{\rm N}$ in this frustrated antiferromagnet Na$_2$BaCo(PO$_4$)$_2$, thus such a state is not as exotic as previously reported.


title: Faraday effect of light caused by plane gravitational wave

abstract: Gravitational field can cause a rotation of polarisation plane of light. This phenomenon is known as the gravitational Faraday effect. We study the gravitational Faraday effect of linearly polarised light propagating in the gravitational field of a weak plane gravitational wave (GW) with $+$, $\times$, and elliptical polarisation modes. The corresponding gravitational Faraday rotation is proportional to the wave amplitude and to the squared distance traveled by light and it is inversely proportional to the GW's squared wavelength. The rotation is also maximal if light propagates in the direction perpendicular to the GW propagation, along directions of its polarisation. There is no gravitational Faraday rotation when light and the GW propagate in the same or opposite directions, or it propagates along directions perpendicular to directions of the GW polarisation. Helicity of elliptically polarised GW gives higher-order contribution to the gravitational Faraday rotation.


title: A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem

abstract: In this paper, we study a class of bilevel optimization problems, also known as simple bilevel optimization, where we minimize a smooth objective function over the optimal solution set of another convex constrained optimization problem. Several iterative methods have been developed for tackling this class of problems. Alas, their convergence guarantees are either asymptotic for the upper-level objective, or the convergence rates are slow and sub-optimal. To address this issue, in this paper, we introduce a novel bilevel optimization method that locally approximates the solution set of the lower-level problem via a cutting plane, and then runs a conditional gradient update to decrease the upper-level objective. When the upper-level objective is convex, we show that our method requires ${\mathcal{O}}(\max\{1/\epsilon_f,1/\epsilon_g\})$ iterations to find a solution that is $\epsilon_f$-optimal for the upper-level objective and $\epsilon_g$-optimal for the lower-level objective. Moreover, when the upper-level objective is non-convex, our method requires ${\mathcal{O}}(\max\{1/\epsilon_f^2,1/(\epsilon_f\epsilon_g)\})$ iterations to find an $(\epsilon_f,\epsilon_g)$-optimal solution. We also prove stronger convergence guarantees under the H\"olderian error bound assumption on the lower-level problem. To the best of our knowledge, our method achieves the best-known iteration complexity for the considered class of bilevel problems.


title: Fast Lossless Neural Compression with Integer-Only Discrete Flows

abstract: By applying entropy codecs with learned data distributions, neural compressors have significantly outperformed traditional codecs in terms of compression ratio. However, the high inference latency of neural networks hinders the deployment of neural compressors in practical applications. In this work, we propose Integer-only Discrete Flows (IODF), an efficient neural compressor with integer-only arithmetic. Our work is built upon integer discrete flows, which consists of invertible transformations between discrete random variables. We propose efficient invertible transformations with integer-only arithmetic based on 8-bit quantization. Our invertible transformation is equipped with learnable binary gates to remove redundant filters during inference. We deploy IODF with TensorRT on GPUs, achieving 10x inference speedup compared to the fastest existing neural compressors, while retaining the high compression rates on ImageNet32 and ImageNet64.


title: The Pixel Luminosity Telescope: A detector for luminosity measurement at CMS using silicon pixel sensors

abstract: The Pixel Luminosity Telescope is a silicon pixel detector dedicated to luminosity measurement at the CMS experiment at the LHC. It is located approximately 1.75 m from the interaction point and arranged into 16 "telescopes", with eight telescopes installed around the beam pipe at either end of the detector and each telescope composed of three individual silicon sensor planes. The per-bunch instantaneous luminosity is measured by counting events where all three planes in the telescope register a hit, using a special readout at the full LHC bunch-crossing rate of 40 MHz. The full pixel information is read out at a lower rate and can be used to determine calibrations, corrections, and systematic uncertainties for the online and offline measurements. This paper details the commissioning, operational history, and performance of the detector during Run 2 (2015-18) of the LHC, as well as preparations for Run 3, which will begin in 2022.


title: How Robust is Unsupervised Representation Learning to Distribution Shift?

abstract: The robustness of machine learning algorithms to distributions shift is primarily discussed in the context of supervised learning (SL). As such, there is a lack of insight on the robustness of the representations learned from unsupervised methods, such as self-supervised learning (SSL) and auto-encoder based algorithms (AE), to distribution shift. We posit that the input-driven objectives of unsupervised algorithms lead to representations that are more robust to distribution shift than the target-driven objective of SL. We verify this by extensively evaluating the performance of SSL and AE on both synthetic and realistic distribution shift datasets. Following observations that the linear layer used for classification itself can be susceptible to spurious correlations, we evaluate the representations using a linear head trained on a small amount of out-of-distribution (OOD) data, to isolate the robustness of the learned representations from that of the linear head. We also develop "controllable" versions of existing realistic domain generalisation datasets with adjustable degrees of distribution shifts. This allows us to study the robustness of different learning algorithms under versatile yet realistic distribution shift conditions. Our experiments show that representations learned from unsupervised learning algorithms generalise better than SL under a wide variety of extreme as well as realistic distribution shifts.


title: Singular cotangent models and complexity in fluids with dissipation

abstract: In this article we analyze several mathematical models with singularities where the classical cotangent model is replaced by a $b$-cotangent model. We provide physical interpretations of the singular symplectic geometry underlying in $b$-cotangent bundles featuring two models: the canonical (or non-twisted) model and the twisted one. The first one models systems on manifolds with boundary and the twisted model represents Hamiltonian systems where the singularity of the system is in the fiber of the bundle. The twisted cotangent model includes (for linear potentials) the case of fluids with dissipation. We relate the complexity of the fluids in terms of the Reynolds number and the (non)-existence of cotangent lift dynamics. We also discuss more general physical interpretations of the twisted and non-twisted $b$-symplectic models. These models offer a Hamiltonian formulation for systems which are dissipative, extending the horizons of Hamiltonian dynamics and opening a new approach to study non-conservative systems.


title: Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM

abstract: Many problems in machine learning can be formulated as optimizing a convex functional over a vector space of measures. This paper studies the convergence of the mirror descent algorithm in this infinite-dimensional setting. Defining Bregman divergences through directional derivatives, we derive the convergence of the scheme for relatively smooth and convex pairs of functionals. Such assumptions allow to handle non-smooth functionals such as the Kullback--Leibler (KL) divergence. Applying our result to joint distributions and KL, we show that Sinkhorn's primal iterations for entropic optimal transport in the continuous setting correspond to a mirror descent, and we obtain a new proof of its (sub)linear convergence. We also show that Expectation Maximization (EM) can always formally be written as a mirror descent. When optimizing only on the latent distribution while fixing the mixtures parameters -- which corresponds to the Richardson--Lucy deconvolution scheme in signal processing -- we derive sublinear rates of convergence.


title: SwarmHawk: Self-Sustaining Multi-Agent System for Landing on a Moving Platform through an Agent Supervision

abstract: Heterogeneous teams of mobile robots and UAVs are offering a substantial benefit in an autonomous exploration of the environment. Nevertheless, although joint exploration scenarios for such systems are widely discussed, they are still suffering from low adaptability to changes in external conditions and faults of swarm agents during the UAV docking. We propose a novel vision-based drone swarm docking system for robust landing on a moving platform when one of the agents lost its position signal. The proposed SwarmHawk system relies on vision-based detection for the mobile platform tracking and navigation of its agents. Each drone of the swarm carries an RGB camera and AprilTag3 QR-code marker on board. SwarmHawk can switch between two modes of operation, acting as a homogeneous swarm in case of global UAV localization or assigning leader drones to navigate its neighbors in case of a camera fault in one of the drones or global localization failure. Two experiments were performed to evaluate SwarmHawk's performance under the global and local localization with static and moving platforms. The experimental results revealed a sufficient accuracy in the swarm landing task on a static mobile platform (error of 4.2 cm in homogeneous formation and 1.9 cm in leader-follower formation) and on moving platform (error of 6.9 cm in homogeneous formation and 4.7 cm in leader-follower formation). Moreover, the drones showed a good landing on a platform moving along a complex trajectory (average error of 19.4 cm) in leader-follower formation. The proposed SwarmHawk technology can be potentially applied in various swarm scenarios, including complex environment exploration, inspection, and drone delivery.


title: Correlators of Four Light-Ray Operators in CCFT

abstract: Building on results in arXiv:2203.04255, we compute the correlator of four gluon light-ray operators in celestial CFT. We find that it is described by Fox H-functions and generalized I-functions of multiple variables. We also analyze light-ray correlators for a scalar amplitude involving the exchange of a massive scalar.


title: Transverse Momentum Dependent Fragmentation Functions from recent BELLE data

abstract: A new formalism for the factorization of the cross section for single hadron production in $e^+e^-$ annihilations, differential in $z_h$, $P_T$ and thrust, is applied to the phenomenological analysis of data recently measured by the BELLE Collaboration. Within this scheme the $e^+e^- \to hX$ cross section can be recast in the convolution of a perturbatively calculable coefficient and a universal transverse momentum dependent fragmentation function. While performing a next-to-leading order calculation of the perturbative part of the process to next-to-leading logarithmic accuracy, we examine and thoroughly discuss the suitability of a number of possible ansatz to model the non-perturbative part of this universal transverse momentum dependent fragmentation function, showing the extent to which present experimental data can actually constrain its shape and functional form in terms of $z_h$, $P_T$ and thrust.


title: BUDDI-MaNGA I: A statistical sample of cleanly decomposed bulge and disc spectra

abstract: Many galaxies display clear bulges and discs, and understanding how these components form is a vital step towards understanding how the galaxy has evolved into what we see today. The BUDDI-MaNGA project aims to study galaxy evolution and morphological transformations through the star-formation histories of the bulges and discs. We have applied our BUDDI software to galaxies from the MaNGA Survey in the SDSS DR15 in order to isolate their bulge and disc spectra, from which we derived their stellar populations. To date, this work provides the largest sample of clean bulge and disc spectra extracted from IFU datacubes using the galaxies light profile information, and will form the basis for a series of papers aiming to answer open questions on how galaxies have formed and evolved, and the role of their individual structures. This paper presents an introduction to the project, including an overview of these fits, a characterisation of the sample, and a series of tests on the fits to ensure reliability.


title: BUDDI-MaNGA II: The Star-Formation Histories of Bulges and Discs of S0s

abstract: Many processes have been proposed to explain the quenching of star formation in spiral galaxies and their transformation into S0s. These processes affect the bulge and disc in different ways, and so by isolating the bulge and disc spectra, we can look for these characteristic signatures. In this work, we used BUDDI to cleanly extract the spectra of the bulges and discs of 78 S0 galaxies in the MaNGA Survey. We compared the luminosity and mass weighted stellar populations of the bulges and discs, finding that bulges are generally older and more metal rich than their discs. When considering the mass and environment of each galaxy, we found that the galaxy stellar mass plays a more significant role on the formation of the bulges. Bulges in galaxies with masses $\geq10^{10}M_\odot$ built up the majority of their mass rapidly early in their lifetimes, while those in lower mass galaxies formed over more extended timescales and more recently. No clear difference was found in the formation or quenching processes of the discs as a function of galaxy environment. We conclude that more massive S0 galaxies formed through an inside-out scenario, where the bulge formed first and evolved passively while the disc underwent a more extended period of star formation. In lower mass S0s, the bulges and discs either formed together from the same material, or through an outside-in scenario. Our results therefore imply multiple formation mechanisms for S0 galaxies, the pathway of which is chiefly determined by a galaxy's current stellar mass.


title: Continuous cohomology of gauge algebras and bornological Loday-Quillen-Tsygan theorems

abstract: We investigate the well-known Loday-Quillen-Tsygan theorem, which calculates the Lie algebra homology of the general linear algebra $\mathfrak{gl}(A)$ for an associative algebra $A$ in terms of cyclic homology, and extend the proof to bornological Lie algebra homology of Fr\'echet and LF-algebras. For Fr\'echet spaces, this equals continuous Lie algebra homology. To this end we prepare several statements about homological algebra of topological vector spaces, and discuss when the differential of the bornological Hochschild and cyclic complex are topological homomorphisms in the setting of Fr\'echet algebras. We apply the results to the algebras of smooth functions on a smooth manifold and compactly supported smooth functions on Euclidean space, and construct from a local-to-global principle a Gelfand-Fuks-like spectral sequence which calculates the stable part of bornological Lie algebra homology of nontrivial gauge algebras. This complements results by Maier, Janssens and Wockel.


title: Improving Generalization of Metric Learning via Listwise Self-distillation

abstract: Most deep metric learning (DML) methods employ a strategy that forces all positive samples to be close in the embedding space while keeping them away from negative ones. However, such a strategy ignores the internal relationships of positive (negative) samples and often leads to overfitting, especially in the presence of hard samples and mislabeled samples. In this work, we propose a simple yet effective regularization, namely Listwise Self-Distillation (LSD), which progressively distills a model's own knowledge to adaptively assign a more appropriate distance target to each sample pair in a batch. LSD encourages smoother embeddings and information mining within positive (negative) samples as a way to mitigate overfitting and thus improve generalization. Our LSD can be directly integrated into general DML frameworks. Extensive experiments show that LSD consistently boosts the performance of various metric learning methods on multiple datasets.


title: Logic-based Reward Shaping for Multi-Agent Reinforcement Learning

abstract: Reinforcement learning (RL) relies heavily on exploration to learn from its environment and maximize observed rewards. Therefore, it is essential to design a reward function that guarantees optimal learning from the received experience. Previous work has combined automata and logic based reward shaping with environment assumptions to provide an automatic mechanism to synthesize the reward function based on the task. However, there is limited work on how to expand logic-based reward shaping to Multi-Agent Reinforcement Learning (MARL). The environment will need to consider the joint state in order to keep track of other agents if the task requires cooperation, thus suffering from the curse of dimensionality with respect to the number of agents. This project explores how logic-based reward shaping for MARL can be designed for different scenarios and tasks. We present a novel method for semi-centralized logic-based MARL reward shaping that is scalable in the number of agents and evaluate it in multiple scenarios.


title: Edge-Aided Sensor Data Sharing in Vehicular Communication Networks

abstract: Sensor data sharing in vehicular networks can significantly improve the range and accuracy of environmental perception for connected automated vehicles. Different concepts and schemes for dissemination and fusion of sensor data have been developed. It is common to these schemes that measurement errors of the sensors impair the perception quality and can result in road traffic accidents. Specifically, when the measurement error from the sensors (also referred as measurement noise) is unknown and time varying, the performance of the data fusion process is restricted, which represents a major challenge in the calibration of sensors. In this paper, we consider sensor data sharing and fusion in a vehicular network with both, vehicle-to-infrastructure and vehicle-to-vehicle communication. We propose a method, named Bidirectional Feedback Noise Estimation (BiFNoE), in which an edge server collects and caches sensor measurement data from vehicles. The edge estimates the noise and the targets alternately in double dynamic sliding time windows and enhances the distributed cooperative environment sensing at each vehicle with low communication costs. We evaluate the proposed algorithm and data dissemination strategy in an application scenario by simulation and show that the perception accuracy is on average improved by around 80 % with only 12 kbps uplink and 28 kbps downlink bandwidth.


title: CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer

abstract: Transformer has achieved great successes in learning vision and language representation, which is general across various downstream tasks. In visual control, learning transferable state representation that can transfer between different control tasks is important to reduce the training sample size. However, porting Transformer to sample-efficient visual control remains a challenging and unsolved problem. To this end, we propose a novel Control Transformer (CtrlFormer), possessing many appealing benefits that prior arts do not have. Firstly, CtrlFormer jointly learns self-attention mechanisms between visual tokens and policy tokens among different control tasks, where multitask representation can be learned and transferred without catastrophic forgetting. Secondly, we carefully design a contrastive reinforcement learning paradigm to train CtrlFormer, enabling it to achieve high sample efficiency, which is important in control problems. For example, in the DMControl benchmark, unlike recent advanced methods that failed by producing a zero score in the "Cartpole" task after transfer learning with 100k samples, CtrlFormer can achieve a state-of-the-art score with only 100k samples while maintaining the performance of previous tasks. The code and models are released in our project homepage.


title: Resolution Limits of Non-Adaptive 20 Questions Search for a Moving Target

abstract: Using the 20 questions estimation framework with query-dependent noise, we study non-adaptive search strategies for a moving target over the unit cube with unknown initial location and velocities under a piecewise constant velocity model. In this search problem, there is an oracle who knows the instantaneous location of the target at any time. Our task is to query the oracle as few times as possible to accurately estimate the location of the target at any specified time. We first study the case where the oracle's answer to each query is corrupted by discrete noise and then generalize our results to the case of additive white Gaussian noise. In our formulation, the performance criterion is the resolution, which is defined as the maximal $L_\infty$ distance between the true locations and estimated locations. We characterize the minimal resolution of an optimal non-adaptive query procedure with a finite number of queries by deriving non-asymptotic and asymptotic bounds. Our bounds are tight in the first-order asymptotic sense when the number of queries satisfies a certain condition and our bounds are tight in the stronger second-order asymptotic sense when the target moves with a constant velocity. To prove our results, we relate the current problem to channel coding, borrow ideas from finite blocklength information theory and construct bounds on the number of possible quantized target trajectories.


title: Incorporating intratumoral heterogeneity into weakly-supervised deep learning models via variance pooling

abstract: Supervised learning tasks such as cancer survival prediction from gigapixel whole slide images (WSIs) are a critical challenge in computational pathology that requires modeling complex features of the tumor microenvironment. These learning tasks are often solved with deep multi-instance learning (MIL) models that do not explicitly capture intratumoral heterogeneity. We develop a novel variance pooling architecture that enables a MIL model to incorporate intratumoral heterogeneity into its predictions. Two interpretability tools based on representative patches are illustrated to probe the biological signals captured by these models. An empirical study with 4,479 gigapixel WSIs from the Cancer Genome Atlas shows that adding variance pooling onto MIL frameworks improves survival prediction performance for five cancer types.


title: Learning to self-fold at a bifurcation

abstract: Disordered mechanical systems can deform along a network of pathways that branch and recombine at special configurations called bifurcation points. Multiple pathways are accessible from these bifurcation points; consequently, computer-aided design algorithms have been sought to achieve a specific structure of pathways at bifurcations by rationally designing the geometry and material properties of these systems. Here, we explore an alternative physical training framework in which the topology of folding pathways in a disordered sheet is changed in a desired manner due to changes in crease stiffnesses induced by prior folding. We study the quality and robustness of such training for different `learning rules', that is, different quantitative ways in which local strain changes the local folding stiffness. We experimentally demonstrate these ideas using sheets with epoxy-filled creases whose stiffnesses change due to folding before the epoxy sets. Our work shows how specific forms of plasticity in materials enable them to learn non-linear behaviors through their prior deformation history in a robust manner.


title: Characteristics of Interference in Licensed and Unlicensed Bands for Intelligent Spectrum Management

abstract: The exponential growth of IoT devices and the demand of smart devices for higher data rates has heightened the need for sharing and managing spectrum resources in cellular 5G/6G operating in licensed bands and Wi-Fi technologies operating in unlicensed bands. Intelligent spectrum management has emerged as a key concept in dynamic spectrum allocation. To understand the interference existing in the spectrum, researchers usually monitor the interference in a fixed location and either focus on the cellular band or Wi-Fi band. In this study, we conduct experiments for collecting real-time spectrum data in indoor and outdoor environments with a mobile receiver, the spectrum analyzer. For outdoor, we mount the spectrum analyzer on a car seat and drive on the selected route in an urban area. We put the analyzer on a cart and moved it around in the laboratory for indoor. The frequency of interest in this study is 1.9 - 2.5 GHz, including both licensed and unlicensed bands. Temporal and frequency domain behavior is compared between licensed and unlicensed bands. We first normalize and binarize the data with a threshold. Then we calculate the spectrum occupancy by counting how many consecutive ones. Based on our observation, the spectrum occupancy of the outdoor environment is more remarkable than the indoor environment. The interference in the licensed band shows more variations in the frequency domain than that in the unlicensed band. This study provides a better understanding of the interference behavior for different environments and frequency bands.


title: Fast Population-Based Reinforcement Learning on a Single Machine

abstract: Training populations of agents has demonstrated great promise in Reinforcement Learning for stabilizing training, improving exploration and asymptotic performance, and generating a diverse set of solutions. However, population-based training is often not considered by practitioners as it is perceived to be either prohibitively slow (when implemented sequentially), or computationally expensive (if agents are trained in parallel on independent accelerators). In this work, we compare implementations and revisit previous studies to show that the judicious use of compilation and vectorization allows population-based training to be performed on a single machine with one accelerator with minimal overhead compared to training a single agent. We also show that, when provided with a few accelerators, our protocols extend to large population sizes for applications such as hyperparameter tuning. We hope that this work and the public release of our code will encourage practitioners to use population-based learning more frequently for their research and applications.


title: Lossy Compression with Gaussian Diffusion

abstract: We consider a novel lossy compression approach based on unconditional diffusion generative models, which we call DiffC. Unlike modern compression schemes which rely on transform coding and quantization to restrict the transmitted information, DiffC relies on the efficient communication of pixels corrupted by Gaussian noise. We implement a proof of concept and find that it works surprisingly well despite the lack of an encoder transform, outperforming the state-of-the-art generative compression method HiFiC on ImageNet 64x64. DiffC only uses a single model to encode and denoise corrupted pixels at arbitrary bitrates. The approach further provides support for progressive coding, that is, decoding from partial bit streams. We perform a rate-distortion analysis to gain a deeper understanding of its performance, providing analytical results for multivariate Gaussian data as well as theoretic bounds for general distributions. Furthermore, we prove that a flow-based reconstruction achieves a 3 dB gain over ancestral sampling at high bitrates.


title: Disentangling Model Multiplicity in Deep Learning

abstract: Model multiplicity is a well-known but poorly understood phenomenon that undermines the generalisation guarantees of machine learning models. It appears when two models with similar training-time performance differ in their predictions and real-world performance characteristics. This observed 'predictive' multiplicity (PM) also implies elusive differences in the internals of the models, their 'representational' multiplicity (RM). We introduce a conceptual and experimental setup for analysing RM by measuring activation similarity via singular vector canonical correlation analysis (SVCCA). We show that certain differences in training methods systematically result in larger RM than others and evaluate RM and PM over a finite sample as predictors for generalizability. We further correlate RM with PM measured by the variance in i.i.d. and out-of-distribution test predictions in four standard image data sets. Finally, instead of attempting to eliminate RM, we call for its systematic measurement and maximal exposure.


title: Cyclic isogenies of elliptic curves over fixed quadratic fields

abstract: Building on Mazur's 1978 work on prime degree isogenies, Kenku determined in 1981 all possible cyclic isogenies of elliptic curves over $\mathbb{Q}$. Although more than 40 years have passed, the determination of cyclic isogenies of elliptic curves over a single other number field has hitherto not been realised. In this paper we develop a procedure to assist in establishing such a determination for a given quadratic field. Executing this procedure on all quadratic fields $\mathbb{Q}(\sqrt{d})$ with $|d| < 10^4$ we obtain, conditional on the Generalised Riemann Hypothesis, the determination of cyclic isogenies of elliptic curves over $19$ quadratic fields, including $\mathbb{Q}(\sqrt{213})$ and $\mathbb{Q}(\sqrt{-2289})$. To make this procedure work, we determine all of the finitely many quadratic points on the modular curves $X_0(125)$ and $X_0(169)$, which may be of independent interest.


title: Cluster-Head Rotation Approaches in Sensor Networks: A Review

abstract: Balanced energy consumption is a major research concern in sensor networks. If some sensor nodes spent energy rapidly compared to other sensor groups, then the energy consumption is not evenly distributed and the lifetime of the network is adversely affected. In cluster-based communication protocol, cluster-head nodes rapidly dissipate their energy compared to respective cluster members as it has additional duties of receiving data from cluster members and aggregation of received data. Clustering protocols periodically rotate the energy-intensive job of cluster-head among other sensors in order to evenly distribute the CH's energy load. In this paper, we review various cluster-head rotation approaches and classify them into different categories based on the number of participating nodes, rotation mechanism, and decision parameter(s). Finally, we discuss open research issues in developing an efficient cluster-head rotation approach, which reduces the premature death of cluster-head nodes and maximizes the network lifetime.


title: A refinement of some previous results of Bernardara-Marcolli-Tabuada and Ornaghi-Pertusi

abstract: The Voevodsky nilpotence conjecture was proved by Bernardara-Marcolli-Tabuada for certain quadric fibrations, intersections of quadrics, linear sections of Grassmannians, linear sections of determinantal varieties, and Moishezon manifolds, and later by Ornaghi-Pertusi for certain cubic fourfolds and Gushel-Mukai fourfolds. In this paper we refine these results by using the algebraic equivalence relation instead of the nilpotence equivalence relation. Along the way, we address also certain cases of K\"uchle fourfolds, families of Sextic del Pezzo surfaces and families of Fano fourfolds of K3 type.


title: Scaling multi-species occupancy models to large citizen science datasets

abstract: Citizen science datasets can be very large and promise to improve species distribution modelling, but detection is imperfect, risking bias when fitting models. In particular, observers may not detect species that are actually present. Occupancy models can estimate and correct for this observation process, and multi-species occupancy models exploit similarities in the observation process, which can improve estimates for rare species. However, the computational methods currently used to fit these models do not scale to large datasets. We develop approximate Bayesian inference methods and use graphics processing units (GPUs) to scale multi-species occupancy models to very large citizen science data. We fit multi-species occupancy models to one month of data from the eBird project consisting of 186,811 checklist records comprising 430 bird species. We evaluate the predictions on a spatially separated test set of 59,338 records, comparing two different inference methods -- Markov chain Monte Carlo (MCMC) and variational inference (VI) -- to occupancy models fitted to each species separately using maximum likelihood. We fitted models to the entire dataset using VI, and up to 32,000 records with MCMC. VI fitted to the entire dataset performed best, outperforming single-species models on both AUC (90.4% compared to 88.7%) and on log likelihood (-0.080 compared to -0.085). We also evaluate how well range maps predicted by the model agree with expert maps. We find that modelling the detection process greatly improves agreement and that the resulting maps agree as closely with expert maps as ones estimated using high quality survey data. Our results demonstrate that multi-species occupancy models are a compelling approach to model large citizen science datasets, and that, once the observation process is taken into account, they can model species distributions accurately.


title: Attosecond Field Emission

abstract: Field-emission of electrons underlies major advances in science and technology, ranging from imaging the atomic-scale structure of matter to signal processing at ever-higher frequencies. The advancement of these applications to their ultimate limits of temporal resolution and frequency calls for techniques that can confine and probe the field emission on the sub-femtosecond time scale. We used intense, sub-cycle transients to induce optical field emission of electron pulses from tungsten nanotips and a weak replica of the same transient to directly probe the emission dynamics in real-time. Access into the temporal profile of the emerging electron pulses, including the duration $\tau$ = (53 as $\pm$ 5 as) and chirp, and the direct probing of nanoscale near-fields, open new prospects for research and applications at the interface of attosecond physics and nanooptics.


title: Evolution through Large Models

abstract: This paper pursues the insight that large language models (LLMs) trained to generate code can vastly improve the effectiveness of mutation operators applied to programs in genetic programming (GP). Because such LLMs benefit from training data that includes sequential changes and modifications, they can approximate likely changes that humans would make. To highlight the breadth of implications of such evolution through large models (ELM), in the main experiment ELM combined with MAP-Elites generates hundreds of thousands of functional examples of Python programs that output working ambulating robots in the Sodarace domain, which the original LLM had never seen in pre-training. These examples then help to bootstrap training a new conditional language model that can output the right walker for a particular terrain. The ability to bootstrap new models that can output appropriate artifacts for a given context in a domain where zero training data was previously available carries implications for open-endedness, deep learning, and reinforcement learning. These implications are explored here in depth in the hope of inspiring new directions of research now opened up by ELM.


title: Loss-driven topological transitions in lasing

abstract: We experimentally observe lasing in a hexamer plasmonic lattice and find that when tuning the scale of the unit cell, the polarization winding of the emission changes. By a theoretical analysis we identify the lasing modes as quasi bound states in continuum (quasi-BICs) of topological charges of zero, one or two. A T-matrix simulation of the structure reveals that the mode quality(Q)-factors depend on the scale of the unit cell, with highest-Q modes favored by lasing. The system thus shows a loss-driven transition between lasing in modes of trivial and high-order topological charge.


title: SimA: Simple Softmax-free Attention for Vision Transformers

abstract: Recently, vision transformers have become very popular. However, deploying them in many applications is computationally expensive partly due to the Softmax layer in the attention block. We introduce a simple but effective, Softmax-free attention block, SimA, which normalizes query and key matrices with simple $\ell_1$-norm instead of using Softmax layer. Then, the attention block in SimA is a simple multiplication of three matrices, so SimA can dynamically change the ordering of the computation at the test time to achieve linear computation on the number of tokens or the number of channels. We empirically show that SimA applied to three SOTA variations of transformers, DeiT, XCiT, and CvT, results in on-par accuracy compared to the SOTA models, without any need for Softmax layer. Interestingly, changing SimA from multi-head to single-head has only a small effect on the accuracy, which simplifies the attention block further. The code is available here: https://github.com/UCDvision/sima


title: Popular decision tree algorithms are provably noise tolerant

abstract: Using the framework of boosting, we prove that all impurity-based decision tree learning algorithms, including the classic ID3, C4.5, and CART, are highly noise tolerant. Our guarantees hold under the strongest noise model of nasty noise, and we provide near-matching upper and lower bounds on the allowable noise rate. We further show that these algorithms, which are simple and have long been central to everyday machine learning, enjoy provable guarantees in the noisy setting that are unmatched by existing algorithms in the theoretical literature on decision tree learning. Taken together, our results add to an ongoing line of research that seeks to place the empirical success of these practical decision tree algorithms on firm theoretical footing.


title: Adapting the Linearised Laplace Model Evidence for Modern Deep Learning

abstract: The linearised Laplace method for estimating model uncertainty has received renewed attention in the Bayesian deep learning community. The method provides reliable error bars and admits a closed-form expression for the model evidence, allowing for scalable selection of model hyperparameters. In this work, we examine the assumptions behind this method, particularly in conjunction with model selection. We show that these interact poorly with some now-standard tools of deep learning--stochastic approximation methods and normalisation layers--and make recommendations for how to better adapt this classic method to the modern setting. We provide theoretical support for our recommendations and validate them empirically on MLPs, classic CNNs, residual networks with and without normalisation layers, generative autoencoders and transformers.