title: Theta bundle, Quillen connection and the Hodge theoretic projective structure

abstract: There are two canonical projective structures on any compact Riemann surface of genus at least two: one coming from the uniformization theorem, and the other from Hodge theory. They produce two (different) families of projective structures over the moduli space $M_g$ of compact Riemann surfaces. A recent work of Biswas, Favale, Pirola, and Torelli shows that families of projective structures over $M_g$ admit an equivalent characterization in terms of complex connections on the dual $\mathcal{L}$ of the determinant of the Hodge line bundle over $M_g$; the same work gave the connection on $\mathcal L$ corresponding to the projective structures coming from uniformization. Here we construct the connection on $\mathcal L$ corresponding to the family of Hodge theoretic projective structures. This connection is described in three different ways: firstly as the connection induced on $\mathcal L$ by the Chern connection of the $L^2$-metric on the Hodge bundle, secondly as an appropriate root of the Quillen metric induced by the (square of the) Theta line bundle on the universal family of abelian varieties, endowed with the natural Hermitian metric given by the polarization, and finally as Quillen connection gotten using the Arakelov metric on the universal curve, modified by Faltings' delta invariant.


title: Certifying ground-state properties of quantum many-body systems

abstract: A ubiquitous problem in quantum physics is to understand the ground-state properties of many-body systems. Confronted with the fact that exact diagonalisation quickly becomes impossible when increasing the system size, variational approaches are typically employed as a scalable alternative: energy is minimised over a subset of all possible states and then different physical quantities are computed over the solution state. Despite remarkable success, rigorously speaking, all what variational methods offer are upper bounds on the ground-state energy. On the other hand, so-called relaxations of the ground-state problem based on semidefinite programming represent a complementary approach, providing lower bounds to the ground-state energy. However, in their current implementation, neither variational nor relaxation methods offer provable bound on other observables in the ground state beyond the energy. In this work, we show that the combination of the two classes of approaches can be used to derive certifiable bounds on the value of any observable in the ground state, such as correlation functions of arbitrary order, structure factors, or order parameters. We illustrate the power of this approach in paradigmatic examples of 1D and 2D spin-one-half Heisenberg models. To improve the scalability of the method, we exploit the symmetries and sparsity of the considered systems to reach sizes of hundreds of particles at much higher precision than previous works. Our analysis therefore shows how to obtain certifiable bounds on many-body ground-state properties beyond energy in a scalable way.


title: GraphLLM: Boosting Graph Reasoning Ability of Large Language Model

abstract: The advancement of Large Language Models (LLMs) has remarkably pushed the boundaries towards artificial general intelligence (AGI), with their exceptional ability on understanding diverse types of information, including but not limited to images and audio. Despite this progress, a critical gap remains in empowering LLMs to proficiently understand and reason on graph data. Recent studies underscore LLMs' underwhelming performance on fundamental graph reasoning tasks. In this paper, we endeavor to unearth the obstacles that impede LLMs in graph reasoning, pinpointing the common practice of converting graphs into natural language descriptions (Graph2Text) as a fundamental bottleneck. To overcome this impediment, we introduce GraphLLM, a pioneering end-to-end approach that synergistically integrates graph learning models with LLMs. This synergy equips LLMs with the ability to proficiently interpret and reason on graph data, harnessing the superior expressive power of graph learning models. Our empirical evaluations across four fundamental graph reasoning tasks validate the effectiveness of GraphLLM. The results exhibit a substantial average accuracy enhancement of 54.44%, alongside a noteworthy context reduction of 96.45% across various graph reasoning tasks.


title: Modeling a Coronal Mass Ejection from an Extended Filament Channel. II. Interplanetary Propagation to 1 au

abstract: We present observations and modeling results of the propagation and impact at Earth of a high-latitude, extended filament channel eruption that commenced on 2015 July 9. The coronal mass ejection (CME) that resulted from the filament eruption was associated with a moderate disturbance at Earth. This event could be classified as a so-called "problem storm" because it lacked the usual solar signatures that are characteristic of large, energetic, Earth-directed CMEs that often result in significant geoeffective impacts. We use solar observations to constrain the initial parameters and therefore to model the propagation of the 2015 July 9 eruption from the solar corona up to Earth using 3D magnetohydrodynamic heliospheric simulations with three different configurations of the modeled CME. We find the best match between observed and modeled arrival at Earth for the simulation run that features a toroidal flux rope structure of the CME ejecta, but caution that different approaches may be more or less useful depending on the CME-observer geometry when evaluating the space weather impact of eruptions that are extreme in terms of their large size and high degree of asymmetry. We discuss our results in the context of both advancing our understanding of the physics of CME evolution and future improvements to space weather forecasting.


title: Making Users Indistinguishable: Attribute-wise Unlearning in Recommender Systems

abstract: With the growing privacy concerns in recommender systems, recommendation unlearning, i.e., forgetting the impact of specific learned targets, is getting increasing attention. Existing studies predominantly use training data, i.e., model inputs, as the unlearning target. However, we find that attackers can extract private information, i.e., gender, race, and age, from a trained model even if it has not been explicitly encountered during training. We name this unseen information as attribute and treat it as the unlearning target. To protect the sensitive attribute of users, Attribute Unlearning (AU) aims to degrade attacking performance and make target attributes indistinguishable. In this paper, we focus on a strict but practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the training of the recommendation model is completed. To address the PoT-AU problem in recommender systems, we design a two-component loss function that consists of i) distinguishability loss: making attribute labels indistinguishable from attackers, and ii) regularization loss: preventing drastic changes in the model that result in a negative impact on recommendation performance. Specifically, we investigate two types of distinguishability measurements, i.e., user-to-user and distribution-to-distribution. We use the stochastic gradient descent algorithm to optimize our proposed loss. Extensive experiments on three real-world datasets demonstrate the effectiveness of our proposed methods.


title: FMM-Head: Enhancing Autoencoder-based ECG anomaly detection with prior knowledge

abstract: Detecting anomalies in electrocardiogram data is crucial to identifying deviations from normal heartbeat patterns and providing timely intervention to at-risk patients. Various AutoEncoder models (AE) have been proposed to tackle the anomaly detection task with ML. However, these models do not consider the specific patterns of ECG leads and are unexplainable black boxes. In contrast, we replace the decoding part of the AE with a reconstruction head (namely, FMM-Head) based on prior knowledge of the ECG shape. Our model consistently achieves higher anomaly detection capabilities than state-of-the-art models, up to 0.31 increase in area under the ROC curve (AUROC), with as little as half the original model size and explainable extracted features. The processing time of our model is four orders of magnitude lower than solving an optimization problem to obtain the same parameters, thus making it suitable for real-time ECG parameters extraction and anomaly detection.


title: Analytical Solution of the Degeneracy Pressure of Two-Dimensional Quantum Gas

abstract: The exact equations of state of two-dimensional Bose gas (Equation 14) and Fermi gas (Equation 24) are derived, to calculate the degeneracy pressure. They are consistent with the second law of thermodynamics in the entire phase space. Under relevant conditions, they converge to the conventional equations.


title: Modified rational six vertex model on the rectangular lattice

abstract: We consider a rational six vertex model on a rectangular lattice with boundary conditions that generalize the usual domain wall type. We find that the partition function of the inhomogeneous version of this model is given by a modified Izergin determinant. The proofs are based on the quantum inverse scattering method and its representation theory together with elementary linear algebra.

categories: math-ph cond-mat.stat-mech math.MP quant-ph


title: Qibosoq: an open-source framework for quantum circuit RFSoC programming

abstract: We present Qibosoq, an open-source server-side software package designed for radio frequency system on chip (RFSoC) for executing arbitrary pulse sequences on self-hosted quantum processing units. Qibosoq bridges the RFSoC firmware provided by Qick, a Quantum Instrumentation Control Kit, with Qibo, a quantum computing middleware framework. It enables experimentalists and developers to delegate all complex aspects of client-server communication protocols to the library, implementing tests and validation protocols. The client-side integration is achieved with dedicated drivers implemented in Qibolab, the specialized software module of Qibo for quantum hardware control. Therefore, this setup provides a seamless mechanism to deploy circuit-based algorithms on custom self-hosted quantum hardware platforms controlled by RFSoC electronics. We first describe the status of all components of the software package, then we show examples of control setup for superconducting qubits platforms. Finally, we present successful application results related to RFSoC performance and circuit-based algorithms.


title: Higher-Order Platonism and Multiversism

abstract: Joel Hamkins has described his multiverse position as being one of `higher-order realism -- Platonism about universes', whereby one takes models of set theory to be actually existing objects (vis-\`a-vis `first-order realism', which takes only sets to be actually existing objects). My goal in this paper is to make sense of the view in the very context of Hamkins' own multiversism. To this end, I will explain what may be considered the central features of higher-order platonism, and then will focus on Zalta and Linsky's Object Theory, which, I will argue, is able to faithfully express Hamkins' conception. I will then show how the embedding of higher-order platonism into Object Theory may help the Hamkinsian multiversist to respond to salient criticisms of the multiverse conception, especially those relating to its articulation, skeptical attitude, and relationship with set-theoretic practice.


title: "Mango Mango, How to Let The Lettuce Dry Without A Spinner?'': Exploring User Perceptions of Using An LLM-Based Conversational Assistant Toward Cooking Partner

abstract: The rapid advancement of the Large Language Model (LLM) has created numerous potentials for integration with conversational assistants (CAs) assisting people in their daily tasks, particularly due to their extensive flexibility. However, users' real-world experiences interacting with these assistants remain unexplored. In this research, we chose cooking, a complex daily task, as a scenario to investigate people's successful and unsatisfactory experiences while receiving assistance from an LLM-based CA, Mango Mango. We discovered that participants value the system's ability to provide extensive information beyond the recipe, offer customized instructions based on context, and assist them in dynamically planning the task. However, they expect the system to be more adaptive to oral conversation and provide more suggestive responses to keep users actively involved. Recognizing that users began treating our LLM-CA as a personal assistant or even a partner rather than just a recipe-reading tool, we propose several design considerations for future development.


title: Time Crystal in a Single-mode Nonlinear Cavity

abstract: Time crystal is a class of non-equilibrium phases with broken time-translational symmetry. Here we demonstrate the time crystal in a single-mode nonlinear cavity. The time crystal originates from the self-oscillation induced by a linear gain and is stabilized by a nonlinear damping. We show in the time crystal phase there are sharp dissipative gap closing and pure imaginary eigenvalues of the Liouvillian spectrum in the thermodynamic limit. Dynamically, we observe a metastable regime with the emergence of quantum oscillation, followed by a dissipative evolution with a time scale much smaller than the oscillating period. Moreover, we show there is a dissipative phase transition at the Hopf bifurcation of the model, which can be characterized by the photon number fluctuation in the steady state. These results pave a new promising way for further experiments and deepen our understanding of time crystals.


title: Validation of a strongly polynomial-time algorithm for the general linear programming problem

abstract: This article presents a validation of a recently proposed strongly polynomial-time algorithm for the general linear programming problem. The algorithm is an implicit, solution-space reduction procedure that works as follows. Primal and dual problems are combined into a special system of linear equations constrained by complementarity relations and non-negative variables. Each iteration of the algorithm consists of applying two complementary Gauss-Jordan pivoting operations, guided by a reduction lemma. The algorithm requires no more than m+n iterations, where m is the number of constraints and n is the number of variables of the general linear programming problem. In another arXiv article, arXiv:2309.01037[math.DC], a numerical illustration of the algorithm is given that includes an instance of a classical problem of Klee and Minty and a problem of Beale.


title: A Reactive Force Field Approach to Modeling Corrosion of NiCr Alloys in Molten FLiNaK Salts

abstract: The interface between NiCr alloys and FLiNaK molten salt exhibits complex corrosion behavior, mainly driven by intricate chemical interactions involving Cr and F$\mathrm{^-}$ ions. Understanding these dynamic reactions is crucial for developing effective corrosion mitigation strategies to ensure the long-term durability of Ni-based alloy components in molten salt technologies. However, obtaining molecular-level understanding through experiments is challenging. To address this, we utilize reactive molecular dynamics simulations enabled by a reactive force field, ReaxFF, to investigate detailed reaction dynamics at the atomic level. Since there is currently no available force field involving fluoride salt and Ni-based alloys, we first present the development of the ReaxFF parameter set for Ni/Cr/F/Li/Na/K based on extensive first-principles calculations. With this force field, we achieve a strong agreement for the structure of FLiNaK molten salt by comparing the pair distribution functions with experimental and simulation results. Furthermore, it successfully reproduces the experimental phenomenon of Cr dissolution in fluoride salt, with the corrosion rate depending on the alloy and salt compositions. Particularly, it reveals that increasing the concentration of Li can enhance the formation of a compact double layer, mitigating Cr dissolution. This work enables a fundamental understanding of the interfacial behavior between fluoride salt and NiCr alloys.


title: Improving Summarization with Human Edits

abstract: Recent work has shown the promise of learning with human feedback paradigms to produce human-determined high-quality text. Existing works use human feedback to train large language models (LLMs) in general domain abstractive summarization and have obtained summary quality exceeding traditional likelihood training. In this paper, we focus on a less explored form of human feedback -- Human Edits. We propose Sequence Alignment (un)Likelihood Training (SALT), a novel technique to use both the human-edited and model-generated data together in the training loop. In addition, we demonstrate simulating Human Edits with ground truth summaries coming from existing training data -- Imitation edits, along with the model-generated summaries obtained after the training, to reduce the need for expensive human-edit data. In our experiments, we extend human feedback exploration from general domain summarization to medical domain summarization. Our results demonstrate the effectiveness of SALT in improving the summary quality with Human and Imitation Edits. Through additional experiments, we show that SALT outperforms the conventional RLHF method (designed for human preferences) -- DPO, when applied to human-edit data. We hope the evidence in our paper prompts researchers to explore, collect, and better use different human feedback approaches scalably.


title: DSAC-T: Distributional Soft Actor-Critic with Three Refinements

abstract: Reinforcement learning (RL) has proven to be highly effective in tackling complex decision-making and control tasks. However, prevalent model-free RL methods often face severe performance degradation due to the well-known overestimation issue. In response to this problem, we recently introduced an off-policy RL algorithm, called distributional soft actor-critic (DSAC or DSAC-v1), which can effectively improve the value estimation accuracy by learning a continuous Gaussian value distribution. Nonetheless, standard DSAC has its own shortcomings, including occasionally unstable learning processes and the necessity for task-specific reward scaling, which may hinder its overall performance and adaptability in some special tasks. This paper further introduces three important refinements to standard DSAC in order to address these shortcomings. These refinements consist of expected value substituting, twin value distribution learning, and variance-based critic gradient adjusting. The modified RL algorithm is named as DSAC with three refinements (DSAC-T or DSAC-v2), and its performances are systematically evaluated on a diverse set of benchmark tasks. Without any task-specific hyperparameter tuning, DSAC-T surpasses or matches a lot of mainstream model-free RL algorithms, including SAC, TD3, DDPG, TRPO, and PPO, in all tested environments. Additionally, DSAC-T, unlike its standard version, ensures a highly stable learning process and delivers similar performance across varying reward scales.


title: Choices of HKR isomorphisms

abstract: In this short note we record the fact that the set of multiplicative HKR natural equivalences defined simultaneously for all derived schemes, functorialy splitting the HKR-filtration and rendering the circle action compatible with the de Rham differential, is, via Cartier duality, in a natural bijection with the set of filtered formal exponential maps $ \widehat{\mathbb{G}_a}\to \widehat{\mathbb{G}_m}$. In particular, when the base $k$ is a field of characteristic zero, the set of choices is $k^\ast$.


title: Optical features of rotating quintessential charged black holes in de-Sitter spacetime

abstract: One of the most important and actual issues in relativistic astrophysics is testing gravity theories and obtaining constraint values for the parameters of black holes using observational data. In this research, we aimed to explore the optical features of a Kerr--Newman black hole model in the presence of a quintessential field, which may be a candidate for a dark-energy model with a nonzero cosmological constant. First, we obtain the equations of motion for photons using the Hamilton--Jacobi formalism. We also study the horizons and shapes of the apparent regions of the photon region around the said black hole. In various scenarios, we investigate shadows cast by the black hole using celestial coordinates. Furthermore, we analyze the effects of the quintessential field and black hole charge on the shadow radius and distortion. Furthermore, we look into the constraints on the spin and charge of supermassive black holes M87$^*$ and Sagittarius A$^*$ for different values of the quintessential field using their shadow size measured by the Event Horizon Telescope Collaboration. Finally, we study the effects of the quintessential field, black hole spin and charge on its energy emission rate by Hawking radiation and compare our results with those of the available literature.


title: Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models

abstract: An increasing number of vision-language tasks can be handled with little to no training, i.e., in a zero and few-shot manner, by marrying large language models (LLMs) to vision encoders, resulting in large vision-language models (LVLMs). While this has huge upsides, such as not requiring training data or custom architectures, how an input is presented to an LVLM can have a major impact on zero-shot model performance. In particular, inputs phrased in an underspecified way can result in incorrect answers due to factors like missing visual information, complex implicit reasoning, or linguistic ambiguity. Therefore, adding visually-grounded information to the input as a preemptive clarification should improve model performance by reducing underspecification, e.g., by localizing objects and disambiguating references. Similarly, in the VQA setting, changing the way questions are framed can make them easier for models to answer. To this end, we present Rephrase, Augment and Reason (RepARe), a gradient-free framework that extracts salient details about the image using the underlying LVLM as a captioner and reasoner, in order to propose modifications to the original question. We then use the LVLM's confidence over a generated answer as an unsupervised scoring function to select the rephrased question most likely to improve zero-shot performance. Focusing on three visual question answering tasks, we show that RepARe can result in a 3.85% (absolute) increase in zero-shot accuracy on VQAv2, 6.41%, and 7.94% points increase on A-OKVQA, and VizWiz respectively. Additionally, we find that using gold answers for oracle question candidate selection achieves a substantial gain in VQA accuracy by up to 14.41%. Through extensive analysis, we demonstrate that outputs from RepARe increase syntactic complexity, and effectively utilize vision-language interaction and the frozen LLM.


title: Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning and Backdoor Attacks

abstract: Contrastive Language-Image Pre-training (CLIP) on large image-caption datasets has achieved remarkable success in zero-shot classification and enabled transferability to new domains. However, CLIP is extremely more vulnerable to targeted data poisoning and backdoor attacks, compared to supervised learning. Perhaps surprisingly, poisoning 0.0001% of CLIP pre-training data is enough to make targeted data poisoning attacks successful. This is four orders of magnitude smaller than what is required to poison supervised models. Despite this vulnerability, existing methods are very limited in defending CLIP models during pre-training. In this work, we propose a strong defense, SAFECLIP, to safely pre-train CLIP against targeted data poisoning and backdoor attacks. SAFECLIP warms up the model by applying unimodal contrastive learning (CL) on image and text modalities separately. Then, it divides the data into safe and risky sets, by applying a Gaussian Mixture Model to the cosine similarity of image-caption pair representations. SAFECLIP pre-trains the model by applying the CLIP loss to the safe set and applying unimodal CL to image and text modalities of the risky set separately. By gradually increasing the size of the safe set during pre-training, SAFECLIP effectively breaks targeted data poisoning and backdoor attacks without harming the CLIP performance. Our extensive experiments on CC3M, Visual Genome, and MSCOCO demonstrate that SAFECLIP significantly reduces the success rate of targeted data poisoning attacks from 93.75% to 0% and that of various backdoor attacks from up to 100% to 0%, without harming CLIP's performance.


title: Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models

abstract: Audio-visual large language models (LLM) have drawn significant attention, yet the fine-grained combination of both input streams is rather under-explored, which is challenging but necessary for LLMs to understand general video inputs. To this end, a fine-grained audio-visual joint representation (FAVOR) learning framework for multimodal LLMs is proposed in this paper, which extends a text-based LLM to simultaneously perceive speech and audio events in the audio input stream and images or videos in the visual input stream, at the frame level. To fuse the audio and visual feature streams into joint representations and to align the joint space with the LLM input embedding space, we propose a causal Q-Former structure with a causal attention module to enhance the capture of causal relations of the audio-visual frames across time. An audio-visual evaluation benchmark (AVEB) is also proposed which comprises six representative single-modal tasks with five cross-modal tasks reflecting audio-visual co-reasoning abilities. While achieving competitive single-modal performance on audio, speech and image tasks in AVEB, FAVOR achieved over 20% accuracy improvements on the video question-answering task when fine-grained information or temporal causal reasoning is required. FAVOR, in addition, demonstrated remarkable video comprehension and reasoning abilities on tasks that are unprecedented by other multimodal LLMs. An interactive demo of FAVOR is available at https://github.com/BriansIDP/AudioVisualLLM.git, and the training code and model checkpoints will be released soon.


title: Helium identification with LHCb

abstract: The identification of helium nuclei at LHCb is achieved using a method based on measurements of ionisation losses in the silicon sensors and timing measurements in the Outer Tracker drift tubes. The background from photon conversions is reduced using the RICH detectors and an isolation requirement. The method is developed using $pp$ collision data at $\sqrt{s}=13\,{\rm TeV}$ recorded by the LHCb experiment in the years 2016 to 2018, corresponding to an integrated luminosity of $5.5\,{\rm fb}^{-1}$. A total of around $10^5$ helium and antihelium candidates are identified with negligible background contamination. The helium identification efficiency is estimated to be approximately $50\%$ with a corresponding background rejection rate of up to $\mathcal O(10^{12})$. These results demonstrate the feasibility of a rich programme of measurements of QCD and astrophysics interest involving light nuclei.


title: A Learning-Based Framework for Safe Human-Robot Collaboration with Multiple Backup Control Barrier Functions

abstract: Ensuring robot safety in complex environments is a difficult task due to actuation limits, such as torque bounds. This paper presents a safety-critical control framework that leverages learning-based switching between multiple backup controllers to formally guarantee safety under bounded control inputs while satisfying driver intention. By leveraging backup controllers designed to uphold safety and input constraints, backup control barrier functions (BCBFs) construct implicitly defined control invariance sets via a feasible quadratic program (QP). However, BCBF performance largely depends on the design and conservativeness of the chosen backup controller, especially in our setting of human-driven vehicles in complex, e.g, off-road, conditions. While conservativeness can be reduced by using multiple backup controllers, determining when to switch is an open problem. Consequently, we develop a broadcast scheme that estimates driver intention and integrates BCBFs with multiple backup strategies for human-robot interaction. An LSTM classifier uses data inputs from the robot, human, and safety algorithms to continually choose a backup controller in real-time. We demonstrate our method's efficacy on a dual-track robot in obstacle avoidance scenarios. Our framework guarantees robot safety while adhering to driver intention.


title: Generative quantum machine learning via denoising diffusion probabilistic models

abstract: Deep generative models are key-enabling technology to computer vision, text generation, and large language models. Denoising diffusion probabilistic models (DDPMs) have recently gained much attention due to their ability to generate diverse and high-quality samples in many computer vision tasks, as well as to incorporate flexible model architectures and a relatively simple training scheme. Quantum generative models, empowered by entanglement and superposition, have brought new insight to learning classical and quantum data. Inspired by the classical counterpart, we propose the quantum denoising diffusion probabilistic model (QuDDPM) to enable efficiently trainable generative learning of quantum data. QuDDPM adopts sufficient layers of circuits to guarantee expressivity, while it introduces multiple intermediate training tasks as interpolation between the target distribution and noise to avoid barren plateau and guarantee efficient training. We provide bounds on the learning error and demonstrate QuDDPM's capability in learning correlated quantum noise model, quantum many-body phases, and topological structure of quantum data. The results provide a paradigm for versatile and efficient quantum generative learning.


title: Domain-wise Invariant Learning for Panoptic Scene Graph Generation

abstract: Panoptic Scene Graph Generation (PSG) involves the detection of objects and the prediction of their corresponding relationships (predicates). However, the presence of biased predicate annotations poses a significant challenge for PSG models, as it hinders their ability to establish a clear decision boundary among different predicates. This issue substantially impedes the practical utility and real-world applicability of PSG models. To address the intrinsic bias above, we propose a novel framework to infer potentially biased annotations by measuring the predicate prediction risks within each subject-object pair (domain), and adaptively transfer the biased annotations to consistent ones by learning invariant predicate representation embeddings. Experiments show that our method significantly improves the performance of benchmark models, achieving a new state-of-the-art performance, and shows great generalization and effectiveness on PSG dataset.


title: Bio-inspired computational memory model of the Hippocampus: an approach to a neuromorphic spike-based Content-Addressable Memory

abstract: The brain has computational capabilities that surpass those of modern systems, being able to solve complex problems efficiently in a simple way. Neuromorphic engineering aims to mimic biology in order to develop new systems capable of incorporating such capabilities. Bio-inspired learning systems continue to be a challenge that must be solved, and much work needs to be done in this regard. Among all brain regions, the hippocampus stands out as an autoassociative short-term memory with the capacity to learn and recall memories from any fragment of them. These characteristics make the hippocampus an ideal candidate for developing bio-inspired learning systems that, in addition, resemble content-addressable memories. Therefore, in this work we propose a bio-inspired spiking content-addressable memory model based on the CA3 region of the hippocampus with the ability to learn, forget and recall memories, both orthogonal and non-orthogonal, from any fragment of them. The model was implemented on the SpiNNaker hardware platform using Spiking Neural Networks. A set of experiments based on functional, stress and applicability tests were performed to demonstrate its correct functioning. This work presents the first hardware implementation of a fully-functional bio-inspired spiking hippocampal content-addressable memory model, paving the way for the development of future more complex neuromorphic systems.


title: HyperAttention: Long-context Attention in Near-Linear Time

abstract: We present an approximate attention mechanism named HyperAttention to address the computational challenges posed by the growing complexity of long contexts used in Large Language Models (LLMs). Recent work suggests that in the worst-case scenario, quadratic time is necessary unless the entries of the attention matrix are bounded or the matrix has low stable rank. We introduce two parameters which measure: (1) the max column norm in the normalized attention matrix, and (2) the ratio of row norms in the unnormalized attention matrix after detecting and removing large entries. We use these fine-grained parameters to capture the hardness of the problem. Despite previous lower bounds, we are able to achieve a linear time sampling algorithm even when the matrix has unbounded entries or a large stable rank, provided the above parameters are small. HyperAttention features a modular design that easily accommodates integration of other fast low-level implementations, particularly FlashAttention. Empirically, employing Locality Sensitive Hashing (LSH) to identify large entries, HyperAttention outperforms existing methods, giving significant speed improvements compared to state-of-the-art solutions like FlashAttention. We validate the empirical performance of HyperAttention on a variety of different long-context length datasets. For example, HyperAttention makes the inference time of ChatGLM2 50\% faster on 32k context length while perplexity increases from 5.6 to 6.3. On larger context length, e.g., 131k, with causal masking, HyperAttention offers 5-fold speedup on a single attention layer.


title: Detecting Multipartite Entanglement Patterns using Single Particle Green's Functions

abstract: We propose a protocol for detecting multipartite entanglement in itinerant many-body electronic systems using the quantum Fisher information (QFI). We establish a connection between the QFI and single-particle Green's functions by identifying a set of non-trivial witness operators. To construct these operators, we employ a doubling of the system wherein we introduce two identical copies of the original model. While the witness operator hops electrons between copies, the copies do not interact with one another. We apply this methodology to a finite-sized fermionic system and showcase its effectiveness in detecting entanglement in spinless itinerant electron models. We show that the detected entanglement level is sensitive to the wave vector associated with the hopping process. We also demonstrate the important role that symmetry has in detecting levels of entanglement. Our protocol paves the way for detecting entanglement in many-body systems using scanning tunneling microscopy and angle-resolved photoemission spectroscopy, thus offering exciting prospects beyond the detection of entanglement via the dynamical spin response accessed in neutron scattering experiments.


title: Dynamic value alignment through preference aggregation of multiple objectives

abstract: The development of ethical AI systems is currently geared toward setting objective functions that align with human objectives. However, finding such functions remains a research challenge, while in RL, setting rewards by hand is a fairly standard approach. We present a methodology for dynamic value alignment, where the values that are to be aligned with are dynamically changing, using a multiple-objective approach. We apply this approach to extend Deep $Q$-Learning to accommodate multiple objectives and evaluate this method on a simplified two-leg intersection controlled by a switching agent.Our approach dynamically accommodates the preferences of drivers on the system and achieves better overall performance across three metrics (speeds, stops, and waits) while integrating objectives that have competing or conflicting actions.


title: ViCor: Bridging Visual Understanding and Commonsense Reasoning with Large Language Models

abstract: In our work, we explore the synergistic capabilities of pre-trained vision-and-language models (VLMs) and large language models (LLMs) on visual commonsense reasoning (VCR) problems. We find that VLMs and LLMs-based decision pipelines are good at different kinds of VCR problems. Pre-trained VLMs exhibit strong performance for problems involving understanding the literal visual content, which we noted as visual commonsense understanding (VCU). For problems where the goal is to infer conclusions beyond image content, which we noted as visual commonsense inference (VCI), VLMs face difficulties, while LLMs, given sufficient visual evidence, can use commonsense to infer the answer well. We empirically validate this by letting LLMs classify VCR problems into these two categories and show the significant difference between VLM and LLM with image caption decision pipelines on two subproblems. Moreover, we identify a challenge with VLMs' passive perception, which may miss crucial context information, leading to incorrect reasoning by LLMs. Based on these, we suggest a collaborative approach, named ViCor, where pre-trained LLMs serve as problem classifiers to analyze the problem category, then either use VLMs to answer the question directly or actively instruct VLMs to concentrate on and gather relevant visual elements to support potential commonsense inferences. We evaluate our framework on two VCR benchmark datasets and outperform all other methods that do not require in-domain fine-tuning.


title: Implicit Concept Removal of Diffusion Models

abstract: Text-to-image (T2I) diffusion models often inadvertently generate unwanted concepts such as watermarks and unsafe images. These concepts, termed as the "implicit concepts", could be unintentionally learned during training and then be generated uncontrollably during inference. Existing removal methods still struggle to eliminate implicit concepts primarily due to their dependency on the model's ability to recognize concepts it actually can not discern. To address this, we utilize the intrinsic geometric characteristics of implicit concepts and present the Geom-Erasing, a novel concept removal method based on geometric-driven control. Specifically, once an unwanted implicit concept is identified, we integrate the existence and geometric information of the concept into text prompts with the help of an accessible classifier or detector model. Subsequently, the model is optimized to identify and disentangle this information, which is adopted as negative prompts for generation. Moreover, we introduce Implicit Concept Dataset (ICD), a novel image-text dataset imbued with three typical implicit concepts (i.e., QR codes, watermarks, and text), reflecting real-life situations where implicit concepts are easily injected. Geom-Erasing effectively mitigates the generation of implicit concepts, achieving state-of-the-art results on the Inappropriate Image Prompts (I2P) and our challenging Implicit Concept Dataset (ICD) benchmarks.


title: Resolvent expansions of 3D magnetic Schroedinger operators and Pauli operators

abstract: We obtain asymptotic resolvent expansions at the threshold of the essential spectrum for magnetic Schr\"odinger and Pauli operators in dimension three. These operators are treated as perturbations of the Laplace operator in $L^2(\mathbb{R}^3)$ and $L^2(\mathbb{R}^3;\mathbb{C}^2)$, respectively. The main novelty of our approach is to show that the relative perturbations, which are first order differential operators, can be factorized in suitably chosen auxiliary spaces. This allows us to derive the desired asymptotic expansions of the resolvents around zero. We then calculate their leading and sub-leading terms explicitly. Analogous factorization schemes for more general perturbations, including e.g.~finite rank perturbations, are discussed as well.


title: Host-feeding preferences and temperature shape the dynamics of West Nile virus: a mathematical model of assessing the abatement planning

abstract: West Nile virus (WNV) is prevalent in the United States but it shows considerable divergence in transmission patterns and spatio-temporal intensity.It is to be noted that the mechanism that drives the transmission potential of WNV is described by the abilities of host species to maintain and disseminate the pathogens pertinent with different eco-epidemiological factors that have an influence on the contact rates amongst the interacting species.There is growing evidence that several vectors exhibit strong feeding preferences towards different host communities.We construct a process based weather driven ordinary differential equation (ODE) model to understand the impact of one vector species Culex pipiens, preferred avian and non-preferred human hosts and compared it surveillance data for the Culex pipiens complex collected in Cook County, Illinois, USA.In our mechanistic model, we also demonstrate that adulticide treatments produced significant reductions in the Culex pipiens population.We take into account the feeding index that can be described as the ratio between observed frequency of mosquitoes feeding on one host compared to another host, divided by the expected frequency of mosquitoes feeding on these two hosts based on the presence of the particular hosts to develop this transmission model for WNV. Our findings demonstrate that the interplay between the feeding index and mosquito abatement strategy is rather a complex phenomenon and it induces a heterogeneous contact rates that should be included while modelling multi-host, multi-vector transmission model.


title: AI Systems of Concern

abstract: Concerns around future dangers from advanced AI often centre on systems hypothesised to have intrinsic characteristics such as agent-like behaviour, strategic awareness, and long-range planning. We label this cluster of characteristics as "Property X". Most present AI systems are low in "Property X"; however, in the absence of deliberate steering, current research directions may rapidly lead to the emergence of highly capable AI systems that are also high in "Property X". We argue that "Property X" characteristics are intrinsically dangerous, and when combined with greater capabilities will result in AI systems for which safety and control is difficult to guarantee. Drawing on several scholars' alternative frameworks for possible AI research trajectories, we argue that most of the proposed benefits of advanced AI can be obtained by systems designed to minimise this property. We then propose indicators and governance interventions to identify and limit the development of systems with risky "Property X" characteristics.


title: The orbital period of the recurrent nova V2487 Oph revealed

abstract: We present the first reliable determination of the orbital period of the recurrent nova V2487 Oph (Nova Oph 1998). We derived a value of $0.753 \pm 0.016$ d ($18.1 \pm 0.4$ h) from the radial velocity curve of the intense He II $\lambda$4686 emission line as detected in time-series X-shooter spectra. The orbital period is significantly shorter than earlier claims, but it makes V2487 Oph one of the longest period cataclysmic variables known. The spectrum of V2487 Oph is prolific in broad Balmer absorptions that resemble a white dwarf spectrum. However, we show that they come from the accretion disc viewed at low inclination. Although highly speculative, the analysis of the radial velocity curves provides a binary mass ratio $q \approx 0.16$ and a donor star mass $M_2 \approx 0.21$ M$_\odot$, assuming the reported white dwarf mass $M_1 = 1.35$ M$_\odot$. A subgiant M-type star is tentatively suggested as the donor star. We were lucky to inadvertently take some of the spectra when V2487 Oph was in a flare state. During the flare, we detected high-velocity emission in the Balmer and He II $\lambda$4686 lines exceeding $-2000$ km s$^{-1}$ at close to orbital phase 0.4. Receding emission up to $1200$ km s$^{-1}$ at about phase 0.3 is also observed. The similarities with the magnetic cataclysmic variables may point to magnetic accretion on to the white dwarf during the repeating flares.


title: A Machine Learning Approach to Predicting Single Event Upsets

abstract: A single event upset (SEU) is a critical soft error that occurs in semiconductor devices on exposure to ionising particles from space environments. SEUs cause bit flips in the memory component of semiconductors. This creates a multitude of safety hazards as stored information becomes less reliable. Currently, SEUs are only detected several hours after their occurrence. CREMER, the model presented in this paper, predicts SEUs in advance using machine learning. CREMER uses only positional data to predict SEU occurrence, making it robust, inexpensive and scalable. Upon implementation, the improved reliability of memory devices will create a digitally safer environment onboard space vehicles.


title: Coarse-Graining Hamiltonian Systems Using WSINDy

abstract: The Weak-form Sparse Identification of Nonlinear Dynamics algorithm (WSINDy) has been demonstrated to offer coarse-graining capabilities in the context of interacting particle systems (https://doi.org/10.1016/j.physd.2022.133406). In this work we extend this capability to the problem of coarse-graining Hamiltonian dynamics which possess approximate symmetries associated with timescale separation. Such approximate symmetries often lead to the existence of a Hamiltonian system of reduced dimension that may be used to efficiently capture the dynamics of the symmetry-invariant dependent variables. Deriving such reduced systems, or approximating them numerically, is an ongoing challenge. We demonstrate that WSINDy can successfully identify this reduced Hamiltonian system in the presence of large intrinsic perturbations while remaining robust to extrinsic noise. This is significant in part due to the nontrivial means by which such systems are derived analytically. WSINDy also naturally preserves the Hamiltonian structure by restricting to a trial basis of Hamiltonian vector fields. The methodology is computational efficient, often requiring only a single trajectory to learn the global reduced Hamiltonian, and avoiding forward solves in the learning process. Using nearly-periodic Hamiltonian systems as a prototypical class of systems with approximate symmetries, we show that WSINDy robustly identifies the correct leading-order system, with dimension reduced by at least two, upon observation of the relevant degrees of freedom. We also provide a contribution to averaging theory by proving that first-order averaging at the level of vector fields preserves Hamiltonian structure in nearly-periodic Hamiltonian systems. We provide physically relevant examples, namely coupled oscillator dynamics, the H\'enon-Heiles system for stellar motion within a galaxy, and the dynamics of charged particles.

categories: physics.comp-ph cs.AI cs.NA math.DS math.NA stat.ML


title: The Hitchin Image in Type-D

abstract: Motivated by their appearance as Coulomb branch geometries of Class S theories, we study the image of the local Hitchin map in tame Hitchin systems of type-D with residue in a special nilpotent orbit $\mathcal{O}_H$. We describe two important features which distinguish it from the type A case studied in arXiv:2008.01020. The first feature, which we term even type constraints, arise iff the partition label $[\mathcal{O}_H]$ has even parts. In this case, our Hitchin image is non-singular and thus different from the one studied by Baraglia and Kamgarpour. We argue that our Hitchin image always globalizes to being the Hitchin base of an integrable system. The second feature, which we term odd type constraints, is related to a particular finite group $\overline{A}_b(\mathcal{O}_H)$ being non-trivial. When this finite group is non-trivial, we have $\mid \overline{A}_b \mid$ choices for the local Hitchin base. Additionally, we also show that the finite group $\overline{A}_b(\mathcal{O}_H)$ encodes the size of the dual special piece.


title: Controllable Chest X-Ray Report Generation from Longitudinal Representations

abstract: Radiology reports are detailed text descriptions of the content of medical scans. Each report describes the presence/absence and location of relevant clinical findings, commonly including comparison with prior exams of the same patient to describe how they evolved. Radiology reporting is a time-consuming process, and scan results are often subject to delays. One strategy to speed up reporting is to integrate automated reporting systems, however clinical deployment requires high accuracy and interpretability. Previous approaches to automated radiology reporting generally do not provide the prior study as input, precluding comparison which is required for clinical accuracy in some types of scans, and offer only unreliable methods of interpretability. Therefore, leveraging an existing visual input format of anatomical tokens, we introduce two novel aspects: (1) longitudinal representation learning -- we input the prior scan as an additional input, proposing a method to align, concatenate and fuse the current and prior visual information into a joint longitudinal representation which can be provided to the multimodal report generation model; (2) sentence-anatomy dropout -- a training strategy for controllability in which the report generator model is trained to predict only sentences from the original report which correspond to the subset of anatomical regions given as input. We show through in-depth experiments on the MIMIC-CXR dataset how the proposed approach achieves state-of-the-art results while enabling anatomy-wise controllable report generation.


title: Evaluating a VR System for Collecting Safety-Critical Vehicle-Pedestrian Interactions

abstract: Autonomous vehicles (AVs) require comprehensive and reliable pedestrian trajectory data to ensure safe operation. However, obtaining data of safety-critical scenarios such as jaywalking and near-collisions, or uncommon agents such as children, disabled pedestrians, and vulnerable road users poses logistical and ethical challenges. This paper evaluates a Virtual Reality (VR) system designed to collect pedestrian trajectory and body pose data in a controlled, low-risk environment. We substantiate the usefulness of such a system through semi-structured interviews with professionals in the AV field, and validate the effectiveness of the system through two empirical studies: a first-person user evaluation involving 62 participants, and a third-person evaluative survey involving 290 respondents. Our findings demonstrate that the VR-based data collection system elicits realistic responses for capturing pedestrian data in safety-critical or uncommon vehicle-pedestrian interaction scenarios.


title: Generation of meter-scale hydrogen plasmas and efficient, pump-depletion-limited wakefield excitation using 10 GeV electron bunches

abstract: High repetition rates and efficient energy transfer to the accelerating beam are important for a future linear collider based on the beam-driven plasma wakefield acceleration scheme (PWFA-LC). This paper reports the first results from the Plasma Wakefield Acceleration Collaboration (E300) that are beginning to address both of these issues using the recently commissioned FACET-II facility at SLAC. We have generated meter-scale hydrogen plasmas using time-structured 10 GeV electron bunches from FACET-II, which hold the promise of dramatically increasing the repetition rate of PWFA by rapidly replenishing the gas between each shot compared to the hitherto used lithium plasmas that operate at 1-10 Hz. Furthermore, we have excited wakes in such plasmas that are suitable for high gradient particle acceleration with high drive-bunch to wake energy transfer efficiency -- a first step in achieving a high overall energy transfer efficiency. We have done this by using time-structured electron drive bunches that typically have one or more ultra-high current (>30 kA) femtosecond spike(s) superimposed on a longer (~0.4 ps) lower current (<10 kA) bunch structure. The first spike effectively field-ionizes the gas and produces a meter-scale (30-160 cm) plasma, whereas the subsequent beam charge creates a wake. The length and amplitude of the wake depends on the longitudinal current profile of the bunch and plasma density. We find that the onset of pump depletion, when some of the drive beam electrons are nearly fully depleted of their energy, occurs for hydrogen pressure >1.5 Torr. We also show that some electrons in the rear of the bunch can gain several GeV energies from the wake. These results are reproduced by particle-in-cell simulations using the QPAD code. At a pressure of ~2 Torr, simulations results and experimental data show that the beam transfers about 60% of its energy to the wake.


title: A Meta-Learning Perspective on Transformers for Causal Language Modeling

abstract: The Transformer architecture has become prominent in developing large causal language models. However, mechanisms to explain its capabilities are not well understood. Focused on the training process, here we establish a meta-learning view of the Transformer architecture when trained for the causal language modeling task, by explicating an inner optimization process within the Transformer. Further, within the inner optimization, we discover and theoretically analyze a special characteristic of the norms of learned token representations within Transformer-based causal language models. Our analysis is supported by experiments in various settings.


title: DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving

abstract: Motion prediction and cost evaluation are vital components in the decision-making system of autonomous vehicles. However, existing methods often ignore the importance of cost learning and treat them as separate modules. In this study, we employ a tree-structured policy planner and propose a differentiable joint training framework for both ego-conditioned prediction and cost models, resulting in a direct improvement of the final planning performance. For conditional prediction, we introduce a query-centric Transformer model that performs efficient ego-conditioned motion prediction. For planning cost, we propose a learnable context-aware cost function with latent interaction features, facilitating differentiable joint learning. We validate our proposed approach using the real-world nuPlan dataset and its associated planning test platform. Our framework not only matches state-of-the-art planning methods but outperforms other learning-based methods in planning quality, while operating more efficiently in terms of runtime. We show that joint training delivers significantly better performance than separate training of the two modules. Additionally, we find that tree-structured policy planning outperforms the conventional single-stage planning approach.


title: Streaming Anchor Loss: Augmenting Supervision with Temporal Significance

abstract: Streaming neural network models for fast frame-wise responses to various speech and sensory signals are widely adopted on resource-constrained platforms. Hence, increasing the learning capacity of such streaming models (i.e., by adding more parameters) to improve the predictive power may not be viable for real-world tasks. In this work, we propose a new loss, Streaming Anchor Loss (SAL), to better utilize the given learning capacity by encouraging the model to learn more from essential frames. More specifically, our SAL and its focal variations dynamically modulate the frame-wise cross entropy loss based on the importance of the corresponding frames so that a higher loss penalty is assigned for frames within the temporal proximity of semantically critical events. Therefore, our loss ensures that the model training focuses on predicting the relatively rare but task-relevant frames. Experimental results with standard lightweight convolutional and recurrent streaming networks on three different speech based detection tasks demonstrate that SAL enables the model to learn the overall task more effectively with improved accuracy and latency, without any additional data, model parameters, or architectural changes.


title: The Sun's Alfven Surface: Recent Insights and Prospects for the Polarimeter to Unify the Corona and Heliosphere (PUNCH)

abstract: The solar wind is the extension of the Sun's hot and ionized corona, and it exists in a state of continuous expansion into interplanetary space. The radial distance at which the wind's outflow speed exceeds the phase speed of Alfvenic and fast-mode magnetohydrodynamic (MHD) waves is called the Alfven radius. In one-dimensional models, this is a singular point beyond which most fluctuations in the plasma and magnetic field cannot propagate back down to the Sun. In the multi-dimensional solar wind, this point can occur at different distances along an irregularly shaped "Alfven surface." In this article, we review the properties of this surface and discuss its importance in models of solar-wind acceleration, angular-momentum transport, MHD waves and turbulence, and the geometry of magnetically closed coronal loops. We also review the results of simulations and data analysis techniques that aim to determine the location of the Alfven surface. Combined with recent perihelia of Parker Solar Probe, these studies seem to indicate that the Alfven surface spends most of its time at heliocentric distances between about 10 and 20 solar radii. It is becoming apparent that this region of the heliosphere is sufficiently turbulent that there often exist multiple (stochastic and time-dependent) crossings of the Alfven surface along any radial ray. Thus, in many contexts, it is more useful to make use of the concept of a topologically complex "Alfven zone" rather than one closed surface. This article also reviews how the Polarimeter to Unify the Corona and Heliosphere (PUNCH) mission will measure the properties of the Alfven surface and provide key constraints on theories of solar-wind acceleration.


title: On the stability of the dnoidal waves for the Schr\"odinger-KdV system

abstract: We study the periodic Schr\"odinger-Korteweg de Vries system. We describe the two-parametetric family of $2T$ periodic traveling waves of dnoidal type. The main objective of the paper is to establish their spectral stability with respect to co-periodic perturbations. In the limit $T\to \infty$, we recover the results of Albert-Angulo, for the stability of the soliton solutions of this system on the real line.


title: Superadditive Communications with the Green Machine: A Practical Demonstration of Nonlocality without Entanglement

abstract: Achieving the ultimate Holevo limit of optical communications capacity requires a joint-detection receiver: a device that makes a collective quantum measurement over multiple modulated symbols. Such superadditivity -- a higher communication rate than that achievable by any physically realizable symbol-by-symbol optical detection -- is a special case of the celebrated nonlocality without entanglement and has yet to be demonstrated in practice. In this article, we propose a practical design of the Green Machine -- a joint-detection receiver that can attain superadditive capacity with a binary-phase-shift-keying (BPSK) modulated Hadamard code. We build this receiver and show that its capacity surpasses that of all practical symbol-by-symbol receivers in the low-received-photon-flux regime after backing out losses within our receiver. Our Green Machine receiver not only reduces the transmitter peak power requirement compared with the pulse-position modulation (the conventional modulation format used for deep space laser communications), but we show that its self-referenced phase also makes it more immune to phase noise, e.g., atmospheric turbulence or platform vibrations, by orders of magnitude compared with other BPSK-compatible receivers.


title: Solaris photometric survey: Search for circumbinary companions using eclipse timing variations

abstract: Eclipse timing variations (ETV) have been a successful tool for detecting circumbinary companions to eclipsing binaries (EB). While TESS and Kepler have been prolific for ETV searches, they sometimes can be limited by time and sky coverage which can be addressed by specialised ground-based ETV surveys. We present the initial results from the Solaris photometric survey which uses four 0.5m robotic telescopes in the southern hemisphere to look for circumbinary companions. We present the method of light curve extraction, detrending, and EB modelling using the observations from the Solaris network. Using these light curves we extract precise eclipse timing for 7 EB and look for companions using a Lomb-Scargle periodogram search. We find two possible periodic signals for the target GSC 08814-01026. With the system having strong activity, we check for the feasibility of orbital solutions at these two periods. We find that the 245 +/- 1 d period is due to an M-dwarf mass companion. This makes GSC 08814-01026 a candidate compact hierarchical triple system. The other periodic signal at 146 +/- 1 d is an artefact of stellar activity.


title: Automated reasoning for proving non-orderability of groups

abstract: We demonstrate how a generic automated theorem prover can be applied to establish the non-orderability of groups. Our approach incorporates various tools such as positive cones, torsions, generalised torsions and cofinal elements.


title: A Generalization Bound of Deep Neural Networks for Dependent Data

abstract: Existing generalization bounds for deep neural networks require data to be independent and identically distributed (iid). This assumption may not hold in real-life applications such as evolutionary biology, infectious disease epidemiology, and stock price prediction. This work establishes a generalization bound of feed-forward neural networks for non-stationary $\phi$-mixing data.


title: One Benders cut to rule all schedules in the neighbourhood

abstract: Logic-Based Benders Decomposition (LBBD) and its Branch-and-Cut variant, namely Branch-and-Check, enjoy an extensive applicability on a broad variety of problems, including scheduling. Although LBBD offers problem-specific cuts to impose tighter dual bounds, its application to resource-constrained scheduling remains less explored. Given a position-based Mixed-Integer Linear Programming (MILP) formulation for scheduling on unrelated parallel machines, we notice that certain $k-$OPT neighbourhoods could implicitly be explored by regular local search operators, thus allowing us to integrate Local Branching into Branch-and-Check schemes. After enumerating such neighbourhoods and obtaining their local optima - hence, proving that they are suboptimal - a local branching cut (applied as a Benders cut) eliminates all their solutions at once, thus avoiding an overload of the master problem with thousands of Benders cuts. However, to guarantee convergence to optimality, the constructed neighbourhood should be exhaustively explored, hence this time-consuming procedure must be accelerated by domination rules or selectively implemented on nodes which are more likely to reduce the optimality gap. In this study, the realisation of this idea is limited on the common 'internal (job) swaps' to construct formulation-specific $4$-OPT neighbourhoods. Nonetheless, the experimentation on two challenging scheduling problems (i.e., the minimisation of total completion times and the minimisation of total tardiness on unrelated machines with sequence-dependent and resource-constrained setups) shows that the proposed methodology offers considerable reductions of optimality gaps or faster convergence to optimality. The simplicity of our approach allows its transferability to other neighbourhoods and different sequencing optimisation problems, hence providing a promising prospect to improve Branch-and-Check methods.


title: On the Value of Linear Quadratic Zero-sum Difference Games with Multiplicative Randomness: Existence and Achievability

abstract: We consider a wireless networked control system (WNCS) with multiple controllers and multiple attackers. The dynamic interaction between the controllers and the attackers is modeled as a linear quadratic (LQ) zero-sum difference game with multiplicative randomness induced by the multiple-input and multiple-output (MIMO) wireless fading channels of the controllers and attackers. We focus on analyzing the existence and achievability of the value of the zero-sum game. We first establish a general sufficient and necessary condition for the existence of the game value. This condition relies on the solvability of a modified game algebraic Riccati equation (MGARE) under an implicit concavity constraint, which is generally difficult to verify due to the intermittent controllability or almost sure uncontrollability caused by the multiplicative randomness. We then introduce a new positive semidefinite (PSD) kernel decomposition method induced by multiplicative randomness, through which we obtain a closed-form tight verifiable sufficient condition. Under the existence condition, we finally construct a saddle-point policy that is able to achieve the game value in a certain class of admissible policies. We demonstrate that the proposed saddle-point policy is backward compatible to the existing strictly feedback stabilizing saddle-point policy.


title: Spatial asymmetry of optically excited spin waves in anisotropic ferromagnetic film

abstract: We analytically discuss and micromagnetically prove the ways to tune the spatial asymmetry of the initial phase, amplitude, and wavevectors of magnetostatic waves driven by ultrafast laser excitation. We consider that the optical pulse heats a thin ferromagnetic metallic film and abruptly decreases the saturation magnetization and the parameter of uniaxial anisotropy. The two corresponding terms of laser-induced torque have different azimuthal symmetries, with the 4-fold symmetry of the demagnetization-related term, and the isotropic distribution of the anisotropy-related term. As a result, the initial phase and amplitude of excited magnetostatic waves have a non-trivial azimuthal distribution tunable with the angle between the external magnetic field and anisotropy axis, and the laser spot diameter. Moreover, the variation of these parameters tunes the distribution of wavevectors, resulting in additional asymmetry between the spectral components of the waves propagating in different directions.


title: Controlling Topology through Targeted Symmetry Manipulation in Magnetic Systems

abstract: The possibility of selecting magnetic space groups by orienting the magnetization direction or tuning magnetic orders offers a vast playground for engineering symmetry protected topological phases in magnetic materials. In this work, we study how selective tuning of symmetry and magnetism can influence and control the resulting topology in a 2D magnetic system, and illustrate such procedure in the ferromagnetic monolayer MnPSe$_3$. Density functional theory calculations reveals a symmetry-protected accidental semimetalic (SM) phase for out-of-plane magnetization which becomes an insulator when the magnetization is tilted in-plane, reaching band gap values close to $100$ meV. We identify an order-two composite antiunitary symmetry and threefold rotational symmetry that induce the band crossing and classify the possible topological phases using symmetry analysis, which we support with tight-binding and $\mathbf{k}\cdot\mathbf{p}$ models. Breaking of inversion symmetry opens a gap in the SM phase, giving rise to a Chern insulator. We demonstrate this explicitly in the isostructural Janus compound Mn$_2$P$_2$S$_3$Se$_3$, which naturally exhibits Rashba spin-orbit coupling that breaks inversion symmetry. Our results map out the phase space of topological properties of ferromagnetic transition metal phosphorus trichalcogenides and demonstrate the potential of the magnetization-dependent metal-to-insulator transition as a spin switch in integrated two-dimensional electronics.


title: Tensor Network Based Efficient Quantum Data Loading of Images

abstract: Image-based data is a popular arena for testing quantum machine learning algorithms. A crucial factor in realizing quantum advantage for these applications is the ability to efficiently represent images as quantum states. Here we present a novel method for creating quantum states that approximately encode images as amplitudes, based on recently proposed techniques that convert matrix product states to quantum circuits. The numbers of gates and qubits in our method scale logarithmically in the number of pixels given a desired accuracy, which make it suitable for near term quantum computers. Finally, we experimentally demonstrate our technique on 8 qubits of a trapped ion quantum computer for complex images of road scenes, making this the first large instance of full amplitude encoding of an image in a quantum state.


title: Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts

abstract: Lion (Evolved Sign Momentum), a new optimizer discovered through program search, has shown promising results in training large AI models. It performs comparably or favorably to AdamW but with greater memory efficiency. As we can expect from the results of a random search program, Lion incorporates elements from several existing algorithms, including signed momentum, decoupled weight decay, Polak, and Nesterov momentum, but does not fit into any existing category of theoretically grounded optimizers. Thus, even though Lion appears to perform well as a general-purpose optimizer for a wide range of tasks, its theoretical basis remains uncertain. This lack of theoretical clarity limits opportunities to further enhance and expand Lion's efficacy. This work aims to demystify Lion. Based on both continuous-time and discrete-time analysis, we demonstrate that Lion is a theoretically novel and principled approach for minimizing a general loss function $f(x)$ while enforcing a bound constraint $\|x\|_\infty \leq 1/\lambda$. Lion achieves this through the incorporation of decoupled weight decay, where $\lambda$ represents the weight decay coefficient. Our analysis is made possible by the development of a new Lyapunov function for the Lion updates. It applies to a broader family of Lion-$\kappa$ algorithms, where the $\text{sign}(\cdot)$ operator in Lion is replaced by the subgradient of a convex function $\kappa$, leading to the solution of a general composite optimization problem of $\min_x f(x) + \kappa^*(x)$. Our findings provide valuable insights into the dynamics of Lion and pave the way for further improvements and extensions of Lion-related algorithms.


title: Enabling Intelligent Vehicular Networks Through Distributed Learning in the Non-Terrestrial Networks 6G Vision

abstract: The forthcoming 6G-enabled Intelligent Transportation System (ITS) is set to redefine conventional transportation networks with advanced intelligent services and applications. These technologies, including edge computing, Machine Learning (ML), and network softwarization, pose stringent requirements for latency, energy efficiency, and user data security. Distributed Learning (DL), such as Federated Learning (FL), is essential to meet these demands by distributing the learning process at the network edge. However, traditional FL approaches often require substantial resources for satisfactory learning performance. In contrast, Transfer Learning (TL) and Split Learning (SL) have shown effectiveness in enhancing learning efficiency in resource-constrained wireless scenarios like ITS. Non-terrestrial Networks (NTNs) have recently acquired a central place in the 6G vision, especially for boosting the coverage, capacity, and resilience of traditional terrestrial facilities. Air-based NTN layers, such as High Altitude Platforms (HAPs), can have added advantages in terms of reduced transmission distances and flexible deployments and thus can be exploited to enable intelligent solutions for latency-critical vehicular scenarios. With this motivation, in this work, we introduce the concept of Federated Split Transfer Learning (FSTL) in joint air-ground networks for resource-constrained vehicular scenarios. Simulations carried out in vehicular scenarios validate the efficacy of FSTL on HAPs in NTN, demonstrating significant improvements in addressing the demands of ITS applications.


title: Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network

abstract: Quantum error-correction is a prerequisite for reliable quantum computation. Towards this goal, we present a recurrent, transformer-based neural network which learns to decode the surface code, the leading quantum error-correction code. Our decoder outperforms state-of-the-art algorithmic decoders on real-world data from Google's Sycamore quantum processor for distance 3 and 5 surface codes. On distances up to 11, the decoder maintains its advantage on simulated data with realistic noise including cross-talk, leakage, and analog readout signals, and sustains its accuracy far beyond the 25 cycles it was trained on. Our work illustrates the ability of machine learning to go beyond human-designed algorithms by learning from data directly, highlighting machine learning as a strong contender for decoding in quantum computers.


title: Trans-Planckian censorship constraints on properties and cosmological applications of axion-like fields

abstract: We use the Trans-Planckian Censorship Conjecture (TCC) to constrain the decay constants $f$ characterizing a set of N identical axion-like fields with cosine potentials, improving upon the precision of other Swampland conjectures and existing string-theoretic arguments. We find that consistency with the TCC requires any such set of axion-like fields to satisfy $f\sqrt{N} \lesssim 0.6M_{pl}$, where $M_{pl}$ is the reduced Planck mass. We show that this bound makes models of axion-driven inflation incapable of simultaneously producing the required number of e-foldings and the observed scalar spectral tilt. In contrast, we find that models of axion quintessence can be simultaneously compatible with the TCC and observational data, provided that the axions' initial field values are set near the maxima of their potentials to within roughly $\pm \frac{\pi}{5}f$.


title: Impact of interface traps on charge noise, mobility and percolation density in Ge/SiGe heterostructures

abstract: Hole spins in Ge/SiGe heterostructure quantum dots have emerged as promising qubits for quantum computation. The strong spin-orbit coupling (SOC), characteristic of heavy-hole states in Ge, enables fast and all-electrical qubit control. However, SOC also increases the susceptibility of spin qubits to charge noise. While qubit coherence can be significantly improved by operating at sweet spots with reduced hyperfine or charge noise sensitivity, the latter ultimately limits coherence, underlining the importance of understanding and reducing charge noise at its source. In this work, we study the voltage-induced hysteresis commonly observed in SiGe-based quantum devices and show that the dominant charge fluctuators are localized at the semiconductor-oxide interface. By applying increasingly negative gate voltages to Hall bar and quantum dot devices, we investigate how the hysteretic filling of interface traps impacts transport metrics and charge noise. We find that the gate-induced accumulation and trapping of charge at the SiGe-oxide interface leads to an increased electrostatic disorder, as probed by transport measurements, as well as the activation of low-frequency relaxation dynamics, resulting in slow drifts and increased charge noise levels. Our results highlight the importance of a conservative device tuning strategy and reveal the critical role of the semiconductor-oxide interface in SiGe heterostructures for spin qubit applications.


title: Graphs with no even holes and no sector wheels are the union of two chordal graphs

abstract: Sivaraman conjectured that if $G$ is a graph with no induced even cycle then there exist sets $X_1, X_2 \subseteq V(G)$ satisfying $V(G) = X_1 \cup X_2$ such that the induced graphs $G[X_1]$ and $G[X_2]$ are both chordal. We prove this conjecture in the special case where $G$ contains no sector wheel, namely, a pair $(H, w)$ where $H$ is an induced cycle of $G$ and $w$ is a vertex in $V(G) \setminus V(H)$ such that $N(w) \cap H$ is either $V(H)$ or a path with at least three vertices.


title: On Multi-Fidelity Impedance Tuning for Human-Robot Cooperative Manipulation

abstract: We examine how a human-robot interaction (HRI) system may be designed when input-output data from previous experiments are available. In particular, we consider how to select an optimal impedance in the assistance design for a cooperative manipulation task with a new operator. Due to the variability between individuals, the design parameters that best suit one operator of the robot may not be the best parameters for another one. However, by incorporating historical data using a linear auto-regressive (AR-1) Gaussian process, the search for a new operator's optimal parameters can be accelerated. We lay out a framework for optimizing the human-robot cooperative manipulation that only requires input-output data. We establish how the AR-1 model improves the bound on the regret and numerically simulate a human-robot cooperative manipulation task to show the regret improvement. Further, we show how our approach's input-output nature provides robustness against modeling error through an additional numerical study.


title: TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models

abstract: The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challenges associated with training or fine-tuning these large models for such applications. Prior work mainly emphasizes either effective pretraining of large models for decision-making or single-task adaptation. But real-world problems will require data-efficient, continual adaptation for new control tasks. Recognizing these constraints, we introduce TAIL (Task-specific Adapters for Imitation Learning), a framework for efficient adaptation to new control tasks. Inspired by recent advancements in parameter-efficient fine-tuning in language domains, we explore efficient fine-tuning techniques -- e.g., Bottleneck Adapters, P-Tuning, and Low-Rank Adaptation (LoRA) -- in TAIL to adapt large pretrained models for new tasks with limited demonstration data. Our extensive experiments in large-scale language-conditioned manipulation tasks comparing prevalent parameter-efficient fine-tuning techniques and adaptation baselines suggest that TAIL with LoRA can achieve the best post-adaptation performance with only 1\% of the trainable parameters of full fine-tuning, while avoiding catastrophic forgetting and preserving adaptation plasticity in continual learning settings.


title: Variational quantum eigensolver boosted by adiabatic connection

abstract: In this work we integrate the variational quantum eigensolver (VQE) with the adiabatic connection (AC) method for efficient simulations of chemical problems on near-term quantum computers. Orbital optimized VQE methods are employed to capture the strong correlation within an active space and classical AC corrections recover the dynamical correlation effects comprising electrons outside of the active space. On two challenging strongly correlated problems, namely the dissociation of N$_2$ and the electronic structure of the tetramethyleneethane biradical, we show that the combined VQE-AC approach enhances the performance of VQE dramatically. Moreover, since the AC corrections do not bring any additional requirements on quantum resources or measurements, they can literally boost the VQE algorithms. Our work paves the way towards quantum simulations of real-life problems on near-term quantum computers.


title: Stochastic wave equation with heavy-tailed noise: Uniqueness of solutions and past-light cone property

abstract: In [5] was proved the existence of a solution for the stochastic wave equation driven by a L\'evy noise, which may have infinite $p$-th moments. In this work, we show the uniqueness of this solution using a compact support property of the fundamental solution of the wave operator. Furthermore, we extend this result to a general class of wave equations. Secondly, as a consequence of the past-light cone property, we show that $p$-th moments of the solution are bounded up to a certain stopping times that depend on a bounded region in $\mathbb{R}^d$.


title: Exploring Axions through the Photon Ring of a Spherically Symmetric Black Hole

abstract: In this study, we examine the phenomenon of photon axion conversion occurring in the spacetime surrounding a black hole. Specifically, we focus on the potential existence of a magnetic field around the supermassive black hole M87*, which could facilitate the conversion of photons into axions in close proximity to the photon sphere. While photons traverse through the curved spacetime, they spend time near the photon sphere, where conversion of these photons into axions takes place. Consequently, this process leads to a decrease in the intensity of the black hole's photon ring. To explore the possibilities of detecting these hypothetical axion particles, we propose observing the photon sphere using higher resolution telescopes. By doing so, we can gain valuable insights into the conversion mechanism as well as the nature of the spherically symmetric black hole geometry. Moreover, we also investigate how the photon ring luminosities are affected if the black hole possesses a charge parameter. For instance apart from U(1) electric charge, the presence of extra dimension may induce a {\em tidal charge} with a characteristic signature. It is important to note that the success of the conversion mechanism relies on the axion-photon coupling and mass. As a result, the modified luminosity of the black hole's photon ring offers a valuable means of constraining the axion's mass and coupling parameter within a certain range. Thus our findings contribute to a better understanding of photon axion conversion in the environment of a black hole spacetime and helps us explore the possible existence of extra spatial dimension.


title: Enumerating Calabi-Yau Manifolds: Placing bounds on the number of diffeomorphism classes in the Kreuzer-Skarke list

abstract: The diffeomorphism class of simply-connected smooth Calabi-Yau threefolds with torsion-free cohomology is determined via certain basic topological invariants: the Hodge numbers, the triple intersection form, and the second Chern class. In the present paper, we shed some light on this classification by placing bounds on the number of diffeomorphism classes present in the set of smooth Calabi-Yau threefolds constructed from the Kreuzer-Skarke list of reflexive polytopes up to Picard number six. The main difficulty arises from the comparison of triple intersection numbers and divisor integrals of the second Chern class up to basis transformations. By using certain basis-independent invariants, some of which appear here for the first time, we are able to place lower bounds on the number of classes. Upper bounds are obtained by explicitly identifying basis transformations, using constraints related to the index of line bundles. Extrapolating our results, we conjecture that the favourable entries of the Kreuzer-Skarke list of reflexive polytopes leads to some $10^{400}$ diffeomorphically distinct Calabi-Yau threefolds.


title: SALMON: Self-Alignment with Instructable Reward Models

abstract: Supervised Fine-Tuning (SFT) on response demonstrations combined with Reinforcement Learning from Human Feedback (RLHF) constitutes a powerful paradigm for aligning LLM-based AI agents. However, a significant limitation of such an approach is its dependency on high-quality human annotations, making its application to intricate tasks challenging due to difficulties in obtaining consistent response demonstrations and in-distribution response preferences. This paper presents a novel approach, namely SALMON, to align base language models with minimal human supervision, using only a small set of human-defined principles, yet achieving superior performance. Central to our approach is an instructable reward model. Trained on synthetic preference data, this model can generate reward scores based on arbitrary human-defined principles. By merely adjusting these principles during the RL training phase, we gain full control over the preferences with the instructable reward model, subsequently influencing the behavior of the RL-trained policy models, and reducing the reliance on the collection of online human preferences. Applying our method to the LLaMA-2-70b base language model, we developed an AI assistant named Dromedary-2. With only 6 exemplars for in-context learning and 31 human-defined principles, Dromedary-2 significantly surpasses the performance of several state-of-the-art AI systems, including LLaMA-2-Chat-70b, on various benchmark datasets. We have open-sourced the code and model weights to encourage further research into aligning LLM-based AI agents with enhanced supervision efficiency, improved controllability, and scalable oversight.


title: Energy Management in a Cooperative Energy Harvesting Wireless Sensor Network

abstract: In this paper, we consider the problem of finding an optimal energy management policy for a network of sensor nodes capable of harvesting their own energy and sharing it with other nodes in the network. We formulate this problem in the discounted cost Markov decision process framework and obtain good energy-sharing policies using the Deep Deterministic Policy Gradient (DDPG) algorithm. Earlier works have attempted to obtain the optimal energy allocation policy for a single sensor and for multiple sensors arranged on a mote with a single centralized energy buffer. Our algorithms, on the other hand, provide optimal policies for a distributed network of sensors individually harvesting energy and capable of sharing energy amongst themselves. Through simulations, we illustrate that the policies obtained by our DDPG algorithm using this enhanced network model outperform algorithms that do not share energy or use a centralized energy buffer in the distributed multi-nodal case.


title: State-independent Black Hole Interiors from the Crossed Product

abstract: Opinion is divided about the nature of state dependence in the black hole interior. Some argue that it is a necessary feature, while others argue it is a bug. In this paper, we consider the extended half-sided modular translation $U(s_0)$ (with $s_0 > 0$) of Leutheusser and Liu that takes us inside the horizon. We note that we can use this operator to construct a modular Hamiltonian $H$ and a conjugation $J$ on the infalling time-evolved wedges. The original thermofield double translates to a new cyclic and separating vector in the shifted algebra. We use these objects and the Connes cocycle to repeat Witten's crossed product construction in this new setting, and to obtain a Type II$_\infty$ algebra that is independent of the various choices, in particular that of the cyclic separating vector. Our emergent times are implicitly boundary-dressed. But if one admits an ``extra'' observer in the interior, we argue that the (state-independent) algebra can be Type I or Type II$_1$ instead of Type II$_\infty$, depending on whether the observer's light cone contains an entire Cauchy slice or not. Along with these general considerations, we present some specific calculations in the setting of the Poincare BTZ black hole. We identify a generalization of modular translations in BTZ-Kruskal coordinates that is pointwise (as opposed to non-local) and is analytically tractable, exploiting a connection with the covering AdS-space. These evolutions can reach the singularity.


title: Protected Fermionic Zero Modes in Periodic Gauge Fields

abstract: It is well-known that macroscopically-normalizable zero-energy wavefunctions of spin-$\frac{1}{2}$ particles in a two-dimensional inhomogeneous magnetic field are spin-polarized and exactly calculable with degeneracy equaling the number of flux quanta linking the whole system. Extending this argument to massless Dirac fermions subjected to magnetic fields that have \textit{zero} net flux but are doubly periodic in real space, we show that there exist \textit{only two} Bloch-normalizable zero-energy eigenstates, one for each spin flavor. This result is immediately relevant to graphene multilayer systems subjected to doubly-periodic strain fields, which at low energies, enter the Hamiltonian as periodic pseudo-gauge vector potentials. Furthermore, we explore various related settings including nonlinearly-dispersing band structure models and systems with singly-periodic magnetic fields.


title: NEFTune: Noisy Embeddings Improve Instruction Finetuning

abstract: We show that language model finetuning can be improved, sometimes dramatically, with a simple augmentation. NEFTune adds noise to the embedding vectors during training. Standard finetuning of LLaMA-2-7B using Alpaca achieves 29.79% on AlpacaEval, which rises to 64.69% using noisy embeddings. NEFTune also improves over strong baselines on modern instruction datasets. Models trained with Evol-Instruct see a 10% improvement, with ShareGPT an 8% improvement, and with OpenPlatypus an 8% improvement. Even powerful models further refined with RLHF such as LLaMA-2-Chat benefit from additional training with NEFTune.


title: FireAct: Toward Language Agent Fine-tuning

abstract: Recent efforts have augmented language models (LMs) with external tools or environments, leading to the development of language agents that can reason and act. However, most of these agents rely on few-shot prompting techniques with off-the-shelf LMs. In this paper, we investigate and argue for the overlooked direction of fine-tuning LMs to obtain language agents. Using a setup of question answering (QA) with a Google search API, we explore a variety of base LMs, prompting methods, fine-tuning data, and QA tasks, and find language agents are consistently improved after fine-tuning their backbone LMs. For example, fine-tuning Llama2-7B with 500 agent trajectories generated by GPT-4 leads to a 77% HotpotQA performance increase. Furthermore, we propose FireAct, a novel approach to fine-tuning LMs with trajectories from multiple tasks and prompting methods, and show having more diverse fine-tuning data can further improve agents. Along with other findings regarding scaling effects, robustness, generalization, efficiency and cost, our work establishes comprehensive benefits of fine-tuning LMs for agents, and provides an initial set of experimental designs, insights, as well as open questions toward language agent fine-tuning.


title: Interpreting CLIP's Image Representation via Text-Based Decomposition

abstract: We investigate the CLIP image encoder by analyzing how individual model components affect the final representation. We decompose the image representation as a sum across individual image patches, model layers, and attention heads, and use CLIP's text representation to interpret the summands. Interpreting the attention heads, we characterize each head's role by automatically finding text representations that span its output space, which reveals property-specific roles for many heads (e.g. location or shape). Next, interpreting the image patches, we uncover an emergent spatial localization within CLIP. Finally, we use this understanding to remove spurious features from CLIP and to create a strong zero-shot image segmenter. Our results indicate that a scalable understanding of transformer models is attainable and can be used to repair and improve models.


title: Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Input

abstract: Clothing is an important part of human appearance but challenging to model in photorealistic avatars. In this work we present avatars with dynamically moving loose clothing that can be faithfully driven by sparse RGB-D inputs as well as body and face motion. We propose a Neural Iterative Closest Point (N-ICP) algorithm that can efficiently track the coarse garment shape given sparse depth input. Given the coarse tracking results, the input RGB-D images are then remapped to texel-aligned features, which are fed into the drivable avatar models to faithfully reconstruct appearance details. We evaluate our method against recent image-driven synthesis baselines, and conduct a comprehensive analysis of the N-ICP algorithm. We demonstrate that our method can generalize to a novel testing environment, while preserving the ability to produce high-fidelity and faithful clothing dynamics and appearance.


title: Grokking as Compression: A Nonlinear Complexity Perspective

abstract: We attribute grokking, the phenomenon where generalization is much delayed after memorization, to compression. To do so, we define linear mapping number (LMN) to measure network complexity, which is a generalized version of linear region number for ReLU networks. LMN can nicely characterize neural network compression before generalization. Although the $L_2$ norm has been a popular choice for characterizing model complexity, we argue in favor of LMN for a number of reasons: (1) LMN can be naturally interpreted as information/computation, while $L_2$ cannot. (2) In the compression phase, LMN has linear relations with test losses, while $L_2$ is correlated with test losses in a complicated nonlinear way. (3) LMN also reveals an intriguing phenomenon of the XOR network switching between two generalization solutions, while $L_2$ does not. Besides explaining grokking, we argue that LMN is a promising candidate as the neural network version of the Kolmogorov complexity since it explicitly considers local or conditioned linear computations aligned with the nature of modern artificial neural networks.


title: Few-Shot Spoken Language Understanding via Joint Speech-Text Models

abstract: Recent work on speech representation models jointly pre-trained with text has demonstrated the potential of improving speech representations by encoding speech and text in a shared space. In this paper, we leverage such shared representations to address the persistent challenge of limited data availability in spoken language understanding tasks. By employing a pre-trained speech-text model, we find that models fine-tuned on text can be effectively transferred to speech testing data. With as little as 1 hour of labeled speech data, our proposed approach achieves comparable performance on spoken language understanding tasks (specifically, sentiment analysis and named entity recognition) when compared to previous methods using speech-only pre-trained models fine-tuned on 10 times more data. Beyond the proof-of-concept study, we also analyze the latent representations. We find that the bottom layers of speech-text models are largely task-agnostic and align speech and text representations into a shared space, while the top layers are more task-specific.


title: SimPLR: A Simple and Plain Transformer for Scaling-Efficient Object Detection and Segmentation

abstract: The ability to detect objects in images at varying scales has played a pivotal role in the design of modern object detectors. Despite considerable progress in removing hand-crafted components and simplifying the architecture with transformers, multi-scale feature maps and/or pyramid design remain a key factor for their empirical success. In this paper, we show that this reliance on either feature pyramids or an hierarchical backbone is unnecessary and a transformer-based detector with scale-aware attention enables the plain detector `SimPLR' whose backbone and detection head are both non-hierarchical and operate on single-scale features. We find through our experiments that SimPLR with scale-aware attention is plain and simple, yet competitive with multi-scale vision transformer alternatives. Compared to the multi-scale and single-scale state-of-the-art, our model scales much better with bigger capacity (self-supervised) models and more pre-training data, allowing us to report a consistently better accuracy and faster runtime for object detection, instance segmentation as well as panoptic segmentation. Code will be released.


title: Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions

abstract: We introduce Conformal Decision Theory, a framework for producing safe autonomous decisions despite imperfect machine learning predictions. Examples of such decisions are ubiquitous, from robot planning algorithms that rely on pedestrian predictions, to calibrating autonomous manufacturing to exhibit high throughput and low error, to the choice of trusting a nominal policy versus switching to a safe backup policy at run-time. The decisions produced by our algorithms are safe in the sense that they come with provable statistical guarantees of having low risk without any assumptions on the world model whatsoever; the observations need not be I.I.D. and can even be adversarial. The theory extends results from conformal prediction to calibrate decisions directly, without requiring the construction of prediction sets. Experiments demonstrate the utility of our approach in robot motion planning around humans, automated stock trading, and robot manufacturing.