title: What Do Language Models Learn in Context? The Structured Task Hypothesis

abstract: Large language models (LLMs) exhibit an intriguing ability to learn a novel task from in-context examples presented in a demonstration, termed in-context learning (ICL). Understandably, a swath of research has been dedicated to uncovering the theories underpinning ICL. One popular hypothesis explains ICL by task selection. LLMs identify the task based on the demonstration and generalize it to the prompt. Another popular hypothesis is that ICL is a form of meta-learning, i.e., the models learn a learning algorithm at pre-training time and apply it to the demonstration. Finally, a third hypothesis argues that LLMs use the demonstration to select a composition of tasks learned during pre-training to perform ICL. In this paper, we empirically explore these three hypotheses that explain LLMs' ability to learn in context with a suite of experiments derived from common text classification tasks. We invalidate the first two hypotheses with counterexamples and provide evidence in support of the last hypothesis. Our results suggest an LLM could learn a novel task in context via composing tasks learned during pre-training.


title: Optomechanical Backaction in the Bistable Regime

abstract: With a variety of realisations, optomechanics utilizes its light matter interaction to test fundamental physics. By coupling the phonons of a mechanical resonator to the photons in a high quality cavity, control of increasingly macroscopic objects has become feasible. In such systems, state manipulation of the mechanical mode is achieved by driving the cavity. To be able to achieve high drive powers the system is typically designed such that it remains in a linear response regime when driven. A nonlinear response and especially bistability in a driven cavity is often considered detrimentally to cooling and state preparation in optomechanical systems and is avoided in experiments. Here we show, that with an intrinsic nonlinear cavity backaction cooling of a mechanical resonator is feasible operating deeply within the nonlinear regime of the cavity. With our theory taking the nonlinearity into account, precise predictions on backaction cooling can be achieved even with a cavity beyond the bifurcation point, where the cavity photon number spectrum starts to deviate from a typical Lorentzian shape.


title: Linguistic Steganalysis via LLMs: Two Modes for Efficient Detection of Strongly Concealed Stego

abstract: To detect stego (steganographic text) in complex scenarios, linguistic steganalysis (LS) with various motivations has been proposed and achieved excellent performance. However, with the development of generative steganography, some stegos have strong concealment, especially after the emergence of LLMs-based steganography, the existing LS has low detection or cannot detect them. We designed a novel LS with two modes called LSGC. In the generation mode, we created an LS-task "description" and used the generation ability of LLM to explain whether texts to be detected are stegos. On this basis, we rethought the principle of LS and LLMs, and proposed the classification mode. In this mode, LSGC deleted the LS-task "description" and used the "causalLM" LLMs to extract steganographic features. The LS features can be extracted by only one pass of the model, and a linear layer with initialization weights is added to obtain the classification probability. Experiments on strongly concealed stegos show that LSGC significantly improves detection and reaches SOTA performance. Additionally, LSGC in classification mode greatly reduces training time while maintaining high performance.


title: Multi-Agent Imitation Learning: Value is Easy, Regret is Hard

abstract: We study a multi-agent imitation learning (MAIL) problem where we take the perspective of a learner attempting to coordinate a group of agents based on demonstrations of an expert doing so. Most prior work in MAIL essentially reduces the problem to matching the behavior of the expert within the support of the demonstrations. While doing so is sufficient to drive the value gap between the learner and the expert to zero under the assumption that agents are non-strategic, it does not guarantee robustness to deviations by strategic agents. Intuitively, this is because strategic deviations can depend on a counterfactual quantity: the coordinator's recommendations outside of the state distribution their recommendations induce. In response, we initiate the study of an alternative objective for MAIL in Markov Games we term the regret gap that explicitly accounts for potential deviations by agents in the group. We first perform an in-depth exploration of the relationship between the value and regret gaps. First, we show that while the value gap can be efficiently minimized via a direct extension of single-agent IL algorithms, even value equivalence can lead to an arbitrarily large regret gap. This implies that achieving regret equivalence is harder than achieving value equivalence in MAIL. We then provide a pair of efficient reductions to no-regret online convex optimization that are capable of minimizing the regret gap (a) under a coverage assumption on the expert (MALICE) or (b) with access to a queryable expert (BLADES).


title: BEADs: Bias Evaluation Across Domains

abstract: Recent improvements in large language models (LLMs) have significantly enhanced natural language processing (NLP) applications. However, these models can also inherit and perpetuate biases from their training data. Addressing this issue is crucial, yet many existing datasets do not offer evaluation across diverse NLP tasks. To tackle this, we introduce the Bias Evaluations Across Domains (BEADs) dataset, designed to support a wide range of NLP tasks, including text classification, bias entity recognition, bias quantification, and benign language generation. BEADs uses AI-driven annotation combined with experts' verification to provide reliable labels. This method overcomes the limitations of existing datasets that typically depend on crowd-sourcing, expert-only annotations with limited bias evaluations, or unverified AI labeling. Our empirical analysis shows that BEADs is effective in detecting and reducing biases across different language models, with smaller models fine-tuned on BEADs often outperforming LLMs in bias classification tasks. However, these models may still exhibit biases towards certain demographics. Fine-tuning LLMs with our benign language data also reduces biases while preserving the models' knowledge. Our findings highlight the importance of comprehensive bias evaluation and the potential of targeted fine-tuning for reducing the bias of LLMs. We are making BEADs publicly available at https://huggingface.co/datasets/shainar/BEAD Warning: This paper contains examples that may be considered offensive.


title: Matching Anything by Segmenting Anything

abstract: The robust association of the same objects across video frames in complex scenes is crucial for many applications, especially Multiple Object Tracking (MOT). Current methods predominantly rely on labeled domain-specific video datasets, which limits the cross-domain generalization of learned similarity embeddings. We propose MASA, a novel method for robust instance association learning, capable of matching any objects within videos across diverse domains without tracking labels. Leveraging the rich object segmentation from the Segment Anything Model (SAM), MASA learns instance-level correspondence through exhaustive data transformations. We treat the SAM outputs as dense object region proposals and learn to match those regions from a vast image collection. We further design a universal MASA adapter which can work in tandem with foundational segmentation or detection models and enable them to track any detected objects. Those combinations present strong zero-shot tracking ability in complex domains. Extensive tests on multiple challenging MOT and MOTS benchmarks indicate that the proposed method, using only unlabeled static images, achieves even better performance than state-of-the-art methods trained with fully annotated in-domain video sequences, in zero-shot association. Project Page: https://matchinganything.github.io/


title: Coarse embeddability, $L^1$-compression and Percolations on General Graphs

abstract: We show that a locally finite, connected graph has a coarse embedding into a Hilbert space if and only if there exist bond percolations with arbitrarily large marginals and two-point function vanishing at infinity. We further show that the decay is stretched exponential with stretching exponent $\alpha\in[0,1]$ if and only if the $L^1$-compression exponent of the graph is at least $\alpha$, leading to a probabilistic characterization of this exponent. These results are new even in the particular setting of Cayley graphs of finitely generated groups. The proofs build on a new probabilistic method introduced recently by the authors to study group-invariant percolation on Cayley graphs [24,25], which is now extended to the general, non-symmetric situation of graphs to study their coarse embeddability and compression exponents.


title: Resolving the Module of Derivations on an $n \times (n+1)$ Determinantal Ring

abstract: We use the construction of the relative bar resolution via differential graded structures to obtain the minimal graded free resolution of $\text{Der}_{R \mid k}$, where $R$ is a determinantal ring defined by the maximal minors of an $n \times (n+1)$ generic matrix and $k$ is its coefficient field. Along the way, we compute an explicit action of the Hilbert-Burch differential graded algebra on a differential graded module resolving the cokernel of the Jacobian matrix whose kernel is $\text{Der}_{R \mid k}$. As a consequence of the minimality of the resulting relative bar resolution, we get a minimal generating set for $\text{Der}_{R \mid k}$ as an $R$-module, which, while already known, has not been obtained via our methods.


title: Rank-2 wobbly bundles from special divisors on spectral curves

abstract: Let $X$ be a compact connected Riemann surface of genus $g \geq 3$ and $S \rightarrow X$ a $2:1$ branched covering defined by a generic quadratic differential on $X$. In the following notes, we explore how taking direct images of line bundles defined by certain distinguished special divisors on $S$ gives rank-2 bundles having nilpotent Higgs fields on $X$. The key ingredient is the notion of Baker-Akhiezer divisors associated to Higgs bundles and choices of subbundles of the underlying rank-2 bundles.


title: Homology of spectral minimal partitions

abstract: A spectral minimal partition of a manifold is its decomposition into disjoint open sets that minimizes a spectral energy functional. It is known that bipartite spectral minimal partitions coincide with nodal partitions of Courant-sharp Laplacian eigenfunctions. However, almost all minimal partitions are non-bipartite. To study those, we define a modified Laplacian operator and prove that the nodal partitions of its Courant-sharp eigenfunctions are minimal within a certain topological class of partitions. This yields new results in the non-bipartite case and recovers the above known result in the bipartite case. Our approach is based on tools from algebraic topology, which we illustrate by a number of examples where the topological types of partitions are characterized by relative homology.


title: C*-framework for higher-order bulk-boundary correspondences

abstract: A typical crystal is a finite piece of a material which may be invariant under some point symmetry group. If it is a so-called intrinsic higher-order topological insulator or superconductor then it displays boundary modes at hinges or corners protected by the crystalline symmetry and the bulk topology. We explain the mechanism behind that using operator K-theory. Specifically, we derive a groupoid $C^\ast$-algebra that 1) encodes the dynamics of the electrons in the infinite size limit of a crystal; 2) remembers the boundary conditions at the crystal's boundaries, and 3) accepts a natural action by the point symmetries of the atomic lattice. The filtrations of the groupoid's unit space by closed subsets that are invariant under the groupoid and point group actions supply equivariant co-filtrations of the groupoid $C^\ast$-algebra. We show that specific derivations of the induced spectral sequences in twisted equivariant K-theories enumerate all non-trivial higher-order bulk-boundary correspondences.

categories: math-ph cond-mat.str-el math.KT math.MP math.OA


title: R-CONV: An Analytical Approach for Efficient Data Reconstruction via Convolutional Gradients

abstract: In the effort to learn from extensive collections of distributed data, federated learning has emerged as a promising approach for preserving privacy by using a gradient-sharing mechanism instead of exchanging raw data. However, recent studies show that private training data can be leaked through many gradient attacks. While previous analytical-based attacks have successfully reconstructed input data from fully connected layers, their effectiveness diminishes when applied to convolutional layers. This paper introduces an advanced data leakage method to efficiently exploit convolutional layers' gradients. We present a surprising finding: even with non-fully invertible activation functions, such as ReLU, we can analytically reconstruct training samples from the gradients. To the best of our knowledge, this is the first analytical approach that successfully reconstructs convolutional layer inputs directly from the gradients, bypassing the need to reconstruct layers' outputs. Prior research has mainly concentrated on the weight constraints of convolution layers, overlooking the significance of gradient constraints. Our findings demonstrate that existing analytical methods used to estimate the risk of gradient attacks lack accuracy. In some layers, attacks can be launched with less than 5% of the reported constraints.


title: Early-time resonances in the three-dimensional wall-bounded axisymmetric Euler and related equations

abstract: We investigate the complex-time analytic structure of solutions of the 3D-axisymmetric, wall-bounded, incompressible Euler equations, by starting with the initial data proposed in Luo and Hou (2014), to study a possible finite-time singularity. We use our pseudospectral Fourier-Chebyshev method, with quadruple-precision arithmetic, to compute the time-Taylor series coefficients of the flow fields, up to a high order. We show that the resulting approximations display early-time resonances; the initial spatial location of these structures is different from that for the tygers, which we have obtained in Kolluru et al. (2022). We then perform asymptotic analysis of the Taylor-series coefficients, by using generalised ratio methods, to extract the location and nature of the convergence-limiting singularities and demonstrate that these singularities are distributed around the origin, in the complex-t2 plane, along two curves that resemble the shape of an eye. We obtain similar results for the 1D wall-approximation (of the full 3D-axisymmetric Euler equation) called the 1D HL model, for which we use Fourier-pseudospectral methods to compute the time-Taylor series coefficients of the flow fields. Our work examines the link between tygers, in Galerkin-truncated pseudospectral studies, and early-time resonances, in truncated time-Taylor expansions of solutions of PDEs, such as those we consider.


title: The CLRS-Text Algorithmic Reasoning Language Benchmark

abstract: Eliciting reasoning capabilities from language models (LMs) is a critical direction on the path towards building intelligent systems. Most recent studies dedicated to reasoning focus on out-of-distribution performance on procedurally-generated synthetic benchmarks, bespoke-built to evaluate specific skills only. This trend makes results hard to transfer across publications, slowing down progress. Three years ago, a similar issue was identified and rectified in the field of neural algorithmic reasoning, with the advent of the CLRS benchmark. CLRS is a dataset generator comprising graph execution traces of classical algorithms from the Introduction to Algorithms textbook. Inspired by this, we propose CLRS-Text -- a textual version of these algorithmic traces. Out of the box, CLRS-Text is capable of procedurally generating trace data for thirty diverse, challenging algorithmic tasks across any desirable input distribution, while offering a standard pipeline in which any additional algorithmic tasks may be created in the benchmark. We fine-tune and evaluate various LMs as generalist executors on this benchmark, validating prior work and revealing a novel, interesting challenge for the LM reasoning community. Our code is available at https://github.com/google-deepmind/clrs/tree/master/clrs/_src/clrs_text.


title: M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and RGB Data

abstract: Satellite-based remote sensing has revolutionised the way we address global challenges in a rapidly evolving world. Huge quantities of Earth Observation (EO) data are generated by satellite sensors daily, but processing these large datasets for use in ML pipelines is technically and computationally challenging. Specifically, different types of EO data are often hosted on a variety of platforms, with differing availability for Python preprocessing tools. In addition, spatial alignment across data sources and data tiling can present significant technical hurdles for novice users. While some preprocessed EO datasets exist, their content is often limited to optical or near-optical wavelength data, which is ineffective at night or in adverse weather conditions. Synthetic Aperture Radar (SAR), an active sensing technique based on microwave length radiation, offers a viable alternative. However, the application of machine learning to SAR has been limited due to a lack of ML-ready data and pipelines, particularly for the full diversity of SAR data, including polarimetry, coherence and interferometry. We introduce M3LEO, a multi-modal, multi-label EO dataset that includes polarimetric, interferometric, and coherence SAR data derived from Sentinel-1, alongside Sentinel-2 RGB imagery and a suite of labelled tasks for model evaluation. M3LEO spans 17.5TB and contains approximately 10M data chips across six geographic regions. The dataset is complemented by a flexible PyTorch Lightning framework, with configuration management using Hydra. We provide tools to process any dataset available on popular platforms such as Google Earth Engine for integration with our framework. Initial experiments validate the utility of our data and framework, showing that SAR imagery contains information additional to that extractable from RGB data. Data at huggingface.co/M3LEO, and code at github.com/spaceml-org/M3LEO.


title: Quantifying Misalignment Between Agents

abstract: Growing concerns about the AI alignment problem have emerged in recent years, with previous work focusing mainly on (1) qualitative descriptions of the alignment problem; (2) attempting to align AI actions with human interests by focusing on value specification and learning; and/or (3) focusing on a single agent or on humanity as a singular unit. Recent work in sociotechnical AI alignment has made some progress in defining alignment inclusively, but the field as a whole still lacks a systematic understanding of how to specify, describe, and analyze misalignment among entities, which may include individual humans, AI agents, and complex compositional entities such as corporations, nation-states, and so forth. Previous work on controversy in computational social science offers a mathematical model of contention among populations (of humans). In this paper, we adapt this contention model to the alignment problem, and show how misalignment can vary depending on the population of agents (human or otherwise) being observed, the domain in question, and the agents' probability-weighted preferences between possible outcomes. Our model departs from value specification approaches and focuses instead on the morass of complex, interlocking, sometimes contradictory goals that agents may have in practice. We apply our model by analyzing several case studies ranging from social media moderation to autonomous vehicle behavior. By applying our model with appropriately representative value data, AI engineers can ensure that their systems learn values maximally aligned with diverse human interests.


title: Multidimensional Stability of Planar Travelling Waves for Stochastically Perturbed Reaction-Diffusion Systems

abstract: We consider reaction-diffusion systems with multiplicative noise on a spatial domain of dimension two or higher. The noise process is white in time, coloured in space, and invariant under translations. In the deterministic setting, multidimensional stability of planar waves on the whole space $\mathbb R^d$ has been studied by many. Inspired by previous works on the real line, we establish the multidimensional stability of planar waves on a cylindrical domain on time scales that are exponentially long with respect to the noise strength. This is achieved by means of a stochastic phase tracking mechanism that can be maintained over such long time scales. The corresponding mild formulation of our problem features stochastic integrals with respect to anticipating integrands, which hence cannot be understood within the well-established setting of It\^o-integrals. To circumvent this problem, we exploit and extend recently developed theory concerning forward integrals.


title: FairytaleQA Translated: Enabling Educational Question and Answer Generation in Less-Resourced Languages

abstract: Question Answering (QA) datasets are crucial in assessing reading comprehension skills for both machines and humans. While numerous datasets have been developed in English for this purpose, a noticeable void exists in less-resourced languages. To alleviate this gap, our paper introduces machine-translated versions of FairytaleQA, a renowned QA dataset designed to assess and enhance narrative comprehension skills in young children. By employing fine-tuned, modest-scale models, we establish benchmarks for both Question Generation (QG) and QA tasks within the translated datasets. In addition, we present a case study proposing a model for generating question-answer pairs, with an evaluation incorporating quality metrics such as question well-formedness, answerability, relevance, and children suitability. Our evaluation prioritizes quantifying and describing error cases, along with providing directions for future work. This paper contributes to the advancement of QA and QG research in less-resourced languages, promoting accessibility and inclusivity in the development of these models for reading comprehension. The code and data is publicly available at github.com/bernardoleite/fairytaleqa-translated.


title: Hidden zeros = secret ultraviolet scaling, and a new path to uniqueness

abstract: We investigate the hidden amplitude zeros discovered by Arkani-Hamed et al, which describe a non-trivial vanishing of scattering amplitudes on special external kinematics. We prove the conjecture that amplitude zeros uniquely fix Tr$(\phi^3)$, by first showing that any amplitude can be divided into special subsets, which independently satisfy the zero condition. We further prove that this condition is in fact equivalent to a novel enhancement under Britto-Cachazo-Feng-Witten shifts of the same subsets. More generally, for one particular zero type, this equivalence can be extended to arbitrary rational functions built from planar variables (such as the non-linear sigma model). In the case of Yang-Mills theory, we conjecture the zeros, combined with color-kinematic duality, uniquely fix the $\lfloor n/2\rfloor$ different tensor structures of $n$-point gluon amplitudes. Our approach suggests a straightforward avenue for understanding previous similar uniqueness results, in which unitarity appears as an emergent property, and naively independent physical principles contain identical (amplitude) information.


title: Toward Artificial Open-Ended Evolution within Lenia using Quality-Diversity

abstract: From the formation of snowflakes to the evolution of diverse life forms, emergence is ubiquitous in our universe. In the quest to understand how complexity can arise from simple rules, abstract computational models, such as cellular automata, have been developed to study self-organization. However, the discovery of self-organizing patterns in artificial systems is challenging and has largely relied on manual or semi-automatic search in the past. In this paper, we show that Quality-Diversity, a family of Evolutionary Algorithms, is an effective framework for the automatic discovery of diverse self-organizing patterns in complex systems. Quality-Diversity algorithms aim to evolve a large population of diverse individuals, each adapted to its ecological niche. Combined with Lenia, a family of continuous cellular automata, we demonstrate that our method is able to evolve a diverse population of lifelike self-organizing autonomous patterns. Our framework, called Leniabreeder, can leverage both manually defined diversity criteria to guide the search toward interesting areas, as well as unsupervised measures of diversity to broaden the scope of discoverable patterns. We demonstrate both qualitatively and quantitatively that Leniabreeder offers a powerful solution for discovering self-organizing patterns. The effectiveness of unsupervised Quality-Diversity methods combined with the rich landscape of Lenia exhibits a sustained generation of diversity and complexity characteristic of biological evolution. We provide empirical evidence that suggests unbounded diversity and argue that Leniabreeder is a step toward replicating open-ended evolution in silico.


title: Understanding Information Storage and Transfer in Multi-modal Large Language Models

abstract: Understanding the mechanisms of information storage and transfer in Transformer-based models is important for driving model understanding progress. Recent work has studied these mechanisms for Large Language Models (LLMs), revealing insights on how information is stored in a model's parameters and how information flows to and from these parameters in response to specific prompts. However, these studies have not yet been extended to Multi-modal Large Language Models (MLLMs). Given their expanding capabilities and real-world use, we start by studying one aspect of these models -- how MLLMs process information in a factual visual question answering task. We use a constraint-based formulation which views a visual question as having a set of visual or textual constraints that the model's generated answer must satisfy to be correct (e.g. What movie directed by the director in this photo has won a Golden Globe?). Under this setting, we contribute i) a method that extends causal information tracing from pure language to the multi-modal setting, and ii) VQA-Constraints, a test-bed of 9.7K visual questions annotated with constraints. We use these tools to study two open-source MLLMs, LLaVa and multi-modal Phi-2. Our key findings show that these MLLMs rely on MLP and self-attention blocks in much earlier layers for information storage, compared to LLMs whose mid-layer MLPs are more important. We also show that a consistent small subset of visual tokens output by the vision encoder are responsible for transferring information from the image to these causal blocks. We validate these mechanisms by introducing MultEdit, a model-editing algorithm that can correct errors and insert new long-tailed information into MLLMs by targeting these causal blocks.


title: On the equational theory of finite modular lattices

abstract: It is shown that the equational theory of modular lattices is undecidable


title: The evolution of the transverse-momentum dependent gluon distribution at small $x$

abstract: Using the colour dipole picture for photon-nucleus interactions at small $x$ together with the Color Glass Condensate (CGC) effective theory, we demonstrate that the next-to-leading (NLO) order corrections to the cross-section for the inclusive production of a pair of hard jets encode not only the JIMWLK evolution with decreasing $x$, but also the DGLAP evolution of the gluon distribution function and the CSS evolution of the gluon transverse momentum dependent (TMD) distribution. The emergent CSS equation takes the form of a rate equation describing the evolution of the dijet distribution in the transverse momentum imbalance $K_\perp$ when increasing the dijet relative momentum $P_\perp$. All three types of evolution become important when both $P_\perp$ and $K_\perp$ are much larger than the nuclear saturation momentum $Q_s(x)$ and we propose a framework which encompasses all of them. The solution to the JIMWLK equation provides the source term for the DGLAP evolution with increasing $K_\perp$, which in turn generates the initial condition for the CSS evolution with increasing $P_\perp$.


title: Solving Inverse Problems in Protein Space Using Diffusion-Based Priors

abstract: The interaction of a protein with its environment can be understood and controlled via its 3D structure. Experimental methods for protein structure determination, such as X-ray crystallography or cryogenic electron microscopy, shed light on biological processes but introduce challenging inverse problems. Learning-based approaches have emerged as accurate and efficient methods to solve these inverse problems for 3D structure determination, but are specialized for a predefined type of measurement. Here, we introduce a versatile framework to turn raw biophysical measurements of varying types into 3D atomic models. Our method combines a physics-based forward model of the measurement process with a pretrained generative model providing a task-agnostic, data-driven prior. Our method outperforms posterior sampling baselines on both linear and non-linear inverse problems. In particular, it is the first diffusion-based method for refining atomic models from cryo-EM density maps.


title: Hypernetworks for Personalizing ASR to Atypical Speech

abstract: Parameter-efficient fine-tuning (PEFT) for personalizing automatic speech recognition (ASR) has recently shown promise for adapting general population models to atypical speech. However, these approaches assume a priori knowledge of the atypical speech disorder being adapted for -- the diagnosis of which requires expert knowledge that is not always available. Even given this knowledge, data scarcity and high inter/intra-speaker variability further limit the effectiveness of traditional fine-tuning. To circumvent these challenges, we first identify the minimal set of model parameters required for ASR adaptation. Our analysis of each individual parameter's effect on adaptation performance allows us to reduce Word Error Rate (WER) by half while adapting 0.03% of all weights. Alleviating the need for cohort-specific models, we next propose the novel use of a meta-learned hypernetwork to generate highly individualized, utterance-level adaptations on-the-fly for a diverse set of atypical speech characteristics. Evaluating adaptation at the global, cohort and individual-level, we show that hypernetworks generalize better to out-of-distribution speakers, while maintaining an overall relative WER reduction of 75.2% using 0.1% of the full parameter budget.


title: Regular one-parameter groups, reflection positivity and their application to Hankel operators and standard subspaces

abstract: Standard subspaces are a well studied object in algebraic quantum field theory (AQFT). Given a standard subspace ${\tt V}$ of a Hilbert space $\mathcal{H}$, one is interested in unitary one-parameter groups on $\mathcal{H}$ with $U_t {\tt V} \subseteq {\tt V}$ for every $t \in \mathbb{R}_+$. If $({\tt V},U)$ is a non-degenerate standard pair on $\mathcal{H}$, i.e. the self-adjoint infinitesimal generator of $U$ is a positive operator with trivial kernel, two classical results are given by Borchers' Theorem, relating non-degenerate standard pairs to positive energy representations of the affine group $\mathrm{Aff}(\mathbb{R})$ and the Longo--Witten Theorem, stating the the semigroup of unitary endomorphisms of ${\tt V}$ can be identified with the semigroup of symmetric operator-valued inner functions on the upper half plane. In this thesis we prove results similar to the theorems of Borchers and of Longo--Witten for a more general framework of unitary one-parameter groups without the assumption that their infinitesimal generator is positive. We replace this assumption by the weaker assumption that the triple $(\mathcal{H},{\tt V},U)$ is a so called real regular one-parameter group.


title: Exploring magnetic fields in merging galaxy: combining polarization and velocity gradient in the Centaurus Galaxy

abstract: In this study, we apply the Velocity Gradient Technique (VGT) to the merging Centaurus galaxy. We compare gradient maps derived from the PHANGS-ALMA survey using CO emission lines with magnetic field tracings from dust polarization data obtained via the HAWC+ instrument. Our analysis reveals a strong correspondence between the directions indicated by these two tracers across most of the galactic image. Specifically, we identify jet regions as areas of anti-alignment, consistent with previous reports that gradients tend to rotate 90 degrees in outflow regions. Statistically, we find that the alignment of magnetic fields, as revealed by polarization, is most accurate in regions with the highest signal-to-noise ratios. Our findings underscore the utility of velocity gradients as a valuable complementary tool for probing magnetic fields and dynamical processes in merging galaxies.


title: Policy Optimization in Control: Geometry and Algorithmic Implications

abstract: This survey explores the geometric perspective on policy optimization within the realm of feedback control systems, emphasizing the intrinsic relationship between control design and optimization. By adopting a geometric viewpoint, we aim to provide a nuanced understanding of how various ``complete parameterization'' -- referring to the policy parameters together with its Riemannian geometry -- of control design problems, influence stability and performance of local search algorithms. The paper is structured to address key themes such as policy parameterization, the topology and geometry of stabilizing policies, and their implications for various (non-convex) dynamic performance measures. We focus on a few iconic control design problems, including the Linear Quadratic Regulator (LQR), Linear Quadratic Gaussian (LQG) control, and $\mathcal{H}_\infty$ control. In particular, we first discuss the topology and Riemannian geometry of stabilizing policies, distinguishing between their static and dynamic realizations. Expanding on this geometric perspective, we then explore structural properties of the aforementioned performance measures and their interplay with the geometry of stabilizing policies in presence of policy constraints; along the way, we address issues such as spurious stationary points, symmetries of dynamic feedback policies, and (non-)smoothness of the corresponding performance measures. We conclude the survey with algorithmic implications of policy optimization in feedback design.


title: Benchmark Data Contamination of Large Language Models: A Survey

abstract: The rapid development of Large Language Models (LLMs) like GPT-4, Claude-3, and Gemini has transformed the field of natural language processing. However, it has also resulted in a significant issue known as Benchmark Data Contamination (BDC). This occurs when language models inadvertently incorporate evaluation benchmark information from their training data, leading to inaccurate or unreliable performance during the evaluation phase of the process. This paper reviews the complex challenge of BDC in LLM evaluation and explores alternative assessment methods to mitigate the risks associated with traditional benchmarks. The paper also examines challenges and future directions in mitigating BDC risks, highlighting the complexity of the issue and the need for innovative solutions to ensure the reliability of LLM evaluation in real-world applications.


title: Online learning of a panoply of quantum objects

abstract: In many quantum tasks, there is an unknown quantum object that one wishes to learn. An online strategy for this task involves adaptively refining a hypothesis to reproduce such an object or its measurement statistics. A common evaluation metric for such a strategy is its regret, or roughly the accumulated errors in hypothesis statistics. We prove a sublinear regret bound for learning over general subsets of positive semidefinite matrices via the regularized-follow-the-leader algorithm and apply it to various settings where one wishes to learn quantum objects. For concrete applications, we present a sublinear regret bound for learning quantum states, effects, channels, interactive measurements, strategies, co-strategies, and the collection of inner products of pure states. Our bound applies to many other quantum objects with compact, convex representations. In proving our regret bound, we establish various matrix analysis results useful in quantum information theory. This includes a generalization of Pinsker's inequality for arbitrary positive semidefinite operators with possibly different traces, which may be of independent interest and applicable to more general classes of divergences.


title: Complementary polynomials in quantum signal processing

abstract: Quantum signal processing is a framework for implementing polynomial functions on quantum computers. To implement a given polynomial $P$, one must first construct a corresponding complementary polynomial $Q$. Existing approaches to this problem employ numerical methods that are not amenable to explicit error analysis. We present a new approach to complementary polynomials using complex analysis. Our main mathematical result is a contour integral representation for a canonical complementary polynomial. The integral representation on the unit circle has a particularly simple and efficacious Fourier analytic interpretation, which we use to develop a Fast Fourier Transform-based algorithm for the efficient calculation of $Q$ in the monomial basis with explicit error guarantees. Numerical evidence that our algorithm outperforms the state-of-the-art optimization-based method for computing complementary polynomials is provided.


title: Spectral flow and the conformal block expansion for strings in AdS$_3$

abstract: We present a detailed study of spectrally flowed four-point functions in the SL(2,$\mathbb{R}$) WZW model, focusing on their conformal block decomposition. Dei and Eberhardt conjectured a general formula relating these observables to their unflowed counterparts. Although the latter are not known in closed form, their conformal block expansion has been formally established. By combining this information with the integral transform that encodes the effect of spectral flow, we show how to describe a considerable number of $s$-channel exchanges, including cases with both flowed and unflowed intermediate states. For all such processes, we compute the normalization of the corresponding conformal blocks in terms of products of the recently derived flowed three-point functions with arbitrary spectral flow charges. Our results constitute a highly non-trivial consistency check, thus strongly supporting the aforementioned conjecture, and establishing its computational power.


title: An adaptive parameter estimator for poor-quality spectral data of white dwarfs

abstract: White dwarfs represent the end stage for 97% of stars, making precise parameter measurement crucial for understanding stellar evolution. Traditional estimation methods involve fitting spectra or photometry, which require high-quality data. In recent years, machine learning has played a crucial role in processing spectral data due to its speed, automation, and accuracy. However, two common issues have been identified. First, most studies rely on data with high signal-to-noise ratios (SNR > 10), leaving many poor-quality datasets underutilized. Second, existing machine learning models, primarily based on convolutional networks, recurrent networks, and their variants, cannot simultaneously capture both the spatial and sequential information of spectra. To address these challenges, we designed the Estimator Network (EstNet), an advanced algorithm integrating multiple techniques, including Residual Networks, Squeeze and Excitation Attention, Gated Recurrent Units, Adaptive Loss, and Monte-Carlo Dropout Layers. We conducted parameter estimation on 5,965 poor-quality white dwarf spectra (R~1800, SNR~1.17), achieving average percentage errors of 14.86% for effective temperature and 3.97% for surface gravity. These results are significantly superior to other mainstream algorithms and consistent with the outcomes of traditional theoretical spectrum fitting methods. In the future, our algorithms will be applied for large-scale parameter estimation on the Chinese Space Station Telescope and the Large Synoptic Survey Telescope.


title: Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals

abstract: Implicit neural representation (INR) has recently emerged as a promising paradigm for signal representations. Typically, INR is parameterized by a multiplayer perceptron (MLP) which takes the coordinates as the inputs and generates corresponding attributes of a signal. However, MLP-based INRs face two critical issues: i) individually considering each coordinate while ignoring the connections; ii) suffering from the spectral bias thus failing to learn high-frequency components. While target visual signals usually exhibit strong local structures and neighborhood dependencies, and high-frequency components are significant in these signals, the issues harm the representational capacity of INRs. This paper proposes Conv-INR, the first INR model fully based on convolution. Due to the inherent attributes of convolution, Conv-INR can simultaneously consider adjacent coordinates and learn high-frequency components effectively. Compared to existing MLP-based INRs, Conv-INR has better representational capacity and trainability without requiring primary function expansion. We conduct extensive experiments on four tasks, including image fitting, CT/MRI reconstruction, and novel view synthesis, Conv-INR all significantly surpasses existing MLP-based INRs, validating the effectiveness. Finally, we raise three reparameterization methods that can further enhance the performance of the vanilla Conv-INR without introducing any extra inference cost.


title: Online learning of quantum processes

abstract: Among recent insights into learning quantum states, online learning and shadow tomography procedures are notable for their ability to accurately predict expectation values even of adaptively chosen observables. In contrast to the state case, quantum process learning tasks with a similarly adaptive nature have received little attention. In this work, we investigate online learning tasks for quantum processes. Whereas online learning is infeasible for general quantum channels, we show that channels of bounded gate complexity as well as Pauli channels can be online learned in the regret and mistake-bounded models of online learning. In fact, we can online learn probabilistic mixtures of any exponentially large set of known channels. We also provide a provably sample-efficient shadow tomography procedure for Pauli channels. Our results extend beyond quantum channels to non-Markovian multi-time processes, with favorable regret and mistake bounds, as well as a shadow tomography procedure. We complement our online learning upper bounds with mistake as well as computational lower bounds. On the technical side, we make use of the multiplicative weights update algorithm, classical adaptive data analysis, and Bell sampling, as well as tools from the theory of quantum combs for multi-time quantum processes. Our work initiates a study of online learning for classes of quantum channels and, more generally, non-Markovian quantum processes. Given the importance of online learning for state shadow tomography, this may serve as a step towards quantum channel variants of adaptive shadow tomography.


title: Gaussian Splatting with Localized Points Management

abstract: Point management is a critical component in optimizing 3D Gaussian Splatting (3DGS) models, as the point initiation (e.g., via structure from motion) is distributionally inappropriate. Typically, the Adaptive Density Control (ADC) algorithm is applied, leveraging view-averaged gradient magnitude thresholding for point densification, opacity thresholding for pruning, and regular all-points opacity reset. However, we reveal that this strategy is limited in tackling intricate/special image regions (e.g., transparent) as it is unable to identify all the 3D zones that require point densification, and lacking an appropriate mechanism to handle the ill-conditioned points with negative impacts (occlusion due to false high opacity). To address these limitations, we propose a Localized Point Management (LPM) strategy, capable of identifying those error-contributing zones in the highest demand for both point addition and geometry calibration. Zone identification is achieved by leveraging the underlying multiview geometry constraints, with the guidance of image rendering errors. We apply point densification in the identified zone, whilst resetting the opacity of those points residing in front of these regions so that a new opportunity is created to correct ill-conditioned points. Serving as a versatile plugin, LPM can be seamlessly integrated into existing 3D Gaussian Splatting models. Experimental evaluation across both static 3D and dynamic 4D scenes validate the efficacy of our LPM strategy in boosting a variety of existing 3DGS models both quantitatively and qualitatively. Notably, LPM improves both vanilla 3DGS and SpaceTimeGS to achieve state-of-the-art rendering quality while retaining real-time speeds, outperforming on challenging datasets such as Tanks & Temples and the Neural 3D Video Dataset.


title: Sub-nanometer depth resolution and single dopant visualization achieved by tilt-coupled multislice electron ptychography

abstract: Real-space imaging of three-dimensional atomic structures is a critical yet challenging task in materials science. Although scanning transmission electron microscopy has achieved sub-angstrom lateral resolution through techniques like electron ptychography1,2, depth resolution remains limited to only 2 to 3 nanometers with a single projection setup3,4. Attaining better depth resolution typically necessitates large sample tilt angles and many projections, as seen in atomic electron tomography5,6. Here, we develop a new algorithm based on multislice electron ptychography which couples only a few projections at small tilt angles, but is sufficient to improve the depth resolution by more than threefold to the sub-nanometer scale, and potentially to the atomic level. This technique maintains high resolving power for both light and heavy atoms, and significantly improves the visibility of single dopants. We are thus able to experimentally detect dilute substitutional praseodymium dopants in a brownmillerite oxide, Ca2Co2O5, in three dimensions and observe the accompanying lattice distortion. This technique requires only a moderate level of data acquisition or processing, and can be seamlessly integrated into electron microscopes equipped with conventional components.


title: A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generation

abstract: 3D modeling has long been an important area in computer vision and computer graphics. Recently, thanks to the breakthroughs in neural representations and generative models, we witnessed a rapid development of 3D modeling. 3D human modeling, lying at the core of many real-world applications, such as gaming and animation, has attracted significant attention. Over the past few years, a large body of work on creating 3D human avatars has been introduced, forming a new and abundant knowledge base for 3D human modeling. The scale of the literature makes it difficult for individuals to keep track of all the works. This survey aims to provide a comprehensive overview of these emerging techniques for 3D human avatar modeling, from both reconstruction and generation perspectives. Firstly, we review representative methods for 3D human reconstruction, including methods based on pixel-aligned implicit function, neural radiance field, and 3D Gaussian Splatting, etc. We then summarize representative methods for 3D human generation, especially those using large language models like CLIP, diffusion models, and various 3D representations, which demonstrate state-of-the-art performance. Finally, we discuss our reflection on existing methods and open challenges for 3D human avatar modeling, shedding light on future research.


title: GeoGen: Geometry-Aware Generative Modeling via Signed Distance Functions

abstract: We introduce a new generative approach for synthesizing 3D geometry and images from single-view collections. Most existing approaches predict volumetric density to render multi-view consistent images. By employing volumetric rendering using neural radiance fields, they inherit a key limitation: the generated geometry is noisy and unconstrained, limiting the quality and utility of the output meshes. To address this issue, we propose GeoGen, a new SDF-based 3D generative model trained in an end-to-end manner. Initially, we reinterpret the volumetric density as a Signed Distance Function (SDF). This allows us to introduce useful priors to generate valid meshes. However, those priors prevent the generative model from learning details, limiting the applicability of the method to real-world scenarios. To alleviate that problem, we make the transformation learnable and constrain the rendered depth map to be consistent with the zero-level set of the SDF. Through the lens of adversarial training, we encourage the network to produce higher fidelity details on the output meshes. For evaluation, we introduce a synthetic dataset of human avatars captured from 360-degree camera angles, to overcome the challenges presented by real-world datasets, which often lack 3D consistency and do not cover all camera angles. Our experiments on multiple datasets show that GeoGen produces visually and quantitatively better geometry than the previous generative models based on neural radiance fields.


title: The frequency process in a non-neutral two-type continuous-state branching process with competition and its genealogy

abstract: We consider a population growth model given by a two-type continuous-state branching process with immigration and competition, introduced by Ma. We study the relative frequency of one of the types in the population when the total mass is forced to be constant at a dense set of times. The resulting process is described as the solution to an SDE, which we call the culled frequency process, generalizing the $\Lambda$-asymmetric frequency process introduced by Caballero et al. We obtain conditions for the culled frequency process to have a moment dual and show that it is given by a branching-coalescing continuous-time Markov chain that describes the genealogy of the two-type CBI with competition. Finally, we obtain a large population limit of the culled frequency process, resulting in a deterministic ordinary differential equation (ODE). Two particular cases of the limiting ODE are studied to determine if general two-type branching mechanisms and general Malthusians can lead to the coexistence of the two types in the population.


title: Gradient Boosting for Hierarchical Data in Small Area Estimation

abstract: This paper introduces Mixed Effect Gradient Boosting (MEGB), which combines the strengths of Gradient Boosting with Mixed Effects models to address complex, hierarchical data structures often encountered in statistical analysis. The methodological foundations, including a review of the Mixed Effects model and the Extreme Gradient Boosting method, leading to the introduction of MEGB are shown in detail. It highlights how MEGB can derive area-level mean estimations from unit-level data and calculate Mean Squared Error (MSE) estimates using a nonparametric bootstrap approach. The paper evaluates MEGB's performance through model-based and design-based simulation studies, comparing it against established estimators. The findings indicate that MEGB provides promising area mean estimations and may outperform existing small area estimators in various scenarios. The paper concludes with a discussion on future research directions, highlighting the possibility of extending MEGB's framework to accommodate different types of outcome variables or non-linear area level indicators.


title: Data Measurements for Decentralized Data Markets

abstract: Decentralized data markets can provide more equitable forms of data acquisition for machine learning. However, to realize practical marketplaces, efficient techniques for seller selection need to be developed. We propose and benchmark federated data measurements to allow a data buyer to find sellers with relevant and diverse datasets. Diversity and relevance measures enable a buyer to make relative comparisons between sellers without requiring intermediate brokers and training task-dependent models.


title: Quiver Hecke algebras from Floer homology in Couloumb branches

abstract: Homology theories categorifying quantum group link invariants are known to be governed by the representation theory of quiver Hecke algebras, also called KLRW algebras. Here we show that certain cylindrical KLRW algebras, relevant in particular for cylindrical generalizations of link homology theories, can be realized by Lagrangian Floer homology in multiplicative Coulomb branches. This confirms a homological mirror symmetry prediction of the first author.


title: Topological Stability and Latschev-type Reconstruction Theorems for $\boldsymbol{\mathrm{CAT}(\kappa)}$ Spaces

abstract: We consider the problem of homotopy-type reconstruction of compact shapes $X\subset\mathbb{R}^N$ that are $\mathrm{CAT}(\kappa)$ in the intrinsic length metric. The reconstructed spaces are in the form of Vietoris--Rips complexes computed from a compact sample $S$, Hausdorff--close to the unknown shape $X$. Instead of the Euclidean metric on the sample, our reconstruction technique leverages a path-based metric to compute these complexes. As naturally emerging in the framework of reconstruction, we also study the Gromov--Hausdorff topological stability and finiteness problem for general compact $\mathrm{CAT}(\kappa)$ spaces. Our techniques provide novel sampling conditions alternative to the existing and commonly used techniques using weak feature size and $\mu$--reach. In particular, we introduce a new parameter, called the {\em restricted distortion}, which is a generalization of the well-known global distortion of embedding. We show examples of Euclidean subspaces, for which the known parameters such as the reach, $\mu$--reach and weak features size vanish, whereas the restricted distortion is finite, making our reconstruction results applicable for such spaces.


title: Embedding induced trees in sparse expanding graphs

abstract: Inspired by the network routing literature \cite{aggarwal1996efficient}, we develop what we call a ``Pre-Emptive Greedy Algorithm" to embed bounded degree induced trees in sparse expanders. This generalises a powerful and central result of Friedman and Pippenger to the induced setting. As corollaries we obtain that a sparse random graph contains all bounded degree trees of linear order (whp) and that the induced and size induced Ramsey numbers of bounded degree trees are linear. No such linear bounds were previously known. We also prove a nearly-tight result on induced forests in bounded degree countable expanders. We expect that our new result will find many more applications.


title: Simulating, Fast and Slow: Learning Policies for Black-Box Optimization

abstract: In recent years, solving optimization problems involving black-box simulators has become a point of focus for the machine learning community due to their ubiquity in science and engineering. The simulators describe a forward process $f_{\mathrm{sim}}: (\psi, x) \rightarrow y$ from simulation parameters $\psi$ and input data $x$ to observations $y$, and the goal of the optimization problem is to find parameters $\psi$ that minimize a desired loss function. Sophisticated optimization algorithms typically require gradient information regarding the forward process, $f_{\mathrm{sim}}$, with respect to the parameters $\psi$. However, obtaining gradients from black-box simulators can often be prohibitively expensive or, in some cases, impossible. Furthermore, in many applications, practitioners aim to solve a set of related problems. Thus, starting the optimization ``ab initio", i.e. from scratch, each time might be inefficient if the forward model is expensive to evaluate. To address those challenges, this paper introduces a novel method for solving classes of similar black-box optimization problems by learning an active learning policy that guides a differentiable surrogate's training and uses the surrogate's gradients to optimize the simulation parameters with gradient descent. After training the policy, downstream optimization of problems involving black-box simulators requires up to $\sim$90\% fewer expensive simulator calls compared to baselines such as local surrogate-based approaches, numerical optimization, and Bayesian methods.


title: Near-field Beam Training with Sparse DFT Codebook

abstract: Extremely large-scale array (XL-array) has emerged as one promising technology to improve the spectral efficiency and spatial resolution of future sixth generation (6G) wireless systems.The upsurge in the antenna number antennas renders communication users more likely to be located in the near-field region, which requires a more accurate spherical (instead of planar) wavefront propagation modeling.This inevitably incurs unaffordable beam training overhead when performing a two-dimensional (2D) beam-search in both the angular and range domains.To address this issue, we first introduce a new sparse discrete Fourier transform (DFT) codebook, which exhibits the angular periodicity in the received beam pattern at the user, which motivates us to propose a three-phase beam training scheme.Specifically, in the first phase, we utilize the sparse DFT codebook for beam sweeping in an angular subspace and estimate candidate user angles according to the received beam pattern.Then, a central sub-array is activated to scan specific candidate angles for resolving the issue of angular ambiguity and identity the user angle.In the third phase, the polar-domain codebook is applied in the estimated angle to search the best effective range of the user.Finally, numerical results show that the proposed beam training scheme enabled by sparse DFT codebook achieves 98.67% reduction as compared with the exhaustive-search scheme, yet without compromising rate performance in the high signal-to-ratio (SNR) regime.


title: No real advantage of photon subtraction and displacement in continuous variable measurement device independent quantum key distribution

abstract: We critically analyse the role of single photon subtraction (SPS) and displacement in improving the performance of continuous variable measurement device independent quantum key distribution (CV-MDI-QKD). We consider CV-MDI-QKD with resource states generated by SPS on a displaced two-mode squeezed vacuum state. Optimizing the secret key rate with state parameters reveals that implementing SPS yields no benefits in improving the loss tolerance of CV-MDI-QKD. Additionally, we find that displacement too is not useful in improving the performance of CV-MDI-QKD. While our result is in contradistinction with the widely held belief in the field regarding the utility of SPS and displacement in CV-MDI-QKD, it also calls for a re-examination of the role of non-Gaussian operations in increasing the efficiency of various quantum information processing protocols.


title: MLVU: A Comprehensive Benchmark for Multi-Task Long Video Understanding

abstract: The evaluation of Long Video Understanding (LVU) performance poses an important but challenging research problem. Despite previous efforts, the existing video understanding benchmarks are severely constrained by several issues, especially the insufficient lengths of videos, a lack of diversity in video types and evaluation tasks, and the inappropriateness for evaluating LVU performances. To address the above problems, we propose a new benchmark, called MLVU (Multi-task Long Video Understanding Benchmark), for the comprehensive and in-depth evaluation of LVU. MLVU presents the following critical values: 1) The substantial and flexible extension of video lengths, which enables the benchmark to evaluate LVU performance across a wide range of durations. 2) The inclusion of various video genres, e.g., movies, surveillance footage, egocentric videos, cartoons, game videos, etc., which reflects the models' LVU performances in different scenarios. 3) The development of diversified evaluation tasks, which enables a comprehensive examination of MLLMs' key abilities in long-video understanding. The empirical study with 20 latest MLLMs reveals significant room for improvement in today's technique, as all existing methods struggle with most of the evaluation tasks and exhibit severe performance degradation when handling longer videos. Additionally, it suggests that factors such as context length, image-understanding quality, and the choice of LLM backbone can play critical roles in future advancements. We anticipate that MLVU will advance the research of long video understanding by providing a comprehensive and in-depth analysis of MLLMs.


title: Apsidal Precession in Binary Asteroids

abstract: While the secondary in a binary asteroid plays an important role in the precession of the mutual orbit, this role has not been thoroughly studied. Given the complex spin-orbit coupled dynamics in binary asteroids, we use a numerical approach to study the relationship between the secondary's shape and spin and the apsidal precession rate of the orbit. Using this approach in conjunction with observations of Didymos, we find it is likely that Dimorphos was significantly reshaped as a result of the DART impact, with its new shape more elongated than the pre-impact shape. Finally, we show that non-principal axis rotation of the secondary can lead to a chaotic evolution of the longitude of the periapsis.


title: Matrices over polynomial rings approached by commutative algebra

abstract: The main goal of the paper is the discussion of a deeper interaction between matrix theory over polynomial rings over a field and typical methods of commutative algebra and related algebraic geometry. This is intended in the sense of bringing numerical algebraic invariants into the picture of determinantal ideals, with an emphasis on non-generic ones. In particular, there is a strong focus on square sparse matrices and features of the dual variety to a determinantal hypersurface. Though the overall goal is not exhausted here, one provides several environments where the present treatment has a degree of success.


title: Transformers need glasses! Information over-squashing in language tasks

abstract: We study how information propagates in decoder-only Transformers, which are the architectural backbone of most existing frontier large language models (LLMs). We rely on a theoretical signal propagation analysis -- specifically, we analyse the representations of the last token in the final layer of the Transformer, as this is the representation used for next-token prediction. Our analysis reveals a representational collapse phenomenon: we prove that certain distinct sequences of inputs to the Transformer can yield arbitrarily close representations in the final token. This effect is exacerbated by the low-precision floating-point formats frequently used in modern LLMs. As a result, the model is provably unable to respond to these sequences in different ways -- leading to errors in, e.g., tasks involving counting or copying. Further, we show that decoder-only Transformer language models can lose sensitivity to specific tokens in the input, which relates to the well-known phenomenon of over-squashing in graph neural networks. We provide empirical evidence supporting our claims on contemporary LLMs. Our theory also points to simple solutions towards ameliorating these issues.


title: Open-Endedness is Essential for Artificial Superhuman Intelligence

abstract: In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, ever self-improving AI remains elusive. In this position paper, we argue that the ingredients are now in place to achieve openendedness in AI systems with respect to a human observer. Furthermore, we claim that such open-endedness is an essential property of any artificial superhuman intelligence (ASI). We begin by providing a concrete formal definition of open-endedness through the lens of novelty and learnability. We then illustrate a path towards ASI via open-ended systems built on top of foundation models, capable of making novel, humanrelevant discoveries. We conclude by examining the safety implications of generally-capable openended AI. We expect that open-ended foundation models will prove to be an increasingly fertile and safety-critical area of research in the near future.


title: Beyond Performance Plateaus: A Comprehensive Study on Scalability in Speech Enhancement

abstract: Deep learning-based speech enhancement (SE) models have achieved impressive performance in the past decade. Numerous advanced architectures have been designed to deliver state-of-the-art performance; however, their scalability potential remains unrevealed. Meanwhile, the majority of research focuses on small-sized datasets with restricted diversity, leading to a plateau in performance improvement. In this paper, we aim to provide new insights for addressing the above issues by exploring the scalability of SE models in terms of architectures, model sizes, compute budgets, and dataset sizes. Our investigation involves several popular SE architectures and speech data from different domains. Experiments reveal both similarities and distinctions between the scaling effects in SE and other tasks such as speech recognition. These findings further provide insights into the under-explored SE directions, e.g., larger-scale multi-domain corpora and efficiently scalable architectures.


title: Optimization of state parameters in displacement assisted photon subtracted measurement-device-independent quantum key distribution

abstract: Non-Gaussian operations, in particular, photon subtraction (PS), have been shown to enhance the performance of various quantum information processing tasks including continuous variable measurement device independent quantum key distribution (CV-MDI-QKD). This work investigates the role of non-Gaussian resource states, namely, the photon subtracted two-mode squeezed coherent (PSTMSC) (which include photon subtracted two-mode squeezed vacuum (PSTMSV) as a special case) states in CV-MDI-QKD. To this end, we derive the Wigner characteristic function for the resource states, from which the covariance matrix and, finally, the secret key rate expressions are extracted. The optimization of the state parameters is undertaken to find the most suitable resource states in this family of states. There have been previous studies on the PSTMSV and PSTMSC states in CV-MDI-QKD that make use of PS operation. We evaluate such proposals and find to our surprise that both PSTMSC and PSTMSV resource states underperform as compared to the TMSV state rendering PS operation and displacement undesirable.


title: Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models

abstract: We introduce Buffer of Thoughts (BoT), a novel and versatile thought-augmented reasoning approach for enhancing accuracy, efficiency and robustness of large language models (LLMs). Specifically, we propose meta-buffer to store a series of informative high-level thoughts, namely thought-template, distilled from the problem-solving processes across various tasks. Then for each problem, we retrieve a relevant thought-template and adaptively instantiate it with specific reasoning structures to conduct efficient reasoning. To guarantee the scalability and stability, we further propose buffer-manager to dynamically update the meta-buffer, thus enhancing the capacity of meta-buffer as more tasks are solved. We conduct extensive experiments on 10 challenging reasoning-intensive tasks, and achieve significant performance improvements over previous SOTA methods: 11% on Game of 24, 20% on Geometric Shapes and 51% on Checkmate-in-One. Further analysis demonstrate the superior generalization ability and model robustness of our BoT, while requiring only 12% of the cost of multi-query prompting methods (e.g., tree/graph of thoughts) on average. Notably, we find that our Llama3-8B+BoT has the potential to surpass Llama3-70B model. Our project is available at: https://github.com/YangLing0818/buffer-of-thought-llm


title: Entangling Quantum Memories at Channel Capacity

abstract: Entangling quantum memories, mediated by optical-frequency or microwave channels, at high rates and fidelities is key for linking qubits across short and long ranges. All well-known protocols encode up to one qubit per optical mode, hence entangling one pair of memory qubits per transmitted mode over the channel, with probability $\eta$, the channel's transmissivity. The rate is proportional to $\eta$ ideal Bell states (ebits) per mode. The quantum capacity, $C(\eta) = -\log_2(1-{\eta})$ ebits per mode, which $\approx 1.44\eta$ for high loss, i.e., $\eta \ll 1$, thereby making these schemes near rate-optimal. However, $C(\eta) \to \infty$ as $\eta \to 1$, making the known schemes highly rate-suboptimal for shorter ranges. We show that a cavity-assisted memory-photon interface can be used to entangle matter memories with Gottesman-Kitaev-Preskill (GKP) photonic qudits, which along with dual-homodyne entanglement swaps that retain analog information, enables entangling memories at capacity-approaching rates at low loss. We benefit from loss resilience of GKP qudits, and their ability to encode multiple qubits in one mode. Our memory-photon interface further supports the preparation of needed ancilla GKP qudits. We expect our result to spur research in low-loss high-cooperativity cavity-coupled qubits with high-efficiency optical coupling, and demonstrations of high-rate short-range quantum links.


title: ELFS: Enhancing Label-Free Coreset Selection via Clustering-based Pseudo-Labeling

abstract: High-quality human-annotated data is crucial for modern deep learning pipelines, yet the human annotation process is both costly and time-consuming. Given a constrained human labeling budget, selecting an informative and representative data subset for labeling can significantly reduce human annotation effort. Well-performing state-of-the-art (SOTA) coreset selection methods require ground-truth labels over the whole dataset, failing to reduce the human labeling burden. Meanwhile, SOTA label-free coreset selection methods deliver inferior performance due to poor geometry-based scores. In this paper, we introduce ELFS, a novel label-free coreset selection method. ELFS employs deep clustering to estimate data difficulty scores without ground-truth labels. Furthermore, ELFS uses a simple but effective double-end pruning method to mitigate bias on calculated scores, which further improves the performance on selected coresets. We evaluate ELFS on five vision benchmarks and show that ELFS consistently outperforms SOTA label-free baselines. For instance, at a 90% pruning rate, ELFS surpasses the best-performing baseline by 5.3% on CIFAR10 and 7.1% on CIFAR100. Moreover, ELFS even achieves comparable performance to supervised coreset selection at low pruning rates (e.g., 30% and 50%) on CIFAR10 and ImageNet-1K.


title: Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models

abstract: This work studies the challenge of aligning large language models (LLMs) with offline preference data. We focus on alignment by Reinforcement Learning from Human Feedback (RLHF) in particular. While popular preference optimization methods exhibit good empirical performance in practice, they are not theoretically guaranteed to converge to the optimal policy and can provably fail when the data coverage is sparse by classical offline reinforcement learning (RL) results. On the other hand, a recent line of work has focused on theoretically motivated preference optimization methods with provable guarantees, but these are not computationally efficient for large-scale applications like LLM alignment. To bridge this gap, we propose SPAC, a new offline preference optimization method with self-play, inspired by the on-average pessimism technique from the offline RL literature, to be the first provable and scalable approach to LLM alignment. We both provide theoretical analysis for its convergence under single-policy concentrability for the general function approximation setting and demonstrate its competitive empirical performance for LLM alignment on a 7B Mistral model with Open LLM Leaderboard evaluations.


title: Interfacing Gottesman-Kitaev-Preskill Qubits to Quantum Memories

abstract: Gottesman-Kitaev-Preskill (GKP) states have been demonstrated to pose significant advantages when utilized for fault-tolerant all optical continuous-variable quantum computing as well as for quantum communications links for entanglement distribution. However interfacing these systems to long-lived solid-state quantum memories has remained an open problem. Here we propose an interface between quantum memories and GKP qubit states based on a cavity-mediated controlled displacement gate. We characterize the quality of memory-GKP entanglement as a function of cavity parameters suggesting optimal regimes of operation for high-quality state transfer between either qubit states. We further extend this protocol to demonstrate the creation of GKP cluster states by avoiding the requirement of ancillary optical quadrature-squeezed light. Utilizing post-selected entanglement swapping operations for GKP qubits, we demonstrate the utility of our protocol for high-rate entanglement generation between quantum memories. Extensions and derivatives of our proposal could enable a wide variety of applications by utilizing the operational trade-offs for qubits encoded in memory and in the GKP basis.


title: Generative AI-in-the-loop: Integrating LLMs and GPTs into the Next Generation Networks

abstract: In recent years, machine learning (ML) techniques have created numerous opportunities for intelligent mobile networks and have accelerated the automation of network operations. However, complex network tasks may involve variables and considerations even beyond the capacity of traditional ML algorithms. On the other hand, large language models (LLMs) have recently emerged, demonstrating near-human-level performance in cognitive tasks across various fields. However, they remain prone to hallucinations and often lack common sense in basic tasks. Therefore, they are regarded as assistive tools for humans. In this work, we propose the concept of "generative AI-in-the-loop" and utilize the semantic understanding, context awareness, and reasoning abilities of LLMs to assist humans in handling complex or unforeseen situations in mobile communication networks. We believe that combining LLMs and ML models allows both to leverage their respective capabilities and achieve better results than either model alone. To support this idea, we begin by analyzing the capabilities of LLMs and compare them with traditional ML algorithms. We then explore potential LLM-based applications in line with the requirements of next-generation networks. We further examine the integration of ML and LLMs, discussing how they can be used together in mobile networks. Unlike existing studies, our research emphasizes the fusion of LLMs with traditional ML-driven next-generation networks and serves as a comprehensive refinement of existing surveys. Finally, we provide a case study to enhance ML-based network intrusion detection with synthesized data generated by LLMs. Our case study further demonstrates the advantages of our proposed idea.


title: VideoTetris: Towards Compositional Text-to-Video Generation

abstract: Diffusion models have demonstrated great success in text-to-video (T2V) generation. However, existing methods may face challenges when handling complex (long) video generation scenarios that involve multiple objects or dynamic changes in object numbers. To address these limitations, we propose VideoTetris, a novel framework that enables compositional T2V generation. Specifically, we propose spatio-temporal compositional diffusion to precisely follow complex textual semantics by manipulating and composing the attention maps of denoising networks spatially and temporally. Moreover, we propose an enhanced video data preprocessing to enhance the training data regarding motion dynamics and prompt understanding, equipped with a new reference frame attention mechanism to improve the consistency of auto-regressive video generation. Extensive experiments demonstrate that our VideoTetris achieves impressive qualitative and quantitative results in compositional T2V generation. Code is available at: https://github.com/YangLing0818/VideoTetris


title: Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs by Sampling with People

abstract: Conversational tones -- the manners and attitudes in which speakers communicate -- are essential to effective communication. Amidst the increasing popularization of Large Language Models (LLMs) over recent years, it becomes necessary to characterize the divergences in their conversational tones relative to humans. However, existing investigations of conversational modalities rely on pre-existing taxonomies or text corpora, which suffer from experimenter bias and may not be representative of real-world distributions for the studies' psycholinguistic domains. Inspired by methods from cognitive science, we propose an iterative method for simultaneously eliciting conversational tones and sentences, where participants alternate between two tasks: (1) one participant identifies the tone of a given sentence and (2) a different participant generates a sentence based on that tone. We run 100 iterations of this process with human participants and GPT-4, then obtain a dataset of sentences and frequent conversational tones. In an additional experiment, humans and GPT-4 annotated all sentences with all tones. With data from 1,339 human participants, 33,370 human judgments, and 29,900 GPT-4 queries, we show how our approach can be used to create an interpretable geometric representation of relations between conversational tones in humans and GPT-4. This work demonstrates how combining ideas from machine learning and cognitive science can address challenges in human-computer interactions.


title: Discovering reduced order model equations of many-body quantum systems using genetic programming: a technical report

abstract: In this technical report we present first results of applying Genetic Programming (GP) to obtain equations for constructing reduced order models of quantum systems, with a particular interest in nuclear Density Functional Theory. We employ the reduced basis method to obtain reduced coordinates as the amplitudes of the reduced basis, and use GP to avoid the need of constructing the reduced equations through, for example, a Galerkin projection. The reduced order models constructed through GP show excellent accuracy and speed performance, including extrapolations in the controlling parameters, and show promise as an effective method for emulating computationally demanding calculations in nuclear physics.


title: xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology

abstract: Multiple instance learning (MIL) is an effective and widely used approach for weakly supervised machine learning. In histopathology, MIL models have achieved remarkable success in tasks like tumor detection, biomarker prediction, and outcome prognostication. However, MIL explanation methods are still lagging behind, as they are limited to small bag sizes or disregard instance interactions. We revisit MIL through the lens of explainable AI (XAI) and introduce xMIL, a refined framework with more general assumptions. We demonstrate how to obtain improved MIL explanations using layer-wise relevance propagation (LRP) and conduct extensive evaluation experiments on three toy settings and four real-world histopathology datasets. Our approach consistently outperforms previous explanation attempts with particularly improved faithfulness scores on challenging biomarker prediction tasks. Finally, we showcase how xMIL explanations enable pathologists to extract insights from MIL models, representing a significant advance for knowledge discovery and model debugging in digital histopathology.


title: Total-Duration-Aware Duration Modeling for Text-to-Speech Systems

abstract: Accurate control of the total duration of generated speech by adjusting the speech rate is crucial for various text-to-speech (TTS) applications. However, the impact of adjusting the speech rate on speech quality, such as intelligibility and speaker characteristics, has been underexplored. In this work, we propose a novel total-duration-aware (TDA) duration model for TTS, where phoneme durations are predicted not only from the text input but also from an additional input of the total target duration. We also propose a MaskGIT-based duration model that enhances the diversity and quality of the predicted phoneme durations. Our results demonstrate that the proposed TDA duration models achieve better intelligibility and speaker similarity for various speech rate configurations compared to the baseline models. We also show that the proposed MaskGIT-based model can generate phoneme durations with higher quality and diversity compared to its regression or flow-matching counterparts.


title: A Statistical Characterization of Wireless Channels Conditioned on Side Information

abstract: Statistical prior channel knowledge, such as the wide-sense-stationary-uncorrelated-scattering (WSSUS) property, and additional side information both can be used to enhance physical layer applications in wireless communication. Generally, the wireless channel's strongly fluctuating path phases and WSSUS property characterize the channel by a zero mean and Toeplitz-structured covariance matrices in different domains. In this work, we derive a framework to comprehensively categorize side information based on whether it preserves or abandons these statistical features conditioned on the given side information. To accomplish this, we combine insights from a generic channel model with the representation of wireless channels as probabilistic graphs. Additionally, we exemplify several applications, ranging from channel modeling to estimation and clustering, which demonstrate how the proposed framework can practically enhance physical layer methods utilizing machine learning (ML).


title: Systolic inequalities and the Horowitz-Myers conjecture

abstract: Let $3 \leq n \leq 7$, and let $g$ be a Riemannian metric on $B^2 \times T^{n-2}$ with scalar curvature at least $-n(n-1)$. We establish an inequality relating the systole of the boundary to the infimum of the mean curvature on the boundary. As a consequence, we confirm a conjecture of Horowitz and Myers.


title: What is Dataset Distillation Learning?

abstract: Dataset distillation has emerged as a strategy to overcome the hurdles associated with large datasets by learning a compact set of synthetic data that retains essential information from the original dataset. While distilled data can be used to train high performing models, little is understood about how the information is stored. In this study, we posit and answer three questions about the behavior, representativeness, and point-wise information content of distilled data. We reveal distilled data cannot serve as a substitute for real data during training outside the standard evaluation setting for dataset distillation. Additionally, the distillation process retains high task performance by compressing information related to the early training dynamics of real models. Finally, we provide an framework for interpreting distilled data and reveal that individual distilled data points contain meaningful semantic information. This investigation sheds light on the intricate nature of distilled data, providing a better understanding on how they can be effectively utilized.


title: Robust preparation of ground state phases under noisy imaginary time evolution

abstract: Non-unitary state preparation protocols such as imaginary time evolution (ITE) offer substantial advantages relative to unitary ones, including the ability to prepare certain long-range correlated states more efficiently. Here, we ask whether such protocols are also robust to noise arising due to coupling to the environment. We consider a non-unitary ITE "circuit" subjected to a variety of noise models and investigate whether the resulting steady state remains in the same phase as the target state of the ITE at finite noise strength. Taking the one-dimensional quantum Ising model as a concrete example, we find that the ground state order and associated phase transition persist in the presence of noise, provided the noise does not explicitly break the symmetry that protects the phase transition. That is, the noise must possess the protecting symmetry in a weak (or average) form. Our analysis is facilitated by a mapping to an effective Hamiltonian picture in a doubled Hilbert space. We discuss possible implications of these findings for quantum simulation on noisy quantum hardware.


title: ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptions

abstract: We present ABEX, a novel and effective generative data augmentation methodology for low-resource Natural Language Understanding (NLU) tasks. ABEX is based on ABstract-and-EXpand, a novel paradigm for generating diverse forms of an input document -- we first convert a document into its concise, abstract description and then generate new documents based on expanding the resultant abstraction. To learn the task of expanding abstract descriptions, we first train BART on a large-scale synthetic dataset with abstract-document pairs. Next, to generate abstract descriptions for a document, we propose a simple, controllable, and training-free method based on editing AMR graphs. ABEX brings the best of both worlds: by expanding from abstract representations, it preserves the original semantic properties of the documents, like style and meaning, thereby maintaining alignment with the original label and data distribution. At the same time, the fundamental process of elaborating on abstract descriptions facilitates diverse generations. We demonstrate the effectiveness of ABEX on 4 NLU tasks spanning 12 datasets and 4 low-resource settings. ABEX outperforms all our baselines qualitatively with improvements of 0.04% - 38.8%. Qualitatively, ABEX outperforms all prior methods from literature in terms of context and length diversity.


title: SpectralZoom: Efficient Segmentation with an Adaptive Hyperspectral Camera

abstract: Hyperspectral image segmentation is crucial for many fields such as agriculture, remote sensing, biomedical imaging, battlefield sensing and astronomy. However, the challenge of hyper and multi spectral imaging is its large data footprint. We propose both a novel camera design and a vision transformer-based (ViT) algorithm that alleviate both the captured data footprint and the computational load for hyperspectral segmentation. Our camera is able to adaptively sample image regions or patches at different resolutions, instead of capturing the entire hyperspectral cube at one high resolution. Our segmentation algorithm works in concert with the camera, applying ViT-based segmentation only to adaptively selected patches. We show results both in simulation and on a real hardware platform demonstrating both accurate segmentation results and reduced computational burden.


title: Trials and Tribulations in the Reanalysis of KELT-24 b: a Case Study for the Importance of Stellar Modeling

abstract: We present a new analysis of the KELT-24 system, comprising a well-aligned hot Jupiter, KELT-24~b, and a bright ($V=8.3$), nearby ($d=96.9~\mathrm{pc}$) F-type host star. KELT-24~b was independently discovered by two groups in 2019, with each reporting best-fit stellar parameters that were notably inconsistent. Here, we present three independent analyses of the KELT-24 system, each incorporating a broad range of photometric and spectroscopic data, including eight sectors of TESS photometry and more than 200 new radial velocities (RVs) from MINERVA. Two of these analyses use KELT-24's observed spectral energy distribution (SED) through a direct comparison to stellar evolutionary models, while our third analysis assumes an unknown additional body contributing to the observed broadband photometry and excludes the SED. Ultimately, we find that the models that include the SED are a poor fit to the available data, so we adopt the system parameters derived without it. We also highlight a single transit-like event observed by TESS, deemed likely to be an eclipsing binary bound to KELT-24, that will require follow-up observations to confirm. We discuss the potential of these additional bodies in the KELT-24 system as a possible explanation for the discrepancies between the results of the different modeling approaches, and explore the system for longer-period planets that may be weakly evident in the RV observations. The comprehensive investigations that we present not only increase the fidelity of our understanding of the KELT-24 system, but also serve as a blueprint for future stellar modeling in global analyses of exoplanet systems.


title: What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages

abstract: What can large language models learn? By definition, language models (LM) are distributions over strings. Therefore, an intuitive way of addressing the above question is to formalize it as a matter of learnability of classes of distributions over strings. While prior work in this direction focused on assessing the theoretical limits, in contrast, we seek to understand the empirical learnability. Unlike prior empirical work, we evaluate neural LMs on their home turf-learning probabilistic languages-rather than as classifiers of formal languages. In particular, we investigate the learnability of regular LMs (RLMs) by RNN and Transformer LMs. We empirically test the learnability of RLMs as a function of various complexity parameters of the RLM and the hidden state size of the neural LM. We find that the RLM rank, which corresponds to the size of linear space spanned by the logits of its conditional distributions, and the expected length of sampled strings are strong and significant predictors of learnability for both RNNs and Transformers. Several other predictors also reach significance, but with differing patterns between RNNs and Transformers.


title: Providing High-Performance Execution with a Sequential Contract for Cryptographic Programs

abstract: Constant-time programming is a widely deployed approach to harden cryptographic programs against side channel attacks. However, modern processors violate the underlying assumptions of constant-time policies by speculatively executing unintended paths of the program. In this work, we propose Cassandra, a novel hardware-software mechanism to protect constant-time cryptographic code against speculative control flow based attacks. Cassandra explores the radical design point of disabling the branch predictor and recording-and-replaying sequential control flow of the program. Two key insights that enable our design are that (1) the sequential control flow of a constant-time program is constant over different runs, and (2) cryptographic programs are highly looped and their control flow patterns repeat in a highly compressible way. These insights allow us to perform an offline branch analysis that significantly compresses control flow traces. We add a small component to a typical processor design, the Branch Trace Unit, to store compressed traces and determine fetch redirections according to the sequential model of the program. Moreover, we provide a formal security analysis and prove that our methodology adheres to a strong security contract by design. Despite providing a higher security guarantee, Cassandra counter-intuitively improves performance by 1.77% by eliminating branch misprediction penalties.


title: Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation

abstract: Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. PPI achieves this by combining small amounts of human-labeled data with larger amounts of data labeled by a reasonably accurate -- but potentially biased -- automatic system, in a way that results in tighter confidence intervals for certain parameters of interest (e.g., the mean performance of a language model). In this paper, we propose a method called Stratified Prediction-Powered Inference (StratPPI), in which we show that the basic PPI estimates can be considerably improved by employing simple data stratification strategies. Without making any assumptions on the underlying automatic labeling system or data distribution, we derive an algorithm for computing provably valid confidence intervals for population parameters (such as averages) that is based on stratified sampling. In particular, we show both theoretically and empirically that, with appropriate choices of stratification and sample allocation, our approach can provide substantially tighter confidence intervals than unstratified approaches. Specifically, StratPPI is expected to improve in cases where the performance of the autorater varies across different conditional distributions of the target data.


title: VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval

abstract: Multi-modal retrieval becomes increasingly popular in practice. However, the existing retrievers are mostly text-oriented, which lack the capability to process visual information. Despite the presence of vision-language models like CLIP, the current methods are severely limited in representing the text-only and image-only data. In this work, we present a new embedding model VISTA for universal multi-modal retrieval. Our work brings forth threefold technical contributions. Firstly, we introduce a flexible architecture which extends a powerful text encoder with the image understanding capability by introducing visual token embeddings. Secondly, we develop two data generation strategies, which bring high-quality composed image-text to facilitate the training of the embedding model. Thirdly, we introduce a multi-stage training algorithm, which first aligns the visual token embedding with the text encoder using massive weakly labeled data, and then develops multi-modal representation capability using the generated composed image-text data. In our experiments, VISTA achieves superior performances across a variety of multi-modal retrieval tasks in both zero-shot and supervised settings. Our model, data, and source code are available at https://github.com/FlagOpen/FlagEmbedding.


title: On the Legendrian realisation of parametric families of knots

abstract: We study the natural inclusion of the space of Legendrian embeddings in $(\mathbb{S}^3,\xi_{\operatorname{std}})$ into the space of smooth embeddings from a homotopical viewpoint. T. K\'alm\'an posed in [Kal] the open question of whether for every fixed knot type $\mathcal{K}$ and Legendrian representative $\mathcal{L}$, the homomorphism $\pi_1(\mathcal{L})\to\pi_1(\mathcal{K})$ is surjective. We positively answer this question for infinitely many knot types $\mathcal{K}$ in the three main families (hyperbolic, torus and satellites) and every stabilised Legendrian representative in $(\mathbb{S}^3,\xi_{\operatorname{std}})$. We then show that for every $n\geq 3$, the homomorphisms $\pi_n(\mathcal{L})\to\pi_n(\mathcal{K})$ and $\pi_n(\mathcal{FL})\to\pi_n(\mathcal{K})$ are never surjective for any knot type $\mathcal K$, Legendrian representative $\mathcal L$ or formal Legendrian representative $\mathcal{FL}$. This shows the existence of rigidity at every higher-homotopy level beyond $\pi_3$. For completeness, we also show that surjectivity at the $\pi_2$-level depends on the smooth knot type.


title: Wilson Loops with Lagrangians: large spin OPE and cusp anomalous dimension dictionary

abstract: In the context of planar conformal gauge theory, we study five-point correlation functions between the interaction Lagrangian and four of the lightest single-trace, gauge-invariant scalar primaries. After performing two light-cone OPEs, we express this correlator in terms of the three-point functions between two leading-twist spinning operators and the Lagrangian. For finite values of spin, we compute these structure constants in perturbation theory up to two loops in $\mathcal{N}=4$ Super Yang-Mills theory. Large values of spin are captured by null polygon kinematics, where we use dualities with null polygon Wilson loops as well as factorization properties to bootstrap the universal behavior of the structure constants at all loops. We find explicit maps that relate the Lagrangian structure constants with the leading-twist anomalous dimension. From the large-spin map, we recover the cusp anomalous dimension at strong and weak coupling, including genus-one terms.