title: Cost Aware Best Arm Identification

abstract: In this paper, we study a best arm identification problem with dual objects. In addition to the classic reward, each arm is associated with a cost distribution and the goal is to identify the largest reward arm using the minimum expected cost. We call it \emph{Cost Aware Best Arm Identification} (CABAI), which captures the separation of testing and implementation phases in product development pipelines and models the objective shift between phases, i.e., cost for testing and reward for implementation. We first derive an theoretic lower bound for CABAI and propose an algorithm called $\mathsf{CTAS}$ to match it asymptotically. To reduce the computation of $\mathsf{CTAS}$, we further propose a low-complexity algorithm called CO, based on a square-root rule, which proves optimal in simplified two-armed models and generalizes surprisingly well in numerical experiments. Our results show (i) ignoring the heterogeneous action cost results in sub-optimality in practice, and (ii) low-complexity algorithms deliver near-optimal performance over a wide range of problems.


title: Action potential propagation properties of 4D, 3D and 2D Hodgkin-Huxley type models

abstract: We explore the relationship between sodium (Na$^+$) and potassium (K$^+$) gating variables in the 4-dimensional (4D) Hodgkin-Huxley (HH) electrophysiology model and reduce its complexity by deriving new 3D and 2D models that retain the original model's dynamic properties. The new 3D and 2D models are based on the relationship $h \simeq c - n$ between the gating variables $h$ and $n$ of the 4D HH model, where $c$ is a constant, which suggests an interdependence between the dynamics of Na$^+$ and K$^+$ transmembrane voltage-gated channels. We derive the corresponding cable equations for the three HH-type models and demonstrate that the action potential propagates along the axon at a speed described by $v(R, C_m) = \alpha / (C_m R^{\beta}) = \gamma D^{\beta}$, where $\alpha > 0$, $0 < \beta < 1$, and $\gamma$ are constants independent of the local stimulus intensity, $D$ is the diffusion coefficient of the electric signal along the axon, $C_m$ is the axon transmembrane capacitance, and $R$ is the axon conducting resistivity. The width $w$ of the action potential spikes is inversely related to the resistivity of the axon, with $w = \alpha_2 / R^{\beta_2}$, where $\alpha_2 > 0$ and $\beta_2 > 0$.


title: l1-norm regularized l1-norm best-fit lines

abstract: In this work, we propose an optimization framework for estimating a sparse robust one-dimensional subspace. Our objective is to minimize both the representation error and the penalty, in terms of the l1-norm criterion. Given that the problem is NP-hard, we introduce a linear relaxation-based approach. Additionally, we present a novel fitting procedure, utilizing simple ratios and sorting techniques. The proposed algorithm demonstrates a worst-case time complexity of $O(n^2 m \log n)$ and, in certain instances, achieves global optimality for the sparse robust subspace, thereby exhibiting polynomial time efficiency. Compared to extant methodologies, the proposed algorithm finds the subspace with the lowest discordance, offering a smoother trade-off between sparsity and fit. Its architecture affords scalability, evidenced by a 16-fold improvement in computational speeds for matrices of 2000x2000 over CPU version. Furthermore, this method is distinguished by several advantages, including its independence from initialization and deterministic and replicable procedures. Furthermore, this method is distinguished by several advantages, including its independence from initialization and deterministic and replicable procedures. The real-world example demonstrates the effectiveness of algorithm in achieving meaningful sparsity, underscoring its precise and useful application across various domains.


title: Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs

abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in solving various tasks, yet they often struggle with comprehensively addressing complex and vague problems. Existing approaches, including multi-agent LLM systems, offer solutions to certain challenges but still require manual setup and lack scalability. To address this gap, we propose a novel approach leveraging decomposition to enable LLMs to tackle vague problems effectively. Our approach involves an orchestrating LLM that interacts with users to understand the problem and then decomposes it into tangible sub-problems. Instead of expecting the LLM to solve the entire problem in one go, we train it to ask follow-up questions to gain a deeper understanding of the user's requirements. Once the problem is adequately understood, the orchestrating LLM divides it into smaller, manageable sub-problems. Each sub-problem is then assigned to specialized LLM agents or non-LLM functions for resolution. These agents work in parallel to solve their respective sub-problems, with the orchestrating LLM overseeing the process and compiling the solutions into a comprehensive answer for the user. By adopting this decomposition approach, we alleviate the constraints imposed by token limitations on LLM outputs and empower them to provide nuanced solutions to complex and ambiguous problems. Through our approach, we aim to enable LLMs to think and operate more like humans, breaking down complex problems into manageable parts and collaboratively solving them. This not only enhances the problem-solving capabilities of LLMs but also offers a scalable and efficient method for addressing a wide range of real-world challenges.


title: Quantum linear algebra is all you need for Transformer architectures

abstract: Generative machine learning methods such as large-language models are revolutionizing the creation of text and images. While these models are powerful they also harness a large amount of computational resources. The transformer is a key component in large language models that aims to generate a suitable completion of a given partial sequence. In this work, we investigate transformer architectures under the lens of fault-tolerant quantum computing. The input model is one where trained weight matrices are given as block encodings and we construct the query, key, and value matrices for the transformer. We show how to prepare a block encoding of the self-attention matrix, with a new subroutine for the row-wise application of the softmax function. In addition, we combine quantum subroutines to construct important building blocks in the transformer, the residual connection and layer normalization, and the feed-forward neural network. Our subroutines prepare an amplitude encoding of the transformer output, which can be measured to obtain a prediction. Based on common open-source large-language models, we provide insights into the behavior of important parameters determining the run time of the quantum algorithm. We discuss the potential and challenges for obtaining a quantum advantage.


title: Numerical performance of correlated-k distribution method in atmospheric escape simulation

abstract: Atmospheric escape is crucial to understand the evolution of planets in and out of the Solar system and to interpret atmospheric observations. While hydrodynamic escape simulations have been actively developed incorporating detailed processes such as UV heating, chemical reactions, and radiative cooling, the radiative cooling by molecules has been treated as emission from selected lines or rotational/vibrational bands to reduce its numerical cost. However, ad hoc selections of radiative lines would risk estimating inaccurate cooling rates because important lines or wavelengths for atmospheric cooling depend on emitting conditions such as temperature and optical thickness. In this study, we apply the correlated-k distribution (CKD) method to cooling rate calculations for H$_2$-dominant transonic atmospheres containing H$_2$O or CO as radiative species, to investigate its numerical performance and the importance of considering all lines of the molecules. Our simulations demonstrate that the sum of weak lines, which provides only 1 % of the line emission energy in total at optically thin conditions, can become the primary source of radiative cooling in optically thick regions, especially for H$_2$O-containing atmospheres. Also, in our hydrodynamic simulations, the CKD method with a wavelength resolution of 1000 is found to be effective, allowing the calculation of escape rate and temperature profiles with acceptable numerical cost. Our results show the importance of treating all radiative lines and the usefulness of the CKD method in hydrodynamic escape simulations. It is particularly practical for heavy-element-enriched atmospheres considered in small exoplanets, including super-Earths, without any prior selections for effective lines.


title: A General Bayesian Algorithm for the Autonomous Alignment of Beamlines

abstract: Autonomous methods to align beamlines can decrease the amount of time spent on diagnostics, and also uncover better global optima leading to better beam quality. The alignment of these beamlines is a high-dimensional, expensive-to-sample optimization problem involving the simultaneous treatment of many optical elements with correlated and nonlinear dynamics. Bayesian optimization is a strategy of efficient global optimization that has proved successful in similar regimes in a wide variety of beamline alignment applications, though it has typically been implemented for particular beamlines and optimization tasks. In this paper, we present a basic formulation of Bayesian inference and Gaussian process models as they relate to multiobjective Bayesian optimization, as well as the practical challenges presented by beamline alignment. We show that the same general implementation of Bayesian optimization with special consideration for beamline alignment can quickly learn the dynamics of particular beamlines in an online fashion through hyperparameter fitting with no prior information. We present the implementation of a concise software framework for beamline alignment and test it on four different optimization problems for experiments at x-ray beamlines of the National Synchrotron Light Source II and the Advanced Light Source and an electron beam at the Accelerator Test Facility, along with benchmarking on a simulated digital twin. We discuss new applications of the framework, and the potential for a unified approach to beamline alignment at synchrotron facilities.


title: CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models

abstract: Adversarial misuse, particularly through `jailbreaking' that circumvents a model's safety and ethical protocols, poses a significant challenge for Large Language Models (LLMs). This paper delves into the mechanisms behind such successful attacks, introducing a hypothesis for the safety mechanism of aligned LLMs: intent security recognition followed by response generation. Grounded in this hypothesis, we propose CodeChameleon, a novel jailbreak framework based on personalized encryption tactics. To elude the intent security recognition phase, we reformulate tasks into a code completion format, enabling users to encrypt queries using personalized encryption functions. To guarantee response generation functionality, we embed a decryption function within the instructions, which allows the LLM to decrypt and execute the encrypted queries successfully. We conduct extensive experiments on 7 LLMs, achieving state-of-the-art average Attack Success Rate (ASR). Remarkably, our method achieves an 86.6\% ASR on GPT-4-1106.


title: An Overview of the Development of Stereotactic Body Radiation Therapy

abstract: Stereotactic body radiation therapy (SBRT) refers to focusing high-energy rays in three-dimensional space on the tumor lesion area, reducing the dose received by surrounding normal tissues, which can effectively improve the local control rate of the tumor and reduce the probability of complications. With the comprehensive development of medical imaging, radiation biology and other disciplines, this less-fractional, high-dose radiotherapy method has been increasingly developed and applied in clinical practice. The background, radio-biological basis, key technologies and main equipment of SBRT are discussed, and its future development direction is prospected.


title: Matrix denoising: Bayes-optimal estimators via low-degree polynomials

abstract: We consider the additive version of the matrix denoising problem, where a random symmetric matrix $S$ of size $n$ has to be inferred from the observation of $Y=S+Z$, with $Z$ an independent random matrix modeling a noise. For prior distributions of $S$ and $Z$ that are invariant under conjugation by orthogonal matrices we determine, using results from first and second order free probability theory, the Bayes-optimal (in terms of the mean square error) polynomial estimators of degree at most $D$, asymptotically in $n$, and show that as $D$ increases they converge towards the estimator introduced by Bun, Allez, Bouchaud and Potters in [IEEE Transactions on Information Theory {\bf 62}, 7475 (2016)]. We conjecture that this optimality holds beyond strictly orthogonally invariant priors, and provide partial evidences of this universality phenomenon when $S$ is an arbitrary Wishart matrix and $Z$ is drawn from the Gaussian Orthogonal Ensemble, a case motivated by the related extensive rank matrix factorization problem.

categories: cond-mat.dis-nn cond-mat.stat-mech cs.IT math.IT math.PR math.ST stat.TH


title: Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)

abstract: Real-world autonomous driving (AD) especially urban driving involves many corner cases. The lately released AD simulator CARLA v2 adds 39 common events in the driving scene, and provide more quasi-realistic testbed compared to CARLA v1. It poses new challenge to the community and so far no literature has reported any success on the new scenarios in V2 as existing works mostly have to rely on specific rules for planning yet they cannot cover the more complex cases in CARLA v2. In this work, we take the initiative of directly training a planner and the hope is to handle the corner cases flexibly and effectively, which we believe is also the future of AD. To our best knowledge, we develop the first model-based RL method named Think2Drive for AD, with a world model to learn the transitions of the environment, and then it acts as a neural simulator to train the planner. This paradigm significantly boosts the training efficiency due to the low dimensional state space and parallel computing of tensors in the world model. As a result, Think2Drive is able to run in an expert-level proficiency in CARLA v2 within 3 days of training on a single A6000 GPU, and to our best knowledge, so far there is no reported success (100\% route completion)on CARLA v2. We also propose CornerCase-Repository, a benchmark that supports the evaluation of driving models by scenarios. Additionally, we propose a new and balanced metric to evaluate the performance by route completion, infraction number, and scenario density, so that the driving score could give more information about the actual driving performance.


title: Unveiling Detuning Effects for Heat-Current Control in Quantum Thermal Devices

abstract: Navigating the intricacies of thermal management at the quantum scale is a challenge in the pursuit of advanced nanoscale technologies. To this extent, theoretical frameworks introducing minimal models mirroring the functionality of electronic current amplifiers and transistors, for instance, have been proposed. Different architectures of the subsystems composing a quantum thermal device can be considered, tacitly bringing drawbacks or advantages if properly engineered. This paper extends the prior research on thermotronics, studying a strongly coupled three-subsystem thermal device with a specific emphasis on a third excited level in the control subsystem. Our setup can be employed as a multipurpose device conditioned on the specific choice of internal parameters: heat switch, rectifier, stabilizer, and amplifier. The exploration of the detuned levels unveils a key role in the performance and working regime of the device. We observe a stable and strong amplification effect persisting over broad ranges of temperature. We conclude that considering a three-level system, as the one directly in contact with the control temperature, boosts output currents and the ability to operate our devices as a switch at various temperatures.


title: All-optical polarization scrambler based on polarization beam splitting with amplified fiber ring

abstract: Optical-fiber-based polarization scramblers can reduce the impact of polarization sensitive performance of various optical fiber systems. Here, we propose a simple and efficient polarization scrambler based on an all optical Mach-Zehnder structure by combining polarization beam splitter and amplified fiber ring. To totally decoherence one polarization splitted beam, a fiber ring together with an amplifier are incorporated. The ratio of two orthogonal beams can be controlled by varying the amplification factor, and we observe different evolution trajectories of the output state of polarizations on Poincare sphere. When the amplification factor exceeds a certain threshold, the scrambler system exhibits chaotical behavior. A commercial single wavelength laser with linewidth of 3 MHz is utilized to characterize the scrambling performance. We found that when the sampling rate is 1.6 MSa/s, a scrambling speed up to 2000 krad/s can be obtained for the average degree of polarization being less than 0.1. We also exploit these chaotic polarization fluctuations to generate random binary number, indicating that the proposed technique is a good candidate for random bit generator.


title: Structure-dependent fragmentation risk of rubble-pile asteroids on low-impacts

abstract: Deflection of stray asteroids by low-momentum impacts is an Earth's key defense strategy. Using a proof-of-principle numerical model, we show that even such impacts on rubble pile asteroids (RPAs) may lead to fragmentation because of inherent internal stress chains. Such chains occur in three- and two-dimensional (2D) aggregates and we study this phenomenon in 2D. We first establish the existence of stress chains in our aggregates and then show that the dynamic shear stress and displacements are highest along the chains, increasing significantly the risk of fracturing there. Our simulations suggest that this phenomenon is independent of the aggregate's particle size distribution. We conclude that future studies are needed to quantitatively assess fragmentation risk as a function of stress chain statistics in RPAs.


title: Alternative models for FX: pricing double barrier options in regime-switching L\'evy models with memory

abstract: This paper is a supplement to our recent paper ``Alternative models for FX, arbitrage opportunities and efficient pricing of double barrier options in L\'evy models". We introduce the class of regime-switching L\'evy models with memory, which take into account the evolution of the stochastic parameters in the past. This generalization of the class of L\'evy models modulated by Markov chains is similar in spirit to rough volatility models. It is flexible and suitable for application of the machine-learning tools. We formulate the modification of the numerical method in ``Alternative models for FX, arbitrage opportunities and efficient pricing of double barrier options in L\'evy models", which has the same number of the main time-consuming blocks as the method for Markovian regime-switching models.


title: Inference on the proportion of variance explained in principal component analysis

abstract: Principal component analysis (PCA) is a longstanding and well-studied approach for dimension reduction. It rests upon the assumption that the underlying signal in the data has low rank, and thus can be well-summarized using a small number of dimensions. The output of PCA is typically represented using a scree plot, which displays the proportion of variance explained (PVE) by each principal component. While the PVE is extensively reported in routine data analyses, to the best of our knowledge the notion of inference on the PVE remains unexplored. In this paper, we consider inference on the PVE. We first introduce a new population quantity for the PVE with respect to an unknown matrix mean. Critically, our interest lies in the PVE of the sample principal components (as opposed to unobserved population principal components); thus, the population PVE that we introduce is defined conditional on the sample singular vectors. We show that it is possible to conduct inference, in the sense of confidence intervals, p-values, and point estimates, on this population quantity. Furthermore, we can conduct valid inference on the PVE of a subset of the principal components, even when the subset is selected using a data-driven approach such as the elbow rule. We demonstrate the proposed approach in simulation and in an application to a gene expression dataset.


title: Interpreting Grokked Transformers in Complex Modular Arithmetic

abstract: Grokking has been actively explored to reveal the mystery of delayed generalization. Identifying interpretable algorithms inside the grokked models is a suggestive hint to understanding its mechanism. In this work, beyond the simplest and well-studied modular addition, we observe the internal circuits learned through grokking in complex modular arithmetic via interpretable reverse engineering, which highlights the significant difference in their dynamics: subtraction poses a strong asymmetry on Transformer; multiplication requires cosine-biased components at all the frequencies in a Fourier domain; polynomials often result in the superposition of the patterns from elementary arithmetic, but clear patterns do not emerge in challenging cases; grokking can easily occur even in higher-degree formulas with basic symmetric and alternating expressions. We also introduce the novel progress measure for modular arithmetic; Fourier Frequency Sparsity and Fourier Coefficient Ratio, which not only indicate the late generalization but also characterize distinctive internal representations of grokked models per modular operation. Our empirical analysis emphasizes the importance of holistic evaluation among various combinations.


title: Modeling error correction with Lindblad dynamics and approximate channels

abstract: We analyze the performance of a quantum error correction code subject to physically-motivated noise modeled by a Lindblad master equation. Working within the code-capacity framework, we consider dissipative and coherent single-qubit terms and two-qubit crosstalk, studying how different approximations of the noise capture the performance of the five-qubit code. A composite-channel approximation where every noise term is considered separately, captures the behavior in many physical cases up to considerably-long timescales, and we analyze its eventual failure due to the effect of noncommuting terms. In contrast, we find that single-qubit approximations do not properly capture the error correction dynamics with two-qubit noise, even for short times. A Pauli approximation going beyond a single-qubit channel, is sensitive to the details of the noise, state, and decoder, and succeeds in many cases at short timescales relative to the noise strength, beyond which it fails. We calculate the code pseudo-threshold emerging within this model, and demonstrate how knowledge of the qubit parameters and connectivity can be used to design better decoders. These results shed light on the performance of error correction codes in the presence of realistic noise and can advance the ongoing efforts toward useful quantum error correction.


title: Auto Tuning for OpenMP Dynamic Scheduling applied to FWI

abstract: Because Full Waveform Inversion (FWI) works with a massive amount of data, its execution requires much time and computational resources, being restricted to large-scale computer systems such as supercomputers. Techniques such as FWI adapt well to parallel computing and can be parallelized in shared memory systems using the application programming interface (API) OpenMP. The management of parallel tasks can be performed through loop schedulers contained in OpenMP. The dynamic scheduler stands out for distributing predefined fixed-size chunk sizes to idle processing cores at runtime. It can better adapt to FWI, where data processing can be irregular. However, the relationship between the size of the chunk size and the runtime is unknown. Optimization techniques can employ meta-heuristics to explore the parameter search space, avoiding testing all possible solutions. Here, we propose a strategy to use the Parameter Auto Tuning for Shared Memory Algorithms (PATSMA), with Coupled Simulated Annealing (CSA) as its optimization method, to automatically adjust the chunk size for the dynamic scheduling of wave propagation, one of the most expensive steps in FWI. Since testing each candidate chunk size in the complete FWI is unpractical, our approach consists of running a PATSMA where the objective function is the runtime of the first time iteration of the first seismic shot of the first FWI iteration. The resulting chunk size is then employed in all wave propagations involved in an FWI. We conducted tests to measure the runtime of an FWI using the proposed autotuning, varying the problem size and running on different computational environments, such as supercomputers and cloud computing instances. The results show that applying the proposed autotuning in an FWI reduces its runtime by up to 70.46% compared to standard OpenMP schedulers.


title: Digraphs modulo primitive positive constructability

abstract: This is my dissertation about digraphs ordered by pp-constructability. We study in particular smooth digraphs, i.e., digraphs without sources or sinks, tournaments and semicomplete digraphs, orientations of paths and cycles, digraphs with at most four vertices, and orientations of trees.


title: The maximum sum of the size of all intersections within intersecting families and crossing-intersecting families

abstract: Let $\omega(\mathcal{F})=\sum_{\{A,B\}\subset\mathcal{F}}|A\cap B|$ and $\omega(\mathcal{A},\mathcal{B})=\sum_{(A,B)\in \mathcal{A}\times \mathcal{B}}|A\cap B|$. A family $\mathcal{F}$ is intersecting if $F_1\cap F_2\neq \emptyset$ for any $F_1,F_2\in\mathcal{F}$ and two family $\mathcal{A}$ and $\mathcal{B}$ are crossing-intersecting if $A\cap B\neq \emptyset$ for any $(A,B)\in \mathcal{A}\times\mathcal{B}$. For an intersecting family $\mathcal{F}$, Erd\H{o}s, Ko and Rado determined the upper bound of $|\mathcal{F}|$, consequently yielding an upper bound of $\binom{|\mathcal{F}|}{2}=\sum_{\{A,B\}\subset\mathcal{F}}1$. If we replace $1$ with $|A\cap B|$ in the summation $\sum_{\{A,B\}\subset\mathcal{F}}1$, then this summation transforms into $\omega(\mathcal{F})$. In this paper, for an intersecting family $\mathcal{F}$, we determine the upper bound of $\omega(\mathcal{F})$, which is a generalization of Erd\H{o}s-Ko-Rado Theorem. Further, for crossing-intersecting families $\mathcal{A}$ and $\mathcal{B}$, we determine the upper bound of $\omega(\mathcal{A},\mathcal{B})$.


title: Accelerating Graph Neural Networks on Real Processing-In-Memory Systems

abstract: Graph Neural Networks (GNNs) are emerging ML models to analyze graph-structure data. Graph Neural Network (GNN) execution involves both compute-intensive and memory-intensive kernels, the latter dominates the total time, being significantly bottlenecked by data movement between memory and processors. Processing-In-Memory (PIM) systems can alleviate this data movement bottleneck by placing simple processors near or inside to memory arrays. In this work, we introduce PyGim, an efficient ML framework that accelerates GNNs on real PIM systems. We propose intelligent parallelization techniques for memory-intensive kernels of GNNs tailored for real PIM systems, and develop handy Python API for them. We provide hybrid GNN execution, in which the compute-intensive and memory-intensive kernels are executed in processor-centric and memory-centric computing systems, respectively, to match their algorithmic nature. We extensively evaluate PyGim on a real-world PIM system with 1992 PIM cores using emerging GNN models, and demonstrate that it outperforms its state-of-the-art CPU counterpart on Intel Xeon by on average 3.04x, and achieves higher resource utilization than CPU and GPU systems. Our work provides useful recommendations for software, system and hardware designers. PyGim will be open-sourced to enable the widespread use of PIM systems in GNNs.


title: C-Band Lithium Niobate on Silicon Carbide SAW Resonator with Figure-of-Merit of 124 at 6.5 GHz

abstract: In this work, we demonstrate a C-band shear-horizontal surface acoustic wave (SH-SAW) resonator with high electromechanical coupling (keff2) of 22% and a quality factor (Q) of 565 based on a thin-film lithium niobate (LN) on silicon carbide (SiC) platform, featuring an excellent figure-of-merit (FoM = keff2*Q ) of 124 at 6.5 GHz, the highest FoM reported in this frequency range. The resonator frequency upscaling is achieved through wavelength ($\lambda$) reduction and the use of thin aluminum (Al) electrodes. The LN/SiC waveguide and synchronous resonator design collectively enable effective acoustic energy confinement for a high FoM, even when the normalized thickness of LN approaches a scale of 0.5$\lambda$ to 1$\lambda$. To perform a comprehensive study, we also designed and fabricated five additional resonators, expending the $\lambda$ studied ranging from 480 to 800 nm, in the same 500 nm-thick transferred Y-cut thin-film LN on SiC. The fabricated SH-SAW resonators, operating from 5 to 8 GHz, experimentally demonstrate a keff2 from 20.3% to 22.9% and a Q from 350 to 575, thereby covering the entire C-band with excellent performance.


title: DREsS: Dataset for Rubric-based Essay Scoring on EFL Writing

abstract: Automated essay scoring (AES) is a useful tool in English as a Foreign Language (EFL) writing education, offering real-time essay scores for students and instructors. However, previous AES models were trained on essays and scores irrelevant to the practical scenarios of EFL writing education and usually provided a single holistic score due to the lack of appropriate datasets. In this paper, we release DREsS, a large-scale, standard dataset for rubric-based automated essay scoring. DREsS comprises three sub-datasets: DREsS_New, DREsS_Std., and DREsS_CASE. We collect DREsS_New, a real-classroom dataset with 1.7K essays authored by EFL undergraduate students and scored by English education experts. We also standardize existing rubric-based essay scoring datasets as DREsS_Std. We suggest CASE, a corruption-based augmentation strategy for essays, which generates 20K synthetic samples of DREsS_CASE and improves the baseline results by 45.44%. DREsS will enable further research to provide a more accurate and practical AES system for EFL writing education.


title: Investigating the Robustness of Vision Transformers against Label Noise in Medical Image Classification

abstract: Label noise in medical image classification datasets significantly hampers the training of supervised deep learning methods, undermining their generalizability. The test performance of a model tends to decrease as the label noise rate increases. Over recent years, several methods have been proposed to mitigate the impact of label noise in medical image classification and enhance the robustness of the model. Predominantly, these works have employed CNN-based architectures as the backbone of their classifiers for feature extraction. However, in recent years, Vision Transformer (ViT)-based backbones have replaced CNNs, demonstrating improved performance and a greater ability to learn more generalizable features, especially when the dataset is large. Nevertheless, no prior work has rigorously investigated how transformer-based backbones handle the impact of label noise in medical image classification. In this paper, we investigate the architectural robustness of ViT against label noise and compare it to that of CNNs. We use two medical image classification datasets -- COVID-DU-Ex, and NCT-CRC-HE-100K -- both corrupted by injecting label noise at various rates. Additionally, we show that pretraining is crucial for ensuring ViT's improved robustness against label noise in supervised training.


title: Multichain Taprootized Atomic Swaps: Introducing Untraceability through Zero-Knowledge Proofs

abstract: Taprootized Atomic Swaps is an extension for Atomic Swaps that enables the untraceability of transactions in a particular swap. Based on Schnorr signatures, Taproot technology, and zero-knowledge proofs, the taprootized atomic swaps hide swap transactions between regular payments. We propose several implementation options: single-transaction protocol, multiple-transaction protocol that splits the receiving amount in an untraceable way, and multichain swap protocol. Our proposed approach works with any smart-contract-compatible chain and multiple Taproot-compatible chains. We describe the concrete implementation of the protocol and release the source code publically.


title: Identifying X(3872) molecular state or the tetraquark state in $pp$ collisions with PACIAE 3.0

abstract: In this work the production of exotic hadron $\rm X(3872)$ in $pp$ collisions at $\sqrt{s}=2.76\,\mathrm{TeV}$ is investigated by the parton and hadron cascade model {\footnotesize PACIAE 3.0}. We firstly use the model to generated, respectively, the final partonic state (Quark Phase Space, QPS) and the final hadronic state (Hadron Phase Space, HPS). The $\rm X(3872)$ tetraquark state and molecular state are then investigated with the quantum statistical mechanics inspired dynamically constrained phase-space coalescence model in the QPS and HPS, respectively. The simulated results of their yields, rapidity distributions, and transverse momentum spectra show a signifcant discrepancy between the $\rm X(3872)$ tetraquark state and the molecular state. Each of these observables may serve as a criterion to identify the $\rm X(3872)$ as either the molecular state or the compact tetraquark state.


title: Probing the inverse moment of $B_s$-meson distribution amplitude via $B_s \to \eta_s$ form factors

abstract: We investigate the inverse moment of the $B_s$-meson light-cone distribution amplitude (LCDA), denoted as $\lambda_{B_s}$ and defined within the heavy quark effective theory, through the calculation of $B_s \to \eta_s$ form factors. The presence of the $s$-quark inside the $B_s$-meson dictates a notable departure of approximately $20\%$ in the $\lambda_{B_s}$ value compared to the non-strange case $\lambda_{B_q}$, as computed within the QCD sum rule approach, albeit with significant uncertainty. First, we compute the decay constant of the $\eta_s$-meson utilizing two-point sum rules while retaining finite $s$-quark mass contributions. Next, we constrain the parameter $\lambda_{B_s}$ by calculating $B_s \to \eta_s$ form factors within the light-cone sum rule approach, using $B_s$-meson LCDAs, and leveraging Lattice QCD estimates at zero momentum transfer from the HPQCD collaboration. Our findings yield $\lambda_{B_s}$ = 480 $\pm$ 92 MeV when expressing the $B_s$-meson LCDAs in the Exponential model, consistent with previous QCD sum rule estimate yet exhibiting a 1.5-fold improvement in uncertainty. Furthermore, we compare the form factor predictions, based on the extracted $\lambda_{B_s}$ value, with earlier analyses for other channels such as $B_s \to D_s$ and $B_s \to K$.


title: Dimensional reduction formulae for spectral traces and Casimir energies

abstract: This short letter considers the case of acoustic scattering by several obstacles in $\mathbb{R}^{d+r}$ for $r,d \geq 1$ of the form $\Omega \times \mathbb{R}^r$, where $\Omega$ is a smooth bounded domain in $\mathbb{R}^d$. As a main result a von-Neumann-trace formula for the relative trace is obtained in this setting. As a special case we obtain a dimensional reduction formula for the Casimir energy for the massive and massless scalar fields in this configuration $\Omega \times \mathbb{R}^r$ per unit volume in $\mathbb{R}^r$.


title: Neural Mesh Fusion: Unsupervised 3D Planar Surface Understanding

abstract: This paper presents Neural Mesh Fusion (NMF), an efficient approach for joint optimization of polygon mesh from multi-view image observations and unsupervised 3D planar-surface parsing of the scene. In contrast to implicit neural representations, NMF directly learns to deform surface triangle mesh and generate an embedding for unsupervised 3D planar segmentation through gradient-based optimization directly on the surface mesh. The conducted experiments show that NMF obtains competitive results compared to state-of-the-art multi-view planar reconstruction, while not requiring any ground-truth 3D or planar supervision. Moreover, NMF is significantly more computationally efficient compared to implicit neural rendering-based scene reconstruction approaches.


title: Decoherence Implies Information Gain

abstract: It is shown that if the wave function of a quantum system undergoes an arbitrary random transformation such that the diagonal elements of the density matrix in the basis of a preferred observable remain constant, then (i) the off-diagonal elements of the density matrix become smaller in magnitude, and (ii) the system gains information about the preferred observable in the sense that the uncertainty of the observable reduces. These results are independent of the details of how the system-environment interaction generates the transformation of the state of the system, and together imply that decoherence leads to information gain, not information loss.


title: Less is More Revisited

abstract: Multiparty session types (MPST) provide a type discipline where a programmer or architect specifies a whole view of communications as a global protocol, and each distributed program is locally type-checked against its end-point projection. After 10 years from the birth of MPST, Scalas and Yoshida discovered that the proofs of type safety in the literature which use the end-point projection with mergeability are flawed. After this paper, researchers wrongly believed that the end-point projection (with mergeability) was unsound. We correct this misunderstanding, proposing a new general proof technique for type soundness of multiparty session $\pi$-calculus, which uses an association relation between a global type and its end-point projection.


title: Trapped ion qubit and clock operations with a visible wavelength photonic coil resonator stabilized integrated Brillouin laser

abstract: Integrating precise, stable, ultra-low noise visible light lasers into atomic systems is critical for advancing quantum information sciences and improving scalability and portability. Trapped ions are a leading approach for high-fidelity quantum computing, high-accuracy optical clocks, and precision quantum sensors. However, current ion-based systems rely on bulky, lab-scale precision lasers and optical stabilization cavities for optical clock and qubit operations, constraining the size, weight, scalability, and portability of atomic systems. Chip-scale integration of ultra-low noise lasers and reference cavities operating directly at optical clock transitions and capable of qubit and clock operations will represent a major transformation in atom and trapped ion-based quantum technologies. However, this goal has remained elusive. Here we report the first demonstration of chip-scale optical clock and qubit operations on a trapped ion using a photonic integrated direct-drive visible wavelength Brillouin laser stabilized to an integrated 3-meter coil-resonator reference cavity and the optical clock transition of a $^{88}$Sr$^+$ ion trapped on a surface electrode chip. We also demonstrate for the first time, to the best of our knowledge, trapped-ion spectroscopy and qubit operations such as Rabi oscillations and high fidelity (99%) qubit state preparation and measurement (SPAM) using direct drive integrated photonic technologies without bulk optic stabilization cavities or second harmonic generation. Our chip-scale stabilized Brillouin laser exhibits a 6 kHz linewidth with the 0.4 Hz quadrupole transition of $^{88}$Sr$^+$ and a self-consistent coherence time of 60 $\mu$s via Ramsey interferometry on the trapped ion qubit. Furthermore, we demonstrate the stability of the locked Brillouin laser to 5$\times10^{-13}/ \sqrt{\tau}$ at 1 second using dual optical clocks.


title: Almost gauge-invariant states and the ground state of Yang-Mills theory

abstract: We consider the problem of the explicit description of the gauge-invariant subspace of pure lattice gauge theories in the Hamiltonian formulation, where the gauge group is either a compact Lie group or a finite group. The latter case is particularly interesting for quantum simulation. A basis of states where configurations are grouped according to their holonomies is shown to have several advantages over other descriptions. Using this basis, we compute some properties of interest for some non- Abelian finite groups on small lattices, and in particular we examine the question of whether a certain ansatz introduced long ago is a good approximation for the ground state.


title: Mathematical foundations of spectral methods for time-dependent PDEs

abstract: The contention of this paper is that a spectral method for time-dependent PDEs is basically no more than a choice of an orthonormal basis of the underlying Hilbert space. This choice is governed by a long list of considerations: stability, speed of convergence, geometric numerical integration, fast approximation and efficient linear algebra. We subject different choices of orthonormal bases, focussing on the real line, to these considerations. While nothing is likely to improve upon a Fourier basis in the presence of periodic boundary conditions, the situation is considerably more interesting in other settings. We introduce two kinds of orthonormal bases, T-systems and W-systems, and investigate in detail their features. T-systems are designed to work with Cauchy boundary conditions, while W-systems are suited to zero Dirichlet boundary conditions.


title: On an evolution equation in sub-Finsler geometry

abstract: We study the gradient flow of an energy with mixed homogeneity which is at the interface of Finsler and sub-Riemannian geometry


title: Asymptotic-preserving and energy stable dynamical low-rank approximation for thermal radiative transfer equations

abstract: The thermal radiative transfer equations model temperature evolution through a background medium as a result of radiation. When a large number of particles are absorbed in a short time scale, the dynamics tend to a non-linear diffusion-type equation called the Rosseland approximation. The main challenges for constructing numerical schemes that exhibit the correct limiting behavior are posed by the solution's high-dimensional phase space and multi-scale effects. In this work, we propose an asymptotic-preserving and rank-adaptive dynamical low-rank approximation scheme based on the macro-micro decomposition of the particle density and a modified augmented basis-update \& Galerkin integrator. We show that this scheme, for linear particle emission by the material, dissipates energy over time under a step size restriction that captures the hyperbolic and parabolic CFL conditions. We demonstrate the efficacy of the proposed method in a series of numerical experiments.


title: Recent progress in the physical principles of dynamic ground self-righting

abstract: Animals and robots must self-right on the ground after overturning. Biology research described various strategies and motor patterns in many species. Robotics research devised many strategies. However, we do not well understand how the physical principles of how the need to generate mechanical energy to overcome the potential energy barrier governs behavioral strategies and 3-D body rotations given the morphology. Here I review progress on this which I led studying cockroaches self-righting on level, flat, solid, low-friction ground, by integrating biology experiments, robotic modeling, and physics modeling.


title: Enhancing Hypergradients Estimation: A Study of Preconditioning and Reparameterization

abstract: Bilevel optimization aims to optimize an outer objective function that depends on the solution to an inner optimization problem. It is routinely used in Machine Learning, notably for hyperparameter tuning. The conventional method to compute the so-called hypergradient of the outer problem is to use the Implicit Function Theorem (IFT). As a function of the error of the inner problem resolution, we study the error of the IFT method. We analyze two strategies to reduce this error: preconditioning the IFT formula and reparameterizing the inner problem. We give a detailed account of the impact of these two modifications on the error, highlighting the role played by higher-order derivatives of the functionals at stake. Our theoretical findings explain when super efficiency, namely reaching an error on the hypergradient that depends quadratically on the error on the inner problem, is achievable and compare the two approaches when this is impossible. Numerical evaluations on hyperparameter tuning for regression problems substantiate our theoretical findings.


title: MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model

abstract: With the evolution of storage and communication protocols, ultra-low bitrate image compression has become a highly demanding topic. However, existing compression algorithms must sacrifice either consistency with the ground truth or perceptual quality at ultra-low bitrate. In recent years, the rapid development of the Large Multimodal Model (LMM) has made it possible to balance these two goals. To solve this problem, this paper proposes a method called Multimodal Image Semantic Compression (MISC), which consists of an LMM encoder for extracting the semantic information of the image, a map encoder to locate the region corresponding to the semantic, an image encoder generates an extremely compressed bitstream, and a decoder reconstructs the image based on the above information. Experimental results show that our proposed MISC is suitable for compressing both traditional Natural Sense Images (NSIs) and emerging AI-Generated Images (AIGIs) content. It can achieve optimal consistency and perception results while saving 50% bitrate, which has strong potential applications in the next generation of storage and communication. The code will be released on https://github.com/lcysyzxdxc/MISC.


title: Optical control and coherent coupling of spin diffusive modes in thermal gases

abstract: Collective spins in thermal gases are at the core of a multitude of science and technology applications. In most of them, the random thermal motion of the particles is considered detrimental as it is responsible for decoherence and noise. In conditions of diffusive propagation, thermal atoms can potentially occupy various stable spatial modes in a glass cell. Extended or localized, diffusive modes have different magnetic properties, depending on the boundary conditions of the atomic cell, and can react differently to external perturbations. Here we demonstrate that few of these modes can be selectively excited, manipulated, and interrogated in atomic thermal vapours using laser light. In particular, we individuate the conditions for the generation of modes that are exceptionally resilient to undesirable effects introduced by optical pumping, such as light shifts and power-broadening, which are often the dominant sources of systematic errors in atomic magnetometers and co-magnetometers. Moreover, we show that the presence of spatial inhomogeneity in the pump, on top of the random diffusive atomic motion, introduces a coupling that leads to a coherent exchange of excitation between the two longest-lived modes. Our results indicate that systematic engineering of the multi-mode nature of diffusive gases has great potential for improving the performance of quantum technology applications based on alkali-metal thermal gases, and promote these simple experimental systems as versatile tools for quantum information applications.


title: Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems

abstract: Understanding citizens' values in participatory systems is crucial for citizen-centric policy-making. We envision a hybrid participatory system where participants make choices and provide motivations for those choices, and AI agents estimate their value preferences by interacting with them. We focus on situations where a conflict is detected between participants' choices and motivations, and propose methods for estimating value preferences while addressing detected inconsistencies by interacting with the participants. We operationalize the philosophical stance that "valuing is deliberatively consequential." That is, if a participant's choice is based on a deliberation of value preferences, the value preferences can be observed in the motivation the participant provides for the choice. Thus, we propose and compare value estimation methods that prioritize the values estimated from motivations over the values estimated from choices alone. Then, we introduce a disambiguation strategy that addresses the detected inconsistencies between choices and motivations by directly interacting with the participants. We evaluate the proposed methods on a dataset of a large-scale survey on energy transition. The results show that explicitly addressing inconsistencies between choices and motivations improves the estimation of an individual's value preferences. The disambiguation strategy does not show substantial improvements when compared to similar baselines--however, we discuss how the novelty of the approach can open new research avenues and propose improvements to address the current limitations.


title: Unambiguous preparation of Bell pairs

abstract: The ability of preparing perfect Bell pairs with a practical scheme is of great relevance for quantum communication as well as distributed quantum computing. We propose a scheme which probabilistically, but unambiguously produces the $\left|\Phi_{+}\right>$ Bell pair from four copies of qubit pairs initially in the same arbitrary pure quantum state. The same scheme, extended to eight qubit pairs initially in the same, moderately mixed quantum state, unambiguously produces the $\left|\Phi_{+}\right>$ Bell pair with quadratically suppressed noise. The core step of the proposed scheme consists of a pair of local two-qubit operations applied at each of the two distant locations, followed by a partial projective measurement and postselection at each party, with results communicated classically. While the scheme resembles standard entanglement distillation protocols, it achieves success within just three iterations, making it attractive for real-world applications.


title: Area preserving Combescure transformations

abstract: Motivated by the design of flexible nets, we classify all nets of arbitrary size m x n that admit a continuous family of area-preserving Combescure transformations. There are just two different classes. The nets in the first class are special cases of cone nets that have been recently studied by Kilian, Mueller, and Tervooren. The second class consists of Koenigs nets having a Christoffel dual with the same areas of corresponding faces. We apply isotropic metric duality to get a new class of flexible nets in isotropic geometry. We also study the smooth analogs of the introduced classes.


title: Ambiguity Function Shaping in FMCW Automotive Radar

abstract: Frequency-modulated continuous wave (FMCW) radar with inter-chirp coding produces high side-lobes in the Doppler and range dimensions of the radar's ambiguity function. The high side-lobes may cause miss-detection due to masking between targets that are at similar range and have large received power difference, as is often the case in automotive scenarios. In this paper, we develop a novel code optimization method that attenuates the side-lobes of the radar's ambiguity function. In particular, we introduce a framework for designing radar transmit sequences by shaping the radar Ambiguity Function (AF) to a desired structure. The proposed approach suppresses the average amplitude of the AF of the transmitted signal in regions of interest by efficiently tackling a longstanding optimization problem. The optimization criterion is quartic in nature with respect to the radar transmit code. A cyclic iterative algorithm is introduced that recasts the quartic problem as a unimodular quadratic problem (UQP) which can be tackled using power-method-like iterations (PMLI). Our numerical results demonstrate the effectiveness of the proposed algorithm in designing sequences with desired AF which is of great interest to the future generations of automotive radar sensors.


title: Towards Bridging the Gap between Near and Far-Field Characterizations of the Wireless Channel

abstract: The "near-field" propagation modeling of wireless channels is necessary to support sixth-generation (6G) technologies, such as intelligent reflecting surface (IRS), that are enabled by large aperture antennas and higher frequency carriers. As the conventional far-field model proves inadequate in this context, there is a pressing need to explore and bridge the gap between near and far-field propagation models. Although far-field models are simple and provide computationally efficient solutions for many practical applications, near-field models provide the most accurate representation of wireless channels. This paper builds upon the foundations of electromagnetic wave propagation theory to derive near and far-field models as approximations of the Green's function (Maxwell's equations). We characterize the near and far-field models both theoretically and with the help of simulations in a line-of-sight (LOS)-only scenario. In particular, for two key applications in multiantenna systems, namely, beamforming and multiple-access, we showcase the advantages of using the near-field model over the far-field, and present a novel scheduling scheme for multiple-access in the near-field regime. Our findings offer insights into the challenge of incorporating near-field models in practical wireless systems, fostering enhanced performance in future communication technologies.


title: Exact Jacobi elliptic solutions of the $abcd$-system

abstract: In this manuscript, consideration is given to the existence of periodic traveling-wave solutions to the $abcd$-system. This system was derived by Bona, Saut, and Chen to describe small amplitude, long wavelength gravity waves on the surface of water. These exact solutions are formulated in terms of the Jacobi elliptic function cnoidal. The existence of explicit traveling-wave solutions is very useful in theoretical investigations such as stability of solutions, as well as other numerical analysis of the system.


title: Towards Environmental Preference Based Speech Enhancement For Individualised Multi-Modal Hearing Aids

abstract: Since the advent of Deep Learning (DL), Speech Enhancement (SE) models have performed well under a variety of noise conditions. However, such systems may still introduce sonic artefacts, sound unnatural, and restrict the ability for a user to hear ambient sound which may be of importance. Hearing Aid (HA) users may wish to customise their SE systems to suit their personal preferences and day-to-day lifestyle. In this paper, we introduce a preference learning based SE (PLSE) model for future multi-modal HAs that can contextually exploit audio information to improve listening comfort, based upon the preferences of the user. The proposed system estimates the Signal-to-noise ratio (SNR) as a basic objective speech quality measure which quantifies the relative amount of background noise present in speech, and directly correlates to the intelligibility of the signal. Additionally, to provide contextual information we predict the acoustic scene in which the user is situated. These tasks are achieved via a multi-task DL model, which surpasses the performance of inferring the acoustic scene or SNR separately, by jointly leveraging a shared encoded feature space. These environmental inferences are exploited in a preference elicitation framework, which linearly learns a set of predictive functions to determine the target SNR of an AV (Audio-Visual) SE system. By greatly reducing noise in challenging listening conditions, and by novelly scaling the output of the SE model, we are able to provide HA users with contextually individualised SE. Preliminary results suggest an improvement over the non-individualised baseline model in some participants.


title: Globalization of Partial Actions of Ordered Groupoids on Rings

abstract: We provide a necessary and sufficient condition to the existence of an ordered globalization of a partial ordered action of an ordered groupoid on a ring and we also present criteria to obtain uniqueness. Furthermore, we apply those results to obtain a Morita context and to show that an inverse semigroup partial action has a globalization (unique up to isomorphism) if, and only if, it is unital.


title: The Door and Drawer Reset Mechanisms: Automated Mechanisms for Testing and Data Collection

abstract: Robotic manipulation in human environments is a challenging problem for researchers and industry alike. In particular, opening doors/drawers can be challenging for robots, as the size, shape, actuation and required force is variable. Because of this, it can be difficult to collect large real-world datasets and to benchmark different control algorithms on the same hardware. In this paper we present two automated testbeds, the Door Reset Mechanism (DORM) and Drawer Reset Mechanism (DWRM), for the purpose of real world testing and data collection. These devices are low-cost, are sensorized, operate with customized variable resistance, and come with open source software. Additionally, we provide a dataset of over 600 grasps using the DORM and DWRM. We use this dataset to highlight how much variability can exist even with the same trial on the same hardware. This data can also serve as a source for real-world noise in simulation environments.


title: Integrating Dark Pattern Taxonomies

abstract: The problem of ``Dark Patterns" in user interface/user experience (UI/UX) design has proven a difficult issue to tackle. Malicious and explotitative design has expanded to multiple domains in the past 10 years and which has in turn led to multiple taxonomies attempting to describe them. While these taxonomies holds their own merit, and constitute unique contributions to the literature, their usefulness as separate entities is limited. We believe that in order to make meaningful progress in regulating malicious interface design, we must first form a globally harmonized system (GHS) for the classification and labeling of Dark Patterns. By leaning on network analysis tools and methods, this paper synthesizes existing taxonomies and their elements through as a directed graph. In doing so, the interconnectedness of Dark patterns can be more clearly revealed via community (cluster) detection. Ultimately, we hope that this work can serve as the inspiration for the creation of a glyph-based GHS for the classification of Dark Patterns.


title: Microscopic pathways of transition from low-density to high-density amorphous phase of water

abstract: Much attention has been devoted to understanding the microscopic pathways of phase transition between two equilibrium condensed phases (such as liquids and solids). However, the microscopic pathways between non-equilibrium, non-diffusive amorphous (glassy) phases still remain poorly understood. In this work, we have employed computer simulations, persistence homology (a tool rooted in topological data analysis), and machine learning to probe the microscopic pathway of pressure-induced non-equilibrium transition between the low- and high-density amorphous (LDA and HDA, respectively) ice phases of TIP4P/2005 and ST2 water models. Using persistence homology and machine learning, we introduced a new order parameter that unambiguously identifies the LDA and HDA-like local environments. The system transitions continuously and collectively in the order parameter space via a pre-ordered intermediate phase during the compression of the LDA phase. The local order parameter susceptibilities show a maximum near the transition pressure ($P^*$) -- suggesting maximum structural heterogeneities near $P^*$. The HDA-like clusters are structurally ramified and spatially delocalized inside the LDA phase near the transition pressure. We have further investigated the (geometrical) structures and topologies of the LDA and HDA ices formed via different protocols and also studied the dependence of the microscopic pathway of phase transition in the order parameter space on the protocol followed to prepare the initial LDA phase. Finally, the method adopted here to study the microscopic pathways of transition is not restricted to the system under consideration and provides a robust way of probing phase transition pathways involving any two condensed phases at both equilibrium and out-of-equilibrium conditions.


title: The influence of the vorticity-scalar correlation on mixing

abstract: We investigate the role of the correlation between a scalar quantity and the vorticity in two-dimensional mixing at infinite P\'eclet number. We assess, using a diffusivity independent mixing-norm, the dynamics of both Galerkin-truncated ensembles and freely evolving two-dimensional scalar mixing. Both statistical mechanics and numerical experiments show how the mixing-rate is attenuated when vorticity and scalar are initially correlated. Since the vorticity is shown to be a poorly mixing scalar, the results suggest that, in general, mixing can be enhanced by minimizing the correlation between vorticity and passive scalar.


title: Order from chaos: Interplay of development and learning in recurrent networks of structured neurons

abstract: Behavior can be described as a temporal sequence of actions driven by neural activity. To learn complex sequential patterns in neural networks, memories of past activities need to persist on significantly longer timescales than relaxation times of single-neuron activity. While recurrent networks can produce such long transients, training these networks in a biologically plausible way is challenging. One approach has been reservoir computing, where only weights from a recurrent network to a readout are learned. Other models achieve learning of recurrent synaptic weights using propagated errors. However, their biological plausibility typically suffers from issues with locality, resource allocation or parameter scales and tuning. We suggest that many of these issues can be alleviated by considering dendritic information storage and computation. By applying a fully local, always-on plasticity rule we are able to learn complex sequences in a recurrent network comprised of two populations. Importantly, our model is resource-efficient, enabling the learning of complex sequences using only a small number of neurons. We demonstrate these features in a mock-up of birdsong learning, in which our networks first learn a long, non-Markovian sequence that they can then reproduce robustly despite external disturbances.


title: Debiased LASSO under Poisson-Gauss Model

abstract: Quantifying uncertainty in high-dimensional sparse linear regression is a fundamental task in statistics that arises in various applications. One of the most successful methods for quantifying uncertainty is the debiased LASSO, which has a solid theoretical foundation but is restricted to settings where the noise is purely additive. Motivated by real-world applications, we study the so-called Poisson inverse problem with additive Gaussian noise and propose a debiased LASSO algorithm that only requires $n \gg s\log^2p$ samples, which is optimal up to a logarithmic factor.


title: Oscillations-Aware Frequency Security Assessment via Efficient Worst-Case Frequency Nadir Computation

abstract: Frequency security assessment following major disturbances has long been one of the central tasks in power system operations. The standard approach is to study the center of inertia frequency, an aggregate signal for an entire system, to avoid analyzing the frequency signal at individual buses. However, as the amount of low-inertia renewable resources in a grid increases, the center of inertia frequency is becoming too coarse to provide reliable frequency security assessment. In this paper, we propose an efficient algorithm to determine the worst-case frequency nadir across all buses for bounded power disturbances, as well as identify the power disturbances leading to that severest scenario. The proposed algorithm allows oscillations-aware frequency security assessment without conducting exhaustive simulations and intractable analysis.


title: The Paradox of Industrial Involvement in Engineering Higher Education

abstract: This paper discusses the importance of reflective and socially conscious education in engineering schools, particularly within the EE/CS sector. While most engineering disciplines have historically aligned themselves with the demands of the technology industry, the lack of critical examination of industry practices and their impact on justice, equality, and sustainability is self-evident. Today, the for-profit engineering/technology companies, some of which are among the largest in the world, also shape the narrative of engineering education and research in universities. As engineering graduates form the largest cohorts within STEM disciplines in Western countries, they become future professionals who will work, lead, or even establish companies in this industry. Unfortunately, the curriculum within engineering education often lacks a deep understanding of social realities, an essential component of a comprehensive university education. Here we establish this unusual connection with the industry that has driven engineering higher education for several decades and its obvious negative impacts to society. We analyse this nexus and highlight the need for engineering schools to hold a more critical viewpoint. Given the wealth and power of modern technology companies, particularly in the ICT domain, questioning their techno-solutionism narrative is essential within the institutes of higher education.


title: CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks

abstract: Knowledge-intensive language tasks (KILTs) typically require retrieving relevant documents from trustworthy corpora, e.g., Wikipedia, to produce specific answers. Very recently, a pre-trained generative retrieval model for KILTs, named CorpusBrain, was proposed and reached new state-of-the-art retrieval performance. However, most existing research on KILTs, including CorpusBrain, has predominantly focused on a static document collection, overlooking the dynamic nature of real-world scenarios, where new documents are continuously being incorporated into the source corpus. To address this gap, it is crucial to explore the capability of retrieval models to effectively handle the dynamic retrieval scenario inherent in KILTs. In this work, we first introduce the continual document learning (CDL) task for KILTs and build a novel benchmark dataset named KILT++ based on the original KILT dataset for evaluation. Then, we conduct a comprehensive study over the use of pre-trained CorpusBrain on KILT++. Unlike the promising results in the stationary scenario, CorpusBrain is prone to catastrophic forgetting in the dynamic scenario, hence hampering the retrieval performance. To alleviate this issue, we propose CorpusBrain++, a continual generative pre-training framework. Empirical results demonstrate the significant effectiveness and remarkable efficiency of CorpusBrain++ in comparison to both traditional and generative IR methods.


title: Synthesis, structural and magnetic characterizations of Li$_4$Cu$_{1-x}$Ni$_x$TeO$_6$ ( $x$ = 0, 0.1, 0.2, 0.5, and 1)

abstract: We investigated the effect of Ni doping in a recently proposed quantum spin liquid (QSL) candidate Li$_4$CuTeO$_6$. We performed a comprehensive study on the structural and magnetic properties. We find that the anti-site disorder between Li$^+$ and Cu$^{2+}$ persists until 50\% Ni doping in which Ni and Cu occupy different crystallographic sites. As a result, while Cu sits in both triangular and honeycomb layers in Li$_4$CuTeO$_6$, Ni forms only honeycomb layer in Li$_4$NiTeO$_6$ and Li$_4$Cu$_{0.5}$Ni$_{0.5}$TeO$_6$. Our magnetic susceptibility measurements show that the Weiss temperature decreases from -145.68 K in Li$_4$CuTeO$_6$ to -6.15 K in Li$_4$NiTeO$_6$ as Ni doping increases, and find no hint of magnetic ordering or freezing down to 1.8 K. Our analysis implies the existence of abundant low energy excitations in these materials.


title: Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval

abstract: In video-text retrieval, most existing methods adopt the dual-encoder architecture for fast retrieval, which employs two individual encoders to extract global latent representations for videos and texts. However, they face challenges in capturing fine-grained semantic concepts. In this work, we propose the UNIFY framework, which learns lexicon representations to capture fine-grained semantics and combines the strengths of latent and lexicon representations for video-text retrieval. Specifically, we map videos and texts into a pre-defined lexicon space, where each dimension corresponds to a semantic concept. A two-stage semantics grounding approach is proposed to activate semantically relevant dimensions and suppress irrelevant dimensions. The learned lexicon representations can thus reflect fine-grained semantics of videos and texts. Furthermore, to leverage the complementarity between latent and lexicon representations, we propose a unified learning scheme to facilitate mutual learning via structure sharing and self-distillation. Experimental results show our UNIFY framework largely outperforms previous video-text retrieval methods, with 4.8% and 8.2% Recall@1 improvement on MSR-VTT and DiDeMo respectively.


title: Neural population geometry and optimal coding of tasks with shared latent structure

abstract: Humans and animals can recognize latent structures in their environment and apply this information to efficiently navigate the world. However, it remains unclear what aspects of neural activity contribute to these computational capabilities. Here, we develop an analytical theory linking the geometry of a neural population's activity to the generalization performance of a linear readout on a set of tasks that depend on a common latent structure. We show that four geometric measures of the activity determine performance across tasks. Using this theory, we find that experimentally observed disentangled representations naturally emerge as an optimal solution to the multi-task learning problem. When data is scarce, these optimal neural codes compress less informative latent variables, and when data is abundant, they expand these variables in the state space. We validate our theory using macaque ventral stream recordings. Our results therefore tie population geometry to multi-task learning.

categories: q-bio.NC cond-mat.dis-nn cond-mat.stat-mech cs.LG cs.NE


title: Wisdom and Foolishness of Noisy Matching Markets

abstract: We consider a many-to-one matching market where colleges share true preferences over students but make decisions using only independent noisy rankings. Each student has a true value $v$, but each college $c$ ranks the student according to an independently drawn estimated value $v + X_c$ for $X_c\sim \mathcal{D}.$ We ask a basic question about the resulting stable matching: How noisy is the set of matched students? Two striking effects can occur in large markets (i.e., with a continuum of students and a large number of colleges). When $\mathcal{D}$ is light-tailed, noise is fully attenuated: only the highest-value students are matched. When $\mathcal{D}$ is long-tailed, noise is fully amplified: students are matched uniformly at random. These results hold for any distribution of student preferences over colleges, and extend to when only subsets of colleges agree on true student valuations instead of the entire market. More broadly, our framework provides a tractable approach to analyze implications of imperfect preference formation in large markets.


title: Chern Bands with Higher-Order Van Hove Singularities on Topological Moir\'e Surface States

abstract: In two-dimensional electronic lattices, changes in the topology of the Fermi surface (Lifshitz transitions) lead to Van Hove singularities characterized by a divergence in the electronic density of states. Van Hove singularities can enhance the effect of electronic interactions, providing a platform to explore novel correlated electronic states. In this work, we investigate the emergence of topological Chern bands on the surface of three-dimensional topological insulators, which host higher-order Van Hove singularities that are characterized by the power-law diverging density of states. These singularities can arise from the interplay between a time-reversal breaking Zeeman field induced by proximity to a ferromagnetic insulator and a time-reversal invariant moir\'e potential on the surface electrons, created by quintuple layer misalignment in a family of topological insulators such as Bi$_2$Se$_3$ and Bi$_2$Te$_3$, which host a single surface Dirac fermion. We establish the onset of Chern bands near charge neutrality with Chern numbers $C = \pm 1$ that also possess a manifold of higher-order Van Hove singularities on the moir\'e Brillouin zone valleys controlled by the Zeeman and moir\'e potential energy scales, unveiling a new platform to realize exotic Lifshitz transitions in topological bands. Furthermore, we show that the strong peaks in the density of states in the vicinity of Lifshitz transitions give rise to characteristic features in the low-temperature intrinsic anomalous Hall conductivity, yielding a path to probe Van Hove singularities in Chern bands through anomalous transport measurements.


title: Hofer geometry of $A_3$-configurations

abstract: Let $L_0,L_1,L_2 \subset M$ be exact Lagrangian spheres in a Liouville domain $M$ with $2c_1(M)=0$. If $L_0,L_1,L_2$ are in an $A_3$-configuration, we show that $\mathscr{L}(L_0)$ and $\mathscr{L}(L_2)$ when endowed with the Hofer metric contain quasi-flats of arbitrary dimension. We then strengthen this result to show that $\mathscr{L}(L_0)$ and $\mathscr{L}(L_2)$ both contain quasi-isometric embeddings of $(\mathbb{R}^\infty, d_\infty)$, i.e. infinite-dimensional quasi-flats. We also obtain as a corollary of the proof that $\text{Ham}_c(M)$ contains an infinite-dimensional quasi-flat if $M$ contains an $A_3$-configuration of exact Lagrangian spheres. Additionally, we show the same for the two zero-sections in an $A_2$-plumbing. Along the way, we obtain the same result for $\mathscr{L}(F)$, where $F$ is any fiber in the cotangent bundle of a sphere $S^n$. This gives an alternative proof of a result of Usher for $n \geq 3$ and Feng-Zhang for $n=2$. Lastly, we use our method to show that for a Dehn twist $\tau: M \to M$ along $L_1$ the boundary depth of $HF(\tau^\ell(L_0), L')$ is unbounded in $L' \in \mathscr{L}(L_2)$ for any $\ell \in 2\mathbb{N}_0$.


title: Video-Based Autism Detection with Deep Learning

abstract: Individuals with Autism Spectrum Disorder (ASD) often experience challenges in health, communication, and sensory processing; therefore, early diagnosis is necessary for proper treatment and care. In this work, we consider the problem of detecting or classifying ASD children to aid medical professionals in early diagnosis. We develop a deep learning model that analyzes video clips of children reacting to sensory stimuli, with the intent of capturing key differences in reactions and behavior between ASD and non-ASD participants. Unlike many recent studies in ASD classification with MRI data, which require expensive specialized equipment, our method utilizes a powerful but relatively affordable GPU, a standard computer setup, and a video camera for inference. Results show that our model effectively generalizes and understands key differences in the distinct movements of the children. It is noteworthy that our model exhibits successful classification performance despite the limited amount of data for a deep learning problem and limited temporal information available for learning, even with the motion artifacts.


title: A Comprehensive Evaluation of Quantization Strategies for Large Language Models

abstract: Increasing the number of parameters in large language models (LLMs) usually improves performance in downstream tasks but raises compute and memory costs, making deployment difficult in resource-limited settings. Quantization techniques, which reduce the bits needed for model weights or activations with minimal performance loss, have become popular due to the rise of LLMs. However, most quantization studies use pre-trained LLMs, and the impact of quantization on instruction-tuned LLMs and the relationship between perplexity and benchmark performance of quantized LLMs are not well understood. Evaluation of quantized LLMs is often limited to language modeling and a few classification tasks, leaving their performance on other benchmarks unclear. To address these gaps, we propose a structured evaluation framework consisting of three critical dimensions: (1) knowledge \& capacity, (2) alignment, and (3) efficiency, and conduct extensive experiments across ten diverse benchmarks. Our experimental results indicate that LLMs with 4-bit quantization can retain performance comparable to their non-quantized counterparts, and perplexity can serve as a proxy metric for quantized LLMs on most benchmarks. Furthermore, quantized LLMs with larger parameter scales can outperform smaller LLMs. Despite the memory savings achieved through quantization, it can also slow down the inference speed of LLMs. Consequently, substantial engineering efforts and hardware support are imperative to achieve a balanced optimization of decoding speed and memory consumption in the context of quantized LLMs.


title: On the length of directed paths in digraphs

abstract: Thomass\'{e} conjectured the following strengthening of the well-known Caccetta-Haggkvist Conjecture: any digraph with minimum out-degree $\delta$ and girth $g$ contains a directed path of length $\delta(g-1)$. Bai and Manoussakis gave counterexamples to Thomass\'{e}'s conjecture for every even $g\geq 4$. In this note, we first generalize their counterexamples to show that Thomass\'{e}'s conjecture is false for every $g\geq 4$. We also obtain the positive result that any digraph with minimum out-degree $\delta$ and girth $g$ contains a directed path of $2\delta(1-\frac{1}{g})$. For small $g$ we obtain better bounds, e.g. for $g=3$ we show that oriented graph with minimum out-degree $\delta$ contains a directed path of length $1.5\delta$. Furthermore, we show that each $d$-regular digraph with girth $g$ contains a directed path of length $\Omega(dg/\log d)$. Our results give the first non-trivial bounds for these problems.


title: Approximating Simplet Frequency Distribution for Simplicial Complexes

abstract: Simplets, constituting elementary units within simplicial complexes (SCs), serve as foundational elements for the structural analysis of SCs. Previous efforts have focused on the exact count or approximation of simplet count rather than their frequencies, with the latter being more practical in large-scale SCs. This paper enables simplet frequency analysis of SCs by introducing the Simplet Frequency Distribution (SFD) vector. In addition, we present a bound on the sample complexity required for accurately approximating the SFD vector by any uniform sampling-based algorithm. We also present a simple algorithm for this purpose and justify the theoretical bounds with experiments on some random simplicial complexes.


title: On the Growth of Mistakes in Differentially Private Online Learning: A Lower Bound Perspective

abstract: In this paper, we provide lower bounds for Differentially Private (DP) Online Learning algorithms. Our result shows that, for a broad class of $(\varepsilon,\delta)$-DP online algorithms, for $T$ such that $\log T\leq O(1 / \delta)$, the expected number of mistakes incurred by the algorithm grows as $\Omega(\log \frac{T}{\delta})$. This matches the upper bound obtained by Golowich and Livni (2021) and is in contrast to non-private online learning where the number of mistakes is independent of $T$. To the best of our knowledge, our work is the first result towards settling lower bounds for DP-Online learning and partially addresses the open question in Sanyal and Ramponi (2022).


title: One-loop quantization of Euclidean D3-branes in holographic backgrounds

abstract: In this note we analyze the semi-classical quantization of D3 branes in three different holographic backgrounds in type IIB string theory. The first background is Euclidean AdS$_5$ with $S^1\times S^3$ boundary accompanied with a twist to preserve supersymmetry. We work out the spectrum of fluctuations around the classical D3-brane configuration, compute its one-loop partition function, and match to the non-perturbative correction to the superconformal index of ${\cal N}=4$ SYM. We then study Euclidean D3-branes in the Pilch-Warner geometry dual to the IR Leigh-Strassler fixed point of ${\cal N}=1^*$ with the aim to find non-perturbative corrections to its index. Finally we study Euclidean D3-branes in the non-geometric ${\cal N}=2$ J-fold background which is dual to the gauging of the 3D Gaiotto-Witten SCFT.


title: Quantum error thresholds for gauge-redundant digitizations of lattice field theories

abstract: In the quantum simulation of lattice gauge theories, gauge symmetry can be either fixed or encoded as a redundancy of the Hilbert space. While gauge-fixing reduces the number of qubits, keeping the gauge redundancy can provide code space to mitigate and correct quantum errors by checking and restoring Gauss's law. In this work, we consider the correctable errors for generic finite gauge groups and design the quantum circuits to detect and correct them. We calculate the error thresholds below which the gauge-redundant digitization with Gauss's law error correction has better fidelity than the gauge-fixed digitization. Our results provide guidance for fault-tolerant quantum simulations of lattice gauge theories.


title: Can we distinguish the adiabatic fluctuations and isocurvature fluctuations with pulsar timing arrays?

abstract: Understanding the nature of primordial fluctuations is critical to our comprehension of the Universe's early stages. While these fluctuations are known to be nearly scale-invariant, quasi-adiabatic, and nearly Gaussian on large scales, their behavior at smaller scales remains less well-defined and may offer insights into new physics. Recent observations by the NANOGrav, PPTA, EPTA, and CPTA collaborations suggest the presence of a stochastic gravitational wave background, which, while consistent with the contribution from supermassive black hole binaries, also opens the possibility of probing new physics. This paper explores whether this signal could stem from primordial isocurvature and adiabatic fluctuations. We adopt parameterized spectra for both types of fluctuations to fit the observations from the latest NANOGrav data. Furthermore, we employ Bayesian analysis to assess the distinguishability of these models in light of current PTA sensitivities. Our findings indicate that with the capabilities, PTAs cannot conclusively differentiate between isocurvature and adiabatic fluctuations.


title: Multiplex measures for higher-order networks

abstract: A wide variety of complex systems are characterized by interactions of different types involving varying numbers of units. Multiplex hypergraphs serve as a tool to describe such structures, capturing distinct types of higher-order interactions among a collection of units. In this work, we introduce a comprehensive set of measures to describe structural connectivity patterns in multiplex hypergraphs, considering scales from node and hyperedge levels to the system's mesoscale. We validate our measures with three real-world datasets: scientific co-authorship in physics, movie collaborations, and high school interactions. This validation reveals new collaboration patterns, identifies trends within and across movie subfields, and provides insights into daily interaction dynamics. Our framework aims to offer a more nuanced characterization of real-world systems marked by both multiplex and higher-order interactions.


title: Conditional optimal sets and the quantization coefficients for some uniform distributions

abstract: Bucklew and Wise (1982) showed that the quantization dimension of an absolutely continuous probability measure on a given Euclidean space is constant and equals the Euclidean dimension of the space, and the quantization coefficient exists as a finite positive number. By giving different examples, in this paper, we have shown that the quantization coefficients for absolutely continuous probability measures defined on the same Euclidean space can be different. We have taken uniform distribution as a prototype of an absolutely continuous probability measure. In addition, we have also calculated the conditional optimal sets of $n$-points and the $n$th conditional quantization errors for the uniform distributions in constrained and unconstrained scenarios.


title: A Strong Version of the Hilbert Nullstellensatz for slice regular polynomials in several quaternionic variables

abstract: In this paper we prove a strong version of the Hilbert Nullstellensatz in the ring $\mathbb H[q_1,\ldots,q_n]$ of slice regular polynomials in several quaternionic variables. Our proof deeply depends on a detailed analysis of the common zeros of slice regular polynomials which belong to an ideal in $\mathbb H[q_1,\ldots,q_n]$. This study motivates the introduction of a new notion of algebraic set in the quaternionic setting, which allows us to define a Zariski-type topology on $\mathbb H^n$.


title: CARTE: Pretraining and Transfer for Tabular Learning

abstract: Pretrained deep-learning models are the go-to solution for images or text. However, for tabular data the standard is still to train tree-based models. Indeed, transfer learning on tables hits the challenge of data integration: finding correspondences, correspondences in the entries (entity matching) where different words may denote the same entity, correspondences across columns (schema matching), which may come in different orders, names... We propose a neural architecture that does not need such correspondences. As a result, we can pretrain it on background data that has not been matched. The architecture -- CARTE for Context Aware Representation of Table Entries -- uses a graph representation of tabular (or relational) data to process tables with different columns, string embedding of entries and columns names to model an open vocabulary, and a graph-attentional network to contextualize entries with column names and neighboring entries. An extensive benchmark shows that CARTE facilitates learning, outperforming a solid set of baselines including the best tree-based models. CARTE also enables joint learning across tables with unmatched columns, enhancing a small table with bigger ones. CARTE opens the door to large pretrained models for tabular data.


title: Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models

abstract: Much recent work seeks to evaluate values and opinions in large language models (LLMs) using multiple-choice surveys and questionnaires. Most of this work is motivated by concerns around real-world LLM applications. For example, politically-biased LLMs may subtly influence society when they are used by millions of people. Such real-world concerns, however, stand in stark contrast to the artificiality of current evaluations: real users do not typically ask LLMs survey questions. Motivated by this discrepancy, we challenge the prevailing constrained evaluation paradigm for values and opinions in LLMs and explore more realistic unconstrained evaluations. As a case study, we focus on the popular Political Compass Test (PCT). In a systematic review, we find that most prior work using the PCT forces models to comply with the PCT's multiple-choice format. We show that models give substantively different answers when not forced; that answers change depending on how models are forced; and that answers lack paraphrase robustness. Then, we demonstrate that models give different answers yet again in a more realistic open-ended answer setting. We distill these findings into recommendations and open challenges in evaluating values and opinions in LLMs.


title: Isoperimetric Profiles and Regular Embeddings of locally compact groups

abstract: In this article we extend the notion of $L^p$-measured subgroups couplings; a quantitative asymmetric version of measure equivalence that was introduced by Delabie, Koivisto, Le Ma\^itre and Tessera for finitely generated groups, to the setting of locally compact compactly generated unimodular groups. As an example of these couplings; using ideas from Bader and Rosendal, we prove a "dynamical criteria" for the existence of regular embeddings between amenable locally compact compactly generated unimodular groups, namely the existence of an $L^\infty$-measured subgroup coupling that is coarsely $m$ to $1$. We then proceed to prove that the existence of an $L^p$-measured subgroup that is coarsely $m$ to $1$ implies the monotonicity of the $L^p$-isoperimetric profile. We conclude then that the $L^p$-isoperimetric profile is monotonous under regular embeddings, as well as coarse embeddings, between amenable unimodular locally compact compactly generated groups.


title: Why Transformers Need Adam: A Hessian Perspective

abstract: SGD performs worse than Adam by a significant margin on Transformers, but the reason remains unclear. In this work, we provide an explanation through the lens of Hessian: (i) Transformers are "heterogeneous": the Hessian spectrum across parameter blocks vary dramatically, a phenomenon we call "block heterogeneity"; (ii) Heterogeneity hampers SGD: SGD performs worse than Adam on problems with block heterogeneity. To validate (i) and (ii), we check various Transformers, CNNs, MLPs, and quadratic problems, and find that SGD can perform on par with Adam on problems without block heterogeneity, but performs worse than Adam when the heterogeneity exists. Our initial theoretical analysis indicates that SGD performs worse because it applies one single learning rate to all blocks, which cannot handle the heterogeneity among blocks. This limitation could be ameliorated if we use coordinate-wise learning rates, as designed in Adam.


title: Entanglement-breaking channels are a quantum memory resource

abstract: Entanglement-breaking channels (equivalently, measure-and-prepare channels) are an important class of quantum operations noted for their ability to destroy multipartite spatial quantum correlations. Inspired by this property, they have also been employed in defining notions of "classical memory", under the assumption that such channels effectively act as a classical resource. We show that, in a single-system multi-time scenario, entanglement-breaking channels are still a quantum memory resource: a qudit going through an entanglement-breaking channel cannot be simulated by a classical system of same dimension. We provide explicit examples of memory-based output generation tasks where entanglement-breaking channels outperform classical memories of the same size. Our results imply that entanglement-breaking channels cannot be generally employed to characterize classical memory effects in temporal scenarios without additional assumptions.


title: Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidance

abstract: Transformer-based models have demonstrated considerable potential for source code modeling tasks in software engineering. However, they are limited by their dependence solely on automatic self-attention weight learning mechanisms. Previous studies have shown that these models overemphasize delimiters added by tokenizers (e.g., [CLS], [SEP]), which may lead to overlooking essential information in the original input source code. To address this challenge, we introduce SyntaGuid, a novel approach that utilizes the observation that attention weights tend to be biased towards specific source code syntax tokens and abstract syntax tree (AST) elements in fine-tuned language models when they make correct predictions. SyntaGuid facilitates the guidance of attention-weight learning, leading to improved model performance on various software engineering tasks. We evaluate the effectiveness of SyntaGuid on multiple tasks and demonstrate that it outperforms existing state-of-the-art models in overall performance without requiring additional data. Experimental result shows that SyntaGuid can improve overall performance up to 3.25% and fix up to 28.3% wrong predictions. Our work represents the first attempt to guide the attention of Transformer-based models towards critical source code tokens during fine-tuning, highlighting the potential for enhancing Transformer-based models in software engineering.


title: Gravitational particle production and the Hubble tension

abstract: The effect of gravitational particle production of scalar particles on the total effective cosmic energy density (in the era after photon decoupling till the present) is considered. The effect is significant for heavy particles. It is found that gravitational particle production results in an effective increase in the directly measured value of the Hubble constant $H_0$ while it does not affect the value of the Hubble constant in the calculation of the number density of baryons at present time that is used to calculate recombination redshift. This may explain why the Hubble constant determined by local measurements and non-local measurements (such as CMB) are different. This suggests that gravitational particle production may have a non-negligible impact on the $H_0$ tension.


title: Rate-Optimal Rank Aggregation with Private Pairwise Rankings

abstract: In various real-world scenarios like recommender systems and political surveys, pairwise rankings are commonly collected and utilized for rank aggregation to obtain an overall ranking of items. However, preference rankings can reveal individuals' personal preferences, underscoring the need to protect them before releasing for downstream analysis. In this paper, we address the challenge of preserving privacy while ensuring the utility of rank aggregation based on pairwise rankings generated from the Bradley-Terry-Luce (BTL) model. Using the randomized response mechanism to perturb raw pairwise rankings is a common privacy protection strategy used in practice, but a critical challenge arises because the privatized rankings no longer adhere to the BTL model, resulting in significant bias in downstream rank aggregation tasks. Motivated from this, we propose a debiased randomized response mechanism to protect the raw pairwise rankings, ensuring consistent estimation of true preferences and rankings in downstream rank aggregation. Theoretically, we offer insights into the relationship between overall privacy guarantees and estimation errors from private ranking data, and establish minimax rates for estimation errors. This enables the determination of optimal privacy guarantees that balance consistency in rank aggregation with robust privacy protection. We also investigate convergence rates of expected ranking errors for partial and full ranking recovery, quantifying how privacy protection influences the specification of top-$K$ item sets and complete rankings. Our findings are validated through extensive simulations and a real application.


title: Failures and Successes of Cross-Validation for Early-Stopped Gradient Descent

abstract: We analyze the statistical properties of generalized cross-validation (GCV) and leave-one-out cross-validation (LOOCV) applied to early-stopped gradient descent (GD) in high-dimensional least squares regression. We prove that GCV is generically inconsistent as an estimator of the prediction risk of early-stopped GD, even for a well-specified linear model with isotropic features. In contrast, we show that LOOCV converges uniformly along the GD trajectory to the prediction risk. Our theory requires only mild assumptions on the data distribution and does not require the underlying regression function to be linear. Furthermore, by leveraging the individual LOOCV errors, we construct consistent estimators for the entire prediction error distribution along the GD trajectory and consistent estimators for a wide class of error functionals. This in particular enables the construction of pathwise prediction intervals based on GD iterates that have asymptotically correct nominal coverage conditional on the training data.


title: BV bialgebra structures in Floer theory and string topology

abstract: We derive the notions of BV unital infinitesimal bialgebra and BV Frobenius algebra from the topology of suitable compactifications of moduli spaces of decorated genus 0 curves. We construct these structures respectively on reduced symplectic homology and Rabinowitz Floer homology. As an application, we construct these structures in nonequivariant string topology. We also show how the Lie bialgebra structure in equivariant string topology, and more generally on $S^1$-equivariant symplectic homology, is obtained as a formal consequence.