title: Shaping the distribution of neural responses with interneurons in a recurrent circuit model

abstract: Efficient coding theory posits that sensory circuits transform natural signals into neural representations that maximize information transmission subject to resource constraints. Local interneurons are thought to play an important role in these transformations, shaping patterns of circuit activity to facilitate and direct information flow. However, the relationship between these coordinated, nonlinear, circuit-level transformations and the properties of interneurons (e.g., connectivity, activation functions, response dynamics) remains unknown. Here, we propose a normative computational model that establishes such a relationship. Our model is derived from an optimal transport objective that conceptualizes the circuit's input-response function as transforming the inputs to achieve a target response distribution. The circuit, which is comprised of primary neurons that are recurrently connected to a set of local interneurons, continuously optimizes this objective by dynamically adjusting both the synaptic connections between neurons as well as the interneuron activation functions. In an application motivated by redundancy reduction theory, we demonstrate that when the inputs are natural image statistics and the target distribution is a spherical Gaussian, the circuit learns a nonlinear transformation that significantly reduces statistical dependencies in neural responses. Overall, our results provide a framework in which the distribution of circuit responses is systematically and nonlinearly controlled by adjustment of interneuron connectivity and activation functions.


title: Rethinking Pruning for Backdoor Mitigation: An Optimization Perspective

abstract: Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks, posing concerning threats to their reliable deployment. Recent research reveals that backdoors can be erased from infected DNNs by pruning a specific group of neurons, while how to effectively identify and remove these backdoor-associated neurons remains an open challenge. Most of the existing defense methods rely on defined rules and focus on neuron's local properties, ignoring the exploration and optimization of pruning policies. To address this gap, we propose an Optimized Neuron Pruning (ONP) method combined with Graph Neural Network (GNN) and Reinforcement Learning (RL) to repair backdoor models. Specifically, ONP first models the target DNN as graphs based on neuron connectivity, and then uses GNN-based RL agents to learn graph embeddings and find a suitable pruning policy. To the best of our knowledge, this is the first attempt to employ GNN and RL for optimizing pruning policies in the field of backdoor defense. Experiments show, with a small amount of clean data, ONP can effectively prune the backdoor neurons implanted by a set of backdoor attacks at the cost of negligible performance degradation, achieving a new state-of-the-art performance for backdoor mitigation.


title: Discriminating between Babcock-Leighton-type solar dynamo models by torsional oscillations

abstract: The details of the dynamo process in the Sun are an important aspect of research in solar-terrestrial physics and astrophysics. The surface part of the dynamo can be constrained by direct observations, but the subsurface part lacks direct observational constraints. The torsional oscillations, a small periodic variation of the Sun's rotation with the solar cycle, are thought to result from the Lorentz force of the cyclic magnetic field generated by the dynamo. In this study, we aim to discriminate between three Babcock-Leighton (BL) dynamo models by comparing the zonal acceleration of the three models with the observed one. The property that the poleward and equatorward branches of the torsional oscillations originate from about $\pm 55^\circ$ latitudes with their own migration time periods serves as an effective discriminator that could constrain the configuration of the magnetic field in the convection zone. The toroidal field, comprising poleward and equatorward branches separated at about $\pm 55^\circ$ latitudes can generate the two branches of the torsional oscillations. The alternating acceleration and deceleration bands in time is the other property of the torsional oscillations that discriminate between the dynamo models. To reproduce this property, the phase difference between the radial ($B_{r}$) and toroidal ($B_{\phi}$) components of the magnetic field near the surface should be about $\pi/2$.


title: Bi-directional models of `radically synthetic' differential geometry

abstract: The radically synthetic foundation for smooth geometry formulated in [Law11] postulates a space T with the property that it has a unique point and, out of the monoid T^T of endomorphisms, it extracts a submonoid R which, in many cases, is the (commutative) multiplication of a rig structure. The rig R is said to be bi-directional if its subobject of invertible elements has two connected components. In this case, R may be equipped with a pre-order compatible with the rig structure. We adjust the construction of `well-adapted' models of Synthetic Differential Geometry in order to build the first pre-cohesive toposes with a bi-directional R. We also show that, in one of these pre-cohesive variants, the pre-order on R, derived radically synthetically from bi-directionality, coincides with that defined in the original model.


title: Pseudo-Hermitian Topology of Multiband Non-Hermitian Systems

abstract: The complex eigenenergies and non-orthogonal eigenstates of non-Hermitian systems exhibit unique topological phenomena that cannot appear in Hermitian systems. Representative examples are the non-Hermitian skin effect and exceptional points. In a two-dimensional parameter space, topological classifications of non-separable bands in multiband non-Hermitian systems can be established by invoking a permutation group, where the product of the permutation represents state exchange due to exceptional points in the space. We unveil in this work the role of pseudo-Hermitian lines in non-Hermitian topology for multiple bands. Contrary to current understanding, the non-separability of non-Hermitian multibands can be topologically non-trivial without exceptional points in two-dimensional space. Our work builds on the fundamental and comprehensive understanding of non-Hermitian multiband systems and also offers versatile applications and realizations of non-Hermitian systems without the need to consider exceptional points.


title: Magnitude-based Neuron Pruning for Backdoor Defens

abstract: Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks, posing concerning threats to their reliable deployment. Recent research reveals that backdoors can be erased from infected DNNs by pruning a specific group of neurons, while how to effectively identify and remove these backdoor-associated neurons remains an open challenge. In this paper, we investigate the correlation between backdoor behavior and neuron magnitude, and find that backdoor neurons deviate from the magnitude-saliency correlation of the model. The deviation inspires us to propose a Magnitude-based Neuron Pruning (MNP) method to detect and prune backdoor neurons. Specifically, MNP uses three magnitude-guided objective functions to manipulate the magnitude-saliency correlation of backdoor neurons, thus achieving the purpose of exposing backdoor behavior, eliminating backdoor neurons and preserving clean neurons, respectively. Experiments show our pruning strategy achieves state-of-the-art backdoor defense performance against a variety of backdoor attacks with a limited amount of clean data, demonstrating the crucial role of magnitude for guiding backdoor defenses.


title: Molecular Structure and Rotational Dynamics in the Acetonitrile:Acetylene (1:2) Plastic Co-Crystal at Titan Conditions

abstract: The surface of Saturn's moon Titan is coated with small molecule organic solids termed cryominerals. Cryominerals play an analogous role to minerals on Earth in Titan's surface geology and geochemistry. To develop a predictive understanding of Titan's surface geochemistry, we need to characterize the structure and dynamics of cryominerals at the molecular scale. We use ab initio molecular dynamics simulations to quantify the structure and dynamics of the acetonitrile:acetylene (1:2) co-crystal at Titan surface conditions. We suggest that acetonitrile:acetylene is in a plastic phase, in which acetonitrile molecules are dynamically disordered about the N-C-C axis on sub-picosecond timescales, and that this rotational, plastic disorder persists at least to temperatures of 30 K. We anticipate that many cryominerals may have plastic phases at or near Titan surface conditions, and understanding this disorder will be crucial to predicting chemistry on Titan's surface.


title: Isotope effects and Alfven eigenmode stability in JET H, D, T, DT, and He plasmas

abstract: While much about Alfven eigenmode (AE) stability has been explored in previous and current tokamaks, open questions remain for future burning plasma experiments, especially regarding exact stability threshold conditions and related isotope effects; the latter, of course, requiring good knowledge of the plasma ion composition. In the JET tokamak, eight in-vessel antennas actively excite stable AEs, from which their frequencies, toroidal mode numbers, and net damping rates are assessed. The effective ion mass can also be inferred using measurements of the plasma density and magnetic geometry. Thousands of AE stability measurements have been collected by the Alfven Eigenmode Active Diagnostic in hundreds of JET plasmas during the recent Hydrogen, Deuterium, Tritium, DT, and Helium-4 campaigns. In this novel AE stability database, spanning all four main ion species, damping is observed to decrease with increasing Hydrogenic mass, but increase for Helium, a trend consistent with radiative damping as the dominant damping mechanism. These data are important for confident predictions of AE stability in both non-nuclear (H/He) and nuclear (D/T) operations in future devices. In particular, if radiative damping plays a significant role in overall stability, some AEs could be more easily destabilized in D/T plasmas than their H/He reference pulses, even before considering fast ion and alpha particle drive. Active MHD spectroscopy is also employed on select HD, HT, and DT plasmas to infer the effective ion mass, thereby closing the loop on isotope analysis and demonstrating a complementary method to typical diagnosis of the isotope ratio.


title: Regression Equilibrium in Electricity Markets

abstract: Renewable power producers participating in electricity markets build forecasting models independently, relying on their own data, model and feature preferences. In this paper, we argue that in renewable-dominated markets, such an uncoordinated approach to forecasting results in substantial opportunity costs for stochastic producers and additional operating costs for the power system. As a solution, we introduce Regression Equilibrium--a welfare-optimal state of electricity markets under uncertainty, where profit-seeking stochastic producers do not benefit by unilaterally deviating from their equilibrium forecast models. While the regression equilibrium maximizes the private welfare, i.e., the average profit of stochastic producers across the day-ahead and real-time markets, it also aligns with the socially optimal, least-cost dispatch solution for the system. We base the equilibrium analysis on the theory of variational inequalities, providing results on the existence and uniqueness of regression equilibrium in energy-only markets. We also devise two methods for computing the regression equilibrium: centralized optimization and a decentralized ADMM-based algorithm that preserves the privacy of regression datasets.


title: Differential Voltage Analysis and Patterns in Parallel-Connected Pairs of Imbalanced Cells

abstract: Diagnosing imbalances in capacity and resistance within parallel-connected cells in battery packs is critical for battery management and fault detection, but it is challenging given that individual currents flowing into each cell are often unmeasured. This work introduces a novel method useful for identifying imbalances in capacity and resistance within a pair of parallel-connected cells using only voltage and current measurements from the pair. Our method utilizes differential voltage analysis (DVA) when the pair is under constant current discharge and demonstrates that features of the pair's differential voltage curve (dV/dQ), namely its mid-to-high SOC dV/dQ peak's height and skewness, are sensitive to imbalances in capacity and resistance. We analyze and explain how and why these dV/dQ peak shape features change in response to these imbalances, highlighting that the underlying current imbalance dynamics resulting from these imbalances contribute to these changes. Ultimately, we demonstrate that dV/dQ peak shape features can identify the product of capacity imbalance and resistance imbalance, but cannot uniquely identify the imbalances. This work lays the groundwork for identifying imbalances in capacity and resistance in parallel-connected cell groups in battery packs, where commonly only a single current sensor is placed for each parallel cell group.


title: XL3M: A Training-free Framework for LLM Length Extension Based on Segment-wise Inference

abstract: Length generalization failure problem, namely the large language model (LLM) fails to generalize to texts longer than its maximum training length, greatly restricts the application of LLM in the scenarios with streaming long inputs. To address this problem, the existing methods either require substantial costs or introduce precision loss. In this paper, we empirically find that the accuracy of the LLM's prediction is highly correlated to its certainty. Based on this, we propose an efficient training free framework, named XL3M (it means extra-long large language model), which enables the LLMs trained on short sequences to reason extremely long sequence without any further training or fine-tuning. Under the XL3M framework, the input context will be firstly decomposed into multiple short sub-contexts, where each sub-context contains an independent segment and a common ``question'' which is a few tokens from the end of the original context. Then XL3M gives a method to measure the relevance between each segment and the ``question'', and constructs a concise key context by splicing all the relevant segments in chronological order. The key context is further used instead of the original context to complete the inference task. Evaluations on comprehensive benchmarks show the superiority of XL3M. Using our framework, a Llama2-7B model is able to reason 20M long sequences on an 8-card Huawei Ascend 910B NPU machine with 64GB memory per card.


title: Motion-Informed Deep Learning for Brain MR Image Reconstruction Framework

abstract: Motion artifacts in Magnetic Resonance Imaging (MRI) are one of the frequently occurring artifacts due to patient movements during scanning. Motion is estimated to be present in approximately 30% of clinical MRI scans; however, motion has not been explicitly modeled within deep learning image reconstruction models. Deep learning (DL) algorithms have been demonstrated to be effective for both the image reconstruction task and the motion correction task, but the two tasks are considered separately. The image reconstruction task involves removing undersampling artifacts such as noise and aliasing artifacts, whereas motion correction involves removing artifacts including blurring, ghosting, and ringing. In this work, we propose a novel method to simultaneously accelerate imaging and correct motion. This is achieved by integrating a motion module into the deep learning-based MRI reconstruction process, enabling real-time detection and correction of motion. We model motion as a tightly integrated auxiliary layer in the deep learning model during training, making the deep learning model 'motion-informed'. During inference, image reconstruction is performed from undersampled raw k-space data using a trained motion-informed DL model. Experimental results demonstrate that the proposed motion-informed deep learning image reconstruction network outperformed the conventional image reconstruction network for motion-degraded MRI datasets.


title: NASPrecision: Neural Architecture Search-Driven Multi-Stage Learning for Surface Roughness Prediction in Ultra-Precision Machining

abstract: Accurate surface roughness prediction is critical for ensuring high product quality, especially in areas like manufacturing and aerospace, where the smallest imperfections can compromise performance or safety. However, this is challenging due to complex, non-linear interactions among variables, which is further exacerbated with limited and imbalanced datasets. Existing methods using traditional machine learning algorithms require extensive domain knowledge for feature engineering and substantial human intervention for model selection. To address these issues, we propose NASPrecision, a Neural Architecture Search (NAS)-Driven Multi-Stage Learning Framework. This innovative approach autonomously identifies the most suitable features and models for various surface roughness prediction tasks and significantly enhances the performance by multi-stage learning. Our framework operates in three stages: 1) architecture search stage, employing NAS to automatically identify the most effective model architecture; 2) initial training stage, where we train the neural network for initial predictions; 3) refinement stage, where a subsequent model is appended to refine and capture subtle variations overlooked by the initial training stage. In light of limited and imbalanced datasets, we adopt a generative data augmentation technique to balance and generate new data by learning the underlying data distribution. We conducted experiments on three distinct real-world datasets linked to different machining techniques. Results show improvements in Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and Standard Deviation (STD) by 18%, 31%, and 22%, respectively. This establishes it as a robust and general solution for precise surface roughness prediction, potentially boosting production efficiency and product quality in key industries while minimizing domain expertise and human intervention.


title: Charge transport through the multiple end zigzag edge states of armchair graphene nanoribbons and heterojunctions

abstract: This comprehensive study investigates charge transport through the multiple end zigzag edge states of finite-size armchair graphene nanoribbons/boron nitride nanoribbons (n-AGNR/w-BNNR) junctions under a longitudinal electric field, where n and w denote the widths of the AGNRs and the BNNRs, respectively. In 13-atom wide AGNR segments, the edge states exhibit a blue Stark shift in response to the electric field, with only the long decay length zigzag edge states showing significant interaction with the red Stark shift subband states. Charge tunneling through such edge states assisted by the subband states is elucidated in the spectra of the transmission coefficient. In the 13-AGNR/6-BNNR heterojunction, notable influences on the energy levels of the end zigzag edge states of 13-AGNRs induced by BNNR segments are observed. We demonstrate the modulation of these energy levels in resonant tunneling situations, as depicted by bias-dependent transmission coefficient spectra. Intriguing nonthermal broadening of tunneling current shows a significant peak-to-valley ratio. Our findings highlight the promising potential of n-AGNR/w-BNNR heterojunctions with long decay length edge states in the realm of GNR-based single electron transistors at room temperature.


title: Wireless Federated Learning over Resource-Constrained Networks: Digital versus Analog Transmissions

abstract: To enable wireless federated learning (FL) in communication resource-constrained networks, two communication schemes, i.e., digital and analog ones, are effective solutions. In this paper, we quantitatively compare these two techniques, highlighting their essential differences as well as respectively suitable scenarios. We first examine both digital and analog transmission schemes, together with a unified and fair comparison framework under imbalanced device sampling, strict latency targets, and transmit power constraints. A universal convergence analysis under various imperfections is established for evaluating the performance of FL over wireless networks. These analytical results reveal that the fundamental difference between the digital and analog communications lies in whether communication and computation are jointly designed or not. The digital scheme decouples the communication design from FL computing tasks, making it difficult to support uplink transmission from massive devices with limited bandwidth and hence the performance is mainly communication-limited. In contrast, the analog communication allows over-the-air computation (AirComp) and achieves better spectrum utilization. However, the computation-oriented analog transmission reduces power efficiency, and its performance is sensitive to computation errors from imperfect channel state information (CSI). Furthermore, device sampling for both schemes are optimized and differences in sampling optimization are analyzed. Numerical results verify the theoretical analysis and affirm the superior performance of the sampling optimization.


title: Implications of neutrino species number and summed mass measurements in cosmological observations

abstract: We confront measurable neutrino degrees of freedom $N_{\rm eff}$ and summed neutrino mass in the early universe to particle physics at the energy scale beyond the standard model (BSM), in particular including the issue of neutrino mass type distinction. The Majorana-type of massive neutrino is perfectly acceptable by Planck observations, while the Dirac-type neutrino may survive in a restricted class of models that suppresses extra right-handed contribution to $\Delta N_{\rm eff} = N_{\rm eff} - 3$ at a nearly indistinguishable level from the Majorana case. There is a chance that supersymmetry energy scale may be identified in supersymmetric extension of left-right symmetric model if improved $N_{\rm eff}$ measurements discover a finite value. Combined analysis of this quantity with the summed neutrino mass helps to determine the neutrino mass ordering pattern, if measurement accuracy of order, $60 \sim 80\,$meV, is achieved, as in CMB-S4.


title: Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient

abstract: Variance reduction techniques are designed to decrease the sampling variance, thereby accelerating convergence rates of first-order (FO) and zeroth-order (ZO) optimization methods. However, in composite optimization problems, ZO methods encounter an additional variance called the coordinate-wise variance, which stems from the random gradient estimation. To reduce this variance, prior works require estimating all partial derivatives, essentially approximating FO information. This approach demands O(d) function evaluations (d is the dimension size), which incurs substantial computational costs and is prohibitive in high-dimensional scenarios. This paper proposes the Zeroth-order Proximal Double Variance Reduction (ZPDVR) method, which utilizes the averaging trick to reduce both sampling and coordinate-wise variances. Compared to prior methods, ZPDVR relies solely on random gradient estimates, calls the stochastic zeroth-order oracle (SZO) in expectation $\mathcal{O}(1)$ times per iteration, and achieves the optimal $\mathcal{O}(d(n + \kappa)\log (\frac{1}{\epsilon}))$ SZO query complexity in the strongly convex and smooth setting, where $\kappa$ represents the condition number and $\epsilon$ is the desired accuracy. Empirical results validate ZPDVR's linear convergence and demonstrate its superior performance over other related methods.


title: Tuition too high? Blame competition

abstract: We develop a feedback theory that includes reinforcing and balancing feedback effects that emerge when colleges compete for reputation, applicants, and tuition revenue. The feedback theory is replicated in a formal duopoly model consisting of two competing colleges. An independent ranking entity determines the relative order of the colleges. College applicants choose between the two colleges based on the rankings and the financial aid offered by the colleges. Contrary to the conventional wisdom that competition lowers prices and benefits consumers, our simulations show that competition between academic institutions for resources and reputation leads to tuition escalation that negatively affects students and their families. Four of the five scenarios -- rankings, a capital campaign, facilities improvements, and an excellence campaign -- increase college tuition, institutional debt, and expenditures per student; only the scenario of ignoring the rankings decreases these measures. By referring to the feedback structure of academic competition, the article makes several recommendations for controlling tuition inflation. This article contributes to the literature on the economics of higher education and illustrates the value of feedback economics in developing economic theory.


title: Capturing dynamics and thermodynamics of a three-level quantum heat engine via programmable quantum circuits

abstract: This research employs the Kraus representation and Sz.-Nagy dilation theorem to model a three-level quantum heat on quantum circuits, investigating its dynamic evolution and thermodynamic performance. The feasibility of the dynamic model is validated by tracking the changes of population. On the basis of reinforcement learning algorithm, the optimal cycle of the quantum heat engine for maximal average power is proposed and verified by the thermodynamic model. The stability of quantum circuit simulations is scrutinized through a comparative analysis of theoretical and simulated results, predicated on an orthogonal test. These results affirm the practicality of simulating quantum heat engines on quantum circuits, offering potential for substantially curtailing the experimental expenses associated with the construction of such engines.

categories: quant-ph physics.app-ph physics.atm-clus physics.optics


title: On the Sequence Evaluation based on Stochastic Processes

abstract: Modeling and analyzing long sequences of text is an essential task for Natural Language Processing. Success in capturing long text dynamics using neural language models will facilitate many downstream tasks such as coherence evaluation, text generation, machine translation and so on. This paper presents a novel approach to model sequences through a stochastic process. We introduce a likelihood-based training objective for the text encoder and design a more thorough measurement (score) for long text evaluation compared to the previous approach. The proposed training objective effectively preserves the sequence coherence, while the new score comprehensively captures both temporal and spatial dependencies. Theoretical properties of our new score show its advantages in sequence evaluation. Experimental results show superior performance in various sequence evaluation tasks, including global and local discrimination within and between documents of different lengths. We also demonstrate the encoder achieves competitive results on discriminating human and AI written text.


title: PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild

abstract: Video quality assessment (VQA) is a challenging problem due to the numerous factors that can affect the perceptual quality of a video, \eg, content attractiveness, distortion type, motion pattern, and level. However, annotating the Mean opinion score (MOS) for videos is expensive and time-consuming, which limits the scale of VQA datasets, and poses a significant obstacle for deep learning-based methods. In this paper, we propose a VQA method named PTM-VQA, which leverages PreTrained Models to transfer knowledge from models pretrained on various pre-tasks, enabling benefits for VQA from different aspects. Specifically, we extract features of videos from different pretrained models with frozen weights and integrate them to generate representation. Since these models possess various fields of knowledge and are often trained with labels irrelevant to quality, we propose an Intra-Consistency and Inter-Divisibility (ICID) loss to impose constraints on features extracted by multiple pretrained models. The intra-consistency constraint ensures that features extracted by different pretrained models are in the same unified quality-aware latent space, while the inter-divisibility introduces pseudo clusters based on the annotation of samples and tries to separate features of samples from different clusters. Furthermore, with a constantly growing number of pretrained models, it is crucial to determine which models to use and how to use them. To address this problem, we propose an efficient scheme to select suitable candidates. Models with better clustering performance on VQA datasets are chosen to be our candidates. Extensive experiments demonstrate the effectiveness of the proposed method.


title: SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals

abstract: Sleep is a complex physiological process evaluated through various modalities recording electrical brain, cardiac, and respiratory activities. We curate a large polysomnography dataset from over 14,000 participants comprising over 100,000 hours of multi-modal sleep recordings. Leveraging this extensive dataset, we developed SleepFM, the first multi-modal foundation model for sleep analysis. We show that a novel leave-one-out approach for contrastive learning significantly improves downstream task performance compared to representations from standard pairwise contrastive learning. A logistic regression model trained on SleepFM's learned embeddings outperforms an end-to-end trained convolutional neural network (CNN) on sleep stage classification (macro AUROC 0.88 vs 0.72 and macro AUPRC 0.72 vs 0.48) and sleep disordered breathing detection (AUROC 0.85 vs 0.69 and AUPRC 0.77 vs 0.61). Notably, the learned embeddings achieve 48% top-1 average accuracy in retrieving the corresponding recording clips of other modalities from 90,000 candidates. This work demonstrates the value of holistic multi-modal sleep modeling to fully capture the richness of sleep recordings. SleepFM is open source and available at https://github.com/rthapa84/sleepfm-codebase.


title: Linguistic Collapse: Neural Collapse in (Large) Language Models

abstract: Neural collapse ($\mathcal{NC}$) is a phenomenon observed in classification tasks where top-layer representations collapse into their class means, which become equinorm, equiangular and aligned with the classifiers. These behaviors -- associated with generalization and robustness -- would manifest under specific conditions: models are trained towards zero loss, with noise-free labels belonging to balanced classes, which do not outnumber the model's hidden dimension. Recent studies have explored $\mathcal{NC}$ in the absence of one or more of these conditions to extend and capitalize on the associated benefits of ideal geometries. Language modeling presents a curious frontier, as \textit{training by token prediction} constitutes a classification task where none of the conditions exist: the vocabulary is imbalanced and exceeds the embedding dimension; different tokens might correspond to similar contextual embeddings; and large language models (LLMs) in particular are typically only trained for a few epochs. This paper empirically investigates the impact of scaling the architectures and training of causal language models (CLMs) on their progression towards $\mathcal{NC}$. We find that $\mathcal{NC}$ properties that develop with scaling are linked to generalization. Moreover, there is evidence of some relationship between $\mathcal{NC}$ and generalization independent of scale. Our work therefore underscores the generality of $\mathcal{NC}$ as it extends to the novel and more challenging setting of language modeling. Downstream, we seek to inspire further research on the phenomenon to deepen our understanding of LLMs -- and neural networks at large -- and improve existing architectures based on $\mathcal{NC}$-related properties.


title: Revisiting the Message Passing in Heterophilous Graph Neural Networks

abstract: Graph Neural Networks (GNNs) have demonstrated strong performance in graph mining tasks due to their message-passing mechanism, which is aligned with the homophily assumption that adjacent nodes exhibit similar behaviors. However, in many real-world graphs, connected nodes may display contrasting behaviors, termed as heterophilous patterns, which has attracted increased interest in heterophilous GNNs (HTGNNs). Although the message-passing mechanism seems unsuitable for heterophilous graphs due to the propagation of class-irrelevant information, it is still widely used in many existing HTGNNs and consistently achieves notable success. This raises the question: why does message passing remain effective on heterophilous graphs? To answer this question, in this paper, we revisit the message-passing mechanisms in heterophilous graph neural networks and reformulate them into a unified heterophilious message-passing (HTMP) mechanism. Based on HTMP and empirical analysis, we reveal that the success of message passing in existing HTGNNs is attributed to implicitly enhancing the compatibility matrix among classes. Moreover, we argue that the full potential of the compatibility matrix is not completely achieved due to the existence of incomplete and noisy semantic neighborhoods in real-world heterophilous graphs. To bridge this gap, we introduce a new approach named CMGNN, which operates within the HTMP mechanism to explicitly leverage and improve the compatibility matrix. A thorough evaluation involving 10 benchmark datasets and comparative analysis against 13 well-established baselines highlights the superior performance of the HTMP mechanism and CMGNN method.


title: Microsaccade-inspired Event Camera for Robotics

abstract: Neuromorphic vision sensors or event cameras have made the visual perception of extremely low reaction time possible, opening new avenues for high-dynamic robotics applications. These event cameras' output is dependent on both motion and texture. However, the event camera fails to capture object edges that are parallel to the camera motion. This is a problem intrinsic to the sensor and therefore challenging to solve algorithmically. Human vision deals with perceptual fading using the active mechanism of small involuntary eye movements, the most prominent ones called microsaccades. By moving the eyes constantly and slightly during fixation, microsaccades can substantially maintain texture stability and persistence. Inspired by microsaccades, we designed an event-based perception system capable of simultaneously maintaining low reaction time and stable texture. In this design, a rotating wedge prism was mounted in front of the aperture of an event camera to redirect light and trigger events. The geometrical optics of the rotating wedge prism allows for algorithmic compensation of the additional rotational motion, resulting in a stable texture appearance and high informational output independent of external motion. The hardware device and software solution are integrated into a system, which we call Artificial MIcrosaccade-enhanced EVent camera (AMI-EV). Benchmark comparisons validate the superior data quality of AMI-EV recordings in scenarios where both standard cameras and event cameras fail to deliver. Various real-world experiments demonstrate the potential of the system to facilitate robotics perception both for low-level and high-level vision tasks.


title: Risk-Neutral Generative Networks

abstract: We present a functional generative approach to extract risk-neutral densities from market prices of options. Specifically, we model the log-returns on the time-to-maturity continuum as a stochastic curve driven by standard normal. We then use neural nets to represent the term structures of the location, the scale, and the higher-order moments, and impose stringent conditions on the learning process to ensure the neural net-based curve representation is free of static arbitrage. This specification is structurally clear in that it separates the modeling of randomness from the modeling of the term structures of the parameters. It is data adaptive in that we use neural nets to represent the shape of the stochastic curve. It is also generative in that the functional form of the stochastic curve, although parameterized by neural nets, is an explicit and deterministic function of the standard normal. This explicitness allows for the efficient generation of samples to price options across strikes and maturities, without compromising data adaptability. We have validated the effectiveness of this approach by benchmarking it against a comprehensive set of baseline models. Experiments show that the extracted risk-neutral densities accommodate a diverse range of shapes. Its accuracy significantly outperforms the extensive set of baseline models--including three parametric models and nine stochastic process models--in terms of accuracy and stability. The success of this approach is attributed to its capacity to offer flexible term structures for risk-neutral skewness and kurtosis.


title: Non-uniform dependence on initial data for the generalized Camassa-Holm equation in $C^1$

abstract: It is shown in \cite[Adv. Differ. Equ(2017)]{HT} that the Cauchy problem for the generalized Camassa-Holm equation is well-posed in $C^1$ and the data-to-solution map is H\"{o}lder continuous from $C^\alpha$ to $\mathcal{C}([0,T];C^\alpha)$ with $\alpha\in[0,1)$. In this paper, we further show that the data-to-solution map of the generalized Camassa-Holm equation is not uniformly continuous on the initial data in $C^1$. In particular, our result also can be a complement of previous work on the classical Camassa-Holm equation in \cite[Geom. Funct. Anal(2002)]{G02}.


title: A Contact Binary Mis-Classified as an Ellipsoidal Variable: Complications for Detached Black Hole Searches

abstract: Identifying sources exhibiting ellipsoidal variability in large photometric surveys is becoming a promising method to search for candidate detached black holes in binaries. This technique aims to exploit the orbital-phase dependent modulation in optical photometry caused by the black hole distorting the shape of the luminous star to constrain the mass ratio of the binary. Without understanding if, or how much, contamination is present in the candidate black hole samples produced by this new technique it is hard to leverage them for black hole discovery. Here, we follow up one of the best candidates identified from Gaia Data Release 3, Gaia DR3 4042390512917208960, with a radial velocity campaign. Combined photometric and radial velocity modelling, along with spectral disentangling, suggests that the true mass ratio (mass of the unseen object divided by the mass of the luminous star) is an order of magnitude smaller than that inferred assuming the modulations arise from ellipsoidal variability. We therefore infer that this system is likely a contact binary, or on the boundary of both stars nearly filling their Roche lobes, however, further observations are required to confidently detect the secondary. We find that the well-known problem of discriminating between ellipsoidal and contact binary light curves results in a larger contamination from contact binaries than previously suggested. Until ellipsoidal variables can be reliably distinguished from contact binaries, samples of black hole candidates selected based on ellipsoidal variability are likely to be highly contaminated by contact binaries or similar systems.


title: Towards a Generalist and Blind RGB-X Tracker

abstract: With the emergence of a single large model capable of successfully solving a multitude of tasks in NLP, there has been growing research interest in achieving similar goals in computer vision. On the one hand, most of these generic models, referred to as generalist vision models, aim at producing unified outputs serving different tasks. On the other hand, some existing models aim to combine different input types (aka data modalities), which are then processed by a single large model. Yet, this step of combination remains specialized, which falls short of serving the initial ambition. In this paper, we showcase that such specialization (during unification) is unnecessary, in the context of RGB-X video object tracking. Our single model tracker, termed XTrack, can remain blind to any modality X during inference time. Our tracker employs a mixture of modal experts comprising those dedicated to shared commonality and others capable of flexibly performing reasoning conditioned on input modality. Such a design ensures the unification of input modalities towards a common latent space, without weakening the modality-specific information representation. With this idea, our training process is extremely simple, integrating multi-label classification loss with a routing function, thereby effectively aligning and unifying all modalities together, even from only paired data. Thus, during inference, we can adopt any modality without relying on the inductive bias of the modal prior and achieve generalist performance. Without any bells and whistles, our generalist and blind tracker can achieve competitive performance compared to well-established modal-specific models on 5 benchmarks across 3 auxiliary modalities, covering commonly used depth, thermal, and event data.


title: Gradually Vanishing Gap in Prototypical Network for Unsupervised Domain Adaptation

abstract: Unsupervised domain adaptation (UDA) is a critical problem for transfer learning, which aims to transfer the semantic information from labeled source domain to unlabeled target domain. Recent advancements in UDA models have demonstrated significant generalization capabilities on the target domain. However, the generalization boundary of UDA models remains unclear. When the domain discrepancy is too large, the model can not preserve the distribution structure, leading to distribution collapse during the alignment. To address this challenge, we propose an efficient UDA framework named Gradually Vanishing Gap in Prototypical Network (GVG-PN), which achieves transfer learning from both global and local perspectives. From the global alignment standpoint, our model generates a domain-biased intermediate domain that helps preserve the distribution structures. By entangling cross-domain features, our model progressively reduces the risk of distribution collapse. However, only relying on global alignment is insufficient to preserve the distribution structure. To further enhance the inner relationships of features, we introduce the local perspective. We utilize the graph convolutional network (GCN) as an intuitive method to explore the internal relationships between features, ensuring the preservation of manifold structures and generating domain-biased prototypes. Additionally, we consider the discriminability of the inner relationships between features. We propose a pro-contrastive loss to enhance the discriminability at the prototype level by separating hard negative pairs. By incorporating both GCN and the pro-contrastive loss, our model fully explores fine-grained semantic relationships. Experiments on several UDA benchmarks validated that the proposed GVG-PN can clearly outperform the SOTA models.


title: Manifestation of antiquark nuggets in collisions with the Earth

abstract: Antiquark nuggets are hypothetical compact composite objects conjectured to account for a significant fraction of dark matter in the Universe. In contrast to quark nuggets, these objects consist of antimatter. They may remain undetected if they possess a sufficiently small cross section relative to their mass. In this paper, we investigate the allowed region in the parameter space of this model that is consistent with the observed neutrino flux from the Sun and the Earth, and the non-observation of seismic events with specific signatures of dark matter particles. We found the allowed values of the antibaryon charge number in this model to be in the interval $2\times 10^{24}<A<8\times 10^{25}$, while the probability of nucleon annihilation upon collisions with the antiquark core is constrained by $0.1\lesssim \kappa<0.25$. Although very large values of the antibaryon charge, $A>10^{33}$, are not fully excluded by the present study, we show that they conflict with the non-observation of rare catastrophic explosion-like events on the Earth.


title: The Binary Quantized Neural Network for Dense Prediction via Specially Designed Upsampling and Attention

abstract: Deep learning-based information processing consumes long time and requires huge computing resources, especially for dense prediction tasks which require an output for each pixel, like semantic segmentation and salient object detection. There are mainly two challenges for quantization of dense prediction tasks. Firstly, directly applying the upsampling operation that dense prediction tasks require is extremely crude and causes unacceptable accuracy reduction. Secondly, the complex structure of dense prediction networks means it is difficult to maintain a fast speed as well as a high accuracy when performing quantization. In this paper, we propose an effective upsampling method and an efficient attention computation strategy to transfer the success of the binary neural networks (BNN) from single prediction tasks to dense prediction tasks. Firstly, we design a simple and robust multi-branch parallel upsampling structure to achieve the high accuracy. Then we further optimize the attention method which plays an important role in segmentation but has huge computation complexity. Our attention method can reduce the computational complexity by a factor of one hundred times but retain the original effect. Experiments on Cityscapes, KITTI road, and ECSSD fully show the effectiveness of our work.


title: RREH: Reconstruction Relations Embedded Hashing for Semi-Paired Cross-Modal Retrieval

abstract: Known for efficient computation and easy storage, hashing has been extensively explored in cross-modal retrieval. The majority of current hashing models are predicated on the premise of a direct one-to-one mapping between data points. However, in real practice, data correspondence across modalities may be partially provided. In this research, we introduce an innovative unsupervised hashing technique designed for semi-paired cross-modal retrieval tasks, named Reconstruction Relations Embedded Hashing (RREH). RREH assumes that multi-modal data share a common subspace. For paired data, RREH explores the latent consistent information of heterogeneous modalities by seeking a shared representation. For unpaired data, to effectively capture the latent discriminative features, the high-order relationships between unpaired data and anchors are embedded into the latent subspace, which are computed by efficient linear reconstruction. The anchors are sampled from paired data, which improves the efficiency of hash learning. The RREH trains the underlying features and the binary encodings in a unified framework with high-order reconstruction relations preserved. With the well devised objective function and discrete optimization algorithm, RREH is designed to be scalable, making it suitable for large-scale datasets and facilitating efficient cross-modal retrieval. In the evaluation process, the proposed is tested with partially paired data to establish its superiority over several existing methods.


title: Synthetic non-Abelian charges in degenerate Fermi gases

abstract: Topological phases associated with non-Abelian charges can exhibit a distinguished bulk-edge correspondence compared to Abelian phases, although elucidating this relationship remains challenging in traditional solid-state systems. In this paper, we propose a theoretical framework for synthesizing non-Abelian quaternion charges in degenerate Fermi gases. By designing artificial spin-orbit coupling patterns, the topological edge modes demonstrate a clear correspondence with the band topology determined by various quaternion charges. This paves the way for observing the interface modes whose existence is attributed to the non-conservation multiplication relation, which is fundamental to non-Abelian charges. This scheme can be readily implemented using current ultracold atom techniques, offering a promising approach to explore the intriguing non-Abelian characteristics of the system.


title: Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task

abstract: In the field of autonomous driving, even a meticulously trained model can encounter failures when faced with unfamiliar sceanrios. One of these scenarios can be formulated as an online continual learning (OCL) problem. That is, data come in an online fashion, and models are updated according to these streaming data. Two major OCL challenges are catastrophic forgetting and data imbalance. To address these challenges, in this paper, we propose an Analytic Exemplar-Free Online Continual Learning (AEF-OCL). The AEF-OCL leverages analytic continual learning principles and employs ridge regression as a classifier for features extracted by a large backbone network. It solves the OCL problem by recursively calculating the analytical solution, ensuring an equalization between the continual learning and its joint-learning counterpart, and works without the need to save any used samples (i.e., exemplar-free). Additionally, we introduce a Pseudo-Features Generator (PFG) module that recursively estimates the deviation of real features. The PFG generates offset pseudo-features following a normal distribution, thereby addressing the data imbalance issue. Experimental results demonstrate that despite being an exemplar-free strategy, our method outperforms various methods on the autonomous driving SODA10M dataset. Source code is available at https://github.com/ZHUANGHP/Analytic-continual-learning.


title: Unmasking Vulnerabilities: Cardinality Sketches under Adaptive Inputs

abstract: Cardinality sketches are popular data structures that enhance the efficiency of working with large data sets. The sketches are randomized representations of sets that are only of logarithmic size but can support set merges and approximate cardinality (i.e., distinct count) queries. When queries are not adaptive, that is, they do not depend on preceding query responses, the design provides strong guarantees of correctly answering a number of queries exponential in the sketch size $k$. In this work, we investigate the performance of cardinality sketches in adaptive settings and unveil inherent vulnerabilities. We design an attack against the ``standard'' estimators that constructs an adversarial input by post-processing responses to a set of simple non-adaptive queries of size linear in the sketch size $k$. Empirically, our attack used only $4k$ queries with the widely used HyperLogLog (HLL++)~\citep{hyperloglog:2007,hyperloglogpractice:EDBT2013} sketch. The simple attack technique suggests it can be effective with post-processed natural workloads. Finally and importantly, we demonstrate that the vulnerability is inherent as \emph{any} estimator applied to known sketch structures can be attacked using a number of queries that is quadratic in $k$, matching a generic upper bound.


title: Dissipation-induced bound states as a two-level system

abstract: Potential wells are employed to constrain quantum particles into forming discrete energy levels, acting as artificial few-level systems. In contrast, an anti-parity-time ($\mathcal{PT}$) symmetric system can have a single pair of real energy levels, while all the remaining levels are unstable due to the negative imaginary part of the energy. In this work, we investigate the formation of bound states in a tight-binding chain induced by a harmonic imaginary potential. Exact solutions show that the real parts of energy levels are equidistant, while the imaginary parts are semi-negative definite and equidistant. This allows for the formation of an effective two-level system. For a given initial state with a wide range of profiles, the evolved state always converges to a superposition of two stable eigenstates. In addition, these two states are orthogonal under the Dirac inner product and can be mutually switched by applying a $\pi$ pulse of a linear field. Our finding provides an alternative method for fabricating quantum devices through dissipation.


title: Post-Fair Federated Learning: Achieving Group and Community Fairness in Federated Learning via Post-processing

abstract: Federated Learning (FL) is a distributed machine learning framework in which a set of local communities collaboratively learn a shared global model while retaining all training data locally within each community. Two notions of fairness have recently emerged as important issues for federated learning: group fairness and community fairness. Group fairness requires that a model's decisions do not favor any particular group based on a set of legally protected attributes such as race or gender. Community fairness requires that global models exhibit similar levels of performance (accuracy) across all collaborating communities. Both fairness concepts can coexist within an FL framework, but the existing literature has focused on either one concept or the other. This paper proposes and analyzes a post-processing fair federated learning (FFL) framework called post-FFL. Post-FFL uses a linear program to simultaneously enforce group and community fairness while maximizing the utility of the global model. Because Post-FFL is a post-processing approach, it can be used with existing FL training pipelines whose convergence properties are well understood. This paper uses post-FFL on real-world datasets to mimic how hospital networks, for example, use federated learning to deliver community health care. Theoretical results bound the accuracy lost when post-FFL enforces both notion of fairness. Experimental results illustrate that post-FFL simultaneously improves both group and community fairness in FL. Moreover, post-FFL outperforms the existing in-processing fair federated learning in terms of improving both notions of fairness, communication efficiency and computation cost.


title: Hyperbolic motions in the $N$-body problem with homogeneous potentials

abstract: In the $N$-body problem, a motion is called hyperbolic, when the mutual distances between the bodies go to infinity with non-zero limiting velocities as time goes to infinity. For Newtonian potential, in \cite{MV20} Maderna and Venturelli proved that starting from any initial position there is a hyperbolic motion with any prescribed limiting velocities at infinity. Recently based on a different approach, Liu, Yan and Zhou \cite{LYZ21} generalized this result to a larger class of $N$-body problem. As the proof in \cite{LYZ21} is quite long and technical, we give a simplified proof for homogeneous potentials following the approach given in the latter paper.


title: Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation

abstract: Model-Free Reinforcement Learning (MFRL), leveraging the policy gradient theorem, has demonstrated considerable success in continuous control tasks. However, these approaches are plagued by high gradient variance due to zeroth-order gradient estimation, resulting in suboptimal policies. Conversely, First-Order Model-Based Reinforcement Learning (FO-MBRL) methods employing differentiable simulation provide gradients with reduced variance but are susceptible to sampling error in scenarios involving stiff dynamics, such as physical contact. This paper investigates the source of this error and introduces Adaptive Horizon Actor-Critic (AHAC), an FO-MBRL algorithm that reduces gradient error by adapting the model-based horizon to avoid stiff dynamics. Empirical findings reveal that AHAC outperforms MFRL baselines, attaining 40% more reward across a set of locomotion tasks and efficiently scaling to high-dimensional control environments with improved wall-clock-time efficiency.


title: Graph Skeletons and Diminishing Minors

abstract: We define coarse skeletons of graphs in terms of two constants. We introduce the notion of coarse bottlenecking in graphs and show how it can guarantee that a skeleton resembles (up to quasi-isometry) the original graph. We show how notions similar to the coarse skeleton have been previously used to classify some coarse families of graphs. We explore the properties of a coarse skeleton and of combinations of skeletons. We show how these tools can be used to simplify the structure of graphs that have an excluded asymptotic minor, reducing it to a skeleton of the original containing at most a 3-fat minor, we give an example to show that a similar result does not hold for 2-fat minors.


title: Effect of insulator end cap thickness on time-dependent Hartmann flow in a rotating mirror

abstract: We present a framework for analyzing plasma flow in a rotating mirror. By making a series of physical assumptions, we reduce the magnetohydrodynamic (MHD) equations in a three-dimensional cylindrical system to a one-dimensional system in a shallow, cuboidal channel within a transverse magnetic field, similar to the Hartmann flow in the ducts. We then solve the system both numerically and analytically for a range of values of the Hartmann number and calculate the dependence of the plasma flow speed on the thickness of the insulating end cap. We observe that the mean flow overshoots and decelerates before achieving a steady-state value, a phenomenon that the analytical model cannot capture. This overshoot is directly proportional to the thickness of the insulating end cap and the external electric field, with a weak dependence on the external magnetic field. Our simplified model can act as a benchmark for future simulations of the supersonic mirror device Compact Magnetic Fusion Experiment (CMFX), which will employ more sophisticated physics and realistic magnetic field geometries.


title: Dyadic Regression with Sample Selection

abstract: This paper addresses the sample selection problem in panel dyadic regression analysis. Dyadic data often include many zeros in the main outcomes due to the underlying network formation process. This not only contaminates popular estimators used in practice but also complicates the inference due to the dyadic dependence structure. We extend Kyriazidou (1997)'s approach to dyadic data and characterize the asymptotic distribution of our proposed estimator. The convergence rates are $\sqrt{n}$ or $\sqrt{n^{2}h_{n}}$, depending on the degeneracy of the H\'{a}jek projection part of the estimator, where $n$ is the number of nodes and $h_{n}$ is a bandwidth. We propose a bias-corrected confidence interval and a variance estimator that adapts to the degeneracy. A Monte Carlo simulation shows the good finite performance of our estimator and highlights the importance of bias correction in both asymptotic regimes when the fraction of zeros in outcomes varies. We illustrate our procedure using data from Moretti and Wilson (2017)'s paper on migration.


title: Enhancing Road Safety: Real-Time Detection of Driver Distraction through Convolutional Neural Networks

abstract: As we navigate our daily commutes, the threat posed by a distracted driver is at a large, resulting in a troubling rise in traffic accidents. Addressing this safety concern, our project harnesses the analytical power of Convolutional Neural Networks (CNNs), with a particular emphasis on the well-established models VGG16 and VGG19. These models are acclaimed for their precision in image recognition and are meticulously tested for their ability to detect nuances in driver behavior under varying environmental conditions. Through a comparative analysis against an array of CNN architectures, this study seeks to identify the most efficient model for real-time detection of driver distractions. The ultimate aim is to incorporate the findings into vehicle safety systems, significantly boosting their capability to prevent accidents triggered by inattention. This research not only enhances our understanding of automotive safety technologies but also marks a pivotal step towards creating vehicles that are intuitively aligned with driver behaviors, ensuring safer roads for all.


title: On the Downlink Average Energy Efficiency of Non-Stationary XL-MIMO

abstract: Extra large-scale multiple-input multiple-output (XL-MIMO) is a key technology for future wireless communication systems. This paper considers the effects of visibility region (VR) at the base station (BS) in a non-stationary multi-user XL-MIMO scenario, where only partial antennas can receive users' signal. In time division duplexing (TDD) mode, we first estimate the VR at the BS by detecting the energy of the received signal during uplink training phase. The probabilities of two detection errors are derived and the uplink channel on the detected VR is estimated. In downlink data transmission, to avoid cumbersome Monte-Carlo trials, we derive a deterministic approximate expression for ergodic {average energy efficiency (EE)} with the regularized zero-forcing (RZF) precoding. In frequency division duplexing (FDD) mode, the VR is estimated in uplink training and then the channel information of detected VR is acquired from the feedback channel. In downlink data transmission, the approximation of ergodic average {EE} is also derived with the RZF precoding. Invoking approximate results, we propose an alternate optimization algorithm to design the detection threshold and the pilot length in both TDD and FDD modes. The numerical results reveal the impacts of VR estimation error on ergodic average {EE} and demonstrate the effectiveness of our proposed algorithm.


title: Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification

abstract: Human intelligence can retrieve any person according to both visual and language descriptions. However, the current computer vision community studies specific person re-identification (ReID) tasks in different scenarios separately, which limits the applications in the real world. This paper strives to resolve this problem by proposing a novel instruct-ReID task that requires the model to retrieve images according to the given image or language instructions. Instruct-ReID is the first exploration of a general ReID setting, where existing 6 ReID tasks can be viewed as special cases by assigning different instructions. To facilitate research in this new instruct-ReID task, we propose a large-scale OmniReID++ benchmark equipped with diverse data and comprehensive evaluation methods e.g., task specific and task-free evaluation settings. In the task-specific evaluation setting, gallery sets are categorized according to specific ReID tasks. We propose a novel baseline model, IRM, with an adaptive triplet loss to handle various retrieval tasks within a unified framework. For task-free evaluation setting, where target person images are retrieved from task-agnostic gallery sets, we further propose a new method called IRM++ with novel memory bank-assisted learning. Extensive evaluations of IRM and IRM++ on OmniReID++ benchmark demonstrate the superiority of our proposed methods, achieving state-of-the-art performance on 10 test sets. The datasets, the model, and the code will be available at https://github.com/hwz-zju/Instruct-ReID


title: Classical and quantum thermodynamics in a non-equilibrium regime: Application to Stirling engine

abstract: We develop a thermodynamic theory in the non-equilibrium regime by extending a theory based on the dimensionless (DL) minimum work principle previously developed for a thermodynamic system-bath model [S.Koyanagi and Y.Tanimura,J.Chem.Phys.160,(2024)]. Our results are described by non-equilibrium thermodynamic potentials expressed in time-derivative form in terms of extensive and intensive variables. This is made possible through the incorporation of waste heat, which is equivalent to the loss work consumed by the bath, into the definitions of thermodynamic potentials in a non-equilibrium regime. These potentials can be evaluated from the DL non-equilibrium-to-equilibrium minimum work principle, which is derived from the principle of DL minimum work and is equivalent to the second law of thermodynamics. We thus obtain the non-equilibrium Massieu-Planck potentials as entropic potentials and the non-equilibrium Helmholtz-Gibbs potentials as free energies. Our results are verified numerically by simulating a Stirling engine consisting of two isothermal and two thermostatic processes using the quantum Fokker-Planck equations and the classical Kramers equation derived from the thermodynamic system-bath model. We then show that any thermodynamic process can be analyzed using a non-equilibrium work diagram analogous to the equilibrium one for given time-dependent intensive variables. The results can be used to develop efficient heat machines in non-equilibrium regimes.


title: JUNO Sensitivity to Invisible Decay Modes of Neutrons

abstract: We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 \nu$ or $nn \rightarrow 2 \nu$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation modes of the excited residual nuclei can produce a time- and space-correlated triple coincidence signal in the JUNO detector. Based on a full Monte Carlo simulation informed with the latest available data, we estimate all backgrounds, including inverse beta decay events of the reactor antineutrino $\bar{\nu}_e$, natural radioactivity, cosmogenic isotopes and neutral current interactions of atmospheric neutrinos. Pulse shape discrimination and multivariate analysis techniques are employed to further suppress backgrounds. With two years of exposure, JUNO is expected to give an order of magnitude improvement compared to the current best limits. After 10 years of data taking, the JUNO expected sensitivities at a 90% confidence level are $\tau/B( n \rightarrow { inv} ) > 5.0 \times 10^{31} \, {\rm yr}$ and $\tau/B( nn \rightarrow { inv} ) > 1.4 \times 10^{32} \, {\rm yr}$.


title: SafeguardGS: 3D Gaussian Primitive Pruning While Avoiding Catastrophic Scene Destruction

abstract: 3D Gaussian Splatting (3DGS) has made a significant stride in novel view synthesis, demonstrating top-notch rendering quality while achieving real-time rendering speed. However, the excessively large number of Gaussian primitives resulting from 3DGS' suboptimal densification process poses a major challenge, slowing down frame-per-second (FPS) and demanding considerable memory cost, making it unfavorable for low-end devices. To cope with this issue, many follow-up studies have suggested various pruning techniques, often in combination with different score functions, to optimize rendering performance. Nonetheless, a comprehensive discussion regarding their effectiveness and implications across all techniques is missing. In this paper, we first categorize 3DGS pruning techniques into two types: Cross-view pruning and pixel-wise pruning, which differ in their approaches to rank primitives. Our subsequent experiments reveal that while cross-view pruning leads to disastrous quality drops under extreme Gaussian primitives decimation, the pixel-wise pruning technique not only sustains relatively high rendering quality with minuscule performance degradation but also provides a reasonable minimum boundary for pruning. Building on this observation, we further propose multiple variations of score functions and empirically discover that the color-weighted score function outperforms others for discriminating insignificant primitives for rendering. We believe our research provides valuable insights for optimizing 3DGS pruning strategies for future works.


title: LNS2+RL: Combining Multi-agent Reinforcement Learning with Large Neighborhood Search in Multi-agent Path Finding

abstract: Multi-Agent Path Finding (MAPF) is a critical component of logistics and warehouse management, which focuses on planning collision-free paths for a team of robots in a known environment. Recent work introduced a novel MAPF approach, LNS2, which proposed to repair a quickly-obtainable set of infeasible paths via iterative re-planning, by relying on a fast, yet lower-quality, priority-based planner. At the same time, there has been a recent push for Multi-Agent Reinforcement Learning (MARL) based MAPF algorithms, which let agents learn decentralized policies that exhibit improved cooperation over such priority planning, although inevitably remaining slower. In this paper, we introduce a new MAPF algorithm, LNS2+RL, which combines the distinct yet complementary characteristics of LNS2 and MARL to effectively balance their individual limitations and get the best from both worlds. During early iterations, LNS2+RL relies on MARL for low-level re-planning, which we show eliminates collisions much more than a priority-based planner. There, our MARL-based planner allows agents to reason about past and future/predicted information to gradually learn cooperative decision-making through a finely designed curriculum learning. At later stages of planning, LNS2+RL adaptively switches to priority-based planning to quickly resolve the remaining collisions, naturally trading-off solution quality and computational efficiency. Our comprehensive experiments on challenging tasks across various team sizes, world sizes, and map structures consistently demonstrate the superior performance of LNS2+RL compared to many MAPF algorithms, including LNS2, LaCAM, and EECBS, where LNS2+RL shows significantly better performance in complex scenarios. We finally experimentally validate our algorithm in a hybrid simulation of a warehouse mockup involving a team of 100 (real-world and simulated) robots.


title: Dataset Regeneration for Sequential Recommendation

abstract: The sequential recommender (SR) system is a crucial component of modern recommender systems, as it aims to capture the evolving preferences of users. Significant efforts have been made to enhance the capabilities of SR systems. These methods typically follow the model-centric paradigm, which involves developing effective models based on fixed datasets. However, this approach often overlooks potential quality issues and flaws inherent in the data. Driven by the potential of data-centric AI, we propose a novel data-centric paradigm for developing an ideal training dataset using a model-agnostic dataset regeneration framework called DR4SR. This framework enables the regeneration of a dataset with exceptional cross-architecture generalizability. Additionally, we introduce the DR4SR+ framework, which incorporates a model-aware dataset personalizer to tailor the regenerated dataset specifically for a target model. To demonstrate the effectiveness of the data-centric paradigm, we integrate our framework with various model-centric methods and observe significant performance improvements across four widely adopted datasets. Furthermore, we conduct in-depth analyses to explore the potential of the data-centric paradigm and provide valuable insights. The code can be found at https://anonymous.4open.science/r/KDD2024-86EA


title: Offline Oracle-Efficient Learning for Contextual MDPs via Layerwise Exploration-Exploitation Tradeoff

abstract: Motivated by the recent discovery of a statistical and computational reduction from contextual bandits to offline regression (Simchi-Levi and Xu, 2021), we address the general (stochastic) Contextual Markov Decision Process (CMDP) problem with horizon H (as known as CMDP with H layers). In this paper, we introduce a reduction from CMDPs to offline density estimation under the realizability assumption, i.e., a model class M containing the true underlying CMDP is provided in advance. We develop an efficient, statistically near-optimal algorithm requiring only O(HlogT) calls to an offline density estimation algorithm (or oracle) across all T rounds of interaction. This number can be further reduced to O(HloglogT) if T is known in advance. Our results mark the first efficient and near-optimal reduction from CMDPs to offline density estimation without imposing any structural assumptions on the model class. A notable feature of our algorithm is the design of a layerwise exploration-exploitation tradeoff tailored to address the layerwise structure of CMDPs. Additionally, our algorithm is versatile and applicable to pure exploration tasks in reward-free reinforcement learning.


title: A note on the second neighborhood problem for $k$-anti-transitive and $m$-free digraphs

abstract: Seymour Second Neighborhood Conjecture (SSNC) asserts that every finite oriented graph has a vertex whose second out-neighborhood is at least as large as its first out-neighborhood. Such a vertex is called a Seymour vertex. A digraph $D = (V, E)$ is $k$-anti-transitive if for every pair of vertices $u, v \in V$, the existence of a directed path of length $k$ from $u$ to $v$ implies that $(u, v) \notin E$. An $m$-free digraph is digraph having no directed cycles with length at most $m$. In this paper, we prove that if $D$ is $k$-anti-transitive and $(k-4)$-free digraph, then $D$ has a Seymour vertex. As a consequence, a special case of Caccetta-Haggkvist Conjecture holds on 7-anti-transitive oriented graphs. This work extends recently known results.


title: Learn to be Fair without Labels: a Distribution-based Learning Framework for Fair Ranking

abstract: Ranking algorithms as an essential component of retrieval systems have been constantly improved in previous studies, especially regarding relevance-based utilities. In recent years, more and more research attempts have been proposed regarding fairness in rankings due to increasing concerns about potential discrimination and the issue of echo chamber. These attempts include traditional score-based methods that allocate exposure resources to different groups using pre-defined scoring functions or selection strategies and learning-based methods that learn the scoring functions based on data samples. Learning-based models are more flexible and achieve better performance than traditional methods. However, most of the learning-based models were trained and tested on outdated datasets where fairness labels are barely available. State-of-art models utilize relevance-based utility scores as a substitute for the fairness labels to train their fairness-aware loss, where plugging in the substitution does not guarantee the minimum loss. This inconsistency challenges the model's accuracy and performance, especially when learning is achieved by gradient descent. Hence, we propose a distribution-based fair learning framework (DLF) that does not require labels by replacing the unavailable fairness labels with target fairness exposure distributions. Experimental studies on TREC fair ranking track dataset confirm that our proposed framework achieves better fairness performance while maintaining better control over the fairness-relevance trade-off than state-of-art fair ranking frameworks.


title: Exploring Activation Patterns of Parameters in Language Models

abstract: Most work treats large language models as black boxes without in-depth understanding of their internal working mechanism. In order to explain the internal representations of LLMs, we propose a gradient-based metric to assess the activation level of model parameters. Based on this metric, we obtain three preliminary findings. (1) When the inputs are in the same domain, parameters in the shallow layers will be activated densely, which means a larger portion of parameters will have great impacts on the outputs. In contrast, parameters in the deep layers are activated sparsely. (2) When the inputs are across different domains, parameters in shallow layers exhibit higher similarity in the activation behavior than deep layers. (3) In deep layers, the similarity of the distributions of activated parameters is positively correlated to the empirical data relevance. Further, we develop three validation experiments to solidify these findings. (1) Firstly, starting from the first finding, we attempt to configure different prune ratios for different layers, and find this method can benefit model pruning. (2) Secondly, we find that a pruned model based on one calibration set can better handle tasks related to the calibration task than those not related, which validate the second finding. (3) Thirdly, Based on the STS-B and SICK benchmark, we find that two sentences with consistent semantics tend to share similar parameter activation patterns in deep layers, which aligns with our third finding. Our work sheds light on the behavior of parameter activation in LLMs, and we hope these findings will have the potential to inspire more practical applications.


title: Moments of $L$-functions Problem List

abstract: This is an ongoing list of problems that has resulted from the PIMS (Pacific Institute of Mathematical Sciences) Collaborative Research Group L-functions in Analytic Number Theory: 2022- 2025. The focus of this list is on Moments of $L$-functions and related topics.


title: Bandwidth Efficient Cache Selection and Content Advertisement

abstract: Caching is extensively used in various networking environments to optimize performance by reducing latency, bandwidth, and energy consumption. To optimize performance, caches often advertise their content using indicators, which are data structures that trade space efficiency for accuracy. However, this tradeoff introduces the risk of false indications. Existing solutions for cache content advertisement and cache selection often lead to inefficiencies, failing to adapt to dynamic network conditions. This paper introduces SALSA2, a Scalable Adaptive and Learning-based Selection and Advertisement Algorithm, which addresses these limitations through a dynamic and adaptive approach. SALSA2 accurately estimates mis-indication probabilities by considering inter-cache dependencies and dynamically adjusts the size and frequency of indicator advertisements to minimize transmission overhead while maintaining high accuracy. Our extensive simulation study, conducted using a variety of real-world cache traces, demonstrates that SALSA2 achieves up to 84\% bandwidth savings compared to the state-of-the-art solution and close-to-optimal service cost in most scenarios. These results highlight SALSA2's effectiveness in enhancing cache management, making it a robust and versatile solution for modern networking challenges.


title: Multi-level Interaction Modeling for Protein Mutational Effect Prediction

abstract: Protein-protein interactions are central mediators in many biological processes. Accurately predicting the effects of mutations on interactions is crucial for guiding the modulation of these interactions, thereby playing a significant role in therapeutic development and drug discovery. Mutations generally affect interactions hierarchically across three levels: mutated residues exhibit different sidechain conformations, which lead to changes in the backbone conformation, eventually affecting the binding affinity between proteins. However, existing methods typically focus only on sidechain-level interaction modeling, resulting in suboptimal predictions. In this work, we propose a self-supervised multi-level pre-training framework, ProMIM, to fully capture all three levels of interactions with well-designed pretraining objectives. Experiments show ProMIM outperforms all the baselines on the standard benchmark, especially on mutations where significant changes in backbone conformations may occur. In addition, leading results from zero-shot evaluations for SARS-CoV-2 mutational effect prediction and antibody optimization underscore the potential of ProMIM as a powerful next-generation tool for developing novel therapeutic approaches and new drugs.


title: Searching for the highest energy of pulsation and critical luminosity of Swift J0243.6+6124 observed by Insight-HXMT

abstract: Owing to the broad energy coverage of Insight-HXMT in the hard X-ray band, we detected the highest energy of pulsation exceeding 200 keV around the 2017-2018 outburst peak of the first Galactic pulsating ultraluminous X-ray source (PULX) Swift J0243.6+6124, which is the highest energy detected from PULXs to date. We also obtained the highest energy of pulsation of every exposure during the outburst in 2017-2018, and found the highest energy is roughly positively correlated with luminosity. Using our newly developed method, we identified the critical luminosity being $4\times 10^{38}\, \rm erg\,s^{-1}$ when the main peaks of the low and high energy pulse profiles became aligned, which separates the fan-beam dominated and pencil-beam dominated accretion regimes. Above the critical luminosity, the phase of the main peak shifted gradually from 0.5 to 0.8 until the outburst peak in all energy bands is reached, which is in agreement with the phase shift found previously at low energies. Our result is consistent with what is derived from spectral analysis.


title: Detection-Correction Structure via General Language Model for Grammatical Error Correction

abstract: Grammatical error correction (GEC) is a task dedicated to rectifying texts with minimal edits, which can be decoupled into two components: detection and correction. However, previous works have predominantly focused on direct correction, with no prior efforts to integrate both into a single model. Moreover, the exploration of the detection-correction paradigm by large language models (LLMs) remains underdeveloped. This paper introduces an integrated detection-correction structure, named DeCoGLM, based on the General Language Model (GLM). The detection phase employs a fault-tolerant detection template, while the correction phase leverages autoregressive mask infilling for localized error correction. Through the strategic organization of input tokens and modification of attention masks, we facilitate multi-task learning within a single model. Our model demonstrates competitive performance against the state-of-the-art models on English and Chinese GEC datasets. Further experiments present the effectiveness of the detection-correction structure in LLMs, suggesting a promising direction for GEC.


title: Strong-lensing cosmography using third-generation gravitational-wave detectors

abstract: We present a detailed exposition of a statistical method for estimating cosmological parameters from the observation of a large number of strongly lensed binary-black-hole (BBH) mergers observable by third-generation gravitational-wave (GW) detectors. This method, first presented in $\href{https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.130.261401}{[1]}$, compares the observed number of strongly lensed GW events and their time delay distribution (between lensed images) with observed events to infer cosmological parameters. We show that the precision of the estimation of the cosmological parameters does not have a strong dependance on the assumed BBH redshift distribution model. An incorrect model for the distribution of lens properties and redshifts can lead to a biased cosmological inference. However, Bayesian model selection can assist in selecting the right model from a set of available parametric models for the distribution of lens redshift and properties. We also present a way to incorporate the effect of contamination in the data due to the limited efficiency of lensing identification methods, so that it will not bias the cosmological inference.


title: Entry-Wise Eigenvector Analysis and Improved Rates for Topic Modeling on Short Documents

abstract: Topic modeling is a widely utilized tool in text analysis. We investigate the optimal rate for estimating a topic model. Specifically, we consider a scenario with $n$ documents, a vocabulary of size $p$, and document lengths at the order $N$. When $N\geq c\cdot p$, referred to as the long-document case, the optimal rate is established in the literature at $\sqrt{p/(Nn)}$. However, when $N=o(p)$, referred to as the short-document case, the optimal rate remains unknown. In this paper, we first provide new entry-wise large-deviation bounds for the empirical singular vectors of a topic model. We then apply these bounds to improve the error rate of a spectral algorithm, Topic-SCORE. Finally, by comparing the improved error rate with the minimax lower bound, we conclude that the optimal rate is still $\sqrt{p/(Nn)}$ in the short-document case.


title: The flavor-dependent $U(1)_F$ model

abstract: A flavor-dependent model (FDM) is proposed in this work. The model extends the Standard Model by an extra $U(1)_F$ local gauge group, two scalar doublets, one scalar singlet and two right-handed neutrinos, where the additional $U(1)_F$ charges are related to the particles' flavor. The new fermion sector in the FDM can explain the flavor mixings puzzle and the mass hierarchy puzzle simultaneously, and the nonzero Majorana neutrino masses can be obtained naturally by the Type I see-saw mechanism. In addition, the $B$ meson rare decay processes $\bar B \to X_s\gamma$, $B_s^0 \to \mu^+\mu^-$, the top quark rare decay processes $t\to ch$, $t\to uh$ and the $\tau$ lepton flavor violation processes $\tau\to 3e$, $\tau\to 3\mu$, $\mu\to 3e$ predicted in the FDM are analyzed.


title: Correlation effects in magic-angle twisted bilayer graphene: An auxiliary-field quantum Monte Carlo study

abstract: Magic angle twisted bilayer graphene (MATBG) presents a fascinating platform for investigating the effects of electron interactions in topological flat bands. The Bistritzer-MacDonald (BM) model provides a simplified quantitative description of the flat bands. Introducing long-range Coulomb interactions leads to an interacting BM (IBM) Hamiltonian, a momentum-space continuum description which offers a very natural starting point for many-body studies of MATBG. Accurate and reliable many-body computations in the IBM model are challenging, however, and have been limited mostly to special fillings, or smaller lattice sizes. We employ state-of-the-art auxiliary-field quantum Monte Carlo (AFQMC) method to study the IBM model, which constrains the sign problem to enable accurate treatment of large system sizes. We determine ground-state properties and quantify errors compared to mean-field theory calculations. Our calculations identify correlated metal states and their competition with the insulating Kramers inter-valley coherent state at both half-filling and charge neutrality. Additionally, we investigate one- and three-quarter fillings, and examine the effect of many-body corrections beyond single Slater determinant solutions. We discuss the effect that details of the IBM Hamiltonian have on the results, including different forms of double-counting corrections, and the need to establish and precisely specify many-body Hamiltonians to allow more direct and quantitative comparisons with experiments in MATBG.


title: TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation

abstract: There is a rising interest and trend in research towards directly translating speech from one language to another, known as end-to-end speech-to-speech translation. However, most end-to-end models struggle to outperform cascade models, i.e., a pipeline framework by concatenating speech recognition, machine translation and text-to-speech models. The primary challenges stem from the inherent complexities involved in direct translation tasks and the scarcity of data. In this study, we introduce a novel model framework TransVIP that leverages diverse datasets in a cascade fashion yet facilitates end-to-end inference through joint probability. Furthermore, we propose two separated encoders to preserve the speaker's voice characteristics and isochrony from the source speech during the translation process, making it highly suitable for scenarios such as video dubbing. Our experiments on the French-English language pair demonstrate that our model outperforms the current state-of-the-art speech-to-speech translation model.


title: A new class of evolution multivalued quasi-variational inequalities I: existence and nonsmooth optimal control

abstract: In this paper, we consider a new kind of evolution multivalued quasi-variational inequalities with feedback effect and a nonlinear bifunction which contain several (evolution) quasi-variational/hemivariational inequalities as special cases. The main contribution of this paper is twofold. The first goal is to establish a novel framework for proving the existence of solutions and the compactness of solution set to the evolution multivalued quasi-variational inequalities, under quite mild assumptions. Whereas, the second contribution is to introduce and study a nonlinear and nonsmooth optimal control problem governed by an evolution multivalued quasi-variational inequality, and then to obtain the sufficient conditions for guaranteeing the solvability of the nonlinear and nonsmooth optimal control problem under consideration. Such nonlinear and nonsmooth optimal control problem could as a useful model to explore the simultaneous distributed-boundary optimal control problems driven by evolution multivalued quasi-variational inequalities, and optimal parameters identification for evolution multivalued quasi-variational inequalities.


title: Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh

abstract: Neural 3D representations such as Neural Radiance Fields (NeRF), excel at producing photo-realistic rendering results but lack the flexibility for manipulation and editing which is crucial for content creation. Previous works have attempted to address this issue by deforming a NeRF in canonical space or manipulating the radiance field based on an explicit mesh. However, manipulating NeRF is not highly controllable and requires a long training and inference time. With the emergence of 3D Gaussian Splatting (3DGS), extremely high-fidelity novel view synthesis can be achieved using an explicit point-based 3D representation with much faster training and rendering speed. However, there is still a lack of effective means to manipulate 3DGS freely while maintaining rendering quality. In this work, we aim to tackle the challenge of achieving manipulable photo-realistic rendering. We propose to utilize a triangular mesh to manipulate 3DGS directly with self-adaptation. This approach reduces the need to design various algorithms for different types of Gaussian manipulation. By utilizing a triangle shape-aware Gaussian binding and adapting method, we can achieve 3DGS manipulation and preserve high-fidelity rendering after manipulation. Our approach is capable of handling large deformations, local manipulations, and soft body simulations while keeping high-quality rendering. Furthermore, we demonstrate that our method is also effective with inaccurate meshes extracted from 3DGS. Experiments conducted demonstrate the effectiveness of our method and its superiority over baseline approaches.


title: Lyndon pairs and the lexicographically greatest perfect necklace

abstract: Fix a finite alphabet. A necklace is a circular word. For positive integers $n$ and~$k$, a necklace is $(n,k)$-perfect if all words of length $n$ occur $k$ times but at positions with different congruence modulo $k$, for any convention of the starting position. We define the notion of a Lyndon pair and we use it to construct the lexicographically greatest $(n,k)$-perfect necklace, for any $n$ and $k$ such that $n$ divides~$k$ or $k$ divides~$n$. Our construction generalizes Fredricksen and Maiorana's construction of the lexicographically greatest de Bruijn sequence of order $n$, based on the concatenation of the Lyndon words whose length divide $n$.


title: The Impacts of Data, Ordering, and Intrinsic Dimensionality on Recall in Hierarchical Navigable Small Worlds

abstract: Vector search systems, pivotal in AI applications, often rely on the Hierarchical Navigable Small Worlds (HNSW) algorithm. However, the behaviour of HNSW under real-world scenarios using vectors generated with deep learning models remains under-explored. Existing Approximate Nearest Neighbours (ANN) benchmarks and research typically has an over-reliance on simplistic datasets like MNIST or SIFT1M and fail to reflect the complexity of current use-cases. Our investigation focuses on HNSW's efficacy across a spectrum of datasets, including synthetic vectors tailored to mimic specific intrinsic dimensionalities, widely-used retrieval benchmarks with popular embedding models, and proprietary e-commerce image data with CLIP models. We survey the most popular HNSW vector databases and collate their default parameters to provide a realistic fixed parameterisation for the duration of the paper. We discover that the recall of approximate HNSW search, in comparison to exact K Nearest Neighbours (KNN) search, is linked to the vector space's intrinsic dimensionality and significantly influenced by the data insertion sequence. Our methodology highlights how insertion order, informed by measurable properties such as the pointwise Local Intrinsic Dimensionality (LID) or known categories, can shift recall by up to 12 percentage points. We also observe that running popular benchmark datasets with HNSW instead of KNN can shift rankings by up to three positions for some models. This work underscores the need for more nuanced benchmarks and design considerations in developing robust vector search systems using approximate vector search algorithms. This study presents a number of scenarios with varying real world applicability which aim to better increase understanding and future development of ANN algorithms and embedding


title: FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Models

abstract: The rapid development and reduced barriers to entry for Text-to-Image (T2I) models have raised concerns about the biases in their outputs, but existing research lacks a holistic definition and evaluation framework of biases, limiting the enhancement of debiasing techniques. To address this issue, we introduce FAIntbench, a holistic and precise benchmark for biases in T2I models. In contrast to existing benchmarks that evaluate bias in limited aspects, FAIntbench evaluate biases from four dimensions: manifestation of bias, visibility of bias, acquired attributes, and protected attributes. We applied FAIntbench to evaluate seven recent large-scale T2I models and conducted human evaluation, whose results demonstrated the effectiveness of FAIntbench in identifying various biases. Our study also revealed new research questions about biases, including the side-effect of distillation. The findings presented here are preliminary, highlighting the potential of FAIntbench to advance future research aimed at mitigating the biases in T2I models. Our benchmark is publicly available to ensure the reproducibility.


title: Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model

abstract: In the realm of Multimodal Large Language Models (MLLMs), vision-language connector plays a crucial role to link the pre-trained vision encoders with Large Language Models (LLMs). Despite its importance, the vision-language connector has been relatively less explored. In this study, we aim to propose a strong vision-language connector that enables MLLMs to achieve high accuracy while maintain low computation cost. We first reveal the existence of the visual anchors in Vision Transformer and propose a cost-effective search algorithm to extract them. Building on these findings, we introduce the Anchor Former (AcFormer), a novel vision-language connector designed to leverage the rich prior knowledge obtained from these visual anchors during pretraining, guiding the aggregation of information. Through extensive experimentation, we demonstrate that the proposed method significantly reduces computational costs by nearly two-thirds compared with baseline, while simultaneously outperforming baseline methods. This highlights the effectiveness and efficiency of AcFormer.


title: Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection

abstract: In the open world, detecting out-of-distribution (OOD) data, whose labels are disjoint with those of in-distribution (ID) samples, is important for reliable deep neural networks (DNNs). To achieve better detection performance, one type of approach proposes to fine-tune the model with auxiliary OOD datasets to amplify the difference between ID and OOD data through a separation loss defined on model outputs. However, none of these studies consider enlarging the feature disparity, which should be more effective compared to outputs. The main difficulty lies in the diversity of OOD samples, which makes it hard to describe their feature distribution, let alone design losses to separate them from ID features. In this paper, we neatly fence off the problem based on an aggregation property of ID features named Neural Collapse (NC). NC means that the penultimate features of ID samples within a class are nearly identical to the last layer weight of the corresponding class. Based on this property, we propose a simple but effective loss called OrthLoss, which binds the features of OOD data in a subspace orthogonal to the principal subspace of ID features formed by NC. In this way, the features of ID and OOD samples are separated by different dimensions. By optimizing the feature separation loss rather than purely enlarging output differences, our detection achieves SOTA performance on CIFAR benchmarks without any additional data augmentation or sampling, demonstrating the importance of feature separation in OOD detection. The code will be published.