title: On a cofinal Reinhardt embedding without Powerset

abstract: In this paper, we provide a positive answer to a question by Matthews whether $\mathsf{ZF}^-$ is consistent with a non-trivial cofinal Reinhardt elementary embedding $j\colon V\to V$. The consistency follows from $\mathsf{ZFC} + I_0$, and more precisely, Schultzenberg's model of $\mathsf{ZF}$ with an elementary embedding $k\colon V_{\lambda+2}\to V_{\lambda+2}$.


title: Quantum Random Walks and Quantum Oscillator in an Infinite-Dimensional Phase Space

abstract: We consider quantum random walks in an infinite-dimensional phase space constructed using Weyl representation of the coordinate and momentum operators in the space of functions on a Hilbert space which are square integrable with respect to a shift-invariant measure. We study unitary groups of shift operators in the phase space and averaging of such shifts by Gaussian vectors, which form semigroups of self-adjoint contractions: we find conditions for their strong continuity and establish properties of their generators. Significant differences in their properties allow us to show the absence of the Fourier transform as a unitary transformation that implements the unitary equivalence of these compressive semigroups. Next, we prove the Taylor formula for a certain special subset of smooth functions for shifting to a non-finite vector. It allows us to prove convergence of quantum random walks in the coordinate representation to the evolution of a diffusion process, as well as convergence of quantum random walks in both coordinate and momentum representations to the evolution semigroup of a quantum oscillator in an infinite-dimensional phase space. We find the special essential common domain of generators of semigroups arising in averaging of random shift operators both in position and momentum representations. The invariance of this common domain with respect to both semigroups allows to establish properties of a convex combination of both generators. That convex combination are Hamiltonians of infinite-dimentional quantum oscillators. Thus, we obtain that a Weyl representation of a random walk in an infinite dimensional phase space describes the semigroup of self-adjoint contractions whose generator is the Hamiltonian of an infinite dimensional harmonic oscillator.


title: Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection

abstract: Serialization-based methods, which serialize the 3D voxels and group them into multiple sequences before inputting to Transformers, have demonstrated their effectiveness in 3D object detection. However, serializing 3D voxels into 1D sequences will inevitably sacrifice the voxel spatial proximity. Such an issue is hard to be addressed by enlarging the group size with existing serialization-based methods due to the quadratic complexity of Transformers with feature sizes. Inspired by the recent advances of state space models (SSMs), we present a Voxel SSM, termed as Voxel Mamba, which employs a group-free strategy to serialize the whole space of voxels into a single sequence. The linear complexity of SSMs encourages our group-free design, alleviating the loss of spatial proximity of voxels. To further enhance the spatial proximity, we propose a Dual-scale SSM Block to establish a hierarchical structure, enabling a larger receptive field in the 1D serialization curve, as well as more complete local regions in 3D space. Moreover, we implicitly apply window partition under the group-free framework by positional encoding, which further enhances spatial proximity by encoding voxel positional information. Our experiments on Waymo Open Dataset and nuScenes dataset show that Voxel Mamba not only achieves higher accuracy than state-of-the-art methods, but also demonstrates significant advantages in computational efficiency.


title: MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding

abstract: Improving user experience and providing personalized search results in E-commerce platforms heavily rely on understanding purchase intention. However, existing methods for acquiring large-scale intentions bank on distilling large language models with human annotation for verification. Such an approach tends to generate product-centric intentions, overlook valuable visual information from product images, and incurs high costs for scalability. To address these issues, we introduce MIND, a multimodal framework that allows Large Vision-Language Models (LVLMs) to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. Using Amazon Review data, we apply MIND and create a multimodal intention knowledge base, which contains 1,264,441 million intentions derived from 126,142 co-buy shopping records across 107,215 products. Extensive human evaluations demonstrate the high plausibility and typicality of our obtained intentions and validate the effectiveness of our distillation framework and filtering mechanism. Additional experiments reveal that our obtained intentions significantly enhance large language models in two intention comprehension tasks.


title: Direct laser acceleration in varying plasma density profiles

abstract: Direct laser acceleration has proven to be an efficient source of high-charge electron bunches and high brilliance X-rays. However, an analytical description of the acceleration in the interaction with varying plasma density targets is still missing. Here, we provide an analytical estimate of the maximum energies that electrons can achieve in such a case. We demonstrate that the maximum energy depends on the local electron properties at the moment when the electron fulfills the resonant condition at the beginning of the acceleration. This knowledge enables density shaping for various purposes. One application is to decrease the required acceleration distance which has important implications for multi-petawatt laser experiments, where strong laser depletion could play a crucial role. Another use for density tailoring is to achieve acceleration beyond the radiation reaction limit. We derive the energy scaling law that is valid for arbitrary density profile that varies slowly compared with the betatron period. Our results can be applied to electron heating in exponential preplasma of thin foils, ablating plasma plumes, or gas jets with long-scale ramp-up.


title: Calibrating Neural Networks' parameters through Optimal Contraction in a Prediction Problem

abstract: This study introduces a novel approach to ensure the existence and uniqueness of optimal parameters in neural networks. The paper details how a recurrent neural networks (RNN) can be transformed into a contraction in a domain where its parameters are linear. It then demonstrates that a prediction problem modeled through an RNN, with a specific regularization term in the loss function, can have its first-order conditions expressed analytically. This system of equations is reduced to two matrix equations involving Sylvester equations, which can be partially solved. We establish that, if certain conditions are met, optimal parameters exist, are unique, and can be found through a straightforward algorithm to any desired precision. Also, as the number of neurons grows the conditions of convergence become easier to fulfill. Feedforward neural networks (FNNs) are also explored by including linear constraints on parameters. According to our model, incorporating loops (with fixed or variable weights) will produce loss functions that train easier, because it assures the existence of a region where an iterative method converges.


title: Interactive Graph Visualization in DDLab

abstract: Interactive visualization of the basin of attraction field, the ``ibaf-graph'', is a new feature in DDLab with the same interactive functions as the ``network-graph'' and ``jump-graph''. These functions allow any node and its connected fragment to be dragged/dropped with the pointer as a graphic animation with elastic links. The fragment itself depends on the node's link setting by inputs, outputs, or either, and a distance in link-steps. Further options include graph geometry, rescaling, node display, and link editing. This article describes the three graph types, network/ibaf/jump, their selection, enhanced functions, and applications to cellular automata, discrete dynamical networks and random maps.


title: Propagation of circular Airy derivative beams in complex media

abstract: Controlling light propagation through complex media plays a significant role in a wide range of applications ranging from astronomical observations to microscopy. Although, several advances have been made based on adaptive optics, optical phase conjugation and wavefront shaping, but many of these involve challenges. Recently, controlling light propagation in complex media by simply structuring light has shown promising capabilities. We present experimental and numerical investigations of abruptly autofocusing of circular Airy derivative beams (CADBs) in complex media. We find that up to a relatively high turbulence strength, CADB possesses relatively good abrupt autofocusing, however, efficiency and autofocusing position vary with the strength of turbulence. Further, the spatial distortions in CADB caused by turbulence are quantified by an overlap integral, which shows that CADB possesses reasonably good resilience against the turbulence. The diffraction efficiency of CADB changes by a factor of ~ 1.7 with increasing strength of turbulence from zero to high, indicating good confinement of intensity at autofocusing. The focused beam spot size grows gradually with increasing the strength of turbulence, specifically, it grows by a factor of ~ 2 for a strong turbulence, indicating reasonably good focusing abilities. The results of CADB are compared with a Gaussian beam, and find that CADB possesses superior focusing abilities in turbulent media. We have carried out a detailed analysis of these observations based on Zernike polynomials, which reveals that different kinds of aberrations present in turbulent media leads to distortions in the spatial structure as well as other properties of CADBs. Our results can be used for various applications, such as in biomedical treatment, seismology, optical tweezers and material processing.


title: Notch Filters without Transient Effects: A Constrained Optimization Design

abstract: Transient responses are an inherent property of recursive filters due to unknown or incorrectly selected initial conditions. Well-designed stable filters are less affected by transient responses, as the impact of initial conditions diminishes over time. However, applications that require very short data acquisition periods (for example, as short as ten seconds), such as biosignals recorded and processed by wearable technologies, can be significantly impacted by transient effects. But how feasible is it to design filters without transient responses? We propose a well-known filter design scheme based on constrained least squares (CLS) optimization to create zero-transient effect notch filters for powerline noise cancellation. We demonstrate that this filter is equivalent to the optimal Wiener smoother in the stationary case. We also discuss its limitations in removing powerline noise with nonstationary amplitude, where a Kalman filter-based formulation can be used instead.


title: DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models

abstract: LLMs have seen rapid adoption in all domains. They need to be trained on high-end high-performance computing (HPC) infrastructures and ingest massive amounts of input data. Unsurprisingly, at such a large scale, unexpected events (e.g., failures of components, instability of the software, undesirable learning patterns, etc.), are frequent and typically impact the training in a negative fashion. Thus, LLMs need to be checkpointed frequently so that they can be rolled back to a stable state and subsequently fine-tuned. However, given the large sizes of LLMs, a straightforward checkpointing solution that directly writes the model parameters and optimizer state to persistent storage (e.g., a parallel file system), incurs significant I/O overheads. To address this challenge, in this paper we study how to reduce the I/O overheads for enabling fast and scalable checkpointing for LLMs that can be applied at high frequency (up to the granularity of individual iterations) without significant impact on the training process. Specifically, we introduce a lazy asynchronous multi-level approach that takes advantage of the fact that the tensors making up the model and optimizer state shards remain immutable for extended periods of time, which makes it possible to copy their content in the background with minimal interference during the training process. We evaluate our approach at scales of up to 180 GPUs using different model sizes, parallelism settings, and checkpointing frequencies. The results show up to 48$\times$ faster checkpointing and 2.2$\times$ faster end-to-end training runtime compared with the state-of-art checkpointing approaches.


title: MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception

abstract: Compared with an extensive list of automotive radar datasets that support autonomous driving, indoor radar datasets are scarce at a smaller scale in the format of low-resolution radar point clouds and usually under an open-space single-room setting. In this paper, we scale up indoor radar data collection using multi-view high-resolution radar heatmap in a multi-day, multi-room, and multi-subject setting, with an emphasis on the diversity of environment and subjects. Referred to as the millimeter-wave multi-view radar (MMVR) dataset, it consists of $345$K multi-view radar frames collected from $25$ human subjects over $6$ different rooms, $446$K annotated bounding boxes/segmentation instances, and $7.59$ million annotated keypoints to support three major perception tasks of object detection, pose estimation, and instance segmentation, respectively. For each task, we report performance benchmarks under two protocols: a single subject in an open space and multiple subjects in several cluttered rooms with two data splits: random split and cross-environment split over $395$ 1-min data segments. We anticipate that MMVR facilitates indoor radar perception development for indoor vehicle (robot/humanoid) navigation, building energy management, and elderly care for better efficiency, user experience, and safety.


title: Intraband collective excitations in fractional Chern insulators are dark

abstract: The low-energy collective excitations of semiconductors and insulators often couple strongly to light, allowing them to be probed optically. We argue here that in fractional Chern insulators intra-band collective excitations are dark in the sense that they couple anomalously weakly to light. This conclusion is based on a relationship between ideal quantum geometry and the structure factor of a Chern band, and on a classical plasma analogy motivated by the vortexibility property of ideal Chern bands.


title: SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task

abstract: Integrating Large Language Models (LLMs) with existing Knowledge Graph (KG) databases presents a promising avenue for enhancing LLMs' efficacy and mitigating their "hallucinations". Given that most KGs reside in graph databases accessible solely through specialized query languages (e.g., Cypher), there exists a critical need to bridge the divide between LLMs and KG databases by automating the translation of natural language into Cypher queries (commonly termed the "Text2Cypher" task). Prior efforts tried to bolster LLMs' proficiency in Cypher generation through Supervised Fine-Tuning. However, these explorations are hindered by the lack of annotated datasets of Query-Cypher pairs, resulting from the labor-intensive and domain-specific nature of annotating such datasets. In this study, we propose SyntheT2C, a methodology for constructing a synthetic Query-Cypher pair dataset, comprising two distinct pipelines: (1) LLM-based prompting and (2) template-filling. SyntheT2C facilitates the generation of extensive Query-Cypher pairs with values sampled from an underlying Neo4j graph database. Subsequently, SyntheT2C is applied to two medical databases, culminating in the creation of a synthetic dataset, MedT2C. Comprehensive experiments demonstrate that the MedT2C dataset effectively enhances the performance of backbone LLMs on the Text2Cypher task. Both the SyntheT2C codebase and the MedT2C dataset will be released soon.


title: Symmetry-driven embedding of networks in hyperbolic space

abstract: Hyperbolic models can reproduce the heavy-tailed degree distribution, high clustering, and hierarchical structure of empirical networks. Current algorithms for finding the hyperbolic coordinates of networks, however, do not quantify uncertainty in the inferred coordinates. We present BIGUE, a Markov chain Monte Carlo (MCMC) algorithm that samples the posterior distribution of a Bayesian hyperbolic random graph model. We show that combining random walk and random cluster transformations significantly improves mixing compared to the commonly used and state-of-the-art dynamic Hamiltonian Monte Carlo algorithm. Using this algorithm, we also provide evidence that the posterior distribution cannot be approximated by a multivariate normal distribution, thereby justifying the use of MCMC to quantify the uncertainty of the inferred parameters.


title: Object Detection using Oriented Window Learning Vi-sion Transformer: Roadway Assets Recognition

abstract: Object detection is a critical component of transportation systems, particularly for applications such as autonomous driving, traffic monitoring, and infrastructure maintenance. Traditional object detection methods often struggle with limited data and variability in object appearance. The Oriented Window Learning Vision Transformer (OWL-ViT) offers a novel approach by adapting window orientations to the geometry and existence of objects, making it highly suitable for detecting diverse roadway assets. This study leverages OWL-ViT within a one-shot learning framework to recognize transportation infrastructure components, such as traffic signs, poles, pavement, and cracks. This study presents a novel method for roadway asset detection using OWL-ViT. We conducted a series of experiments to evaluate the performance of the model in terms of detection consistency, semantic flexibility, visual context adaptability, resolution robustness, and impact of non-max suppression. The results demonstrate the high efficiency and reliability of the OWL-ViT across various scenarios, underscoring its potential to enhance the safety and efficiency of intelligent transportation systems.


title: Nonlocal cooperative behaviour, psychological effects, and collective decision-making: an exemplification with predator-prey models

abstract: In bio-social models, cooperative behaviour has evolved as an adaptive strategy, playing multi-functional roles. One of such roles in populations is to increase the success of survival and reproduction of individuals and their families or social groups. Moreover, collective decision-making in cooperative behaviour is an aspect that is used to study the dynamic behaviour of individuals within a social group. In this paper, we have focused on population dynamics by considering a predator-prey model as our main exemplification, where the generalist predator has adopted a cooperative hunting strategy while consuming their prey. In particular, we have analyzed the dynamic nature of the system when a nonlocal term is introduced in the cooperation. First, the Turing instability condition has been studied for the local model around the coexisting steady-state, followed by the Turing and non-Turing patterns in the presence of the nonlocal interaction term. This work is also concerned with the existence of travelling wave solutions for predator-prey interaction with the nonlocal cooperative hunting strategy. Such solutions are reported for local as well as for nonlocal models. We have characterized the invading speed of the predator with the help of the minimal wave speed of travelling wave solutions connecting the predator-free state to the co-existence state. The travelling waves are found to be non-monotonic in this system. The formation of wave trains has been demonstrated for an extended range of nonlocal interactions. Finally, the importance of psychological effects in shaping the dynamics of nonlocal collective behaviour is demonstrated with several representative examples.


title: Planning with Adaptive World Models for Autonomous Driving

abstract: Motion planning is crucial for safe navigation in complex urban environments. Historically, motion planners (MPs) have been evaluated with procedurally-generated simulators like CARLA. However, such synthetic benchmarks do not capture real-world multi-agent interactions. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic, effectively turning the fixed dataset into a reactive simulator. We analyze the characteristics of nuPlan's recorded logs and find that each city has its own unique driving behaviors, suggesting that robust planners must adapt to different environments. We learn to model such unique behaviors with BehaviorNet, a graph convolutional neural network (GCNN) that predicts reactive agent behaviors using features derived from recently-observed agent histories; intuitively, some aggressive agents may tailgate lead vehicles, while others may not. To model such phenomena, BehaviorNet predicts parameters of an agent's motion controller rather than predicting its spacetime trajectory (as most forecasters do). Finally, we present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions. Our extensive experiments demonstrate that AdaptiveDriver achieves state-of-the-art results on the nuPlan closed-loop planning benchmark, reducing test error from 6.4% to 4.6%, even when applied to never-before-seen cities.


title: Chip-scale generation of 60-mode continuous-variable cluster states

abstract: Increasing the number of entangled entities is crucial for achieving exponential computational speedups and secure quantum networks. Despite recent progress in generating large-scale entanglement through continuous-variable (CV) cluster states, translating these technologies to photonic chips has been hindered by decoherence, limiting the number of entangled entities to 8. Here, we demonstrate 60-mode CVcluster states in a chip-based optical microresonator pumped by chromatic lasers. Resonantly-enhanced four-wave mixing processes establish entanglement between equidistant spectral quantum modes (qumodes), forming a quantum analogue of optical frequency combs. Decoherence is minimized to achieve unprecedented two-mode raw squeezing (>3 dB) from a chip. Using bichromatic and trichromatic pump lasers, we realize one- and two-dimensional cluster states with up to 60 qumodes. Our work provides a compact and scalable platform for constructing large-scale entangled quantum resources, which are appealing for performing computational and communicational tasks with quantum advantages.


title: On two recent extensions of the Big Five of Reverse Mathematics

abstract: The program Reverse Mathematics in the foundations of mathematics seeks to identify the minimal axioms required to prove theorems of ordinary mathematics. One always assumes the base theory, a logical system embodying computable mathematics. As it turns out, many (most?) theorems are either provable in said base theory, or equivalent to one of four logical systems, collectively called the Big Five. This paper provides an overview of two recent extensions of the Big Five, working in Kohlenbach's higher-order framework. On one hand, we obtain a large number of equivalences between the second-order Big Five and third-order theorems of real analysis dealing with possibly discontinuous functions. On the other hand, we identify four new 'Big' systems, i.e. boasting many equivalences over the base theory, namely the uncountability of the reals, the Jordan decomposition theorem, the Baire category theorem, and Tao's pigeon hole principle for the Lebesgue measure. We discuss a connection to hyperarithmetical analysis, completing the picture.


title: Economical representation of spatial networks

abstract: Network visualization is essential for many scientific, societal, technological and artistic domains. The primary goal is to highlight patterns out of nodes interconnected by edges that are easy to understand, facilitate communication and support decision-making. This is typically achieved by rearranging the nodes to minimize the edge crossings responsible of unintelligible and often unaesthetic trends. But when the nodes cannot be moved, as in spatial and physical networks, this procedure is not viable. Here, we overcome this situation by turning the edge crossing problem into a graph filtering optimization. We demonstrate that the presence of longer connections prompt the optimal solution to yield sparser networks, thereby limiting the number of intersections and getting more readable layouts. This theoretical result matches human behavior and provides an ecologically-inspired criterion to visualize and model real-world interconnected systems.


title: Stacking for Probabilistic Short-term Load Forecasting

abstract: In this study, we delve into the realm of meta-learning to combine point base forecasts for probabilistic short-term electricity demand forecasting. Our approach encompasses the utilization of quantile linear regression, quantile regression forest, and post-processing techniques involving residual simulation to generate quantile forecasts. Furthermore, we introduce both global and local variants of meta-learning. In the local-learning mode, the meta-model is trained using patterns most similar to the query pattern.Through extensive experimental studies across 35 forecasting scenarios and employing 16 base forecasting models, our findings underscored the superiority of quantile regression forest over its competitors


title: Trading Devil: Robust backdoor attack via Stochastic investment models and Bayesian approach

abstract: With the growing use of voice-activated systems and speech recognition technologies, the danger of backdoor attacks on audio data has grown significantly. This research looks at a specific type of attack, known as a Stochastic investment-based backdoor attack (MarketBack), in which adversaries strategically manipulate the stylistic properties of audio to fool speech recognition systems. The security and integrity of machine learning models are seriously threatened by backdoor attacks, in order to maintain the reliability of audio applications and systems, the identification of such attacks becomes crucial in the context of audio data. Experimental results demonstrated that MarketBack is feasible to achieve an average attack success rate close to 100% in seven victim models when poisoning less than 1% of the training data.


title: Dampening Long-Period Doppler Shift Oscillations using Deep Machine Learning Techniques in the Solar Network and Internetwork

abstract: This study explores the Doppler shift at different wavelengths in the Interface Region Imaging Spectrograph (IRIS) solar spectrum and implements a comprehensive consideration of Doppler velocity oscillations in the IRIS channels. This comprehensive consideration reveals a propagating periodic perturbation in a large number of chromosphere and transition region (TR) bright points (BPs). To our knowledge, this is the first investigation of the longitudinal oscillations with damping in BPs using comprehensive consideration of the Doppler velocity at various wavelengths. The phenomena of attenuation in the red and blue Doppler shifts of the solar wavelength range were seen several times during the experiments. We utilized deep learning techniques to examine the statistical properties of damping in network and internetwork BPs, as well as active, quiet areas, and coronal hole areas. Our results revealed varying damping rates across different regions, with 80 percent of network BPs exhibiting damping in quiet areas and 72 in coronal hole areas. In active areas, the figure approached 33. For internetwork BPs, the values were 65, 54, and 63 percent for quiet areas, coronal hole areas, and active regions, respectively. The damping rate in active regions is twice as high at Internetwork's BPs. The damping components in this study were computed, and the findings show that the damping at all points is underdamped. The observed damping process suggests the propagation and leaking of energetic waves out of TR bright points, potentially contributing to the energy transport from the bright magnetic footpoints to the upper chromosphere, transition region, and corona.


title: RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics

abstract: From rearranging objects on a table to putting groceries into shelves, robots must plan precise action points to perform tasks accurately and reliably. In spite of the recent adoption of vision language models (VLMs) to control robot behavior, VLMs struggle to precisely articulate robot actions using language. We introduce an automatic synthetic data generation pipeline that instruction-tunes VLMs to robotic domains and needs. Using the pipeline, we train RoboPoint, a VLM that predicts image keypoint affordances given language instructions. Compared to alternative approaches, our method requires no real-world data collection or human demonstration, making it much more scalable to diverse environments and viewpoints. In addition, RoboPoint is a general model that enables several downstream applications such as robot navigation, manipulation, and augmented reality (AR) assistance. Our experiments demonstrate that RoboPoint outperforms state-of-the-art VLMs (GPT-4o) and visual prompting techniques (PIVOT) by 21.8% in the accuracy of predicting spatial affordance and by 30.5% in the success rate of downstream tasks. Project website: https://robo-point.github.io.


title: GenMM: Geometrically and Temporally Consistent Multimodal Data Generation for Video and LiDAR

abstract: Multimodal synthetic data generation is crucial in domains such as autonomous driving, robotics, augmented/virtual reality, and retail. We propose a novel approach, GenMM, for jointly editing RGB videos and LiDAR scans by inserting temporally and geometrically consistent 3D objects. Our method uses a reference image and 3D bounding boxes to seamlessly insert and blend new objects into target videos. We inpaint the 2D Regions of Interest (consistent with 3D boxes) using a diffusion-based video inpainting model. We then compute semantic boundaries of the object and estimate it's surface depth using state-of-the-art semantic segmentation and monocular depth estimation techniques. Subsequently, we employ a geometry-based optimization algorithm to recover the 3D shape of the object's surface, ensuring it fits precisely within the 3D bounding box. Finally, LiDAR rays intersecting with the new object surface are updated to reflect consistent depths with its geometry. Our experiments demonstrate the effectiveness of GenMM in inserting various 3D objects across video and LiDAR modalities.


title: Eye in the Sky: Detection and Compliance Monitoring of Brick Kilns using Satellite Imagery

abstract: Air pollution kills 7 million people annually. The brick manufacturing industry accounts for 8%-14% of air pollution in the densely populated Indo-Gangetic plain. Due to the unorganized nature of brick kilns, policy violation detection, such as proximity to human habitats, remains challenging. While previous studies have utilized computer vision-based machine learning methods for brick kiln detection from satellite imagery, they utilize proprietary satellite data and rarely focus on compliance with government policies. In this research, we introduce a scalable framework for brick kiln detection and automatic compliance monitoring. We use Google Maps Static API to download the satellite imagery followed by the YOLOv8x model for detection. We identified and hand-verified 19579 new brick kilns across 9 states within the Indo-Gangetic plain. Furthermore, we automate and test the compliance to the policies affecting human habitats, rivers and hospitals. Our results show that a substantial number of brick kilns do not meet the compliance requirements. Our framework offers a valuable tool for governments worldwide to automate and enforce policy regulations for brick kilns, addressing critical environmental and public health concerns.


title: Beyond the Visible: Jointly Attending to Spectral and Spatial Dimensions with HSI-Diffusion for the FINCH Spacecraft

abstract: Satellite remote sensing missions have gained popularity over the past fifteen years due to their ability to cover large swaths of land at regular intervals, making them ideal for monitoring environmental trends. The FINCH mission, a 3U+ CubeSat equipped with a hyperspectral camera, aims to monitor crop residue cover in agricultural fields. Although hyperspectral imaging captures both spectral and spatial information, it is prone to various types of noise, including random noise, stripe noise, and dead pixels. Effective denoising of these images is crucial for downstream scientific tasks. Traditional methods, including hand-crafted techniques encoding strong priors, learned 2D image denoising methods applied across different hyperspectral bands, or diffusion generative models applied independently on bands, often struggle with varying noise strengths across spectral bands, leading to significant spectral distortion. This paper presents a novel approach to hyperspectral image denoising using latent diffusion models that integrate spatial and spectral information. We particularly do so by building a 3D diffusion model and presenting a 3-stage training approach on real and synthetically crafted datasets. The proposed method preserves image structure while reducing noise. Evaluations on both popular hyperspectral denoising datasets and synthetically crafted datasets for the FINCH mission demonstrate the effectiveness of this approach.


title: A Conditional Upper Bound for the Moving Sofa Problem

abstract: The moving sofa problem asks for the connected shape with the largest area $\mu_{\text{max}}$ that can move around the right-angled corner of a hallway $L$ with unit width. The best bounds currently known on $\mu_{\max}$ are summarized as $2.2195\ldots \leq \mu_{\max} \leq 2.37$. The lower bound $2.2195\ldots \leq \mu_{\max}$ comes from Gerver's sofa $S_G$ of area $\mu_G := 2.2195\ldots$. The upper bound $\mu_{\max} \leq 2.37$ was proved by Kallus and Romik using extensive computer assistance. It is conjectured that the equality $\mu_{\max} = \mu_G$ holds at the lower bound. We develop a new approach to the moving sofa problem by approximating it as an infinite-dimensional convex quadratic optimization problem. The problem is then explicitly solved using a calculus of variation based on the Brunn-Minkowski theory. Consequently, we prove that any moving sofa satisfying a property named the injectivity condition has an area of at most $1 + \pi^2/8 = 2.2337\dots$. The new conditional bound does not rely on any computer assistance, yet it is much closer to the lower bound $2.2195\ldots$ of Gerver than the computer-assisted upper bound $2.37$ of Kallus and Romik. Gerver's sofa $S_G$, the conjectured optimum, satisfies the injectivity condition in particular.


title: Integer quadratic forms and extensions of subsets of linearly independent roots

abstract: We consider subsets of linearly independent roots in a certain root system $\varPhi$. Let $S'$ be such a subset, and let $S'$ be associated with any Carter diagram $\Gamma'$. The main question of the paper: what root $\gamma \in \varPhi$ can be added to $S'$ so that $S' \cup \gamma$ is also a subset of linearly independent roots? This extra root $\gamma$ is called the linkage root. The vector $\gamma^{\nabla}$ of inner products $\{(\gamma,\tau'_i)\mid \tau'_i \in S'\}$ is called the linkage label vector. Let $B_{\Gamma'}$ be the Cartan matrix associated with $\Gamma'$. It is shown that $\gamma$ is a linkage root if and only if $\mathscr{B}^{\vee}_{\Gamma'}(\gamma^{\nabla}) < 2$, where $\mathscr{B}^{\vee}_{\Gamma'}$ is a quadratic form with the matrix inverse to $B_{\Gamma'}$. The set of all linkage roots for $\Gamma'$ is called a linkage system and is denoted by $\mathscr{L}(\Gamma')$. The Cartan matrix associated with any Carter diagram $\Gamma'$ is conjugate to the Cartan matrix associated with some Dynkin diagram $\Gamma$, [St23]. The sizes of $\mathscr{L}(\Gamma')$ and $\mathscr{L}(\Gamma)$ are the same. Let $W^{\vee}$ be the Weyl group of the quadratic form $\mathscr{B}^{\vee}_{\Gamma'}$. This group acts on the linkage system and forms several orbits. The sizes and structure of orbits for linkage systems $\mathscr{L}(D_l)$ and $\mathscr{L}(D_l(a_k))$ are presented.


title: Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights

abstract: Given the ubiquity of graph data and its applications in diverse domains, building a Graph Foundation Model (GFM) that can work well across different graphs and tasks with a unified backbone has recently garnered significant interests. A major obstacle to achieving this goal stems from the fact that graphs from different domains often exhibit diverse node features. Inspired by multi-modal models that align different modalities with natural language, the text has recently been adopted to provide a unified feature space for diverse graphs. Despite the great potential of these text-space GFMs, current research in this field is hampered by two problems. First, the absence of a comprehensive benchmark with unified problem settings hinders a clear understanding of the comparative effectiveness and practical value of different text-space GFMs. Second, there is a lack of sufficient datasets to thoroughly explore the methods' full potential and verify their effectiveness across diverse settings. To address these issues, we conduct a comprehensive benchmark providing novel text-space datasets and comprehensive evaluation under unified problem settings. Empirical results provide new insights and inspire future research directions. Our code and data are publicly available from \url{https://github.com/CurryTang/TSGFM}.


title: Breaking the Memory Wall: A Study of I/O Patterns and GPU Memory Utilization for Hybrid CPU-GPU Offloaded Optimizers

abstract: Transformers and LLMs have seen rapid adoption in all domains. Their sizes have exploded to hundreds of billions of parameters and keep increasing. Under these circumstances, the training of transformers is slow and often takes in the order of weeks or months. Thanks to 3D model parallelism (data, pipeline, and tensor-level parallelism), the training can scale to a large number of GPUs, which reduces the duration of the training but dramatically increases the cost. Even when a large number of GPUs are available, the aggregated GPU memory is often not enough to hold the full training state (optimizer state, model parameters, and gradients). To compensate, state-of-the-art approaches offload the optimizer state at least partially to the host memory and perform hybrid CPU-GPU computations. Such flexible solutions dramatically reduce the GPU memory utilization, which makes it feasible to run the training on a smaller number of GPUs at the cost of performance penalty. Unfortunately, the challenges and bottlenecks of adopting this strategy are not sufficiently studied by state-of-the-art, which results in poor management of the combined host-GPU memory and poor overlapping between data movements and computations. In this paper, we aim to fill this gap by characterizing the behavior of offloaded training using the DeepSpeed runtime. Specifically, we study the GPU memory utilization over time during each iteration, the activity on the PCIe related to transfers between the host memory and the GPU memory, and the relationship between resource utilization and the steps involved in each iteration. Thanks to this study, we reveal opportunities for future improvements of offloading solutions, which enable greater flexibility to optimize the cost-performance trade-off in the context of transformer and LLM training.


title: A Comprehensive Survey of Foundation Models in Medicine

abstract: Foundation models (FMs) are large-scale deep-learning models trained on extensive datasets using self-supervised techniques. These models serve as a base for various downstream tasks, including healthcare. FMs have been adopted with great success across various domains within healthcare, including natural language processing (NLP), computer vision, graph learning, biology, and omics. Existing healthcare-based surveys have not yet included all of these domains. Therefore, this survey provides a comprehensive overview of FMs in healthcare. We focus on the history, learning strategies, flagship models, applications, and challenges of FMs. We explore how FMs such as the BERT and GPT families are reshaping various healthcare domains, including clinical large language models, medical image analysis, and omics data. Furthermore, we provide a detailed taxonomy of healthcare applications facilitated by FMs, such as clinical NLP, medical computer vision, graph learning, and other biology-related tasks. Despite the promising opportunities FMs provide, they also have several associated challenges, which are explained in detail. We also outline potential future directions to provide researchers and practitioners with insights into the potential and limitations of FMs in healthcare to advance their deployment and mitigate associated risks.


title: Order-theoretic models for decision-making: Learning, optimization, complexity and computation

abstract: The study of intelligent systems explains behaviour in terms of economic rationality. This results in an optimization principle involving a function or utility, which states that the system will evolve until the configuration of maximum utility is achieved. Recently, this theory has incorporated constraints, i.e., the optimum is achieved when the utility is maximized while respecting some information-processing constraints. This is reminiscent of thermodynamic systems. As such, the study of intelligent systems has benefited from the tools of thermodynamics. The first aim of this thesis is to clarify the applicability of these results in the study of intelligent systems. We can think of the local transition steps in thermodynamic or intelligent systems as being driven by uncertainty. In fact, the transitions in both systems can be described in terms of majorization. Hence, real-valued uncertainty measures like Shannon entropy are simply a proxy for their more involved behaviour. More in general, real-valued functions are fundamental to study optimization and complexity in the order-theoretic approach to several topics, including economics, thermodynamics, and quantum mechanics. The second aim of this thesis is to improve on this classification. The basic similarity between thermodynamic and intelligent systems is based on an uncertainty notion expressed by a preorder. We can also think of the transitions in the steps of a computational process as a decision-making procedure. In fact, by adding some requirements on the considered order structures, we can build an abstract model of uncertainty reduction that allows to incorporate computability, that is, to distinguish the objects that can be constructed by following a finite set of instructions from those that cannot. The third aim of this thesis is to clarify the requirements on the order structure that allow such a framework.


title: Anisotropic Heating and Parallel Heat Flux in Electron-only Magnetic Reconnection with Intense Guide Fields

abstract: Electron-only reconnection is a process recently observed in the Earth's magnetosheath, where magnetic reconnection occurs at electron kinetic scales with only electrons involved. Electron-only reconnection is likely to have a significant impact on the energy conversion and dissipation of turbulence cascades at kinetic scales. This paper investigates electron-only reconnection under different intensities of strong guide fields via two-dimensional fully kinetic Particle-in-Cell (PIC) simulations, focusing on electron heating. The simulations are initialized with a force-free current sheet equilibrium under various intensities of strong guide fields. The quadrupolar structures of the out-of-plane magnetic field and asymmetric structures of density are found to become narrower in space as the guide field increases. Electron velocities are considerably larger than ion velocities, and the motion of both species is notably affected by the strength of guide fields. Similarly to previous experiments studies, electron temperature anisotropy along separatrices is observed, which is found to be mainly caused by the variations of parallel temperature. Both regions of anisotropy and parallel temperature increase/decrease along separatrices become thinner with increasing guide fields. Besides, we find a transition from a quadrupolar to a six-polar to an eight-polar structure in temperature anisotropy and parallel temperature as the guide field intensifies. Non-Maxwellian electron velocity distribution functions (EVDFs) at different locations in the three simulations are observed. Our results show that parallel electron velocity varies notably with different guide field intensities and strong parallel electron heat flux is observed.


title: Character Animation in AR: Character Animation in AR: a mobile application development study

abstract: Digital preservation of the cultural heritages is one of the major applications of various computer graphics and vision algorithms. The advancement in the AR/VR technologies is giving the cultural heritage preservation an interesting spin due to its immense visualization ability. The use of these technologies to digitally recreate heritage sites and art is becoming a popular trend. A project, called Indian Digital Heritage (IDH), for recreating the heritage site of Hampi, Karnataka during Vijaynagara empire ($1336$ - $1646$ CE) has been initiated by the Department of Science and Technology (DST) few years back. Immense work on surveying the site, collecting geographical and historic information about the life in Hampi, creating 3D models for buildings and people of Hampi and many other related tasks has been undertaken by various participants of this project. A major part of this project is to make tourists visiting Hampi visualize the life of people in ancient Hampi through any handy device. With such a requirement, the mobile AR based platform becomes a natural choice for developing any application for this purpose. We contributed to the project by developing an AR based mobile application to recreate a scene from Virupaksha Bazaar of Hampi with two components - author scene with augmented virtual contents at any scale, and visualize the same scene by reaching the physical location of augmentation. We develop an interactive application for the purpose of digitally recreating ancient Hampi. Though the focus of this work is not creating any static or dynamic content from scratch, it shows an interesting application of the content created in a real world scenario.


title: A Laplace transform-based test for the equality of positive semidefinite matrix distributions

abstract: In this paper, we present a novel test for determining equality in distribution of matrix distributions. Our approach is based on the integral squared difference of the empirical Laplace transforms with respect to the noncentral Wishart measure. We conduct an extensive power study to assess the performance of the test and determine the optimal choice of parameters. Furthermore, we demonstrate the applicability of the test on financial and non-life insurance data, illustrating its effectiveness in practical scenarios.


title: Polynomial Chaos-based Stochastic Model Predictive Control: An Overview and Future Research Directions

abstract: This article is devoted to providing a review of mathematical formulations in which Polynomial Chaos Theory (PCT) has been incorporated into stochastic model predictive control (SMPC). In the past decade, PCT has been shown to provide a computationally tractable way to perform complete and accurate uncertainty propagation through (smooth) nonlinear dynamic systems. As such, it represents a very useful computational tool for accelerating the computations needed in SMPC with time invariant uncertainties. It turns out that it can also be used to reduce complexity of chance constraints, which are an important component of SMPC. In this paper, we provide an overview of PCT and discuss how it can be applied in such time invariant settings.


title: How Should We Extract Discrete Audio Tokens from Self-Supervised Models?

abstract: Discrete audio tokens have recently gained attention for their potential to bridge the gap between audio and language processing. Ideal audio tokens must preserve content, paralinguistic elements, speaker identity, and many other audio details. Current audio tokenization methods fall into two categories: Semantic tokens, acquired through quantization of Self-Supervised Learning (SSL) models, and Neural compression-based tokens (codecs). Although previous studies have benchmarked codec models to identify optimal configurations, the ideal setup for quantizing pretrained SSL models remains unclear. This paper explores the optimal configuration of semantic tokens across discriminative and generative tasks. We propose a scalable solution to train a universal vocoder across multiple SSL layers. Furthermore, an attention mechanism is employed to identify task-specific influential layers, enhancing the adaptability and performance of semantic tokens in diverse audio applications.


title: Coherent amplitude modulation of continuous-wave light in cesium vapor

abstract: We report on observations of coherent, sustained oscillations in the absorption of continuous-wave light at 388 nm that excites the $6S_{1/2}\rightarrow 8P_{3/2}$ transition in cesium vapor. The oscillation frequency is close to the spacing of hyperfine levels of the $8P_{3/2}$ level that are excited simultaneously by the 388 nm field. We observe threshold behavior of the oscillation amplitude with pump power, and suggest that the effect is associated with infrared directional emission due to amplified spontaneous emission from the $8P_{3/2}\rightarrow 8S_{1/2}$ transition, that is assisted by retro-reflections from the cell windows. The effect may be used to probe a lasing process in an atomic vapor, by checking the temporal properties of the pump field transmitted through the vapor.


title: Dynamic Domains, Dynamic Solutions: DPCore for Continual Test-Time Adaptation

abstract: Continual Test-Time Adaptation (TTA) seeks to adapt a source pre-trained model to continually changing, unlabeled target domains. Existing TTA methods are typically designed for environments where domain changes occur gradually and can struggle in more dynamic scenarios. Inspired by the principles of online K-Means, this paper introduces a novel approach to continual TTA through visual prompting. We propose a Dynamic Prompt Coreset that not only preserves knowledge from previously visited domains but also accommodates learning from new potential domains. This is complemented by a distance-based weight updating mechanism that ensures the coreset remains current and relevant. Our approach employs a fixed model architecture alongside the coreset and an innovative updating system to effectively mitigate challenges such as catastrophic forgetting and error accumulation. Extensive testing across various benchmarks-including ImageNet-C, CIFAR100-C, and CIFAR10-C-demonstrates that our method consistently outperforms state-of-the-art (SOTA) alternatives, particularly excelling in dynamically changing environments.


title: Adaptive Experimentation When You Can't Experiment

abstract: This paper introduces the \emph{confounded pure exploration transductive linear bandit} (\texttt{CPET-LB}) problem. As a motivating example, often online services cannot directly assign users to specific control or treatment experiences either for business or practical reasons. In these settings, naively comparing treatment and control groups that may result from self-selection can lead to biased estimates of underlying treatment effects. Instead, online services can employ a properly randomized encouragement that incentivizes users toward a specific treatment. Our methodology provides online services with an adaptive experimental design approach for learning the best-performing treatment for such \textit{encouragement designs}. We consider a more general underlying model captured by a linear structural equation and formulate pure exploration linear bandits in this setting. Though pure exploration has been extensively studied in standard adaptive experimental design settings, we believe this is the first work considering a setting where noise is confounded. Elimination-style algorithms using experimental design methods in combination with a novel finite-time confidence interval on an instrumental variable style estimator are presented with sample complexity upper bounds nearly matching a minimax lower bound. Finally, experiments are conducted that demonstrate the efficacy of our approach.


title: A stochastic field theory for the evolution of quantitative traits in finite populations

abstract: Infinitely many distinct trait values may arise in populations bearing quantitative traits, and modelling their population dynamics is thus a formidable task. While classical models assume fixed or infinite population size, models in which the total population size fluctuates due to demographic noise in births and deaths can behave qualitatively differently from constant or infinite population models due to density-dependent dynamics. In this paper, I present a stochastic field theory for the eco-evolutionary dynamics of finite populations bearing one-dimensional quantitative traits. I derive stochastic field equations that describe the evolution of population densities, trait frequencies, and the mean value of any trait in the population. These equations recover well-known results such as the replicator-mutator equation, Price equation, and gradient dynamics in the infinite population limit. For finite populations, the equations describe the intricate interplay between natural selection, noise-induced selection, eco-evolutionary feedback, and neutral genetic drift in determining evolutionary trajectories. My methods use ideas from statistical physics and present an alternative to some recently proposed measure-theoretic frameworks.

categories: q-bio.PE cond-mat.stat-mech math.PR physics.bio-ph


title: FreeMotion: MoCap-Free Human Motion Synthesis with Multimodal Large Language Models

abstract: Human motion synthesis is a fundamental task in computer animation. Despite recent progress in this field utilizing deep learning and motion capture data, existing methods are always limited to specific motion categories, environments, and styles. This poor generalizability can be partially attributed to the difficulty and expense of collecting large-scale and high-quality motion data. At the same time, foundation models trained with internet-scale image and text data have demonstrated surprising world knowledge and reasoning ability for various downstream tasks. Utilizing these foundation models may help with human motion synthesis, which some recent works have superficially explored. However, these methods didn't fully unveil the foundation models' potential for this task and only support several simple actions and environments. In this paper, we for the first time, without any motion data, explore open-set human motion synthesis using natural language instructions as user control signals based on MLLMs across any motion task and environment. Our framework can be split into two stages: 1) sequential keyframe generation by utilizing MLLMs as a keyframe designer and animator; 2) motion filling between keyframes through interpolation and motion tracking. Our method can achieve general human motion synthesis for many downstream tasks. The promising results demonstrate the worth of mocap-free human motion synthesis aided by MLLMs and pave the way for future research.


title: Speech Emotion Recognition Using CNN and Its Use Case in Digital Healthcare

abstract: The process of identifying human emotion and affective states from speech is known as speech emotion recognition (SER). This is based on the observation that tone and pitch in the voice frequently convey underlying emotion. Speech recognition includes the ability to recognize emotions, which is becoming increasingly popular and in high demand. With the help of appropriate factors (such modalities, emotions, intensities, repetitions, etc.) found in the data, my research seeks to use the Convolutional Neural Network (CNN) to distinguish emotions from audio recordings and label them in accordance with the range of different emotions. I have developed a machine learning model to identify emotions from supplied audio files with the aid of machine learning methods. The evaluation is mostly focused on precision, recall, and F1 score, which are common machine learning metrics. To properly set up and train the machine learning framework, the main objective is to investigate the influence and cross-relation of all input and output parameters. To improve the ability to recognize intentions, a key condition for communication, I have evaluated emotions using my specialized machine learning algorithm via voice that would address the emotional state from voice with the help of digital healthcare, bridging the gap between human and artificial intelligence (AI).


title: Spuriousness-Aware Meta-Learning for Learning Robust Classifiers

abstract: Spurious correlations are brittle associations between certain attributes of inputs and target variables, such as the correlation between an image background and an object class. Deep image classifiers often leverage them for predictions, leading to poor generalization on the data where the correlations do not hold. Mitigating the impact of spurious correlations is crucial towards robust model generalization, but it often requires annotations of the spurious correlations in data -- a strong assumption in practice. In this paper, we propose a novel learning framework based on meta-learning, termed SPUME -- SPUriousness-aware MEta-learning, to train an image classifier to be robust to spurious correlations. We design the framework to iteratively detect and mitigate the spurious correlations that the classifier excessively relies on for predictions. To achieve this, we first propose to utilize a pre-trained vision-language model to extract text-format attributes from images. These attributes enable us to curate data with various class-attribute correlations, and we formulate a novel metric to measure the degree of these correlations' spuriousness. Then, to mitigate the reliance on spurious correlations, we propose a meta-learning strategy in which the support (training) sets and query (test) sets in tasks are curated with different spurious correlations that have high degrees of spuriousness. By meta-training the classifier on these spuriousness-aware meta-learning tasks, our classifier can learn to be invariant to the spurious correlations. We demonstrate that our method is robust to spurious correlations without knowing them a priori and achieves the best on five benchmark datasets with different robustness measures.


title: Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?

abstract: Deep Learning is often depicted as a trio of data-architecture-loss. Yet, recent Self Supervised Learning (SSL) solutions have introduced numerous additional design choices, e.g., a projector network, positive views, or teacher-student networks. These additions pose two challenges. First, they limit the impact of theoretical studies that often fail to incorporate all those intertwined designs. Second, they slow-down the deployment of SSL methods to new domains as numerous hyper-parameters need to be carefully tuned. In this study, we bring forward the surprising observation that--at least for pretraining datasets of up to a few hundred thousands samples--the additional designs introduced by SSL do not contribute to the quality of the learned representations. That finding not only provides legitimacy to existing theoretical studies, but also simplifies the practitioner's path to SSL deployment in numerous small and medium scale settings. Our finding answers a long-lasting question: the often-experienced sensitivity to training settings and hyper-parameters encountered in SSL come from their design, rather than the absence of supervised guidance.


title: Technique Report of CVPR 2024 PBDL Challenges

abstract: The intersection of physics-based vision and deep learning presents an exciting frontier for advancing computer vision technologies. By leveraging the principles of physics to inform and enhance deep learning models, we can develop more robust and accurate vision systems. Physics-based vision aims to invert the processes to recover scene properties such as shape, reflectance, light distribution, and medium properties from images. In recent years, deep learning has shown promising improvements for various vision tasks, and when combined with physics-based vision, these approaches can enhance the robustness and accuracy of vision systems. This technical report summarizes the outcomes of the Physics-Based Vision Meets Deep Learning (PBDL) 2024 challenge, held in CVPR 2024 workshop. The challenge consisted of eight tracks, focusing on Low-Light Enhancement and Detection as well as High Dynamic Range (HDR) Imaging. This report details the objectives, methodologies, and results of each track, highlighting the top-performing solutions and their innovative approaches.


title: Strong Brandt-Thomass\'e Theorems

abstract: Solving a long standing conjecture of Erd\H{o}s and Simonovits, Brandt and Thomass\'e proved that the chromatic number of each triangle-free graph $G$ such that $\delta(G)>|V(G)|/3$ is at most four. In fact, they showed the much stronger result that every maximal triangle-free graph $G$ satisfying this minimum degree condition is a blow-up of either an Andr\'asfai or a Vega graph. Here we establish the same structural conclusion on $G$ under the weaker assumption that for $m\in\{2, 3, 4\}$ every sequence of $3m$ vertices has a subsequence of length $m+1$ with a common neighbour. In forthcoming work this will be used to solve an old problem of Andr\'asfai in Ramsey-Tur\'an theory.


title: SparseCL: Sparse Contrastive Learning for Contradiction Retrieval

abstract: Contradiction retrieval refers to identifying and extracting documents that explicitly disagree with or refute the content of a query, which is important to many downstream applications like fact checking and data cleaning. To retrieve contradiction argument to the query from large document corpora, existing methods such as similarity search and crossencoder models exhibit significant limitations. The former struggles to capture the essence of contradiction due to its inherent nature of favoring similarity, while the latter suffers from computational inefficiency, especially when the size of corpora is large. To address these challenges, we introduce a novel approach: SparseCL that leverages specially trained sentence embeddings designed to preserve subtle, contradictory nuances between sentences. Our method utilizes a combined metric of cosine similarity and a sparsity function to efficiently identify and retrieve documents that contradict a given query. This approach dramatically enhances the speed of contradiction detection by reducing the need for exhaustive document comparisons to simple vector calculations. We validate our model using the Arguana dataset, a benchmark dataset specifically geared towards contradiction retrieval, as well as synthetic contradictions generated from the MSMARCO and HotpotQA datasets using GPT-4. Our experiments demonstrate the efficacy of our approach not only in contradiction retrieval with more than 30% accuracy improvements on MSMARCO and HotpotQA across different model architectures but also in applications such as cleaning corrupted corpora to restore high-quality QA retrieval. This paper outlines a promising direction for improving the accuracy and efficiency of contradiction retrieval in large-scale text corpora.


title: Threshold bounce -- occupancy-dependent modulation of the discriminating threshold in silicon detectors

abstract: The front-end electronics of silicon detectors are typically designed to ensure optimal noise performance for the expected input charge. A combination of preamplifiers and shaper circuits result in a nontrivial response of the front-end to injected charge, and the magnitude of the response may be sizeable in readout windows subsequent to that in which the charge was initially injected. The modulation of the discriminator threshold due to the superposition of the front-end response across multiple readout windows is coined "threshold bounce". In this paper, we report a measurement of threshold bounce using silicon modules built for the Phase-II Upgrade of the ATLAS detector at the Large Hadron Collider. These modules utilize ATLAS Binary Chips for their hit readout. The measurement was performed using a micro-focused 15 keV photon beam at the Diamond Light Source synchrotron. The effect of the choice of photon flux and discriminator threshold on the magnitude of the threshold bounce is studied. A Monte Carlo simulation which accounts for the front-end behaviour of the silicon modules is developed, and its predicted hit efficiency is found to be in good agreement with the measured hit efficiency.


title: Combinatorial properties of Circulant Hadamard matrices

abstract: We describe combinatorial properties of the defining row of a circulant Hadamard matrix by exploiting its orthogonality to subsequent rows, and show how to exclude several particular forms of these matrices.


title: Unique continuation of Schr\"odinger-type equations for $\bar\partial$ II

abstract: In this paper, we extend our earlier unique continuation results \cite{PZ2} for the Schr\"odinger-type inequality $ |\bar\partial u| \le V|u|$ on a domain in $\mathbb C^n$ by removing the smoothness assumption on solutions $u = (u_1, \ldots, u_N)$. More specifically, we establish the unique continuation property for $W_{loc}^{1,1}$ solutions when the potential $V\in L_{loc}^p $, $ p>2n$; and for $W_{loc}^{1,2n+\epsilon}$ solutions when $V\in L_{loc}^{2n}$ with $N=1$ or $n = 2$. Although the unique continuation property fails in general if $V\in L_{loc}^{p}, p<2n$, we show that the property still holds for $W_{loc}^{1,1}$ solutions when $V $ is a small constant multiple of $ \frac{1}{|z|}$.


title: EchoGuide: Active Acoustic Guidance for LLM-Based Eating Event Analysis from Egocentric Videos

abstract: Self-recording eating behaviors is a step towards a healthy lifestyle recommended by many health professionals. However, the current practice of manually recording eating activities using paper records or smartphone apps is often unsustainable and inaccurate. Smart glasses have emerged as a promising wearable form factor for tracking eating behaviors, but existing systems primarily identify when eating occurs without capturing details of the eating activities (E.g., what is being eaten). In this paper, we present EchoGuide, an application and system pipeline that leverages low-power active acoustic sensing to guide head-mounted cameras to capture egocentric videos, enabling efficient and detailed analysis of eating activities. By combining active acoustic sensing for eating detection with video captioning models and large-scale language models for retrieval augmentation, EchoGuide intelligently clips and analyzes videos to create concise, relevant activity records on eating. We evaluated EchoGuide with 9 participants in naturalistic settings involving eating activities, demonstrating high-quality summarization and significant reductions in video data needed, paving the way for practical, scalable eating activity tracking.


title: Strong field behavior of Wang-Yau Quasi-local energy

abstract: We look at the strong field behavior of the Wang-Yau quasi-local energy. In particular, we examine the limit of the Wang-Yau quasi-local energy as the defining spacelike 2-surface {\Sigma} approaches an apparent horizon from outside. Assuming any smooth isometric embedding, we find that the Wang-Yau quasi-local energy blows up in this limit, unless the horizon can be isometrically embedded into R3. Moreover, if the horizon cannot be embedded into R3, the optimal embedding equation is not solvable for {\Sigma} near the horizon. We discuss the implications of our results in the conclusion section.


title: On The Pursuit of EFX for Chores: Non-Existence and Approximations

abstract: We study the problem of fairly allocating a set of chores to a group of agents. The existence of envy-free up to any item (EFX) allocations is a long-standing open question for both goods and chores. We resolve this question by providing a negative answer for the latter, presenting a simple construction that admits no EFX solutions for allocating six items to three agents equipped with superadditive cost functions, thus proving a separation result between goods and bads. In fact, we uncover a deeper insight, showing that the instance has unbounded approximation ratio. Moreover, we show that deciding whether an EFX allocation exists is NP-complete. On the positive side, we establish the existence of EFX allocations under general monotone cost functions when the number of items is at most $n+2$. We then shift our attention to additive cost functions. We employ a general framework in order to improve the approximation guarantees in the well-studied case of three additive agents, and provide several conditional approximation bounds that leverage ordinal information.


title: Testing the parametric model for self-interacting dark matter using matched halos in cosmological simulations

abstract: We systemically evaluate the performance of the self-interacting dark matter (SIDM) halo model proposed in arXiv:2305.16176 with matched halos from high-resolution cosmological CDM and SIDM simulations. The model incorporates SIDM effects along mass evolution histories of CDM halos and it is applicable to both isolated halos and suhbhalos. We focus on the accuracy of the model in predicting halo density profiles at $z=0$ and the evolution of maximum circular velocity. We find the model predictions agree with the simulations within $10\%-50\%$ for most of the simulated (sub)halos, $50\%-100\%$ for extreme cases. This indicates that the model effectively captures the gravothermal evolution of the halos with very strong, velocity-dependent self-interactions. For an example application, we apply the model to study the impact of various SIDM scenarios on strong lensing perturber systems, demonstrating its utility in predicting SIDM effects for small-scale structure analyses. Our findings confirm that the model is an effective tool for mapping CDM halos into their SIDM counterparts.


title: Electronic and magnetic properties of graphene-fluorographene nanoribbons: controllable semiconductor-metal transition

abstract: We investigate the electronic and magnetic properties of graphene channels ($2$-$4$ nm wide) embedded within fluorographene, focusing on two distinct interfaces: the fully-fluorinated $\alpha$ interface and the half-fluorinated $\beta$ interface. Density Functional Theory (DFT) calculations reveal that $\alpha\alpha$ systems exhibit semiconducting behavior with antiferromagnetic ordering, closely resembling pristine zigzag graphene nanoribbons. In contrast, $\alpha\beta$ systems display ferromagnetism and a width-dependent semiconductor-to-metal transition. To enable the study of larger systems, we develop and validate effective Hubbard models for both $\alpha\alpha$ and $\alpha\beta$ channels. Building upon DFT results and a Wannier function analysis, these models accurately reproduce the electronic structure and magnetic ordering observed in DFT calculations. Crucially, our $\alpha\beta$ model successfully captures the semiconductor-to-metal transition. Application of this model to larger systems reveals the persistence of a ferromagnetic state with spin polarization localized at the $\alpha$ edge. Our results demonstrate the potential of fluorination for targeted property engineering and provide a basis for exploring graphene-fluorographene systems in device applications ranging from microelectronics to spintronics.


title: Role of the symmetry energy slope in neutron stars: exploring the model-dependency

abstract: Using six different parametrizations of the quantum hadrodynamics (one of which is original), I study how different values of the symmetry energy slope ($L)$ affect some microscopic and macroscopic properties of neutron stars, such as the proton fraction, the maximum mass, the radius of the canonical 1.4$M_\odot$ star and its dimensionless tidal parameter $\Lambda$. I show that while most quantities present the same qualitative results, the tidal parameter can increase or decrease with the slope, depending on the model. Moreover, special attention is given to the minimum mass that enables the direct URCA process to occur in neutron stars' interiors ($M_{DU})$. Assuming the weak constraint $M_{DU}~>~1.35M_\odot$, we see that the maximum value of $L$ that satisfies it lies between 79 and 86 MeV. A range of only 7 MeV. Therefore, $M_{DU}$ is an easy way to impose upper bounds to the slope.


title: Astronomical Spectroscopy with Skipper CCDs: First Results from a Skipper CCD Focal Plane Prototype at SIFS

abstract: We present the first on-sky results from an ultra-low-readout-noise Skipper CCD focal plane prototype for the SOAR Integral Field Spectrograph (SIFS). The Skipper CCD focal plane consists of four 6k x 1k, 15 $\mu$m pixel, fully-depleted, p-channel devices that have been thinned to ~250 $\mu$m, backside processed, and treated with an anti-reflective coating. These Skipper CCDs were configured for astronomical spectroscopy, i.e., single-sample readout noise < 4.3 e- rms/pixel, the ability to achieve multi-sample readout noise $\ll$ 1 e- rms/pixel, full-well capacities ~40,000-65,000 e-, low dark current and charge transfer inefficiency (~2 x 10$^{-4}$ e-/pixel/s and 3.44 x 10$^{-7}$, respectively), and an absolute quantum efficiency of $\gtrsim$ 80% between 450 nm and 980 nm ($\gtrsim$ 90% between 600 nm and 900 nm). We optimized the readout sequence timing to achieve sub-electron noise (~0.5 e- rms/pixel) in a region of 2k x 4k pixels and photon-counting noise (~0.22 e- rms/pixel) in a region of 220 x 4k pixels, each with a readout time of $\lesssim$ 17 min. We observed two quasars (HB89 1159+123 and QSO J1621-0042) at redshift z ~ 3.5, two high-redshift galaxy clusters (CL J1001+0220 and SPT-CL J2040-4451), an emission line galaxy at z = 0.3239, a candidate member star of the Bo\"{o}tes II ultra-faint dwarf galaxy, and five CALSPEC spectrophotometric standard stars (HD074000, HD60753, HD106252, HD101452, HD200654). We present charge-quantized, photon-counting observations of the quasar HB89 1159+123 and show the detector sensitivity increase for faint spectral features. We demonstrate signal-to-noise performance improvements for SIFS observations in the low-background, readout-noise-dominated regime. We outline scientific studies that will leverage the SIFS-Skipper CCD data and new detector architectures that utilize the Skipper floating gate amplifier with faster readout times.


title: Searching for cosmological stochastic backgrounds by notching out resolvable compact binary foregrounds with next-generation gravitational-wave detectors

abstract: Stochastic gravitational-wave backgrounds can be of either cosmological or astrophysical origin. The detection of an astrophysical stochastic gravitational-wave background with ground-based interferometers is expected in the near future. Perhaps even more excitingly, the detection of stochastic backgrounds of cosmological origin by future ground-based interferometers could reveal invaluable information about the early Universe. From this perspective, the astrophysical background is a {\it foreground} that can prevent the extraction of this information from the data. In this paper, we revisit a time-frequency domain notching procedure previously proposed to remove the astrophysical foreground in the context of next-generation ground-based detectors, but we consider the more realistic scenario where we remove individually detectable signals by taking into account the uncertainty in the estimation of their parameters. We find that time-frequency domain masks can still efficiently remove the astrophysical foreground and suppress it to about $5\%$ of its original level. Further removal of the foreground formed by unresolvable events (in particular, unresolvable binary neutron stars), which is about $10$ times larger than the residual foreground from realistic notching, would require detector sensitivity improvements. Therefore, the main limitation in the search for a cosmological background is the unresolvable foreground itself, and not the residual of the notching procedure.


title: Finite-difference least square methods for solving Hamilton-Jacobi equations using neural networks

abstract: We present a simple algorithm to approximate the viscosity solution of Hamilton-Jacobi~(HJ) equations by means of an artificial deep neural network. The algorithm uses a stochastic gradient descent-based algorithm to minimize the least square principle defined by a monotone, consistent numerical scheme. We analyze the least square principle's critical points and derive conditions that guarantee that any critical point approximates the sought viscosity solution. The use of a deep artificial neural network on a finite difference scheme lifts the restriction of conventional finite difference methods that rely on computing functions on a fixed grid. This feature makes it possible to solve HJ equations posed in higher dimensions where conventional methods are infeasible. We demonstrate the efficacy of our algorithm through numerical studies on various canonical HJ equations across different dimensions, showcasing its potential and versatility.


title: Humanoid Parkour Learning

abstract: Parkour is a grand challenge for legged locomotion, even for quadruped robots, requiring active perception and various maneuvers to overcome multiple challenging obstacles. Existing methods for humanoid locomotion either optimize a trajectory for a single parkour track or train a reinforcement learning policy only to walk with a significant amount of motion references. In this work, we propose a framework for learning an end-to-end vision-based whole-body-control parkour policy for humanoid robots that overcomes multiple parkour skills without any motion prior. Using the parkour policy, the humanoid robot can jump on a 0.42m platform, leap over hurdles, 0.8m gaps, and much more. It can also run at 1.8m/s in the wild and walk robustly on different terrains. We test our policy in indoor and outdoor environments to demonstrate that it can autonomously select parkour skills while following the rotation command of the joystick. We override the arm actions and show that this framework can easily transfer to humanoid mobile manipulation tasks. Videos can be found at https://humanoid4parkour.github.io


title: Joint parameter estimations for spin glasses

abstract: Spin glass models with quadratic-type Hamiltonians are disordered statistical physics systems with competing ferromagnetic and anti-ferromagnetic spin interactions. The corresponding Gibbs measures belong to the exponential family parametrized by (inverse) temperature $\beta>0$ and external field $h\in\mathbb{R}$. Given a sample from these Gibbs measures, a statistically fundamental question is to infer the temperature and external field parameters. In 2007, Chatterjee (Ann. Statist. 35 (2007), no.5, 1931-1946) first proved that in the absence of external field $h=0$, the maximum pseudolikelihood estimator for $\beta$ is $\sqrt{N}$-consistent under some mild assumptions on the disorder matrices. It was left open whether the same method can be used to estimate the temperature and external field simultaneously. In this paper, under some easily verifiable conditions, we prove that the bivariate maximum pseudolikelihood estimator is indeed jointly $\sqrt{N}$-consistent for the temperature and external field parameters. The examples cover the classical Sherrington-Kirkpatrick model and its diluted variants.

categories: math.PR cond-mat.dis-nn math-ph math.MP math.ST stat.TH


title: A note on best n-term approximation for generalized Wiener classes

abstract: We determine the best n-term approximation of generalized Wiener model classes in a Hilbert space $H $. This theory is then applied to several special cases.


title: Analysis and approximation of elliptic problems with Uhlenbeck structure in convex polytopes

abstract: We prove the well posedness in weighted Sobolev spaces of certain linear and nonlinear elliptic boundary value problems posed on convex domains and under singular forcing. It is assumed that the weights belong to the Muckenhoupt class $A_p$ with $p \in (1,\infty$). We also propose and analyze a convergent finite element discretization for the nonlinear elliptic boundary value problems mentioned above. As an instrumental result, we prove that the discretization of certain linear problems are well posed in weighted spaces.


title: A note on the emission spectrum and trapping states in the Jaynes-Cummings model

abstract: The emission of light from an atom represents a fundamental process that provides valuable insights into the atom-light interaction. The Jaynes-Cummings model is one of the simplest fully quantized models to deal with these interactions, allowing for an analytical solution, while exhibiting notable non-trivial effects. We explore new features in the fluorescence emission spectrum for initial "trapping states", which suppress the atomic population inversion. Despite the seemingly dormant activity of the atom, the resulting emission spectra exhibit rich features, and using a dressed-state coordinates formalism, we are able to quantitatively explain the different profiles in the spectrum. We generalize the trapping conditions for non-zero atom-field detuning and also unveil two types of trapping states that lead to spectra with three peaks, in contrast to previously known states: a center peak and one secondary peak on each side. These are a trapping state formed by a Schr\"odinger cat state with Poissonian statistics (Yurke-Stoler state) and also a different type of "perfect trapping state".


title: GNOME: Generating Negotiations through Open-Domain Mapping of Exchanges

abstract: Language Models have previously shown strong negotiation capabilities in closed domains where the negotiation strategy prediction scope is constrained to a specific setup. In this paper, we first show that these models are not generalizable beyond their original training domain despite their wide-scale pretraining. Following this, we propose an automated framework called GNOME, which processes existing human-annotated, closed-domain datasets using Large Language Models and produces synthetic open-domain dialogues for negotiation. GNOME improves the generalizability of negotiation systems while reducing the expensive and subjective task of manual data curation. Through our experimental setup, we create a benchmark comparing encoder and decoder models trained on existing datasets against datasets created through GNOME. Our results show that models trained on our dataset not only perform better than previous state of the art models on domain specific strategy prediction, but also generalize better to previously unseen domains.


title: PWDFT-SW: Extending the Limit of Plane-Wave DFT Calculations to 16K Atoms on the New Sunway Supercomputer

abstract: First-principles density functional theory (DFT) with plane wave (PW) basis set is the most widely used method in quantum mechanical material simulations due to its advantages in accuracy and universality. However, a perceived drawback of PW-based DFT calculations is their substantial computational cost and memory usage, which currently limits their ability to simulate large-scale complex systems containing thousands of atoms. This situation is exacerbated in the new Sunway supercomputer, where each process is limited to a mere 16 GB of memory. Herein, we present a novel parallel implementation of plane wave density functional theory on the new Sunway supercomputer (PWDFT-SW). PWDFT-SW fully extracts the benefits of Sunway supercomputer by extensively refactoring and calibrating our algorithms to align with the system characteristics of the Sunway system. Through extensive numerical experiments, we demonstrate that our methods can substantially decrease both computational costs and memory usage. Our optimizations translate to a speedup of 64.8x for a physical system containing 4,096 silicon atoms, enabling us to push the limit of PW-based DFT calculations to large-scale systems containing 16,384 carbon atoms.


title: Uncertainty principles for the imaginary Ornstein-Uhlenbeck operator

abstract: We prove two forms of uncertainty principle for the Schr\"odinger group generated by the Ornstein-Uhlenbeck operator. As a consequence, we derive a related (in fact, equivalent) result for the imaginary harmonic oscillator.


title: Linear Inverse Problems Using a Generative Compound Gaussian Prior

abstract: Since most inverse problems arising in scientific and engineering applications are ill-posed, prior information about the solution space is incorporated, typically through regularization, to establish a well-posed problem with a unique solution. Often, this prior information is an assumed statistical distribution of the desired inverse problem solution. Recently, due to the unprecedented success of generative adversarial networks (GANs), the generative network from a GAN has been implemented as the prior information in imaging inverse problems. In this paper, we devise a novel iterative algorithm to solve inverse problems in imaging where a dual-structured prior is imposed by combining a GAN prior with the compound Gaussian (CG) class of distributions. A rigorous computational theory for the convergence of the proposed iterative algorithm, which is based upon the alternating direction method of multipliers, is established. Furthermore, elaborate empirical results for the proposed iterative algorithm are presented. By jointly exploiting the powerful CG and GAN classes of image priors, we find, in compressive sensing and tomographic imaging problems, our proposed algorithm outperforms and provides improved generalizability over competitive prior art approaches while avoiding performance saturation issues in previous GAN prior-based methods.


title: Rideshare Transparency: Translating Gig Worker Insights on AI Platform Design to Policy

abstract: Rideshare platforms exert significant control over workers through algorithmic systems that can result in financial, emotional, and physical harm. What steps can platforms, designers, and practitioners take to mitigate these negative impacts and meet worker needs? In this paper, through a novel mixed methods study combining a LLM-based analysis of over 1 million comments posted to online platform worker communities with semi-structured interviews of workers, we thickly characterize transparency-related harms, mitigation strategies, and worker needs while validating and contextualizing our findings within the broader worker community. Our findings expose a transparency gap between existing platform designs and the information drivers need, particularly concerning promotions, fares, routes, and task allocation. Our analysis suggests that rideshare workers need key pieces of information, which we refer to as indicators, to make informed work decisions. These indicators include details about rides, driver statistics, algorithmic implementation details, and platform policy information. We argue that instead of relying on platforms to include such information in their designs, new regulations that require platforms to publish public transparency reports may be a more effective solution to improve worker well-being. We offer recommendations for implementing such a policy.


title: Exact complex mobility edges and flagellate spectra for non-Hermitian quasicrystals with exponential hoppings

abstract: We propose a class of general non-Hermitian quasiperiodic lattice models with exponential hoppings and analytically determine the genuine complex mobility edges by solving its dual counterpart exactly utilizing Avila's global theory. Our analytical formula unveils that the complex mobility edges usually form a loop structure in the complex energy plane. By shifting the eigenenergy a constant $t$, the complex mobility edges of the family of models with different hopping parameter $t$ can be described by a unified formula, formally independent of $t$. By scanning the hopping parameter, we demonstrate the existence of a type of intriguing flagellate-like spectra in complex energy plane, in which the localized states and extended states are well separated by the complex mobility edges. Our result provides a firm ground for understanding the complex mobility edges in non-Hermitian quasiperiodic lattices.


title: Modal logics of almost sure validities in some classes of euclidean and transitive frames

abstract: Given a class C of finite Kripke frames, we consider the uniform distribution on the frames from C with n states. A formula is almost surely valid in C if the probability that it is valid in a random C-frame with n states tends to 1 as n tends to infinity. The formulas that are almost surely valid in C form a normal modal logic. We find complete and sound axiomatizations for the logics of almost sure validities in the classes of finite frames defined by the logics K5, KD5, K45, KD45, K5B, S5, Grz.3, and GL.3.


title: Predicting Exoplanetary Features with a Residual Model for Uniform and Gaussian Distributions

abstract: The advancement of technology has led to rampant growth in data collection across almost every field, including astrophysics, with researchers turning to machine learning to process and analyze this data. One prominent example of this data in astrophysics is the atmospheric retrievals of exoplanets. In order to help bridge the gap between machine learning and astrophysics domain experts, the 2023 Ariel Data Challenge was hosted to predict posterior distributions of 7 exoplanetary features. The procedure outlined in this paper leveraged a combination of two deep learning models to address this challenge: a Multivariate Gaussian model that generates the mean and covariance matrix of a multivariate Gaussian distribution, and a Uniform Quantile model that predicts quantiles for use as the upper and lower bounds of a uniform distribution. Training of the Multivariate Gaussian model was found to be unstable, while training of the Uniform Quantile model was stable. An ensemble of uniform distributions was found to have competitive results during testing (posterior score of 696.43), and when combined with a multivariate Gaussian distribution achieved a final rank of third in the 2023 Ariel Data Challenge (final score of 681.57).

categories: astro-ph.EP astro-ph.IM cs.LG physics.data-an


title: On the maximal L1 influence of real-valued boolean functions

abstract: We show that any sequence of well-behaved (e.g. bounded and non-constant) real-valued functions of $n$ boolean variables $\{f_n\}$ admits a sequence of coordinates whose $L^1$ influence under the $p$-biased distribution, for any $p\in(0,1)$, is $\Omega(\text{var}(f_n) \frac{\ln n}{n})$.


title: Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articles

abstract: Large language models (LLMs) are increasingly being utilised across a range of tasks and domains, with a burgeoning interest in their application within the field of journalism. This trend raises concerns due to our limited understanding of LLM behaviour in this domain, especially with respect to political bias. Existing studies predominantly focus on LLMs undertaking political questionnaires, which offers only limited insights into their biases and operational nuances. To address this gap, our study establishes a new curated dataset that contains 2,100 human-written articles and utilises their descriptions to generate 56,700 synthetic articles using nine LLMs. This enables us to analyse shifts in properties between human-authored and machine-generated articles, with this study focusing on political bias, detecting it using both supervised models and LLMs. Our findings reveal significant disparities between base and instruction-tuned LLMs, with instruction-tuned models exhibiting consistent political bias. Furthermore, we are able to study how LLMs behave as classifiers, observing their display of political bias even in this role. Overall, for the first time within the journalistic domain, this study outlines a framework and provides a structured dataset for quantifiable experiments, serving as a foundation for further research into LLM political bias and its implications.


title: Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference

abstract: As the demand for long-context large language models (LLMs) increases, models with context windows of up to 128K or 1M tokens are becoming increasingly prevalent. However, long-context LLM inference is challenging since the inference speed decreases significantly as the sequence length grows. This slowdown is primarily caused by loading a large KV cache during self-attention. Previous works have shown that a small portion of critical tokens will dominate the attention outcomes. However, we observe the criticality of a token highly depends on the query. To this end, we propose Quest, a query-aware KV cache selection algorithm. Quest keeps track of the minimal and maximal Key values in KV cache pages and estimates the criticality of a given page using Query vectors. By only loading the Top-K critical KV cache pages for attention, Quest significantly speeds up self-attention without sacrificing accuracy. We show that Quest can achieve up to 2.23x self-attention speedup, which reduces inference latency by 7.03x while performing well on tasks with long dependencies with negligible accuracy loss. Code is available at http://github.com/mit-han-lab/Quest .


title: A Rate-Distortion View of Uncertainty Quantification

abstract: In supervised learning, understanding an input's proximity to the training data can help a model decide whether it has sufficient evidence for reaching a reliable prediction. While powerful probabilistic models such as Gaussian Processes naturally have this property, deep neural networks often lack it. In this paper, we introduce Distance Aware Bottleneck (DAB), i.e., a new method for enriching deep neural networks with this property. Building on prior information bottleneck approaches, our method learns a codebook that stores a compressed representation of all inputs seen during training. The distance of a new example from this codebook can serve as an uncertainty estimate for the example. The resulting model is simple to train and provides deterministic uncertainty estimates by a single forward pass. Finally, our method achieves better out-of-distribution (OOD) detection and misclassification prediction than prior methods, including expensive ensemble methods, deep kernel Gaussian Processes, and approaches based on the standard information bottleneck.


title: High-level Codes and Fine-grained Weights for Online Multi-modal Hashing Retrieval

abstract: In the real world, multi-modal data often appears in a streaming fashion, and there is a growing demand for similarity retrieval from such non-stationary data, especially at a large scale. In response to this need, online multi-modal hashing has gained significant attention. However, existing online multi-modal hashing methods face challenges related to the inconsistency of hash codes during long-term learning and inefficient fusion of different modalities. In this paper, we present a novel approach to supervised online multi-modal hashing, called High-level Codes, Fine-grained Weights (HCFW). To address these problems, HCFW is designed by its non-trivial contributions from two primary dimensions: 1) Online Hashing Perspective. To ensure the long-term consistency of hash codes, especially in incremental learning scenarios, HCFW learns high-level codes derived from category-level semantics. Besides, these codes are adept at handling the category-incremental challenge. 2) Multi-modal Hashing Aspect. HCFW introduces the concept of fine-grained weights designed to facilitate the seamless fusion of complementary multi-modal data, thereby generating multi-modal weights at the instance level and enhancing the overall hashing performance. A comprehensive battery of experiments conducted on two benchmark datasets convincingly underscores the effectiveness and efficiency of HCFW.