title: Arithmetic Reasoning with LLM: Prolog Generation & Permutation

abstract: Instructing large language models (LLMs) to solve elementary school math problems has shown great success using Chain of Thought (CoT). However, the CoT approach relies on an LLM to generate a sequence of arithmetic calculations which can be prone to cascaded calculation errors. We hypothesize that an LLM should focus on extracting predicates and generating symbolic formulas from the math problem description so that the underlying calculation can be done via an external code interpreter. We investigate using LLM to generate Prolog programs to solve mathematical questions. Experimental results show that our Prolog-based arithmetic problem-solving outperforms CoT generation in the GSM8K benchmark across three distinct LLMs. In addition, given the insensitive ordering of predicates and symbolic formulas in Prolog, we propose to permute the ground truth predicates for more robust LLM training via data augmentation.


title: White-box Multimodal Jailbreaks Against Large Vision-Language Models

abstract: Recent advancements in Large Vision-Language Models (VLMs) have underscored their superiority in various multimodal tasks. However, the adversarial robustness of VLMs has not been fully explored. Existing methods mainly assess robustness through unimodal adversarial attacks that perturb images, while assuming inherent resilience against text-based attacks. Different from existing attacks, in this work we propose a more comprehensive strategy that jointly attacks both text and image modalities to exploit a broader spectrum of vulnerability within VLMs. Specifically, we propose a dual optimization objective aimed at guiding the model to generate affirmative responses with high toxicity. Our attack method begins by optimizing an adversarial image prefix from random noise to generate diverse harmful responses in the absence of text input, thus imbuing the image with toxic semantics. Subsequently, an adversarial text suffix is integrated and co-optimized with the adversarial image prefix to maximize the probability of eliciting affirmative responses to various harmful instructions. The discovered adversarial image prefix and text suffix are collectively denoted as a Universal Master Key (UMK). When integrated into various malicious queries, UMK can circumvent the alignment defenses of VLMs and lead to the generation of objectionable content, known as jailbreaks. The experimental results demonstrate that our universal attack strategy can effectively jailbreak MiniGPT-4 with a 96% success rate, highlighting the vulnerability of VLMs and the urgent need for new alignment strategies.


title: Enhanced dissipation and temporal decay in the Euler-Poisson-Navier-Stokes equations

abstract: This paper investigates the global well-posedness and large-time behavior of solutions for a coupled fluid model in $\mathbb{R}^3$ consisting of the isothermal compressible Euler-Poisson system and incompressible Navier-Stokes equations coupled through the drag force. Notably, we exploit the dissipation effects inherent in the Poisson equation to achieve a faster decay of fluid density compared to velocities. This strategic utilization of dissipation, together with the influence of the electric field and the damping structure induced by the drag force, leads to a remarkable decay behavior: the fluid density converges to equilibrium at a rate of $(1+t)^{-11/4}$, significantly faster than the decay rates of velocity differences $(1+t)^{-7/4}$ and velocities themselves $(1+t)^{-3/4}$ in the $L^2$ norm. Furthermore, under the condition of vanishing coupled incompressible flow, we demonstrate an exponential decay to a constant state for the solution of the corresponding system, the damped Euler-Poisson system.


title: The expected evolution of the binary system PTF J2238+743015.1

abstract: Binary systems made by a low-mass CO WD and a He-donor represent possible progenitors of explosive events via He-detonation, producing low-luminosity thermonuclear Supernovae with a peculiar nucleosynthetis. Recently, the binary system PTF J223857.11+743015.1 has been suggested as one. We investigate the evolution of the PTF J223857.11+743015.1 system, composed by a 0.75Msun CO WD and a 0.390Msun subdwarf, capped by a thin H-rich layer, considering rotation of the WD component. We compute the evolution of two stars simultaneously, accounting for the possible evolution of the orbital parameters, as determined by mass transfer between components and by mass ejection from the system during RLOF episodes. We consider that the WD gains angular momentum due to accretion and we follow the evolution of the angular velocity profile as due to angular momentum transport via convection and rotation-induced instabilities. As the donor H-rich envelope is transferred, the WD experiences recurrent very strong H-flashes triggering RLOF episodes during which the entire accreted matter is lost from the system. Due to mixing of chemicals by rotation-induced instabilities during the accretion phase, H-flashes occur inside the original WD. Hence, pulse-by pulse, the accretor mass is reduced down to 0.7453Msun. When He-rich matter is transferred, He-detonation does not occur in the rotating WD, which undergoes 6 very strong He-flashes and subsequent RLOF episodes. Also in this case, due to rotation-induced mixing of the accreted layers with the underlying core, the WD is eroded. Finally, when the mass transfer rate from the donor decreases, a massive He-buffer is piled-up onto the accretor which ends its life as a cooling WD. The binary system PTF J2238+743015.1 as all those binaries having similar components masses and orbital parameters are not good candidates as thermonuclear explosions progenitors.


title: $C^2M^3$: Cycle-Consistent Multi-Model Merging

abstract: In this paper, we present a novel data-free method for merging neural networks in weight space. Differently from most existing works, our method optimizes for the permutations of network neurons globally across all layers. This allows us to enforce cycle consistency of the permutations when merging $N \geq 3$ models, allowing circular compositions of permutations to be computed without accumulating error along the path. We qualitatively and quantitatively motivate the need for such a constraint, showing its benefits when merging sets of models in scenarios spanning varying architectures and datasets. We finally show that, when coupled with activation renormalization, our approach yields the best results in the task.


title: FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction

abstract: The objective of traffic prediction is to accurately forecast and analyze the dynamics of transportation patterns, considering both space and time. However, the presence of distribution shift poses a significant challenge in this field, as existing models struggle to generalize well when faced with test data that significantly differs from the training distribution. To tackle this issue, this paper introduces a simple and universal spatio-temporal prompt-tuning framework-FlashST, which adapts pre-trained models to the specific characteristics of diverse downstream datasets, improving generalization in diverse traffic prediction scenarios. Specifically, the FlashST framework employs a lightweight spatio-temporal prompt network for in-context learning, capturing spatio-temporal invariant knowledge and facilitating effective adaptation to diverse scenarios. Additionally, we incorporate a distribution mapping mechanism to align the data distributions of pre-training and downstream data, facilitating effective knowledge transfer in spatio-temporal forecasting. Empirical evaluations demonstrate the effectiveness of our FlashST across different spatio-temporal prediction tasks using diverse urban datasets. Code is available at https://github.com/HKUDS/FlashST.


title: Near IR bandgap semiconductive 2D conjugated metal-organic framework with rhombic lattice and high mobility

abstract: Two-dimensional conjugated metal-organic frameworks (2D c-MOFs) are emerging as a unique class of 2D electronic materials. However, intrinsically semiconducting 2D c-MOFs with gaps in the Vis-NIR and high charge carrier mobility have been rare. Most of the reported semiconducting 2D c-MOFs are metallic (i.e. gapless), which limits their use in applications where larger band gaps are needed for logic devices. Herein, we design a new D2h-geometric ligand, 2,3,6,7,11,12,15,16-octahydroxyphenanthro(9,10b)triphenylene (OHPTP), and synthesize the first example of a 2D c-MOF single crystal (OHPTP-Cu) with a rhombohedral pore geometry after coordination with copper. The continuous rotation electron diffraction (cRED) analysis unveils the orthorhombic crystal structure at the atomic level with a unique AB layer stacking. The resultant Cu2(OHPTP) is a p-type semiconductor with an indirect band gap of about 0.50 eV and exhibits high electrical conductivity of 0.10 S cm-1 and high charge carrier mobility of 10.0 cm2V-1s-1. Density-functional theory calculations underline the predominant role of the out-of-plane charge transport in this semiquinone-based 2D c-MOFs.


title: Enhancing Emotion Recognition in Conversation through Emotional Cross-Modal Fusion and Inter-class Contrastive Learning

abstract: The purpose of emotion recognition in conversation (ERC) is to identify the emotion category of an utterance based on contextual information. Previous ERC methods relied on simple connections for cross-modal fusion and ignored the information differences between modalities, resulting in the model being unable to focus on modality-specific emotional information. At the same time, the shared information between modalities was not processed to generate emotions. Information redundancy problem. To overcome these limitations, we propose a cross-modal fusion emotion prediction network based on vector connections. The network mainly includes two stages: the multi-modal feature fusion stage based on connection vectors and the emotion classification stage based on fused features. Furthermore, we design a supervised inter-class contrastive learning module based on emotion labels. Experimental results confirm the effectiveness of the proposed method, demonstrating excellent performance on the IEMOCAP and MELD datasets.


title: Near-Infrared and Low-Rank Adaptation of Vision Transformers in Remote Sensing

abstract: Plant health can be monitored dynamically using multispectral sensors that measure Near-Infrared reflectance (NIR). Despite this potential, obtaining and annotating high-resolution NIR images poses a significant challenge for training deep neural networks. Typically, large networks pre-trained on the RGB domain are utilized to fine-tune infrared images. This practice introduces a domain shift issue because of the differing visual traits between RGB and NIR images.As an alternative to fine-tuning, a method called low-rank adaptation (LoRA) enables more efficient training by optimizing rank-decomposition matrices while keeping the original network weights frozen. However, existing parameter-efficient adaptation strategies for remote sensing images focus on RGB images and overlook domain shift issues in the NIR domain. Therefore, this study investigates the potential benefits of using vision transformer (ViT) backbones pre-trained in the RGB domain, with low-rank adaptation for downstream tasks in the NIR domain. Extensive experiments demonstrate that employing LoRA with pre-trained ViT backbones yields the best performance for downstream tasks applied to NIR images.


title: Boosting Protein Language Models with Negative Sample Mining

abstract: We introduce a pioneering methodology for boosting large language models in the domain of protein representation learning. Our primary contribution lies in the refinement process for correlating the over-reliance on co-evolution knowledge, in a way that networks are trained to distill invaluable insights from negative samples, constituted by protein pairs sourced from disparate categories. By capitalizing on this novel approach, our technique steers the training of transformer-based models within the attention score space. This advanced strategy not only amplifies performance but also reflects the nuanced biological behaviors exhibited by proteins, offering aligned evidence with traditional biological mechanisms such as protein-protein interaction. We experimentally observed improved performance on various tasks over datasets, on top of several well-established large protein models. This innovative paradigm opens up promising horizons for further progress in the realms of protein research and computational biology.


title: Reliable Object Tracking by Multimodal Hybrid Feature Extraction and Transformer-Based Fusion

abstract: Visual object tracking, which is primarily based on visible light image sequences, encounters numerous challenges in complicated scenarios, such as low light conditions, high dynamic ranges, and background clutter. To address these challenges, incorporating the advantages of multiple visual modalities is a promising solution for achieving reliable object tracking. However, the existing approaches usually integrate multimodal inputs through adaptive local feature interactions, which cannot leverage the full potential of visual cues, thus resulting in insufficient feature modeling. In this study, we propose a novel multimodal hybrid tracker (MMHT) that utilizes frame-event-based data for reliable single object tracking. The MMHT model employs a hybrid backbone consisting of an artificial neural network (ANN) and a spiking neural network (SNN) to extract dominant features from different visual modalities and then uses a unified encoder to align the features across different domains. Moreover, we propose an enhanced transformer-based module to fuse multimodal features using attention mechanisms. With these methods, the MMHT model can effectively construct a multiscale and multidimensional visual feature space and achieve discriminative feature modeling. Extensive experiments demonstrate that the MMHT model exhibits competitive performance in comparison with that of other state-of-the-art methods. Overall, our results highlight the effectiveness of the MMHT model in terms of addressing the challenges faced in visual object tracking tasks.


title: A beam profile monitor for GeV photon with high spatial resolution and fast readout capability

abstract: A beam profile monitor (BPM) has been developed to measure photon beams at the BM4 beamline of the Mikamine site, Research Center for Accelerator and Radioisotope Science (RARIS-Mikamine; previously known as ELPH) at Tohoku University. The BPM comprises plastic scintillation fibers and SiPMs, enabling high-precision, real-time measurements of photon beams in the 1 GeV region. Data acquisition utilized streaming TDC, a firmware commonly employed in the J-PARC Hadron-hall, enabling real-time detection of high-intensity photon beams with count rates reaching several tens of MHz. With sufficient statistical data, the BPM achieved a 1 s beam-profiling accuracy of 10 {\mu}m. The proposed BPM serves as a valuable resource for future physics experiments at the BM4 photon beamline and will contribute significantly to ongoing accelerator research endeavors.


title: Cycle-YOLO: A Efficient and Robust Framework for Pavement Damage Detection

abstract: With the development of modern society, traffic volume continues to increase in most countries worldwide, leading to an increase in the rate of pavement damage Therefore, the real-time and highly accurate pavement damage detection and maintenance have become the current need. In this paper, an enhanced pavement damage detection method with CycleGAN and improved YOLOv5 algorithm is presented. We selected 7644 self-collected images of pavement damage samples as the initial dataset and augmented it by CycleGAN. Due to a substantial difference between the images generated by CycleGAN and real road images, we proposed a data enhancement method based on an improved Scharr filter, CycleGAN, and Laplacian pyramid. To improve the target recognition effect on a complex background and solve the problem that the spatial pyramid pooling-fast module in the YOLOv5 network cannot handle multiscale targets, we introduced the convolutional block attention module attention mechanism and proposed the atrous spatial pyramid pooling with squeeze-and-excitation structure. In addition, we optimized the loss function of YOLOv5 by replacing the CIoU with EIoU. The experimental results showed that our algorithm achieved a precision of 0.872, recall of 0.854, and mean average precision@0.5 of 0.882 in detecting three main types of pavement damage: cracks, potholes, and patching. On the GPU, its frames per second reached 68, meeting the requirements for real-time detection. Its overall performance even exceeded the current more advanced YOLOv7 and achieved good results in practical applications, providing a basis for decision-making in pavement damage detection and prevention.


title: Laser-scanning optical-frequency-comb microscopy for multimodal imaging

abstract: We introduce a novel laser-scanning optical microscopy technique that employs optical-frequency-comb (OFC) lasers. This method facilitates multimodal spectroscopic imaging by analyzing interferograms produced via a dual-comb spectroscopic approach. Such interferograms capture comprehensive light information, including amplitude, phase, polarization, frequency, and time of flight information, enabling multimodal imaging from a single measurement. We demonstrate the potential of this technique across several spectroscopic imaging applications.


title: Biunits of ternary algebra of hypermatrices

abstract: In this paper we study ternary algebras of third-order hypermatrices. By hypermatrix we mean a complex-valued variable with three indices, which is also called a three-dimensional matrix or spatial matrix. We assume that a hypermatrix is defined in three-dimensional Euclidean space and when this space is rotated, it transforms as a SO(3)-tensor. We consider two ternary multiplications of hypermatrices, which have the property of generalized associativity. We explore the geometric meaning of two independent SO(3)-invariants of hypermatrices and show that one of them defines a Hermitian metric. We study the 10-dimensional subspace of hypermatrices, known in the theory of representations of the rotation group, as the space of the weight 2 tensor representation of the rotation group. It is proved that the elements of this subspace that satisfy the regularity condition are right biunits of the ternary algebra of hypermatrices. The motivation for studying biunits of ternary algebra of hypermatrices was a ternary generalization of the Pauli exclusion principle.


title: Deep learning inference of the neutron star equation of state

abstract: We present a pipeline to infer the equation of state of neutron stars from observations based on deep neural networks. In particular, using the standard (deterministic), as well as Bayesian (probabilistic) deep networks, we explore how one can infer the interior speed of sound of the star given a set of mock observations of total stellar mass, stellar radius and tidal deformability. We discuss in detail the construction of our simulated dataset of stellar observables starting from the solution of the gravitational equations, as well as the relevant architectures for the deep networks, along with their performance and accuracy. We further explain how our pipeline is capable to detect a possible QCD phase transition in the stellar core. Our results show that deep networks offer a promising tool towards solving the inverse problem of neutron stars, and the accurate inference of their interior from future stellar observations.

categories: astro-ph.HE astro-ph.IM gr-qc hep-ph hep-th


title: $\infty$-Categorical Generalized Langlands Correspondence II: Langlands Program Formalism

abstract: We extend the Langlands program in various subprograms with certain different generalizations: (1) Mixed-parity functorial perturbation of the usual Langlands program after Fargues-Scholze in all characteristics; (2) Robba-Frobenius sheafified functorial perturbation of the usual Langlands program after Fargues-Scholze and Kedlaya-Liu in all characteristics; (3) Arithmetic $D$-module theoretic functorial perturbation of the usual Langlands program after Fargues-Scholze and Abe-Kedlaya-Xu. In certain localized setting we construct Langlands correspondence from smooth representation side to Weil side through geometrized generalized Bernstein center.


title: Human-Cobot collaboration's impact on success, time completion, errors, workload, gestures and acceptability during an assembly task

abstract: The 5.0 industry promotes collaborative robots (cobots). This research studies the impacts of cobot collaboration using an experimental setup. 120 participants realized a simple and a complex assembly task. 50% collaborated with another human (H/H) and 50% with a cobot (H/C). The workload and the acceptability of the cobotic collaboration were measured. Working with a cobot decreases the effect of the task complexity on the human workload and on the output quality. However, it increases the time completion and the number of gestures (while decreasing their frequency). The H/C couples have a higher chance of success but they take more time and more gestures to realize the task. The results of this research could help developers and stakeholders to understand the impacts of implementing a cobot in production chains.


title: Memory-induced absolute negative mobility

abstract: Non-Markovian systems form a broad area of physics that remains greatly unexplored despite years of intensive investigations. The spotlight is on memory as a source of effects that are absent in their Markovian counterparts. In this work we dive into this problem and analyze a driven Brownian particle moving in a spatially periodic potential and exposed to correlated thermal noise. We show that the absolute negative mobility effect, in which the net movement of the particle is in direction opposite to the average force acting on it, may be induced by the memory of the setup. To explain the origin of this phenomenon we resort to the recently developed effective mass approach to dynamics of non-Markovian systems.


title: Lateral migration and bouncing of a deformable bubble rising near a vertical wall. Part 1. Moderately inertial regimes

abstract: The buoyancy-driven motion of a deformable bubble rising near a vertical hydrophilic wall is studied numerically. We focus on moderately inertial regimes in which the bubble undergoes low-to-moderate deformations and would rise in a straight line in the absence of the wall. Three different types of near-wall motion are observed, depending on the buoyancy-to-viscous and buoyancy-to-capillary force ratios defining the Galilei ($Ga$) and Bond ($Bo$) numbers of the system, respectively. For low enough $Ga$ or large enough $Bo$, bubbles consistently migrate away from the wall. Conversely, for large enough $Ga$ and low enough $Bo$, they perform periodic near-wall bounces. At intermediate $Ga$ and $Bo$, they are first attracted to the wall down to a certain critical distance, and then perform bounces with a decreasing amplitude before stabilizing at this critical separation. Periodic bounces are accompanied by the shedding of a pair of streamwise vortices in the wake, the formation of which is governed by the near-wall shear resulting from the no-slip condition. These vortices provide a repulsive force that overcomes the viscous resistance of the fluid to the departing motion, making the bubble capable of returning to the region where it is attracted again to the wall. Although periodic, the shedding/regeneration cycle of these vortices is highly asymmetric with respect to the lateral bubble displacements, vortices being shed when the gap left between the bubble and the wall reaches its maximum, and reborn only when this gap comes back to its minimum.


title: OV-DQUO: Open-Vocabulary DETR with Denoising Text Query Training and Open-World Unknown Objects Supervision

abstract: Open-Vocabulary Detection (OVD) aims to detect objects from novel categories beyond the base categories on which the detector is trained. However, existing open-vocabulary detectors trained on known category data tend to assign higher confidence to trained categories and confuse novel categories with background. To resolve this, we propose OV-DQUO, an \textbf{O}pen-\textbf{V}ocabulary DETR with \textbf{D}enoising text \textbf{Q}uery training and open-world \textbf{U}nknown \textbf{O}bjects supervision. Specifically, we introduce a wildcard matching method that enables the detector to learn from pairs of unknown objects recognized by the open-world detector and text embeddings with general semantics, mitigating the confidence bias between base and novel categories. Additionally, we propose a denoising text query training strategy that synthesizes additional noisy query-box pairs from open-world unknown objects to trains the detector through contrastive learning, enhancing its ability to distinguish novel objects from the background. We conducted extensive experiments on the challenging OV-COCO and OV-LVIS benchmarks, achieving new state-of-the-art results of 45.6 AP50 and 39.3 mAP on novel categories respectively, without the need for additional training data. Models and code are released at https://github.com/xiaomoguhz/OV-DQUO


title: Trustworthy DNN Partition for Blockchain-enabled Digital Twin in Wireless IIoT Networks

abstract: Digital twin (DT) has emerged as a promising solution to enhance manufacturing efficiency in industrial Internet of Things (IIoT) networks. To promote the efficiency and trustworthiness of DT for wireless IIoT networks, we propose a blockchain-enabled DT (B-DT) framework that employs deep neural network (DNN) partitioning technique and reputation-based consensus mechanism, wherein the DTs maintained at the gateway side execute DNN inference tasks using the data collected from their associated IIoT devices. First, we employ DNN partitioning technique to offload the top-layer DNN inference tasks to the access point (AP) side, which alleviates the computation burden at the gateway side and thereby improves the efficiency of DNN inference. Second, we propose a reputation-based consensus mechanism that integrates Proof of Work (PoW) and Proof of Stake (PoS). Specifically, the proposed consensus mechanism evaluates the off-chain reputation of each AP according to its computation resource contributions to the DNN inference tasks, and utilizes the off-chain reputation as a stake to adjust the block generation difficulty. Third, we formulate a stochastic optimization problem of communication resource (i.e., partition point) and computation resource allocation (i.e., computation frequency of APs for top-layer DNN inference and block generation) to minimize system latency under the time-varying channel state and long-term constraints of off-chain reputation, and solve the problem using Lyapunov optimization method. Experimental results show that the proposed dynamic DNN partitioning and resource allocation (DPRA) algorithm outperforms the baselines in terms of reducing the overall latency while guaranteeing the trustworthiness of the B-DT system.


title: Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models

abstract: Long-context modeling capabilities are important for large language models (LLMs) in various applications. However, directly training LLMs with long context windows is insufficient to enhance this capability since some training samples do not exhibit strong semantic dependencies across long contexts. In this study, we propose a data mining framework \textbf{ProLong} that can assign each training sample with a long dependency score, which can be used to rank and filter samples that are more advantageous for enhancing long-context modeling abilities in LLM training. Specifically, we first use delta perplexity scores to measure the \textit{Dependency Strength} between text segments in a given document. Then we refine this metric based on the \textit{Dependency Distance} of these segments to incorporate spatial relationships across long-contexts. Final results are calibrated with a \textit{Dependency Specificity} metric to prevent trivial dependencies introduced by repetitive patterns. Moreover, a random sampling approach is proposed to optimize the computational efficiency of ProLong. Comprehensive experiments on multiple benchmarks indicate that ProLong effectively identifies documents that carry long dependencies and LLMs trained on these documents exhibit significantly enhanced long-context modeling capabilities.


title: Boosting General Trimap-free Matting in the Real-World Image

abstract: Image matting aims to obtain an alpha matte that separates foreground objects from the background accurately. Recently, trimap-free matting has been well studied because it requires only the original image without any extra input. Such methods usually extract a rough foreground by itself to take place trimap as further guidance. However, the definition of 'foreground' lacks a unified standard and thus ambiguities arise. Besides, the extracted foreground is sometimes incomplete due to inadequate network design. Most importantly, there is not a large-scale real-world matting dataset, and current trimap-free methods trained with synthetic images suffer from large domain shift problems in practice. In this paper, we define the salient object as foreground, which is consistent with human cognition and annotations of the current matting dataset. Meanwhile, data and technologies in salient object detection can be transferred to matting in a breeze. To obtain a more accurate and complete alpha matte, we propose a network called \textbf{M}ulti-\textbf{F}eature fusion-based \textbf{C}oarse-to-fine Network \textbf{(MFC-Net)}, which fully integrates multiple features for an accurate and complete alpha matte. Furthermore, we introduce image harmony in data composition to bridge the gap between synthetic and real images. More importantly, we establish the largest general matting dataset \textbf{(Real-19k)} in the real world to date. Experiments show that our method is significantly effective on both synthetic and real-world images, and the performance in the real-world dataset is far better than existing matting-free methods. Our code and data will be released soon.


title: Cascaded Group Testing

abstract: In this paper, we introduce a variation of the group testing problem where each test is specified by an ordered subset of items, and returns the first defective item in the specified order. We refer to this as \textit{cascaded group testing} and the goal is to identify a small set of $K$ defective items amongst a collection of size $N$, using as few tests as possible. For the adaptive testing regime, we show that a simple scheme is able to find all defective items in at most $K$ tests, which is optimal. For the non-adaptive setting, we first come up with a necessary and sufficient condition for any collection of tests to be feasible for recovering all the defectives. Using this, we are able to show that any feasible non-adaptive strategy requires at least $\Omega(K^2)$ tests. In terms of achievability, it is easy to show that a collection of $O(K^2 \log (N/K))$ randomly constructed tests is feasible. We show via carefully constructed explicit designs that one can do significantly better. We provide two simple schemes for $K = 1, 2$ which only require one and two tests respectively irrespective of the number of items $N$. Note that this is in contrast to standard binary group testing, where at least $\Omega(\log N)$ tests are required. The case of $K \ge 3$ is more challenging and here we come up with an iterative design which requires only $\text{poly}(\log \log N)$ tests.


title: Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation

abstract: In this paper, we address the problem of cost-sensitive multi-fidelity Bayesian Optimization (BO) for efficient hyperparameter optimization (HPO). Specifically, we assume a scenario where users want to early-stop the BO when the performance improvement is not satisfactory with respect to the required computational cost. Motivated by this scenario, we introduce utility, which is a function predefined by each user and describes the trade-off between cost and performance of BO. This utility function, combined with our novel acquisition function and stopping criterion, allows us to dynamically choose for each BO step the best configuration that we expect to maximally improve the utility in future, and also automatically stop the BO around the maximum utility. Further, we improve the sample efficiency of existing learning curve (LC) extrapolation methods with transfer learning, while successfully capturing the correlations between different configurations to develop a sensible surrogate function for multi-fidelity BO. We validate our algorithm on various LC datasets and found it outperform all the previous multi-fidelity BO and transfer-BO baselines we consider, achieving significantly better trade-off between cost and performance of BO.


title: Fisher's Legacy of Directional Statistics, and Beyond to Statistics on Manifolds

abstract: It will not be an exaggeration to say that R A Fisher is the Albert Einstein of Statistics. He pioneered almost all the main branches of statistics, but it is not as well known that he opened the area of Directional Statistics with his 1953 paper introducing a distribution on the sphere which is now known as the Fisher distribution. He stressed that for spherical data one should take into account that the data is on a manifold. We will describe this Fisher distribution and reanalyse his geological data. We also comment on the two goals he set himself in that paper, and how he reinvented the von Mises distribution on the circle. Since then, many extensions of this distribution have appeared bearing Fisher's name such as the von Mises Fisher distribution and the matrix Fisher distribution. In fact, the subject of Directional Statistics has grown tremendously in the last two decades with new applications emerging in Life Sciences, Image Analysis, Machine Learning and so on. We give a recent new method of constructing the Fisher type distribution which has been motivated by some problems in Machine Learning. The subject related to his distribution has evolved since then more broadly as Statistics on Manifolds which also includes the new field of Shape Analysis. We end with a historical note pointing out some correspondence between D'Arcy Thompson and R A Fisher related to Shape Analysis.


title: Banana Trees for the Persistence in Time Series Experimentally

abstract: In numerous fields, dynamic time series data require continuous updates, necessitating efficient data processing techniques for accurate analysis. This paper examines the banana tree data structure, specifically designed to efficiently maintain persistent homology -- a multi-scale topological descriptor -- for dynamically changing time series data. We implement this data structure and conduct an experimental study to assess its properties and runtime for update operations. Our findings indicate that banana trees are highly effective with unbiased random data, outperforming state-of-the-art static algorithms in these scenarios. Additionally, our results show that real-world time series share structural properties with unbiased random walks, suggesting potential practical utility for our implementation.


title: Towards Clinical AI Fairness: Filling Gaps in the Puzzle

abstract: The ethical integration of Artificial Intelligence (AI) in healthcare necessitates addressing fairness-a concept that is highly context-specific across medical fields. Extensive studies have been conducted to expand the technical components of AI fairness, while tremendous calls for AI fairness have been raised from healthcare. Despite this, a significant disconnect persists between technical advancements and their practical clinical applications, resulting in a lack of contextualized discussion of AI fairness in clinical settings. Through a detailed evidence gap analysis, our review systematically pinpoints several deficiencies concerning both healthcare data and the provided AI fairness solutions. We highlight the scarcity of research on AI fairness in many medical domains where AI technology is increasingly utilized. Additionally, our analysis highlights a substantial reliance on group fairness, aiming to ensure equality among demographic groups from a macro healthcare system perspective; in contrast, individual fairness, focusing on equity at a more granular level, is frequently overlooked. To bridge these gaps, our review advances actionable strategies for both the healthcare and AI research communities. Beyond applying existing AI fairness methods in healthcare, we further emphasize the importance of involving healthcare professionals to refine AI fairness concepts and methods to ensure contextually relevant and ethically sound AI applications in healthcare.


title: Stochastic Optimization Schemes for Performative Prediction with Nonconvex Loss

abstract: This paper studies a risk minimization problem with decision dependent data distribution. The problem pertains to the performative prediction setting where a trained model can affect the outcome that the model estimates. Such dependency creates a feedback loop that influences the stability of optimization algorithms such as stochastic gradient descent (SGD). We present the first study on performative prediction with smooth but possibly non-convex loss. We analyze a greedy deployment scheme with SGD (SGD-GD). Note that in the literature, SGD-GD is often studied with strongly convex loss. We first propose the definition of stationary performative stable (SPS) solutions through relaxing the popular performative stable condition. We then prove that SGD-GD converges to a biased SPS solution in expectation. We consider two conditions of sensitivity on the distribution shifts: (i) the sensitivity is characterized by Wasserstein-1 distance and the loss is Lipschitz w.r.t.~data samples, or (ii) the sensitivity is characterized by $\chi^2$-divergence and the loss is bounded. In both conditions, the bias levels are proportional to stochastic gradient's variance and sensitivity level. Our analysis is extended to a lazy deployment scheme where models are deployed once per several SGD updates, and we show that it converges to a bias-free SPS solution. Numerical experiments corroborate our theories.


title: Unraveling friction forces of droplets on a non-wetting surface

abstract: This paper explores the friction forces encountered by droplets on non-wetting surfaces, specifically focusing on superhydrophobic and superheated substrates. Employing a combination of experimental techniques, including inclined plane tests and cantilever force sensor measurements, we quantify friction forces across a broad range of velocities and surface types. Our results demonstrate that friction forces vary significantly with changes in droplet velocity and surface characteristics, transitioning from contact line pinning to viscous dissipation in the bulk of the droplet. We propose a universal scaling law that accounts for contact angle hysteresis, viscous dissipation, and aerodynamic drag, providing a comprehensive framework for understanding droplet dynamics on non-wetting surfaces. These findings offer valuable insights for optimizing surface designs in fluid transport and microfluidic applications, paving the way for enhanced efficiency and innovation in these technologies.


title: Generative AI Enhances Team Performance and Reduces Need for Traditional Teams

abstract: Recent advancements in generative artificial intelligence (AI) have transformed collaborative work processes, yet the impact on team performance remains underexplored. Here we examine the role of generative AI in enhancing or replacing traditional team dynamics using a randomized controlled experiment with 435 participants across 122 teams. We show that teams augmented with generative AI significantly outperformed those relying solely on human collaboration across various performance measures. Interestingly, teams with multiple AIs did not exhibit further gains, indicating diminishing returns with increased AI integration. Our analysis suggests that centralized AI usage by a few team members is more effective than distributed engagement. Additionally, individual-AI pairs matched the performance of conventional teams, suggesting a reduced need for traditional team structures in some contexts. However, despite this capability, individual-AI pairs still fell short of the performance levels achieved by AI-assisted teams. These findings underscore that while generative AI can replace some traditional team functions, more comprehensively integrating AI within team structures provides superior benefits, enhancing overall effectiveness beyond individual efforts.


title: A real/fast-time simulator for impact assessment of spoofing & jamming attacks on GNSS receivers

abstract: In aviation, the impact of threats is becoming increasingly significant, particularly for global navigation satellite system (GNSS). Two relevant GNSS threats are represented by jamming and spoofing. In order to evaluate the technological solutions to counter GNSS attacks, such attacks should be assessed by means of a proper GNSS threat simulator. This work shows the implementation and the testing results of a GNSS security impact simulator which injects the desired threat scenarios as a deviations on the GNSS actual measurements. The proposed simulator can be integrated in both real- and fast-time simulation environments. The provided results confirm the effectiveness of the simulator, and include in-flight demonstrations by means of a flight experimental vehicle.


title: SarcNet: A Novel AI-based Framework to Automatically Analyze and Score Sarcomere Organizations in Fluorescently Tagged hiPSC-CMs

abstract: Quantifying sarcomere structure organization in human-induced pluripotent stem cell-derived cardiomyocytes (hiPSC-CMs) is crucial for understanding cardiac disease pathology, improving drug screening, and advancing regenerative medicine. Traditional methods, such as manual annotation and Fourier transform analysis, are labor-intensive, error-prone, and lack high-throughput capabilities. In this study, we present a novel deep learning-based framework that leverages cell images and integrates cell features to automatically evaluate the sarcomere structure of hiPSC-CMs from the onset of differentiation. This framework overcomes the limitations of traditional methods through automated, high-throughput analysis, providing consistent, reliable results while accurately detecting complex sarcomere patterns across diverse samples. The proposed framework contains the SarcNet, a linear layers-added ResNet-18 module, to output a continuous score ranging from one to five that captures the level of sarcomere structure organization. It is trained and validated on an open-source dataset of hiPSC-CMs images with the endogenously GFP-tagged alpha-actinin-2 structure developed by the Allen Institute for Cell Science (AICS). SarcNet achieves a Spearman correlation of 0.831 with expert evaluations, demonstrating superior performance and an improvement of 0.075 over the current state-of-the-art approach, which uses Linear Regression. Our results also show a consistent pattern of increasing organization from day 18 to day 32 of differentiation, aligning with expert evaluations. By integrating the quantitative features calculated directly from the images with the visual features learned during the deep learning model, our framework offers a more comprehensive and accurate assessment, thereby enhancing the further utility of hiPSC-CMs in medical research and therapy development.


title: The Evolution of Multimodal Model Architectures

abstract: This work uniquely identifies and characterizes four prevalent multimodal model architectural patterns in the contemporary multimodal landscape. Systematically categorizing models by architecture type facilitates monitoring of developments in the multimodal domain. Distinct from recent survey papers that present general information on multimodal architectures, this research conducts a comprehensive exploration of architectural details and identifies four specific architectural types. The types are distinguished by their respective methodologies for integrating multimodal inputs into the deep neural network model. The first two types (Type A and B) deeply fuses multimodal inputs within the internal layers of the model, whereas the following two types (Type C and D) facilitate early fusion at the input stage. Type-A employs standard cross-attention, whereas Type-B utilizes custom-designed layers for modality fusion within the internal layers. On the other hand, Type-C utilizes modality-specific encoders, while Type-D leverages tokenizers to process the modalities at the model's input stage. The identified architecture types aid the monitoring of any-to-any multimodal model development. Notably, Type-C and Type-D are currently favored in the construction of any-to-any multimodal models. Type-C, distinguished by its non-tokenizing multimodal model architecture, is emerging as a viable alternative to Type-D, which utilizes input-tokenizing techniques. To assist in model selection, this work highlights the advantages and disadvantages of each architecture type based on data and compute requirements, architecture complexity, scalability, simplification of adding modalities, training objectives, and any-to-any multimodal generation capability.


title: Relational Self-supervised Distillation with Compact Descriptors for Image Copy Detection

abstract: This paper addresses image copy detection, a task in online sharing platforms for copyright protection. While previous approaches have performed exceptionally well, the large size of their networks and descriptors remains a significant disadvantage, complicating their practical application. In this paper, we propose a novel method that achieves a competitive performance by using a lightweight network and compact descriptors. By utilizing relational self-supervised distillation to transfer knowledge from a large network to a small network, we enable the training of lightweight networks with a small descriptor size. Our approach, which we call Relational self-supervised Distillation with Compact Descriptors (RDCD), introduces relational self-supervised distillation (RSD) for flexible representation in a smaller feature space and applies contrastive learning with a hard negative (HN) loss to prevent dimensional collapse. We demonstrate the effectiveness of our method using the DISC2021, Copydays, and NDEC benchmark datasets, with which our lightweight network with compact descriptors achieves a competitive performance. For the DISC2021 benchmark, ResNet-50/EfficientNet- B0 are used as a teacher and student respectively, the micro average precision improved by 5.0%/4.9%/5.9% for 64/128/256 descriptor sizes compared to the baseline method.


title: Towards Unified Robustness Against Both Backdoor and Adversarial Attacks

abstract: Deep Neural Networks (DNNs) are known to be vulnerable to both backdoor and adversarial attacks. In the literature, these two types of attacks are commonly treated as distinct robustness problems and solved separately, since they belong to training-time and inference-time attacks respectively. However, this paper revealed that there is an intriguing connection between them: (1) planting a backdoor into a model will significantly affect the model's adversarial examples; (2) for an infected model, its adversarial examples have similar features as the triggered images. Based on these observations, a novel Progressive Unified Defense (PUD) algorithm is proposed to defend against backdoor and adversarial attacks simultaneously. Specifically, our PUD has a progressive model purification scheme to jointly erase backdoors and enhance the model's adversarial robustness. At the early stage, the adversarial examples of infected models are utilized to erase backdoors. With the backdoor gradually erased, our model purification can naturally turn into a stage to boost the model's robustness against adversarial attacks. Besides, our PUD algorithm can effectively identify poisoned images, which allows the initial extra dataset not to be completely clean. Extensive experimental results show that, our discovered connection between backdoor and adversarial attacks is ubiquitous, no matter what type of backdoor attack. The proposed PUD outperforms the state-of-the-art backdoor defense, including the model repairing-based and data filtering-based methods. Besides, it also has the ability to compete with the most advanced adversarial defense methods.


title: Modified double brackets and a conjecture of S. Arthamonov

abstract: Around 20 years ago, M. Van den Bergh introduced double Poisson brackets as operations on associative algebras inducing Poisson brackets under the representation functor. Weaker versions of these operations, called modified double Poisson brackets, were later introduced by S. Arthamonov in order to induce a Poisson bracket on moduli spaces of representations of the corresponding associative algebras. Moreover, he defined two operations that he conjectured to be modified double Poisson brackets. The first case of this conjecture was recently proved by M. Goncharov and V. Gubarev motivated by the theory of Rota-Baxter operators of nonzero weight. We settle the conjecture by realising the second case as part of a new family of modified double Poisson brackets.


title: Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment

abstract: Effectively aligning Large Language Models (LLMs) with human-centric values while preventing the degradation of abilities acquired through Pre-training and Supervised Fine-tuning (SFT) poses a central challenge in Reinforcement Learning from Human Feedback (RLHF). In this paper, we first discover that interpolating RLHF and SFT model parameters can adjust the trade-off between human preference and basic capabilities, thereby reducing the alignment tax at the cost of alignment reward. Inspired by this, we propose integrating the RL policy and SFT models at each optimization step in RLHF to continuously regulate the training direction, introducing the Online Merging Optimizer. Specifically, we merge gradients with the parameter differences between SFT and pretrained models, effectively steering the gradient towards maximizing rewards in the direction of SFT optimization. We demonstrate that our optimizer works well with different LLM families, such as Qwen and LLaMA, across various model sizes ranging from 1.8B to 8B, various RLHF algorithms like DPO and KTO, and existing model merging methods. It significantly enhances alignment reward while mitigating alignment tax, achieving higher overall performance across 14 benchmarks.


title: Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization

abstract: Adaptive moment estimation (Adam), as a Stochastic Gradient Descent (SGD) variant, has gained widespread popularity in federated learning (FL) due to its fast convergence. However, federated Adam (FedAdam) algorithms suffer from a threefold increase in uplink communication overhead compared to federated SGD (FedSGD) algorithms, which arises from the necessity to transmit both local model updates and first and second moment estimates from distributed devices to the centralized server for aggregation. Driven by this issue, we propose a novel sparse FedAdam algorithm called FedAdam-SSM, wherein distributed devices sparsify the updates of local model parameters and moment estimates and subsequently upload the sparse representations to the centralized server. To further reduce the communication overhead, the updates of local model parameters and moment estimates incorporate a shared sparse mask (SSM) into the sparsification process, eliminating the need for three separate sparse masks. Theoretically, we develop an upper bound on the divergence between the local model trained by FedAdam-SSM and the desired model trained by centralized Adam, which is related to sparsification error and imbalanced data distribution. By minimizing the divergence bound between the model trained by FedAdam-SSM and centralized Adam, we optimize the SSM to mitigate the learning performance degradation caused by sparsification error. Additionally, we provide convergence bounds for FedAdam-SSM in both convex and non-convex objective function settings, and investigate the impact of local epoch, learning rate and sparsification ratio on the convergence rate of FedAdam-SSM. Experimental results show that FedAdam-SSM outperforms baselines in terms of convergence rate (over 1.1$\times$ faster than the sparse FedAdam baselines) and test accuracy (over 14.5\% ahead of the quantized FedAdam baselines).


title: ToonCrafter: Generative Cartoon Interpolation

abstract: We introduce ToonCrafter, a novel approach that transcends traditional correspondence-based cartoon video interpolation, paving the way for generative interpolation. Traditional methods, that implicitly assume linear motion and the absence of complicated phenomena like dis-occlusion, often struggle with the exaggerated non-linear and large motions with occlusion commonly found in cartoons, resulting in implausible or even failed interpolation results. To overcome these limitations, we explore the potential of adapting live-action video priors to better suit cartoon interpolation within a generative framework. ToonCrafter effectively addresses the challenges faced when applying live-action video motion priors to generative cartoon interpolation. First, we design a toon rectification learning strategy that seamlessly adapts live-action video priors to the cartoon domain, resolving the domain gap and content leakage issues. Next, we introduce a dual-reference-based 3D decoder to compensate for lost details due to the highly compressed latent prior spaces, ensuring the preservation of fine details in interpolation results. Finally, we design a flexible sketch encoder that empowers users with interactive control over the interpolation results. Experimental results demonstrate that our proposed method not only produces visually convincing and more natural dynamics, but also effectively handles dis-occlusion. The comparative evaluation demonstrates the notable superiority of our approach over existing competitors.


title: Proof of Quality: A Costless Paradigm for Trustless Generative AI Model Inference on Blockchains

abstract: Generative AI models, such as GPT-4 and Stable Diffusion, have demonstrated powerful and disruptive capabilities in natural language and image tasks. However, deploying these models in decentralized environments remains challenging. Unlike traditional centralized deployment, systematically guaranteeing the integrity of AI model services in fully decentralized environments, particularly on trustless blockchains, is both crucial and difficult. In this paper, we present a new inference paradigm called \emph{proof of quality} (PoQ) to enable the deployment of arbitrarily large generative models on blockchain architecture. Unlike traditional approaches based on validating inference procedures, such as ZKML or OPML, our PoQ paradigm focuses on the outcome quality of model inference. Using lightweight BERT-based cross-encoders as our underlying quality evaluation model, we design and implement PQML, the first practical protocol for real-world NLP generative model inference on blockchains, tailored for popular open-source models such as Llama 3 and Mixtral. Our analysis demonstrates that our protocol is robust against adversarial but rational participants in ecosystems, where lazy or dishonest behavior results in fewer benefits compared to well-behaving participants. The computational overhead of validating the quality evaluation is minimal, allowing quality validators to complete the quality check within a second, even using only a CPU. Preliminary simulation results show that PoQ consensus is generated in milliseconds, 1,000 times faster than any existing scheme.


title: Tool Learning with Large Language Models: A Survey

abstract: Recently, tool learning with large language models (LLMs) has emerged as a promising paradigm for augmenting the capabilities of LLMs to tackle highly complex problems. Despite growing attention and rapid advancements in this field, the existing literature remains fragmented and lacks systematic organization, posing barriers to entry for newcomers. This gap motivates us to conduct a comprehensive survey of existing works on tool learning with LLMs. In this survey, we focus on reviewing existing literature from the two primary aspects (1) why tool learning is beneficial and (2) how tool learning is implemented, enabling a comprehensive understanding of tool learning with LLMs. We first explore the "why" by reviewing both the benefits of tool integration and the inherent benefits of the tool learning paradigm from six specific aspects. In terms of "how", we systematically review the literature according to a taxonomy of four key stages in the tool learning workflow: task planning, tool selection, tool calling, and response generation. Additionally, we provide a detailed summary of existing benchmarks and evaluation methods, categorizing them according to their relevance to different stages. Finally, we discuss current challenges and outline potential future directions, aiming to inspire both researchers and industrial developers to further explore this emerging and promising area. We also maintain a GitHub repository to continually keep track of the relevant papers and resources in this rising area at \url{https://github.com/quchangle1/LLM-Tool-Survey}.


title: The 4-Planes of Spin(7) Manifolds

abstract: This paper investigates the geometric structures and properties of 8-dimensional manifolds with Spin(7)-holonomy. We focus on the characterization and implications of 4-planes within these manifolds, which are endowed with an almost symplectic structure compatible with the Spin(7)-structure. We provide a detailed analysis of the differential forms defining these structures, including the Cayley 4-form and its stabilization under the Spin(7) group actions. Furthermore, we explore the concept of mirror duality within the context of exceptional holonomy, elucidating the relationship between dual structures and their topological constraints. By examining the conditions under which mirror duality arises, we enhance the understanding of the interplay between Spin(7)-structures and symplectic geometry. This endeavor contributes to our deeper understanding of the intricate symmetries and topological properties of these remarkable manifolds.


title: Data-driven background model for the CUORE experiment

abstract: We present the model we developed to reconstruct the CUORE radioactive background based on the analysis of an experimental exposure of 1038.4 kg yr. The data reconstruction relies on a simultaneous Bayesian fit applied to energy spectra over a broad energy range. The high granularity of the CUORE detector, together with the large exposure and extended stable operations, allow for an in-depth exploration of both spatial and time dependence of backgrounds. We achieve high sensitivity to both bulk and surface activities of the materials of the setup, detecting levels as low as 10 nBq kg$^{-1}$ and 0.1 nBq cm$^{-2}$, respectively. We compare the contamination levels we extract from the background model with prior radio-assay data, which informs future background risk mitigation strategies. The results of this background model play a crucial role in constructing the background budget for the CUPID experiment as it will exploit the same CUORE infrastructure.


title: RC-Mixup: A Data Augmentation Strategy against Noisy Data for Regression Tasks

abstract: We study the problem of robust data augmentation for regression tasks in the presence of noisy data. Data augmentation is essential for generalizing deep learning models, but most of the techniques like the popular Mixup are primarily designed for classification tasks on image data. Recently, there are also Mixup techniques that are specialized to regression tasks like C-Mixup. In comparison to Mixup, which takes linear interpolations of pairs of samples, C-Mixup is more selective in which samples to mix based on their label distances for better regression performance. However, C-Mixup does not distinguish noisy versus clean samples, which can be problematic when mixing and lead to suboptimal model performance. At the same time, robust training has been heavily studied where the goal is to train accurate models against noisy data through multiple rounds of model training. We thus propose our data augmentation strategy RC-Mixup, which tightly integrates C-Mixup with multi-round robust training methods for a synergistic effect. In particular, C-Mixup improves robust training in identifying clean data, while robust training provides cleaner data to C-Mixup for it to perform better. A key advantage of RC-Mixup is that it is data-centric where the robust model training algorithm itself does not need to be modified, but can simply benefit from data mixing. We show in our experiments that RC-Mixup significantly outperforms C-Mixup and robust training baselines on noisy data benchmarks and can be integrated with various robust training methods.


title: An empirical study of bloated dependencies in CommonJS packages

abstract: JavaScript packages are notoriously prone to bloat, a factor that significantly impacts the performance and maintainability of web applications. While web bundlers and tree-shaking can mitigate this issue in client-side applications at the function level, they cannot effectively detect and remove bloat in server-side applications. In this paper, we conduct an empirical study to investigate the bloated dependencies that are entirely unused within server-side applications. Our study focuses on applications built with the widely used and highly dynamic CommonJS module system. We propose a trace-based dynamic analysis that monitors file access, to determine which dependencies are not accessed during runtime. To conduct our study, we curate an original dataset of 92 CommonJS packages with a median test coverage of 96.9% and a total of 50,661 dependencies. Our dynamic analysis identifies and successfully removes 50.7% of these dependencies while maintaining the correct build of all packages. Furthermore, we find that 14.9% of directly used dependencies and 51.3% of indirect dependencies are bloated. A key insight is that focusing on removing only the direct bloated dependencies by cleaning the package.json file, also removes a significant share of unnecessary bloated indirect dependencies. Compared to the state-of-the-art dynamic debloating technique, our analysis based on file accesses has fewer false positives, and demonstrates higher accuracy in detecting bloated dependencies. Our findings suggest that native support for dependency debloating in package managers could significantly alleviate the burden of maintaining dependencies.


title: World Models for General Surgical Grasping

abstract: Intelligent vision control systems for surgical robots should adapt to unknown and diverse objects while being robust to system disturbances. Previous methods did not meet these requirements due to mainly relying on pose estimation and feature tracking. We propose a world-model-based deep reinforcement learning framework "Grasp Anything for Surgery" (GAS), that learns a pixel-level visuomotor policy for surgical grasping, enhancing both generality and robustness. In particular, a novel method is proposed to estimate the values and uncertainties of depth pixels for a rigid-link object's inaccurate region based on the empirical prior of the object's size; both depth and mask images of task objects are encoded to a single compact 3-channel image (size: 64x64x3) by dynamically zooming in the mask regions, minimizing the information loss. The learned controller's effectiveness is extensively evaluated in simulation and in a real robot. Our learned visuomotor policy handles: i) unseen objects, including 5 types of target grasping objects and a robot gripper, in unstructured real-world surgery environments, and ii) disturbances in perception and control. Note that we are the first work to achieve a unified surgical control system that grasps diverse surgical objects using different robot grippers on real robots in complex surgery scenes (average success rate: 69%). Our system also demonstrates significant robustness across 6 conditions including background variation, target disturbance, camera pose variation, kinematic control error, image noise, and re-grasping after the gripped target object drops from the gripper. Videos and codes can be found on our project page: https://linhongbin.github.io/gas/.


title: Programmable Nonlinear Quantum Photonic Circuits

abstract: The lack of interactions between single photons prohibits direct nonlinear operations in quantum optical circuits, representing a central obstacle in photonic quantum technologies. Here, we demonstrate multi-mode nonlinear photonic circuits where both linear and direct nonlinear operations can be programmed with high precision at the single-photon level. Deterministic nonlinear interaction is realized with a tunable quantum dot embedded in a nanophotonic waveguide mediating interactions between individual photons within a temporal linear optical interferometer. We demonstrate the capability to reprogram the nonlinear photonic circuits and implement protocols where strong nonlinearities are required, in particular for quantum simulation of anharmonic molecular dynamics, thereby showcasing the new key functionalities enabled by our technology.


title: Self-supervised Pre-training for Transferable Multi-modal Perception

abstract: In autonomous driving, multi-modal perception models leveraging inputs from multiple sensors exhibit strong robustness in degraded environments. However, these models face challenges in efficiently and effectively transferring learned representations across different modalities and tasks. This paper presents NeRF-Supervised Masked Auto Encoder (NS-MAE), a self-supervised pre-training paradigm for transferable multi-modal representation learning. NS-MAE is designed to provide pre-trained model initializations for efficient and high-performance fine-tuning. Our approach uses masked multi-modal reconstruction in neural radiance fields (NeRF), training the model to reconstruct missing or corrupted input data across multiple modalities. Specifically, multi-modal embeddings are extracted from corrupted LiDAR point clouds and images, conditioned on specific view directions and locations. These embeddings are then rendered into projected multi-modal feature maps using neural rendering techniques. The original multi-modal signals serve as reconstruction targets for the rendered feature maps, facilitating self-supervised representation learning. Extensive experiments demonstrate the promising transferability of NS-MAE representations across diverse multi-modal and single-modal perception models. This transferability is evaluated on various 3D perception downstream tasks, such as 3D object detection and BEV map segmentation, using different amounts of fine-tuning labeled data. Our code will be released to support the community.


title: Shift-invariant subspaces of Sobolev type

abstract: This paper has the characteristics of a review paper in which results of shift-invariant subspaces of Sobolev type are summarized without proofs. The structure of shift-invariant spaces $V_s$, $s\in\mathbb{R}$, generated by at most countable family of generators, which are subspaces of Sobolev spaces $H^s(\mathbb{R}^n)$, are announced in \cite{aap} and Bessel sequences, frames and Riesz families of such spaces are characterized. With the Fourier multiplier $\left(1-\frac{\Delta}{4\pi^2}\right)^{s/2}f=\mathcal{F}^{-1}\big((1+|t|^2)^{s/2}\widehat{f}(t)\big)$, we are able to extend notions and theorems in \cite{MB} to spaces of the Sobolev type.


title: Remeasuring the Arbitrage and Sandwich Attacks of Maximal Extractable Value in Ethereum

abstract: Maximal Extractable Value (MEV) drives the prosperity of the blockchain ecosystem. By strategically including, excluding, or reordering transactions within blocks, block producers/validators can extract additional value, which in turn incentivizes them to keep the decentralization of the whole blockchain platform. Before The Merge of Ethereum in Sep. 2022, around \$675M was extracted in terms of MEV. Despite its importance, current measurement works on MEV suffer some limitations. First, current works only focus on transactions of a very limited number of DApps. Second, current methods heavily rely on fixed heuristic rule-based patterns, leading to false negative/positive. Third, the observations and conclusions are outdated to some extent due to the continuously introduced features, like The Merge in Ethereum. To address these challenges, in this work, we first propose two robust methods to identify arbitrage transactions and sandwich attacks, respectively. Then, we apply them to the largest-ever dataset to filter out related MEV transactions. Based on the identified results, we have characterized the overall landscape of the Ethereum MEV ecosystem, the impact the private transaction architectures bring, and the adoption of back-running mechanism. Our research will shed light on future MEV-related work.


title: Defining Root-$T\overline{T}$

abstract: We give a tentative definition of the recently introduced Root-$T\bar{T}$ operator in a generic, two dimensional quantum conformal field theory with continuous spectrum of scaling weights. The definition assumes certain factorization properties and uses Schwinger parametrization to introduce the square root. Properties of the operator thus defined are investigated by explicit computation of variations of two- and three-point correlation functions.


title: The communication power of a noisy qubit

abstract: A fundamental property of quantum mechanics is that a single qubit can carry at most 1 bit of classical information. For an important class of quantum communication channels, known as entanglement-breaking, this limitation remains valid even if the sender and receiver share entangled particles before the start of the communication: for every entanglement-breaking channel, the rate at which classical messages can be reliably communicated cannot exceed 1 bit per transmitted qubit even with the assistance of quantum entanglement. But does this mean that, for the purpose of communicating classical messages, a noisy entanglement-breaking qubit channel can be replaced by a noisy bit channel? Here we answer the question in the negative. We introduce a game where a player (the sender) assists another player (the receiver) in finding a prize hidden into one of four possible boxes, while avoiding a bomb hidden in one of the three remaining boxes. In this game, the bomb cannot be avoided with certainty if the players communicate through a noisy bit channel. In contrast, the players can deterministically avoid the bomb and find the prize with a guaranteed 1/3 probability if they communicate through an entanglement-breaking qubit channel known as the universal NOT channel. We show that the features of the quantum strategy can be simulated with a noiseless bit channel, but this simulation requires the transmission to be assisted by shared randomness: without shared randomness, even the noiseless transmission of a three-level classical system cannot match the transmission of a single noisy qubit.


title: High capacity NbS2-based anodes for Li-ion batteries

abstract: We have investigated the lithium capacity of the 2H phase of niobium sulphide (NbS2) using density functional theory calculations and experiments. Theoretically, this material is found to allow the intercalation of a double layer of Li in between each NbS2 layer when in equilibrium with metal Li. The resulting specific capacity (340.8 mAh/g for the pristine material, 681.6 mAh/g for oxidized material) can reach more than double the specific capacity of graphite anodes. The presence of various defects leads to an even higher capacity with a partially reversible conversion of the material, indicating that the performance of the anodes is robust with respect to the presence of defects. Experiments in battery prototypes with NbS2-based anodes find a first specific capacity of about 1,130 mAh/g, exceeding the theoretical predictions.


title: A poset-like approach to positive opetopes

abstract: We introduce in this paper a new formalisation of positive opetopes where faces are organised in a poset. Then we show that our definition is equivalent to that of positives opetopes as given by Marek Zawadowski.


title: Attosecond spectroscopy using vacuum-ultraviolet pulses emitted from laser-driven semiconductors

abstract: Strongly laser-driven semiconductor crystals offer substantial advantages for the study of many-body physics and ultrafast optoelectronics via the high harmonic generation process. While this phenomenon has been employed to investigate the dynamics of solids in the presence of strong laser fields, its potential to be utilized as an attosecond light source has remained unexploited. Here, we demonstrate that the high harmonics generated through the interaction of mid--infrared pulses with a ZnO crystal leads to the production of attosecond pulses, that can be used to trace the ultrafast ionization dynamics of alkali metals. In a cross--correlation approach, we photoionize Cesium atoms with the vacuum-ultraviolet (VUV) high-harmonics in the presence of a mid-infrared laser field. We observe strong oscillations of the photoelectron yield originating from the instantaneous polarization of the atoms by the laser field. The phase of the oscillations encodes the attosecond synchronization of the ionizing high-harmonics and is used for attosecond pulse metrology. This light source opens a new spectral window for attosecond spectroscopy, paving the way for studies of systems with low ionization potentials including neutral atoms, molecules and solids. Additionally, our results highlight the significance of the source for generating non--classical massively entangled light states in the visible--VUV spectral region.


title: Self-Guiding Exploration for Combinatorial Problems

abstract: Large Language Models (LLMs) have become pivotal in addressing reasoning tasks across diverse domains, including arithmetic, commonsense, and symbolic reasoning. They utilize prompting techniques such as Exploration-of-Thought, Decomposition, and Refinement to effectively navigate and solve intricate tasks. Despite these advancements, the application of LLMs to Combinatorial Problems (CPs), known for their NP-hardness and critical roles in logistics and resource management remains underexplored. To address this gap, we introduce a novel prompting strategy: Self-Guiding Exploration (SGE), designed to enhance the performance of solving CPs. SGE operates autonomously, generating multiple thought trajectories for each CP task. It then breaks these trajectories down into actionable subtasks, executes them sequentially, and refines the results to ensure optimal outcomes. We present our research as the first to apply LLMs to a broad range of CPs and demonstrate that SGE outperforms existing prompting strategies by over 27.84% in CP optimization performance. Additionally, SGE achieves a 2.46% higher accuracy over the best existing results in other reasoning tasks (arithmetic, commonsense, and symbolic).


title: Efficient Time Series Processing for Transformers and State-Space Models through Token Merging

abstract: Transformer architectures have shown promising results in time series processing. However, despite recent advances in subquadratic attention mechanisms or state-space models, processing very long sequences still imposes significant computational requirements. Token merging, which involves replacing multiple tokens with a single one calculated as their linear combination, has shown to considerably improve the throughput of vision transformer architectures while maintaining accuracy. In this work, we go beyond computer vision and perform the first investigations of token merging in time series analysis on both time series transformers and state-space models. To effectively scale token merging to long sequences, we introduce local merging, a domain-specific token merging algorithm that selectively combines tokens within a local neighborhood, adjusting the computational complexity from linear to quadratic based on the neighborhood size. Our comprehensive empirical evaluation demonstrates that token merging offers substantial computational benefits with minimal impact on accuracy across various models and datasets. On the recently proposed Chronos foundation model, we achieve accelerations up to 5400% with only minor accuracy degradations.


title: Upper Bounds on the Average Height of Random Binary Trees

abstract: We study the average height of random trees generated by leaf-centric binary tree sources as introduced by Zhang, Yang and Kieffer. A leaf-centric binary tree source induces for every $n \geq 2$ a probability distribution on the set of binary trees with $n$ leaves. Our results generalize a result by Devroye, according to which the average height of a random binary search tree of size $n$ is in $\mathcal{O}(\log n)$.


title: Graph Threading with Turn Costs

abstract: How should we thread a single string through a set of tubes so that pulling the string taut self-assembles the tubes into a desired graph? While prior work [ITCS 2024] solves this problem with the goal of minimizing the length of string, we study here the objective of minimizing the total turn cost. The frictional force required to pull the string through the tubes grows exponentially with the total absolute turn angles (by the Capstan equation), so this metric often dominates the friction in real-world applications such as deployable structures. We show that minimum-turn threading is NP-hard, even for graphs of maximum degree 4, and even when restricted to some special cases of threading. On the other hand, we show that these special cases can in fact be solved efficiently for graphs of maximum degree 4, thereby fully characterizing their dependence on maximum degree. We further provide polynomial-time exact and approximation algorithms for variants of turn-cost threading: restricting to threading each edge exactly twice, and on rectangular grid graphs.


title: Comparison of predictive values with paired samples

abstract: Positive predictive value and negative predictive value are two widely used parameters to assess the clinical usefulness of a medical diagnostic test. When there are two diagnostic tests, it is recommendable to make a comparative assessment of the values of these two parameters after applying the two tests to the same subjects (paired samples). The objective is then to make individual or global inferences about the difference or the ratio of the predictive value of the two diagnostic tests. These inferences are usually based on complex and not very intuitive expressions, some of which have subsequently been reformulated. We define the two properties of symmetry which any inference method must verify - symmetry in diagnoses and symmetry in the tests -, we propose new inference methods, and we define them with simple expressions. All of the methods are compared with each other, selecting the optimal method: (a) to obtain a confidence interval for the difference or ratio; (b) to perform an individual homogeneity test of the two predictive values; and (c) to carry out a global homogeneity test of the two predictive values.


title: Efficient Prior Calibration From Indirect Data

abstract: Bayesian inversion is central to the quantification of uncertainty within problems arising from numerous applications in science and engineering. To formulate the approach, four ingredients are required: a forward model mapping the unknown parameter to an element of a solution space, often the solution space for a differential equation; an observation operator mapping an element of the solution space to the data space; a noise model describing how noise pollutes the observations; and a prior model describing knowledge about the unknown parameter before the data is acquired. This paper is concerned with learning the prior model from data; in particular, learning the prior from multiple realizations of indirect data obtained through the noisy observation process. The prior is represented, using a generative model, as the pushforward of a Gaussian in a latent space; the pushforward map is learned by minimizing an appropriate loss function. A metric that is well-defined under empirical approximation is used to define the loss function for the pushforward map to make an implementable methodology. Furthermore, an efficient residual-based neural operator approximation of the forward model is proposed and it is shown that this may be learned concurrently with the pushforward map, using a bilevel optimization formulation of the problem; this use of neural operator approximation has the potential to make prior learning from indirect data more computationally efficient, especially when the observation process is expensive, non-smooth or not known. The ideas are illustrated with the Darcy flow inverse problem of finding permeability from piezometric head measurements.


title: Hybrid Preference Optimization: Augmenting Direct Preference Optimization with Auxiliary Objectives

abstract: For aligning large language models (LLMs), prior work has leveraged reinforcement learning via human feedback (RLHF) or variations of direct preference optimization (DPO). While DPO offers a simpler framework based on maximum likelihood estimation, it compromises on the ability to tune language models to easily maximize non-differentiable and non-binary objectives according to the LLM designer's preferences (e.g., using simpler language or minimizing specific kinds of harmful content). These may neither align with user preferences nor even be able to be captured tractably by binary preference data. To leverage the simplicity and performance of DPO with the generalizability of RL, we propose a hybrid approach between DPO and RLHF. With a simple augmentation to the implicit reward decomposition of DPO, we allow for tuning LLMs to maximize a set of arbitrary auxiliary rewards using offline RL. The proposed method, Hybrid Preference Optimization (HPO), shows the ability to effectively generalize to both user preferences and auxiliary designer objectives, while preserving alignment performance across a range of challenging benchmarks and model sizes.


title: Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion

abstract: Dynamic topic models track the evolution of topics in sequential documents, which have derived various applications like trend analysis and opinion mining. However, existing models suffer from repetitive topic and unassociated topic issues, failing to reveal the evolution and hindering further applications. To address these issues, we break the tradition of simply chaining topics in existing work and propose a novel neural \modelfullname. We introduce a new evolution-tracking contrastive learning method that builds the similarity relations among dynamic topics. This not only tracks topic evolution but also maintains topic diversity, mitigating the repetitive topic issue. To avoid unassociated topics, we further present an unassociated word exclusion method that consistently excludes unassociated words from discovered topics. Extensive experiments demonstrate our model significantly outperforms state-of-the-art baselines, tracking topic evolution with high-quality topics, showing better performance on downstream tasks, and remaining robust to the hyperparameter for evolution intensities. Our code is available at https://github.com/bobxwu/CFDTM .


title: FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes

abstract: Empowering 3D Gaussian Splatting with generalization ability is appealing. However, existing generalizable 3D Gaussian Splatting methods are largely confined to narrow-range interpolation between stereo images due to their heavy backbones, thus lacking the ability to accurately localize 3D Gaussian and support free-view synthesis across wide view range. In this paper, we present a novel framework FreeSplat that is capable of reconstructing geometrically consistent 3D scenes from long sequence input towards free-view synthesis.Specifically, we firstly introduce Low-cost Cross-View Aggregation achieved by constructing adaptive cost volumes among nearby views and aggregating features using a multi-scale structure. Subsequently, we present the Pixel-wise Triplet Fusion to eliminate redundancy of 3D Gaussians in overlapping view regions and to aggregate features observed across multiple views. Additionally, we propose a simple but effective free-view training strategy that ensures robust view synthesis across broader view range regardless of the number of views. Our empirical results demonstrate state-of-the-art novel view synthesis peformances in both novel view rendered color maps quality and depth maps accuracy across different numbers of input views. We also show that FreeSplat performs inference more efficiently and can effectively reduce redundant Gaussians, offering the possibility of feed-forward large scene reconstruction without depth priors.


title: Attention-based sequential recommendation system using multimodal data

abstract: Sequential recommendation systems that model dynamic preferences based on a use's past behavior are crucial to e-commerce. Recent studies on these systems have considered various types of information such as images and texts. However, multimodal data have not yet been utilized directly to recommend products to users. In this study, we propose an attention-based sequential recommendation method that employs multimodal data of items such as images, texts, and categories. First, we extract image and text features from pre-trained VGG and BERT and convert categories into multi-labeled forms. Subsequently, attention operations are performed independent of the item sequence and multimodal representations. Finally, the individual attention information is integrated through an attention fusion function. In addition, we apply multitask learning loss for each modality to improve the generalization performance. The experimental results obtained from the Amazon datasets show that the proposed method outperforms those of conventional sequential recommendation systems.


title: Elementary Flux Modes as CRN Gears for Free Energy Transduction

abstract: We demonstrate that, for a chemical reaction network (CRN) engaged in energy transduction, its optimal operation from a thermodynamic efficiency standpoint is contingent upon its working conditions. Analogously to the bicycle gear system, CRNs have at their disposal several transducing mechanisms characterized by different yields. We highlight the critical role of the CRN's elementary flux modes in determining this "gearing" and their impact on maximizing energy transduction efficiency. Furthermore, we introduce an enzymatically regulated CRN, engineered to autonomously adjust its "gear", thereby optimizing its efficiency under different external conditions.


title: Global $L^p$ estimate for some kind of Kolmogorov-Fokker-Planck Equations in nondivergence form

abstract: In this paper, we mainly investigate a class of Kolmogorov-Fokker-Planck operator with 4 different scalings in nondivergence form. And we assume the coefficients $a^{ij}$ are only measurable in $t$ and satisfy the vanishing mean oscillation in space variables. We establish a global priori estimates of $\nabla_x^u$, $\dy u$ and $\dz u$ in $L^p$ space which extend the work of Dong and Yastrzhembskiy \cite{ref49} where they focus on the 3 different scalings KFP operator. Moreover we establish a kind of Poincare inequality for homogeneous equations.


title: Detection of carbon monoxide using a polarization-multiplexed erbium dual-comb fiber laser

abstract: We present a compact, reliable, and robust free-running all-polarization-maintaining erbium (Er) single-cavity dual-comb laser generated via polarization multiplexing with gain sharing. Polarization multiplexing exploits the fast and slow axes of the fiber, while modelocking is achieved through a nonlinear amplifying loop mirror scheme using readily available components. The laser operates at a repetition rate of around 74.74 MHz with a tuning capability in the difference in repetition rates from 500 Hz to 200 kHz. This tunability makes the system more flexible for dual-comb spectroscopy experiments. Consequently, using this laser, we demonstrated a proof-of-principle dual-comb spectroscopy of carbon monoxide (CO), operating without any active stabilization.


title: Band geometry induced electro-optic effect and polarization rotation

abstract: Electric field-induced modulation of the optical properties is crucial for amplitude and phase modulators used in photonic devices. Here, we present a comprehensive study of the band geometry-induced electro-optic effect, specifically focusing on the Fermi surface and disorder-induced contributions. These contributions are crucial for metallic and semimetallic systems and for optical frequencies comparable to or smaller than the scattering rates. We highlight the importance of the quantum metric and metric connection in generating the electro-optic effect in parity-time reversal ($\mathcal{PT}$) symmetric systems such as CuMnAs. Our findings establish the electro-optic effect as a novel tool to probe band geometric effects and open new avenues to design electrically controlled efficient amplitude and phase modulators for photonic applications.


title: Transformer and Hybrid Deep Learning Based Models for Machine-Generated Text Detection

abstract: This paper describes the approach of the UniBuc - NLP team in tackling the SemEval 2024 Task 8: Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection. We explored transformer-based and hybrid deep learning architectures. For subtask B, our transformer-based model achieved a strong \textbf{second-place} out of $77$ teams with an accuracy of \textbf{86.95\%}, demonstrating the architecture's suitability for this task. However, our models showed overfitting in subtask A which could potentially be fixed with less fine-tunning and increasing maximum sequence length. For subtask C (token-level classification), our hybrid model overfit during training, hindering its ability to detect transitions between human and machine-generated text.


title: AttenCraft: Attention-guided Disentanglement of Multiple Concepts for Text-to-Image Customization

abstract: With the unprecedented performance being achieved by text-to-image (T2I) diffusion models, T2I customization further empowers users to tailor the diffusion model to new concepts absent in the pre-training dataset, termed subject-driven generation. Moreover, extracting several new concepts from a single image enables the model to learn multiple concepts, and simultaneously decreases the difficulties of training data preparation, urging the disentanglement of multiple concepts to be a new challenge. However, existing models for disentanglement commonly require pre-determined masks or retain background elements. To this end, we propose an attention-guided method, AttenCraft, for multiple concept disentanglement. In particular, our method leverages self-attention and cross-attention maps to create accurate masks for each concept within a single initialization step, omitting any required mask preparation by humans or other models. The created masks are then applied to guide the cross-attention activation of each target concept during training and achieve concept disentanglement. Additionally, we introduce Uniform sampling and Reweighted sampling schemes to alleviate the non-synchronicity of feature acquisition from different concepts, and improve generation quality. Our method outperforms baseline models in terms of image-alignment, and behaves comparably on text-alignment. Finally, we showcase the applicability of AttenCraft to more complicated settings, such as an input image containing three concepts. The project is available at https://github.com/junjie-shentu/AttenCraft.


title: An even-parity precession protocol for detecting nonclassicality and entanglement

abstract: We introduce an even-parity precession protocol that can detect nonclassicality of some quantum states using only measurements of a uniformly-precessing variable at different points in time. Depending on the system under study, the protocol may detect the Wigner negativity of a single quantum harmonic oscillator or of a single spin $j\geq 2$; the non-Gaussian entanglement of two harmonic oscillators; or genuine multipartite entanglement of a spin ensemble, whose total spin is integer. Unlike other nonclassicality tests, simultaneous or sequential measurements are not required. Our protocol can also detect states that commute with the parity operator, which were missed by similar protocols built from Tsirelson's original precession protocol. This work also closes a long-standing gap by showing the possibility of detecting the Greenberger--Horne--Zeilinger entanglement of an even number of qubits using only collective spin measurements.


title: Charge Amplification in Low Pressure CF4:SF6:He Mixtures with a Multi-Mesh ThGEM for Directional Dark Matter Searches

abstract: The CYGNO collaboration is developing next generation directional Dark Matter (DM) detection experiments, using gaseous Time Projection Chambers (TPCs), as a robust method for identifying Weakly Interacting Massive Particles (WIMPs) below the Neutrino Fog. SF6 is potentially ideal for this since it provides a high fluorine content, enhancing sensitivity to spin-dependent interactions and, as a Negative Ion Drift (NID) gas, reduces charge diffusion leading to improved positional resolution. CF4, although not a NID gas, has also been identified as a favourable gas target as it provides a scintillation signal which can be used for a complimentary light/charge readout approach. These gases can operate at low pressures to elongate Nuclear Recoil (NR) tracks and facilitate directional measurements. In principle, He could be added to low pressure SF6/CF4 without significant detriment to the length of 16S, 12C, and 19F recoils. This would improve the target mass, sensitivity to lower WIMP masses, and offer the possibility of atmospheric operation; potentially reducing the cost of a containment vessel. In this article, we present gas gain and energy resolution measurements, taken with a Multi-Mesh Thick Gaseous Electron Multiplier (MMThGEM), in low pressure SF6 and CF4:SF6 mixtures following the addition of He. We find that the CF4:SF6:He mixtures tested were able to produce gas gains on the order of 10^4 up to a total pressure of 100 Torr. These results demonstrate an order of magnitude improvement in charge amplification in NID gas mixtures with a He component.


title: Matroid Semi-Bandits in Sublinear Time

abstract: We study the matroid semi-bandits problem, where at each round the learner plays a subset of $K$ arms from a feasible set, and the goal is to maximize the expected cumulative linear rewards. Existing algorithms have per-round time complexity at least $\Omega(K)$, which becomes expensive when $K$ is large. To address this computational issue, we propose FasterCUCB whose sampling rule takes time sublinear in $K$ for common classes of matroids: $O(D\text{ polylog}(K)\text{ polylog}(T))$ for uniform matroids, partition matroids, and graphical matroids, and $O(D\sqrt{K}\text{ polylog}(T))$ for transversal matroids. Here, $D$ is the maximum number of elements in any feasible subset of arms, and $T$ is the horizon. Our technique is based on dynamic maintenance of an approximate maximum-weight basis over inner-product weights. Although the introduction of an approximate maximum-weight basis presents a challenge in regret analysis, we can still guarantee an upper bound on regret as tight as CUCB in the sense that it matches the gap-dependent lower bound by Kveton et al. (2014a) asymptotically.


title: Knowledge Circuits in Pretrained Transformers

abstract: The remarkable capabilities of modern large language models are rooted in their vast repositories of knowledge encoded within their parameters, enabling them to perceive the world and engage in reasoning. The inner workings of how these models store knowledge have long been a subject of intense interest and investigation among researchers. To date, most studies have concentrated on isolated components within these models, such as the Multilayer Perceptrons and attention head. In this paper, we delve into the computation graph of the language model to uncover the knowledge circuits that are instrumental in articulating specific knowledge. The experiments, conducted with GPT2 and TinyLLAMA, has allowed us to observe how certain information heads, relation heads, and Multilayer Perceptrons collaboratively encode knowledge within the model. Moreover, we evaluate the impact of current knowledge editing techniques on these knowledge circuits, providing deeper insights into the functioning and constraints of these editing methodologies. Finally, we utilize knowledge circuits to analyze and interpret language model behaviors such as hallucinations and in-context learning. We believe the knowledge circuit holds potential for advancing our understanding of Transformers and guiding the improved design of knowledge editing. Code and data are available in https://github.com/zjunlp/KnowledgeCircuits.