title: HOACS: Homomorphic Obfuscation Assisted Concealing of Secrets to Thwart Trojan Attacks in COTS Processor

abstract: Commercial-off-the-shelf (COTS) components are often preferred over custom Integrated Circuits (ICs) to achieve reduced system development time and cost, easy adoption of new technologies, and replaceability. Unfortunately, the integration of COTS components introduces serious security concerns. None of the entities in the COTS IC supply chain are trusted from a consumer's perspective, leading to a ''zero trust'' threat model. Any of these entities could introduce hidden malicious circuits or hardware Trojans within the component, allowing an attacker in the field to extract secret information (e.g., cryptographic keys) or cause a functional failure. Existing solutions to counter hardware Trojans are inapplicable in such a zero-trust scenario as they assume either the design house or the foundry to be trusted and consider the design to be available for either analysis or modification. In this work, we have proposed a software-oriented countermeasure to ensure the confidentiality of secret assets against hardware Trojans that can be seamlessly integrated in existing COTS microprocessors. The proposed solution does not require any supply chain entity to be trusted and does not require analysis or modification of the IC design. To protect secret assets in an untrusted microprocessor, the proposed method leverages the concept of residue number coding (RNC) to transform the software functions operating on the asset to be fully homomorphic. We have implemented the proposed solution to protect the secret key within the Advanced Encryption Standard (AES) program and presented a detailed security analysis. We also have developed a plugin for the LLVM compiler toolchain that automatically integrates the solution in AES. Finally, we compare the execution time overhead of the operations in the RNC-based technique with comparable homomorphic solutions and demonstrate significant improvement.


title: Sparse and Faithful Explanations Without Sparse Models

abstract: Even if a model is not globally sparse, it is possible for decisions made from that model to be accurately and faithfully described by a small number of features. For instance, an application for a large loan might be denied to someone because they have no credit history, which overwhelms any evidence towards their creditworthiness. In this work, we introduce the Sparse Explanation Value (SEV), a new way of measuring sparsity in machine learning models. In the loan denial example above, the SEV is 1 because only one factor is needed to explain why the loan was denied. SEV is a measure of decision sparsity rather than overall model sparsity, and we are able to show that many machine learning models -- even if they are not sparse -- actually have low decision sparsity, as measured by SEV. SEV is defined using movements over a hypercube, allowing SEV to be defined consistently over various model classes, with movement restrictions reflecting real-world constraints. We proposed the algorithms that reduce SEV without sacrificing accuracy, providing sparse and completely faithful explanations, even without globally sparse models.


title: Canalization reduces the nonlinearity of regulation in biological networks

abstract: Biological networks such as gene regulatory networks possess desirable properties. They are more robust and controllable than random networks. This motivates the search for structural and dynamical features that evolution has incorporated in biological networks. A recent meta-analysis of published, expert-curated Boolean biological network models has revealed several such features, often referred to as design principles. Among others, the biological networks are enriched for certain recurring network motifs, the dynamic update rules are more redundant, more biased and more canalizing than expected, and the dynamics of biological networks are better approximable by linear and lower-order approximations than those of comparable random networks. Since most of these features are interrelated, it is paramount to disentangle cause and effect, that is, to understand which features evolution actively selects for, and thus truly constitute evolutionary design principles. Here, we show that approximability is strongly dependent on the dynamical robustness of a network, and that increased canalization in biological networks can almost completely explain their recently postulated high approximability.


title: A Study of Gram Determinants in Knot Theory

abstract: Historically originated as a sub-field of topology, knot theory is an active area of mathematical investigation that has strong connections with a diverse set of scientific fields such as algebra, biology, and statistical mechanics. A popular and important concept in linear algebra, Gram determinants enjoy a connection with the mathematical theory of knots. In this article, we expose this concept and present several types of Gram determinants in what can be considered as a survey of the current Gram determinants of interest to knot theorists; examples are included to illustrate the definitions. In particular, we pay special attention to a recently defined determinant from a M\"obius band and we further study its structure. At the end, some speculation is presented regarding the closed formula for the Gram determinant of type $(Mb)_1$, a problem that arouses serious interest among knot theorists.


title: Linear Depth QFT over IBM Heavy-hex Architecture

abstract: Compiling a given quantum algorithm into a target hardware architecture is a challenging optimization problem. The compiler must take into consideration the coupling graph of physical qubits and the gate operation dependencies. The existing noise in hardware architectures requires the compilation to use as few running cycles as possible. Existing approaches include using SAT solver or heuristics to complete the mapping but these may cause the issue of either long compilation time (e.g., timeout after hours) or suboptimal compilation results in terms of running cycles (e.g., exponentially increasing number of total cycles). In this paper, we propose an efficient mapping approach for Quantum Fourier Transformation (QFT) circuits over the existing IBM heavy-hex architecture. Such proposal first of all turns the architecture into a structure consisting of a straight line with dangling qubits, and then do the mapping over this generated structure recursively. The calculation shows that there is a linear depth upper bound for the time complexity of these structures and for a special case where there is 1 dangling qubit in every 5 qubits, the time complexity is 5N+O(1). All these results are better than state of the art methods.


title: Site-selective cobalt substitution in La-Co co-substituted magnetoplumbite-type ferrites: $^{59}$Co-NMR and DFT calculation study

abstract: The La-Co co-substituted magnetoplumbite-type (M-type) ferrites $A$Fe$_{12}$O$_{19}$ ($A$ = Ca, Sr and Ba, ion sizes Ca$^{2+}$ $<$ Sr$^{2+}$ $<$ Ba$^{2+}$) with Co compositions around 0.2 have been subjected to $^{59}$Co-NMR. The results show that Co occupies the 4f$_1$, 2a and 12k sites, and that the smaller the $A$ ion, the more Co tends to occupy the 4f$_1$ minority spin site, which is effective in enhancing both uniaxial anisotropy and magnetisation. First-principles total energy calculations based on density functional theory (DFT) of undoped $A$Fe$_{12}$O$_{19}$ and a supercell ($2 \times 2 \times 1$ of the unit cell) in which 1/96 of Fe$^{3+}$ is replaced by Co$^{2+}$ were performed to predict the stable structure and Co occupancy sites. The results show that regardless of $A$, Co is most stable when it occupies the 4f$_1$ site, followed by the 2a and 12k sites with energy differences on the order of 100 meV, and Co practically does not occupy the 2b and 4f$_2$ sites. As the $A$ ion becomes smaller, the energy difference when Co occupies each Fe site tends to increase, and the Co occupancy of the 4f$_1$ site also increases. The site selectivity of Co can be roughly explained as a result of the difference in uniaxial strain along the $c$-axis associated with the difference in $A$. However, the influence of the $A$ ion differs between the R and S blocks and the local strain also has a secondary effect on the Co distribution. Based on these results, the guidelines for improving the performance (anisotropy and magnetisation) of La-Co co-substituted M-type ferrite magnets with a limited amount of Co can be summarised as follows: It is effective to select as small $A$ ions as possible and to post-anneal at low temperature or cool slowly to concentrate Co at the 4f$_1$ site in tetrahedral coordination.


title: On the adversarial robustness of Locality-Sensitive Hashing in Hamming space

abstract: Locality-sensitive hashing~[Indyk,Motwani'98] is a classical data structure for approximate nearest neighbor search. It allows, after a close to linear time preprocessing of the input dataset, to find an approximately nearest neighbor of any fixed query in sublinear time in the dataset size. The resulting data structure is randomized and succeeds with high probability for every fixed query. In many modern applications of nearest neighbor search the queries are chosen adaptively. In this paper, we study the robustness of the locality-sensitive hashing to adaptive queries in Hamming space. We present a simple adversary that can, under mild assumptions on the initial point set, provably find a query to the approximate near neighbor search data structure that the data structure fails on. Crucially, our adaptive algorithm finds the hard query exponentially faster than random sampling.


title: Are Odd Radio Circles phoenixes of powerful radio galaxies?

abstract: Odd Radio Circles (ORCs) are a class of low surface brightness, circular objects approximately one arcminute in diameter. ORCs were recently discovered in the Australian Square Kilometre Array Pathfinder (ASKAP) data, and subsequently confirmed with follow-up observations on other instruments, yet their origins remain uncertain. In this paper, we suggest that ORCs could be remnant lobes of powerful radio galaxies, re-energised by the passage of a shock. Using relativistic hydrodynamic simulations with synchrotron emission calculated in post-processing, we show that buoyant evolution of remnant radio lobes is alone too slow to produce the observed ORC morphology. However, the passage of a shock can produce both filled and edge-brightnened ORC-like morphologies for a wide variety of shock and observing orientations. Circular ORCs are predicted to have host galaxies near the geometric centre of the radio emission, consistent with observations of these objects. Significantly offset hosts are possible for elliptical ORCs, potentially causing challenges for accurate host galaxy identification. Observed ORC number counts are broadly consistent with a paradigm in which moderately powerful radio galaxies are their progenitors.


title: ME-ViT: A Single-Load Memory-Efficient FPGA Accelerator for Vision Transformers

abstract: Vision Transformers (ViTs) have emerged as a state-of-the-art solution for object classification tasks. However, their computational demands and high parameter count make them unsuitable for real-time inference, prompting the need for efficient hardware implementations. Existing hardware accelerators for ViTs suffer from frequent off-chip memory access, restricting the achievable throughput by memory bandwidth. In devices with a high compute-to-communication ratio (e.g., edge FPGAs with limited bandwidth), off-chip memory access imposes a severe bottleneck on overall throughput. This work proposes ME-ViT, a novel \underline{M}emory \underline{E}fficient FPGA accelerator for \underline{ViT} inference that minimizes memory traffic. We propose a \textit{single-load policy} in designing ME-ViT: model parameters are only loaded once, intermediate results are stored on-chip, and all operations are implemented in a single processing element. To achieve this goal, we design a memory-efficient processing element (ME-PE), which processes multiple key operations of ViT inference on the same architecture through the reuse of \textit{multi-purpose buffers}. We also integrate the Softmax and LayerNorm functions into the ME-PE, minimizing stalls between matrix multiplications. We evaluate ME-ViT on systolic array sizes of 32 and 16, achieving up to a 9.22$\times$ and 17.89$\times$ overall improvement in memory bandwidth, and a 2.16$\times$ improvement in throughput per DSP for both designs over state-of-the-art ViT accelerators on FPGA. ME-ViT achieves a power efficiency improvement of up to 4.00$\times$ (1.03$\times$) over a GPU (FPGA) baseline. ME-ViT enables up to 5 ME-PE instantiations on a Xilinx Alveo U200, achieving a 5.10$\times$ improvement in throughput over the state-of-the art FPGA baseline, and a 5.85$\times$ (1.51$\times$) improvement in power efficiency over the GPU (FPGA) baseline.


title: Preserving Data Privacy for ML-driven Applications in Open Radio Access Networks

abstract: Deep learning offers a promising solution to improve spectrum access techniques by utilizing data-driven approaches to manage and share limited spectrum resources for emerging applications. For several of these applications, the sensitive wireless data (such as spectrograms) are stored in a shared database or multistakeholder cloud environment and are therefore prone to privacy leaks. This paper aims to address such privacy concerns by examining the representative case study of shared database scenarios in 5G Open Radio Access Network (O-RAN) networks where we have a shared database within the near-real-time (near-RT) RAN intelligent controller. We focus on securing the data that can be used by machine learning (ML) models for spectrum sharing and interference mitigation applications without compromising the model and network performances. The underlying idea is to leverage a (i) Shuffling-based learnable encryption technique to encrypt the data, following which, (ii) employ a custom Vision transformer (ViT) as the trained ML model that is capable of performing accurate inferences on such encrypted data. The paper offers a thorough analysis and comparisons with analogous convolutional neural networks (CNN) as well as deeper architectures (such as ResNet-50) as baselines. Our experiments showcase that the proposed approach significantly outperforms the baseline CNN with an improvement of 24.5% and 23.9% for the percent accuracy and F1-Score respectively when operated on encrypted data. Though deeper ResNet-50 architecture is obtained as a slightly more accurate model, with an increase of 4.4%, the proposed approach boasts a reduction of parameters by 99.32%, and thus, offers a much-improved prediction time by nearly 60%.


title: Node Duplication Improves Cold-start Link Prediction

abstract: Graph Neural Networks (GNNs) are prominent in graph machine learning and have shown state-of-the-art performance in Link Prediction (LP) tasks. Nonetheless, recent studies show that GNNs struggle to produce good results on low-degree nodes despite their overall strong performance. In practical applications of LP, like recommendation systems, improving performance on low-degree nodes is critical, as it amounts to tackling the cold-start problem of improving the experiences of users with few observed interactions. In this paper, we investigate improving GNNs' LP performance on low-degree nodes while preserving their performance on high-degree nodes and propose a simple yet surprisingly effective augmentation technique called NodeDup. Specifically, NodeDup duplicates low-degree nodes and creates links between nodes and their own duplicates before following the standard supervised LP training scheme. By leveraging a ''multi-view'' perspective for low-degree nodes, NodeDup shows significant LP performance improvements on low-degree nodes without compromising any performance on high-degree nodes. Additionally, as a plug-and-play augmentation module, NodeDup can be easily applied to existing GNNs with very light computational cost. Extensive experiments show that NodeDup achieves 38.49%, 13.34%, and 6.76% improvements on isolated, low-degree, and warm nodes, respectively, on average across all datasets compared to GNNs and state-of-the-art cold-start methods.


title: Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement

abstract: Disentangled representation learning strives to extract the intrinsic factors within observed data. Factorizing these representations in an unsupervised manner is notably challenging and usually requires tailored loss functions or specific structural designs. In this paper, we introduce a new perspective and framework, demonstrating that diffusion models with cross-attention can serve as a powerful inductive bias to facilitate the learning of disentangled representations. We propose to encode an image to a set of concept tokens and treat them as the condition of the latent diffusion for image reconstruction, where cross-attention over the concept tokens is used to bridge the interaction between the encoder and diffusion. Without any additional regularization, this framework achieves superior disentanglement performance on the benchmark datasets, surpassing all previous methods with intricate designs. We have conducted comprehensive ablation studies and visualization analysis, shedding light on the functioning of this model. This is the first work to reveal the potent disentanglement capability of diffusion models with cross-attention, requiring no complex designs. We anticipate that our findings will inspire more investigation on exploring diffusion for disentangled representation learning towards more sophisticated data analysis and understanding.


title: Variations on quantum de Finetti theorems and operator valued Martin boundaries: a Choquet-Deny approach

abstract: We revisit the quantum de Finetti theorem. We state and prove a couple of variants thereof. In parallel, we introduce an operator version of the Martin boundary on quantum groups and prove generalizations of Biane's theoresm. Our proof of the de Finetti theorem is new in the sense that it is based on an analogy with the theory of operator valued Martin boundary that we introduce.


title: An Accelerated Distributed Stochastic Gradient Method with Momentum

abstract: In this paper, we introduce an accelerated distributed stochastic gradient method with momentum for solving the distributed optimization problem, where a group of $n$ agents collaboratively minimize the average of the local objective functions over a connected network. The method, termed ``Distributed Stochastic Momentum Tracking (DSMT)'', is a single-loop algorithm that utilizes the momentum tracking technique as well as the Loopless Chebyshev Acceleration (LCA) method. We show that DSMT can asymptotically achieve comparable convergence rates as centralized stochastic gradient descent (SGD) method under a general variance condition regarding the stochastic gradients. Moreover, the number of iterations (transient times) required for DSMT to achieve such rates behaves as $\mathcal{O}(n^{5/3}/(1-\lambda))$ for minimizing general smooth objective functions, and $\mathcal{O}(\sqrt{n/(1-\lambda)})$ under the Polyak-{\L}ojasiewicz (PL) condition. Here, the term $1-\lambda$ denotes the spectral gap of the mixing matrix related to the underlying network topology. Notably, the obtained results do not rely on multiple inter-node communications or stochastic gradient accumulation per iteration, and the transient times are the shortest under the setting to the best of our knowledge.


title: DPBalance: Efficient and Fair Privacy Budget Scheduling for Federated Learning as a Service

abstract: Federated learning (FL) has emerged as a prevalent distributed machine learning scheme that enables collaborative model training without aggregating raw data. Cloud service providers further embrace Federated Learning as a Service (FLaaS), allowing data analysts to execute their FL training pipelines over differentially-protected data. Due to the intrinsic properties of differential privacy, the enforced privacy level on data blocks can be viewed as a privacy budget that requires careful scheduling to cater to diverse training pipelines. Existing privacy budget scheduling studies prioritize either efficiency or fairness individually. In this paper, we propose DPBalance, a novel privacy budget scheduling mechanism that jointly optimizes both efficiency and fairness. We first develop a comprehensive utility function incorporating data analyst-level dominant shares and FL-specific performance metrics. A sequential allocation mechanism is then designed using the Lagrange multiplier method and effective greedy heuristics. We theoretically prove that DPBalance satisfies Pareto Efficiency, Sharing Incentive, Envy-Freeness, and Weak Strategy Proofness. We also theoretically prove the existence of a fairness-efficiency tradeoff in privacy budgeting. Extensive experiments demonstrate that DPBalance outperforms state-of-the-art solutions, achieving an average efficiency improvement of $1.44\times \sim 3.49 \times$, and an average fairness improvement of $1.37\times \sim 24.32 \times$.


title: User Privacy Harms and Risks in Conversational AI: A Proposed Framework

abstract: This study presents a unique framework that applies and extends Solove (2006)'s taxonomy to address privacy concerns in interactions with text-based AI chatbots. As chatbot prevalence grows, concerns about user privacy have heightened. While existing literature highlights design elements compromising privacy, a comprehensive framework is lacking. Through semi-structured interviews with 13 participants interacting with two AI chatbots, this study identifies 9 privacy harms and 9 privacy risks in text-based interactions. Using a grounded theory approach for interview and chatlog analysis, the framework examines privacy implications at various interaction stages. The aim is to offer developers, policymakers, and researchers a tool for responsible and secure implementation of conversational AI, filling the existing gap in addressing privacy issues associated with text-based AI chatbots.


title: Visually Dehallucinative Instruction Generation: Know What You Don't Know

abstract: "When did the emperor Napoleon invented iPhone?" Such hallucination-inducing question is well known challenge in generative language modeling. In this study, we present an innovative concept of visual hallucination, referred to as "I Know (IK)" hallucination, to address scenarios where "I Don't Know" is the desired response. To effectively tackle this issue, we propose the VQAv2-IDK benchmark, the subset of VQAv2 comprising unanswerable image-question pairs as determined by human annotators. Stepping further, we present the visually dehallucinative instruction generation method for IK hallucination and introduce the IDK-Instructions visual instruction database. Our experiments show that current methods struggle with IK hallucination. Yet, our approach effectively reduces these hallucinations, proving its versatility across different frameworks and datasets.


title: Strong Purcell Effect in All-Dielectric Epsilon-Near-Zero Microcavities

abstract: We theoretically study the quality and Purcell enhancement factors in epsilon-near-zero (ENZ) optical microcavities with ideally lossless or slightly lossy structures. In the case of ENZ rectangular cavities with perfect electric conductor (PEC) walls, our results supported by numerical simulations, show that both the quality and Purcell factors grow with increasing the length of cavity, having respective functional dependencies of $n^{-3}$ and $n^{-2}$ on the effective index of refraction, $n$. Further, we study extremely low-loss all-dielectric Bragg reflection waveguide cavities, near their cutoff frequencies. In such ENZ structures, the respective quality and Purcell factors scale as \((n\kappa)^{-1}\) and $n/\kappa$, in terms of real ($n$) and imaginary ($\kappa$) parts of the cavity effective index of refraction. We believe such structures exhibiting remarkably high quality ($\sim10^6$) and Purcell factors ($\sim10^3$), can drastically enhance various linear and nonlinear optical processes, and thus find use in applications ranging from laser development to optical sensing and metrology.


title: Effects of correlated magnetic noises from shared control lines on two-qubit gate

abstract: Recent proposals for building scalable quantum computational devices in semi-conductor based spin qubits introduce shared control lines in order to reduce the overhead of qubits controls. In principle, noises from the shared controls could introduce correlated errors to multi-qubit gates, and identifying them will be helpful for achieving higher gate fidelity in those setups. Here, we introduce a method based on the randomized benchmarking protocols that is capable of distinguishing among different correlated noises in a particular two-qubit model motivated by the crossbar architecture.


title: SpaceMeta: Global-Scale Massive Multi-User Virtual Interaction over LEO Satellite Constellations

abstract: Low latency and high synchronization among users are critical for emerging multi-user virtual interaction applications. However, the existing ground-based cloud solutions are naturally limited by the complex ground topology and fiber speeds, making it difficult to pace with the requirement of multi-user virtual interaction. The growth of low earth orbit (LEO) satellite constellations becomes a promising alternative to ground solutions. To fully exploit the potential of the LEO satellite, in this paper, we study the satellite server selection problem for global-scale multi-user interaction applications over LEO constellations. We propose an effective server selection framework, called SpaceMeta, that jointly selects the ingress satellite servers and relay servers on the communication path to minimize latency and latency discrepancy among users. Extensive experiments using real-world Starlink topology demonstrate that SpaceMeta reduces the latency by 6.72% and the interquartile range (IQR) of user latency by 39.50% compared with state-of-the-art methods.


title: Generalized Principal-Agent Problem with a Learning Agent

abstract: Generalized principal-agent problems, including Stackelberg games, contract design, and Bayesian persuasion, are a class of economic problems where an agent best responds to a principal's committed strategy. We study repeated generalized principal-agent problems under the assumption that the principal does not have commitment power and the agent uses algorithms to learn to respond to the principal. We reduce this problem to a one-shot generalized principal-agent problem with an approximately-best-responding agent. Using this reduction, we show that: (1) if the agent uses contextual no-regret learning algorithms, then the principal can guarantee a utility that is at least the principal's optimal utility in the classic non-learning model minus the square root of the agent's regret; (2) if the agent uses contextual no-swap-regret learning algorithms, then the principal cannot obtain any utility more than the optimal utility in the non-learning model plus the agent's swap regret. But (3) if the agent uses mean-based learning algorithms (which can be no-regret but not no-swap-regret), then the principal can do significantly better than the non-learning model. These general results not only refine previous results in Stackelberg games and contract design with learning agents but also lead to new results for Bayesian persuasion with a learning agent.


title: Reg-NF: Efficient Registration of Implicit Surfaces within Neural Fields

abstract: Neural fields, coordinate-based neural networks, have recently gained popularity for implicitly representing a scene. In contrast to classical methods that are based on explicit representations such as point clouds, neural fields provide a continuous scene representation able to represent 3D geometry and appearance in a way which is compact and ideal for robotics applications. However, limited prior methods have investigated registering multiple neural fields by directly utilising these continuous implicit representations. In this paper, we present Reg-NF, a neural fields-based registration that optimises for the relative 6-DoF transformation between two arbitrary neural fields, even if those two fields have different scale factors. Key components of Reg-NF include a bidirectional registration loss, multi-view surface sampling, and utilisation of volumetric signed distance functions (SDFs). We showcase our approach on a new neural field dataset for evaluating registration problems. We provide an exhaustive set of experiments and ablation studies to identify the performance of our approach, while also discussing limitations to provide future direction to the research community on open challenges in utilizing neural fields in unconstrained environments.


title: Efficient Prompt Optimization Through the Lens of Best Arm Identification

abstract: The remarkable instruction-following capability of large language models (LLMs) has sparked a growing interest in automatically finding good prompts, i.e., prompt optimization. Most existing works follow the scheme of selecting from a pre-generated pool of candidate prompts. However, these designs mainly focus on the generation strategy, while limited attention has been paid to the selection method. Especially, the cost incurred during the selection (e.g., accessing LLM and evaluating the responses) is rarely explicitly considered. To overcome this limitation, this work provides a principled framework, TRIPLE, to efficiently perform prompt selection under an explicit budget constraint. TRIPLE is built on a novel connection established between prompt optimization and fixed-budget best arm identification (BAI-FB) in multi-armed bandits (MAB); thus, it is capable of leveraging the rich toolbox from BAI-FB systematically and also incorporating unique characteristics of prompt optimization. Extensive experiments on multiple well-adopted tasks using various LLMs demonstrate the remarkable performance improvement of TRIPLE over baselines while satisfying the limited budget constraints. As an extension, variants of TRIPLE are proposed to efficiently select examples for few-shot prompts, also achieving superior empirical performance.


title: Region Feature Descriptor Adapted to High Affine Transformations

abstract: To address the issue of feature descriptors being ineffective in representing grayscale feature information when images undergo high affine transformations, leading to a rapid decline in feature matching accuracy, this paper proposes a region feature descriptor based on simulating affine transformations using classification. The proposed method initially categorizes images with different affine degrees to simulate affine transformations and generate a new set of images. Subsequently, it calculates neighborhood information for feature points on this new image set. Finally, the descriptor is generated by combining the grayscale histogram of the maximum stable extremal region to which the feature point belongs and the normalized position relative to the grayscale centroid of the feature point's region. Experimental results, comparing feature matching metrics under affine transformation scenarios, demonstrate that the proposed descriptor exhibits higher precision and robustness compared to existing classical descriptors. Additionally, it shows robustness when integrated with other descriptors.


title: Improving Non-autoregressive Machine Translation with Error Exposure and Consistency Regularization

abstract: Being one of the IR-NAT (Iterative-refinemennt-based NAT) frameworks, the Conditional Masked Language Model (CMLM) adopts the mask-predict paradigm to re-predict the masked low-confidence tokens. However, CMLM suffers from the data distribution discrepancy between training and inference, where the observed tokens are generated differently in the two cases. In this paper, we address this problem with the training approaches of error exposure and consistency regularization (EECR). We construct the mixed sequences based on model prediction during training, and propose to optimize over the masked tokens under imperfect observation conditions. We also design a consistency learning method to constrain the data distribution for the masked tokens under different observing situations to narrow down the gap between training and inference. The experiments on five translation benchmarks obtains an average improvement of 0.68 and 0.40 BLEU scores compared to the base models, respectively, and our CMLMC-EECR achieves the best performance with a comparable translation quality with the Transformer. The experiments results demonstrate the effectiveness of our method.


title: Detecting the effect of non-thermal sources on the warm-hot Galactic halo

abstract: We report the first detection of non-thermal broadening of OVII lines in the warm-hot $\approx 10^6$ K circumgalactic medium (CGM) of the Milky Way. We use $z$=0 absorption of OVII K$\alpha$, OVII K$\beta$, and OVIII K$\alpha$ lines in archival grating data of $b>$15$^\circ$ quasar sightlines from $Chandra$ and $XMM$-$Newton$. Non-thermal line broadening is evident in two-third of the sightlines considered, and on average is constrained at 4.6$\sigma$ significance. Non-thermal line broadening dominates over thermal broadening. We extensively test whether the appearance of non-thermal line broadening could instead be because of multiple thermally broadened velocity components and robustly rule it out. Non-thermal line broadening is more evident toward sightlines at lower galactic latitude indicating the Galactic disk origin of the nonthermal sources. There is weak/no correlation between non-thermal line broadening and the angular separation of sightlines from the Galactic center, indicating that the nuclear region might not be a major source of non-thermal factors.


title: A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts

abstract: Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we propose ReadAgent, an LLM agent system that increases effective context length up to 20x in our experiments. Inspired by how humans interactively read long documents, we implement ReadAgent as a simple prompting system that uses the advanced language capabilities of LLMs to (1) decide what content to store together in a memory episode, (2) compress those memory episodes into short episodic memories called gist memories, and (3) take actions to look up passages in the original text if ReadAgent needs to remind itself of relevant details to complete a task. We evaluate ReadAgent against baselines using retrieval methods, using the original long contexts, and using the gist memories. These evaluations are performed on three long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum. ReadAgent outperforms the baselines on all three tasks while extending the effective context window by 3-20x.


title: AbuseGPT: Abuse of Generative AI ChatBots to Create Smishing Campaigns

abstract: SMS phishing, also known as "smishing", is a growing threat that tricks users into disclosing private information or clicking into URLs with malicious content through fraudulent mobile text messages. In recent past, we have also observed a rapid advancement of conversational generative AI chatbot services (e.g., OpenAI's ChatGPT, Google's BARD), which are powered by pre-trained large language models (LLMs). These AI chatbots certainly have a lot of utilities but it is not systematically understood how they can play a role in creating threats and attacks. In this paper, we propose AbuseGPT method to show how the existing generative AI-based chatbot services can be exploited by attackers in real world to create smishing texts and eventually lead to craftier smishing campaigns. To the best of our knowledge, there is no pre-existing work that evidently shows the impacts of these generative text-based models on creating SMS phishing. Thus, we believe this study is the first of its kind to shed light on this emerging cybersecurity threat. We have found strong empirical evidences to show that attackers can exploit ethical standards in the existing generative AI-based chatbot services by crafting prompt injection attacks to create newer smishing campaigns. We also discuss some future research directions and guidelines to protect the abuse of generative AI-based services and safeguard users from smishing attacks.


title: Federated Prompt-based Decision Transformer for Customized VR Services in Mobile Edge Computing System

abstract: This paper investigates resource allocation to provide heterogeneous users with customized virtual reality (VR) services in a mobile edge computing (MEC) system. We first introduce a quality of experience (QoE) metric to measure user experience, which considers the MEC system's latency, user attention levels, and preferred resolutions. Then, a QoE maximization problem is formulated for resource allocation to ensure the highest possible user experience,which is cast as a reinforcement learning problem, aiming to learn a generalized policy applicable across diverse user environments for all MEC servers. To learn the generalized policy, we propose a framework that employs federated learning (FL) and prompt-based sequence modeling to pre-train a common decision model across MEC servers, which is named FedPromptDT. Using FL solves the problem of insufficient local MEC data while protecting user privacy during offline training. The design of prompts integrating user-environment cues and user-preferred allocation improves the model's adaptability to various user environments during online execution.


title: DOF: Accelerating High-order Differential Operators with Forward Propagation

abstract: Solving partial differential equations (PDEs) efficiently is essential for analyzing complex physical systems. Recent advancements in leveraging deep learning for solving PDE have shown significant promise. However, machine learning methods, such as Physics-Informed Neural Networks (PINN), face challenges in handling high-order derivatives of neural network-parameterized functions. Inspired by Forward Laplacian, a recent method of accelerating Laplacian computation, we propose an efficient computational framework, Differential Operator with Forward-propagation (DOF), for calculating general second-order differential operators without losing any precision. We provide rigorous proof of the advantages of our method over existing methods, demonstrating two times improvement in efficiency and reduced memory consumption on any architectures. Empirical results illustrate that our method surpasses traditional automatic differentiation (AutoDiff) techniques, achieving 2x improvement on the MLP structure and nearly 20x improvement on the MLP with Jacobian sparsity.


title: POBEVM: Real-time Video Matting via Progressively Optimize the Target Body and Edge

abstract: Deep convolutional neural networks (CNNs) based approaches have achieved great performance in video matting. Many of these methods can produce accurate alpha estimation for the target body but typically yield fuzzy or incorrect target edges. This is usually caused by the following reasons: 1) The current methods always treat the target body and edge indiscriminately; 2) Target body dominates the whole target with only a tiny proportion target edge. For the first problem, we propose a CNN-based module that separately optimizes the matting target body and edge (SOBE). And on this basis, we introduce a real-time, trimap-free video matting method via progressively optimizing the matting target body and edge (POBEVM) that is much lighter than previous approaches and achieves significant improvements in the predicted target edge. For the second problem, we propose an Edge-L1-Loss (ELL) function that enforces our network on the matting target edge. Experiments demonstrate our method outperforms prior trimap-free matting methods on both Distinctions-646 (D646) and VideoMatte240K(VM) dataset, especially in edge optimization.


title: Study of size and shell composition effect of TiO2 core-shell mesoporous microsphere on UV absorption effectivity for photocatalytic application

abstract: Microdispersed photocatalysts based on titanium dioxide (TiO2) in the form of hollow core-shell microspheres (microcapsules) with mesoporous structure are widely demanded in modern critical technologies related to the catalysis of various chemicals, solving environmental problems, and obtaining cheap fuel. To date, a number of experimental works are known, showing that geometrical parameters of microcapsules (size, shell thickness), as well as microstructural composition (nanosized metal additives, additional inner dielectric core -- the "yolk") noticeably affect their photocatalytic activity. At the same time, a valuable physical description of the optical properties of porous microcapsules has not been presented in the literature so far. Using the finite element method, we perform a full-wave simulation of the optical field inside a hollow microsphere whose shell is assembled from multiple TiO2 nanoparticles to form an irregular nanoporous structure. We give a physical explanation of the published experimental data on the optical activity of titanium-dioxide microcapsules and show that the existing theoretical models do not always give a correct interpretation of the observed empirical behaviors.


title: Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States

abstract: Large Language Models (LLMs) can make up answers that are not real, and this is known as hallucination. This research aims to see if, how, and to what extent LLMs are aware of hallucination. More specifically, we check whether and how an LLM reacts differently in its hidden states when it answers a question right versus when it hallucinates. To do this, we introduce an experimental framework which allows examining LLM's hidden states in different hallucination situations. Building upon this framework, we conduct a series of experiments with language models in the LLaMA family (Touvron et al., 2023). Our empirical findings suggest that LLMs react differently when processing a genuine response versus a fabricated one. We then apply various model interpretation techniques to help understand and explain the findings better. Moreover, informed by the empirical observations, we show great potential of using the guidance derived from LLM's hidden representation space to mitigate hallucination. We believe this work provides insights into how LLMs produce hallucinated answers and how to make them occur less often.


title: Agents Need Not Know Their Purpose

abstract: Ensuring artificial intelligence behaves in such a way that is aligned with human values is commonly referred to as the alignment challenge. Prior work has shown that rational agents, behaving in such a way that maximizes a utility function, will inevitably behave in such a way that is not aligned with human values, especially as their level of intelligence goes up. Prior work has also shown that there is no "one true utility function"; solutions must include a more holistic approach to alignment. This paper describes oblivious agents: agents that are architected in such a way that their effective utility function is an aggregation of a known and hidden sub-functions. The hidden component, to be maximized, is internally implemented as a black box, preventing the agent from examining it. The known component, to be minimized, is knowledge of the hidden sub-function. Architectural constraints further influence how agent actions can evolve its internal environment model. We show that an oblivious agent, behaving rationally, constructs an internal approximation of designers' intentions (i.e., infers alignment), and, as a consequence of its architecture and effective utility function, behaves in such a way that maximizes alignment; i.e., maximizing the approximated intention function. We show that, paradoxically, it does this for whatever utility function is used as the hidden component and, in contrast with extant techniques, chances of alignment actually improve as agent intelligence grows.


title: DFORM: Diffeomorphic vector field alignment for assessing dynamics across learned models

abstract: Dynamical system models such as Recurrent Neural Networks (RNNs) have become increasingly popular as hypothesis-generating tools in scientific research. Evaluating the dynamics in such networks is key to understanding their learned generative mechanisms. However, comparison of learned dynamics across models is challenging due to their inherent nonlinearity and because a priori there is no enforced equivalence of their coordinate systems. Here, we propose the DFORM (Diffeomorphic vector field alignment for comparing dynamics across learned models) framework. DFORM learns a nonlinear coordinate transformation which provides a continuous, maximally one-to-one mapping between the trajectories of learned models, thus approximating a diffeomorphism between them. The mismatch between DFORM-transformed vector fields defines the orbital similarity between two models, thus providing a generalization of the concepts of smooth orbital and topological equivalence. As an example, we apply DFORM to models trained on a canonical neuroscience task, showing that learned dynamics may be functionally similar, despite overt differences in attractor landscapes.


title: Federated Analytics-Empowered Frequent Pattern Mining for Decentralized Web 3.0 Applications

abstract: The emerging Web 3.0 paradigm aims to decentralize existing web services, enabling desirable properties such as transparency, incentives, and privacy preservation. However, current Web 3.0 applications supported by blockchain infrastructure still cannot support complex data analytics tasks in a scalable and privacy-preserving way. This paper introduces the emerging federated analytics (FA) paradigm into the realm of Web 3.0 services, enabling data to stay local while still contributing to complex web analytics tasks in a privacy-preserving way. We propose FedWeb, a tailored FA design for important frequent pattern mining tasks in Web 3.0. FedWeb remarkably reduces the number of required participating data owners to support privacy-preserving Web 3.0 data analytics based on a novel distributed differential privacy technique. The correctness of mining results is guaranteed by a theoretically rigid candidate filtering scheme based on Hoeffding's inequality and Chebychev's inequality. Two response budget saving solutions are proposed to further reduce participating data owners. Experiments on three representative Web 3.0 scenarios show that FedWeb can improve data utility by ~25.3% and reduce the participating data owners by ~98.4%.


title: Dynamics and precise control of fluid V-states using an electron plasma

abstract: An electron plasma can be confined for a theoretically infinite time in a Penning-Malmberg trap, a linear, azimuthally-symmetric magneto-electrostatic device where upon suitable conditions (high magnetization) the transverse dynamics of the plasma column is isomorphic to the one displayed by a two-dimensional ideal fluid. Fluid dynamics can thus be reproduced in these systems with a very high degree of control on the system's parameters and active excitation of fluid perturbations is made possible by the use of static or time-dependent electric fields (i.e., fluid strains) imparted by electric potentials applied to the azimuthal patches of a sectored electrode of the trap. An example is represented by azimuthal velocity shear phenomena and the insurgence of Kelvin-Helmholtz (KH) instabilities in fluid vortices. We present a study where we exploit multipolar rotating electric fields to generate V-states and observe their dynamics and stability properties. A V-state is the generalization of the 2D Kirchhoff (elliptical) fluid vortex to a generic KH mode, in the nonlinear regime. In particular, we discuss first how we can exploit a combination of techniques (plasma evaporation and tilt-induced transport) to tune the radial vorticity profile, which may have an effect on the dynamics of the growth and decay of the selected KH wave. We also investigate autoresonant (swept-frequency, self-locking) excitation - useful, e.g., for the precise control of the KH mode growth - and discuss the features of autoresonance applied to higher-order KH waves.


title: Align before Attend: Aligning Visual and Textual Features for Multimodal Hateful Content Detection

abstract: Multimodal hateful content detection is a challenging task that requires complex reasoning across visual and textual modalities. Therefore, creating a meaningful multimodal representation that effectively captures the interplay between visual and textual features through intermediate fusion is critical. Conventional fusion techniques are unable to attend to the modality-specific features effectively. Moreover, most studies exclusively concentrated on English and overlooked other low-resource languages. This paper proposes a context-aware attention framework for multimodal hateful content detection and assesses it for both English and non-English languages. The proposed approach incorporates an attention layer to meaningfully align the visual and textual features. This alignment enables selective focus on modality-specific features before fusing them. We evaluate the proposed approach on two benchmark hateful meme datasets, viz. MUTE (Bengali code-mixed) and MultiOFF (English). Evaluation results demonstrate our proposed approach's effectiveness with F1-scores of $69.7$% and $70.3$% for the MUTE and MultiOFF datasets. The scores show approximately $2.5$% and $3.2$% performance improvement over the state-of-the-art systems on these datasets. Our implementation is available at https://github.com/eftekhar-hossain/Bengali-Hateful-Memes.


title: QuRating: Selecting High-Quality Data for Training Language Models

abstract: Selecting high-quality pre-training data is important for creating capable language models, but existing methods rely on simple heuristics. We introduce QuRating, a method for selecting pre-training data that can capture human intuitions about data quality. In this paper, we investigate four qualities - writing style, required expertise, facts & trivia, and educational value - and find that LLMs are able to discern these qualities, especially when making pairwise judgments of texts. We train a QuRater model to learn scalar ratings from pairwise judgments, and use it to annotate a 260B training corpus with quality ratings for each of the four criteria. In our experiments, we select 30B tokens according to the different quality ratings and train 1.3B-parameter language models on the selected data. We find that it is important to balance quality and diversity. When we sample using quality ratings as logits over documents, our models obtain lower perplexity and stronger in-context learning performance than baselines. Our best model is based on educational value and performs similarly to a model trained with uniform sampling for 50% more steps. Beyond data selection, we use the quality ratings to construct a training curriculum which improves performance without changing the training dataset. We extensively analyze the quality ratings and discuss their characteristics, biases, and wider implications.


title: Inversion of limited-aperture Fresnel experimental data using orthogonality sampling method with single and multiple sources

abstract: In this study, we consider the application of orthogonality sampling method (OSM) with single and multiple sources for a fast identification of small objects in limited-aperture inverse scattering problem. We first apply the OSM with single source and show that the indicator function with single source can be expressed by the Bessel function of order zero of the first kind, infinite series of Bessel function of nonzero integer order of the first kind, range of signal receiver, and the location of emitter. Based on this result, we explain that the objects can be identified through the OSM with single source but the identification is significantly influenced by the location of source and applied frequency. For a successful improvement, we then consider the OSM with multiple sources. Based on the identified structure of the OSM with single source, we design an indicator function of the OSM with multiple sources and show that it can be expressed by the square of the Bessel function of order zero of the first kind an infinite series of the square of Bessel function of nonzero integer order of the first kind. Based on the theoretical results, we explain that the objects can be identified uniquely through the designed OSM. Several numerical experiments with experimental data provided by the Institute Fresnel demonstrate the pros and cons of the OSM with single source and how the designed OSM with multiple sources behave.


title: Dimension-Dependent Critical Scaling Analysis and Emergent Competing Interaction Scales in a 2D Van der Waals magnet Cr$_{2}$Ge$_{2}$Te$_{6}$

abstract: We investigate thickness-dependent transformation from a paramagnetic to ferromagnetic phase in Cr$_{2}$Ge$_{2}$Te$_{6}$ (CGT) in bulk and few-layer flake forms. 2D Ising-like critical transition in bulk CGT occurs at $T_{c}$ = 67 K with out-of-plane magnetic anisotropy. Few-layer CGT on hBN/SiO$_{2}$/Si substrate displays the same $T_{c}$ but also exhibits a new critical transition at $T^{\prime}_c$ = 14.2 K. Here, critical scaling analysis reveals the critical exponents differ significantly from those in bulk and do not align with the known universality classes. Our Density Functional Theory (DFT) and classical calculations indicate competition between magnetocrystalline and dipolar anisotropy emerges with reduced dimensions. The observed behavior is due to minor structural distortions in low dimensional CGT, which modify the balance between spin-orbit coupling, exchange interactions and dipolar anisotropy. This triggers a critical crossover at $T^{\prime}_c$. Our study shows the emergence of a complex interplay of short- and long-range interactions below $T^{\prime}_c$ as CGT approaches the 2D limit.


title: AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator

abstract: Artificial intelligence has significantly advanced healthcare, particularly through large language models (LLMs) that excel in medical question answering benchmarks. However, their real-world clinical application remains limited due to the complexities of doctor-patient interactions. To address this, we introduce \textbf{AI Hospital}, a multi-agent framework simulating dynamic medical interactions between \emph{Doctor} as player and NPCs including \emph{Patient}, \emph{Examiner}, \emph{Chief Physician}. This setup allows for realistic assessments of LLMs in clinical scenarios. We develop the Multi-View Medical Evaluation (MVME) benchmark, utilizing high-quality Chinese medical records and NPCs to evaluate LLMs' performance in symptom collection, examination recommendations, and diagnoses. Additionally, a dispute resolution collaborative mechanism is proposed to enhance diagnostic accuracy through iterative discussions. Despite improvements, current LLMs exhibit significant performance gaps in multi-turn interactions compared to one-step approaches. Our findings highlight the need for further research to bridge these gaps and improve LLMs' clinical diagnostic capabilities. Our data, code, and experimental results are all open-sourced at \url{https://github.com/LibertFan/AI_Hospital}.


title: Quickest Detection of False Data Injection Attack in Distributed Process Tracking

abstract: This paper addresses the problem of detecting false data injection (FDI) attacks in a distributed network without a fusion center, represented by a connected graph among multiple agent nodes. Each agent node is equipped with a sensor, and uses a Kalman consensus information filter (KCIF) to track a discrete time global process with linear dynamics and additive Gaussian noise. The state estimate of the global process at any sensor is computed from the local observation history and the information received by that agent node from its neighbors. At an unknown time, an attacker starts altering the local observation of one agent node. In the Bayesian setting where there is a known prior distribution of the attack beginning instant, we formulate a Bayesian quickest change detection (QCD) problem for FDI detection in order to minimize the mean detection delay subject to a false alarm probability constraint. While it is well-known that the optimal Bayesian QCD rule involves checking the Shriyaev's statistic against a threshold, we demonstrate how to compute the Shriyaev's statistic at each node in a recursive fashion given our non-i.i.d. observations. Next, we consider non-Bayesian QCD where the attack begins at an arbitrary and unknown time, and the detector seeks to minimize the worst case detection delay subject to a constraint on the mean time to false alarm and probability of misidentification. We use the multiple hypothesis sequential probability ratio test for attack detection and identification at each sensor. For unknown attack strategy, we use the window-limited generalized likelihood ratio (WL-GLR) algorithm to solve the QCD problem. Numerical results demonstrate the performances and trade-offs of the proposed algorithms.


title: Quantile Granger Causality in the Presence of Instability

abstract: We propose a new framework for assessing Granger causality in quantiles in unstable environments, for a fixed quantile or over a continuum of quantile levels. Our proposed test statistics are consistent against fixed alternatives, they have nontrivial power against local alternatives, and they are pivotal in certain important special cases. In addition, we show the validity of a bootstrap procedure when asymptotic distributions depend on nuisance parameters. Monte Carlo simulations reveal that the proposed test statistics have correct empirical size and high power, even in absence of structural breaks. Finally, two empirical applications in energy economics and macroeconomics highlight the applicability of our method as the new tests provide stronger evidence of Granger causality.


title: WEFix: Intelligent Automatic Generation of Explicit Waits for Efficient Web End-to-End Flaky Tests

abstract: Web end-to-end (e2e) testing evaluates the workflow of a web application. It simulates real-world user scenarios to ensure the application flows behave as expected. However, web e2e tests are notorious for being flaky, i.e., the tests can produce inconsistent results despite no changes to the code. One common type of flakiness is caused by nondeterministic execution orders between the test code and the client-side code under test. In particular, UI-based flakiness emerges as a notably prevalent and challenging issue to fix because the test code has limited knowledge about the client-side code execution. In this paper, we propose WEFix, a technique that can automatically generate fix code for UI-based flakiness in web e2e testing. The core of our approach is to leverage browser UI changes to predict the client-side code execution and generate proper wait oracles. We evaluate the effectiveness and efficiency of WEFix against 122 web e2e flaky tests from seven popular real-world projects. Our results show that WEFix dramatically reduces the overhead (from 3.7$\times$ to 1.25$\times$) while achieving a high correctness (98%).


title: Alpha-GPT 2.0: Human-in-the-Loop AI for Quantitative Investment

abstract: Recently, we introduced a new paradigm for alpha mining in the realm of quantitative investment, developing a new interactive alpha mining system framework, Alpha-GPT. This system is centered on iterative Human-AI interaction based on large language models, introducing a Human-in-the-Loop approach to alpha discovery. In this paper, we present the next-generation Alpha-GPT 2.0 \footnote{Draft. Work in progress}, a quantitative investment framework that further encompasses crucial modeling and analysis phases in quantitative investment. This framework emphasizes the iterative, interactive research between humans and AI, embodying a Human-in-the-Loop strategy throughout the entire quantitative investment pipeline. By assimilating the insights of human researchers into the systematic alpha research process, we effectively leverage the Human-in-the-Loop approach, enhancing the efficiency and precision of quantitative investment research.


title: Less is more: Ensemble Learning for Retinal Disease Recognition Under Limited Resources

abstract: Retinal optical coherence tomography (OCT) images provide crucial insights into the health of the posterior ocular segment. Therefore, the advancement of automated image analysis methods is imperative to equip clinicians and researchers with quantitative data, thereby facilitating informed decision-making. The application of deep learning (DL)-based approaches has gained extensive traction for executing these analysis tasks, demonstrating remarkable performance compared to labor-intensive manual analyses. However, the acquisition of Retinal OCT images often presents challenges stemming from privacy concerns and the resource-intensive labeling procedures, which contradicts the prevailing notion that DL models necessitate substantial data volumes for achieving superior performance. Moreover, limitations in available computational resources constrain the progress of high-performance medical artificial intelligence, particularly in less developed regions and countries. This paper introduces a novel ensemble learning mechanism designed for recognizing retinal diseases under limited resources (e.g., data, computation). The mechanism leverages insights from multiple pre-trained models, facilitating the transfer and adaptation of their knowledge to Retinal OCT images. This approach establishes a robust model even when confronted with limited labeled data, eliminating the need for an extensive array of parameters, as required in learning from scratch. Comprehensive experimentation on real-world datasets demonstrates that the proposed approach can achieve superior performance in recognizing Retinal OCT images, even when dealing with exceedingly restricted labeled datasets. Furthermore, this method obviates the necessity of learning extensive-scale parameters, making it well-suited for deployment in low-resource scenarios.


title: Model Compression and Efficient Inference for Large Language Models: A Survey

abstract: Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it challenging to deploy large models on resource-constrained devices. In this paper, we investigate compression and efficient inference methods for large language models from an algorithmic perspective. Regarding taxonomy, similar to smaller models, compression and acceleration algorithms for large language models can still be categorized into quantization, pruning, distillation, compact architecture design, dynamic networks. However, Large language models have two prominent characteristics compared to smaller models: (1) Most of compression algorithms require finetuning or even retraining the model after compression. The most notable aspect of large models is the very high cost associated with model finetuning or training. Therefore, many algorithms for large models, such as quantization and pruning, start to explore tuning-free algorithms. (2) Large models emphasize versatility and generalization rather than performance on a single task. Hence, many algorithms, such as knowledge distillation, focus on how to preserving their versatility and generalization after compression. Since these two characteristics were not very pronounced in early large models, we further distinguish large language models into medium models and ``real'' large models. Additionally, we also provide an introduction to some mature frameworks for efficient inference of large models, which can support basic compression or acceleration algorithms, greatly facilitating model deployment for users.


title: The $\mathcal{PT}$-symmetric quantum Rabi model: Solutions and exceptional points

abstract: The non-Hermitian one-photon and two-photon quantum Rabi models (QRMs) with imaginary couplings are respectively solved through the Bogoliubov operators approach. Transcendental functions responsible for exact solutions are derived, whose zeros produce the complete spectra. Exceptional points (EPs) can be identified in terms of the transcendental function. The EP is formed in the two nearest-neighboring excited energy levels, and shifts towards lower coupling strength at higher energy levels. Interestingly, under the resonant condition in the non-Hermitian two-photon QRM, the lowest two excited states within the same parity in the even photonic number subspace coalesce at an EP, but take always the purely real energy, in sharp contrast to the conventional EP in the non-Hermitian systems. For both non-Hermitian QRMs, the fidelity susceptibility goes to negative infinity at the EPs, consistent with the recent observations in non-Hermitian systems. All eigenstates can be labeled by the conserved energy and the QRM parity, we argue that the non-Hermitian QRMs are also integrable, similar to their Hermitian counterparts.


title: Exploring the Potential of Large Language Models in Artistic Creation: Collaboration and Reflection on Creative Programming

abstract: Recently, the potential of large language models (LLMs) has been widely used in assisting programming. However, current research does not explore the artist potential of LLMs in creative coding within artist and AI collaboration. Our work probes the reflection type of artists in the creation process with such collaboration. We compare two common collaboration approaches: invoking the entire program and multiple subtasks. Our findings exhibit artists' different stimulated reflections in two different methods. Our finding also shows the correlation of reflection type with user performance, user satisfaction, and subjective experience in two collaborations through conducting two methods, including experimental data and qualitative interviews. In this sense, our work reveals the artistic potential of LLM in creative coding. Meanwhile, we provide a critical lens of human-AI collaboration from the artists' perspective and expound design suggestions for future work of AI-assisted creative tasks.


title: Long-time behavior towards viscous-dispersive shock for Navier-Stokes equations of Korteweg type

abstract: We consider the so-called Naiver-Stokes-Korteweg(NSK) equations for the dynamics of compressible barotropic viscous fluids with internal capillarity. We handle the time-asymptotic stability in 1D of the viscous-dispersive shock wave that is a traveling wave solution to NSK as a viscous-dispersive counterpart of a Riemann shock. More precisely, we prove that when the prescribed far-field states of NSK are connected by a single Hugoniot curve, then solutions of NSK tend to the viscous-dispersive shock wave as time goes to infinity. To obtain the convergence, we extend the theory of $a$-contraction with shifts, used for the Navier-Stokes equations, to the NSK system. The main difficulty in analysis for NSK is due to the third-order derivative terms of the specific volume in the momentum equation. To resolve the problem, we introduce an auxiliary variable that is equivalent to the derivative of the specific volume.


title: Vector spectrometer with Hertz-level resolution and super-recognition capability

abstract: High-resolution optical spectrometers are crucial in revealing intricate characteristics of signals, determining laser frequencies, measuring physical constants, identifying substances, and advancing biosensing applications. Conventional spectrometers, however, often grapple with inherent trade-offs among spectral resolution, wavelength range, and accuracy. Furthermore, even at high resolution, resolving overlapping spectral lines during spectroscopic analyses remains a huge challenge. Here, we propose a vector spectrometer with ultrahigh resolution, combining broadband optical frequency hopping, ultrafine microwave-photonic scanning, and vector detection. A programmable frequency-hopping laser was developed, facilitating a sub-Hz linewidth and Hz-level frequency stability, an improvement of four and six orders of magnitude, respectively, compared to those of state-of-the-art tunable lasers. We also designed an asymmetric optical transmitter and receiver to eliminate measurement errors arising from modulation nonlinearity and multi-channel crosstalk. The resultant vector spectrometer exhibits an unprecedented frequency resolution of 2 Hz, surpassing the state-of-the-art by four orders of magnitude, over a 33-nm range. Through high-resolution vector analysis, we observed that group delay information enhances the separation capability of overlapping spectral lines by over 47%, significantly streamlining the real-time identification of diverse substances. Our technique fills the gap in optical spectrometers with resolutions below 10 kHz and enables vector measurement to embrace revolution in functionality.

categories: physics.optics cs.SY eess.SY physics.app-ph quant-ph


title: A note on the universal supersingular quotients of $U(2,1)$

abstract: Let $G$ be the unramified unitary group $U(2, 1)(E/F)$ defined over a non-archimedean local field $F$ of residue characteristic $p\neq 2$. In this note, we prove the universal supersingular quotients of $G$ are not irreducible in general.


title: Robust SVD Made Easy: A fast and reliable algorithm for large-scale data analysis

abstract: The singular value decomposition (SVD) is a crucial tool in machine learning and statistical data analysis. However, it is highly susceptible to outliers in the data matrix. Existing robust SVD algorithms often sacrifice speed for robustness or fail in the presence of only a few outliers. This study introduces an efficient algorithm, called Spherically Normalized SVD, for robust SVD approximation that is highly insensitive to outliers, computationally scalable, and provides accurate approximations of singular vectors. The proposed algorithm achieves remarkable speed by utilizing only two applications of a standard reduced-rank SVD algorithm to appropriately scaled data, significantly outperforming competing algorithms in computation times. To assess the robustness of the approximated singular vectors and their subspaces against data contamination, we introduce new notions of breakdown points for matrix-valued input, including row-wise, column-wise, and block-wise breakdown points. Theoretical and empirical analyses demonstrate that our algorithm exhibits higher breakdown points compared to standard SVD and its modifications. We empirically validate the effectiveness of our approach in applications such as robust low-rank approximation and robust principal component analysis of high-dimensional microarray datasets. Overall, our study presents a highly efficient and robust solution for SVD approximation that overcomes the limitations of existing algorithms in the presence of outliers.


title: Data Smoothing Filling Method based on ScRNA-Seq Data Zero-Value Identification

abstract: Single-cell RNA sequencing (scRNA-seq) determines RNA expression at single-cell resolution. It provides a powerful tool for studying immunity, regulation, and other life activities of cells. However, due to the limitations of the sequencing technique, the scRNA-seq data are represented with sparsity, whichcontains missing gene values, i.e., zero values, called dropout. Therefore, it is necessary to impute missing values before analyzing scRNA-seq data. However, existing imputation computation methods often only focus on the identification of technical zeros or imputing all zeros based on cell similarity. This study proposes a new method (SFAG) to reconstruct the gene expression relationship matrix by usinggraph regularization technology to preserve the high-dimensional manifold information of the data, andto mine the relationship between genes and cells in the data, and then uses a method of averaging the clustering results to fill in the identified technical zeros. Experimental results show that SFAGcan helpimprove downstream analysis and reconstruct cell trajectory


title: Mixture of Experts for Network Optimization: A Large Language Model-enabled Approach

abstract: Optimizing various wireless user tasks poses a significant challenge for networking systems because of the expanding range of user requirements. Despite advancements in Deep Reinforcement Learning (DRL), the need for customized optimization tasks for individual users complicates developing and applying numerous DRL models, leading to substantial computation resource and energy consumption and can lead to inconsistent outcomes. To address this issue, we propose a novel approach utilizing a Mixture of Experts (MoE) framework, augmented with Large Language Models (LLMs), to analyze user objectives and constraints effectively, select specialized DRL experts, and weigh each decision from the participating experts. Specifically, we develop a gate network to oversee the expert models, allowing a collective of experts to tackle a wide array of new tasks. Furthermore, we innovatively substitute the traditional gate network with an LLM, leveraging its advanced reasoning capabilities to manage expert model selection for joint decisions. Our proposed method reduces the need to train new DRL models for each unique optimization problem, decreasing energy consumption and AI model implementation costs. The LLM-enabled MoE approach is validated through a general maze navigation task and a specific network service provider utility maximization task, demonstrating its effectiveness and practical applicability in optimizing complex networking systems.


title: Construction of CCC and ZCCS Through Additive Characters Over Galois Field

abstract: The rapid progression in wireless communication technologies, especially in multicarrier code-division multiple access (MC-CDMA), there is a need of advanced code construction methods. Traditional approaches, mainly based on generalized Boolean functions, have limitations in code length versatility. This paper introduces a novel approach to constructing complete complementary codes (CCC) and Z-complementary code sets (ZCCS), for reducing interference in MC-CDMA systems. The proposed construction, distinct from Boolean function-based approaches, employs additive characters over Galois fields GF($p^{r}$), where $p$ is prime and $r$ is a positive integer. First, we develop CCCs with lengths of $p^{r}$, which are then extended to construct ZCCS with both unreported lengths and sizes of $np^{r}$, where $n$ are arbitrary positive integers. The versatility of this method is further highlighted as it includes the lengths of ZCCS reported in prior studies as special cases, underscoring the method's comprehensive nature and superiority.


title: Extrapolation-Aware Nonparametric Statistical Inference

abstract: We define extrapolation as any type of statistical inference on a conditional function (e.g., a conditional expectation or conditional quantile) evaluated outside of the support of the conditioning variable. This type of extrapolation occurs in many data analysis applications and can invalidate the resulting conclusions if not taken into account. While extrapolating is straightforward in parametric models, it becomes challenging in nonparametric models. In this work, we extend the nonparametric statistical model to explicitly allow for extrapolation and introduce a class of extrapolation assumptions that can be combined with existing inference techniques to draw extrapolation-aware conclusions. The proposed class of extrapolation assumptions stipulate that the conditional function attains its minimal and maximal directional derivative, in each direction, within the observed support. We illustrate how the framework applies to several statistical applications including prediction and uncertainty quantification. We furthermore propose a consistent estimation procedure that can be used to adjust existing nonparametric estimates to account for extrapolation by providing lower and upper extrapolation bounds. The procedure is empirically evaluated on both simulated and real-world data.


title: Efficient Language Adaptive Pre-training: Extending State-of-the-Art Large Language Models for Polish

abstract: This study explores the potential of fine-tuning foundational English Large Language Models (LLMs) for generating Polish text. The first step involves Language Adaptive Pre-training (LAPT) on a high-quality dataset of 3.11 GB, consisting of 276 million Polish tokens. The LAPT is followed by additional fine-tuning aimed at solving nine KLEJ challenges. Our trained model Curie-7B-v1 not only generates Polish text with the lowest perplexity of 3.02 among decoder-based Polish models but also closely rivals the performance of the best Polish encoder-decoder models with a less than 2% gap on 8 out of 9 tasks. Curie-7B-v1 used approximately 2-3% of a typical dataset size to learn Polish. The LAPT was completed in less than five days using a consumer GPU, highlighting the method's efficiency. The proficiency of the model in Polish was significantly enhanced, demonstrating the viability of this approach for adding new languages to existing LLMs by training just 1.2% of its parameters. To contribute to the community's collaborative progress, the model has been released as open-source.


title: Grounding Language Model with Chunking-Free In-Context Retrieval

abstract: This paper presents a novel Chunking-Free In-Context (CFIC) retrieval approach, specifically tailored for Retrieval-Augmented Generation (RAG) systems. Traditional RAG systems often struggle with grounding responses using precise evidence text due to the challenges of processing lengthy documents and filtering out irrelevant content. Commonly employed solutions, such as document chunking and adapting language models to handle longer contexts, have their limitations. These methods either disrupt the semantic coherence of the text or fail to effectively address the issues of noise and inaccuracy in evidence retrieval. CFIC addresses these challenges by circumventing the conventional chunking process. It utilizes the encoded hidden states of documents for in-context retrieval, employing auto-aggressive decoding to accurately identify the specific evidence text required for user queries, eliminating the need for chunking. CFIC is further enhanced by incorporating two decoding strategies, namely Constrained Sentence Prefix Decoding and Skip Decoding. These strategies not only improve the efficiency of the retrieval process but also ensure that the fidelity of the generated grounding text evidence is maintained. Our evaluations of CFIC on a range of open QA datasets demonstrate its superiority in retrieving relevant and accurate evidence, offering a significant improvement over traditional methods. By doing away with the need for document chunking, CFIC presents a more streamlined, effective, and efficient retrieval solution, making it a valuable advancement in the field of RAG systems.


title: A Framework For Gait-Based User Demography Estimation Using Inertial Sensors

abstract: Human gait has been shown to provide crucial motion cues for various applications. Recognizing patterns in human gait has been widely adopted in various application areas such as security, virtual reality gaming, medical rehabilitation, and ailment identification. Furthermore, wearable inertial sensors have been widely used for not only recording gait but also to predict users' demography. Machine Learning techniques such as deep learning, combined with inertial sensor signals, have shown promising results in recognizing patterns in human gait and estimate users' demography. However, the black-box nature of such deep learning models hinders the researchers from uncovering the reasons behind the model's predictions. Therefore, we propose leveraging deep learning and Layer-Wise Relevance Propagation (LRP) to identify the important variables that play a vital role in identifying the users' demography such as age and gender. To assess the efficacy of this approach we train a deep neural network model on a large sensor-based gait dataset consisting of 745 subjects to identify users' age and gender. Using LRP we identify the variables relevant for characterizing the gait patterns. Thus, we enable interpretation of non-linear ML models which are experts in identifying the users' demography based on inertial signals. We believe this approach can not only provide clinicians information about the gait parameters relevant to age and gender but also can be expanded to analyze and diagnose gait disorders.


title: Peaceful Colourings

abstract: We introduce peaceful colourings, a variant of $h$-conflict free colourings. We call a colouring with no monochromatic edges $p$-peaceful if for each vertex $v$, there are at most $p$ neighbours of $v$ coloured with a colour appearing on another neighbour of $v$. An $h$-conflict-free colouring of a graph is a (vertex)-colouring with no monochromatic edges so that for every vertex $v$, the number of neighbours of $v$ which are coloured with a colour appearing on no other neighbour of $v$ is at least the minimum of $h$ and the degree of $v$. If $G$ is $\Delta$-regular then it has an $h$-conflict free colouring precisely if it has a $(\Delta-h)$-peaceful colouring. We focus on the minimum $p_\Delta$ of those $p$ for which every graph of maximum degree $\Delta$ has a $p$-peaceful colouring with $\Delta+1$ colours. We show that $p_\Delta > (1-\frac{1}{e}-o(1))\Delta$ and that for graphs of bounded codegree, $p_\Delta \leq (1-\frac{1}{e}+o(1))\Delta$. We ask if the latter result can be improved by dropping the bound on the codegree. As a partial result, we show that $p_\Delta \leq \frac{8000}{8001}\Delta$ for sufficiently large $\Delta$.


title: Quantifying Systematic Uncertainties in Experimental Physics: An Approximation Method

abstract: In the domain of physics experiments, data fitting is a pivotal technique for extracting insights from both experimental and simulated datasets. This article presents an approximation method designed to estimate the systematic errors prevalent in data analyses. By applying our method to the Nab experiment, we compare our findings with simulation-derived results, thereby confirming the concordance of our approach with established simulation outcomes. This corroboration highlights the versatility of our method as a good tool for validating simulation results across various experimental contexts.


title: Aligning Crowd Feedback via Distributional Preference Reward Modeling

abstract: Deep Reinforcement Learning is widely used for aligning Large Language Models (LLM) with human preference. However, the conventional reward modelling is predominantly dependent on human annotations provided by a select cohort of individuals. Such dependence may unintentionally result in skewed models that reflect the inclinations of these annotators, thereby failing to adequately represent the wider population's expectations. We propose the Distributional Preference Reward Model (DPRM), a simple yet effective framework to align large language models with diverse human preferences. To this end, we characterize multiple preferences by a categorical distribution and introduce a Bayesian updater to accommodate shifted or new preferences. On top of that, we design an optimal-transportation-based loss to calibrate DPRM to align with the preference distribution. Finally, the expected reward is utilized to fine-tune an LLM policy to generate responses favoured by the population. Our experiments show that DPRM significantly enhances the alignment of LLMs with population preference, yielding more accurate, unbiased, and contextually appropriate responses.


title: Reinforcement Learning for Solving Stochastic Vehicle Routing Problem with Time Windows

abstract: This paper introduces a reinforcement learning approach to optimize the Stochastic Vehicle Routing Problem with Time Windows (SVRP), focusing on reducing travel costs in goods delivery. We develop a novel SVRP formulation that accounts for uncertain travel costs and demands, alongside specific customer time windows. An attention-based neural network trained through reinforcement learning is employed to minimize routing costs. Our approach addresses a gap in SVRP research, which traditionally relies on heuristic methods, by leveraging machine learning. The model outperforms the Ant-Colony Optimization algorithm, achieving a 1.73% reduction in travel costs. It uniquely integrates external information, demonstrating robustness in diverse environments, making it a valuable benchmark for future SVRP studies and industry application.


title: From Variability to Stability: Advancing RecSys Benchmarking Practices

abstract: In the rapidly evolving domain of Recommender Systems (RecSys), new algorithms frequently claim state-of-the-art performance based on evaluations over a limited set of arbitrarily selected datasets. However, this approach may fail to holistically reflect their effectiveness due to the significant impact of dataset characteristics on algorithm performance. Addressing this deficiency, this paper introduces a novel benchmarking methodology to facilitate a fair and robust comparison of RecSys algorithms, thereby advancing evaluation practices. By utilizing a diverse set of $30$ open datasets, including two introduced in this work, and evaluating $11$ collaborative filtering algorithms across $9$ metrics, we critically examine the influence of dataset characteristics on algorithm performance. We further investigate the feasibility of aggregating outcomes from multiple datasets into a unified ranking. Through rigorous experimental analysis, we validate the reliability of our methodology under the variability of datasets, offering a benchmarking strategy that balances quality and computational demands. This methodology enables a fair yet effective means of evaluating RecSys algorithms, providing valuable guidance for future research endeavors.


title: One-loop vacuum energy in 10D super-Yang-Mills theory on magnetized tori with/without 4D N=1 supersymmetric completion

abstract: We discuss the behavior of the one-loop vacuum energy of 10 dimensional (10D) super Yang-Mills theory on magnetized tori $\mathbb{R}^{1,3}\times (\mathbb{T}^2)^3$ in the presence of the Abelian magnetic fluxes, including all the contributions from Kaluza-Klein (KK) modes. Higher-dimensional super Yang-Mills action is known to be repackaged in terms of 4D $\mathcal{N}=1$ superfield. We, however, find that such a superspace action differs from the original 10D super Yang-Mills action in the presence of magnetic fluxes. We show that the KK mass spectrum and hence the vacuum energy computed from these two actions differ from each other. In particular, we find that the UV divergence of the vacuum energy based on the original action precisely cancels independently of flux configuration whereas that based on the superspace completion does only when flux configuration preserves supersymmetry, which implies spontaneous or explicit breaking of hidden extended supersymmetry.


title: Reeb Complements for Exploring Inclusions Between Isosurfaces From Two Scalar Fields

abstract: This article proposes to integrate two Reeb graphs with the information of their isosurfaces' inclusion relation. As computing power evolves, there arise numerical data that have small-scale physics inside larger ones -- for example, small clouds in a simulation can be contained inside an atmospheric layer, which is further contained in an enormous hurricane. Extracting such inclusions between isosurfaces is a challenge for isosurfacing: the user would have to explore the vast combinations of isosurfaces $(f_1^{-1}(l_1), f_2^{-1}(l_2))$ from scalar fields $f_i: M(n) \to \mathbb{R}$, $i = 1, 2$, where $M$ is an $n$-dimensional domain manifold and $f_i$ are physical quantities, to find inclusion of one isosurface within another. For this, we propose the \textit{Reeb complement}, a topological space that integrates two Reeb graphs with the inclusion relation. The Reeb complement has a natural partition that classifies equivalent containment of isosurfaces. This is a handy characteristic that lets the Reeb complement serve as an overview of the inclusion relationship in the data. We also propose level-of-detail control of the inclusions through simplification of the Reeb complement. We demonstrate that the relationship of two independent scalar fields can be extracted by taking the product of Reeb graphs (which we call the Reeb product) and by then subtracting the projection of the Reeb space, which opens up a new possibility for feature analysis.


title: Representation Learning Using a Single Forward Pass

abstract: We propose a neuroscience-inspired Solo Pass Embedded Learning Algorithm (SPELA). SPELA is a prime candidate for training and inference applications in Edge AI devices. At the same time, SPELA can optimally cater to the need for a framework to study perceptual representation learning and formation. SPELA has distinctive features such as neural priors (in the form of embedded vectors), no weight transport, no update locking of weights, complete local Hebbian learning, single forward pass with no storage of activations, and single weight update per sample. Juxtaposed with traditional approaches, SPELA operates without the need for backpropagation. We show that our algorithm can perform nonlinear classification on a noisy boolean operation dataset. Additionally, we exhibit high performance using SPELA across MNIST, KMNIST, and Fashion MNIST. Lastly, we show the few-shot and 1-epoch learning capabilities of SPELA on MNIST, KMNIST, and Fashion MNIST, where it consistently outperforms backpropagation.


title: Doping induced multiferroicity and quantum anomalous Hall effect in $\alpha$-In$_2$Se$_3$ thin films

abstract: In flat-band materials, the strong Coulomb interaction between electrons can lead to exotic physical phenomena. Recently, $\alpha$-In$_2$Se$_3$ thin films were found to possess ferroelectricity and flat bands. In this work, using first-principles calculations, we find that for the monolayer, there is a Weyl point at $\Gamma$ in the flat band, where the inclusion of the spin-orbit coupling opens a gap. Shifting the Fermi level into the spin-orbit gap gives rise to nontrivial band topology, which is preserved for the bilayer regardless of the interlayer polarization couplings. We further calculate the Chern number and edge states for both the monolayer and bilayer, for which the results suggest that they become quantum anomalous Hall insulators under appropriate dopings. Moreover, we find that the doping-induced magnetism for In$_2$Se$_3$ bilayer is strongly dependent on the interlayer polarization coupling. Therefore, doping the flat bands in In$_2$Se$_3$ bilayer can also yield multiferroicity, where the magnetism is electrically tunable as the system transforms between different polarization states. Our study thus reveals that multiferroicity and nontrivial band topology can be unified into one material for designing multifunctional electronic devices.


title: AlN Nanowire Based Vertically Integrated Piezoelectric Nanogenerators

abstract: In this study, detailed analysis of the direct piezo-response of AlN nanowire-based vertically integrated nanogenerators (VINGs) is undertaken as a function of mechanical excitation frequency. We show that the piezo-charge, piezo-voltage, and impedance measured at the same position of the devices can be directly correlated through an equivalent circuit model, in the whole frequency range of investigation. Our presented results are utilized to determine the performance figures of merit (FoM) of nanowire-based VINGs, namely the piezoelectric voltage constant (g) for sensing, and the product d g for energy harvesting, where d is the piezoelectric charge constant. By comparison of these metrics with those of freestanding single crystal GaN and quartz substrates, as well as sputtered AlN thin films, we suggest that the nanowires can outperform their rigid counterparts in terms of mechanical sensing and energy generation. This work provides experimental guidelines for understanding the direct piezo-characteristics of VINGs and facilitates a quantitative comparison between nanostructured piezoelectric devices fabricated using different materials or architectures.


title: Optimal Experimental Design for Partially Observable Pure Birth Processes

abstract: We develop an efficient algorithm to find optimal observation times by maximizing the Fisher information for the birth rate of a partially observable pure birth process involving $n$ observations. Partially observable implies that at each of the $n$ observation time points for counting the number of individuals present in the pure birth process, each individual is observed independently with a fixed probability $p$, modeling detection difficulties or constraints on resources. We apply concepts and techniques from generating functions, using a combination of symbolic and numeric computation, to establish a recursion for evaluating and optimizing the Fisher information. Our numerical results reveal the efficacy of this new method. An implementation of the algorithm is available publicly.


title: NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models

abstract: The considerable size of Large Language Models (LLMs) presents notable deployment challenges, particularly on resource-constrained hardware. Structured pruning, offers an effective means to compress LLMs, thereby reducing storage costs and enhancing inference speed for more efficient utilization. In this work, we study data-efficient and resource-efficient structure pruning methods to obtain smaller yet still powerful models. Knowledge Distillation is well-suited for pruning, as the intact model can serve as an excellent teacher for pruned students. However, it becomes challenging in the context of LLMs due to memory constraints. To address this, we propose an efficient progressive Numerous-teacher pruning method (NutePrune). NutePrune mitigates excessive memory costs by loading only one intact model and integrating it with various masks and LoRA modules, enabling it to seamlessly switch between teacher and student roles. This approach allows us to leverage numerous teachers with varying capacities to progressively guide the pruned model, enhancing overall performance. Extensive experiments across various tasks demonstrate the effectiveness of NutePrune. In LLaMA-7B zero-shot experiments, NutePrune retains 97.17% of the performance of the original model at 20% sparsity and 95.07% at 25% sparsity. Our code is available at https://github.com/Lucius-lsr/NutePrune.


title: A lattice formulation of Weyl fermions on a single curved surface

abstract: In the standard lattice domain-wall fermion formulation, one needs two flat domain-walls where both of the left- and right-handed massless modes appear. In this work we investigate a single domain-wall system with a nontrivial curved background. Specifically we consider a massive fermion on a $3D$ square lattice, whose domain-wall is a $2D$ sphere. In the free theory, we find that a single Weyl fermion is localized at the wall and it feels gravity through the induced spin connection. With a topologically nontrivial $U(1)$ link gauge field, however, we find a zero mode with the opposite chirality localized at the center where the gauge field is singular. In the latter case, the low-energy effective theory is not chiral but vectorlike. We discuss how to circumvent this obstacle in formulating lattice chiral gauge theory in the single domain-wall fermion system.


title: Chemotactic particles as strong electrolytes: Debye-H\"uckel approximation and effective mobility law

abstract: We consider a binary mixture of chemically active particles, that produce or consume solute molecules, and that interact with each other through the long-range concentrations fields they generate. We analytically calculate the effective phoretic mobility of these particles when the mixture is submitted to a constant, external concentration gradient, at leading order in the overall concentration. Relying on a analogy with the modeling of strong electrolytes, we show that the effective phoretic mobility decays with the square-root of the concentration: our result is therefore a nonequilibrium counterpart to the celebrated Kohlrausch and Debye-H\"uckel-Onsager conductivity laws for electrolytes, which are extended here to particles with long-range nonreciprocal interactions. The effective mobility law we derive reveals the existence of a regime of maximal mobility, and could find applications in the description of nanoscale transport phenomena in living cells.


title: Strategic Vote Timing in Online Elections With Public Tallies

abstract: We study the effect of public tallies on online elections, in a setting where voting is costly and voters are allowed to strategically time their votes. The strategic importance of choosing \emph{when} to vote arises when votes are public, such as in online event scheduling polls (e.g., Doodle), or in blockchain governance mechanisms. In particular, there is a tension between voting early to influence future votes and waiting to observe interim results and avoid voting costs if the outcome has already been decided. Our study draws on empirical findings showing that "temporal" bandwagon effects occur when interim results are revealed to the electorate: late voters are more likely to vote for leading candidates. To capture this phenomenon, we analyze a novel model where the electorate consists of informed voters who have a preferred candidate, and uninformed swing voters who can be swayed according to the interim outcome at the time of voting. In our main results, we prove the existence of equilibria where both early and late voting occur with a positive probability, and we characterize conditions that lead to the appearance of "last minute" voting behavior, where all informed voters vote late.