title: Concept Formation and Alignment in Language Models: Bridging Statistical Patterns in Latent Space to Concept Taxonomy

abstract: This paper explores the concept formation and alignment within the realm of language models (LMs). We propose a mechanism for identifying concepts and their hierarchical organization within the semantic representations learned by various LMs, encompassing a spectrum from early models like Glove to the transformer-based language models like ALBERT and T5. Our approach leverages the inherent structure present in the semantic embeddings generated by these models to extract a taxonomy of concepts and their hierarchical relationships. This investigation sheds light on how LMs develop conceptual understanding and opens doors to further research to improve their ability to reason and leverage real-world knowledge. We further conducted experiments and observed the possibility of isolating these extracted conceptual representations from the reasoning modules of the transformer-based LMs. The observed concept formation along with the isolation of conceptual representations from the reasoning modules can enable targeted token engineering to open the door for potential applications in knowledge transfer, explainable AI, and the development of more modular and conceptually grounded language models.


title: C-Mamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting

abstract: In recent years, significant progress has been made in multivariate time series forecasting using Linear-based, Transformer-based, and Convolution-based models. However, these approaches face notable limitations: linear forecasters struggle with representation capacities, attention mechanisms suffer from quadratic complexity, and convolutional models have a restricted receptive field. These constraints impede their effectiveness in modeling complex time series, particularly those with numerous variables. Additionally, many models adopt the Channel-Independent (CI) strategy, treating multivariate time series as uncorrelated univariate series while ignoring their correlations. For models considering inter-channel relationships, whether through the self-attention mechanism, linear combination, or convolution, they all incur high computational costs and focus solely on weighted summation relationships, neglecting potential proportional relationships between channels. In this work, we address these issues by leveraging the newly introduced state space model and propose \textbf{C-Mamba}, a novel approach that captures cross-channel dependencies while maintaining linear complexity without losing the global receptive field. Our model consists of two key components: (i) channel mixup, where two channels are mixed to enhance the training sets; (ii) channel attention enhanced patch-wise Mamba encoder that leverages the ability of the state space models to capture cross-time dependencies and models correlations between channels by mining their weight relationships. Our model achieves state-of-the-art performance on seven real-world time series datasets. Moreover, the proposed mixup and attention strategy exhibits strong generalizability across other frameworks.


title: LoCoCo: Dropping In Convolutions for Long Context Compression

abstract: This paper tackles the memory hurdle of processing long context sequences in Large Language Models (LLMs), by presenting a novel approach, Dropping In Convolutions for Long Context Compression (LoCoCo). LoCoCo employs only a fixed-size Key-Value (KV) cache, and can enhance efficiency in both inference and fine-tuning stages. Diverging from prior methods that selectively drop KV pairs based on heuristics, LoCoCo leverages a data-driven adaptive fusion technique, blending previous KV pairs with incoming tokens to minimize the loss of contextual information and ensure accurate attention modeling. This token integration is achieved through injecting one-dimensional convolutional kernels that dynamically calculate mixing weights for each KV cache slot. Designed for broad compatibility with existing LLM frameworks, LoCoCo allows for straightforward "drop-in" integration without needing architectural modifications, while incurring minimal tuning overhead. Experiments demonstrate that LoCoCo maintains consistently outstanding performance across various context lengths and can achieve a high context compression rate during both inference and fine-tuning phases. During inference, we successfully compressed up to 3482 tokens into a 128-size KV cache, while retaining comparable performance to the full sequence - an accuracy improvement of up to 0.2791 compared to baselines at the same cache size. During post-training tuning, we also effectively extended the context length from 4K to 32K using a KV cache of fixed size 512, achieving performance similar to fine-tuning with entire sequences.


title: Integrating Text and Image Pre-training for Multi-modal Algorithmic Reasoning

abstract: In this paper, we present our solution for SMART-101 Challenge of CVPR Multi-modal Algorithmic Reasoning Task 2024. Unlike traditional visual questions and answer tasks, this challenge evaluates abstraction, deduction and generalization ability of neural network in solving visuo-linguistic puzzles designed for specially children in the 6-8 age group. Our model is based on two pre-trained models, dedicated to extract features from text and image respectively. To integrate the features from different modalities, we employed a fusion layer with attention mechanism. We explored different text and image pre-trained models, and fine-tune the integrated classifier on the SMART-101 dataset. Experiment results show that under the data splitting style of puzzle split, our proposed integrated classifier achieves superior performance, verifying the effectiveness of multi-modal pre-trained representations.


title: Spin and lattice dynamics of a van der Waals antiferromagnet MnPSe$_3$

abstract: Antiferromagnetic van der Waals family $\rm \textit{M}P\textit{X}_{3}\ (M=Fe,\ Mn,\ Co,\text{ and}\ Ni; X=S\text{ and}\ Se)$ have attracted significant research attention due to the possibility of realizing long-range magnetic order down to the monolayer limit. Here, we perform inelastic neutron scattering measurements on single crystal samples of MnPSe$_3$, a member of the $\rm \textit{M}P\textit{X}_{3}$ family, to study the spin dynamics and determine the effective spin model. The excited magnon bands are well characterized by a spin model, which includes a Heisenberg term with three intraplane exchange parameters ($J_{1}=-0.73$~meV, $J_{2}=-0.014$~meV, $J_{3}=-0.43$~meV) and one interplane parameter ($J_{c}=-0.054$~meV), and an easy-plane single-ion anisotropy term ($D=-0.035$~meV). Additionally, we observe the intersection of the magnon and phonon bands but no anomalous spectral features induced by the formation of magnon-phonon hybrid excitations at the intersecting region. We discuss possible reasons for the absence of such hybrid excitations in MnPSe$_3$.


title: Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation

abstract: Deep learning has exhibited remarkable results across diverse areas. To understand its success, substantial research has been directed towards its theoretical foundations. Nevertheless, the majority of these studies examine how well deep neural networks can model functions with uniform regularity. In this paper, we explore a different angle: how deep neural networks can adapt to different regularity in functions across different locations and scales and nonuniform data distributions. More precisely, we focus on a broad class of functions defined by nonlinear tree-based approximation. This class encompasses a range of function types, such as functions with uniform regularity and discontinuous functions. We develop nonparametric approximation and estimation theories for this function class using deep ReLU networks. Our results show that deep neural networks are adaptive to different regularity of functions and nonuniform data distributions at different locations and scales. We apply our results to several function classes, and derive the corresponding approximation and generalization errors. The validity of our results is demonstrated through numerical experiments.


title: Instrumental Broadening Of The SPOL Spectropolarimeter At The University Of Arizona

abstract: The Ground-based Observational Support of the Fermi Gamma-ray Space Telescope is conducted by the University of Arizona using the 2.3m Bok and 1.54m Kuiper telescopes operated by the Steward Observatory (SO). This program monitors blazar sources with spectroscopic (among others) observations. Yet, the instrumental broadenings for the different slit widths used in the spectra, are unavailable (the widths range from 2.0 to 12.7 arcsec). Using quasi-simultaneous spectroscopic observations of the blazar 3C 273 between the SO, Observatorio Astrof\'isico Guillermo Haro (OAGH), and Observatorio Astron\'omico Nacional San Pedro M\'artir (OAN-SPM), we can provide an estimation of the instrumental profile for two of the slit widths. Since the instrumental broadening and the slit width are directly proportional, we were able to estimate the instrumental broadening for all six slit widths used at the SO. We found significant variations between the instrumental profiles, emphasizing the need to correct the instrumental broadening for each aperture.


title: Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios

abstract: There is increasing interest in distilling task-specific knowledge from large language models (LLM) to smaller student models. Nonetheless, LLM distillation presents a dual challenge: 1) there is a high cost associated with querying the teacher LLM, such as GPT-4, for gathering an ample number of demonstrations; 2) the teacher LLM might provide imperfect outputs with a negative impact on the student's learning process. To enhance sample efficiency within resource-constrained, imperfect teacher scenarios, we propose a three-component framework leveraging three signal types. The first signal is the student's self-consistency (consistency of student multiple outputs), which is a proxy of the student's confidence. Specifically, we introduce a ``teaching assistant'' (TA) model to assess the uncertainty of both the student's and the teacher's outputs via confidence scoring, which serves as another two signals for student training. Furthermore, we propose a two-stage training schema to first warm up the student with a small proportion of data to better utilize student's signal. Experiments have shown the superiority of our proposed framework for four complex reasoning tasks. On average, our proposed two-stage framework brings a relative improvement of up to 20.79% compared to fine-tuning without any signals across datasets.


title: The $F$-pure threshold and squarefree initial ideal as invariants of linkage

abstract: The generic link of an unmixed, radical (even a generic complete intersection) ideal is radical (in fact, prime). We show that this cannot be strengthened in general: The squarefreeness of the initial ideal and $F$-purity are not preserved along generic links. However, for several important cases in liaison theory, including generic height three Gorenstein ideals and the maximal minors of a generic matrix, we show that the squarefreeness of the initial ideal, $F$-purity, and even the $F$-pure threshold are each preserved along the $n$-th generic link for every $n>0$ by finding a property of such ideals which propagates along generic links. We use this property to establish the $F$-regularity of their generic links. Finally, we study the $F$-pure thresholds of the generic residual intersections of a complete intersection ideal and answer a related question of Kim--Miller--Niu.


title: Probably the simplest and cheapest quantum Monte Carlo method so far for extracting high-precision entanglement entropy and its derivative

abstract: Measuring entanglement entropy (EE) to probe the intrinsic physics of quantum many-body systems is an important but challenging topic in condensed matter, high energy and computational physics. Designing quantum Monte Carlo (QMC) algorithm to obtain the R\'enyi EE is a promising solution in large-scale many-body systems. However, to gain high-precision EE, the QMC-based algorithm for EE becomes more and more complex at the designing level. The entangled region needs being changed during the QMC simulation, and the detailed balance condition becomes more complicated. Moreover, the intermediately incremental processes introduced cannot be exploited neither. In this paper, we propose a simple QMC scheme able to extract EE and its derivative with high-precision, which requires neither changing replica manifold during the simulation nor adding extra detailed balance conditions. All the values measured in the incremental process are the EE under physical parameters, which greatly improves the efficiency. It opens an access to numerically probe the novel phases and phase transitions by scanning EE in a wide parameter-region in 2D and higher dimensional systems. The method has low-technical barrier and is natural for parallel computing. Our algorithm makes it no longer a dream to calculate a large amount of high-precision EE values without complicated techniques and huge computational cost.

categories: cond-mat.str-el cond-mat.stat-mech hep-th physics.comp-ph quant-ph


title: LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance

abstract: Any-to-any singing voice conversion (SVC) is an interesting audio editing technique, aiming to convert the singing voice of one singer into that of another, given only a few seconds of singing data. However, during the conversion process, the issue of timbre leakage is inevitable: the converted singing voice still sounds like the original singer's voice. To tackle this, we propose a latent diffusion model for SVC (LDM-SVC) in this work, which attempts to perform SVC in the latent space using an LDM. We pretrain a variational autoencoder structure using the noted open-source So-VITS-SVC project based on the VITS framework, which is then used for the LDM training. Besides, we propose a singer guidance training method based on classifier-free guidance to further suppress the timbre of the original singer. Experimental results show the superiority of the proposed method over previous works in both subjective and objective evaluations of timbre similarity.


title: Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss

abstract: Since the introduction of BERT and RoBERTa, research on Semantic Textual Similarity (STS) has made groundbreaking progress. Particularly, the adoption of contrastive learning has substantially elevated state-of-the-art performance across various STS benchmarks. However, contrastive learning categorizes text pairs as either semantically similar or dissimilar, failing to leverage fine-grained annotated information and necessitating large batch sizes to prevent model collapse. These constraints pose challenges for researchers engaged in STS tasks that require nuanced similarity levels or those with limited computational resources, compelling them to explore alternatives like Sentence-BERT. Nonetheless, Sentence-BERT tackles STS tasks from a classification perspective, overlooking the progressive nature of semantic relationships, which results in suboptimal performance. To bridge this gap, this paper presents an innovative regression framework and proposes two simple yet effective loss functions: Translated ReLU and Smooth K2 Loss. Experimental analyses demonstrate that our method achieves convincing performance across seven established STS benchmarks, especially when supplemented with task-specific training data.


title: Multi-Entry Generalized Search Trees for Indexing Trajectories

abstract: The idea of generalized indices is one of the success stories of database systems research. It has found its way to implementation in common database systems. GiST (Generalized Search Tree) and SP-GiST (Space-Partitioned Generalized Search Tree) are two widely-used generalized indices that are typically used for multidimensional data. Currently, the generalized indices GiST and SP-GiST represent one database object using one index entry, e.g., a bounding box for each spatio-temporal object. However, when dealing with complex objects, e.g., moving object trajectories, a single entry per object is inadequate for creating efficient indices. Previous research has highlighted that splitting trajectories into multiple bounding boxes prior to indexing can enhance query performance as it leads to a higher index filter. In this paper, we introduce MGiST and MSP-GiST, the multi-entry generalized search tree counterparts of GiST and SP-GiST, respectively, that are designed to enable the partitioning of objects into multiple entries during insertion. The methods for decomposing a complex object into multiple sub-objects differ from one data type to another, and may depend on some domain-specific parameters. Thus, MGiST and MSP-GiST are designed to allow for pluggable modules that aid in optimizing the split of an object into multiple sub-objects. We demonstrate the usefulness of MGiST and MSP-GiST using a trajectory indexing scenario, where we realize several trajectory indexes using MGiST and MSP-GiST and instantiate these search trees with trajectory-specific splitting algorithms. We create and test the performance of several multi-entry versions of widely-used spatial index structures, e.g., R-Tree, Quad-Tree, and KD-Tree. We conduct evaluations using both synthetic and real-world data, and observe up to an order of magnitude enhancement in performance of point, range, and KNN queries.


title: Hidden Question Representations Tell Non-Factuality Within and Across Large Language Models

abstract: Despite the remarkable advance of large language models (LLMs), the prevalence of non-factual responses remains a common issue. This work studies non-factuality prediction (NFP), which predicts whether an LLM will generate non-factual responses to a question before the generation process. Previous efforts on NFP usually rely on extensive computation. In this work, we conduct extensive analysis to explore the capabilities of using a lightweight probe to elicit ``whether an LLM knows'' from the hidden representations of questions. Additionally, we discover that the non-factuality probe employs similar patterns for NFP across multiple LLMs. Motivated by the intriguing finding, we conduct effective transfer learning for cross-LLM NFP and propose a question-aligned strategy to ensure the efficacy of mini-batch based training.


title: Global stability of large Fourier mode for 3-D anisotropic Navier-Stokes equations in cylindrical domain

abstract: In this paper, we first establish the global existence and stability of solutions to 3-D classical Navier-Stokes equations $(NS)$ in an infinite cylindrical domain with large Fourier mode initial data. Then we extend similar result for 3-D anisotropic Navier-Stokes equations $(ANS).$ We remark that due to the loss of vertical viscosity in $(ANS),$ the construction of the energy functionals for $(ANS)$ is much more subtle than that of $(NS).$ Compared with our previous paper for $(NS)$, we improve the polynomial decay in $k$ for the Fourier coefficients of the solution to be exponential decay in $k$ here.


title: Formation of transfermium elements in reactions with $^{208}$Pb

abstract: Within the Langevin framework, we investigate the dynamics of the fusion process for production of transfermium elements in reactions of $^{48}$Ca, $^{50}$Ti, $^{54}$Cr, and $^{58}$Fe with $^{208}$Pb. After the reacting nuclei have made contact, the early dynamical stage is dominated by the dissipation of the initial radial kinetic energy, while the subsequent shape evolution is diffusive. The probability for surmounting the inner barrier and forming a compound system is obtained by simulating the evolution as a Metropolis random walk in a five-dimensional potential-energy landscape. Good agreement with the available data is obtained, especially for the maximal formation probability.


title: Autonomous Robotic Assembly: From Part Singulation to Precise Assembly

abstract: Imagine a robot that can assemble a functional product from the individual parts presented in any configuration to the robot. Designing such a robotic system is a complex problem which presents several open challenges. To bypass these challenges, the current generation of assembly systems is built with a lot of system integration effort to provide the structure and precision necessary for assembly. These systems are mostly responsible for part singulation, part kitting, and part detection, which is accomplished by intelligent system design. In this paper, we present autonomous assembly of a gear box with minimum requirements on structure. The assembly parts are randomly placed in a two-dimensional work environment for the robot. The proposed system makes use of several different manipulation skills such as sliding for grasping, in-hand manipulation, and insertion to assemble the gear box. All these tasks are run in a closed-loop fashion using vision, tactile, and Force-Torque (F/T) sensors. We perform extensive hardware experiments to show the robustness of the proposed methods as well as the overall system. See supplementary video at https://www.youtube.com/watch?v=cZ9M1DQ23OI.


title: Transformer Conformal Prediction for Time Series

abstract: We present a conformal prediction method for time series using the Transformer architecture to capture long-memory and long-range dependencies. Specifically, we use the Transformer decoder as a conditional quantile estimator to predict the quantiles of prediction residuals, which are used to estimate the prediction interval. We hypothesize that the Transformer decoder benefits the estimation of the prediction interval by learning temporal dependencies across past prediction residuals. Our comprehensive experiments using simulated and real data empirically demonstrate the superiority of the proposed method compared to the existing state-of-the-art conformal prediction methods.


title: Dissipation rates from experimental uncertainty

abstract: Experimental uncertainty prompted the early development of the quantum uncertainty relations nowknownasspeedlimits. However, it has not yet been a part of the development of thermodynamic speed limits on dissipation. Here, we predict the maximal rates of heat and entropy production using experimentally accessible uncertainties in a thermodynamic speed limit. Because these rates can be difficult to measure directly, we reparametrize the speed limit to predict these observables indirectly from quantities that are readily measurable with experiments. From this transformed speed limit, we identify the resolution an experiment will need to upper bound nonequilibrium rates. Without models for the dynamics, these speed limit predictions agree with calorimetric measurements of the energy dissipated by a pulled Brownian particle and a microtubule active gel, validating the approach and suggesting potential for the design of experiments.


title: Purely Quantum Nonreciprocity by Spatially Separated Transmission Scheme

abstract: Nonreciprocal photon blockade is of particular interest due to its potential applications in chiral quantum technologies and topological photonics. In the regular cases, nonreciprocal transmission (classical nonreciprocity) and nonreciprocal photon blockade (quantum nonreciprocity) often appear simultaneously. Nevertheless, how to achieve purely quantum nonreciprocity (no classical nonreciprocity) remains largely unexplored. Here, we propose a spatially separated transmission scheme, that the photons transport in different directions take different paths, in an optical system consisting of two spinning cavities coupled indirectly by two common drop-filter waveguides. Based on the spatially separated transmission scheme, we demonstrate a purely quantum nonreciprocity (nonreciprocal photon blockade) by considering the Kerr nonlinear interaction in one of the paths. Interestingly, we find that the nonreciprocal photon blockade is enhanced nonreciprocally, i.e., the nonreciprocal photon blockade is enhanced when the photons transport in one direction but suppressed in the reverse direction. We identify that the nonreciprocal enhancement of nonreciprocal photon blockade is induced by the destructive or constructive interference between two paths for two photons passing through the whole system. The spatially separated transmission scheme proposed in the work provides a novel approach to observe purely quantum nonreciprocal effects.


title: Critical Phase Transition in a Large Language Model

abstract: The performance of large language models (LLMs) strongly depends on the \textit{temperature} parameter. Empirically, at very low temperatures, LLMs generate sentences with clear repetitive structures, while at very high temperatures, generated sentences are often incomprehensible. In this study, using GPT-2, we numerically demonstrate that the difference between the two regimes is not just a smooth change but a phase transition with singular, divergent statistical quantities. Our extensive analysis shows that critical behaviors, such as a power-law decay of correlation in a text, emerge in the LLM at the transition temperature as well as in a natural language dataset. We also discuss that several statistical quantities characterizing the criticality should be useful to evaluate the performance of LLMs.


title: Multi-Vehicle Trajectory Planning at V2I-enabled Intersections based on Correlated Equilibrium

abstract: Generating trajectories that ensure both vehicle safety and improve traffic efficiency remains a challenging task at intersections. Many existing works utilize Nash equilibrium (NE) for the trajectory planning at intersections. However, NE-based planning can hardly guarantee that all vehicles are in the same equilibrium, leading to a risk of collision. In this work, we propose a framework for trajectory planning based on Correlated Equilibrium (CE) when V2I communication is also enabled. The recommendation with CE allows all vehicles to reach a safe and consensual equilibrium and meanwhile keeps the rationality as NE-based methods that no vehicle has the incentive to deviate. The Intersection Manager (IM) first collects the trajectory library and the personal preference probabilities over the library from each vehicle in a low-resolution spatial-temporal grid map. Then, the IM optimizes the recommendation probability distribution for each vehicle's trajectory by minimizing overall collision probability under the CE constraint. Finally, each vehicle samples a trajectory of the low-resolution map to construct a safety corridor and derive a smooth trajectory with a local refinement optimization. We conduct comparative experiments at a crossroad intersection involving two and four vehicles, validating the effectiveness of our method in balancing vehicle safety and traffic efficiency.


title: On Onsager's type conjecture for the inviscid Boussinesq equations

abstract: In this paper, we investigate the Cauchy problem for the three dimensional inviscid Boussinesq system in the periodic setting. For $1\le p\le \infty$, we show that the threshold regularity exponent for $L^p$-norm conservation of temperature of this system is $1/3$, consistent with Onsager exponent. More precisely, for $1\le p\le\infty$, every weak solution $(v,\theta)\in C_tC^{\beta}_x$ to the inviscid Boussinesq equations satisfies that $\|\theta(t)\|_{L^p(\mathbb{T}^3)}=\|\theta_0\|_{L^p(\mathbb{T}^3)}$ if $\beta>\frac{1}{3}$, while if $\beta<\frac{1}{3}$, there exist infinitely many weak solutions $(v,\theta)\in C_tC^{\beta}_x$ such that the $L^p$-norm of temperature is not conserved. As a byproduct, we are able to construct many weak solutions in $C_tC^{\beta}_x$ for $\beta<\frac{1}{3}$ displaying wild behavior, such as fast kinetic energy dissipation and high oscillation of velocity. Moreover, we also show that if a weak solution $(v, \theta)$ of this system has at least one interval of regularity, then this weak solution $(v,\theta)$ is not unique in $C_tC^{\beta}_x$ for $\beta<\frac{1}{3}$.


title: MotionClone: Training-Free Motion Cloning for Controllable Video Generation

abstract: Motion-based controllable text-to-video generation involves motions to control the video generation. Previous methods typically require the training of models to encode motion cues or the fine-tuning of video diffusion models. However, these approaches often result in suboptimal motion generation when applied outside the trained domain. In this work, we propose MotionClone, a training-free framework that enables motion cloning from a reference video to control text-to-video generation. We employ temporal attention in video inversion to represent the motions in the reference video and introduce primary temporal-attention guidance to mitigate the influence of noisy or very subtle motions within the attention weights. Furthermore, to assist the generation model in synthesizing reasonable spatial relationships and enhance its prompt-following capability, we propose a location-aware semantic guidance mechanism that leverages the coarse location of the foreground from the reference video and original classifier-free guidance features to guide the video generation. Extensive experiments demonstrate that MotionClone exhibits proficiency in both global camera motion and local object motion, with notable superiority in terms of motion fidelity, textual alignment, and temporal consistency.


title: To what extent can ASV systems naturally defend against spoofing attacks?

abstract: The current automatic speaker verification (ASV) task involves making binary decisions on two types of trials: target and non-target. However, emerging advancements in speech generation technology pose significant threats to the reliability of ASV systems. This study investigates whether ASV effortlessly acquires robustness against spoofing attacks (i.e., zero-shot capability) by systematically exploring diverse ASV systems and spoofing attacks, ranging from traditional to cutting-edge techniques. Through extensive analyses conducted on eight distinct ASV systems and 29 spoofing attack systems, we demonstrate that the evolution of ASV inherently incorporates defense mechanisms against spoofing attacks. Nevertheless, our findings also underscore that the advancement of spoofing attacks far outpaces that of ASV systems, hence necessitating further research on spoofing-robust ASV methodologies.


title: Selecting the Number of Communities for Weighted Degree-Corrected Stochastic Block Models

abstract: We investigate how to select the number of communities for weighted networks without a full likelihood modeling. First, we propose a novel weighted degree-corrected stochastic block model (DCSBM), in which the mean adjacency matrix is modeled as the same as in standard DCSBM, while the variance profile matrix is assumed to be related to the mean adjacency matrix through a given variance function. Our method of selection the number of communities is based on a sequential testing framework, in each step the weighed DCSBM is fitted via some spectral clustering method. A key step is to carry out matrix scaling on the estimated variance profile matrix. The resulting scaling factors can be used to normalize the adjacency matrix, from which the testing statistic is obtained. Under mild conditions on the weighted DCSBM, our proposed procedure is shown to be consistent in estimating the true number of communities. Numerical experiments on both simulated and real network data also demonstrate the desirable empirical properties of our method.


title: Diversifying and Expanding Frequency-Adaptive Convolution Kernels for Sound Event Detection

abstract: Frequency dynamic convolution (FDY conv) has shown the state-of-the-art performance in sound event detection (SED) using frequency-adaptive kernels obtained by frequency-varying combination of basis kernels. However, FDY conv lacks an explicit mean to diversify frequency-adaptive kernels, potentially limiting the performance. In addition, size of basis kernels is limited while time-frequency patterns span larger spectro-temporal range. Therefore, we propose dilated frequency dynamic convolution (DFD conv) which diversifies and expands frequency-adaptive kernels by introducing different dilation sizes to basis kernels. Experiments showed advantages of varying dilation sizes along frequency dimension, and analysis on attention weight variance proved dilated basis kernels are effectively diversified. By adapting class-wise median filter with intersection-based F1 score, proposed DFD-CRNN outperforms FDY-CRNN by 3.12% in terms of polyphonic sound detection score (PSDS).


title: Compensation for reactive power and harmonic current drawn by a non-linear load in a pv-micro hydro grid

abstract: This paper presents a simulation approach to enhance the power quality of a PV-micro hydro grid supplying both linear consumer load and non-linear industrial load by integrating Shunt Active Power Filter (SAPF), utilizing instantaneous PQ theory and hysteresis current control band logic. The non-linear load draws reactive power and harmonic current from the source thereby affecting the power quality. The integration of the SAPF at the point of common coupling (PCC) offers reactive power and harmonic current compensation, ensuring that the current supply to the grid remains nearly sinusoidal and proportional to the active power. By injecting equal and opposite harmonic components, the SAPF effectively reduces Total Harmonic Distortion (THD) from 7% to 2.96%, thereby enhancing the overall power quality of the PV-micro hydro grid system.


title: M3GIA: A Cognition Inspired Multilingual and Multimodal General Intelligence Ability Benchmark

abstract: As recent multi-modality large language models (MLLMs) have shown formidable proficiency on various complex tasks, there has been increasing attention on debating whether these models could eventually mirror human intelligence. However, existing benchmarks mainly focus on evaluating solely on task performance, such as the accuracy of identifying the attribute of an object. Combining well-developed cognitive science to understand the intelligence of MLLMs beyond superficial achievements remains largely unexplored. To this end, we introduce the first cognitive-driven multi-lingual and multi-modal benchmark to evaluate the general intelligence ability of MLLMs, dubbed M3GIA. Specifically, we identify five key cognitive factors based on the well-recognized Cattell-Horn-Carrol (CHC) model of intelligence and propose a novel evaluation metric. In addition, since most MLLMs are trained to perform in different languages, a natural question arises: is language a key factor influencing the cognitive ability of MLLMs? As such, we go beyond English to encompass other languages based on their popularity, including Chinese, French, Spanish, Portuguese and Korean, to construct our M3GIA. We make sure all the data relevant to the cultural backgrounds are collected from their native context to avoid English-centric bias. We collected a significant corpus of data from human participants, revealing that the most advanced MLLM reaches the lower boundary of human intelligence in English. Yet, there remains a pronounced disparity in the other five languages assessed. We also reveals an interesting winner takes all phenomenon that are aligned with the discovery in cognitive studies. Our benchmark will be open-sourced, with the aspiration of facilitating the enhancement of cognitive capabilities in MLLMs.


title: MemeGuard: An LLM and VLM-based Framework for Advancing Content Moderation via Meme Intervention

abstract: In the digital world, memes present a unique challenge for content moderation due to their potential to spread harmful content. Although detection methods have improved, proactive solutions such as intervention are still limited, with current research focusing mostly on text-based content, neglecting the widespread influence of multimodal content like memes. Addressing this gap, we present \textit{MemeGuard}, a comprehensive framework leveraging Large Language Models (LLMs) and Visual Language Models (VLMs) for meme intervention. \textit{MemeGuard} harnesses a specially fine-tuned VLM, \textit{VLMeme}, for meme interpretation, and a multimodal knowledge selection and ranking mechanism (\textit{MKS}) for distilling relevant knowledge. This knowledge is then employed by a general-purpose LLM to generate contextually appropriate interventions. Another key contribution of this work is the \textit{\textbf{I}ntervening} \textit{\textbf{C}yberbullying in \textbf{M}ultimodal \textbf{M}emes (ICMM)} dataset, a high-quality, labeled dataset featuring toxic memes and their corresponding human-annotated interventions. We leverage \textit{ICMM} to test \textit{MemeGuard}, demonstrating its proficiency in generating relevant and effective responses to toxic memes.


title: Dynamical and thermodynamic crossovers in the supercritical region of a holographic superfluid model

abstract: Many physical systems, including classical fluids, present in their phase diagram the competition between two phases that are separated by a line of first-order phase transitions which terminates at a so-called critical point. Despite several proposals, in the supercritical region beyond the critical point, whether the two phases can still be distinguished and by which criterion remain open questions. In this work, we study the thermodynamics and linear dynamics of a holographic superfluid model with nonlinear potential terms in the supercritical region. We identify the presence of a dynamical crossover, akin to the liquid-like to gas-like Frenkel transition in supercritical fluids, and we define other separation lines of thermodynamic origin based on higher order derivatives of the free energy with respect to the charge density. Our results highlight the universal dynamical and thermodynamic features of supercritical systems from nuclear matter and classical fluids to superfluid systems.


title: ProG: A Graph Prompt Learning Benchmark

abstract: Artificial general intelligence on graphs has shown significant advancements across various applications, yet the traditional 'Pre-train & Fine-tune' paradigm faces inefficiencies and negative transfer issues, particularly in complex and few-shot settings. Graph prompt learning emerges as a promising alternative, leveraging lightweight prompts to manipulate data and fill the task gap by reformulating downstream tasks to the pretext. However, several critical challenges still remain: how to unify diverse graph prompt models, how to evaluate the quality of graph prompts, and to improve their usability for practical comparisons and selection. In response to these challenges, we introduce the first comprehensive benchmark for graph prompt learning. Our benchmark integrates SIX pre-training methods and FIVE state-of-the-art graph prompt techniques, evaluated across FIFTEEN diverse datasets to assess performance, flexibility, and efficiency. We also present 'ProG', an easy-to-use open-source library that streamlines the execution of various graph prompt models, facilitating objective evaluations. Additionally, we propose a unified framework that categorizes existing graph prompt methods into two main approaches: prompts as graphs and prompts as tokens. This framework enhances the applicability and comparison of graph prompt techniques. The code is available at: https://github.com/sheldonresearch/ProG.


title: MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training

abstract: Multiple Sequence Alignment (MSA) plays a pivotal role in unveiling the evolutionary trajectories of protein families. The accuracy of protein structure predictions is often compromised for protein sequences that lack sufficient homologous information to construct high quality MSA. Although various methods have been proposed to generate virtual MSA under these conditions, they fall short in comprehensively capturing the intricate coevolutionary patterns within MSA or require guidance from external oracle models. Here we introduce MSAGPT, a novel approach to prompt protein structure predictions via MSA generative pretraining in the low MSA regime. MSAGPT employs a simple yet effective 2D evolutionary positional encoding scheme to model complex evolutionary patterns. Endowed by this, its flexible 1D MSA decoding framework facilitates zero or few shot learning. Moreover, we demonstrate that leveraging the feedback from AlphaFold2 can further enhance the model capacity via Rejective Fine tuning (RFT) and Reinforcement Learning from AF2 Feedback (RLAF). Extensive experiments confirm the efficacy of MSAGPT in generating faithful virtual MSA to enhance the structure prediction accuracy. The transfer learning capabilities also highlight its great potential for facilitating other protein tasks.


title: Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets

abstract: We explore the ability of GPT-4 to perform ad-hoc schema based information extraction from scientific literature. We assess specifically whether it can, with a basic prompting approach, replicate two existing material science datasets, given the manuscripts from which they were originally manually extracted. We employ materials scientists to perform a detailed manual error analysis to assess where the model struggles to faithfully extract the desired information, and draw on their insights to suggest research directions to address this broadly important task.


title: Blurry-Consistency Segmentation Framework with Selective Stacking on Differential Interference Contrast 3D Breast Cancer Spheroid

abstract: The ability of three-dimensional (3D) spheroid modeling to study the invasive behavior of breast cancer cells has drawn increased attention. The deep learning-based image processing framework is very effective at speeding up the cell morphological analysis process. Out-of-focus photos taken while capturing 3D cells under several z-slices, however, could negatively impact the deep learning model. In this work, we created a new algorithm to handle blurry images while preserving the stacked image quality. Furthermore, we proposed a unique training architecture that leverages consistency training to help reduce the bias of the model when dense-slice stacking is applied. Additionally, the model's stability is increased under the sparse-slice stacking effect by utilizing the self-training approach. The new blurring stacking technique and training flow are combined with the suggested architecture and self-training mechanism to provide an innovative yet easy-to-use framework. Our methods produced noteworthy experimental outcomes in terms of both quantitative and qualitative aspects.


title: Exploiting Monotonicity to Design an Adaptive PI Passivity-Based Controller for a Fuel-Cell System

abstract: We present a controller for a power electronic system composed of a fuel cell (FC) connected to a boost converter which feeds a resistive load. The controller aims to regulate the output voltage of the converter regardless of the uncertainty of the load. Leveraging the monotonicity feature of the fuel cell polarization curve we prove that the nonlinear system can be controlled by means of a passivity-based proportional-integral approach. We afterward extend the result to an adaptive version, allowing the controller to deal with parameter uncertainties, such as inductor parasitic resistance, load, and FC polarization curve parameters. This adaptive design is based on an indirect control approach with online parameter identification performed by a ``hybrid'' estimator which combines two techniques: the gradient-descent and immersion-and-invariance algorithms. The overall system is proved to be stable with the output voltage regulated to its reference. Experimental results validate our proposal under two real-life scenarios: pulsating load and output voltage reference changes.


title: Effects of rotation and anisotropy on the properties of type-II holographic superconductors

abstract: The present work concerns the detailed construction of a holographic model for a type-II s-wave superconductor defined on a 5-dimensional anisotropic rotating black hole. We examine the role of rotation and anisotropy on the properties of the superconductor model focusing on the condensate and the AC conductivity, for which we obtain closed formulas, using both analytical and numerical methods. The results reveal that the rotation is responsible for the appearance of a peak and for introducing an exponentially vanishing behavior in the high-frequency limit of the real component of the AC conductivity. Such a behavior aligns with that observed in high-temperature superconductor models and experiments, where the peak and vanishing behavior result from quasiparticle damping, suggesting a plausible relation between the rotation of a black hole and quasiparticle damping effects in a superconducting material. In addition, we provide a detailed construction of the vortex lattice presented in arXiv:2208.05988 and study its behavior as a function of an external uniform magnetic field. Once again, it is shown that the vortex lattice can be continuously deformed along with a change in the vortex population by virtue of the magnetic field, providing a promising avenue for holographically modeling the vortex lattice deformations observed in experimental superconducting materials. As an observed experimental effect, we describe both the vortex lattice deformation and the vortex population increment under the action of an external magnetic field in the LiFeAs type-II superconductor. These effects supplement those previously found for the FeSe type-II superconductor studied in arXiv:2208.05988.


title: 1st Place Winner of the 2024 Pixel-level Video Understanding in the Wild (CVPR'24 PVUW) Challenge in Video Panoptic Segmentation and Best Long Video Consistency of Video Semantic Segmentation

abstract: The third Pixel-level Video Understanding in the Wild (PVUW CVPR 2024) challenge aims to advance the state of art in video understanding through benchmarking Video Panoptic Segmentation (VPS) and Video Semantic Segmentation (VSS) on challenging videos and scenes introduced in the large-scale Video Panoptic Segmentation in the Wild (VIPSeg) test set and the large-scale Video Scene Parsing in the Wild (VSPW) test set, respectively. This paper details our research work that achieved the 1st place winner in the PVUW'24 VPS challenge, establishing state of art results in all metrics, including the Video Panoptic Quality (VPQ) and Segmentation and Tracking Quality (STQ). With minor fine-tuning our approach also achieved the 3rd place in the PVUW'24 VSS challenge ranked by the mIoU (mean intersection over union) metric and the first place ranked by the VC16 (16-frame video consistency) metric. Our winning solution stands on the shoulders of giant foundational vision transformer model (DINOv2 ViT-g) and proven multi-stage Decoupled Video Instance Segmentation (DVIS) frameworks for video understanding.


title: Nanoparticle uptake by a semi-permeable, spherical cell from an external planar diffusive field. I. Mathematical model and asymptotic solution

abstract: In this paper we consider the diffusion of nanoparticles taken up by a semi-permeable spherical cell placed in the path of a diffusive particle field generated by an external planar source. The cell interior and exterior are characterized by different diffusive properties, while the cell is able to accommodate a different saturation level of particles at steady state than is present in the external medium. The situation models the practical problem of biological cells exposed from one direction. The conflict of geometries is handled by the introduction of an effective boundary condition at a virtual spherical boundary. A closed-form, large-time asymptotic solution for the local concentration interior to the cell is developed. We consequently derive an asymptotic approximation for the rate of nanoparticle accumulation in the cell. We contrast the resulting time dependence with that of the corresponding quantity found under strictly spherically symmetric conditions.


title: Investigating Memory Failure Prediction Across CPU Architectures

abstract: Large-scale datacenters often experience memory failures, where Uncorrectable Errors (UEs) highlight critical malfunction in Dual Inline Memory Modules (DIMMs). Existing approaches primarily utilize Correctable Errors (CEs) to predict UEs, yet they typically neglect how these errors vary between different CPU architectures, especially in terms of Error Correction Code (ECC) applicability. In this paper, we investigate the correlation between CEs and UEs across different CPU architectures, including X86 and ARM. Our analysis identifies unique patterns of memory failure associated with each processor platform. Leveraging Machine Learning (ML) techniques on production datasets, we conduct the memory failure prediction in different processors' platforms, achieving up to 15% improvements in F1-score compared to the existing algorithm. Finally, an MLOps (Machine Learning Operations) framework is provided to consistently improve the failure prediction in the production environment.


title: Revisit to the WGVC schemes: a nonlinear order-preserving and spectral-property-optimized methodology and its enhancement

abstract: The numerical simulation of supersonic complex flow problems demands capabilities in identifying multiscale structures and capturing shocks, imposing stringent requirements on the numerical scheme. The capability to identify multiscale structures is closely related to the spectral properties of the numerical scheme. Currently, existing methods to improve the spectral properties of finite difference schemes face shortcomings such as parallel difficulties (compact methods) or introducing unnecessary dispersion errors at low wavenumbers due to accuracy loss (spectral-like optimization methods). In this paper, we proposed an order-preserving spectral properties optimization method based on the group velocity control theory: the weighted group velocity control (WGVC) scheme. This method, centered around the concept of group velocity, achieves low-wavenumber accuracy control and mid-wavenumber group velocity control by designing smoothness indicators and nonlinear weighting approach for wave packets. Furthermore, by embedding the WGVC scheme into shock-capturing schemes such as WENO/TENO scheme, we not only preserve the spectral properties of the WGVC scheme at medium to low wavenumbers but also enhance the shock-capturing capability of the scheme. Theoretical and numerical experiments verify that the new method has advantages such as order-preserving, small dispersion and dissipation errors, and is very suitable for numerical simulation of complex flow problems such as turbulence-shock boundary layer interactions.


title: Thermalization dynamics in photonic lattices of different geometries

abstract: The statistical mechanical behavior of weakly nonlinear multimoded optical settings is attracting increased interest during the last few years. The main purpose of this work is to numerically investigate the main factors that affect the thermalization process in photonic lattices. In particular, we find that lattices with identically selected properties (such as temperature, coupling coefficient, lattice size, and excitation conditions) can exhibit very different thermalization dynamics and thus thermalization distances. Our investigation is focused on two different two-dimensional lattices: the honeycomb lattice and the triangular lattice. Our numerical results show that, independently of the excitation conditions, the honeycomb lattice always thermalizes faster than the triangular lattice. We mainly explain this behavior to the quasilinear spectrum that promotes wave-mixing in the honeycomb lattice in comparison to the power-like spectrum of the triangular lattice. In addition, we investigate the combined effects of temperature as well as the sign and magnitude of the nonlinearity. Switching either the sign of the Kerr nonlinear coefficient or the sign of the temperature can lead to significant differences in the thermalization dynamics, a phenomenon that can be physically explained in terms of wave instabilities. Larger absolute values of the temperature |T| result in more uniform distributions for the power occupation numbers and faster thermalization speeds. Finally, as expected, increasing the magnitude of the nonlinearity results in accelerated thermalization. Our findings provide valuable insights into optical thermalization in discrete systems where experimental realization may bring about new possibilities for light manipulation and applications.


title: Classification of Fermi Gamma-Ray Bursts Based on Machine Learning

abstract: Gamma-ray bursts (GRBs) are typically classified into long and short GRBs based on their durations. However, there is a significant overlapping in the duration distributions of these two categories. In this paper, we apply the unsupervised dimensionality reduction algorithm called t-SNE and UMAP to classify 2061 Fermi GRBs based on four observed quantities: duration, peak energy, fluence, and peak flux. The map results of t-SNE and UMAP show a clear division of these GRBs into two clusters. We mark the two clusters as GRBs-I and GRBs-II, and find that all GRBs associated with supernovae are classified as GRBs-II. It includes the peculiar short GRB 200826A, which was confirmed to originate from the death of a massive star. Furthermore, except for two extreme events GRB 211211A and GRB 230307A, all GRBs associated with kilonovae fall into GRBs-I population. By comparing to the traditional classification of short and long GRBs, the distribution of durations for GRBs-I and GRBs-II do not have a fixed boundary. We find that more than 10% of GRBs-I have a duration greater than 2 seconds, while approximately 1% of GRBs-II have a duration shorter than 2 seconds.


title: Reinforcement Learning for Intensity Control: An Application to Choice-Based Network Revenue Management

abstract: Intensity control is a type of continuous-time dynamic optimization problems with many important applications in Operations Research including queueing and revenue management. In this study, we adapt the reinforcement learning framework to intensity control using choice-based network revenue management as a case study, which is a classical problem in revenue management that features a large state space, a large action space and a continuous time horizon. We show that by utilizing the inherent discretization of the sample paths created by the jump points, a unique and defining feature of intensity control, one does not need to discretize the time horizon in advance, which was believed to be necessary because most reinforcement learning algorithms are designed for discrete-time problems. As a result, the computation can be facilitated and the discretization error is significantly reduced. We lay the theoretical foundation for the Monte Carlo and temporal difference learning algorithms for policy evaluation and develop policy gradient based actor critic algorithms for intensity control. Via a comprehensive numerical study, we demonstrate the benefit of our approach versus other state-of-the-art benchmarks.


title: Towards Lightweight Speaker Verification via Adaptive Neural Network Quantization

abstract: Modern speaker verification (SV) systems typically demand expensive storage and computing resources, thereby hindering their deployment on mobile devices. In this paper, we explore adaptive neural network quantization for lightweight speaker verification. Firstly, we propose a novel adaptive uniform precision quantization method which enables the dynamic generation of quantization centroids customized for each network layer based on k-means clustering. By applying it to the pre-trained SV systems, we obtain a series of quantized variants with different bit widths. To enhance the performance of low-bit quantized models, a mixed precision quantization algorithm along with a multi-stage fine-tuning (MSFT) strategy is further introduced. Unlike uniform precision quantization, mixed precision approach allows for the assignment of varying bit widths to different network layers. When bit combination is determined, MSFT is employed to progressively quantize and fine-tune network in a specific order. Finally, we design two distinct binary quantization schemes to mitigate performance degradation of 1-bit quantized models: the static and adaptive quantizers. Experiments on VoxCeleb demonstrate that lossless 4-bit uniform precision quantization is achieved on both ResNets and DF-ResNets, yielding a promising compression ratio of around 8. Moreover, compared to uniform precision approach, mixed precision quantization not only obtains additional performance improvements with a similar model size but also offers the flexibility to generate bit combination for any desirable model size. In addition, our suggested 1-bit quantization schemes remarkably boost the performance of binarized models. Finally, a thorough comparison with existing lightweight SV systems reveals that our proposed models outperform all previous methods by a large margin across various model size ranges.


title: Flexible and Adaptable Summarization via Expertise Separation

abstract: A proficient summarization model should exhibit both flexibility -- the capacity to handle a range of in-domain summarization tasks, and adaptability -- the competence to acquire new knowledge and adjust to unseen out-of-domain tasks. Unlike large language models (LLMs) that achieve this through parameter scaling, we propose a more parameter-efficient approach in this study. Our motivation rests on the principle that the general summarization ability to capture salient information can be shared across different tasks, while the domain-specific summarization abilities need to be distinct and tailored. Concretely, we propose MoeSumm, a Mixture-of-Expert Summarization architecture, which utilizes a main expert for gaining the general summarization capability and deputy experts that selectively collaborate to meet specific summarization task requirements. We further propose a max-margin loss to stimulate the separation of these abilities. Our model's distinct separation of general and domain-specific summarization abilities grants it with notable flexibility and adaptability, all while maintaining parameter efficiency. MoeSumm achieves flexibility by managing summarization across multiple domains with a single model, utilizing a shared main expert and selected deputy experts. It exhibits adaptability by tailoring deputy experts to cater to out-of-domain few-shot and zero-shot scenarios. Experimental results on 11 datasets show the superiority of our model compared with recent baselines and LLMs. We also provide statistical and visual evidence of the distinct separation of the two abilities in MoeSumm (https://github.com/iriscxy/MoE_Summ).


title: Write Summary Step-by-Step: A Pilot Study of Stepwise Summarization

abstract: Nowadays, neural text generation has made tremendous progress in abstractive summarization tasks. However, most of the existing summarization models take in the whole document all at once, which sometimes cannot meet the needs in practice. Practically, social text streams such as news events and tweets keep growing from time to time, and can only be fed to the summarization system step by step. Hence, in this paper, we propose the task of Stepwise Summarization, which aims to generate a new appended summary each time a new document is proposed. The appended summary should not only summarize the newly added content but also be coherent with the previous summary, to form an up-to-date complete summary. To tackle this challenge, we design an adversarial learning model, named Stepwise Summary Generator (SSG). First, SSG selectively processes the new document under the guidance of the previous summary, obtaining polished document representation. Next, SSG generates the summary considering both the previous summary and the document. Finally, a convolutional-based discriminator is employed to determine whether the newly generated summary is coherent with the previous summary. For the experiment, we extend the traditional two-step update summarization setting to a multi-step stepwise setting, and re-propose a large-scale stepwise summarization dataset based on a public story generation dataset. Extensive experiments on this dataset show that SSG achieves state-of-the-art performance in terms of both automatic metrics and human evaluations. Ablation studies demonstrate the effectiveness of each module in our framework. We also discuss the benefits and limitations of recent large language models on this task.


title: RAPID: Robust APT Detection and Investigation Using Context-Aware Deep Learning

abstract: Advanced persistent threats (APTs) pose significant challenges for organizations, leading to data breaches, financial losses, and reputational damage. Existing provenance-based approaches for APT detection often struggle with high false positive rates, a lack of interpretability, and an inability to adapt to evolving system behavior. We introduce RAPID, a novel deep learning-based method for robust APT detection and investigation, leveraging context-aware anomaly detection and alert tracing. By utilizing self-supervised sequence learning and iteratively learned embeddings, our approach effectively adapts to dynamic system behavior. The use of provenance tracing both enriches the alerts and enhances the detection capabilities of our approach. Our extensive evaluation demonstrates RAPID's effectiveness and computational efficiency in real-world scenarios. In addition, RAPID achieves higher precision and recall than state-of-the-art methods, significantly reducing false positives. RAPID integrates contextual information and facilitates a smooth transition from detection to investigation, providing security teams with detailed insights to efficiently address APT threats.


title: The symplectic characteristic polynomial

abstract: We introduce the notion of the symplectic characteristic polynomial of an endomorphism of a symplectic vector space. This is a polynomial in two variables and can be considered as a generalization of the characteristic polynomial of the endomorphism in the context of symplectic linear algebra. One of the goal of this paper is to prove that the symplectic characteristic polynomial is a complete symplectic invariant of symplectically diagonalizable endomorphisms.


title: Is On-Device AI Broken and Exploitable? Assessing the Trust and Ethics in Small Language Models

abstract: In this paper, we present a very first study to investigate trust and ethical implications of on-device artificial intelligence (AI), focusing on ''small'' language models (SLMs) amenable for personal devices like smartphones. While on-device SLMs promise enhanced privacy, reduced latency, and improved user experience compared to cloud-based services, we posit that they might also introduce significant challenges and vulnerabilities compared to on-server counterparts. As part of our trust assessment study, we conduct a systematic evaluation of the state-of-the-art on-devices SLMs, contrasted to their on-server counterparts, based on a well-established trustworthiness measurement framework. Our results show on-device SLMs to be (statistically) significantly less trustworthy, specifically demonstrating more stereotypical, unfair and privacy-breaching behavior. Informed by these findings, we then perform our ethics assessment study by inferring whether SLMs would provide responses to potentially unethical vanilla prompts, collated from prior jailbreaking and prompt engineering studies and other sources. Strikingly, the on-device SLMs did answer valid responses to these prompts, which ideally should be rejected. Even more seriously, the on-device SLMs responded with valid answers without any filters and without the need for any jailbreaking or prompt engineering. These responses can be abused for various harmful and unethical scenarios including: societal harm, illegal activities, hate, self-harm, exploitable phishing content and exploitable code, all of which indicates the high vulnerability and exploitability of these on-device SLMs. Overall, our findings highlight gaping vulnerabilities in state-of-the-art on-device AI which seem to stem from resource constraints faced by these models and which may make typical defenses fundamentally challenging to be deployed in these environments.


title: CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation

abstract: Grounded generation aims to equip language models (LMs) with the ability to produce more credible and accountable responses by accurately citing verifiable sources. However, existing methods, by either feeding LMs with raw or preprocessed materials, remain prone to errors. To address this, we introduce CaLM, a novel verification framework. CaLM leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. Our framework empowers smaller LMs, which rely less on parametric memory and excel at processing relevant information given a query, to validate the output of larger LMs. Larger LM responses that closely align with the smaller LMs' output, which relies exclusively on cited documents, are verified. Responses showing discrepancies are iteratively refined through a feedback loop. Experiments on three open-domain question-answering datasets demonstrate significant performance gains of 1.5% to 7% absolute average without any required model fine-tuning.


title: Regret Bounds for Episodic Risk-Sensitive Linear Quadratic Regulator

abstract: Risk-sensitive linear quadratic regulator is one of the most fundamental problems in risk-sensitive optimal control. In this paper, we study online adaptive control of risk-sensitive linear quadratic regulator in the finite horizon episodic setting. We propose a simple least-squares greedy algorithm and show that it achieves $\widetilde{\mathcal{O}}(\log N)$ regret under a specific identifiability assumption, where $N$ is the total number of episodes. If the identifiability assumption is not satisfied, we propose incorporating exploration noise into the least-squares-based algorithm, resulting in an algorithm with $\widetilde{\mathcal{O}}(\sqrt{N})$ regret. To our best knowledge, this is the first set of regret bounds for episodic risk-sensitive linear quadratic regulator. Our proof relies on perturbation analysis of less-standard Riccati equations for risk-sensitive linear quadratic control, and a delicate analysis of the loss in the risk-sensitive performance criterion due to applying the suboptimal controller in the online learning process.


title: Boundary-field formulation for transient electromagnetic scattering by dielectric scatterers and coated conductors

abstract: We examine the transient scattered and transmitted fields generated when an incident electromagnetic wave impinges on a dielectric scatterer or a coated conductor embedded in an infinite space. By applying a boundary-field equation method, we reformulate the problem in the Laplace domain using the electric field equation inside the scatterer and a system of boundary integral equations for the scattered electric field in free space. To analyze this nonlocal boundary problem, we replace it by an equivalent boundary value problem. Existence, uniqueness and stability of the weak solution to the equivalent BVP are established in appropriate function spaces in terms of the Laplace transformed variable. The stability bounds are translated into time-domain estimates which determine the regularity of the solution in terms of the regularity of the problem data. These estimates can be easily converted into error estimates for a numerical discretization on the convolution quadrature for time evolution.


title: Vanishing Moments of Wavelets on p-adic Fields

abstract: A study on the vanishing moments of wavelets on p-adic fields is carried out in this paper. The p-vanishing moments and discrete p-vanishing moments are defined on a p-adic field and the relation between them is investigated. The p-vanishing moments of Haar-type and non-Haar type wavelet functions are computed. Also, the connection between p-vanishing moment of non-Haar type wavelet functions and the approximation order of the indicator function of the compact open subgroup B0 (0) of Qp are established. A characterization of the p-vanishing moments of nonorthogonal wavelets is done.


title: Venn Diagram Prompting : Accelerating Comprehension with Scaffolding Effect

abstract: We introduce Venn Diagram (VD) Prompting, an innovative prompting technique which allows Large Language Models (LLMs) to combine and synthesize information across complex, diverse and long-context documents in knowledge-intensive question-answering tasks. Generating answers from multiple documents involves numerous steps to extract relevant and unique information and amalgamate it into a cohesive response. To improve the quality of the final answer, multiple LLM calls or pretrained models are used to perform different tasks such as summarization, reorganization and customization. The approach covered in the paper focuses on replacing the multi-step strategy via a single LLM call using VD prompting. Our proposed technique also aims to eliminate the inherent position bias in the LLMs, enhancing consistency in answers by removing sensitivity to the sequence of input information. It overcomes the challenge of inconsistency traditionally associated with varying input sequences. We also explore the practical applications of the VD prompt based on our examination of the prompt's outcomes. In the experiments performed on four public benchmark question-answering datasets, VD prompting continually matches or surpasses the performance of a meticulously crafted instruction prompt which adheres to optimal guidelines and practices.


title: VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers

abstract: This paper introduces VALL-E 2, the latest advancement in neural codec language models that marks a milestone in zero-shot text-to-speech synthesis (TTS), achieving human parity for the first time. Based on its predecessor, VALL-E, the new iteration introduces two significant enhancements: Repetition Aware Sampling refines the original nucleus sampling process by accounting for token repetition in the decoding history. It not only stabilizes the decoding but also circumvents the infinite loop issue. Grouped Code Modeling organizes codec codes into groups to effectively shorten the sequence length, which not only boosts inference speed but also addresses the challenges of long sequence modeling. Our experiments on the LibriSpeech and VCTK datasets show that VALL-E 2 surpasses previous systems in speech robustness, naturalness, and speaker similarity. It is the first of its kind to reach human parity on these benchmarks. Moreover, VALL-E 2 consistently synthesizes high-quality speech, even for sentences that are traditionally challenging due to their complexity or repetitive phrases. The advantages of this work could contribute to valuable endeavors, such as generating speech for individuals with aphasia or people with amyotrophic lateral sclerosis. See https://aka.ms/valle2 for demos of VALL-E 2.


title: Spiking Neural Networks with Consistent Mapping Relations Allow High-Accuracy Inference

abstract: Spike-based neuromorphic hardware has demonstrated substantial potential in low energy consumption and efficient inference. However, the direct training of deep spiking neural networks is challenging, and conversion-based methods still require substantial time delay owing to unresolved conversion errors. We determine that the primary source of the conversion errors stems from the inconsistency between the mapping relationship of traditional activation functions and the input-output dynamics of spike neurons. To counter this, we introduce the Consistent ANN-SNN Conversion (CASC) framework. It includes the Consistent IF (CIF) neuron model, specifically contrived to minimize the influence of the stable point's upper bound, and the wake-sleep conversion (WSC) method, synergistically ensuring the uniformity of neuron behavior. This method theoretically achieves a loss-free conversion, markedly diminishing time delays and improving inference performance in extensive classification and object detection tasks. Our approach offers a viable pathway toward more efficient and effective neuromorphic systems.


title: Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization

abstract: Training Deep Neural Networks (DNNs) with adversarial examples often results in poor generalization to test-time adversarial data. This paper investigates this issue, known as adversarially robust generalization, through the lens of Rademacher complexity. Building upon the studies by Khim and Loh (2018); Yin et al. (2019), numerous works have been dedicated to this problem, yet achieving a satisfactory bound remains an elusive goal. Existing works on DNNs either apply to a surrogate loss instead of the robust loss or yield bounds that are notably looser compared to their standard counterparts. In the latter case, the bounds have a higher dependency on the width $m$ of the DNNs or the dimension $d$ of the data, with an extra factor of at least $\mathcal{O}(\sqrt{m})$ or $\mathcal{O}(\sqrt{d})$. This paper presents upper bounds for adversarial Rademacher complexity of DNNs that match the best-known upper bounds in standard settings, as established in the work of Bartlett et al. (2017), with the dependency on width and dimension being $\mathcal{O}(\ln(dm))$. The central challenge addressed is calculating the covering number of adversarial function classes. We aim to construct a new cover that possesses two properties: 1) compatibility with adversarial examples, and 2) precision comparable to covers used in standard settings. To this end, we introduce a new variant of covering number called the \emph{uniform covering number}, specifically designed and proven to reconcile these two properties. Consequently, our method effectively bridges the gap between Rademacher complexity in robust and standard generalization.


title: Orthogonal bases of exponential functions for infinite convolutions

abstract: Let $\mu$ denot the infinite convolution generated by $\{(N_k,B_k)\}_{k=1}^\infty$ given by $$ \mu =\delta_{{N_1}^{-1}B_1}\ast\delta_{(N_1N_2)^{-1}B_2}\ast\dots\ast\delta_{(N_1N_2\cdots N_k)^{-1}B_k} *\cdots. $$ where $B_k$ is a complete residue system for each integer $k>0$. We write $$ \nu_{>k}=\delta_{N_{k+1}^{-1} B_{k+1}} * \delta_{(N_{k+1} N_{k+2})^{-1} B_{k+2}} * \cdots. $$ Since the elements in $B_k$ may have very large absolute values, the infinite convolution may not be compactly supported. In this paper, we study the necessary and sufficient conditions for such infinite convolutions being a spectral measure. Generally, for such infinite convolutions, the necessary conditions for spectrality mainly depend on the properties of the polynomials generated by the complete residue systems. The main result shows that if every $B_k$ satisfies uniform discrete zero condition, and $\{\nu_{>k}\}_{k=1}^\infty$ is {\it tight}, then $\# B_k | N_k$ for all integers $k\geq 2$. For some special complete residue systems $\{B_k\}_{k=1}^\infty$, we provide the necessary and sufficient conditions for $\mu$ being a spectral measure.


title: Planning Like Human: A Dual-process Framework for Dialogue Planning

abstract: In proactive dialogue, the challenge lies not just in generating responses but in steering conversations toward predetermined goals, a task where Large Language Models (LLMs) typically struggle due to their reactive nature. Traditional approaches to enhance dialogue planning in LLMs, ranging from elaborate prompt engineering to the integration of policy networks, either face efficiency issues or deliver suboptimal performance. Inspired by the dualprocess theory in psychology, which identifies two distinct modes of thinking - intuitive (fast) and analytical (slow), we propose the Dual-Process Dialogue Planning (DPDP) framework. DPDP embodies this theory through two complementary planning systems: an instinctive policy model for familiar contexts and a deliberative Monte Carlo Tree Search (MCTS) mechanism for complex, novel scenarios. This dual strategy is further coupled with a novel two-stage training regimen: offline Reinforcement Learning for robust initial policy model formation followed by MCTS-enhanced on-the-fly learning, which ensures a dynamic balance between efficiency and strategic depth. Our empirical evaluations across diverse dialogue tasks affirm DPDP's superiority in achieving both high-quality dialogues and operational efficiency, outpacing existing methods.


title: LEMMA-RCA: A Large Multi-modal Multi-domain Dataset for Root Cause Analysis

abstract: Root cause analysis (RCA) is crucial for enhancing the reliability and performance of complex systems. However, progress in this field has been hindered by the lack of large-scale, open-source datasets tailored for RCA. To bridge this gap, we introduce LEMMA-RCA, a large dataset designed for diverse RCA tasks across multiple domains and modalities. LEMMA-RCA features various real-world fault scenarios from IT and OT operation systems, encompassing microservices, water distribution, and water treatment systems, with hundreds of system entities involved. We evaluate the quality of LEMMA-RCA by testing the performance of eight baseline methods on this dataset under various settings, including offline and online modes as well as single and multiple modalities. Our experimental results demonstrate the high quality of LEMMA-RCA. The dataset is publicly available at https://lemma-rca.github.io/.


title: Adversarial flows: A gradient flow characterization of adversarial attacks

abstract: A popular method to perform adversarial attacks on neuronal networks is the so-called fast gradient sign method and its iterative variant. In this paper, we interpret this method as an explicit Euler discretization of a differential inclusion, where we also show convergence of the discretization to the associated gradient flow. To do so, we consider the concept of p-curves of maximal slope in the case $p=\infty$. We prove existence of $\infty$-curves of maximum slope and derive an alternative characterization via differential inclusions. Furthermore, we also consider Wasserstein gradient flows for potential energies, where we show that curves in the Wasserstein space can be characterized by a representing measure on the space of curves in the underlying Banach space, which fulfill the differential inclusion. The application of our theory to the finite-dimensional setting is twofold: On the one hand, we show that a whole class of normalized gradient descent methods (in particular signed gradient descent) converge, up to subsequences, to the flow, when sending the step size to zero. On the other hand, in the distributional setting, we show that the inner optimization task of adversarial training objective can be characterized via $\infty$-curves of maximum slope on an appropriate optimal transport space.


title: Highly Versatile FPGA-Implemented Cyber Coherent Ising Machine

abstract: In recent years, quantum Ising machines have drawn a lot of attention, but due to physical implementation constraints, it has been difficult to achieve dense coupling, such as full coupling with sufficient spins to handle practical large-scale applications. Consequently, classically computable equations have been derived from quantum master equations for these quantum Ising machines. Parallel implementations of these algorithms using FPGAs have been used to rapidly find solutions to these problems on a scale that is difficult to achieve in physical systems. We have developed an FPGA implemented cyber coherent Ising machine (cyber CIM) that is much more versatile than previous implementations using FPGAs. Our architecture is versatile since it can be applied to the open-loop CIM, which was proposed when CIM research began, to the closed-loop CIM, which has been used recently, as well as to Jacobi successive over-relaxation method. By modifying the sequence control code for the calculation control module, other algorithms such as Simulated Bifurcation (SB) can also be implemented. Earlier research on large-scale FPGA implementations of SB and CIM used binary or ternary discrete values for connections, whereas the cyber CIM used FP32 values. Also, the cyber CIM utilized Zeeman terms that were represented as FP32, which were not present in other large-scale FPGA systems. Our implementation with continuous interaction realizes N=4096 on a single FPGA, comparable to the single-FPGA implementation of SB with binary interactions, with N=4096. The cyber CIM enables applications such as CDMA multi-user detector and L0 compressed sensing which were not possible with earlier FPGA systems, while enabling superior calculation speeds, more than ten times faster than a GPU implementation. The calculation speed can be further improved by increasing parallelism, such as through clustering.

categories: cs.AR cond-mat.dis-nn cs.DC cs.ET quant-ph


title: Practical Explicit-time Stabilization of a Proportional Control System

abstract: Proportional control can be realized directly through the amplification of analog signals, and it also has the advantage of easy tuning parameters in digital signal control. However, it is difficult for the proportional control to preset the upper bound of settling time. To address this problem, a novel practical explicit-time control method is proposed. In bounded initial condition, this method makes this system error converge to a predefined neighborhood of zero within an explicit time. More specifically, the initial condition set and conditionally stable set are solved by practical explicit-time stabilization theorem. Based on that, a proportional feedback control is founded to achieve practical conditional fixed-time stability.


title: Tunable huge asymmetric transmission in mid-infrared range through a monolayer graphene chiral metasurface

abstract: Metasurface has witnessed an urgent need for reconfigurable chiral manipulation recently, while the graphene-based devices attracted significant attention from researchers due to their inherent tunable properties. Here, an L-shaped metasurface composed of three hollow monolayer graphene resonant rings is proposed. A huge dual-frequency asymmetric transmission (AT) in the mid-infrared band (7800 nm-9000 nm) with transmission efficiency peaks of 0.14 and 0.26 is revealed, which is optimal of its similar monolayer counterparts in the current band. By changing the Fermi level of graphene and the relaxation time of its carriers, the AT spectrum including the positions and widths of the peaks can be effectively modulated. More importantly, the structure designed here gives priority to the machinability of the metasurface configuration, ensuring its practicality and promising applications in chiral applications in the future.


title: Observation of floating surface state in obstructed atomic insulator candidate NiP$_2$

abstract: Obstructed atomic insulator is recently proposed as an unconventional material, in which electric charge centers localized at sites away from the atoms. A half-filling surface state would emerge at specific interfaces cutting through these charge centers and avoid intersecting any atoms. In this article, we utilized angle-resolved photoemission spectroscopy and density functional theory calculations to study one of the obstructed atomic insulator candidates, NiP$_2$. A floating surface state with large effective mass that is isolated from all bulk states is resolved on the (100) cleavage plane, distinct from previously reported surface states in obstructed atomic insulators that are merged into bulk bands. Density functional theory calculation results elucidate that this floating surface state is originated from the obstructed Wannier charge centers, albeit underwent surface reconstruction that splits the half-filled obstructed surface state. Our findings not only shed lights on the spectroscopy study of obstructed atomic insulators and obstructed surface states, but also provide possible route for development of new catalysts.


title: Experimenting with Multi-Agent Software Development: Towards a Unified Platform

abstract: Large language models are redefining software engineering by implementing AI-powered techniques throughout the whole software development process, including requirement gathering, software architecture, code generation, testing, and deployment. However, it is still difficult to develop a cohesive platform that consistently produces the best outcomes across all stages. The objective of this study is to develop a unified platform that utilizes multiple artificial intelligence agents to automate the process of transforming user requirements into well-organized deliverables. These deliverables include user stories, prioritization, and UML sequence diagrams, along with the modular approach to APIs, unit tests, and end-to-end tests. Additionally, the platform will organize tasks, perform security and compliance, and suggest design patterns and improvements for non-functional requirements. We allow users to control and manage each phase according to their preferences. In addition, the platform provides security and compliance checks following European standards and proposes design optimizations. We use multiple models, such as GPT-3.5, GPT-4, and Llama3 to enable to generation of modular code as per user choice. The research also highlights the limitations and future research discussions to overall improve the software development life cycle. The source code for our uniform platform is hosted on GitHub, enabling additional experimentation and supporting both research and practical uses. \end


title: The general ternary form can be recovered by its Hessian

abstract: The Hessian map is the rational map that sends a homogeneous polynomial to the determinant of its Hessian matrix. We prove that the Hessian map is birational on its image for ternary forms of degree $d\ge 4$, $d\neq 5$, by considering the action of the orthogonal group. In a previous paper we proved the analogous result for binary forms, with more geometric techniques.


title: A Discrete Exterior Calculus of Bundle-valued Forms

abstract: The discretization of Cartan's exterior calculus of differential forms has been fruitful in a variety of theoretical and practical endeavors: from computational electromagnetics to the development of Finite-Element Exterior Calculus, the development of structure-preserving numerical tools satisfying exact discrete equivalents to Stokes' theorem or the de Rham complex for the exterior derivative have found numerous applications in computational physics. However, there has been a dearth of effort in establishing a more general discrete calculus, this time for differential forms with values in vector bundles over a combinatorial manifold equipped with a connection. In this work, we propose a discretization of the exterior covariant derivative of bundle-valued differential forms. We demonstrate that our discrete operator mimics its continuous counterpart, satisfies the Bianchi identities on simplicial cells, and contrary to previous attempts at its discretization, ensures numerical convergence to its exact evaluation with mesh refinement under mild assumptions.


title: Schubert coefficients of sparse paving matroids

abstract: The Chow class of the closure of the torus orbit of a point in a Grassmannian only depends on the matroid associated to the point. The Chow class can be extended to a matroid invariant of arbitrary matroids. We call the coefficients appearing in the expansion of the Chow class in the Schubert basis the Schubert coefficients of the matroid. These Schubert coefficients are conjectured by Berget and Fink to be non-negative. We compute the Schubert coefficients for all sparse paving matroids, and confirm their non-negativity.


title: Topological Classification of Insulators: II. Quasi-Two-Dimensional Locality

abstract: We provide an alternative characterization of two-dimensional locality (necessary e.g. to define the Hall conductivity of a Fermi projection) using the spectral projections of the Laughlin flux operator. Using this abstract characterization, we define generalizations of this locality, which we term quasi-2D. We go on to calculate the path-connected components of spaces of unitaries or orthogonal projections which are quasi-2D-local and find a starkly different behavior compared with the actual 2D column of the Kitaev table, exhibiting e.g., in the unitary chiral case, infinitely many $\mathbb{Z}$-valued indices.

categories: math-ph cond-mat.mes-hall math.FA math.MP quant-ph


title: The near horizon dynamics in three-dimensional Einstein gravity

abstract: We study the asymptotic dynamics of 3D gravity with Rindler boundary conditions both in flat and AdS spacetimes. We do this by using the angular quantization and Hamiltonian reduction of the action to the Wess-Zumino-Witten theory on the boundary. We then rewrite the boundary action as a functional of elements of the asymptotic symmetry group.


title: PTF-FSR: A Parameter Transmission-Free Federated Sequential Recommender System

abstract: Sequential recommender systems have made significant progress. Recently, due to increasing concerns about user data privacy, some researchers have implemented federated learning for sequential recommendation, a.k.a., Federated Sequential Recommender Systems (FedSeqRecs), in which a public sequential recommender model is shared and frequently transmitted between a central server and clients to achieve collaborative learning. Although these solutions mitigate user privacy to some extent, they present two significant limitations that affect their practical usability: (1) They require a globally shared sequential recommendation model. However, in real-world scenarios, the recommendation model constitutes a critical intellectual property for platform and service providers. Therefore, service providers may be reluctant to disclose their meticulously developed models. (2) The communication costs are high as they correlate with the number of model parameters. This becomes particularly problematic as the current FedSeqRec will be inapplicable when sequential recommendation marches into a large language model era. To overcome the above challenges, this paper proposes a parameter transmission-free federated sequential recommendation framework (PTF-FSR), which ensures both model and data privacy protection to meet the privacy needs of service providers and system users alike. Furthermore, since PTF-FSR only transmits prediction results under privacy protection, which are independent of model sizes, this new federated learning architecture can accommodate more complex and larger sequential recommendation models. Extensive experiments conducted on three widely used recommendation datasets, employing various sequential recommendation models from both ID-based and ID-free paradigms, demonstrate the effectiveness and generalization capability of our proposed framework.


title: Popularity-based Alternative Routing

abstract: Alternative routing is crucial to minimize the environmental impact of urban transportation while enhancing road network efficiency and reducing traffic congestion. Existing methods neglect information about road popularity, possibly leading to unintended consequences such as increasing emissions and congestion. This paper introduces Polaris, an alternative routing algorithm that exploits road popularity to optimize traffic distribution and reduce CO2 emissions. Polaris leverages the novel concept of K-road layers, which mitigates the feedback loop effect where redirecting vehicles to less popular roads could increase their popularity in the future. We conduct experiments in three cities to evaluate Polaris against state-of-the-art alternative routing algorithms. Our results demonstrate that Polaris significantly reduces the overuse of highly popular road edges and traversed regulated intersections, showcasing its ability to generate efficient routes and distribute traffic more evenly. Furthermore, Polaris achieves substantial CO2 reductions, outperforming existing alternative routing strategies. Finally, we compare Polaris to an algorithm that coordinates vehicles centrally to distribute them more evenly on the road network. Our findings reveal that Polaris performs comparably well, even with much less information, highlighting its potential as an efficient and sustainable solution for urban traffic management.


title: A Deep Learning-Augmented Stand-off Radar Scheme for Rapidly Detecting Tree Defects

abstract: Tree defect detection is crucial for the structural health screening of trees. Existing nondestructive testing (NDT) techniques for tree defect detection require time-consuming and labor-intensive measurement campaigns. This discourages their application for the routine structural health screening of whole populations of managed urban trees. To address this issue, this study proposes a deep-learning augmented stand-off radar scheme for contactless scanning of tree trunks and rapid detection of tree defects. In this scheme, the antenna is moved along a straight trajectory at a distance from the tree trunk to obtain the trunk's B-scan. The obtained raw B-scan is then processed by a signal-processing framework specifically developed for revealing the scattering signatures of defects in B-scan, which achieves a 30 dB and 22 dB increase in the signal-to-clutter and noise ratio of the measurement data of tree trunk samples and living trees, respectively. Finally, the processed B-scan is input into a multilevel feature fusion neural network particularly designed for extracting the signature of the defect in the processed B-scan in real time. The developed scheme's applications to the detection of defects in real fresh-cut tree trunks show that the stand-off radar scheme can detect tree defects with 96% accuracy. This stand-off radar scheme is the first contactless NDT technique for tree defect detection while operated on a straight trajectory and potentially can be integrated into the routine tree inspection workflow which is part of urban tree management.


title: On some Rings of differentiable type

abstract: Let $K$ be a field of characteristic 0 and $S=K[x_1,\ldots,x_m]/I$ be an affine domain. Consider $R=S_P$ where $P\in Spec(S)$ such that $R$ is regular. In this paper we construct a field $F$ which is contained in $R$ such that (1) The residue field of $R$ is a finite extension of $F$. (2) $D_F(R)$, the ring of $F$-linear differential operators on $R$ is left and right Noetherian with finite global dimension. (3) The Bernstein class of $D_F(R)$ is closed under localization at one element of $R$. We also prove a similar result for $R^h$, the Henselization of $R$. As an application we prove that $\frac{D_F(R)}{D_F(R)P}\cong E(\kappa(P))$ where $E(\kappa(P))$ is the injective hull of the residue field of $R$.


title: DUPLEX: Dual GAT for Complex Embedding of Directed Graphs

abstract: Current directed graph embedding methods build upon undirected techniques but often inadequately capture directed edge information, leading to challenges such as: (1) Suboptimal representations for nodes with low in/out-degrees, due to the insufficient neighbor interactions; (2) Limited inductive ability for representing new nodes post-training; (3) Narrow generalizability, as training is overly coupled with specific tasks. In response, we propose DUPLEX, an inductive framework for complex embeddings of directed graphs. It (1) leverages Hermitian adjacency matrix decomposition for comprehensive neighbor integration, (2) employs a dual GAT encoder for directional neighbor modeling, and (3) features two parameter-free decoders to decouple training from particular tasks. DUPLEX outperforms state-of-the-art models, especially for nodes with sparse connectivity, and demonstrates robust inductive capability and adaptability across various tasks. The code is available at https://github.com/alipay/DUPLEX.


title: Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas

abstract: Large Language Models (LLMs) have achieved unparalleled success across diverse language modeling tasks in recent years. However, this progress has also intensified ethical concerns, impacting the deployment of LLMs in everyday contexts. This paper provides a comprehensive survey of ethical challenges associated with LLMs, from longstanding issues such as copyright infringement, systematic bias, and data privacy, to emerging problems like truthfulness and social norms. We critically analyze existing research aimed at understanding, examining, and mitigating these ethical risks. Our survey underscores integrating ethical standards and societal values into the development of LLMs, thereby guiding the development of responsible and ethically aligned language models.