title: A Labeled Array Distance Metric for Measuring Image Segmentation Quality

abstract: This work introduces two new distance metrics for comparing labeled arrays, which are common outputs of image segmentation algorithms. Each pixel in an image is assigned a label, with binary segmentation providing only two labels ('foreground' and 'background'). These can be represented by a simple binary matrix and compared using pixel differences. However, many segmentation algorithms output multiple regions in a labeled array. We propose two distance metrics, named LAD and MADLAD, that calculate the distance between two labeled images. By doing so, the accuracy of different image segmentation algorithms can be evaluated by measuring their outputs against a 'ground truth' labeling. Both proposed metrics, operating with a complexity of $O(N)$ for images with $N$ pixels, are designed to quickly identify similar labeled arrays, even when different labeling methods are used. Comparisons are made between images labeled manually and those labeled by segmentation algorithms. This evaluation is crucial when searching through a space of segmentation algorithms and their hyperparameters via a genetic algorithm to identify the optimal solution for automated segmentation, which is the goal in our lab, SEE-Insight. By measuring the distance from the ground truth, these metrics help determine which algorithm provides the most accurate segmentation.


title: DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition

abstract: In this paper, we address the problem of plausible object placement for the challenging task of realistic image composition. We propose DiffPop, the first framework that utilizes plausibility-guided denoising diffusion probabilistic model to learn the scale and spatial relations among multiple objects and the corresponding scene image. First, we train an unguided diffusion model to directly learn the object placement parameters in a self-supervised manner. Then, we develop a human-in-the-loop pipeline which exploits human labeling on the diffusion-generated composite images to provide the weak supervision for training a structural plausibility classifier. The classifier is further used to guide the diffusion sampling process towards generating the plausible object placement. Experimental results verify the superiority of our method for producing plausible and diverse composite images on the new Cityscapes-OP dataset and the public OPA dataset, as well as demonstrate its potential in applications such as data augmentation and multi-object placement tasks. Our dataset and code will be released.


title: Targeted marine cloud brightening can dampen El Ni\~no

abstract: Many record-breaking climate extremes arise from both greenhouse gas-induced warming and natural climate variability. Marine cloud brightening, a solar geoengineering strategy originally proposed to reduce long-term warming, could potentially mitigate extreme events by instead targeting seasonal phenomena, such as El Ni\~no-Southern Oscillation (ENSO). By exploiting the 2019-2020 Australian wildfire experiment-of-opportunity, we show that simulated marine cloud brightening in the southeast Pacific reproduces observed cloud changes and induces La Ni\~na-like responses. We then explore how cloud brightening timing and duration modifies the 1997-1998 and 2015-2016 El Ni\~no events. We find the earliest and longest interventions effectively restore neutral ENSO conditions and dampen El Ni\~no's impacts. Solar geoengineering that targets climate variability could complement tools such as ENSO forecasting and provide a pathway for climate risk mitigation.


title: Zero-Shot Fake Video Detection by Audio-Visual Consistency

abstract: Recent studies have advocated the detection of fake videos as a one-class detection task, predicated on the hypothesis that the consistency between audio and visual modalities of genuine data is more significant than that of fake data. This methodology, which solely relies on genuine audio-visual data while negating the need for forged counterparts, is thus delineated as a `zero-shot' detection paradigm. This paper introduces a novel zero-shot detection approach anchored in content consistency across audio and video. By employing pre-trained ASR and VSR models, we recognize the audio and video content sequences, respectively. Then, the edit distance between the two sequences is computed to assess whether the claimed video is genuine. Experimental results indicate that, compared to two mainstream approaches based on semantic consistency and temporal consistency, our approach achieves superior generalizability across various deepfake techniques and demonstrates strong robustness against audio-visual perturbations. Finally, state-of-the-art performance gains can be achieved by simply integrating the decision scores of these three systems.


title: VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment

abstract: With the help of discrete neural audio codecs, large language models (LLM) have increasingly been recognized as a promising methodology for zero-shot Text-to-Speech (TTS) synthesis. However, sampling based decoding strategies bring astonishing diversity to generation, but also pose robustness issues such as typos, omissions and repetition. In addition, the high sampling rate of audio also brings huge computational overhead to the inference process of autoregression. To address these issues, we propose VALL-E R, a robust and efficient zero-shot TTS system, building upon the foundation of VALL-E. Specifically, we introduce a phoneme monotonic alignment strategy to strengthen the connection between phonemes and acoustic sequence, ensuring a more precise alignment by constraining the acoustic tokens to match their associated phonemes. Furthermore, we employ a codec-merging approach to downsample the discrete codes in shallow quantization layer, thereby accelerating the decoding speed while preserving the high quality of speech output. Benefiting from these strategies, VALL-E R obtains controllablity over phonemes and demonstrates its strong robustness by approaching the WER of ground truth. In addition, it requires fewer autoregressive steps, with over 60% time reduction during inference. This research has the potential to be applied to meaningful projects, including the creation of speech for those affected by aphasia. Audio samples will be available at: https://aka.ms/valler.


title: Learning dynamical behaviors in physical systems

abstract: Physical learning is an emerging paradigm in science and engineering whereby (meta)materials acquire desired macroscopic behaviors by exposure to examples. So far, it has been applied to static properties such as elastic moduli and self-assembled structures encoded in minima of an energy landscape. Here, we extend this paradigm to dynamic functionalities, such as motion and shape change, that are instead encoded in limit cycles or pathways of a dynamical system. We identify the two ingredients needed to learn time-dependent behaviors irrespective of experimental platforms: (i) learning rules with time delays and (ii) exposure to examples that break time-reversal symmetry during training. After providing a hands-on demonstration of these requirements using programmable LEGO toys, we turn to realistic particle-based simulations where the training rules are not programmed on a computer. Instead, we elucidate how they emerge from physico-chemical processes involving the causal propagation of fields, like in recent experiments on moving oil droplets with chemotactic signalling. Our trainable particles can self-assemble into structures that move or change shape on demand, either by retrieving the dynamic behavior previously seen during training, or by learning on the fly. This rich phenomenology is captured by a modified Hopfield spin model amenable to analytical treatment. The principles illustrated here provide a step towards von Neumann's dream of engineering synthetic living systems that adapt to the environment.


title: Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges

abstract: This article presents a digital twin (DT)-enhanced reinforcement learning (RL) framework aimed at optimizing performance and reliability in network resource management, since the traditional RL methods face several unified challenges when applied to physical networks, including limited exploration efficiency, slow convergence, poor long-term performance, and safety concerns during the exploration phase. To deal with the above challenges, a comprehensive DT-based framework is proposed to enhance the convergence speed and performance for unified RL-based resource management. The proposed framework provides safe action exploration, more accurate estimates of long-term returns, faster training convergence, higher convergence performance, and real-time adaptation to varying network conditions. Then, two case studies on ultra-reliable and low-latency communication (URLLC) services and multiple unmanned aerial vehicles (UAV) network are presented, demonstrating improvements of the proposed framework in performance, convergence speed, and training cost reduction both on traditional RL and neural network based Deep RL (DRL). Finally, the article identifies and explores some of the research challenges and open issues in this rapidly evolving field.


title: Random Combinatorial Billiards and Stoned Exclusion Processes

abstract: We introduce and study several random combinatorial billiard trajectories. Such a system, which depends on a fixed parameter $p\in(0,1)$, models a beam of light that travels in a Euclidean space, occasionally randomly reflecting off of a hyperplane in the Coxeter arrangement of an affine Weyl group with some probability that depends on the side of the hyperplane that it hits. In one case, we (essentially) recover Lam's reduced random walk in the limit as $p$ tends to $0$. The investigation of our random billiard trajectories relies on an analysis of new finite Markov chains that we call stoned exclusion processes. These processes have remarkable stationary distributions determined by well-studied polynomials such as ASEP polynomials, inhomogeneous TASEP polynomials, and open boundary ASEP polynomials; in many cases, it was previously not known how to construct Markov chains with these stationary distributions. Using multiline queues, we analyze correlations in the stoned multispecies TASEP, allowing us to determine limit directions for reduced random billiard trajectories and limit shapes for new random growth processes for $n$-core partitions. Our perspective coming from combinatorial billiards naturally leads us to formulate a new variant of the ASEP on $\mathbb{Z}$ called the scan ASEP, which we deem interesting in its own right.


title: Relations between monotone complexity measures based on decision tree complexity

abstract: In a recent result, Knop, Lovett, McGuire and Yuan (STOC 2021) proved the log-rank conjecture for communication complexity, up to log n factor, for any Boolean function composed with AND function as the inner gadget. One of the main tools in this result was the relationship between monotone analogues of well-studied Boolean complexity measures like block sensitivity and certificate complexity. The relationship between the standard measures has been a long line of research, with a landmark result by Huang (Annals of Mathematics 2019), finally showing that sensitivity is polynomially related to all other standard measures. In this article, we study the monotone analogues of standard measures like block sensitivity (mbs(f)), certificate complexity (MCC(f)) and fractional block sensitivity (fmbs(f)); and study the relationship between these measures given their connection with AND-decision tree and sparsity of a Boolean function. We show the following results: 1) Given a Boolean function $f : \{0, 1\}^{n} \rightarrow \{0, 1\}$, the ratio $fmbs(f^l )/mbs(f^l )$ is bounded by a function of n (and not l). A similar result was known for the corresponding standard measures (Tal, ITCS 2013). This result allows us to extend any upper bound by a well behaved measure on monotone block sensitivity to monotone fractional block sensitivity. 2) The question of the best possible upper bound on monotone block sensitivity by the logarithm of sparsity is equivalent to the natural question of best upper bound by degree on sensitivity. One side of this relationship was used in the proof by Knop, Lovett, McGuire and Yuan (STOC 2021). 3) For two natural classes of functions, symmetric and monotone, hitting set complexity (MCC) is equal to monotone sensitivity.


title: BookSQL: A Large Scale Text-to-SQL Dataset for Accounting Domain

abstract: Several large-scale datasets (e.g., WikiSQL, Spider) for developing natural language interfaces to databases have recently been proposed. These datasets cover a wide breadth of domains but fall short on some essential domains, such as finance and accounting. Given that accounting databases are used worldwide, particularly by non-technical people, there is an imminent need to develop models that could help extract information from accounting databases via natural language queries. In this resource paper, we aim to fill this gap by proposing a new large-scale Text-to-SQL dataset for the accounting and financial domain: BookSQL. The dataset consists of 100k natural language queries-SQL pairs, and accounting databases of 1 million records. We experiment with and analyze existing state-of-the-art models (including GPT-4) for the Text-to-SQL task on BookSQL. We find significant performance gaps, thus pointing towards developing more focused models for this domain.


title: A Measurement of CO(3-2) Line Emission from eBOSS Galaxies at $z\sim 0.5$ using Planck Data

abstract: Line intensity mapping (LIM) is a novel observational technique in astrophysics that utilizes the integrated emission from multiple atomic and molecular transition lines from galaxies to probe the complex physics of galaxy formation and evolution, as well as the large-scale structure of the universe. Modeling multiple line luminosities of galaxies with varying masses or their host halo masses poses significant uncertainty due to the lack of observational data across a wide redshift range and the intricate nature of astrophysical processes, making them challenging to model analytically or in simulations. While future experiments aim to measure multiple line intensities up to $z\sim 8$ across a wide volume using tomographic methods, we leverage publicly available datasets from the CMB experiment Planck and the galaxy survey eBOSS to constrain the CO(3-2) emission from galaxies. We correlate galaxies from eBOSS data onto the full-sky CO(2-1) map produced by Planck and report the first measurement of the average CO(3-2) intensity, $I_{CO} = 45.7 \pm 14.2\, \mathrm{Jy/sr}$ at $z\sim 0.5$ with $3.2\sigma$ confidence. Our findings demonstrate that stacking methods are already viable with existing observations from CMB experiments and galaxy surveys, and are complementary to traditional LIM experiments.


title: Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networks

abstract: Spiking neural networks (SNNs) have attracted considerable attention for their event-driven, low-power characteristics and high biological interpretability. Inspired by knowledge distillation (KD), recent research has improved the performance of the SNN model with a pre-trained teacher model. However, additional teacher models require significant computational resources, and it is tedious to manually define the appropriate teacher network architecture. In this paper, we explore cost-effective self-distillation learning of SNNs to circumvent these concerns. Without an explicit defined teacher, the SNN generates pseudo-labels and learns consistency during training. On the one hand, we extend the timestep of the SNN during training to create an implicit temporal ``teacher" that guides the learning of the original ``student", i.e., the temporal self-distillation. On the other hand, we guide the output of the weak classifier at the intermediate stage by the final output of the SNN, i.e., the spatial self-distillation. Our temporal-spatial self-distillation (TSSD) learning method does not introduce any inference overhead and has excellent generalization ability. Extensive experiments on the static image datasets CIFAR10/100 and ImageNet as well as the neuromorphic datasets CIFAR10-DVS and DVS-Gesture validate the superior performance of the TSSD method. This paper presents a novel manner of fusing SNNs with KD, providing insights into high-performance SNN learning methods.


title: The Size-luminosity Relation of the AGN Torus Determined from the Comparison between Optical and Mid-infrared Variability

abstract: We investigate the optical variability of low-redshift ($0.15< z\leq0.4$) active galactic nuclei using the multi-epoch data from the Zwicky Transient Facility. We find that a damped random walk model well describes the ensemble structure function in the $g$ band. Consistent with previous studies, more luminous active galactic nuclei tend to have a steeper structure function at a timescale less than the break timescale and smaller variability amplitude. By comparing the structure functions in the optical with the mid-infrared obtained from the Wide-field Infrared Survey Explorer, we derive the size of the dusty torus using a toy model for the geometry of the torus. The size of the torus positively correlates with the luminosity of the active nucleus, following a relation that agrees well with previous studies based on reverberation mapping. This result demonstrates that the structure function method can be used as a powerful and highly efficient tool to examine the size of the torus.


title: Weak anti-localization and spin-momentum locking in topological insulator Ta$_2$Ni$_3$Te$_5$

abstract: We report the synthesis, structural characterization, and investigation of electrical transport, magnetic and specific heat properties of bulk semiconducting layered material Ta$_2$Ni$_3$Te$_5$. Ta$_2$Ni$_3$Te$_5$ crystallizes in the centrosymmetric orthorhombic structure with space group Pnma. Temperature-dependent resistivity shows a transition from semiconducting to metallic nature below 7 K. Low-temperature magnetotransport studies show large magnetoresistance with the signature of weak anti-localization (WAL) effect. The magnetoconductivity data has been used to explore the origin of the WAL effect, extract relevant parameters, and study their variation with temperature. The presence of significant electron-phonon interaction is evident from the MR vs. B/R plot (Kohler's plot). Isothermal field-dependent magnetization studies show Berry paramagnetism as the signature for spin-orbit coupling-induced spin-momentum locking. Observation of WAL effect and spin-momentum locking phenomenon demonstrate Ta$_2$Ni$_3$Te$_5$ as a promising low dimensional material for quantum spin Hall insulator-based applications.


title: FaithFill: Faithful Inpainting for Object Completion Using a Single Reference Image

abstract: We present FaithFill, a diffusion-based inpainting object completion approach for realistic generation of missing object parts. Typically, multiple reference images are needed to achieve such realistic generation, otherwise the generation would not faithfully preserve shape, texture, color, and background. In this work, we propose a pipeline that utilizes only a single input reference image -having varying lighting, background, object pose, and/or viewpoint. The singular reference image is used to generate multiple views of the object to be inpainted. We demonstrate that FaithFill produces faithful generation of the object's missing parts, together with background/scene preservation, from a single reference image. This is demonstrated through standard similarity metrics, human judgement, and GPT evaluation. Our results are presented on the DreamBooth dataset, and a novel proposed dataset.


title: Asymptotically Optimal Regret for Black-Box Predict-then-Optimize

abstract: We consider the predict-then-optimize paradigm for decision-making in which a practitioner (1) trains a supervised learning model on historical data of decisions, contexts, and rewards, and then (2) uses the resulting model to make future binary decisions for new contexts by finding the decision that maximizes the model's predicted reward. This approach is common in industry. Past analysis assumes that rewards are observed for all actions for all historical contexts, which is possible only in problems with special structure. Motivated by problems from ads targeting and recommender systems, we study new black-box predict-then-optimize problems that lack this special structure and where we only observe the reward from the action taken. We present a novel loss function, which we call Empirical Soft Regret (ESR), designed to significantly improve reward when used in training compared to classical accuracy-based metrics like mean-squared error. This loss function targets the regret achieved when taking a suboptimal decision; because the regret is generally not differentiable, we propose a differentiable "soft" regret term that allows the use of neural networks and other flexible machine learning models dependent on gradient-based training. In the particular case of paired data, we show theoretically that optimizing our loss function yields asymptotically optimal regret within the class of supervised learning models. We also show our approach significantly outperforms state-of-the-art algorithms on real-world decision-making problems in news recommendation and personalized healthcare compared to benchmark methods from contextual bandits and conditional average treatment effect estimation.


title: Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation

abstract: In this paper, we introduce a novel Face-to-Face spoken dialogue model. It processes audio-visual speech from user input and generates audio-visual speech as the response, marking the initial step towards creating an avatar chatbot system without relying on intermediate text. To this end, we newly introduce MultiDialog, the first large-scale multimodal (i.e., audio and visual) spoken dialogue corpus containing 340 hours of approximately 9,000 dialogues, recorded based on the open domain dialogue dataset, TopicalChat. The MultiDialog contains parallel audio-visual recordings of conversation partners acting according to the given script with emotion annotations, which we expect to open up research opportunities in multimodal synthesis. Our Face-to-Face spoken dialogue model incorporates a textually pretrained large language model and adapts it into the audio-visual spoken dialogue domain by incorporating speech-text joint pretraining. Through extensive experiments, we validate the effectiveness of our model in facilitating a face-to-face conversation. Demo and data are available at https://multidialog.github.io and https://huggingface.co/datasets/IVLLab/MultiDialog, respectively.


title: Bridging multiple worlds: multi-marginal optimal transport for causal partial-identification problem

abstract: Under the prevalent potential outcome model in causal inference, each unit is associated with multiple potential outcomes but at most one of which is observed, leading to many causal quantities being only partially identified. The inherent missing data issue echoes the multi-marginal optimal transport (MOT) problem, where marginal distributions are known, but how the marginals couple to form the joint distribution is unavailable. In this paper, we cast the causal partial identification problem in the framework of MOT with $K$ margins and $d$-dimensional outcomes and obtain the exact partial identified set. In order to estimate the partial identified set via MOT, statistically, we establish a convergence rate of the plug-in MOT estimator for general quadratic objective functions and prove it is minimax optimal for a quadratic objective function stemming from the variance minimization problem with arbitrary $K$ and $d \le 4$. Numerically, we demonstrate the efficacy of our method over several real-world datasets where our proposal consistently outperforms the baseline by a significant margin (over 70%). In addition, we provide efficient off-the-shelf implementations of MOT with general objective functions.


title: Unveiling the Power of Wavelets: A Wavelet-based Kolmogorov-Arnold Network for Hyperspectral Image Classification

abstract: Hyperspectral image classification is a crucial but challenging task due to the high dimensionality and complex spatial-spectral correlations inherent in hyperspectral data. This paper employs Wavelet-based Kolmogorov-Arnold Network (wav-kan) architecture tailored for efficient modeling of these intricate dependencies. Inspired by the Kolmogorov-Arnold representation theorem, Wav-KAN incorporates wavelet functions as learnable activation functions, enabling non-linear mapping of the input spectral signatures. The wavelet-based activation allows Wav-KAN to effectively capture multi-scale spatial and spectral patterns through dilations and translations. Experimental evaluation on three benchmark hyperspectral datasets (Salinas, Pavia, Indian Pines) demonstrates the superior performance of Wav-KAN compared to traditional multilayer perceptrons (MLPs) and the recently proposed Spline-based KAN (Spline-KAN) model. In this work we are: (1) conducting more experiments on additional hyperspectral datasets (Pavia University, WHU-Hi, and Urban Hyperspectral Image) to further validate the generalizability of Wav-KAN; (2) developing a multiresolution Wav-KAN architecture to capture scale-invariant features; (3) analyzing the effect of dimensional reduction techniques on classification performance; (4) exploring optimization methods for tuning the hyperparameters of KAN models; and (5) comparing Wav-KAN with other state-of-the-art models in hyperspectral image classification.


title: Event-Triggered Optimal Tracking Control for Strict-Feedback Nonlinear Systems With Non-Affine Nonlinear Faults

abstract: This article studies the control ideas of the optimal backstepping technique, proposing an event-triggered optimal tracking control scheme for a class of strict-feedback nonlinear systems with non-affine and nonlinear faults. A simplified identifier-critic-actor framework is employed in the reinforcement learning algorithm to achieve optimal control. The identifier estimates the unknown dynamic functions, the critic evaluates the system performance, and the actor implements control actions, enabling modeling and control of anonymous systems for achieving optimal control performance. In this paper, a simplified reinforcement learning algorithm is designed by deriving update rules from the negative gradient of a simple positive function related to the Hamilton-Jacobi-Bellman equation, and it also releases the stringent persistent excitation condition. Then, a fault-tolerant control method is developed by applying filtered signals for controller design. Additionally, to address communication resource reduction, an event-triggered mechanism is employed for designing the actual controller. Finally, the proposed scheme's feasibility is validated through theoretical analysis and simulation.


title: Flexible Music-Conditioned Dance Generation with Style Description Prompts

abstract: Dance plays an important role as an artistic form and expression in human culture, yet the creation of dance remains a challenging task. Most dance generation methods primarily rely solely on music, seldom taking into consideration intrinsic attributes such as music style or genre. In this work, we introduce Flexible Dance Generation with Style Description Prompts (DGSDP), a diffusion-based framework suitable for diversified tasks of dance generation by fully leveraging the semantics of music style. The core component of this framework is Music-Conditioned Style-Aware Diffusion (MCSAD), which comprises a Transformer-based network and a music Style Modulation module. The MCSAD seemly integrates music conditions and style description prompts into the dance generation framework, ensuring that generated dances are consistent with the music content and style. To facilitate flexible dance generation and accommodate different tasks, a spatial-temporal masking strategy is effectively applied in the backward diffusion process. The proposed framework successfully generates realistic dance sequences that are accurately aligned with music for a variety of tasks such as long-term generation, dance in-betweening, dance inpainting, and etc. We hope that this work has the potential to inspire dance generation and creation, with promising applications in entertainment, art, and education.


title: How to accurately measure the mobility and viscosity of two-dimensional carriers?

abstract: Recently, a hydrodynamic behavior of ultrahigh mobility viscous two-dimensional electron gas in GaAs/AlGaAs quantum well was intensively studied based on relevant negative magnetoresistance (NMR) data. The two-dimensional Gurzhi model with(without) simplification combined with extra assumptions on electron-electron collisions rate often lead to a certain agreement with experiment. We provide ab initio analysis of the NMR data[Q. Shi et al, Phys. Rev. B. 89, 201301(R) (2014)] and demonstrate exceptional agreement with the uncut Gurzhi model attracting a single fitting parameter of the effective stream width. Our approach allows one to identify the intrinsic carrier mobility higher than that deduced by conventional methods. The NMR data analysis is proposed for powerful tool for two-dimensional viscometry.


title: Robust 3D Face Alignment with Multi-Path Neural Architecture Search

abstract: 3D face alignment is a very challenging and fundamental problem in computer vision. Existing deep learning-based methods manually design different networks to regress either parameters of a 3D face model or 3D positions of face vertices. However, designing such networks relies on expert knowledge, and these methods often struggle to produce consistent results across various face poses. To address this limitation, we employ Neural Architecture Search (NAS) to automatically discover the optimal architecture for 3D face alignment. We propose a novel Multi-path One-shot Neural Architecture Search (MONAS) framework that leverages multi-scale features and contextual information to enhance face alignment across various poses. The MONAS comprises two key algorithms: Multi-path Networks Unbiased Sampling Based Training and Simulated Annealing based Multi-path One-shot Search. Experimental results on three popular benchmarks demonstrate the superior performance of the MONAS for both sparse alignment and dense alignment.


title: Visual instrument co-design embracing the unique movement capabilities of a dancer with physical disability

abstract: This paper explores the design of an expressive visual instrument that embraces the unique movement style of a dancer living with physical disability. Through a collaboration between the dancer and an interaction designer/visual artist, the creative qualities of wearable devices for motion tracking are investigated, with emphasis on integrating the dancer's specific movement capabilities with their creative goals. The affordances of this technology for imagining new forms of creative expression play a critical role in the design process. These themes are drawn together through an experiential performance which augments an improvised dance with an ephemeral real-time visualisation of the performer's movements. Through practice-based research, the design, development and presentation of this performance work is examined as a 'testbed' for new ideas, allowing for the exploration of HCI concepts within a creative context. This paper outlines the creative process behind the development of the work, the insights derived from the practice-based research enquiry, and the role of movement technology in encouraging new ways of moving through creative expression.


title: Carbon Market Simulation with Adaptive Mechanism Design

abstract: A carbon market is a market-based tool that incentivizes economic agents to align individual profits with the global utility, i.e., reducing carbon emissions to tackle climate change. Cap and trade stands as a critical principle based on allocating and trading carbon allowances (carbon emission credit), enabling economic agents to follow planned emissions and penalizing excess emissions. A central authority is responsible for introducing and allocating those allowances in cap and trade. However, the complexity of carbon market dynamics makes accurate simulation intractable, which in turn hinders the design of effective allocation strategies. To address this, we propose an adaptive mechanism design framework, simulating the market using hierarchical, model-free multi-agent reinforcement learning (MARL). Government agents allocate carbon credits, while enterprises engage in economic activities and carbon trading. This framework illustrates agents' behavior comprehensively. Numerical results show MARL enables government agents to balance productivity, equality, and carbon emissions. Our project is available at https://github.com/xwanghan/Carbon-Simulator.


title: Small Scale Data-Free Knowledge Distillation

abstract: Data-free knowledge distillation is able to utilize the knowledge learned by a large teacher network to augment the training of a smaller student network without accessing the original training data, avoiding privacy, security, and proprietary risks in real applications. In this line of research, existing methods typically follow an inversion-and-distillation paradigm in which a generative adversarial network on-the-fly trained with the guidance of the pre-trained teacher network is used to synthesize a large-scale sample set for knowledge distillation. In this paper, we reexamine this common data-free knowledge distillation paradigm, showing that there is considerable room to improve the overall training efficiency through a lens of ``small-scale inverted data for knowledge distillation". In light of three empirical observations indicating the importance of how to balance class distributions in terms of synthetic sample diversity and difficulty during both data inversion and distillation processes, we propose Small Scale Data-free Knowledge Distillation SSD-KD. In formulation, SSD-KD introduces a modulating function to balance synthetic samples and a priority sampling function to select proper samples, facilitated by a dynamic replay buffer and a reinforcement learning strategy. As a result, SSD-KD can perform distillation training conditioned on an extremely small scale of synthetic samples (e.g., 10X less than the original training data scale), making the overall training efficiency one or two orders of magnitude faster than many mainstream methods while retaining superior or competitive model performance, as demonstrated on popular image classification and semantic segmentation benchmarks. The code is available at https://github.com/OSVAI/SSD-KD.


title: Hierarchical Reinforcement Learning for Swarm Confrontation with High Uncertainty

abstract: In swarm robotics, confrontation including the pursuit-evasion game is a key scenario. High uncertainty caused by unknown opponents' strategies and dynamic obstacles complicates the action space into a hybrid decision process. Although the deep reinforcement learning method is significant for swarm confrontation since it can handle various sizes, as an end-to-end implementation, it cannot deal with the hybrid process. Here, we propose a novel hierarchical reinforcement learning approach consisting of a target allocation layer, a path planning layer, and the underlying dynamic interaction mechanism between the two layers, which indicates the quantified uncertainty. It decouples the hybrid process into discrete allocation and continuous planning layers, with a probabilistic ensemble model to quantify the uncertainty and regulate the interaction frequency adaptively. Furthermore, to overcome the unstable training process introduced by the two layers, we design an integration training method including pre-training and cross-training, which enhances the training efficiency and stability. Experiment results in both comparison and ablation studies validate the effectiveness and generalization performance of our proposed approach.


title: A Game Theoretic Analysis of the Three-Gambler Ruin Game

abstract: We study the following game. Three players start with initial capitals of $s_{1},s_{2},s_{3}$ dollars; in each round player $P_{m}$ is selected with probability $\frac{1}{3}$; then \emph{he} selects player $P_{n}$ and they play a game in which $P_{m}$ wins from (resp. loses to) $P_{n}$ one dollar with probability $p_{mn}$ (resp. $p_{nm}=1-p_{mn}$). When a player loses all his capital he drops out; the game continues until a single player wins by collecting everybody's money. This is a "strategic" version of the classical Gambler's Ruin game. It seems reasonable that a player may improve his winning probability by judicious selection of which opponent to engage in each round. We formulate the situation as a \emph{stochastic game} and prove that it has at least one Nash equilibrium in deterministic stationary strategies.


title: KernelWarehouse: Rethinking the Design of Dynamic Convolution

abstract: Dynamic convolution learns a linear mixture of n static kernels weighted with their input-dependent attentions, demonstrating superior performance than normal convolution. However, it increases the number of convolutional parameters by n times, and thus is not parameter efficient. This leads to no research progress that can allow researchers to explore the setting n>100 (an order of magnitude larger than the typical setting n<10) for pushing forward the performance boundary of dynamic convolution while enjoying parameter efficiency. To fill this gap, in this paper, we propose KernelWarehouse, a more general form of dynamic convolution, which redefines the basic concepts of ``kernels", ``assembling kernels" and ``attention function" through the lens of exploiting convolutional parameter dependencies within the same layer and across neighboring layers of a ConvNet. We testify the effectiveness of KernelWarehouse on ImageNet and MS-COCO datasets using various ConvNet architectures. Intriguingly, KernelWarehouse is also applicable to Vision Transformers, and it can even reduce the model size of a backbone while improving the model accuracy. For instance, KernelWarehouse (n=4) achieves 5.61%|3.90%|4.38% absolute top-1 accuracy gain on the ResNet18|MobileNetV2|DeiT-Tiny backbone, and KernelWarehouse (n=1/4) with 65.10% model size reduction still achieves 2.29% gain on the ResNet18 backbone. The code and models are available at https://github.com/OSVAI/KernelWarehouse.


title: A Comprehensive Survey on Machine Learning Driven Material Defect Detection: Challenges, Solutions, and Future Prospects

abstract: Material defects (MD) represent a primary challenge affecting product performance and giving rise to safety issues in related products. The rapid and accurate identification and localization of MD constitute crucial research endeavours in addressing contemporary challenges associated with MD. Although conventional non-destructive testing methods such as ultrasonic and X-ray approaches have mitigated issues related to low efficiency in manual inspections, they struggle to meet the diverse requirements of high precision, real-time speed, automation, and intelligence. In recent years, propelled by the swift advancement of machine learning (ML) technologies, particularly exemplified by deep learning, ML has swiftly emerged as the core technology and a prominent research direction for material defect detection (MDD). Through a comprehensive review of the latest literature, we systematically survey the ML techniques applied in MDD into five categories: unsupervised learning, supervised learning, semi-supervised learning, reinforcement learning, and generative learning. We provide a detailed analysis of the main principles and techniques used, together with the advantages and potential challenges associated with these techniques. Furthermore, the survey focuses on the techniques for defect detection in composite materials, which are important types of materials enjoying increasingly wide application in various industries such as aerospace, automotive, construction, and renewable energy. Finally, the survey explores potential future directions in MDD utilizing ML technologies. This comprehensive survey not only consolidates existing literature on ML-based MDD technologies but also serves as a foundational reference for future researchers and industrial practitioners, providing valuable insights and guidance in developing advanced and efficient MDD systems.


title: McKean-Vlasov Forward-backward doubly stochastic differential equations and applications to stochastic control

abstract: This paper investigates first the existence and uniqueness of solutions for McKean-Vlasov forward-backward doubly stochastic differential equations (MV-FBDSDEs) in infinite-dimensional real separable Hilbert spaces. These equations combine the features of forward-backward doubly stochastic differential equations with the mean-field approach, allowing the coefficients to depend on the solution distribution. We establish the existence and uniqueness of solutions for MV-FBDSDEs using the method of continuation and provide an example and a counterexample to illustrate our findings. Moreover, we extend the practical applicability of our results by employing them within the context of the stochastic maximum principle for a control problem governed by MV-FBDSDEs. This study contributes to the field of stochastic control problems and presents the first analysis of MV-FBDSDEs in infinite-dimensional spaces.


title: Designing a Dashboard for Transparency and Control of Conversational AI

abstract: Conversational LLMs function as black box systems, leaving users guessing about why they see the output they do. This lack of transparency is potentially problematic, especially given concerns around bias and truthfulness. To address this issue, we present an end-to-end prototype-connecting interpretability techniques with user experience design-that seeks to make chatbots more transparent. We begin by showing evidence that a prominent open-source LLM has a "user model": examining the internal state of the system, we can extract data related to a user's age, gender, educational level, and socioeconomic status. Next, we describe the design of a dashboard that accompanies the chatbot interface, displaying this user model in real time. The dashboard can also be used to control the user model and the system's behavior. Finally, we discuss a study in which users conversed with the instrumented system. Our results suggest that users appreciate seeing internal states, which helped them expose biased behavior and increased their sense of control. Participants also made valuable suggestions that point to future directions for both design and machine learning research. The project page and video demo of our TalkTuner system are available at https://bit.ly/talktuner-project-page


title: Continuity estimates for the gradient of solutions to the Monge-Amp\`ere equation with nature boundary conditions

abstract: We study the first derivative estimates for solutions to Monge-Amp\`ere equations in terms of modulus of continuity. As a result, we establish the optimal global log-Lipschitz continuity for the gradient of solutions to the Monge-Amp\`ere equation with natural boundary conditions.


title: Reinforcement Learning to Disentangle Multiqubit Quantum States from Partial Observations

abstract: Using partial knowledge of a quantum state to control multiqubit entanglement is a largely unexplored paradigm in the emerging field of quantum interactive dynamics with the potential to address outstanding challenges in quantum state preparation and compression, quantum control, and quantum complexity. We present a deep reinforcement learning (RL) approach to constructing short disentangling circuits for arbitrary 4-, 5-, and 6-qubit states using an actor-critic algorithm. With access to only two-qubit reduced density matrices, our agent decides which pairs of qubits to apply two-qubit gates on; requiring only local information makes it directly applicable on modern NISQ devices. Utilizing a permutation-equivariant transformer architecture, the agent can autonomously identify qubit permutations within the state, and adjusts the disentangling protocol accordingly. Once trained, it provides circuits from different initial states without further optimization. We demonstrate the agent's ability to identify and exploit the entanglement structure of multiqubit states. For 4-, 5-, and 6-qubit Haar-random states, the agent learns to construct disentangling circuits that exhibit strong correlations both between consecutive gates and among the qubits involved. Through extensive benchmarking, we show the efficacy of the RL approach to find disentangling protocols with minimal gate resources. We explore the resilience of our trained agents to noise, highlighting their potential for real-world quantum computing applications. Analyzing optimal disentangling protocols, we report a general circuit to prepare an arbitrary 4-qubit state using at most 5 two-qubit (10 CNOT) gates.


title: GENIU: A Restricted Data Access Unlearning for Imbalanced Data

abstract: With the increasing emphasis on data privacy, the significance of machine unlearning has grown substantially. Class unlearning, which involves enabling a trained model to forget data belonging to a specific class learned before, is important as classification tasks account for the majority of today's machine learning as a service (MLaaS). Retraining the model on the original data, excluding the data to be forgotten (a.k.a forgetting data), is a common approach to class unlearning. However, the availability of original data during the unlearning phase is not always guaranteed, leading to the exploration of class unlearning with restricted data access. While current unlearning methods with restricted data access usually generate proxy sample via the trained neural network classifier, they typically focus on training and forgetting balanced data. However, the imbalanced original data can cause trouble for these proxies and unlearning, particularly when the forgetting data consists predominantly of the majority class. To address this issue, we propose the GENerative Imbalanced Unlearning (GENIU) framework. GENIU utilizes a Variational Autoencoder (VAE) to concurrently train a proxy generator alongside the original model. These generated proxies accurately represent each class and are leveraged in the unlearning phase, eliminating the reliance on the original training data. To further mitigate the performance degradation resulting from forgetting the majority class, we introduce an in-batch tuning strategy that works with the generated proxies. GENIU is the first practical framework for class unlearning in imbalanced data settings and restricted data access, ensuring the preservation of essential information for future unlearning. Experimental results confirm the superiority of GENIU over existing methods, establishing its effectiveness in empirical scenarios.


title: Label-aware Hard Negative Sampling Strategies with Momentum Contrastive Learning for Implicit Hate Speech Detection

abstract: Detecting implicit hate speech that is not directly hateful remains a challenge. Recent research has attempted to detect implicit hate speech by applying contrastive learning to pre-trained language models such as BERT and RoBERTa, but the proposed models still do not have a significant advantage over cross-entropy loss-based learning. We found that contrastive learning based on randomly sampled batch data does not encourage the model to learn hard negative samples. In this work, we propose Label-aware Hard Negative sampling strategies (LAHN) that encourage the model to learn detailed features from hard negative samples, instead of naive negative samples in random batch, using momentum-integrated contrastive learning. LAHN outperforms the existing models for implicit hate speech detection both in- and cross-datasets. The code is available at https://github.com/Hanyang-HCC-Lab/LAHN


title: An Empirical Study of Mamba-based Language Models

abstract: Selective state-space models (SSMs) like Mamba overcome some of the shortcomings of Transformers, such as quadratic computational complexity with sequence length and large inference-time memory requirements from the key-value cache. Moreover, recent studies have shown that SSMs can match or exceed the language modeling capabilities of Transformers, making them an attractive alternative. In a controlled setting (e.g., same data), however, studies so far have only presented small scale experiments comparing SSMs to Transformers. To understand the strengths and weaknesses of these architectures at larger scales, we present a direct comparison between 8B-parameter Mamba, Mamba-2, and Transformer models trained on the same datasets of up to 3.5T tokens. We also compare these models to a hybrid architecture consisting of 43% Mamba-2, 7% attention, and 50% MLP layers (Mamba-2-Hybrid). Using a diverse set of tasks, we answer the question of whether Mamba models can match Transformers at larger training budgets. Our results show that while pure SSMs match or exceed Transformers on many tasks, they lag behind Transformers on tasks which require strong copying or in-context learning abilities (e.g., 5-shot MMLU, Phonebook) or long-context reasoning. In contrast, we find that the 8B Mamba-2-Hybrid exceeds the 8B Transformer on all 12 standard tasks we evaluated (+2.65 points on average) and is predicted to be up to 8x faster when generating tokens at inference time. To validate long-context capabilities, we provide additional experiments evaluating variants of the Mamba-2-Hybrid and Transformer extended to support 16K, 32K, and 128K sequences. On an additional 23 long-context tasks, the hybrid model continues to closely match or exceed the Transformer on average. To enable further study, we release the checkpoints as well as the code used to train our models as part of NVIDIA's Megatron-LM project.


title: Classification Modeling with RNN-Based, Random Forest, and XGBoost for Imbalanced Data: A Case of Early Crash Detection in ASEAN-5 Stock Markets

abstract: This research aims to evaluate the performance of several Recurrent Neural Network (RNN) architectures including Simple RNN, Gated Recurrent Units (GRU), and Long Short-Term Memory (LSTM), compared to classic algorithms such as Random Forest and XGBoost in building classification models for early crash detection in ASEAN-5 stock markets. The study is examined using imbalanced data, which is common due to the rarity of market crashes. The study analyzes daily data from 2010 to 2023 across the major stock markets of the ASEAN-5 countries, including Indonesia, Malaysia, Singapore, Thailand, and Philippines. Market crash is identified as the target variable when the major stock price indices fall below the Value at Risk (VaR) thresholds of 5%, 2.5% and 1%. predictors involving technical indicators of major local and global markets as well as commodity markets. This study includes 213 predictors with their respective lags (5, 10, 15, 22, 50, 200) and uses a time step of 7, expanding the total number of predictors to 1491. The challenge of data imbalance is addressed with SMOTE-ENN. The results show that all RNN-Based architectures outperform Random Forest and XGBoost. Among the various RNN architectures, Simple RNN stands out as the most superior, mainly due to the data characteristics that are not overly complex and focus more on short-term information. This study enhances and extends the range of phenomena observed in previous studies by incorporating variables like different geographical zones and time periods, as well as methodological adjustments.


title: Nonparametric estimation of linear multiplier for processes driven by a bifractional Brownian motion

abstract: We study the problem of nonparametric estimation of the linear multiplier function $\theta(t)$ for processes satisfying stochastic differential equations of the type $$dX_t=\theta(t)X_tdt+\epsilon dW_t^{H,K}, X_0=x_0,0\leq t \leq T$$ where $\{W_t^{H,K}, t \geq 0\}$ is a bifractional Brownian motion with known parameters $H\in (0,1), K\in (0,1]$ and $HK\in (\frac{1}{2},1).$ We investigate the asymptotic behaviour of the estimator of the unknown function $\theta(t)$ as $\epsilon \rightarrow 0.$


title: Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions

abstract: Speech foundation models, trained on vast datasets, have opened unique opportunities in addressing challenging low-resource speech understanding, such as child speech. In this work, we explore the capabilities of speech foundation models on child-adult speaker diarization. We show that exemplary foundation models can achieve 39.5% and 62.3% relative reductions in Diarization Error Rate and Speaker Confusion Rate, respectively, compared to previous speaker diarization methods. In addition, we benchmark and evaluate the speaker diarization results of the speech foundation models with varying the input audio window size, speaker demographics, and training data ratio. Our results highlight promising pathways for understanding and adopting speech foundation models to facilitate child speech understanding.


title: McCormick envelopes in mixed-integer PDE-constrained optimization

abstract: McCormick envelopes are a standard tool for deriving convex relaxations of optimization problems that involve polynomial terms. McCormick envelopes provide lower bounds, for example, in branch-and-bound procedures for mixed-integer nonlinear programs but have not gained much attention in infinite-dimensional optimization so far. This lack of attention may be due to the distributed nature of infinite-dimensional problems, which on the one hand leads to infinitely many constraints (generally state constraints that may be difficult to handle) that are added to the problem under consideration and on the other hand renders bound-tightening procedures that successively improve these convex envelopes computationally intractable. We take on the challenge and analyze McCormick envelopes for a model problem class that is governed by a semilinear PDE involving a bilinearity. We approximate the McCormick envelopes by averaging them over the cells of a discretization of the computational domain. This process yields convex problems that underestimate the original problem up to an a priori error estimate that depends on the mesh size of the discretization. These approximate McCormick envelopes can be improved by means of an optimization-based bound-tightening procedure and can be shown to $\Gamma$-converge to a limit problem with a pointwise formulation of the McCormick envelopes. We provide a computational example, for which we certify all of our imposed assumptions. The results point to both the potential of the methodology and the gaps in the research that need to be closed. Our methodology provides a framework first for obtaining pointwise underestimators for nonconvexities and second for approximating them with finitely many linear inequalities in a superordinate infinite-dimensional setting.


title: Finite Time Analysis of Temporal Difference Learning for Mean-Variance in a Discounted MDP

abstract: Motivated by risk-sensitive reinforcement learning scenarios, we consider the problem of policy evaluation for variance in a discounted reward Markov decision process (MDP). For this problem, a temporal difference (TD) type learning algorithm with linear function approximation (LFA) exists in the literature, though only asymptotic guarantees are available for this algorithm. We derive finite sample bounds that hold (i) in the mean-squared sense; and (ii) with high probability, when tail iterate averaging is employed with/without regularization. Our bounds exhibit exponential decay for the initial error, while the overall bound is $O(1/t)$, where $t$ is the number of update iterations of the TD algorithm. Further, the bound for the regularized TD variant is for a universal step size. Our bounds open avenues for analysis of actor-critic algorithms for mean-variance optimization in a discounted MDP.


title: Parameter Estimation in Quantum Metrology Technique for Time Series Prediction

abstract: The paper investigates the techniques of quantum computation in metrological predictions, with a particular emphasis on enhancing prediction potential through variational parameter estimation. The applicability of quantum simulations and quantum metrology techniques for modelling complex physical systems and achieving high-resolution measurements are proposed. The impacts of various parameter distributions and learning rates on predictive accuracy are investigated. Modelling the time evolution of physical systems Hamiltonian simulation and the product formula procedure are adopted. The time block method is analyzed in order to reduce simulation errors, while the Schatten-infinite norm is used to evaluate the simulation precision. Methodology requires estimation of optimized parameters by minimizing loss functions and resource needs. For this purpose, the mathematical formulations of Cramer Rao Bound and Fischer Information are indispensable requirements. The impact of learning rates on regulating the loss function for various parameter values. Using parameterized quantum circuits, the article outlines a four-step procedure for extracting information. This method involves the preparation of input states, the evolution of parameterized quantum states, the measurement of outputs, and the estimation of parameters based on multiple measurements. The study analyses variational unitary circuits with optimized parameter estimation for more precise predictions. The findings shed light on the effects of normal parameter distributions and learning rates on attaining the most optimal state and comparison with classical Long Short Term Memory (LSTM) predictions, providing valuable insights for the development of more appropriate approaches in quantum computing.


title: 100 Drivers, 2200 km: A Natural Dataset of Driving Style toward Human-centered Intelligent Driving Systems

abstract: Effective driving style analysis is critical to developing human-centered intelligent driving systems that consider drivers' preferences. However, the approaches and conclusions of most related studies are diverse and inconsistent because no unified datasets tagged with driving styles exist as a reliable benchmark. The absence of explicit driving style labels makes verifying different approaches and algorithms difficult. This paper provides a new benchmark by constructing a natural dataset of Driving Style (100-DrivingStyle) tagged with the subjective evaluation of 100 drivers' driving styles. In this dataset, the subjective quantification of each driver's driving style is from themselves and an expert according to the Likert-scale questionnaire. The testing routes are selected to cover various driving scenarios, including highways, urban, highway ramps, and signalized traffic. The collected driving data consists of lateral and longitudinal manipulation information, including steering angle, steering speed, lateral acceleration, throttle position, throttle rate, brake pressure, etc. This dataset is the first to provide detailed manipulation data with driving-style tags, and we demonstrate its benchmark function using six classifiers. The 100-DrivingStyle dataset is available via https://github.com/chaopengzhang/100-DrivingStyle-Dataset


title: Emotional Conversation: Empowering Talking Faces with Cohesive Expression, Gaze and Pose Generation

abstract: Vivid talking face generation holds immense potential applications across diverse multimedia domains, such as film and game production. While existing methods accurately synchronize lip movements with input audio, they typically ignore crucial alignments between emotion and facial cues, which include expression, gaze, and head pose. These alignments are indispensable for synthesizing realistic videos. To address these issues, we propose a two-stage audio-driven talking face generation framework that employs 3D facial landmarks as intermediate variables. This framework achieves collaborative alignment of expression, gaze, and pose with emotions through self-supervised learning. Specifically, we decompose this task into two key steps, namely speech-to-landmarks synthesis and landmarks-to-face generation. The first step focuses on simultaneously synthesizing emotionally aligned facial cues, including normalized landmarks that represent expressions, gaze, and head pose. These cues are subsequently reassembled into relocated facial landmarks. In the second step, these relocated landmarks are mapped to latent key points using self-supervised learning and then input into a pretrained model to create high-quality face images. Extensive experiments on the MEAD dataset demonstrate that our model significantly advances the state-of-the-art performance in both visual quality and emotional alignment.


title: Investigating Sulfur Chemistry in the HD 163296 disk

abstract: Sulfur chemistry in the formation process of low-mass stars and planets remains poorly understood. The protoplanetary disks (PPDs) are the birthplace of planets and its distinctive environment provides an intriguing platform for investigating models of sulfur chemistry. We analyzed the ALMA observations of CS 7-6 transitions in the HD 163296 disk and perform astrochemical modeling to explore its sulfur chemistry. We simulated the distribution of sulfur-containing molecules and compared it with observationally deduced fractional column densities. We have found that the simulated column density of CS is consistent with the observationally deduced fractional column densities, while the simulated column density of C$_2$S is lower than the observationally deduced upper limits on column densities. This results indicate that we have a good understanding of the chemical properties of CS and C$_2$S in the disk. We also investigated the influence of the C/O ratio on sulfur-containing molecules and found that the column densities of SO, SO$_2$, and H$_2$S near the central star are dependent on the C/O ratio. Additionally, we found that the $N$[CS]/$N$[SO] ratio can serve as a promising indicator of the disk's C/O ratio in the HD 163296. Overall, the disk of HD 163296 provides a favorable environment for the detection of sulfur-containing molecules.


title: When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions

abstract: Skills are temporal abstractions that are intended to improve reinforcement learning (RL) performance through hierarchical RL. Despite our intuition about the properties of an environment that make skills useful, a precise characterization has been absent. We provide the first such characterization, focusing on the utility of deterministic skills in deterministic sparse-reward environments with finite action spaces. We show theoretically and empirically that RL performance gain from skills is worse in environments where solutions to states are less compressible. Additional theoretical results suggest that skills benefit exploration more than they benefit learning from existing experience, and that using unexpressive skills such as macroactions may worsen RL performance. We hope our findings can guide research on automatic skill discovery and help RL practitioners better decide when and how to use skills.


title: Content Provider Contributions to Capacity Expansion of a Neutral ISP: Effect of Private Option

abstract: Increasing content consumption by users and the expectation of a better Internet experience requires Internet service providers (ISPs) to expand the capacity of the access network continually. The ISPs have been demanding the participation of the content providers (CPs) in sharing the cost of upgrading the infrastructure. From CPs' perspective, investing in the ISP infrastructure, termed as \emph{public investment}, seems rational as it will boost their profit. However, the CPs can alternatively invest in making content delivery more efficient, termed as \emph{private investment}, as it also boosts their profit. Thus, in this work, we investigate this trade-off between public and private investment of the CPs for a net-neutral ISP. Specifically, we consider centralized decision and non-cooperative forms of interaction between CPs and an ISP and determine the optimum public and private investments of the CPs for each model. In the non-cooperative interaction, we find that at most one CP contributes to the public infrastructure, whereas all invest in their private infrastructure.


title: Josephson Parametric Amplifier based Quantum Noise Limited Amplifier Development for Axion Search Experiments in CAPP

abstract: This paper provides a comprehensive overview of the development of flux-driven Josephson Parametric Amplifiers (JPAs) as Quantum Noise Limited Amplifier for axion search experiments conducted at the Center for Axion and Precision Physics Research (CAPP) of the Institute for Basic Science. It focuses on the characterization, and optimization of JPAs, which are crucial for achieving the highest sensitivity in axion particle detection. We discuss various characterization techniques, methods for improving bandwidth, and the attainment of ultra-low noise temperatures. JPAs have emerged as indispensable tools in CAPPs axion search endeavors, playing a significant role in advancing our understanding of fundamental physics and unraveling the mysteries of the universe.


title: Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations

abstract: Recent advancements in Deep and Self-Supervised Learning (SSL) have led to substantial improvements in Speech Emotion Recognition (SER) performance, reaching unprecedented levels. However, obtaining sufficient amounts of accurately labeled data for training or fine-tuning the models remains a costly and challenging task. In this paper, we propose a multi-view SSL pre-training technique that can be applied to various representations of speech, including the ones generated by large speech models, to improve SER performance in scenarios where annotations are limited. Our experiments, based on wav2vec 2.0, spectral and paralinguistic features, demonstrate that the proposed framework boosts the SER performance, by up to 10% in Unweighted Average Recall, in settings with extremely sparse data annotations.


title: The resolved star formation law in NGC 7469 from JWST, ALMA and VLA

abstract: We investigate the star formation process within the central 3.3 kpc region of the nearby luminous infrared Seyfert NGC 7469, probing scales ranging from 88 to 330 pc. We combine JWST/MIRI imaging with the F770W filter, with CO(2-1) and the underlying 1.3 mm dust continuum data from ALMA, along with VLA radio continuum observations at 22 GHz. NGC 7469 hosts a starburst ring which dominates the overall star formation activity. We estimate a global star formation rate SFR $\sim 11.5$ $\rm M_{\odot}~yr^{-1}$ from the radio at 22 GHz, and a cold molecular gas mass M(H2) $\sim$ 6.4 $\times$ $\rm 10^9 M_{\odot}$ from the CO(2-1) emission. We find that the 1.3 mm map shows a morphology remarkably similar to those traced by the 22 GHz and the 7.7 $\rm \mu m$ polycyclic aromatic hydrocarbon (PAH) emission observed with JWST. The three tracers reproduce the morphology of the starburst ring with good agreement. We further investigate the correlations between the PAHs, the star formation rate and the cold molecular gas. We find a stronger correlation of the PAHs with the star formation than with the CO, with steeper correlations within the starburst ring ($n > 2$) than in the outer region ($n < 1$). We derive the correlation between the star formation rate and the cold molecular gas mass surface densities, the Kennicutt-Schmidt star formation law. Comparisons with other galaxy populations, including starburst galaxies and active galactic nuclei, highlighted that NGC 7469 exhibits an intermediate behavior to the Kennicutt-Schmidt relations found for these galaxy populations.


title: Collinear three-photon excitation of a strongly forbidden optical clock transition

abstract: The ${}^1\!S_0-{}^3\!P_0$ clock transition in strontium serves as the foundation for the world's best atomic clocks and for gravitational wave detector concepts in clock atom interferometry. This transition is weakly allowed in the fermionic isotope $^{87}$Sr, but strongly forbidden in bosonic isotopes unless a strong external magnetic field is applied. Here we demonstrate coherent excitation of the clock transition in bosonic ${}^{88}$Sr using a novel collinear three-photon process in a weak magnetic field. We observe Rabi oscillations with frequencies of up to $50~\text{kHz}$ using $\text{W}/\text{cm}^{2}$ laser intensities and Gauss-level magnetic field amplitudes. The absence of nuclear spin in bosonic isotopes offers decreased sensitivity to magnetic fields and optical lattice light shifts, enabling clocks with reduced systematic errors. The collinear propagation of the laser fields permits the interrogation of spatially separated atomic ensembles with common laser pulses, a key requirement for dark matter searches and gravitational wave detection with clock atom interferometers.


title: GAPses: Versatile smart glasses for comfortable and fully-dry acquisition and parallel ultra-low-power processing of EEG and EOG

abstract: Recent advancements in head-mounted wearable technology are revolutionizing the field of biopotential measurement, but the integration of these technologies into practical, user-friendly devices remains challenging due to issues with design intrusiveness, comfort, and data privacy. To address these challenges, this paper presents GAPSES, a novel smart glasses platform designed for unobtrusive, comfortable, and secure acquisition and processing of electroencephalography (EEG) and electrooculography (EOG) signals. We introduce a direct electrode-electronics interface with custom fully dry soft electrodes to enhance comfort for long wear. An integrated parallel ultra-low-power RISC-V processor (GAP9, Greenwaves Technologies) processes data at the edge, thereby eliminating the need for continuous data streaming through a wireless link, enhancing privacy, and increasing system reliability in adverse channel conditions. We demonstrate the broad applicability of the designed prototype through validation in a number of EEG-based interaction tasks, including alpha waves, steady-state visual evoked potential analysis, and motor movement classification. Furthermore, we demonstrate an EEG-based biometric subject recognition task, where we reach a sensitivity and specificity of 98.87% and 99.86% respectively, with only 8 EEG channels and an energy consumption per inference on the edge as low as 121 uJ. Moreover, in an EOG-based eye movement classification task, we reach an accuracy of 96.68% on 11 classes, resulting in an information transfer rate of 94.78 bit/min, which can be further increased to 161.43 bit/min by reducing the accuracy to 81.43%. The deployed implementation has an energy consumption of 24 uJ per inference and a total system power of only 16.28 mW, allowing for continuous operation of more than 12 h with a small 75 mAh battery.


title: Grounding Multimodal Large Language Models in Actions

abstract: Multimodal Large Language Models (MLLMs) have demonstrated a wide range of capabilities across many domains, including Embodied AI. In this work, we study how to best ground a MLLM into different embodiments and their associated action spaces, with the goal of leveraging the multimodal world knowledge of the MLLM. We first generalize a number of methods through a unified architecture and the lens of action space adaptors. For continuous actions, we show that a learned tokenization allows for sufficient modeling precision, yielding the best performance on downstream tasks. For discrete actions, we demonstrate that semantically aligning these actions with the native output token space of the MLLM leads to the strongest performance. We arrive at these lessons via a thorough study of seven action space adapters on five different environments, encompassing over 114 embodied tasks.


title: Arithmetic density and congruences of $\ell$-regular bipartitions $II$

abstract: Let $ B_{\ell}(n)$ denote the number of $\ell-$regular bipartitions of $n.$ In 2013, Lin \cite{Lin2013} proved a density result for $B_4(n).$ He showed that for any positive integer $k,$ $B_4(n)$ is almost always divisible by $2^k.$ In this article, we improved his result. We prove that $B_{2^{\alpha}m}(n)$ and $B_{3^{\alpha}m}(n)$ are almost always divisible by arbitrary power of $2$ and $3$ respectively. Further, we obtain an infinities families of congruences and multiplicative formulae for $B_2(n)$ and $B_4(n)$ by using Hecke eigenform theory. Next, by using a result of Ono and Taguchi on nilpotency of Hecke operator, we also find an infinite families of congruences modulo arbitrary power of $2$ satisfied by $B_{2^{\alpha}}(n).$


title: Hybrid Rendering for Dynamic Scenes

abstract: Despite significant advances in algorithms and hardware, global illumination continues to be a challenge in the real-time domain. Time constraints often force developers to either compromise on the quality of global illumination or disregard it altogether. We take advantage of a common setup in modern games: having a set of a level, which is a static scene with dynamic characters and lighting. We introduce a novel method for efficiently and accurately rendering global illumination in dynamic scenes. Our hybrid technique leverages precomputation and neural networks to capture the light transport of a static scene. Then, we introduce a method to compute the difference between the current scene and the static scene, which we already precomputed. By handling the bulk of the light transport through precomputation, our method only requires the rendering of a minimal difference, reducing the noise and increasing the quality.


title: Wall-crossing for K-moduli spaces of certain families of weighted projective hypersurfaces

abstract: We describe the K-moduli spaces of weighted hypersurfaces of degree $2(n+3)$ in $\mathbb{P}(1,2,n+2,n+3)$. We show that the K-polystable limits of these weighted hypersurfaces are also weighted hypersurfaces of the same degree in the same weighted projective space. This is achieved by an explicit study of the wall crossing for K-moduli spaces $M_w$ of certain log Fano pairs with coefficient $w$ whose double cover gives the weighted hypersurface. Moreover, we show that the wall crossing of $M_w$ coincides with variation of GIT except at the last K-moduli wall which gives a divisorial contraction. Our K-moduli spaces provide new birational models for some natural loci in the moduli space of marked hyperelliptic curves.


title: Ablation Based Counterfactuals

abstract: Diffusion models are a class of generative models that generate high-quality samples, but at present it is difficult to characterize how they depend upon their training data. This difficulty raises scientific and regulatory questions, and is a consequence of the complexity of diffusion models and their sampling process. To analyze this dependence, we introduce Ablation Based Counterfactuals (ABC), a method of performing counterfactual analysis that relies on model ablation rather than model retraining. In our approach, we train independent components of a model on different but overlapping splits of a training set. These components are then combined into a single model, from which the causal influence of any training sample can be removed by ablating a combination of model components. We demonstrate how we can construct a model like this using an ensemble of diffusion models. We then use this model to study the limits of training data attribution by enumerating full counterfactual landscapes, and show that single source attributability diminishes with increasing training data size. Finally, we demonstrate the existence of unattributable samples.


title: Guiding Frame-Level CTC Alignments Using Self-knowledge Distillation

abstract: Transformer encoder with connectionist temporal classification (CTC) framework is widely used for automatic speech recognition (ASR). However, knowledge distillation (KD) for ASR displays a problem of disagreement between teacher-student models in frame-level alignment which ultimately hinders it from improving the student model's performance. In order to resolve this problem, this paper introduces a self-knowledge distillation (SKD) method that guides the frame-level alignment during the training time. In contrast to the conventional method using separate teacher and student models, this study introduces a simple and effective method sharing encoder layers and applying the sub-model as the student model. Overall, our approach is effective in improving both the resource efficiency as well as performance. We also conducted an experimental analysis of the spike timings to illustrate that the proposed method improves performance by reducing the alignment disagreement.


title: Demonstration of Safe Electromagnetic Radiation Emitted by 5G Active Antenna Systems

abstract: The careful planning and safe deployment of 5G technologies will bring enormous benefits to society and the economy. Higher frequency, beamforming, and small-cells are key technologies that will provide unmatched throughput and seamless connectivity to 5G users. Superficial knowledge of these technologies has raised concerns among the general public about the harmful effects of radiation. Several standardization bodies are active to put limits on the emissions which are based on a defined set of radiation measurement methodologies. However, due to the peculiarity of 5G such as dynamicity of the beams, network densification, Time Division Duplexing mode of operation, etc, using existing EMF measurement methods may provide inaccurate results. In this context, we discuss our experimental studies aimed towards the measurement of radiation caused by beam-based transmissions from a 5G base station equipped with an Active Antenna System(AAS). We elaborate on the shortcomings of current measurement methodologies and address several open questions. Next, we demonstrate that using user-specific downlink beamforming, not only better performance is achieved compared to non-beamformed downlink, but also the radiation in the vicinity of the intended user is significantly decreased. Further, we show that under weak reception conditions, an uplink transmission can cause significantly high radiation in the vicinity of the user equipment. We believe that our work will help in clearing several misleading concepts about the 5G EMF radiation effects. We conclude the work by providing guidelines to improve the methodology of EMF measurement by considering the spatiotemporal dynamicity of the 5G transmission.


title: Proposal for realizing and probing topological crystalline insulators in optical lattices

abstract: We develop a lattice model which exhibits topological transitions from $Z_2$ topological insulators to mirror symmetry-protected topological crystalline insulators by introducing additional spin-orbit coupling terms. The topological phase is characterized by the mirror winding number, defined within the mirror symmetry invariant subspace, which ensures the protection of gapless edge states and zero-energy corner states under specific boundary conditions. Additionally, we propose a feasible scheme using ultracold atoms confined in a stacked hexagonal optical lattice with Raman fields to realize the two-dimensional topological crystalline insulators. Detection of the mirror winding number in these systems can be achieved by implementing a simple quench sequence and observing the evolution of the time-of-flight patterns.


title: A quark core-gluon model for heavy hybrid baryons

abstract: Besides the ordinary hadrons, QCD allows the existence of states in which excitations of the gluonic field can play the role of valence particles, either alone in a glueball, or coupled to quarks in a hybrid. So, hybrid baryons, made of three quarks and a gluon, can a priori exist. Till now, there is no experimental evidence for such exotic hadrons but experimental efforts are being made to search for them at CEBAF Large Acceptance Spectrometer. In this work, a hybrid baryon is considered as a two-body system composed of a color octet three-quark core and a gluon, interacting via a QCD-inspired interaction. A semirelativistic potential model is built in which the dominant interaction is a potential simulating the flux tube confinement, and the Casimir scaling is assumed to link interactions between triplet and octet color sources. This picture is similar to the quark-diquark description for baryons. It is chosen in order to take properly into account the helicity of the gluon. Only $cccg$ and $bbbg$ states are considered because the strong mass asymmetry between the quark core and the gluon is expected to favor the formation of the core. As the results for heavy hybrid baryons seem relevant, we consider this paper as a proof of concept which can be extended for the study of light hybrid baryons.


title: DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning

abstract: While in-context Learning (ICL) has proven to be an effective technique to improve the performance of Large Language Models (LLMs) in a variety of complex tasks, notably in translating natural language questions into Structured Query Language (NL2SQL), the question of how to select the most beneficial demonstration examples remains an open research problem. While prior works often adapted off-the-shelf encoders to retrieve examples dynamically, an inherent discrepancy exists in the representational capacities between the external retrievers and the LLMs. Further, optimizing the selection of examples is a non-trivial task, since there are no straightforward methods to assess the relative benefits of examples without performing pairwise inference. To address these shortcomings, we propose DeTriever, a novel demonstration retrieval framework that learns a weighted combination of LLM hidden states, where rich semantic information is encoded. To train the model, we propose a proxy score that estimates the relative benefits of examples based on the similarities between output queries. Experiments on two popular NL2SQL benchmarks demonstrate that our method significantly outperforms the state-of-the-art baselines on one-shot NL2SQL tasks.


title: Can Large Language Models Understand Spatial Audio?

abstract: This paper explores enabling large language models (LLMs) to understand spatial information from multichannel audio, a skill currently lacking in auditory LLMs. By leveraging LLMs' advanced cognitive and inferential abilities, the aim is to enhance understanding of 3D environments via audio. We study 3 spatial audio tasks: sound source localization (SSL), far-field speech recognition (FSR), and localisation-informed speech extraction (LSE), achieving notable progress in each task. For SSL, our approach achieves an MAE of $2.70^{\circ}$ on the Spatial LibriSpeech dataset, substantially surpassing the prior benchmark of about $6.60^{\circ}$. Moreover, our model can employ spatial cues to improve FSR accuracy and execute LSE by selectively attending to sounds originating from a specified direction via text prompts, even amidst overlapping speech. These findings highlight the potential of adapting LLMs to grasp physical audio concepts, paving the way for LLM-based agents in 3D environments.


title: Aggregation Design for Personalized Federated Multi-Modal Learning over Wireless Networks

abstract: Federated Multi-Modal Learning (FMML) is an emerging field that integrates information from different modalities in federated learning to improve the learning performance. In this letter, we develop a parameter scheduling scheme to improve personalized performance and communication efficiency in personalized FMML, considering the non-independent and nonidentically distributed (non-IID) data along with the modality heterogeneity. Specifically, a learning-based approach is utilized to obtain the aggregation coefficients for parameters of different modalities on distinct devices. Based on the aggregation coefficients and channel state, a subset of parameters is scheduled to be uploaded to a server for each modality. Experimental results show that the proposed algorithm can effectively improve the personalized performance of FMML.


title: Revisiting strong-coupling determinations from $e^+ e^-$ event shapes

abstract: We reassess the theoretical uncertainties of strong-coupling determinations from a global fit to the $e^+ e^-$ thrust data, focusing in particular on the scheme dependence associated with the applied renormalon-cancellation prescription and on the choice of scale parameters that are used to estimate higher-order perturbative corrections in all sectors of the calculation.


title: Graph Transductive Defense: a Two-Stage Defense for Graph Membership Inference Attacks

abstract: Graph neural networks (GNNs) have become instrumental in diverse real-world applications, offering powerful graph learning capabilities for tasks such as social networks and medical data analysis. Despite their successes, GNNs are vulnerable to adversarial attacks, including membership inference attacks (MIA), which threaten privacy by identifying whether a record was part of the model's training data. While existing research has explored MIA in GNNs under graph inductive learning settings, the more common and challenging graph transductive learning setting remains understudied in this context. This paper addresses this gap and proposes an effective two-stage defense, Graph Transductive Defense (GTD), tailored to graph transductive learning characteristics. The gist of our approach is a combination of a train-test alternate training schedule and flattening strategy, which successfully reduces the difference between the training and testing loss distributions. Extensive empirical results demonstrate the superior performance of our method (a decrease in attack AUROC by $9.42\%$ and an increase in utility performance by $18.08\%$ on average compared to LBP), highlighting its potential for seamless integration into various classification models with minimal overhead.


title: Micro-expression recognition based on depth map to point cloud

abstract: Micro-expressions are nonverbal facial expressions that reveal the covert emotions of individuals, making the micro-expression recognition task receive widespread attention. However, the micro-expression recognition task is challenging due to the subtle facial motion and brevity in duration. Many 2D image-based methods have been developed in recent years to recognize MEs effectively, but, these approaches are restricted by facial texture information and are susceptible to environmental factors, such as lighting. Conversely, depth information can effectively represent motion information related to facial structure changes and is not affected by lighting. Motion information derived from facial structures can describe motion features that pixel textures cannot delineate. We proposed a network for micro-expression recognition based on facial depth information, and our experiments have demonstrated the crucial role of depth maps in the micro-expression recognition task. Initially, we transform the depth map into a point cloud and obtain the motion information for each point by aligning the initiating frame with the apex frame and performing a differential operation. Subsequently, we adjusted all point cloud motion feature input dimensions and used them as inputs for multiple point cloud networks to assess the efficacy of this representation. PointNet++ was chosen as the ultimate outcome for micro-expression recognition due to its superior performance. Our experiments show that our proposed method significantly outperforms the existing deep learning methods, including the baseline, on the $CAS(ME)^3$ dataset, which includes depth information.


title: Theory for Tunnel Magnetoresistance Oscillation

abstract: The universal oscillation of the tunnel magnetoresistance (TMR) ratio as a function of the insulating barrier thickness in crystalline magnetic tunnel junctions (MTJs) is a long-standing unsolved problem in condensed matter physics. To explain this, we here introduce a superposition of wave functions with opposite spins and different Fermi momenta, based on the fact that spin-flip scattering near the interface provides a hybridization between majority- and minority-spin states. In a typical Fe/MgO/Fe MTJ, we solve the tunneling problem and show that the TMR ratio oscillates with a period of $\sim3\,$\r{A} by varying the MgO thickness, consistent with previous and present experimental observations.


title: Near-Optimal Learning and Planning in Separated Latent MDPs

abstract: We study computational and statistical aspects of learning Latent Markov Decision Processes (LMDPs). In this model, the learner interacts with an MDP drawn at the beginning of each epoch from an unknown mixture of MDPs. To sidestep known impossibility results, we consider several notions of separation of the constituent MDPs. The main thrust of this paper is in establishing a nearly-sharp *statistical threshold* for the horizon length necessary for efficient learning. On the computational side, we show that under a weaker assumption of separability under the optimal policy, there is a quasi-polynomial algorithm with time complexity scaling in terms of the statistical threshold. We further show a near-matching time complexity lower bound under the exponential time hypothesis.

categories: cs.LG cs.AI cs.CC math.ST stat.ML stat.TH


title: A Two-Stage Online Algorithm for EV Charging Station Energy Management and Carbon Trading

abstract: The increasing electric vehicle (EV) adoption challenges the energy management of charging stations (CSs) due to the large number of EVs and the underlying uncertainties. Moreover, the carbon footprint of CSs is growing significantly due to the rising charging power demand. This makes it important for CSs to properly manage their energy usage and ensure their carbon footprint stay within their carbon emission quotas. This paper proposes a two-stage online algorithm for this purpose, considering the different time scales of energy management and carbon trading. In the first stage, the CS characterizes the real-time aggregate EV power flexibility, in terms of upper and lower bounds on the total charging power, by a Lyapunov optimization-based online algorithm. In the second stage, the CS co-optimizes energy management and carbon trading, with EV charging power chosen within the aggregate flexibility region provided by the first stage. A generalized battery model is proposed to capture the dynamic carbon footprint changes and carbon trading. A virtual carbon queue is designed to develop an online algorithm for the second stage, which can ensure the carbon footprint of CS be within its carbon emission quota and its total operation cost is nearly offline optimal. Case studies validate the effectiveness and advantages of the proposed algorithm.


title: Automated Information Extraction from Thyroid Operation Narrative: A Comparative Study of GPT-4 and Fine-tuned KoELECTRA

abstract: In the rapidly evolving field of healthcare, the integration of artificial intelligence (AI) has become a pivotal component in the automation of clinical workflows, ushering in a new era of efficiency and accuracy. This study focuses on the transformative capabilities of the fine-tuned KoELECTRA model in comparison to the GPT-4 model, aiming to facilitate automated information extraction from thyroid operation narratives. The current research landscape is dominated by traditional methods heavily reliant on regular expressions, which often face challenges in processing free-style text formats containing critical details of operation records, including frozen biopsy reports. Addressing this, the study leverages advanced natural language processing (NLP) techniques to foster a paradigm shift towards more sophisticated data processing systems. Through this comparative study, we aspire to unveil a more streamlined, precise, and efficient approach to document processing in the healthcare domain, potentially revolutionizing the way medical data is handled and analyzed.


title: CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting

abstract: This paper introduces a novel approach for streaming openvocabulary keyword spotting (KWS) with text-based keyword enrollment. For every input frame, the proposed method finds the optimal alignment ending at the frame using connectionist temporal classification (CTC) and aggregates the frame-level acoustic embedding (AE) to obtain higher-level (i.e., character, word, or phrase) AE that aligns with the text embedding (TE) of the target keyword text. After that, we calculate the similarity of the aggregated AE and the TE. To the best of our knowledge, this is the first attempt to dynamically align the audio and the keyword text on-the-fly to attain the joint audio-text embedding for KWS. Despite operating in a streaming fashion, our approach achieves competitive performance on the LibriPhrase dataset compared to the non-streaming methods with a mere 155K model parameters and a decoding algorithm with time complexity O(U), where U is the length of the target keyword at inference time.


title: The Galerkin method for a regularised combined field integral equation without a dual basis function

abstract: We propose discretisation of a regularised combined field integral equation (regularised CFIE) only with the Rao-Wilton-Glisson (RWG) basis function. The CFIE is a formulation of integral equations, which avoids the so-called ficticious frequencies of integral equations. The most typical CFIE, which is a linear combination of the electric field integral equation (EFIE) and magnetic field integral equation (MFIE), is known to be ill-conditioned and requires many iterations when solved with iteration methods such as the generalised minimum residual (GMRES) method. The regularised CFIE is another formulation of the CFIE to solve this problem by applying a regularising operator to the part of the EFIE. In several previous studies the regularising operator is determined based on the Calderon preconditioning. This regularising operator however takes much more computatonal time than the standard CFIE since discretising the EFIE with the Calderon preconditioner requires the dual basis function. In this article we propose a formulation of the regularised CFIE, which can be discretised with the Galerkin method without the dual basis function.


title: FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning

abstract: Large language models (LLMs) have emerged as important components across various fields, yet their training requires substantial computation resources and abundant labeled data. It poses a challenge to robustly training LLMs for individual users (clients). To tackle this challenge, the intuitive idea is to introduce federated learning (FL), which can collaboratively train models on distributed private data. However, existing methods suffer from the challenges of data heterogeneity, system heterogeneity, and model size, resulting in suboptimal performance and high costs. In this work, we proposed a variant of personalized federated learning (PFL) framework, namely FDLoRA, which allows the client to be a single device or a cluster and adopts low-rank adaptation (LoRA) tuning. FDLoRA sets dual LoRA modules on each client to capture personalized and global knowledge, respectively, and only the global LoRA module uploads parameters to the central server to aggregate cross-client knowledge. Finally, an adaptive fusion approach is employed to combine the parameters of the dual LoRAs. This enables FDLoRA to make effective use of private data distributed across different clients, thereby improving performance on the client without incurring high communication and computing costs. We conducted extensive experiments in two practice scenarios. The results demonstrate that FDLoRA outperforms six baselines in terms of performance, stability, robustness, computation cost, and communication cost.


title: Efficient Neural Common Neighbor for Temporal Graph Link Prediction

abstract: Temporal graphs are ubiquitous in real-world scenarios, such as social network, trade and transportation. Predicting dynamic links between nodes in a temporal graph is of vital importance. Traditional methods usually leverage the temporal neighborhood of interaction history to generate node embeddings first and then aggregate the source and target node embeddings to predict the link. However, such methods focus on learning individual node representations, but overlook the pairwise representation learning nature of link prediction and fail to capture the important pairwise features of links such as common neighbors (CN). Motivated by the success of Neural Common Neighbor (NCN) for static graph link prediction, we propose TNCN, a temporal version of NCN for link prediction in temporal graphs. TNCN dynamically updates a temporal neighbor dictionary for each node, and utilizes multi-hop common neighbors between the source and target node to learn a more effective pairwise representation. We validate our model on five large-scale real-world datasets from the Temporal Graph Benchmark (TGB), and find that it achieves new state-of-the-art performance on three of them. Additionally, TNCN demonstrates excellent scalability on large datasets, outperforming popular GNN baselines by up to 6.4 times in speed. Our code is available at https: //github.com/GraphPKU/TNCN.